From bsdhenrymartin at gmail.com  Mon Aug  3 00:33:52 2026
From: bsdhenrymartin at gmail.com (Henry Martin)
Date: Mon,  3 Aug 2026 15:33:52 +0800
Subject: [PATCH 0/3] rxrpc: fix three bugs found by autokbug fuzzing
Message-ID: <20260803073355.3895949-1-bsdhenrymartin@gmail.com>

These three patches fix independent rxrpc bugs found by the autokbug
dynamic kernel fuzzer at Tencent Yunding Lab.

  [1/3] Stack OOB read in TLP soft-ACK processing. A malicious peer
        can deterministically trigger it over an unencrypted
        (security-none) AF_RXRPC association - an immediate KASAN
        stack-out-of-bounds report even at the minimum offset.

  [2/3] kernel BUG in netns teardown: the conn_proc_list assertion
        in rxrpc_destroy_all_connections() races with connection
        destructors deferred to system_wq and fires on a healthy
        state, killing the cleanup_net kworker. Unprivileged
        userns+netns reachable.

  [3/3] WARN_ON in rxrpc_alloc_txqueue(): the TX_NO_MORE entry check
        from commit ae4f89989479 ("rxrpc: Fix ability to add more
        data to a call once MSG_MORE deasserted") is not re-done
        after sleeping in wait_for_space, so a racing sendmsg on the
        same call reaches the WARN on a legal finalized-call state
        (spurious splat and -ENOMEM; panic_on_warn DoS).

Thanks,
Henry

Henry Martin (3):
  rxrpc: fix stack OOB read in TLP soft-ACK processing
  rxrpc: wait for deferred conn destruction before conn_proc_list check
  rxrpc: recheck RXRPC_CALL_TX_NO_MORE after sleeping in rxrpc_send_data()

 net/rxrpc/ar-internal.h | 3 ++-
 net/rxrpc/conn_object.c | 10 ++++++----
 net/rxrpc/sendmsg.c     | 8 ++++++++
 3 files changed, 16 insertions(+), 5 deletions(-)

-- 
2.43.0


From bsdhenrymartin at gmail.com  Mon Aug  3 00:33:53 2026
From: bsdhenrymartin at gmail.com (Henry Martin)
Date: Mon,  3 Aug 2026 15:33:53 +0800
Subject: [PATCH 1/3] rxrpc: fix stack OOB read in TLP soft-ACK processing
In-Reply-To: <20260803073355.3895949-1-bsdhenrymartin@gmail.com>
References: <20260803073355.3895949-1-bsdhenrymartin@gmail.com>
Message-ID: <20260803073355.3895949-2-bsdhenrymartin@gmail.com>

rxrpc_seq_in_txq() is meant to test whether a sequence number belongs
to a given txqueue segment, but it compares the in-segment slot
number (seq & 63, range 0..63) against the segment's absolute base
sequence (tq->qbase, 0/64/128/...). Two consequences:

- For the first segment (qbase == 0), any seq that is a multiple of
  64 is wrongly judged to belong to it;
- For any later segment (qbase >= 64), the test is always false, so
  TLP probe handling is silently skipped for them.

In rxrpc_input_soft_ack_tq() the first case leads to

	test_bit(call->tlp_seq - tq->qbase, &new_acks)

being evaluated with tlp_seq - qbase == 64*N while new_acks is a
single unsigned long on the stack, i.e. a stack out-of-bounds read
8*N bytes above new_acks (KASAN reports stack-out-of-bounds at
offset 40 for tlp_seq == 64). With a large enough tlp_seq the read
walks off the vmalloc'd kthread stack into the guard page and panics.

Turn the broken slot comparison into a real range check. This bounds
tlp_seq - tq->qbase to [0, RXRPC_NR_TXQUEUE), keeping the test_bit()
inside new_acks, and also fixes TLP probe handling for non-first
segments.

Found by the autokbug dynamic kernel fuzzer at Tencent Yunding Lab.

Fixes: 7c482665931b ("rxrpc: Implement RACK/TLP to deal with transmission stalls [RFC8985]")
Signed-off-by: Henry Martin <bsdhenrymartin at gmail.com>
---
 net/rxrpc/ar-internal.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 865f05fe37ab9..27992e10d82ad 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -1580,7 +1580,8 @@ static inline u32 latest(u32 seq1, u32 seq2)
 
 static inline bool rxrpc_seq_in_txq(const struct rxrpc_txqueue *tq, rxrpc_seq_t seq)
 {
-	return (seq & (RXRPC_NR_TXQUEUE - 1)) == tq->qbase;
+	return after_eq(seq, tq->qbase) &&
+	       before(seq, tq->qbase + RXRPC_NR_TXQUEUE);
 }
 
 static inline void rxrpc_queue_rx_call_packet(struct rxrpc_call *call, struct sk_buff *skb)
-- 
2.43.0


From bsdhenrymartin at gmail.com  Mon Aug  3 00:33:54 2026
From: bsdhenrymartin at gmail.com (Henry Martin)
Date: Mon,  3 Aug 2026 15:33:54 +0800
Subject: [PATCH 2/3] rxrpc: wait for deferred conn destruction before conn_proc_list check
In-Reply-To: <20260803073355.3895949-1-bsdhenrymartin@gmail.com>
References: <20260803073355.3895949-1-bsdhenrymartin@gmail.com>
Message-ID: <20260803073355.3895949-3-bsdhenrymartin@gmail.com>

rxrpc_destroy_all_connections() flushes rxrpc_workqueue and then
immediately asserts that rxnet->conn_proc_list is empty. However,
connection destruction is normally deferred to system_wq: the
final-ACK timer is still pending when the last ref is dropped, so
rxrpc_put_connection() schedules conn->destructor instead of running
it inline. conn->proc_link is only removed by the deferred
destructor (rxrpc_clean_up_connection()), which the
flush_workqueue(rxrpc_workqueue) call does not cover - so the
assertion can fire on a perfectly healthy intermediate state:

	rxrpc: Assertion failed
	kernel BUG at net/rxrpc/conn_object.c:488!
	Workqueue: netns cleanup_net

The BUG kills the cleanup_net kworker mid-teardown, leaving the
netns half-destroyed and potentially wedging later netns operations.
This is easily reachable from an unprivileged userns+netns running
loopback AF_RXRPC traffic.

The existing wait_var_event() on nr_conns is exactly the right
synchronization: nr_conns only reaches zero after every destructor
and RCU free has completed, which implies proc_link has been removed
from every connection. It is, however, placed *after* the assertion
it is meant to make reliable. Move it between the service_conns
leak check (whose outcome is already final once the reaper has been
flushed) and the conn_proc_list assertion.

This also silences the spurious "AF_RXRPC: Leaked peer" messages
seen during netns teardown, which share the same root cause: peer
references are dropped by the same deferred destructors.

Found by the autokbug dynamic kernel fuzzer at Tencent Yunding Lab.

Signed-off-by: Henry Martin <bsdhenrymartin at gmail.com>
---
 net/rxrpc/conn_object.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/net/rxrpc/conn_object.c b/net/rxrpc/conn_object.c
index 0ece717db0f85..df3f92b1a42e9 100644
--- a/net/rxrpc/conn_object.c
+++ b/net/rxrpc/conn_object.c
@@ -485,11 +485,13 @@ void rxrpc_destroy_all_connections(struct rxrpc_net *rxnet)
 	write_unlock(&rxnet->conn_lock);
 	BUG_ON(leak);
 
-	ASSERT(list_empty(&rxnet->conn_proc_list));
-
-	/* We need to wait for the connections to be destroyed by RCU as they
-	 * pin things that we still need to get rid of.
+	/* Connection destruction is normally deferred to system_wq because
+	 * the final-ACK timer is still pending when the last ref is dropped.
+	 * Wait for the deferred destructors (and the RCU frees) to complete
+	 * before checking conn_proc_list; they remove conns from it.
 	 */
 	wait_var_event(&rxnet->nr_conns, !atomic_read(&rxnet->nr_conns));
+
+	ASSERT(list_empty(&rxnet->conn_proc_list));
 	_leave("");
 }
-- 
2.43.0


From bsdhenrymartin at gmail.com  Mon Aug  3 00:33:55 2026
From: bsdhenrymartin at gmail.com (Henry Martin)
Date: Mon,  3 Aug 2026 15:33:55 +0800
Subject: [PATCH 3/3] rxrpc: recheck RXRPC_CALL_TX_NO_MORE after sleeping in rxrpc_send_data()
In-Reply-To: <20260803073355.3895949-1-bsdhenrymartin@gmail.com>
References: <20260803073355.3895949-1-bsdhenrymartin@gmail.com>
Message-ID: <20260803073355.3895949-4-bsdhenrymartin@gmail.com>

Commit ae4f89989479 ("rxrpc: Fix ability to add more data to a call
once MSG_MORE deasserted") added the RXRPC_CALL_TX_NO_MORE flag and an
entry check in rxrpc_send_data() to stop late sends on a finalized
call. However, the check runs only once at function entry: when the
transmit window is full, wait_for_space drops call->user_mutex and
sleeps, during which another thread sharing the same user_call_ID can
finalize the call (queue the LAST packet, setting TX_NO_MORE and
clearing send_queue). Upon waking, the reload path only re-verifies
the shutdown flag and call state - and a just-finalized call is still
in RXRPC_CALL_CLIENT_SEND_REQUEST until its data is hard-acked - so
the thread proceeds to rxrpc_alloc_txqueue() with send_queue == NULL
but tx_queue != NULL, triggering:

	WARNING: net/rxrpc/sendmsg.c:297 at rxrpc_alloc_txqueue

and returning a spurious -ENOMEM. The call state itself is entirely
legal at that point; the WARN_ON's implied assumption that
send_queue == NULL implies tx_queue == NULL simply does not hold after
finalization.

Re-check RXRPC_CALL_TX_NO_MORE in the reload path after reacquiring
user_mutex, mirroring the entry check, and bail out with -EPROTO.

Found by the autokbug dynamic kernel fuzzer at Tencent Yunding Lab.

Fixes: ae4f89989479 ("rxrpc: Fix ability to add more data to a call once MSG_MORE deasserted")
Signed-off-by: Henry Martin <bsdhenrymartin at gmail.com>
---
 net/rxrpc/sendmsg.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index ed2c9a51005ad..b39af552c6b46 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -358,6 +358,14 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	if (txb)
 		rxrpc_see_txbuf(txb, rxrpc_txbuf_see_send_more);
 
+	ret = -EPROTO;
+	if (test_bit(RXRPC_CALL_TX_NO_MORE, &call->flags)) {
+		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_late_send,
+				  call->cid, call->call_id, call->rx_consumed,
+				  0, -EPROTO);
+		goto maybe_error;
+	}
+
 	ret = -EPIPE;
 	if (sk->sk_shutdown & SEND_SHUTDOWN)
 		goto maybe_error;
-- 
2.43.0


From marsy12010123 at gmail.com  Tue Aug  4 05:33:30 2026
From: marsy12010123 at gmail.com (Qi Zhang)
Date: Tue,  4 Aug 2026 20:33:30 +0800
Subject: [PATCH net] afs: Clear stale peer app data after address list changes
Message-ID: <20260804123330.2214405-1-marsy12010123@gmail.com>

From: Chengfeng Ye <nicoyip.dev at gmail.com>

afs_fs_probe_fileserver() fetches the current endpoint state under
server->fs_lock, but leaves old_alist as NULL.  Consequently,
afs_set_peer_appdata() treats every address list replacement as initial
setup and only binds the new peers; it never unbinds peers removed from
the old list.

An address refresh can therefore proceed as follows.  CPU 0 replaces
server S's list and drops Pold without clearing Pold->app_data.  The
server destroyer then clears only S's current peers and lets S reach its
RCU callback.  After the callback frees S, CPU 1 handles a callback
through an RxRPC connection that still pins Pold, reads Pold->app_data,
and calls afs_use_server() on the freed object.

KASAN reported:

  BUG: KASAN: slab-use-after-free in afs_find_server+0x3c/0xa0
  Read of size 4 at addr ffff8881013e1af0 by task krxrpcio/7001/74
  Call Trace:
   afs_find_server+0x3c/0xa0
   afs_rx_new_call+0x15c/0x390
   rxrpc_new_incoming_call+0x97c/0x1730
   rxrpc_input_packet.constprop.0+0xd03/0xec0
   rxrpc_io_thread+0x967/0x1640
  Allocated by task 93:
   afs_lookup_server+0x1a7/0x14c0
   afs_alloc_server_list+0x43f/0xb60
   afs_create_volume+0x923/0x1490
   afs_get_tree+0x1c6/0x10a0
  Freed by task 0:
   kfree+0x131/0x3c0
   rcu_core+0x50a/0x1850
  Last potentially related work creation:
   __call_rcu_common.constprop.0+0x71/0xa10
   afs_put_server+0x213/0x2b0

Preserve old->addresses for the peer app-data update so that removed
peers are cleared before the endpoint state is replaced.  Also advance
both cursors when the old and new lists share a peer; activating the
old/new comparison without this would otherwise loop forever on the
shared entry.

Fixes: 40e8b52fe8c8 ("afs: Use the per-peer app data provided by rxrpc")
Cc: stable at vger.kernel.org
Signed-off-by: Chengfeng Ye <nicoyip.dev at gmail.com>
Signed-off-by: Qi Zhang <marsy12010123 at gmail.com>
---
 fs/afs/addr_list.c | 5 ++++-
 fs/afs/fs_probe.c  | 1 +
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/fs/afs/addr_list.c b/fs/afs/addr_list.c
index 63bf096b721a..73195d76b481 100644
--- a/fs/afs/addr_list.c
+++ b/fs/afs/addr_list.c
@@ -394,8 +394,11 @@ void afs_set_peer_appdata(struct afs_server *server,
 		struct rxrpc_peer *pn = new_alist->addrs[n].peer;
 		struct rxrpc_peer *po = old_alist->addrs[o].peer;
 
-		if (pn == po)
+		if (pn == po) {
+			n++;
+			o++;
 			continue;
+		}
 		if (pn < po) {
 			rxrpc_kernel_set_peer_data(pn, data);
 			n++;
diff --git a/fs/afs/fs_probe.c b/fs/afs/fs_probe.c
index a91ad1938d07..8c62334dbfe7 100644
--- a/fs/afs/fs_probe.c
+++ b/fs/afs/fs_probe.c
@@ -258,6 +258,7 @@ int afs_fs_probe_fileserver(struct afs_net *net, struct afs_server *server,
 					lockdep_is_held(&server->fs_lock));
 	if (old) {
 		estate->responsive_set = old->responsive_set;
+		old_alist = old->addresses;
 		if (!new_alist)
 			new_alist = old->addresses;
 	}
-- 
2.43.0



From syzbot+750f21d691e244b473b1 at syzkaller.appspotmail.com  Tue Aug  4 06:08:02 2026
From: syzbot+750f21d691e244b473b1 at syzkaller.appspotmail.com (syzbot)
Date: Tue, 04 Aug 2026 06:08:02 -0700
Subject: [syzbot] [afs?] INFO: task hung in afs_cell_purge (2)
In-Reply-To: <68185b7b.a70a0220.254cdc.0046.GAE@google.com>
Message-ID: <6a71e432.40259c87.584f4.04b6.GAE@google.com>

syzbot suspects this issue was fixed by commit:

commit c9c3b615a462a4023bd148f02c564e175ed10502
Author: David Howells <dhowells at redhat.com>
Date:   Mon Jun 22 09:08:45 2026 +0000

    afs: Fix misplaced inc of net->cells_outstanding

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=10a6be32580000
start commit:   7a13c14ee59d Merge tag 'for-6.15-rc4-tag' of git://git.ker..
git tree:       upstream
kernel config:  https://syzkaller.appspot.com/x/.config?x=a42a9d552788177b
dashboard link: https://syzkaller.appspot.com/bug?extid=750f21d691e244b473b1
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=13a101cc580000

If the result looks correct, please mark the issue as fixed by replying with:

#syz fix: afs: Fix misplaced inc of net->cells_outstanding

For information about bisection process see: https://goo.gl/tpsmEJ#bisection


From horms at kernel.org  Mon Aug  3 09:02:24 2026
From: horms at kernel.org (Simon Horman)
Date: Mon,  3 Aug 2026 17:02:24 +0100
Subject: [PATCH net v5 02/10] rxrpc: Fix packet encryption error handling
In-Reply-To: <20260729160108.2031453-3-dhowells@redhat.com>
References: <20260729160108.2031453-3-dhowells@redhat.com>
Message-ID: <20260803160224.431797-1-horms@kernel.org>

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev/#/patchset/20260729160108.2031453-2-dhowells at redhat.com?part=2
---
--- Patch [2]: [PATCH net v5 02/10] rxrpc: Fix packet encryption error handling ---
commit 59ddcd5fc94e6b2d9346d2e5e32c548184c0070b
Author: David Howells <dhowells at redhat.com>

rxrpc: Fix packet encryption error handling

This commit changes the error handling in rxrpc_send_data() when
secure_packet() fails. It adds logic to rewind the buffer by one byte on
retryable -ENOMEM errors and sets a new RXRPC_CALL_TX_ERROR flag for terminal
errors to prevent further sends.

Closes: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com

> diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
> index 4c754f78ece9a..d117a25b031dd 100644
> --- a/net/rxrpc/sendmsg.c
> +++ b/net/rxrpc/sendmsg.c
> @@ -330,12 +330,18 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
>  	bool more = msg->msg_flags & MSG_MORE;
>  	int ret, copied = 0;
>  
> -	if (test_bit(RXRPC_CALL_TX_NO_MORE, &call->flags)) {
> +	if (unlikely(test_bit(RXRPC_CALL_TX_NO_MORE, &call->flags))) {
>  		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_late_send,
>  				  call->cid, call->call_id, call->rx_consumed,
>  				  0, -EPROTO);
>  		return -EPROTO;
>  	}
> +	if (unlikely(test_bit(RXRPC_CALL_TX_ERROR, &call->flags))) {
> +		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_tx_error,
> +				  call->cid, call->call_id, call->rx_consumed,
> +				  0, -EIO);
> +		return -EIO;
> +	}

[Severity: High]
Does this check get bypassed after waiting for transmit space?

If the transmit window is full, rxrpc_send_data() jumps to wait_for_space
where it drops call->user_mutex and sleeps:

	ret = rxrpc_wait_for_tx_window(rx, call, &timeo,
				       msg->msg_flags & MSG_WAITALL);
    ...
	if (call->interruptibility == RXRPC_INTERRUPTIBLE) {
    ...
	} else {
		mutex_lock(&call->user_mutex);
	}
	goto reload;

When it reacquires the lock, it jumps to the reload label, which is located
after the RXRPC_CALL_TX_ERROR check.

If another thread encounters a terminal crypto error during the lock-drop
window and sets RXRPC_CALL_TX_ERROR, won't this thread blindly reuse the
corrupted call->tx_pending buffer when it wakes up?

[ ... ]

> @@ -440,12 +446,21 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
>  		/* add the packet to the send queue if it's now full */
>  		if (!txb->space ||
>  		    (msg_data_left(msg) == 0 && !more)) {
> -			if (msg_data_left(msg) == 0 && !more)
> -				txb->flags |= RXRPC_LAST_PACKET;
> -
> +			/* Do any required crypto.  If this fails, it could
> +			 * have corrupted the txbuf content with a partial
> +			 * encrypt.  Assume that ENOMEM is retryable, but
> +			 * everything else is terminal.
> +			 */
>  			ret = call->security->secure_packet(call, txb);
> -			if (ret < 0)
> +			if (ret < 0) {
> +				if (ret == -ENOMEM)
> +					goto maybe_error_rewind;

[ ... ]

> @@ -463,6 +478,20 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
>  	_leave(" = %d", call->error);
>  	return call->error;
>  
> +maybe_error_rewind:
> +	/* If we got a retryable error after copying all the supplied data into
> +	 * the last packet, we need to rewind the buffer by one byte so the
> +	 * caller knows they need to retry.
> +	 */
> +	if (copied && !more && !msg_data_left(msg)) {
> +		txb->space  += 1;
> +		txb->len    -= 1;
> +		txb->offset -= 1;
> +		copied      -= 1;
> +		if (call->tx_total_len != -1)
> +			call->tx_total_len += 1;
> +		iov_iter_revert(&msg->msg_iter, 1);
> +	}
>  maybe_error:

[Severity: High]
Does this error path drop the -ENOMEM error code and incorrectly return
success?

When secure_packet() fails with -ENOMEM on the final packet, it jumps to
maybe_error_rewind, and falls through to maybe_error.

If data was copied prior to the failure (copied > 0), maybe_error jumps
to success:

maybe_error:
	if (copied) {
		if (rxrpc_call_is_complete(call) &&
		    call->error < 0) {
			ret = call->error;
			goto out_txb;
		}
		goto success;
	}

Will this cause kernel callers like AFS (which use MSG_WAITALL) to see a
positive return value, assume the send succeeded, and hang forever waiting
for a server reply that will never arrive because the final packet was
dropped?

>  	if (copied) {
>  		if (rxrpc_call_is_complete(call) &&
>  		    call->error < 0) {


From horms at kernel.org  Mon Aug  3 09:06:27 2026
From: horms at kernel.org (Simon Horman)
Date: Mon, 3 Aug 2026 17:06:27 +0100
Subject: [PATCH net v5 07/10] afs: Create a server appdata key
In-Reply-To: <20260729160108.2031453-8-dhowells@redhat.com>
References: <20260729160108.2031453-1-dhowells@redhat.com>
 <20260729160108.2031453-8-dhowells@redhat.com>
Message-ID: <20260803160627.GL51943@horms.kernel.org>

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev/#/patchset/20260729160108.2031453-2-dhowells at redhat.com?part=7
---
--- Patch [7]: [PATCH net v5 07/10] afs: Create a server appdata key ---
commit 46d1f7800458c15b841827dd597a26f2786367f4
Author: David Howells <dhowells at redhat.com>
Subject: afs: Create a server appdata key

This patch introduces an appdata key for each AFS fileserver the client
communicates with using RxGK. It shifts the responsibility of generating
application data from responding to challenges in the OOB queue to creating
it when setting up the server record.

Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com

> diff --git a/fs/afs/cm_security.c b/fs/afs/cm_security.c
> index 103168c70dd4d..7f08ede12206a 100644
> --- a/fs/afs/cm_security.c
> +++ b/fs/afs/cm_security.c
[ ... ]
> @@ -23,6 +25,236 @@ static int afs_create_yfs_cm_token(struct sk_buff *challenge,
[ ... ]
> +	/* Store the appdata before the key pointer */
> +	smp_store_release(&server->yfs_rxgk_appdata, appdata_key);
> +
> +out_aead:
> +	crypto_free_aead(aead);
> +out_token:
> +	kfree(appdata);

Simon says:

Hi David,

I'm wondering if you could take a look into the issue flagged below.

[Severity: Medium]
Could this leave the plaintext cryptographic callback key (K0) in the slab
cache?

When creating an RxGK appdata key, afs_create_yfs_rxgk_cm_appdata()
allocates a temporary appdata buffer, generates a random cryptographic key
into it, encrypts it, and then frees the temporary buffer using kfree().

Can an unprivileged process or remote attacker who exploits an information leak
recover the plaintext key from the heap and forge RxGK callback connections?
Would kfree_sensitive() be more appropriate here to ensure the buffer is
securely zeroed?

> +out:
> +	return ret;
> +}
> +#endif /* CONFIG_RXGK */
[ ... ]

Simon says: But I don't think we need to consider the pre-existing
            issues flagged below in the context of this patch-set.

I noticed a couple of pre-existing issues while reviewing this code:

> fs/afs/cm_security.c:afs_respond_to_challenge()
>
> 			server = (struct afs_server *)peer_data;
> 			if (!server->cm_rxgk_appdata.data) {

[Severity: Critical]
This is a pre-existing issue, but is it safe to cast and dereference peer_data
here?

When a network CHALLENGE packet arrives and is processed in the OOB queue in
afs_respond_to_challenge(), it reads peer_data and casts it to an afs_server
pointer.

Since the code dereferences this pointer without holding rcu_read_lock() or
taking a reference on the server object, what happens if the server is
concurrently destroyed or updated?

If peer_data is set to 0, this causes a direct NULL dereference. If the
server memory is freed, could this result in a use-after-free triggered by a
remote attacker?


> fs/afs/server.c:afs_lookup_server()
>
> out:
> 	afs_put_addrlist(alist, afs_alist_trace_put_server_create);
> 	if (candidate) {
> 		kfree(rcu_access_pointer(server->endpoint_state));
> 		kfree(candidate);
> 		afs_dec_servers_outstanding(cell->net);
> 	}

[Severity: High]
This isn't a bug introduced by this patch, but could this cleanup path free
the active server's endpoint state?

When two threads concurrently execute afs_lookup_server() for a new server,
one thread installs its candidate and the other receives the installed server.
On the exit path (out:), if candidate is non-null, the second thread
erroneously calls kfree(rcu_access_pointer(server->endpoint_state)) instead
of freeing the candidate's state.

Also, if the first thread fails server creation, it sets server = NULL.
Would accessing server->endpoint_state here then trigger a direct NULL
pointer dereference?


From dhowells at redhat.com  Tue Aug  4 03:01:55 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:01:55 +0100
Subject: [PATCH v8 00/25] netfs: Keep track of folios in a segmented bio_vec[] chain
Message-ID: <20260804100224.2748935-1-dhowells@redhat.com>

Hi Christian,

Could you add these patches to the VFS tree for next?

The patches get rid of folio_queue, rolling_buffer and ITER_FOLIOQ,
replacing the folio queue construct used to manage buffers in netfslib with
one based around a segmented chain of bio_vec arrays instead.  There are
three main aims here:

 (1) The kernel file I/O subsystem seems to be moving towards consolidating
     on the use of bio_vec arrays, so embrace this by moving netfslib to
     keep track of its buffers for buffered I/O in bio_vec[] form.

 (2) Netfslib already uses a bio_vec[] to handle unbuffered/DIO, so the
     number of different buffering schemes used can be reduced to just a
     single one.

 (3) Always send an entire filesystem RPC request message to a TCP socket
     with single kernel_sendmsg() call as this is faster, more efficient
     and doesn't require the use of corking as it puts the entire
     transmission loop inside of a single tcp_sendmsg().

For the replacement of folio_queue, a segmented chain of bio_vec arrays
rather than a single monolithic array is provided:

	struct bvecq {
		struct bvecq		*next;
		struct bvecq		*prev;
		unsigned long long	fpos;
		refcount_t		ref;
		u32			priv;
		u16			nr_slots;
		u16			max_slots;
		enum bvecq_mem		mem_type:2;
		bool			inline_bv:1;
		bool			discontig:1;
		bool			from_pool:1;
		struct bio_vec		*bv;
		struct bio_vec		__bv[];
	};

The fields are:

 (1) next, prev - Link segments together in a list.  I want this to be
     NULL-terminated linear rather than circular to make it possible to
     arbitrarily glue bits on the front.

 (2) fpos, discontig - Note the current file position of the first byte of
     the segment and whether this bvecq is discontiguous with the previous.
     When accessing the pagecache to clear flags/locks, the fpos can be
     used to look up folios by file position rather than by finding those
     folios from the info stored in the bio_vecs.

     When the file position is relevant, the model I'm working with is that
     all the segments pointed to by a single bvecq must represent
     contiguous data, but adjacent bvecqs within a chain need not be
     contiguous.  This allows a bvecq chain to be used to provide bufferage
     for a sparse read or write RPC such as can be done with Ceph.

     If a bvecq segment is not contiguous with the previous one,
     ->discontig should be set (this is technically redundant if one keeps
     track of the fpos as a bvecq chain is processed).

     Note that the beginning and end file positions in a segment need not
     be aligned to any filesystem block size.

 (3) ref - Refcount.  Each bvecq keeps a ref on the next.  I'm not sure
     this is entirely necessary, but it makes sharing slices easier.

 (4) priv - Private data for the owner.  Dispensible; currently only used
     for storing a debug ID for tracing in a patch not included here.

 (5) max_slots, nr_slots.  The size of bv[] and the number of slots used.
     I've assumed a maximum of 65535 bio_vecs in the array (which would
     represent a ~1MiB allocation).

 (6) bv, __bv, inline_bv.  bv points to the bio_vec[] array handled by
     this segment.  This may begin at __bv and if it does inline_bv should
     be set (otherwise it's impossible to distinguish a separately
     allocated bio_vec[] that follows immediately by coincidence).

 (7) mem_type.  Indicates how the memory attached to the bio_vecs should be
     disposed of when the bvecq is destroyed.  It can be one of:

	BVECQ_MEM_EXTERNAL	- Externally tracked ref; don't put
	BVECQ_MEM_PAGECACHE	- Pagecache; must be put
	BVECQ_MEM_GUP		- Pinned by from GUP; needs unpin
	BVECQ_MEM_ALLOCED	- Plain alloc'd pages; can be mempooled

     [!] I'm not sure that this is a good name for this member or for the
     	 enum values.

 (8) from_pool.  Set if the bvecq is allocated from netfslib's mempool and
     should be freed to it.

I've also defined an iov_iter iterator type ITER_BVECQ to walk this sort of
construct so that it can be passed directly to sendmsg() or block-based DIO
(as cachefiles does).


This series makes the following changes to netfslib:

 (1) Remove the writethrough code as the locking is really tricky to get
     right and it looks like it could deadlock with Ceph if snapshots are
     used.

 (2) The folio_queue chain used to hold folios for buffered I/O is replaced
     with a bvecq chain.  Each bio_vec then holds (a portion of) one folio.
     Each bvecq holds a contiguous sequence of folios, but adjacent bvecqs
     in a chain may be discontiguous.

 (3) For unbuffered/DIO, the source iov_iter is extracted into a bvecq
     chain.

 (4) An abstract position representation ('bvecq_pos') is created that can
     used to hold a position in a bvecq chain.  For the moment, this takes
     a ref on the bvecq it points to, but that may be excessive.

 (5) Buffer tracking is managed with three cursors:  The load_cursor, at
     which new folios are added as we go; the dispatch_cursor, at which new
     subrequests' buffers start when they're created; and the
     collect_cursor, the point at which folios are being unlocked.

     Not all cursors are necessarily needed in all situations and during
     buffered writeback, we need a dispatch cursor per stream (one for the
     network filesystem and one for the cache).

 (6) ->prepare_read(), buffer setting up and ->issue_read() are merged, as
     are the write variants, with the filesystem calling back up to
     netfslib to prepare its buffer.  This simplifies the process of
     setting up a subrequest.  It may even make sense to have the
     filesystem allocate the subrequest.

 (7) Retry dispatch tracking is added to netfs_io_request so that the
     buffer preparation functions can find it.  Retry requires an
     additional buffer cursor.

 (8) Netfslib dispatches I/O by accumulating enough bufferage to dispatch
     at least one subrequest, then looping to generate as many as the
     filesystem wants to (they may be limited by other constraints,
     e.g. max RDMA segment count or negotiated max size).  This loop could
     be moved down into the filesystem.  A new method is provided by which
     netfslib can ask the filesystem to provide an estimate of the data
     that should be accumulated before dispatch begins.

 (9) Reading from the cache is now managed by querying the cache to provide
     a list of the next two data extents within the cache.

(10) AFS directories are switched to using a bvecq rather than a
     folio_queue to hold their contents.

(11) CIFS is switch to using a bvecq rather than a folio_queue for holding
     a temporary encryption buffer.

(12) CIFS RDMA is given the ability to extract ITER_BVECQ and support for
     extracting ITER_FOLIOQ is removed.

(13) All the folio_queue and rolling_buffer code is removed.

Cachefiles is also modified:

 (1) On-demand mode for Erofs is removed.

 (2) The object type in the cachefiles file xattr is now correctly set to
     CACHEFILES_CONTENT_{SINGLE,ALL,BACKFS_MAP} rather than just being 0,
     to indicate whether we have a single monolithic blob, all the data up
     to cache i_size with no holes or a sparse file with the data mapped by
     the backing file system (as currently upstream).

 (3) For "ALL" type files, the cache's i_size is used to track how much
     data is saved in the cache and no longer bears any relation to the
     netfs i_size.  The actual object size is stored in the xattr.

 (4) For most typical files which are contiguous and written progressively,
     the object type is now set to "ALL".  For anything else, cachefiles
     uses SEEK_DATA/HOLE to find extent outlines at before (this is the
     current behaviour and needs to be fixed, but in a separate set of
     patches as it's not trivial).

 (5) Preset the xattr on a cachefile to try to avoid having to deal with
     ENOSPC from setxattr when committing the object.

Two further things that I'm working on (but not in this branch) are:

 (1) Make it so that a filesystem can be given a copy of a subchain which
     it can then tack header and trailer protocol elements upon to form a
     single message (I have this working for cifs) and even join copies
     together with intervening protocol elements to form compounds.

 (2) Make it so that a filesystem can 'splice' out the contents of the TCP
     receive queue into a bvecq chain.  This allows the socket lock to be
     dropped much more quickly and the copying of data read to the
     destination buffers to happen without the lock.  I have this working
     for cifs too.  Kernel recvmsg() doesn't then block kernel sendmsg()
     for anywhere near as long.

There are also some things I want to consider for the future:

 (1) Create one or more batched iteration functions to 'unlock' all the
     folios in a bio_vec[], where 'unlock' is the appropriate action for
     ending a read or a write.  Batching should hopefully also improve the
     efficiency of wrangling the marks on the xarray.  Very often these
     marks are going to be represented by contiguous bits, so there may be
     a way to change them in bulk.

 (2) Rather than walking the bvecq chain to get each individual folio out
     via bv_page, use the file position stored on the bvecq and the sum of
     bv_len to iterate over the appropriate range in i_pages.

 (3) Change iov_iter to store the initial starting point and for
     iov_iter_revert() to reset to that and advance.  This would (a) help
     prevent over-reversion and (b) dispense with the need for a prev
     pointer.

 (4) Use bvecq to replace scatterlist.  One problem with replacing
     scatterlist is that crypto drivers like to glue bits on the front of
     the scatterlists they're given (something trivial with that API) - and
     this is one way to achieve it.

The patches can also be found here:

	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-next

Thanks,
David

Changes
=======
ver #8)
- Removed the fix patches to their own branches and posted separately.
- Rebased on v7.2-rc6 as a number of fixes went in.
- Fixed cachefiles_collect_write() to handle cres->cache_priv2 (file) being
  NULL due to failure to create an object (e.g. due to ENOSPC).
- Added a patch to cachefiles to preset the xattr on a cachefiles to avoid
  dealing with ENOSPC from setxattr when committing an object.
- Fixed a number of bugs reported by Sashiko[6]:
  - Fixed bvecq_alloc_buffer2() to break out of the loop if pre_slots > 0
    and size == 0.
  - Made netfs_extract_iter() limit max_pages to protect iov_iter_npages().
  - Remove include/uapi/linux/cachefiles.h when ondemand mode is removed.
  - Remove more ondemand-related trace bits.
  - Remove ondemand-related BPF bits.
  - Fixed a missing ENOMEM check in netfs_pgpriv2_begin_copy_to_cache().
  - Fixed netfs_retry_read_subrequests() to unset dispatch_cursor on the
    "abandon_after" error path.
  - Fixed netfs_issue_write() to clear stream->construct if a subreq is
    terminated due to ENOMEM.
  - Fixed netfs_collect_read_results() to unpause the app if it decides to
    abandon the read.
  - Fixed cachefiles_issue_read() to not wait for the object state to match
    FSCACHE_WANT_READ as cachefiles_query_occupancy() should have done that
    already.
  - Fixed netfs_retry_write_stream() to handle superfluous subreqs
    correctly (borrowing from read retry).

ver #7)
- Rebased on v7.2-rc4.
- Added three fix patches to the front from Yichong Chen that conflict with
  other patches in the series.
- Fixed a number of bugs reported by Sashiko[5].
  - Fixed double page put in iov_kunit_destroy_bvecq().
  - Fixed bvecq_alloc_buffer2() to handle pre_slots>0 and size==0
    correctly.
  - Fixes bvecq_expand_buffer() to use a barrer to set tail->nr_slots.
  - Added functions to insert barriers when reading or setting bvecq::next
    and fixed some places to use them.
  - Fixed more meta block mapping in afs_edit_dir_remove().
  - Added a patch to fix missing unmap in afs_dir_search_bucket().
  - Removed the writethrough stat counter as the code gets removed.
  - Cleaned up the cachefiles-ondemand removal patch:
    - Fix removal of code that's conditional on !ONDEMAND.
    - Removed msg_id_next field.
    - Removed ondemand trace elements.
    - Removed documentation.
  - Fixed cachefiles_resize_cookie() to only consider a resize a shrink if
    the new size is less than the object size.
  - Fixed cachefiles_resize_cookie() to update object->object_size on
    expansion.
  - Add a comment on exclusion in cachefiles_collect_write().
  - Fixed netfs_single_dispatch_read() to allow for the cache to have
    rounded out the stored data to DIO size.
  - Fixed netfs_prepare_read_iterator() to clamp subreq len to sreq_max_len
    (obsoleted by last patch).
  - Changed write subreq collection to never retry cache writes.  Shouldn't
    see them in the retry path either then.
  - Fixed netfs_pgpriv2_unlock_copied_folios() to return made_progress
    instead of false if we hit the end of the bvecq chain.
  - Modified buffered read collection to abandon the rest of a read request
    if a subreq over-reads as the buffer may have been corrupted.  The read
    termination function no longer winds subreq->transferred back.
  - Fixed netfs_issue_read() to call the right termination func in the fail
    path.
  - Fixed netfs_dispatch_unbuffered_reads(), netfs_single_dispatch_read()
    and netfs_retry_read_subrequests() to handle failed ->issue_read().
  - Fixed afs_issue_read() to return after calling afs_end_read() as this
    op is then released.
  - Fixed cifs_issue_write() to always release credits on failure.
  - Fixed cifs_issue_read() to not release credits twice.
  - Fixed netfs_issue_writes() to advance stream->issued_to and
    stream->dispatch_cursor on early failure of ->issue_write().
  - Fixed netfs_issue_read() to only call netfs_all_subreqs_queued() if
    stream->buffered is zero.
  - Fixed ceph_netfs_issue_op_inline() to do a retry if file got uninlined
    by someone else.
  - Fixed ceph_netfs_issue_op_inline() to not return an error directly
    once it's got a prepared buffer.
  - Fixed nfs_netfs_issue_read() to return an int.

ver #6)
- Rebased on v7.2-rc3.
- Added a patch to remove erofs on-demand support from cachefiles.
- Added a patch to remove the writethrough code as it's really tricky to
  get the locking right and it looks like can deadlock itself with Ceph.
- Fixed a number of bugs reported by Sashiko[4].
  - Fixed iov_iter_single_seg_count(), iov_iter_alignment_bvecq() and
    iov_npages_bvecq() to handle zero-length segments and segments with no
    stuff remaining.
  - Fixed bvecq_extract() to use GFP_NOFS rather than GFP_KERNEL.
  - Fixed bvecq_alloc_buffer2()'s use of alloc_pages_bulk().
  - Fixed netfs_extract_iter() to break out of the inner loop if max_pages
    hits zero.
  - Added a patch to fix meta block mapping in afs_edit_dir_remove().
  - Fixed afs_init_new_symlink() to clear the tail of the symlink page.
  - Fixed cachefiles_begin_operation() to deal with a NULL file.
  - Fixed slot check in netfs_read_gaps() to be >=, not >.
  - Fixed netfs_pgpriv2_copy_folio() to update creq->last_end.
  - Fixed netfs_read_single() to return -EIO if the buffer isn't big
    enough.
  - Add a comment to netfs_write_folio() to indicate that truncate must
    not run concurrently with writeback and must exclude/wait for it.
  - In netfs_read_subreq_terminated(), move the netfs_sreq_trace_too_much
    trace line earlier before ->transferred is clobbered.
  - In netfs_read_subreq_terminated(), make the transfer-too-long case
    avoid retrying and just end with EIO (in case data further along in the
    buffer got corrupted).
  - Fixed netfs_collect_read_results() if it sees front->transferred >
    front->len, to just use front->len instead.
  - Fixed ceph_netfs_issue_op_inline() to prep the buffer before setting
    NETFS_SREQ_HIT_EOF so that subreq->content is set.
  - Alter netfs_read_to_pagecache() to not round up reading from
    zero-containing cache granules as no I/O is required.
  - Made ->issue_write() return an error so that errors before any buffer
    is sliced off don't result in an infinite loop in netfs_issue_writes().
  - Made ->issue_read() return an error also for similiar reasons.
  - In netfs_writepages(), handle -ENOMEM from bvecq prealloc before
    caling netfs_queue_wb_folio() rather than doing it on the combined
    error return of that and netfs_issue_streams() which can result in a
    double unlock.
  - Fix netfs_writeback_single() to write nothing if len == 0.

ver #5)
- Rebased on v7.2-rc2 as that has a bunch of outstanding netfs and afs
  bugfixes included.

ver #4)
- Fixed a number of bugs reported by Sashiko[3].
  - Added a patch to fix an underflow in iov_iter_extract_xarray_pages().
  - Added a patch to fix alloc failure in iov_iter_extract_bvec_pages().
  - Added a patch to remove an unused var in kunit code.
  - Added a patch to fix the folio offset in extract_xarray_to_sg().
  - Added a patch to fix the exclusion over writeback to make it cover
    collection too.
  - Fixed double fput() in cachefiles.
  - Fixed the collection of cache writes to handle cancellation better.
  - Fixed iterate_bvecq() to skip bvecq structs with nr_slots==0.
  - Add a comment into iterate_bvecq() that a slot with bv_len>0 must have a
    valid bv_page.
  - Fixed iov_iter_bvecq_advance(), iov_iter_bvecq_revert(),
    iter_count_bvecq_pages(), iov_iter_extract_bvecq_pages() and
    extract_bvecq_to_sg() to correctly handle empty bvecqs.
  - Fixed extract_bvecq_to_sg() to be limited by iter->count.
  - Fixed bvecq_expand_buffer() to take an unsigned size param.
  - Fixed bvecq_expand_buffer() to not mix memory types in alloc'd bvecqs.
  - Fixed bvecq_shorten_buffer() occasional retention of zero-length slots.
  - Fixed slot validity check polarity in bvecq_pos_advance(); also don't use
    inner loop otherwise break then exits the wrong loop.
  - Fixed bvecq_zero(), bvecq_slice() and bvecq_extract to use a barrier when
    checking bq->nr_slots.
  - Restructured bvecq_zero() to be similar to bvecq_pos_advance().
  - Fixed an off-by-one error in bvecq_pos_step() and added a missing slot
    reset.
  - Fixed a break in netfs_extract_iter() that should have been a goto.
  - Fixed netfs_extract_iter() to limit number of pages extracted to remnant
    of max_pages.
  - Fixed an uninit var in afs_do_read_symlink().
  - Fixed netfs_read_gaps() to fill a multipart bvecq chain correctly.
  - Fixed netfs_dispatch_unbuffered_reads() to initialise collect_cursor as
    netfs_rreq_assess_dio() uses it to flush the data read.
  - Fixed netfs_extract_iter() to init the slot counter outside the extract
    loop to avoid overwriting already loaded slots.
  - Fixed callers of bvecq_delete_spent() to update bvecq_pos::slot before
    calling.
  - Fixed netfs_reissue_write() to make sure subreq->content is unset before
    setting.
  - Altered netfs_extract_iter() to free any allocated bvecq chain if no pages
    were extracted and an error occurred  (and to initialise the return
    pointer to NULL).  Also, made it return an empty bvecq if nothing was
    extracted, but no error occurred.
  - Fixed ceph_netfs_issue_read() to just return if
    ceph_netfs_issue_op_inline() returns anything other than 1 to avoid a
    double termination.
  - Fixed ceph_netfs_issue_read() to do the size calculation in the right
    order to avoid the op expanding to larger than the buffer.
  - Fixed netfs_issue_read(), in the NETFS_FILL_WITH_ZEROES case, to deduct
    subreq->len from stream->buffered rather than just setting it to 0.
  - Fixed netfs_perform_write() to put the folio if netfs_advance_writethrough()
    fails.
  - Restored the old ->prepare_write op specifically for
    fscache_write_to_cache() which is still used by Ceph.
  - Fixed undefined return in netfs_pgpriv2_issue_stream().
  - Fixed netfs_collect_write_results() to try to make sure a request isn't
    left paused if there are no further server-bound subreqs.
  - Fixed netfs_queue_wb_folio() to redirty the folio before unlocking it if
    it can't allocate a bvecq.
  - Fixed netfs_writepages() to cancel the pagecache iteration after ENOMEM.
  - Fixed netfs_advance/end_writethrough() to advance the dispatch cursor.
  - Fixed netfs_retry_read_subrequests() to use barriers when walking
    stream->subrequests as the app may add another subreq before pausing.
  - Fixed netfs_prepare_write_retry_buffer() to use ->retry_start and
    ->retry_buffered rather than ->issue_from and ->buffered.
  - Fixed netfs_retry_write_stream() to use barriers when walking
    stream->subrequests as the app may add another subreq before pausing.
  - Fixed netfs_retry_write_stream() to check the correct length when adding
    additional subreqs.
  - Fixed nfs_netfs_issue_read() to set -ENOMEM, not 0, on alloc failure.
  - Fixed nfs_netfs_issue_read() to only terminate the subreq once.
  - Fixed cifs_issue_read() to release the credits if cifs_reopen_file()
    fails.
- Rebased on v7.1.

ver #3)
- Rebased to -rc7 as the patches wouldn't apply for Christian.
- Prepended a fix for a warning from generic/464 (the problem also exists
  upstream, just not the warning).
- Renamed kmap_local_bvec() to bvec_kmap_partial() as requested by
  Christoph.
- Adjusted smbdirect patch descriptions as requested by Stefan Metzmacher.

ver #2)
- Fixed a number of bugs reported by Sashiko[1].
- Split a bunch of fixes out and posted them separately[2].

[1] https://sashiko.dev/#/patchset/20260326104544.509518-1-dhowells%40redhat.com
[2] https://lore.kernel.org/linux-fsdevel/20260512-infozentrum-becher-7f86c47c96c8 at brauner/T/#t
[3] https://sashiko.dev/#/patchset/20260608145432.681865-1-dhowells%40redhat.com
[4] https://sashiko.dev/#/patchset/20260706153408.1231650-1-dhowells%40redhat.com
[5] https://sashiko.dev/#/patchset/20260716103030.3065561-1-dhowells%40redhat.com
[6] https://sashiko.dev/#/patchset/20260722130218.78958-1-dhowells%40redhat.com

David Howells (24):
  mm: Make readahead store folio count in readahead_control
  netfs: Bulk load the readahead-provided folios up front
  Add a function to kmap one page of a multipage bio_vec
  iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
  iov_iter: Add a segmented queue of bio_vec[]
  netfs: Add some tools for managing bvecq chains
  netfs: Make mempool available for bvecq
  netfs: Add a function to extract from an iter into a bvecq
  afs: Use a bvecq to hold dir content rather than folioq
  cifs: Use a bvecq for buffering instead of a folioq
  smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
  netfs: Remove the writethrough code
  cachefiles: Don't rely on backing fs storage map for most use cases
  netfs: Add the cache object ID to netfs_read/write tracepoints
  netfs: Switch to using bvecq rather than folio_queue and
    rolling_buffer
  smbdirect: Remove support for ITER_FOLIOQ from
    smbdirect_map_sges_from_iter()
  netfs: Remove netfs_alloc/free_folioq_buffer()
  netfs: Remove netfs_extract_user_iter()
  iov_iter: Remove ITER_FOLIOQ
  netfs: Remove folio_queue and rolling_buffer
  netfs: Check for too much data being read
  netfs: Limit the minimum trigger for progress reporting
  netfs: Combine prepare and issue ops and grab the buffers on request
  cachefiles: Preset the state xattr when creating a new file

Gao Xiang (1):
  cachefiles,netfs: sunset ondemand mode

 Documentation/core-api/folio_queue.rst        |  209 ----
 Documentation/core-api/index.rst              |    1 -
 .../filesystems/caching/cachefiles.rst        |  179 ---
 Documentation/filesystems/netfs_library.rst   |    2 +-
 fs/9p/vfs_addr.c                              |   61 +-
 fs/afs/dir.c                                  |   40 +-
 fs/afs/dir_edit.c                             |   43 +-
 fs/afs/dir_search.c                           |   33 +-
 fs/afs/file.c                                 |   31 +-
 fs/afs/fsclient.c                             |    8 +-
 fs/afs/inode.c                                |    2 +-
 fs/afs/internal.h                             |   14 +-
 fs/afs/symlink.c                              |   37 +-
 fs/afs/write.c                                |   36 +-
 fs/afs/yfsclient.c                            |    6 +-
 fs/cachefiles/Kconfig                         |   12 -
 fs/cachefiles/Makefile                        |    1 -
 fs/cachefiles/daemon.c                        |   76 +-
 fs/cachefiles/interface.c                     |   98 +-
 fs/cachefiles/internal.h                      |  144 +--
 fs/cachefiles/io.c                            |  600 ++++++---
 fs/cachefiles/namei.c                         |   42 +-
 fs/cachefiles/ondemand.c                      |  761 ------------
 fs/cachefiles/xattr.c                         |   84 +-
 fs/ceph/Kconfig                               |    1 +
 fs/ceph/addr.c                                |  123 +-
 fs/netfs/Kconfig                              |    3 +
 fs/netfs/Makefile                             |    4 +-
 fs/netfs/buffered_read.c                      |  529 +++++---
 fs/netfs/buffered_write.c                     |   59 +-
 fs/netfs/bvecq.c                              |  787 ++++++++++++
 fs/netfs/direct_read.c                        |  115 +-
 fs/netfs/direct_write.c                       |  162 +--
 fs/netfs/fscache_io.c                         |    4 +-
 fs/netfs/internal.h                           |  109 +-
 fs/netfs/iterator.c                           |  393 ++----
 fs/netfs/main.c                               |   13 +-
 fs/netfs/misc.c                               |  168 +--
 fs/netfs/objects.c                            |   22 +-
 fs/netfs/read_collect.c                       |  209 ++--
 fs/netfs/read_pgpriv2.c                       |  204 ++-
 fs/netfs/read_retry.c                         |  254 ++--
 fs/netfs/read_single.c                        |  182 ++-
 fs/netfs/rolling_buffer.c                     |  226 ----
 fs/netfs/stats.c                              |   10 +-
 fs/netfs/write_collect.c                      |  246 +++-
 fs/netfs/write_issue.c                        | 1089 +++++++++--------
 fs/netfs/write_retry.c                        |  187 +--
 fs/nfs/Kconfig                                |    1 +
 fs/nfs/fscache.c                              |   24 +-
 fs/smb/client/cifsglob.h                      |    2 +-
 fs/smb/client/cifssmb.c                       |   13 +-
 fs/smb/client/file.c                          |  145 +--
 fs/smb/client/smb2ops.c                       |   84 +-
 fs/smb/client/smb2pdu.c                       |   28 +-
 fs/smb/client/transport.c                     |   15 +-
 fs/smb/smbdirect/connection.c                 |  135 +-
 include/linux/bvec.h                          |   18 +
 include/linux/bvecq.h                         |  361 ++++++
 include/linux/folio_queue.h                   |  282 -----
 include/linux/fscache.h                       |   17 +
 include/linux/iov_iter.h                      |   87 +-
 include/linux/netfs.h                         |  184 +--
 include/linux/pagemap.h                       |   10 +
 include/linux/rolling_buffer.h                |   61 -
 include/linux/uio.h                           |   17 +-
 include/trace/events/cachefiles.h             |  209 +---
 include/trace/events/netfs.h                  |  161 ++-
 include/uapi/linux/cachefiles.h               |   68 -
 kernel/bpf/btf.c                              |    9 -
 lib/iov_iter.c                                |  557 +++++----
 lib/scatterlist.c                             |   82 +-
 lib/tests/kunit_iov_iter.c                    |  131 +-
 mm/readahead.c                                |    5 +
 net/9p/client.c                               |    8 +-
 75 files changed, 5101 insertions(+), 5232 deletions(-)
 delete mode 100644 Documentation/core-api/folio_queue.rst
 delete mode 100644 fs/cachefiles/ondemand.c
 create mode 100644 fs/netfs/bvecq.c
 delete mode 100644 fs/netfs/rolling_buffer.c
 create mode 100644 include/linux/bvecq.h
 delete mode 100644 include/linux/folio_queue.h
 delete mode 100644 include/linux/rolling_buffer.h
 delete mode 100644 include/uapi/linux/cachefiles.h



From dhowells at redhat.com  Tue Aug  4 03:01:56 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:01:56 +0100
Subject: [PATCH v8 01/25] mm: Make readahead store folio count in readahead_control
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-2-dhowells@redhat.com>

Make readahead store folio count in readahead_control so that the
filesystem can know in advance how many folios it needs to keep track of.

This is cleared by read_pages() in case it is called from a loop.

The count is accessed by the filesystem with readahead_folio_count().

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: netfs at lists.linux.dev
cc: linux-mm at kvack.org
cc: linux-fsdevel at vger.kernel.org
---
 include/linux/pagemap.h | 10 ++++++++++
 mm/readahead.c          |  5 +++++
 2 files changed, 15 insertions(+)

diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 2c3718d592d6..e1e51bace388 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -1348,6 +1348,7 @@ struct readahead_control {
 	struct file_ra_state *ra;
 /* private: use the readahead_* accessors instead */
 	pgoff_t _index;
+	unsigned int _nr_folios;
 	unsigned int _nr_pages;
 	unsigned int _batch_count;
 	bool dropbehind;
@@ -1527,6 +1528,15 @@ static inline size_t readahead_batch_length(const struct readahead_control *rac)
 	return rac->_batch_count * PAGE_SIZE;
 }
 
+/**
+ * readahead_folio_count - Get the number of folios in this readahead request.
+ * @rac: The readahead request.
+ */
+static inline unsigned int readahead_folio_count(const struct readahead_control *rac)
+{
+	return rac->_nr_folios;
+}
+
 static inline unsigned long dir_pages(const struct inode *inode)
 {
 	return (unsigned long)(inode->i_size + PAGE_SIZE - 1) >>
diff --git a/mm/readahead.c b/mm/readahead.c
index 558c92957518..069ded56fd80 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -188,6 +188,7 @@ static void read_pages(struct readahead_control *rac)
 	if (unlikely(rac->_workingset))
 		psi_memstall_leave(&rac->_pflags);
 	rac->_workingset = false;
+	rac->_nr_folios = 0;
 
 	BUG_ON(readahead_count(rac));
 }
@@ -303,6 +304,7 @@ void page_cache_ra_unbounded(struct readahead_control *ractl,
 		if (i == mark)
 			folio_set_readahead(folio);
 		ractl->_workingset |= folio_test_workingset(folio);
+		ractl->_nr_folios++;
 		ractl->_nr_pages += min_nrpages;
 		i += min_nrpages;
 	}
@@ -473,6 +475,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index,
 		return err;
 	}
 
+	ractl->_nr_folios++;
 	ractl->_nr_pages += 1UL << order;
 	ractl->_workingset |= folio_test_workingset(folio);
 	return 0;
@@ -822,6 +825,7 @@ void readahead_expand(struct readahead_control *ractl,
 			ractl->_workingset = true;
 			psi_memstall_enter(&ractl->_pflags);
 		}
+		ractl->_nr_folios++;
 		ractl->_nr_pages += min_nrpages;
 		ractl->_index = folio->index;
 	}
@@ -851,6 +855,7 @@ void readahead_expand(struct readahead_control *ractl,
 			ractl->_workingset = true;
 			psi_memstall_enter(&ractl->_pflags);
 		}
+		ractl->_nr_folios++;
 		ractl->_nr_pages += min_nrpages;
 		if (ra) {
 			ra->size += min_nrpages;



From dhowells at redhat.com  Tue Aug  4 03:01:57 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:01:57 +0100
Subject: [PATCH v8 02/25] netfs: Bulk load the readahead-provided folios up front
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-3-dhowells@redhat.com>

Load all the folios by the VM for readahead up front into the folio queue.
With the number of folios provided by the VM, the folio queue can be fully
allocated first and then the loading happen in one go inside the RCU read
lock.  The folio refs acquired from readahead are dropped in bulk once the
first subrequest is dispatched as it's quite a slow operation.  The
collector waits for NETFS_RREQ_NEED_PUT_RA_REFS to be cleared so that it
doesn't unlock folios before the xarray has been scanned for them.

This simplifies the buffer handling later and isn't noticeably slower as
the xarray doesn't need to be modified and the folios are all already
pre-locked.

Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: netfs at lists.linux.dev
cc: linux-mm at kvack.org
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/buffered_read.c       | 99 ++++++++++++++++++++--------------
 fs/netfs/internal.h            |  1 +
 fs/netfs/misc.c                | 19 +++++++
 fs/netfs/read_collect.c        |  7 +++
 fs/netfs/rolling_buffer.c      | 75 ++++++++++++++++++++++++++
 include/linux/netfs.h          |  1 +
 include/linux/rolling_buffer.h |  3 ++
 include/trace/events/netfs.h   |  3 ++
 8 files changed, 169 insertions(+), 39 deletions(-)

diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 7fdfa4f27e34..4752d9325c4b 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -54,6 +54,42 @@ static void netfs_rreq_expand(struct netfs_io_request *rreq,
 	}
 }
 
+/*
+ * Drop the folio refs acquired from the readahead API.
+ */
+static void netfs_bulk_drop_ra_refs(struct netfs_io_request *rreq)
+{
+	struct folio_batch fbatch;
+	struct folio *folio;
+	pgoff_t nr_pages = DIV_ROUND_UP(rreq->len, PAGE_SIZE);
+	pgoff_t first = rreq->start / PAGE_SIZE;
+	XA_STATE(xas, &rreq->mapping->i_pages, first);
+
+	folio_batch_init(&fbatch);
+
+	rcu_read_lock();
+
+	xas_for_each(&xas, folio,  first + nr_pages - 1) {
+		if (xas_retry(&xas, folio))
+			continue;
+
+		if (!folio_batch_add(&fbatch, folio))
+			folio_batch_release(&fbatch);
+	}
+
+	rcu_read_unlock();
+	folio_batch_release(&fbatch);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_ra_put_ref);
+	clear_bit_unlock(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags);
+	wake_up(&rreq->waitq);
+}
+
+static void netfs_maybe_bulk_drop_ra_refs(struct netfs_io_request *rreq)
+{
+	if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+		netfs_bulk_drop_ra_refs(rreq);
+}
+
 /*
  * Begin an operation, and fetch the stored zero point value from the cookie if
  * available.
@@ -74,12 +110,8 @@ static int netfs_begin_cache_read(struct netfs_io_request *rreq, struct netfs_in
  *
  * Returns the limited size if successful and -ENOMEM if insufficient memory
  * available.
- *
- * [!] NOTE: This must be run in the same thread as ->issue_read() was called
- * in as we access the readahead_control struct.
  */
-static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq,
-					   struct readahead_control *ractl)
+static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	size_t rsize = subreq->len;
@@ -87,30 +119,6 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq,
 	if (subreq->source == NETFS_DOWNLOAD_FROM_SERVER)
 		rsize = umin(rsize, rreq->io_streams[0].sreq_max_len);
 
-	if (ractl) {
-		/* If we don't have sufficient folios in the rolling buffer,
-		 * extract a folioq's worth from the readahead region at a time
-		 * into the buffer.  Note that this acquires a ref on each page
-		 * that we will need to release later - but we don't want to do
-		 * that until after we've started the I/O.
-		 */
-		struct folio_batch put_batch;
-
-		folio_batch_init(&put_batch);
-		while (rreq->submitted < subreq->start + rsize) {
-			ssize_t added;
-
-			added = rolling_buffer_load_from_ra(&rreq->buffer, ractl,
-							    &put_batch);
-			if (added < 0) {
-				folio_batch_release(&put_batch);
-				return added;
-			}
-			rreq->submitted += added;
-		}
-		folio_batch_release(&put_batch);
-	}
-
 	subreq->len = rsize;
 	if (unlikely(rreq->io_streams[0].sreq_max_segs)) {
 		size_t limit = netfs_limit_iter(&rreq->buffer.iter, 0, rsize,
@@ -208,8 +216,7 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
  * slicing up the region to be read according to available cache blocks and
  * network rsize.
  */
-static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
-				    struct readahead_control *ractl)
+static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 {
 	unsigned long long start = rreq->start;
 	ssize_t size = rreq->len;
@@ -288,7 +295,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
 		break;
 
 	issue:
-		slice = netfs_prepare_read_iterator(subreq, ractl);
+		slice = netfs_prepare_read_iterator(subreq);
 		if (slice < 0) {
 			ret = slice;
 			netfs_cancel_read(subreq, ret);
@@ -302,6 +309,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
 		}
 
 		netfs_issue_read(rreq, subreq);
+		netfs_maybe_bulk_drop_ra_refs(rreq);
 
 		if (test_bit(NETFS_RREQ_PAUSE, &rreq->flags))
 			netfs_wait_for_paused_read(rreq);
@@ -340,6 +348,7 @@ void netfs_readahead(struct readahead_control *ractl)
 	struct netfs_io_request *rreq;
 	struct netfs_inode *ictx = netfs_inode(ractl->mapping->host);
 	unsigned long long start = readahead_pos(ractl);
+	ssize_t added;
 	size_t size = readahead_length(ractl);
 	int ret;
 
@@ -360,11 +369,23 @@ void netfs_readahead(struct readahead_control *ractl)
 
 	netfs_rreq_expand(rreq, ractl);
 
-	rreq->submitted = rreq->start;
-	if (rolling_buffer_init(&rreq->buffer, rreq->debug_id, ITER_DEST, rreq->gfp) < 0)
+	/* Load the folios to be read into a bvecq chain.  Note that this
+	 * acquires a ref on each folio that we will need to release later -
+	 * but we don't want to do that until after we've started the I/O.
+	 */
+	added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl, rreq->debug_id);
+	if (added < 0) {
+		ret = added;
 		goto cleanup_free;
-	netfs_read_to_pagecache(rreq, ractl);
+	}
+	__set_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags);
+
+	rreq->submitted = rreq->start + added;
+	rreq->cleaned_to = rreq->start;
+	rreq->front_folio_order = folio_order(rreq->buffer.tail->vec.folios[0]);
 
+	netfs_read_to_pagecache(rreq);
+	netfs_maybe_bulk_drop_ra_refs(rreq);
 	return netfs_put_request(rreq, netfs_rreq_trace_put_return);
 
 cleanup_free:
@@ -457,7 +478,7 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 	iov_iter_bvec(&rreq->buffer.iter, ITER_DEST, bvec, i, rreq->len);
 	rreq->submitted = rreq->start + flen;
 
-	netfs_read_to_pagecache(rreq, NULL);
+	netfs_read_to_pagecache(rreq);
 
 	ret = netfs_wait_for_read(rreq);
 	if (ret >= 0) {
@@ -532,7 +553,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
 	if (ret < 0)
 		goto discard;
 
-	netfs_read_to_pagecache(rreq, NULL);
+	netfs_read_to_pagecache(rreq);
 	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret < 0 ? ret : 0;
@@ -689,7 +710,7 @@ int netfs_write_begin(struct netfs_inode *ctx,
 	if (ret < 0)
 		goto error_put;
 
-	netfs_read_to_pagecache(rreq, NULL);
+	netfs_read_to_pagecache(rreq);
 	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	if (ret < 0)
@@ -754,7 +775,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 	if (ret < 0)
 		goto error_put;
 
-	netfs_read_to_pagecache(rreq, NULL);
+	netfs_read_to_pagecache(rreq);
 	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret < 0 ? ret : 0;
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 420ee7b26580..bd8b2d633f96 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -79,6 +79,7 @@ ssize_t netfs_wait_for_read(struct netfs_io_request *rreq);
 ssize_t netfs_wait_for_write(struct netfs_io_request *rreq);
 void netfs_wait_for_paused_read(struct netfs_io_request *rreq);
 void netfs_wait_for_paused_write(struct netfs_io_request *rreq);
+void netfs_wait_for_put_ra_refs(struct netfs_io_request *rreq);
 
 /*
  * objects.c
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index 5d554512ed23..f5c1c463f4ff 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -563,3 +563,22 @@ void netfs_wait_for_paused_write(struct netfs_io_request *rreq)
 {
 	return netfs_wait_for_pause(rreq, netfs_write_collection);
 }
+
+/*
+ * Wait for the readahead-acquired refs to be put.
+ */
+void netfs_wait_for_put_ra_refs(struct netfs_io_request *rreq)
+{
+	DEFINE_WAIT(myself);
+
+	for (;;) {
+		trace_netfs_rreq(rreq, netfs_rreq_trace_wait_put_ra_refs);
+		prepare_to_wait(&rreq->waitq, &myself, TASK_UNINTERRUPTIBLE);
+		if (!test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+			break;
+		schedule();
+	}
+
+	trace_netfs_rreq(rreq, netfs_rreq_trace_waited_put_ra_refs);
+	finish_wait(&rreq->waitq, &myself);
+}
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 23660a590124..edf7cea7e2f9 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -118,6 +118,13 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 		slot = 0;
 	}
 
+	/* We have to wait for readahead refs to have been released before we
+	 * can unlock any folios as the ref-dropper walks i_pages and the only
+	 * thing preventing these folios from being removed is the folio lock.
+	 */
+	if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+		netfs_wait_for_put_ra_refs(rreq);
+
 	for (;;) {
 		struct folio *folio;
 		unsigned long long fpos, fend;
diff --git a/fs/netfs/rolling_buffer.c b/fs/netfs/rolling_buffer.c
index 8c0026836f9c..4e8584725b6a 100644
--- a/fs/netfs/rolling_buffer.c
+++ b/fs/netfs/rolling_buffer.c
@@ -153,6 +153,81 @@ ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
 	return size;
 }
 
+/*
+ * Decant the entire list of folios to read into a rolling buffer.
+ */
+ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
+					 struct readahead_control *ractl,
+					 unsigned int rreq_id)
+{
+	XA_STATE(xas, &ractl->mapping->i_pages, ractl->_index);
+	struct folio_queue *fq;
+	struct folio *folio;
+	ssize_t loaded = 0;
+	int nr, slot = 0, npages = 0;
+
+	/* First allocate all the folioqs we're going to need to avoid having
+	 * to deal with ENOMEM later.
+	 */
+	nr = ractl->_nr_folios;
+	do {
+		fq = netfs_folioq_alloc(rreq_id, GFP_KERNEL,
+					netfs_trace_folioq_make_space);
+		if (!fq) {
+			rolling_buffer_clear(roll);
+			return -ENOMEM;
+		}
+		fq->prev = roll->head;
+		if (!roll->tail)
+			roll->tail = fq;
+		else
+			roll->head->next = fq;
+		roll->head = fq;
+
+		nr -= folioq_nr_slots(fq);
+	} while (nr > 0);
+
+	rcu_read_lock();
+
+	fq = roll->tail;
+	xas_for_each(&xas, folio, ractl->_index + ractl->_nr_pages - 1) {
+		unsigned int order;
+
+		if (xas_retry(&xas, folio))
+			continue;
+		VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
+
+		order = folio_order(folio);
+		fq->orders[slot] = order;
+		fq->vec.folios[slot] = folio;
+		loaded += PAGE_SIZE << order;
+		npages += 1 << order;
+		trace_netfs_folio(folio, netfs_folio_trace_read);
+
+		slot++;
+		if (slot >= folioq_nr_slots(fq)) {
+			fq->vec.nr = slot;
+			fq = fq->next;
+			if (!fq) {
+				WARN_ON_ONCE(npages < readahead_count(ractl));
+				break;
+			}
+			slot = 0;
+		}
+	}
+
+	rcu_read_unlock();
+
+	if (fq)
+		fq->vec.nr = slot;
+
+	WRITE_ONCE(roll->iter.count, loaded);
+	iov_iter_folio_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
+	ractl->_index    += npages;
+	ractl->_nr_pages -= npages;
+	return loaded;
+}
+
 /*
  * Append a folio to the rolling buffer.
  */
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index d0b62d53eea9..e60539f039f1 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -279,6 +279,7 @@ struct netfs_io_request {
 #define NETFS_RREQ_FOLIO_COPY_TO_CACHE	10	/* Copy current folio to cache from read */
 #define NETFS_RREQ_UPLOAD_TO_SERVER	11	/* Need to write to the server */
 #define NETFS_RREQ_USE_IO_ITER		12	/* Use ->io_iter rather than ->i_pages */
+#define NETFS_RREQ_NEED_PUT_RA_REFS	17	/* Need to put the folio refs RA gave us */
 #define NETFS_RREQ_USE_PGPRIV2		31	/* [DEPRECATED] Use PG_private_2 to mark
 						 * write to cache on read */
 	const struct netfs_request_ops *netfs_ops;
diff --git a/include/linux/rolling_buffer.h b/include/linux/rolling_buffer.h
index 9e5dad29669c..1e37750e4ff8 100644
--- a/include/linux/rolling_buffer.h
+++ b/include/linux/rolling_buffer.h
@@ -48,6 +48,9 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp);
 ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
 				    struct readahead_control *ractl,
 				    struct folio_batch *put_batch);
+ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
+					 struct readahead_control *ractl,
+					 unsigned int rreq_id);
 ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
 			      unsigned int flags, gfp_t gfp);
 struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll);
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 082cb03c6131..9bda9302be90 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -59,6 +59,7 @@
 	EM(netfs_rreq_trace_free,		"FREE   ")	\
 	EM(netfs_rreq_trace_intr,		"INTR   ")	\
 	EM(netfs_rreq_trace_ki_complete,	"KI-CMPL")	\
+	EM(netfs_rreq_trace_ra_put_ref,		"RA-PUT ")	\
 	EM(netfs_rreq_trace_recollect,		"RECLLCT")	\
 	EM(netfs_rreq_trace_redirty,		"REDIRTY")	\
 	EM(netfs_rreq_trace_resubmit,		"RESUBMT")	\
@@ -70,9 +71,11 @@
 	EM(netfs_rreq_trace_unpause,		"UNPAUSE")	\
 	EM(netfs_rreq_trace_wait_ip,		"WAIT-IP")	\
 	EM(netfs_rreq_trace_wait_pause,		"--PAUSED--")	\
+	EM(netfs_rreq_trace_wait_put_ra_refs,	"WAIT-P-RA")	\
 	EM(netfs_rreq_trace_wait_quiesce,	"WAIT-QUIESCE")	\
 	EM(netfs_rreq_trace_waited_ip,		"DONE-IP")	\
 	EM(netfs_rreq_trace_waited_pause,	"--UNPAUSED--")	\
+	EM(netfs_rreq_trace_waited_put_ra_refs,	"DONE-P-RA")	\
 	EM(netfs_rreq_trace_waited_quiesce,	"DONE-QUIESCE")	\
 	EM(netfs_rreq_trace_wake_ip,		"WAKE-IP")	\
 	EM(netfs_rreq_trace_wake_queue,		"WAKE-Q ")	\



From dhowells at redhat.com  Tue Aug  4 03:01:59 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:01:59 +0100
Subject: [PATCH v8 04/25] iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-5-dhowells@redhat.com>

Make iov_iter_get_pages*() wrap iov_iter_extract_pages() for kernel
iterator types (e.g. ITER_BVEC, ITER_FOLIOQ, ITER_XARRAY).  The pages
obtained have their refcounts incremented afterwards if they're not slab
pages.  ITER_KVEC is left returning -EFAULT.

Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Jens Axboe <axboe at kernel.dk>
cc: linux-block at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 lib/iov_iter.c | 164 ++++++-------------------------------------------
 1 file changed, 19 insertions(+), 145 deletions(-)

diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index c2484551a4e8..31afc9687eb6 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -910,118 +910,34 @@ static int want_pages_array(struct page ***res, size_t size,
 	return count;
 }
 
-static ssize_t iter_folioq_get_pages(struct iov_iter *iter,
+/*
+ * Wrap iov_iter_extract_pages() and then pin the non-slab pages we got back.
+ * This only works for non-user iterator types as get_pages uses get_user_pages
+ * not pin_user_pages.
+ */
+static ssize_t iter_get_kernel_pages(struct iov_iter *iter,
 				     struct page ***ppages, size_t maxsize,
 				     unsigned maxpages, size_t *_start_offset)
 {
-	const struct folio_queue *folioq = iter->folioq;
 	struct page **pages;
-	unsigned int slot = iter->folioq_slot;
-	size_t extracted = 0, count = iter->count, iov_offset = iter->iov_offset;
+	ssize_t ret, done;
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		slot = 0;
-		if (WARN_ON(iov_offset != 0))
-			return -EIO;
-	}
+	ret = iov_iter_extract_pages(iter, ppages, maxsize, maxpages,
+				     0, _start_offset);
+	if (ret <= 0)
+		return ret;
 
-	maxpages = want_pages_array(ppages, maxsize, iov_offset & ~PAGE_MASK, maxpages);
-	if (!maxpages)
-		return -ENOMEM;
-	*_start_offset = iov_offset & ~PAGE_MASK;
 	pages = *ppages;
+	for (done = ret + *_start_offset; done > 0; done -= PAGE_SIZE) {
+		struct folio *folio = page_folio(*pages);
 
-	for (;;) {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t offset = iov_offset, fsize = folioq_folio_size(folioq, slot);
-		size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
-		if (offset < fsize) {
-			part = umin(part, umin(maxsize - extracted, fsize - offset));
-			count -= part;
-			iov_offset += part;
-			extracted += part;
-
-			*pages = folio_page(folio, offset / PAGE_SIZE);
-			get_page(*pages);
-			pages++;
-			maxpages--;
-		}
-
-		if (maxpages == 0 || extracted >= maxsize)
-			break;
-
-		if (iov_offset >= fsize) {
-			iov_offset = 0;
-			slot++;
-			if (slot == folioq_nr_slots(folioq) && folioq->next) {
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	}
-
-	iter->count = count;
-	iter->iov_offset = iov_offset;
-	iter->folioq = folioq;
-	iter->folioq_slot = slot;
-	return extracted;
-}
-
-static ssize_t iter_xarray_populate_pages(struct page **pages, struct xarray *xa,
-					  pgoff_t index, unsigned int nr_pages)
-{
-	XA_STATE(xas, xa, index);
-	struct folio *folio;
-	unsigned int ret = 0;
-
-	rcu_read_lock();
-	for (folio = xas_load(&xas); folio; folio = xas_next(&xas)) {
-		if (xas_retry(&xas, folio))
-			continue;
-
-		/* Has the folio moved or been split? */
-		if (unlikely(folio != xas_reload(&xas))) {
-			xas_reset(&xas);
-			continue;
-		}
-
-		pages[ret] = folio_file_page(folio, xas.xa_index);
-		folio_get(folio);
-		if (++ret == nr_pages)
-			break;
+		if (!folio_test_slab(folio))
+			folio_get(folio);
+		pages++;
 	}
-	rcu_read_unlock();
 	return ret;
 }
 
-static ssize_t iter_xarray_get_pages(struct iov_iter *i,
-				     struct page ***pages, size_t maxsize,
-				     unsigned maxpages, size_t *_start_offset)
-{
-	unsigned nr, offset, count;
-	pgoff_t index;
-	loff_t pos;
-
-	pos = i->xarray_start + i->iov_offset;
-	index = pos >> PAGE_SHIFT;
-	offset = pos & ~PAGE_MASK;
-	*_start_offset = offset;
-
-	count = want_pages_array(pages, maxsize, offset, maxpages);
-	if (!count)
-		return -ENOMEM;
-	nr = iter_xarray_populate_pages(*pages, i->xarray, index, count);
-	if (nr == 0)
-		return 0;
-
-	maxsize = min_t(size_t, nr * PAGE_SIZE - offset, maxsize);
-	i->iov_offset += maxsize;
-	i->count -= maxsize;
-	return maxsize;
-}
-
 /* must be done on non-empty ITER_UBUF or ITER_IOVEC one */
 static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
 {
@@ -1044,22 +960,6 @@ static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
 	BUG(); // if it had been empty, we wouldn't get called
 }
 
-/* must be done on non-empty ITER_BVEC one */
-static struct page *first_bvec_segment(const struct iov_iter *i,
-				       size_t *size, size_t *start)
-{
-	struct page *page;
-	size_t skip = i->iov_offset, len;
-
-	len = i->bvec->bv_len - skip;
-	if (*size > len)
-		*size = len;
-	skip += i->bvec->bv_offset;
-	page = i->bvec->bv_page + skip / PAGE_SIZE;
-	*start = skip % PAGE_SIZE;
-	return page;
-}
-
 static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
 		   struct page ***pages, size_t maxsize,
 		   unsigned int maxpages, size_t *start)
@@ -1095,36 +995,10 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
 		iov_iter_advance(i, maxsize);
 		return maxsize;
 	}
-	if (iov_iter_is_bvec(i)) {
-		struct page **p;
-		struct page *page;
 
-		page = first_bvec_segment(i, &maxsize, start);
-		n = want_pages_array(pages, maxsize, *start, maxpages);
-		if (!n)
-			return -ENOMEM;
-		p = *pages;
-		for (int k = 0; k < n; k++) {
-			struct folio *folio = page_folio(page + k);
-			p[k] = page + k;
-			if (!folio_test_slab(folio))
-				folio_get(folio);
-		}
-		maxsize = min_t(size_t, maxsize, n * PAGE_SIZE - *start);
-		i->count -= maxsize;
-		i->iov_offset += maxsize;
-		if (i->iov_offset == i->bvec->bv_len) {
-			i->iov_offset = 0;
-			i->bvec++;
-			i->nr_segs--;
-		}
-		return maxsize;
-	}
-	if (iov_iter_is_folioq(i))
-		return iter_folioq_get_pages(i, pages, maxsize, maxpages, start);
-	if (iov_iter_is_xarray(i))
-		return iter_xarray_get_pages(i, pages, maxsize, maxpages, start);
-	return -EFAULT;
+	if (iov_iter_is_kvec(i))
+		return -EFAULT;
+	return iter_get_kernel_pages(i, pages, maxsize, maxpages, start);
 }
 
 ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,



From dhowells at redhat.com  Tue Aug  4 03:01:58 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:01:58 +0100
Subject: [PATCH v8 03/25] Add a function to kmap one page of a multipage bio_vec
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-4-dhowells@redhat.com>

Add a function to kmap one page of a multipage bio_vec by offset (which is
added to the offset in the bio_vec internally).  The caller is responsible
for calculating how much of the page is then available.

Signed-off-by: David Howells <dhowells at redhat.com>
Acked-by: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Jens Axboe <axboe at kernel.dk>
cc: linux-block at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 include/linux/bvec.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 92837e2743f1..53cf36e73967 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -343,4 +343,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
 	return page_to_phys(bvec->bv_page) + bvec->bv_offset;
 }
 
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space.  The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
+{
+	offset += bvec->bv_offset;
+
+	return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+		(offset & ~PAGE_MASK);
+}
+
 #endif /* __LINUX_BVEC_H */



From dhowells at redhat.com  Tue Aug  4 03:02:00 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:00 +0100
Subject: [PATCH v8 05/25] iov_iter: Add a segmented queue of bio_vec[]
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-6-dhowells@redhat.com>

Add the concept of a segmented queue of bio_vec[] arrays.  This allows an
indefinite quantity of elements to be handled and allows things like
network filesystems and crypto drivers to glue bits on the ends without
having to reallocate the array.

The bvecq struct that defines each segment also carries capacity/usage
information along with flags indicating whether the constituent memory
regions need freeing or unpinning and the file position of the first
element in a segment.  The bvecq structs are refcounted to allow a queue to
be extracted in batches and split between a number of subrequests.

The bvecq can have the bio_vec[] it manages allocated in with it, but this
is not required.  A flag is provided for if this is the case as comparing
->bv to ->__bv is not sufficient to detect this case.

Add an iterator type ITER_BVECQ for it.  This is intended to replace
ITER_FOLIOQ (and ITER_XARRAY).

Note that the prev pointer is only really needed for iov_iter_revert() and
could be dispensed with if struct iov_iter contained the head information
as well as the current point.

Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Jens Axboe <axboe at kernel.dk>
cc: linux-block at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 include/linux/bvecq.h      |  56 +++++++
 include/linux/iov_iter.h   |  74 +++++++-
 include/linux/uio.h        |  11 ++
 lib/iov_iter.c             | 334 ++++++++++++++++++++++++++++++++++++-
 lib/scatterlist.c          |  67 +++++++-
 lib/tests/kunit_iov_iter.c | 260 +++++++++++++++++++++++++++++
 6 files changed, 796 insertions(+), 6 deletions(-)
 create mode 100644 include/linux/bvecq.h

diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
new file mode 100644
index 000000000000..15f16f905877
--- /dev/null
+++ b/include/linux/bvecq.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Implementation of a segmented queue of bio_vec[].
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells at redhat.com)
+ */
+
+#ifndef _LINUX_BVECQ_H
+#define _LINUX_BVECQ_H
+
+#include <linux/bvec.h>
+
+/*
+ * The type of memory retention used by the elements in bvecq->bv[] and how to
+ * clean it up.
+ */
+enum bvecq_mem {
+	BVECQ_MEM_EXTERNAL,	/* Externally retained memory - no freeing */
+	BVECQ_MEM_PAGECACHE,	/* Ref'd pagecache pages - must put */
+	BVECQ_MEM_GUP,		/* Pinned memory from get_user_pages() - unpin */
+	BVECQ_MEM_ALLOCED,	/* Memory alloc'd by bvecq - can be freed/pooled */
+} __mode(byte);
+
+/*
+ * Segmented bio_vec queue.
+ *
+ * These can be linked together to form messages of indefinite length and
+ * iterated over with an ITER_BVECQ iterator.  The list is non-circular; next
+ * and prev are NULL at the ends.
+ *
+ * The bv pointer points to the bio_vec array; this may be __bv if allocated
+ * together.  The caller is responsible for determining whether or not this is
+ * the case as the array pointed to by bv may be follow on directly from the
+ * bvecq by accident of allocation (ie. ->bv == ->__bv is *not* sufficient to
+ * determine this).
+ *
+ * The file position and discontiguity flag allow non-contiguous data sets to
+ * be chained together, but still teased apart without the need to convert the
+ * info in the bio_vec back into a folio pointer.
+ */
+struct bvecq {
+	struct bvecq	*next;		/* Next bvec in the list or NULL */
+	struct bvecq	*prev;		/* Prev bvec in the list or NULL */
+	unsigned long long fpos;	/* File position */
+	refcount_t	ref;
+	u32		priv;		/* Private data */
+	u16		nr_slots;	/* Number of elements in bv[] used */
+	u16		max_slots;	/* Number of elements allocated in bv[] */
+	enum bvecq_mem	mem_type:3;	/* What sort of memory and how to free it */
+	bool		inline_bv:1;	/* T if __bv[] is being used */
+	bool		discontig:1;	/* T if not contiguous with previous bvecq */
+	struct bio_vec	*bv;		/* Pointer to array of page fragments */
+	struct bio_vec	__bv[];		/* Default array (if ->inline_bv) */
+};
+
+#endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index f9a17fbbd398..04b8a6d943fa 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -10,6 +10,7 @@
 
 #include <linux/uio.h>
 #include <linux/bvec.h>
+#include <linux/bvecq.h>
 #include <linux/folio_queue.h>
 
 typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
@@ -141,6 +142,71 @@ size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2,
 	return progress;
 }
 
+/*
+ * Handle ITER_BVECQ.
+ */
+static __always_inline
+size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
+		     iov_step_f step)
+{
+	const struct bvecq *bq = iter->bvecq;
+	unsigned int slot = iter->bvecq_slot;
+	size_t progress = 0, skip = iter->iov_offset;
+
+	do {
+		const struct bio_vec *bvec;
+		struct page *page;
+		size_t poff, plen;
+		void *base;
+
+		if (slot >= bq->nr_slots) {
+			if (!bq->next)
+				break;
+			bq = bq->next;
+			slot = 0;
+			continue;
+		}
+
+		bvec = &bq->bv[slot];
+		/*
+		 * The caller must ensure that a slot with bv_len>0 has a valid
+		 * bv_page.
+		 */
+		page = bvec->bv_page + (bvec->bv_offset + skip) / PAGE_SIZE;
+		poff = (bvec->bv_offset + skip) % PAGE_SIZE;
+		plen = umin(bvec->bv_len - skip, len);
+
+		while (plen > 0) {
+			size_t part, remain, consumed;
+
+			part = umin(plen, PAGE_SIZE - poff);
+			base = kmap_local_page(page) + poff;
+			remain = step(base, progress, part, priv, priv2);
+			kunmap_local(base);
+
+			consumed = part - remain;
+			progress += consumed;
+			skip += consumed;
+			len -= consumed;
+			if (!len || remain)
+				goto stop;
+			page++;
+			poff = 0;
+			plen -= consumed;
+		}
+
+		skip = 0;
+		slot++;
+	} while (len);
+
+stop:
+	iter->bvecq_slot = slot;
+	iter->bvecq = bq;
+	iter->iov_offset = skip;
+	iter->count -= progress;
+	return progress;
+}
+
 /*
  * Handle ITER_FOLIOQ.
  */
@@ -306,6 +372,8 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_bvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_kvec(iter))
 		return iterate_kvec(iter, len, priv, priv2, step);
+	if (iov_iter_is_bvecq(iter))
+		return iterate_bvecq(iter, len, priv, priv2, step);
 	if (iov_iter_is_folioq(iter))
 		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
@@ -342,8 +410,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
  * buffer is presented in segments, which for kernel iteration are broken up by
  * physical pages and mapped, with the mapped address being presented.
  *
- * [!] Note This will only handle BVEC, KVEC, FOLIOQ, XARRAY and DISCARD-type
- * iterators; it will not handle UBUF or IOVEC-type iterators.
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and
+ * DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators.
  *
  * A step functions, @step, must be provided, one for handling mapped kernel
  * addresses and the other is given user addresses which have the potential to
@@ -370,6 +438,8 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_bvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_kvec(iter))
 		return iterate_kvec(iter, len, priv, priv2, step);
+	if (iov_iter_is_bvecq(iter))
+		return iterate_bvecq(iter, len, priv, priv2, step);
 	if (iov_iter_is_folioq(iter))
 		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
diff --git a/include/linux/uio.h b/include/linux/uio.h
index a9bc5b3067e3..f7cfa6ea8213 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -26,6 +26,7 @@ enum iter_type {
 	ITER_IOVEC,
 	ITER_BVEC,
 	ITER_KVEC,
+	ITER_BVECQ,
 	ITER_FOLIOQ,
 	ITER_XARRAY,
 	ITER_DISCARD,
@@ -68,6 +69,7 @@ struct iov_iter {
 				const struct iovec *__iov;
 				const struct kvec *kvec;
 				const struct bio_vec *bvec;
+				const struct bvecq *bvecq;
 				const struct folio_queue *folioq;
 				struct xarray *xarray;
 				void __user *ubuf;
@@ -77,6 +79,7 @@ struct iov_iter {
 	};
 	union {
 		unsigned long nr_segs;
+		u16 bvecq_slot;
 		u8 folioq_slot;
 		loff_t xarray_start;
 	};
@@ -145,6 +148,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
 	return iov_iter_type(i) == ITER_DISCARD;
 }
 
+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
+{
+	return iov_iter_type(i) == ITER_BVECQ;
+}
+
 static inline bool iov_iter_is_folioq(const struct iov_iter *i)
 {
 	return iov_iter_type(i) == ITER_FOLIOQ;
@@ -295,6 +303,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec
 void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,
 			unsigned long nr_segs, size_t count);
 void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+			 const struct bvecq *bvecq,
+			 unsigned int first_slot, unsigned int offset, size_t count);
 void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
 			  const struct folio_queue *folioq,
 			  unsigned int first_slot, unsigned int offset, size_t count);
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 31afc9687eb6..ff12ddd917cd 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -538,6 +538,40 @@ static void iov_iter_iovec_advance(struct iov_iter *i, size_t size)
 	i->__iov = iov;
 }
 
+static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
+{
+	const struct bvecq *bq = i->bvecq;
+	unsigned int slot = i->bvecq_slot;
+
+	if (!i->count)
+		return;
+	i->count -= by;
+
+	by += i->iov_offset; /* From beginning of current segment. */
+	do {
+		size_t len;
+
+		if (slot >= bq->nr_slots) {
+			if (!bq->next)
+				break;
+			bq = bq->next;
+			slot = 0;
+			continue;
+		}
+
+		len = bq->bv[slot].bv_len;
+
+		if (likely(by < len))
+			break;
+		by -= len;
+		slot++;
+	} while (by);
+
+	i->iov_offset = by;
+	i->bvecq_slot = slot;
+	i->bvecq = bq;
+}
+
 static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
 {
 	const struct folio_queue *folioq = i->folioq;
@@ -583,6 +617,8 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
 		iov_iter_iovec_advance(i, size);
 	} else if (iov_iter_is_bvec(i)) {
 		iov_iter_bvec_advance(i, size);
+	} else if (iov_iter_is_bvecq(i)) {
+		iov_iter_bvecq_advance(i, size);
 	} else if (iov_iter_is_folioq(i)) {
 		iov_iter_folioq_advance(i, size);
 	} else if (iov_iter_is_discard(i)) {
@@ -591,6 +627,33 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
 }
 EXPORT_SYMBOL(iov_iter_advance);
 
+static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
+{
+	const struct bvecq *bq = i->bvecq;
+	unsigned int slot = i->bvecq_slot;
+
+	for (;;) {
+		size_t len;
+
+		if (slot == 0) {
+			bq = bq->prev;
+			slot = bq->nr_slots;
+			continue;
+		}
+		slot--;
+
+		len = bq->bv[slot].bv_len;
+		if (unroll <= len) {
+			i->iov_offset = len - unroll;
+			break;
+		}
+		unroll -= len;
+	}
+
+	i->bvecq_slot = slot;
+	i->bvecq = bq;
+}
+
 static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
 {
 	const struct folio_queue *folioq = i->folioq;
@@ -648,6 +711,9 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
 			}
 			unroll -= n;
 		}
+	} else if (iov_iter_is_bvecq(i)) {
+		i->iov_offset = 0;
+		iov_iter_bvecq_revert(i, unroll);
 	} else if (iov_iter_is_folioq(i)) {
 		i->iov_offset = 0;
 		iov_iter_folioq_revert(i, unroll);
@@ -678,9 +744,30 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
 		if (iov_iter_is_bvec(i))
 			return min(i->count, i->bvec->bv_len - i->iov_offset);
 	}
+	if (!i->count)
+		return 0;
+	if (unlikely(iov_iter_is_bvecq(i))) {
+		const struct bvecq *bq = i->bvecq;
+		unsigned int slot = i->bvecq_slot;
+		size_t offset = i->iov_offset;
+
+		for (;;) {
+			while (slot >= bq->nr_slots) {
+				bq = bq->next;
+				if (!bq)
+					return 0;
+				slot = 0;
+				offset = 0;
+			}
+			if (bq->bv[slot].bv_len > offset)
+				break;
+			slot++;
+			offset = 0;
+		}
+		return umin(i->count, bq->bv[slot].bv_len - offset);
+	}
 	if (unlikely(iov_iter_is_folioq(i)))
-		return !i->count ? 0 :
-			umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
+		return umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
 	return i->count;
 }
 EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -717,6 +804,35 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction,
 }
 EXPORT_SYMBOL(iov_iter_bvec);
 
+/**
+ * iov_iter_bvec_queue - Initialise an I/O iterator to use a segmented bvec queue
+ * @i: The iterator to initialise.
+ * @direction: The direction of the transfer.
+ * @bvecq: The starting point in the bvec queue.
+ * @first_slot: The first slot in the bvec queue to use
+ * @offset: The offset into the bvec in the first slot to start at
+ * @count: The size of the I/O buffer in bytes.
+ *
+ * Set up an I/O iterator to either draw data out of the buffers attached to an
+ * inode or to inject data into those buffers.  The pages *must* be prevented
+ * from evaporation, either by the caller.
+ */
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+			 const struct bvecq *bvecq, unsigned int first_slot,
+			 unsigned int offset, size_t count)
+{
+	WARN_ON(direction & ~(READ | WRITE));
+	*i = (struct iov_iter) {
+		.iter_type	= ITER_BVECQ,
+		.data_source	= direction,
+		.bvecq		= bvecq,
+		.bvecq_slot	= first_slot,
+		.count		= count,
+		.iov_offset	= offset,
+	};
+}
+EXPORT_SYMBOL(iov_iter_bvec_queue);
+
 /**
  * iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
  * @i: The iterator to initialise.
@@ -839,6 +955,39 @@ static unsigned long iov_iter_alignment_bvec(const struct iov_iter *i)
 	return res;
 }
 
+static unsigned long iov_iter_alignment_bvecq(const struct iov_iter *iter)
+{
+	const struct bvecq *bq;
+	unsigned long res = 0;
+	unsigned int slot = iter->bvecq_slot;
+	size_t skip = iter->iov_offset;
+	size_t size = iter->count;
+
+	if (!size)
+		return res;
+
+	for (bq = iter->bvecq; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			const struct bio_vec *bvec = &bq->bv[slot];
+			size_t part = umin(bvec->bv_len - skip, size);
+
+			if (part) {
+				res |= bvec->bv_offset + skip;
+				res |= part;
+			}
+
+			size -= part;
+			if (size == 0)
+				return res;
+			skip = 0;
+		}
+
+		slot = 0;
+	}
+
+	return res;
+}
+
 unsigned long iov_iter_alignment(const struct iov_iter *i)
 {
 	if (likely(iter_is_ubuf(i))) {
@@ -854,6 +1003,8 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
 
 	if (iov_iter_is_bvec(i))
 		return iov_iter_alignment_bvec(i);
+	if (iov_iter_is_bvecq(i))
+		return iov_iter_alignment_bvecq(i);
 
 	/* With both xarray and folioq types, we're dealing with whole folios. */
 	if (iov_iter_is_folioq(i))
@@ -1066,6 +1217,38 @@ static int bvec_npages(const struct iov_iter *i, int maxpages)
 	return npages;
 }
 
+static size_t iov_npages_bvecq(const struct iov_iter *iter, size_t maxpages)
+{
+	const struct bvecq *bq;
+	unsigned int slot = iter->bvecq_slot;
+	size_t npages = 0;
+	size_t skip = iter->iov_offset;
+	size_t size = iter->count;
+
+	for (bq = iter->bvecq; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			const struct bio_vec *bvec = &bq->bv[slot];
+			size_t offs = (bvec->bv_offset + skip) % PAGE_SIZE;
+			size_t part = umin(bvec->bv_len - skip, size);
+
+			if (part) {
+				npages += DIV_ROUND_UP(offs + part, PAGE_SIZE);
+				if (npages >= maxpages)
+					goto out;
+			}
+
+			size -= part;
+			if (!size)
+				goto out;
+			skip = 0;
+		}
+
+		slot = 0;
+	}
+out:
+	return umin(npages, maxpages);
+}
+
 int iov_iter_npages(const struct iov_iter *i, int maxpages)
 {
 	if (unlikely(!i->count))
@@ -1080,6 +1263,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
 		return iov_npages(i, maxpages);
 	if (iov_iter_is_bvec(i))
 		return bvec_npages(i, maxpages);
+	if (iov_iter_is_bvecq(i))
+		return iov_npages_bvecq(i, maxpages);
 	if (iov_iter_is_folioq(i)) {
 		unsigned offset = i->iov_offset % PAGE_SIZE;
 		int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1366,6 +1551,147 @@ void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state)
 	i->nr_segs = state->nr_segs;
 }
 
+/*
+ * Count the number of virtually contiguous pages coming up next in an
+ * ITER_BVECQ iterator, up to the specified maxima.
+ */
+static unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,
+					   size_t maxsize,
+					   unsigned int maxpages)
+{
+	const struct bvecq *bvecq = iter->bvecq;
+	unsigned int slot = iter->bvecq_slot;
+	ssize_t remain = umin(maxsize, iter->count);
+	size_t count = 0, offset = iter->iov_offset;
+
+	do {
+		const struct bio_vec *bv;
+		size_t boff, blen;
+
+		if (slot >= bvecq->nr_slots) {
+			if (!bvecq->next) {
+				WARN_ON_ONCE(remain > 0);
+				break;
+			}
+			bvecq = bvecq->next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		bv = &bvecq->bv[slot++];
+		boff = bv->bv_offset;
+		blen = bv->bv_len;
+
+		if (unlikely(!bv->bv_page)) {
+			if (blen && count > 0)
+				break;
+			continue;
+		}
+		if (!PAGE_ALIGNED(boff) && count > 0)
+			break;
+
+		boff += offset;
+		blen -= offset;
+		offset = 0;
+		if (!blen)
+			continue;
+
+		blen = umin(blen, remain);
+		remain -= blen;
+		blen += offset_in_page(boff);
+		count += DIV_ROUND_UP(blen, PAGE_SIZE);
+
+		if (!PAGE_ALIGNED(blen))
+			break;
+	} while (remain > 0 && count < maxpages);
+
+	return umin(count, maxpages);
+}
+
+/*
+ * Extract a list of virtually contiguous pages from an ITER_BVECQ iterator.
+ * This does not get references on the pages, nor does it get a pin on them.
+ */
+static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
+					    struct page ***pages, size_t maxsize,
+					    unsigned int maxpages,
+					    iov_iter_extraction_t extraction_flags,
+					    size_t *offset0)
+{
+	const struct bvecq *bvecq;
+	struct page **p;
+	unsigned int slot, nr = 0;
+	size_t extracted = 0, offset;
+
+	/* Count the next run of virtually contiguous pages. */
+	maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+
+	if (!*pages) {
+		*pages = kvmalloc_array(maxpages, sizeof(struct page *), GFP_KERNEL);
+		if (!*pages)
+			return -ENOMEM;
+	}
+
+	p = *pages;
+
+	/* Now transcribe the page pointers. */
+	extracted = 0;
+	bvecq = iter->bvecq;
+	offset = iter->iov_offset;
+	slot = iter->bvecq_slot;
+
+	do {
+		const struct bio_vec *bv;
+		size_t boff, blen;
+
+		if (slot >= bvecq->nr_slots) {
+			if (!bvecq->next) {
+				WARN_ON_ONCE(extracted < iter->count);
+				break;
+			}
+			bvecq = bvecq->next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		bv = &bvecq->bv[slot];
+		boff = bv->bv_offset;
+		blen = bv->bv_len;
+
+		if (!bv->bv_page)
+			blen = 0;
+
+		if (offset < blen) {
+			size_t part = umin(maxsize - extracted, blen - offset);
+			size_t poff = (boff + offset) % PAGE_SIZE;
+			size_t pix = (boff + offset) / PAGE_SIZE;
+
+			if (poff + part > PAGE_SIZE)
+				part = PAGE_SIZE - poff;
+
+			if (!extracted)
+				*offset0 = poff;
+
+			p[nr++] = bv->bv_page + pix;
+			offset += part;
+			extracted += part;
+		}
+
+		if (offset >= blen) {
+			offset = 0;
+			slot++;
+		}
+	} while (nr < maxpages && extracted < maxsize);
+
+	iter->bvecq = bvecq;
+	iter->bvecq_slot = slot;
+	iter->iov_offset = offset;
+	iter->count -= extracted;
+	return extracted;
+}
+
 /*
  * Extract a list of contiguous pages from an ITER_FOLIOQ iterator.  This does
  * not get references on the pages, nor does it get a pin on them.
@@ -1726,6 +2052,10 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
 		return iov_iter_extract_bvec_pages(i, pages, maxsize,
 						   maxpages, extraction_flags,
 						   offset0);
+	if (iov_iter_is_bvecq(i))
+		return iov_iter_extract_bvecq_pages(i, pages, maxsize,
+						    maxpages, extraction_flags,
+						    offset0);
 	if (iov_iter_is_folioq(i))
 		return iov_iter_extract_folioq_pages(i, pages, maxsize,
 						     maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index 6ea40d2e6247..23e5a180103b 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -10,6 +10,7 @@
 #include <linux/highmem.h>
 #include <linux/kmemleak.h>
 #include <linux/bvec.h>
+#include <linux/bvecq.h>
 #include <linux/uio.h>
 #include <linux/folio_queue.h>
 
@@ -1267,6 +1268,65 @@ static ssize_t extract_kvec_to_sg(struct iov_iter *iter,
 	return ret;
 }
 
+/*
+ * Extract up to sg_max folios from an BVECQ-type iterator and add them to
+ * the scatterlist.  The pages are not pinned.
+ */
+static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
+				   ssize_t maxsize,
+				   struct sg_table *sgtable,
+				   unsigned int sg_max,
+				   iov_iter_extraction_t extraction_flags)
+{
+	const struct bvecq *bvecq = iter->bvecq;
+	struct scatterlist *sg = sgtable->sgl + sgtable->nents;
+	unsigned int slot = iter->bvecq_slot;
+	ssize_t ret = 0;
+	size_t offset = iter->iov_offset;
+
+	maxsize = umin(maxsize, iov_iter_count(iter));
+
+	while (sg_max > 0 && ret < maxsize) {
+		const struct bio_vec *bv;
+		size_t blen, part;
+
+		if (slot >= bvecq->nr_slots) {
+			if (!bvecq->next) {
+				WARN_ON_ONCE(ret < iter->count);
+				break;
+			}
+			bvecq = bvecq->next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		bv = &bvecq->bv[slot];
+		blen = bv->bv_len;
+
+		if (offset >= blen) {
+			offset = 0;
+			slot++;
+			continue;
+		}
+
+		part = umin(maxsize - ret, blen - offset);
+
+		sg_set_page(sg, bv->bv_page, part, bv->bv_offset + offset);
+		sgtable->nents++;
+		sg++;
+		sg_max--;
+		offset += part;
+		ret += part;
+	}
+
+	iter->bvecq = bvecq;
+	iter->bvecq_slot = slot;
+	iter->iov_offset = offset;
+	iter->count -= ret;
+	return ret;
+}
+
 /*
  * Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
  * the scatterlist.  The pages are not pinned.
@@ -1391,8 +1451,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
  * addition of @sg_max elements.
  *
  * The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, KVEC-, FOLIOQ- and XARRAY-type are extracted but aren't
- * pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, FOLIOQ- and XARRAY-type are extracted but
+ * aren't pinned; DISCARD-type is not supported.
  *
  * No end mark is placed on the scatterlist; that's left to the caller.
  *
@@ -1424,6 +1484,9 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
 	case ITER_KVEC:
 		return extract_kvec_to_sg(iter, maxsize, sgtable, sg_max,
 					  extraction_flags);
+	case ITER_BVECQ:
+		return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
+					   extraction_flags);
 	case ITER_FOLIOQ:
 		return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
 					    extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index d9690ba1db88..f6413e6b925a 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -12,6 +12,7 @@
 #include <linux/mm.h>
 #include <linux/uio.h>
 #include <linux/bvec.h>
+#include <linux/bvecq.h>
 #include <linux/folio_queue.h>
 #include <linux/scatterlist.h>
 #include <linux/minmax.h>
@@ -552,6 +553,183 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
 	KUNIT_SUCCEED(test);
 }
 
+static void iov_kunit_destroy_bvecq(void *data)
+{
+	struct bvecq *bq, *next;
+
+	for (bq = data; bq; bq = next) {
+		next = bq->next;
+		/* The pages are freed by vmap with VM_MAP_PUT_PAGES. */
+		kfree(bq);
+	}
+}
+
+static struct bvecq *iov_kunit_alloc_bvecq(struct kunit *test, unsigned int max_slots)
+{
+	struct bvecq *bq;
+
+	bq = kzalloc(struct_size(bq, __bv, max_slots), GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, bq);
+	bq->max_slots = max_slots;
+	bq->bv = bq->__bv;
+	bq->inline_bv = true;
+	return bq;
+}
+
+static struct bvecq *iov_kunit_create_bvecq(struct kunit *test, unsigned int max_slots)
+{
+	struct bvecq *bq;
+
+	bq = iov_kunit_alloc_bvecq(test, max_slots);
+	kunit_add_action_or_reset(test, iov_kunit_destroy_bvecq, bq);
+	return bq;
+}
+
+static void __init iov_kunit_load_bvecq(struct kunit *test,
+					struct iov_iter *iter, int dir,
+					struct bvecq *bq_head,
+					struct page **pages, size_t npages)
+{
+	struct bvecq *bq = bq_head;
+	size_t size = 0;
+
+	for (int i = 0; i < npages; i++) {
+		if (bq->nr_slots >= bq->max_slots) {
+			bq->next = iov_kunit_alloc_bvecq(test, 13);
+			bq->next->prev = bq;
+			bq = bq->next;
+		}
+		bvec_set_page(&bq->bv[bq->nr_slots], pages[i], PAGE_SIZE, 0);
+		bq->nr_slots++;
+		size += PAGE_SIZE;
+	}
+	iov_iter_bvec_queue(iter, dir, bq_head, 0, 0, size);
+}
+
+/*
+ * Test copying to a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_to_bvecq(struct kunit *test)
+{
+	const struct kvec_test_range *pr;
+	struct iov_iter iter;
+	struct bvecq *bq;
+	struct page **spages, **bpages;
+	u8 *scratch, *buffer;
+	size_t bufsize, npages, size, copied;
+	int i, patt;
+
+	bufsize = 0x100000;
+	npages = bufsize / PAGE_SIZE;
+
+	bq = iov_kunit_create_bvecq(test, 13);
+
+	scratch = iov_kunit_create_buffer(test, &spages, npages);
+	for (i = 0; i < bufsize; i++)
+		scratch[i] = pattern(i);
+
+	buffer = iov_kunit_create_buffer(test, &bpages, npages);
+	memset(buffer, 0, bufsize);
+
+	iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+	i = 0;
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+		size = pr->to - pr->from;
+		KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+		iov_iter_bvec_queue(&iter, READ, bq, 0, 0, pr->to);
+		iov_iter_advance(&iter, pr->from);
+		copied = copy_to_iter(scratch + i, size, &iter);
+
+		KUNIT_EXPECT_EQ(test, copied, size);
+		KUNIT_EXPECT_EQ(test, iter.count, 0);
+		i += size;
+		if (test->status == KUNIT_FAILURE)
+			goto stop;
+	}
+
+	/* Build the expected image in the scratch buffer. */
+	patt = 0;
+	memset(scratch, 0, bufsize);
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++)
+		for (i = pr->from; i < pr->to; i++)
+			scratch[i] = pattern(patt++);
+
+	/* Compare the images */
+	for (i = 0; i < bufsize; i++) {
+		KUNIT_EXPECT_EQ_MSG(test, buffer[i], scratch[i], "at i=%x", i);
+		if (buffer[i] != scratch[i])
+			return;
+	}
+
+stop:
+	KUNIT_SUCCEED(test);
+}
+
+/*
+ * Test copying from a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_from_bvecq(struct kunit *test)
+{
+	const struct kvec_test_range *pr;
+	struct iov_iter iter;
+	struct bvecq *bq;
+	struct page **spages, **bpages;
+	u8 *scratch, *buffer;
+	size_t bufsize, npages, size, copied;
+	int i, j;
+
+	bufsize = 0x100000;
+	npages = bufsize / PAGE_SIZE;
+
+	bq = iov_kunit_create_bvecq(test, 13);
+
+	buffer = iov_kunit_create_buffer(test, &bpages, npages);
+	for (i = 0; i < bufsize; i++)
+		buffer[i] = pattern(i);
+
+	scratch = iov_kunit_create_buffer(test, &spages, npages);
+	memset(scratch, 0, bufsize);
+
+	iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+	i = 0;
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+		size = pr->to - pr->from;
+		KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+		iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+		iov_iter_advance(&iter, pr->from);
+		copied = copy_from_iter(scratch + i, size, &iter);
+
+		KUNIT_EXPECT_EQ(test, copied, size);
+		KUNIT_EXPECT_EQ(test, iter.count, 0);
+		i += size;
+	}
+
+	/* Build the expected image in the main buffer. */
+	i = 0;
+	memset(buffer, 0, bufsize);
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+		for (j = pr->from; j < pr->to; j++) {
+			buffer[i++] = pattern(j);
+			if (i >= bufsize)
+				goto stop;
+		}
+	}
+stop:
+
+	/* Compare the images */
+	for (i = 0; i < bufsize; i++) {
+		KUNIT_EXPECT_EQ_MSG(test, scratch[i], buffer[i], "at i=%x", i);
+		if (scratch[i] != buffer[i])
+			return;
+	}
+
+	KUNIT_SUCCEED(test);
+}
+
 static void iov_kunit_destroy_xarray(void *data)
 {
 	struct xarray *xarray = data;
@@ -867,6 +1045,85 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
 	KUNIT_SUCCEED(test);
 }
 
+/*
+ * Test the extraction of ITER_BVECQ-type iterators.
+ */
+static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
+{
+	const struct kvec_test_range *pr;
+	struct iov_iter iter;
+	struct bvecq *bq;
+	struct page **bpages, *pagelist[8], **pages = pagelist;
+	ssize_t len;
+	size_t bufsize, size = 0, npages;
+	int i, from;
+
+	bufsize = 0x100000;
+	npages = bufsize / PAGE_SIZE;
+
+	bq = iov_kunit_create_bvecq(test, 13);
+
+	iov_kunit_create_buffer(test, &bpages, npages);
+	iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+		from = pr->from;
+		size = pr->to - from;
+		KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+		iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+		iov_iter_advance(&iter, from);
+
+		do {
+			size_t offset0 = LONG_MAX;
+
+			for (i = 0; i < ARRAY_SIZE(pagelist); i++)
+				pagelist[i] = (void *)(unsigned long)0xaa55aa55aa55aa55ULL;
+
+			len = iov_iter_extract_pages(&iter, &pages, 100 * 1024,
+						     ARRAY_SIZE(pagelist), 0, &offset0);
+			KUNIT_EXPECT_GE(test, len, 0);
+			if (len < 0)
+				break;
+			KUNIT_EXPECT_LE(test, len, size);
+			KUNIT_EXPECT_EQ(test, iter.count, size - len);
+			if (len == 0)
+				break;
+			size -= len;
+			KUNIT_EXPECT_GE(test, (ssize_t)offset0, 0);
+			KUNIT_EXPECT_LT(test, offset0, PAGE_SIZE);
+
+			for (i = 0; i < ARRAY_SIZE(pagelist); i++) {
+				struct page *p;
+				ssize_t part = min_t(ssize_t, len, PAGE_SIZE - offset0);
+				int ix;
+
+				KUNIT_ASSERT_GE(test, part, 0);
+				ix = from / PAGE_SIZE;
+				KUNIT_ASSERT_LT(test, ix, npages);
+				p = bpages[ix];
+				KUNIT_EXPECT_PTR_EQ(test, pagelist[i], p);
+				KUNIT_EXPECT_EQ(test, offset0, from % PAGE_SIZE);
+				from += part;
+				len -= part;
+				KUNIT_ASSERT_GE(test, len, 0);
+				if (len == 0)
+					break;
+				offset0 = 0;
+			}
+
+			if (test->status == KUNIT_FAILURE)
+				goto stop;
+		} while (iov_iter_count(&iter) > 0);
+
+		KUNIT_EXPECT_EQ(test, size, 0);
+		KUNIT_EXPECT_EQ(test, iter.count, 0);
+	}
+
+stop:
+	KUNIT_SUCCEED(test);
+}
+
 /*
  * Test the extraction of ITER_FOLIOQ-type iterators.
  */
@@ -1226,12 +1483,15 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
 	KUNIT_CASE(iov_kunit_copy_from_kvec),
 	KUNIT_CASE(iov_kunit_copy_to_bvec),
 	KUNIT_CASE(iov_kunit_copy_from_bvec),
+	KUNIT_CASE(iov_kunit_copy_to_bvecq),
+	KUNIT_CASE(iov_kunit_copy_from_bvecq),
 	KUNIT_CASE(iov_kunit_copy_to_folioq),
 	KUNIT_CASE(iov_kunit_copy_from_folioq),
 	KUNIT_CASE(iov_kunit_copy_to_xarray),
 	KUNIT_CASE(iov_kunit_copy_from_xarray),
 	KUNIT_CASE(iov_kunit_extract_pages_kvec),
 	KUNIT_CASE(iov_kunit_extract_pages_bvec),
+	KUNIT_CASE(iov_kunit_extract_pages_bvecq),
 	KUNIT_CASE(iov_kunit_extract_pages_folioq),
 	KUNIT_CASE(iov_kunit_extract_pages_xarray),
 	KUNIT_CASE(iov_kunit_iter_to_sg_kvec),



From dhowells at redhat.com  Tue Aug  4 03:02:01 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:01 +0100
Subject: [PATCH v8 06/25] netfs: Add some tools for managing bvecq chains
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-7-dhowells@redhat.com>

Provide a selection of tools for managing bvec queue chains.  This
includes:

 (1) Allocation, prepopulation, expansion, shortening and refcounting of
     bvecqs and bvecq chains.

     This can be used to do things like creating an encryption buffer in
     cifs or a directory content buffer in afs.  The memory segments will
     be appropriate disposed off according to the flags on the bvecq.

 (2) Management of a bvecq chain as a rolling buffer and the management of
     positions within it.

 (3) Loading folios, slicing chains and clearing content.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/Makefile            |   1 +
 fs/netfs/bvecq.c             | 770 +++++++++++++++++++++++++++++++++++
 fs/netfs/internal.h          |   1 +
 fs/netfs/stats.c             |   4 +-
 include/linux/bvecq.h        | 302 ++++++++++++++
 include/linux/netfs.h        |   1 +
 include/trace/events/netfs.h |  24 ++
 7 files changed, 1102 insertions(+), 1 deletion(-)
 create mode 100644 fs/netfs/bvecq.c

diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index b43188d64bd8..e1f12ecb5abf 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -3,6 +3,7 @@
 netfs-y := \
 	buffered_read.o \
 	buffered_write.o \
+	bvecq.o \
 	direct_read.o \
 	direct_write.o \
 	iterator.o \
diff --git a/fs/netfs/bvecq.c b/fs/netfs/bvecq.c
new file mode 100644
index 000000000000..b8dd5250ec05
--- /dev/null
+++ b/fs/netfs/bvecq.c
@@ -0,0 +1,770 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Buffering helpers for bvec queues
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells at redhat.com)
+ */
+
+#include <linux/bvecq.h>
+#include "internal.h"
+
+void bvecq_dump(const struct bvecq *bq)
+{
+	int b = 0;
+
+	for (; bq; bq = bvecq_next(bq), b++) {
+		int skipz = 0;
+
+		pr_notice("BQ[%u] %u/%u fp=%llx%s\n",
+			  b, bq->nr_slots, bq->max_slots, bq->fpos,
+			  bq->discontig ? " discontig" : "");
+		for (int s = 0; s < bq->nr_slots; s++) {
+			const struct bio_vec *bv = &bq->bv[s];
+
+			if (!bv->bv_page && !bv->bv_len && skipz < 2) {
+				skipz = 1;
+				continue;
+			}
+			if (skipz == 1)
+				pr_notice("BQ[%u:00-%02u] ...\n", b, s - 1);
+			skipz = 2;
+			pr_notice("BQ[%u:%02u] %10lx %04x %04x %u\n",
+				  b, s,
+				  bv->bv_page ? page_to_pfn(bv->bv_page) : 0,
+				  bv->bv_offset, bv->bv_len,
+				  bv->bv_page ? page_count(bv->bv_page) : 0);
+		}
+	}
+}
+EXPORT_SYMBOL(bvecq_dump);
+
+/**
+ * bvecq_alloc_one - Allocate a single bvecq node with unpopulated slots
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ *
+ * Allocate a single bvecq node and initialise the header.  A number of inline
+ * slots are also allocated, rounded up to fit after the header in a power-of-2
+ * slab object of up to 512 bytes (up to 29 slots on a 64-bit cpu).  The slot
+ * array is not initialised.
+ *
+ * Return: The node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp)
+{
+	struct bvecq *bq;
+	const size_t max_size = 512;
+	const size_t max_slots = (max_size - sizeof(*bq)) / sizeof(bq->__bv[0]);
+	size_t part = umin(nr_slots, max_slots);
+	size_t size = roundup_pow_of_two(struct_size(bq, __bv, part));
+
+	bq = kmalloc(size, gfp & ~GFP_ZONEMASK);
+	if (bq) {
+		*bq = (struct bvecq) {
+			.ref		= REFCOUNT_INIT(1),
+			.bv		= bq->__bv,
+			.inline_bv	= true,
+			.max_slots	= (size - sizeof(*bq)) / sizeof(bq->__bv[0]),
+		};
+		netfs_stat(&netfs_n_bvecq);
+	}
+	return bq;
+}
+EXPORT_SYMBOL(bvecq_alloc_one);
+
+/**
+ * bvecq_alloc_chain - Allocate an unpopulated bvecq chain
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ *
+ * Allocate a chain of bvecq nodes providing at least the requested cumulative
+ * number of slots.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp)
+{
+	struct bvecq *head = NULL, *tail = NULL;
+
+	_enter("%zu", nr_slots);
+
+	for (;;) {
+		struct bvecq *bq;
+
+		bq = bvecq_alloc_one(nr_slots, gfp);
+		if (!bq)
+			goto oom;
+
+		if (tail)
+			bvecq_append(tail, bq);
+		else
+			head = bq;
+		tail = bq;
+		if (tail->max_slots >= nr_slots)
+			break;
+		nr_slots -= tail->max_slots;
+	}
+
+	return head;
+oom:
+	bvecq_put(head);
+	return NULL;
+}
+EXPORT_SYMBOL(bvecq_alloc_chain);
+
+/**
+ * bvecq_alloc_buffer2 - Allocate a bvecq chain and populate with buffers
+ * @size: Target size of the buffer (can be 0 for an empty buffer)
+ * @pre_slots: Number of preamble slots to set aside
+ * @gfp: The allocation constraints.
+ *
+ * Allocate a chain of bvecq nodes and populate the slots with sufficient pages
+ * to provide at least the requested amount of space, leaving the first
+ * @pre_slots slots unset.  The pre-slots must all fit into the the first
+ * bvecq.
+ *
+ * The pages allocated may be compound pages larger than PAGE_SIZE and thus
+ * occupy fewer slots.  The pages have their refcounts set to 1 and can be
+ * passed to MSG_SPLICE_PAGES.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp)
+{
+	struct bvecq *head = NULL, *p = NULL;
+	size_t nr_per_bq = BVECQ_STD_SLOTS;
+	size_t count = pre_slots + DIV_ROUND_UP(size, PAGE_SIZE);
+
+	_enter("%zx,%zx,%u", size, count, pre_slots);
+
+	if (WARN_ON_ONCE(pre_slots > nr_per_bq))
+		return NULL;
+
+	head = bvecq_alloc_chain(count, gfp);
+	if (!head)
+		return NULL;
+
+	p = head;
+	do {
+		struct page **pages;
+		size_t unused, want, got, slot;
+
+		if (!count)
+			break;
+		if (WARN_ON_ONCE(!p))
+			goto oom;
+
+		if (p->nr_slots == 0) {
+			/* Need to clear pre slots and pages[], so just clear all. */
+			memset(p->bv, 0, p->max_slots * sizeof(p->bv[0]));
+			p->mem_type = BVECQ_MEM_ALLOCED;
+			p->nr_slots = pre_slots;
+			count -= pre_slots;
+			pre_slots = 0;
+			if (!count)
+				break;
+		}
+
+		if (p->nr_slots >= p->max_slots) {
+			p = p->next;
+			continue;
+		}
+		unused = p->max_slots - p->nr_slots;
+
+		pages = (struct page **)&p->bv[p->max_slots];
+		pages -= unused;
+
+		want = min(count, unused);
+		got = alloc_pages_bulk(gfp, want, pages);
+		if (!got)
+			goto oom;
+
+		slot = p->nr_slots;
+		for (int i = 0; i < got; i++) {
+			set_page_count(pages[i], 1);
+			bvec_set_page(&p->bv[slot++], pages[i], PAGE_SIZE, 0);
+		}
+
+		bvecq_filled_to(p, slot);
+		count -= got;
+	} while (count > 0);
+
+	return head;
+oom:
+	bvecq_put(head);
+	return NULL;
+}
+EXPORT_SYMBOL(bvecq_alloc_buffer2);
+
+/*
+ * Free the page pointed to by a slot as necessary.
+ */
+static void bvecq_free_slot(struct bvecq *bq, unsigned int slot)
+{
+	struct page *page = bq->bv[slot].bv_page;
+
+	if (!page)
+		return;
+
+	switch (bq->mem_type) {
+	case BVECQ_MEM_EXTERNAL:
+		break;
+	case BVECQ_MEM_PAGECACHE:
+		put_page(page);
+		break;
+	case BVECQ_MEM_GUP:
+		unpin_user_page(page);
+		break;
+	case BVECQ_MEM_ALLOCED:
+		__free_pages(page, compound_order(page));
+		break;
+	default:
+		WARN_ON_ONCE(1);
+		break;
+	}
+}
+
+/**
+ * bvecq_put - Put a ref on a bvec queue
+ * @bq: The start of the folio queue to free
+ *
+ * Put the ref(s) on the nodes in a bvec queue, freeing up the node and the
+ * page fragments it points to as the refcounts become zero.
+ */
+void bvecq_put(struct bvecq *bq)
+{
+	struct bvecq *next;
+
+	for (; bq; bq = next) {
+		if (!refcount_dec_and_test(&bq->ref))
+			break;
+		for (int slot = 0; slot < bq->nr_slots; slot++)
+			bvecq_free_slot(bq, slot);
+		next = bq->next;
+		netfs_stat_d(&netfs_n_bvecq);
+		kfree(bq);
+	}
+}
+EXPORT_SYMBOL(bvecq_put);
+
+/**
+ * bvecq_expand_buffer - Allocate buffer space into a bvec queue
+ * @_buffer: Pointer to the bvecq chain to expand (may point to a NULL; updated).
+ * @_cur_size: Current size of the buffer (updated).
+ * @size: Target size of the buffer.
+ * @gfp: The allocation constraints.
+ *
+ * Append extra pages to a buffer to increase its capacity to the @size
+ * specified.  If the current tail has space, but is not of the
+ * BVECQ_MEM_ALLOCED memory type, a separate bvecq will be allocated to hold
+ * the new memory.
+ */
+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp)
+{
+	struct bvecq *tail = *_buffer;
+
+	size = round_up(size, PAGE_SIZE);
+	if (tail)
+		while (tail->next)
+			tail = tail->next;
+
+	while (*_cur_size < size) {
+		struct page *page;
+		size_t need = size - *_cur_size;
+		int order = 0;
+
+		if (!tail || bvecq_is_full(tail) || tail->mem_type != BVECQ_MEM_ALLOCED) {
+			struct bvecq *p;
+
+			p = bvecq_alloc_one(BVECQ_STD_SLOTS, gfp);
+			if (!p)
+				return -ENOMEM;
+			if (tail)
+				bvecq_append(tail, p);
+			else
+				*_buffer = p;
+			tail = p;
+			p->mem_type = BVECQ_MEM_ALLOCED;
+		}
+
+		if (need > PAGE_SIZE)
+			order = umin(ilog2(need) - PAGE_SHIFT, MAX_PAGECACHE_ORDER);
+
+		page = alloc_pages(gfp | __GFP_COMP, order);
+		if (!page && order > 0) {
+			page = alloc_pages(gfp | __GFP_COMP, 0);
+			order = 0;
+		}
+		if (!page)
+			return -ENOMEM;
+
+		bvec_set_page(&tail->bv[tail->nr_slots], page, PAGE_SIZE << order, 0);
+		*_cur_size += PAGE_SIZE << order;
+		bvecq_filled_to(tail, tail->nr_slots + 1);
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL(bvecq_expand_buffer);
+
+/**
+ * bvecq_shorten_buffer - Shorten a bvec queue buffer
+ * @bq: The start of the buffer to shorten
+ * @slot: The slot to start from
+ * @size: The size to retain
+ *
+ * Shorten the content of a bvec queue down to the minimum number of slots,
+ * starting at the specified slot, to retain the specified size.
+ *
+ * Return: 0 if successful; -EMSGSIZE if there is insufficient content.
+ */
+int bvecq_shorten_buffer(struct bvecq *bq, unsigned int slot, size_t size)
+{
+	struct bvecq *next;
+
+	/* Skip through the segments we want to keep. */
+	for (; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			if (size < bq->bv[slot].bv_len)
+				goto found;
+			size -= bq->bv[slot].bv_len;
+		}
+		slot = 0;
+	}
+	if (WARN_ON_ONCE(size > 0))
+		return -EMSGSIZE;
+	return 0;
+
+found:
+	/* Shorten any partial entry and clean the rest of this bvecq. */
+	if (size > 0) {
+		bq->bv[slot].bv_len = size;
+		slot++;
+	}
+	for (int i = slot; i < bq->nr_slots; i++)
+		bvecq_free_slot(bq, i);
+	bq->nr_slots = slot;
+
+	/* Free the queue tail. */
+	next = bq->next;
+	bq->next = NULL;
+	bvecq_put(next);
+	return 0;
+}
+EXPORT_SYMBOL(bvecq_shorten_buffer);
+
+/**
+ * bvecq_buffer_init - Initialise a buffer and set position
+ * @pos: The position to point at the new buffer.
+ * @gfp: The allocation constraints.
+ *
+ * Initialise a rolling buffer.  We allocate an unpopulated bvecq node to so
+ * that the pointers can be independently driven by the producer and the
+ * consumer.
+ *
+ * Return 0 if successful; -ENOMEM on allocation failure.
+ */
+int bvecq_buffer_init(struct bvecq_pos *pos, gfp_t gfp)
+{
+	struct bvecq *bq;
+
+	bq = bvecq_alloc_one(BVECQ_STD_SLOTS, gfp);
+	if (!bq)
+		return -ENOMEM;
+
+	pos->bvecq  = bq; /* Comes with a ref. */
+	pos->slot   = 0;
+	pos->offset = 0;
+	return 0;
+}
+
+/**
+ * bvecq_buffer_append - Append a new bvecq node to a buffer
+ * @pos: The position of the last node.
+ * @bq: The buffer to add.
+ *
+ * Add a new node on to the buffer chain at the specified position, either
+ * because the previous one is full or because we have a discontiguity to
+ * contend with, and update @pos to point to it.
+ */
+void bvecq_buffer_append(struct bvecq_pos *pos, struct bvecq *bq)
+{
+	struct bvecq *head = pos->bvecq;
+
+	pos->bvecq = bvecq_get(bq);
+	pos->slot = 0;
+	pos->offset = 0;
+
+	/* [!] NOTE: After we set head->next, the consumer is at liberty to
+	 * immediately delete the old head.
+	 */
+	bvecq_append(head, bq);
+	bvecq_put(head);
+}
+
+/**
+ * bvecq_pos_advance - Advance a bvecq position
+ * @pos: The position to advance.
+ * @amount: The amount of bytes to advance by.
+ *
+ * Advance the specified bvecq position by @amount bytes.  @pos is updated and
+ * bvecq ref counts may have been manipulated.  If the position hits the end of
+ * the queue, then it is left pointing beyond the last slot of the last bvecq
+ * so that it doesn't break the chain.
+ */
+void bvecq_pos_advance(struct bvecq_pos *pos, size_t amount)
+{
+	struct bvecq *bq = pos->bvecq, *next;
+	unsigned int slot = pos->slot;
+	size_t offset = pos->offset;
+
+	while (amount) {
+		size_t part;
+
+		if (!bvecq_acquire_slot(bq, slot)) {
+			next = bvecq_next(bq);
+			if (!next) {
+				WARN_ON_ONCE(amount > 0);
+				break;
+			}
+			bq = next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		part = bq->bv[slot].bv_len - offset;
+
+		if (part > amount) {
+			offset += amount;
+			break;
+		}
+		amount -= part;
+		offset = 0;
+		slot++;
+	}
+
+	pos->slot   = slot;
+	pos->offset = offset;
+	bvecq_pos_move(pos, bq);
+}
+
+/*
+ * Clear part of the memory pointed to by a bio_vec.
+ */
+static void bvec_zero(const struct bio_vec *bv, size_t offset, size_t len)
+{
+	struct page *page = bv->bv_page;
+
+	offset += bv->bv_offset;
+
+	page  += offset / PAGE_SIZE;
+	offset = offset % PAGE_SIZE;
+
+	while (len) {
+		size_t part = umin(len, PAGE_SIZE - offset);
+		char *p = kmap_local_page(page);
+
+		memset(p + offset, 0, part);
+		kunmap_local(p);
+
+		len -= part;
+		offset = 0;
+		page++;
+	}
+}
+
+/**
+ * bvecq_zero - Clear memory starting at the bvecq position.
+ * @pos: The position in the bvecq chain to start clearing.
+ * @amount: The number of bytes to clear.
+ *
+ * Clear memory fragments pointed to by a bvec queue.  @pos is updated and
+ * bvecq ref counts may have been manipulated.  If the position hits the end of
+ * the queue, then it is left pointing beyond the last slot of the last bvecq
+ * so that it doesn't break the chain.
+ *
+ * Return: The number of bytes cleared.
+ */
+ssize_t bvecq_zero(struct bvecq_pos *pos, size_t amount)
+{
+	struct bvecq *bq = pos->bvecq, *next;
+	unsigned int slot = pos->slot;
+	ssize_t cleared = 0;
+	size_t offset = pos->offset;
+
+	while (amount) {
+		const struct bio_vec *bv;
+		size_t part;
+
+		if (!bvecq_acquire_slot(bq, slot)) {
+			next = bvecq_next(bq);
+			if (!next) {
+				WARN_ON_ONCE(amount > 0);
+				break;
+			}
+			bq = next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		bv = &bq->bv[slot];
+		if (offset >= bv->bv_len) {
+			slot++;
+			offset = 0;
+			continue;
+		}
+
+		part = min(bv->bv_len - offset, amount);
+		bvec_zero(bv, offset, part);
+		cleared += part;
+		offset += part;
+		amount -= part;
+	}
+
+	pos->slot   = slot;
+	pos->offset = offset;
+	bvecq_pos_move(pos, bq);
+	return cleared;
+}
+
+/**
+ * bvecq_slice - Find a slice of a bvecq queue
+ * @pos: The position to start at.
+ * @max_size: The maximum size of the slice (or ULONG_MAX).
+ * @max_slots: The maximum number of slots in the slice (or INT_MAX).
+ * @_nr_slots: Where to put the number of slots (updated).
+ *
+ * Determine the size and number of slots that can be obtained the next slice
+ * of bvec queue up to the maximum size and slot count specified.  The slice is
+ * also limited if a discontiguity is found.
+ *
+ * @pos is updated to the end of the slice.  If the position hits the end of
+ * the queue, then it is left pointing beyond the last slot of the last bvecq
+ * so that it doesn't break the chain.
+ *
+ * Return: The number of bytes in the slice.
+ */
+size_t bvecq_slice(struct bvecq_pos *pos, size_t max_size,
+		   unsigned int max_slots, unsigned int *_nr_slots)
+{
+	struct bvecq *bq, *next;
+	unsigned int slot = pos->slot, nslots = 0;
+	size_t size = 0, offset = pos->offset;
+
+	bq = pos->bvecq;
+	for (;;) {
+		for (; slot < bvecq_nr_slots_acquire(bq); slot++) {
+			const struct bio_vec *bvec = &bq->bv[slot];
+
+			if (offset < bvec->bv_len && bvec->bv_page) {
+				size_t part = umin(bvec->bv_len - offset, max_size);
+
+				size += part;
+				offset += part;
+				max_size -= part;
+				nslots++;
+				if (!max_size || nslots >= max_slots)
+					goto out;
+			}
+			offset = 0;
+		}
+
+		/* pos->bvecq isn't allowed to go NULL as the queue may get
+		 * extended and we would lose our place.
+		 */
+		next = bvecq_next(bq);
+		if (!next)
+			break;
+		slot = 0;
+		bq = next;
+		if (bq->discontig && size > 0)
+			break;
+	}
+
+out:
+	*_nr_slots = nslots;
+	if (slot == bvecq_nr_slots_acquire(bq)) {
+		next = bvecq_next(bq);
+		if (next) {
+			bq = next;
+			slot = 0;
+			offset = 0;
+		}
+	}
+	bvecq_pos_move(pos, bq);
+	pos->slot = slot;
+	pos->offset = offset;
+	return size;
+}
+
+/**
+ * bvecq_extract - Extract a slice of a bvecq queue into a new bvecq queue
+ * @pos: The position to start at.
+ * @max_size: The maximum size of the slice (or ULONG_MAX).
+ * @max_slots: The maximum number of slots in the slice (or INT_MAX).
+ * @to: Where to put the extraction bvecq chain head (updated).
+ *
+ * Allocate a new bvecq and extract into it memory fragments from a slice of
+ * bvec queue, starting at @pos.  The slice is also limited if a discontiguity
+ * is found.  No refs are taken on the page.
+ *
+ * @pos is updated to the end of the slice.  If the position hits the end of
+ * the queue, then it is left pointing beyond the last slot of the last bvecq
+ * so that it doesn't break the chain.
+ *
+ * If successful, *@to is set to point to the head of the newly allocated chain
+ * and the caller inherits a ref to it.
+ *
+ * Return: The number of bytes extracted; -ENOMEM on allocation failure or -EIO
+ * if no slots were available to extract.
+ */
+ssize_t bvecq_extract(struct bvecq_pos *pos, size_t max_size,
+		      unsigned int max_slots, struct bvecq **to)
+{
+	struct bvecq_pos tmp_pos;
+	struct bvecq *src, *dst = NULL, *next;
+	unsigned int slot = pos->slot, dslot = 0, nslots;
+	ssize_t extracted = 0;
+	size_t offset = pos->offset, amount;
+
+	*to = NULL;
+	if (WARN_ON_ONCE(!max_slots))
+		max_slots = INT_MAX;
+
+	bvecq_pos_set(&tmp_pos, pos);
+	amount = bvecq_slice(&tmp_pos, max_size, max_slots, &nslots);
+	bvecq_pos_unset(&tmp_pos);
+	if (nslots == 0)
+		return -EIO;
+
+	dst = bvecq_alloc_chain(nslots, GFP_NOFS);
+	if (!dst)
+		return -ENOMEM;
+	*to = dst;
+	max_slots = nslots;
+	nslots = 0;
+
+	/* Transcribe the slots */
+	src = pos->bvecq;
+	for (;;) {
+		for (; slot < bvecq_nr_slots_acquire(src); slot++) {
+			const struct bio_vec *sv = &src->bv[slot];
+			struct bio_vec *dv = &dst->bv[dslot];
+
+			_debug("EXTR BQ=%x[%x] off=%zx am=%zx p=%lx",
+			       src->priv, slot, offset, amount, page_to_pfn(sv->bv_page));
+
+			if (offset < sv->bv_len && sv->bv_page) {
+				size_t part = umin(sv->bv_len - offset, amount);
+
+				bvec_set_page(dv, sv->bv_page, part,
+					      sv->bv_offset + offset);
+				extracted += part;
+				amount -= part;
+				offset += part;
+				trace_netfs_bv_slot(dst, dslot);
+				dslot++;
+				nslots++;
+				if (dslot >= dst->max_slots) {
+					bvecq_filled_to(dst, dslot);
+					dst = dst->next;
+					dslot = 0;
+				}
+				if (nslots >= max_slots)
+					goto out;
+				if (amount == 0)
+					goto out;
+			}
+			offset = 0;
+		}
+
+		/* pos->bvecq isn't allowed to go NULL as the queue may get
+		 * extended and we would lose our place.
+		 */
+		next = bvecq_next(src);
+		if (!next)
+			break;
+		slot = 0;
+		src = next;
+		if (src->discontig && extracted > 0)
+			break;
+	}
+
+out:
+	if (dst)
+		bvecq_filled_to(dst, dslot);
+	if (slot == bvecq_nr_slots_acquire(src)) {
+		next = bvecq_next(src);
+		if (next) {
+			src = next;
+			slot = 0;
+			offset = 0;
+		}
+	}
+	bvecq_pos_move(pos, src);
+	pos->slot = slot;
+	pos->offset = offset;
+	return extracted;
+}
+
+/**
+ * bvecq_load_from_ra - Allocate a bvecq chain and load from readahead
+ * @pos: Blank position object to attach the new chain to.
+ * @ractl: The readahead control context.
+ *
+ * Decant the set of folios to be read from the readahead context into a bvecq
+ * chain.  Each folio occupies one bio_vec element.
+ *
+ * Return: Amount of data loaded or -ENOMEM on allocation failure.
+ */
+ssize_t bvecq_load_from_ra(struct bvecq_pos *pos, struct readahead_control *ractl)
+{
+	XA_STATE(xas, &ractl->mapping->i_pages, ractl->_index);
+	struct folio *folio;
+	struct bvecq *bq;
+	unsigned int slot = 0;
+	size_t loaded = 0;
+
+	bq = bvecq_alloc_chain(ractl->_nr_folios, GFP_NOFS);
+	if (!bq)
+		return -ENOMEM;
+
+	pos->bvecq  = bq;
+	pos->slot   = 0;
+	pos->offset = 0;
+
+	rcu_read_lock();
+
+	xas_for_each(&xas, folio, ractl->_index + ractl->_nr_pages - 1) {
+		size_t len;
+
+		if (xas_retry(&xas, folio))
+			continue;
+		VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
+
+		len = folio_size(folio);
+		bvec_set_folio(&bq->bv[slot], folio, len, 0);
+		loaded += len;
+		slot++;
+		trace_netfs_folio(folio, netfs_folio_trace_read);
+
+		if (slot >= bq->max_slots) {
+			bvecq_filled_to(bq, slot);
+			bq = bq->next;
+			if (!bq)
+				break;
+			slot = 0;
+		}
+	}
+
+	rcu_read_unlock();
+
+	if (bq)
+		bvecq_filled_to(bq, slot);
+
+	ractl->_index += ractl->_nr_pages;
+	ractl->_nr_pages = 0;
+	return loaded;
+}
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index bd8b2d633f96..46fb92601774 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -171,6 +171,7 @@ extern atomic_t netfs_n_wh_retry_write_subreq;
 extern atomic_t netfs_n_wb_lock_skip;
 extern atomic_t netfs_n_wb_lock_wait;
 extern atomic_t netfs_n_folioq;
+extern atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v);
 
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
index ab6b916addc4..84c2a4bcc762 100644
--- a/fs/netfs/stats.c
+++ b/fs/netfs/stats.c
@@ -48,6 +48,7 @@ atomic_t netfs_n_wh_retry_write_subreq;
 atomic_t netfs_n_wb_lock_skip;
 atomic_t netfs_n_wb_lock_wait;
 atomic_t netfs_n_folioq;
+atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v)
 {
@@ -90,9 +91,10 @@ int netfs_stats_show(struct seq_file *m, void *v)
 		   atomic_read(&netfs_n_rh_retry_read_subreq),
 		   atomic_read(&netfs_n_wh_retry_write_req),
 		   atomic_read(&netfs_n_wh_retry_write_subreq));
-	seq_printf(m, "Objs   : rr=%u sr=%u foq=%u wsc=%u\n",
+	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u foq=%u wsc=%u\n",
 		   atomic_read(&netfs_n_rh_rreq),
 		   atomic_read(&netfs_n_rh_sreq),
+		   atomic_read(&netfs_n_bvecq),
 		   atomic_read(&netfs_n_folioq),
 		   atomic_read(&netfs_n_wh_wstream_conflict));
 	seq_printf(m, "WbLock : skip=%u wait=%u\n",
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
index 15f16f905877..fa46be520649 100644
--- a/include/linux/bvecq.h
+++ b/include/linux/bvecq.h
@@ -53,4 +53,306 @@ struct bvecq {
 	struct bio_vec	__bv[];		/* Default array (if ->inline_bv) */
 };
 
+#if BITS_PER_LONG == 64
+/* Number of slots in __bv[] for a bvecq in a 512-byte kmalloc block. */
+#define BVECQ_STD_SLOTS		29	/* 2 words/slot; 32 slots; bvecq is 6 words (3 slots) */
+#elif  BITS_PER_LONG == 32
+/* Number of slots in __bv[] for a bvecq in a 256-byte kmalloc block. */
+#define BVECQ_STD_SLOTS		18	/* 3 words/slot; 21 slots; bvecq is 9 words (3 slots) */
+#else
+#error BVECQ_STD_SLOTS undetermined
+#endif
+
+/*
+ * Position in a bio_vec queue.  The bvecq holds a ref on the queue segment it
+ * points to.
+ */
+struct bvecq_pos {
+	struct bvecq		*bvecq;		/* The first bvecq */
+	unsigned int		offset;		/* The offset within the starting slot */
+	u16			slot;		/* The starting slot */
+};
+
+void bvecq_dump(const struct bvecq *bq);
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp);
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp);
+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp);
+void bvecq_put(struct bvecq *bq);
+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp);
+int bvecq_shorten_buffer(struct bvecq *bq, unsigned int slot, size_t size);
+int bvecq_buffer_init(struct bvecq_pos *pos, gfp_t gfp);
+void bvecq_buffer_append(struct bvecq_pos *pos, struct bvecq *bq);
+void bvecq_pos_advance(struct bvecq_pos *pos, size_t amount);
+ssize_t bvecq_zero(struct bvecq_pos *pos, size_t amount);
+size_t bvecq_slice(struct bvecq_pos *pos, size_t max_size,
+		   unsigned int max_slots, unsigned int *_nr_slots);
+ssize_t bvecq_extract(struct bvecq_pos *pos, size_t max_size,
+		      unsigned int max_slots, struct bvecq **to);
+ssize_t bvecq_load_from_ra(struct bvecq_pos *pos, struct readahead_control *ractl);
+
+/**
+ * bvecq_alloc_buffer - Allocate a bvecq chain and populate with buffers
+ * @size: Target size of the buffer (can be 0 for an empty buffer)
+ * @gfp: The allocation constraints.
+ *
+ * Wrapper around %bvecq_alloc_buffer2().
+ */
+static inline struct bvecq *bvecq_alloc_buffer(size_t size, gfp_t gfp)
+{
+	return bvecq_alloc_buffer2(size, 0, gfp);
+}
+
+/**
+ * bvecq_get - Get a ref on a bvecq
+ * @bq: The bvecq to get a ref on
+ */
+static inline struct bvecq *bvecq_get(struct bvecq *bq)
+{
+	refcount_inc(&bq->ref);
+	return bq;
+}
+
+/**
+ * bvecq_is_full - Determine if a bvecq is full
+ * @bvecq: The object to query
+ *
+ * Return: true if full; false if not.
+ */
+static inline bool bvecq_is_full(const struct bvecq *bvecq)
+{
+	return bvecq->nr_slots >= bvecq->max_slots;
+}
+
+/**
+ * bvecq_filled_to - Release filled slots with release barrier
+ * @bvecq: The object modified
+ * @to: The latest slot filled + 1
+ */
+static inline void bvecq_filled_to(struct bvecq *bvecq, unsigned int to)
+{
+	/* Set the slot counter after filling the slot */
+	smp_store_release(&bvecq->nr_slots, to);
+}
+
+/**
+ * bvecq_nr_slots_acquire - Get the number of filled slots with acquire barrier
+ * @bvecq: The object to query
+ *
+ * Return: The number of filled slots
+ */
+static inline unsigned int bvecq_nr_slots_acquire(const struct bvecq *bvecq)
+{
+	/* Read the slot counter before looking at the slot */
+	return smp_load_acquire(&bvecq->nr_slots);
+}
+
+/**
+ * bvecq_acquire_slot - Determine if a slot is valid with acquire barrier
+ * @bvecq: The object to query
+ * @slot: The next slot
+ *
+ * Return: true if valid; false if might not be valid
+ */
+static inline bool bvecq_acquire_slot(const struct bvecq *bvecq, unsigned int slot)
+{
+	/* Read the slot counter before looking at the slot */
+	return slot < bvecq_nr_slots_acquire(bvecq);
+}
+
+/**
+ * bvecq_append - Get the next bvecq with appropriate barrier
+ * @to: The bvecq to append to
+ * @add: The bvecq to append
+ *
+ * Attach a new bvecq to a chain using an appropriate barrier to protect the
+ * write.
+ *
+ * [!] Note that this function transfers the caller's ref to the chain.
+ */
+static inline void bvecq_append(struct bvecq *to, struct bvecq *add)
+{
+	add->prev = to;
+
+	/* Make sure the initialisation is stored before the next pointer. */
+	smp_store_release(&to->next, add);
+}
+
+/**
+ * bvecq_next - Get the next bvecq with appropriate barrier
+ * @bq: The bvecq to start from
+ *
+ * Return the next bvecq in a chain, using an appropriate barrier to protect
+ * the access.
+ */
+static inline struct bvecq *bvecq_next(const struct bvecq *bq)
+{
+	/* Read the contents of the next node after the pointer to it. */
+	return smp_load_acquire(&bq->next);
+}
+
+/**
+ * bvecq_pos_set - Set one position to be the same as another
+ * @pos: The position object to set
+ * @at: The source position.
+ *
+ * Set @pos to have the same position as @at.  This may take a ref on the
+ * bvecq pointed to.
+ */
+static inline void bvecq_pos_set(struct bvecq_pos *pos, const struct bvecq_pos *at)
+{
+	*pos = *at;
+	bvecq_get(pos->bvecq);
+}
+
+/**
+ * bvecq_pos_unset - Unset a position
+ * @pos: The position object to unset
+ *
+ * Unset @pos.  This does any needed ref cleanup.
+ */
+static inline void bvecq_pos_unset(struct bvecq_pos *pos)
+{
+	bvecq_put(pos->bvecq);
+	pos->bvecq = NULL;
+	pos->slot = 0;
+	pos->offset = 0;
+}
+
+/**
+ * bvecq_pos_transfer - Transfer one position to another, clearing the first
+ * @pos: The position object to set
+ * @from: The source position to clear.
+ *
+ * Set @pos to have the same position as @from and then clear @from.  This may
+ * transfer a ref on the bvecq pointed to.
+ */
+static inline void bvecq_pos_transfer(struct bvecq_pos *pos, struct bvecq_pos *from)
+{
+	*pos = *from;
+	from->bvecq = NULL;
+	from->slot = 0;
+	from->offset = 0;
+}
+
+/**
+ * bvecq_pos_move - Update a position to a new bvecq
+ * @pos: The position object to update.
+ * @to: The new bvecq to point at.
+ *
+ * Update @pos to point to @to if it doesn't already do so.  This may
+ * manipulate refs on the bvecqs pointed to.
+ */
+static inline void bvecq_pos_move(struct bvecq_pos *pos, struct bvecq *to)
+{
+	struct bvecq *old = pos->bvecq;
+
+	if (old != to) {
+		pos->bvecq = bvecq_get(to);
+		bvecq_put(old);
+	}
+}
+
+/**
+ * bvecq_pos_nudge - Nudge a position onto the next segment if current used up
+ * @pos: The position object to nudge.
+ *
+ * Update @pos to point to the next segment in the chain if we've used up the
+ * current segment.  This may manipulate refs on the bvecqs pointed to.
+ *
+ * Return: true if found a new segment, false if hit the end.
+ */
+static inline bool bvecq_pos_nudge(struct bvecq_pos *pos)
+{
+	struct bvecq *bq = pos->bvecq;
+
+	for (;;) {
+		if (!bvecq_acquire_slot(bq, pos->slot)) {
+			bq = bvecq_next(bq);
+			if (!bq)
+				return false;
+			bvecq_pos_move(pos, bq);
+			pos->slot = 0;
+			pos->offset = 0;
+			continue;
+		}
+		if (pos->offset >= bq->bv[pos->slot].bv_len) {
+			pos->slot++;
+			pos->offset = 0;
+			continue;
+		}
+		return true;
+	}
+}
+
+/**
+ * bvecq_pos_step - Step a position to the next slot if possible
+ * @pos: The position object to step.
+ *
+ * Update @pos to point to the next slot in the queue if not at the end.  This
+ * may manipulate refs on the bvecqs pointed to.
+ *
+ * Return: true if successful, false if was at the end.
+ */
+static inline bool bvecq_pos_step(struct bvecq_pos *pos)
+{
+	struct bvecq *bq = pos->bvecq, *next;
+
+	pos->slot++;
+	pos->offset = 0;
+	if (pos->slot < bq->nr_slots)
+		return true;
+	next = bvecq_next(bq);
+	if (!next)
+		return false;
+	bvecq_pos_move(pos, next);
+	pos->slot = 0;
+	return true;
+}
+
+/**
+ * bvecq_delete_spent - Delete the bvecq at the front if possible
+ * @pos: The position object to update.
+ *
+ * Delete the used up bvecq at the front of the queue that @pos points to if it
+ * is not the last node in the queue; if it is the last node in the queue, it
+ * is kept so that the queue doesn't become detached from the other end.  This
+ * may manipulate refs on the bvecqs pointed to.  It is also possible that the
+ * producer will fill more slots in the current bvecq.
+ *
+ * Also, we have to be very careful: the consumer can catch the producer, which
+ * could lead to us having nothing left in the queue, causing the front and
+ * back pointers to end up on different tracks.  To avoid this, we must always
+ * keep at least one segment in the queue.
+ *
+ * The caller must reload from @pos after calling this.
+ *
+ * Return: true if there's more available; false if not.
+ */
+static inline bool bvecq_delete_spent(struct bvecq_pos *pos)
+{
+	struct bvecq *spent = pos->bvecq;
+	struct bvecq *next;
+	unsigned int slot = pos->slot;
+
+again:
+	/* Read the contents of the queue node after the pointer to it. */
+	next = bvecq_next(spent);
+	if (!next)
+		return false; /* Nothing more to consume at the moment. */
+	if (slot < bvecq_nr_slots_acquire(spent))
+		return true; /* The producer added more. */
+	next->prev = NULL;
+	spent->next = NULL;
+	bvecq_put(spent);
+	pos->bvecq = next; /* We take spent's ref. */
+	pos->slot = 0;
+	pos->offset = 0;
+	if (!bvecq_acquire_slot(next, 0)) {
+		spent = next;
+		slot = 0;
+		goto again;
+	}
+	return true;
+}
+
 #endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index e60539f039f1..2010d24975ee 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -17,6 +17,7 @@
 #include <linux/workqueue.h>
 #include <linux/fs.h>
 #include <linux/pagemap.h>
+#include <linux/bvecq.h>
 #include <linux/uio.h>
 #include <linux/rolling_buffer.h>
 
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 9bda9302be90..5354a0439324 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -789,6 +789,30 @@ TRACE_EVENT(netfs_folioq,
 		      __print_symbolic(__entry->trace, netfs_folioq_traces))
 	    );
 
+TRACE_EVENT(netfs_bv_slot,
+	    TP_PROTO(const struct bvecq *bq, int slot),
+
+	    TP_ARGS(bq, slot),
+
+	    TP_STRUCT__entry(
+		    __field(unsigned long,		pfn)
+		    __field(unsigned int,		offset)
+		    __field(unsigned int,		len)
+		    __field(unsigned int,		slot)
+			     ),
+
+	    TP_fast_assign(
+		    __entry->slot = slot;
+		    __entry->pfn = page_to_pfn(bq->bv[slot].bv_page);
+		    __entry->offset = bq->bv[slot].bv_offset;
+		    __entry->len = bq->bv[slot].bv_len;
+			   ),
+
+	    TP_printk("bq[%x] p=%lx %x-%x",
+		      __entry->slot,
+		      __entry->pfn, __entry->offset, __entry->offset + __entry->len)
+	    );
+
 #undef EM
 #undef E_
 #endif /* _TRACE_NETFS_H */



From dhowells at redhat.com  Tue Aug  4 03:02:02 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:02 +0100
Subject: [PATCH v8 07/25] netfs: Make mempool available for bvecq
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-8-dhowells@redhat.com>

Make a mempool available for allocating bvecq structs.  Use it
automatically if anything other than GFP_KERNEL (with GFP_ZONEMASK flags
masked off) is specified.  Reading from a file can use GFP_KERNEL as the
failure mode is straightforward and the same for DIO reads and writes.
When it comes to writeback, however, the writeback_iter() API does not
permit temporary failure, including ENOMEM, if WB_SYNC_ALL is set and the
caller must process all folios to completion.  (I'm not sure if EINTR
constitutes an acceptable failure).

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/bvecq.c      | 27 ++++++++++++++++++++++-----
 fs/netfs/internal.h   |  1 +
 fs/netfs/main.c       |  7 +++++++
 include/linux/bvecq.h |  3 +++
 4 files changed, 33 insertions(+), 5 deletions(-)

diff --git a/fs/netfs/bvecq.c b/fs/netfs/bvecq.c
index b8dd5250ec05..9be1d07a158e 100644
--- a/fs/netfs/bvecq.c
+++ b/fs/netfs/bvecq.c
@@ -45,8 +45,9 @@ EXPORT_SYMBOL(bvecq_dump);
  *
  * Allocate a single bvecq node and initialise the header.  A number of inline
  * slots are also allocated, rounded up to fit after the header in a power-of-2
- * slab object of up to 512 bytes (up to 29 slots on a 64-bit cpu).  The slot
- * array is not initialised.
+ * slab object of up to 512 bytes (up to 29 slots on a 64-bit cpu).  The caller
+ * should be aware that the number of slots allocated may be more or less than
+ * the number requested.  The slot array is not initialised.
  *
  * Return: The node pointer or NULL on allocation failure.
  */
@@ -57,14 +58,27 @@ struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp)
 	const size_t max_slots = (max_size - sizeof(*bq)) / sizeof(bq->__bv[0]);
 	size_t part = umin(nr_slots, max_slots);
 	size_t size = roundup_pow_of_two(struct_size(bq, __bv, part));
-
-	bq = kmalloc(size, gfp & ~GFP_ZONEMASK);
+	bool from_pool = false;
+
+	gfp &= ~GFP_ZONEMASK;
+	if (size != BVECQ_STD_SIZE) {
+		bq = kmalloc(size, gfp);
+	} else {
+		bq = netfs_bvecq_pool.alloc(gfp, netfs_bvecq_pool.pool_data);
+		from_pool = true;
+	}
+	if (!bq && gfp != GFP_KERNEL) {
+		bq = mempool_alloc(&netfs_bvecq_pool, gfp);
+		from_pool = true;
+		size = BVECQ_STD_SIZE;
+	}
 	if (bq) {
 		*bq = (struct bvecq) {
 			.ref		= REFCOUNT_INIT(1),
 			.bv		= bq->__bv,
 			.inline_bv	= true,
 			.max_slots	= (size - sizeof(*bq)) / sizeof(bq->__bv[0]),
+			.from_pool	= from_pool,
 		};
 		netfs_stat(&netfs_n_bvecq);
 	}
@@ -242,7 +256,10 @@ void bvecq_put(struct bvecq *bq)
 			bvecq_free_slot(bq, slot);
 		next = bq->next;
 		netfs_stat_d(&netfs_n_bvecq);
-		kfree(bq);
+		if (bq->from_pool)
+			mempool_free(bq, &netfs_bvecq_pool);
+		else
+			kfree(bq);
 	}
 }
 EXPORT_SYMBOL(bvecq_put);
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 46fb92601774..c68eea2ecc60 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -43,6 +43,7 @@ extern struct list_head netfs_io_requests;
 extern spinlock_t netfs_proc_lock;
 extern mempool_t netfs_request_pool;
 extern mempool_t netfs_subrequest_pool;
+extern mempool_t netfs_bvecq_pool;
 extern mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 927badf3989d..9f72e5054aff 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -28,6 +28,7 @@ static struct kmem_cache *netfs_request_slab;
 static struct kmem_cache *netfs_subrequest_slab;
 mempool_t netfs_request_pool;
 mempool_t netfs_subrequest_pool;
+mempool_t netfs_bvecq_pool;
 mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
@@ -112,6 +113,9 @@ static int __init netfs_init(void)
 	if (mempool_init_kmalloc_pool(&netfs_folioq_pool, 100, sizeof(struct folio_queue)) < 0)
 		goto error_folioq_pool;
 
+	if (mempool_init_kmalloc_pool(&netfs_bvecq_pool, 100, BVECQ_STD_SIZE) < 0)
+		goto error_bvecq_pool;
+
 	netfs_request_slab = kmem_cache_create("netfs_request",
 					       sizeof(struct netfs_io_request), 0,
 					       SLAB_HWCACHE_ALIGN | SLAB_ACCOUNT,
@@ -164,6 +168,8 @@ static int __init netfs_init(void)
 error_reqpool:
 	kmem_cache_destroy(netfs_request_slab);
 error_req:
+	mempool_exit(&netfs_bvecq_pool);
+error_bvecq_pool:
 	mempool_exit(&netfs_folioq_pool);
 error_folioq_pool:
 	return ret;
@@ -178,6 +184,7 @@ static void __exit netfs_exit(void)
 	kmem_cache_destroy(netfs_subrequest_slab);
 	mempool_exit(&netfs_request_pool);
 	kmem_cache_destroy(netfs_request_slab);
+	mempool_exit(&netfs_bvecq_pool);
 	mempool_exit(&netfs_folioq_pool);
 }
 module_exit(netfs_exit);
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
index fa46be520649..d1476393942e 100644
--- a/include/linux/bvecq.h
+++ b/include/linux/bvecq.h
@@ -49,15 +49,18 @@ struct bvecq {
 	enum bvecq_mem	mem_type:3;	/* What sort of memory and how to free it */
 	bool		inline_bv:1;	/* T if __bv[] is being used */
 	bool		discontig:1;	/* T if not contiguous with previous bvecq */
+	bool		from_pool:1;	/* T if bvecq from mempool */
 	struct bio_vec	*bv;		/* Pointer to array of page fragments */
 	struct bio_vec	__bv[];		/* Default array (if ->inline_bv) */
 };
 
 #if BITS_PER_LONG == 64
 /* Number of slots in __bv[] for a bvecq in a 512-byte kmalloc block. */
+#define BVECQ_STD_SIZE		512
 #define BVECQ_STD_SLOTS		29	/* 2 words/slot; 32 slots; bvecq is 6 words (3 slots) */
 #elif  BITS_PER_LONG == 32
 /* Number of slots in __bv[] for a bvecq in a 256-byte kmalloc block. */
+#define BVECQ_STD_SIZE		256
 #define BVECQ_STD_SLOTS		18	/* 3 words/slot; 21 slots; bvecq is 9 words (3 slots) */
 #else
 #error BVECQ_STD_SLOTS undetermined



From dhowells at redhat.com  Tue Aug  4 03:02:03 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:03 +0100
Subject: [PATCH v8 08/25] netfs: Add a function to extract from an iter into a bvecq
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-9-dhowells@redhat.com>

Add a function to extract a slice of data from an iterator of any type into
a bvec queue chain.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/iterator.c   | 145 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/netfs.h |   3 +
 2 files changed, 148 insertions(+)

diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index b375567e0520..4b58b4fb3d77 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -13,6 +13,151 @@
 #include <linux/netfs.h>
 #include "internal.h"
 
+/**
+ * netfs_extract_iter - Extract virtually contiguous pages from an iterator into a bvecq
+ * @orig: The original iterator
+ * @max_len: Maximum number of bytes to extract
+ * @max_pages: Maximum number of pages to extract
+ * @fpos: Starting file position to label the bvecq with
+ * @_bvecq_head: Where to cache the bvec queue
+ * @extraction_flags: Flags to qualify the request
+ * @gfp: Allocation mode for bvecq structs.
+ *
+ * Extract virtually contiguous page fragments from the source iterator up to
+ * the given maxima and build bvec queue that refers to all of those bits.
+ * This allows the original iterator to disposed of.
+ *
+ * @extraction_flags can have ITER_ALLOW_P2PDMA set to request peer-to-peer DMA be
+ * allowed on the pages extracted.
+ *
+ * On success or partial success, the amount of data in the bvec is returned,
+ * the original iterator will have been advanced by the amount extracted.
+ *
+ * If an error occurs and no pages are extracted, an error will be returned and
+ * any allocated bvecq will be freed.  If there is no data to be extracted (or
+ * @max_len or @max_pages are zero), a single empty bvecq will be returned.
+ *
+ * The bvecq segments are marked with indications on how to get clean up the
+ * extracted fragments.
+ */
+ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
+			   unsigned long long fpos, struct bvecq **_bvecq_head,
+			   iov_iter_extraction_t extraction_flags, gfp_t gfp)
+{
+	struct bvecq *bq_tail = NULL, *bq;
+	ssize_t ret = 0;
+	size_t extracted = 0;
+
+	_enter("{%u,%zx},%zx", orig->iter_type, orig->count, max_len);
+
+	*_bvecq_head = NULL;
+	if (max_len > orig->count)
+		max_len = orig->count;
+	if (WARN_ON_ONCE(!max_len || !max_pages))
+		goto alloc_empty;
+	if (WARN_ON_ONCE(max_pages > INT_MAX))
+		max_pages = INT_MAX; /* Protect iov_iter_npages(). */
+
+	max_pages = iov_iter_npages(orig, max_pages);
+	if (!max_pages)
+		goto alloc_empty;
+
+	do {
+		bq = bvecq_alloc_one(max_pages, gfp);
+		if (!bq) {
+			ret = -ENOMEM;
+			break;
+		}
+		if (user_backed_iter(orig))
+			bq->mem_type = iov_iter_extract_will_pin(orig) ?
+				BVECQ_MEM_GUP : BVECQ_MEM_PAGECACHE;
+		bq->fpos	= fpos + extracted;
+
+		if (bq_tail)
+			bvecq_append(bq_tail, bq);
+		else
+			*_bvecq_head = bq;
+		bq_tail = bq;
+
+		if (max_len == 0)
+			break;
+
+		struct bio_vec *bv = bq->bv;
+		unsigned int slot = 0;
+		do {
+			struct page **pages;
+			ssize_t got;
+			size_t offset;
+			size_t space = bq->max_slots - slot;
+			size_t bv_size = array_size(bq->max_slots, sizeof(*bv));
+			size_t pg_size = array_size(space, sizeof(*pages));
+
+			/* Put the page list at the end of the bvec list
+			 * storage.  bvec elements are larger than page
+			 * pointers, so as long as we work 0->last, we should
+			 * be fine.
+			 */
+			pages = (void *)bv + bv_size - pg_size;
+
+			got = iov_iter_extract_pages(orig, &pages, max_len,
+						     min(space, max_pages),
+						     extraction_flags, &offset);
+			if (got < 0) {
+				ret = got;
+				goto out;
+			}
+
+			if (got == 0) {
+				pr_err("extract_pages gave nothing from %zu, %zu\n",
+				       extracted, max_len);
+				ret = -EIO;
+				goto out;
+			}
+
+			if (WARN(got > max_len,
+				 "%s: extract_pages overrun %zd > %zu bytes\n",
+				 __func__, got, max_len)) {
+				ret = -EIO;
+				goto out;
+			}
+
+			extracted += got;
+			max_len -= got;
+
+			do {
+				size_t len = umin(got, PAGE_SIZE - offset);
+
+				BUG_ON(slot >= bq->max_slots);
+
+				bvec_set_page(&bq->bv[slot], *pages++, len, offset);
+				slot++;
+				max_pages--;
+				got -= len;
+				offset = 0;
+			} while (got > 0);
+
+			bvecq_filled_to(bq, slot);
+		} while (max_len > 0 && max_pages > 0 && !bvecq_is_full(bq));
+
+	} while (max_len > 0 && max_pages > 0);
+
+out:
+	if (extracted || ret == 0)
+		return extracted;
+	bvecq_put(*_bvecq_head);
+	*_bvecq_head = NULL;
+	return ret;
+
+alloc_empty:
+	bq = bvecq_alloc_one(1, gfp);
+	if (!bq)
+		return -ENOMEM;
+	*_bvecq_head = bq;
+	return 0;
+
+}
+EXPORT_SYMBOL_GPL(netfs_extract_iter);
+
 /**
  * netfs_extract_user_iter - Extract the pages from a user iterator into a bvec
  * @orig: The original iterator
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 2010d24975ee..d22fcae8d67d 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -440,6 +440,9 @@ void netfs_get_subrequest(struct netfs_io_subrequest *subreq,
 			  enum netfs_sreq_ref_trace what);
 void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 			  enum netfs_sreq_ref_trace what);
+ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
+			   unsigned long long fpos, struct bvecq **_bvecq_head,
+			   iov_iter_extraction_t extraction_flags, gfp_t gfp);
 ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
 				struct iov_iter *new,
 				iov_iter_extraction_t extraction_flags);



From dhowells at redhat.com  Tue Aug  4 03:02:04 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:04 +0100
Subject: [PATCH v8 09/25] afs: Use a bvecq to hold dir content rather than folioq
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-10-dhowells@redhat.com>

Use a bvecq to hold the contents of a directory rather than the folioq so
that the latter can be phased out.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/afs/dir.c           |  35 +++++----
 fs/afs/dir_edit.c      |  42 +++++------
 fs/afs/dir_search.c    |  33 ++++-----
 fs/afs/inode.c         |   2 +-
 fs/afs/internal.h      |   6 +-
 fs/afs/symlink.c       |  34 ++++-----
 fs/netfs/write_issue.c | 161 ++++++-----------------------------------
 7 files changed, 92 insertions(+), 221 deletions(-)

diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 6df56fe9163f..547faa06c80b 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -140,9 +140,9 @@ static void afs_dir_dump(struct afs_vnode *dvnode)
 	pr_warn("DIR %llx:%llx is=%llx\n",
 		dvnode->fid.vid, dvnode->fid.vnode, i_size);
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
-	iterate_folioq(&iter, iov_iter_count(&iter), NULL, NULL,
-		       afs_dir_dump_step);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	iterate_bvecq(&iter, iov_iter_count(&iter), NULL, NULL,
+		      afs_dir_dump_step);
 }
 
 /*
@@ -203,9 +203,9 @@ static int afs_dir_check(struct afs_vnode *dvnode)
 	if (unlikely(!i_size))
 		return 0;
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
-	checked = iterate_folioq(&iter, iov_iter_count(&iter), dvnode, NULL,
-				 afs_dir_check_step);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	checked = iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, NULL,
+				afs_dir_check_step);
 	if (checked != i_size) {
 		afs_dir_dump(dvnode);
 		return -EIO;
@@ -250,15 +250,14 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
 	if (dvnode->directory_size < i_size) {
 		size_t cur_size = dvnode->directory_size;
 
-		ret = netfs_alloc_folioq_buffer(NULL,
-						&dvnode->directory, &cur_size, i_size,
-						mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size, i_size,
+					  GFP_KERNEL);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			return ret;
 	}
 
-	iov_iter_folio_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
+	iov_iter_bvec_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
 
 	/* AFS requires us to perform the read of a directory synchronously as
 	 * a single unit to avoid issues with the directory contents being
@@ -294,8 +293,8 @@ static ssize_t afs_read_single(struct afs_vnode *dvnode, struct file *file)
 }
 
 /*
- * Read the directory into a folio_queue buffer in one go, scrubbing the
- * previous contents.  We return -ESTALE if the caller needs to call us again.
+ * Read the directory into the buffer in one go, scrubbing the previous
+ * contents.  We return -ESTALE if the caller needs to call us again.
  */
 ssize_t afs_read_dir(struct afs_vnode *dvnode, struct file *file)
 	__acquires(&dvnode->validate_lock)
@@ -483,7 +482,7 @@ static size_t afs_dir_iterate_step(void *iter_base, size_t progress, size_t len,
 }
 
 /*
- * Iterate through the directory folios.
+ * Iterate through the directory content.
  */
 static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_ctx)
 {
@@ -498,11 +497,11 @@ static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_c
 	if (i_size <= 0 || dir_ctx->pos >= i_size)
 		return 0;
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
 	iov_iter_advance(&iter, round_down(dir_ctx->pos, AFS_DIR_BLOCK_SIZE));
 
-	iterate_folioq(&iter, iov_iter_count(&iter), dvnode, &ctx,
-		       afs_dir_iterate_step);
+	iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, &ctx,
+		      afs_dir_iterate_step);
 
 	if (ctx.error == -ESTALE)
 		afs_invalidate_dir(dvnode, afs_dir_invalid_iter_stale);
@@ -2228,8 +2227,8 @@ static int afs_dir_writepages(struct address_space *mapping,
 	}
 
 	if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
-		iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
-				     i_size_read(&dvnode->netfs.inode));
+		iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
+				    i_size_read(&dvnode->netfs.inode));
 		ret = netfs_writeback_single(mapping, wbc, &iter);
 		if (ret == 1)
 			ret = 0; /* Skipped write due to lock conflict. */
diff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c
index 3ead36a07048..01c6265e8865 100644
--- a/fs/afs/dir_edit.c
+++ b/fs/afs/dir_edit.c
@@ -110,9 +110,8 @@ static void afs_clear_contig_bits(union afs_xdr_dir_block *block,
  */
 static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, size_t block)
 {
-	struct folio_queue *fq;
 	struct afs_vnode *dvnode = iter->dvnode;
-	struct folio *folio;
+	struct bvecq *bq;
 	size_t blpos = block * AFS_DIR_BLOCK_SIZE;
 	size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
 	int ret;
@@ -120,41 +119,38 @@ static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, siz
 	if (dvnode->directory_size < blend) {
 		size_t cur_size = dvnode->directory_size;
 
-		ret = netfs_alloc_folioq_buffer(
-			NULL, &dvnode->directory, &cur_size, blend,
-			mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size, blend,
+					  GFP_KERNEL);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			goto fail;
 	}
 
-	fq = iter->fq;
-	if (!fq)
-		fq = dvnode->directory;
+	bq = iter->bq;
+	if (!bq)
+		bq = dvnode->directory;
 
-	/* Search the folio queue for the folio containing the block... */
-	for (; fq; fq = fq->next) {
-		for (int s = iter->fq_slot; s < folioq_count(fq); s++) {
-			size_t fsize = folioq_folio_size(fq, s);
+	/* Search the contents for the region containing the block... */
+	for (; bq; bq = bq->next) {
+		for (int s = iter->bq_slot; s < bq->nr_slots; s++) {
+			struct bio_vec *bv = &bq->bv[s];
+			size_t bsize = bv->bv_len;
 
-			if (blend <= fpos + fsize) {
+			if (blend <= fpos + bsize) {
 				/* ... and then return the mapped block. */
-				folio = folioq_folio(fq, s);
-				if (WARN_ON_ONCE(folio_pos(folio) != fpos))
-					goto fail;
-				iter->fq = fq;
-				iter->fq_slot = s;
+				iter->bq = bq;
+				iter->bq_slot = s;
 				iter->fpos = fpos;
-				return kmap_local_folio(folio, blpos - fpos);
+				return bvec_kmap_partial(bv, blpos - fpos);
 			}
-			fpos += fsize;
+			fpos += bsize;
 		}
-		iter->fq_slot = 0;
+		iter->bq_slot = 0;
 	}
 
 fail:
-	iter->fq = NULL;
-	iter->fq_slot = 0;
+	iter->bq = NULL;
+	iter->bq_slot = 0;
 	afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
 	return NULL;
 }
diff --git a/fs/afs/dir_search.c b/fs/afs/dir_search.c
index 104411c0692f..01751c680115 100644
--- a/fs/afs/dir_search.c
+++ b/fs/afs/dir_search.c
@@ -66,12 +66,11 @@ bool afs_dir_init_iter(struct afs_dir_iter *iter, const struct qstr *name)
  */
 union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t block)
 {
-	struct folio_queue *fq = iter->fq;
 	struct afs_vnode *dvnode = iter->dvnode;
-	struct folio *folio;
+	struct bvecq *bq = iter->bq;
 	size_t blpos = block * AFS_DIR_BLOCK_SIZE;
 	size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
-	int slot = iter->fq_slot;
+	int slot = iter->bq_slot;
 
 	_enter("%zx,%d", block, slot);
 
@@ -83,36 +82,34 @@ union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t bl
 	if (dvnode->directory_size < blend)
 		goto fail;
 
-	if (!fq || blpos < fpos) {
-		fq = dvnode->directory;
+	if (!bq || blpos < fpos) {
+		bq = dvnode->directory;
 		slot = 0;
 		fpos = 0;
 	}
 
 	/* Search the folio queue for the folio containing the block... */
-	for (; fq; fq = fq->next) {
-		for (; slot < folioq_count(fq); slot++) {
-			size_t fsize = folioq_folio_size(fq, slot);
+	for (; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			struct bio_vec *bv = &bq->bv[slot];
+			size_t bsize = bv->bv_len;
 
-			if (blend <= fpos + fsize) {
+			if (blend <= fpos + bsize) {
 				/* ... and then return the mapped block. */
-				folio = folioq_folio(fq, slot);
-				if (WARN_ON_ONCE(folio_pos(folio) != fpos))
-					goto fail;
-				iter->fq = fq;
-				iter->fq_slot = slot;
+				iter->bq = bq;
+				iter->bq_slot = slot;
 				iter->fpos = fpos;
-				iter->block = kmap_local_folio(folio, blpos - fpos);
+				iter->block = bvec_kmap_partial(bv, blpos - fpos);
 				return iter->block;
 			}
-			fpos += fsize;
+			fpos += bsize;
 		}
 		slot = 0;
 	}
 
 fail:
-	iter->fq = NULL;
-	iter->fq_slot = 0;
+	iter->bq = NULL;
+	iter->bq_slot = 0;
 	afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
 	return NULL;
 }
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 14f39a9bea6c..634fbf8eb212 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -683,7 +683,7 @@ void afs_evict_inode(struct inode *inode)
 	flush_delayed_work(&vnode->lock_work);
 	netfs_wait_for_outstanding_io(inode);
 	truncate_inode_pages_final(&inode->i_data);
-	netfs_free_folioq_buffer(vnode->directory);
+	bvecq_put(vnode->directory);
 	if (vnode->symlink)
 		afs_evict_symlink(vnode);
 
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 290873bac89b..556e7e7777f2 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -710,7 +710,7 @@ struct afs_vnode {
 #define AFS_VNODE_MODIFYING	10		/* Set if we're performing a modification op */
 #define AFS_VNODE_DIR_READ	11		/* Set if we've read a dir's contents */
 
-	struct folio_queue	*directory;	/* Directory contents */
+	struct bvecq		*directory;	/* Directory contents */
 	struct afs_symlink __rcu *symlink;	/* Symlink content */
 	struct list_head	wb_keys;	/* List of keys available for writeback */
 	struct list_head	pending_locks;	/* locks waiting to be granted */
@@ -991,9 +991,9 @@ static inline void afs_invalidate_cache(struct afs_vnode *vnode, unsigned int fl
 struct afs_dir_iter {
 	struct afs_vnode	*dvnode;
 	union afs_xdr_dir_block *block;
-	struct folio_queue	*fq;
+	struct bvecq		*bq;
 	unsigned int		fpos;
-	int			fq_slot;
+	int			bq_slot;
 	unsigned int		loop_check;
 	u8			nr_slots;
 	u8			bucket;
diff --git a/fs/afs/symlink.c b/fs/afs/symlink.c
index 16b4823cb7b7..84f8ab222dd6 100644
--- a/fs/afs/symlink.c
+++ b/fs/afs/symlink.c
@@ -56,7 +56,6 @@ void afs_evict_symlink(struct afs_vnode *vnode)
 void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)
 {
 	struct afs_symlink *symlink = op->create.symlink;
-	size_t dsize = 0;
 	size_t size = strlen(symlink->content) + 1;
 	char *p;
 
@@ -66,13 +65,15 @@ void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)
 	if (!fscache_cookie_enabled(netfs_i_cookie(&vnode->netfs)))
 		return;
 
-	if (netfs_alloc_folioq_buffer(NULL, &vnode->directory, &dsize, size,
-				      mapping_gfp_mask(vnode->netfs.inode.i_mapping)) < 0)
+	vnode->directory = bvecq_alloc_buffer(PAGE_SIZE, GFP_KERNEL);
+	if (!vnode->directory)
 		return;
 
-	vnode->directory_size = dsize;
-	p = kmap_local_folio(folioq_folio(vnode->directory, 0), 0);
+	vnode->directory_size = size;
+	p = bvec_kmap_partial(&vnode->directory->bv[0], 0);
 	memcpy(p, symlink->content, size);
+	if (size < PAGE_SIZE)
+		memset(p + size, 0, PAGE_SIZE - size);
 	kunmap_local(p);
 	netfs_single_mark_inode_dirty(&vnode->netfs.inode);
 }
@@ -94,17 +95,12 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
 	}
 
 	if (!vnode->directory) {
-		size_t cur_size = 0;
-
-		ret = netfs_alloc_folioq_buffer(NULL,
-						&vnode->directory, &cur_size, PAGE_SIZE,
-						mapping_gfp_mask(vnode->netfs.inode.i_mapping));
-		vnode->directory_size = PAGE_SIZE - 1;
-		if (ret < 0)
-			return ret;
+		vnode->directory = bvecq_alloc_buffer(PAGE_SIZE, GFP_KERNEL);
+		if (!vnode->directory)
+			return -ENOMEM;
 	}
 
-	iov_iter_folio_queue(&iter, ITER_DEST, vnode->directory, 0, 0, PAGE_SIZE);
+	iov_iter_bvec_queue(&iter, ITER_DEST, vnode->directory, 0, 0, PAGE_SIZE);
 
 	/* AFS requires us to perform the read of a symlink as a single unit to
 	 * avoid issues with the content being changed between reads.
@@ -127,7 +123,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
 		refcount_set(&symlink->ref, 1);
 		symlink->content[i_size] = 0;
 
-		const char *s = kmap_local_folio(folioq_folio(vnode->directory, 0), 0);
+		const char *s = bvec_kmap_partial(&vnode->directory->bv[0], 0);
 
 		memcpy(symlink->content, s, i_size);
 		kunmap_local(s);
@@ -136,7 +132,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
 	}
 
 	if (!fscache_cookie_enabled(netfs_i_cookie(&vnode->netfs))) {
-		netfs_free_folioq_buffer(vnode->directory);
+		bvecq_put(vnode->directory);
 		vnode->directory = NULL;
 		vnode->directory_size = 0;
 	}
@@ -249,14 +245,14 @@ int afs_symlink_writepages(struct address_space *mapping,
 
 	if (vnode->directory &&
 	    atomic64_read(&vnode->cb_expires_at) != AFS_NO_CB_PROMISE) {
-		iov_iter_folio_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
-				     i_size_read(&vnode->netfs.inode));
+		iov_iter_bvec_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
+				    i_size_read(&vnode->netfs.inode));
 		ret = netfs_writeback_single(mapping, wbc, &iter);
 	}
 
 	if (ret == 0) {
 		netfs_wb_begin(&vnode->netfs, false);
-		netfs_free_folioq_buffer(vnode->directory);
+		bvecq_put(vnode->directory);
 		vnode->directory = NULL;
 		vnode->directory_size = 0;
 		netfs_wb_end(&vnode->netfs);
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 2d9cfcd43658..a9f24b47f4a8 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -708,129 +708,11 @@ ssize_t netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_c
 	return ret;
 }
 
-/*
- * Write some of a pending folio data back to the server and/or the cache.
- */
-static int netfs_write_folio_single(struct netfs_io_request *wreq,
-				    struct folio *folio)
-{
-	struct netfs_io_stream *upload = &wreq->io_streams[0];
-	struct netfs_io_stream *cache  = &wreq->io_streams[1];
-	struct netfs_io_stream *stream;
-	size_t iter_off = 0;
-	size_t fsize = folio_size(folio), flen;
-	loff_t fpos = folio_pos(folio);
-	ssize_t ret;
-	bool to_eof = false;
-	bool no_debug = false;
-
-	_enter("");
-
-	flen = folio_size(folio);
-	if (flen > wreq->i_size - fpos) {
-		flen = wreq->i_size - fpos;
-		folio_zero_segment(folio, flen, fsize);
-		to_eof = true;
-	} else if (flen == wreq->i_size - fpos) {
-		to_eof = true;
-	}
-
-	_debug("folio %zx/%zx", flen, fsize);
-
-	if (!upload->avail && !cache->avail) {
-		trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
-		return 0;
-	}
-
-	if (!upload->construct)
-		trace_netfs_folio(folio, netfs_folio_trace_store);
-	else
-		trace_netfs_folio(folio, netfs_folio_trace_store_plus);
-
-	/* Attach the folio to the rolling buffer. */
-	folio_get(folio);
-	ret = rolling_buffer_append(&wreq->buffer, folio, NETFS_ROLLBUF_PUT_MARK, wreq->gfp);
-	if (ret < 0) {
-		folio_put(folio);
-		return ret;
-	}
-
-	/* Move the submission point forward to allow for write-streaming data
-	 * not starting at the front of the page.  We don't do write-streaming
-	 * with the cache as the cache requires DIO alignment.
-	 *
-	 * Also skip uploading for data that's been read and just needs copying
-	 * to the cache.
-	 */
-	for (int s = 0; s < NR_IO_STREAMS; s++) {
-		stream = &wreq->io_streams[s];
-		stream->submit_off = 0;
-		stream->submit_len = flen;
-		if (!stream->avail) {
-			stream->submit_off = UINT_MAX;
-			stream->submit_len = 0;
-		}
-	}
-
-	/* Attach the folio to one or more subrequests.  For a big folio, we
-	 * could end up with thousands of subrequests if the wsize is small -
-	 * but we might need to wait during the creation of subrequests for
-	 * network resources (eg. SMB credits).
-	 */
-	for (;;) {
-		ssize_t part;
-		size_t lowest_off = ULONG_MAX;
-		int choose_s = -1;
-
-		/* Always add to the lowest-submitted stream first. */
-		for (int s = 0; s < NR_IO_STREAMS; s++) {
-			stream = &wreq->io_streams[s];
-			if (stream->submit_len > 0 &&
-			    stream->submit_off < lowest_off) {
-				lowest_off = stream->submit_off;
-				choose_s = s;
-			}
-		}
-
-		if (choose_s < 0)
-			break;
-		stream = &wreq->io_streams[choose_s];
-
-		/* Advance the iterator(s). */
-		if (stream->submit_off > iter_off) {
-			rolling_buffer_advance(&wreq->buffer, stream->submit_off - iter_off);
-			iter_off = stream->submit_off;
-		}
-
-		atomic64_set(&wreq->issued_to, fpos + stream->submit_off);
-		stream->submit_extendable_to = fsize - stream->submit_off;
-		part = netfs_advance_write(wreq, stream, fpos + stream->submit_off,
-					   stream->submit_len, to_eof);
-		stream->submit_off += part;
-		if (part > stream->submit_len)
-			stream->submit_len = 0;
-		else
-			stream->submit_len -= part;
-		if (part > 0)
-			no_debug = true;
-	}
-
-	wreq->buffer.iter.iov_offset = 0;
-	if (fsize > iter_off)
-		rolling_buffer_advance(&wreq->buffer, fsize - iter_off);
-	atomic64_set(&wreq->issued_to, fpos + fsize);
-
-	if (!no_debug)
-		kdebug("R=%x: No submit", wreq->debug_id);
-	_leave(" = 0");
-	return 0;
-}
-
 /**
  * netfs_writeback_single - Write back a monolithic payload
  * @mapping: The mapping to write from
  * @wbc: Hints from the VM
- * @iter: Data to write, must be ITER_FOLIOQ.
+ * @iter: Data to write.
  *
  * Write a monolithic, non-pagecache object back to the server and/or
  * the cache.
@@ -844,13 +726,8 @@ int netfs_writeback_single(struct address_space *mapping,
 {
 	struct netfs_io_request *wreq;
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
-	struct folio_queue *fq;
-	size_t size = iov_iter_count(iter);
 	int ret;
 
-	if (WARN_ON_ONCE(!iov_iter_is_folioq(iter)))
-		return -EIO;
-
 	if (!netfs_wb_begin(ictx, wbc->sync_mode == WB_SYNC_NONE)) {
 		/* The VFS will have undirtied the inode. */
 		netfs_single_mark_inode_dirty(&ictx->inode);
@@ -863,6 +740,9 @@ int netfs_writeback_single(struct address_space *mapping,
 		goto couldnt_start;
 	}
 
+	wreq->buffer.iter = *iter;
+	wreq->len = iov_iter_count(iter);
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback_single);
 	netfs_stat(&netfs_n_wh_writepages);
@@ -870,30 +750,33 @@ int netfs_writeback_single(struct address_space *mapping,
 	if (__test_and_set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
 		wreq->netfs_ops->begin_writeback(wreq);
 
-	for (fq = (struct folio_queue *)iter->folioq; fq; fq = fq->next) {
-		for (int slot = 0; slot < folioq_count(fq); slot++) {
-			struct folio *folio = folioq_folio(fq, slot);
-			size_t part = umin(folioq_folio_size(fq, slot), size);
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_subrequest *subreq;
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
 
-			_debug("wbiter %lx %llx", folio->index, atomic64_read(&wreq->issued_to));
+		if (!stream->avail)
+			continue;
 
-			ret = netfs_write_folio_single(wreq, folio);
-			if (ret < 0)
-				goto stop;
-			size -= part;
-			if (size <= 0)
-				goto stop;
-		}
+		netfs_prepare_write(wreq, stream, 0);
+
+		subreq = stream->construct;
+		subreq->len = wreq->len;
+		stream->submit_len = subreq->len;
+		stream->submit_extendable_to = round_up(wreq->len, PAGE_SIZE);
+
+		netfs_issue_write(wreq, stream);
 	}
 
-stop:
-	for (int s = 0; s < NR_IO_STREAMS; s++)
-		netfs_issue_write(wreq, &wreq->io_streams[s]);
 	smp_wmb(); /* Write lists before ALL_QUEUED. */
 	set_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags);
 
 	netfs_wake_collector(wreq);
 
+	/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to
+	 * wait before modifying.
+	 */
+	ret = netfs_wait_for_write(wreq);
+
 	netfs_put_request(wreq, netfs_rreq_trace_put_return);
 	_leave(" = %d", ret);
 	return ret;



From dhowells at redhat.com  Tue Aug  4 03:02:05 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:05 +0100
Subject: [PATCH v8 10/25] cifs: Use a bvecq for buffering instead of a folioq
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-11-dhowells@redhat.com>

Use a bvecq for internal buffering for crypto purposes instead of a folioq
so that the latter can be phased out.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/smb/client/cifsglob.h |  2 +-
 fs/smb/client/smb2ops.c  | 76 +++++++++++++++++++---------------------
 2 files changed, 37 insertions(+), 41 deletions(-)

diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 79e4e84f8985..c8b1c23ef494 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -289,7 +289,7 @@ struct smb_rqst {
 	struct kvec	*rq_iov;	/* array of kvecs */
 	unsigned int	rq_nvec;	/* number of kvecs in array */
 	struct iov_iter	rq_iter;	/* Data iterator */
-	struct folio_queue *rq_buffer;	/* Buffer for encryption */
+	struct bvecq	*rq_buffer;	/* Buffer for encryption */
 };
 
 struct mid_q_entry;
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 192649fec25d..2b5d745ba16e 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -4683,19 +4683,18 @@ crypt_message(struct TCP_Server_Info *server, int num_rqst,
 }
 
 /*
- * Copy data from an iterator to the folios in a folio queue buffer.
+ * Copy data from an iterator to the pages in a bvec queue buffer.
  */
-static bool cifs_copy_iter_to_folioq(struct iov_iter *iter, size_t size,
-				     struct folio_queue *buffer)
+static bool cifs_copy_iter_to_bvecq(struct iov_iter *iter, size_t size,
+				    struct bvecq *buffer)
 {
 	for (; buffer; buffer = buffer->next) {
-		for (int s = 0; s < folioq_count(buffer); s++) {
-			struct folio *folio = folioq_folio(buffer, s);
-			size_t part = folioq_folio_size(buffer, s);
+		for (int s = 0; s < buffer->nr_slots; s++) {
+			struct bio_vec *bv = &buffer->bv[s];
+			size_t part = umin(bv->bv_len, size);
 
-			part = umin(part, size);
-
-			if (copy_folio_from_iter(folio, 0, part, iter) != part)
+			if (copy_page_from_iter(bv->bv_page, bv->bv_offset,
+						part, iter) != part)
 				return false;
 			size -= part;
 		}
@@ -4707,7 +4706,7 @@ void
 smb3_free_compound_rqst(int num_rqst, struct smb_rqst *rqst)
 {
 	for (int i = 0; i < num_rqst; i++)
-		netfs_free_folioq_buffer(rqst[i].rq_buffer);
+		bvecq_put(rqst[i].rq_buffer);
 }
 
 /*
@@ -4734,7 +4733,7 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,
 	for (int i = 1; i < num_rqst; i++) {
 		struct smb_rqst *old = &old_rq[i - 1];
 		struct smb_rqst *new = &new_rq[i];
-		struct folio_queue *buffer = NULL;
+		struct bvecq *buffer = NULL;
 		size_t size = iov_iter_count(&old->rq_iter);
 
 		orig_len += smb_rqst_len(server, old);
@@ -4742,17 +4741,16 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,
 		new->rq_nvec = old->rq_nvec;
 
 		if (size > 0) {
-			size_t cur_size = 0;
-			rc = netfs_alloc_folioq_buffer(NULL, &buffer, &cur_size,
-						       size, GFP_NOFS);
-			new->rq_buffer = buffer;
-			if (rc < 0)
+			rc = -ENOMEM;
+			buffer = bvecq_alloc_buffer(size, GFP_NOFS);
+			if (!buffer)
 				goto err_free;
 
-			iov_iter_folio_queue(&new->rq_iter, ITER_SOURCE,
-					     buffer, 0, 0, size);
+			new->rq_buffer = buffer;
+			iov_iter_bvec_queue(&new->rq_iter, ITER_SOURCE,
+					    buffer, 0, 0, size);
 
-			if (!cifs_copy_iter_to_folioq(&old->rq_iter, size, buffer)) {
+			if (!cifs_copy_iter_to_bvecq(&old->rq_iter, size, buffer)) {
 				rc = smb_EIO1(smb_eio_trace_tx_copy_iter_to_buf, size);
 				goto err_free;
 			}
@@ -4842,22 +4840,20 @@ decrypt_raw_data(struct TCP_Server_Info *server, char *buf,
 }
 
 static int
-cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,
-			 size_t skip, struct iov_iter *iter)
+cifs_copy_bvecq_to_iter(struct bvecq *bq, size_t data_size,
+			size_t skip, struct iov_iter *iter)
 {
-	for (; folioq; folioq = folioq->next) {
-		for (int s = 0; s < folioq_count(folioq); s++) {
-			struct folio *folio;
-			size_t fsize, n, len;
+	for (; bq; bq = bq->next) {
+		for (int s = 0; s < bq->nr_slots; s++) {
+			struct bio_vec *bv = &bq->bv[s];
+			size_t n, len;
 
 			if (data_size == 0)
 				return 0;
 
-			folio = folioq_folio(folioq, s);
-			fsize = folio_size(folio);
-			len = umin(fsize - skip, data_size);
+			len = umin(bv->bv_len - skip, data_size);
 
-			n = copy_folio_to_iter(folio, skip, len, iter);
+			n = copy_page_to_iter(bv->bv_page, bv->bv_offset + skip, len, iter);
 			if (n != len) {
 				cifs_dbg(VFS, "%s: something went wrong\n", __func__);
 				return smb_EIO2(smb_eio_trace_rx_copy_to_iter,
@@ -4879,7 +4875,7 @@ cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,
 
 static int
 handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
-		 char *buf, unsigned int buf_len, struct folio_queue *buffer,
+		 char *buf, unsigned int buf_len, struct bvecq *buffer,
 		 unsigned int buffer_len, bool is_offloaded)
 {
 	unsigned int data_offset;
@@ -4989,8 +4985,8 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 		}
 
 		/* Copy the data to the output I/O iterator. */
-		rdata->result = cifs_copy_folioq_to_iter(buffer, data_len,
-							 cur_off, &rdata->subreq.io_iter);
+		rdata->result = cifs_copy_bvecq_to_iter(buffer, data_len,
+							cur_off, &rdata->subreq.io_iter);
 		if (rdata->result != 0) {
 			if (is_offloaded)
 				mid->mid_state = MID_RESPONSE_MALFORMED;
@@ -5029,7 +5025,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 struct smb2_decrypt_work {
 	struct work_struct decrypt;
 	struct TCP_Server_Info *server;
-	struct folio_queue *buffer;
+	struct bvecq *buffer;
 	char *buf;
 	unsigned int len;
 };
@@ -5043,7 +5039,7 @@ static void smb2_decrypt_offload(struct work_struct *work)
 	struct mid_q_entry *mid;
 	struct iov_iter iter;
 
-	iov_iter_folio_queue(&iter, ITER_DEST, dw->buffer, 0, 0, dw->len);
+	iov_iter_bvec_queue(&iter, ITER_DEST, dw->buffer, 0, 0, dw->len);
 	rc = decrypt_raw_data(dw->server, dw->buf, dw->server->vals->read_rsp_size,
 			      &iter, true);
 	if (rc) {
@@ -5092,7 +5088,7 @@ static void smb2_decrypt_offload(struct work_struct *work)
 	}
 
 free_pages:
-	netfs_free_folioq_buffer(dw->buffer);
+	bvecq_put(dw->buffer);
 	cifs_small_buf_release(dw->buf);
 	kfree(dw);
 }
@@ -5138,12 +5134,12 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
 	dw->len = len;
 	len = round_up(dw->len, PAGE_SIZE);
 
-	size_t cur_size = 0;
-	rc = netfs_alloc_folioq_buffer(NULL, &dw->buffer, &cur_size, len, GFP_NOFS);
-	if (rc < 0)
+	rc = -ENOMEM;
+	dw->buffer = bvecq_alloc_buffer(len, GFP_NOFS);
+	if (!dw->buffer)
 		goto discard_data;
 
-	iov_iter_folio_queue(&iter, ITER_DEST, dw->buffer, 0, 0, len);
+	iov_iter_bvec_queue(&iter, ITER_DEST, dw->buffer, 0, 0, len);
 
 	/* Read the data into the buffer and clear excess bufferage. */
 	rc = cifs_read_iter_from_socket(server, &iter, dw->len);
@@ -5201,7 +5197,7 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
 	}
 
 free_pages:
-	netfs_free_folioq_buffer(dw->buffer);
+	bvecq_put(dw->buffer);
 free_dw:
 	kfree(dw);
 	return rc;



From dhowells at redhat.com  Tue Aug  4 03:02:06 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:06 +0100
Subject: [PATCH v8 11/25] smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-12-dhowells@redhat.com>

Add support for ITER_BVECQ to smbdirect_map_sges_from_iter().

Signed-off-by: David Howells <dhowells at redhat.com>
Acked-by: Stefan Metzmacher <metze at samba.org>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: Shyam Prasad N <sprasad at microsoft.com>
cc: Tom Talpey <tom at talpey.com>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/smb/smbdirect/connection.c | 67 +++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/fs/smb/smbdirect/connection.c b/fs/smb/smbdirect/connection.c
index 8adf58097534..f5e7e75dd809 100644
--- a/fs/smb/smbdirect/connection.c
+++ b/fs/smb/smbdirect/connection.c
@@ -5,6 +5,7 @@
  */
 
 #include "internal.h"
+#include <linux/bvecq.h>
 #include <linux/folio_queue.h>
 
 struct smbdirect_map_sges {
@@ -2006,6 +2007,69 @@ static ssize_t smbdirect_map_sges_from_bvec(struct iov_iter *iter,
 	return ret;
 }
 
+/*
+ * Extract memory fragments from a BVECQ-class iterator and add them to an RDMA
+ * list.  The fragments are not pinned.
+ */
+static ssize_t smbdirect_map_sges_from_bvecq(struct iov_iter *iter,
+					     struct smbdirect_map_sges *state,
+					     ssize_t maxsize)
+{
+	const struct bvecq *bq = iter->bvecq, *next;
+	unsigned int slot = iter->bvecq_slot;
+	ssize_t extracted = 0;
+	size_t offset = iter->iov_offset;
+
+	maxsize = umin(maxsize, iov_iter_count(iter));
+
+	do {
+		struct bio_vec *bv;
+		size_t bsize;
+
+		while (slot >= bq->nr_slots) {
+			next = bvecq_next(bq);
+			if (!next) {
+				if (WARN_ON_ONCE(maxsize > 0))
+					return -EIO;
+				goto out;
+			}
+			bq = next;
+			slot = 0;
+		}
+
+		bv = &bq->bv[slot];
+		bsize = bv->bv_len;
+
+		if (offset < bsize) {
+			size_t part = umin(maxsize, bsize - offset);
+			bool ok;
+
+			ok = smbdirect_map_sges_single_page(state,
+							    bv->bv_page,
+							    bv->bv_offset + offset,
+							    part);
+			if (!ok)
+				return -EIO;
+
+			offset += part;
+			extracted += part;
+			maxsize -= part;
+		}
+
+		if (offset >= bsize) {
+			offset = 0;
+			slot++;
+		}
+	} while (state->num_sge < state->max_sge && maxsize > 0);
+
+out:
+	iter->bvecq = bq;
+	iter->bvecq_slot = slot;
+	iter->iov_offset = offset;
+	iter->count -= extracted;
+	return extracted;
+}
+
 /*
  * Extract fragments from a KVEC-class iterator and add them to an ib_sge list.
  * This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers.
@@ -2155,6 +2219,9 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,
 	case ITER_BVEC:
 		ret = smbdirect_map_sges_from_bvec(iter, state, len);
 		break;
+	case ITER_BVECQ:
+		ret = smbdirect_map_sges_from_bvecq(iter, state, len);
+		break;
 	case ITER_KVEC:
 		ret = smbdirect_map_sges_from_kvec(iter, state, len);
 		break;



From dhowells at redhat.com  Tue Aug  4 03:02:07 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:07 +0100
Subject: [PATCH v8 12/25] netfs: Remove the writethrough code
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-13-dhowells@redhat.com>

Remove the netfs writethrough code as it's very tricky to get the locking
right and it will probably deadlock if used in conjunction with Ceph snapshots
because it excludes writeback for the duration, but to flush out old
snapshots, it does a synchronous flush that invokes writeback.

Instead, O_SYNC writes do a flush after performing the write - which is
already there as the callers of netfs_perform_write() all call
generic_write_sync().

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/9p/vfs_addr.c             |   1 -
 fs/afs/file.c                |   1 -
 fs/netfs/buffered_write.c    |  56 ++-----------------
 fs/netfs/internal.h          |   7 ---
 fs/netfs/main.c              |   1 -
 fs/netfs/stats.c             |   4 +-
 fs/netfs/write_collect.c     |   2 -
 fs/netfs/write_issue.c       | 104 +----------------------------------
 include/linux/netfs.h        |   1 -
 include/trace/events/netfs.h |   8 +--
 10 files changed, 9 insertions(+), 176 deletions(-)

diff --git a/fs/9p/vfs_addr.c b/fs/9p/vfs_addr.c
index 1ac0b3dcc077..2129fcb0f65c 100644
--- a/fs/9p/vfs_addr.c
+++ b/fs/9p/vfs_addr.c
@@ -124,7 +124,6 @@ static int v9fs_init_request(struct netfs_io_request *rreq, struct file *file)
 	struct p9_fid *fid;
 	struct dentry *dentry;
 	bool writing = (rreq->origin == NETFS_READ_FOR_WRITE ||
-			rreq->origin == NETFS_WRITETHROUGH ||
 			rreq->origin == NETFS_UNBUFFERED_WRITE ||
 			rreq->origin == NETFS_DIO_WRITE);
 
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 0467742bfeee..3617c245f319 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -400,7 +400,6 @@ static int afs_init_request(struct netfs_io_request *rreq, struct file *file)
 		}
 		break;
 	case NETFS_WRITEBACK:
-	case NETFS_WRITETHROUGH:
 	case NETFS_UNBUFFERED_WRITE:
 	case NETFS_DIO_WRITE:
 		if (S_ISREG(rreq->inode->i_mode))
diff --git a/fs/netfs/buffered_write.c b/fs/netfs/buffered_write.c
index 2cdb68e6b16f..39f00b326e0f 100644
--- a/fs/netfs/buffered_write.c
+++ b/fs/netfs/buffered_write.c
@@ -91,44 +91,14 @@ ssize_t netfs_perform_write(struct kiocb *iocb, struct iov_iter *iter,
 	struct inode *inode = file_inode(file);
 	struct address_space *mapping = inode->i_mapping;
 	struct netfs_inode *ctx = netfs_inode(inode);
-	struct writeback_control wbc = {
-		.sync_mode	= WB_SYNC_NONE,
-		.for_sync	= true,
-		.nr_to_write	= LONG_MAX,
-		.range_start	= iocb->ki_pos,
-		.range_end	= iocb->ki_pos + iter->count,
-	};
-	struct netfs_io_request *wreq = NULL;
-	struct folio *folio = NULL, *writethrough = NULL;
+	struct folio *folio = NULL;
 	unsigned int bdp_flags = (iocb->ki_flags & IOCB_NOWAIT) ? BDP_ASYNC : 0;
-	ssize_t written = 0, ret, ret2;
+	ssize_t written = 0, ret;
 	loff_t pos = iocb->ki_pos;
 	size_t max_chunk = mapping_max_folio_size(mapping);
 	bool maybe_trouble = false;
 
-	if (unlikely(iocb->ki_flags & (IOCB_DSYNC | IOCB_SYNC))
-	    ) {
-		wbc_attach_fdatawrite_inode(&wbc, mapping->host);
-
-		ret = filemap_write_and_wait_range(mapping, pos, pos + iter->count);
-		if (ret < 0) {
-			wbc_detach_inode(&wbc);
-			goto out;
-		}
-
-		wreq = netfs_begin_writethrough(iocb, iter->count);
-		if (IS_ERR(wreq)) {
-			wbc_detach_inode(&wbc);
-			ret = PTR_ERR(wreq);
-			wreq = NULL;
-			goto out;
-		}
-		if (!is_sync_kiocb(iocb))
-			wreq->iocb = iocb;
-		netfs_stat(&netfs_n_wh_writethrough);
-	} else {
-		netfs_stat(&netfs_n_wh_buffered_write);
-	}
+	netfs_stat(&netfs_n_wh_buffered_write);
 
 	do {
 		enum netfs_folio_trace trace;
@@ -390,15 +360,8 @@ ssize_t netfs_perform_write(struct kiocb *iocb, struct iov_iter *iter,
 		pos += copied;
 		written += copied;
 
-		if (likely(!wreq)) {
-			folio_mark_dirty(folio);
-			folio_unlock(folio);
-		} else {
-			netfs_advance_writethrough(wreq, &wbc, folio, copied,
-						   offset + copied == flen,
-						   &writethrough);
-			/* Folio unlocked */
-		}
+		folio_mark_dirty(folio);
+		folio_unlock(folio);
 	retry:
 		folio_put(folio);
 		folio = NULL;
@@ -420,15 +383,6 @@ ssize_t netfs_perform_write(struct kiocb *iocb, struct iov_iter *iter,
 			ctx->ops->post_modify(inode);
 	}
 
-	if (unlikely(wreq)) {
-		ret2 = netfs_end_writethrough(wreq, &wbc, writethrough);
-		wbc_detach_inode(&wbc);
-		if (ret2 == -EIOCBQUEUED)
-			return ret2;
-		if (ret == 0 && ret2 < 0)
-			ret = ret2;
-	}
-
 	iocb->ki_pos += written;
 	_leave(" = %zd [%zd]", written, ret);
 	return written ? written : ret;
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index c68eea2ecc60..5aea8d34adcf 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -156,7 +156,6 @@ extern atomic_t netfs_n_rh_write_zskip;
 extern atomic_t netfs_n_rh_retry_read_req;
 extern atomic_t netfs_n_rh_retry_read_subreq;
 extern atomic_t netfs_n_wh_buffered_write;
-extern atomic_t netfs_n_wh_writethrough;
 extern atomic_t netfs_n_wh_dio_write;
 extern atomic_t netfs_n_wh_writepages;
 extern atomic_t netfs_n_wh_copy_to_cache;
@@ -216,12 +215,6 @@ void netfs_issue_write(struct netfs_io_request *wreq,
 size_t netfs_advance_write(struct netfs_io_request *wreq,
 			   struct netfs_io_stream *stream,
 			   loff_t start, size_t len, bool to_eof);
-struct netfs_io_request *netfs_begin_writethrough(struct kiocb *iocb, size_t len);
-int netfs_advance_writethrough(struct netfs_io_request *wreq, struct writeback_control *wbc,
-			       struct folio *folio, size_t copied, bool to_page_end,
-			       struct folio **writethrough_cache);
-ssize_t netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_control *wbc,
-			       struct folio *writethrough_cache);
 
 /*
  * write_retry.c
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 9f72e5054aff..3908ea2422c8 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -45,7 +45,6 @@ static const char *netfs_origins[nr__netfs_io_origin] = {
 	[NETFS_DIO_READ]		= "DR",
 	[NETFS_WRITEBACK]		= "WB",
 	[NETFS_WRITEBACK_SINGLE]	= "W1",
-	[NETFS_WRITETHROUGH]		= "WT",
 	[NETFS_UNBUFFERED_WRITE]	= "UW",
 	[NETFS_DIO_WRITE]		= "DW",
 	[NETFS_PGPRIV2_COPY_TO_CACHE]	= "2C",
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
index 84c2a4bcc762..a10d34f88597 100644
--- a/fs/netfs/stats.c
+++ b/fs/netfs/stats.c
@@ -32,7 +32,6 @@ atomic_t netfs_n_rh_write_zskip;
 atomic_t netfs_n_rh_retry_read_req;
 atomic_t netfs_n_rh_retry_read_subreq;
 atomic_t netfs_n_wh_buffered_write;
-atomic_t netfs_n_wh_writethrough;
 atomic_t netfs_n_wh_dio_write;
 atomic_t netfs_n_wh_writepages;
 atomic_t netfs_n_wh_copy_to_cache;
@@ -59,9 +58,8 @@ int netfs_stats_show(struct seq_file *m, void *v)
 		   atomic_read(&netfs_n_rh_read_single),
 		   atomic_read(&netfs_n_rh_write_begin),
 		   atomic_read(&netfs_n_rh_write_zskip));
-	seq_printf(m, "Writes : BW=%u WT=%u DW=%u WP=%u 2C=%u\n",
+	seq_printf(m, "Writes : BW=%u DW=%u WP=%u 2C=%u\n",
 		   atomic_read(&netfs_n_wh_buffered_write),
-		   atomic_read(&netfs_n_wh_writethrough),
 		   atomic_read(&netfs_n_wh_dio_write),
 		   atomic_read(&netfs_n_wh_writepages),
 		   atomic_read(&netfs_n_wh_copy_to_cache));
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 210eb8f3958d..2eac588e37de 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -214,7 +214,6 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 	smp_rmb();
 	collected_to = ULLONG_MAX;
 	if (wreq->origin == NETFS_WRITEBACK ||
-	    wreq->origin == NETFS_WRITETHROUGH ||
 	    wreq->origin == NETFS_PGPRIV2_COPY_TO_CACHE)
 		notes = NEED_UNLOCK;
 	else
@@ -411,7 +410,6 @@ bool netfs_write_collection(struct netfs_io_request *wreq)
 	switch (wreq->origin) {
 	case NETFS_WRITEBACK:
 	case NETFS_WRITEBACK_SINGLE:
-	case NETFS_WRITETHROUGH:
 		netfs_wb_end(ictx);
 		break;
 	default:
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index a9f24b47f4a8..5590ffe56750 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -96,7 +96,6 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 	struct netfs_inode *ictx;
 	bool is_cacheable = (origin == NETFS_WRITEBACK ||
 			     origin == NETFS_WRITEBACK_SINGLE ||
-			     origin == NETFS_WRITETHROUGH ||
 			     origin == NETFS_PGPRIV2_COPY_TO_CACHE);
 
 	wreq = netfs_alloc_request(mapping, file, start, 0, origin);
@@ -365,11 +364,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 		streamw = true;
 	}
 
-	if (wreq->origin == NETFS_WRITETHROUGH) {
-		to_eof = false;
-		if (flen > i_size - fpos)
-			flen = i_size - fpos;
-	} else if (flen > i_size - fpos) {
+	if (flen > i_size - fpos) {
 		flen = i_size - fpos;
 		if (!streamw)
 			folio_zero_segment(folio, flen, fsize);
@@ -611,103 +606,6 @@ int netfs_writepages(struct address_space *mapping,
 }
 EXPORT_SYMBOL(netfs_writepages);
 
-/*
- * Begin a write operation for writing through the pagecache.
- */
-struct netfs_io_request *netfs_begin_writethrough(struct kiocb *iocb, size_t len)
-{
-	struct netfs_io_request *wreq = NULL;
-	struct netfs_inode *ictx = netfs_inode(file_inode(iocb->ki_filp));
-
-	netfs_wb_begin(ictx, false);
-
-	wreq = netfs_create_write_req(iocb->ki_filp->f_mapping, iocb->ki_filp,
-				      iocb->ki_pos, NETFS_WRITETHROUGH);
-	if (IS_ERR(wreq)) {
-		netfs_wb_end(ictx);
-		return wreq;
-	}
-
-	wreq->io_streams[0].avail = true;
-	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
-	trace_netfs_write(wreq, netfs_write_trace_writethrough);
-	return wreq;
-}
-
-/*
- * Advance the state of the write operation used when writing through the
- * pagecache.  Data has been copied into the pagecache that we need to append
- * to the request.  If we've added more than wsize then we need to create a new
- * subrequest.
- */
-int netfs_advance_writethrough(struct netfs_io_request *wreq, struct writeback_control *wbc,
-			       struct folio *folio, size_t copied, bool to_page_end,
-			       struct folio **writethrough_cache)
-{
-	int ret;
-
-	_enter("R=%x ic=%zu ws=%u cp=%zu tp=%u",
-	       wreq->debug_id, wreq->buffer.iter.count, wreq->wsize, copied, to_page_end);
-
-	/* The folio is locked. */
-
-	if (*writethrough_cache != folio) {
-		if (*writethrough_cache) {
-			/* Did the folio get moved? */
-			folio_put(*writethrough_cache);
-			*writethrough_cache = NULL;
-		}
-		/* We can make multiple writes to the folio... */
-		if (wreq->len == 0)
-			trace_netfs_folio(folio, netfs_folio_trace_wthru);
-		else
-			trace_netfs_folio(folio, netfs_folio_trace_wthru_plus);
-		*writethrough_cache = folio;
-		folio_get(folio);
-	}
-
-	wreq->len += copied;
-
-	if (!to_page_end) {
-		folio_mark_dirty(folio);
-		folio_unlock(folio);
-		return 0;
-	}
-
-	ret = netfs_write_folio(wreq, wbc, folio);
-	folio_put(*writethrough_cache);
-	*writethrough_cache = NULL;
-	wreq->submitted = wreq->len;
-	return ret;
-}
-
-/*
- * End a write operation used when writing through the pagecache.
- */
-ssize_t netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_control *wbc,
-			       struct folio *writethrough_cache)
-{
-	ssize_t ret;
-
-	_enter("R=%x", wreq->debug_id);
-
-	if (writethrough_cache) {
-		folio_lock(writethrough_cache);
-		netfs_write_folio(wreq, wbc, writethrough_cache);
-		folio_put(writethrough_cache);
-		wreq->submitted = wreq->len;
-	}
-
-	netfs_end_issue_write(wreq);
-
-	if (wreq->iocb)
-		ret = -EIOCBQUEUED;
-	else
-		ret = netfs_wait_for_write(wreq);
-	netfs_put_request(wreq, netfs_rreq_trace_put_return);
-	return ret;
-}
-
 /**
  * netfs_writeback_single - Write back a monolithic payload
  * @mapping: The mapping to write from
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index d22fcae8d67d..4b071d451d4b 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -210,7 +210,6 @@ enum netfs_io_origin {
 	NETFS_DIO_READ,			/* This is a direct I/O read */
 	NETFS_WRITEBACK,		/* This write was triggered by writepages */
 	NETFS_WRITEBACK_SINGLE,		/* This monolithic write was triggered by writepages */
-	NETFS_WRITETHROUGH,		/* This write was made by netfs_perform_write() */
 	NETFS_UNBUFFERED_WRITE,		/* This is an unbuffered write */
 	NETFS_DIO_WRITE,		/* This is a direct I/O write */
 	NETFS_PGPRIV2_COPY_TO_CACHE,	/* [DEPRECATED] This is writing read data to the cache */
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 5354a0439324..2ce162baaad1 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -30,8 +30,7 @@
 	EM(netfs_write_trace_dio_write,		"DIO-WRITE")	\
 	EM(netfs_write_trace_unbuffered_write,	"UNB-WRITE")	\
 	EM(netfs_write_trace_writeback,		"WRITEBACK")	\
-	EM(netfs_write_trace_writeback_single,	"WB-SINGLE") \
-	E_(netfs_write_trace_writethrough,	"WRITETHRU")
+	E_(netfs_write_trace_writeback_single,	"WB-SINGLE")
 
 #define netfs_rreq_origins					\
 	EM(NETFS_READAHEAD,			"RA")		\
@@ -43,7 +42,6 @@
 	EM(NETFS_DIO_READ,			"DR")		\
 	EM(NETFS_WRITEBACK,			"WB")		\
 	EM(NETFS_WRITEBACK_SINGLE,		"W1")		\
-	EM(NETFS_WRITETHROUGH,			"WT")		\
 	EM(NETFS_UNBUFFERED_WRITE,		"UW")		\
 	EM(NETFS_DIO_WRITE,			"DW")		\
 	E_(NETFS_PGPRIV2_COPY_TO_CACHE,		"2C")
@@ -221,9 +219,7 @@
 	EM(netfs_folio_trace_redirtied,		"redirtied")	\
 	EM(netfs_folio_trace_store,		"store")	\
 	EM(netfs_folio_trace_store_copy,	"store-copy")	\
-	EM(netfs_folio_trace_store_plus,	"store+")	\
-	EM(netfs_folio_trace_wthru,		"wthru")	\
-	E_(netfs_folio_trace_wthru_plus,	"wthru+")
+	E_(netfs_folio_trace_store_plus,	"store+")
 
 #define netfs_collect_contig_traces				\
 	EM(netfs_contig_trace_collect,		"Collect")	\



From dhowells at redhat.com  Tue Aug  4 03:02:08 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:08 +0100
Subject: [PATCH v8 13/25] cachefiles,netfs: sunset ondemand mode
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-14-dhowells@redhat.com>

From: Gao Xiang <xiang at kernel.org>

It was an effort to enhance fscache as a kernel cache for lazy
pulling (at least according to previous Incremental FS discussion [1])
and EROFS over fscache was the in-tree user of this mode.

fscache has since evolved to be netfslib-oriented, serving network
filesystem inodes via the netfs library, but EROFS never acts as a
network filesystem and we need to cache golden filesystem images rather
than individual EROFS inodes.

Since EROFS over fscache is now removed, clean up netfs/fscache/
cachefiles upstream too.

[1] https://lore.kernel.org/r/CAOQ4uxi4dzxArY24YO=+kBCK2gGoq3Ptb8WkzCqSogPgU_R3dQ at mail.gmail.com

[dh] Fixed up comments on:
https://sashiko.dev/#/patchset/20260716103030.3065561-1-dhowells%40redhat.com
https://sashiko.dev/#/patchset/20260722130218.78958-1-dhowells%40redhat.com

Signed-off-by: Gao Xiang <xiang at kernel.org>
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-erofs at lists.ozlabs.org
cc: bpf at vger.kernel.org
cc: linux-fsdevel at vger.kernel.org
---
 .../filesystems/caching/cachefiles.rst        | 179 ----
 fs/cachefiles/Kconfig                         |  12 -
 fs/cachefiles/Makefile                        |   1 -
 fs/cachefiles/daemon.c                        |  76 +-
 fs/cachefiles/interface.c                     |   8 -
 fs/cachefiles/internal.h                      | 130 ---
 fs/cachefiles/io.c                            |  26 +-
 fs/cachefiles/namei.c                         |   8 -
 fs/cachefiles/ondemand.c                      | 761 ------------------
 include/linux/netfs.h                         |   9 -
 include/trace/events/cachefiles.h             | 184 +----
 include/uapi/linux/cachefiles.h               |  68 --
 kernel/bpf/btf.c                              |   7 -
 13 files changed, 6 insertions(+), 1463 deletions(-)
 delete mode 100644 fs/cachefiles/ondemand.c
 delete mode 100644 include/uapi/linux/cachefiles.h

diff --git a/Documentation/filesystems/caching/cachefiles.rst b/Documentation/filesystems/caching/cachefiles.rst
index b3ccc782cb3b..adfb7d079027 100644
--- a/Documentation/filesystems/caching/cachefiles.rst
+++ b/Documentation/filesystems/caching/cachefiles.rst
@@ -28,8 +28,6 @@ Cache on Already Mounted Filesystem
 
  (*) Debugging.
 
- (*) On-demand Read.
-
 
 Overview
 ========
@@ -483,180 +481,3 @@ the control file.  For example::
 	echo $((1|4|8)) >/sys/module/cachefiles/parameters/debug
 
 will turn on all function entry debugging.
-
-
-On-demand Read
-==============
-
-When working in its original mode, CacheFiles serves as a local cache for a
-remote networking fs - while in on-demand read mode, CacheFiles can boost the
-scenario where on-demand read semantics are needed, e.g. container image
-distribution.
-
-The essential difference between these two modes is seen when a cache miss
-occurs: In the original mode, the netfs will fetch the data from the remote
-server and then write it to the cache file; in on-demand read mode, fetching
-the data and writing it into the cache is delegated to a user daemon.
-
-``CONFIG_CACHEFILES_ONDEMAND`` should be enabled to support on-demand read mode.
-
-
-Protocol Communication
-----------------------
-
-The on-demand read mode uses a simple protocol for communication between kernel
-and user daemon. The protocol can be modeled as::
-
-	kernel --[request]--> user daemon --[reply]--> kernel
-
-CacheFiles will send requests to the user daemon when needed.  The user daemon
-should poll the devnode ('/dev/cachefiles') to check if there's a pending
-request to be processed.  A POLLIN event will be returned when there's a pending
-request.
-
-The user daemon then reads the devnode to fetch a request to process.  It should
-be noted that each read only gets one request. When it has finished processing
-the request, the user daemon should write the reply to the devnode.
-
-Each request starts with a message header of the form::
-
-	struct cachefiles_msg {
-		__u32 msg_id;
-		__u32 opcode;
-		__u32 len;
-		__u32 object_id;
-		__u8  data[];
-	};
-
-where:
-
-	* ``msg_id`` is a unique ID identifying this request among all pending
-	  requests.
-
-	* ``opcode`` indicates the type of this request.
-
-	* ``object_id`` is a unique ID identifying the cache file operated on.
-
-	* ``data`` indicates the payload of this request.
-
-	* ``len`` indicates the whole length of this request, including the
-	  header and following type-specific payload.
-
-
-Turning on On-demand Mode
--------------------------
-
-An optional parameter becomes available to the "bind" command::
-
-	bind [ondemand]
-
-When the "bind" command is given no argument, it defaults to the original mode.
-When it is given the "ondemand" argument, i.e. "bind ondemand", on-demand read
-mode will be enabled.
-
-
-The OPEN Request
-----------------
-
-When the netfs opens a cache file for the first time, a request with the
-CACHEFILES_OP_OPEN opcode, a.k.a an OPEN request will be sent to the user
-daemon.  The payload format is of the form::
-
-	struct cachefiles_open {
-		__u32 volume_key_size;
-		__u32 cookie_key_size;
-		__u32 fd;
-		__u32 flags;
-		__u8  data[];
-	};
-
-where:
-
-	* ``data`` contains the volume_key followed directly by the cookie_key.
-	  The volume key is a NUL-terminated string; the cookie key is binary
-	  data.
-
-	* ``volume_key_size`` indicates the size of the volume key in bytes.
-
-	* ``cookie_key_size`` indicates the size of the cookie key in bytes.
-
-	* ``fd`` indicates an anonymous fd referring to the cache file, through
-	  which the user daemon can perform write/llseek file operations on the
-	  cache file.
-
-
-The user daemon can use the given (volume_key, cookie_key) pair to distinguish
-the requested cache file.  With the given anonymous fd, the user daemon can
-fetch the data and write it to the cache file in the background, even when
-kernel has not triggered a cache miss yet.
-
-Be noted that each cache file has a unique object_id, while it may have multiple
-anonymous fds.  The user daemon may duplicate anonymous fds from the initial
-anonymous fd indicated by the @fd field through dup().  Thus each object_id can
-be mapped to multiple anonymous fds, while the usr daemon itself needs to
-maintain the mapping.
-
-When implementing a user daemon, please be careful of RLIMIT_NOFILE,
-``/proc/sys/fs/nr_open`` and ``/proc/sys/fs/file-max``.  Typically these needn't
-be huge since they're related to the number of open device blobs rather than
-open files of each individual filesystem.
-
-The user daemon should reply the OPEN request by issuing a "copen" (complete
-open) command on the devnode::
-
-	copen <msg_id>,<cache_size>
-
-where:
-
-	* ``msg_id`` must match the msg_id field of the OPEN request.
-
-	* When >= 0, ``cache_size`` indicates the size of the cache file;
-	  when < 0, ``cache_size`` indicates any error code encountered by the
-	  user daemon.
-
-
-The CLOSE Request
------------------
-
-When a cookie withdrawn, a CLOSE request (opcode CACHEFILES_OP_CLOSE) will be
-sent to the user daemon.  This tells the user daemon to close all anonymous fds
-associated with the given object_id.  The CLOSE request has no extra payload,
-and shouldn't be replied.
-
-
-The READ Request
-----------------
-
-When a cache miss is encountered in on-demand read mode, CacheFiles will send a
-READ request (opcode CACHEFILES_OP_READ) to the user daemon. This tells the user
-daemon to fetch the contents of the requested file range.  The payload is of the
-form::
-
-	struct cachefiles_read {
-		__u64 off;
-		__u64 len;
-	};
-
-where:
-
-	* ``off`` indicates the starting offset of the requested file range.
-
-	* ``len`` indicates the length of the requested file range.
-
-
-When it receives a READ request, the user daemon should fetch the requested data
-and write it to the cache file identified by object_id.
-
-When it has finished processing the READ request, the user daemon should reply
-by using the CACHEFILES_IOC_READ_COMPLETE ioctl on one of the anonymous fds
-associated with the object_id given in the READ request.  The ioctl is of the
-form::
-
-	ioctl(fd, CACHEFILES_IOC_READ_COMPLETE, msg_id);
-
-where:
-
-	* ``fd`` is one of the anonymous fds associated with the object_id
-	  given.
-
-	* ``msg_id`` must match the msg_id field of the READ request.
diff --git a/fs/cachefiles/Kconfig b/fs/cachefiles/Kconfig
index c5a070550ee3..afb25b6af5aa 100644
--- a/fs/cachefiles/Kconfig
+++ b/fs/cachefiles/Kconfig
@@ -26,15 +26,3 @@ config CACHEFILES_ERROR_INJECTION
 	help
 	  This permits error injection to be enabled in cachefiles whilst a
 	  cache is in service.
-
-config CACHEFILES_ONDEMAND
-	bool "Support for on-demand read"
-	depends on CACHEFILES
-	default n
-	help
-	  This permits userspace to enable the cachefiles on-demand read mode.
-	  In this mode, when a cache miss occurs, responsibility for fetching
-	  the data lies with the cachefiles backend instead of with the netfs
-	  and is delegated to userspace.
-
-	  If unsure, say N.
diff --git a/fs/cachefiles/Makefile b/fs/cachefiles/Makefile
index c37a7a9af10b..16d811f1a2fa 100644
--- a/fs/cachefiles/Makefile
+++ b/fs/cachefiles/Makefile
@@ -16,6 +16,5 @@ cachefiles-y := \
 	xattr.o
 
 cachefiles-$(CONFIG_CACHEFILES_ERROR_INJECTION) += error_inject.o
-cachefiles-$(CONFIG_CACHEFILES_ONDEMAND) += ondemand.o
 
 obj-$(CONFIG_CACHEFILES) := cachefiles.o
diff --git a/fs/cachefiles/daemon.c b/fs/cachefiles/daemon.c
index 4117b145ac94..6e0248a62c13 100644
--- a/fs/cachefiles/daemon.c
+++ b/fs/cachefiles/daemon.c
@@ -76,10 +76,6 @@ static const struct cachefiles_daemon_cmd cachefiles_daemon_cmds[] = {
 	{ "inuse",	cachefiles_daemon_inuse		},
 	{ "secctx",	cachefiles_daemon_secctx	},
 	{ "tag",	cachefiles_daemon_tag		},
-#ifdef CONFIG_CACHEFILES_ONDEMAND
-	{ "copen",	cachefiles_ondemand_copen	},
-	{ "restore",	cachefiles_ondemand_restore	},
-#endif
 	{ "",		NULL				}
 };
 
@@ -114,8 +110,6 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
 	INIT_LIST_HEAD(&cache->object_list);
 	spin_lock_init(&cache->object_list_lock);
 	refcount_set(&cache->unbind_pincount, 1);
-	xa_init_flags(&cache->reqs, XA_FLAGS_ALLOC);
-	xa_init_flags(&cache->ondemand_ids, XA_FLAGS_ALLOC1);
 
 	/* set default caching limits
 	 * - limit at 1% free space and/or free files
@@ -134,40 +128,6 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
 	return 0;
 }
 
-void cachefiles_flush_reqs(struct cachefiles_cache *cache)
-{
-	struct xarray *xa = &cache->reqs;
-	struct cachefiles_req *req;
-	unsigned long index;
-
-	/*
-	 * Make sure the following two operations won't be reordered.
-	 *   1) set CACHEFILES_DEAD bit
-	 *   2) flush requests in the xarray
-	 * Otherwise the request may be enqueued after xarray has been
-	 * flushed, leaving the orphan request never being completed.
-	 *
-	 * CPU 1			CPU 2
-	 * =====			=====
-	 * flush requests in the xarray
-	 *				test CACHEFILES_DEAD bit
-	 *				enqueue the request
-	 * set CACHEFILES_DEAD bit
-	 */
-	smp_mb();
-
-	xa_lock(xa);
-	xa_for_each(xa, index, req) {
-		req->error = -EIO;
-		complete(&req->done);
-		__xa_erase(xa, index);
-	}
-	xa_unlock(xa);
-
-	xa_destroy(&cache->reqs);
-	xa_destroy(&cache->ondemand_ids);
-}
-
 void cachefiles_put_unbind_pincount(struct cachefiles_cache *cache)
 {
 	if (refcount_dec_and_test(&cache->unbind_pincount)) {
@@ -195,9 +155,6 @@ static int cachefiles_daemon_release(struct inode *inode, struct file *file)
 
 	set_bit(CACHEFILES_DEAD, &cache->flags);
 
-	if (cachefiles_in_ondemand_mode(cache))
-		cachefiles_flush_reqs(cache);
-
 	/* clean up the control file interface */
 	cache->cachefilesd = NULL;
 	file->private_data = NULL;
@@ -266,10 +223,7 @@ static ssize_t cachefiles_daemon_read(struct file *file, char __user *_buffer,
 	if (!test_bit(CACHEFILES_READY, &cache->flags))
 		return 0;
 
-	if (cachefiles_in_ondemand_mode(cache))
-		return cachefiles_ondemand_daemon_read(cache, _buffer, buflen);
-	else
-		return cachefiles_do_daemon_read(cache, _buffer, buflen);
+	return cachefiles_do_daemon_read(cache, _buffer, buflen);
 }
 
 /*
@@ -358,28 +312,13 @@ static __poll_t cachefiles_daemon_poll(struct file *file,
 					   struct poll_table_struct *poll)
 {
 	struct cachefiles_cache *cache = file->private_data;
-	XA_STATE(xas, &cache->reqs, 0);
-	struct cachefiles_req *req;
 	__poll_t mask;
 
 	poll_wait(file, &cache->daemon_pollwq, poll);
 	mask = 0;
 
-	if (cachefiles_in_ondemand_mode(cache)) {
-		if (!xa_empty(&cache->reqs)) {
-			xas_lock(&xas);
-			xas_for_each_marked(&xas, req, ULONG_MAX, CACHEFILES_REQ_NEW) {
-				if (!cachefiles_ondemand_is_reopening_read(req)) {
-					mask |= EPOLLIN;
-					break;
-				}
-			}
-			xas_unlock(&xas);
-		}
-	} else {
-		if (test_bit(CACHEFILES_STATE_CHANGED, &cache->flags))
-			mask |= EPOLLIN;
-	}
+	if (test_bit(CACHEFILES_STATE_CHANGED, &cache->flags))
+		mask |= EPOLLIN;
 
 	if (test_bit(CACHEFILES_CULLING, &cache->flags))
 		mask |= EPOLLOUT;
@@ -779,14 +718,7 @@ static int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args)
 		return -EBUSY;
 	}
 
-	if (IS_ENABLED(CONFIG_CACHEFILES_ONDEMAND)) {
-		if (!strcmp(args, "ondemand")) {
-			set_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags);
-		} else if (*args) {
-			pr_err("Invalid argument to the 'bind' command\n");
-			return -EINVAL;
-		}
-	} else if (*args) {
+	if (*args) {
 		pr_err("'bind' command doesn't take an argument\n");
 		return -EINVAL;
 	}
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index a08250d244ea..50a000310a8c 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -32,11 +32,6 @@ struct cachefiles_object *cachefiles_alloc_object(struct fscache_cookie *cookie)
 	if (!object)
 		return NULL;
 
-	if (cachefiles_ondemand_init_obj_info(object, volume)) {
-		kmem_cache_free(cachefiles_object_jar, object);
-		return NULL;
-	}
-
 	refcount_set(&object->ref, 1);
 
 	spin_lock_init(&object->lock);
@@ -94,7 +89,6 @@ void cachefiles_put_object(struct cachefiles_object *object,
 		ASSERTCMP(object->file, ==, NULL);
 
 		kfree(object->d_name);
-		cachefiles_ondemand_deinit_obj_info(object);
 		cache = object->volume->cache->cache;
 		fscache_put_cookie(object->cookie, fscache_cookie_put_object);
 		object->cookie = NULL;
@@ -374,8 +368,6 @@ static void cachefiles_withdraw_cookie(struct fscache_cookie *cookie)
 		spin_unlock(&cache->object_list_lock);
 	}
 
-	cachefiles_ondemand_clean_object(object);
-
 	if (object->file) {
 		cachefiles_begin_secure(cache, &saved_cred);
 		cachefiles_clean_up_object(object, cache);
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index b62cd3e9a18e..48ea38ba1bde 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -15,8 +15,6 @@
 #include <linux/fscache-cache.h>
 #include <linux/cred.h>
 #include <linux/security.h>
-#include <linux/xarray.h>
-#include <linux/cachefiles.h>
 
 #define CACHEFILES_DIO_BLOCK_SIZE 4096
 
@@ -44,21 +42,6 @@ struct cachefiles_volume {
 	struct dentry			*fanout[256];	/* Fanout subdirs */
 };
 
-enum cachefiles_object_state {
-	CACHEFILES_ONDEMAND_OBJSTATE_CLOSE, /* Anonymous fd closed by daemon or initial state */
-	CACHEFILES_ONDEMAND_OBJSTATE_OPEN, /* Anonymous fd associated with object is available */
-	CACHEFILES_ONDEMAND_OBJSTATE_REOPENING, /* Object that was closed and is being reopened. */
-	CACHEFILES_ONDEMAND_OBJSTATE_DROPPING, /* Object is being dropped. */
-};
-
-struct cachefiles_ondemand_info {
-	struct work_struct		ondemand_work;
-	int				ondemand_id;
-	enum cachefiles_object_state	state;
-	struct cachefiles_object	*object;
-	spinlock_t			lock;
-};
-
 /*
  * Backing file state.
  */
@@ -74,13 +57,8 @@ struct cachefiles_object {
 	enum cachefiles_content		content_info:8;	/* Info about content presence */
 	unsigned long			flags;
 #define CACHEFILES_OBJECT_USING_TMPFILE	0		/* Have an unlinked tmpfile */
-#ifdef CONFIG_CACHEFILES_ONDEMAND
-	struct cachefiles_ondemand_info	*ondemand;
-#endif
 };
 
-#define CACHEFILES_ONDEMAND_ID_CLOSED	-1
-
 /*
  * Cache files cache definition
  */
@@ -119,35 +97,13 @@ struct cachefiles_cache {
 #define CACHEFILES_DEAD			1	/* T if cache dead */
 #define CACHEFILES_CULLING		2	/* T if cull engaged */
 #define CACHEFILES_STATE_CHANGED	3	/* T if state changed (poll trigger) */
-#define CACHEFILES_ONDEMAND_MODE	4	/* T if in on-demand read mode */
 	char				*rootdirname;	/* name of cache root directory */
 	char				*tag;		/* cache binding tag */
 	refcount_t			unbind_pincount;/* refcount to do daemon unbind */
-	struct xarray			reqs;		/* xarray of pending on-demand requests */
-	unsigned long			req_id_next;
-	struct xarray			ondemand_ids;	/* xarray for ondemand_id allocation */
-	u32				ondemand_id_next;
-	u32				msg_id_next;
 	u32				secid;		/* LSM security id */
 	bool				have_secid;	/* whether "secid" was set */
 };
 
-static inline bool cachefiles_in_ondemand_mode(struct cachefiles_cache *cache)
-{
-	return IS_ENABLED(CONFIG_CACHEFILES_ONDEMAND) &&
-		test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags);
-}
-
-struct cachefiles_req {
-	struct cachefiles_object *object;
-	struct completion done;
-	refcount_t ref;
-	int error;
-	struct cachefiles_msg msg;
-};
-
-#define CACHEFILES_REQ_NEW	XA_MARK_1
-
 #include <trace/events/cachefiles.h>
 
 static inline
@@ -298,90 +254,6 @@ extern struct file *cachefiles_create_tmpfile(struct cachefiles_object *object);
 extern bool cachefiles_commit_tmpfile(struct cachefiles_cache *cache,
 				      struct cachefiles_object *object);
 
-/*
- * ondemand.c
- */
-#ifdef CONFIG_CACHEFILES_ONDEMAND
-extern ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
-					char __user *_buffer, size_t buflen);
-
-extern int cachefiles_ondemand_copen(struct cachefiles_cache *cache,
-				     char *args);
-
-extern int cachefiles_ondemand_restore(struct cachefiles_cache *cache,
-					char *args);
-
-extern int cachefiles_ondemand_init_object(struct cachefiles_object *object);
-extern void cachefiles_ondemand_clean_object(struct cachefiles_object *object);
-
-extern int cachefiles_ondemand_read(struct cachefiles_object *object,
-				    loff_t pos, size_t len);
-
-extern int cachefiles_ondemand_init_obj_info(struct cachefiles_object *obj,
-					struct cachefiles_volume *volume);
-extern void cachefiles_ondemand_deinit_obj_info(struct cachefiles_object *obj);
-
-#define CACHEFILES_OBJECT_STATE_FUNCS(_state, _STATE)	\
-static inline bool								\
-cachefiles_ondemand_object_is_##_state(const struct cachefiles_object *object) \
-{												\
-	return object->ondemand->state == CACHEFILES_ONDEMAND_OBJSTATE_##_STATE; \
-}												\
-												\
-static inline void								\
-cachefiles_ondemand_set_object_##_state(struct cachefiles_object *object) \
-{												\
-	object->ondemand->state = CACHEFILES_ONDEMAND_OBJSTATE_##_STATE; \
-}
-
-CACHEFILES_OBJECT_STATE_FUNCS(open, OPEN);
-CACHEFILES_OBJECT_STATE_FUNCS(close, CLOSE);
-CACHEFILES_OBJECT_STATE_FUNCS(reopening, REOPENING);
-CACHEFILES_OBJECT_STATE_FUNCS(dropping, DROPPING);
-
-static inline bool cachefiles_ondemand_is_reopening_read(struct cachefiles_req *req)
-{
-	return cachefiles_ondemand_object_is_reopening(req->object) &&
-			req->msg.opcode == CACHEFILES_OP_READ;
-}
-
-#else
-static inline ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
-					char __user *_buffer, size_t buflen)
-{
-	return -EOPNOTSUPP;
-}
-
-static inline int cachefiles_ondemand_init_object(struct cachefiles_object *object)
-{
-	return 0;
-}
-
-static inline void cachefiles_ondemand_clean_object(struct cachefiles_object *object)
-{
-}
-
-static inline int cachefiles_ondemand_read(struct cachefiles_object *object,
-					   loff_t pos, size_t len)
-{
-	return -EOPNOTSUPP;
-}
-
-static inline int cachefiles_ondemand_init_obj_info(struct cachefiles_object *obj,
-						struct cachefiles_volume *volume)
-{
-	return 0;
-}
-static inline void cachefiles_ondemand_deinit_obj_info(struct cachefiles_object *obj)
-{
-}
-
-static inline bool cachefiles_ondemand_is_reopening_read(struct cachefiles_req *req)
-{
-	return false;
-}
-#endif
-
 /*
  * security.c
  */
@@ -430,8 +302,6 @@ do {							\
 	pr_err("I/O Error: " FMT"\n", ##__VA_ARGS__);	\
 	fscache_io_error((___cache)->cache);		\
 	set_bit(CACHEFILES_DEAD, &(___cache)->flags);	\
-	if (cachefiles_in_ondemand_mode(___cache))	\
-		cachefiles_flush_reqs(___cache);	\
 } while (0)
 
 #define cachefiles_io_error_obj(object, FMT, ...)			\
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index d879b80a0bed..9540ec25b3cb 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -390,7 +390,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 	size_t len = *_len;
 	loff_t off, to;
 	ino_t ino = file ? file_inode(file)->i_ino : 0;
-	int rc;
 
 	_enter("%zx @%llx/%llx", len, start, i_size);
 
@@ -403,8 +402,7 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 	if (test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags)) {
 		__set_bit(NETFS_SREQ_COPY_TO_CACHE, _flags);
 		why = cachefiles_trace_read_no_data;
-		if (!test_bit(NETFS_SREQ_ONDEMAND, _flags))
-			goto out_no_object;
+		goto out_no_object;
 	}
 
 	/* The object and the file may be being created in the background. */
@@ -421,7 +419,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 	object = cachefiles_cres_object(cres);
 	cache = object->volume->cache;
 	cachefiles_begin_secure(cache, &saved_cred);
-retry:
 	off = cachefiles_inject_read_error();
 	if (off == 0)
 		off = vfs_llseek(file, start, SEEK_DATA);
@@ -474,14 +471,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 
 download_and_store:
 	__set_bit(NETFS_SREQ_COPY_TO_CACHE, _flags);
-	if (test_bit(NETFS_SREQ_ONDEMAND, _flags)) {
-		rc = cachefiles_ondemand_read(object, start, len);
-		if (!rc) {
-			__clear_bit(NETFS_SREQ_ONDEMAND, _flags);
-			goto retry;
-		}
-		ret = NETFS_INVALID_READ;
-	}
 out:
 	cachefiles_end_secure(cache, saved_cred);
 out_no_object:
@@ -501,18 +490,6 @@ static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *
 					  &subreq->flags, subreq->rreq->inode->i_ino);
 }
 
-/*
- * Prepare an on-demand read operation, shortening it to a cached/uncached
- * boundary as appropriate.
- */
-static enum netfs_io_source
-cachefiles_prepare_ondemand_read(struct netfs_cache_resources *cres,
-				 loff_t start, size_t *_len, loff_t i_size,
-				 unsigned long *_flags, ino_t ino)
-{
-	return cachefiles_do_prepare_read(cres, start, _len, i_size, _flags, ino);
-}
-
 /*
  * Prepare for a write to occur.
  */
@@ -731,7 +708,6 @@ static const struct netfs_cache_ops cachefiles_netfs_cache_ops = {
 	.prepare_read		= cachefiles_prepare_read,
 	.prepare_write		= cachefiles_prepare_write,
 	.prepare_write_subreq	= cachefiles_prepare_write_subreq,
-	.prepare_ondemand_read	= cachefiles_prepare_ondemand_read,
 	.query_occupancy	= cachefiles_query_occupancy,
 };
 
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 8a9f6be15828..88955249a1a6 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -442,10 +442,6 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 	if (!cachefiles_mark_inode_in_use(object, file_inode(file)))
 		WARN_ON(1);
 
-	ret = cachefiles_ondemand_init_object(object);
-	if (ret < 0)
-		goto err_unuse;
-
 	ni_size = object->cookie->object_size;
 	ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
 
@@ -546,10 +542,6 @@ static bool cachefiles_open_file(struct cachefiles_object *object,
 	}
 	_debug("file -> %pd positive", dentry);
 
-	ret = cachefiles_ondemand_init_object(object);
-	if (ret < 0)
-		goto error_fput;
-
 	ret = cachefiles_check_auxdata(object, file);
 	if (ret < 0)
 		goto check_failed;
diff --git a/fs/cachefiles/ondemand.c b/fs/cachefiles/ondemand.c
deleted file mode 100644
index 0849eaf583cd..000000000000
--- a/fs/cachefiles/ondemand.c
+++ /dev/null
@@ -1,761 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-#include <linux/anon_inodes.h>
-#include <linux/uio.h>
-#include "internal.h"
-
-struct ondemand_anon_file {
-	struct file *file;
-	int fd;
-};
-
-static inline void cachefiles_req_put(struct cachefiles_req *req)
-{
-	if (refcount_dec_and_test(&req->ref))
-		kfree(req);
-}
-
-static int cachefiles_ondemand_fd_release(struct inode *inode,
-					  struct file *file)
-{
-	struct cachefiles_object *object = file->private_data;
-	struct cachefiles_cache *cache;
-	struct cachefiles_ondemand_info *info;
-	int object_id;
-	struct cachefiles_req *req;
-	XA_STATE(xas, NULL, 0);
-
-	if (!object)
-		return 0;
-
-	info = object->ondemand;
-	cache = object->volume->cache;
-	xas.xa = &cache->reqs;
-
-	xa_lock(&cache->reqs);
-	spin_lock(&info->lock);
-	object_id = info->ondemand_id;
-	info->ondemand_id = CACHEFILES_ONDEMAND_ID_CLOSED;
-	cachefiles_ondemand_set_object_close(object);
-	spin_unlock(&info->lock);
-
-	/* Only flush CACHEFILES_REQ_NEW marked req to avoid race with daemon_read */
-	xas_for_each_marked(&xas, req, ULONG_MAX, CACHEFILES_REQ_NEW) {
-		if (req->msg.object_id == object_id &&
-		    req->msg.opcode == CACHEFILES_OP_CLOSE) {
-			complete(&req->done);
-			xas_store(&xas, NULL);
-		}
-	}
-	xa_unlock(&cache->reqs);
-
-	xa_erase(&cache->ondemand_ids, object_id);
-	trace_cachefiles_ondemand_fd_release(object, object_id);
-	cachefiles_put_object(object, cachefiles_obj_put_ondemand_fd);
-	cachefiles_put_unbind_pincount(cache);
-	return 0;
-}
-
-static ssize_t cachefiles_ondemand_fd_write_iter(struct kiocb *kiocb,
-						 struct iov_iter *iter)
-{
-	struct cachefiles_object *object = kiocb->ki_filp->private_data;
-	struct cachefiles_cache *cache = object->volume->cache;
-	struct file *file;
-	size_t len = iter->count, aligned_len = len;
-	loff_t pos = kiocb->ki_pos;
-	const struct cred *saved_cred;
-	int ret;
-
-	spin_lock(&object->lock);
-	file = object->file;
-	if (!file) {
-		spin_unlock(&object->lock);
-		return -ENOBUFS;
-	}
-	get_file(file);
-	spin_unlock(&object->lock);
-
-	cachefiles_begin_secure(cache, &saved_cred);
-	ret = __cachefiles_prepare_write(object, file, &pos, &aligned_len, len, true);
-	cachefiles_end_secure(cache, saved_cred);
-	if (ret < 0)
-		goto out;
-
-	trace_cachefiles_ondemand_fd_write(object, file_inode(file), pos, len);
-	ret = __cachefiles_write(object, file, pos, iter, NULL, NULL);
-	if (ret > 0)
-		kiocb->ki_pos += ret;
-
-out:
-	fput(file);
-	return ret;
-}
-
-static loff_t cachefiles_ondemand_fd_llseek(struct file *filp, loff_t pos,
-					    int whence)
-{
-	struct cachefiles_object *object = filp->private_data;
-	struct file *file;
-	loff_t ret;
-
-	spin_lock(&object->lock);
-	file = object->file;
-	if (!file) {
-		spin_unlock(&object->lock);
-		return -ENOBUFS;
-	}
-	get_file(file);
-	spin_unlock(&object->lock);
-
-	ret = vfs_llseek(file, pos, whence);
-	fput(file);
-
-	return ret;
-}
-
-static long cachefiles_ondemand_fd_ioctl(struct file *filp, unsigned int ioctl,
-					 unsigned long id)
-{
-	struct cachefiles_object *object = filp->private_data;
-	struct cachefiles_cache *cache = object->volume->cache;
-	struct cachefiles_req *req;
-	XA_STATE(xas, &cache->reqs, id);
-
-	if (ioctl != CACHEFILES_IOC_READ_COMPLETE)
-		return -EINVAL;
-
-	if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
-		return -EOPNOTSUPP;
-
-	xa_lock(&cache->reqs);
-	req = xas_load(&xas);
-	if (!req || req->msg.opcode != CACHEFILES_OP_READ ||
-	    req->object != object) {
-		xa_unlock(&cache->reqs);
-		return -EINVAL;
-	}
-	xas_store(&xas, NULL);
-	xa_unlock(&cache->reqs);
-
-	trace_cachefiles_ondemand_cread(object, id);
-	complete(&req->done);
-	return 0;
-}
-
-static const struct file_operations cachefiles_ondemand_fd_fops = {
-	.owner		= THIS_MODULE,
-	.release	= cachefiles_ondemand_fd_release,
-	.write_iter	= cachefiles_ondemand_fd_write_iter,
-	.llseek		= cachefiles_ondemand_fd_llseek,
-	.unlocked_ioctl	= cachefiles_ondemand_fd_ioctl,
-};
-
-/*
- * OPEN request Completion (copen)
- * - command: "copen <id>,<cache_size>"
- *   <cache_size> indicates the object size if >=0, error code if negative
- */
-int cachefiles_ondemand_copen(struct cachefiles_cache *cache, char *args)
-{
-	struct cachefiles_req *req;
-	struct fscache_cookie *cookie;
-	struct cachefiles_ondemand_info *info;
-	char *pid, *psize;
-	unsigned long id;
-	long size;
-	int ret;
-	XA_STATE(xas, &cache->reqs, 0);
-
-	if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
-		return -EOPNOTSUPP;
-
-	if (!*args) {
-		pr_err("Empty id specified\n");
-		return -EINVAL;
-	}
-
-	pid = args;
-	psize = strchr(args, ',');
-	if (!psize) {
-		pr_err("Cache size is not specified\n");
-		return -EINVAL;
-	}
-
-	*psize = 0;
-	psize++;
-
-	ret = kstrtoul(pid, 0, &id);
-	if (ret)
-		return ret;
-
-	xa_lock(&cache->reqs);
-	xas.xa_index = id;
-	req = xas_load(&xas);
-	if (!req || req->msg.opcode != CACHEFILES_OP_OPEN ||
-	    !req->object->ondemand->ondemand_id) {
-		xa_unlock(&cache->reqs);
-		return -EINVAL;
-	}
-	xas_store(&xas, NULL);
-	xa_unlock(&cache->reqs);
-
-	info = req->object->ondemand;
-	/* fail OPEN request if copen format is invalid */
-	ret = kstrtol(psize, 0, &size);
-	if (ret) {
-		req->error = ret;
-		goto out;
-	}
-
-	/* fail OPEN request if daemon reports an error */
-	if (size < 0) {
-		if (!IS_ERR_VALUE(size)) {
-			req->error = -EINVAL;
-			ret = -EINVAL;
-		} else {
-			req->error = size;
-			ret = 0;
-		}
-		goto out;
-	}
-
-	spin_lock(&info->lock);
-	/*
-	 * The anonymous fd was closed before copen ? Fail the request.
-	 *
-	 *             t1             |             t2
-	 * ---------------------------------------------------------
-	 *                             cachefiles_ondemand_copen
-	 *                             req = xa_erase(&cache->reqs, id)
-	 * // Anon fd is maliciously closed.
-	 * cachefiles_ondemand_fd_release
-	 * xa_lock(&cache->reqs)
-	 * cachefiles_ondemand_set_object_close(object)
-	 * xa_unlock(&cache->reqs)
-	 *                             cachefiles_ondemand_set_object_open
-	 *                             // No one will ever close it again.
-	 * cachefiles_ondemand_daemon_read
-	 * cachefiles_ondemand_select_req
-	 *
-	 * Get a read req but its fd is already closed. The daemon can't
-	 * issue a cread ioctl with an closed fd, then hung.
-	 */
-	if (info->ondemand_id == CACHEFILES_ONDEMAND_ID_CLOSED) {
-		spin_unlock(&info->lock);
-		req->error = -EBADFD;
-		goto out;
-	}
-	cookie = req->object->cookie;
-	cookie->object_size = size;
-	if (size)
-		clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags);
-	else
-		set_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags);
-	trace_cachefiles_ondemand_copen(req->object, id, size);
-
-	cachefiles_ondemand_set_object_open(req->object);
-	spin_unlock(&info->lock);
-	wake_up_all(&cache->daemon_pollwq);
-
-out:
-	spin_lock(&info->lock);
-	/* Need to set object close to avoid reopen status continuing */
-	if (info->ondemand_id == CACHEFILES_ONDEMAND_ID_CLOSED)
-		cachefiles_ondemand_set_object_close(req->object);
-	spin_unlock(&info->lock);
-	complete(&req->done);
-	return ret;
-}
-
-int cachefiles_ondemand_restore(struct cachefiles_cache *cache, char *args)
-{
-	struct cachefiles_req *req;
-
-	XA_STATE(xas, &cache->reqs, 0);
-
-	if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
-		return -EOPNOTSUPP;
-
-	/*
-	 * Reset the requests to CACHEFILES_REQ_NEW state, so that the
-	 * requests have been processed halfway before the crash of the
-	 * user daemon could be reprocessed after the recovery.
-	 */
-	xas_lock(&xas);
-	xas_for_each(&xas, req, ULONG_MAX)
-		xas_set_mark(&xas, CACHEFILES_REQ_NEW);
-	xas_unlock(&xas);
-
-	wake_up_all(&cache->daemon_pollwq);
-	return 0;
-}
-
-static int cachefiles_ondemand_get_fd(struct cachefiles_req *req,
-				      struct ondemand_anon_file *anon_file)
-{
-	struct cachefiles_object *object;
-	struct cachefiles_cache *cache;
-	struct cachefiles_open *load;
-	u32 object_id;
-	int ret;
-
-	object = cachefiles_grab_object(req->object,
-			cachefiles_obj_get_ondemand_fd);
-	cache = object->volume->cache;
-
-	ret = xa_alloc_cyclic(&cache->ondemand_ids, &object_id, NULL,
-			      XA_LIMIT(1, INT_MAX),
-			      &cache->ondemand_id_next, GFP_KERNEL);
-	if (ret < 0)
-		goto err;
-
-	anon_file->fd = get_unused_fd_flags(O_WRONLY);
-	if (anon_file->fd < 0) {
-		ret = anon_file->fd;
-		goto err_free_id;
-	}
-
-	anon_file->file = anon_inode_getfile_fmode("[cachefiles]",
-				&cachefiles_ondemand_fd_fops, object,
-				O_WRONLY, FMODE_PWRITE | FMODE_LSEEK);
-	if (IS_ERR(anon_file->file)) {
-		ret = PTR_ERR(anon_file->file);
-		goto err_put_fd;
-	}
-
-	spin_lock(&object->ondemand->lock);
-	if (object->ondemand->ondemand_id > 0) {
-		spin_unlock(&object->ondemand->lock);
-		/* Pair with check in cachefiles_ondemand_fd_release(). */
-		anon_file->file->private_data = NULL;
-		ret = -EEXIST;
-		goto err_put_file;
-	}
-
-	load = (void *)req->msg.data;
-	load->fd = anon_file->fd;
-	object->ondemand->ondemand_id = object_id;
-	spin_unlock(&object->ondemand->lock);
-
-	cachefiles_get_unbind_pincount(cache);
-	trace_cachefiles_ondemand_open(object, &req->msg, load);
-	return 0;
-
-err_put_file:
-	fput(anon_file->file);
-	anon_file->file = NULL;
-err_put_fd:
-	put_unused_fd(anon_file->fd);
-	anon_file->fd = ret;
-err_free_id:
-	xa_erase(&cache->ondemand_ids, object_id);
-err:
-	spin_lock(&object->ondemand->lock);
-	/* Avoid marking an opened object as closed. */
-	if (object->ondemand->ondemand_id <= 0)
-		cachefiles_ondemand_set_object_close(object);
-	spin_unlock(&object->ondemand->lock);
-	cachefiles_put_object(object, cachefiles_obj_put_ondemand_fd);
-	return ret;
-}
-
-static void ondemand_object_worker(struct work_struct *work)
-{
-	struct cachefiles_ondemand_info *info =
-		container_of(work, struct cachefiles_ondemand_info, ondemand_work);
-
-	cachefiles_ondemand_init_object(info->object);
-}
-
-/*
- * If there are any inflight or subsequent READ requests on the
- * closed object, reopen it.
- * Skip read requests whose related object is reopening.
- */
-static struct cachefiles_req *cachefiles_ondemand_select_req(struct xa_state *xas,
-							      unsigned long xa_max)
-{
-	struct cachefiles_req *req;
-	struct cachefiles_object *object;
-	struct cachefiles_ondemand_info *info;
-
-	xas_for_each_marked(xas, req, xa_max, CACHEFILES_REQ_NEW) {
-		if (req->msg.opcode != CACHEFILES_OP_READ)
-			return req;
-		object = req->object;
-		info = object->ondemand;
-		if (cachefiles_ondemand_object_is_close(object)) {
-			cachefiles_ondemand_set_object_reopening(object);
-			queue_work(fscache_wq, &info->ondemand_work);
-			continue;
-		}
-		if (cachefiles_ondemand_object_is_reopening(object))
-			continue;
-		return req;
-	}
-	return NULL;
-}
-
-static inline bool cachefiles_ondemand_finish_req(struct cachefiles_req *req,
-						  struct xa_state *xas, int err)
-{
-	if (unlikely(!xas || !req))
-		return false;
-
-	if (xa_cmpxchg(xas->xa, xas->xa_index, req, NULL, 0) != req)
-		return false;
-
-	req->error = err;
-	complete(&req->done);
-	return true;
-}
-
-ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
-					char __user *_buffer, size_t buflen)
-{
-	struct cachefiles_req *req;
-	struct cachefiles_msg *msg;
-	size_t n;
-	int ret = 0;
-	struct ondemand_anon_file anon_file;
-	XA_STATE(xas, &cache->reqs, cache->req_id_next);
-
-	xa_lock(&cache->reqs);
-	/*
-	 * Cyclically search for a request that has not ever been processed,
-	 * to prevent requests from being processed repeatedly, and make
-	 * request distribution fair.
-	 */
-	req = cachefiles_ondemand_select_req(&xas, ULONG_MAX);
-	if (!req && cache->req_id_next > 0) {
-		xas_set(&xas, 0);
-		req = cachefiles_ondemand_select_req(&xas, cache->req_id_next - 1);
-	}
-	if (!req) {
-		xa_unlock(&cache->reqs);
-		return 0;
-	}
-
-	msg = &req->msg;
-	n = msg->len;
-
-	if (n > buflen) {
-		xa_unlock(&cache->reqs);
-		return -EMSGSIZE;
-	}
-
-	xas_clear_mark(&xas, CACHEFILES_REQ_NEW);
-	cache->req_id_next = xas.xa_index + 1;
-	refcount_inc(&req->ref);
-	cachefiles_grab_object(req->object, cachefiles_obj_get_read_req);
-	xa_unlock(&cache->reqs);
-
-	if (msg->opcode == CACHEFILES_OP_OPEN) {
-		ret = cachefiles_ondemand_get_fd(req, &anon_file);
-		if (ret)
-			goto out;
-	}
-
-	msg->msg_id = xas.xa_index;
-	msg->object_id = req->object->ondemand->ondemand_id;
-
-	if (copy_to_user(_buffer, msg, n) != 0)
-		ret = -EFAULT;
-
-	if (msg->opcode == CACHEFILES_OP_OPEN) {
-		if (ret < 0) {
-			fput(anon_file.file);
-			put_unused_fd(anon_file.fd);
-			goto out;
-		}
-		fd_install(anon_file.fd, anon_file.file);
-	}
-out:
-	cachefiles_put_object(req->object, cachefiles_obj_put_read_req);
-	/* Remove error request and CLOSE request has no reply */
-	if (ret || msg->opcode == CACHEFILES_OP_CLOSE)
-		cachefiles_ondemand_finish_req(req, &xas, ret);
-	cachefiles_req_put(req);
-	return ret ? ret : n;
-}
-
-typedef int (*init_req_fn)(struct cachefiles_req *req, void *private);
-
-static int cachefiles_ondemand_send_req(struct cachefiles_object *object,
-					enum cachefiles_opcode opcode,
-					size_t data_len,
-					init_req_fn init_req,
-					void *private)
-{
-	struct cachefiles_cache *cache = object->volume->cache;
-	struct cachefiles_req *req = NULL;
-	XA_STATE(xas, &cache->reqs, 0);
-	int ret;
-
-	if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
-		return 0;
-
-	if (test_bit(CACHEFILES_DEAD, &cache->flags)) {
-		ret = -EIO;
-		goto out;
-	}
-
-	req = kzalloc(sizeof(*req) + data_len, GFP_KERNEL);
-	if (!req) {
-		ret = -ENOMEM;
-		goto out;
-	}
-
-	refcount_set(&req->ref, 1);
-	req->object = object;
-	init_completion(&req->done);
-	req->msg.opcode = opcode;
-	req->msg.len = sizeof(struct cachefiles_msg) + data_len;
-
-	ret = init_req(req, private);
-	if (ret)
-		goto out;
-
-	do {
-		/*
-		 * Stop enqueuing the request when daemon is dying. The
-		 * following two operations need to be atomic as a whole.
-		 *   1) check cache state, and
-		 *   2) enqueue request if cache is alive.
-		 * Otherwise the request may be enqueued after xarray has been
-		 * flushed, leaving the orphan request never being completed.
-		 *
-		 * CPU 1			CPU 2
-		 * =====			=====
-		 *				test CACHEFILES_DEAD bit
-		 * set CACHEFILES_DEAD bit
-		 * flush requests in the xarray
-		 *				enqueue the request
-		 */
-		xas_lock(&xas);
-
-		if (test_bit(CACHEFILES_DEAD, &cache->flags) ||
-		    cachefiles_ondemand_object_is_dropping(object)) {
-			xas_unlock(&xas);
-			ret = -EIO;
-			goto out;
-		}
-
-		/* coupled with the barrier in cachefiles_flush_reqs() */
-		smp_mb();
-
-		if (opcode == CACHEFILES_OP_CLOSE &&
-		    !cachefiles_ondemand_object_is_open(object)) {
-			WARN_ON_ONCE(object->ondemand->ondemand_id == 0);
-			xas_unlock(&xas);
-			ret = -EIO;
-			goto out;
-		}
-
-		/*
-		 * Cyclically find a free xas to avoid msg_id reuse that would
-		 * cause the daemon to successfully copen a stale msg_id.
-		 */
-		xas.xa_index = cache->msg_id_next;
-		xas_find_marked(&xas, UINT_MAX, XA_FREE_MARK);
-		if (xas.xa_node == XAS_RESTART) {
-			xas.xa_index = 0;
-			xas_find_marked(&xas, cache->msg_id_next - 1, XA_FREE_MARK);
-		}
-		if (xas.xa_node == XAS_RESTART)
-			xas_set_err(&xas, -EBUSY);
-
-		xas_store(&xas, req);
-		if (xas_valid(&xas)) {
-			cache->msg_id_next = xas.xa_index + 1;
-			xas_clear_mark(&xas, XA_FREE_MARK);
-			xas_set_mark(&xas, CACHEFILES_REQ_NEW);
-		}
-		xas_unlock(&xas);
-	} while (xas_nomem(&xas, GFP_KERNEL));
-
-	ret = xas_error(&xas);
-	if (ret)
-		goto out;
-
-	wake_up_all(&cache->daemon_pollwq);
-wait:
-	ret = wait_for_completion_killable(&req->done);
-	if (!ret) {
-		ret = req->error;
-	} else {
-		ret = -EINTR;
-		if (!cachefiles_ondemand_finish_req(req, &xas, ret)) {
-			/* Someone will complete it soon. */
-			cpu_relax();
-			goto wait;
-		}
-	}
-	cachefiles_req_put(req);
-	return ret;
-out:
-	/* Reset the object to close state in error handling path.
-	 * If error occurs after creating the anonymous fd,
-	 * cachefiles_ondemand_fd_release() will set object to close.
-	 */
-	if (opcode == CACHEFILES_OP_OPEN &&
-	    !cachefiles_ondemand_object_is_dropping(object))
-		cachefiles_ondemand_set_object_close(object);
-	kfree(req);
-	return ret;
-}
-
-static int cachefiles_ondemand_init_open_req(struct cachefiles_req *req,
-					     void *private)
-{
-	struct cachefiles_object *object = req->object;
-	struct fscache_cookie *cookie = object->cookie;
-	struct fscache_volume *volume = object->volume->vcookie;
-	struct cachefiles_open *load = (void *)req->msg.data;
-	size_t volume_key_size, cookie_key_size;
-	void *volume_key, *cookie_key;
-
-	/*
-	 * Volume key is a NUL-terminated string. key[0] stores strlen() of the
-	 * string, followed by the content of the string (excluding '\0').
-	 */
-	volume_key_size = volume->key[0] + 1;
-	volume_key = volume->key + 1;
-
-	/* Cookie key is binary data, which is netfs specific. */
-	cookie_key_size = cookie->key_len;
-	cookie_key = fscache_get_key(cookie);
-
-	if (!(object->cookie->advice & FSCACHE_ADV_WANT_CACHE_SIZE)) {
-		pr_err("WANT_CACHE_SIZE is needed for on-demand mode\n");
-		return -EINVAL;
-	}
-
-	load->volume_key_size = volume_key_size;
-	load->cookie_key_size = cookie_key_size;
-	memcpy(load->data, volume_key, volume_key_size);
-	memcpy(load->data + volume_key_size, cookie_key, cookie_key_size);
-
-	return 0;
-}
-
-static int cachefiles_ondemand_init_close_req(struct cachefiles_req *req,
-					      void *private)
-{
-	struct cachefiles_object *object = req->object;
-
-	if (!cachefiles_ondemand_object_is_open(object))
-		return -ENOENT;
-
-	trace_cachefiles_ondemand_close(object, &req->msg);
-	return 0;
-}
-
-struct cachefiles_read_ctx {
-	loff_t off;
-	size_t len;
-};
-
-static int cachefiles_ondemand_init_read_req(struct cachefiles_req *req,
-					     void *private)
-{
-	struct cachefiles_object *object = req->object;
-	struct cachefiles_read *load = (void *)req->msg.data;
-	struct cachefiles_read_ctx *read_ctx = private;
-
-	load->off = read_ctx->off;
-	load->len = read_ctx->len;
-	trace_cachefiles_ondemand_read(object, &req->msg, load);
-	return 0;
-}
-
-int cachefiles_ondemand_init_object(struct cachefiles_object *object)
-{
-	struct fscache_cookie *cookie = object->cookie;
-	struct fscache_volume *volume = object->volume->vcookie;
-	size_t volume_key_size, cookie_key_size, data_len;
-
-	if (!object->ondemand)
-		return 0;
-
-	/*
-	 * CacheFiles will firstly check the cache file under the root cache
-	 * directory. If the coherency check failed, it will fallback to
-	 * creating a new tmpfile as the cache file. Reuse the previously
-	 * allocated object ID if any.
-	 */
-	if (cachefiles_ondemand_object_is_open(object))
-		return 0;
-
-	volume_key_size = volume->key[0] + 1;
-	cookie_key_size = cookie->key_len;
-	data_len = sizeof(struct cachefiles_open) +
-		   volume_key_size + cookie_key_size;
-
-	return cachefiles_ondemand_send_req(object, CACHEFILES_OP_OPEN,
-			data_len, cachefiles_ondemand_init_open_req, NULL);
-}
-
-void cachefiles_ondemand_clean_object(struct cachefiles_object *object)
-{
-	unsigned long index;
-	struct cachefiles_req *req;
-	struct cachefiles_cache *cache;
-
-	if (!object->ondemand)
-		return;
-
-	cachefiles_ondemand_send_req(object, CACHEFILES_OP_CLOSE, 0,
-			cachefiles_ondemand_init_close_req, NULL);
-
-	if (!object->ondemand->ondemand_id)
-		return;
-
-	/* Cancel all requests for the object that is being dropped. */
-	cache = object->volume->cache;
-	xa_lock(&cache->reqs);
-	cachefiles_ondemand_set_object_dropping(object);
-	xa_for_each(&cache->reqs, index, req) {
-		if (req->object == object) {
-			req->error = -EIO;
-			complete(&req->done);
-			__xa_erase(&cache->reqs, index);
-		}
-	}
-	xa_unlock(&cache->reqs);
-
-	/* Wait for ondemand_object_worker() to finish to avoid UAF. */
-	cancel_work_sync(&object->ondemand->ondemand_work);
-}
-
-int cachefiles_ondemand_init_obj_info(struct cachefiles_object *object,
-				struct cachefiles_volume *volume)
-{
-	if (!cachefiles_in_ondemand_mode(volume->cache))
-		return 0;
-
-	object->ondemand = kzalloc_obj(struct cachefiles_ondemand_info);
-	if (!object->ondemand)
-		return -ENOMEM;
-
-	object->ondemand->object = object;
-	spin_lock_init(&object->ondemand->lock);
-	INIT_WORK(&object->ondemand->ondemand_work, ondemand_object_worker);
-	return 0;
-}
-
-void cachefiles_ondemand_deinit_obj_info(struct cachefiles_object *object)
-{
-	kfree(object->ondemand);
-	object->ondemand = NULL;
-}
-
-int cachefiles_ondemand_read(struct cachefiles_object *object,
-			     loff_t pos, size_t len)
-{
-	struct cachefiles_read_ctx read_ctx = {pos, len};
-
-	return cachefiles_ondemand_send_req(object, CACHEFILES_OP_READ,
-			sizeof(struct cachefiles_read),
-			cachefiles_ondemand_init_read_req, &read_ctx);
-}
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 4b071d451d4b..70c0937f6797 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -192,7 +192,6 @@ struct netfs_io_subrequest {
 #define NETFS_SREQ_COPY_TO_CACHE	0	/* Set if should copy the data to the cache */
 #define NETFS_SREQ_CLEAR_TAIL		1	/* Set if the rest of the read should be cleared */
 #define NETFS_SREQ_MADE_PROGRESS	4	/* Set if we transferred at least some data */
-#define NETFS_SREQ_ONDEMAND		5	/* Set if it's from on-demand read mode */
 #define NETFS_SREQ_BOUNDARY		6	/* Set if ends on hard boundary (eg. ceph object) */
 #define NETFS_SREQ_HIT_EOF		7	/* Set if short due to EOF */
 #define NETFS_SREQ_IN_PROGRESS		8	/* Unlocked when the subrequest completes */
@@ -375,14 +374,6 @@ struct netfs_cache_ops {
 			     loff_t *_start, size_t *_len, size_t upper_len,
 			     loff_t i_size, bool no_space_allocated_yet);
 
-	/* Prepare an on-demand read operation, shortening it to a cached/uncached
-	 * boundary as appropriate.
-	 */
-	enum netfs_io_source (*prepare_ondemand_read)(struct netfs_cache_resources *cres,
-						      loff_t start, size_t *_len,
-						      loff_t i_size,
-						      unsigned long *_flags, ino_t ino);
-
 	/* Query the occupancy of the cache in a region, returning where the
 	 * next chunk of data starts and how long it is.
 	 */
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 6e3b1424eea4..9259bc71049e 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -31,10 +31,6 @@ enum cachefiles_obj_ref_trace {
 	cachefiles_obj_see_lookup_failed,
 	cachefiles_obj_see_withdraw_cookie,
 	cachefiles_obj_see_withdrawal,
-	cachefiles_obj_get_ondemand_fd,
-	cachefiles_obj_put_ondemand_fd,
-	cachefiles_obj_get_read_req,
-	cachefiles_obj_put_read_req,
 };
 
 enum fscache_why_object_killed {
@@ -129,11 +125,7 @@ enum cachefiles_error_trace {
 	EM(cachefiles_obj_see_lookup_cookie,	"SEE lookup_cookie")	\
 	EM(cachefiles_obj_see_lookup_failed,	"SEE lookup_failed")	\
 	EM(cachefiles_obj_see_withdraw_cookie,	"SEE withdraw_cookie")	\
-	EM(cachefiles_obj_see_withdrawal,	"SEE withdrawal")	\
-	EM(cachefiles_obj_get_ondemand_fd,      "GET ondemand_fd")	\
-	EM(cachefiles_obj_put_ondemand_fd,      "PUT ondemand_fd")	\
-	EM(cachefiles_obj_get_read_req,		"GET read_req")		\
-	E_(cachefiles_obj_put_read_req,		"PUT read_req")
+	E_(cachefiles_obj_see_withdrawal,	"SEE withdrawal")
 
 #define cachefiles_coherency_traces					\
 	EM(cachefiles_coherency_check_aux,	"BAD aux ")		\
@@ -687,180 +679,6 @@ TRACE_EVENT(cachefiles_io_error,
 		      __entry->error)
 	    );
 
-TRACE_EVENT(cachefiles_ondemand_open,
-	    TP_PROTO(struct cachefiles_object *obj, struct cachefiles_msg *msg,
-		     struct cachefiles_open *load),
-
-	    TP_ARGS(obj, msg, load),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-		    __field(unsigned int,	object_id)
-		    __field(unsigned int,	fd)
-		    __field(unsigned int,	flags)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg->msg_id;
-		    __entry->object_id	= msg->object_id;
-		    __entry->fd		= load->fd;
-		    __entry->flags	= load->flags;
-			   ),
-
-	    TP_printk("o=%08x mid=%x oid=%x fd=%d f=%x",
-		      __entry->obj,
-		      __entry->msg_id,
-		      __entry->object_id,
-		      __entry->fd,
-		      __entry->flags)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_copen,
-	    TP_PROTO(struct cachefiles_object *obj, unsigned int msg_id,
-		     long len),
-
-	    TP_ARGS(obj, msg_id, len),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-		    __field(long,		len)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg_id;
-		    __entry->len	= len;
-			   ),
-
-	    TP_printk("o=%08x mid=%x l=%lx",
-		      __entry->obj,
-		      __entry->msg_id,
-		      __entry->len)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_close,
-	    TP_PROTO(struct cachefiles_object *obj, struct cachefiles_msg *msg),
-
-	    TP_ARGS(obj, msg),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-		    __field(unsigned int,	object_id)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg->msg_id;
-		    __entry->object_id	= msg->object_id;
-			   ),
-
-	    TP_printk("o=%08x mid=%x oid=%x",
-		      __entry->obj,
-		      __entry->msg_id,
-		      __entry->object_id)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_read,
-	    TP_PROTO(struct cachefiles_object *obj, struct cachefiles_msg *msg,
-		     struct cachefiles_read *load),
-
-	    TP_ARGS(obj, msg, load),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-		    __field(unsigned int,	object_id)
-		    __field(loff_t,		start)
-		    __field(size_t,		len)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg->msg_id;
-		    __entry->object_id	= msg->object_id;
-		    __entry->start	= load->off;
-		    __entry->len	= load->len;
-			   ),
-
-	    TP_printk("o=%08x mid=%x oid=%x s=%llx l=%zx",
-		      __entry->obj,
-		      __entry->msg_id,
-		      __entry->object_id,
-		      __entry->start,
-		      __entry->len)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_cread,
-	    TP_PROTO(struct cachefiles_object *obj, unsigned int msg_id),
-
-	    TP_ARGS(obj, msg_id),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg_id;
-			   ),
-
-	    TP_printk("o=%08x mid=%x",
-		      __entry->obj,
-		      __entry->msg_id)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_fd_write,
-	    TP_PROTO(struct cachefiles_object *obj, struct inode *backer,
-		     loff_t start, size_t len),
-
-	    TP_ARGS(obj, backer, start, len),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	backer)
-		    __field(loff_t,		start)
-		    __field(size_t,		len)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->backer	= backer->i_ino;
-		    __entry->start	= start;
-		    __entry->len	= len;
-			   ),
-
-	    TP_printk("o=%08x iB=%x s=%llx l=%zx",
-		      __entry->obj,
-		      __entry->backer,
-		      __entry->start,
-		      __entry->len)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_fd_release,
-	    TP_PROTO(struct cachefiles_object *obj, int object_id),
-
-	    TP_ARGS(obj, object_id),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	object_id)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->object_id	= object_id;
-			   ),
-
-	    TP_printk("o=%08x oid=%x",
-		      __entry->obj,
-		      __entry->object_id)
-	    );
-
 #endif /* _TRACE_CACHEFILES_H */
 
 /* This part must be outside protection */
diff --git a/include/uapi/linux/cachefiles.h b/include/uapi/linux/cachefiles.h
deleted file mode 100644
index 78caa73e5343..000000000000
--- a/include/uapi/linux/cachefiles.h
+++ /dev/null
@@ -1,68 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
-#ifndef _LINUX_CACHEFILES_H
-#define _LINUX_CACHEFILES_H
-
-#include <linux/types.h>
-#include <linux/ioctl.h>
-
-/*
- * Fscache ensures that the maximum length of cookie key is 255. The volume key
- * is controlled by netfs, and generally no bigger than 255.
- */
-#define CACHEFILES_MSG_MAX_SIZE	1024
-
-enum cachefiles_opcode {
-	CACHEFILES_OP_OPEN,
-	CACHEFILES_OP_CLOSE,
-	CACHEFILES_OP_READ,
-};
-
-/*
- * Message Header
- *
- * @msg_id	a unique ID identifying this message
- * @opcode	message type, CACHEFILE_OP_*
- * @len		message length, including message header and following data
- * @object_id	a unique ID identifying a cache file
- * @data	message type specific payload
- */
-struct cachefiles_msg {
-	__u32 msg_id;
-	__u32 opcode;
-	__u32 len;
-	__u32 object_id;
-	__u8  data[];
-};
-
-/*
- * @data contains the volume_key followed directly by the cookie_key. volume_key
- * is a NUL-terminated string; @volume_key_size indicates the size of the volume
- * key in bytes. cookie_key is binary data, which is netfs specific;
- * @cookie_key_size indicates the size of the cookie key in bytes.
- *
- * @fd identifies an anon_fd referring to the cache file.
- */
-struct cachefiles_open {
-	__u32 volume_key_size;
-	__u32 cookie_key_size;
-	__u32 fd;
-	__u32 flags;
-	__u8  data[];
-};
-
-/*
- * @off		indicates the starting offset of the requested file range
- * @len		indicates the length of the requested file range
- */
-struct cachefiles_read {
-	__u64 off;
-	__u64 len;
-};
-
-/*
- * Reply for READ request
- * @arg for this ioctl is the @id field of READ request.
- */
-#define CACHEFILES_IOC_READ_COMPLETE	_IOW(0x98, 1, int)
-
-#endif
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index c4673a54c4ba..7e25051dc1ea 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -6676,13 +6676,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
 	{ "cachefiles_mark_inactive", 0x1 },
 	{ "cachefiles_vfs_error", 0x1 },
 	{ "cachefiles_io_error", 0x1 },
-	{ "cachefiles_ondemand_open", 0x1 },
-	{ "cachefiles_ondemand_copen", 0x1 },
-	{ "cachefiles_ondemand_close", 0x1 },
-	{ "cachefiles_ondemand_read", 0x1 },
-	{ "cachefiles_ondemand_cread", 0x1 },
-	{ "cachefiles_ondemand_fd_write", 0x1 },
-	{ "cachefiles_ondemand_fd_release", 0x1 },
 	/* ext4, from ext4__mballoc event class */
 	{ "ext4_mballoc_discard", 0x10 },
 	{ "ext4_mballoc_free", 0x10 },



From dhowells at redhat.com  Tue Aug  4 03:02:09 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:09 +0100
Subject: [PATCH v8 14/25] cachefiles: Don't rely on backing fs storage map for most use cases
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-15-dhowells@redhat.com>

Cachefiles currently uses the backing filesystem's idea of what data is
held in a backing file and queries this by means of SEEK_DATA and
SEEK_HOLE.  However, this means it does two seek operations on the backing
file for each individual read call it wants to prepare (unless the first
returns -ENXIO).  Worse, the backing filesystem is at liberty to insert or
remove blocks of zeros in order to optimise its layout which may cause
false positives and false negatives.

The problem is that keeping track of what is dirty is tricky (if storing
info in xattrs, which may have limited capacity and must be read and
written as one piece) and expensive (in terms of diskspace at least) and is
basically duplicating what a filesystem does.

However, the most common write case, in which the application does {
open(O_TRUNC); write(); write(); ... write(); close(); } where each write
follows directly on from the previous and leaves no gaps in the file is
reasonably easy to detect and can be noted in the primary xattr as
CACHEFILES_CONTENT_ALL, indicating we have everything up to the object size
stored.

In this specific case, given that it is known that there are no holes in
the file, there's no need to call SEEK_DATA/HOLE or use any other mechanism
to track the contents.  That speeds things up enormously.

Even when it is necessary to use SEEK_DATA/HOLE, it may not be necessary to
call it for each cache read subrequest generated.

Implement this by adding support for the CACHEFILES_CONTENT_ALL content
type (which is defined, but currently unused), which requires a slight
adjustment in how backing files are managed.  Specifically, the driver
needs to know how much of the tail block is data and whether storing more
data will create a hole.

To this end, the way that the size of a backing file is managed is changed.
Currently, the backing file is expanded to strictly match the size of the
network file, but this can be changed to carry more useful information.
This makes two pieces of metadata available: xattr.object_size and the
backing file's i_size.  Apply the following schema:

  (a) i_size is always a multiple of the DIO block size.

  (b) i_size is only updated to the end of the highest write stored.  This
      is used to work out if we are following on without leaving a hole.

  (c) xattr.object_size is the size of the network filesystem file cached
      in this backing file.

  (d) xattr.object_size must point after the start of the last block
      (unless both are 0).

  (e) If xattr.object_size is at or after the block at the current end of
      the backing file (ie. i_size), then we have all the contents of the
      block (if xattr.content == CACHEFILES_CONTENT_ALL).

  (f) If xattr.object_size is somewhere in the middle of the last block,
      then the data following it is invalid and must be ignored.

  (g) If data is added to the last block, then that block must be fetched,
      modified and rewritten (it must be a buffered write through the
      pagecache and not DIO).

  (h) Writes to cache are rounded out to blocks on both sides and the
      folios used as sources must contain data for any lower gap and must
      have been cleared for any upper gap, and so will rewrite any
      non-data area in the tail block.

To implement this, the following changes are made:

 (1) cookie->object_size is no longer updated when writes are copied into
     the pagecache, but rather only updated when a write request completes.

     This prevents object size miscomparison when checking the xattr
     causing the backing file to be invalidated (opening and marking the
     backing file and modifying the pagecache run in parallel).

 (2) The cache's current idea of the amount of data that should be stored
     in the backing file is kept track of in object->object_size.

     Possibly this is redundant with cookie->object_size, but the latter
     gets updated in some addition circumstances.

 (3) The size of the backing file at the start of a request is now tracked
     in struct netfs_cache_resources so that the partial EOF block can be
     located and cleaned.

 (4) The cache block size is now used consistently rather than using
     CACHEFILES_DIO_BLOCK_SIZE (4096).

 (5) The backing file size is no longer adjusted when looking up an object.

 (6) When shortening a file, if the new size is not block aligned, the part
     beyond the new size is cleared.  If the file is truncated to zero, the
     content_info gets reset to CACHEFILES_CONTENT_NO_DATA.

 (7) A new struct, fscache_occupancy, is instituted to track the region
     being read.  Netfslib allocates it and fills in the start and end of
     the region to be read then calls the ->query_occupancy() method to
     find and fill in the extents.  It also indicates whether a recorded
     extent contains data or just contains a region that's all zeros
     (FSCACHE_EXTENT_DATA or FSCACHE_EXTENT_ZERO).

 (8) The ->prepare_read() cache method is changed such that, if given, it
     just limits the amount that can be read from the cache in one go.  It
     no longer indicates what source of read should be done; that
     information is now obtained from ->query_occupancy().

 (9) A new cache method, ->collect_write(), is added that is called when a
     contiguous series of writes have completed and a discontiguity or the
     end of the request has been hit.  It it supplied with the start and
     length of the write made to the backing file and can use this
     information to update the cache metadata.

(10) cachefiles_query_occupancy() is altered to find the next two "extents"
     of data stored in the backing file by doing SEEK_DATA/HOLE between the
     bounds set - unless it is known that there are no holes, in which case
     a whole-file first extent can be set.

(11) cachefiles_collect_write() is implemented to take the collated write
     completion information and use this to update the cache metadata, in
     particular working out whether there's now a hole in the backing file
     requiring future use of SEEK_DATA/HOLE instead of just assuming the
     data is all present.

     It also uses fallocate(FALLOC_FL_ZERO_RANGE) to clean the part of a
     partial block that extended beyond the old object size.  It might be
     better to perform a synchronous DIO write for this purpose, but that
     would mandate an RMW cycle.  Ideally, it should be all zeros anyway,
     but, unfortunately, shared-writable mmap can interfere.

(12) cachefiles_begin_operation() is updated to note the current backing
     file size and the cache DIO size.

(13) cachefiles_create_tmpfile() no longer expands the backing file when it
     creates it.

(14) cachefiles_set_object_xattr() is changed to use object->object_size
     rather than cookie->object_size.

(15) cachefiles_check_auxdata() is altered to actually store the content
     type and to also set object->object_size.  The cachefiles_coherency
     tracepoint is also modified to display xattr.object_size.

(16) netfs_read_to_pagecache() is reworked.  The cache ->prepare_read()
     method is replaced with ->query_occupancy() as the arbiter of what
     region of the file is read from where, and that retrieves up to two
     occupied extents of the backing file at once.

     The cache ->prepare_read() method is now repurposed to be the same as
     the equivalent network filesystem method and allows the cache to limit
     the size of the read before the iterator is prepared.

     netfs_single_dispatch_read() is similarly modified.

(17) netfs_update_i_size() and afs_update_i_size() no longer call
     fscache_update_cookie() to update cookie->object_size.

(18) Write collection now collates contiguous sequences of writes to the
     cache and calls the cache ->collect_write() method.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/afs/file.c                     |   1 -
 fs/cachefiles/interface.c         |  90 ++------
 fs/cachefiles/internal.h          |  13 +-
 fs/cachefiles/io.c                | 334 ++++++++++++++++++++++++------
 fs/cachefiles/namei.c             |  19 +-
 fs/cachefiles/xattr.c             |  24 ++-
 fs/netfs/buffered_read.c          | 176 ++++++++++------
 fs/netfs/buffered_write.c         |   3 -
 fs/netfs/internal.h               |   2 +
 fs/netfs/read_retry.c             |   2 +
 fs/netfs/read_single.c            |  39 ++--
 fs/netfs/write_collect.c          | 132 ++++++++++--
 fs/netfs/write_issue.c            |  18 ++
 fs/netfs/write_retry.c            |   3 +
 include/linux/fscache.h           |  17 ++
 include/linux/netfs.h             |  40 +++-
 include/trace/events/cachefiles.h |  21 +-
 include/trace/events/netfs.h      |   9 +-
 18 files changed, 656 insertions(+), 287 deletions(-)

diff --git a/fs/afs/file.c b/fs/afs/file.c
index 3617c245f319..92380dc25781 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -447,7 +447,6 @@ void afs_set_i_size(struct afs_vnode *vnode, loff_t new_i_size)
 	}
 	spin_unlock(&inode->i_lock);
 	write_sequnlock(&vnode->cb_lock);
-	fscache_update_cookie(afs_vnode_cache(vnode), NULL, &new_i_size);
 }
 
 static void afs_update_i_size(struct inode *inode, loff_t new_i_size)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 50a000310a8c..638e00717119 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -99,73 +99,6 @@ void cachefiles_put_object(struct cachefiles_object *object,
 	_leave("");
 }
 
-/*
- * Adjust the size of a cache file if necessary to match the DIO size.  We keep
- * the EOF marker a multiple of DIO blocks so that we don't fall back to doing
- * non-DIO for a partial block straddling the EOF, but we also have to be
- * careful of someone expanding the file and accidentally accreting the
- * padding.
- */
-static int cachefiles_adjust_size(struct cachefiles_object *object)
-{
-	struct iattr newattrs;
-	struct file *file = object->file;
-	uint64_t ni_size;
-	loff_t oi_size;
-	int ret;
-
-	ni_size = object->cookie->object_size;
-	ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
-
-	_enter("{OBJ%x},[%llu]",
-	       object->debug_id, (unsigned long long) ni_size);
-
-	if (!file)
-		return -ENOBUFS;
-
-	oi_size = i_size_read(file_inode(file));
-	if (oi_size == ni_size)
-		return 0;
-
-	inode_lock(file_inode(file));
-
-	/* if there's an extension to a partial page at the end of the backing
-	 * file, we need to discard the partial page so that we pick up new
-	 * data after it */
-	if (oi_size & ~PAGE_MASK && ni_size > oi_size) {
-		_debug("discard tail %llx", oi_size);
-		newattrs.ia_valid = ATTR_SIZE;
-		newattrs.ia_size = oi_size & PAGE_MASK;
-		ret = cachefiles_inject_remove_error();
-		if (ret == 0)
-			ret = notify_change(&nop_mnt_idmap, file->f_path.dentry,
-					    &newattrs, NULL);
-		if (ret < 0)
-			goto truncate_failed;
-	}
-
-	newattrs.ia_valid = ATTR_SIZE;
-	newattrs.ia_size = ni_size;
-	ret = cachefiles_inject_write_error();
-	if (ret == 0)
-		ret = notify_change(&nop_mnt_idmap, file->f_path.dentry,
-				    &newattrs, NULL);
-
-truncate_failed:
-	inode_unlock(file_inode(file));
-
-	if (ret < 0)
-		trace_cachefiles_io_error(NULL, file_inode(file), ret,
-					  cachefiles_trace_notify_change_error);
-	if (ret == -EIO) {
-		cachefiles_io_error_obj(object, "Size set failed");
-		ret = -ENOBUFS;
-	}
-
-	_leave(" = %d", ret);
-	return ret;
-}
-
 /*
  * Attempt to look up the nominated node in this cache
  */
@@ -198,7 +131,6 @@ static bool cachefiles_lookup_cookie(struct fscache_cookie *cookie)
 	spin_lock(&cache->object_list_lock);
 	list_add(&object->cache_link, &cache->object_list);
 	spin_unlock(&cache->object_list_lock);
-	cachefiles_adjust_size(object);
 
 	cachefiles_end_secure(cache, saved_cred);
 	_leave(" = t");
@@ -232,7 +164,7 @@ static bool cachefiles_shorten_object(struct cachefiles_object *object,
 	loff_t i_size, dio_size;
 	int ret;
 
-	dio_size = round_up(new_size, CACHEFILES_DIO_BLOCK_SIZE);
+	dio_size = round_up(new_size, cache->bsize);
 	i_size = i_size_read(inode);
 
 	trace_cachefiles_trunc(object, inode, i_size, dio_size,
@@ -264,6 +196,7 @@ static bool cachefiles_shorten_object(struct cachefiles_object *object,
 		}
 	}
 
+	object->object_size = new_size;
 	return true;
 }
 
@@ -278,22 +211,31 @@ static void cachefiles_resize_cookie(struct netfs_cache_resources *cres,
 	struct fscache_cookie *cookie = object->cookie;
 	const struct cred *saved_cred;
 	struct file *file = cachefiles_cres_file(cres);
-	loff_t old_size = cookie->object_size;
+	unsigned long long i_size = i_size_read(file_inode(file));
 
-	_enter("%llu->%llu", old_size, new_size);
+	_enter("%llu->%llu", object->object_size, new_size);
 
-	if (new_size < old_size) {
+	/* If the file is being shrunk, we need to downsize the backing file
+	 * and clear the end of the final block.
+	 */
+	if (new_size < object->object_size) {
+		if (new_size >= i_size)
+			goto out;
 		cachefiles_begin_secure(cache, &saved_cred);
 		cachefiles_shorten_object(object, file, new_size);
 		cachefiles_end_secure(cache, saved_cred);
 		object->cookie->object_size = new_size;
+		if (new_size == 0)
+			object->content_info = CACHEFILES_CONTENT_NO_DATA;
 		return;
 	}
 
 	/* The file is being expanded.  We don't need to do anything
-	 * particularly.  cookie->initial_size doesn't change and so the point
-	 * at which we have to download before doesn't change.
+	 * particularly.  The tail of the last block should have been cleared
+	 * both when it is written and when it is shrunk.
 	 */
+out:
+	object->object_size = new_size;
 	cookie->object_size = new_size;
 }
 
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 48ea38ba1bde..99d5f9077a31 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -16,8 +16,6 @@
 #include <linux/cred.h>
 #include <linux/security.h>
 
-#define CACHEFILES_DIO_BLOCK_SIZE 4096
-
 struct cachefiles_cache;
 struct cachefiles_object;
 
@@ -51,12 +49,17 @@ struct cachefiles_object {
 	struct list_head		cache_link;	/* Link in cache->*_list */
 	struct file			*file;		/* The file representing this object */
 	char				*d_name;	/* Backing file name */
+	unsigned long			flags;
+#define CACHEFILES_OBJECT_USING_TMPFILE	0		/* Have an unlinked tmpfile */
+	unsigned long long		object_size;	/* Size of the object stored
+							 * (independent of cookie->object_size for
+							 * coherency reasons)
+							 */
+	atomic64_t			read_limit;	/* Point beyond which uncommitted writes */
 	int				debug_id;
 	spinlock_t			lock;
 	refcount_t			ref;
-	enum cachefiles_content		content_info:8;	/* Info about content presence */
-	unsigned long			flags;
-#define CACHEFILES_OBJECT_USING_TMPFILE	0		/* Have an unlinked tmpfile */
+	enum cachefiles_content		content_info;	/* Info about content presence */
 };
 
 /*
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 9540ec25b3cb..64d1e3d526d6 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -32,6 +32,8 @@ struct cachefiles_kiocb {
 	u64			b_writing;
 };
 
+#define IS_ERR_VALUE_LL(x) unlikely((x) >= (unsigned long long)-MAX_ERRNO)
+
 static inline void cachefiles_put_kiocb(struct cachefiles_kiocb *ki)
 {
 	if (refcount_dec_and_test(&ki->ki_refcnt)) {
@@ -193,60 +195,81 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
 }
 
 /*
- * Query the occupancy of the cache in a region, returning where the next chunk
- * of data starts and how long it is.
+ * Query the occupancy of the cache in a region, returning the extent of the
+ * next two chunks of cached data and the next hole.
  */
 static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
-				      loff_t start, size_t len, size_t granularity,
-				      loff_t *_data_start, size_t *_data_len)
+				      struct fscache_occupancy *occ)
 {
 	struct cachefiles_object *object;
+	struct inode *inode;
 	struct file *file;
-	loff_t off, off2;
-
-	*_data_start = -1;
-	*_data_len = 0;
+	unsigned long long read_limit;
+	loff_t ret;
+	int i;
 
 	if (!fscache_wait_for_operation(cres, FSCACHE_WANT_READ))
 		return -ENOBUFS;
 
 	object = cachefiles_cres_object(cres);
 	file = cachefiles_cres_file(cres);
-	granularity = max_t(size_t, object->volume->cache->bsize, granularity);
+	inode = file_inode(file);
+	occ->granularity = object->volume->cache->bsize;
+	/* Read read_limit before content_info. */
+	read_limit = atomic64_read_acquire(&object->read_limit);
+
+	_enter("%pD,%llu,%llx-%llx/%llx",
+	       file, inode->i_ino, occ->query_from, occ->query_to, read_limit);
+
+	if (read_limit == 0)
+		goto done;
+
+	switch (READ_ONCE(object->content_info)) {
+	case CACHEFILES_CONTENT_ALL:
+	case CACHEFILES_CONTENT_SINGLE:
+		if (read_limit > occ->query_from) {
+			occ->cached_from[0] = 0;
+			occ->cached_to[0] = read_limit;
+			occ->cached_type[0] = FSCACHE_EXTENT_DATA;
+			occ->query_from = ULLONG_MAX;
+		}
+		goto done;
+	default:
+		break;
+	}
 
-	_enter("%pD,%llu,%llx,%zx/%llx",
-	       file, file_inode(file)->i_ino, start, len,
-	       i_size_read(file_inode(file)));
+	for (i = 0; i < ARRAY_SIZE(occ->cached_from); i++) {
+		ret = cachefiles_inject_read_error();
+		if (ret == 0)
+			ret = vfs_llseek(file, occ->query_from, SEEK_DATA);
+		if (IS_ERR_VALUE_LL(ret)) {
+			if (ret != -ENXIO)
+				return ret;
+			occ->query_from = ULLONG_MAX;
+			goto done;
+		}
+		occ->cached_type[i] = FSCACHE_EXTENT_DATA;
+		occ->cached_from[i] = ret;
+		occ->query_from = ret;
+
+		ret = cachefiles_inject_read_error();
+		if (ret == 0)
+			ret = vfs_llseek(file, occ->query_from, SEEK_HOLE);
+		if (IS_ERR_VALUE_LL(ret)) {
+			if (ret != -ENXIO)
+				return ret;
+			occ->query_from = ULLONG_MAX;
+			goto done;
+		}
+		occ->cached_to[i] = ret;
+		occ->query_from = ret;
+		if (occ->query_from >= occ->query_to)
+			break;
+	}
 
-	off = cachefiles_inject_read_error();
-	if (off == 0)
-		off = vfs_llseek(file, start, SEEK_DATA);
-	if (off == -ENXIO)
-		return -ENODATA; /* Beyond EOF */
-	if (off < 0 && off >= (loff_t)-MAX_ERRNO)
-		return -ENOBUFS; /* Error. */
-	if (round_up(off, granularity) >= start + len)
-		return -ENODATA; /* No data in range */
-
-	off2 = cachefiles_inject_read_error();
-	if (off2 == 0)
-		off2 = vfs_llseek(file, off, SEEK_HOLE);
-	if (off2 == -ENXIO)
-		return -ENODATA; /* Beyond EOF */
-	if (off2 < 0 && off2 >= (loff_t)-MAX_ERRNO)
-		return -ENOBUFS; /* Error. */
-
-	/* Round away partial blocks */
-	off = round_up(off, granularity);
-	off2 = round_down(off2, granularity);
-	if (off2 <= off)
-		return -ENODATA;
-
-	*_data_start = off;
-	if (off2 > start + len)
-		*_data_len = len;
-	else
-		*_data_len = off2 - off;
+done:
+	_debug("query[0] %llx-%llx", occ->cached_from[0], occ->cached_to[0]);
+	_debug("query[1] %llx-%llx", occ->cached_from[1], occ->cached_to[1]);
 	return 0;
 }
 
@@ -478,18 +501,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 	return ret;
 }
 
-/*
- * Prepare a read operation, shortening it to a cached/uncached
- * boundary as appropriate.
- */
-static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *subreq,
-						    unsigned long long i_size)
-{
-	return cachefiles_do_prepare_read(&subreq->rreq->cache_resources,
-					  subreq->start, &subreq->len, i_size,
-					  &subreq->flags, subreq->rreq->inode->i_ino);
-}
-
 /*
  * Prepare for a write to occur.
  */
@@ -504,7 +515,7 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	int ret;
 
 	/* Round to DIO size */
-	start = round_down(*_start, PAGE_SIZE);
+	start = round_down(*_start, cache->bsize);
 	if (start != *_start || *_len > upper_len) {
 		/* Probably asked to cache a streaming write written into the
 		 * pagecache when the cookie was temporarily out of service to
@@ -514,7 +525,7 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 		return -ENOBUFS;
 	}
 
-	*_len = round_up(len, PAGE_SIZE);
+	*_len = round_up(len, cache->bsize);
 
 	/* We need to work out whether there's sufficient disk space to perform
 	 * the write - but we can skip that check if we have space already
@@ -540,7 +551,7 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	 * space, we need to see if it's fully allocated.  If it's not, we may
 	 * want to cull it.
 	 */
-	if (cachefiles_has_space(cache, 0, *_len / PAGE_SIZE,
+	if (cachefiles_has_space(cache, 0, *_len / cache->bsize,
 				 cachefiles_has_space_check) == 0)
 		return 0; /* Enough space to simply overwrite the whole block */
 
@@ -572,7 +583,7 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	return ret;
 
 check_space:
-	return cachefiles_has_space(cache, 0, *_len / PAGE_SIZE,
+	return cachefiles_has_space(cache, 0, *_len / cache->bsize,
 				    cachefiles_has_space_for_write);
 }
 
@@ -635,9 +646,9 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 	       wreq->debug_id, subreq->debug_index, start, start + len - 1);
 
 	/* We need to start on the cache granularity boundary */
-	off = start & (CACHEFILES_DIO_BLOCK_SIZE - 1);
+	off = start & (cache->bsize - 1);
 	if (off) {
-		pre = CACHEFILES_DIO_BLOCK_SIZE - off;
+		pre = cache->bsize - off;
 		if (pre >= len) {
 			fscache_count_dio_misfit();
 			netfs_write_subrequest_terminated(subreq, len);
@@ -651,8 +662,8 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 
 	/* We also need to end on the cache granularity boundary */
 	if (start + len == wreq->i_size) {
-		size_t part = len % CACHEFILES_DIO_BLOCK_SIZE;
-		size_t need = CACHEFILES_DIO_BLOCK_SIZE - part;
+		size_t part = len & (cache->bsize - 1);
+		size_t need = cache->bsize - part;
 
 		if (part && stream->submit_extendable_to >= need) {
 			len += need;
@@ -661,7 +672,7 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 		}
 	}
 
-	post = len & (CACHEFILES_DIO_BLOCK_SIZE - 1);
+	post = len & (cache->bsize - 1);
 	if (post) {
 		len -= post;
 		if (len == 0) {
@@ -688,6 +699,194 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 			 netfs_write_subrequest_terminated, subreq);
 }
 
+/*
+ * Collect the result of buffered writeback to the cache.  This includes
+ * copying a read to the cache.  Netfslib collates the results, which might
+ * occur out of order, and delivers them to the cache so that it can update its
+ * content record.
+ *
+ * block_type is one of:
+ * - NETFS_CACHE_COLLECT_WRITE_DATA for a contiguous block of data
+ * - NETFS_CACHE_COLLECT_WRITE_GAP if a discontiguity was skipped
+ * - NETFS_CACHE_COLLECT_WRITE_CANCEL for a hole due to a failed/cancelled write
+ *
+ * The writes we made are all rounded out at both sides to the nearest DIO
+ * block boundary, so if the final block contains the EOF in the middle of it
+ * (rather than at the end), padding will have been written to the file.  The
+ * backing file's filesize will have been updated if the write extended the
+ * file; the filesize may still change due to outstanding subreqs.
+ *
+ * The metadata in the cache file xattr records the size of the object we have
+ * stored, but the cache file EOF only goes up to where we've cached data to
+ * and, furthermore, is rounded up to the nearest DIO block boundary.
+ *
+ * Concurrent updates should be protected against by the caller.  Netfslib
+ * holds NETFS_ICTX_WB_LOCK as a lock on writeback requests.  DIO writes
+ * invalidate the cookie and caching is kept disabled until all users have
+ * unused the cookie.
+ */
+static void cachefiles_collect_write(struct netfs_io_request *wreq,
+				     unsigned long long start, size_t len,
+				     enum netfs_cache_collect block_type)
+{
+	struct netfs_cache_resources *cres = &wreq->cache_resources;
+	struct cachefiles_object *object = cachefiles_cres_object(cres);
+	struct cachefiles_cache *cache = object->volume->cache;
+	struct inode *inode;
+	struct file *file = cachefiles_cres_file(cres);
+	unsigned long long read_limit;
+	unsigned long long old_size = cres->cache_i_size;
+	unsigned long long new_size;
+	unsigned long long data_to = object->object_size;
+	unsigned long long end = start + len;
+	int ret;
+
+	if (!file)
+		return;
+
+	inode = file_inode(file);
+	new_size = i_size_read(inode);
+
+	_enter("%llx,%zx,%x", start, len, cache->bsize);
+
+	if (WARN_ON(old_size	& (cache->bsize - 1)) ||
+	    WARN_ON(new_size	& (cache->bsize - 1)) ||
+	    WARN_ON(start	& (cache->bsize - 1)) ||
+	    WARN_ON(len		& (cache->bsize - 1))) {
+		trace_cachefiles_io_error(object, inode, -EIO,
+					  cachefiles_trace_alignment_error);
+		cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
+		return;
+	}
+
+	/* If this is recording a gap, due to discontiguous writes or lack of
+	 * cache space, then a hole may have been introduced into the backing
+	 * file.  Treat it as a zero-length data block.
+	 */
+	if (block_type == NETFS_CACHE_COLLECT_WRITE_GAP ||
+	    block_type == NETFS_CACHE_COLLECT_WRITE_CANCEL) {
+		start = end;
+		len = 0;
+	}
+
+	/* Zeroth case: Single monolithic files are handled specially.
+	 */
+	if (wreq->origin == NETFS_WRITEBACK_SINGLE) {
+		if (block_type == NETFS_CACHE_COLLECT_WRITE_GAP ||
+		    block_type == NETFS_CACHE_COLLECT_WRITE_CANCEL) {
+			trace_cachefiles_trunc(object, inode, data_to, 0,
+					       cachefiles_trunc_zap);
+			ret = cachefiles_inject_remove_error();
+			if (ret == 0)
+				ret = vfs_truncate(&file->f_path, 0);
+			if (ret < 0) {
+				trace_cachefiles_io_error(object, inode, ret,
+							  cachefiles_trace_trunc_error);
+				cachefiles_io_error_obj(object, "truncate failed %d", ret);
+				cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
+				return;
+			}
+
+		}
+		object->content_info = CACHEFILES_CONTENT_SINGLE;
+		goto update_sizes;
+	}
+
+	/* First case: The backing file was empty. */
+	if (old_size == 0) {
+		if (start == 0)
+			object->content_info = CACHEFILES_CONTENT_ALL;
+		else
+			object->content_info = CACHEFILES_CONTENT_BACKFS_MAP;
+		goto update_sizes;
+	}
+
+	/* Second case: The backing file is entirely within the old object size
+	 * and thus there can be no partial tail block to deal with in the
+	 * cache file.
+	 */
+	if (old_size <= data_to) {
+		if (start > old_size)
+			goto discontiguous;
+		goto update_sizes;
+	}
+
+	/* Third case: The write happened entirely within the bounds of the
+	 * current cache file's size.
+	 */
+	if (end <= old_size)
+		goto update_sizes;
+
+	/* Fourth case: The write overwrote the partial tail block and extended
+	 * the file.  We only need to update the object size because netfslib
+	 * rounds out/pads cache writes to whole disk blocks.
+	 */
+	if (start < old_size)
+		goto update_sizes;
+
+	/* Fifth case: The write started from the end of the whole tail block
+	 * and extended the file.  Just extend our notion of the filesize.
+	 */
+	if (start == old_size && old_size == data_to)
+		goto update_sizes;
+
+	/* Sixth case: The write continued on from the partial tail block and
+	 * extended the file.  Need to clear the gap.
+	 */
+	if (start == old_size && old_size > data_to)
+		goto clear_gap;
+
+discontiguous:
+	/* Seventh case: The write was beyond the EOF on the cache file, so now
+	 * there's a hole in the file and we can no longer say in the metadata
+	 * that we can assume we have it all.  We may also need to clear the
+	 * end of the partial tail block.
+	 */
+	/* TODO: For the moment, we will have to use SEEK_HOLE/SEEK_DATA. */
+	if (object->content_info != CACHEFILES_CONTENT_BACKFS_MAP) {
+		object->content_info = CACHEFILES_CONTENT_BACKFS_MAP;
+		trace_cachefiles_coherency(object, inode->i_ino, data_to,
+					   be64_to_cpup((__be64 *)object->cookie->inline_aux),
+					   CACHEFILES_CONTENT_BACKFS_MAP,
+					   cachefiles_coherency_discontiguous);
+	}
+
+clear_gap:
+	/* We need to clear any partial padding that got jumped over.  It
+	 * *should* be all zeros, but shared-writable mmap exists...
+	 */
+	if (old_size > data_to) {
+		trace_cachefiles_trunc(object, inode, data_to, old_size,
+				       cachefiles_trunc_clear_padding);
+		ret = cachefiles_inject_write_error();
+		if (ret == 0)
+			ret = vfs_fallocate(file, FALLOC_FL_ZERO_RANGE,
+					    data_to, old_size - data_to);
+		if (ret < 0) {
+			trace_cachefiles_io_error(object, inode, ret,
+						  cachefiles_trace_fallocate_error);
+			cachefiles_io_error_obj(object, "fallocate zero pad failed %d", ret);
+			cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
+			return;
+		}
+	}
+
+update_sizes:
+	read_limit = umax(old_size, end);
+	cres->cache_i_size = read_limit;
+
+	/* We need to be careful setting the object_size: we may have written
+	 * more to the cache than to the server (due to cache DIO rounding) and
+	 * the i_size set on the netfs inode may include unwritten data that
+	 * the server doesn't know about yet.
+	 */
+	object->object_size = umin(read_limit, wreq->i_size);
+
+	/* Raise the limit at which reads can access the file. */
+	/* Update read_limit after content_info */
+	atomic64_set_release(&object->read_limit, read_limit);
+}
+
 /*
  * Clean up an operation.
  */
@@ -705,10 +904,10 @@ static const struct netfs_cache_ops cachefiles_netfs_cache_ops = {
 	.read			= cachefiles_read,
 	.write			= cachefiles_write,
 	.issue_write		= cachefiles_issue_write,
-	.prepare_read		= cachefiles_prepare_read,
 	.prepare_write		= cachefiles_prepare_write,
 	.prepare_write_subreq	= cachefiles_prepare_write_subreq,
 	.query_occupancy	= cachefiles_query_occupancy,
+	.collect_write		= cachefiles_collect_write,
 };
 
 /*
@@ -718,13 +917,18 @@ bool cachefiles_begin_operation(struct netfs_cache_resources *cres,
 				enum fscache_want_state want_state)
 {
 	struct cachefiles_object *object = cachefiles_cres_object(cres);
+	struct file *file;
 
 	if (!cachefiles_cres_file(cres)) {
 		cres->ops = &cachefiles_netfs_cache_ops;
 		if (object->file) {
 			spin_lock(&object->lock);
-			if (!cres->cache_priv2 && object->file)
-				cres->cache_priv2 = get_file(object->file);
+			file = object->file;
+			if (!cres->cache_priv2 && file) {
+				cres->cache_priv2 = get_file(file);
+				cres->cache_i_size = i_size_read(file_inode(file));
+				cres->dio_size = object->volume->cache->bsize;
+			}
 			spin_unlock(&object->lock);
 		}
 	}
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 88955249a1a6..a30df0f91e5e 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -414,7 +414,6 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 	struct dentry *fan = volume->fanout[(u8)object->cookie->key_hash];
 	struct file *file;
 	const struct path parentpath = { .mnt = cache->mnt, .dentry = fan };
-	uint64_t ni_size;
 	long ret;
 
 
@@ -442,23 +441,6 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 	if (!cachefiles_mark_inode_in_use(object, file_inode(file)))
 		WARN_ON(1);
 
-	ni_size = object->cookie->object_size;
-	ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
-
-	if (ni_size > 0) {
-		trace_cachefiles_trunc(object, file_inode(file), 0, ni_size,
-				       cachefiles_trunc_expand_tmpfile);
-		ret = cachefiles_inject_write_error();
-		if (ret == 0)
-			ret = vfs_truncate(&file->f_path, ni_size);
-		if (ret < 0) {
-			trace_cachefiles_vfs_error(
-				object, file_inode(file), ret,
-				cachefiles_trace_trunc_error);
-			goto err_unuse;
-		}
-	}
-
 	ret = -EINVAL;
 	if (unlikely(!file->f_op->read_iter) ||
 	    unlikely(!file->f_op->write_iter)) {
@@ -467,6 +449,7 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 	}
 out:
 	cachefiles_end_secure(cache, saved_cred);
+	object->content_info = CACHEFILES_CONTENT_ALL;
 	return file;
 
 err_unuse:
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index f8ae78b3f7b6..25f2a906f984 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -54,7 +54,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
 	if (!buf)
 		return -ENOMEM;
 
-	buf->object_size	= cpu_to_be64(object->cookie->object_size);
+	buf->object_size	= cpu_to_be64(object->object_size);
 	buf->zero_point		= 0;
 	buf->type		= CACHEFILES_COOKIE_TYPE_DATA;
 	buf->content		= object->content_info;
@@ -77,6 +77,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
 		trace_cachefiles_vfs_error(object, file_inode(file), ret,
 					   cachefiles_trace_setxattr_error);
 		trace_cachefiles_coherency(object, file_inode(file)->i_ino,
+					   object->object_size,
 					   be64_to_cpup((__be64 *)buf->data),
 					   buf->content,
 					   cachefiles_coherency_set_fail);
@@ -86,6 +87,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
 				"Failed to set xattr with error %d", ret);
 	} else {
 		trace_cachefiles_coherency(object, file_inode(file)->i_ino,
+					   object->object_size,
 					   be64_to_cpup((__be64 *)buf->data),
 					   buf->content,
 					   cachefiles_coherency_set_ok);
@@ -103,9 +105,11 @@ int cachefiles_check_auxdata(struct cachefiles_object *object, struct file *file
 {
 	struct cachefiles_xattr *buf;
 	struct dentry *dentry = file->f_path.dentry;
+	struct inode *inode = file_inode(file);
 	unsigned int len = object->cookie->aux_len, tlen;
 	const void *p = fscache_get_aux(object->cookie);
 	enum cachefiles_coherency_trace why;
+	unsigned long long obj_size;
 	ssize_t xlen;
 	int ret = -ESTALE;
 
@@ -120,36 +124,41 @@ int cachefiles_check_auxdata(struct cachefiles_object *object, struct file *file
 	if (xlen != tlen) {
 		if (xlen < 0) {
 			ret = xlen;
-			trace_cachefiles_vfs_error(object, file_inode(file), xlen,
+			trace_cachefiles_vfs_error(object, inode, xlen,
 						   cachefiles_trace_getxattr_error);
 		}
 		if (xlen == -EIO)
 			cachefiles_io_error_obj(
 				object,
 				"Failed to read aux with error %zd", xlen);
-		why = cachefiles_coherency_check_xattr;
+		trace_cachefiles_coherency(object, inode->i_ino, 0, 0, 0,
+					   cachefiles_coherency_check_xattr);
 		goto out;
 	}
 
+	obj_size = be64_to_cpu(buf->object_size);
 	if (buf->type != CACHEFILES_COOKIE_TYPE_DATA) {
 		why = cachefiles_coherency_check_type;
 	} else if (memcmp(buf->data, p, len) != 0) {
 		why = cachefiles_coherency_check_aux;
-	} else if (be64_to_cpu(buf->object_size) != object->cookie->object_size) {
+	} else if (obj_size != object->cookie->object_size) {
 		why = cachefiles_coherency_check_objsize;
 	} else if (buf->content == CACHEFILES_CONTENT_DIRTY) {
 		// TODO: Begin conflict resolution
 		pr_warn("Dirty object in cache\n");
 		why = cachefiles_coherency_check_dirty;
 	} else {
+		object->content_info = buf->content;
+		object->object_size = obj_size;
+		atomic64_set(&object->read_limit, i_size_read(inode));
 		why = cachefiles_coherency_check_ok;
 		ret = 0;
 	}
 
-out:
-	trace_cachefiles_coherency(object, file_inode(file)->i_ino,
+	trace_cachefiles_coherency(object, inode->i_ino, obj_size,
 				   be64_to_cpup((__be64 *)buf->data),
 				   buf->content, why);
+out:
 	kfree(buf);
 	return ret;
 }
@@ -163,6 +172,9 @@ int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
 {
 	int ret;
 
+	trace_cachefiles_coherency(object, d_inode(dentry)->i_ino, 0, 0, 0,
+				   cachefiles_coherency_remove);
+
 	ret = cachefiles_inject_remove_error();
 	if (ret == 0) {
 		ret = mnt_want_write(cache->mnt);
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 4752d9325c4b..0f4070c8c0d5 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -137,21 +137,6 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
 	return subreq->len;
 }
 
-static enum netfs_io_source netfs_cache_prepare_read(struct netfs_io_request *rreq,
-						     struct netfs_io_subrequest *subreq,
-						     loff_t i_size)
-{
-	struct netfs_cache_resources *cres = &rreq->cache_resources;
-	enum netfs_io_source source;
-
-	if (!cres->ops)
-		return NETFS_DOWNLOAD_FROM_SERVER;
-	source = cres->ops->prepare_read(subreq, i_size);
-	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-	return source;
-
-}
-
 /*
  * Issue a read against the cache.
  * - Eats the caller's ref on subreq.
@@ -166,6 +151,19 @@ static void netfs_read_cache_to_pagecache(struct netfs_io_request *rreq,
 			netfs_cache_read_terminated, subreq);
 }
 
+int netfs_read_query_cache(struct netfs_io_request *rreq, struct fscache_occupancy *occ)
+{
+	struct netfs_cache_resources *cres = &rreq->cache_resources;
+
+	occ->granularity = PAGE_SIZE;
+	if (occ->query_from >= occ->query_to)
+		return 0;
+	if (!cres->ops)
+		return 0;
+	occ->query_from = round_up(occ->query_from, occ->granularity);
+	return cres->ops->query_occupancy(cres, occ);
+}
+
 void netfs_queue_read(struct netfs_io_request *rreq,
 		      struct netfs_io_subrequest *subreq)
 {
@@ -218,15 +216,54 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
  */
 static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 {
+	struct fscache_occupancy _occ = {
+		.query_from	= rreq->start,
+		.query_to	= rreq->start + rreq->len,
+		.cached_from[0]	= 0,
+		.cached_to[0]	= 0,
+		.cached_from[1]	= ULLONG_MAX,
+		.cached_to[1]	= ULLONG_MAX,
+	};
+	struct fscache_occupancy *occ = &_occ;
 	unsigned long long start = rreq->start;
 	ssize_t size = rreq->len;
 	int ret = 0;
 
 	do {
+		int (*prepare_read)(struct netfs_io_subrequest *subreq) = NULL;
 		struct netfs_io_subrequest *subreq;
-		enum netfs_io_source source = NETFS_SOURCE_UNKNOWN;
+		unsigned long long hole_to, cache_to;
 		ssize_t slice;
 
+		/* If we don't have any, find out the next couple of data
+		 * extents from the cache, containing of following the
+		 * specified start offset.  Holes have to be fetched from the
+		 * server; data regions from the cache.
+		 */
+		hole_to = occ->cached_from[0];
+		cache_to = occ->cached_to[0];
+		if (start >= cache_to) {
+			/* Extent exhausted; shuffle down. */
+			int i;
+
+			for (i = 0; i < ARRAY_SIZE(occ->cached_from) - 1; i++) {
+				occ->cached_from[i] = occ->cached_from[i + 1];
+				occ->cached_to[i]   = occ->cached_to[i + 1];
+				occ->cached_type[i] = occ->cached_type[i + 1];
+			}
+			occ->cached_from[i] = ULLONG_MAX;
+			occ->cached_to[i]   = ULLONG_MAX;
+
+			if (occ->cached_from[0] != ULLONG_MAX)
+				continue;
+
+			/* Get new extents */
+			ret = netfs_read_query_cache(rreq, occ);
+			if (ret < 0)
+				break;
+			continue;
+		}
+
 		subreq = netfs_alloc_subrequest(rreq);
 		if (!subreq) {
 			ret = -ENOMEM;
@@ -238,63 +275,75 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 
 		netfs_queue_read(rreq, subreq);
 
-		source = netfs_cache_prepare_read(rreq, subreq, rreq->i_size);
-		subreq->source = source;
-		if (source == NETFS_DOWNLOAD_FROM_SERVER) {
-			unsigned long long zero_point = netfs_read_zero_point(rreq->inode);
-			unsigned long long zp = umin(zero_point, rreq->i_size);
-			size_t len = subreq->len;
-
-			if (unlikely(rreq->origin == NETFS_READ_SINGLE))
-				zp = rreq->i_size;
-			if (subreq->start >= zp) {
-				subreq->source = source = NETFS_FILL_WITH_ZEROES;
-				goto fill_with_zeroes;
+		unsigned long long zero_point = netfs_read_zero_point(rreq->inode);
+		unsigned long long zlimit = umin(zero_point, rreq->i_size);
+
+		_debug("rsub %llx %llx-%llx", subreq->start, hole_to, cache_to);
+
+		if (start >= hole_to && start < cache_to) {
+			/* Overlap with a cached region, where the cache may
+			 * record a block of zeroes.
+			 */
+			_debug("cached s=%llx c=%llx l=%zx", start, cache_to, size);
+			subreq->len = umin(cache_to - start, size);
+			subreq->len = round_up(subreq->len, occ->granularity);
+			if (occ->cached_type[0] == FSCACHE_EXTENT_ZERO) {
+				subreq->source = NETFS_FILL_WITH_ZEROES;
+				netfs_stat(&netfs_n_rh_zero);
+			} else {
+				subreq->source = NETFS_READ_FROM_CACHE;
+				prepare_read = rreq->cache_resources.ops->prepare_read;
 			}
 
-			if (len > zp - subreq->start)
-				len = zp - subreq->start;
-			if (len == 0) {
-				pr_err("ZERO-LEN READ: R=%08x[%x] l=%zx/%zx s=%llx z=%llx i=%llx",
-				       rreq->debug_id, subreq->debug_index,
-				       subreq->len, size,
-				       subreq->start, zero_point, rreq->i_size);
-				netfs_cancel_read(subreq, ret);
-				break;
-			}
-			subreq->len = len;
-
-			netfs_stat(&netfs_n_rh_download);
-			if (rreq->netfs_ops->prepare_read) {
-				ret = rreq->netfs_ops->prepare_read(subreq);
-				if (ret < 0) {
-					netfs_cancel_read(subreq, ret);
-					break;
-				}
-				trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-			}
-			goto issue;
-		}
+			trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
 
-	fill_with_zeroes:
-		if (source == NETFS_FILL_WITH_ZEROES) {
+		} else if (subreq->start >= zlimit && size > 0) {
+			/* If this range lies beyond the zero-point, that part
+			 * can just be cleared locally.
+			 */
+			_debug("zero %llx-%llx", start, start + size);
+			subreq->len = size;
 			subreq->source = NETFS_FILL_WITH_ZEROES;
-			trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+			if (rreq->cache_resources.ops)
+				__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
 			netfs_stat(&netfs_n_rh_zero);
-			goto issue;
+		} else {
+			/* Read a cache hole from the server.  If any part of
+			 * this range lies beyond the zero-point or the EOF,
+			 * that part can just be cleared locally.
+			 */
+			unsigned long long limit = min3(zlimit, start + size, hole_to);
+
+			_debug("limit %llx %llx", rreq->i_size, zero_point);
+			_debug("download %llx-%llx", start, start + size);
+			subreq->len = umin(limit - subreq->start, ULONG_MAX);
+			subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
+			if (rreq->cache_resources.ops)
+				__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
+			netfs_stat(&netfs_n_rh_download);
 		}
 
-		if (source == NETFS_READ_FROM_CACHE) {
-			trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
-			goto issue;
+		if (size == 0) {
+			pr_err("ZERO-LEN READ: R=%08x[%x] l=%zx/%zx s=%llx z=%llx i=%llx",
+			       rreq->debug_id, subreq->debug_index,
+			       subreq->len, size,
+			       subreq->start, zero_point, rreq->i_size);
+			netfs_cancel_read(subreq, ret);
+			break;
 		}
 
-		pr_err("Unexpected read source %u\n", source);
-		WARN_ON_ONCE(1);
-		netfs_cancel_read(subreq, ret);
-		break;
+		rreq->io_streams[0].sreq_max_len = MAX_RW_COUNT;
+		rreq->io_streams[0].sreq_max_segs = INT_MAX;
+
+		if (prepare_read) {
+			ret = prepare_read(subreq);
+			if (ret < 0) {
+				netfs_cancel_read(subreq, ret);
+				break;
+			}
+			trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
+		}
 
-	issue:
 		slice = netfs_prepare_read_iterator(subreq);
 		if (slice < 0) {
 			ret = slice;
@@ -308,6 +357,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 			set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
 		}
 
+		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 		netfs_issue_read(rreq, subreq);
 		netfs_maybe_bulk_drop_ra_refs(rreq);
 
diff --git a/fs/netfs/buffered_write.c b/fs/netfs/buffered_write.c
index 39f00b326e0f..8e84ec6d26c6 100644
--- a/fs/netfs/buffered_write.c
+++ b/fs/netfs/buffered_write.c
@@ -54,9 +54,6 @@ void netfs_update_i_size(struct netfs_inode *ctx, struct inode *inode,
 	i_size = i_size_read(inode);
 	if (end > i_size) {
 		i_size_write(inode, end);
-#if IS_ENABLED(CONFIG_FSCACHE)
-		fscache_update_cookie(ctx->cache, NULL, &end);
-#endif
 
 		gap = SECTOR_SIZE - (i_size & (SECTOR_SIZE - 1));
 		if (copied > gap) {
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 5aea8d34adcf..3c10f125f13b 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -23,6 +23,8 @@
 /*
  * buffered_read.c
  */
+int netfs_read_query_cache(struct netfs_io_request *rreq,
+			   struct fscache_occupancy *occ);
 void netfs_queue_read(struct netfs_io_request *rreq,
 		      struct netfs_io_subrequest *subreq);
 void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error);
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index 2b42758e01ec..ce61f4d8d301 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -272,6 +272,7 @@ void netfs_retry_reads(struct netfs_io_request *rreq)
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
 
 	netfs_stat(&netfs_n_rh_retry_read_req);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_retry_begin);
 
 	/* Wait for all outstanding I/O to quiesce before performing retries as
 	 * we may need to renegotiate the I/O sizes.
@@ -282,6 +283,7 @@ void netfs_retry_reads(struct netfs_io_request *rreq)
 
 	trace_netfs_rreq(rreq, netfs_rreq_trace_resubmit);
 	netfs_retry_read_subrequests(rreq);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_retry_end);
 }
 
 /*
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index 8833550d2eb6..65ef16ada1bf 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -58,20 +58,6 @@ static int netfs_single_begin_cache_read(struct netfs_io_request *rreq, struct n
 	return fscache_begin_read_operation(&rreq->cache_resources, netfs_i_cookie(ctx));
 }
 
-static void netfs_single_cache_prepare_read(struct netfs_io_request *rreq,
-					    struct netfs_io_subrequest *subreq)
-{
-	struct netfs_cache_resources *cres = &rreq->cache_resources;
-
-	if (!cres->ops) {
-		subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
-		return;
-	}
-	subreq->source = cres->ops->prepare_read(subreq, rreq->i_size);
-	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-
-}
-
 static void netfs_single_read_cache(struct netfs_io_request *rreq,
 				    struct netfs_io_subrequest *subreq)
 {
@@ -89,6 +75,14 @@ static void netfs_single_read_cache(struct netfs_io_request *rreq,
  */
 static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 {
+	struct fscache_occupancy occ = {
+		.query_from	= 0,
+		.query_to	= rreq->len,
+		.cached_from[0]	= ULLONG_MAX,
+		.cached_to[0]	= ULLONG_MAX,
+		.cached_from[1]	= ULLONG_MAX,
+		.cached_to[1]	= ULLONG_MAX,
+	};
 	struct netfs_io_subrequest *subreq;
 	int ret = 0;
 
@@ -96,14 +90,21 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 	if (!subreq)
 		return -ENOMEM;
 
-	subreq->source	= NETFS_SOURCE_UNKNOWN;
+	subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
 	subreq->start	= 0;
 	subreq->len	= rreq->len;
 	subreq->io_iter	= rreq->buffer.iter;
 
 	netfs_queue_read(rreq, subreq);
 
-	netfs_single_cache_prepare_read(rreq, subreq);
+	/* Try to use the cache if the cache content matches the size of the
+	 * remote file.
+	 */
+	netfs_read_query_cache(rreq, &occ);
+	if (occ.cached_from[0] == 0 &&
+	    occ.cached_to[0] >= rreq->len)
+		subreq->source = NETFS_READ_FROM_CACHE;
+
 	switch (subreq->source) {
 	case NETFS_DOWNLOAD_FROM_SERVER:
 		netfs_stat(&netfs_n_rh_download);
@@ -119,6 +120,12 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 		rreq->submitted += subreq->len;
 		break;
 	case NETFS_READ_FROM_CACHE:
+		if (rreq->cache_resources.ops->prepare_read) {
+			ret = rreq->cache_resources.ops->prepare_read(subreq);
+			if (ret < 0)
+				goto cancel;
+		}
+
 		smp_wmb(); /* Write lists before ALL_QUEUED. */
 		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
 		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 2eac588e37de..5b96463808d5 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -188,6 +188,26 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 	wreq->buffer.first_tail_slot = slot;
 }
 
+/*
+ * Collect cache results.
+ */
+static void netfs_cache_collect(struct netfs_io_request *wreq,
+				struct netfs_io_stream *stream,
+				enum netfs_cache_collect block_type)
+{
+	struct netfs_cache_resources *cres = &wreq->cache_resources;
+
+	if (stream->source != NETFS_WRITE_TO_CACHE ||
+	    wreq->cache_coll_to >= stream->collected_to)
+		return;
+
+	if (cres->ops && cres->ops->collect_write)
+		cres->ops->collect_write(wreq, wreq->cache_coll_to,
+					 stream->collected_to - wreq->cache_coll_to,
+					 block_type);
+	wreq->cache_coll_to = stream->collected_to;
+}
+
 /*
  * Collect and assess the results of various write subrequests.  We may need to
  * retry some of the results - or even do an RMW cycle for content crypto.
@@ -235,13 +255,16 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 		/* Read first subreq pointer before IN_PROGRESS flag. */
 
 		while (front) {
+			bool cancelled;
 			trace_netfs_collect_sreq(wreq, front);
 			//_debug("sreq [%x] %llx %zx/%zx",
 			//       front->debug_index, front->start, front->transferred, front->len);
 
 			if (stream->collected_to < front->start) {
 				trace_netfs_collect_gap(wreq, stream, issued_to, 'F');
+				netfs_cache_collect(wreq, stream, NETFS_CACHE_COLLECT_WRITE_DATA);
 				stream->collected_to = front->start;
+				netfs_cache_collect(wreq, stream, NETFS_CACHE_COLLECT_WRITE_GAP);
 			}
 
 			/* Stall if the front is still undergoing I/O. */
@@ -249,7 +272,6 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 				notes |= HIT_PENDING;
 				break;
 			}
-			smp_rmb(); /* Read counters after I-P flag. */
 
 			if (stream->failed) {
 				stream->collected_to = front->start + front->len;
@@ -262,15 +284,44 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 				stream->transferred_valid = true;
 				notes |= MADE_PROGRESS;
 			}
-			if (test_bit(NETFS_SREQ_FAILED, &front->flags)) {
-				stream->failed = true;
-				stream->error = front->error;
-				if (stream->source == NETFS_UPLOAD_TO_SERVER)
-					mapping_set_error(wreq->mapping, front->error);
-				notes |= NEED_REASSESS | SAW_FAILURE;
+
+			/* Handle failed or cancelled subreqs.  Failure of
+			 * cache writes are handled differently to upload
+			 * failures.  Cache writes aren't fatal, provided we're
+			 * not doing disconnected operation, and so we can kind
+			 * of treat them as if they had succeeded - except that
+			 * we need to log any holes they cause.
+			 */
+			switch (stream->source) {
+			case NETFS_UPLOAD_TO_SERVER:
+				if (test_bit(NETFS_SREQ_FAILED, &front->flags)) {
+					if (!stream->failed) {
+						stream->failed = true;
+						stream->error = front->error;
+						mapping_set_error(wreq->mapping, front->error);
+						break;
+					}
+					notes |= NEED_REASSESS | SAW_FAILURE;
+				}
+				break;
+
+			case NETFS_WRITE_TO_CACHE:
+				cancelled = test_bit(NETFS_SREQ_CANCELLED, &front->flags);
+				if (cancelled != stream->cancelled) {
+					trace_netfs_rreq(wreq, netfs_rreq_trace_cache_fail_collect);
+					netfs_cache_collect(wreq, stream,
+							    cancelled ?
+							    NETFS_CACHE_COLLECT_WRITE_CANCEL :
+							    NETFS_CACHE_COLLECT_WRITE_DATA);
+					stream->cancelled = !stream->cancelled;
+				}
+				break;
+
+			default:
+				WARN_ON(1);
 				break;
 			}
-			if (front->transferred < front->len) {
+			if (test_bit(NETFS_SREQ_NEED_RETRY, &front->flags)) {
 				stream->need_retry = true;
 				notes |= NEED_RETRY | MADE_PROGRESS;
 				break;
@@ -359,6 +410,7 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
  */
 bool netfs_write_collection(struct netfs_io_request *wreq)
 {
+	struct netfs_io_stream *cstream = &wreq->io_streams[1];
 	struct netfs_inode *ictx = netfs_inode(wreq->inode);
 	size_t transferred;
 	bool transferred_valid = false;
@@ -394,13 +446,22 @@ bool netfs_write_collection(struct netfs_io_request *wreq)
 		wreq->transferred = transferred;
 	trace_netfs_rreq(wreq, netfs_rreq_trace_write_done);
 
-	if (wreq->io_streams[1].active &&
-	    wreq->io_streams[1].failed &&
-	    ictx->ops->invalidate_cache) {
-		/* Cache write failure doesn't prevent writeback completion
-		 * unless we're in disconnected mode.
-		 */
-		ictx->ops->invalidate_cache(wreq);
+	if (cstream->active) {
+		if (test_bit(NETFS_RREQ_CACHE_ERROR, &wreq->flags)) {
+			if (ictx->ops->invalidate_cache) {
+				/* Cache write failure doesn't prevent
+				 * writeback completion unless we're in
+				 * disconnected mode.
+				 */
+				trace_netfs_rreq(wreq, netfs_rreq_trace_inval_cache);
+				ictx->ops->invalidate_cache(wreq);
+			}
+		} else if (!cstream->failed) {
+			netfs_cache_collect(wreq, cstream,
+					    cstream->cancelled ?
+					    NETFS_CACHE_COLLECT_WRITE_CANCEL :
+					    NETFS_CACHE_COLLECT_WRITE_DATA);
+		}
 	}
 
 	_debug("finished");
@@ -484,24 +545,51 @@ void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error)
 
 	if (IS_ERR_VALUE(transferred_or_error)) {
 		subreq->error = transferred_or_error;
-		/* if need retry is set, error should not matter */
-		if (!test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-			set_bit(NETFS_SREQ_FAILED, &subreq->flags);
-			trace_netfs_failure(wreq, subreq, transferred_or_error, netfs_fail_write);
-		}
 
 		switch (subreq->source) {
 		case NETFS_WRITE_TO_CACHE:
+			/* We don't mark a cache-write subreq as failed.
+			 * Instead we tell the issuer to produce dummy subreqs
+			 * instead and make a note if we need to invalidate the
+			 * cache at the end.  We also don't pause the loop that
+			 * grabs pages and launches upload subreqs.
+			 *
+			 * Note that we need to distinguish between -ENOBUFS
+			 * (no space available in the cache) and other errors.
+			 * In the former case, we can keep the data we have,
+			 * though we might have to change the way the on-disk
+			 * data is tracked.
+			 */
 			netfs_stat(&netfs_n_wh_write_failed);
+			if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
+				break;
+
+			trace_netfs_failure(wreq, subreq, transferred_or_error, netfs_fail_write);
+			__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
+			set_bit(NETFS_RREQ_CACHE_STOP, &wreq->flags);
+			if (transferred_or_error == -ENOBUFS)
+				trace_netfs_rreq(wreq, netfs_rreq_trace_cache_no_space);
+			else if (!test_and_set_bit(NETFS_RREQ_CACHE_ERROR, &wreq->flags))
+				trace_netfs_rreq(wreq, netfs_rreq_trace_cache_failed);
+			subreq->transferred = subreq->len;
 			break;
+
 		case NETFS_UPLOAD_TO_SERVER:
+			/* If need_retry is set, error should not matter */
+			if (!test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
+				set_bit(NETFS_SREQ_FAILED, &subreq->flags);
+				trace_netfs_failure(wreq, subreq, transferred_or_error,
+						    netfs_fail_upload);
+			}
+
+			set_bit(NETFS_RREQ_PAUSE, &wreq->flags);
+			trace_netfs_rreq(wreq, netfs_rreq_trace_set_pause);
 			netfs_stat(&netfs_n_wh_upload_failed);
 			break;
+
 		default:
 			break;
 		}
-		trace_netfs_rreq(wreq, netfs_rreq_trace_set_pause);
-		set_bit(NETFS_RREQ_PAUSE, &wreq->flags);
 	} else {
 		if (WARN(transferred_or_error > subreq->len - subreq->transferred,
 			 "Subreq excess write: R=%x[%x] %zd > %zu - %zu",
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 5590ffe56750..9d16f63536dc 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -111,6 +111,8 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 		goto nomem;
 
 	wreq->cleaned_to = wreq->start;
+	if (wreq->cache_resources.dio_size > 1)
+		wreq->cache_coll_to = round_down(wreq->start, wreq->cache_resources.dio_size);
 
 	wreq->io_streams[0].stream_nr		= 0;
 	wreq->io_streams[0].source		= NETFS_UPLOAD_TO_SERVER;
@@ -230,6 +232,21 @@ static void netfs_do_issue_write(struct netfs_io_stream *stream,
 
 	_enter("R=%x[%x],%zx", wreq->debug_id, subreq->debug_index, subreq->len);
 
+	if (stream->source == NETFS_WRITE_TO_CACHE &&
+	    unlikely(test_bit(NETFS_RREQ_CACHE_STOP, &wreq->flags))) {
+		size_t dio_size = wreq->cache_resources.dio_size;
+		size_t len, disp;
+
+		disp = subreq->start & (dio_size - 1);
+		len = round_up(subreq->len + disp, dio_size);
+
+		subreq->start -= disp;
+		subreq->len = len;
+
+		__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
+		return netfs_write_subrequest_terminated(subreq, subreq->len);
+	}
+
 	if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
 		return netfs_write_subrequest_terminated(subreq, subreq->error);
 
@@ -263,6 +280,7 @@ void netfs_issue_write(struct netfs_io_request *wreq,
 
 	if (!subreq)
 		return;
+
 	stream->construct = NULL;
 	subreq->io_iter.count = subreq->len;
 	netfs_do_issue_write(stream, subreq);
diff --git a/fs/netfs/write_retry.c b/fs/netfs/write_retry.c
index 058bc7a166a5..508dbcbb3b19 100644
--- a/fs/netfs/write_retry.c
+++ b/fs/netfs/write_retry.c
@@ -211,6 +211,7 @@ void netfs_retry_writes(struct netfs_io_request *wreq)
 	int s;
 
 	netfs_stat(&netfs_n_wh_retry_write_req);
+	trace_netfs_rreq(wreq, netfs_rreq_trace_retry_begin);
 
 	/* Wait for all outstanding I/O to quiesce before performing retries as
 	 * we may need to renegotiate the I/O sizes.
@@ -235,4 +236,6 @@ void netfs_retry_writes(struct netfs_io_request *wreq)
 			netfs_retry_write_stream(wreq, stream);
 		}
 	}
+
+	trace_netfs_rreq(wreq, netfs_rreq_trace_retry_end);
 }
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 58fdb9605425..850d20241075 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -147,6 +147,23 @@ struct fscache_cookie {
 	};
 };
 
+enum fscache_extent_type {
+	FSCACHE_EXTENT_DATA,
+	FSCACHE_EXTENT_ZERO,
+} __mode(byte);
+
+/*
+ * Cache occupancy information.
+ */
+struct fscache_occupancy {
+	unsigned long long	query_from;	/* Point to query from */
+	unsigned long long	query_to;	/* Point to query to */
+	unsigned long long	cached_from[2];	/* Point at which cache extents start */
+	unsigned long long	cached_to[2];	/* Point at which cache extents end */
+	unsigned int		granularity;	/* Granularity desired */
+	enum fscache_extent_type cached_type[2];	/* Type of cache extent */
+};
+
 /*
  * slow-path functions for when there is actually caching available, and the
  * netfs does actually have a valid token
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 70c0937f6797..10be7180e6f3 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -23,6 +23,7 @@
 
 enum netfs_sreq_ref_trace;
 typedef struct mempool mempool_t;
+struct fscache_occupancy;
 struct folio_queue;
 
 /**
@@ -153,6 +154,7 @@ struct netfs_io_stream {
 	bool			need_retry;	/* T if this stream needs retrying */
 	bool			failed;		/* T if this stream failed */
 	bool			transferred_valid; /* T is ->transferred is valid */
+	bool			cancelled;	/* T if stream is cancelled */
 };
 
 /*
@@ -162,8 +164,10 @@ struct netfs_cache_resources {
 	const struct netfs_cache_ops	*ops;
 	void				*cache_priv;
 	void				*cache_priv2;
+	unsigned long long		cache_i_size;	/* Initial size of cache file */
 	unsigned int			debug_id;	/* Cookie debug ID */
 	unsigned int			inval_counter;	/* object->inval_counter at begin_op */
+	unsigned int			dio_size;	/* DIO block size */
 };
 
 /*
@@ -197,6 +201,7 @@ struct netfs_io_subrequest {
 #define NETFS_SREQ_IN_PROGRESS		8	/* Unlocked when the subrequest completes */
 #define NETFS_SREQ_NEED_RETRY		9	/* Set if the filesystem requests a retry */
 #define NETFS_SREQ_FAILED		10	/* Set if the subreq failed unretryably */
+#define NETFS_SREQ_CANCELLED		11	/* Set if the subreq was cancelled by netfslib */
 };
 
 enum netfs_io_origin {
@@ -251,6 +256,7 @@ struct netfs_io_request {
 	unsigned long long	start;		/* Start position */
 	atomic64_t		issued_to;	/* Write issuer folio cursor */
 	unsigned long long	collected_to;	/* Point we've collected to */
+	unsigned long long	cache_coll_to;	/* Point the cache has collected to */
 	unsigned long long	cleaned_to;	/* Position we've cleaned folios to */
 	unsigned long long	abandon_to;	/* Position to abandon folios to */
 	const struct folio	*no_unlock_folio; /* Don't unlock this folio after read */
@@ -273,11 +279,13 @@ struct netfs_io_request {
 #define NETFS_RREQ_FAILED		3	/* The request failed */
 #define NETFS_RREQ_RETRYING		4	/* Set if we're in the retry path */
 #define NETFS_RREQ_SHORT_TRANSFER	5	/* Set if we have a short transfer */
-#define NETFS_RREQ_OFFLOAD_COLLECTION	8	/* Offload collection to workqueue */
-#define NETFS_RREQ_NO_UNLOCK_FOLIO	9	/* Don't unlock no_unlock_folio on completion */
-#define NETFS_RREQ_FOLIO_COPY_TO_CACHE	10	/* Copy current folio to cache from read */
-#define NETFS_RREQ_UPLOAD_TO_SERVER	11	/* Need to write to the server */
-#define NETFS_RREQ_USE_IO_ITER		12	/* Use ->io_iter rather than ->i_pages */
+#define NETFS_RREQ_CACHE_STOP		8	/* Set to stop caching (ENOBUFS or error) */
+#define NETFS_RREQ_CACHE_ERROR		9	/* Set if we got an error from the cache */
+#define NETFS_RREQ_OFFLOAD_COLLECTION	12	/* Offload collection to workqueue */
+#define NETFS_RREQ_NO_UNLOCK_FOLIO	13	/* Don't unlock no_unlock_folio on completion */
+#define NETFS_RREQ_FOLIO_COPY_TO_CACHE	14	/* Copy current folio to cache from read */
+#define NETFS_RREQ_UPLOAD_TO_SERVER	15	/* Need to write to the server */
+#define NETFS_RREQ_USE_IO_ITER		16	/* Use ->io_iter rather than ->i_pages */
 #define NETFS_RREQ_NEED_PUT_RA_REFS	17	/* Need to put the folio refs RA gave us */
 #define NETFS_RREQ_USE_PGPRIV2		31	/* [DEPRECATED] Use PG_private_2 to mark
 						 * write to cache on read */
@@ -323,6 +331,12 @@ enum netfs_read_from_hole {
 	NETFS_READ_HOLE_FAIL,
 };
 
+enum netfs_cache_collect {
+	NETFS_CACHE_COLLECT_WRITE_DATA,
+	NETFS_CACHE_COLLECT_WRITE_GAP,
+	NETFS_CACHE_COLLECT_WRITE_CANCEL,
+};
+
 /*
  * Table of operations for access to a cache.
  */
@@ -357,8 +371,7 @@ struct netfs_cache_ops {
 	/* Prepare a read operation, shortening it to a cached/uncached
 	 * boundary as appropriate.
 	 */
-	enum netfs_io_source (*prepare_read)(struct netfs_io_subrequest *subreq,
-					     unsigned long long i_size);
+	int (*prepare_read)(struct netfs_io_subrequest *subreq);
 
 	/* Prepare a write subrequest, working out if we're allowed to do it
 	 * and finding out the maximum amount of data to gather before
@@ -378,8 +391,17 @@ struct netfs_cache_ops {
 	 * next chunk of data starts and how long it is.
 	 */
 	int (*query_occupancy)(struct netfs_cache_resources *cres,
-			       loff_t start, size_t len, size_t granularity,
-			       loff_t *_data_start, size_t *_data_len);
+			       struct fscache_occupancy *occ);
+
+	/* Collect the result of buffered writeback to the cache.  This
+	 * includes copying a read to the cache.  block_type is one of:
+	 * - NETFS_CACHE_COLLECT_WRITE_DATA for a block of data
+	 * - NETFS_CACHE_COLLECT_WRITE_GAP if a discontiguity was skipped
+	 * - NETFS_CACHE_COLLECT_WRITE_CANCEL for a cancellation gap
+	 */
+	void (*collect_write)(struct netfs_io_request *wreq,
+			      unsigned long long start, size_t len,
+			      enum netfs_cache_collect block_type);
 };
 
 /* High-level read API. */
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 9259bc71049e..eccdb5abeedc 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -52,6 +52,8 @@ enum cachefiles_coherency_trace {
 	cachefiles_coherency_check_ok,
 	cachefiles_coherency_check_type,
 	cachefiles_coherency_check_xattr,
+	cachefiles_coherency_discontiguous,
+	cachefiles_coherency_remove,
 	cachefiles_coherency_set_fail,
 	cachefiles_coherency_set_ok,
 	cachefiles_coherency_vol_check_cmp,
@@ -63,9 +65,11 @@ enum cachefiles_coherency_trace {
 };
 
 enum cachefiles_trunc_trace {
+	cachefiles_trunc_clear_padding,
 	cachefiles_trunc_dio_adjust,
 	cachefiles_trunc_expand_tmpfile,
 	cachefiles_trunc_shrink,
+	cachefiles_trunc_zap,
 };
 
 enum cachefiles_prepare_read_trace {
@@ -80,6 +84,7 @@ enum cachefiles_prepare_read_trace {
 };
 
 enum cachefiles_error_trace {
+	cachefiles_trace_alignment_error,
 	cachefiles_trace_fallocate_error,
 	cachefiles_trace_getxattr_error,
 	cachefiles_trace_link_error,
@@ -136,6 +141,8 @@ enum cachefiles_error_trace {
 	EM(cachefiles_coherency_check_ok,	"OK      ")		\
 	EM(cachefiles_coherency_check_type,	"BAD type")		\
 	EM(cachefiles_coherency_check_xattr,	"BAD xatt")		\
+	EM(cachefiles_coherency_discontiguous,	"--- gap ")		\
+	EM(cachefiles_coherency_remove,		"REMOVE  ")		\
 	EM(cachefiles_coherency_set_fail,	"SET fail")		\
 	EM(cachefiles_coherency_set_ok,		"SET ok  ")		\
 	EM(cachefiles_coherency_vol_check_cmp,	"VOL BAD cmp ")		\
@@ -146,9 +153,11 @@ enum cachefiles_error_trace {
 	E_(cachefiles_coherency_vol_set_ok,	"VOL SET ok  ")
 
 #define cachefiles_trunc_traces						\
+	EM(cachefiles_trunc_clear_padding,	"CLRPAD")		\
 	EM(cachefiles_trunc_dio_adjust,		"DIOADJ")		\
 	EM(cachefiles_trunc_expand_tmpfile,	"EXPTMP")		\
-	E_(cachefiles_trunc_shrink,		"SHRINK")
+	EM(cachefiles_trunc_shrink,		"SHRINK")		\
+	E_(cachefiles_trunc_zap,		"ZAP   ")
 
 #define cachefiles_prepare_read_traces					\
 	EM(cachefiles_trace_read_after_eof,	"after-eof ")		\
@@ -161,6 +170,7 @@ enum cachefiles_error_trace {
 	E_(cachefiles_trace_read_seek_nxio,	"seek-enxio")
 
 #define cachefiles_error_traces						\
+	EM(cachefiles_trace_alignment_error,	"align")		\
 	EM(cachefiles_trace_fallocate_error,	"fallocate")		\
 	EM(cachefiles_trace_getxattr_error,	"getxattr")		\
 	EM(cachefiles_trace_link_error,		"link")			\
@@ -371,12 +381,12 @@ TRACE_EVENT(cachefiles_rename,
 
 TRACE_EVENT(cachefiles_coherency,
 	    TP_PROTO(struct cachefiles_object *obj,
-		     ino_t ino,
+		     ino_t ino, unsigned long long obj_size,
 		     u64 disk_aux,
 		     enum cachefiles_content content,
 		     enum cachefiles_coherency_trace why),
 
-	    TP_ARGS(obj, ino, disk_aux, content, why),
+	    TP_ARGS(obj, ino, obj_size, disk_aux, content, why),
 
 	    /* Note that obj may be NULL */
 	    TP_STRUCT__entry(
@@ -384,6 +394,7 @@ TRACE_EVENT(cachefiles_coherency,
 		    __field(enum cachefiles_coherency_trace,	why)
 		    __field(enum cachefiles_content,		content)
 		    __field(u64,				ino)
+		    __field(u64,				obj_size)
 		    __field(u64,				aux)
 		    __field(u64,				disk_aux)
 			     ),
@@ -393,14 +404,16 @@ TRACE_EVENT(cachefiles_coherency,
 		    __entry->why	= why;
 		    __entry->content	= content;
 		    __entry->ino	= ino;
+		    __entry->obj_size	= obj_size;
 		    __entry->aux	= be64_to_cpup((__be64 *)obj->cookie->inline_aux);
 		    __entry->disk_aux	= disk_aux;
 			   ),
 
-	    TP_printk("o=%08x %s B=%llx c=%u aux=%llx dsk=%llx",
+	    TP_printk("o=%08x %s B=%llx oz=%llx c=%u aux=%llx dsk=%llx",
 		      __entry->obj,
 		      __print_symbolic(__entry->why, cachefiles_coherency_traces),
 		      __entry->ino,
+		      __entry->obj_size,
 		      __entry->content,
 		      __entry->aux,
 		      __entry->disk_aux)
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 2ce162baaad1..cc3806e383f5 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -48,6 +48,10 @@
 
 #define netfs_rreq_traces					\
 	EM(netfs_rreq_trace_assess,		"ASSESS ")	\
+	EM(netfs_rreq_trace_cache_cancelled,	"CA-CNCL")	\
+	EM(netfs_rreq_trace_cache_failed,	"CA-FAIL")	\
+	EM(netfs_rreq_trace_cache_fail_collect,	"CA-F-CO")	\
+	EM(netfs_rreq_trace_cache_no_space,	"CA-NOSP")	\
 	EM(netfs_rreq_trace_collect,		"COLLECT")	\
 	EM(netfs_rreq_trace_complete,		"COMPLET")	\
 	EM(netfs_rreq_trace_copy,		"COPY   ")	\
@@ -56,11 +60,14 @@
 	EM(netfs_rreq_trace_end_copy_to_cache,	"END-C2C")	\
 	EM(netfs_rreq_trace_free,		"FREE   ")	\
 	EM(netfs_rreq_trace_intr,		"INTR   ")	\
+	EM(netfs_rreq_trace_inval_cache,	"INVL-CA")	\
 	EM(netfs_rreq_trace_ki_complete,	"KI-CMPL")	\
 	EM(netfs_rreq_trace_ra_put_ref,		"RA-PUT ")	\
 	EM(netfs_rreq_trace_recollect,		"RECLLCT")	\
 	EM(netfs_rreq_trace_redirty,		"REDIRTY")	\
 	EM(netfs_rreq_trace_resubmit,		"RESUBMT")	\
+	EM(netfs_rreq_trace_retry_begin,	"RETRY-BEGIN")	\
+	EM(netfs_rreq_trace_retry_end,		"RETRY-END")	\
 	EM(netfs_rreq_trace_set_abandon,	"S-ABNDN")	\
 	EM(netfs_rreq_trace_set_pause,		"PAUSE  ")	\
 	EM(netfs_rreq_trace_unlock,		"UNLOCK ")	\
@@ -132,12 +139,12 @@
 
 #define netfs_failures							\
 	EM(netfs_fail_check_write_begin,	"check-write-begin")	\
-	EM(netfs_fail_copy_to_cache,		"copy-to-cache")	\
 	EM(netfs_fail_dio_read_short,		"dio-read-short")	\
 	EM(netfs_fail_dio_read_zero,		"dio-read-zero")	\
 	EM(netfs_fail_read,			"read")			\
 	EM(netfs_fail_short_read,		"short-read")		\
 	EM(netfs_fail_prepare_write,		"prep-write")		\
+	EM(netfs_fail_upload,			"upload")		\
 	E_(netfs_fail_write,			"write")
 
 #define netfs_rreq_ref_traces					\



From dhowells at redhat.com  Tue Aug  4 03:02:10 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:10 +0100
Subject: [PATCH v8 15/25] netfs: Add the cache object ID to netfs_read/write tracepoints
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-16-dhowells@redhat.com>

Add the cache object debug ID to netfs_read/write tracepoints to make
debugging easier as there's now a direct cross-reference with the
cachefiles tracepoints that only log that debug ID.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/cachefiles/io.c           |  1 +
 fs/netfs/fscache_io.c        |  2 +-
 include/linux/netfs.h        |  3 ++-
 include/trace/events/netfs.h | 27 +++++++++++++++------------
 4 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 64d1e3d526d6..2b5471ca8378 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -921,6 +921,7 @@ bool cachefiles_begin_operation(struct netfs_cache_resources *cres,
 
 	if (!cachefiles_cres_file(cres)) {
 		cres->ops = &cachefiles_netfs_cache_ops;
+		cres->object_id = object->debug_id;
 		if (object->file) {
 			spin_lock(&object->lock);
 			file = object->file;
diff --git a/fs/netfs/fscache_io.c b/fs/netfs/fscache_io.c
index 37f05b4d3469..fafa8c6bec57 100644
--- a/fs/netfs/fscache_io.c
+++ b/fs/netfs/fscache_io.c
@@ -79,7 +79,7 @@ static int fscache_begin_operation(struct netfs_cache_resources *cres,
 	cres->ops		= NULL;
 	cres->cache_priv	= cookie;
 	cres->cache_priv2	= NULL;
-	cres->debug_id		= cookie->debug_id;
+	cres->cookie_id		= cookie->debug_id;
 	cres->inval_counter	= cookie->inval_counter;
 
 	if (!fscache_begin_cookie_access(cookie, why)) {
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 10be7180e6f3..09e81475e899 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -165,7 +165,8 @@ struct netfs_cache_resources {
 	void				*cache_priv;
 	void				*cache_priv2;
 	unsigned long long		cache_i_size;	/* Initial size of cache file */
-	unsigned int			debug_id;	/* Cookie debug ID */
+	unsigned int			cookie_id;	/* Cache cookie debug ID */
+	unsigned int			object_id;	/* Cache object debug ID */
 	unsigned int			inval_counter;	/* object->inval_counter at begin_op */
 	unsigned int			dio_size;	/* DIO block size */
 };
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index cc3806e383f5..5dc61a4ba5f0 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -310,6 +310,7 @@ TRACE_EVENT(netfs_read,
 	    TP_STRUCT__entry(
 		    __field(unsigned int,		rreq)
 		    __field(unsigned int,		cookie)
+		    __field(unsigned int,		object)
 		    __field(loff_t,			i_size)
 		    __field(loff_t,			start)
 		    __field(size_t,			len)
@@ -319,7 +320,8 @@ TRACE_EVENT(netfs_read,
 
 	    TP_fast_assign(
 		    __entry->rreq	= rreq->debug_id;
-		    __entry->cookie	= rreq->cache_resources.debug_id;
+		    __entry->cookie	= rreq->cache_resources.cookie_id;
+		    __entry->object	= rreq->cache_resources.object_id;
 		    __entry->i_size	= rreq->i_size;
 		    __entry->start	= start;
 		    __entry->len	= len;
@@ -327,10 +329,10 @@ TRACE_EVENT(netfs_read,
 		    __entry->netfs_inode = rreq->inode->i_ino;
 			   ),
 
-	    TP_printk("R=%08x %s c=%08x ni=%llx s=%llx l=%zx sz=%llx",
+	    TP_printk("R=%08x %s c=%08x o=%08x ni=%llx s=%llx l=%zx sz=%llx",
 		      __entry->rreq,
 		      __print_symbolic(__entry->what, netfs_read_traces),
-		      __entry->cookie,
+		      __entry->cookie, __entry->object,
 		      __entry->netfs_inode,
 		      __entry->start, __entry->len, __entry->i_size)
 	    );
@@ -551,6 +553,7 @@ TRACE_EVENT(netfs_write,
 	    TP_STRUCT__entry(
 		    __field(unsigned int,		wreq)
 		    __field(unsigned int,		cookie)
+		    __field(unsigned int,		object)
 		    __field(unsigned int,		ino)
 		    __field(enum netfs_write_trace,	what)
 		    __field(unsigned long long,		start)
@@ -558,20 +561,19 @@ TRACE_EVENT(netfs_write,
 			     ),
 
 	    TP_fast_assign(
-		    struct netfs_inode *__ctx = netfs_inode(wreq->inode);
-		    struct fscache_cookie *__cookie = netfs_i_cookie(__ctx);
 		    __entry->wreq	= wreq->debug_id;
-		    __entry->cookie	= __cookie ? __cookie->debug_id : 0;
+		    __entry->cookie	= wreq->cache_resources.cookie_id;
+		    __entry->object	= wreq->cache_resources.object_id;
 		    __entry->ino	= wreq->inode->i_ino;
 		    __entry->what	= what;
 		    __entry->start	= wreq->start;
 		    __entry->len	= wreq->len;
 			   ),
 
-	    TP_printk("R=%08x %s c=%08x i=%x by=%llx-%llx",
+	    TP_printk("R=%08x %s c=%08x o=%08x i=%x by=%llx-%llx",
 		      __entry->wreq,
 		      __print_symbolic(__entry->what, netfs_write_traces),
-		      __entry->cookie,
+		      __entry->cookie, __entry->object,
 		      __entry->ino,
 		      __entry->start, __entry->start + __entry->len - 1)
 	    );
@@ -586,22 +588,23 @@ TRACE_EVENT(netfs_copy2cache,
 		    __field(unsigned int,		rreq)
 		    __field(unsigned int,		creq)
 		    __field(unsigned int,		cookie)
+		    __field(unsigned int,		object)
 		    __field(unsigned int,		ino)
 			     ),
 
 	    TP_fast_assign(
-		    struct netfs_inode *__ctx = netfs_inode(rreq->inode);
-		    struct fscache_cookie *__cookie = netfs_i_cookie(__ctx);
 		    __entry->rreq	= rreq->debug_id;
 		    __entry->creq	= creq->debug_id;
-		    __entry->cookie	= __cookie ? __cookie->debug_id : 0;
+		    __entry->cookie	= rreq->cache_resources.cookie_id;
+		    __entry->object	= rreq->cache_resources.object_id;
 		    __entry->ino	= rreq->inode->i_ino;
 			   ),
 
-	    TP_printk("R=%08x CR=%08x c=%08x i=%x ",
+	    TP_printk("R=%08x CR=%08x c=%08x o=%08x i=%x ",
 		      __entry->rreq,
 		      __entry->creq,
 		      __entry->cookie,
+		      __entry->object,
 		      __entry->ino)
 	    );
 



From dhowells at redhat.com  Tue Aug  4 03:02:12 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:12 +0100
Subject: [PATCH v8 17/25] smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter()
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-18-dhowells@redhat.com>

netfslib now only presents an bvecq queue and an associated ITER_BVECQ
iterator to the filesystem, so it isn't going to see the ITER_FOLIOQ
iterator.  So remove that code.

Netfslib also won't supply ITER_BVEC/KVEC iterators, though smbdirect
might; further in future, it won't supply iterators at all, but rather a
bvecq slice (that can be used to construct an iterator).

Signed-off-by: David Howells <dhowells at redhat.com>
Acked-by: Stefan Metzmacher <metze at samba.org>
cc: Steve French <sfrench at samba.org>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Stefan Metzmacher <metze at samba.org>
cc: Shyam Prasad N <sprasad at microsoft.com>
cc: Tom Talpey <tom at talpey.com>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/smb/smbdirect/connection.c | 68 -----------------------------------
 1 file changed, 68 deletions(-)

diff --git a/fs/smb/smbdirect/connection.c b/fs/smb/smbdirect/connection.c
index f5e7e75dd809..f217cf127cf9 100644
--- a/fs/smb/smbdirect/connection.c
+++ b/fs/smb/smbdirect/connection.c
@@ -6,7 +6,6 @@
 
 #include "internal.h"
 #include <linux/bvecq.h>
-#include <linux/folio_queue.h>
 
 struct smbdirect_map_sges {
 	struct ib_sge *sge;
@@ -2131,70 +2130,6 @@ static ssize_t smbdirect_map_sges_from_kvec(struct iov_iter *iter,
 	return ret;
 }
 
-/*
- * Extract folio fragments from a FOLIOQ-class iterator and add them to an
- * ib_sge list.  The folios are not pinned.
- */
-static ssize_t smbdirect_map_sges_from_folioq(struct iov_iter *iter,
-					      struct smbdirect_map_sges *state,
-					      ssize_t maxsize)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	unsigned int slot = iter->folioq_slot;
-	ssize_t ret = 0;
-	size_t offset = iter->iov_offset;
-
-	if (WARN_ON_ONCE(!folioq))
-		return -EIO;
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		if (WARN_ON_ONCE(!folioq))
-			return -EIO;
-		slot = 0;
-	}
-
-	do {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t fsize = folioq_folio_size(folioq, slot);
-
-		if (offset < fsize) {
-			size_t part = umin(maxsize, fsize - offset);
-			bool ok;
-
-			ok = smbdirect_map_sges_single_page(state,
-							    folio_page(folio, 0),
-							    offset,
-							    part);
-			if (!ok)
-				return -EIO;
-
-			offset += part;
-			ret += part;
-			maxsize -= part;
-		}
-
-		if (offset >= fsize) {
-			offset = 0;
-			slot++;
-			if (slot >= folioq_nr_slots(folioq)) {
-				if (!folioq->next) {
-					WARN_ON_ONCE(ret < iter->count);
-					break;
-				}
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	} while (state->num_sge < state->max_sge && maxsize > 0);
-
-	iter->folioq = folioq;
-	iter->folioq_slot = slot;
-	iter->iov_offset = offset;
-	iter->count -= ret;
-	return ret;
-}
-
 /*
  * Extract page fragments from up to the given amount of the source iterator
  * and build up an ib_sge list that refers to all of those bits.  The ib_sge list
@@ -2225,9 +2160,6 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,
 	case ITER_KVEC:
 		ret = smbdirect_map_sges_from_kvec(iter, state, len);
 		break;
-	case ITER_FOLIOQ:
-		ret = smbdirect_map_sges_from_folioq(iter, state, len);
-		break;
 	default:
 		WARN_ONCE(1, "iov_iter_type[%u]\n", iov_iter_type(iter));
 		return -EIO;



From dhowells at redhat.com  Tue Aug  4 03:02:13 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:13 +0100
Subject: [PATCH v8 18/25] netfs: Remove netfs_alloc/free_folioq_buffer()
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-19-dhowells@redhat.com>

Remove netfs_alloc/free_folioq_buffer() as bvecq_alloc_*() and bvecq_put()
should now be used instead.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/afs/dir_edit.c       |  1 -
 fs/netfs/misc.c         | 98 -----------------------------------------
 fs/smb/client/smb2ops.c |  1 -
 include/linux/netfs.h   |  6 ---
 4 files changed, 106 deletions(-)

diff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c
index 01c6265e8865..31fdef23c6b6 100644
--- a/fs/afs/dir_edit.c
+++ b/fs/afs/dir_edit.c
@@ -10,7 +10,6 @@
 #include <linux/namei.h>
 #include <linux/pagemap.h>
 #include <linux/iversion.h>
-#include <linux/folio_queue.h>
 #include "internal.h"
 #include "xdr_fs.h"
 
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index ee67a0681784..8fc4e5ef2152 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -8,104 +8,6 @@
 #include <linux/swap.h>
 #include "internal.h"
 
-#if 0
-/**
- * netfs_alloc_folioq_buffer - Allocate buffer space into a folio queue
- * @mapping: Address space to set on the folio (or NULL).
- * @_buffer: Pointer to the folio queue to add to (may point to a NULL; updated).
- * @_cur_size: Current size of the buffer (updated).
- * @size: Target size of the buffer.
- * @gfp: The allocation constraints.
- */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
-			      struct folio_queue **_buffer,
-			      size_t *_cur_size, ssize_t size, gfp_t gfp)
-{
-	struct folio_queue *tail = *_buffer, *p;
-
-	size = round_up(size, PAGE_SIZE);
-	if (*_cur_size >= size)
-		return 0;
-
-	if (tail)
-		while (tail->next)
-			tail = tail->next;
-
-	do {
-		struct folio *folio;
-		int order = 0, slot;
-
-		if (!tail || folioq_full(tail)) {
-			p = netfs_folioq_alloc(0, GFP_NOFS, netfs_trace_folioq_alloc_buffer);
-			if (!p)
-				return -ENOMEM;
-			if (tail) {
-				tail->next = p;
-				p->prev = tail;
-			} else {
-				*_buffer = p;
-			}
-			tail = p;
-		}
-
-		if (size - *_cur_size > PAGE_SIZE)
-			order = umin(ilog2(size - *_cur_size) - PAGE_SHIFT,
-				     MAX_PAGECACHE_ORDER);
-
-		folio = folio_alloc(gfp, order);
-		if (!folio && order > 0)
-			folio = folio_alloc(gfp, 0);
-		if (!folio)
-			return -ENOMEM;
-
-		folio->mapping = mapping;
-		folio->index = *_cur_size / PAGE_SIZE;
-		trace_netfs_folio(folio, netfs_folio_trace_alloc_buffer);
-		slot = folioq_append_mark(tail, folio);
-		*_cur_size += folioq_folio_size(tail, slot);
-	} while (*_cur_size < size);
-
-	return 0;
-}
-EXPORT_SYMBOL(netfs_alloc_folioq_buffer);
-
-/**
- * netfs_free_folioq_buffer - Free a folio queue.
- * @fq: The start of the folio queue to free
- *
- * Free up a chain of folio_queues and, if marked, the marked folios they point
- * to.
- */
-void netfs_free_folioq_buffer(struct folio_queue *fq)
-{
-	struct folio_queue *next;
-	struct folio_batch fbatch;
-
-	folio_batch_init(&fbatch);
-
-	for (; fq; fq = next) {
-		for (int slot = 0; slot < folioq_count(fq); slot++) {
-			struct folio *folio = folioq_folio(fq, slot);
-
-			if (!folio ||
-			    !folioq_is_marked(fq, slot))
-				continue;
-
-			trace_netfs_folio(folio, netfs_folio_trace_put);
-			if (folio_batch_add(&fbatch, folio))
-				folio_batch_release(&fbatch);
-		}
-
-		netfs_stat_d(&netfs_n_folioq);
-		next = fq->next;
-		kfree(fq);
-	}
-
-	folio_batch_release(&fbatch);
-}
-EXPORT_SYMBOL(netfs_free_folioq_buffer);
-#endif
-
 /**
  * netfs_dirty_folio - Mark folio dirty and pin a cache object for writeback
  * @mapping: The mapping the folio belongs to.
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 2b5d745ba16e..28902d3c5ef4 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -13,7 +13,6 @@
 #include <linux/sort.h>
 #include <crypto/aead.h>
 #include <linux/fiemap.h>
-#include <linux/folio_queue.h>
 #include <uapi/linux/magic.h>
 #include "cifsfs.h"
 #include "cifsglob.h"
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 6b43978bfcda..621f1d41e9f5 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -472,12 +472,6 @@ void netfs_end_io_write(struct inode *inode);
 int netfs_start_io_direct(struct inode *inode);
 void netfs_end_io_direct(struct inode *inode);
 
-/* Buffer wrangling helpers API. */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
-			      struct folio_queue **_buffer,
-			      size_t *_cur_size, ssize_t size, gfp_t gfp);
-void netfs_free_folioq_buffer(struct folio_queue *fq);
-
 /* Writeback exclusion API. */
 bool netfs_wb_begin(struct netfs_inode *ictx, bool nowait);
 void netfs_wb_end(struct netfs_inode *ictx);



From dhowells at redhat.com  Tue Aug  4 03:02:14 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:14 +0100
Subject: [PATCH v8 19/25] netfs: Remove netfs_extract_user_iter()
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-20-dhowells@redhat.com>

Remove netfs_extract_user_iter() as it has been replaced with
netfs_extract_iter().

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/iterator.c   | 104 ------------------------------------------
 include/linux/netfs.h |   3 --
 2 files changed, 107 deletions(-)

diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index 2a977b1f8f15..9635a3ed67ea 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -159,110 +159,6 @@ ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pag
 EXPORT_SYMBOL_GPL(netfs_extract_iter);
 
 #if 0
-/**
- * netfs_extract_user_iter - Extract the pages from a user iterator into a bvec
- * @orig: The original iterator
- * @orig_len: The amount of iterator to copy
- * @new: The iterator to be set up
- * @extraction_flags: Flags to qualify the request
- *
- * Extract the page fragments from the given amount of the source iterator and
- * build up a second iterator that refers to all of those bits.  This allows
- * the original iterator to be disposed of.
- *
- * @extraction_flags can have ITER_ALLOW_P2PDMA set to request peer-to-peer DMA be
- * allowed on the pages extracted.
- *
- * On success, the number of elements in the bvec is returned, the original
- * iterator will have been advanced by the amount extracted.
- *
- * The iov_iter_extract_mode() function should be used to query how cleanup
- * should be performed.
- */
-ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
-				struct iov_iter *new,
-				iov_iter_extraction_t extraction_flags)
-{
-	struct bio_vec *bv = NULL;
-	struct page **pages;
-	unsigned int cur_npages;
-	unsigned int max_pages;
-	unsigned int npages = 0;
-	unsigned int i;
-	ssize_t ret = 0;
-	size_t count = orig_len, offset, len;
-	size_t bv_size, pg_size;
-
-	if (WARN_ON_ONCE(!iter_is_ubuf(orig) && !iter_is_iovec(orig)))
-		return -EIO;
-
-	max_pages = iov_iter_npages(orig, INT_MAX);
-	bv_size = array_size(max_pages, sizeof(*bv));
-	bv = kvmalloc(bv_size, GFP_KERNEL);
-	if (!bv)
-		return -ENOMEM;
-
-	/* Put the page list at the end of the bvec list storage.  bvec
-	 * elements are larger than page pointers, so as long as we work
-	 * 0->last, we should be fine.
-	 */
-	pg_size = array_size(max_pages, sizeof(*pages));
-	pages = (void *)bv + bv_size - pg_size;
-
-	while (count && npages < max_pages) {
-		ret = iov_iter_extract_pages(orig, &pages, count,
-					     max_pages - npages, extraction_flags,
-					     &offset);
-		if (unlikely(ret <= 0)) {
-			ret = ret ?: -EIO;
-			break;
-		}
-
-		if (WARN(ret > count,
-			 "%s: extract_pages overrun %zd > %zu bytes\n",
-			 __func__, ret, count)) {
-			ret = -EIO;
-			break;
-		}
-
-		cur_npages = DIV_ROUND_UP(offset + ret, PAGE_SIZE);
-		if (WARN(cur_npages > max_pages - npages,
-			 "%s: extract_pages overrun %u > %u pages\n",
-			 __func__, npages + cur_npages, max_pages)) {
-			ret = -EIO;
-			break;
-		}
-
-		count -= ret;
-		ret += offset;
-
-		for (i = 0; i < cur_npages; i++) {
-			len = ret > PAGE_SIZE ? PAGE_SIZE : ret;
-			bvec_set_page(bv + npages + i, *pages++, len - offset, offset);
-			ret -= len;
-			offset = 0;
-		}
-
-		npages += cur_npages;
-	}
-
-	/* Note: Don't try to clean up after EIO.  Either we got no pages, so
-	 * nothing to clean up, or we got a buffer overrun, memory corruption
-	 * and can't trust the stuff in the buffer (a WARN was emitted).
-	 */
-
-	if (ret < 0 && (ret == -ENOMEM || npages == 0)) {
-		for (i = 0; i < npages; i++)
-			unpin_user_page(bv[i].bv_page);
-		kvfree(bv);
-		return ret;
-	}
-
-	iov_iter_bvec(new, orig->data_source, bv, npages, orig_len - count);
-	return npages;
-}
-EXPORT_SYMBOL_GPL(netfs_extract_user_iter);
-
 /*
  * Select the span of a bvec iterator we're going to use.  Limit it by both maximum
  * size and maximum number of segments.  Returns the size of the span in bytes.
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 621f1d41e9f5..85e4d49479eb 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -457,9 +457,6 @@ void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
 			   unsigned long long fpos, struct bvecq **_bvecq_head,
 			   iov_iter_extraction_t extraction_flags, gfp_t gfp);
-ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
-				struct iov_iter *new,
-				iov_iter_extraction_t extraction_flags);
 size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
 			size_t max_size, size_t max_segs);
 void netfs_prepare_write_failed(struct netfs_io_subrequest *subreq);



From dhowells at redhat.com  Tue Aug  4 03:02:15 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:15 +0100
Subject: [PATCH v8 20/25] iov_iter: Remove ITER_FOLIOQ
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-21-dhowells@redhat.com>

Remove ITER_FOLIOQ as it's no longer used.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 include/linux/iov_iter.h   |  65 +--------
 include/linux/uio.h        |  12 --
 lib/iov_iter.c             | 175 +-----------------------
 lib/scatterlist.c          |  69 +---------
 lib/tests/kunit_iov_iter.c | 271 -------------------------------------
 5 files changed, 7 insertions(+), 585 deletions(-)

diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index 04b8a6d943fa..4e7d69b4ace2 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -11,7 +11,6 @@
 #include <linux/uio.h>
 #include <linux/bvec.h>
 #include <linux/bvecq.h>
-#include <linux/folio_queue.h>
 
 typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
 			     void *priv, void *priv2);
@@ -207,62 +206,6 @@ size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
 	return progress;
 }
 
-/*
- * Handle ITER_FOLIOQ.
- */
-static __always_inline
-size_t iterate_folioq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
-		      iov_step_f step)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	unsigned int slot = iter->folioq_slot;
-	size_t progress = 0, skip = iter->iov_offset;
-
-	if (slot == folioq_nr_slots(folioq)) {
-		/* The iterator may have been extended. */
-		folioq = folioq->next;
-		slot = 0;
-	}
-
-	do {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t part, remain = 0, consumed;
-		size_t fsize;
-		void *base;
-
-		if (!folio)
-			break;
-
-		fsize = folioq_folio_size(folioq, slot);
-		if (skip < fsize) {
-			base = kmap_local_folio(folio, skip);
-			part = umin(len, PAGE_SIZE - skip % PAGE_SIZE);
-			remain = step(base, progress, part, priv, priv2);
-			kunmap_local(base);
-			consumed = part - remain;
-			len -= consumed;
-			progress += consumed;
-			skip += consumed;
-		}
-		if (skip >= fsize) {
-			skip = 0;
-			slot++;
-			if (slot == folioq_nr_slots(folioq) && folioq->next) {
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-		if (remain)
-			break;
-	} while (len);
-
-	iter->folioq_slot = slot;
-	iter->folioq = folioq;
-	iter->iov_offset = skip;
-	iter->count -= progress;
-	return progress;
-}
-
 /*
  * Handle ITER_XARRAY.
  */
@@ -374,8 +317,6 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_kvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_bvecq(iter))
 		return iterate_bvecq(iter, len, priv, priv2, step);
-	if (iov_iter_is_folioq(iter))
-		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
 		return iterate_xarray(iter, len, priv, priv2, step);
 	return iterate_discard(iter, len, priv, priv2, step);
@@ -410,8 +351,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
  * buffer is presented in segments, which for kernel iteration are broken up by
  * physical pages and mapped, with the mapped address being presented.
  *
- * [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and
- * DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators.
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, XARRAY and DISCARD-type
+ * iterators; it will not handle UBUF or IOVEC-type iterators.
  *
  * A step functions, @step, must be provided, one for handling mapped kernel
  * addresses and the other is given user addresses which have the potential to
@@ -440,8 +381,6 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_kvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_bvecq(iter))
 		return iterate_bvecq(iter, len, priv, priv2, step);
-	if (iov_iter_is_folioq(iter))
-		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
 		return iterate_xarray(iter, len, priv, priv2, step);
 	return iterate_discard(iter, len, priv, priv2, step);
diff --git a/include/linux/uio.h b/include/linux/uio.h
index f7cfa6ea8213..e84a0c4f28c6 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -11,7 +11,6 @@
 #include <uapi/linux/uio.h>
 
 struct page;
-struct folio_queue;
 
 typedef unsigned int __bitwise iov_iter_extraction_t;
 
@@ -27,7 +26,6 @@ enum iter_type {
 	ITER_BVEC,
 	ITER_KVEC,
 	ITER_BVECQ,
-	ITER_FOLIOQ,
 	ITER_XARRAY,
 	ITER_DISCARD,
 };
@@ -70,7 +68,6 @@ struct iov_iter {
 				const struct kvec *kvec;
 				const struct bio_vec *bvec;
 				const struct bvecq *bvecq;
-				const struct folio_queue *folioq;
 				struct xarray *xarray;
 				void __user *ubuf;
 			};
@@ -80,7 +77,6 @@ struct iov_iter {
 	union {
 		unsigned long nr_segs;
 		u16 bvecq_slot;
-		u8 folioq_slot;
 		loff_t xarray_start;
 	};
 };
@@ -153,11 +149,6 @@ static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
 	return iov_iter_type(i) == ITER_BVECQ;
 }
 
-static inline bool iov_iter_is_folioq(const struct iov_iter *i)
-{
-	return iov_iter_type(i) == ITER_FOLIOQ;
-}
-
 static inline bool iov_iter_is_xarray(const struct iov_iter *i)
 {
 	return iov_iter_type(i) == ITER_XARRAY;
@@ -306,9 +297,6 @@ void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
 void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
 			 const struct bvecq *bvecq,
 			 unsigned int first_slot, unsigned int offset, size_t count);
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
-			  const struct folio_queue *folioq,
-			  unsigned int first_slot, unsigned int offset, size_t count);
 void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xarray *xarray,
 		     loff_t start, size_t count);
 ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index ff12ddd917cd..b1d253f69862 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -572,39 +572,6 @@ static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
 	i->bvecq = bq;
 }
 
-static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
-{
-	const struct folio_queue *folioq = i->folioq;
-	unsigned int slot = i->folioq_slot;
-
-	if (!i->count)
-		return;
-	i->count -= size;
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		slot = 0;
-	}
-
-	size += i->iov_offset; /* From beginning of current segment. */
-	do {
-		size_t fsize = folioq_folio_size(folioq, slot);
-
-		if (likely(size < fsize))
-			break;
-		size -= fsize;
-		slot++;
-		if (slot >= folioq_nr_slots(folioq) && folioq->next) {
-			folioq = folioq->next;
-			slot = 0;
-		}
-	} while (size);
-
-	i->iov_offset = size;
-	i->folioq_slot = slot;
-	i->folioq = folioq;
-}
-
 void iov_iter_advance(struct iov_iter *i, size_t size)
 {
 	if (unlikely(i->count < size))
@@ -619,8 +586,6 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
 		iov_iter_bvec_advance(i, size);
 	} else if (iov_iter_is_bvecq(i)) {
 		iov_iter_bvecq_advance(i, size);
-	} else if (iov_iter_is_folioq(i)) {
-		iov_iter_folioq_advance(i, size);
 	} else if (iov_iter_is_discard(i)) {
 		i->count -= size;
 	}
@@ -654,32 +619,6 @@ static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
 	i->bvecq = bq;
 }
 
-static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
-{
-	const struct folio_queue *folioq = i->folioq;
-	unsigned int slot = i->folioq_slot;
-
-	for (;;) {
-		size_t fsize;
-
-		if (slot == 0) {
-			folioq = folioq->prev;
-			slot = folioq_nr_slots(folioq);
-		}
-		slot--;
-
-		fsize = folioq_folio_size(folioq, slot);
-		if (unroll <= fsize) {
-			i->iov_offset = fsize - unroll;
-			break;
-		}
-		unroll -= fsize;
-	}
-
-	i->folioq_slot = slot;
-	i->folioq = folioq;
-}
-
 void iov_iter_revert(struct iov_iter *i, size_t unroll)
 {
 	if (!unroll)
@@ -714,9 +653,6 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
 	} else if (iov_iter_is_bvecq(i)) {
 		i->iov_offset = 0;
 		iov_iter_bvecq_revert(i, unroll);
-	} else if (iov_iter_is_folioq(i)) {
-		i->iov_offset = 0;
-		iov_iter_folioq_revert(i, unroll);
 	} else { /* same logics for iovec and kvec */
 		const struct iovec *iov = iter_iov(i);
 		while (1) {
@@ -766,8 +702,6 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
 		}
 		return umin(i->count, bq->bv[slot].bv_len - offset);
 	}
-	if (unlikely(iov_iter_is_folioq(i)))
-		return umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
 	return i->count;
 }
 EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -833,36 +767,6 @@ void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
 }
 EXPORT_SYMBOL(iov_iter_bvec_queue);
 
-/**
- * iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
- * @i: The iterator to initialise.
- * @direction: The direction of the transfer.
- * @folioq: The starting point in the folio queue.
- * @first_slot: The first slot in the folio queue to use
- * @offset: The offset into the folio in the first slot to start at
- * @count: The size of the I/O buffer in bytes.
- *
- * Set up an I/O iterator to either draw data out of the pages attached to an
- * inode or to inject data into those pages.  The pages *must* be prevented
- * from evaporation, either by taking a ref on them or locking them by the
- * caller.
- */
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
-			  const struct folio_queue *folioq, unsigned int first_slot,
-			  unsigned int offset, size_t count)
-{
-	BUG_ON(direction & ~1);
-	*i = (struct iov_iter) {
-		.iter_type = ITER_FOLIOQ,
-		.data_source = direction,
-		.folioq = folioq,
-		.folioq_slot = first_slot,
-		.count = count,
-		.iov_offset = offset,
-	};
-}
-EXPORT_SYMBOL(iov_iter_folio_queue);
-
 /**
  * iov_iter_xarray - Initialise an I/O iterator to use the pages in an xarray
  * @i: The iterator to initialise.
@@ -1006,9 +910,7 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
 	if (iov_iter_is_bvecq(i))
 		return iov_iter_alignment_bvecq(i);
 
-	/* With both xarray and folioq types, we're dealing with whole folios. */
-	if (iov_iter_is_folioq(i))
-		return i->iov_offset | i->count;
+	/* With the xarray type, we're dealing with whole folios. */
 	if (iov_iter_is_xarray(i))
 		return (i->xarray_start + i->iov_offset) | i->count;
 
@@ -1265,11 +1167,6 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
 		return bvec_npages(i, maxpages);
 	if (iov_iter_is_bvecq(i))
 		return iov_npages_bvecq(i, maxpages);
-	if (iov_iter_is_folioq(i)) {
-		unsigned offset = i->iov_offset % PAGE_SIZE;
-		int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
-		return min(npages, maxpages);
-	}
 	if (iov_iter_is_xarray(i)) {
 		unsigned offset = (i->xarray_start + i->iov_offset) % PAGE_SIZE;
 		int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1692,68 +1589,6 @@ static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
 	return extracted;
 }
 
-/*
- * Extract a list of contiguous pages from an ITER_FOLIOQ iterator.  This does
- * not get references on the pages, nor does it get a pin on them.
- */
-static ssize_t iov_iter_extract_folioq_pages(struct iov_iter *i,
-					     struct page ***pages, size_t maxsize,
-					     unsigned int maxpages,
-					     iov_iter_extraction_t extraction_flags,
-					     size_t *offset0)
-{
-	const struct folio_queue *folioq = i->folioq;
-	struct page **p;
-	unsigned int nr = 0;
-	size_t extracted = 0, offset, slot = i->folioq_slot;
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		slot = 0;
-		if (WARN_ON(i->iov_offset != 0))
-			return -EIO;
-	}
-
-	offset = i->iov_offset & ~PAGE_MASK;
-	*offset0 = offset;
-
-	maxpages = want_pages_array(pages, maxsize, offset, maxpages);
-	if (!maxpages)
-		return -ENOMEM;
-	p = *pages;
-
-	for (;;) {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t offset = i->iov_offset, fsize = folioq_folio_size(folioq, slot);
-		size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
-		if (offset < fsize) {
-			part = umin(part, umin(maxsize - extracted, fsize - offset));
-			i->count -= part;
-			i->iov_offset += part;
-			extracted += part;
-
-			p[nr++] = folio_page(folio, offset / PAGE_SIZE);
-		}
-
-		if (nr >= maxpages || extracted >= maxsize)
-			break;
-
-		if (i->iov_offset >= fsize) {
-			i->iov_offset = 0;
-			slot++;
-			if (slot == folioq_nr_slots(folioq) && folioq->next) {
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	}
-
-	i->folioq = folioq;
-	i->folioq_slot = slot;
-	return extracted;
-}
-
 /*
  * Extract a list of contiguous pages from an ITER_XARRAY iterator.  This does not
  * get references on the pages, nor does it get a pin on them.
@@ -2016,8 +1851,8 @@ static ssize_t iov_iter_extract_user_pages(struct iov_iter *i,
  *      added to the pages, but refs will not be taken.
  *      iov_iter_extract_will_pin() will return true.
  *
- *  (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_FOLIOQ or ITER_XARRAY, the
- *      pages are merely listed; no extra refs or pins are obtained.
+ *  (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_XARRAY, the pages are
+ *      merely listed; no extra refs or pins are obtained.
  *      iov_iter_extract_will_pin() will return 0.
  *
  * Note also:
@@ -2056,10 +1891,6 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
 		return iov_iter_extract_bvecq_pages(i, pages, maxsize,
 						    maxpages, extraction_flags,
 						    offset0);
-	if (iov_iter_is_folioq(i))
-		return iov_iter_extract_folioq_pages(i, pages, maxsize,
-						     maxpages, extraction_flags,
-						     offset0);
 	if (iov_iter_is_xarray(i))
 		return iov_iter_extract_xarray_pages(i, pages, maxsize,
 						     maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index 23e5a180103b..b9a7298306d9 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -12,7 +12,6 @@
 #include <linux/bvec.h>
 #include <linux/bvecq.h>
 #include <linux/uio.h>
-#include <linux/folio_queue.h>
 
 /**
  * sg_nents - return total count of entries in scatterlist
@@ -1327,67 +1326,6 @@ static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
 	return ret;
 }
 
-/*
- * Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
- * the scatterlist.  The pages are not pinned.
- */
-static ssize_t extract_folioq_to_sg(struct iov_iter *iter,
-				   ssize_t maxsize,
-				   struct sg_table *sgtable,
-				   unsigned int sg_max,
-				   iov_iter_extraction_t extraction_flags)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	struct scatterlist *sg = sgtable->sgl + sgtable->nents;
-	unsigned int slot = iter->folioq_slot;
-	ssize_t ret = 0;
-	size_t offset = iter->iov_offset;
-
-	BUG_ON(!folioq);
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		if (WARN_ON_ONCE(!folioq))
-			return 0;
-		slot = 0;
-	}
-
-	do {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t fsize = folioq_folio_size(folioq, slot);
-
-		if (offset < fsize) {
-			size_t part = umin(maxsize - ret, fsize - offset);
-
-			sg_set_page(sg, folio_page(folio, 0), part, offset);
-			sgtable->nents++;
-			sg++;
-			sg_max--;
-			offset += part;
-			ret += part;
-		}
-
-		if (offset >= fsize) {
-			offset = 0;
-			slot++;
-			if (slot >= folioq_nr_slots(folioq)) {
-				if (!folioq->next) {
-					WARN_ON_ONCE(ret < iter->count);
-					break;
-				}
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	} while (sg_max > 0 && ret < maxsize);
-
-	iter->folioq = folioq;
-	iter->folioq_slot = slot;
-	iter->iov_offset = offset;
-	iter->count -= ret;
-	return ret;
-}
-
 /*
  * Extract up to sg_max folios from an XARRAY-type iterator and add them to
  * the scatterlist.  The pages are not pinned.
@@ -1451,8 +1389,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
  * addition of @sg_max elements.
  *
  * The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, BVECQ-, KVEC-, FOLIOQ- and XARRAY-type are extracted but
- * aren't pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, XARRAY-type are extracted but aren't pinned;
+ * DISCARD-type is not supported.
  *
  * No end mark is placed on the scatterlist; that's left to the caller.
  *
@@ -1487,9 +1425,6 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
 	case ITER_BVECQ:
 		return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
 					   extraction_flags);
-	case ITER_FOLIOQ:
-		return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
-					    extraction_flags);
 	case ITER_XARRAY:
 		return extract_xarray_to_sg(iter, maxsize, sgtable, sg_max,
 					    extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index f6413e6b925a..1c7085234006 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -13,7 +13,6 @@
 #include <linux/uio.h>
 #include <linux/bvec.h>
 #include <linux/bvecq.h>
-#include <linux/folio_queue.h>
 #include <linux/scatterlist.h>
 #include <linux/minmax.h>
 #include <linux/mman.h>
@@ -383,176 +382,6 @@ static void __init iov_kunit_copy_from_bvec(struct kunit *test)
 	KUNIT_SUCCEED(test);
 }
 
-static void iov_kunit_destroy_folioq(void *data)
-{
-	struct folio_queue *folioq, *next;
-
-	for (folioq = data; folioq; folioq = next) {
-		next = folioq->next;
-		kfree(folioq);
-	}
-}
-
-static void __init iov_kunit_load_folioq(struct kunit *test,
-					struct iov_iter *iter, int dir,
-					struct folio_queue *folioq,
-					struct page **pages, size_t npages)
-{
-	struct folio_queue *p = folioq;
-	size_t size = 0;
-	int i;
-
-	for (i = 0; i < npages; i++) {
-		if (folioq_full(p)) {
-			p->next = kzalloc_obj(struct folio_queue);
-			KUNIT_ASSERT_NOT_ERR_OR_NULL(test, p->next);
-			folioq_init(p->next, 0);
-			p->next->prev = p;
-			p = p->next;
-		}
-		folioq_append(p, page_folio(pages[i]));
-		size += PAGE_SIZE;
-	}
-	iov_iter_folio_queue(iter, dir, folioq, 0, 0, size);
-}
-
-static struct folio_queue *iov_kunit_create_folioq(struct kunit *test)
-{
-	struct folio_queue *folioq;
-
-	folioq = kzalloc_obj(struct folio_queue);
-	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, folioq);
-	kunit_add_action_or_reset(test, iov_kunit_destroy_folioq, folioq);
-	folioq_init(folioq, 0);
-	return folioq;
-}
-
-/*
- * Test copying to a ITER_FOLIOQ-type iterator.
- */
-static void __init iov_kunit_copy_to_folioq(struct kunit *test)
-{
-	const struct kvec_test_range *pr;
-	struct iov_iter iter;
-	struct folio_queue *folioq;
-	struct page **spages, **bpages;
-	u8 *scratch, *buffer;
-	size_t bufsize, npages, size, copied;
-	int i, patt;
-
-	bufsize = 0x100000;
-	npages = bufsize / PAGE_SIZE;
-
-	folioq = iov_kunit_create_folioq(test);
-
-	scratch = iov_kunit_create_buffer(test, &spages, npages);
-	for (i = 0; i < bufsize; i++)
-		scratch[i] = pattern(i);
-
-	buffer = iov_kunit_create_buffer(test, &bpages, npages);
-	memset(buffer, 0, bufsize);
-
-	iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
-
-	i = 0;
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
-		size = pr->to - pr->from;
-		KUNIT_ASSERT_LE(test, pr->to, bufsize);
-
-		iov_iter_folio_queue(&iter, READ, folioq, 0, 0, pr->to);
-		iov_iter_advance(&iter, pr->from);
-		copied = copy_to_iter(scratch + i, size, &iter);
-
-		KUNIT_EXPECT_EQ(test, copied, size);
-		KUNIT_EXPECT_EQ(test, iter.count, 0);
-		KUNIT_EXPECT_EQ(test, iter.iov_offset, pr->to % PAGE_SIZE);
-		i += size;
-		if (test->status == KUNIT_FAILURE)
-			goto stop;
-	}
-
-	/* Build the expected image in the scratch buffer. */
-	patt = 0;
-	memset(scratch, 0, bufsize);
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++)
-		for (i = pr->from; i < pr->to; i++)
-			scratch[i] = pattern(patt++);
-
-	/* Compare the images */
-	for (i = 0; i < bufsize; i++) {
-		KUNIT_EXPECT_EQ_MSG(test, buffer[i], scratch[i], "at i=%x", i);
-		if (buffer[i] != scratch[i])
-			return;
-	}
-
-stop:
-	KUNIT_SUCCEED(test);
-}
-
-/*
- * Test copying from a ITER_FOLIOQ-type iterator.
- */
-static void __init iov_kunit_copy_from_folioq(struct kunit *test)
-{
-	const struct kvec_test_range *pr;
-	struct iov_iter iter;
-	struct folio_queue *folioq;
-	struct page **spages, **bpages;
-	u8 *scratch, *buffer;
-	size_t bufsize, npages, size, copied;
-	int i, j;
-
-	bufsize = 0x100000;
-	npages = bufsize / PAGE_SIZE;
-
-	folioq = iov_kunit_create_folioq(test);
-
-	buffer = iov_kunit_create_buffer(test, &bpages, npages);
-	for (i = 0; i < bufsize; i++)
-		buffer[i] = pattern(i);
-
-	scratch = iov_kunit_create_buffer(test, &spages, npages);
-	memset(scratch, 0, bufsize);
-
-	iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
-
-	i = 0;
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
-		size = pr->to - pr->from;
-		KUNIT_ASSERT_LE(test, pr->to, bufsize);
-
-		iov_iter_folio_queue(&iter, WRITE, folioq, 0, 0, pr->to);
-		iov_iter_advance(&iter, pr->from);
-		copied = copy_from_iter(scratch + i, size, &iter);
-
-		KUNIT_EXPECT_EQ(test, copied, size);
-		KUNIT_EXPECT_EQ(test, iter.count, 0);
-		KUNIT_EXPECT_EQ(test, iter.iov_offset, pr->to % PAGE_SIZE);
-		i += size;
-	}
-
-	/* Build the expected image in the main buffer. */
-	i = 0;
-	memset(buffer, 0, bufsize);
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
-		for (j = pr->from; j < pr->to; j++) {
-			buffer[i++] = pattern(j);
-			if (i >= bufsize)
-				goto stop;
-		}
-	}
-stop:
-
-	/* Compare the images */
-	for (i = 0; i < bufsize; i++) {
-		KUNIT_EXPECT_EQ_MSG(test, scratch[i], buffer[i], "at i=%x", i);
-		if (scratch[i] != buffer[i])
-			return;
-	}
-
-	KUNIT_SUCCEED(test);
-}
-
 static void iov_kunit_destroy_bvecq(void *data)
 {
 	struct bvecq *bq, *next;
@@ -1124,85 +953,6 @@ static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
 	KUNIT_SUCCEED(test);
 }
 
-/*
- * Test the extraction of ITER_FOLIOQ-type iterators.
- */
-static void __init iov_kunit_extract_pages_folioq(struct kunit *test)
-{
-	const struct kvec_test_range *pr;
-	struct folio_queue *folioq;
-	struct iov_iter iter;
-	struct page **bpages, *pagelist[8], **pages = pagelist;
-	ssize_t len;
-	size_t bufsize, size = 0, npages;
-	int i, from;
-
-	bufsize = 0x100000;
-	npages = bufsize / PAGE_SIZE;
-
-	folioq = iov_kunit_create_folioq(test);
-
-	iov_kunit_create_buffer(test, &bpages, npages);
-	iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
-
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
-		from = pr->from;
-		size = pr->to - from;
-		KUNIT_ASSERT_LE(test, pr->to, bufsize);
-
-		iov_iter_folio_queue(&iter, WRITE, folioq, 0, 0, pr->to);
-		iov_iter_advance(&iter, from);
-
-		do {
-			size_t offset0 = LONG_MAX;
-
-			for (i = 0; i < ARRAY_SIZE(pagelist); i++)
-				pagelist[i] = (void *)(unsigned long)0xaa55aa55aa55aa55ULL;
-
-			len = iov_iter_extract_pages(&iter, &pages, 100 * 1024,
-						     ARRAY_SIZE(pagelist), 0, &offset0);
-			KUNIT_EXPECT_GE(test, len, 0);
-			if (len < 0)
-				break;
-			KUNIT_EXPECT_LE(test, len, size);
-			KUNIT_EXPECT_EQ(test, iter.count, size - len);
-			if (len == 0)
-				break;
-			size -= len;
-			KUNIT_EXPECT_GE(test, (ssize_t)offset0, 0);
-			KUNIT_EXPECT_LT(test, offset0, PAGE_SIZE);
-
-			for (i = 0; i < ARRAY_SIZE(pagelist); i++) {
-				struct page *p;
-				ssize_t part = min_t(ssize_t, len, PAGE_SIZE - offset0);
-				int ix;
-
-				KUNIT_ASSERT_GE(test, part, 0);
-				ix = from / PAGE_SIZE;
-				KUNIT_ASSERT_LT(test, ix, npages);
-				p = bpages[ix];
-				KUNIT_EXPECT_PTR_EQ(test, pagelist[i], p);
-				KUNIT_EXPECT_EQ(test, offset0, from % PAGE_SIZE);
-				from += part;
-				len -= part;
-				KUNIT_ASSERT_GE(test, len, 0);
-				if (len == 0)
-					break;
-				offset0 = 0;
-			}
-
-			if (test->status == KUNIT_FAILURE)
-				goto stop;
-		} while (iov_iter_count(&iter) > 0);
-
-		KUNIT_EXPECT_EQ(test, size, 0);
-		KUNIT_EXPECT_EQ(test, iter.count, 0);
-	}
-
-stop:
-	KUNIT_SUCCEED(test);
-}
-
 /*
  * Test the extraction of ITER_XARRAY-type iterators.
  */
@@ -1430,23 +1180,6 @@ static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)
 	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
 }
 
-static void __init iov_kunit_iter_to_sg_folioq(struct kunit *test)
-{
-	struct iov_kunit_iter_to_sg_data data;
-	struct folio_queue *folioq;
-	struct iov_iter iter;
-	size_t bufsize;
-
-	bufsize = 0x200000;
-	iov_kunit_iter_to_sg_init(test, bufsize, false, &data);
-
-	folioq = iov_kunit_create_folioq(test);
-	iov_kunit_load_folioq(test, &iter, READ, folioq, data.pages,
-			      data.npages);
-
-	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
-}
-
 static void __init iov_kunit_iter_to_sg_xarray(struct kunit *test)
 {
 	struct iov_kunit_iter_to_sg_data data;
@@ -1485,18 +1218,14 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
 	KUNIT_CASE(iov_kunit_copy_from_bvec),
 	KUNIT_CASE(iov_kunit_copy_to_bvecq),
 	KUNIT_CASE(iov_kunit_copy_from_bvecq),
-	KUNIT_CASE(iov_kunit_copy_to_folioq),
-	KUNIT_CASE(iov_kunit_copy_from_folioq),
 	KUNIT_CASE(iov_kunit_copy_to_xarray),
 	KUNIT_CASE(iov_kunit_copy_from_xarray),
 	KUNIT_CASE(iov_kunit_extract_pages_kvec),
 	KUNIT_CASE(iov_kunit_extract_pages_bvec),
 	KUNIT_CASE(iov_kunit_extract_pages_bvecq),
-	KUNIT_CASE(iov_kunit_extract_pages_folioq),
 	KUNIT_CASE(iov_kunit_extract_pages_xarray),
 	KUNIT_CASE(iov_kunit_iter_to_sg_kvec),
 	KUNIT_CASE(iov_kunit_iter_to_sg_bvec),
-	KUNIT_CASE(iov_kunit_iter_to_sg_folioq),
 	KUNIT_CASE(iov_kunit_iter_to_sg_xarray),
 	KUNIT_CASE(iov_kunit_iter_to_sg_ubuf),
 	{}



From dhowells at redhat.com  Tue Aug  4 03:02:16 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:16 +0100
Subject: [PATCH v8 21/25] netfs: Remove folio_queue and rolling_buffer
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-22-dhowells@redhat.com>

Remove folio_queue and rolling_buffer as they're no longer used.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 Documentation/core-api/folio_queue.rst      | 209 --------------
 Documentation/core-api/index.rst            |   1 -
 Documentation/filesystems/netfs_library.rst |   2 +-
 fs/netfs/internal.h                         |   1 -
 fs/netfs/iterator.c                         | 192 -------------
 fs/netfs/main.c                             |   1 -
 fs/netfs/rolling_buffer.c                   | 301 --------------------
 include/linux/folio_queue.h                 | 282 ------------------
 include/linux/netfs.h                       |   2 -
 include/linux/rolling_buffer.h              |  64 -----
 kernel/bpf/btf.c                            |   2 -
 11 files changed, 1 insertion(+), 1056 deletions(-)
 delete mode 100644 Documentation/core-api/folio_queue.rst
 delete mode 100644 fs/netfs/rolling_buffer.c
 delete mode 100644 include/linux/folio_queue.h
 delete mode 100644 include/linux/rolling_buffer.h

diff --git a/Documentation/core-api/folio_queue.rst b/Documentation/core-api/folio_queue.rst
deleted file mode 100644
index b7628896d2b6..000000000000
--- a/Documentation/core-api/folio_queue.rst
+++ /dev/null
@@ -1,209 +0,0 @@
-.. SPDX-License-Identifier: GPL-2.0+
-
-===========
-Folio Queue
-===========
-
-:Author: David Howells <dhowells at redhat.com>
-
-.. Contents:
-
- * Overview
- * Initialisation
- * Adding and removing folios
- * Querying information about a folio
- * Querying information about a folio_queue
- * Folio queue iteration
- * Folio marks
- * Lockless simultaneous production/consumption issues
-
-
-Overview
-========
-
-The folio_queue struct forms a single segment in a segmented list of folios
-that can be used to form an I/O buffer.  As such, the list can be iterated over
-using the ITER_FOLIOQ iov_iter type.
-
-The publicly accessible members of the structure are::
-
-	struct folio_queue {
-		struct folio_queue *next;
-		struct folio_queue *prev;
-		...
-	};
-
-A pair of pointers are provided, ``next`` and ``prev``, that point to the
-segments on either side of the segment being accessed.  Whilst this is a
-doubly-linked list, it is intentionally not a circular list; the outward
-sibling pointers in terminal segments should be NULL.
-
-Each segment in the list also stores:
-
- * an ordered sequence of folio pointers,
- * the size of each folio and
- * three 1-bit marks per folio,
-
-but these should not be accessed directly as the underlying data structure may
-change, but rather the access functions outlined below should be used.
-
-The facility can be made accessible by::
-
-	#include <linux/folio_queue.h>
-
-and to use the iterator::
-
-	#include <linux/uio.h>
-
-
-Initialisation
-==============
-
-A segment should be initialised by calling::
-
-	void folioq_init(struct folio_queue *folioq);
-
-with a pointer to the segment to be initialised.  Note that this will not
-necessarily initialise all the folio pointers, so care must be taken to check
-the number of folios added.
-
-
-Adding and removing folios
-==========================
-
-Folios can be set in the next unused slot in a segment struct by calling one
-of::
-
-	unsigned int folioq_append(struct folio_queue *folioq,
-				   struct folio *folio);
-
-	unsigned int folioq_append_mark(struct folio_queue *folioq,
-					struct folio *folio);
-
-Both functions update the stored folio count, store the folio and note its
-size.  The second function also sets the first mark for the folio added.  Both
-functions return the number of the slot used.  [!] Note that no attempt is made
-to check that the capacity wasn't overrun and the list will not be extended
-automatically.
-
-A folio can be excised by calling::
-
-	void folioq_clear(struct folio_queue *folioq, unsigned int slot);
-
-This clears the slot in the array and also clears all the marks for that folio,
-but doesn't change the folio count - so future accesses of that slot must check
-if the slot is occupied.
-
-
-Querying information about a folio
-==================================
-
-Information about the folio in a particular slot may be queried by the
-following function::
-
-	struct folio *folioq_folio(const struct folio_queue *folioq,
-				   unsigned int slot);
-
-If a folio has not yet been set in that slot, this may yield an undefined
-pointer.  The size of the folio in a slot may be queried with either of::
-
-	unsigned int folioq_folio_order(const struct folio_queue *folioq,
-					unsigned int slot);
-
-	size_t folioq_folio_size(const struct folio_queue *folioq,
-				 unsigned int slot);
-
-The first function returns the size as an order and the second as a number of
-bytes.
-
-
-Querying information about a folio_queue
-========================================
-
-Information may be retrieved about a particular segment with the following
-functions::
-
-	unsigned int folioq_nr_slots(const struct folio_queue *folioq);
-
-	unsigned int folioq_count(struct folio_queue *folioq);
-
-	bool folioq_full(struct folio_queue *folioq);
-
-The first function returns the maximum capacity of a segment.  It must not be
-assumed that this won't vary between segments.  The second returns the number
-of folios added to a segments and the third is a shorthand to indicate if the
-segment has been filled to capacity.
-
-Not that the count and fullness are not affected by clearing folios from the
-segment.  These are more about indicating how many slots in the array have been
-initialised, and it assumed that slots won't get reused, but rather the segment
-will get discarded as the queue is consumed.
-
-
-Folio marks
-===========
-
-Folios within a queue can also have marks assigned to them.  These marks can be
-used to note information such as if a folio needs folio_put() calling upon it.
-There are three marks available to be set for each folio.
-
-The marks can be set by::
-
-	void folioq_mark(struct folio_queue *folioq, unsigned int slot);
-	void folioq_mark2(struct folio_queue *folioq, unsigned int slot);
-
-Cleared by::
-
-	void folioq_unmark(struct folio_queue *folioq, unsigned int slot);
-	void folioq_unmark2(struct folio_queue *folioq, unsigned int slot);
-
-And the marks can be queried by::
-
-	bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot);
-	bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot);
-
-The marks can be used for any purpose and are not interpreted by this API.
-
-
-Folio queue iteration
-=====================
-
-A list of segments may be iterated over using the I/O iterator facility using
-an ``iov_iter`` iterator of ``ITER_FOLIOQ`` type.  The iterator may be
-initialised with::
-
-	void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
-				  const struct folio_queue *folioq,
-				  unsigned int first_slot, unsigned int offset,
-				  size_t count);
-
-This may be told to start at a particular segment, slot and offset within a
-queue.  The iov iterator functions will follow the next pointers when advancing
-and prev pointers when reverting when needed.
-
-
-Lockless simultaneous production/consumption issues
-===================================================
-
-If properly managed, the list can be extended by the producer at the head end
-and shortened by the consumer at the tail end simultaneously without the need
-to take locks.  The ITER_FOLIOQ iterator inserts appropriate barriers to aid
-with this.
-
-Care must be taken when simultaneously producing and consuming a list.  If the
-last segment is reached and the folios it refers to are entirely consumed by
-the IOV iterators, an iov_iter struct will be left pointing to the last segment
-with a slot number equal to the capacity of that segment.  The iterator will
-try to continue on from this if there's another segment available when it is
-used again, but care must be taken lest the segment got removed and freed by
-the consumer before the iterator was advanced.
-
-It is recommended that the queue always contain at least one segment, even if
-that segment has never been filled or is entirely spent.  This prevents the
-head and tail pointers from collapsing.
-
-
-API Function Reference
-======================
-
-.. kernel-doc:: include/linux/folio_queue.h
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index 13769d5c40bf..16c529a33ac4 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -39,7 +39,6 @@ Library functionality that is used throughout the kernel.
    kref
    cleanup
    assoc_array
-   folio_queue
    xarray
    maple_tree
    idr
diff --git a/Documentation/filesystems/netfs_library.rst b/Documentation/filesystems/netfs_library.rst
index ddd799df6ce3..18e3c3aae57c 100644
--- a/Documentation/filesystems/netfs_library.rst
+++ b/Documentation/filesystems/netfs_library.rst
@@ -449,7 +449,7 @@ be called from the writeback code to write the data to the cache, if there is
 one.
 
 The inode should be marked ``NETFS_ICTX_SINGLE_NO_UPLOAD`` if this API is to be
-used.  The writeback function requires the buffer to be of ITER_FOLIOQ type.
+used.
 
 High-Level VM API
 ==================
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 667d18911949..315b02f5fa67 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -45,7 +45,6 @@ extern spinlock_t netfs_proc_lock;
 extern mempool_t netfs_request_pool;
 extern mempool_t netfs_subrequest_pool;
 extern mempool_t netfs_bvecq_pool;
-extern mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
 static inline void netfs_proc_add_rreq(struct netfs_io_request *rreq)
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index 9635a3ed67ea..c923ba68e0af 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -157,195 +157,3 @@ ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pag
 
 }
 EXPORT_SYMBOL_GPL(netfs_extract_iter);
-
-#if 0
-/*
- * Select the span of a bvec iterator we're going to use.  Limit it by both maximum
- * size and maximum number of segments.  Returns the size of the span in bytes.
- */
-static size_t netfs_limit_bvec(const struct iov_iter *iter, size_t start_offset,
-			       size_t max_size, size_t max_segs)
-{
-	const struct bio_vec *bvecs = iter->bvec;
-	unsigned int nbv = iter->nr_segs, ix = 0, nsegs = 0;
-	size_t len, span = 0, n = iter->count;
-	size_t skip = iter->iov_offset + start_offset;
-
-	if (WARN_ON(!iov_iter_is_bvec(iter)) ||
-	    WARN_ON(start_offset > n) ||
-	    n == 0)
-		return 0;
-
-	while (n && ix < nbv && skip) {
-		len = bvecs[ix].bv_len;
-		if (skip < len)
-			break;
-		skip -= len;
-		n -= len;
-		ix++;
-	}
-
-	while (n && ix < nbv) {
-		len = min3(n, bvecs[ix].bv_len - skip, max_size);
-		span += len;
-		nsegs++;
-		ix++;
-		if (span >= max_size || nsegs >= max_segs)
-			break;
-		skip = 0;
-		n -= len;
-	}
-
-	return min(span, max_size);
-}
-
-/*
- * Select the span of a kvec iterator we're going to use.  Limit it by both
- * maximum size and maximum number of segments.  Returns the size of the span
- * in bytes.
- */
-static size_t netfs_limit_kvec(const struct iov_iter *iter, size_t start_offset,
-			       size_t max_size, size_t max_segs)
-{
-	const struct kvec *kvecs = iter->kvec;
-	unsigned int nkv = iter->nr_segs, ix = 0, nsegs = 0;
-	size_t len, span = 0, n = iter->count;
-	size_t skip = iter->iov_offset + start_offset;
-
-	if (WARN_ON(!iov_iter_is_kvec(iter)) ||
-	    WARN_ON(start_offset > n) ||
-	    n == 0)
-		return 0;
-
-	while (n && ix < nkv && skip) {
-		len = kvecs[ix].iov_len;
-		if (skip < len)
-			break;
-		skip -= len;
-		n -= len;
-		ix++;
-	}
-
-	while (n && ix < nkv) {
-		len = min3(n, kvecs[ix].iov_len - skip, max_size);
-		span += len;
-		nsegs++;
-		ix++;
-		if (span >= max_size || nsegs >= max_segs)
-			break;
-		skip = 0;
-		n -= len;
-	}
-
-	return min(span, max_size);
-}
-
-/*
- * Select the span of an xarray iterator we're going to use.  Limit it by both
- * maximum size and maximum number of segments.  It is assumed that segments
- * can be larger than a page in size, provided they're physically contiguous.
- * Returns the size of the span in bytes.
- */
-static size_t netfs_limit_xarray(const struct iov_iter *iter, size_t start_offset,
-				 size_t max_size, size_t max_segs)
-{
-	struct folio *folio;
-	unsigned int nsegs = 0;
-	loff_t pos = iter->xarray_start + iter->iov_offset;
-	pgoff_t index = pos / PAGE_SIZE;
-	size_t span = 0, n = iter->count;
-
-	XA_STATE(xas, iter->xarray, index);
-
-	if (WARN_ON(!iov_iter_is_xarray(iter)) ||
-	    WARN_ON(start_offset > n) ||
-	    n == 0)
-		return 0;
-	max_size = min(max_size, n - start_offset);
-
-	rcu_read_lock();
-	xas_for_each(&xas, folio, ULONG_MAX) {
-		size_t offset, flen, len;
-		if (xas_retry(&xas, folio))
-			continue;
-		if (WARN_ON(xa_is_value(folio)))
-			break;
-		if (WARN_ON(folio_test_hugetlb(folio)))
-			break;
-
-		flen = folio_size(folio);
-		offset = offset_in_folio(folio, pos);
-		len = min(max_size, flen - offset);
-		span += len;
-		nsegs++;
-		if (span >= max_size || nsegs >= max_segs)
-			break;
-	}
-
-	rcu_read_unlock();
-	return min(span, max_size);
-}
-
-/*
- * Select the span of a folio queue iterator we're going to use.  Limit it by
- * both maximum size and maximum number of segments.  Returns the size of the
- * span in bytes.
- */
-static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offset,
-				 size_t max_size, size_t max_segs)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	unsigned int nsegs = 0;
-	unsigned int slot = iter->folioq_slot;
-	size_t span = 0, n = iter->count;
-
-	if (WARN_ON(!iov_iter_is_folioq(iter)) ||
-	    WARN_ON(start_offset > n) ||
-	    n == 0)
-		return 0;
-	max_size = umin(max_size, n - start_offset);
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		slot = 0;
-	}
-
-	start_offset += iter->iov_offset;
-	do {
-		size_t flen = folioq_folio_size(folioq, slot);
-
-		if (start_offset < flen) {
-			span += flen - start_offset;
-			nsegs++;
-			start_offset = 0;
-		} else {
-			start_offset -= flen;
-		}
-		if (span >= max_size || nsegs >= max_segs)
-			break;
-
-		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = folioq->next;
-			slot = 0;
-		}
-	} while (folioq);
-
-	return umin(span, max_size);
-}
-
-size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
-			size_t max_size, size_t max_segs)
-{
-	if (iov_iter_is_folioq(iter))
-		return netfs_limit_folioq(iter, start_offset, max_size, max_segs);
-	if (iov_iter_is_bvec(iter))
-		return netfs_limit_bvec(iter, start_offset, max_size, max_segs);
-	if (iov_iter_is_xarray(iter))
-		return netfs_limit_xarray(iter, start_offset, max_size, max_segs);
-	if (iov_iter_is_kvec(iter))
-		return netfs_limit_kvec(iter, start_offset, max_size, max_segs);
-	BUG();
-}
-EXPORT_SYMBOL(netfs_limit_iter);
-#endif
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 041810707eb4..5d8b87f71888 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -29,7 +29,6 @@ static struct kmem_cache *netfs_subrequest_slab;
 mempool_t netfs_request_pool;
 mempool_t netfs_subrequest_pool;
 mempool_t netfs_bvecq_pool;
-mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
 LIST_HEAD(netfs_io_requests);
diff --git a/fs/netfs/rolling_buffer.c b/fs/netfs/rolling_buffer.c
deleted file mode 100644
index 4e8584725b6a..000000000000
--- a/fs/netfs/rolling_buffer.c
+++ /dev/null
@@ -1,301 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* Rolling buffer helpers
- *
- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells at redhat.com)
- */
-
-#include <linux/bitops.h>
-#include <linux/mempool.h>
-#include <linux/pagemap.h>
-#include <linux/rolling_buffer.h>
-#include <linux/slab.h>
-#include "internal.h"
-
-static atomic_t debug_ids;
-
-/**
- * netfs_folioq_alloc - Allocate a folio_queue struct
- * @rreq_id: Associated debugging ID for tracing purposes
- * @gfp: Allocation constraints
- * @trace: Trace tag to indicate the purpose of the allocation
- *
- * Allocate, initialise and account the folio_queue struct and log a trace line
- * to mark the allocation.
- */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
-				       unsigned int /*enum netfs_folioq_trace*/ trace)
-{
-	struct folio_queue *fq;
-
-	if (gfp == GFP_KERNEL)
-		fq = netfs_folioq_pool.alloc(gfp, netfs_folioq_pool.pool_data);
-	else
-		fq = mempool_alloc(&netfs_folioq_pool, gfp);
-	if (fq) {
-		netfs_stat(&netfs_n_folioq);
-		folioq_init(fq, rreq_id);
-		fq->debug_id = atomic_inc_return(&debug_ids);
-		trace_netfs_folioq(fq, trace);
-	}
-	return fq;
-}
-EXPORT_SYMBOL(netfs_folioq_alloc);
-
-/**
- * netfs_folioq_free - Free a folio_queue struct
- * @folioq: The object to free
- * @trace: Trace tag to indicate which free
- *
- * Free and unaccount the folio_queue struct.
- */
-void netfs_folioq_free(struct folio_queue *folioq,
-		       unsigned int /*enum netfs_trace_folioq*/ trace)
-{
-	trace_netfs_folioq(folioq, trace);
-	netfs_stat_d(&netfs_n_folioq);
-	mempool_free(folioq, &netfs_folioq_pool);
-}
-EXPORT_SYMBOL(netfs_folioq_free);
-
-/*
- * Initialise a rolling buffer.  We allocate an empty folio queue struct to so
- * that the pointers can be independently driven by the producer and the
- * consumer.
- */
-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
-			unsigned int direction, gfp_t gfp)
-{
-	struct folio_queue *fq;
-
-	fq = netfs_folioq_alloc(rreq_id, gfp, netfs_trace_folioq_rollbuf_init);
-	if (!fq)
-		return -ENOMEM;
-
-	roll->head = fq;
-	roll->tail = fq;
-	iov_iter_folio_queue(&roll->iter, direction, fq, 0, 0, 0);
-	return 0;
-}
-
-/*
- * Add another folio_queue to a rolling buffer if there's no space left.
- */
-int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
-{
-	struct folio_queue *fq, *head = roll->head;
-
-	if (!folioq_full(head))
-		return 0;
-
-	fq = netfs_folioq_alloc(head->rreq_id, gfp, netfs_trace_folioq_make_space);
-	if (!fq)
-		return -ENOMEM;
-	fq->prev = head;
-
-	roll->head = fq;
-	if (folioq_full(head)) {
-		/* Make sure we don't leave the master iterator pointing to a
-		 * block that might get immediately consumed.
-		 */
-		if (roll->iter.folioq == head &&
-		    roll->iter.folioq_slot == folioq_nr_slots(head)) {
-			roll->iter.folioq = fq;
-			roll->iter.folioq_slot = 0;
-		}
-	}
-
-	/* Make sure the initialisation is stored before the next pointer.
-	 *
-	 * [!] NOTE: After we set head->next, the consumer is at liberty to
-	 * immediately delete the old head.
-	 */
-	smp_store_release(&head->next, fq);
-	return 0;
-}
-
-/*
- * Decant the list of folios to read into a rolling buffer.
- */
-ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
-				    struct readahead_control *ractl,
-				    struct folio_batch *put_batch)
-{
-	struct folio_queue *fq;
-	struct page **vec;
-	int nr, ix, to;
-	ssize_t size = 0;
-
-	if (rolling_buffer_make_space(roll, GFP_KERNEL) < 0)
-		return -ENOMEM;
-
-	fq = roll->head;
-	vec = (struct page **)fq->vec.folios;
-	nr = __readahead_batch(ractl, vec + folio_batch_count(&fq->vec),
-			       folio_batch_space(&fq->vec));
-	ix = fq->vec.nr;
-	to = ix + nr;
-	fq->vec.nr = to;
-	for (; ix < to; ix++) {
-		struct folio *folio = folioq_folio(fq, ix);
-		unsigned int order = folio_order(folio);
-
-		fq->orders[ix] = order;
-		size += PAGE_SIZE << order;
-		trace_netfs_folio(folio, netfs_folio_trace_read);
-		if (!folio_batch_add(put_batch, folio))
-			folio_batch_release(put_batch);
-	}
-	WRITE_ONCE(roll->iter.count, roll->iter.count + size);
-
-	/* Store the counter after setting the slot. */
-	smp_store_release(&roll->next_head_slot, to);
-	return size;
-}
-
-/*
- * Decant the entire list of folios to read into a rolling buffer.
- */
-ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
-					 struct readahead_control *ractl,
-					 unsigned int rreq_id)
-{
-	XA_STATE(xas, &ractl->mapping->i_pages, ractl->_index);
-	struct folio_queue *fq;
-	struct folio *folio;
-	ssize_t loaded = 0;
-	int nr, slot = 0, npages = 0;
-
-	/* First allocate all the folioqs we're going to need to avoid having
-	 * to deal with ENOMEM later.
-	 */
-	nr = ractl->_nr_folios;
-	do {
-		fq = netfs_folioq_alloc(rreq_id, GFP_KERNEL,
-					netfs_trace_folioq_make_space);
-		if (!fq) {
-			rolling_buffer_clear(roll);
-			return -ENOMEM;
-		}
-		fq->prev = roll->head;
-		if (!roll->tail)
-			roll->tail = fq;
-		else
-			roll->head->next = fq;
-		roll->head = fq;
-
-		nr -= folioq_nr_slots(fq);
-	} while (nr > 0);
-
-	rcu_read_lock();
-
-	fq = roll->tail;
-	xas_for_each(&xas, folio, ractl->_index + ractl->_nr_pages - 1) {
-		unsigned int order;
-
-		if (xas_retry(&xas, folio))
-			continue;
-		VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
-
-		order = folio_order(folio);
-		fq->orders[slot] = order;
-		fq->vec.folios[slot] = folio;
-		loaded += PAGE_SIZE << order;
-		npages += 1 << order;
-		trace_netfs_folio(folio, netfs_folio_trace_read);
-
-		slot++;
-		if (slot >= folioq_nr_slots(fq)) {
-			fq->vec.nr = slot;
-			fq = fq->next;
-			if (!fq) {
-				WARN_ON_ONCE(npages < readahead_count(ractl));
-				break;
-			}
-			slot = 0;
-		}
-	}
-
-	rcu_read_unlock();
-
-	if (fq)
-		fq->vec.nr = slot;
-
-	WRITE_ONCE(roll->iter.count, loaded);
-	iov_iter_folio_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
-	ractl->_index    += npages;
-	ractl->_nr_pages -= npages;
-	return loaded;
-}
-
-/*
- * Append a folio to the rolling buffer.
- */
-ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
-			      unsigned int flags, gfp_t gfp)
-{
-	ssize_t size = folio_size(folio);
-	int slot;
-
-	if (rolling_buffer_make_space(roll, gfp) < 0)
-		return -ENOMEM;
-
-	slot = folioq_append(roll->head, folio);
-	if (flags & ROLLBUF_MARK_1)
-		folioq_mark(roll->head, slot);
-	if (flags & ROLLBUF_MARK_2)
-		folioq_mark2(roll->head, slot);
-
-	WRITE_ONCE(roll->iter.count, roll->iter.count + size);
-
-	/* Store the counter after setting the slot. */
-	smp_store_release(&roll->next_head_slot, slot);
-	return size;
-}
-
-/*
- * Delete a spent buffer from a rolling queue and return the next in line.  We
- * don't return the last buffer to keep the pointers independent, but return
- * NULL instead.
- */
-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll)
-{
-	struct folio_queue *spent = roll->tail, *next = READ_ONCE(spent->next);
-
-	if (!next)
-		return NULL;
-	next->prev = NULL;
-	netfs_folioq_free(spent, netfs_trace_folioq_delete);
-	roll->tail = next;
-	return next;
-}
-
-/*
- * Clear out a rolling queue.  Folios that have mark 1 set are put.
- */
-void rolling_buffer_clear(struct rolling_buffer *roll)
-{
-	struct folio_batch fbatch;
-	struct folio_queue *p;
-
-	folio_batch_init(&fbatch);
-
-	while ((p = roll->tail)) {
-		roll->tail = p->next;
-		for (int slot = 0; slot < folioq_count(p); slot++) {
-			struct folio *folio = folioq_folio(p, slot);
-
-			if (!folio)
-				continue;
-			if (folioq_is_marked(p, slot)) {
-				trace_netfs_folio(folio, netfs_folio_trace_put);
-				if (!folio_batch_add(&fbatch, folio))
-					folio_batch_release(&fbatch);
-			}
-		}
-
-		netfs_folioq_free(p, netfs_trace_folioq_clear);
-	}
-
-	folio_batch_release(&fbatch);
-}
diff --git a/include/linux/folio_queue.h b/include/linux/folio_queue.h
deleted file mode 100644
index f6d5f1f127c9..000000000000
--- a/include/linux/folio_queue.h
+++ /dev/null
@@ -1,282 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/* Queue of folios definitions
- *
- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells at redhat.com)
- *
- * See:
- *
- *	Documentation/core-api/folio_queue.rst
- *
- * for a description of the API.
- */
-
-#ifndef _LINUX_FOLIO_QUEUE_H
-#define _LINUX_FOLIO_QUEUE_H
-
-#include <linux/folio_batch.h>
-#include <linux/mm.h>
-
-/*
- * Segment in a queue of running buffers.  Each segment can hold a number of
- * folios and a portion of the queue can be referenced with the ITER_FOLIOQ
- * iterator.  The possibility exists of inserting non-folio elements into the
- * queue (such as gaps).
- *
- * Explicit prev and next pointers are used instead of a list_head to make it
- * easier to add segments to tail and remove them from the head without the
- * need for a lock.
- */
-struct folio_queue {
-	struct folio_batch	vec;		/* Folios in the queue segment */
-	u8			orders[FOLIO_BATCH_SIZE]; /* Order of each folio */
-	struct folio_queue	*next;		/* Next queue segment or NULL */
-	struct folio_queue	*prev;		/* Previous queue segment of NULL */
-	unsigned long		marks;		/* 1-bit mark per folio */
-	unsigned long		marks2;		/* Second 1-bit mark per folio */
-#if FOLIO_BATCH_SIZE > BITS_PER_LONG
-#error marks is not big enough
-#endif
-	unsigned int		rreq_id;
-	unsigned int		debug_id;
-};
-
-/**
- * folioq_init - Initialise a folio queue segment
- * @folioq: The segment to initialise
- * @rreq_id: The request identifier to use in tracelines.
- *
- * Initialise a folio queue segment and set an identifier to be used in traces.
- *
- * Note that the folio pointers are left uninitialised.
- */
-static inline void folioq_init(struct folio_queue *folioq, unsigned int rreq_id)
-{
-	folio_batch_init(&folioq->vec);
-	folioq->next = NULL;
-	folioq->prev = NULL;
-	folioq->marks = 0;
-	folioq->marks2 = 0;
-	folioq->rreq_id = rreq_id;
-	folioq->debug_id = 0;
-}
-
-/**
- * folioq_nr_slots: Query the capacity of a folio queue segment
- * @folioq: The segment to query
- *
- * Query the number of folios that a particular folio queue segment might hold.
- * [!] NOTE: This must not be assumed to be the same for every segment!
- */
-static inline unsigned int folioq_nr_slots(const struct folio_queue *folioq)
-{
-	return FOLIO_BATCH_SIZE;
-}
-
-/**
- * folioq_count: Query the occupancy of a folio queue segment
- * @folioq: The segment to query
- *
- * Query the number of folios that have been added to a folio queue segment.
- * Note that this is not decreased as folios are removed from a segment.
- */
-static inline unsigned int folioq_count(struct folio_queue *folioq)
-{
-	return folio_batch_count(&folioq->vec);
-}
-
-/**
- * folioq_full: Query if a folio queue segment is full
- * @folioq: The segment to query
- *
- * Query if a folio queue segment is fully occupied.  Note that this does not
- * change if folios are removed from a segment.
- */
-static inline bool folioq_full(struct folio_queue *folioq)
-{
-	//return !folio_batch_space(&folioq->vec);
-	return folioq_count(folioq) >= folioq_nr_slots(folioq);
-}
-
-/**
- * folioq_is_marked: Check first folio mark in a folio queue segment
- * @folioq: The segment to query
- * @slot: The slot number of the folio to query
- *
- * Determine if the first mark is set for the folio in the specified slot in a
- * folio queue segment.
- */
-static inline bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot)
-{
-	return test_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_mark: Set the first mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Set the first mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_mark(struct folio_queue *folioq, unsigned int slot)
-{
-	set_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_unmark: Clear the first mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Clear the first mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_unmark(struct folio_queue *folioq, unsigned int slot)
-{
-	clear_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_is_marked2: Check second folio mark in a folio queue segment
- * @folioq: The segment to query
- * @slot: The slot number of the folio to query
- *
- * Determine if the second mark is set for the folio in the specified slot in a
- * folio queue segment.
- */
-static inline bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot)
-{
-	return test_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_mark2: Set the second mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Set the second mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_mark2(struct folio_queue *folioq, unsigned int slot)
-{
-	set_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_unmark2: Clear the second mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Clear the second mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_unmark2(struct folio_queue *folioq, unsigned int slot)
-{
-	clear_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_append: Add a folio to a folio queue segment
- * @folioq: The segment to add to
- * @folio: The folio to add
- *
- * Add a folio to the tail of the sequence in a folio queue segment, increasing
- * the occupancy count and returning the slot number for the folio just added.
- * The folio size is extracted and stored in the queue and the marks are left
- * unmodified.
- *
- * Note that it's left up to the caller to check that the segment capacity will
- * not be exceeded and to extend the queue.
- */
-static inline unsigned int folioq_append(struct folio_queue *folioq, struct folio *folio)
-{
-	unsigned int slot = folioq->vec.nr++;
-
-	folioq->vec.folios[slot] = folio;
-	folioq->orders[slot] = folio_order(folio);
-	return slot;
-}
-
-/**
- * folioq_append_mark: Add a folio to a folio queue segment
- * @folioq: The segment to add to
- * @folio: The folio to add
- *
- * Add a folio to the tail of the sequence in a folio queue segment, increasing
- * the occupancy count and returning the slot number for the folio just added.
- * The folio size is extracted and stored in the queue, the first mark is set
- * and and the second and third marks are left unmodified.
- *
- * Note that it's left up to the caller to check that the segment capacity will
- * not be exceeded and to extend the queue.
- */
-static inline unsigned int folioq_append_mark(struct folio_queue *folioq, struct folio *folio)
-{
-	unsigned int slot = folioq->vec.nr++;
-
-	folioq->vec.folios[slot] = folio;
-	folioq->orders[slot] = folio_order(folio);
-	folioq_mark(folioq, slot);
-	return slot;
-}
-
-/**
- * folioq_folio: Get a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the folio in the specified slot from a folio queue segment.  Note
- * that no bounds check is made and if the slot hasn't been added into yet, the
- * pointer will be undefined.  If the slot has been cleared, NULL will be
- * returned.
- */
-static inline struct folio *folioq_folio(const struct folio_queue *folioq, unsigned int slot)
-{
-	return folioq->vec.folios[slot];
-}
-
-/**
- * folioq_folio_order: Get the order of a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the order of the folio in the specified slot from a folio queue
- * segment.  Note that no bounds check is made and if the slot hasn't been
- * added into yet, the order returned will be 0.
- */
-static inline unsigned int folioq_folio_order(const struct folio_queue *folioq, unsigned int slot)
-{
-	return folioq->orders[slot];
-}
-
-/**
- * folioq_folio_size: Get the size of a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the size of the folio in the specified slot from a folio queue
- * segment.  Note that no bounds check is made and if the slot hasn't been
- * added into yet, the size returned will be PAGE_SIZE.
- */
-static inline size_t folioq_folio_size(const struct folio_queue *folioq, unsigned int slot)
-{
-	return PAGE_SIZE << folioq_folio_order(folioq, slot);
-}
-
-/**
- * folioq_clear: Clear a folio from a folio queue segment
- * @folioq: The segment to clear
- * @slot: The folio slot to clear
- *
- * Clear a folio from a sequence in a folio queue segment and clear its marks.
- * The occupancy count is left unchanged.
- */
-static inline void folioq_clear(struct folio_queue *folioq, unsigned int slot)
-{
-	folioq->vec.folios[slot] = NULL;
-	folioq_unmark(folioq, slot);
-	folioq_unmark2(folioq, slot);
-}
-
-#endif /* _LINUX_FOLIO_QUEUE_H */
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 85e4d49479eb..fc316682dddb 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -457,8 +457,6 @@ void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
 			   unsigned long long fpos, struct bvecq **_bvecq_head,
 			   iov_iter_extraction_t extraction_flags, gfp_t gfp);
-size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
-			size_t max_size, size_t max_segs);
 void netfs_prepare_write_failed(struct netfs_io_subrequest *subreq);
 void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error);
 
diff --git a/include/linux/rolling_buffer.h b/include/linux/rolling_buffer.h
deleted file mode 100644
index 1e37750e4ff8..000000000000
--- a/include/linux/rolling_buffer.h
+++ /dev/null
@@ -1,64 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/* Rolling buffer of folios
- *
- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells at redhat.com)
- */
-
-#ifndef _ROLLING_BUFFER_H
-#define _ROLLING_BUFFER_H
-
-#include <linux/folio_queue.h>
-#include <linux/uio.h>
-
-/*
- * Rolling buffer.  Whilst the buffer is live and in use, folios and folio
- * queue segments can be added to one end by one thread and removed from the
- * other end by another thread.  The buffer isn't allowed to be empty; it must
- * always have at least one folio_queue in it so that neither side has to
- * modify both queue pointers.
- *
- * The iterator in the buffer is extended as buffers are inserted.  It can be
- * snapshotted to use a segment of the buffer.
- */
-struct rolling_buffer {
-	struct folio_queue	*head;		/* Producer's insertion point */
-	struct folio_queue	*tail;		/* Consumer's removal point */
-	struct iov_iter		iter;		/* Iterator tracking what's left in the buffer */
-	u8			next_head_slot;	/* Next slot in ->head */
-	u8			first_tail_slot; /* First slot in ->tail */
-};
-
-/*
- * Snapshot of a rolling buffer.
- */
-struct rolling_buffer_snapshot {
-	struct folio_queue	*curr_folioq;	/* Queue segment in which current folio resides */
-	unsigned char		curr_slot;	/* Folio currently being read */
-	unsigned char		curr_order;	/* Order of folio */
-};
-
-/* Marks to store per-folio in the internal folio_queue structs. */
-#define ROLLBUF_MARK_1	BIT(0)
-#define ROLLBUF_MARK_2	BIT(1)
-
-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
-			unsigned int direction, gfp_t gfp);
-int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp);
-ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
-				    struct readahead_control *ractl,
-				    struct folio_batch *put_batch);
-ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
-					 struct readahead_control *ractl,
-					 unsigned int rreq_id);
-ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
-			      unsigned int flags, gfp_t gfp);
-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll);
-void rolling_buffer_clear(struct rolling_buffer *roll);
-
-static inline void rolling_buffer_advance(struct rolling_buffer *roll, size_t amount)
-{
-	iov_iter_advance(&roll->iter, amount);
-}
-
-#endif /* _ROLLING_BUFFER_H */
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 7e25051dc1ea..b9558a0c2c1e 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -6742,8 +6742,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
 	/* amdgpu */
 	{ "amdgpu_vm_bo_map", 0x1 },
 	{ "amdgpu_vm_bo_unmap", 0x1 },
-	/* netfs */
-	{ "netfs_folioq", 0x1 },
 	/* xfs from xfs_defer_pending_class */
 	{ "xfs_defer_create_intent", 0x1 },
 	{ "xfs_defer_cancel_list", 0x1 },



From dhowells at redhat.com  Tue Aug  4 03:02:18 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:18 +0100
Subject: [PATCH v8 23/25] netfs: Limit the minimum trigger for progress reporting
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-24-dhowells@redhat.com>

For really big read RPC ops that span multiple folios, netfslib allows the
filesystem to give progress notifications to wake up the collector thread
to do a collection of folios that have now been fetched, even if the RPC is
still ongoing, thereby allowing the application to make progress.

The trigger for this is that at least one folio has been downloaded since
the clean point.  If, however, the folios are small, this means the
collector thread is constantly being woken up - which has a negative
performance impact on the system.

Set a minimum trigger of 256KiB or the size of the folio at the front of
the queue, whichever is larger.

Also, fix the base to be the stream collection point, not the point at
which the collector has cleaned up to (which is currently 0 until something
has been collected).

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/read_collect.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 8ab27103a86d..408208f29c0c 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -528,15 +528,15 @@ void netfs_read_collection_worker(struct work_struct *work)
 void netfs_read_subreq_progress(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
-	struct netfs_io_stream *stream = &rreq->io_streams[0];
-	size_t fsize = PAGE_SIZE << rreq->front_folio_order;
+	struct netfs_io_stream *stream = &rreq->io_streams[subreq->stream_nr];
+	size_t fsize = umax(PAGE_SIZE << rreq->front_folio_order, 256 * 1024);
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_progress);
 
 	/* If we are at the head of the queue, wake up the collector,
 	 * getting a ref to it if we were the ones to do so.
 	 */
-	if (subreq->start + subreq->transferred > rreq->cleaned_to + fsize &&
+	if (subreq->start + subreq->transferred >= stream->collected_to + fsize &&
 	    (rreq->origin == NETFS_READAHEAD ||
 	     rreq->origin == NETFS_READPAGE ||
 	     rreq->origin == NETFS_READ_FOR_WRITE) &&



From dhowells at redhat.com  Tue Aug  4 03:02:17 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:17 +0100
Subject: [PATCH v8 22/25] netfs: Check for too much data being read
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-23-dhowells@redhat.com>

Put in a check in read subreq termination to detect more data being read
for a subrequest than was requested.  In the event that this happens, abort
the rest of the read request on the basis that some of the read buffer may
have been corrupted and return -EIO.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/read_collect.c      | 57 ++++++++++++++++++++++++++++++++++--
 include/linux/netfs.h        |  1 +
 include/trace/events/netfs.h |  1 +
 3 files changed, 56 insertions(+), 3 deletions(-)

diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index e0fe7b13dd7a..8ab27103a86d 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -19,8 +19,9 @@
 #define MADE_PROGRESS	0x04	/* Made progress cleaning up a stream or the folio set */
 #define BUFFERED	0x08	/* The pagecache needs cleaning up */
 #define NEED_RETRY	0x10	/* A front op requests retrying */
-#define COPY_TO_CACHE	0x40	/* Need to copy subrequest to cache */
-#define ABANDON_SREQ	0x80	/* Need to abandon untransferred part of subrequest */
+#define COPY_TO_CACHE	0x20	/* Need to copy subrequest to cache */
+#define ABANDON_SREQ	0x40	/* Need to abandon untransferred part of subrequest */
+#define ABANDON_RREQ	0x80	/* Need to abandon the rest of a request */
 
 /*
  * Clear the unread part of an I/O request.
@@ -201,6 +202,8 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 		notes = BUFFERED;
 	else
 		notes = 0;
+	if (test_bit(NETFS_RREQ_ABANDON_REQ, &rreq->flags))
+		notes |= ABANDON_RREQ;
 
 	/* Remove completed subrequests from the front of the stream and
 	 * advance the completion point.  We stop when we hit something that's
@@ -226,14 +229,44 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 		if (netfs_check_subreq_in_progress(front))
 			notes |= HIT_PENDING;
 		smp_rmb(); /* Read counters after IN_PROGRESS flag. */
+
 		transferred = READ_ONCE(front->transferred);
+		if (unlikely(transferred > front->len)) {
+			/* Ugh...  A subreq overran its allotted length.  It
+			 * may have corrupted the read buffer.
+			 */
+			set_bit(NETFS_RREQ_FAILED, &rreq->flags);
+			set_bit(NETFS_RREQ_ABANDON_REQ, &rreq->flags);
+			notes |= ABANDON_RREQ;
+			netfs_wake_rreq_flag(rreq, NETFS_RREQ_PAUSE, netfs_rreq_trace_unpause);
+		}
+
+		/* Deal with an abandoned request.  Wait for each subreq to
+		 * complete before abandoning them.
+		 */
+		if (unlikely(notes & ABANDON_RREQ)) {
+			if (notes & HIT_PENDING)
+				break;
+
+			/* The subreq now belongs to us. */
+			stream->error = -EIO;
+			stream->failed = true;
+			rreq->abandon_to = front->start + front->len;
+			rreq->error = -EIO;
+			transferred = 0;
+			trace_netfs_rreq(rreq, netfs_rreq_trace_set_abandon);
+			notes |= ABANDON_SREQ;
+			goto sreq_complete;
+		}
 
 		/* If we can now collect the next folio, do so.  We don't want
 		 * to defer this as we have to decide whether we need to copy
 		 * to the cache or not, and that may differ between adjacent
 		 * subreqs.
 		 */
-		if (notes & BUFFERED) {
+		if (notes & ABANDON_SREQ) {
+			/* Don't collect anything. */
+		} else if (notes & BUFFERED) {
 			size_t fsize = PAGE_SIZE << rreq->front_folio_order;
 
 			/* Clear the tail of a short read. */
@@ -295,6 +328,7 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 			notes |= MADE_PROGRESS;
 		}
 
+sreq_complete:
 		/* Remove if completely consumed. */
 		stream->source = front->source;
 		spin_lock(&rreq->lock);
@@ -319,6 +353,8 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 	if (!(notes & BUFFERED))
 		rreq->cleaned_to = rreq->collected_to;
 
+	if (notes & ABANDON_RREQ)
+		goto out;
 	if (notes & NEED_RETRY)
 		goto need_retry;
 	if (notes & MADE_PROGRESS) {
@@ -543,6 +579,21 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq)
 		break;
 	}
 
+	/* If the subrequest read more than it was supposed to, abort
+	 * the request with EIO as we may have clobbered other parts
+	 * of the buffer that are already read.
+	 */
+	if (subreq->transferred > subreq->len) {
+		trace_netfs_sreq(subreq, netfs_sreq_trace_too_much);
+		__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
+		__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+		subreq->error = -EIO;
+		trace_netfs_failure(rreq, subreq, subreq->error, netfs_fail_read);
+		trace_netfs_rreq(rreq, netfs_rreq_trace_set_pause);
+		set_bit(NETFS_RREQ_PAUSE, &rreq->flags);
+		goto skip_error_checks;
+	}
+
 	/* Deal with retry requests, short reads and errors.  If we retry
 	 * but don't make progress, we abandon the attempt.
 	 */
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index fc316682dddb..856a6ba3fc71 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -281,6 +281,7 @@ struct netfs_io_request {
 #define NETFS_RREQ_FAILED		3	/* The request failed */
 #define NETFS_RREQ_RETRYING		4	/* Set if we're in the retry path */
 #define NETFS_RREQ_SHORT_TRANSFER	5	/* Set if we have a short transfer */
+#define NETFS_RREQ_ABANDON_REQ		6	/* Set if the request is to be abandoned */
 #define NETFS_RREQ_CACHE_STOP		8	/* Set to stop caching (ENOBUFS or error) */
 #define NETFS_RREQ_CACHE_ERROR		9	/* Set if we got an error from the cache */
 #define NETFS_RREQ_OFFLOAD_COLLECTION	12	/* Offload collection to workqueue */
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 071d20e80f13..723cb7315308 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -132,6 +132,7 @@
 	EM(netfs_sreq_trace_submit,		"SUBMT")	\
 	EM(netfs_sreq_trace_superfluous,	"SPRFL")	\
 	EM(netfs_sreq_trace_terminated,		"TERM ")	\
+	EM(netfs_sreq_trace_too_much,		"!TOOM")	\
 	EM(netfs_sreq_trace_wait_for,		"_WAIT")	\
 	EM(netfs_sreq_trace_write,		"WRITE")	\
 	EM(netfs_sreq_trace_write_skip,		"SKIP ")	\



From dhowells at redhat.com  Tue Aug  4 03:02:19 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:19 +0100
Subject: [PATCH v8 24/25] netfs: Combine prepare and issue ops and grab the buffers on request
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-25-dhowells@redhat.com>

Modify the way subrequests are generated in netfslib to try and simplify
the code.  The issue, primarily, is in writeback: the code has to create
multiple streams of write requests to disparate targets with different
properties (e.g. server and fscache), where not every folio needs to go to
every target (e.g. data just read from the server may only need writing to
the cache).

The current model in writeback, at least, is to go carefully through every
folio, preparing a subrequest for each stream when it was detected that
part of the current folio needed to go to that stream, and repeating this
within and across contiguous folios; then to issue subrequests as they
become full or hit boundaries after first setting up the buffer.  However,
this is quite difficult to follow - and makes it tricky to handle
discontiguous folios in a request.

This is changed such that netfs now accumulates buffers and attaches them
to each stream when they become valid for that stream, then flushes the
stream when a limit or a boundary is hit.  The issuing code in netfs then
loops around creating and issuing subrequests without calling a separate
prepare stage (though a function is provided to get an estimate of when
flushing should occur).  The filesystem (or cache) then gets to take a
slice of the master bvec chain as its I/O buffer for each subrequest,
including discontiguities if it can support a sparse/vectored RPC (as Ceph
can).

Similar-ish changes also apply to buffered read and unbuffered read and
write, though in each of those cases there is only a single contiguous
stream.  Though for buffered read this consists of interwoven requests from
multiple sources (server or cache).

To this end, netfslib is changed in the following ways:

 (1) ->prepare_xxx(), buffer selection and ->issue_xxx() are now collapsed
     together such that one ->issue_xxx() call is made with the subrequest
     defined to the maximum extent; the filesystem/cache then reduces the
     length of the subrequest and calls back to netfslib to grab a slice of
     the buffer, which may reduce the subrequest further if a maximum
     segment limit is set.  The filesystem/cache then dispatches the
     operation.

 (2) Retry buffer tracking is added to the netfs_io_request struct.  This
     is then selected by the subrequest retry counter being non-zero.

 (3) The use of iov_iter is pushed down to the filesystem.  Netfslib now
     provides the filesystem with a bvecq holding the buffer rather than an
     iov_iter.  The bvecq can be duplicated and headers/trailers attached
     to hold protocol and several bvecqs can be linked together to create a
     compound operation.

 (4) If the ->issue_xxx() functions terminate with -ENOMEM, a flag is set
     on the request to abort further subrequest generation/retrying.  If
     it fails early with any other error, generally the subreq will be
     extended as much as possible and failed with the error returned.

 (5) During writeback, netfslib now builds up an accumulation of buffered
     data before issuing writes on each stream (one server, one cache).  It
     asks each stream for an estimate of how much data to accumulate before
     it next generates subrequests on the stream.  The filesystem or cache
     is not required to use up all the data accumulated on a stream at that
     time unless the end of the pagecache is hit.

 (6) During read-gaps, in which there are two gaps on either end of a dirty
     streaming write page that need to be filled, a buffer is constructed
     consisting of the two ends plus a sink page repeated to cover the
     middle portion.  This is passed to the server as a single write.  For
     something like Ceph, this should probably be done either as a
     vectored/sparse read or as two separate reads (if different Ceph
     objects are involved).

 (7) During unbuffered/DIO read/write, there is a single contiguous file
     region to be read or written as a single stream.  The dispatching
     function just creates subrequests and calls ->issue_xxx() repeatedly
     to eat through the bufferage.

 (8) At the start of buffered read, the entire set of folios allocated by
     VM readahead is loaded into a bvecq chain, rather than trying to do it
     piecemeal as-needed.  As the pages were already added and locked by
     the VM, this is slightly more efficient than loading piecemeal as only
     a single iteration of the xarray is required.

 (9) During buffered read, there is a single contiguous file region, to
     read as a single stream - however, this stream may be stitched
     together from subrequests to multiple sources.  Which sources are used
     where is now determined by querying the cache to find the next couple
     of extents in which it has data; netfslib uses this to direct the
     subrequests towards the appropriate sources.

     Each subrequest is given the maximum length in the current extent and
     then ->issue_read() is called.  The filesystem then limits the size
     and slices off a piece of the buffer for that extent.

(10) Cachefiles now provides an estimation function that indicates the
     standard maxima for doing DIO (MAX_RW_COUNT and BIO_MAX_VECS).

Note that sparse cachefiles still rely on the backing filesystem for
content mapping.  That will need to be addressed in a future patch and is
not trivial to fix.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/9p/vfs_addr.c             |  60 ++-
 fs/afs/dir.c                 |  11 +-
 fs/afs/file.c                |  29 +-
 fs/afs/fsclient.c            |   8 +-
 fs/afs/internal.h            |   8 +-
 fs/afs/symlink.c             |   5 +-
 fs/afs/write.c               |  36 +-
 fs/afs/yfsclient.c           |   6 +-
 fs/cachefiles/io.c           | 240 ++++++++---
 fs/ceph/Kconfig              |   1 +
 fs/ceph/addr.c               | 123 +++---
 fs/netfs/Kconfig             |   3 +
 fs/netfs/Makefile            |   2 +-
 fs/netfs/buffered_read.c     | 254 ++++++-----
 fs/netfs/direct_read.c       |  87 ++--
 fs/netfs/direct_write.c      | 135 +++---
 fs/netfs/fscache_io.c        |   2 +-
 fs/netfs/internal.h          |  92 +++-
 fs/netfs/iterator.c          |   4 +-
 fs/netfs/misc.c              |  35 +-
 fs/netfs/objects.c           |   7 +-
 fs/netfs/read_collect.c      |  43 +-
 fs/netfs/read_pgpriv2.c      | 122 ++++--
 fs/netfs/read_retry.c        | 208 ++++-----
 fs/netfs/read_single.c       | 158 ++++---
 fs/netfs/write_collect.c     |  62 ++-
 fs/netfs/write_issue.c       | 816 +++++++++++++++++++++--------------
 fs/netfs/write_retry.c       | 168 ++++----
 fs/nfs/Kconfig               |   1 +
 fs/nfs/fscache.c             |  24 +-
 fs/smb/client/cifssmb.c      |  13 +-
 fs/smb/client/file.c         | 145 +++----
 fs/smb/client/smb2ops.c      |   9 +-
 fs/smb/client/smb2pdu.c      |  28 +-
 fs/smb/client/transport.c    |  15 +-
 include/linux/netfs.h        |  97 +++--
 include/trace/events/netfs.h |  51 ++-
 net/9p/client.c              |   8 +-
 38 files changed, 1899 insertions(+), 1217 deletions(-)

diff --git a/fs/9p/vfs_addr.c b/fs/9p/vfs_addr.c
index 2129fcb0f65c..fc5b9c677986 100644
--- a/fs/9p/vfs_addr.c
+++ b/fs/9p/vfs_addr.c
@@ -48,31 +48,70 @@ static void v9fs_begin_writeback(struct netfs_io_request *wreq)
 	wreq->io_streams[0].avail = true;
 }
 
+/*
+ * Estimate how much data should be accumulated before we start issuing
+ * write subrequests.
+ */
+static int v9fs_estimate_write(struct netfs_io_request *wreq,
+			       struct netfs_io_stream *stream,
+			       struct netfs_write_estimate *estimate)
+{
+	struct p9_fid *fid = wreq->netfs_priv;
+	unsigned long long limit = ULLONG_MAX - stream->issue_from;
+	unsigned long long max_len = fid->clnt->msize - P9_IOHDRSZ;
+
+	estimate->issue_at = stream->issue_from + umin(max_len, limit);
+	return 0;
+}
+
 /*
  * Issue a subrequest to write to the server.
  */
-static void v9fs_issue_write(struct netfs_io_subrequest *subreq)
+static int v9fs_issue_write(struct netfs_io_subrequest *subreq)
 {
+	struct iov_iter iter;
 	struct p9_fid *fid = subreq->rreq->netfs_priv;
-	int err, len;
+	int err, len = 0;
+
+	subreq->len = umin(subreq->len, fid->clnt->msize - P9_IOHDRSZ);
 
-	len = p9_client_write(fid, subreq->start, &subreq->io_iter, &err);
+	err = netfs_prepare_write_buffer(subreq, INT_MAX);
+	if (err < 0)
+		return err;
+	/* After this point, must fail by termination. */
+
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
+	len = p9_client_write(fid, subreq->start, &iter, &err);
 	if (len > 0)
 		__set_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+
 	netfs_write_subrequest_terminated(subreq, len ?: err);
+	return 0;
 }
 
 /**
  * v9fs_issue_read - Issue a read from 9P
  * @subreq: The read to make
  */
-static void v9fs_issue_read(struct netfs_io_subrequest *subreq)
+static int v9fs_issue_read(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
+	struct iov_iter iter;
 	struct p9_fid *fid = rreq->netfs_priv;
 	char *target;
 	unsigned long long pos = subreq->start + subreq->transferred;
-	int total = 0, err, len, n;
+	size_t len;
+	int total = 0, err;
+
+	err = netfs_prepare_read_buffer(subreq, INT_MAX);
+	if (err < 0)
+		return err;
+	/* After this point, must fail by termination. */
+
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
 
 	if (S_ISLNK(rreq->inode->i_mode)) {
 		/* p9_client_readlink() must not be called for legacy protocols
@@ -89,12 +128,13 @@ static void v9fs_issue_read(struct netfs_io_subrequest *subreq)
 		err = p9_client_readlink(fid, &target);
 		if (err != 0)
 			goto fill_subreq;
-		len = strlen(target);
-		n = copy_to_iter(target, len, &subreq->io_iter);
+		len = min(strlen(target), subreq->len);
+		total = copy_to_iter(target, len, &iter);
 		kfree(target);
-		total = n;
 	} else {
-		total = p9_client_read(fid, pos, &subreq->io_iter, &err);
+		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+
+		total = p9_client_read(fid, pos, &iter, &err);
 	}
 
 fill_subreq:
@@ -112,6 +152,7 @@ static void v9fs_issue_read(struct netfs_io_subrequest *subreq)
 
 	subreq->error = err;
 	netfs_read_subreq_terminated(subreq);
+	return 0;
 }
 
 /**
@@ -185,6 +226,7 @@ const struct netfs_request_ops v9fs_req_ops = {
 	.free_request		= v9fs_free_request,
 	.issue_read		= v9fs_issue_read,
 	.begin_writeback	= v9fs_begin_writeback,
+	.estimate_write		= v9fs_estimate_write,
 	.issue_write		= v9fs_issue_write,
 };
 
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 547faa06c80b..174114c39639 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -250,8 +250,8 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
 	if (dvnode->directory_size < i_size) {
 		size_t cur_size = dvnode->directory_size;
 
-		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size, i_size,
-					  GFP_KERNEL);
+		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size,
+					  round_up(i_size, PAGE_SIZE), GFP_KERNEL);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			return ret;
@@ -2227,11 +2227,10 @@ static int afs_dir_writepages(struct address_space *mapping,
 	}
 
 	if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
+		size_t len = i_size_read(&dvnode->netfs.inode);
 		iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
-				    i_size_read(&dvnode->netfs.inode));
-		ret = netfs_writeback_single(mapping, wbc, &iter);
-		if (ret == 1)
-			ret = 0; /* Skipped write due to lock conflict. */
+				    round_up(len, PAGE_SIZE));
+		ret = netfs_writeback_single(mapping, wbc, &iter, len);
 	}
 
 	up_read(&dvnode->validate_lock);
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 92380dc25781..476b9f3a776e 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -332,11 +332,12 @@ void afs_fetch_data_immediate_cancel(struct afs_call *call)
 /*
  * Fetch file data from the volume.
  */
-static void afs_issue_read(struct netfs_io_subrequest *subreq)
+static int afs_issue_read(struct netfs_io_subrequest *subreq)
 {
 	struct afs_operation *op;
 	struct afs_vnode *vnode = AFS_FS_I(subreq->rreq->inode);
 	struct key *key = subreq->rreq->netfs_priv;
+	int ret;
 
 	_enter("%s{%llx:%llu.%u},%x,,,",
 	       vnode->volume->name,
@@ -345,11 +346,15 @@ static void afs_issue_read(struct netfs_io_subrequest *subreq)
 	       vnode->fid.unique,
 	       key_serial(key));
 
+	ret = netfs_prepare_read_buffer(subreq, INT_MAX);
+	if (ret < 0)
+		return ret;
+	/* After this point, must fail by termination. */
+
 	op = afs_alloc_operation(key, vnode->volume);
 	if (IS_ERR(op)) {
-		subreq->error = PTR_ERR(op);
-		netfs_read_subreq_terminated(subreq);
-		return;
+		ret = PTR_ERR(op);
+		goto failed;
 	}
 
 	afs_op_set_vnode(op, 0, vnode);
@@ -364,20 +369,24 @@ static void afs_issue_read(struct netfs_io_subrequest *subreq)
 		op->flags |= AFS_OPERATION_ASYNC;
 
 		if (!afs_begin_vnode_operation(op)) {
-			subreq->error = afs_put_operation(op);
-			netfs_read_subreq_terminated(subreq);
-			return;
+			ret = afs_put_operation(op);
+			goto failed;
 		}
 
 		if (!afs_select_fileserver(op)) {
-			afs_end_read(op);
-			return;
+			afs_end_read(op); /* Error recorded here. */
+			return 0;
 		}
 
 		afs_issue_read_call(op);
 	} else {
 		afs_do_sync_operation(op);
 	}
+	return 0;
+failed:
+	subreq->error = ret;
+	netfs_read_subreq_terminated(subreq);
+	return 0;
 }
 
 static int afs_init_request(struct netfs_io_request *rreq, struct file *file)
@@ -469,7 +478,7 @@ const struct netfs_request_ops afs_req_ops = {
 	.update_i_size		= afs_update_i_size,
 	.invalidate_cache	= afs_netfs_invalidate_cache,
 	.begin_writeback	= afs_begin_writeback,
-	.prepare_write		= afs_prepare_write,
+	.estimate_write		= afs_estimate_write,
 	.issue_write		= afs_issue_write,
 	.retry_request		= afs_retry_request,
 };
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 1a3f186a6a11..9713551dd842 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -339,7 +339,9 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
 		if (call->remaining == 0)
 			goto no_more_data;
 
-		call->iter = &subreq->io_iter;
+		iov_iter_bvec_queue(&call->def_iter, ITER_DEST, subreq->content.bvecq,
+				    subreq->content.slot, subreq->content.offset, subreq->len);
+
 		call->iov_len = umin(call->remaining, subreq->len - subreq->transferred);
 		call->unmarshall++;
 		fallthrough;
@@ -1088,7 +1090,7 @@ static void afs_fs_store_data64(struct afs_operation *op)
 	if (!call)
 		return afs_op_nomem(op);
 
-	call->write_iter = op->store.write_iter;
+	call->write_iter = &op->store.write_iter;
 
 	/* marshall the parameters */
 	bp = call->request;
@@ -1142,7 +1144,7 @@ void afs_fs_store_data(struct afs_operation *op)
 	if (!call)
 		return afs_op_nomem(op);
 
-	call->write_iter = op->store.write_iter;
+	call->write_iter = &op->store.write_iter;
 
 	/* marshall the parameters */
 	bp = call->request;
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 556e7e7777f2..5c536cc7b53c 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -914,7 +914,7 @@ struct afs_operation {
 			afs_lock_type_t type;
 		} lock;
 		struct {
-			struct iov_iter	*write_iter;
+			struct iov_iter	write_iter;
 			loff_t	pos;
 			loff_t	size;
 			loff_t	i_size;
@@ -1698,8 +1698,10 @@ extern int afs_check_volume_status(struct afs_volume *, struct afs_operation *);
 /*
  * write.c
  */
-void afs_prepare_write(struct netfs_io_subrequest *subreq);
-void afs_issue_write(struct netfs_io_subrequest *subreq);
+int afs_estimate_write(struct netfs_io_request *wreq,
+		       struct netfs_io_stream *stream,
+		       struct netfs_write_estimate *estimate);
+int afs_issue_write(struct netfs_io_subrequest *subreq);
 void afs_begin_writeback(struct netfs_io_request *wreq);
 void afs_retry_request(struct netfs_io_request *wreq, struct netfs_io_stream *stream);
 extern int afs_writepages(struct address_space *, struct writeback_control *);
diff --git a/fs/afs/symlink.c b/fs/afs/symlink.c
index 84f8ab222dd6..ddd3307adccf 100644
--- a/fs/afs/symlink.c
+++ b/fs/afs/symlink.c
@@ -245,9 +245,10 @@ int afs_symlink_writepages(struct address_space *mapping,
 
 	if (vnode->directory &&
 	    atomic64_read(&vnode->cb_expires_at) != AFS_NO_CB_PROMISE) {
+		size_t len = i_size_read(&vnode->netfs.inode);
 		iov_iter_bvec_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
-				    i_size_read(&vnode->netfs.inode));
-		ret = netfs_writeback_single(mapping, wbc, &iter);
+				    round_up(len, PAGE_SIZE));
+		ret = netfs_writeback_single(mapping, wbc, &iter, len);
 	}
 
 	if (ret == 0) {
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 7f34b939706a..46f81d9e7d7f 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -83,17 +83,20 @@ static const struct afs_operation_ops afs_store_data_operation = {
 };
 
 /*
- * Prepare a subrequest to write to the server.  This sets the max_len
- * parameter.
+ * Estimate the maximum size of a write we can send to the server.
  */
-void afs_prepare_write(struct netfs_io_subrequest *subreq)
+int afs_estimate_write(struct netfs_io_request *wreq,
+		       struct netfs_io_stream *stream,
+		       struct netfs_write_estimate *estimate)
 {
-	struct netfs_io_stream *stream = &subreq->rreq->io_streams[subreq->stream_nr];
+	unsigned long long limit = ULLONG_MAX - stream->issue_from;
+	unsigned long long max_len = 256 * 1024 * 1024;
 
 	//if (test_bit(NETFS_SREQ_RETRYING, &subreq->flags))
-	//	subreq->max_len = 512 * 1024;
-	//else
-	stream->sreq_max_len = 256 * 1024 * 1024;
+	//	max_len = 512 * 1024;
+
+	estimate->issue_at = stream->issue_from + umin(max_len, limit);
+	return 0;
 }
 
 /*
@@ -139,12 +142,15 @@ static void afs_issue_write_worker(struct work_struct *work)
 	op->flags		|= AFS_OPERATION_UNINTR;
 	op->ops			= &afs_store_data_operation;
 
+	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 	afs_begin_vnode_operation(op);
 
-	op->store.write_iter	= &subreq->io_iter;
 	op->store.i_size	= umax(pos + len, netfs_read_remote_i_size(&vnode->netfs.inode));
 	op->mtime		= inode_get_mtime(&vnode->netfs.inode);
 
+	iov_iter_bvec_queue(&op->store.write_iter, ITER_SOURCE, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
 	afs_wait_for_operation(op);
 	ret = afs_put_operation(op);
 	switch (ret) {
@@ -168,11 +174,21 @@ static void afs_issue_write_worker(struct work_struct *work)
 	netfs_write_subrequest_terminated(subreq, ret < 0 ? ret : subreq->len);
 }
 
-void afs_issue_write(struct netfs_io_subrequest *subreq)
+int afs_issue_write(struct netfs_io_subrequest *subreq)
 {
+	int ret;
+
+	if (subreq->len > 256 * 1024 * 1024)
+		subreq->len = 256 * 1024 * 1024;
+	ret = netfs_prepare_write_buffer(subreq, INT_MAX);
+	if (ret < 0)
+		return ret;
+	/* After this point, must fail by termination. */
+
 	subreq->work.func = afs_issue_write_worker;
 	if (!queue_work(system_dfl_wq, &subreq->work))
 		WARN_ON_ONCE(1);
+	return 0;
 }
 
 /*
@@ -183,6 +199,8 @@ void afs_begin_writeback(struct netfs_io_request *wreq)
 {
 	if (S_ISREG(wreq->inode->i_mode))
 		afs_get_writeback_key(wreq);
+
+	wreq->io_streams[0].avail = true;
 }
 
 /*
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index d941179730a9..52c588092050 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -385,7 +385,9 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
 		if (call->remaining == 0)
 			goto no_more_data;
 
-		call->iter = &subreq->io_iter;
+		iov_iter_bvec_queue(&call->def_iter, ITER_DEST, subreq->content.bvecq,
+				    subreq->content.slot, subreq->content.offset, subreq->len);
+
 		call->iov_len = min(call->remaining, subreq->len - subreq->transferred);
 		call->unmarshall++;
 		fallthrough;
@@ -1357,7 +1359,7 @@ void yfs_fs_store_data(struct afs_operation *op)
 	if (!call)
 		return afs_op_nomem(op);
 
-	call->write_iter = op->store.write_iter;
+	call->write_iter = &op->store.write_iter;
 
 	/* marshall the parameters */
 	bp = call->request;
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index d7ad65fc840e..b06d0fbcc81e 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -26,7 +26,10 @@ struct cachefiles_kiocb {
 	};
 	struct cachefiles_object *object;
 	netfs_io_terminated_t	term_func;
-	void			*term_func_priv;
+	union {
+		struct netfs_io_subrequest *subreq;
+		void			*term_func_priv;
+	};
 	bool			was_async;
 	unsigned int		inval_counter;	/* Copy of cookie->inval_counter */
 	u64			b_writing;
@@ -194,12 +197,131 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
 	return ret;
 }
 
+/*
+ * Handle completion of a read from the cache issued by netfslib.
+ */
+static void cachefiles_issue_read_complete(struct kiocb *iocb, long ret)
+{
+	struct cachefiles_kiocb *ki = container_of(iocb, struct cachefiles_kiocb, iocb);
+	struct netfs_io_subrequest *subreq = ki->subreq;
+	struct inode *inode = file_inode(ki->iocb.ki_filp);
+
+	_enter("%ld", ret);
+
+	if (ret < 0) {
+		subreq->error = -ESTALE;
+		trace_cachefiles_io_error(ki->object, inode, ret,
+					  cachefiles_trace_read_error);
+	}
+
+	if (ret >= 0) {
+		if (ki->object->cookie->inval_counter == ki->inval_counter) {
+			subreq->error = 0;
+			if (ret > 0) {
+				subreq->transferred += ret;
+				__set_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+			}
+		} else {
+			subreq->error = -ESTALE;
+		}
+	}
+
+	netfs_read_subreq_terminated(subreq);
+	cachefiles_put_kiocb(ki);
+}
+
+/*
+ * Issue a read operation to the cache.  We assume that the cache is readable
+ * as the caller should have called ->query_occupancy() first.
+ */
+static int cachefiles_issue_read(struct netfs_io_subrequest *subreq)
+{
+	struct netfs_cache_resources *cres = &subreq->rreq->cache_resources;
+	struct cachefiles_object *object = cachefiles_cres_object(cres);
+	struct cachefiles_kiocb *ki;
+	struct iov_iter iter;
+	struct file *file = cachefiles_cres_file(cres);
+	unsigned int old_nofs;
+	ssize_t ret;
+
+	fscache_count_read();
+
+	_enter("%pD,%lli,%llx,%zx/%llx",
+	       file, file_inode(file)->i_ino, subreq->start, subreq->len,
+	       i_size_read(file_inode(file)));
+
+	if (subreq->len > MAX_RW_COUNT)
+		subreq->len = MAX_RW_COUNT;
+
+	ret = netfs_prepare_read_buffer(subreq, BIO_MAX_VECS);
+	if (ret < 0)
+		return ret;
+	/* After this point, must fail by termination. */
+
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
+	ret = -ENOMEM;
+	ki = kzalloc_obj(struct cachefiles_kiocb);
+	if (!ki)
+		goto failed;
+
+	refcount_set(&ki->ki_refcnt, 2);
+	ki->iocb.ki_filp	= file;
+	ki->iocb.ki_pos		= subreq->start;
+	ki->iocb.ki_flags	= IOCB_DIRECT;
+	ki->iocb.ki_ioprio	= get_current_ioprio();
+	ki->iocb.ki_complete	= cachefiles_issue_read_complete;
+	ki->object		= object;
+	ki->inval_counter	= cres->inval_counter;
+	ki->subreq		= subreq;
+	ki->was_async		= true;
+
+	get_file(ki->iocb.ki_filp);
+	cachefiles_grab_object(object, cachefiles_obj_get_ioreq);
+
+	trace_cachefiles_read(object, file_inode(file), ki->iocb.ki_pos, subreq->len);
+	old_nofs = memalloc_nofs_save();
+	ret = cachefiles_inject_read_error();
+	if (ret == 0)
+		ret = vfs_iocb_iter_read(file, &ki->iocb, &iter);
+	memalloc_nofs_restore(old_nofs);
+
+	switch (ret) {
+	case -EIOCBQUEUED:
+		break;
+
+	case -ERESTARTSYS:
+	case -ERESTARTNOINTR:
+	case -ERESTARTNOHAND:
+	case -ERESTART_RESTARTBLOCK:
+		/* There's no easy way to restart the syscall since other AIO's
+		 * may be already running. Just fail this IO with EINTR.
+		 */
+		ret = -EINTR;
+		fallthrough;
+	default:
+		ki->was_async = false;
+		cachefiles_issue_read_complete(&ki->iocb, ret);
+		break;
+	}
+
+	cachefiles_put_kiocb(ki);
+	_leave(" = %zd", ret);
+	return 0;
+failed:
+	subreq->error = ret;
+	netfs_read_subreq_terminated(subreq);
+	return 0;
+}
+
 /*
  * Query the occupancy of the cache in a region, returning the extent of the
- * next two chunks of cached data and the next hole.
+ * next two chunks of cached data and the next hole.  The occupancy map is
+ * preloaded to show just one giant hole.
  */
-static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
-				      struct fscache_occupancy *occ)
+static void cachefiles_query_occupancy(struct netfs_cache_resources *cres,
+				       struct fscache_occupancy *occ)
 {
 	struct cachefiles_object *object;
 	struct inode *inode;
@@ -209,7 +331,7 @@ static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
 	int i;
 
 	if (!fscache_wait_for_operation(cres, FSCACHE_WANT_READ))
-		return -ENOBUFS;
+		return;
 
 	object = cachefiles_cres_object(cres);
 	file = cachefiles_cres_file(cres);
@@ -244,7 +366,7 @@ static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
 			ret = vfs_llseek(file, occ->query_from, SEEK_DATA);
 		if (IS_ERR_VALUE_LL(ret)) {
 			if (ret != -ENXIO)
-				return ret;
+				goto done;
 			occ->query_from = ULLONG_MAX;
 			goto done;
 		}
@@ -257,7 +379,7 @@ static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
 			ret = vfs_llseek(file, occ->query_from, SEEK_HOLE);
 		if (IS_ERR_VALUE_LL(ret)) {
 			if (ret != -ENXIO)
-				return ret;
+				goto done;
 			occ->query_from = ULLONG_MAX;
 			goto done;
 		}
@@ -270,7 +392,6 @@ static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
 done:
 	_debug("query[0] %llx-%llx", occ->cached_from[0], occ->cached_to[0]);
 	_debug("query[1] %llx-%llx", occ->cached_from[1], occ->cached_to[1]);
-	return 0;
 }
 
 /*
@@ -587,9 +708,9 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 				    cachefiles_has_space_for_write);
 }
 
-static int cachefiles_prepare_write(struct netfs_cache_resources *cres,
-				    loff_t *_start, size_t *_len, size_t upper_len,
-				    loff_t i_size, bool no_space_allocated_yet)
+static int cachefiles_prepare_write_old(struct netfs_cache_resources *cres,
+					loff_t *_start, size_t *_len, size_t upper_len,
+					loff_t i_size, bool no_space_allocated_yet)
 {
 	struct cachefiles_object *object = cachefiles_cres_object(cres);
 	struct cachefiles_cache *cache = object->volume->cache;
@@ -611,80 +732,72 @@ static int cachefiles_prepare_write(struct netfs_cache_resources *cres,
 	return ret;
 }
 
-static void cachefiles_prepare_write_subreq(struct netfs_io_subrequest *subreq)
+static int cachefiles_estimate_write(struct netfs_io_request *wreq,
+				     struct netfs_io_stream *stream,
+				     struct netfs_write_estimate *estimate)
 {
-	struct netfs_io_request *wreq = subreq->rreq;
-	struct netfs_cache_resources *cres = &wreq->cache_resources;
-	struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];
-
-	_enter("W=%x[%x] %llx", wreq->debug_id, subreq->debug_index, subreq->start);
-
-	stream->sreq_max_len = MAX_RW_COUNT;
-	stream->sreq_max_segs = BIO_MAX_VECS;
-
-	if (!cachefiles_cres_file(cres)) {
-		if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
-			return netfs_prepare_write_failed(subreq);
-		if (!cachefiles_cres_file(cres))
-			return netfs_prepare_write_failed(subreq);
-	}
+	estimate->issue_at = stream->issue_from + MAX_RW_COUNT;
+	estimate->max_segs = BIO_MAX_VECS;
+	return 0;
 }
 
-static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
+static int cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *wreq = subreq->rreq;
 	struct netfs_cache_resources *cres = &wreq->cache_resources;
 	struct cachefiles_object *object = cachefiles_cres_object(cres);
 	struct cachefiles_cache *cache = object->volume->cache;
+	struct iov_iter iter;
 	const struct cred *saved_cred;
-	size_t off, pre, post, len = subreq->len;
 	loff_t start = subreq->start;
-	int ret;
+	size_t len = subreq->len;
+	int ret = -EINVAL;
 
 	_enter("W=%x[%x] %llx-%llx",
 	       wreq->debug_id, subreq->debug_index, start, start + len - 1);
 
-	/* We need to start on the cache granularity boundary */
-	off = start & (cache->bsize - 1);
-	if (off) {
-		pre = cache->bsize - off;
-		if (pre >= len) {
-			fscache_count_dio_misfit();
-			netfs_write_subrequest_terminated(subreq, len);
-			return;
-		}
-		subreq->transferred += pre;
-		start += pre;
-		len -= pre;
-		iov_iter_advance(&subreq->io_iter, pre);
-	}
-
-	/* We also need to end on the cache granularity boundary */
-	post = len & (cache->bsize - 1);
-	if (post) {
-		len -= post;
-		if (len == 0) {
-			fscache_count_dio_misfit();
-			netfs_write_subrequest_terminated(subreq, post);
-			return;
-		}
-		iov_iter_truncate(&subreq->io_iter, len);
+	if (!cachefiles_cres_file(cres)) {
+		if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
+			return -EINVAL;
+		if (!cachefiles_cres_file(cres))
+			return -EINVAL;
 	}
 
+	ret = netfs_prepare_write_buffer(subreq, BIO_MAX_VECS);
+	if (ret < 0)
+		return ret;
+	/* After this point, must fail by termination. */
+
+	/* The buffer extraction func may round out start and end. */
+	start = subreq->start;
+	len = subreq->len;
+
+	/* We need to start and end on cache granularity boundaries. */
+	if (WARN_ON_ONCE(start & (cache->bsize - 1)) ||
+	    WARN_ON_ONCE(len   & (cache->bsize - 1))) {
+		fscache_count_dio_misfit();
+		ret = -EIO;
+		goto failed;
+	}
+
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, len);
+
 	trace_netfs_sreq(subreq, netfs_sreq_trace_cache_prepare);
 	cachefiles_begin_secure(cache, &saved_cred);
 	ret = __cachefiles_prepare_write(object, cachefiles_cres_file(cres),
 					 &start, &len, len, true);
 	cachefiles_end_secure(cache, saved_cred);
-	if (ret < 0) {
-		netfs_write_subrequest_terminated(subreq, ret);
-		return;
-	}
+	if (ret < 0)
+		goto failed;
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_cache_write);
-	cachefiles_write(&subreq->rreq->cache_resources,
-			 subreq->start, &subreq->io_iter,
+	cachefiles_write(&subreq->rreq->cache_resources, subreq->start, &iter,
 			 netfs_write_subrequest_terminated, subreq);
+	return 0;
+failed:
+	netfs_write_subrequest_terminated(subreq, ret);
+	return 0;
 }
 
 /*
@@ -891,9 +1004,10 @@ static const struct netfs_cache_ops cachefiles_netfs_cache_ops = {
 	.end_operation		= cachefiles_end_operation,
 	.read			= cachefiles_read,
 	.write			= cachefiles_write,
+	.issue_read		= cachefiles_issue_read,
 	.issue_write		= cachefiles_issue_write,
-	.prepare_write		= cachefiles_prepare_write,
-	.prepare_write_subreq	= cachefiles_prepare_write_subreq,
+	.estimate_write		= cachefiles_estimate_write,
+	.prepare_write_old	= cachefiles_prepare_write_old,
 	.query_occupancy	= cachefiles_query_occupancy,
 	.collect_write		= cachefiles_collect_write,
 };
diff --git a/fs/ceph/Kconfig b/fs/ceph/Kconfig
index 3d64a316ca31..aa6ccd7794d2 100644
--- a/fs/ceph/Kconfig
+++ b/fs/ceph/Kconfig
@@ -4,6 +4,7 @@ config CEPH_FS
 	depends on INET
 	select CEPH_LIB
 	select NETFS_SUPPORT
+	select NETFS_PGPRIV2
 	select FS_ENCRYPTION_ALGS if FS_ENCRYPTION
 	default n
 	help
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index ecf33b66610c..0e1cb493fe6e 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -274,7 +274,7 @@ static void finish_netfs_read(struct ceph_osd_request *req)
 	ceph_dec_osd_stopping_blocker(fsc->mdsc);
 }
 
-static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
+static int ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	struct inode *inode = rreq->inode;
@@ -283,7 +283,8 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 	struct ceph_mds_request *req;
 	struct ceph_mds_client *mdsc = ceph_sb_to_mdsc(inode->i_sb);
 	struct ceph_inode_info *ci = ceph_inode(inode);
-	ssize_t err = 0;
+	struct iov_iter iter;
+	ssize_t err;
 	size_t len;
 	int mode;
 
@@ -292,8 +293,20 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 		__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
 	__clear_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
 
-	if (subreq->start >= inode->i_size)
+	err = netfs_prepare_read_buffer(subreq, INT_MAX);
+	if (err < 0)
+		return err;
+	/* After this point, must fail by termination. */
+
+	if (subreq->start >= inode->i_size) {
+		__set_bit(NETFS_SREQ_HIT_EOF, &subreq->flags);
+		err = 0;
 		goto out;
+	}
+
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset,
+			    subreq->len);
 
 	/* We need to fetch the inline data. */
 	mode = ceph_try_to_choose_auth_mds(inode, CEPH_STAT_CAP_INLINE_DATA);
@@ -302,11 +315,13 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 		err = PTR_ERR(req);
 		goto out;
 	}
+
 	req->r_ino1 = ci->i_vino;
 	req->r_args.getattr.mask = cpu_to_le32(CEPH_STAT_CAP_INLINE_DATA);
 	req->r_num_caps = 2;
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+
 	err = ceph_mdsc_do_request(mdsc, NULL, req);
 	if (err < 0)
 		goto out;
@@ -316,11 +331,13 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 	if (iinfo->inline_version == CEPH_INLINE_NONE) {
 		/* The data got uninlined */
 		ceph_mdsc_put_request(req);
-		return false;
+		__set_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+		err = -EAGAIN;
+		goto out;
 	}
 
 	len = min_t(size_t, iinfo->inline_len - subreq->start, subreq->len);
-	err = copy_to_iter(iinfo->inline_data + subreq->start, len, &subreq->io_iter);
+	err = copy_to_iter(iinfo->inline_data + subreq->start, len, &iter);
 	if (err == 0) {
 		err = -EFAULT;
 	} else {
@@ -333,26 +350,10 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 	subreq->error = err;
 	trace_netfs_sreq(subreq, netfs_sreq_trace_io_progress);
 	netfs_read_subreq_terminated(subreq);
-	return true;
-}
-
-static int ceph_netfs_prepare_read(struct netfs_io_subrequest *subreq)
-{
-	struct netfs_io_request *rreq = subreq->rreq;
-	struct inode *inode = rreq->inode;
-	struct ceph_inode_info *ci = ceph_inode(inode);
-	struct ceph_fs_client *fsc = ceph_inode_to_fs_client(inode);
-	u64 objno, objoff;
-	u32 xlen;
-
-	/* Truncate the extent at the end of the current block */
-	ceph_calc_file_object_mapping(&ci->i_layout, subreq->start, subreq->len,
-				      &objno, &objoff, &xlen);
-	rreq->io_streams[0].sreq_max_len = umin(xlen, fsc->mount_options->rsize);
 	return 0;
 }
 
-static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
+static int ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	struct inode *inode = rreq->inode;
@@ -361,19 +362,18 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 	struct ceph_client *cl = fsc->client;
 	struct ceph_osd_request *req = NULL;
 	struct ceph_vino vino = ceph_vino(inode);
-	int err;
-	u64 len;
+	struct iov_iter iter;
+	u64 objno, objoff, len, off = subreq->start;
+	u32 maxlen;
+	int err = -EIO;
 	bool sparse = IS_ENCRYPTED(inode) || ceph_test_mount_opt(fsc, SPARSEREAD);
-	u64 off = subreq->start;
 	int extent_cnt;
 
-	if (ceph_inode_is_shutdown(inode)) {
-		err = -EIO;
-		goto out;
-	}
+	if (ceph_inode_is_shutdown(inode))
+		return -EIO;
 
-	if (ceph_has_inline_data(ci) && ceph_netfs_issue_op_inline(subreq))
-		return;
+	if (ceph_has_inline_data(ci))
+		return ceph_netfs_issue_op_inline(subreq);
 
 	// TODO: This rounding here is slightly dodgy.  It *should* work, for
 	// now, as the cache only deals in blocks that are a multiple of
@@ -383,26 +383,48 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 	len = subreq->len;
 	ceph_fscrypt_adjust_off_and_len(inode, &off, &len);
 
+	/* Truncate the extent at the end of the current block */
+	ceph_calc_file_object_mapping(&ci->i_layout, subreq->start, len,
+				      &objno, &objoff, &maxlen);
+	maxlen = min(maxlen, fsc->mount_options->rsize);
+	len = min(len, maxlen);
+	subreq->len = len;
+
+	/* Grab a slice of read buffer.  This may shrink the subreq. */
+	err = netfs_prepare_read_buffer(subreq, INT_MAX);
+	if (err < 0)
+		return err;
+	/* After this point, must fail by termination. */
+
+	/* Create a request.  In theory, this may shrink the request again, but
+	 * it shouldn't since we calculated the object size above and already
+	 * shrank to that, but if it does, we'll just end up doing a short read
+	 * and retrying to get the rest.
+	 */
+	len = subreq->len;
 	req = ceph_osdc_new_request(&fsc->client->osdc, &ci->i_layout, vino,
 			off, &len, 0, 1, sparse ? CEPH_OSD_OP_SPARSE_READ : CEPH_OSD_OP_READ,
 			CEPH_OSD_FLAG_READ, NULL, ci->i_truncate_seq,
 			ci->i_truncate_size, false);
 	if (IS_ERR(req)) {
 		err = PTR_ERR(req);
-		req = NULL;
-		goto out;
+		goto failed_noput;
 	}
 
 	if (sparse) {
 		extent_cnt = __ceph_sparse_read_ext_count(inode, len);
 		err = ceph_alloc_sparse_ext_map(&req->r_ops[0], extent_cnt);
 		if (err)
-			goto out;
+			goto failed;
 	}
 
 	doutc(cl, "%llx.%llx pos=%llu orig_len=%zu len=%llu\n",
 	      ceph_vinop(inode), subreq->start, subreq->len, len);
 
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset,
+			    subreq->len);
+
 	/*
 	 * FIXME: For now, use CEPH_OSD_DATA_TYPE_PAGES instead of _ITER for
 	 * encrypted inodes. We'd need infrastructure that handles an iov_iter
@@ -421,13 +443,11 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 		 * ceph_msg_data_cursor_init() triggers BUG_ON() in the case
 		 * if msg->sparse_read_total > msg->data_length.
 		 */
-		subreq->io_iter.count = len;
-
-		err = iov_iter_get_pages_alloc2(&subreq->io_iter, &pages, len, &page_off);
+		err = iov_iter_get_pages_alloc2(&iter, &pages, len, &page_off);
 		if (err < 0) {
 			doutc(cl, "%llx.%llx failed to allocate pages, %d\n",
 			      ceph_vinop(inode), err);
-			goto out;
+			goto eio;
 		}
 
 		/* should always give us a page-aligned read */
@@ -438,12 +458,10 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 		osd_req_op_extent_osd_data_pages(req, 0, pages, len, 0, false,
 						 false);
 	} else {
-		osd_req_op_extent_osd_iter(req, 0, &subreq->io_iter);
-	}
-	if (!ceph_inc_osd_stopping_blocker(fsc->mdsc)) {
-		err = -EIO;
-		goto out;
+		osd_req_op_extent_osd_iter(req, 0, &iter);
 	}
+	if (!ceph_inc_osd_stopping_blocker(fsc->mdsc))
+		goto eio;
 	req->r_callback = finish_netfs_read;
 	req->r_priv = subreq;
 	req->r_inode = inode;
@@ -451,19 +469,22 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 	ceph_osdc_start_request(req->r_osdc, req);
-out:
 	ceph_osdc_put_request(req);
-	if (err) {
-		subreq->error = err;
-		netfs_read_subreq_terminated(subreq);
-	}
-	doutc(cl, "%llx.%llx result %d\n", ceph_vinop(inode), err);
+	doutc(cl, "%llx.%llx result -EIOCBQUEUED\n", ceph_vinop(inode));
+	return 0;
+eio:
+	err = -EIO;
+failed:
+	ceph_osdc_put_request(req);
+failed_noput:
+	subreq->error = err;
+	netfs_read_subreq_terminated(subreq);
+	return 0;
 }
 
 static int ceph_init_request(struct netfs_io_request *rreq, struct file *file)
 {
 	struct inode *inode = rreq->inode;
-	struct ceph_fs_client *fsc = ceph_inode_to_fs_client(inode);
 	struct ceph_client *cl = ceph_inode_to_client(inode);
 	int got = 0, want = CEPH_CAP_FILE_CACHE;
 	struct ceph_netfs_request_data *priv;
@@ -515,7 +536,6 @@ static int ceph_init_request(struct netfs_io_request *rreq, struct file *file)
 
 	priv->caps = got;
 	rreq->netfs_priv = priv;
-	rreq->io_streams[0].sreq_max_len = fsc->mount_options->rsize;
 
 out:
 	if (ret < 0) {
@@ -543,7 +563,6 @@ static void ceph_netfs_free_request(struct netfs_io_request *rreq)
 const struct netfs_request_ops ceph_netfs_ops = {
 	.init_request		= ceph_init_request,
 	.free_request		= ceph_netfs_free_request,
-	.prepare_read		= ceph_netfs_prepare_read,
 	.issue_read		= ceph_netfs_issue_read,
 	.expand_readahead	= ceph_netfs_expand_readahead,
 	.check_write_begin	= ceph_netfs_check_write_begin,
diff --git a/fs/netfs/Kconfig b/fs/netfs/Kconfig
index 7701c037c328..d0e7b0971fa3 100644
--- a/fs/netfs/Kconfig
+++ b/fs/netfs/Kconfig
@@ -22,6 +22,9 @@ config NETFS_STATS
 	  between CPUs.  On the other hand, the stats are very useful for
 	  debugging purposes.  Saying 'Y' here is recommended.
 
+config NETFS_PGPRIV2
+	bool
+
 config NETFS_DEBUG
 	bool "Enable dynamic debugging netfslib and FS-Cache"
 	depends on NETFS_SUPPORT
diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index 0621e6870cbd..421dd0be413b 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -12,13 +12,13 @@ netfs-y := \
 	misc.o \
 	objects.o \
 	read_collect.o \
-	read_pgpriv2.o \
 	read_retry.o \
 	read_single.o \
 	write_collect.o \
 	write_issue.o \
 	write_retry.o
 
+netfs-$(CONFIG_NETFS_PGPRIV2) += read_pgpriv2.o
 netfs-$(CONFIG_NETFS_STATS) += stats.o
 
 netfs-$(CONFIG_FSCACHE) += \
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 1f052bb6c861..cb230d41180a 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -100,70 +100,98 @@ static int netfs_begin_cache_read(struct netfs_io_request *rreq, struct netfs_in
 }
 
 /*
- * netfs_prepare_read_iterator - Prepare the subreq iterator for I/O
- * @subreq: The subrequest to be set up
- *
- * Prepare the I/O iterator representing the read buffer on a subrequest for
- * the filesystem to use for I/O (it can be passed directly to a socket).  This
- * is intended to be called from the ->issue_read() method once the filesystem
- * has trimmed the request to the size it wants.
- *
- * Returns the limited size if successful and -ENOMEM if insufficient memory
- * available.
+ * Prepare the I/O buffer on a buffered read subrequest for the filesystem to
+ * use as a bvec queue.
  */
-static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
+static int netfs_prepare_buffered_read_buffer(struct netfs_io_subrequest *subreq,
+					      unsigned int max_segs)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
 	ssize_t extracted;
-	size_t rsize = subreq->len;
 
-	if (subreq->source == NETFS_DOWNLOAD_FROM_SERVER)
-		rsize = umin(rsize, stream->sreq_max_len);
+	_enter("R=%08x[%x] l=%zx s=%u",
+	       rreq->debug_id, subreq->debug_index, subreq->len, max_segs);
 
-	bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
-	extracted = bvecq_slice(&rreq->dispatch_cursor, rsize,
-				stream->sreq_max_segs, &subreq->nr_segs);
-	if (extracted < rsize) {
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	extracted = bvecq_slice(&stream->dispatch_cursor, subreq->len,
+				max_segs, &subreq->nr_segs);
+
+	if (extracted < subreq->len) {
 		subreq->len = extracted;
 		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
 	}
+	stream->buffered -= extracted;
+	stream->issue_from = subreq->start + subreq->len;
+	rreq->submitted = stream->issue_from;
 
-	return subreq->len;
+	if (!stream->buffered)
+		netfs_all_subreqs_queued(rreq);
+	return 0;
 }
 
-/*
- * Issue a read against the cache.
- * - Eats the caller's ref on subreq.
+/**
+ * netfs_prepare_read_buffer - Get the buffer for a subrequest
+ * @subreq: The subrequest to get the buffer for
+ * @max_segs: Maximum number of segments in buffer (or INT_MAX)
+ *
+ * Extract a slice of buffer from the stream and attach it to the subrequest as
+ * a bio_vec queue.  The maximum amount of data attached is set by
+ * @subreq->len, but this may be shortened if @max_segs would be exceeded.
+ *
+ * [!] NOTE: This must be run in the same thread as ->issue_read() was called
+ * in as we access the readahead_control struct if there is one.
  */
-static void netfs_read_cache_to_pagecache(struct netfs_io_request *rreq,
-					  struct netfs_io_subrequest *subreq)
+int netfs_prepare_read_buffer(struct netfs_io_subrequest *subreq,
+			      unsigned int max_segs)
 {
-	struct netfs_cache_resources *cres = &rreq->cache_resources;
-
-	netfs_stat(&netfs_n_rh_read);
-	cres->ops->read(cres, subreq->start, &subreq->io_iter, NETFS_READ_HOLE_IGNORE,
-			netfs_cache_read_terminated, subreq);
+	switch (subreq->rreq->origin) {
+	case NETFS_READAHEAD:
+	case NETFS_READPAGE:
+	case NETFS_READ_FOR_WRITE:
+		if (subreq->retry_count)
+			return netfs_prepare_buffered_read_retry_buffer(subreq, max_segs);
+		return netfs_prepare_buffered_read_buffer(subreq, max_segs);
+
+	case NETFS_UNBUFFERED_READ:
+	case NETFS_DIO_READ:
+	case NETFS_READ_GAPS:
+		return netfs_prepare_unbuffered_read_buffer(subreq, max_segs);
+	case NETFS_READ_SINGLE:
+		return netfs_prepare_read_single_buffer(subreq, max_segs);
+	default:
+		WARN_ON_ONCE(1);
+		return -EIO;
+	}
 }
+EXPORT_SYMBOL(netfs_prepare_read_buffer);
 
-int netfs_read_query_cache(struct netfs_io_request *rreq, struct fscache_occupancy *occ)
+void netfs_read_query_cache(struct netfs_io_request *rreq, struct fscache_occupancy *occ)
 {
 	struct netfs_cache_resources *cres = &rreq->cache_resources;
 
 	occ->granularity = PAGE_SIZE;
 	if (occ->query_from >= occ->query_to)
-		return 0;
+		return;
 	if (!cres->ops)
-		return 0;
+		return;
 	occ->query_from = round_up(occ->query_from, occ->granularity);
-	return cres->ops->query_occupancy(cres, occ);
+	cres->ops->query_occupancy(cres, occ);
 }
 
-void netfs_queue_read(struct netfs_io_request *rreq,
-		      struct netfs_io_subrequest *subreq)
+/*
+ * Allocate and prepare a read subrequest.
+ */
+struct netfs_io_subrequest *netfs_alloc_read_subrequest(struct netfs_io_request *rreq)
 {
+	struct netfs_io_subrequest *subreq;
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
 
+	subreq = netfs_alloc_subrequest(rreq);
+	if (!subreq)
+		return subreq;
+
 	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 
 	/* We add to the end of the list whilst the collector may be walking
@@ -182,31 +210,51 @@ void netfs_queue_read(struct netfs_io_request *rreq,
 	}
 
 	spin_unlock(&rreq->lock);
+	return subreq;
 }
 
 static void netfs_issue_read(struct netfs_io_request *rreq,
 			     struct netfs_io_subrequest *subreq)
 {
-	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
-			    subreq->content.slot, subreq->content.offset, subreq->len);
+	struct netfs_cache_resources *cres = &rreq->cache_resources;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	int ret;
+
+	_enter("R=%08x[%x]", rreq->debug_id, subreq->debug_index);
 
 	switch (subreq->source) {
 	case NETFS_DOWNLOAD_FROM_SERVER:
-		rreq->netfs_ops->issue_read(subreq);
-		break;
+		ret = rreq->netfs_ops->issue_read(subreq);
+		if (ret < 0)
+			goto fail;
+		return;
 	case NETFS_READ_FROM_CACHE:
-		netfs_read_cache_to_pagecache(rreq, subreq);
-		break;
+		netfs_stat(&netfs_n_rh_read);
+		ret = cres->ops->issue_read(subreq);
+		if (ret < 0)
+			goto fail;
+		return;
 	default:
-		bvecq_zero(&subreq->content, subreq->len);
+		WARN_ON_ONCE(1);
+		fallthrough;
+	case NETFS_FILL_WITH_ZEROES:
+		stream->issue_from = subreq->start + subreq->len;
+		stream->buffered -= subreq->len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+		if (!stream->buffered)
+			netfs_all_subreqs_queued(rreq);
+		bvecq_zero(&stream->dispatch_cursor, subreq->len);
 		subreq->transferred = subreq->len;
 		subreq->error = 0;
-		iov_iter_zero(subreq->len, &subreq->io_iter);
-		subreq->transferred = subreq->len;
-		netfs_read_subreq_terminated(subreq);
-		break;
+		return netfs_read_subreq_terminated(subreq);
 	}
+
+fail:
+	/* Ownership of subreq was returned to us. */
+	trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+	stream->buffered -= subreq->len;
+	subreq->error = ret;
+	netfs_read_subreq_terminated(subreq);
 }
 
 /*
@@ -225,20 +273,17 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		.cached_to[1]	= ULLONG_MAX,
 	};
 	struct fscache_occupancy *occ = &_occ;
-	unsigned long long start = rreq->start;
-	ssize_t size = rreq->len;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
 	int ret = 0;
 
 	_enter("R=%08x", rreq->debug_id);
 
-	bvecq_pos_set(&rreq->dispatch_cursor, &rreq->load_cursor);
-	bvecq_pos_set(&rreq->collect_cursor, &rreq->dispatch_cursor);
+	bvecq_pos_set(&stream->dispatch_cursor, &rreq->load_cursor);
+	bvecq_pos_set(&rreq->collect_cursor, &rreq->load_cursor);
 
 	do {
-		int (*prepare_read)(struct netfs_io_subrequest *subreq) = NULL;
 		struct netfs_io_subrequest *subreq;
-		unsigned long long hole_to, cache_to;
-		ssize_t slice;
+		unsigned long long hole_to, cache_to, stop;
 
 		/* If we don't have any, find out the next couple of data
 		 * extents from the cache, containing of following the
@@ -247,7 +292,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		 */
 		hole_to = occ->cached_from[0];
 		cache_to = occ->cached_to[0];
-		if (start >= cache_to) {
+		if (stream->issue_from >= cache_to) {
 			/* Extent exhausted; shuffle down. */
 			int i;
 
@@ -262,52 +307,50 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 			if (occ->cached_from[0] != ULLONG_MAX)
 				continue;
 
-			/* Get new extents */
-			ret = netfs_read_query_cache(rreq, occ);
-			if (ret < 0)
-				break;
+			/* Wait for the cache to enter the readable state and
+			 * get more extents.  If the cache doesn't become
+			 * readable, this leaves the default negative extent in
+			 * place.
+			 */
+			netfs_read_query_cache(rreq, occ);
 			continue;
 		}
 
-		subreq = netfs_alloc_subrequest(rreq);
+		subreq = netfs_alloc_read_subrequest(rreq);
 		if (!subreq) {
 			ret = -ENOMEM;
 			break;
 		}
 
-		subreq->start	= start;
-		subreq->len	= size;
-
-		netfs_queue_read(rreq, subreq);
+		subreq->start = stream->issue_from;
+		stop = stream->issue_from + stream->buffered;
 
 		unsigned long long zero_point = netfs_read_zero_point(rreq->inode);
 		unsigned long long zlimit = umin(zero_point, rreq->i_size);
 
 		_debug("rsub %llx %llx-%llx", subreq->start, hole_to, cache_to);
 
-		if (start >= hole_to && start < cache_to) {
+		if (stream->issue_from >= hole_to && stream->issue_from < cache_to) {
 			/* Overlap with a cached region, where the cache may
 			 * record a block of zeroes.
 			 */
-			_debug("cached s=%llx c=%llx l=%zx", start, cache_to, size);
-			subreq->len = umin(cache_to - start, size);
-			subreq->len = round_up(subreq->len, occ->granularity);
+			_debug("cached s=%llx c=%llx l=%zx",
+			       stream->issue_from, cache_to, stream->buffered);
+			subreq->len = umin(cache_to - stream->issue_from, stream->buffered);
 			if (occ->cached_type[0] == FSCACHE_EXTENT_ZERO) {
 				subreq->source = NETFS_FILL_WITH_ZEROES;
 				netfs_stat(&netfs_n_rh_zero);
 			} else {
+				subreq->len = round_up(subreq->len, occ->granularity);
 				subreq->source = NETFS_READ_FROM_CACHE;
-				prepare_read = rreq->cache_resources.ops->prepare_read;
 			}
-
-			trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-
-		} else if (subreq->start >= zlimit && size > 0) {
+		} else if (subreq->start >= zlimit &&
+			   subreq->start < stop) {
 			/* If this range lies beyond the zero-point, that part
 			 * can just be cleared locally.
 			 */
-			_debug("zero %llx-%llx", start, start + size);
-			subreq->len = size;
+			_debug("zero %llx-%llx", subreq->start, stop);
+			subreq->len = stream->buffered;
 			subreq->source = NETFS_FILL_WITH_ZEROES;
 			if (rreq->cache_resources.ops)
 				__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
@@ -317,10 +360,10 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 			 * this range lies beyond the zero-point or the EOF,
 			 * that part can just be cleared locally.
 			 */
-			unsigned long long limit = min3(zlimit, start + size, hole_to);
+			unsigned long long limit = min3(zlimit, stop, hole_to);
 
 			_debug("limit %llx %llx", rreq->i_size, zero_point);
-			_debug("download %llx-%llx", start, start + size);
+			_debug("download %llx-%llx", subreq->start, stop);
 			subreq->len = umin(limit - subreq->start, ULONG_MAX);
 			subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
 			if (rreq->cache_resources.ops)
@@ -328,41 +371,15 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 			netfs_stat(&netfs_n_rh_download);
 		}
 
-		if (size == 0) {
+		if (subreq->len == 0) {
 			pr_err("ZERO-LEN READ: R=%08x[%x] l=%zx/%zx s=%llx z=%llx i=%llx",
 			       rreq->debug_id, subreq->debug_index,
-			       subreq->len, size,
+			       subreq->len, stream->buffered,
 			       subreq->start, zero_point, rreq->i_size);
 			netfs_cancel_read(subreq, ret);
 			break;
 		}
 
-		rreq->io_streams[0].sreq_max_len = MAX_RW_COUNT;
-		rreq->io_streams[0].sreq_max_segs = INT_MAX;
-
-		if (prepare_read) {
-			ret = prepare_read(subreq);
-			if (ret < 0) {
-				netfs_cancel_read(subreq, ret);
-				break;
-			}
-			trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-		}
-
-		slice = netfs_prepare_read_iterator(subreq);
-		if (slice < 0) {
-			ret = slice;
-			netfs_cancel_read(subreq, ret);
-			break;
-		}
-		start += slice;
-		size -= slice;
-		if (size <= 0) {
-			smp_wmb(); /* Write lists before ALL_QUEUED. */
-			set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
-		}
-
-		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 		netfs_issue_read(rreq, subreq);
 		netfs_maybe_bulk_drop_ra_refs(rreq);
 
@@ -371,19 +388,19 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		if (test_bit(NETFS_RREQ_FAILED, &rreq->flags))
 			break;
 		cond_resched();
-	} while (size > 0);
+	} while (stream->buffered > 0);
 
-	if (unlikely(size > 0)) {
-		smp_wmb(); /* Write lists before ALL_QUEUED. */
-		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+	if (unlikely(!netfs_are_all_subreqs_queued(rreq))) {
+		netfs_all_subreqs_queued(rreq);
 		netfs_wake_collector(rreq);
 	}
 
 	/* Defer error return as we may need to wait for outstanding I/O. */
-	cmpxchg(&rreq->error, 0, ret);
+	if (ret < 0)
+		cmpxchg(&rreq->error, 0, ret);
 
 	bvecq_pos_unset(&rreq->load_cursor);
-	bvecq_pos_unset(&rreq->dispatch_cursor);
+	bvecq_pos_unset(&stream->dispatch_cursor);
 }
 
 /**
@@ -404,17 +421,22 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 void netfs_readahead(struct readahead_control *ractl)
 {
 	struct netfs_io_request *rreq;
+	struct netfs_io_stream *stream;
 	struct netfs_inode *ictx = netfs_inode(ractl->mapping->host);
 	unsigned long long start = readahead_pos(ractl);
 	ssize_t added;
 	size_t size = readahead_length(ractl);
 	int ret;
 
+	_enter("");
+
 	rreq = netfs_alloc_request(ractl->mapping, ractl->file, start, size,
 				   NETFS_READAHEAD);
 	if (IS_ERR(rreq))
 		return;
 
+	stream = &rreq->io_streams[0];
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &rreq->flags);
 
 	ret = netfs_begin_cache_read(rreq, ictx);
@@ -441,6 +463,8 @@ void netfs_readahead(struct readahead_control *ractl)
 	rreq->submitted = rreq->start + added;
 	rreq->cleaned_to = rreq->start;
 	rreq->front_folio_order = get_order(rreq->load_cursor.bvecq->bv[0].bv_len);
+	stream->issue_from = rreq->start;
+	stream->buffered = added;
 
 	netfs_read_to_pagecache(rreq);
 	netfs_maybe_bulk_drop_ra_refs(rreq);
@@ -456,6 +480,7 @@ EXPORT_SYMBOL(netfs_readahead);
  */
 static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio)
 {
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
 	struct bvecq *bq;
 	size_t fsize = folio_size(folio);
 
@@ -466,6 +491,8 @@ static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct fo
 	bvec_set_folio(&bq->bv[0], folio, fsize, 0);
 	bvecq_filled_to(bq, 1);
 	rreq->submitted = rreq->start + fsize;
+	stream->issue_from = rreq->start;
+	stream->buffered = fsize;
 	return 0;
 }
 
@@ -475,6 +502,7 @@ static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct fo
 static int netfs_read_gaps(struct file *file, struct folio *folio)
 {
 	struct netfs_io_request *rreq;
+	struct netfs_io_stream *stream;
 	struct address_space *mapping = folio->mapping;
 	struct netfs_group *group = netfs_folio_group(folio);
 	struct netfs_folio *finfo = netfs_folio_info(folio);
@@ -496,6 +524,7 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 		ret = PTR_ERR(rreq);
 		goto alloc_error;
 	}
+	stream = &rreq->io_streams[0];
 
 	ret = netfs_begin_cache_read(rreq, ctx);
 	if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
@@ -550,6 +579,8 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 	bvecq_filled_to(bq, slot);
 
 	rreq->submitted = rreq->start + flen;
+	stream->issue_from = rreq->start;
+	stream->buffered = flen;
 
 	netfs_read_to_pagecache(rreq);
 
@@ -628,6 +659,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
 		goto discard;
 
 	netfs_read_to_pagecache(rreq);
+
 	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret < 0 ? ret : 0;
diff --git a/fs/netfs/direct_read.c b/fs/netfs/direct_read.c
index 4d35f8efc633..5b19145ee4dd 100644
--- a/fs/netfs/direct_read.c
+++ b/fs/netfs/direct_read.c
@@ -16,6 +16,32 @@
 #include <linux/netfs.h>
 #include "internal.h"
 
+int netfs_prepare_unbuffered_read_buffer(struct netfs_io_subrequest *subreq,
+					 unsigned int max_segs)
+{
+	struct netfs_io_request *rreq = subreq->rreq;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &stream->dispatch_cursor);
+	len = bvecq_slice(&stream->dispatch_cursor, subreq->len, max_segs,
+			  &subreq->nr_segs);
+
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+
+	stream->buffered   -= subreq->len;
+	stream->issue_from += subreq->len;
+	rreq->submitted = stream->issue_from;
+
+	if (stream->buffered == 0)
+		netfs_all_subreqs_queued(rreq);
+	return 0;
+}
+
 /*
  * Perform a read to a buffer from the server, slicing up the region to be read
  * according to the network rsize.
@@ -23,17 +49,15 @@
 static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 {
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
-	unsigned long long start = rreq->start;
-	ssize_t size = rreq->len;
-	int ret;
 
-	bvecq_pos_set(&rreq->dispatch_cursor, &rreq->load_cursor);
-	bvecq_pos_transfer(&rreq->collect_cursor, &rreq->load_cursor);
+	bvecq_pos_set(&rreq->collect_cursor, &rreq->load_cursor);
+	bvecq_pos_transfer(&stream->dispatch_cursor, &rreq->load_cursor);
 
 	do {
 		struct netfs_io_subrequest *subreq;
+		int ret;
 
-		subreq = netfs_alloc_subrequest(rreq);
+		subreq = netfs_alloc_read_subrequest(rreq);
 		if (!subreq) {
 			/* Stash the error in the request if there's not
 			 * already an error set.
@@ -43,54 +67,32 @@ static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 		}
 
 		subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
-		subreq->start	= start;
-		subreq->len	= size;
-
-		netfs_queue_read(rreq, subreq);
+		subreq->start	= stream->issue_from;
+		subreq->len	= stream->buffered;
 
 		netfs_stat(&netfs_n_rh_download);
-		if (rreq->netfs_ops->prepare_read) {
-			ret = rreq->netfs_ops->prepare_read(subreq);
-			if (ret < 0) {
-				netfs_cancel_read(subreq, ret);
-				break;
-			}
-		}
 
-		bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
-		bvecq_pos_set(&subreq->content, &rreq->dispatch_cursor);
-		subreq->len = bvecq_slice(&rreq->dispatch_cursor,
-					  umin(size, stream->sreq_max_len),
-					  stream->sreq_max_segs,
-					  &subreq->nr_segs);
-
-		size -= subreq->len;
-		start += subreq->len;
-		rreq->submitted += subreq->len;
-		if (size <= 0) {
-			smp_wmb(); /* Write lists before ALL_QUEUED. */
-			set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+		ret = rreq->netfs_ops->issue_read(subreq);
+		if (ret < 0) {
+			stream->buffered -= subreq->len;
+			subreq->error = ret;
+			netfs_read_subreq_terminated(subreq);
+			break;
 		}
 
-		iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
-				    subreq->content.slot, subreq->content.offset, subreq->len);
-
-		rreq->netfs_ops->issue_read(subreq);
-
 		if (test_bit(NETFS_RREQ_PAUSE, &rreq->flags))
 			netfs_wait_for_paused_read(rreq);
 		if (test_bit(NETFS_RREQ_FAILED, &rreq->flags))
 			break;
 		cond_resched();
-	} while (size > 0);
+	} while (stream->buffered > 0);
 
-	if (unlikely(size > 0)) {
-		smp_wmb(); /* Write lists before ALL_QUEUED. */
-		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+	if (unlikely(stream->buffered > 0)) {
+		netfs_all_subreqs_queued(rreq);
 		netfs_wake_collector(rreq);
 	}
 
-	bvecq_pos_unset(&rreq->dispatch_cursor);
+	bvecq_pos_unset(&stream->dispatch_cursor);
 }
 
 /*
@@ -141,6 +143,7 @@ static ssize_t netfs_unbuffered_read(struct netfs_io_request *rreq, bool sync)
 ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *iter)
 {
 	struct netfs_io_request *rreq;
+	struct netfs_io_stream *stream;
 	ssize_t ret;
 	size_t orig_count = iov_iter_count(iter);
 	bool sync = is_sync_kiocb(iocb);
@@ -165,6 +168,8 @@ ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *i
 	netfs_stat(&netfs_n_rh_dio_read);
 	trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_dio_read);
 
+	stream = &rreq->io_streams[0];
+
 	/* If this is an async op, we have to keep track of the destination
 	 * buffer for ourselves as the caller's iterator will be trashed when
 	 * we return.
@@ -180,6 +185,8 @@ ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *i
 		goto error_put;
 
 	rreq->len = ret;
+	stream->buffered = ret;
+	stream->issue_from = rreq->start;
 
 	// TODO: Set up bounce buffer if needed
 
diff --git a/fs/netfs/direct_write.c b/fs/netfs/direct_write.c
index d81245a29eff..87f7bd02fe7c 100644
--- a/fs/netfs/direct_write.c
+++ b/fs/netfs/direct_write.c
@@ -9,6 +9,34 @@
 #include <linux/uio.h>
 #include "internal.h"
 
+/*
+ * Prepare the buffer for an unbuffered/DIO write.
+ */
+int netfs_prepare_unbuffered_write_buffer(struct netfs_io_subrequest *subreq,
+					  unsigned int max_segs)
+{
+	struct netfs_io_stream *stream = &subreq->rreq->io_streams[subreq->stream_nr];
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &stream->dispatch_cursor);
+	len = bvecq_slice(&stream->dispatch_cursor, subreq->len, max_segs,
+			  &subreq->nr_segs);
+
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+
+	// TODO: Wait here for completion of prev subreq
+
+	stream->issue_from += subreq->len;
+	stream->buffered   -= subreq->len;
+	if (stream->buffered == 0)
+		netfs_all_subreqs_queued(subreq->rreq);
+	return 0;
+}
+
 /*
  * Perform the cleanup rituals after an unbuffered write is complete.
  */
@@ -74,9 +102,9 @@ static void netfs_unbuffered_write_collect(struct netfs_io_request *wreq,
 
 	wreq->transferred += subreq->transferred;
 	if (subreq->transferred < subreq->len) {
-		bvecq_pos_unset(&wreq->dispatch_cursor);
-		bvecq_pos_transfer(&wreq->dispatch_cursor, &subreq->dispatch_pos);
-		bvecq_pos_advance(&wreq->dispatch_cursor, subreq->transferred);
+		bvecq_pos_unset(&stream->dispatch_cursor);
+		bvecq_pos_transfer(&stream->dispatch_cursor, &subreq->dispatch_pos);
+		bvecq_pos_advance(&stream->dispatch_cursor, subreq->transferred);
 	}
 
 	stream->collected_to = subreq->start + subreq->transferred;
@@ -85,6 +113,7 @@ static void netfs_unbuffered_write_collect(struct netfs_io_request *wreq,
 
 	trace_netfs_collect_stream(wreq, stream);
 	trace_netfs_collect_state(wreq, wreq->collected_to, 0);
+	/* TODO: Progressively clean up wreq->direct_bq */
 }
 
 /*
@@ -103,60 +132,44 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 
 	_enter("%llx", wreq->len);
 
-	bvecq_pos_set(&wreq->dispatch_cursor, &wreq->load_cursor);
-	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+	stream->issue_from = wreq->start;
+	stream->buffered = wreq->len;
+	bvecq_pos_set(&stream->dispatch_cursor, &wreq->load_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &stream->dispatch_cursor);
 
 	if (wreq->origin == NETFS_DIO_WRITE)
 		inode_dio_begin(wreq->inode);
 
-	stream->collected_to = wreq->start;
-
 	for (;;) {
 		bool retry = false;
 
 		if (!subreq) {
-			netfs_prepare_write(wreq, stream, wreq->start + wreq->transferred);
-			subreq = stream->construct;
-			stream->construct = NULL;
-		} else {
-			bvecq_pos_set(&subreq->dispatch_pos, &wreq->dispatch_cursor);
+			subreq = netfs_alloc_write_subreq(wreq, stream);
+			if (!subreq)
+				return -ENOMEM;
 		}
 
-		/* Check if (re-)preparation failed. */
-		if (unlikely(test_bit(NETFS_SREQ_FAILED, &subreq->flags))) {
-			netfs_write_subrequest_terminated(subreq, subreq->error);
-			wreq->error = subreq->error;
-			break;
+		ret = stream->issue_write(subreq);
+		if (ret < 0) {
+			/* Ownership of subreq was returned to us. */
+			trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+			netfs_write_subrequest_terminated(subreq, ret);
 		}
 
-		subreq->len = bvecq_slice(&wreq->dispatch_cursor, stream->sreq_max_len,
-					  stream->sreq_max_segs, &subreq->nr_segs);
-		bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-
-		iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
-				    subreq->content.bvecq, subreq->content.slot,
-				    subreq->content.offset,
-				    subreq->len);
-
-		if (!iov_iter_count(&subreq->io_iter))
-			break;
-
-		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
-		stream->issue_write(subreq);
-
-		/* Async, need to wait. */
-		netfs_wait_for_in_progress_stream(wreq, stream);
-
-		if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
+		ret = netfs_wait_for_in_progress_subreq(wreq, subreq);
+		if (ret < 0) {
+			if (ret != -EAGAIN) {
+				/* Don't need to lock here as the collection is
+				 * done in this thread.
+				 */
+				list_del_init(&subreq->rreq_link);
+				ret = subreq->error;
+				netfs_put_subrequest(subreq, netfs_sreq_trace_put_failed);
+				subreq = NULL;
+				goto failed;
+			}
 			retry = true;
-		} else if (test_bit(NETFS_SREQ_FAILED, &subreq->flags)) {
-			ret = subreq->error;
-			wreq->error = ret;
-			netfs_see_subrequest(subreq, netfs_sreq_trace_see_failed);
-			subreq = NULL;
-			break;
 		}
-		ret = 0;
 
 		if (!retry) {
 			netfs_unbuffered_write_collect(wreq, stream, subreq);
@@ -171,20 +184,21 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 			continue;
 		}
 
-		/* We need to retry the last subrequest, so first reset the
-		 * iterator, taking into account what, if anything, we managed
-		 * to transfer.
+		/* We need to retry the last subrequest, so first wind back the
+		 * buffer position.
 		 */
 		subreq->error = -EAGAIN;
 		trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
 		bvecq_pos_unset(&subreq->content);
-		bvecq_pos_unset(&wreq->dispatch_cursor);
-		bvecq_pos_transfer(&wreq->dispatch_cursor, &subreq->dispatch_pos);
+		bvecq_pos_unset(&stream->dispatch_cursor);
+		bvecq_pos_transfer(&stream->dispatch_cursor, &subreq->dispatch_pos);
 
+		stream->issue_from -= subreq->len - subreq->transferred;
+		stream->buffered   += subreq->len - subreq->transferred;
 		if (subreq->transferred > 0) {
-			wreq->transferred += subreq->transferred;
-			bvecq_pos_advance(&wreq->dispatch_cursor, subreq->transferred);
+			wreq->transferred  += subreq->transferred;
+			bvecq_pos_advance(&stream->dispatch_cursor, subreq->transferred);
 		}
 
 		if (stream->source == NETFS_UPLOAD_TO_SERVER &&
@@ -193,25 +207,21 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 
 		__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 		__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
-		__clear_bit(NETFS_SREQ_BOUNDARY, &subreq->flags);
 		__clear_bit(NETFS_SREQ_FAILED, &subreq->flags);
-		subreq->start		= wreq->start + wreq->transferred;
-		subreq->len		= wreq->len   - wreq->transferred;
+		__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+		subreq->start		= stream->issue_from;
+		subreq->len		= stream->buffered;
 		subreq->transferred	= 0;
 		subreq->retry_count	+= 1;
-		stream->sreq_max_len	= UINT_MAX;
-		stream->sreq_max_segs	= INT_MAX;
 
 		netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
 
-		if (stream->prepare_write)
-			stream->prepare_write(subreq);
 		__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 		netfs_stat(&netfs_n_wh_retry_write_subreq);
 	}
 
-	bvecq_pos_unset(&wreq->dispatch_cursor);
-	bvecq_pos_unset(&wreq->load_cursor);
+failed:
+	bvecq_pos_unset(&stream->dispatch_cursor);
 	netfs_unbuffered_write_done(wreq);
 	_leave(" = %d", ret);
 	return ret;
@@ -255,6 +265,7 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 	if (IS_ERR(wreq))
 		return PTR_ERR(wreq);
 
+	wreq->len = iov_iter_count(iter);
 	wreq->io_streams[0].avail = true;
 	trace_netfs_write(wreq, (iocb->ki_flags & IOCB_DIRECT ?
 				 netfs_write_trace_dio_write :
@@ -265,9 +276,7 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 		 * we have to save the source buffer as the iterator is only
 		 * good until we return.  In such a case, extract an iterator
 		 * to represent as much of the the output buffer as we can
-		 * manage.  Note that the extraction might not be able to
-		 * allocate a sufficiently large bvec array and may shorten the
-		 * request.
+		 * manage.  Note that the extraction may shorten the request.
 		 */
 		ssize_t n = netfs_extract_iter(iter, len, INT_MAX, iocb->ki_pos,
 					       &wreq->load_cursor.bvecq, 0, wreq->gfp);
@@ -282,8 +291,6 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 		       wreq->load_cursor.bvecq->max_slots);
 	}
 
-	__set_bit(NETFS_RREQ_USE_IO_ITER, &wreq->flags);
-
 	/* Copy the data into the bounce buffer and encrypt it. */
 	// TODO
 
diff --git a/fs/netfs/fscache_io.c b/fs/netfs/fscache_io.c
index fafa8c6bec57..a04cd96cd09a 100644
--- a/fs/netfs/fscache_io.c
+++ b/fs/netfs/fscache_io.c
@@ -239,7 +239,7 @@ void __fscache_write_to_cache(struct fscache_cookie *cookie,
 				    fscache_access_io_write) < 0)
 		goto abandon_free;
 
-	ret = cres->ops->prepare_write(cres, &start, &len, len, i_size, false);
+	ret = cres->ops->prepare_write_old(cres, &start, &len, len, i_size, false);
 	if (ret < 0)
 		goto abandon_end;
 
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 315b02f5fa67..52223743a2df 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -22,10 +22,9 @@
 /*
  * buffered_read.c
  */
-int netfs_read_query_cache(struct netfs_io_request *rreq,
-			   struct fscache_occupancy *occ);
-void netfs_queue_read(struct netfs_io_request *rreq,
-		      struct netfs_io_subrequest *subreq);
+void netfs_read_query_cache(struct netfs_io_request *rreq,
+			    struct fscache_occupancy *occ);
+struct netfs_io_subrequest *netfs_alloc_read_subrequest(struct netfs_io_request *rreq);
 void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error);
 int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 			     size_t offset, size_t len);
@@ -36,6 +35,18 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 void netfs_update_i_size(struct netfs_inode *ctx, struct inode *inode,
 			 loff_t pos, size_t copied);
 
+/*
+ * direct_read.c
+ */
+int netfs_prepare_unbuffered_read_buffer(struct netfs_io_subrequest *subreq,
+					 unsigned int max_segs);
+
+/*
+ * direct_write.c
+ */
+int netfs_prepare_unbuffered_write_buffer(struct netfs_io_subrequest *subreq,
+					  unsigned int max_segs);
+
 /*
  * main.c
  */
@@ -73,6 +84,8 @@ struct bvecq *netfs_buffer_make_space(struct netfs_io_request *rreq,
 				      enum netfs_bvecq_trace trace);
 void netfs_wake_collector(struct netfs_io_request *rreq);
 void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq);
+int netfs_wait_for_in_progress_subreq(struct netfs_io_request *rreq,
+				      struct netfs_io_subrequest *subreq);
 void netfs_wait_for_in_progress_stream(struct netfs_io_request *rreq,
 				       struct netfs_io_stream *stream);
 ssize_t netfs_wait_for_read(struct netfs_io_request *rreq);
@@ -118,16 +131,53 @@ void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error);
 /*
  * read_pgpriv2.c
  */
+#ifdef CONFIG_NETFS_PGPRIV2
+int netfs_prepare_pgpriv2_write_buffer(struct netfs_io_subrequest *subreq,
+				       unsigned int max_segs);
 void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *folio);
 void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq);
 bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *wreq);
+static inline bool netfs_using_pgpriv2(const struct netfs_io_request *rreq)
+{
+	return test_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags);
+}
+#else
+static inline int netfs_prepare_pgpriv2_write_buffer(struct netfs_io_subrequest *subreq,
+						     unsigned int max_segs)
+{
+	return -EIO;
+}
+static inline void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *folio)
+{
+}
+static inline void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
+{
+}
+static inline bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *wreq)
+{
+	return true;
+}
+static inline bool netfs_using_pgpriv2(const struct netfs_io_request *rreq)
+{
+	return false;
+}
+#endif
 
 /*
  * read_retry.c
  */
+int netfs_prepare_buffered_read_retry_buffer(struct netfs_io_subrequest *subreq,
+					     unsigned int max_segs);
+int netfs_reset_for_read_retry(struct netfs_io_subrequest *subreq);
 void netfs_retry_reads(struct netfs_io_request *rreq);
 void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq);
 
+/*
+ * read_single.c
+ */
+int netfs_prepare_read_single_buffer(struct netfs_io_subrequest *subreq,
+				     unsigned int max_segs);
+
 /*
  * stats.c
  */
@@ -198,24 +248,19 @@ void netfs_write_collection_worker(struct work_struct *work);
 /*
  * write_issue.c
  */
+struct netfs_writethrough;
 struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 						struct file *file,
 						loff_t start,
 						enum netfs_io_origin origin);
-void netfs_prepare_write(struct netfs_io_request *wreq,
-			 struct netfs_io_stream *stream,
-			 loff_t start);
-void netfs_reissue_write(struct netfs_io_stream *stream,
-			 struct netfs_io_subrequest *subreq);
-void netfs_issue_write(struct netfs_io_request *wreq,
-		       struct netfs_io_stream *stream);
-size_t netfs_advance_write(struct netfs_io_request *wreq,
-			   struct netfs_io_stream *stream,
-			   loff_t start, size_t len, bool to_eof);
+struct netfs_io_subrequest *netfs_alloc_write_subreq(struct netfs_io_request *wreq,
+						     struct netfs_io_stream *stream);
 
 /*
  * write_retry.c
  */
+int netfs_prepare_write_retry_buffer(struct netfs_io_subrequest *subreq,
+				     unsigned int max_segs);
 void netfs_retry_writes(struct netfs_io_request *wreq);
 
 /*
@@ -310,6 +355,25 @@ static inline bool netfs_check_subreq_in_progress(const struct netfs_io_subreque
 	return test_bit_acquire(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 }
 
+/*
+ * Indicate that we've generated and queued all the subrequests we're going to.
+ */
+static inline void netfs_all_subreqs_queued(struct netfs_io_request *rreq)
+{
+	smp_wmb(); /* Write lists before ALL_QUEUED. */
+	set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_all_queued);
+}
+
+/*
+ * Query if all subrequests are queued.
+ */
+static inline bool netfs_are_all_subreqs_queued(const struct netfs_io_request *rreq)
+{
+	/* Read lists after ALL_QUEUED. */
+	return test_bit_acquire(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+}
+
 /*
  * fscache-cache.c
  */
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index c923ba68e0af..9c56a51a9ac0 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -108,14 +108,14 @@ ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pag
 			}
 
 			if (got == 0) {
-				pr_err("extract_pages gave nothing from %zu, %zu\n",
+				pr_err("extract_pages gave nothing from %zx, %zx\n",
 				       extracted, max_len);
 				ret = -EIO;
 				goto out;
 			}
 
 			if (WARN(got > max_len,
-				 "%s: extract_pages overrun %zd > %zu bytes\n",
+				 "%s: extract_pages overrun %zx > %zx bytes\n",
 				 __func__, got, max_len)) {
 				ret = -EIO;
 				goto out;
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index 8fc4e5ef2152..0af45204fabc 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -250,6 +250,37 @@ void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq)
 		netfs_wake_collector(rreq);
 }
 
+/*
+ * Wait for a subrequest to come to completion.
+ */
+int netfs_wait_for_in_progress_subreq(struct netfs_io_request *rreq,
+				      struct netfs_io_subrequest *subreq)
+{
+	if (netfs_check_subreq_in_progress(subreq)) {
+		DEFINE_WAIT(myself);
+
+		trace_netfs_rreq(rreq, netfs_rreq_trace_wait_quiesce);
+		for (;;) {
+			prepare_to_wait(&rreq->waitq, &myself, TASK_UNINTERRUPTIBLE);
+
+			if (!netfs_check_subreq_in_progress(subreq))
+				break;
+
+			trace_netfs_sreq(subreq, netfs_sreq_trace_wait_for);
+			schedule();
+		}
+
+		trace_netfs_rreq(rreq, netfs_rreq_trace_waited_quiesce);
+		finish_wait(&rreq->waitq, &myself);
+	}
+
+	if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
+		return -EAGAIN;
+	if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
+		return subreq->error;
+	return 0;
+}
+
 /*
  * Wait for all outstanding I/O in a stream to quiesce.
  */
@@ -310,7 +341,7 @@ static int netfs_collect_in_app(struct netfs_io_request *rreq,
 			need_collect = true;
 			break;
 		}
-		if (subreq || !test_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags))
+		if (subreq || !netfs_are_all_subreqs_queued(rreq))
 			done = false;
 	}
 
@@ -380,7 +411,7 @@ static ssize_t netfs_wait_for_in_progress(struct netfs_io_request *rreq,
 		case NETFS_UNBUFFERED_WRITE:
 			break;
 		default:
-			if (rreq->submitted < rreq->len) {
+			if (rreq->transferred < rreq->len) {
 				trace_netfs_failure(rreq, NULL, ret, netfs_fail_short_read);
 				ret = -EIO;
 			}
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index 889590c6da69..906613261ee0 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -51,8 +51,6 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
 	rreq->i_size	= i_size_read(inode);
 	rreq->debug_id	= atomic_inc_return(&debug_ids);
 	rreq->wsize	= INT_MAX;
-	rreq->io_streams[0].sreq_max_len = ULONG_MAX;
-	rreq->io_streams[0].sreq_max_segs = 0;
 	spin_lock_init(&rreq->lock);
 	INIT_LIST_HEAD(&rreq->io_streams[0].subrequests);
 	INIT_LIST_HEAD(&rreq->io_streams[1].subrequests);
@@ -139,9 +137,11 @@ static void netfs_deinit_request(struct netfs_io_request *rreq)
 	if (rreq->cache_resources.ops)
 		rreq->cache_resources.ops->end_operation(&rreq->cache_resources);
 	bvecq_pos_unset(&rreq->load_cursor);
-	bvecq_pos_unset(&rreq->dispatch_cursor);
 	bvecq_pos_unset(&rreq->collect_cursor);
+	bvecq_pos_unset(&rreq->retry_cursor);
 	bvecq_put(rreq->spare);
+	for (int i = 0; i < NR_IO_STREAMS; i++)
+		bvecq_pos_unset(&rreq->io_streams[i].dispatch_cursor);
 
 	if (atomic_dec_and_test(&ictx->io_count))
 		wake_up_var(&ictx->io_count);
@@ -231,6 +231,7 @@ static void netfs_free_subrequest(struct netfs_io_subrequest *subreq)
 	struct netfs_io_request *rreq = subreq->rreq;
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_free);
+	WARN_ON_ONCE(!list_empty(&subreq->rreq_link));
 	if (rreq->netfs_ops->free_subrequest)
 		rreq->netfs_ops->free_subrequest(subreq);
 	bvecq_pos_unset(&subreq->dispatch_pos);
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 408208f29c0c..3c76fd0e755c 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -37,6 +37,7 @@ static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
 
 	if (subreq->start + subreq->transferred >= subreq->rreq->i_size)
 		__set_bit(NETFS_SREQ_HIT_EOF, &subreq->flags);
+	trace_netfs_rreq(subreq->rreq, netfs_rreq_trace_zero_unread);
 }
 
 /*
@@ -59,7 +60,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 	flush_dcache_folio(folio);
 	folio_mark_uptodate(folio);
 
-	if (!test_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags)) {
+	if (!netfs_using_pgpriv2(rreq)) {
 		finfo = netfs_folio_info(folio);
 		if (finfo) {
 			trace_netfs_folio(folio, netfs_folio_trace_filled_gaps);
@@ -292,8 +293,7 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 				transferred = front->len;
 				trace_netfs_rreq(rreq, netfs_rreq_trace_set_abandon);
 			}
-			if (front->start + transferred >= rreq->cleaned_to + fsize ||
-			    test_bit(NETFS_SREQ_HIT_EOF, &front->flags))
+			if (front->start + transferred >= rreq->cleaned_to + fsize)
 				netfs_read_unlock_folios(rreq, &notes);
 		} else {
 			stream->collected_to = front->start + transferred;
@@ -415,31 +415,6 @@ static void netfs_rreq_assess_dio(struct netfs_io_request *rreq)
 		inode_dio_end(rreq->inode);
 }
 
-/*
- * Do processing after reading a monolithic single object.
- */
-static void netfs_rreq_assess_single(struct netfs_io_request *rreq)
-{
-	struct netfs_io_stream *stream = &rreq->io_streams[0];
-
-	if (!rreq->error && stream->source == NETFS_DOWNLOAD_FROM_SERVER &&
-	    fscache_resources_valid(&rreq->cache_resources)) {
-		trace_netfs_rreq(rreq, netfs_rreq_trace_dirty);
-		netfs_single_mark_inode_dirty(rreq->inode);
-	}
-
-	if (rreq->iocb) {
-		rreq->iocb->ki_pos += rreq->transferred;
-		if (rreq->iocb->ki_complete) {
-			trace_netfs_rreq(rreq, netfs_rreq_trace_ki_complete);
-			rreq->iocb->ki_complete(
-				rreq->iocb, rreq->error ? rreq->error : rreq->transferred);
-		}
-	}
-	if (rreq->netfs_ops->done)
-		rreq->netfs_ops->done(rreq);
-}
-
 /*
  * Perform the collection of subrequests and folios.
  *
@@ -455,9 +430,8 @@ bool netfs_read_collection(struct netfs_io_request *rreq)
 	/* We're done when the app thread has finished posting subreqs and the
 	 * queue is empty.
 	 */
-	if (!test_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags))
+	if (!netfs_are_all_subreqs_queued(rreq))
 		return false;
-	smp_rmb(); /* Read ALL_QUEUED before subreq lists. */
 
 	if (!list_empty(&stream->subrequests))
 		return false;
@@ -475,7 +449,7 @@ bool netfs_read_collection(struct netfs_io_request *rreq)
 		netfs_rreq_assess_dio(rreq);
 		break;
 	case NETFS_READ_SINGLE:
-		netfs_rreq_assess_single(rreq);
+		WARN_ON_ONCE(1);
 		break;
 	default:
 		break;
@@ -607,6 +581,11 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq)
 		} else if (test_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags)) {
 			__set_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 			trace_netfs_sreq(subreq, netfs_sreq_trace_partial_read);
+		} else if (subreq->source == NETFS_READ_FROM_CACHE) {
+			netfs_stat(&netfs_n_rh_read_failed);
+			__set_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+			subreq->error = -ENODATA;
+			trace_netfs_sreq(subreq, netfs_sreq_trace_short);
 		} else {
 			__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
 			subreq->error = -ENODATA;
@@ -625,6 +604,8 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq)
 
 	if (unlikely(subreq->error < 0)) {
 		trace_netfs_failure(rreq, subreq, subreq->error, netfs_fail_read);
+		if (subreq->error == -ENOMEM)
+			set_bit(NETFS_RREQ_SAW_ENOMEM, &rreq->flags);
 		if (subreq->source == NETFS_READ_FROM_CACHE) {
 			netfs_stat(&netfs_n_rh_read_failed);
 			__set_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
diff --git a/fs/netfs/read_pgpriv2.c b/fs/netfs/read_pgpriv2.c
index b64a8594735a..f1b685c23eb6 100644
--- a/fs/netfs/read_pgpriv2.c
+++ b/fs/netfs/read_pgpriv2.c
@@ -13,8 +13,37 @@
 #include <linux/task_io_accounting_ops.h>
 #include "internal.h"
 
+int netfs_prepare_pgpriv2_write_buffer(struct netfs_io_subrequest *subreq,
+				       unsigned int max_segs)
+{
+	struct netfs_io_request *creq = subreq->rreq;
+	struct netfs_io_stream *stream = &creq->io_streams[1];
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &stream->dispatch_cursor);
+	len = bvecq_slice(&stream->dispatch_cursor, subreq->len, max_segs,
+			  &subreq->nr_segs);
+
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+
+	// TODO: Wait here for completion of prev subreq
+
+	stream->issue_from += subreq->len;
+	stream->buffered   -= subreq->len;
+	if (stream->buffered == 0)
+		netfs_all_subreqs_queued(creq);
+	return 0;
+}
+
 /*
- * [DEPRECATED] Copy a folio to the cache with PG_private_2 set.
+ * [DEPRECATED] Copy a folio to the cache with PG_private_2 set.  Note that the
+ * folio won't necessarily be contiguous with the previous one as there might
+ * be a mixture of folios read from the cache and downloaded from the server
+ * (or just zeroed).
  */
 static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio *folio)
 {
@@ -24,7 +53,6 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	size_t dio_size = PAGE_SIZE;
 	size_t fsize = folio_size(folio), flen = fsize;
 	loff_t fpos = folio_pos(folio), i_size;
-	bool to_eof = false;
 
 	_enter("");
 
@@ -44,12 +72,8 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	if (fpos + fsize > creq->i_size)
 		creq->i_size = i_size;
 
-	if (flen > i_size - fpos) {
+	if (flen > i_size - fpos)
 		flen = i_size - fpos;
-		to_eof = true;
-	} else if (flen == i_size - fpos) {
-		to_eof = true;
-	}
 
 	flen = round_up(flen, dio_size);
 
@@ -82,37 +106,9 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	creq->load_cursor.slot = slot;
 	creq->load_cursor.offset = 0;
 	creq->last_end = fpos + flen;
+	trace_netfs_wback(creq, folio, 0);
 
-	bvecq_pos_nudge(&creq->dispatch_cursor);
-	
-	cache->submit_off = 0;
-	cache->submit_len = flen;
-
-	/* Attach the folio to one or more subrequests.  For a big folio, we
-	 * could end up with thousands of subrequests if the wsize is small -
-	 * but we might need to wait during the creation of subrequests for
-	 * network resources (eg. SMB credits).
-	 */
-	do {
-		ssize_t part;
-
-		creq->dispatch_cursor.offset = cache->submit_off;
-
-		atomic64_set(&creq->issued_to, fpos + cache->submit_off);
-		part = netfs_advance_write(creq, cache, fpos + cache->submit_off,
-					   cache->submit_len, to_eof);
-		cache->submit_off += part;
-		if (part > cache->submit_len)
-			cache->submit_len = 0;
-		else
-			cache->submit_len -= part;
-	} while (cache->submit_len > 0);
-
-	bvecq_pos_step(&creq->dispatch_cursor);
-	atomic64_set(&creq->issued_to, fpos + fsize);
-
-	if (flen < fsize)
-		netfs_issue_write(creq, cache);
+	cache->buffered += flen;
 }
 
 /*
@@ -122,6 +118,7 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(
 	struct netfs_io_request *rreq, struct folio *folio)
 {
 	struct netfs_io_request *creq;
+	struct netfs_io_stream *cache;
 
 	if (!fscache_resources_valid(&rreq->cache_resources))
 		goto cancel;
@@ -131,13 +128,14 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(
 	if (IS_ERR(creq))
 		goto cancel;
 
-	if (!creq->io_streams[1].avail)
+	cache = &creq->io_streams[1];
+	if (!cache->avail)
 		goto cancel_put;
 
 	if (bvecq_buffer_init(&creq->load_cursor, creq->gfp) < 0)
 		goto cancel_put;
-	bvecq_pos_set(&creq->dispatch_cursor, &creq->load_cursor);
-	bvecq_pos_set(&creq->collect_cursor, &creq->dispatch_cursor);
+	bvecq_pos_set(&cache->dispatch_cursor, &creq->load_cursor);
+	bvecq_pos_set(&creq->collect_cursor, &creq->load_cursor);
 
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &creq->flags);
 	trace_netfs_copy2cache(rreq, creq);
@@ -180,25 +178,61 @@ void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *fo
 	netfs_pgpriv2_copy_folio(creq, folio);
 }
 
+/*
+ * Issue all pending writes on the cache stream.
+ */
+static void netfs_pgpriv2_issue_stream(struct netfs_io_request *wreq,
+				       struct netfs_io_stream *stream)
+{
+	atomic64_set_release(&stream->issued_to, wreq->start);
+
+	do {
+		struct netfs_io_subrequest *subreq;
+		int ret;
+
+		subreq = netfs_alloc_write_subreq(wreq, stream);
+		if (!subreq)
+			break;
+
+		ret = stream->issue_write(subreq);
+		if (ret < 0) {
+			/* Ownership of subreq was returned to us.  Punt the
+			 * error to the collector and stop the issuance of new
+			 * subreqs.
+			 */
+			trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+			stream->buffered -= subreq->len;
+			netfs_write_subrequest_terminated(subreq, ret);
+			break;
+		}
+		/* We no longer own subreq. */
+
+		if (test_bit(NETFS_RREQ_SAW_ENOMEM, &wreq->flags))
+			break;
+
+	} while (stream->buffered > 0);
+
+	netfs_all_subreqs_queued(wreq);
+}
+
 /*
  * [DEPRECATED] End writing to the cache, flushing out any outstanding writes.
  */
 void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
 {
 	struct netfs_io_request *creq = rreq->copy_to_cache;
+	struct netfs_io_stream *stream = &creq->io_streams[1];
 
 	if (IS_ERR_OR_NULL(creq))
 		return;
 
-	netfs_issue_write(creq, &creq->io_streams[1]);
-	smp_wmb(); /* Write lists before ALL_QUEUED. */
-	set_bit(NETFS_RREQ_ALL_QUEUED, &creq->flags);
+	netfs_pgpriv2_issue_stream(creq, stream);
 	trace_netfs_rreq(rreq, netfs_rreq_trace_end_copy_to_cache);
 	if (list_empty_careful(&creq->io_streams[1].subrequests))
 		netfs_wake_collector(creq);
 
 	netfs_put_request(creq, netfs_rreq_trace_put_return);
-	creq->copy_to_cache = NULL;
+	rreq->copy_to_cache = NULL;
 }
 
 /*
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index e691b42c19b9..df74c8209cad 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -9,20 +9,55 @@
 #include <linux/slab.h>
 #include "internal.h"
 
-static void netfs_reissue_read(struct netfs_io_request *rreq,
-			       struct netfs_io_subrequest *subreq)
+/*
+ * Prepare the I/O buffer on a buffered read subrequest for the filesystem to
+ * use as a bvec queue.
+ */
+int netfs_prepare_buffered_read_retry_buffer(struct netfs_io_subrequest *subreq,
+					     unsigned int max_segs)
 {
-	bvecq_pos_unset(&subreq->content);
+	struct netfs_io_request *rreq = subreq->rreq;
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &rreq->retry_cursor);
 	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
-			    subreq->content.slot, subreq->content.offset, subreq->len);
-	iov_iter_advance(&subreq->io_iter, subreq->transferred);
+	len = bvecq_slice(&rreq->retry_cursor, subreq->len, max_segs,
+			  &subreq->nr_segs);
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+	rreq->retry_buffered -= subreq->len;
+	rreq->retry_start    += subreq->len;
+	return 0;
+}
 
-	subreq->error = 0;
+/*
+ * Reset the state of the subrequest and discard any buffering so that we can
+ * retry (where this may include sending it to the server instead of the
+ * cache).
+ */
+int netfs_reset_for_read_retry(struct netfs_io_subrequest *subreq)
+{
+	trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+
+	if (subreq->retry_count > 3) {
+		trace_netfs_sreq(subreq, netfs_sreq_trace_too_many_retries);
+		return subreq->error;
+	}
+
+	subreq->retry_count++;
 	__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+	__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+	__clear_bit(NETFS_SREQ_FAILED, &subreq->flags);
 	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
-	netfs_stat(&netfs_n_rh_retry_read_subreq);
-	subreq->rreq->netfs_ops->issue_read(subreq);
+	bvecq_pos_unset(&subreq->content);
+	bvecq_pos_unset(&subreq->dispatch_pos);
+	subreq->error = 0;
+	subreq->transferred = 0;
+	netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
+	netfs_stat(&netfs_n_wh_retry_write_subreq);
+	return 0;
 }
 
 /*
@@ -33,8 +68,8 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 {
 	struct netfs_io_subrequest *subreq;
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
-	struct bvecq_pos dispatch_cursor = {};
 	struct list_head *next;
+	int ret;
 
 	_enter("R=%x", rreq->debug_id);
 
@@ -44,46 +79,19 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 	if (rreq->netfs_ops->retry_request)
 		rreq->netfs_ops->retry_request(rreq, NULL);
 
-	/* If there's no renegotiation to do, just resend each retryable subreq
-	 * up to the first permanently failed one.
-	 */
-	if (!rreq->netfs_ops->prepare_read &&
-	    !rreq->cache_resources.ops) {
-		list_for_each_entry(subreq, &stream->subrequests, rreq_link) {
-			if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
-				break;
-			if (__test_and_clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-				subreq->retry_count++;
-				netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-				netfs_reissue_read(rreq, subreq);
-			}
-		}
-		return;
-	}
+	/* Read pointer to subreq before reading subreq state. */
+	next = smp_load_acquire(&stream->subrequests.next);
 
-	/* Okay, we need to renegotiate all the download requests and flip any
-	 * failed cache reads over to being download requests and negotiate
-	 * those also.  All fully successful subreqs have been removed from the
-	 * list and any spare data from those has been donated.
-	 *
-	 * What we do is decant the list and rebuild it one subreq at a time so
-	 * that we don't end up with donations jumping over a gap we're busy
-	 * populating with smaller subrequests.  In the event that the subreq
-	 * we just launched finishes before we insert the next subreq, it'll
-	 * fill in rreq->prev_donated instead.
-	 *
-	 * Note: Alternatively, we could split the tail subrequest right before
-	 * we reissue it and fix up the donations under lock.
+	/* Renegotiate all the download requests and flip any failed cache
+	 * reads over to being download requests and negotiate those also.
 	 */
-	next = stream->subrequests.next;
-
 	do {
 		struct netfs_io_subrequest *from, *to, *tmp;
-		unsigned long long start, len;
-		size_t part;
-		bool boundary = false, subreq_superfluous = false;
+		unsigned long long start;
+		size_t len;
+		bool subreq_superfluous = false;
 
-		bvecq_pos_unset(&dispatch_cursor);
+		bvecq_pos_unset(&rreq->retry_cursor);
 
 		/* Go through the subreqs and find the next span of contiguous
 		 * buffer that we then rejig (cifs, for example, needs the
@@ -98,8 +106,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		       rreq->debug_id, from->debug_index,
 		       from->start, from->transferred, from->len);
 
-		if (test_bit(NETFS_SREQ_FAILED, &from->flags) ||
-		    !test_bit(NETFS_SREQ_NEED_RETRY, &from->flags)) {
+		if (!test_bit(NETFS_SREQ_NEED_RETRY, &from->flags)) {
 			subreq = from;
 			goto abandon;
 		}
@@ -113,20 +120,21 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 			subreq = list_entry(next, struct netfs_io_subrequest, rreq_link);
 			if (subreq->start != start + len ||
 			    subreq->transferred > 0 ||
-			    test_bit(NETFS_SREQ_BOUNDARY, &subreq->flags) ||
 			    !test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
 				break;
 			to = subreq;
 			len += to->len;
 		}
 
-		_debug(" - range: %llx-%llx %llx", start, start + len - 1, len);
+		_debug(" - range: %llx-%llx %zx", start, start + len - 1, len);
 
 		/* Determine the set of buffers we're going to use.  Each
-		 * subreq gets a subset of a single overall contiguous buffer.
+		 * subreq takes a subset of a single overall contiguous buffer.
 		 */
-		bvecq_pos_transfer(&dispatch_cursor, &from->dispatch_pos);
-		bvecq_pos_advance(&dispatch_cursor, from->transferred);
+		bvecq_pos_transfer(&rreq->retry_cursor, &from->dispatch_pos);
+		bvecq_pos_advance(&rreq->retry_cursor, from->transferred);
+		rreq->retry_start = start;
+		rreq->retry_buffered = len;
 		from->transferred = 0;
 
 		/* Work through the sublist.  The chain of buffers we're going
@@ -135,51 +143,30 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		 */
 		subreq = from;
 		list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
-			if (!len) {
+			if (rreq->retry_buffered == 0) {
 				subreq_superfluous = true;
 				break;
 			}
 			subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
-			subreq->start	= start;
-			subreq->len	= len;
-			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
-			__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
-			subreq->retry_count++;
-			subreq->transferred = 0;
+			subreq->start	= rreq->retry_start;
+			subreq->len	= rreq->retry_buffered;
 
-			bvecq_pos_unset(&subreq->content);
-			bvecq_pos_unset(&subreq->dispatch_pos);
-			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
-
-			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
-
-			/* Renegotiate max_len (rsize) */
-			stream->sreq_max_len = len;
-			stream->sreq_max_segs = INT_MAX;
-			if (rreq->netfs_ops->prepare_read &&
-			    rreq->netfs_ops->prepare_read(subreq) < 0) {
-				trace_netfs_sreq(subreq, netfs_sreq_trace_reprep_failed);
+			ret = netfs_reset_for_read_retry(subreq);
+			if (ret < 0) {
 				__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
+				rreq->error = ret;
 				goto abandon;
 			}
 
-			part = bvecq_slice(&dispatch_cursor,
-					   umin(len, stream->sreq_max_len),
-					   stream->sreq_max_segs,
-					   &subreq->nr_segs);
-			subreq->len = part;
-
-			len -= part;
-			start += part;
-			if (!len) {
-				if (boundary)
-					__set_bit(NETFS_SREQ_BOUNDARY, &subreq->flags);
-			} else {
-				__clear_bit(NETFS_SREQ_BOUNDARY, &subreq->flags);
+			netfs_stat(&netfs_n_rh_download);
+			ret = rreq->netfs_ops->issue_read(subreq);
+			if (ret < 0) {
+				subreq->error = ret;
+				netfs_read_subreq_terminated(subreq);
+				goto abandon_after;
 			}
-
-			netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-			netfs_reissue_read(rreq, subreq);
+			if (test_bit(NETFS_RREQ_SAW_ENOMEM, &rreq->flags))
+				goto abandon_after;
 			if (subreq == to) {
 				subreq_superfluous = false;
 				break;
@@ -189,7 +176,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		/* If we managed to use fewer subreqs, we can discard the
 		 * excess; if we used the same number, then we're done.
 		 */
-		if (!len) {
+		if (rreq->retry_buffered == 0) {
 			if (!subreq_superfluous)
 				continue;
 			list_for_each_entry_safe_from(subreq, tmp,
@@ -207,7 +194,8 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		}
 
 		/* We ran out of subrequests, so we need to allocate some more
-		 * and insert them after.
+		 * and insert them after.  They must start with being marked
+		 * for retry to switch to the retry cursor.
 		 */
 		do {
 			subreq = netfs_alloc_subrequest(rreq);
@@ -216,8 +204,8 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				goto abandon_after;
 			}
 			subreq->source		= NETFS_DOWNLOAD_FROM_SERVER;
-			subreq->start		= start;
-			subreq->len		= len;
+			subreq->start		= rreq->retry_start;
+			subreq->len		= rreq->retry_buffered;
 			subreq->stream_nr	= stream->stream_nr;
 			subreq->retry_count	= 1;
 
@@ -225,43 +213,32 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 					     refcount_read(&subreq->ref),
 					     netfs_sreq_trace_new);
 
+			__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
+
 			spin_lock(&rreq->lock);
+			/* Write IN_PROGRESS before pointer to new subreq */
+			smp_wmb();
 			list_add(&subreq->rreq_link, &to->rreq_link);
 			spin_unlock(&rreq->lock);
 			to = subreq;
 			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
-			stream->sreq_max_len	= umin(len, rreq->rsize);
-			stream->sreq_max_segs	= INT_MAX;
-
 			netfs_stat(&netfs_n_rh_download);
-			if (rreq->netfs_ops->prepare_read(subreq) < 0) {
-				trace_netfs_sreq(subreq, netfs_sreq_trace_reprep_failed);
-				__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
-				goto abandon;
-			}
-
-			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
-			part = bvecq_slice(&dispatch_cursor,
-					   umin(len, stream->sreq_max_len),
-					   stream->sreq_max_segs,
-					   &subreq->nr_segs);
-			subreq->len = part;
-
-			len -= part;
-			start += part;
-			if (!len && boundary) {
-				__set_bit(NETFS_SREQ_BOUNDARY, &to->flags);
-				boundary = false;
+			ret = rreq->netfs_ops->issue_read(subreq);
+			if (ret < 0) {
+				subreq->error = ret;
+				netfs_read_subreq_terminated(subreq);
+				goto abandon_after;
 			}
+			if (test_bit(NETFS_RREQ_SAW_ENOMEM, &rreq->flags))
+				goto abandon_after;
 
-			netfs_reissue_read(rreq, subreq);
-		} while (len);
+		} while (rreq->retry_buffered > 0);
 
 	} while (!list_is_head(next, &stream->subrequests));
 
 out:
-	bvecq_pos_unset(&dispatch_cursor);
+	bvecq_pos_unset(&rreq->retry_cursor);
 	return;
 
 	/* If we hit an error, fail all remaining incomplete subrequests */
@@ -327,6 +304,7 @@ void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
 			}
 			trace_netfs_folio(folio, netfs_folio_trace_abandon);
 			folio_unlock(folio);
+			p->bv[slot].bv_page = NULL;
 		}
 	}
 }
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index cd770703d99f..12bbdf0f70a8 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -16,6 +16,22 @@
 #include <linux/netfs.h>
 #include "internal.h"
 
+int netfs_prepare_read_single_buffer(struct netfs_io_subrequest *subreq,
+				     unsigned int max_segs)
+{
+	struct netfs_io_request *rreq = subreq->rreq;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+
+	bvecq_pos_set(&subreq->dispatch_pos, &rreq->load_cursor);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+
+	stream->buffered = 0;
+	stream->issue_from += subreq->len;
+	rreq->submitted = stream->issue_from;
+	netfs_all_subreqs_queued(rreq);
+	return 0;
+}
+
 /**
  * netfs_single_mark_inode_dirty - Mark a single, monolithic object inode dirty
  * @inode: The inode to mark
@@ -58,17 +74,6 @@ static int netfs_single_begin_cache_read(struct netfs_io_request *rreq, struct n
 	return fscache_begin_read_operation(&rreq->cache_resources, netfs_i_cookie(ctx));
 }
 
-static void netfs_single_read_cache(struct netfs_io_request *rreq,
-				    struct netfs_io_subrequest *subreq)
-{
-	struct netfs_cache_resources *cres = &rreq->cache_resources;
-
-	_enter("R=%08x[%x]", rreq->debug_id, subreq->debug_index);
-	netfs_stat(&netfs_n_rh_read);
-	cres->ops->read(cres, subreq->start, &subreq->io_iter, NETFS_READ_HOLE_FAIL,
-			netfs_cache_read_terminated, subreq);
-}
-
 /*
  * Perform a read to a buffer from the cache or the server.  Only a single
  * subreq is permitted as the object must be fetched in a single transaction.
@@ -84,73 +89,84 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 		.cached_to[1]	= ULLONG_MAX,
 	};
 	struct netfs_io_subrequest *subreq;
-	int ret = 0;
+	int ret;
+
+	netfs_read_query_cache(rreq, &occ);
 
-	subreq = netfs_alloc_subrequest(rreq);
+	subreq = netfs_alloc_read_subrequest(rreq);
 	if (!subreq)
 		return -ENOMEM;
 
-	subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
 	subreq->start	= 0;
 	subreq->len	= rreq->len;
 
-	bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
-	bvecq_pos_set(&subreq->content, &rreq->dispatch_cursor);
-
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
-			    subreq->content.slot, subreq->content.offset, subreq->len);
-
-	netfs_queue_read(rreq, subreq);
+	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
 
 	/* Try to use the cache if the cache content matches the size of the
 	 * remote file.
 	 */
-	netfs_read_query_cache(rreq, &occ);
 	if (occ.cached_from[0] == 0 &&
-	    occ.cached_to[0] >= rreq->len)
-		subreq->source = NETFS_READ_FROM_CACHE;
+	    occ.cached_to[0] >= rreq->len) {
+		struct netfs_cache_resources *cres = &rreq->cache_resources;
 
-	switch (subreq->source) {
-	case NETFS_DOWNLOAD_FROM_SERVER:
-		netfs_stat(&netfs_n_rh_download);
-		if (rreq->netfs_ops->prepare_read) {
-			ret = rreq->netfs_ops->prepare_read(subreq);
-			if (ret < 0)
-				goto cancel;
+		subreq->source = NETFS_READ_FROM_CACHE;
+		netfs_stat(&netfs_n_rh_read);
+		ret = cres->ops->issue_read(subreq);
+		if (ret < 0) {
+			subreq->error = ret;
+			netfs_read_subreq_terminated(subreq);
 		}
 
-		smp_wmb(); /* Write lists before ALL_QUEUED. */
-		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
-		rreq->netfs_ops->issue_read(subreq);
-		rreq->submitted += subreq->len;
-		break;
-	case NETFS_READ_FROM_CACHE:
-		if (rreq->cache_resources.ops->prepare_read) {
-			ret = rreq->cache_resources.ops->prepare_read(subreq);
-			if (ret < 0)
-				goto cancel;
+		ret = netfs_wait_for_in_progress_subreq(rreq, subreq);
+		if (ret == 0)
+			goto success;
+		if (ret == -ENOMEM)
+			goto cancel;
+
+		/* Didn't manage to retrieve from the cache, so toss it to the
+		 * server instead.
+		 */
+		if (netfs_reset_for_read_retry(subreq) < 0)
+			goto cancel;
+	}
+
+	__set_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
+
+	/* Try to send it to the cache. */
+	for (;;) {
+		subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
+		netfs_stat(&netfs_n_rh_download);
+		ret = rreq->netfs_ops->issue_read(subreq);
+		if (ret < 0) {
+			subreq->error = ret;
+			netfs_read_subreq_terminated(subreq);
 		}
 
-		smp_wmb(); /* Write lists before ALL_QUEUED. */
-		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
-		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
-		netfs_single_read_cache(rreq, subreq);
-		rreq->submitted += subreq->len;
-		ret = 0;
-		break;
-	default:
-		pr_warn("Unexpected single-read source %u\n", subreq->source);
-		WARN_ON_ONCE(true);
-		ret = -EIO;
-		goto cancel;
+		ret = netfs_wait_for_in_progress_subreq(rreq, subreq);
+		if (ret == 0)
+			goto success;
+		if (ret == -ENOMEM)
+			goto cancel;
+		if (ret != -EAGAIN)
+			goto failed;
+		if (netfs_reset_for_read_retry(subreq) < 0)
+			goto cancel;
 	}
 
-	return ret;
+success:
+	rreq->transferred = subreq->transferred;
+	list_del_init(&subreq->rreq_link);
+	netfs_put_subrequest(subreq, netfs_sreq_trace_put_consumed);
+	return 0;
 cancel:
-	netfs_cancel_read(subreq, ret);
-	smp_wmb(); /* Write lists before ALL_QUEUED. */
-	set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
-	netfs_wake_collector(rreq);
+	rreq->error = ret;
+	list_del_init(&subreq->rreq_link);
+	netfs_put_subrequest(subreq, netfs_sreq_trace_put_cancel);
+	return ret;
+failed:
+	rreq->error = ret;
+	list_del_init(&subreq->rreq_link);
+	netfs_put_subrequest(subreq, netfs_sreq_trace_put_failed);
 	return ret;
 }
 
@@ -182,7 +198,7 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	if (IS_ERR(rreq))
 		return PTR_ERR(rreq);
 
-	ret = netfs_extract_iter(iter, rreq->len, INT_MAX, 0, &rreq->dispatch_cursor.bvecq,
+	ret = netfs_extract_iter(iter, rreq->len, INT_MAX, 0, &rreq->load_cursor.bvecq,
 				 0, rreq->gfp);
 	if (ret < 0)
 		goto cleanup_free;
@@ -198,9 +214,29 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	netfs_stat(&netfs_n_rh_read_single);
 	trace_netfs_read(rreq, 0, rreq->len, netfs_read_trace_read_single);
 
-	netfs_single_dispatch_read(rreq);
+	ret = netfs_single_dispatch_read(rreq);
+
+	trace_netfs_rreq(rreq, netfs_rreq_trace_complete);
+	if (ret == 0) {
+		task_io_account_read(rreq->transferred);
+
+		if (test_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags) &&
+		    fscache_resources_valid(&rreq->cache_resources)) {
+			trace_netfs_rreq(rreq, netfs_rreq_trace_dirty);
+			netfs_single_mark_inode_dirty(rreq->inode);
+		}
+		ret = rreq->transferred;
+	}
+
+	if (rreq->netfs_ops->done)
+		rreq->netfs_ops->done(rreq);
+
+	netfs_wake_rreq_flag(rreq, NETFS_RREQ_IN_PROGRESS, netfs_rreq_trace_wake_ip);
+	/* As we cleared NETFS_RREQ_IN_PROGRESS, we acquired its ref. */
+	netfs_put_request(rreq, netfs_rreq_trace_put_work_ip);
+
+	trace_netfs_rreq(rreq, netfs_rreq_trace_done);
 
-	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret;
 
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 0cd145472a17..33302cdb08f0 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -28,8 +28,8 @@ static void netfs_dump_request(const struct netfs_io_request *rreq)
 	       rreq->origin, rreq->error);
 	pr_err("  st=%llx tsl=%zx/%llx/%llx\n",
 	       rreq->start, rreq->transferred, rreq->submitted, rreq->len);
-	pr_err("  cci=%llx/%llx/%llx\n",
-	       rreq->cleaned_to, rreq->collected_to, atomic64_read(&rreq->issued_to));
+	pr_err("  cci=%llx/%llx\n",
+	       rreq->cleaned_to, rreq->collected_to);
 	pr_err("  iw=%pSR\n", rreq->netfs_ops->issue_write);
 	for (int i = 0; i < NR_IO_STREAMS; i++) {
 		const struct netfs_io_subrequest *sreq;
@@ -38,8 +38,9 @@ static void netfs_dump_request(const struct netfs_io_request *rreq)
 		pr_err("  str[%x] s=%x e=%d acnf=%u,%u,%u,%u\n",
 		       s->stream_nr, s->source, s->error,
 		       s->avail, s->active, s->need_retry, s->failed);
-		pr_err("  str[%x] ct=%llx t=%zx\n",
-		       s->stream_nr, s->collected_to, s->transferred);
+		pr_err("  str[%x] it=%llx ct=%llx t=%zx\n",
+		       s->stream_nr, atomic64_read(&s->issued_to),
+		       s->collected_to, s->transferred);
 		list_for_each_entry(sreq, &s->subrequests, rreq_link) {
 			pr_err("  sreq[%x:%x] sc=%u s=%llx t=%zx/%zx r=%d f=%lx\n",
 			       sreq->stream_nr, sreq->debug_index, sreq->source,
@@ -56,7 +57,7 @@ static void netfs_dump_request(const struct netfs_io_request *rreq)
  */
 int netfs_folio_written_back(struct folio *folio)
 {
-	enum netfs_folio_trace why = netfs_folio_trace_clear;
+	enum netfs_folio_trace why = netfs_folio_trace_endwb;
 	struct inode *inode = folio_inode(folio);
 	struct netfs_inode *ictx = netfs_inode(inode);
 	struct netfs_folio *finfo;
@@ -79,13 +80,13 @@ int netfs_folio_written_back(struct folio *folio)
 		group = finfo->netfs_group;
 		gcount++;
 		kfree(finfo);
-		why = netfs_folio_trace_clear_s;
+		why = netfs_folio_trace_endwb_s;
 		goto end_wb;
 	}
 
 	if ((group = netfs_folio_group(folio))) {
 		if (group == NETFS_FOLIO_COPY_TO_CACHE) {
-			why = netfs_folio_trace_clear_cc;
+			why = netfs_folio_trace_endwb_cc;
 			folio_detach_private(folio);
 			goto end_wb;
 		}
@@ -98,7 +99,7 @@ int netfs_folio_written_back(struct folio *folio)
 		if (!folio_test_dirty(folio)) {
 			folio_detach_private(folio);
 			gcount++;
-			why = netfs_folio_trace_clear_g;
+			why = netfs_folio_trace_endwb_g;
 		}
 	}
 
@@ -151,6 +152,12 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 			slot  = wreq->collect_cursor.slot;
 		}
 
+		if (!bvecq->bv[slot].bv_page) {
+			WARN_ONCE(1, "R=%08x slot already cleared?\n", wreq->debug_id);
+			fsize = bvecq->bv[slot].bv_len;
+			goto skip;
+		}
+
 		folio = page_folio(bvecq->bv[slot].bv_page);
 		if (WARN_ONCE(!folio_test_writeback(folio),
 			      "R=%08x: folio %lx is not under writeback\n",
@@ -175,6 +182,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		*notes |= MADE_PROGRESS;
 
 		bvecq->bv[slot].bv_page = NULL;
+	skip:
 		slot++;
 		if (fpos + fsize >= collected_to)
 			break;
@@ -225,9 +233,7 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 	trace_netfs_rreq(wreq, netfs_rreq_trace_collect);
 
 reassess_streams:
-	/* Order reading the issued_to point before reading the queue it refers to. */
-	issued_to = atomic64_read_acquire(&wreq->issued_to);
-	smp_rmb();
+	issued_to = ULLONG_MAX;
 	collected_to = ULLONG_MAX;
 	if (wreq->origin == NETFS_WRITEBACK ||
 	    wreq->origin == NETFS_PGPRIV2_COPY_TO_CACHE)
@@ -241,17 +247,34 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 	 * to the tail whilst we're doing this.
 	 */
 	for (s = 0; s < NR_IO_STREAMS; s++) {
+		unsigned long long s_issued_to;
+
 		stream = &wreq->io_streams[s];
-		/* Read active flag before list pointers */
+		/* Read active flag before issued_to */
 		if (!smp_load_acquire(&stream->active))
 			continue;
 
-		front = list_first_entry_or_null_acquire(&stream->subrequests,
-							 struct netfs_io_subrequest, rreq_link);
-		/* Read first subreq pointer before IN_PROGRESS flag. */
-
-		while (front) {
+		for (;;) {
 			bool cancelled;
+
+			/* Order reading the issued_to point before reading the
+			 * queue it refers to.
+			 */
+			s_issued_to = atomic64_read_acquire(&stream->issued_to);
+			if (s_issued_to < issued_to)
+				issued_to = s_issued_to;
+
+			front = list_first_entry_or_null_acquire(&stream->subrequests,
+								 struct netfs_io_subrequest,
+								 rreq_link);
+			/* Read first subreq pointer before IN_PROGRESS flag. */
+			if (!front) {
+				if (stream->source == NETFS_UPLOAD_TO_SERVER &&
+				    test_bit(NETFS_RREQ_PAUSE, &wreq->flags))
+					notes |= MADE_PROGRESS;
+				break;
+			}
+
 			trace_netfs_collect_sreq(wreq, front);
 			//_debug("sreq [%x] %llx %zx/%zx",
 			//       front->debug_index, front->start, front->transferred, front->len);
@@ -419,9 +442,8 @@ bool netfs_write_collection(struct netfs_io_request *wreq)
 	/* We're done when the app thread has finished posting subreqs and all
 	 * the queues in all the streams are empty.
 	 */
-	if (!test_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags))
+	if (!netfs_are_all_subreqs_queued(wreq))
 		return false;
-	smp_rmb(); /* Read ALL_QUEUED before lists. */
 
 	transferred = LONG_MAX;
 	for (s = 0; s < NR_IO_STREAMS; s++) {
@@ -541,6 +563,8 @@ void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error)
 
 	if (IS_ERR_VALUE(transferred_or_error)) {
 		subreq->error = transferred_or_error;
+		if (transferred_or_error == -ENOMEM)
+			set_bit(NETFS_RREQ_SAW_ENOMEM, &wreq->flags);
 
 		switch (subreq->source) {
 		case NETFS_WRITE_TO_CACHE:
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 70a7a92028e4..b9a6f9385fdb 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -36,6 +36,33 @@
 #include <linux/pagemap.h>
 #include "internal.h"
 
+#define NOTE_UPLOAD_AVAIL	0x001	/* Upload is available */
+#define NOTE_CACHE_AVAIL	0x002	/* Local cache is available */
+#define NOTE_CACHE_COPY		0x004	/* Copy folio to cache */
+#define NOTE_UPLOAD		0x008	/* Upload folio to server */
+#define NOTE_UPLOAD_STARTED	0x010	/* Upload started */
+#define NOTE_STREAMW		0x020	/* Folio is from a streaming write */
+#define NOTE_DISCONTIG_BEFORE	0x040	/* Folio discontiguous with the previous folio */
+#define NOTE_DISCONTIG_AFTER	0x080	/* Folio discontiguous with the next folio */
+#define NOTE_TO_EOF		0x100	/* Data in folio ends at EOF */
+#define NOTE_FLUSH_ANYWAY	0x200	/* Flush data, even if not hit estimated limit */
+
+#define NOTES__KEEP_MASK (NOTE_UPLOAD_AVAIL | NOTE_CACHE_AVAIL | NOTE_UPLOAD_STARTED)
+
+struct netfs_wb_params {
+	unsigned long long	last_end;	/* End file pos of previous folio */
+	unsigned long long	folio_start;	/* File pos of folio */
+	unsigned int		folio_len;	/* Length of folio */
+	unsigned int		dirty_offset;	/* Offset of dirty region in folio */
+	unsigned int		dirty_len;	/* Length of dirty region in folio */
+	unsigned int		notes;		/* Notes on applicability */
+	struct bvecq_pos	dispatch_cursor; /* Folio queue anchor for issue_at */
+	struct netfs_write_estimate estimates[NR_IO_STREAMS];
+};
+
+static int netfs_prepare_write_single_buffer(struct netfs_io_subrequest *subreq,
+					     unsigned int max_segs);
+
 /*
  * Kill all dirty folios in the event of an unrecoverable error, starting with
  * a locked folio we've already obtained from writeback_iter().
@@ -114,65 +141,53 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 
 	wreq->io_streams[0].stream_nr		= 0;
 	wreq->io_streams[0].source		= NETFS_UPLOAD_TO_SERVER;
-	wreq->io_streams[0].prepare_write	= ictx->ops->prepare_write;
+	wreq->io_streams[0].applicable		= NOTE_UPLOAD;
+	wreq->io_streams[0].estimate_write	= ictx->ops->estimate_write;
 	wreq->io_streams[0].issue_write		= ictx->ops->issue_write;
 	wreq->io_streams[0].collected_to	= start;
 	wreq->io_streams[0].transferred		= 0;
 
 	wreq->io_streams[1].stream_nr		= 1;
 	wreq->io_streams[1].source		= NETFS_WRITE_TO_CACHE;
+	wreq->io_streams[1].applicable		= NOTE_CACHE_COPY;
 	wreq->io_streams[1].collected_to	= start;
 	wreq->io_streams[1].transferred		= 0;
 	if (fscache_resources_valid(&wreq->cache_resources)) {
 		wreq->io_streams[1].avail	= true;
 		wreq->io_streams[1].active	= true;
-		wreq->io_streams[1].prepare_write = wreq->cache_resources.ops->prepare_write_subreq;
+		wreq->io_streams[1].estimate_write = wreq->cache_resources.ops->estimate_write;
 		wreq->io_streams[1].issue_write = wreq->cache_resources.ops->issue_write;
 	}
 
 	return wreq;
 }
 
-/**
- * netfs_prepare_write_failed - Note write preparation failed
- * @subreq: The subrequest to mark
- *
- * Mark a subrequest to note that preparation for write failed.
- */
-void netfs_prepare_write_failed(struct netfs_io_subrequest *subreq)
-{
-	__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
-	trace_netfs_sreq(subreq, netfs_sreq_trace_prep_failed);
-}
-EXPORT_SYMBOL(netfs_prepare_write_failed);
-
 /*
- * Prepare a write subrequest.  We need to allocate a new subrequest
- * if we don't have one.
+ * Allocate and prepare a write subrequest.  Will only return NULL if not
+ * performing writeback; if performing writeback, mempools may be accessed and
+ * the allocator may wait forever.
  */
-void netfs_prepare_write(struct netfs_io_request *wreq,
-			 struct netfs_io_stream *stream,
-			 loff_t start)
+struct netfs_io_subrequest *netfs_alloc_write_subreq(struct netfs_io_request *wreq,
+						     struct netfs_io_stream *stream)
 {
 	struct netfs_io_subrequest *subreq;
 
 	subreq = netfs_alloc_subrequest(wreq);
+	if (!subreq)
+		return subreq;
+
 	subreq->source		= stream->source;
-	subreq->start		= start;
+	subreq->start		= stream->issue_from;
+	subreq->len		= stream->buffered;
 	subreq->stream_nr	= stream->stream_nr;
 
-	bvecq_pos_set(&subreq->dispatch_pos, &wreq->dispatch_cursor);
-
 	_enter("R=%x[%x]", wreq->debug_id, subreq->debug_index);
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
 
-	stream->sreq_max_len	= UINT_MAX;
-	stream->sreq_max_segs	= INT_MAX;
 	switch (stream->source) {
 	case NETFS_UPLOAD_TO_SERVER:
 		netfs_stat(&netfs_n_wh_upload);
-		stream->sreq_max_len = wreq->wsize;
 		break;
 	case NETFS_WRITE_TO_CACHE:
 		netfs_stat(&netfs_n_wh_write);
@@ -182,9 +197,6 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 		break;
 	}
 
-	if (stream->prepare_write)
-		stream->prepare_write(subreq);
-
 	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 
 	/* We add to the end of the list whilst the collector may be walking
@@ -194,101 +206,46 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 	spin_lock(&wreq->lock);
 	/* Write IN_PROGRESS before pointer to new subreq */
 	list_add_tail_release(&subreq->rreq_link, &stream->subrequests);
-	if (list_is_first(&subreq->rreq_link, &stream->subrequests)) {
-		if (!stream->active) {
-			stream->collected_to = subreq->start;
-			/* Write list pointers before active flag */
-			smp_store_release(&stream->active, true);
-		}
-	}
+	if (list_is_first(&subreq->rreq_link, &stream->subrequests) &&
+	    stream->collected_to == 0)
+		stream->collected_to = subreq->start;
 
 	spin_unlock(&wreq->lock);
-
-	stream->construct = subreq;
+	return subreq;
 }
 
 /*
- * Set the I/O iterator for the filesystem/cache to use and dispatch the I/O
- * operation.  The operation may be asynchronous and should call
- * netfs_write_subrequest_terminated() when complete.
+ * Prepare the buffer for a buffered write.
  */
-static void netfs_do_issue_write(struct netfs_io_stream *stream,
-				 struct netfs_io_subrequest *subreq)
+static int netfs_prepare_buffered_write_buffer(struct netfs_io_subrequest *subreq,
+					       unsigned int max_segs)
 {
 	struct netfs_io_request *wreq = subreq->rreq;
+	struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];
+	ssize_t len;
 
-	_enter("R=%x[%x],%zx", wreq->debug_id, subreq->debug_index, subreq->len);
-
-	if (stream->source == NETFS_WRITE_TO_CACHE &&
-	    unlikely(test_bit(NETFS_RREQ_CACHE_STOP, &wreq->flags))) {
-		size_t dio_size = wreq->cache_resources.dio_size;
-		size_t len, disp;
-
-		disp = subreq->start & (dio_size - 1);
-		len = round_up(subreq->len + disp, dio_size);
-
-		subreq->start -= disp;
-		subreq->len = len;
+	_enter("%zx,{,%u,%u},%u",
+	       subreq->len, stream->dispatch_cursor.slot, stream->dispatch_cursor.offset, max_segs);
 
-		__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
-		return netfs_write_subrequest_terminated(subreq, subreq->len);
-	}
-
-	if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
-		return netfs_write_subrequest_terminated(subreq, subreq->error);
-
-	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
-	stream->issue_write(subreq);
-}
-
-void netfs_reissue_write(struct netfs_io_stream *stream,
-			 struct netfs_io_subrequest *subreq)
-{
-	// TODO: Use encrypted buffer
-	bvecq_pos_unset(&subreq->content);
-	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
-			    subreq->content.bvecq, subreq->content.slot,
-			    subreq->content.offset,
-			    subreq->len);
-	iov_iter_advance(&subreq->io_iter, subreq->transferred);
-
-	subreq->retry_count++;
-	subreq->error = 0;
-	__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
-	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
-	netfs_stat(&netfs_n_wh_retry_write_subreq);
-	netfs_do_issue_write(stream, subreq);
-}
-
-void netfs_issue_write(struct netfs_io_request *wreq,
-		       struct netfs_io_stream *stream)
-{
-	struct netfs_io_subrequest *subreq = stream->construct;
-
-	if (!subreq)
-		return;
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
 
 	/* If we have a write to the cache, we need to round out the first and
 	 * last entries (only those as the data will be on virtually contiguous
 	 * folios) to cache DIO boundaries.
 	 */
 	if (subreq->source == NETFS_WRITE_TO_CACHE) {
-		struct bvecq_pos tmp_pos;
 		struct bio_vec *bv;
 		struct bvecq *bq;
 		size_t dio_size = wreq->cache_resources.dio_size;
-		size_t disp, len;
-		int ret;
+		size_t disp, dlen;
 
-		bvecq_pos_set(&tmp_pos, &subreq->dispatch_pos);
-		ret = bvecq_extract(&tmp_pos, subreq->len, INT_MAX, &subreq->content.bvecq);
-		bvecq_pos_unset(&tmp_pos);
-		if (ret < 0) {
-			netfs_write_subrequest_terminated(subreq, -ENOMEM);
-			stream->construct = NULL;
-			return;
-		}
+		len = bvecq_extract(&stream->dispatch_cursor, subreq->len, max_segs,
+				    &subreq->content.bvecq);
+		if (len < 0)
+			return -ENOMEM;
+
+		_debug("extract %zx/%zx", len, subreq->len);
+		subreq->len = len;
 
 		/* Round the first entry down.  We should be able to get away
 		 * with this as this path only happens for buffered reads and
@@ -316,94 +273,302 @@ void netfs_issue_write(struct netfs_io_request *wreq,
 		while (bq->next)
 			bq = bq->next;
 		bv = &bq->bv[bq->nr_slots - 1];
-		len = round_up(bv->bv_len, dio_size);
-		if (len > bv->bv_len) {
-			subreq->len += len - bv->bv_len;
-			bv->bv_len = len;
+		dlen = round_up(bv->bv_len, dio_size);
+		if (dlen > bv->bv_len) {
+			subreq->len += dlen - bv->bv_len;
+			bv->bv_len = dlen;
 		}
 	} else {
-		bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+		bvecq_pos_set(&subreq->content, &stream->dispatch_cursor);
+		len = bvecq_slice(&stream->dispatch_cursor, subreq->len, max_segs,
+				  &subreq->nr_segs);
+
+		if (len < subreq->len) {
+			subreq->len = len;
+			trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+		}
+	}
+
+	stream->issue_from += len;
+	stream->buffered   -= len;
+	if (stream->buffered == 0) {
+		stream->buffering = false;
+		bvecq_pos_unset(&stream->dispatch_cursor);
+	}
+	/* Order loading the queue before updating the issue_to point */
+	atomic64_set_release(&stream->issued_to, stream->issue_from);
+	return 0;
+}
+
+/**
+ * netfs_prepare_write_buffer - Get the buffer for a subrequest
+ * @subreq: The subrequest to get the buffer for
+ * @max_segs: Maximum number of segments in buffer (or INT_MAX)
+ *
+ * Extract a slice of buffer from the stream and attach it to the subrequest as
+ * a bio_vec queue.  The maximum amount of data attached is set by
+ * @subreq->len, but this may be shortened if @max_segs would be exceeded.
+ */
+int netfs_prepare_write_buffer(struct netfs_io_subrequest *subreq,
+			       unsigned int max_segs)
+{
+	struct netfs_io_request *rreq = subreq->rreq;
+
+	switch (rreq->origin) {
+	case NETFS_WRITEBACK:
+		if (test_bit(NETFS_RREQ_RETRYING, &rreq->flags))
+			return netfs_prepare_write_retry_buffer(subreq, max_segs);
+		return netfs_prepare_buffered_write_buffer(subreq, max_segs);
+
+	case NETFS_UNBUFFERED_WRITE:
+	case NETFS_DIO_WRITE:
+		return netfs_prepare_unbuffered_write_buffer(subreq, max_segs);
+
+	case NETFS_WRITEBACK_SINGLE:
+		return netfs_prepare_write_single_buffer(subreq, max_segs);
+
+	case NETFS_PGPRIV2_COPY_TO_CACHE:
+		return netfs_prepare_pgpriv2_write_buffer(subreq, max_segs);
+
+	default:
+		WARN_ON_ONCE(1);
+		return -EIO;
 	}
+}
+EXPORT_SYMBOL(netfs_prepare_write_buffer);
+
+/*
+ * Issue writes for a stream.
+ */
+static int netfs_issue_writes(struct netfs_io_request *wreq,
+			      struct netfs_io_stream *stream,
+			      struct netfs_wb_params *params)
+{
+	struct netfs_write_estimate *estimate = &params->estimates[stream->stream_nr];
+
+	for (;;) {
+		struct netfs_io_subrequest *subreq;
+		int ret;
+
+		if (test_bit(NETFS_RREQ_PAUSE, &wreq->flags))
+			netfs_wait_for_paused_write(wreq);
+
+		subreq = netfs_alloc_write_subreq(wreq, stream);
+		/* subreq allocation in a writeback is backed by a mempool and
+		 * will wait for an new one to come available.
+		 */
+
+		if (stream->source == NETFS_WRITE_TO_CACHE &&
+		    unlikely(test_bit(NETFS_RREQ_CACHE_STOP, &wreq->flags))) {
+			size_t dio_size = wreq->cache_resources.dio_size;
+			size_t len, disp;
+
+			disp = subreq->start & (dio_size - 1);
+			len = round_up(subreq->len + disp, dio_size);
+
+			subreq->start -= disp;
+			subreq->len = len;
+
+			stream->issue_from = subreq->start + len;
+			stream->buffered = 0;
+			stream->buffering = false;
+			bvecq_pos_unset(&stream->dispatch_cursor);
+			estimate->issue_at = subreq->start + len + 16 * 1024 * 1024;
+			estimate->max_segs = INT_MAX;
+			__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
+			netfs_write_subrequest_terminated(subreq, len);
+			return 0;
+		}
 
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
-			    subreq->content.bvecq, subreq->content.slot,
-			    subreq->content.offset,
-			    subreq->len);
+		ret = stream->issue_write(subreq);
+		if (ret < 0) {
+			/* Ownership of subreq was returned to us. */
+			trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+			bvecq_pos_advance(&stream->dispatch_cursor, subreq->len);
+			stream->issue_from += subreq->len;
+			stream->buffered   -= subreq->len;
+			atomic64_set_release(&stream->issued_to, stream->issue_from);
+			netfs_write_subrequest_terminated(subreq, ret);
+		}
+		/* We no longer own subreq. */
 
-	stream->construct = NULL;
-	netfs_do_issue_write(stream, subreq);
+		if (test_bit(NETFS_RREQ_SAW_ENOMEM, &wreq->flags))
+			return -ENOMEM;
+		if (stream->buffered == 0) {
+			if (stream->stream_nr == 0)
+				params->notes &= ~NOTE_UPLOAD_STARTED;
+			return 0;
+		}
+
+		if (!(params->notes & NOTE_FLUSH_ANYWAY)) {
+			estimate->issue_at = ULLONG_MAX;
+			estimate->max_segs = INT_MAX;
+			stream->estimate_write(wreq, stream, estimate);
+			if (stream->issue_from + stream->buffered < estimate->issue_at &&
+			    estimate->max_segs > 0)
+				return 0;
+		}
+	}
 }
 
 /*
- * Add data to the write subrequest, dispatching each as we fill it up or if it
- * is discontiguous with the previous.  We only fill one part at a time so that
- * we can avoid overrunning the credits obtained (cifs) and try to parallelise
- * content-crypto preparation with network writes.
+ * Issue pending writes on a stream.
  */
-size_t netfs_advance_write(struct netfs_io_request *wreq,
-			   struct netfs_io_stream *stream,
-			   loff_t start, size_t len, bool to_eof)
+static int netfs_issue_stream(struct netfs_io_request *wreq,
+			      struct netfs_wb_params *params, int s)
 {
-	struct netfs_io_subrequest *subreq = stream->construct;
-	size_t part;
+	struct netfs_write_estimate *estimate = &params->estimates[s];
+	struct netfs_io_stream *stream = &wreq->io_streams[s];
+	unsigned long long dirty_start;
+	bool discontig_before = params->notes & NOTE_DISCONTIG_BEFORE;
+	int ret;
 
-	if (!stream->avail) {
-		_leave("no write");
-		return len;
+	_enter("%x", params->notes);
+
+	/* If the current folio doesn't contribute to this stream, see if we
+	 * need to flush it.
+	 */
+	if (!(params->notes & stream->applicable)) {
+		if (!stream->buffering) {
+			atomic64_set_release(&stream->issued_to,
+					     params->folio_start + params->folio_len);
+			return 0;
+		}
+		discontig_before = true;
 	}
 
-	_enter("R=%x[%x]", wreq->debug_id, subreq ? subreq->debug_index : 0);
+	/* Issue writes if we meet a discontiguity before the current folio.
+	 * Even if the filesystem can do sparse/vectored writes, we still
+	 * generate a subreq per contiguous region rather than generating
+	 * separate extent lists.
+	 */
+	if (stream->buffering && discontig_before) {
+		params->notes |= NOTE_FLUSH_ANYWAY;
+		ret = netfs_issue_writes(wreq, stream, params);
+		if (ret < 0)
+			return ret;
+		stream->buffering = false;
+		params->notes &= ~NOTE_FLUSH_ANYWAY;
+	}
 
-	if (subreq && start != subreq->start + subreq->len) {
-		netfs_issue_write(wreq, stream);
-		subreq = NULL;
+	if (!(params->notes & stream->applicable)) {
+		atomic64_set_release(&stream->issued_to,
+				     params->folio_start + params->folio_len);
+		return 0;
 	}
 
-	if (!stream->construct)
-		netfs_prepare_write(wreq, stream, start);
-	subreq = stream->construct;
+	/* If we're not currently buffering on this stream, we need to get an
+	 * estimate of when we need to issue a write.  It might be within the
+	 * starting folio.
+	 */
+	dirty_start = params->folio_start + params->dirty_offset;
+	if (!stream->buffering) {
+		stream->buffering = true;
+		stream->issue_from = dirty_start;
+		bvecq_pos_set(&stream->dispatch_cursor, &params->dispatch_cursor);
+		estimate->issue_at = ULLONG_MAX;
+		estimate->max_segs = INT_MAX;
+		stream->estimate_write(wreq, stream, estimate);
+	}
 
-	part = umin(stream->sreq_max_len - subreq->len, len);
-	_debug("part %zx/%zx %zx/%zx", subreq->len, stream->sreq_max_len, part, len);
-	subreq->len += part;
-	subreq->nr_segs++;
+	stream->buffered += params->dirty_len;
+	estimate->max_segs--;
 
-	if (subreq->len >= stream->sreq_max_len ||
-	    subreq->nr_segs >= stream->sreq_max_segs ||
-	    to_eof) {
-		netfs_issue_write(wreq, stream);
-		subreq = NULL;
+	/* Poke the filesystem to issue writes when we hit the limit it set or
+	 * if the data ends before the end of the page.
+	 */
+	if (params->notes & NOTE_DISCONTIG_AFTER)
+		params->notes |= NOTE_FLUSH_ANYWAY;
+	_debug("[%u] %llx + %zx >= %llx, %u %x",
+	       s, stream->issue_from, stream->buffered, estimate->issue_at,
+	       estimate->max_segs, params->notes);
+	if (stream->issue_from + stream->buffered >= estimate->issue_at ||
+	    estimate->max_segs <= 0 ||
+	    (params->notes & NOTE_FLUSH_ANYWAY)) {
+		ret = netfs_issue_writes(wreq, stream, params);
+		if (ret < 0)
+			return ret;
 	}
 
-	return part;
+	return 0;
 }
 
 /*
- * Write some of a pending folio data back to the server.
+ * See which streams need writes issuing and issue them.
  */
-static int netfs_write_folio(struct netfs_io_request *wreq,
-			     struct writeback_control *wbc,
-			     struct folio *folio)
+static int netfs_issue_streams(struct netfs_io_request *wreq,
+			       struct netfs_wb_params *params)
+{
+	int ret = 0, ret2;
+
+	_enter("%x", params->notes);
+
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		ret2 = netfs_issue_stream(wreq, params, s);
+		if (ret2 < 0)
+			ret = ret2;
+	}
+	return ret;
+}
+
+/*
+ * End the issuing of writes, let the collector know we're done.
+ */
+static void netfs_end_issue_write(struct netfs_io_request *wreq,
+				  struct netfs_wb_params *params)
+{
+	bool needs_poke = true;
+
+	params->notes |= NOTE_FLUSH_ANYWAY;
+
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
+		int ret;
+
+		if (stream->buffering) {
+			ret = netfs_issue_writes(wreq, stream, params);
+			if (ret < 0 && stream->source != NETFS_WRITE_TO_CACHE) {
+				/* Leave the error somewhere the completion
+				 * path can pick it up if there isn't already
+				 * another error logged.
+				 */
+				cmpxchg(&wreq->error, 0, ret);
+			}
+			stream->buffering = false;
+		}
+	}
+
+	netfs_all_subreqs_queued(wreq);
+
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
+
+		if (!stream->active)
+			continue;
+		if (!list_empty(&stream->subrequests))
+			needs_poke = false;
+	}
+
+	if (needs_poke)
+		netfs_wake_collector(wreq);
+}
+
+/*
+ * Queue a folio for writeback.
+ */
+static void netfs_queue_wb_folio(struct netfs_io_request *wreq,
+				 struct writeback_control *wbc,
+				 struct folio *folio,
+				 struct netfs_wb_params *params)
 {
-	struct netfs_io_stream *upload = &wreq->io_streams[0];
-	struct netfs_io_stream *cache  = &wreq->io_streams[1];
-	struct netfs_io_stream *stream;
 	struct netfs_group *fgroup; /* TODO: Use this with ceph */
 	struct netfs_folio *finfo;
 	struct bvecq *queue = wreq->load_cursor.bvecq;
 	unsigned int slot;
 	size_t fsize = folio_size(folio), flen = fsize, foff = 0;
 	loff_t fpos = folio_pos(folio), i_size;
-	bool to_eof = false, streamw = false;
-	bool debug = false;
 
-	_enter("");
-
-	if (!wreq->spare) {
-		wreq->spare = bvecq_alloc_one(BVECQ_STD_SLOTS, wreq->gfp);
-		if (!wreq->spare)
-			return -ENOMEM;
-	}
+	_enter("%x", params->notes);
 
 	/* netfs_perform_write() may shift i_size around the folio or from out
 	 * of the folio to beyond it, but cannot move i_size into or through
@@ -424,7 +589,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 		wreq->nr_group_rel += netfs_folio_written_back(folio);
 		netfs_put_group_many(wreq->group, wreq->nr_group_rel);
 		wreq->nr_group_rel = 0;
-		return 0;
+		return;
 	}
 
 	if (fpos + fsize > wreq->i_size)
@@ -435,19 +600,32 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	if (finfo) {
 		foff = finfo->dirty_offset;
 		flen = foff + finfo->dirty_len;
-		streamw = true;
+		params->notes |= NOTE_STREAMW;
+		if (foff > 0)
+			params->notes |= NOTE_DISCONTIG_BEFORE;
+		if (flen < fsize)
+			params->notes |= NOTE_DISCONTIG_AFTER;
 	}
 
+	if (params->last_end && fpos != params->last_end)
+		params->notes |= NOTE_DISCONTIG_BEFORE;
+	params->last_end = fpos + fsize;
+
 	if (flen > i_size - fpos) {
 		flen = i_size - fpos;
-		if (!streamw)
+		if (!(params->notes & NOTE_STREAMW))
 			folio_zero_segment(folio, flen, fsize);
-		to_eof = true;
+		params->notes |= NOTE_TO_EOF;
 	} else if (flen == i_size - fpos) {
-		to_eof = true;
+		params->notes |= NOTE_TO_EOF;
 	}
 	flen -= foff;
 
+	params->folio_start	= fpos;
+	params->folio_len	= fsize;
+	params->dirty_offset	= foff;
+	params->dirty_len	= flen;
+
 	_debug("folio %zx %zx %zx", foff, flen, fsize);
 
 	/* Deal with discontinuities in the stream of dirty pages.  These can
@@ -467,58 +645,50 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	 *     write-back group.
 	 */
 	if (fgroup == NETFS_FOLIO_COPY_TO_CACHE) {
-		netfs_issue_write(wreq, upload);
+		if (!(params->notes & NOTE_CACHE_AVAIL)) {
+			trace_netfs_folio(folio, netfs_folio_trace_cancel_copy);
+			goto cancel_folio;
+		}
+		params->notes |= NOTE_CACHE_COPY;
+		trace_netfs_folio(folio, netfs_folio_trace_store_copy);
 	} else if (fgroup != wreq->group) {
 		/* We can't write this page to the server yet. */
 		kdebug("wrong group");
-		folio_redirty_for_writepage(wbc, folio);
-		folio_unlock(folio);
-		netfs_issue_write(wreq, upload);
-		netfs_issue_write(wreq, cache);
-		return 0;
+		goto skip_folio;
+	} else if (!(params->notes & (NOTE_UPLOAD_AVAIL | NOTE_CACHE_AVAIL))) {
+		trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
+		goto cancel_folio_discard;
+	} else {
+		if (params->notes & NOTE_UPLOAD_STARTED) {
+			params->notes |= NOTE_UPLOAD;
+			trace_netfs_folio(folio, netfs_folio_trace_store_plus);
+		} else {
+			params->notes |= NOTE_UPLOAD | NOTE_UPLOAD_STARTED;
+			trace_netfs_folio(folio, netfs_folio_trace_store);
+		}
+		if ((params->notes & NOTE_CACHE_AVAIL) &&
+		    !(params->notes & NOTE_STREAMW))
+			params->notes |= NOTE_CACHE_COPY;
 	}
 
-	if (foff > 0)
-		netfs_issue_write(wreq, upload);
-	if (streamw)
-		netfs_issue_write(wreq, cache);
-
 	folio_start_writeback(folio);
 	folio_unlock(folio);
 
-	if (fgroup == NETFS_FOLIO_COPY_TO_CACHE) {
-		if (!cache->avail) {
-			trace_netfs_folio(folio, netfs_folio_trace_cancel_copy);
-			netfs_issue_write(wreq, upload);
-			netfs_folio_written_back(folio);
-			return 0;
-		}
-		trace_netfs_folio(folio, netfs_folio_trace_store_copy);
-	} else if (!upload->avail && !cache->avail) {
-		trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
-		netfs_folio_written_back(folio);
-		return 0;
-	} else if (!upload->construct) {
-		trace_netfs_folio(folio, netfs_folio_trace_store);
-	} else {
-		trace_netfs_folio(folio, netfs_folio_trace_store_plus);
-	}
-
 	/* Institute a new bvec queue segment if the current one is full or if
 	 * we encounter a discontiguity.  The discontiguity break is important
 	 * when it comes to bulk unlocking folios by file range.
 	 */
 	if (bvecq_is_full(queue) ||
-	    (fpos != wreq->last_end && wreq->last_end > 0)) {
+	    ((params->notes & NOTE_DISCONTIG_BEFORE) && queue->nr_slots > 0)) {
 		bvecq_buffer_append(&wreq->load_cursor, wreq->spare);
 		wreq->spare = NULL;
 
 		queue = wreq->load_cursor.bvecq;
 		queue->fpos = fpos;
-		if (fpos != wreq->last_end)
+		if (params->notes & NOTE_DISCONTIG_BEFORE)
 			queue->discontig = true;
-		bvecq_pos_move(&wreq->dispatch_cursor, queue);
-		wreq->dispatch_cursor.slot = 0;
+		bvecq_pos_move(&params->dispatch_cursor, queue);
+		params->dispatch_cursor.slot = 0;
 	}
 
 	/* Attach the folio to the rolling buffer. */
@@ -529,106 +699,26 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	bvecq_filled_to(queue, slot);
 	wreq->load_cursor.slot = slot;
 	wreq->load_cursor.offset = 0;
-	wreq->last_end = fpos + foff + flen;
+	trace_netfs_wback(wreq, folio, params->notes);
 
-	/* Move the submission point forward to allow for write-streaming data
-	 * not starting at the front of the page.  We don't do write-streaming
-	 * with the cache as the cache requires DIO alignment.
-	 *
-	 * Also skip uploading for data that's been read and just needs copying
-	 * to the cache.
-	 */
-	bvecq_pos_nudge(&wreq->dispatch_cursor);
-	
-	for (int s = 0; s < NR_IO_STREAMS; s++) {
-		stream = &wreq->io_streams[s];
-		stream->submit_off = 0;
-		stream->submit_len = flen;
-		if (!stream->avail ||
-		    (stream->source == NETFS_WRITE_TO_CACHE && streamw) ||
-		    (stream->source == NETFS_UPLOAD_TO_SERVER &&
-		     fgroup == NETFS_FOLIO_COPY_TO_CACHE)) {
-			stream->submit_off = UINT_MAX;
-			stream->submit_len = 0;
-		}
-	}
-
-	/* Attach the folio to one or more subrequests.  For a big folio, we
-	 * could end up with thousands of subrequests if the wsize is small -
-	 * but we might need to wait during the creation of subrequests for
-	 * network resources (eg. SMB credits).
-	 */
-	for (;;) {
-		ssize_t part;
-		size_t lowest_off = ULONG_MAX;
-		int choose_s = -1;
-
-		/* Always add to the lowest-submitted stream first. */
-		for (int s = 0; s < NR_IO_STREAMS; s++) {
-			stream = &wreq->io_streams[s];
-			if (stream->submit_len > 0 &&
-			    stream->submit_off < lowest_off) {
-				lowest_off = stream->submit_off;
-				choose_s = s;
-			}
-		}
-
-		if (choose_s < 0)
-			break;
-		stream = &wreq->io_streams[choose_s];
-
-		/* Advance the cursor. */
-		wreq->dispatch_cursor.offset = stream->submit_off;
-
-		atomic64_set(&wreq->issued_to, fpos + foff + stream->submit_off);
-		part = netfs_advance_write(wreq, stream, fpos + foff + stream->submit_off,
-					   stream->submit_len, to_eof);
-		stream->submit_off += part;
-		if (part > stream->submit_len)
-			stream->submit_len = 0;
-		else
-			stream->submit_len -= part;
-		if (part > 0)
-			debug = true;
-	}
-
-	bvecq_pos_step(&wreq->dispatch_cursor);
-	/* Order loading the queue before updating the issue_to point */
-	atomic64_set_release(&wreq->issued_to, fpos + fsize);
-
-	if (!debug)
-		kdebug("R=%x: No submit", wreq->debug_id);
-
-	if (foff + flen < fsize)
-		for (int s = 0; s < NR_IO_STREAMS; s++)
-			netfs_issue_write(wreq, &wreq->io_streams[s]);
-
-	_leave(" = 0");
-	return 0;
-}
-
-/*
- * End the issuing of writes, letting the collector know we're done.
- */
-static void netfs_end_issue_write(struct netfs_io_request *wreq)
-{
-	bool needs_poke = true;
-
-	smp_wmb(); /* Write subreq lists before ALL_QUEUED. */
-	set_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags);
-
-	for (int s = 0; s < NR_IO_STREAMS; s++) {
-		struct netfs_io_stream *stream = &wreq->io_streams[s];
-
-		if (!stream->active)
-			continue;
-		if (!list_empty(&stream->subrequests))
-			needs_poke = false;
-		netfs_issue_write(wreq, stream);
-	}
+out:
+	_leave(" = %x", params->notes);
+	return;
 
-	if (needs_poke)
-		netfs_wake_collector(wreq);
+skip_folio:
+	folio_redirty_for_writepage(wbc, folio);
+	folio_unlock(folio);
+	params->notes |= NOTE_DISCONTIG_BEFORE;
+	goto out;
+cancel_folio_discard:
+	netfs_put_group(fgroup);
+cancel_folio:
+	folio_detach_private(folio);
+	kfree(finfo);
+	folio_unlock(folio);
+	folio_cancel_dirty(folio);
+	params->notes |= NOTE_DISCONTIG_BEFORE;
+	goto out;
 }
 
 /*
@@ -639,6 +729,7 @@ int netfs_writepages(struct address_space *mapping,
 {
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
 	struct netfs_io_request *wreq = NULL;
+	struct netfs_wb_params params = {};
 	struct folio *folio;
 	int error = 0;
 
@@ -658,35 +749,54 @@ int netfs_writepages(struct address_space *mapping,
 
 	if (bvecq_buffer_init(&wreq->load_cursor, wreq->gfp) < 0)
 		goto nomem;
-	bvecq_pos_set(&wreq->dispatch_cursor, &wreq->load_cursor);
-	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+	bvecq_pos_set(&params.dispatch_cursor, &wreq->load_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->load_cursor);
 
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback);
 	netfs_stat(&netfs_n_wh_writepages);
 
-	do {
-		_debug("wbiter %lx %llx", folio->index, atomic64_read(&wreq->issued_to));
+	if (wreq->io_streams[1].avail)
+		params.notes |= NOTE_CACHE_AVAIL;
 
-		/* It appears we don't have to handle cyclic writeback wrapping. */
-		WARN_ON_ONCE(wreq && folio_pos(folio) < atomic64_read(&wreq->issued_to));
+	do {
+		_debug("wbiter %lx", folio->index);
+
+		if (!wreq->spare) {
+			wreq->spare = bvecq_alloc_one(BVECQ_STD_SLOTS, wreq->gfp);
+			if (!wreq->spare) {
+				folio_redirty_for_writepage(wbc, folio);
+				folio_unlock(folio);
+				error = -ENOMEM;
+				goto end_loop;
+			}
+		}
 
 		if (netfs_folio_group(folio) != NETFS_FOLIO_COPY_TO_CACHE &&
 		    unlikely(!test_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))) {
 			set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags);
 			wreq->netfs_ops->begin_writeback(wreq);
+			if (wreq->io_streams[0].avail) {
+				params.notes |= NOTE_UPLOAD_AVAIL;
+				/* Order setting the active flag after other fields. */
+				smp_store_release(&wreq->io_streams[0].active, true);
+			}
 		}
 
-		error = netfs_write_folio(wreq, wbc, folio);
-		if (error == -ENOMEM) {
-			folio_redirty_for_writepage(wbc, folio);
-			folio_unlock(folio);
-		}
+		params.notes &= NOTES__KEEP_MASK;
+		netfs_queue_wb_folio(wreq, wbc, folio, &params);
+		error = netfs_issue_streams(wreq, &params);
+		if (!error)
+			bvecq_pos_step(&params.dispatch_cursor);
+end_loop:
 	} while ((folio = writeback_iter(mapping, wbc, folio, &error)));
 
-	netfs_end_issue_write(wreq);
+	netfs_end_issue_write(wreq, &params);
+
 	bvecq_pos_unset(&wreq->load_cursor);
-	bvecq_pos_unset(&wreq->dispatch_cursor);
+	bvecq_pos_unset(&params.dispatch_cursor);
+	for (int i = 0; i < NR_IO_STREAMS; i++)
+		bvecq_pos_unset(&wreq->io_streams[i].dispatch_cursor);
 	netfs_wake_collector(wreq);
 
 	netfs_put_request(wreq, netfs_rreq_trace_put_return);
@@ -712,11 +822,46 @@ int netfs_writepages(struct address_space *mapping,
 }
 EXPORT_SYMBOL(netfs_writepages);
 
+/*
+ * Prepare a buffer for a single monolithic write.
+ */
+static int netfs_prepare_write_single_buffer(struct netfs_io_subrequest *subreq,
+					     unsigned int max_segs)
+{
+	struct netfs_io_request *wreq = subreq->rreq;
+	struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];
+	struct bio_vec *bv;
+	struct bvecq *bq;
+	size_t dio_size = wreq->cache_resources.dio_size;
+	size_t dlen;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+
+	/* Round the end of the last entry up. */
+	bq = subreq->content.bvecq;
+	while (bq->next)
+		bq = bq->next;
+	bv = &bq->bv[bq->nr_slots - 1];
+	dlen = round_up(bv->bv_len, dio_size);
+	if (dlen > bv->bv_len) {
+		subreq->len += dlen - bv->bv_len;
+		bv->bv_len = dlen;
+	}
+
+	stream->buffered   = 0;
+	stream->issue_from = subreq->len;
+	wreq->submitted    = subreq->len;
+	netfs_all_subreqs_queued(wreq);
+	return 0;
+}
+
 /**
  * netfs_writeback_single - Write back a monolithic payload
  * @mapping: The mapping to write from
  * @wbc: Hints from the VM
- * @iter: Data to write.
+ * @iter: Data to write
+ * @len: Amount of data to write
  *
  * Write a monolithic, non-pagecache object back to the server and/or
  * the cache.  There's a maximum of one subrequest per stream.
@@ -727,12 +872,18 @@ EXPORT_SYMBOL(netfs_writepages);
  */
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
-			   struct iov_iter *iter)
+			   struct iov_iter *iter,
+			   size_t len)
 {
 	struct netfs_io_request *wreq;
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
 	int ret;
 
+	_enter("%zx,%zx", iov_iter_count(iter), len);
+
+	if (!len)
+		return 0;
+
 	if (!netfs_wb_begin(ictx, wbc->sync_mode == WB_SYNC_NONE)) {
 		/* The VFS will have undirtied the inode. */
 		netfs_single_mark_inode_dirty(&ictx->inode);
@@ -744,24 +895,25 @@ int netfs_writeback_single(struct address_space *mapping,
 		ret = PTR_ERR(wreq);
 		goto couldnt_start;
 	}
-	wreq->len = iov_iter_count(iter);
 
-	ret = netfs_extract_iter(iter, wreq->len, INT_MAX, 0, &wreq->dispatch_cursor.bvecq,
+	wreq->len = len;
+
+	ret = netfs_extract_iter(iter, len, INT_MAX, 0, &wreq->load_cursor.bvecq,
 				 0, wreq->gfp);
 	if (ret < 0)
 		goto cleanup_free;
-	if (ret < wreq->len) {
+	if (ret < len) {
 		ret = -EIO;
 		goto cleanup_free;
 	}
 
-	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->load_cursor);
 
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback_single);
 	netfs_stat(&netfs_n_wh_writepages);
 
-	if (__test_and_set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
+	if (test_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
 		wreq->netfs_ops->begin_writeback(wreq);
 
 	for (int s = 0; s < NR_IO_STREAMS; s++) {
@@ -771,20 +923,34 @@ int netfs_writeback_single(struct address_space *mapping,
 		if (!stream->avail)
 			continue;
 
-		netfs_prepare_write(wreq, stream, 0);
+		stream->issue_from = 0;
+		stream->buffered   = len;
+
+		subreq = netfs_alloc_write_subreq(wreq, stream);
+		if (!subreq) {
+			ret = -ENOMEM;
+			break;
+		}
+
+		bvecq_pos_set(&stream->dispatch_cursor, &wreq->load_cursor);
 
-		subreq = stream->construct;
-		subreq->len = wreq->len;
-		stream->submit_len = subreq->len;
+		ret = stream->issue_write(subreq);
+		if (ret < 0) {
+			/* Ownership of subreq was returned to us. */
+			trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+			stream->buffered -= subreq->len;
+			netfs_write_subrequest_terminated(subreq, ret);
+			/* Punt the error to the collector. */
+		}
 
-		netfs_issue_write(wreq, stream);
+		bvecq_pos_unset(&stream->dispatch_cursor);
 	}
 
 	wreq->submitted = wreq->len;
-	smp_wmb(); /* Write lists before ALL_QUEUED. */
-	set_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags);
-
-	netfs_wake_collector(wreq);
+	if (unlikely(!netfs_are_all_subreqs_queued(wreq))) {
+		netfs_all_subreqs_queued(wreq);
+		netfs_wake_collector(wreq);
+	}
 
 	/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to
 	 * wait before modifying.
diff --git a/fs/netfs/write_retry.c b/fs/netfs/write_retry.c
index 4e320591f83a..ff783ea2dc61 100644
--- a/fs/netfs/write_retry.c
+++ b/fs/netfs/write_retry.c
@@ -12,13 +12,44 @@
 #include "internal.h"
 
 /*
- * Perform retries on the streams that need it.
+ * Prepare the write buffer for a retry.  We can't necessarily reuse the write
+ * buffer from the previous run of a subrequest because the filesystem is
+ * permitted to modify it (add headers/trailers, encrypt it).  Further, the
+ * subrequest may now be a different size (e.g. cifs has to negotiate for
+ * maximum transfer size).  Also, we can't look at *stream as that may still
+ * refer to the source material being broken up into original subrequests.
+ */
+int netfs_prepare_write_retry_buffer(struct netfs_io_subrequest *subreq,
+				     unsigned int max_segs)
+{
+	struct netfs_io_request *wreq = subreq->rreq;
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &wreq->retry_cursor);
+	bvecq_pos_set(&subreq->content, &wreq->retry_cursor);
+	len = bvecq_slice(&wreq->retry_cursor, subreq->len, max_segs, &subreq->nr_segs);
+
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+
+	wreq->retry_start += len;
+	wreq->retry_buffered   -= len;
+	if (wreq->retry_buffered == 0)
+		bvecq_pos_unset(&wreq->retry_cursor);
+	return 0;
+}
+
+/*
+ * Perform retries on the streams that need it.  This only has to deal with
+ * buffered writes; unbuffered write retry is handled in direct_write.c.
  */
 static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				     struct netfs_io_stream *stream)
 {
-	struct bvecq_pos dispatch_cursor = {};
 	struct list_head *next;
+	int ret;
 
 	_enter("R=%x[%x:]", wreq->debug_id, stream->stream_nr);
 
@@ -33,30 +64,15 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 	if (unlikely(stream->failed))
 		return;
 
-	/* If there's no renegotiation to do, just resend each failed subreq. */
-	if (!stream->prepare_write) {
-		struct netfs_io_subrequest *subreq;
-
-		list_for_each_entry(subreq, &stream->subrequests, rreq_link) {
-			if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
-				break;
-			if (__test_and_clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-				netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-				netfs_reissue_write(stream, subreq);
-			}
-		}
-		return;
-	}
-
-	next = stream->subrequests.next;
+	/* Read pointer to subreq before reading subreq state. */
+	next = smp_load_acquire(&stream->subrequests.next);
 
 	do {
 		struct netfs_io_subrequest *subreq = NULL, *from, *to, *tmp;
 		unsigned long long start, len;
-		size_t part;
-		bool boundary = false;
+		bool subreq_superfluous = false;
 
-		bvecq_pos_unset(&dispatch_cursor);
+		bvecq_pos_unset(&wreq->retry_cursor);
 
 		/* Go through the stream and find the next span of contiguous
 		 * data that we then rejig (cifs, for example, needs the wsize
@@ -80,7 +96,6 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			subreq = list_entry(next, struct netfs_io_subrequest, rreq_link);
 			if (subreq->start != start + len ||
 			    subreq->transferred > 0 ||
-			    test_bit(NETFS_SREQ_BOUNDARY, &subreq->flags) ||
 			    !test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
 				break;
 			to = subreq;
@@ -90,8 +105,10 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 		/* Determine the set of buffers we're going to use.  Each
 		 * subreq gets a subset of a single overall contiguous buffer.
 		 */
-		bvecq_pos_transfer(&dispatch_cursor, &from->dispatch_pos);
-		bvecq_pos_advance(&dispatch_cursor, from->transferred);
+		bvecq_pos_transfer(&wreq->retry_cursor, &from->dispatch_pos);
+		bvecq_pos_advance(&wreq->retry_cursor, from->transferred);
+		wreq->retry_start = start;
+		wreq->retry_buffered = len;
 
 		/* Work through the sublist.  The chain of buffers we're going
 		 * to fill is attached to dispatch_cursor and we need to read
@@ -99,47 +116,50 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 		 */
 		subreq = from;
 		list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
-			if (!len)
+			if (!wreq->retry_buffered) {
+				subreq_superfluous = true;
 				break;
-
-			subreq->start	= start;
-			subreq->len	= len;
-			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
-			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
-			subreq->transferred = 0;
+			}
 
 			bvecq_pos_unset(&subreq->content);
 			bvecq_pos_unset(&subreq->dispatch_pos);
 
-			/* Renegotiate max_len (wsize) */
-			stream->sreq_max_len = len;
-			stream->sreq_max_segs = INT_MAX;
-			stream->prepare_write(subreq);
-
-			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
-			part = bvecq_slice(&dispatch_cursor,
-					   umin(len, stream->sreq_max_len),
-					   stream->sreq_max_segs,
-					   &subreq->nr_segs);
-			subreq->len = part;
-
-			len -= part;
-			start += part;
-			if (len && subreq == to &&
-			    __test_and_clear_bit(NETFS_SREQ_BOUNDARY, &to->flags))
-				boundary = true;
-
+			__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+			__clear_bit(NETFS_SREQ_FAILED, &subreq->flags);
+			__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
+			subreq->start		= wreq->retry_start;
+			subreq->len		= wreq->retry_buffered;
+			subreq->transferred	= 0;
+			subreq->retry_count	+= 1;
+			subreq->error		= 0;
+
+			netfs_stat(&netfs_n_wh_retry_write_subreq);
+			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 			netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-			netfs_reissue_write(stream, subreq);
-			if (subreq == to)
+			ret = stream->issue_write(subreq);
+			if (ret < 0) {
+				/* Ownership of subreq was returned to us.
+				 * Expand the subreq to consume the entire
+				 * remaining amount to be retried and fail it.
+				 */
+				trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+				wreq->retry_buffered -= subreq->len;
+				netfs_write_subrequest_terminated(subreq, ret);
+				break;
+			}
+
+			if (subreq == to) {
+				subreq_superfluous = false;
 				break;
+			}
 		}
 
 		/* If we managed to use fewer subreqs, we can discard the
 		 * excess; if we used the same number, then we're done.
 		 */
-		if (!len) {
-			if (subreq == to)
+		if (!wreq->retry_buffered) {
+			if (!subreq_superfluous)
 				continue;
 			list_for_each_entry_safe_from(subreq, tmp,
 						      &stream->subrequests, rreq_link) {
@@ -161,6 +181,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			subreq = netfs_alloc_subrequest(wreq);
 			subreq->source		= to->source;
 			subreq->start		= start;
+			subreq->len		= wreq->retry_buffered;
 			subreq->stream_nr	= to->stream_nr;
 			subreq->retry_count	= 1;
 
@@ -169,49 +190,40 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 					     netfs_sreq_trace_new);
 			trace_netfs_sreq(subreq, netfs_sreq_trace_split);
 
+			__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 			spin_lock(&wreq->lock);
+			/* Write IN_PROGRESS before pointer to new subreq */
+			smp_wmb();
 			list_add(&subreq->rreq_link, &to->rreq_link);
 			spin_unlock(&wreq->lock);
 			to = subreq;
-			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
-			stream->sreq_max_len	= len;
-			stream->sreq_max_segs	= INT_MAX;
 			switch (stream->source) {
 			case NETFS_UPLOAD_TO_SERVER:
 				netfs_stat(&netfs_n_wh_upload);
-				stream->sreq_max_len = umin(len, wreq->wsize);
 				break;
 			default:
 				WARN_ON_ONCE(1);
 			}
 
-			stream->prepare_write(subreq);
-
-			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
-			part = bvecq_slice(&dispatch_cursor,
-					   umin(len, stream->sreq_max_len),
-					   stream->sreq_max_segs,
-					   &subreq->nr_segs);
-			subreq->len = subreq->transferred + part;
-
-			len -= part;
-			start += part;
-			if (!len && boundary) {
-				__set_bit(NETFS_SREQ_BOUNDARY, &to->flags);
-				boundary = false;
-			}
-
-			netfs_reissue_write(stream, subreq);
-			if (!len)
+			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+			ret = stream->issue_write(subreq);
+			if (ret < 0) {
+				/* Ownership of subreq was returned to us.
+				 * Expand the subreq to consume the entire
+				 * remaining amount to be retried and fail it.
+				 */
+				trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+				wreq->retry_buffered -= subreq->len;
+				netfs_write_subrequest_terminated(subreq, ret);
 				break;
-
-		} while (len);
+			}
+		} while (wreq->retry_buffered > 0);
 
 	} while (!list_is_head(next, &stream->subrequests));
 
 out:
-	bvecq_pos_unset(&dispatch_cursor);
+	bvecq_pos_unset(&wreq->retry_cursor);
 }
 
 /*
diff --git a/fs/nfs/Kconfig b/fs/nfs/Kconfig
index 6bb30543eff0..e7862f35b72c 100644
--- a/fs/nfs/Kconfig
+++ b/fs/nfs/Kconfig
@@ -174,6 +174,7 @@ config NFS_FSCACHE
 	bool "Provide NFS client caching support"
 	depends on NFS_FS
 	select NETFS_SUPPORT
+	select NETFS_PGPRIV2
 	select FSCACHE
 	help
 	  Say Y here if you want NFS data to be cached locally on disc through
diff --git a/fs/nfs/fscache.c b/fs/nfs/fscache.c
index 9b7fdad4a920..fb1441f88661 100644
--- a/fs/nfs/fscache.c
+++ b/fs/nfs/fscache.c
@@ -23,6 +23,7 @@
 #include "iostat.h"
 #include "fscache.h"
 #include "nfstrace.h"
+#include <trace/events/netfs.h>
 
 #define NFS_MAX_KEY_LEN 1000
 
@@ -273,8 +274,6 @@ static int nfs_netfs_init_request(struct netfs_io_request *rreq, struct file *fi
 	rreq->debug_id = atomic_inc_return(&nfs_netfs_debug_id);
 	/* [DEPRECATED] Use PG_private_2 to mark folio being written to the cache. */
 	__set_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags);
-	rreq->io_streams[0].sreq_max_len = NFS_SB(rreq->inode->i_sb)->rsize;
-
 	return 0;
 }
 
@@ -296,8 +295,9 @@ static struct nfs_netfs_io_data *nfs_netfs_alloc(struct netfs_io_subrequest *sre
 	return netfs;
 }
 
-static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
+static int nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
 {
+	struct netfs_io_request		*rreq = sreq->rreq;
 	struct nfs_netfs_io_data	*netfs;
 	struct nfs_pageio_descriptor	pgio;
 	struct inode *inode = sreq->rreq->inode;
@@ -307,6 +307,16 @@ static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
 	pgoff_t start, last;
 	int err;
 
+	if (sreq->len > NFS_SB(rreq->inode->i_sb)->rsize)
+		sreq->len = NFS_SB(rreq->inode->i_sb)->rsize;
+
+	err = netfs_prepare_read_buffer(sreq, INT_MAX);
+	if (err < 0) {
+		sreq->error = err;
+		return err;
+	}
+	/* After this point, must fail by termination. */
+
 	start = (sreq->start + sreq->transferred) >> PAGE_SHIFT;
 	last = ((sreq->start + sreq->len - sreq->transferred - 1) >> PAGE_SHIFT);
 
@@ -316,12 +326,15 @@ static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
 	netfs = nfs_netfs_alloc(sreq);
 	if (!netfs) {
 		sreq->error = -ENOMEM;
-		return netfs_read_subreq_terminated(sreq);
+		netfs_read_subreq_terminated(sreq);
+		return 0;
 	}
 
+	trace_netfs_sreq(sreq, netfs_sreq_trace_submit);
+
 	pgio.pg_netfs = netfs; /* used in completion */
 
-	xa_for_each_range(&sreq->rreq->mapping->i_pages, idx, page, start, last) {
+	xa_for_each_range(&rreq->mapping->i_pages, idx, page, start, last) {
 		/* nfs_read_add_folio() may schedule() due to pNFS layout and other RPCs  */
 		err = nfs_read_add_folio(&pgio, ctx, page_folio(page));
 		if (err < 0) {
@@ -332,6 +345,7 @@ static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
 out:
 	nfs_pageio_complete_read(&pgio);
 	nfs_netfs_put(netfs);
+	return 0;
 }
 
 void nfs_netfs_initiate_read(struct nfs_pgio_header *hdr)
diff --git a/fs/smb/client/cifssmb.c b/fs/smb/client/cifssmb.c
index 1f77512252e7..37d576d38837 100644
--- a/fs/smb/client/cifssmb.c
+++ b/fs/smb/client/cifssmb.c
@@ -1467,8 +1467,7 @@ cifs_readv_callback(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 	struct cifs_tcon *tcon = tlink_tcon(rdata->req->cfile->tlink);
 	struct inode *inode = &ictx->inode;
 	struct smb_rqst rqst = { .rq_iov = rdata->iov,
-				 .rq_nvec = 1,
-				 .rq_iter = rdata->subreq.io_iter };
+				 .rq_nvec = 1};
 	struct cifs_credits credits = {
 		.value = 1,
 		.instance = 0,
@@ -1482,6 +1481,11 @@ cifs_readv_callback(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 		 __func__, mid->mid, mid->mid_state, rdata->result,
 		 rdata->subreq.len);
 
+	if (rdata->got_bytes)
+		iov_iter_bvec_queue(&rqst.rq_iter, ITER_DEST,
+				    rdata->subreq.content.bvecq, rdata->subreq.content.slot,
+				    rdata->subreq.content.offset, rdata->subreq.len);
+
 	switch (mid->mid_state) {
 	case MID_RESPONSE_RECEIVED:
 		/* result already set, check signature */
@@ -2007,7 +2011,10 @@ cifs_async_writev(struct cifs_io_subrequest *wdata)
 
 	rqst.rq_iov = iov;
 	rqst.rq_nvec = 1;
-	rqst.rq_iter = wdata->subreq.io_iter;
+
+	iov_iter_bvec_queue(&rqst.rq_iter, ITER_SOURCE,
+			    wdata->subreq.content.bvecq, wdata->subreq.content.slot,
+			    wdata->subreq.content.offset, wdata->subreq.len);
 
 	cifs_dbg(FYI, "async write at %llu %zu bytes\n",
 		 wdata->subreq.start, wdata->subreq.len);
diff --git a/fs/smb/client/file.c b/fs/smb/client/file.c
index ac89c1ba56b1..bc08e5808cf0 100644
--- a/fs/smb/client/file.c
+++ b/fs/smb/client/file.c
@@ -43,18 +43,34 @@ static int cifs_reopen_file(struct cifsFileInfo *cfile, bool can_flush);
  * Prepare a subrequest to upload to the server.  We need to allocate credits
  * so that we know the maximum amount of data that we can include in it.
  */
-static void cifs_prepare_write(struct netfs_io_subrequest *subreq)
+static int cifs_estimate_write(struct netfs_io_request *wreq,
+			       struct netfs_io_stream *stream,
+			       struct netfs_write_estimate *estimate)
+{
+	struct cifs_sb_info *cifs_sb = CIFS_SB(wreq->inode->i_sb);
+
+	estimate->issue_at = stream->issue_from + cifs_sb->ctx->wsize;
+	return 0;
+}
+
+/*
+ * Issue a subrequest to upload to the server.
+ */
+static int cifs_issue_write(struct netfs_io_subrequest *subreq)
 {
 	struct cifs_io_subrequest *wdata =
 		container_of(subreq, struct cifs_io_subrequest, subreq);
 	struct cifs_io_request *req = wdata->req;
-	struct netfs_io_stream *stream = &req->rreq.io_streams[subreq->stream_nr];
 	struct TCP_Server_Info *server;
 	struct cifsFileInfo *open_file = req->cfile;
-	struct cifs_sb_info *cifs_sb = CIFS_SB(wdata->rreq->inode->i_sb);
-	size_t wsize = req->rreq.wsize;
+	struct cifs_sb_info *cifs_sb = CIFS_SB(subreq->rreq->inode->i_sb);
+	unsigned int max_segs = INT_MAX;
+	size_t len;
 	int rc;
 
+	if (cifs_forced_shutdown(cifs_sb))
+		return smb_EIO(smb_eio_trace_forced_shutdown);
+
 	if (!wdata->have_xid) {
 		wdata->xid = get_xid();
 		wdata->have_xid = true;
@@ -73,18 +89,16 @@ static void cifs_prepare_write(struct netfs_io_subrequest *subreq)
 		if (rc < 0) {
 			if (rc == -EAGAIN)
 				goto retry;
-			subreq->error = rc;
-			return netfs_prepare_write_failed(subreq);
+			return rc;
 		}
 	}
 
-	rc = server->ops->wait_mtu_credits(server, wsize, &stream->sreq_max_len,
-					   &wdata->credits);
-	if (rc < 0) {
-		subreq->error = rc;
-		return netfs_prepare_write_failed(subreq);
-	}
+	len = umin(subreq->len, cifs_sb->ctx->wsize);
+	rc = server->ops->wait_mtu_credits(server, len, &len, &wdata->credits);
+	if (rc < 0)
+		return rc;
 
+	subreq->len = len;
 	wdata->credits.rreq_debug_id = subreq->rreq->debug_id;
 	wdata->credits.rreq_debug_index = subreq->debug_index;
 	wdata->credits.in_flight_check = 1;
@@ -100,46 +114,34 @@ static void cifs_prepare_write(struct netfs_io_subrequest *subreq)
 		const struct smbdirect_socket_parameters *sp =
 			smbd_get_parameters(server->smbd_conn);
 
-		stream->sreq_max_segs = sp->max_frmr_depth;
+		max_segs = sp->max_frmr_depth;
 	}
 #endif
-}
 
-/*
- * Issue a subrequest to upload to the server.
- */
-static void cifs_issue_write(struct netfs_io_subrequest *subreq)
-{
-	struct cifs_io_subrequest *wdata =
-		container_of(subreq, struct cifs_io_subrequest, subreq);
-	struct cifs_sb_info *sbi = CIFS_SB(subreq->rreq->inode->i_sb);
-	int rc;
-
-	if (cifs_forced_shutdown(sbi)) {
-		rc = smb_EIO(smb_eio_trace_forced_shutdown);
-		goto fail;
+	rc = netfs_prepare_write_buffer(subreq, max_segs);
+	if (rc < 0) {
+		add_credits_and_wake_if(wdata->server, &wdata->credits, 0);
+		return rc;
 	}
+	/* After this point, must fail by termination. */
 
-	rc = adjust_credits(wdata->server, wdata, cifs_trace_rw_credits_issue_write_adjust);
+	rc = adjust_credits(server, wdata, cifs_trace_rw_credits_issue_write_adjust);
 	if (rc)
-		goto fail;
+		goto fail_with_credits;
 
 	rc = -EAGAIN;
 	if (wdata->req->cfile->invalidHandle)
-		goto fail;
+		goto fail_with_credits;
 
 	wdata->server->ops->async_writev(wdata);
-out:
-	return;
+	return 0;
 
-fail:
+fail_with_credits:
 	if (rc == -EAGAIN)
 		trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
-	else
-		trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
 	add_credits_and_wake_if(wdata->server, &wdata->credits, 0);
 	cifs_write_subrequest_terminated(wdata, rc);
-	goto out;
+	return 0;
 }
 
 static void cifs_netfs_invalidate_cache(struct netfs_io_request *wreq)
@@ -148,17 +150,25 @@ static void cifs_netfs_invalidate_cache(struct netfs_io_request *wreq)
 }
 
 /*
- * Negotiate the size of a read operation on behalf of the netfs library.
+ * Issue a read operation on behalf of the netfs helper functions.  We're asked
+ * to make a read of a certain size at a point in the file.  We are permitted
+ * to only read a portion of that, but as long as we read something, the netfs
+ * helper will call us again so that we can issue another read.
  */
-static int cifs_prepare_read(struct netfs_io_subrequest *subreq)
+static int cifs_issue_read(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	struct cifs_io_subrequest *rdata = container_of(subreq, struct cifs_io_subrequest, subreq);
 	struct cifs_io_request *req = container_of(subreq->rreq, struct cifs_io_request, rreq);
-	struct TCP_Server_Info *server;
+	struct TCP_Server_Info *server = rdata->server;
 	struct cifs_sb_info *cifs_sb = CIFS_SB(rreq->inode->i_sb);
-	size_t size;
-	int rc = 0;
+	unsigned int max_segs = INT_MAX;
+	size_t len;
+	int rc;
+
+	cifs_dbg(FYI, "%s: op=%08x[%x] mapping=%p len=%zu/%zu\n",
+		 __func__, rreq->debug_id, subreq->debug_index, rreq->mapping,
+		 subreq->transferred, subreq->len);
 
 	if (!rdata->have_xid) {
 		rdata->xid = get_xid();
@@ -172,17 +182,15 @@ static int cifs_prepare_read(struct netfs_io_subrequest *subreq)
 		cifs_negotiate_rsize(server, cifs_sb->ctx,
 				     tlink_tcon(req->cfile->tlink));
 
-	rc = server->ops->wait_mtu_credits(server, cifs_sb->ctx->rsize,
-					   &size, &rdata->credits);
+	len = umin(subreq->len, cifs_sb->ctx->rsize);
+	rc = server->ops->wait_mtu_credits(server, len, &len, &rdata->credits);
 	if (rc)
 		return rc;
 
-	rreq->io_streams[0].sreq_max_len = size;
-
-	rdata->credits.in_flight_check = 1;
+	subreq->len = len;
 	rdata->credits.rreq_debug_id = rreq->debug_id;
 	rdata->credits.rreq_debug_index = subreq->debug_index;
-
+	rdata->credits.in_flight_check = 1;
 	trace_smb3_rw_credits(rdata->rreq->debug_id,
 			      rdata->subreq.debug_index,
 			      rdata->credits.value,
@@ -194,40 +202,27 @@ static int cifs_prepare_read(struct netfs_io_subrequest *subreq)
 		const struct smbdirect_socket_parameters *sp =
 			smbd_get_parameters(server->smbd_conn);
 
-		rreq->io_streams[0].sreq_max_segs = sp->max_frmr_depth;
+		max_segs = sp->max_frmr_depth;
 	}
 #endif
-	return 0;
-}
-
-/*
- * Issue a read operation on behalf of the netfs helper functions.  We're asked
- * to make a read of a certain size at a point in the file.  We are permitted
- * to only read a portion of that, but as long as we read something, the netfs
- * helper will call us again so that we can issue another read.
- */
-static void cifs_issue_read(struct netfs_io_subrequest *subreq)
-{
-	struct netfs_io_request *rreq = subreq->rreq;
-	struct cifs_io_subrequest *rdata = container_of(subreq, struct cifs_io_subrequest, subreq);
-	struct cifs_io_request *req = container_of(subreq->rreq, struct cifs_io_request, rreq);
-	struct TCP_Server_Info *server = rdata->server;
-	int rc = 0;
 
-	cifs_dbg(FYI, "%s: op=%08x[%x] mapping=%p len=%zu/%zu\n",
-		 __func__, rreq->debug_id, subreq->debug_index, rreq->mapping,
-		 subreq->transferred, subreq->len);
+	rc = netfs_prepare_read_buffer(subreq, max_segs);
+	if (rc < 0) {
+		add_credits_and_wake_if(rdata->server, &rdata->credits, 0);
+		return rc;
+	}
+	/* After this point, must fail by termination. */
 
 	rc = adjust_credits(server, rdata, cifs_trace_rw_credits_issue_read_adjust);
 	if (rc)
-		goto failed;
+		goto fail_with_credits;
 
 	if (req->cfile->invalidHandle) {
 		do {
 			rc = cifs_reopen_file(req->cfile, true);
 		} while (rc == -EAGAIN);
 		if (rc)
-			goto failed;
+			goto fail_with_credits;
 	}
 
 	if (subreq->rreq->origin != NETFS_UNBUFFERED_READ &&
@@ -235,15 +230,22 @@ static void cifs_issue_read(struct netfs_io_subrequest *subreq)
 		__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+
 	rc = rdata->server->ops->async_readv(rdata);
 	if (rc)
 		goto failed;
-	return;
+	return 0;
 
+fail_with_credits:
+	if (rc == -EAGAIN)
+		trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+	else
+		trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
 failed:
 	add_credits_and_wake_if(rdata->server, &rdata->credits, 0);
 	subreq->error = rc;
 	netfs_read_subreq_terminated(subreq);
+	return 0;
 }
 
 /*
@@ -358,11 +360,10 @@ const struct netfs_request_ops cifs_req_ops = {
 	.init_request		= cifs_init_request,
 	.free_request		= cifs_free_request,
 	.free_subrequest	= cifs_free_subrequest,
-	.prepare_read		= cifs_prepare_read,
 	.issue_read		= cifs_issue_read,
 	.done			= cifs_rreq_done,
 	.begin_writeback	= cifs_begin_writeback,
-	.prepare_write		= cifs_prepare_write,
+	.estimate_write		= cifs_estimate_write,
 	.issue_write		= cifs_issue_write,
 	.invalidate_cache	= cifs_netfs_invalidate_cache,
 };
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 28902d3c5ef4..28553b74333a 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -4884,6 +4884,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 	unsigned int cur_page_idx;
 	unsigned int pad_len;
 	struct cifs_io_subrequest *rdata = mid->callback_data;
+	struct iov_iter iter;
 	struct smb2_hdr *shdr = (struct smb2_hdr *)buf;
 	size_t copied;
 	bool use_rdma_mr = false;
@@ -4956,6 +4957,10 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 
 	pad_len = data_offset - server->vals->read_rsp_size;
 
+	iov_iter_bvec_queue(&iter, ITER_DEST,
+			    rdata->subreq.content.bvecq, rdata->subreq.content.slot,
+			    rdata->subreq.content.offset, rdata->subreq.len);
+
 	if (buf_len <= data_offset) {
 		/* read response payload is in pages */
 		cur_page_idx = pad_len / PAGE_SIZE;
@@ -4985,7 +4990,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 
 		/* Copy the data to the output I/O iterator. */
 		rdata->result = cifs_copy_bvecq_to_iter(buffer, data_len,
-							cur_off, &rdata->subreq.io_iter);
+							cur_off, &iter);
 		if (rdata->result != 0) {
 			if (is_offloaded)
 				mid->mid_state = MID_RESPONSE_MALFORMED;
@@ -4999,7 +5004,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 		   buf_len >= end_off) {
 		/* read response payload is in buf */
 		WARN_ONCE(buffer, "read data can be either in buf or in buffer");
-		copied = copy_to_iter(buf + data_offset, data_len, &rdata->subreq.io_iter);
+		copied = copy_to_iter(buf + data_offset, data_len, &iter);
 		if (copied == 0)
 			return smb_EIO2(smb_eio_trace_rx_copy_to_iter, copied, data_len);
 		rdata->got_bytes = copied;
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index 4ce165e40657..085fdc7d8bec 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -4592,9 +4592,13 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
 	 */
 	if (rdata && smb3_use_rdma_offload(io_parms)) {
 		struct smbdirect_buffer_descriptor_v1 *v1;
+		struct iov_iter iter;
 		bool need_invalidate = server->dialect == SMB30_PROT_ID;
 
-		rdata->mr = smbd_register_mr(server->smbd_conn, &rdata->subreq.io_iter,
+		iov_iter_bvec_queue(&iter, ITER_DEST,
+				    rdata->subreq.content.bvecq, rdata->subreq.content.slot,
+				    rdata->subreq.content.offset, rdata->subreq.len);
+		rdata->mr = smbd_register_mr(server->smbd_conn, &iter,
 					     true, need_invalidate);
 		if (!rdata->mr)
 			return -EAGAIN;
@@ -4658,9 +4662,10 @@ smb2_readv_callback(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 	unsigned int rreq_debug_id = rdata->rreq->debug_id;
 	unsigned int subreq_debug_index = rdata->subreq.debug_index;
 
-	if (rdata->got_bytes) {
-		rqst.rq_iter	  = rdata->subreq.io_iter;
-	}
+	if (rdata->got_bytes)
+		iov_iter_bvec_queue(&rqst.rq_iter, ITER_DEST,
+				    rdata->subreq.content.bvecq, rdata->subreq.content.slot,
+				    rdata->subreq.content.offset, rdata->subreq.len);
 
 	WARN_ONCE(rdata->server != server,
 		  "rdata server %p != mid server %p",
@@ -5148,7 +5153,9 @@ smb2_async_writev(struct cifs_io_subrequest *wdata)
 		goto out;
 
 	rqst.rq_iov = iov;
-	rqst.rq_iter = wdata->subreq.io_iter;
+	iov_iter_bvec_queue(&rqst.rq_iter, ITER_SOURCE,
+			    wdata->subreq.content.bvecq, wdata->subreq.content.slot,
+			    wdata->subreq.content.offset, wdata->subreq.len);
 
 	rqst.rq_iov[0].iov_len = total_len - 1;
 	rqst.rq_iov[0].iov_base = (char *)req;
@@ -5187,9 +5194,14 @@ smb2_async_writev(struct cifs_io_subrequest *wdata)
 	 */
 	if (smb3_use_rdma_offload(io_parms)) {
 		struct smbdirect_buffer_descriptor_v1 *v1;
+		struct iov_iter iter;
 		bool need_invalidate = server->dialect == SMB30_PROT_ID;
 
-		wdata->mr = smbd_register_mr(server->smbd_conn, &wdata->subreq.io_iter,
+		iov_iter_bvec_queue(&iter, ITER_SOURCE,
+				    wdata->subreq.content.bvecq, wdata->subreq.content.slot,
+				    wdata->subreq.content.offset, wdata->subreq.len);
+
+		wdata->mr = smbd_register_mr(server->smbd_conn, &iter,
 					     false, need_invalidate);
 		if (!wdata->mr) {
 			rc = -EAGAIN;
@@ -5226,8 +5238,8 @@ smb2_async_writev(struct cifs_io_subrequest *wdata)
 		smb2_set_replay(server, &rqst);
 	}
 
-	cifs_dbg(FYI, "async write at %llu %u bytes iter=%zx\n",
-		 io_parms->offset, io_parms->length, iov_iter_count(&wdata->subreq.io_iter));
+	cifs_dbg(FYI, "async write at %llu %u bytes len=%zx\n",
+		 io_parms->offset, io_parms->length, wdata->subreq.len);
 
 	if (wdata->credits.value > 0) {
 		shdr->CreditCharge = cpu_to_le16(DIV_ROUND_UP(wdata->subreq.len,
diff --git a/fs/smb/client/transport.c b/fs/smb/client/transport.c
index fdf4e50c27ce..be2f6b909c34 100644
--- a/fs/smb/client/transport.c
+++ b/fs/smb/client/transport.c
@@ -1267,12 +1267,19 @@ cifs_readv_receive(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 	}
 
 #ifdef CONFIG_CIFS_SMB_DIRECT
-	if (rdata->mr)
+	if (rdata->mr) {
 		length = data_len; /* An RDMA read is already done. */
-	else
+	} else {
+#endif
+		struct iov_iter iter;
+
+		iov_iter_bvec_queue(&iter, ITER_DEST, rdata->subreq.content.bvecq,
+				    rdata->subreq.content.slot, rdata->subreq.content.offset,
+				    data_len);
+		length = cifs_read_iter_from_socket(server, &iter, data_len);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	}
 #endif
-		length = cifs_read_iter_from_socket(server, &rdata->subreq.io_iter,
-						    data_len);
 	if (length > 0)
 		rdata->got_bytes += length;
 	server->total_read += length;
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 856a6ba3fc71..d780d1d9adf9 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -66,7 +66,7 @@ struct netfs_inode {
 #endif
 	struct list_head	wb_queue;	/* Queue of processes wanting to do writeback */
 	loff_t			_remote_i_size;	/* Size of the remote file */
-	loff_t			_zero_point;	/* Size after which we assume there's no data
+	unsigned long long	_zero_point;	/* Size after which we assume there's no data
 						 * on the server */
 	spinlock_t		lock;		/* Lock covering wb_queue */
 	atomic_t		io_count;	/* Number of outstanding reqs */
@@ -128,25 +128,39 @@ static inline struct netfs_group *netfs_folio_group(struct folio *folio)
 	return priv;
 }
 
+/*
+ * Estimate of maximum write subrequest for writeback.  The filesystem is
+ * responsible for filling this in when called from ->estimate_write(), though
+ * netfslib will preset infinite defaults.
+ */
+struct netfs_write_estimate {
+	unsigned long long	issue_at;	/* Point at which we must submit */
+	int			max_segs;	/* Max number of segments in a single RPC */
+};
+
 /*
  * Stream of I/O subrequests going to a particular destination, such as the
  * server or the local cache.  This is mainly intended for writing where we may
  * have to write to multiple destinations concurrently.
  */
 struct netfs_io_stream {
-	/* Submission tracking */
-	struct netfs_io_subrequest *construct;	/* Op being constructed */
-	size_t			sreq_max_len;	/* Maximum size of a subrequest */
-	unsigned int		sreq_max_segs;	/* 0 or max number of segments in an iterator */
-	unsigned int		submit_off;	/* Folio offset we're submitting from */
-	unsigned int		submit_len;	/* Amount of data left to submit */
-	void (*prepare_write)(struct netfs_io_subrequest *subreq);
-	void (*issue_write)(struct netfs_io_subrequest *subreq);
+	/* Submission tracking (main dispatch only; not retry) */
+	struct bvecq_pos	dispatch_cursor; /* Point from which buffers are dispatched */
+	unsigned long long	issue_from;	/* Current issue point */
+	size_t			buffered;	/* Amount in buffer */
+	u8			applicable;	/* What sources are applicable (NOTE_* mask) */
+	bool			buffering;	/* T if buffering on this stream */
+	int (*estimate_write)(struct netfs_io_request *wreq,
+			      struct netfs_io_stream *stream,
+			      struct netfs_write_estimate *estimate);
+	int (*issue_write)(struct netfs_io_subrequest *subreq);
+	atomic64_t		issued_to;	/* Point to which can be considered issued */
+
 	/* Collection tracking */
 	struct list_head	subrequests;	/* Contributory I/O operations */
 	unsigned long long	collected_to;	/* Position we've collected results to */
 	size_t			transferred;	/* The amount transferred from this stream */
-	unsigned short		error;		/* Aggregate error for the stream */
+	short			error;		/* Aggregate error for the stream */
 	enum netfs_io_source	source;		/* Where to read from/write to */
 	unsigned char		stream_nr;	/* Index of stream in parent table */
 	bool			avail;		/* T if stream is available */
@@ -184,14 +198,13 @@ struct netfs_io_subrequest {
 	struct list_head	rreq_link;	/* Link in rreq->subrequests */
 	struct bvecq_pos	dispatch_pos;	/* Bookmark in the combined queue of the start */
 	struct bvecq_pos	content;	/* The (copied) content of the subrequest */
-	struct iov_iter		io_iter;	/* Iterator for this subrequest */
 	unsigned long long	start;		/* Where to start the I/O */
 	size_t			len;		/* Size of the I/O */
 	size_t			transferred;	/* Amount of data transferred */
+	unsigned int		nr_segs;	/* Number of segments in content */
 	refcount_t		ref;
 	short			error;		/* 0 or error that occurred */
 	unsigned short		debug_index;	/* Index in list (for debugging output) */
-	unsigned int		nr_segs;	/* Number of segs in io_iter */
 	u8			retry_count;	/* The number of retries (0 on initial pass) */
 	enum netfs_io_source	source;		/* Where to read from/write to */
 	unsigned char		stream_nr;	/* I/O stream this belongs to */
@@ -199,7 +212,6 @@ struct netfs_io_subrequest {
 #define NETFS_SREQ_COPY_TO_CACHE	0	/* Set if should copy the data to the cache */
 #define NETFS_SREQ_CLEAR_TAIL		1	/* Set if the rest of the read should be cleared */
 #define NETFS_SREQ_MADE_PROGRESS	4	/* Set if we transferred at least some data */
-#define NETFS_SREQ_BOUNDARY		6	/* Set if ends on hard boundary (eg. ceph object) */
 #define NETFS_SREQ_HIT_EOF		7	/* Set if short due to EOF */
 #define NETFS_SREQ_IN_PROGRESS		8	/* Unlocked when the subrequest completes */
 #define NETFS_SREQ_NEED_RETRY		9	/* Set if the filesystem requests a retry */
@@ -246,23 +258,26 @@ struct netfs_io_request {
 	struct netfs_group	*group;		/* Writeback group being written back */
 	struct bvecq		*spare;		/* Advance allocation of bvecq */
 	struct bvecq_pos	load_cursor;	/* Point at which new folios are loaded in */
-	struct bvecq_pos	dispatch_cursor; /* Point from which buffers are dispatched */
 	struct bvecq_pos	collect_cursor;	/* Clear-up point of I/O buffer */
+	struct bvecq_pos	retry_cursor;	/* Point from which retries are dispatched */
 	wait_queue_head_t	waitq;		/* Processor waiter */
 	void			*netfs_priv;	/* Private data for the netfs */
 	void			*netfs_priv2;	/* Private data for the netfs */
-	unsigned long long	last_end;	/* End pos of last folio submitted */
 	unsigned long long	submitted;	/* Amount submitted for I/O so far */
 	unsigned long long	len;		/* Length of the request */
 	size_t			transferred;	/* Amount to be indicated as transferred */
 	long			error;		/* 0 or error that occurred */
 	unsigned long long	i_size;		/* Size of the file */
 	unsigned long long	start;		/* Start position */
-	atomic64_t		issued_to;	/* Write issuer folio cursor */
 	unsigned long long	collected_to;	/* Point we've collected to */
 	unsigned long long	cache_coll_to;	/* Point the cache has collected to */
 	unsigned long long	cleaned_to;	/* Position we've cleaned folios to */
 	unsigned long long	abandon_to;	/* Position to abandon folios to */
+#ifdef CONFIG_NETFS_PGPRIV2
+	unsigned long long	last_end;	/* End of last folio added */
+#endif
+	unsigned long long	retry_start;	/* Position to retry from */
+	size_t			retry_buffered;	/* Amount of data to retry */
 	const struct folio	*no_unlock_folio; /* Don't unlock this folio after read */
 	gfp_t			gfp;		/* GFP flags to use */
 	unsigned int		debug_id;
@@ -282,6 +297,7 @@ struct netfs_io_request {
 #define NETFS_RREQ_RETRYING		4	/* Set if we're in the retry path */
 #define NETFS_RREQ_SHORT_TRANSFER	5	/* Set if we have a short transfer */
 #define NETFS_RREQ_ABANDON_REQ		6	/* Set if the request is to be abandoned */
+#define NETFS_RREQ_SAW_ENOMEM		7	/* Set if we encounted ENOMEM */
 #define NETFS_RREQ_CACHE_STOP		8	/* Set to stop caching (ENOBUFS or error) */
 #define NETFS_RREQ_CACHE_ERROR		9	/* Set if we got an error from the cache */
 #define NETFS_RREQ_OFFLOAD_COLLECTION	12	/* Offload collection to workqueue */
@@ -290,8 +306,10 @@ struct netfs_io_request {
 #define NETFS_RREQ_UPLOAD_TO_SERVER	15	/* Need to write to the server */
 #define NETFS_RREQ_USE_IO_ITER		16	/* Use ->io_iter rather than ->i_pages */
 #define NETFS_RREQ_NEED_PUT_RA_REFS	17	/* Need to put the folio refs RA gave us */
+#ifdef CONFIG_NETFS_PGPRIV2
 #define NETFS_RREQ_USE_PGPRIV2		31	/* [DEPRECATED] Use PG_private_2 to mark
 						 * write to cache on read */
+#endif
 	const struct netfs_request_ops *netfs_ops;
 };
 
@@ -307,8 +325,7 @@ struct netfs_request_ops {
 
 	/* Read request handling */
 	void (*expand_readahead)(struct netfs_io_request *rreq);
-	int (*prepare_read)(struct netfs_io_subrequest *subreq);
-	void (*issue_read)(struct netfs_io_subrequest *subreq);
+	int (*issue_read)(struct netfs_io_subrequest *subreq);
 	bool (*is_still_valid)(struct netfs_io_request *rreq);
 	int (*check_write_begin)(struct file *file, loff_t pos, unsigned len,
 				 struct folio **foliop, void **_fsdata);
@@ -320,8 +337,10 @@ struct netfs_request_ops {
 
 	/* Write request handling */
 	void (*begin_writeback)(struct netfs_io_request *wreq);
-	void (*prepare_write)(struct netfs_io_subrequest *subreq);
-	void (*issue_write)(struct netfs_io_subrequest *subreq);
+	int (*estimate_write)(struct netfs_io_request *wreq,
+			      struct netfs_io_stream *stream,
+			      struct netfs_write_estimate *estimate);
+	int (*issue_write)(struct netfs_io_subrequest *subreq);
 	void (*retry_request)(struct netfs_io_request *wreq, struct netfs_io_stream *stream);
 	void (*invalidate_cache)(struct netfs_io_request *wreq);
 };
@@ -362,8 +381,16 @@ struct netfs_cache_ops {
 		     netfs_io_terminated_t term_func,
 		     void *term_func_priv);
 
+	/* Estimate the amount of data that can be written in an op. */
+	int (*estimate_write)(struct netfs_io_request *wreq,
+			      struct netfs_io_stream *stream,
+			      struct netfs_write_estimate *estimate);
+
+	/* Read data from the cache for a netfs subrequest. */
+	int (*issue_read)(struct netfs_io_subrequest *subreq);
+
 	/* Write data to the cache from a netfs subrequest. */
-	void (*issue_write)(struct netfs_io_subrequest *subreq);
+	int (*issue_write)(struct netfs_io_subrequest *subreq);
 
 	/* Expand readahead request */
 	void (*expand_readahead)(struct netfs_cache_resources *cres,
@@ -371,30 +398,18 @@ struct netfs_cache_ops {
 				 unsigned long long *_len,
 				 unsigned long long i_size);
 
-	/* Prepare a read operation, shortening it to a cached/uncached
-	 * boundary as appropriate.
-	 */
-	int (*prepare_read)(struct netfs_io_subrequest *subreq);
-
-	/* Prepare a write subrequest, working out if we're allowed to do it
-	 * and finding out the maximum amount of data to gather before
-	 * attempting to submit.  If we're not permitted to do it, the
-	 * subrequest should be marked failed.
-	 */
-	void (*prepare_write_subreq)(struct netfs_io_subrequest *subreq);
-
 	/* Prepare a write operation, working out what part of the write we can
 	 * actually do.
 	 */
-	int (*prepare_write)(struct netfs_cache_resources *cres,
-			     loff_t *_start, size_t *_len, size_t upper_len,
-			     loff_t i_size, bool no_space_allocated_yet);
+	int (*prepare_write_old)(struct netfs_cache_resources *cres,
+				 loff_t *_start, size_t *_len, size_t upper_len,
+				 loff_t i_size, bool no_space_allocated_yet);
 
 	/* Query the occupancy of the cache in a region, returning where the
 	 * next chunk of data starts and how long it is.
 	 */
-	int (*query_occupancy)(struct netfs_cache_resources *cres,
-			       struct fscache_occupancy *occ);
+	void (*query_occupancy)(struct netfs_cache_resources *cres,
+				struct fscache_occupancy *occ);
 
 	/* Collect the result of buffered writeback to the cache.  This
 	 * includes copying a read to the cache.  block_type is one of:
@@ -428,10 +443,9 @@ void netfs_single_mark_inode_dirty(struct inode *inode);
 ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_iter *iter);
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
-			   struct iov_iter *iter);
+			   struct iov_iter *iter, size_t len);
 
 /* Address operations API */
-struct readahead_control;
 void netfs_readahead(struct readahead_control *);
 int netfs_read_folio(struct file *, struct folio *);
 int netfs_write_begin(struct netfs_inode *, struct file *,
@@ -458,7 +472,8 @@ void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
 			   unsigned long long fpos, struct bvecq **_bvecq_head,
 			   iov_iter_extraction_t extraction_flags, gfp_t gfp);
-void netfs_prepare_write_failed(struct netfs_io_subrequest *subreq);
+int netfs_prepare_read_buffer(struct netfs_io_subrequest *subreq, unsigned int max_segs);
+int netfs_prepare_write_buffer(struct netfs_io_subrequest *subreq, unsigned int max_segs);
 void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error);
 
 int netfs_start_io_read(struct inode *inode);
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 723cb7315308..e6bbf2a09d92 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -47,6 +47,7 @@
 	E_(NETFS_PGPRIV2_COPY_TO_CACHE,		"2C")
 
 #define netfs_rreq_traces					\
+	EM(netfs_rreq_trace_all_queued,		"ALL-Q  ")	\
 	EM(netfs_rreq_trace_assess,		"ASSESS ")	\
 	EM(netfs_rreq_trace_cache_cancelled,	"CA-CNCL")	\
 	EM(netfs_rreq_trace_cache_failed,	"CA-FAIL")	\
@@ -84,7 +85,8 @@
 	EM(netfs_rreq_trace_waited_quiesce,	"DONE-QUIESCE")	\
 	EM(netfs_rreq_trace_wake_ip,		"WAKE-IP")	\
 	EM(netfs_rreq_trace_wake_queue,		"WAKE-Q ")	\
-	E_(netfs_rreq_trace_write_done,		"WR-DONE")
+	EM(netfs_rreq_trace_write_done,		"WR-DONE")	\
+	E_(netfs_rreq_trace_zero_unread,	"ZERO-UR")
 
 #define netfs_sreq_sources					\
 	EM(NETFS_SOURCE_UNKNOWN,		"----")		\
@@ -133,6 +135,7 @@
 	EM(netfs_sreq_trace_superfluous,	"SPRFL")	\
 	EM(netfs_sreq_trace_terminated,		"TERM ")	\
 	EM(netfs_sreq_trace_too_much,		"!TOOM")	\
+	EM(netfs_sreq_trace_too_many_retries,	"!RETR")	\
 	EM(netfs_sreq_trace_wait_for,		"_WAIT")	\
 	EM(netfs_sreq_trace_write,		"WRITE")	\
 	EM(netfs_sreq_trace_write_skip,		"SKIP ")	\
@@ -200,12 +203,12 @@
 	EM(netfs_folio_trace_alloc_buffer,	"alloc-buf")	\
 	EM(netfs_folio_trace_cancel_copy,	"cancel-copy")	\
 	EM(netfs_folio_trace_cancel_store,	"cancel-store")	\
-	EM(netfs_folio_trace_clear,		"clear")	\
-	EM(netfs_folio_trace_clear_cc,		"clear-cc")	\
-	EM(netfs_folio_trace_clear_g,		"clear-g")	\
-	EM(netfs_folio_trace_clear_s,		"clear-s")	\
 	EM(netfs_folio_trace_copy_to_cache,	"mark-copy")	\
 	EM(netfs_folio_trace_end_copy,		"end-copy")	\
+	EM(netfs_folio_trace_endwb,		"endwb")	\
+	EM(netfs_folio_trace_endwb_cc,		"endwb-cc")	\
+	EM(netfs_folio_trace_endwb_g,		"endwb-g")	\
+	EM(netfs_folio_trace_endwb_s,		"endwb-s")	\
 	EM(netfs_folio_trace_filled_gaps,	"filled-gaps")	\
 	EM(netfs_folio_trace_invalidate_all,	"inval-all")	\
 	EM(netfs_folio_trace_invalidate_front,	"inval-front")	\
@@ -396,10 +399,10 @@ TRACE_EVENT(netfs_sreq,
 		    __entry->len	= sreq->len;
 		    __entry->transferred = sreq->transferred;
 		    __entry->start	= sreq->start;
-		    __entry->slot	= sreq->dispatch_pos.slot;
+		    __entry->slot	= sreq->content.slot;
 			   ),
 
-	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx qs=%u e=%d",
+	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx bv=%u e=%d",
 		      __entry->rreq, __entry->index,
 		      __print_symbolic(__entry->source, netfs_sreq_sources),
 		      __print_symbolic(__entry->what, netfs_sreq_traces),
@@ -507,6 +510,7 @@ TRACE_EVENT(netfs_folio,
 	    TP_STRUCT__entry(
 		    __field(u64,			ino)
 		    __field(pgoff_t,			index)
+		    __field(unsigned long,		pfn)
 		    __field(unsigned int,		nr)
 		    __field(enum netfs_folio_trace,	why)
 			     ),
@@ -517,13 +521,40 @@ TRACE_EVENT(netfs_folio,
 		    __entry->why = why;
 		    __entry->index = folio->index;
 		    __entry->nr = folio_nr_pages(folio);
+		    __entry->pfn = folio_pfn(folio);
 			   ),
 
-	    TP_printk("i=%05llx ix=%05lx-%05lx %s",
+	    TP_printk("p=%lx i=%05llx ix=%05lx-%05lx %s",
+		      __entry->pfn,
 		      __entry->ino, __entry->index, __entry->index + __entry->nr - 1,
 		      __print_symbolic(__entry->why, netfs_folio_traces))
 	    );
 
+TRACE_EVENT(netfs_wback,
+	    TP_PROTO(struct netfs_io_request *wreq, struct folio *folio, unsigned int notes),
+
+	    TP_ARGS(wreq, folio, notes),
+
+	    TP_STRUCT__entry(
+		    __field(pgoff_t,			index)
+		    __field(unsigned int,		wreq)
+		    __field(unsigned int,		nr)
+		    __field(unsigned int,		notes)
+			     ),
+
+	    TP_fast_assign(
+		    __entry->wreq = wreq->debug_id;
+		    __entry->notes = notes;
+		    __entry->index = folio->index;
+		    __entry->nr = folio_nr_pages(folio);
+			   ),
+
+	    TP_printk("R=%08x ix=%05lx-%05lx n=%02x",
+		      __entry->wreq,
+		      __entry->index, __entry->index + __entry->nr - 1,
+		      __entry->notes)
+	    );
+
 TRACE_EVENT(netfs_write_iter,
 	    TP_PROTO(const struct kiocb *iocb, const struct iov_iter *from),
 
@@ -767,7 +798,7 @@ TRACE_EVENT(netfs_collect_stream,
 		    __entry->wreq	= wreq->debug_id;
 		    __entry->stream	= stream->stream_nr;
 		    __entry->collected_to = stream->collected_to;
-		    __entry->issued_to	= atomic64_read(&wreq->issued_to);
+		    __entry->issued_to	= atomic64_read(&stream->issued_to);
 			   ),
 
 	    TP_printk("R=%08x[%x:] cto=%llx ito=%llx",
@@ -791,7 +822,7 @@ TRACE_EVENT(netfs_bvecq,
 		    __entry->trace	= trace;
 			   ),
 
-	    TP_printk("fq=%x %s",
+	    TP_printk("bq=%x %s",
 		      __entry->id,
 		      __print_symbolic(__entry->trace, netfs_bvecq_traces))
 	    );
diff --git a/net/9p/client.c b/net/9p/client.c
index ef64546c6d52..68d82f677a7a 100644
--- a/net/9p/client.c
+++ b/net/9p/client.c
@@ -1564,6 +1564,7 @@ void
 p9_client_write_subreq(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *wreq = subreq->rreq;
+	struct iov_iter iter;
 	struct p9_fid *fid = wreq->netfs_priv;
 	struct p9_client *clnt = fid->clnt;
 	struct p9_req_t *req;
@@ -1574,14 +1575,17 @@ p9_client_write_subreq(struct netfs_io_subrequest *subreq)
 	p9_debug(P9_DEBUG_9P, ">>> TWRITE fid %d offset %llu len %d\n",
 		 fid->fid, start, len);
 
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
 	/* Don't bother zerocopy for small IO (< 1024) */
 	if (clnt->trans_mod->zc_request && len > 1024) {
-		req = p9_client_zc_rpc(clnt, P9_TWRITE, NULL, &subreq->io_iter,
+		req = p9_client_zc_rpc(clnt, P9_TWRITE, NULL, &iter,
 				       0, wreq->len, P9_ZC_HDR_SZ, "dqd",
 				       fid->fid, start, len);
 	} else {
 		req = p9_client_rpc(clnt, P9_TWRITE, "dqV", fid->fid,
-				    start, len, &subreq->io_iter);
+				    start, len, &iter);
 	}
 	if (IS_ERR(req)) {
 		netfs_write_subrequest_terminated(subreq, PTR_ERR(req));



From dhowells at redhat.com  Tue Aug  4 03:02:20 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:20 +0100
Subject: [PATCH v8 25/25] cachefiles: Preset the state xattr when creating a new file
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-26-dhowells@redhat.com>

With a really small cache, cachefiles is likely to see a lot of writes
hitting ENOSPC - and this can include setxattr that sets the state xattr on
a cachefile - but we don't really want to successfully fill a cache file
only to have to scrap it because we can't set the xattr.

Instead, preset the xattr when we create the tmpfile we're going to use,
and scrap the file at that point if we get ENOSPC.  Only if setxattr
succeeds do we allow data to be written to the file.

Note that there is a potential performance loss in that writes to the cache
have to be delayed until this is completed - but we do the tmpfile/setxattr
in parallel, starting when the file is opened and only have to wait once
writeback occurs.

Further, a new tracepoint, cachefiles_no_space, has been added to log ENOSPC
events, along with a couple of subreq traces to log failure detection in
netfslib.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/cachefiles/internal.h          |  1 +
 fs/cachefiles/io.c                | 23 ++++++++----
 fs/cachefiles/namei.c             | 15 ++++++--
 fs/cachefiles/xattr.c             | 60 ++++++++++++++++++++++++++++++-
 include/trace/events/cachefiles.h | 28 ++++++++++++++-
 include/trace/events/netfs.h      |  2 ++
 6 files changed, 119 insertions(+), 10 deletions(-)

diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 99d5f9077a31..71832fdb3770 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -287,6 +287,7 @@ void cachefiles_withdraw_volume(struct cachefiles_volume *volume);
 /*
  * xattr.c
  */
+int cachefiles_preset_object_xattr(struct cachefiles_object *object, struct file *file);
 extern int cachefiles_set_object_xattr(struct cachefiles_object *object);
 extern int cachefiles_check_auxdata(struct cachefiles_object *object,
 				    struct file *file);
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index b06d0fbcc81e..e8722b9bf183 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -672,10 +672,14 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	 * space, we need to see if it's fully allocated.  If it's not, we may
 	 * want to cull it.
 	 */
-	if (cachefiles_has_space(cache, 0, *_len / cache->bsize,
-				 cachefiles_has_space_check) == 0)
+	ret = cachefiles_has_space(cache, 0, *_len / cache->bsize,
+				   cachefiles_has_space_check);
+	if (ret == 0)
 		return 0; /* Enough space to simply overwrite the whole block */
 
+	if (ret == -ENOBUFS)
+		trace_cachefiles_no_space(object, cachefiles_trace_write_nospace);
+
 	pos = cachefiles_inject_read_error();
 	if (pos == 0)
 		pos = vfs_llseek(file, start, SEEK_HOLE);
@@ -704,8 +708,11 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	return ret;
 
 check_space:
-	return cachefiles_has_space(cache, 0, *_len / cache->bsize,
-				    cachefiles_has_space_for_write);
+	ret = cachefiles_has_space(cache, 0, *_len / cache->bsize,
+				   cachefiles_has_space_for_write);
+	if (ret == -ENOBUFS)
+		trace_cachefiles_no_space(object, cachefiles_trace_write_nospace);
+	return ret;
 }
 
 static int cachefiles_prepare_write_old(struct netfs_cache_resources *cres,
@@ -757,10 +764,14 @@ static int cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 	       wreq->debug_id, subreq->debug_index, start, start + len - 1);
 
 	if (!cachefiles_cres_file(cres)) {
-		if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
+		if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE)) {
+			trace_netfs_sreq(subreq, netfs_sreq_trace_cache_waitfail);
 			return -EINVAL;
-		if (!cachefiles_cres_file(cres))
+		}
+		if (!cachefiles_cres_file(cres)) {
+			trace_netfs_sreq(subreq, netfs_sreq_trace_cache_nofile);
 			return -EINVAL;
+		}
 	}
 
 	ret = netfs_prepare_write_buffer(subreq, BIO_MAX_VECS);
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index a30df0f91e5e..ef656a319ede 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -117,8 +117,11 @@ struct dentry *cachefiles_get_directory(struct cachefiles_cache *cache,
 	if (d_is_negative(subdir)) {
 		ret = cachefiles_has_space(cache, 1, 0,
 					   cachefiles_has_space_for_create);
-		if (ret < 0)
+		if (ret < 0) {
+			if (ret == -ENOBUFS)
+				trace_cachefiles_no_space(NULL, cachefiles_trace_mkdir_nospace);
 			goto mkdir_error;
+		}
 
 		_debug("attempt mkdir");
 
@@ -447,6 +450,11 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 		pr_notice("Cache does not support read_iter and write_iter\n");
 		goto err_unuse;
 	}
+
+	/* Preallocate space for the xattr. */
+	ret = cachefiles_preset_object_xattr(object, file);
+	if (ret < 0)
+		goto err_unuse;
 out:
 	cachefiles_end_secure(cache, saved_cred);
 	object->content_info = CACHEFILES_CONTENT_ALL;
@@ -470,8 +478,11 @@ static bool cachefiles_create_file(struct cachefiles_object *object)
 
 	ret = cachefiles_has_space(object->volume->cache, 1, 0,
 				   cachefiles_has_space_for_create);
-	if (ret < 0)
+	if (ret < 0) {
+		if (ret == -ENOBUFS)
+			trace_cachefiles_no_space(object, cachefiles_trace_create_nospace);
 		return false;
+	}
 
 	file = cachefiles_create_tmpfile(object);
 	if (IS_ERR(file))
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 25f2a906f984..2a7cc514f1a5 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -33,6 +33,58 @@ struct cachefiles_vol_xattr {
 	__u8	data[];		/* netfs volume coherency data */
 } __packed;
 
+/*
+ * Preset the state xattr on a cache file to allocate space for it.
+ */
+int cachefiles_preset_object_xattr(struct cachefiles_object *object, struct file *file)
+{
+	struct cachefiles_xattr *buf;
+	struct dentry *dentry = file->f_path.dentry;
+	unsigned int len = object->cookie->aux_len;
+	int ret;
+
+	buf = kzalloc(sizeof(struct cachefiles_xattr) + len, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	buf->type		= CACHEFILES_COOKIE_TYPE_DATA;
+	buf->content		= CACHEFILES_CONTENT_DIRTY;
+
+	ret = cachefiles_inject_write_error();
+	if (ret == 0) {
+		ret = mnt_want_write_file(file);
+		if (ret == 0) {
+			ret = vfs_setxattr(&nop_mnt_idmap, dentry,
+					   cachefiles_xattr_cache, buf,
+					   sizeof(struct cachefiles_xattr) + len, 0);
+			mnt_drop_write_file(file);
+		}
+	}
+	if (ret < 0) {
+		trace_cachefiles_vfs_error(object, file_inode(file), ret,
+					   cachefiles_trace_setxattr_error);
+		trace_cachefiles_coherency(object, file_inode(file)->i_ino,
+					   object->object_size,
+					   be64_to_cpup((__be64 *)buf->data),
+					   buf->content,
+					   cachefiles_coherency_set_fail);
+		switch (ret) {
+		case -ENOMEM:
+		case -ENOSPC:
+			break;
+		default:
+			cachefiles_io_error_obj(
+				object,
+				"Failed to set xattr with error %d", ret);
+			break;
+		}
+	}
+
+	kfree(buf);
+	_leave(" = %d", ret);
+	return ret;
+}
+
 /*
  * set the state xattr on a cache file
  */
@@ -81,10 +133,16 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
 					   be64_to_cpup((__be64 *)buf->data),
 					   buf->content,
 					   cachefiles_coherency_set_fail);
-		if (ret != -ENOMEM)
+		switch (ret) {
+		case -ENOMEM:
+			break;
+		case -ENOSPC:
+		default:
 			cachefiles_io_error_obj(
 				object,
 				"Failed to set xattr with error %d", ret);
+			break;
+		}
 	} else {
 		trace_cachefiles_coherency(object, file_inode(file)->i_ino,
 					   object->object_size,
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index eccdb5abeedc..c7b5b8587cca 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -85,11 +85,13 @@ enum cachefiles_prepare_read_trace {
 
 enum cachefiles_error_trace {
 	cachefiles_trace_alignment_error,
+	cachefiles_trace_create_nospace,
 	cachefiles_trace_fallocate_error,
 	cachefiles_trace_getxattr_error,
 	cachefiles_trace_link_error,
 	cachefiles_trace_lookup_error,
 	cachefiles_trace_mkdir_error,
+	cachefiles_trace_mkdir_nospace,
 	cachefiles_trace_notify_change_error,
 	cachefiles_trace_open_error,
 	cachefiles_trace_read_error,
@@ -102,6 +104,7 @@ enum cachefiles_error_trace {
 	cachefiles_trace_trunc_error,
 	cachefiles_trace_unlink_error,
 	cachefiles_trace_write_error,
+	cachefiles_trace_write_nospace,
 };
 
 #endif
@@ -171,11 +174,13 @@ enum cachefiles_error_trace {
 
 #define cachefiles_error_traces						\
 	EM(cachefiles_trace_alignment_error,	"align")		\
+	EM(cachefiles_trace_create_nospace,	"create-nospace")	\
 	EM(cachefiles_trace_fallocate_error,	"fallocate")		\
 	EM(cachefiles_trace_getxattr_error,	"getxattr")		\
 	EM(cachefiles_trace_link_error,		"link")			\
 	EM(cachefiles_trace_lookup_error,	"lookup")		\
 	EM(cachefiles_trace_mkdir_error,	"mkdir")		\
+	EM(cachefiles_trace_mkdir_nospace,	"mkdir-nospace")	\
 	EM(cachefiles_trace_notify_change_error, "notify_change")	\
 	EM(cachefiles_trace_open_error,		"open")			\
 	EM(cachefiles_trace_read_error,		"read")			\
@@ -187,7 +192,8 @@ enum cachefiles_error_trace {
 	EM(cachefiles_trace_tmpfile_error,	"tmpfile")		\
 	EM(cachefiles_trace_trunc_error,	"trunc")		\
 	EM(cachefiles_trace_unlink_error,	"unlink")		\
-	E_(cachefiles_trace_write_error,	"write")
+	EM(cachefiles_trace_write_error,	"write")		\
+	E_(cachefiles_trace_write_nospace,	"write-nospace")
 
 
 /*
@@ -692,6 +698,26 @@ TRACE_EVENT(cachefiles_io_error,
 		      __entry->error)
 	    );
 
+TRACE_EVENT(cachefiles_no_space,
+	    TP_PROTO(struct cachefiles_object *obj, enum cachefiles_error_trace trace),
+
+	    TP_ARGS(obj, trace),
+
+	    TP_STRUCT__entry(
+		    __field(unsigned int,			obj)
+		    __field(enum cachefiles_error_trace,	trace)
+			     ),
+
+	    TP_fast_assign(
+		    __entry->obj	= obj ? obj->debug_id : 0;
+		    __entry->trace	= trace;
+			   ),
+
+	    TP_printk("o=%08x %s",
+		      __entry->obj,
+		      __print_symbolic(__entry->trace, cachefiles_error_traces))
+	    );
+
 #endif /* _TRACE_CACHEFILES_H */
 
 /* This part must be outside protection */
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index e6bbf2a09d92..45c1924c0b2f 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -101,8 +101,10 @@
 	EM(netfs_sreq_trace_abandoned,		"ABNDN")	\
 	EM(netfs_sreq_trace_add_donations,	"+DON ")	\
 	EM(netfs_sreq_trace_added,		"ADD  ")	\
+	EM(netfs_sreq_trace_cache_nofile,	"CA-!F")	\
 	EM(netfs_sreq_trace_cache_nowrite,	"CA-NW")	\
 	EM(netfs_sreq_trace_cache_prepare,	"CA-PR")	\
+	EM(netfs_sreq_trace_cache_waitfail,	"CA-!W")	\
 	EM(netfs_sreq_trace_cache_write,	"CA-WR")	\
 	EM(netfs_sreq_trace_cancel,		"CANCL")	\
 	EM(netfs_sreq_trace_clear,		"CLEAR")	\



From dhowells at redhat.com  Tue Aug  4 03:02:11 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 11:02:11 +0100
Subject: [PATCH v8 16/25] netfs: Switch to using bvecq rather than folio_queue and rolling_buffer
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260804100224.2748935-17-dhowells@redhat.com>

Switch netfslib to using bvecq, a segmented bio_vec[] queue, instead of the
folio_queue and rolling_buffer constructs, to keep track of the regions of
memory it is performing I/O upon.  Each bvecq struct in the chain is marked
with the starting file position of that sequence so that discontiguities
can be handled (the contents of each individual bvecq must be contiguous).

For unbuffered/direct I/O, the iterator is extracted into the queue up
front.  For buffered I/O, the folios are added to the queue as the
operation proceeds, much as it does now with folio_queues.  There is one
important change for buffered writes: only the relevant part of the folio
is included; this is expanded for writes to the cache in a copy of the
bvecq segment (it is known that each bio_vec corresponds to part of a
folio in this case).

The bvecq structs are marked with information as to how the regions
contained therein should be disposed of (unlock-only, free, unpin).

When setting up a subrequest, netfslib will furnish it with a slice of the
main buffer queue as a pointer to starting bvecq, slot and offset and, for
the moment, an ITER_BVECQ iterator is set to cover the slice in
subreq->io_iter.

Notes on the implementation:

 (1) This patch uses the concept of a 'bvecq position', which is a tuple of
     { bvecq, slot, offset }.  This is lighter weight than using a full
     iov_iter, though that would also suffice.  If not NULL, the position
     also holds a reference on the bvecq it is pointing to.  This is
     probably overkill as only the hindmost position (that of collection)
     needs to hold a reference.

 (2) There are three positions on the netfs_io_request struct.  Not all are
     used by every request type.

     Firstly, there's ->load_cursor, which is used by buffered read and
     write to point to the next slot to have a folio inserted into it
     (either loaded from the readahead_control or from writeback_iter()).

     Secondly, there's ->dispatch_cursor, which is used to provide the
     position in the buffer from which we start dispatching a subrequest.

     Thirdly, there's the ->collect_cursor, which is used by the collection
     routines to point to the next memory region to be cleaned up.

 (3) There are two positions on the netfs_io_subrequest struct.

     Firstly, there's ->dispatch_pos, which indicates the position from
     which a subrequest's buffer begins.  This is used as the base of the
     position from which to retry (advanced by ->transfer).

     Secondly, there's ->content, which is normally the same as
     ->dispatch_pos but if the bvecq chain got duplicated or the content
     got copied, then this will point to that and will that will be
     disposed of on retry.

 (4) Maintenance of the position structs is done with helper functions,
     such as bvecq_pos_attach() to hide the refcounting.

 (5) When sending a write to the cache, the bvecq will be duplicated and
     the ends rounded up/down to the backing file's DIO block alignment.

 (6) bvec_slice() is used to select a slice of the source buffer and assign
     it to a subrequest.  The source buffer position is advanced.

 (7) netfs_extract_iter() is used by unbuffered/direct I/O API functions to
     decant a chunk of the iov_iter supplied by the VFS into a bvecq chain
     - and to label the bvecqs with appropriate disposal information
     (e.g. unpin, free, nothing).

There are further options that can be explored in the future:

 (1) Allow the provision of a duplicated bvecq chain for just that region
     so that the filesystem can add bits on either end (such as adding
     protocol headers and trailers and gluing several things together into
     a compound operation).

 (2) If a filesystem supports vectored/sparse read and write ops, it can be
     given a chain with discontiguities in it to perform in a single op
     (Ceph, for example, can do this).

 (3) Because each bvecq notes the start file position of the regions
     contained therein, there's no need to translate the info in the
     bio_vec into folio pointers in order to unlock the page after I/O.
     Instead, the inode's pagecache can be iterated over and the xarray
     marks cleared en masse.

 (4) Make MSG_SPLICE_PAGES handling read the disposal info in the bvecq and
     use that to indicate how it should get rid of the stuff it pasted into
     a sk_buff.

 (5) If a bounce buffer is needed (encryption, for example), the bounce
     buffer can be held in a bvecq and sliced up instead of the main buffer
     queue.

 (6) Get rid of subreq->io_iter and move the iov_iter stuff down into the
     filesystem.  The I/O iterators are normally only needed transitorily,
     and the one currently in netfs_io_subrequest is unnecessary most of
     the time.

folio_queue and rolling_buffer will be removed in a follow up patch.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: Shyam Prasad N <sprasad at microsoft.com>
cc: Tom Talpey <tom at talpey.com>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/cachefiles/io.c           |  12 ---
 fs/netfs/Makefile            |   1 -
 fs/netfs/buffered_read.c     | 122 +++++++++++++---------
 fs/netfs/direct_read.c       |  78 ++++++--------
 fs/netfs/direct_write.c      |  69 +++++++------
 fs/netfs/internal.h          |  10 +-
 fs/netfs/iterator.c          |   2 +
 fs/netfs/main.c              |   6 --
 fs/netfs/misc.c              |  20 +---
 fs/netfs/objects.c           |  17 ++-
 fs/netfs/read_collect.c      |  98 ++++++++++--------
 fs/netfs/read_pgpriv2.c      |  94 +++++++++++------
 fs/netfs/read_retry.c        | 104 +++++++++++--------
 fs/netfs/read_single.c       |  17 ++-
 fs/netfs/stats.c             |   4 +-
 fs/netfs/write_collect.c     |  60 +++++------
 fs/netfs/write_issue.c       | 194 ++++++++++++++++++++++++++---------
 fs/netfs/write_retry.c       |  62 ++++++-----
 include/linux/netfs.h        |  25 ++---
 include/trace/events/netfs.h |  46 ++++-----
 20 files changed, 596 insertions(+), 445 deletions(-)

diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 2b5471ca8378..d7ad65fc840e 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -636,7 +636,6 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 	struct netfs_cache_resources *cres = &wreq->cache_resources;
 	struct cachefiles_object *object = cachefiles_cres_object(cres);
 	struct cachefiles_cache *cache = object->volume->cache;
-	struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];
 	const struct cred *saved_cred;
 	size_t off, pre, post, len = subreq->len;
 	loff_t start = subreq->start;
@@ -661,17 +660,6 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 	}
 
 	/* We also need to end on the cache granularity boundary */
-	if (start + len == wreq->i_size) {
-		size_t part = len & (cache->bsize - 1);
-		size_t need = cache->bsize - part;
-
-		if (part && stream->submit_extendable_to >= need) {
-			len += need;
-			subreq->len += need;
-			subreq->io_iter.count += need;
-		}
-	}
-
 	post = len & (cache->bsize - 1);
 	if (post) {
 		len -= post;
diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index e1f12ecb5abf..0621e6870cbd 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -15,7 +15,6 @@ netfs-y := \
 	read_pgpriv2.o \
 	read_retry.o \
 	read_single.o \
-	rolling_buffer.o \
 	write_collect.o \
 	write_issue.o \
 	write_retry.o
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 0f4070c8c0d5..1f052bb6c861 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -114,26 +114,21 @@ static int netfs_begin_cache_read(struct netfs_io_request *rreq, struct netfs_in
 static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	ssize_t extracted;
 	size_t rsize = subreq->len;
 
 	if (subreq->source == NETFS_DOWNLOAD_FROM_SERVER)
-		rsize = umin(rsize, rreq->io_streams[0].sreq_max_len);
-
-	subreq->len = rsize;
-	if (unlikely(rreq->io_streams[0].sreq_max_segs)) {
-		size_t limit = netfs_limit_iter(&rreq->buffer.iter, 0, rsize,
-						rreq->io_streams[0].sreq_max_segs);
-
-		if (limit < rsize) {
-			subreq->len = limit;
-			trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
-		}
+		rsize = umin(rsize, stream->sreq_max_len);
+
+	bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
+	extracted = bvecq_slice(&rreq->dispatch_cursor, rsize,
+				stream->sreq_max_segs, &subreq->nr_segs);
+	if (extracted < rsize) {
+		subreq->len = extracted;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
 	}
 
-	subreq->io_iter	= rreq->buffer.iter;
-
-	iov_iter_truncate(&subreq->io_iter, subreq->len);
-	rolling_buffer_advance(&rreq->buffer, subreq->len);
 	return subreq->len;
 }
 
@@ -192,6 +187,10 @@ void netfs_queue_read(struct netfs_io_request *rreq,
 static void netfs_issue_read(struct netfs_io_request *rreq,
 			     struct netfs_io_subrequest *subreq)
 {
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
 	switch (subreq->source) {
 	case NETFS_DOWNLOAD_FROM_SERVER:
 		rreq->netfs_ops->issue_read(subreq);
@@ -200,7 +199,8 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
 		netfs_read_cache_to_pagecache(rreq, subreq);
 		break;
 	default:
-		__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
+		bvecq_zero(&subreq->content, subreq->len);
+		subreq->transferred = subreq->len;
 		subreq->error = 0;
 		iov_iter_zero(subreq->len, &subreq->io_iter);
 		subreq->transferred = subreq->len;
@@ -229,6 +229,11 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 	ssize_t size = rreq->len;
 	int ret = 0;
 
+	_enter("R=%08x", rreq->debug_id);
+
+	bvecq_pos_set(&rreq->dispatch_cursor, &rreq->load_cursor);
+	bvecq_pos_set(&rreq->collect_cursor, &rreq->dispatch_cursor);
+
 	do {
 		int (*prepare_read)(struct netfs_io_subrequest *subreq) = NULL;
 		struct netfs_io_subrequest *subreq;
@@ -376,6 +381,9 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 
 	/* Defer error return as we may need to wait for outstanding I/O. */
 	cmpxchg(&rreq->error, 0, ret);
+
+	bvecq_pos_unset(&rreq->load_cursor);
+	bvecq_pos_unset(&rreq->dispatch_cursor);
 }
 
 /**
@@ -423,7 +431,7 @@ void netfs_readahead(struct readahead_control *ractl)
 	 * acquires a ref on each folio that we will need to release later -
 	 * but we don't want to do that until after we've started the I/O.
 	 */
-	added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl, rreq->debug_id);
+	added = bvecq_load_from_ra(&rreq->load_cursor, ractl);
 	if (added < 0) {
 		ret = added;
 		goto cleanup_free;
@@ -432,7 +440,7 @@ void netfs_readahead(struct readahead_control *ractl)
 
 	rreq->submitted = rreq->start + added;
 	rreq->cleaned_to = rreq->start;
-	rreq->front_folio_order = folio_order(rreq->buffer.tail->vec.folios[0]);
+	rreq->front_folio_order = get_order(rreq->load_cursor.bvecq->bv[0].bv_len);
 
 	netfs_read_to_pagecache(rreq);
 	netfs_maybe_bulk_drop_ra_refs(rreq);
@@ -444,20 +452,20 @@ void netfs_readahead(struct readahead_control *ractl)
 EXPORT_SYMBOL(netfs_readahead);
 
 /*
- * Create a rolling buffer with a single occupying folio.
+ * Create a buffer queue with a single occupying folio.
  */
-static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio,
-					unsigned int rollbuf_flags)
+static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio)
 {
-	ssize_t added;
+	struct bvecq *bq;
+	size_t fsize = folio_size(folio);
 
-	if (rolling_buffer_init(&rreq->buffer, rreq->debug_id, ITER_DEST, rreq->gfp) < 0)
+	if (bvecq_buffer_init(&rreq->load_cursor, rreq->gfp) < 0)
 		return -ENOMEM;
 
-	added = rolling_buffer_append(&rreq->buffer, folio, rollbuf_flags, rreq->gfp);
-	if (added < 0)
-		return added;
-	rreq->submitted = rreq->start + added;
+	bq = rreq->load_cursor.bvecq;
+	bvec_set_folio(&bq->bv[0], folio, fsize, 0);
+	bvecq_filled_to(bq, 1);
+	rreq->submitted = rreq->start + fsize;
 	return 0;
 }
 
@@ -471,11 +479,11 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 	struct netfs_group *group = netfs_folio_group(folio);
 	struct netfs_folio *finfo = netfs_folio_info(folio);
 	struct netfs_inode *ctx = netfs_inode(mapping->host);
-	struct folio *sink = NULL;
-	struct bio_vec *bvec;
+	struct bvecq *bq = NULL;
+	struct page *sink = NULL;
 	unsigned int from = finfo->dirty_offset;
 	unsigned int to = from + finfo->dirty_len;
-	unsigned int off = 0, i = 0;
+	unsigned int off = 0, slot = 0;
 	size_t flen = folio_size(folio);
 	size_t nr_bvec = flen / PAGE_SIZE + 2;
 	size_t part;
@@ -500,32 +508,47 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 	 * end get copied to, but the middle is discarded.
 	 */
 	ret = -ENOMEM;
-	bvec = kmalloc_objs(*bvec, nr_bvec);
-	if (!bvec)
+	bq = bvecq_alloc_chain(nr_bvec, rreq->gfp);
+	if (!bq)
 		goto discard;
+	rreq->load_cursor.bvecq = bq;
 
-	sink = folio_alloc(GFP_KERNEL, 0);
-	if (!sink) {
-		kfree(bvec);
+	sink = alloc_page(rreq->gfp);
+	if (!sink)
 		goto discard;
-	}
 
 	trace_netfs_folio(folio, netfs_folio_trace_read_gaps);
 
-	rreq->direct_bv = bvec;
-	rreq->direct_bv_count = nr_bvec;
+	for (struct bvecq *p = bq; p; p = p->next)
+		p->mem_type = BVECQ_MEM_PAGECACHE;
+
 	if (from > 0) {
-		bvec_set_folio(&bvec[i++], folio, from, 0);
+		folio_get(folio);
+		bvec_set_folio(&bq->bv[slot++], folio, from, 0);
 		off = from;
 	}
 	while (off < to) {
-		part = min_t(size_t, to - off, PAGE_SIZE);
-		bvec_set_folio(&bvec[i++], sink, part, 0);
+		if (slot >= bq->max_slots) {
+			bvecq_filled_to(bq, slot);
+			bq = bq->next;
+			slot = 0;
+		}
+		part = min(to - off, PAGE_SIZE);
+		get_page(sink);
+		bvec_set_page(&bq->bv[slot++], sink, part, 0);
 		off += part;
 	}
-	if (to < flen)
-		bvec_set_folio(&bvec[i++], folio, flen - to, to);
-	iov_iter_bvec(&rreq->buffer.iter, ITER_DEST, bvec, i, rreq->len);
+	if (to < flen) {
+		if (slot >= bq->max_slots) {
+			bvecq_filled_to(bq, slot);
+			bq = bq->next;
+			slot = 0;
+		}
+		folio_get(folio);
+		bvec_set_folio(&bq->bv[slot++], folio, flen - to, to);
+	}
+	bvecq_filled_to(bq, slot);
+
 	rreq->submitted = rreq->start + flen;
 
 	netfs_read_to_pagecache(rreq);
@@ -542,13 +565,14 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 		folio_mark_uptodate(folio);
 	}
 
-	if (sink)
-		folio_put(sink);
+	put_page(sink);
 	folio_unlock(folio);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret < 0 ? ret : 0;
 
 discard:
+	if (sink)
+		put_page(sink);
 	netfs_put_failed_request(rreq);
 alloc_error:
 	folio_unlock(folio);
@@ -599,7 +623,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
 	trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, 0);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto discard;
 
@@ -756,7 +780,7 @@ int netfs_write_begin(struct netfs_inode *ctx,
 	trace_netfs_read(rreq, pos, len, netfs_read_trace_write_begin);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, 0);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto error_put;
 
@@ -821,7 +845,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 	trace_netfs_read(rreq, start, flen, netfs_read_trace_prefetch_for_write);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, NETFS_ROLLBUF_PAGECACHE_MARK);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto error_put;
 
diff --git a/fs/netfs/direct_read.c b/fs/netfs/direct_read.c
index 6a8fb0d55e04..4d35f8efc633 100644
--- a/fs/netfs/direct_read.c
+++ b/fs/netfs/direct_read.c
@@ -16,44 +16,22 @@
 #include <linux/netfs.h>
 #include "internal.h"
 
-static void netfs_prepare_dio_read_iterator(struct netfs_io_subrequest *subreq)
-{
-	struct netfs_io_request *rreq = subreq->rreq;
-	size_t rsize;
-
-	rsize = umin(subreq->len, rreq->io_streams[0].sreq_max_len);
-	subreq->len = rsize;
-
-	if (unlikely(rreq->io_streams[0].sreq_max_segs)) {
-		size_t limit = netfs_limit_iter(&rreq->buffer.iter, 0, rsize,
-						rreq->io_streams[0].sreq_max_segs);
-
-		if (limit < rsize) {
-			subreq->len = limit;
-			trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
-		}
-	}
-
-	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-
-	subreq->io_iter	= rreq->buffer.iter;
-	iov_iter_truncate(&subreq->io_iter, subreq->len);
-	iov_iter_advance(&rreq->buffer.iter, subreq->len);
-}
-
 /*
  * Perform a read to a buffer from the server, slicing up the region to be read
  * according to the network rsize.
  */
 static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 {
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
 	unsigned long long start = rreq->start;
 	ssize_t size = rreq->len;
 	int ret;
 
+	bvecq_pos_set(&rreq->dispatch_cursor, &rreq->load_cursor);
+	bvecq_pos_transfer(&rreq->collect_cursor, &rreq->load_cursor);
+
 	do {
 		struct netfs_io_subrequest *subreq;
-		ssize_t slice;
 
 		subreq = netfs_alloc_subrequest(rreq);
 		if (!subreq) {
@@ -79,16 +57,24 @@ static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 			}
 		}
 
-		netfs_prepare_dio_read_iterator(subreq);
-		slice = subreq->len;
-		size -= slice;
-		start += slice;
-		rreq->submitted += slice;
+		bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
+		bvecq_pos_set(&subreq->content, &rreq->dispatch_cursor);
+		subreq->len = bvecq_slice(&rreq->dispatch_cursor,
+					  umin(size, stream->sreq_max_len),
+					  stream->sreq_max_segs,
+					  &subreq->nr_segs);
+
+		size -= subreq->len;
+		start += subreq->len;
+		rreq->submitted += subreq->len;
 		if (size <= 0) {
 			smp_wmb(); /* Write lists before ALL_QUEUED. */
 			set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
 		}
 
+		iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
+				    subreq->content.slot, subreq->content.offset, subreq->len);
+
 		rreq->netfs_ops->issue_read(subreq);
 
 		if (test_bit(NETFS_RREQ_PAUSE, &rreq->flags))
@@ -103,6 +89,8 @@ static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
 		netfs_wake_collector(rreq);
 	}
+
+	bvecq_pos_unset(&rreq->dispatch_cursor);
 }
 
 /*
@@ -181,25 +169,17 @@ ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *i
 	 * buffer for ourselves as the caller's iterator will be trashed when
 	 * we return.
 	 *
-	 * In such a case, extract an iterator to represent as much of the the
-	 * output buffer as we can manage.  Note that the extraction might not
-	 * be able to allocate a sufficiently large bvec array and may shorten
-	 * the request.
+	 * Extract a buffer queue to represent as much of the output buffer as
+	 * we can manage.  The fragments are extracted into a bvecq which will
+	 * have sufficient nodes allocated to hold all the data, though this
+	 * may end up truncated if ENOMEM is encountered.
 	 */
-	if (user_backed_iter(iter)) {
-		ret = netfs_extract_user_iter(iter, rreq->len, &rreq->buffer.iter, 0);
-		if (ret < 0)
-			goto error_put;
-		rreq->direct_bv = (struct bio_vec *)rreq->buffer.iter.bvec;
-		rreq->direct_bv_count = ret;
-		rreq->direct_bv_unpin = iov_iter_extract_will_pin(iter);
-		rreq->len = iov_iter_count(&rreq->buffer.iter);
-	} else {
-		rreq->buffer.iter = *iter;
-		rreq->len = orig_count;
-		rreq->direct_bv_unpin = false;
-		iov_iter_advance(iter, orig_count);
-	}
+	ret = netfs_extract_iter(iter, rreq->len, INT_MAX, iocb->ki_pos,
+				 &rreq->load_cursor.bvecq, 0, rreq->gfp);
+	if (ret < 0)
+		goto error_put;
+
+	rreq->len = ret;
 
 	// TODO: Set up bounce buffer if needed
 
diff --git a/fs/netfs/direct_write.c b/fs/netfs/direct_write.c
index c16fbad286a1..d81245a29eff 100644
--- a/fs/netfs/direct_write.c
+++ b/fs/netfs/direct_write.c
@@ -73,7 +73,11 @@ static void netfs_unbuffered_write_collect(struct netfs_io_request *wreq,
 	spin_unlock(&wreq->lock);
 
 	wreq->transferred += subreq->transferred;
-	iov_iter_advance(&wreq->buffer.iter, subreq->transferred);
+	if (subreq->transferred < subreq->len) {
+		bvecq_pos_unset(&wreq->dispatch_cursor);
+		bvecq_pos_transfer(&wreq->dispatch_cursor, &subreq->dispatch_pos);
+		bvecq_pos_advance(&wreq->dispatch_cursor, subreq->transferred);
+	}
 
 	stream->collected_to = subreq->start + subreq->transferred;
 	wreq->collected_to = stream->collected_to;
@@ -99,6 +103,9 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 
 	_enter("%llx", wreq->len);
 
+	bvecq_pos_set(&wreq->dispatch_cursor, &wreq->load_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+
 	if (wreq->origin == NETFS_DIO_WRITE)
 		inode_dio_begin(wreq->inode);
 
@@ -111,6 +118,8 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 			netfs_prepare_write(wreq, stream, wreq->start + wreq->transferred);
 			subreq = stream->construct;
 			stream->construct = NULL;
+		} else {
+			bvecq_pos_set(&subreq->dispatch_pos, &wreq->dispatch_cursor);
 		}
 
 		/* Check if (re-)preparation failed. */
@@ -120,16 +129,18 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 			break;
 		}
 
-		iov_iter_truncate(&subreq->io_iter, wreq->len - wreq->transferred);
+		subreq->len = bvecq_slice(&wreq->dispatch_cursor, stream->sreq_max_len,
+					  stream->sreq_max_segs, &subreq->nr_segs);
+		bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+
+		iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
+				    subreq->content.bvecq, subreq->content.slot,
+				    subreq->content.offset,
+				    subreq->len);
+
 		if (!iov_iter_count(&subreq->io_iter))
 			break;
 
-		subreq->len = netfs_limit_iter(&subreq->io_iter, 0,
-					       stream->sreq_max_len,
-					       stream->sreq_max_segs);
-		iov_iter_truncate(&subreq->io_iter, subreq->len);
-		stream->submit_extendable_to = subreq->len;
-
 		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 		stream->issue_write(subreq);
 
@@ -166,9 +177,14 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 		 */
 		subreq->error = -EAGAIN;
 		trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+
+		bvecq_pos_unset(&subreq->content);
+		bvecq_pos_unset(&wreq->dispatch_cursor);
+		bvecq_pos_transfer(&wreq->dispatch_cursor, &subreq->dispatch_pos);
+
 		if (subreq->transferred > 0) {
-			iov_iter_advance(&wreq->buffer.iter, subreq->transferred);
 			wreq->transferred += subreq->transferred;
+			bvecq_pos_advance(&wreq->dispatch_cursor, subreq->transferred);
 		}
 
 		if (stream->source == NETFS_UPLOAD_TO_SERVER &&
@@ -179,7 +195,6 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 		__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 		__clear_bit(NETFS_SREQ_BOUNDARY, &subreq->flags);
 		__clear_bit(NETFS_SREQ_FAILED, &subreq->flags);
-		subreq->io_iter		= wreq->buffer.iter;
 		subreq->start		= wreq->start + wreq->transferred;
 		subreq->len		= wreq->len   - wreq->transferred;
 		subreq->transferred	= 0;
@@ -195,6 +210,8 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 		netfs_stat(&netfs_n_wh_retry_write_subreq);
 	}
 
+	bvecq_pos_unset(&wreq->dispatch_cursor);
+	bvecq_pos_unset(&wreq->load_cursor);
 	netfs_unbuffered_write_done(wreq);
 	_leave(" = %d", ret);
 	return ret;
@@ -213,12 +230,12 @@ static void netfs_unbuffered_write_async(struct work_struct *work)
  * encrypted file.  This can also be used for direct I/O writes.
  */
 ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *iter,
-						  struct netfs_group *netfs_group)
+					   struct netfs_group *netfs_group)
 {
 	struct netfs_io_request *wreq;
 	unsigned long long start = iocb->ki_pos;
 	unsigned long long end = start + iov_iter_count(iter);
-	ssize_t ret, n;
+	ssize_t ret;
 	size_t len = iov_iter_count(iter);
 	bool async = !is_sync_kiocb(iocb);
 
@@ -252,25 +269,17 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 		 * allocate a sufficiently large bvec array and may shorten the
 		 * request.
 		 */
-		if (user_backed_iter(iter)) {
-			n = netfs_extract_user_iter(iter, len, &wreq->buffer.iter, 0);
-			if (n < 0) {
-				ret = n;
-				goto error_put;
-			}
-			wreq->direct_bv = (struct bio_vec *)wreq->buffer.iter.bvec;
-			wreq->direct_bv_count = n;
-			wreq->direct_bv_unpin = iov_iter_extract_will_pin(iter);
-		} else {
-			/* If this is a kernel-generated async DIO request,
-			 * assume that any resources the iterator points to
-			 * (eg. a bio_vec array) will persist till the end of
-			 * the op.
-			 */
-			wreq->buffer.iter = *iter;
-		}
+		ssize_t n = netfs_extract_iter(iter, len, INT_MAX, iocb->ki_pos,
+					       &wreq->load_cursor.bvecq, 0, wreq->gfp);
 
-		wreq->len = iov_iter_count(&wreq->buffer.iter);
+		if (n < 0) {
+			ret = n;
+			goto error_put;
+		}
+		wreq->len = n;
+		_debug("dio-write %zx/%zx %u/%u",
+		       n, len, wreq->load_cursor.bvecq->nr_slots,
+		       wreq->load_cursor.bvecq->max_slots);
 	}
 
 	__set_bit(NETFS_RREQ_USE_IO_ITER, &wreq->flags);
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 3c10f125f13b..667d18911949 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -7,7 +7,6 @@
 
 #include <linux/slab.h>
 #include <linux/seq_file.h>
-#include <linux/folio_queue.h>
 #include <linux/netfs.h>
 #include <linux/fscache.h>
 #include <linux/fscache-cache.h>
@@ -71,9 +70,8 @@ static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}
 /*
  * misc.c
  */
-struct folio_queue *netfs_buffer_make_space(struct netfs_io_request *rreq,
-					    enum netfs_folioq_trace trace);
-void netfs_reset_iter(struct netfs_io_subrequest *subreq);
+struct bvecq *netfs_buffer_make_space(struct netfs_io_request *rreq,
+				      enum netfs_bvecq_trace trace);
 void netfs_wake_collector(struct netfs_io_request *rreq);
 void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq);
 void netfs_wait_for_in_progress_stream(struct netfs_io_request *rreq,
@@ -172,7 +170,6 @@ extern atomic_t netfs_n_wh_retry_write_req;
 extern atomic_t netfs_n_wh_retry_write_subreq;
 extern atomic_t netfs_n_wb_lock_skip;
 extern atomic_t netfs_n_wb_lock_wait;
-extern atomic_t netfs_n_folioq;
 extern atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v);
@@ -210,8 +207,7 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 			 struct netfs_io_stream *stream,
 			 loff_t start);
 void netfs_reissue_write(struct netfs_io_stream *stream,
-			 struct netfs_io_subrequest *subreq,
-			 struct iov_iter *source);
+			 struct netfs_io_subrequest *subreq);
 void netfs_issue_write(struct netfs_io_request *wreq,
 		       struct netfs_io_stream *stream);
 size_t netfs_advance_write(struct netfs_io_request *wreq,
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index 4b58b4fb3d77..2a977b1f8f15 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -158,6 +158,7 @@ ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pag
 }
 EXPORT_SYMBOL_GPL(netfs_extract_iter);
 
+#if 0
 /**
  * netfs_extract_user_iter - Extract the pages from a user iterator into a bvec
  * @orig: The original iterator
@@ -451,3 +452,4 @@ size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
 	BUG();
 }
 EXPORT_SYMBOL(netfs_limit_iter);
+#endif
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 3908ea2422c8..041810707eb4 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -109,9 +109,6 @@ static int __init netfs_init(void)
 {
 	int ret = -ENOMEM;
 
-	if (mempool_init_kmalloc_pool(&netfs_folioq_pool, 100, sizeof(struct folio_queue)) < 0)
-		goto error_folioq_pool;
-
 	if (mempool_init_kmalloc_pool(&netfs_bvecq_pool, 100, BVECQ_STD_SIZE) < 0)
 		goto error_bvecq_pool;
 
@@ -169,8 +166,6 @@ static int __init netfs_init(void)
 error_req:
 	mempool_exit(&netfs_bvecq_pool);
 error_bvecq_pool:
-	mempool_exit(&netfs_folioq_pool);
-error_folioq_pool:
 	return ret;
 }
 fs_initcall(netfs_init);
@@ -184,6 +179,5 @@ static void __exit netfs_exit(void)
 	mempool_exit(&netfs_request_pool);
 	kmem_cache_destroy(netfs_request_slab);
 	mempool_exit(&netfs_bvecq_pool);
-	mempool_exit(&netfs_folioq_pool);
 }
 module_exit(netfs_exit);
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index f5c1c463f4ff..ee67a0681784 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -8,6 +8,7 @@
 #include <linux/swap.h>
 #include "internal.h"
 
+#if 0
 /**
  * netfs_alloc_folioq_buffer - Allocate buffer space into a folio queue
  * @mapping: Address space to set on the folio (or NULL).
@@ -103,24 +104,7 @@ void netfs_free_folioq_buffer(struct folio_queue *fq)
 	folio_batch_release(&fbatch);
 }
 EXPORT_SYMBOL(netfs_free_folioq_buffer);
-
-/*
- * Reset the subrequest iterator to refer just to the region remaining to be
- * read.  The iterator may or may not have been advanced by socket ops or
- * extraction ops to an extent that may or may not match the amount actually
- * read.
- */
-void netfs_reset_iter(struct netfs_io_subrequest *subreq)
-{
-	struct iov_iter *io_iter = &subreq->io_iter;
-	size_t remain = subreq->len - subreq->transferred;
-
-	if (io_iter->count > remain)
-		iov_iter_advance(io_iter, io_iter->count - remain);
-	else if (io_iter->count < remain)
-		iov_iter_revert(io_iter, remain - io_iter->count);
-	iov_iter_truncate(&subreq->io_iter, remain);
-}
+#endif
 
 /**
  * netfs_dirty_folio - Mark folio dirty and pin a cache object for writeback
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index 01461a74642d..889590c6da69 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -124,7 +124,6 @@ static void netfs_free_request_rcu(struct rcu_head *rcu)
 static void netfs_deinit_request(struct netfs_io_request *rreq)
 {
 	struct netfs_inode *ictx = netfs_inode(rreq->inode);
-	unsigned int i;
 
 	trace_netfs_rreq(rreq, netfs_rreq_trace_free);
 
@@ -139,16 +138,10 @@ static void netfs_deinit_request(struct netfs_io_request *rreq)
 		rreq->netfs_ops->free_request(rreq);
 	if (rreq->cache_resources.ops)
 		rreq->cache_resources.ops->end_operation(&rreq->cache_resources);
-	if (rreq->direct_bv) {
-		for (i = 0; i < rreq->direct_bv_count; i++) {
-			if (rreq->direct_bv[i].bv_page) {
-				if (rreq->direct_bv_unpin)
-					unpin_user_page(rreq->direct_bv[i].bv_page);
-			}
-		}
-		kvfree(rreq->direct_bv);
-	}
-	rolling_buffer_clear(&rreq->buffer);
+	bvecq_pos_unset(&rreq->load_cursor);
+	bvecq_pos_unset(&rreq->dispatch_cursor);
+	bvecq_pos_unset(&rreq->collect_cursor);
+	bvecq_put(rreq->spare);
 
 	if (atomic_dec_and_test(&ictx->io_count))
 		wake_up_var(&ictx->io_count);
@@ -240,6 +233,8 @@ static void netfs_free_subrequest(struct netfs_io_subrequest *subreq)
 	trace_netfs_sreq(subreq, netfs_sreq_trace_free);
 	if (rreq->netfs_ops->free_subrequest)
 		rreq->netfs_ops->free_subrequest(subreq);
+	bvecq_pos_unset(&subreq->dispatch_pos);
+	bvecq_pos_unset(&subreq->content);
 	mempool_free(subreq, rreq->netfs_ops->subrequest_pool ?: &netfs_subrequest_pool);
 	netfs_stat_d(&netfs_n_rh_sreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_subreq);
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index edf7cea7e2f9..e0fe7b13dd7a 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -27,9 +27,13 @@
  */
 static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
 {
-	netfs_reset_iter(subreq);
-	WARN_ON_ONCE(subreq->len - subreq->transferred != iov_iter_count(&subreq->io_iter));
-	iov_iter_zero(iov_iter_count(&subreq->io_iter), &subreq->io_iter);
+	struct iov_iter iter;
+
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+	iov_iter_advance(&iter, subreq->transferred);
+	iov_iter_zero(subreq->len, &iter);
+
 	if (subreq->start + subreq->transferred >= subreq->rreq->i_size)
 		__set_bit(NETFS_SREQ_HIT_EOF, &subreq->flags);
 }
@@ -40,11 +44,11 @@ static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
  * dirty and let writeback handle it.
  */
 static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
-				    struct folio_queue *folioq,
+				    struct bvecq *bvecq,
 				    int slot)
 {
 	struct netfs_folio *finfo;
-	struct folio *folio = folioq_folio(folioq, slot);
+	struct folio *folio = page_folio(bvecq->bv[slot].bv_page);
 
 	if (unlikely(folio_pos(folio) < rreq->abandon_to)) {
 		trace_netfs_folio(folio, netfs_folio_trace_abandon);
@@ -75,7 +79,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 			trace_netfs_folio(folio, netfs_folio_trace_read_done);
 		}
 
-		folioq_clear(folioq, slot);
+		bvecq->bv[slot].bv_page = NULL;
 	} else {
 		// TODO: Use of PG_private_2 is deprecated.
 		if (test_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags))
@@ -91,7 +95,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 		folio_unlock(folio);
 	}
 
-	folioq_clear(folioq, slot);
+	bvecq->bv[slot].bv_page = NULL;
 }
 
 /*
@@ -100,24 +104,15 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 				     unsigned int *notes)
 {
-	struct folio_queue *folioq = rreq->buffer.tail;
+	struct bvecq *bvecq = rreq->collect_cursor.bvecq;
 	unsigned long long collected_to = rreq->collected_to;
-	unsigned int slot = rreq->buffer.first_tail_slot;
+	unsigned int slot = rreq->collect_cursor.slot;
 
 	if (rreq->cleaned_to >= rreq->collected_to)
 		return;
 
 	// TODO: Begin decryption
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&rreq->buffer);
-		if (!folioq) {
-			rreq->front_folio_order = 0;
-			return;
-		}
-		slot = 0;
-	}
-
 	/* We have to wait for readahead refs to have been released before we
 	 * can unlock any folios as the ref-dropper walks i_pages and the only
 	 * thing preventing these folios from being removed is the folio lock.
@@ -131,16 +126,30 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 		unsigned int order;
 		size_t fsize;
 
+		/* Clean up the head bvecq segment.  If we clear an entire
+		 * segment, then we can get rid of it provided it's not also
+		 * the tail segment being filled by the issuer.
+		 */
+		if (!bvecq_acquire_slot(bvecq, slot)) {
+			rreq->collect_cursor.slot = slot;
+			if (!bvecq_delete_spent(&rreq->collect_cursor)) {
+				rreq->front_folio_order = 0;
+				return;
+			}
+			bvecq = rreq->collect_cursor.bvecq;
+			slot  = rreq->collect_cursor.slot;
+		}
+
 		if (*notes & COPY_TO_CACHE)
 			set_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
 
-		folio = folioq_folio(folioq, slot);
+		folio = page_folio(bvecq->bv[slot].bv_page);
 		if (WARN_ONCE(!folio_test_locked(folio),
 			      "R=%08x: folio %lx is not locked\n",
 			      rreq->debug_id, folio->index))
 			trace_netfs_folio(folio, netfs_folio_trace_not_locked);
 
-		order = folioq_folio_order(folioq, slot);
+		order = folio_order(folio);
 		rreq->front_folio_order = order;
 		fsize = PAGE_SIZE << order;
 		fpos = folio_pos(folio);
@@ -152,33 +161,19 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 		if (collected_to < fend)
 			break;
 
-		netfs_unlock_read_folio(rreq, folioq, slot);
+		netfs_unlock_read_folio(rreq, bvecq, slot);
+		slot++;
 		WRITE_ONCE(rreq->cleaned_to, fpos + fsize);
 		*notes |= MADE_PROGRESS;
 
 		clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
-		 * being filled by the issuer.
-		 */
-		folioq_clear(folioq, slot);
-		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&rreq->buffer);
-			if (!folioq)
-				goto done;
-			slot = 0;
-			trace_netfs_folioq(folioq, netfs_trace_folioq_read_progress);
-		}
-
 		if (fpos + fsize >= collected_to)
 			break;
 	}
 
-	rreq->buffer.tail = folioq;
-done:
-	rreq->buffer.first_tail_slot = slot;
+	bvecq_pos_move(&rreq->collect_cursor, bvecq);
+	rreq->collect_cursor.slot = slot;
 }
 
 /*
@@ -355,12 +350,17 @@ static void netfs_rreq_assess_dio(struct netfs_io_request *rreq)
 
 	if (rreq->origin == NETFS_UNBUFFERED_READ ||
 	    rreq->origin == NETFS_DIO_READ) {
-		for (i = 0; i < rreq->direct_bv_count; i++) {
-			flush_dcache_page(rreq->direct_bv[i].bv_page);
-			// TODO: cifs marks pages in the destination buffer
-			// dirty under some circumstances after a read.  Do we
-			// need to do that too?
-			set_page_dirty(rreq->direct_bv[i].bv_page);
+		for (struct bvecq *bq = rreq->collect_cursor.bvecq; bq; bq = bvecq_next(bq)) {
+			unsigned int nr_slots = bvecq_nr_slots_acquire(bq);
+			/* Read the slot count before the slots. */
+
+			for (i = 0; i < nr_slots; i++) {
+				flush_dcache_page(bq->bv[i].bv_page);
+				// TODO: cifs marks pages in the destination buffer
+				// dirty under some circumstances after a read.  Do we
+				// need to do that too?
+				set_page_dirty(bq->bv[i].bv_page);
+			}
 		}
 	}
 
@@ -451,7 +451,15 @@ bool netfs_read_collection(struct netfs_io_request *rreq)
 
 	trace_netfs_rreq(rreq, netfs_rreq_trace_done);
 	netfs_clear_subrequests(rreq);
-	netfs_unlock_abandoned_read_pages(rreq);
+	switch (rreq->origin) {
+	case NETFS_READAHEAD:
+	case NETFS_READPAGE:
+	case NETFS_READ_FOR_WRITE:
+		netfs_unlock_abandoned_read_pages(rreq);
+		break;
+	default:
+		break;
+	}
 	if (unlikely(rreq->copy_to_cache))
 		netfs_pgpriv2_end_copy_to_cache(rreq);
 	return true;
diff --git a/fs/netfs/read_pgpriv2.c b/fs/netfs/read_pgpriv2.c
index c31190993b76..b64a8594735a 100644
--- a/fs/netfs/read_pgpriv2.c
+++ b/fs/netfs/read_pgpriv2.c
@@ -19,6 +19,9 @@
 static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio *folio)
 {
 	struct netfs_io_stream *cache = &creq->io_streams[1];
+	struct bvecq *queue;
+	unsigned int slot;
+	size_t dio_size = PAGE_SIZE;
 	size_t fsize = folio_size(folio), flen = fsize;
 	loff_t fpos = folio_pos(folio), i_size;
 	bool to_eof = false;
@@ -48,18 +51,40 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 		to_eof = true;
 	}
 
+	flen = round_up(flen, dio_size);
+
 	_debug("folio %zx %zx", flen, fsize);
 
 	trace_netfs_folio(folio, netfs_folio_trace_store_copy);
 
-	/* Attach the folio to the rolling buffer. */
-	if (rolling_buffer_append(&creq->buffer, folio, 0, creq->gfp) < 0) {
-		folio_end_private_2(folio);
-		clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &creq->flags);
-		return;
+	/* Institute a new bvec queue segment if the current one is full or if
+	 * we encounter a discontiguity.  The discontiguity break is important
+	 * when it comes to bulk unlocking folios by file range.
+	 */
+	queue = creq->load_cursor.bvecq;
+	if (bvecq_is_full(queue) ||
+	    (fpos != creq->last_end && creq->last_end > 0 && queue->nr_slots > 0)) {
+		bvecq_buffer_append(&creq->load_cursor, creq->spare);
+		creq->spare = NULL;
+
+		queue = creq->load_cursor.bvecq;
+		queue->fpos = fpos;
+		if (fpos != creq->last_end)
+			queue->discontig = true;
 	}
 
-	cache->submit_extendable_to = fsize;
+	/* Attach the folio to the rolling buffer. */
+	slot = queue->nr_slots;
+	bvec_set_folio(&queue->bv[slot], folio, fsize, 0);
+	trace_netfs_bv_slot(queue, slot);
+	slot++;
+	bvecq_filled_to(queue, slot);
+	creq->load_cursor.slot = slot;
+	creq->load_cursor.offset = 0;
+	creq->last_end = fpos + flen;
+
+	bvecq_pos_nudge(&creq->dispatch_cursor);
+	
 	cache->submit_off = 0;
 	cache->submit_len = flen;
 
@@ -71,10 +96,9 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	do {
 		ssize_t part;
 
-		creq->buffer.iter.iov_offset = cache->submit_off;
+		creq->dispatch_cursor.offset = cache->submit_off;
 
 		atomic64_set(&creq->issued_to, fpos + cache->submit_off);
-		cache->submit_extendable_to = fsize - cache->submit_off;
 		part = netfs_advance_write(creq, cache, fpos + cache->submit_off,
 					   cache->submit_len, to_eof);
 		cache->submit_off += part;
@@ -84,8 +108,7 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 			cache->submit_len -= part;
 	} while (cache->submit_len > 0);
 
-	creq->buffer.iter.iov_offset = 0;
-	rolling_buffer_advance(&creq->buffer, fsize);
+	bvecq_pos_step(&creq->dispatch_cursor);
 	atomic64_set(&creq->issued_to, fpos + fsize);
 
 	if (flen < fsize)
@@ -111,6 +134,11 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(
 	if (!creq->io_streams[1].avail)
 		goto cancel_put;
 
+	if (bvecq_buffer_init(&creq->load_cursor, creq->gfp) < 0)
+		goto cancel_put;
+	bvecq_pos_set(&creq->dispatch_cursor, &creq->load_cursor);
+	bvecq_pos_set(&creq->collect_cursor, &creq->dispatch_cursor);
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &creq->flags);
 	trace_netfs_copy2cache(rreq, creq);
 	trace_netfs_write(creq, netfs_write_trace_copy_to_cache);
@@ -139,6 +167,14 @@ void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *fo
 	if (IS_ERR(creq))
 		return;
 
+	if (!creq->spare) {
+		creq->spare = bvecq_alloc_one(BVECQ_STD_SLOTS, creq->gfp);
+		if (!creq->spare) {
+			clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &creq->flags);
+			return;
+		}
+	}
+
 	trace_netfs_folio(folio, netfs_folio_trace_copy_to_cache);
 	folio_start_private_2(folio);
 	netfs_pgpriv2_copy_folio(creq, folio);
@@ -171,22 +207,27 @@ void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
  */
 bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
 {
-	struct folio_queue *folioq = creq->buffer.tail;
+	struct bvecq *bq = creq->collect_cursor.bvecq;
 	unsigned long long collected_to = creq->collected_to;
-	unsigned int slot = creq->buffer.first_tail_slot;
+	unsigned int slot;
 	bool made_progress = false;
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&creq->buffer);
-		slot = 0;
-	}
+	slot = creq->collect_cursor.slot;
 
 	for (;;) {
 		struct folio *folio;
 		unsigned long long fpos, fend;
 		size_t fsize, flen;
 
-		folio = folioq_folio(folioq, slot);
+		if (!bvecq_acquire_slot(bq, slot)) {
+			creq->collect_cursor.slot = slot;
+			if (!bvecq_delete_spent(&creq->collect_cursor))
+				goto out;
+			bq   = creq->collect_cursor.bvecq;
+			slot = creq->collect_cursor.slot;
+		}
+
+		folio = page_folio(bq->bv[slot].bv_page);
 		if (WARN_ONCE(!folio_test_private_2(folio),
 			      "R=%08x: folio %lx is not marked private_2\n",
 			      creq->debug_id, folio->index))
@@ -209,25 +250,18 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
 		creq->cleaned_to = fpos + fsize;
 		made_progress = true;
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
-		 * being filled by the issuer.
+		/* Clean up the head segment.  If we clear an entire segment,
+		 * then we can get rid of it provided it's not also the tail
+		 * segment being filled by the issuer.
 		 */
-		folioq_clear(folioq, slot);
+		bq->bv[slot].bv_page = NULL;
 		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&creq->buffer);
-			if (!folioq)
-				goto done;
-			slot = 0;
-		}
 
 		if (fpos + fsize >= collected_to)
 			break;
 	}
 
-	creq->buffer.tail = folioq;
-done:
-	creq->buffer.first_tail_slot = slot;
+	creq->collect_cursor.slot = slot;
+out:
 	return made_progress;
 }
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index ce61f4d8d301..e691b42c19b9 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -12,6 +12,12 @@
 static void netfs_reissue_read(struct netfs_io_request *rreq,
 			       struct netfs_io_subrequest *subreq)
 {
+	bvecq_pos_unset(&subreq->content);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+	iov_iter_advance(&subreq->io_iter, subreq->transferred);
+
 	subreq->error = 0;
 	__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
@@ -27,6 +33,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 {
 	struct netfs_io_subrequest *subreq;
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	struct bvecq_pos dispatch_cursor = {};
 	struct list_head *next;
 
 	_enter("R=%x", rreq->debug_id);
@@ -46,9 +53,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 			if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
 				break;
 			if (__test_and_clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-				__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 				subreq->retry_count++;
-				netfs_reset_iter(subreq);
 				netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
 				netfs_reissue_read(rreq, subreq);
 			}
@@ -74,11 +79,12 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 
 	do {
 		struct netfs_io_subrequest *from, *to, *tmp;
-		struct iov_iter source;
 		unsigned long long start, len;
 		size_t part;
 		bool boundary = false, subreq_superfluous = false;
 
+		bvecq_pos_unset(&dispatch_cursor);
+
 		/* Go through the subreqs and find the next span of contiguous
 		 * buffer that we then rejig (cifs, for example, needs the
 		 * rsize renegotiating) and reissue.
@@ -105,7 +111,8 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				break;
 
 			subreq = list_entry(next, struct netfs_io_subrequest, rreq_link);
-			if (subreq->start + subreq->transferred != start + len ||
+			if (subreq->start != start + len ||
+			    subreq->transferred > 0 ||
 			    test_bit(NETFS_SREQ_BOUNDARY, &subreq->flags) ||
 			    !test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
 				break;
@@ -118,11 +125,14 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		/* Determine the set of buffers we're going to use.  Each
 		 * subreq gets a subset of a single overall contiguous buffer.
 		 */
-		netfs_reset_iter(from);
-		source = from->io_iter;
-		source.count = len;
+		bvecq_pos_transfer(&dispatch_cursor, &from->dispatch_pos);
+		bvecq_pos_advance(&dispatch_cursor, from->transferred);
+		from->transferred = 0;
 
-		/* Work through the sublist. */
+		/* Work through the sublist.  The chain of buffers we're going
+		 * to fill is attached to dispatch_cursor and we need to read
+		 * 'len' amount of data from 'start'.
+		 */
 		subreq = from;
 		list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
 			if (!len) {
@@ -130,16 +140,22 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				break;
 			}
 			subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
-			subreq->start	= start - subreq->transferred;
-			subreq->len	= len   + subreq->transferred;
+			subreq->start	= start;
+			subreq->len	= len;
 			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 			__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 			subreq->retry_count++;
+			subreq->transferred = 0;
+
+			bvecq_pos_unset(&subreq->content);
+			bvecq_pos_unset(&subreq->dispatch_pos);
+			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
 
 			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
 			/* Renegotiate max_len (rsize) */
-			stream->sreq_max_len = subreq->len;
+			stream->sreq_max_len = len;
+			stream->sreq_max_segs = INT_MAX;
 			if (rreq->netfs_ops->prepare_read &&
 			    rreq->netfs_ops->prepare_read(subreq) < 0) {
 				trace_netfs_sreq(subreq, netfs_sreq_trace_reprep_failed);
@@ -147,13 +163,12 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				goto abandon;
 			}
 
-			part = umin(len, stream->sreq_max_len);
-			if (unlikely(stream->sreq_max_segs))
-				part = netfs_limit_iter(&source, 0, part, stream->sreq_max_segs);
-			subreq->len = subreq->transferred + part;
-			subreq->io_iter = source;
-			iov_iter_truncate(&subreq->io_iter, part);
-			iov_iter_advance(&source, part);
+			part = bvecq_slice(&dispatch_cursor,
+					   umin(len, stream->sreq_max_len),
+					   stream->sreq_max_segs,
+					   &subreq->nr_segs);
+			subreq->len = part;
+
 			len -= part;
 			start += part;
 			if (!len) {
@@ -217,9 +232,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
 			stream->sreq_max_len	= umin(len, rreq->rsize);
-			stream->sreq_max_segs	= 0;
-			if (unlikely(stream->sreq_max_segs))
-				part = netfs_limit_iter(&source, 0, part, stream->sreq_max_segs);
+			stream->sreq_max_segs	= INT_MAX;
 
 			netfs_stat(&netfs_n_rh_download);
 			if (rreq->netfs_ops->prepare_read(subreq) < 0) {
@@ -228,11 +241,12 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				goto abandon;
 			}
 
-			part = umin(len, stream->sreq_max_len);
-			subreq->len = subreq->transferred + part;
-			subreq->io_iter = source;
-			iov_iter_truncate(&subreq->io_iter, part);
-			iov_iter_advance(&source, part);
+			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
+			part = bvecq_slice(&dispatch_cursor,
+					   umin(len, stream->sreq_max_len),
+					   stream->sreq_max_segs,
+					   &subreq->nr_segs);
+			subreq->len = part;
 
 			len -= part;
 			start += part;
@@ -246,12 +260,14 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 
 	} while (!list_is_head(next, &stream->subrequests));
 
+out:
+	bvecq_pos_unset(&dispatch_cursor);
 	return;
 
 	/* If we hit an error, fail all remaining incomplete subrequests */
 abandon_after:
 	if (list_is_last(&subreq->rreq_link, &stream->subrequests))
-		return;
+		goto out;
 	subreq = list_next_entry(subreq, rreq_link);
 abandon:
 	list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
@@ -262,6 +278,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
 		__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 	}
+	goto out;
 }
 
 /*
@@ -292,23 +309,24 @@ void netfs_retry_reads(struct netfs_io_request *rreq)
  */
 void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
 {
-	struct folio_queue *p;
-
-	for (p = rreq->buffer.tail; p; p = p->next) {
-		for (int slot = 0; slot < folioq_count(p); slot++) {
-			struct folio *folio = folioq_folio(p, slot);
-
-			if (folio && !folioq_is_marked2(p, slot)) {
-				if (folio == rreq->no_unlock_folio &&
-				    test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO,
-					     &rreq->flags)) {
-					_debug("no unlock");
-				} else {
-					trace_netfs_folio(folio,
-						netfs_folio_trace_abandon);
-					folio_unlock(folio);
-				}
+	struct bvecq *p;
+
+	for (p = rreq->collect_cursor.bvecq; p; p = bvecq_next(p)) {
+		unsigned int nr_slots = bvecq_nr_slots_acquire(p);
+
+		for (int slot = 0; slot < nr_slots; slot++) {
+			if (!p->bv[slot].bv_page)
+				continue;
+
+			struct folio *folio = page_folio(p->bv[slot].bv_page);
+
+			if (folio == rreq->no_unlock_folio &&
+			    test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags)) {
+				_debug("no unlock");
+				continue;
 			}
+			trace_netfs_folio(folio, netfs_folio_trace_abandon);
+			folio_unlock(folio);
 		}
 	}
 }
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index 65ef16ada1bf..cd770703d99f 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -93,7 +93,12 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 	subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
 	subreq->start	= 0;
 	subreq->len	= rreq->len;
-	subreq->io_iter	= rreq->buffer.iter;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &rreq->dispatch_cursor);
+
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
 
 	netfs_queue_read(rreq, subreq);
 
@@ -177,6 +182,15 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	if (IS_ERR(rreq))
 		return PTR_ERR(rreq);
 
+	ret = netfs_extract_iter(iter, rreq->len, INT_MAX, 0, &rreq->dispatch_cursor.bvecq,
+				 0, rreq->gfp);
+	if (ret < 0)
+		goto cleanup_free;
+	if (ret < rreq->len) {
+		ret = -EIO;
+		goto cleanup_free;
+	}
+
 	ret = netfs_single_begin_cache_read(rreq, ictx);
 	if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
 		goto cleanup_free;
@@ -184,7 +198,6 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	netfs_stat(&netfs_n_rh_read_single);
 	trace_netfs_read(rreq, 0, rreq->len, netfs_read_trace_read_single);
 
-	rreq->buffer.iter = *iter;
 	netfs_single_dispatch_read(rreq);
 
 	ret = netfs_wait_for_read(rreq);
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
index a10d34f88597..0ba6ce9295b2 100644
--- a/fs/netfs/stats.c
+++ b/fs/netfs/stats.c
@@ -46,7 +46,6 @@ atomic_t netfs_n_wh_retry_write_req;
 atomic_t netfs_n_wh_retry_write_subreq;
 atomic_t netfs_n_wb_lock_skip;
 atomic_t netfs_n_wb_lock_wait;
-atomic_t netfs_n_folioq;
 atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v)
@@ -89,11 +88,10 @@ int netfs_stats_show(struct seq_file *m, void *v)
 		   atomic_read(&netfs_n_rh_retry_read_subreq),
 		   atomic_read(&netfs_n_wh_retry_write_req),
 		   atomic_read(&netfs_n_wh_retry_write_subreq));
-	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u foq=%u wsc=%u\n",
+	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u wsc=%u\n",
 		   atomic_read(&netfs_n_rh_rreq),
 		   atomic_read(&netfs_n_rh_sreq),
 		   atomic_read(&netfs_n_bvecq),
-		   atomic_read(&netfs_n_folioq),
 		   atomic_read(&netfs_n_wh_wstream_conflict));
 	seq_printf(m, "WbLock : skip=%u wait=%u\n",
 		   atomic_read(&netfs_n_wb_lock_skip),
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 5b96463808d5..0cd145472a17 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -114,12 +114,12 @@ int netfs_folio_written_back(struct folio *folio)
 static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 					  unsigned int *notes)
 {
-	struct folio_queue *folioq = wreq->buffer.tail;
+	struct bvecq *bvecq = wreq->collect_cursor.bvecq;
 	unsigned long long collected_to = wreq->collected_to;
-	unsigned int slot = wreq->buffer.first_tail_slot;
+	unsigned int slot = wreq->collect_cursor.slot;
 
-	if (WARN_ON_ONCE(!folioq)) {
-		pr_err("[!] Writeback unlock found empty rolling buffer!\n");
+	if (WARN_ON_ONCE(!bvecq)) {
+		pr_err("[!] Writeback unlock found empty buffer!\n");
 		netfs_dump_request(wreq);
 		return;
 	}
@@ -130,20 +130,28 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		return;
 	}
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&wreq->buffer);
-		if (!folioq)
-			return;
-		slot = 0;
-	}
-
 	for (;;) {
 		struct folio *folio;
 		struct netfs_folio *finfo;
 		unsigned long long fpos, fend;
 		size_t fsize, flen;
 
-		folio = folioq_folio(folioq, slot);
+		/* Try to clean up the head of the queue if it appears to be
+		 * used up, but we need to be very careful - the cleanup can
+		 * catch the dispatcher, which could lead to us having nothing
+		 * left in the queue, causing the front and back pointers to
+		 * end up on different tracks.  To avoid this, we must always
+		 * keep at least one segment in the queue.
+		 */
+		if (!bvecq_acquire_slot(bvecq, slot)) {
+			wreq->collect_cursor.slot = slot;
+			if (!bvecq_delete_spent(&wreq->collect_cursor))
+				return;
+			bvecq = wreq->collect_cursor.bvecq;
+			slot  = wreq->collect_cursor.slot;
+		}
+
+		folio = page_folio(bvecq->bv[slot].bv_page);
 		if (WARN_ONCE(!folio_test_writeback(folio),
 			      "R=%08x: folio %lx is not under writeback\n",
 			      wreq->debug_id, folio->index))
@@ -166,26 +174,13 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		wreq->cleaned_to = fpos + fsize;
 		*notes |= MADE_PROGRESS;
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
-		 * being filled by the issuer.
-		 */
-		folioq_clear(folioq, slot);
+		bvecq->bv[slot].bv_page = NULL;
 		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&wreq->buffer);
-			if (!folioq)
-				goto done;
-			slot = 0;
-		}
-
 		if (fpos + fsize >= collected_to)
 			break;
 	}
 
-	wreq->buffer.tail = folioq;
-done:
-	wreq->buffer.first_tail_slot = slot;
+	wreq->collect_cursor.slot = slot;
 }
 
 /*
@@ -230,7 +225,8 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 	trace_netfs_rreq(wreq, netfs_rreq_trace_collect);
 
 reassess_streams:
-	issued_to = atomic64_read(&wreq->issued_to);
+	/* Order reading the issued_to point before reading the queue it refers to. */
+	issued_to = atomic64_read_acquire(&wreq->issued_to);
 	smp_rmb();
 	collected_to = ULLONG_MAX;
 	if (wreq->origin == NETFS_WRITEBACK ||
@@ -561,8 +557,12 @@ void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error)
 			 * data is tracked.
 			 */
 			netfs_stat(&netfs_n_wh_write_failed);
-			if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
-				break;
+			if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
+				/* We don't retry failed cache writes. */
+				__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+				if (!subreq->error)
+					subreq->error = -ENOBUFS;
+			}
 
 			trace_netfs_failure(wreq, subreq, transferred_or_error, netfs_fail_write);
 			__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 9d16f63536dc..70a7a92028e4 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -107,8 +107,6 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 	ictx = netfs_inode(wreq->inode);
 	if (is_cacheable)
 		fscache_begin_write_operation(&wreq->cache_resources, netfs_i_cookie(ictx));
-	if (rolling_buffer_init(&wreq->buffer, wreq->debug_id, ITER_SOURCE, wreq->gfp) < 0)
-		goto nomem;
 
 	wreq->cleaned_to = wreq->start;
 	if (wreq->cache_resources.dio_size > 1)
@@ -133,9 +131,6 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 	}
 
 	return wreq;
-nomem:
-	netfs_put_failed_request(wreq);
-	return ERR_PTR(-ENOMEM);
 }
 
 /**
@@ -160,21 +155,13 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 			 loff_t start)
 {
 	struct netfs_io_subrequest *subreq;
-	struct iov_iter *wreq_iter = &wreq->buffer.iter;
-
-	/* Make sure we don't point the iterator at a used-up folio_queue
-	 * struct being used as a placeholder to prevent the queue from
-	 * collapsing.  In such a case, extend the queue.
-	 */
-	if (iov_iter_is_folioq(wreq_iter) &&
-	    wreq_iter->folioq_slot >= folioq_nr_slots(wreq_iter->folioq))
-		rolling_buffer_make_space(&wreq->buffer, wreq->gfp);
 
 	subreq = netfs_alloc_subrequest(wreq);
 	subreq->source		= stream->source;
 	subreq->start		= start;
 	subreq->stream_nr	= stream->stream_nr;
-	subreq->io_iter		= *wreq_iter;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &wreq->dispatch_cursor);
 
 	_enter("R=%x[%x]", wreq->debug_id, subreq->debug_index);
 
@@ -255,15 +242,16 @@ static void netfs_do_issue_write(struct netfs_io_stream *stream,
 }
 
 void netfs_reissue_write(struct netfs_io_stream *stream,
-			 struct netfs_io_subrequest *subreq,
-			 struct iov_iter *source)
+			 struct netfs_io_subrequest *subreq)
 {
-	size_t size = subreq->len - subreq->transferred;
-
 	// TODO: Use encrypted buffer
-	subreq->io_iter = *source;
-	iov_iter_advance(source, size);
-	iov_iter_truncate(&subreq->io_iter, size);
+	bvecq_pos_unset(&subreq->content);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
+			    subreq->content.bvecq, subreq->content.slot,
+			    subreq->content.offset,
+			    subreq->len);
+	iov_iter_advance(&subreq->io_iter, subreq->transferred);
 
 	subreq->retry_count++;
 	subreq->error = 0;
@@ -281,8 +269,68 @@ void netfs_issue_write(struct netfs_io_request *wreq,
 	if (!subreq)
 		return;
 
+	/* If we have a write to the cache, we need to round out the first and
+	 * last entries (only those as the data will be on virtually contiguous
+	 * folios) to cache DIO boundaries.
+	 */
+	if (subreq->source == NETFS_WRITE_TO_CACHE) {
+		struct bvecq_pos tmp_pos;
+		struct bio_vec *bv;
+		struct bvecq *bq;
+		size_t dio_size = wreq->cache_resources.dio_size;
+		size_t disp, len;
+		int ret;
+
+		bvecq_pos_set(&tmp_pos, &subreq->dispatch_pos);
+		ret = bvecq_extract(&tmp_pos, subreq->len, INT_MAX, &subreq->content.bvecq);
+		bvecq_pos_unset(&tmp_pos);
+		if (ret < 0) {
+			netfs_write_subrequest_terminated(subreq, -ENOMEM);
+			stream->construct = NULL;
+			return;
+		}
+
+		/* Round the first entry down.  We should be able to get away
+		 * with this as this path only happens for buffered reads and
+		 * writes.  As such, a bio_vec must always point to a whole
+		 * folio (or part thereof) in the pagecache with writeback set,
+		 * so presuming that dio_size < folio size, we should be able
+		 * to round out bv_offset and bv_len.
+		 *
+		 * Further, streaming-write pages don't get sent to the cache
+		 * (and aren't normally generated if there is a cache), so we
+		 * only see fully uptodate pages here.
+		 */
+		bq = subreq->content.bvecq;
+		bv = &bq->bv[0];
+		disp = bv->bv_offset & (dio_size - 1);
+		if (disp) {
+			bv->bv_offset -= disp;
+			bv->bv_len += disp;
+			bq->fpos -= disp;
+			subreq->start -= disp;
+			subreq->len += disp;
+		}
+
+		/* Round the end of the last entry up. */
+		while (bq->next)
+			bq = bq->next;
+		bv = &bq->bv[bq->nr_slots - 1];
+		len = round_up(bv->bv_len, dio_size);
+		if (len > bv->bv_len) {
+			subreq->len += len - bv->bv_len;
+			bv->bv_len = len;
+		}
+	} else {
+		bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	}
+
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
+			    subreq->content.bvecq, subreq->content.slot,
+			    subreq->content.offset,
+			    subreq->len);
+
 	stream->construct = NULL;
-	subreq->io_iter.count = subreq->len;
 	netfs_do_issue_write(stream, subreq);
 }
 
@@ -319,7 +367,6 @@ size_t netfs_advance_write(struct netfs_io_request *wreq,
 	_debug("part %zx/%zx %zx/%zx", subreq->len, stream->sreq_max_len, part, len);
 	subreq->len += part;
 	subreq->nr_segs++;
-	stream->submit_extendable_to -= part;
 
 	if (subreq->len >= stream->sreq_max_len ||
 	    subreq->nr_segs >= stream->sreq_max_segs ||
@@ -343,7 +390,8 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	struct netfs_io_stream *stream;
 	struct netfs_group *fgroup; /* TODO: Use this with ceph */
 	struct netfs_folio *finfo;
-	size_t iter_off = 0;
+	struct bvecq *queue = wreq->load_cursor.bvecq;
+	unsigned int slot;
 	size_t fsize = folio_size(folio), flen = fsize, foff = 0;
 	loff_t fpos = folio_pos(folio), i_size;
 	bool to_eof = false, streamw = false;
@@ -351,12 +399,20 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 
 	_enter("");
 
-	if (rolling_buffer_make_space(&wreq->buffer, wreq->gfp) < 0)
-		return -ENOMEM;
+	if (!wreq->spare) {
+		wreq->spare = bvecq_alloc_one(BVECQ_STD_SLOTS, wreq->gfp);
+		if (!wreq->spare)
+			return -ENOMEM;
+	}
 
-	/* netfs_perform_write() may shift i_size around the page or from out
-	 * of the page to beyond it, but cannot move i_size into or through the
-	 * page since we have it locked.
+	/* netfs_perform_write() may shift i_size around the folio or from out
+	 * of the folio to beyond it, but cannot move i_size into or through
+	 * the folio since we have it locked.
+	 *
+	 * Truncate could in theory move i_size into or before the folio, but
+	 * it should take steps to prevent writeback from happening
+	 * concurrently and should wait for any in-progress writebacks before
+	 * proceeding.
 	 */
 	i_size = i_size_read(wreq->inode);
 
@@ -448,8 +504,32 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 		trace_netfs_folio(folio, netfs_folio_trace_store_plus);
 	}
 
+	/* Institute a new bvec queue segment if the current one is full or if
+	 * we encounter a discontiguity.  The discontiguity break is important
+	 * when it comes to bulk unlocking folios by file range.
+	 */
+	if (bvecq_is_full(queue) ||
+	    (fpos != wreq->last_end && wreq->last_end > 0)) {
+		bvecq_buffer_append(&wreq->load_cursor, wreq->spare);
+		wreq->spare = NULL;
+
+		queue = wreq->load_cursor.bvecq;
+		queue->fpos = fpos;
+		if (fpos != wreq->last_end)
+			queue->discontig = true;
+		bvecq_pos_move(&wreq->dispatch_cursor, queue);
+		wreq->dispatch_cursor.slot = 0;
+	}
+
 	/* Attach the folio to the rolling buffer. */
-	rolling_buffer_append(&wreq->buffer, folio, 0, wreq->gfp);
+	slot = queue->nr_slots;
+	bvec_set_folio(&queue->bv[slot], folio, flen, foff);
+	trace_netfs_bv_slot(queue, slot);
+	slot++;
+	bvecq_filled_to(queue, slot);
+	wreq->load_cursor.slot = slot;
+	wreq->load_cursor.offset = 0;
+	wreq->last_end = fpos + foff + flen;
 
 	/* Move the submission point forward to allow for write-streaming data
 	 * not starting at the front of the page.  We don't do write-streaming
@@ -458,9 +538,11 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	 * Also skip uploading for data that's been read and just needs copying
 	 * to the cache.
 	 */
+	bvecq_pos_nudge(&wreq->dispatch_cursor);
+	
 	for (int s = 0; s < NR_IO_STREAMS; s++) {
 		stream = &wreq->io_streams[s];
-		stream->submit_off = foff;
+		stream->submit_off = 0;
 		stream->submit_len = flen;
 		if (!stream->avail ||
 		    (stream->source == NETFS_WRITE_TO_CACHE && streamw) ||
@@ -495,15 +577,11 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 			break;
 		stream = &wreq->io_streams[choose_s];
 
-		/* Advance the iterator(s). */
-		if (stream->submit_off > iter_off) {
-			rolling_buffer_advance(&wreq->buffer, stream->submit_off - iter_off);
-			iter_off = stream->submit_off;
-		}
+		/* Advance the cursor. */
+		wreq->dispatch_cursor.offset = stream->submit_off;
 
-		atomic64_set(&wreq->issued_to, fpos + stream->submit_off);
-		stream->submit_extendable_to = fsize - stream->submit_off;
-		part = netfs_advance_write(wreq, stream, fpos + stream->submit_off,
+		atomic64_set(&wreq->issued_to, fpos + foff + stream->submit_off);
+		part = netfs_advance_write(wreq, stream, fpos + foff + stream->submit_off,
 					   stream->submit_len, to_eof);
 		stream->submit_off += part;
 		if (part > stream->submit_len)
@@ -514,9 +592,9 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 			debug = true;
 	}
 
-	if (fsize > iter_off)
-		rolling_buffer_advance(&wreq->buffer, fsize - iter_off);
-	atomic64_set(&wreq->issued_to, fpos + fsize);
+	bvecq_pos_step(&wreq->dispatch_cursor);
+	/* Order loading the queue before updating the issue_to point */
+	atomic64_set_release(&wreq->issued_to, fpos + fsize);
 
 	if (!debug)
 		kdebug("R=%x: No submit", wreq->debug_id);
@@ -578,6 +656,11 @@ int netfs_writepages(struct address_space *mapping,
 		goto couldnt_start;
 	}
 
+	if (bvecq_buffer_init(&wreq->load_cursor, wreq->gfp) < 0)
+		goto nomem;
+	bvecq_pos_set(&wreq->dispatch_cursor, &wreq->load_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback);
 	netfs_stat(&netfs_n_wh_writepages);
@@ -602,12 +685,17 @@ int netfs_writepages(struct address_space *mapping,
 	} while ((folio = writeback_iter(mapping, wbc, folio, &error)));
 
 	netfs_end_issue_write(wreq);
+	bvecq_pos_unset(&wreq->load_cursor);
+	bvecq_pos_unset(&wreq->dispatch_cursor);
 	netfs_wake_collector(wreq);
 
 	netfs_put_request(wreq, netfs_rreq_trace_put_return);
 	_leave(" = %d", error);
 	return error;
 
+nomem:
+	error = -ENOMEM;
+	netfs_put_failed_request(wreq);
 couldnt_start:
 	if (error == -ENOMEM) {
 		folio_redirty_for_writepage(wbc, folio);
@@ -631,10 +719,11 @@ EXPORT_SYMBOL(netfs_writepages);
  * @iter: Data to write.
  *
  * Write a monolithic, non-pagecache object back to the server and/or
- * the cache.
+ * the cache.  There's a maximum of one subrequest per stream.
  *
  * Return: 0 if successful; 1 if skipped due to lock conflict and WB_SYNC_NONE;
  * or a negative error code.
+ * the cache.  There's a maximum of one subrequest per stream.
  */
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
@@ -655,10 +744,19 @@ int netfs_writeback_single(struct address_space *mapping,
 		ret = PTR_ERR(wreq);
 		goto couldnt_start;
 	}
-
-	wreq->buffer.iter = *iter;
 	wreq->len = iov_iter_count(iter);
 
+	ret = netfs_extract_iter(iter, wreq->len, INT_MAX, 0, &wreq->dispatch_cursor.bvecq,
+				 0, wreq->gfp);
+	if (ret < 0)
+		goto cleanup_free;
+	if (ret < wreq->len) {
+		ret = -EIO;
+		goto cleanup_free;
+	}
+
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback_single);
 	netfs_stat(&netfs_n_wh_writepages);
@@ -678,11 +776,11 @@ int netfs_writeback_single(struct address_space *mapping,
 		subreq = stream->construct;
 		subreq->len = wreq->len;
 		stream->submit_len = subreq->len;
-		stream->submit_extendable_to = round_up(wreq->len, PAGE_SIZE);
 
 		netfs_issue_write(wreq, stream);
 	}
 
+	wreq->submitted = wreq->len;
 	smp_wmb(); /* Write lists before ALL_QUEUED. */
 	set_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags);
 
@@ -697,6 +795,8 @@ int netfs_writeback_single(struct address_space *mapping,
 	_leave(" = %d", ret);
 	return ret;
 
+cleanup_free:
+	netfs_put_failed_request(wreq);
 couldnt_start:
 	netfs_wb_end(ictx);
 	_leave(" = %d", ret);
diff --git a/fs/netfs/write_retry.c b/fs/netfs/write_retry.c
index 508dbcbb3b19..4e320591f83a 100644
--- a/fs/netfs/write_retry.c
+++ b/fs/netfs/write_retry.c
@@ -17,15 +17,17 @@
 static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				     struct netfs_io_stream *stream)
 {
+	struct bvecq_pos dispatch_cursor = {};
 	struct list_head *next;
 
 	_enter("R=%x[%x:]", wreq->debug_id, stream->stream_nr);
 
 	if (list_empty(&stream->subrequests))
 		return;
+	if (WARN_ON_ONCE(stream->source != NETFS_UPLOAD_TO_SERVER))
+		return; /* Shouldn't be retrying cache writes. */
 
-	if (stream->source == NETFS_UPLOAD_TO_SERVER &&
-	    wreq->netfs_ops->retry_request)
+	if (wreq->netfs_ops->retry_request)
 		wreq->netfs_ops->retry_request(wreq, stream);
 
 	if (unlikely(stream->failed))
@@ -39,12 +41,8 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
 				break;
 			if (__test_and_clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-				struct iov_iter source;
-
-				netfs_reset_iter(subreq);
-				source = subreq->io_iter;
 				netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-				netfs_reissue_write(stream, subreq, &source);
+				netfs_reissue_write(stream, subreq);
 			}
 		}
 		return;
@@ -54,11 +52,12 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 
 	do {
 		struct netfs_io_subrequest *subreq = NULL, *from, *to, *tmp;
-		struct iov_iter source;
 		unsigned long long start, len;
 		size_t part;
 		bool boundary = false;
 
+		bvecq_pos_unset(&dispatch_cursor);
+
 		/* Go through the stream and find the next span of contiguous
 		 * data that we then rejig (cifs, for example, needs the wsize
 		 * renegotiating) and reissue.
@@ -70,7 +69,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 
 		if (test_bit(NETFS_SREQ_FAILED, &from->flags) ||
 		    !test_bit(NETFS_SREQ_NEED_RETRY, &from->flags))
-			return;
+			goto out;
 
 		for (;;) {
 			/* Read pointer to subreq before reading subreq state. */
@@ -79,7 +78,8 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				break;
 
 			subreq = list_entry(next, struct netfs_io_subrequest, rreq_link);
-			if (subreq->start + subreq->transferred != start + len ||
+			if (subreq->start != start + len ||
+			    subreq->transferred > 0 ||
 			    test_bit(NETFS_SREQ_BOUNDARY, &subreq->flags) ||
 			    !test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
 				break;
@@ -90,11 +90,13 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 		/* Determine the set of buffers we're going to use.  Each
 		 * subreq gets a subset of a single overall contiguous buffer.
 		 */
-		netfs_reset_iter(from);
-		source = from->io_iter;
-		source.count = len;
+		bvecq_pos_transfer(&dispatch_cursor, &from->dispatch_pos);
+		bvecq_pos_advance(&dispatch_cursor, from->transferred);
 
-		/* Work through the sublist. */
+		/* Work through the sublist.  The chain of buffers we're going
+		 * to fill is attached to dispatch_cursor and we need to read
+		 * 'len' amount of data from 'start'.
+		 */
 		subreq = from;
 		list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
 			if (!len)
@@ -104,16 +106,23 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			subreq->len	= len;
 			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+			subreq->transferred = 0;
+
+			bvecq_pos_unset(&subreq->content);
+			bvecq_pos_unset(&subreq->dispatch_pos);
 
 			/* Renegotiate max_len (wsize) */
 			stream->sreq_max_len = len;
+			stream->sreq_max_segs = INT_MAX;
 			stream->prepare_write(subreq);
 
-			part = umin(len, stream->sreq_max_len);
-			if (unlikely(stream->sreq_max_segs))
-				part = netfs_limit_iter(&source, 0, part, stream->sreq_max_segs);
+			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
+			part = bvecq_slice(&dispatch_cursor,
+					   umin(len, stream->sreq_max_len),
+					   stream->sreq_max_segs,
+					   &subreq->nr_segs);
 			subreq->len = part;
-			subreq->transferred = 0;
+
 			len -= part;
 			start += part;
 			if (len && subreq == to &&
@@ -121,7 +130,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				boundary = true;
 
 			netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-			netfs_reissue_write(stream, subreq, &source);
+			netfs_reissue_write(stream, subreq);
 			if (subreq == to)
 				break;
 		}
@@ -173,17 +182,19 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				netfs_stat(&netfs_n_wh_upload);
 				stream->sreq_max_len = umin(len, wreq->wsize);
 				break;
-			case NETFS_WRITE_TO_CACHE:
-				netfs_stat(&netfs_n_wh_write);
-				break;
 			default:
 				WARN_ON_ONCE(1);
 			}
 
 			stream->prepare_write(subreq);
 
-			part = umin(len, stream->sreq_max_len);
+			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
+			part = bvecq_slice(&dispatch_cursor,
+					   umin(len, stream->sreq_max_len),
+					   stream->sreq_max_segs,
+					   &subreq->nr_segs);
 			subreq->len = subreq->transferred + part;
+
 			len -= part;
 			start += part;
 			if (!len && boundary) {
@@ -191,13 +202,16 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				boundary = false;
 			}
 
-			netfs_reissue_write(stream, subreq, &source);
+			netfs_reissue_write(stream, subreq);
 			if (!len)
 				break;
 
 		} while (len);
 
 	} while (!list_is_head(next, &stream->subrequests));
+
+out:
+	bvecq_pos_unset(&dispatch_cursor);
 }
 
 /*
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 09e81475e899..6b43978bfcda 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -19,12 +19,13 @@
 #include <linux/pagemap.h>
 #include <linux/bvecq.h>
 #include <linux/uio.h>
-#include <linux/rolling_buffer.h>
 
 enum netfs_sreq_ref_trace;
 typedef struct mempool mempool_t;
+struct readahead_control;
+struct netfs_io_request;
+struct netfs_io_subrequest;
 struct fscache_occupancy;
-struct folio_queue;
 
 /**
  * folio_start_private_2 - Start an fscache write on a folio.  [DEPRECATED]
@@ -139,7 +140,6 @@ struct netfs_io_stream {
 	unsigned int		sreq_max_segs;	/* 0 or max number of segments in an iterator */
 	unsigned int		submit_off;	/* Folio offset we're submitting from */
 	unsigned int		submit_len;	/* Amount of data left to submit */
-	unsigned int		submit_extendable_to; /* Amount I/O can be rounded up to */
 	void (*prepare_write)(struct netfs_io_subrequest *subreq);
 	void (*issue_write)(struct netfs_io_subrequest *subreq);
 	/* Collection tracking */
@@ -182,6 +182,8 @@ struct netfs_io_subrequest {
 	struct netfs_io_request *rreq;		/* Supervising I/O request */
 	struct work_struct	work;
 	struct list_head	rreq_link;	/* Link in rreq->subrequests */
+	struct bvecq_pos	dispatch_pos;	/* Bookmark in the combined queue of the start */
+	struct bvecq_pos	content;	/* The (copied) content of the subrequest */
 	struct iov_iter		io_iter;	/* Iterator for this subrequest */
 	unsigned long long	start;		/* Where to start the I/O */
 	size_t			len;		/* Size of the I/O */
@@ -242,13 +244,14 @@ struct netfs_io_request {
 	struct netfs_io_stream	io_streams[2];	/* Streams of parallel I/O operations */
 #define NR_IO_STREAMS 2 //wreq->nr_io_streams
 	struct netfs_group	*group;		/* Writeback group being written back */
-	struct rolling_buffer	buffer;		/* Unencrypted buffer */
-#define NETFS_ROLLBUF_PUT_MARK		ROLLBUF_MARK_1
-#define NETFS_ROLLBUF_PAGECACHE_MARK	ROLLBUF_MARK_2
+	struct bvecq		*spare;		/* Advance allocation of bvecq */
+	struct bvecq_pos	load_cursor;	/* Point at which new folios are loaded in */
+	struct bvecq_pos	dispatch_cursor; /* Point from which buffers are dispatched */
+	struct bvecq_pos	collect_cursor;	/* Clear-up point of I/O buffer */
 	wait_queue_head_t	waitq;		/* Processor waiter */
 	void			*netfs_priv;	/* Private data for the netfs */
 	void			*netfs_priv2;	/* Private data for the netfs */
-	struct bio_vec		*direct_bv;	/* DIO buffer list (when handling iovec-iter) */
+	unsigned long long	last_end;	/* End pos of last folio submitted */
 	unsigned long long	submitted;	/* Amount submitted for I/O so far */
 	unsigned long long	len;		/* Length of the request */
 	size_t			transferred;	/* Amount to be indicated as transferred */
@@ -262,7 +265,6 @@ struct netfs_io_request {
 	unsigned long long	abandon_to;	/* Position to abandon folios to */
 	const struct folio	*no_unlock_folio; /* Don't unlock this folio after read */
 	gfp_t			gfp;		/* GFP flags to use */
-	unsigned int		direct_bv_count; /* Number of elements in direct_bv[] */
 	unsigned int		debug_id;
 	unsigned int		rsize;		/* Maximum read size (0 for none) */
 	unsigned int		wsize;		/* Maximum write size (0 for none) */
@@ -271,7 +273,6 @@ struct netfs_io_request {
 	spinlock_t		lock;		/* Lock for queuing subreqs */
 	unsigned char		front_folio_order; /* Order (size) of front folio */
 	enum netfs_io_origin	origin;		/* Origin of the request */
-	bool			direct_bv_unpin; /* T if direct_bv[] must be unpinned */
 	refcount_t		ref;
 	unsigned long		flags;
 #define NETFS_RREQ_IN_PROGRESS		0	/* Unlocked when the request completes (has ref) */
@@ -471,12 +472,6 @@ void netfs_end_io_write(struct inode *inode);
 int netfs_start_io_direct(struct inode *inode);
 void netfs_end_io_direct(struct inode *inode);
 
-/* Miscellaneous APIs. */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
-				       unsigned int trace /*enum netfs_folioq_trace*/);
-void netfs_folioq_free(struct folio_queue *folioq,
-		       unsigned int trace /*enum netfs_trace_folioq*/);
-
 /* Buffer wrangling helpers API. */
 int netfs_alloc_folioq_buffer(struct address_space *mapping,
 			      struct folio_queue **_buffer,
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 5dc61a4ba5f0..071d20e80f13 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -226,7 +226,9 @@
 	EM(netfs_folio_trace_redirtied,		"redirtied")	\
 	EM(netfs_folio_trace_store,		"store")	\
 	EM(netfs_folio_trace_store_copy,	"store-copy")	\
-	E_(netfs_folio_trace_store_plus,	"store+")
+	EM(netfs_folio_trace_store_plus,	"store+")	\
+	EM(netfs_folio_trace_zero,		"zero")		\
+	E_(netfs_folio_trace_zero_ra,		"zero-ra")
 
 #define netfs_collect_contig_traces				\
 	EM(netfs_contig_trace_collect,		"Collect")	\
@@ -239,13 +241,13 @@
 	EM(netfs_trace_donate_to_next,		"to-next")	\
 	E_(netfs_trace_donate_to_deferred_next,	"defer-next")
 
-#define netfs_folioq_traces					\
-	EM(netfs_trace_folioq_alloc_buffer,	"alloc-buf")	\
-	EM(netfs_trace_folioq_clear,		"clear")	\
-	EM(netfs_trace_folioq_delete,		"delete")	\
-	EM(netfs_trace_folioq_make_space,	"make-space")	\
-	EM(netfs_trace_folioq_rollbuf_init,	"roll-init")	\
-	E_(netfs_trace_folioq_read_progress,	"r-progress")
+#define netfs_bvecq_traces					\
+	EM(netfs_trace_bvecq_alloc_buffer,	"alloc-buf")	\
+	EM(netfs_trace_bvecq_clear,		"clear")	\
+	EM(netfs_trace_bvecq_delete,		"delete")	\
+	EM(netfs_trace_bvecq_make_space,	"make-space")	\
+	EM(netfs_trace_bvecq_rollbuf_init,	"roll-init")	\
+	E_(netfs_trace_bvecq_read_progress,	"r-progress")
 
 #ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
 #define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
@@ -265,7 +267,7 @@ enum netfs_sreq_ref_trace { netfs_sreq_ref_traces } __mode(byte);
 enum netfs_folio_trace { netfs_folio_traces } __mode(byte);
 enum netfs_collect_contig_trace { netfs_collect_contig_traces } __mode(byte);
 enum netfs_donate_trace { netfs_donate_traces } __mode(byte);
-enum netfs_folioq_trace { netfs_folioq_traces } __mode(byte);
+enum netfs_bvecq_trace { netfs_bvecq_traces } __mode(byte);
 
 #endif
 
@@ -289,7 +291,7 @@ netfs_sreq_ref_traces;
 netfs_folio_traces;
 netfs_collect_contig_traces;
 netfs_donate_traces;
-netfs_folioq_traces;
+netfs_bvecq_traces;
 
 /*
  * Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -393,10 +395,10 @@ TRACE_EVENT(netfs_sreq,
 		    __entry->len	= sreq->len;
 		    __entry->transferred = sreq->transferred;
 		    __entry->start	= sreq->start;
-		    __entry->slot	= sreq->io_iter.folioq_slot;
+		    __entry->slot	= sreq->dispatch_pos.slot;
 			   ),
 
-	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx s=%u e=%d",
+	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx qs=%u e=%d",
 		      __entry->rreq, __entry->index,
 		      __print_symbolic(__entry->source, netfs_sreq_sources),
 		      __print_symbolic(__entry->what, netfs_sreq_traces),
@@ -772,27 +774,25 @@ TRACE_EVENT(netfs_collect_stream,
 		      __entry->collected_to, __entry->issued_to)
 	    );
 
-TRACE_EVENT(netfs_folioq,
-	    TP_PROTO(const struct folio_queue *fq,
-		     enum netfs_folioq_trace trace),
+TRACE_EVENT(netfs_bvecq,
+	    TP_PROTO(const struct bvecq *bq,
+		     enum netfs_bvecq_trace trace),
 
-	    TP_ARGS(fq, trace),
+	    TP_ARGS(bq, trace),
 
 	    TP_STRUCT__entry(
-		    __field(unsigned int,		rreq)
 		    __field(unsigned int,		id)
-		    __field(enum netfs_folioq_trace,	trace)
+		    __field(enum netfs_bvecq_trace,	trace)
 			     ),
 
 	    TP_fast_assign(
-		    __entry->rreq	= fq ? fq->rreq_id : 0;
-		    __entry->id		= fq ? fq->debug_id : 0;
+		    __entry->id		= bq ? bq->priv : 0;
 		    __entry->trace	= trace;
 			   ),
 
-	    TP_printk("R=%08x fq=%x %s",
-		      __entry->rreq, __entry->id,
-		      __print_symbolic(__entry->trace, netfs_folioq_traces))
+	    TP_printk("fq=%x %s",
+		      __entry->id,
+		      __print_symbolic(__entry->trace, netfs_bvecq_traces))
 	    );
 
 TRACE_EVENT(netfs_bv_slot,



From gregkh at linuxfoundation.org  Wed Aug  5 02:50:45 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Wed, 05 Aug 2026 11:50:45 +0200
Subject: Patch "afs: Fix afs_fs_fetch_data() to subtract transferred from len" has been added to the 6.18-stable tree
Message-ID: <2026080545-gibberish-fondling-4abb@gregkh>


This is a note to let you know that I've just added the patch titled

    afs: Fix afs_fs_fetch_data() to subtract transferred from len

to the 6.18-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     afs-fix-afs_fs_fetch_data-to-subtract-transferred-from-len.patch
and it can be found in the queue-6.18 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From 222052c6be186f2074b3a4d741d5de200f654c43 Mon Sep 17 00:00:00 2001
From: David Howells <dhowells at redhat.com>
Date: Thu, 23 Jul 2026 12:34:47 +0100
Subject: afs: Fix afs_fs_fetch_data() to subtract transferred from len

From: David Howells <dhowells at redhat.com>

commit 222052c6be186f2074b3a4d741d5de200f654c43 upstream.

Fix afs_fs_fetch_data() to subtract subreq->transferred from subreq->len
rather than adding it.

Fixes: f28fc2010d62 ("afs: Eliminate afs_read")
Link: https://sashiko.dev/#/patchset/20260713081022.2186481-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-3-dhowells at redhat.com
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/fsclient.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 626e1d37b915..1a3f186a6a11 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -487,7 +487,7 @@ void afs_fs_fetch_data(struct afs_operation *op)
 	bp[2] = htonl(vp->fid.vnode);
 	bp[3] = htonl(vp->fid.unique);
 	bp[4] = htonl(lower_32_bits(subreq->start + subreq->transferred));
-	bp[5] = htonl(lower_32_bits(subreq->len   + subreq->transferred));
+	bp[5] = htonl(lower_32_bits(subreq->len   - subreq->transferred));
 
 	call->fid = vp->fid;
 	trace_afs_make_fs_call(call, &vp->fid);
-- 
2.55.0



Patches currently in stable-queue which might be from dhowells at redhat.com are

queue-6.18/netfs-release-readahead-folios-on-iterator-preparati.patch
queue-6.18/netfs-handle-single-writeback-rolling-buffer-allocat.patch
queue-6.18/netfs-clear-pg_private_2-on-copy-to-cache-append-fai.patch
queue-6.18/afs-fix-uaf-when-sending-a-message.patch
queue-6.18/afs-fix-afs_fs_fetch_data-to-set-call-async.patch
queue-6.18/afs-fix-afs_fs_fetch_data-to-subtract-transferred-from-len.patch


From gregkh at linuxfoundation.org  Wed Aug  5 02:50:45 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Wed, 05 Aug 2026 11:50:45 +0200
Subject: Patch "afs: Fix UAF when sending a message" has been added to the 6.18-stable tree
Message-ID: <2026080545-subtype-dab-4bed@gregkh>


This is a note to let you know that I've just added the patch titled

    afs: Fix UAF when sending a message

to the 6.18-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     afs-fix-uaf-when-sending-a-message.patch
and it can be found in the queue-6.18 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From 4af1ec68d54b3871155914d584fb10669c41a861 Mon Sep 17 00:00:00 2001
From: David Howells <dhowells at redhat.com>
Date: Thu, 23 Jul 2026 12:34:48 +0100
Subject: afs: Fix UAF when sending a message

From: David Howells <dhowells at redhat.com>

commit 4af1ec68d54b3871155914d584fb10669c41a861 upstream.

In afs_make_call(), there's a race with async call reception and
destruction.  If a call is dispatched that doesn't have call->write_iter
set (used to specify the data content for FS.StoreData), then the first
rxrpc_kernel_send_data() will not set MSG_MORE in the msghdr.

Once rxrpc_send_data() queues the last request packet, the response could
come in at any time and cause the call to be completed and put.  However,
afs_make_call() will look at the call again to see it ->write_iter should
be handled - something it's only allowed to do if it has its own ref on the
call.  Whilst this is the case for synchronous calls, it isn't true for
async calls such as FS.FetchData.

There's also a potential UAF in afs_make_call() in the event that an
asynchronous call is being sent, but the call fails in some way (e.g. it
gets aborted from the server).  The problem there is that afs_make_call()
tries to abort a call if the rxrpc send fails, but the asynchronous
notification from rxrpc may have caused the afs_call to be torn down.

generic/650 plays games with randomly taking CPUs offline, and can
interject a significant delay such that the call is deallocated before
afs_make_call() gets to check call->write_iter - and a UAF ensues (caught
by KASAN).

   BUG: KASAN: slab-use-after-free in afs_make_call+0x1c90/0x2210 [kafs]
   Read of size 8 at addr ffff888035e050e8 by task fsstress/1409

Fix this by making afs_make_op_call() give the op->call its own ref rather
than transferring the caller's ref to it and then dropping the ref when
afs_make_call() returns.

This also means that the afs_make_call() func never loses its ref on the
call now.

Fixes: eddf51f2bb2c ("afs: Make {Y,}FS.FetchData an asynchronous operation")
Fixes: e49c7b2f6de7 ("afs: Build an abstraction around an "operation" concept")
Link: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Reported-by: Marc Dionne <marc.dionne at auristor.com>
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-4-dhowells at redhat.com
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/internal.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -1420,7 +1420,7 @@ static inline void afs_make_op_call(stru
 {
 	struct afs_addr_list *alist = op->estate->addresses;
 
-	op->call	= call;
+	op->call	= afs_get_call(call, afs_call_trace_get);
 	op->type	= call->type;
 	call->op	= op;
 	call->key	= op->key;
@@ -1428,6 +1428,7 @@ static inline void afs_make_op_call(stru
 	call->peer	= rxrpc_kernel_get_peer(alist->addrs[op->addr_index].peer);
 	call->service_id = op->server->service_id;
 	afs_make_call(call, gfp);
+	afs_put_call(call);
 }
 
 static inline void afs_extract_begin(struct afs_call *call, void *buf, size_t size)


Patches currently in stable-queue which might be from dhowells at redhat.com are

queue-6.18/netfs-release-readahead-folios-on-iterator-preparati.patch
queue-6.18/netfs-handle-single-writeback-rolling-buffer-allocat.patch
queue-6.18/netfs-clear-pg_private_2-on-copy-to-cache-append-fai.patch
queue-6.18/afs-fix-uaf-when-sending-a-message.patch
queue-6.18/afs-fix-afs_fs_fetch_data-to-set-call-async.patch
queue-6.18/afs-fix-afs_fs_fetch_data-to-subtract-transferred-from-len.patch


From gregkh at linuxfoundation.org  Wed Aug  5 02:50:45 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Wed, 05 Aug 2026 11:50:45 +0200
Subject: Patch "afs: Fix afs_fs_fetch_data() to set call->async" has been added to the 6.18-stable tree
Message-ID: <2026080545-strut-doing-a38a@gregkh>


This is a note to let you know that I've just added the patch titled

    afs: Fix afs_fs_fetch_data() to set call->async

to the 6.18-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     afs-fix-afs_fs_fetch_data-to-set-call-async.patch
and it can be found in the queue-6.18 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From d568a43f6dbba3ba006304d95fd09862bd482a2f Mon Sep 17 00:00:00 2001
From: David Howells <dhowells at redhat.com>
Date: Thu, 23 Jul 2026 12:34:46 +0100
Subject: afs: Fix afs_fs_fetch_data() to set call->async

From: David Howells <dhowells at redhat.com>

commit d568a43f6dbba3ba006304d95fd09862bd482a2f upstream.

Fix afs_fs_fetch_data() to set call->async on an async operation as does
afs_fs_fetch_data64().

Fixes: eddf51f2bb2c ("afs: Make {Y,}FS.FetchData an asynchronous operation")
Link: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-2-dhowells at redhat.com
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/fsclient.c |    3 +++
 1 file changed, 3 insertions(+)

--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -477,6 +477,9 @@ void afs_fs_fetch_data(struct afs_operat
 	if (!call)
 		return afs_op_nomem(op);
 
+	if (op->flags & AFS_OPERATION_ASYNC)
+		call->async = true;
+
 	/* marshall the parameters */
 	bp = call->request;
 	bp[0] = htonl(FSFETCHDATA);


Patches currently in stable-queue which might be from dhowells at redhat.com are

queue-6.18/netfs-release-readahead-folios-on-iterator-preparati.patch
queue-6.18/netfs-handle-single-writeback-rolling-buffer-allocat.patch
queue-6.18/netfs-clear-pg_private_2-on-copy-to-cache-append-fai.patch
queue-6.18/afs-fix-uaf-when-sending-a-message.patch
queue-6.18/afs-fix-afs_fs_fetch_data-to-set-call-async.patch
queue-6.18/afs-fix-afs_fs_fetch_data-to-subtract-transferred-from-len.patch


From gregkh at linuxfoundation.org  Wed Aug  5 02:50:58 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Wed, 05 Aug 2026 11:50:58 +0200
Subject: Patch "afs: Fix afs_fs_fetch_data() to subtract transferred from len" has been added to the 7.1-stable tree
Message-ID: <2026080558-untrimmed-taco-787d@gregkh>


This is a note to let you know that I've just added the patch titled

    afs: Fix afs_fs_fetch_data() to subtract transferred from len

to the 7.1-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     afs-fix-afs_fs_fetch_data-to-subtract-transferred-from-len.patch
and it can be found in the queue-7.1 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From 222052c6be186f2074b3a4d741d5de200f654c43 Mon Sep 17 00:00:00 2001
From: David Howells <dhowells at redhat.com>
Date: Thu, 23 Jul 2026 12:34:47 +0100
Subject: afs: Fix afs_fs_fetch_data() to subtract transferred from len

From: David Howells <dhowells at redhat.com>

commit 222052c6be186f2074b3a4d741d5de200f654c43 upstream.

Fix afs_fs_fetch_data() to subtract subreq->transferred from subreq->len
rather than adding it.

Fixes: f28fc2010d62 ("afs: Eliminate afs_read")
Link: https://sashiko.dev/#/patchset/20260713081022.2186481-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-3-dhowells at redhat.com
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/fsclient.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -487,7 +487,7 @@ void afs_fs_fetch_data(struct afs_operat
 	bp[2] = htonl(vp->fid.vnode);
 	bp[3] = htonl(vp->fid.unique);
 	bp[4] = htonl(lower_32_bits(subreq->start + subreq->transferred));
-	bp[5] = htonl(lower_32_bits(subreq->len   + subreq->transferred));
+	bp[5] = htonl(lower_32_bits(subreq->len   - subreq->transferred));
 
 	call->fid = vp->fid;
 	trace_afs_make_fs_call(call, &vp->fid);


Patches currently in stable-queue which might be from dhowells at redhat.com are

queue-7.1/netfs-fix-folio_queue-enomem-in-writeback-by-adding-.patch
queue-7.1/netfs-release-readahead-folios-on-iterator-preparati.patch
queue-7.1/netfs-handle-single-writeback-rolling-buffer-allocat.patch
queue-7.1/netfs-clear-pg_private_2-on-copy-to-cache-append-fai.patch
queue-7.1/afs-fix-uaf-when-sending-a-message.patch
queue-7.1/afs-fix-afs_fs_fetch_data-to-set-call-async.patch
queue-7.1/afs-fix-afs_fs_fetch_data-to-subtract-transferred-from-len.patch


From gregkh at linuxfoundation.org  Wed Aug  5 02:50:58 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Wed, 05 Aug 2026 11:50:58 +0200
Subject: Patch "afs: Fix UAF when sending a message" has been added to the 7.1-stable tree
Message-ID: <2026080558-surfacing-faster-b2d6@gregkh>


This is a note to let you know that I've just added the patch titled

    afs: Fix UAF when sending a message

to the 7.1-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     afs-fix-uaf-when-sending-a-message.patch
and it can be found in the queue-7.1 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From 4af1ec68d54b3871155914d584fb10669c41a861 Mon Sep 17 00:00:00 2001
From: David Howells <dhowells at redhat.com>
Date: Thu, 23 Jul 2026 12:34:48 +0100
Subject: afs: Fix UAF when sending a message

From: David Howells <dhowells at redhat.com>

commit 4af1ec68d54b3871155914d584fb10669c41a861 upstream.

In afs_make_call(), there's a race with async call reception and
destruction.  If a call is dispatched that doesn't have call->write_iter
set (used to specify the data content for FS.StoreData), then the first
rxrpc_kernel_send_data() will not set MSG_MORE in the msghdr.

Once rxrpc_send_data() queues the last request packet, the response could
come in at any time and cause the call to be completed and put.  However,
afs_make_call() will look at the call again to see it ->write_iter should
be handled - something it's only allowed to do if it has its own ref on the
call.  Whilst this is the case for synchronous calls, it isn't true for
async calls such as FS.FetchData.

There's also a potential UAF in afs_make_call() in the event that an
asynchronous call is being sent, but the call fails in some way (e.g. it
gets aborted from the server).  The problem there is that afs_make_call()
tries to abort a call if the rxrpc send fails, but the asynchronous
notification from rxrpc may have caused the afs_call to be torn down.

generic/650 plays games with randomly taking CPUs offline, and can
interject a significant delay such that the call is deallocated before
afs_make_call() gets to check call->write_iter - and a UAF ensues (caught
by KASAN).

   BUG: KASAN: slab-use-after-free in afs_make_call+0x1c90/0x2210 [kafs]
   Read of size 8 at addr ffff888035e050e8 by task fsstress/1409

Fix this by making afs_make_op_call() give the op->call its own ref rather
than transferring the caller's ref to it and then dropping the ref when
afs_make_call() returns.

This also means that the afs_make_call() func never loses its ref on the
call now.

Fixes: eddf51f2bb2c ("afs: Make {Y,}FS.FetchData an asynchronous operation")
Fixes: e49c7b2f6de7 ("afs: Build an abstraction around an "operation" concept")
Link: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Reported-by: Marc Dionne <marc.dionne at auristor.com>
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-4-dhowells at redhat.com
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/internal.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -1421,7 +1421,7 @@ static inline void afs_make_op_call(stru
 {
 	struct afs_addr_list *alist = op->estate->addresses;
 
-	op->call	= call;
+	op->call	= afs_get_call(call, afs_call_trace_get);
 	op->type	= call->type;
 	call->op	= op;
 	call->key	= op->key;
@@ -1429,6 +1429,7 @@ static inline void afs_make_op_call(stru
 	call->peer	= rxrpc_kernel_get_peer(alist->addrs[op->addr_index].peer);
 	call->service_id = op->server->service_id;
 	afs_make_call(call, gfp);
+	afs_put_call(call);
 }
 
 static inline void afs_extract_begin(struct afs_call *call, void *buf, size_t size)


Patches currently in stable-queue which might be from dhowells at redhat.com are

queue-7.1/netfs-fix-folio_queue-enomem-in-writeback-by-adding-.patch
queue-7.1/netfs-release-readahead-folios-on-iterator-preparati.patch
queue-7.1/netfs-handle-single-writeback-rolling-buffer-allocat.patch
queue-7.1/netfs-clear-pg_private_2-on-copy-to-cache-append-fai.patch
queue-7.1/afs-fix-uaf-when-sending-a-message.patch
queue-7.1/afs-fix-afs_fs_fetch_data-to-set-call-async.patch
queue-7.1/afs-fix-afs_fs_fetch_data-to-subtract-transferred-from-len.patch


From gregkh at linuxfoundation.org  Wed Aug  5 02:50:58 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Wed, 05 Aug 2026 11:50:58 +0200
Subject: Patch "afs: Fix afs_fs_fetch_data() to set call->async" has been added to the 7.1-stable tree
Message-ID: <2026080558-helper-corny-fab9@gregkh>


This is a note to let you know that I've just added the patch titled

    afs: Fix afs_fs_fetch_data() to set call->async

to the 7.1-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     afs-fix-afs_fs_fetch_data-to-set-call-async.patch
and it can be found in the queue-7.1 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From d568a43f6dbba3ba006304d95fd09862bd482a2f Mon Sep 17 00:00:00 2001
From: David Howells <dhowells at redhat.com>
Date: Thu, 23 Jul 2026 12:34:46 +0100
Subject: afs: Fix afs_fs_fetch_data() to set call->async

From: David Howells <dhowells at redhat.com>

commit d568a43f6dbba3ba006304d95fd09862bd482a2f upstream.

Fix afs_fs_fetch_data() to set call->async on an async operation as does
afs_fs_fetch_data64().

Fixes: eddf51f2bb2c ("afs: Make {Y,}FS.FetchData an asynchronous operation")
Link: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-2-dhowells at redhat.com
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/fsclient.c |    3 +++
 1 file changed, 3 insertions(+)

--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -477,6 +477,9 @@ void afs_fs_fetch_data(struct afs_operat
 	if (!call)
 		return afs_op_nomem(op);
 
+	if (op->flags & AFS_OPERATION_ASYNC)
+		call->async = true;
+
 	/* marshall the parameters */
 	bp = call->request;
 	bp[0] = htonl(FSFETCHDATA);


Patches currently in stable-queue which might be from dhowells at redhat.com are

queue-7.1/netfs-fix-folio_queue-enomem-in-writeback-by-adding-.patch
queue-7.1/netfs-release-readahead-folios-on-iterator-preparati.patch
queue-7.1/netfs-handle-single-writeback-rolling-buffer-allocat.patch
queue-7.1/netfs-clear-pg_private_2-on-copy-to-cache-append-fai.patch
queue-7.1/afs-fix-uaf-when-sending-a-message.patch
queue-7.1/afs-fix-afs_fs_fetch_data-to-set-call-async.patch
queue-7.1/afs-fix-afs_fs_fetch_data-to-subtract-transferred-from-len.patch


From dhowells at redhat.com  Tue Aug  4 08:46:08 2026
From: dhowells at redhat.com (David Howells)
Date: Tue, 04 Aug 2026 16:46:08 +0100
Subject: [PATCH net v5 07/10] afs: Create a server appdata key
In-Reply-To: <20260803160627.GL51943@horms.kernel.org>
References: <20260803160627.GL51943@horms.kernel.org> <20260729160108.2031453-1-dhowells@redhat.com> <20260729160108.2031453-8-dhowells@redhat.com>
Message-ID: <2804996.1785858368@warthog.procyon.org.uk>

Simon Horman <horms at kernel.org> wrote:

> [Severity: Critical]
> This is a pre-existing issue, but is it safe to cast and dereference peer_data
> here?
> 
> When a network CHALLENGE packet arrives and is processed in the OOB queue in
> afs_respond_to_challenge(), it reads peer_data and casts it to an afs_server
> pointer.
> 
> Since the code dereferences this pointer without holding rcu_read_lock() or
> taking a reference on the server object, what happens if the server is
> concurrently destroyed or updated?
> 
> If peer_data is set to 0, this causes a direct NULL dereference. If the
> server memory is freed, could this result in a use-after-free triggered by a
> remote attacker?

Note that the OOB code is removed by these patches.

The other two issues here need addressing, though for one of them, the fix
will need to go via the vfs tree.

David



From dhowells at redhat.com  Tue Aug  4 10:26:27 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:27 +0100
Subject: [PATCH net v6 00/10] rxrpc: Fix CHALLENGE packet handling
Message-ID: <20260804172639.2844491-1-dhowells@redhat.com>

Here's a fix for AF_RXRPC's CHALLENGE packet handling, addressing an issue
raised by Sashiko[1], plus some miscellaneous fixes found in the process of
fixing this, plus a number of things raised by Sashiko[2][3][4][5][6].

Firstly, the miscellaneous patches:

 (1) Fix rxrpc_sendmsg so that it doesn't return an error if it queued the
     last packet of a call.  After that point, the error will be returned
     by recvmsg() and returned it twice in two different places may
     complicate userspace cleaning up its own structures.

 (2) Fix error handling in rxrpc_send_data() for if ->secure_packet()
     returns an error.

 (3) Fix the update of call->pending in rxrpc_send_data() in paths when the
     call lock has been dropped.

 (4) Fix the generation of notifications from rxrpc after call completion.

And then there are the patches to fix CHALLENGE packet overqueuing and
simplify RESPONSE packet generation by pre-creating the RxGK application
data up front and passing it in a user key (thereby allowing userspace to
partake).  This is split into five patches:

 (5) Expand the abort trace enum to be larger than a signed char as the
     number of elements will exceed 128.

 (6) Add a refcount to the user key payload.

 (7) Make the AFS filesystem generate per-server appdata keys.

 (8) Pass the appdata from AFS (or userspace) to rxrpc.

 (9) Change over to using the appdata key to supply the appdata.

(10) Remove all the OOB stuff.

[!] Note that this entails a significant change in the UAPI for AF_RXRPC,
with the CMSG types and sockopt to support the OOB queuing being removed
and replaced with a new single CMSG type that conveys the user key ID.  I
don't think it likely anyone is using this outside of my kafs-utils
package.

This also involves a change to the user-defined key type, making the
payload refcounted so that it can be accessed and the length read, then a
buffer allocated that will hold it and other data, and then the content
copied.  The problem is that the user is perfectly at liberty to change the
content of a user-defined key (which will RCU-replace the content of the
key), so the length might change when we drop the RCU read lock in order to
allocate.  This could be got around by locking the key->rwsem sharedly, but
that might be able to deadlock part of the rxrpc protocol engine if memory
reclaim occurs.

David

The patches can be found here also:

	http://git.kernel.org/cgit/linux/kernel/git/dhowells/linux-fs.git/log/?h=rxrpc-fixes

Changes
=======
ver #6)
- Rebased on latest net/main.
- Fixed more Sashiko-reported bugs[6]:
  - Fixed rxrpc_send_data() to redo the RXRPC_CALL_TX_ERROR and the
    RXRPC_CALL_TX_NO_MORE checks after having dropped the call mutex.
  - Altered rxrpc_send_data(), as discussed with Paulo Abeni, to rewind as
    much as possible on retryable crypto error (e.g. ENOMEM) rather than
    rewinding just one byte.
  - Fixed afs_make_call() to keep trying rxrpc_kernel_send_data() until the
    iterator is drained unless an error occurs.
  - Fixed afs_create_yfs_rxgk_cm_appdata() to use kfree_sensitive().
  - Removed remaining OOB trace constants.

ver #5)
- Rebased on latest net/main.
- Fixed more Sashiko-reported bugs[5]:
  - Removed dropped_lock from rxrpc_do_sendmsg() as it's now always false.
  - Fix rxrpc_send_data() to not leak a txbuf from the "maybe_error:" path
    by reattaching it to call->tx_pending.
  - Increased the size of the rxrpc_abort_reason enum value by removing the
    __mode(byte) specifier.

ver #4)
- Rebased on latest net/main.
- Split out non-relevant AFS patches.
- Allow logon-type key as well as user-type key as they're basically the
  same thing internally.
- Fixed more Sashiko-reported bugs[4]:
  - Fixed rxrpc_send_data() to wind the transmitted data back by 1 byte if
    ENOMEM is hit when encrypting the final packet so that the caller can
    retry.
  - Fixed afs_create_yfs_rxgk_cm_appdata() to add the 4 bytes for the level
    into toksize.
  - Changed bundle code to include the appdata key as part of the client
    connection bundle lookup criteria (don't share connections with
    different appdata keys).
  - Fixed rxrpc_sendmsg_cmsg() to allow only, not disallow user-type keys
    for RXRPC_RESPONSE_APPDATA.
  - Reversed the removal of the rejection of MSG_OOB passed to
    rxrpc_recvmsg().
  - Fixed rxgk_construct_response() to use xdr_object_len() to calculate
    the space needed for the appdata and also the space needed for the
    token and the authenticator.
  - Added a check into rxgk_respond_to_challenge() to make sure the key
    type is user or login before we access its payload.
  - Remove ->sendmsg_respond_to_challenge() too.

ver #3)
- Rebased on latest net/main.
  - Removed two obsoleted patches.

ver #2)
- Split the CHALLENGE/RESPONSE fix into smaller patches.
- Fixed more Sashiko-reported bugs[2][3]:
  - Added some more patches to fix some more bugs.
  - Get rid of the AFS_SERVER_FL_APPDATA flag and check the pointer to the
    appdata instead.
  - Rename the appdata key pointer in the AFS_SERVER to reflect this one is
    only for the YFS-RxGK security class.
  - Use barriers when reading or writing the server appdata key pointer.
  - Ignore the appdata for RxNULL, RxKAD and OpenAFS's RxGK for now.
  - Check that sendmsg() with RXRPC_RESPONSE_APPDATA is passed a user key.
  - Check that the appdata key's payload isn't NULL, for instance if it
    gets revoked.
  - Add some error path key_put()s in rxrpc_do_sendmsg().

[1] https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
[2] https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
[3] https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
[4] https://sashiko.dev/#/patchset/20260713081022.2186481-1-dhowells%40redhat.com
[5] https://sashiko.dev/#/patchset/20260723100309.530157-1-dhowells%40redhat.com
[6] https://sashiko.dev/#/patchset/20260729160108.2031453-1-dhowells%40redhat.com

David Howells (10):
  rxrpc: Fix sendmsg to not return an error if last packet queued
  rxrpc: Fix packet encryption error handling
  rxrpc: Fix update of call->tx_pending without holding lock
  rxrpc: Fix generation of notifications after call completion
  rxrpc: Expand abort trace enum
  keys: Add refcounting to user-defined key type payload
  afs: Create a server appdata key
  rxrpc: Pass appdata key to rxrpc_call and thence to rxrpc_bundle
  rxrpc: Fix CHALLENGE packet overqueuing and simplify RESPONSE
    generation
  rxrpc: Remove OOB challenge/response code

 fs/afs/cm_security.c         | 315 +++++++++++++---------------
 fs/afs/fs_probe.c            |   5 +
 fs/afs/internal.h            |  36 ++--
 fs/afs/main.c                |   1 -
 fs/afs/rxrpc.c               |  62 +++---
 fs/afs/server.c              |   2 +-
 include/keys/user-type.h     |   2 +
 include/net/af_rxrpc.h       |  21 +-
 include/trace/events/afs.h   |   1 +
 include/trace/events/rxrpc.h |  15 +-
 include/uapi/linux/rxrpc.h   |   6 +-
 net/dns_resolver/dns_key.c   |   1 +
 net/rxrpc/Makefile           |   1 -
 net/rxrpc/af_rxrpc.c         |  49 +----
 net/rxrpc/ar-internal.h      |  27 +--
 net/rxrpc/call_object.c      |   2 +
 net/rxrpc/call_state.c       |  57 +++++-
 net/rxrpc/conn_client.c      |   4 +
 net/rxrpc/conn_event.c       |  70 +------
 net/rxrpc/insecure.c         |   7 -
 net/rxrpc/key.c              |  37 ++++
 net/rxrpc/oob.c              | 387 -----------------------------------
 net/rxrpc/proc.c             |   5 +-
 net/rxrpc/recvmsg.c          | 124 ++---------
 net/rxrpc/rxgk.c             | 138 ++++---------
 net/rxrpc/rxkad.c            |  27 ---
 net/rxrpc/sendmsg.c          | 140 +++++++++----
 net/rxrpc/server_key.c       |  40 ----
 security/keys/user_defined.c |  23 ++-
 29 files changed, 506 insertions(+), 1099 deletions(-)
 delete mode 100644 net/rxrpc/oob.c



From dhowells at redhat.com  Tue Aug  4 10:26:28 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:28 +0100
Subject: [PATCH net v6 01/10] rxrpc: Fix sendmsg to not return an error if last packet queued
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-2-dhowells@redhat.com>

Fix AF_RXRPC sendmsg() so that it doesn't return an error if it has
successfully queued the last packet of a call, but the call has seen to
have completed after it did that.  Rather, leave it to recvmsg() to report
the completion (which it will do anyway).

The problem with trying to report the error twice is that the caller may
try to clean up the dead call twice.

Fixes: d41b3f5b9688 ("rxrpc: Wrap accesses to get call state to put the barrier in one place")
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
---
 net/rxrpc/sendmsg.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index ed2c9a51005a..4c754f78ece9 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -453,9 +453,6 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 
 success:
 	ret = copied;
-	if (rxrpc_call_is_complete(call) &&
-	    call->error < 0)
-		ret = call->error;
 out:
 	call->tx_pending = txb;
 	_leave(" = %d", ret);
@@ -467,8 +464,14 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	return call->error;
 
 maybe_error:
-	if (copied)
+	if (copied) {
+		if (rxrpc_call_is_complete(call) &&
+		    call->error < 0) {
+			ret = call->error;
+			goto out;
+		}
 		goto success;
+	}
 	goto out;
 
 efault:



From dhowells at redhat.com  Tue Aug  4 10:26:29 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:29 +0100
Subject: [PATCH net v6 02/10] rxrpc: Fix packet encryption error handling
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-3-dhowells@redhat.com>

In rxrpc_send_data(), if ->secure_packet() returns an error, the code
currently just jumps to out: and returns the error to the app on the
assumption that any error returned by this is automatically fatal for the
call, and may even have corrupted the transmission queue - but leaving it
to userspace to deal with.  Nothing stops the application from retrying the
sendmsg(), which will try to encrypt the buffer again, and might succeed
with a corrupt buffer.

Fix rxrpc_send_data() in the following ways:

 (1) If -ENOMEM is returned, assume we never got as far as the encryption
     and that the operation is retryable.  In which case, jump to
     maybe_error_rewind and, if we've copied data into the last packet,
     remove a byte from it so that we don't tell the caller that we've
     completed the transmission phase.

 (2) If any other error occurs, set the TX_ERROR flag on the call and
     return that error directly; on all subsequent attempts to add data to
     the call, return -EIO.  The app must then abort the call to get rid of
     it (this allows the app to choose the abort code to use).

Fixes: 17926a79320a ("[AF_RXRPC]: Provide secure RxRPC sockets for use by userspace and kernel both")
Closes: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
---
 fs/afs/rxrpc.c               | 22 +++++++++------
 include/trace/events/rxrpc.h |  1 +
 net/rxrpc/ar-internal.h      |  1 +
 net/rxrpc/sendmsg.c          | 53 ++++++++++++++++++++++++++++--------
 4 files changed, 57 insertions(+), 20 deletions(-)

diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index d82916657a3d..b435f4b2da7f 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -412,21 +412,25 @@ void afs_make_call(struct afs_call *call, gfp_t gfp)
 	msg.msg_controllen	= 0;
 	msg.msg_flags		= MSG_WAITALL | (call->write_iter ? MSG_MORE : 0);
 
-	ret = rxrpc_kernel_send_data(call->net->socket, rxcall,
-				     &msg, call->request_size,
-				     afs_notify_end_request_tx);
-	if (ret < 0)
-		goto error_do_abort;
+	do {
+		ret = rxrpc_kernel_send_data(call->net->socket, rxcall,
+					     &msg, call->request_size,
+					     afs_notify_end_request_tx);
+		if (ret < 0)
+			goto error_do_abort;
+	} while (iov_iter_count(&msg.msg_iter) > 0);
 
 	if (call->write_iter) {
 		msg.msg_iter = *call->write_iter;
 		msg.msg_flags &= ~MSG_MORE;
 		trace_afs_send_data(call, &msg);
 
-		ret = rxrpc_kernel_send_data(call->net->socket,
-					     call->rxcall, &msg,
-					     iov_iter_count(&msg.msg_iter),
-					     afs_notify_end_request_tx);
+		do {
+			ret = rxrpc_kernel_send_data(call->net->socket,
+						     call->rxcall, &msg,
+						     iov_iter_count(&msg.msg_iter),
+						     afs_notify_end_request_tx);
+		} while (iov_iter_count(&msg.msg_iter) > 0);
 		*call->write_iter = msg.msg_iter;
 
 		trace_afs_sent_data(call, &msg, ret);
diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index 704a10de6670..8f3e3967885a 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -148,6 +148,7 @@
 	EM(rxrpc_eproto_wrong_security,		"wrong-sec")		\
 	EM(rxrpc_recvmsg_excess_data,		"recvmsg-excess")	\
 	EM(rxrpc_recvmsg_short_data,		"recvmsg-short")	\
+	EM(rxrpc_sendmsg_tx_error,		"tx-error")		\
 	E_(rxrpc_sendmsg_late_send,		"sendmsg-late")
 
 #define rxrpc_call_poke_traces \
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 865f05fe37ab..a6f830c1621f 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -642,6 +642,7 @@ enum rxrpc_call_flag {
 	RXRPC_CALL_TX_LAST,		/* Last packet in Tx buffer (at rxtx_top) */
 	RXRPC_CALL_TX_ALL_ACKED,	/* Last packet has been hard-acked */
 	RXRPC_CALL_TX_NO_MORE,		/* No more data to transmit (MSG_MORE deasserted) */
+	RXRPC_CALL_TX_ERROR,		/* Terminal error; call needs abort */
 	RXRPC_CALL_SEND_PING,		/* A ping will need to be sent */
 	RXRPC_CALL_RETRANS_TIMEOUT,	/* Retransmission due to timeout occurred */
 	RXRPC_CALL_BEGAN_RX_TIMER,	/* We began the expect_rx_by timer */
diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index 4c754f78ece9..5ea10dc497a7 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -330,13 +330,6 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	bool more = msg->msg_flags & MSG_MORE;
 	int ret, copied = 0;
 
-	if (test_bit(RXRPC_CALL_TX_NO_MORE, &call->flags)) {
-		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_late_send,
-				  call->cid, call->call_id, call->rx_consumed,
-				  0, -EPROTO);
-		return -EPROTO;
-	}
-
 	timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
 
 	ret = rxrpc_wait_to_be_connected(call, &timeo);
@@ -353,6 +346,19 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk);
 
 reload:
+	if (unlikely(test_bit(RXRPC_CALL_TX_NO_MORE, &call->flags))) {
+		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_late_send,
+				  call->cid, call->call_id, call->rx_consumed,
+				  0, -EPROTO);
+		return -EPROTO;
+	}
+	if (unlikely(test_bit(RXRPC_CALL_TX_ERROR, &call->flags))) {
+		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_tx_error,
+				  call->cid, call->call_id, call->rx_consumed,
+				  0, -EIO);
+		return -EIO;
+	}
+
 	txb = call->tx_pending;
 	call->tx_pending = NULL;
 	if (txb)
@@ -440,12 +446,21 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 		/* add the packet to the send queue if it's now full */
 		if (!txb->space ||
 		    (msg_data_left(msg) == 0 && !more)) {
-			if (msg_data_left(msg) == 0 && !more)
-				txb->flags |= RXRPC_LAST_PACKET;
-
+			/* Do any required crypto.  If this fails, it could
+			 * have corrupted the txbuf content with a partial
+			 * encrypt.  Assume that ENOMEM is retryable, but
+			 * everything else is terminal.
+			 */
 			ret = call->security->secure_packet(call, txb);
-			if (ret < 0)
+			if (ret < 0) {
+				if (ret == -ENOMEM)
+					goto maybe_error_rewind;
+				set_bit(RXRPC_CALL_TX_ERROR, &call->flags);
 				goto out;
+			}
+
+			if (msg_data_left(msg) == 0 && !more)
+				txb->flags |= RXRPC_LAST_PACKET;
 			rxrpc_queue_packet(rx, call, txb, notify_end_tx);
 			txb = NULL;
 		}
@@ -463,6 +478,22 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	_leave(" = %d", call->error);
 	return call->error;
 
+maybe_error_rewind:
+	/* If we got a retryable error after copying all the supplied data into
+	 * the last packet, we need to rewind as much as we can so the caller
+	 * knows they need to retry the sendmsg.
+	 */
+	if (copied && !more && !msg_data_left(msg)) {
+		unsigned int rewind_by = umin(copied, txb->len);
+
+		txb->space  += rewind_by;
+		txb->len    -= rewind_by;
+		txb->offset -= rewind_by;
+		copied      -= rewind_by;
+		if (call->tx_total_len != -1)
+			call->tx_total_len += rewind_by;
+		iov_iter_revert(&msg->msg_iter, rewind_by);
+	}
 maybe_error:
 	if (copied) {
 		if (rxrpc_call_is_complete(call) &&



From dhowells at redhat.com  Tue Aug  4 10:26:31 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:31 +0100
Subject: [PATCH net v6 04/10] rxrpc: Fix generation of notifications after call completion
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-5-dhowells@redhat.com>

AF_RXRPC may generate a notification to the application after a call has
completed because it generates one notification when
rxrpc_input_split_jumbo() queues the final packet and completes the call
and then generates another when rxrpc_input_split_jumbo() does the
aggregated data receive notification at the end of the function.

This might cause the AFS filesystem to malfunction because it tries to
queue the afs_call for processing an extra time.  Most of the time this
happens quickly enough that the second queue_work skips, but sometimes this
means that the call work may happen a second time with implications for
afs_call lifetime management.

Fix this by:

 (1) Create a lighter version of rxrpc_notify_socket() that's just used to
     requeue a call for rxrpc_recvmsg() without creating another
     notification.

 (2) Move rxrpc_notify_socket() to call_state.c and rename it to
     __rxrpc_notify_socket().

 (3) Create a wrapper called rxrpc_notify_socket() that skips the
     notification if a call is completed.

 (4) Make rxrpc_set_call_completion() call __rxrpc_notify_socket() to avoid
     the skip-if-completed check.

Fixes: 2d1faf7a0ca3 ("rxrpc: Simplify skbuff accounting in receive path")
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
---
 include/trace/events/rxrpc.h |  1 +
 net/rxrpc/call_state.c       | 57 +++++++++++++++++++++++++++++++++++-
 net/rxrpc/recvmsg.c          | 42 +++++++++-----------------
 3 files changed, 71 insertions(+), 29 deletions(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index 8f3e3967885a..d7c7b04d69fc 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -343,6 +343,7 @@
 	EM(rxrpc_call_see_distribute_error,	"SEE dist-err") \
 	EM(rxrpc_call_see_input,		"SEE input   ") \
 	EM(rxrpc_call_see_notify_released,	"SEE nfy-rlsd") \
+	EM(rxrpc_call_see_notify_skipped,	"SEE nfy-skip") \
 	EM(rxrpc_call_see_recvmsg,		"SEE recvmsg ") \
 	EM(rxrpc_call_see_recvmsg_requeue,	"SEE recv-rqu") \
 	EM(rxrpc_call_see_recvmsg_requeue_first, "SEE recv-rqF") \
diff --git a/net/rxrpc/call_state.c b/net/rxrpc/call_state.c
index 6afb54373ebb..6e402312e145 100644
--- a/net/rxrpc/call_state.c
+++ b/net/rxrpc/call_state.c
@@ -7,6 +7,61 @@
 
 #include "ar-internal.h"
 
+/*
+ * Post a call for attention by the socket or kernel service.  Further
+ * notifications are suppressed by putting recvmsg_link on a dummy queue.
+ */
+static void __rxrpc_notify_socket(struct rxrpc_call *call)
+{
+	struct rxrpc_sock *rx;
+	struct sock *sk;
+
+	if (test_bit(RXRPC_CALL_RELEASED, &call->flags)) {
+		rxrpc_see_call(call, rxrpc_call_see_notify_released);
+		return;
+	}
+
+	rcu_read_lock();
+
+	rx = rcu_dereference(call->socket);
+	sk = &rx->sk;
+	if (rx && sk->sk_state < RXRPC_CLOSE) {
+		if (call->notify_rx) {
+			spin_lock_irq(&call->notify_lock);
+			call->notify_rx(sk, call, call->user_call_ID);
+			spin_unlock_irq(&call->notify_lock);
+		} else {
+			spin_lock_irq(&rx->recvmsg_lock);
+			if (list_empty(&call->recvmsg_link)) {
+				rxrpc_get_call(call, rxrpc_call_get_notify_socket);
+				list_add_tail(&call->recvmsg_link, &rx->recvmsg_q);
+			}
+			spin_unlock_irq(&rx->recvmsg_lock);
+
+			if (!sock_flag(sk, SOCK_DEAD)) {
+				_debug("call %ps", sk->sk_data_ready);
+				sk->sk_data_ready(sk);
+			}
+		}
+	}
+
+	rcu_read_unlock();
+}
+
+/*
+ * Post a call for attention by the socket or kernel service.  Further
+ * notifications are suppressed by putting recvmsg_link on a dummy queue.
+ */
+void rxrpc_notify_socket(struct rxrpc_call *call)
+{
+	if (rxrpc_call_is_complete(call)) {
+		rxrpc_see_call(call, rxrpc_call_see_notify_skipped);
+		return;
+	}
+
+	__rxrpc_notify_socket(call);
+}
+
 /*
  * Transition a call to the complete state.
  */
@@ -25,7 +80,7 @@ bool rxrpc_set_call_completion(struct rxrpc_call *call,
 	rxrpc_set_call_state(call, RXRPC_CALL_COMPLETE);
 	trace_rxrpc_call_complete(call);
 	wake_up(&call->waitq);
-	rxrpc_notify_socket(call);
+	__rxrpc_notify_socket(call);
 	return true;
 }
 
diff --git a/net/rxrpc/recvmsg.c b/net/rxrpc/recvmsg.c
index efcba4b2e74f..28b2148b5693 100644
--- a/net/rxrpc/recvmsg.c
+++ b/net/rxrpc/recvmsg.c
@@ -17,13 +17,12 @@
 #include "ar-internal.h"
 
 /*
- * Post a call for attention by the socket or kernel service.  Further
- * notifications are suppressed by putting recvmsg_link on a dummy queue.
+ * Requeue a call for recvmsg() to pick up.
  */
-void rxrpc_notify_socket(struct rxrpc_call *call)
+static void rxrpc_requeue_call(struct socket *sock, struct rxrpc_call *call)
 {
-	struct rxrpc_sock *rx;
-	struct sock *sk;
+	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
+	struct sock *sk = &rx->sk;
 
 	_enter("%d", call->debug_id);
 
@@ -32,31 +31,18 @@ void rxrpc_notify_socket(struct rxrpc_call *call)
 		return;
 	}
 
-	rcu_read_lock();
-
-	rx = rcu_dereference(call->socket);
-	sk = &rx->sk;
-	if (rx && sk->sk_state < RXRPC_CLOSE) {
-		if (call->notify_rx) {
-			spin_lock_irq(&call->notify_lock);
-			call->notify_rx(sk, call, call->user_call_ID);
-			spin_unlock_irq(&call->notify_lock);
-		} else {
-			spin_lock_irq(&rx->recvmsg_lock);
-			if (list_empty(&call->recvmsg_link)) {
-				rxrpc_get_call(call, rxrpc_call_get_notify_socket);
-				list_add_tail(&call->recvmsg_link, &rx->recvmsg_q);
-			}
-			spin_unlock_irq(&rx->recvmsg_lock);
+	spin_lock_irq(&rx->recvmsg_lock);
+	if (list_empty(&call->recvmsg_link)) {
+		rxrpc_get_call(call, rxrpc_call_get_notify_socket);
+		list_add_tail(&call->recvmsg_link, &rx->recvmsg_q);
+	}
+	spin_unlock_irq(&rx->recvmsg_lock);
 
-			if (!sock_flag(sk, SOCK_DEAD)) {
-				_debug("call %ps", sk->sk_data_ready);
-				sk->sk_data_ready(sk);
-			}
-		}
+	if (!sock_flag(sk, SOCK_DEAD)) {
+		_debug("call %ps", sk->sk_data_ready);
+		sk->sk_data_ready(sk);
 	}
 
-	rcu_read_unlock();
 	_leave("");
 }
 
@@ -561,7 +547,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 
 	if (!(flags & MSG_PEEK) &&
 	    !skb_queue_empty(&call->recvmsg_queue))
-		rxrpc_notify_socket(call);
+		rxrpc_requeue_call(sock, call);
 	goto not_yet_complete;
 
 call_failed:



From dhowells at redhat.com  Tue Aug  4 10:26:32 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:32 +0100
Subject: [PATCH net v6 05/10] rxrpc: Expand abort trace enum
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-6-dhowells@redhat.com>

The rxrpc_abort_reason enum is marked __mode(byte), but there are currently
exactly 128 in the enum (0-127), but this is about to be increased and the
enum is represented by a signed byte and an increase causes compilation
errors.

Remove the mode marking to return it to a 32-bit int.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 include/trace/events/rxrpc.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index d7c7b04d69fc..ba2c6e3c78a6 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -602,7 +602,7 @@
 #define EM(a, b) a,
 #define E_(a, b) a
 
-enum rxrpc_abort_reason		{ rxrpc_abort_reasons } __mode(byte);
+enum rxrpc_abort_reason		{ rxrpc_abort_reasons };
 enum rxrpc_bundle_trace		{ rxrpc_bundle_traces } __mode(byte);
 enum rxrpc_call_poke_trace	{ rxrpc_call_poke_traces } __mode(byte);
 enum rxrpc_call_trace		{ rxrpc_call_traces } __mode(byte);



From dhowells at redhat.com  Tue Aug  4 10:26:30 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:30 +0100
Subject: [PATCH net v6 03/10] rxrpc: Fix update of call->tx_pending without holding lock
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-4-dhowells@redhat.com>

Currently, rxrpc_send_data() updates call->tx_pending just before it
returns - but it won't be holding the call->lock when it does this if a
wait was interrupted by a signal.  This would allow a parallel sendmsg() to
race.

Further, both the callers of rxrpc_send_data() call it with the lock held,
and then it returns an indication through the parameter list to say whether
it has dropped the lock or not - after which the callers both just drop the
lock if it's still held.

Fix this by:

 (1) Moving the release of call->lock down into rxrpc_send_data() and get
     rid of the indicator parameter.  This makes it easier to see where the
     lock is held.

 (2) Make the wait_for_space path move the value in txb back into
     call->tx_pending before dropping the lock prior to the wait.

 (3) After waiting, if the attempt to reacquire the mutex is interrupted,
     just return directly there rather than going to out_unlock

Fixes: b0f571ecd794 ("rxrpc: Fix locking in rxrpc's sendmsg")
Closes: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
---
 net/rxrpc/sendmsg.c | 56 ++++++++++++++++++++++++---------------------
 1 file changed, 30 insertions(+), 26 deletions(-)

diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index 5ea10dc497a7..79fd31b02d11 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -320,8 +320,8 @@ static int rxrpc_alloc_txqueue(struct sock *sk, struct rxrpc_call *call)
 static int rxrpc_send_data(struct rxrpc_sock *rx,
 			   struct rxrpc_call *call,
 			   struct msghdr *msg, size_t len,
-			   rxrpc_notify_end_tx_t notify_end_tx,
-			   bool *_dropped_lock)
+			   rxrpc_notify_end_tx_t notify_end_tx)
+	__releases(&call->user_mutex)
 {
 	struct rxrpc_txbuf *txb;
 	struct sock *sk = &rx->sk;
@@ -334,12 +334,12 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 
 	ret = rxrpc_wait_to_be_connected(call, &timeo);
 	if (ret < 0)
-		return ret;
+		goto out_unlock;
 
 	if (call->conn->state == RXRPC_CONN_CLIENT_UNSECURED) {
 		ret = rxrpc_init_client_conn_security(call->conn);
 		if (ret < 0)
-			return ret;
+			goto out_unlock;
 	}
 
 	/* this should be in poll */
@@ -350,13 +350,15 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_late_send,
 				  call->cid, call->call_id, call->rx_consumed,
 				  0, -EPROTO);
-		return -EPROTO;
+		ret = -EPROTO;
+		goto out_unlock;
 	}
 	if (unlikely(test_bit(RXRPC_CALL_TX_ERROR, &call->flags))) {
 		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_tx_error,
 				  call->cid, call->call_id, call->rx_consumed,
 				  0, -EIO);
-		return -EIO;
+		ret = -EIO;
+		goto out_unlock;
 	}
 
 	txb = call->tx_pending;
@@ -456,7 +458,7 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 				if (ret == -ENOMEM)
 					goto maybe_error_rewind;
 				set_bit(RXRPC_CALL_TX_ERROR, &call->flags);
-				goto out;
+				goto out_txb;
 			}
 
 			if (msg_data_left(msg) == 0 && !more)
@@ -468,15 +470,18 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 
 success:
 	ret = copied;
-out:
+out_txb:
 	call->tx_pending = txb;
+out_unlock:
+	mutex_unlock(&call->user_mutex);
 	_leave(" = %d", ret);
 	return ret;
 
 call_terminated:
 	rxrpc_put_txbuf(txb, rxrpc_txbuf_put_send_aborted);
-	_leave(" = %d", call->error);
-	return call->error;
+	call->tx_pending = NULL;
+	ret = call->error;
+	goto out_unlock;
 
 maybe_error_rewind:
 	/* If we got a retryable error after copying all the supplied data into
@@ -499,36 +504,40 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 		if (rxrpc_call_is_complete(call) &&
 		    call->error < 0) {
 			ret = call->error;
-			goto out;
+			goto out_txb;
 		}
 		goto success;
 	}
-	goto out;
+	goto out_txb;
 
 efault:
 	ret = -EFAULT;
-	goto out;
+	goto out_txb;
 
 wait_for_space:
 	ret = -EAGAIN;
 	if (msg->msg_flags & MSG_DONTWAIT)
 		goto maybe_error;
+	call->tx_pending = txb;
+	txb = NULL;
 	mutex_unlock(&call->user_mutex);
-	*_dropped_lock = true;
+
 	ret = rxrpc_wait_for_tx_window(rx, call, &timeo,
 				       msg->msg_flags & MSG_WAITALL);
 	if (ret < 0)
-		goto maybe_error;
+		goto out_nolock;
 	if (call->interruptibility == RXRPC_INTERRUPTIBLE) {
 		if (mutex_lock_interruptible(&call->user_mutex) < 0) {
 			ret = sock_intr_errno(timeo);
-			goto maybe_error;
+			goto out_nolock;
 		}
 	} else {
 		mutex_lock(&call->user_mutex);
 	}
-	*_dropped_lock = false;
 	goto reload;
+out_nolock:
+	_leave(" = %d [intr]", ret);
+	return copied ?: ret;
 }
 
 /*
@@ -694,7 +703,6 @@ rxrpc_new_client_call_for_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg,
 int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
 {
 	struct rxrpc_call *call;
-	bool dropped_lock = false;
 	int ret;
 
 	struct rxrpc_send_params p = {
@@ -803,16 +811,15 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
 		ret = 0;
 		break;
 	case RXRPC_CMD_SEND_DATA:
-		ret = rxrpc_send_data(rx, call, msg, len, NULL, &dropped_lock);
-		break;
+		ret = rxrpc_send_data(rx, call, msg, len, NULL);
+		goto error_put;
 	default:
 		ret = -EINVAL;
 		break;
 	}
 
 out_put_unlock:
-	if (!dropped_lock)
-		mutex_unlock(&call->user_mutex);
+	mutex_unlock(&call->user_mutex);
 error_put:
 	rxrpc_put_call(call, rxrpc_call_put_sendmsg);
 	_leave(" = %d", ret);
@@ -842,7 +849,6 @@ int rxrpc_kernel_send_data(struct socket *sock, struct rxrpc_call *call,
 			   struct msghdr *msg, size_t len,
 			   rxrpc_notify_end_tx_t notify_end_tx)
 {
-	bool dropped_lock = false;
 	int ret;
 
 	_enter("{%d},", call->debug_id);
@@ -853,12 +859,10 @@ int rxrpc_kernel_send_data(struct socket *sock, struct rxrpc_call *call,
 	mutex_lock(&call->user_mutex);
 
 	ret = rxrpc_send_data(rxrpc_sk(sock->sk), call, msg, len,
-			      notify_end_tx, &dropped_lock);
+			      notify_end_tx);
 	if (ret == -ESHUTDOWN)
 		ret = call->error;
 
-	if (!dropped_lock)
-		mutex_unlock(&call->user_mutex);
 	_leave(" = %d", ret);
 	return ret;
 }



From dhowells at redhat.com  Tue Aug  4 10:26:33 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:33 +0100
Subject: [PATCH net v6 06/10] keys: Add refcounting to user-defined key type payload
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-7-dhowells@redhat.com>

Add refcounting to user-defined key type payload so that a kernel service
wanting to use such a key can hold onto the payload without the RCU read
lock held in order that it can do an allocation without having to be
concerned with the key getting updated.

This is the first part of the fix for the AF_RXRPC challenge response
generation code.

Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 include/keys/user-type.h     |  2 ++
 net/dns_resolver/dns_key.c   |  1 +
 security/keys/user_defined.c | 23 ++++++++++++++++-------
 3 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/include/keys/user-type.h b/include/keys/user-type.h
index 386c31432789..7002a993a472 100644
--- a/include/keys/user-type.h
+++ b/include/keys/user-type.h
@@ -26,6 +26,7 @@
  */
 struct user_key_payload {
 	struct rcu_head	rcu;		/* RCU destructor */
+	refcount_t	ref;
 	unsigned short	datalen;	/* length of this data */
 	char		data[] __aligned(__alignof__(u64)); /* actual data */
 };
@@ -37,6 +38,7 @@ struct key_preparsed_payload;
 
 extern int user_preparse(struct key_preparsed_payload *prep);
 extern void user_free_preparse(struct key_preparsed_payload *prep);
+void put_user_key_payload(struct user_key_payload *payload);
 extern int user_update(struct key *key, struct key_preparsed_payload *prep);
 extern void user_revoke(struct key *key);
 extern void user_destroy(struct key *key);
diff --git a/net/dns_resolver/dns_key.c b/net/dns_resolver/dns_key.c
index c3c8c3240ef9..aa3c058f4095 100644
--- a/net/dns_resolver/dns_key.c
+++ b/net/dns_resolver/dns_key.c
@@ -208,6 +208,7 @@ dns_resolver_preparse(struct key_preparsed_payload *prep)
 		kleave(" = -ENOMEM");
 		return -ENOMEM;
 	}
+	refcount_set(&upayload->ref, 1);
 
 	upayload->datalen = result_len;
 	memcpy(upayload->data, data, result_len);
diff --git a/security/keys/user_defined.c b/security/keys/user_defined.c
index 6f88b507f927..90c1bd5d7dfe 100644
--- a/security/keys/user_defined.c
+++ b/security/keys/user_defined.c
@@ -67,6 +67,7 @@ int user_preparse(struct key_preparsed_payload *prep)
 	upayload = kmalloc_flex(*upayload, data, datalen);
 	if (!upayload)
 		return -ENOMEM;
+	refcount_set(&upayload->ref, 1);
 
 	/* attach the data */
 	prep->quotalen = datalen;
@@ -88,12 +89,22 @@ EXPORT_SYMBOL_GPL(user_free_preparse);
 
 static void user_free_payload_rcu(struct rcu_head *head)
 {
-	struct user_key_payload *payload;
+	struct user_key_payload *payload =
+		container_of(head, struct user_key_payload, rcu);
 
-	payload = container_of(head, struct user_key_payload, rcu);
 	kfree_sensitive(payload);
 }
 
+/*
+ * Free a user defined key payload.
+ */
+void put_user_key_payload(struct user_key_payload *payload)
+{
+	if (payload && refcount_dec_and_test(&payload->ref))
+		call_rcu(&payload->rcu, user_free_payload_rcu);
+}
+EXPORT_SYMBOL_GPL(put_user_key_payload);
+
 /*
  * update a user defined key
  * - the key's semaphore is write-locked
@@ -115,8 +126,7 @@ int user_update(struct key *key, struct key_preparsed_payload *prep)
 	rcu_assign_keypointer(key, prep->payload.data[0]);
 	prep->payload.data[0] = NULL;
 
-	if (zap)
-		call_rcu(&zap->rcu, user_free_payload_rcu);
+	put_user_key_payload(zap);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(user_update);
@@ -134,7 +144,7 @@ void user_revoke(struct key *key)
 
 	if (upayload) {
 		rcu_assign_keypointer(key, NULL);
-		call_rcu(&upayload->rcu, user_free_payload_rcu);
+		put_user_key_payload(upayload);
 	}
 }
 
@@ -147,9 +157,8 @@ void user_destroy(struct key *key)
 {
 	struct user_key_payload *upayload = key->payload.data[0];
 
-	kfree_sensitive(upayload);
+	put_user_key_payload(upayload);
 }
-
 EXPORT_SYMBOL_GPL(user_destroy);
 
 /*



From dhowells at redhat.com  Tue Aug  4 10:26:34 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:34 +0100
Subject: [PATCH net v6 07/10] afs: Create a server appdata key
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-8-dhowells@redhat.com>

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

This is a part of the fix.  With this, the AFS filesystem creates an
appdata key for each fileserver it talks to with RxGK and attaches it to
the afs_server record.

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 fs/afs/cm_security.c   | 232 +++++++++++++++++++++++++++++++++++++++++
 fs/afs/fs_probe.c      |   5 +
 fs/afs/internal.h      |   2 +
 fs/afs/server.c        |   1 +
 include/net/af_rxrpc.h |   2 +
 net/rxrpc/key.c        |  37 +++++++
 6 files changed, 279 insertions(+)

diff --git a/fs/afs/cm_security.c b/fs/afs/cm_security.c
index 103168c70dd4..8f04fec20a2a 100644
--- a/fs/afs/cm_security.c
+++ b/fs/afs/cm_security.c
@@ -6,7 +6,9 @@
  */
 
 #include <linux/slab.h>
+#include <linux/key-type.h>
 #include <crypto/krb5.h>
+#include <keys/user-type.h>
 #include "internal.h"
 #include "afs_cm.h"
 #include "afs_fs.h"
@@ -23,6 +25,236 @@ static int afs_create_yfs_cm_token(struct sk_buff *challenge,
 				   struct afs_server *server);
 #endif
 
+#ifdef CONFIG_RXGK
+/*
+ * As the YFS RxGK appdata to be passed in the YFS.FS-service RESPONSE packet,
+ * create a GSS token to use as a ticket to the specified fileserver.
+ */
+static int afs_create_yfs_rxgk_cm_appdata(struct afs_server *server, u32 enctype)
+{
+	const struct krb5_enctype *conn_krb5, *token_krb5;
+	const struct krb5_buffer *token_key;
+	struct crypto_aead *aead;
+	struct scatterlist sg;
+	struct afs_net *net = server->cell->net;
+	const struct key *cm_key = net->fs_cm_token_key;
+	struct key *appdata_key = NULL;
+	size_t keysize, uuidsize, authsize, toksize, encsize, contsize;
+	size_t adatasize, offset;
+	__be32 caps[1] = {
+		[0] = htonl(AFS_CAP_ERROR_TRANSLATION),
+	};
+	__be32 *xdr;
+	void *appdata, *K0, *encbase;
+	int ret;
+
+	if (!cm_key)
+		return -ENOKEY;
+
+	/* Assume that the fileserver is happy to use the same encoding type as
+	 * we were told to use by the token obtained by the user.
+	 */
+	conn_krb5 = crypto_krb5_find_enctype(enctype);
+	if (!conn_krb5)
+		return -ENOPKG;
+	token_krb5 = cm_key->payload.data[0];
+	token_key = (const struct krb5_buffer *)&cm_key->payload.data[2];
+
+	/* struct rxgk_key {
+	 *	afs_uint32	enctype;
+	 *	opaque		key<>;
+	 * };
+	 */
+	keysize = 4 + xdr_len_object(conn_krb5->key_len);
+
+	/* struct RXGK_AuthName {
+	 *	afs_int32	kind;
+	 *	opaque		data<AUTHDATAMAX>;
+	 *	opaque		display<AUTHPRINTABLEMAX>;
+	 * };
+	 */
+	uuidsize = sizeof(server->uuid);
+	authsize = 4 + xdr_len_object(uuidsize) + xdr_len_object(0);
+
+	/* struct RXGK_Token {
+	 *	rxgk_key		K0;
+	 *	RXGK_Level		level;
+	 *	rxgkTime		starttime;
+	 *	afs_int32		lifetime;
+	 *	afs_int32		bytelife;
+	 *	rxgkTime		expirationtime;
+	 *	struct RXGK_AuthName	identities<>;
+	 * };
+	 */
+	toksize = keysize + 4 + 8 + 4 + 4 + 8 + xdr_len_object(authsize);
+
+	offset = 0;
+	encsize = crypto_krb5_how_much_buffer(token_krb5, KRB5_ENCRYPT_MODE, toksize, &offset);
+
+	/* struct RXGK_TokenContainer {
+	 *	afs_int32	kvno;
+	 *	afs_int32	enctype;
+	 *	opaque		encrypted_token<>;
+	 * };
+	 */
+	contsize = 4 + 4 + xdr_len_object(encsize);
+
+	/* struct YFSAppData {
+	 *	opr_uuid	initiatorUuid;
+	 *	opr_uuid	acceptorUuid;
+	 *	Capabilities	caps;
+	 *	afs_int32	enctype;
+	 *	opaque		callbackKey<>;
+	 *	opaque		callbackToken<>;
+	 * };
+	 */
+	adatasize = 16 + 16 +
+		xdr_len_object(sizeof(caps)) +
+		4 +
+		xdr_len_object(conn_krb5->key_len) +
+		xdr_len_object(contsize);
+
+	ret = -ENOMEM;
+	appdata = kzalloc(adatasize, GFP_KERNEL);
+	if (!appdata)
+		goto out;
+	xdr = appdata;
+
+	memcpy(xdr, &net->uuid, 16);		/* appdata.initiatorUuid */
+	xdr += 16 / 4;
+	memcpy(xdr, &server->uuid, 16);		/* appdata.acceptorUuid */
+	xdr += 16 / 4;
+	*xdr++ = htonl(ARRAY_SIZE(caps));	/* appdata.caps.len */
+	memcpy(xdr, &caps, sizeof(caps));	/* appdata.caps */
+	xdr += ARRAY_SIZE(caps);
+	*xdr++ = htonl(conn_krb5->etype);	/* appdata.enctype */
+
+	*xdr++ = htonl(conn_krb5->key_len);	/* appdata.callbackKey.len */
+	K0 = xdr;
+	get_random_bytes(K0, conn_krb5->key_len); /* appdata.callbackKey.data */
+	xdr += xdr_round_up(conn_krb5->key_len) / 4;
+
+	*xdr++ = htonl(contsize);		/* appdata.callbackToken.len */
+	*xdr++ = htonl(1);			/* cont.kvno */
+	*xdr++ = htonl(token_krb5->etype);	/* cont.enctype */
+	*xdr++ = htonl(encsize);		/* cont.encrypted_token.len */
+
+	encbase = xdr;
+	xdr += offset / 4;
+	*xdr++ = htonl(conn_krb5->etype);	/* token.K0.enctype */
+	*xdr++ = htonl(conn_krb5->key_len);	/* token.K0.key.len */
+	memcpy(xdr, K0, conn_krb5->key_len);	/* token.K0.key.data */
+	xdr += xdr_round_up(conn_krb5->key_len) / 4;
+
+	*xdr++ = htonl(RXRPC_SECURITY_ENCRYPT);	/* token.level */
+	*xdr++ = htonl(0);			/* token.starttime */
+	*xdr++ = htonl(0);			/* " */
+	*xdr++ = htonl(0);			/* token.lifetime */
+	*xdr++ = htonl(0);			/* token.bytelife */
+	*xdr++ = htonl(0);			/* token.expirationtime */
+	*xdr++ = htonl(0);			/* " */
+	*xdr++ = htonl(1);			/* token.identities.count */
+	*xdr++ = htonl(0);			/* token.identities[0].kind */
+	*xdr++ = htonl(uuidsize);		/* token.identities[0].data.len */
+	memcpy(xdr, &server->uuid, uuidsize);
+	xdr += xdr_round_up(uuidsize) / 4;
+	*xdr++ = htonl(0);			/* token.identities[0].display.len */
+
+	xdr = encbase + xdr_round_up(encsize);
+
+	if ((unsigned long)xdr - (unsigned long)appdata != adatasize)
+		pr_err("Appdata size incorrect %lx != %zx\n",
+		       (unsigned long)xdr - (unsigned long)appdata, adatasize);
+
+	aead = crypto_krb5_prepare_encryption(token_krb5, token_key, RXGK_SERVER_ENC_TOKEN,
+					      GFP_KERNEL);
+	if (IS_ERR(aead)) {
+		ret = PTR_ERR(aead);
+		goto out_token;
+	}
+
+	sg_init_one(&sg, encbase, encsize);
+	ret = crypto_krb5_encrypt(token_krb5, aead, &sg, 1, encsize, offset, toksize, false);
+	if (ret < 0)
+		goto out_aead;
+
+	appdata_key = key_alloc(&key_type_user, "rxrpc: afs rxgk appdata",
+				GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, current_cred(),
+				KEY_POS_VIEW | KEY_POS_SEARCH | KEY_USR_VIEW,
+				KEY_ALLOC_NOT_IN_QUOTA, NULL);
+	if (IS_ERR(appdata_key)) {
+		ret = PTR_ERR(appdata_key);
+		goto out_aead;
+	}
+
+	ret = key_instantiate_and_link(appdata_key, appdata, adatasize, NULL, NULL);
+	if (ret < 0) {
+		key_put(appdata_key);
+		goto out_aead;
+	}
+
+	/* Store the appdata before the key pointer */
+	smp_store_release(&server->yfs_rxgk_appdata, appdata_key);
+
+out_aead:
+	crypto_free_aead(aead);
+out_token:
+	kfree_sensitive(appdata);
+out:
+	return ret;
+}
+#endif /* CONFIG_RXGK */
+
+/*
+ * Create the application data to go in a RESPONSE packet a server's CHALLENGE
+ * from the parameters contained in a key.  The key specifies the security
+ * index and other appropriate parameters such as the encoding type for RxGK.
+ */
+int afs_create_server_appdata(struct afs_server *server, struct key *key)
+{
+	u32 krb5_enctype;
+	int ret;
+	u8 security_index;
+
+	if (!key)
+		return 0;
+
+	rxrpc_kernel_query_key(key, &security_index, &krb5_enctype);
+
+	_enter("%u,%u", security_index, krb5_enctype);
+
+	switch (security_index) {
+#ifdef CONFIG_RXGK
+	case RXRPC_SECURITY_YFS_RXGK:
+		/* Read the key pointer before the appdata */
+		if (smp_load_acquire(&server->yfs_rxgk_appdata))
+			return 0;
+		break;
+#endif
+	default:
+		return 0;
+	}
+
+	ret = 0;
+	mutex_lock(&server->cm_token_lock);
+
+	switch (security_index) {
+#ifdef CONFIG_RXGK
+	case RXRPC_SECURITY_YFS_RXGK:
+		/* Read the key pointer before the appdata */
+		if (smp_load_acquire(&server->yfs_rxgk_appdata))
+			break;
+		ret = afs_create_yfs_rxgk_cm_appdata(server, krb5_enctype);
+		break;
+#endif
+	default:
+		break;
+	}
+
+	mutex_unlock(&server->cm_token_lock);
+	return ret;
+}
+
 /*
  * Respond to an RxGK challenge, adding appdata.
  */
diff --git a/fs/afs/fs_probe.c b/fs/afs/fs_probe.c
index a91ad1938d07..e26f7f1e30a2 100644
--- a/fs/afs/fs_probe.c
+++ b/fs/afs/fs_probe.c
@@ -241,9 +241,14 @@ int afs_fs_probe_fileserver(struct afs_net *net, struct afs_server *server,
 	struct afs_endpoint_state *estate, *old;
 	struct afs_addr_list *old_alist = NULL, *alist;
 	unsigned long unprobed;
+	int ret;
 
 	_enter("%pU", &server->uuid);
 
+	ret = afs_create_server_appdata(server, key);
+	if (ret < 0)
+		return ret;
+
 	estate = kzalloc_obj(*estate);
 	if (!estate)
 		return -ENOMEM;
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 601f01e5c15f..a8f303c20ef7 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -549,6 +549,7 @@ struct afs_server {
 	struct timer_list	timer;		/* Management timer */
 	struct mutex		cm_token_lock;	/* Lock governing creation of appdata */
 	struct krb5_buffer	cm_rxgk_appdata; /* Appdata to be included in RESPONSE packet */
+	struct key		*yfs_rxgk_appdata; /* Appdata to be included in RESPONSE packet */
 	time64_t		unuse_time;	/* Time at which last unused */
 	unsigned long		flags;
 #define AFS_SERVER_FL_RESPONDING 0		/* The server is responding */
@@ -1089,6 +1090,7 @@ extern bool afs_cm_incoming_call(struct afs_call *);
 /*
  * cm_security.c
  */
+int afs_create_server_appdata(struct afs_server *server, struct key *key);
 void afs_process_oob_queue(struct work_struct *work);
 #ifdef CONFIG_RXGK
 int afs_create_token_key(struct afs_net *net, struct socket *socket);
diff --git a/fs/afs/server.c b/fs/afs/server.c
index 0fe162ea2a36..b08d9080b0c5 100644
--- a/fs/afs/server.c
+++ b/fs/afs/server.c
@@ -399,6 +399,7 @@ static void afs_server_rcu(struct rcu_head *rcu)
 			       afs_estate_trace_put_server);
 	afs_put_cell(server->cell, afs_cell_trace_put_server);
 	kfree(server->cm_rxgk_appdata.data);
+	key_put(server->yfs_rxgk_appdata);
 	kfree(server);
 }
 
diff --git a/include/net/af_rxrpc.h b/include/net/af_rxrpc.h
index 0fb4c41c9bbf..c4b68049c06f 100644
--- a/include/net/af_rxrpc.h
+++ b/include/net/af_rxrpc.h
@@ -109,6 +109,8 @@ int rxkad_kernel_respond_to_challenge(struct sk_buff *challenge);
 u32 rxgk_kernel_query_challenge(struct sk_buff *challenge);
 int rxgk_kernel_respond_to_challenge(struct sk_buff *challenge,
 				     struct krb5_buffer *appdata);
+void rxrpc_kernel_query_key(const struct key *key, u8 *_security_index,
+			    u32 *_krb5_enctype);
 u8 rxrpc_kernel_query_call_security(struct rxrpc_call *call,
 				    u16 *_service_id, u32 *_enctype);
 
diff --git a/net/rxrpc/key.c b/net/rxrpc/key.c
index a0aa78d89289..16bab72e6f07 100644
--- a/net/rxrpc/key.c
+++ b/net/rxrpc/key.c
@@ -893,3 +893,40 @@ static long rxrpc_read(const struct key *key,
 	_leave(" = %zu", size);
 	return size;
 }
+
+/**
+ * rxrpc_kernel_query_key - Query parameters from an rxrpc key
+ * @key: The key to query
+ * @_security_index: Where to return the security index
+ * @_krb5_enctype: Where to return the krb5 encryption type if applicable
+ *
+ * Query an rxrpc authentication key, extracting the security index from the
+ * first token therein.
+ */
+void rxrpc_kernel_query_key(const struct key *key, u8 *_security_index,
+			    u32 *_krb5_enctype)
+{
+	const struct rxrpc_key_token *token;
+
+	token = key->payload.data[0];
+	if (!token) {
+		*_security_index = 0;
+		*_krb5_enctype = 0;
+		return;
+	}
+
+	*_security_index = token->security_index;
+	switch (token->security_index) {
+	case RXRPC_SECURITY_RXKAD:
+		*_krb5_enctype = 0;
+		break;
+	case RXRPC_SECURITY_YFS_RXGK:
+		*_krb5_enctype = token->rxgk->enctype;
+		break;
+	default:
+		WARN_ON_ONCE(1);
+		*_krb5_enctype = 0;
+		break;
+	}
+}
+EXPORT_SYMBOL(rxrpc_kernel_query_key);



From dhowells at redhat.com  Tue Aug  4 10:26:35 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:35 +0100
Subject: [PATCH net v6 08/10] rxrpc: Pass appdata key to rxrpc_call and thence to rxrpc_bundle
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-9-dhowells@redhat.com>

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

This is a part of the fix.  With this, the appdata key created by the AFS
filesystem or passed in via sendmsg CMSG to a user AF_RXRPC socket is added
to an rxrpc_call struct and will then be added to an rxrpc_bundle struct.

Note that afs_make_op_call() has to be moved so that it can call
afs_use_server().  afs_operation-based calls did not heretofore 'use' the
server and server will be 'un-used' by afs_free_call().

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 fs/afs/internal.h          | 31 ++++++++++++++++---------------
 fs/afs/rxrpc.c             | 21 ++++++++++++++++++++-
 include/net/af_rxrpc.h     |  1 +
 include/trace/events/afs.h |  1 +
 include/uapi/linux/rxrpc.h |  1 +
 net/rxrpc/af_rxrpc.c       |  3 +++
 net/rxrpc/ar-internal.h    |  3 +++
 net/rxrpc/call_object.c    |  2 ++
 net/rxrpc/conn_client.c    |  4 ++++
 net/rxrpc/proc.c           |  5 +++--
 net/rxrpc/sendmsg.c        | 26 +++++++++++++++++++++++++-
 11 files changed, 79 insertions(+), 19 deletions(-)

diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index a8f303c20ef7..f75e007e483e 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -1418,21 +1418,6 @@ static inline void afs_see_call(struct afs_call *call, enum afs_call_trace why)
 		       __builtin_return_address(0));
 }
 
-static inline void afs_make_op_call(struct afs_operation *op, struct afs_call *call,
-				    gfp_t gfp)
-{
-	struct afs_addr_list *alist = op->estate->addresses;
-
-	op->call	= call;
-	op->type	= call->type;
-	call->op	= op;
-	call->key	= op->key;
-	call->intr	= !(op->flags & AFS_OPERATION_UNINTR);
-	call->peer	= rxrpc_kernel_get_peer(alist->addrs[op->addr_index].peer);
-	call->service_id = op->server->service_id;
-	afs_make_call(call, gfp);
-}
-
 static inline void afs_extract_begin(struct afs_call *call, void *buf, size_t size)
 {
 	call->iov_len = size;
@@ -1764,6 +1749,22 @@ static inline struct inode *AFS_VNODE_TO_I(struct afs_vnode *vnode)
 	return &vnode->netfs.inode;
 }
 
+static inline void afs_make_op_call(struct afs_operation *op, struct afs_call *call,
+				    gfp_t gfp)
+{
+	struct afs_addr_list *alist = op->estate->addresses;
+
+	op->call	= call;
+	op->type	= call->type;
+	call->op	= op;
+	call->server	= afs_use_server(op->server, false, afs_server_trace_use_call);
+	call->key	= op->key;
+	call->intr	= !(op->flags & AFS_OPERATION_UNINTR);
+	call->peer	= rxrpc_kernel_get_peer(alist->addrs[op->addr_index].peer);
+	call->service_id = op->server->service_id;
+	afs_make_call(call, gfp);
+}
+
 /*
  * Note that a dentry got changed.  We need to set d_fsdata to the data version
  * number derived from the result of the operation.  It doesn't matter if
diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index b435f4b2da7f..9f2c4972f2b4 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -347,6 +347,7 @@ void afs_make_call(struct afs_call *call, gfp_t gfp)
 	struct rxrpc_call *rxcall;
 	struct msghdr msg;
 	struct kvec iov[1];
+	struct key *app_data = NULL;
 	size_t len;
 	s64 tx_total_len;
 	int ret;
@@ -378,8 +379,26 @@ void afs_make_call(struct afs_call *call, gfp_t gfp)
 		call->drop_ref = true;
 	}
 
+	if (call->key && call->server) {
+		u32 krb5_enctype = 0;
+		u8 security_index = 0;
+
+		rxrpc_kernel_query_key(call->key, &security_index, &krb5_enctype);
+		switch (security_index) {
+#ifdef CONFIG_RXGK
+		case RXRPC_SECURITY_YFS_RXGK:
+			/* Read the key pointer before the appdata */
+			app_data = smp_load_acquire(&call->server->yfs_rxgk_appdata);
+			break;
+#endif
+		default:
+			break;
+		}
+	}
+
 	/* create a call */
-	rxcall = rxrpc_kernel_begin_call(call->net->socket, call->peer, call->key,
+	rxcall = rxrpc_kernel_begin_call(call->net->socket, call->peer,
+					 call->key, app_data,
 					 (unsigned long)call,
 					 tx_total_len,
 					 call->max_lifespan,
diff --git a/include/net/af_rxrpc.h b/include/net/af_rxrpc.h
index c4b68049c06f..19c61a2f5af3 100644
--- a/include/net/af_rxrpc.h
+++ b/include/net/af_rxrpc.h
@@ -55,6 +55,7 @@ void rxrpc_kernel_set_notifications(struct socket *sock,
 struct rxrpc_call *rxrpc_kernel_begin_call(struct socket *sock,
 					   struct rxrpc_peer *peer,
 					   struct key *key,
+					   struct key *app_data,
 					   unsigned long user_call_ID,
 					   s64 tx_total_len,
 					   u32 hard_timeout,
diff --git a/include/trace/events/afs.h b/include/trace/events/afs.h
index 1b3c48b5591d..04b0bb682b81 100644
--- a/include/trace/events/afs.h
+++ b/include/trace/events/afs.h
@@ -148,6 +148,7 @@ enum yfs_cm_operation {
 	EM(afs_server_trace_unuse_slist_isort,	"UNU isort") \
 	EM(afs_server_trace_update,		"UPDATE   ") \
 	EM(afs_server_trace_use_by_uuid,	"USE uuid ") \
+	EM(afs_server_trace_use_call,		"USE call ") \
 	EM(afs_server_trace_use_cm_call,	"USE cm-cl") \
 	EM(afs_server_trace_use_get_caps,	"USE gcaps") \
 	EM(afs_server_trace_use_give_up_cb,	"USE gvupc") \
diff --git a/include/uapi/linux/rxrpc.h b/include/uapi/linux/rxrpc.h
index d9735abd4c79..bcdfdf9c67a1 100644
--- a/include/uapi/linux/rxrpc.h
+++ b/include/uapi/linux/rxrpc.h
@@ -63,6 +63,7 @@ enum rxrpc_cmsg_type {
 	RXRPC_RESPOND		= 17,	/* Cs-: Respond to a challenge */
 	RXRPC_RESPONDED		= 18,	/* S-r: Data received in RESPONSE */
 	RXRPC_RESP_RXGK_APPDATA	= 19,	/* Cs-: RESPONSE: RxGK app data to include */
+	RXRPC_RESPONSE_APPDATA	= 20,	/* Cs-: User key holding app data for RESPONSE */
 	RXRPC__SUPPORTED
 };
 
diff --git a/net/rxrpc/af_rxrpc.c b/net/rxrpc/af_rxrpc.c
index 9ab0f22c881e..a19c0fd3c51a 100644
--- a/net/rxrpc/af_rxrpc.c
+++ b/net/rxrpc/af_rxrpc.c
@@ -318,6 +318,7 @@ EXPORT_SYMBOL(rxrpc_kernel_put_peer);
  * @sock: The socket on which to make the call
  * @peer: The peer to contact
  * @key: The security context to use (defaults to socket setting)
+ * @app_data: The security response application data (or NULL)
  * @user_call_ID: The ID to use
  * @tx_total_len: Total length of data to transmit during the call (or -1)
  * @hard_timeout: The maximum lifespan of the call in sec
@@ -340,6 +341,7 @@ EXPORT_SYMBOL(rxrpc_kernel_put_peer);
 struct rxrpc_call *rxrpc_kernel_begin_call(struct socket *sock,
 					   struct rxrpc_peer *peer,
 					   struct key *key,
+					   struct key *app_data,
 					   unsigned long user_call_ID,
 					   s64 tx_total_len,
 					   u32 hard_timeout,
@@ -368,6 +370,7 @@ struct rxrpc_call *rxrpc_kernel_begin_call(struct socket *sock,
 		key = NULL; /* a no-security key */
 
 	memset(&p, 0, sizeof(p));
+	p.app_data		= app_data;
 	p.user_call_ID		= user_call_ID;
 	p.tx_total_len		= tx_total_len;
 	p.interruptibility	= interruptibility;
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index a6f830c1621f..6e16df7173d3 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -516,6 +516,7 @@ struct rxrpc_bundle {
 	struct rxrpc_local	*local;		/* Representation of local endpoint */
 	struct rxrpc_peer	*peer;		/* Remote endpoint */
 	struct key		*key;		/* Security details */
+	struct key		*app_data;	/* Security response app data */
 	struct list_head	proc_link;	/* Link in net->bundle_proc_list */
 	const struct rxrpc_security *security;	/* applied security module */
 	refcount_t		ref;
@@ -720,6 +721,7 @@ struct rxrpc_call {
 	struct rxrpc_sock __rcu	*socket;	/* socket responsible */
 	struct rxrpc_net	*rxnet;		/* Network namespace to which call belongs */
 	struct key		*key;		/* Security details */
+	struct key		*app_data;	/* Security response app data */
 	const struct rxrpc_security *security;	/* applied security module */
 	struct mutex		user_mutex;	/* User access mutex */
 	struct sockaddr_rxrpc	dest_srx;	/* Destination address */
@@ -914,6 +916,7 @@ enum rxrpc_command {
 };
 
 struct rxrpc_call_params {
+	struct key		*app_data;	/* Security response app data */
 	s64			tx_total_len;	/* Total Tx data length (if send data) */
 	unsigned long		user_call_ID;	/* User's call ID */
 	struct {
diff --git a/net/rxrpc/call_object.c b/net/rxrpc/call_object.c
index 817ed9acb91e..9f6130d90c4c 100644
--- a/net/rxrpc/call_object.c
+++ b/net/rxrpc/call_object.c
@@ -211,6 +211,7 @@ static struct rxrpc_call *rxrpc_alloc_client_call(struct rxrpc_sock *rx,
 	call->interruptibility	= p->interruptibility;
 	call->tx_total_len	= p->tx_total_len;
 	call->key		= key_get(cp->key);
+	call->app_data		= key_get(p->app_data);
 	call->peer		= rxrpc_get_peer(cp->peer, rxrpc_peer_get_call);
 	call->local		= rxrpc_get_local(cp->local, rxrpc_local_get_call);
 	call->security_level	= cp->security_level;
@@ -697,6 +698,7 @@ static void rxrpc_destroy_call(struct work_struct *work)
 	rxrpc_put_peer(call->peer, rxrpc_peer_put_call);
 	rxrpc_put_local(call->local, rxrpc_local_put_call);
 	key_put(call->key);
+	key_put(call->app_data);
 	call_rcu(&call->rcu, rxrpc_rcu_free_call);
 }
 
diff --git a/net/rxrpc/conn_client.c b/net/rxrpc/conn_client.c
index 48519f0de185..dd79630d3066 100644
--- a/net/rxrpc/conn_client.c
+++ b/net/rxrpc/conn_client.c
@@ -81,6 +81,7 @@ static struct rxrpc_bundle *rxrpc_alloc_bundle(struct rxrpc_call *call,
 		bundle->local		= call->local;
 		bundle->peer		= rxrpc_get_peer(call->peer, rxrpc_peer_get_bundle);
 		bundle->key		= key_get(call->key);
+		bundle->app_data	= key_get(call->app_data);
 		bundle->security	= call->security;
 		bundle->exclusive	= test_bit(RXRPC_CALL_EXCLUSIVE, &call->flags);
 		bundle->upgrade		= test_bit(RXRPC_CALL_UPGRADE, &call->flags);
@@ -118,6 +119,7 @@ static void rxrpc_free_bundle(struct rxrpc_bundle *bundle)
 	write_unlock(&bundle->local->rxnet->conn_lock);
 	rxrpc_put_peer(bundle->peer, rxrpc_peer_put_bundle);
 	key_put(bundle->key);
+	key_put(bundle->app_data);
 	kfree(bundle);
 }
 
@@ -274,6 +276,7 @@ int rxrpc_look_up_bundle(struct rxrpc_call *call, gfp_t gfp)
 #define cmp(X, Y) ((long)(X) - (long)(Y))
 		diff = (cmp(bundle->peer, call->peer) ?:
 			cmp(bundle->key, call->key) ?:
+			cmp(bundle->app_data, call->app_data) ?:
 			cmp(bundle->security_level, call->security_level) ?:
 			cmp(bundle->upgrade, upgrade));
 #undef cmp
@@ -303,6 +306,7 @@ int rxrpc_look_up_bundle(struct rxrpc_call *call, gfp_t gfp)
 #define cmp(X, Y) ((long)(X) - (long)(Y))
 		diff = (cmp(bundle->peer, call->peer) ?:
 			cmp(bundle->key, call->key) ?:
+			cmp(bundle->app_data, call->app_data) ?:
 			cmp(bundle->security_level, call->security_level) ?:
 			cmp(bundle->upgrade, upgrade));
 #undef cmp
diff --git a/net/rxrpc/proc.c b/net/rxrpc/proc.c
index e9a27fa7b25d..f300491b06bd 100644
--- a/net/rxrpc/proc.c
+++ b/net/rxrpc/proc.c
@@ -238,7 +238,7 @@ static int rxrpc_bundle_seq_show(struct seq_file *seq, void *v)
 		seq_puts(seq,
 			 "Proto Local                                          "
 			 " Remote                                         "
-			 " SvID Ref Act Flg Key      |"
+			 " SvID Ref Act Flg Key      AppData  |"
 			 " Bundle   Conn_0   Conn_1   Conn_2   Conn_3\n"
 			 );
 		return 0;
@@ -250,7 +250,7 @@ static int rxrpc_bundle_seq_show(struct seq_file *seq, void *v)
 	scnprintf(rbuff, sizeof(rbuff), "%pISpc", &bundle->peer->srx.transport);
 	seq_printf(seq,
 		   "UDP   %-47.47s %-47.47s %4x %3u %3d"
-		   " %c%c%c %08x | %08x %08x %08x %08x %08x\n",
+		   " %c%c%c %08x %08x | %08x %08x %08x %08x %08x\n",
 		   lbuff,
 		   rbuff,
 		   bundle->service_id,
@@ -260,6 +260,7 @@ static int rxrpc_bundle_seq_show(struct seq_file *seq, void *v)
 		   bundle->exclusive ? 'e' : '-',
 		   bundle->upgrade ? 'u' : '-',
 		   key_serial(bundle->key),
+		   key_serial(bundle->app_data),
 		   bundle->debug_id,
 		   bundle->conn_ids[0],
 		   bundle->conn_ids[1],
diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index 79fd31b02d11..db002fc83240 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -12,6 +12,7 @@
 #include <linux/skbuff.h>
 #include <linux/export.h>
 #include <linux/sched/signal.h>
+#include <keys/user-type.h>
 
 #include <net/sock.h>
 #include <net/af_rxrpc.h>
@@ -546,6 +547,8 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 static int rxrpc_sendmsg_cmsg(struct msghdr *msg, struct rxrpc_send_params *p)
 {
 	struct cmsghdr *cmsg;
+	key_serial_t key_id;
+	key_ref_t key;
 	bool got_user_ID = false;
 	int len;
 
@@ -630,6 +633,23 @@ static int rxrpc_sendmsg_cmsg(struct msghdr *msg, struct rxrpc_send_params *p)
 				return -ERANGE;
 			break;
 
+		case RXRPC_RESPONSE_APPDATA:
+			if (len != sizeof(key_serial_t))
+				return -EINVAL;
+			if (p->call.app_data)
+				return -EINVAL;
+			key_id = *(key_serial_t *)CMSG_DATA(cmsg);
+			key = lookup_user_key(key_id, 0, KEY_NEED_SEARCH);
+			if (IS_ERR(key))
+				return PTR_ERR(key);
+			if (key_ref_to_ptr(key)->type != &key_type_user &&
+			    key_ref_to_ptr(key)->type != &key_type_logon) {
+				key_ref_put(key);
+				return -EINVAL;
+			}
+			p->call.app_data = key_ref_to_ptr(key);
+			break;
+
 		default:
 			return -EINVAL;
 		}
@@ -737,8 +757,10 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
 			goto error_release_sock;
 		call = rxrpc_new_client_call_for_sendmsg(rx, msg, &p);
 		/* The socket is now unlocked... */
-		if (IS_ERR(call))
+		if (IS_ERR(call)) {
+			key_put(p.call.app_data);
 			return PTR_ERR(call);
+		}
 		/* ... and we have the call lock. */
 		p.call.nr_timeouts = 0;
 		ret = 0;
@@ -822,11 +844,13 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
 	mutex_unlock(&call->user_mutex);
 error_put:
 	rxrpc_put_call(call, rxrpc_call_put_sendmsg);
+	key_put(p.call.app_data);
 	_leave(" = %d", ret);
 	return ret;
 
 error_release_sock:
 	release_sock(&rx->sk);
+	key_put(p.call.app_data);
 	return ret;
 }
 



From dhowells at redhat.com  Tue Aug  4 10:26:36 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:36 +0100
Subject: [PATCH net v6 09/10] rxrpc: Fix CHALLENGE packet overqueuing and simplify RESPONSE generation
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-10-dhowells@redhat.com>

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

This is the main part of the fix.  It switches the code over from using the
OOB communication mechanism to get the appdata on the fly from the
application (either the AFS filesystem or userspace) to using the appdata
key preemptively provided by the application.

The following changes are made:

 (1) Revert to making the connection event processor work item parse the
     CHALLENGE and generate the RESPONSE directly.

 (2) Add another (optional) parameter that is passed in when an rxrpc
     client call is created and ends up attached to the connection bundle
     and is a user-type or logon-type key containing the application data.

 (3) RESPONSE generation looks at the bundle and if the app-data is there,
     it will include it (if the security class is YFS-RxGK; RxKAD ignores
     it).

The AFS filesystem driver creates an app-data key when it probes a
fileserver and attaches it to the afs_server struct.  This is picked up
when a call is made to that server and thence passed to rxrpc.

Direct userspace users of AF_RXRPC can partake by creating a user-type or
logon-type key containing the app-data they want to use and passing its
serial ID in a CMSG of type RXRPC_RESPONSE_APPDATA in the initial sendmsg()
of a call.

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 include/trace/events/rxrpc.h |  1 +
 net/rxrpc/af_rxrpc.c         |  5 +--
 net/rxrpc/conn_event.c       | 68 ++----------------------------------
 net/rxrpc/rxgk.c             | 67 ++++++++++++++++++++++-------------
 4 files changed, 48 insertions(+), 93 deletions(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index ba2c6e3c78a6..48e774034195 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -78,6 +78,7 @@
 	EM(rxgk_abort_2_short_encdata,		"rxgk2-short-encdata")	\
 	EM(rxgk_abort_2_short_header,		"rxgk2-short-hdr")	\
 	EM(rxgk_abort_bad_key_number,		"rxgk-bad-key-num")	\
+	EM(rxgk_abort_chall_bad_app_key,	"rxgk-chall-bad-app-key")\
 	EM(rxgk_abort_chall_key_expired,	"rxgk-chall-key-exp")	\
 	EM(rxgk_abort_chall_no_key,		"rxgk-chall-nokey")	\
 	EM(rxgk_abort_chall_short,		"rxgk-chall-short")	\
diff --git a/net/rxrpc/af_rxrpc.c b/net/rxrpc/af_rxrpc.c
index a19c0fd3c51a..9eb6ca3c5ebf 100644
--- a/net/rxrpc/af_rxrpc.c
+++ b/net/rxrpc/af_rxrpc.c
@@ -598,10 +598,7 @@ static int rxrpc_sendmsg(struct socket *sock, struct msghdr *m, size_t len)
 		fallthrough;
 	case RXRPC_SERVER_BOUND:
 	case RXRPC_SERVER_LISTENING:
-		if (m->msg_flags & MSG_OOB)
-			ret = rxrpc_sendmsg_oob(rx, m, len);
-		else
-			ret = rxrpc_do_sendmsg(rx, m, len);
+		ret = rxrpc_do_sendmsg(rx, m, len);
 		/* The socket has been unlocked */
 		goto out;
 	default:
diff --git a/net/rxrpc/conn_event.c b/net/rxrpc/conn_event.c
index 611c790bc6d0..9146b9d4c2ac 100644
--- a/net/rxrpc/conn_event.c
+++ b/net/rxrpc/conn_event.c
@@ -279,10 +279,7 @@ static int rxrpc_process_event(struct rxrpc_connection *conn,
 
 	switch (sp->hdr.type) {
 	case RXRPC_PACKET_TYPE_CHALLENGE:
-		ret = conn->security->respond_to_challenge(conn, skb);
-		sp->chall.conn = NULL;
-		rxrpc_put_connection(conn, rxrpc_conn_put_challenge_input);
-		return ret;
+		return conn->security->respond_to_challenge(conn, skb);
 
 	case RXRPC_PACKET_TYPE_RESPONSE:
 		spin_lock_irq(&conn->state_lock);
@@ -425,66 +422,6 @@ static void rxrpc_post_packet_to_conn(struct rxrpc_connection *conn,
 	rxrpc_queue_conn(conn, rxrpc_conn_queue_rx_work);
 }
 
-/*
- * Post a CHALLENGE packet to the socket of one of a connection's calls so that
- * it can get application data to include in the packet, possibly querying
- * userspace.
- */
-static bool rxrpc_post_challenge(struct rxrpc_connection *conn,
-				 struct sk_buff *skb)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
-	struct rxrpc_call *call = NULL;
-	struct rxrpc_sock *rx;
-	bool respond = false, queued = false;
-
-	sp->chall.conn =
-		rxrpc_get_connection(conn, rxrpc_conn_get_challenge_input);
-
-	if (!conn->security->challenge_to_recvmsg) {
-		rxrpc_post_packet_to_conn(conn, skb);
-		return true;
-	}
-
-	rcu_read_lock();
-
-	for (int i = 0; i < ARRAY_SIZE(conn->channels); i++) {
-		if (conn->channels[i].call) {
-			call = conn->channels[i].call;
-			rx = rcu_dereference(call->socket);
-			if (!rx) {
-				call = NULL;
-				continue;
-			}
-
-			respond = true;
-			if (test_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags))
-				break;
-			call = NULL;
-		}
-	}
-
-	if (!respond) {
-		rcu_read_unlock();
-		rxrpc_put_connection(conn, rxrpc_conn_put_challenge_input);
-		sp->chall.conn = NULL;
-		return false;
-	}
-
-	if (call)
-		queued = rxrpc_notify_socket_oob(call, skb);
-	rcu_read_unlock();
-	if (call && !queued) {
-		rxrpc_put_connection(conn, rxrpc_conn_put_challenge_input);
-		sp->chall.conn = NULL;
-		return false;
-	}
-
-	if (!call)
-		rxrpc_post_packet_to_conn(conn, skb);
-	return true;
-}
-
 /*
  * Input a connection-level packet.
  */
@@ -513,7 +450,8 @@ bool rxrpc_input_conn_packet(struct rxrpc_connection *conn, struct sk_buff *skb)
 		}
 		if (!conn->security->validate_challenge(conn, skb))
 			return false;
-		return rxrpc_post_challenge(conn, skb);
+		rxrpc_post_packet_to_conn(conn, skb);
+		return true;
 
 	case RXRPC_PACKET_TYPE_RESPONSE:
 		if (rxrpc_is_conn_aborted(conn)) {
diff --git a/net/rxrpc/rxgk.c b/net/rxrpc/rxgk.c
index 77a67ace1d24..031c6719d33e 100644
--- a/net/rxrpc/rxgk.c
+++ b/net/rxrpc/rxgk.c
@@ -11,6 +11,7 @@
 #include <linux/skbuff.h>
 #include <linux/slab.h>
 #include <linux/key-type.h>
+#include <keys/user-type.h>
 #include "ar-internal.h"
 #include "rxgk_common.h"
 
@@ -837,7 +838,7 @@ static noinline ssize_t rxgk_insert_response_header(struct rxrpc_connection *con
  */
 static ssize_t rxgk_construct_authenticator(struct rxrpc_connection *conn,
 					    struct sk_buff *challenge,
-					    const struct krb5_buffer *appdata,
+					    const struct user_key_payload *appdata,
 					    struct sk_buff *response,
 					    size_t offset)
 {
@@ -859,20 +860,20 @@ static ssize_t rxgk_construct_authenticator(struct rxrpc_connection *conn,
 	if (ret < 0)
 		return -EPROTO;
 
-	a.appdata_len = htonl(appdata->len);
+	a.appdata_len = htonl(appdata->datalen);
 
 	ret = skb_store_bits(response, offset, &a, sizeof(a));
 	if (ret < 0)
 		return ret;
 	offset += sizeof(a);
 
-	if (appdata->len) {
-		ret = skb_store_bits(response, offset, appdata->data, appdata->len);
+	if (appdata->datalen) {
+		ret = skb_store_bits(response, offset, appdata->data, appdata->datalen);
 		if (ret < 0)
 			return ret;
-		offset += appdata->len;
+		offset += appdata->datalen;
 
-		ret = rxgk_pad_out(response, appdata->len, offset);
+		ret = rxgk_pad_out(response, appdata->datalen, offset);
 		if (ret < 0)
 			return ret;
 		offset += ret;
@@ -890,7 +891,7 @@ static ssize_t rxgk_construct_authenticator(struct rxrpc_connection *conn,
 	ret = skb_store_bits(response, offset, &b, sizeof(b));
 	if (ret < 0)
 		return ret;
-	return sizeof(a) + xdr_round_up(appdata->len) + sizeof(b);
+	return sizeof(a) + xdr_round_up(appdata->datalen) + sizeof(b);
 }
 
 static ssize_t rxgk_encrypt_authenticator(struct rxrpc_connection *conn,
@@ -923,7 +924,7 @@ static ssize_t rxgk_encrypt_authenticator(struct rxrpc_connection *conn,
  */
 static int rxgk_construct_response(struct rxrpc_connection *conn,
 				   struct sk_buff *challenge,
-				   struct krb5_buffer *appdata)
+				   const struct user_key_payload *appdata)
 {
 	struct rxrpc_skb_priv *csp, *rsp;
 	struct rxgk_context *gk;
@@ -936,11 +937,11 @@ static int rxgk_construct_response(struct rxrpc_connection *conn,
 	if (IS_ERR(gk))
 		return PTR_ERR(gk);
 
-	auth_len = 20 + (4 + appdata->len) + 12 + (1 + 4) * 4;
+	auth_len = 20 + xdr_object_len(appdata->datalen) + 12 + (1 + 4) * 4;
 	authx_len = crypto_krb5_how_much_buffer(gk->krb5, KRB5_ENCRYPT_MODE,
 						auth_len, &auth_offset);
 	len = sizeof(struct rxrpc_wire_header) +
-		8 + (4 + xdr_round_up(gk->key->ticket.len)) + (4 + authx_len);
+		8 + xdr_object_len(gk->key->ticket.len) + xdr_object_len(authx_len);
 
 	response = alloc_skb_with_frags(0, len, 0, &ret, GFP_NOFS);
 	if (!response)
@@ -1011,24 +1012,44 @@ static int rxgk_construct_response(struct rxrpc_connection *conn,
  * Respond to a challenge packet.
  */
 static int rxgk_respond_to_challenge(struct rxrpc_connection *conn,
-				     struct sk_buff *challenge,
-				     struct krb5_buffer *appdata)
+				     struct sk_buff *challenge)
 {
-	_enter("{%d,%x}", conn->debug_id, key_serial(conn->key));
+	struct user_key_payload dummy = {}, *appdata = &dummy;
+	int ret;
+
+	_enter("{%d,%u,%x,%x}",
+	       conn->debug_id, conn->service_id,
+	       key_serial(conn->key), key_serial(conn->bundle->app_data));
 
 	if (key_validate(conn->key) < 0)
 		return rxrpc_abort_conn(conn, NULL, RXGK_EXPIRED, -EPROTO,
 					rxgk_abort_chall_key_expired);
 
-	return rxgk_construct_response(conn, challenge, appdata);
-}
+	if (conn->bundle->app_data) {
+		struct key *app_key = conn->bundle->app_data;
+
+		/* The wrong sort of key shouldn't get this far. */
+		if (WARN_ON_ONCE(app_key->type != &key_type_user &&
+				 app_key->type != &key_type_logon))
+			return rxrpc_abort_conn(conn, NULL, RXGK_EXPIRED, -EKEYREJECTED,
+						rxgk_abort_chall_bad_app_key);
+
+		rcu_read_lock();
+		appdata = (struct user_key_payload *)
+			user_key_payload_rcu(conn->bundle->app_data);
+		if (appdata && !refcount_inc_not_zero(&appdata->ref))
+			appdata = NULL;
+		rcu_read_unlock();
+		if (!appdata)
+			return rxrpc_abort_conn(conn, NULL, RXGK_EXPIRED, -EKEYREVOKED,
+						rxgk_abort_chall_key_expired);
+	}
 
-static int rxgk_respond_to_challenge_no_appdata(struct rxrpc_connection *conn,
-						struct sk_buff *challenge)
-{
-	struct krb5_buffer appdata = {};
+	ret = rxgk_construct_response(conn, challenge, appdata);
 
-	return rxgk_respond_to_challenge(conn, challenge, &appdata);
+	if (appdata != &dummy)
+		put_user_key_payload(appdata);
+	return ret;
 }
 
 /**
@@ -1044,9 +1065,7 @@ static int rxgk_respond_to_challenge_no_appdata(struct rxrpc_connection *conn,
 int rxgk_kernel_respond_to_challenge(struct sk_buff *challenge,
 				     struct krb5_buffer *appdata)
 {
-	struct rxrpc_skb_priv *csp = rxrpc_skb(challenge);
-
-	return rxgk_respond_to_challenge(csp->chall.conn, challenge, appdata);
+	return -EINVAL;
 }
 EXPORT_SYMBOL(rxgk_kernel_respond_to_challenge);
 
@@ -1348,7 +1367,7 @@ const struct rxrpc_security rxgk_yfs = {
 	.validate_challenge		= rxgk_validate_challenge,
 	.challenge_to_recvmsg		= rxgk_challenge_to_recvmsg,
 	.sendmsg_respond_to_challenge	= rxgk_sendmsg_respond_to_challenge,
-	.respond_to_challenge		= rxgk_respond_to_challenge_no_appdata,
+	.respond_to_challenge		= rxgk_respond_to_challenge,
 	.verify_response		= rxgk_verify_response,
 	.clear				= rxgk_clear,
 	.default_decode_ticket		= rxgk_yfs_decode_ticket,



From dhowells at redhat.com  Tue Aug  4 10:26:37 2026
From: dhowells at redhat.com (David Howells)
Date: Tue,  4 Aug 2026 18:26:37 +0100
Subject: [PATCH net v6 10/10] rxrpc: Remove OOB challenge/response code
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260804172639.2844491-11-dhowells@redhat.com>

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

This is the final part of the fix.  It removes all the OOB communication
stuff along with the UAPI elements related to them.

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 fs/afs/cm_security.c         | 259 -----------------------
 fs/afs/internal.h            |   3 -
 fs/afs/main.c                |   1 -
 fs/afs/rxrpc.c               |  19 --
 fs/afs/server.c              |   1 -
 include/net/af_rxrpc.h       |  18 --
 include/trace/events/rxrpc.h |  10 +-
 include/uapi/linux/rxrpc.h   |   5 -
 net/rxrpc/Makefile           |   1 -
 net/rxrpc/af_rxrpc.c         |  41 +---
 net/rxrpc/ar-internal.h      |  23 ---
 net/rxrpc/conn_event.c       |   2 +-
 net/rxrpc/insecure.c         |   7 -
 net/rxrpc/oob.c              | 387 -----------------------------------
 net/rxrpc/recvmsg.c          |  82 +-------
 net/rxrpc/rxgk.c             |  73 -------
 net/rxrpc/rxkad.c            |  27 ---
 net/rxrpc/server_key.c       |  40 ----
 18 files changed, 7 insertions(+), 992 deletions(-)
 delete mode 100644 net/rxrpc/oob.c

diff --git a/fs/afs/cm_security.c b/fs/afs/cm_security.c
index 8f04fec20a2a..0d4700b062f6 100644
--- a/fs/afs/cm_security.c
+++ b/fs/afs/cm_security.c
@@ -20,11 +20,6 @@
 #define xdr_round_up(x) (round_up((x), sizeof(__be32)))
 #define xdr_len_object(x) (4 + round_up((x), sizeof(__be32)))
 
-#ifdef CONFIG_RXGK
-static int afs_create_yfs_cm_token(struct sk_buff *challenge,
-				   struct afs_server *server);
-#endif
-
 #ifdef CONFIG_RXGK
 /*
  * As the YFS RxGK appdata to be passed in the YFS.FS-service RESPONSE packet,
@@ -255,95 +250,6 @@ int afs_create_server_appdata(struct afs_server *server, struct key *key)
 	return ret;
 }
 
-/*
- * Respond to an RxGK challenge, adding appdata.
- */
-static int afs_respond_to_challenge(struct sk_buff *challenge)
-{
-#ifdef CONFIG_RXGK
-	struct krb5_buffer appdata = {};
-	struct afs_server *server;
-#endif
-	struct rxrpc_peer *peer;
-	unsigned long peer_data;
-	u16 service_id;
-	u8 security_index;
-
-	rxrpc_kernel_query_challenge(challenge, &peer, &peer_data,
-				     &service_id, &security_index);
-
-	_enter("%u,%u", service_id, security_index);
-
-	switch (service_id) {
-		/* We don't send CM_SERVICE RPCs, so don't expect a challenge
-		 * therefrom.
-		 */
-	case FS_SERVICE:
-	case VL_SERVICE:
-	case YFS_FS_SERVICE:
-	case YFS_VL_SERVICE:
-		break;
-	default:
-		pr_warn("Can't respond to unknown challenge %u:%u",
-			service_id, security_index);
-		return rxrpc_kernel_reject_challenge(challenge, RX_USER_ABORT, -EPROTO,
-						     afs_abort_unsupported_sec_class);
-	}
-
-	switch (security_index) {
-#ifdef CONFIG_RXKAD
-	case RXRPC_SECURITY_RXKAD:
-		return rxkad_kernel_respond_to_challenge(challenge);
-#endif
-
-#ifdef CONFIG_RXGK
-	case RXRPC_SECURITY_RXGK:
-		return rxgk_kernel_respond_to_challenge(challenge, &appdata);
-
-	case RXRPC_SECURITY_YFS_RXGK:
-		switch (service_id) {
-		case FS_SERVICE:
-		case YFS_FS_SERVICE:
-			server = (struct afs_server *)peer_data;
-			if (!server->cm_rxgk_appdata.data) {
-				mutex_lock(&server->cm_token_lock);
-				if (!server->cm_rxgk_appdata.data)
-					afs_create_yfs_cm_token(challenge, server);
-				mutex_unlock(&server->cm_token_lock);
-			}
-			if (server->cm_rxgk_appdata.data)
-				appdata = server->cm_rxgk_appdata;
-			break;
-		}
-		return rxgk_kernel_respond_to_challenge(challenge, &appdata);
-#endif
-
-	default:
-		return rxrpc_kernel_reject_challenge(challenge, RX_USER_ABORT, -EPROTO,
-						     afs_abort_unsupported_sec_class);
-	}
-}
-
-/*
- * Process the OOB message queue, processing challenge packets.
- */
-void afs_process_oob_queue(struct work_struct *work)
-{
-	struct afs_net *net = container_of(work, struct afs_net, rx_oob_work);
-	struct sk_buff *oob;
-	enum rxrpc_oob_type type;
-
-	while (READ_ONCE(net->live) &&
-	       (oob = rxrpc_kernel_dequeue_oob(net->socket, &type))) {
-		switch (type) {
-		case RXRPC_OOB_CHALLENGE:
-			afs_respond_to_challenge(oob);
-			break;
-		}
-		rxrpc_kernel_free_oob(oob);
-	}
-}
-
 #ifdef CONFIG_RXGK
 /*
  * Create a securities keyring for the cache manager and attach a key to it for
@@ -405,169 +311,4 @@ int afs_create_token_key(struct afs_net *net, struct socket *socket)
 	key_put(ring);
 	return ret;
 }
-
-/*
- * Create an YFS RxGK GSS token to use as a ticket to the specified fileserver.
- */
-static int afs_create_yfs_cm_token(struct sk_buff *challenge,
-				   struct afs_server *server)
-{
-	const struct krb5_enctype *conn_krb5, *token_krb5;
-	const struct krb5_buffer *token_key;
-	struct crypto_aead *aead;
-	struct scatterlist sg;
-	struct afs_net *net = server->cell->net;
-	const struct key *key = net->fs_cm_token_key;
-	size_t keysize, uuidsize, authsize, toksize, encsize, contsize, adatasize, offset;
-	__be32 caps[1] = {
-		[0] = htonl(AFS_CAP_ERROR_TRANSLATION),
-	};
-	__be32 *xdr;
-	void *appdata, *K0, *encbase;
-	u32 enctype;
-	int ret;
-
-	if (!key)
-		return -ENOKEY;
-
-	/* Assume that the fileserver is happy to use the same encoding type as
-	 * we were told to use by the token obtained by the user.
-	 */
-	enctype = rxgk_kernel_query_challenge(challenge);
-
-	conn_krb5 = crypto_krb5_find_enctype(enctype);
-	if (!conn_krb5)
-		return -ENOPKG;
-	token_krb5 = key->payload.data[0];
-	token_key = (const struct krb5_buffer *)&key->payload.data[2];
-
-	/* struct rxgk_key {
-	 *	afs_uint32	enctype;
-	 *	opaque		key<>;
-	 * };
-	 */
-	keysize = 4 + xdr_len_object(conn_krb5->key_len);
-
-	/* struct RXGK_AuthName {
-	 *	afs_int32	kind;
-	 *	opaque		data<AUTHDATAMAX>;
-	 *	opaque		display<AUTHPRINTABLEMAX>;
-	 * };
-	 */
-	uuidsize = sizeof(server->uuid);
-	authsize = 4 + xdr_len_object(uuidsize) + xdr_len_object(0);
-
-	/* struct RXGK_Token {
-	 *	rxgk_key		K0;
-	 *	RXGK_Level		level;
-	 *	rxgkTime		starttime;
-	 *	afs_int32		lifetime;
-	 *	afs_int32		bytelife;
-	 *	rxgkTime		expirationtime;
-	 *	struct RXGK_AuthName	identities<>;
-	 * };
-	 */
-	toksize = keysize + 8 + 4 + 4 + 8 + xdr_len_object(authsize);
-
-	offset = 0;
-	encsize = crypto_krb5_how_much_buffer(token_krb5, KRB5_ENCRYPT_MODE, toksize, &offset);
-
-	/* struct RXGK_TokenContainer {
-	 *	afs_int32	kvno;
-	 *	afs_int32	enctype;
-	 *	opaque		encrypted_token<>;
-	 * };
-	 */
-	contsize = 4 + 4 + xdr_len_object(encsize);
-
-	/* struct YFSAppData {
-	 *	opr_uuid	initiatorUuid;
-	 *	opr_uuid	acceptorUuid;
-	 *	Capabilities	caps;
-	 *	afs_int32	enctype;
-	 *	opaque		callbackKey<>;
-	 *	opaque		callbackToken<>;
-	 * };
-	 */
-	adatasize = 16 + 16 +
-		xdr_len_object(sizeof(caps)) +
-		4 +
-		xdr_len_object(conn_krb5->key_len) +
-		xdr_len_object(contsize);
-
-	ret = -ENOMEM;
-	appdata = kzalloc(adatasize, GFP_KERNEL);
-	if (!appdata)
-		goto out;
-	xdr = appdata;
-
-	memcpy(xdr, &net->uuid, 16);		/* appdata.initiatorUuid */
-	xdr += 16 / 4;
-	memcpy(xdr, &server->uuid, 16);		/* appdata.acceptorUuid */
-	xdr += 16 / 4;
-	*xdr++ = htonl(ARRAY_SIZE(caps));	/* appdata.caps.len */
-	memcpy(xdr, &caps, sizeof(caps));	/* appdata.caps */
-	xdr += ARRAY_SIZE(caps);
-	*xdr++ = htonl(conn_krb5->etype);	/* appdata.enctype */
-
-	*xdr++ = htonl(conn_krb5->key_len);	/* appdata.callbackKey.len */
-	K0 = xdr;
-	get_random_bytes(K0, conn_krb5->key_len); /* appdata.callbackKey.data */
-	xdr += xdr_round_up(conn_krb5->key_len) / 4;
-
-	*xdr++ = htonl(contsize);		/* appdata.callbackToken.len */
-	*xdr++ = htonl(1);			/* cont.kvno */
-	*xdr++ = htonl(token_krb5->etype);	/* cont.enctype */
-	*xdr++ = htonl(encsize);		/* cont.encrypted_token.len */
-
-	encbase = xdr;
-	xdr += offset / 4;
-	*xdr++ = htonl(conn_krb5->etype);	/* token.K0.enctype */
-	*xdr++ = htonl(conn_krb5->key_len);	/* token.K0.key.len */
-	memcpy(xdr, K0, conn_krb5->key_len);	/* token.K0.key.data */
-	xdr += xdr_round_up(conn_krb5->key_len) / 4;
-
-	*xdr++ = htonl(RXRPC_SECURITY_ENCRYPT);	/* token.level */
-	*xdr++ = htonl(0);			/* token.starttime */
-	*xdr++ = htonl(0);			/* " */
-	*xdr++ = htonl(0);			/* token.lifetime */
-	*xdr++ = htonl(0);			/* token.bytelife */
-	*xdr++ = htonl(0);			/* token.expirationtime */
-	*xdr++ = htonl(0);			/* " */
-	*xdr++ = htonl(1);			/* token.identities.count */
-	*xdr++ = htonl(0);			/* token.identities[0].kind */
-	*xdr++ = htonl(uuidsize);		/* token.identities[0].data.len */
-	memcpy(xdr, &server->uuid, uuidsize);
-	xdr += xdr_round_up(uuidsize) / 4;
-	*xdr++ = htonl(0);			/* token.identities[0].display.len */
-
-	xdr = encbase + xdr_round_up(encsize);
-
-	if ((unsigned long)xdr - (unsigned long)appdata != adatasize)
-		pr_err("Appdata size incorrect %lx != %zx\n",
-		       (unsigned long)xdr - (unsigned long)appdata, adatasize);
-
-	aead = crypto_krb5_prepare_encryption(token_krb5, token_key, RXGK_SERVER_ENC_TOKEN,
-					      GFP_KERNEL);
-	if (IS_ERR(aead)) {
-		ret = PTR_ERR(aead);
-		goto out_token;
-	}
-
-	sg_init_one(&sg, encbase, encsize);
-	ret = crypto_krb5_encrypt(token_krb5, aead, &sg, 1, encsize, offset, toksize, false);
-	if (ret < 0)
-		goto out_aead;
-
-	server->cm_rxgk_appdata.len  = adatasize;
-	server->cm_rxgk_appdata.data = appdata;
-	appdata = NULL;
-
-out_aead:
-	crypto_free_aead(aead);
-out_token:
-	kfree(appdata);
-out:
-	return ret;
-}
 #endif /* CONFIG_RXGK */
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index f75e007e483e..ef7bd9f943e0 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -283,7 +283,6 @@ struct afs_net {
 	struct socket		*socket;
 	struct afs_call		*spare_incoming_call;
 	struct work_struct	charge_preallocation_work;
-	struct work_struct	rx_oob_work;
 	struct mutex		socket_mutex;
 	atomic_t		nr_outstanding_calls;
 	atomic_t		nr_superblocks;
@@ -548,7 +547,6 @@ struct afs_server {
 	struct work_struct	destroyer;	/* Work item to try and destroy a server */
 	struct timer_list	timer;		/* Management timer */
 	struct mutex		cm_token_lock;	/* Lock governing creation of appdata */
-	struct krb5_buffer	cm_rxgk_appdata; /* Appdata to be included in RESPONSE packet */
 	struct key		*yfs_rxgk_appdata; /* Appdata to be included in RESPONSE packet */
 	time64_t		unuse_time;	/* Time at which last unused */
 	unsigned long		flags;
@@ -1091,7 +1089,6 @@ extern bool afs_cm_incoming_call(struct afs_call *);
  * cm_security.c
  */
 int afs_create_server_appdata(struct afs_server *server, struct key *key);
-void afs_process_oob_queue(struct work_struct *work);
 #ifdef CONFIG_RXGK
 int afs_create_token_key(struct afs_net *net, struct socket *socket);
 #else
diff --git a/fs/afs/main.c b/fs/afs/main.c
index 7a883c59976f..6353feaa5fb4 100644
--- a/fs/afs/main.c
+++ b/fs/afs/main.c
@@ -73,7 +73,6 @@ static int __net_init afs_net_init(struct net *net_ns)
 	generate_random_uuid((unsigned char *)&net->uuid);
 
 	INIT_WORK(&net->charge_preallocation_work, afs_charge_preallocation);
-	INIT_WORK(&net->rx_oob_work, afs_process_oob_queue);
 	mutex_init(&net->socket_mutex);
 
 	net->cells = RB_ROOT;
diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index 9f2c4972f2b4..b73e73a791ea 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -25,14 +25,12 @@ static void afs_process_async_call(struct work_struct *);
 static void afs_rx_new_call(struct sock *, struct rxrpc_call *, unsigned long);
 static void afs_rx_discard_new_call(struct rxrpc_call *, unsigned long);
 static void afs_rx_attach(struct rxrpc_call *rxcall, unsigned long user_call_ID);
-static void afs_rx_notify_oob(struct sock *sk, struct sk_buff *oob);
 static int afs_deliver_cm_op_id(struct afs_call *);
 
 static const struct rxrpc_kernel_ops afs_rxrpc_callback_ops = {
 	.notify_new_call	= afs_rx_new_call,
 	.discard_new_call	= afs_rx_discard_new_call,
 	.user_attach_call	= afs_rx_attach,
-	.notify_oob		= afs_rx_notify_oob,
 };
 
 /* asynchronous incoming call initial processing */
@@ -74,10 +72,6 @@ int afs_open_socket(struct afs_net *net)
 	if (ret < 0)
 		goto error_2;
 
-	ret = rxrpc_sock_set_manage_response(socket->sk, true);
-	if (ret < 0)
-		goto error_2;
-
 	ret = afs_create_token_key(net, socket);
 	if (ret < 0)
 		pr_err("Couldn't create RxGK CM key: %d\n", ret);
@@ -128,7 +122,6 @@ void afs_close_socket(struct afs_net *net)
 	_enter("");
 
 	cancel_work_sync(&net->charge_preallocation_work);
-	cancel_work_sync(&net->rx_oob_work);
 	/* Future work items should now see ->live is false. */
 
 	kernel_listen(net->socket, 0);
@@ -149,7 +142,6 @@ void afs_close_socket(struct afs_net *net)
 
 	kernel_sock_shutdown(net->socket, SHUT_RDWR);
 	flush_workqueue(afs_async_calls);
-	cancel_work_sync(&net->rx_oob_work);
 	net->socket->sk->sk_user_data = NULL;
 	sock_release(net->socket);
 	key_put(net->fs_cm_token_key);
@@ -1006,14 +998,3 @@ noinline int afs_protocol_error(struct afs_call *call,
 		call->unmarshalling_error = true;
 	return -EBADMSG;
 }
-
-/*
- * Wake up OOB notification processing.
- */
-static void afs_rx_notify_oob(struct sock *sk, struct sk_buff *oob)
-{
-	struct afs_net *net = sk->sk_user_data;
-
-	if (READ_ONCE(net->live))
-		queue_work(afs_wq, &net->rx_oob_work);
-}
diff --git a/fs/afs/server.c b/fs/afs/server.c
index b08d9080b0c5..4c3a7bd0f7d8 100644
--- a/fs/afs/server.c
+++ b/fs/afs/server.c
@@ -398,7 +398,6 @@ static void afs_server_rcu(struct rcu_head *rcu)
 	afs_put_endpoint_state(rcu_access_pointer(server->endpoint_state),
 			       afs_estate_trace_put_server);
 	afs_put_cell(server->cell, afs_cell_trace_put_server);
-	kfree(server->cm_rxgk_appdata.data);
 	key_put(server->yfs_rxgk_appdata);
 	kfree(server);
 }
diff --git a/include/net/af_rxrpc.h b/include/net/af_rxrpc.h
index 19c61a2f5af3..a9b8e1ba67ef 100644
--- a/include/net/af_rxrpc.h
+++ b/include/net/af_rxrpc.h
@@ -92,24 +92,6 @@ bool rxrpc_kernel_check_life(const struct socket *, const struct rxrpc_call *);
 
 int rxrpc_sock_set_min_security_level(struct sock *sk, unsigned int val);
 int rxrpc_sock_set_security_keyring(struct sock *, struct key *);
-int rxrpc_sock_set_manage_response(struct sock *sk, bool set);
-
-enum rxrpc_oob_type rxrpc_kernel_query_oob(struct sk_buff *oob,
-					   struct rxrpc_peer **_peer,
-					   unsigned long *_peer_appdata);
-struct sk_buff *rxrpc_kernel_dequeue_oob(struct socket *sock,
-					 enum rxrpc_oob_type *_type);
-void rxrpc_kernel_free_oob(struct sk_buff *oob);
-void rxrpc_kernel_query_challenge(struct sk_buff *challenge,
-				  struct rxrpc_peer **_peer,
-				  unsigned long *_peer_appdata,
-				  u16 *_service_id, u8 *_security_index);
-int rxrpc_kernel_reject_challenge(struct sk_buff *challenge, u32 abort_code,
-				  int error, enum rxrpc_abort_reason why);
-int rxkad_kernel_respond_to_challenge(struct sk_buff *challenge);
-u32 rxgk_kernel_query_challenge(struct sk_buff *challenge);
-int rxgk_kernel_respond_to_challenge(struct sk_buff *challenge,
-				     struct krb5_buffer *appdata);
 void rxrpc_kernel_query_key(const struct key *key, u8 *_security_index,
 			    u32 *_krb5_enctype);
 u8 rxrpc_kernel_query_call_security(struct rxrpc_call *call,
diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index 48e774034195..0eb5158f4e3d 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -169,7 +169,6 @@
 	EM(rxrpc_skb_get_conn_secured,		"GET conn-secd") \
 	EM(rxrpc_skb_get_conn_work,		"GET conn-work") \
 	EM(rxrpc_skb_get_local_work,		"GET locl-work") \
-	EM(rxrpc_skb_get_post_oob,		"GET post-oob ") \
 	EM(rxrpc_skb_get_reject_work,		"GET rej-work ") \
 	EM(rxrpc_skb_get_to_recvmsg,		"GET to-recv  ") \
 	EM(rxrpc_skb_get_to_recvmsg_oos,	"GET to-recv-o") \
@@ -186,18 +185,15 @@
 	EM(rxrpc_skb_put_error_report,		"PUT error-rep") \
 	EM(rxrpc_skb_put_input,			"PUT input    ") \
 	EM(rxrpc_skb_put_jumbo_subpacket,	"PUT jumbo-sub") \
-	EM(rxrpc_skb_put_oob,			"PUT oob      ") \
 	EM(rxrpc_skb_put_old_response,		"PUT old-resp ") \
 	EM(rxrpc_skb_put_purge,			"PUT purge    ") \
-	EM(rxrpc_skb_put_purge_oob,		"PUT purge-oob") \
 	EM(rxrpc_skb_put_response,		"PUT response ") \
 	EM(rxrpc_skb_put_response_copy,		"PUT resp-cpy ") \
 	EM(rxrpc_skb_put_rotate,		"PUT rotate   ") \
 	EM(rxrpc_skb_put_unknown,		"PUT unknown  ") \
 	EM(rxrpc_skb_see_conn_work,		"SEE conn-work") \
-	EM(rxrpc_skb_see_oob_challenge,		"SEE oob-chall") \
+	EM(rxrpc_skb_see_challenge,		"SEE challenge") \
 	EM(rxrpc_skb_see_recvmsg,		"SEE recvmsg  ") \
-	EM(rxrpc_skb_see_recvmsg_oob,		"SEE recvm-oob") \
 	EM(rxrpc_skb_see_reject,		"SEE reject   ") \
 	EM(rxrpc_skb_see_rotate,		"SEE rotate   ") \
 	EM(rxrpc_skb_see_unshare_nomem,		"SEE unshar-nm") \
@@ -264,7 +260,6 @@
 	EM(rxrpc_conn_free,			"FREE        ") \
 	EM(rxrpc_conn_get_activate_call,	"GET act-call") \
 	EM(rxrpc_conn_get_call_input,		"GET inp-call") \
-	EM(rxrpc_conn_get_challenge_input,	"GET inp-chal") \
 	EM(rxrpc_conn_get_conn_input,		"GET inp-conn") \
 	EM(rxrpc_conn_get_idle,			"GET idle    ") \
 	EM(rxrpc_conn_get_poke_abort,		"GET pk-abort") \
@@ -276,12 +271,10 @@
 	EM(rxrpc_conn_new_service,		"NEW service ") \
 	EM(rxrpc_conn_put_call,			"PUT call    ") \
 	EM(rxrpc_conn_put_call_input,		"PUT inp-call") \
-	EM(rxrpc_conn_put_challenge_input,	"PUT inp-chal") \
 	EM(rxrpc_conn_put_conn_input,		"PUT inp-conn") \
 	EM(rxrpc_conn_put_discard_idle,		"PUT disc-idl") \
 	EM(rxrpc_conn_put_local_dead,		"PUT loc-dead") \
 	EM(rxrpc_conn_put_noreuse,		"PUT noreuse ") \
-	EM(rxrpc_conn_put_oob,			"PUT oob     ") \
 	EM(rxrpc_conn_put_poke,			"PUT poke    ") \
 	EM(rxrpc_conn_put_service_reaped,	"PUT svc-reap") \
 	EM(rxrpc_conn_put_unbundle,		"PUT unbundle") \
@@ -391,7 +384,6 @@
 	EM(rxrpc_recvmsg_full,			"FULL") \
 	EM(rxrpc_recvmsg_hole,			"HOLE") \
 	EM(rxrpc_recvmsg_next,			"NEXT") \
-	EM(rxrpc_recvmsg_oobq,			"OOBQ") \
 	EM(rxrpc_recvmsg_requeue,		"REQU") \
 	EM(rxrpc_recvmsg_return,		"RETN") \
 	EM(rxrpc_recvmsg_terminal,		"TERM") \
diff --git a/include/uapi/linux/rxrpc.h b/include/uapi/linux/rxrpc.h
index bcdfdf9c67a1..7d6a50151931 100644
--- a/include/uapi/linux/rxrpc.h
+++ b/include/uapi/linux/rxrpc.h
@@ -58,11 +58,6 @@ enum rxrpc_cmsg_type {
 	RXRPC_TX_LENGTH		= 12,	/* -s-: Total length of Tx data */
 	RXRPC_SET_CALL_TIMEOUT	= 13,	/* -s-: Set one or more call timeouts */
 	RXRPC_CHARGE_ACCEPT	= 14,	/* Ss-: Charge the accept pool with a user call ID */
-	RXRPC_OOB_ID		= 15,	/* -sr: OOB message ID */
-	RXRPC_CHALLENGED	= 16,	/* C-r: Info on a received CHALLENGE */
-	RXRPC_RESPOND		= 17,	/* Cs-: Respond to a challenge */
-	RXRPC_RESPONDED		= 18,	/* S-r: Data received in RESPONSE */
-	RXRPC_RESP_RXGK_APPDATA	= 19,	/* Cs-: RESPONSE: RxGK app data to include */
 	RXRPC_RESPONSE_APPDATA	= 20,	/* Cs-: User key holding app data for RESPONSE */
 	RXRPC__SUPPORTED
 };
diff --git a/net/rxrpc/Makefile b/net/rxrpc/Makefile
index f994f9f30a29..f7a5e0a592ee 100644
--- a/net/rxrpc/Makefile
+++ b/net/rxrpc/Makefile
@@ -24,7 +24,6 @@ rxrpc-y := \
 	local_object.o \
 	misc.o \
 	net_ns.o \
-	oob.o \
 	output.o \
 	peer_event.o \
 	peer_object.o \
diff --git a/net/rxrpc/af_rxrpc.c b/net/rxrpc/af_rxrpc.c
index 9eb6ca3c5ebf..9d5ce1c7ad0d 100644
--- a/net/rxrpc/af_rxrpc.c
+++ b/net/rxrpc/af_rxrpc.c
@@ -633,7 +633,7 @@ static int rxrpc_setsockopt(struct socket *sock, int level, int optname,
 			    sockptr_t optval, unsigned int optlen)
 {
 	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
-	unsigned int min_sec_level, val;
+	unsigned int min_sec_level;
 	u16 service_upgrade[2];
 	int ret;
 
@@ -710,23 +710,7 @@ static int rxrpc_setsockopt(struct socket *sock, int level, int optname,
 
 		case RXRPC_MANAGE_RESPONSE:
 			ret = -EINVAL;
-			if (optlen != sizeof(unsigned int))
-				goto error;
-			ret = -EISCONN;
-			if (rx->sk.sk_state != RXRPC_UNBOUND)
-				goto error;
-			ret = copy_safe_from_sockptr(&val, sizeof(val),
-						     optval, optlen);
-			if (ret)
-				goto error;
-			ret = -EINVAL;
-			if (val > 1)
-				goto error;
-			if (val)
-				set_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-			else
-				clear_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-			goto success;
+			goto error;
 
 		default:
 			break;
@@ -835,8 +819,6 @@ static int rxrpc_create(struct net *net, struct socket *sock, int protocol,
 	rx->calls = RB_ROOT;
 
 	spin_lock_init(&rx->incoming_lock);
-	skb_queue_head_init(&rx->recvmsg_oobq);
-	rx->pending_oobq = RB_ROOT;
 	INIT_LIST_HEAD(&rx->sock_calls);
 	INIT_LIST_HEAD(&rx->to_be_accepted);
 	INIT_LIST_HEAD(&rx->recvmsg_q);
@@ -884,23 +866,6 @@ static int rxrpc_shutdown(struct socket *sock, int flags)
 	return ret;
 }
 
-/*
- * Purge the out-of-band queue.
- */
-static void rxrpc_purge_oob_queue(struct sock *sk)
-{
-	struct rxrpc_sock *rx = rxrpc_sk(sk);
-	struct sk_buff *skb;
-
-	while ((skb = skb_dequeue(&rx->recvmsg_oobq)))
-		rxrpc_kernel_free_oob(skb);
-	while (!RB_EMPTY_ROOT(&rx->pending_oobq)) {
-		skb = rb_entry(rx->pending_oobq.rb_node, struct sk_buff, rbnode);
-		rb_erase(&skb->rbnode, &rx->pending_oobq);
-		rxrpc_kernel_free_oob(skb);
-	}
-}
-
 /*
  * RxRPC socket destructor
  */
@@ -908,7 +873,6 @@ static void rxrpc_sock_destructor(struct sock *sk)
 {
 	_enter("%p", sk);
 
-	rxrpc_purge_oob_queue(sk);
 	rxrpc_purge_queue(&sk->sk_receive_queue);
 
 	WARN_ON(refcount_read(&sk->sk_wmem_alloc));
@@ -961,7 +925,6 @@ static int rxrpc_release_sock(struct sock *sk)
 	rxrpc_discard_prealloc(rx);
 	rxrpc_release_calls_on_socket(rx);
 	flush_workqueue(rxrpc_workqueue);
-	rxrpc_purge_oob_queue(sk);
 	rxrpc_purge_queue(&sk->sk_receive_queue);
 
 	rxrpc_unuse_local(rx->local, rxrpc_local_unuse_release_sock);
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 6e16df7173d3..83858c45fa4a 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -171,9 +171,6 @@ struct rxrpc_sock {
 	const struct rxrpc_kernel_ops *app_ops;	/* Table of kernel app notification funcs */
 	struct rxrpc_local	*local;		/* local endpoint */
 	struct rxrpc_backlog	*backlog;	/* Preallocation for services */
-	struct sk_buff_head	recvmsg_oobq;	/* OOB messages for recvmsg to pick up */
-	struct rb_root		pending_oobq;	/* OOB messages awaiting userspace to respond to */
-	u64			oob_id_counter;	/* OOB message ID counter */
 	spinlock_t		incoming_lock;	/* Incoming call vs service shutdown lock */
 	struct list_head	sock_calls;	/* List of calls owned by this socket */
 	struct list_head	to_be_accepted;	/* calls awaiting acceptance */
@@ -184,7 +181,6 @@ struct rxrpc_sock {
 	struct rb_root		calls;		/* User ID -> call mapping */
 	unsigned long		flags;
 #define RXRPC_SOCK_CONNECTED		0	/* connect_srx is set */
-#define RXRPC_SOCK_MANAGE_RESPONSE	1	/* User wants to manage RESPONSE packets */
 	rwlock_t		call_lock;	/* lock for calls */
 	u32			min_sec_level;	/* minimum security level */
 #define RXRPC_SECURITY_MAX	RXRPC_SECURITY_ENCRYPT
@@ -241,7 +237,6 @@ struct rxrpc_skb_priv {
 			u8		reason;		/* Reason for ack */
 		} ack;
 		struct {
-			struct rxrpc_connection *conn;	/* Connection referred to */
 			union {
 				u32 rxkad_nonce;
 			};
@@ -310,17 +305,6 @@ struct rxrpc_security {
 	bool (*validate_challenge)(struct rxrpc_connection *conn,
 				   struct sk_buff *skb);
 
-	/* Fill out the cmsg for recvmsg() to pass on a challenge to userspace.
-	 * The security class gets to add additional information.
-	 */
-	int (*challenge_to_recvmsg)(struct rxrpc_connection *conn,
-				    struct sk_buff *challenge,
-				    struct msghdr *msg);
-
-	/* Parse sendmsg() control message and respond to challenge. */
-	int (*sendmsg_respond_to_challenge)(struct sk_buff *challenge,
-					    struct msghdr *msg);
-
 	/* respond to a challenge */
 	int (*respond_to_challenge)(struct rxrpc_connection *conn,
 				    struct sk_buff *challenge);
@@ -1381,13 +1365,6 @@ static inline struct rxrpc_net *rxrpc_net(struct net *net)
 	return net_generic(net, rxrpc_net_id);
 }
 
-/*
- * oob.c
- */
-bool rxrpc_notify_socket_oob(struct rxrpc_call *call, struct sk_buff *skb);
-void rxrpc_add_pending_oob(struct rxrpc_sock *rx, struct sk_buff *skb);
-int rxrpc_sendmsg_oob(struct rxrpc_sock *rx, struct msghdr *msg, size_t len);
-
 /*
  * output.c
  */
diff --git a/net/rxrpc/conn_event.c b/net/rxrpc/conn_event.c
index 9146b9d4c2ac..267f2d0f883d 100644
--- a/net/rxrpc/conn_event.c
+++ b/net/rxrpc/conn_event.c
@@ -442,7 +442,7 @@ bool rxrpc_input_conn_packet(struct rxrpc_connection *conn, struct sk_buff *skb)
 		return true;
 
 	case RXRPC_PACKET_TYPE_CHALLENGE:
-		rxrpc_see_skb(skb, rxrpc_skb_see_oob_challenge);
+		rxrpc_see_skb(skb, rxrpc_skb_see_challenge);
 		if (rxrpc_is_conn_aborted(conn)) {
 			if (conn->completion == RXRPC_CALL_LOCALLY_ABORTED)
 				rxrpc_send_conn_abort(conn);
diff --git a/net/rxrpc/insecure.c b/net/rxrpc/insecure.c
index 0b39046bdc61..c083c82c4e7f 100644
--- a/net/rxrpc/insecure.c
+++ b/net/rxrpc/insecure.c
@@ -47,12 +47,6 @@ static bool none_validate_challenge(struct rxrpc_connection *conn,
 	return true;
 }
 
-static int none_sendmsg_respond_to_challenge(struct sk_buff *challenge,
-					     struct msghdr *msg)
-{
-	return -EINVAL;
-}
-
 static int none_verify_response(struct rxrpc_connection *conn,
 				struct sk_buff *response_skb,
 				void *response, unsigned int len)
@@ -88,7 +82,6 @@ const struct rxrpc_security rxrpc_no_security = {
 	.secure_packet			= none_secure_packet,
 	.verify_packet			= none_verify_packet,
 	.validate_challenge		= none_validate_challenge,
-	.sendmsg_respond_to_challenge	= none_sendmsg_respond_to_challenge,
 	.verify_response		= none_verify_response,
 	.clear				= none_clear,
 };
diff --git a/net/rxrpc/oob.c b/net/rxrpc/oob.c
deleted file mode 100644
index c80ee2487d09..000000000000
--- a/net/rxrpc/oob.c
+++ /dev/null
@@ -1,387 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* Out of band message handling (e.g. challenge-response)
- *
- * Copyright (C) 2025 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells at redhat.com)
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/net.h>
-#include <linux/gfp.h>
-#include <linux/skbuff.h>
-#include <linux/export.h>
-#include <linux/sched/signal.h>
-#include <net/sock.h>
-#include <net/af_rxrpc.h>
-#include "ar-internal.h"
-
-enum rxrpc_oob_command {
-	RXRPC_OOB_CMD_UNSET,
-	RXRPC_OOB_CMD_RESPOND,
-} __mode(byte);
-
-struct rxrpc_oob_params {
-	u64			oob_id;		/* ID number of message if reply */
-	s32			abort_code;
-	enum rxrpc_oob_command	command;
-	bool			have_oob_id:1;
-};
-
-/*
- * Post an out-of-band message for attention by the socket or kernel service
- * associated with a reference call.
- */
-bool rxrpc_notify_socket_oob(struct rxrpc_call *call, struct sk_buff *skb)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
-	struct rxrpc_sock *rx;
-	struct sock *sk;
-	bool queued = false;
-
-	rcu_read_lock();
-
-	rx = rcu_dereference(call->socket);
-	if (rx) {
-		sk = &rx->sk;
-		spin_lock_irq(&rx->recvmsg_lock);
-
-		if (sk->sk_state < RXRPC_CLOSE) {
-			skb->skb_mstamp_ns = rx->oob_id_counter++;
-			rxrpc_get_skb(skb, rxrpc_skb_get_post_oob);
-			skb_queue_tail(&rx->recvmsg_oobq, skb);
-			queued = true;
-
-			trace_rxrpc_notify_socket(call->debug_id, sp->hdr.serial);
-			if (rx->app_ops)
-				rx->app_ops->notify_oob(sk, skb);
-		}
-
-		spin_unlock_irq(&rx->recvmsg_lock);
-		if (queued && !rx->app_ops && !sock_flag(sk, SOCK_DEAD))
-			sk->sk_data_ready(sk);
-	}
-
-	rcu_read_unlock();
-	return queued;
-}
-
-/*
- * Locate the OOB message to respond to by its ID.
- */
-static struct sk_buff *rxrpc_find_pending_oob(struct rxrpc_sock *rx, u64 oob_id)
-{
-	struct rb_node *p;
-	struct sk_buff *skb;
-
-	p = rx->pending_oobq.rb_node;
-	while (p) {
-		skb = rb_entry(p, struct sk_buff, rbnode);
-
-		if (oob_id < skb->skb_mstamp_ns)
-			p = p->rb_left;
-		else if (oob_id > skb->skb_mstamp_ns)
-			p = p->rb_right;
-		else
-			return skb;
-	}
-
-	return NULL;
-}
-
-/*
- * Add an OOB message into the pending-response set.  We always assign the next
- * value from a 64-bit counter to the oob_id, so just assume we're always going
- * to be on the right-hand edge of the tree and that the counter won't wrap.
- * The tree is also given a ref to the message.
- */
-void rxrpc_add_pending_oob(struct rxrpc_sock *rx, struct sk_buff *skb)
-{
-	struct rb_node **pp = &rx->pending_oobq.rb_node, *p = NULL;
-
-	while (*pp) {
-		p = *pp;
-		pp = &(*pp)->rb_right;
-	}
-
-	rb_link_node(&skb->rbnode, p, pp);
-	rb_insert_color(&skb->rbnode, &rx->pending_oobq);
-}
-
-/*
- * Extract control messages from the sendmsg() control buffer.
- */
-static int rxrpc_sendmsg_oob_cmsg(struct msghdr *msg, struct rxrpc_oob_params *p)
-{
-	struct cmsghdr *cmsg;
-	int len;
-
-	if (msg->msg_controllen == 0)
-		return -EINVAL;
-
-	for_each_cmsghdr(cmsg, msg) {
-		if (!CMSG_OK(msg, cmsg))
-			return -EINVAL;
-
-		len = cmsg->cmsg_len - sizeof(struct cmsghdr);
-		_debug("CMSG %d, %d, %d",
-		       cmsg->cmsg_level, cmsg->cmsg_type, len);
-
-		if (cmsg->cmsg_level != SOL_RXRPC)
-			continue;
-
-		switch (cmsg->cmsg_type) {
-		case RXRPC_OOB_ID:
-			if (len != sizeof(p->oob_id) || p->have_oob_id)
-				return -EINVAL;
-			memcpy(&p->oob_id, CMSG_DATA(cmsg), sizeof(p->oob_id));
-			p->have_oob_id = true;
-			break;
-		case RXRPC_RESPOND:
-			if (p->command != RXRPC_OOB_CMD_UNSET)
-				return -EINVAL;
-			p->command = RXRPC_OOB_CMD_RESPOND;
-			break;
-		case RXRPC_ABORT:
-			if (len != sizeof(p->abort_code) || p->abort_code)
-				return -EINVAL;
-			memcpy(&p->abort_code, CMSG_DATA(cmsg), sizeof(p->abort_code));
-			if (p->abort_code == 0)
-				return -EINVAL;
-			break;
-		case RXRPC_RESP_RXGK_APPDATA:
-			if (p->command != RXRPC_OOB_CMD_RESPOND)
-				return -EINVAL;
-			break;
-		default:
-			return -EINVAL;
-		}
-	}
-
-	switch (p->command) {
-	case RXRPC_OOB_CMD_RESPOND:
-		if (!p->have_oob_id)
-			return -EBADSLT;
-		break;
-	default:
-		return -EINVAL;
-	}
-
-	return 0;
-}
-
-/*
- * Allow userspace to respond to an OOB using sendmsg().
- */
-static int rxrpc_respond_to_oob(struct rxrpc_sock *rx,
-				struct rxrpc_oob_params *p,
-				struct msghdr *msg)
-{
-	struct rxrpc_connection *conn;
-	struct rxrpc_skb_priv *sp;
-	struct sk_buff *skb;
-	int ret;
-
-	skb = rxrpc_find_pending_oob(rx, p->oob_id);
-	if (skb)
-		rb_erase(&skb->rbnode, &rx->pending_oobq);
-	release_sock(&rx->sk);
-	if (!skb)
-		return -EBADSLT;
-
-	sp = rxrpc_skb(skb);
-
-	switch (p->command) {
-	case RXRPC_OOB_CMD_RESPOND:
-		ret = -EPROTO;
-		if (skb->mark != RXRPC_OOB_CHALLENGE)
-			break;
-		conn = sp->chall.conn;
-		ret = -EOPNOTSUPP;
-		if (!conn->security->sendmsg_respond_to_challenge)
-			break;
-		if (p->abort_code) {
-			rxrpc_abort_conn(conn, NULL, p->abort_code, -ECONNABORTED,
-					 rxrpc_abort_response_sendmsg);
-			ret = 0;
-		} else {
-			ret = conn->security->sendmsg_respond_to_challenge(skb, msg);
-		}
-		break;
-	default:
-		ret = -EINVAL;
-		break;
-	}
-
-	switch (skb->mark) {
-	case RXRPC_OOB_CHALLENGE:
-		rxrpc_put_connection(sp->chall.conn, rxrpc_conn_put_oob);
-		break;
-	}
-	rxrpc_free_skb(skb, rxrpc_skb_put_oob);
-	return ret;
-}
-
-/*
- * Send an out-of-band message or respond to a received out-of-band message.
- * - caller gives us the socket lock
- * - the socket may be either a client socket or a server socket
- */
-int rxrpc_sendmsg_oob(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
-{
-	struct rxrpc_oob_params p = {};
-	int ret;
-
-	_enter("");
-
-	ret = rxrpc_sendmsg_oob_cmsg(msg, &p);
-	if (ret < 0)
-		goto error_release_sock;
-
-	if (p.have_oob_id)
-		return rxrpc_respond_to_oob(rx, &p, msg);
-
-	release_sock(&rx->sk);
-
-	switch (p.command) {
-	default:
-		ret = -EINVAL;
-		break;
-	}
-
-	_leave(" = %d", ret);
-	return ret;
-
-error_release_sock:
-	release_sock(&rx->sk);
-	return ret;
-}
-
-/**
- * rxrpc_kernel_query_oob - Query the parameters of an out-of-band message
- * @oob: The message to query
- * @_peer: Where to return the peer record
- * @_peer_appdata: The application data attached to a peer record
- *
- * Extract useful parameters from an out-of-band message.  The source peer
- * parameters are returned through the argument list and the message type is
- * returned.
- *
- * Return:
- * * %RXRPC_OOB_CHALLENGE - Challenge wanting a response.
- */
-enum rxrpc_oob_type rxrpc_kernel_query_oob(struct sk_buff *oob,
-					   struct rxrpc_peer **_peer,
-					   unsigned long *_peer_appdata)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(oob);
-	enum rxrpc_oob_type type = oob->mark;
-
-	switch (type) {
-	case RXRPC_OOB_CHALLENGE:
-		*_peer		= sp->chall.conn->peer;
-		*_peer_appdata	= sp->chall.conn->peer->app_data;
-		break;
-	default:
-		WARN_ON_ONCE(1);
-		*_peer		= NULL;
-		*_peer_appdata	= 0;
-		break;
-	}
-
-	return type;
-}
-EXPORT_SYMBOL(rxrpc_kernel_query_oob);
-
-/**
- * rxrpc_kernel_dequeue_oob - Dequeue and return the front OOB message
- * @sock: The socket to query
- * @_type: Where to return the message type
- *
- * Dequeue the front OOB message, if there is one, and return it and
- * its type.
- *
- * Return: The sk_buff representing the OOB message or %NULL if the queue was
- * empty.
- */
-struct sk_buff *rxrpc_kernel_dequeue_oob(struct socket *sock,
-					 enum rxrpc_oob_type *_type)
-{
-	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
-	struct sk_buff *oob;
-
-	oob = skb_dequeue(&rx->recvmsg_oobq);
-	if (oob)
-		*_type = oob->mark;
-	return oob;
-}
-EXPORT_SYMBOL(rxrpc_kernel_dequeue_oob);
-
-/**
- * rxrpc_kernel_free_oob - Free an out-of-band message
- * @oob: The OOB message to free
- *
- * Free an OOB message along with any resources it holds.
- */
-void rxrpc_kernel_free_oob(struct sk_buff *oob)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(oob);
-
-	switch (oob->mark) {
-	case RXRPC_OOB_CHALLENGE:
-		rxrpc_put_connection(sp->chall.conn, rxrpc_conn_put_oob);
-		break;
-	}
-
-	rxrpc_free_skb(oob, rxrpc_skb_put_purge_oob);
-}
-EXPORT_SYMBOL(rxrpc_kernel_free_oob);
-
-/**
- * rxrpc_kernel_query_challenge - Query the parameters of a challenge
- * @challenge: The challenge to query
- * @_peer: Where to return the peer record
- * @_peer_appdata: The application data attached to a peer record
- * @_service_id: Where to return the connection service ID
- * @_security_index: Where to return the connection security index
- *
- * Extract useful parameters from a CHALLENGE message.
- */
-void rxrpc_kernel_query_challenge(struct sk_buff *challenge,
-				  struct rxrpc_peer **_peer,
-				  unsigned long *_peer_appdata,
-				  u16 *_service_id, u8 *_security_index)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(challenge);
-
-	*_peer		= sp->chall.conn->peer;
-	*_peer_appdata	= sp->chall.conn->peer->app_data;
-	*_service_id	= sp->hdr.serviceId;
-	*_security_index = sp->hdr.securityIndex;
-}
-EXPORT_SYMBOL(rxrpc_kernel_query_challenge);
-
-/**
- * rxrpc_kernel_reject_challenge - Allow a kernel service to reject a challenge
- * @challenge: The challenge to be rejected
- * @abort_code: The abort code to stick into the ABORT packet
- * @error: Local error value
- * @why: Indication as to why.
- *
- * Allow a kernel service to reject a challenge by aborting the connection if
- * it's still in an abortable state.  The error is returned so this function
- * can be used with a return statement.
- *
- * Return: The %error parameter.
- */
-int rxrpc_kernel_reject_challenge(struct sk_buff *challenge, u32 abort_code,
-				  int error, enum rxrpc_abort_reason why)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(challenge);
-
-	_enter("{%x},%d,%d,%u", sp->hdr.serial, abort_code, error, why);
-
-	rxrpc_abort_conn(sp->chall.conn, NULL, abort_code, error, why);
-	return error;
-}
-EXPORT_SYMBOL(rxrpc_kernel_reject_challenge);
diff --git a/net/rxrpc/recvmsg.c b/net/rxrpc/recvmsg.c
index 28b2148b5693..8c4571982edc 100644
--- a/net/rxrpc/recvmsg.c
+++ b/net/rxrpc/recvmsg.c
@@ -201,61 +201,6 @@ static int rxrpc_recvmsg_user_id(struct rxrpc_call *call, struct msghdr *msg,
 	}
 }
 
-/*
- * Deal with a CHALLENGE packet.
- */
-static int rxrpc_recvmsg_challenge(struct socket *sock, struct msghdr *msg,
-				   struct sk_buff *challenge, unsigned int flags)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(challenge);
-	struct rxrpc_connection *conn = sp->chall.conn;
-
-	return conn->security->challenge_to_recvmsg(conn, challenge, msg);
-}
-
-/*
- * Process OOB packets.  Called with the socket locked.
- */
-static int rxrpc_recvmsg_oob(struct socket *sock, struct msghdr *msg,
-			     unsigned int flags)
-{
-	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
-	struct sk_buff *skb;
-	bool need_response = false;
-	int ret;
-
-	skb = skb_peek(&rx->recvmsg_oobq);
-	if (!skb)
-		return -EAGAIN;
-	rxrpc_see_skb(skb, rxrpc_skb_see_recvmsg);
-
-	ret = put_cmsg(msg, SOL_RXRPC, RXRPC_OOB_ID, sizeof(u64),
-		       &skb->skb_mstamp_ns);
-	if (ret < 0)
-		return ret;
-
-	switch ((enum rxrpc_oob_type)skb->mark) {
-	case RXRPC_OOB_CHALLENGE:
-		need_response = true;
-		ret = rxrpc_recvmsg_challenge(sock, msg, skb, flags);
-		break;
-	default:
-		WARN_ONCE(1, "recvmsg() can't process unknown OOB type %u\n",
-			  skb->mark);
-		ret = -EIO;
-		break;
-	}
-
-	if (!(flags & MSG_PEEK)) {
-		skb_unlink(skb, &rx->recvmsg_oobq);
-		if (need_response)
-			rxrpc_add_pending_oob(rx, skb);
-		else
-			rxrpc_free_skb(skb, rxrpc_skb_put_oob);
-	}
-	return ret;
-}
-
 /*
  * Deliver messages to a call.  This keeps processing packets until the buffer
  * is filled and we find either more DATA (returns 0) or the end of the DATA
@@ -267,7 +212,6 @@ static int rxrpc_recvmsg_data(struct socket *sock, struct rxrpc_call *call,
 			      size_t len, int flags, size_t *_offset)
 {
 	struct rxrpc_skb_priv *sp;
-	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
 	struct sk_buff *skb;
 	rxrpc_seq_t seq = 0;
 	size_t remain;
@@ -362,13 +306,6 @@ static int rxrpc_recvmsg_data(struct socket *sock, struct rxrpc_call *call,
 
 		if (!(flags & MSG_PEEK))
 			rxrpc_rotate_rx_window(call);
-
-		if (!rx->app_ops &&
-		    !skb_queue_empty_lockless(&rx->recvmsg_oobq)) {
-			trace_rxrpc_recvdata(call, rxrpc_recvmsg_oobq, seq,
-					     rx_pkt_offset, rx_pkt_len, ret);
-			break;
-		}
 	}
 
 out:
@@ -416,14 +353,12 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 	/* Return immediately if a client socket has no outstanding calls */
 	if (RB_EMPTY_ROOT(&rx->calls) &&
 	    list_empty(&rx->recvmsg_q) &&
-	    skb_queue_empty_lockless(&rx->recvmsg_oobq) &&
 	    rx->sk.sk_state != RXRPC_SERVER_LISTENING) {
 		release_sock(&rx->sk);
 		return -EAGAIN;
 	}
 
-	if (list_empty(&rx->recvmsg_q) &&
-	    skb_queue_empty_lockless(&rx->recvmsg_oobq)) {
+	if (list_empty(&rx->recvmsg_q)) {
 		ret = -EWOULDBLOCK;
 		if (timeo == 0) {
 			call = NULL;
@@ -439,8 +374,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 		if (ret)
 			goto wait_error;
 
-		if (list_empty(&rx->recvmsg_q) &&
-		    skb_queue_empty_lockless(&rx->recvmsg_oobq)) {
+		if (list_empty(&rx->recvmsg_q)) {
 			if (signal_pending(current))
 				goto wait_interrupted;
 			trace_rxrpc_recvmsg(0, rxrpc_recvmsg_wait, 0);
@@ -450,15 +384,6 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 		goto try_again;
 	}
 
-	/* Deal with OOB messages before we consider getting normal data. */
-	if (!skb_queue_empty_lockless(&rx->recvmsg_oobq)) {
-		ret = rxrpc_recvmsg_oob(sock, msg, flags);
-		release_sock(&rx->sk);
-		if (ret == -EAGAIN)
-			goto try_again;
-		goto error_trace;
-	}
-
 	/* Find the next call and dequeue it if we're not just peeking.  If we
 	 * do dequeue it, that comes with a ref that we will need to release.
 	 * We also want to weed out calls that got requeued whilst we were
@@ -469,8 +394,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 	call = list_entry(l, struct rxrpc_call, recvmsg_link);
 
 	if (!rxrpc_call_is_complete(call) &&
-	    skb_queue_empty(&call->recvmsg_queue) &&
-	    skb_queue_empty(&rx->recvmsg_oobq)) {
+	    skb_queue_empty(&call->recvmsg_queue)) {
 		list_del_init(&call->recvmsg_link);
 		spin_unlock_irq(&rx->recvmsg_lock);
 		release_sock(&rx->sk);
diff --git a/net/rxrpc/rxgk.c b/net/rxrpc/rxgk.c
index 031c6719d33e..c64ef96763ae 100644
--- a/net/rxrpc/rxgk.c
+++ b/net/rxrpc/rxgk.c
@@ -735,37 +735,6 @@ static bool rxgk_validate_challenge(struct rxrpc_connection *conn,
 	return true;
 }
 
-/**
- * rxgk_kernel_query_challenge - Query RxGK-specific challenge parameters
- * @challenge: The challenge packet to query
- *
- * Return: The Kerberos 5 encoding type for the challenged connection.
- */
-u32 rxgk_kernel_query_challenge(struct sk_buff *challenge)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(challenge);
-
-	return sp->chall.conn->rxgk.enctype;
-}
-EXPORT_SYMBOL(rxgk_kernel_query_challenge);
-
-/*
- * Fill out the control message to pass to userspace to inform about the
- * challenge.
- */
-static int rxgk_challenge_to_recvmsg(struct rxrpc_connection *conn,
-				     struct sk_buff *challenge,
-				     struct msghdr *msg)
-{
-	struct rxgk_challenge chall;
-
-	chall.base.service_id		= conn->service_id;
-	chall.base.security_index	= conn->security_ix;
-	chall.enctype			= conn->rxgk.enctype;
-
-	return put_cmsg(msg, SOL_RXRPC, RXRPC_CHALLENGED, sizeof(chall), &chall);
-}
-
 /*
  * Insert the requisite amount of XDR padding for the length given.
  */
@@ -1052,46 +1021,6 @@ static int rxgk_respond_to_challenge(struct rxrpc_connection *conn,
 	return ret;
 }
 
-/**
- * rxgk_kernel_respond_to_challenge - Respond to a challenge with appdata
- * @challenge: The challenge to respond to
- * @appdata: The application data to include in the RESPONSE authenticator
- *
- * Allow a kernel application to respond to a CHALLENGE with application data
- * to be included in the RxGK RESPONSE Authenticator.
- *
- * Return: %0 if successful and a negative error code otherwise.
- */
-int rxgk_kernel_respond_to_challenge(struct sk_buff *challenge,
-				     struct krb5_buffer *appdata)
-{
-	return -EINVAL;
-}
-EXPORT_SYMBOL(rxgk_kernel_respond_to_challenge);
-
-/*
- * Parse sendmsg() control message and respond to challenge.  We need to see if
- * there's an appdata to fish out.
- */
-static int rxgk_sendmsg_respond_to_challenge(struct sk_buff *challenge,
-					     struct msghdr *msg)
-{
-	struct krb5_buffer appdata = {};
-	struct cmsghdr *cmsg;
-
-	for_each_cmsghdr(cmsg, msg) {
-		if (cmsg->cmsg_level != SOL_RXRPC ||
-		    cmsg->cmsg_type != RXRPC_RESP_RXGK_APPDATA)
-			continue;
-		if (appdata.data)
-			return -EINVAL;
-		appdata.data = CMSG_DATA(cmsg);
-		appdata.len = cmsg->cmsg_len - sizeof(struct cmsghdr);
-	}
-
-	return rxgk_kernel_respond_to_challenge(challenge, &appdata);
-}
-
 /*
  * Verify the authenticator.
  *
@@ -1365,8 +1294,6 @@ const struct rxrpc_security rxgk_yfs = {
 	.free_call_crypto		= rxgk_free_call_crypto,
 	.issue_challenge		= rxgk_issue_challenge,
 	.validate_challenge		= rxgk_validate_challenge,
-	.challenge_to_recvmsg		= rxgk_challenge_to_recvmsg,
-	.sendmsg_respond_to_challenge	= rxgk_sendmsg_respond_to_challenge,
 	.respond_to_challenge		= rxgk_respond_to_challenge,
 	.verify_response		= rxgk_verify_response,
 	.clear				= rxgk_clear,
diff --git a/net/rxrpc/rxkad.c b/net/rxrpc/rxkad.c
index ca9f0e82cb9a..e55bfca10e1a 100644
--- a/net/rxrpc/rxkad.c
+++ b/net/rxrpc/rxkad.c
@@ -748,32 +748,6 @@ static int rxkad_respond_to_challenge(struct rxrpc_connection *conn,
 	return ret;
 }
 
-/*
- * RxKAD does automatic response only as there's nothing to manage that isn't
- * already in the key.
- */
-static int rxkad_sendmsg_respond_to_challenge(struct sk_buff *challenge,
-					      struct msghdr *msg)
-{
-	return -EINVAL;
-}
-
-/**
- * rxkad_kernel_respond_to_challenge - Respond to a challenge with appdata
- * @challenge: The challenge to respond to
- *
- * Allow a kernel application to respond to a CHALLENGE.
- *
- * Return: %0 if successful and a negative error code otherwise.
- */
-int rxkad_kernel_respond_to_challenge(struct sk_buff *challenge)
-{
-	struct rxrpc_skb_priv *csp = rxrpc_skb(challenge);
-
-	return rxkad_respond_to_challenge(csp->chall.conn, challenge);
-}
-EXPORT_SYMBOL(rxkad_kernel_respond_to_challenge);
-
 /* Decrypt data in-place using DES-PCBC.  @len must be a multiple of 8. */
 VISIBLE_IF_KUNIT void des_pcbc_decrypt_inplace(const struct des_ctx *key,
 					       __le64 iv, u8 *data, size_t len)
@@ -1134,7 +1108,6 @@ const struct rxrpc_security rxkad = {
 	.free_call_crypto		= rxkad_free_call_crypto,
 	.issue_challenge		= rxkad_issue_challenge,
 	.validate_challenge		= rxkad_validate_challenge,
-	.sendmsg_respond_to_challenge	= rxkad_sendmsg_respond_to_challenge,
 	.respond_to_challenge		= rxkad_respond_to_challenge,
 	.verify_response		= rxkad_verify_response,
 	.clear				= rxkad_clear,
diff --git a/net/rxrpc/server_key.c b/net/rxrpc/server_key.c
index 3efe104b1930..fc6f478e1762 100644
--- a/net/rxrpc/server_key.c
+++ b/net/rxrpc/server_key.c
@@ -173,43 +173,3 @@ int rxrpc_sock_set_security_keyring(struct sock *sk, struct key *keyring)
 	return ret;
 }
 EXPORT_SYMBOL(rxrpc_sock_set_security_keyring);
-
-/**
- * rxrpc_sock_set_manage_response - Set the manage-response flag for a kernel service
- * @sk: The socket to set the keyring on
- * @set: True to set, false to clear the flag
- *
- * Set the flag on an rxrpc socket to say that the caller wants to manage the
- * RESPONSE packet and the user-defined data it may contain.  Setting this
- * means that recvmsg() will return messages with RXRPC_CHALLENGED in the
- * control message buffer containing information about the challenge.
- *
- * The user should respond to the challenge by passing RXRPC_RESPOND or
- * RXRPC_RESPOND_ABORT control messages with sendmsg() to the same call.
- * Supplementary control messages, such as RXRPC_RESP_RXGK_APPDATA, may be
- * included to indicate the parts the user wants to supply.
- *
- * The server will be passed the response data with a RXRPC_RESPONDED control
- * message when it gets the first data from each call.
- *
- * Note that this is only honoured by security classes that need auxiliary data
- * (e.g. RxGK).  Those that don't offer the facility (e.g. RxKAD) respond
- * without consulting userspace.
- *
- * Return: The previous setting.
- */
-int rxrpc_sock_set_manage_response(struct sock *sk, bool set)
-{
-	struct rxrpc_sock *rx = rxrpc_sk(sk);
-	int ret;
-
-	lock_sock(sk);
-	ret = !!test_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-	if (set)
-		set_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-	else
-		clear_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-	release_sock(sk);
-	return ret;
-}
-EXPORT_SYMBOL(rxrpc_sock_set_manage_response);



From sashal at kernel.org  Thu Aug  6 06:49:59 2026
From: sashal at kernel.org (Sasha Levin)
Date: Thu,  6 Aug 2026 09:49:59 -0400
Subject: [PATCH 6.12.y 1/5] rxrpc: Generate rtt_min
In-Reply-To: <2026073013-sanding-unlined-9b91@gregkh>
References: <2026073013-sanding-unlined-9b91@gregkh>
Message-ID: <20260806135003.602726-1-sashal@kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit c637bd066841de6d0a204898a62f1d9bb8fa1b7f ]

Generate rtt_min as this is required by RACK-TLP.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20241204074710.990092-27-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 lib/win_minmax.c        |  1 +
 net/rxrpc/ar-internal.h |  2 ++
 net/rxrpc/rtt.c         | 20 ++++++++++++++++----
 3 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/lib/win_minmax.c b/lib/win_minmax.c
index ec10506834b62..1682e614309c7 100644
--- a/lib/win_minmax.c
+++ b/lib/win_minmax.c
@@ -97,3 +97,4 @@ u32 minmax_running_min(struct minmax *m, u32 win, u32 t, u32 meas)
 
 	return minmax_subwin_update(m, win, &val);
 }
+EXPORT_SYMBOL(minmax_running_min);
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 79a377f1b5517..f7ef2790f07af 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -349,6 +349,8 @@ struct rxrpc_peer {
 	spinlock_t		rtt_input_lock;	/* RTT lock for input routine */
 	ktime_t			rtt_last_req;	/* Time of last RTT request */
 	unsigned int		rtt_count;	/* Number of samples we've got */
+	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
+	struct minmax		min_rtt;	/* Estimated minimum RTT */
 
 	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
 	u32			mdev_us;	/* medium deviation			*/
diff --git a/net/rxrpc/rtt.c b/net/rxrpc/rtt.c
index cdab7b7d08a00..bce6e8f32d334 100644
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -127,16 +127,27 @@ static void rxrpc_set_rto(struct rxrpc_peer *peer)
 	peer->rto_us = rxrpc_bound_rto(rto);
 }
 
-static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, long rtt_us)
+static void rxrpc_update_rtt_min(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+{
+	/* Window size 5mins in approx usec (ipv4.sysctl_tcp_min_rtt_wlen) */
+	u32 wlen_us = 5ULL * NSEC_PER_SEC / 1024;
+
+	minmax_running_min(&peer->min_rtt, wlen_us, resp_time / 1024,
+			   (u32)rtt_us ? : jiffies_to_usecs(1));
+}
+
+static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
 {
 	if (rtt_us < 0)
 		return;
 
-	//rxrpc_update_rtt_min(peer, rtt_us);
+	/* Update RACK min RTT [RFC8985 6.1 Step 1]. */
+	rxrpc_update_rtt_min(peer, resp_time, rtt_us);
+
 	rxrpc_rtt_estimator(peer, rtt_us);
 	rxrpc_set_rto(peer);
 
-	/* RFC6298: only reset backoff on valid RTT measurement. */
+	/* Only reset backoff on valid RTT measurement [RFC6298]. */
 	peer->backoff = 0;
 }
 
@@ -157,9 +168,10 @@ void rxrpc_peer_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
 		return;
 
 	spin_lock(&peer->rtt_input_lock);
-	rxrpc_ack_update_rtt(peer, rtt_us);
+	rxrpc_ack_update_rtt(peer, resp_time, rtt_us);
 	if (peer->rtt_count < 3)
 		peer->rtt_count++;
+	peer->rtt_taken++;
 	spin_unlock(&peer->rtt_input_lock);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,
-- 
2.53.0



From sashal at kernel.org  Thu Aug  6 06:50:00 2026
From: sashal at kernel.org (Sasha Levin)
Date: Thu,  6 Aug 2026 09:50:00 -0400
Subject: [PATCH 6.12.y 2/5] rxrpc: Adjust the rxrpc_rtt_rx tracepoint
In-Reply-To: <20260806135003.602726-1-sashal@kernel.org>
References: <2026073013-sanding-unlined-9b91@gregkh>
 <20260806135003.602726-1-sashal@kernel.org>
Message-ID: <20260806135003.602726-2-sashal@kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit 93dfca65a1df42a3c8b1094299dc42ab8f18e5c8 ]

Adjust the rxrpc_rtt_rx tracepoint in the following ways:

 (1) Display the collected RTT sample in the rxrpc_rtt_rx trace.

 (2) Move the division of srtt by 8 to the TP_printk() rather doing it
     before invoking the trace point.

 (3) Display the min_rtt value.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 include/trace/events/rxrpc.h | 14 ++++++++++----
 net/rxrpc/input.c            |  4 ++--
 net/rxrpc/rtt.c              |  2 +-
 3 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index fd913d7f79b87..c398897052a2a 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -1308,9 +1308,9 @@ TRACE_EVENT(rxrpc_rtt_rx,
 	    TP_PROTO(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
 		     int slot,
 		     rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
-		     u32 rtt, u32 rto),
+		     u32 rtt, u32 srtt, u32 rto),
 
-	    TP_ARGS(call, why, slot, send_serial, resp_serial, rtt, rto),
+	    TP_ARGS(call, why, slot, send_serial, resp_serial, rtt, srtt, rto),
 
 	    TP_STRUCT__entry(
 		    __field(unsigned int,		call)
@@ -1319,7 +1319,9 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __field(rxrpc_serial_t,		send_serial)
 		    __field(rxrpc_serial_t,		resp_serial)
 		    __field(u32,			rtt)
+		    __field(u32,			srtt)
 		    __field(u32,			rto)
+		    __field(u32,			min_rtt)
 			     ),
 
 	    TP_fast_assign(
@@ -1329,17 +1331,21 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __entry->send_serial = send_serial;
 		    __entry->resp_serial = resp_serial;
 		    __entry->rtt = rtt;
+		    __entry->srtt = srtt;
 		    __entry->rto = rto;
+		    __entry->min_rtt = minmax_get(&call->peer->min_rtt)
 			   ),
 
-	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u rto=%u",
+	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u srtt=%u rto=%u min=%u",
 		      __entry->call,
 		      __entry->slot,
 		      __print_symbolic(__entry->why, rxrpc_rtt_rx_traces),
 		      __entry->send_serial,
 		      __entry->resp_serial,
 		      __entry->rtt,
-		      __entry->rto)
+		      __entry->srtt / 8,
+		      __entry->rto,
+		      __entry->min_rtt)
 	    );
 
 TRACE_EVENT(rxrpc_timer_set,
diff --git a/net/rxrpc/input.c b/net/rxrpc/input.c
index c2a2fb7210c3d..501a049b42f86 100644
--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -674,7 +674,7 @@ static void rxrpc_complete_rtt_probe(struct rxrpc_call *call,
 		 */
 		if (after(acked_serial, orig_serial)) {
 			trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_obsolete, i,
-					   orig_serial, acked_serial, 0, 0);
+					   orig_serial, acked_serial, 0, 0, 0);
 			clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail);
 			smp_wmb();
 			set_bit(i, &call->rtt_avail);
@@ -682,7 +682,7 @@ static void rxrpc_complete_rtt_probe(struct rxrpc_call *call,
 	}
 
 	if (!matched)
-		trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_lost, 9, 0, acked_serial, 0, 0);
+		trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_lost, 9, 0, acked_serial, 0, 0, 0);
 }
 
 /*
diff --git a/net/rxrpc/rtt.c b/net/rxrpc/rtt.c
index bce6e8f32d334..dc26754ffd5fb 100644
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -175,7 +175,7 @@ void rxrpc_peer_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
 	spin_unlock(&peer->rtt_input_lock);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,
-			   peer->srtt_us >> 3, peer->rto_us);
+			   rtt_us, peer->srtt_us, peer->rto_us);
 }
 
 /*
-- 
2.53.0



From sashal at kernel.org  Thu Aug  6 06:50:01 2026
From: sashal at kernel.org (Sasha Levin)
Date: Thu,  6 Aug 2026 09:50:01 -0400
Subject: [PATCH 6.12.y 3/5] rxrpc: Fix the calculation and use of RTO
In-Reply-To: <20260806135003.602726-1-sashal@kernel.org>
References: <2026073013-sanding-unlined-9b91@gregkh>
 <20260806135003.602726-1-sashal@kernel.org>
Message-ID: <20260806135003.602726-3-sashal@kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit 5c0ceba23bb47085d6c9c53bff08a29634ee4e7e ]

Make the following changes to the calculation and use of RTO:

 (1) Fix rxrpc_resend() to use the backed-off RTO value obtained by calling
     rxrpc_get_rto_backoff() rather than extracting the value itself.
     Without this, it may retransmit packets too early.

 (2) The RTO value being similar to the RTT causes a lot of extraneous
     resends because the RTT doesn't end up taking account of clearing out
     of the receive queue on the server.  Worse, responses to PING-ACKs are
     made as fast as possible and so are less than the DATA-requested-ACK
     RTT and so skew the RTT down.

     Fix this by putting a lower bound on the RTO by adding 100ms to it and
     limiting the lower end to 200ms.

Fixes: c410bf01933e ("rxrpc: Fix the excessive initial retransmission timeout")
Fixes: 37473e416234 ("rxrpc: Clean up the resend algorithm")
Signed-off-by: David Howells <dhowells at redhat.com>
Suggested-by: Simon Wilkinson <sxw at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 net/rxrpc/call_event.c | 3 ++-
 net/rxrpc/rtt.c        | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/net/rxrpc/call_event.c b/net/rxrpc/call_event.c
index 1181976972a31..c85e89b371b2a 100644
--- a/net/rxrpc/call_event.c
+++ b/net/rxrpc/call_event.c
@@ -71,7 +71,8 @@ void rxrpc_resend(struct rxrpc_call *call, struct sk_buff *ack_skb)
 	struct rxrpc_skb_priv *sp;
 	struct rxrpc_txbuf *txb;
 	rxrpc_seq_t transmitted = call->tx_transmitted;
-	ktime_t next_resend = KTIME_MAX, rto = ns_to_ktime(call->peer->rto_us * NSEC_PER_USEC);
+	ktime_t next_resend = KTIME_MAX;
+	ktime_t rto = rxrpc_get_rto_backoff(call->peer, false);
 	ktime_t resend_at = KTIME_MAX, now, delay;
 	bool unacked = false, did_send = false;
 	unsigned int i;
diff --git a/net/rxrpc/rtt.c b/net/rxrpc/rtt.c
index dc26754ffd5fb..b5615401fc944 100644
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -27,7 +27,7 @@ static u32 __rxrpc_set_rto(const struct rxrpc_peer *peer)
 
 static u32 rxrpc_bound_rto(u32 rto)
 {
-	return min(rto, RXRPC_RTO_MAX);
+	return clamp(200000, rto + 100000, RXRPC_RTO_MAX);
 }
 
 /*
-- 
2.53.0



From sashal at kernel.org  Thu Aug  6 06:50:02 2026
From: sashal at kernel.org (Sasha Levin)
Date: Thu,  6 Aug 2026 09:50:02 -0400
Subject: [PATCH 6.12.y 4/5] rxrpc: Manage RTT per-call rather than per-peer
In-Reply-To: <20260806135003.602726-1-sashal@kernel.org>
References: <2026073013-sanding-unlined-9b91@gregkh>
 <20260806135003.602726-1-sashal@kernel.org>
Message-ID: <20260806135003.602726-4-sashal@kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit b40ef2b85a7d117dd323b5910e504899e0a3e7dc ]

Manage the determination of RTT on a per-call (ie. per-RPC op) basis rather
than on a per-peer basis, averaging across all calls going to that peer.
The problem is that the RTT measurements from the initial packets on a call
may be off because the server may do some setting up (such as getting a
lock on a file) before accepting the rest of the data in the RPC and,
further, the RTT may be affected by server-side file operations, for
instance if a large amount of data is being written or read.

Note: When handling the FS.StoreData-type RPCs, for example, the server
uses the userStatus field in the header of ACK packets as supplementary
flow control to aid in managing this.  AF_RXRPC does not yet support this,
but it should be added.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 include/trace/events/rxrpc.h |  2 +-
 net/rxrpc/ar-internal.h      | 39 ++++++++-------
 net/rxrpc/call_event.c       | 18 +++----
 net/rxrpc/call_object.c      |  2 +
 net/rxrpc/input.c            |  8 ++--
 net/rxrpc/output.c           | 14 +++---
 net/rxrpc/peer_object.c      |  8 +---
 net/rxrpc/proc.c             |  6 +--
 net/rxrpc/rtt.c              | 93 ++++++++++++++++++------------------
 net/rxrpc/sendmsg.c          |  2 +-
 10 files changed, 97 insertions(+), 95 deletions(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index c398897052a2a..8c4d791507f18 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -1333,7 +1333,7 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __entry->rtt = rtt;
 		    __entry->srtt = srtt;
 		    __entry->rto = rto;
-		    __entry->min_rtt = minmax_get(&call->peer->min_rtt)
+		    __entry->min_rtt = minmax_get(&call->min_rtt)
 			   ),
 
 	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u srtt=%u rto=%u min=%u",
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index f7ef2790f07af..a3afe7413e771 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -344,20 +344,9 @@ struct rxrpc_peer {
 	int			debug_id;	/* debug ID for printks */
 	struct sockaddr_rxrpc	srx;		/* remote address */
 
-	/* calculated RTT cache */
-#define RXRPC_RTT_CACHE_SIZE 32
-	spinlock_t		rtt_input_lock;	/* RTT lock for input routine */
-	ktime_t			rtt_last_req;	/* Time of last RTT request */
-	unsigned int		rtt_count;	/* Number of samples we've got */
-	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
-	struct minmax		min_rtt;	/* Estimated minimum RTT */
-
-	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
-	u32			mdev_us;	/* medium deviation			*/
-	u32			mdev_max_us;	/* maximal mdev for the last rtt period	*/
-	u32			rttvar_us;	/* smoothed mdev_max			*/
-	u32			rto_us;		/* Retransmission timeout in usec */
-	u8			backoff;	/* Backoff timeout (as shift) */
+	/* Calculated RTT cache */
+	unsigned int		recent_srtt_us;
+	unsigned int		recent_rto_us;
 
 	u8			cong_ssthresh;	/* Congestion slow-start threshold */
 };
@@ -742,6 +731,18 @@ struct rxrpc_call {
 	rxrpc_seq_t		acks_hard_ack;	/* Latest hard-ack point */
 	rxrpc_seq_t		acks_lowest_nak; /* Lowest NACK in the buffer (or ==tx_hard_ack) */
 	rxrpc_serial_t		acks_highest_serial; /* Highest serial number ACK'd */
+
+	/* Calculated RTT cache */
+	ktime_t			rtt_last_req;	/* Time of last RTT request */
+	unsigned int		rtt_count;	/* Number of samples we've got */
+	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
+	struct minmax		min_rtt;	/* Estimated minimum RTT */
+	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
+	u32			mdev_us;	/* medium deviation			*/
+	u32			mdev_max_us;	/* maximal mdev for the last rtt period	*/
+	u32			rttvar_us;	/* smoothed mdev_max			*/
+	u32			rto_us;		/* Retransmission timeout in usec */
+	u8			backoff;	/* Backoff timeout (as shift) */
 };
 
 /*
@@ -1222,10 +1223,12 @@ static inline int rxrpc_abort_eproto(struct rxrpc_call *call,
 /*
  * rtt.c
  */
-void rxrpc_peer_add_rtt(struct rxrpc_call *, enum rxrpc_rtt_rx_trace, int,
-			rxrpc_serial_t, rxrpc_serial_t, ktime_t, ktime_t);
-ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans);
-void rxrpc_peer_init_rtt(struct rxrpc_peer *);
+void rxrpc_call_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
+			int rtt_slot,
+			rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
+			ktime_t send_time, ktime_t resp_time);
+ktime_t rxrpc_get_rto_backoff(struct rxrpc_call *call, bool retrans);
+void rxrpc_call_init_rtt(struct rxrpc_call *call);
 
 /*
  * rxkad.c
diff --git a/net/rxrpc/call_event.c b/net/rxrpc/call_event.c
index c85e89b371b2a..3425e778d24f1 100644
--- a/net/rxrpc/call_event.c
+++ b/net/rxrpc/call_event.c
@@ -44,8 +44,8 @@ void rxrpc_propose_delay_ACK(struct rxrpc_call *call, rxrpc_serial_t serial,
 
 	trace_rxrpc_propose_ack(call, why, RXRPC_ACK_DELAY, serial);
 
-	if (call->peer->srtt_us)
-		delay = (call->peer->srtt_us >> 3) * NSEC_PER_USEC;
+	if (call->srtt_us)
+		delay = (call->srtt_us >> 3) * NSEC_PER_USEC;
 	else
 		delay = ms_to_ktime(READ_ONCE(rxrpc_soft_ack_delay));
 	ktime_add_ms(delay, call->tx_backoff);
@@ -72,7 +72,7 @@ void rxrpc_resend(struct rxrpc_call *call, struct sk_buff *ack_skb)
 	struct rxrpc_txbuf *txb;
 	rxrpc_seq_t transmitted = call->tx_transmitted;
 	ktime_t next_resend = KTIME_MAX;
-	ktime_t rto = rxrpc_get_rto_backoff(call->peer, false);
+	ktime_t rto = rxrpc_get_rto_backoff(call, false);
 	ktime_t resend_at = KTIME_MAX, now, delay;
 	bool unacked = false, did_send = false;
 	unsigned int i;
@@ -174,7 +174,7 @@ void rxrpc_resend(struct rxrpc_call *call, struct sk_buff *ack_skb)
 no_further_resend:
 no_resend:
 	if (resend_at < KTIME_MAX) {
-		delay = rxrpc_get_rto_backoff(call->peer, did_send);
+		delay = rxrpc_get_rto_backoff(call, did_send);
 		resend_at = ktime_add(resend_at, delay);
 		trace_rxrpc_timer_set(call, resend_at - now, rxrpc_timer_trace_resend_reset);
 	}
@@ -189,7 +189,7 @@ void rxrpc_resend(struct rxrpc_call *call, struct sk_buff *ack_skb)
 	 */
 	if (!did_send) {
 		ktime_t next_ping = ktime_add_us(call->acks_latest_ts,
-						 call->peer->srtt_us >> 3);
+						 call->srtt_us >> 3);
 
 		if (ktime_sub(next_ping, now) <= 0)
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
@@ -306,8 +306,8 @@ static void rxrpc_transmit_some_data(struct rxrpc_call *call)
  */
 static void rxrpc_send_initial_ping(struct rxrpc_call *call)
 {
-	if (call->peer->rtt_count < 3 ||
-	    ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000),
+	if (call->rtt_count < 3 ||
+	    ktime_before(ktime_add_ms(call->rtt_last_req, 1000),
 			 ktime_get_real()))
 		rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 			       rxrpc_propose_ack_ping_for_params);
@@ -438,10 +438,10 @@ bool rxrpc_input_call_event(struct rxrpc_call *call)
 			       rxrpc_propose_ack_rx_idle);
 
 	if (call->ackr_nr_unacked > 2) {
-		if (call->peer->rtt_count < 3)
+		if (call->rtt_count < 3)
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 				       rxrpc_propose_ack_ping_for_rtt);
-		else if (ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000),
+		else if (ktime_before(ktime_add_ms(call->rtt_last_req, 1000),
 				      ktime_get_real()))
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 				       rxrpc_propose_ack_ping_for_old_rtt);
diff --git a/net/rxrpc/call_object.c b/net/rxrpc/call_object.c
index 6e011a1e8e713..485c481481fc9 100644
--- a/net/rxrpc/call_object.c
+++ b/net/rxrpc/call_object.c
@@ -178,6 +178,8 @@ struct rxrpc_call *rxrpc_alloc_call(struct rxrpc_sock *rx, gfp_t gfp,
 	call->cong_cwnd = RXRPC_MIN_CWND;
 	call->cong_ssthresh = RXRPC_TX_MAX_WINDOW;
 
+	rxrpc_call_init_rtt(call);
+
 	call->rxnet = rxnet;
 	call->rtt_avail = RXRPC_CALL_RTT_AVAIL_MASK;
 	atomic_inc(&rxnet->nr_calls);
diff --git a/net/rxrpc/input.c b/net/rxrpc/input.c
index 501a049b42f86..d312feb4aaf45 100644
--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -83,11 +83,11 @@ static void rxrpc_congestion_management(struct rxrpc_call *call,
 		/* We analyse the number of packets that get ACK'd per RTT
 		 * period and increase the window if we managed to fill it.
 		 */
-		if (call->peer->rtt_count == 0)
+		if (call->rtt_count == 0)
 			goto out;
 		if (ktime_before(skb->tstamp,
 				 ktime_add_us(call->cong_tstamp,
-					      call->peer->srtt_us >> 3)))
+					      call->srtt_us >> 3)))
 			goto out_no_clear_ca;
 		change = rxrpc_cong_rtt_window_end;
 		call->cong_tstamp = skb->tstamp;
@@ -197,7 +197,7 @@ void rxrpc_congestion_degrade(struct rxrpc_call *call)
 	if (__rxrpc_call_state(call) == RXRPC_CALL_CLIENT_AWAIT_REPLY)
 		return;
 
-	rtt = ns_to_ktime(call->peer->srtt_us * (1000 / 8));
+	rtt = ns_to_ktime(call->srtt_us * (NSEC_PER_USEC / 8));
 	now = ktime_get_real();
 	if (!ktime_before(ktime_add(call->tx_last_sent, rtt), now))
 		return;
@@ -664,7 +664,7 @@ static void rxrpc_complete_rtt_probe(struct rxrpc_call *call,
 			clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail);
 			smp_mb(); /* Read data before setting avail bit */
 			set_bit(i, &call->rtt_avail);
-			rxrpc_peer_add_rtt(call, type, i, acked_serial, ack_serial,
+			rxrpc_call_add_rtt(call, type, i, acked_serial, ack_serial,
 					   sent_at, resp_time);
 			matched = true;
 		}
diff --git a/net/rxrpc/output.c b/net/rxrpc/output.c
index ad7e61066d2b5..20085175471ff 100644
--- a/net/rxrpc/output.c
+++ b/net/rxrpc/output.c
@@ -220,7 +220,7 @@ static void rxrpc_send_ack_packet(struct rxrpc_call *call, struct rxrpc_txbuf *t
 		if (ack->reason == RXRPC_ACK_PING)
 			rxrpc_begin_rtt_probe(call, txb->serial, now, rxrpc_rtt_tx_ping);
 		if (txb->flags & RXRPC_REQUEST_ACK)
-			call->peer->rtt_last_req = now;
+			call->rtt_last_req = now;
 		rxrpc_set_keepalive(call, now);
 	}
 	rxrpc_tx_backoff(call, ret);
@@ -358,9 +358,9 @@ static void rxrpc_prepare_data_subpacket(struct rxrpc_call *call, struct rxrpc_t
 		why = rxrpc_reqack_slow_start;
 	else if (call->tx_winsize <= 2)
 		why = rxrpc_reqack_small_txwin;
-	else if (call->peer->rtt_count < 3 && txb->seq & 1)
+	else if (call->rtt_count < 3)
 		why = rxrpc_reqack_more_rtt;
-	else if (ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000), ktime_get_real()))
+	else if (ktime_before(ktime_add_ms(call->rtt_last_req, 1000), ktime_get_real()))
 		why = rxrpc_reqack_old_rtt;
 	else
 		goto dont_set_request_ack;
@@ -407,9 +407,9 @@ static void rxrpc_tstamp_data_packets(struct rxrpc_call *call, struct rxrpc_txbu
 	if (ack_requested) {
 		rxrpc_begin_rtt_probe(call, txb->serial, now, rxrpc_rtt_tx_data);
 
-		call->peer->rtt_last_req = now;
-		if (call->peer->rtt_count > 1) {
-			ktime_t delay = rxrpc_get_rto_backoff(call->peer, false);
+		call->rtt_last_req = now;
+		if (call->rtt_count > 1) {
+			ktime_t delay = rxrpc_get_rto_backoff(call, false);
 
 			call->ack_lost_at = ktime_add(now, delay);
 			trace_rxrpc_timer_set(call, delay, rxrpc_timer_trace_lost_ack);
@@ -727,7 +727,7 @@ void rxrpc_transmit_one(struct rxrpc_call *call, struct rxrpc_txbuf *txb)
 			rxrpc_instant_resend(call, txb);
 		}
 	} else {
-		ktime_t delay = ns_to_ktime(call->peer->rto_us * NSEC_PER_USEC);
+		ktime_t delay = ns_to_ktime(call->rto_us * NSEC_PER_USEC);
 
 		call->resend_at = ktime_add(ktime_get_real(), delay);
 		trace_rxrpc_timer_set(call, delay, rxrpc_timer_trace_resend_tx);
diff --git a/net/rxrpc/peer_object.c b/net/rxrpc/peer_object.c
index a09b8aaf9bcdf..45d5f25db40b2 100644
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -222,11 +222,8 @@ struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *local, gfp_t gfp,
 		peer->service_conns = RB_ROOT;
 		seqlock_init(&peer->service_conn_lock);
 		spin_lock_init(&peer->lock);
-		spin_lock_init(&peer->rtt_input_lock);
 		peer->debug_id = atomic_inc_return(&rxrpc_debug_id);
-
-		rxrpc_peer_init_rtt(peer);
-
+		peer->recent_srtt_us = UINT_MAX;
 		peer->cong_ssthresh = RXRPC_TX_MAX_WINDOW;
 		trace_rxrpc_peer(peer->debug_id, 1, why);
 	}
@@ -244,7 +241,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer,
 	peer->hash_key = hash_key;
 	rxrpc_assess_MTU_size(local, peer);
 	peer->mtu = peer->if_mtu;
-	peer->rtt_last_req = ktime_get_real();
 
 	switch (peer->srx.transport.family) {
 	case AF_INET:
@@ -480,7 +476,7 @@ EXPORT_SYMBOL(rxrpc_kernel_get_call_peer);
  */
 unsigned int rxrpc_kernel_get_srtt(const struct rxrpc_peer *peer)
 {
-	return peer->rtt_count > 0 ? peer->srtt_us >> 3 : UINT_MAX;
+	return READ_ONCE(peer->recent_srtt_us);
 }
 EXPORT_SYMBOL(rxrpc_kernel_get_srtt);
 
diff --git a/net/rxrpc/proc.c b/net/rxrpc/proc.c
index ca85ac764f82b..2dd9567cb1cfd 100644
--- a/net/rxrpc/proc.c
+++ b/net/rxrpc/proc.c
@@ -299,15 +299,15 @@ static int rxrpc_peer_seq_show(struct seq_file *seq, void *v)
 	now = ktime_get_seconds();
 	seq_printf(seq,
 		   "UDP   %-47.47s %-47.47s %3u"
-		   " %3u %5u %6ds %8u %8u\n",
+		   " %3u %5u %6ds %8d %8d\n",
 		   lbuff,
 		   rbuff,
 		   refcount_read(&peer->ref),
 		   peer->cong_ssthresh,
 		   peer->mtu,
 		   (s32)now - (s32)READ_ONCE(peer->last_tx_at),
-		   peer->srtt_us >> 3,
-		   peer->rto_us);
+		   READ_ONCE(peer->recent_srtt_us),
+		   READ_ONCE(peer->recent_rto_us));
 
 	return 0;
 }
diff --git a/net/rxrpc/rtt.c b/net/rxrpc/rtt.c
index b5615401fc944..7474f88d7b180 100644
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -12,17 +12,17 @@
 #include "ar-internal.h"
 
 #define RXRPC_RTO_MAX	(120 * USEC_PER_SEC)
-#define RXRPC_TIMEOUT_INIT ((unsigned int)(1 * MSEC_PER_SEC)) /* RFC6298 2.1 initial RTO value */
+#define RXRPC_TIMEOUT_INIT ((unsigned int)(1 * USEC_PER_SEC)) /* RFC6298 2.1 initial RTO value */
 #define rxrpc_jiffies32 ((u32)jiffies)		/* As rxrpc_jiffies32 */
 
-static u32 rxrpc_rto_min_us(struct rxrpc_peer *peer)
+static u32 rxrpc_rto_min_us(struct rxrpc_call *call)
 {
 	return 200;
 }
 
-static u32 __rxrpc_set_rto(const struct rxrpc_peer *peer)
+static u32 __rxrpc_set_rto(const struct rxrpc_call *call)
 {
-	return (peer->srtt_us >> 3) + peer->rttvar_us;
+	return (call->srtt_us >> 3) + call->rttvar_us;
 }
 
 static u32 rxrpc_bound_rto(u32 rto)
@@ -40,10 +40,10 @@ static u32 rxrpc_bound_rto(u32 rto)
  * To save cycles in the RFC 1323 implementation it was better to break
  * it up into three procedures. -- erics
  */
-static void rxrpc_rtt_estimator(struct rxrpc_peer *peer, long sample_rtt_us)
+static void rxrpc_rtt_estimator(struct rxrpc_call *call, long sample_rtt_us)
 {
 	long m = sample_rtt_us; /* RTT */
-	u32 srtt = peer->srtt_us;
+	u32 srtt = call->srtt_us;
 
 	/*	The following amusing code comes from Jacobson's
 	 *	article in SIGCOMM '88.  Note that rtt and mdev
@@ -66,7 +66,7 @@ static void rxrpc_rtt_estimator(struct rxrpc_peer *peer, long sample_rtt_us)
 		srtt += m;		/* rtt = 7/8 rtt + 1/8 new */
 		if (m < 0) {
 			m = -m;		/* m is now abs(error) */
-			m -= (peer->mdev_us >> 2);   /* similar update on mdev */
+			m -= (call->mdev_us >> 2);   /* similar update on mdev */
 			/* This is similar to one of Eifel findings.
 			 * Eifel blocks mdev updates when rtt decreases.
 			 * This solution is a bit different: we use finer gain
@@ -78,31 +78,31 @@ static void rxrpc_rtt_estimator(struct rxrpc_peer *peer, long sample_rtt_us)
 			if (m > 0)
 				m >>= 3;
 		} else {
-			m -= (peer->mdev_us >> 2);   /* similar update on mdev */
+			m -= (call->mdev_us >> 2);   /* similar update on mdev */
 		}
 
-		peer->mdev_us += m;		/* mdev = 3/4 mdev + 1/4 new */
-		if (peer->mdev_us > peer->mdev_max_us) {
-			peer->mdev_max_us = peer->mdev_us;
-			if (peer->mdev_max_us > peer->rttvar_us)
-				peer->rttvar_us = peer->mdev_max_us;
+		call->mdev_us += m;		/* mdev = 3/4 mdev + 1/4 new */
+		if (call->mdev_us > call->mdev_max_us) {
+			call->mdev_max_us = call->mdev_us;
+			if (call->mdev_max_us > call->rttvar_us)
+				call->rttvar_us = call->mdev_max_us;
 		}
 	} else {
 		/* no previous measure. */
 		srtt = m << 3;		/* take the measured time to be rtt */
-		peer->mdev_us = m << 1;	/* make sure rto = 3*rtt */
-		peer->rttvar_us = max(peer->mdev_us, rxrpc_rto_min_us(peer));
-		peer->mdev_max_us = peer->rttvar_us;
+		call->mdev_us = m << 1;	/* make sure rto = 3*rtt */
+		call->rttvar_us = umax(call->mdev_us, rxrpc_rto_min_us(call));
+		call->mdev_max_us = call->rttvar_us;
 	}
 
-	peer->srtt_us = max(1U, srtt);
+	call->srtt_us = umax(srtt, 1);
 }
 
 /*
  * Calculate rto without backoff.  This is the second half of Van Jacobson's
  * routine referred to above.
  */
-static void rxrpc_set_rto(struct rxrpc_peer *peer)
+static void rxrpc_set_rto(struct rxrpc_call *call)
 {
 	u32 rto;
 
@@ -113,7 +113,7 @@ static void rxrpc_set_rto(struct rxrpc_peer *peer)
 	 *    is invisible. Actually, Linux-2.4 also generates erratic
 	 *    ACKs in some circumstances.
 	 */
-	rto = __rxrpc_set_rto(peer);
+	rto = __rxrpc_set_rto(call);
 
 	/* 2. Fixups made earlier cannot be right.
 	 *    If we do not estimate RTO correctly without them,
@@ -124,73 +124,73 @@ static void rxrpc_set_rto(struct rxrpc_peer *peer)
 	/* NOTE: clamping at RXRPC_RTO_MIN is not required, current algo
 	 * guarantees that rto is higher.
 	 */
-	peer->rto_us = rxrpc_bound_rto(rto);
+	call->rto_us = rxrpc_bound_rto(rto);
 }
 
-static void rxrpc_update_rtt_min(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+static void rxrpc_update_rtt_min(struct rxrpc_call *call, ktime_t resp_time, long rtt_us)
 {
 	/* Window size 5mins in approx usec (ipv4.sysctl_tcp_min_rtt_wlen) */
 	u32 wlen_us = 5ULL * NSEC_PER_SEC / 1024;
 
-	minmax_running_min(&peer->min_rtt, wlen_us, resp_time / 1024,
+	minmax_running_min(&call->min_rtt, wlen_us, resp_time / 1024,
 			   (u32)rtt_us ? : jiffies_to_usecs(1));
 }
 
-static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+static void rxrpc_ack_update_rtt(struct rxrpc_call *call, ktime_t resp_time, long rtt_us)
 {
 	if (rtt_us < 0)
 		return;
 
 	/* Update RACK min RTT [RFC8985 6.1 Step 1]. */
-	rxrpc_update_rtt_min(peer, resp_time, rtt_us);
+	rxrpc_update_rtt_min(call, resp_time, rtt_us);
 
-	rxrpc_rtt_estimator(peer, rtt_us);
-	rxrpc_set_rto(peer);
+	rxrpc_rtt_estimator(call, rtt_us);
+	rxrpc_set_rto(call);
 
 	/* Only reset backoff on valid RTT measurement [RFC6298]. */
-	peer->backoff = 0;
+	call->backoff = 0;
 }
 
 /*
  * Add RTT information to cache.  This is called in softirq mode and has
- * exclusive access to the peer RTT data.
+ * exclusive access to the call RTT data.
  */
-void rxrpc_peer_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
+void rxrpc_call_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
 			int rtt_slot,
 			rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
 			ktime_t send_time, ktime_t resp_time)
 {
-	struct rxrpc_peer *peer = call->peer;
 	s64 rtt_us;
 
 	rtt_us = ktime_to_us(ktime_sub(resp_time, send_time));
 	if (rtt_us < 0)
 		return;
 
-	spin_lock(&peer->rtt_input_lock);
-	rxrpc_ack_update_rtt(peer, resp_time, rtt_us);
-	if (peer->rtt_count < 3)
-		peer->rtt_count++;
-	peer->rtt_taken++;
-	spin_unlock(&peer->rtt_input_lock);
+	rxrpc_ack_update_rtt(call, resp_time, rtt_us);
+	if (call->rtt_count < 3)
+		call->rtt_count++;
+	call->rtt_taken++;
+
+	WRITE_ONCE(call->peer->recent_srtt_us, call->srtt_us / 8);
+	WRITE_ONCE(call->peer->recent_rto_us, call->rto_us);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,
-			   rtt_us, peer->srtt_us, peer->rto_us);
+			   rtt_us, call->srtt_us, call->rto_us);
 }
 
 /*
  * Get the retransmission timeout to set in nanoseconds, backing it off each
  * time we retransmit.
  */
-ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans)
+ktime_t rxrpc_get_rto_backoff(struct rxrpc_call *call, bool retrans)
 {
 	u64 timo_us;
-	u32 backoff = READ_ONCE(peer->backoff);
+	u32 backoff = READ_ONCE(call->backoff);
 
-	timo_us = peer->rto_us;
+	timo_us = call->rto_us;
 	timo_us <<= backoff;
 	if (retrans && timo_us * 2 <= RXRPC_RTO_MAX)
-		WRITE_ONCE(peer->backoff, backoff + 1);
+		WRITE_ONCE(call->backoff, backoff + 1);
 
 	if (timo_us < 1)
 		timo_us = 1;
@@ -198,10 +198,11 @@ ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans)
 	return ns_to_ktime(timo_us * NSEC_PER_USEC);
 }
 
-void rxrpc_peer_init_rtt(struct rxrpc_peer *peer)
+void rxrpc_call_init_rtt(struct rxrpc_call *call)
 {
-	peer->rto_us	= RXRPC_TIMEOUT_INIT;
-	peer->mdev_us	= RXRPC_TIMEOUT_INIT;
-	peer->backoff	= 0;
-	//minmax_reset(&peer->rtt_min, rxrpc_jiffies32, ~0U);
+	call->rtt_last_req = KTIME_MIN;
+	call->rto_us	= RXRPC_TIMEOUT_INIT;
+	call->mdev_us	= RXRPC_TIMEOUT_INIT;
+	call->backoff	= 0;
+	//minmax_reset(&call->rtt_min, rxrpc_jiffies32, ~0U);
 }
diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index dc43bf8cbac95..1d8065468e90a 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -133,7 +133,7 @@ static int rxrpc_wait_for_tx_window_waitall(struct rxrpc_sock *rx,
 	rxrpc_seq_t tx_start, tx_win;
 	signed long rtt, timeout;
 
-	rtt = READ_ONCE(call->peer->srtt_us) >> 3;
+	rtt = READ_ONCE(call->srtt_us) >> 3;
 	rtt = usecs_to_jiffies(rtt) * 2;
 	if (rtt < 2)
 		rtt = 2;
-- 
2.53.0



From kuba at kernel.org  Thu Aug  6 09:15:11 2026
From: kuba at kernel.org (Jakub Kicinski)
Date: Thu, 6 Aug 2026 09:15:11 -0700
Subject: [PATCH net v6 00/10] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <20260804172639.2844491-1-dhowells@redhat.com>
References: <20260804172639.2844491-1-dhowells@redhat.com>
Message-ID: <20260806091511.46a2fabb@kernel.org>

On Tue,  4 Aug 2026 18:26:27 +0100 David Howells wrote:
> [!] Note that this entails a significant change in the UAPI for AF_RXRPC,
> with the CMSG types and sockopt to support the OOB queuing being removed
> and replaced with a new single CMSG type that conveys the user key ID.  I
> don't think it likely anyone is using this outside of my kafs-utils
> package.

If the remaining Sashiko issues are false-positives - should we queue
this to net-next, then?


From gregkh at linuxfoundation.org  Fri Aug  7 06:07:26 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Fri, 07 Aug 2026 15:07:26 +0200
Subject: Patch "rxrpc: Adjust the rxrpc_rtt_rx tracepoint" has been added to the 6.12-stable tree
In-Reply-To: <20260806135003.602726-2-sashal@kernel.org>
Message-ID: <2026080726-occultist-stipend-7e9c@gregkh>


This is a note to let you know that I've just added the patch titled

    rxrpc: Adjust the rxrpc_rtt_rx tracepoint

to the 6.12-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     rxrpc-adjust-the-rxrpc_rtt_rx-tracepoint.patch
and it can be found in the queue-6.12 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From stable+bounces-296835-greg=kroah.com at vger.kernel.org Thu Aug  6 15:55:57 2026
From: Sasha Levin <sashal at kernel.org>
Date: Thu,  6 Aug 2026 09:50:00 -0400
Subject: rxrpc: Adjust the rxrpc_rtt_rx tracepoint
To: stable at vger.kernel.org
Cc: David Howells <dhowells at redhat.com>, Marc Dionne <marc.dionne at auristor.com>, linux-afs at lists.infradead.org, Jakub Kicinski <kuba at kernel.org>, Sasha Levin <sashal at kernel.org>
Message-ID: <20260806135003.602726-2-sashal at kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit 93dfca65a1df42a3c8b1094299dc42ab8f18e5c8 ]

Adjust the rxrpc_rtt_rx tracepoint in the following ways:

 (1) Display the collected RTT sample in the rxrpc_rtt_rx trace.

 (2) Move the division of srtt by 8 to the TP_printk() rather doing it
     before invoking the trace point.

 (3) Display the min_rtt value.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 include/trace/events/rxrpc.h |   14 ++++++++++----
 net/rxrpc/input.c            |    4 ++--
 net/rxrpc/rtt.c              |    2 +-
 3 files changed, 13 insertions(+), 7 deletions(-)

--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -1308,9 +1308,9 @@ TRACE_EVENT(rxrpc_rtt_rx,
 	    TP_PROTO(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
 		     int slot,
 		     rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
-		     u32 rtt, u32 rto),
+		     u32 rtt, u32 srtt, u32 rto),
 
-	    TP_ARGS(call, why, slot, send_serial, resp_serial, rtt, rto),
+	    TP_ARGS(call, why, slot, send_serial, resp_serial, rtt, srtt, rto),
 
 	    TP_STRUCT__entry(
 		    __field(unsigned int,		call)
@@ -1319,7 +1319,9 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __field(rxrpc_serial_t,		send_serial)
 		    __field(rxrpc_serial_t,		resp_serial)
 		    __field(u32,			rtt)
+		    __field(u32,			srtt)
 		    __field(u32,			rto)
+		    __field(u32,			min_rtt)
 			     ),
 
 	    TP_fast_assign(
@@ -1329,17 +1331,21 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __entry->send_serial = send_serial;
 		    __entry->resp_serial = resp_serial;
 		    __entry->rtt = rtt;
+		    __entry->srtt = srtt;
 		    __entry->rto = rto;
+		    __entry->min_rtt = minmax_get(&call->peer->min_rtt)
 			   ),
 
-	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u rto=%u",
+	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u srtt=%u rto=%u min=%u",
 		      __entry->call,
 		      __entry->slot,
 		      __print_symbolic(__entry->why, rxrpc_rtt_rx_traces),
 		      __entry->send_serial,
 		      __entry->resp_serial,
 		      __entry->rtt,
-		      __entry->rto)
+		      __entry->srtt / 8,
+		      __entry->rto,
+		      __entry->min_rtt)
 	    );
 
 TRACE_EVENT(rxrpc_timer_set,
--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -674,7 +674,7 @@ static void rxrpc_complete_rtt_probe(str
 		 */
 		if (after(acked_serial, orig_serial)) {
 			trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_obsolete, i,
-					   orig_serial, acked_serial, 0, 0);
+					   orig_serial, acked_serial, 0, 0, 0);
 			clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail);
 			smp_wmb();
 			set_bit(i, &call->rtt_avail);
@@ -682,7 +682,7 @@ static void rxrpc_complete_rtt_probe(str
 	}
 
 	if (!matched)
-		trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_lost, 9, 0, acked_serial, 0, 0);
+		trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_lost, 9, 0, acked_serial, 0, 0, 0);
 }
 
 /*
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -175,7 +175,7 @@ void rxrpc_peer_add_rtt(struct rxrpc_cal
 	spin_unlock(&peer->rtt_input_lock);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,
-			   peer->srtt_us >> 3, peer->rto_us);
+			   rtt_us, peer->srtt_us, peer->rto_us);
 }
 
 /*


Patches currently in stable-queue which might be from sashal at kernel.org are

queue-6.12/wifi-brcmfmac-set-f2-blocksize-to-256-for-bcm43752.patch
queue-6.12/net-dsa-mt7530-check-bus-read-errors-in-the-mdio-reg.patch
queue-6.12/um-set-parent-death-signal-for-userspace-process.patch
queue-6.12/usb-gadget-f_tcm-synchronize-delayed-set_alt-with-teardown.patch
queue-6.12/drm-xe-rtp-de-whitelist-oa-registers-for-all-hwe-s-f.patch
queue-6.12/media-uapi-rkisp-correct-name-version-enum.patch
queue-6.12/net-sxgbe-free-tx-rings-on-rx-allocation-failure.patch
queue-6.12/rtase-fix-double-free-of-multi-frag-skb-on-dma-map-f.patch
queue-6.12/usb-typec-ucsi-fix-race-condition-and-ordering-in-port-unregistration.patch
queue-6.12/drm-xe-introduce-xe_gt_dbg_printer.patch
queue-6.12/drm-fb-helper-allocate-and-release-fb_info-in-single-place.patch
queue-6.12/drm-xe-rtp-maintain-oa-whitelists-separately.patch
queue-6.12/pinctrl-qcom-sc8280xp-add-missing-wakeup-entries-for.patch
queue-6.12/forcedeth-fix-uaf-of-txrx_stats-in-nv_remove.patch
queue-6.12/alsa-hda-codecs-hdmi-disable-keep-alive-before-audio-format-change.patch
queue-6.12/hid-logitech-dj-standardise-hid_report_enum-variable.patch
queue-6.12/scsi-libsas-fix-ha-resume-deadlock-and-hisi_sas-disk.patch
queue-6.12/netfilter-nft_payload-fix-mask-build-for-partial-fie.patch
queue-6.12/bluetooth-iso-fix-timeout-vs-sync_timeout-typo-in-ch.patch
queue-6.12/um-set-parent-death-signal-for-ubd-io-thread-process.patch
queue-6.12/net-sxgbe-check-descriptor-ring-allocation-failures.patch
queue-6.12/ipvs-fix-the-checksum-validations.patch
queue-6.12/mptcp-pm-avoid-code-duplication-to-lookup-endp.patch
queue-6.12/hwmon-ina2xx-fix-various-overflow-issues.patch
queue-6.12/spi-spi-cadence-move-tx-fifo-full-busy-wait-into-fif.patch
queue-6.12/powerpc-boot-fix-simpleboot-cpu-node-lookup-check.patch
queue-6.12/ata-sata_mv-accept-1-or-2-resources-in-platform-prob.patch
queue-6.12/drm-xe-rtp-generalize-whitelist_apply_to_hwe.patch
queue-6.12/wifi-mac80211-validate-individual-twt-params-before-.patch
queue-6.12/mm-hugetlb-fix-swap-entry-corruption-when-clearing-u.patch
queue-6.12/dmaengine-idxd-fix-fdev-setup-failure-cleanup-in-idx.patch
queue-6.12/asoc-max98090-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/keys-trusted-dcp-fix-key_len-validation-and-calc_blo.patch
queue-6.12/drm-xe-oa-de-whitelist-oa-registers-on-oa-stream-ope.patch
queue-6.12/mptcp-pm-use-addr-entry-for-get_local_id.patch
queue-6.12/drm-xe-rtp-save-oa-nonpriv-registers-to-register-sav.patch
queue-6.12/bluetooth-hci_sync-remove-unnecessary-hci_conn_get-i.patch
queue-6.12/drm-mediatek-check-crtc-state-before-freeing.patch
queue-6.12/bluetooth-iso-clear-iso_data-always-when-detaching-c.patch
queue-6.12/bluetooth-iso-fix-leaking-sk-after-socket-release.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina260.patch
queue-6.12/accel-qaic-use-sizeof-trans_hdr-for-transaction-leng.patch
queue-6.12/pinctrl-amd-don-t-clear-s4-wake-bits-at-probe.patch
queue-6.12/net-bridge-mrp-fix-option-tlv-length-in-mrp_test-fra.patch
queue-6.12/kmemleak-iommu-iova-fix-transient-kmemleak-false-positive.patch
queue-6.12/gpio-pch-use-raw_spinlock_t-for-the-register-lock.patch
queue-6.12/rxrpc-generate-rtt_min.patch
queue-6.12/drm-xe-rtp-refactor-oag-mmio-trigger-register-whitel.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina234.patch
queue-6.12/drm-xe-apply-whitelist-to-engine-save-restore.patch
queue-6.12/powerpc-boot-fix-treeboot-currituck-cpu-node-lookup-.patch
queue-6.12/wifi-ath6kl-fix-use-after-free-in-aggr_reset_state.patch
queue-6.12/wifi-brcmfmac-fix-43752-sdio-fwvid-incorrectly-labelled-as-cypress-cyw.patch
queue-6.12/keys-make-keyring-key-chunk-byte-order-agree-with-ke.patch
queue-6.12/drm-xe-pt-reset-current_op-in-xe_pt_update_ops_init.patch
queue-6.12/drm-xe-rtp-toggle-deny-bit-to-de-whitelist-oa-regs.patch
queue-6.12/hwmon-adt7470-fix-fans-stuck-in-manual-mode-on-i2c-e.patch
queue-6.12/net-airoha-fix-register-index-for-tx-fwd-counter-con.patch
queue-6.12/ata-ahci_ceva-fix-error-paths-in-ceva_ahci_platform_.patch
queue-6.12/qede-sync-udp_tunnel-ports-outside-qede_lock-in-the-.patch
queue-6.12/rxrpc-fix-the-calculation-and-use-of-rto.patch
queue-6.12/tracing-mmiotrace-reset-dropped_count-in-mmio_reset_.patch
queue-6.12/netfilter-nf_conntrack_sip-widen-nat-rewrite-delta-t.patch
queue-6.12/drm-i915-hdcp-move-to-using-intel_display-in-intel_hdcp.patch
queue-6.12/usb-typec-ucsi-split-connector-lock-classes.patch
queue-6.12/ahci-introduce-ahci_ignore_port-helper.patch
queue-6.12/drm-i915-hdcp-require-monotonically-increasing-seq_num_v.patch
queue-6.12/net-udp_tunnel-fix-memory-leak-in-udp_tunnel_nic_unr.patch
queue-6.12/bluetooth-iso-validate-sockaddr_iso-first-in-iso_soc.patch
queue-6.12/net-libwx-fix-fdir-atr-queue-mismatch-for-software-v.patch
queue-6.12/riscv-mm-fix-out-of-bounds-page-table-walk-during-me.patch
queue-6.12/netfilter-nf_tables-make-nft_object-rhltable-per-tab.patch
queue-6.12/mm-huge_memory-unlock-i_mmap_rwsem-before-releasing-.patch
queue-6.12/bluetooth-l2cap-fix-uaf-in-l2cap_le_connect_rsp.patch
queue-6.12/hwmon-ina226-add-support-for-sy24655.patch
queue-6.12/scsi-zfcp-fix-memory-leak-during-adapter-release-by-.patch
queue-6.12/mptcp-pm-userspace-fix-use-after-free-in-get_local_id.patch
queue-6.12/hwmon-ina2xx-make-it-easier-to-add-more-devices.patch
queue-6.12/net-dsa-mt7530-error-out-on-failed-reads-in-mt7531-p.patch
queue-6.12/scsi-libiscsi_tcp-bound-scsi-response-data-segment-t.patch
queue-6.12/bluetooth-hci_conn-hold-conn-reference-in-abort_conn.patch
queue-6.12/ipvs-do-not-mangle-icmp-replies-for-non-first-fragme.patch
queue-6.12/drm-i915-hdcp-check-streams-bounds-before-overflow.patch
queue-6.12/phy-zynqmp-fix-clock-error-handling-in-xpsgtr_phy_in.patch
queue-6.12/asoc-max98095-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/hid-logitech-dj-fix-wrong-detection-of-bad-dj_short-.patch
queue-6.12/rds-tcp-hold-the-rcu-lock-across-ipv6_chk_addr-in-rd.patch
queue-6.12/drm-xe-rtp-ensure-locking-ref-counting-for-oa-whitel.patch
queue-6.12/um-add-os_set_pdeathsig-helper-function.patch
queue-6.12/hwmon-lm90-only-report-alarms-if-driver-is-ready.patch
queue-6.12/of-reserved_mem-add-code-to-dynamically-allocate-res.patch
queue-6.12/bpf-reset-register-bounds-before-narrowing-retval-ra.patch
queue-6.12/drm-xe-rtp-keep-track-of-non-oa-nonpriv-slots.patch
queue-6.12/net-ethernet-mtk_eth_soc-support-named-irqs.patch
queue-6.12/wifi-brcmfmac-drain-bus_reset-work-on-device-removal.patch
queue-6.12/hwmon-adt7470-fix-divide-by-zero-toctou-crash-in-fan.patch
queue-6.12/mptcp-add-mptcp_userspace_pm_lookup_addr-helper.patch
queue-6.12/drm-sched-store-the-drm-client_id-in-drm_sched_fence.patch
queue-6.12/of-reserved_mem-prevent-oob-when-too-many-dynamic-re.patch
queue-6.12/hwmon-adt7470-fix-temperature-alarm-logic-in-hwmon_t.patch
queue-6.12/ksmbd-return-success-for-deferred-final-close.patch
queue-6.12/hwmon-ina2xx-add-support-for-has_alerts-configuratio.patch
queue-6.12/drm-tegra-fbdev-remove-offset-into-framebuffer-memory.patch
queue-6.12/scsi-ufs-core-cancel-rtc-work-in-active-active-suspe.patch
queue-6.12/drm-exec-remove-the-index-parameter-from-drm_exec_for_each_locked_obj.patch
queue-6.12/hwmon-adt7470-fix-cache-updated-before-hardware-writ.patch
queue-6.12/assoc_array-trim-the-final-shortcut-word-using-the-c.patch
queue-6.12/mm-kmemleak-fix-checksum-computation-for-per-cpu-objects.patch
queue-6.12/ksmbd-fix-use-after-free-in-__close_file_table_ids.patch
queue-6.12/drm-xe-wait-on-external-bo-kernel-fences-in-exec-ioctl.patch
queue-6.12/drm-amdgpu-gfx-fix-cleaner-shader-ib-buffer-overflow.patch
queue-6.12/netfilter-xt_hashlimit-validate-hashtable-supports-x.patch
queue-6.12/scsi-target-iblock-fix-wrong-pr-ops-null-check-for-p.patch
queue-6.12/net-ethernet-mtk_eth_soc-add-consts-for-irq-index.patch
queue-6.12/hwmon-adt7470-fix-swapped-pwm3-and-pwm4-auto-mode-ma.patch
queue-6.12/media-chips-media-wave5-support-cbp-profile.patch
queue-6.12/um-use-os_set_pdeathsig-helper-in-winch-thread-proce.patch
queue-6.12/idpf-fix-mailbox-irq-name-leak-on-request-failure.patch
queue-6.12/drm-i915-vrr-check-has_vrr-first-in-intel_vrr_is_capable.patch
queue-6.12/btrfs-fix-leaking-btrfs_fs_state_remounting-flag.patch
queue-6.12/netfilter-br_netfilter-reallocate-headroom-if-necess.patch
queue-6.12/idpf-adjust-txq-ring-count-minimum.patch
queue-6.12/pinctrl-qcom-unconditionally-mark-gpio-as-wakeup-ena.patch
queue-6.12/gpio-sloppy-logic-analyzer-fix-memory-leak-in-gpio_l.patch
queue-6.12/keys-fix-out-of-bounds-read-in-keyring_get_key_chunk.patch
queue-6.12/drm-amdgpu-fix-context-pstate-override-handling.patch
queue-6.12/hwmon-sht3x-fix-unaligned-accesses.patch
queue-6.12/mm-slab-prevent-unbounded-recursion-in-free-path-wit.patch
queue-6.12/fs-proc-task_mmu-fix-pagemap_scan-written-state-for-.patch
queue-6.12/hwmon-adt7470-use-cached-pwm-frequency-value.patch
queue-6.12/hwmon-pmbus-fix-return-value-from-pmbus_update_byte_.patch
queue-6.12/phy-zynqmp-fix-runtime-pm-leak-on-probe-allocation-f.patch
queue-6.12/scsi-libiscsi-fix-stale-data-leak-into-the-scsi-sens.patch
queue-6.12/tracing-mmiotrace-add-null-check-for-mmio_trace_arra.patch
queue-6.12/drm-xe-rtp-add-ring_force_to_nonpriv_deny-to-oa-whit.patch
queue-6.12/spi-spi-cadence-supports-transmission-with-bits_per_.patch
queue-6.12/kunit-tool-skip-stty-when-stdin-is-not-a-tty.patch
queue-6.12/um-preserve-errno-within-signal-handler.patch
queue-6.12/ipvs-fix-places-with-wrong-packet-offsets.patch
queue-6.12/hwmon-adt7470-fix-pwm-auto-temp-state-array-and-boun.patch
queue-6.12/hwmon-ina2xx-shift-ina234-shunt-and-current-register.patch
queue-6.12/drm-amdgpu-give-each-kernel-job-a-unique-id.patch
queue-6.12/netconsole-avoid-oob-reads-msg-is-not-nul-terminated.patch
queue-6.12/bluetooth-hci_sync-make-hci_cmd_sync_run_once-return.patch
queue-6.12/hwmon-adt7470-fix-busy-loop-and-i2c-flooding-in-upda.patch
queue-6.12/bluetooth-iso-avoid-deadlocks-in-iso_sock_timeout.patch
queue-6.12/rxrpc-fix-irq-disabled-in-local_bh_enable.patch
queue-6.12/drm-i915-hdcp-skip-inactive-mst-connectors-when-building-stream-list.patch
queue-6.12/hwmon-nct6775-core-fix-number-of-temperature-registe.patch
queue-6.12/hwmon-nct6775-core-prevent-access-to-unsupported-wei.patch
queue-6.12/dmaengine-sun6i-dma-fix-reclaim-descriptors-while-te.patch
queue-6.12/drm-xe-hold-a-dma-buf-reference-for-imported-bos.patch
queue-6.12/sched-deadline-use-revised-wakeup-rule-only-for-runn.patch
queue-6.12/rds-fix-inet6_addr_lst-null-dereference-when-ipv6-is.patch
queue-6.12/rxrpc-adjust-the-rxrpc_rtt_rx-tracepoint.patch
queue-6.12/media-i2c-imx219-rename-vts-to-frm_length.patch
queue-6.12/smb-client-fix-buffer-leaks-in-smb1-read-and-write.patch
queue-6.12/scsi-target-clear-cmd_cnt-when-initial-counter-enrol.patch
queue-6.12/lib-alloc_tag-introduce-mem_alloc_profiling_permanen.patch
queue-6.12/rxrpc-manage-rtt-per-call-rather-than-per-peer.patch
queue-6.12/bluetooth-hci_sync-fix-hci_conn_del-use-in-hci_le_cr.patch
queue-6.12/net-phylink-put-link_gpio-if-phylink_create-fails.patch
queue-6.12/btrfs-zoned-fix-deadlock-between-metadata-writeback-.patch
queue-6.12/media-imx219-fix-maximum-frame-length-in-lines.patch
queue-6.12/tracing-remove-trace_event_fl_filtered-logic.patch
queue-6.12/drm-i915-vrr-require-valid-min-max-vfreq-for-vrr.patch
queue-6.12/um-set-parent-death-signal-for-write_sigio-thread-pr.patch
queue-6.12/hwmon-nzxt-smart2-dma-align-output-buffer.patch
queue-6.12/rhashtable-clear-stale-iter-p-on-table-restart.patch
queue-6.12/drm-amdgpu-respect-placement-requirements-in-amdgpu_gtt_mgr-functions.patch
queue-6.12/powerpc-boot-fix-treeboot-akebono-cpu-node-lookup-ch.patch
queue-6.12/hwmon-ltc4282-fix-reading-the-minimum-alarm-voltage.patch
queue-6.12/thunderbolt-prevent-xdomain-delayed-work-use-after-f.patch
queue-6.12/ata-libahci_platform-support-non-consecutive-port-nu.patch
queue-6.12/octeontx2-pf-set-correct-sequence-for-carrier-off-an.patch
queue-6.12/net-mpls-initialize-rtm_tos-in-mpls_getroute.patch
queue-6.12/um-set-parent-death-signal-for-winch-thread-process.patch
queue-6.12/netfilter-nf_conntrack_expect-restore-helper-propaga.patch
queue-6.12/drm-xe-stub-out-new-pagefault-layer.patch
queue-6.12/bluetooth-btintel-validate-length-before-parsing-dia.patch
queue-6.12/net-ethernet-mtk_eth_soc-pass-eth-to-mtk_handle_irq_.patch
queue-6.12/drivers-hv-vmbus-replace-lockdep_hardirq_threaded-wi.patch
queue-6.12/drm-xe-rename-___xe_bo_create_locked.patch
queue-6.12/hid-logitech-dj-prevent-report_id_dj_short-related-u.patch
queue-6.12/net-do-not-send-icmp-ndisc-redirects-when-peer-alloc.patch
queue-6.12/phy-zynqmp-postpone-getting-clock-rate-until-actuall.patch
queue-6.12/kunit-tool-terminate-kernel-under-test-on-sigint.patch
queue-6.12/can-isotp-check-register_netdevice_notifier-error-in.patch


From gregkh at linuxfoundation.org  Fri Aug  7 06:07:26 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Fri, 07 Aug 2026 15:07:26 +0200
Subject: Patch "rxrpc: Fix the calculation and use of RTO" has been added to the 6.12-stable tree
In-Reply-To: <20260806135003.602726-3-sashal@kernel.org>
Message-ID: <2026080726-activate-crafty-7749@gregkh>


This is a note to let you know that I've just added the patch titled

    rxrpc: Fix the calculation and use of RTO

to the 6.12-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     rxrpc-fix-the-calculation-and-use-of-rto.patch
and it can be found in the queue-6.12 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From stable+bounces-296836-greg=kroah.com at vger.kernel.org Thu Aug  6 15:51:40 2026
From: Sasha Levin <sashal at kernel.org>
Date: Thu,  6 Aug 2026 09:50:01 -0400
Subject: rxrpc: Fix the calculation and use of RTO
To: stable at vger.kernel.org
Cc: David Howells <dhowells at redhat.com>, Simon Wilkinson <sxw at auristor.com>, Marc Dionne <marc.dionne at auristor.com>, linux-afs at lists.infradead.org, Jakub Kicinski <kuba at kernel.org>, Sasha Levin <sashal at kernel.org>
Message-ID: <20260806135003.602726-3-sashal at kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit 5c0ceba23bb47085d6c9c53bff08a29634ee4e7e ]

Make the following changes to the calculation and use of RTO:

 (1) Fix rxrpc_resend() to use the backed-off RTO value obtained by calling
     rxrpc_get_rto_backoff() rather than extracting the value itself.
     Without this, it may retransmit packets too early.

 (2) The RTO value being similar to the RTT causes a lot of extraneous
     resends because the RTT doesn't end up taking account of clearing out
     of the receive queue on the server.  Worse, responses to PING-ACKs are
     made as fast as possible and so are less than the DATA-requested-ACK
     RTT and so skew the RTT down.

     Fix this by putting a lower bound on the RTO by adding 100ms to it and
     limiting the lower end to 200ms.

Fixes: c410bf01933e ("rxrpc: Fix the excessive initial retransmission timeout")
Fixes: 37473e416234 ("rxrpc: Clean up the resend algorithm")
Signed-off-by: David Howells <dhowells at redhat.com>
Suggested-by: Simon Wilkinson <sxw at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 net/rxrpc/call_event.c |    3 ++-
 net/rxrpc/rtt.c        |    2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

--- a/net/rxrpc/call_event.c
+++ b/net/rxrpc/call_event.c
@@ -71,7 +71,8 @@ void rxrpc_resend(struct rxrpc_call *cal
 	struct rxrpc_skb_priv *sp;
 	struct rxrpc_txbuf *txb;
 	rxrpc_seq_t transmitted = call->tx_transmitted;
-	ktime_t next_resend = KTIME_MAX, rto = ns_to_ktime(call->peer->rto_us * NSEC_PER_USEC);
+	ktime_t next_resend = KTIME_MAX;
+	ktime_t rto = rxrpc_get_rto_backoff(call->peer, false);
 	ktime_t resend_at = KTIME_MAX, now, delay;
 	bool unacked = false, did_send = false;
 	unsigned int i;
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -27,7 +27,7 @@ static u32 __rxrpc_set_rto(const struct
 
 static u32 rxrpc_bound_rto(u32 rto)
 {
-	return min(rto, RXRPC_RTO_MAX);
+	return clamp(200000, rto + 100000, RXRPC_RTO_MAX);
 }
 
 /*


Patches currently in stable-queue which might be from sashal at kernel.org are

queue-6.12/wifi-brcmfmac-set-f2-blocksize-to-256-for-bcm43752.patch
queue-6.12/net-dsa-mt7530-check-bus-read-errors-in-the-mdio-reg.patch
queue-6.12/um-set-parent-death-signal-for-userspace-process.patch
queue-6.12/usb-gadget-f_tcm-synchronize-delayed-set_alt-with-teardown.patch
queue-6.12/drm-xe-rtp-de-whitelist-oa-registers-for-all-hwe-s-f.patch
queue-6.12/media-uapi-rkisp-correct-name-version-enum.patch
queue-6.12/net-sxgbe-free-tx-rings-on-rx-allocation-failure.patch
queue-6.12/rtase-fix-double-free-of-multi-frag-skb-on-dma-map-f.patch
queue-6.12/usb-typec-ucsi-fix-race-condition-and-ordering-in-port-unregistration.patch
queue-6.12/drm-xe-introduce-xe_gt_dbg_printer.patch
queue-6.12/drm-fb-helper-allocate-and-release-fb_info-in-single-place.patch
queue-6.12/drm-xe-rtp-maintain-oa-whitelists-separately.patch
queue-6.12/pinctrl-qcom-sc8280xp-add-missing-wakeup-entries-for.patch
queue-6.12/forcedeth-fix-uaf-of-txrx_stats-in-nv_remove.patch
queue-6.12/alsa-hda-codecs-hdmi-disable-keep-alive-before-audio-format-change.patch
queue-6.12/hid-logitech-dj-standardise-hid_report_enum-variable.patch
queue-6.12/scsi-libsas-fix-ha-resume-deadlock-and-hisi_sas-disk.patch
queue-6.12/netfilter-nft_payload-fix-mask-build-for-partial-fie.patch
queue-6.12/bluetooth-iso-fix-timeout-vs-sync_timeout-typo-in-ch.patch
queue-6.12/um-set-parent-death-signal-for-ubd-io-thread-process.patch
queue-6.12/net-sxgbe-check-descriptor-ring-allocation-failures.patch
queue-6.12/ipvs-fix-the-checksum-validations.patch
queue-6.12/mptcp-pm-avoid-code-duplication-to-lookup-endp.patch
queue-6.12/hwmon-ina2xx-fix-various-overflow-issues.patch
queue-6.12/spi-spi-cadence-move-tx-fifo-full-busy-wait-into-fif.patch
queue-6.12/powerpc-boot-fix-simpleboot-cpu-node-lookup-check.patch
queue-6.12/ata-sata_mv-accept-1-or-2-resources-in-platform-prob.patch
queue-6.12/drm-xe-rtp-generalize-whitelist_apply_to_hwe.patch
queue-6.12/wifi-mac80211-validate-individual-twt-params-before-.patch
queue-6.12/mm-hugetlb-fix-swap-entry-corruption-when-clearing-u.patch
queue-6.12/dmaengine-idxd-fix-fdev-setup-failure-cleanup-in-idx.patch
queue-6.12/asoc-max98090-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/keys-trusted-dcp-fix-key_len-validation-and-calc_blo.patch
queue-6.12/drm-xe-oa-de-whitelist-oa-registers-on-oa-stream-ope.patch
queue-6.12/mptcp-pm-use-addr-entry-for-get_local_id.patch
queue-6.12/drm-xe-rtp-save-oa-nonpriv-registers-to-register-sav.patch
queue-6.12/bluetooth-hci_sync-remove-unnecessary-hci_conn_get-i.patch
queue-6.12/drm-mediatek-check-crtc-state-before-freeing.patch
queue-6.12/bluetooth-iso-clear-iso_data-always-when-detaching-c.patch
queue-6.12/bluetooth-iso-fix-leaking-sk-after-socket-release.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina260.patch
queue-6.12/accel-qaic-use-sizeof-trans_hdr-for-transaction-leng.patch
queue-6.12/pinctrl-amd-don-t-clear-s4-wake-bits-at-probe.patch
queue-6.12/net-bridge-mrp-fix-option-tlv-length-in-mrp_test-fra.patch
queue-6.12/kmemleak-iommu-iova-fix-transient-kmemleak-false-positive.patch
queue-6.12/gpio-pch-use-raw_spinlock_t-for-the-register-lock.patch
queue-6.12/rxrpc-generate-rtt_min.patch
queue-6.12/drm-xe-rtp-refactor-oag-mmio-trigger-register-whitel.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina234.patch
queue-6.12/drm-xe-apply-whitelist-to-engine-save-restore.patch
queue-6.12/powerpc-boot-fix-treeboot-currituck-cpu-node-lookup-.patch
queue-6.12/wifi-ath6kl-fix-use-after-free-in-aggr_reset_state.patch
queue-6.12/wifi-brcmfmac-fix-43752-sdio-fwvid-incorrectly-labelled-as-cypress-cyw.patch
queue-6.12/keys-make-keyring-key-chunk-byte-order-agree-with-ke.patch
queue-6.12/drm-xe-pt-reset-current_op-in-xe_pt_update_ops_init.patch
queue-6.12/drm-xe-rtp-toggle-deny-bit-to-de-whitelist-oa-regs.patch
queue-6.12/hwmon-adt7470-fix-fans-stuck-in-manual-mode-on-i2c-e.patch
queue-6.12/net-airoha-fix-register-index-for-tx-fwd-counter-con.patch
queue-6.12/ata-ahci_ceva-fix-error-paths-in-ceva_ahci_platform_.patch
queue-6.12/qede-sync-udp_tunnel-ports-outside-qede_lock-in-the-.patch
queue-6.12/rxrpc-fix-the-calculation-and-use-of-rto.patch
queue-6.12/tracing-mmiotrace-reset-dropped_count-in-mmio_reset_.patch
queue-6.12/netfilter-nf_conntrack_sip-widen-nat-rewrite-delta-t.patch
queue-6.12/drm-i915-hdcp-move-to-using-intel_display-in-intel_hdcp.patch
queue-6.12/usb-typec-ucsi-split-connector-lock-classes.patch
queue-6.12/ahci-introduce-ahci_ignore_port-helper.patch
queue-6.12/drm-i915-hdcp-require-monotonically-increasing-seq_num_v.patch
queue-6.12/net-udp_tunnel-fix-memory-leak-in-udp_tunnel_nic_unr.patch
queue-6.12/bluetooth-iso-validate-sockaddr_iso-first-in-iso_soc.patch
queue-6.12/net-libwx-fix-fdir-atr-queue-mismatch-for-software-v.patch
queue-6.12/riscv-mm-fix-out-of-bounds-page-table-walk-during-me.patch
queue-6.12/netfilter-nf_tables-make-nft_object-rhltable-per-tab.patch
queue-6.12/mm-huge_memory-unlock-i_mmap_rwsem-before-releasing-.patch
queue-6.12/bluetooth-l2cap-fix-uaf-in-l2cap_le_connect_rsp.patch
queue-6.12/hwmon-ina226-add-support-for-sy24655.patch
queue-6.12/scsi-zfcp-fix-memory-leak-during-adapter-release-by-.patch
queue-6.12/mptcp-pm-userspace-fix-use-after-free-in-get_local_id.patch
queue-6.12/hwmon-ina2xx-make-it-easier-to-add-more-devices.patch
queue-6.12/net-dsa-mt7530-error-out-on-failed-reads-in-mt7531-p.patch
queue-6.12/scsi-libiscsi_tcp-bound-scsi-response-data-segment-t.patch
queue-6.12/bluetooth-hci_conn-hold-conn-reference-in-abort_conn.patch
queue-6.12/ipvs-do-not-mangle-icmp-replies-for-non-first-fragme.patch
queue-6.12/drm-i915-hdcp-check-streams-bounds-before-overflow.patch
queue-6.12/phy-zynqmp-fix-clock-error-handling-in-xpsgtr_phy_in.patch
queue-6.12/asoc-max98095-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/hid-logitech-dj-fix-wrong-detection-of-bad-dj_short-.patch
queue-6.12/rds-tcp-hold-the-rcu-lock-across-ipv6_chk_addr-in-rd.patch
queue-6.12/drm-xe-rtp-ensure-locking-ref-counting-for-oa-whitel.patch
queue-6.12/um-add-os_set_pdeathsig-helper-function.patch
queue-6.12/hwmon-lm90-only-report-alarms-if-driver-is-ready.patch
queue-6.12/of-reserved_mem-add-code-to-dynamically-allocate-res.patch
queue-6.12/bpf-reset-register-bounds-before-narrowing-retval-ra.patch
queue-6.12/drm-xe-rtp-keep-track-of-non-oa-nonpriv-slots.patch
queue-6.12/net-ethernet-mtk_eth_soc-support-named-irqs.patch
queue-6.12/wifi-brcmfmac-drain-bus_reset-work-on-device-removal.patch
queue-6.12/hwmon-adt7470-fix-divide-by-zero-toctou-crash-in-fan.patch
queue-6.12/mptcp-add-mptcp_userspace_pm_lookup_addr-helper.patch
queue-6.12/drm-sched-store-the-drm-client_id-in-drm_sched_fence.patch
queue-6.12/of-reserved_mem-prevent-oob-when-too-many-dynamic-re.patch
queue-6.12/hwmon-adt7470-fix-temperature-alarm-logic-in-hwmon_t.patch
queue-6.12/ksmbd-return-success-for-deferred-final-close.patch
queue-6.12/hwmon-ina2xx-add-support-for-has_alerts-configuratio.patch
queue-6.12/drm-tegra-fbdev-remove-offset-into-framebuffer-memory.patch
queue-6.12/scsi-ufs-core-cancel-rtc-work-in-active-active-suspe.patch
queue-6.12/drm-exec-remove-the-index-parameter-from-drm_exec_for_each_locked_obj.patch
queue-6.12/hwmon-adt7470-fix-cache-updated-before-hardware-writ.patch
queue-6.12/assoc_array-trim-the-final-shortcut-word-using-the-c.patch
queue-6.12/mm-kmemleak-fix-checksum-computation-for-per-cpu-objects.patch
queue-6.12/ksmbd-fix-use-after-free-in-__close_file_table_ids.patch
queue-6.12/drm-xe-wait-on-external-bo-kernel-fences-in-exec-ioctl.patch
queue-6.12/drm-amdgpu-gfx-fix-cleaner-shader-ib-buffer-overflow.patch
queue-6.12/netfilter-xt_hashlimit-validate-hashtable-supports-x.patch
queue-6.12/scsi-target-iblock-fix-wrong-pr-ops-null-check-for-p.patch
queue-6.12/net-ethernet-mtk_eth_soc-add-consts-for-irq-index.patch
queue-6.12/hwmon-adt7470-fix-swapped-pwm3-and-pwm4-auto-mode-ma.patch
queue-6.12/media-chips-media-wave5-support-cbp-profile.patch
queue-6.12/um-use-os_set_pdeathsig-helper-in-winch-thread-proce.patch
queue-6.12/idpf-fix-mailbox-irq-name-leak-on-request-failure.patch
queue-6.12/drm-i915-vrr-check-has_vrr-first-in-intel_vrr_is_capable.patch
queue-6.12/btrfs-fix-leaking-btrfs_fs_state_remounting-flag.patch
queue-6.12/netfilter-br_netfilter-reallocate-headroom-if-necess.patch
queue-6.12/idpf-adjust-txq-ring-count-minimum.patch
queue-6.12/pinctrl-qcom-unconditionally-mark-gpio-as-wakeup-ena.patch
queue-6.12/gpio-sloppy-logic-analyzer-fix-memory-leak-in-gpio_l.patch
queue-6.12/keys-fix-out-of-bounds-read-in-keyring_get_key_chunk.patch
queue-6.12/drm-amdgpu-fix-context-pstate-override-handling.patch
queue-6.12/hwmon-sht3x-fix-unaligned-accesses.patch
queue-6.12/mm-slab-prevent-unbounded-recursion-in-free-path-wit.patch
queue-6.12/fs-proc-task_mmu-fix-pagemap_scan-written-state-for-.patch
queue-6.12/hwmon-adt7470-use-cached-pwm-frequency-value.patch
queue-6.12/hwmon-pmbus-fix-return-value-from-pmbus_update_byte_.patch
queue-6.12/phy-zynqmp-fix-runtime-pm-leak-on-probe-allocation-f.patch
queue-6.12/scsi-libiscsi-fix-stale-data-leak-into-the-scsi-sens.patch
queue-6.12/tracing-mmiotrace-add-null-check-for-mmio_trace_arra.patch
queue-6.12/drm-xe-rtp-add-ring_force_to_nonpriv_deny-to-oa-whit.patch
queue-6.12/spi-spi-cadence-supports-transmission-with-bits_per_.patch
queue-6.12/kunit-tool-skip-stty-when-stdin-is-not-a-tty.patch
queue-6.12/um-preserve-errno-within-signal-handler.patch
queue-6.12/ipvs-fix-places-with-wrong-packet-offsets.patch
queue-6.12/hwmon-adt7470-fix-pwm-auto-temp-state-array-and-boun.patch
queue-6.12/hwmon-ina2xx-shift-ina234-shunt-and-current-register.patch
queue-6.12/drm-amdgpu-give-each-kernel-job-a-unique-id.patch
queue-6.12/netconsole-avoid-oob-reads-msg-is-not-nul-terminated.patch
queue-6.12/bluetooth-hci_sync-make-hci_cmd_sync_run_once-return.patch
queue-6.12/hwmon-adt7470-fix-busy-loop-and-i2c-flooding-in-upda.patch
queue-6.12/bluetooth-iso-avoid-deadlocks-in-iso_sock_timeout.patch
queue-6.12/rxrpc-fix-irq-disabled-in-local_bh_enable.patch
queue-6.12/drm-i915-hdcp-skip-inactive-mst-connectors-when-building-stream-list.patch
queue-6.12/hwmon-nct6775-core-fix-number-of-temperature-registe.patch
queue-6.12/hwmon-nct6775-core-prevent-access-to-unsupported-wei.patch
queue-6.12/dmaengine-sun6i-dma-fix-reclaim-descriptors-while-te.patch
queue-6.12/drm-xe-hold-a-dma-buf-reference-for-imported-bos.patch
queue-6.12/sched-deadline-use-revised-wakeup-rule-only-for-runn.patch
queue-6.12/rds-fix-inet6_addr_lst-null-dereference-when-ipv6-is.patch
queue-6.12/rxrpc-adjust-the-rxrpc_rtt_rx-tracepoint.patch
queue-6.12/media-i2c-imx219-rename-vts-to-frm_length.patch
queue-6.12/smb-client-fix-buffer-leaks-in-smb1-read-and-write.patch
queue-6.12/scsi-target-clear-cmd_cnt-when-initial-counter-enrol.patch
queue-6.12/lib-alloc_tag-introduce-mem_alloc_profiling_permanen.patch
queue-6.12/rxrpc-manage-rtt-per-call-rather-than-per-peer.patch
queue-6.12/bluetooth-hci_sync-fix-hci_conn_del-use-in-hci_le_cr.patch
queue-6.12/net-phylink-put-link_gpio-if-phylink_create-fails.patch
queue-6.12/btrfs-zoned-fix-deadlock-between-metadata-writeback-.patch
queue-6.12/media-imx219-fix-maximum-frame-length-in-lines.patch
queue-6.12/tracing-remove-trace_event_fl_filtered-logic.patch
queue-6.12/drm-i915-vrr-require-valid-min-max-vfreq-for-vrr.patch
queue-6.12/um-set-parent-death-signal-for-write_sigio-thread-pr.patch
queue-6.12/hwmon-nzxt-smart2-dma-align-output-buffer.patch
queue-6.12/rhashtable-clear-stale-iter-p-on-table-restart.patch
queue-6.12/drm-amdgpu-respect-placement-requirements-in-amdgpu_gtt_mgr-functions.patch
queue-6.12/powerpc-boot-fix-treeboot-akebono-cpu-node-lookup-ch.patch
queue-6.12/hwmon-ltc4282-fix-reading-the-minimum-alarm-voltage.patch
queue-6.12/thunderbolt-prevent-xdomain-delayed-work-use-after-f.patch
queue-6.12/ata-libahci_platform-support-non-consecutive-port-nu.patch
queue-6.12/octeontx2-pf-set-correct-sequence-for-carrier-off-an.patch
queue-6.12/net-mpls-initialize-rtm_tos-in-mpls_getroute.patch
queue-6.12/um-set-parent-death-signal-for-winch-thread-process.patch
queue-6.12/netfilter-nf_conntrack_expect-restore-helper-propaga.patch
queue-6.12/drm-xe-stub-out-new-pagefault-layer.patch
queue-6.12/bluetooth-btintel-validate-length-before-parsing-dia.patch
queue-6.12/net-ethernet-mtk_eth_soc-pass-eth-to-mtk_handle_irq_.patch
queue-6.12/drivers-hv-vmbus-replace-lockdep_hardirq_threaded-wi.patch
queue-6.12/drm-xe-rename-___xe_bo_create_locked.patch
queue-6.12/hid-logitech-dj-prevent-report_id_dj_short-related-u.patch
queue-6.12/net-do-not-send-icmp-ndisc-redirects-when-peer-alloc.patch
queue-6.12/phy-zynqmp-postpone-getting-clock-rate-until-actuall.patch
queue-6.12/kunit-tool-terminate-kernel-under-test-on-sigint.patch
queue-6.12/can-isotp-check-register_netdevice_notifier-error-in.patch


From gregkh at linuxfoundation.org  Fri Aug  7 06:07:26 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Fri, 07 Aug 2026 15:07:26 +0200
Subject: Patch "rxrpc: Generate rtt_min" has been added to the 6.12-stable tree
In-Reply-To: <20260806135003.602726-1-sashal@kernel.org>
Message-ID: <2026080726-aerobics-starlit-7862@gregkh>


This is a note to let you know that I've just added the patch titled

    rxrpc: Generate rtt_min

to the 6.12-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     rxrpc-generate-rtt_min.patch
and it can be found in the queue-6.12 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From stable+bounces-296834-greg=kroah.com at vger.kernel.org Thu Aug  6 15:52:10 2026
From: Sasha Levin <sashal at kernel.org>
Date: Thu,  6 Aug 2026 09:49:59 -0400
Subject: rxrpc: Generate rtt_min
To: stable at vger.kernel.org
Cc: David Howells <dhowells at redhat.com>, Marc Dionne <marc.dionne at auristor.com>, linux-afs at lists.infradead.org, Jakub Kicinski <kuba at kernel.org>, Sasha Levin <sashal at kernel.org>
Message-ID: <20260806135003.602726-1-sashal at kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit c637bd066841de6d0a204898a62f1d9bb8fa1b7f ]

Generate rtt_min as this is required by RACK-TLP.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20241204074710.990092-27-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 lib/win_minmax.c        |    1 +
 net/rxrpc/ar-internal.h |    2 ++
 net/rxrpc/rtt.c         |   20 ++++++++++++++++----
 3 files changed, 19 insertions(+), 4 deletions(-)

--- a/lib/win_minmax.c
+++ b/lib/win_minmax.c
@@ -97,3 +97,4 @@ u32 minmax_running_min(struct minmax *m,
 
 	return minmax_subwin_update(m, win, &val);
 }
+EXPORT_SYMBOL(minmax_running_min);
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -349,6 +349,8 @@ struct rxrpc_peer {
 	spinlock_t		rtt_input_lock;	/* RTT lock for input routine */
 	ktime_t			rtt_last_req;	/* Time of last RTT request */
 	unsigned int		rtt_count;	/* Number of samples we've got */
+	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
+	struct minmax		min_rtt;	/* Estimated minimum RTT */
 
 	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
 	u32			mdev_us;	/* medium deviation			*/
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -127,16 +127,27 @@ static void rxrpc_set_rto(struct rxrpc_p
 	peer->rto_us = rxrpc_bound_rto(rto);
 }
 
-static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, long rtt_us)
+static void rxrpc_update_rtt_min(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+{
+	/* Window size 5mins in approx usec (ipv4.sysctl_tcp_min_rtt_wlen) */
+	u32 wlen_us = 5ULL * NSEC_PER_SEC / 1024;
+
+	minmax_running_min(&peer->min_rtt, wlen_us, resp_time / 1024,
+			   (u32)rtt_us ? : jiffies_to_usecs(1));
+}
+
+static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
 {
 	if (rtt_us < 0)
 		return;
 
-	//rxrpc_update_rtt_min(peer, rtt_us);
+	/* Update RACK min RTT [RFC8985 6.1 Step 1]. */
+	rxrpc_update_rtt_min(peer, resp_time, rtt_us);
+
 	rxrpc_rtt_estimator(peer, rtt_us);
 	rxrpc_set_rto(peer);
 
-	/* RFC6298: only reset backoff on valid RTT measurement. */
+	/* Only reset backoff on valid RTT measurement [RFC6298]. */
 	peer->backoff = 0;
 }
 
@@ -157,9 +168,10 @@ void rxrpc_peer_add_rtt(struct rxrpc_cal
 		return;
 
 	spin_lock(&peer->rtt_input_lock);
-	rxrpc_ack_update_rtt(peer, rtt_us);
+	rxrpc_ack_update_rtt(peer, resp_time, rtt_us);
 	if (peer->rtt_count < 3)
 		peer->rtt_count++;
+	peer->rtt_taken++;
 	spin_unlock(&peer->rtt_input_lock);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,


Patches currently in stable-queue which might be from sashal at kernel.org are

queue-6.12/wifi-brcmfmac-set-f2-blocksize-to-256-for-bcm43752.patch
queue-6.12/net-dsa-mt7530-check-bus-read-errors-in-the-mdio-reg.patch
queue-6.12/um-set-parent-death-signal-for-userspace-process.patch
queue-6.12/usb-gadget-f_tcm-synchronize-delayed-set_alt-with-teardown.patch
queue-6.12/drm-xe-rtp-de-whitelist-oa-registers-for-all-hwe-s-f.patch
queue-6.12/media-uapi-rkisp-correct-name-version-enum.patch
queue-6.12/net-sxgbe-free-tx-rings-on-rx-allocation-failure.patch
queue-6.12/rtase-fix-double-free-of-multi-frag-skb-on-dma-map-f.patch
queue-6.12/usb-typec-ucsi-fix-race-condition-and-ordering-in-port-unregistration.patch
queue-6.12/drm-xe-introduce-xe_gt_dbg_printer.patch
queue-6.12/drm-fb-helper-allocate-and-release-fb_info-in-single-place.patch
queue-6.12/drm-xe-rtp-maintain-oa-whitelists-separately.patch
queue-6.12/pinctrl-qcom-sc8280xp-add-missing-wakeup-entries-for.patch
queue-6.12/forcedeth-fix-uaf-of-txrx_stats-in-nv_remove.patch
queue-6.12/alsa-hda-codecs-hdmi-disable-keep-alive-before-audio-format-change.patch
queue-6.12/hid-logitech-dj-standardise-hid_report_enum-variable.patch
queue-6.12/scsi-libsas-fix-ha-resume-deadlock-and-hisi_sas-disk.patch
queue-6.12/netfilter-nft_payload-fix-mask-build-for-partial-fie.patch
queue-6.12/bluetooth-iso-fix-timeout-vs-sync_timeout-typo-in-ch.patch
queue-6.12/um-set-parent-death-signal-for-ubd-io-thread-process.patch
queue-6.12/net-sxgbe-check-descriptor-ring-allocation-failures.patch
queue-6.12/ipvs-fix-the-checksum-validations.patch
queue-6.12/mptcp-pm-avoid-code-duplication-to-lookup-endp.patch
queue-6.12/hwmon-ina2xx-fix-various-overflow-issues.patch
queue-6.12/spi-spi-cadence-move-tx-fifo-full-busy-wait-into-fif.patch
queue-6.12/powerpc-boot-fix-simpleboot-cpu-node-lookup-check.patch
queue-6.12/ata-sata_mv-accept-1-or-2-resources-in-platform-prob.patch
queue-6.12/drm-xe-rtp-generalize-whitelist_apply_to_hwe.patch
queue-6.12/wifi-mac80211-validate-individual-twt-params-before-.patch
queue-6.12/mm-hugetlb-fix-swap-entry-corruption-when-clearing-u.patch
queue-6.12/dmaengine-idxd-fix-fdev-setup-failure-cleanup-in-idx.patch
queue-6.12/asoc-max98090-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/keys-trusted-dcp-fix-key_len-validation-and-calc_blo.patch
queue-6.12/drm-xe-oa-de-whitelist-oa-registers-on-oa-stream-ope.patch
queue-6.12/mptcp-pm-use-addr-entry-for-get_local_id.patch
queue-6.12/drm-xe-rtp-save-oa-nonpriv-registers-to-register-sav.patch
queue-6.12/bluetooth-hci_sync-remove-unnecessary-hci_conn_get-i.patch
queue-6.12/drm-mediatek-check-crtc-state-before-freeing.patch
queue-6.12/bluetooth-iso-clear-iso_data-always-when-detaching-c.patch
queue-6.12/bluetooth-iso-fix-leaking-sk-after-socket-release.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina260.patch
queue-6.12/accel-qaic-use-sizeof-trans_hdr-for-transaction-leng.patch
queue-6.12/pinctrl-amd-don-t-clear-s4-wake-bits-at-probe.patch
queue-6.12/net-bridge-mrp-fix-option-tlv-length-in-mrp_test-fra.patch
queue-6.12/kmemleak-iommu-iova-fix-transient-kmemleak-false-positive.patch
queue-6.12/gpio-pch-use-raw_spinlock_t-for-the-register-lock.patch
queue-6.12/rxrpc-generate-rtt_min.patch
queue-6.12/drm-xe-rtp-refactor-oag-mmio-trigger-register-whitel.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina234.patch
queue-6.12/drm-xe-apply-whitelist-to-engine-save-restore.patch
queue-6.12/powerpc-boot-fix-treeboot-currituck-cpu-node-lookup-.patch
queue-6.12/wifi-ath6kl-fix-use-after-free-in-aggr_reset_state.patch
queue-6.12/wifi-brcmfmac-fix-43752-sdio-fwvid-incorrectly-labelled-as-cypress-cyw.patch
queue-6.12/keys-make-keyring-key-chunk-byte-order-agree-with-ke.patch
queue-6.12/drm-xe-pt-reset-current_op-in-xe_pt_update_ops_init.patch
queue-6.12/drm-xe-rtp-toggle-deny-bit-to-de-whitelist-oa-regs.patch
queue-6.12/hwmon-adt7470-fix-fans-stuck-in-manual-mode-on-i2c-e.patch
queue-6.12/net-airoha-fix-register-index-for-tx-fwd-counter-con.patch
queue-6.12/ata-ahci_ceva-fix-error-paths-in-ceva_ahci_platform_.patch
queue-6.12/qede-sync-udp_tunnel-ports-outside-qede_lock-in-the-.patch
queue-6.12/rxrpc-fix-the-calculation-and-use-of-rto.patch
queue-6.12/tracing-mmiotrace-reset-dropped_count-in-mmio_reset_.patch
queue-6.12/netfilter-nf_conntrack_sip-widen-nat-rewrite-delta-t.patch
queue-6.12/drm-i915-hdcp-move-to-using-intel_display-in-intel_hdcp.patch
queue-6.12/usb-typec-ucsi-split-connector-lock-classes.patch
queue-6.12/ahci-introduce-ahci_ignore_port-helper.patch
queue-6.12/drm-i915-hdcp-require-monotonically-increasing-seq_num_v.patch
queue-6.12/net-udp_tunnel-fix-memory-leak-in-udp_tunnel_nic_unr.patch
queue-6.12/bluetooth-iso-validate-sockaddr_iso-first-in-iso_soc.patch
queue-6.12/net-libwx-fix-fdir-atr-queue-mismatch-for-software-v.patch
queue-6.12/riscv-mm-fix-out-of-bounds-page-table-walk-during-me.patch
queue-6.12/netfilter-nf_tables-make-nft_object-rhltable-per-tab.patch
queue-6.12/mm-huge_memory-unlock-i_mmap_rwsem-before-releasing-.patch
queue-6.12/bluetooth-l2cap-fix-uaf-in-l2cap_le_connect_rsp.patch
queue-6.12/hwmon-ina226-add-support-for-sy24655.patch
queue-6.12/scsi-zfcp-fix-memory-leak-during-adapter-release-by-.patch
queue-6.12/mptcp-pm-userspace-fix-use-after-free-in-get_local_id.patch
queue-6.12/hwmon-ina2xx-make-it-easier-to-add-more-devices.patch
queue-6.12/net-dsa-mt7530-error-out-on-failed-reads-in-mt7531-p.patch
queue-6.12/scsi-libiscsi_tcp-bound-scsi-response-data-segment-t.patch
queue-6.12/bluetooth-hci_conn-hold-conn-reference-in-abort_conn.patch
queue-6.12/ipvs-do-not-mangle-icmp-replies-for-non-first-fragme.patch
queue-6.12/drm-i915-hdcp-check-streams-bounds-before-overflow.patch
queue-6.12/phy-zynqmp-fix-clock-error-handling-in-xpsgtr_phy_in.patch
queue-6.12/asoc-max98095-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/hid-logitech-dj-fix-wrong-detection-of-bad-dj_short-.patch
queue-6.12/rds-tcp-hold-the-rcu-lock-across-ipv6_chk_addr-in-rd.patch
queue-6.12/drm-xe-rtp-ensure-locking-ref-counting-for-oa-whitel.patch
queue-6.12/um-add-os_set_pdeathsig-helper-function.patch
queue-6.12/hwmon-lm90-only-report-alarms-if-driver-is-ready.patch
queue-6.12/of-reserved_mem-add-code-to-dynamically-allocate-res.patch
queue-6.12/bpf-reset-register-bounds-before-narrowing-retval-ra.patch
queue-6.12/drm-xe-rtp-keep-track-of-non-oa-nonpriv-slots.patch
queue-6.12/net-ethernet-mtk_eth_soc-support-named-irqs.patch
queue-6.12/wifi-brcmfmac-drain-bus_reset-work-on-device-removal.patch
queue-6.12/hwmon-adt7470-fix-divide-by-zero-toctou-crash-in-fan.patch
queue-6.12/mptcp-add-mptcp_userspace_pm_lookup_addr-helper.patch
queue-6.12/drm-sched-store-the-drm-client_id-in-drm_sched_fence.patch
queue-6.12/of-reserved_mem-prevent-oob-when-too-many-dynamic-re.patch
queue-6.12/hwmon-adt7470-fix-temperature-alarm-logic-in-hwmon_t.patch
queue-6.12/ksmbd-return-success-for-deferred-final-close.patch
queue-6.12/hwmon-ina2xx-add-support-for-has_alerts-configuratio.patch
queue-6.12/drm-tegra-fbdev-remove-offset-into-framebuffer-memory.patch
queue-6.12/scsi-ufs-core-cancel-rtc-work-in-active-active-suspe.patch
queue-6.12/drm-exec-remove-the-index-parameter-from-drm_exec_for_each_locked_obj.patch
queue-6.12/hwmon-adt7470-fix-cache-updated-before-hardware-writ.patch
queue-6.12/assoc_array-trim-the-final-shortcut-word-using-the-c.patch
queue-6.12/mm-kmemleak-fix-checksum-computation-for-per-cpu-objects.patch
queue-6.12/ksmbd-fix-use-after-free-in-__close_file_table_ids.patch
queue-6.12/drm-xe-wait-on-external-bo-kernel-fences-in-exec-ioctl.patch
queue-6.12/drm-amdgpu-gfx-fix-cleaner-shader-ib-buffer-overflow.patch
queue-6.12/netfilter-xt_hashlimit-validate-hashtable-supports-x.patch
queue-6.12/scsi-target-iblock-fix-wrong-pr-ops-null-check-for-p.patch
queue-6.12/net-ethernet-mtk_eth_soc-add-consts-for-irq-index.patch
queue-6.12/hwmon-adt7470-fix-swapped-pwm3-and-pwm4-auto-mode-ma.patch
queue-6.12/media-chips-media-wave5-support-cbp-profile.patch
queue-6.12/um-use-os_set_pdeathsig-helper-in-winch-thread-proce.patch
queue-6.12/idpf-fix-mailbox-irq-name-leak-on-request-failure.patch
queue-6.12/drm-i915-vrr-check-has_vrr-first-in-intel_vrr_is_capable.patch
queue-6.12/btrfs-fix-leaking-btrfs_fs_state_remounting-flag.patch
queue-6.12/netfilter-br_netfilter-reallocate-headroom-if-necess.patch
queue-6.12/idpf-adjust-txq-ring-count-minimum.patch
queue-6.12/pinctrl-qcom-unconditionally-mark-gpio-as-wakeup-ena.patch
queue-6.12/gpio-sloppy-logic-analyzer-fix-memory-leak-in-gpio_l.patch
queue-6.12/keys-fix-out-of-bounds-read-in-keyring_get_key_chunk.patch
queue-6.12/drm-amdgpu-fix-context-pstate-override-handling.patch
queue-6.12/hwmon-sht3x-fix-unaligned-accesses.patch
queue-6.12/mm-slab-prevent-unbounded-recursion-in-free-path-wit.patch
queue-6.12/fs-proc-task_mmu-fix-pagemap_scan-written-state-for-.patch
queue-6.12/hwmon-adt7470-use-cached-pwm-frequency-value.patch
queue-6.12/hwmon-pmbus-fix-return-value-from-pmbus_update_byte_.patch
queue-6.12/phy-zynqmp-fix-runtime-pm-leak-on-probe-allocation-f.patch
queue-6.12/scsi-libiscsi-fix-stale-data-leak-into-the-scsi-sens.patch
queue-6.12/tracing-mmiotrace-add-null-check-for-mmio_trace_arra.patch
queue-6.12/drm-xe-rtp-add-ring_force_to_nonpriv_deny-to-oa-whit.patch
queue-6.12/spi-spi-cadence-supports-transmission-with-bits_per_.patch
queue-6.12/kunit-tool-skip-stty-when-stdin-is-not-a-tty.patch
queue-6.12/um-preserve-errno-within-signal-handler.patch
queue-6.12/ipvs-fix-places-with-wrong-packet-offsets.patch
queue-6.12/hwmon-adt7470-fix-pwm-auto-temp-state-array-and-boun.patch
queue-6.12/hwmon-ina2xx-shift-ina234-shunt-and-current-register.patch
queue-6.12/drm-amdgpu-give-each-kernel-job-a-unique-id.patch
queue-6.12/netconsole-avoid-oob-reads-msg-is-not-nul-terminated.patch
queue-6.12/bluetooth-hci_sync-make-hci_cmd_sync_run_once-return.patch
queue-6.12/hwmon-adt7470-fix-busy-loop-and-i2c-flooding-in-upda.patch
queue-6.12/bluetooth-iso-avoid-deadlocks-in-iso_sock_timeout.patch
queue-6.12/rxrpc-fix-irq-disabled-in-local_bh_enable.patch
queue-6.12/drm-i915-hdcp-skip-inactive-mst-connectors-when-building-stream-list.patch
queue-6.12/hwmon-nct6775-core-fix-number-of-temperature-registe.patch
queue-6.12/hwmon-nct6775-core-prevent-access-to-unsupported-wei.patch
queue-6.12/dmaengine-sun6i-dma-fix-reclaim-descriptors-while-te.patch
queue-6.12/drm-xe-hold-a-dma-buf-reference-for-imported-bos.patch
queue-6.12/sched-deadline-use-revised-wakeup-rule-only-for-runn.patch
queue-6.12/rds-fix-inet6_addr_lst-null-dereference-when-ipv6-is.patch
queue-6.12/rxrpc-adjust-the-rxrpc_rtt_rx-tracepoint.patch
queue-6.12/media-i2c-imx219-rename-vts-to-frm_length.patch
queue-6.12/smb-client-fix-buffer-leaks-in-smb1-read-and-write.patch
queue-6.12/scsi-target-clear-cmd_cnt-when-initial-counter-enrol.patch
queue-6.12/lib-alloc_tag-introduce-mem_alloc_profiling_permanen.patch
queue-6.12/rxrpc-manage-rtt-per-call-rather-than-per-peer.patch
queue-6.12/bluetooth-hci_sync-fix-hci_conn_del-use-in-hci_le_cr.patch
queue-6.12/net-phylink-put-link_gpio-if-phylink_create-fails.patch
queue-6.12/btrfs-zoned-fix-deadlock-between-metadata-writeback-.patch
queue-6.12/media-imx219-fix-maximum-frame-length-in-lines.patch
queue-6.12/tracing-remove-trace_event_fl_filtered-logic.patch
queue-6.12/drm-i915-vrr-require-valid-min-max-vfreq-for-vrr.patch
queue-6.12/um-set-parent-death-signal-for-write_sigio-thread-pr.patch
queue-6.12/hwmon-nzxt-smart2-dma-align-output-buffer.patch
queue-6.12/rhashtable-clear-stale-iter-p-on-table-restart.patch
queue-6.12/drm-amdgpu-respect-placement-requirements-in-amdgpu_gtt_mgr-functions.patch
queue-6.12/powerpc-boot-fix-treeboot-akebono-cpu-node-lookup-ch.patch
queue-6.12/hwmon-ltc4282-fix-reading-the-minimum-alarm-voltage.patch
queue-6.12/thunderbolt-prevent-xdomain-delayed-work-use-after-f.patch
queue-6.12/ata-libahci_platform-support-non-consecutive-port-nu.patch
queue-6.12/octeontx2-pf-set-correct-sequence-for-carrier-off-an.patch
queue-6.12/net-mpls-initialize-rtm_tos-in-mpls_getroute.patch
queue-6.12/um-set-parent-death-signal-for-winch-thread-process.patch
queue-6.12/netfilter-nf_conntrack_expect-restore-helper-propaga.patch
queue-6.12/drm-xe-stub-out-new-pagefault-layer.patch
queue-6.12/bluetooth-btintel-validate-length-before-parsing-dia.patch
queue-6.12/net-ethernet-mtk_eth_soc-pass-eth-to-mtk_handle_irq_.patch
queue-6.12/drivers-hv-vmbus-replace-lockdep_hardirq_threaded-wi.patch
queue-6.12/drm-xe-rename-___xe_bo_create_locked.patch
queue-6.12/hid-logitech-dj-prevent-report_id_dj_short-related-u.patch
queue-6.12/net-do-not-send-icmp-ndisc-redirects-when-peer-alloc.patch
queue-6.12/phy-zynqmp-postpone-getting-clock-rate-until-actuall.patch
queue-6.12/kunit-tool-terminate-kernel-under-test-on-sigint.patch
queue-6.12/can-isotp-check-register_netdevice_notifier-error-in.patch


From gregkh at linuxfoundation.org  Fri Aug  7 06:07:26 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Fri, 07 Aug 2026 15:07:26 +0200
Subject: Patch "rxrpc: Manage RTT per-call rather than per-peer" has been added to the 6.12-stable tree
In-Reply-To: <20260806135003.602726-4-sashal@kernel.org>
Message-ID: <2026080726-deplete-spinach-f3a1@gregkh>


This is a note to let you know that I've just added the patch titled

    rxrpc: Manage RTT per-call rather than per-peer

to the 6.12-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     rxrpc-manage-rtt-per-call-rather-than-per-peer.patch
and it can be found in the queue-6.12 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From stable+bounces-296837-greg=kroah.com at vger.kernel.org Thu Aug  6 15:51:40 2026
From: Sasha Levin <sashal at kernel.org>
Date: Thu,  6 Aug 2026 09:50:02 -0400
Subject: rxrpc: Manage RTT per-call rather than per-peer
To: stable at vger.kernel.org
Cc: David Howells <dhowells at redhat.com>, Marc Dionne <marc.dionne at auristor.com>, linux-afs at lists.infradead.org, Jakub Kicinski <kuba at kernel.org>, Sasha Levin <sashal at kernel.org>
Message-ID: <20260806135003.602726-4-sashal at kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit b40ef2b85a7d117dd323b5910e504899e0a3e7dc ]

Manage the determination of RTT on a per-call (ie. per-RPC op) basis rather
than on a per-peer basis, averaging across all calls going to that peer.
The problem is that the RTT measurements from the initial packets on a call
may be off because the server may do some setting up (such as getting a
lock on a file) before accepting the rest of the data in the RPC and,
further, the RTT may be affected by server-side file operations, for
instance if a large amount of data is being written or read.

Note: When handling the FS.StoreData-type RPCs, for example, the server
uses the userStatus field in the header of ACK packets as supplementary
flow control to aid in managing this.  AF_RXRPC does not yet support this,
but it should be added.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 include/trace/events/rxrpc.h |    2 
 net/rxrpc/ar-internal.h      |   39 +++++++++---------
 net/rxrpc/call_event.c       |   18 ++++----
 net/rxrpc/call_object.c      |    2 
 net/rxrpc/input.c            |    8 +--
 net/rxrpc/output.c           |   14 +++---
 net/rxrpc/peer_object.c      |    8 ---
 net/rxrpc/proc.c             |    6 +-
 net/rxrpc/rtt.c              |   93 +++++++++++++++++++++----------------------
 net/rxrpc/sendmsg.c          |    2 
 10 files changed, 97 insertions(+), 95 deletions(-)

--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -1333,7 +1333,7 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __entry->rtt = rtt;
 		    __entry->srtt = srtt;
 		    __entry->rto = rto;
-		    __entry->min_rtt = minmax_get(&call->peer->min_rtt)
+		    __entry->min_rtt = minmax_get(&call->min_rtt)
 			   ),
 
 	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u srtt=%u rto=%u min=%u",
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -344,20 +344,9 @@ struct rxrpc_peer {
 	int			debug_id;	/* debug ID for printks */
 	struct sockaddr_rxrpc	srx;		/* remote address */
 
-	/* calculated RTT cache */
-#define RXRPC_RTT_CACHE_SIZE 32
-	spinlock_t		rtt_input_lock;	/* RTT lock for input routine */
-	ktime_t			rtt_last_req;	/* Time of last RTT request */
-	unsigned int		rtt_count;	/* Number of samples we've got */
-	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
-	struct minmax		min_rtt;	/* Estimated minimum RTT */
-
-	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
-	u32			mdev_us;	/* medium deviation			*/
-	u32			mdev_max_us;	/* maximal mdev for the last rtt period	*/
-	u32			rttvar_us;	/* smoothed mdev_max			*/
-	u32			rto_us;		/* Retransmission timeout in usec */
-	u8			backoff;	/* Backoff timeout (as shift) */
+	/* Calculated RTT cache */
+	unsigned int		recent_srtt_us;
+	unsigned int		recent_rto_us;
 
 	u8			cong_ssthresh;	/* Congestion slow-start threshold */
 };
@@ -742,6 +731,18 @@ struct rxrpc_call {
 	rxrpc_seq_t		acks_hard_ack;	/* Latest hard-ack point */
 	rxrpc_seq_t		acks_lowest_nak; /* Lowest NACK in the buffer (or ==tx_hard_ack) */
 	rxrpc_serial_t		acks_highest_serial; /* Highest serial number ACK'd */
+
+	/* Calculated RTT cache */
+	ktime_t			rtt_last_req;	/* Time of last RTT request */
+	unsigned int		rtt_count;	/* Number of samples we've got */
+	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
+	struct minmax		min_rtt;	/* Estimated minimum RTT */
+	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
+	u32			mdev_us;	/* medium deviation			*/
+	u32			mdev_max_us;	/* maximal mdev for the last rtt period	*/
+	u32			rttvar_us;	/* smoothed mdev_max			*/
+	u32			rto_us;		/* Retransmission timeout in usec */
+	u8			backoff;	/* Backoff timeout (as shift) */
 };
 
 /*
@@ -1222,10 +1223,12 @@ static inline int rxrpc_abort_eproto(str
 /*
  * rtt.c
  */
-void rxrpc_peer_add_rtt(struct rxrpc_call *, enum rxrpc_rtt_rx_trace, int,
-			rxrpc_serial_t, rxrpc_serial_t, ktime_t, ktime_t);
-ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans);
-void rxrpc_peer_init_rtt(struct rxrpc_peer *);
+void rxrpc_call_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
+			int rtt_slot,
+			rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
+			ktime_t send_time, ktime_t resp_time);
+ktime_t rxrpc_get_rto_backoff(struct rxrpc_call *call, bool retrans);
+void rxrpc_call_init_rtt(struct rxrpc_call *call);
 
 /*
  * rxkad.c
--- a/net/rxrpc/call_event.c
+++ b/net/rxrpc/call_event.c
@@ -44,8 +44,8 @@ void rxrpc_propose_delay_ACK(struct rxrp
 
 	trace_rxrpc_propose_ack(call, why, RXRPC_ACK_DELAY, serial);
 
-	if (call->peer->srtt_us)
-		delay = (call->peer->srtt_us >> 3) * NSEC_PER_USEC;
+	if (call->srtt_us)
+		delay = (call->srtt_us >> 3) * NSEC_PER_USEC;
 	else
 		delay = ms_to_ktime(READ_ONCE(rxrpc_soft_ack_delay));
 	ktime_add_ms(delay, call->tx_backoff);
@@ -72,7 +72,7 @@ void rxrpc_resend(struct rxrpc_call *cal
 	struct rxrpc_txbuf *txb;
 	rxrpc_seq_t transmitted = call->tx_transmitted;
 	ktime_t next_resend = KTIME_MAX;
-	ktime_t rto = rxrpc_get_rto_backoff(call->peer, false);
+	ktime_t rto = rxrpc_get_rto_backoff(call, false);
 	ktime_t resend_at = KTIME_MAX, now, delay;
 	bool unacked = false, did_send = false;
 	unsigned int i;
@@ -174,7 +174,7 @@ void rxrpc_resend(struct rxrpc_call *cal
 no_further_resend:
 no_resend:
 	if (resend_at < KTIME_MAX) {
-		delay = rxrpc_get_rto_backoff(call->peer, did_send);
+		delay = rxrpc_get_rto_backoff(call, did_send);
 		resend_at = ktime_add(resend_at, delay);
 		trace_rxrpc_timer_set(call, resend_at - now, rxrpc_timer_trace_resend_reset);
 	}
@@ -189,7 +189,7 @@ no_resend:
 	 */
 	if (!did_send) {
 		ktime_t next_ping = ktime_add_us(call->acks_latest_ts,
-						 call->peer->srtt_us >> 3);
+						 call->srtt_us >> 3);
 
 		if (ktime_sub(next_ping, now) <= 0)
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
@@ -306,8 +306,8 @@ static void rxrpc_transmit_some_data(str
  */
 static void rxrpc_send_initial_ping(struct rxrpc_call *call)
 {
-	if (call->peer->rtt_count < 3 ||
-	    ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000),
+	if (call->rtt_count < 3 ||
+	    ktime_before(ktime_add_ms(call->rtt_last_req, 1000),
 			 ktime_get_real()))
 		rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 			       rxrpc_propose_ack_ping_for_params);
@@ -438,10 +438,10 @@ bool rxrpc_input_call_event(struct rxrpc
 			       rxrpc_propose_ack_rx_idle);
 
 	if (call->ackr_nr_unacked > 2) {
-		if (call->peer->rtt_count < 3)
+		if (call->rtt_count < 3)
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 				       rxrpc_propose_ack_ping_for_rtt);
-		else if (ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000),
+		else if (ktime_before(ktime_add_ms(call->rtt_last_req, 1000),
 				      ktime_get_real()))
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 				       rxrpc_propose_ack_ping_for_old_rtt);
--- a/net/rxrpc/call_object.c
+++ b/net/rxrpc/call_object.c
@@ -178,6 +178,8 @@ struct rxrpc_call *rxrpc_alloc_call(stru
 	call->cong_cwnd = RXRPC_MIN_CWND;
 	call->cong_ssthresh = RXRPC_TX_MAX_WINDOW;
 
+	rxrpc_call_init_rtt(call);
+
 	call->rxnet = rxnet;
 	call->rtt_avail = RXRPC_CALL_RTT_AVAIL_MASK;
 	atomic_inc(&rxnet->nr_calls);
--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -83,11 +83,11 @@ static void rxrpc_congestion_management(
 		/* We analyse the number of packets that get ACK'd per RTT
 		 * period and increase the window if we managed to fill it.
 		 */
-		if (call->peer->rtt_count == 0)
+		if (call->rtt_count == 0)
 			goto out;
 		if (ktime_before(skb->tstamp,
 				 ktime_add_us(call->cong_tstamp,
-					      call->peer->srtt_us >> 3)))
+					      call->srtt_us >> 3)))
 			goto out_no_clear_ca;
 		change = rxrpc_cong_rtt_window_end;
 		call->cong_tstamp = skb->tstamp;
@@ -197,7 +197,7 @@ void rxrpc_congestion_degrade(struct rxr
 	if (__rxrpc_call_state(call) == RXRPC_CALL_CLIENT_AWAIT_REPLY)
 		return;
 
-	rtt = ns_to_ktime(call->peer->srtt_us * (1000 / 8));
+	rtt = ns_to_ktime(call->srtt_us * (NSEC_PER_USEC / 8));
 	now = ktime_get_real();
 	if (!ktime_before(ktime_add(call->tx_last_sent, rtt), now))
 		return;
@@ -664,7 +664,7 @@ static void rxrpc_complete_rtt_probe(str
 			clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail);
 			smp_mb(); /* Read data before setting avail bit */
 			set_bit(i, &call->rtt_avail);
-			rxrpc_peer_add_rtt(call, type, i, acked_serial, ack_serial,
+			rxrpc_call_add_rtt(call, type, i, acked_serial, ack_serial,
 					   sent_at, resp_time);
 			matched = true;
 		}
--- a/net/rxrpc/output.c
+++ b/net/rxrpc/output.c
@@ -220,7 +220,7 @@ static void rxrpc_send_ack_packet(struct
 		if (ack->reason == RXRPC_ACK_PING)
 			rxrpc_begin_rtt_probe(call, txb->serial, now, rxrpc_rtt_tx_ping);
 		if (txb->flags & RXRPC_REQUEST_ACK)
-			call->peer->rtt_last_req = now;
+			call->rtt_last_req = now;
 		rxrpc_set_keepalive(call, now);
 	}
 	rxrpc_tx_backoff(call, ret);
@@ -358,9 +358,9 @@ static void rxrpc_prepare_data_subpacket
 		why = rxrpc_reqack_slow_start;
 	else if (call->tx_winsize <= 2)
 		why = rxrpc_reqack_small_txwin;
-	else if (call->peer->rtt_count < 3 && txb->seq & 1)
+	else if (call->rtt_count < 3)
 		why = rxrpc_reqack_more_rtt;
-	else if (ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000), ktime_get_real()))
+	else if (ktime_before(ktime_add_ms(call->rtt_last_req, 1000), ktime_get_real()))
 		why = rxrpc_reqack_old_rtt;
 	else
 		goto dont_set_request_ack;
@@ -407,9 +407,9 @@ static void rxrpc_tstamp_data_packets(st
 	if (ack_requested) {
 		rxrpc_begin_rtt_probe(call, txb->serial, now, rxrpc_rtt_tx_data);
 
-		call->peer->rtt_last_req = now;
-		if (call->peer->rtt_count > 1) {
-			ktime_t delay = rxrpc_get_rto_backoff(call->peer, false);
+		call->rtt_last_req = now;
+		if (call->rtt_count > 1) {
+			ktime_t delay = rxrpc_get_rto_backoff(call, false);
 
 			call->ack_lost_at = ktime_add(now, delay);
 			trace_rxrpc_timer_set(call, delay, rxrpc_timer_trace_lost_ack);
@@ -727,7 +727,7 @@ void rxrpc_transmit_one(struct rxrpc_cal
 			rxrpc_instant_resend(call, txb);
 		}
 	} else {
-		ktime_t delay = ns_to_ktime(call->peer->rto_us * NSEC_PER_USEC);
+		ktime_t delay = ns_to_ktime(call->rto_us * NSEC_PER_USEC);
 
 		call->resend_at = ktime_add(ktime_get_real(), delay);
 		trace_rxrpc_timer_set(call, delay, rxrpc_timer_trace_resend_tx);
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -222,11 +222,8 @@ struct rxrpc_peer *rxrpc_alloc_peer(stru
 		peer->service_conns = RB_ROOT;
 		seqlock_init(&peer->service_conn_lock);
 		spin_lock_init(&peer->lock);
-		spin_lock_init(&peer->rtt_input_lock);
 		peer->debug_id = atomic_inc_return(&rxrpc_debug_id);
-
-		rxrpc_peer_init_rtt(peer);
-
+		peer->recent_srtt_us = UINT_MAX;
 		peer->cong_ssthresh = RXRPC_TX_MAX_WINDOW;
 		trace_rxrpc_peer(peer->debug_id, 1, why);
 	}
@@ -244,7 +241,6 @@ static void rxrpc_init_peer(struct rxrpc
 	peer->hash_key = hash_key;
 	rxrpc_assess_MTU_size(local, peer);
 	peer->mtu = peer->if_mtu;
-	peer->rtt_last_req = ktime_get_real();
 
 	switch (peer->srx.transport.family) {
 	case AF_INET:
@@ -480,7 +476,7 @@ EXPORT_SYMBOL(rxrpc_kernel_get_call_peer
  */
 unsigned int rxrpc_kernel_get_srtt(const struct rxrpc_peer *peer)
 {
-	return peer->rtt_count > 0 ? peer->srtt_us >> 3 : UINT_MAX;
+	return READ_ONCE(peer->recent_srtt_us);
 }
 EXPORT_SYMBOL(rxrpc_kernel_get_srtt);
 
--- a/net/rxrpc/proc.c
+++ b/net/rxrpc/proc.c
@@ -299,15 +299,15 @@ static int rxrpc_peer_seq_show(struct se
 	now = ktime_get_seconds();
 	seq_printf(seq,
 		   "UDP   %-47.47s %-47.47s %3u"
-		   " %3u %5u %6ds %8u %8u\n",
+		   " %3u %5u %6ds %8d %8d\n",
 		   lbuff,
 		   rbuff,
 		   refcount_read(&peer->ref),
 		   peer->cong_ssthresh,
 		   peer->mtu,
 		   (s32)now - (s32)READ_ONCE(peer->last_tx_at),
-		   peer->srtt_us >> 3,
-		   peer->rto_us);
+		   READ_ONCE(peer->recent_srtt_us),
+		   READ_ONCE(peer->recent_rto_us));
 
 	return 0;
 }
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -12,17 +12,17 @@
 #include "ar-internal.h"
 
 #define RXRPC_RTO_MAX	(120 * USEC_PER_SEC)
-#define RXRPC_TIMEOUT_INIT ((unsigned int)(1 * MSEC_PER_SEC)) /* RFC6298 2.1 initial RTO value */
+#define RXRPC_TIMEOUT_INIT ((unsigned int)(1 * USEC_PER_SEC)) /* RFC6298 2.1 initial RTO value */
 #define rxrpc_jiffies32 ((u32)jiffies)		/* As rxrpc_jiffies32 */
 
-static u32 rxrpc_rto_min_us(struct rxrpc_peer *peer)
+static u32 rxrpc_rto_min_us(struct rxrpc_call *call)
 {
 	return 200;
 }
 
-static u32 __rxrpc_set_rto(const struct rxrpc_peer *peer)
+static u32 __rxrpc_set_rto(const struct rxrpc_call *call)
 {
-	return (peer->srtt_us >> 3) + peer->rttvar_us;
+	return (call->srtt_us >> 3) + call->rttvar_us;
 }
 
 static u32 rxrpc_bound_rto(u32 rto)
@@ -40,10 +40,10 @@ static u32 rxrpc_bound_rto(u32 rto)
  * To save cycles in the RFC 1323 implementation it was better to break
  * it up into three procedures. -- erics
  */
-static void rxrpc_rtt_estimator(struct rxrpc_peer *peer, long sample_rtt_us)
+static void rxrpc_rtt_estimator(struct rxrpc_call *call, long sample_rtt_us)
 {
 	long m = sample_rtt_us; /* RTT */
-	u32 srtt = peer->srtt_us;
+	u32 srtt = call->srtt_us;
 
 	/*	The following amusing code comes from Jacobson's
 	 *	article in SIGCOMM '88.  Note that rtt and mdev
@@ -66,7 +66,7 @@ static void rxrpc_rtt_estimator(struct r
 		srtt += m;		/* rtt = 7/8 rtt + 1/8 new */
 		if (m < 0) {
 			m = -m;		/* m is now abs(error) */
-			m -= (peer->mdev_us >> 2);   /* similar update on mdev */
+			m -= (call->mdev_us >> 2);   /* similar update on mdev */
 			/* This is similar to one of Eifel findings.
 			 * Eifel blocks mdev updates when rtt decreases.
 			 * This solution is a bit different: we use finer gain
@@ -78,31 +78,31 @@ static void rxrpc_rtt_estimator(struct r
 			if (m > 0)
 				m >>= 3;
 		} else {
-			m -= (peer->mdev_us >> 2);   /* similar update on mdev */
+			m -= (call->mdev_us >> 2);   /* similar update on mdev */
 		}
 
-		peer->mdev_us += m;		/* mdev = 3/4 mdev + 1/4 new */
-		if (peer->mdev_us > peer->mdev_max_us) {
-			peer->mdev_max_us = peer->mdev_us;
-			if (peer->mdev_max_us > peer->rttvar_us)
-				peer->rttvar_us = peer->mdev_max_us;
+		call->mdev_us += m;		/* mdev = 3/4 mdev + 1/4 new */
+		if (call->mdev_us > call->mdev_max_us) {
+			call->mdev_max_us = call->mdev_us;
+			if (call->mdev_max_us > call->rttvar_us)
+				call->rttvar_us = call->mdev_max_us;
 		}
 	} else {
 		/* no previous measure. */
 		srtt = m << 3;		/* take the measured time to be rtt */
-		peer->mdev_us = m << 1;	/* make sure rto = 3*rtt */
-		peer->rttvar_us = max(peer->mdev_us, rxrpc_rto_min_us(peer));
-		peer->mdev_max_us = peer->rttvar_us;
+		call->mdev_us = m << 1;	/* make sure rto = 3*rtt */
+		call->rttvar_us = umax(call->mdev_us, rxrpc_rto_min_us(call));
+		call->mdev_max_us = call->rttvar_us;
 	}
 
-	peer->srtt_us = max(1U, srtt);
+	call->srtt_us = umax(srtt, 1);
 }
 
 /*
  * Calculate rto without backoff.  This is the second half of Van Jacobson's
  * routine referred to above.
  */
-static void rxrpc_set_rto(struct rxrpc_peer *peer)
+static void rxrpc_set_rto(struct rxrpc_call *call)
 {
 	u32 rto;
 
@@ -113,7 +113,7 @@ static void rxrpc_set_rto(struct rxrpc_p
 	 *    is invisible. Actually, Linux-2.4 also generates erratic
 	 *    ACKs in some circumstances.
 	 */
-	rto = __rxrpc_set_rto(peer);
+	rto = __rxrpc_set_rto(call);
 
 	/* 2. Fixups made earlier cannot be right.
 	 *    If we do not estimate RTO correctly without them,
@@ -124,73 +124,73 @@ static void rxrpc_set_rto(struct rxrpc_p
 	/* NOTE: clamping at RXRPC_RTO_MIN is not required, current algo
 	 * guarantees that rto is higher.
 	 */
-	peer->rto_us = rxrpc_bound_rto(rto);
+	call->rto_us = rxrpc_bound_rto(rto);
 }
 
-static void rxrpc_update_rtt_min(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+static void rxrpc_update_rtt_min(struct rxrpc_call *call, ktime_t resp_time, long rtt_us)
 {
 	/* Window size 5mins in approx usec (ipv4.sysctl_tcp_min_rtt_wlen) */
 	u32 wlen_us = 5ULL * NSEC_PER_SEC / 1024;
 
-	minmax_running_min(&peer->min_rtt, wlen_us, resp_time / 1024,
+	minmax_running_min(&call->min_rtt, wlen_us, resp_time / 1024,
 			   (u32)rtt_us ? : jiffies_to_usecs(1));
 }
 
-static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+static void rxrpc_ack_update_rtt(struct rxrpc_call *call, ktime_t resp_time, long rtt_us)
 {
 	if (rtt_us < 0)
 		return;
 
 	/* Update RACK min RTT [RFC8985 6.1 Step 1]. */
-	rxrpc_update_rtt_min(peer, resp_time, rtt_us);
+	rxrpc_update_rtt_min(call, resp_time, rtt_us);
 
-	rxrpc_rtt_estimator(peer, rtt_us);
-	rxrpc_set_rto(peer);
+	rxrpc_rtt_estimator(call, rtt_us);
+	rxrpc_set_rto(call);
 
 	/* Only reset backoff on valid RTT measurement [RFC6298]. */
-	peer->backoff = 0;
+	call->backoff = 0;
 }
 
 /*
  * Add RTT information to cache.  This is called in softirq mode and has
- * exclusive access to the peer RTT data.
+ * exclusive access to the call RTT data.
  */
-void rxrpc_peer_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
+void rxrpc_call_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
 			int rtt_slot,
 			rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
 			ktime_t send_time, ktime_t resp_time)
 {
-	struct rxrpc_peer *peer = call->peer;
 	s64 rtt_us;
 
 	rtt_us = ktime_to_us(ktime_sub(resp_time, send_time));
 	if (rtt_us < 0)
 		return;
 
-	spin_lock(&peer->rtt_input_lock);
-	rxrpc_ack_update_rtt(peer, resp_time, rtt_us);
-	if (peer->rtt_count < 3)
-		peer->rtt_count++;
-	peer->rtt_taken++;
-	spin_unlock(&peer->rtt_input_lock);
+	rxrpc_ack_update_rtt(call, resp_time, rtt_us);
+	if (call->rtt_count < 3)
+		call->rtt_count++;
+	call->rtt_taken++;
+
+	WRITE_ONCE(call->peer->recent_srtt_us, call->srtt_us / 8);
+	WRITE_ONCE(call->peer->recent_rto_us, call->rto_us);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,
-			   rtt_us, peer->srtt_us, peer->rto_us);
+			   rtt_us, call->srtt_us, call->rto_us);
 }
 
 /*
  * Get the retransmission timeout to set in nanoseconds, backing it off each
  * time we retransmit.
  */
-ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans)
+ktime_t rxrpc_get_rto_backoff(struct rxrpc_call *call, bool retrans)
 {
 	u64 timo_us;
-	u32 backoff = READ_ONCE(peer->backoff);
+	u32 backoff = READ_ONCE(call->backoff);
 
-	timo_us = peer->rto_us;
+	timo_us = call->rto_us;
 	timo_us <<= backoff;
 	if (retrans && timo_us * 2 <= RXRPC_RTO_MAX)
-		WRITE_ONCE(peer->backoff, backoff + 1);
+		WRITE_ONCE(call->backoff, backoff + 1);
 
 	if (timo_us < 1)
 		timo_us = 1;
@@ -198,10 +198,11 @@ ktime_t rxrpc_get_rto_backoff(struct rxr
 	return ns_to_ktime(timo_us * NSEC_PER_USEC);
 }
 
-void rxrpc_peer_init_rtt(struct rxrpc_peer *peer)
+void rxrpc_call_init_rtt(struct rxrpc_call *call)
 {
-	peer->rto_us	= RXRPC_TIMEOUT_INIT;
-	peer->mdev_us	= RXRPC_TIMEOUT_INIT;
-	peer->backoff	= 0;
-	//minmax_reset(&peer->rtt_min, rxrpc_jiffies32, ~0U);
+	call->rtt_last_req = KTIME_MIN;
+	call->rto_us	= RXRPC_TIMEOUT_INIT;
+	call->mdev_us	= RXRPC_TIMEOUT_INIT;
+	call->backoff	= 0;
+	//minmax_reset(&call->rtt_min, rxrpc_jiffies32, ~0U);
 }
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -133,7 +133,7 @@ static int rxrpc_wait_for_tx_window_wait
 	rxrpc_seq_t tx_start, tx_win;
 	signed long rtt, timeout;
 
-	rtt = READ_ONCE(call->peer->srtt_us) >> 3;
+	rtt = READ_ONCE(call->srtt_us) >> 3;
 	rtt = usecs_to_jiffies(rtt) * 2;
 	if (rtt < 2)
 		rtt = 2;


Patches currently in stable-queue which might be from sashal at kernel.org are

queue-6.12/wifi-brcmfmac-set-f2-blocksize-to-256-for-bcm43752.patch
queue-6.12/net-dsa-mt7530-check-bus-read-errors-in-the-mdio-reg.patch
queue-6.12/um-set-parent-death-signal-for-userspace-process.patch
queue-6.12/usb-gadget-f_tcm-synchronize-delayed-set_alt-with-teardown.patch
queue-6.12/drm-xe-rtp-de-whitelist-oa-registers-for-all-hwe-s-f.patch
queue-6.12/media-uapi-rkisp-correct-name-version-enum.patch
queue-6.12/net-sxgbe-free-tx-rings-on-rx-allocation-failure.patch
queue-6.12/rtase-fix-double-free-of-multi-frag-skb-on-dma-map-f.patch
queue-6.12/usb-typec-ucsi-fix-race-condition-and-ordering-in-port-unregistration.patch
queue-6.12/drm-xe-introduce-xe_gt_dbg_printer.patch
queue-6.12/drm-fb-helper-allocate-and-release-fb_info-in-single-place.patch
queue-6.12/drm-xe-rtp-maintain-oa-whitelists-separately.patch
queue-6.12/pinctrl-qcom-sc8280xp-add-missing-wakeup-entries-for.patch
queue-6.12/forcedeth-fix-uaf-of-txrx_stats-in-nv_remove.patch
queue-6.12/alsa-hda-codecs-hdmi-disable-keep-alive-before-audio-format-change.patch
queue-6.12/hid-logitech-dj-standardise-hid_report_enum-variable.patch
queue-6.12/scsi-libsas-fix-ha-resume-deadlock-and-hisi_sas-disk.patch
queue-6.12/netfilter-nft_payload-fix-mask-build-for-partial-fie.patch
queue-6.12/bluetooth-iso-fix-timeout-vs-sync_timeout-typo-in-ch.patch
queue-6.12/um-set-parent-death-signal-for-ubd-io-thread-process.patch
queue-6.12/net-sxgbe-check-descriptor-ring-allocation-failures.patch
queue-6.12/ipvs-fix-the-checksum-validations.patch
queue-6.12/mptcp-pm-avoid-code-duplication-to-lookup-endp.patch
queue-6.12/hwmon-ina2xx-fix-various-overflow-issues.patch
queue-6.12/spi-spi-cadence-move-tx-fifo-full-busy-wait-into-fif.patch
queue-6.12/powerpc-boot-fix-simpleboot-cpu-node-lookup-check.patch
queue-6.12/ata-sata_mv-accept-1-or-2-resources-in-platform-prob.patch
queue-6.12/drm-xe-rtp-generalize-whitelist_apply_to_hwe.patch
queue-6.12/wifi-mac80211-validate-individual-twt-params-before-.patch
queue-6.12/mm-hugetlb-fix-swap-entry-corruption-when-clearing-u.patch
queue-6.12/dmaengine-idxd-fix-fdev-setup-failure-cleanup-in-idx.patch
queue-6.12/asoc-max98090-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/keys-trusted-dcp-fix-key_len-validation-and-calc_blo.patch
queue-6.12/drm-xe-oa-de-whitelist-oa-registers-on-oa-stream-ope.patch
queue-6.12/mptcp-pm-use-addr-entry-for-get_local_id.patch
queue-6.12/drm-xe-rtp-save-oa-nonpriv-registers-to-register-sav.patch
queue-6.12/bluetooth-hci_sync-remove-unnecessary-hci_conn_get-i.patch
queue-6.12/drm-mediatek-check-crtc-state-before-freeing.patch
queue-6.12/bluetooth-iso-clear-iso_data-always-when-detaching-c.patch
queue-6.12/bluetooth-iso-fix-leaking-sk-after-socket-release.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina260.patch
queue-6.12/accel-qaic-use-sizeof-trans_hdr-for-transaction-leng.patch
queue-6.12/pinctrl-amd-don-t-clear-s4-wake-bits-at-probe.patch
queue-6.12/net-bridge-mrp-fix-option-tlv-length-in-mrp_test-fra.patch
queue-6.12/kmemleak-iommu-iova-fix-transient-kmemleak-false-positive.patch
queue-6.12/gpio-pch-use-raw_spinlock_t-for-the-register-lock.patch
queue-6.12/rxrpc-generate-rtt_min.patch
queue-6.12/drm-xe-rtp-refactor-oag-mmio-trigger-register-whitel.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina234.patch
queue-6.12/drm-xe-apply-whitelist-to-engine-save-restore.patch
queue-6.12/powerpc-boot-fix-treeboot-currituck-cpu-node-lookup-.patch
queue-6.12/wifi-ath6kl-fix-use-after-free-in-aggr_reset_state.patch
queue-6.12/wifi-brcmfmac-fix-43752-sdio-fwvid-incorrectly-labelled-as-cypress-cyw.patch
queue-6.12/keys-make-keyring-key-chunk-byte-order-agree-with-ke.patch
queue-6.12/drm-xe-pt-reset-current_op-in-xe_pt_update_ops_init.patch
queue-6.12/drm-xe-rtp-toggle-deny-bit-to-de-whitelist-oa-regs.patch
queue-6.12/hwmon-adt7470-fix-fans-stuck-in-manual-mode-on-i2c-e.patch
queue-6.12/net-airoha-fix-register-index-for-tx-fwd-counter-con.patch
queue-6.12/ata-ahci_ceva-fix-error-paths-in-ceva_ahci_platform_.patch
queue-6.12/qede-sync-udp_tunnel-ports-outside-qede_lock-in-the-.patch
queue-6.12/rxrpc-fix-the-calculation-and-use-of-rto.patch
queue-6.12/tracing-mmiotrace-reset-dropped_count-in-mmio_reset_.patch
queue-6.12/netfilter-nf_conntrack_sip-widen-nat-rewrite-delta-t.patch
queue-6.12/drm-i915-hdcp-move-to-using-intel_display-in-intel_hdcp.patch
queue-6.12/usb-typec-ucsi-split-connector-lock-classes.patch
queue-6.12/ahci-introduce-ahci_ignore_port-helper.patch
queue-6.12/drm-i915-hdcp-require-monotonically-increasing-seq_num_v.patch
queue-6.12/net-udp_tunnel-fix-memory-leak-in-udp_tunnel_nic_unr.patch
queue-6.12/bluetooth-iso-validate-sockaddr_iso-first-in-iso_soc.patch
queue-6.12/net-libwx-fix-fdir-atr-queue-mismatch-for-software-v.patch
queue-6.12/riscv-mm-fix-out-of-bounds-page-table-walk-during-me.patch
queue-6.12/netfilter-nf_tables-make-nft_object-rhltable-per-tab.patch
queue-6.12/mm-huge_memory-unlock-i_mmap_rwsem-before-releasing-.patch
queue-6.12/bluetooth-l2cap-fix-uaf-in-l2cap_le_connect_rsp.patch
queue-6.12/hwmon-ina226-add-support-for-sy24655.patch
queue-6.12/scsi-zfcp-fix-memory-leak-during-adapter-release-by-.patch
queue-6.12/mptcp-pm-userspace-fix-use-after-free-in-get_local_id.patch
queue-6.12/hwmon-ina2xx-make-it-easier-to-add-more-devices.patch
queue-6.12/net-dsa-mt7530-error-out-on-failed-reads-in-mt7531-p.patch
queue-6.12/scsi-libiscsi_tcp-bound-scsi-response-data-segment-t.patch
queue-6.12/bluetooth-hci_conn-hold-conn-reference-in-abort_conn.patch
queue-6.12/ipvs-do-not-mangle-icmp-replies-for-non-first-fragme.patch
queue-6.12/drm-i915-hdcp-check-streams-bounds-before-overflow.patch
queue-6.12/phy-zynqmp-fix-clock-error-handling-in-xpsgtr_phy_in.patch
queue-6.12/asoc-max98095-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/hid-logitech-dj-fix-wrong-detection-of-bad-dj_short-.patch
queue-6.12/rds-tcp-hold-the-rcu-lock-across-ipv6_chk_addr-in-rd.patch
queue-6.12/drm-xe-rtp-ensure-locking-ref-counting-for-oa-whitel.patch
queue-6.12/um-add-os_set_pdeathsig-helper-function.patch
queue-6.12/hwmon-lm90-only-report-alarms-if-driver-is-ready.patch
queue-6.12/of-reserved_mem-add-code-to-dynamically-allocate-res.patch
queue-6.12/bpf-reset-register-bounds-before-narrowing-retval-ra.patch
queue-6.12/drm-xe-rtp-keep-track-of-non-oa-nonpriv-slots.patch
queue-6.12/net-ethernet-mtk_eth_soc-support-named-irqs.patch
queue-6.12/wifi-brcmfmac-drain-bus_reset-work-on-device-removal.patch
queue-6.12/hwmon-adt7470-fix-divide-by-zero-toctou-crash-in-fan.patch
queue-6.12/mptcp-add-mptcp_userspace_pm_lookup_addr-helper.patch
queue-6.12/drm-sched-store-the-drm-client_id-in-drm_sched_fence.patch
queue-6.12/of-reserved_mem-prevent-oob-when-too-many-dynamic-re.patch
queue-6.12/hwmon-adt7470-fix-temperature-alarm-logic-in-hwmon_t.patch
queue-6.12/ksmbd-return-success-for-deferred-final-close.patch
queue-6.12/hwmon-ina2xx-add-support-for-has_alerts-configuratio.patch
queue-6.12/drm-tegra-fbdev-remove-offset-into-framebuffer-memory.patch
queue-6.12/scsi-ufs-core-cancel-rtc-work-in-active-active-suspe.patch
queue-6.12/drm-exec-remove-the-index-parameter-from-drm_exec_for_each_locked_obj.patch
queue-6.12/hwmon-adt7470-fix-cache-updated-before-hardware-writ.patch
queue-6.12/assoc_array-trim-the-final-shortcut-word-using-the-c.patch
queue-6.12/mm-kmemleak-fix-checksum-computation-for-per-cpu-objects.patch
queue-6.12/ksmbd-fix-use-after-free-in-__close_file_table_ids.patch
queue-6.12/drm-xe-wait-on-external-bo-kernel-fences-in-exec-ioctl.patch
queue-6.12/drm-amdgpu-gfx-fix-cleaner-shader-ib-buffer-overflow.patch
queue-6.12/netfilter-xt_hashlimit-validate-hashtable-supports-x.patch
queue-6.12/scsi-target-iblock-fix-wrong-pr-ops-null-check-for-p.patch
queue-6.12/net-ethernet-mtk_eth_soc-add-consts-for-irq-index.patch
queue-6.12/hwmon-adt7470-fix-swapped-pwm3-and-pwm4-auto-mode-ma.patch
queue-6.12/media-chips-media-wave5-support-cbp-profile.patch
queue-6.12/um-use-os_set_pdeathsig-helper-in-winch-thread-proce.patch
queue-6.12/idpf-fix-mailbox-irq-name-leak-on-request-failure.patch
queue-6.12/drm-i915-vrr-check-has_vrr-first-in-intel_vrr_is_capable.patch
queue-6.12/btrfs-fix-leaking-btrfs_fs_state_remounting-flag.patch
queue-6.12/netfilter-br_netfilter-reallocate-headroom-if-necess.patch
queue-6.12/idpf-adjust-txq-ring-count-minimum.patch
queue-6.12/pinctrl-qcom-unconditionally-mark-gpio-as-wakeup-ena.patch
queue-6.12/gpio-sloppy-logic-analyzer-fix-memory-leak-in-gpio_l.patch
queue-6.12/keys-fix-out-of-bounds-read-in-keyring_get_key_chunk.patch
queue-6.12/drm-amdgpu-fix-context-pstate-override-handling.patch
queue-6.12/hwmon-sht3x-fix-unaligned-accesses.patch
queue-6.12/mm-slab-prevent-unbounded-recursion-in-free-path-wit.patch
queue-6.12/fs-proc-task_mmu-fix-pagemap_scan-written-state-for-.patch
queue-6.12/hwmon-adt7470-use-cached-pwm-frequency-value.patch
queue-6.12/hwmon-pmbus-fix-return-value-from-pmbus_update_byte_.patch
queue-6.12/phy-zynqmp-fix-runtime-pm-leak-on-probe-allocation-f.patch
queue-6.12/scsi-libiscsi-fix-stale-data-leak-into-the-scsi-sens.patch
queue-6.12/tracing-mmiotrace-add-null-check-for-mmio_trace_arra.patch
queue-6.12/drm-xe-rtp-add-ring_force_to_nonpriv_deny-to-oa-whit.patch
queue-6.12/spi-spi-cadence-supports-transmission-with-bits_per_.patch
queue-6.12/kunit-tool-skip-stty-when-stdin-is-not-a-tty.patch
queue-6.12/um-preserve-errno-within-signal-handler.patch
queue-6.12/ipvs-fix-places-with-wrong-packet-offsets.patch
queue-6.12/hwmon-adt7470-fix-pwm-auto-temp-state-array-and-boun.patch
queue-6.12/hwmon-ina2xx-shift-ina234-shunt-and-current-register.patch
queue-6.12/drm-amdgpu-give-each-kernel-job-a-unique-id.patch
queue-6.12/netconsole-avoid-oob-reads-msg-is-not-nul-terminated.patch
queue-6.12/bluetooth-hci_sync-make-hci_cmd_sync_run_once-return.patch
queue-6.12/hwmon-adt7470-fix-busy-loop-and-i2c-flooding-in-upda.patch
queue-6.12/bluetooth-iso-avoid-deadlocks-in-iso_sock_timeout.patch
queue-6.12/rxrpc-fix-irq-disabled-in-local_bh_enable.patch
queue-6.12/drm-i915-hdcp-skip-inactive-mst-connectors-when-building-stream-list.patch
queue-6.12/hwmon-nct6775-core-fix-number-of-temperature-registe.patch
queue-6.12/hwmon-nct6775-core-prevent-access-to-unsupported-wei.patch
queue-6.12/dmaengine-sun6i-dma-fix-reclaim-descriptors-while-te.patch
queue-6.12/drm-xe-hold-a-dma-buf-reference-for-imported-bos.patch
queue-6.12/sched-deadline-use-revised-wakeup-rule-only-for-runn.patch
queue-6.12/rds-fix-inet6_addr_lst-null-dereference-when-ipv6-is.patch
queue-6.12/rxrpc-adjust-the-rxrpc_rtt_rx-tracepoint.patch
queue-6.12/media-i2c-imx219-rename-vts-to-frm_length.patch
queue-6.12/smb-client-fix-buffer-leaks-in-smb1-read-and-write.patch
queue-6.12/scsi-target-clear-cmd_cnt-when-initial-counter-enrol.patch
queue-6.12/lib-alloc_tag-introduce-mem_alloc_profiling_permanen.patch
queue-6.12/rxrpc-manage-rtt-per-call-rather-than-per-peer.patch
queue-6.12/bluetooth-hci_sync-fix-hci_conn_del-use-in-hci_le_cr.patch
queue-6.12/net-phylink-put-link_gpio-if-phylink_create-fails.patch
queue-6.12/btrfs-zoned-fix-deadlock-between-metadata-writeback-.patch
queue-6.12/media-imx219-fix-maximum-frame-length-in-lines.patch
queue-6.12/tracing-remove-trace_event_fl_filtered-logic.patch
queue-6.12/drm-i915-vrr-require-valid-min-max-vfreq-for-vrr.patch
queue-6.12/um-set-parent-death-signal-for-write_sigio-thread-pr.patch
queue-6.12/hwmon-nzxt-smart2-dma-align-output-buffer.patch
queue-6.12/rhashtable-clear-stale-iter-p-on-table-restart.patch
queue-6.12/drm-amdgpu-respect-placement-requirements-in-amdgpu_gtt_mgr-functions.patch
queue-6.12/powerpc-boot-fix-treeboot-akebono-cpu-node-lookup-ch.patch
queue-6.12/hwmon-ltc4282-fix-reading-the-minimum-alarm-voltage.patch
queue-6.12/thunderbolt-prevent-xdomain-delayed-work-use-after-f.patch
queue-6.12/ata-libahci_platform-support-non-consecutive-port-nu.patch
queue-6.12/octeontx2-pf-set-correct-sequence-for-carrier-off-an.patch
queue-6.12/net-mpls-initialize-rtm_tos-in-mpls_getroute.patch
queue-6.12/um-set-parent-death-signal-for-winch-thread-process.patch
queue-6.12/netfilter-nf_conntrack_expect-restore-helper-propaga.patch
queue-6.12/drm-xe-stub-out-new-pagefault-layer.patch
queue-6.12/bluetooth-btintel-validate-length-before-parsing-dia.patch
queue-6.12/net-ethernet-mtk_eth_soc-pass-eth-to-mtk_handle_irq_.patch
queue-6.12/drivers-hv-vmbus-replace-lockdep_hardirq_threaded-wi.patch
queue-6.12/drm-xe-rename-___xe_bo_create_locked.patch
queue-6.12/hid-logitech-dj-prevent-report_id_dj_short-related-u.patch
queue-6.12/net-do-not-send-icmp-ndisc-redirects-when-peer-alloc.patch
queue-6.12/phy-zynqmp-postpone-getting-clock-rate-until-actuall.patch
queue-6.12/kunit-tool-terminate-kernel-under-test-on-sigint.patch
queue-6.12/can-isotp-check-register_netdevice_notifier-error-in.patch


From gregkh at linuxfoundation.org  Fri Aug  7 07:38:51 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:38:51 +0200
Subject: [PATCH 6.12 328/337] rxrpc: Generate rtt_min
In-Reply-To: <20260807143418.516897842@linuxfoundation.org>
References: <20260807143418.516897842@linuxfoundation.org>
Message-ID: <20260807143425.652017439@linuxfoundation.org>

6.12-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

[ Upstream commit c637bd066841de6d0a204898a62f1d9bb8fa1b7f ]

Generate rtt_min as this is required by RACK-TLP.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20241204074710.990092-27-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 lib/win_minmax.c        |    1 +
 net/rxrpc/ar-internal.h |    2 ++
 net/rxrpc/rtt.c         |   20 ++++++++++++++++----
 3 files changed, 19 insertions(+), 4 deletions(-)

--- a/lib/win_minmax.c
+++ b/lib/win_minmax.c
@@ -97,3 +97,4 @@ u32 minmax_running_min(struct minmax *m,
 
 	return minmax_subwin_update(m, win, &val);
 }
+EXPORT_SYMBOL(minmax_running_min);
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -349,6 +349,8 @@ struct rxrpc_peer {
 	spinlock_t		rtt_input_lock;	/* RTT lock for input routine */
 	ktime_t			rtt_last_req;	/* Time of last RTT request */
 	unsigned int		rtt_count;	/* Number of samples we've got */
+	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
+	struct minmax		min_rtt;	/* Estimated minimum RTT */
 
 	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
 	u32			mdev_us;	/* medium deviation			*/
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -127,16 +127,27 @@ static void rxrpc_set_rto(struct rxrpc_p
 	peer->rto_us = rxrpc_bound_rto(rto);
 }
 
-static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, long rtt_us)
+static void rxrpc_update_rtt_min(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+{
+	/* Window size 5mins in approx usec (ipv4.sysctl_tcp_min_rtt_wlen) */
+	u32 wlen_us = 5ULL * NSEC_PER_SEC / 1024;
+
+	minmax_running_min(&peer->min_rtt, wlen_us, resp_time / 1024,
+			   (u32)rtt_us ? : jiffies_to_usecs(1));
+}
+
+static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
 {
 	if (rtt_us < 0)
 		return;
 
-	//rxrpc_update_rtt_min(peer, rtt_us);
+	/* Update RACK min RTT [RFC8985 6.1 Step 1]. */
+	rxrpc_update_rtt_min(peer, resp_time, rtt_us);
+
 	rxrpc_rtt_estimator(peer, rtt_us);
 	rxrpc_set_rto(peer);
 
-	/* RFC6298: only reset backoff on valid RTT measurement. */
+	/* Only reset backoff on valid RTT measurement [RFC6298]. */
 	peer->backoff = 0;
 }
 
@@ -157,9 +168,10 @@ void rxrpc_peer_add_rtt(struct rxrpc_cal
 		return;
 
 	spin_lock(&peer->rtt_input_lock);
-	rxrpc_ack_update_rtt(peer, rtt_us);
+	rxrpc_ack_update_rtt(peer, resp_time, rtt_us);
 	if (peer->rtt_count < 3)
 		peer->rtt_count++;
+	peer->rtt_taken++;
 	spin_unlock(&peer->rtt_input_lock);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,




From gregkh at linuxfoundation.org  Fri Aug  7 07:38:52 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:38:52 +0200
Subject: [PATCH 6.12 329/337] rxrpc: Adjust the rxrpc_rtt_rx tracepoint
In-Reply-To: <20260807143418.516897842@linuxfoundation.org>
References: <20260807143418.516897842@linuxfoundation.org>
Message-ID: <20260807143425.673911720@linuxfoundation.org>

6.12-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

[ Upstream commit 93dfca65a1df42a3c8b1094299dc42ab8f18e5c8 ]

Adjust the rxrpc_rtt_rx tracepoint in the following ways:

 (1) Display the collected RTT sample in the rxrpc_rtt_rx trace.

 (2) Move the division of srtt by 8 to the TP_printk() rather doing it
     before invoking the trace point.

 (3) Display the min_rtt value.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 include/trace/events/rxrpc.h |   14 ++++++++++----
 net/rxrpc/input.c            |    4 ++--
 net/rxrpc/rtt.c              |    2 +-
 3 files changed, 13 insertions(+), 7 deletions(-)

--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -1308,9 +1308,9 @@ TRACE_EVENT(rxrpc_rtt_rx,
 	    TP_PROTO(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
 		     int slot,
 		     rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
-		     u32 rtt, u32 rto),
+		     u32 rtt, u32 srtt, u32 rto),
 
-	    TP_ARGS(call, why, slot, send_serial, resp_serial, rtt, rto),
+	    TP_ARGS(call, why, slot, send_serial, resp_serial, rtt, srtt, rto),
 
 	    TP_STRUCT__entry(
 		    __field(unsigned int,		call)
@@ -1319,7 +1319,9 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __field(rxrpc_serial_t,		send_serial)
 		    __field(rxrpc_serial_t,		resp_serial)
 		    __field(u32,			rtt)
+		    __field(u32,			srtt)
 		    __field(u32,			rto)
+		    __field(u32,			min_rtt)
 			     ),
 
 	    TP_fast_assign(
@@ -1329,17 +1331,21 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __entry->send_serial = send_serial;
 		    __entry->resp_serial = resp_serial;
 		    __entry->rtt = rtt;
+		    __entry->srtt = srtt;
 		    __entry->rto = rto;
+		    __entry->min_rtt = minmax_get(&call->peer->min_rtt)
 			   ),
 
-	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u rto=%u",
+	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u srtt=%u rto=%u min=%u",
 		      __entry->call,
 		      __entry->slot,
 		      __print_symbolic(__entry->why, rxrpc_rtt_rx_traces),
 		      __entry->send_serial,
 		      __entry->resp_serial,
 		      __entry->rtt,
-		      __entry->rto)
+		      __entry->srtt / 8,
+		      __entry->rto,
+		      __entry->min_rtt)
 	    );
 
 TRACE_EVENT(rxrpc_timer_set,
--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -674,7 +674,7 @@ static void rxrpc_complete_rtt_probe(str
 		 */
 		if (after(acked_serial, orig_serial)) {
 			trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_obsolete, i,
-					   orig_serial, acked_serial, 0, 0);
+					   orig_serial, acked_serial, 0, 0, 0);
 			clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail);
 			smp_wmb();
 			set_bit(i, &call->rtt_avail);
@@ -682,7 +682,7 @@ static void rxrpc_complete_rtt_probe(str
 	}
 
 	if (!matched)
-		trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_lost, 9, 0, acked_serial, 0, 0);
+		trace_rxrpc_rtt_rx(call, rxrpc_rtt_rx_lost, 9, 0, acked_serial, 0, 0, 0);
 }
 
 /*
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -175,7 +175,7 @@ void rxrpc_peer_add_rtt(struct rxrpc_cal
 	spin_unlock(&peer->rtt_input_lock);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,
-			   peer->srtt_us >> 3, peer->rto_us);
+			   rtt_us, peer->srtt_us, peer->rto_us);
 }
 
 /*




From gregkh at linuxfoundation.org  Fri Aug  7 07:38:53 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:38:53 +0200
Subject: [PATCH 6.12 330/337] rxrpc: Fix the calculation and use of RTO
In-Reply-To: <20260807143418.516897842@linuxfoundation.org>
References: <20260807143418.516897842@linuxfoundation.org>
Message-ID: <20260807143425.695475885@linuxfoundation.org>

6.12-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

[ Upstream commit 5c0ceba23bb47085d6c9c53bff08a29634ee4e7e ]

Make the following changes to the calculation and use of RTO:

 (1) Fix rxrpc_resend() to use the backed-off RTO value obtained by calling
     rxrpc_get_rto_backoff() rather than extracting the value itself.
     Without this, it may retransmit packets too early.

 (2) The RTO value being similar to the RTT causes a lot of extraneous
     resends because the RTT doesn't end up taking account of clearing out
     of the receive queue on the server.  Worse, responses to PING-ACKs are
     made as fast as possible and so are less than the DATA-requested-ACK
     RTT and so skew the RTT down.

     Fix this by putting a lower bound on the RTO by adding 100ms to it and
     limiting the lower end to 200ms.

Fixes: c410bf01933e ("rxrpc: Fix the excessive initial retransmission timeout")
Fixes: 37473e416234 ("rxrpc: Clean up the resend algorithm")
Signed-off-by: David Howells <dhowells at redhat.com>
Suggested-by: Simon Wilkinson <sxw at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 net/rxrpc/call_event.c |    3 ++-
 net/rxrpc/rtt.c        |    2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

--- a/net/rxrpc/call_event.c
+++ b/net/rxrpc/call_event.c
@@ -71,7 +71,8 @@ void rxrpc_resend(struct rxrpc_call *cal
 	struct rxrpc_skb_priv *sp;
 	struct rxrpc_txbuf *txb;
 	rxrpc_seq_t transmitted = call->tx_transmitted;
-	ktime_t next_resend = KTIME_MAX, rto = ns_to_ktime(call->peer->rto_us * NSEC_PER_USEC);
+	ktime_t next_resend = KTIME_MAX;
+	ktime_t rto = rxrpc_get_rto_backoff(call->peer, false);
 	ktime_t resend_at = KTIME_MAX, now, delay;
 	bool unacked = false, did_send = false;
 	unsigned int i;
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -27,7 +27,7 @@ static u32 __rxrpc_set_rto(const struct
 
 static u32 rxrpc_bound_rto(u32 rto)
 {
-	return min(rto, RXRPC_RTO_MAX);
+	return clamp(200000, rto + 100000, RXRPC_RTO_MAX);
 }
 
 /*




From gregkh at linuxfoundation.org  Fri Aug  7 07:38:54 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:38:54 +0200
Subject: [PATCH 6.12 331/337] rxrpc: Manage RTT per-call rather than per-peer
In-Reply-To: <20260807143418.516897842@linuxfoundation.org>
References: <20260807143418.516897842@linuxfoundation.org>
Message-ID: <20260807143425.717848644@linuxfoundation.org>

6.12-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

[ Upstream commit b40ef2b85a7d117dd323b5910e504899e0a3e7dc ]

Manage the determination of RTT on a per-call (ie. per-RPC op) basis rather
than on a per-peer basis, averaging across all calls going to that peer.
The problem is that the RTT measurements from the initial packets on a call
may be off because the server may do some setting up (such as getting a
lock on a file) before accepting the rest of the data in the RPC and,
further, the RTT may be affected by server-side file operations, for
instance if a large amount of data is being written or read.

Note: When handling the FS.StoreData-type RPCs, for example, the server
uses the userStatus field in the header of ACK packets as supplementary
flow control to aid in managing this.  AF_RXRPC does not yet support this,
but it should be added.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Stable-dep-of: e4d2878369d5 ("rxrpc: Fix irq-disabled in local_bh_enable()")
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 include/trace/events/rxrpc.h |    2 
 net/rxrpc/ar-internal.h      |   39 +++++++++---------
 net/rxrpc/call_event.c       |   18 ++++----
 net/rxrpc/call_object.c      |    2 
 net/rxrpc/input.c            |    8 +--
 net/rxrpc/output.c           |   14 +++---
 net/rxrpc/peer_object.c      |    8 ---
 net/rxrpc/proc.c             |    6 +-
 net/rxrpc/rtt.c              |   93 +++++++++++++++++++++----------------------
 net/rxrpc/sendmsg.c          |    2 
 10 files changed, 97 insertions(+), 95 deletions(-)

--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -1333,7 +1333,7 @@ TRACE_EVENT(rxrpc_rtt_rx,
 		    __entry->rtt = rtt;
 		    __entry->srtt = srtt;
 		    __entry->rto = rto;
-		    __entry->min_rtt = minmax_get(&call->peer->min_rtt)
+		    __entry->min_rtt = minmax_get(&call->min_rtt)
 			   ),
 
 	    TP_printk("c=%08x [%d] %s sr=%08x rr=%08x rtt=%u srtt=%u rto=%u min=%u",
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -344,20 +344,9 @@ struct rxrpc_peer {
 	int			debug_id;	/* debug ID for printks */
 	struct sockaddr_rxrpc	srx;		/* remote address */
 
-	/* calculated RTT cache */
-#define RXRPC_RTT_CACHE_SIZE 32
-	spinlock_t		rtt_input_lock;	/* RTT lock for input routine */
-	ktime_t			rtt_last_req;	/* Time of last RTT request */
-	unsigned int		rtt_count;	/* Number of samples we've got */
-	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
-	struct minmax		min_rtt;	/* Estimated minimum RTT */
-
-	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
-	u32			mdev_us;	/* medium deviation			*/
-	u32			mdev_max_us;	/* maximal mdev for the last rtt period	*/
-	u32			rttvar_us;	/* smoothed mdev_max			*/
-	u32			rto_us;		/* Retransmission timeout in usec */
-	u8			backoff;	/* Backoff timeout (as shift) */
+	/* Calculated RTT cache */
+	unsigned int		recent_srtt_us;
+	unsigned int		recent_rto_us;
 
 	u8			cong_ssthresh;	/* Congestion slow-start threshold */
 };
@@ -742,6 +731,18 @@ struct rxrpc_call {
 	rxrpc_seq_t		acks_hard_ack;	/* Latest hard-ack point */
 	rxrpc_seq_t		acks_lowest_nak; /* Lowest NACK in the buffer (or ==tx_hard_ack) */
 	rxrpc_serial_t		acks_highest_serial; /* Highest serial number ACK'd */
+
+	/* Calculated RTT cache */
+	ktime_t			rtt_last_req;	/* Time of last RTT request */
+	unsigned int		rtt_count;	/* Number of samples we've got */
+	unsigned int		rtt_taken;	/* Number of samples taken (wrapping) */
+	struct minmax		min_rtt;	/* Estimated minimum RTT */
+	u32			srtt_us;	/* smoothed round trip time << 3 in usecs */
+	u32			mdev_us;	/* medium deviation			*/
+	u32			mdev_max_us;	/* maximal mdev for the last rtt period	*/
+	u32			rttvar_us;	/* smoothed mdev_max			*/
+	u32			rto_us;		/* Retransmission timeout in usec */
+	u8			backoff;	/* Backoff timeout (as shift) */
 };
 
 /*
@@ -1222,10 +1223,12 @@ static inline int rxrpc_abort_eproto(str
 /*
  * rtt.c
  */
-void rxrpc_peer_add_rtt(struct rxrpc_call *, enum rxrpc_rtt_rx_trace, int,
-			rxrpc_serial_t, rxrpc_serial_t, ktime_t, ktime_t);
-ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans);
-void rxrpc_peer_init_rtt(struct rxrpc_peer *);
+void rxrpc_call_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
+			int rtt_slot,
+			rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
+			ktime_t send_time, ktime_t resp_time);
+ktime_t rxrpc_get_rto_backoff(struct rxrpc_call *call, bool retrans);
+void rxrpc_call_init_rtt(struct rxrpc_call *call);
 
 /*
  * rxkad.c
--- a/net/rxrpc/call_event.c
+++ b/net/rxrpc/call_event.c
@@ -44,8 +44,8 @@ void rxrpc_propose_delay_ACK(struct rxrp
 
 	trace_rxrpc_propose_ack(call, why, RXRPC_ACK_DELAY, serial);
 
-	if (call->peer->srtt_us)
-		delay = (call->peer->srtt_us >> 3) * NSEC_PER_USEC;
+	if (call->srtt_us)
+		delay = (call->srtt_us >> 3) * NSEC_PER_USEC;
 	else
 		delay = ms_to_ktime(READ_ONCE(rxrpc_soft_ack_delay));
 	ktime_add_ms(delay, call->tx_backoff);
@@ -72,7 +72,7 @@ void rxrpc_resend(struct rxrpc_call *cal
 	struct rxrpc_txbuf *txb;
 	rxrpc_seq_t transmitted = call->tx_transmitted;
 	ktime_t next_resend = KTIME_MAX;
-	ktime_t rto = rxrpc_get_rto_backoff(call->peer, false);
+	ktime_t rto = rxrpc_get_rto_backoff(call, false);
 	ktime_t resend_at = KTIME_MAX, now, delay;
 	bool unacked = false, did_send = false;
 	unsigned int i;
@@ -174,7 +174,7 @@ void rxrpc_resend(struct rxrpc_call *cal
 no_further_resend:
 no_resend:
 	if (resend_at < KTIME_MAX) {
-		delay = rxrpc_get_rto_backoff(call->peer, did_send);
+		delay = rxrpc_get_rto_backoff(call, did_send);
 		resend_at = ktime_add(resend_at, delay);
 		trace_rxrpc_timer_set(call, resend_at - now, rxrpc_timer_trace_resend_reset);
 	}
@@ -189,7 +189,7 @@ no_resend:
 	 */
 	if (!did_send) {
 		ktime_t next_ping = ktime_add_us(call->acks_latest_ts,
-						 call->peer->srtt_us >> 3);
+						 call->srtt_us >> 3);
 
 		if (ktime_sub(next_ping, now) <= 0)
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
@@ -306,8 +306,8 @@ static void rxrpc_transmit_some_data(str
  */
 static void rxrpc_send_initial_ping(struct rxrpc_call *call)
 {
-	if (call->peer->rtt_count < 3 ||
-	    ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000),
+	if (call->rtt_count < 3 ||
+	    ktime_before(ktime_add_ms(call->rtt_last_req, 1000),
 			 ktime_get_real()))
 		rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 			       rxrpc_propose_ack_ping_for_params);
@@ -438,10 +438,10 @@ bool rxrpc_input_call_event(struct rxrpc
 			       rxrpc_propose_ack_rx_idle);
 
 	if (call->ackr_nr_unacked > 2) {
-		if (call->peer->rtt_count < 3)
+		if (call->rtt_count < 3)
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 				       rxrpc_propose_ack_ping_for_rtt);
-		else if (ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000),
+		else if (ktime_before(ktime_add_ms(call->rtt_last_req, 1000),
 				      ktime_get_real()))
 			rxrpc_send_ACK(call, RXRPC_ACK_PING, 0,
 				       rxrpc_propose_ack_ping_for_old_rtt);
--- a/net/rxrpc/call_object.c
+++ b/net/rxrpc/call_object.c
@@ -178,6 +178,8 @@ struct rxrpc_call *rxrpc_alloc_call(stru
 	call->cong_cwnd = RXRPC_MIN_CWND;
 	call->cong_ssthresh = RXRPC_TX_MAX_WINDOW;
 
+	rxrpc_call_init_rtt(call);
+
 	call->rxnet = rxnet;
 	call->rtt_avail = RXRPC_CALL_RTT_AVAIL_MASK;
 	atomic_inc(&rxnet->nr_calls);
--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -83,11 +83,11 @@ static void rxrpc_congestion_management(
 		/* We analyse the number of packets that get ACK'd per RTT
 		 * period and increase the window if we managed to fill it.
 		 */
-		if (call->peer->rtt_count == 0)
+		if (call->rtt_count == 0)
 			goto out;
 		if (ktime_before(skb->tstamp,
 				 ktime_add_us(call->cong_tstamp,
-					      call->peer->srtt_us >> 3)))
+					      call->srtt_us >> 3)))
 			goto out_no_clear_ca;
 		change = rxrpc_cong_rtt_window_end;
 		call->cong_tstamp = skb->tstamp;
@@ -197,7 +197,7 @@ void rxrpc_congestion_degrade(struct rxr
 	if (__rxrpc_call_state(call) == RXRPC_CALL_CLIENT_AWAIT_REPLY)
 		return;
 
-	rtt = ns_to_ktime(call->peer->srtt_us * (1000 / 8));
+	rtt = ns_to_ktime(call->srtt_us * (NSEC_PER_USEC / 8));
 	now = ktime_get_real();
 	if (!ktime_before(ktime_add(call->tx_last_sent, rtt), now))
 		return;
@@ -664,7 +664,7 @@ static void rxrpc_complete_rtt_probe(str
 			clear_bit(i + RXRPC_CALL_RTT_PEND_SHIFT, &call->rtt_avail);
 			smp_mb(); /* Read data before setting avail bit */
 			set_bit(i, &call->rtt_avail);
-			rxrpc_peer_add_rtt(call, type, i, acked_serial, ack_serial,
+			rxrpc_call_add_rtt(call, type, i, acked_serial, ack_serial,
 					   sent_at, resp_time);
 			matched = true;
 		}
--- a/net/rxrpc/output.c
+++ b/net/rxrpc/output.c
@@ -220,7 +220,7 @@ static void rxrpc_send_ack_packet(struct
 		if (ack->reason == RXRPC_ACK_PING)
 			rxrpc_begin_rtt_probe(call, txb->serial, now, rxrpc_rtt_tx_ping);
 		if (txb->flags & RXRPC_REQUEST_ACK)
-			call->peer->rtt_last_req = now;
+			call->rtt_last_req = now;
 		rxrpc_set_keepalive(call, now);
 	}
 	rxrpc_tx_backoff(call, ret);
@@ -358,9 +358,9 @@ static void rxrpc_prepare_data_subpacket
 		why = rxrpc_reqack_slow_start;
 	else if (call->tx_winsize <= 2)
 		why = rxrpc_reqack_small_txwin;
-	else if (call->peer->rtt_count < 3 && txb->seq & 1)
+	else if (call->rtt_count < 3)
 		why = rxrpc_reqack_more_rtt;
-	else if (ktime_before(ktime_add_ms(call->peer->rtt_last_req, 1000), ktime_get_real()))
+	else if (ktime_before(ktime_add_ms(call->rtt_last_req, 1000), ktime_get_real()))
 		why = rxrpc_reqack_old_rtt;
 	else
 		goto dont_set_request_ack;
@@ -407,9 +407,9 @@ static void rxrpc_tstamp_data_packets(st
 	if (ack_requested) {
 		rxrpc_begin_rtt_probe(call, txb->serial, now, rxrpc_rtt_tx_data);
 
-		call->peer->rtt_last_req = now;
-		if (call->peer->rtt_count > 1) {
-			ktime_t delay = rxrpc_get_rto_backoff(call->peer, false);
+		call->rtt_last_req = now;
+		if (call->rtt_count > 1) {
+			ktime_t delay = rxrpc_get_rto_backoff(call, false);
 
 			call->ack_lost_at = ktime_add(now, delay);
 			trace_rxrpc_timer_set(call, delay, rxrpc_timer_trace_lost_ack);
@@ -727,7 +727,7 @@ void rxrpc_transmit_one(struct rxrpc_cal
 			rxrpc_instant_resend(call, txb);
 		}
 	} else {
-		ktime_t delay = ns_to_ktime(call->peer->rto_us * NSEC_PER_USEC);
+		ktime_t delay = ns_to_ktime(call->rto_us * NSEC_PER_USEC);
 
 		call->resend_at = ktime_add(ktime_get_real(), delay);
 		trace_rxrpc_timer_set(call, delay, rxrpc_timer_trace_resend_tx);
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -222,11 +222,8 @@ struct rxrpc_peer *rxrpc_alloc_peer(stru
 		peer->service_conns = RB_ROOT;
 		seqlock_init(&peer->service_conn_lock);
 		spin_lock_init(&peer->lock);
-		spin_lock_init(&peer->rtt_input_lock);
 		peer->debug_id = atomic_inc_return(&rxrpc_debug_id);
-
-		rxrpc_peer_init_rtt(peer);
-
+		peer->recent_srtt_us = UINT_MAX;
 		peer->cong_ssthresh = RXRPC_TX_MAX_WINDOW;
 		trace_rxrpc_peer(peer->debug_id, 1, why);
 	}
@@ -244,7 +241,6 @@ static void rxrpc_init_peer(struct rxrpc
 	peer->hash_key = hash_key;
 	rxrpc_assess_MTU_size(local, peer);
 	peer->mtu = peer->if_mtu;
-	peer->rtt_last_req = ktime_get_real();
 
 	switch (peer->srx.transport.family) {
 	case AF_INET:
@@ -480,7 +476,7 @@ EXPORT_SYMBOL(rxrpc_kernel_get_call_peer
  */
 unsigned int rxrpc_kernel_get_srtt(const struct rxrpc_peer *peer)
 {
-	return peer->rtt_count > 0 ? peer->srtt_us >> 3 : UINT_MAX;
+	return READ_ONCE(peer->recent_srtt_us);
 }
 EXPORT_SYMBOL(rxrpc_kernel_get_srtt);
 
--- a/net/rxrpc/proc.c
+++ b/net/rxrpc/proc.c
@@ -299,15 +299,15 @@ static int rxrpc_peer_seq_show(struct se
 	now = ktime_get_seconds();
 	seq_printf(seq,
 		   "UDP   %-47.47s %-47.47s %3u"
-		   " %3u %5u %6ds %8u %8u\n",
+		   " %3u %5u %6ds %8d %8d\n",
 		   lbuff,
 		   rbuff,
 		   refcount_read(&peer->ref),
 		   peer->cong_ssthresh,
 		   peer->mtu,
 		   (s32)now - (s32)READ_ONCE(peer->last_tx_at),
-		   peer->srtt_us >> 3,
-		   peer->rto_us);
+		   READ_ONCE(peer->recent_srtt_us),
+		   READ_ONCE(peer->recent_rto_us));
 
 	return 0;
 }
--- a/net/rxrpc/rtt.c
+++ b/net/rxrpc/rtt.c
@@ -12,17 +12,17 @@
 #include "ar-internal.h"
 
 #define RXRPC_RTO_MAX	(120 * USEC_PER_SEC)
-#define RXRPC_TIMEOUT_INIT ((unsigned int)(1 * MSEC_PER_SEC)) /* RFC6298 2.1 initial RTO value */
+#define RXRPC_TIMEOUT_INIT ((unsigned int)(1 * USEC_PER_SEC)) /* RFC6298 2.1 initial RTO value */
 #define rxrpc_jiffies32 ((u32)jiffies)		/* As rxrpc_jiffies32 */
 
-static u32 rxrpc_rto_min_us(struct rxrpc_peer *peer)
+static u32 rxrpc_rto_min_us(struct rxrpc_call *call)
 {
 	return 200;
 }
 
-static u32 __rxrpc_set_rto(const struct rxrpc_peer *peer)
+static u32 __rxrpc_set_rto(const struct rxrpc_call *call)
 {
-	return (peer->srtt_us >> 3) + peer->rttvar_us;
+	return (call->srtt_us >> 3) + call->rttvar_us;
 }
 
 static u32 rxrpc_bound_rto(u32 rto)
@@ -40,10 +40,10 @@ static u32 rxrpc_bound_rto(u32 rto)
  * To save cycles in the RFC 1323 implementation it was better to break
  * it up into three procedures. -- erics
  */
-static void rxrpc_rtt_estimator(struct rxrpc_peer *peer, long sample_rtt_us)
+static void rxrpc_rtt_estimator(struct rxrpc_call *call, long sample_rtt_us)
 {
 	long m = sample_rtt_us; /* RTT */
-	u32 srtt = peer->srtt_us;
+	u32 srtt = call->srtt_us;
 
 	/*	The following amusing code comes from Jacobson's
 	 *	article in SIGCOMM '88.  Note that rtt and mdev
@@ -66,7 +66,7 @@ static void rxrpc_rtt_estimator(struct r
 		srtt += m;		/* rtt = 7/8 rtt + 1/8 new */
 		if (m < 0) {
 			m = -m;		/* m is now abs(error) */
-			m -= (peer->mdev_us >> 2);   /* similar update on mdev */
+			m -= (call->mdev_us >> 2);   /* similar update on mdev */
 			/* This is similar to one of Eifel findings.
 			 * Eifel blocks mdev updates when rtt decreases.
 			 * This solution is a bit different: we use finer gain
@@ -78,31 +78,31 @@ static void rxrpc_rtt_estimator(struct r
 			if (m > 0)
 				m >>= 3;
 		} else {
-			m -= (peer->mdev_us >> 2);   /* similar update on mdev */
+			m -= (call->mdev_us >> 2);   /* similar update on mdev */
 		}
 
-		peer->mdev_us += m;		/* mdev = 3/4 mdev + 1/4 new */
-		if (peer->mdev_us > peer->mdev_max_us) {
-			peer->mdev_max_us = peer->mdev_us;
-			if (peer->mdev_max_us > peer->rttvar_us)
-				peer->rttvar_us = peer->mdev_max_us;
+		call->mdev_us += m;		/* mdev = 3/4 mdev + 1/4 new */
+		if (call->mdev_us > call->mdev_max_us) {
+			call->mdev_max_us = call->mdev_us;
+			if (call->mdev_max_us > call->rttvar_us)
+				call->rttvar_us = call->mdev_max_us;
 		}
 	} else {
 		/* no previous measure. */
 		srtt = m << 3;		/* take the measured time to be rtt */
-		peer->mdev_us = m << 1;	/* make sure rto = 3*rtt */
-		peer->rttvar_us = max(peer->mdev_us, rxrpc_rto_min_us(peer));
-		peer->mdev_max_us = peer->rttvar_us;
+		call->mdev_us = m << 1;	/* make sure rto = 3*rtt */
+		call->rttvar_us = umax(call->mdev_us, rxrpc_rto_min_us(call));
+		call->mdev_max_us = call->rttvar_us;
 	}
 
-	peer->srtt_us = max(1U, srtt);
+	call->srtt_us = umax(srtt, 1);
 }
 
 /*
  * Calculate rto without backoff.  This is the second half of Van Jacobson's
  * routine referred to above.
  */
-static void rxrpc_set_rto(struct rxrpc_peer *peer)
+static void rxrpc_set_rto(struct rxrpc_call *call)
 {
 	u32 rto;
 
@@ -113,7 +113,7 @@ static void rxrpc_set_rto(struct rxrpc_p
 	 *    is invisible. Actually, Linux-2.4 also generates erratic
 	 *    ACKs in some circumstances.
 	 */
-	rto = __rxrpc_set_rto(peer);
+	rto = __rxrpc_set_rto(call);
 
 	/* 2. Fixups made earlier cannot be right.
 	 *    If we do not estimate RTO correctly without them,
@@ -124,73 +124,73 @@ static void rxrpc_set_rto(struct rxrpc_p
 	/* NOTE: clamping at RXRPC_RTO_MIN is not required, current algo
 	 * guarantees that rto is higher.
 	 */
-	peer->rto_us = rxrpc_bound_rto(rto);
+	call->rto_us = rxrpc_bound_rto(rto);
 }
 
-static void rxrpc_update_rtt_min(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+static void rxrpc_update_rtt_min(struct rxrpc_call *call, ktime_t resp_time, long rtt_us)
 {
 	/* Window size 5mins in approx usec (ipv4.sysctl_tcp_min_rtt_wlen) */
 	u32 wlen_us = 5ULL * NSEC_PER_SEC / 1024;
 
-	minmax_running_min(&peer->min_rtt, wlen_us, resp_time / 1024,
+	minmax_running_min(&call->min_rtt, wlen_us, resp_time / 1024,
 			   (u32)rtt_us ? : jiffies_to_usecs(1));
 }
 
-static void rxrpc_ack_update_rtt(struct rxrpc_peer *peer, ktime_t resp_time, long rtt_us)
+static void rxrpc_ack_update_rtt(struct rxrpc_call *call, ktime_t resp_time, long rtt_us)
 {
 	if (rtt_us < 0)
 		return;
 
 	/* Update RACK min RTT [RFC8985 6.1 Step 1]. */
-	rxrpc_update_rtt_min(peer, resp_time, rtt_us);
+	rxrpc_update_rtt_min(call, resp_time, rtt_us);
 
-	rxrpc_rtt_estimator(peer, rtt_us);
-	rxrpc_set_rto(peer);
+	rxrpc_rtt_estimator(call, rtt_us);
+	rxrpc_set_rto(call);
 
 	/* Only reset backoff on valid RTT measurement [RFC6298]. */
-	peer->backoff = 0;
+	call->backoff = 0;
 }
 
 /*
  * Add RTT information to cache.  This is called in softirq mode and has
- * exclusive access to the peer RTT data.
+ * exclusive access to the call RTT data.
  */
-void rxrpc_peer_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
+void rxrpc_call_add_rtt(struct rxrpc_call *call, enum rxrpc_rtt_rx_trace why,
 			int rtt_slot,
 			rxrpc_serial_t send_serial, rxrpc_serial_t resp_serial,
 			ktime_t send_time, ktime_t resp_time)
 {
-	struct rxrpc_peer *peer = call->peer;
 	s64 rtt_us;
 
 	rtt_us = ktime_to_us(ktime_sub(resp_time, send_time));
 	if (rtt_us < 0)
 		return;
 
-	spin_lock(&peer->rtt_input_lock);
-	rxrpc_ack_update_rtt(peer, resp_time, rtt_us);
-	if (peer->rtt_count < 3)
-		peer->rtt_count++;
-	peer->rtt_taken++;
-	spin_unlock(&peer->rtt_input_lock);
+	rxrpc_ack_update_rtt(call, resp_time, rtt_us);
+	if (call->rtt_count < 3)
+		call->rtt_count++;
+	call->rtt_taken++;
+
+	WRITE_ONCE(call->peer->recent_srtt_us, call->srtt_us / 8);
+	WRITE_ONCE(call->peer->recent_rto_us, call->rto_us);
 
 	trace_rxrpc_rtt_rx(call, why, rtt_slot, send_serial, resp_serial,
-			   rtt_us, peer->srtt_us, peer->rto_us);
+			   rtt_us, call->srtt_us, call->rto_us);
 }
 
 /*
  * Get the retransmission timeout to set in nanoseconds, backing it off each
  * time we retransmit.
  */
-ktime_t rxrpc_get_rto_backoff(struct rxrpc_peer *peer, bool retrans)
+ktime_t rxrpc_get_rto_backoff(struct rxrpc_call *call, bool retrans)
 {
 	u64 timo_us;
-	u32 backoff = READ_ONCE(peer->backoff);
+	u32 backoff = READ_ONCE(call->backoff);
 
-	timo_us = peer->rto_us;
+	timo_us = call->rto_us;
 	timo_us <<= backoff;
 	if (retrans && timo_us * 2 <= RXRPC_RTO_MAX)
-		WRITE_ONCE(peer->backoff, backoff + 1);
+		WRITE_ONCE(call->backoff, backoff + 1);
 
 	if (timo_us < 1)
 		timo_us = 1;
@@ -198,10 +198,11 @@ ktime_t rxrpc_get_rto_backoff(struct rxr
 	return ns_to_ktime(timo_us * NSEC_PER_USEC);
 }
 
-void rxrpc_peer_init_rtt(struct rxrpc_peer *peer)
+void rxrpc_call_init_rtt(struct rxrpc_call *call)
 {
-	peer->rto_us	= RXRPC_TIMEOUT_INIT;
-	peer->mdev_us	= RXRPC_TIMEOUT_INIT;
-	peer->backoff	= 0;
-	//minmax_reset(&peer->rtt_min, rxrpc_jiffies32, ~0U);
+	call->rtt_last_req = KTIME_MIN;
+	call->rto_us	= RXRPC_TIMEOUT_INIT;
+	call->mdev_us	= RXRPC_TIMEOUT_INIT;
+	call->backoff	= 0;
+	//minmax_reset(&call->rtt_min, rxrpc_jiffies32, ~0U);
 }
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -133,7 +133,7 @@ static int rxrpc_wait_for_tx_window_wait
 	rxrpc_seq_t tx_start, tx_win;
 	signed long rtt, timeout;
 
-	rtt = READ_ONCE(call->peer->srtt_us) >> 3;
+	rtt = READ_ONCE(call->srtt_us) >> 3;
 	rtt = usecs_to_jiffies(rtt) * 2;
 	if (rtt < 2)
 		rtt = 2;




From gregkh at linuxfoundation.org  Fri Aug  7 07:35:58 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:35:58 +0200
Subject: [PATCH 6.18 198/396] afs: Fix afs_fs_fetch_data() to set call->async
In-Reply-To: <20260807143424.272339768@linuxfoundation.org>
References: <20260807143424.272339768@linuxfoundation.org>
Message-ID: <20260807143428.552740560@linuxfoundation.org>

6.18-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

commit d568a43f6dbba3ba006304d95fd09862bd482a2f upstream.

Fix afs_fs_fetch_data() to set call->async on an async operation as does
afs_fs_fetch_data64().

Fixes: eddf51f2bb2c ("afs: Make {Y,}FS.FetchData an asynchronous operation")
Link: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-2-dhowells at redhat.com
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/fsclient.c |    3 +++
 1 file changed, 3 insertions(+)

--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -477,6 +477,9 @@ void afs_fs_fetch_data(struct afs_operat
 	if (!call)
 		return afs_op_nomem(op);
 
+	if (op->flags & AFS_OPERATION_ASYNC)
+		call->async = true;
+
 	/* marshall the parameters */
 	bp = call->request;
 	bp[0] = htonl(FSFETCHDATA);




From gregkh at linuxfoundation.org  Fri Aug  7 07:35:59 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:35:59 +0200
Subject: [PATCH 6.18 199/396] afs: Fix afs_fs_fetch_data() to subtract transferred from len
In-Reply-To: <20260807143424.272339768@linuxfoundation.org>
References: <20260807143424.272339768@linuxfoundation.org>
Message-ID: <20260807143428.575054891@linuxfoundation.org>

6.18-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

commit 222052c6be186f2074b3a4d741d5de200f654c43 upstream.

Fix afs_fs_fetch_data() to subtract subreq->transferred from subreq->len
rather than adding it.

Fixes: f28fc2010d62 ("afs: Eliminate afs_read")
Link: https://sashiko.dev/#/patchset/20260713081022.2186481-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-3-dhowells at redhat.com
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/fsclient.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 626e1d37b915..1a3f186a6a11 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -487,7 +487,7 @@ void afs_fs_fetch_data(struct afs_operation *op)
 	bp[2] = htonl(vp->fid.vnode);
 	bp[3] = htonl(vp->fid.unique);
 	bp[4] = htonl(lower_32_bits(subreq->start + subreq->transferred));
-	bp[5] = htonl(lower_32_bits(subreq->len   + subreq->transferred));
+	bp[5] = htonl(lower_32_bits(subreq->len   - subreq->transferred));
 
 	call->fid = vp->fid;
 	trace_afs_make_fs_call(call, &vp->fid);
-- 
2.55.0





From gregkh at linuxfoundation.org  Fri Aug  7 07:36:00 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:36:00 +0200
Subject: [PATCH 6.18 200/396] afs: Fix UAF when sending a message
In-Reply-To: <20260807143424.272339768@linuxfoundation.org>
References: <20260807143424.272339768@linuxfoundation.org>
Message-ID: <20260807143428.596343543@linuxfoundation.org>

6.18-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

commit 4af1ec68d54b3871155914d584fb10669c41a861 upstream.

In afs_make_call(), there's a race with async call reception and
destruction.  If a call is dispatched that doesn't have call->write_iter
set (used to specify the data content for FS.StoreData), then the first
rxrpc_kernel_send_data() will not set MSG_MORE in the msghdr.

Once rxrpc_send_data() queues the last request packet, the response could
come in at any time and cause the call to be completed and put.  However,
afs_make_call() will look at the call again to see it ->write_iter should
be handled - something it's only allowed to do if it has its own ref on the
call.  Whilst this is the case for synchronous calls, it isn't true for
async calls such as FS.FetchData.

There's also a potential UAF in afs_make_call() in the event that an
asynchronous call is being sent, but the call fails in some way (e.g. it
gets aborted from the server).  The problem there is that afs_make_call()
tries to abort a call if the rxrpc send fails, but the asynchronous
notification from rxrpc may have caused the afs_call to be torn down.

generic/650 plays games with randomly taking CPUs offline, and can
interject a significant delay such that the call is deallocated before
afs_make_call() gets to check call->write_iter - and a UAF ensues (caught
by KASAN).

   BUG: KASAN: slab-use-after-free in afs_make_call+0x1c90/0x2210 [kafs]
   Read of size 8 at addr ffff888035e050e8 by task fsstress/1409

Fix this by making afs_make_op_call() give the op->call its own ref rather
than transferring the caller's ref to it and then dropping the ref when
afs_make_call() returns.

This also means that the afs_make_call() func never loses its ref on the
call now.

Fixes: eddf51f2bb2c ("afs: Make {Y,}FS.FetchData an asynchronous operation")
Fixes: e49c7b2f6de7 ("afs: Build an abstraction around an "operation" concept")
Link: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Reported-by: Marc Dionne <marc.dionne at auristor.com>
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-4-dhowells at redhat.com
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/internal.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -1420,7 +1420,7 @@ static inline void afs_make_op_call(stru
 {
 	struct afs_addr_list *alist = op->estate->addresses;
 
-	op->call	= call;
+	op->call	= afs_get_call(call, afs_call_trace_get);
 	op->type	= call->type;
 	call->op	= op;
 	call->key	= op->key;
@@ -1428,6 +1428,7 @@ static inline void afs_make_op_call(stru
 	call->peer	= rxrpc_kernel_get_peer(alist->addrs[op->addr_index].peer);
 	call->service_id = op->server->service_id;
 	afs_make_call(call, gfp);
+	afs_put_call(call);
 }
 
 static inline void afs_extract_begin(struct afs_call *call, void *buf, size_t size)




From gregkh at linuxfoundation.org  Fri Aug  7 07:37:25 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:37:25 +0200
Subject: [PATCH 7.1 249/438] afs: Fix afs_fs_fetch_data() to set call->async
In-Reply-To: <20260807143428.008222056@linuxfoundation.org>
References: <20260807143428.008222056@linuxfoundation.org>
Message-ID: <20260807143433.307219717@linuxfoundation.org>

7.1-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

commit d568a43f6dbba3ba006304d95fd09862bd482a2f upstream.

Fix afs_fs_fetch_data() to set call->async on an async operation as does
afs_fs_fetch_data64().

Fixes: eddf51f2bb2c ("afs: Make {Y,}FS.FetchData an asynchronous operation")
Link: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-2-dhowells at redhat.com
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/fsclient.c |    3 +++
 1 file changed, 3 insertions(+)

--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -477,6 +477,9 @@ void afs_fs_fetch_data(struct afs_operat
 	if (!call)
 		return afs_op_nomem(op);
 
+	if (op->flags & AFS_OPERATION_ASYNC)
+		call->async = true;
+
 	/* marshall the parameters */
 	bp = call->request;
 	bp[0] = htonl(FSFETCHDATA);




From gregkh at linuxfoundation.org  Fri Aug  7 07:37:26 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:37:26 +0200
Subject: [PATCH 7.1 250/438] afs: Fix afs_fs_fetch_data() to subtract transferred from len
In-Reply-To: <20260807143428.008222056@linuxfoundation.org>
References: <20260807143428.008222056@linuxfoundation.org>
Message-ID: <20260807143433.327791084@linuxfoundation.org>

7.1-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

commit 222052c6be186f2074b3a4d741d5de200f654c43 upstream.

Fix afs_fs_fetch_data() to subtract subreq->transferred from subreq->len
rather than adding it.

Fixes: f28fc2010d62 ("afs: Eliminate afs_read")
Link: https://sashiko.dev/#/patchset/20260713081022.2186481-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-3-dhowells at redhat.com
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/fsclient.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -487,7 +487,7 @@ void afs_fs_fetch_data(struct afs_operat
 	bp[2] = htonl(vp->fid.vnode);
 	bp[3] = htonl(vp->fid.unique);
 	bp[4] = htonl(lower_32_bits(subreq->start + subreq->transferred));
-	bp[5] = htonl(lower_32_bits(subreq->len   + subreq->transferred));
+	bp[5] = htonl(lower_32_bits(subreq->len   - subreq->transferred));
 
 	call->fid = vp->fid;
 	trace_afs_make_fs_call(call, &vp->fid);




From gregkh at linuxfoundation.org  Fri Aug  7 07:37:27 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:37:27 +0200
Subject: [PATCH 7.1 251/438] afs: Fix UAF when sending a message
In-Reply-To: <20260807143428.008222056@linuxfoundation.org>
References: <20260807143428.008222056@linuxfoundation.org>
Message-ID: <20260807143433.349605054@linuxfoundation.org>

7.1-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

commit 4af1ec68d54b3871155914d584fb10669c41a861 upstream.

In afs_make_call(), there's a race with async call reception and
destruction.  If a call is dispatched that doesn't have call->write_iter
set (used to specify the data content for FS.StoreData), then the first
rxrpc_kernel_send_data() will not set MSG_MORE in the msghdr.

Once rxrpc_send_data() queues the last request packet, the response could
come in at any time and cause the call to be completed and put.  However,
afs_make_call() will look at the call again to see it ->write_iter should
be handled - something it's only allowed to do if it has its own ref on the
call.  Whilst this is the case for synchronous calls, it isn't true for
async calls such as FS.FetchData.

There's also a potential UAF in afs_make_call() in the event that an
asynchronous call is being sent, but the call fails in some way (e.g. it
gets aborted from the server).  The problem there is that afs_make_call()
tries to abort a call if the rxrpc send fails, but the asynchronous
notification from rxrpc may have caused the afs_call to be torn down.

generic/650 plays games with randomly taking CPUs offline, and can
interject a significant delay such that the call is deallocated before
afs_make_call() gets to check call->write_iter - and a UAF ensues (caught
by KASAN).

   BUG: KASAN: slab-use-after-free in afs_make_call+0x1c90/0x2210 [kafs]
   Read of size 8 at addr ffff888035e050e8 by task fsstress/1409

Fix this by making afs_make_op_call() give the op->call its own ref rather
than transferring the caller's ref to it and then dropping the ref when
afs_make_call() returns.

This also means that the afs_make_call() func never loses its ref on the
call now.

Fixes: eddf51f2bb2c ("afs: Make {Y,}FS.FetchData an asynchronous operation")
Fixes: e49c7b2f6de7 ("afs: Build an abstraction around an "operation" concept")
Link: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Reported-by: Marc Dionne <marc.dionne at auristor.com>
Signed-off-by: David Howells <dhowells at redhat.com>
Link: https://patch.msgid.link/20260723113452.566619-4-dhowells at redhat.com
cc: Jeffrey Altman <jaltman at auristor.com>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 fs/afs/internal.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -1421,7 +1421,7 @@ static inline void afs_make_op_call(stru
 {
 	struct afs_addr_list *alist = op->estate->addresses;
 
-	op->call	= call;
+	op->call	= afs_get_call(call, afs_call_trace_get);
 	op->type	= call->type;
 	call->op	= op;
 	call->key	= op->key;
@@ -1429,6 +1429,7 @@ static inline void afs_make_op_call(stru
 	call->peer	= rxrpc_kernel_get_peer(alist->addrs[op->addr_index].peer);
 	call->service_id = op->server->service_id;
 	afs_make_call(call, gfp);
+	afs_put_call(call);
 }
 
 static inline void afs_extract_begin(struct afs_call *call, void *buf, size_t size)




From kuba at kernel.org  Fri Aug  7 18:05:18 2026
From: kuba at kernel.org (Jakub Kicinski)
Date: Fri, 7 Aug 2026 18:05:18 -0700
Subject: [PATCH net v6 00/10] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <20260806091511.46a2fabb@kernel.org>
References: <20260804172639.2844491-1-dhowells@redhat.com>
	<20260806091511.46a2fabb@kernel.org>
Message-ID: <20260807180518.0287b58e@kernel.org>

On Thu, 6 Aug 2026 09:15:11 -0700 Jakub Kicinski wrote:
> On Tue,  4 Aug 2026 18:26:27 +0100 David Howells wrote:
> > [!] Note that this entails a significant change in the UAPI for AF_RXRPC,
> > with the CMSG types and sockopt to support the OOB queuing being removed
> > and replaced with a new single CMSG type that conveys the user key ID.  I
> > don't think it likely anyone is using this outside of my kafs-utils
> > package.  
> 
> If the remaining Sashiko issues are false-positives - should we queue
> this to net-next, then?

Blah, doesn't apply or pull cleanly now to either tree :(


From syzbot+89001660f14fb7684c83 at syzkaller.appspotmail.com  Wed Aug  5 17:12:36 2026
From: syzbot+89001660f14fb7684c83 at syzkaller.appspotmail.com (syzbot)
Date: Wed, 05 Aug 2026 17:12:36 -0700
Subject: [syzbot] [afs?] [net?] kernel BUG in rxrpc_destroy_all_connections (2)
Message-ID: <6a73d174.5aeeccc9.207e63.000c.GAE@google.com>

Hello,

syzbot found the following issue on:

HEAD commit:    c21bb4193868 Merge tag 'for_linus' of git://git.kernel.org..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=1126cbb9580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=b7079b368dcec295
dashboard link: https://syzkaller.appspot.com/bug?extid=89001660f14fb7684c83
compiler:       gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/9f5212fcea24/disk-c21bb419.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/9a166d81a082/vmlinux-c21bb419.xz
kernel image: https://storage.googleapis.com/syzbot-assets/b36458f10930/bzImage-c21bb419.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+89001660f14fb7684c83 at syzkaller.appspotmail.com

tipc: Left network mode
rxrpc: Assertion failed
------------[ cut here ]------------
kernel BUG at net/rxrpc/conn_object.c:488!
Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
CPU: 0 UID: 0 PID: 1040 Comm: kworker/u8:6 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/16/2026
Workqueue: netns cleanup_net
RIP: 0010:rxrpc_destroy_all_connections.cold+0x12/0x14 net/rxrpc/conn_object.c:488
Code: ff 90 0f 0b e8 ca 82 98 00 48 c7 c7 80 12 3a 8d e8 9e 79 b0 ff 90 0f 0b e8 b6 82 98 00 48 c7 c7 80 12 3a 8d e8 8a 79 b0 ff 90 <0f> 0b e8 a2 82 98 00 49 63 d6 48 83 ce ff 48 c7 c7 20 7c de 90 e8
RSP: 0018:ffffc9000519f990 EFLAGS: 00010282
RAX: 0000000000000017 RBX: ffff888029c40000 RCX: 0000000000000000
RDX: 0000000000000017 RSI: ffffffff81e89fd9 RDI: fffff52000a33f23
RBP: ffff888029c40064 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000080000000 R11: 0000000000000000 R12: 1ffff92000a33f34
R13: ffff888029c40078 R14: ffff888029c40088 R15: ffff888029c40088
FS:  0000000000000000(0000) GS:ffff888123dd9000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6c701e4b45 CR3: 0000000057648000 CR4: 0000000000350ef0
Call Trace:
 <TASK>
 rxrpc_exit_net+0x7b/0xc0 net/rxrpc/net_ns.c:113
 ops_exit_list net/core/net_namespace.c:199 [inline]
 ops_undo_list+0x2ee/0xab0 net/core/net_namespace.c:252
 cleanup_net+0x499/0x920 net/core/net_namespace.c:702
 process_one_work+0xa23/0x1940 kernel/workqueue.c:3322
 process_scheduled_works kernel/workqueue.c:3405 [inline]
 worker_thread+0x5ef/0xe50 kernel/workqueue.c:3486
 kthread+0x370/0x450 kernel/kthread.c:436
 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
 </TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:rxrpc_destroy_all_connections.cold+0x12/0x14 net/rxrpc/conn_object.c:488
Code: ff 90 0f 0b e8 ca 82 98 00 48 c7 c7 80 12 3a 8d e8 9e 79 b0 ff 90 0f 0b e8 b6 82 98 00 48 c7 c7 80 12 3a 8d e8 8a 79 b0 ff 90 <0f> 0b e8 a2 82 98 00 49 63 d6 48 83 ce ff 48 c7 c7 20 7c de 90 e8
RSP: 0018:ffffc9000519f990 EFLAGS: 00010282
RAX: 0000000000000017 RBX: ffff888029c40000 RCX: 0000000000000000
RDX: 0000000000000017 RSI: ffffffff81e89fd9 RDI: fffff52000a33f23
RBP: ffff888029c40064 R08: 0000000000000005 R09: 0000000000000000
R10: 0000000080000000 R11: 0000000000000000 R12: 1ffff92000a33f34
R13: ffff888029c40078 R14: ffff888029c40088 R15: ffff888029c40088
FS:  0000000000000000(0000) GS:ffff888123dd9000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558cf9997138 CR3: 000000000e994000 CR4: 0000000000350ef0


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller at googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup


From sashal at kernel.org  Thu Aug  6 05:19:41 2026
From: sashal at kernel.org (Sasha Levin)
Date: Thu,  6 Aug 2026 08:19:41 -0400
Subject: [PATCH 6.12.y] rxrpc: Fix irq-disabled in local_bh_enable()
In-Reply-To: <2026073013-sanding-unlined-9b91@gregkh>
References: <2026073013-sanding-unlined-9b91@gregkh>
Message-ID: <20260806121941.265456-1-sashal@kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit e4d2878369d590bf8455e3678a644e503172eafa ]

The rxrpc_assess_MTU_size() function calls down into the IP layer to find
out the MTU size for a route.  When accepting an incoming call, this is
called from rxrpc_new_incoming_call() which holds interrupts disabled
across the code that calls down to it.  Unfortunately, the IP layer uses
local_bh_enable() which, config dependent, throws a warning if IRQs are
enabled:

WARNING: CPU: 1 PID: 5544 at kernel/softirq.c:387 __local_bh_enable_ip+0x43/0xd0
...
RIP: 0010:__local_bh_enable_ip+0x43/0xd0
...
Call Trace:
 <TASK>
 rt_cache_route+0x7e/0xa0
 rt_set_nexthop.isra.0+0x3b3/0x3f0
 __mkroute_output+0x43a/0x460
 ip_route_output_key_hash+0xf7/0x140
 ip_route_output_flow+0x1b/0x90
 rxrpc_assess_MTU_size.isra.0+0x2a0/0x590
 rxrpc_new_incoming_peer+0x46/0x120
 rxrpc_alloc_incoming_call+0x1b1/0x400
 rxrpc_new_incoming_call+0x1da/0x5e0
 rxrpc_input_packet+0x827/0x900
 rxrpc_io_thread+0x403/0xb60
 kthread+0x2f7/0x310
 ret_from_fork+0x2a/0x230
 ret_from_fork_asm+0x1a/0x30
...
hardirqs last  enabled at (23): _raw_spin_unlock_irq+0x24/0x50
hardirqs last disabled at (24): _raw_read_lock_irq+0x17/0x70
softirqs last  enabled at (0): copy_process+0xc61/0x2730
softirqs last disabled at (25): rt_add_uncached_list+0x3c/0x90

Fix this by moving the call to rxrpc_assess_MTU_size() out of
rxrpc_init_peer() and further up the stack where it can be done without
interrupts disabled.

It shouldn't be a problem for rxrpc_new_incoming_call() to do it after the
locks are dropped as pmtud is going to be performed by the I/O thread - and
we're in the I/O thread at this point.

Fixes: a2ea9a907260 ("rxrpc: Use irq-disabling spinlocks between app and I/O thread")
Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Jeffrey Altman <jaltman at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Junvyyang, Tencent Zhuque Lab <zhuque at tencent.com>
cc: LePremierHomme <kwqcheii at proton.me>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20250717074350.3767366-2-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
[ moved `peer->mtu`/`peer->maxdata` derivation into `rxrpc_assess_MTU_size()` at both exits since 6.12 predates the `max_data` rework ]
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 net/rxrpc/ar-internal.h |  1 +
 net/rxrpc/call_accept.c |  1 +
 net/rxrpc/peer_object.c | 12 +++++++-----
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 79a377f1b5517..c78e86ba2261f 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -1180,6 +1180,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *,
 					 const struct sockaddr_rxrpc *);
 struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local,
 				     struct sockaddr_rxrpc *srx, gfp_t gfp);
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer);
 struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t,
 				    enum rxrpc_peer_trace);
 void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer);
diff --git a/net/rxrpc/call_accept.c b/net/rxrpc/call_accept.c
index 15e63b1ae7134..82766feb34e51 100644
--- a/net/rxrpc/call_accept.c
+++ b/net/rxrpc/call_accept.c
@@ -407,6 +407,7 @@ bool rxrpc_new_incoming_call(struct rxrpc_local *local,
 
 	spin_unlock(&rx->incoming_lock);
 	read_unlock_irq(&local->services_lock);
+	rxrpc_assess_MTU_size(local, call->peer);
 
 	if (hlist_unhashed(&call->error_link)) {
 		spin_lock_irq(&call->peer->lock);
diff --git a/net/rxrpc/peer_object.c b/net/rxrpc/peer_object.c
index a09b8aaf9bcdf..882c8397b859e 100644
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -149,8 +149,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *local,
  * assess the MTU size for the network interface through which this peer is
  * reached
  */
-static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
-				  struct rxrpc_peer *peer)
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer)
 {
 	struct net *net = local->net;
 	struct dst_entry *dst;
@@ -162,6 +161,8 @@ static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
 #endif
 
 	peer->if_mtu = 1500;
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
 
 	memset(&fl, 0, sizeof(fl));
 	switch (peer->srx.transport.family) {
@@ -201,6 +202,9 @@ static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
 	peer->if_mtu = dst_mtu(dst);
 	dst_release(dst);
 
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
+
 	_leave(" [if_mtu %u]", peer->if_mtu);
 }
 
@@ -242,8 +246,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer,
 			    unsigned long hash_key)
 {
 	peer->hash_key = hash_key;
-	rxrpc_assess_MTU_size(local, peer);
-	peer->mtu = peer->if_mtu;
 	peer->rtt_last_req = ktime_get_real();
 
 	switch (peer->srx.transport.family) {
@@ -268,7 +270,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer,
 	}
 
 	peer->hdrsize += sizeof(struct rxrpc_wire_header);
-	peer->maxdata = peer->mtu - peer->hdrsize;
 }
 
 /*
@@ -287,6 +288,7 @@ static struct rxrpc_peer *rxrpc_create_peer(struct rxrpc_local *local,
 	if (peer) {
 		memcpy(&peer->srx, srx, sizeof(*srx));
 		rxrpc_init_peer(local, peer, hash_key);
+		rxrpc_assess_MTU_size(local, peer);
 	}
 
 	_leave(" = %p", peer);
-- 
2.53.0



From sashal at kernel.org  Thu Aug  6 05:46:13 2026
From: sashal at kernel.org (Sasha Levin)
Date: Thu,  6 Aug 2026 08:46:13 -0400
Subject: [PATCH 6.6.y] rxrpc: Fix irq-disabled in local_bh_enable()
In-Reply-To: <2026073004-doodle-trickery-d104@gregkh>
References: <2026073004-doodle-trickery-d104@gregkh>
Message-ID: <20260806124613.299390-1-sashal@kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit e4d2878369d590bf8455e3678a644e503172eafa ]

The rxrpc_assess_MTU_size() function calls down into the IP layer to find
out the MTU size for a route.  When accepting an incoming call, this is
called from rxrpc_new_incoming_call() which holds interrupts disabled
across the code that calls down to it.  Unfortunately, the IP layer uses
local_bh_enable() which, config dependent, throws a warning if IRQs are
enabled:

WARNING: CPU: 1 PID: 5544 at kernel/softirq.c:387 __local_bh_enable_ip+0x43/0xd0
...
RIP: 0010:__local_bh_enable_ip+0x43/0xd0
...
Call Trace:
 <TASK>
 rt_cache_route+0x7e/0xa0
 rt_set_nexthop.isra.0+0x3b3/0x3f0
 __mkroute_output+0x43a/0x460
 ip_route_output_key_hash+0xf7/0x140
 ip_route_output_flow+0x1b/0x90
 rxrpc_assess_MTU_size.isra.0+0x2a0/0x590
 rxrpc_new_incoming_peer+0x46/0x120
 rxrpc_alloc_incoming_call+0x1b1/0x400
 rxrpc_new_incoming_call+0x1da/0x5e0
 rxrpc_input_packet+0x827/0x900
 rxrpc_io_thread+0x403/0xb60
 kthread+0x2f7/0x310
 ret_from_fork+0x2a/0x230
 ret_from_fork_asm+0x1a/0x30
...
hardirqs last  enabled at (23): _raw_spin_unlock_irq+0x24/0x50
hardirqs last disabled at (24): _raw_read_lock_irq+0x17/0x70
softirqs last  enabled at (0): copy_process+0xc61/0x2730
softirqs last disabled at (25): rt_add_uncached_list+0x3c/0x90

Fix this by moving the call to rxrpc_assess_MTU_size() out of
rxrpc_init_peer() and further up the stack where it can be done without
interrupts disabled.

It shouldn't be a problem for rxrpc_new_incoming_call() to do it after the
locks are dropped as pmtud is going to be performed by the I/O thread - and
we're in the I/O thread at this point.

Fixes: a2ea9a907260 ("rxrpc: Use irq-disabling spinlocks between app and I/O thread")
Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Jeffrey Altman <jaltman at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Junvyyang, Tencent Zhuque Lab <zhuque at tencent.com>
cc: LePremierHomme <kwqcheii at proton.me>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20250717074350.3767366-2-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
[ moved `peer->mtu`/`peer->maxdata` derivation into `rxrpc_assess_MTU_size()` at both exits since 6.12 predates the `max_data` rework ]
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 net/rxrpc/ar-internal.h |  1 +
 net/rxrpc/call_accept.c |  1 +
 net/rxrpc/peer_object.c | 12 +++++++-----
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 5f5421680b2eb..494a87be3d854 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -1188,6 +1188,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *,
 					 const struct sockaddr_rxrpc *);
 struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local,
 				     struct sockaddr_rxrpc *srx, gfp_t gfp);
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer);
 struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t,
 				    enum rxrpc_peer_trace);
 void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer);
diff --git a/net/rxrpc/call_accept.c b/net/rxrpc/call_accept.c
index cbf6187d68edf..66994e3698efb 100644
--- a/net/rxrpc/call_accept.c
+++ b/net/rxrpc/call_accept.c
@@ -407,6 +407,7 @@ bool rxrpc_new_incoming_call(struct rxrpc_local *local,
 
 	spin_unlock(&rx->incoming_lock);
 	read_unlock_irq(&local->services_lock);
+	rxrpc_assess_MTU_size(local, call->peer);
 
 	if (hlist_unhashed(&call->error_link)) {
 		spin_lock_irq(&call->peer->lock);
diff --git a/net/rxrpc/peer_object.c b/net/rxrpc/peer_object.c
index 3a4cf95ecddfc..7826081410290 100644
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -147,8 +147,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *local,
  * assess the MTU size for the network interface through which this peer is
  * reached
  */
-static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
-				  struct rxrpc_peer *peer)
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer)
 {
 	struct net *net = local->net;
 	struct dst_entry *dst;
@@ -160,6 +159,8 @@ static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
 #endif
 
 	peer->if_mtu = 1500;
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
 
 	memset(&fl, 0, sizeof(fl));
 	switch (peer->srx.transport.family) {
@@ -199,6 +200,9 @@ static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
 	peer->if_mtu = dst_mtu(dst);
 	dst_release(dst);
 
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
+
 	_leave(" [if_mtu %u]", peer->if_mtu);
 }
 
@@ -240,8 +244,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer,
 			    unsigned long hash_key)
 {
 	peer->hash_key = hash_key;
-	rxrpc_assess_MTU_size(local, peer);
-	peer->mtu = peer->if_mtu;
 	peer->rtt_last_req = ktime_get_real();
 
 	switch (peer->srx.transport.family) {
@@ -266,7 +268,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer,
 	}
 
 	peer->hdrsize += sizeof(struct rxrpc_wire_header);
-	peer->maxdata = peer->mtu - peer->hdrsize;
 }
 
 /*
@@ -285,6 +286,7 @@ static struct rxrpc_peer *rxrpc_create_peer(struct rxrpc_local *local,
 	if (peer) {
 		memcpy(&peer->srx, srx, sizeof(*srx));
 		rxrpc_init_peer(local, peer, hash_key);
+		rxrpc_assess_MTU_size(local, peer);
 	}
 
 	_leave(" = %p", peer);
-- 
2.53.0



From sashal at kernel.org  Thu Aug  6 06:50:03 2026
From: sashal at kernel.org (Sasha Levin)
Date: Thu,  6 Aug 2026 09:50:03 -0400
Subject: [PATCH 6.12.y 5/5] rxrpc: Fix irq-disabled in local_bh_enable()
In-Reply-To: <20260806135003.602726-1-sashal@kernel.org>
References: <2026073013-sanding-unlined-9b91@gregkh>
 <20260806135003.602726-1-sashal@kernel.org>
Message-ID: <20260806135003.602726-5-sashal@kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit e4d2878369d590bf8455e3678a644e503172eafa ]

The rxrpc_assess_MTU_size() function calls down into the IP layer to find
out the MTU size for a route.  When accepting an incoming call, this is
called from rxrpc_new_incoming_call() which holds interrupts disabled
across the code that calls down to it.  Unfortunately, the IP layer uses
local_bh_enable() which, config dependent, throws a warning if IRQs are
enabled:

WARNING: CPU: 1 PID: 5544 at kernel/softirq.c:387 __local_bh_enable_ip+0x43/0xd0
...
RIP: 0010:__local_bh_enable_ip+0x43/0xd0
...
Call Trace:
 <TASK>
 rt_cache_route+0x7e/0xa0
 rt_set_nexthop.isra.0+0x3b3/0x3f0
 __mkroute_output+0x43a/0x460
 ip_route_output_key_hash+0xf7/0x140
 ip_route_output_flow+0x1b/0x90
 rxrpc_assess_MTU_size.isra.0+0x2a0/0x590
 rxrpc_new_incoming_peer+0x46/0x120
 rxrpc_alloc_incoming_call+0x1b1/0x400
 rxrpc_new_incoming_call+0x1da/0x5e0
 rxrpc_input_packet+0x827/0x900
 rxrpc_io_thread+0x403/0xb60
 kthread+0x2f7/0x310
 ret_from_fork+0x2a/0x230
 ret_from_fork_asm+0x1a/0x30
...
hardirqs last  enabled at (23): _raw_spin_unlock_irq+0x24/0x50
hardirqs last disabled at (24): _raw_read_lock_irq+0x17/0x70
softirqs last  enabled at (0): copy_process+0xc61/0x2730
softirqs last disabled at (25): rt_add_uncached_list+0x3c/0x90

Fix this by moving the call to rxrpc_assess_MTU_size() out of
rxrpc_init_peer() and further up the stack where it can be done without
interrupts disabled.

It shouldn't be a problem for rxrpc_new_incoming_call() to do it after the
locks are dropped as pmtud is going to be performed by the I/O thread - and
we're in the I/O thread at this point.

Fixes: a2ea9a907260 ("rxrpc: Use irq-disabling spinlocks between app and I/O thread")
Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Jeffrey Altman <jaltman at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Junvyyang, Tencent Zhuque Lab <zhuque at tencent.com>
cc: LePremierHomme <kwqcheii at proton.me>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20250717074350.3767366-2-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Signed-off-by: Sasha Levin <sashal at kernel.org>
---
 net/rxrpc/ar-internal.h |  1 +
 net/rxrpc/call_accept.c |  1 +
 net/rxrpc/peer_object.c | 12 +++++++-----
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index a3afe7413e771..48202528a2438 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -1183,6 +1183,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *,
 					 const struct sockaddr_rxrpc *);
 struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local,
 				     struct sockaddr_rxrpc *srx, gfp_t gfp);
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer);
 struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t,
 				    enum rxrpc_peer_trace);
 void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer);
diff --git a/net/rxrpc/call_accept.c b/net/rxrpc/call_accept.c
index 15e63b1ae7134..82766feb34e51 100644
--- a/net/rxrpc/call_accept.c
+++ b/net/rxrpc/call_accept.c
@@ -407,6 +407,7 @@ bool rxrpc_new_incoming_call(struct rxrpc_local *local,
 
 	spin_unlock(&rx->incoming_lock);
 	read_unlock_irq(&local->services_lock);
+	rxrpc_assess_MTU_size(local, call->peer);
 
 	if (hlist_unhashed(&call->error_link)) {
 		spin_lock_irq(&call->peer->lock);
diff --git a/net/rxrpc/peer_object.c b/net/rxrpc/peer_object.c
index 45d5f25db40b2..3af39fc67d34b 100644
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -149,8 +149,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu(struct rxrpc_local *local,
  * assess the MTU size for the network interface through which this peer is
  * reached
  */
-static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
-				  struct rxrpc_peer *peer)
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer)
 {
 	struct net *net = local->net;
 	struct dst_entry *dst;
@@ -162,6 +161,8 @@ static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
 #endif
 
 	peer->if_mtu = 1500;
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
 
 	memset(&fl, 0, sizeof(fl));
 	switch (peer->srx.transport.family) {
@@ -201,6 +202,9 @@ static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
 	peer->if_mtu = dst_mtu(dst);
 	dst_release(dst);
 
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
+
 	_leave(" [if_mtu %u]", peer->if_mtu);
 }
 
@@ -239,8 +243,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer,
 			    unsigned long hash_key)
 {
 	peer->hash_key = hash_key;
-	rxrpc_assess_MTU_size(local, peer);
-	peer->mtu = peer->if_mtu;
 
 	switch (peer->srx.transport.family) {
 	case AF_INET:
@@ -264,7 +266,6 @@ static void rxrpc_init_peer(struct rxrpc_local *local, struct rxrpc_peer *peer,
 	}
 
 	peer->hdrsize += sizeof(struct rxrpc_wire_header);
-	peer->maxdata = peer->mtu - peer->hdrsize;
 }
 
 /*
@@ -283,6 +284,7 @@ static struct rxrpc_peer *rxrpc_create_peer(struct rxrpc_local *local,
 	if (peer) {
 		memcpy(&peer->srx, srx, sizeof(*srx));
 		rxrpc_init_peer(local, peer, hash_key);
+		rxrpc_assess_MTU_size(local, peer);
 	}
 
 	_leave(" = %p", peer);
-- 
2.53.0



From gregkh at linuxfoundation.org  Fri Aug  7 06:07:26 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Fri, 07 Aug 2026 15:07:26 +0200
Subject: Patch "rxrpc: Fix irq-disabled in local_bh_enable()" has been added to the 6.12-stable tree
In-Reply-To: <20260806135003.602726-5-sashal@kernel.org>
Message-ID: <2026080726-luster-library-1437@gregkh>


This is a note to let you know that I've just added the patch titled

    rxrpc: Fix irq-disabled in local_bh_enable()

to the 6.12-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     rxrpc-fix-irq-disabled-in-local_bh_enable.patch
and it can be found in the queue-6.12 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From stable+bounces-296838-greg=kroah.com at vger.kernel.org Thu Aug  6 15:52:26 2026
From: Sasha Levin <sashal at kernel.org>
Date: Thu,  6 Aug 2026 09:50:03 -0400
Subject: rxrpc: Fix irq-disabled in local_bh_enable()
To: stable at vger.kernel.org
Cc: David Howells <dhowells at redhat.com>, Jeffrey Altman <jaltman at auristor.com>, Marc Dionne <marc.dionne at auristor.com>, "Junvyyang, Tencent Zhuque Lab" <zhuque at tencent.com>, LePremierHomme <kwqcheii at proton.me>, Simon Horman <horms at kernel.org>, linux-afs at lists.infradead.org, Jakub Kicinski <kuba at kernel.org>, Sasha Levin <sashal at kernel.org>
Message-ID: <20260806135003.602726-5-sashal at kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit e4d2878369d590bf8455e3678a644e503172eafa ]

The rxrpc_assess_MTU_size() function calls down into the IP layer to find
out the MTU size for a route.  When accepting an incoming call, this is
called from rxrpc_new_incoming_call() which holds interrupts disabled
across the code that calls down to it.  Unfortunately, the IP layer uses
local_bh_enable() which, config dependent, throws a warning if IRQs are
enabled:

WARNING: CPU: 1 PID: 5544 at kernel/softirq.c:387 __local_bh_enable_ip+0x43/0xd0
...
RIP: 0010:__local_bh_enable_ip+0x43/0xd0
...
Call Trace:
 <TASK>
 rt_cache_route+0x7e/0xa0
 rt_set_nexthop.isra.0+0x3b3/0x3f0
 __mkroute_output+0x43a/0x460
 ip_route_output_key_hash+0xf7/0x140
 ip_route_output_flow+0x1b/0x90
 rxrpc_assess_MTU_size.isra.0+0x2a0/0x590
 rxrpc_new_incoming_peer+0x46/0x120
 rxrpc_alloc_incoming_call+0x1b1/0x400
 rxrpc_new_incoming_call+0x1da/0x5e0
 rxrpc_input_packet+0x827/0x900
 rxrpc_io_thread+0x403/0xb60
 kthread+0x2f7/0x310
 ret_from_fork+0x2a/0x230
 ret_from_fork_asm+0x1a/0x30
...
hardirqs last  enabled at (23): _raw_spin_unlock_irq+0x24/0x50
hardirqs last disabled at (24): _raw_read_lock_irq+0x17/0x70
softirqs last  enabled at (0): copy_process+0xc61/0x2730
softirqs last disabled at (25): rt_add_uncached_list+0x3c/0x90

Fix this by moving the call to rxrpc_assess_MTU_size() out of
rxrpc_init_peer() and further up the stack where it can be done without
interrupts disabled.

It shouldn't be a problem for rxrpc_new_incoming_call() to do it after the
locks are dropped as pmtud is going to be performed by the I/O thread - and
we're in the I/O thread at this point.

Fixes: a2ea9a907260 ("rxrpc: Use irq-disabling spinlocks between app and I/O thread")
Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Jeffrey Altman <jaltman at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Junvyyang, Tencent Zhuque Lab <zhuque at tencent.com>
cc: LePremierHomme <kwqcheii at proton.me>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20250717074350.3767366-2-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 net/rxrpc/ar-internal.h |    1 +
 net/rxrpc/call_accept.c |    1 +
 net/rxrpc/peer_object.c |   12 +++++++-----
 3 files changed, 9 insertions(+), 5 deletions(-)

--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -1183,6 +1183,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu
 					 const struct sockaddr_rxrpc *);
 struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local,
 				     struct sockaddr_rxrpc *srx, gfp_t gfp);
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer);
 struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t,
 				    enum rxrpc_peer_trace);
 void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer);
--- a/net/rxrpc/call_accept.c
+++ b/net/rxrpc/call_accept.c
@@ -407,6 +407,7 @@ bool rxrpc_new_incoming_call(struct rxrp
 
 	spin_unlock(&rx->incoming_lock);
 	read_unlock_irq(&local->services_lock);
+	rxrpc_assess_MTU_size(local, call->peer);
 
 	if (hlist_unhashed(&call->error_link)) {
 		spin_lock_irq(&call->peer->lock);
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -149,8 +149,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu
  * assess the MTU size for the network interface through which this peer is
  * reached
  */
-static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
-				  struct rxrpc_peer *peer)
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer)
 {
 	struct net *net = local->net;
 	struct dst_entry *dst;
@@ -162,6 +161,8 @@ static void rxrpc_assess_MTU_size(struct
 #endif
 
 	peer->if_mtu = 1500;
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
 
 	memset(&fl, 0, sizeof(fl));
 	switch (peer->srx.transport.family) {
@@ -201,6 +202,9 @@ static void rxrpc_assess_MTU_size(struct
 	peer->if_mtu = dst_mtu(dst);
 	dst_release(dst);
 
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
+
 	_leave(" [if_mtu %u]", peer->if_mtu);
 }
 
@@ -239,8 +243,6 @@ static void rxrpc_init_peer(struct rxrpc
 			    unsigned long hash_key)
 {
 	peer->hash_key = hash_key;
-	rxrpc_assess_MTU_size(local, peer);
-	peer->mtu = peer->if_mtu;
 
 	switch (peer->srx.transport.family) {
 	case AF_INET:
@@ -264,7 +266,6 @@ static void rxrpc_init_peer(struct rxrpc
 	}
 
 	peer->hdrsize += sizeof(struct rxrpc_wire_header);
-	peer->maxdata = peer->mtu - peer->hdrsize;
 }
 
 /*
@@ -283,6 +284,7 @@ static struct rxrpc_peer *rxrpc_create_p
 	if (peer) {
 		memcpy(&peer->srx, srx, sizeof(*srx));
 		rxrpc_init_peer(local, peer, hash_key);
+		rxrpc_assess_MTU_size(local, peer);
 	}
 
 	_leave(" = %p", peer);


Patches currently in stable-queue which might be from sashal at kernel.org are

queue-6.12/wifi-brcmfmac-set-f2-blocksize-to-256-for-bcm43752.patch
queue-6.12/net-dsa-mt7530-check-bus-read-errors-in-the-mdio-reg.patch
queue-6.12/um-set-parent-death-signal-for-userspace-process.patch
queue-6.12/usb-gadget-f_tcm-synchronize-delayed-set_alt-with-teardown.patch
queue-6.12/drm-xe-rtp-de-whitelist-oa-registers-for-all-hwe-s-f.patch
queue-6.12/media-uapi-rkisp-correct-name-version-enum.patch
queue-6.12/net-sxgbe-free-tx-rings-on-rx-allocation-failure.patch
queue-6.12/rtase-fix-double-free-of-multi-frag-skb-on-dma-map-f.patch
queue-6.12/usb-typec-ucsi-fix-race-condition-and-ordering-in-port-unregistration.patch
queue-6.12/drm-xe-introduce-xe_gt_dbg_printer.patch
queue-6.12/drm-fb-helper-allocate-and-release-fb_info-in-single-place.patch
queue-6.12/drm-xe-rtp-maintain-oa-whitelists-separately.patch
queue-6.12/pinctrl-qcom-sc8280xp-add-missing-wakeup-entries-for.patch
queue-6.12/forcedeth-fix-uaf-of-txrx_stats-in-nv_remove.patch
queue-6.12/alsa-hda-codecs-hdmi-disable-keep-alive-before-audio-format-change.patch
queue-6.12/hid-logitech-dj-standardise-hid_report_enum-variable.patch
queue-6.12/scsi-libsas-fix-ha-resume-deadlock-and-hisi_sas-disk.patch
queue-6.12/netfilter-nft_payload-fix-mask-build-for-partial-fie.patch
queue-6.12/bluetooth-iso-fix-timeout-vs-sync_timeout-typo-in-ch.patch
queue-6.12/um-set-parent-death-signal-for-ubd-io-thread-process.patch
queue-6.12/net-sxgbe-check-descriptor-ring-allocation-failures.patch
queue-6.12/ipvs-fix-the-checksum-validations.patch
queue-6.12/mptcp-pm-avoid-code-duplication-to-lookup-endp.patch
queue-6.12/hwmon-ina2xx-fix-various-overflow-issues.patch
queue-6.12/spi-spi-cadence-move-tx-fifo-full-busy-wait-into-fif.patch
queue-6.12/powerpc-boot-fix-simpleboot-cpu-node-lookup-check.patch
queue-6.12/ata-sata_mv-accept-1-or-2-resources-in-platform-prob.patch
queue-6.12/drm-xe-rtp-generalize-whitelist_apply_to_hwe.patch
queue-6.12/wifi-mac80211-validate-individual-twt-params-before-.patch
queue-6.12/mm-hugetlb-fix-swap-entry-corruption-when-clearing-u.patch
queue-6.12/dmaengine-idxd-fix-fdev-setup-failure-cleanup-in-idx.patch
queue-6.12/asoc-max98090-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/keys-trusted-dcp-fix-key_len-validation-and-calc_blo.patch
queue-6.12/drm-xe-oa-de-whitelist-oa-registers-on-oa-stream-ope.patch
queue-6.12/mptcp-pm-use-addr-entry-for-get_local_id.patch
queue-6.12/drm-xe-rtp-save-oa-nonpriv-registers-to-register-sav.patch
queue-6.12/bluetooth-hci_sync-remove-unnecessary-hci_conn_get-i.patch
queue-6.12/drm-mediatek-check-crtc-state-before-freeing.patch
queue-6.12/bluetooth-iso-clear-iso_data-always-when-detaching-c.patch
queue-6.12/bluetooth-iso-fix-leaking-sk-after-socket-release.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina260.patch
queue-6.12/accel-qaic-use-sizeof-trans_hdr-for-transaction-leng.patch
queue-6.12/pinctrl-amd-don-t-clear-s4-wake-bits-at-probe.patch
queue-6.12/net-bridge-mrp-fix-option-tlv-length-in-mrp_test-fra.patch
queue-6.12/kmemleak-iommu-iova-fix-transient-kmemleak-false-positive.patch
queue-6.12/gpio-pch-use-raw_spinlock_t-for-the-register-lock.patch
queue-6.12/rxrpc-generate-rtt_min.patch
queue-6.12/drm-xe-rtp-refactor-oag-mmio-trigger-register-whitel.patch
queue-6.12/hwmon-ina2xx-add-support-for-ina234.patch
queue-6.12/drm-xe-apply-whitelist-to-engine-save-restore.patch
queue-6.12/powerpc-boot-fix-treeboot-currituck-cpu-node-lookup-.patch
queue-6.12/wifi-ath6kl-fix-use-after-free-in-aggr_reset_state.patch
queue-6.12/wifi-brcmfmac-fix-43752-sdio-fwvid-incorrectly-labelled-as-cypress-cyw.patch
queue-6.12/keys-make-keyring-key-chunk-byte-order-agree-with-ke.patch
queue-6.12/drm-xe-pt-reset-current_op-in-xe_pt_update_ops_init.patch
queue-6.12/drm-xe-rtp-toggle-deny-bit-to-de-whitelist-oa-regs.patch
queue-6.12/hwmon-adt7470-fix-fans-stuck-in-manual-mode-on-i2c-e.patch
queue-6.12/net-airoha-fix-register-index-for-tx-fwd-counter-con.patch
queue-6.12/ata-ahci_ceva-fix-error-paths-in-ceva_ahci_platform_.patch
queue-6.12/qede-sync-udp_tunnel-ports-outside-qede_lock-in-the-.patch
queue-6.12/rxrpc-fix-the-calculation-and-use-of-rto.patch
queue-6.12/tracing-mmiotrace-reset-dropped_count-in-mmio_reset_.patch
queue-6.12/netfilter-nf_conntrack_sip-widen-nat-rewrite-delta-t.patch
queue-6.12/drm-i915-hdcp-move-to-using-intel_display-in-intel_hdcp.patch
queue-6.12/usb-typec-ucsi-split-connector-lock-classes.patch
queue-6.12/ahci-introduce-ahci_ignore_port-helper.patch
queue-6.12/drm-i915-hdcp-require-monotonically-increasing-seq_num_v.patch
queue-6.12/net-udp_tunnel-fix-memory-leak-in-udp_tunnel_nic_unr.patch
queue-6.12/bluetooth-iso-validate-sockaddr_iso-first-in-iso_soc.patch
queue-6.12/net-libwx-fix-fdir-atr-queue-mismatch-for-software-v.patch
queue-6.12/riscv-mm-fix-out-of-bounds-page-table-walk-during-me.patch
queue-6.12/netfilter-nf_tables-make-nft_object-rhltable-per-tab.patch
queue-6.12/mm-huge_memory-unlock-i_mmap_rwsem-before-releasing-.patch
queue-6.12/bluetooth-l2cap-fix-uaf-in-l2cap_le_connect_rsp.patch
queue-6.12/hwmon-ina226-add-support-for-sy24655.patch
queue-6.12/scsi-zfcp-fix-memory-leak-during-adapter-release-by-.patch
queue-6.12/mptcp-pm-userspace-fix-use-after-free-in-get_local_id.patch
queue-6.12/hwmon-ina2xx-make-it-easier-to-add-more-devices.patch
queue-6.12/net-dsa-mt7530-error-out-on-failed-reads-in-mt7531-p.patch
queue-6.12/scsi-libiscsi_tcp-bound-scsi-response-data-segment-t.patch
queue-6.12/bluetooth-hci_conn-hold-conn-reference-in-abort_conn.patch
queue-6.12/ipvs-do-not-mangle-icmp-replies-for-non-first-fragme.patch
queue-6.12/drm-i915-hdcp-check-streams-bounds-before-overflow.patch
queue-6.12/phy-zynqmp-fix-clock-error-handling-in-xpsgtr_phy_in.patch
queue-6.12/asoc-max98095-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.12/hid-logitech-dj-fix-wrong-detection-of-bad-dj_short-.patch
queue-6.12/rds-tcp-hold-the-rcu-lock-across-ipv6_chk_addr-in-rd.patch
queue-6.12/drm-xe-rtp-ensure-locking-ref-counting-for-oa-whitel.patch
queue-6.12/um-add-os_set_pdeathsig-helper-function.patch
queue-6.12/hwmon-lm90-only-report-alarms-if-driver-is-ready.patch
queue-6.12/of-reserved_mem-add-code-to-dynamically-allocate-res.patch
queue-6.12/bpf-reset-register-bounds-before-narrowing-retval-ra.patch
queue-6.12/drm-xe-rtp-keep-track-of-non-oa-nonpriv-slots.patch
queue-6.12/net-ethernet-mtk_eth_soc-support-named-irqs.patch
queue-6.12/wifi-brcmfmac-drain-bus_reset-work-on-device-removal.patch
queue-6.12/hwmon-adt7470-fix-divide-by-zero-toctou-crash-in-fan.patch
queue-6.12/mptcp-add-mptcp_userspace_pm_lookup_addr-helper.patch
queue-6.12/drm-sched-store-the-drm-client_id-in-drm_sched_fence.patch
queue-6.12/of-reserved_mem-prevent-oob-when-too-many-dynamic-re.patch
queue-6.12/hwmon-adt7470-fix-temperature-alarm-logic-in-hwmon_t.patch
queue-6.12/ksmbd-return-success-for-deferred-final-close.patch
queue-6.12/hwmon-ina2xx-add-support-for-has_alerts-configuratio.patch
queue-6.12/drm-tegra-fbdev-remove-offset-into-framebuffer-memory.patch
queue-6.12/scsi-ufs-core-cancel-rtc-work-in-active-active-suspe.patch
queue-6.12/drm-exec-remove-the-index-parameter-from-drm_exec_for_each_locked_obj.patch
queue-6.12/hwmon-adt7470-fix-cache-updated-before-hardware-writ.patch
queue-6.12/assoc_array-trim-the-final-shortcut-word-using-the-c.patch
queue-6.12/mm-kmemleak-fix-checksum-computation-for-per-cpu-objects.patch
queue-6.12/ksmbd-fix-use-after-free-in-__close_file_table_ids.patch
queue-6.12/drm-xe-wait-on-external-bo-kernel-fences-in-exec-ioctl.patch
queue-6.12/drm-amdgpu-gfx-fix-cleaner-shader-ib-buffer-overflow.patch
queue-6.12/netfilter-xt_hashlimit-validate-hashtable-supports-x.patch
queue-6.12/scsi-target-iblock-fix-wrong-pr-ops-null-check-for-p.patch
queue-6.12/net-ethernet-mtk_eth_soc-add-consts-for-irq-index.patch
queue-6.12/hwmon-adt7470-fix-swapped-pwm3-and-pwm4-auto-mode-ma.patch
queue-6.12/media-chips-media-wave5-support-cbp-profile.patch
queue-6.12/um-use-os_set_pdeathsig-helper-in-winch-thread-proce.patch
queue-6.12/idpf-fix-mailbox-irq-name-leak-on-request-failure.patch
queue-6.12/drm-i915-vrr-check-has_vrr-first-in-intel_vrr_is_capable.patch
queue-6.12/btrfs-fix-leaking-btrfs_fs_state_remounting-flag.patch
queue-6.12/netfilter-br_netfilter-reallocate-headroom-if-necess.patch
queue-6.12/idpf-adjust-txq-ring-count-minimum.patch
queue-6.12/pinctrl-qcom-unconditionally-mark-gpio-as-wakeup-ena.patch
queue-6.12/gpio-sloppy-logic-analyzer-fix-memory-leak-in-gpio_l.patch
queue-6.12/keys-fix-out-of-bounds-read-in-keyring_get_key_chunk.patch
queue-6.12/drm-amdgpu-fix-context-pstate-override-handling.patch
queue-6.12/hwmon-sht3x-fix-unaligned-accesses.patch
queue-6.12/mm-slab-prevent-unbounded-recursion-in-free-path-wit.patch
queue-6.12/fs-proc-task_mmu-fix-pagemap_scan-written-state-for-.patch
queue-6.12/hwmon-adt7470-use-cached-pwm-frequency-value.patch
queue-6.12/hwmon-pmbus-fix-return-value-from-pmbus_update_byte_.patch
queue-6.12/phy-zynqmp-fix-runtime-pm-leak-on-probe-allocation-f.patch
queue-6.12/scsi-libiscsi-fix-stale-data-leak-into-the-scsi-sens.patch
queue-6.12/tracing-mmiotrace-add-null-check-for-mmio_trace_arra.patch
queue-6.12/drm-xe-rtp-add-ring_force_to_nonpriv_deny-to-oa-whit.patch
queue-6.12/spi-spi-cadence-supports-transmission-with-bits_per_.patch
queue-6.12/kunit-tool-skip-stty-when-stdin-is-not-a-tty.patch
queue-6.12/um-preserve-errno-within-signal-handler.patch
queue-6.12/ipvs-fix-places-with-wrong-packet-offsets.patch
queue-6.12/hwmon-adt7470-fix-pwm-auto-temp-state-array-and-boun.patch
queue-6.12/hwmon-ina2xx-shift-ina234-shunt-and-current-register.patch
queue-6.12/drm-amdgpu-give-each-kernel-job-a-unique-id.patch
queue-6.12/netconsole-avoid-oob-reads-msg-is-not-nul-terminated.patch
queue-6.12/bluetooth-hci_sync-make-hci_cmd_sync_run_once-return.patch
queue-6.12/hwmon-adt7470-fix-busy-loop-and-i2c-flooding-in-upda.patch
queue-6.12/bluetooth-iso-avoid-deadlocks-in-iso_sock_timeout.patch
queue-6.12/rxrpc-fix-irq-disabled-in-local_bh_enable.patch
queue-6.12/drm-i915-hdcp-skip-inactive-mst-connectors-when-building-stream-list.patch
queue-6.12/hwmon-nct6775-core-fix-number-of-temperature-registe.patch
queue-6.12/hwmon-nct6775-core-prevent-access-to-unsupported-wei.patch
queue-6.12/dmaengine-sun6i-dma-fix-reclaim-descriptors-while-te.patch
queue-6.12/drm-xe-hold-a-dma-buf-reference-for-imported-bos.patch
queue-6.12/sched-deadline-use-revised-wakeup-rule-only-for-runn.patch
queue-6.12/rds-fix-inet6_addr_lst-null-dereference-when-ipv6-is.patch
queue-6.12/rxrpc-adjust-the-rxrpc_rtt_rx-tracepoint.patch
queue-6.12/media-i2c-imx219-rename-vts-to-frm_length.patch
queue-6.12/smb-client-fix-buffer-leaks-in-smb1-read-and-write.patch
queue-6.12/scsi-target-clear-cmd_cnt-when-initial-counter-enrol.patch
queue-6.12/lib-alloc_tag-introduce-mem_alloc_profiling_permanen.patch
queue-6.12/rxrpc-manage-rtt-per-call-rather-than-per-peer.patch
queue-6.12/bluetooth-hci_sync-fix-hci_conn_del-use-in-hci_le_cr.patch
queue-6.12/net-phylink-put-link_gpio-if-phylink_create-fails.patch
queue-6.12/btrfs-zoned-fix-deadlock-between-metadata-writeback-.patch
queue-6.12/media-imx219-fix-maximum-frame-length-in-lines.patch
queue-6.12/tracing-remove-trace_event_fl_filtered-logic.patch
queue-6.12/drm-i915-vrr-require-valid-min-max-vfreq-for-vrr.patch
queue-6.12/um-set-parent-death-signal-for-write_sigio-thread-pr.patch
queue-6.12/hwmon-nzxt-smart2-dma-align-output-buffer.patch
queue-6.12/rhashtable-clear-stale-iter-p-on-table-restart.patch
queue-6.12/drm-amdgpu-respect-placement-requirements-in-amdgpu_gtt_mgr-functions.patch
queue-6.12/powerpc-boot-fix-treeboot-akebono-cpu-node-lookup-ch.patch
queue-6.12/hwmon-ltc4282-fix-reading-the-minimum-alarm-voltage.patch
queue-6.12/thunderbolt-prevent-xdomain-delayed-work-use-after-f.patch
queue-6.12/ata-libahci_platform-support-non-consecutive-port-nu.patch
queue-6.12/octeontx2-pf-set-correct-sequence-for-carrier-off-an.patch
queue-6.12/net-mpls-initialize-rtm_tos-in-mpls_getroute.patch
queue-6.12/um-set-parent-death-signal-for-winch-thread-process.patch
queue-6.12/netfilter-nf_conntrack_expect-restore-helper-propaga.patch
queue-6.12/drm-xe-stub-out-new-pagefault-layer.patch
queue-6.12/bluetooth-btintel-validate-length-before-parsing-dia.patch
queue-6.12/net-ethernet-mtk_eth_soc-pass-eth-to-mtk_handle_irq_.patch
queue-6.12/drivers-hv-vmbus-replace-lockdep_hardirq_threaded-wi.patch
queue-6.12/drm-xe-rename-___xe_bo_create_locked.patch
queue-6.12/hid-logitech-dj-prevent-report_id_dj_short-related-u.patch
queue-6.12/net-do-not-send-icmp-ndisc-redirects-when-peer-alloc.patch
queue-6.12/phy-zynqmp-postpone-getting-clock-rate-until-actuall.patch
queue-6.12/kunit-tool-terminate-kernel-under-test-on-sigint.patch
queue-6.12/can-isotp-check-register_netdevice_notifier-error-in.patch


From gregkh at linuxfoundation.org  Fri Aug  7 06:49:55 2026
From: gregkh at linuxfoundation.org (gregkh at linuxfoundation.org)
Date: Fri, 07 Aug 2026 15:49:55 +0200
Subject: Patch "rxrpc: Fix irq-disabled in local_bh_enable()" has been added to the 6.6-stable tree
In-Reply-To: <20260806124613.299390-1-sashal@kernel.org>
Message-ID: <2026080755-troubling-compress-2e35@gregkh>


This is a note to let you know that I've just added the patch titled

    rxrpc: Fix irq-disabled in local_bh_enable()

to the 6.6-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     rxrpc-fix-irq-disabled-in-local_bh_enable.patch
and it can be found in the queue-6.6 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable at vger.kernel.org> know about it.


>From stable+bounces-296804-greg=kroah.com at vger.kernel.org Thu Aug  6 14:49:19 2026
From: Sasha Levin <sashal at kernel.org>
Date: Thu,  6 Aug 2026 08:46:13 -0400
Subject: rxrpc: Fix irq-disabled in local_bh_enable()
To: stable at vger.kernel.org
Cc: David Howells <dhowells at redhat.com>, Jeffrey Altman <jaltman at auristor.com>, Marc Dionne <marc.dionne at auristor.com>, "Junvyyang, Tencent Zhuque Lab" <zhuque at tencent.com>, LePremierHomme <kwqcheii at proton.me>, Simon Horman <horms at kernel.org>, linux-afs at lists.infradead.org, Jakub Kicinski <kuba at kernel.org>, Sasha Levin <sashal at kernel.org>
Message-ID: <20260806124613.299390-1-sashal at kernel.org>

From: David Howells <dhowells at redhat.com>

[ Upstream commit e4d2878369d590bf8455e3678a644e503172eafa ]

The rxrpc_assess_MTU_size() function calls down into the IP layer to find
out the MTU size for a route.  When accepting an incoming call, this is
called from rxrpc_new_incoming_call() which holds interrupts disabled
across the code that calls down to it.  Unfortunately, the IP layer uses
local_bh_enable() which, config dependent, throws a warning if IRQs are
enabled:

WARNING: CPU: 1 PID: 5544 at kernel/softirq.c:387 __local_bh_enable_ip+0x43/0xd0
...
RIP: 0010:__local_bh_enable_ip+0x43/0xd0
...
Call Trace:
 <TASK>
 rt_cache_route+0x7e/0xa0
 rt_set_nexthop.isra.0+0x3b3/0x3f0
 __mkroute_output+0x43a/0x460
 ip_route_output_key_hash+0xf7/0x140
 ip_route_output_flow+0x1b/0x90
 rxrpc_assess_MTU_size.isra.0+0x2a0/0x590
 rxrpc_new_incoming_peer+0x46/0x120
 rxrpc_alloc_incoming_call+0x1b1/0x400
 rxrpc_new_incoming_call+0x1da/0x5e0
 rxrpc_input_packet+0x827/0x900
 rxrpc_io_thread+0x403/0xb60
 kthread+0x2f7/0x310
 ret_from_fork+0x2a/0x230
 ret_from_fork_asm+0x1a/0x30
...
hardirqs last  enabled at (23): _raw_spin_unlock_irq+0x24/0x50
hardirqs last disabled at (24): _raw_read_lock_irq+0x17/0x70
softirqs last  enabled at (0): copy_process+0xc61/0x2730
softirqs last disabled at (25): rt_add_uncached_list+0x3c/0x90

Fix this by moving the call to rxrpc_assess_MTU_size() out of
rxrpc_init_peer() and further up the stack where it can be done without
interrupts disabled.

It shouldn't be a problem for rxrpc_new_incoming_call() to do it after the
locks are dropped as pmtud is going to be performed by the I/O thread - and
we're in the I/O thread at this point.

Fixes: a2ea9a907260 ("rxrpc: Use irq-disabling spinlocks between app and I/O thread")
Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Jeffrey Altman <jaltman at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Junvyyang, Tencent Zhuque Lab <zhuque at tencent.com>
cc: LePremierHomme <kwqcheii at proton.me>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20250717074350.3767366-2-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
[ moved `peer->mtu`/`peer->maxdata` derivation into `rxrpc_assess_MTU_size()` at both exits since 6.12 predates the `max_data` rework ]
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 net/rxrpc/ar-internal.h |    1 +
 net/rxrpc/call_accept.c |    1 +
 net/rxrpc/peer_object.c |   12 +++++++-----
 3 files changed, 9 insertions(+), 5 deletions(-)

--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -1188,6 +1188,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu
 					 const struct sockaddr_rxrpc *);
 struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local,
 				     struct sockaddr_rxrpc *srx, gfp_t gfp);
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer);
 struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t,
 				    enum rxrpc_peer_trace);
 void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer);
--- a/net/rxrpc/call_accept.c
+++ b/net/rxrpc/call_accept.c
@@ -407,6 +407,7 @@ bool rxrpc_new_incoming_call(struct rxrp
 
 	spin_unlock(&rx->incoming_lock);
 	read_unlock_irq(&local->services_lock);
+	rxrpc_assess_MTU_size(local, call->peer);
 
 	if (hlist_unhashed(&call->error_link)) {
 		spin_lock_irq(&call->peer->lock);
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -147,8 +147,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu
  * assess the MTU size for the network interface through which this peer is
  * reached
  */
-static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
-				  struct rxrpc_peer *peer)
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer)
 {
 	struct net *net = local->net;
 	struct dst_entry *dst;
@@ -160,6 +159,8 @@ static void rxrpc_assess_MTU_size(struct
 #endif
 
 	peer->if_mtu = 1500;
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
 
 	memset(&fl, 0, sizeof(fl));
 	switch (peer->srx.transport.family) {
@@ -199,6 +200,9 @@ static void rxrpc_assess_MTU_size(struct
 	peer->if_mtu = dst_mtu(dst);
 	dst_release(dst);
 
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
+
 	_leave(" [if_mtu %u]", peer->if_mtu);
 }
 
@@ -240,8 +244,6 @@ static void rxrpc_init_peer(struct rxrpc
 			    unsigned long hash_key)
 {
 	peer->hash_key = hash_key;
-	rxrpc_assess_MTU_size(local, peer);
-	peer->mtu = peer->if_mtu;
 	peer->rtt_last_req = ktime_get_real();
 
 	switch (peer->srx.transport.family) {
@@ -266,7 +268,6 @@ static void rxrpc_init_peer(struct rxrpc
 	}
 
 	peer->hdrsize += sizeof(struct rxrpc_wire_header);
-	peer->maxdata = peer->mtu - peer->hdrsize;
 }
 
 /*
@@ -285,6 +286,7 @@ static struct rxrpc_peer *rxrpc_create_p
 	if (peer) {
 		memcpy(&peer->srx, srx, sizeof(*srx));
 		rxrpc_init_peer(local, peer, hash_key);
+		rxrpc_assess_MTU_size(local, peer);
 	}
 
 	_leave(" = %p", peer);


Patches currently in stable-queue which might be from sashal at kernel.org are

queue-6.6/sysctl-treewide-constify-ctl_table_header-ctl_table_arg.patch
queue-6.6/wifi-brcmfmac-set-f2-blocksize-to-256-for-bcm43752.patch
queue-6.6/usb-gadget-f_tcm-synchronize-delayed-set_alt-with-teardown.patch
queue-6.6/net-sxgbe-free-tx-rings-on-rx-allocation-failure.patch
queue-6.6/usb-typec-ucsi-fix-race-condition-and-ordering-in-port-unregistration.patch
queue-6.6/drm-fb-helper-allocate-and-release-fb_info-in-single-place.patch
queue-6.6/pinctrl-qcom-sc8280xp-add-missing-wakeup-entries-for.patch
queue-6.6/forcedeth-fix-uaf-of-txrx_stats-in-nv_remove.patch
queue-6.6/hid-logitech-dj-standardise-hid_report_enum-variable.patch
queue-6.6/scsi-libsas-fix-ha-resume-deadlock-and-hisi_sas-disk.patch
queue-6.6/netfilter-nft_payload-fix-mask-build-for-partial-fie.patch
queue-6.6/bluetooth-iso-fix-timeout-vs-sync_timeout-typo-in-ch.patch
queue-6.6/net-sxgbe-check-descriptor-ring-allocation-failures.patch
queue-6.6/ipvs-fix-the-checksum-validations.patch
queue-6.6/mptcp-pm-avoid-code-duplication-to-lookup-endp.patch
queue-6.6/spi-spi-cadence-move-tx-fifo-full-busy-wait-into-fif.patch
queue-6.6/powerpc-boot-fix-simpleboot-cpu-node-lookup-check.patch
queue-6.6/ata-sata_mv-accept-1-or-2-resources-in-platform-prob.patch
queue-6.6/wifi-mac80211-validate-individual-twt-params-before-.patch
queue-6.6/mm-hugetlb-fix-swap-entry-corruption-when-clearing-u.patch
queue-6.6/dmaengine-idxd-fix-fdev-setup-failure-cleanup-in-idx.patch
queue-6.6/asoc-max98090-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.6/mptcp-pm-use-addr-entry-for-get_local_id.patch
queue-6.6/sched-add-task_struct-faults_disabled_mapping.patch
queue-6.6/libceph-add-doutc-and-_client-debug-macros-support.patch
queue-6.6/gve-fix-rx-queue-stall-on-alloc-failure.patch
queue-6.6/bluetooth-hci_sync-remove-unnecessary-hci_conn_get-i.patch
queue-6.6/drm-mediatek-check-crtc-state-before-freeing.patch
queue-6.6/usb-musb-omap2430-clean-up-probe-error-handling.patch
queue-6.6/bluetooth-iso-clear-iso_data-always-when-detaching-c.patch
queue-6.6/accel-qaic-use-sizeof-trans_hdr-for-transaction-leng.patch
queue-6.6/pinctrl-amd-don-t-clear-s4-wake-bits-at-probe.patch
queue-6.6/net-bridge-mrp-fix-option-tlv-length-in-mrp_test-fra.patch
queue-6.6/gpio-pch-use-raw_spinlock_t-for-the-register-lock.patch
queue-6.6/powerpc-boot-fix-treeboot-currituck-cpu-node-lookup-.patch
queue-6.6/wifi-ath6kl-fix-use-after-free-in-aggr_reset_state.patch
queue-6.6/wifi-brcmfmac-fix-43752-sdio-fwvid-incorrectly-labelled-as-cypress-cyw.patch
queue-6.6/keys-make-keyring-key-chunk-byte-order-agree-with-ke.patch
queue-6.6/hwmon-adt7470-fix-fans-stuck-in-manual-mode-on-i2c-e.patch
queue-6.6/ata-ahci_ceva-fix-error-paths-in-ceva_ahci_platform_.patch
queue-6.6/qede-sync-udp_tunnel-ports-outside-qede_lock-in-the-.patch
queue-6.6/tracing-mmiotrace-reset-dropped_count-in-mmio_reset_.patch
queue-6.6/netfilter-nf_conntrack_sip-widen-nat-rewrite-delta-t.patch
queue-6.6/usb-typec-ucsi-split-connector-lock-classes.patch
queue-6.6/ahci-introduce-ahci_ignore_port-helper.patch
queue-6.6/mm-huge_memory-unlock-i_mmap_rwsem-before-releasing-.patch
queue-6.6/bluetooth-l2cap-fix-uaf-in-l2cap_le_connect_rsp.patch
queue-6.6/scsi-zfcp-fix-memory-leak-during-adapter-release-by-.patch
queue-6.6/mptcp-pm-userspace-fix-use-after-free-in-get_local_id.patch
queue-6.6/net-dsa-mt7530-error-out-on-failed-reads-in-mt7531-p.patch
queue-6.6/scsi-libiscsi_tcp-bound-scsi-response-data-segment-t.patch
queue-6.6/bluetooth-hci_conn-hold-conn-reference-in-abort_conn.patch
queue-6.6/ipvs-do-not-mangle-icmp-replies-for-non-first-fragme.patch
queue-6.6/phy-zynqmp-fix-clock-error-handling-in-xpsgtr_phy_in.patch
queue-6.6/asoc-max98095-fix-missing-is_err-before-ptr_err-on-m.patch
queue-6.6/hid-logitech-dj-fix-wrong-detection-of-bad-dj_short-.patch
queue-6.6/rds-tcp-hold-the-rcu-lock-across-ipv6_chk_addr-in-rd.patch
queue-6.6/hwmon-lm90-only-report-alarms-if-driver-is-ready.patch
queue-6.6/ceph-print-cluster-fsid-and-client-global_id-in-all-debug-logs.patch
queue-6.6/media-v4l2-fwnode-fix-subdev-owner-overwritten-in-v4l2_async_register_subdev_sensor.patch
queue-6.6/wifi-brcmfmac-drain-bus_reset-work-on-device-removal.patch
queue-6.6/hwmon-adt7470-fix-divide-by-zero-toctou-crash-in-fan.patch
queue-6.6/mptcp-add-mptcp_userspace_pm_lookup_addr-helper.patch
queue-6.6/hwmon-adt7470-fix-temperature-alarm-logic-in-hwmon_t.patch
queue-6.6/ksmbd-return-success-for-deferred-final-close.patch
queue-6.6/drm-tegra-fbdev-remove-offset-into-framebuffer-memory.patch
queue-6.6/drm-renesas-move-rz-g2l-mipi-dsi-driver-to-rz-du.patch
queue-6.6/scsi-ufs-core-cancel-rtc-work-in-active-active-suspe.patch
queue-6.6/hwmon-adt7470-fix-cache-updated-before-hardware-writ.patch
queue-6.6/assoc_array-trim-the-final-shortcut-word-using-the-c.patch
queue-6.6/usb-musb-omap2430-do-not-put-borrowed-of_node-in-probe.patch
queue-6.6/ksmbd-fix-use-after-free-in-__close_file_table_ids.patch
queue-6.6/netfilter-xt_hashlimit-validate-hashtable-supports-x.patch
queue-6.6/scsi-target-iblock-fix-wrong-pr-ops-null-check-for-p.patch
queue-6.6/hwmon-adt7470-fix-swapped-pwm3-and-pwm4-auto-mode-ma.patch
queue-6.6/media-i2c-imx219-access-height-from-active-format-in-imx219_set_ctrl.patch
queue-6.6/usb-typec-ucsi-only-enable-supported-notifications.patch
queue-6.6/drm-i915-vrr-check-has_vrr-first-in-intel_vrr_is_capable.patch
queue-6.6/ipv6-fib6-fix-null-deref-in-fib6_walk_continue-on-mu.patch
queue-6.6/netfilter-br_netfilter-reallocate-headroom-if-necess.patch
queue-6.6/keys-fix-out-of-bounds-read-in-keyring_get_key_chunk.patch
queue-6.6/hwmon-adt7470-use-cached-pwm-frequency-value.patch
queue-6.6/hwmon-pmbus-fix-return-value-from-pmbus_update_byte_.patch
queue-6.6/phy-zynqmp-fix-runtime-pm-leak-on-probe-allocation-f.patch
queue-6.6/scsi-libiscsi-fix-stale-data-leak-into-the-scsi-sens.patch
queue-6.6/soc-qcom-ice-allow-explicit-votes-on-iface-clock-for.patch
queue-6.6/spi-spi-cadence-supports-transmission-with-bits_per_.patch
queue-6.6/sctp-close-udp-tunnel-sockets-during-netns-teardown.patch
queue-6.6/ipvs-fix-places-with-wrong-packet-offsets.patch
queue-6.6/drm-renesas-rzg2l_mipi_dsi-increase-reset-deassertion-delay.patch
queue-6.6/hwmon-adt7470-fix-pwm-auto-temp-state-array-and-boun.patch
queue-6.6/bluetooth-hci_sync-make-hci_cmd_sync_run_once-return.patch
queue-6.6/hwmon-adt7470-fix-busy-loop-and-i2c-flooding-in-upda.patch
queue-6.6/rxrpc-fix-irq-disabled-in-local_bh_enable.patch
queue-6.6/hwmon-nct6775-core-fix-number-of-temperature-registe.patch
queue-6.6/hwmon-nct6775-core-prevent-access-to-unsupported-wei.patch
queue-6.6/dmaengine-sun6i-dma-fix-reclaim-descriptors-while-te.patch
queue-6.6/media-i2c-imx219-drop-imx219_vts_-macros.patch
queue-6.6/rds-fix-inet6_addr_lst-null-dereference-when-ipv6-is.patch
queue-6.6/media-i2c-imx219-rename-vts-to-frm_length.patch
queue-6.6/smb-client-fix-buffer-leaks-in-smb1-read-and-write.patch
queue-6.6/scsi-target-clear-cmd_cnt-when-initial-counter-enrol.patch
queue-6.6/bluetooth-hci_sync-fix-hci_conn_del-use-in-hci_le_cr.patch
queue-6.6/net-phylink-put-link_gpio-if-phylink_create-fails.patch
queue-6.6/btrfs-zoned-fix-deadlock-between-metadata-writeback-.patch
queue-6.6/media-imx219-fix-maximum-frame-length-in-lines.patch
queue-6.6/media-i2c-imx219-group-functions-by-purpose.patch
queue-6.6/drm-i915-vrr-require-valid-min-max-vfreq-for-vrr.patch
queue-6.6/media-i2c-imx219-don-t-store-the-current-mode-in-the-imx219-structure.patch
queue-6.6/hwmon-nzxt-smart2-dma-align-output-buffer.patch
queue-6.6/rhashtable-clear-stale-iter-p-on-table-restart.patch
queue-6.6/drm-amdgpu-respect-placement-requirements-in-amdgpu_gtt_mgr-functions.patch
queue-6.6/powerpc-boot-fix-treeboot-akebono-cpu-node-lookup-ch.patch
queue-6.6/media-v4l-async-set-owner-for-async-sub-devices.patch
queue-6.6/thunderbolt-prevent-xdomain-delayed-work-use-after-f.patch
queue-6.6/netfilter-nf_tables-clone-set-on-flush-only.patch
queue-6.6/ata-libahci_platform-support-non-consecutive-port-nu.patch
queue-6.6/octeontx2-pf-set-correct-sequence-for-carrier-off-an.patch
queue-6.6/net-mpls-initialize-rtm_tos-in-mpls_getroute.patch
queue-6.6/ceph-fix-refcount-leak-in-ceph_readdir.patch
queue-6.6/netfilter-nf_conntrack_expect-restore-helper-propaga.patch
queue-6.6/bluetooth-btintel-validate-length-before-parsing-dia.patch
queue-6.6/firmware-stratix10-svc-fix-memory-leaks-and-list-cor.patch
queue-6.6/hid-logitech-dj-prevent-report_id_dj_short-related-u.patch
queue-6.6/sctp-avoid-auth_enable-sysctl-uaf-during-netns-teardown.patch
queue-6.6/iommu-sva-move-x86-disable-check-before-allocation.patch
queue-6.6/media-i2c-imx219-calculate-crop-rectangle-dynamically.patch
queue-6.6/net-do-not-send-icmp-ndisc-redirects-when-peer-alloc.patch
queue-6.6/phy-zynqmp-postpone-getting-clock-rate-until-actuall.patch
queue-6.6/can-isotp-check-register_netdevice_notifier-error-in.patch


From gregkh at linuxfoundation.org  Fri Aug  7 07:38:55 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:38:55 +0200
Subject: [PATCH 6.12 332/337] rxrpc: Fix irq-disabled in local_bh_enable()
In-Reply-To: <20260807143418.516897842@linuxfoundation.org>
References: <20260807143418.516897842@linuxfoundation.org>
Message-ID: <20260807143425.738699850@linuxfoundation.org>

6.12-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

[ Upstream commit e4d2878369d590bf8455e3678a644e503172eafa ]

The rxrpc_assess_MTU_size() function calls down into the IP layer to find
out the MTU size for a route.  When accepting an incoming call, this is
called from rxrpc_new_incoming_call() which holds interrupts disabled
across the code that calls down to it.  Unfortunately, the IP layer uses
local_bh_enable() which, config dependent, throws a warning if IRQs are
enabled:

WARNING: CPU: 1 PID: 5544 at kernel/softirq.c:387 __local_bh_enable_ip+0x43/0xd0
...
RIP: 0010:__local_bh_enable_ip+0x43/0xd0
...
Call Trace:
 <TASK>
 rt_cache_route+0x7e/0xa0
 rt_set_nexthop.isra.0+0x3b3/0x3f0
 __mkroute_output+0x43a/0x460
 ip_route_output_key_hash+0xf7/0x140
 ip_route_output_flow+0x1b/0x90
 rxrpc_assess_MTU_size.isra.0+0x2a0/0x590
 rxrpc_new_incoming_peer+0x46/0x120
 rxrpc_alloc_incoming_call+0x1b1/0x400
 rxrpc_new_incoming_call+0x1da/0x5e0
 rxrpc_input_packet+0x827/0x900
 rxrpc_io_thread+0x403/0xb60
 kthread+0x2f7/0x310
 ret_from_fork+0x2a/0x230
 ret_from_fork_asm+0x1a/0x30
...
hardirqs last  enabled at (23): _raw_spin_unlock_irq+0x24/0x50
hardirqs last disabled at (24): _raw_read_lock_irq+0x17/0x70
softirqs last  enabled at (0): copy_process+0xc61/0x2730
softirqs last disabled at (25): rt_add_uncached_list+0x3c/0x90

Fix this by moving the call to rxrpc_assess_MTU_size() out of
rxrpc_init_peer() and further up the stack where it can be done without
interrupts disabled.

It shouldn't be a problem for rxrpc_new_incoming_call() to do it after the
locks are dropped as pmtud is going to be performed by the I/O thread - and
we're in the I/O thread at this point.

Fixes: a2ea9a907260 ("rxrpc: Use irq-disabling spinlocks between app and I/O thread")
Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Jeffrey Altman <jaltman at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Junvyyang, Tencent Zhuque Lab <zhuque at tencent.com>
cc: LePremierHomme <kwqcheii at proton.me>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20250717074350.3767366-2-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 net/rxrpc/ar-internal.h |    1 +
 net/rxrpc/call_accept.c |    1 +
 net/rxrpc/peer_object.c |   12 +++++++-----
 3 files changed, 9 insertions(+), 5 deletions(-)

--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -1183,6 +1183,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu
 					 const struct sockaddr_rxrpc *);
 struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local,
 				     struct sockaddr_rxrpc *srx, gfp_t gfp);
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer);
 struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t,
 				    enum rxrpc_peer_trace);
 void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer);
--- a/net/rxrpc/call_accept.c
+++ b/net/rxrpc/call_accept.c
@@ -407,6 +407,7 @@ bool rxrpc_new_incoming_call(struct rxrp
 
 	spin_unlock(&rx->incoming_lock);
 	read_unlock_irq(&local->services_lock);
+	rxrpc_assess_MTU_size(local, call->peer);
 
 	if (hlist_unhashed(&call->error_link)) {
 		spin_lock_irq(&call->peer->lock);
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -149,8 +149,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu
  * assess the MTU size for the network interface through which this peer is
  * reached
  */
-static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
-				  struct rxrpc_peer *peer)
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer)
 {
 	struct net *net = local->net;
 	struct dst_entry *dst;
@@ -162,6 +161,8 @@ static void rxrpc_assess_MTU_size(struct
 #endif
 
 	peer->if_mtu = 1500;
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
 
 	memset(&fl, 0, sizeof(fl));
 	switch (peer->srx.transport.family) {
@@ -201,6 +202,9 @@ static void rxrpc_assess_MTU_size(struct
 	peer->if_mtu = dst_mtu(dst);
 	dst_release(dst);
 
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
+
 	_leave(" [if_mtu %u]", peer->if_mtu);
 }
 
@@ -239,8 +243,6 @@ static void rxrpc_init_peer(struct rxrpc
 			    unsigned long hash_key)
 {
 	peer->hash_key = hash_key;
-	rxrpc_assess_MTU_size(local, peer);
-	peer->mtu = peer->if_mtu;
 
 	switch (peer->srx.transport.family) {
 	case AF_INET:
@@ -264,7 +266,6 @@ static void rxrpc_init_peer(struct rxrpc
 	}
 
 	peer->hdrsize += sizeof(struct rxrpc_wire_header);
-	peer->maxdata = peer->mtu - peer->hdrsize;
 }
 
 /*
@@ -283,6 +284,7 @@ static struct rxrpc_peer *rxrpc_create_p
 	if (peer) {
 		memcpy(&peer->srx, srx, sizeof(*srx));
 		rxrpc_init_peer(local, peer, hash_key);
+		rxrpc_assess_MTU_size(local, peer);
 	}
 
 	_leave(" = %p", peer);




From gregkh at linuxfoundation.org  Fri Aug  7 07:39:51 2026
From: gregkh at linuxfoundation.org (Greg Kroah-Hartman)
Date: Fri,  7 Aug 2026 16:39:51 +0200
Subject: [PATCH 6.6 234/261] rxrpc: Fix irq-disabled in local_bh_enable()
In-Reply-To: <20260807143415.358597922@linuxfoundation.org>
References: <20260807143415.358597922@linuxfoundation.org>
Message-ID: <20260807143420.426507157@linuxfoundation.org>

6.6-stable review patch.  If anyone has any objections, please let me know.

------------------

From: David Howells <dhowells at redhat.com>

[ Upstream commit e4d2878369d590bf8455e3678a644e503172eafa ]

The rxrpc_assess_MTU_size() function calls down into the IP layer to find
out the MTU size for a route.  When accepting an incoming call, this is
called from rxrpc_new_incoming_call() which holds interrupts disabled
across the code that calls down to it.  Unfortunately, the IP layer uses
local_bh_enable() which, config dependent, throws a warning if IRQs are
enabled:

WARNING: CPU: 1 PID: 5544 at kernel/softirq.c:387 __local_bh_enable_ip+0x43/0xd0
...
RIP: 0010:__local_bh_enable_ip+0x43/0xd0
...
Call Trace:
 <TASK>
 rt_cache_route+0x7e/0xa0
 rt_set_nexthop.isra.0+0x3b3/0x3f0
 __mkroute_output+0x43a/0x460
 ip_route_output_key_hash+0xf7/0x140
 ip_route_output_flow+0x1b/0x90
 rxrpc_assess_MTU_size.isra.0+0x2a0/0x590
 rxrpc_new_incoming_peer+0x46/0x120
 rxrpc_alloc_incoming_call+0x1b1/0x400
 rxrpc_new_incoming_call+0x1da/0x5e0
 rxrpc_input_packet+0x827/0x900
 rxrpc_io_thread+0x403/0xb60
 kthread+0x2f7/0x310
 ret_from_fork+0x2a/0x230
 ret_from_fork_asm+0x1a/0x30
...
hardirqs last  enabled at (23): _raw_spin_unlock_irq+0x24/0x50
hardirqs last disabled at (24): _raw_read_lock_irq+0x17/0x70
softirqs last  enabled at (0): copy_process+0xc61/0x2730
softirqs last disabled at (25): rt_add_uncached_list+0x3c/0x90

Fix this by moving the call to rxrpc_assess_MTU_size() out of
rxrpc_init_peer() and further up the stack where it can be done without
interrupts disabled.

It shouldn't be a problem for rxrpc_new_incoming_call() to do it after the
locks are dropped as pmtud is going to be performed by the I/O thread - and
we're in the I/O thread at this point.

Fixes: a2ea9a907260 ("rxrpc: Use irq-disabling spinlocks between app and I/O thread")
Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Jeffrey Altman <jaltman at auristor.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Junvyyang, Tencent Zhuque Lab <zhuque at tencent.com>
cc: LePremierHomme <kwqcheii at proton.me>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
Link: https://patch.msgid.link/20250717074350.3767366-2-dhowells at redhat.com
Signed-off-by: Jakub Kicinski <kuba at kernel.org>
[ moved `peer->mtu`/`peer->maxdata` derivation into `rxrpc_assess_MTU_size()` at both exits since 6.12 predates the `max_data` rework ]
Signed-off-by: Sasha Levin <sashal at kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh at linuxfoundation.org>
---
 net/rxrpc/ar-internal.h |    1 +
 net/rxrpc/call_accept.c |    1 +
 net/rxrpc/peer_object.c |   12 +++++++-----
 3 files changed, 9 insertions(+), 5 deletions(-)

--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -1188,6 +1188,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu
 					 const struct sockaddr_rxrpc *);
 struct rxrpc_peer *rxrpc_lookup_peer(struct rxrpc_local *local,
 				     struct sockaddr_rxrpc *srx, gfp_t gfp);
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer);
 struct rxrpc_peer *rxrpc_alloc_peer(struct rxrpc_local *, gfp_t,
 				    enum rxrpc_peer_trace);
 void rxrpc_new_incoming_peer(struct rxrpc_local *local, struct rxrpc_peer *peer);
--- a/net/rxrpc/call_accept.c
+++ b/net/rxrpc/call_accept.c
@@ -407,6 +407,7 @@ bool rxrpc_new_incoming_call(struct rxrp
 
 	spin_unlock(&rx->incoming_lock);
 	read_unlock_irq(&local->services_lock);
+	rxrpc_assess_MTU_size(local, call->peer);
 
 	if (hlist_unhashed(&call->error_link)) {
 		spin_lock_irq(&call->peer->lock);
--- a/net/rxrpc/peer_object.c
+++ b/net/rxrpc/peer_object.c
@@ -147,8 +147,7 @@ struct rxrpc_peer *rxrpc_lookup_peer_rcu
  * assess the MTU size for the network interface through which this peer is
  * reached
  */
-static void rxrpc_assess_MTU_size(struct rxrpc_local *local,
-				  struct rxrpc_peer *peer)
+void rxrpc_assess_MTU_size(struct rxrpc_local *local, struct rxrpc_peer *peer)
 {
 	struct net *net = local->net;
 	struct dst_entry *dst;
@@ -160,6 +159,8 @@ static void rxrpc_assess_MTU_size(struct
 #endif
 
 	peer->if_mtu = 1500;
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
 
 	memset(&fl, 0, sizeof(fl));
 	switch (peer->srx.transport.family) {
@@ -199,6 +200,9 @@ static void rxrpc_assess_MTU_size(struct
 	peer->if_mtu = dst_mtu(dst);
 	dst_release(dst);
 
+	peer->mtu = peer->if_mtu;
+	peer->maxdata = peer->mtu - peer->hdrsize;
+
 	_leave(" [if_mtu %u]", peer->if_mtu);
 }
 
@@ -240,8 +244,6 @@ static void rxrpc_init_peer(struct rxrpc
 			    unsigned long hash_key)
 {
 	peer->hash_key = hash_key;
-	rxrpc_assess_MTU_size(local, peer);
-	peer->mtu = peer->if_mtu;
 	peer->rtt_last_req = ktime_get_real();
 
 	switch (peer->srx.transport.family) {
@@ -266,7 +268,6 @@ static void rxrpc_init_peer(struct rxrpc
 	}
 
 	peer->hdrsize += sizeof(struct rxrpc_wire_header);
-	peer->maxdata = peer->mtu - peer->hdrsize;
 }
 
 /*
@@ -285,6 +286,7 @@ static struct rxrpc_peer *rxrpc_create_p
 	if (peer) {
 		memcpy(&peer->srx, srx, sizeof(*srx));
 		rxrpc_init_peer(local, peer, hash_key);
+		rxrpc_assess_MTU_size(local, peer);
 	}
 
 	_leave(" = %p", peer);




From syzbot+c876adfab6362679008c at syzkaller.appspotmail.com  Sun Aug  9 17:23:26 2026
From: syzbot+c876adfab6362679008c at syzkaller.appspotmail.com (syzbot)
Date: Sun, 09 Aug 2026 17:23:26 -0700
Subject: [syzbot] [net?] [afs?] KASAN: slab-use-after-free Read in rxrpc_destroy_all_peers
Message-ID: <6a7919fe.9c11d2ce.289b96.00df.GAE@google.com>

Hello,

syzbot found the following issue on:

HEAD commit:    06cf61899d64 Merge tag 'x86-urgent-2026-08-08' of git://gi..
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=16765fb9580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=c44651ea7dd2f307
dashboard link: https://syzkaller.appspot.com/bug?extid=c876adfab6362679008c
compiler:       gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/260ea970a68e/disk-06cf6189.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/0005e8a7195f/vmlinux-06cf6189.xz
kernel image: https://storage.googleapis.com/syzbot-assets/f061ed2152ca/bzImage-06cf6189.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+c876adfab6362679008c at syzkaller.appspotmail.com

rxrpc: Leaked peer e1 {1} [0000:0000:0000:0000:0000:0000:0000:0001]:7003
==================================================================
BUG: KASAN: slab-use-after-free in rxrpc_destroy_all_peers+0x1ce/0x1d0 net/rxrpc/peer_object.c:461
Read of size 8 at addr ffff88803528cc20 by task kworker/u8:6/1029

CPU: 0 UID: 0 PID: 1029 Comm: kworker/u8:6 Tainted: G             L      syzkaller #0 PREEMPT(full) 
Tainted: [L]=SOFTLOCKUP
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/16/2026
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120
 print_address_description mm/kasan/report.c:378 [inline]
 print_report+0x13d/0x4b0 mm/kasan/report.c:482
 kasan_report+0xdf/0x1c0 mm/kasan/report.c:595
 rxrpc_destroy_all_peers+0x1ce/0x1d0 net/rxrpc/peer_object.c:461
 rxrpc_exit_net+0x83/0xc0 net/rxrpc/net_ns.c:114
 ops_exit_list net/core/net_namespace.c:199 [inline]
 ops_undo_list+0x2ee/0xab0 net/core/net_namespace.c:252
 cleanup_net+0x499/0x920 net/core/net_namespace.c:702
 process_one_work+0xa23/0x1940 kernel/workqueue.c:3322
 process_scheduled_works kernel/workqueue.c:3405 [inline]
 worker_thread+0x5ef/0xe50 kernel/workqueue.c:3486
 kthread+0x370/0x450 kernel/kthread.c:436
 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
 </TASK>

Allocated by task 5852:
 kasan_save_stack+0x30/0x50 mm/kasan/common.c:57
 kasan_save_track+0x14/0x30 mm/kasan/common.c:78
 poison_kmalloc_redzone mm/kasan/common.c:398 [inline]
 __kasan_kmalloc+0xaa/0xb0 mm/kasan/common.c:415
 kasan_kmalloc include/linux/kasan.h:263 [inline]
 __kmalloc_cache_noprof+0x2e5/0x6c0 mm/slub.c:5489
 _kmalloc_noprof include/linux/slab.h:988 [inline]
 _kzalloc_noprof include/linux/slab.h:1309 [inline]
 rxrpc_alloc_peer+0xa7/0x400 net/rxrpc/peer_object.c:229
 rxrpc_create_peer net/rxrpc/peer_object.c:293 [inline]
 rxrpc_lookup_peer+0x168/0x680 net/rxrpc/peer_object.c:354
 rxrpc_kernel_lookup_peer+0xa8/0xe0 net/rxrpc/af_rxrpc.c:285
 afs_merge_fs_addr6+0x186/0x460 fs/afs/addr_list.c:344
 afs_parse_text_addrs+0x3f2/0xa10 fs/afs/addr_list.c:222
 afs_alloc_cell fs/afs/cell.c:181 [inline]
 afs_lookup_cell+0xbb6/0x1940 fs/afs/cell.c:273
 afs_proc_cells_write+0x2a3/0x3a0 fs/afs/proc.c:125
 proc_simple_write+0x10e/0x1a0 fs/proc/generic.c:843
 pde_write fs/proc/inode.c:330 [inline]
 proc_reg_write+0x240/0x330 fs/proc/inode.c:342
 vfs_write+0x2aa/0x1050 fs/read_write.c:685
 ksys_write+0x12a/0x250 fs/read_write.c:739
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0x115/0x870 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

Freed by task 15:
 kasan_save_stack+0x30/0x50 mm/kasan/common.c:57
 kasan_save_track+0x14/0x30 mm/kasan/common.c:78
 kasan_save_free_info+0x3b/0x70 mm/kasan/generic.c:584
 poison_slab_object mm/kasan/common.c:253 [inline]
 __kasan_slab_free+0x5f/0x80 mm/kasan/common.c:285
 kasan_slab_free include/linux/kasan.h:235 [inline]
 slab_free_hook mm/slub.c:2677 [inline]
 __rcu_free_sheaf_prepare+0x5d/0x2f0 mm/slub.c:2928
 rcu_free_sheaf+0x1a/0xe0 mm/slub.c:5978
 rcu_do_batch kernel/rcu/tree.c:2645 [inline]
 rcu_core+0x5a2/0x10d0 kernel/rcu/tree.c:2897
 handle_softirqs+0x1ea/0x9b0 kernel/softirq.c:622
 run_ksoftirqd kernel/softirq.c:1076 [inline]
 run_ksoftirqd+0x38/0x60 kernel/softirq.c:1068
 smpboot_thread_fn+0x3d3/0xaa0 kernel/smpboot.c:160
 kthread+0x370/0x450 kernel/kthread.c:436
 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

The buggy address belongs to the object at ffff88803528cc00
 which belongs to the cache kmalloc-512 of size 512
The buggy address is located 32 bytes inside of
 freed 512-byte region [ffff88803528cc00, ffff88803528ce00)

The buggy address belongs to the physical page:
page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x3528c
head: order:2 mapcount:0 entire_mapcount:0 nr_pages_mapped:0 pincount:0
flags: 0xfff00000000040(head|node=0|zone=1|lastcpupid=0x7ff)
page_type: f5(slab)
raw: 00fff00000000040 ffff88813fe21c80 dead000000000100 dead000000000122
raw: 0000000000000000 0000000800100010 00000000f5000000 0000000000000000
head: 00fff00000000040 ffff88813fe21c80 dead000000000100 dead000000000122
head: 0000000000000000 0000000800100010 00000000f5000000 0000000000000000
head: 00fff00000000002 ffffffffffffff01 00000000ffffffff 00000000ffffffff
head: ffffffffffffffff 0000000000000000 00000000ffffffff 0000000000000004
page dumped because: kasan: bad access detected
page_owner tracks the page as allocated
page last allocated via order 2, migratetype Unmovable, gfp_mask 0xd20c0(__GFP_IO|__GFP_FS|__GFP_NOWARN|__GFP_NORETRY|__GFP_COMP|__GFP_NOMEMALLOC), pid 1, tgid 1 (swapper/0), ts 16783483369, free_ts 0
 set_page_owner include/linux/page_owner.h:32 [inline]
 post_alloc_hook+0xfd/0x120 mm/page_alloc.c:1859
 prep_new_page mm/page_alloc.c:1867 [inline]
 get_page_from_freelist+0xf48/0x3530 mm/page_alloc.c:3946
 __alloc_frozen_pages_noprof+0x299/0x2dc0 mm/page_alloc.c:5304
 alloc_slab_page mm/slub.c:3266 [inline]
 allocate_slab mm/slub.c:3380 [inline]
 new_slab+0xa2/0x640 mm/slub.c:3426
 refill_objects+0xe3/0x410 mm/slub.c:7310
 refill_sheaf mm/slub.c:2804 [inline]
 __pcs_replace_empty_main+0x376/0x680 mm/slub.c:4675
 alloc_from_pcs mm/slub.c:4773 [inline]
 slab_alloc_node mm/slub.c:4905 [inline]
 __do_kmalloc_node mm/slub.c:5333 [inline]
 __kmalloc_node_track_caller_noprof+0x679/0x830 mm/slub.c:5471
 __do_krealloc mm/slub.c:6837 [inline]
 krealloc_node_align_noprof+0x1f9/0x3e0 mm/slub.c:6858
 add_sysfs_param+0xdb/0x970 kernel/params.c:648
 kernel_add_sysfs_param kernel/params.c:797 [inline]
 param_sysfs_builtin kernel/params.c:836 [inline]
 param_sysfs_builtin_init+0x310/0x3f0 kernel/params.c:972
 do_one_initcall+0x11d/0x700 init/main.c:1347
 do_initcall_level init/main.c:1409 [inline]
 do_initcalls init/main.c:1425 [inline]
 do_basic_setup init/main.c:1445 [inline]
 kernel_init_freeable+0x6ea/0x7b0 init/main.c:1658
 kernel_init+0x1f/0x1e0 init/main.c:1548
 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
page_owner free stack trace missing

Memory state around the buggy address:
 ffff88803528cb00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
 ffff88803528cb80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>ffff88803528cc00: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                               ^
 ffff88803528cc80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff88803528cd00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller at googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup


From jarkko at kernel.org  Mon Aug 10 05:57:48 2026
From: jarkko at kernel.org (Jarkko Sakkinen)
Date: Mon, 10 Aug 2026 15:57:48 +0300
Subject: [PATCH net v5 06/10] keys: Add refcounting to user-defined key
 type payload
In-Reply-To: <20260729160108.2031453-7-dhowells@redhat.com>
References: <20260729160108.2031453-1-dhowells@redhat.com>
 <20260729160108.2031453-7-dhowells@redhat.com>
Message-ID: <annKzJSDhvHjk-wK@kernel.org>

On Wed, Jul 29, 2026 at 05:01:03PM +0100, David Howells wrote:
> Add refcounting to user-defined key type payload so that a kernel service
> wanting to use such a key can hold onto the payload without the RCU read
> lock held in order that it can do an allocation without having to be
> concerned with the key getting updated.
> 
> This is the first part of the fix for the AF_RXRPC challenge response
> generation code.
> 
> Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
> Signed-off-by: David Howells <dhowells at redhat.com>
> cc: Marc Dionne <marc.dionne at auristor.com>
> cc: Jeffrey Altman <jaltman at auristor.com>
> cc: Eric Dumazet <edumazet at google.com>
> cc: "David S. Miller" <davem at davemloft.net>
> cc: Jakub Kicinski <kuba at kernel.org>
> cc: Paolo Abeni <pabeni at redhat.com>
> cc: Simon Horman <horms at kernel.org>
> cc: Jarkko Sakkinen <jarkko at kernel.org>
> cc: linux-afs at lists.infradead.org
> cc: keyrings at vger.kernel.org
> cc: stable at kernel.org
> ---
>  include/keys/user-type.h     |  2 ++
>  net/dns_resolver/dns_key.c   |  1 +
>  security/keys/user_defined.c | 23 ++++++++++++++++-------
>  3 files changed, 19 insertions(+), 7 deletions(-)
> 
> diff --git a/include/keys/user-type.h b/include/keys/user-type.h
> index 386c31432789..7002a993a472 100644
> --- a/include/keys/user-type.h
> +++ b/include/keys/user-type.h
> @@ -26,6 +26,7 @@
>   */
>  struct user_key_payload {
>  	struct rcu_head	rcu;		/* RCU destructor */
> +	refcount_t	ref;
>  	unsigned short	datalen;	/* length of this data */
>  	char		data[] __aligned(__alignof__(u64)); /* actual data */
>  };
> @@ -37,6 +38,7 @@ struct key_preparsed_payload;
>  
>  extern int user_preparse(struct key_preparsed_payload *prep);
>  extern void user_free_preparse(struct key_preparsed_payload *prep);
> +void put_user_key_payload(struct user_key_payload *payload);
>  extern int user_update(struct key *key, struct key_preparsed_payload *prep);
>  extern void user_revoke(struct key *key);
>  extern void user_destroy(struct key *key);
> diff --git a/net/dns_resolver/dns_key.c b/net/dns_resolver/dns_key.c
> index c3c8c3240ef9..aa3c058f4095 100644
> --- a/net/dns_resolver/dns_key.c
> +++ b/net/dns_resolver/dns_key.c
> @@ -208,6 +208,7 @@ dns_resolver_preparse(struct key_preparsed_payload *prep)
>  		kleave(" = -ENOMEM");
>  		return -ENOMEM;
>  	}
> +	refcount_set(&upayload->ref, 1);
>  
>  	upayload->datalen = result_len;
>  	memcpy(upayload->data, data, result_len);
> diff --git a/security/keys/user_defined.c b/security/keys/user_defined.c
> index 6f88b507f927..90c1bd5d7dfe 100644
> --- a/security/keys/user_defined.c
> +++ b/security/keys/user_defined.c
> @@ -67,6 +67,7 @@ int user_preparse(struct key_preparsed_payload *prep)
>  	upayload = kmalloc_flex(*upayload, data, datalen);
>  	if (!upayload)
>  		return -ENOMEM;
> +	refcount_set(&upayload->ref, 1);
>  
>  	/* attach the data */
>  	prep->quotalen = datalen;
> @@ -88,12 +89,22 @@ EXPORT_SYMBOL_GPL(user_free_preparse);
>  
>  static void user_free_payload_rcu(struct rcu_head *head)
>  {
> -	struct user_key_payload *payload;
> +	struct user_key_payload *payload =
> +		container_of(head, struct user_key_payload, rcu);
>  
> -	payload = container_of(head, struct user_key_payload, rcu);
>  	kfree_sensitive(payload);
>  }
>  
> +/*
> + * Free a user defined key payload.
> + */
> +void put_user_key_payload(struct user_key_payload *payload)
> +{
> +	if (payload && refcount_dec_and_test(&payload->ref))
> +		call_rcu(&payload->rcu, user_free_payload_rcu);
> +}
> +EXPORT_SYMBOL_GPL(put_user_key_payload);
> +
>  /*
>   * update a user defined key
>   * - the key's semaphore is write-locked
> @@ -115,8 +126,7 @@ int user_update(struct key *key, struct key_preparsed_payload *prep)
>  	rcu_assign_keypointer(key, prep->payload.data[0]);
>  	prep->payload.data[0] = NULL;
>  
> -	if (zap)
> -		call_rcu(&zap->rcu, user_free_payload_rcu);
> +	put_user_key_payload(zap);
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(user_update);
> @@ -134,7 +144,7 @@ void user_revoke(struct key *key)
>  
>  	if (upayload) {
>  		rcu_assign_keypointer(key, NULL);
> -		call_rcu(&upayload->rcu, user_free_payload_rcu);
> +		put_user_key_payload(upayload);
>  	}
>  }
>  
> @@ -147,9 +157,8 @@ void user_destroy(struct key *key)
>  {
>  	struct user_key_payload *upayload = key->payload.data[0];
>  
> -	kfree_sensitive(upayload);
> +	put_user_key_payload(upayload);
>  }
> -
>  EXPORT_SYMBOL_GPL(user_destroy);
>  
>  /*
> 

It's quite solid change imho.

Reviewed-by: Jarkko Sakkinen <jarkko at kernel.org>

BR, Jarkko


From brauner at kernel.org  Mon Aug 10 06:12:23 2026
From: brauner at kernel.org (Christian Brauner)
Date: Mon, 10 Aug 2026 15:12:23 +0200
Subject: [PATCH v8 00/25] netfs: Keep track of folios in a segmented bio_vec[] chain
In-Reply-To: <20260804100224.2748935-1-dhowells@redhat.com>
References: <20260804100224.2748935-1-dhowells@redhat.com>
Message-ID: <20260810-luftdicht-popmusik-hypen-61dd8d9e3c06@brauner>

On Tue, 04 Aug 2026 11:01:55 +0100, David Howells wrote:
> Could you add these patches to the VFS tree for next?
> 
> The patches get rid of folio_queue, rolling_buffer and ITER_FOLIOQ,
> replacing the folio queue construct used to manage buffers in netfslib with
> one based around a segmented chain of bio_vec arrays instead.  There are
> three main aims here:
> 
> [...]

Applied to the vfs-7.3.netfs branch of the vfs/vfs.git tree.
Patches in the vfs-7.3.netfs branch should appear in linux-next soon.

Please report any outstanding bugs that were missed during review in a
new review to the original patch series allowing us to drop it.

It's encouraged to provide Acked-bys and Reviewed-bys even though the
patch has now been applied. If possible patch trailers will be updated.

Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. If in doubt, please check the listed branch.

tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
branch: vfs-7.3.netfs

[01/25] mm: Make readahead store folio count in readahead_control
        https://git.kernel.org/vfs/vfs/c/bb16bd52f872
[02/25] netfs: Bulk load the readahead-provided folios up front
        https://git.kernel.org/vfs/vfs/c/49fe2955409b
[03/25] Add a function to kmap one page of a multipage bio_vec
        https://git.kernel.org/vfs/vfs/c/71947db0b6b2
[04/25] iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
        https://git.kernel.org/vfs/vfs/c/fc0f8aa28cb0
[05/25] iov_iter: Add a segmented queue of bio_vec[]
        https://git.kernel.org/vfs/vfs/c/b7eab1a852c3
[06/25] netfs: Add some tools for managing bvecq chains
        https://git.kernel.org/vfs/vfs/c/a8e973126d3f
[07/25] netfs: Make mempool available for bvecq
        https://git.kernel.org/vfs/vfs/c/c95787e047bf
[08/25] netfs: Add a function to extract from an iter into a bvecq
        https://git.kernel.org/vfs/vfs/c/8854b625d977
[09/25] afs: Use a bvecq to hold dir content rather than folioq
        https://git.kernel.org/vfs/vfs/c/acb8eed74013
[10/25] cifs: Use a bvecq for buffering instead of a folioq
        https://git.kernel.org/vfs/vfs/c/b0d72aea6001
[11/25] smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
        https://git.kernel.org/vfs/vfs/c/27a6832d7b93
[12/25] netfs: Remove the writethrough code
        https://git.kernel.org/vfs/vfs/c/de4a0d855ae3
[13/25] cachefiles,netfs: sunset ondemand mode
        https://git.kernel.org/vfs/vfs/c/3b09163b7bed
[14/25] cachefiles: Don't rely on backing fs storage map for most use cases
        https://git.kernel.org/vfs/vfs/c/f3ac229c89a0
[15/25] netfs: Add the cache object ID to netfs_read/write tracepoints
        https://git.kernel.org/vfs/vfs/c/592472ff5a77
[16/25] netfs: Switch to using bvecq rather than folio_queue and rolling_buffer
        https://git.kernel.org/vfs/vfs/c/4c9ea0ae64f9
[17/25] smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter()
        https://git.kernel.org/vfs/vfs/c/c9529300bd22
[18/25] netfs: Remove netfs_alloc/free_folioq_buffer()
        https://git.kernel.org/vfs/vfs/c/bc89b7f32adb
[19/25] netfs: Remove netfs_extract_user_iter()
        https://git.kernel.org/vfs/vfs/c/e533d0d598c4
[20/25] iov_iter: Remove ITER_FOLIOQ
        https://git.kernel.org/vfs/vfs/c/ca929d6a6044
[21/25] netfs: Remove folio_queue and rolling_buffer
        https://git.kernel.org/vfs/vfs/c/6440cb5fd08b
[22/25] netfs: Check for too much data being read
        https://git.kernel.org/vfs/vfs/c/353f336ead13
[23/25] netfs: Limit the minimum trigger for progress reporting
        https://git.kernel.org/vfs/vfs/c/0150b23b0a63
[24/25] netfs: Combine prepare and issue ops and grab the buffers on request
        https://git.kernel.org/vfs/vfs/c/473527604b71
[25/25] cachefiles: Preset the state xattr when creating a new file
        https://git.kernel.org/vfs/vfs/c/9476c221814c


From syzbot+c876adfab6362679008c at syzkaller.appspotmail.com  Mon Aug 10 06:22:30 2026
From: syzbot+c876adfab6362679008c at syzkaller.appspotmail.com (syzbot)
Date: Mon, 10 Aug 2026 06:22:30 -0700
Subject: [syzbot] [afs?] [net?] KASAN: slab-use-after-free Read in rxrpc_destroy_all_peers
In-Reply-To: <6a7919fe.9c11d2ce.289b96.00df.GAE@google.com>
Message-ID: <6a79d096.9c11d2ce.289b96.00ed.GAE@google.com>

syzbot has found a reproducer for the following issue on:

HEAD commit:    db2ddb871435 Linux 7.2-rc7
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=152cd079580000
kernel config:  https://syzkaller.appspot.com/x/.config?x=c44651ea7dd2f307
dashboard link: https://syzkaller.appspot.com/bug?extid=c876adfab6362679008c
compiler:       gcc (Debian 14.2.0-19) 14.2.0, GNU ld (GNU Binutils for Debian) 2.44
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=17e35132580000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+c876adfab6362679008c at syzkaller.appspotmail.com

rxrpc: Leaked peer 4dfc {1} 127.0.0.1:8051
rxrpc: Leaked peer 5009 {1} 127.0.0.1:8167
rxrpc: Leaked peer 5726 {1} 127.0.0.1:8581
rxrpc: Leaked peer 5523 {0} 127.0.0.1:8374
==================================================================
BUG: KASAN: slab-use-after-free in rxrpc_destroy_all_peers+0x1ce/0x1d0 net/rxrpc/peer_object.c:461
Read of size 8 at addr ffff888054f70c20 by task kworker/u8:3/48

CPU: 0 UID: 0 PID: 48 Comm: kworker/u8:3 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/16/2026
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:94 [inline]
 dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120
 print_address_description mm/kasan/report.c:378 [inline]
 print_report+0x13d/0x4b0 mm/kasan/report.c:482
 kasan_report+0xdf/0x1c0 mm/kasan/report.c:595
 rxrpc_destroy_all_peers+0x1ce/0x1d0 net/rxrpc/peer_object.c:461
 rxrpc_exit_net+0x83/0xc0 net/rxrpc/net_ns.c:114
 ops_exit_list net/core/net_namespace.c:199 [inline]
 ops_undo_list+0x2ee/0xab0 net/core/net_namespace.c:252
 cleanup_net+0x499/0x920 net/core/net_namespace.c:702
 process_one_work+0xa23/0x1940 kernel/workqueue.c:3322
 process_scheduled_works kernel/workqueue.c:3405 [inline]
 worker_thread+0x5ef/0xe50 kernel/workqueue.c:3486
 kthread+0x370/0x450 kernel/kthread.c:436
 ret_from_fork+0x72b/0xd50 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
 </TASK>

Allocated by task 6240:
 kasan_save_stack+0x30/0x50 mm/kasan/common.c:57
 kasan_save_track+0x14/0x30 mm/kasan/common.c:78
 poison_kmalloc_redzone mm/kasan/common.c:398 [inline]
 __kasan_kmalloc+0xaa/0xb0 mm/kasan/common.c:415
 kasan_kmalloc include/linux/kasan.h:263 [inline]
 __kmalloc_cache_noprof+0x2e5/0x6c0 mm/slub.c:5489
 _kmalloc_noprof include/linux/slab.h:988 [inline]
 _kzalloc_noprof include/linux/slab.h:1309 [inline]
 rxrpc_alloc_peer+0xa7/0x400 net/rxrpc/peer_object.c:229
 rxrpc_create_peer net/rxrpc/peer_object.c:293 [inline]
 rxrpc_lookup_peer+0x168/0x680 net/rxrpc/peer_object.c:354
 rxrpc_kernel_lookup_peer+0xa8/0xe0 net/rxrpc/af_rxrpc.c:285
 afs_merge_fs_addr4+0x1cd/0x4c0 fs/afs/addr_list.c:300
 afs_parse_text_addrs+0x655/0xa10 fs/afs/addr_list.c:220
 afs_alloc_cell fs/afs/cell.c:181 [inline]
 afs_lookup_cell+0xbb6/0x1940 fs/afs/cell.c:273
 afs_proc_cells_write+0x2a3/0x3a0 fs/afs/proc.c:125
 proc_simple_write+0x10e/0x1a0 fs/proc/generic.c:843
 pde_write fs/proc/inode.c:330 [inline]
 proc_reg_write+0x240/0x330 fs/proc/inode.c:342
 vfs_write+0x2aa/0x1050 fs/read_write.c:685
 ksys_write+0x12a/0x250 fs/read_write.c:739
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0x115/0x870 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

Freed by task 16:
 kasan_save_stack+0x30/0x50 mm/kasan/common.c:57
 kasan_save_track+0x14/0x30 mm/kasan/common.c:78
 kasan_save_free_info+0x3b/0x70 mm/kasan/generic.c:584
 poison_slab_object mm/kasan/common.c:253 [inline]
 __kasan_slab_free+0x5f/0x80 mm/kasan/common.c:285
 kasan_slab_free include/linux/kasan.h:235 [inline]
 slab_free_hook mm/slub.c:2677 [inline]
 __rcu_free_sheaf_prepare+0x5d/0x2f0 mm/slub.c:2928
 rcu_free_sheaf+0x1a/0xe0 mm/slub.c:5978
 rcu_do_batch kernel/rcu/tree.c:2645 [inline]
 rcu_core+0x5a2/0x10d0 kernel/rcu/tree.c:2897
 handle_softirqs+0x1ea/0x9b0 kernel/softirq.c:622
 __do_softirq kernel/softirq.c:656 [inline]
 invoke_softirq kernel/softirq.c:496 [inline]
 __irq_exit_rcu+0x162/0x210 kernel/softirq.c:735
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:752
 instr_sysvec_call_function_single arch/x86/kernel/smp.c:267 [inline]
 sysvec_call_function_single+0xa3/0xc0 arch/x86/kernel/smp.c:267
 asm_sysvec_call_function_single+0x1a/0x20 arch/x86/include/asm/idtentry.h:681

The buggy address belongs to the object at ffff888054f70c00
 which belongs to the cache kmalloc-512 of size 512
The buggy address is located 32 bytes inside of
 freed 512-byte region [ffff888054f70c00, ffff888054f70e00)

The buggy address belongs to the physical page:
page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x54f70
head: order:2 mapcount:0 entire_mapcount:0 nr_pages_mapped:0 pincount:0
flags: 0xfff00000000040(head|node=0|zone=1|lastcpupid=0x7ff)
page_type: f5(slab)
raw: 00fff00000000040 ffff88813fe21c80 dead000000000100 dead000000000122
raw: 0000000000000000 0000000800100010 00000000f5000000 0000000000000000
head: 00fff00000000040 ffff88813fe21c80 dead000000000100 dead000000000122
head: 0000000000000000 0000000800100010 00000000f5000000 0000000000000000
head: 00fff00000000002 ffffffffffffff01 00000000ffffffff 00000000ffffffff
head: ffffffffffffffff 0000000000000000 00000000ffffffff 0000000000000004
page dumped because: kasan: bad access detected
page_owner tracks the page as allocated
page last allocated via order 2, migratetype Unmovable, gfp_mask 0xd20c0(__GFP_IO|__GFP_FS|__GFP_NOWARN|__GFP_NORETRY|__GFP_COMP|__GFP_NOMEMALLOC), pid 6240, tgid 6240 (syz-executor166), ts 101070694469, free_ts 79760049897
 set_page_owner include/linux/page_owner.h:32 [inline]
 post_alloc_hook+0xfd/0x120 mm/page_alloc.c:1859
 prep_new_page mm/page_alloc.c:1867 [inline]
 get_page_from_freelist+0xf48/0x3530 mm/page_alloc.c:3946
 __alloc_frozen_pages_noprof+0x299/0x2dc0 mm/page_alloc.c:5304
 alloc_slab_page mm/slub.c:3266 [inline]
 allocate_slab mm/slub.c:3380 [inline]
 new_slab+0xa2/0x640 mm/slub.c:3426
 refill_objects+0xe3/0x410 mm/slub.c:7310
 refill_sheaf mm/slub.c:2804 [inline]
 __pcs_replace_empty_main+0x376/0x680 mm/slub.c:4675
 alloc_from_pcs mm/slub.c:4773 [inline]
 slab_alloc_node mm/slub.c:4905 [inline]
 __kmalloc_cache_noprof+0x479/0x6c0 mm/slub.c:5485
 _kmalloc_noprof include/linux/slab.h:988 [inline]
 _kzalloc_noprof include/linux/slab.h:1309 [inline]
 rxrpc_alloc_peer+0xa7/0x400 net/rxrpc/peer_object.c:229
 rxrpc_create_peer net/rxrpc/peer_object.c:293 [inline]
 rxrpc_lookup_peer+0x168/0x680 net/rxrpc/peer_object.c:354
 rxrpc_kernel_lookup_peer+0xa8/0xe0 net/rxrpc/af_rxrpc.c:285
 afs_merge_fs_addr4+0x1cd/0x4c0 fs/afs/addr_list.c:300
 afs_parse_text_addrs+0x655/0xa10 fs/afs/addr_list.c:220
 afs_alloc_cell fs/afs/cell.c:181 [inline]
 afs_lookup_cell+0xbb6/0x1940 fs/afs/cell.c:273
 afs_proc_cells_write+0x2a3/0x3a0 fs/afs/proc.c:125
 proc_simple_write+0x10e/0x1a0 fs/proc/generic.c:843
 pde_write fs/proc/inode.c:330 [inline]
 proc_reg_write+0x240/0x330 fs/proc/inode.c:342
page last free pid 5674 tgid 5674 stack trace:
 reset_page_owner include/linux/page_owner.h:25 [inline]
 __free_pages_prepare mm/page_alloc.c:1406 [inline]
 free_pages_prepare+0x586/0xd80 mm/page_alloc.c:1451
 __free_contig_range_common+0x14f/0x250 mm/page_alloc.c:6897
 __free_contig_range mm/page_alloc.c:6942 [inline]
 free_pages_bulk+0x12a/0x200 mm/page_alloc.c:5257
 vm_area_free_pages+0xad/0x2b0 mm/vmalloc.c:3461
 vfree mm/vmalloc.c:3510 [inline]
 vfree+0x107/0x750 mm/vmalloc.c:3484
 kcov_put kernel/kcov.c:447 [inline]
 kcov_put kernel/kcov.c:443 [inline]
 kcov_close+0x34/0x60 kernel/kcov.c:554
 __fput+0x3ff/0xb50 fs/file_table.c:512
 task_work_run+0x150/0x240 kernel/task_work.c:233
 exit_task_work include/linux/task_work.h:40 [inline]
 do_exit+0x951/0x2ae0 kernel/exit.c:1009
 do_group_exit+0xd5/0x2a0 kernel/exit.c:1152
 get_signal+0x1ec7/0x21e0 kernel/signal.c:3046
 arch_do_signal_or_restart+0x91/0x7e0 arch/x86/kernel/signal.c:337
 __exit_to_user_mode_loop kernel/entry/common.c:66 [inline]
 exit_to_user_mode_loop+0x139/0x6f0 kernel/entry/common.c:101
 __exit_to_user_mode_prepare include/linux/irq-entry-common.h:207 [inline]
 syscall_exit_to_user_mode_prepare include/linux/irq-entry-common.h:230 [inline]
 syscall_exit_to_user_mode include/linux/entry-common.h:318 [inline]
 do_syscall_64+0x666/0x870 arch/x86/entry/syscall_64.c:100
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

Memory state around the buggy address:
 ffff888054f70b00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
 ffff888054f70b80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>ffff888054f70c00: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                               ^
 ffff888054f70c80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff888054f70d00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================


---
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.


From brauner at kernel.org  Mon Aug 10 06:24:52 2026
From: brauner at kernel.org (Christian Brauner)
Date: Mon, 10 Aug 2026 15:24:52 +0200
Subject: [PATCH v3 0/2] vfs, afs, bash: Fix miscomparison of foreign
 user IDs in the VFS
In-Reply-To: <20260728141344.1811120-1-dhowells@redhat.com>
References: <20260728141344.1811120-1-dhowells@redhat.com>
Message-ID: <20260810-jazzfest-unglaubhaft-miene-4d1cf4cf42d3@brauner>

On 2026-07-28 15:13 +0100, David Howells wrote:
> Hi Al, Christian,
> 
> Here's a pair of fixes that deal with some places the VFS mishandles
> foreign user ID checks.  By "foreign" I mean that the user IDs from the
> filesystem do not belong in the same number space as the system's user IDs.
> Network filesystems are prime examples of this, but it may also impact
> things like USB drives or cdroms.
> 
> Take AFS as example: Whilst each file does have a numeric user ID, the file
> may be accessed from a world-accessible public-facing server from some
> other organisation with its own idea of what that user ID refers to.  IDs
> from AFS may also collide with the system's own set of IDs and may also be
> unrepresentable as a 32-bit UID (in the case of AuriStor servers).
> 
> Further, kAFS uses a key containing an authentication token to specify the
> subject doing an RPC operation to the server - and, as such, this needs to
> be used instead of current_fsuid() in determining whether the current user
> has ownership rights over a file.
> 
> Additionally, filesystems (CIFS being a notable example) may also have user
> identifiers that aren't simple integers.
> 
> Now the problem in the VFS is that there are a number of places where it
> assumes it can directly compare i_uid (possibly id-mapped) to either than
> on another inode or a UID drawn from elsewhere (e.g. current_uid()) - but
> this doesn't work right.
> 
> This causes the write-to-sticky check to work incorrectly for AFS (though
> this is currently masked by a workaround in bash that has now been removed
> as of bash-5.3) whereby open(O_CREAT) of such a file will fail when it
> shouldn't.
> 
> Two patches are provided:
> 
>  (1) Add a pair of inode operations, one to compare the ownership of a pair
>      of inodes and the other to see if the current process has ownership
>      rights over an inode with an IOP flag to control access to them.
> 
>      Usage of this is then extended out into the VFS, replacing comparisons
>      between i_uid and i_uid and between i_uid and current_fsuid().  The
>      default, it the inode ops are unimplemented, is to do those direct i_uid
>      comparisons.
> 
>  (2) Fixes the bash workaround issue with regard to AFS, overriding the
>      checks as to whether two inodes have the same owner and the check as
>      to whether the current user owns an inode to work within the AFS
>      model.
> 
> kAFS uses the result of a status-fetch with a suitable key to determine
> file ownership (if the ADMINISTER bit is set) and just compares the 64-bit
> owner IDs to determine if two inodes have the same ownership.
> 
> Note that chown may also need modifying in some way - but that can't
> necessarily supply the information required (for instance, an AuriStor YFS ID
> is 64 bits, but chown can only handle a 32-bit integer; CIFS might use a
> GUID).

Seems that some other changes are required for this. I'll take a look
once a new version is out.



From dhowells at redhat.com  Mon Aug 10 07:47:17 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:17 +0100
Subject: [PATCH v9 00/26] netfs: Keep track of folios in a segmented bio_vec[] chain
Message-ID: <20260810144746.574036-1-dhowells@redhat.com>

Hi Christian,

Could you add these patches to the VFS tree for next?  Though, given it may
be a little late for that, could you at least take the fix patch at the
front?

The patches get rid of folio_queue, rolling_buffer and ITER_FOLIOQ,
replacing the folio queue construct used to manage buffers in netfslib with
one based around a segmented chain of bio_vec arrays instead.  There are
three main aims here:

 (1) The kernel file I/O subsystem seems to be moving towards consolidating
     on the use of bio_vec arrays, so embrace this by moving netfslib to
     keep track of its buffers for buffered I/O in bio_vec[] form.

 (2) Netfslib already uses a bio_vec[] to handle unbuffered/DIO, so the
     number of different buffering schemes used can be reduced to just a
     single one.

 (3) Always send an entire filesystem RPC request message to a TCP socket
     with single kernel_sendmsg() call as this is faster, more efficient
     and doesn't require the use of corking as it puts the entire
     transmission loop inside of a single tcp_sendmsg().

For the replacement of folio_queue, a segmented chain of bio_vec arrays
rather than a single monolithic array is provided:

	struct bvecq {
		struct bvecq		*next;
		struct bvecq		*prev;
		unsigned long long	fpos;
		refcount_t		ref;
		u32			priv;
		u16			nr_slots;
		u16			max_slots;
		enum bvecq_mem		mem_type:2;
		bool			inline_bv:1;
		bool			discontig:1;
		bool			from_pool:1;
		struct bio_vec		*bv;
		struct bio_vec		__bv[];
	};

The fields are:

 (1) next, prev - Link segments together in a list.  I want this to be
     NULL-terminated linear rather than circular to make it possible to
     arbitrarily glue bits on the front.

 (2) fpos, discontig - Note the current file position of the first byte of
     the segment and whether this bvecq is discontiguous with the previous.
     When accessing the pagecache to clear flags/locks, the fpos can be
     used to look up folios by file position rather than by finding those
     folios from the info stored in the bio_vecs.

     When the file position is relevant, the model I'm working with is that
     all the segments pointed to by a single bvecq must represent
     contiguous data, but adjacent bvecqs within a chain need not be
     contiguous.  This allows a bvecq chain to be used to provide bufferage
     for a sparse read or write RPC such as can be done with Ceph.

     If a bvecq segment is not contiguous with the previous one,
     ->discontig should be set (this is technically redundant if one keeps
     track of the fpos as a bvecq chain is processed).

     Note that the beginning and end file positions in a segment need not
     be aligned to any filesystem block size.

 (3) ref - Refcount.  Each bvecq keeps a ref on the next.  I'm not sure
     this is entirely necessary, but it makes sharing slices easier.

 (4) priv - Private data for the owner.  Dispensible; currently only used
     for storing a debug ID for tracing in a patch not included here.

 (5) max_slots, nr_slots.  The size of bv[] and the number of slots used.
     I've assumed a maximum of 65535 bio_vecs in the array (which would
     represent a ~1MiB allocation).

 (6) bv, __bv, inline_bv.  bv points to the bio_vec[] array handled by
     this segment.  This may begin at __bv and if it does inline_bv should
     be set (otherwise it's impossible to distinguish a separately
     allocated bio_vec[] that follows immediately by coincidence).

 (7) mem_type.  Indicates how the memory attached to the bio_vecs should be
     disposed of when the bvecq is destroyed.  It can be one of:

	BVECQ_MEM_EXTERNAL	- Externally tracked ref; don't put
	BVECQ_MEM_PAGECACHE	- Pagecache; must be put
	BVECQ_MEM_GUP		- Pinned by from GUP; needs unpin
	BVECQ_MEM_ALLOCED	- Plain alloc'd pages; can be mempooled

     [!] I'm not sure that this is a good name for this member or for the
     	 enum values.

 (8) from_pool.  Set if the bvecq is allocated from netfslib's mempool and
     should be freed to it.

I've also defined an iov_iter iterator type ITER_BVECQ to walk this sort of
construct so that it can be passed directly to sendmsg() or block-based DIO
(as cachefiles does).


This series makes the following changes to netfslib:

 (1) Remove the writethrough code as the locking is really tricky to get
     right and it looks like it could deadlock with Ceph if snapshots are
     used.

 (2) The folio_queue chain used to hold folios for buffered I/O is replaced
     with a bvecq chain.  Each bio_vec then holds (a portion of) one folio.
     Each bvecq holds a contiguous sequence of folios, but adjacent bvecqs
     in a chain may be discontiguous.

 (3) For unbuffered/DIO, the source iov_iter is extracted into a bvecq
     chain.

 (4) An abstract position representation ('bvecq_pos') is created that can
     used to hold a position in a bvecq chain.  For the moment, this takes
     a ref on the bvecq it points to, but that may be excessive.

 (5) Buffer tracking is managed with three cursors:  The load_cursor, at
     which new folios are added as we go; the dispatch_cursor, at which new
     subrequests' buffers start when they're created; and the
     collect_cursor, the point at which folios are being unlocked.

     Not all cursors are necessarily needed in all situations and during
     buffered writeback, we need a dispatch cursor per stream (one for the
     network filesystem and one for the cache).

 (6) ->prepare_read(), buffer setting up and ->issue_read() are merged, as
     are the write variants, with the filesystem calling back up to
     netfslib to prepare its buffer.  This simplifies the process of
     setting up a subrequest.  It may even make sense to have the
     filesystem allocate the subrequest.

 (7) Retry dispatch tracking is added to netfs_io_request so that the
     buffer preparation functions can find it.  Retry requires an
     additional buffer cursor.

 (8) Netfslib dispatches I/O by accumulating enough bufferage to dispatch
     at least one subrequest, then looping to generate as many as the
     filesystem wants to (they may be limited by other constraints,
     e.g. max RDMA segment count or negotiated max size).  This loop could
     be moved down into the filesystem.  A new method is provided by which
     netfslib can ask the filesystem to provide an estimate of the data
     that should be accumulated before dispatch begins.

 (9) Reading from the cache is now managed by querying the cache to provide
     a list of the next two data extents within the cache.

(10) AFS directories are switched to using a bvecq rather than a
     folio_queue to hold their contents.

(11) CIFS is switch to using a bvecq rather than a folio_queue for holding
     a temporary encryption buffer.

(12) CIFS RDMA is given the ability to extract ITER_BVECQ and support for
     extracting ITER_FOLIOQ is removed.

(13) All the folio_queue and rolling_buffer code is removed.

Cachefiles is also modified:

 (1) On-demand mode for Erofs is removed.

 (2) The object type in the cachefiles file xattr is now correctly set to
     CACHEFILES_CONTENT_{SINGLE,ALL,BACKFS_MAP} rather than just being 0,
     to indicate whether we have a single monolithic blob, all the data up
     to cache i_size with no holes or a sparse file with the data mapped by
     the backing file system (as currently upstream).

 (3) For "ALL" type files, the cache's i_size is used to track how much
     data is saved in the cache and no longer bears any relation to the
     netfs i_size.  The actual object size is stored in the xattr.

 (4) For most typical files which are contiguous and written progressively,
     the object type is now set to "ALL".  For anything else, cachefiles
     uses SEEK_DATA/HOLE to find extent outlines at before (this is the
     current behaviour and needs to be fixed, but in a separate set of
     patches as it's not trivial).

 (5) Preset the xattr on a cachefile to try to avoid having to deal with
     ENOSPC from setxattr when committing the object.

Two further things that I'm working on (but not in this branch) are:

 (1) Make it so that a filesystem can be given a copy of a subchain which
     it can then tack header and trailer protocol elements upon to form a
     single message (I have this working for cifs) and even join copies
     together with intervening protocol elements to form compounds.

 (2) Make it so that a filesystem can 'splice' out the contents of the TCP
     receive queue into a bvecq chain.  This allows the socket lock to be
     dropped much more quickly and the copying of data read to the
     destination buffers to happen without the lock.  I have this working
     for cifs too.  Kernel recvmsg() doesn't then block kernel sendmsg()
     for anywhere near as long.

There are also some things I want to consider for the future:

 (1) Create one or more batched iteration functions to 'unlock' all the
     folios in a bio_vec[], where 'unlock' is the appropriate action for
     ending a read or a write.  Batching should hopefully also improve the
     efficiency of wrangling the marks on the xarray.  Very often these
     marks are going to be represented by contiguous bits, so there may be
     a way to change them in bulk.

 (2) Rather than walking the bvecq chain to get each individual folio out
     via bv_page, use the file position stored on the bvecq and the sum of
     bv_len to iterate over the appropriate range in i_pages.

 (3) Change iov_iter to store the initial starting point and for
     iov_iter_revert() to reset to that and advance.  This would (a) help
     prevent over-reversion and (b) dispense with the need for a prev
     pointer.

 (4) Use bvecq to replace scatterlist.  One problem with replacing
     scatterlist is that crypto drivers like to glue bits on the front of
     the scatterlists they're given (something trivial with that API) - and
     this is one way to achieve it.

The patches can also be found here:

	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-next

Thanks,
David

Changes
=======
ver #9)
- Fixed a number of issues reported by Sashiko[7]:
  - Fixed netfs_unlock_abandoned_read_pages() to make sure the refs are
    dropped before unlocking.
  - Fixed iov_iter_extract_bvecq_pages() to immediately return 0 if
    there are no pages to be extracted from the iterator (e.g. if maxpages
    is 0).
  - Pass a flag into bvecq_alloc_one() to indicate if we can access
    writeback memory pools rather than trying to divine this from gfp.
  - Fixed bvecq_alloc_one() to use the same gfp adjustments as
    mempool_alloc() when prospectively allocating what the caller asked for
    during writeback.
  - Simplified the bvecq-based rolling buffer in writeback by always adding
    whole folios and then setting the offset appropriately.  The offset and
    length of the part added can then be aligned at that point when
    tracking the new folio in the cache stream rather than copying and
    rounding out later.
  - Changed the writeback loop to ignore ENOMEM.  There are now backup
    memory pools for writeback.
  - Simplified the writeback loop to update stream tracking as folios are
    added and to flush at that point too if needed.
  - Removed the warning given by netfs_extract_iter() if max_len or
    max_pages is 0.
  - Made netfs_extract_iter() set the fpos on the blank bvecq it allocates
    in the alloc_empty path.
  - Fixed netfs_writeback_single() to test NETFS_RREQ_UPLOAD_TO_SERVER
    rather than test-and-setting it (nothing currently wants to
    writeback-single to the server anyway, only to the cache).
  - Removed cachefiles ondemand pincount.
  - Fixed netfs_collect_write_results() to get the cancel/data flags to
    netfs_collect_write_results() the right way round.
  - Fixed netfs_writeback_unlock_folios() to just warn and oops in the
    unlikely event that it sees a NULL (ie. already unlocked) folio
    pointer.
  - Moved some common initialisation of fields to rreq->start to
    netfs_alloc_request().
  - Fixed bvecq_load_from_ra() to set bq->fpos.
  - Fixed read progress reporting to use an offset from rreq->start
    (unchanging) rather than trying to use rreq->clean_to (tearable) as a
    base.

ver #8)
- Removed the fix patches to their own branches and posted separately.
- Rebased on v7.2-rc6 as a number of fixes went in.
- Fixed cachefiles_collect_write() to handle cres->cache_priv2 (file) being
  NULL due to failure to create an object (e.g. due to ENOSPC).
- Added a patch to cachefiles to preset the xattr on a cachefiles to avoid
  dealing with ENOSPC from setxattr when committing an object.
- Fixed a number of bugs reported by Sashiko[6]:
  - Fixed bvecq_alloc_buffer2() to break out of the loop if pre_slots > 0
    and size == 0.
  - Made netfs_extract_iter() limit max_pages to protect iov_iter_npages().
  - Remove include/uapi/linux/cachefiles.h when ondemand mode is removed.
  - Remove more ondemand-related trace bits.
  - Remove ondemand-related BPF bits.
  - Fixed a missing ENOMEM check in netfs_pgpriv2_begin_copy_to_cache().
  - Fixed netfs_retry_read_subrequests() to unset dispatch_cursor on the
    "abandon_after" error path.
  - Fixed netfs_issue_write() to clear stream->construct if a subreq is
    terminated due to ENOMEM.
  - Fixed netfs_collect_read_results() to unpause the app if it decides to
    abandon the read.
  - Fixed cachefiles_issue_read() to not wait for the object state to match
    FSCACHE_WANT_READ as cachefiles_query_occupancy() should have done that
    already.
  - Fixed netfs_retry_write_stream() to handle superfluous subreqs
    correctly (borrowing from read retry).

ver #7)
- Rebased on v7.2-rc4.
- Added three fix patches to the front from Yichong Chen that conflict with
  other patches in the series.
- Fixed a number of bugs reported by Sashiko[5].
  - Fixed double page put in iov_kunit_destroy_bvecq().
  - Fixed bvecq_alloc_buffer2() to handle pre_slots>0 and size==0
    correctly.
  - Fixes bvecq_expand_buffer() to use a barrer to set tail->nr_slots.
  - Added functions to insert barriers when reading or setting bvecq::next
    and fixed some places to use them.
  - Fixed more meta block mapping in afs_edit_dir_remove().
  - Added a patch to fix missing unmap in afs_dir_search_bucket().
  - Removed the writethrough stat counter as the code gets removed.
  - Cleaned up the cachefiles-ondemand removal patch:
    - Fix removal of code that's conditional on !ONDEMAND.
    - Removed msg_id_next field.
    - Removed ondemand trace elements.
    - Removed documentation.
  - Fixed cachefiles_resize_cookie() to only consider a resize a shrink if
    the new size is less than the object size.
  - Fixed cachefiles_resize_cookie() to update object->object_size on
    expansion.
  - Add a comment on exclusion in cachefiles_collect_write().
  - Fixed netfs_single_dispatch_read() to allow for the cache to have
    rounded out the stored data to DIO size.
  - Fixed netfs_prepare_read_iterator() to clamp subreq len to sreq_max_len
    (obsoleted by last patch).
  - Changed write subreq collection to never retry cache writes.  Shouldn't
    see them in the retry path either then.
  - Fixed netfs_pgpriv2_unlock_copied_folios() to return made_progress
    instead of false if we hit the end of the bvecq chain.
  - Modified buffered read collection to abandon the rest of a read request
    if a subreq over-reads as the buffer may have been corrupted.  The read
    termination function no longer winds subreq->transferred back.
  - Fixed netfs_issue_read() to call the right termination func in the fail
    path.
  - Fixed netfs_dispatch_unbuffered_reads(), netfs_single_dispatch_read()
    and netfs_retry_read_subrequests() to handle failed ->issue_read().
  - Fixed afs_issue_read() to return after calling afs_end_read() as this
    op is then released.
  - Fixed cifs_issue_write() to always release credits on failure.
  - Fixed cifs_issue_read() to not release credits twice.
  - Fixed netfs_issue_writes() to advance stream->issued_to and
    stream->dispatch_cursor on early failure of ->issue_write().
  - Fixed netfs_issue_read() to only call netfs_all_subreqs_queued() if
    stream->buffered is zero.
  - Fixed ceph_netfs_issue_op_inline() to do a retry if file got uninlined
    by someone else.
  - Fixed ceph_netfs_issue_op_inline() to not return an error directly
    once it's got a prepared buffer.
  - Fixed nfs_netfs_issue_read() to return an int.

ver #6)
- Rebased on v7.2-rc3.
- Added a patch to remove erofs on-demand support from cachefiles.
- Added a patch to remove the writethrough code as it's really tricky to
  get the locking right and it looks like can deadlock itself with Ceph.
- Fixed a number of bugs reported by Sashiko[4].
  - Fixed iov_iter_single_seg_count(), iov_iter_alignment_bvecq() and
    iov_npages_bvecq() to handle zero-length segments and segments with no
    stuff remaining.
  - Fixed bvecq_extract() to use GFP_NOFS rather than GFP_KERNEL.
  - Fixed bvecq_alloc_buffer2()'s use of alloc_pages_bulk().
  - Fixed netfs_extract_iter() to break out of the inner loop if max_pages
    hits zero.
  - Added a patch to fix meta block mapping in afs_edit_dir_remove().
  - Fixed afs_init_new_symlink() to clear the tail of the symlink page.
  - Fixed cachefiles_begin_operation() to deal with a NULL file.
  - Fixed slot check in netfs_read_gaps() to be >=, not >.
  - Fixed netfs_pgpriv2_copy_folio() to update creq->last_end.
  - Fixed netfs_read_single() to return -EIO if the buffer isn't big
    enough.
  - Add a comment to netfs_write_folio() to indicate that truncate must
    not run concurrently with writeback and must exclude/wait for it.
  - In netfs_read_subreq_terminated(), move the netfs_sreq_trace_too_much
    trace line earlier before ->transferred is clobbered.
  - In netfs_read_subreq_terminated(), make the transfer-too-long case
    avoid retrying and just end with EIO (in case data further along in the
    buffer got corrupted).
  - Fixed netfs_collect_read_results() if it sees front->transferred >
    front->len, to just use front->len instead.
  - Fixed ceph_netfs_issue_op_inline() to prep the buffer before setting
    NETFS_SREQ_HIT_EOF so that subreq->content is set.
  - Alter netfs_read_to_pagecache() to not round up reading from
    zero-containing cache granules as no I/O is required.
  - Made ->issue_write() return an error so that errors before any buffer
    is sliced off don't result in an infinite loop in netfs_issue_writes().
  - Made ->issue_read() return an error also for similiar reasons.
  - In netfs_writepages(), handle -ENOMEM from bvecq prealloc before
    caling netfs_queue_wb_folio() rather than doing it on the combined
    error return of that and netfs_issue_streams() which can result in a
    double unlock.
  - Fix netfs_writeback_single() to write nothing if len == 0.

ver #5)
- Rebased on v7.2-rc2 as that has a bunch of outstanding netfs and afs
  bugfixes included.

ver #4)
- Fixed a number of bugs reported by Sashiko[3].
  - Added a patch to fix an underflow in iov_iter_extract_xarray_pages().
  - Added a patch to fix alloc failure in iov_iter_extract_bvec_pages().
  - Added a patch to remove an unused var in kunit code.
  - Added a patch to fix the folio offset in extract_xarray_to_sg().
  - Added a patch to fix the exclusion over writeback to make it cover
    collection too.
  - Fixed double fput() in cachefiles.
  - Fixed the collection of cache writes to handle cancellation better.
  - Fixed iterate_bvecq() to skip bvecq structs with nr_slots==0.
  - Add a comment into iterate_bvecq() that a slot with bv_len>0 must have a
    valid bv_page.
  - Fixed iov_iter_bvecq_advance(), iov_iter_bvecq_revert(),
    iter_count_bvecq_pages(), iov_iter_extract_bvecq_pages() and
    extract_bvecq_to_sg() to correctly handle empty bvecqs.
  - Fixed extract_bvecq_to_sg() to be limited by iter->count.
  - Fixed bvecq_expand_buffer() to take an unsigned size param.
  - Fixed bvecq_expand_buffer() to not mix memory types in alloc'd bvecqs.
  - Fixed bvecq_shorten_buffer() occasional retention of zero-length slots.
  - Fixed slot validity check polarity in bvecq_pos_advance(); also don't use
    inner loop otherwise break then exits the wrong loop.
  - Fixed bvecq_zero(), bvecq_slice() and bvecq_extract to use a barrier when
    checking bq->nr_slots.
  - Restructured bvecq_zero() to be similar to bvecq_pos_advance().
  - Fixed an off-by-one error in bvecq_pos_step() and added a missing slot
    reset.
  - Fixed a break in netfs_extract_iter() that should have been a goto.
  - Fixed netfs_extract_iter() to limit number of pages extracted to remnant
    of max_pages.
  - Fixed an uninit var in afs_do_read_symlink().
  - Fixed netfs_read_gaps() to fill a multipart bvecq chain correctly.
  - Fixed netfs_dispatch_unbuffered_reads() to initialise collect_cursor as
    netfs_rreq_assess_dio() uses it to flush the data read.
  - Fixed netfs_extract_iter() to init the slot counter outside the extract
    loop to avoid overwriting already loaded slots.
  - Fixed callers of bvecq_delete_spent() to update bvecq_pos::slot before
    calling.
  - Fixed netfs_reissue_write() to make sure subreq->content is unset before
    setting.
  - Altered netfs_extract_iter() to free any allocated bvecq chain if no pages
    were extracted and an error occurred  (and to initialise the return
    pointer to NULL).  Also, made it return an empty bvecq if nothing was
    extracted, but no error occurred.
  - Fixed ceph_netfs_issue_read() to just return if
    ceph_netfs_issue_op_inline() returns anything other than 1 to avoid a
    double termination.
  - Fixed ceph_netfs_issue_read() to do the size calculation in the right
    order to avoid the op expanding to larger than the buffer.
  - Fixed netfs_issue_read(), in the NETFS_FILL_WITH_ZEROES case, to deduct
    subreq->len from stream->buffered rather than just setting it to 0.
  - Fixed netfs_perform_write() to put the folio if netfs_advance_writethrough()
    fails.
  - Restored the old ->prepare_write op specifically for
    fscache_write_to_cache() which is still used by Ceph.
  - Fixed undefined return in netfs_pgpriv2_issue_stream().
  - Fixed netfs_collect_write_results() to try to make sure a request isn't
    left paused if there are no further server-bound subreqs.
  - Fixed netfs_queue_wb_folio() to redirty the folio before unlocking it if
    it can't allocate a bvecq.
  - Fixed netfs_writepages() to cancel the pagecache iteration after ENOMEM.
  - Fixed netfs_advance/end_writethrough() to advance the dispatch cursor.
  - Fixed netfs_retry_read_subrequests() to use barriers when walking
    stream->subrequests as the app may add another subreq before pausing.
  - Fixed netfs_prepare_write_retry_buffer() to use ->retry_start and
    ->retry_buffered rather than ->issue_from and ->buffered.
  - Fixed netfs_retry_write_stream() to use barriers when walking
    stream->subrequests as the app may add another subreq before pausing.
  - Fixed netfs_retry_write_stream() to check the correct length when adding
    additional subreqs.
  - Fixed nfs_netfs_issue_read() to set -ENOMEM, not 0, on alloc failure.
  - Fixed nfs_netfs_issue_read() to only terminate the subreq once.
  - Fixed cifs_issue_read() to release the credits if cifs_reopen_file()
    fails.
- Rebased on v7.1.

ver #3)
- Rebased to -rc7 as the patches wouldn't apply for Christian.
- Prepended a fix for a warning from generic/464 (the problem also exists
  upstream, just not the warning).
- Renamed kmap_local_bvec() to bvec_kmap_partial() as requested by
  Christoph.
- Adjusted smbdirect patch descriptions as requested by Stefan Metzmacher.

ver #2)
- Fixed a number of bugs reported by Sashiko[1].
- Split a bunch of fixes out and posted them separately[2].

[1] https://sashiko.dev/#/patchset/20260326104544.509518-1-dhowells%40redhat.com
[2] https://lore.kernel.org/linux-fsdevel/20260512-infozentrum-becher-7f86c47c96c8 at brauner/T/#t
[3] https://sashiko.dev/#/patchset/20260608145432.681865-1-dhowells%40redhat.com
[4] https://sashiko.dev/#/patchset/20260706153408.1231650-1-dhowells%40redhat.com
[5] https://sashiko.dev/#/patchset/20260716103030.3065561-1-dhowells%40redhat.com
[6] https://sashiko.dev/#/patchset/20260722130218.78958-1-dhowells%40redhat.com
[7] https://sashiko.dev/#/patchset/20260804100224.2748935-1-dhowells%40redhat.com

David Howells (25):
  netfs: Fix read progress reporting
  mm: Make readahead store folio count in readahead_control
  netfs: Bulk load the readahead-provided folios up front
  Add a function to kmap one page of a multipage bio_vec
  iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
  iov_iter: Add a segmented queue of bio_vec[]
  netfs: Add some tools for managing bvecq chains
  netfs: Make mempool available for bvecq
  netfs: Add a function to extract from an iter into a bvecq
  afs: Use a bvecq to hold dir content rather than folioq
  cifs: Use a bvecq for buffering instead of a folioq
  smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
  netfs: Remove the writethrough code
  cachefiles: Don't rely on backing fs storage map for most use cases
  netfs: Add the cache object ID to netfs_read/write tracepoints
  netfs: Switch to using bvecq rather than folio_queue and
    rolling_buffer
  smbdirect: Remove support for ITER_FOLIOQ from
    smbdirect_map_sges_from_iter()
  netfs: Remove netfs_alloc/free_folioq_buffer()
  netfs: Remove netfs_extract_user_iter()
  iov_iter: Remove ITER_FOLIOQ
  netfs: Remove folio_queue and rolling_buffer
  netfs: Simplify read abandonment
  netfs: Check for too much data being read
  netfs: Combine prepare and issue ops and grab the buffers on request
  cachefiles: Preset the state xattr when creating a new file

Gao Xiang (1):
  cachefiles,netfs: sunset ondemand mode

 Documentation/core-api/folio_queue.rst        | 209 ----
 Documentation/core-api/index.rst              |   1 -
 .../filesystems/caching/cachefiles.rst        | 179 ----
 Documentation/filesystems/netfs_library.rst   |   2 +-
 fs/9p/vfs_addr.c                              |  61 +-
 fs/afs/dir.c                                  |  40 +-
 fs/afs/dir_edit.c                             |  43 +-
 fs/afs/dir_search.c                           |  33 +-
 fs/afs/file.c                                 |  31 +-
 fs/afs/fsclient.c                             |   8 +-
 fs/afs/inode.c                                |   2 +-
 fs/afs/internal.h                             |  14 +-
 fs/afs/symlink.c                              |  37 +-
 fs/afs/write.c                                |  36 +-
 fs/afs/yfsclient.c                            |   6 +-
 fs/cachefiles/Kconfig                         |  12 -
 fs/cachefiles/Makefile                        |   1 -
 fs/cachefiles/daemon.c                        |  96 +-
 fs/cachefiles/interface.c                     |  98 +-
 fs/cachefiles/internal.h                      | 147 +--
 fs/cachefiles/io.c                            | 602 ++++++++---
 fs/cachefiles/namei.c                         |  42 +-
 fs/cachefiles/ondemand.c                      | 761 --------------
 fs/cachefiles/xattr.c                         |  84 +-
 fs/ceph/Kconfig                               |   1 +
 fs/ceph/addr.c                                | 123 ++-
 fs/netfs/Kconfig                              |   3 +
 fs/netfs/Makefile                             |   4 +-
 fs/netfs/buffered_read.c                      | 526 ++++++----
 fs/netfs/buffered_write.c                     |  59 +-
 fs/netfs/bvecq.c                              | 808 +++++++++++++++
 fs/netfs/direct_read.c                        | 115 +--
 fs/netfs/direct_write.c                       | 162 +--
 fs/netfs/fscache_io.c                         |   4 +-
 fs/netfs/internal.h                           | 109 +-
 fs/netfs/iterator.c                           | 394 +++-----
 fs/netfs/main.c                               |  13 +-
 fs/netfs/misc.c                               | 168 +---
 fs/netfs/objects.c                            |  57 +-
 fs/netfs/read_collect.c                       | 377 ++++---
 fs/netfs/read_pgpriv2.c                       | 204 ++--
 fs/netfs/read_retry.c                         | 261 ++---
 fs/netfs/read_single.c                        | 184 ++--
 fs/netfs/rolling_buffer.c                     | 226 -----
 fs/netfs/stats.c                              |  10 +-
 fs/netfs/write_collect.c                      | 245 +++--
 fs/netfs/write_issue.c                        | 935 ++++++++----------
 fs/netfs/write_retry.c                        | 187 ++--
 fs/nfs/Kconfig                                |   1 +
 fs/nfs/fscache.c                              |  24 +-
 fs/smb/client/cifsglob.h                      |   2 +-
 fs/smb/client/cifssmb.c                       |  13 +-
 fs/smb/client/file.c                          | 145 +--
 fs/smb/client/smb2ops.c                       |  84 +-
 fs/smb/client/smb2pdu.c                       |  28 +-
 fs/smb/client/transport.c                     |  15 +-
 fs/smb/smbdirect/connection.c                 | 135 ++-
 include/linux/bvec.h                          |  18 +
 include/linux/bvecq.h                         | 363 +++++++
 include/linux/folio_queue.h                   | 282 ------
 include/linux/fscache.h                       |  17 +
 include/linux/iov_iter.h                      |  87 +-
 include/linux/netfs.h                         | 188 ++--
 include/linux/pagemap.h                       |  10 +
 include/linux/rolling_buffer.h                |  61 --
 include/linux/uio.h                           |  17 +-
 include/trace/events/cachefiles.h             | 209 +---
 include/trace/events/netfs.h                  | 162 ++-
 include/uapi/linux/cachefiles.h               |  68 --
 kernel/bpf/btf.c                              |   9 -
 lib/iov_iter.c                                | 559 ++++++-----
 lib/scatterlist.c                             |  82 +-
 lib/tests/kunit_iov_iter.c                    | 131 ++-
 mm/readahead.c                                |   5 +
 net/9p/client.c                               |   8 +-
 75 files changed, 5134 insertions(+), 5309 deletions(-)
 delete mode 100644 Documentation/core-api/folio_queue.rst
 delete mode 100644 fs/cachefiles/ondemand.c
 create mode 100644 fs/netfs/bvecq.c
 delete mode 100644 fs/netfs/rolling_buffer.c
 create mode 100644 include/linux/bvecq.h
 delete mode 100644 include/linux/folio_queue.h
 delete mode 100644 include/linux/rolling_buffer.h
 delete mode 100644 include/uapi/linux/cachefiles.h



From dhowells at redhat.com  Mon Aug 10 07:47:18 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:18 +0100
Subject: [PATCH v9 01/26] netfs: Fix read progress reporting
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-2-dhowells@redhat.com>

For really big read RPC ops that span multiple folios, netfslib allows the
filesystem to give progress notifications to wake up the collector thread
to do a collection of folios that have now been fetched, even if the RPC is
still ongoing, thereby allowing the application to make progress.

This works by taking the current rreq->cleaned_to value (which indicates
which folios have been unlocked) and adding the stashed size of the next
folio to it.  cleaned_to, however, is subject to 64-bit tearing on a 32-bit
arch.

Fix this by stashing the next progress notification point as a size_t
(which won't tear) to be added to rreq->start (which won't change), with
the collector thread calculating that from cleaned_to plus the next folio
size.

Further, however, if the folios are small, the collector thread gets
constantly woken up - which has a negative performance impact on the
system.

Fix that too by setting a minimum trigger of 256KiB or the size of the
folio at the front of the queue, whichever is larger.

Also, make sure rreq->cleaned_to is initialised up front, along with
rreq->collected_to and stream->collected_to.

Fixes: e2d46f2ec332 ("netfs: Change the read result collector to only use one work item")
Link: https://sashiko.dev/#/patchset/20260804100224.2748935-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/buffered_read.c |  4 ++++
 fs/netfs/objects.c       | 32 ++++++++++++++++++++------------
 fs/netfs/read_collect.c  | 27 +++++++++++++++------------
 fs/netfs/read_single.c   |  2 ++
 include/linux/netfs.h    |  2 +-
 5 files changed, 42 insertions(+), 25 deletions(-)

diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 7fdfa4f27e34..af14beeb6d11 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -106,6 +106,9 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq,
 				folio_batch_release(&put_batch);
 				return added;
 			}
+
+			if (!rreq->progress_at)
+				rreq->progress_at = folioq_folio_size(rreq->buffer.tail, 0);
 			rreq->submitted += added;
 		}
 		folio_batch_release(&put_batch);
@@ -387,6 +390,7 @@ static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct fo
 	if (added < 0)
 		return added;
 	rreq->submitted = rreq->start + added;
+	rreq->progress_at = added;
 	return 0;
 }
 
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index 01461a74642d..e88293cdb834 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -41,24 +41,32 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
 
 	memset(rreq, 0, kmem_cache_size(cache));
 	INIT_WORK(&rreq->cleanup_work, netfs_free_request);
-	rreq->gfp	= gfp;
-	rreq->start	= start;
-	rreq->len	= len;
-	rreq->origin	= origin;
-	rreq->netfs_ops	= ctx->ops;
-	rreq->mapping	= mapping;
-	rreq->inode	= inode;
-	rreq->i_size	= i_size_read(inode);
-	rreq->debug_id	= atomic_inc_return(&debug_ids);
-	rreq->wsize	= INT_MAX;
+	rreq->gfp		= gfp;
+	rreq->start		= start;
+	rreq->collected_to	= start;
+	rreq->cleaned_to	= start;
+	rreq->len		= len;
+	rreq->progress_at	= ULONG_MAX;
+	rreq->origin		= origin;
+	rreq->netfs_ops		= ctx->ops;
+	rreq->mapping		= mapping;
+	rreq->inode		= inode;
+	rreq->i_size		= i_size_read(inode);
+	rreq->debug_id		= atomic_inc_return(&debug_ids);
+	rreq->wsize		= INT_MAX;
 	rreq->io_streams[0].sreq_max_len = ULONG_MAX;
 	rreq->io_streams[0].sreq_max_segs = 0;
 	spin_lock_init(&rreq->lock);
-	INIT_LIST_HEAD(&rreq->io_streams[0].subrequests);
-	INIT_LIST_HEAD(&rreq->io_streams[1].subrequests);
 	init_waitqueue_head(&rreq->waitq);
 	refcount_set(&rreq->ref, 2);
 
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_stream *stream = &rreq->io_streams[s];
+
+		INIT_LIST_HEAD(&stream->subrequests);
+		stream->collected_to = rreq->start;
+	}
+
 	if (origin == NETFS_READAHEAD ||
 	    origin == NETFS_READPAGE ||
 	    origin == NETFS_READ_GAPS ||
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 23660a590124..91b187db0bca 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -112,7 +112,7 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 	if (slot >= folioq_nr_slots(folioq)) {
 		folioq = rolling_buffer_delete_spent(&rreq->buffer);
 		if (!folioq) {
-			rreq->front_folio_order = 0;
+			WRITE_ONCE(rreq->progress_at, ULONG_MAX);
 			return;
 		}
 		slot = 0;
@@ -120,8 +120,7 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 
 	for (;;) {
 		struct folio *folio;
-		unsigned long long fpos, fend;
-		unsigned int order;
+		unsigned long long fpos = rreq->cleaned_to, fend;
 		size_t fsize;
 
 		if (*notes & COPY_TO_CACHE)
@@ -133,12 +132,12 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 			      rreq->debug_id, folio->index))
 			trace_netfs_folio(folio, netfs_folio_trace_not_locked);
 
-		order = folioq_folio_order(folioq, slot);
-		rreq->front_folio_order = order;
-		fsize = PAGE_SIZE << order;
+		fsize = folioq_folio_size(folioq, slot);
 		fpos = folio_pos(folio);
 		fend = fpos + fsize;
 
+		WRITE_ONCE(rreq->progress_at, fend - rreq->start);
+
 		trace_netfs_collect_folio(rreq, folio, fend, collected_to);
 
 		/* Unlock any folio we've transferred all of. */
@@ -146,7 +145,7 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 			break;
 
 		netfs_unlock_read_folio(rreq, folioq, slot);
-		WRITE_ONCE(rreq->cleaned_to, fpos + fsize);
+		WRITE_ONCE(rreq->cleaned_to, fend);
 		*notes |= MADE_PROGRESS;
 
 		clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
@@ -232,7 +231,7 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 		 * subreqs.
 		 */
 		if (notes & BUFFERED) {
-			size_t fsize = PAGE_SIZE << rreq->front_folio_order;
+			uoff_t unlock_at = rreq->start + rreq->progress_at;
 
 			/* Clear the tail of a short read. */
 			if (!(notes & HIT_PENDING) &&
@@ -257,7 +256,7 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 				transferred = front->len;
 				trace_netfs_rreq(rreq, netfs_rreq_trace_set_abandon);
 			}
-			if (front->start + transferred >= rreq->cleaned_to + fsize ||
+			if (front->start + transferred >= unlock_at ||
 			    test_bit(NETFS_SREQ_HIT_EOF, &front->flags))
 				netfs_read_unlock_folios(rreq, &notes);
 		} else {
@@ -477,15 +476,19 @@ void netfs_read_collection_worker(struct work_struct *work)
 void netfs_read_subreq_progress(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
-	struct netfs_io_stream *stream = &rreq->io_streams[0];
-	size_t fsize = PAGE_SIZE << rreq->front_folio_order;
+	struct netfs_io_stream *stream = &rreq->io_streams[subreq->stream_nr];
+	size_t progress_at = READ_ONCE(rreq->progress_at);
+	uoff_t update_at = rreq->start + progress_at;
+	uoff_t transferred_to = subreq->start + subreq->transferred;
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_progress);
 
 	/* If we are at the head of the queue, wake up the collector,
 	 * getting a ref to it if we were the ones to do so.
 	 */
-	if (subreq->start + subreq->transferred > rreq->cleaned_to + fsize &&
+	if (progress_at != ULONG_MAX &&
+	    transferred_to >= update_at &&
+	    transferred_to - update_at >= 256 * 1024 &&
 	    (rreq->origin == NETFS_READAHEAD ||
 	     rreq->origin == NETFS_READPAGE ||
 	     rreq->origin == NETFS_READ_FOR_WRITE) &&
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index 8833550d2eb6..de67ac41548d 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -170,6 +170,8 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	if (IS_ERR(rreq))
 		return PTR_ERR(rreq);
 
+	rreq->progress_at = rreq->len;
+
 	ret = netfs_single_begin_cache_read(rreq, ictx);
 	if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
 		goto cleanup_free;
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index d0b62d53eea9..cc8d6500d059 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -247,6 +247,7 @@ struct netfs_io_request {
 	unsigned long long	submitted;	/* Amount submitted for I/O so far */
 	unsigned long long	len;		/* Length of the request */
 	size_t			transferred;	/* Amount to be indicated as transferred */
+	size_t			progress_at;	/* Report read progress when hit this much read */
 	long			error;		/* 0 or error that occurred */
 	unsigned long long	i_size;		/* Size of the file */
 	unsigned long long	start;		/* Start position */
@@ -263,7 +264,6 @@ struct netfs_io_request {
 	atomic_t		subreq_counter;	/* Next subreq->debug_index */
 	unsigned int		nr_group_rel;	/* Number of refs to release on ->group */
 	spinlock_t		lock;		/* Lock for queuing subreqs */
-	unsigned char		front_folio_order; /* Order (size) of front folio */
 	enum netfs_io_origin	origin;		/* Origin of the request */
 	bool			direct_bv_unpin; /* T if direct_bv[] must be unpinned */
 	refcount_t		ref;



From dhowells at redhat.com  Mon Aug 10 07:47:19 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:19 +0100
Subject: [PATCH v9 02/26] mm: Make readahead store folio count in readahead_control
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-3-dhowells@redhat.com>

Make readahead store folio count in readahead_control so that the
filesystem can know in advance how many folios it needs to keep track of.

This is cleared by read_pages() in case it is called from a loop.

The count is accessed by the filesystem with readahead_folio_count().

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: netfs at lists.linux.dev
cc: linux-mm at kvack.org
cc: linux-fsdevel at vger.kernel.org
---
 include/linux/pagemap.h | 10 ++++++++++
 mm/readahead.c          |  5 +++++
 2 files changed, 15 insertions(+)

diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 2c3718d592d6..e1e51bace388 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -1348,6 +1348,7 @@ struct readahead_control {
 	struct file_ra_state *ra;
 /* private: use the readahead_* accessors instead */
 	pgoff_t _index;
+	unsigned int _nr_folios;
 	unsigned int _nr_pages;
 	unsigned int _batch_count;
 	bool dropbehind;
@@ -1527,6 +1528,15 @@ static inline size_t readahead_batch_length(const struct readahead_control *rac)
 	return rac->_batch_count * PAGE_SIZE;
 }
 
+/**
+ * readahead_folio_count - Get the number of folios in this readahead request.
+ * @rac: The readahead request.
+ */
+static inline unsigned int readahead_folio_count(const struct readahead_control *rac)
+{
+	return rac->_nr_folios;
+}
+
 static inline unsigned long dir_pages(const struct inode *inode)
 {
 	return (unsigned long)(inode->i_size + PAGE_SIZE - 1) >>
diff --git a/mm/readahead.c b/mm/readahead.c
index 558c92957518..069ded56fd80 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -188,6 +188,7 @@ static void read_pages(struct readahead_control *rac)
 	if (unlikely(rac->_workingset))
 		psi_memstall_leave(&rac->_pflags);
 	rac->_workingset = false;
+	rac->_nr_folios = 0;
 
 	BUG_ON(readahead_count(rac));
 }
@@ -303,6 +304,7 @@ void page_cache_ra_unbounded(struct readahead_control *ractl,
 		if (i == mark)
 			folio_set_readahead(folio);
 		ractl->_workingset |= folio_test_workingset(folio);
+		ractl->_nr_folios++;
 		ractl->_nr_pages += min_nrpages;
 		i += min_nrpages;
 	}
@@ -473,6 +475,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index,
 		return err;
 	}
 
+	ractl->_nr_folios++;
 	ractl->_nr_pages += 1UL << order;
 	ractl->_workingset |= folio_test_workingset(folio);
 	return 0;
@@ -822,6 +825,7 @@ void readahead_expand(struct readahead_control *ractl,
 			ractl->_workingset = true;
 			psi_memstall_enter(&ractl->_pflags);
 		}
+		ractl->_nr_folios++;
 		ractl->_nr_pages += min_nrpages;
 		ractl->_index = folio->index;
 	}
@@ -851,6 +855,7 @@ void readahead_expand(struct readahead_control *ractl,
 			ractl->_workingset = true;
 			psi_memstall_enter(&ractl->_pflags);
 		}
+		ractl->_nr_folios++;
 		ractl->_nr_pages += min_nrpages;
 		if (ra) {
 			ra->size += min_nrpages;



From dhowells at redhat.com  Mon Aug 10 07:47:20 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:20 +0100
Subject: [PATCH v9 03/26] netfs: Bulk load the readahead-provided folios up front
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-4-dhowells@redhat.com>

Load all the folios by the VM for readahead up front into the folio queue.
With the number of folios provided by the VM, the folio queue can be fully
allocated first and then the loading happen in one go inside the RCU read
lock.  The folio refs acquired from readahead are dropped in bulk once the
first subrequest is dispatched as it's quite a slow operation.  The
collector waits for NETFS_RREQ_NEED_PUT_RA_REFS to be cleared so that it
doesn't unlock folios before the xarray has been scanned for them.

This simplifies the buffer handling later and isn't noticeably slower as
the xarray doesn't need to be modified and the folios are all already
pre-locked.

Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: netfs at lists.linux.dev
cc: linux-mm at kvack.org
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/buffered_read.c       | 102 +++++++++++++++++++--------------
 fs/netfs/internal.h            |   1 +
 fs/netfs/misc.c                |  19 ++++++
 fs/netfs/read_collect.c        |   7 +++
 fs/netfs/read_retry.c          |   7 +++
 fs/netfs/rolling_buffer.c      |  75 ++++++++++++++++++++++++
 include/linux/netfs.h          |   1 +
 include/linux/rolling_buffer.h |   3 +
 include/trace/events/netfs.h   |   3 +
 9 files changed, 176 insertions(+), 42 deletions(-)

diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index af14beeb6d11..55ef81173a44 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -54,6 +54,42 @@ static void netfs_rreq_expand(struct netfs_io_request *rreq,
 	}
 }
 
+/*
+ * Drop the folio refs acquired from the readahead API.
+ */
+static void netfs_bulk_drop_ra_refs(struct netfs_io_request *rreq)
+{
+	struct folio_batch fbatch;
+	struct folio *folio;
+	pgoff_t nr_pages = DIV_ROUND_UP(rreq->len, PAGE_SIZE);
+	pgoff_t first = rreq->start / PAGE_SIZE;
+	XA_STATE(xas, &rreq->mapping->i_pages, first);
+
+	folio_batch_init(&fbatch);
+
+	rcu_read_lock();
+
+	xas_for_each(&xas, folio,  first + nr_pages - 1) {
+		if (xas_retry(&xas, folio))
+			continue;
+
+		if (!folio_batch_add(&fbatch, folio))
+			folio_batch_release(&fbatch);
+	}
+
+	rcu_read_unlock();
+	folio_batch_release(&fbatch);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_ra_put_ref);
+	clear_bit_unlock(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags);
+	wake_up(&rreq->waitq);
+}
+
+static void netfs_maybe_bulk_drop_ra_refs(struct netfs_io_request *rreq)
+{
+	if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+		netfs_bulk_drop_ra_refs(rreq);
+}
+
 /*
  * Begin an operation, and fetch the stored zero point value from the cookie if
  * available.
@@ -74,12 +110,8 @@ static int netfs_begin_cache_read(struct netfs_io_request *rreq, struct netfs_in
  *
  * Returns the limited size if successful and -ENOMEM if insufficient memory
  * available.
- *
- * [!] NOTE: This must be run in the same thread as ->issue_read() was called
- * in as we access the readahead_control struct.
  */
-static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq,
-					   struct readahead_control *ractl)
+static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	size_t rsize = subreq->len;
@@ -87,33 +119,6 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq,
 	if (subreq->source == NETFS_DOWNLOAD_FROM_SERVER)
 		rsize = umin(rsize, rreq->io_streams[0].sreq_max_len);
 
-	if (ractl) {
-		/* If we don't have sufficient folios in the rolling buffer,
-		 * extract a folioq's worth from the readahead region at a time
-		 * into the buffer.  Note that this acquires a ref on each page
-		 * that we will need to release later - but we don't want to do
-		 * that until after we've started the I/O.
-		 */
-		struct folio_batch put_batch;
-
-		folio_batch_init(&put_batch);
-		while (rreq->submitted < subreq->start + rsize) {
-			ssize_t added;
-
-			added = rolling_buffer_load_from_ra(&rreq->buffer, ractl,
-							    &put_batch);
-			if (added < 0) {
-				folio_batch_release(&put_batch);
-				return added;
-			}
-
-			if (!rreq->progress_at)
-				rreq->progress_at = folioq_folio_size(rreq->buffer.tail, 0);
-			rreq->submitted += added;
-		}
-		folio_batch_release(&put_batch);
-	}
-
 	subreq->len = rsize;
 	if (unlikely(rreq->io_streams[0].sreq_max_segs)) {
 		size_t limit = netfs_limit_iter(&rreq->buffer.iter, 0, rsize,
@@ -211,8 +216,7 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
  * slicing up the region to be read according to available cache blocks and
  * network rsize.
  */
-static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
-				    struct readahead_control *ractl)
+static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 {
 	unsigned long long start = rreq->start;
 	ssize_t size = rreq->len;
@@ -291,7 +295,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
 		break;
 
 	issue:
-		slice = netfs_prepare_read_iterator(subreq, ractl);
+		slice = netfs_prepare_read_iterator(subreq);
 		if (slice < 0) {
 			ret = slice;
 			netfs_cancel_read(subreq, ret);
@@ -305,6 +309,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq,
 		}
 
 		netfs_issue_read(rreq, subreq);
+		netfs_maybe_bulk_drop_ra_refs(rreq);
 
 		if (test_bit(NETFS_RREQ_PAUSE, &rreq->flags))
 			netfs_wait_for_paused_read(rreq);
@@ -343,6 +348,7 @@ void netfs_readahead(struct readahead_control *ractl)
 	struct netfs_io_request *rreq;
 	struct netfs_inode *ictx = netfs_inode(ractl->mapping->host);
 	unsigned long long start = readahead_pos(ractl);
+	ssize_t added;
 	size_t size = readahead_length(ractl);
 	int ret;
 
@@ -363,11 +369,23 @@ void netfs_readahead(struct readahead_control *ractl)
 
 	netfs_rreq_expand(rreq, ractl);
 
-	rreq->submitted = rreq->start;
-	if (rolling_buffer_init(&rreq->buffer, rreq->debug_id, ITER_DEST, rreq->gfp) < 0)
+	/* Load the folios to be read into a bvecq chain.  Note that this
+	 * acquires a ref on each folio that we will need to release later -
+	 * but we don't want to do that until after we've started the I/O.
+	 */
+	added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl, rreq->debug_id);
+	if (added < 0) {
+		ret = added;
 		goto cleanup_free;
-	netfs_read_to_pagecache(rreq, ractl);
+	}
+	__set_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags);
+
+	rreq->submitted = rreq->start + added;
+	rreq->cleaned_to = rreq->start;
+	rreq->progress_at = folioq_folio_size(rreq->buffer.tail, 0);
 
+	netfs_read_to_pagecache(rreq);
+	netfs_maybe_bulk_drop_ra_refs(rreq);
 	return netfs_put_request(rreq, netfs_rreq_trace_put_return);
 
 cleanup_free:
@@ -461,7 +479,7 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 	iov_iter_bvec(&rreq->buffer.iter, ITER_DEST, bvec, i, rreq->len);
 	rreq->submitted = rreq->start + flen;
 
-	netfs_read_to_pagecache(rreq, NULL);
+	netfs_read_to_pagecache(rreq);
 
 	ret = netfs_wait_for_read(rreq);
 	if (ret >= 0) {
@@ -536,7 +554,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
 	if (ret < 0)
 		goto discard;
 
-	netfs_read_to_pagecache(rreq, NULL);
+	netfs_read_to_pagecache(rreq);
 	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret < 0 ? ret : 0;
@@ -693,7 +711,7 @@ int netfs_write_begin(struct netfs_inode *ctx,
 	if (ret < 0)
 		goto error_put;
 
-	netfs_read_to_pagecache(rreq, NULL);
+	netfs_read_to_pagecache(rreq);
 	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	if (ret < 0)
@@ -758,7 +776,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 	if (ret < 0)
 		goto error_put;
 
-	netfs_read_to_pagecache(rreq, NULL);
+	netfs_read_to_pagecache(rreq);
 	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret < 0 ? ret : 0;
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 420ee7b26580..bd8b2d633f96 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -79,6 +79,7 @@ ssize_t netfs_wait_for_read(struct netfs_io_request *rreq);
 ssize_t netfs_wait_for_write(struct netfs_io_request *rreq);
 void netfs_wait_for_paused_read(struct netfs_io_request *rreq);
 void netfs_wait_for_paused_write(struct netfs_io_request *rreq);
+void netfs_wait_for_put_ra_refs(struct netfs_io_request *rreq);
 
 /*
  * objects.c
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index 5d554512ed23..f5c1c463f4ff 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -563,3 +563,22 @@ void netfs_wait_for_paused_write(struct netfs_io_request *rreq)
 {
 	return netfs_wait_for_pause(rreq, netfs_write_collection);
 }
+
+/*
+ * Wait for the readahead-acquired refs to be put.
+ */
+void netfs_wait_for_put_ra_refs(struct netfs_io_request *rreq)
+{
+	DEFINE_WAIT(myself);
+
+	for (;;) {
+		trace_netfs_rreq(rreq, netfs_rreq_trace_wait_put_ra_refs);
+		prepare_to_wait(&rreq->waitq, &myself, TASK_UNINTERRUPTIBLE);
+		if (!test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+			break;
+		schedule();
+	}
+
+	trace_netfs_rreq(rreq, netfs_rreq_trace_waited_put_ra_refs);
+	finish_wait(&rreq->waitq, &myself);
+}
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 91b187db0bca..764579c76d3d 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -118,6 +118,13 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 		slot = 0;
 	}
 
+	/* We have to wait for readahead refs to have been released before we
+	 * can unlock any folios as the ref-dropper walks i_pages and the only
+	 * thing preventing these folios from being removed is the folio lock.
+	 */
+	if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+		netfs_wait_for_put_ra_refs(rreq);
+
 	for (;;) {
 		struct folio *folio;
 		unsigned long long fpos = rreq->cleaned_to, fend;
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index 2b42758e01ec..dd463a485139 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -292,6 +292,13 @@ void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
 {
 	struct folio_queue *p;
 
+	/* We have to wait for readahead refs to have been released before we
+	 * can unlock any folios as the ref-dropper walks i_pages and the only
+	 * thing preventing these folios from being removed is the folio lock.
+	 */
+	if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
+		netfs_wait_for_put_ra_refs(rreq);
+
 	for (p = rreq->buffer.tail; p; p = p->next) {
 		for (int slot = 0; slot < folioq_count(p); slot++) {
 			struct folio *folio = folioq_folio(p, slot);
diff --git a/fs/netfs/rolling_buffer.c b/fs/netfs/rolling_buffer.c
index 8c0026836f9c..4e8584725b6a 100644
--- a/fs/netfs/rolling_buffer.c
+++ b/fs/netfs/rolling_buffer.c
@@ -153,6 +153,81 @@ ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
 	return size;
 }
 
+/*
+ * Decant the entire list of folios to read into a rolling buffer.
+ */
+ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
+					 struct readahead_control *ractl,
+					 unsigned int rreq_id)
+{
+	XA_STATE(xas, &ractl->mapping->i_pages, ractl->_index);
+	struct folio_queue *fq;
+	struct folio *folio;
+	ssize_t loaded = 0;
+	int nr, slot = 0, npages = 0;
+
+	/* First allocate all the folioqs we're going to need to avoid having
+	 * to deal with ENOMEM later.
+	 */
+	nr = ractl->_nr_folios;
+	do {
+		fq = netfs_folioq_alloc(rreq_id, GFP_KERNEL,
+					netfs_trace_folioq_make_space);
+		if (!fq) {
+			rolling_buffer_clear(roll);
+			return -ENOMEM;
+		}
+		fq->prev = roll->head;
+		if (!roll->tail)
+			roll->tail = fq;
+		else
+			roll->head->next = fq;
+		roll->head = fq;
+
+		nr -= folioq_nr_slots(fq);
+	} while (nr > 0);
+
+	rcu_read_lock();
+
+	fq = roll->tail;
+	xas_for_each(&xas, folio, ractl->_index + ractl->_nr_pages - 1) {
+		unsigned int order;
+
+		if (xas_retry(&xas, folio))
+			continue;
+		VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
+
+		order = folio_order(folio);
+		fq->orders[slot] = order;
+		fq->vec.folios[slot] = folio;
+		loaded += PAGE_SIZE << order;
+		npages += 1 << order;
+		trace_netfs_folio(folio, netfs_folio_trace_read);
+
+		slot++;
+		if (slot >= folioq_nr_slots(fq)) {
+			fq->vec.nr = slot;
+			fq = fq->next;
+			if (!fq) {
+				WARN_ON_ONCE(npages < readahead_count(ractl));
+				break;
+			}
+			slot = 0;
+		}
+	}
+
+	rcu_read_unlock();
+
+	if (fq)
+		fq->vec.nr = slot;
+
+	WRITE_ONCE(roll->iter.count, loaded);
+	iov_iter_folio_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
+	ractl->_index    += npages;
+	ractl->_nr_pages -= npages;
+	return loaded;
+}
+
 /*
  * Append a folio to the rolling buffer.
  */
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index cc8d6500d059..f9dcf3f31cb8 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -279,6 +279,7 @@ struct netfs_io_request {
 #define NETFS_RREQ_FOLIO_COPY_TO_CACHE	10	/* Copy current folio to cache from read */
 #define NETFS_RREQ_UPLOAD_TO_SERVER	11	/* Need to write to the server */
 #define NETFS_RREQ_USE_IO_ITER		12	/* Use ->io_iter rather than ->i_pages */
+#define NETFS_RREQ_NEED_PUT_RA_REFS	17	/* Need to put the folio refs RA gave us */
 #define NETFS_RREQ_USE_PGPRIV2		31	/* [DEPRECATED] Use PG_private_2 to mark
 						 * write to cache on read */
 	const struct netfs_request_ops *netfs_ops;
diff --git a/include/linux/rolling_buffer.h b/include/linux/rolling_buffer.h
index 9e5dad29669c..1e37750e4ff8 100644
--- a/include/linux/rolling_buffer.h
+++ b/include/linux/rolling_buffer.h
@@ -48,6 +48,9 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp);
 ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
 				    struct readahead_control *ractl,
 				    struct folio_batch *put_batch);
+ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
+					 struct readahead_control *ractl,
+					 unsigned int rreq_id);
 ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
 			      unsigned int flags, gfp_t gfp);
 struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll);
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 082cb03c6131..9bda9302be90 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -59,6 +59,7 @@
 	EM(netfs_rreq_trace_free,		"FREE   ")	\
 	EM(netfs_rreq_trace_intr,		"INTR   ")	\
 	EM(netfs_rreq_trace_ki_complete,	"KI-CMPL")	\
+	EM(netfs_rreq_trace_ra_put_ref,		"RA-PUT ")	\
 	EM(netfs_rreq_trace_recollect,		"RECLLCT")	\
 	EM(netfs_rreq_trace_redirty,		"REDIRTY")	\
 	EM(netfs_rreq_trace_resubmit,		"RESUBMT")	\
@@ -70,9 +71,11 @@
 	EM(netfs_rreq_trace_unpause,		"UNPAUSE")	\
 	EM(netfs_rreq_trace_wait_ip,		"WAIT-IP")	\
 	EM(netfs_rreq_trace_wait_pause,		"--PAUSED--")	\
+	EM(netfs_rreq_trace_wait_put_ra_refs,	"WAIT-P-RA")	\
 	EM(netfs_rreq_trace_wait_quiesce,	"WAIT-QUIESCE")	\
 	EM(netfs_rreq_trace_waited_ip,		"DONE-IP")	\
 	EM(netfs_rreq_trace_waited_pause,	"--UNPAUSED--")	\
+	EM(netfs_rreq_trace_waited_put_ra_refs,	"DONE-P-RA")	\
 	EM(netfs_rreq_trace_waited_quiesce,	"DONE-QUIESCE")	\
 	EM(netfs_rreq_trace_wake_ip,		"WAKE-IP")	\
 	EM(netfs_rreq_trace_wake_queue,		"WAKE-Q ")	\



From dhowells at redhat.com  Mon Aug 10 07:47:22 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:22 +0100
Subject: [PATCH v9 05/26] iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-6-dhowells@redhat.com>

Make iov_iter_get_pages*() wrap iov_iter_extract_pages() for kernel
iterator types (e.g. ITER_BVEC, ITER_FOLIOQ, ITER_XARRAY).  The pages
obtained have their refcounts incremented afterwards if they're not slab
pages.  ITER_KVEC is left returning -EFAULT.

Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Jens Axboe <axboe at kernel.dk>
cc: linux-block at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 lib/iov_iter.c | 164 ++++++-------------------------------------------
 1 file changed, 19 insertions(+), 145 deletions(-)

diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index c2484551a4e8..31afc9687eb6 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -910,118 +910,34 @@ static int want_pages_array(struct page ***res, size_t size,
 	return count;
 }
 
-static ssize_t iter_folioq_get_pages(struct iov_iter *iter,
+/*
+ * Wrap iov_iter_extract_pages() and then pin the non-slab pages we got back.
+ * This only works for non-user iterator types as get_pages uses get_user_pages
+ * not pin_user_pages.
+ */
+static ssize_t iter_get_kernel_pages(struct iov_iter *iter,
 				     struct page ***ppages, size_t maxsize,
 				     unsigned maxpages, size_t *_start_offset)
 {
-	const struct folio_queue *folioq = iter->folioq;
 	struct page **pages;
-	unsigned int slot = iter->folioq_slot;
-	size_t extracted = 0, count = iter->count, iov_offset = iter->iov_offset;
+	ssize_t ret, done;
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		slot = 0;
-		if (WARN_ON(iov_offset != 0))
-			return -EIO;
-	}
+	ret = iov_iter_extract_pages(iter, ppages, maxsize, maxpages,
+				     0, _start_offset);
+	if (ret <= 0)
+		return ret;
 
-	maxpages = want_pages_array(ppages, maxsize, iov_offset & ~PAGE_MASK, maxpages);
-	if (!maxpages)
-		return -ENOMEM;
-	*_start_offset = iov_offset & ~PAGE_MASK;
 	pages = *ppages;
+	for (done = ret + *_start_offset; done > 0; done -= PAGE_SIZE) {
+		struct folio *folio = page_folio(*pages);
 
-	for (;;) {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t offset = iov_offset, fsize = folioq_folio_size(folioq, slot);
-		size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
-		if (offset < fsize) {
-			part = umin(part, umin(maxsize - extracted, fsize - offset));
-			count -= part;
-			iov_offset += part;
-			extracted += part;
-
-			*pages = folio_page(folio, offset / PAGE_SIZE);
-			get_page(*pages);
-			pages++;
-			maxpages--;
-		}
-
-		if (maxpages == 0 || extracted >= maxsize)
-			break;
-
-		if (iov_offset >= fsize) {
-			iov_offset = 0;
-			slot++;
-			if (slot == folioq_nr_slots(folioq) && folioq->next) {
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	}
-
-	iter->count = count;
-	iter->iov_offset = iov_offset;
-	iter->folioq = folioq;
-	iter->folioq_slot = slot;
-	return extracted;
-}
-
-static ssize_t iter_xarray_populate_pages(struct page **pages, struct xarray *xa,
-					  pgoff_t index, unsigned int nr_pages)
-{
-	XA_STATE(xas, xa, index);
-	struct folio *folio;
-	unsigned int ret = 0;
-
-	rcu_read_lock();
-	for (folio = xas_load(&xas); folio; folio = xas_next(&xas)) {
-		if (xas_retry(&xas, folio))
-			continue;
-
-		/* Has the folio moved or been split? */
-		if (unlikely(folio != xas_reload(&xas))) {
-			xas_reset(&xas);
-			continue;
-		}
-
-		pages[ret] = folio_file_page(folio, xas.xa_index);
-		folio_get(folio);
-		if (++ret == nr_pages)
-			break;
+		if (!folio_test_slab(folio))
+			folio_get(folio);
+		pages++;
 	}
-	rcu_read_unlock();
 	return ret;
 }
 
-static ssize_t iter_xarray_get_pages(struct iov_iter *i,
-				     struct page ***pages, size_t maxsize,
-				     unsigned maxpages, size_t *_start_offset)
-{
-	unsigned nr, offset, count;
-	pgoff_t index;
-	loff_t pos;
-
-	pos = i->xarray_start + i->iov_offset;
-	index = pos >> PAGE_SHIFT;
-	offset = pos & ~PAGE_MASK;
-	*_start_offset = offset;
-
-	count = want_pages_array(pages, maxsize, offset, maxpages);
-	if (!count)
-		return -ENOMEM;
-	nr = iter_xarray_populate_pages(*pages, i->xarray, index, count);
-	if (nr == 0)
-		return 0;
-
-	maxsize = min_t(size_t, nr * PAGE_SIZE - offset, maxsize);
-	i->iov_offset += maxsize;
-	i->count -= maxsize;
-	return maxsize;
-}
-
 /* must be done on non-empty ITER_UBUF or ITER_IOVEC one */
 static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
 {
@@ -1044,22 +960,6 @@ static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
 	BUG(); // if it had been empty, we wouldn't get called
 }
 
-/* must be done on non-empty ITER_BVEC one */
-static struct page *first_bvec_segment(const struct iov_iter *i,
-				       size_t *size, size_t *start)
-{
-	struct page *page;
-	size_t skip = i->iov_offset, len;
-
-	len = i->bvec->bv_len - skip;
-	if (*size > len)
-		*size = len;
-	skip += i->bvec->bv_offset;
-	page = i->bvec->bv_page + skip / PAGE_SIZE;
-	*start = skip % PAGE_SIZE;
-	return page;
-}
-
 static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
 		   struct page ***pages, size_t maxsize,
 		   unsigned int maxpages, size_t *start)
@@ -1095,36 +995,10 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
 		iov_iter_advance(i, maxsize);
 		return maxsize;
 	}
-	if (iov_iter_is_bvec(i)) {
-		struct page **p;
-		struct page *page;
 
-		page = first_bvec_segment(i, &maxsize, start);
-		n = want_pages_array(pages, maxsize, *start, maxpages);
-		if (!n)
-			return -ENOMEM;
-		p = *pages;
-		for (int k = 0; k < n; k++) {
-			struct folio *folio = page_folio(page + k);
-			p[k] = page + k;
-			if (!folio_test_slab(folio))
-				folio_get(folio);
-		}
-		maxsize = min_t(size_t, maxsize, n * PAGE_SIZE - *start);
-		i->count -= maxsize;
-		i->iov_offset += maxsize;
-		if (i->iov_offset == i->bvec->bv_len) {
-			i->iov_offset = 0;
-			i->bvec++;
-			i->nr_segs--;
-		}
-		return maxsize;
-	}
-	if (iov_iter_is_folioq(i))
-		return iter_folioq_get_pages(i, pages, maxsize, maxpages, start);
-	if (iov_iter_is_xarray(i))
-		return iter_xarray_get_pages(i, pages, maxsize, maxpages, start);
-	return -EFAULT;
+	if (iov_iter_is_kvec(i))
+		return -EFAULT;
+	return iter_get_kernel_pages(i, pages, maxsize, maxpages, start);
 }
 
 ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,



From dhowells at redhat.com  Mon Aug 10 07:47:21 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:21 +0100
Subject: [PATCH v9 04/26] Add a function to kmap one page of a multipage bio_vec
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-5-dhowells@redhat.com>

Add a function to kmap one page of a multipage bio_vec by offset (which is
added to the offset in the bio_vec internally).  The caller is responsible
for calculating how much of the page is then available.

Signed-off-by: David Howells <dhowells at redhat.com>
Acked-by: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Jens Axboe <axboe at kernel.dk>
cc: linux-block at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 include/linux/bvec.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 92837e2743f1..53cf36e73967 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -343,4 +343,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
 	return page_to_phys(bvec->bv_page) + bvec->bv_offset;
 }
 
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space.  The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
+{
+	offset += bvec->bv_offset;
+
+	return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+		(offset & ~PAGE_MASK);
+}
+
 #endif /* __LINUX_BVEC_H */



From dhowells at redhat.com  Mon Aug 10 07:47:23 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:23 +0100
Subject: [PATCH v9 06/26] iov_iter: Add a segmented queue of bio_vec[]
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-7-dhowells@redhat.com>

Add the concept of a segmented queue of bio_vec[] arrays.  This allows an
indefinite quantity of elements to be handled and allows things like
network filesystems and crypto drivers to glue bits on the ends without
having to reallocate the array.

The bvecq struct that defines each segment also carries capacity/usage
information along with flags indicating whether the constituent memory
regions need freeing or unpinning and the file position of the first
element in a segment.  The bvecq structs are refcounted to allow a queue to
be extracted in batches and split between a number of subrequests.

The bvecq can have the bio_vec[] it manages allocated in with it, but this
is not required.  A flag is provided for if this is the case as comparing
->bv to ->__bv is not sufficient to detect this case.

Add an iterator type ITER_BVECQ for it.  This is intended to replace
ITER_FOLIOQ (and ITER_XARRAY).

Note that the prev pointer is only really needed for iov_iter_revert() and
could be dispensed with if struct iov_iter contained the head information
as well as the current point.

Signed-off-by: David Howells <dhowells at redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Jens Axboe <axboe at kernel.dk>
cc: linux-block at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 include/linux/bvecq.h      |  56 +++++++
 include/linux/iov_iter.h   |  74 +++++++-
 include/linux/uio.h        |  11 ++
 lib/iov_iter.c             | 336 ++++++++++++++++++++++++++++++++++++-
 lib/scatterlist.c          |  67 +++++++-
 lib/tests/kunit_iov_iter.c | 260 ++++++++++++++++++++++++++++
 6 files changed, 798 insertions(+), 6 deletions(-)
 create mode 100644 include/linux/bvecq.h

diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
new file mode 100644
index 000000000000..15f16f905877
--- /dev/null
+++ b/include/linux/bvecq.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Implementation of a segmented queue of bio_vec[].
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells at redhat.com)
+ */
+
+#ifndef _LINUX_BVECQ_H
+#define _LINUX_BVECQ_H
+
+#include <linux/bvec.h>
+
+/*
+ * The type of memory retention used by the elements in bvecq->bv[] and how to
+ * clean it up.
+ */
+enum bvecq_mem {
+	BVECQ_MEM_EXTERNAL,	/* Externally retained memory - no freeing */
+	BVECQ_MEM_PAGECACHE,	/* Ref'd pagecache pages - must put */
+	BVECQ_MEM_GUP,		/* Pinned memory from get_user_pages() - unpin */
+	BVECQ_MEM_ALLOCED,	/* Memory alloc'd by bvecq - can be freed/pooled */
+} __mode(byte);
+
+/*
+ * Segmented bio_vec queue.
+ *
+ * These can be linked together to form messages of indefinite length and
+ * iterated over with an ITER_BVECQ iterator.  The list is non-circular; next
+ * and prev are NULL at the ends.
+ *
+ * The bv pointer points to the bio_vec array; this may be __bv if allocated
+ * together.  The caller is responsible for determining whether or not this is
+ * the case as the array pointed to by bv may be follow on directly from the
+ * bvecq by accident of allocation (ie. ->bv == ->__bv is *not* sufficient to
+ * determine this).
+ *
+ * The file position and discontiguity flag allow non-contiguous data sets to
+ * be chained together, but still teased apart without the need to convert the
+ * info in the bio_vec back into a folio pointer.
+ */
+struct bvecq {
+	struct bvecq	*next;		/* Next bvec in the list or NULL */
+	struct bvecq	*prev;		/* Prev bvec in the list or NULL */
+	unsigned long long fpos;	/* File position */
+	refcount_t	ref;
+	u32		priv;		/* Private data */
+	u16		nr_slots;	/* Number of elements in bv[] used */
+	u16		max_slots;	/* Number of elements allocated in bv[] */
+	enum bvecq_mem	mem_type:3;	/* What sort of memory and how to free it */
+	bool		inline_bv:1;	/* T if __bv[] is being used */
+	bool		discontig:1;	/* T if not contiguous with previous bvecq */
+	struct bio_vec	*bv;		/* Pointer to array of page fragments */
+	struct bio_vec	__bv[];		/* Default array (if ->inline_bv) */
+};
+
+#endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index f9a17fbbd398..04b8a6d943fa 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -10,6 +10,7 @@
 
 #include <linux/uio.h>
 #include <linux/bvec.h>
+#include <linux/bvecq.h>
 #include <linux/folio_queue.h>
 
 typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
@@ -141,6 +142,71 @@ size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2,
 	return progress;
 }
 
+/*
+ * Handle ITER_BVECQ.
+ */
+static __always_inline
+size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
+		     iov_step_f step)
+{
+	const struct bvecq *bq = iter->bvecq;
+	unsigned int slot = iter->bvecq_slot;
+	size_t progress = 0, skip = iter->iov_offset;
+
+	do {
+		const struct bio_vec *bvec;
+		struct page *page;
+		size_t poff, plen;
+		void *base;
+
+		if (slot >= bq->nr_slots) {
+			if (!bq->next)
+				break;
+			bq = bq->next;
+			slot = 0;
+			continue;
+		}
+
+		bvec = &bq->bv[slot];
+		/*
+		 * The caller must ensure that a slot with bv_len>0 has a valid
+		 * bv_page.
+		 */
+		page = bvec->bv_page + (bvec->bv_offset + skip) / PAGE_SIZE;
+		poff = (bvec->bv_offset + skip) % PAGE_SIZE;
+		plen = umin(bvec->bv_len - skip, len);
+
+		while (plen > 0) {
+			size_t part, remain, consumed;
+
+			part = umin(plen, PAGE_SIZE - poff);
+			base = kmap_local_page(page) + poff;
+			remain = step(base, progress, part, priv, priv2);
+			kunmap_local(base);
+
+			consumed = part - remain;
+			progress += consumed;
+			skip += consumed;
+			len -= consumed;
+			if (!len || remain)
+				goto stop;
+			page++;
+			poff = 0;
+			plen -= consumed;
+		}
+
+		skip = 0;
+		slot++;
+	} while (len);
+
+stop:
+	iter->bvecq_slot = slot;
+	iter->bvecq = bq;
+	iter->iov_offset = skip;
+	iter->count -= progress;
+	return progress;
+}
+
 /*
  * Handle ITER_FOLIOQ.
  */
@@ -306,6 +372,8 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_bvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_kvec(iter))
 		return iterate_kvec(iter, len, priv, priv2, step);
+	if (iov_iter_is_bvecq(iter))
+		return iterate_bvecq(iter, len, priv, priv2, step);
 	if (iov_iter_is_folioq(iter))
 		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
@@ -342,8 +410,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
  * buffer is presented in segments, which for kernel iteration are broken up by
  * physical pages and mapped, with the mapped address being presented.
  *
- * [!] Note This will only handle BVEC, KVEC, FOLIOQ, XARRAY and DISCARD-type
- * iterators; it will not handle UBUF or IOVEC-type iterators.
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and
+ * DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators.
  *
  * A step functions, @step, must be provided, one for handling mapped kernel
  * addresses and the other is given user addresses which have the potential to
@@ -370,6 +438,8 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_bvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_kvec(iter))
 		return iterate_kvec(iter, len, priv, priv2, step);
+	if (iov_iter_is_bvecq(iter))
+		return iterate_bvecq(iter, len, priv, priv2, step);
 	if (iov_iter_is_folioq(iter))
 		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
diff --git a/include/linux/uio.h b/include/linux/uio.h
index a9bc5b3067e3..f7cfa6ea8213 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -26,6 +26,7 @@ enum iter_type {
 	ITER_IOVEC,
 	ITER_BVEC,
 	ITER_KVEC,
+	ITER_BVECQ,
 	ITER_FOLIOQ,
 	ITER_XARRAY,
 	ITER_DISCARD,
@@ -68,6 +69,7 @@ struct iov_iter {
 				const struct iovec *__iov;
 				const struct kvec *kvec;
 				const struct bio_vec *bvec;
+				const struct bvecq *bvecq;
 				const struct folio_queue *folioq;
 				struct xarray *xarray;
 				void __user *ubuf;
@@ -77,6 +79,7 @@ struct iov_iter {
 	};
 	union {
 		unsigned long nr_segs;
+		u16 bvecq_slot;
 		u8 folioq_slot;
 		loff_t xarray_start;
 	};
@@ -145,6 +148,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
 	return iov_iter_type(i) == ITER_DISCARD;
 }
 
+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
+{
+	return iov_iter_type(i) == ITER_BVECQ;
+}
+
 static inline bool iov_iter_is_folioq(const struct iov_iter *i)
 {
 	return iov_iter_type(i) == ITER_FOLIOQ;
@@ -295,6 +303,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec
 void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,
 			unsigned long nr_segs, size_t count);
 void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+			 const struct bvecq *bvecq,
+			 unsigned int first_slot, unsigned int offset, size_t count);
 void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
 			  const struct folio_queue *folioq,
 			  unsigned int first_slot, unsigned int offset, size_t count);
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 31afc9687eb6..8c647783df2f 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -538,6 +538,40 @@ static void iov_iter_iovec_advance(struct iov_iter *i, size_t size)
 	i->__iov = iov;
 }
 
+static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
+{
+	const struct bvecq *bq = i->bvecq;
+	unsigned int slot = i->bvecq_slot;
+
+	if (!i->count)
+		return;
+	i->count -= by;
+
+	by += i->iov_offset; /* From beginning of current segment. */
+	do {
+		size_t len;
+
+		if (slot >= bq->nr_slots) {
+			if (!bq->next)
+				break;
+			bq = bq->next;
+			slot = 0;
+			continue;
+		}
+
+		len = bq->bv[slot].bv_len;
+
+		if (likely(by < len))
+			break;
+		by -= len;
+		slot++;
+	} while (by);
+
+	i->iov_offset = by;
+	i->bvecq_slot = slot;
+	i->bvecq = bq;
+}
+
 static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
 {
 	const struct folio_queue *folioq = i->folioq;
@@ -583,6 +617,8 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
 		iov_iter_iovec_advance(i, size);
 	} else if (iov_iter_is_bvec(i)) {
 		iov_iter_bvec_advance(i, size);
+	} else if (iov_iter_is_bvecq(i)) {
+		iov_iter_bvecq_advance(i, size);
 	} else if (iov_iter_is_folioq(i)) {
 		iov_iter_folioq_advance(i, size);
 	} else if (iov_iter_is_discard(i)) {
@@ -591,6 +627,33 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
 }
 EXPORT_SYMBOL(iov_iter_advance);
 
+static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
+{
+	const struct bvecq *bq = i->bvecq;
+	unsigned int slot = i->bvecq_slot;
+
+	for (;;) {
+		size_t len;
+
+		if (slot == 0) {
+			bq = bq->prev;
+			slot = bq->nr_slots;
+			continue;
+		}
+		slot--;
+
+		len = bq->bv[slot].bv_len;
+		if (unroll <= len) {
+			i->iov_offset = len - unroll;
+			break;
+		}
+		unroll -= len;
+	}
+
+	i->bvecq_slot = slot;
+	i->bvecq = bq;
+}
+
 static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
 {
 	const struct folio_queue *folioq = i->folioq;
@@ -648,6 +711,9 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
 			}
 			unroll -= n;
 		}
+	} else if (iov_iter_is_bvecq(i)) {
+		i->iov_offset = 0;
+		iov_iter_bvecq_revert(i, unroll);
 	} else if (iov_iter_is_folioq(i)) {
 		i->iov_offset = 0;
 		iov_iter_folioq_revert(i, unroll);
@@ -678,9 +744,30 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
 		if (iov_iter_is_bvec(i))
 			return min(i->count, i->bvec->bv_len - i->iov_offset);
 	}
+	if (!i->count)
+		return 0;
+	if (unlikely(iov_iter_is_bvecq(i))) {
+		const struct bvecq *bq = i->bvecq;
+		unsigned int slot = i->bvecq_slot;
+		size_t offset = i->iov_offset;
+
+		for (;;) {
+			while (slot >= bq->nr_slots) {
+				bq = bq->next;
+				if (!bq)
+					return 0;
+				slot = 0;
+				offset = 0;
+			}
+			if (bq->bv[slot].bv_len > offset)
+				break;
+			slot++;
+			offset = 0;
+		}
+		return umin(i->count, bq->bv[slot].bv_len - offset);
+	}
 	if (unlikely(iov_iter_is_folioq(i)))
-		return !i->count ? 0 :
-			umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
+		return umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
 	return i->count;
 }
 EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -717,6 +804,35 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction,
 }
 EXPORT_SYMBOL(iov_iter_bvec);
 
+/**
+ * iov_iter_bvec_queue - Initialise an I/O iterator to use a segmented bvec queue
+ * @i: The iterator to initialise.
+ * @direction: The direction of the transfer.
+ * @bvecq: The starting point in the bvec queue.
+ * @first_slot: The first slot in the bvec queue to use
+ * @offset: The offset into the bvec in the first slot to start at
+ * @count: The size of the I/O buffer in bytes.
+ *
+ * Set up an I/O iterator to either draw data out of the buffers attached to an
+ * inode or to inject data into those buffers.  The pages *must* be prevented
+ * from evaporation, either by the caller.
+ */
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+			 const struct bvecq *bvecq, unsigned int first_slot,
+			 unsigned int offset, size_t count)
+{
+	WARN_ON(direction & ~(READ | WRITE));
+	*i = (struct iov_iter) {
+		.iter_type	= ITER_BVECQ,
+		.data_source	= direction,
+		.bvecq		= bvecq,
+		.bvecq_slot	= first_slot,
+		.count		= count,
+		.iov_offset	= offset,
+	};
+}
+EXPORT_SYMBOL(iov_iter_bvec_queue);
+
 /**
  * iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
  * @i: The iterator to initialise.
@@ -839,6 +955,39 @@ static unsigned long iov_iter_alignment_bvec(const struct iov_iter *i)
 	return res;
 }
 
+static unsigned long iov_iter_alignment_bvecq(const struct iov_iter *iter)
+{
+	const struct bvecq *bq;
+	unsigned long res = 0;
+	unsigned int slot = iter->bvecq_slot;
+	size_t skip = iter->iov_offset;
+	size_t size = iter->count;
+
+	if (!size)
+		return res;
+
+	for (bq = iter->bvecq; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			const struct bio_vec *bvec = &bq->bv[slot];
+			size_t part = umin(bvec->bv_len - skip, size);
+
+			if (part) {
+				res |= bvec->bv_offset + skip;
+				res |= part;
+			}
+
+			size -= part;
+			if (size == 0)
+				return res;
+			skip = 0;
+		}
+
+		slot = 0;
+	}
+
+	return res;
+}
+
 unsigned long iov_iter_alignment(const struct iov_iter *i)
 {
 	if (likely(iter_is_ubuf(i))) {
@@ -854,6 +1003,8 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
 
 	if (iov_iter_is_bvec(i))
 		return iov_iter_alignment_bvec(i);
+	if (iov_iter_is_bvecq(i))
+		return iov_iter_alignment_bvecq(i);
 
 	/* With both xarray and folioq types, we're dealing with whole folios. */
 	if (iov_iter_is_folioq(i))
@@ -1066,6 +1217,38 @@ static int bvec_npages(const struct iov_iter *i, int maxpages)
 	return npages;
 }
 
+static size_t iov_npages_bvecq(const struct iov_iter *iter, size_t maxpages)
+{
+	const struct bvecq *bq;
+	unsigned int slot = iter->bvecq_slot;
+	size_t npages = 0;
+	size_t skip = iter->iov_offset;
+	size_t size = iter->count;
+
+	for (bq = iter->bvecq; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			const struct bio_vec *bvec = &bq->bv[slot];
+			size_t offs = (bvec->bv_offset + skip) % PAGE_SIZE;
+			size_t part = umin(bvec->bv_len - skip, size);
+
+			if (part) {
+				npages += DIV_ROUND_UP(offs + part, PAGE_SIZE);
+				if (npages >= maxpages)
+					goto out;
+			}
+
+			size -= part;
+			if (!size)
+				goto out;
+			skip = 0;
+		}
+
+		slot = 0;
+	}
+out:
+	return umin(npages, maxpages);
+}
+
 int iov_iter_npages(const struct iov_iter *i, int maxpages)
 {
 	if (unlikely(!i->count))
@@ -1080,6 +1263,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
 		return iov_npages(i, maxpages);
 	if (iov_iter_is_bvec(i))
 		return bvec_npages(i, maxpages);
+	if (iov_iter_is_bvecq(i))
+		return iov_npages_bvecq(i, maxpages);
 	if (iov_iter_is_folioq(i)) {
 		unsigned offset = i->iov_offset % PAGE_SIZE;
 		int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1366,6 +1551,149 @@ void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state)
 	i->nr_segs = state->nr_segs;
 }
 
+/*
+ * Count the number of virtually contiguous pages coming up next in an
+ * ITER_BVECQ iterator, up to the specified maxima.
+ */
+static unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,
+					   size_t maxsize,
+					   unsigned int maxpages)
+{
+	const struct bvecq *bvecq = iter->bvecq;
+	unsigned int slot = iter->bvecq_slot;
+	ssize_t remain = umin(maxsize, iter->count);
+	size_t count = 0, offset = iter->iov_offset;
+
+	do {
+		const struct bio_vec *bv;
+		size_t boff, blen;
+
+		if (slot >= bvecq->nr_slots) {
+			if (!bvecq->next) {
+				WARN_ON_ONCE(remain > 0);
+				break;
+			}
+			bvecq = bvecq->next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		bv = &bvecq->bv[slot++];
+		boff = bv->bv_offset;
+		blen = bv->bv_len;
+
+		if (unlikely(!bv->bv_page)) {
+			if (blen && count > 0)
+				break;
+			continue;
+		}
+		if (!PAGE_ALIGNED(boff) && count > 0)
+			break;
+
+		boff += offset;
+		blen -= offset;
+		offset = 0;
+		if (!blen)
+			continue;
+
+		blen = umin(blen, remain);
+		remain -= blen;
+		blen += offset_in_page(boff);
+		count += DIV_ROUND_UP(blen, PAGE_SIZE);
+
+		if (!PAGE_ALIGNED(blen))
+			break;
+	} while (remain > 0 && count < maxpages);
+
+	return umin(count, maxpages);
+}
+
+/*
+ * Extract a list of virtually contiguous pages from an ITER_BVECQ iterator.
+ * This does not get references on the pages, nor does it get a pin on them.
+ */
+static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
+					    struct page ***pages, size_t maxsize,
+					    unsigned int maxpages,
+					    iov_iter_extraction_t extraction_flags,
+					    size_t *offset0)
+{
+	const struct bvecq *bvecq;
+	struct page **p;
+	unsigned int slot, nr = 0;
+	size_t extracted = 0, offset;
+
+	/* Count the next run of virtually contiguous pages. */
+	maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+	if (maxpages < 0)
+		return 0;
+
+	if (!*pages) {
+		*pages = kvmalloc_array(maxpages, sizeof(struct page *), GFP_KERNEL);
+		if (!*pages)
+			return -ENOMEM;
+	}
+
+	p = *pages;
+
+	/* Now transcribe the page pointers. */
+	extracted = 0;
+	bvecq = iter->bvecq;
+	offset = iter->iov_offset;
+	slot = iter->bvecq_slot;
+
+	do {
+		const struct bio_vec *bv;
+		size_t boff, blen;
+
+		if (slot >= bvecq->nr_slots) {
+			if (!bvecq->next) {
+				WARN_ON_ONCE(extracted < iter->count);
+				break;
+			}
+			bvecq = bvecq->next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		bv = &bvecq->bv[slot];
+		boff = bv->bv_offset;
+		blen = bv->bv_len;
+
+		if (!bv->bv_page)
+			blen = 0;
+
+		if (offset < blen) {
+			size_t part = umin(maxsize - extracted, blen - offset);
+			size_t poff = (boff + offset) % PAGE_SIZE;
+			size_t pix = (boff + offset) / PAGE_SIZE;
+
+			if (poff + part > PAGE_SIZE)
+				part = PAGE_SIZE - poff;
+
+			if (!extracted)
+				*offset0 = poff;
+
+			p[nr++] = bv->bv_page + pix;
+			offset += part;
+			extracted += part;
+		}
+
+		if (offset >= blen) {
+			offset = 0;
+			slot++;
+		}
+	} while (nr < maxpages && extracted < maxsize);
+
+	iter->bvecq = bvecq;
+	iter->bvecq_slot = slot;
+	iter->iov_offset = offset;
+	iter->count -= extracted;
+	return extracted;
+}
+
 /*
  * Extract a list of contiguous pages from an ITER_FOLIOQ iterator.  This does
  * not get references on the pages, nor does it get a pin on them.
@@ -1726,6 +2054,10 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
 		return iov_iter_extract_bvec_pages(i, pages, maxsize,
 						   maxpages, extraction_flags,
 						   offset0);
+	if (iov_iter_is_bvecq(i))
+		return iov_iter_extract_bvecq_pages(i, pages, maxsize,
+						    maxpages, extraction_flags,
+						    offset0);
 	if (iov_iter_is_folioq(i))
 		return iov_iter_extract_folioq_pages(i, pages, maxsize,
 						     maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index 6ea40d2e6247..23e5a180103b 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -10,6 +10,7 @@
 #include <linux/highmem.h>
 #include <linux/kmemleak.h>
 #include <linux/bvec.h>
+#include <linux/bvecq.h>
 #include <linux/uio.h>
 #include <linux/folio_queue.h>
 
@@ -1267,6 +1268,65 @@ static ssize_t extract_kvec_to_sg(struct iov_iter *iter,
 	return ret;
 }
 
+/*
+ * Extract up to sg_max folios from an BVECQ-type iterator and add them to
+ * the scatterlist.  The pages are not pinned.
+ */
+static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
+				   ssize_t maxsize,
+				   struct sg_table *sgtable,
+				   unsigned int sg_max,
+				   iov_iter_extraction_t extraction_flags)
+{
+	const struct bvecq *bvecq = iter->bvecq;
+	struct scatterlist *sg = sgtable->sgl + sgtable->nents;
+	unsigned int slot = iter->bvecq_slot;
+	ssize_t ret = 0;
+	size_t offset = iter->iov_offset;
+
+	maxsize = umin(maxsize, iov_iter_count(iter));
+
+	while (sg_max > 0 && ret < maxsize) {
+		const struct bio_vec *bv;
+		size_t blen, part;
+
+		if (slot >= bvecq->nr_slots) {
+			if (!bvecq->next) {
+				WARN_ON_ONCE(ret < iter->count);
+				break;
+			}
+			bvecq = bvecq->next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		bv = &bvecq->bv[slot];
+		blen = bv->bv_len;
+
+		if (offset >= blen) {
+			offset = 0;
+			slot++;
+			continue;
+		}
+
+		part = umin(maxsize - ret, blen - offset);
+
+		sg_set_page(sg, bv->bv_page, part, bv->bv_offset + offset);
+		sgtable->nents++;
+		sg++;
+		sg_max--;
+		offset += part;
+		ret += part;
+	}
+
+	iter->bvecq = bvecq;
+	iter->bvecq_slot = slot;
+	iter->iov_offset = offset;
+	iter->count -= ret;
+	return ret;
+}
+
 /*
  * Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
  * the scatterlist.  The pages are not pinned.
@@ -1391,8 +1451,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
  * addition of @sg_max elements.
  *
  * The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, KVEC-, FOLIOQ- and XARRAY-type are extracted but aren't
- * pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, FOLIOQ- and XARRAY-type are extracted but
+ * aren't pinned; DISCARD-type is not supported.
  *
  * No end mark is placed on the scatterlist; that's left to the caller.
  *
@@ -1424,6 +1484,9 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
 	case ITER_KVEC:
 		return extract_kvec_to_sg(iter, maxsize, sgtable, sg_max,
 					  extraction_flags);
+	case ITER_BVECQ:
+		return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
+					   extraction_flags);
 	case ITER_FOLIOQ:
 		return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
 					    extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index d9690ba1db88..f6413e6b925a 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -12,6 +12,7 @@
 #include <linux/mm.h>
 #include <linux/uio.h>
 #include <linux/bvec.h>
+#include <linux/bvecq.h>
 #include <linux/folio_queue.h>
 #include <linux/scatterlist.h>
 #include <linux/minmax.h>
@@ -552,6 +553,183 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
 	KUNIT_SUCCEED(test);
 }
 
+static void iov_kunit_destroy_bvecq(void *data)
+{
+	struct bvecq *bq, *next;
+
+	for (bq = data; bq; bq = next) {
+		next = bq->next;
+		/* The pages are freed by vmap with VM_MAP_PUT_PAGES. */
+		kfree(bq);
+	}
+}
+
+static struct bvecq *iov_kunit_alloc_bvecq(struct kunit *test, unsigned int max_slots)
+{
+	struct bvecq *bq;
+
+	bq = kzalloc(struct_size(bq, __bv, max_slots), GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, bq);
+	bq->max_slots = max_slots;
+	bq->bv = bq->__bv;
+	bq->inline_bv = true;
+	return bq;
+}
+
+static struct bvecq *iov_kunit_create_bvecq(struct kunit *test, unsigned int max_slots)
+{
+	struct bvecq *bq;
+
+	bq = iov_kunit_alloc_bvecq(test, max_slots);
+	kunit_add_action_or_reset(test, iov_kunit_destroy_bvecq, bq);
+	return bq;
+}
+
+static void __init iov_kunit_load_bvecq(struct kunit *test,
+					struct iov_iter *iter, int dir,
+					struct bvecq *bq_head,
+					struct page **pages, size_t npages)
+{
+	struct bvecq *bq = bq_head;
+	size_t size = 0;
+
+	for (int i = 0; i < npages; i++) {
+		if (bq->nr_slots >= bq->max_slots) {
+			bq->next = iov_kunit_alloc_bvecq(test, 13);
+			bq->next->prev = bq;
+			bq = bq->next;
+		}
+		bvec_set_page(&bq->bv[bq->nr_slots], pages[i], PAGE_SIZE, 0);
+		bq->nr_slots++;
+		size += PAGE_SIZE;
+	}
+	iov_iter_bvec_queue(iter, dir, bq_head, 0, 0, size);
+}
+
+/*
+ * Test copying to a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_to_bvecq(struct kunit *test)
+{
+	const struct kvec_test_range *pr;
+	struct iov_iter iter;
+	struct bvecq *bq;
+	struct page **spages, **bpages;
+	u8 *scratch, *buffer;
+	size_t bufsize, npages, size, copied;
+	int i, patt;
+
+	bufsize = 0x100000;
+	npages = bufsize / PAGE_SIZE;
+
+	bq = iov_kunit_create_bvecq(test, 13);
+
+	scratch = iov_kunit_create_buffer(test, &spages, npages);
+	for (i = 0; i < bufsize; i++)
+		scratch[i] = pattern(i);
+
+	buffer = iov_kunit_create_buffer(test, &bpages, npages);
+	memset(buffer, 0, bufsize);
+
+	iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+	i = 0;
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+		size = pr->to - pr->from;
+		KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+		iov_iter_bvec_queue(&iter, READ, bq, 0, 0, pr->to);
+		iov_iter_advance(&iter, pr->from);
+		copied = copy_to_iter(scratch + i, size, &iter);
+
+		KUNIT_EXPECT_EQ(test, copied, size);
+		KUNIT_EXPECT_EQ(test, iter.count, 0);
+		i += size;
+		if (test->status == KUNIT_FAILURE)
+			goto stop;
+	}
+
+	/* Build the expected image in the scratch buffer. */
+	patt = 0;
+	memset(scratch, 0, bufsize);
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++)
+		for (i = pr->from; i < pr->to; i++)
+			scratch[i] = pattern(patt++);
+
+	/* Compare the images */
+	for (i = 0; i < bufsize; i++) {
+		KUNIT_EXPECT_EQ_MSG(test, buffer[i], scratch[i], "at i=%x", i);
+		if (buffer[i] != scratch[i])
+			return;
+	}
+
+stop:
+	KUNIT_SUCCEED(test);
+}
+
+/*
+ * Test copying from a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_from_bvecq(struct kunit *test)
+{
+	const struct kvec_test_range *pr;
+	struct iov_iter iter;
+	struct bvecq *bq;
+	struct page **spages, **bpages;
+	u8 *scratch, *buffer;
+	size_t bufsize, npages, size, copied;
+	int i, j;
+
+	bufsize = 0x100000;
+	npages = bufsize / PAGE_SIZE;
+
+	bq = iov_kunit_create_bvecq(test, 13);
+
+	buffer = iov_kunit_create_buffer(test, &bpages, npages);
+	for (i = 0; i < bufsize; i++)
+		buffer[i] = pattern(i);
+
+	scratch = iov_kunit_create_buffer(test, &spages, npages);
+	memset(scratch, 0, bufsize);
+
+	iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+	i = 0;
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+		size = pr->to - pr->from;
+		KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+		iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+		iov_iter_advance(&iter, pr->from);
+		copied = copy_from_iter(scratch + i, size, &iter);
+
+		KUNIT_EXPECT_EQ(test, copied, size);
+		KUNIT_EXPECT_EQ(test, iter.count, 0);
+		i += size;
+	}
+
+	/* Build the expected image in the main buffer. */
+	i = 0;
+	memset(buffer, 0, bufsize);
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+		for (j = pr->from; j < pr->to; j++) {
+			buffer[i++] = pattern(j);
+			if (i >= bufsize)
+				goto stop;
+		}
+	}
+stop:
+
+	/* Compare the images */
+	for (i = 0; i < bufsize; i++) {
+		KUNIT_EXPECT_EQ_MSG(test, scratch[i], buffer[i], "at i=%x", i);
+		if (scratch[i] != buffer[i])
+			return;
+	}
+
+	KUNIT_SUCCEED(test);
+}
+
 static void iov_kunit_destroy_xarray(void *data)
 {
 	struct xarray *xarray = data;
@@ -867,6 +1045,85 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
 	KUNIT_SUCCEED(test);
 }
 
+/*
+ * Test the extraction of ITER_BVECQ-type iterators.
+ */
+static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
+{
+	const struct kvec_test_range *pr;
+	struct iov_iter iter;
+	struct bvecq *bq;
+	struct page **bpages, *pagelist[8], **pages = pagelist;
+	ssize_t len;
+	size_t bufsize, size = 0, npages;
+	int i, from;
+
+	bufsize = 0x100000;
+	npages = bufsize / PAGE_SIZE;
+
+	bq = iov_kunit_create_bvecq(test, 13);
+
+	iov_kunit_create_buffer(test, &bpages, npages);
+	iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+		from = pr->from;
+		size = pr->to - from;
+		KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+		iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+		iov_iter_advance(&iter, from);
+
+		do {
+			size_t offset0 = LONG_MAX;
+
+			for (i = 0; i < ARRAY_SIZE(pagelist); i++)
+				pagelist[i] = (void *)(unsigned long)0xaa55aa55aa55aa55ULL;
+
+			len = iov_iter_extract_pages(&iter, &pages, 100 * 1024,
+						     ARRAY_SIZE(pagelist), 0, &offset0);
+			KUNIT_EXPECT_GE(test, len, 0);
+			if (len < 0)
+				break;
+			KUNIT_EXPECT_LE(test, len, size);
+			KUNIT_EXPECT_EQ(test, iter.count, size - len);
+			if (len == 0)
+				break;
+			size -= len;
+			KUNIT_EXPECT_GE(test, (ssize_t)offset0, 0);
+			KUNIT_EXPECT_LT(test, offset0, PAGE_SIZE);
+
+			for (i = 0; i < ARRAY_SIZE(pagelist); i++) {
+				struct page *p;
+				ssize_t part = min_t(ssize_t, len, PAGE_SIZE - offset0);
+				int ix;
+
+				KUNIT_ASSERT_GE(test, part, 0);
+				ix = from / PAGE_SIZE;
+				KUNIT_ASSERT_LT(test, ix, npages);
+				p = bpages[ix];
+				KUNIT_EXPECT_PTR_EQ(test, pagelist[i], p);
+				KUNIT_EXPECT_EQ(test, offset0, from % PAGE_SIZE);
+				from += part;
+				len -= part;
+				KUNIT_ASSERT_GE(test, len, 0);
+				if (len == 0)
+					break;
+				offset0 = 0;
+			}
+
+			if (test->status == KUNIT_FAILURE)
+				goto stop;
+		} while (iov_iter_count(&iter) > 0);
+
+		KUNIT_EXPECT_EQ(test, size, 0);
+		KUNIT_EXPECT_EQ(test, iter.count, 0);
+	}
+
+stop:
+	KUNIT_SUCCEED(test);
+}
+
 /*
  * Test the extraction of ITER_FOLIOQ-type iterators.
  */
@@ -1226,12 +1483,15 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
 	KUNIT_CASE(iov_kunit_copy_from_kvec),
 	KUNIT_CASE(iov_kunit_copy_to_bvec),
 	KUNIT_CASE(iov_kunit_copy_from_bvec),
+	KUNIT_CASE(iov_kunit_copy_to_bvecq),
+	KUNIT_CASE(iov_kunit_copy_from_bvecq),
 	KUNIT_CASE(iov_kunit_copy_to_folioq),
 	KUNIT_CASE(iov_kunit_copy_from_folioq),
 	KUNIT_CASE(iov_kunit_copy_to_xarray),
 	KUNIT_CASE(iov_kunit_copy_from_xarray),
 	KUNIT_CASE(iov_kunit_extract_pages_kvec),
 	KUNIT_CASE(iov_kunit_extract_pages_bvec),
+	KUNIT_CASE(iov_kunit_extract_pages_bvecq),
 	KUNIT_CASE(iov_kunit_extract_pages_folioq),
 	KUNIT_CASE(iov_kunit_extract_pages_xarray),
 	KUNIT_CASE(iov_kunit_iter_to_sg_kvec),



From dhowells at redhat.com  Mon Aug 10 07:47:25 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:25 +0100
Subject: [PATCH v9 08/26] netfs: Make mempool available for bvecq
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-9-dhowells@redhat.com>

Make a mempool available for allocating bvecq structs.  Use it
automatically if anything other than GFP_KERNEL (with GFP_ZONEMASK flags
masked off) is specified.  Reading from a file can use GFP_KERNEL as the
failure mode is straightforward and the same for DIO reads and writes.
When it comes to writeback, however, the writeback_iter() API does not
permit temporary failure, including ENOMEM, if WB_SYNC_ALL is set and the
caller must process all folios to completion.  (I'm not sure if EINTR
constitutes an acceptable failure).

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/bvecq.c      | 39 +++++++++++++++++++++++++++++++++------
 fs/netfs/internal.h   |  1 +
 fs/netfs/main.c       |  7 +++++++
 include/linux/bvecq.h |  3 +++
 4 files changed, 44 insertions(+), 6 deletions(-)

diff --git a/fs/netfs/bvecq.c b/fs/netfs/bvecq.c
index 00e321a796c9..0bd2ee70c656 100644
--- a/fs/netfs/bvecq.c
+++ b/fs/netfs/bvecq.c
@@ -46,8 +46,9 @@ EXPORT_SYMBOL(bvecq_dump);
  *
  * Allocate a single bvecq node and initialise the header.  A number of inline
  * slots are also allocated, rounded up to fit after the header in a power-of-2
- * slab object of up to 512 bytes (up to 29 slots on a 64-bit cpu).  The slot
- * array is not initialised.
+ * slab object of up to 512 bytes (up to 29 slots on a 64-bit cpu).  The caller
+ * should be aware that the number of slots allocated may be more or less than
+ * the number requested.  The slot array is not initialised.
  *
  * Return: The node pointer or NULL on allocation failure.
  */
@@ -58,16 +59,39 @@ struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback)
 	const size_t max_slots = (max_size - sizeof(*bq)) / sizeof(bq->__bv[0]);
 	size_t part = umin(nr_slots, max_slots);
 	size_t size = roundup_pow_of_two(struct_size(bq, __bv, part));
+	bool from_pool = false;
 
-	bq = kmalloc(size, gfp & ~GFP_ZONEMASK);
-	if (!bq)
-		return bq;
+	gfp &= ~GFP_ZONEMASK;
+
+	if (for_writeback) {
+		if (size != BVECQ_STD_SIZE) {
+			gfp_t gfp_temp = gfp;
 
+			gfp_temp |= __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN;
+			gfp_temp &= ~(__GFP_DIRECT_RECLAIM | __GFP_IO);
+			bq = kmalloc(size, gfp_temp);
+			if (bq)
+				goto success;
+		}
+
+		bq = mempool_alloc(&netfs_bvecq_pool, gfp);
+		if (!bq)
+			return bq;
+		from_pool = true;
+		size = BVECQ_STD_SIZE;
+	} else {
+		bq = kmalloc(size, gfp);
+		if (!bq)
+			return bq;
+	}
+
+success:
 	*bq = (struct bvecq) {
 		.ref		= REFCOUNT_INIT(1),
 		.bv		= bq->__bv,
 		.inline_bv	= true,
 		.max_slots	= (size - sizeof(*bq)) / sizeof(bq->__bv[0]),
+		.from_pool	= from_pool,
 	};
 	netfs_stat(&netfs_n_bvecq);
 	return bq;
@@ -247,7 +271,10 @@ void bvecq_put(struct bvecq *bq)
 			bvecq_free_slot(bq, slot);
 		next = bq->next;
 		netfs_stat_d(&netfs_n_bvecq);
-		kfree(bq);
+		if (bq->from_pool)
+			mempool_free(bq, &netfs_bvecq_pool);
+		else
+			kfree(bq);
 	}
 }
 EXPORT_SYMBOL(bvecq_put);
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 46fb92601774..c68eea2ecc60 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -43,6 +43,7 @@ extern struct list_head netfs_io_requests;
 extern spinlock_t netfs_proc_lock;
 extern mempool_t netfs_request_pool;
 extern mempool_t netfs_subrequest_pool;
+extern mempool_t netfs_bvecq_pool;
 extern mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 927badf3989d..9f72e5054aff 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -28,6 +28,7 @@ static struct kmem_cache *netfs_request_slab;
 static struct kmem_cache *netfs_subrequest_slab;
 mempool_t netfs_request_pool;
 mempool_t netfs_subrequest_pool;
+mempool_t netfs_bvecq_pool;
 mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
@@ -112,6 +113,9 @@ static int __init netfs_init(void)
 	if (mempool_init_kmalloc_pool(&netfs_folioq_pool, 100, sizeof(struct folio_queue)) < 0)
 		goto error_folioq_pool;
 
+	if (mempool_init_kmalloc_pool(&netfs_bvecq_pool, 100, BVECQ_STD_SIZE) < 0)
+		goto error_bvecq_pool;
+
 	netfs_request_slab = kmem_cache_create("netfs_request",
 					       sizeof(struct netfs_io_request), 0,
 					       SLAB_HWCACHE_ALIGN | SLAB_ACCOUNT,
@@ -164,6 +168,8 @@ static int __init netfs_init(void)
 error_reqpool:
 	kmem_cache_destroy(netfs_request_slab);
 error_req:
+	mempool_exit(&netfs_bvecq_pool);
+error_bvecq_pool:
 	mempool_exit(&netfs_folioq_pool);
 error_folioq_pool:
 	return ret;
@@ -178,6 +184,7 @@ static void __exit netfs_exit(void)
 	kmem_cache_destroy(netfs_subrequest_slab);
 	mempool_exit(&netfs_request_pool);
 	kmem_cache_destroy(netfs_request_slab);
+	mempool_exit(&netfs_bvecq_pool);
 	mempool_exit(&netfs_folioq_pool);
 }
 module_exit(netfs_exit);
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
index 3d4ab0e554d4..07c96d1fa4b2 100644
--- a/include/linux/bvecq.h
+++ b/include/linux/bvecq.h
@@ -49,15 +49,18 @@ struct bvecq {
 	enum bvecq_mem	mem_type:3;	/* What sort of memory and how to free it */
 	bool		inline_bv:1;	/* T if __bv[] is being used */
 	bool		discontig:1;	/* T if not contiguous with previous bvecq */
+	bool		from_pool:1;	/* T if bvecq from mempool */
 	struct bio_vec	*bv;		/* Pointer to array of page fragments */
 	struct bio_vec	__bv[];		/* Default array (if ->inline_bv) */
 };
 
 #if BITS_PER_LONG == 64
 /* Number of slots in __bv[] for a bvecq in a 512-byte kmalloc block. */
+#define BVECQ_STD_SIZE		512
 #define BVECQ_STD_SLOTS		29	/* 2 words/slot; 32 slots; bvecq is 6 words (3 slots) */
 #elif  BITS_PER_LONG == 32
 /* Number of slots in __bv[] for a bvecq in a 256-byte kmalloc block. */
+#define BVECQ_STD_SIZE		256
 #define BVECQ_STD_SLOTS		18	/* 3 words/slot; 21 slots; bvecq is 9 words (3 slots) */
 #else
 #error BVECQ_STD_SLOTS undetermined



From dhowells at redhat.com  Mon Aug 10 07:47:24 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:24 +0100
Subject: [PATCH v9 07/26] netfs: Add some tools for managing bvecq chains
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-8-dhowells@redhat.com>

Provide a selection of tools for managing bvec queue chains.  This
includes:

 (1) Allocation, prepopulation, expansion, shortening and refcounting of
     bvecqs and bvecq chains.

     This can be used to do things like creating an encryption buffer in
     cifs or a directory content buffer in afs.  The memory segments will
     be appropriate disposed off according to the flags on the bvecq.

 (2) Management of a bvecq chain as a rolling buffer and the management of
     positions within it.

 (3) Loading folios, slicing chains and clearing content.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/Makefile            |   1 +
 fs/netfs/bvecq.c             | 781 +++++++++++++++++++++++++++++++++++
 fs/netfs/internal.h          |   1 +
 fs/netfs/stats.c             |   4 +-
 include/linux/bvecq.h        | 304 ++++++++++++++
 include/linux/netfs.h        |   1 +
 include/trace/events/netfs.h |  24 ++
 7 files changed, 1115 insertions(+), 1 deletion(-)
 create mode 100644 fs/netfs/bvecq.c

diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index b43188d64bd8..e1f12ecb5abf 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -3,6 +3,7 @@
 netfs-y := \
 	buffered_read.o \
 	buffered_write.o \
+	bvecq.o \
 	direct_read.o \
 	direct_write.o \
 	iterator.o \
diff --git a/fs/netfs/bvecq.c b/fs/netfs/bvecq.c
new file mode 100644
index 000000000000..00e321a796c9
--- /dev/null
+++ b/fs/netfs/bvecq.c
@@ -0,0 +1,781 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Buffering helpers for bvec queues
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells at redhat.com)
+ */
+
+#include <linux/bvecq.h>
+#include "internal.h"
+
+void bvecq_dump(const struct bvecq *bq)
+{
+	int b = 0;
+
+	for (; bq; bq = bvecq_next(bq), b++) {
+		int skipz = 0;
+
+		pr_notice("BQ[%u] %u/%u fp=%llx%s\n",
+			  b, bq->nr_slots, bq->max_slots, bq->fpos,
+			  bq->discontig ? " discontig" : "");
+		for (int s = 0; s < bq->nr_slots; s++) {
+			const struct bio_vec *bv = &bq->bv[s];
+
+			if (!bv->bv_page && !bv->bv_len && skipz < 2) {
+				skipz = 1;
+				continue;
+			}
+			if (skipz == 1)
+				pr_notice("BQ[%u:00-%02u] ...\n", b, s - 1);
+			skipz = 2;
+			pr_notice("BQ[%u:%02u] %10lx %04x %04x %u\n",
+				  b, s,
+				  bv->bv_page ? page_to_pfn(bv->bv_page) : 0,
+				  bv->bv_offset, bv->bv_len,
+				  bv->bv_page ? page_count(bv->bv_page) : 0);
+		}
+	}
+}
+EXPORT_SYMBOL(bvecq_dump);
+
+/**
+ * bvecq_alloc_one - Allocate a single bvecq node with unpopulated slots
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a single bvecq node and initialise the header.  A number of inline
+ * slots are also allocated, rounded up to fit after the header in a power-of-2
+ * slab object of up to 512 bytes (up to 29 slots on a 64-bit cpu).  The slot
+ * array is not initialised.
+ *
+ * Return: The node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback)
+{
+	struct bvecq *bq;
+	const size_t max_size = 512;
+	const size_t max_slots = (max_size - sizeof(*bq)) / sizeof(bq->__bv[0]);
+	size_t part = umin(nr_slots, max_slots);
+	size_t size = roundup_pow_of_two(struct_size(bq, __bv, part));
+
+	bq = kmalloc(size, gfp & ~GFP_ZONEMASK);
+	if (!bq)
+		return bq;
+
+	*bq = (struct bvecq) {
+		.ref		= REFCOUNT_INIT(1),
+		.bv		= bq->__bv,
+		.inline_bv	= true,
+		.max_slots	= (size - sizeof(*bq)) / sizeof(bq->__bv[0]),
+	};
+	netfs_stat(&netfs_n_bvecq);
+	return bq;
+}
+EXPORT_SYMBOL(bvecq_alloc_one);
+
+/**
+ * bvecq_alloc_chain - Allocate an unpopulated bvecq chain
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a chain of bvecq nodes providing at least the requested cumulative
+ * number of slots.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback)
+{
+	struct bvecq *head = NULL, *tail = NULL;
+
+	_enter("%zu", nr_slots);
+
+	for (;;) {
+		struct bvecq *bq;
+
+		bq = bvecq_alloc_one(nr_slots, gfp, for_writeback);
+		if (!bq)
+			goto oom;
+
+		if (tail)
+			bvecq_append(tail, bq);
+		else
+			head = bq;
+		tail = bq;
+		if (tail->max_slots >= nr_slots)
+			break;
+		nr_slots -= tail->max_slots;
+	}
+
+	return head;
+oom:
+	bvecq_put(head);
+	return NULL;
+}
+EXPORT_SYMBOL(bvecq_alloc_chain);
+
+/**
+ * bvecq_alloc_buffer2 - Allocate a bvecq chain and populate with buffers
+ * @size: Target size of the buffer (can be 0 for an empty buffer)
+ * @pre_slots: Number of preamble slots to set aside
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a chain of bvecq nodes and populate the slots with sufficient pages
+ * to provide at least the requested amount of space, leaving the first
+ * @pre_slots slots unset.  The pre-slots must all fit into the the first
+ * bvecq.
+ *
+ * The pages allocated may be compound pages larger than PAGE_SIZE and thus
+ * occupy fewer slots.  The pages have their refcounts set to 1 and can be
+ * passed to MSG_SPLICE_PAGES.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,
+				  bool for_writeback)
+{
+	struct bvecq *head = NULL, *p = NULL;
+	size_t nr_per_bq = BVECQ_STD_SLOTS;
+	size_t count = pre_slots + DIV_ROUND_UP(size, PAGE_SIZE);
+
+	_enter("%zx,%zx,%u", size, count, pre_slots);
+
+	if (WARN_ON_ONCE(pre_slots > nr_per_bq))
+		return NULL;
+
+	head = bvecq_alloc_chain(count, gfp, for_writeback);
+	if (!head)
+		return NULL;
+
+	p = head;
+	do {
+		struct page **pages;
+		size_t unused, want, got, slot;
+
+		if (!count)
+			break;
+		if (WARN_ON_ONCE(!p))
+			goto oom;
+
+		if (p->nr_slots == 0) {
+			/* Need to clear pre slots and pages[], so just clear all. */
+			memset(p->bv, 0, p->max_slots * sizeof(p->bv[0]));
+			p->mem_type = BVECQ_MEM_ALLOCED;
+			p->nr_slots = pre_slots;
+			count -= pre_slots;
+			pre_slots = 0;
+			if (!count)
+				break;
+		}
+
+		if (p->nr_slots >= p->max_slots) {
+			p = p->next;
+			continue;
+		}
+		unused = p->max_slots - p->nr_slots;
+
+		pages = (struct page **)&p->bv[p->max_slots];
+		pages -= unused;
+
+		want = min(count, unused);
+		got = alloc_pages_bulk(gfp, want, pages);
+		if (!got)
+			goto oom;
+
+		slot = p->nr_slots;
+		for (int i = 0; i < got; i++) {
+			set_page_count(pages[i], 1);
+			bvec_set_page(&p->bv[slot++], pages[i], PAGE_SIZE, 0);
+		}
+
+		bvecq_filled_to(p, slot);
+		count -= got;
+	} while (count > 0);
+
+	return head;
+oom:
+	bvecq_put(head);
+	return NULL;
+}
+EXPORT_SYMBOL(bvecq_alloc_buffer2);
+
+/*
+ * Free the page pointed to by a slot as necessary.
+ */
+static void bvecq_free_slot(struct bvecq *bq, unsigned int slot)
+{
+	struct page *page = bq->bv[slot].bv_page;
+
+	if (!page)
+		return;
+
+	switch (bq->mem_type) {
+	case BVECQ_MEM_EXTERNAL:
+		break;
+	case BVECQ_MEM_PAGECACHE:
+		put_page(page);
+		break;
+	case BVECQ_MEM_GUP:
+		unpin_user_page(page);
+		break;
+	case BVECQ_MEM_ALLOCED:
+		__free_pages(page, compound_order(page));
+		break;
+	default:
+		WARN_ON_ONCE(1);
+		break;
+	}
+}
+
+/**
+ * bvecq_put - Put a ref on a bvec queue
+ * @bq: The start of the folio queue to free
+ *
+ * Put the ref(s) on the nodes in a bvec queue, freeing up the node and the
+ * page fragments it points to as the refcounts become zero.
+ */
+void bvecq_put(struct bvecq *bq)
+{
+	struct bvecq *next;
+
+	for (; bq; bq = next) {
+		if (!refcount_dec_and_test(&bq->ref))
+			break;
+		for (int slot = 0; slot < bq->nr_slots; slot++)
+			bvecq_free_slot(bq, slot);
+		next = bq->next;
+		netfs_stat_d(&netfs_n_bvecq);
+		kfree(bq);
+	}
+}
+EXPORT_SYMBOL(bvecq_put);
+
+/**
+ * bvecq_expand_buffer - Allocate buffer space into a bvec queue
+ * @_buffer: Pointer to the bvecq chain to expand (may point to a NULL; updated).
+ * @_cur_size: Current size of the buffer (updated).
+ * @size: Target size of the buffer.
+ * @gfp: The allocation constraints.
+ *
+ * Append extra pages to a buffer to increase its capacity to the @size
+ * specified.  If the current tail has space, but is not of the
+ * BVECQ_MEM_ALLOCED memory type, a separate bvecq will be allocated to hold
+ * the new memory.
+ */
+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp)
+{
+	struct bvecq *tail = *_buffer;
+
+	size = round_up(size, PAGE_SIZE);
+	if (tail)
+		while (tail->next)
+			tail = tail->next;
+
+	while (*_cur_size < size) {
+		struct page *page;
+		size_t need = size - *_cur_size;
+		int order = 0;
+
+		if (!tail || bvecq_is_full(tail) || tail->mem_type != BVECQ_MEM_ALLOCED) {
+			struct bvecq *p;
+
+			p = bvecq_alloc_one(BVECQ_STD_SLOTS, gfp, false);
+			if (!p)
+				return -ENOMEM;
+			if (tail)
+				bvecq_append(tail, p);
+			else
+				*_buffer = p;
+			tail = p;
+			p->mem_type = BVECQ_MEM_ALLOCED;
+		}
+
+		if (need > PAGE_SIZE)
+			order = umin(ilog2(need) - PAGE_SHIFT, MAX_PAGECACHE_ORDER);
+
+		page = alloc_pages(gfp | __GFP_COMP, order);
+		if (!page && order > 0) {
+			page = alloc_pages(gfp | __GFP_COMP, 0);
+			order = 0;
+		}
+		if (!page)
+			return -ENOMEM;
+
+		bvec_set_page(&tail->bv[tail->nr_slots], page, PAGE_SIZE << order, 0);
+		*_cur_size += PAGE_SIZE << order;
+		bvecq_filled_to(tail, tail->nr_slots + 1);
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL(bvecq_expand_buffer);
+
+/**
+ * bvecq_shorten_buffer - Shorten a bvec queue buffer
+ * @bq: The start of the buffer to shorten
+ * @slot: The slot to start from
+ * @size: The size to retain
+ *
+ * Shorten the content of a bvec queue down to the minimum number of slots,
+ * starting at the specified slot, to retain the specified size.
+ *
+ * Return: 0 if successful; -EMSGSIZE if there is insufficient content.
+ */
+int bvecq_shorten_buffer(struct bvecq *bq, unsigned int slot, size_t size)
+{
+	struct bvecq *next;
+
+	/* Skip through the segments we want to keep. */
+	for (; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			if (size < bq->bv[slot].bv_len)
+				goto found;
+			size -= bq->bv[slot].bv_len;
+		}
+		slot = 0;
+	}
+	if (WARN_ON_ONCE(size > 0))
+		return -EMSGSIZE;
+	return 0;
+
+found:
+	/* Shorten any partial entry and clean the rest of this bvecq. */
+	if (size > 0) {
+		bq->bv[slot].bv_len = size;
+		slot++;
+	}
+	for (int i = slot; i < bq->nr_slots; i++)
+		bvecq_free_slot(bq, i);
+	bq->nr_slots = slot;
+
+	/* Free the queue tail. */
+	next = bq->next;
+	bq->next = NULL;
+	bvecq_put(next);
+	return 0;
+}
+EXPORT_SYMBOL(bvecq_shorten_buffer);
+
+/**
+ * bvecq_buffer_init - Initialise a buffer and set position
+ * @pos: The position to point at the new buffer.
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Initialise a rolling buffer.  We allocate an unpopulated bvecq node to so
+ * that the pointers can be independently driven by the producer and the
+ * consumer.
+ *
+ * Return 0 if successful; -ENOMEM on allocation failure.
+ */
+int bvecq_buffer_init(struct bvecq_pos *pos, gfp_t gfp, bool for_writeback)
+{
+	struct bvecq *bq;
+
+	bq = bvecq_alloc_one(BVECQ_STD_SLOTS, gfp, for_writeback);
+	if (!bq)
+		return -ENOMEM;
+
+	pos->bvecq  = bq; /* Comes with a ref. */
+	pos->slot   = 0;
+	pos->offset = 0;
+	return 0;
+}
+
+/**
+ * bvecq_buffer_append - Append a new bvecq node to a buffer
+ * @pos: The position of the last node.
+ * @bq: The buffer to add.
+ *
+ * Add a new node on to the buffer chain at the specified position, either
+ * because the previous one is full or because we have a discontiguity to
+ * contend with, and update @pos to point to it.
+ */
+void bvecq_buffer_append(struct bvecq_pos *pos, struct bvecq *bq)
+{
+	struct bvecq *head = pos->bvecq;
+
+	pos->bvecq = bvecq_get(bq);
+	pos->slot = 0;
+	pos->offset = 0;
+
+	/* [!] NOTE: After we set head->next, the consumer is at liberty to
+	 * immediately delete the old head.
+	 */
+	bvecq_append(head, bq);
+	bvecq_put(head);
+}
+
+/**
+ * bvecq_pos_advance - Advance a bvecq position
+ * @pos: The position to advance.
+ * @amount: The amount of bytes to advance by.
+ *
+ * Advance the specified bvecq position by @amount bytes.  @pos is updated and
+ * bvecq ref counts may have been manipulated.  If the position hits the end of
+ * the queue, then it is left pointing beyond the last slot of the last bvecq
+ * so that it doesn't break the chain.
+ */
+void bvecq_pos_advance(struct bvecq_pos *pos, size_t amount)
+{
+	struct bvecq *bq = pos->bvecq, *next;
+	unsigned int slot = pos->slot;
+	size_t offset = pos->offset;
+
+	while (amount) {
+		size_t part;
+
+		if (!bvecq_acquire_slot(bq, slot)) {
+			next = bvecq_next(bq);
+			if (!next) {
+				WARN_ON_ONCE(amount > 0);
+				break;
+			}
+			bq = next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		part = bq->bv[slot].bv_len - offset;
+
+		if (part > amount) {
+			offset += amount;
+			break;
+		}
+		amount -= part;
+		offset = 0;
+		slot++;
+	}
+
+	pos->slot   = slot;
+	pos->offset = offset;
+	bvecq_pos_move(pos, bq);
+}
+
+/*
+ * Clear part of the memory pointed to by a bio_vec.
+ */
+static void bvec_zero(const struct bio_vec *bv, size_t offset, size_t len)
+{
+	struct page *page = bv->bv_page;
+
+	offset += bv->bv_offset;
+
+	page  += offset / PAGE_SIZE;
+	offset = offset % PAGE_SIZE;
+
+	while (len) {
+		size_t part = umin(len, PAGE_SIZE - offset);
+		char *p = kmap_local_page(page);
+
+		memset(p + offset, 0, part);
+		kunmap_local(p);
+
+		len -= part;
+		offset = 0;
+		page++;
+	}
+}
+
+/**
+ * bvecq_zero - Clear memory starting at the bvecq position.
+ * @pos: The position in the bvecq chain to start clearing.
+ * @amount: The number of bytes to clear.
+ *
+ * Clear memory fragments pointed to by a bvec queue.  @pos is updated and
+ * bvecq ref counts may have been manipulated.  If the position hits the end of
+ * the queue, then it is left pointing beyond the last slot of the last bvecq
+ * so that it doesn't break the chain.
+ *
+ * Return: The number of bytes cleared.
+ */
+ssize_t bvecq_zero(struct bvecq_pos *pos, size_t amount)
+{
+	struct bvecq *bq = pos->bvecq, *next;
+	unsigned int slot = pos->slot;
+	ssize_t cleared = 0;
+	size_t offset = pos->offset;
+
+	while (amount) {
+		const struct bio_vec *bv;
+		size_t part;
+
+		if (!bvecq_acquire_slot(bq, slot)) {
+			next = bvecq_next(bq);
+			if (!next) {
+				WARN_ON_ONCE(amount > 0);
+				break;
+			}
+			bq = next;
+			slot = 0;
+			offset = 0;
+			continue;
+		}
+
+		bv = &bq->bv[slot];
+		if (offset >= bv->bv_len) {
+			slot++;
+			offset = 0;
+			continue;
+		}
+
+		part = min(bv->bv_len - offset, amount);
+		bvec_zero(bv, offset, part);
+		cleared += part;
+		offset += part;
+		amount -= part;
+	}
+
+	pos->slot   = slot;
+	pos->offset = offset;
+	bvecq_pos_move(pos, bq);
+	return cleared;
+}
+
+/**
+ * bvecq_slice - Find a slice of a bvecq queue
+ * @pos: The position to start at.
+ * @max_size: The maximum size of the slice (or ULONG_MAX).
+ * @max_slots: The maximum number of slots in the slice (or INT_MAX).
+ * @_nr_slots: Where to put the number of slots (updated).
+ *
+ * Determine the size and number of slots that can be obtained the next slice
+ * of bvec queue up to the maximum size and slot count specified.  The slice is
+ * also limited if a discontiguity is found.
+ *
+ * @pos is updated to the end of the slice.  If the position hits the end of
+ * the queue, then it is left pointing beyond the last slot of the last bvecq
+ * so that it doesn't break the chain.
+ *
+ * Return: The number of bytes in the slice.
+ */
+size_t bvecq_slice(struct bvecq_pos *pos, size_t max_size,
+		   unsigned int max_slots, unsigned int *_nr_slots)
+{
+	struct bvecq *bq, *next;
+	unsigned int slot = pos->slot, nslots = 0;
+	size_t size = 0, offset = pos->offset;
+
+	bq = pos->bvecq;
+	for (;;) {
+		for (; slot < bvecq_nr_slots_acquire(bq); slot++) {
+			const struct bio_vec *bvec = &bq->bv[slot];
+
+			if (offset < bvec->bv_len && bvec->bv_page) {
+				size_t part = umin(bvec->bv_len - offset, max_size);
+
+				size += part;
+				offset += part;
+				max_size -= part;
+				nslots++;
+				if (!max_size || nslots >= max_slots)
+					goto out;
+			}
+			offset = 0;
+		}
+
+		/* pos->bvecq isn't allowed to go NULL as the queue may get
+		 * extended and we would lose our place.
+		 */
+		next = bvecq_next(bq);
+		if (!next)
+			break;
+		slot = 0;
+		bq = next;
+		if (bq->discontig && size > 0)
+			break;
+	}
+
+out:
+	*_nr_slots = nslots;
+	if (slot == bvecq_nr_slots_acquire(bq)) {
+		next = bvecq_next(bq);
+		if (next) {
+			bq = next;
+			slot = 0;
+			offset = 0;
+		}
+	}
+	bvecq_pos_move(pos, bq);
+	pos->slot = slot;
+	pos->offset = offset;
+	return size;
+}
+
+/**
+ * bvecq_extract - Extract a slice of a bvecq queue into a new bvecq queue
+ * @pos: The position to start at.
+ * @max_size: The maximum size of the slice (or ULONG_MAX).
+ * @max_slots: The maximum number of slots in the slice (or INT_MAX).
+ * @to: Where to put the extraction bvecq chain head (updated).
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a new bvecq and extract into it memory fragments from a slice of
+ * bvec queue, starting at @pos.  The slice is also limited if a discontiguity
+ * is found.  No refs are taken on the page.
+ *
+ * @pos is updated to the end of the slice.  If the position hits the end of
+ * the queue, then it is left pointing beyond the last slot of the last bvecq
+ * so that it doesn't break the chain.
+ *
+ * If successful, *@to is set to point to the head of the newly allocated chain
+ * and the caller inherits a ref to it.
+ *
+ * Return: The number of bytes extracted; -ENOMEM on allocation failure or -EIO
+ * if no slots were available to extract.
+ */
+ssize_t bvecq_extract(struct bvecq_pos *pos, size_t max_size, unsigned int max_slots,
+		      struct bvecq **to, bool for_writeback)
+{
+	struct bvecq_pos tmp_pos;
+	struct bvecq *src, *dst = NULL, *next;
+	unsigned int slot = pos->slot, dslot = 0, nslots;
+	ssize_t extracted = 0;
+	size_t offset = pos->offset, amount;
+
+	*to = NULL;
+	if (WARN_ON_ONCE(!max_slots))
+		max_slots = INT_MAX;
+
+	bvecq_pos_set(&tmp_pos, pos);
+	amount = bvecq_slice(&tmp_pos, max_size, max_slots, &nslots);
+	bvecq_pos_unset(&tmp_pos);
+	if (nslots == 0)
+		return -EIO;
+
+	dst = bvecq_alloc_chain(nslots, GFP_NOFS, for_writeback);
+	if (!dst)
+		return -ENOMEM;
+	*to = dst;
+	max_slots = nslots;
+	nslots = 0;
+
+	/* Transcribe the slots */
+	src = pos->bvecq;
+	for (;;) {
+		for (; slot < bvecq_nr_slots_acquire(src); slot++) {
+			const struct bio_vec *sv = &src->bv[slot];
+			struct bio_vec *dv = &dst->bv[dslot];
+
+			_debug("EXTR BQ=%x[%x] off=%zx am=%zx p=%lx",
+			       src->priv, slot, offset, amount, page_to_pfn(sv->bv_page));
+
+			if (offset < sv->bv_len && sv->bv_page) {
+				size_t part = umin(sv->bv_len - offset, amount);
+
+				bvec_set_page(dv, sv->bv_page, part,
+					      sv->bv_offset + offset);
+				extracted += part;
+				amount -= part;
+				offset += part;
+				trace_netfs_bv_slot(dst, dslot);
+				dslot++;
+				nslots++;
+				if (dslot >= dst->max_slots) {
+					bvecq_filled_to(dst, dslot);
+					dst = dst->next;
+					dslot = 0;
+				}
+				if (nslots >= max_slots)
+					goto out;
+				if (amount == 0)
+					goto out;
+			}
+			offset = 0;
+		}
+
+		/* pos->bvecq isn't allowed to go NULL as the queue may get
+		 * extended and we would lose our place.
+		 */
+		next = bvecq_next(src);
+		if (!next)
+			break;
+		slot = 0;
+		src = next;
+		if (src->discontig && extracted > 0)
+			break;
+	}
+
+out:
+	if (dst)
+		bvecq_filled_to(dst, dslot);
+	if (slot == bvecq_nr_slots_acquire(src)) {
+		next = bvecq_next(src);
+		if (next) {
+			src = next;
+			slot = 0;
+			offset = 0;
+		}
+	}
+	bvecq_pos_move(pos, src);
+	pos->slot = slot;
+	pos->offset = offset;
+	return extracted;
+}
+
+/**
+ * bvecq_load_from_ra - Allocate a bvecq chain and load from readahead
+ * @pos: Blank position object to attach the new chain to.
+ * @ractl: The readahead control context.
+ *
+ * Decant the set of folios to be read from the readahead context into a bvecq
+ * chain.  Each folio occupies one bio_vec element.
+ *
+ * Return: Amount of data loaded or -ENOMEM on allocation failure.
+ */
+ssize_t bvecq_load_from_ra(struct bvecq_pos *pos, struct readahead_control *ractl)
+{
+	XA_STATE(xas, &ractl->mapping->i_pages, ractl->_index);
+	struct folio *folio;
+	struct bvecq *bq;
+	unsigned int slot = 0;
+	size_t loaded = 0;
+	uoff_t fpos = readahead_pos(ractl);
+
+	bq = bvecq_alloc_chain(ractl->_nr_folios, GFP_KERNEL, false);
+	if (!bq)
+		return -ENOMEM;
+	bq->fpos = fpos;
+
+	pos->bvecq  = bq;
+	pos->slot   = 0;
+	pos->offset = 0;
+
+	rcu_read_lock();
+
+	xas_for_each(&xas, folio, ractl->_index + ractl->_nr_pages - 1) {
+		size_t len;
+
+		if (xas_retry(&xas, folio))
+			continue;
+		VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
+
+		len = folio_size(folio);
+		bvec_set_folio(&bq->bv[slot], folio, len, 0);
+		loaded += len;
+		fpos += len;
+		slot++;
+		trace_netfs_folio(folio, netfs_folio_trace_read);
+
+		if (slot >= bq->max_slots) {
+			bvecq_filled_to(bq, slot);
+			bq = bq->next;
+			if (!bq)
+				break;
+			slot = 0;
+			bq->fpos = fpos;
+		}
+	}
+
+	rcu_read_unlock();
+
+	if (bq)
+		bvecq_filled_to(bq, slot);
+
+	ractl->_index += ractl->_nr_pages;
+	ractl->_nr_pages = 0;
+	return loaded;
+}
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index bd8b2d633f96..46fb92601774 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -171,6 +171,7 @@ extern atomic_t netfs_n_wh_retry_write_subreq;
 extern atomic_t netfs_n_wb_lock_skip;
 extern atomic_t netfs_n_wb_lock_wait;
 extern atomic_t netfs_n_folioq;
+extern atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v);
 
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
index ab6b916addc4..84c2a4bcc762 100644
--- a/fs/netfs/stats.c
+++ b/fs/netfs/stats.c
@@ -48,6 +48,7 @@ atomic_t netfs_n_wh_retry_write_subreq;
 atomic_t netfs_n_wb_lock_skip;
 atomic_t netfs_n_wb_lock_wait;
 atomic_t netfs_n_folioq;
+atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v)
 {
@@ -90,9 +91,10 @@ int netfs_stats_show(struct seq_file *m, void *v)
 		   atomic_read(&netfs_n_rh_retry_read_subreq),
 		   atomic_read(&netfs_n_wh_retry_write_req),
 		   atomic_read(&netfs_n_wh_retry_write_subreq));
-	seq_printf(m, "Objs   : rr=%u sr=%u foq=%u wsc=%u\n",
+	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u foq=%u wsc=%u\n",
 		   atomic_read(&netfs_n_rh_rreq),
 		   atomic_read(&netfs_n_rh_sreq),
+		   atomic_read(&netfs_n_bvecq),
 		   atomic_read(&netfs_n_folioq),
 		   atomic_read(&netfs_n_wh_wstream_conflict));
 	seq_printf(m, "WbLock : skip=%u wait=%u\n",
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
index 15f16f905877..3d4ab0e554d4 100644
--- a/include/linux/bvecq.h
+++ b/include/linux/bvecq.h
@@ -53,4 +53,308 @@ struct bvecq {
 	struct bio_vec	__bv[];		/* Default array (if ->inline_bv) */
 };
 
+#if BITS_PER_LONG == 64
+/* Number of slots in __bv[] for a bvecq in a 512-byte kmalloc block. */
+#define BVECQ_STD_SLOTS		29	/* 2 words/slot; 32 slots; bvecq is 6 words (3 slots) */
+#elif  BITS_PER_LONG == 32
+/* Number of slots in __bv[] for a bvecq in a 256-byte kmalloc block. */
+#define BVECQ_STD_SLOTS		18	/* 3 words/slot; 21 slots; bvecq is 9 words (3 slots) */
+#else
+#error BVECQ_STD_SLOTS undetermined
+#endif
+
+/*
+ * Position in a bio_vec queue.  The bvecq holds a ref on the queue segment it
+ * points to.
+ */
+struct bvecq_pos {
+	struct bvecq		*bvecq;		/* The first bvecq */
+	unsigned int		offset;		/* The offset within the starting slot */
+	u16			slot;		/* The starting slot */
+};
+
+void bvecq_dump(const struct bvecq *bq);
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback);
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback);
+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,
+				  bool for_writeback);
+void bvecq_put(struct bvecq *bq);
+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp);
+int bvecq_shorten_buffer(struct bvecq *bq, unsigned int slot, size_t size);
+int bvecq_buffer_init(struct bvecq_pos *pos, gfp_t gfp, bool for_writeback);
+void bvecq_buffer_append(struct bvecq_pos *pos, struct bvecq *bq);
+void bvecq_pos_advance(struct bvecq_pos *pos, size_t amount);
+ssize_t bvecq_zero(struct bvecq_pos *pos, size_t amount);
+size_t bvecq_slice(struct bvecq_pos *pos, size_t max_size,
+		   unsigned int max_slots, unsigned int *_nr_slots);
+ssize_t bvecq_extract(struct bvecq_pos *pos, size_t max_size, unsigned int max_slots,
+		      struct bvecq **to, bool for_writeback);
+ssize_t bvecq_load_from_ra(struct bvecq_pos *pos, struct readahead_control *ractl);
+
+/**
+ * bvecq_alloc_buffer - Allocate a bvecq chain and populate with buffers
+ * @size: Target size of the buffer (can be 0 for an empty buffer)
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Wrapper around %bvecq_alloc_buffer2().
+ */
+static inline struct bvecq *bvecq_alloc_buffer(size_t size, gfp_t gfp, bool for_writeback)
+{
+	return bvecq_alloc_buffer2(size, 0, gfp, for_writeback);
+}
+
+/**
+ * bvecq_get - Get a ref on a bvecq
+ * @bq: The bvecq to get a ref on
+ */
+static inline struct bvecq *bvecq_get(struct bvecq *bq)
+{
+	refcount_inc(&bq->ref);
+	return bq;
+}
+
+/**
+ * bvecq_is_full - Determine if a bvecq is full
+ * @bvecq: The object to query
+ *
+ * Return: true if full; false if not.
+ */
+static inline bool bvecq_is_full(const struct bvecq *bvecq)
+{
+	return bvecq->nr_slots >= bvecq->max_slots;
+}
+
+/**
+ * bvecq_filled_to - Release filled slots with release barrier
+ * @bvecq: The object modified
+ * @to: The latest slot filled + 1
+ */
+static inline void bvecq_filled_to(struct bvecq *bvecq, unsigned int to)
+{
+	/* Set the slot counter after filling the slot */
+	smp_store_release(&bvecq->nr_slots, to);
+}
+
+/**
+ * bvecq_nr_slots_acquire - Get the number of filled slots with acquire barrier
+ * @bvecq: The object to query
+ *
+ * Return: The number of filled slots
+ */
+static inline unsigned int bvecq_nr_slots_acquire(const struct bvecq *bvecq)
+{
+	/* Read the slot counter before looking at the slot */
+	return smp_load_acquire(&bvecq->nr_slots);
+}
+
+/**
+ * bvecq_acquire_slot - Determine if a slot is valid with acquire barrier
+ * @bvecq: The object to query
+ * @slot: The next slot
+ *
+ * Return: true if valid; false if might not be valid
+ */
+static inline bool bvecq_acquire_slot(const struct bvecq *bvecq, unsigned int slot)
+{
+	/* Read the slot counter before looking at the slot */
+	return slot < bvecq_nr_slots_acquire(bvecq);
+}
+
+/**
+ * bvecq_append - Get the next bvecq with appropriate barrier
+ * @to: The bvecq to append to
+ * @add: The bvecq to append
+ *
+ * Attach a new bvecq to a chain using an appropriate barrier to protect the
+ * write.
+ *
+ * [!] Note that this function transfers the caller's ref to the chain.
+ */
+static inline void bvecq_append(struct bvecq *to, struct bvecq *add)
+{
+	add->prev = to;
+
+	/* Make sure the initialisation is stored before the next pointer. */
+	smp_store_release(&to->next, add);
+}
+
+/**
+ * bvecq_next - Get the next bvecq with appropriate barrier
+ * @bq: The bvecq to start from
+ *
+ * Return the next bvecq in a chain, using an appropriate barrier to protect
+ * the access.
+ */
+static inline struct bvecq *bvecq_next(const struct bvecq *bq)
+{
+	/* Read the contents of the next node after the pointer to it. */
+	return smp_load_acquire(&bq->next);
+}
+
+/**
+ * bvecq_pos_set - Set one position to be the same as another
+ * @pos: The position object to set
+ * @at: The source position.
+ *
+ * Set @pos to have the same position as @at.  This may take a ref on the
+ * bvecq pointed to.
+ */
+static inline void bvecq_pos_set(struct bvecq_pos *pos, const struct bvecq_pos *at)
+{
+	*pos = *at;
+	bvecq_get(pos->bvecq);
+}
+
+/**
+ * bvecq_pos_unset - Unset a position
+ * @pos: The position object to unset
+ *
+ * Unset @pos.  This does any needed ref cleanup.
+ */
+static inline void bvecq_pos_unset(struct bvecq_pos *pos)
+{
+	bvecq_put(pos->bvecq);
+	pos->bvecq = NULL;
+	pos->slot = 0;
+	pos->offset = 0;
+}
+
+/**
+ * bvecq_pos_transfer - Transfer one position to another, clearing the first
+ * @pos: The position object to set
+ * @from: The source position to clear.
+ *
+ * Set @pos to have the same position as @from and then clear @from.  This may
+ * transfer a ref on the bvecq pointed to.
+ */
+static inline void bvecq_pos_transfer(struct bvecq_pos *pos, struct bvecq_pos *from)
+{
+	*pos = *from;
+	from->bvecq = NULL;
+	from->slot = 0;
+	from->offset = 0;
+}
+
+/**
+ * bvecq_pos_move - Update a position to a new bvecq
+ * @pos: The position object to update.
+ * @to: The new bvecq to point at.
+ *
+ * Update @pos to point to @to if it doesn't already do so.  This may
+ * manipulate refs on the bvecqs pointed to.
+ */
+static inline void bvecq_pos_move(struct bvecq_pos *pos, struct bvecq *to)
+{
+	struct bvecq *old = pos->bvecq;
+
+	if (old != to) {
+		pos->bvecq = bvecq_get(to);
+		bvecq_put(old);
+	}
+}
+
+/**
+ * bvecq_pos_nudge - Nudge a position onto the next segment if current used up
+ * @pos: The position object to nudge.
+ *
+ * Update @pos to point to the next segment in the chain if we've used up the
+ * current segment.  This may manipulate refs on the bvecqs pointed to.
+ *
+ * Return: true if found a new segment, false if hit the end.
+ */
+static inline bool bvecq_pos_nudge(struct bvecq_pos *pos)
+{
+	struct bvecq *bq = pos->bvecq;
+
+	for (;;) {
+		if (!bvecq_acquire_slot(bq, pos->slot)) {
+			bq = bvecq_next(bq);
+			if (!bq)
+				return false;
+			bvecq_pos_move(pos, bq);
+			pos->slot = 0;
+			pos->offset = 0;
+			continue;
+		}
+		if (pos->offset >= bq->bv[pos->slot].bv_len) {
+			pos->slot++;
+			pos->offset = 0;
+			continue;
+		}
+		return true;
+	}
+}
+
+/**
+ * bvecq_pos_step - Step a position to the next slot if possible
+ * @pos: The position object to step.
+ *
+ * Update @pos to point to the next slot in the queue if not at the end.  This
+ * may manipulate refs on the bvecqs pointed to.
+ *
+ * Return: true if successful, false if was at the end.
+ */
+static inline bool bvecq_pos_step(struct bvecq_pos *pos)
+{
+	struct bvecq *bq = pos->bvecq, *next;
+
+	pos->slot++;
+	pos->offset = 0;
+	if (pos->slot < bq->nr_slots)
+		return true;
+	next = bvecq_next(bq);
+	if (!next)
+		return false;
+	bvecq_pos_move(pos, next);
+	pos->slot = 0;
+	return true;
+}
+
+/**
+ * bvecq_delete_spent - Delete the bvecq at the front if possible
+ * @pos: The position object to update.
+ *
+ * Delete the used up bvecq at the front of the queue that @pos points to if it
+ * is not the last node in the queue; if it is the last node in the queue, it
+ * is kept so that the queue doesn't become detached from the other end.  This
+ * may manipulate refs on the bvecqs pointed to.  It is also possible that the
+ * producer will fill more slots in the current bvecq.
+ *
+ * Also, we have to be very careful: the consumer can catch the producer, which
+ * could lead to us having nothing left in the queue, causing the front and
+ * back pointers to end up on different tracks.  To avoid this, we must always
+ * keep at least one segment in the queue.
+ *
+ * The caller must reload from @pos after calling this.
+ *
+ * Return: true if there's more available; false if not.
+ */
+static inline bool bvecq_delete_spent(struct bvecq_pos *pos)
+{
+	struct bvecq *spent = pos->bvecq;
+	struct bvecq *next;
+	unsigned int slot = pos->slot;
+
+again:
+	/* Read the contents of the queue node after the pointer to it. */
+	next = bvecq_next(spent);
+	if (!next)
+		return false; /* Nothing more to consume at the moment. */
+	if (slot < bvecq_nr_slots_acquire(spent))
+		return true; /* The producer added more. */
+	next->prev = NULL;
+	spent->next = NULL;
+	bvecq_put(spent);
+	pos->bvecq = next; /* We take spent's ref. */
+	pos->slot = 0;
+	pos->offset = 0;
+	if (!bvecq_acquire_slot(next, 0)) {
+		spent = next;
+		slot = 0;
+		goto again;
+	}
+	return true;
+}
+
 #endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index f9dcf3f31cb8..2562bf1e4fe4 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -17,6 +17,7 @@
 #include <linux/workqueue.h>
 #include <linux/fs.h>
 #include <linux/pagemap.h>
+#include <linux/bvecq.h>
 #include <linux/uio.h>
 #include <linux/rolling_buffer.h>
 
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 9bda9302be90..5354a0439324 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -789,6 +789,30 @@ TRACE_EVENT(netfs_folioq,
 		      __print_symbolic(__entry->trace, netfs_folioq_traces))
 	    );
 
+TRACE_EVENT(netfs_bv_slot,
+	    TP_PROTO(const struct bvecq *bq, int slot),
+
+	    TP_ARGS(bq, slot),
+
+	    TP_STRUCT__entry(
+		    __field(unsigned long,		pfn)
+		    __field(unsigned int,		offset)
+		    __field(unsigned int,		len)
+		    __field(unsigned int,		slot)
+			     ),
+
+	    TP_fast_assign(
+		    __entry->slot = slot;
+		    __entry->pfn = page_to_pfn(bq->bv[slot].bv_page);
+		    __entry->offset = bq->bv[slot].bv_offset;
+		    __entry->len = bq->bv[slot].bv_len;
+			   ),
+
+	    TP_printk("bq[%x] p=%lx %x-%x",
+		      __entry->slot,
+		      __entry->pfn, __entry->offset, __entry->offset + __entry->len)
+	    );
+
 #undef EM
 #undef E_
 #endif /* _TRACE_NETFS_H */



From dhowells at redhat.com  Mon Aug 10 07:47:26 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:26 +0100
Subject: [PATCH v9 09/26] netfs: Add a function to extract from an iter into a bvecq
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-10-dhowells@redhat.com>

Add a function to extract a slice of data from an iterator of any type into
a bvec queue chain.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/iterator.c   | 146 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/netfs.h |   3 +
 2 files changed, 149 insertions(+)

diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index b375567e0520..b977a759a5d9 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -13,6 +13,152 @@
 #include <linux/netfs.h>
 #include "internal.h"
 
+/**
+ * netfs_extract_iter - Extract virtually contiguous pages from an iterator into a bvecq
+ * @orig: The original iterator
+ * @max_len: Maximum number of bytes to extract
+ * @max_pages: Maximum number of pages to extract
+ * @fpos: Starting file position to label the bvecq with
+ * @_bvecq_head: Where to cache the bvec queue
+ * @extraction_flags: Flags to qualify the request
+ * @gfp: Allocation mode for bvecq structs.
+ *
+ * Extract virtually contiguous page fragments from the source iterator up to
+ * the given maxima and build bvec queue that refers to all of those bits.
+ * This allows the original iterator to disposed of.
+ *
+ * @extraction_flags can have ITER_ALLOW_P2PDMA set to request peer-to-peer DMA be
+ * allowed on the pages extracted.
+ *
+ * On success or partial success, the amount of data in the bvec is returned,
+ * the original iterator will have been advanced by the amount extracted.
+ *
+ * If an error occurs and no pages are extracted, an error will be returned and
+ * any allocated bvecq will be freed.  If there is no data to be extracted (or
+ * @max_len or @max_pages are zero), a single empty bvecq will be returned.
+ *
+ * The bvecq segments are marked with indications on how to get clean up the
+ * extracted fragments.
+ */
+ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
+			   unsigned long long fpos, struct bvecq **_bvecq_head,
+			   iov_iter_extraction_t extraction_flags, gfp_t gfp)
+{
+	struct bvecq *bq_tail = NULL, *bq;
+	ssize_t ret = 0;
+	size_t extracted = 0;
+
+	_enter("{%u,%zx},%zx", orig->iter_type, orig->count, max_len);
+
+	*_bvecq_head = NULL;
+	if (max_len > orig->count)
+		max_len = orig->count;
+	if (!max_len || !max_pages)
+		goto alloc_empty;
+	if (WARN_ON_ONCE(max_pages > INT_MAX))
+		max_pages = INT_MAX; /* Protect iov_iter_npages(). */
+
+	max_pages = iov_iter_npages(orig, max_pages);
+	if (!max_pages)
+		goto alloc_empty;
+
+	do {
+		bq = bvecq_alloc_one(max_pages, gfp, false);
+		if (!bq) {
+			ret = -ENOMEM;
+			break;
+		}
+		if (user_backed_iter(orig))
+			bq->mem_type = iov_iter_extract_will_pin(orig) ?
+				BVECQ_MEM_GUP : BVECQ_MEM_PAGECACHE;
+		bq->fpos = fpos + extracted;
+
+		if (bq_tail)
+			bvecq_append(bq_tail, bq);
+		else
+			*_bvecq_head = bq;
+		bq_tail = bq;
+
+		if (max_len == 0)
+			break;
+
+		struct bio_vec *bv = bq->bv;
+		unsigned int slot = 0;
+		do {
+			struct page **pages;
+			ssize_t got;
+			size_t offset;
+			size_t space = bq->max_slots - slot;
+			size_t bv_size = array_size(bq->max_slots, sizeof(*bv));
+			size_t pg_size = array_size(space, sizeof(*pages));
+
+			/* Put the page list at the end of the bvec list
+			 * storage.  bvec elements are larger than page
+			 * pointers, so as long as we work 0->last, we should
+			 * be fine.
+			 */
+			pages = (void *)bv + bv_size - pg_size;
+
+			got = iov_iter_extract_pages(orig, &pages, max_len,
+						     min(space, max_pages),
+						     extraction_flags, &offset);
+			if (got < 0) {
+				ret = got;
+				goto out;
+			}
+
+			if (got == 0) {
+				pr_err("extract_pages gave nothing from %zu, %zu\n",
+				       extracted, max_len);
+				ret = -EIO;
+				goto out;
+			}
+
+			if (WARN(got > max_len,
+				 "%s: extract_pages overrun %zd > %zu bytes\n",
+				 __func__, got, max_len)) {
+				ret = -EIO;
+				goto out;
+			}
+
+			extracted += got;
+			max_len -= got;
+
+			do {
+				size_t len = umin(got, PAGE_SIZE - offset);
+
+				BUG_ON(slot >= bq->max_slots);
+
+				bvec_set_page(&bq->bv[slot], *pages++, len, offset);
+				slot++;
+				max_pages--;
+				got -= len;
+				offset = 0;
+			} while (got > 0);
+
+			bvecq_filled_to(bq, slot);
+		} while (max_len > 0 && max_pages > 0 && !bvecq_is_full(bq));
+
+	} while (max_len > 0 && max_pages > 0);
+
+out:
+	if (extracted || ret == 0)
+		return extracted;
+	bvecq_put(*_bvecq_head);
+	*_bvecq_head = NULL;
+	return ret;
+
+alloc_empty:
+	bq = bvecq_alloc_one(1, gfp, false);
+	if (!bq)
+		return -ENOMEM;
+	bq->fpos = fpos;
+	*_bvecq_head = bq;
+	return 0;
+
+}
+EXPORT_SYMBOL_GPL(netfs_extract_iter);
+
 /**
  * netfs_extract_user_iter - Extract the pages from a user iterator into a bvec
  * @orig: The original iterator
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 2562bf1e4fe4..6a4d4fd2551d 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -440,6 +440,9 @@ void netfs_get_subrequest(struct netfs_io_subrequest *subreq,
 			  enum netfs_sreq_ref_trace what);
 void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 			  enum netfs_sreq_ref_trace what);
+ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
+			   unsigned long long fpos, struct bvecq **_bvecq_head,
+			   iov_iter_extraction_t extraction_flags, gfp_t gfp);
 ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
 				struct iov_iter *new,
 				iov_iter_extraction_t extraction_flags);



From dhowells at redhat.com  Mon Aug 10 07:47:27 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:27 +0100
Subject: [PATCH v9 10/26] afs: Use a bvecq to hold dir content rather than folioq
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-11-dhowells@redhat.com>

Use a bvecq to hold the contents of a directory rather than the folioq so
that the latter can be phased out.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: linux-afs at lists.infradead.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/afs/dir.c           |  35 +++++----
 fs/afs/dir_edit.c      |  42 +++++------
 fs/afs/dir_search.c    |  33 ++++-----
 fs/afs/inode.c         |   2 +-
 fs/afs/internal.h      |   6 +-
 fs/afs/symlink.c       |  34 ++++-----
 fs/netfs/write_issue.c | 163 ++++++-----------------------------------
 7 files changed, 93 insertions(+), 222 deletions(-)

diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 6df56fe9163f..547faa06c80b 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -140,9 +140,9 @@ static void afs_dir_dump(struct afs_vnode *dvnode)
 	pr_warn("DIR %llx:%llx is=%llx\n",
 		dvnode->fid.vid, dvnode->fid.vnode, i_size);
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
-	iterate_folioq(&iter, iov_iter_count(&iter), NULL, NULL,
-		       afs_dir_dump_step);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	iterate_bvecq(&iter, iov_iter_count(&iter), NULL, NULL,
+		      afs_dir_dump_step);
 }
 
 /*
@@ -203,9 +203,9 @@ static int afs_dir_check(struct afs_vnode *dvnode)
 	if (unlikely(!i_size))
 		return 0;
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
-	checked = iterate_folioq(&iter, iov_iter_count(&iter), dvnode, NULL,
-				 afs_dir_check_step);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	checked = iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, NULL,
+				afs_dir_check_step);
 	if (checked != i_size) {
 		afs_dir_dump(dvnode);
 		return -EIO;
@@ -250,15 +250,14 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
 	if (dvnode->directory_size < i_size) {
 		size_t cur_size = dvnode->directory_size;
 
-		ret = netfs_alloc_folioq_buffer(NULL,
-						&dvnode->directory, &cur_size, i_size,
-						mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size, i_size,
+					  GFP_KERNEL);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			return ret;
 	}
 
-	iov_iter_folio_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
+	iov_iter_bvec_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
 
 	/* AFS requires us to perform the read of a directory synchronously as
 	 * a single unit to avoid issues with the directory contents being
@@ -294,8 +293,8 @@ static ssize_t afs_read_single(struct afs_vnode *dvnode, struct file *file)
 }
 
 /*
- * Read the directory into a folio_queue buffer in one go, scrubbing the
- * previous contents.  We return -ESTALE if the caller needs to call us again.
+ * Read the directory into the buffer in one go, scrubbing the previous
+ * contents.  We return -ESTALE if the caller needs to call us again.
  */
 ssize_t afs_read_dir(struct afs_vnode *dvnode, struct file *file)
 	__acquires(&dvnode->validate_lock)
@@ -483,7 +482,7 @@ static size_t afs_dir_iterate_step(void *iter_base, size_t progress, size_t len,
 }
 
 /*
- * Iterate through the directory folios.
+ * Iterate through the directory content.
  */
 static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_ctx)
 {
@@ -498,11 +497,11 @@ static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_c
 	if (i_size <= 0 || dir_ctx->pos >= i_size)
 		return 0;
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
 	iov_iter_advance(&iter, round_down(dir_ctx->pos, AFS_DIR_BLOCK_SIZE));
 
-	iterate_folioq(&iter, iov_iter_count(&iter), dvnode, &ctx,
-		       afs_dir_iterate_step);
+	iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, &ctx,
+		      afs_dir_iterate_step);
 
 	if (ctx.error == -ESTALE)
 		afs_invalidate_dir(dvnode, afs_dir_invalid_iter_stale);
@@ -2228,8 +2227,8 @@ static int afs_dir_writepages(struct address_space *mapping,
 	}
 
 	if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
-		iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
-				     i_size_read(&dvnode->netfs.inode));
+		iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
+				    i_size_read(&dvnode->netfs.inode));
 		ret = netfs_writeback_single(mapping, wbc, &iter);
 		if (ret == 1)
 			ret = 0; /* Skipped write due to lock conflict. */
diff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c
index 3ead36a07048..01c6265e8865 100644
--- a/fs/afs/dir_edit.c
+++ b/fs/afs/dir_edit.c
@@ -110,9 +110,8 @@ static void afs_clear_contig_bits(union afs_xdr_dir_block *block,
  */
 static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, size_t block)
 {
-	struct folio_queue *fq;
 	struct afs_vnode *dvnode = iter->dvnode;
-	struct folio *folio;
+	struct bvecq *bq;
 	size_t blpos = block * AFS_DIR_BLOCK_SIZE;
 	size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
 	int ret;
@@ -120,41 +119,38 @@ static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, siz
 	if (dvnode->directory_size < blend) {
 		size_t cur_size = dvnode->directory_size;
 
-		ret = netfs_alloc_folioq_buffer(
-			NULL, &dvnode->directory, &cur_size, blend,
-			mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size, blend,
+					  GFP_KERNEL);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			goto fail;
 	}
 
-	fq = iter->fq;
-	if (!fq)
-		fq = dvnode->directory;
+	bq = iter->bq;
+	if (!bq)
+		bq = dvnode->directory;
 
-	/* Search the folio queue for the folio containing the block... */
-	for (; fq; fq = fq->next) {
-		for (int s = iter->fq_slot; s < folioq_count(fq); s++) {
-			size_t fsize = folioq_folio_size(fq, s);
+	/* Search the contents for the region containing the block... */
+	for (; bq; bq = bq->next) {
+		for (int s = iter->bq_slot; s < bq->nr_slots; s++) {
+			struct bio_vec *bv = &bq->bv[s];
+			size_t bsize = bv->bv_len;
 
-			if (blend <= fpos + fsize) {
+			if (blend <= fpos + bsize) {
 				/* ... and then return the mapped block. */
-				folio = folioq_folio(fq, s);
-				if (WARN_ON_ONCE(folio_pos(folio) != fpos))
-					goto fail;
-				iter->fq = fq;
-				iter->fq_slot = s;
+				iter->bq = bq;
+				iter->bq_slot = s;
 				iter->fpos = fpos;
-				return kmap_local_folio(folio, blpos - fpos);
+				return bvec_kmap_partial(bv, blpos - fpos);
 			}
-			fpos += fsize;
+			fpos += bsize;
 		}
-		iter->fq_slot = 0;
+		iter->bq_slot = 0;
 	}
 
 fail:
-	iter->fq = NULL;
-	iter->fq_slot = 0;
+	iter->bq = NULL;
+	iter->bq_slot = 0;
 	afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
 	return NULL;
 }
diff --git a/fs/afs/dir_search.c b/fs/afs/dir_search.c
index 104411c0692f..01751c680115 100644
--- a/fs/afs/dir_search.c
+++ b/fs/afs/dir_search.c
@@ -66,12 +66,11 @@ bool afs_dir_init_iter(struct afs_dir_iter *iter, const struct qstr *name)
  */
 union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t block)
 {
-	struct folio_queue *fq = iter->fq;
 	struct afs_vnode *dvnode = iter->dvnode;
-	struct folio *folio;
+	struct bvecq *bq = iter->bq;
 	size_t blpos = block * AFS_DIR_BLOCK_SIZE;
 	size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
-	int slot = iter->fq_slot;
+	int slot = iter->bq_slot;
 
 	_enter("%zx,%d", block, slot);
 
@@ -83,36 +82,34 @@ union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t bl
 	if (dvnode->directory_size < blend)
 		goto fail;
 
-	if (!fq || blpos < fpos) {
-		fq = dvnode->directory;
+	if (!bq || blpos < fpos) {
+		bq = dvnode->directory;
 		slot = 0;
 		fpos = 0;
 	}
 
 	/* Search the folio queue for the folio containing the block... */
-	for (; fq; fq = fq->next) {
-		for (; slot < folioq_count(fq); slot++) {
-			size_t fsize = folioq_folio_size(fq, slot);
+	for (; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			struct bio_vec *bv = &bq->bv[slot];
+			size_t bsize = bv->bv_len;
 
-			if (blend <= fpos + fsize) {
+			if (blend <= fpos + bsize) {
 				/* ... and then return the mapped block. */
-				folio = folioq_folio(fq, slot);
-				if (WARN_ON_ONCE(folio_pos(folio) != fpos))
-					goto fail;
-				iter->fq = fq;
-				iter->fq_slot = slot;
+				iter->bq = bq;
+				iter->bq_slot = slot;
 				iter->fpos = fpos;
-				iter->block = kmap_local_folio(folio, blpos - fpos);
+				iter->block = bvec_kmap_partial(bv, blpos - fpos);
 				return iter->block;
 			}
-			fpos += fsize;
+			fpos += bsize;
 		}
 		slot = 0;
 	}
 
 fail:
-	iter->fq = NULL;
-	iter->fq_slot = 0;
+	iter->bq = NULL;
+	iter->bq_slot = 0;
 	afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
 	return NULL;
 }
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 14f39a9bea6c..634fbf8eb212 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -683,7 +683,7 @@ void afs_evict_inode(struct inode *inode)
 	flush_delayed_work(&vnode->lock_work);
 	netfs_wait_for_outstanding_io(inode);
 	truncate_inode_pages_final(&inode->i_data);
-	netfs_free_folioq_buffer(vnode->directory);
+	bvecq_put(vnode->directory);
 	if (vnode->symlink)
 		afs_evict_symlink(vnode);
 
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 290873bac89b..556e7e7777f2 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -710,7 +710,7 @@ struct afs_vnode {
 #define AFS_VNODE_MODIFYING	10		/* Set if we're performing a modification op */
 #define AFS_VNODE_DIR_READ	11		/* Set if we've read a dir's contents */
 
-	struct folio_queue	*directory;	/* Directory contents */
+	struct bvecq		*directory;	/* Directory contents */
 	struct afs_symlink __rcu *symlink;	/* Symlink content */
 	struct list_head	wb_keys;	/* List of keys available for writeback */
 	struct list_head	pending_locks;	/* locks waiting to be granted */
@@ -991,9 +991,9 @@ static inline void afs_invalidate_cache(struct afs_vnode *vnode, unsigned int fl
 struct afs_dir_iter {
 	struct afs_vnode	*dvnode;
 	union afs_xdr_dir_block *block;
-	struct folio_queue	*fq;
+	struct bvecq		*bq;
 	unsigned int		fpos;
-	int			fq_slot;
+	int			bq_slot;
 	unsigned int		loop_check;
 	u8			nr_slots;
 	u8			bucket;
diff --git a/fs/afs/symlink.c b/fs/afs/symlink.c
index 16b4823cb7b7..7fd4f985c441 100644
--- a/fs/afs/symlink.c
+++ b/fs/afs/symlink.c
@@ -56,7 +56,6 @@ void afs_evict_symlink(struct afs_vnode *vnode)
 void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)
 {
 	struct afs_symlink *symlink = op->create.symlink;
-	size_t dsize = 0;
 	size_t size = strlen(symlink->content) + 1;
 	char *p;
 
@@ -66,13 +65,15 @@ void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)
 	if (!fscache_cookie_enabled(netfs_i_cookie(&vnode->netfs)))
 		return;
 
-	if (netfs_alloc_folioq_buffer(NULL, &vnode->directory, &dsize, size,
-				      mapping_gfp_mask(vnode->netfs.inode.i_mapping)) < 0)
+	vnode->directory = bvecq_alloc_buffer(PAGE_SIZE, GFP_KERNEL, false);
+	if (!vnode->directory)
 		return;
 
-	vnode->directory_size = dsize;
-	p = kmap_local_folio(folioq_folio(vnode->directory, 0), 0);
+	vnode->directory_size = size;
+	p = bvec_kmap_partial(&vnode->directory->bv[0], 0);
 	memcpy(p, symlink->content, size);
+	if (size < PAGE_SIZE)
+		memset(p + size, 0, PAGE_SIZE - size);
 	kunmap_local(p);
 	netfs_single_mark_inode_dirty(&vnode->netfs.inode);
 }
@@ -94,17 +95,12 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
 	}
 
 	if (!vnode->directory) {
-		size_t cur_size = 0;
-
-		ret = netfs_alloc_folioq_buffer(NULL,
-						&vnode->directory, &cur_size, PAGE_SIZE,
-						mapping_gfp_mask(vnode->netfs.inode.i_mapping));
-		vnode->directory_size = PAGE_SIZE - 1;
-		if (ret < 0)
-			return ret;
+		vnode->directory = bvecq_alloc_buffer(PAGE_SIZE, GFP_KERNEL, false);
+		if (!vnode->directory)
+			return -ENOMEM;
 	}
 
-	iov_iter_folio_queue(&iter, ITER_DEST, vnode->directory, 0, 0, PAGE_SIZE);
+	iov_iter_bvec_queue(&iter, ITER_DEST, vnode->directory, 0, 0, PAGE_SIZE);
 
 	/* AFS requires us to perform the read of a symlink as a single unit to
 	 * avoid issues with the content being changed between reads.
@@ -127,7 +123,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
 		refcount_set(&symlink->ref, 1);
 		symlink->content[i_size] = 0;
 
-		const char *s = kmap_local_folio(folioq_folio(vnode->directory, 0), 0);
+		const char *s = bvec_kmap_partial(&vnode->directory->bv[0], 0);
 
 		memcpy(symlink->content, s, i_size);
 		kunmap_local(s);
@@ -136,7 +132,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
 	}
 
 	if (!fscache_cookie_enabled(netfs_i_cookie(&vnode->netfs))) {
-		netfs_free_folioq_buffer(vnode->directory);
+		bvecq_put(vnode->directory);
 		vnode->directory = NULL;
 		vnode->directory_size = 0;
 	}
@@ -249,14 +245,14 @@ int afs_symlink_writepages(struct address_space *mapping,
 
 	if (vnode->directory &&
 	    atomic64_read(&vnode->cb_expires_at) != AFS_NO_CB_PROMISE) {
-		iov_iter_folio_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
-				     i_size_read(&vnode->netfs.inode));
+		iov_iter_bvec_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
+				    i_size_read(&vnode->netfs.inode));
 		ret = netfs_writeback_single(mapping, wbc, &iter);
 	}
 
 	if (ret == 0) {
 		netfs_wb_begin(&vnode->netfs, false);
-		netfs_free_folioq_buffer(vnode->directory);
+		bvecq_put(vnode->directory);
 		vnode->directory = NULL;
 		vnode->directory_size = 0;
 		netfs_wb_end(&vnode->netfs);
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 2d9cfcd43658..11f59e5a9953 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -708,129 +708,11 @@ ssize_t netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_c
 	return ret;
 }
 
-/*
- * Write some of a pending folio data back to the server and/or the cache.
- */
-static int netfs_write_folio_single(struct netfs_io_request *wreq,
-				    struct folio *folio)
-{
-	struct netfs_io_stream *upload = &wreq->io_streams[0];
-	struct netfs_io_stream *cache  = &wreq->io_streams[1];
-	struct netfs_io_stream *stream;
-	size_t iter_off = 0;
-	size_t fsize = folio_size(folio), flen;
-	loff_t fpos = folio_pos(folio);
-	ssize_t ret;
-	bool to_eof = false;
-	bool no_debug = false;
-
-	_enter("");
-
-	flen = folio_size(folio);
-	if (flen > wreq->i_size - fpos) {
-		flen = wreq->i_size - fpos;
-		folio_zero_segment(folio, flen, fsize);
-		to_eof = true;
-	} else if (flen == wreq->i_size - fpos) {
-		to_eof = true;
-	}
-
-	_debug("folio %zx/%zx", flen, fsize);
-
-	if (!upload->avail && !cache->avail) {
-		trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
-		return 0;
-	}
-
-	if (!upload->construct)
-		trace_netfs_folio(folio, netfs_folio_trace_store);
-	else
-		trace_netfs_folio(folio, netfs_folio_trace_store_plus);
-
-	/* Attach the folio to the rolling buffer. */
-	folio_get(folio);
-	ret = rolling_buffer_append(&wreq->buffer, folio, NETFS_ROLLBUF_PUT_MARK, wreq->gfp);
-	if (ret < 0) {
-		folio_put(folio);
-		return ret;
-	}
-
-	/* Move the submission point forward to allow for write-streaming data
-	 * not starting at the front of the page.  We don't do write-streaming
-	 * with the cache as the cache requires DIO alignment.
-	 *
-	 * Also skip uploading for data that's been read and just needs copying
-	 * to the cache.
-	 */
-	for (int s = 0; s < NR_IO_STREAMS; s++) {
-		stream = &wreq->io_streams[s];
-		stream->submit_off = 0;
-		stream->submit_len = flen;
-		if (!stream->avail) {
-			stream->submit_off = UINT_MAX;
-			stream->submit_len = 0;
-		}
-	}
-
-	/* Attach the folio to one or more subrequests.  For a big folio, we
-	 * could end up with thousands of subrequests if the wsize is small -
-	 * but we might need to wait during the creation of subrequests for
-	 * network resources (eg. SMB credits).
-	 */
-	for (;;) {
-		ssize_t part;
-		size_t lowest_off = ULONG_MAX;
-		int choose_s = -1;
-
-		/* Always add to the lowest-submitted stream first. */
-		for (int s = 0; s < NR_IO_STREAMS; s++) {
-			stream = &wreq->io_streams[s];
-			if (stream->submit_len > 0 &&
-			    stream->submit_off < lowest_off) {
-				lowest_off = stream->submit_off;
-				choose_s = s;
-			}
-		}
-
-		if (choose_s < 0)
-			break;
-		stream = &wreq->io_streams[choose_s];
-
-		/* Advance the iterator(s). */
-		if (stream->submit_off > iter_off) {
-			rolling_buffer_advance(&wreq->buffer, stream->submit_off - iter_off);
-			iter_off = stream->submit_off;
-		}
-
-		atomic64_set(&wreq->issued_to, fpos + stream->submit_off);
-		stream->submit_extendable_to = fsize - stream->submit_off;
-		part = netfs_advance_write(wreq, stream, fpos + stream->submit_off,
-					   stream->submit_len, to_eof);
-		stream->submit_off += part;
-		if (part > stream->submit_len)
-			stream->submit_len = 0;
-		else
-			stream->submit_len -= part;
-		if (part > 0)
-			no_debug = true;
-	}
-
-	wreq->buffer.iter.iov_offset = 0;
-	if (fsize > iter_off)
-		rolling_buffer_advance(&wreq->buffer, fsize - iter_off);
-	atomic64_set(&wreq->issued_to, fpos + fsize);
-
-	if (!no_debug)
-		kdebug("R=%x: No submit", wreq->debug_id);
-	_leave(" = 0");
-	return 0;
-}
-
 /**
  * netfs_writeback_single - Write back a monolithic payload
  * @mapping: The mapping to write from
  * @wbc: Hints from the VM
- * @iter: Data to write, must be ITER_FOLIOQ.
+ * @iter: Data to write.
  *
  * Write a monolithic, non-pagecache object back to the server and/or
  * the cache.
@@ -844,13 +726,8 @@ int netfs_writeback_single(struct address_space *mapping,
 {
 	struct netfs_io_request *wreq;
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
-	struct folio_queue *fq;
-	size_t size = iov_iter_count(iter);
 	int ret;
 
-	if (WARN_ON_ONCE(!iov_iter_is_folioq(iter)))
-		return -EIO;
-
 	if (!netfs_wb_begin(ictx, wbc->sync_mode == WB_SYNC_NONE)) {
 		/* The VFS will have undirtied the inode. */
 		netfs_single_mark_inode_dirty(&ictx->inode);
@@ -863,37 +740,43 @@ int netfs_writeback_single(struct address_space *mapping,
 		goto couldnt_start;
 	}
 
+	wreq->buffer.iter = *iter;
+	wreq->len = iov_iter_count(iter);
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback_single);
 	netfs_stat(&netfs_n_wh_writepages);
 
-	if (__test_and_set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
+	if (test_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
 		wreq->netfs_ops->begin_writeback(wreq);
 
-	for (fq = (struct folio_queue *)iter->folioq; fq; fq = fq->next) {
-		for (int slot = 0; slot < folioq_count(fq); slot++) {
-			struct folio *folio = folioq_folio(fq, slot);
-			size_t part = umin(folioq_folio_size(fq, slot), size);
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_subrequest *subreq;
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
 
-			_debug("wbiter %lx %llx", folio->index, atomic64_read(&wreq->issued_to));
+		if (!stream->avail)
+			continue;
 
-			ret = netfs_write_folio_single(wreq, folio);
-			if (ret < 0)
-				goto stop;
-			size -= part;
-			if (size <= 0)
-				goto stop;
-		}
+		netfs_prepare_write(wreq, stream, 0);
+
+		subreq = stream->construct;
+		subreq->len = wreq->len;
+		stream->submit_len = subreq->len;
+		stream->submit_extendable_to = round_up(wreq->len, PAGE_SIZE);
+
+		netfs_issue_write(wreq, stream);
 	}
 
-stop:
-	for (int s = 0; s < NR_IO_STREAMS; s++)
-		netfs_issue_write(wreq, &wreq->io_streams[s]);
 	smp_wmb(); /* Write lists before ALL_QUEUED. */
 	set_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags);
 
 	netfs_wake_collector(wreq);
 
+	/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to
+	 * wait before modifying.
+	 */
+	ret = netfs_wait_for_write(wreq);
+
 	netfs_put_request(wreq, netfs_rreq_trace_put_return);
 	_leave(" = %d", ret);
 	return ret;



From dhowells at redhat.com  Mon Aug 10 07:47:28 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:28 +0100
Subject: [PATCH v9 11/26] cifs: Use a bvecq for buffering instead of a folioq
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-12-dhowells@redhat.com>

Use a bvecq for internal buffering for crypto purposes instead of a folioq
so that the latter can be phased out.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/smb/client/cifsglob.h |  2 +-
 fs/smb/client/smb2ops.c  | 76 +++++++++++++++++++---------------------
 2 files changed, 37 insertions(+), 41 deletions(-)

diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 79e4e84f8985..c8b1c23ef494 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -289,7 +289,7 @@ struct smb_rqst {
 	struct kvec	*rq_iov;	/* array of kvecs */
 	unsigned int	rq_nvec;	/* number of kvecs in array */
 	struct iov_iter	rq_iter;	/* Data iterator */
-	struct folio_queue *rq_buffer;	/* Buffer for encryption */
+	struct bvecq	*rq_buffer;	/* Buffer for encryption */
 };
 
 struct mid_q_entry;
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 192649fec25d..885f0cc861ed 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -4683,19 +4683,18 @@ crypt_message(struct TCP_Server_Info *server, int num_rqst,
 }
 
 /*
- * Copy data from an iterator to the folios in a folio queue buffer.
+ * Copy data from an iterator to the pages in a bvec queue buffer.
  */
-static bool cifs_copy_iter_to_folioq(struct iov_iter *iter, size_t size,
-				     struct folio_queue *buffer)
+static bool cifs_copy_iter_to_bvecq(struct iov_iter *iter, size_t size,
+				    struct bvecq *buffer)
 {
 	for (; buffer; buffer = buffer->next) {
-		for (int s = 0; s < folioq_count(buffer); s++) {
-			struct folio *folio = folioq_folio(buffer, s);
-			size_t part = folioq_folio_size(buffer, s);
+		for (int s = 0; s < buffer->nr_slots; s++) {
+			struct bio_vec *bv = &buffer->bv[s];
+			size_t part = umin(bv->bv_len, size);
 
-			part = umin(part, size);
-
-			if (copy_folio_from_iter(folio, 0, part, iter) != part)
+			if (copy_page_from_iter(bv->bv_page, bv->bv_offset,
+						part, iter) != part)
 				return false;
 			size -= part;
 		}
@@ -4707,7 +4706,7 @@ void
 smb3_free_compound_rqst(int num_rqst, struct smb_rqst *rqst)
 {
 	for (int i = 0; i < num_rqst; i++)
-		netfs_free_folioq_buffer(rqst[i].rq_buffer);
+		bvecq_put(rqst[i].rq_buffer);
 }
 
 /*
@@ -4734,7 +4733,7 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,
 	for (int i = 1; i < num_rqst; i++) {
 		struct smb_rqst *old = &old_rq[i - 1];
 		struct smb_rqst *new = &new_rq[i];
-		struct folio_queue *buffer = NULL;
+		struct bvecq *buffer = NULL;
 		size_t size = iov_iter_count(&old->rq_iter);
 
 		orig_len += smb_rqst_len(server, old);
@@ -4742,17 +4741,16 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,
 		new->rq_nvec = old->rq_nvec;
 
 		if (size > 0) {
-			size_t cur_size = 0;
-			rc = netfs_alloc_folioq_buffer(NULL, &buffer, &cur_size,
-						       size, GFP_NOFS);
-			new->rq_buffer = buffer;
-			if (rc < 0)
+			rc = -ENOMEM;
+			buffer = bvecq_alloc_buffer(size, GFP_NOFS, true);
+			if (!buffer)
 				goto err_free;
 
-			iov_iter_folio_queue(&new->rq_iter, ITER_SOURCE,
-					     buffer, 0, 0, size);
+			new->rq_buffer = buffer;
+			iov_iter_bvec_queue(&new->rq_iter, ITER_SOURCE,
+					    buffer, 0, 0, size);
 
-			if (!cifs_copy_iter_to_folioq(&old->rq_iter, size, buffer)) {
+			if (!cifs_copy_iter_to_bvecq(&old->rq_iter, size, buffer)) {
 				rc = smb_EIO1(smb_eio_trace_tx_copy_iter_to_buf, size);
 				goto err_free;
 			}
@@ -4842,22 +4840,20 @@ decrypt_raw_data(struct TCP_Server_Info *server, char *buf,
 }
 
 static int
-cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,
-			 size_t skip, struct iov_iter *iter)
+cifs_copy_bvecq_to_iter(struct bvecq *bq, size_t data_size,
+			size_t skip, struct iov_iter *iter)
 {
-	for (; folioq; folioq = folioq->next) {
-		for (int s = 0; s < folioq_count(folioq); s++) {
-			struct folio *folio;
-			size_t fsize, n, len;
+	for (; bq; bq = bq->next) {
+		for (int s = 0; s < bq->nr_slots; s++) {
+			struct bio_vec *bv = &bq->bv[s];
+			size_t n, len;
 
 			if (data_size == 0)
 				return 0;
 
-			folio = folioq_folio(folioq, s);
-			fsize = folio_size(folio);
-			len = umin(fsize - skip, data_size);
+			len = umin(bv->bv_len - skip, data_size);
 
-			n = copy_folio_to_iter(folio, skip, len, iter);
+			n = copy_page_to_iter(bv->bv_page, bv->bv_offset + skip, len, iter);
 			if (n != len) {
 				cifs_dbg(VFS, "%s: something went wrong\n", __func__);
 				return smb_EIO2(smb_eio_trace_rx_copy_to_iter,
@@ -4879,7 +4875,7 @@ cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,
 
 static int
 handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
-		 char *buf, unsigned int buf_len, struct folio_queue *buffer,
+		 char *buf, unsigned int buf_len, struct bvecq *buffer,
 		 unsigned int buffer_len, bool is_offloaded)
 {
 	unsigned int data_offset;
@@ -4989,8 +4985,8 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 		}
 
 		/* Copy the data to the output I/O iterator. */
-		rdata->result = cifs_copy_folioq_to_iter(buffer, data_len,
-							 cur_off, &rdata->subreq.io_iter);
+		rdata->result = cifs_copy_bvecq_to_iter(buffer, data_len,
+							cur_off, &rdata->subreq.io_iter);
 		if (rdata->result != 0) {
 			if (is_offloaded)
 				mid->mid_state = MID_RESPONSE_MALFORMED;
@@ -5029,7 +5025,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 struct smb2_decrypt_work {
 	struct work_struct decrypt;
 	struct TCP_Server_Info *server;
-	struct folio_queue *buffer;
+	struct bvecq *buffer;
 	char *buf;
 	unsigned int len;
 };
@@ -5043,7 +5039,7 @@ static void smb2_decrypt_offload(struct work_struct *work)
 	struct mid_q_entry *mid;
 	struct iov_iter iter;
 
-	iov_iter_folio_queue(&iter, ITER_DEST, dw->buffer, 0, 0, dw->len);
+	iov_iter_bvec_queue(&iter, ITER_DEST, dw->buffer, 0, 0, dw->len);
 	rc = decrypt_raw_data(dw->server, dw->buf, dw->server->vals->read_rsp_size,
 			      &iter, true);
 	if (rc) {
@@ -5092,7 +5088,7 @@ static void smb2_decrypt_offload(struct work_struct *work)
 	}
 
 free_pages:
-	netfs_free_folioq_buffer(dw->buffer);
+	bvecq_put(dw->buffer);
 	cifs_small_buf_release(dw->buf);
 	kfree(dw);
 }
@@ -5138,12 +5134,12 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
 	dw->len = len;
 	len = round_up(dw->len, PAGE_SIZE);
 
-	size_t cur_size = 0;
-	rc = netfs_alloc_folioq_buffer(NULL, &dw->buffer, &cur_size, len, GFP_NOFS);
-	if (rc < 0)
+	rc = -ENOMEM;
+	dw->buffer = bvecq_alloc_buffer(len, GFP_NOFS, false);
+	if (!dw->buffer)
 		goto discard_data;
 
-	iov_iter_folio_queue(&iter, ITER_DEST, dw->buffer, 0, 0, len);
+	iov_iter_bvec_queue(&iter, ITER_DEST, dw->buffer, 0, 0, len);
 
 	/* Read the data into the buffer and clear excess bufferage. */
 	rc = cifs_read_iter_from_socket(server, &iter, dw->len);
@@ -5201,7 +5197,7 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
 	}
 
 free_pages:
-	netfs_free_folioq_buffer(dw->buffer);
+	bvecq_put(dw->buffer);
 free_dw:
 	kfree(dw);
 	return rc;



From dhowells at redhat.com  Mon Aug 10 07:47:31 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:31 +0100
Subject: [PATCH v9 14/26] cachefiles,netfs: sunset ondemand mode
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-15-dhowells@redhat.com>

From: Gao Xiang <xiang at kernel.org>

It was an effort to enhance fscache as a kernel cache for lazy
pulling (at least according to previous Incremental FS discussion [1])
and EROFS over fscache was the in-tree user of this mode.

fscache has since evolved to be netfslib-oriented, serving network
filesystem inodes via the netfs library, but EROFS never acts as a
network filesystem and we need to cache golden filesystem images rather
than individual EROFS inodes.

Since EROFS over fscache is now removed, clean up netfs/fscache/
cachefiles upstream too.

[1] https://lore.kernel.org/r/CAOQ4uxi4dzxArY24YO=+kBCK2gGoq3Ptb8WkzCqSogPgU_R3dQ at mail.gmail.com

[dh] Fixed up comments on:
https://sashiko.dev/#/patchset/20260716103030.3065561-1-dhowells%40redhat.com
https://sashiko.dev/#/patchset/20260722130218.78958-1-dhowells%40redhat.com

Signed-off-by: Gao Xiang <xiang at kernel.org>
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-erofs at lists.ozlabs.org
cc: bpf at vger.kernel.org
cc: linux-fsdevel at vger.kernel.org
---
 .../filesystems/caching/cachefiles.rst        | 179 ----
 fs/cachefiles/Kconfig                         |  12 -
 fs/cachefiles/Makefile                        |   1 -
 fs/cachefiles/daemon.c                        |  96 +--
 fs/cachefiles/interface.c                     |   8 -
 fs/cachefiles/internal.h                      | 133 ---
 fs/cachefiles/io.c                            |  26 +-
 fs/cachefiles/namei.c                         |   8 -
 fs/cachefiles/ondemand.c                      | 761 ------------------
 include/linux/netfs.h                         |   9 -
 include/trace/events/cachefiles.h             | 184 +----
 include/uapi/linux/cachefiles.h               |  68 --
 kernel/bpf/btf.c                              |   7 -
 13 files changed, 9 insertions(+), 1483 deletions(-)
 delete mode 100644 fs/cachefiles/ondemand.c
 delete mode 100644 include/uapi/linux/cachefiles.h

diff --git a/Documentation/filesystems/caching/cachefiles.rst b/Documentation/filesystems/caching/cachefiles.rst
index b3ccc782cb3b..adfb7d079027 100644
--- a/Documentation/filesystems/caching/cachefiles.rst
+++ b/Documentation/filesystems/caching/cachefiles.rst
@@ -28,8 +28,6 @@ Cache on Already Mounted Filesystem
 
  (*) Debugging.
 
- (*) On-demand Read.
-
 
 Overview
 ========
@@ -483,180 +481,3 @@ the control file.  For example::
 	echo $((1|4|8)) >/sys/module/cachefiles/parameters/debug
 
 will turn on all function entry debugging.
-
-
-On-demand Read
-==============
-
-When working in its original mode, CacheFiles serves as a local cache for a
-remote networking fs - while in on-demand read mode, CacheFiles can boost the
-scenario where on-demand read semantics are needed, e.g. container image
-distribution.
-
-The essential difference between these two modes is seen when a cache miss
-occurs: In the original mode, the netfs will fetch the data from the remote
-server and then write it to the cache file; in on-demand read mode, fetching
-the data and writing it into the cache is delegated to a user daemon.
-
-``CONFIG_CACHEFILES_ONDEMAND`` should be enabled to support on-demand read mode.
-
-
-Protocol Communication
-----------------------
-
-The on-demand read mode uses a simple protocol for communication between kernel
-and user daemon. The protocol can be modeled as::
-
-	kernel --[request]--> user daemon --[reply]--> kernel
-
-CacheFiles will send requests to the user daemon when needed.  The user daemon
-should poll the devnode ('/dev/cachefiles') to check if there's a pending
-request to be processed.  A POLLIN event will be returned when there's a pending
-request.
-
-The user daemon then reads the devnode to fetch a request to process.  It should
-be noted that each read only gets one request. When it has finished processing
-the request, the user daemon should write the reply to the devnode.
-
-Each request starts with a message header of the form::
-
-	struct cachefiles_msg {
-		__u32 msg_id;
-		__u32 opcode;
-		__u32 len;
-		__u32 object_id;
-		__u8  data[];
-	};
-
-where:
-
-	* ``msg_id`` is a unique ID identifying this request among all pending
-	  requests.
-
-	* ``opcode`` indicates the type of this request.
-
-	* ``object_id`` is a unique ID identifying the cache file operated on.
-
-	* ``data`` indicates the payload of this request.
-
-	* ``len`` indicates the whole length of this request, including the
-	  header and following type-specific payload.
-
-
-Turning on On-demand Mode
--------------------------
-
-An optional parameter becomes available to the "bind" command::
-
-	bind [ondemand]
-
-When the "bind" command is given no argument, it defaults to the original mode.
-When it is given the "ondemand" argument, i.e. "bind ondemand", on-demand read
-mode will be enabled.
-
-
-The OPEN Request
-----------------
-
-When the netfs opens a cache file for the first time, a request with the
-CACHEFILES_OP_OPEN opcode, a.k.a an OPEN request will be sent to the user
-daemon.  The payload format is of the form::
-
-	struct cachefiles_open {
-		__u32 volume_key_size;
-		__u32 cookie_key_size;
-		__u32 fd;
-		__u32 flags;
-		__u8  data[];
-	};
-
-where:
-
-	* ``data`` contains the volume_key followed directly by the cookie_key.
-	  The volume key is a NUL-terminated string; the cookie key is binary
-	  data.
-
-	* ``volume_key_size`` indicates the size of the volume key in bytes.
-
-	* ``cookie_key_size`` indicates the size of the cookie key in bytes.
-
-	* ``fd`` indicates an anonymous fd referring to the cache file, through
-	  which the user daemon can perform write/llseek file operations on the
-	  cache file.
-
-
-The user daemon can use the given (volume_key, cookie_key) pair to distinguish
-the requested cache file.  With the given anonymous fd, the user daemon can
-fetch the data and write it to the cache file in the background, even when
-kernel has not triggered a cache miss yet.
-
-Be noted that each cache file has a unique object_id, while it may have multiple
-anonymous fds.  The user daemon may duplicate anonymous fds from the initial
-anonymous fd indicated by the @fd field through dup().  Thus each object_id can
-be mapped to multiple anonymous fds, while the usr daemon itself needs to
-maintain the mapping.
-
-When implementing a user daemon, please be careful of RLIMIT_NOFILE,
-``/proc/sys/fs/nr_open`` and ``/proc/sys/fs/file-max``.  Typically these needn't
-be huge since they're related to the number of open device blobs rather than
-open files of each individual filesystem.
-
-The user daemon should reply the OPEN request by issuing a "copen" (complete
-open) command on the devnode::
-
-	copen <msg_id>,<cache_size>
-
-where:
-
-	* ``msg_id`` must match the msg_id field of the OPEN request.
-
-	* When >= 0, ``cache_size`` indicates the size of the cache file;
-	  when < 0, ``cache_size`` indicates any error code encountered by the
-	  user daemon.
-
-
-The CLOSE Request
------------------
-
-When a cookie withdrawn, a CLOSE request (opcode CACHEFILES_OP_CLOSE) will be
-sent to the user daemon.  This tells the user daemon to close all anonymous fds
-associated with the given object_id.  The CLOSE request has no extra payload,
-and shouldn't be replied.
-
-
-The READ Request
-----------------
-
-When a cache miss is encountered in on-demand read mode, CacheFiles will send a
-READ request (opcode CACHEFILES_OP_READ) to the user daemon. This tells the user
-daemon to fetch the contents of the requested file range.  The payload is of the
-form::
-
-	struct cachefiles_read {
-		__u64 off;
-		__u64 len;
-	};
-
-where:
-
-	* ``off`` indicates the starting offset of the requested file range.
-
-	* ``len`` indicates the length of the requested file range.
-
-
-When it receives a READ request, the user daemon should fetch the requested data
-and write it to the cache file identified by object_id.
-
-When it has finished processing the READ request, the user daemon should reply
-by using the CACHEFILES_IOC_READ_COMPLETE ioctl on one of the anonymous fds
-associated with the object_id given in the READ request.  The ioctl is of the
-form::
-
-	ioctl(fd, CACHEFILES_IOC_READ_COMPLETE, msg_id);
-
-where:
-
-	* ``fd`` is one of the anonymous fds associated with the object_id
-	  given.
-
-	* ``msg_id`` must match the msg_id field of the READ request.
diff --git a/fs/cachefiles/Kconfig b/fs/cachefiles/Kconfig
index c5a070550ee3..afb25b6af5aa 100644
--- a/fs/cachefiles/Kconfig
+++ b/fs/cachefiles/Kconfig
@@ -26,15 +26,3 @@ config CACHEFILES_ERROR_INJECTION
 	help
 	  This permits error injection to be enabled in cachefiles whilst a
 	  cache is in service.
-
-config CACHEFILES_ONDEMAND
-	bool "Support for on-demand read"
-	depends on CACHEFILES
-	default n
-	help
-	  This permits userspace to enable the cachefiles on-demand read mode.
-	  In this mode, when a cache miss occurs, responsibility for fetching
-	  the data lies with the cachefiles backend instead of with the netfs
-	  and is delegated to userspace.
-
-	  If unsure, say N.
diff --git a/fs/cachefiles/Makefile b/fs/cachefiles/Makefile
index c37a7a9af10b..16d811f1a2fa 100644
--- a/fs/cachefiles/Makefile
+++ b/fs/cachefiles/Makefile
@@ -16,6 +16,5 @@ cachefiles-y := \
 	xattr.o
 
 cachefiles-$(CONFIG_CACHEFILES_ERROR_INJECTION) += error_inject.o
-cachefiles-$(CONFIG_CACHEFILES_ONDEMAND) += ondemand.o
 
 obj-$(CONFIG_CACHEFILES) := cachefiles.o
diff --git a/fs/cachefiles/daemon.c b/fs/cachefiles/daemon.c
index 4117b145ac94..1a66e0af2837 100644
--- a/fs/cachefiles/daemon.c
+++ b/fs/cachefiles/daemon.c
@@ -76,10 +76,6 @@ static const struct cachefiles_daemon_cmd cachefiles_daemon_cmds[] = {
 	{ "inuse",	cachefiles_daemon_inuse		},
 	{ "secctx",	cachefiles_daemon_secctx	},
 	{ "tag",	cachefiles_daemon_tag		},
-#ifdef CONFIG_CACHEFILES_ONDEMAND
-	{ "copen",	cachefiles_ondemand_copen	},
-	{ "restore",	cachefiles_ondemand_restore	},
-#endif
 	{ "",		NULL				}
 };
 
@@ -113,9 +109,6 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
 	INIT_LIST_HEAD(&cache->volumes);
 	INIT_LIST_HEAD(&cache->object_list);
 	spin_lock_init(&cache->object_list_lock);
-	refcount_set(&cache->unbind_pincount, 1);
-	xa_init_flags(&cache->reqs, XA_FLAGS_ALLOC);
-	xa_init_flags(&cache->ondemand_ids, XA_FLAGS_ALLOC1);
 
 	/* set default caching limits
 	 * - limit at 1% free space and/or free files
@@ -134,54 +127,6 @@ static int cachefiles_daemon_open(struct inode *inode, struct file *file)
 	return 0;
 }
 
-void cachefiles_flush_reqs(struct cachefiles_cache *cache)
-{
-	struct xarray *xa = &cache->reqs;
-	struct cachefiles_req *req;
-	unsigned long index;
-
-	/*
-	 * Make sure the following two operations won't be reordered.
-	 *   1) set CACHEFILES_DEAD bit
-	 *   2) flush requests in the xarray
-	 * Otherwise the request may be enqueued after xarray has been
-	 * flushed, leaving the orphan request never being completed.
-	 *
-	 * CPU 1			CPU 2
-	 * =====			=====
-	 * flush requests in the xarray
-	 *				test CACHEFILES_DEAD bit
-	 *				enqueue the request
-	 * set CACHEFILES_DEAD bit
-	 */
-	smp_mb();
-
-	xa_lock(xa);
-	xa_for_each(xa, index, req) {
-		req->error = -EIO;
-		complete(&req->done);
-		__xa_erase(xa, index);
-	}
-	xa_unlock(xa);
-
-	xa_destroy(&cache->reqs);
-	xa_destroy(&cache->ondemand_ids);
-}
-
-void cachefiles_put_unbind_pincount(struct cachefiles_cache *cache)
-{
-	if (refcount_dec_and_test(&cache->unbind_pincount)) {
-		cachefiles_daemon_unbind(cache);
-		cachefiles_open = 0;
-		kfree(cache);
-	}
-}
-
-void cachefiles_get_unbind_pincount(struct cachefiles_cache *cache)
-{
-	refcount_inc(&cache->unbind_pincount);
-}
-
 /*
  * Release a cache.
  */
@@ -194,16 +139,14 @@ static int cachefiles_daemon_release(struct inode *inode, struct file *file)
 	ASSERT(cache);
 
 	set_bit(CACHEFILES_DEAD, &cache->flags);
-
-	if (cachefiles_in_ondemand_mode(cache))
-		cachefiles_flush_reqs(cache);
+	cachefiles_daemon_unbind(cache);
 
 	/* clean up the control file interface */
 	cache->cachefilesd = NULL;
 	file->private_data = NULL;
+	cachefiles_open = 0;
 
-	cachefiles_put_unbind_pincount(cache);
-
+	kfree(cache);
 	_leave("");
 	return 0;
 }
@@ -266,10 +209,7 @@ static ssize_t cachefiles_daemon_read(struct file *file, char __user *_buffer,
 	if (!test_bit(CACHEFILES_READY, &cache->flags))
 		return 0;
 
-	if (cachefiles_in_ondemand_mode(cache))
-		return cachefiles_ondemand_daemon_read(cache, _buffer, buflen);
-	else
-		return cachefiles_do_daemon_read(cache, _buffer, buflen);
+	return cachefiles_do_daemon_read(cache, _buffer, buflen);
 }
 
 /*
@@ -358,28 +298,13 @@ static __poll_t cachefiles_daemon_poll(struct file *file,
 					   struct poll_table_struct *poll)
 {
 	struct cachefiles_cache *cache = file->private_data;
-	XA_STATE(xas, &cache->reqs, 0);
-	struct cachefiles_req *req;
 	__poll_t mask;
 
 	poll_wait(file, &cache->daemon_pollwq, poll);
 	mask = 0;
 
-	if (cachefiles_in_ondemand_mode(cache)) {
-		if (!xa_empty(&cache->reqs)) {
-			xas_lock(&xas);
-			xas_for_each_marked(&xas, req, ULONG_MAX, CACHEFILES_REQ_NEW) {
-				if (!cachefiles_ondemand_is_reopening_read(req)) {
-					mask |= EPOLLIN;
-					break;
-				}
-			}
-			xas_unlock(&xas);
-		}
-	} else {
-		if (test_bit(CACHEFILES_STATE_CHANGED, &cache->flags))
-			mask |= EPOLLIN;
-	}
+	if (test_bit(CACHEFILES_STATE_CHANGED, &cache->flags))
+		mask |= EPOLLIN;
 
 	if (test_bit(CACHEFILES_CULLING, &cache->flags))
 		mask |= EPOLLOUT;
@@ -779,14 +704,7 @@ static int cachefiles_daemon_bind(struct cachefiles_cache *cache, char *args)
 		return -EBUSY;
 	}
 
-	if (IS_ENABLED(CONFIG_CACHEFILES_ONDEMAND)) {
-		if (!strcmp(args, "ondemand")) {
-			set_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags);
-		} else if (*args) {
-			pr_err("Invalid argument to the 'bind' command\n");
-			return -EINVAL;
-		}
-	} else if (*args) {
+	if (*args) {
 		pr_err("'bind' command doesn't take an argument\n");
 		return -EINVAL;
 	}
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index a08250d244ea..50a000310a8c 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -32,11 +32,6 @@ struct cachefiles_object *cachefiles_alloc_object(struct fscache_cookie *cookie)
 	if (!object)
 		return NULL;
 
-	if (cachefiles_ondemand_init_obj_info(object, volume)) {
-		kmem_cache_free(cachefiles_object_jar, object);
-		return NULL;
-	}
-
 	refcount_set(&object->ref, 1);
 
 	spin_lock_init(&object->lock);
@@ -94,7 +89,6 @@ void cachefiles_put_object(struct cachefiles_object *object,
 		ASSERTCMP(object->file, ==, NULL);
 
 		kfree(object->d_name);
-		cachefiles_ondemand_deinit_obj_info(object);
 		cache = object->volume->cache->cache;
 		fscache_put_cookie(object->cookie, fscache_cookie_put_object);
 		object->cookie = NULL;
@@ -374,8 +368,6 @@ static void cachefiles_withdraw_cookie(struct fscache_cookie *cookie)
 		spin_unlock(&cache->object_list_lock);
 	}
 
-	cachefiles_ondemand_clean_object(object);
-
 	if (object->file) {
 		cachefiles_begin_secure(cache, &saved_cred);
 		cachefiles_clean_up_object(object, cache);
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index b62cd3e9a18e..409492744bde 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -15,8 +15,6 @@
 #include <linux/fscache-cache.h>
 #include <linux/cred.h>
 #include <linux/security.h>
-#include <linux/xarray.h>
-#include <linux/cachefiles.h>
 
 #define CACHEFILES_DIO_BLOCK_SIZE 4096
 
@@ -44,21 +42,6 @@ struct cachefiles_volume {
 	struct dentry			*fanout[256];	/* Fanout subdirs */
 };
 
-enum cachefiles_object_state {
-	CACHEFILES_ONDEMAND_OBJSTATE_CLOSE, /* Anonymous fd closed by daemon or initial state */
-	CACHEFILES_ONDEMAND_OBJSTATE_OPEN, /* Anonymous fd associated with object is available */
-	CACHEFILES_ONDEMAND_OBJSTATE_REOPENING, /* Object that was closed and is being reopened. */
-	CACHEFILES_ONDEMAND_OBJSTATE_DROPPING, /* Object is being dropped. */
-};
-
-struct cachefiles_ondemand_info {
-	struct work_struct		ondemand_work;
-	int				ondemand_id;
-	enum cachefiles_object_state	state;
-	struct cachefiles_object	*object;
-	spinlock_t			lock;
-};
-
 /*
  * Backing file state.
  */
@@ -74,13 +57,8 @@ struct cachefiles_object {
 	enum cachefiles_content		content_info:8;	/* Info about content presence */
 	unsigned long			flags;
 #define CACHEFILES_OBJECT_USING_TMPFILE	0		/* Have an unlinked tmpfile */
-#ifdef CONFIG_CACHEFILES_ONDEMAND
-	struct cachefiles_ondemand_info	*ondemand;
-#endif
 };
 
-#define CACHEFILES_ONDEMAND_ID_CLOSED	-1
-
 /*
  * Cache files cache definition
  */
@@ -119,35 +97,12 @@ struct cachefiles_cache {
 #define CACHEFILES_DEAD			1	/* T if cache dead */
 #define CACHEFILES_CULLING		2	/* T if cull engaged */
 #define CACHEFILES_STATE_CHANGED	3	/* T if state changed (poll trigger) */
-#define CACHEFILES_ONDEMAND_MODE	4	/* T if in on-demand read mode */
 	char				*rootdirname;	/* name of cache root directory */
 	char				*tag;		/* cache binding tag */
-	refcount_t			unbind_pincount;/* refcount to do daemon unbind */
-	struct xarray			reqs;		/* xarray of pending on-demand requests */
-	unsigned long			req_id_next;
-	struct xarray			ondemand_ids;	/* xarray for ondemand_id allocation */
-	u32				ondemand_id_next;
-	u32				msg_id_next;
 	u32				secid;		/* LSM security id */
 	bool				have_secid;	/* whether "secid" was set */
 };
 
-static inline bool cachefiles_in_ondemand_mode(struct cachefiles_cache *cache)
-{
-	return IS_ENABLED(CONFIG_CACHEFILES_ONDEMAND) &&
-		test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags);
-}
-
-struct cachefiles_req {
-	struct cachefiles_object *object;
-	struct completion done;
-	refcount_t ref;
-	int error;
-	struct cachefiles_msg msg;
-};
-
-#define CACHEFILES_REQ_NEW	XA_MARK_1
-
 #include <trace/events/cachefiles.h>
 
 static inline
@@ -191,8 +146,6 @@ extern int cachefiles_has_space(struct cachefiles_cache *cache,
  */
 extern const struct file_operations cachefiles_daemon_fops;
 extern void cachefiles_flush_reqs(struct cachefiles_cache *cache);
-extern void cachefiles_get_unbind_pincount(struct cachefiles_cache *cache);
-extern void cachefiles_put_unbind_pincount(struct cachefiles_cache *cache);
 
 /*
  * error_inject.c
@@ -298,90 +251,6 @@ extern struct file *cachefiles_create_tmpfile(struct cachefiles_object *object);
 extern bool cachefiles_commit_tmpfile(struct cachefiles_cache *cache,
 				      struct cachefiles_object *object);
 
-/*
- * ondemand.c
- */
-#ifdef CONFIG_CACHEFILES_ONDEMAND
-extern ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
-					char __user *_buffer, size_t buflen);
-
-extern int cachefiles_ondemand_copen(struct cachefiles_cache *cache,
-				     char *args);
-
-extern int cachefiles_ondemand_restore(struct cachefiles_cache *cache,
-					char *args);
-
-extern int cachefiles_ondemand_init_object(struct cachefiles_object *object);
-extern void cachefiles_ondemand_clean_object(struct cachefiles_object *object);
-
-extern int cachefiles_ondemand_read(struct cachefiles_object *object,
-				    loff_t pos, size_t len);
-
-extern int cachefiles_ondemand_init_obj_info(struct cachefiles_object *obj,
-					struct cachefiles_volume *volume);
-extern void cachefiles_ondemand_deinit_obj_info(struct cachefiles_object *obj);
-
-#define CACHEFILES_OBJECT_STATE_FUNCS(_state, _STATE)	\
-static inline bool								\
-cachefiles_ondemand_object_is_##_state(const struct cachefiles_object *object) \
-{												\
-	return object->ondemand->state == CACHEFILES_ONDEMAND_OBJSTATE_##_STATE; \
-}												\
-												\
-static inline void								\
-cachefiles_ondemand_set_object_##_state(struct cachefiles_object *object) \
-{												\
-	object->ondemand->state = CACHEFILES_ONDEMAND_OBJSTATE_##_STATE; \
-}
-
-CACHEFILES_OBJECT_STATE_FUNCS(open, OPEN);
-CACHEFILES_OBJECT_STATE_FUNCS(close, CLOSE);
-CACHEFILES_OBJECT_STATE_FUNCS(reopening, REOPENING);
-CACHEFILES_OBJECT_STATE_FUNCS(dropping, DROPPING);
-
-static inline bool cachefiles_ondemand_is_reopening_read(struct cachefiles_req *req)
-{
-	return cachefiles_ondemand_object_is_reopening(req->object) &&
-			req->msg.opcode == CACHEFILES_OP_READ;
-}
-
-#else
-static inline ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
-					char __user *_buffer, size_t buflen)
-{
-	return -EOPNOTSUPP;
-}
-
-static inline int cachefiles_ondemand_init_object(struct cachefiles_object *object)
-{
-	return 0;
-}
-
-static inline void cachefiles_ondemand_clean_object(struct cachefiles_object *object)
-{
-}
-
-static inline int cachefiles_ondemand_read(struct cachefiles_object *object,
-					   loff_t pos, size_t len)
-{
-	return -EOPNOTSUPP;
-}
-
-static inline int cachefiles_ondemand_init_obj_info(struct cachefiles_object *obj,
-						struct cachefiles_volume *volume)
-{
-	return 0;
-}
-static inline void cachefiles_ondemand_deinit_obj_info(struct cachefiles_object *obj)
-{
-}
-
-static inline bool cachefiles_ondemand_is_reopening_read(struct cachefiles_req *req)
-{
-	return false;
-}
-#endif
-
 /*
  * security.c
  */
@@ -430,8 +299,6 @@ do {							\
 	pr_err("I/O Error: " FMT"\n", ##__VA_ARGS__);	\
 	fscache_io_error((___cache)->cache);		\
 	set_bit(CACHEFILES_DEAD, &(___cache)->flags);	\
-	if (cachefiles_in_ondemand_mode(___cache))	\
-		cachefiles_flush_reqs(___cache);	\
 } while (0)
 
 #define cachefiles_io_error_obj(object, FMT, ...)			\
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index d879b80a0bed..9540ec25b3cb 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -390,7 +390,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 	size_t len = *_len;
 	loff_t off, to;
 	ino_t ino = file ? file_inode(file)->i_ino : 0;
-	int rc;
 
 	_enter("%zx @%llx/%llx", len, start, i_size);
 
@@ -403,8 +402,7 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 	if (test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags)) {
 		__set_bit(NETFS_SREQ_COPY_TO_CACHE, _flags);
 		why = cachefiles_trace_read_no_data;
-		if (!test_bit(NETFS_SREQ_ONDEMAND, _flags))
-			goto out_no_object;
+		goto out_no_object;
 	}
 
 	/* The object and the file may be being created in the background. */
@@ -421,7 +419,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 	object = cachefiles_cres_object(cres);
 	cache = object->volume->cache;
 	cachefiles_begin_secure(cache, &saved_cred);
-retry:
 	off = cachefiles_inject_read_error();
 	if (off == 0)
 		off = vfs_llseek(file, start, SEEK_DATA);
@@ -474,14 +471,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 
 download_and_store:
 	__set_bit(NETFS_SREQ_COPY_TO_CACHE, _flags);
-	if (test_bit(NETFS_SREQ_ONDEMAND, _flags)) {
-		rc = cachefiles_ondemand_read(object, start, len);
-		if (!rc) {
-			__clear_bit(NETFS_SREQ_ONDEMAND, _flags);
-			goto retry;
-		}
-		ret = NETFS_INVALID_READ;
-	}
 out:
 	cachefiles_end_secure(cache, saved_cred);
 out_no_object:
@@ -501,18 +490,6 @@ static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *
 					  &subreq->flags, subreq->rreq->inode->i_ino);
 }
 
-/*
- * Prepare an on-demand read operation, shortening it to a cached/uncached
- * boundary as appropriate.
- */
-static enum netfs_io_source
-cachefiles_prepare_ondemand_read(struct netfs_cache_resources *cres,
-				 loff_t start, size_t *_len, loff_t i_size,
-				 unsigned long *_flags, ino_t ino)
-{
-	return cachefiles_do_prepare_read(cres, start, _len, i_size, _flags, ino);
-}
-
 /*
  * Prepare for a write to occur.
  */
@@ -731,7 +708,6 @@ static const struct netfs_cache_ops cachefiles_netfs_cache_ops = {
 	.prepare_read		= cachefiles_prepare_read,
 	.prepare_write		= cachefiles_prepare_write,
 	.prepare_write_subreq	= cachefiles_prepare_write_subreq,
-	.prepare_ondemand_read	= cachefiles_prepare_ondemand_read,
 	.query_occupancy	= cachefiles_query_occupancy,
 };
 
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 8a9f6be15828..88955249a1a6 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -442,10 +442,6 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 	if (!cachefiles_mark_inode_in_use(object, file_inode(file)))
 		WARN_ON(1);
 
-	ret = cachefiles_ondemand_init_object(object);
-	if (ret < 0)
-		goto err_unuse;
-
 	ni_size = object->cookie->object_size;
 	ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
 
@@ -546,10 +542,6 @@ static bool cachefiles_open_file(struct cachefiles_object *object,
 	}
 	_debug("file -> %pd positive", dentry);
 
-	ret = cachefiles_ondemand_init_object(object);
-	if (ret < 0)
-		goto error_fput;
-
 	ret = cachefiles_check_auxdata(object, file);
 	if (ret < 0)
 		goto check_failed;
diff --git a/fs/cachefiles/ondemand.c b/fs/cachefiles/ondemand.c
deleted file mode 100644
index 0849eaf583cd..000000000000
--- a/fs/cachefiles/ondemand.c
+++ /dev/null
@@ -1,761 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-#include <linux/anon_inodes.h>
-#include <linux/uio.h>
-#include "internal.h"
-
-struct ondemand_anon_file {
-	struct file *file;
-	int fd;
-};
-
-static inline void cachefiles_req_put(struct cachefiles_req *req)
-{
-	if (refcount_dec_and_test(&req->ref))
-		kfree(req);
-}
-
-static int cachefiles_ondemand_fd_release(struct inode *inode,
-					  struct file *file)
-{
-	struct cachefiles_object *object = file->private_data;
-	struct cachefiles_cache *cache;
-	struct cachefiles_ondemand_info *info;
-	int object_id;
-	struct cachefiles_req *req;
-	XA_STATE(xas, NULL, 0);
-
-	if (!object)
-		return 0;
-
-	info = object->ondemand;
-	cache = object->volume->cache;
-	xas.xa = &cache->reqs;
-
-	xa_lock(&cache->reqs);
-	spin_lock(&info->lock);
-	object_id = info->ondemand_id;
-	info->ondemand_id = CACHEFILES_ONDEMAND_ID_CLOSED;
-	cachefiles_ondemand_set_object_close(object);
-	spin_unlock(&info->lock);
-
-	/* Only flush CACHEFILES_REQ_NEW marked req to avoid race with daemon_read */
-	xas_for_each_marked(&xas, req, ULONG_MAX, CACHEFILES_REQ_NEW) {
-		if (req->msg.object_id == object_id &&
-		    req->msg.opcode == CACHEFILES_OP_CLOSE) {
-			complete(&req->done);
-			xas_store(&xas, NULL);
-		}
-	}
-	xa_unlock(&cache->reqs);
-
-	xa_erase(&cache->ondemand_ids, object_id);
-	trace_cachefiles_ondemand_fd_release(object, object_id);
-	cachefiles_put_object(object, cachefiles_obj_put_ondemand_fd);
-	cachefiles_put_unbind_pincount(cache);
-	return 0;
-}
-
-static ssize_t cachefiles_ondemand_fd_write_iter(struct kiocb *kiocb,
-						 struct iov_iter *iter)
-{
-	struct cachefiles_object *object = kiocb->ki_filp->private_data;
-	struct cachefiles_cache *cache = object->volume->cache;
-	struct file *file;
-	size_t len = iter->count, aligned_len = len;
-	loff_t pos = kiocb->ki_pos;
-	const struct cred *saved_cred;
-	int ret;
-
-	spin_lock(&object->lock);
-	file = object->file;
-	if (!file) {
-		spin_unlock(&object->lock);
-		return -ENOBUFS;
-	}
-	get_file(file);
-	spin_unlock(&object->lock);
-
-	cachefiles_begin_secure(cache, &saved_cred);
-	ret = __cachefiles_prepare_write(object, file, &pos, &aligned_len, len, true);
-	cachefiles_end_secure(cache, saved_cred);
-	if (ret < 0)
-		goto out;
-
-	trace_cachefiles_ondemand_fd_write(object, file_inode(file), pos, len);
-	ret = __cachefiles_write(object, file, pos, iter, NULL, NULL);
-	if (ret > 0)
-		kiocb->ki_pos += ret;
-
-out:
-	fput(file);
-	return ret;
-}
-
-static loff_t cachefiles_ondemand_fd_llseek(struct file *filp, loff_t pos,
-					    int whence)
-{
-	struct cachefiles_object *object = filp->private_data;
-	struct file *file;
-	loff_t ret;
-
-	spin_lock(&object->lock);
-	file = object->file;
-	if (!file) {
-		spin_unlock(&object->lock);
-		return -ENOBUFS;
-	}
-	get_file(file);
-	spin_unlock(&object->lock);
-
-	ret = vfs_llseek(file, pos, whence);
-	fput(file);
-
-	return ret;
-}
-
-static long cachefiles_ondemand_fd_ioctl(struct file *filp, unsigned int ioctl,
-					 unsigned long id)
-{
-	struct cachefiles_object *object = filp->private_data;
-	struct cachefiles_cache *cache = object->volume->cache;
-	struct cachefiles_req *req;
-	XA_STATE(xas, &cache->reqs, id);
-
-	if (ioctl != CACHEFILES_IOC_READ_COMPLETE)
-		return -EINVAL;
-
-	if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
-		return -EOPNOTSUPP;
-
-	xa_lock(&cache->reqs);
-	req = xas_load(&xas);
-	if (!req || req->msg.opcode != CACHEFILES_OP_READ ||
-	    req->object != object) {
-		xa_unlock(&cache->reqs);
-		return -EINVAL;
-	}
-	xas_store(&xas, NULL);
-	xa_unlock(&cache->reqs);
-
-	trace_cachefiles_ondemand_cread(object, id);
-	complete(&req->done);
-	return 0;
-}
-
-static const struct file_operations cachefiles_ondemand_fd_fops = {
-	.owner		= THIS_MODULE,
-	.release	= cachefiles_ondemand_fd_release,
-	.write_iter	= cachefiles_ondemand_fd_write_iter,
-	.llseek		= cachefiles_ondemand_fd_llseek,
-	.unlocked_ioctl	= cachefiles_ondemand_fd_ioctl,
-};
-
-/*
- * OPEN request Completion (copen)
- * - command: "copen <id>,<cache_size>"
- *   <cache_size> indicates the object size if >=0, error code if negative
- */
-int cachefiles_ondemand_copen(struct cachefiles_cache *cache, char *args)
-{
-	struct cachefiles_req *req;
-	struct fscache_cookie *cookie;
-	struct cachefiles_ondemand_info *info;
-	char *pid, *psize;
-	unsigned long id;
-	long size;
-	int ret;
-	XA_STATE(xas, &cache->reqs, 0);
-
-	if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
-		return -EOPNOTSUPP;
-
-	if (!*args) {
-		pr_err("Empty id specified\n");
-		return -EINVAL;
-	}
-
-	pid = args;
-	psize = strchr(args, ',');
-	if (!psize) {
-		pr_err("Cache size is not specified\n");
-		return -EINVAL;
-	}
-
-	*psize = 0;
-	psize++;
-
-	ret = kstrtoul(pid, 0, &id);
-	if (ret)
-		return ret;
-
-	xa_lock(&cache->reqs);
-	xas.xa_index = id;
-	req = xas_load(&xas);
-	if (!req || req->msg.opcode != CACHEFILES_OP_OPEN ||
-	    !req->object->ondemand->ondemand_id) {
-		xa_unlock(&cache->reqs);
-		return -EINVAL;
-	}
-	xas_store(&xas, NULL);
-	xa_unlock(&cache->reqs);
-
-	info = req->object->ondemand;
-	/* fail OPEN request if copen format is invalid */
-	ret = kstrtol(psize, 0, &size);
-	if (ret) {
-		req->error = ret;
-		goto out;
-	}
-
-	/* fail OPEN request if daemon reports an error */
-	if (size < 0) {
-		if (!IS_ERR_VALUE(size)) {
-			req->error = -EINVAL;
-			ret = -EINVAL;
-		} else {
-			req->error = size;
-			ret = 0;
-		}
-		goto out;
-	}
-
-	spin_lock(&info->lock);
-	/*
-	 * The anonymous fd was closed before copen ? Fail the request.
-	 *
-	 *             t1             |             t2
-	 * ---------------------------------------------------------
-	 *                             cachefiles_ondemand_copen
-	 *                             req = xa_erase(&cache->reqs, id)
-	 * // Anon fd is maliciously closed.
-	 * cachefiles_ondemand_fd_release
-	 * xa_lock(&cache->reqs)
-	 * cachefiles_ondemand_set_object_close(object)
-	 * xa_unlock(&cache->reqs)
-	 *                             cachefiles_ondemand_set_object_open
-	 *                             // No one will ever close it again.
-	 * cachefiles_ondemand_daemon_read
-	 * cachefiles_ondemand_select_req
-	 *
-	 * Get a read req but its fd is already closed. The daemon can't
-	 * issue a cread ioctl with an closed fd, then hung.
-	 */
-	if (info->ondemand_id == CACHEFILES_ONDEMAND_ID_CLOSED) {
-		spin_unlock(&info->lock);
-		req->error = -EBADFD;
-		goto out;
-	}
-	cookie = req->object->cookie;
-	cookie->object_size = size;
-	if (size)
-		clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags);
-	else
-		set_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags);
-	trace_cachefiles_ondemand_copen(req->object, id, size);
-
-	cachefiles_ondemand_set_object_open(req->object);
-	spin_unlock(&info->lock);
-	wake_up_all(&cache->daemon_pollwq);
-
-out:
-	spin_lock(&info->lock);
-	/* Need to set object close to avoid reopen status continuing */
-	if (info->ondemand_id == CACHEFILES_ONDEMAND_ID_CLOSED)
-		cachefiles_ondemand_set_object_close(req->object);
-	spin_unlock(&info->lock);
-	complete(&req->done);
-	return ret;
-}
-
-int cachefiles_ondemand_restore(struct cachefiles_cache *cache, char *args)
-{
-	struct cachefiles_req *req;
-
-	XA_STATE(xas, &cache->reqs, 0);
-
-	if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
-		return -EOPNOTSUPP;
-
-	/*
-	 * Reset the requests to CACHEFILES_REQ_NEW state, so that the
-	 * requests have been processed halfway before the crash of the
-	 * user daemon could be reprocessed after the recovery.
-	 */
-	xas_lock(&xas);
-	xas_for_each(&xas, req, ULONG_MAX)
-		xas_set_mark(&xas, CACHEFILES_REQ_NEW);
-	xas_unlock(&xas);
-
-	wake_up_all(&cache->daemon_pollwq);
-	return 0;
-}
-
-static int cachefiles_ondemand_get_fd(struct cachefiles_req *req,
-				      struct ondemand_anon_file *anon_file)
-{
-	struct cachefiles_object *object;
-	struct cachefiles_cache *cache;
-	struct cachefiles_open *load;
-	u32 object_id;
-	int ret;
-
-	object = cachefiles_grab_object(req->object,
-			cachefiles_obj_get_ondemand_fd);
-	cache = object->volume->cache;
-
-	ret = xa_alloc_cyclic(&cache->ondemand_ids, &object_id, NULL,
-			      XA_LIMIT(1, INT_MAX),
-			      &cache->ondemand_id_next, GFP_KERNEL);
-	if (ret < 0)
-		goto err;
-
-	anon_file->fd = get_unused_fd_flags(O_WRONLY);
-	if (anon_file->fd < 0) {
-		ret = anon_file->fd;
-		goto err_free_id;
-	}
-
-	anon_file->file = anon_inode_getfile_fmode("[cachefiles]",
-				&cachefiles_ondemand_fd_fops, object,
-				O_WRONLY, FMODE_PWRITE | FMODE_LSEEK);
-	if (IS_ERR(anon_file->file)) {
-		ret = PTR_ERR(anon_file->file);
-		goto err_put_fd;
-	}
-
-	spin_lock(&object->ondemand->lock);
-	if (object->ondemand->ondemand_id > 0) {
-		spin_unlock(&object->ondemand->lock);
-		/* Pair with check in cachefiles_ondemand_fd_release(). */
-		anon_file->file->private_data = NULL;
-		ret = -EEXIST;
-		goto err_put_file;
-	}
-
-	load = (void *)req->msg.data;
-	load->fd = anon_file->fd;
-	object->ondemand->ondemand_id = object_id;
-	spin_unlock(&object->ondemand->lock);
-
-	cachefiles_get_unbind_pincount(cache);
-	trace_cachefiles_ondemand_open(object, &req->msg, load);
-	return 0;
-
-err_put_file:
-	fput(anon_file->file);
-	anon_file->file = NULL;
-err_put_fd:
-	put_unused_fd(anon_file->fd);
-	anon_file->fd = ret;
-err_free_id:
-	xa_erase(&cache->ondemand_ids, object_id);
-err:
-	spin_lock(&object->ondemand->lock);
-	/* Avoid marking an opened object as closed. */
-	if (object->ondemand->ondemand_id <= 0)
-		cachefiles_ondemand_set_object_close(object);
-	spin_unlock(&object->ondemand->lock);
-	cachefiles_put_object(object, cachefiles_obj_put_ondemand_fd);
-	return ret;
-}
-
-static void ondemand_object_worker(struct work_struct *work)
-{
-	struct cachefiles_ondemand_info *info =
-		container_of(work, struct cachefiles_ondemand_info, ondemand_work);
-
-	cachefiles_ondemand_init_object(info->object);
-}
-
-/*
- * If there are any inflight or subsequent READ requests on the
- * closed object, reopen it.
- * Skip read requests whose related object is reopening.
- */
-static struct cachefiles_req *cachefiles_ondemand_select_req(struct xa_state *xas,
-							      unsigned long xa_max)
-{
-	struct cachefiles_req *req;
-	struct cachefiles_object *object;
-	struct cachefiles_ondemand_info *info;
-
-	xas_for_each_marked(xas, req, xa_max, CACHEFILES_REQ_NEW) {
-		if (req->msg.opcode != CACHEFILES_OP_READ)
-			return req;
-		object = req->object;
-		info = object->ondemand;
-		if (cachefiles_ondemand_object_is_close(object)) {
-			cachefiles_ondemand_set_object_reopening(object);
-			queue_work(fscache_wq, &info->ondemand_work);
-			continue;
-		}
-		if (cachefiles_ondemand_object_is_reopening(object))
-			continue;
-		return req;
-	}
-	return NULL;
-}
-
-static inline bool cachefiles_ondemand_finish_req(struct cachefiles_req *req,
-						  struct xa_state *xas, int err)
-{
-	if (unlikely(!xas || !req))
-		return false;
-
-	if (xa_cmpxchg(xas->xa, xas->xa_index, req, NULL, 0) != req)
-		return false;
-
-	req->error = err;
-	complete(&req->done);
-	return true;
-}
-
-ssize_t cachefiles_ondemand_daemon_read(struct cachefiles_cache *cache,
-					char __user *_buffer, size_t buflen)
-{
-	struct cachefiles_req *req;
-	struct cachefiles_msg *msg;
-	size_t n;
-	int ret = 0;
-	struct ondemand_anon_file anon_file;
-	XA_STATE(xas, &cache->reqs, cache->req_id_next);
-
-	xa_lock(&cache->reqs);
-	/*
-	 * Cyclically search for a request that has not ever been processed,
-	 * to prevent requests from being processed repeatedly, and make
-	 * request distribution fair.
-	 */
-	req = cachefiles_ondemand_select_req(&xas, ULONG_MAX);
-	if (!req && cache->req_id_next > 0) {
-		xas_set(&xas, 0);
-		req = cachefiles_ondemand_select_req(&xas, cache->req_id_next - 1);
-	}
-	if (!req) {
-		xa_unlock(&cache->reqs);
-		return 0;
-	}
-
-	msg = &req->msg;
-	n = msg->len;
-
-	if (n > buflen) {
-		xa_unlock(&cache->reqs);
-		return -EMSGSIZE;
-	}
-
-	xas_clear_mark(&xas, CACHEFILES_REQ_NEW);
-	cache->req_id_next = xas.xa_index + 1;
-	refcount_inc(&req->ref);
-	cachefiles_grab_object(req->object, cachefiles_obj_get_read_req);
-	xa_unlock(&cache->reqs);
-
-	if (msg->opcode == CACHEFILES_OP_OPEN) {
-		ret = cachefiles_ondemand_get_fd(req, &anon_file);
-		if (ret)
-			goto out;
-	}
-
-	msg->msg_id = xas.xa_index;
-	msg->object_id = req->object->ondemand->ondemand_id;
-
-	if (copy_to_user(_buffer, msg, n) != 0)
-		ret = -EFAULT;
-
-	if (msg->opcode == CACHEFILES_OP_OPEN) {
-		if (ret < 0) {
-			fput(anon_file.file);
-			put_unused_fd(anon_file.fd);
-			goto out;
-		}
-		fd_install(anon_file.fd, anon_file.file);
-	}
-out:
-	cachefiles_put_object(req->object, cachefiles_obj_put_read_req);
-	/* Remove error request and CLOSE request has no reply */
-	if (ret || msg->opcode == CACHEFILES_OP_CLOSE)
-		cachefiles_ondemand_finish_req(req, &xas, ret);
-	cachefiles_req_put(req);
-	return ret ? ret : n;
-}
-
-typedef int (*init_req_fn)(struct cachefiles_req *req, void *private);
-
-static int cachefiles_ondemand_send_req(struct cachefiles_object *object,
-					enum cachefiles_opcode opcode,
-					size_t data_len,
-					init_req_fn init_req,
-					void *private)
-{
-	struct cachefiles_cache *cache = object->volume->cache;
-	struct cachefiles_req *req = NULL;
-	XA_STATE(xas, &cache->reqs, 0);
-	int ret;
-
-	if (!test_bit(CACHEFILES_ONDEMAND_MODE, &cache->flags))
-		return 0;
-
-	if (test_bit(CACHEFILES_DEAD, &cache->flags)) {
-		ret = -EIO;
-		goto out;
-	}
-
-	req = kzalloc(sizeof(*req) + data_len, GFP_KERNEL);
-	if (!req) {
-		ret = -ENOMEM;
-		goto out;
-	}
-
-	refcount_set(&req->ref, 1);
-	req->object = object;
-	init_completion(&req->done);
-	req->msg.opcode = opcode;
-	req->msg.len = sizeof(struct cachefiles_msg) + data_len;
-
-	ret = init_req(req, private);
-	if (ret)
-		goto out;
-
-	do {
-		/*
-		 * Stop enqueuing the request when daemon is dying. The
-		 * following two operations need to be atomic as a whole.
-		 *   1) check cache state, and
-		 *   2) enqueue request if cache is alive.
-		 * Otherwise the request may be enqueued after xarray has been
-		 * flushed, leaving the orphan request never being completed.
-		 *
-		 * CPU 1			CPU 2
-		 * =====			=====
-		 *				test CACHEFILES_DEAD bit
-		 * set CACHEFILES_DEAD bit
-		 * flush requests in the xarray
-		 *				enqueue the request
-		 */
-		xas_lock(&xas);
-
-		if (test_bit(CACHEFILES_DEAD, &cache->flags) ||
-		    cachefiles_ondemand_object_is_dropping(object)) {
-			xas_unlock(&xas);
-			ret = -EIO;
-			goto out;
-		}
-
-		/* coupled with the barrier in cachefiles_flush_reqs() */
-		smp_mb();
-
-		if (opcode == CACHEFILES_OP_CLOSE &&
-		    !cachefiles_ondemand_object_is_open(object)) {
-			WARN_ON_ONCE(object->ondemand->ondemand_id == 0);
-			xas_unlock(&xas);
-			ret = -EIO;
-			goto out;
-		}
-
-		/*
-		 * Cyclically find a free xas to avoid msg_id reuse that would
-		 * cause the daemon to successfully copen a stale msg_id.
-		 */
-		xas.xa_index = cache->msg_id_next;
-		xas_find_marked(&xas, UINT_MAX, XA_FREE_MARK);
-		if (xas.xa_node == XAS_RESTART) {
-			xas.xa_index = 0;
-			xas_find_marked(&xas, cache->msg_id_next - 1, XA_FREE_MARK);
-		}
-		if (xas.xa_node == XAS_RESTART)
-			xas_set_err(&xas, -EBUSY);
-
-		xas_store(&xas, req);
-		if (xas_valid(&xas)) {
-			cache->msg_id_next = xas.xa_index + 1;
-			xas_clear_mark(&xas, XA_FREE_MARK);
-			xas_set_mark(&xas, CACHEFILES_REQ_NEW);
-		}
-		xas_unlock(&xas);
-	} while (xas_nomem(&xas, GFP_KERNEL));
-
-	ret = xas_error(&xas);
-	if (ret)
-		goto out;
-
-	wake_up_all(&cache->daemon_pollwq);
-wait:
-	ret = wait_for_completion_killable(&req->done);
-	if (!ret) {
-		ret = req->error;
-	} else {
-		ret = -EINTR;
-		if (!cachefiles_ondemand_finish_req(req, &xas, ret)) {
-			/* Someone will complete it soon. */
-			cpu_relax();
-			goto wait;
-		}
-	}
-	cachefiles_req_put(req);
-	return ret;
-out:
-	/* Reset the object to close state in error handling path.
-	 * If error occurs after creating the anonymous fd,
-	 * cachefiles_ondemand_fd_release() will set object to close.
-	 */
-	if (opcode == CACHEFILES_OP_OPEN &&
-	    !cachefiles_ondemand_object_is_dropping(object))
-		cachefiles_ondemand_set_object_close(object);
-	kfree(req);
-	return ret;
-}
-
-static int cachefiles_ondemand_init_open_req(struct cachefiles_req *req,
-					     void *private)
-{
-	struct cachefiles_object *object = req->object;
-	struct fscache_cookie *cookie = object->cookie;
-	struct fscache_volume *volume = object->volume->vcookie;
-	struct cachefiles_open *load = (void *)req->msg.data;
-	size_t volume_key_size, cookie_key_size;
-	void *volume_key, *cookie_key;
-
-	/*
-	 * Volume key is a NUL-terminated string. key[0] stores strlen() of the
-	 * string, followed by the content of the string (excluding '\0').
-	 */
-	volume_key_size = volume->key[0] + 1;
-	volume_key = volume->key + 1;
-
-	/* Cookie key is binary data, which is netfs specific. */
-	cookie_key_size = cookie->key_len;
-	cookie_key = fscache_get_key(cookie);
-
-	if (!(object->cookie->advice & FSCACHE_ADV_WANT_CACHE_SIZE)) {
-		pr_err("WANT_CACHE_SIZE is needed for on-demand mode\n");
-		return -EINVAL;
-	}
-
-	load->volume_key_size = volume_key_size;
-	load->cookie_key_size = cookie_key_size;
-	memcpy(load->data, volume_key, volume_key_size);
-	memcpy(load->data + volume_key_size, cookie_key, cookie_key_size);
-
-	return 0;
-}
-
-static int cachefiles_ondemand_init_close_req(struct cachefiles_req *req,
-					      void *private)
-{
-	struct cachefiles_object *object = req->object;
-
-	if (!cachefiles_ondemand_object_is_open(object))
-		return -ENOENT;
-
-	trace_cachefiles_ondemand_close(object, &req->msg);
-	return 0;
-}
-
-struct cachefiles_read_ctx {
-	loff_t off;
-	size_t len;
-};
-
-static int cachefiles_ondemand_init_read_req(struct cachefiles_req *req,
-					     void *private)
-{
-	struct cachefiles_object *object = req->object;
-	struct cachefiles_read *load = (void *)req->msg.data;
-	struct cachefiles_read_ctx *read_ctx = private;
-
-	load->off = read_ctx->off;
-	load->len = read_ctx->len;
-	trace_cachefiles_ondemand_read(object, &req->msg, load);
-	return 0;
-}
-
-int cachefiles_ondemand_init_object(struct cachefiles_object *object)
-{
-	struct fscache_cookie *cookie = object->cookie;
-	struct fscache_volume *volume = object->volume->vcookie;
-	size_t volume_key_size, cookie_key_size, data_len;
-
-	if (!object->ondemand)
-		return 0;
-
-	/*
-	 * CacheFiles will firstly check the cache file under the root cache
-	 * directory. If the coherency check failed, it will fallback to
-	 * creating a new tmpfile as the cache file. Reuse the previously
-	 * allocated object ID if any.
-	 */
-	if (cachefiles_ondemand_object_is_open(object))
-		return 0;
-
-	volume_key_size = volume->key[0] + 1;
-	cookie_key_size = cookie->key_len;
-	data_len = sizeof(struct cachefiles_open) +
-		   volume_key_size + cookie_key_size;
-
-	return cachefiles_ondemand_send_req(object, CACHEFILES_OP_OPEN,
-			data_len, cachefiles_ondemand_init_open_req, NULL);
-}
-
-void cachefiles_ondemand_clean_object(struct cachefiles_object *object)
-{
-	unsigned long index;
-	struct cachefiles_req *req;
-	struct cachefiles_cache *cache;
-
-	if (!object->ondemand)
-		return;
-
-	cachefiles_ondemand_send_req(object, CACHEFILES_OP_CLOSE, 0,
-			cachefiles_ondemand_init_close_req, NULL);
-
-	if (!object->ondemand->ondemand_id)
-		return;
-
-	/* Cancel all requests for the object that is being dropped. */
-	cache = object->volume->cache;
-	xa_lock(&cache->reqs);
-	cachefiles_ondemand_set_object_dropping(object);
-	xa_for_each(&cache->reqs, index, req) {
-		if (req->object == object) {
-			req->error = -EIO;
-			complete(&req->done);
-			__xa_erase(&cache->reqs, index);
-		}
-	}
-	xa_unlock(&cache->reqs);
-
-	/* Wait for ondemand_object_worker() to finish to avoid UAF. */
-	cancel_work_sync(&object->ondemand->ondemand_work);
-}
-
-int cachefiles_ondemand_init_obj_info(struct cachefiles_object *object,
-				struct cachefiles_volume *volume)
-{
-	if (!cachefiles_in_ondemand_mode(volume->cache))
-		return 0;
-
-	object->ondemand = kzalloc_obj(struct cachefiles_ondemand_info);
-	if (!object->ondemand)
-		return -ENOMEM;
-
-	object->ondemand->object = object;
-	spin_lock_init(&object->ondemand->lock);
-	INIT_WORK(&object->ondemand->ondemand_work, ondemand_object_worker);
-	return 0;
-}
-
-void cachefiles_ondemand_deinit_obj_info(struct cachefiles_object *object)
-{
-	kfree(object->ondemand);
-	object->ondemand = NULL;
-}
-
-int cachefiles_ondemand_read(struct cachefiles_object *object,
-			     loff_t pos, size_t len)
-{
-	struct cachefiles_read_ctx read_ctx = {pos, len};
-
-	return cachefiles_ondemand_send_req(object, CACHEFILES_OP_READ,
-			sizeof(struct cachefiles_read),
-			cachefiles_ondemand_init_read_req, &read_ctx);
-}
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index c8f4941b0ad7..dc8db260260a 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -192,7 +192,6 @@ struct netfs_io_subrequest {
 #define NETFS_SREQ_COPY_TO_CACHE	0	/* Set if should copy the data to the cache */
 #define NETFS_SREQ_CLEAR_TAIL		1	/* Set if the rest of the read should be cleared */
 #define NETFS_SREQ_MADE_PROGRESS	4	/* Set if we transferred at least some data */
-#define NETFS_SREQ_ONDEMAND		5	/* Set if it's from on-demand read mode */
 #define NETFS_SREQ_BOUNDARY		6	/* Set if ends on hard boundary (eg. ceph object) */
 #define NETFS_SREQ_HIT_EOF		7	/* Set if short due to EOF */
 #define NETFS_SREQ_IN_PROGRESS		8	/* Unlocked when the subrequest completes */
@@ -375,14 +374,6 @@ struct netfs_cache_ops {
 			     loff_t *_start, size_t *_len, size_t upper_len,
 			     loff_t i_size, bool no_space_allocated_yet);
 
-	/* Prepare an on-demand read operation, shortening it to a cached/uncached
-	 * boundary as appropriate.
-	 */
-	enum netfs_io_source (*prepare_ondemand_read)(struct netfs_cache_resources *cres,
-						      loff_t start, size_t *_len,
-						      loff_t i_size,
-						      unsigned long *_flags, ino_t ino);
-
 	/* Query the occupancy of the cache in a region, returning where the
 	 * next chunk of data starts and how long it is.
 	 */
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 6e3b1424eea4..9259bc71049e 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -31,10 +31,6 @@ enum cachefiles_obj_ref_trace {
 	cachefiles_obj_see_lookup_failed,
 	cachefiles_obj_see_withdraw_cookie,
 	cachefiles_obj_see_withdrawal,
-	cachefiles_obj_get_ondemand_fd,
-	cachefiles_obj_put_ondemand_fd,
-	cachefiles_obj_get_read_req,
-	cachefiles_obj_put_read_req,
 };
 
 enum fscache_why_object_killed {
@@ -129,11 +125,7 @@ enum cachefiles_error_trace {
 	EM(cachefiles_obj_see_lookup_cookie,	"SEE lookup_cookie")	\
 	EM(cachefiles_obj_see_lookup_failed,	"SEE lookup_failed")	\
 	EM(cachefiles_obj_see_withdraw_cookie,	"SEE withdraw_cookie")	\
-	EM(cachefiles_obj_see_withdrawal,	"SEE withdrawal")	\
-	EM(cachefiles_obj_get_ondemand_fd,      "GET ondemand_fd")	\
-	EM(cachefiles_obj_put_ondemand_fd,      "PUT ondemand_fd")	\
-	EM(cachefiles_obj_get_read_req,		"GET read_req")		\
-	E_(cachefiles_obj_put_read_req,		"PUT read_req")
+	E_(cachefiles_obj_see_withdrawal,	"SEE withdrawal")
 
 #define cachefiles_coherency_traces					\
 	EM(cachefiles_coherency_check_aux,	"BAD aux ")		\
@@ -687,180 +679,6 @@ TRACE_EVENT(cachefiles_io_error,
 		      __entry->error)
 	    );
 
-TRACE_EVENT(cachefiles_ondemand_open,
-	    TP_PROTO(struct cachefiles_object *obj, struct cachefiles_msg *msg,
-		     struct cachefiles_open *load),
-
-	    TP_ARGS(obj, msg, load),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-		    __field(unsigned int,	object_id)
-		    __field(unsigned int,	fd)
-		    __field(unsigned int,	flags)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg->msg_id;
-		    __entry->object_id	= msg->object_id;
-		    __entry->fd		= load->fd;
-		    __entry->flags	= load->flags;
-			   ),
-
-	    TP_printk("o=%08x mid=%x oid=%x fd=%d f=%x",
-		      __entry->obj,
-		      __entry->msg_id,
-		      __entry->object_id,
-		      __entry->fd,
-		      __entry->flags)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_copen,
-	    TP_PROTO(struct cachefiles_object *obj, unsigned int msg_id,
-		     long len),
-
-	    TP_ARGS(obj, msg_id, len),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-		    __field(long,		len)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg_id;
-		    __entry->len	= len;
-			   ),
-
-	    TP_printk("o=%08x mid=%x l=%lx",
-		      __entry->obj,
-		      __entry->msg_id,
-		      __entry->len)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_close,
-	    TP_PROTO(struct cachefiles_object *obj, struct cachefiles_msg *msg),
-
-	    TP_ARGS(obj, msg),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-		    __field(unsigned int,	object_id)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg->msg_id;
-		    __entry->object_id	= msg->object_id;
-			   ),
-
-	    TP_printk("o=%08x mid=%x oid=%x",
-		      __entry->obj,
-		      __entry->msg_id,
-		      __entry->object_id)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_read,
-	    TP_PROTO(struct cachefiles_object *obj, struct cachefiles_msg *msg,
-		     struct cachefiles_read *load),
-
-	    TP_ARGS(obj, msg, load),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-		    __field(unsigned int,	object_id)
-		    __field(loff_t,		start)
-		    __field(size_t,		len)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg->msg_id;
-		    __entry->object_id	= msg->object_id;
-		    __entry->start	= load->off;
-		    __entry->len	= load->len;
-			   ),
-
-	    TP_printk("o=%08x mid=%x oid=%x s=%llx l=%zx",
-		      __entry->obj,
-		      __entry->msg_id,
-		      __entry->object_id,
-		      __entry->start,
-		      __entry->len)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_cread,
-	    TP_PROTO(struct cachefiles_object *obj, unsigned int msg_id),
-
-	    TP_ARGS(obj, msg_id),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	msg_id)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->msg_id	= msg_id;
-			   ),
-
-	    TP_printk("o=%08x mid=%x",
-		      __entry->obj,
-		      __entry->msg_id)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_fd_write,
-	    TP_PROTO(struct cachefiles_object *obj, struct inode *backer,
-		     loff_t start, size_t len),
-
-	    TP_ARGS(obj, backer, start, len),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	backer)
-		    __field(loff_t,		start)
-		    __field(size_t,		len)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->backer	= backer->i_ino;
-		    __entry->start	= start;
-		    __entry->len	= len;
-			   ),
-
-	    TP_printk("o=%08x iB=%x s=%llx l=%zx",
-		      __entry->obj,
-		      __entry->backer,
-		      __entry->start,
-		      __entry->len)
-	    );
-
-TRACE_EVENT(cachefiles_ondemand_fd_release,
-	    TP_PROTO(struct cachefiles_object *obj, int object_id),
-
-	    TP_ARGS(obj, object_id),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,	obj)
-		    __field(unsigned int,	object_id)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->obj	= obj ? obj->debug_id : 0;
-		    __entry->object_id	= object_id;
-			   ),
-
-	    TP_printk("o=%08x oid=%x",
-		      __entry->obj,
-		      __entry->object_id)
-	    );
-
 #endif /* _TRACE_CACHEFILES_H */
 
 /* This part must be outside protection */
diff --git a/include/uapi/linux/cachefiles.h b/include/uapi/linux/cachefiles.h
deleted file mode 100644
index 78caa73e5343..000000000000
--- a/include/uapi/linux/cachefiles.h
+++ /dev/null
@@ -1,68 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
-#ifndef _LINUX_CACHEFILES_H
-#define _LINUX_CACHEFILES_H
-
-#include <linux/types.h>
-#include <linux/ioctl.h>
-
-/*
- * Fscache ensures that the maximum length of cookie key is 255. The volume key
- * is controlled by netfs, and generally no bigger than 255.
- */
-#define CACHEFILES_MSG_MAX_SIZE	1024
-
-enum cachefiles_opcode {
-	CACHEFILES_OP_OPEN,
-	CACHEFILES_OP_CLOSE,
-	CACHEFILES_OP_READ,
-};
-
-/*
- * Message Header
- *
- * @msg_id	a unique ID identifying this message
- * @opcode	message type, CACHEFILE_OP_*
- * @len		message length, including message header and following data
- * @object_id	a unique ID identifying a cache file
- * @data	message type specific payload
- */
-struct cachefiles_msg {
-	__u32 msg_id;
-	__u32 opcode;
-	__u32 len;
-	__u32 object_id;
-	__u8  data[];
-};
-
-/*
- * @data contains the volume_key followed directly by the cookie_key. volume_key
- * is a NUL-terminated string; @volume_key_size indicates the size of the volume
- * key in bytes. cookie_key is binary data, which is netfs specific;
- * @cookie_key_size indicates the size of the cookie key in bytes.
- *
- * @fd identifies an anon_fd referring to the cache file.
- */
-struct cachefiles_open {
-	__u32 volume_key_size;
-	__u32 cookie_key_size;
-	__u32 fd;
-	__u32 flags;
-	__u8  data[];
-};
-
-/*
- * @off		indicates the starting offset of the requested file range
- * @len		indicates the length of the requested file range
- */
-struct cachefiles_read {
-	__u64 off;
-	__u64 len;
-};
-
-/*
- * Reply for READ request
- * @arg for this ioctl is the @id field of READ request.
- */
-#define CACHEFILES_IOC_READ_COMPLETE	_IOW(0x98, 1, int)
-
-#endif
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index c4673a54c4ba..7e25051dc1ea 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -6676,13 +6676,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
 	{ "cachefiles_mark_inactive", 0x1 },
 	{ "cachefiles_vfs_error", 0x1 },
 	{ "cachefiles_io_error", 0x1 },
-	{ "cachefiles_ondemand_open", 0x1 },
-	{ "cachefiles_ondemand_copen", 0x1 },
-	{ "cachefiles_ondemand_close", 0x1 },
-	{ "cachefiles_ondemand_read", 0x1 },
-	{ "cachefiles_ondemand_cread", 0x1 },
-	{ "cachefiles_ondemand_fd_write", 0x1 },
-	{ "cachefiles_ondemand_fd_release", 0x1 },
 	/* ext4, from ext4__mballoc event class */
 	{ "ext4_mballoc_discard", 0x10 },
 	{ "ext4_mballoc_free", 0x10 },



From dhowells at redhat.com  Mon Aug 10 07:47:32 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:32 +0100
Subject: [PATCH v9 15/26] cachefiles: Don't rely on backing fs storage map for most use cases
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-16-dhowells@redhat.com>

Cachefiles currently uses the backing filesystem's idea of what data is
held in a backing file and queries this by means of SEEK_DATA and
SEEK_HOLE.  However, this means it does two seek operations on the backing
file for each individual read call it wants to prepare (unless the first
returns -ENXIO).  Worse, the backing filesystem is at liberty to insert or
remove blocks of zeros in order to optimise its layout which may cause
false positives and false negatives.

The problem is that keeping track of what is dirty is tricky (if storing
info in xattrs, which may have limited capacity and must be read and
written as one piece) and expensive (in terms of diskspace at least) and is
basically duplicating what a filesystem does.

However, the most common write case, in which the application does {
open(O_TRUNC); write(); write(); ... write(); close(); } where each write
follows directly on from the previous and leaves no gaps in the file is
reasonably easy to detect and can be noted in the primary xattr as
CACHEFILES_CONTENT_ALL, indicating we have everything up to the object size
stored.

In this specific case, given that it is known that there are no holes in
the file, there's no need to call SEEK_DATA/HOLE or use any other mechanism
to track the contents.  That speeds things up enormously.

Even when it is necessary to use SEEK_DATA/HOLE, it may not be necessary to
call it for each cache read subrequest generated.

Implement this by adding support for the CACHEFILES_CONTENT_ALL content
type (which is defined, but currently unused), which requires a slight
adjustment in how backing files are managed.  Specifically, the driver
needs to know how much of the tail block is data and whether storing more
data will create a hole.

To this end, the way that the size of a backing file is managed is changed.
Currently, the backing file is expanded to strictly match the size of the
network file, but this can be changed to carry more useful information.
This makes two pieces of metadata available: xattr.object_size and the
backing file's i_size.  Apply the following schema:

  (a) i_size is always a multiple of the DIO block size.

  (b) i_size is only updated to the end of the highest write stored.  This
      is used to work out if we are following on without leaving a hole.

  (c) xattr.object_size is the size of the network filesystem file cached
      in this backing file.

  (d) xattr.object_size must point after the start of the last block
      (unless both are 0).

  (e) If xattr.object_size is at or after the block at the current end of
      the backing file (ie. i_size), then we have all the contents of the
      block (if xattr.content == CACHEFILES_CONTENT_ALL).

  (f) If xattr.object_size is somewhere in the middle of the last block,
      then the data following it is invalid and must be ignored.

  (g) If data is added to the last block, then that block must be fetched,
      modified and rewritten (it must be a buffered write through the
      pagecache and not DIO).

  (h) Writes to cache are rounded out to blocks on both sides and the
      folios used as sources must contain data for any lower gap and must
      have been cleared for any upper gap, and so will rewrite any
      non-data area in the tail block.

To implement this, the following changes are made:

 (1) cookie->object_size is no longer updated when writes are copied into
     the pagecache, but rather only updated when a write request completes.

     This prevents object size miscomparison when checking the xattr
     causing the backing file to be invalidated (opening and marking the
     backing file and modifying the pagecache run in parallel).

 (2) The cache's current idea of the amount of data that should be stored
     in the backing file is kept track of in object->object_size.

     Possibly this is redundant with cookie->object_size, but the latter
     gets updated in some addition circumstances.

 (3) The size of the backing file at the start of a request is now tracked
     in struct netfs_cache_resources so that the partial EOF block can be
     located and cleaned.

 (4) The cache block size is now used consistently rather than using
     CACHEFILES_DIO_BLOCK_SIZE (4096).

 (5) The backing file size is no longer adjusted when looking up an object.

 (6) When shortening a file, if the new size is not block aligned, the part
     beyond the new size is cleared.  If the file is truncated to zero, the
     content_info gets reset to CACHEFILES_CONTENT_NO_DATA.

 (7) A new struct, fscache_occupancy, is instituted to track the region
     being read.  Netfslib allocates it and fills in the start and end of
     the region to be read then calls the ->query_occupancy() method to
     find and fill in the extents.  It also indicates whether a recorded
     extent contains data or just contains a region that's all zeros
     (FSCACHE_EXTENT_DATA or FSCACHE_EXTENT_ZERO).

 (8) The ->prepare_read() cache method is changed such that, if given, it
     just limits the amount that can be read from the cache in one go.  It
     no longer indicates what source of read should be done; that
     information is now obtained from ->query_occupancy().

 (9) A new cache method, ->collect_write(), is added that is called when a
     contiguous series of writes have completed and a discontiguity or the
     end of the request has been hit.  It it supplied with the start and
     length of the write made to the backing file and can use this
     information to update the cache metadata.

(10) cachefiles_query_occupancy() is altered to find the next two "extents"
     of data stored in the backing file by doing SEEK_DATA/HOLE between the
     bounds set - unless it is known that there are no holes, in which case
     a whole-file first extent can be set.

(11) cachefiles_collect_write() is implemented to take the collated write
     completion information and use this to update the cache metadata, in
     particular working out whether there's now a hole in the backing file
     requiring future use of SEEK_DATA/HOLE instead of just assuming the
     data is all present.

     It also uses fallocate(FALLOC_FL_ZERO_RANGE) to clean the part of a
     partial block that extended beyond the old object size.  It might be
     better to perform a synchronous DIO write for this purpose, but that
     would mandate an RMW cycle.  Ideally, it should be all zeros anyway,
     but, unfortunately, shared-writable mmap can interfere.

(12) cachefiles_begin_operation() is updated to note the current backing
     file size and the cache DIO size.

(13) cachefiles_create_tmpfile() no longer expands the backing file when it
     creates it.

(14) cachefiles_set_object_xattr() is changed to use object->object_size
     rather than cookie->object_size.

(15) cachefiles_check_auxdata() is altered to actually store the content
     type and to also set object->object_size.  The cachefiles_coherency
     tracepoint is also modified to display xattr.object_size.

(16) netfs_read_to_pagecache() is reworked.  The cache ->prepare_read()
     method is replaced with ->query_occupancy() as the arbiter of what
     region of the file is read from where, and that retrieves up to two
     occupied extents of the backing file at once.

     The cache ->prepare_read() method is now repurposed to be the same as
     the equivalent network filesystem method and allows the cache to limit
     the size of the read before the iterator is prepared.

     netfs_single_dispatch_read() is similarly modified.

(17) netfs_update_i_size() and afs_update_i_size() no longer call
     fscache_update_cookie() to update cookie->object_size.

(18) Write collection now collates contiguous sequences of writes to the
     cache and calls the cache ->collect_write() method.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/afs/file.c                     |   1 -
 fs/cachefiles/interface.c         |  90 ++------
 fs/cachefiles/internal.h          |  13 +-
 fs/cachefiles/io.c                | 335 ++++++++++++++++++++++++------
 fs/cachefiles/namei.c             |  19 +-
 fs/cachefiles/xattr.c             |  24 ++-
 fs/netfs/buffered_read.c          | 176 ++++++++++------
 fs/netfs/buffered_write.c         |   3 -
 fs/netfs/internal.h               |   2 +
 fs/netfs/objects.c                |   1 +
 fs/netfs/read_retry.c             |   2 +
 fs/netfs/read_single.c            |  39 ++--
 fs/netfs/write_collect.c          | 132 ++++++++++--
 fs/netfs/write_issue.c            |  18 ++
 fs/netfs/write_retry.c            |   3 +
 include/linux/fscache.h           |  17 ++
 include/linux/netfs.h             |  40 +++-
 include/trace/events/cachefiles.h |  21 +-
 include/trace/events/netfs.h      |   9 +-
 19 files changed, 658 insertions(+), 287 deletions(-)

diff --git a/fs/afs/file.c b/fs/afs/file.c
index 3617c245f319..92380dc25781 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -447,7 +447,6 @@ void afs_set_i_size(struct afs_vnode *vnode, loff_t new_i_size)
 	}
 	spin_unlock(&inode->i_lock);
 	write_sequnlock(&vnode->cb_lock);
-	fscache_update_cookie(afs_vnode_cache(vnode), NULL, &new_i_size);
 }
 
 static void afs_update_i_size(struct inode *inode, loff_t new_i_size)
diff --git a/fs/cachefiles/interface.c b/fs/cachefiles/interface.c
index 50a000310a8c..638e00717119 100644
--- a/fs/cachefiles/interface.c
+++ b/fs/cachefiles/interface.c
@@ -99,73 +99,6 @@ void cachefiles_put_object(struct cachefiles_object *object,
 	_leave("");
 }
 
-/*
- * Adjust the size of a cache file if necessary to match the DIO size.  We keep
- * the EOF marker a multiple of DIO blocks so that we don't fall back to doing
- * non-DIO for a partial block straddling the EOF, but we also have to be
- * careful of someone expanding the file and accidentally accreting the
- * padding.
- */
-static int cachefiles_adjust_size(struct cachefiles_object *object)
-{
-	struct iattr newattrs;
-	struct file *file = object->file;
-	uint64_t ni_size;
-	loff_t oi_size;
-	int ret;
-
-	ni_size = object->cookie->object_size;
-	ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
-
-	_enter("{OBJ%x},[%llu]",
-	       object->debug_id, (unsigned long long) ni_size);
-
-	if (!file)
-		return -ENOBUFS;
-
-	oi_size = i_size_read(file_inode(file));
-	if (oi_size == ni_size)
-		return 0;
-
-	inode_lock(file_inode(file));
-
-	/* if there's an extension to a partial page at the end of the backing
-	 * file, we need to discard the partial page so that we pick up new
-	 * data after it */
-	if (oi_size & ~PAGE_MASK && ni_size > oi_size) {
-		_debug("discard tail %llx", oi_size);
-		newattrs.ia_valid = ATTR_SIZE;
-		newattrs.ia_size = oi_size & PAGE_MASK;
-		ret = cachefiles_inject_remove_error();
-		if (ret == 0)
-			ret = notify_change(&nop_mnt_idmap, file->f_path.dentry,
-					    &newattrs, NULL);
-		if (ret < 0)
-			goto truncate_failed;
-	}
-
-	newattrs.ia_valid = ATTR_SIZE;
-	newattrs.ia_size = ni_size;
-	ret = cachefiles_inject_write_error();
-	if (ret == 0)
-		ret = notify_change(&nop_mnt_idmap, file->f_path.dentry,
-				    &newattrs, NULL);
-
-truncate_failed:
-	inode_unlock(file_inode(file));
-
-	if (ret < 0)
-		trace_cachefiles_io_error(NULL, file_inode(file), ret,
-					  cachefiles_trace_notify_change_error);
-	if (ret == -EIO) {
-		cachefiles_io_error_obj(object, "Size set failed");
-		ret = -ENOBUFS;
-	}
-
-	_leave(" = %d", ret);
-	return ret;
-}
-
 /*
  * Attempt to look up the nominated node in this cache
  */
@@ -198,7 +131,6 @@ static bool cachefiles_lookup_cookie(struct fscache_cookie *cookie)
 	spin_lock(&cache->object_list_lock);
 	list_add(&object->cache_link, &cache->object_list);
 	spin_unlock(&cache->object_list_lock);
-	cachefiles_adjust_size(object);
 
 	cachefiles_end_secure(cache, saved_cred);
 	_leave(" = t");
@@ -232,7 +164,7 @@ static bool cachefiles_shorten_object(struct cachefiles_object *object,
 	loff_t i_size, dio_size;
 	int ret;
 
-	dio_size = round_up(new_size, CACHEFILES_DIO_BLOCK_SIZE);
+	dio_size = round_up(new_size, cache->bsize);
 	i_size = i_size_read(inode);
 
 	trace_cachefiles_trunc(object, inode, i_size, dio_size,
@@ -264,6 +196,7 @@ static bool cachefiles_shorten_object(struct cachefiles_object *object,
 		}
 	}
 
+	object->object_size = new_size;
 	return true;
 }
 
@@ -278,22 +211,31 @@ static void cachefiles_resize_cookie(struct netfs_cache_resources *cres,
 	struct fscache_cookie *cookie = object->cookie;
 	const struct cred *saved_cred;
 	struct file *file = cachefiles_cres_file(cres);
-	loff_t old_size = cookie->object_size;
+	unsigned long long i_size = i_size_read(file_inode(file));
 
-	_enter("%llu->%llu", old_size, new_size);
+	_enter("%llu->%llu", object->object_size, new_size);
 
-	if (new_size < old_size) {
+	/* If the file is being shrunk, we need to downsize the backing file
+	 * and clear the end of the final block.
+	 */
+	if (new_size < object->object_size) {
+		if (new_size >= i_size)
+			goto out;
 		cachefiles_begin_secure(cache, &saved_cred);
 		cachefiles_shorten_object(object, file, new_size);
 		cachefiles_end_secure(cache, saved_cred);
 		object->cookie->object_size = new_size;
+		if (new_size == 0)
+			object->content_info = CACHEFILES_CONTENT_NO_DATA;
 		return;
 	}
 
 	/* The file is being expanded.  We don't need to do anything
-	 * particularly.  cookie->initial_size doesn't change and so the point
-	 * at which we have to download before doesn't change.
+	 * particularly.  The tail of the last block should have been cleared
+	 * both when it is written and when it is shrunk.
 	 */
+out:
+	object->object_size = new_size;
 	cookie->object_size = new_size;
 }
 
diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index 409492744bde..de5e5ef01c8f 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -16,8 +16,6 @@
 #include <linux/cred.h>
 #include <linux/security.h>
 
-#define CACHEFILES_DIO_BLOCK_SIZE 4096
-
 struct cachefiles_cache;
 struct cachefiles_object;
 
@@ -51,12 +49,17 @@ struct cachefiles_object {
 	struct list_head		cache_link;	/* Link in cache->*_list */
 	struct file			*file;		/* The file representing this object */
 	char				*d_name;	/* Backing file name */
+	unsigned long			flags;
+#define CACHEFILES_OBJECT_USING_TMPFILE	0		/* Have an unlinked tmpfile */
+	unsigned long long		object_size;	/* Size of the object stored
+							 * (independent of cookie->object_size for
+							 * coherency reasons)
+							 */
+	atomic64_t			read_limit;	/* Point beyond which uncommitted writes */
 	int				debug_id;
 	spinlock_t			lock;
 	refcount_t			ref;
-	enum cachefiles_content		content_info:8;	/* Info about content presence */
-	unsigned long			flags;
-#define CACHEFILES_OBJECT_USING_TMPFILE	0		/* Have an unlinked tmpfile */
+	enum cachefiles_content		content_info;	/* Info about content presence */
 };
 
 /*
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 9540ec25b3cb..0289c4935635 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -32,6 +32,8 @@ struct cachefiles_kiocb {
 	u64			b_writing;
 };
 
+#define IS_ERR_VALUE_LL(x) unlikely((x) >= (unsigned long long)-MAX_ERRNO)
+
 static inline void cachefiles_put_kiocb(struct cachefiles_kiocb *ki)
 {
 	if (refcount_dec_and_test(&ki->ki_refcnt)) {
@@ -193,60 +195,81 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
 }
 
 /*
- * Query the occupancy of the cache in a region, returning where the next chunk
- * of data starts and how long it is.
+ * Query the occupancy of the cache in a region, returning the extent of the
+ * next two chunks of cached data and the next hole.
  */
 static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
-				      loff_t start, size_t len, size_t granularity,
-				      loff_t *_data_start, size_t *_data_len)
+				      struct fscache_occupancy *occ)
 {
 	struct cachefiles_object *object;
+	struct inode *inode;
 	struct file *file;
-	loff_t off, off2;
-
-	*_data_start = -1;
-	*_data_len = 0;
+	unsigned long long read_limit;
+	loff_t ret;
+	int i;
 
 	if (!fscache_wait_for_operation(cres, FSCACHE_WANT_READ))
 		return -ENOBUFS;
 
 	object = cachefiles_cres_object(cres);
 	file = cachefiles_cres_file(cres);
-	granularity = max_t(size_t, object->volume->cache->bsize, granularity);
+	inode = file_inode(file);
+	occ->granularity = object->volume->cache->bsize;
+	/* Read read_limit before content_info. */
+	read_limit = atomic64_read_acquire(&object->read_limit);
+
+	_enter("%pD,%llu,%llx-%llx/%llx",
+	       file, inode->i_ino, occ->query_from, occ->query_to, read_limit);
+
+	if (read_limit == 0)
+		goto done;
+
+	switch (READ_ONCE(object->content_info)) {
+	case CACHEFILES_CONTENT_ALL:
+	case CACHEFILES_CONTENT_SINGLE:
+		if (read_limit > occ->query_from) {
+			occ->cached_from[0] = 0;
+			occ->cached_to[0] = read_limit;
+			occ->cached_type[0] = FSCACHE_EXTENT_DATA;
+			occ->query_from = ULLONG_MAX;
+		}
+		goto done;
+	default:
+		break;
+	}
 
-	_enter("%pD,%llu,%llx,%zx/%llx",
-	       file, file_inode(file)->i_ino, start, len,
-	       i_size_read(file_inode(file)));
+	for (i = 0; i < ARRAY_SIZE(occ->cached_from); i++) {
+		ret = cachefiles_inject_read_error();
+		if (ret == 0)
+			ret = vfs_llseek(file, occ->query_from, SEEK_DATA);
+		if (IS_ERR_VALUE_LL(ret)) {
+			if (ret != -ENXIO)
+				return ret;
+			occ->query_from = ULLONG_MAX;
+			goto done;
+		}
+		occ->cached_type[i] = FSCACHE_EXTENT_DATA;
+		occ->cached_from[i] = ret;
+		occ->query_from = ret;
+
+		ret = cachefiles_inject_read_error();
+		if (ret == 0)
+			ret = vfs_llseek(file, occ->query_from, SEEK_HOLE);
+		if (IS_ERR_VALUE_LL(ret)) {
+			if (ret != -ENXIO)
+				return ret;
+			occ->query_from = ULLONG_MAX;
+			goto done;
+		}
+		occ->cached_to[i] = ret;
+		occ->query_from = ret;
+		if (occ->query_from >= occ->query_to)
+			break;
+	}
 
-	off = cachefiles_inject_read_error();
-	if (off == 0)
-		off = vfs_llseek(file, start, SEEK_DATA);
-	if (off == -ENXIO)
-		return -ENODATA; /* Beyond EOF */
-	if (off < 0 && off >= (loff_t)-MAX_ERRNO)
-		return -ENOBUFS; /* Error. */
-	if (round_up(off, granularity) >= start + len)
-		return -ENODATA; /* No data in range */
-
-	off2 = cachefiles_inject_read_error();
-	if (off2 == 0)
-		off2 = vfs_llseek(file, off, SEEK_HOLE);
-	if (off2 == -ENXIO)
-		return -ENODATA; /* Beyond EOF */
-	if (off2 < 0 && off2 >= (loff_t)-MAX_ERRNO)
-		return -ENOBUFS; /* Error. */
-
-	/* Round away partial blocks */
-	off = round_up(off, granularity);
-	off2 = round_down(off2, granularity);
-	if (off2 <= off)
-		return -ENODATA;
-
-	*_data_start = off;
-	if (off2 > start + len)
-		*_data_len = len;
-	else
-		*_data_len = off2 - off;
+done:
+	_debug("query[0] %llx-%llx", occ->cached_from[0], occ->cached_to[0]);
+	_debug("query[1] %llx-%llx", occ->cached_from[1], occ->cached_to[1]);
 	return 0;
 }
 
@@ -478,18 +501,6 @@ cachefiles_do_prepare_read(struct netfs_cache_resources *cres,
 	return ret;
 }
 
-/*
- * Prepare a read operation, shortening it to a cached/uncached
- * boundary as appropriate.
- */
-static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *subreq,
-						    unsigned long long i_size)
-{
-	return cachefiles_do_prepare_read(&subreq->rreq->cache_resources,
-					  subreq->start, &subreq->len, i_size,
-					  &subreq->flags, subreq->rreq->inode->i_ino);
-}
-
 /*
  * Prepare for a write to occur.
  */
@@ -504,7 +515,7 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	int ret;
 
 	/* Round to DIO size */
-	start = round_down(*_start, PAGE_SIZE);
+	start = round_down(*_start, cache->bsize);
 	if (start != *_start || *_len > upper_len) {
 		/* Probably asked to cache a streaming write written into the
 		 * pagecache when the cookie was temporarily out of service to
@@ -514,7 +525,7 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 		return -ENOBUFS;
 	}
 
-	*_len = round_up(len, PAGE_SIZE);
+	*_len = round_up(len, cache->bsize);
 
 	/* We need to work out whether there's sufficient disk space to perform
 	 * the write - but we can skip that check if we have space already
@@ -540,7 +551,7 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	 * space, we need to see if it's fully allocated.  If it's not, we may
 	 * want to cull it.
 	 */
-	if (cachefiles_has_space(cache, 0, *_len / PAGE_SIZE,
+	if (cachefiles_has_space(cache, 0, *_len / cache->bsize,
 				 cachefiles_has_space_check) == 0)
 		return 0; /* Enough space to simply overwrite the whole block */
 
@@ -572,7 +583,7 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	return ret;
 
 check_space:
-	return cachefiles_has_space(cache, 0, *_len / PAGE_SIZE,
+	return cachefiles_has_space(cache, 0, *_len / cache->bsize,
 				    cachefiles_has_space_for_write);
 }
 
@@ -635,9 +646,9 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 	       wreq->debug_id, subreq->debug_index, start, start + len - 1);
 
 	/* We need to start on the cache granularity boundary */
-	off = start & (CACHEFILES_DIO_BLOCK_SIZE - 1);
+	off = start & (cache->bsize - 1);
 	if (off) {
-		pre = CACHEFILES_DIO_BLOCK_SIZE - off;
+		pre = cache->bsize - off;
 		if (pre >= len) {
 			fscache_count_dio_misfit();
 			netfs_write_subrequest_terminated(subreq, len);
@@ -651,8 +662,8 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 
 	/* We also need to end on the cache granularity boundary */
 	if (start + len == wreq->i_size) {
-		size_t part = len % CACHEFILES_DIO_BLOCK_SIZE;
-		size_t need = CACHEFILES_DIO_BLOCK_SIZE - part;
+		size_t part = len & (cache->bsize - 1);
+		size_t need = cache->bsize - part;
 
 		if (part && stream->submit_extendable_to >= need) {
 			len += need;
@@ -661,7 +672,7 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 		}
 	}
 
-	post = len & (CACHEFILES_DIO_BLOCK_SIZE - 1);
+	post = len & (cache->bsize - 1);
 	if (post) {
 		len -= post;
 		if (len == 0) {
@@ -688,6 +699,194 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 			 netfs_write_subrequest_terminated, subreq);
 }
 
+/*
+ * Collect the result of buffered writeback to the cache.  This includes
+ * copying a read to the cache.  Netfslib collates the results, which might
+ * occur out of order, and delivers them to the cache so that it can update its
+ * content record.
+ *
+ * block_type is one of:
+ * - NETFS_CACHE_COLLECT_WRITE_DATA for a contiguous block of data
+ * - NETFS_CACHE_COLLECT_WRITE_GAP if a discontiguity was skipped
+ * - NETFS_CACHE_COLLECT_WRITE_CANCEL for a hole due to a failed/cancelled write
+ *
+ * The writes we made are all rounded out at both sides to the nearest DIO
+ * block boundary, so if the final block contains the EOF in the middle of it
+ * (rather than at the end), padding will have been written to the file.  The
+ * backing file's filesize will have been updated if the write extended the
+ * file; the filesize may still change due to outstanding subreqs.
+ *
+ * The metadata in the cache file xattr records the size of the object we have
+ * stored, but the cache file EOF only goes up to where we've cached data to
+ * and, furthermore, is rounded up to the nearest DIO block boundary.
+ *
+ * Concurrent updates should be protected against by the caller.  Netfslib
+ * holds NETFS_ICTX_WB_LOCK as a lock on writeback requests.  DIO writes
+ * invalidate the cookie and caching is kept disabled until all users have
+ * unused the cookie.
+ */
+static void cachefiles_collect_write(struct netfs_io_request *wreq,
+				     unsigned long long start, size_t len,
+				     enum netfs_cache_collect block_type)
+{
+	struct netfs_cache_resources *cres = &wreq->cache_resources;
+	struct cachefiles_object *object = cachefiles_cres_object(cres);
+	struct cachefiles_cache *cache = object->volume->cache;
+	struct inode *inode;
+	struct file *file = cachefiles_cres_file(cres);
+	unsigned long long read_limit;
+	unsigned long long old_size = cres->cache_i_size;
+	unsigned long long new_size;
+	unsigned long long data_to = object->object_size;
+	unsigned long long end = start + len;
+	int ret;
+
+	if (!file)
+		return;
+
+	inode = file_inode(file);
+	new_size = i_size_read(inode);
+
+	_enter("%llx,%zx,%x", start, len, cache->bsize);
+
+	if (WARN_ON(old_size	& (cache->bsize - 1)) ||
+	    WARN_ON(new_size	& (cache->bsize - 1)) ||
+	    WARN_ON(start	& (cache->bsize - 1)) ||
+	    WARN_ON(len		& (cache->bsize - 1))) {
+		trace_cachefiles_io_error(object, inode, -EIO,
+					  cachefiles_trace_alignment_error);
+		cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
+		return;
+	}
+
+	/* If this is recording a gap, due to discontiguous writes or lack of
+	 * cache space, then a hole may have been introduced into the backing
+	 * file.  Treat it as a zero-length data block.
+	 */
+	if (block_type == NETFS_CACHE_COLLECT_WRITE_GAP ||
+	    block_type == NETFS_CACHE_COLLECT_WRITE_CANCEL) {
+		start = end;
+		len = 0;
+	}
+
+	/* Zeroth case: Single monolithic files are handled specially.
+	 */
+	if (wreq->origin == NETFS_WRITEBACK_SINGLE) {
+		if (block_type == NETFS_CACHE_COLLECT_WRITE_GAP ||
+		    block_type == NETFS_CACHE_COLLECT_WRITE_CANCEL) {
+			trace_cachefiles_trunc(object, inode, data_to, 0,
+					       cachefiles_trunc_zap);
+			ret = cachefiles_inject_remove_error();
+			if (ret == 0)
+				ret = vfs_truncate(&file->f_path, 0);
+			if (ret < 0) {
+				trace_cachefiles_io_error(object, inode, ret,
+							  cachefiles_trace_trunc_error);
+				cachefiles_io_error_obj(object, "truncate failed %d", ret);
+				cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
+				return;
+			}
+
+		}
+		object->content_info = CACHEFILES_CONTENT_SINGLE;
+		goto update_sizes;
+	}
+
+	/* First case: The backing file was empty. */
+	if (old_size == 0) {
+		if (start == 0)
+			object->content_info = CACHEFILES_CONTENT_ALL;
+		else
+			object->content_info = CACHEFILES_CONTENT_BACKFS_MAP;
+		goto update_sizes;
+	}
+
+	/* Second case: The backing file is entirely within the old object size
+	 * and thus there can be no partial tail block to deal with in the
+	 * cache file.
+	 */
+	if (old_size <= data_to) {
+		if (start > old_size)
+			goto discontiguous;
+		goto update_sizes;
+	}
+
+	/* Third case: The write happened entirely within the bounds of the
+	 * current cache file's size.
+	 */
+	if (end <= old_size)
+		goto update_sizes;
+
+	/* Fourth case: The write overwrote the partial tail block and extended
+	 * the file.  We only need to update the object size because netfslib
+	 * rounds out/pads cache writes to whole disk blocks.
+	 */
+	if (start < old_size)
+		goto update_sizes;
+
+	/* Fifth case: The write started from the end of the whole tail block
+	 * and extended the file.  Just extend our notion of the filesize.
+	 */
+	if (start == old_size && old_size == data_to)
+		goto update_sizes;
+
+	/* Sixth case: The write continued on from the partial tail block and
+	 * extended the file.  Need to clear the gap.
+	 */
+	if (start == old_size && old_size > data_to)
+		goto clear_gap;
+
+discontiguous:
+	/* Seventh case: The write was beyond the EOF on the cache file, so now
+	 * there's a hole in the file and we can no longer say in the metadata
+	 * that we can assume we have it all.  We may also need to clear the
+	 * end of the partial tail block.
+	 */
+	/* TODO: For the moment, we will have to use SEEK_HOLE/SEEK_DATA. */
+	if (object->content_info != CACHEFILES_CONTENT_BACKFS_MAP) {
+		object->content_info = CACHEFILES_CONTENT_BACKFS_MAP;
+		trace_cachefiles_coherency(object, inode->i_ino, data_to,
+					   be64_to_cpup((__be64 *)object->cookie->inline_aux),
+					   CACHEFILES_CONTENT_BACKFS_MAP,
+					   cachefiles_coherency_discontiguous);
+	}
+
+clear_gap:
+	/* We need to clear any partial padding that got jumped over.  It
+	 * *should* be all zeros, but shared-writable mmap exists...
+	 */
+	if (old_size > data_to) {
+		trace_cachefiles_trunc(object, inode, data_to, old_size,
+				       cachefiles_trunc_clear_padding);
+		ret = cachefiles_inject_write_error();
+		if (ret == 0)
+			ret = vfs_fallocate(file, FALLOC_FL_ZERO_RANGE,
+					    data_to, old_size - data_to);
+		if (ret < 0) {
+			trace_cachefiles_io_error(object, inode, ret,
+						  cachefiles_trace_fallocate_error);
+			cachefiles_io_error_obj(object, "fallocate zero pad failed %d", ret);
+			cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
+			return;
+		}
+	}
+
+update_sizes:
+	read_limit = umax(old_size, end);
+	cres->cache_i_size = read_limit;
+
+	/* We need to be careful setting the object_size: we may have written
+	 * more to the cache than to the server (due to cache DIO rounding) and
+	 * the i_size set on the netfs inode may include unwritten data that
+	 * the server doesn't know about yet.
+	 */
+	object->object_size = umin(read_limit, wreq->i_size);
+
+	/* Raise the limit at which reads can access the file. */
+	/* Update read_limit after content_info */
+	atomic64_set_release(&object->read_limit, read_limit);
+}
+
 /*
  * Clean up an operation.
  */
@@ -705,10 +904,10 @@ static const struct netfs_cache_ops cachefiles_netfs_cache_ops = {
 	.read			= cachefiles_read,
 	.write			= cachefiles_write,
 	.issue_write		= cachefiles_issue_write,
-	.prepare_read		= cachefiles_prepare_read,
 	.prepare_write		= cachefiles_prepare_write,
 	.prepare_write_subreq	= cachefiles_prepare_write_subreq,
 	.query_occupancy	= cachefiles_query_occupancy,
+	.collect_write		= cachefiles_collect_write,
 };
 
 /*
@@ -718,13 +917,19 @@ bool cachefiles_begin_operation(struct netfs_cache_resources *cres,
 				enum fscache_want_state want_state)
 {
 	struct cachefiles_object *object = cachefiles_cres_object(cres);
+	struct file *file;
+
+	cres->dio_size = object->volume->cache->bsize;
 
 	if (!cachefiles_cres_file(cres)) {
 		cres->ops = &cachefiles_netfs_cache_ops;
 		if (object->file) {
 			spin_lock(&object->lock);
-			if (!cres->cache_priv2 && object->file)
-				cres->cache_priv2 = get_file(object->file);
+			file = object->file;
+			if (!cres->cache_priv2 && file) {
+				cres->cache_priv2 = get_file(file);
+				cres->cache_i_size = i_size_read(file_inode(file));
+			}
 			spin_unlock(&object->lock);
 		}
 	}
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index 88955249a1a6..a30df0f91e5e 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -414,7 +414,6 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 	struct dentry *fan = volume->fanout[(u8)object->cookie->key_hash];
 	struct file *file;
 	const struct path parentpath = { .mnt = cache->mnt, .dentry = fan };
-	uint64_t ni_size;
 	long ret;
 
 
@@ -442,23 +441,6 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 	if (!cachefiles_mark_inode_in_use(object, file_inode(file)))
 		WARN_ON(1);
 
-	ni_size = object->cookie->object_size;
-	ni_size = round_up(ni_size, CACHEFILES_DIO_BLOCK_SIZE);
-
-	if (ni_size > 0) {
-		trace_cachefiles_trunc(object, file_inode(file), 0, ni_size,
-				       cachefiles_trunc_expand_tmpfile);
-		ret = cachefiles_inject_write_error();
-		if (ret == 0)
-			ret = vfs_truncate(&file->f_path, ni_size);
-		if (ret < 0) {
-			trace_cachefiles_vfs_error(
-				object, file_inode(file), ret,
-				cachefiles_trace_trunc_error);
-			goto err_unuse;
-		}
-	}
-
 	ret = -EINVAL;
 	if (unlikely(!file->f_op->read_iter) ||
 	    unlikely(!file->f_op->write_iter)) {
@@ -467,6 +449,7 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 	}
 out:
 	cachefiles_end_secure(cache, saved_cred);
+	object->content_info = CACHEFILES_CONTENT_ALL;
 	return file;
 
 err_unuse:
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index f8ae78b3f7b6..25f2a906f984 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -54,7 +54,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
 	if (!buf)
 		return -ENOMEM;
 
-	buf->object_size	= cpu_to_be64(object->cookie->object_size);
+	buf->object_size	= cpu_to_be64(object->object_size);
 	buf->zero_point		= 0;
 	buf->type		= CACHEFILES_COOKIE_TYPE_DATA;
 	buf->content		= object->content_info;
@@ -77,6 +77,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
 		trace_cachefiles_vfs_error(object, file_inode(file), ret,
 					   cachefiles_trace_setxattr_error);
 		trace_cachefiles_coherency(object, file_inode(file)->i_ino,
+					   object->object_size,
 					   be64_to_cpup((__be64 *)buf->data),
 					   buf->content,
 					   cachefiles_coherency_set_fail);
@@ -86,6 +87,7 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
 				"Failed to set xattr with error %d", ret);
 	} else {
 		trace_cachefiles_coherency(object, file_inode(file)->i_ino,
+					   object->object_size,
 					   be64_to_cpup((__be64 *)buf->data),
 					   buf->content,
 					   cachefiles_coherency_set_ok);
@@ -103,9 +105,11 @@ int cachefiles_check_auxdata(struct cachefiles_object *object, struct file *file
 {
 	struct cachefiles_xattr *buf;
 	struct dentry *dentry = file->f_path.dentry;
+	struct inode *inode = file_inode(file);
 	unsigned int len = object->cookie->aux_len, tlen;
 	const void *p = fscache_get_aux(object->cookie);
 	enum cachefiles_coherency_trace why;
+	unsigned long long obj_size;
 	ssize_t xlen;
 	int ret = -ESTALE;
 
@@ -120,36 +124,41 @@ int cachefiles_check_auxdata(struct cachefiles_object *object, struct file *file
 	if (xlen != tlen) {
 		if (xlen < 0) {
 			ret = xlen;
-			trace_cachefiles_vfs_error(object, file_inode(file), xlen,
+			trace_cachefiles_vfs_error(object, inode, xlen,
 						   cachefiles_trace_getxattr_error);
 		}
 		if (xlen == -EIO)
 			cachefiles_io_error_obj(
 				object,
 				"Failed to read aux with error %zd", xlen);
-		why = cachefiles_coherency_check_xattr;
+		trace_cachefiles_coherency(object, inode->i_ino, 0, 0, 0,
+					   cachefiles_coherency_check_xattr);
 		goto out;
 	}
 
+	obj_size = be64_to_cpu(buf->object_size);
 	if (buf->type != CACHEFILES_COOKIE_TYPE_DATA) {
 		why = cachefiles_coherency_check_type;
 	} else if (memcmp(buf->data, p, len) != 0) {
 		why = cachefiles_coherency_check_aux;
-	} else if (be64_to_cpu(buf->object_size) != object->cookie->object_size) {
+	} else if (obj_size != object->cookie->object_size) {
 		why = cachefiles_coherency_check_objsize;
 	} else if (buf->content == CACHEFILES_CONTENT_DIRTY) {
 		// TODO: Begin conflict resolution
 		pr_warn("Dirty object in cache\n");
 		why = cachefiles_coherency_check_dirty;
 	} else {
+		object->content_info = buf->content;
+		object->object_size = obj_size;
+		atomic64_set(&object->read_limit, i_size_read(inode));
 		why = cachefiles_coherency_check_ok;
 		ret = 0;
 	}
 
-out:
-	trace_cachefiles_coherency(object, file_inode(file)->i_ino,
+	trace_cachefiles_coherency(object, inode->i_ino, obj_size,
 				   be64_to_cpup((__be64 *)buf->data),
 				   buf->content, why);
+out:
 	kfree(buf);
 	return ret;
 }
@@ -163,6 +172,9 @@ int cachefiles_remove_object_xattr(struct cachefiles_cache *cache,
 {
 	int ret;
 
+	trace_cachefiles_coherency(object, d_inode(dentry)->i_ino, 0, 0, 0,
+				   cachefiles_coherency_remove);
+
 	ret = cachefiles_inject_remove_error();
 	if (ret == 0) {
 		ret = mnt_want_write(cache->mnt);
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 55ef81173a44..67c2300fd16b 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -137,21 +137,6 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
 	return subreq->len;
 }
 
-static enum netfs_io_source netfs_cache_prepare_read(struct netfs_io_request *rreq,
-						     struct netfs_io_subrequest *subreq,
-						     loff_t i_size)
-{
-	struct netfs_cache_resources *cres = &rreq->cache_resources;
-	enum netfs_io_source source;
-
-	if (!cres->ops)
-		return NETFS_DOWNLOAD_FROM_SERVER;
-	source = cres->ops->prepare_read(subreq, i_size);
-	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-	return source;
-
-}
-
 /*
  * Issue a read against the cache.
  * - Eats the caller's ref on subreq.
@@ -166,6 +151,19 @@ static void netfs_read_cache_to_pagecache(struct netfs_io_request *rreq,
 			netfs_cache_read_terminated, subreq);
 }
 
+int netfs_read_query_cache(struct netfs_io_request *rreq, struct fscache_occupancy *occ)
+{
+	struct netfs_cache_resources *cres = &rreq->cache_resources;
+
+	occ->granularity = PAGE_SIZE;
+	if (occ->query_from >= occ->query_to)
+		return 0;
+	if (!cres->ops)
+		return 0;
+	occ->query_from = round_up(occ->query_from, occ->granularity);
+	return cres->ops->query_occupancy(cres, occ);
+}
+
 void netfs_queue_read(struct netfs_io_request *rreq,
 		      struct netfs_io_subrequest *subreq)
 {
@@ -218,15 +216,54 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
  */
 static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 {
+	struct fscache_occupancy _occ = {
+		.query_from	= rreq->start,
+		.query_to	= rreq->start + rreq->len,
+		.cached_from[0]	= 0,
+		.cached_to[0]	= 0,
+		.cached_from[1]	= ULLONG_MAX,
+		.cached_to[1]	= ULLONG_MAX,
+	};
+	struct fscache_occupancy *occ = &_occ;
 	unsigned long long start = rreq->start;
 	ssize_t size = rreq->len;
 	int ret = 0;
 
 	do {
+		int (*prepare_read)(struct netfs_io_subrequest *subreq) = NULL;
 		struct netfs_io_subrequest *subreq;
-		enum netfs_io_source source = NETFS_SOURCE_UNKNOWN;
+		unsigned long long hole_to, cache_to;
 		ssize_t slice;
 
+		/* If we don't have any, find out the next couple of data
+		 * extents from the cache, containing of following the
+		 * specified start offset.  Holes have to be fetched from the
+		 * server; data regions from the cache.
+		 */
+		hole_to = occ->cached_from[0];
+		cache_to = occ->cached_to[0];
+		if (start >= cache_to) {
+			/* Extent exhausted; shuffle down. */
+			int i;
+
+			for (i = 0; i < ARRAY_SIZE(occ->cached_from) - 1; i++) {
+				occ->cached_from[i] = occ->cached_from[i + 1];
+				occ->cached_to[i]   = occ->cached_to[i + 1];
+				occ->cached_type[i] = occ->cached_type[i + 1];
+			}
+			occ->cached_from[i] = ULLONG_MAX;
+			occ->cached_to[i]   = ULLONG_MAX;
+
+			if (occ->cached_from[0] != ULLONG_MAX)
+				continue;
+
+			/* Get new extents */
+			ret = netfs_read_query_cache(rreq, occ);
+			if (ret < 0)
+				break;
+			continue;
+		}
+
 		subreq = netfs_alloc_subrequest(rreq);
 		if (!subreq) {
 			ret = -ENOMEM;
@@ -238,63 +275,75 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 
 		netfs_queue_read(rreq, subreq);
 
-		source = netfs_cache_prepare_read(rreq, subreq, rreq->i_size);
-		subreq->source = source;
-		if (source == NETFS_DOWNLOAD_FROM_SERVER) {
-			unsigned long long zero_point = netfs_read_zero_point(rreq->inode);
-			unsigned long long zp = umin(zero_point, rreq->i_size);
-			size_t len = subreq->len;
-
-			if (unlikely(rreq->origin == NETFS_READ_SINGLE))
-				zp = rreq->i_size;
-			if (subreq->start >= zp) {
-				subreq->source = source = NETFS_FILL_WITH_ZEROES;
-				goto fill_with_zeroes;
+		unsigned long long zero_point = netfs_read_zero_point(rreq->inode);
+		unsigned long long zlimit = umin(zero_point, rreq->i_size);
+
+		_debug("rsub %llx %llx-%llx", subreq->start, hole_to, cache_to);
+
+		if (start >= hole_to && start < cache_to) {
+			/* Overlap with a cached region, where the cache may
+			 * record a block of zeroes.
+			 */
+			_debug("cached s=%llx c=%llx l=%zx", start, cache_to, size);
+			subreq->len = umin(cache_to - start, size);
+			subreq->len = round_up(subreq->len, occ->granularity);
+			if (occ->cached_type[0] == FSCACHE_EXTENT_ZERO) {
+				subreq->source = NETFS_FILL_WITH_ZEROES;
+				netfs_stat(&netfs_n_rh_zero);
+			} else {
+				subreq->source = NETFS_READ_FROM_CACHE;
+				prepare_read = rreq->cache_resources.ops->prepare_read;
 			}
 
-			if (len > zp - subreq->start)
-				len = zp - subreq->start;
-			if (len == 0) {
-				pr_err("ZERO-LEN READ: R=%08x[%x] l=%zx/%zx s=%llx z=%llx i=%llx",
-				       rreq->debug_id, subreq->debug_index,
-				       subreq->len, size,
-				       subreq->start, zero_point, rreq->i_size);
-				netfs_cancel_read(subreq, ret);
-				break;
-			}
-			subreq->len = len;
-
-			netfs_stat(&netfs_n_rh_download);
-			if (rreq->netfs_ops->prepare_read) {
-				ret = rreq->netfs_ops->prepare_read(subreq);
-				if (ret < 0) {
-					netfs_cancel_read(subreq, ret);
-					break;
-				}
-				trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-			}
-			goto issue;
-		}
+			trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
 
-	fill_with_zeroes:
-		if (source == NETFS_FILL_WITH_ZEROES) {
+		} else if (subreq->start >= zlimit && size > 0) {
+			/* If this range lies beyond the zero-point, that part
+			 * can just be cleared locally.
+			 */
+			_debug("zero %llx-%llx", start, start + size);
+			subreq->len = size;
 			subreq->source = NETFS_FILL_WITH_ZEROES;
-			trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+			if (rreq->cache_resources.ops)
+				__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
 			netfs_stat(&netfs_n_rh_zero);
-			goto issue;
+		} else {
+			/* Read a cache hole from the server.  If any part of
+			 * this range lies beyond the zero-point or the EOF,
+			 * that part can just be cleared locally.
+			 */
+			unsigned long long limit = min3(zlimit, start + size, hole_to);
+
+			_debug("limit %llx %llx", rreq->i_size, zero_point);
+			_debug("download %llx-%llx", start, start + size);
+			subreq->len = umin(limit - subreq->start, ULONG_MAX);
+			subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
+			if (rreq->cache_resources.ops)
+				__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
+			netfs_stat(&netfs_n_rh_download);
 		}
 
-		if (source == NETFS_READ_FROM_CACHE) {
-			trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
-			goto issue;
+		if (size == 0) {
+			pr_err("ZERO-LEN READ: R=%08x[%x] l=%zx/%zx s=%llx z=%llx i=%llx",
+			       rreq->debug_id, subreq->debug_index,
+			       subreq->len, size,
+			       subreq->start, zero_point, rreq->i_size);
+			netfs_cancel_read(subreq, ret);
+			break;
 		}
 
-		pr_err("Unexpected read source %u\n", source);
-		WARN_ON_ONCE(1);
-		netfs_cancel_read(subreq, ret);
-		break;
+		rreq->io_streams[0].sreq_max_len = MAX_RW_COUNT;
+		rreq->io_streams[0].sreq_max_segs = INT_MAX;
+
+		if (prepare_read) {
+			ret = prepare_read(subreq);
+			if (ret < 0) {
+				netfs_cancel_read(subreq, ret);
+				break;
+			}
+			trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
+		}
 
-	issue:
 		slice = netfs_prepare_read_iterator(subreq);
 		if (slice < 0) {
 			ret = slice;
@@ -308,6 +357,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 			set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
 		}
 
+		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 		netfs_issue_read(rreq, subreq);
 		netfs_maybe_bulk_drop_ra_refs(rreq);
 
diff --git a/fs/netfs/buffered_write.c b/fs/netfs/buffered_write.c
index 39f00b326e0f..8e84ec6d26c6 100644
--- a/fs/netfs/buffered_write.c
+++ b/fs/netfs/buffered_write.c
@@ -54,9 +54,6 @@ void netfs_update_i_size(struct netfs_inode *ctx, struct inode *inode,
 	i_size = i_size_read(inode);
 	if (end > i_size) {
 		i_size_write(inode, end);
-#if IS_ENABLED(CONFIG_FSCACHE)
-		fscache_update_cookie(ctx->cache, NULL, &end);
-#endif
 
 		gap = SECTOR_SIZE - (i_size & (SECTOR_SIZE - 1));
 		if (copied > gap) {
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 5aea8d34adcf..3c10f125f13b 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -23,6 +23,8 @@
 /*
  * buffered_read.c
  */
+int netfs_read_query_cache(struct netfs_io_request *rreq,
+			   struct fscache_occupancy *occ);
 void netfs_queue_read(struct netfs_io_request *rreq,
 		      struct netfs_io_subrequest *subreq);
 void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error);
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index e88293cdb834..b3185c27edf2 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -44,6 +44,7 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
 	rreq->gfp		= gfp;
 	rreq->start		= start;
 	rreq->collected_to	= start;
+	rreq->cache_coll_to	= start;
 	rreq->cleaned_to	= start;
 	rreq->len		= len;
 	rreq->progress_at	= ULONG_MAX;
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index dd463a485139..74869b1e876e 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -272,6 +272,7 @@ void netfs_retry_reads(struct netfs_io_request *rreq)
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
 
 	netfs_stat(&netfs_n_rh_retry_read_req);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_retry_begin);
 
 	/* Wait for all outstanding I/O to quiesce before performing retries as
 	 * we may need to renegotiate the I/O sizes.
@@ -282,6 +283,7 @@ void netfs_retry_reads(struct netfs_io_request *rreq)
 
 	trace_netfs_rreq(rreq, netfs_rreq_trace_resubmit);
 	netfs_retry_read_subrequests(rreq);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_retry_end);
 }
 
 /*
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index de67ac41548d..d5be31e55f38 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -58,20 +58,6 @@ static int netfs_single_begin_cache_read(struct netfs_io_request *rreq, struct n
 	return fscache_begin_read_operation(&rreq->cache_resources, netfs_i_cookie(ctx));
 }
 
-static void netfs_single_cache_prepare_read(struct netfs_io_request *rreq,
-					    struct netfs_io_subrequest *subreq)
-{
-	struct netfs_cache_resources *cres = &rreq->cache_resources;
-
-	if (!cres->ops) {
-		subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
-		return;
-	}
-	subreq->source = cres->ops->prepare_read(subreq, rreq->i_size);
-	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-
-}
-
 static void netfs_single_read_cache(struct netfs_io_request *rreq,
 				    struct netfs_io_subrequest *subreq)
 {
@@ -89,6 +75,14 @@ static void netfs_single_read_cache(struct netfs_io_request *rreq,
  */
 static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 {
+	struct fscache_occupancy occ = {
+		.query_from	= 0,
+		.query_to	= rreq->len,
+		.cached_from[0]	= ULLONG_MAX,
+		.cached_to[0]	= ULLONG_MAX,
+		.cached_from[1]	= ULLONG_MAX,
+		.cached_to[1]	= ULLONG_MAX,
+	};
 	struct netfs_io_subrequest *subreq;
 	int ret = 0;
 
@@ -96,14 +90,21 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 	if (!subreq)
 		return -ENOMEM;
 
-	subreq->source	= NETFS_SOURCE_UNKNOWN;
+	subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
 	subreq->start	= 0;
 	subreq->len	= rreq->len;
 	subreq->io_iter	= rreq->buffer.iter;
 
 	netfs_queue_read(rreq, subreq);
 
-	netfs_single_cache_prepare_read(rreq, subreq);
+	/* Try to use the cache if the cache content matches the size of the
+	 * remote file.
+	 */
+	netfs_read_query_cache(rreq, &occ);
+	if (occ.cached_from[0] == 0 &&
+	    occ.cached_to[0] >= rreq->len)
+		subreq->source = NETFS_READ_FROM_CACHE;
+
 	switch (subreq->source) {
 	case NETFS_DOWNLOAD_FROM_SERVER:
 		netfs_stat(&netfs_n_rh_download);
@@ -119,6 +120,12 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 		rreq->submitted += subreq->len;
 		break;
 	case NETFS_READ_FROM_CACHE:
+		if (rreq->cache_resources.ops->prepare_read) {
+			ret = rreq->cache_resources.ops->prepare_read(subreq);
+			if (ret < 0)
+				goto cancel;
+		}
+
 		smp_wmb(); /* Write lists before ALL_QUEUED. */
 		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
 		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 2eac588e37de..bf77db6bd403 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -188,6 +188,26 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 	wreq->buffer.first_tail_slot = slot;
 }
 
+/*
+ * Collect cache results.
+ */
+static void netfs_cache_collect(struct netfs_io_request *wreq,
+				struct netfs_io_stream *stream,
+				enum netfs_cache_collect block_type)
+{
+	struct netfs_cache_resources *cres = &wreq->cache_resources;
+
+	if (stream->source != NETFS_WRITE_TO_CACHE ||
+	    wreq->cache_coll_to >= stream->collected_to)
+		return;
+
+	if (cres->ops && cres->ops->collect_write)
+		cres->ops->collect_write(wreq, wreq->cache_coll_to,
+					 stream->collected_to - wreq->cache_coll_to,
+					 block_type);
+	wreq->cache_coll_to = stream->collected_to;
+}
+
 /*
  * Collect and assess the results of various write subrequests.  We may need to
  * retry some of the results - or even do an RMW cycle for content crypto.
@@ -235,13 +255,16 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 		/* Read first subreq pointer before IN_PROGRESS flag. */
 
 		while (front) {
+			bool cancelled;
 			trace_netfs_collect_sreq(wreq, front);
 			//_debug("sreq [%x] %llx %zx/%zx",
 			//       front->debug_index, front->start, front->transferred, front->len);
 
 			if (stream->collected_to < front->start) {
 				trace_netfs_collect_gap(wreq, stream, issued_to, 'F');
+				netfs_cache_collect(wreq, stream, NETFS_CACHE_COLLECT_WRITE_DATA);
 				stream->collected_to = front->start;
+				netfs_cache_collect(wreq, stream, NETFS_CACHE_COLLECT_WRITE_GAP);
 			}
 
 			/* Stall if the front is still undergoing I/O. */
@@ -249,7 +272,6 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 				notes |= HIT_PENDING;
 				break;
 			}
-			smp_rmb(); /* Read counters after I-P flag. */
 
 			if (stream->failed) {
 				stream->collected_to = front->start + front->len;
@@ -262,15 +284,44 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 				stream->transferred_valid = true;
 				notes |= MADE_PROGRESS;
 			}
-			if (test_bit(NETFS_SREQ_FAILED, &front->flags)) {
-				stream->failed = true;
-				stream->error = front->error;
-				if (stream->source == NETFS_UPLOAD_TO_SERVER)
-					mapping_set_error(wreq->mapping, front->error);
-				notes |= NEED_REASSESS | SAW_FAILURE;
+
+			/* Handle failed or cancelled subreqs.  Failure of
+			 * cache writes are handled differently to upload
+			 * failures.  Cache writes aren't fatal, provided we're
+			 * not doing disconnected operation, and so we can kind
+			 * of treat them as if they had succeeded - except that
+			 * we need to log any holes they cause.
+			 */
+			switch (stream->source) {
+			case NETFS_UPLOAD_TO_SERVER:
+				if (test_bit(NETFS_SREQ_FAILED, &front->flags)) {
+					if (!stream->failed) {
+						stream->failed = true;
+						stream->error = front->error;
+						mapping_set_error(wreq->mapping, front->error);
+						break;
+					}
+					notes |= NEED_REASSESS | SAW_FAILURE;
+				}
+				break;
+
+			case NETFS_WRITE_TO_CACHE:
+				cancelled = test_bit(NETFS_SREQ_CANCELLED, &front->flags);
+				if (cancelled != stream->cancelled) {
+					trace_netfs_rreq(wreq, netfs_rreq_trace_cache_fail_collect);
+					netfs_cache_collect(wreq, stream,
+							    stream->cancelled ?
+							    NETFS_CACHE_COLLECT_WRITE_CANCEL :
+							    NETFS_CACHE_COLLECT_WRITE_DATA);
+					stream->cancelled = !stream->cancelled;
+				}
+				break;
+
+			default:
+				WARN_ON(1);
 				break;
 			}
-			if (front->transferred < front->len) {
+			if (test_bit(NETFS_SREQ_NEED_RETRY, &front->flags)) {
 				stream->need_retry = true;
 				notes |= NEED_RETRY | MADE_PROGRESS;
 				break;
@@ -359,6 +410,7 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
  */
 bool netfs_write_collection(struct netfs_io_request *wreq)
 {
+	struct netfs_io_stream *cstream = &wreq->io_streams[1];
 	struct netfs_inode *ictx = netfs_inode(wreq->inode);
 	size_t transferred;
 	bool transferred_valid = false;
@@ -394,13 +446,22 @@ bool netfs_write_collection(struct netfs_io_request *wreq)
 		wreq->transferred = transferred;
 	trace_netfs_rreq(wreq, netfs_rreq_trace_write_done);
 
-	if (wreq->io_streams[1].active &&
-	    wreq->io_streams[1].failed &&
-	    ictx->ops->invalidate_cache) {
-		/* Cache write failure doesn't prevent writeback completion
-		 * unless we're in disconnected mode.
-		 */
-		ictx->ops->invalidate_cache(wreq);
+	if (cstream->active) {
+		if (test_bit(NETFS_RREQ_CACHE_ERROR, &wreq->flags)) {
+			if (ictx->ops->invalidate_cache) {
+				/* Cache write failure doesn't prevent
+				 * writeback completion unless we're in
+				 * disconnected mode.
+				 */
+				trace_netfs_rreq(wreq, netfs_rreq_trace_inval_cache);
+				ictx->ops->invalidate_cache(wreq);
+			}
+		} else if (!cstream->failed) {
+			netfs_cache_collect(wreq, cstream,
+					    cstream->cancelled ?
+					    NETFS_CACHE_COLLECT_WRITE_CANCEL :
+					    NETFS_CACHE_COLLECT_WRITE_DATA);
+		}
 	}
 
 	_debug("finished");
@@ -484,24 +545,51 @@ void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error)
 
 	if (IS_ERR_VALUE(transferred_or_error)) {
 		subreq->error = transferred_or_error;
-		/* if need retry is set, error should not matter */
-		if (!test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-			set_bit(NETFS_SREQ_FAILED, &subreq->flags);
-			trace_netfs_failure(wreq, subreq, transferred_or_error, netfs_fail_write);
-		}
 
 		switch (subreq->source) {
 		case NETFS_WRITE_TO_CACHE:
+			/* We don't mark a cache-write subreq as failed.
+			 * Instead we tell the issuer to produce dummy subreqs
+			 * instead and make a note if we need to invalidate the
+			 * cache at the end.  We also don't pause the loop that
+			 * grabs pages and launches upload subreqs.
+			 *
+			 * Note that we need to distinguish between -ENOBUFS
+			 * (no space available in the cache) and other errors.
+			 * In the former case, we can keep the data we have,
+			 * though we might have to change the way the on-disk
+			 * data is tracked.
+			 */
 			netfs_stat(&netfs_n_wh_write_failed);
+			if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
+				break;
+
+			trace_netfs_failure(wreq, subreq, transferred_or_error, netfs_fail_write);
+			__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
+			set_bit(NETFS_RREQ_CACHE_STOP, &wreq->flags);
+			if (transferred_or_error == -ENOBUFS)
+				trace_netfs_rreq(wreq, netfs_rreq_trace_cache_no_space);
+			else if (!test_and_set_bit(NETFS_RREQ_CACHE_ERROR, &wreq->flags))
+				trace_netfs_rreq(wreq, netfs_rreq_trace_cache_failed);
+			subreq->transferred = subreq->len;
 			break;
+
 		case NETFS_UPLOAD_TO_SERVER:
+			/* If need_retry is set, error should not matter */
+			if (!test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
+				set_bit(NETFS_SREQ_FAILED, &subreq->flags);
+				trace_netfs_failure(wreq, subreq, transferred_or_error,
+						    netfs_fail_upload);
+			}
+
+			set_bit(NETFS_RREQ_PAUSE, &wreq->flags);
+			trace_netfs_rreq(wreq, netfs_rreq_trace_set_pause);
 			netfs_stat(&netfs_n_wh_upload_failed);
 			break;
+
 		default:
 			break;
 		}
-		trace_netfs_rreq(wreq, netfs_rreq_trace_set_pause);
-		set_bit(NETFS_RREQ_PAUSE, &wreq->flags);
 	} else {
 		if (WARN(transferred_or_error > subreq->len - subreq->transferred,
 			 "Subreq excess write: R=%x[%x] %zd > %zu - %zu",
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index f531dda7856a..2004d6853702 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -111,6 +111,8 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 		goto nomem;
 
 	wreq->cleaned_to = wreq->start;
+	if (wreq->cache_resources.dio_size > 1)
+		wreq->cache_coll_to = round_down(wreq->start, wreq->cache_resources.dio_size);
 
 	wreq->io_streams[0].stream_nr		= 0;
 	wreq->io_streams[0].source		= NETFS_UPLOAD_TO_SERVER;
@@ -230,6 +232,21 @@ static void netfs_do_issue_write(struct netfs_io_stream *stream,
 
 	_enter("R=%x[%x],%zx", wreq->debug_id, subreq->debug_index, subreq->len);
 
+	if (stream->source == NETFS_WRITE_TO_CACHE &&
+	    unlikely(test_bit(NETFS_RREQ_CACHE_STOP, &wreq->flags))) {
+		size_t dio_size = wreq->cache_resources.dio_size;
+		size_t len, disp;
+
+		disp = subreq->start & (dio_size - 1);
+		len = round_up(subreq->len + disp, dio_size);
+
+		subreq->start -= disp;
+		subreq->len = len;
+
+		__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
+		return netfs_write_subrequest_terminated(subreq, subreq->len);
+	}
+
 	if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
 		return netfs_write_subrequest_terminated(subreq, subreq->error);
 
@@ -263,6 +280,7 @@ void netfs_issue_write(struct netfs_io_request *wreq,
 
 	if (!subreq)
 		return;
+
 	stream->construct = NULL;
 	subreq->io_iter.count = subreq->len;
 	netfs_do_issue_write(stream, subreq);
diff --git a/fs/netfs/write_retry.c b/fs/netfs/write_retry.c
index 058bc7a166a5..508dbcbb3b19 100644
--- a/fs/netfs/write_retry.c
+++ b/fs/netfs/write_retry.c
@@ -211,6 +211,7 @@ void netfs_retry_writes(struct netfs_io_request *wreq)
 	int s;
 
 	netfs_stat(&netfs_n_wh_retry_write_req);
+	trace_netfs_rreq(wreq, netfs_rreq_trace_retry_begin);
 
 	/* Wait for all outstanding I/O to quiesce before performing retries as
 	 * we may need to renegotiate the I/O sizes.
@@ -235,4 +236,6 @@ void netfs_retry_writes(struct netfs_io_request *wreq)
 			netfs_retry_write_stream(wreq, stream);
 		}
 	}
+
+	trace_netfs_rreq(wreq, netfs_rreq_trace_retry_end);
 }
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 58fdb9605425..850d20241075 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -147,6 +147,23 @@ struct fscache_cookie {
 	};
 };
 
+enum fscache_extent_type {
+	FSCACHE_EXTENT_DATA,
+	FSCACHE_EXTENT_ZERO,
+} __mode(byte);
+
+/*
+ * Cache occupancy information.
+ */
+struct fscache_occupancy {
+	unsigned long long	query_from;	/* Point to query from */
+	unsigned long long	query_to;	/* Point to query to */
+	unsigned long long	cached_from[2];	/* Point at which cache extents start */
+	unsigned long long	cached_to[2];	/* Point at which cache extents end */
+	unsigned int		granularity;	/* Granularity desired */
+	enum fscache_extent_type cached_type[2];	/* Type of cache extent */
+};
+
 /*
  * slow-path functions for when there is actually caching available, and the
  * netfs does actually have a valid token
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index dc8db260260a..8db5e5e1af3f 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -23,6 +23,7 @@
 
 enum netfs_sreq_ref_trace;
 typedef struct mempool mempool_t;
+struct fscache_occupancy;
 struct folio_queue;
 
 /**
@@ -153,6 +154,7 @@ struct netfs_io_stream {
 	bool			need_retry;	/* T if this stream needs retrying */
 	bool			failed;		/* T if this stream failed */
 	bool			transferred_valid; /* T is ->transferred is valid */
+	bool			cancelled;	/* T if stream is cancelled */
 };
 
 /*
@@ -162,8 +164,10 @@ struct netfs_cache_resources {
 	const struct netfs_cache_ops	*ops;
 	void				*cache_priv;
 	void				*cache_priv2;
+	unsigned long long		cache_i_size;	/* Initial size of cache file */
 	unsigned int			debug_id;	/* Cookie debug ID */
 	unsigned int			inval_counter;	/* object->inval_counter at begin_op */
+	unsigned int			dio_size;	/* DIO block size */
 };
 
 /*
@@ -197,6 +201,7 @@ struct netfs_io_subrequest {
 #define NETFS_SREQ_IN_PROGRESS		8	/* Unlocked when the subrequest completes */
 #define NETFS_SREQ_NEED_RETRY		9	/* Set if the filesystem requests a retry */
 #define NETFS_SREQ_FAILED		10	/* Set if the subreq failed unretryably */
+#define NETFS_SREQ_CANCELLED		11	/* Set if the subreq was cancelled by netfslib */
 };
 
 enum netfs_io_origin {
@@ -252,6 +257,7 @@ struct netfs_io_request {
 	unsigned long long	start;		/* Start position */
 	atomic64_t		issued_to;	/* Write issuer folio cursor */
 	unsigned long long	collected_to;	/* Point we've collected to */
+	unsigned long long	cache_coll_to;	/* Point the cache has collected to */
 	unsigned long long	cleaned_to;	/* Position we've cleaned folios to */
 	unsigned long long	abandon_to;	/* Position to abandon folios to */
 	const struct folio	*no_unlock_folio; /* Don't unlock this folio after read */
@@ -273,11 +279,13 @@ struct netfs_io_request {
 #define NETFS_RREQ_FAILED		3	/* The request failed */
 #define NETFS_RREQ_RETRYING		4	/* Set if we're in the retry path */
 #define NETFS_RREQ_SHORT_TRANSFER	5	/* Set if we have a short transfer */
-#define NETFS_RREQ_OFFLOAD_COLLECTION	8	/* Offload collection to workqueue */
-#define NETFS_RREQ_NO_UNLOCK_FOLIO	9	/* Don't unlock no_unlock_folio on completion */
-#define NETFS_RREQ_FOLIO_COPY_TO_CACHE	10	/* Copy current folio to cache from read */
-#define NETFS_RREQ_UPLOAD_TO_SERVER	11	/* Need to write to the server */
-#define NETFS_RREQ_USE_IO_ITER		12	/* Use ->io_iter rather than ->i_pages */
+#define NETFS_RREQ_CACHE_STOP		8	/* Set to stop caching (ENOBUFS or error) */
+#define NETFS_RREQ_CACHE_ERROR		9	/* Set if we got an error from the cache */
+#define NETFS_RREQ_OFFLOAD_COLLECTION	12	/* Offload collection to workqueue */
+#define NETFS_RREQ_NO_UNLOCK_FOLIO	13	/* Don't unlock no_unlock_folio on completion */
+#define NETFS_RREQ_FOLIO_COPY_TO_CACHE	14	/* Copy current folio to cache from read */
+#define NETFS_RREQ_UPLOAD_TO_SERVER	15	/* Need to write to the server */
+#define NETFS_RREQ_USE_IO_ITER		16	/* Use ->io_iter rather than ->i_pages */
 #define NETFS_RREQ_NEED_PUT_RA_REFS	17	/* Need to put the folio refs RA gave us */
 #define NETFS_RREQ_USE_PGPRIV2		31	/* [DEPRECATED] Use PG_private_2 to mark
 						 * write to cache on read */
@@ -323,6 +331,12 @@ enum netfs_read_from_hole {
 	NETFS_READ_HOLE_FAIL,
 };
 
+enum netfs_cache_collect {
+	NETFS_CACHE_COLLECT_WRITE_DATA,
+	NETFS_CACHE_COLLECT_WRITE_GAP,
+	NETFS_CACHE_COLLECT_WRITE_CANCEL,
+};
+
 /*
  * Table of operations for access to a cache.
  */
@@ -357,8 +371,7 @@ struct netfs_cache_ops {
 	/* Prepare a read operation, shortening it to a cached/uncached
 	 * boundary as appropriate.
 	 */
-	enum netfs_io_source (*prepare_read)(struct netfs_io_subrequest *subreq,
-					     unsigned long long i_size);
+	int (*prepare_read)(struct netfs_io_subrequest *subreq);
 
 	/* Prepare a write subrequest, working out if we're allowed to do it
 	 * and finding out the maximum amount of data to gather before
@@ -378,8 +391,17 @@ struct netfs_cache_ops {
 	 * next chunk of data starts and how long it is.
 	 */
 	int (*query_occupancy)(struct netfs_cache_resources *cres,
-			       loff_t start, size_t len, size_t granularity,
-			       loff_t *_data_start, size_t *_data_len);
+			       struct fscache_occupancy *occ);
+
+	/* Collect the result of buffered writeback to the cache.  This
+	 * includes copying a read to the cache.  block_type is one of:
+	 * - NETFS_CACHE_COLLECT_WRITE_DATA for a block of data
+	 * - NETFS_CACHE_COLLECT_WRITE_GAP if a discontiguity was skipped
+	 * - NETFS_CACHE_COLLECT_WRITE_CANCEL for a cancellation gap
+	 */
+	void (*collect_write)(struct netfs_io_request *wreq,
+			      unsigned long long start, size_t len,
+			      enum netfs_cache_collect block_type);
 };
 
 /* High-level read API. */
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index 9259bc71049e..eccdb5abeedc 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -52,6 +52,8 @@ enum cachefiles_coherency_trace {
 	cachefiles_coherency_check_ok,
 	cachefiles_coherency_check_type,
 	cachefiles_coherency_check_xattr,
+	cachefiles_coherency_discontiguous,
+	cachefiles_coherency_remove,
 	cachefiles_coherency_set_fail,
 	cachefiles_coherency_set_ok,
 	cachefiles_coherency_vol_check_cmp,
@@ -63,9 +65,11 @@ enum cachefiles_coherency_trace {
 };
 
 enum cachefiles_trunc_trace {
+	cachefiles_trunc_clear_padding,
 	cachefiles_trunc_dio_adjust,
 	cachefiles_trunc_expand_tmpfile,
 	cachefiles_trunc_shrink,
+	cachefiles_trunc_zap,
 };
 
 enum cachefiles_prepare_read_trace {
@@ -80,6 +84,7 @@ enum cachefiles_prepare_read_trace {
 };
 
 enum cachefiles_error_trace {
+	cachefiles_trace_alignment_error,
 	cachefiles_trace_fallocate_error,
 	cachefiles_trace_getxattr_error,
 	cachefiles_trace_link_error,
@@ -136,6 +141,8 @@ enum cachefiles_error_trace {
 	EM(cachefiles_coherency_check_ok,	"OK      ")		\
 	EM(cachefiles_coherency_check_type,	"BAD type")		\
 	EM(cachefiles_coherency_check_xattr,	"BAD xatt")		\
+	EM(cachefiles_coherency_discontiguous,	"--- gap ")		\
+	EM(cachefiles_coherency_remove,		"REMOVE  ")		\
 	EM(cachefiles_coherency_set_fail,	"SET fail")		\
 	EM(cachefiles_coherency_set_ok,		"SET ok  ")		\
 	EM(cachefiles_coherency_vol_check_cmp,	"VOL BAD cmp ")		\
@@ -146,9 +153,11 @@ enum cachefiles_error_trace {
 	E_(cachefiles_coherency_vol_set_ok,	"VOL SET ok  ")
 
 #define cachefiles_trunc_traces						\
+	EM(cachefiles_trunc_clear_padding,	"CLRPAD")		\
 	EM(cachefiles_trunc_dio_adjust,		"DIOADJ")		\
 	EM(cachefiles_trunc_expand_tmpfile,	"EXPTMP")		\
-	E_(cachefiles_trunc_shrink,		"SHRINK")
+	EM(cachefiles_trunc_shrink,		"SHRINK")		\
+	E_(cachefiles_trunc_zap,		"ZAP   ")
 
 #define cachefiles_prepare_read_traces					\
 	EM(cachefiles_trace_read_after_eof,	"after-eof ")		\
@@ -161,6 +170,7 @@ enum cachefiles_error_trace {
 	E_(cachefiles_trace_read_seek_nxio,	"seek-enxio")
 
 #define cachefiles_error_traces						\
+	EM(cachefiles_trace_alignment_error,	"align")		\
 	EM(cachefiles_trace_fallocate_error,	"fallocate")		\
 	EM(cachefiles_trace_getxattr_error,	"getxattr")		\
 	EM(cachefiles_trace_link_error,		"link")			\
@@ -371,12 +381,12 @@ TRACE_EVENT(cachefiles_rename,
 
 TRACE_EVENT(cachefiles_coherency,
 	    TP_PROTO(struct cachefiles_object *obj,
-		     ino_t ino,
+		     ino_t ino, unsigned long long obj_size,
 		     u64 disk_aux,
 		     enum cachefiles_content content,
 		     enum cachefiles_coherency_trace why),
 
-	    TP_ARGS(obj, ino, disk_aux, content, why),
+	    TP_ARGS(obj, ino, obj_size, disk_aux, content, why),
 
 	    /* Note that obj may be NULL */
 	    TP_STRUCT__entry(
@@ -384,6 +394,7 @@ TRACE_EVENT(cachefiles_coherency,
 		    __field(enum cachefiles_coherency_trace,	why)
 		    __field(enum cachefiles_content,		content)
 		    __field(u64,				ino)
+		    __field(u64,				obj_size)
 		    __field(u64,				aux)
 		    __field(u64,				disk_aux)
 			     ),
@@ -393,14 +404,16 @@ TRACE_EVENT(cachefiles_coherency,
 		    __entry->why	= why;
 		    __entry->content	= content;
 		    __entry->ino	= ino;
+		    __entry->obj_size	= obj_size;
 		    __entry->aux	= be64_to_cpup((__be64 *)obj->cookie->inline_aux);
 		    __entry->disk_aux	= disk_aux;
 			   ),
 
-	    TP_printk("o=%08x %s B=%llx c=%u aux=%llx dsk=%llx",
+	    TP_printk("o=%08x %s B=%llx oz=%llx c=%u aux=%llx dsk=%llx",
 		      __entry->obj,
 		      __print_symbolic(__entry->why, cachefiles_coherency_traces),
 		      __entry->ino,
+		      __entry->obj_size,
 		      __entry->content,
 		      __entry->aux,
 		      __entry->disk_aux)
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 2ce162baaad1..cc3806e383f5 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -48,6 +48,10 @@
 
 #define netfs_rreq_traces					\
 	EM(netfs_rreq_trace_assess,		"ASSESS ")	\
+	EM(netfs_rreq_trace_cache_cancelled,	"CA-CNCL")	\
+	EM(netfs_rreq_trace_cache_failed,	"CA-FAIL")	\
+	EM(netfs_rreq_trace_cache_fail_collect,	"CA-F-CO")	\
+	EM(netfs_rreq_trace_cache_no_space,	"CA-NOSP")	\
 	EM(netfs_rreq_trace_collect,		"COLLECT")	\
 	EM(netfs_rreq_trace_complete,		"COMPLET")	\
 	EM(netfs_rreq_trace_copy,		"COPY   ")	\
@@ -56,11 +60,14 @@
 	EM(netfs_rreq_trace_end_copy_to_cache,	"END-C2C")	\
 	EM(netfs_rreq_trace_free,		"FREE   ")	\
 	EM(netfs_rreq_trace_intr,		"INTR   ")	\
+	EM(netfs_rreq_trace_inval_cache,	"INVL-CA")	\
 	EM(netfs_rreq_trace_ki_complete,	"KI-CMPL")	\
 	EM(netfs_rreq_trace_ra_put_ref,		"RA-PUT ")	\
 	EM(netfs_rreq_trace_recollect,		"RECLLCT")	\
 	EM(netfs_rreq_trace_redirty,		"REDIRTY")	\
 	EM(netfs_rreq_trace_resubmit,		"RESUBMT")	\
+	EM(netfs_rreq_trace_retry_begin,	"RETRY-BEGIN")	\
+	EM(netfs_rreq_trace_retry_end,		"RETRY-END")	\
 	EM(netfs_rreq_trace_set_abandon,	"S-ABNDN")	\
 	EM(netfs_rreq_trace_set_pause,		"PAUSE  ")	\
 	EM(netfs_rreq_trace_unlock,		"UNLOCK ")	\
@@ -132,12 +139,12 @@
 
 #define netfs_failures							\
 	EM(netfs_fail_check_write_begin,	"check-write-begin")	\
-	EM(netfs_fail_copy_to_cache,		"copy-to-cache")	\
 	EM(netfs_fail_dio_read_short,		"dio-read-short")	\
 	EM(netfs_fail_dio_read_zero,		"dio-read-zero")	\
 	EM(netfs_fail_read,			"read")			\
 	EM(netfs_fail_short_read,		"short-read")		\
 	EM(netfs_fail_prepare_write,		"prep-write")		\
+	EM(netfs_fail_upload,			"upload")		\
 	E_(netfs_fail_write,			"write")
 
 #define netfs_rreq_ref_traces					\



From dhowells at redhat.com  Mon Aug 10 07:47:33 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:33 +0100
Subject: [PATCH v9 16/26] netfs: Add the cache object ID to netfs_read/write tracepoints
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-17-dhowells@redhat.com>

Add the cache object debug ID to netfs_read/write tracepoints to make
debugging easier as there's now a direct cross-reference with the
cachefiles tracepoints that only log that debug ID.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/cachefiles/io.c           |  1 +
 fs/netfs/fscache_io.c        |  2 +-
 include/linux/netfs.h        |  3 ++-
 include/trace/events/netfs.h | 27 +++++++++++++++------------
 4 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 0289c4935635..fb12110c84ed 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -923,6 +923,7 @@ bool cachefiles_begin_operation(struct netfs_cache_resources *cres,
 
 	if (!cachefiles_cres_file(cres)) {
 		cres->ops = &cachefiles_netfs_cache_ops;
+		cres->object_id = object->debug_id;
 		if (object->file) {
 			spin_lock(&object->lock);
 			file = object->file;
diff --git a/fs/netfs/fscache_io.c b/fs/netfs/fscache_io.c
index 37f05b4d3469..fafa8c6bec57 100644
--- a/fs/netfs/fscache_io.c
+++ b/fs/netfs/fscache_io.c
@@ -79,7 +79,7 @@ static int fscache_begin_operation(struct netfs_cache_resources *cres,
 	cres->ops		= NULL;
 	cres->cache_priv	= cookie;
 	cres->cache_priv2	= NULL;
-	cres->debug_id		= cookie->debug_id;
+	cres->cookie_id		= cookie->debug_id;
 	cres->inval_counter	= cookie->inval_counter;
 
 	if (!fscache_begin_cookie_access(cookie, why)) {
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 8db5e5e1af3f..453998053969 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -165,7 +165,8 @@ struct netfs_cache_resources {
 	void				*cache_priv;
 	void				*cache_priv2;
 	unsigned long long		cache_i_size;	/* Initial size of cache file */
-	unsigned int			debug_id;	/* Cookie debug ID */
+	unsigned int			cookie_id;	/* Cache cookie debug ID */
+	unsigned int			object_id;	/* Cache object debug ID */
 	unsigned int			inval_counter;	/* object->inval_counter at begin_op */
 	unsigned int			dio_size;	/* DIO block size */
 };
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index cc3806e383f5..5dc61a4ba5f0 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -310,6 +310,7 @@ TRACE_EVENT(netfs_read,
 	    TP_STRUCT__entry(
 		    __field(unsigned int,		rreq)
 		    __field(unsigned int,		cookie)
+		    __field(unsigned int,		object)
 		    __field(loff_t,			i_size)
 		    __field(loff_t,			start)
 		    __field(size_t,			len)
@@ -319,7 +320,8 @@ TRACE_EVENT(netfs_read,
 
 	    TP_fast_assign(
 		    __entry->rreq	= rreq->debug_id;
-		    __entry->cookie	= rreq->cache_resources.debug_id;
+		    __entry->cookie	= rreq->cache_resources.cookie_id;
+		    __entry->object	= rreq->cache_resources.object_id;
 		    __entry->i_size	= rreq->i_size;
 		    __entry->start	= start;
 		    __entry->len	= len;
@@ -327,10 +329,10 @@ TRACE_EVENT(netfs_read,
 		    __entry->netfs_inode = rreq->inode->i_ino;
 			   ),
 
-	    TP_printk("R=%08x %s c=%08x ni=%llx s=%llx l=%zx sz=%llx",
+	    TP_printk("R=%08x %s c=%08x o=%08x ni=%llx s=%llx l=%zx sz=%llx",
 		      __entry->rreq,
 		      __print_symbolic(__entry->what, netfs_read_traces),
-		      __entry->cookie,
+		      __entry->cookie, __entry->object,
 		      __entry->netfs_inode,
 		      __entry->start, __entry->len, __entry->i_size)
 	    );
@@ -551,6 +553,7 @@ TRACE_EVENT(netfs_write,
 	    TP_STRUCT__entry(
 		    __field(unsigned int,		wreq)
 		    __field(unsigned int,		cookie)
+		    __field(unsigned int,		object)
 		    __field(unsigned int,		ino)
 		    __field(enum netfs_write_trace,	what)
 		    __field(unsigned long long,		start)
@@ -558,20 +561,19 @@ TRACE_EVENT(netfs_write,
 			     ),
 
 	    TP_fast_assign(
-		    struct netfs_inode *__ctx = netfs_inode(wreq->inode);
-		    struct fscache_cookie *__cookie = netfs_i_cookie(__ctx);
 		    __entry->wreq	= wreq->debug_id;
-		    __entry->cookie	= __cookie ? __cookie->debug_id : 0;
+		    __entry->cookie	= wreq->cache_resources.cookie_id;
+		    __entry->object	= wreq->cache_resources.object_id;
 		    __entry->ino	= wreq->inode->i_ino;
 		    __entry->what	= what;
 		    __entry->start	= wreq->start;
 		    __entry->len	= wreq->len;
 			   ),
 
-	    TP_printk("R=%08x %s c=%08x i=%x by=%llx-%llx",
+	    TP_printk("R=%08x %s c=%08x o=%08x i=%x by=%llx-%llx",
 		      __entry->wreq,
 		      __print_symbolic(__entry->what, netfs_write_traces),
-		      __entry->cookie,
+		      __entry->cookie, __entry->object,
 		      __entry->ino,
 		      __entry->start, __entry->start + __entry->len - 1)
 	    );
@@ -586,22 +588,23 @@ TRACE_EVENT(netfs_copy2cache,
 		    __field(unsigned int,		rreq)
 		    __field(unsigned int,		creq)
 		    __field(unsigned int,		cookie)
+		    __field(unsigned int,		object)
 		    __field(unsigned int,		ino)
 			     ),
 
 	    TP_fast_assign(
-		    struct netfs_inode *__ctx = netfs_inode(rreq->inode);
-		    struct fscache_cookie *__cookie = netfs_i_cookie(__ctx);
 		    __entry->rreq	= rreq->debug_id;
 		    __entry->creq	= creq->debug_id;
-		    __entry->cookie	= __cookie ? __cookie->debug_id : 0;
+		    __entry->cookie	= rreq->cache_resources.cookie_id;
+		    __entry->object	= rreq->cache_resources.object_id;
 		    __entry->ino	= rreq->inode->i_ino;
 			   ),
 
-	    TP_printk("R=%08x CR=%08x c=%08x i=%x ",
+	    TP_printk("R=%08x CR=%08x c=%08x o=%08x i=%x ",
 		      __entry->rreq,
 		      __entry->creq,
 		      __entry->cookie,
+		      __entry->object,
 		      __entry->ino)
 	    );
 



From dhowells at redhat.com  Mon Aug 10 07:47:34 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:34 +0100
Subject: [PATCH v9 17/26] netfs: Switch to using bvecq rather than folio_queue and rolling_buffer
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-18-dhowells@redhat.com>

Switch netfslib to using bvecq, a segmented bio_vec[] queue, instead of the
folio_queue and rolling_buffer constructs, to keep track of the regions of
memory it is performing I/O upon.  Each bvecq struct in the chain is marked
with the starting file position of that sequence so that discontiguities
can be handled (the contents of each individual bvecq must be contiguous).

For unbuffered/direct I/O, the iterator is extracted into the queue up
front.  For buffered I/O, the folios are added to the queue as the
operation proceeds, much as it does now with folio_queues.  For buffered
reads and writes through the pagecache, bv_len is set to the length of the
entire folio and then slices of the relevant parts are passed to
subrequests.

The bvecq structs are marked with information as to how the regions
contained therein should be disposed of (unlock-only, free, unpin).

When setting up a subrequest, netfslib will furnish it with a slice of the
main buffer queue as a pointer to starting bvecq, slot and offset and, for
the moment, an ITER_BVECQ iterator is set to cover the slice in
subreq->io_iter.

Notes on the implementation:

 (1) This patch uses the concept of a 'bvecq position', which is a tuple of
     { bvecq, slot, offset }.  This is lighter weight than using a full
     iov_iter, though that would also suffice.  If not NULL, the position
     also holds a reference on the bvecq it is pointing to.  This is
     probably overkill as only the hindmost position (that of collection)
     needs to hold a reference.

 (2) There are three positions on the netfs_io_request struct.  Not all are
     used by every request type.

     Firstly, there's ->load_cursor, which is used by buffered read and
     write to point to the next slot to have a folio inserted into it
     (either loaded from the readahead_control or from writeback_iter()).

     Secondly, there's ->dispatch_cursor, which is used to provide the
     position in the buffer from which we start dispatching a subrequest.

     Thirdly, there's the ->collect_cursor, which is used by the collection
     routines to point to the next memory region to be cleaned up.

 (3) There are two positions on the netfs_io_subrequest struct.

     Firstly, there's ->dispatch_pos, which indicates the position from
     which a subrequest's buffer begins.  This is used as the base of the
     position from which to retry (advanced by ->transfer).

     Secondly, there's ->content, which is normally the same as
     ->dispatch_pos but if the bvecq chain got duplicated or the content
     got copied, then this will point to that and will that will be
     disposed of on retry.  The filesystem may modify ->content, but not
     ->dispatch_pos.

 (4) Maintenance of the position structs is done with helper functions,
     such as bvecq_pos_attach() to hide the refcounting.

 (5) When sending a write to the cache, the ends of the slice will be
     rounded up/down to the backing file's DIO block alignment.

 (6) bvec_slice() is used to select a slice of the source buffer and assign
     it to a subrequest.  The source buffer position is advanced.

 (7) netfs_extract_iter() is used by unbuffered/direct I/O API functions to
     decant a chunk of the iov_iter supplied by the VFS into a bvecq chain
     - and to label the bvecqs with appropriate disposal information
     (e.g. unpin, free, nothing).

There are further options that can be explored in the future:

 (1) Allow the provision of a duplicated bvecq chain for a specific slice
     so that the filesystem can add bits on either end (such as adding
     protocol headers and trailers and gluing several things together into
     a compound operation).

 (2) If a filesystem supports vectored/sparse read and write ops, it can be
     given a chain with discontiguities in it to perform in a single op
     (Ceph, for example, can do this).

 (3) Because each bvecq notes the start file position of the regions
     contained therein, there's no need to translate the info in the
     bio_vec into folio pointers in order to unlock the page after I/O.
     Instead, the inode's pagecache can be iterated over and the xarray
     marks cleared en masse.

 (4) Make MSG_SPLICE_PAGES handling read the disposal info in the bvecq and
     use that to indicate how it should get rid of the stuff it pasted into
     a sk_buff.

 (5) If a bounce buffer is needed (encryption, for example), the bounce
     buffer can be held in a bvecq and sliced up instead of the main buffer
     queue.

 (6) Get rid of subreq->io_iter and move the iov_iter stuff down into the
     filesystem.  The I/O iterators are normally only needed transitorily,
     and the one currently in netfs_io_subrequest is unnecessary most of
     the time.

folio_queue and rolling_buffer will be removed in a follow up patch.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: Shyam Prasad N <sprasad at microsoft.com>
cc: Tom Talpey <tom at talpey.com>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/cachefiles/io.c           |  12 ---
 fs/netfs/Makefile            |   1 -
 fs/netfs/buffered_read.c     | 124 ++++++++++++++++++------------
 fs/netfs/direct_read.c       |  78 +++++++------------
 fs/netfs/direct_write.c      |  69 +++++++++--------
 fs/netfs/internal.h          |  10 +--
 fs/netfs/iterator.c          |   2 +
 fs/netfs/main.c              |   6 --
 fs/netfs/misc.c              |  20 +----
 fs/netfs/objects.c           |  17 ++---
 fs/netfs/read_collect.c      |  99 +++++++++++++-----------
 fs/netfs/read_pgpriv2.c      |  94 +++++++++++++++--------
 fs/netfs/read_retry.c        | 102 +++++++++++++++----------
 fs/netfs/read_single.c       |  17 ++++-
 fs/netfs/stats.c             |   4 +-
 fs/netfs/write_collect.c     |  60 +++++++--------
 fs/netfs/write_issue.c       | 143 ++++++++++++++++++++++++-----------
 fs/netfs/write_retry.c       |  62 +++++++++------
 include/linux/netfs.h        |  25 +++---
 include/trace/events/netfs.h |  46 +++++------
 20 files changed, 546 insertions(+), 445 deletions(-)

diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index fb12110c84ed..d22a8a88b93d 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -636,7 +636,6 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 	struct netfs_cache_resources *cres = &wreq->cache_resources;
 	struct cachefiles_object *object = cachefiles_cres_object(cres);
 	struct cachefiles_cache *cache = object->volume->cache;
-	struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];
 	const struct cred *saved_cred;
 	size_t off, pre, post, len = subreq->len;
 	loff_t start = subreq->start;
@@ -661,17 +660,6 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 	}
 
 	/* We also need to end on the cache granularity boundary */
-	if (start + len == wreq->i_size) {
-		size_t part = len & (cache->bsize - 1);
-		size_t need = cache->bsize - part;
-
-		if (part && stream->submit_extendable_to >= need) {
-			len += need;
-			subreq->len += need;
-			subreq->io_iter.count += need;
-		}
-	}
-
 	post = len & (cache->bsize - 1);
 	if (post) {
 		len -= post;
diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index e1f12ecb5abf..0621e6870cbd 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -15,7 +15,6 @@ netfs-y := \
 	read_pgpriv2.o \
 	read_retry.o \
 	read_single.o \
-	rolling_buffer.o \
 	write_collect.o \
 	write_issue.o \
 	write_retry.o
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 67c2300fd16b..3da20ed981a4 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -114,26 +114,21 @@ static int netfs_begin_cache_read(struct netfs_io_request *rreq, struct netfs_in
 static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	ssize_t extracted;
 	size_t rsize = subreq->len;
 
 	if (subreq->source == NETFS_DOWNLOAD_FROM_SERVER)
-		rsize = umin(rsize, rreq->io_streams[0].sreq_max_len);
-
-	subreq->len = rsize;
-	if (unlikely(rreq->io_streams[0].sreq_max_segs)) {
-		size_t limit = netfs_limit_iter(&rreq->buffer.iter, 0, rsize,
-						rreq->io_streams[0].sreq_max_segs);
-
-		if (limit < rsize) {
-			subreq->len = limit;
-			trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
-		}
+		rsize = umin(rsize, stream->sreq_max_len);
+
+	bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
+	extracted = bvecq_slice(&rreq->dispatch_cursor, rsize,
+				stream->sreq_max_segs, &subreq->nr_segs);
+	if (extracted < rsize) {
+		subreq->len = extracted;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
 	}
 
-	subreq->io_iter	= rreq->buffer.iter;
-
-	iov_iter_truncate(&subreq->io_iter, subreq->len);
-	rolling_buffer_advance(&rreq->buffer, subreq->len);
 	return subreq->len;
 }
 
@@ -192,6 +187,10 @@ void netfs_queue_read(struct netfs_io_request *rreq,
 static void netfs_issue_read(struct netfs_io_request *rreq,
 			     struct netfs_io_subrequest *subreq)
 {
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
 	switch (subreq->source) {
 	case NETFS_DOWNLOAD_FROM_SERVER:
 		rreq->netfs_ops->issue_read(subreq);
@@ -200,7 +199,8 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
 		netfs_read_cache_to_pagecache(rreq, subreq);
 		break;
 	default:
-		__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
+		bvecq_zero(&subreq->content, subreq->len);
+		subreq->transferred = subreq->len;
 		subreq->error = 0;
 		iov_iter_zero(subreq->len, &subreq->io_iter);
 		subreq->transferred = subreq->len;
@@ -229,6 +229,11 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 	ssize_t size = rreq->len;
 	int ret = 0;
 
+	_enter("R=%08x", rreq->debug_id);
+
+	bvecq_pos_set(&rreq->dispatch_cursor, &rreq->load_cursor);
+	bvecq_pos_set(&rreq->collect_cursor, &rreq->dispatch_cursor);
+
 	do {
 		int (*prepare_read)(struct netfs_io_subrequest *subreq) = NULL;
 		struct netfs_io_subrequest *subreq;
@@ -376,6 +381,9 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 
 	/* Defer error return as we may need to wait for outstanding I/O. */
 	cmpxchg(&rreq->error, 0, ret);
+
+	bvecq_pos_unset(&rreq->load_cursor);
+	bvecq_pos_unset(&rreq->dispatch_cursor);
 }
 
 /**
@@ -423,7 +431,7 @@ void netfs_readahead(struct readahead_control *ractl)
 	 * acquires a ref on each folio that we will need to release later -
 	 * but we don't want to do that until after we've started the I/O.
 	 */
-	added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl, rreq->debug_id);
+	added = bvecq_load_from_ra(&rreq->load_cursor, ractl);
 	if (added < 0) {
 		ret = added;
 		goto cleanup_free;
@@ -432,7 +440,7 @@ void netfs_readahead(struct readahead_control *ractl)
 
 	rreq->submitted = rreq->start + added;
 	rreq->cleaned_to = rreq->start;
-	rreq->progress_at = folioq_folio_size(rreq->buffer.tail, 0);
+	rreq->progress_at = rreq->load_cursor.bvecq->bv[0].bv_len;
 
 	netfs_read_to_pagecache(rreq);
 	netfs_maybe_bulk_drop_ra_refs(rreq);
@@ -444,21 +452,21 @@ void netfs_readahead(struct readahead_control *ractl)
 EXPORT_SYMBOL(netfs_readahead);
 
 /*
- * Create a rolling buffer with a single occupying folio.
+ * Create a buffer queue with a single occupying folio.
  */
-static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio,
-					unsigned int rollbuf_flags)
+static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio)
 {
-	ssize_t added;
+	struct bvecq *bq;
+	size_t fsize = folio_size(folio);
 
-	if (rolling_buffer_init(&rreq->buffer, rreq->debug_id, ITER_DEST, rreq->gfp) < 0)
+	if (bvecq_buffer_init(&rreq->load_cursor, rreq->gfp, false) < 0)
 		return -ENOMEM;
 
-	added = rolling_buffer_append(&rreq->buffer, folio, rollbuf_flags, rreq->gfp);
-	if (added < 0)
-		return added;
-	rreq->submitted = rreq->start + added;
-	rreq->progress_at = added;
+	bq = rreq->load_cursor.bvecq;
+	bvec_set_folio(&bq->bv[0], folio, fsize, 0);
+	bvecq_filled_to(bq, 1);
+	rreq->submitted = rreq->start + fsize;
+	rreq->progress_at = fsize;
 	return 0;
 }
 
@@ -472,11 +480,11 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 	struct netfs_group *group = netfs_folio_group(folio);
 	struct netfs_folio *finfo = netfs_folio_info(folio);
 	struct netfs_inode *ctx = netfs_inode(mapping->host);
-	struct folio *sink = NULL;
-	struct bio_vec *bvec;
+	struct bvecq *bq = NULL;
+	struct page *sink = NULL;
 	unsigned int from = finfo->dirty_offset;
 	unsigned int to = from + finfo->dirty_len;
-	unsigned int off = 0, i = 0;
+	unsigned int off = 0, slot = 0;
 	size_t flen = folio_size(folio);
 	size_t nr_bvec = flen / PAGE_SIZE + 2;
 	size_t part;
@@ -501,32 +509,47 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 	 * end get copied to, but the middle is discarded.
 	 */
 	ret = -ENOMEM;
-	bvec = kmalloc_objs(*bvec, nr_bvec);
-	if (!bvec)
+	bq = bvecq_alloc_chain(nr_bvec, rreq->gfp, false);
+	if (!bq)
 		goto discard;
+	rreq->load_cursor.bvecq = bq;
 
-	sink = folio_alloc(GFP_KERNEL, 0);
-	if (!sink) {
-		kfree(bvec);
+	sink = alloc_page(rreq->gfp);
+	if (!sink)
 		goto discard;
-	}
 
 	trace_netfs_folio(folio, netfs_folio_trace_read_gaps);
 
-	rreq->direct_bv = bvec;
-	rreq->direct_bv_count = nr_bvec;
+	for (struct bvecq *p = bq; p; p = p->next)
+		p->mem_type = BVECQ_MEM_PAGECACHE;
+
 	if (from > 0) {
-		bvec_set_folio(&bvec[i++], folio, from, 0);
+		folio_get(folio);
+		bvec_set_folio(&bq->bv[slot++], folio, from, 0);
 		off = from;
 	}
 	while (off < to) {
-		part = min_t(size_t, to - off, PAGE_SIZE);
-		bvec_set_folio(&bvec[i++], sink, part, 0);
+		if (slot >= bq->max_slots) {
+			bvecq_filled_to(bq, slot);
+			bq = bq->next;
+			slot = 0;
+		}
+		part = min(to - off, PAGE_SIZE);
+		get_page(sink);
+		bvec_set_page(&bq->bv[slot++], sink, part, 0);
 		off += part;
 	}
-	if (to < flen)
-		bvec_set_folio(&bvec[i++], folio, flen - to, to);
-	iov_iter_bvec(&rreq->buffer.iter, ITER_DEST, bvec, i, rreq->len);
+	if (to < flen) {
+		if (slot >= bq->max_slots) {
+			bvecq_filled_to(bq, slot);
+			bq = bq->next;
+			slot = 0;
+		}
+		folio_get(folio);
+		bvec_set_folio(&bq->bv[slot++], folio, flen - to, to);
+	}
+	bvecq_filled_to(bq, slot);
+
 	rreq->submitted = rreq->start + flen;
 
 	netfs_read_to_pagecache(rreq);
@@ -543,13 +566,14 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 		folio_mark_uptodate(folio);
 	}
 
-	if (sink)
-		folio_put(sink);
+	put_page(sink);
 	folio_unlock(folio);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret < 0 ? ret : 0;
 
 discard:
+	if (sink)
+		put_page(sink);
 	netfs_put_failed_request(rreq);
 alloc_error:
 	folio_unlock(folio);
@@ -600,7 +624,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
 	trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, 0);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto discard;
 
@@ -757,7 +781,7 @@ int netfs_write_begin(struct netfs_inode *ctx,
 	trace_netfs_read(rreq, pos, len, netfs_read_trace_write_begin);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, 0);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto error_put;
 
@@ -822,7 +846,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 	trace_netfs_read(rreq, start, flen, netfs_read_trace_prefetch_for_write);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, NETFS_ROLLBUF_PAGECACHE_MARK);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto error_put;
 
diff --git a/fs/netfs/direct_read.c b/fs/netfs/direct_read.c
index 6a8fb0d55e04..4d35f8efc633 100644
--- a/fs/netfs/direct_read.c
+++ b/fs/netfs/direct_read.c
@@ -16,44 +16,22 @@
 #include <linux/netfs.h>
 #include "internal.h"
 
-static void netfs_prepare_dio_read_iterator(struct netfs_io_subrequest *subreq)
-{
-	struct netfs_io_request *rreq = subreq->rreq;
-	size_t rsize;
-
-	rsize = umin(subreq->len, rreq->io_streams[0].sreq_max_len);
-	subreq->len = rsize;
-
-	if (unlikely(rreq->io_streams[0].sreq_max_segs)) {
-		size_t limit = netfs_limit_iter(&rreq->buffer.iter, 0, rsize,
-						rreq->io_streams[0].sreq_max_segs);
-
-		if (limit < rsize) {
-			subreq->len = limit;
-			trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
-		}
-	}
-
-	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-
-	subreq->io_iter	= rreq->buffer.iter;
-	iov_iter_truncate(&subreq->io_iter, subreq->len);
-	iov_iter_advance(&rreq->buffer.iter, subreq->len);
-}
-
 /*
  * Perform a read to a buffer from the server, slicing up the region to be read
  * according to the network rsize.
  */
 static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 {
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
 	unsigned long long start = rreq->start;
 	ssize_t size = rreq->len;
 	int ret;
 
+	bvecq_pos_set(&rreq->dispatch_cursor, &rreq->load_cursor);
+	bvecq_pos_transfer(&rreq->collect_cursor, &rreq->load_cursor);
+
 	do {
 		struct netfs_io_subrequest *subreq;
-		ssize_t slice;
 
 		subreq = netfs_alloc_subrequest(rreq);
 		if (!subreq) {
@@ -79,16 +57,24 @@ static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 			}
 		}
 
-		netfs_prepare_dio_read_iterator(subreq);
-		slice = subreq->len;
-		size -= slice;
-		start += slice;
-		rreq->submitted += slice;
+		bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
+		bvecq_pos_set(&subreq->content, &rreq->dispatch_cursor);
+		subreq->len = bvecq_slice(&rreq->dispatch_cursor,
+					  umin(size, stream->sreq_max_len),
+					  stream->sreq_max_segs,
+					  &subreq->nr_segs);
+
+		size -= subreq->len;
+		start += subreq->len;
+		rreq->submitted += subreq->len;
 		if (size <= 0) {
 			smp_wmb(); /* Write lists before ALL_QUEUED. */
 			set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
 		}
 
+		iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
+				    subreq->content.slot, subreq->content.offset, subreq->len);
+
 		rreq->netfs_ops->issue_read(subreq);
 
 		if (test_bit(NETFS_RREQ_PAUSE, &rreq->flags))
@@ -103,6 +89,8 @@ static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
 		netfs_wake_collector(rreq);
 	}
+
+	bvecq_pos_unset(&rreq->dispatch_cursor);
 }
 
 /*
@@ -181,25 +169,17 @@ ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *i
 	 * buffer for ourselves as the caller's iterator will be trashed when
 	 * we return.
 	 *
-	 * In such a case, extract an iterator to represent as much of the the
-	 * output buffer as we can manage.  Note that the extraction might not
-	 * be able to allocate a sufficiently large bvec array and may shorten
-	 * the request.
+	 * Extract a buffer queue to represent as much of the output buffer as
+	 * we can manage.  The fragments are extracted into a bvecq which will
+	 * have sufficient nodes allocated to hold all the data, though this
+	 * may end up truncated if ENOMEM is encountered.
 	 */
-	if (user_backed_iter(iter)) {
-		ret = netfs_extract_user_iter(iter, rreq->len, &rreq->buffer.iter, 0);
-		if (ret < 0)
-			goto error_put;
-		rreq->direct_bv = (struct bio_vec *)rreq->buffer.iter.bvec;
-		rreq->direct_bv_count = ret;
-		rreq->direct_bv_unpin = iov_iter_extract_will_pin(iter);
-		rreq->len = iov_iter_count(&rreq->buffer.iter);
-	} else {
-		rreq->buffer.iter = *iter;
-		rreq->len = orig_count;
-		rreq->direct_bv_unpin = false;
-		iov_iter_advance(iter, orig_count);
-	}
+	ret = netfs_extract_iter(iter, rreq->len, INT_MAX, iocb->ki_pos,
+				 &rreq->load_cursor.bvecq, 0, rreq->gfp);
+	if (ret < 0)
+		goto error_put;
+
+	rreq->len = ret;
 
 	// TODO: Set up bounce buffer if needed
 
diff --git a/fs/netfs/direct_write.c b/fs/netfs/direct_write.c
index c16fbad286a1..d81245a29eff 100644
--- a/fs/netfs/direct_write.c
+++ b/fs/netfs/direct_write.c
@@ -73,7 +73,11 @@ static void netfs_unbuffered_write_collect(struct netfs_io_request *wreq,
 	spin_unlock(&wreq->lock);
 
 	wreq->transferred += subreq->transferred;
-	iov_iter_advance(&wreq->buffer.iter, subreq->transferred);
+	if (subreq->transferred < subreq->len) {
+		bvecq_pos_unset(&wreq->dispatch_cursor);
+		bvecq_pos_transfer(&wreq->dispatch_cursor, &subreq->dispatch_pos);
+		bvecq_pos_advance(&wreq->dispatch_cursor, subreq->transferred);
+	}
 
 	stream->collected_to = subreq->start + subreq->transferred;
 	wreq->collected_to = stream->collected_to;
@@ -99,6 +103,9 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 
 	_enter("%llx", wreq->len);
 
+	bvecq_pos_set(&wreq->dispatch_cursor, &wreq->load_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+
 	if (wreq->origin == NETFS_DIO_WRITE)
 		inode_dio_begin(wreq->inode);
 
@@ -111,6 +118,8 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 			netfs_prepare_write(wreq, stream, wreq->start + wreq->transferred);
 			subreq = stream->construct;
 			stream->construct = NULL;
+		} else {
+			bvecq_pos_set(&subreq->dispatch_pos, &wreq->dispatch_cursor);
 		}
 
 		/* Check if (re-)preparation failed. */
@@ -120,16 +129,18 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 			break;
 		}
 
-		iov_iter_truncate(&subreq->io_iter, wreq->len - wreq->transferred);
+		subreq->len = bvecq_slice(&wreq->dispatch_cursor, stream->sreq_max_len,
+					  stream->sreq_max_segs, &subreq->nr_segs);
+		bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+
+		iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
+				    subreq->content.bvecq, subreq->content.slot,
+				    subreq->content.offset,
+				    subreq->len);
+
 		if (!iov_iter_count(&subreq->io_iter))
 			break;
 
-		subreq->len = netfs_limit_iter(&subreq->io_iter, 0,
-					       stream->sreq_max_len,
-					       stream->sreq_max_segs);
-		iov_iter_truncate(&subreq->io_iter, subreq->len);
-		stream->submit_extendable_to = subreq->len;
-
 		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 		stream->issue_write(subreq);
 
@@ -166,9 +177,14 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 		 */
 		subreq->error = -EAGAIN;
 		trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+
+		bvecq_pos_unset(&subreq->content);
+		bvecq_pos_unset(&wreq->dispatch_cursor);
+		bvecq_pos_transfer(&wreq->dispatch_cursor, &subreq->dispatch_pos);
+
 		if (subreq->transferred > 0) {
-			iov_iter_advance(&wreq->buffer.iter, subreq->transferred);
 			wreq->transferred += subreq->transferred;
+			bvecq_pos_advance(&wreq->dispatch_cursor, subreq->transferred);
 		}
 
 		if (stream->source == NETFS_UPLOAD_TO_SERVER &&
@@ -179,7 +195,6 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 		__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 		__clear_bit(NETFS_SREQ_BOUNDARY, &subreq->flags);
 		__clear_bit(NETFS_SREQ_FAILED, &subreq->flags);
-		subreq->io_iter		= wreq->buffer.iter;
 		subreq->start		= wreq->start + wreq->transferred;
 		subreq->len		= wreq->len   - wreq->transferred;
 		subreq->transferred	= 0;
@@ -195,6 +210,8 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 		netfs_stat(&netfs_n_wh_retry_write_subreq);
 	}
 
+	bvecq_pos_unset(&wreq->dispatch_cursor);
+	bvecq_pos_unset(&wreq->load_cursor);
 	netfs_unbuffered_write_done(wreq);
 	_leave(" = %d", ret);
 	return ret;
@@ -213,12 +230,12 @@ static void netfs_unbuffered_write_async(struct work_struct *work)
  * encrypted file.  This can also be used for direct I/O writes.
  */
 ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *iter,
-						  struct netfs_group *netfs_group)
+					   struct netfs_group *netfs_group)
 {
 	struct netfs_io_request *wreq;
 	unsigned long long start = iocb->ki_pos;
 	unsigned long long end = start + iov_iter_count(iter);
-	ssize_t ret, n;
+	ssize_t ret;
 	size_t len = iov_iter_count(iter);
 	bool async = !is_sync_kiocb(iocb);
 
@@ -252,25 +269,17 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 		 * allocate a sufficiently large bvec array and may shorten the
 		 * request.
 		 */
-		if (user_backed_iter(iter)) {
-			n = netfs_extract_user_iter(iter, len, &wreq->buffer.iter, 0);
-			if (n < 0) {
-				ret = n;
-				goto error_put;
-			}
-			wreq->direct_bv = (struct bio_vec *)wreq->buffer.iter.bvec;
-			wreq->direct_bv_count = n;
-			wreq->direct_bv_unpin = iov_iter_extract_will_pin(iter);
-		} else {
-			/* If this is a kernel-generated async DIO request,
-			 * assume that any resources the iterator points to
-			 * (eg. a bio_vec array) will persist till the end of
-			 * the op.
-			 */
-			wreq->buffer.iter = *iter;
-		}
+		ssize_t n = netfs_extract_iter(iter, len, INT_MAX, iocb->ki_pos,
+					       &wreq->load_cursor.bvecq, 0, wreq->gfp);
 
-		wreq->len = iov_iter_count(&wreq->buffer.iter);
+		if (n < 0) {
+			ret = n;
+			goto error_put;
+		}
+		wreq->len = n;
+		_debug("dio-write %zx/%zx %u/%u",
+		       n, len, wreq->load_cursor.bvecq->nr_slots,
+		       wreq->load_cursor.bvecq->max_slots);
 	}
 
 	__set_bit(NETFS_RREQ_USE_IO_ITER, &wreq->flags);
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 3c10f125f13b..667d18911949 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -7,7 +7,6 @@
 
 #include <linux/slab.h>
 #include <linux/seq_file.h>
-#include <linux/folio_queue.h>
 #include <linux/netfs.h>
 #include <linux/fscache.h>
 #include <linux/fscache-cache.h>
@@ -71,9 +70,8 @@ static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}
 /*
  * misc.c
  */
-struct folio_queue *netfs_buffer_make_space(struct netfs_io_request *rreq,
-					    enum netfs_folioq_trace trace);
-void netfs_reset_iter(struct netfs_io_subrequest *subreq);
+struct bvecq *netfs_buffer_make_space(struct netfs_io_request *rreq,
+				      enum netfs_bvecq_trace trace);
 void netfs_wake_collector(struct netfs_io_request *rreq);
 void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq);
 void netfs_wait_for_in_progress_stream(struct netfs_io_request *rreq,
@@ -172,7 +170,6 @@ extern atomic_t netfs_n_wh_retry_write_req;
 extern atomic_t netfs_n_wh_retry_write_subreq;
 extern atomic_t netfs_n_wb_lock_skip;
 extern atomic_t netfs_n_wb_lock_wait;
-extern atomic_t netfs_n_folioq;
 extern atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v);
@@ -210,8 +207,7 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 			 struct netfs_io_stream *stream,
 			 loff_t start);
 void netfs_reissue_write(struct netfs_io_stream *stream,
-			 struct netfs_io_subrequest *subreq,
-			 struct iov_iter *source);
+			 struct netfs_io_subrequest *subreq);
 void netfs_issue_write(struct netfs_io_request *wreq,
 		       struct netfs_io_stream *stream);
 size_t netfs_advance_write(struct netfs_io_request *wreq,
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index b977a759a5d9..2a4b12264608 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -159,6 +159,7 @@ ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pag
 }
 EXPORT_SYMBOL_GPL(netfs_extract_iter);
 
+#if 0
 /**
  * netfs_extract_user_iter - Extract the pages from a user iterator into a bvec
  * @orig: The original iterator
@@ -452,3 +453,4 @@ size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
 	BUG();
 }
 EXPORT_SYMBOL(netfs_limit_iter);
+#endif
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 3908ea2422c8..041810707eb4 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -109,9 +109,6 @@ static int __init netfs_init(void)
 {
 	int ret = -ENOMEM;
 
-	if (mempool_init_kmalloc_pool(&netfs_folioq_pool, 100, sizeof(struct folio_queue)) < 0)
-		goto error_folioq_pool;
-
 	if (mempool_init_kmalloc_pool(&netfs_bvecq_pool, 100, BVECQ_STD_SIZE) < 0)
 		goto error_bvecq_pool;
 
@@ -169,8 +166,6 @@ static int __init netfs_init(void)
 error_req:
 	mempool_exit(&netfs_bvecq_pool);
 error_bvecq_pool:
-	mempool_exit(&netfs_folioq_pool);
-error_folioq_pool:
 	return ret;
 }
 fs_initcall(netfs_init);
@@ -184,6 +179,5 @@ static void __exit netfs_exit(void)
 	mempool_exit(&netfs_request_pool);
 	kmem_cache_destroy(netfs_request_slab);
 	mempool_exit(&netfs_bvecq_pool);
-	mempool_exit(&netfs_folioq_pool);
 }
 module_exit(netfs_exit);
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index f5c1c463f4ff..ee67a0681784 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -8,6 +8,7 @@
 #include <linux/swap.h>
 #include "internal.h"
 
+#if 0
 /**
  * netfs_alloc_folioq_buffer - Allocate buffer space into a folio queue
  * @mapping: Address space to set on the folio (or NULL).
@@ -103,24 +104,7 @@ void netfs_free_folioq_buffer(struct folio_queue *fq)
 	folio_batch_release(&fbatch);
 }
 EXPORT_SYMBOL(netfs_free_folioq_buffer);
-
-/*
- * Reset the subrequest iterator to refer just to the region remaining to be
- * read.  The iterator may or may not have been advanced by socket ops or
- * extraction ops to an extent that may or may not match the amount actually
- * read.
- */
-void netfs_reset_iter(struct netfs_io_subrequest *subreq)
-{
-	struct iov_iter *io_iter = &subreq->io_iter;
-	size_t remain = subreq->len - subreq->transferred;
-
-	if (io_iter->count > remain)
-		iov_iter_advance(io_iter, io_iter->count - remain);
-	else if (io_iter->count < remain)
-		iov_iter_revert(io_iter, remain - io_iter->count);
-	iov_iter_truncate(&subreq->io_iter, remain);
-}
+#endif
 
 /**
  * netfs_dirty_folio - Mark folio dirty and pin a cache object for writeback
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index b3185c27edf2..40ba1f058bc7 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -133,7 +133,6 @@ static void netfs_free_request_rcu(struct rcu_head *rcu)
 static void netfs_deinit_request(struct netfs_io_request *rreq)
 {
 	struct netfs_inode *ictx = netfs_inode(rreq->inode);
-	unsigned int i;
 
 	trace_netfs_rreq(rreq, netfs_rreq_trace_free);
 
@@ -148,16 +147,10 @@ static void netfs_deinit_request(struct netfs_io_request *rreq)
 		rreq->netfs_ops->free_request(rreq);
 	if (rreq->cache_resources.ops)
 		rreq->cache_resources.ops->end_operation(&rreq->cache_resources);
-	if (rreq->direct_bv) {
-		for (i = 0; i < rreq->direct_bv_count; i++) {
-			if (rreq->direct_bv[i].bv_page) {
-				if (rreq->direct_bv_unpin)
-					unpin_user_page(rreq->direct_bv[i].bv_page);
-			}
-		}
-		kvfree(rreq->direct_bv);
-	}
-	rolling_buffer_clear(&rreq->buffer);
+	bvecq_pos_unset(&rreq->load_cursor);
+	bvecq_pos_unset(&rreq->dispatch_cursor);
+	bvecq_pos_unset(&rreq->collect_cursor);
+	bvecq_put(rreq->spare);
 
 	if (atomic_dec_and_test(&ictx->io_count))
 		wake_up_var(&ictx->io_count);
@@ -249,6 +242,8 @@ static void netfs_free_subrequest(struct netfs_io_subrequest *subreq)
 	trace_netfs_sreq(subreq, netfs_sreq_trace_free);
 	if (rreq->netfs_ops->free_subrequest)
 		rreq->netfs_ops->free_subrequest(subreq);
+	bvecq_pos_unset(&subreq->dispatch_pos);
+	bvecq_pos_unset(&subreq->content);
 	mempool_free(subreq, rreq->netfs_ops->subrequest_pool ?: &netfs_subrequest_pool);
 	netfs_stat_d(&netfs_n_rh_sreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_subreq);
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 764579c76d3d..5d0a8a75ce26 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -27,9 +27,13 @@
  */
 static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
 {
-	netfs_reset_iter(subreq);
-	WARN_ON_ONCE(subreq->len - subreq->transferred != iov_iter_count(&subreq->io_iter));
-	iov_iter_zero(iov_iter_count(&subreq->io_iter), &subreq->io_iter);
+	struct iov_iter iter;
+
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+	iov_iter_advance(&iter, subreq->transferred);
+	iov_iter_zero(subreq->len, &iter);
+
 	if (subreq->start + subreq->transferred >= subreq->rreq->i_size)
 		__set_bit(NETFS_SREQ_HIT_EOF, &subreq->flags);
 }
@@ -40,11 +44,11 @@ static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
  * dirty and let writeback handle it.
  */
 static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
-				    struct folio_queue *folioq,
+				    struct bvecq *bvecq,
 				    int slot)
 {
 	struct netfs_folio *finfo;
-	struct folio *folio = folioq_folio(folioq, slot);
+	struct folio *folio = page_folio(bvecq->bv[slot].bv_page);
 
 	if (unlikely(folio_pos(folio) < rreq->abandon_to)) {
 		trace_netfs_folio(folio, netfs_folio_trace_abandon);
@@ -75,7 +79,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 			trace_netfs_folio(folio, netfs_folio_trace_read_done);
 		}
 
-		folioq_clear(folioq, slot);
+		bvecq->bv[slot].bv_page = NULL;
 	} else {
 		// TODO: Use of PG_private_2 is deprecated.
 		if (test_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags))
@@ -91,7 +95,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 		folio_unlock(folio);
 	}
 
-	folioq_clear(folioq, slot);
+	bvecq->bv[slot].bv_page = NULL;
 }
 
 /*
@@ -100,24 +104,15 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 				     unsigned int *notes)
 {
-	struct folio_queue *folioq = rreq->buffer.tail;
+	struct bvecq *bvecq = rreq->collect_cursor.bvecq;
 	unsigned long long collected_to = rreq->collected_to;
-	unsigned int slot = rreq->buffer.first_tail_slot;
+	unsigned int slot = rreq->collect_cursor.slot;
 
 	if (rreq->cleaned_to >= rreq->collected_to)
 		return;
 
 	// TODO: Begin decryption
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&rreq->buffer);
-		if (!folioq) {
-			WRITE_ONCE(rreq->progress_at, ULONG_MAX);
-			return;
-		}
-		slot = 0;
-	}
-
 	/* We have to wait for readahead refs to have been released before we
 	 * can unlock any folios as the ref-dropper walks i_pages and the only
 	 * thing preventing these folios from being removed is the folio lock.
@@ -130,17 +125,30 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 		unsigned long long fpos = rreq->cleaned_to, fend;
 		size_t fsize;
 
+		/* Clean up the head bvecq segment.  If we clear an entire
+		 * segment, then we can get rid of it provided it's not also
+		 * the tail segment being filled by the issuer.
+		 */
+		if (!bvecq_acquire_slot(bvecq, slot)) {
+			rreq->collect_cursor.slot = slot;
+			if (!bvecq_delete_spent(&rreq->collect_cursor)) {
+				WRITE_ONCE(rreq->progress_at, ULONG_MAX);
+				return;
+			}
+			bvecq = rreq->collect_cursor.bvecq;
+			slot  = rreq->collect_cursor.slot;
+		}
+
 		if (*notes & COPY_TO_CACHE)
 			set_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
 
-		folio = folioq_folio(folioq, slot);
+		folio = page_folio(bvecq->bv[slot].bv_page);
 		if (WARN_ONCE(!folio_test_locked(folio),
 			      "R=%08x: folio %lx is not locked\n",
 			      rreq->debug_id, folio->index))
 			trace_netfs_folio(folio, netfs_folio_trace_not_locked);
 
-		fsize = folioq_folio_size(folioq, slot);
-		fpos = folio_pos(folio);
+		fsize = bvecq->bv[slot].bv_len;
 		fend = fpos + fsize;
 
 		WRITE_ONCE(rreq->progress_at, fend - rreq->start);
@@ -151,33 +159,19 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 		if (collected_to < fend)
 			break;
 
-		netfs_unlock_read_folio(rreq, folioq, slot);
+		netfs_unlock_read_folio(rreq, bvecq, slot);
+		slot++;
 		WRITE_ONCE(rreq->cleaned_to, fend);
 		*notes |= MADE_PROGRESS;
 
 		clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
-		 * being filled by the issuer.
-		 */
-		folioq_clear(folioq, slot);
-		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&rreq->buffer);
-			if (!folioq)
-				goto done;
-			slot = 0;
-			trace_netfs_folioq(folioq, netfs_trace_folioq_read_progress);
-		}
-
 		if (fpos + fsize >= collected_to)
 			break;
 	}
 
-	rreq->buffer.tail = folioq;
-done:
-	rreq->buffer.first_tail_slot = slot;
+	bvecq_pos_move(&rreq->collect_cursor, bvecq);
+	rreq->collect_cursor.slot = slot;
 }
 
 /*
@@ -354,12 +348,17 @@ static void netfs_rreq_assess_dio(struct netfs_io_request *rreq)
 
 	if (rreq->origin == NETFS_UNBUFFERED_READ ||
 	    rreq->origin == NETFS_DIO_READ) {
-		for (i = 0; i < rreq->direct_bv_count; i++) {
-			flush_dcache_page(rreq->direct_bv[i].bv_page);
-			// TODO: cifs marks pages in the destination buffer
-			// dirty under some circumstances after a read.  Do we
-			// need to do that too?
-			set_page_dirty(rreq->direct_bv[i].bv_page);
+		for (struct bvecq *bq = rreq->collect_cursor.bvecq; bq; bq = bvecq_next(bq)) {
+			unsigned int nr_slots = bvecq_nr_slots_acquire(bq);
+			/* Read the slot count before the slots. */
+
+			for (i = 0; i < nr_slots; i++) {
+				flush_dcache_page(bq->bv[i].bv_page);
+				// TODO: cifs marks pages in the destination buffer
+				// dirty under some circumstances after a read.  Do we
+				// need to do that too?
+				set_page_dirty(bq->bv[i].bv_page);
+			}
 		}
 	}
 
@@ -450,7 +449,15 @@ bool netfs_read_collection(struct netfs_io_request *rreq)
 
 	trace_netfs_rreq(rreq, netfs_rreq_trace_done);
 	netfs_clear_subrequests(rreq);
-	netfs_unlock_abandoned_read_pages(rreq);
+	switch (rreq->origin) {
+	case NETFS_READAHEAD:
+	case NETFS_READPAGE:
+	case NETFS_READ_FOR_WRITE:
+		netfs_unlock_abandoned_read_pages(rreq);
+		break;
+	default:
+		break;
+	}
 	if (unlikely(rreq->copy_to_cache))
 		netfs_pgpriv2_end_copy_to_cache(rreq);
 	return true;
diff --git a/fs/netfs/read_pgpriv2.c b/fs/netfs/read_pgpriv2.c
index c31190993b76..eb7e2959358d 100644
--- a/fs/netfs/read_pgpriv2.c
+++ b/fs/netfs/read_pgpriv2.c
@@ -19,6 +19,9 @@
 static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio *folio)
 {
 	struct netfs_io_stream *cache = &creq->io_streams[1];
+	struct bvecq *queue;
+	unsigned int slot;
+	size_t dio_size = PAGE_SIZE;
 	size_t fsize = folio_size(folio), flen = fsize;
 	loff_t fpos = folio_pos(folio), i_size;
 	bool to_eof = false;
@@ -48,18 +51,40 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 		to_eof = true;
 	}
 
+	flen = round_up(flen, dio_size);
+
 	_debug("folio %zx %zx", flen, fsize);
 
 	trace_netfs_folio(folio, netfs_folio_trace_store_copy);
 
-	/* Attach the folio to the rolling buffer. */
-	if (rolling_buffer_append(&creq->buffer, folio, 0, creq->gfp) < 0) {
-		folio_end_private_2(folio);
-		clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &creq->flags);
-		return;
+	/* Institute a new bvec queue segment if the current one is full or if
+	 * we encounter a discontiguity.  The discontiguity break is important
+	 * when it comes to bulk unlocking folios by file range.
+	 */
+	queue = creq->load_cursor.bvecq;
+	if (bvecq_is_full(queue) ||
+	    (fpos != creq->last_end && creq->last_end > 0 && queue->nr_slots > 0)) {
+		bvecq_buffer_append(&creq->load_cursor, creq->spare);
+		creq->spare = NULL;
+
+		queue = creq->load_cursor.bvecq;
+		queue->fpos = fpos;
+		if (fpos != creq->last_end)
+			queue->discontig = true;
 	}
 
-	cache->submit_extendable_to = fsize;
+	/* Attach the folio to the rolling buffer. */
+	slot = queue->nr_slots;
+	bvec_set_folio(&queue->bv[slot], folio, fsize, 0);
+	trace_netfs_bv_slot(queue, slot);
+	slot++;
+	bvecq_filled_to(queue, slot);
+	creq->load_cursor.slot = slot;
+	creq->load_cursor.offset = 0;
+	creq->last_end = fpos + flen;
+
+	bvecq_pos_nudge(&creq->dispatch_cursor);
+	
 	cache->submit_off = 0;
 	cache->submit_len = flen;
 
@@ -71,10 +96,9 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	do {
 		ssize_t part;
 
-		creq->buffer.iter.iov_offset = cache->submit_off;
+		creq->dispatch_cursor.offset = cache->submit_off;
 
 		atomic64_set(&creq->issued_to, fpos + cache->submit_off);
-		cache->submit_extendable_to = fsize - cache->submit_off;
 		part = netfs_advance_write(creq, cache, fpos + cache->submit_off,
 					   cache->submit_len, to_eof);
 		cache->submit_off += part;
@@ -84,8 +108,7 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 			cache->submit_len -= part;
 	} while (cache->submit_len > 0);
 
-	creq->buffer.iter.iov_offset = 0;
-	rolling_buffer_advance(&creq->buffer, fsize);
+	bvecq_pos_step(&creq->dispatch_cursor);
 	atomic64_set(&creq->issued_to, fpos + fsize);
 
 	if (flen < fsize)
@@ -111,6 +134,11 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(
 	if (!creq->io_streams[1].avail)
 		goto cancel_put;
 
+	if (bvecq_buffer_init(&creq->load_cursor, creq->gfp, false) < 0)
+		goto cancel_put;
+	bvecq_pos_set(&creq->dispatch_cursor, &creq->load_cursor);
+	bvecq_pos_set(&creq->collect_cursor, &creq->dispatch_cursor);
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &creq->flags);
 	trace_netfs_copy2cache(rreq, creq);
 	trace_netfs_write(creq, netfs_write_trace_copy_to_cache);
@@ -139,6 +167,14 @@ void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *fo
 	if (IS_ERR(creq))
 		return;
 
+	if (!creq->spare) {
+		creq->spare = bvecq_alloc_one(BVECQ_STD_SLOTS, creq->gfp, false);
+		if (!creq->spare) {
+			clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &creq->flags);
+			return;
+		}
+	}
+
 	trace_netfs_folio(folio, netfs_folio_trace_copy_to_cache);
 	folio_start_private_2(folio);
 	netfs_pgpriv2_copy_folio(creq, folio);
@@ -171,22 +207,27 @@ void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
  */
 bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
 {
-	struct folio_queue *folioq = creq->buffer.tail;
+	struct bvecq *bq = creq->collect_cursor.bvecq;
 	unsigned long long collected_to = creq->collected_to;
-	unsigned int slot = creq->buffer.first_tail_slot;
+	unsigned int slot;
 	bool made_progress = false;
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&creq->buffer);
-		slot = 0;
-	}
+	slot = creq->collect_cursor.slot;
 
 	for (;;) {
 		struct folio *folio;
 		unsigned long long fpos, fend;
 		size_t fsize, flen;
 
-		folio = folioq_folio(folioq, slot);
+		if (!bvecq_acquire_slot(bq, slot)) {
+			creq->collect_cursor.slot = slot;
+			if (!bvecq_delete_spent(&creq->collect_cursor))
+				goto out;
+			bq   = creq->collect_cursor.bvecq;
+			slot = creq->collect_cursor.slot;
+		}
+
+		folio = page_folio(bq->bv[slot].bv_page);
 		if (WARN_ONCE(!folio_test_private_2(folio),
 			      "R=%08x: folio %lx is not marked private_2\n",
 			      creq->debug_id, folio->index))
@@ -209,25 +250,18 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
 		creq->cleaned_to = fpos + fsize;
 		made_progress = true;
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
-		 * being filled by the issuer.
+		/* Clean up the head segment.  If we clear an entire segment,
+		 * then we can get rid of it provided it's not also the tail
+		 * segment being filled by the issuer.
 		 */
-		folioq_clear(folioq, slot);
+		bq->bv[slot].bv_page = NULL;
 		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&creq->buffer);
-			if (!folioq)
-				goto done;
-			slot = 0;
-		}
 
 		if (fpos + fsize >= collected_to)
 			break;
 	}
 
-	creq->buffer.tail = folioq;
-done:
-	creq->buffer.first_tail_slot = slot;
+	creq->collect_cursor.slot = slot;
+out:
 	return made_progress;
 }
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index 74869b1e876e..f93a0da08c72 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -12,6 +12,12 @@
 static void netfs_reissue_read(struct netfs_io_request *rreq,
 			       struct netfs_io_subrequest *subreq)
 {
+	bvecq_pos_unset(&subreq->content);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+	iov_iter_advance(&subreq->io_iter, subreq->transferred);
+
 	subreq->error = 0;
 	__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
@@ -27,6 +33,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 {
 	struct netfs_io_subrequest *subreq;
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	struct bvecq_pos dispatch_cursor = {};
 	struct list_head *next;
 
 	_enter("R=%x", rreq->debug_id);
@@ -46,9 +53,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 			if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
 				break;
 			if (__test_and_clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-				__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 				subreq->retry_count++;
-				netfs_reset_iter(subreq);
 				netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
 				netfs_reissue_read(rreq, subreq);
 			}
@@ -74,11 +79,12 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 
 	do {
 		struct netfs_io_subrequest *from, *to, *tmp;
-		struct iov_iter source;
 		unsigned long long start, len;
 		size_t part;
 		bool boundary = false, subreq_superfluous = false;
 
+		bvecq_pos_unset(&dispatch_cursor);
+
 		/* Go through the subreqs and find the next span of contiguous
 		 * buffer that we then rejig (cifs, for example, needs the
 		 * rsize renegotiating) and reissue.
@@ -105,7 +111,8 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				break;
 
 			subreq = list_entry(next, struct netfs_io_subrequest, rreq_link);
-			if (subreq->start + subreq->transferred != start + len ||
+			if (subreq->start != start + len ||
+			    subreq->transferred > 0 ||
 			    test_bit(NETFS_SREQ_BOUNDARY, &subreq->flags) ||
 			    !test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
 				break;
@@ -118,11 +125,14 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		/* Determine the set of buffers we're going to use.  Each
 		 * subreq gets a subset of a single overall contiguous buffer.
 		 */
-		netfs_reset_iter(from);
-		source = from->io_iter;
-		source.count = len;
+		bvecq_pos_transfer(&dispatch_cursor, &from->dispatch_pos);
+		bvecq_pos_advance(&dispatch_cursor, from->transferred);
+		from->transferred = 0;
 
-		/* Work through the sublist. */
+		/* Work through the sublist.  The chain of buffers we're going
+		 * to fill is attached to dispatch_cursor and we need to read
+		 * 'len' amount of data from 'start'.
+		 */
 		subreq = from;
 		list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
 			if (!len) {
@@ -130,16 +140,22 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				break;
 			}
 			subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
-			subreq->start	= start - subreq->transferred;
-			subreq->len	= len   + subreq->transferred;
+			subreq->start	= start;
+			subreq->len	= len;
 			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 			__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 			subreq->retry_count++;
+			subreq->transferred = 0;
+
+			bvecq_pos_unset(&subreq->content);
+			bvecq_pos_unset(&subreq->dispatch_pos);
+			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
 
 			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
 			/* Renegotiate max_len (rsize) */
-			stream->sreq_max_len = subreq->len;
+			stream->sreq_max_len = len;
+			stream->sreq_max_segs = INT_MAX;
 			if (rreq->netfs_ops->prepare_read &&
 			    rreq->netfs_ops->prepare_read(subreq) < 0) {
 				trace_netfs_sreq(subreq, netfs_sreq_trace_reprep_failed);
@@ -147,13 +163,12 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				goto abandon;
 			}
 
-			part = umin(len, stream->sreq_max_len);
-			if (unlikely(stream->sreq_max_segs))
-				part = netfs_limit_iter(&source, 0, part, stream->sreq_max_segs);
-			subreq->len = subreq->transferred + part;
-			subreq->io_iter = source;
-			iov_iter_truncate(&subreq->io_iter, part);
-			iov_iter_advance(&source, part);
+			part = bvecq_slice(&dispatch_cursor,
+					   umin(len, stream->sreq_max_len),
+					   stream->sreq_max_segs,
+					   &subreq->nr_segs);
+			subreq->len = part;
+
 			len -= part;
 			start += part;
 			if (!len) {
@@ -217,9 +232,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
 			stream->sreq_max_len	= umin(len, rreq->rsize);
-			stream->sreq_max_segs	= 0;
-			if (unlikely(stream->sreq_max_segs))
-				part = netfs_limit_iter(&source, 0, part, stream->sreq_max_segs);
+			stream->sreq_max_segs	= INT_MAX;
 
 			netfs_stat(&netfs_n_rh_download);
 			if (rreq->netfs_ops->prepare_read(subreq) < 0) {
@@ -228,11 +241,12 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				goto abandon;
 			}
 
-			part = umin(len, stream->sreq_max_len);
-			subreq->len = subreq->transferred + part;
-			subreq->io_iter = source;
-			iov_iter_truncate(&subreq->io_iter, part);
-			iov_iter_advance(&source, part);
+			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
+			part = bvecq_slice(&dispatch_cursor,
+					   umin(len, stream->sreq_max_len),
+					   stream->sreq_max_segs,
+					   &subreq->nr_segs);
+			subreq->len = part;
 
 			len -= part;
 			start += part;
@@ -246,12 +260,14 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 
 	} while (!list_is_head(next, &stream->subrequests));
 
+out:
+	bvecq_pos_unset(&dispatch_cursor);
 	return;
 
 	/* If we hit an error, fail all remaining incomplete subrequests */
 abandon_after:
 	if (list_is_last(&subreq->rreq_link, &stream->subrequests))
-		return;
+		goto out;
 	subreq = list_next_entry(subreq, rreq_link);
 abandon:
 	list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
@@ -262,6 +278,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
 		__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 	}
+	goto out;
 }
 
 /*
@@ -292,7 +309,7 @@ void netfs_retry_reads(struct netfs_io_request *rreq)
  */
 void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
 {
-	struct folio_queue *p;
+	struct bvecq *p;
 
 	/* We have to wait for readahead refs to have been released before we
 	 * can unlock any folios as the ref-dropper walks i_pages and the only
@@ -301,21 +318,22 @@ void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
 	if (test_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags))
 		netfs_wait_for_put_ra_refs(rreq);
 
-	for (p = rreq->buffer.tail; p; p = p->next) {
-		for (int slot = 0; slot < folioq_count(p); slot++) {
-			struct folio *folio = folioq_folio(p, slot);
-
-			if (folio && !folioq_is_marked2(p, slot)) {
-				if (folio == rreq->no_unlock_folio &&
-				    test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO,
-					     &rreq->flags)) {
-					_debug("no unlock");
-				} else {
-					trace_netfs_folio(folio,
-						netfs_folio_trace_abandon);
-					folio_unlock(folio);
-				}
+	for (p = rreq->collect_cursor.bvecq; p; p = bvecq_next(p)) {
+		unsigned int nr_slots = bvecq_nr_slots_acquire(p);
+
+		for (int slot = 0; slot < nr_slots; slot++) {
+			if (!p->bv[slot].bv_page)
+				continue;
+
+			struct folio *folio = page_folio(p->bv[slot].bv_page);
+
+			if (folio == rreq->no_unlock_folio &&
+			    test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags)) {
+				_debug("no unlock");
+				continue;
 			}
+			trace_netfs_folio(folio, netfs_folio_trace_abandon);
+			folio_unlock(folio);
 		}
 	}
 }
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index d5be31e55f38..0a4542cd87e7 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -93,7 +93,12 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 	subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
 	subreq->start	= 0;
 	subreq->len	= rreq->len;
-	subreq->io_iter	= rreq->buffer.iter;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &rreq->dispatch_cursor);
+
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
 
 	netfs_queue_read(rreq, subreq);
 
@@ -177,6 +182,15 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	if (IS_ERR(rreq))
 		return PTR_ERR(rreq);
 
+	ret = netfs_extract_iter(iter, rreq->len, INT_MAX, 0, &rreq->dispatch_cursor.bvecq,
+				 0, rreq->gfp);
+	if (ret < 0)
+		goto cleanup_free;
+	if (ret < rreq->len) {
+		ret = -EIO;
+		goto cleanup_free;
+	}
+
 	rreq->progress_at = rreq->len;
 
 	ret = netfs_single_begin_cache_read(rreq, ictx);
@@ -186,7 +200,6 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	netfs_stat(&netfs_n_rh_read_single);
 	trace_netfs_read(rreq, 0, rreq->len, netfs_read_trace_read_single);
 
-	rreq->buffer.iter = *iter;
 	netfs_single_dispatch_read(rreq);
 
 	ret = netfs_wait_for_read(rreq);
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
index a10d34f88597..0ba6ce9295b2 100644
--- a/fs/netfs/stats.c
+++ b/fs/netfs/stats.c
@@ -46,7 +46,6 @@ atomic_t netfs_n_wh_retry_write_req;
 atomic_t netfs_n_wh_retry_write_subreq;
 atomic_t netfs_n_wb_lock_skip;
 atomic_t netfs_n_wb_lock_wait;
-atomic_t netfs_n_folioq;
 atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v)
@@ -89,11 +88,10 @@ int netfs_stats_show(struct seq_file *m, void *v)
 		   atomic_read(&netfs_n_rh_retry_read_subreq),
 		   atomic_read(&netfs_n_wh_retry_write_req),
 		   atomic_read(&netfs_n_wh_retry_write_subreq));
-	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u foq=%u wsc=%u\n",
+	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u wsc=%u\n",
 		   atomic_read(&netfs_n_rh_rreq),
 		   atomic_read(&netfs_n_rh_sreq),
 		   atomic_read(&netfs_n_bvecq),
-		   atomic_read(&netfs_n_folioq),
 		   atomic_read(&netfs_n_wh_wstream_conflict));
 	seq_printf(m, "WbLock : skip=%u wait=%u\n",
 		   atomic_read(&netfs_n_wb_lock_skip),
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index bf77db6bd403..35e29a01f57c 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -114,12 +114,12 @@ int netfs_folio_written_back(struct folio *folio)
 static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 					  unsigned int *notes)
 {
-	struct folio_queue *folioq = wreq->buffer.tail;
+	struct bvecq *bvecq = wreq->collect_cursor.bvecq;
 	unsigned long long collected_to = wreq->collected_to;
-	unsigned int slot = wreq->buffer.first_tail_slot;
+	unsigned int slot = wreq->collect_cursor.slot;
 
-	if (WARN_ON_ONCE(!folioq)) {
-		pr_err("[!] Writeback unlock found empty rolling buffer!\n");
+	if (WARN_ON_ONCE(!bvecq)) {
+		pr_err("[!] Writeback unlock found empty buffer!\n");
 		netfs_dump_request(wreq);
 		return;
 	}
@@ -130,20 +130,28 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		return;
 	}
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&wreq->buffer);
-		if (!folioq)
-			return;
-		slot = 0;
-	}
-
 	for (;;) {
 		struct folio *folio;
 		struct netfs_folio *finfo;
 		unsigned long long fpos, fend;
 		size_t fsize, flen;
 
-		folio = folioq_folio(folioq, slot);
+		/* Try to clean up the head of the queue if it appears to be
+		 * used up, but we need to be very careful - the cleanup can
+		 * catch the dispatcher, which could lead to us having nothing
+		 * left in the queue, causing the front and back pointers to
+		 * end up on different tracks.  To avoid this, we must always
+		 * keep at least one segment in the queue.
+		 */
+		if (!bvecq_acquire_slot(bvecq, slot)) {
+			wreq->collect_cursor.slot = slot;
+			if (!bvecq_delete_spent(&wreq->collect_cursor))
+				return;
+			bvecq = wreq->collect_cursor.bvecq;
+			slot  = wreq->collect_cursor.slot;
+		}
+
+		folio = page_folio(bvecq->bv[slot].bv_page);
 		if (WARN_ONCE(!folio_test_writeback(folio),
 			      "R=%08x: folio %lx is not under writeback\n",
 			      wreq->debug_id, folio->index))
@@ -166,26 +174,13 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		wreq->cleaned_to = fpos + fsize;
 		*notes |= MADE_PROGRESS;
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
-		 * being filled by the issuer.
-		 */
-		folioq_clear(folioq, slot);
+		bvecq->bv[slot].bv_page = NULL;
 		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&wreq->buffer);
-			if (!folioq)
-				goto done;
-			slot = 0;
-		}
-
 		if (fpos + fsize >= collected_to)
 			break;
 	}
 
-	wreq->buffer.tail = folioq;
-done:
-	wreq->buffer.first_tail_slot = slot;
+	wreq->collect_cursor.slot = slot;
 }
 
 /*
@@ -230,7 +225,8 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 	trace_netfs_rreq(wreq, netfs_rreq_trace_collect);
 
 reassess_streams:
-	issued_to = atomic64_read(&wreq->issued_to);
+	/* Order reading the issued_to point before reading the queue it refers to. */
+	issued_to = atomic64_read_acquire(&wreq->issued_to);
 	smp_rmb();
 	collected_to = ULLONG_MAX;
 	if (wreq->origin == NETFS_WRITEBACK ||
@@ -561,8 +557,12 @@ void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error)
 			 * data is tracked.
 			 */
 			netfs_stat(&netfs_n_wh_write_failed);
-			if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
-				break;
+			if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
+				/* We don't retry failed cache writes. */
+				__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+				if (!subreq->error)
+					subreq->error = -ENOBUFS;
+			}
 
 			trace_netfs_failure(wreq, subreq, transferred_or_error, netfs_fail_write);
 			__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 2004d6853702..69f24d53363b 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -107,8 +107,6 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 	ictx = netfs_inode(wreq->inode);
 	if (is_cacheable)
 		fscache_begin_write_operation(&wreq->cache_resources, netfs_i_cookie(ictx));
-	if (rolling_buffer_init(&wreq->buffer, wreq->debug_id, ITER_SOURCE, wreq->gfp) < 0)
-		goto nomem;
 
 	wreq->cleaned_to = wreq->start;
 	if (wreq->cache_resources.dio_size > 1)
@@ -133,9 +131,6 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 	}
 
 	return wreq;
-nomem:
-	netfs_put_failed_request(wreq);
-	return ERR_PTR(-ENOMEM);
 }
 
 /**
@@ -160,21 +155,13 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 			 loff_t start)
 {
 	struct netfs_io_subrequest *subreq;
-	struct iov_iter *wreq_iter = &wreq->buffer.iter;
-
-	/* Make sure we don't point the iterator at a used-up folio_queue
-	 * struct being used as a placeholder to prevent the queue from
-	 * collapsing.  In such a case, extend the queue.
-	 */
-	if (iov_iter_is_folioq(wreq_iter) &&
-	    wreq_iter->folioq_slot >= folioq_nr_slots(wreq_iter->folioq))
-		rolling_buffer_make_space(&wreq->buffer, wreq->gfp);
 
 	subreq = netfs_alloc_subrequest(wreq);
 	subreq->source		= stream->source;
 	subreq->start		= start;
 	subreq->stream_nr	= stream->stream_nr;
-	subreq->io_iter		= *wreq_iter;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &wreq->dispatch_cursor);
 
 	_enter("R=%x[%x]", wreq->debug_id, subreq->debug_index);
 
@@ -255,15 +242,16 @@ static void netfs_do_issue_write(struct netfs_io_stream *stream,
 }
 
 void netfs_reissue_write(struct netfs_io_stream *stream,
-			 struct netfs_io_subrequest *subreq,
-			 struct iov_iter *source)
+			 struct netfs_io_subrequest *subreq)
 {
-	size_t size = subreq->len - subreq->transferred;
-
 	// TODO: Use encrypted buffer
-	subreq->io_iter = *source;
-	iov_iter_advance(source, size);
-	iov_iter_truncate(&subreq->io_iter, size);
+	bvecq_pos_unset(&subreq->content);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
+			    subreq->content.bvecq, subreq->content.slot,
+			    subreq->content.offset,
+			    subreq->len);
+	iov_iter_advance(&subreq->io_iter, subreq->transferred);
 
 	subreq->retry_count++;
 	subreq->error = 0;
@@ -281,8 +269,13 @@ void netfs_issue_write(struct netfs_io_request *wreq,
 	if (!subreq)
 		return;
 
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
+			    subreq->content.bvecq, subreq->content.slot,
+			    subreq->content.offset,
+			    subreq->len);
+
 	stream->construct = NULL;
-	subreq->io_iter.count = subreq->len;
 	netfs_do_issue_write(stream, subreq);
 }
 
@@ -319,7 +312,6 @@ size_t netfs_advance_write(struct netfs_io_request *wreq,
 	_debug("part %zx/%zx %zx/%zx", subreq->len, stream->sreq_max_len, part, len);
 	subreq->len += part;
 	subreq->nr_segs++;
-	stream->submit_extendable_to -= part;
 
 	if (subreq->len >= stream->sreq_max_len ||
 	    subreq->nr_segs >= stream->sreq_max_segs ||
@@ -343,7 +335,8 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	struct netfs_io_stream *stream;
 	struct netfs_group *fgroup; /* TODO: Use this with ceph */
 	struct netfs_folio *finfo;
-	size_t iter_off = 0;
+	struct bvecq *queue = wreq->load_cursor.bvecq;
+	unsigned int slot;
 	size_t fsize = folio_size(folio), flen = fsize, foff = 0;
 	loff_t fpos = folio_pos(folio), i_size;
 	bool to_eof = false, streamw = false;
@@ -351,12 +344,20 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 
 	_enter("");
 
-	if (rolling_buffer_make_space(&wreq->buffer, wreq->gfp) < 0)
-		return -ENOMEM;
+	if (!wreq->spare) {
+		wreq->spare = bvecq_alloc_one(BVECQ_STD_SLOTS, wreq->gfp, true);
+		if (!wreq->spare)
+			return -ENOMEM;
+	}
 
-	/* netfs_perform_write() may shift i_size around the page or from out
-	 * of the page to beyond it, but cannot move i_size into or through the
-	 * page since we have it locked.
+	/* netfs_perform_write() may shift i_size around the folio or from out
+	 * of the folio to beyond it, but cannot move i_size into or through
+	 * the folio since we have it locked.
+	 *
+	 * Truncate could in theory move i_size into or before the folio, but
+	 * it should take steps to prevent writeback from happening
+	 * concurrently and should wait for any in-progress writebacks before
+	 * proceeding.
 	 */
 	i_size = i_size_read(wreq->inode);
 
@@ -448,8 +449,32 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 		trace_netfs_folio(folio, netfs_folio_trace_store_plus);
 	}
 
+	/* Institute a new bvec queue segment if the current one is full or if
+	 * we encounter a discontiguity.  The discontiguity break is important
+	 * when it comes to bulk unlocking folios by file range.
+	 */
+	if (bvecq_is_full(queue) ||
+	    (fpos != wreq->last_end && wreq->last_end > 0)) {
+		bvecq_buffer_append(&wreq->load_cursor, wreq->spare);
+		wreq->spare = NULL;
+
+		queue = wreq->load_cursor.bvecq;
+		queue->fpos = fpos;
+		if (fpos != wreq->last_end)
+			queue->discontig = true;
+		bvecq_pos_move(&wreq->dispatch_cursor, queue);
+		wreq->dispatch_cursor.slot = 0;
+	}
+
 	/* Attach the folio to the rolling buffer. */
-	rolling_buffer_append(&wreq->buffer, folio, 0, wreq->gfp);
+	slot = queue->nr_slots;
+	bvec_set_folio(&queue->bv[slot], folio, fsize, 0);
+	trace_netfs_bv_slot(queue, slot);
+	slot++;
+	bvecq_filled_to(queue, slot);
+	wreq->load_cursor.slot = slot;
+	wreq->load_cursor.offset = 0;
+	wreq->last_end = fpos + fsize;
 
 	/* Move the submission point forward to allow for write-streaming data
 	 * not starting at the front of the page.  We don't do write-streaming
@@ -458,10 +483,18 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	 * Also skip uploading for data that's been read and just needs copying
 	 * to the cache.
 	 */
+	bvecq_pos_nudge(&wreq->dispatch_cursor);
+	
 	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		size_t soff = foff, slen = flen, alignment = 1;
+
+		if (stream->source == NETFS_WRITE_TO_CACHE)
+			alignment = wreq->cache_resources.dio_size;
 		stream = &wreq->io_streams[s];
-		stream->submit_off = foff;
-		stream->submit_len = flen;
+		stream->submit_off = round_down(soff, alignment);
+		slen += foff - stream->submit_off;
+		stream->submit_len = round_up(slen, alignment);
+
 		if (!stream->avail ||
 		    (stream->source == NETFS_WRITE_TO_CACHE && streamw) ||
 		    (stream->source == NETFS_UPLOAD_TO_SERVER &&
@@ -495,14 +528,10 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 			break;
 		stream = &wreq->io_streams[choose_s];
 
-		/* Advance the iterator(s). */
-		if (stream->submit_off > iter_off) {
-			rolling_buffer_advance(&wreq->buffer, stream->submit_off - iter_off);
-			iter_off = stream->submit_off;
-		}
+		/* Advance the cursor. */
+		wreq->dispatch_cursor.offset = stream->submit_off;
 
 		atomic64_set(&wreq->issued_to, fpos + stream->submit_off);
-		stream->submit_extendable_to = fsize - stream->submit_off;
 		part = netfs_advance_write(wreq, stream, fpos + stream->submit_off,
 					   stream->submit_len, to_eof);
 		stream->submit_off += part;
@@ -514,9 +543,9 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 			debug = true;
 	}
 
-	if (fsize > iter_off)
-		rolling_buffer_advance(&wreq->buffer, fsize - iter_off);
-	atomic64_set(&wreq->issued_to, fpos + fsize);
+	bvecq_pos_step(&wreq->dispatch_cursor);
+	/* Order loading the queue before updating the issue_to point */
+	atomic64_set_release(&wreq->issued_to, fpos + fsize);
 
 	if (!debug)
 		kdebug("R=%x: No submit", wreq->debug_id);
@@ -578,6 +607,11 @@ int netfs_writepages(struct address_space *mapping,
 		goto couldnt_start;
 	}
 
+	if (bvecq_buffer_init(&wreq->load_cursor, wreq->gfp, true) < 0)
+		goto nomem;
+	bvecq_pos_set(&wreq->dispatch_cursor, &wreq->load_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback);
 	netfs_stat(&netfs_n_wh_writepages);
@@ -602,12 +636,17 @@ int netfs_writepages(struct address_space *mapping,
 	} while ((folio = writeback_iter(mapping, wbc, folio, &error)));
 
 	netfs_end_issue_write(wreq);
+	bvecq_pos_unset(&wreq->load_cursor);
+	bvecq_pos_unset(&wreq->dispatch_cursor);
 	netfs_wake_collector(wreq);
 
 	netfs_put_request(wreq, netfs_rreq_trace_put_return);
 	_leave(" = %d", error);
 	return error;
 
+nomem:
+	error = -ENOMEM;
+	netfs_put_failed_request(wreq);
 couldnt_start:
 	if (error == -ENOMEM) {
 		folio_redirty_for_writepage(wbc, folio);
@@ -631,10 +670,11 @@ EXPORT_SYMBOL(netfs_writepages);
  * @iter: Data to write.
  *
  * Write a monolithic, non-pagecache object back to the server and/or
- * the cache.
+ * the cache.  There's a maximum of one subrequest per stream.
  *
  * Return: 0 if successful; 1 if skipped due to lock conflict and WB_SYNC_NONE;
  * or a negative error code.
+ * the cache.  There's a maximum of one subrequest per stream.
  */
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
@@ -655,10 +695,19 @@ int netfs_writeback_single(struct address_space *mapping,
 		ret = PTR_ERR(wreq);
 		goto couldnt_start;
 	}
-
-	wreq->buffer.iter = *iter;
 	wreq->len = iov_iter_count(iter);
 
+	ret = netfs_extract_iter(iter, wreq->len, INT_MAX, 0, &wreq->dispatch_cursor.bvecq,
+				 0, wreq->gfp);
+	if (ret < 0)
+		goto cleanup_free;
+	if (ret < wreq->len) {
+		ret = -EIO;
+		goto cleanup_free;
+	}
+
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback_single);
 	netfs_stat(&netfs_n_wh_writepages);
@@ -678,11 +727,11 @@ int netfs_writeback_single(struct address_space *mapping,
 		subreq = stream->construct;
 		subreq->len = wreq->len;
 		stream->submit_len = subreq->len;
-		stream->submit_extendable_to = round_up(wreq->len, PAGE_SIZE);
 
 		netfs_issue_write(wreq, stream);
 	}
 
+	wreq->submitted = wreq->len;
 	smp_wmb(); /* Write lists before ALL_QUEUED. */
 	set_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags);
 
@@ -697,6 +746,8 @@ int netfs_writeback_single(struct address_space *mapping,
 	_leave(" = %d", ret);
 	return ret;
 
+cleanup_free:
+	netfs_put_failed_request(wreq);
 couldnt_start:
 	netfs_wb_end(ictx);
 	_leave(" = %d", ret);
diff --git a/fs/netfs/write_retry.c b/fs/netfs/write_retry.c
index 508dbcbb3b19..4e320591f83a 100644
--- a/fs/netfs/write_retry.c
+++ b/fs/netfs/write_retry.c
@@ -17,15 +17,17 @@
 static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				     struct netfs_io_stream *stream)
 {
+	struct bvecq_pos dispatch_cursor = {};
 	struct list_head *next;
 
 	_enter("R=%x[%x:]", wreq->debug_id, stream->stream_nr);
 
 	if (list_empty(&stream->subrequests))
 		return;
+	if (WARN_ON_ONCE(stream->source != NETFS_UPLOAD_TO_SERVER))
+		return; /* Shouldn't be retrying cache writes. */
 
-	if (stream->source == NETFS_UPLOAD_TO_SERVER &&
-	    wreq->netfs_ops->retry_request)
+	if (wreq->netfs_ops->retry_request)
 		wreq->netfs_ops->retry_request(wreq, stream);
 
 	if (unlikely(stream->failed))
@@ -39,12 +41,8 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
 				break;
 			if (__test_and_clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-				struct iov_iter source;
-
-				netfs_reset_iter(subreq);
-				source = subreq->io_iter;
 				netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-				netfs_reissue_write(stream, subreq, &source);
+				netfs_reissue_write(stream, subreq);
 			}
 		}
 		return;
@@ -54,11 +52,12 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 
 	do {
 		struct netfs_io_subrequest *subreq = NULL, *from, *to, *tmp;
-		struct iov_iter source;
 		unsigned long long start, len;
 		size_t part;
 		bool boundary = false;
 
+		bvecq_pos_unset(&dispatch_cursor);
+
 		/* Go through the stream and find the next span of contiguous
 		 * data that we then rejig (cifs, for example, needs the wsize
 		 * renegotiating) and reissue.
@@ -70,7 +69,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 
 		if (test_bit(NETFS_SREQ_FAILED, &from->flags) ||
 		    !test_bit(NETFS_SREQ_NEED_RETRY, &from->flags))
-			return;
+			goto out;
 
 		for (;;) {
 			/* Read pointer to subreq before reading subreq state. */
@@ -79,7 +78,8 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				break;
 
 			subreq = list_entry(next, struct netfs_io_subrequest, rreq_link);
-			if (subreq->start + subreq->transferred != start + len ||
+			if (subreq->start != start + len ||
+			    subreq->transferred > 0 ||
 			    test_bit(NETFS_SREQ_BOUNDARY, &subreq->flags) ||
 			    !test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
 				break;
@@ -90,11 +90,13 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 		/* Determine the set of buffers we're going to use.  Each
 		 * subreq gets a subset of a single overall contiguous buffer.
 		 */
-		netfs_reset_iter(from);
-		source = from->io_iter;
-		source.count = len;
+		bvecq_pos_transfer(&dispatch_cursor, &from->dispatch_pos);
+		bvecq_pos_advance(&dispatch_cursor, from->transferred);
 
-		/* Work through the sublist. */
+		/* Work through the sublist.  The chain of buffers we're going
+		 * to fill is attached to dispatch_cursor and we need to read
+		 * 'len' amount of data from 'start'.
+		 */
 		subreq = from;
 		list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
 			if (!len)
@@ -104,16 +106,23 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			subreq->len	= len;
 			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+			subreq->transferred = 0;
+
+			bvecq_pos_unset(&subreq->content);
+			bvecq_pos_unset(&subreq->dispatch_pos);
 
 			/* Renegotiate max_len (wsize) */
 			stream->sreq_max_len = len;
+			stream->sreq_max_segs = INT_MAX;
 			stream->prepare_write(subreq);
 
-			part = umin(len, stream->sreq_max_len);
-			if (unlikely(stream->sreq_max_segs))
-				part = netfs_limit_iter(&source, 0, part, stream->sreq_max_segs);
+			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
+			part = bvecq_slice(&dispatch_cursor,
+					   umin(len, stream->sreq_max_len),
+					   stream->sreq_max_segs,
+					   &subreq->nr_segs);
 			subreq->len = part;
-			subreq->transferred = 0;
+
 			len -= part;
 			start += part;
 			if (len && subreq == to &&
@@ -121,7 +130,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				boundary = true;
 
 			netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-			netfs_reissue_write(stream, subreq, &source);
+			netfs_reissue_write(stream, subreq);
 			if (subreq == to)
 				break;
 		}
@@ -173,17 +182,19 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				netfs_stat(&netfs_n_wh_upload);
 				stream->sreq_max_len = umin(len, wreq->wsize);
 				break;
-			case NETFS_WRITE_TO_CACHE:
-				netfs_stat(&netfs_n_wh_write);
-				break;
 			default:
 				WARN_ON_ONCE(1);
 			}
 
 			stream->prepare_write(subreq);
 
-			part = umin(len, stream->sreq_max_len);
+			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
+			part = bvecq_slice(&dispatch_cursor,
+					   umin(len, stream->sreq_max_len),
+					   stream->sreq_max_segs,
+					   &subreq->nr_segs);
 			subreq->len = subreq->transferred + part;
+
 			len -= part;
 			start += part;
 			if (!len && boundary) {
@@ -191,13 +202,16 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				boundary = false;
 			}
 
-			netfs_reissue_write(stream, subreq, &source);
+			netfs_reissue_write(stream, subreq);
 			if (!len)
 				break;
 
 		} while (len);
 
 	} while (!list_is_head(next, &stream->subrequests));
+
+out:
+	bvecq_pos_unset(&dispatch_cursor);
 }
 
 /*
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 453998053969..224a7cc7f4d4 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -19,12 +19,13 @@
 #include <linux/pagemap.h>
 #include <linux/bvecq.h>
 #include <linux/uio.h>
-#include <linux/rolling_buffer.h>
 
 enum netfs_sreq_ref_trace;
 typedef struct mempool mempool_t;
+struct readahead_control;
+struct netfs_io_request;
+struct netfs_io_subrequest;
 struct fscache_occupancy;
-struct folio_queue;
 
 /**
  * folio_start_private_2 - Start an fscache write on a folio.  [DEPRECATED]
@@ -139,7 +140,6 @@ struct netfs_io_stream {
 	unsigned int		sreq_max_segs;	/* 0 or max number of segments in an iterator */
 	unsigned int		submit_off;	/* Folio offset we're submitting from */
 	unsigned int		submit_len;	/* Amount of data left to submit */
-	unsigned int		submit_extendable_to; /* Amount I/O can be rounded up to */
 	void (*prepare_write)(struct netfs_io_subrequest *subreq);
 	void (*issue_write)(struct netfs_io_subrequest *subreq);
 	/* Collection tracking */
@@ -182,6 +182,8 @@ struct netfs_io_subrequest {
 	struct netfs_io_request *rreq;		/* Supervising I/O request */
 	struct work_struct	work;
 	struct list_head	rreq_link;	/* Link in rreq->subrequests */
+	struct bvecq_pos	dispatch_pos;	/* Bookmark in the combined queue of the start */
+	struct bvecq_pos	content;	/* The (copied) content of the subrequest */
 	struct iov_iter		io_iter;	/* Iterator for this subrequest */
 	unsigned long long	start;		/* Where to start the I/O */
 	size_t			len;		/* Size of the I/O */
@@ -242,13 +244,14 @@ struct netfs_io_request {
 	struct netfs_io_stream	io_streams[2];	/* Streams of parallel I/O operations */
 #define NR_IO_STREAMS 2 //wreq->nr_io_streams
 	struct netfs_group	*group;		/* Writeback group being written back */
-	struct rolling_buffer	buffer;		/* Unencrypted buffer */
-#define NETFS_ROLLBUF_PUT_MARK		ROLLBUF_MARK_1
-#define NETFS_ROLLBUF_PAGECACHE_MARK	ROLLBUF_MARK_2
+	struct bvecq		*spare;		/* Advance allocation of bvecq */
+	struct bvecq_pos	load_cursor;	/* Point at which new folios are loaded in */
+	struct bvecq_pos	dispatch_cursor; /* Point from which buffers are dispatched */
+	struct bvecq_pos	collect_cursor;	/* Clear-up point of I/O buffer */
 	wait_queue_head_t	waitq;		/* Processor waiter */
 	void			*netfs_priv;	/* Private data for the netfs */
 	void			*netfs_priv2;	/* Private data for the netfs */
-	struct bio_vec		*direct_bv;	/* DIO buffer list (when handling iovec-iter) */
+	unsigned long long	last_end;	/* End pos of last folio submitted */
 	unsigned long long	submitted;	/* Amount submitted for I/O so far */
 	unsigned long long	len;		/* Length of the request */
 	size_t			transferred;	/* Amount to be indicated as transferred */
@@ -263,7 +266,6 @@ struct netfs_io_request {
 	unsigned long long	abandon_to;	/* Position to abandon folios to */
 	const struct folio	*no_unlock_folio; /* Don't unlock this folio after read */
 	gfp_t			gfp;		/* GFP flags to use */
-	unsigned int		direct_bv_count; /* Number of elements in direct_bv[] */
 	unsigned int		debug_id;
 	unsigned int		rsize;		/* Maximum read size (0 for none) */
 	unsigned int		wsize;		/* Maximum write size (0 for none) */
@@ -271,7 +273,6 @@ struct netfs_io_request {
 	unsigned int		nr_group_rel;	/* Number of refs to release on ->group */
 	spinlock_t		lock;		/* Lock for queuing subreqs */
 	enum netfs_io_origin	origin;		/* Origin of the request */
-	bool			direct_bv_unpin; /* T if direct_bv[] must be unpinned */
 	refcount_t		ref;
 	unsigned long		flags;
 #define NETFS_RREQ_IN_PROGRESS		0	/* Unlocked when the request completes (has ref) */
@@ -471,12 +472,6 @@ void netfs_end_io_write(struct inode *inode);
 int netfs_start_io_direct(struct inode *inode);
 void netfs_end_io_direct(struct inode *inode);
 
-/* Miscellaneous APIs. */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
-				       unsigned int trace /*enum netfs_folioq_trace*/);
-void netfs_folioq_free(struct folio_queue *folioq,
-		       unsigned int trace /*enum netfs_trace_folioq*/);
-
 /* Buffer wrangling helpers API. */
 int netfs_alloc_folioq_buffer(struct address_space *mapping,
 			      struct folio_queue **_buffer,
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 5dc61a4ba5f0..071d20e80f13 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -226,7 +226,9 @@
 	EM(netfs_folio_trace_redirtied,		"redirtied")	\
 	EM(netfs_folio_trace_store,		"store")	\
 	EM(netfs_folio_trace_store_copy,	"store-copy")	\
-	E_(netfs_folio_trace_store_plus,	"store+")
+	EM(netfs_folio_trace_store_plus,	"store+")	\
+	EM(netfs_folio_trace_zero,		"zero")		\
+	E_(netfs_folio_trace_zero_ra,		"zero-ra")
 
 #define netfs_collect_contig_traces				\
 	EM(netfs_contig_trace_collect,		"Collect")	\
@@ -239,13 +241,13 @@
 	EM(netfs_trace_donate_to_next,		"to-next")	\
 	E_(netfs_trace_donate_to_deferred_next,	"defer-next")
 
-#define netfs_folioq_traces					\
-	EM(netfs_trace_folioq_alloc_buffer,	"alloc-buf")	\
-	EM(netfs_trace_folioq_clear,		"clear")	\
-	EM(netfs_trace_folioq_delete,		"delete")	\
-	EM(netfs_trace_folioq_make_space,	"make-space")	\
-	EM(netfs_trace_folioq_rollbuf_init,	"roll-init")	\
-	E_(netfs_trace_folioq_read_progress,	"r-progress")
+#define netfs_bvecq_traces					\
+	EM(netfs_trace_bvecq_alloc_buffer,	"alloc-buf")	\
+	EM(netfs_trace_bvecq_clear,		"clear")	\
+	EM(netfs_trace_bvecq_delete,		"delete")	\
+	EM(netfs_trace_bvecq_make_space,	"make-space")	\
+	EM(netfs_trace_bvecq_rollbuf_init,	"roll-init")	\
+	E_(netfs_trace_bvecq_read_progress,	"r-progress")
 
 #ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
 #define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
@@ -265,7 +267,7 @@ enum netfs_sreq_ref_trace { netfs_sreq_ref_traces } __mode(byte);
 enum netfs_folio_trace { netfs_folio_traces } __mode(byte);
 enum netfs_collect_contig_trace { netfs_collect_contig_traces } __mode(byte);
 enum netfs_donate_trace { netfs_donate_traces } __mode(byte);
-enum netfs_folioq_trace { netfs_folioq_traces } __mode(byte);
+enum netfs_bvecq_trace { netfs_bvecq_traces } __mode(byte);
 
 #endif
 
@@ -289,7 +291,7 @@ netfs_sreq_ref_traces;
 netfs_folio_traces;
 netfs_collect_contig_traces;
 netfs_donate_traces;
-netfs_folioq_traces;
+netfs_bvecq_traces;
 
 /*
  * Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -393,10 +395,10 @@ TRACE_EVENT(netfs_sreq,
 		    __entry->len	= sreq->len;
 		    __entry->transferred = sreq->transferred;
 		    __entry->start	= sreq->start;
-		    __entry->slot	= sreq->io_iter.folioq_slot;
+		    __entry->slot	= sreq->dispatch_pos.slot;
 			   ),
 
-	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx s=%u e=%d",
+	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx qs=%u e=%d",
 		      __entry->rreq, __entry->index,
 		      __print_symbolic(__entry->source, netfs_sreq_sources),
 		      __print_symbolic(__entry->what, netfs_sreq_traces),
@@ -772,27 +774,25 @@ TRACE_EVENT(netfs_collect_stream,
 		      __entry->collected_to, __entry->issued_to)
 	    );
 
-TRACE_EVENT(netfs_folioq,
-	    TP_PROTO(const struct folio_queue *fq,
-		     enum netfs_folioq_trace trace),
+TRACE_EVENT(netfs_bvecq,
+	    TP_PROTO(const struct bvecq *bq,
+		     enum netfs_bvecq_trace trace),
 
-	    TP_ARGS(fq, trace),
+	    TP_ARGS(bq, trace),
 
 	    TP_STRUCT__entry(
-		    __field(unsigned int,		rreq)
 		    __field(unsigned int,		id)
-		    __field(enum netfs_folioq_trace,	trace)
+		    __field(enum netfs_bvecq_trace,	trace)
 			     ),
 
 	    TP_fast_assign(
-		    __entry->rreq	= fq ? fq->rreq_id : 0;
-		    __entry->id		= fq ? fq->debug_id : 0;
+		    __entry->id		= bq ? bq->priv : 0;
 		    __entry->trace	= trace;
 			   ),
 
-	    TP_printk("R=%08x fq=%x %s",
-		      __entry->rreq, __entry->id,
-		      __print_symbolic(__entry->trace, netfs_folioq_traces))
+	    TP_printk("fq=%x %s",
+		      __entry->id,
+		      __print_symbolic(__entry->trace, netfs_bvecq_traces))
 	    );
 
 TRACE_EVENT(netfs_bv_slot,



From dhowells at redhat.com  Mon Aug 10 07:47:35 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:35 +0100
Subject: [PATCH v9 18/26] smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter()
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-19-dhowells@redhat.com>

netfslib now only presents an bvecq queue and an associated ITER_BVECQ
iterator to the filesystem, so it isn't going to see the ITER_FOLIOQ
iterator.  So remove that code.

Netfslib also won't supply ITER_BVEC/KVEC iterators, though smbdirect
might; further in future, it won't supply iterators at all, but rather a
bvecq slice (that can be used to construct an iterator).

Signed-off-by: David Howells <dhowells at redhat.com>
Acked-by: Stefan Metzmacher <metze at samba.org>
cc: Steve French <sfrench at samba.org>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Stefan Metzmacher <metze at samba.org>
cc: Shyam Prasad N <sprasad at microsoft.com>
cc: Tom Talpey <tom at talpey.com>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/smb/smbdirect/connection.c | 68 -----------------------------------
 1 file changed, 68 deletions(-)

diff --git a/fs/smb/smbdirect/connection.c b/fs/smb/smbdirect/connection.c
index f5e7e75dd809..f217cf127cf9 100644
--- a/fs/smb/smbdirect/connection.c
+++ b/fs/smb/smbdirect/connection.c
@@ -6,7 +6,6 @@
 
 #include "internal.h"
 #include <linux/bvecq.h>
-#include <linux/folio_queue.h>
 
 struct smbdirect_map_sges {
 	struct ib_sge *sge;
@@ -2131,70 +2130,6 @@ static ssize_t smbdirect_map_sges_from_kvec(struct iov_iter *iter,
 	return ret;
 }
 
-/*
- * Extract folio fragments from a FOLIOQ-class iterator and add them to an
- * ib_sge list.  The folios are not pinned.
- */
-static ssize_t smbdirect_map_sges_from_folioq(struct iov_iter *iter,
-					      struct smbdirect_map_sges *state,
-					      ssize_t maxsize)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	unsigned int slot = iter->folioq_slot;
-	ssize_t ret = 0;
-	size_t offset = iter->iov_offset;
-
-	if (WARN_ON_ONCE(!folioq))
-		return -EIO;
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		if (WARN_ON_ONCE(!folioq))
-			return -EIO;
-		slot = 0;
-	}
-
-	do {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t fsize = folioq_folio_size(folioq, slot);
-
-		if (offset < fsize) {
-			size_t part = umin(maxsize, fsize - offset);
-			bool ok;
-
-			ok = smbdirect_map_sges_single_page(state,
-							    folio_page(folio, 0),
-							    offset,
-							    part);
-			if (!ok)
-				return -EIO;
-
-			offset += part;
-			ret += part;
-			maxsize -= part;
-		}
-
-		if (offset >= fsize) {
-			offset = 0;
-			slot++;
-			if (slot >= folioq_nr_slots(folioq)) {
-				if (!folioq->next) {
-					WARN_ON_ONCE(ret < iter->count);
-					break;
-				}
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	} while (state->num_sge < state->max_sge && maxsize > 0);
-
-	iter->folioq = folioq;
-	iter->folioq_slot = slot;
-	iter->iov_offset = offset;
-	iter->count -= ret;
-	return ret;
-}
-
 /*
  * Extract page fragments from up to the given amount of the source iterator
  * and build up an ib_sge list that refers to all of those bits.  The ib_sge list
@@ -2225,9 +2160,6 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,
 	case ITER_KVEC:
 		ret = smbdirect_map_sges_from_kvec(iter, state, len);
 		break;
-	case ITER_FOLIOQ:
-		ret = smbdirect_map_sges_from_folioq(iter, state, len);
-		break;
 	default:
 		WARN_ONCE(1, "iov_iter_type[%u]\n", iov_iter_type(iter));
 		return -EIO;



From dhowells at redhat.com  Mon Aug 10 07:47:36 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:36 +0100
Subject: [PATCH v9 19/26] netfs: Remove netfs_alloc/free_folioq_buffer()
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-20-dhowells@redhat.com>

Remove netfs_alloc/free_folioq_buffer() as bvecq_alloc_*() and bvecq_put()
should now be used instead.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/afs/dir_edit.c       |  1 -
 fs/netfs/misc.c         | 98 -----------------------------------------
 fs/smb/client/smb2ops.c |  1 -
 include/linux/netfs.h   |  6 ---
 4 files changed, 106 deletions(-)

diff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c
index 01c6265e8865..31fdef23c6b6 100644
--- a/fs/afs/dir_edit.c
+++ b/fs/afs/dir_edit.c
@@ -10,7 +10,6 @@
 #include <linux/namei.h>
 #include <linux/pagemap.h>
 #include <linux/iversion.h>
-#include <linux/folio_queue.h>
 #include "internal.h"
 #include "xdr_fs.h"
 
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index ee67a0681784..8fc4e5ef2152 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -8,104 +8,6 @@
 #include <linux/swap.h>
 #include "internal.h"
 
-#if 0
-/**
- * netfs_alloc_folioq_buffer - Allocate buffer space into a folio queue
- * @mapping: Address space to set on the folio (or NULL).
- * @_buffer: Pointer to the folio queue to add to (may point to a NULL; updated).
- * @_cur_size: Current size of the buffer (updated).
- * @size: Target size of the buffer.
- * @gfp: The allocation constraints.
- */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
-			      struct folio_queue **_buffer,
-			      size_t *_cur_size, ssize_t size, gfp_t gfp)
-{
-	struct folio_queue *tail = *_buffer, *p;
-
-	size = round_up(size, PAGE_SIZE);
-	if (*_cur_size >= size)
-		return 0;
-
-	if (tail)
-		while (tail->next)
-			tail = tail->next;
-
-	do {
-		struct folio *folio;
-		int order = 0, slot;
-
-		if (!tail || folioq_full(tail)) {
-			p = netfs_folioq_alloc(0, GFP_NOFS, netfs_trace_folioq_alloc_buffer);
-			if (!p)
-				return -ENOMEM;
-			if (tail) {
-				tail->next = p;
-				p->prev = tail;
-			} else {
-				*_buffer = p;
-			}
-			tail = p;
-		}
-
-		if (size - *_cur_size > PAGE_SIZE)
-			order = umin(ilog2(size - *_cur_size) - PAGE_SHIFT,
-				     MAX_PAGECACHE_ORDER);
-
-		folio = folio_alloc(gfp, order);
-		if (!folio && order > 0)
-			folio = folio_alloc(gfp, 0);
-		if (!folio)
-			return -ENOMEM;
-
-		folio->mapping = mapping;
-		folio->index = *_cur_size / PAGE_SIZE;
-		trace_netfs_folio(folio, netfs_folio_trace_alloc_buffer);
-		slot = folioq_append_mark(tail, folio);
-		*_cur_size += folioq_folio_size(tail, slot);
-	} while (*_cur_size < size);
-
-	return 0;
-}
-EXPORT_SYMBOL(netfs_alloc_folioq_buffer);
-
-/**
- * netfs_free_folioq_buffer - Free a folio queue.
- * @fq: The start of the folio queue to free
- *
- * Free up a chain of folio_queues and, if marked, the marked folios they point
- * to.
- */
-void netfs_free_folioq_buffer(struct folio_queue *fq)
-{
-	struct folio_queue *next;
-	struct folio_batch fbatch;
-
-	folio_batch_init(&fbatch);
-
-	for (; fq; fq = next) {
-		for (int slot = 0; slot < folioq_count(fq); slot++) {
-			struct folio *folio = folioq_folio(fq, slot);
-
-			if (!folio ||
-			    !folioq_is_marked(fq, slot))
-				continue;
-
-			trace_netfs_folio(folio, netfs_folio_trace_put);
-			if (folio_batch_add(&fbatch, folio))
-				folio_batch_release(&fbatch);
-		}
-
-		netfs_stat_d(&netfs_n_folioq);
-		next = fq->next;
-		kfree(fq);
-	}
-
-	folio_batch_release(&fbatch);
-}
-EXPORT_SYMBOL(netfs_free_folioq_buffer);
-#endif
-
 /**
  * netfs_dirty_folio - Mark folio dirty and pin a cache object for writeback
  * @mapping: The mapping the folio belongs to.
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 885f0cc861ed..97823bcf198c 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -13,7 +13,6 @@
 #include <linux/sort.h>
 #include <crypto/aead.h>
 #include <linux/fiemap.h>
-#include <linux/folio_queue.h>
 #include <uapi/linux/magic.h>
 #include "cifsfs.h"
 #include "cifsglob.h"
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 224a7cc7f4d4..7b7cd7d7e143 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -472,12 +472,6 @@ void netfs_end_io_write(struct inode *inode);
 int netfs_start_io_direct(struct inode *inode);
 void netfs_end_io_direct(struct inode *inode);
 
-/* Buffer wrangling helpers API. */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
-			      struct folio_queue **_buffer,
-			      size_t *_cur_size, ssize_t size, gfp_t gfp);
-void netfs_free_folioq_buffer(struct folio_queue *fq);
-
 /* Writeback exclusion API. */
 bool netfs_wb_begin(struct netfs_inode *ictx, bool nowait);
 void netfs_wb_end(struct netfs_inode *ictx);



From dhowells at redhat.com  Mon Aug 10 07:47:37 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:37 +0100
Subject: [PATCH v9 20/26] netfs: Remove netfs_extract_user_iter()
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-21-dhowells@redhat.com>

Remove netfs_extract_user_iter() as it has been replaced with
netfs_extract_iter().

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/iterator.c   | 104 ------------------------------------------
 include/linux/netfs.h |   3 --
 2 files changed, 107 deletions(-)

diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index 2a4b12264608..499516ba6399 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -160,110 +160,6 @@ ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pag
 EXPORT_SYMBOL_GPL(netfs_extract_iter);
 
 #if 0
-/**
- * netfs_extract_user_iter - Extract the pages from a user iterator into a bvec
- * @orig: The original iterator
- * @orig_len: The amount of iterator to copy
- * @new: The iterator to be set up
- * @extraction_flags: Flags to qualify the request
- *
- * Extract the page fragments from the given amount of the source iterator and
- * build up a second iterator that refers to all of those bits.  This allows
- * the original iterator to be disposed of.
- *
- * @extraction_flags can have ITER_ALLOW_P2PDMA set to request peer-to-peer DMA be
- * allowed on the pages extracted.
- *
- * On success, the number of elements in the bvec is returned, the original
- * iterator will have been advanced by the amount extracted.
- *
- * The iov_iter_extract_mode() function should be used to query how cleanup
- * should be performed.
- */
-ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
-				struct iov_iter *new,
-				iov_iter_extraction_t extraction_flags)
-{
-	struct bio_vec *bv = NULL;
-	struct page **pages;
-	unsigned int cur_npages;
-	unsigned int max_pages;
-	unsigned int npages = 0;
-	unsigned int i;
-	ssize_t ret = 0;
-	size_t count = orig_len, offset, len;
-	size_t bv_size, pg_size;
-
-	if (WARN_ON_ONCE(!iter_is_ubuf(orig) && !iter_is_iovec(orig)))
-		return -EIO;
-
-	max_pages = iov_iter_npages(orig, INT_MAX);
-	bv_size = array_size(max_pages, sizeof(*bv));
-	bv = kvmalloc(bv_size, GFP_KERNEL);
-	if (!bv)
-		return -ENOMEM;
-
-	/* Put the page list at the end of the bvec list storage.  bvec
-	 * elements are larger than page pointers, so as long as we work
-	 * 0->last, we should be fine.
-	 */
-	pg_size = array_size(max_pages, sizeof(*pages));
-	pages = (void *)bv + bv_size - pg_size;
-
-	while (count && npages < max_pages) {
-		ret = iov_iter_extract_pages(orig, &pages, count,
-					     max_pages - npages, extraction_flags,
-					     &offset);
-		if (unlikely(ret <= 0)) {
-			ret = ret ?: -EIO;
-			break;
-		}
-
-		if (WARN(ret > count,
-			 "%s: extract_pages overrun %zd > %zu bytes\n",
-			 __func__, ret, count)) {
-			ret = -EIO;
-			break;
-		}
-
-		cur_npages = DIV_ROUND_UP(offset + ret, PAGE_SIZE);
-		if (WARN(cur_npages > max_pages - npages,
-			 "%s: extract_pages overrun %u > %u pages\n",
-			 __func__, npages + cur_npages, max_pages)) {
-			ret = -EIO;
-			break;
-		}
-
-		count -= ret;
-		ret += offset;
-
-		for (i = 0; i < cur_npages; i++) {
-			len = ret > PAGE_SIZE ? PAGE_SIZE : ret;
-			bvec_set_page(bv + npages + i, *pages++, len - offset, offset);
-			ret -= len;
-			offset = 0;
-		}
-
-		npages += cur_npages;
-	}
-
-	/* Note: Don't try to clean up after EIO.  Either we got no pages, so
-	 * nothing to clean up, or we got a buffer overrun, memory corruption
-	 * and can't trust the stuff in the buffer (a WARN was emitted).
-	 */
-
-	if (ret < 0 && (ret == -ENOMEM || npages == 0)) {
-		for (i = 0; i < npages; i++)
-			unpin_user_page(bv[i].bv_page);
-		kvfree(bv);
-		return ret;
-	}
-
-	iov_iter_bvec(new, orig->data_source, bv, npages, orig_len - count);
-	return npages;
-}
-EXPORT_SYMBOL_GPL(netfs_extract_user_iter);
-
 /*
  * Select the span of a bvec iterator we're going to use.  Limit it by both maximum
  * size and maximum number of segments.  Returns the size of the span in bytes.
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 7b7cd7d7e143..08b7b7e643a3 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -457,9 +457,6 @@ void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
 			   unsigned long long fpos, struct bvecq **_bvecq_head,
 			   iov_iter_extraction_t extraction_flags, gfp_t gfp);
-ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,
-				struct iov_iter *new,
-				iov_iter_extraction_t extraction_flags);
 size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
 			size_t max_size, size_t max_segs);
 void netfs_prepare_write_failed(struct netfs_io_subrequest *subreq);



From dhowells at redhat.com  Mon Aug 10 07:47:29 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:29 +0100
Subject: [PATCH v9 12/26] smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-13-dhowells@redhat.com>

Add support for ITER_BVECQ to smbdirect_map_sges_from_iter().

Signed-off-by: David Howells <dhowells at redhat.com>
Acked-by: Stefan Metzmacher <metze at samba.org>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: Shyam Prasad N <sprasad at microsoft.com>
cc: Tom Talpey <tom at talpey.com>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/smb/smbdirect/connection.c | 67 +++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/fs/smb/smbdirect/connection.c b/fs/smb/smbdirect/connection.c
index 8adf58097534..f5e7e75dd809 100644
--- a/fs/smb/smbdirect/connection.c
+++ b/fs/smb/smbdirect/connection.c
@@ -5,6 +5,7 @@
  */
 
 #include "internal.h"
+#include <linux/bvecq.h>
 #include <linux/folio_queue.h>
 
 struct smbdirect_map_sges {
@@ -2006,6 +2007,69 @@ static ssize_t smbdirect_map_sges_from_bvec(struct iov_iter *iter,
 	return ret;
 }
 
+/*
+ * Extract memory fragments from a BVECQ-class iterator and add them to an RDMA
+ * list.  The fragments are not pinned.
+ */
+static ssize_t smbdirect_map_sges_from_bvecq(struct iov_iter *iter,
+					     struct smbdirect_map_sges *state,
+					     ssize_t maxsize)
+{
+	const struct bvecq *bq = iter->bvecq, *next;
+	unsigned int slot = iter->bvecq_slot;
+	ssize_t extracted = 0;
+	size_t offset = iter->iov_offset;
+
+	maxsize = umin(maxsize, iov_iter_count(iter));
+
+	do {
+		struct bio_vec *bv;
+		size_t bsize;
+
+		while (slot >= bq->nr_slots) {
+			next = bvecq_next(bq);
+			if (!next) {
+				if (WARN_ON_ONCE(maxsize > 0))
+					return -EIO;
+				goto out;
+			}
+			bq = next;
+			slot = 0;
+		}
+
+		bv = &bq->bv[slot];
+		bsize = bv->bv_len;
+
+		if (offset < bsize) {
+			size_t part = umin(maxsize, bsize - offset);
+			bool ok;
+
+			ok = smbdirect_map_sges_single_page(state,
+							    bv->bv_page,
+							    bv->bv_offset + offset,
+							    part);
+			if (!ok)
+				return -EIO;
+
+			offset += part;
+			extracted += part;
+			maxsize -= part;
+		}
+
+		if (offset >= bsize) {
+			offset = 0;
+			slot++;
+		}
+	} while (state->num_sge < state->max_sge && maxsize > 0);
+
+out:
+	iter->bvecq = bq;
+	iter->bvecq_slot = slot;
+	iter->iov_offset = offset;
+	iter->count -= extracted;
+	return extracted;
+}
+
 /*
  * Extract fragments from a KVEC-class iterator and add them to an ib_sge list.
  * This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers.
@@ -2155,6 +2219,9 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,
 	case ITER_BVEC:
 		ret = smbdirect_map_sges_from_bvec(iter, state, len);
 		break;
+	case ITER_BVECQ:
+		ret = smbdirect_map_sges_from_bvecq(iter, state, len);
+		break;
 	case ITER_KVEC:
 		ret = smbdirect_map_sges_from_kvec(iter, state, len);
 		break;



From dhowells at redhat.com  Mon Aug 10 07:47:38 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:38 +0100
Subject: [PATCH v9 21/26] iov_iter: Remove ITER_FOLIOQ
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-22-dhowells@redhat.com>

Remove ITER_FOLIOQ as it's no longer used.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 include/linux/iov_iter.h   |  65 +--------
 include/linux/uio.h        |  12 --
 lib/iov_iter.c             | 175 +-----------------------
 lib/scatterlist.c          |  69 +---------
 lib/tests/kunit_iov_iter.c | 271 -------------------------------------
 5 files changed, 7 insertions(+), 585 deletions(-)

diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index 04b8a6d943fa..4e7d69b4ace2 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -11,7 +11,6 @@
 #include <linux/uio.h>
 #include <linux/bvec.h>
 #include <linux/bvecq.h>
-#include <linux/folio_queue.h>
 
 typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
 			     void *priv, void *priv2);
@@ -207,62 +206,6 @@ size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
 	return progress;
 }
 
-/*
- * Handle ITER_FOLIOQ.
- */
-static __always_inline
-size_t iterate_folioq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
-		      iov_step_f step)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	unsigned int slot = iter->folioq_slot;
-	size_t progress = 0, skip = iter->iov_offset;
-
-	if (slot == folioq_nr_slots(folioq)) {
-		/* The iterator may have been extended. */
-		folioq = folioq->next;
-		slot = 0;
-	}
-
-	do {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t part, remain = 0, consumed;
-		size_t fsize;
-		void *base;
-
-		if (!folio)
-			break;
-
-		fsize = folioq_folio_size(folioq, slot);
-		if (skip < fsize) {
-			base = kmap_local_folio(folio, skip);
-			part = umin(len, PAGE_SIZE - skip % PAGE_SIZE);
-			remain = step(base, progress, part, priv, priv2);
-			kunmap_local(base);
-			consumed = part - remain;
-			len -= consumed;
-			progress += consumed;
-			skip += consumed;
-		}
-		if (skip >= fsize) {
-			skip = 0;
-			slot++;
-			if (slot == folioq_nr_slots(folioq) && folioq->next) {
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-		if (remain)
-			break;
-	} while (len);
-
-	iter->folioq_slot = slot;
-	iter->folioq = folioq;
-	iter->iov_offset = skip;
-	iter->count -= progress;
-	return progress;
-}
-
 /*
  * Handle ITER_XARRAY.
  */
@@ -374,8 +317,6 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_kvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_bvecq(iter))
 		return iterate_bvecq(iter, len, priv, priv2, step);
-	if (iov_iter_is_folioq(iter))
-		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
 		return iterate_xarray(iter, len, priv, priv2, step);
 	return iterate_discard(iter, len, priv, priv2, step);
@@ -410,8 +351,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
  * buffer is presented in segments, which for kernel iteration are broken up by
  * physical pages and mapped, with the mapped address being presented.
  *
- * [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and
- * DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators.
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, XARRAY and DISCARD-type
+ * iterators; it will not handle UBUF or IOVEC-type iterators.
  *
  * A step functions, @step, must be provided, one for handling mapped kernel
  * addresses and the other is given user addresses which have the potential to
@@ -440,8 +381,6 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_kvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_bvecq(iter))
 		return iterate_bvecq(iter, len, priv, priv2, step);
-	if (iov_iter_is_folioq(iter))
-		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
 		return iterate_xarray(iter, len, priv, priv2, step);
 	return iterate_discard(iter, len, priv, priv2, step);
diff --git a/include/linux/uio.h b/include/linux/uio.h
index f7cfa6ea8213..e84a0c4f28c6 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -11,7 +11,6 @@
 #include <uapi/linux/uio.h>
 
 struct page;
-struct folio_queue;
 
 typedef unsigned int __bitwise iov_iter_extraction_t;
 
@@ -27,7 +26,6 @@ enum iter_type {
 	ITER_BVEC,
 	ITER_KVEC,
 	ITER_BVECQ,
-	ITER_FOLIOQ,
 	ITER_XARRAY,
 	ITER_DISCARD,
 };
@@ -70,7 +68,6 @@ struct iov_iter {
 				const struct kvec *kvec;
 				const struct bio_vec *bvec;
 				const struct bvecq *bvecq;
-				const struct folio_queue *folioq;
 				struct xarray *xarray;
 				void __user *ubuf;
 			};
@@ -80,7 +77,6 @@ struct iov_iter {
 	union {
 		unsigned long nr_segs;
 		u16 bvecq_slot;
-		u8 folioq_slot;
 		loff_t xarray_start;
 	};
 };
@@ -153,11 +149,6 @@ static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
 	return iov_iter_type(i) == ITER_BVECQ;
 }
 
-static inline bool iov_iter_is_folioq(const struct iov_iter *i)
-{
-	return iov_iter_type(i) == ITER_FOLIOQ;
-}
-
 static inline bool iov_iter_is_xarray(const struct iov_iter *i)
 {
 	return iov_iter_type(i) == ITER_XARRAY;
@@ -306,9 +297,6 @@ void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
 void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
 			 const struct bvecq *bvecq,
 			 unsigned int first_slot, unsigned int offset, size_t count);
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
-			  const struct folio_queue *folioq,
-			  unsigned int first_slot, unsigned int offset, size_t count);
 void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xarray *xarray,
 		     loff_t start, size_t count);
 ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 8c647783df2f..4898da340d96 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -572,39 +572,6 @@ static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
 	i->bvecq = bq;
 }
 
-static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
-{
-	const struct folio_queue *folioq = i->folioq;
-	unsigned int slot = i->folioq_slot;
-
-	if (!i->count)
-		return;
-	i->count -= size;
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		slot = 0;
-	}
-
-	size += i->iov_offset; /* From beginning of current segment. */
-	do {
-		size_t fsize = folioq_folio_size(folioq, slot);
-
-		if (likely(size < fsize))
-			break;
-		size -= fsize;
-		slot++;
-		if (slot >= folioq_nr_slots(folioq) && folioq->next) {
-			folioq = folioq->next;
-			slot = 0;
-		}
-	} while (size);
-
-	i->iov_offset = size;
-	i->folioq_slot = slot;
-	i->folioq = folioq;
-}
-
 void iov_iter_advance(struct iov_iter *i, size_t size)
 {
 	if (unlikely(i->count < size))
@@ -619,8 +586,6 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
 		iov_iter_bvec_advance(i, size);
 	} else if (iov_iter_is_bvecq(i)) {
 		iov_iter_bvecq_advance(i, size);
-	} else if (iov_iter_is_folioq(i)) {
-		iov_iter_folioq_advance(i, size);
 	} else if (iov_iter_is_discard(i)) {
 		i->count -= size;
 	}
@@ -654,32 +619,6 @@ static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
 	i->bvecq = bq;
 }
 
-static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
-{
-	const struct folio_queue *folioq = i->folioq;
-	unsigned int slot = i->folioq_slot;
-
-	for (;;) {
-		size_t fsize;
-
-		if (slot == 0) {
-			folioq = folioq->prev;
-			slot = folioq_nr_slots(folioq);
-		}
-		slot--;
-
-		fsize = folioq_folio_size(folioq, slot);
-		if (unroll <= fsize) {
-			i->iov_offset = fsize - unroll;
-			break;
-		}
-		unroll -= fsize;
-	}
-
-	i->folioq_slot = slot;
-	i->folioq = folioq;
-}
-
 void iov_iter_revert(struct iov_iter *i, size_t unroll)
 {
 	if (!unroll)
@@ -714,9 +653,6 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
 	} else if (iov_iter_is_bvecq(i)) {
 		i->iov_offset = 0;
 		iov_iter_bvecq_revert(i, unroll);
-	} else if (iov_iter_is_folioq(i)) {
-		i->iov_offset = 0;
-		iov_iter_folioq_revert(i, unroll);
 	} else { /* same logics for iovec and kvec */
 		const struct iovec *iov = iter_iov(i);
 		while (1) {
@@ -766,8 +702,6 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
 		}
 		return umin(i->count, bq->bv[slot].bv_len - offset);
 	}
-	if (unlikely(iov_iter_is_folioq(i)))
-		return umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
 	return i->count;
 }
 EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -833,36 +767,6 @@ void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
 }
 EXPORT_SYMBOL(iov_iter_bvec_queue);
 
-/**
- * iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
- * @i: The iterator to initialise.
- * @direction: The direction of the transfer.
- * @folioq: The starting point in the folio queue.
- * @first_slot: The first slot in the folio queue to use
- * @offset: The offset into the folio in the first slot to start at
- * @count: The size of the I/O buffer in bytes.
- *
- * Set up an I/O iterator to either draw data out of the pages attached to an
- * inode or to inject data into those pages.  The pages *must* be prevented
- * from evaporation, either by taking a ref on them or locking them by the
- * caller.
- */
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
-			  const struct folio_queue *folioq, unsigned int first_slot,
-			  unsigned int offset, size_t count)
-{
-	BUG_ON(direction & ~1);
-	*i = (struct iov_iter) {
-		.iter_type = ITER_FOLIOQ,
-		.data_source = direction,
-		.folioq = folioq,
-		.folioq_slot = first_slot,
-		.count = count,
-		.iov_offset = offset,
-	};
-}
-EXPORT_SYMBOL(iov_iter_folio_queue);
-
 /**
  * iov_iter_xarray - Initialise an I/O iterator to use the pages in an xarray
  * @i: The iterator to initialise.
@@ -1006,9 +910,7 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
 	if (iov_iter_is_bvecq(i))
 		return iov_iter_alignment_bvecq(i);
 
-	/* With both xarray and folioq types, we're dealing with whole folios. */
-	if (iov_iter_is_folioq(i))
-		return i->iov_offset | i->count;
+	/* With the xarray type, we're dealing with whole folios. */
 	if (iov_iter_is_xarray(i))
 		return (i->xarray_start + i->iov_offset) | i->count;
 
@@ -1265,11 +1167,6 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
 		return bvec_npages(i, maxpages);
 	if (iov_iter_is_bvecq(i))
 		return iov_npages_bvecq(i, maxpages);
-	if (iov_iter_is_folioq(i)) {
-		unsigned offset = i->iov_offset % PAGE_SIZE;
-		int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
-		return min(npages, maxpages);
-	}
 	if (iov_iter_is_xarray(i)) {
 		unsigned offset = (i->xarray_start + i->iov_offset) % PAGE_SIZE;
 		int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1694,68 +1591,6 @@ static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
 	return extracted;
 }
 
-/*
- * Extract a list of contiguous pages from an ITER_FOLIOQ iterator.  This does
- * not get references on the pages, nor does it get a pin on them.
- */
-static ssize_t iov_iter_extract_folioq_pages(struct iov_iter *i,
-					     struct page ***pages, size_t maxsize,
-					     unsigned int maxpages,
-					     iov_iter_extraction_t extraction_flags,
-					     size_t *offset0)
-{
-	const struct folio_queue *folioq = i->folioq;
-	struct page **p;
-	unsigned int nr = 0;
-	size_t extracted = 0, offset, slot = i->folioq_slot;
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		slot = 0;
-		if (WARN_ON(i->iov_offset != 0))
-			return -EIO;
-	}
-
-	offset = i->iov_offset & ~PAGE_MASK;
-	*offset0 = offset;
-
-	maxpages = want_pages_array(pages, maxsize, offset, maxpages);
-	if (!maxpages)
-		return -ENOMEM;
-	p = *pages;
-
-	for (;;) {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t offset = i->iov_offset, fsize = folioq_folio_size(folioq, slot);
-		size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
-		if (offset < fsize) {
-			part = umin(part, umin(maxsize - extracted, fsize - offset));
-			i->count -= part;
-			i->iov_offset += part;
-			extracted += part;
-
-			p[nr++] = folio_page(folio, offset / PAGE_SIZE);
-		}
-
-		if (nr >= maxpages || extracted >= maxsize)
-			break;
-
-		if (i->iov_offset >= fsize) {
-			i->iov_offset = 0;
-			slot++;
-			if (slot == folioq_nr_slots(folioq) && folioq->next) {
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	}
-
-	i->folioq = folioq;
-	i->folioq_slot = slot;
-	return extracted;
-}
-
 /*
  * Extract a list of contiguous pages from an ITER_XARRAY iterator.  This does not
  * get references on the pages, nor does it get a pin on them.
@@ -2018,8 +1853,8 @@ static ssize_t iov_iter_extract_user_pages(struct iov_iter *i,
  *      added to the pages, but refs will not be taken.
  *      iov_iter_extract_will_pin() will return true.
  *
- *  (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_FOLIOQ or ITER_XARRAY, the
- *      pages are merely listed; no extra refs or pins are obtained.
+ *  (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_XARRAY, the pages are
+ *      merely listed; no extra refs or pins are obtained.
  *      iov_iter_extract_will_pin() will return 0.
  *
  * Note also:
@@ -2058,10 +1893,6 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
 		return iov_iter_extract_bvecq_pages(i, pages, maxsize,
 						    maxpages, extraction_flags,
 						    offset0);
-	if (iov_iter_is_folioq(i))
-		return iov_iter_extract_folioq_pages(i, pages, maxsize,
-						     maxpages, extraction_flags,
-						     offset0);
 	if (iov_iter_is_xarray(i))
 		return iov_iter_extract_xarray_pages(i, pages, maxsize,
 						     maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index 23e5a180103b..b9a7298306d9 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -12,7 +12,6 @@
 #include <linux/bvec.h>
 #include <linux/bvecq.h>
 #include <linux/uio.h>
-#include <linux/folio_queue.h>
 
 /**
  * sg_nents - return total count of entries in scatterlist
@@ -1327,67 +1326,6 @@ static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
 	return ret;
 }
 
-/*
- * Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
- * the scatterlist.  The pages are not pinned.
- */
-static ssize_t extract_folioq_to_sg(struct iov_iter *iter,
-				   ssize_t maxsize,
-				   struct sg_table *sgtable,
-				   unsigned int sg_max,
-				   iov_iter_extraction_t extraction_flags)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	struct scatterlist *sg = sgtable->sgl + sgtable->nents;
-	unsigned int slot = iter->folioq_slot;
-	ssize_t ret = 0;
-	size_t offset = iter->iov_offset;
-
-	BUG_ON(!folioq);
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		if (WARN_ON_ONCE(!folioq))
-			return 0;
-		slot = 0;
-	}
-
-	do {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t fsize = folioq_folio_size(folioq, slot);
-
-		if (offset < fsize) {
-			size_t part = umin(maxsize - ret, fsize - offset);
-
-			sg_set_page(sg, folio_page(folio, 0), part, offset);
-			sgtable->nents++;
-			sg++;
-			sg_max--;
-			offset += part;
-			ret += part;
-		}
-
-		if (offset >= fsize) {
-			offset = 0;
-			slot++;
-			if (slot >= folioq_nr_slots(folioq)) {
-				if (!folioq->next) {
-					WARN_ON_ONCE(ret < iter->count);
-					break;
-				}
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	} while (sg_max > 0 && ret < maxsize);
-
-	iter->folioq = folioq;
-	iter->folioq_slot = slot;
-	iter->iov_offset = offset;
-	iter->count -= ret;
-	return ret;
-}
-
 /*
  * Extract up to sg_max folios from an XARRAY-type iterator and add them to
  * the scatterlist.  The pages are not pinned.
@@ -1451,8 +1389,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
  * addition of @sg_max elements.
  *
  * The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, BVECQ-, KVEC-, FOLIOQ- and XARRAY-type are extracted but
- * aren't pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, XARRAY-type are extracted but aren't pinned;
+ * DISCARD-type is not supported.
  *
  * No end mark is placed on the scatterlist; that's left to the caller.
  *
@@ -1487,9 +1425,6 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
 	case ITER_BVECQ:
 		return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
 					   extraction_flags);
-	case ITER_FOLIOQ:
-		return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
-					    extraction_flags);
 	case ITER_XARRAY:
 		return extract_xarray_to_sg(iter, maxsize, sgtable, sg_max,
 					    extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index f6413e6b925a..1c7085234006 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -13,7 +13,6 @@
 #include <linux/uio.h>
 #include <linux/bvec.h>
 #include <linux/bvecq.h>
-#include <linux/folio_queue.h>
 #include <linux/scatterlist.h>
 #include <linux/minmax.h>
 #include <linux/mman.h>
@@ -383,176 +382,6 @@ static void __init iov_kunit_copy_from_bvec(struct kunit *test)
 	KUNIT_SUCCEED(test);
 }
 
-static void iov_kunit_destroy_folioq(void *data)
-{
-	struct folio_queue *folioq, *next;
-
-	for (folioq = data; folioq; folioq = next) {
-		next = folioq->next;
-		kfree(folioq);
-	}
-}
-
-static void __init iov_kunit_load_folioq(struct kunit *test,
-					struct iov_iter *iter, int dir,
-					struct folio_queue *folioq,
-					struct page **pages, size_t npages)
-{
-	struct folio_queue *p = folioq;
-	size_t size = 0;
-	int i;
-
-	for (i = 0; i < npages; i++) {
-		if (folioq_full(p)) {
-			p->next = kzalloc_obj(struct folio_queue);
-			KUNIT_ASSERT_NOT_ERR_OR_NULL(test, p->next);
-			folioq_init(p->next, 0);
-			p->next->prev = p;
-			p = p->next;
-		}
-		folioq_append(p, page_folio(pages[i]));
-		size += PAGE_SIZE;
-	}
-	iov_iter_folio_queue(iter, dir, folioq, 0, 0, size);
-}
-
-static struct folio_queue *iov_kunit_create_folioq(struct kunit *test)
-{
-	struct folio_queue *folioq;
-
-	folioq = kzalloc_obj(struct folio_queue);
-	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, folioq);
-	kunit_add_action_or_reset(test, iov_kunit_destroy_folioq, folioq);
-	folioq_init(folioq, 0);
-	return folioq;
-}
-
-/*
- * Test copying to a ITER_FOLIOQ-type iterator.
- */
-static void __init iov_kunit_copy_to_folioq(struct kunit *test)
-{
-	const struct kvec_test_range *pr;
-	struct iov_iter iter;
-	struct folio_queue *folioq;
-	struct page **spages, **bpages;
-	u8 *scratch, *buffer;
-	size_t bufsize, npages, size, copied;
-	int i, patt;
-
-	bufsize = 0x100000;
-	npages = bufsize / PAGE_SIZE;
-
-	folioq = iov_kunit_create_folioq(test);
-
-	scratch = iov_kunit_create_buffer(test, &spages, npages);
-	for (i = 0; i < bufsize; i++)
-		scratch[i] = pattern(i);
-
-	buffer = iov_kunit_create_buffer(test, &bpages, npages);
-	memset(buffer, 0, bufsize);
-
-	iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
-
-	i = 0;
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
-		size = pr->to - pr->from;
-		KUNIT_ASSERT_LE(test, pr->to, bufsize);
-
-		iov_iter_folio_queue(&iter, READ, folioq, 0, 0, pr->to);
-		iov_iter_advance(&iter, pr->from);
-		copied = copy_to_iter(scratch + i, size, &iter);
-
-		KUNIT_EXPECT_EQ(test, copied, size);
-		KUNIT_EXPECT_EQ(test, iter.count, 0);
-		KUNIT_EXPECT_EQ(test, iter.iov_offset, pr->to % PAGE_SIZE);
-		i += size;
-		if (test->status == KUNIT_FAILURE)
-			goto stop;
-	}
-
-	/* Build the expected image in the scratch buffer. */
-	patt = 0;
-	memset(scratch, 0, bufsize);
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++)
-		for (i = pr->from; i < pr->to; i++)
-			scratch[i] = pattern(patt++);
-
-	/* Compare the images */
-	for (i = 0; i < bufsize; i++) {
-		KUNIT_EXPECT_EQ_MSG(test, buffer[i], scratch[i], "at i=%x", i);
-		if (buffer[i] != scratch[i])
-			return;
-	}
-
-stop:
-	KUNIT_SUCCEED(test);
-}
-
-/*
- * Test copying from a ITER_FOLIOQ-type iterator.
- */
-static void __init iov_kunit_copy_from_folioq(struct kunit *test)
-{
-	const struct kvec_test_range *pr;
-	struct iov_iter iter;
-	struct folio_queue *folioq;
-	struct page **spages, **bpages;
-	u8 *scratch, *buffer;
-	size_t bufsize, npages, size, copied;
-	int i, j;
-
-	bufsize = 0x100000;
-	npages = bufsize / PAGE_SIZE;
-
-	folioq = iov_kunit_create_folioq(test);
-
-	buffer = iov_kunit_create_buffer(test, &bpages, npages);
-	for (i = 0; i < bufsize; i++)
-		buffer[i] = pattern(i);
-
-	scratch = iov_kunit_create_buffer(test, &spages, npages);
-	memset(scratch, 0, bufsize);
-
-	iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
-
-	i = 0;
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
-		size = pr->to - pr->from;
-		KUNIT_ASSERT_LE(test, pr->to, bufsize);
-
-		iov_iter_folio_queue(&iter, WRITE, folioq, 0, 0, pr->to);
-		iov_iter_advance(&iter, pr->from);
-		copied = copy_from_iter(scratch + i, size, &iter);
-
-		KUNIT_EXPECT_EQ(test, copied, size);
-		KUNIT_EXPECT_EQ(test, iter.count, 0);
-		KUNIT_EXPECT_EQ(test, iter.iov_offset, pr->to % PAGE_SIZE);
-		i += size;
-	}
-
-	/* Build the expected image in the main buffer. */
-	i = 0;
-	memset(buffer, 0, bufsize);
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
-		for (j = pr->from; j < pr->to; j++) {
-			buffer[i++] = pattern(j);
-			if (i >= bufsize)
-				goto stop;
-		}
-	}
-stop:
-
-	/* Compare the images */
-	for (i = 0; i < bufsize; i++) {
-		KUNIT_EXPECT_EQ_MSG(test, scratch[i], buffer[i], "at i=%x", i);
-		if (scratch[i] != buffer[i])
-			return;
-	}
-
-	KUNIT_SUCCEED(test);
-}
-
 static void iov_kunit_destroy_bvecq(void *data)
 {
 	struct bvecq *bq, *next;
@@ -1124,85 +953,6 @@ static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
 	KUNIT_SUCCEED(test);
 }
 
-/*
- * Test the extraction of ITER_FOLIOQ-type iterators.
- */
-static void __init iov_kunit_extract_pages_folioq(struct kunit *test)
-{
-	const struct kvec_test_range *pr;
-	struct folio_queue *folioq;
-	struct iov_iter iter;
-	struct page **bpages, *pagelist[8], **pages = pagelist;
-	ssize_t len;
-	size_t bufsize, size = 0, npages;
-	int i, from;
-
-	bufsize = 0x100000;
-	npages = bufsize / PAGE_SIZE;
-
-	folioq = iov_kunit_create_folioq(test);
-
-	iov_kunit_create_buffer(test, &bpages, npages);
-	iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
-
-	for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
-		from = pr->from;
-		size = pr->to - from;
-		KUNIT_ASSERT_LE(test, pr->to, bufsize);
-
-		iov_iter_folio_queue(&iter, WRITE, folioq, 0, 0, pr->to);
-		iov_iter_advance(&iter, from);
-
-		do {
-			size_t offset0 = LONG_MAX;
-
-			for (i = 0; i < ARRAY_SIZE(pagelist); i++)
-				pagelist[i] = (void *)(unsigned long)0xaa55aa55aa55aa55ULL;
-
-			len = iov_iter_extract_pages(&iter, &pages, 100 * 1024,
-						     ARRAY_SIZE(pagelist), 0, &offset0);
-			KUNIT_EXPECT_GE(test, len, 0);
-			if (len < 0)
-				break;
-			KUNIT_EXPECT_LE(test, len, size);
-			KUNIT_EXPECT_EQ(test, iter.count, size - len);
-			if (len == 0)
-				break;
-			size -= len;
-			KUNIT_EXPECT_GE(test, (ssize_t)offset0, 0);
-			KUNIT_EXPECT_LT(test, offset0, PAGE_SIZE);
-
-			for (i = 0; i < ARRAY_SIZE(pagelist); i++) {
-				struct page *p;
-				ssize_t part = min_t(ssize_t, len, PAGE_SIZE - offset0);
-				int ix;
-
-				KUNIT_ASSERT_GE(test, part, 0);
-				ix = from / PAGE_SIZE;
-				KUNIT_ASSERT_LT(test, ix, npages);
-				p = bpages[ix];
-				KUNIT_EXPECT_PTR_EQ(test, pagelist[i], p);
-				KUNIT_EXPECT_EQ(test, offset0, from % PAGE_SIZE);
-				from += part;
-				len -= part;
-				KUNIT_ASSERT_GE(test, len, 0);
-				if (len == 0)
-					break;
-				offset0 = 0;
-			}
-
-			if (test->status == KUNIT_FAILURE)
-				goto stop;
-		} while (iov_iter_count(&iter) > 0);
-
-		KUNIT_EXPECT_EQ(test, size, 0);
-		KUNIT_EXPECT_EQ(test, iter.count, 0);
-	}
-
-stop:
-	KUNIT_SUCCEED(test);
-}
-
 /*
  * Test the extraction of ITER_XARRAY-type iterators.
  */
@@ -1430,23 +1180,6 @@ static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)
 	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
 }
 
-static void __init iov_kunit_iter_to_sg_folioq(struct kunit *test)
-{
-	struct iov_kunit_iter_to_sg_data data;
-	struct folio_queue *folioq;
-	struct iov_iter iter;
-	size_t bufsize;
-
-	bufsize = 0x200000;
-	iov_kunit_iter_to_sg_init(test, bufsize, false, &data);
-
-	folioq = iov_kunit_create_folioq(test);
-	iov_kunit_load_folioq(test, &iter, READ, folioq, data.pages,
-			      data.npages);
-
-	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
-}
-
 static void __init iov_kunit_iter_to_sg_xarray(struct kunit *test)
 {
 	struct iov_kunit_iter_to_sg_data data;
@@ -1485,18 +1218,14 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
 	KUNIT_CASE(iov_kunit_copy_from_bvec),
 	KUNIT_CASE(iov_kunit_copy_to_bvecq),
 	KUNIT_CASE(iov_kunit_copy_from_bvecq),
-	KUNIT_CASE(iov_kunit_copy_to_folioq),
-	KUNIT_CASE(iov_kunit_copy_from_folioq),
 	KUNIT_CASE(iov_kunit_copy_to_xarray),
 	KUNIT_CASE(iov_kunit_copy_from_xarray),
 	KUNIT_CASE(iov_kunit_extract_pages_kvec),
 	KUNIT_CASE(iov_kunit_extract_pages_bvec),
 	KUNIT_CASE(iov_kunit_extract_pages_bvecq),
-	KUNIT_CASE(iov_kunit_extract_pages_folioq),
 	KUNIT_CASE(iov_kunit_extract_pages_xarray),
 	KUNIT_CASE(iov_kunit_iter_to_sg_kvec),
 	KUNIT_CASE(iov_kunit_iter_to_sg_bvec),
-	KUNIT_CASE(iov_kunit_iter_to_sg_folioq),
 	KUNIT_CASE(iov_kunit_iter_to_sg_xarray),
 	KUNIT_CASE(iov_kunit_iter_to_sg_ubuf),
 	{}



From dhowells at redhat.com  Mon Aug 10 07:47:30 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:30 +0100
Subject: [PATCH v9 13/26] netfs: Remove the writethrough code
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-14-dhowells@redhat.com>

Remove the netfs writethrough code as it's very tricky to get the locking
right and it will probably deadlock if used in conjunction with Ceph snapshots
because it excludes writeback for the duration, but to flush out old
snapshots, it does a synchronous flush that invokes writeback.

Instead, O_SYNC writes do a flush after performing the write - which is
already there as the callers of netfs_perform_write() all call
generic_write_sync().

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/9p/vfs_addr.c             |   1 -
 fs/afs/file.c                |   1 -
 fs/netfs/buffered_write.c    |  56 ++-----------------
 fs/netfs/internal.h          |   7 ---
 fs/netfs/main.c              |   1 -
 fs/netfs/stats.c             |   4 +-
 fs/netfs/write_collect.c     |   2 -
 fs/netfs/write_issue.c       | 104 +----------------------------------
 include/linux/netfs.h        |   1 -
 include/trace/events/netfs.h |   8 +--
 10 files changed, 9 insertions(+), 176 deletions(-)

diff --git a/fs/9p/vfs_addr.c b/fs/9p/vfs_addr.c
index 1ac0b3dcc077..2129fcb0f65c 100644
--- a/fs/9p/vfs_addr.c
+++ b/fs/9p/vfs_addr.c
@@ -124,7 +124,6 @@ static int v9fs_init_request(struct netfs_io_request *rreq, struct file *file)
 	struct p9_fid *fid;
 	struct dentry *dentry;
 	bool writing = (rreq->origin == NETFS_READ_FOR_WRITE ||
-			rreq->origin == NETFS_WRITETHROUGH ||
 			rreq->origin == NETFS_UNBUFFERED_WRITE ||
 			rreq->origin == NETFS_DIO_WRITE);
 
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 0467742bfeee..3617c245f319 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -400,7 +400,6 @@ static int afs_init_request(struct netfs_io_request *rreq, struct file *file)
 		}
 		break;
 	case NETFS_WRITEBACK:
-	case NETFS_WRITETHROUGH:
 	case NETFS_UNBUFFERED_WRITE:
 	case NETFS_DIO_WRITE:
 		if (S_ISREG(rreq->inode->i_mode))
diff --git a/fs/netfs/buffered_write.c b/fs/netfs/buffered_write.c
index 2cdb68e6b16f..39f00b326e0f 100644
--- a/fs/netfs/buffered_write.c
+++ b/fs/netfs/buffered_write.c
@@ -91,44 +91,14 @@ ssize_t netfs_perform_write(struct kiocb *iocb, struct iov_iter *iter,
 	struct inode *inode = file_inode(file);
 	struct address_space *mapping = inode->i_mapping;
 	struct netfs_inode *ctx = netfs_inode(inode);
-	struct writeback_control wbc = {
-		.sync_mode	= WB_SYNC_NONE,
-		.for_sync	= true,
-		.nr_to_write	= LONG_MAX,
-		.range_start	= iocb->ki_pos,
-		.range_end	= iocb->ki_pos + iter->count,
-	};
-	struct netfs_io_request *wreq = NULL;
-	struct folio *folio = NULL, *writethrough = NULL;
+	struct folio *folio = NULL;
 	unsigned int bdp_flags = (iocb->ki_flags & IOCB_NOWAIT) ? BDP_ASYNC : 0;
-	ssize_t written = 0, ret, ret2;
+	ssize_t written = 0, ret;
 	loff_t pos = iocb->ki_pos;
 	size_t max_chunk = mapping_max_folio_size(mapping);
 	bool maybe_trouble = false;
 
-	if (unlikely(iocb->ki_flags & (IOCB_DSYNC | IOCB_SYNC))
-	    ) {
-		wbc_attach_fdatawrite_inode(&wbc, mapping->host);
-
-		ret = filemap_write_and_wait_range(mapping, pos, pos + iter->count);
-		if (ret < 0) {
-			wbc_detach_inode(&wbc);
-			goto out;
-		}
-
-		wreq = netfs_begin_writethrough(iocb, iter->count);
-		if (IS_ERR(wreq)) {
-			wbc_detach_inode(&wbc);
-			ret = PTR_ERR(wreq);
-			wreq = NULL;
-			goto out;
-		}
-		if (!is_sync_kiocb(iocb))
-			wreq->iocb = iocb;
-		netfs_stat(&netfs_n_wh_writethrough);
-	} else {
-		netfs_stat(&netfs_n_wh_buffered_write);
-	}
+	netfs_stat(&netfs_n_wh_buffered_write);
 
 	do {
 		enum netfs_folio_trace trace;
@@ -390,15 +360,8 @@ ssize_t netfs_perform_write(struct kiocb *iocb, struct iov_iter *iter,
 		pos += copied;
 		written += copied;
 
-		if (likely(!wreq)) {
-			folio_mark_dirty(folio);
-			folio_unlock(folio);
-		} else {
-			netfs_advance_writethrough(wreq, &wbc, folio, copied,
-						   offset + copied == flen,
-						   &writethrough);
-			/* Folio unlocked */
-		}
+		folio_mark_dirty(folio);
+		folio_unlock(folio);
 	retry:
 		folio_put(folio);
 		folio = NULL;
@@ -420,15 +383,6 @@ ssize_t netfs_perform_write(struct kiocb *iocb, struct iov_iter *iter,
 			ctx->ops->post_modify(inode);
 	}
 
-	if (unlikely(wreq)) {
-		ret2 = netfs_end_writethrough(wreq, &wbc, writethrough);
-		wbc_detach_inode(&wbc);
-		if (ret2 == -EIOCBQUEUED)
-			return ret2;
-		if (ret == 0 && ret2 < 0)
-			ret = ret2;
-	}
-
 	iocb->ki_pos += written;
 	_leave(" = %zd [%zd]", written, ret);
 	return written ? written : ret;
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index c68eea2ecc60..5aea8d34adcf 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -156,7 +156,6 @@ extern atomic_t netfs_n_rh_write_zskip;
 extern atomic_t netfs_n_rh_retry_read_req;
 extern atomic_t netfs_n_rh_retry_read_subreq;
 extern atomic_t netfs_n_wh_buffered_write;
-extern atomic_t netfs_n_wh_writethrough;
 extern atomic_t netfs_n_wh_dio_write;
 extern atomic_t netfs_n_wh_writepages;
 extern atomic_t netfs_n_wh_copy_to_cache;
@@ -216,12 +215,6 @@ void netfs_issue_write(struct netfs_io_request *wreq,
 size_t netfs_advance_write(struct netfs_io_request *wreq,
 			   struct netfs_io_stream *stream,
 			   loff_t start, size_t len, bool to_eof);
-struct netfs_io_request *netfs_begin_writethrough(struct kiocb *iocb, size_t len);
-int netfs_advance_writethrough(struct netfs_io_request *wreq, struct writeback_control *wbc,
-			       struct folio *folio, size_t copied, bool to_page_end,
-			       struct folio **writethrough_cache);
-ssize_t netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_control *wbc,
-			       struct folio *writethrough_cache);
 
 /*
  * write_retry.c
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 9f72e5054aff..3908ea2422c8 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -45,7 +45,6 @@ static const char *netfs_origins[nr__netfs_io_origin] = {
 	[NETFS_DIO_READ]		= "DR",
 	[NETFS_WRITEBACK]		= "WB",
 	[NETFS_WRITEBACK_SINGLE]	= "W1",
-	[NETFS_WRITETHROUGH]		= "WT",
 	[NETFS_UNBUFFERED_WRITE]	= "UW",
 	[NETFS_DIO_WRITE]		= "DW",
 	[NETFS_PGPRIV2_COPY_TO_CACHE]	= "2C",
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
index 84c2a4bcc762..a10d34f88597 100644
--- a/fs/netfs/stats.c
+++ b/fs/netfs/stats.c
@@ -32,7 +32,6 @@ atomic_t netfs_n_rh_write_zskip;
 atomic_t netfs_n_rh_retry_read_req;
 atomic_t netfs_n_rh_retry_read_subreq;
 atomic_t netfs_n_wh_buffered_write;
-atomic_t netfs_n_wh_writethrough;
 atomic_t netfs_n_wh_dio_write;
 atomic_t netfs_n_wh_writepages;
 atomic_t netfs_n_wh_copy_to_cache;
@@ -59,9 +58,8 @@ int netfs_stats_show(struct seq_file *m, void *v)
 		   atomic_read(&netfs_n_rh_read_single),
 		   atomic_read(&netfs_n_rh_write_begin),
 		   atomic_read(&netfs_n_rh_write_zskip));
-	seq_printf(m, "Writes : BW=%u WT=%u DW=%u WP=%u 2C=%u\n",
+	seq_printf(m, "Writes : BW=%u DW=%u WP=%u 2C=%u\n",
 		   atomic_read(&netfs_n_wh_buffered_write),
-		   atomic_read(&netfs_n_wh_writethrough),
 		   atomic_read(&netfs_n_wh_dio_write),
 		   atomic_read(&netfs_n_wh_writepages),
 		   atomic_read(&netfs_n_wh_copy_to_cache));
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 210eb8f3958d..2eac588e37de 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -214,7 +214,6 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 	smp_rmb();
 	collected_to = ULLONG_MAX;
 	if (wreq->origin == NETFS_WRITEBACK ||
-	    wreq->origin == NETFS_WRITETHROUGH ||
 	    wreq->origin == NETFS_PGPRIV2_COPY_TO_CACHE)
 		notes = NEED_UNLOCK;
 	else
@@ -411,7 +410,6 @@ bool netfs_write_collection(struct netfs_io_request *wreq)
 	switch (wreq->origin) {
 	case NETFS_WRITEBACK:
 	case NETFS_WRITEBACK_SINGLE:
-	case NETFS_WRITETHROUGH:
 		netfs_wb_end(ictx);
 		break;
 	default:
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 11f59e5a9953..f531dda7856a 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -96,7 +96,6 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 	struct netfs_inode *ictx;
 	bool is_cacheable = (origin == NETFS_WRITEBACK ||
 			     origin == NETFS_WRITEBACK_SINGLE ||
-			     origin == NETFS_WRITETHROUGH ||
 			     origin == NETFS_PGPRIV2_COPY_TO_CACHE);
 
 	wreq = netfs_alloc_request(mapping, file, start, 0, origin);
@@ -365,11 +364,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 		streamw = true;
 	}
 
-	if (wreq->origin == NETFS_WRITETHROUGH) {
-		to_eof = false;
-		if (flen > i_size - fpos)
-			flen = i_size - fpos;
-	} else if (flen > i_size - fpos) {
+	if (flen > i_size - fpos) {
 		flen = i_size - fpos;
 		if (!streamw)
 			folio_zero_segment(folio, flen, fsize);
@@ -611,103 +606,6 @@ int netfs_writepages(struct address_space *mapping,
 }
 EXPORT_SYMBOL(netfs_writepages);
 
-/*
- * Begin a write operation for writing through the pagecache.
- */
-struct netfs_io_request *netfs_begin_writethrough(struct kiocb *iocb, size_t len)
-{
-	struct netfs_io_request *wreq = NULL;
-	struct netfs_inode *ictx = netfs_inode(file_inode(iocb->ki_filp));
-
-	netfs_wb_begin(ictx, false);
-
-	wreq = netfs_create_write_req(iocb->ki_filp->f_mapping, iocb->ki_filp,
-				      iocb->ki_pos, NETFS_WRITETHROUGH);
-	if (IS_ERR(wreq)) {
-		netfs_wb_end(ictx);
-		return wreq;
-	}
-
-	wreq->io_streams[0].avail = true;
-	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
-	trace_netfs_write(wreq, netfs_write_trace_writethrough);
-	return wreq;
-}
-
-/*
- * Advance the state of the write operation used when writing through the
- * pagecache.  Data has been copied into the pagecache that we need to append
- * to the request.  If we've added more than wsize then we need to create a new
- * subrequest.
- */
-int netfs_advance_writethrough(struct netfs_io_request *wreq, struct writeback_control *wbc,
-			       struct folio *folio, size_t copied, bool to_page_end,
-			       struct folio **writethrough_cache)
-{
-	int ret;
-
-	_enter("R=%x ic=%zu ws=%u cp=%zu tp=%u",
-	       wreq->debug_id, wreq->buffer.iter.count, wreq->wsize, copied, to_page_end);
-
-	/* The folio is locked. */
-
-	if (*writethrough_cache != folio) {
-		if (*writethrough_cache) {
-			/* Did the folio get moved? */
-			folio_put(*writethrough_cache);
-			*writethrough_cache = NULL;
-		}
-		/* We can make multiple writes to the folio... */
-		if (wreq->len == 0)
-			trace_netfs_folio(folio, netfs_folio_trace_wthru);
-		else
-			trace_netfs_folio(folio, netfs_folio_trace_wthru_plus);
-		*writethrough_cache = folio;
-		folio_get(folio);
-	}
-
-	wreq->len += copied;
-
-	if (!to_page_end) {
-		folio_mark_dirty(folio);
-		folio_unlock(folio);
-		return 0;
-	}
-
-	ret = netfs_write_folio(wreq, wbc, folio);
-	folio_put(*writethrough_cache);
-	*writethrough_cache = NULL;
-	wreq->submitted = wreq->len;
-	return ret;
-}
-
-/*
- * End a write operation used when writing through the pagecache.
- */
-ssize_t netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_control *wbc,
-			       struct folio *writethrough_cache)
-{
-	ssize_t ret;
-
-	_enter("R=%x", wreq->debug_id);
-
-	if (writethrough_cache) {
-		folio_lock(writethrough_cache);
-		netfs_write_folio(wreq, wbc, writethrough_cache);
-		folio_put(writethrough_cache);
-		wreq->submitted = wreq->len;
-	}
-
-	netfs_end_issue_write(wreq);
-
-	if (wreq->iocb)
-		ret = -EIOCBQUEUED;
-	else
-		ret = netfs_wait_for_write(wreq);
-	netfs_put_request(wreq, netfs_rreq_trace_put_return);
-	return ret;
-}
-
 /**
  * netfs_writeback_single - Write back a monolithic payload
  * @mapping: The mapping to write from
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 6a4d4fd2551d..c8f4941b0ad7 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -210,7 +210,6 @@ enum netfs_io_origin {
 	NETFS_DIO_READ,			/* This is a direct I/O read */
 	NETFS_WRITEBACK,		/* This write was triggered by writepages */
 	NETFS_WRITEBACK_SINGLE,		/* This monolithic write was triggered by writepages */
-	NETFS_WRITETHROUGH,		/* This write was made by netfs_perform_write() */
 	NETFS_UNBUFFERED_WRITE,		/* This is an unbuffered write */
 	NETFS_DIO_WRITE,		/* This is a direct I/O write */
 	NETFS_PGPRIV2_COPY_TO_CACHE,	/* [DEPRECATED] This is writing read data to the cache */
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 5354a0439324..2ce162baaad1 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -30,8 +30,7 @@
 	EM(netfs_write_trace_dio_write,		"DIO-WRITE")	\
 	EM(netfs_write_trace_unbuffered_write,	"UNB-WRITE")	\
 	EM(netfs_write_trace_writeback,		"WRITEBACK")	\
-	EM(netfs_write_trace_writeback_single,	"WB-SINGLE") \
-	E_(netfs_write_trace_writethrough,	"WRITETHRU")
+	E_(netfs_write_trace_writeback_single,	"WB-SINGLE")
 
 #define netfs_rreq_origins					\
 	EM(NETFS_READAHEAD,			"RA")		\
@@ -43,7 +42,6 @@
 	EM(NETFS_DIO_READ,			"DR")		\
 	EM(NETFS_WRITEBACK,			"WB")		\
 	EM(NETFS_WRITEBACK_SINGLE,		"W1")		\
-	EM(NETFS_WRITETHROUGH,			"WT")		\
 	EM(NETFS_UNBUFFERED_WRITE,		"UW")		\
 	EM(NETFS_DIO_WRITE,			"DW")		\
 	E_(NETFS_PGPRIV2_COPY_TO_CACHE,		"2C")
@@ -221,9 +219,7 @@
 	EM(netfs_folio_trace_redirtied,		"redirtied")	\
 	EM(netfs_folio_trace_store,		"store")	\
 	EM(netfs_folio_trace_store_copy,	"store-copy")	\
-	EM(netfs_folio_trace_store_plus,	"store+")	\
-	EM(netfs_folio_trace_wthru,		"wthru")	\
-	E_(netfs_folio_trace_wthru_plus,	"wthru+")
+	E_(netfs_folio_trace_store_plus,	"store+")
 
 #define netfs_collect_contig_traces				\
 	EM(netfs_contig_trace_collect,		"Collect")	\



From dhowells at redhat.com  Mon Aug 10 07:47:39 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:39 +0100
Subject: [PATCH v9 22/26] netfs: Remove folio_queue and rolling_buffer
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-23-dhowells@redhat.com>

Remove folio_queue and rolling_buffer as they're no longer used.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: Steve French <sfrench at samba.org>
cc: linux-cifs at vger.kernel.org
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 Documentation/core-api/folio_queue.rst      | 209 --------------
 Documentation/core-api/index.rst            |   1 -
 Documentation/filesystems/netfs_library.rst |   2 +-
 fs/netfs/internal.h                         |   1 -
 fs/netfs/iterator.c                         | 192 -------------
 fs/netfs/main.c                             |   1 -
 fs/netfs/rolling_buffer.c                   | 301 --------------------
 include/linux/folio_queue.h                 | 282 ------------------
 include/linux/netfs.h                       |   2 -
 include/linux/rolling_buffer.h              |  64 -----
 kernel/bpf/btf.c                            |   2 -
 11 files changed, 1 insertion(+), 1056 deletions(-)
 delete mode 100644 Documentation/core-api/folio_queue.rst
 delete mode 100644 fs/netfs/rolling_buffer.c
 delete mode 100644 include/linux/folio_queue.h
 delete mode 100644 include/linux/rolling_buffer.h

diff --git a/Documentation/core-api/folio_queue.rst b/Documentation/core-api/folio_queue.rst
deleted file mode 100644
index b7628896d2b6..000000000000
--- a/Documentation/core-api/folio_queue.rst
+++ /dev/null
@@ -1,209 +0,0 @@
-.. SPDX-License-Identifier: GPL-2.0+
-
-===========
-Folio Queue
-===========
-
-:Author: David Howells <dhowells at redhat.com>
-
-.. Contents:
-
- * Overview
- * Initialisation
- * Adding and removing folios
- * Querying information about a folio
- * Querying information about a folio_queue
- * Folio queue iteration
- * Folio marks
- * Lockless simultaneous production/consumption issues
-
-
-Overview
-========
-
-The folio_queue struct forms a single segment in a segmented list of folios
-that can be used to form an I/O buffer.  As such, the list can be iterated over
-using the ITER_FOLIOQ iov_iter type.
-
-The publicly accessible members of the structure are::
-
-	struct folio_queue {
-		struct folio_queue *next;
-		struct folio_queue *prev;
-		...
-	};
-
-A pair of pointers are provided, ``next`` and ``prev``, that point to the
-segments on either side of the segment being accessed.  Whilst this is a
-doubly-linked list, it is intentionally not a circular list; the outward
-sibling pointers in terminal segments should be NULL.
-
-Each segment in the list also stores:
-
- * an ordered sequence of folio pointers,
- * the size of each folio and
- * three 1-bit marks per folio,
-
-but these should not be accessed directly as the underlying data structure may
-change, but rather the access functions outlined below should be used.
-
-The facility can be made accessible by::
-
-	#include <linux/folio_queue.h>
-
-and to use the iterator::
-
-	#include <linux/uio.h>
-
-
-Initialisation
-==============
-
-A segment should be initialised by calling::
-
-	void folioq_init(struct folio_queue *folioq);
-
-with a pointer to the segment to be initialised.  Note that this will not
-necessarily initialise all the folio pointers, so care must be taken to check
-the number of folios added.
-
-
-Adding and removing folios
-==========================
-
-Folios can be set in the next unused slot in a segment struct by calling one
-of::
-
-	unsigned int folioq_append(struct folio_queue *folioq,
-				   struct folio *folio);
-
-	unsigned int folioq_append_mark(struct folio_queue *folioq,
-					struct folio *folio);
-
-Both functions update the stored folio count, store the folio and note its
-size.  The second function also sets the first mark for the folio added.  Both
-functions return the number of the slot used.  [!] Note that no attempt is made
-to check that the capacity wasn't overrun and the list will not be extended
-automatically.
-
-A folio can be excised by calling::
-
-	void folioq_clear(struct folio_queue *folioq, unsigned int slot);
-
-This clears the slot in the array and also clears all the marks for that folio,
-but doesn't change the folio count - so future accesses of that slot must check
-if the slot is occupied.
-
-
-Querying information about a folio
-==================================
-
-Information about the folio in a particular slot may be queried by the
-following function::
-
-	struct folio *folioq_folio(const struct folio_queue *folioq,
-				   unsigned int slot);
-
-If a folio has not yet been set in that slot, this may yield an undefined
-pointer.  The size of the folio in a slot may be queried with either of::
-
-	unsigned int folioq_folio_order(const struct folio_queue *folioq,
-					unsigned int slot);
-
-	size_t folioq_folio_size(const struct folio_queue *folioq,
-				 unsigned int slot);
-
-The first function returns the size as an order and the second as a number of
-bytes.
-
-
-Querying information about a folio_queue
-========================================
-
-Information may be retrieved about a particular segment with the following
-functions::
-
-	unsigned int folioq_nr_slots(const struct folio_queue *folioq);
-
-	unsigned int folioq_count(struct folio_queue *folioq);
-
-	bool folioq_full(struct folio_queue *folioq);
-
-The first function returns the maximum capacity of a segment.  It must not be
-assumed that this won't vary between segments.  The second returns the number
-of folios added to a segments and the third is a shorthand to indicate if the
-segment has been filled to capacity.
-
-Not that the count and fullness are not affected by clearing folios from the
-segment.  These are more about indicating how many slots in the array have been
-initialised, and it assumed that slots won't get reused, but rather the segment
-will get discarded as the queue is consumed.
-
-
-Folio marks
-===========
-
-Folios within a queue can also have marks assigned to them.  These marks can be
-used to note information such as if a folio needs folio_put() calling upon it.
-There are three marks available to be set for each folio.
-
-The marks can be set by::
-
-	void folioq_mark(struct folio_queue *folioq, unsigned int slot);
-	void folioq_mark2(struct folio_queue *folioq, unsigned int slot);
-
-Cleared by::
-
-	void folioq_unmark(struct folio_queue *folioq, unsigned int slot);
-	void folioq_unmark2(struct folio_queue *folioq, unsigned int slot);
-
-And the marks can be queried by::
-
-	bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot);
-	bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot);
-
-The marks can be used for any purpose and are not interpreted by this API.
-
-
-Folio queue iteration
-=====================
-
-A list of segments may be iterated over using the I/O iterator facility using
-an ``iov_iter`` iterator of ``ITER_FOLIOQ`` type.  The iterator may be
-initialised with::
-
-	void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
-				  const struct folio_queue *folioq,
-				  unsigned int first_slot, unsigned int offset,
-				  size_t count);
-
-This may be told to start at a particular segment, slot and offset within a
-queue.  The iov iterator functions will follow the next pointers when advancing
-and prev pointers when reverting when needed.
-
-
-Lockless simultaneous production/consumption issues
-===================================================
-
-If properly managed, the list can be extended by the producer at the head end
-and shortened by the consumer at the tail end simultaneously without the need
-to take locks.  The ITER_FOLIOQ iterator inserts appropriate barriers to aid
-with this.
-
-Care must be taken when simultaneously producing and consuming a list.  If the
-last segment is reached and the folios it refers to are entirely consumed by
-the IOV iterators, an iov_iter struct will be left pointing to the last segment
-with a slot number equal to the capacity of that segment.  The iterator will
-try to continue on from this if there's another segment available when it is
-used again, but care must be taken lest the segment got removed and freed by
-the consumer before the iterator was advanced.
-
-It is recommended that the queue always contain at least one segment, even if
-that segment has never been filled or is entirely spent.  This prevents the
-head and tail pointers from collapsing.
-
-
-API Function Reference
-======================
-
-.. kernel-doc:: include/linux/folio_queue.h
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index 13769d5c40bf..16c529a33ac4 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -39,7 +39,6 @@ Library functionality that is used throughout the kernel.
    kref
    cleanup
    assoc_array
-   folio_queue
    xarray
    maple_tree
    idr
diff --git a/Documentation/filesystems/netfs_library.rst b/Documentation/filesystems/netfs_library.rst
index ddd799df6ce3..18e3c3aae57c 100644
--- a/Documentation/filesystems/netfs_library.rst
+++ b/Documentation/filesystems/netfs_library.rst
@@ -449,7 +449,7 @@ be called from the writeback code to write the data to the cache, if there is
 one.
 
 The inode should be marked ``NETFS_ICTX_SINGLE_NO_UPLOAD`` if this API is to be
-used.  The writeback function requires the buffer to be of ITER_FOLIOQ type.
+used.
 
 High-Level VM API
 ==================
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 667d18911949..315b02f5fa67 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -45,7 +45,6 @@ extern spinlock_t netfs_proc_lock;
 extern mempool_t netfs_request_pool;
 extern mempool_t netfs_subrequest_pool;
 extern mempool_t netfs_bvecq_pool;
-extern mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
 static inline void netfs_proc_add_rreq(struct netfs_io_request *rreq)
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index 499516ba6399..271ede100746 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -158,195 +158,3 @@ ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pag
 
 }
 EXPORT_SYMBOL_GPL(netfs_extract_iter);
-
-#if 0
-/*
- * Select the span of a bvec iterator we're going to use.  Limit it by both maximum
- * size and maximum number of segments.  Returns the size of the span in bytes.
- */
-static size_t netfs_limit_bvec(const struct iov_iter *iter, size_t start_offset,
-			       size_t max_size, size_t max_segs)
-{
-	const struct bio_vec *bvecs = iter->bvec;
-	unsigned int nbv = iter->nr_segs, ix = 0, nsegs = 0;
-	size_t len, span = 0, n = iter->count;
-	size_t skip = iter->iov_offset + start_offset;
-
-	if (WARN_ON(!iov_iter_is_bvec(iter)) ||
-	    WARN_ON(start_offset > n) ||
-	    n == 0)
-		return 0;
-
-	while (n && ix < nbv && skip) {
-		len = bvecs[ix].bv_len;
-		if (skip < len)
-			break;
-		skip -= len;
-		n -= len;
-		ix++;
-	}
-
-	while (n && ix < nbv) {
-		len = min3(n, bvecs[ix].bv_len - skip, max_size);
-		span += len;
-		nsegs++;
-		ix++;
-		if (span >= max_size || nsegs >= max_segs)
-			break;
-		skip = 0;
-		n -= len;
-	}
-
-	return min(span, max_size);
-}
-
-/*
- * Select the span of a kvec iterator we're going to use.  Limit it by both
- * maximum size and maximum number of segments.  Returns the size of the span
- * in bytes.
- */
-static size_t netfs_limit_kvec(const struct iov_iter *iter, size_t start_offset,
-			       size_t max_size, size_t max_segs)
-{
-	const struct kvec *kvecs = iter->kvec;
-	unsigned int nkv = iter->nr_segs, ix = 0, nsegs = 0;
-	size_t len, span = 0, n = iter->count;
-	size_t skip = iter->iov_offset + start_offset;
-
-	if (WARN_ON(!iov_iter_is_kvec(iter)) ||
-	    WARN_ON(start_offset > n) ||
-	    n == 0)
-		return 0;
-
-	while (n && ix < nkv && skip) {
-		len = kvecs[ix].iov_len;
-		if (skip < len)
-			break;
-		skip -= len;
-		n -= len;
-		ix++;
-	}
-
-	while (n && ix < nkv) {
-		len = min3(n, kvecs[ix].iov_len - skip, max_size);
-		span += len;
-		nsegs++;
-		ix++;
-		if (span >= max_size || nsegs >= max_segs)
-			break;
-		skip = 0;
-		n -= len;
-	}
-
-	return min(span, max_size);
-}
-
-/*
- * Select the span of an xarray iterator we're going to use.  Limit it by both
- * maximum size and maximum number of segments.  It is assumed that segments
- * can be larger than a page in size, provided they're physically contiguous.
- * Returns the size of the span in bytes.
- */
-static size_t netfs_limit_xarray(const struct iov_iter *iter, size_t start_offset,
-				 size_t max_size, size_t max_segs)
-{
-	struct folio *folio;
-	unsigned int nsegs = 0;
-	loff_t pos = iter->xarray_start + iter->iov_offset;
-	pgoff_t index = pos / PAGE_SIZE;
-	size_t span = 0, n = iter->count;
-
-	XA_STATE(xas, iter->xarray, index);
-
-	if (WARN_ON(!iov_iter_is_xarray(iter)) ||
-	    WARN_ON(start_offset > n) ||
-	    n == 0)
-		return 0;
-	max_size = min(max_size, n - start_offset);
-
-	rcu_read_lock();
-	xas_for_each(&xas, folio, ULONG_MAX) {
-		size_t offset, flen, len;
-		if (xas_retry(&xas, folio))
-			continue;
-		if (WARN_ON(xa_is_value(folio)))
-			break;
-		if (WARN_ON(folio_test_hugetlb(folio)))
-			break;
-
-		flen = folio_size(folio);
-		offset = offset_in_folio(folio, pos);
-		len = min(max_size, flen - offset);
-		span += len;
-		nsegs++;
-		if (span >= max_size || nsegs >= max_segs)
-			break;
-	}
-
-	rcu_read_unlock();
-	return min(span, max_size);
-}
-
-/*
- * Select the span of a folio queue iterator we're going to use.  Limit it by
- * both maximum size and maximum number of segments.  Returns the size of the
- * span in bytes.
- */
-static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offset,
-				 size_t max_size, size_t max_segs)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	unsigned int nsegs = 0;
-	unsigned int slot = iter->folioq_slot;
-	size_t span = 0, n = iter->count;
-
-	if (WARN_ON(!iov_iter_is_folioq(iter)) ||
-	    WARN_ON(start_offset > n) ||
-	    n == 0)
-		return 0;
-	max_size = umin(max_size, n - start_offset);
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		slot = 0;
-	}
-
-	start_offset += iter->iov_offset;
-	do {
-		size_t flen = folioq_folio_size(folioq, slot);
-
-		if (start_offset < flen) {
-			span += flen - start_offset;
-			nsegs++;
-			start_offset = 0;
-		} else {
-			start_offset -= flen;
-		}
-		if (span >= max_size || nsegs >= max_segs)
-			break;
-
-		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = folioq->next;
-			slot = 0;
-		}
-	} while (folioq);
-
-	return umin(span, max_size);
-}
-
-size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
-			size_t max_size, size_t max_segs)
-{
-	if (iov_iter_is_folioq(iter))
-		return netfs_limit_folioq(iter, start_offset, max_size, max_segs);
-	if (iov_iter_is_bvec(iter))
-		return netfs_limit_bvec(iter, start_offset, max_size, max_segs);
-	if (iov_iter_is_xarray(iter))
-		return netfs_limit_xarray(iter, start_offset, max_size, max_segs);
-	if (iov_iter_is_kvec(iter))
-		return netfs_limit_kvec(iter, start_offset, max_size, max_segs);
-	BUG();
-}
-EXPORT_SYMBOL(netfs_limit_iter);
-#endif
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 041810707eb4..5d8b87f71888 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -29,7 +29,6 @@ static struct kmem_cache *netfs_subrequest_slab;
 mempool_t netfs_request_pool;
 mempool_t netfs_subrequest_pool;
 mempool_t netfs_bvecq_pool;
-mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
 LIST_HEAD(netfs_io_requests);
diff --git a/fs/netfs/rolling_buffer.c b/fs/netfs/rolling_buffer.c
deleted file mode 100644
index 4e8584725b6a..000000000000
--- a/fs/netfs/rolling_buffer.c
+++ /dev/null
@@ -1,301 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* Rolling buffer helpers
- *
- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells at redhat.com)
- */
-
-#include <linux/bitops.h>
-#include <linux/mempool.h>
-#include <linux/pagemap.h>
-#include <linux/rolling_buffer.h>
-#include <linux/slab.h>
-#include "internal.h"
-
-static atomic_t debug_ids;
-
-/**
- * netfs_folioq_alloc - Allocate a folio_queue struct
- * @rreq_id: Associated debugging ID for tracing purposes
- * @gfp: Allocation constraints
- * @trace: Trace tag to indicate the purpose of the allocation
- *
- * Allocate, initialise and account the folio_queue struct and log a trace line
- * to mark the allocation.
- */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
-				       unsigned int /*enum netfs_folioq_trace*/ trace)
-{
-	struct folio_queue *fq;
-
-	if (gfp == GFP_KERNEL)
-		fq = netfs_folioq_pool.alloc(gfp, netfs_folioq_pool.pool_data);
-	else
-		fq = mempool_alloc(&netfs_folioq_pool, gfp);
-	if (fq) {
-		netfs_stat(&netfs_n_folioq);
-		folioq_init(fq, rreq_id);
-		fq->debug_id = atomic_inc_return(&debug_ids);
-		trace_netfs_folioq(fq, trace);
-	}
-	return fq;
-}
-EXPORT_SYMBOL(netfs_folioq_alloc);
-
-/**
- * netfs_folioq_free - Free a folio_queue struct
- * @folioq: The object to free
- * @trace: Trace tag to indicate which free
- *
- * Free and unaccount the folio_queue struct.
- */
-void netfs_folioq_free(struct folio_queue *folioq,
-		       unsigned int /*enum netfs_trace_folioq*/ trace)
-{
-	trace_netfs_folioq(folioq, trace);
-	netfs_stat_d(&netfs_n_folioq);
-	mempool_free(folioq, &netfs_folioq_pool);
-}
-EXPORT_SYMBOL(netfs_folioq_free);
-
-/*
- * Initialise a rolling buffer.  We allocate an empty folio queue struct to so
- * that the pointers can be independently driven by the producer and the
- * consumer.
- */
-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
-			unsigned int direction, gfp_t gfp)
-{
-	struct folio_queue *fq;
-
-	fq = netfs_folioq_alloc(rreq_id, gfp, netfs_trace_folioq_rollbuf_init);
-	if (!fq)
-		return -ENOMEM;
-
-	roll->head = fq;
-	roll->tail = fq;
-	iov_iter_folio_queue(&roll->iter, direction, fq, 0, 0, 0);
-	return 0;
-}
-
-/*
- * Add another folio_queue to a rolling buffer if there's no space left.
- */
-int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
-{
-	struct folio_queue *fq, *head = roll->head;
-
-	if (!folioq_full(head))
-		return 0;
-
-	fq = netfs_folioq_alloc(head->rreq_id, gfp, netfs_trace_folioq_make_space);
-	if (!fq)
-		return -ENOMEM;
-	fq->prev = head;
-
-	roll->head = fq;
-	if (folioq_full(head)) {
-		/* Make sure we don't leave the master iterator pointing to a
-		 * block that might get immediately consumed.
-		 */
-		if (roll->iter.folioq == head &&
-		    roll->iter.folioq_slot == folioq_nr_slots(head)) {
-			roll->iter.folioq = fq;
-			roll->iter.folioq_slot = 0;
-		}
-	}
-
-	/* Make sure the initialisation is stored before the next pointer.
-	 *
-	 * [!] NOTE: After we set head->next, the consumer is at liberty to
-	 * immediately delete the old head.
-	 */
-	smp_store_release(&head->next, fq);
-	return 0;
-}
-
-/*
- * Decant the list of folios to read into a rolling buffer.
- */
-ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
-				    struct readahead_control *ractl,
-				    struct folio_batch *put_batch)
-{
-	struct folio_queue *fq;
-	struct page **vec;
-	int nr, ix, to;
-	ssize_t size = 0;
-
-	if (rolling_buffer_make_space(roll, GFP_KERNEL) < 0)
-		return -ENOMEM;
-
-	fq = roll->head;
-	vec = (struct page **)fq->vec.folios;
-	nr = __readahead_batch(ractl, vec + folio_batch_count(&fq->vec),
-			       folio_batch_space(&fq->vec));
-	ix = fq->vec.nr;
-	to = ix + nr;
-	fq->vec.nr = to;
-	for (; ix < to; ix++) {
-		struct folio *folio = folioq_folio(fq, ix);
-		unsigned int order = folio_order(folio);
-
-		fq->orders[ix] = order;
-		size += PAGE_SIZE << order;
-		trace_netfs_folio(folio, netfs_folio_trace_read);
-		if (!folio_batch_add(put_batch, folio))
-			folio_batch_release(put_batch);
-	}
-	WRITE_ONCE(roll->iter.count, roll->iter.count + size);
-
-	/* Store the counter after setting the slot. */
-	smp_store_release(&roll->next_head_slot, to);
-	return size;
-}
-
-/*
- * Decant the entire list of folios to read into a rolling buffer.
- */
-ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
-					 struct readahead_control *ractl,
-					 unsigned int rreq_id)
-{
-	XA_STATE(xas, &ractl->mapping->i_pages, ractl->_index);
-	struct folio_queue *fq;
-	struct folio *folio;
-	ssize_t loaded = 0;
-	int nr, slot = 0, npages = 0;
-
-	/* First allocate all the folioqs we're going to need to avoid having
-	 * to deal with ENOMEM later.
-	 */
-	nr = ractl->_nr_folios;
-	do {
-		fq = netfs_folioq_alloc(rreq_id, GFP_KERNEL,
-					netfs_trace_folioq_make_space);
-		if (!fq) {
-			rolling_buffer_clear(roll);
-			return -ENOMEM;
-		}
-		fq->prev = roll->head;
-		if (!roll->tail)
-			roll->tail = fq;
-		else
-			roll->head->next = fq;
-		roll->head = fq;
-
-		nr -= folioq_nr_slots(fq);
-	} while (nr > 0);
-
-	rcu_read_lock();
-
-	fq = roll->tail;
-	xas_for_each(&xas, folio, ractl->_index + ractl->_nr_pages - 1) {
-		unsigned int order;
-
-		if (xas_retry(&xas, folio))
-			continue;
-		VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
-
-		order = folio_order(folio);
-		fq->orders[slot] = order;
-		fq->vec.folios[slot] = folio;
-		loaded += PAGE_SIZE << order;
-		npages += 1 << order;
-		trace_netfs_folio(folio, netfs_folio_trace_read);
-
-		slot++;
-		if (slot >= folioq_nr_slots(fq)) {
-			fq->vec.nr = slot;
-			fq = fq->next;
-			if (!fq) {
-				WARN_ON_ONCE(npages < readahead_count(ractl));
-				break;
-			}
-			slot = 0;
-		}
-	}
-
-	rcu_read_unlock();
-
-	if (fq)
-		fq->vec.nr = slot;
-
-	WRITE_ONCE(roll->iter.count, loaded);
-	iov_iter_folio_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
-	ractl->_index    += npages;
-	ractl->_nr_pages -= npages;
-	return loaded;
-}
-
-/*
- * Append a folio to the rolling buffer.
- */
-ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
-			      unsigned int flags, gfp_t gfp)
-{
-	ssize_t size = folio_size(folio);
-	int slot;
-
-	if (rolling_buffer_make_space(roll, gfp) < 0)
-		return -ENOMEM;
-
-	slot = folioq_append(roll->head, folio);
-	if (flags & ROLLBUF_MARK_1)
-		folioq_mark(roll->head, slot);
-	if (flags & ROLLBUF_MARK_2)
-		folioq_mark2(roll->head, slot);
-
-	WRITE_ONCE(roll->iter.count, roll->iter.count + size);
-
-	/* Store the counter after setting the slot. */
-	smp_store_release(&roll->next_head_slot, slot);
-	return size;
-}
-
-/*
- * Delete a spent buffer from a rolling queue and return the next in line.  We
- * don't return the last buffer to keep the pointers independent, but return
- * NULL instead.
- */
-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll)
-{
-	struct folio_queue *spent = roll->tail, *next = READ_ONCE(spent->next);
-
-	if (!next)
-		return NULL;
-	next->prev = NULL;
-	netfs_folioq_free(spent, netfs_trace_folioq_delete);
-	roll->tail = next;
-	return next;
-}
-
-/*
- * Clear out a rolling queue.  Folios that have mark 1 set are put.
- */
-void rolling_buffer_clear(struct rolling_buffer *roll)
-{
-	struct folio_batch fbatch;
-	struct folio_queue *p;
-
-	folio_batch_init(&fbatch);
-
-	while ((p = roll->tail)) {
-		roll->tail = p->next;
-		for (int slot = 0; slot < folioq_count(p); slot++) {
-			struct folio *folio = folioq_folio(p, slot);
-
-			if (!folio)
-				continue;
-			if (folioq_is_marked(p, slot)) {
-				trace_netfs_folio(folio, netfs_folio_trace_put);
-				if (!folio_batch_add(&fbatch, folio))
-					folio_batch_release(&fbatch);
-			}
-		}
-
-		netfs_folioq_free(p, netfs_trace_folioq_clear);
-	}
-
-	folio_batch_release(&fbatch);
-}
diff --git a/include/linux/folio_queue.h b/include/linux/folio_queue.h
deleted file mode 100644
index f6d5f1f127c9..000000000000
--- a/include/linux/folio_queue.h
+++ /dev/null
@@ -1,282 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/* Queue of folios definitions
- *
- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells at redhat.com)
- *
- * See:
- *
- *	Documentation/core-api/folio_queue.rst
- *
- * for a description of the API.
- */
-
-#ifndef _LINUX_FOLIO_QUEUE_H
-#define _LINUX_FOLIO_QUEUE_H
-
-#include <linux/folio_batch.h>
-#include <linux/mm.h>
-
-/*
- * Segment in a queue of running buffers.  Each segment can hold a number of
- * folios and a portion of the queue can be referenced with the ITER_FOLIOQ
- * iterator.  The possibility exists of inserting non-folio elements into the
- * queue (such as gaps).
- *
- * Explicit prev and next pointers are used instead of a list_head to make it
- * easier to add segments to tail and remove them from the head without the
- * need for a lock.
- */
-struct folio_queue {
-	struct folio_batch	vec;		/* Folios in the queue segment */
-	u8			orders[FOLIO_BATCH_SIZE]; /* Order of each folio */
-	struct folio_queue	*next;		/* Next queue segment or NULL */
-	struct folio_queue	*prev;		/* Previous queue segment of NULL */
-	unsigned long		marks;		/* 1-bit mark per folio */
-	unsigned long		marks2;		/* Second 1-bit mark per folio */
-#if FOLIO_BATCH_SIZE > BITS_PER_LONG
-#error marks is not big enough
-#endif
-	unsigned int		rreq_id;
-	unsigned int		debug_id;
-};
-
-/**
- * folioq_init - Initialise a folio queue segment
- * @folioq: The segment to initialise
- * @rreq_id: The request identifier to use in tracelines.
- *
- * Initialise a folio queue segment and set an identifier to be used in traces.
- *
- * Note that the folio pointers are left uninitialised.
- */
-static inline void folioq_init(struct folio_queue *folioq, unsigned int rreq_id)
-{
-	folio_batch_init(&folioq->vec);
-	folioq->next = NULL;
-	folioq->prev = NULL;
-	folioq->marks = 0;
-	folioq->marks2 = 0;
-	folioq->rreq_id = rreq_id;
-	folioq->debug_id = 0;
-}
-
-/**
- * folioq_nr_slots: Query the capacity of a folio queue segment
- * @folioq: The segment to query
- *
- * Query the number of folios that a particular folio queue segment might hold.
- * [!] NOTE: This must not be assumed to be the same for every segment!
- */
-static inline unsigned int folioq_nr_slots(const struct folio_queue *folioq)
-{
-	return FOLIO_BATCH_SIZE;
-}
-
-/**
- * folioq_count: Query the occupancy of a folio queue segment
- * @folioq: The segment to query
- *
- * Query the number of folios that have been added to a folio queue segment.
- * Note that this is not decreased as folios are removed from a segment.
- */
-static inline unsigned int folioq_count(struct folio_queue *folioq)
-{
-	return folio_batch_count(&folioq->vec);
-}
-
-/**
- * folioq_full: Query if a folio queue segment is full
- * @folioq: The segment to query
- *
- * Query if a folio queue segment is fully occupied.  Note that this does not
- * change if folios are removed from a segment.
- */
-static inline bool folioq_full(struct folio_queue *folioq)
-{
-	//return !folio_batch_space(&folioq->vec);
-	return folioq_count(folioq) >= folioq_nr_slots(folioq);
-}
-
-/**
- * folioq_is_marked: Check first folio mark in a folio queue segment
- * @folioq: The segment to query
- * @slot: The slot number of the folio to query
- *
- * Determine if the first mark is set for the folio in the specified slot in a
- * folio queue segment.
- */
-static inline bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot)
-{
-	return test_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_mark: Set the first mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Set the first mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_mark(struct folio_queue *folioq, unsigned int slot)
-{
-	set_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_unmark: Clear the first mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Clear the first mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_unmark(struct folio_queue *folioq, unsigned int slot)
-{
-	clear_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_is_marked2: Check second folio mark in a folio queue segment
- * @folioq: The segment to query
- * @slot: The slot number of the folio to query
- *
- * Determine if the second mark is set for the folio in the specified slot in a
- * folio queue segment.
- */
-static inline bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot)
-{
-	return test_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_mark2: Set the second mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Set the second mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_mark2(struct folio_queue *folioq, unsigned int slot)
-{
-	set_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_unmark2: Clear the second mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Clear the second mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_unmark2(struct folio_queue *folioq, unsigned int slot)
-{
-	clear_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_append: Add a folio to a folio queue segment
- * @folioq: The segment to add to
- * @folio: The folio to add
- *
- * Add a folio to the tail of the sequence in a folio queue segment, increasing
- * the occupancy count and returning the slot number for the folio just added.
- * The folio size is extracted and stored in the queue and the marks are left
- * unmodified.
- *
- * Note that it's left up to the caller to check that the segment capacity will
- * not be exceeded and to extend the queue.
- */
-static inline unsigned int folioq_append(struct folio_queue *folioq, struct folio *folio)
-{
-	unsigned int slot = folioq->vec.nr++;
-
-	folioq->vec.folios[slot] = folio;
-	folioq->orders[slot] = folio_order(folio);
-	return slot;
-}
-
-/**
- * folioq_append_mark: Add a folio to a folio queue segment
- * @folioq: The segment to add to
- * @folio: The folio to add
- *
- * Add a folio to the tail of the sequence in a folio queue segment, increasing
- * the occupancy count and returning the slot number for the folio just added.
- * The folio size is extracted and stored in the queue, the first mark is set
- * and and the second and third marks are left unmodified.
- *
- * Note that it's left up to the caller to check that the segment capacity will
- * not be exceeded and to extend the queue.
- */
-static inline unsigned int folioq_append_mark(struct folio_queue *folioq, struct folio *folio)
-{
-	unsigned int slot = folioq->vec.nr++;
-
-	folioq->vec.folios[slot] = folio;
-	folioq->orders[slot] = folio_order(folio);
-	folioq_mark(folioq, slot);
-	return slot;
-}
-
-/**
- * folioq_folio: Get a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the folio in the specified slot from a folio queue segment.  Note
- * that no bounds check is made and if the slot hasn't been added into yet, the
- * pointer will be undefined.  If the slot has been cleared, NULL will be
- * returned.
- */
-static inline struct folio *folioq_folio(const struct folio_queue *folioq, unsigned int slot)
-{
-	return folioq->vec.folios[slot];
-}
-
-/**
- * folioq_folio_order: Get the order of a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the order of the folio in the specified slot from a folio queue
- * segment.  Note that no bounds check is made and if the slot hasn't been
- * added into yet, the order returned will be 0.
- */
-static inline unsigned int folioq_folio_order(const struct folio_queue *folioq, unsigned int slot)
-{
-	return folioq->orders[slot];
-}
-
-/**
- * folioq_folio_size: Get the size of a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the size of the folio in the specified slot from a folio queue
- * segment.  Note that no bounds check is made and if the slot hasn't been
- * added into yet, the size returned will be PAGE_SIZE.
- */
-static inline size_t folioq_folio_size(const struct folio_queue *folioq, unsigned int slot)
-{
-	return PAGE_SIZE << folioq_folio_order(folioq, slot);
-}
-
-/**
- * folioq_clear: Clear a folio from a folio queue segment
- * @folioq: The segment to clear
- * @slot: The folio slot to clear
- *
- * Clear a folio from a sequence in a folio queue segment and clear its marks.
- * The occupancy count is left unchanged.
- */
-static inline void folioq_clear(struct folio_queue *folioq, unsigned int slot)
-{
-	folioq->vec.folios[slot] = NULL;
-	folioq_unmark(folioq, slot);
-	folioq_unmark2(folioq, slot);
-}
-
-#endif /* _LINUX_FOLIO_QUEUE_H */
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 08b7b7e643a3..43696fc67a2a 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -457,8 +457,6 @@ void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
 			   unsigned long long fpos, struct bvecq **_bvecq_head,
 			   iov_iter_extraction_t extraction_flags, gfp_t gfp);
-size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
-			size_t max_size, size_t max_segs);
 void netfs_prepare_write_failed(struct netfs_io_subrequest *subreq);
 void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error);
 
diff --git a/include/linux/rolling_buffer.h b/include/linux/rolling_buffer.h
deleted file mode 100644
index 1e37750e4ff8..000000000000
--- a/include/linux/rolling_buffer.h
+++ /dev/null
@@ -1,64 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/* Rolling buffer of folios
- *
- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells at redhat.com)
- */
-
-#ifndef _ROLLING_BUFFER_H
-#define _ROLLING_BUFFER_H
-
-#include <linux/folio_queue.h>
-#include <linux/uio.h>
-
-/*
- * Rolling buffer.  Whilst the buffer is live and in use, folios and folio
- * queue segments can be added to one end by one thread and removed from the
- * other end by another thread.  The buffer isn't allowed to be empty; it must
- * always have at least one folio_queue in it so that neither side has to
- * modify both queue pointers.
- *
- * The iterator in the buffer is extended as buffers are inserted.  It can be
- * snapshotted to use a segment of the buffer.
- */
-struct rolling_buffer {
-	struct folio_queue	*head;		/* Producer's insertion point */
-	struct folio_queue	*tail;		/* Consumer's removal point */
-	struct iov_iter		iter;		/* Iterator tracking what's left in the buffer */
-	u8			next_head_slot;	/* Next slot in ->head */
-	u8			first_tail_slot; /* First slot in ->tail */
-};
-
-/*
- * Snapshot of a rolling buffer.
- */
-struct rolling_buffer_snapshot {
-	struct folio_queue	*curr_folioq;	/* Queue segment in which current folio resides */
-	unsigned char		curr_slot;	/* Folio currently being read */
-	unsigned char		curr_order;	/* Order of folio */
-};
-
-/* Marks to store per-folio in the internal folio_queue structs. */
-#define ROLLBUF_MARK_1	BIT(0)
-#define ROLLBUF_MARK_2	BIT(1)
-
-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
-			unsigned int direction, gfp_t gfp);
-int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp);
-ssize_t rolling_buffer_load_from_ra(struct rolling_buffer *roll,
-				    struct readahead_control *ractl,
-				    struct folio_batch *put_batch);
-ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
-					 struct readahead_control *ractl,
-					 unsigned int rreq_id);
-ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
-			      unsigned int flags, gfp_t gfp);
-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll);
-void rolling_buffer_clear(struct rolling_buffer *roll);
-
-static inline void rolling_buffer_advance(struct rolling_buffer *roll, size_t amount)
-{
-	iov_iter_advance(&roll->iter, amount);
-}
-
-#endif /* _ROLLING_BUFFER_H */
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 7e25051dc1ea..b9558a0c2c1e 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -6742,8 +6742,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
 	/* amdgpu */
 	{ "amdgpu_vm_bo_map", 0x1 },
 	{ "amdgpu_vm_bo_unmap", 0x1 },
-	/* netfs */
-	{ "netfs_folioq", 0x1 },
 	/* xfs from xfs_defer_pending_class */
 	{ "xfs_defer_create_intent", 0x1 },
 	{ "xfs_defer_cancel_list", 0x1 },



From hch at infradead.org  Mon Aug 10 07:51:07 2026
From: hch at infradead.org (Christoph Hellwig)
Date: Mon, 10 Aug 2026 07:51:07 -0700
Subject: [PATCH v9 00/26] netfs: Keep track of folios in a segmented
 bio_vec[] chain
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <annlWwsCPjNgcKa-@infradead.org>

On Mon, Aug 10, 2026 at 03:47:17PM +0100, David Howells wrote:
> Hi Christian,
> 
> Could you add these patches to the VFS tree for next?  Though, given it may
> be a little late for that, could you at least take the fix patch at the
> front?
> 
> The patches get rid of folio_queue, rolling_buffer and ITER_FOLIOQ,
> replacing the folio queue construct used to manage buffers in netfslib with
> one based around a segmented chain of bio_vec arrays instead.  There are
> three main aims here:

Christian had a mail that a previous version was applied, but I think
this is a bad idea.  This is a giant series that also touches core code
with no review whatsoever.  And I repeatedly asked for:

 1) the core changes to be split out
 2) come up with users for this very core primitive in the hot path
    (iov_iter) that are not just for your rather niche use case

There's been absolutely no follow on to this, and we should not treat
the absolute core I/O path like this.

> 
>  (1) The kernel file I/O subsystem seems to be moving towards consolidating
>      on the use of bio_vec arrays, so embrace this by moving netfslib to
>      keep track of its buffers for buffered I/O in bio_vec[] form.
> 
>  (2) Netfslib already uses a bio_vec[] to handle unbuffered/DIO, so the
>      number of different buffering schemes used can be reduced to just a
>      single one.
> 
>  (3) Always send an entire filesystem RPC request message to a TCP socket
>      with single kernel_sendmsg() call as this is faster, more efficient
>      and doesn't require the use of corking as it puts the entire
>      transmission loop inside of a single tcp_sendmsg().
> 
> For the replacement of folio_queue, a segmented chain of bio_vec arrays
> rather than a single monolithic array is provided:
> 
> 	struct bvecq {
> 		struct bvecq		*next;
> 		struct bvecq		*prev;
> 		unsigned long long	fpos;
> 		refcount_t		ref;
> 		u32			priv;
> 		u16			nr_slots;
> 		u16			max_slots;
> 		enum bvecq_mem		mem_type:2;
> 		bool			inline_bv:1;
> 		bool			discontig:1;
> 		bool			from_pool:1;
> 		struct bio_vec		*bv;
> 		struct bio_vec		__bv[];
> 	};
> 
> The fields are:
> 
>  (1) next, prev - Link segments together in a list.  I want this to be
>      NULL-terminated linear rather than circular to make it possible to
>      arbitrarily glue bits on the front.
> 
>  (2) fpos, discontig - Note the current file position of the first byte of
>      the segment and whether this bvecq is discontiguous with the previous.
>      When accessing the pagecache to clear flags/locks, the fpos can be
>      used to look up folios by file position rather than by finding those
>      folios from the info stored in the bio_vecs.
> 
>      When the file position is relevant, the model I'm working with is that
>      all the segments pointed to by a single bvecq must represent
>      contiguous data, but adjacent bvecqs within a chain need not be
>      contiguous.  This allows a bvecq chain to be used to provide bufferage
>      for a sparse read or write RPC such as can be done with Ceph.
> 
>      If a bvecq segment is not contiguous with the previous one,
>      ->discontig should be set (this is technically redundant if one keeps
>      track of the fpos as a bvecq chain is processed).
> 
>      Note that the beginning and end file positions in a segment need not
>      be aligned to any filesystem block size.
> 
>  (3) ref - Refcount.  Each bvecq keeps a ref on the next.  I'm not sure
>      this is entirely necessary, but it makes sharing slices easier.
> 
>  (4) priv - Private data for the owner.  Dispensible; currently only used
>      for storing a debug ID for tracing in a patch not included here.
> 
>  (5) max_slots, nr_slots.  The size of bv[] and the number of slots used.
>      I've assumed a maximum of 65535 bio_vecs in the array (which would
>      represent a ~1MiB allocation).
> 
>  (6) bv, __bv, inline_bv.  bv points to the bio_vec[] array handled by
>      this segment.  This may begin at __bv and if it does inline_bv should
>      be set (otherwise it's impossible to distinguish a separately
>      allocated bio_vec[] that follows immediately by coincidence).
> 
>  (7) mem_type.  Indicates how the memory attached to the bio_vecs should be
>      disposed of when the bvecq is destroyed.  It can be one of:
> 
> 	BVECQ_MEM_EXTERNAL	- Externally tracked ref; don't put
> 	BVECQ_MEM_PAGECACHE	- Pagecache; must be put
> 	BVECQ_MEM_GUP		- Pinned by from GUP; needs unpin
> 	BVECQ_MEM_ALLOCED	- Plain alloc'd pages; can be mempooled
> 
>      [!] I'm not sure that this is a good name for this member or for the
>      	 enum values.
> 
>  (8) from_pool.  Set if the bvecq is allocated from netfslib's mempool and
>      should be freed to it.
> 
> I've also defined an iov_iter iterator type ITER_BVECQ to walk this sort of
> construct so that it can be passed directly to sendmsg() or block-based DIO
> (as cachefiles does).
> 
> 
> This series makes the following changes to netfslib:
> 
>  (1) Remove the writethrough code as the locking is really tricky to get
>      right and it looks like it could deadlock with Ceph if snapshots are
>      used.
> 
>  (2) The folio_queue chain used to hold folios for buffered I/O is replaced
>      with a bvecq chain.  Each bio_vec then holds (a portion of) one folio.
>      Each bvecq holds a contiguous sequence of folios, but adjacent bvecqs
>      in a chain may be discontiguous.
> 
>  (3) For unbuffered/DIO, the source iov_iter is extracted into a bvecq
>      chain.
> 
>  (4) An abstract position representation ('bvecq_pos') is created that can
>      used to hold a position in a bvecq chain.  For the moment, this takes
>      a ref on the bvecq it points to, but that may be excessive.
> 
>  (5) Buffer tracking is managed with three cursors:  The load_cursor, at
>      which new folios are added as we go; the dispatch_cursor, at which new
>      subrequests' buffers start when they're created; and the
>      collect_cursor, the point at which folios are being unlocked.
> 
>      Not all cursors are necessarily needed in all situations and during
>      buffered writeback, we need a dispatch cursor per stream (one for the
>      network filesystem and one for the cache).
> 
>  (6) ->prepare_read(), buffer setting up and ->issue_read() are merged, as
>      are the write variants, with the filesystem calling back up to
>      netfslib to prepare its buffer.  This simplifies the process of
>      setting up a subrequest.  It may even make sense to have the
>      filesystem allocate the subrequest.
> 
>  (7) Retry dispatch tracking is added to netfs_io_request so that the
>      buffer preparation functions can find it.  Retry requires an
>      additional buffer cursor.
> 
>  (8) Netfslib dispatches I/O by accumulating enough bufferage to dispatch
>      at least one subrequest, then looping to generate as many as the
>      filesystem wants to (they may be limited by other constraints,
>      e.g. max RDMA segment count or negotiated max size).  This loop could
>      be moved down into the filesystem.  A new method is provided by which
>      netfslib can ask the filesystem to provide an estimate of the data
>      that should be accumulated before dispatch begins.
> 
>  (9) Reading from the cache is now managed by querying the cache to provide
>      a list of the next two data extents within the cache.
> 
> (10) AFS directories are switched to using a bvecq rather than a
>      folio_queue to hold their contents.
> 
> (11) CIFS is switch to using a bvecq rather than a folio_queue for holding
>      a temporary encryption buffer.
> 
> (12) CIFS RDMA is given the ability to extract ITER_BVECQ and support for
>      extracting ITER_FOLIOQ is removed.
> 
> (13) All the folio_queue and rolling_buffer code is removed.
> 
> Cachefiles is also modified:
> 
>  (1) On-demand mode for Erofs is removed.
> 
>  (2) The object type in the cachefiles file xattr is now correctly set to
>      CACHEFILES_CONTENT_{SINGLE,ALL,BACKFS_MAP} rather than just being 0,
>      to indicate whether we have a single monolithic blob, all the data up
>      to cache i_size with no holes or a sparse file with the data mapped by
>      the backing file system (as currently upstream).
> 
>  (3) For "ALL" type files, the cache's i_size is used to track how much
>      data is saved in the cache and no longer bears any relation to the
>      netfs i_size.  The actual object size is stored in the xattr.
> 
>  (4) For most typical files which are contiguous and written progressively,
>      the object type is now set to "ALL".  For anything else, cachefiles
>      uses SEEK_DATA/HOLE to find extent outlines at before (this is the
>      current behaviour and needs to be fixed, but in a separate set of
>      patches as it's not trivial).
> 
>  (5) Preset the xattr on a cachefile to try to avoid having to deal with
>      ENOSPC from setxattr when committing the object.
> 
> Two further things that I'm working on (but not in this branch) are:
> 
>  (1) Make it so that a filesystem can be given a copy of a subchain which
>      it can then tack header and trailer protocol elements upon to form a
>      single message (I have this working for cifs) and even join copies
>      together with intervening protocol elements to form compounds.
> 
>  (2) Make it so that a filesystem can 'splice' out the contents of the TCP
>      receive queue into a bvecq chain.  This allows the socket lock to be
>      dropped much more quickly and the copying of data read to the
>      destination buffers to happen without the lock.  I have this working
>      for cifs too.  Kernel recvmsg() doesn't then block kernel sendmsg()
>      for anywhere near as long.
> 
> There are also some things I want to consider for the future:
> 
>  (1) Create one or more batched iteration functions to 'unlock' all the
>      folios in a bio_vec[], where 'unlock' is the appropriate action for
>      ending a read or a write.  Batching should hopefully also improve the
>      efficiency of wrangling the marks on the xarray.  Very often these
>      marks are going to be represented by contiguous bits, so there may be
>      a way to change them in bulk.
> 
>  (2) Rather than walking the bvecq chain to get each individual folio out
>      via bv_page, use the file position stored on the bvecq and the sum of
>      bv_len to iterate over the appropriate range in i_pages.
> 
>  (3) Change iov_iter to store the initial starting point and for
>      iov_iter_revert() to reset to that and advance.  This would (a) help
>      prevent over-reversion and (b) dispense with the need for a prev
>      pointer.
> 
>  (4) Use bvecq to replace scatterlist.  One problem with replacing
>      scatterlist is that crypto drivers like to glue bits on the front of
>      the scatterlists they're given (something trivial with that API) - and
>      this is one way to achieve it.
> 
> The patches can also be found here:
> 
> 	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-next
> 
> Thanks,
> David
> 
> Changes
> =======
> ver #9)
> - Fixed a number of issues reported by Sashiko[7]:
>   - Fixed netfs_unlock_abandoned_read_pages() to make sure the refs are
>     dropped before unlocking.
>   - Fixed iov_iter_extract_bvecq_pages() to immediately return 0 if
>     there are no pages to be extracted from the iterator (e.g. if maxpages
>     is 0).
>   - Pass a flag into bvecq_alloc_one() to indicate if we can access
>     writeback memory pools rather than trying to divine this from gfp.
>   - Fixed bvecq_alloc_one() to use the same gfp adjustments as
>     mempool_alloc() when prospectively allocating what the caller asked for
>     during writeback.
>   - Simplified the bvecq-based rolling buffer in writeback by always adding
>     whole folios and then setting the offset appropriately.  The offset and
>     length of the part added can then be aligned at that point when
>     tracking the new folio in the cache stream rather than copying and
>     rounding out later.
>   - Changed the writeback loop to ignore ENOMEM.  There are now backup
>     memory pools for writeback.
>   - Simplified the writeback loop to update stream tracking as folios are
>     added and to flush at that point too if needed.
>   - Removed the warning given by netfs_extract_iter() if max_len or
>     max_pages is 0.
>   - Made netfs_extract_iter() set the fpos on the blank bvecq it allocates
>     in the alloc_empty path.
>   - Fixed netfs_writeback_single() to test NETFS_RREQ_UPLOAD_TO_SERVER
>     rather than test-and-setting it (nothing currently wants to
>     writeback-single to the server anyway, only to the cache).
>   - Removed cachefiles ondemand pincount.
>   - Fixed netfs_collect_write_results() to get the cancel/data flags to
>     netfs_collect_write_results() the right way round.
>   - Fixed netfs_writeback_unlock_folios() to just warn and oops in the
>     unlikely event that it sees a NULL (ie. already unlocked) folio
>     pointer.
>   - Moved some common initialisation of fields to rreq->start to
>     netfs_alloc_request().
>   - Fixed bvecq_load_from_ra() to set bq->fpos.
>   - Fixed read progress reporting to use an offset from rreq->start
>     (unchanging) rather than trying to use rreq->clean_to (tearable) as a
>     base.
> 
> ver #8)
> - Removed the fix patches to their own branches and posted separately.
> - Rebased on v7.2-rc6 as a number of fixes went in.
> - Fixed cachefiles_collect_write() to handle cres->cache_priv2 (file) being
>   NULL due to failure to create an object (e.g. due to ENOSPC).
> - Added a patch to cachefiles to preset the xattr on a cachefiles to avoid
>   dealing with ENOSPC from setxattr when committing an object.
> - Fixed a number of bugs reported by Sashiko[6]:
>   - Fixed bvecq_alloc_buffer2() to break out of the loop if pre_slots > 0
>     and size == 0.
>   - Made netfs_extract_iter() limit max_pages to protect iov_iter_npages().
>   - Remove include/uapi/linux/cachefiles.h when ondemand mode is removed.
>   - Remove more ondemand-related trace bits.
>   - Remove ondemand-related BPF bits.
>   - Fixed a missing ENOMEM check in netfs_pgpriv2_begin_copy_to_cache().
>   - Fixed netfs_retry_read_subrequests() to unset dispatch_cursor on the
>     "abandon_after" error path.
>   - Fixed netfs_issue_write() to clear stream->construct if a subreq is
>     terminated due to ENOMEM.
>   - Fixed netfs_collect_read_results() to unpause the app if it decides to
>     abandon the read.
>   - Fixed cachefiles_issue_read() to not wait for the object state to match
>     FSCACHE_WANT_READ as cachefiles_query_occupancy() should have done that
>     already.
>   - Fixed netfs_retry_write_stream() to handle superfluous subreqs
>     correctly (borrowing from read retry).
> 
> ver #7)
> - Rebased on v7.2-rc4.
> - Added three fix patches to the front from Yichong Chen that conflict with
>   other patches in the series.
> - Fixed a number of bugs reported by Sashiko[5].
>   - Fixed double page put in iov_kunit_destroy_bvecq().
>   - Fixed bvecq_alloc_buffer2() to handle pre_slots>0 and size==0
>     correctly.
>   - Fixes bvecq_expand_buffer() to use a barrer to set tail->nr_slots.
>   - Added functions to insert barriers when reading or setting bvecq::next
>     and fixed some places to use them.
>   - Fixed more meta block mapping in afs_edit_dir_remove().
>   - Added a patch to fix missing unmap in afs_dir_search_bucket().
>   - Removed the writethrough stat counter as the code gets removed.
>   - Cleaned up the cachefiles-ondemand removal patch:
>     - Fix removal of code that's conditional on !ONDEMAND.
>     - Removed msg_id_next field.
>     - Removed ondemand trace elements.
>     - Removed documentation.
>   - Fixed cachefiles_resize_cookie() to only consider a resize a shrink if
>     the new size is less than the object size.
>   - Fixed cachefiles_resize_cookie() to update object->object_size on
>     expansion.
>   - Add a comment on exclusion in cachefiles_collect_write().
>   - Fixed netfs_single_dispatch_read() to allow for the cache to have
>     rounded out the stored data to DIO size.
>   - Fixed netfs_prepare_read_iterator() to clamp subreq len to sreq_max_len
>     (obsoleted by last patch).
>   - Changed write subreq collection to never retry cache writes.  Shouldn't
>     see them in the retry path either then.
>   - Fixed netfs_pgpriv2_unlock_copied_folios() to return made_progress
>     instead of false if we hit the end of the bvecq chain.
>   - Modified buffered read collection to abandon the rest of a read request
>     if a subreq over-reads as the buffer may have been corrupted.  The read
>     termination function no longer winds subreq->transferred back.
>   - Fixed netfs_issue_read() to call the right termination func in the fail
>     path.
>   - Fixed netfs_dispatch_unbuffered_reads(), netfs_single_dispatch_read()
>     and netfs_retry_read_subrequests() to handle failed ->issue_read().
>   - Fixed afs_issue_read() to return after calling afs_end_read() as this
>     op is then released.
>   - Fixed cifs_issue_write() to always release credits on failure.
>   - Fixed cifs_issue_read() to not release credits twice.
>   - Fixed netfs_issue_writes() to advance stream->issued_to and
>     stream->dispatch_cursor on early failure of ->issue_write().
>   - Fixed netfs_issue_read() to only call netfs_all_subreqs_queued() if
>     stream->buffered is zero.
>   - Fixed ceph_netfs_issue_op_inline() to do a retry if file got uninlined
>     by someone else.
>   - Fixed ceph_netfs_issue_op_inline() to not return an error directly
>     once it's got a prepared buffer.
>   - Fixed nfs_netfs_issue_read() to return an int.
> 
> ver #6)
> - Rebased on v7.2-rc3.
> - Added a patch to remove erofs on-demand support from cachefiles.
> - Added a patch to remove the writethrough code as it's really tricky to
>   get the locking right and it looks like can deadlock itself with Ceph.
> - Fixed a number of bugs reported by Sashiko[4].
>   - Fixed iov_iter_single_seg_count(), iov_iter_alignment_bvecq() and
>     iov_npages_bvecq() to handle zero-length segments and segments with no
>     stuff remaining.
>   - Fixed bvecq_extract() to use GFP_NOFS rather than GFP_KERNEL.
>   - Fixed bvecq_alloc_buffer2()'s use of alloc_pages_bulk().
>   - Fixed netfs_extract_iter() to break out of the inner loop if max_pages
>     hits zero.
>   - Added a patch to fix meta block mapping in afs_edit_dir_remove().
>   - Fixed afs_init_new_symlink() to clear the tail of the symlink page.
>   - Fixed cachefiles_begin_operation() to deal with a NULL file.
>   - Fixed slot check in netfs_read_gaps() to be >=, not >.
>   - Fixed netfs_pgpriv2_copy_folio() to update creq->last_end.
>   - Fixed netfs_read_single() to return -EIO if the buffer isn't big
>     enough.
>   - Add a comment to netfs_write_folio() to indicate that truncate must
>     not run concurrently with writeback and must exclude/wait for it.
>   - In netfs_read_subreq_terminated(), move the netfs_sreq_trace_too_much
>     trace line earlier before ->transferred is clobbered.
>   - In netfs_read_subreq_terminated(), make the transfer-too-long case
>     avoid retrying and just end with EIO (in case data further along in the
>     buffer got corrupted).
>   - Fixed netfs_collect_read_results() if it sees front->transferred >
>     front->len, to just use front->len instead.
>   - Fixed ceph_netfs_issue_op_inline() to prep the buffer before setting
>     NETFS_SREQ_HIT_EOF so that subreq->content is set.
>   - Alter netfs_read_to_pagecache() to not round up reading from
>     zero-containing cache granules as no I/O is required.
>   - Made ->issue_write() return an error so that errors before any buffer
>     is sliced off don't result in an infinite loop in netfs_issue_writes().
>   - Made ->issue_read() return an error also for similiar reasons.
>   - In netfs_writepages(), handle -ENOMEM from bvecq prealloc before
>     caling netfs_queue_wb_folio() rather than doing it on the combined
>     error return of that and netfs_issue_streams() which can result in a
>     double unlock.
>   - Fix netfs_writeback_single() to write nothing if len == 0.
> 
> ver #5)
> - Rebased on v7.2-rc2 as that has a bunch of outstanding netfs and afs
>   bugfixes included.
> 
> ver #4)
> - Fixed a number of bugs reported by Sashiko[3].
>   - Added a patch to fix an underflow in iov_iter_extract_xarray_pages().
>   - Added a patch to fix alloc failure in iov_iter_extract_bvec_pages().
>   - Added a patch to remove an unused var in kunit code.
>   - Added a patch to fix the folio offset in extract_xarray_to_sg().
>   - Added a patch to fix the exclusion over writeback to make it cover
>     collection too.
>   - Fixed double fput() in cachefiles.
>   - Fixed the collection of cache writes to handle cancellation better.
>   - Fixed iterate_bvecq() to skip bvecq structs with nr_slots==0.
>   - Add a comment into iterate_bvecq() that a slot with bv_len>0 must have a
>     valid bv_page.
>   - Fixed iov_iter_bvecq_advance(), iov_iter_bvecq_revert(),
>     iter_count_bvecq_pages(), iov_iter_extract_bvecq_pages() and
>     extract_bvecq_to_sg() to correctly handle empty bvecqs.
>   - Fixed extract_bvecq_to_sg() to be limited by iter->count.
>   - Fixed bvecq_expand_buffer() to take an unsigned size param.
>   - Fixed bvecq_expand_buffer() to not mix memory types in alloc'd bvecqs.
>   - Fixed bvecq_shorten_buffer() occasional retention of zero-length slots.
>   - Fixed slot validity check polarity in bvecq_pos_advance(); also don't use
>     inner loop otherwise break then exits the wrong loop.
>   - Fixed bvecq_zero(), bvecq_slice() and bvecq_extract to use a barrier when
>     checking bq->nr_slots.
>   - Restructured bvecq_zero() to be similar to bvecq_pos_advance().
>   - Fixed an off-by-one error in bvecq_pos_step() and added a missing slot
>     reset.
>   - Fixed a break in netfs_extract_iter() that should have been a goto.
>   - Fixed netfs_extract_iter() to limit number of pages extracted to remnant
>     of max_pages.
>   - Fixed an uninit var in afs_do_read_symlink().
>   - Fixed netfs_read_gaps() to fill a multipart bvecq chain correctly.
>   - Fixed netfs_dispatch_unbuffered_reads() to initialise collect_cursor as
>     netfs_rreq_assess_dio() uses it to flush the data read.
>   - Fixed netfs_extract_iter() to init the slot counter outside the extract
>     loop to avoid overwriting already loaded slots.
>   - Fixed callers of bvecq_delete_spent() to update bvecq_pos::slot before
>     calling.
>   - Fixed netfs_reissue_write() to make sure subreq->content is unset before
>     setting.
>   - Altered netfs_extract_iter() to free any allocated bvecq chain if no pages
>     were extracted and an error occurred  (and to initialise the return
>     pointer to NULL).  Also, made it return an empty bvecq if nothing was
>     extracted, but no error occurred.
>   - Fixed ceph_netfs_issue_read() to just return if
>     ceph_netfs_issue_op_inline() returns anything other than 1 to avoid a
>     double termination.
>   - Fixed ceph_netfs_issue_read() to do the size calculation in the right
>     order to avoid the op expanding to larger than the buffer.
>   - Fixed netfs_issue_read(), in the NETFS_FILL_WITH_ZEROES case, to deduct
>     subreq->len from stream->buffered rather than just setting it to 0.
>   - Fixed netfs_perform_write() to put the folio if netfs_advance_writethrough()
>     fails.
>   - Restored the old ->prepare_write op specifically for
>     fscache_write_to_cache() which is still used by Ceph.
>   - Fixed undefined return in netfs_pgpriv2_issue_stream().
>   - Fixed netfs_collect_write_results() to try to make sure a request isn't
>     left paused if there are no further server-bound subreqs.
>   - Fixed netfs_queue_wb_folio() to redirty the folio before unlocking it if
>     it can't allocate a bvecq.
>   - Fixed netfs_writepages() to cancel the pagecache iteration after ENOMEM.
>   - Fixed netfs_advance/end_writethrough() to advance the dispatch cursor.
>   - Fixed netfs_retry_read_subrequests() to use barriers when walking
>     stream->subrequests as the app may add another subreq before pausing.
>   - Fixed netfs_prepare_write_retry_buffer() to use ->retry_start and
>     ->retry_buffered rather than ->issue_from and ->buffered.
>   - Fixed netfs_retry_write_stream() to use barriers when walking
>     stream->subrequests as the app may add another subreq before pausing.
>   - Fixed netfs_retry_write_stream() to check the correct length when adding
>     additional subreqs.
>   - Fixed nfs_netfs_issue_read() to set -ENOMEM, not 0, on alloc failure.
>   - Fixed nfs_netfs_issue_read() to only terminate the subreq once.
>   - Fixed cifs_issue_read() to release the credits if cifs_reopen_file()
>     fails.
> - Rebased on v7.1.
> 
> ver #3)
> - Rebased to -rc7 as the patches wouldn't apply for Christian.
> - Prepended a fix for a warning from generic/464 (the problem also exists
>   upstream, just not the warning).
> - Renamed kmap_local_bvec() to bvec_kmap_partial() as requested by
>   Christoph.
> - Adjusted smbdirect patch descriptions as requested by Stefan Metzmacher.
> 
> ver #2)
> - Fixed a number of bugs reported by Sashiko[1].
> - Split a bunch of fixes out and posted them separately[2].
> 
> [1] https://sashiko.dev/#/patchset/20260326104544.509518-1-dhowells%40redhat.com
> [2] https://lore.kernel.org/linux-fsdevel/20260512-infozentrum-becher-7f86c47c96c8 at brauner/T/#t
> [3] https://sashiko.dev/#/patchset/20260608145432.681865-1-dhowells%40redhat.com
> [4] https://sashiko.dev/#/patchset/20260706153408.1231650-1-dhowells%40redhat.com
> [5] https://sashiko.dev/#/patchset/20260716103030.3065561-1-dhowells%40redhat.com
> [6] https://sashiko.dev/#/patchset/20260722130218.78958-1-dhowells%40redhat.com
> [7] https://sashiko.dev/#/patchset/20260804100224.2748935-1-dhowells%40redhat.com
> 
> David Howells (25):
>   netfs: Fix read progress reporting
>   mm: Make readahead store folio count in readahead_control
>   netfs: Bulk load the readahead-provided folios up front
>   Add a function to kmap one page of a multipage bio_vec
>   iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
>   iov_iter: Add a segmented queue of bio_vec[]
>   netfs: Add some tools for managing bvecq chains
>   netfs: Make mempool available for bvecq
>   netfs: Add a function to extract from an iter into a bvecq
>   afs: Use a bvecq to hold dir content rather than folioq
>   cifs: Use a bvecq for buffering instead of a folioq
>   smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
>   netfs: Remove the writethrough code
>   cachefiles: Don't rely on backing fs storage map for most use cases
>   netfs: Add the cache object ID to netfs_read/write tracepoints
>   netfs: Switch to using bvecq rather than folio_queue and
>     rolling_buffer
>   smbdirect: Remove support for ITER_FOLIOQ from
>     smbdirect_map_sges_from_iter()
>   netfs: Remove netfs_alloc/free_folioq_buffer()
>   netfs: Remove netfs_extract_user_iter()
>   iov_iter: Remove ITER_FOLIOQ
>   netfs: Remove folio_queue and rolling_buffer
>   netfs: Simplify read abandonment
>   netfs: Check for too much data being read
>   netfs: Combine prepare and issue ops and grab the buffers on request
>   cachefiles: Preset the state xattr when creating a new file
> 
> Gao Xiang (1):
>   cachefiles,netfs: sunset ondemand mode
> 
>  Documentation/core-api/folio_queue.rst        | 209 ----
>  Documentation/core-api/index.rst              |   1 -
>  .../filesystems/caching/cachefiles.rst        | 179 ----
>  Documentation/filesystems/netfs_library.rst   |   2 +-
>  fs/9p/vfs_addr.c                              |  61 +-
>  fs/afs/dir.c                                  |  40 +-
>  fs/afs/dir_edit.c                             |  43 +-
>  fs/afs/dir_search.c                           |  33 +-
>  fs/afs/file.c                                 |  31 +-
>  fs/afs/fsclient.c                             |   8 +-
>  fs/afs/inode.c                                |   2 +-
>  fs/afs/internal.h                             |  14 +-
>  fs/afs/symlink.c                              |  37 +-
>  fs/afs/write.c                                |  36 +-
>  fs/afs/yfsclient.c                            |   6 +-
>  fs/cachefiles/Kconfig                         |  12 -
>  fs/cachefiles/Makefile                        |   1 -
>  fs/cachefiles/daemon.c                        |  96 +-
>  fs/cachefiles/interface.c                     |  98 +-
>  fs/cachefiles/internal.h                      | 147 +--
>  fs/cachefiles/io.c                            | 602 ++++++++---
>  fs/cachefiles/namei.c                         |  42 +-
>  fs/cachefiles/ondemand.c                      | 761 --------------
>  fs/cachefiles/xattr.c                         |  84 +-
>  fs/ceph/Kconfig                               |   1 +
>  fs/ceph/addr.c                                | 123 ++-
>  fs/netfs/Kconfig                              |   3 +
>  fs/netfs/Makefile                             |   4 +-
>  fs/netfs/buffered_read.c                      | 526 ++++++----
>  fs/netfs/buffered_write.c                     |  59 +-
>  fs/netfs/bvecq.c                              | 808 +++++++++++++++
>  fs/netfs/direct_read.c                        | 115 +--
>  fs/netfs/direct_write.c                       | 162 +--
>  fs/netfs/fscache_io.c                         |   4 +-
>  fs/netfs/internal.h                           | 109 +-
>  fs/netfs/iterator.c                           | 394 +++-----
>  fs/netfs/main.c                               |  13 +-
>  fs/netfs/misc.c                               | 168 +---
>  fs/netfs/objects.c                            |  57 +-
>  fs/netfs/read_collect.c                       | 377 ++++---
>  fs/netfs/read_pgpriv2.c                       | 204 ++--
>  fs/netfs/read_retry.c                         | 261 ++---
>  fs/netfs/read_single.c                        | 184 ++--
>  fs/netfs/rolling_buffer.c                     | 226 -----
>  fs/netfs/stats.c                              |  10 +-
>  fs/netfs/write_collect.c                      | 245 +++--
>  fs/netfs/write_issue.c                        | 935 ++++++++----------
>  fs/netfs/write_retry.c                        | 187 ++--
>  fs/nfs/Kconfig                                |   1 +
>  fs/nfs/fscache.c                              |  24 +-
>  fs/smb/client/cifsglob.h                      |   2 +-
>  fs/smb/client/cifssmb.c                       |  13 +-
>  fs/smb/client/file.c                          | 145 +--
>  fs/smb/client/smb2ops.c                       |  84 +-
>  fs/smb/client/smb2pdu.c                       |  28 +-
>  fs/smb/client/transport.c                     |  15 +-
>  fs/smb/smbdirect/connection.c                 | 135 ++-
>  include/linux/bvec.h                          |  18 +
>  include/linux/bvecq.h                         | 363 +++++++
>  include/linux/folio_queue.h                   | 282 ------
>  include/linux/fscache.h                       |  17 +
>  include/linux/iov_iter.h                      |  87 +-
>  include/linux/netfs.h                         | 188 ++--
>  include/linux/pagemap.h                       |  10 +
>  include/linux/rolling_buffer.h                |  61 --
>  include/linux/uio.h                           |  17 +-
>  include/trace/events/cachefiles.h             | 209 +---
>  include/trace/events/netfs.h                  | 162 ++-
>  include/uapi/linux/cachefiles.h               |  68 --
>  kernel/bpf/btf.c                              |   9 -
>  lib/iov_iter.c                                | 559 ++++++-----
>  lib/scatterlist.c                             |  82 +-
>  lib/tests/kunit_iov_iter.c                    | 131 ++-
>  mm/readahead.c                                |   5 +
>  net/9p/client.c                               |   8 +-
>  75 files changed, 5134 insertions(+), 5309 deletions(-)
>  delete mode 100644 Documentation/core-api/folio_queue.rst
>  delete mode 100644 fs/cachefiles/ondemand.c
>  create mode 100644 fs/netfs/bvecq.c
>  delete mode 100644 fs/netfs/rolling_buffer.c
>  create mode 100644 include/linux/bvecq.h
>  delete mode 100644 include/linux/folio_queue.h
>  delete mode 100644 include/linux/rolling_buffer.h
>  delete mode 100644 include/uapi/linux/cachefiles.h
> 
---end quoted text---


From dhowells at redhat.com  Mon Aug 10 07:47:40 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:40 +0100
Subject: [PATCH v9 23/26] netfs: Simplify read abandonment
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-24-dhowells@redhat.com>

Currently, if one or more of the subrequests in a read request fails, the
read collection algorithm will attempt to salvage folios that are fully
downloaded but that span multiple subreqs, provided all of the contributory
subreqs succeeded, but this makes the algorithm quite complicated as a
subreq can contribute to multiple folios and a folio can be contributed to
by multiple subreqs.

Simplify this by just abandoning the rest of a read request once a
permanently failed subreq is hit.  This is what read_folio, DIO/unbuffered
read, gap filling, RMW and write preload all need to do; readahead is the
odd one out as it doesn't have any return other than unlocking folios.

With this change, even for readahead, the rest of the read is simply
abandoned; read() and suchlike will call ->read_folio() on each
non-uptodate folio to retry and retrieve the error.

Note that retryable failures still get retried by netfslib as part of the
request as before.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/read_collect.c | 190 ++++++++++++++++++++++++++++------------
 include/linux/netfs.h   |   1 +
 2 files changed, 137 insertions(+), 54 deletions(-)

diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 5d0a8a75ce26..cf7c36037edc 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -19,8 +19,8 @@
 #define MADE_PROGRESS	0x04	/* Made progress cleaning up a stream or the folio set */
 #define BUFFERED	0x08	/* The pagecache needs cleaning up */
 #define NEED_RETRY	0x10	/* A front op requests retrying */
-#define COPY_TO_CACHE	0x40	/* Need to copy subrequest to cache */
-#define ABANDON_SREQ	0x80	/* Need to abandon untransferred part of subrequest */
+#define COPY_TO_CACHE	0x20	/* Need to copy subrequest to cache */
+#define ABANDON_RREQ	0x40	/* Need to abandon the rest of a request */
 
 /*
  * Clear the unread part of an I/O request.
@@ -174,6 +174,78 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 	rreq->collect_cursor.slot = slot;
 }
 
+/*
+ * Abandon all remaining read results.  Once we've hit a permanent failure, we
+ * assume that the file is probably unusable.  In the event of readahead, in
+ * theory we might manage to read some data later in the file, that we could
+ * still unlock, but ->read_folio() will be called again.
+ */
+static void netfs_abandon_read_results(struct netfs_io_request *rreq)
+{
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	unsigned int notes = ABANDON_RREQ;
+
+	_enter("%llx-%llx", rreq->start, rreq->start + rreq->len);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_collect);
+	trace_netfs_collect(rreq);
+
+	if (rreq->origin == NETFS_READAHEAD ||
+	    rreq->origin == NETFS_READPAGE ||
+	    rreq->origin == NETFS_READ_FOR_WRITE)
+		notes |= BUFFERED;
+
+	/* Remove completed subrequests from the front of the stream and
+	 * advance the completion point.  We stop when we hit something that's
+	 * in progress.  The issuer thread may be adding stuff to the tail
+	 * whilst we're doing this.
+	 */
+	for (;;) {
+		struct netfs_io_subrequest *front;
+		unsigned long front_flags;
+
+		front = list_first_entry_or_null_acquire(&stream->subrequests,
+							 struct netfs_io_subrequest, rreq_link);
+		/* Read first subreq pointer before IN_PROGRESS flag. */
+		if (!front)
+			break;
+
+		front_flags = smp_load_acquire(&front->flags);
+		/* Order read of flags before read of anything else, such as error. */
+
+		/* Wait for each subreq to complete. */
+		if (test_bit(NETFS_SREQ_IN_PROGRESS, &front_flags)) {
+			notes |= HIT_PENDING;
+			break;
+		}
+
+		/* The subreq now belongs to us. */
+		if (!stream->failed) {
+			stream->failed = true;
+			stream->error = front->error;
+			rreq->error = front->error;
+			trace_netfs_rreq(rreq, netfs_rreq_trace_set_abandon);
+		}
+
+		stream->collected_to = front->start + front->len;
+		trace_netfs_sreq(front, netfs_sreq_trace_abandoned);
+
+		spin_lock(&rreq->lock);
+		list_del_init(&front->rreq_link);
+		spin_unlock(&rreq->lock);
+		netfs_put_subrequest(front, netfs_sreq_trace_put_abandon);
+	}
+
+	rreq->collected_to = stream->collected_to;
+	rreq->abandon_to = rreq->collected_to;
+	if (notes & BUFFERED)
+		netfs_read_unlock_folios(rreq, &notes);
+	else
+		rreq->cleaned_to = rreq->collected_to;
+
+	trace_netfs_collect_stream(rreq, stream);
+	trace_netfs_collect_state(rreq, rreq->collected_to, notes);
+}
+
 /*
  * Collect and assess the results of various read subrequests.  We may need to
  * retry some of the results.
@@ -193,6 +265,9 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 	trace_netfs_collect(rreq);
 
 reassess:
+	if (test_bit(NETFS_RREQ_ABANDON_REQ, &rreq->flags))
+		goto abandon_request;
+
 	if (rreq->origin == NETFS_READAHEAD ||
 	    rreq->origin == NETFS_READPAGE ||
 	    rreq->origin == NETFS_READ_FOR_WRITE)
@@ -210,7 +285,9 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 	/* Read first subreq pointer before IN_PROGRESS flag. */
 
 	while (front) {
+		unsigned long front_flags;
 		size_t transferred;
+		uoff_t unlock_at = rreq->start + rreq->progress_at;
 
 		trace_netfs_collect_sreq(rreq, front);
 		_debug("sreq [%x] %llx %zx/%zx",
@@ -221,25 +298,63 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 			stream->collected_to = front->start;
 		}
 
-		if (netfs_check_subreq_in_progress(front))
+		front_flags = smp_load_acquire(&front->flags);
+		/* Order read of flags before read of anything else, such as error. */
+
+		if (test_bit(NETFS_SREQ_FAILED, &front_flags))
+			goto abandon_request;
+		if (test_bit(NETFS_SREQ_IN_PROGRESS, &front_flags))
 			notes |= HIT_PENDING;
-		smp_rmb(); /* Read counters after IN_PROGRESS flag. */
+
 		transferred = READ_ONCE(front->transferred);
 
+		/* If we can collect the next folio from a pending op, do so,
+		 * but we should only do it if we don't otherwise need to wait
+		 * for completion.
+		 */
+		if ((notes & HIT_PENDING) &&
+		    (notes & BUFFERED) &&
+		    !test_bit(NETFS_SREQ_HIT_EOF, &front_flags) &&
+		    front->error == 0 &&
+		    transferred < front->len
+		    ) {
+			stream->collected_to = front->start + transferred;
+			rreq->collected_to = stream->collected_to;
+
+			if (test_bit(NETFS_SREQ_COPY_TO_CACHE, &front_flags))
+				notes |= COPY_TO_CACHE;
+			if (front->start + transferred >= unlock_at)
+				netfs_read_unlock_folios(rreq, &notes);
+		}
+
+		/* Stall if the front is still undergoing I/O. */
+		if (notes & HIT_PENDING)
+			break;
+
+		if (test_bit(NETFS_SREQ_NEED_RETRY, &front_flags)) {
+			stream->need_retry = true;
+			notes |= NEED_RETRY | MADE_PROGRESS;
+			break;
+		} else if (test_bit(NETFS_RREQ_SHORT_TRANSFER, &rreq->flags)) {
+			notes |= MADE_PROGRESS;
+		} else {
+			stream->transferred += transferred;
+			stream->transferred_valid = true;
+			if (front->transferred < front->len)
+				set_bit(NETFS_RREQ_SHORT_TRANSFER, &rreq->flags);
+			notes |= MADE_PROGRESS;
+		}
+
 		/* If we can now collect the next folio, do so.  We don't want
 		 * to defer this as we have to decide whether we need to copy
 		 * to the cache or not, and that may differ between adjacent
 		 * subreqs.
 		 */
 		if (notes & BUFFERED) {
-			uoff_t unlock_at = rreq->start + rreq->progress_at;
-
 			/* Clear the tail of a short read. */
-			if (!(notes & HIT_PENDING) &&
-			    front->error == 0 &&
-			    transferred < front->len &&
-			    (test_bit(NETFS_SREQ_HIT_EOF, &front->flags) ||
-			     test_bit(NETFS_SREQ_CLEAR_TAIL, &front->flags))) {
+			if (transferred < front->len &&
+			    (test_bit(NETFS_SREQ_HIT_EOF, &front_flags) ||
+			     test_bit(NETFS_SREQ_CLEAR_TAIL, &front_flags))) {
 				netfs_clear_unread(front);
 				transferred = front->transferred = front->len;
 				trace_netfs_sreq(front, netfs_sreq_trace_clear);
@@ -248,67 +363,28 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 			stream->collected_to = front->start + transferred;
 			rreq->collected_to = stream->collected_to;
 
-			if (test_bit(NETFS_SREQ_COPY_TO_CACHE, &front->flags))
+			if (test_bit(NETFS_SREQ_COPY_TO_CACHE, &front_flags))
 				notes |= COPY_TO_CACHE;
 
-			if (test_bit(NETFS_SREQ_FAILED, &front->flags)) {
-				rreq->abandon_to = front->start + front->len;
-				front->transferred = front->len;
-				transferred = front->len;
-				trace_netfs_rreq(rreq, netfs_rreq_trace_set_abandon);
-			}
 			if (front->start + transferred >= unlock_at ||
-			    test_bit(NETFS_SREQ_HIT_EOF, &front->flags))
+			    test_bit(NETFS_SREQ_HIT_EOF, &front_flags))
 				netfs_read_unlock_folios(rreq, &notes);
 		} else {
 			stream->collected_to = front->start + transferred;
 			rreq->collected_to = stream->collected_to;
 		}
 
-		/* Stall if the front is still undergoing I/O. */
-		if (notes & HIT_PENDING)
-			break;
-
-		if (test_bit(NETFS_SREQ_FAILED, &front->flags)) {
-			if (!stream->failed) {
-				stream->error = front->error;
-				rreq->error = front->error;
-				set_bit(NETFS_RREQ_FAILED, &rreq->flags);
-				stream->failed = true;
-			}
-			notes |= MADE_PROGRESS | ABANDON_SREQ;
-		} else if (test_bit(NETFS_SREQ_NEED_RETRY, &front->flags)) {
-			stream->need_retry = true;
-			notes |= NEED_RETRY | MADE_PROGRESS;
-			break;
-		} else if (test_bit(NETFS_RREQ_SHORT_TRANSFER, &rreq->flags)) {
-			notes |= MADE_PROGRESS;
-		} else {
-			if (!stream->failed) {
-				stream->transferred += transferred;
-				stream->transferred_valid = true;
-			}
-			if (front->transferred < front->len)
-				set_bit(NETFS_RREQ_SHORT_TRANSFER, &rreq->flags);
-			notes |= MADE_PROGRESS;
-		}
-
 		/* Remove if completely consumed. */
 		stream->source = front->source;
 		spin_lock(&rreq->lock);
 
 		remove = front;
-		trace_netfs_sreq(front,
-				 notes & ABANDON_SREQ ?
-				 netfs_sreq_trace_abandoned : netfs_sreq_trace_consumed);
+		trace_netfs_sreq(front, netfs_sreq_trace_consumed);
 		list_del_init(&front->rreq_link);
 		front = list_first_entry_or_null(&stream->subrequests,
 						 struct netfs_io_subrequest, rreq_link);
 		spin_unlock(&rreq->lock);
-		netfs_put_subrequest(remove,
-				     notes & ABANDON_SREQ ?
-				     netfs_sreq_trace_put_abandon :
-				     netfs_sreq_trace_put_done);
+		netfs_put_subrequest(remove, netfs_sreq_trace_put_done);
 	}
 
 	trace_netfs_collect_stream(rreq, stream);
@@ -337,6 +413,12 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 	_debug("retry");
 	netfs_retry_reads(rreq);
 	goto out;
+
+abandon_request:
+	set_bit(NETFS_RREQ_FAILED, &rreq->flags);
+	set_bit(NETFS_RREQ_ABANDON_REQ, &rreq->flags);
+	netfs_wake_rreq_flag(rreq, NETFS_RREQ_PAUSE, netfs_rreq_trace_unpause);
+	return netfs_abandon_read_results(rreq);
 }
 
 /*
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 43696fc67a2a..f277c187c408 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -281,6 +281,7 @@ struct netfs_io_request {
 #define NETFS_RREQ_FAILED		3	/* The request failed */
 #define NETFS_RREQ_RETRYING		4	/* Set if we're in the retry path */
 #define NETFS_RREQ_SHORT_TRANSFER	5	/* Set if we have a short transfer */
+#define NETFS_RREQ_ABANDON_REQ		6	/* Set if the request is to be abandoned */
 #define NETFS_RREQ_CACHE_STOP		8	/* Set to stop caching (ENOBUFS or error) */
 #define NETFS_RREQ_CACHE_ERROR		9	/* Set if we got an error from the cache */
 #define NETFS_RREQ_OFFLOAD_COLLECTION	12	/* Offload collection to workqueue */



From dhowells at redhat.com  Mon Aug 10 07:47:41 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:41 +0100
Subject: [PATCH v9 24/26] netfs: Check for too much data being read
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-25-dhowells@redhat.com>

Put in a check in read subreq termination to detect more data being read
for a subrequest than was requested.  In the event that this happens, abort
the rest of the read request on the basis that some of the read buffer may
have been corrupted and return -EIO.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/read_collect.c      | 23 +++++++++++++++++++++++
 include/trace/events/netfs.h |  1 +
 2 files changed, 24 insertions(+)

diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index cf7c36037edc..21eb7f74d66e 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -307,6 +307,14 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 			notes |= HIT_PENDING;
 
 		transferred = READ_ONCE(front->transferred);
+		if (unlikely(transferred > front->len)) {
+			/* Ugh...  A subreq overran its allotted length.  It
+			 * may have corrupted the read buffer.
+			 */
+			stream->error = -EIO;
+			stream->failed = true;
+			goto abandon_request;
+		}
 
 		/* If we can collect the next folio from a pending op, do so,
 		 * but we should only do it if we don't otherwise need to wait
@@ -627,6 +635,21 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq)
 		break;
 	}
 
+	/* If the subrequest read more than it was supposed to, abort
+	 * the request with EIO as we may have clobbered other parts
+	 * of the buffer that are already read.
+	 */
+	if (subreq->transferred > subreq->len) {
+		trace_netfs_sreq(subreq, netfs_sreq_trace_too_much);
+		__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
+		__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+		subreq->error = -EIO;
+		trace_netfs_failure(rreq, subreq, subreq->error, netfs_fail_read);
+		trace_netfs_rreq(rreq, netfs_rreq_trace_set_pause);
+		set_bit(NETFS_RREQ_PAUSE, &rreq->flags);
+		goto skip_error_checks;
+	}
+
 	/* Deal with retry requests, short reads and errors.  If we retry
 	 * but don't make progress, we abandon the attempt.
 	 */
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 071d20e80f13..723cb7315308 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -132,6 +132,7 @@
 	EM(netfs_sreq_trace_submit,		"SUBMT")	\
 	EM(netfs_sreq_trace_superfluous,	"SPRFL")	\
 	EM(netfs_sreq_trace_terminated,		"TERM ")	\
+	EM(netfs_sreq_trace_too_much,		"!TOOM")	\
 	EM(netfs_sreq_trace_wait_for,		"_WAIT")	\
 	EM(netfs_sreq_trace_write,		"WRITE")	\
 	EM(netfs_sreq_trace_write_skip,		"SKIP ")	\



From dhowells at redhat.com  Mon Aug 10 07:47:42 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:42 +0100
Subject: [PATCH v9 25/26] netfs: Combine prepare and issue ops and grab the buffers on request
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-26-dhowells@redhat.com>

Modify the way subrequests are generated in netfslib to try and simplify
the code.  The issue, primarily, is in writeback: the code has to create
multiple streams of write requests to disparate targets with different
properties (e.g. server and fscache), where not every folio needs to go to
every target (e.g. data just read from the server may only need writing to
the cache).

The current model in writeback, at least, is to go carefully through every
folio, preparing a subrequest for each stream when it was detected that
part of the current folio needed to go to that stream, and repeating this
within and across contiguous folios; then to issue subrequests as they
become full or hit boundaries after first setting up the buffer.  However,
this is quite difficult to follow - and makes it tricky to handle
discontiguous folios in a request.

This is changed such that netfs now accumulates buffers and attaches them
to each stream when they become valid for that stream, then flushes the
stream when a limit or a boundary is hit.  The issuing code in netfs then
loops around creating and issuing subrequests without calling a separate
prepare stage (though a function is provided to get an estimate of when
flushing should occur).  The filesystem (or cache) then gets to take a
slice of the master bvec chain as its I/O buffer for each subrequest,
including discontiguities if it can support a sparse/vectored RPC (as Ceph
can).

Similar-ish changes also apply to buffered read and unbuffered read and
write, though in each of those cases there is only a single contiguous
stream.  Though for buffered read this consists of interwoven requests from
multiple sources (server or cache).

To this end, netfslib is changed in the following ways:

 (1) ->prepare_xxx(), buffer selection and ->issue_xxx() are now collapsed
     together such that one ->issue_xxx() call is made with the subrequest
     defined to the maximum extent; the filesystem/cache then reduces the
     length of the subrequest and calls back to netfslib to grab a slice of
     the buffer, which may reduce the subrequest further if a maximum
     segment limit is set.  The filesystem/cache then dispatches the
     operation.

 (2) Retry buffer tracking is added to the netfs_io_request struct.  This
     is then selected by the subrequest retry counter being non-zero.

 (3) The use of iov_iter is pushed down to the filesystem.  Netfslib now
     provides the filesystem with a bvecq holding the buffer rather than an
     iov_iter.  The bvecq can be duplicated and headers/trailers attached
     to hold protocol and several duplicated bvecqs can be linked together
     to create a compound operation.

 (4) If the ->issue_xxx() functions terminate with -ENOMEM, a flag is set
     on the request to abort further subrequest generation/retrying.  If
     it fails early with any other error, generally the subreq will be
     extended as much as possible and failed with the error returned.

 (5) During writeback, netfslib now builds up an accumulation of buffered
     data before issuing writes on each stream (one server, one cache).  It
     asks each stream for an estimate of how much data to accumulate before
     it next generates subrequests on the stream.  The filesystem or cache
     is not required to use up all the data accumulated on a stream at that
     time unless the end of the pagecache is hit.

 (6) During read-gaps, in which there are two gaps on either end of a dirty
     streaming write page that need to be filled, a buffer is constructed
     consisting of the two ends plus a sink page repeated to cover the
     middle portion.  This is passed to the server as a single write.  For
     something like Ceph, this should probably be done either as a
     vectored/sparse read or as two separate reads (if different Ceph
     objects are involved).

 (7) During unbuffered/DIO read/write, there is a single contiguous file
     region to be read or written as a single stream.  The dispatching
     function just creates subrequests and calls ->issue_xxx() repeatedly
     to eat through the bufferage.

 (8) At the start of buffered read, the entire set of folios allocated by
     VM readahead is loaded into a bvecq chain, rather than trying to do it
     piecemeal as-needed.  As the pages were already added and locked by
     the VM, this is slightly more efficient than loading piecemeal as only
     a single iteration of the xarray is required.

 (9) During buffered read, there is a single contiguous file region, to
     read as a single stream - however, this stream may be stitched
     together from subrequests to multiple sources.  Which sources are used
     where is now determined by querying the cache to find the next couple
     of extents in which it has data; netfslib uses this to direct the
     subrequests towards the appropriate sources.

     Each subrequest is given the maximum length in the current extent and
     then ->issue_read() is called.  The filesystem then limits the size
     and slices off a piece of the buffer for that extent.

(10) Cachefiles now provides an estimation function that indicates the
     standard maxima for doing DIO (MAX_RW_COUNT and BIO_MAX_VECS).

Note that sparse cachefiles still rely on the backing filesystem for
content mapping.  That will need to be addressed in a future patch and is
not trivial to fix.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: Matthew Wilcox <willy at infradead.org>
cc: Christoph Hellwig <hch at infradead.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/9p/vfs_addr.c             |  60 +++-
 fs/afs/dir.c                 |  11 +-
 fs/afs/file.c                |  29 +-
 fs/afs/fsclient.c            |   8 +-
 fs/afs/internal.h            |   8 +-
 fs/afs/symlink.c             |   5 +-
 fs/afs/write.c               |  36 +-
 fs/afs/yfsclient.c           |   6 +-
 fs/cachefiles/io.c           | 241 +++++++++----
 fs/ceph/Kconfig              |   1 +
 fs/ceph/addr.c               | 123 ++++---
 fs/netfs/Kconfig             |   3 +
 fs/netfs/Makefile            |   2 +-
 fs/netfs/buffered_read.c     | 252 +++++++------
 fs/netfs/direct_read.c       |  87 ++---
 fs/netfs/direct_write.c      | 135 +++----
 fs/netfs/fscache_io.c        |   2 +-
 fs/netfs/internal.h          |  92 ++++-
 fs/netfs/iterator.c          |   4 +-
 fs/netfs/misc.c              |  35 +-
 fs/netfs/objects.c           |   9 +-
 fs/netfs/read_collect.c      |  43 +--
 fs/netfs/read_pgpriv2.c      | 122 ++++---
 fs/netfs/read_retry.c        | 208 +++++------
 fs/netfs/read_single.c       | 158 ++++----
 fs/netfs/write_collect.c     |  61 ++--
 fs/netfs/write_issue.c       | 673 ++++++++++++++++++++---------------
 fs/netfs/write_retry.c       | 168 +++++----
 fs/nfs/Kconfig               |   1 +
 fs/nfs/fscache.c             |  24 +-
 fs/smb/client/cifssmb.c      |  13 +-
 fs/smb/client/file.c         | 145 ++++----
 fs/smb/client/smb2ops.c      |   9 +-
 fs/smb/client/smb2pdu.c      |  28 +-
 fs/smb/client/transport.c    |  15 +-
 include/linux/netfs.h        |  99 +++---
 include/trace/events/netfs.h |  52 ++-
 net/9p/client.c              |   8 +-
 38 files changed, 1789 insertions(+), 1187 deletions(-)

diff --git a/fs/9p/vfs_addr.c b/fs/9p/vfs_addr.c
index 2129fcb0f65c..fc5b9c677986 100644
--- a/fs/9p/vfs_addr.c
+++ b/fs/9p/vfs_addr.c
@@ -48,31 +48,70 @@ static void v9fs_begin_writeback(struct netfs_io_request *wreq)
 	wreq->io_streams[0].avail = true;
 }
 
+/*
+ * Estimate how much data should be accumulated before we start issuing
+ * write subrequests.
+ */
+static int v9fs_estimate_write(struct netfs_io_request *wreq,
+			       struct netfs_io_stream *stream,
+			       struct netfs_write_estimate *estimate)
+{
+	struct p9_fid *fid = wreq->netfs_priv;
+	unsigned long long limit = ULLONG_MAX - stream->issue_from;
+	unsigned long long max_len = fid->clnt->msize - P9_IOHDRSZ;
+
+	estimate->issue_at = stream->issue_from + umin(max_len, limit);
+	return 0;
+}
+
 /*
  * Issue a subrequest to write to the server.
  */
-static void v9fs_issue_write(struct netfs_io_subrequest *subreq)
+static int v9fs_issue_write(struct netfs_io_subrequest *subreq)
 {
+	struct iov_iter iter;
 	struct p9_fid *fid = subreq->rreq->netfs_priv;
-	int err, len;
+	int err, len = 0;
+
+	subreq->len = umin(subreq->len, fid->clnt->msize - P9_IOHDRSZ);
 
-	len = p9_client_write(fid, subreq->start, &subreq->io_iter, &err);
+	err = netfs_prepare_write_buffer(subreq, INT_MAX);
+	if (err < 0)
+		return err;
+	/* After this point, must fail by termination. */
+
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
+	len = p9_client_write(fid, subreq->start, &iter, &err);
 	if (len > 0)
 		__set_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+
 	netfs_write_subrequest_terminated(subreq, len ?: err);
+	return 0;
 }
 
 /**
  * v9fs_issue_read - Issue a read from 9P
  * @subreq: The read to make
  */
-static void v9fs_issue_read(struct netfs_io_subrequest *subreq)
+static int v9fs_issue_read(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
+	struct iov_iter iter;
 	struct p9_fid *fid = rreq->netfs_priv;
 	char *target;
 	unsigned long long pos = subreq->start + subreq->transferred;
-	int total = 0, err, len, n;
+	size_t len;
+	int total = 0, err;
+
+	err = netfs_prepare_read_buffer(subreq, INT_MAX);
+	if (err < 0)
+		return err;
+	/* After this point, must fail by termination. */
+
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
 
 	if (S_ISLNK(rreq->inode->i_mode)) {
 		/* p9_client_readlink() must not be called for legacy protocols
@@ -89,12 +128,13 @@ static void v9fs_issue_read(struct netfs_io_subrequest *subreq)
 		err = p9_client_readlink(fid, &target);
 		if (err != 0)
 			goto fill_subreq;
-		len = strlen(target);
-		n = copy_to_iter(target, len, &subreq->io_iter);
+		len = min(strlen(target), subreq->len);
+		total = copy_to_iter(target, len, &iter);
 		kfree(target);
-		total = n;
 	} else {
-		total = p9_client_read(fid, pos, &subreq->io_iter, &err);
+		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+
+		total = p9_client_read(fid, pos, &iter, &err);
 	}
 
 fill_subreq:
@@ -112,6 +152,7 @@ static void v9fs_issue_read(struct netfs_io_subrequest *subreq)
 
 	subreq->error = err;
 	netfs_read_subreq_terminated(subreq);
+	return 0;
 }
 
 /**
@@ -185,6 +226,7 @@ const struct netfs_request_ops v9fs_req_ops = {
 	.free_request		= v9fs_free_request,
 	.issue_read		= v9fs_issue_read,
 	.begin_writeback	= v9fs_begin_writeback,
+	.estimate_write		= v9fs_estimate_write,
 	.issue_write		= v9fs_issue_write,
 };
 
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 547faa06c80b..174114c39639 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -250,8 +250,8 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
 	if (dvnode->directory_size < i_size) {
 		size_t cur_size = dvnode->directory_size;
 
-		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size, i_size,
-					  GFP_KERNEL);
+		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size,
+					  round_up(i_size, PAGE_SIZE), GFP_KERNEL);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			return ret;
@@ -2227,11 +2227,10 @@ static int afs_dir_writepages(struct address_space *mapping,
 	}
 
 	if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
+		size_t len = i_size_read(&dvnode->netfs.inode);
 		iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
-				    i_size_read(&dvnode->netfs.inode));
-		ret = netfs_writeback_single(mapping, wbc, &iter);
-		if (ret == 1)
-			ret = 0; /* Skipped write due to lock conflict. */
+				    round_up(len, PAGE_SIZE));
+		ret = netfs_writeback_single(mapping, wbc, &iter, len);
 	}
 
 	up_read(&dvnode->validate_lock);
diff --git a/fs/afs/file.c b/fs/afs/file.c
index 92380dc25781..476b9f3a776e 100644
--- a/fs/afs/file.c
+++ b/fs/afs/file.c
@@ -332,11 +332,12 @@ void afs_fetch_data_immediate_cancel(struct afs_call *call)
 /*
  * Fetch file data from the volume.
  */
-static void afs_issue_read(struct netfs_io_subrequest *subreq)
+static int afs_issue_read(struct netfs_io_subrequest *subreq)
 {
 	struct afs_operation *op;
 	struct afs_vnode *vnode = AFS_FS_I(subreq->rreq->inode);
 	struct key *key = subreq->rreq->netfs_priv;
+	int ret;
 
 	_enter("%s{%llx:%llu.%u},%x,,,",
 	       vnode->volume->name,
@@ -345,11 +346,15 @@ static void afs_issue_read(struct netfs_io_subrequest *subreq)
 	       vnode->fid.unique,
 	       key_serial(key));
 
+	ret = netfs_prepare_read_buffer(subreq, INT_MAX);
+	if (ret < 0)
+		return ret;
+	/* After this point, must fail by termination. */
+
 	op = afs_alloc_operation(key, vnode->volume);
 	if (IS_ERR(op)) {
-		subreq->error = PTR_ERR(op);
-		netfs_read_subreq_terminated(subreq);
-		return;
+		ret = PTR_ERR(op);
+		goto failed;
 	}
 
 	afs_op_set_vnode(op, 0, vnode);
@@ -364,20 +369,24 @@ static void afs_issue_read(struct netfs_io_subrequest *subreq)
 		op->flags |= AFS_OPERATION_ASYNC;
 
 		if (!afs_begin_vnode_operation(op)) {
-			subreq->error = afs_put_operation(op);
-			netfs_read_subreq_terminated(subreq);
-			return;
+			ret = afs_put_operation(op);
+			goto failed;
 		}
 
 		if (!afs_select_fileserver(op)) {
-			afs_end_read(op);
-			return;
+			afs_end_read(op); /* Error recorded here. */
+			return 0;
 		}
 
 		afs_issue_read_call(op);
 	} else {
 		afs_do_sync_operation(op);
 	}
+	return 0;
+failed:
+	subreq->error = ret;
+	netfs_read_subreq_terminated(subreq);
+	return 0;
 }
 
 static int afs_init_request(struct netfs_io_request *rreq, struct file *file)
@@ -469,7 +478,7 @@ const struct netfs_request_ops afs_req_ops = {
 	.update_i_size		= afs_update_i_size,
 	.invalidate_cache	= afs_netfs_invalidate_cache,
 	.begin_writeback	= afs_begin_writeback,
-	.prepare_write		= afs_prepare_write,
+	.estimate_write		= afs_estimate_write,
 	.issue_write		= afs_issue_write,
 	.retry_request		= afs_retry_request,
 };
diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c
index 1a3f186a6a11..9713551dd842 100644
--- a/fs/afs/fsclient.c
+++ b/fs/afs/fsclient.c
@@ -339,7 +339,9 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call)
 		if (call->remaining == 0)
 			goto no_more_data;
 
-		call->iter = &subreq->io_iter;
+		iov_iter_bvec_queue(&call->def_iter, ITER_DEST, subreq->content.bvecq,
+				    subreq->content.slot, subreq->content.offset, subreq->len);
+
 		call->iov_len = umin(call->remaining, subreq->len - subreq->transferred);
 		call->unmarshall++;
 		fallthrough;
@@ -1088,7 +1090,7 @@ static void afs_fs_store_data64(struct afs_operation *op)
 	if (!call)
 		return afs_op_nomem(op);
 
-	call->write_iter = op->store.write_iter;
+	call->write_iter = &op->store.write_iter;
 
 	/* marshall the parameters */
 	bp = call->request;
@@ -1142,7 +1144,7 @@ void afs_fs_store_data(struct afs_operation *op)
 	if (!call)
 		return afs_op_nomem(op);
 
-	call->write_iter = op->store.write_iter;
+	call->write_iter = &op->store.write_iter;
 
 	/* marshall the parameters */
 	bp = call->request;
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 556e7e7777f2..5c536cc7b53c 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -914,7 +914,7 @@ struct afs_operation {
 			afs_lock_type_t type;
 		} lock;
 		struct {
-			struct iov_iter	*write_iter;
+			struct iov_iter	write_iter;
 			loff_t	pos;
 			loff_t	size;
 			loff_t	i_size;
@@ -1698,8 +1698,10 @@ extern int afs_check_volume_status(struct afs_volume *, struct afs_operation *);
 /*
  * write.c
  */
-void afs_prepare_write(struct netfs_io_subrequest *subreq);
-void afs_issue_write(struct netfs_io_subrequest *subreq);
+int afs_estimate_write(struct netfs_io_request *wreq,
+		       struct netfs_io_stream *stream,
+		       struct netfs_write_estimate *estimate);
+int afs_issue_write(struct netfs_io_subrequest *subreq);
 void afs_begin_writeback(struct netfs_io_request *wreq);
 void afs_retry_request(struct netfs_io_request *wreq, struct netfs_io_stream *stream);
 extern int afs_writepages(struct address_space *, struct writeback_control *);
diff --git a/fs/afs/symlink.c b/fs/afs/symlink.c
index 7fd4f985c441..40f4bb5ac983 100644
--- a/fs/afs/symlink.c
+++ b/fs/afs/symlink.c
@@ -245,9 +245,10 @@ int afs_symlink_writepages(struct address_space *mapping,
 
 	if (vnode->directory &&
 	    atomic64_read(&vnode->cb_expires_at) != AFS_NO_CB_PROMISE) {
+		size_t len = i_size_read(&vnode->netfs.inode);
 		iov_iter_bvec_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
-				    i_size_read(&vnode->netfs.inode));
-		ret = netfs_writeback_single(mapping, wbc, &iter);
+				    round_up(len, PAGE_SIZE));
+		ret = netfs_writeback_single(mapping, wbc, &iter, len);
 	}
 
 	if (ret == 0) {
diff --git a/fs/afs/write.c b/fs/afs/write.c
index 7f34b939706a..46f81d9e7d7f 100644
--- a/fs/afs/write.c
+++ b/fs/afs/write.c
@@ -83,17 +83,20 @@ static const struct afs_operation_ops afs_store_data_operation = {
 };
 
 /*
- * Prepare a subrequest to write to the server.  This sets the max_len
- * parameter.
+ * Estimate the maximum size of a write we can send to the server.
  */
-void afs_prepare_write(struct netfs_io_subrequest *subreq)
+int afs_estimate_write(struct netfs_io_request *wreq,
+		       struct netfs_io_stream *stream,
+		       struct netfs_write_estimate *estimate)
 {
-	struct netfs_io_stream *stream = &subreq->rreq->io_streams[subreq->stream_nr];
+	unsigned long long limit = ULLONG_MAX - stream->issue_from;
+	unsigned long long max_len = 256 * 1024 * 1024;
 
 	//if (test_bit(NETFS_SREQ_RETRYING, &subreq->flags))
-	//	subreq->max_len = 512 * 1024;
-	//else
-	stream->sreq_max_len = 256 * 1024 * 1024;
+	//	max_len = 512 * 1024;
+
+	estimate->issue_at = stream->issue_from + umin(max_len, limit);
+	return 0;
 }
 
 /*
@@ -139,12 +142,15 @@ static void afs_issue_write_worker(struct work_struct *work)
 	op->flags		|= AFS_OPERATION_UNINTR;
 	op->ops			= &afs_store_data_operation;
 
+	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 	afs_begin_vnode_operation(op);
 
-	op->store.write_iter	= &subreq->io_iter;
 	op->store.i_size	= umax(pos + len, netfs_read_remote_i_size(&vnode->netfs.inode));
 	op->mtime		= inode_get_mtime(&vnode->netfs.inode);
 
+	iov_iter_bvec_queue(&op->store.write_iter, ITER_SOURCE, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
 	afs_wait_for_operation(op);
 	ret = afs_put_operation(op);
 	switch (ret) {
@@ -168,11 +174,21 @@ static void afs_issue_write_worker(struct work_struct *work)
 	netfs_write_subrequest_terminated(subreq, ret < 0 ? ret : subreq->len);
 }
 
-void afs_issue_write(struct netfs_io_subrequest *subreq)
+int afs_issue_write(struct netfs_io_subrequest *subreq)
 {
+	int ret;
+
+	if (subreq->len > 256 * 1024 * 1024)
+		subreq->len = 256 * 1024 * 1024;
+	ret = netfs_prepare_write_buffer(subreq, INT_MAX);
+	if (ret < 0)
+		return ret;
+	/* After this point, must fail by termination. */
+
 	subreq->work.func = afs_issue_write_worker;
 	if (!queue_work(system_dfl_wq, &subreq->work))
 		WARN_ON_ONCE(1);
+	return 0;
 }
 
 /*
@@ -183,6 +199,8 @@ void afs_begin_writeback(struct netfs_io_request *wreq)
 {
 	if (S_ISREG(wreq->inode->i_mode))
 		afs_get_writeback_key(wreq);
+
+	wreq->io_streams[0].avail = true;
 }
 
 /*
diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c
index d941179730a9..52c588092050 100644
--- a/fs/afs/yfsclient.c
+++ b/fs/afs/yfsclient.c
@@ -385,7 +385,9 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call)
 		if (call->remaining == 0)
 			goto no_more_data;
 
-		call->iter = &subreq->io_iter;
+		iov_iter_bvec_queue(&call->def_iter, ITER_DEST, subreq->content.bvecq,
+				    subreq->content.slot, subreq->content.offset, subreq->len);
+
 		call->iov_len = min(call->remaining, subreq->len - subreq->transferred);
 		call->unmarshall++;
 		fallthrough;
@@ -1357,7 +1359,7 @@ void yfs_fs_store_data(struct afs_operation *op)
 	if (!call)
 		return afs_op_nomem(op);
 
-	call->write_iter = op->store.write_iter;
+	call->write_iter = &op->store.write_iter;
 
 	/* marshall the parameters */
 	bp = call->request;
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index d22a8a88b93d..24a93cc0aae8 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -26,7 +26,10 @@ struct cachefiles_kiocb {
 	};
 	struct cachefiles_object *object;
 	netfs_io_terminated_t	term_func;
-	void			*term_func_priv;
+	union {
+		struct netfs_io_subrequest *subreq;
+		void			*term_func_priv;
+	};
 	bool			was_async;
 	unsigned int		inval_counter;	/* Copy of cookie->inval_counter */
 	u64			b_writing;
@@ -194,12 +197,131 @@ static int cachefiles_read(struct netfs_cache_resources *cres,
 	return ret;
 }
 
+/*
+ * Handle completion of a read from the cache issued by netfslib.
+ */
+static void cachefiles_issue_read_complete(struct kiocb *iocb, long ret)
+{
+	struct cachefiles_kiocb *ki = container_of(iocb, struct cachefiles_kiocb, iocb);
+	struct netfs_io_subrequest *subreq = ki->subreq;
+	struct inode *inode = file_inode(ki->iocb.ki_filp);
+
+	_enter("%ld", ret);
+
+	if (ret < 0) {
+		subreq->error = -ESTALE;
+		trace_cachefiles_io_error(ki->object, inode, ret,
+					  cachefiles_trace_read_error);
+	}
+
+	if (ret >= 0) {
+		if (ki->object->cookie->inval_counter == ki->inval_counter) {
+			subreq->error = 0;
+			if (ret > 0) {
+				subreq->transferred += ret;
+				__set_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+			}
+		} else {
+			subreq->error = -ESTALE;
+		}
+	}
+
+	netfs_read_subreq_terminated(subreq);
+	cachefiles_put_kiocb(ki);
+}
+
+/*
+ * Issue a read operation to the cache.  We assume that the cache is readable
+ * as the caller should have called ->query_occupancy() first.
+ */
+static int cachefiles_issue_read(struct netfs_io_subrequest *subreq)
+{
+	struct netfs_cache_resources *cres = &subreq->rreq->cache_resources;
+	struct cachefiles_object *object = cachefiles_cres_object(cres);
+	struct cachefiles_kiocb *ki;
+	struct iov_iter iter;
+	struct file *file = cachefiles_cres_file(cres);
+	unsigned int old_nofs;
+	ssize_t ret;
+
+	fscache_count_read();
+
+	_enter("%pD,%lli,%llx,%zx/%llx",
+	       file, file_inode(file)->i_ino, subreq->start, subreq->len,
+	       i_size_read(file_inode(file)));
+
+	if (subreq->len > MAX_RW_COUNT)
+		subreq->len = MAX_RW_COUNT;
+
+	ret = netfs_prepare_read_buffer(subreq, BIO_MAX_VECS);
+	if (ret < 0)
+		return ret;
+	/* After this point, must fail by termination. */
+
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
+	ret = -ENOMEM;
+	ki = kzalloc_obj(struct cachefiles_kiocb);
+	if (!ki)
+		goto failed;
+
+	refcount_set(&ki->ki_refcnt, 2);
+	ki->iocb.ki_filp	= file;
+	ki->iocb.ki_pos		= subreq->start;
+	ki->iocb.ki_flags	= IOCB_DIRECT;
+	ki->iocb.ki_ioprio	= get_current_ioprio();
+	ki->iocb.ki_complete	= cachefiles_issue_read_complete;
+	ki->object		= object;
+	ki->inval_counter	= cres->inval_counter;
+	ki->subreq		= subreq;
+	ki->was_async		= true;
+
+	get_file(ki->iocb.ki_filp);
+	cachefiles_grab_object(object, cachefiles_obj_get_ioreq);
+
+	trace_cachefiles_read(object, file_inode(file), ki->iocb.ki_pos, subreq->len);
+	old_nofs = memalloc_nofs_save();
+	ret = cachefiles_inject_read_error();
+	if (ret == 0)
+		ret = vfs_iocb_iter_read(file, &ki->iocb, &iter);
+	memalloc_nofs_restore(old_nofs);
+
+	switch (ret) {
+	case -EIOCBQUEUED:
+		break;
+
+	case -ERESTARTSYS:
+	case -ERESTARTNOINTR:
+	case -ERESTARTNOHAND:
+	case -ERESTART_RESTARTBLOCK:
+		/* There's no easy way to restart the syscall since other AIO's
+		 * may be already running. Just fail this IO with EINTR.
+		 */
+		ret = -EINTR;
+		fallthrough;
+	default:
+		ki->was_async = false;
+		cachefiles_issue_read_complete(&ki->iocb, ret);
+		break;
+	}
+
+	cachefiles_put_kiocb(ki);
+	_leave(" = %zd", ret);
+	return 0;
+failed:
+	subreq->error = ret;
+	netfs_read_subreq_terminated(subreq);
+	return 0;
+}
+
 /*
  * Query the occupancy of the cache in a region, returning the extent of the
- * next two chunks of cached data and the next hole.
+ * next two chunks of cached data and the next hole.  The occupancy map is
+ * preloaded to show just one giant hole.
  */
-static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
-				      struct fscache_occupancy *occ)
+static void cachefiles_query_occupancy(struct netfs_cache_resources *cres,
+				       struct fscache_occupancy *occ)
 {
 	struct cachefiles_object *object;
 	struct inode *inode;
@@ -209,7 +331,7 @@ static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
 	int i;
 
 	if (!fscache_wait_for_operation(cres, FSCACHE_WANT_READ))
-		return -ENOBUFS;
+		return;
 
 	object = cachefiles_cres_object(cres);
 	file = cachefiles_cres_file(cres);
@@ -244,7 +366,7 @@ static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
 			ret = vfs_llseek(file, occ->query_from, SEEK_DATA);
 		if (IS_ERR_VALUE_LL(ret)) {
 			if (ret != -ENXIO)
-				return ret;
+				goto done;
 			occ->query_from = ULLONG_MAX;
 			goto done;
 		}
@@ -257,7 +379,7 @@ static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
 			ret = vfs_llseek(file, occ->query_from, SEEK_HOLE);
 		if (IS_ERR_VALUE_LL(ret)) {
 			if (ret != -ENXIO)
-				return ret;
+				goto done;
 			occ->query_from = ULLONG_MAX;
 			goto done;
 		}
@@ -270,7 +392,6 @@ static int cachefiles_query_occupancy(struct netfs_cache_resources *cres,
 done:
 	_debug("query[0] %llx-%llx", occ->cached_from[0], occ->cached_to[0]);
 	_debug("query[1] %llx-%llx", occ->cached_from[1], occ->cached_to[1]);
-	return 0;
 }
 
 /*
@@ -587,9 +708,9 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 				    cachefiles_has_space_for_write);
 }
 
-static int cachefiles_prepare_write(struct netfs_cache_resources *cres,
-				    loff_t *_start, size_t *_len, size_t upper_len,
-				    loff_t i_size, bool no_space_allocated_yet)
+static int cachefiles_prepare_write_old(struct netfs_cache_resources *cres,
+					loff_t *_start, size_t *_len, size_t upper_len,
+					loff_t i_size, bool no_space_allocated_yet)
 {
 	struct cachefiles_object *object = cachefiles_cres_object(cres);
 	struct cachefiles_cache *cache = object->volume->cache;
@@ -611,80 +732,72 @@ static int cachefiles_prepare_write(struct netfs_cache_resources *cres,
 	return ret;
 }
 
-static void cachefiles_prepare_write_subreq(struct netfs_io_subrequest *subreq)
+static int cachefiles_estimate_write(struct netfs_io_request *wreq,
+				     struct netfs_io_stream *stream,
+				     struct netfs_write_estimate *estimate)
 {
-	struct netfs_io_request *wreq = subreq->rreq;
-	struct netfs_cache_resources *cres = &wreq->cache_resources;
-	struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];
-
-	_enter("W=%x[%x] %llx", wreq->debug_id, subreq->debug_index, subreq->start);
-
-	stream->sreq_max_len = MAX_RW_COUNT;
-	stream->sreq_max_segs = BIO_MAX_VECS;
-
-	if (!cachefiles_cres_file(cres)) {
-		if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
-			return netfs_prepare_write_failed(subreq);
-		if (!cachefiles_cres_file(cres))
-			return netfs_prepare_write_failed(subreq);
-	}
+	estimate->issue_at = stream->issue_from + MAX_RW_COUNT;
+	estimate->max_segs = BIO_MAX_VECS;
+	return 0;
 }
 
-static void cachefiles_issue_write(struct netfs_io_subrequest *subreq)
+static int cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *wreq = subreq->rreq;
 	struct netfs_cache_resources *cres = &wreq->cache_resources;
 	struct cachefiles_object *object = cachefiles_cres_object(cres);
 	struct cachefiles_cache *cache = object->volume->cache;
+	struct iov_iter iter;
 	const struct cred *saved_cred;
-	size_t off, pre, post, len = subreq->len;
 	loff_t start = subreq->start;
-	int ret;
+	size_t len = subreq->len;
+	int ret = -EINVAL;
 
 	_enter("W=%x[%x] %llx-%llx",
 	       wreq->debug_id, subreq->debug_index, start, start + len - 1);
 
-	/* We need to start on the cache granularity boundary */
-	off = start & (cache->bsize - 1);
-	if (off) {
-		pre = cache->bsize - off;
-		if (pre >= len) {
-			fscache_count_dio_misfit();
-			netfs_write_subrequest_terminated(subreq, len);
-			return;
-		}
-		subreq->transferred += pre;
-		start += pre;
-		len -= pre;
-		iov_iter_advance(&subreq->io_iter, pre);
-	}
-
-	/* We also need to end on the cache granularity boundary */
-	post = len & (cache->bsize - 1);
-	if (post) {
-		len -= post;
-		if (len == 0) {
-			fscache_count_dio_misfit();
-			netfs_write_subrequest_terminated(subreq, post);
-			return;
-		}
-		iov_iter_truncate(&subreq->io_iter, len);
+	if (!cachefiles_cres_file(cres)) {
+		if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
+			return -ENOBUFS;
+		if (!cachefiles_cres_file(cres))
+			return -ENOBUFS;
+	}
+
+	ret = netfs_prepare_write_buffer(subreq, BIO_MAX_VECS);
+	if (ret < 0)
+		return ret;
+	/* After this point, must fail by termination. */
+
+	/* The buffer extraction func may round out start and end. */
+	start = subreq->start;
+	len = subreq->len;
+
+	/* We need to start and end on cache granularity boundaries. */
+	if (WARN_ON_ONCE(start & (cache->bsize - 1)) ||
+	    WARN_ON_ONCE(len   & (cache->bsize - 1))) {
+		fscache_count_dio_misfit();
+		ret = -EIO;
+		goto failed;
 	}
 
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, len);
+
 	trace_netfs_sreq(subreq, netfs_sreq_trace_cache_prepare);
 	cachefiles_begin_secure(cache, &saved_cred);
 	ret = __cachefiles_prepare_write(object, cachefiles_cres_file(cres),
 					 &start, &len, len, true);
 	cachefiles_end_secure(cache, saved_cred);
-	if (ret < 0) {
-		netfs_write_subrequest_terminated(subreq, ret);
-		return;
-	}
+	if (ret < 0)
+		goto failed;
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_cache_write);
-	cachefiles_write(&subreq->rreq->cache_resources,
-			 subreq->start, &subreq->io_iter,
+	cachefiles_write(&subreq->rreq->cache_resources, subreq->start, &iter,
 			 netfs_write_subrequest_terminated, subreq);
+	return 0;
+failed:
+	netfs_write_subrequest_terminated(subreq, ret);
+	return 0;
 }
 
 /*
@@ -741,6 +854,7 @@ static void cachefiles_collect_write(struct netfs_io_request *wreq,
 	    WARN_ON(new_size	& (cache->bsize - 1)) ||
 	    WARN_ON(start	& (cache->bsize - 1)) ||
 	    WARN_ON(len		& (cache->bsize - 1))) {
+		trace_netfs_rreq(wreq, netfs_rreq_trace_cache_align_error);
 		trace_cachefiles_io_error(object, inode, -EIO,
 					  cachefiles_trace_alignment_error);
 		cachefiles_remove_object_xattr(cache, object, file->f_path.dentry);
@@ -891,9 +1005,10 @@ static const struct netfs_cache_ops cachefiles_netfs_cache_ops = {
 	.end_operation		= cachefiles_end_operation,
 	.read			= cachefiles_read,
 	.write			= cachefiles_write,
+	.issue_read		= cachefiles_issue_read,
 	.issue_write		= cachefiles_issue_write,
-	.prepare_write		= cachefiles_prepare_write,
-	.prepare_write_subreq	= cachefiles_prepare_write_subreq,
+	.estimate_write		= cachefiles_estimate_write,
+	.prepare_write_old	= cachefiles_prepare_write_old,
 	.query_occupancy	= cachefiles_query_occupancy,
 	.collect_write		= cachefiles_collect_write,
 };
diff --git a/fs/ceph/Kconfig b/fs/ceph/Kconfig
index 3d64a316ca31..aa6ccd7794d2 100644
--- a/fs/ceph/Kconfig
+++ b/fs/ceph/Kconfig
@@ -4,6 +4,7 @@ config CEPH_FS
 	depends on INET
 	select CEPH_LIB
 	select NETFS_SUPPORT
+	select NETFS_PGPRIV2
 	select FS_ENCRYPTION_ALGS if FS_ENCRYPTION
 	default n
 	help
diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c
index ecf33b66610c..0e1cb493fe6e 100644
--- a/fs/ceph/addr.c
+++ b/fs/ceph/addr.c
@@ -274,7 +274,7 @@ static void finish_netfs_read(struct ceph_osd_request *req)
 	ceph_dec_osd_stopping_blocker(fsc->mdsc);
 }
 
-static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
+static int ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	struct inode *inode = rreq->inode;
@@ -283,7 +283,8 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 	struct ceph_mds_request *req;
 	struct ceph_mds_client *mdsc = ceph_sb_to_mdsc(inode->i_sb);
 	struct ceph_inode_info *ci = ceph_inode(inode);
-	ssize_t err = 0;
+	struct iov_iter iter;
+	ssize_t err;
 	size_t len;
 	int mode;
 
@@ -292,8 +293,20 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 		__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
 	__clear_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
 
-	if (subreq->start >= inode->i_size)
+	err = netfs_prepare_read_buffer(subreq, INT_MAX);
+	if (err < 0)
+		return err;
+	/* After this point, must fail by termination. */
+
+	if (subreq->start >= inode->i_size) {
+		__set_bit(NETFS_SREQ_HIT_EOF, &subreq->flags);
+		err = 0;
 		goto out;
+	}
+
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset,
+			    subreq->len);
 
 	/* We need to fetch the inline data. */
 	mode = ceph_try_to_choose_auth_mds(inode, CEPH_STAT_CAP_INLINE_DATA);
@@ -302,11 +315,13 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 		err = PTR_ERR(req);
 		goto out;
 	}
+
 	req->r_ino1 = ci->i_vino;
 	req->r_args.getattr.mask = cpu_to_le32(CEPH_STAT_CAP_INLINE_DATA);
 	req->r_num_caps = 2;
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+
 	err = ceph_mdsc_do_request(mdsc, NULL, req);
 	if (err < 0)
 		goto out;
@@ -316,11 +331,13 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 	if (iinfo->inline_version == CEPH_INLINE_NONE) {
 		/* The data got uninlined */
 		ceph_mdsc_put_request(req);
-		return false;
+		__set_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+		err = -EAGAIN;
+		goto out;
 	}
 
 	len = min_t(size_t, iinfo->inline_len - subreq->start, subreq->len);
-	err = copy_to_iter(iinfo->inline_data + subreq->start, len, &subreq->io_iter);
+	err = copy_to_iter(iinfo->inline_data + subreq->start, len, &iter);
 	if (err == 0) {
 		err = -EFAULT;
 	} else {
@@ -333,26 +350,10 @@ static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
 	subreq->error = err;
 	trace_netfs_sreq(subreq, netfs_sreq_trace_io_progress);
 	netfs_read_subreq_terminated(subreq);
-	return true;
-}
-
-static int ceph_netfs_prepare_read(struct netfs_io_subrequest *subreq)
-{
-	struct netfs_io_request *rreq = subreq->rreq;
-	struct inode *inode = rreq->inode;
-	struct ceph_inode_info *ci = ceph_inode(inode);
-	struct ceph_fs_client *fsc = ceph_inode_to_fs_client(inode);
-	u64 objno, objoff;
-	u32 xlen;
-
-	/* Truncate the extent at the end of the current block */
-	ceph_calc_file_object_mapping(&ci->i_layout, subreq->start, subreq->len,
-				      &objno, &objoff, &xlen);
-	rreq->io_streams[0].sreq_max_len = umin(xlen, fsc->mount_options->rsize);
 	return 0;
 }
 
-static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
+static int ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	struct inode *inode = rreq->inode;
@@ -361,19 +362,18 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 	struct ceph_client *cl = fsc->client;
 	struct ceph_osd_request *req = NULL;
 	struct ceph_vino vino = ceph_vino(inode);
-	int err;
-	u64 len;
+	struct iov_iter iter;
+	u64 objno, objoff, len, off = subreq->start;
+	u32 maxlen;
+	int err = -EIO;
 	bool sparse = IS_ENCRYPTED(inode) || ceph_test_mount_opt(fsc, SPARSEREAD);
-	u64 off = subreq->start;
 	int extent_cnt;
 
-	if (ceph_inode_is_shutdown(inode)) {
-		err = -EIO;
-		goto out;
-	}
+	if (ceph_inode_is_shutdown(inode))
+		return -EIO;
 
-	if (ceph_has_inline_data(ci) && ceph_netfs_issue_op_inline(subreq))
-		return;
+	if (ceph_has_inline_data(ci))
+		return ceph_netfs_issue_op_inline(subreq);
 
 	// TODO: This rounding here is slightly dodgy.  It *should* work, for
 	// now, as the cache only deals in blocks that are a multiple of
@@ -383,26 +383,48 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 	len = subreq->len;
 	ceph_fscrypt_adjust_off_and_len(inode, &off, &len);
 
+	/* Truncate the extent at the end of the current block */
+	ceph_calc_file_object_mapping(&ci->i_layout, subreq->start, len,
+				      &objno, &objoff, &maxlen);
+	maxlen = min(maxlen, fsc->mount_options->rsize);
+	len = min(len, maxlen);
+	subreq->len = len;
+
+	/* Grab a slice of read buffer.  This may shrink the subreq. */
+	err = netfs_prepare_read_buffer(subreq, INT_MAX);
+	if (err < 0)
+		return err;
+	/* After this point, must fail by termination. */
+
+	/* Create a request.  In theory, this may shrink the request again, but
+	 * it shouldn't since we calculated the object size above and already
+	 * shrank to that, but if it does, we'll just end up doing a short read
+	 * and retrying to get the rest.
+	 */
+	len = subreq->len;
 	req = ceph_osdc_new_request(&fsc->client->osdc, &ci->i_layout, vino,
 			off, &len, 0, 1, sparse ? CEPH_OSD_OP_SPARSE_READ : CEPH_OSD_OP_READ,
 			CEPH_OSD_FLAG_READ, NULL, ci->i_truncate_seq,
 			ci->i_truncate_size, false);
 	if (IS_ERR(req)) {
 		err = PTR_ERR(req);
-		req = NULL;
-		goto out;
+		goto failed_noput;
 	}
 
 	if (sparse) {
 		extent_cnt = __ceph_sparse_read_ext_count(inode, len);
 		err = ceph_alloc_sparse_ext_map(&req->r_ops[0], extent_cnt);
 		if (err)
-			goto out;
+			goto failed;
 	}
 
 	doutc(cl, "%llx.%llx pos=%llu orig_len=%zu len=%llu\n",
 	      ceph_vinop(inode), subreq->start, subreq->len, len);
 
+	iov_iter_bvec_queue(&iter, ITER_DEST, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset,
+			    subreq->len);
+
 	/*
 	 * FIXME: For now, use CEPH_OSD_DATA_TYPE_PAGES instead of _ITER for
 	 * encrypted inodes. We'd need infrastructure that handles an iov_iter
@@ -421,13 +443,11 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 		 * ceph_msg_data_cursor_init() triggers BUG_ON() in the case
 		 * if msg->sparse_read_total > msg->data_length.
 		 */
-		subreq->io_iter.count = len;
-
-		err = iov_iter_get_pages_alloc2(&subreq->io_iter, &pages, len, &page_off);
+		err = iov_iter_get_pages_alloc2(&iter, &pages, len, &page_off);
 		if (err < 0) {
 			doutc(cl, "%llx.%llx failed to allocate pages, %d\n",
 			      ceph_vinop(inode), err);
-			goto out;
+			goto eio;
 		}
 
 		/* should always give us a page-aligned read */
@@ -438,12 +458,10 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 		osd_req_op_extent_osd_data_pages(req, 0, pages, len, 0, false,
 						 false);
 	} else {
-		osd_req_op_extent_osd_iter(req, 0, &subreq->io_iter);
-	}
-	if (!ceph_inc_osd_stopping_blocker(fsc->mdsc)) {
-		err = -EIO;
-		goto out;
+		osd_req_op_extent_osd_iter(req, 0, &iter);
 	}
+	if (!ceph_inc_osd_stopping_blocker(fsc->mdsc))
+		goto eio;
 	req->r_callback = finish_netfs_read;
 	req->r_priv = subreq;
 	req->r_inode = inode;
@@ -451,19 +469,22 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 	ceph_osdc_start_request(req->r_osdc, req);
-out:
 	ceph_osdc_put_request(req);
-	if (err) {
-		subreq->error = err;
-		netfs_read_subreq_terminated(subreq);
-	}
-	doutc(cl, "%llx.%llx result %d\n", ceph_vinop(inode), err);
+	doutc(cl, "%llx.%llx result -EIOCBQUEUED\n", ceph_vinop(inode));
+	return 0;
+eio:
+	err = -EIO;
+failed:
+	ceph_osdc_put_request(req);
+failed_noput:
+	subreq->error = err;
+	netfs_read_subreq_terminated(subreq);
+	return 0;
 }
 
 static int ceph_init_request(struct netfs_io_request *rreq, struct file *file)
 {
 	struct inode *inode = rreq->inode;
-	struct ceph_fs_client *fsc = ceph_inode_to_fs_client(inode);
 	struct ceph_client *cl = ceph_inode_to_client(inode);
 	int got = 0, want = CEPH_CAP_FILE_CACHE;
 	struct ceph_netfs_request_data *priv;
@@ -515,7 +536,6 @@ static int ceph_init_request(struct netfs_io_request *rreq, struct file *file)
 
 	priv->caps = got;
 	rreq->netfs_priv = priv;
-	rreq->io_streams[0].sreq_max_len = fsc->mount_options->rsize;
 
 out:
 	if (ret < 0) {
@@ -543,7 +563,6 @@ static void ceph_netfs_free_request(struct netfs_io_request *rreq)
 const struct netfs_request_ops ceph_netfs_ops = {
 	.init_request		= ceph_init_request,
 	.free_request		= ceph_netfs_free_request,
-	.prepare_read		= ceph_netfs_prepare_read,
 	.issue_read		= ceph_netfs_issue_read,
 	.expand_readahead	= ceph_netfs_expand_readahead,
 	.check_write_begin	= ceph_netfs_check_write_begin,
diff --git a/fs/netfs/Kconfig b/fs/netfs/Kconfig
index 7701c037c328..d0e7b0971fa3 100644
--- a/fs/netfs/Kconfig
+++ b/fs/netfs/Kconfig
@@ -22,6 +22,9 @@ config NETFS_STATS
 	  between CPUs.  On the other hand, the stats are very useful for
 	  debugging purposes.  Saying 'Y' here is recommended.
 
+config NETFS_PGPRIV2
+	bool
+
 config NETFS_DEBUG
 	bool "Enable dynamic debugging netfslib and FS-Cache"
 	depends on NETFS_SUPPORT
diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index 0621e6870cbd..421dd0be413b 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -12,13 +12,13 @@ netfs-y := \
 	misc.o \
 	objects.o \
 	read_collect.o \
-	read_pgpriv2.o \
 	read_retry.o \
 	read_single.o \
 	write_collect.o \
 	write_issue.o \
 	write_retry.o
 
+netfs-$(CONFIG_NETFS_PGPRIV2) += read_pgpriv2.o
 netfs-$(CONFIG_NETFS_STATS) += stats.o
 
 netfs-$(CONFIG_FSCACHE) += \
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 3da20ed981a4..c84bd9949abb 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -100,70 +100,98 @@ static int netfs_begin_cache_read(struct netfs_io_request *rreq, struct netfs_in
 }
 
 /*
- * netfs_prepare_read_iterator - Prepare the subreq iterator for I/O
- * @subreq: The subrequest to be set up
- *
- * Prepare the I/O iterator representing the read buffer on a subrequest for
- * the filesystem to use for I/O (it can be passed directly to a socket).  This
- * is intended to be called from the ->issue_read() method once the filesystem
- * has trimmed the request to the size it wants.
- *
- * Returns the limited size if successful and -ENOMEM if insufficient memory
- * available.
+ * Prepare the I/O buffer on a buffered read subrequest for the filesystem to
+ * use as a bvec queue.
  */
-static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
+static int netfs_prepare_buffered_read_buffer(struct netfs_io_subrequest *subreq,
+					      unsigned int max_segs)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
 	ssize_t extracted;
-	size_t rsize = subreq->len;
 
-	if (subreq->source == NETFS_DOWNLOAD_FROM_SERVER)
-		rsize = umin(rsize, stream->sreq_max_len);
+	_enter("R=%08x[%x] l=%zx s=%u",
+	       rreq->debug_id, subreq->debug_index, subreq->len, max_segs);
 
-	bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
-	extracted = bvecq_slice(&rreq->dispatch_cursor, rsize,
-				stream->sreq_max_segs, &subreq->nr_segs);
-	if (extracted < rsize) {
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+	extracted = bvecq_slice(&stream->dispatch_cursor, subreq->len,
+				max_segs, &subreq->nr_segs);
+
+	if (extracted < subreq->len) {
 		subreq->len = extracted;
 		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
 	}
+	stream->buffered -= extracted;
+	stream->issue_from = subreq->start + subreq->len;
+	rreq->submitted = stream->issue_from;
 
-	return subreq->len;
+	if (!stream->buffered)
+		netfs_all_subreqs_queued(rreq);
+	return 0;
 }
 
-/*
- * Issue a read against the cache.
- * - Eats the caller's ref on subreq.
+/**
+ * netfs_prepare_read_buffer - Get the buffer for a subrequest
+ * @subreq: The subrequest to get the buffer for
+ * @max_segs: Maximum number of segments in buffer (or INT_MAX)
+ *
+ * Extract a slice of buffer from the stream and attach it to the subrequest as
+ * a bio_vec queue.  The maximum amount of data attached is set by
+ * @subreq->len, but this may be shortened if @max_segs would be exceeded.
+ *
+ * [!] NOTE: This must be run in the same thread as ->issue_read() was called
+ * in as we access the readahead_control struct if there is one.
  */
-static void netfs_read_cache_to_pagecache(struct netfs_io_request *rreq,
-					  struct netfs_io_subrequest *subreq)
+int netfs_prepare_read_buffer(struct netfs_io_subrequest *subreq,
+			      unsigned int max_segs)
 {
-	struct netfs_cache_resources *cres = &rreq->cache_resources;
-
-	netfs_stat(&netfs_n_rh_read);
-	cres->ops->read(cres, subreq->start, &subreq->io_iter, NETFS_READ_HOLE_IGNORE,
-			netfs_cache_read_terminated, subreq);
+	switch (subreq->rreq->origin) {
+	case NETFS_READAHEAD:
+	case NETFS_READPAGE:
+	case NETFS_READ_FOR_WRITE:
+		if (subreq->retry_count)
+			return netfs_prepare_buffered_read_retry_buffer(subreq, max_segs);
+		return netfs_prepare_buffered_read_buffer(subreq, max_segs);
+
+	case NETFS_UNBUFFERED_READ:
+	case NETFS_DIO_READ:
+	case NETFS_READ_GAPS:
+		return netfs_prepare_unbuffered_read_buffer(subreq, max_segs);
+	case NETFS_READ_SINGLE:
+		return netfs_prepare_read_single_buffer(subreq, max_segs);
+	default:
+		WARN_ON_ONCE(1);
+		return -EIO;
+	}
 }
+EXPORT_SYMBOL(netfs_prepare_read_buffer);
 
-int netfs_read_query_cache(struct netfs_io_request *rreq, struct fscache_occupancy *occ)
+void netfs_read_query_cache(struct netfs_io_request *rreq, struct fscache_occupancy *occ)
 {
 	struct netfs_cache_resources *cres = &rreq->cache_resources;
 
 	occ->granularity = PAGE_SIZE;
 	if (occ->query_from >= occ->query_to)
-		return 0;
+		return;
 	if (!cres->ops)
-		return 0;
+		return;
 	occ->query_from = round_up(occ->query_from, occ->granularity);
-	return cres->ops->query_occupancy(cres, occ);
+	cres->ops->query_occupancy(cres, occ);
 }
 
-void netfs_queue_read(struct netfs_io_request *rreq,
-		      struct netfs_io_subrequest *subreq)
+/*
+ * Allocate and prepare a read subrequest.
+ */
+struct netfs_io_subrequest *netfs_alloc_read_subrequest(struct netfs_io_request *rreq)
 {
+	struct netfs_io_subrequest *subreq;
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
 
+	subreq = netfs_alloc_subrequest(rreq);
+	if (!subreq)
+		return subreq;
+
 	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 
 	/* We add to the end of the list whilst the collector may be walking
@@ -182,31 +210,51 @@ void netfs_queue_read(struct netfs_io_request *rreq,
 	}
 
 	spin_unlock(&rreq->lock);
+	return subreq;
 }
 
 static void netfs_issue_read(struct netfs_io_request *rreq,
 			     struct netfs_io_subrequest *subreq)
 {
-	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
-			    subreq->content.slot, subreq->content.offset, subreq->len);
+	struct netfs_cache_resources *cres = &rreq->cache_resources;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	int ret;
+
+	_enter("R=%08x[%x]", rreq->debug_id, subreq->debug_index);
 
 	switch (subreq->source) {
 	case NETFS_DOWNLOAD_FROM_SERVER:
-		rreq->netfs_ops->issue_read(subreq);
-		break;
+		ret = rreq->netfs_ops->issue_read(subreq);
+		if (ret < 0)
+			goto fail;
+		return;
 	case NETFS_READ_FROM_CACHE:
-		netfs_read_cache_to_pagecache(rreq, subreq);
-		break;
+		netfs_stat(&netfs_n_rh_read);
+		ret = cres->ops->issue_read(subreq);
+		if (ret < 0)
+			goto fail;
+		return;
 	default:
-		bvecq_zero(&subreq->content, subreq->len);
+		WARN_ON_ONCE(1);
+		fallthrough;
+	case NETFS_FILL_WITH_ZEROES:
+		stream->issue_from = subreq->start + subreq->len;
+		stream->buffered -= subreq->len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+		if (!stream->buffered)
+			netfs_all_subreqs_queued(rreq);
+		bvecq_zero(&stream->dispatch_cursor, subreq->len);
 		subreq->transferred = subreq->len;
 		subreq->error = 0;
-		iov_iter_zero(subreq->len, &subreq->io_iter);
-		subreq->transferred = subreq->len;
-		netfs_read_subreq_terminated(subreq);
-		break;
+		return netfs_read_subreq_terminated(subreq);
 	}
+
+fail:
+	/* Ownership of subreq was returned to us. */
+	trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+	stream->buffered -= subreq->len;
+	subreq->error = ret;
+	netfs_read_subreq_terminated(subreq);
 }
 
 /*
@@ -225,20 +273,17 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		.cached_to[1]	= ULLONG_MAX,
 	};
 	struct fscache_occupancy *occ = &_occ;
-	unsigned long long start = rreq->start;
-	ssize_t size = rreq->len;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
 	int ret = 0;
 
 	_enter("R=%08x", rreq->debug_id);
 
-	bvecq_pos_set(&rreq->dispatch_cursor, &rreq->load_cursor);
-	bvecq_pos_set(&rreq->collect_cursor, &rreq->dispatch_cursor);
+	bvecq_pos_set(&stream->dispatch_cursor, &rreq->load_cursor);
+	bvecq_pos_set(&rreq->collect_cursor, &rreq->load_cursor);
 
 	do {
-		int (*prepare_read)(struct netfs_io_subrequest *subreq) = NULL;
 		struct netfs_io_subrequest *subreq;
-		unsigned long long hole_to, cache_to;
-		ssize_t slice;
+		unsigned long long hole_to, cache_to, stop;
 
 		/* If we don't have any, find out the next couple of data
 		 * extents from the cache, containing of following the
@@ -247,7 +292,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		 */
 		hole_to = occ->cached_from[0];
 		cache_to = occ->cached_to[0];
-		if (start >= cache_to) {
+		if (stream->issue_from >= cache_to) {
 			/* Extent exhausted; shuffle down. */
 			int i;
 
@@ -262,52 +307,50 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 			if (occ->cached_from[0] != ULLONG_MAX)
 				continue;
 
-			/* Get new extents */
-			ret = netfs_read_query_cache(rreq, occ);
-			if (ret < 0)
-				break;
+			/* Wait for the cache to enter the readable state and
+			 * get more extents.  If the cache doesn't become
+			 * readable, this leaves the default negative extent in
+			 * place.
+			 */
+			netfs_read_query_cache(rreq, occ);
 			continue;
 		}
 
-		subreq = netfs_alloc_subrequest(rreq);
+		subreq = netfs_alloc_read_subrequest(rreq);
 		if (!subreq) {
 			ret = -ENOMEM;
 			break;
 		}
 
-		subreq->start	= start;
-		subreq->len	= size;
-
-		netfs_queue_read(rreq, subreq);
+		subreq->start = stream->issue_from;
+		stop = stream->issue_from + stream->buffered;
 
 		unsigned long long zero_point = netfs_read_zero_point(rreq->inode);
 		unsigned long long zlimit = umin(zero_point, rreq->i_size);
 
 		_debug("rsub %llx %llx-%llx", subreq->start, hole_to, cache_to);
 
-		if (start >= hole_to && start < cache_to) {
+		if (stream->issue_from >= hole_to && stream->issue_from < cache_to) {
 			/* Overlap with a cached region, where the cache may
 			 * record a block of zeroes.
 			 */
-			_debug("cached s=%llx c=%llx l=%zx", start, cache_to, size);
-			subreq->len = umin(cache_to - start, size);
-			subreq->len = round_up(subreq->len, occ->granularity);
+			_debug("cached s=%llx c=%llx l=%zx",
+			       stream->issue_from, cache_to, stream->buffered);
+			subreq->len = umin(cache_to - stream->issue_from, stream->buffered);
 			if (occ->cached_type[0] == FSCACHE_EXTENT_ZERO) {
 				subreq->source = NETFS_FILL_WITH_ZEROES;
 				netfs_stat(&netfs_n_rh_zero);
 			} else {
+				subreq->len = round_up(subreq->len, occ->granularity);
 				subreq->source = NETFS_READ_FROM_CACHE;
-				prepare_read = rreq->cache_resources.ops->prepare_read;
 			}
-
-			trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-
-		} else if (subreq->start >= zlimit && size > 0) {
+		} else if (subreq->start >= zlimit &&
+			   subreq->start < stop) {
 			/* If this range lies beyond the zero-point, that part
 			 * can just be cleared locally.
 			 */
-			_debug("zero %llx-%llx", start, start + size);
-			subreq->len = size;
+			_debug("zero %llx-%llx", subreq->start, stop);
+			subreq->len = stream->buffered;
 			subreq->source = NETFS_FILL_WITH_ZEROES;
 			if (rreq->cache_resources.ops)
 				__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
@@ -317,10 +360,10 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 			 * this range lies beyond the zero-point or the EOF,
 			 * that part can just be cleared locally.
 			 */
-			unsigned long long limit = min3(zlimit, start + size, hole_to);
+			unsigned long long limit = min3(zlimit, stop, hole_to);
 
 			_debug("limit %llx %llx", rreq->i_size, zero_point);
-			_debug("download %llx-%llx", start, start + size);
+			_debug("download %llx-%llx", subreq->start, stop);
 			subreq->len = umin(limit - subreq->start, ULONG_MAX);
 			subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
 			if (rreq->cache_resources.ops)
@@ -328,41 +371,15 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 			netfs_stat(&netfs_n_rh_download);
 		}
 
-		if (size == 0) {
+		if (subreq->len == 0) {
 			pr_err("ZERO-LEN READ: R=%08x[%x] l=%zx/%zx s=%llx z=%llx i=%llx",
 			       rreq->debug_id, subreq->debug_index,
-			       subreq->len, size,
+			       subreq->len, stream->buffered,
 			       subreq->start, zero_point, rreq->i_size);
 			netfs_cancel_read(subreq, ret);
 			break;
 		}
 
-		rreq->io_streams[0].sreq_max_len = MAX_RW_COUNT;
-		rreq->io_streams[0].sreq_max_segs = INT_MAX;
-
-		if (prepare_read) {
-			ret = prepare_read(subreq);
-			if (ret < 0) {
-				netfs_cancel_read(subreq, ret);
-				break;
-			}
-			trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
-		}
-
-		slice = netfs_prepare_read_iterator(subreq);
-		if (slice < 0) {
-			ret = slice;
-			netfs_cancel_read(subreq, ret);
-			break;
-		}
-		start += slice;
-		size -= slice;
-		if (size <= 0) {
-			smp_wmb(); /* Write lists before ALL_QUEUED. */
-			set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
-		}
-
-		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
 		netfs_issue_read(rreq, subreq);
 		netfs_maybe_bulk_drop_ra_refs(rreq);
 
@@ -371,19 +388,19 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		if (test_bit(NETFS_RREQ_FAILED, &rreq->flags))
 			break;
 		cond_resched();
-	} while (size > 0);
+	} while (stream->buffered > 0);
 
-	if (unlikely(size > 0)) {
-		smp_wmb(); /* Write lists before ALL_QUEUED. */
-		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+	if (unlikely(!netfs_are_all_subreqs_queued(rreq))) {
+		netfs_all_subreqs_queued(rreq);
 		netfs_wake_collector(rreq);
 	}
 
 	/* Defer error return as we may need to wait for outstanding I/O. */
-	cmpxchg(&rreq->error, 0, ret);
+	if (ret < 0)
+		cmpxchg(&rreq->error, 0, ret);
 
 	bvecq_pos_unset(&rreq->load_cursor);
-	bvecq_pos_unset(&rreq->dispatch_cursor);
+	bvecq_pos_unset(&stream->dispatch_cursor);
 }
 
 /**
@@ -404,17 +421,22 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 void netfs_readahead(struct readahead_control *ractl)
 {
 	struct netfs_io_request *rreq;
+	struct netfs_io_stream *stream;
 	struct netfs_inode *ictx = netfs_inode(ractl->mapping->host);
 	unsigned long long start = readahead_pos(ractl);
 	ssize_t added;
 	size_t size = readahead_length(ractl);
 	int ret;
 
+	_enter("");
+
 	rreq = netfs_alloc_request(ractl->mapping, ractl->file, start, size,
 				   NETFS_READAHEAD);
 	if (IS_ERR(rreq))
 		return;
 
+	stream = &rreq->io_streams[0];
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &rreq->flags);
 
 	ret = netfs_begin_cache_read(rreq, ictx);
@@ -439,8 +461,8 @@ void netfs_readahead(struct readahead_control *ractl)
 	__set_bit(NETFS_RREQ_NEED_PUT_RA_REFS, &rreq->flags);
 
 	rreq->submitted = rreq->start + added;
-	rreq->cleaned_to = rreq->start;
 	rreq->progress_at = rreq->load_cursor.bvecq->bv[0].bv_len;
+	stream->buffered = added;
 
 	netfs_read_to_pagecache(rreq);
 	netfs_maybe_bulk_drop_ra_refs(rreq);
@@ -456,6 +478,7 @@ EXPORT_SYMBOL(netfs_readahead);
  */
 static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio)
 {
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
 	struct bvecq *bq;
 	size_t fsize = folio_size(folio);
 
@@ -467,6 +490,7 @@ static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct fo
 	bvecq_filled_to(bq, 1);
 	rreq->submitted = rreq->start + fsize;
 	rreq->progress_at = fsize;
+	stream->buffered = fsize;
 	return 0;
 }
 
@@ -476,6 +500,7 @@ static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct fo
 static int netfs_read_gaps(struct file *file, struct folio *folio)
 {
 	struct netfs_io_request *rreq;
+	struct netfs_io_stream *stream;
 	struct address_space *mapping = folio->mapping;
 	struct netfs_group *group = netfs_folio_group(folio);
 	struct netfs_folio *finfo = netfs_folio_info(folio);
@@ -497,6 +522,7 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 		ret = PTR_ERR(rreq);
 		goto alloc_error;
 	}
+	stream = &rreq->io_streams[0];
 
 	ret = netfs_begin_cache_read(rreq, ctx);
 	if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
@@ -551,6 +577,7 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)
 	bvecq_filled_to(bq, slot);
 
 	rreq->submitted = rreq->start + flen;
+	stream->buffered = flen;
 
 	netfs_read_to_pagecache(rreq);
 
@@ -629,6 +656,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
 		goto discard;
 
 	netfs_read_to_pagecache(rreq);
+
 	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret < 0 ? ret : 0;
diff --git a/fs/netfs/direct_read.c b/fs/netfs/direct_read.c
index 4d35f8efc633..5b19145ee4dd 100644
--- a/fs/netfs/direct_read.c
+++ b/fs/netfs/direct_read.c
@@ -16,6 +16,32 @@
 #include <linux/netfs.h>
 #include "internal.h"
 
+int netfs_prepare_unbuffered_read_buffer(struct netfs_io_subrequest *subreq,
+					 unsigned int max_segs)
+{
+	struct netfs_io_request *rreq = subreq->rreq;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &stream->dispatch_cursor);
+	len = bvecq_slice(&stream->dispatch_cursor, subreq->len, max_segs,
+			  &subreq->nr_segs);
+
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+
+	stream->buffered   -= subreq->len;
+	stream->issue_from += subreq->len;
+	rreq->submitted = stream->issue_from;
+
+	if (stream->buffered == 0)
+		netfs_all_subreqs_queued(rreq);
+	return 0;
+}
+
 /*
  * Perform a read to a buffer from the server, slicing up the region to be read
  * according to the network rsize.
@@ -23,17 +49,15 @@
 static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 {
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
-	unsigned long long start = rreq->start;
-	ssize_t size = rreq->len;
-	int ret;
 
-	bvecq_pos_set(&rreq->dispatch_cursor, &rreq->load_cursor);
-	bvecq_pos_transfer(&rreq->collect_cursor, &rreq->load_cursor);
+	bvecq_pos_set(&rreq->collect_cursor, &rreq->load_cursor);
+	bvecq_pos_transfer(&stream->dispatch_cursor, &rreq->load_cursor);
 
 	do {
 		struct netfs_io_subrequest *subreq;
+		int ret;
 
-		subreq = netfs_alloc_subrequest(rreq);
+		subreq = netfs_alloc_read_subrequest(rreq);
 		if (!subreq) {
 			/* Stash the error in the request if there's not
 			 * already an error set.
@@ -43,54 +67,32 @@ static void netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)
 		}
 
 		subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
-		subreq->start	= start;
-		subreq->len	= size;
-
-		netfs_queue_read(rreq, subreq);
+		subreq->start	= stream->issue_from;
+		subreq->len	= stream->buffered;
 
 		netfs_stat(&netfs_n_rh_download);
-		if (rreq->netfs_ops->prepare_read) {
-			ret = rreq->netfs_ops->prepare_read(subreq);
-			if (ret < 0) {
-				netfs_cancel_read(subreq, ret);
-				break;
-			}
-		}
 
-		bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
-		bvecq_pos_set(&subreq->content, &rreq->dispatch_cursor);
-		subreq->len = bvecq_slice(&rreq->dispatch_cursor,
-					  umin(size, stream->sreq_max_len),
-					  stream->sreq_max_segs,
-					  &subreq->nr_segs);
-
-		size -= subreq->len;
-		start += subreq->len;
-		rreq->submitted += subreq->len;
-		if (size <= 0) {
-			smp_wmb(); /* Write lists before ALL_QUEUED. */
-			set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+		ret = rreq->netfs_ops->issue_read(subreq);
+		if (ret < 0) {
+			stream->buffered -= subreq->len;
+			subreq->error = ret;
+			netfs_read_subreq_terminated(subreq);
+			break;
 		}
 
-		iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
-				    subreq->content.slot, subreq->content.offset, subreq->len);
-
-		rreq->netfs_ops->issue_read(subreq);
-
 		if (test_bit(NETFS_RREQ_PAUSE, &rreq->flags))
 			netfs_wait_for_paused_read(rreq);
 		if (test_bit(NETFS_RREQ_FAILED, &rreq->flags))
 			break;
 		cond_resched();
-	} while (size > 0);
+	} while (stream->buffered > 0);
 
-	if (unlikely(size > 0)) {
-		smp_wmb(); /* Write lists before ALL_QUEUED. */
-		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+	if (unlikely(stream->buffered > 0)) {
+		netfs_all_subreqs_queued(rreq);
 		netfs_wake_collector(rreq);
 	}
 
-	bvecq_pos_unset(&rreq->dispatch_cursor);
+	bvecq_pos_unset(&stream->dispatch_cursor);
 }
 
 /*
@@ -141,6 +143,7 @@ static ssize_t netfs_unbuffered_read(struct netfs_io_request *rreq, bool sync)
 ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *iter)
 {
 	struct netfs_io_request *rreq;
+	struct netfs_io_stream *stream;
 	ssize_t ret;
 	size_t orig_count = iov_iter_count(iter);
 	bool sync = is_sync_kiocb(iocb);
@@ -165,6 +168,8 @@ ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *i
 	netfs_stat(&netfs_n_rh_dio_read);
 	trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_dio_read);
 
+	stream = &rreq->io_streams[0];
+
 	/* If this is an async op, we have to keep track of the destination
 	 * buffer for ourselves as the caller's iterator will be trashed when
 	 * we return.
@@ -180,6 +185,8 @@ ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *i
 		goto error_put;
 
 	rreq->len = ret;
+	stream->buffered = ret;
+	stream->issue_from = rreq->start;
 
 	// TODO: Set up bounce buffer if needed
 
diff --git a/fs/netfs/direct_write.c b/fs/netfs/direct_write.c
index d81245a29eff..87f7bd02fe7c 100644
--- a/fs/netfs/direct_write.c
+++ b/fs/netfs/direct_write.c
@@ -9,6 +9,34 @@
 #include <linux/uio.h>
 #include "internal.h"
 
+/*
+ * Prepare the buffer for an unbuffered/DIO write.
+ */
+int netfs_prepare_unbuffered_write_buffer(struct netfs_io_subrequest *subreq,
+					  unsigned int max_segs)
+{
+	struct netfs_io_stream *stream = &subreq->rreq->io_streams[subreq->stream_nr];
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &stream->dispatch_cursor);
+	len = bvecq_slice(&stream->dispatch_cursor, subreq->len, max_segs,
+			  &subreq->nr_segs);
+
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+
+	// TODO: Wait here for completion of prev subreq
+
+	stream->issue_from += subreq->len;
+	stream->buffered   -= subreq->len;
+	if (stream->buffered == 0)
+		netfs_all_subreqs_queued(subreq->rreq);
+	return 0;
+}
+
 /*
  * Perform the cleanup rituals after an unbuffered write is complete.
  */
@@ -74,9 +102,9 @@ static void netfs_unbuffered_write_collect(struct netfs_io_request *wreq,
 
 	wreq->transferred += subreq->transferred;
 	if (subreq->transferred < subreq->len) {
-		bvecq_pos_unset(&wreq->dispatch_cursor);
-		bvecq_pos_transfer(&wreq->dispatch_cursor, &subreq->dispatch_pos);
-		bvecq_pos_advance(&wreq->dispatch_cursor, subreq->transferred);
+		bvecq_pos_unset(&stream->dispatch_cursor);
+		bvecq_pos_transfer(&stream->dispatch_cursor, &subreq->dispatch_pos);
+		bvecq_pos_advance(&stream->dispatch_cursor, subreq->transferred);
 	}
 
 	stream->collected_to = subreq->start + subreq->transferred;
@@ -85,6 +113,7 @@ static void netfs_unbuffered_write_collect(struct netfs_io_request *wreq,
 
 	trace_netfs_collect_stream(wreq, stream);
 	trace_netfs_collect_state(wreq, wreq->collected_to, 0);
+	/* TODO: Progressively clean up wreq->direct_bq */
 }
 
 /*
@@ -103,60 +132,44 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 
 	_enter("%llx", wreq->len);
 
-	bvecq_pos_set(&wreq->dispatch_cursor, &wreq->load_cursor);
-	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+	stream->issue_from = wreq->start;
+	stream->buffered = wreq->len;
+	bvecq_pos_set(&stream->dispatch_cursor, &wreq->load_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &stream->dispatch_cursor);
 
 	if (wreq->origin == NETFS_DIO_WRITE)
 		inode_dio_begin(wreq->inode);
 
-	stream->collected_to = wreq->start;
-
 	for (;;) {
 		bool retry = false;
 
 		if (!subreq) {
-			netfs_prepare_write(wreq, stream, wreq->start + wreq->transferred);
-			subreq = stream->construct;
-			stream->construct = NULL;
-		} else {
-			bvecq_pos_set(&subreq->dispatch_pos, &wreq->dispatch_cursor);
+			subreq = netfs_alloc_write_subreq(wreq, stream);
+			if (!subreq)
+				return -ENOMEM;
 		}
 
-		/* Check if (re-)preparation failed. */
-		if (unlikely(test_bit(NETFS_SREQ_FAILED, &subreq->flags))) {
-			netfs_write_subrequest_terminated(subreq, subreq->error);
-			wreq->error = subreq->error;
-			break;
+		ret = stream->issue_write(subreq);
+		if (ret < 0) {
+			/* Ownership of subreq was returned to us. */
+			trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+			netfs_write_subrequest_terminated(subreq, ret);
 		}
 
-		subreq->len = bvecq_slice(&wreq->dispatch_cursor, stream->sreq_max_len,
-					  stream->sreq_max_segs, &subreq->nr_segs);
-		bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-
-		iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
-				    subreq->content.bvecq, subreq->content.slot,
-				    subreq->content.offset,
-				    subreq->len);
-
-		if (!iov_iter_count(&subreq->io_iter))
-			break;
-
-		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
-		stream->issue_write(subreq);
-
-		/* Async, need to wait. */
-		netfs_wait_for_in_progress_stream(wreq, stream);
-
-		if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
+		ret = netfs_wait_for_in_progress_subreq(wreq, subreq);
+		if (ret < 0) {
+			if (ret != -EAGAIN) {
+				/* Don't need to lock here as the collection is
+				 * done in this thread.
+				 */
+				list_del_init(&subreq->rreq_link);
+				ret = subreq->error;
+				netfs_put_subrequest(subreq, netfs_sreq_trace_put_failed);
+				subreq = NULL;
+				goto failed;
+			}
 			retry = true;
-		} else if (test_bit(NETFS_SREQ_FAILED, &subreq->flags)) {
-			ret = subreq->error;
-			wreq->error = ret;
-			netfs_see_subrequest(subreq, netfs_sreq_trace_see_failed);
-			subreq = NULL;
-			break;
 		}
-		ret = 0;
 
 		if (!retry) {
 			netfs_unbuffered_write_collect(wreq, stream, subreq);
@@ -171,20 +184,21 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 			continue;
 		}
 
-		/* We need to retry the last subrequest, so first reset the
-		 * iterator, taking into account what, if anything, we managed
-		 * to transfer.
+		/* We need to retry the last subrequest, so first wind back the
+		 * buffer position.
 		 */
 		subreq->error = -EAGAIN;
 		trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
 		bvecq_pos_unset(&subreq->content);
-		bvecq_pos_unset(&wreq->dispatch_cursor);
-		bvecq_pos_transfer(&wreq->dispatch_cursor, &subreq->dispatch_pos);
+		bvecq_pos_unset(&stream->dispatch_cursor);
+		bvecq_pos_transfer(&stream->dispatch_cursor, &subreq->dispatch_pos);
 
+		stream->issue_from -= subreq->len - subreq->transferred;
+		stream->buffered   += subreq->len - subreq->transferred;
 		if (subreq->transferred > 0) {
-			wreq->transferred += subreq->transferred;
-			bvecq_pos_advance(&wreq->dispatch_cursor, subreq->transferred);
+			wreq->transferred  += subreq->transferred;
+			bvecq_pos_advance(&stream->dispatch_cursor, subreq->transferred);
 		}
 
 		if (stream->source == NETFS_UPLOAD_TO_SERVER &&
@@ -193,25 +207,21 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 
 		__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 		__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
-		__clear_bit(NETFS_SREQ_BOUNDARY, &subreq->flags);
 		__clear_bit(NETFS_SREQ_FAILED, &subreq->flags);
-		subreq->start		= wreq->start + wreq->transferred;
-		subreq->len		= wreq->len   - wreq->transferred;
+		__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+		subreq->start		= stream->issue_from;
+		subreq->len		= stream->buffered;
 		subreq->transferred	= 0;
 		subreq->retry_count	+= 1;
-		stream->sreq_max_len	= UINT_MAX;
-		stream->sreq_max_segs	= INT_MAX;
 
 		netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
 
-		if (stream->prepare_write)
-			stream->prepare_write(subreq);
 		__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 		netfs_stat(&netfs_n_wh_retry_write_subreq);
 	}
 
-	bvecq_pos_unset(&wreq->dispatch_cursor);
-	bvecq_pos_unset(&wreq->load_cursor);
+failed:
+	bvecq_pos_unset(&stream->dispatch_cursor);
 	netfs_unbuffered_write_done(wreq);
 	_leave(" = %d", ret);
 	return ret;
@@ -255,6 +265,7 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 	if (IS_ERR(wreq))
 		return PTR_ERR(wreq);
 
+	wreq->len = iov_iter_count(iter);
 	wreq->io_streams[0].avail = true;
 	trace_netfs_write(wreq, (iocb->ki_flags & IOCB_DIRECT ?
 				 netfs_write_trace_dio_write :
@@ -265,9 +276,7 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 		 * we have to save the source buffer as the iterator is only
 		 * good until we return.  In such a case, extract an iterator
 		 * to represent as much of the the output buffer as we can
-		 * manage.  Note that the extraction might not be able to
-		 * allocate a sufficiently large bvec array and may shorten the
-		 * request.
+		 * manage.  Note that the extraction may shorten the request.
 		 */
 		ssize_t n = netfs_extract_iter(iter, len, INT_MAX, iocb->ki_pos,
 					       &wreq->load_cursor.bvecq, 0, wreq->gfp);
@@ -282,8 +291,6 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 		       wreq->load_cursor.bvecq->max_slots);
 	}
 
-	__set_bit(NETFS_RREQ_USE_IO_ITER, &wreq->flags);
-
 	/* Copy the data into the bounce buffer and encrypt it. */
 	// TODO
 
diff --git a/fs/netfs/fscache_io.c b/fs/netfs/fscache_io.c
index fafa8c6bec57..a04cd96cd09a 100644
--- a/fs/netfs/fscache_io.c
+++ b/fs/netfs/fscache_io.c
@@ -239,7 +239,7 @@ void __fscache_write_to_cache(struct fscache_cookie *cookie,
 				    fscache_access_io_write) < 0)
 		goto abandon_free;
 
-	ret = cres->ops->prepare_write(cres, &start, &len, len, i_size, false);
+	ret = cres->ops->prepare_write_old(cres, &start, &len, len, i_size, false);
 	if (ret < 0)
 		goto abandon_end;
 
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 315b02f5fa67..52223743a2df 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -22,10 +22,9 @@
 /*
  * buffered_read.c
  */
-int netfs_read_query_cache(struct netfs_io_request *rreq,
-			   struct fscache_occupancy *occ);
-void netfs_queue_read(struct netfs_io_request *rreq,
-		      struct netfs_io_subrequest *subreq);
+void netfs_read_query_cache(struct netfs_io_request *rreq,
+			    struct fscache_occupancy *occ);
+struct netfs_io_subrequest *netfs_alloc_read_subrequest(struct netfs_io_request *rreq);
 void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error);
 int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 			     size_t offset, size_t len);
@@ -36,6 +35,18 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 void netfs_update_i_size(struct netfs_inode *ctx, struct inode *inode,
 			 loff_t pos, size_t copied);
 
+/*
+ * direct_read.c
+ */
+int netfs_prepare_unbuffered_read_buffer(struct netfs_io_subrequest *subreq,
+					 unsigned int max_segs);
+
+/*
+ * direct_write.c
+ */
+int netfs_prepare_unbuffered_write_buffer(struct netfs_io_subrequest *subreq,
+					  unsigned int max_segs);
+
 /*
  * main.c
  */
@@ -73,6 +84,8 @@ struct bvecq *netfs_buffer_make_space(struct netfs_io_request *rreq,
 				      enum netfs_bvecq_trace trace);
 void netfs_wake_collector(struct netfs_io_request *rreq);
 void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq);
+int netfs_wait_for_in_progress_subreq(struct netfs_io_request *rreq,
+				      struct netfs_io_subrequest *subreq);
 void netfs_wait_for_in_progress_stream(struct netfs_io_request *rreq,
 				       struct netfs_io_stream *stream);
 ssize_t netfs_wait_for_read(struct netfs_io_request *rreq);
@@ -118,16 +131,53 @@ void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error);
 /*
  * read_pgpriv2.c
  */
+#ifdef CONFIG_NETFS_PGPRIV2
+int netfs_prepare_pgpriv2_write_buffer(struct netfs_io_subrequest *subreq,
+				       unsigned int max_segs);
 void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *folio);
 void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq);
 bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *wreq);
+static inline bool netfs_using_pgpriv2(const struct netfs_io_request *rreq)
+{
+	return test_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags);
+}
+#else
+static inline int netfs_prepare_pgpriv2_write_buffer(struct netfs_io_subrequest *subreq,
+						     unsigned int max_segs)
+{
+	return -EIO;
+}
+static inline void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *folio)
+{
+}
+static inline void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
+{
+}
+static inline bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *wreq)
+{
+	return true;
+}
+static inline bool netfs_using_pgpriv2(const struct netfs_io_request *rreq)
+{
+	return false;
+}
+#endif
 
 /*
  * read_retry.c
  */
+int netfs_prepare_buffered_read_retry_buffer(struct netfs_io_subrequest *subreq,
+					     unsigned int max_segs);
+int netfs_reset_for_read_retry(struct netfs_io_subrequest *subreq);
 void netfs_retry_reads(struct netfs_io_request *rreq);
 void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq);
 
+/*
+ * read_single.c
+ */
+int netfs_prepare_read_single_buffer(struct netfs_io_subrequest *subreq,
+				     unsigned int max_segs);
+
 /*
  * stats.c
  */
@@ -198,24 +248,19 @@ void netfs_write_collection_worker(struct work_struct *work);
 /*
  * write_issue.c
  */
+struct netfs_writethrough;
 struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 						struct file *file,
 						loff_t start,
 						enum netfs_io_origin origin);
-void netfs_prepare_write(struct netfs_io_request *wreq,
-			 struct netfs_io_stream *stream,
-			 loff_t start);
-void netfs_reissue_write(struct netfs_io_stream *stream,
-			 struct netfs_io_subrequest *subreq);
-void netfs_issue_write(struct netfs_io_request *wreq,
-		       struct netfs_io_stream *stream);
-size_t netfs_advance_write(struct netfs_io_request *wreq,
-			   struct netfs_io_stream *stream,
-			   loff_t start, size_t len, bool to_eof);
+struct netfs_io_subrequest *netfs_alloc_write_subreq(struct netfs_io_request *wreq,
+						     struct netfs_io_stream *stream);
 
 /*
  * write_retry.c
  */
+int netfs_prepare_write_retry_buffer(struct netfs_io_subrequest *subreq,
+				     unsigned int max_segs);
 void netfs_retry_writes(struct netfs_io_request *wreq);
 
 /*
@@ -310,6 +355,25 @@ static inline bool netfs_check_subreq_in_progress(const struct netfs_io_subreque
 	return test_bit_acquire(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 }
 
+/*
+ * Indicate that we've generated and queued all the subrequests we're going to.
+ */
+static inline void netfs_all_subreqs_queued(struct netfs_io_request *rreq)
+{
+	smp_wmb(); /* Write lists before ALL_QUEUED. */
+	set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+	trace_netfs_rreq(rreq, netfs_rreq_trace_all_queued);
+}
+
+/*
+ * Query if all subrequests are queued.
+ */
+static inline bool netfs_are_all_subreqs_queued(const struct netfs_io_request *rreq)
+{
+	/* Read lists after ALL_QUEUED. */
+	return test_bit_acquire(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
+}
+
 /*
  * fscache-cache.c
  */
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index 271ede100746..81e1197396cd 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -108,14 +108,14 @@ ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pag
 			}
 
 			if (got == 0) {
-				pr_err("extract_pages gave nothing from %zu, %zu\n",
+				pr_err("extract_pages gave nothing from %zx, %zx\n",
 				       extracted, max_len);
 				ret = -EIO;
 				goto out;
 			}
 
 			if (WARN(got > max_len,
-				 "%s: extract_pages overrun %zd > %zu bytes\n",
+				 "%s: extract_pages overrun %zx > %zx bytes\n",
 				 __func__, got, max_len)) {
 				ret = -EIO;
 				goto out;
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index 8fc4e5ef2152..0af45204fabc 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -250,6 +250,37 @@ void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq)
 		netfs_wake_collector(rreq);
 }
 
+/*
+ * Wait for a subrequest to come to completion.
+ */
+int netfs_wait_for_in_progress_subreq(struct netfs_io_request *rreq,
+				      struct netfs_io_subrequest *subreq)
+{
+	if (netfs_check_subreq_in_progress(subreq)) {
+		DEFINE_WAIT(myself);
+
+		trace_netfs_rreq(rreq, netfs_rreq_trace_wait_quiesce);
+		for (;;) {
+			prepare_to_wait(&rreq->waitq, &myself, TASK_UNINTERRUPTIBLE);
+
+			if (!netfs_check_subreq_in_progress(subreq))
+				break;
+
+			trace_netfs_sreq(subreq, netfs_sreq_trace_wait_for);
+			schedule();
+		}
+
+		trace_netfs_rreq(rreq, netfs_rreq_trace_waited_quiesce);
+		finish_wait(&rreq->waitq, &myself);
+	}
+
+	if (test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
+		return -EAGAIN;
+	if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
+		return subreq->error;
+	return 0;
+}
+
 /*
  * Wait for all outstanding I/O in a stream to quiesce.
  */
@@ -310,7 +341,7 @@ static int netfs_collect_in_app(struct netfs_io_request *rreq,
 			need_collect = true;
 			break;
 		}
-		if (subreq || !test_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags))
+		if (subreq || !netfs_are_all_subreqs_queued(rreq))
 			done = false;
 	}
 
@@ -380,7 +411,7 @@ static ssize_t netfs_wait_for_in_progress(struct netfs_io_request *rreq,
 		case NETFS_UNBUFFERED_WRITE:
 			break;
 		default:
-			if (rreq->submitted < rreq->len) {
+			if (rreq->transferred < rreq->len) {
 				trace_netfs_failure(rreq, NULL, ret, netfs_fail_short_read);
 				ret = -EIO;
 			}
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index 40ba1f058bc7..95e33849f665 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -55,8 +55,6 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
 	rreq->i_size		= i_size_read(inode);
 	rreq->debug_id		= atomic_inc_return(&debug_ids);
 	rreq->wsize		= INT_MAX;
-	rreq->io_streams[0].sreq_max_len = ULONG_MAX;
-	rreq->io_streams[0].sreq_max_segs = 0;
 	spin_lock_init(&rreq->lock);
 	init_waitqueue_head(&rreq->waitq);
 	refcount_set(&rreq->ref, 2);
@@ -66,6 +64,8 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
 
 		INIT_LIST_HEAD(&stream->subrequests);
 		stream->collected_to = rreq->start;
+		stream->issue_from = rreq->start;
+		stream->alignment = 1;
 	}
 
 	if (origin == NETFS_READAHEAD ||
@@ -148,9 +148,11 @@ static void netfs_deinit_request(struct netfs_io_request *rreq)
 	if (rreq->cache_resources.ops)
 		rreq->cache_resources.ops->end_operation(&rreq->cache_resources);
 	bvecq_pos_unset(&rreq->load_cursor);
-	bvecq_pos_unset(&rreq->dispatch_cursor);
 	bvecq_pos_unset(&rreq->collect_cursor);
+	bvecq_pos_unset(&rreq->retry_cursor);
 	bvecq_put(rreq->spare);
+	for (int i = 0; i < NR_IO_STREAMS; i++)
+		bvecq_pos_unset(&rreq->io_streams[i].dispatch_cursor);
 
 	if (atomic_dec_and_test(&ictx->io_count))
 		wake_up_var(&ictx->io_count);
@@ -240,6 +242,7 @@ static void netfs_free_subrequest(struct netfs_io_subrequest *subreq)
 	struct netfs_io_request *rreq = subreq->rreq;
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_free);
+	WARN_ON_ONCE(!list_empty(&subreq->rreq_link));
 	if (rreq->netfs_ops->free_subrequest)
 		rreq->netfs_ops->free_subrequest(subreq);
 	bvecq_pos_unset(&subreq->dispatch_pos);
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 21eb7f74d66e..e1352068b77f 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -36,6 +36,7 @@ static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
 
 	if (subreq->start + subreq->transferred >= subreq->rreq->i_size)
 		__set_bit(NETFS_SREQ_HIT_EOF, &subreq->flags);
+	trace_netfs_rreq(subreq->rreq, netfs_rreq_trace_zero_unread);
 }
 
 /*
@@ -58,7 +59,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 	flush_dcache_folio(folio);
 	folio_mark_uptodate(folio);
 
-	if (!test_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags)) {
+	if (!netfs_using_pgpriv2(rreq)) {
 		finfo = netfs_folio_info(folio);
 		if (finfo) {
 			trace_netfs_folio(folio, netfs_folio_trace_filled_gaps);
@@ -374,8 +375,7 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 			if (test_bit(NETFS_SREQ_COPY_TO_CACHE, &front_flags))
 				notes |= COPY_TO_CACHE;
 
-			if (front->start + transferred >= unlock_at ||
-			    test_bit(NETFS_SREQ_HIT_EOF, &front_flags))
+			if (front->start + transferred >= unlock_at)
 				netfs_read_unlock_folios(rreq, &notes);
 		} else {
 			stream->collected_to = front->start + transferred;
@@ -467,31 +467,6 @@ static void netfs_rreq_assess_dio(struct netfs_io_request *rreq)
 		inode_dio_end(rreq->inode);
 }
 
-/*
- * Do processing after reading a monolithic single object.
- */
-static void netfs_rreq_assess_single(struct netfs_io_request *rreq)
-{
-	struct netfs_io_stream *stream = &rreq->io_streams[0];
-
-	if (!rreq->error && stream->source == NETFS_DOWNLOAD_FROM_SERVER &&
-	    fscache_resources_valid(&rreq->cache_resources)) {
-		trace_netfs_rreq(rreq, netfs_rreq_trace_dirty);
-		netfs_single_mark_inode_dirty(rreq->inode);
-	}
-
-	if (rreq->iocb) {
-		rreq->iocb->ki_pos += rreq->transferred;
-		if (rreq->iocb->ki_complete) {
-			trace_netfs_rreq(rreq, netfs_rreq_trace_ki_complete);
-			rreq->iocb->ki_complete(
-				rreq->iocb, rreq->error ? rreq->error : rreq->transferred);
-		}
-	}
-	if (rreq->netfs_ops->done)
-		rreq->netfs_ops->done(rreq);
-}
-
 /*
  * Perform the collection of subrequests and folios.
  *
@@ -507,9 +482,8 @@ bool netfs_read_collection(struct netfs_io_request *rreq)
 	/* We're done when the app thread has finished posting subreqs and the
 	 * queue is empty.
 	 */
-	if (!test_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags))
+	if (!netfs_are_all_subreqs_queued(rreq))
 		return false;
-	smp_rmb(); /* Read ALL_QUEUED before subreq lists. */
 
 	if (!list_empty(&stream->subrequests))
 		return false;
@@ -527,7 +501,7 @@ bool netfs_read_collection(struct netfs_io_request *rreq)
 		netfs_rreq_assess_dio(rreq);
 		break;
 	case NETFS_READ_SINGLE:
-		netfs_rreq_assess_single(rreq);
+		WARN_ON_ONCE(1);
 		break;
 	default:
 		break;
@@ -663,6 +637,11 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq)
 		} else if (test_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags)) {
 			__set_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
 			trace_netfs_sreq(subreq, netfs_sreq_trace_partial_read);
+		} else if (subreq->source == NETFS_READ_FROM_CACHE) {
+			netfs_stat(&netfs_n_rh_read_failed);
+			__set_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+			subreq->error = -ENODATA;
+			trace_netfs_sreq(subreq, netfs_sreq_trace_short);
 		} else {
 			__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
 			subreq->error = -ENODATA;
@@ -681,6 +660,8 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq)
 
 	if (unlikely(subreq->error < 0)) {
 		trace_netfs_failure(rreq, subreq, subreq->error, netfs_fail_read);
+		if (subreq->error == -ENOMEM)
+			set_bit(NETFS_RREQ_SAW_ENOMEM, &rreq->flags);
 		if (subreq->source == NETFS_READ_FROM_CACHE) {
 			netfs_stat(&netfs_n_rh_read_failed);
 			__set_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
diff --git a/fs/netfs/read_pgpriv2.c b/fs/netfs/read_pgpriv2.c
index eb7e2959358d..de70b46889ca 100644
--- a/fs/netfs/read_pgpriv2.c
+++ b/fs/netfs/read_pgpriv2.c
@@ -13,8 +13,37 @@
 #include <linux/task_io_accounting_ops.h>
 #include "internal.h"
 
+int netfs_prepare_pgpriv2_write_buffer(struct netfs_io_subrequest *subreq,
+				       unsigned int max_segs)
+{
+	struct netfs_io_request *creq = subreq->rreq;
+	struct netfs_io_stream *stream = &creq->io_streams[1];
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &stream->dispatch_cursor);
+	len = bvecq_slice(&stream->dispatch_cursor, subreq->len, max_segs,
+			  &subreq->nr_segs);
+
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+
+	// TODO: Wait here for completion of prev subreq
+
+	stream->issue_from += subreq->len;
+	stream->buffered   -= subreq->len;
+	if (stream->buffered == 0)
+		netfs_all_subreqs_queued(creq);
+	return 0;
+}
+
 /*
- * [DEPRECATED] Copy a folio to the cache with PG_private_2 set.
+ * [DEPRECATED] Copy a folio to the cache with PG_private_2 set.  Note that the
+ * folio won't necessarily be contiguous with the previous one as there might
+ * be a mixture of folios read from the cache and downloaded from the server
+ * (or just zeroed).
  */
 static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio *folio)
 {
@@ -24,7 +53,6 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	size_t dio_size = PAGE_SIZE;
 	size_t fsize = folio_size(folio), flen = fsize;
 	loff_t fpos = folio_pos(folio), i_size;
-	bool to_eof = false;
 
 	_enter("");
 
@@ -44,12 +72,8 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	if (fpos + fsize > creq->i_size)
 		creq->i_size = i_size;
 
-	if (flen > i_size - fpos) {
+	if (flen > i_size - fpos)
 		flen = i_size - fpos;
-		to_eof = true;
-	} else if (flen == i_size - fpos) {
-		to_eof = true;
-	}
 
 	flen = round_up(flen, dio_size);
 
@@ -82,37 +106,9 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	creq->load_cursor.slot = slot;
 	creq->load_cursor.offset = 0;
 	creq->last_end = fpos + flen;
+	trace_netfs_wback(creq, folio, 0);
 
-	bvecq_pos_nudge(&creq->dispatch_cursor);
-	
-	cache->submit_off = 0;
-	cache->submit_len = flen;
-
-	/* Attach the folio to one or more subrequests.  For a big folio, we
-	 * could end up with thousands of subrequests if the wsize is small -
-	 * but we might need to wait during the creation of subrequests for
-	 * network resources (eg. SMB credits).
-	 */
-	do {
-		ssize_t part;
-
-		creq->dispatch_cursor.offset = cache->submit_off;
-
-		atomic64_set(&creq->issued_to, fpos + cache->submit_off);
-		part = netfs_advance_write(creq, cache, fpos + cache->submit_off,
-					   cache->submit_len, to_eof);
-		cache->submit_off += part;
-		if (part > cache->submit_len)
-			cache->submit_len = 0;
-		else
-			cache->submit_len -= part;
-	} while (cache->submit_len > 0);
-
-	bvecq_pos_step(&creq->dispatch_cursor);
-	atomic64_set(&creq->issued_to, fpos + fsize);
-
-	if (flen < fsize)
-		netfs_issue_write(creq, cache);
+	cache->buffered += flen;
 }
 
 /*
@@ -122,6 +118,7 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(
 	struct netfs_io_request *rreq, struct folio *folio)
 {
 	struct netfs_io_request *creq;
+	struct netfs_io_stream *cache;
 
 	if (!fscache_resources_valid(&rreq->cache_resources))
 		goto cancel;
@@ -131,13 +128,14 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(
 	if (IS_ERR(creq))
 		goto cancel;
 
-	if (!creq->io_streams[1].avail)
+	cache = &creq->io_streams[1];
+	if (!cache->avail)
 		goto cancel_put;
 
 	if (bvecq_buffer_init(&creq->load_cursor, creq->gfp, false) < 0)
 		goto cancel_put;
-	bvecq_pos_set(&creq->dispatch_cursor, &creq->load_cursor);
-	bvecq_pos_set(&creq->collect_cursor, &creq->dispatch_cursor);
+	bvecq_pos_set(&cache->dispatch_cursor, &creq->load_cursor);
+	bvecq_pos_set(&creq->collect_cursor, &creq->load_cursor);
 
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &creq->flags);
 	trace_netfs_copy2cache(rreq, creq);
@@ -180,25 +178,61 @@ void netfs_pgpriv2_copy_to_cache(struct netfs_io_request *rreq, struct folio *fo
 	netfs_pgpriv2_copy_folio(creq, folio);
 }
 
+/*
+ * Issue all pending writes on the cache stream.
+ */
+static void netfs_pgpriv2_issue_stream(struct netfs_io_request *wreq,
+				       struct netfs_io_stream *stream)
+{
+	atomic64_set_release(&stream->issued_to, wreq->start);
+
+	do {
+		struct netfs_io_subrequest *subreq;
+		int ret;
+
+		subreq = netfs_alloc_write_subreq(wreq, stream);
+		if (!subreq)
+			break;
+
+		ret = stream->issue_write(subreq);
+		if (ret < 0) {
+			/* Ownership of subreq was returned to us.  Punt the
+			 * error to the collector and stop the issuance of new
+			 * subreqs.
+			 */
+			trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+			stream->buffered -= subreq->len;
+			netfs_write_subrequest_terminated(subreq, ret);
+			break;
+		}
+		/* We no longer own subreq. */
+
+		if (test_bit(NETFS_RREQ_SAW_ENOMEM, &wreq->flags))
+			break;
+
+	} while (stream->buffered > 0);
+
+	netfs_all_subreqs_queued(wreq);
+}
+
 /*
  * [DEPRECATED] End writing to the cache, flushing out any outstanding writes.
  */
 void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
 {
 	struct netfs_io_request *creq = rreq->copy_to_cache;
+	struct netfs_io_stream *stream = &creq->io_streams[1];
 
 	if (IS_ERR_OR_NULL(creq))
 		return;
 
-	netfs_issue_write(creq, &creq->io_streams[1]);
-	smp_wmb(); /* Write lists before ALL_QUEUED. */
-	set_bit(NETFS_RREQ_ALL_QUEUED, &creq->flags);
+	netfs_pgpriv2_issue_stream(creq, stream);
 	trace_netfs_rreq(rreq, netfs_rreq_trace_end_copy_to_cache);
 	if (list_empty_careful(&creq->io_streams[1].subrequests))
 		netfs_wake_collector(creq);
 
 	netfs_put_request(creq, netfs_rreq_trace_put_return);
-	creq->copy_to_cache = NULL;
+	rreq->copy_to_cache = NULL;
 }
 
 /*
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index f93a0da08c72..8e77b81d27a4 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -9,20 +9,55 @@
 #include <linux/slab.h>
 #include "internal.h"
 
-static void netfs_reissue_read(struct netfs_io_request *rreq,
-			       struct netfs_io_subrequest *subreq)
+/*
+ * Prepare the I/O buffer on a buffered read subrequest for the filesystem to
+ * use as a bvec queue.
+ */
+int netfs_prepare_buffered_read_retry_buffer(struct netfs_io_subrequest *subreq,
+					     unsigned int max_segs)
 {
-	bvecq_pos_unset(&subreq->content);
+	struct netfs_io_request *rreq = subreq->rreq;
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &rreq->retry_cursor);
 	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
-			    subreq->content.slot, subreq->content.offset, subreq->len);
-	iov_iter_advance(&subreq->io_iter, subreq->transferred);
+	len = bvecq_slice(&rreq->retry_cursor, subreq->len, max_segs,
+			  &subreq->nr_segs);
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+	rreq->retry_buffered -= subreq->len;
+	rreq->retry_start    += subreq->len;
+	return 0;
+}
 
-	subreq->error = 0;
+/*
+ * Reset the state of the subrequest and discard any buffering so that we can
+ * retry (where this may include sending it to the server instead of the
+ * cache).
+ */
+int netfs_reset_for_read_retry(struct netfs_io_subrequest *subreq)
+{
+	trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+
+	if (subreq->retry_count > 3) {
+		trace_netfs_sreq(subreq, netfs_sreq_trace_too_many_retries);
+		return subreq->error;
+	}
+
+	subreq->retry_count++;
 	__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+	__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+	__clear_bit(NETFS_SREQ_FAILED, &subreq->flags);
 	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
-	netfs_stat(&netfs_n_rh_retry_read_subreq);
-	subreq->rreq->netfs_ops->issue_read(subreq);
+	bvecq_pos_unset(&subreq->content);
+	bvecq_pos_unset(&subreq->dispatch_pos);
+	subreq->error = 0;
+	subreq->transferred = 0;
+	netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
+	netfs_stat(&netfs_n_wh_retry_write_subreq);
+	return 0;
 }
 
 /*
@@ -33,8 +68,8 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 {
 	struct netfs_io_subrequest *subreq;
 	struct netfs_io_stream *stream = &rreq->io_streams[0];
-	struct bvecq_pos dispatch_cursor = {};
 	struct list_head *next;
+	int ret;
 
 	_enter("R=%x", rreq->debug_id);
 
@@ -44,46 +79,19 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 	if (rreq->netfs_ops->retry_request)
 		rreq->netfs_ops->retry_request(rreq, NULL);
 
-	/* If there's no renegotiation to do, just resend each retryable subreq
-	 * up to the first permanently failed one.
-	 */
-	if (!rreq->netfs_ops->prepare_read &&
-	    !rreq->cache_resources.ops) {
-		list_for_each_entry(subreq, &stream->subrequests, rreq_link) {
-			if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
-				break;
-			if (__test_and_clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-				subreq->retry_count++;
-				netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-				netfs_reissue_read(rreq, subreq);
-			}
-		}
-		return;
-	}
+	/* Read pointer to subreq before reading subreq state. */
+	next = smp_load_acquire(&stream->subrequests.next);
 
-	/* Okay, we need to renegotiate all the download requests and flip any
-	 * failed cache reads over to being download requests and negotiate
-	 * those also.  All fully successful subreqs have been removed from the
-	 * list and any spare data from those has been donated.
-	 *
-	 * What we do is decant the list and rebuild it one subreq at a time so
-	 * that we don't end up with donations jumping over a gap we're busy
-	 * populating with smaller subrequests.  In the event that the subreq
-	 * we just launched finishes before we insert the next subreq, it'll
-	 * fill in rreq->prev_donated instead.
-	 *
-	 * Note: Alternatively, we could split the tail subrequest right before
-	 * we reissue it and fix up the donations under lock.
+	/* Renegotiate all the download requests and flip any failed cache
+	 * reads over to being download requests and negotiate those also.
 	 */
-	next = stream->subrequests.next;
-
 	do {
 		struct netfs_io_subrequest *from, *to, *tmp;
-		unsigned long long start, len;
-		size_t part;
-		bool boundary = false, subreq_superfluous = false;
+		unsigned long long start;
+		size_t len;
+		bool subreq_superfluous = false;
 
-		bvecq_pos_unset(&dispatch_cursor);
+		bvecq_pos_unset(&rreq->retry_cursor);
 
 		/* Go through the subreqs and find the next span of contiguous
 		 * buffer that we then rejig (cifs, for example, needs the
@@ -98,8 +106,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		       rreq->debug_id, from->debug_index,
 		       from->start, from->transferred, from->len);
 
-		if (test_bit(NETFS_SREQ_FAILED, &from->flags) ||
-		    !test_bit(NETFS_SREQ_NEED_RETRY, &from->flags)) {
+		if (!test_bit(NETFS_SREQ_NEED_RETRY, &from->flags)) {
 			subreq = from;
 			goto abandon;
 		}
@@ -113,20 +120,21 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 			subreq = list_entry(next, struct netfs_io_subrequest, rreq_link);
 			if (subreq->start != start + len ||
 			    subreq->transferred > 0 ||
-			    test_bit(NETFS_SREQ_BOUNDARY, &subreq->flags) ||
 			    !test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
 				break;
 			to = subreq;
 			len += to->len;
 		}
 
-		_debug(" - range: %llx-%llx %llx", start, start + len - 1, len);
+		_debug(" - range: %llx-%llx %zx", start, start + len - 1, len);
 
 		/* Determine the set of buffers we're going to use.  Each
-		 * subreq gets a subset of a single overall contiguous buffer.
+		 * subreq takes a subset of a single overall contiguous buffer.
 		 */
-		bvecq_pos_transfer(&dispatch_cursor, &from->dispatch_pos);
-		bvecq_pos_advance(&dispatch_cursor, from->transferred);
+		bvecq_pos_transfer(&rreq->retry_cursor, &from->dispatch_pos);
+		bvecq_pos_advance(&rreq->retry_cursor, from->transferred);
+		rreq->retry_start = start;
+		rreq->retry_buffered = len;
 		from->transferred = 0;
 
 		/* Work through the sublist.  The chain of buffers we're going
@@ -135,51 +143,30 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		 */
 		subreq = from;
 		list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
-			if (!len) {
+			if (rreq->retry_buffered == 0) {
 				subreq_superfluous = true;
 				break;
 			}
 			subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
-			subreq->start	= start;
-			subreq->len	= len;
-			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
-			__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
-			subreq->retry_count++;
-			subreq->transferred = 0;
+			subreq->start	= rreq->retry_start;
+			subreq->len	= rreq->retry_buffered;
 
-			bvecq_pos_unset(&subreq->content);
-			bvecq_pos_unset(&subreq->dispatch_pos);
-			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
-
-			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
-
-			/* Renegotiate max_len (rsize) */
-			stream->sreq_max_len = len;
-			stream->sreq_max_segs = INT_MAX;
-			if (rreq->netfs_ops->prepare_read &&
-			    rreq->netfs_ops->prepare_read(subreq) < 0) {
-				trace_netfs_sreq(subreq, netfs_sreq_trace_reprep_failed);
+			ret = netfs_reset_for_read_retry(subreq);
+			if (ret < 0) {
 				__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
+				rreq->error = ret;
 				goto abandon;
 			}
 
-			part = bvecq_slice(&dispatch_cursor,
-					   umin(len, stream->sreq_max_len),
-					   stream->sreq_max_segs,
-					   &subreq->nr_segs);
-			subreq->len = part;
-
-			len -= part;
-			start += part;
-			if (!len) {
-				if (boundary)
-					__set_bit(NETFS_SREQ_BOUNDARY, &subreq->flags);
-			} else {
-				__clear_bit(NETFS_SREQ_BOUNDARY, &subreq->flags);
+			netfs_stat(&netfs_n_rh_download);
+			ret = rreq->netfs_ops->issue_read(subreq);
+			if (ret < 0) {
+				subreq->error = ret;
+				netfs_read_subreq_terminated(subreq);
+				goto abandon_after;
 			}
-
-			netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-			netfs_reissue_read(rreq, subreq);
+			if (test_bit(NETFS_RREQ_SAW_ENOMEM, &rreq->flags))
+				goto abandon_after;
 			if (subreq == to) {
 				subreq_superfluous = false;
 				break;
@@ -189,7 +176,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		/* If we managed to use fewer subreqs, we can discard the
 		 * excess; if we used the same number, then we're done.
 		 */
-		if (!len) {
+		if (rreq->retry_buffered == 0) {
 			if (!subreq_superfluous)
 				continue;
 			list_for_each_entry_safe_from(subreq, tmp,
@@ -207,7 +194,8 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 		}
 
 		/* We ran out of subrequests, so we need to allocate some more
-		 * and insert them after.
+		 * and insert them after.  They must start with being marked
+		 * for retry to switch to the retry cursor.
 		 */
 		do {
 			subreq = netfs_alloc_subrequest(rreq);
@@ -216,8 +204,8 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 				goto abandon_after;
 			}
 			subreq->source		= NETFS_DOWNLOAD_FROM_SERVER;
-			subreq->start		= start;
-			subreq->len		= len;
+			subreq->start		= rreq->retry_start;
+			subreq->len		= rreq->retry_buffered;
 			subreq->stream_nr	= stream->stream_nr;
 			subreq->retry_count	= 1;
 
@@ -225,43 +213,32 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)
 					     refcount_read(&subreq->ref),
 					     netfs_sreq_trace_new);
 
+			__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
+
 			spin_lock(&rreq->lock);
+			/* Write IN_PROGRESS before pointer to new subreq */
+			smp_wmb();
 			list_add(&subreq->rreq_link, &to->rreq_link);
 			spin_unlock(&rreq->lock);
 			to = subreq;
 			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
-			stream->sreq_max_len	= umin(len, rreq->rsize);
-			stream->sreq_max_segs	= INT_MAX;
-
 			netfs_stat(&netfs_n_rh_download);
-			if (rreq->netfs_ops->prepare_read(subreq) < 0) {
-				trace_netfs_sreq(subreq, netfs_sreq_trace_reprep_failed);
-				__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
-				goto abandon;
-			}
-
-			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
-			part = bvecq_slice(&dispatch_cursor,
-					   umin(len, stream->sreq_max_len),
-					   stream->sreq_max_segs,
-					   &subreq->nr_segs);
-			subreq->len = part;
-
-			len -= part;
-			start += part;
-			if (!len && boundary) {
-				__set_bit(NETFS_SREQ_BOUNDARY, &to->flags);
-				boundary = false;
+			ret = rreq->netfs_ops->issue_read(subreq);
+			if (ret < 0) {
+				subreq->error = ret;
+				netfs_read_subreq_terminated(subreq);
+				goto abandon_after;
 			}
+			if (test_bit(NETFS_RREQ_SAW_ENOMEM, &rreq->flags))
+				goto abandon_after;
 
-			netfs_reissue_read(rreq, subreq);
-		} while (len);
+		} while (rreq->retry_buffered > 0);
 
 	} while (!list_is_head(next, &stream->subrequests));
 
 out:
-	bvecq_pos_unset(&dispatch_cursor);
+	bvecq_pos_unset(&rreq->retry_cursor);
 	return;
 
 	/* If we hit an error, fail all remaining incomplete subrequests */
@@ -334,6 +311,7 @@ void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
 			}
 			trace_netfs_folio(folio, netfs_folio_trace_abandon);
 			folio_unlock(folio);
+			p->bv[slot].bv_page = NULL;
 		}
 	}
 }
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index 0a4542cd87e7..ab040952c281 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -16,6 +16,22 @@
 #include <linux/netfs.h>
 #include "internal.h"
 
+int netfs_prepare_read_single_buffer(struct netfs_io_subrequest *subreq,
+				     unsigned int max_segs)
+{
+	struct netfs_io_request *rreq = subreq->rreq;
+	struct netfs_io_stream *stream = &rreq->io_streams[0];
+
+	bvecq_pos_set(&subreq->dispatch_pos, &rreq->load_cursor);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+
+	stream->buffered = 0;
+	stream->issue_from += subreq->len;
+	rreq->submitted = stream->issue_from;
+	netfs_all_subreqs_queued(rreq);
+	return 0;
+}
+
 /**
  * netfs_single_mark_inode_dirty - Mark a single, monolithic object inode dirty
  * @inode: The inode to mark
@@ -58,17 +74,6 @@ static int netfs_single_begin_cache_read(struct netfs_io_request *rreq, struct n
 	return fscache_begin_read_operation(&rreq->cache_resources, netfs_i_cookie(ctx));
 }
 
-static void netfs_single_read_cache(struct netfs_io_request *rreq,
-				    struct netfs_io_subrequest *subreq)
-{
-	struct netfs_cache_resources *cres = &rreq->cache_resources;
-
-	_enter("R=%08x[%x]", rreq->debug_id, subreq->debug_index);
-	netfs_stat(&netfs_n_rh_read);
-	cres->ops->read(cres, subreq->start, &subreq->io_iter, NETFS_READ_HOLE_FAIL,
-			netfs_cache_read_terminated, subreq);
-}
-
 /*
  * Perform a read to a buffer from the cache or the server.  Only a single
  * subreq is permitted as the object must be fetched in a single transaction.
@@ -84,73 +89,84 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)
 		.cached_to[1]	= ULLONG_MAX,
 	};
 	struct netfs_io_subrequest *subreq;
-	int ret = 0;
+	int ret;
+
+	netfs_read_query_cache(rreq, &occ);
 
-	subreq = netfs_alloc_subrequest(rreq);
+	subreq = netfs_alloc_read_subrequest(rreq);
 	if (!subreq)
 		return -ENOMEM;
 
-	subreq->source	= NETFS_DOWNLOAD_FROM_SERVER;
 	subreq->start	= 0;
 	subreq->len	= rreq->len;
 
-	bvecq_pos_set(&subreq->dispatch_pos, &rreq->dispatch_cursor);
-	bvecq_pos_set(&subreq->content, &rreq->dispatch_cursor);
-
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_DEST, subreq->content.bvecq,
-			    subreq->content.slot, subreq->content.offset, subreq->len);
-
-	netfs_queue_read(rreq, subreq);
+	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
 
 	/* Try to use the cache if the cache content matches the size of the
 	 * remote file.
 	 */
-	netfs_read_query_cache(rreq, &occ);
 	if (occ.cached_from[0] == 0 &&
-	    occ.cached_to[0] >= rreq->len)
-		subreq->source = NETFS_READ_FROM_CACHE;
+	    occ.cached_to[0] >= rreq->len) {
+		struct netfs_cache_resources *cres = &rreq->cache_resources;
 
-	switch (subreq->source) {
-	case NETFS_DOWNLOAD_FROM_SERVER:
-		netfs_stat(&netfs_n_rh_download);
-		if (rreq->netfs_ops->prepare_read) {
-			ret = rreq->netfs_ops->prepare_read(subreq);
-			if (ret < 0)
-				goto cancel;
+		subreq->source = NETFS_READ_FROM_CACHE;
+		netfs_stat(&netfs_n_rh_read);
+		ret = cres->ops->issue_read(subreq);
+		if (ret < 0) {
+			subreq->error = ret;
+			netfs_read_subreq_terminated(subreq);
 		}
 
-		smp_wmb(); /* Write lists before ALL_QUEUED. */
-		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
-		rreq->netfs_ops->issue_read(subreq);
-		rreq->submitted += subreq->len;
-		break;
-	case NETFS_READ_FROM_CACHE:
-		if (rreq->cache_resources.ops->prepare_read) {
-			ret = rreq->cache_resources.ops->prepare_read(subreq);
-			if (ret < 0)
-				goto cancel;
+		ret = netfs_wait_for_in_progress_subreq(rreq, subreq);
+		if (ret == 0)
+			goto success;
+		if (ret == -ENOMEM)
+			goto cancel;
+
+		/* Didn't manage to retrieve from the cache, so toss it to the
+		 * server instead.
+		 */
+		if (netfs_reset_for_read_retry(subreq) < 0)
+			goto cancel;
+	}
+
+	__set_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
+
+	/* Try to send it to the cache. */
+	for (;;) {
+		subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
+		netfs_stat(&netfs_n_rh_download);
+		ret = rreq->netfs_ops->issue_read(subreq);
+		if (ret < 0) {
+			subreq->error = ret;
+			netfs_read_subreq_terminated(subreq);
 		}
 
-		smp_wmb(); /* Write lists before ALL_QUEUED. */
-		set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
-		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
-		netfs_single_read_cache(rreq, subreq);
-		rreq->submitted += subreq->len;
-		ret = 0;
-		break;
-	default:
-		pr_warn("Unexpected single-read source %u\n", subreq->source);
-		WARN_ON_ONCE(true);
-		ret = -EIO;
-		goto cancel;
+		ret = netfs_wait_for_in_progress_subreq(rreq, subreq);
+		if (ret == 0)
+			goto success;
+		if (ret == -ENOMEM)
+			goto cancel;
+		if (ret != -EAGAIN)
+			goto failed;
+		if (netfs_reset_for_read_retry(subreq) < 0)
+			goto cancel;
 	}
 
-	return ret;
+success:
+	rreq->transferred = subreq->transferred;
+	list_del_init(&subreq->rreq_link);
+	netfs_put_subrequest(subreq, netfs_sreq_trace_put_consumed);
+	return 0;
 cancel:
-	netfs_cancel_read(subreq, ret);
-	smp_wmb(); /* Write lists before ALL_QUEUED. */
-	set_bit(NETFS_RREQ_ALL_QUEUED, &rreq->flags);
-	netfs_wake_collector(rreq);
+	rreq->error = ret;
+	list_del_init(&subreq->rreq_link);
+	netfs_put_subrequest(subreq, netfs_sreq_trace_put_cancel);
+	return ret;
+failed:
+	rreq->error = ret;
+	list_del_init(&subreq->rreq_link);
+	netfs_put_subrequest(subreq, netfs_sreq_trace_put_failed);
 	return ret;
 }
 
@@ -182,7 +198,7 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	if (IS_ERR(rreq))
 		return PTR_ERR(rreq);
 
-	ret = netfs_extract_iter(iter, rreq->len, INT_MAX, 0, &rreq->dispatch_cursor.bvecq,
+	ret = netfs_extract_iter(iter, rreq->len, INT_MAX, 0, &rreq->load_cursor.bvecq,
 				 0, rreq->gfp);
 	if (ret < 0)
 		goto cleanup_free;
@@ -200,9 +216,29 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	netfs_stat(&netfs_n_rh_read_single);
 	trace_netfs_read(rreq, 0, rreq->len, netfs_read_trace_read_single);
 
-	netfs_single_dispatch_read(rreq);
+	ret = netfs_single_dispatch_read(rreq);
+
+	trace_netfs_rreq(rreq, netfs_rreq_trace_complete);
+	if (ret == 0) {
+		task_io_account_read(rreq->transferred);
+
+		if (test_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags) &&
+		    fscache_resources_valid(&rreq->cache_resources)) {
+			trace_netfs_rreq(rreq, netfs_rreq_trace_dirty);
+			netfs_single_mark_inode_dirty(rreq->inode);
+		}
+		ret = rreq->transferred;
+	}
+
+	if (rreq->netfs_ops->done)
+		rreq->netfs_ops->done(rreq);
+
+	netfs_wake_rreq_flag(rreq, NETFS_RREQ_IN_PROGRESS, netfs_rreq_trace_wake_ip);
+	/* As we cleared NETFS_RREQ_IN_PROGRESS, we acquired its ref. */
+	netfs_put_request(rreq, netfs_rreq_trace_put_work_ip);
+
+	trace_netfs_rreq(rreq, netfs_rreq_trace_done);
 
-	ret = netfs_wait_for_read(rreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_return);
 	return ret;
 
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 35e29a01f57c..0e8dc986eeac 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -28,8 +28,8 @@ static void netfs_dump_request(const struct netfs_io_request *rreq)
 	       rreq->origin, rreq->error);
 	pr_err("  st=%llx tsl=%zx/%llx/%llx\n",
 	       rreq->start, rreq->transferred, rreq->submitted, rreq->len);
-	pr_err("  cci=%llx/%llx/%llx\n",
-	       rreq->cleaned_to, rreq->collected_to, atomic64_read(&rreq->issued_to));
+	pr_err("  cci=%llx/%llx\n",
+	       rreq->cleaned_to, rreq->collected_to);
 	pr_err("  iw=%pSR\n", rreq->netfs_ops->issue_write);
 	for (int i = 0; i < NR_IO_STREAMS; i++) {
 		const struct netfs_io_subrequest *sreq;
@@ -38,8 +38,9 @@ static void netfs_dump_request(const struct netfs_io_request *rreq)
 		pr_err("  str[%x] s=%x e=%d acnf=%u,%u,%u,%u\n",
 		       s->stream_nr, s->source, s->error,
 		       s->avail, s->active, s->need_retry, s->failed);
-		pr_err("  str[%x] ct=%llx t=%zx\n",
-		       s->stream_nr, s->collected_to, s->transferred);
+		pr_err("  str[%x] it=%llx ct=%llx t=%zx\n",
+		       s->stream_nr, atomic64_read(&s->issued_to),
+		       s->collected_to, s->transferred);
 		list_for_each_entry(sreq, &s->subrequests, rreq_link) {
 			pr_err("  sreq[%x:%x] sc=%u s=%llx t=%zx/%zx r=%d f=%lx\n",
 			       sreq->stream_nr, sreq->debug_index, sreq->source,
@@ -56,7 +57,7 @@ static void netfs_dump_request(const struct netfs_io_request *rreq)
  */
 int netfs_folio_written_back(struct folio *folio)
 {
-	enum netfs_folio_trace why = netfs_folio_trace_clear;
+	enum netfs_folio_trace why = netfs_folio_trace_endwb;
 	struct inode *inode = folio_inode(folio);
 	struct netfs_inode *ictx = netfs_inode(inode);
 	struct netfs_folio *finfo;
@@ -79,13 +80,13 @@ int netfs_folio_written_back(struct folio *folio)
 		group = finfo->netfs_group;
 		gcount++;
 		kfree(finfo);
-		why = netfs_folio_trace_clear_s;
+		why = netfs_folio_trace_endwb_s;
 		goto end_wb;
 	}
 
 	if ((group = netfs_folio_group(folio))) {
 		if (group == NETFS_FOLIO_COPY_TO_CACHE) {
-			why = netfs_folio_trace_clear_cc;
+			why = netfs_folio_trace_endwb_cc;
 			folio_detach_private(folio);
 			goto end_wb;
 		}
@@ -98,7 +99,7 @@ int netfs_folio_written_back(struct folio *folio)
 		if (!folio_test_dirty(folio)) {
 			folio_detach_private(folio);
 			gcount++;
-			why = netfs_folio_trace_clear_g;
+			why = netfs_folio_trace_endwb_g;
 		}
 	}
 
@@ -151,6 +152,9 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 			slot  = wreq->collect_cursor.slot;
 		}
 
+		WARN_ONCE(!bvecq->bv[slot].bv_page,
+			  "R=%08x slot already cleared?\n", wreq->debug_id);
+
 		folio = page_folio(bvecq->bv[slot].bv_page);
 		if (WARN_ONCE(!folio_test_writeback(folio),
 			      "R=%08x: folio %lx is not under writeback\n",
@@ -225,9 +229,7 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 	trace_netfs_rreq(wreq, netfs_rreq_trace_collect);
 
 reassess_streams:
-	/* Order reading the issued_to point before reading the queue it refers to. */
-	issued_to = atomic64_read_acquire(&wreq->issued_to);
-	smp_rmb();
+	issued_to = ULLONG_MAX;
 	collected_to = ULLONG_MAX;
 	if (wreq->origin == NETFS_WRITEBACK ||
 	    wreq->origin == NETFS_PGPRIV2_COPY_TO_CACHE)
@@ -241,18 +243,34 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 	 * to the tail whilst we're doing this.
 	 */
 	for (s = 0; s < NR_IO_STREAMS; s++) {
+		unsigned long long s_issued_to;
+
 		stream = &wreq->io_streams[s];
-		/* Read active flag before list pointers */
+		/* Read active flag before issued_to */
 		if (!smp_load_acquire(&stream->active))
 			continue;
 
-		front = list_first_entry_or_null_acquire(&stream->subrequests,
-							 struct netfs_io_subrequest, rreq_link);
-		/* Read first subreq pointer before IN_PROGRESS flag. */
-
-		while (front) {
+		for (;;) {
 			bool cancelled;
-			trace_netfs_collect_sreq(wreq, front);
+
+			/* Order reading the issued_to point before reading the
+			 * queue it refers to.
+			 */
+			s_issued_to = atomic64_read_acquire(&stream->issued_to);
+			if (s_issued_to < issued_to)
+				issued_to = s_issued_to;
+
+			front = list_first_entry_or_null_acquire(&stream->subrequests,
+								 struct netfs_io_subrequest,
+								 rreq_link);
+			/* Read first subreq pointer before IN_PROGRESS flag. */
+			if (!front) {
+				if (stream->source == NETFS_UPLOAD_TO_SERVER &&
+				    test_bit(NETFS_RREQ_PAUSE, &wreq->flags))
+					notes |= MADE_PROGRESS;
+				break;
+			}
+
 			//_debug("sreq [%x] %llx %zx/%zx",
 			//       front->debug_index, front->start, front->transferred, front->len);
 
@@ -269,6 +287,8 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)
 				break;
 			}
 
+			trace_netfs_collect_sreq(wreq, front);
+
 			if (stream->failed) {
 				stream->collected_to = front->start + front->len;
 				notes |= MADE_PROGRESS | SAW_FAILURE;
@@ -419,9 +439,8 @@ bool netfs_write_collection(struct netfs_io_request *wreq)
 	/* We're done when the app thread has finished posting subreqs and all
 	 * the queues in all the streams are empty.
 	 */
-	if (!test_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags))
+	if (!netfs_are_all_subreqs_queued(wreq))
 		return false;
-	smp_rmb(); /* Read ALL_QUEUED before lists. */
 
 	transferred = LONG_MAX;
 	for (s = 0; s < NR_IO_STREAMS; s++) {
@@ -541,6 +560,8 @@ void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error)
 
 	if (IS_ERR_VALUE(transferred_or_error)) {
 		subreq->error = transferred_or_error;
+		if (transferred_or_error == -ENOMEM)
+			set_bit(NETFS_RREQ_SAW_ENOMEM, &wreq->flags);
 
 		switch (subreq->source) {
 		case NETFS_WRITE_TO_CACHE:
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 69f24d53363b..eb1fdcb22aba 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -36,6 +36,27 @@
 #include <linux/pagemap.h>
 #include "internal.h"
 
+#define NOTE_UPLOAD_AVAIL	0x001	/* Upload is available */
+#define NOTE_CACHE_AVAIL	0x002	/* Local cache is available */
+#define NOTE_CACHE_COPY		0x004	/* Copy folio to cache */
+#define NOTE_UPLOAD		0x008	/* Upload folio to server */
+#define NOTE_UPLOAD_STARTED	0x010	/* Upload started */
+#define NOTE_STREAMW		0x020	/* Folio is from a streaming write */
+#define NOTE_DISCONTIG_BEFORE	0x040	/* Folio discontiguous with the previous folio */
+#define NOTE_TO_EOF		0x100	/* Data in folio ends at EOF */
+#define NOTE_FLUSH_ANYWAY	0x200	/* Flush data, even if not hit estimated limit */
+
+#define NOTES__KEEP_MASK (NOTE_UPLOAD_AVAIL | NOTE_CACHE_AVAIL | NOTE_UPLOAD_STARTED)
+
+struct netfs_wb_params {
+	uoff_t			last_end;	/* End file pos of previous folio */
+	unsigned int		notes;		/* Notes on applicability */
+	struct netfs_write_estimate estimates[NR_IO_STREAMS];
+};
+
+static int netfs_prepare_write_single_buffer(struct netfs_io_subrequest *subreq,
+					     unsigned int max_segs);
+
 /*
  * Kill all dirty folios in the event of an unrecoverable error, starting with
  * a locked folio we've already obtained from writeback_iter().
@@ -114,65 +135,54 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 
 	wreq->io_streams[0].stream_nr		= 0;
 	wreq->io_streams[0].source		= NETFS_UPLOAD_TO_SERVER;
-	wreq->io_streams[0].prepare_write	= ictx->ops->prepare_write;
+	wreq->io_streams[0].applicable		= NOTE_UPLOAD;
+	wreq->io_streams[0].estimate_write	= ictx->ops->estimate_write;
 	wreq->io_streams[0].issue_write		= ictx->ops->issue_write;
 	wreq->io_streams[0].collected_to	= start;
 	wreq->io_streams[0].transferred		= 0;
 
 	wreq->io_streams[1].stream_nr		= 1;
 	wreq->io_streams[1].source		= NETFS_WRITE_TO_CACHE;
+	wreq->io_streams[1].applicable		= NOTE_CACHE_COPY;
 	wreq->io_streams[1].collected_to	= start;
 	wreq->io_streams[1].transferred		= 0;
 	if (fscache_resources_valid(&wreq->cache_resources)) {
 		wreq->io_streams[1].avail	= true;
 		wreq->io_streams[1].active	= true;
-		wreq->io_streams[1].prepare_write = wreq->cache_resources.ops->prepare_write_subreq;
+		wreq->io_streams[1].estimate_write = wreq->cache_resources.ops->estimate_write;
 		wreq->io_streams[1].issue_write = wreq->cache_resources.ops->issue_write;
+		wreq->io_streams[1].alignment	= wreq->cache_resources.dio_size;
 	}
 
 	return wreq;
 }
 
-/**
- * netfs_prepare_write_failed - Note write preparation failed
- * @subreq: The subrequest to mark
- *
- * Mark a subrequest to note that preparation for write failed.
- */
-void netfs_prepare_write_failed(struct netfs_io_subrequest *subreq)
-{
-	__set_bit(NETFS_SREQ_FAILED, &subreq->flags);
-	trace_netfs_sreq(subreq, netfs_sreq_trace_prep_failed);
-}
-EXPORT_SYMBOL(netfs_prepare_write_failed);
-
 /*
- * Prepare a write subrequest.  We need to allocate a new subrequest
- * if we don't have one.
+ * Allocate and prepare a write subrequest.  Will only return NULL if not
+ * performing writeback; if performing writeback, mempools may be accessed and
+ * the allocator may wait forever.
  */
-void netfs_prepare_write(struct netfs_io_request *wreq,
-			 struct netfs_io_stream *stream,
-			 loff_t start)
+struct netfs_io_subrequest *netfs_alloc_write_subreq(struct netfs_io_request *wreq,
+						     struct netfs_io_stream *stream)
 {
 	struct netfs_io_subrequest *subreq;
 
 	subreq = netfs_alloc_subrequest(wreq);
+	if (!subreq)
+		return subreq;
+
 	subreq->source		= stream->source;
-	subreq->start		= start;
+	subreq->start		= stream->issue_from;
+	subreq->len		= stream->buffered;
 	subreq->stream_nr	= stream->stream_nr;
 
-	bvecq_pos_set(&subreq->dispatch_pos, &wreq->dispatch_cursor);
-
 	_enter("R=%x[%x]", wreq->debug_id, subreq->debug_index);
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
 
-	stream->sreq_max_len	= UINT_MAX;
-	stream->sreq_max_segs	= INT_MAX;
 	switch (stream->source) {
 	case NETFS_UPLOAD_TO_SERVER:
 		netfs_stat(&netfs_n_wh_upload);
-		stream->sreq_max_len = wreq->wsize;
 		break;
 	case NETFS_WRITE_TO_CACHE:
 		netfs_stat(&netfs_n_wh_write);
@@ -182,9 +192,6 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 		break;
 	}
 
-	if (stream->prepare_write)
-		stream->prepare_write(subreq);
-
 	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 
 	/* We add to the end of the list whilst the collector may be walking
@@ -194,161 +201,202 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 	spin_lock(&wreq->lock);
 	/* Write IN_PROGRESS before pointer to new subreq */
 	list_add_tail_release(&subreq->rreq_link, &stream->subrequests);
-	if (list_is_first(&subreq->rreq_link, &stream->subrequests)) {
-		if (!stream->active) {
-			stream->collected_to = subreq->start;
-			/* Write list pointers before active flag */
-			smp_store_release(&stream->active, true);
-		}
-	}
+	if (list_is_first(&subreq->rreq_link, &stream->subrequests) &&
+	    stream->collected_to == 0)
+		stream->collected_to = subreq->start;
 
 	spin_unlock(&wreq->lock);
+	return subreq;
+}
 
-	stream->construct = subreq;
+/*
+ * Advance the state of the amount of data buffered on a stream.
+ */
+static void netfs_advance_stream(struct netfs_io_request *wreq,
+				 struct netfs_io_stream *stream,
+				 size_t len)
+{
+	stream->issue_from += len;
+	stream->buffered   -= len;
+	if (stream->buffered == 0) {
+		stream->buffering = false;
+		bvecq_pos_unset(&stream->dispatch_cursor);
+	}
+	/* Order loading the queue before updating the issue_to point */
+	atomic64_set_release(&stream->issued_to, stream->issue_from);
 }
 
 /*
- * Set the I/O iterator for the filesystem/cache to use and dispatch the I/O
- * operation.  The operation may be asynchronous and should call
- * netfs_write_subrequest_terminated() when complete.
+ * Prepare the buffer for a buffered write.
  */
-static void netfs_do_issue_write(struct netfs_io_stream *stream,
-				 struct netfs_io_subrequest *subreq)
+static int netfs_prepare_buffered_write_buffer(struct netfs_io_subrequest *subreq,
+					       unsigned int max_segs)
 {
 	struct netfs_io_request *wreq = subreq->rreq;
+	struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];
+	ssize_t len;
 
-	_enter("R=%x[%x],%zx", wreq->debug_id, subreq->debug_index, subreq->len);
-
-	if (stream->source == NETFS_WRITE_TO_CACHE &&
-	    unlikely(test_bit(NETFS_RREQ_CACHE_STOP, &wreq->flags))) {
-		size_t dio_size = wreq->cache_resources.dio_size;
-		size_t len, disp;
+	_enter("%zx,{,%u,%u},%u",
+	       subreq->len, stream->dispatch_cursor.slot, stream->dispatch_cursor.offset, max_segs);
 
-		disp = subreq->start & (dio_size - 1);
-		len = round_up(subreq->len + disp, dio_size);
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &stream->dispatch_cursor);
 
-		subreq->start -= disp;
+	len = bvecq_slice(&stream->dispatch_cursor, subreq->len, max_segs, &subreq->nr_segs);
+	if (len < subreq->len) {
 		subreq->len = len;
-
-		__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
-		return netfs_write_subrequest_terminated(subreq, subreq->len);
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
 	}
 
-	if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
-		return netfs_write_subrequest_terminated(subreq, subreq->error);
-
-	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
-	stream->issue_write(subreq);
+	netfs_advance_stream(wreq, stream, len);
+	return 0;
 }
 
-void netfs_reissue_write(struct netfs_io_stream *stream,
-			 struct netfs_io_subrequest *subreq)
+/**
+ * netfs_prepare_write_buffer - Get the buffer for a subrequest
+ * @subreq: The subrequest to get the buffer for
+ * @max_segs: Maximum number of segments in buffer (or INT_MAX)
+ *
+ * Extract a slice of buffer from the stream and attach it to the subrequest as
+ * a bio_vec queue.  The maximum amount of data attached is set by
+ * @subreq->len, but this may be shortened if @max_segs would be exceeded.
+ */
+int netfs_prepare_write_buffer(struct netfs_io_subrequest *subreq,
+			       unsigned int max_segs)
 {
-	// TODO: Use encrypted buffer
-	bvecq_pos_unset(&subreq->content);
-	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
-			    subreq->content.bvecq, subreq->content.slot,
-			    subreq->content.offset,
-			    subreq->len);
-	iov_iter_advance(&subreq->io_iter, subreq->transferred);
-
-	subreq->retry_count++;
-	subreq->error = 0;
-	__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
-	__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
-	netfs_stat(&netfs_n_wh_retry_write_subreq);
-	netfs_do_issue_write(stream, subreq);
+	struct netfs_io_request *rreq = subreq->rreq;
+
+	switch (rreq->origin) {
+	case NETFS_WRITEBACK:
+		if (test_bit(NETFS_RREQ_RETRYING, &rreq->flags))
+			return netfs_prepare_write_retry_buffer(subreq, max_segs);
+		return netfs_prepare_buffered_write_buffer(subreq, max_segs);
+
+	case NETFS_UNBUFFERED_WRITE:
+	case NETFS_DIO_WRITE:
+		return netfs_prepare_unbuffered_write_buffer(subreq, max_segs);
+
+	case NETFS_WRITEBACK_SINGLE:
+		return netfs_prepare_write_single_buffer(subreq, max_segs);
+
+	case NETFS_PGPRIV2_COPY_TO_CACHE:
+		return netfs_prepare_pgpriv2_write_buffer(subreq, max_segs);
+
+	default:
+		WARN_ON_ONCE(1);
+		return -EIO;
+	}
 }
+EXPORT_SYMBOL(netfs_prepare_write_buffer);
 
-void netfs_issue_write(struct netfs_io_request *wreq,
-		       struct netfs_io_stream *stream)
+/*
+ * Issue writes for a stream.
+ */
+static void netfs_writeback_flush(struct netfs_io_request *wreq,
+				  struct netfs_io_stream *stream,
+				  struct netfs_wb_params *params)
 {
-	struct netfs_io_subrequest *subreq = stream->construct;
+	struct netfs_write_estimate *estimate = &params->estimates[stream->stream_nr];
 
-	if (!subreq)
-		return;
+	for (;;) {
+		struct netfs_io_subrequest *subreq;
+		int ret;
+
+		if (test_bit(NETFS_RREQ_PAUSE, &wreq->flags))
+			netfs_wait_for_paused_write(wreq);
+
+		subreq = netfs_alloc_write_subreq(wreq, stream);
+		/* subreq allocation in a writeback is backed by a mempool and
+		 * will wait for an new one to come available.
+		 */
+
+		if (stream->source == NETFS_WRITE_TO_CACHE &&
+		    unlikely(test_bit(NETFS_RREQ_CACHE_STOP, &wreq->flags))) {
+			estimate->issue_at = ULLONG_MAX;
+			estimate->max_segs = INT_MAX;
+			__set_bit(NETFS_SREQ_CANCELLED, &subreq->flags);
+			netfs_advance_stream(wreq, stream, subreq->len);
+			netfs_write_subrequest_terminated(subreq, subreq->len);
+			return;
+		}
 
-	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
-	iov_iter_bvec_queue(&subreq->io_iter, ITER_SOURCE,
-			    subreq->content.bvecq, subreq->content.slot,
-			    subreq->content.offset,
-			    subreq->len);
+		ret = stream->issue_write(subreq);
+		if (ret < 0) {
+			/* Ownership of subreq was returned to us. */
+			trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+			bvecq_pos_advance(&stream->dispatch_cursor, subreq->len);
+			netfs_advance_stream(wreq, stream, subreq->len);
+			netfs_write_subrequest_terminated(subreq, ret);
+		}
+		/* We no longer own subreq. */
+
+		if (stream->buffered == 0) {
+			if (stream->stream_nr == 0)
+				params->notes &= ~NOTE_UPLOAD_STARTED;
+			return;
+		}
 
-	stream->construct = NULL;
-	netfs_do_issue_write(stream, subreq);
+		if (!(params->notes & NOTE_FLUSH_ANYWAY)) {
+			estimate->issue_at = ULLONG_MAX;
+			estimate->max_segs = INT_MAX;
+			stream->estimate_write(wreq, stream, estimate);
+			if (stream->issue_from + stream->buffered < estimate->issue_at &&
+			    estimate->max_segs > 0)
+				return;
+		}
+	}
 }
 
 /*
- * Add data to the write subrequest, dispatching each as we fill it up or if it
- * is discontiguous with the previous.  We only fill one part at a time so that
- * we can avoid overrunning the credits obtained (cifs) and try to parallelise
- * content-crypto preparation with network writes.
+ * End the issuing of writes, let the collector know we're done.
  */
-size_t netfs_advance_write(struct netfs_io_request *wreq,
-			   struct netfs_io_stream *stream,
-			   loff_t start, size_t len, bool to_eof)
+static void netfs_writeback_end(struct netfs_io_request *wreq,
+				struct netfs_wb_params *params)
 {
-	struct netfs_io_subrequest *subreq = stream->construct;
-	size_t part;
+	bool needs_poke = true;
 
-	if (!stream->avail) {
-		_leave("no write");
-		return len;
-	}
+	params->notes |= NOTE_FLUSH_ANYWAY;
 
-	_enter("R=%x[%x]", wreq->debug_id, subreq ? subreq->debug_index : 0);
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
 
-	if (subreq && start != subreq->start + subreq->len) {
-		netfs_issue_write(wreq, stream);
-		subreq = NULL;
+		if (stream->buffering) {
+			netfs_writeback_flush(wreq, stream, params);
+			stream->buffering = false;
+		}
 	}
 
-	if (!stream->construct)
-		netfs_prepare_write(wreq, stream, start);
-	subreq = stream->construct;
+	netfs_all_subreqs_queued(wreq);
 
-	part = umin(stream->sreq_max_len - subreq->len, len);
-	_debug("part %zx/%zx %zx/%zx", subreq->len, stream->sreq_max_len, part, len);
-	subreq->len += part;
-	subreq->nr_segs++;
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
 
-	if (subreq->len >= stream->sreq_max_len ||
-	    subreq->nr_segs >= stream->sreq_max_segs ||
-	    to_eof) {
-		netfs_issue_write(wreq, stream);
-		subreq = NULL;
+		if (!stream->active)
+			continue;
+		if (!list_empty(&stream->subrequests))
+			needs_poke = false;
 	}
 
-	return part;
+	if (needs_poke)
+		netfs_wake_collector(wreq);
 }
 
 /*
- * Write some of a pending folio data back to the server.
+ * Queue a folio for writeback.
  */
-static int netfs_write_folio(struct netfs_io_request *wreq,
-			     struct writeback_control *wbc,
-			     struct folio *folio)
+static void netfs_writeback_folio(struct netfs_io_request *wreq,
+				  struct writeback_control *wbc,
+				  struct folio *folio,
+				  struct netfs_wb_params *params)
 {
-	struct netfs_io_stream *upload = &wreq->io_streams[0];
-	struct netfs_io_stream *cache  = &wreq->io_streams[1];
-	struct netfs_io_stream *stream;
 	struct netfs_group *fgroup; /* TODO: Use this with ceph */
 	struct netfs_folio *finfo;
 	struct bvecq *queue = wreq->load_cursor.bvecq;
 	unsigned int slot;
 	size_t fsize = folio_size(folio), flen = fsize, foff = 0;
-	loff_t fpos = folio_pos(folio), i_size;
-	bool to_eof = false, streamw = false;
-	bool debug = false;
-
-	_enter("");
+	uoff_t fpos = folio_pos(folio), i_size;
 
-	if (!wreq->spare) {
-		wreq->spare = bvecq_alloc_one(BVECQ_STD_SLOTS, wreq->gfp, true);
-		if (!wreq->spare)
-			return -ENOMEM;
-	}
+	_enter("%x", params->notes);
 
 	/* netfs_perform_write() may shift i_size around the folio or from out
 	 * of the folio to beyond it, but cannot move i_size into or through
@@ -369,7 +417,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 		wreq->nr_group_rel += netfs_folio_written_back(folio);
 		netfs_put_group_many(wreq->group, wreq->nr_group_rel);
 		wreq->nr_group_rel = 0;
-		return 0;
+		return;
 	}
 
 	if (fpos + fsize > wreq->i_size)
@@ -380,16 +428,20 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	if (finfo) {
 		foff = finfo->dirty_offset;
 		flen = foff + finfo->dirty_len;
-		streamw = true;
+		params->notes |= NOTE_STREAMW;
 	}
 
+	if (params->last_end && fpos != params->last_end)
+		params->notes |= NOTE_DISCONTIG_BEFORE;
+	params->last_end = fpos + fsize;
+
 	if (flen > i_size - fpos) {
 		flen = i_size - fpos;
-		if (!streamw)
+		if (!(params->notes & NOTE_STREAMW))
 			folio_zero_segment(folio, flen, fsize);
-		to_eof = true;
+		params->notes |= NOTE_TO_EOF;
 	} else if (flen == i_size - fpos) {
-		to_eof = true;
+		params->notes |= NOTE_TO_EOF;
 	}
 	flen -= foff;
 
@@ -412,58 +464,48 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	 *     write-back group.
 	 */
 	if (fgroup == NETFS_FOLIO_COPY_TO_CACHE) {
-		netfs_issue_write(wreq, upload);
+		if (!(params->notes & NOTE_CACHE_AVAIL)) {
+			trace_netfs_folio(folio, netfs_folio_trace_cancel_copy);
+			goto cancel_folio;
+		}
+		params->notes |= NOTE_CACHE_COPY;
+		trace_netfs_folio(folio, netfs_folio_trace_store_copy);
 	} else if (fgroup != wreq->group) {
 		/* We can't write this page to the server yet. */
 		kdebug("wrong group");
-		folio_redirty_for_writepage(wbc, folio);
-		folio_unlock(folio);
-		netfs_issue_write(wreq, upload);
-		netfs_issue_write(wreq, cache);
-		return 0;
+		goto skip_folio;
+	} else if (!(params->notes & (NOTE_UPLOAD_AVAIL | NOTE_CACHE_AVAIL))) {
+		trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
+		goto cancel_folio_discard;
+	} else {
+		if (params->notes & NOTE_UPLOAD_STARTED) {
+			params->notes |= NOTE_UPLOAD;
+			trace_netfs_folio(folio, netfs_folio_trace_store_plus);
+		} else {
+			params->notes |= NOTE_UPLOAD | NOTE_UPLOAD_STARTED;
+			trace_netfs_folio(folio, netfs_folio_trace_store);
+		}
+		if ((params->notes & NOTE_CACHE_AVAIL) &&
+		    !(params->notes & NOTE_STREAMW))
+			params->notes |= NOTE_CACHE_COPY;
 	}
 
-	if (foff > 0)
-		netfs_issue_write(wreq, upload);
-	if (streamw)
-		netfs_issue_write(wreq, cache);
-
 	folio_start_writeback(folio);
 	folio_unlock(folio);
 
-	if (fgroup == NETFS_FOLIO_COPY_TO_CACHE) {
-		if (!cache->avail) {
-			trace_netfs_folio(folio, netfs_folio_trace_cancel_copy);
-			netfs_issue_write(wreq, upload);
-			netfs_folio_written_back(folio);
-			return 0;
-		}
-		trace_netfs_folio(folio, netfs_folio_trace_store_copy);
-	} else if (!upload->avail && !cache->avail) {
-		trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
-		netfs_folio_written_back(folio);
-		return 0;
-	} else if (!upload->construct) {
-		trace_netfs_folio(folio, netfs_folio_trace_store);
-	} else {
-		trace_netfs_folio(folio, netfs_folio_trace_store_plus);
-	}
-
 	/* Institute a new bvec queue segment if the current one is full or if
 	 * we encounter a discontiguity.  The discontiguity break is important
 	 * when it comes to bulk unlocking folios by file range.
 	 */
 	if (bvecq_is_full(queue) ||
-	    (fpos != wreq->last_end && wreq->last_end > 0)) {
+	    ((params->notes & NOTE_DISCONTIG_BEFORE) && queue->nr_slots > 0)) {
 		bvecq_buffer_append(&wreq->load_cursor, wreq->spare);
 		wreq->spare = NULL;
 
 		queue = wreq->load_cursor.bvecq;
 		queue->fpos = fpos;
-		if (fpos != wreq->last_end)
+		if (params->notes & NOTE_DISCONTIG_BEFORE)
 			queue->discontig = true;
-		bvecq_pos_move(&wreq->dispatch_cursor, queue);
-		wreq->dispatch_cursor.slot = 0;
 	}
 
 	/* Attach the folio to the rolling buffer. */
@@ -472,114 +514,92 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	trace_netfs_bv_slot(queue, slot);
 	slot++;
 	bvecq_filled_to(queue, slot);
-	wreq->load_cursor.slot = slot;
-	wreq->load_cursor.offset = 0;
-	wreq->last_end = fpos + fsize;
+	trace_netfs_wback(wreq, folio, params->notes);
 
-	/* Move the submission point forward to allow for write-streaming data
-	 * not starting at the front of the page.  We don't do write-streaming
-	 * with the cache as the cache requires DIO alignment.
-	 *
-	 * Also skip uploading for data that's been read and just needs copying
-	 * to the cache.
-	 */
-	bvecq_pos_nudge(&wreq->dispatch_cursor);
-	
+	/* Initiate or extend the dispatch of each selected stream. */
 	for (int s = 0; s < NR_IO_STREAMS; s++) {
-		size_t soff = foff, slen = flen, alignment = 1;
-
-		if (stream->source == NETFS_WRITE_TO_CACHE)
-			alignment = wreq->cache_resources.dio_size;
-		stream = &wreq->io_streams[s];
-		stream->submit_off = round_down(soff, alignment);
-		slen += foff - stream->submit_off;
-		stream->submit_len = round_up(slen, alignment);
-
-		if (!stream->avail ||
-		    (stream->source == NETFS_WRITE_TO_CACHE && streamw) ||
-		    (stream->source == NETFS_UPLOAD_TO_SERVER &&
-		     fgroup == NETFS_FOLIO_COPY_TO_CACHE)) {
-			stream->submit_off = UINT_MAX;
-			stream->submit_len = 0;
-		}
-	}
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
+		struct netfs_write_estimate *estimate = &params->estimates[stream->stream_nr];
+		size_t soff = foff, slen = flen;
+		uoff_t start, end;
 
-	/* Attach the folio to one or more subrequests.  For a big folio, we
-	 * could end up with thousands of subrequests if the wsize is small -
-	 * but we might need to wait during the creation of subrequests for
-	 * network resources (eg. SMB credits).
-	 */
-	for (;;) {
-		ssize_t part;
-		size_t lowest_off = ULONG_MAX;
-		int choose_s = -1;
-
-		/* Always add to the lowest-submitted stream first. */
-		for (int s = 0; s < NR_IO_STREAMS; s++) {
-			stream = &wreq->io_streams[s];
-			if (stream->submit_len > 0 &&
-			    stream->submit_off < lowest_off) {
-				lowest_off = stream->submit_off;
-				choose_s = s;
+		params->notes &= ~NOTE_FLUSH_ANYWAY;
+
+		if (!stream->active || !(params->notes & stream->applicable)) {
+			if (stream->buffering) {
+				params->notes |= NOTE_FLUSH_ANYWAY;
+				netfs_writeback_flush(wreq, stream, params);
 			}
+			atomic64_set_release(&stream->issued_to, fpos + fsize);
+			continue;
 		}
 
-		if (choose_s < 0)
-			break;
-		stream = &wreq->io_streams[choose_s];
-
-		/* Advance the cursor. */
-		wreq->dispatch_cursor.offset = stream->submit_off;
-
-		atomic64_set(&wreq->issued_to, fpos + stream->submit_off);
-		part = netfs_advance_write(wreq, stream, fpos + stream->submit_off,
-					   stream->submit_len, to_eof);
-		stream->submit_off += part;
-		if (part > stream->submit_len)
-			stream->submit_len = 0;
-		else
-			stream->submit_len -= part;
-		if (part > 0)
-			debug = true;
-	}
+		/* Round out the sizes to fscache alignment. */
+		if (stream->alignment > 1) {
+			soff = round_down(soff, stream->alignment);
+			slen += foff - soff;
+			slen = round_up(slen, stream->alignment);
+		}
 
-	bvecq_pos_step(&wreq->dispatch_cursor);
-	/* Order loading the queue before updating the issue_to point */
-	atomic64_set_release(&wreq->issued_to, fpos + fsize);
+		start = fpos + soff;
+		end = start + slen;
 
-	if (!debug)
-		kdebug("R=%x: No submit", wreq->debug_id);
+		/* Flush if not contiguous with the previous slice. */
+		if (stream->buffering && start != stream->last_end) {
+			params->notes |= NOTE_FLUSH_ANYWAY;
+			netfs_writeback_flush(wreq, stream, params);
+			params->notes &= ~NOTE_FLUSH_ANYWAY;
+		}
 
-	if (foff + flen < fsize)
-		for (int s = 0; s < NR_IO_STREAMS; s++)
-			netfs_issue_write(wreq, &wreq->io_streams[s]);
+		/* Begin the assembly of a slice and get an estimate of how
+		 * much we can accumulate before we have to flush.
+		 */
+		if (!stream->buffering) {
+			stream->issue_from = start;
+			bvecq_pos_set(&stream->dispatch_cursor, &wreq->load_cursor);
+			stream->dispatch_cursor.offset = soff;
+			stream->buffering = true;
+			stream->buffered = 0;
+			estimate->issue_at = ULLONG_MAX;
+			estimate->max_segs = INT_MAX;
+			stream->estimate_write(wreq, stream, estimate);
+		}
 
-	_leave(" = 0");
-	return 0;
-}
+		stream->buffered += slen;
+		stream->last_end = end;
+		estimate->max_segs--;
 
-/*
- * End the issuing of writes, letting the collector know we're done.
- */
-static void netfs_end_issue_write(struct netfs_io_request *wreq)
-{
-	bool needs_poke = true;
+		_debug("[%u] %llx + %zx >= %llx, %u %x",
+		       stream->stream_nr, stream->issue_from, stream->buffered,
+		       estimate->issue_at, estimate->max_segs, params->notes);
 
-	smp_wmb(); /* Write subreq lists before ALL_QUEUED. */
-	set_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags);
+		if (stream->issue_from + stream->buffered >= estimate->issue_at ||
+		    estimate->max_segs <= 0)
+			netfs_writeback_flush(wreq, stream, params);
+	}
 
-	for (int s = 0; s < NR_IO_STREAMS; s++) {
-		struct netfs_io_stream *stream = &wreq->io_streams[s];
+	/* And finally advance the load cursor. */
+	wreq->load_cursor.slot = slot;
+	wreq->load_cursor.offset = 0;
 
-		if (!stream->active)
-			continue;
-		if (!list_empty(&stream->subrequests))
-			needs_poke = false;
-		netfs_issue_write(wreq, stream);
-	}
+out:
+	_leave(" = %x", params->notes);
+	return;
 
-	if (needs_poke)
-		netfs_wake_collector(wreq);
+skip_folio:
+	folio_redirty_for_writepage(wbc, folio);
+	folio_unlock(folio);
+	params->notes |= NOTE_DISCONTIG_BEFORE;
+	goto out;
+cancel_folio_discard:
+	netfs_put_group(fgroup);
+cancel_folio:
+	folio_detach_private(folio);
+	kfree(finfo);
+	folio_unlock(folio);
+	folio_cancel_dirty(folio);
+	params->notes |= NOTE_DISCONTIG_BEFORE;
+	goto out;
 }
 
 /*
@@ -590,6 +610,7 @@ int netfs_writepages(struct address_space *mapping,
 {
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
 	struct netfs_io_request *wreq = NULL;
+	struct netfs_wb_params params = {};
 	struct folio *folio;
 	int error = 0;
 
@@ -609,35 +630,45 @@ int netfs_writepages(struct address_space *mapping,
 
 	if (bvecq_buffer_init(&wreq->load_cursor, wreq->gfp, true) < 0)
 		goto nomem;
-	bvecq_pos_set(&wreq->dispatch_cursor, &wreq->load_cursor);
-	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->load_cursor);
 
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback);
 	netfs_stat(&netfs_n_wh_writepages);
 
+	if (wreq->io_streams[1].avail)
+		params.notes |= NOTE_CACHE_AVAIL;
+
 	do {
-		_debug("wbiter %lx %llx", folio->index, atomic64_read(&wreq->issued_to));
+		_debug("wbiter %lx", folio->index);
 
-		/* It appears we don't have to handle cyclic writeback wrapping. */
-		WARN_ON_ONCE(wreq && folio_pos(folio) < atomic64_read(&wreq->issued_to));
+		if (!wreq->spare) {
+			wreq->spare = bvecq_alloc_one(BVECQ_STD_SLOTS, wreq->gfp, true);
+			/* bvecq allocation in writeback is backed by a mempool
+			 * and will wait for an new one to come available.
+			 */
+		}
 
 		if (netfs_folio_group(folio) != NETFS_FOLIO_COPY_TO_CACHE &&
 		    unlikely(!test_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))) {
 			set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags);
 			wreq->netfs_ops->begin_writeback(wreq);
+			if (wreq->io_streams[0].avail) {
+				params.notes |= NOTE_UPLOAD_AVAIL;
+				/* Order setting the active flag after other fields. */
+				smp_store_release(&wreq->io_streams[0].active, true);
+			}
 		}
 
-		error = netfs_write_folio(wreq, wbc, folio);
-		if (error == -ENOMEM) {
-			folio_redirty_for_writepage(wbc, folio);
-			folio_unlock(folio);
-		}
+		params.notes &= NOTES__KEEP_MASK;
+		netfs_writeback_folio(wreq, wbc, folio, &params);
 	} while ((folio = writeback_iter(mapping, wbc, folio, &error)));
 
-	netfs_end_issue_write(wreq);
+	netfs_writeback_end(wreq, &params);
+
 	bvecq_pos_unset(&wreq->load_cursor);
-	bvecq_pos_unset(&wreq->dispatch_cursor);
+	for (int i = 0; i < NR_IO_STREAMS; i++)
+		bvecq_pos_unset(&wreq->io_streams[i].dispatch_cursor);
 	netfs_wake_collector(wreq);
 
 	netfs_put_request(wreq, netfs_rreq_trace_put_return);
@@ -663,11 +694,46 @@ int netfs_writepages(struct address_space *mapping,
 }
 EXPORT_SYMBOL(netfs_writepages);
 
+/*
+ * Prepare a buffer for a single monolithic write.
+ */
+static int netfs_prepare_write_single_buffer(struct netfs_io_subrequest *subreq,
+					     unsigned int max_segs)
+{
+	struct netfs_io_request *wreq = subreq->rreq;
+	struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];
+	struct bio_vec *bv;
+	struct bvecq *bq;
+	size_t dio_size = wreq->cache_resources.dio_size;
+	size_t dlen;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &stream->dispatch_cursor);
+	bvecq_pos_set(&subreq->content, &subreq->dispatch_pos);
+
+	/* Round the end of the last entry up. */
+	bq = subreq->content.bvecq;
+	while (bq->next)
+		bq = bq->next;
+	bv = &bq->bv[bq->nr_slots - 1];
+	dlen = round_up(bv->bv_len, dio_size);
+	if (dlen > bv->bv_len) {
+		subreq->len += dlen - bv->bv_len;
+		bv->bv_len = dlen;
+	}
+
+	stream->buffered   = 0;
+	stream->issue_from = subreq->len;
+	wreq->submitted    = subreq->len;
+	netfs_all_subreqs_queued(wreq);
+	return 0;
+}
+
 /**
  * netfs_writeback_single - Write back a monolithic payload
  * @mapping: The mapping to write from
  * @wbc: Hints from the VM
- * @iter: Data to write.
+ * @iter: Data to write
+ * @len: Amount of data to write
  *
  * Write a monolithic, non-pagecache object back to the server and/or
  * the cache.  There's a maximum of one subrequest per stream.
@@ -678,12 +744,18 @@ EXPORT_SYMBOL(netfs_writepages);
  */
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
-			   struct iov_iter *iter)
+			   struct iov_iter *iter,
+			   size_t len)
 {
 	struct netfs_io_request *wreq;
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
 	int ret;
 
+	_enter("%zx,%zx", iov_iter_count(iter), len);
+
+	if (!len)
+		return 0;
+
 	if (!netfs_wb_begin(ictx, wbc->sync_mode == WB_SYNC_NONE)) {
 		/* The VFS will have undirtied the inode. */
 		netfs_single_mark_inode_dirty(&ictx->inode);
@@ -695,18 +767,19 @@ int netfs_writeback_single(struct address_space *mapping,
 		ret = PTR_ERR(wreq);
 		goto couldnt_start;
 	}
-	wreq->len = iov_iter_count(iter);
 
-	ret = netfs_extract_iter(iter, wreq->len, INT_MAX, 0, &wreq->dispatch_cursor.bvecq,
+	wreq->len = len;
+
+	ret = netfs_extract_iter(iter, len, INT_MAX, 0, &wreq->load_cursor.bvecq,
 				 0, wreq->gfp);
 	if (ret < 0)
 		goto cleanup_free;
-	if (ret < wreq->len) {
+	if (ret < len) {
 		ret = -EIO;
 		goto cleanup_free;
 	}
 
-	bvecq_pos_set(&wreq->collect_cursor, &wreq->dispatch_cursor);
+	bvecq_pos_set(&wreq->collect_cursor, &wreq->load_cursor);
 
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback_single);
@@ -722,20 +795,34 @@ int netfs_writeback_single(struct address_space *mapping,
 		if (!stream->avail)
 			continue;
 
-		netfs_prepare_write(wreq, stream, 0);
+		stream->issue_from = 0;
+		stream->buffered   = len;
 
-		subreq = stream->construct;
-		subreq->len = wreq->len;
-		stream->submit_len = subreq->len;
+		subreq = netfs_alloc_write_subreq(wreq, stream);
+		if (!subreq) {
+			ret = -ENOMEM;
+			break;
+		}
+
+		bvecq_pos_set(&stream->dispatch_cursor, &wreq->load_cursor);
+
+		ret = stream->issue_write(subreq);
+		if (ret < 0) {
+			/* Ownership of subreq was returned to us. */
+			trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+			stream->buffered -= subreq->len;
+			netfs_write_subrequest_terminated(subreq, ret);
+			/* Punt the error to the collector. */
+		}
 
-		netfs_issue_write(wreq, stream);
+		bvecq_pos_unset(&stream->dispatch_cursor);
 	}
 
 	wreq->submitted = wreq->len;
-	smp_wmb(); /* Write lists before ALL_QUEUED. */
-	set_bit(NETFS_RREQ_ALL_QUEUED, &wreq->flags);
-
-	netfs_wake_collector(wreq);
+	if (unlikely(!netfs_are_all_subreqs_queued(wreq))) {
+		netfs_all_subreqs_queued(wreq);
+		netfs_wake_collector(wreq);
+	}
 
 	/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to
 	 * wait before modifying.
diff --git a/fs/netfs/write_retry.c b/fs/netfs/write_retry.c
index 4e320591f83a..ff783ea2dc61 100644
--- a/fs/netfs/write_retry.c
+++ b/fs/netfs/write_retry.c
@@ -12,13 +12,44 @@
 #include "internal.h"
 
 /*
- * Perform retries on the streams that need it.
+ * Prepare the write buffer for a retry.  We can't necessarily reuse the write
+ * buffer from the previous run of a subrequest because the filesystem is
+ * permitted to modify it (add headers/trailers, encrypt it).  Further, the
+ * subrequest may now be a different size (e.g. cifs has to negotiate for
+ * maximum transfer size).  Also, we can't look at *stream as that may still
+ * refer to the source material being broken up into original subrequests.
+ */
+int netfs_prepare_write_retry_buffer(struct netfs_io_subrequest *subreq,
+				     unsigned int max_segs)
+{
+	struct netfs_io_request *wreq = subreq->rreq;
+	size_t len;
+
+	bvecq_pos_set(&subreq->dispatch_pos, &wreq->retry_cursor);
+	bvecq_pos_set(&subreq->content, &wreq->retry_cursor);
+	len = bvecq_slice(&wreq->retry_cursor, subreq->len, max_segs, &subreq->nr_segs);
+
+	if (len < subreq->len) {
+		subreq->len = len;
+		trace_netfs_sreq(subreq, netfs_sreq_trace_limited);
+	}
+
+	wreq->retry_start += len;
+	wreq->retry_buffered   -= len;
+	if (wreq->retry_buffered == 0)
+		bvecq_pos_unset(&wreq->retry_cursor);
+	return 0;
+}
+
+/*
+ * Perform retries on the streams that need it.  This only has to deal with
+ * buffered writes; unbuffered write retry is handled in direct_write.c.
  */
 static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				     struct netfs_io_stream *stream)
 {
-	struct bvecq_pos dispatch_cursor = {};
 	struct list_head *next;
+	int ret;
 
 	_enter("R=%x[%x:]", wreq->debug_id, stream->stream_nr);
 
@@ -33,30 +64,15 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 	if (unlikely(stream->failed))
 		return;
 
-	/* If there's no renegotiation to do, just resend each failed subreq. */
-	if (!stream->prepare_write) {
-		struct netfs_io_subrequest *subreq;
-
-		list_for_each_entry(subreq, &stream->subrequests, rreq_link) {
-			if (test_bit(NETFS_SREQ_FAILED, &subreq->flags))
-				break;
-			if (__test_and_clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags)) {
-				netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-				netfs_reissue_write(stream, subreq);
-			}
-		}
-		return;
-	}
-
-	next = stream->subrequests.next;
+	/* Read pointer to subreq before reading subreq state. */
+	next = smp_load_acquire(&stream->subrequests.next);
 
 	do {
 		struct netfs_io_subrequest *subreq = NULL, *from, *to, *tmp;
 		unsigned long long start, len;
-		size_t part;
-		bool boundary = false;
+		bool subreq_superfluous = false;
 
-		bvecq_pos_unset(&dispatch_cursor);
+		bvecq_pos_unset(&wreq->retry_cursor);
 
 		/* Go through the stream and find the next span of contiguous
 		 * data that we then rejig (cifs, for example, needs the wsize
@@ -80,7 +96,6 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			subreq = list_entry(next, struct netfs_io_subrequest, rreq_link);
 			if (subreq->start != start + len ||
 			    subreq->transferred > 0 ||
-			    test_bit(NETFS_SREQ_BOUNDARY, &subreq->flags) ||
 			    !test_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags))
 				break;
 			to = subreq;
@@ -90,8 +105,10 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 		/* Determine the set of buffers we're going to use.  Each
 		 * subreq gets a subset of a single overall contiguous buffer.
 		 */
-		bvecq_pos_transfer(&dispatch_cursor, &from->dispatch_pos);
-		bvecq_pos_advance(&dispatch_cursor, from->transferred);
+		bvecq_pos_transfer(&wreq->retry_cursor, &from->dispatch_pos);
+		bvecq_pos_advance(&wreq->retry_cursor, from->transferred);
+		wreq->retry_start = start;
+		wreq->retry_buffered = len;
 
 		/* Work through the sublist.  The chain of buffers we're going
 		 * to fill is attached to dispatch_cursor and we need to read
@@ -99,47 +116,50 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 		 */
 		subreq = from;
 		list_for_each_entry_from(subreq, &stream->subrequests, rreq_link) {
-			if (!len)
+			if (!wreq->retry_buffered) {
+				subreq_superfluous = true;
 				break;
-
-			subreq->start	= start;
-			subreq->len	= len;
-			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
-			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
-			subreq->transferred = 0;
+			}
 
 			bvecq_pos_unset(&subreq->content);
 			bvecq_pos_unset(&subreq->dispatch_pos);
 
-			/* Renegotiate max_len (wsize) */
-			stream->sreq_max_len = len;
-			stream->sreq_max_segs = INT_MAX;
-			stream->prepare_write(subreq);
-
-			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
-			part = bvecq_slice(&dispatch_cursor,
-					   umin(len, stream->sreq_max_len),
-					   stream->sreq_max_segs,
-					   &subreq->nr_segs);
-			subreq->len = part;
-
-			len -= part;
-			start += part;
-			if (len && subreq == to &&
-			    __test_and_clear_bit(NETFS_SREQ_BOUNDARY, &to->flags))
-				boundary = true;
-
+			__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
+			__clear_bit(NETFS_SREQ_NEED_RETRY, &subreq->flags);
+			__clear_bit(NETFS_SREQ_FAILED, &subreq->flags);
+			__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
+			subreq->start		= wreq->retry_start;
+			subreq->len		= wreq->retry_buffered;
+			subreq->transferred	= 0;
+			subreq->retry_count	+= 1;
+			subreq->error		= 0;
+
+			netfs_stat(&netfs_n_wh_retry_write_subreq);
+			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 			netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
-			netfs_reissue_write(stream, subreq);
-			if (subreq == to)
+			ret = stream->issue_write(subreq);
+			if (ret < 0) {
+				/* Ownership of subreq was returned to us.
+				 * Expand the subreq to consume the entire
+				 * remaining amount to be retried and fail it.
+				 */
+				trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+				wreq->retry_buffered -= subreq->len;
+				netfs_write_subrequest_terminated(subreq, ret);
+				break;
+			}
+
+			if (subreq == to) {
+				subreq_superfluous = false;
 				break;
+			}
 		}
 
 		/* If we managed to use fewer subreqs, we can discard the
 		 * excess; if we used the same number, then we're done.
 		 */
-		if (!len) {
-			if (subreq == to)
+		if (!wreq->retry_buffered) {
+			if (!subreq_superfluous)
 				continue;
 			list_for_each_entry_safe_from(subreq, tmp,
 						      &stream->subrequests, rreq_link) {
@@ -161,6 +181,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			subreq = netfs_alloc_subrequest(wreq);
 			subreq->source		= to->source;
 			subreq->start		= start;
+			subreq->len		= wreq->retry_buffered;
 			subreq->stream_nr	= to->stream_nr;
 			subreq->retry_count	= 1;
 
@@ -169,49 +190,40 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 					     netfs_sreq_trace_new);
 			trace_netfs_sreq(subreq, netfs_sreq_trace_split);
 
+			__set_bit(NETFS_SREQ_IN_PROGRESS, &subreq->flags);
 			spin_lock(&wreq->lock);
+			/* Write IN_PROGRESS before pointer to new subreq */
+			smp_wmb();
 			list_add(&subreq->rreq_link, &to->rreq_link);
 			spin_unlock(&wreq->lock);
 			to = subreq;
-			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
 
-			stream->sreq_max_len	= len;
-			stream->sreq_max_segs	= INT_MAX;
 			switch (stream->source) {
 			case NETFS_UPLOAD_TO_SERVER:
 				netfs_stat(&netfs_n_wh_upload);
-				stream->sreq_max_len = umin(len, wreq->wsize);
 				break;
 			default:
 				WARN_ON_ONCE(1);
 			}
 
-			stream->prepare_write(subreq);
-
-			bvecq_pos_set(&subreq->dispatch_pos, &dispatch_cursor);
-			part = bvecq_slice(&dispatch_cursor,
-					   umin(len, stream->sreq_max_len),
-					   stream->sreq_max_segs,
-					   &subreq->nr_segs);
-			subreq->len = subreq->transferred + part;
-
-			len -= part;
-			start += part;
-			if (!len && boundary) {
-				__set_bit(NETFS_SREQ_BOUNDARY, &to->flags);
-				boundary = false;
-			}
-
-			netfs_reissue_write(stream, subreq);
-			if (!len)
+			trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+			ret = stream->issue_write(subreq);
+			if (ret < 0) {
+				/* Ownership of subreq was returned to us.
+				 * Expand the subreq to consume the entire
+				 * remaining amount to be retried and fail it.
+				 */
+				trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
+				wreq->retry_buffered -= subreq->len;
+				netfs_write_subrequest_terminated(subreq, ret);
 				break;
-
-		} while (len);
+			}
+		} while (wreq->retry_buffered > 0);
 
 	} while (!list_is_head(next, &stream->subrequests));
 
 out:
-	bvecq_pos_unset(&dispatch_cursor);
+	bvecq_pos_unset(&wreq->retry_cursor);
 }
 
 /*
diff --git a/fs/nfs/Kconfig b/fs/nfs/Kconfig
index 6bb30543eff0..e7862f35b72c 100644
--- a/fs/nfs/Kconfig
+++ b/fs/nfs/Kconfig
@@ -174,6 +174,7 @@ config NFS_FSCACHE
 	bool "Provide NFS client caching support"
 	depends on NFS_FS
 	select NETFS_SUPPORT
+	select NETFS_PGPRIV2
 	select FSCACHE
 	help
 	  Say Y here if you want NFS data to be cached locally on disc through
diff --git a/fs/nfs/fscache.c b/fs/nfs/fscache.c
index 9b7fdad4a920..fb1441f88661 100644
--- a/fs/nfs/fscache.c
+++ b/fs/nfs/fscache.c
@@ -23,6 +23,7 @@
 #include "iostat.h"
 #include "fscache.h"
 #include "nfstrace.h"
+#include <trace/events/netfs.h>
 
 #define NFS_MAX_KEY_LEN 1000
 
@@ -273,8 +274,6 @@ static int nfs_netfs_init_request(struct netfs_io_request *rreq, struct file *fi
 	rreq->debug_id = atomic_inc_return(&nfs_netfs_debug_id);
 	/* [DEPRECATED] Use PG_private_2 to mark folio being written to the cache. */
 	__set_bit(NETFS_RREQ_USE_PGPRIV2, &rreq->flags);
-	rreq->io_streams[0].sreq_max_len = NFS_SB(rreq->inode->i_sb)->rsize;
-
 	return 0;
 }
 
@@ -296,8 +295,9 @@ static struct nfs_netfs_io_data *nfs_netfs_alloc(struct netfs_io_subrequest *sre
 	return netfs;
 }
 
-static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
+static int nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
 {
+	struct netfs_io_request		*rreq = sreq->rreq;
 	struct nfs_netfs_io_data	*netfs;
 	struct nfs_pageio_descriptor	pgio;
 	struct inode *inode = sreq->rreq->inode;
@@ -307,6 +307,16 @@ static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
 	pgoff_t start, last;
 	int err;
 
+	if (sreq->len > NFS_SB(rreq->inode->i_sb)->rsize)
+		sreq->len = NFS_SB(rreq->inode->i_sb)->rsize;
+
+	err = netfs_prepare_read_buffer(sreq, INT_MAX);
+	if (err < 0) {
+		sreq->error = err;
+		return err;
+	}
+	/* After this point, must fail by termination. */
+
 	start = (sreq->start + sreq->transferred) >> PAGE_SHIFT;
 	last = ((sreq->start + sreq->len - sreq->transferred - 1) >> PAGE_SHIFT);
 
@@ -316,12 +326,15 @@ static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
 	netfs = nfs_netfs_alloc(sreq);
 	if (!netfs) {
 		sreq->error = -ENOMEM;
-		return netfs_read_subreq_terminated(sreq);
+		netfs_read_subreq_terminated(sreq);
+		return 0;
 	}
 
+	trace_netfs_sreq(sreq, netfs_sreq_trace_submit);
+
 	pgio.pg_netfs = netfs; /* used in completion */
 
-	xa_for_each_range(&sreq->rreq->mapping->i_pages, idx, page, start, last) {
+	xa_for_each_range(&rreq->mapping->i_pages, idx, page, start, last) {
 		/* nfs_read_add_folio() may schedule() due to pNFS layout and other RPCs  */
 		err = nfs_read_add_folio(&pgio, ctx, page_folio(page));
 		if (err < 0) {
@@ -332,6 +345,7 @@ static void nfs_netfs_issue_read(struct netfs_io_subrequest *sreq)
 out:
 	nfs_pageio_complete_read(&pgio);
 	nfs_netfs_put(netfs);
+	return 0;
 }
 
 void nfs_netfs_initiate_read(struct nfs_pgio_header *hdr)
diff --git a/fs/smb/client/cifssmb.c b/fs/smb/client/cifssmb.c
index 1f77512252e7..37d576d38837 100644
--- a/fs/smb/client/cifssmb.c
+++ b/fs/smb/client/cifssmb.c
@@ -1467,8 +1467,7 @@ cifs_readv_callback(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 	struct cifs_tcon *tcon = tlink_tcon(rdata->req->cfile->tlink);
 	struct inode *inode = &ictx->inode;
 	struct smb_rqst rqst = { .rq_iov = rdata->iov,
-				 .rq_nvec = 1,
-				 .rq_iter = rdata->subreq.io_iter };
+				 .rq_nvec = 1};
 	struct cifs_credits credits = {
 		.value = 1,
 		.instance = 0,
@@ -1482,6 +1481,11 @@ cifs_readv_callback(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 		 __func__, mid->mid, mid->mid_state, rdata->result,
 		 rdata->subreq.len);
 
+	if (rdata->got_bytes)
+		iov_iter_bvec_queue(&rqst.rq_iter, ITER_DEST,
+				    rdata->subreq.content.bvecq, rdata->subreq.content.slot,
+				    rdata->subreq.content.offset, rdata->subreq.len);
+
 	switch (mid->mid_state) {
 	case MID_RESPONSE_RECEIVED:
 		/* result already set, check signature */
@@ -2007,7 +2011,10 @@ cifs_async_writev(struct cifs_io_subrequest *wdata)
 
 	rqst.rq_iov = iov;
 	rqst.rq_nvec = 1;
-	rqst.rq_iter = wdata->subreq.io_iter;
+
+	iov_iter_bvec_queue(&rqst.rq_iter, ITER_SOURCE,
+			    wdata->subreq.content.bvecq, wdata->subreq.content.slot,
+			    wdata->subreq.content.offset, wdata->subreq.len);
 
 	cifs_dbg(FYI, "async write at %llu %zu bytes\n",
 		 wdata->subreq.start, wdata->subreq.len);
diff --git a/fs/smb/client/file.c b/fs/smb/client/file.c
index ac89c1ba56b1..bc08e5808cf0 100644
--- a/fs/smb/client/file.c
+++ b/fs/smb/client/file.c
@@ -43,18 +43,34 @@ static int cifs_reopen_file(struct cifsFileInfo *cfile, bool can_flush);
  * Prepare a subrequest to upload to the server.  We need to allocate credits
  * so that we know the maximum amount of data that we can include in it.
  */
-static void cifs_prepare_write(struct netfs_io_subrequest *subreq)
+static int cifs_estimate_write(struct netfs_io_request *wreq,
+			       struct netfs_io_stream *stream,
+			       struct netfs_write_estimate *estimate)
+{
+	struct cifs_sb_info *cifs_sb = CIFS_SB(wreq->inode->i_sb);
+
+	estimate->issue_at = stream->issue_from + cifs_sb->ctx->wsize;
+	return 0;
+}
+
+/*
+ * Issue a subrequest to upload to the server.
+ */
+static int cifs_issue_write(struct netfs_io_subrequest *subreq)
 {
 	struct cifs_io_subrequest *wdata =
 		container_of(subreq, struct cifs_io_subrequest, subreq);
 	struct cifs_io_request *req = wdata->req;
-	struct netfs_io_stream *stream = &req->rreq.io_streams[subreq->stream_nr];
 	struct TCP_Server_Info *server;
 	struct cifsFileInfo *open_file = req->cfile;
-	struct cifs_sb_info *cifs_sb = CIFS_SB(wdata->rreq->inode->i_sb);
-	size_t wsize = req->rreq.wsize;
+	struct cifs_sb_info *cifs_sb = CIFS_SB(subreq->rreq->inode->i_sb);
+	unsigned int max_segs = INT_MAX;
+	size_t len;
 	int rc;
 
+	if (cifs_forced_shutdown(cifs_sb))
+		return smb_EIO(smb_eio_trace_forced_shutdown);
+
 	if (!wdata->have_xid) {
 		wdata->xid = get_xid();
 		wdata->have_xid = true;
@@ -73,18 +89,16 @@ static void cifs_prepare_write(struct netfs_io_subrequest *subreq)
 		if (rc < 0) {
 			if (rc == -EAGAIN)
 				goto retry;
-			subreq->error = rc;
-			return netfs_prepare_write_failed(subreq);
+			return rc;
 		}
 	}
 
-	rc = server->ops->wait_mtu_credits(server, wsize, &stream->sreq_max_len,
-					   &wdata->credits);
-	if (rc < 0) {
-		subreq->error = rc;
-		return netfs_prepare_write_failed(subreq);
-	}
+	len = umin(subreq->len, cifs_sb->ctx->wsize);
+	rc = server->ops->wait_mtu_credits(server, len, &len, &wdata->credits);
+	if (rc < 0)
+		return rc;
 
+	subreq->len = len;
 	wdata->credits.rreq_debug_id = subreq->rreq->debug_id;
 	wdata->credits.rreq_debug_index = subreq->debug_index;
 	wdata->credits.in_flight_check = 1;
@@ -100,46 +114,34 @@ static void cifs_prepare_write(struct netfs_io_subrequest *subreq)
 		const struct smbdirect_socket_parameters *sp =
 			smbd_get_parameters(server->smbd_conn);
 
-		stream->sreq_max_segs = sp->max_frmr_depth;
+		max_segs = sp->max_frmr_depth;
 	}
 #endif
-}
 
-/*
- * Issue a subrequest to upload to the server.
- */
-static void cifs_issue_write(struct netfs_io_subrequest *subreq)
-{
-	struct cifs_io_subrequest *wdata =
-		container_of(subreq, struct cifs_io_subrequest, subreq);
-	struct cifs_sb_info *sbi = CIFS_SB(subreq->rreq->inode->i_sb);
-	int rc;
-
-	if (cifs_forced_shutdown(sbi)) {
-		rc = smb_EIO(smb_eio_trace_forced_shutdown);
-		goto fail;
+	rc = netfs_prepare_write_buffer(subreq, max_segs);
+	if (rc < 0) {
+		add_credits_and_wake_if(wdata->server, &wdata->credits, 0);
+		return rc;
 	}
+	/* After this point, must fail by termination. */
 
-	rc = adjust_credits(wdata->server, wdata, cifs_trace_rw_credits_issue_write_adjust);
+	rc = adjust_credits(server, wdata, cifs_trace_rw_credits_issue_write_adjust);
 	if (rc)
-		goto fail;
+		goto fail_with_credits;
 
 	rc = -EAGAIN;
 	if (wdata->req->cfile->invalidHandle)
-		goto fail;
+		goto fail_with_credits;
 
 	wdata->server->ops->async_writev(wdata);
-out:
-	return;
+	return 0;
 
-fail:
+fail_with_credits:
 	if (rc == -EAGAIN)
 		trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
-	else
-		trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
 	add_credits_and_wake_if(wdata->server, &wdata->credits, 0);
 	cifs_write_subrequest_terminated(wdata, rc);
-	goto out;
+	return 0;
 }
 
 static void cifs_netfs_invalidate_cache(struct netfs_io_request *wreq)
@@ -148,17 +150,25 @@ static void cifs_netfs_invalidate_cache(struct netfs_io_request *wreq)
 }
 
 /*
- * Negotiate the size of a read operation on behalf of the netfs library.
+ * Issue a read operation on behalf of the netfs helper functions.  We're asked
+ * to make a read of a certain size at a point in the file.  We are permitted
+ * to only read a portion of that, but as long as we read something, the netfs
+ * helper will call us again so that we can issue another read.
  */
-static int cifs_prepare_read(struct netfs_io_subrequest *subreq)
+static int cifs_issue_read(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
 	struct cifs_io_subrequest *rdata = container_of(subreq, struct cifs_io_subrequest, subreq);
 	struct cifs_io_request *req = container_of(subreq->rreq, struct cifs_io_request, rreq);
-	struct TCP_Server_Info *server;
+	struct TCP_Server_Info *server = rdata->server;
 	struct cifs_sb_info *cifs_sb = CIFS_SB(rreq->inode->i_sb);
-	size_t size;
-	int rc = 0;
+	unsigned int max_segs = INT_MAX;
+	size_t len;
+	int rc;
+
+	cifs_dbg(FYI, "%s: op=%08x[%x] mapping=%p len=%zu/%zu\n",
+		 __func__, rreq->debug_id, subreq->debug_index, rreq->mapping,
+		 subreq->transferred, subreq->len);
 
 	if (!rdata->have_xid) {
 		rdata->xid = get_xid();
@@ -172,17 +182,15 @@ static int cifs_prepare_read(struct netfs_io_subrequest *subreq)
 		cifs_negotiate_rsize(server, cifs_sb->ctx,
 				     tlink_tcon(req->cfile->tlink));
 
-	rc = server->ops->wait_mtu_credits(server, cifs_sb->ctx->rsize,
-					   &size, &rdata->credits);
+	len = umin(subreq->len, cifs_sb->ctx->rsize);
+	rc = server->ops->wait_mtu_credits(server, len, &len, &rdata->credits);
 	if (rc)
 		return rc;
 
-	rreq->io_streams[0].sreq_max_len = size;
-
-	rdata->credits.in_flight_check = 1;
+	subreq->len = len;
 	rdata->credits.rreq_debug_id = rreq->debug_id;
 	rdata->credits.rreq_debug_index = subreq->debug_index;
-
+	rdata->credits.in_flight_check = 1;
 	trace_smb3_rw_credits(rdata->rreq->debug_id,
 			      rdata->subreq.debug_index,
 			      rdata->credits.value,
@@ -194,40 +202,27 @@ static int cifs_prepare_read(struct netfs_io_subrequest *subreq)
 		const struct smbdirect_socket_parameters *sp =
 			smbd_get_parameters(server->smbd_conn);
 
-		rreq->io_streams[0].sreq_max_segs = sp->max_frmr_depth;
+		max_segs = sp->max_frmr_depth;
 	}
 #endif
-	return 0;
-}
-
-/*
- * Issue a read operation on behalf of the netfs helper functions.  We're asked
- * to make a read of a certain size at a point in the file.  We are permitted
- * to only read a portion of that, but as long as we read something, the netfs
- * helper will call us again so that we can issue another read.
- */
-static void cifs_issue_read(struct netfs_io_subrequest *subreq)
-{
-	struct netfs_io_request *rreq = subreq->rreq;
-	struct cifs_io_subrequest *rdata = container_of(subreq, struct cifs_io_subrequest, subreq);
-	struct cifs_io_request *req = container_of(subreq->rreq, struct cifs_io_request, rreq);
-	struct TCP_Server_Info *server = rdata->server;
-	int rc = 0;
 
-	cifs_dbg(FYI, "%s: op=%08x[%x] mapping=%p len=%zu/%zu\n",
-		 __func__, rreq->debug_id, subreq->debug_index, rreq->mapping,
-		 subreq->transferred, subreq->len);
+	rc = netfs_prepare_read_buffer(subreq, max_segs);
+	if (rc < 0) {
+		add_credits_and_wake_if(rdata->server, &rdata->credits, 0);
+		return rc;
+	}
+	/* After this point, must fail by termination. */
 
 	rc = adjust_credits(server, rdata, cifs_trace_rw_credits_issue_read_adjust);
 	if (rc)
-		goto failed;
+		goto fail_with_credits;
 
 	if (req->cfile->invalidHandle) {
 		do {
 			rc = cifs_reopen_file(req->cfile, true);
 		} while (rc == -EAGAIN);
 		if (rc)
-			goto failed;
+			goto fail_with_credits;
 	}
 
 	if (subreq->rreq->origin != NETFS_UNBUFFERED_READ &&
@@ -235,15 +230,22 @@ static void cifs_issue_read(struct netfs_io_subrequest *subreq)
 		__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
 
 	trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
+
 	rc = rdata->server->ops->async_readv(rdata);
 	if (rc)
 		goto failed;
-	return;
+	return 0;
 
+fail_with_credits:
+	if (rc == -EAGAIN)
+		trace_netfs_sreq(subreq, netfs_sreq_trace_retry);
+	else
+		trace_netfs_sreq(subreq, netfs_sreq_trace_fail);
 failed:
 	add_credits_and_wake_if(rdata->server, &rdata->credits, 0);
 	subreq->error = rc;
 	netfs_read_subreq_terminated(subreq);
+	return 0;
 }
 
 /*
@@ -358,11 +360,10 @@ const struct netfs_request_ops cifs_req_ops = {
 	.init_request		= cifs_init_request,
 	.free_request		= cifs_free_request,
 	.free_subrequest	= cifs_free_subrequest,
-	.prepare_read		= cifs_prepare_read,
 	.issue_read		= cifs_issue_read,
 	.done			= cifs_rreq_done,
 	.begin_writeback	= cifs_begin_writeback,
-	.prepare_write		= cifs_prepare_write,
+	.estimate_write		= cifs_estimate_write,
 	.issue_write		= cifs_issue_write,
 	.invalidate_cache	= cifs_netfs_invalidate_cache,
 };
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index 97823bcf198c..1f3266f89cce 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -4884,6 +4884,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 	unsigned int cur_page_idx;
 	unsigned int pad_len;
 	struct cifs_io_subrequest *rdata = mid->callback_data;
+	struct iov_iter iter;
 	struct smb2_hdr *shdr = (struct smb2_hdr *)buf;
 	size_t copied;
 	bool use_rdma_mr = false;
@@ -4956,6 +4957,10 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 
 	pad_len = data_offset - server->vals->read_rsp_size;
 
+	iov_iter_bvec_queue(&iter, ITER_DEST,
+			    rdata->subreq.content.bvecq, rdata->subreq.content.slot,
+			    rdata->subreq.content.offset, rdata->subreq.len);
+
 	if (buf_len <= data_offset) {
 		/* read response payload is in pages */
 		cur_page_idx = pad_len / PAGE_SIZE;
@@ -4985,7 +4990,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 
 		/* Copy the data to the output I/O iterator. */
 		rdata->result = cifs_copy_bvecq_to_iter(buffer, data_len,
-							cur_off, &rdata->subreq.io_iter);
+							cur_off, &iter);
 		if (rdata->result != 0) {
 			if (is_offloaded)
 				mid->mid_state = MID_RESPONSE_MALFORMED;
@@ -4999,7 +5004,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
 		   buf_len >= end_off) {
 		/* read response payload is in buf */
 		WARN_ONCE(buffer, "read data can be either in buf or in buffer");
-		copied = copy_to_iter(buf + data_offset, data_len, &rdata->subreq.io_iter);
+		copied = copy_to_iter(buf + data_offset, data_len, &iter);
 		if (copied == 0)
 			return smb_EIO2(smb_eio_trace_rx_copy_to_iter, copied, data_len);
 		rdata->got_bytes = copied;
diff --git a/fs/smb/client/smb2pdu.c b/fs/smb/client/smb2pdu.c
index 4ce165e40657..085fdc7d8bec 100644
--- a/fs/smb/client/smb2pdu.c
+++ b/fs/smb/client/smb2pdu.c
@@ -4592,9 +4592,13 @@ smb2_new_read_req(void **buf, unsigned int *total_len,
 	 */
 	if (rdata && smb3_use_rdma_offload(io_parms)) {
 		struct smbdirect_buffer_descriptor_v1 *v1;
+		struct iov_iter iter;
 		bool need_invalidate = server->dialect == SMB30_PROT_ID;
 
-		rdata->mr = smbd_register_mr(server->smbd_conn, &rdata->subreq.io_iter,
+		iov_iter_bvec_queue(&iter, ITER_DEST,
+				    rdata->subreq.content.bvecq, rdata->subreq.content.slot,
+				    rdata->subreq.content.offset, rdata->subreq.len);
+		rdata->mr = smbd_register_mr(server->smbd_conn, &iter,
 					     true, need_invalidate);
 		if (!rdata->mr)
 			return -EAGAIN;
@@ -4658,9 +4662,10 @@ smb2_readv_callback(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 	unsigned int rreq_debug_id = rdata->rreq->debug_id;
 	unsigned int subreq_debug_index = rdata->subreq.debug_index;
 
-	if (rdata->got_bytes) {
-		rqst.rq_iter	  = rdata->subreq.io_iter;
-	}
+	if (rdata->got_bytes)
+		iov_iter_bvec_queue(&rqst.rq_iter, ITER_DEST,
+				    rdata->subreq.content.bvecq, rdata->subreq.content.slot,
+				    rdata->subreq.content.offset, rdata->subreq.len);
 
 	WARN_ONCE(rdata->server != server,
 		  "rdata server %p != mid server %p",
@@ -5148,7 +5153,9 @@ smb2_async_writev(struct cifs_io_subrequest *wdata)
 		goto out;
 
 	rqst.rq_iov = iov;
-	rqst.rq_iter = wdata->subreq.io_iter;
+	iov_iter_bvec_queue(&rqst.rq_iter, ITER_SOURCE,
+			    wdata->subreq.content.bvecq, wdata->subreq.content.slot,
+			    wdata->subreq.content.offset, wdata->subreq.len);
 
 	rqst.rq_iov[0].iov_len = total_len - 1;
 	rqst.rq_iov[0].iov_base = (char *)req;
@@ -5187,9 +5194,14 @@ smb2_async_writev(struct cifs_io_subrequest *wdata)
 	 */
 	if (smb3_use_rdma_offload(io_parms)) {
 		struct smbdirect_buffer_descriptor_v1 *v1;
+		struct iov_iter iter;
 		bool need_invalidate = server->dialect == SMB30_PROT_ID;
 
-		wdata->mr = smbd_register_mr(server->smbd_conn, &wdata->subreq.io_iter,
+		iov_iter_bvec_queue(&iter, ITER_SOURCE,
+				    wdata->subreq.content.bvecq, wdata->subreq.content.slot,
+				    wdata->subreq.content.offset, wdata->subreq.len);
+
+		wdata->mr = smbd_register_mr(server->smbd_conn, &iter,
 					     false, need_invalidate);
 		if (!wdata->mr) {
 			rc = -EAGAIN;
@@ -5226,8 +5238,8 @@ smb2_async_writev(struct cifs_io_subrequest *wdata)
 		smb2_set_replay(server, &rqst);
 	}
 
-	cifs_dbg(FYI, "async write at %llu %u bytes iter=%zx\n",
-		 io_parms->offset, io_parms->length, iov_iter_count(&wdata->subreq.io_iter));
+	cifs_dbg(FYI, "async write at %llu %u bytes len=%zx\n",
+		 io_parms->offset, io_parms->length, wdata->subreq.len);
 
 	if (wdata->credits.value > 0) {
 		shdr->CreditCharge = cpu_to_le16(DIV_ROUND_UP(wdata->subreq.len,
diff --git a/fs/smb/client/transport.c b/fs/smb/client/transport.c
index fdf4e50c27ce..be2f6b909c34 100644
--- a/fs/smb/client/transport.c
+++ b/fs/smb/client/transport.c
@@ -1267,12 +1267,19 @@ cifs_readv_receive(struct TCP_Server_Info *server, struct mid_q_entry *mid)
 	}
 
 #ifdef CONFIG_CIFS_SMB_DIRECT
-	if (rdata->mr)
+	if (rdata->mr) {
 		length = data_len; /* An RDMA read is already done. */
-	else
+	} else {
+#endif
+		struct iov_iter iter;
+
+		iov_iter_bvec_queue(&iter, ITER_DEST, rdata->subreq.content.bvecq,
+				    rdata->subreq.content.slot, rdata->subreq.content.offset,
+				    data_len);
+		length = cifs_read_iter_from_socket(server, &iter, data_len);
+#ifdef CONFIG_CIFS_SMB_DIRECT
+	}
 #endif
-		length = cifs_read_iter_from_socket(server, &rdata->subreq.io_iter,
-						    data_len);
 	if (length > 0)
 		rdata->got_bytes += length;
 	server->total_read += length;
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index f277c187c408..2d323f5f34e6 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -66,7 +66,7 @@ struct netfs_inode {
 #endif
 	struct list_head	wb_queue;	/* Queue of processes wanting to do writeback */
 	loff_t			_remote_i_size;	/* Size of the remote file */
-	loff_t			_zero_point;	/* Size after which we assume there's no data
+	unsigned long long	_zero_point;	/* Size after which we assume there's no data
 						 * on the server */
 	spinlock_t		lock;		/* Lock covering wb_queue */
 	atomic_t		io_count;	/* Number of outstanding reqs */
@@ -128,25 +128,41 @@ static inline struct netfs_group *netfs_folio_group(struct folio *folio)
 	return priv;
 }
 
+/*
+ * Estimate of maximum write subrequest for writeback.  The filesystem is
+ * responsible for filling this in when called from ->estimate_write(), though
+ * netfslib will preset infinite defaults.
+ */
+struct netfs_write_estimate {
+	uoff_t			issue_at;	/* Point at which we must submit */
+	int			max_segs;	/* Max number of segments in a single RPC */
+};
+
 /*
  * Stream of I/O subrequests going to a particular destination, such as the
  * server or the local cache.  This is mainly intended for writing where we may
  * have to write to multiple destinations concurrently.
  */
 struct netfs_io_stream {
-	/* Submission tracking */
-	struct netfs_io_subrequest *construct;	/* Op being constructed */
-	size_t			sreq_max_len;	/* Maximum size of a subrequest */
-	unsigned int		sreq_max_segs;	/* 0 or max number of segments in an iterator */
-	unsigned int		submit_off;	/* Folio offset we're submitting from */
-	unsigned int		submit_len;	/* Amount of data left to submit */
-	void (*prepare_write)(struct netfs_io_subrequest *subreq);
-	void (*issue_write)(struct netfs_io_subrequest *subreq);
+	/* Submission tracking (main dispatch only; not retry) */
+	struct bvecq_pos	dispatch_cursor; /* Point from which buffers are dispatched */
+	uoff_t			issue_from;	/* Current issue point */
+	uoff_t			last_end;	/* End file pos of last folio added */
+	size_t			buffered;	/* Amount in buffer */
+	unsigned int            alignment;      /* Required alignment */
+	u8			applicable;	/* What sources are applicable (NOTE_* mask) */
+	bool			buffering;	/* T if buffering on this stream */
+	int (*estimate_write)(struct netfs_io_request *wreq,
+			      struct netfs_io_stream *stream,
+			      struct netfs_write_estimate *estimate);
+	int (*issue_write)(struct netfs_io_subrequest *subreq);
+	atomic64_t		issued_to;	/* Point to which can be considered issued */
+
 	/* Collection tracking */
 	struct list_head	subrequests;	/* Contributory I/O operations */
 	unsigned long long	collected_to;	/* Position we've collected results to */
 	size_t			transferred;	/* The amount transferred from this stream */
-	unsigned short		error;		/* Aggregate error for the stream */
+	short			error;		/* Aggregate error for the stream */
 	enum netfs_io_source	source;		/* Where to read from/write to */
 	unsigned char		stream_nr;	/* Index of stream in parent table */
 	bool			avail;		/* T if stream is available */
@@ -184,14 +200,13 @@ struct netfs_io_subrequest {
 	struct list_head	rreq_link;	/* Link in rreq->subrequests */
 	struct bvecq_pos	dispatch_pos;	/* Bookmark in the combined queue of the start */
 	struct bvecq_pos	content;	/* The (copied) content of the subrequest */
-	struct iov_iter		io_iter;	/* Iterator for this subrequest */
 	unsigned long long	start;		/* Where to start the I/O */
 	size_t			len;		/* Size of the I/O */
 	size_t			transferred;	/* Amount of data transferred */
+	unsigned int		nr_segs;	/* Number of segments in content */
 	refcount_t		ref;
 	short			error;		/* 0 or error that occurred */
 	unsigned short		debug_index;	/* Index in list (for debugging output) */
-	unsigned int		nr_segs;	/* Number of segs in io_iter */
 	u8			retry_count;	/* The number of retries (0 on initial pass) */
 	enum netfs_io_source	source;		/* Where to read from/write to */
 	unsigned char		stream_nr;	/* I/O stream this belongs to */
@@ -199,7 +214,6 @@ struct netfs_io_subrequest {
 #define NETFS_SREQ_COPY_TO_CACHE	0	/* Set if should copy the data to the cache */
 #define NETFS_SREQ_CLEAR_TAIL		1	/* Set if the rest of the read should be cleared */
 #define NETFS_SREQ_MADE_PROGRESS	4	/* Set if we transferred at least some data */
-#define NETFS_SREQ_BOUNDARY		6	/* Set if ends on hard boundary (eg. ceph object) */
 #define NETFS_SREQ_HIT_EOF		7	/* Set if short due to EOF */
 #define NETFS_SREQ_IN_PROGRESS		8	/* Unlocked when the subrequest completes */
 #define NETFS_SREQ_NEED_RETRY		9	/* Set if the filesystem requests a retry */
@@ -246,12 +260,11 @@ struct netfs_io_request {
 	struct netfs_group	*group;		/* Writeback group being written back */
 	struct bvecq		*spare;		/* Advance allocation of bvecq */
 	struct bvecq_pos	load_cursor;	/* Point at which new folios are loaded in */
-	struct bvecq_pos	dispatch_cursor; /* Point from which buffers are dispatched */
 	struct bvecq_pos	collect_cursor;	/* Clear-up point of I/O buffer */
+	struct bvecq_pos	retry_cursor;	/* Point from which retries are dispatched */
 	wait_queue_head_t	waitq;		/* Processor waiter */
 	void			*netfs_priv;	/* Private data for the netfs */
 	void			*netfs_priv2;	/* Private data for the netfs */
-	unsigned long long	last_end;	/* End pos of last folio submitted */
 	unsigned long long	submitted;	/* Amount submitted for I/O so far */
 	unsigned long long	len;		/* Length of the request */
 	size_t			transferred;	/* Amount to be indicated as transferred */
@@ -259,11 +272,15 @@ struct netfs_io_request {
 	long			error;		/* 0 or error that occurred */
 	unsigned long long	i_size;		/* Size of the file */
 	unsigned long long	start;		/* Start position */
-	atomic64_t		issued_to;	/* Write issuer folio cursor */
 	unsigned long long	collected_to;	/* Point we've collected to */
 	unsigned long long	cache_coll_to;	/* Point the cache has collected to */
 	unsigned long long	cleaned_to;	/* Position we've cleaned folios to */
 	unsigned long long	abandon_to;	/* Position to abandon folios to */
+#ifdef CONFIG_NETFS_PGPRIV2
+	unsigned long long	last_end;	/* End of last folio added */
+#endif
+	unsigned long long	retry_start;	/* Position to retry from */
+	size_t			retry_buffered;	/* Amount of data to retry */
 	const struct folio	*no_unlock_folio; /* Don't unlock this folio after read */
 	gfp_t			gfp;		/* GFP flags to use */
 	unsigned int		debug_id;
@@ -282,6 +299,7 @@ struct netfs_io_request {
 #define NETFS_RREQ_RETRYING		4	/* Set if we're in the retry path */
 #define NETFS_RREQ_SHORT_TRANSFER	5	/* Set if we have a short transfer */
 #define NETFS_RREQ_ABANDON_REQ		6	/* Set if the request is to be abandoned */
+#define NETFS_RREQ_SAW_ENOMEM		7	/* Set if we encounted ENOMEM */
 #define NETFS_RREQ_CACHE_STOP		8	/* Set to stop caching (ENOBUFS or error) */
 #define NETFS_RREQ_CACHE_ERROR		9	/* Set if we got an error from the cache */
 #define NETFS_RREQ_OFFLOAD_COLLECTION	12	/* Offload collection to workqueue */
@@ -290,8 +308,10 @@ struct netfs_io_request {
 #define NETFS_RREQ_UPLOAD_TO_SERVER	15	/* Need to write to the server */
 #define NETFS_RREQ_USE_IO_ITER		16	/* Use ->io_iter rather than ->i_pages */
 #define NETFS_RREQ_NEED_PUT_RA_REFS	17	/* Need to put the folio refs RA gave us */
+#ifdef CONFIG_NETFS_PGPRIV2
 #define NETFS_RREQ_USE_PGPRIV2		31	/* [DEPRECATED] Use PG_private_2 to mark
 						 * write to cache on read */
+#endif
 	const struct netfs_request_ops *netfs_ops;
 };
 
@@ -307,8 +327,7 @@ struct netfs_request_ops {
 
 	/* Read request handling */
 	void (*expand_readahead)(struct netfs_io_request *rreq);
-	int (*prepare_read)(struct netfs_io_subrequest *subreq);
-	void (*issue_read)(struct netfs_io_subrequest *subreq);
+	int (*issue_read)(struct netfs_io_subrequest *subreq);
 	bool (*is_still_valid)(struct netfs_io_request *rreq);
 	int (*check_write_begin)(struct file *file, loff_t pos, unsigned len,
 				 struct folio **foliop, void **_fsdata);
@@ -320,8 +339,10 @@ struct netfs_request_ops {
 
 	/* Write request handling */
 	void (*begin_writeback)(struct netfs_io_request *wreq);
-	void (*prepare_write)(struct netfs_io_subrequest *subreq);
-	void (*issue_write)(struct netfs_io_subrequest *subreq);
+	int (*estimate_write)(struct netfs_io_request *wreq,
+			      struct netfs_io_stream *stream,
+			      struct netfs_write_estimate *estimate);
+	int (*issue_write)(struct netfs_io_subrequest *subreq);
 	void (*retry_request)(struct netfs_io_request *wreq, struct netfs_io_stream *stream);
 	void (*invalidate_cache)(struct netfs_io_request *wreq);
 };
@@ -362,8 +383,16 @@ struct netfs_cache_ops {
 		     netfs_io_terminated_t term_func,
 		     void *term_func_priv);
 
+	/* Estimate the amount of data that can be written in an op. */
+	int (*estimate_write)(struct netfs_io_request *wreq,
+			      struct netfs_io_stream *stream,
+			      struct netfs_write_estimate *estimate);
+
+	/* Read data from the cache for a netfs subrequest. */
+	int (*issue_read)(struct netfs_io_subrequest *subreq);
+
 	/* Write data to the cache from a netfs subrequest. */
-	void (*issue_write)(struct netfs_io_subrequest *subreq);
+	int (*issue_write)(struct netfs_io_subrequest *subreq);
 
 	/* Expand readahead request */
 	void (*expand_readahead)(struct netfs_cache_resources *cres,
@@ -371,30 +400,18 @@ struct netfs_cache_ops {
 				 unsigned long long *_len,
 				 unsigned long long i_size);
 
-	/* Prepare a read operation, shortening it to a cached/uncached
-	 * boundary as appropriate.
-	 */
-	int (*prepare_read)(struct netfs_io_subrequest *subreq);
-
-	/* Prepare a write subrequest, working out if we're allowed to do it
-	 * and finding out the maximum amount of data to gather before
-	 * attempting to submit.  If we're not permitted to do it, the
-	 * subrequest should be marked failed.
-	 */
-	void (*prepare_write_subreq)(struct netfs_io_subrequest *subreq);
-
 	/* Prepare a write operation, working out what part of the write we can
 	 * actually do.
 	 */
-	int (*prepare_write)(struct netfs_cache_resources *cres,
-			     loff_t *_start, size_t *_len, size_t upper_len,
-			     loff_t i_size, bool no_space_allocated_yet);
+	int (*prepare_write_old)(struct netfs_cache_resources *cres,
+				 loff_t *_start, size_t *_len, size_t upper_len,
+				 loff_t i_size, bool no_space_allocated_yet);
 
 	/* Query the occupancy of the cache in a region, returning where the
 	 * next chunk of data starts and how long it is.
 	 */
-	int (*query_occupancy)(struct netfs_cache_resources *cres,
-			       struct fscache_occupancy *occ);
+	void (*query_occupancy)(struct netfs_cache_resources *cres,
+				struct fscache_occupancy *occ);
 
 	/* Collect the result of buffered writeback to the cache.  This
 	 * includes copying a read to the cache.  block_type is one of:
@@ -428,10 +445,9 @@ void netfs_single_mark_inode_dirty(struct inode *inode);
 ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_iter *iter);
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
-			   struct iov_iter *iter);
+			   struct iov_iter *iter, size_t len);
 
 /* Address operations API */
-struct readahead_control;
 void netfs_readahead(struct readahead_control *);
 int netfs_read_folio(struct file *, struct folio *);
 int netfs_write_begin(struct netfs_inode *, struct file *,
@@ -458,7 +474,8 @@ void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
 ssize_t netfs_extract_iter(struct iov_iter *orig, size_t max_len, size_t max_pages,
 			   unsigned long long fpos, struct bvecq **_bvecq_head,
 			   iov_iter_extraction_t extraction_flags, gfp_t gfp);
-void netfs_prepare_write_failed(struct netfs_io_subrequest *subreq);
+int netfs_prepare_read_buffer(struct netfs_io_subrequest *subreq, unsigned int max_segs);
+int netfs_prepare_write_buffer(struct netfs_io_subrequest *subreq, unsigned int max_segs);
 void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error);
 
 int netfs_start_io_read(struct inode *inode);
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 723cb7315308..8b40fe870d21 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -47,7 +47,9 @@
 	E_(NETFS_PGPRIV2_COPY_TO_CACHE,		"2C")
 
 #define netfs_rreq_traces					\
+	EM(netfs_rreq_trace_all_queued,		"ALL-Q  ")	\
 	EM(netfs_rreq_trace_assess,		"ASSESS ")	\
+	EM(netfs_rreq_trace_cache_align_error,	"CA-ALN!")	\
 	EM(netfs_rreq_trace_cache_cancelled,	"CA-CNCL")	\
 	EM(netfs_rreq_trace_cache_failed,	"CA-FAIL")	\
 	EM(netfs_rreq_trace_cache_fail_collect,	"CA-F-CO")	\
@@ -84,7 +86,8 @@
 	EM(netfs_rreq_trace_waited_quiesce,	"DONE-QUIESCE")	\
 	EM(netfs_rreq_trace_wake_ip,		"WAKE-IP")	\
 	EM(netfs_rreq_trace_wake_queue,		"WAKE-Q ")	\
-	E_(netfs_rreq_trace_write_done,		"WR-DONE")
+	EM(netfs_rreq_trace_write_done,		"WR-DONE")	\
+	E_(netfs_rreq_trace_zero_unread,	"ZERO-UR")
 
 #define netfs_sreq_sources					\
 	EM(NETFS_SOURCE_UNKNOWN,		"----")		\
@@ -133,6 +136,7 @@
 	EM(netfs_sreq_trace_superfluous,	"SPRFL")	\
 	EM(netfs_sreq_trace_terminated,		"TERM ")	\
 	EM(netfs_sreq_trace_too_much,		"!TOOM")	\
+	EM(netfs_sreq_trace_too_many_retries,	"!RETR")	\
 	EM(netfs_sreq_trace_wait_for,		"_WAIT")	\
 	EM(netfs_sreq_trace_write,		"WRITE")	\
 	EM(netfs_sreq_trace_write_skip,		"SKIP ")	\
@@ -200,12 +204,12 @@
 	EM(netfs_folio_trace_alloc_buffer,	"alloc-buf")	\
 	EM(netfs_folio_trace_cancel_copy,	"cancel-copy")	\
 	EM(netfs_folio_trace_cancel_store,	"cancel-store")	\
-	EM(netfs_folio_trace_clear,		"clear")	\
-	EM(netfs_folio_trace_clear_cc,		"clear-cc")	\
-	EM(netfs_folio_trace_clear_g,		"clear-g")	\
-	EM(netfs_folio_trace_clear_s,		"clear-s")	\
 	EM(netfs_folio_trace_copy_to_cache,	"mark-copy")	\
 	EM(netfs_folio_trace_end_copy,		"end-copy")	\
+	EM(netfs_folio_trace_endwb,		"endwb")	\
+	EM(netfs_folio_trace_endwb_cc,		"endwb-cc")	\
+	EM(netfs_folio_trace_endwb_g,		"endwb-g")	\
+	EM(netfs_folio_trace_endwb_s,		"endwb-s")	\
 	EM(netfs_folio_trace_filled_gaps,	"filled-gaps")	\
 	EM(netfs_folio_trace_invalidate_all,	"inval-all")	\
 	EM(netfs_folio_trace_invalidate_front,	"inval-front")	\
@@ -396,10 +400,10 @@ TRACE_EVENT(netfs_sreq,
 		    __entry->len	= sreq->len;
 		    __entry->transferred = sreq->transferred;
 		    __entry->start	= sreq->start;
-		    __entry->slot	= sreq->dispatch_pos.slot;
+		    __entry->slot	= sreq->content.slot;
 			   ),
 
-	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx qs=%u e=%d",
+	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx bv=%u e=%d",
 		      __entry->rreq, __entry->index,
 		      __print_symbolic(__entry->source, netfs_sreq_sources),
 		      __print_symbolic(__entry->what, netfs_sreq_traces),
@@ -507,6 +511,7 @@ TRACE_EVENT(netfs_folio,
 	    TP_STRUCT__entry(
 		    __field(u64,			ino)
 		    __field(pgoff_t,			index)
+		    __field(unsigned long,		pfn)
 		    __field(unsigned int,		nr)
 		    __field(enum netfs_folio_trace,	why)
 			     ),
@@ -517,13 +522,40 @@ TRACE_EVENT(netfs_folio,
 		    __entry->why = why;
 		    __entry->index = folio->index;
 		    __entry->nr = folio_nr_pages(folio);
+		    __entry->pfn = folio_pfn(folio);
 			   ),
 
-	    TP_printk("i=%05llx ix=%05lx-%05lx %s",
+	    TP_printk("p=%lx i=%05llx ix=%05lx-%05lx %s",
+		      __entry->pfn,
 		      __entry->ino, __entry->index, __entry->index + __entry->nr - 1,
 		      __print_symbolic(__entry->why, netfs_folio_traces))
 	    );
 
+TRACE_EVENT(netfs_wback,
+	    TP_PROTO(struct netfs_io_request *wreq, struct folio *folio, unsigned int notes),
+
+	    TP_ARGS(wreq, folio, notes),
+
+	    TP_STRUCT__entry(
+		    __field(pgoff_t,			index)
+		    __field(unsigned int,		wreq)
+		    __field(unsigned int,		nr)
+		    __field(unsigned int,		notes)
+			     ),
+
+	    TP_fast_assign(
+		    __entry->wreq = wreq->debug_id;
+		    __entry->notes = notes;
+		    __entry->index = folio->index;
+		    __entry->nr = folio_nr_pages(folio);
+			   ),
+
+	    TP_printk("R=%08x ix=%05lx-%05lx n=%02x",
+		      __entry->wreq,
+		      __entry->index, __entry->index + __entry->nr - 1,
+		      __entry->notes)
+	    );
+
 TRACE_EVENT(netfs_write_iter,
 	    TP_PROTO(const struct kiocb *iocb, const struct iov_iter *from),
 
@@ -767,7 +799,7 @@ TRACE_EVENT(netfs_collect_stream,
 		    __entry->wreq	= wreq->debug_id;
 		    __entry->stream	= stream->stream_nr;
 		    __entry->collected_to = stream->collected_to;
-		    __entry->issued_to	= atomic64_read(&wreq->issued_to);
+		    __entry->issued_to	= atomic64_read(&stream->issued_to);
 			   ),
 
 	    TP_printk("R=%08x[%x:] cto=%llx ito=%llx",
@@ -791,7 +823,7 @@ TRACE_EVENT(netfs_bvecq,
 		    __entry->trace	= trace;
 			   ),
 
-	    TP_printk("fq=%x %s",
+	    TP_printk("bq=%x %s",
 		      __entry->id,
 		      __print_symbolic(__entry->trace, netfs_bvecq_traces))
 	    );
diff --git a/net/9p/client.c b/net/9p/client.c
index ef64546c6d52..68d82f677a7a 100644
--- a/net/9p/client.c
+++ b/net/9p/client.c
@@ -1564,6 +1564,7 @@ void
 p9_client_write_subreq(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *wreq = subreq->rreq;
+	struct iov_iter iter;
 	struct p9_fid *fid = wreq->netfs_priv;
 	struct p9_client *clnt = fid->clnt;
 	struct p9_req_t *req;
@@ -1574,14 +1575,17 @@ p9_client_write_subreq(struct netfs_io_subrequest *subreq)
 	p9_debug(P9_DEBUG_9P, ">>> TWRITE fid %d offset %llu len %d\n",
 		 fid->fid, start, len);
 
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, subreq->content.bvecq,
+			    subreq->content.slot, subreq->content.offset, subreq->len);
+
 	/* Don't bother zerocopy for small IO (< 1024) */
 	if (clnt->trans_mod->zc_request && len > 1024) {
-		req = p9_client_zc_rpc(clnt, P9_TWRITE, NULL, &subreq->io_iter,
+		req = p9_client_zc_rpc(clnt, P9_TWRITE, NULL, &iter,
 				       0, wreq->len, P9_ZC_HDR_SZ, "dqd",
 				       fid->fid, start, len);
 	} else {
 		req = p9_client_rpc(clnt, P9_TWRITE, "dqV", fid->fid,
-				    start, len, &subreq->io_iter);
+				    start, len, &iter);
 	}
 	if (IS_ERR(req)) {
 		netfs_write_subrequest_terminated(subreq, PTR_ERR(req));



From dhowells at redhat.com  Mon Aug 10 07:47:43 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 15:47:43 +0100
Subject: [PATCH v9 26/26] cachefiles: Preset the state xattr when creating a new file
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
Message-ID: <20260810144746.574036-27-dhowells@redhat.com>

With a really small cache, cachefiles is likely to see a lot of writes
hitting ENOSPC - and this can include setxattr that sets the state xattr on
a cachefile - but we don't really want to successfully fill a cache file
only to have to scrap it because we can't set the xattr.

Instead, preset the xattr when we create the tmpfile we're going to use,
and scrap the file at that point if we get ENOSPC.  Only if setxattr
succeeds do we allow data to be written to the file.

Note that there is a potential performance loss in that writes to the cache
have to be delayed until this is completed - but we do the tmpfile/setxattr
in parallel, starting when the file is opened and only have to wait once
writeback occurs.

Further, a new tracepoint, cachefiles_no_space, has been added to log ENOSPC
events, along with a couple of subreq traces to log failure detection in
netfslib.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/cachefiles/internal.h          |  1 +
 fs/cachefiles/io.c                | 23 ++++++++----
 fs/cachefiles/namei.c             | 15 ++++++--
 fs/cachefiles/xattr.c             | 60 ++++++++++++++++++++++++++++++-
 include/trace/events/cachefiles.h | 28 ++++++++++++++-
 include/trace/events/netfs.h      |  2 ++
 6 files changed, 119 insertions(+), 10 deletions(-)

diff --git a/fs/cachefiles/internal.h b/fs/cachefiles/internal.h
index de5e5ef01c8f..59a7599d3f2f 100644
--- a/fs/cachefiles/internal.h
+++ b/fs/cachefiles/internal.h
@@ -284,6 +284,7 @@ void cachefiles_withdraw_volume(struct cachefiles_volume *volume);
 /*
  * xattr.c
  */
+int cachefiles_preset_object_xattr(struct cachefiles_object *object, struct file *file);
 extern int cachefiles_set_object_xattr(struct cachefiles_object *object);
 extern int cachefiles_check_auxdata(struct cachefiles_object *object,
 				    struct file *file);
diff --git a/fs/cachefiles/io.c b/fs/cachefiles/io.c
index 24a93cc0aae8..f2e8fc25c94e 100644
--- a/fs/cachefiles/io.c
+++ b/fs/cachefiles/io.c
@@ -672,10 +672,14 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	 * space, we need to see if it's fully allocated.  If it's not, we may
 	 * want to cull it.
 	 */
-	if (cachefiles_has_space(cache, 0, *_len / cache->bsize,
-				 cachefiles_has_space_check) == 0)
+	ret = cachefiles_has_space(cache, 0, *_len / cache->bsize,
+				   cachefiles_has_space_check);
+	if (ret == 0)
 		return 0; /* Enough space to simply overwrite the whole block */
 
+	if (ret == -ENOBUFS)
+		trace_cachefiles_no_space(object, cachefiles_trace_write_nospace);
+
 	pos = cachefiles_inject_read_error();
 	if (pos == 0)
 		pos = vfs_llseek(file, start, SEEK_HOLE);
@@ -704,8 +708,11 @@ int __cachefiles_prepare_write(struct cachefiles_object *object,
 	return ret;
 
 check_space:
-	return cachefiles_has_space(cache, 0, *_len / cache->bsize,
-				    cachefiles_has_space_for_write);
+	ret = cachefiles_has_space(cache, 0, *_len / cache->bsize,
+				   cachefiles_has_space_for_write);
+	if (ret == -ENOBUFS)
+		trace_cachefiles_no_space(object, cachefiles_trace_write_nospace);
+	return ret;
 }
 
 static int cachefiles_prepare_write_old(struct netfs_cache_resources *cres,
@@ -757,10 +764,14 @@ static int cachefiles_issue_write(struct netfs_io_subrequest *subreq)
 	       wreq->debug_id, subreq->debug_index, start, start + len - 1);
 
 	if (!cachefiles_cres_file(cres)) {
-		if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE))
+		if (!fscache_wait_for_operation(cres, FSCACHE_WANT_WRITE)) {
+			trace_netfs_sreq(subreq, netfs_sreq_trace_cache_waitfail);
 			return -ENOBUFS;
-		if (!cachefiles_cres_file(cres))
+		}
+		if (!cachefiles_cres_file(cres)) {
+			trace_netfs_sreq(subreq, netfs_sreq_trace_cache_nofile);
 			return -ENOBUFS;
+		}
 	}
 
 	ret = netfs_prepare_write_buffer(subreq, BIO_MAX_VECS);
diff --git a/fs/cachefiles/namei.c b/fs/cachefiles/namei.c
index a30df0f91e5e..ef656a319ede 100644
--- a/fs/cachefiles/namei.c
+++ b/fs/cachefiles/namei.c
@@ -117,8 +117,11 @@ struct dentry *cachefiles_get_directory(struct cachefiles_cache *cache,
 	if (d_is_negative(subdir)) {
 		ret = cachefiles_has_space(cache, 1, 0,
 					   cachefiles_has_space_for_create);
-		if (ret < 0)
+		if (ret < 0) {
+			if (ret == -ENOBUFS)
+				trace_cachefiles_no_space(NULL, cachefiles_trace_mkdir_nospace);
 			goto mkdir_error;
+		}
 
 		_debug("attempt mkdir");
 
@@ -447,6 +450,11 @@ struct file *cachefiles_create_tmpfile(struct cachefiles_object *object)
 		pr_notice("Cache does not support read_iter and write_iter\n");
 		goto err_unuse;
 	}
+
+	/* Preallocate space for the xattr. */
+	ret = cachefiles_preset_object_xattr(object, file);
+	if (ret < 0)
+		goto err_unuse;
 out:
 	cachefiles_end_secure(cache, saved_cred);
 	object->content_info = CACHEFILES_CONTENT_ALL;
@@ -470,8 +478,11 @@ static bool cachefiles_create_file(struct cachefiles_object *object)
 
 	ret = cachefiles_has_space(object->volume->cache, 1, 0,
 				   cachefiles_has_space_for_create);
-	if (ret < 0)
+	if (ret < 0) {
+		if (ret == -ENOBUFS)
+			trace_cachefiles_no_space(object, cachefiles_trace_create_nospace);
 		return false;
+	}
 
 	file = cachefiles_create_tmpfile(object);
 	if (IS_ERR(file))
diff --git a/fs/cachefiles/xattr.c b/fs/cachefiles/xattr.c
index 25f2a906f984..2a7cc514f1a5 100644
--- a/fs/cachefiles/xattr.c
+++ b/fs/cachefiles/xattr.c
@@ -33,6 +33,58 @@ struct cachefiles_vol_xattr {
 	__u8	data[];		/* netfs volume coherency data */
 } __packed;
 
+/*
+ * Preset the state xattr on a cache file to allocate space for it.
+ */
+int cachefiles_preset_object_xattr(struct cachefiles_object *object, struct file *file)
+{
+	struct cachefiles_xattr *buf;
+	struct dentry *dentry = file->f_path.dentry;
+	unsigned int len = object->cookie->aux_len;
+	int ret;
+
+	buf = kzalloc(sizeof(struct cachefiles_xattr) + len, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	buf->type		= CACHEFILES_COOKIE_TYPE_DATA;
+	buf->content		= CACHEFILES_CONTENT_DIRTY;
+
+	ret = cachefiles_inject_write_error();
+	if (ret == 0) {
+		ret = mnt_want_write_file(file);
+		if (ret == 0) {
+			ret = vfs_setxattr(&nop_mnt_idmap, dentry,
+					   cachefiles_xattr_cache, buf,
+					   sizeof(struct cachefiles_xattr) + len, 0);
+			mnt_drop_write_file(file);
+		}
+	}
+	if (ret < 0) {
+		trace_cachefiles_vfs_error(object, file_inode(file), ret,
+					   cachefiles_trace_setxattr_error);
+		trace_cachefiles_coherency(object, file_inode(file)->i_ino,
+					   object->object_size,
+					   be64_to_cpup((__be64 *)buf->data),
+					   buf->content,
+					   cachefiles_coherency_set_fail);
+		switch (ret) {
+		case -ENOMEM:
+		case -ENOSPC:
+			break;
+		default:
+			cachefiles_io_error_obj(
+				object,
+				"Failed to set xattr with error %d", ret);
+			break;
+		}
+	}
+
+	kfree(buf);
+	_leave(" = %d", ret);
+	return ret;
+}
+
 /*
  * set the state xattr on a cache file
  */
@@ -81,10 +133,16 @@ int cachefiles_set_object_xattr(struct cachefiles_object *object)
 					   be64_to_cpup((__be64 *)buf->data),
 					   buf->content,
 					   cachefiles_coherency_set_fail);
-		if (ret != -ENOMEM)
+		switch (ret) {
+		case -ENOMEM:
+			break;
+		case -ENOSPC:
+		default:
 			cachefiles_io_error_obj(
 				object,
 				"Failed to set xattr with error %d", ret);
+			break;
+		}
 	} else {
 		trace_cachefiles_coherency(object, file_inode(file)->i_ino,
 					   object->object_size,
diff --git a/include/trace/events/cachefiles.h b/include/trace/events/cachefiles.h
index eccdb5abeedc..c7b5b8587cca 100644
--- a/include/trace/events/cachefiles.h
+++ b/include/trace/events/cachefiles.h
@@ -85,11 +85,13 @@ enum cachefiles_prepare_read_trace {
 
 enum cachefiles_error_trace {
 	cachefiles_trace_alignment_error,
+	cachefiles_trace_create_nospace,
 	cachefiles_trace_fallocate_error,
 	cachefiles_trace_getxattr_error,
 	cachefiles_trace_link_error,
 	cachefiles_trace_lookup_error,
 	cachefiles_trace_mkdir_error,
+	cachefiles_trace_mkdir_nospace,
 	cachefiles_trace_notify_change_error,
 	cachefiles_trace_open_error,
 	cachefiles_trace_read_error,
@@ -102,6 +104,7 @@ enum cachefiles_error_trace {
 	cachefiles_trace_trunc_error,
 	cachefiles_trace_unlink_error,
 	cachefiles_trace_write_error,
+	cachefiles_trace_write_nospace,
 };
 
 #endif
@@ -171,11 +174,13 @@ enum cachefiles_error_trace {
 
 #define cachefiles_error_traces						\
 	EM(cachefiles_trace_alignment_error,	"align")		\
+	EM(cachefiles_trace_create_nospace,	"create-nospace")	\
 	EM(cachefiles_trace_fallocate_error,	"fallocate")		\
 	EM(cachefiles_trace_getxattr_error,	"getxattr")		\
 	EM(cachefiles_trace_link_error,		"link")			\
 	EM(cachefiles_trace_lookup_error,	"lookup")		\
 	EM(cachefiles_trace_mkdir_error,	"mkdir")		\
+	EM(cachefiles_trace_mkdir_nospace,	"mkdir-nospace")	\
 	EM(cachefiles_trace_notify_change_error, "notify_change")	\
 	EM(cachefiles_trace_open_error,		"open")			\
 	EM(cachefiles_trace_read_error,		"read")			\
@@ -187,7 +192,8 @@ enum cachefiles_error_trace {
 	EM(cachefiles_trace_tmpfile_error,	"tmpfile")		\
 	EM(cachefiles_trace_trunc_error,	"trunc")		\
 	EM(cachefiles_trace_unlink_error,	"unlink")		\
-	E_(cachefiles_trace_write_error,	"write")
+	EM(cachefiles_trace_write_error,	"write")		\
+	E_(cachefiles_trace_write_nospace,	"write-nospace")
 
 
 /*
@@ -692,6 +698,26 @@ TRACE_EVENT(cachefiles_io_error,
 		      __entry->error)
 	    );
 
+TRACE_EVENT(cachefiles_no_space,
+	    TP_PROTO(struct cachefiles_object *obj, enum cachefiles_error_trace trace),
+
+	    TP_ARGS(obj, trace),
+
+	    TP_STRUCT__entry(
+		    __field(unsigned int,			obj)
+		    __field(enum cachefiles_error_trace,	trace)
+			     ),
+
+	    TP_fast_assign(
+		    __entry->obj	= obj ? obj->debug_id : 0;
+		    __entry->trace	= trace;
+			   ),
+
+	    TP_printk("o=%08x %s",
+		      __entry->obj,
+		      __print_symbolic(__entry->trace, cachefiles_error_traces))
+	    );
+
 #endif /* _TRACE_CACHEFILES_H */
 
 /* This part must be outside protection */
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 8b40fe870d21..e30358f99906 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -102,8 +102,10 @@
 	EM(netfs_sreq_trace_abandoned,		"ABNDN")	\
 	EM(netfs_sreq_trace_add_donations,	"+DON ")	\
 	EM(netfs_sreq_trace_added,		"ADD  ")	\
+	EM(netfs_sreq_trace_cache_nofile,	"CA-!F")	\
 	EM(netfs_sreq_trace_cache_nowrite,	"CA-NW")	\
 	EM(netfs_sreq_trace_cache_prepare,	"CA-PR")	\
+	EM(netfs_sreq_trace_cache_waitfail,	"CA-!W")	\
 	EM(netfs_sreq_trace_cache_write,	"CA-WR")	\
 	EM(netfs_sreq_trace_cancel,		"CANCL")	\
 	EM(netfs_sreq_trace_clear,		"CLEAR")	\



From willy at infradead.org  Mon Aug 10 12:36:11 2026
From: willy at infradead.org (Matthew Wilcox)
Date: Mon, 10 Aug 2026 20:36:11 +0100
Subject: [PATCH v9 04/26] Add a function to kmap one page of a multipage
 bio_vec
In-Reply-To: <20260810144746.574036-5-dhowells@redhat.com>
References: <20260810144746.574036-1-dhowells@redhat.com>
 <20260810144746.574036-5-dhowells@redhat.com>
Message-ID: <anooK9-UCNTmoSAq@casper.infradead.org>

On Mon, Aug 10, 2026 at 03:47:21PM +0100, David Howells wrote:
> Add a function to kmap one page of a multipage bio_vec by offset (which is
> added to the offset in the bio_vec internally).  The caller is responsible
> for calculating how much of the page is then available.

I think this should be kmap_local_bvec().  Living in bvec.h makes sense,
but keeping it in the kmap_local_ namespace is better than putting it in
the bvec_ namespace.

The bvec should be const-qualified.  We're not very good at
being const-clean, but let's try harder ;-)

I think it's better to steal the same terminology for the kernel-doc
used in highmem.h, just for consistency:

/**
 * kmap_local_bvec - Map a page in this bvec for temporary usage
 * @bvec: The bvec containing the page.
 * @offset: The first byte within the bvec to map.
 *
 * This will not map the entire bvec, only the page containing @offset
 * from the start of @bvec.  Unmap the address with kunmap_local().
 * See kmap_local_page() for nesting rules.
 *
 * Context: Interrupt or process context.
 * Return: A pointer to a temporary mapping.
 */

> +/**
> + * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
> + * @bvec: bvec to map
> + * @offset: Offset into bvec
> + *
> + * Map the page containing the byte at @offset into the kernel virtual address
> + * space.  The caller is responsible for making sure this doesn't overrun.
> + *
> + * Call kunmap_local on the returned address to unmap.
> + */
> +static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
> +{
> +	offset += bvec->bv_offset;
> +
> +	return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
> +		(offset & ~PAGE_MASK);
> +}
> +
>  #endif /* __LINUX_BVEC_H */
> 
> 


From dhowells at redhat.com  Mon Aug 10 13:31:03 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 10 Aug 2026 21:31:03 +0100
Subject: [PATCH v9 04/26] Add a function to kmap one page of a multipage bio_vec
In-Reply-To: <anooK9-UCNTmoSAq@casper.infradead.org>
References: <anooK9-UCNTmoSAq@casper.infradead.org> <20260810144746.574036-1-dhowells@redhat.com> <20260810144746.574036-5-dhowells@redhat.com>
Message-ID: <586048.1786393863@warthog.procyon.org.uk>

Matthew Wilcox <willy at infradead.org> wrote:

> On Mon, Aug 10, 2026 at 03:47:21PM +0100, David Howells wrote:
> > Add a function to kmap one page of a multipage bio_vec by offset (which is
> > added to the offset in the bio_vec internally).  The caller is responsible
> > for calculating how much of the page is then available.
> 
> I think this should be kmap_local_bvec().  Living in bvec.h makes sense,
> but keeping it in the kmap_local_ namespace is better than putting it in
> the bvec_ namespace.

Christoph disagrees with you:-/

David



From brauner at kernel.org  Tue Aug 11 00:26:58 2026
From: brauner at kernel.org (Christian Brauner)
Date: Tue, 11 Aug 2026 09:26:58 +0200
Subject: [PATCH v9 00/26] netfs: Keep track of folios in a segmented
 bio_vec[] chain
In-Reply-To: <annlWwsCPjNgcKa-@infradead.org>
References: <20260810144746.574036-1-dhowells@redhat.com>
 <annlWwsCPjNgcKa-@infradead.org>
Message-ID: <20260811-wegkommen-unfehlbar-selektiert-547aaf3c775b@brauner>

> this is a bad idea.  This is a giant series that also touches core code
> with no review whatsoever.  And I repeatedly asked for:

Since it's post the cut off point the series does not make the coming
merge window. It's also not in -next on purpose. The series was just
pulled for testing.

> 
>  1) the core changes to be split out
>  2) come up with users for this very core primitive in the hot path
>     (iov_iter) that are not just for your rather niche use case
> 
> There's been absolutely no follow on to this, and we should not treat
> the absolute core I/O path like this.



From linux at jaseg.de  Tue Aug 11 10:52:49 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:49 +0200
Subject: [PATCH v2 00/13] CRASH_WIPE_SECRETS: Wipe secrets before kdump
 (was: CRASH_ZEROIZE)
Message-ID: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>

I'm using linux on an embedded target in a Hardware Security Module-like
application. One requirement is that I want the system to be able to
quickly erase its memory when it detects physical tampering. I'm
approaching that by using kdump to load into a small payload that
instead of dumping RAM, erases RAM from start to end. However, writing
all of RAM, especially on an embedded target, is rather slow. For this
reason, I propose the mechanism in this patch series:

Add CONFIG_CRASH_ZEROIZE (default off), which when enabled makes various
subsystems handling secret data do a quick, targeted wipe of these
secrets before kdump. This behavior might also be interesting in cases
where you run a normal kdump kernel but you still want to keep things
like fde crypto keys out of these dumps.

CONFIG_CRASH_ZEROIZE is a best effort, defense in depth solution. There
are circumstances, such as when a panic is triggered after memory
corruption, or when a panic interrupts some operation that mutates data
structures under locks, when the kernel cannot safely wipe some memory
areas. The handlers proposed in this series will just print a warning
and skip the affected areas in this case.

This series introduces handlers for the major locations I found where
having this sort of thing makes sense. Notable omissions right now are
the Ceph and CIFS subsystems. I have WIP patches for these, but since I
can't easily test them right now, I omitted them from this patch set for
now. Currently included locations are:

* various key types in security/keys
* rxrpc
* fscrypt
* dm-crypt
* crypto tfm instances
* secretmem (which I'm going to start using in my application)

I've verified this patch series on an ARM64 target using the helper code
at https://codeberg.org/yasec/crash-wipe-test . This code stuffs the
affected kernel subsystems with keys and secret data, then crashes the
system, takes a RAM dump and verifies the dump is clean of secrets. Note
that the helper code is partially LLM-generated, so read with care. It
passes a positive control test with the config option disabled.

The patch series applies on top of linux-next but should work on 7.0.0,
too. I've tested the patches on a Arduino uno Q (Qualcomm QRB2210,
ARM64) embedded target.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
Changes in v2:
- New keyring key types
- New handlers: rxrpc, fscrypt, dm-crypt and crypto tfm
- Renamed from "zeroize" to "wipe"
- Add ARM64-specific cache flush logic
- Link to v1: https://patch.msgid.link/20260731162739.158320-1-linux at jaseg.de

To: Andrew Morton <akpm at linux-foundation.org>
To: Baoquan He <baoquan.he at linux.dev>
To: Mike Rapoport <rppt at kernel.org>
To: Pasha Tatashin <pasha.tatashin at soleen.com>
To: Pratyush Yadav <pratyush at kernel.org>
To: Dave Young <ruirui.yang at linux.dev>
To: Catalin Marinas <catalin.marinas at arm.com>
To: Will Deacon <will at kernel.org>
To: David Howells <dhowells at redhat.com>
To: Jarkko Sakkinen <jarkko at kernel.org>
To: Jonathan Corbet <corbet at lwn.net>
To: Shuah Khan <skhan at linuxfoundation.org>
To: Paul Moore <paul at paul-moore.com>
To: James Morris <jmorris at namei.org>
To: "Serge E. Hallyn" <serge at hallyn.com>
To: Lukas Wunner <lukas at wunner.de>
To: Ignat Korchagin <ignat at linux.win>
To: Herbert Xu <herbert at gondor.apana.org.au>
To: "David S. Miller" <davem at davemloft.net>
To: Keith Busch <kbusch at kernel.org>
To: Jens Axboe <axboe at kernel.dk>
To: Christoph Hellwig <hch at lst.de>
To: Sagi Grimberg <sagi at grimberg.me>
To: Trond Myklebust <trondmy at kernel.org>
To: Anna Schumaker <anna at kernel.org>
To: Mimi Zohar <zohar at linux.ibm.com>
To: James Bottomley <James.Bottomley at HansenPartnership.com>
To: Marc Dionne <marc.dionne at auristor.com>
To: Eric Dumazet <edumazet at google.com>
To: Jakub Kicinski <kuba at kernel.org>
To: Paolo Abeni <pabeni at redhat.com>
To: Simon Horman <horms at kernel.org>
To: Eric Biggers <ebiggers at kernel.org>
To: "Theodore Y. Ts'o" <tytso at mit.edu>
To: Jaegeuk Kim <jaegeuk at kernel.org>
To: Alexander Viro <viro at zeniv.linux.org.uk>
To: Christian Brauner <brauner at kernel.org>
To: Jan Kara <jack at suse.cz>
To: Alasdair Kergon <agk at redhat.com>
To: Mike Snitzer <snitzer at kernel.org>
To: Mikulas Patocka <mpatocka at redhat.com>
To: Benjamin Marzinski <bmarzins at redhat.com>
Cc: kexec at lists.infradead.org
Cc: linux-kernel at vger.kernel.org
Cc: linux-arm-kernel at lists.infradead.org
Cc: linux-mm at kvack.org
Cc: keyrings at vger.kernel.org
Cc: linux-doc at vger.kernel.org
Cc: linux-security-module at vger.kernel.org
Cc: linux-crypto at vger.kernel.org
Cc: linux-nvme at lists.infradead.org
Cc: linux-nfs at vger.kernel.org
Cc: linux-integrity at vger.kernel.org
Cc: linux-afs at lists.infradead.org
Cc: netdev at vger.kernel.org
Cc: linux-fscrypt at vger.kernel.org
Cc: linux-fsdevel at vger.kernel.org
Cc: dm-devel at lists.linux.dev

---
Jan Sebastian G?tte (13):
      kexec: add CRASH_WIPE_SECRETS to wipe secrets before kdump
      crash-core: Flush caches on CRASH_WIPE_SECRETS
      arm64/mm: add set_direct_map_default_nosplit()
      mm/secretmem: wipe secret pages before kdump
      security/keys: wipe key payloads before kdump
      security/keys: implement wipe op for user-type keys
      security/keys: implement wipe op for big_key
      security/keys: implement wipe op for trusted and encrypted keys
      security/keys: implement wipe op for asymmetric keys
      rxrpc: implement wipe op for rxrpc keys
      fscrypt: wipe master keys before kdump
      crypto: api - wipe tfm contexts before kdump
      dm crypt: wipe key material before kdump

 Documentation/security/keys/core.rst      |  13 +++
 arch/arm64/include/asm/set_memory.h       |   2 +
 arch/arm64/kernel/machine_kexec.c         |  20 ++++
 arch/arm64/mm/pageattr.c                  |  21 +++++
 crypto/api.c                              |  78 +++++++++++++--
 crypto/asymmetric_keys/asymmetric_type.c  |  11 +++
 crypto/asymmetric_keys/pkcs7_key_type.c   |   1 +
 crypto/asymmetric_keys/public_key.c       |  15 +++
 drivers/md/dm-crypt.c                     | 151 ++++++++++++++++++++++++++----
 drivers/nvme/common/keyring.c             |   1 +
 fs/crypto/block.c                         |  10 ++
 fs/crypto/fscrypt_private.h               |  15 +++
 fs/crypto/keyring.c                       |  68 +++++++++++++-
 fs/crypto/keysetup_v1.c                   |  15 +++
 fs/nfs/nfs4idmap.c                        |   2 +
 fs/super.c                                |  29 ++++++
 include/keys/asymmetric-subtype.h         |   5 +
 include/keys/user-type.h                  |   1 +
 include/linux/crash_core.h                |  37 ++++++++
 include/linux/crypto.h                    |  11 +++
 include/linux/fs.h                        |   4 +
 include/linux/key-type.h                  |   9 ++
 include/linux/set_memory.h                |  17 +++-
 kernel/Kconfig.kexec                      |  16 ++++
 kernel/crash_core.c                       |  52 ++++++++++
 mm/secretmem.c                            |  53 +++++++++++
 net/rxrpc/ar-internal.h                   |   5 +
 net/rxrpc/key.c                           |  34 +++++++
 net/rxrpc/rxgk.c                          |  11 +++
 net/rxrpc/rxkad.c                         |  14 +++
 net/rxrpc/server_key.c                    |  11 +++
 security/keys/big_key.c                   |  17 ++++
 security/keys/encrypted-keys/encrypted.c  |  13 +++
 security/keys/key.c                       |  43 +++++++++
 security/keys/trusted-keys/trusted_core.c |  15 +++
 security/keys/user_defined.c              |  15 +++
 36 files changed, 808 insertions(+), 27 deletions(-)
---
base-commit: a0dbb7d3457bdebe2dbe198bd08bf9690be5f1ec
change-id: 20260811-crash-zeroize-rework-bb1a5d917577

Best regards,
--  
Jan Sebastian G?tte <linux at jaseg.de>



From linux at jaseg.de  Tue Aug 11 10:52:50 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:50 +0200
Subject: [PATCH v2 01/13] kexec: add CRASH_WIPE_SECRETS to wipe secrets
 before kdump
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-1-9561d13c2340@jaseg.de>

When kdump is used to capture system memory after a panic(), any secret
keys currently in RAM end up in the dump. Add an opt-in atomic notifier
chain, crash_wipe_secrets_notifier_list, invoked late into
__crash_kexec(). Subsystems holding secrets can register a callback to
wipe them.

Callbacks are run after machine_crash_shutdown() has already stopped the
other CPUs and disabled preemption. Callbacks must not wait on locks,
which will never be released.

This is a best-effort, defence-in-depth measure, not a guarantee.
Secrets in flight on the stack, in registers, in DMA buffers, or in
other places in memory are out of scope.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 include/linux/crash_core.h | 16 ++++++++++++++++
 kernel/Kconfig.kexec       | 16 ++++++++++++++++
 kernel/crash_core.c        | 29 +++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index bc087124cd78..4230463f3faa 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -5,6 +5,7 @@
 #include <linux/linkage.h>
 #include <linux/elfcore.h>
 #include <linux/elf.h>
+#include <linux/notifier.h>
 
 struct kimage;
 
@@ -34,6 +35,21 @@ static inline void arch_kexec_protect_crashkres(void) { }
 static inline void arch_kexec_unprotect_crashkres(void) { }
 #endif
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+int crash_wipe_secrets_register(struct notifier_block *nb);
+int crash_wipe_secrets_unregister(struct notifier_block *nb);
+#else
+static inline int crash_wipe_secrets_register(struct notifier_block *nb)
+{
+	return 0;
+}
+
+static inline int crash_wipe_secrets_unregister(struct notifier_block *nb)
+{
+	return 0;
+}
+#endif
+
 #ifndef arch_crash_handle_hotplug_event
 static inline void arch_crash_handle_hotplug_event(struct kimage *image, void *arg) { }
 #endif
diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
index 15632358bcf7..1d2d273145df 100644
--- a/kernel/Kconfig.kexec
+++ b/kernel/Kconfig.kexec
@@ -179,4 +179,20 @@ config CRASH_MAX_MEMORY_RANGES
 	  the computation behind the value provided through the
 	  /sys/kernel/crash_elfcorehdr_size attribute.
 
+config CRASH_WIPE_SECRETS
+	bool "Wipe secrets before kdump"
+	depends on CRASH_DUMP
+	help
+	  Wipe secrets (e.g. kernel keyring and memfd_secret pages) on crash or
+	  panic. This is a best effort, defense-in-depth feature: If the panic
+	  happens at a really bad time, or if copies of the secrets are present
+	  in places like on the stack, in I/O buffers, or in userspace memory
+	  not allocated through memfd_secret, they may still be leaked.
+
+	  Note that enabling this feature carries some risk of crashing the
+	  system during the wipe process if the kernel was already unstable
+	  when the panic happened.
+
+	  If unsure, say N.
+
 endmenu
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 2b36aa9fade0..95f5c0415e60 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -23,6 +23,7 @@
 #include <linux/objtool.h>
 #include <linux/delay.h>
 #include <linux/panic.h>
+#include <linux/timekeeping.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -33,6 +34,33 @@
 /* Per cpu memory for storing cpu states in case of system crash. */
 note_buf_t __percpu *crash_notes;
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+ATOMIC_NOTIFIER_HEAD(crash_wipe_secrets_notifier_list);
+
+int crash_wipe_secrets_register(struct notifier_block *nb)
+{
+	return atomic_notifier_chain_register(
+			&crash_wipe_secrets_notifier_list, nb);
+}
+EXPORT_SYMBOL_GPL(crash_wipe_secrets_register);
+
+int crash_wipe_secrets_unregister(struct notifier_block *nb)
+{
+	return atomic_notifier_chain_unregister(
+			&crash_wipe_secrets_notifier_list, nb);
+}
+EXPORT_SYMBOL_GPL(crash_wipe_secrets_unregister);
+
+static void crash_wipe_secrets(void)
+{
+	pr_info("Wiping sensitive secrets...\n");
+	atomic_notifier_call_chain(&crash_wipe_secrets_notifier_list, 0, NULL);
+	pr_info("Done wiping secrets.\n");
+}
+#else
+static inline void crash_wipe_secrets(void) { }
+#endif /* CONFIG_CRASH_WIPE_SECRETS */
+
 /* time to wait for possible DMA to finish before starting the kdump kernel
  * when a CMA reservation is used
  */
@@ -142,6 +170,7 @@ void __noclone __crash_kexec(struct pt_regs *regs)
 			crash_save_vmcoreinfo();
 			machine_crash_shutdown(&fixed_regs);
 			crash_cma_clear_pending_dma();
+			crash_wipe_secrets();
 			machine_kexec(kexec_crash_image);
 		}
 		kexec_unlock();

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:51 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:51 +0200
Subject: [PATCH v2 02/13] crash-core: Flush caches on CRASH_WIPE_SECRETS
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-2-9561d13c2340@jaseg.de>

Replace memzero_expclit with our custom wrapper that additionally
flushes the target address ranges from CPU caches.

On ARM64, while memzero_explicit is already reasonably reliable for
wiping secrets from memory during kdump, it can theoretically leave
residue in DRAM when the last memzero_explicit writes are still in
flight in caches when shutting down the caches in machine_kexec.

We need to swap every memzero_explicit call because ARM64 do not have a
wholesale "flush all caches" primitive and only support flushes targeted
to a particular address range.

Architectures other than ARM64, notably x86-64, are not affected by this
cache flush issue.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 arch/arm64/kernel/machine_kexec.c | 20 ++++++++++++++++++++
 include/linux/crash_core.h        | 21 +++++++++++++++++++++
 kernel/crash_core.c               | 23 +++++++++++++++++++++++
 3 files changed, 64 insertions(+)

diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index c5693a32e49b..12bf3c90f519 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -6,6 +6,7 @@
  * Copyright (C) Huawei Futurewei Technologies.
  */
 
+#include <linux/crash_core.h>
 #include <linux/interrupt.h>
 #include <linux/irq.h>
 #include <linux/kernel.h>
@@ -15,6 +16,7 @@
 #include <linux/set_memory.h>
 #include <linux/smp.h>
 
+#include <asm/barrier.h>
 #include <asm/cacheflush.h>
 #include <asm/cpu_ops.h>
 #include <asm/daifflags.h>
@@ -221,6 +223,24 @@ void machine_crash_shutdown(struct pt_regs *regs)
 	pr_info("Starting crashdump kernel...\n");
 }
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+/*
+ * Queue each wiped range for cleaning to the point of coherency.  Deferring
+ * the barrier until the notifier chain is complete avoids one DSB per range.
+ */
+void arch_crash_wipe_range(void *addr, size_t size)
+{
+	unsigned long start = (unsigned long)addr;
+
+	dcache_clean_poc_nosync(start, start + size);
+}
+
+void arch_crash_wipe_flush(void)
+{
+	dsb(sy);
+}
+#endif
+
 #if defined(CONFIG_CRASH_DUMP) && defined(CONFIG_HIBERNATION)
 /*
  * To preserve the crash dump kernel image, the relevant memory segments
diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 4230463f3faa..d3e4192b7e6d 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -6,6 +6,7 @@
 #include <linux/elfcore.h>
 #include <linux/elf.h>
 #include <linux/notifier.h>
+#include <linux/string.h>
 
 struct kimage;
 
@@ -15,6 +16,26 @@ struct crash_mem {
 	struct range ranges[] __counted_by(max_nr_ranges);
 };
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+/*
+ * Record a range that has already been wiped, or wipe and record it in one
+ * operation.  Architectures may use the ranges to push the wipes out to
+ * memory before kexec disables the caches.
+ */
+void crash_wipe_cache_range(void *addr, size_t size);
+void crash_wipe_memzero(void *addr, size_t size);
+
+void arch_crash_wipe_range(void *addr, size_t size);
+void arch_crash_wipe_flush(void);
+#else
+static inline void crash_wipe_cache_range(void *addr, size_t size) { }
+
+static inline void crash_wipe_memzero(void *addr, size_t size)
+{
+	memzero_explicit(addr, size);
+}
+#endif
+
 #ifdef CONFIG_CRASH_DUMP
 
 int crash_shrink_memory(unsigned long new_size);
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 95f5c0415e60..42faf8d0a4c4 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -51,10 +51,33 @@ int crash_wipe_secrets_unregister(struct notifier_block *nb)
 }
 EXPORT_SYMBOL_GPL(crash_wipe_secrets_unregister);
 
+/*
+ * Some kexec paths disable the data cache without first cleaning it.  Give
+ * architectures valid virtual ranges for the wiped data, then let them defer
+ * any completion barrier until all crash-wipe callbacks have run.
+ */
+void __weak arch_crash_wipe_range(void *addr, size_t size) { }
+void __weak arch_crash_wipe_flush(void) { }
+
+void crash_wipe_cache_range(void *addr, size_t size)
+{
+	if (size)
+		arch_crash_wipe_range(addr, size);
+}
+EXPORT_SYMBOL_GPL(crash_wipe_cache_range);
+
+void crash_wipe_memzero(void *addr, size_t size)
+{
+	memzero_explicit(addr, size);
+	crash_wipe_cache_range(addr, size);
+}
+EXPORT_SYMBOL_GPL(crash_wipe_memzero);
+
 static void crash_wipe_secrets(void)
 {
 	pr_info("Wiping sensitive secrets...\n");
 	atomic_notifier_call_chain(&crash_wipe_secrets_notifier_list, 0, NULL);
+	arch_crash_wipe_flush();
 	pr_info("Done wiping secrets.\n");
 }
 #else

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:52 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:52 +0200
Subject: [PATCH v2 03/13] arm64/mm: add set_direct_map_default_nosplit()
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-3-9561d13c2340@jaseg.de>

set_direct_map_default_noflush() goes through split_kernel_leaf_mapping(),
which takes pgtable_split_lock and may sleep. That makes it unusable from
atomic contexts such as the panic path.

Add set_direct_map_default_nosplit(), which skips the split and only walks
the page tables to apply the permission change. Callers must guarantee the
page is already mapped at page granularity. This holds for any page that
had set_direct_map_invalid_noflush() applied to it, as that call performed
the split.

Architectures that do not need to split the linear map get a generic
fallback to set_direct_map_default_noflush().

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 arch/arm64/include/asm/set_memory.h |  2 ++
 arch/arm64/mm/pageattr.c            | 21 +++++++++++++++++++++
 include/linux/set_memory.h          | 17 ++++++++++++++++-
 3 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h
index 90f61b17275e..cb5f44bb5a94 100644
--- a/arch/arm64/include/asm/set_memory.h
+++ b/arch/arm64/include/asm/set_memory.h
@@ -13,6 +13,8 @@ int set_memory_valid(unsigned long addr, int numpages, int enable);
 
 int set_direct_map_invalid_noflush(struct page *page);
 int set_direct_map_default_noflush(struct page *page);
+int set_direct_map_default_nosplit(struct page *page);
+#define set_direct_map_default_nosplit set_direct_map_default_nosplit
 int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid);
 bool kernel_page_present(struct page *page);
 
diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c
index bbe98ac9ad8c..a7bb6bfa12ed 100644
--- a/arch/arm64/mm/pageattr.c
+++ b/arch/arm64/mm/pageattr.c
@@ -275,6 +275,27 @@ int set_direct_map_default_noflush(struct page *page)
 				 PAGE_SIZE, set_mask, clear_mask);
 }
 
+/* Caller must guarantee @page is already mapped at page granularity. */
+int set_direct_map_default_nosplit(struct page *page)
+{
+	struct page_change_data data = {
+		.set_mask = __pgprot(PTE_PRESENT_VALID_KERNEL | PTE_WRITE),
+		.clear_mask = __pgprot(PTE_PRESENT_INVALID | PTE_RDONLY),
+	};
+	unsigned long addr = (unsigned long)page_address(page);
+	int ret;
+
+	if (!can_set_direct_map())
+		return 0;
+
+	lazy_mmu_mode_enable();
+	ret = walk_kernel_page_table_range_lockless(addr, addr + PAGE_SIZE,
+						    &pageattr_ops, NULL, &data);
+	lazy_mmu_mode_disable();
+
+	return ret;
+}
+
 static int __set_memory_enc_dec(unsigned long addr,
 				int numpages,
 				bool encrypt)
diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h
index 3030d9245f5a..65d0fa010368 100644
--- a/include/linux/set_memory.h
+++ b/include/linux/set_memory.h
@@ -44,7 +44,22 @@ static inline bool kernel_page_present(struct page *page)
 {
 	return true;
 }
-#else /* CONFIG_ARCH_HAS_SET_DIRECT_MAP */
+#endif /* !CONFIG_ARCH_HAS_SET_DIRECT_MAP */
+
+/*
+ * Variant of set_direct_map_default_noflush() that is safe to call from atomic
+ * context, for pages the caller knows are already mapped at page granularity.
+ * Architectures that need to split the linear map, and can therefore sleep,
+ * override this.
+ */
+#ifndef set_direct_map_default_nosplit
+static inline int set_direct_map_default_nosplit(struct page *page)
+{
+	return set_direct_map_default_noflush(page);
+}
+#endif
+
+#ifdef CONFIG_ARCH_HAS_SET_DIRECT_MAP
 /*
  * Some architectures, e.g. ARM64 can disable direct map modifications at
  * boot time. Let them overrive this query.

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:53 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:53 +0200
Subject: [PATCH v2 04/13] mm/secretmem: wipe secret pages before kdump
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-4-9561d13c2340@jaseg.de>

Register a CRASH_WIPE_SECRETS notifier that wipes secretmem folios. As a
result, when CONFIG_CRASH_WIPE_SECRETS is set, secretmem areas will be
cleared before the kdump kernel is kexec'ed.

The notifier runs after the other CPUs have been stopped, so the page
cache cannot be mutated concurrently and the xarray may be walked
without taking the i_pages lock. This is a best effort, defense in depth
measure. s_inode_list_lock is taken with trylock only. If a CPU was
stopped mid-modification the list may be inconsistent, and this late
into the panic path, there's nothing we can do about it.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 mm/secretmem.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/mm/secretmem.c b/mm/secretmem.c
index d29865075b6e..db00b7d06080 100644
--- a/mm/secretmem.c
+++ b/mm/secretmem.c
@@ -13,9 +13,11 @@
 #include <linux/bitops.h>
 #include <linux/printk.h>
 #include <linux/pagemap.h>
+#include <linux/notifier.h>
 #include <linux/syscalls.h>
 #include <linux/pseudo_fs.h>
 #include <linux/secretmem.h>
+#include <linux/crash_core.h>
 #include <linux/set_memory.h>
 #include <linux/sched/signal.h>
 
@@ -187,6 +189,54 @@ static const struct inode_operations secretmem_iops = {
 
 static struct vfsmount *secretmem_mnt;
 
+/* Called far into vpanic from crash_core.c with other CPUs stopped and
+ * preemption disabled
+ */
+static int secretmem_crash_wipe(struct notifier_block *nb, unsigned long action,
+		void *data)
+{
+	struct super_block *sb;
+	struct inode *inode;
+
+	if (!secretmem_mnt)
+		return NOTIFY_DONE;
+	sb = secretmem_mnt->mnt_sb;
+
+	/* If the list was modified in the exact moment we panic'ed, it might be
+	 * in an inconsistent state that would be unsafe to iterate. If we can't
+	 * get the lock, too bad, that's all we can do here.
+	 */
+	if (!spin_trylock(&sb->s_inode_list_lock)) {
+		pr_crit("crash_wipe_secrets: can't acquire secretmem superblock lock.\n"
+			 "crash_wipe_secrets: skipping zeroizing secretmem.\n");
+		return NOTIFY_DONE;
+	}
+
+	list_for_each_entry(inode, &sb->s_inodes, i_sb_list) {
+		XA_STATE(xas, &inode->i_mapping->i_pages, 0);
+		struct folio *folio;
+
+		/* no need for locks if we're burning down the house :) */
+		xas_for_each(&xas, folio, ULONG_MAX) {
+			if (xas_retry(&xas, folio) || xa_is_value(folio))
+				continue;
+
+			/* secretmem_fault() already split the linear map */
+			set_direct_map_default_nosplit(folio_page(folio, 0));
+			folio_zero_segment(folio, 0, folio_size(folio));
+			crash_wipe_cache_range(folio_address(folio),
+					       folio_size(folio));
+		}
+	}
+	spin_unlock(&sb->s_inode_list_lock);
+	/* off to kexec()! */
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block secretmem_wipe_nb = {
+	.notifier_call = secretmem_crash_wipe
+};
+
 static struct file *secretmem_file_create(unsigned long flags)
 {
 	struct file *file;
@@ -212,6 +262,8 @@ static struct file *secretmem_file_create(unsigned long flags)
 	inode->i_mode |= S_IFREG;
 	inode->i_size = 0;
 
+	inode_sb_list_add(inode);
+
 	atomic_inc(&secretmem_users);
 
 	return file;
@@ -263,6 +315,7 @@ static int __init secretmem_init(void)
 	if (IS_ERR(secretmem_mnt))
 		return PTR_ERR(secretmem_mnt);
 
+	crash_wipe_secrets_register(&secretmem_wipe_nb);
 	return 0;
 }
 fs_initcall(secretmem_init);

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:54 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:54 +0200
Subject: [PATCH v2 05/13] security/keys: wipe key payloads before kdump
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-5-9561d13c2340@jaseg.de>

When CONFIG_CRASH_WIPE_SECRETS is set, try to erase key payloads on
panic before jumping to the kdump kernel.

CRASH_WIPE_SECRETS notifiers run during panic() with other CPUs stopped
and preemption disabled. In this state, we can't rely on free()'ing
being safe, so we define a new `wipe` key op.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 Documentation/security/keys/core.rst | 13 +++++++++++
 include/linux/key-type.h             |  9 ++++++++
 security/keys/key.c                  | 43 ++++++++++++++++++++++++++++++++++++
 3 files changed, 65 insertions(+)

diff --git a/Documentation/security/keys/core.rst b/Documentation/security/keys/core.rst
index 326b8a973828..35cb00a87c46 100644
--- a/Documentation/security/keys/core.rst
+++ b/Documentation/security/keys/core.rst
@@ -1596,6 +1596,19 @@ The structure has a number of fields, some of which are mandatory:
      It is not safe to sleep in this method; the caller may hold spinlocks.
 
 
+  *  ``void (*wipe)(struct key *key);``
+
+     This method is optional. It is called from the panic path when
+     CONFIG_CRASH_WIPE_SECRETS is enabled, to erase the key material from
+     memory before the kdump kernel is started, so that it does not end up in
+     the crash dump. Unlike destroy(), it must only clear the payload, not
+     free it.
+
+     This method is called with all other CPUs stopped and preemption
+     disabled, and only for positively instantiated keys. It must not sleep,
+     allocate, free or take locks, as they will never be released.
+
+
   *  ``void (*describe)(const struct key *key, struct seq_file *p);``
 
      This method is optional. It is called during /proc/keys reading to
diff --git a/include/linux/key-type.h b/include/linux/key-type.h
index bb97bd3e5af4..21e07db0c5f2 100644
--- a/include/linux/key-type.h
+++ b/include/linux/key-type.h
@@ -122,6 +122,15 @@ struct key_type {
 	/* clear the data from a key (optional) */
 	void (*destroy)(struct key *key);
 
+	/* wipe the key material without free'ing (optional)
+	 * - used from CONFIG_CRASH_WIPE_SECRETS during panic to keep keys out
+	 *   of crash dumps
+	 * - called from the panic path with other CPUs stopped and preemption
+	 *   disabled
+	 * - must not sleep, allocate, free or take locks
+	 */
+	void (*wipe)(struct key *key);
+
 	/* describe a key */
 	void (*describe)(const struct key *key, struct seq_file *p);
 
diff --git a/security/keys/key.c b/security/keys/key.c
index b34a64d81d47..213e6f1d5d83 100644
--- a/security/keys/key.c
+++ b/security/keys/key.c
@@ -12,6 +12,7 @@
 #include <linux/slab.h>
 #include <linux/security.h>
 #include <linux/workqueue.h>
+#include <linux/crash_core.h>
 #include <linux/random.h>
 #include <linux/err.h>
 #include "internal.h"
@@ -1268,6 +1269,47 @@ void unregister_key_type(struct key_type *ktype)
 }
 EXPORT_SYMBOL(unregister_key_type);
 
+/* Called far into vpanic from crash_core.c with other CPUs stopped and
+ * preemption disabled
+ */
+static int key_crash_wipe(struct notifier_block *nb, unsigned long action,
+		void *data)
+{
+	struct rb_node *node;
+
+	/* If we can't acquire the lock, the rbtree might be in an inconsistent
+	 * state. That's all we can do then, as there's no point to waiting
+	 * at this stage.
+	 */
+	if (!spin_trylock(&key_serial_lock)) {
+		pr_crit("crash_wipe_secrets: can't acquire key_serial_lock. skipping keyrings.\n");
+		return NOTIFY_DONE;
+	}
+
+	for (node = rb_first(&key_serial_tree); node; node = rb_next(node)) {
+		struct key *key = rb_entry(node, struct key, serial_node);
+
+		/* Negatively instantiated keys have key->state < 0 and never
+		 * had a payload attached, so only wipe positive ones.
+		 */
+		if (key->type == &key_type_keyring || !key_is_positive(key))
+			continue;
+
+		/* We have a dedicated wipe callback for this since free'ing
+		 * isn't safe at this point
+		 */
+		if (key->type->wipe)
+			key->type->wipe(key);
+	}
+	spin_unlock(&key_serial_lock);
+	/* off to kexec()! */
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block key_crash_wipe_nb = {
+	.notifier_call = key_crash_wipe
+};
+
 /*
  * Initialise the key management state.
  */
@@ -1290,4 +1332,5 @@ void __init key_init(void)
 
 	rb_insert_color(&root_key_user.node,
 			&key_user_tree);
+	crash_wipe_secrets_register(&key_crash_wipe_nb);
 }

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:55 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:55 +0200
Subject: [PATCH v2 06/13] security/keys: implement wipe op for user-type
 keys
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-6-9561d13c2340@jaseg.de>

Wipe the payload of "user" and "logon" keys, and of the key types reusing
the user-type ops: s390 cert_store, NVMe TLS PSK, NFS idmap and PKCS#7.

Used by CONFIG_CRASH_WIPE_SECRETS.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 crypto/asymmetric_keys/pkcs7_key_type.c |  1 +
 drivers/nvme/common/keyring.c           |  1 +
 fs/nfs/nfs4idmap.c                      |  2 ++
 include/keys/user-type.h                |  1 +
 security/keys/user_defined.c            | 15 +++++++++++++++
 5 files changed, 20 insertions(+)

diff --git a/crypto/asymmetric_keys/pkcs7_key_type.c b/crypto/asymmetric_keys/pkcs7_key_type.c
index b930d3bbf1af..663a246742f7 100644
--- a/crypto/asymmetric_keys/pkcs7_key_type.c
+++ b/crypto/asymmetric_keys/pkcs7_key_type.c
@@ -74,6 +74,7 @@ static struct key_type key_type_pkcs7 = {
 	.instantiate		= generic_key_instantiate,
 	.revoke			= user_revoke,
 	.destroy		= user_destroy,
+	.wipe			= user_wipe,
 	.describe		= user_describe,
 	.read			= user_read,
 };
diff --git a/drivers/nvme/common/keyring.c b/drivers/nvme/common/keyring.c
index 32d16c53133b..33303f035fb1 100644
--- a/drivers/nvme/common/keyring.c
+++ b/drivers/nvme/common/keyring.c
@@ -84,6 +84,7 @@ static struct key_type nvme_tls_psk_key_type = {
 	.instantiate    = generic_key_instantiate,
 	.revoke         = user_revoke,
 	.destroy        = user_destroy,
+	.wipe		= user_wipe,
 	.describe       = nvme_tls_psk_describe,
 	.read           = user_read,
 };
diff --git a/fs/nfs/nfs4idmap.c b/fs/nfs/nfs4idmap.c
index bc397110d977..26f579ae2c73 100644
--- a/fs/nfs/nfs4idmap.c
+++ b/fs/nfs/nfs4idmap.c
@@ -190,6 +190,7 @@ static struct key_type key_type_id_resolver = {
 	.instantiate	= generic_key_instantiate,
 	.revoke		= user_revoke,
 	.destroy	= user_destroy,
+	.wipe		= user_wipe,
 	.describe	= user_describe,
 	.read		= user_read,
 };
@@ -412,6 +413,7 @@ static struct key_type key_type_id_resolver_legacy = {
 	.instantiate	= generic_key_instantiate,
 	.revoke		= user_revoke,
 	.destroy	= user_destroy,
+	.wipe		= user_wipe,
 	.describe	= user_describe,
 	.read		= user_read,
 	.request_key	= nfs_idmap_legacy_upcall,
diff --git a/include/keys/user-type.h b/include/keys/user-type.h
index 386c31432789..685fa054b0ac 100644
--- a/include/keys/user-type.h
+++ b/include/keys/user-type.h
@@ -40,6 +40,7 @@ extern void user_free_preparse(struct key_preparsed_payload *prep);
 extern int user_update(struct key *key, struct key_preparsed_payload *prep);
 extern void user_revoke(struct key *key);
 extern void user_destroy(struct key *key);
+extern void user_wipe(struct key *key);
 extern void user_describe(const struct key *user, struct seq_file *m);
 extern long user_read(const struct key *key, char *buffer, size_t buflen);
 
diff --git a/security/keys/user_defined.c b/security/keys/user_defined.c
index 6f88b507f927..f4897c19c2dd 100644
--- a/security/keys/user_defined.c
+++ b/security/keys/user_defined.c
@@ -6,6 +6,7 @@
  */
 
 #include <linux/export.h>
+#include <linux/crash_core.h>
 #include <linux/init.h>
 #include <linux/slab.h>
 #include <linux/seq_file.h>
@@ -28,6 +29,7 @@ struct key_type key_type_user = {
 	.update			= user_update,
 	.revoke			= user_revoke,
 	.destroy		= user_destroy,
+	.wipe			= user_wipe,
 	.describe		= user_describe,
 	.read			= user_read,
 };
@@ -48,6 +50,7 @@ struct key_type key_type_logon = {
 	.update			= user_update,
 	.revoke			= user_revoke,
 	.destroy		= user_destroy,
+	.wipe			= user_wipe,
 	.describe		= user_describe,
 	.vet_description	= logon_vet_description,
 };
@@ -152,6 +155,18 @@ void user_destroy(struct key *key)
 
 EXPORT_SYMBOL_GPL(user_destroy);
 
+void user_wipe(struct key *key)
+{
+	struct user_key_payload *upayload = key->payload.data[0];
+
+	if (upayload)
+		crash_wipe_memzero(upayload->data, upayload->datalen);
+	key->payload.data[1] = NULL;
+	key->payload.data[2] = NULL;
+	key->payload.data[3] = NULL;
+}
+EXPORT_SYMBOL_GPL(user_wipe);
+
 /*
  * describe the user key
  */

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:56 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:56 +0200
Subject: [PATCH v2 07/13] security/keys: implement wipe op for big_key
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-7-9561d13c2340@jaseg.de>

For keys stored in shmem only the ChaCha20-Poly1305 key is reachable here,
so wiping it is enough to render the payload unreadable.

Used by CONFIG_CRASH_WIPE_SECRETS.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 security/keys/big_key.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/security/keys/big_key.c b/security/keys/big_key.c
index 268f702df380..c01a212bec90 100644
--- a/security/keys/big_key.c
+++ b/security/keys/big_key.c
@@ -8,6 +8,7 @@
 
 #define pr_fmt(fmt) "big_key: "fmt
 #include <linux/init.h>
+#include <linux/crash_core.h>
 #include <linux/seq_file.h>
 #include <linux/file.h>
 #include <linux/shmem_fs.h>
@@ -35,6 +36,8 @@ struct big_key_payload {
  */
 #define BIG_KEY_FILE_THRESHOLD (sizeof(struct inode) + sizeof(struct dentry))
 
+static void big_key_wipe(struct key *key);
+
 /*
  * big_key defined keys take an arbitrary string as the description and an
  * arbitrary blob of data as the payload
@@ -46,6 +49,7 @@ struct key_type key_type_big_key = {
 	.instantiate		= generic_key_instantiate,
 	.revoke			= big_key_revoke,
 	.destroy		= big_key_destroy,
+	.wipe			= big_key_wipe,
 	.describe		= big_key_describe,
 	.read			= big_key_read,
 	.update			= big_key_update,
@@ -279,6 +283,19 @@ long big_key_read(const struct key *key, char *buffer, size_t buflen)
 	return ret;
 }
 
+static void big_key_wipe(struct key *key)
+{
+	struct big_key_payload *payload = to_big_key_payload(key->payload);
+
+	if (payload->data) {
+		if (payload->length > BIG_KEY_FILE_THRESHOLD)
+			crash_wipe_memzero(payload->data,
+					   CHACHA20POLY1305_KEY_SIZE);
+		else
+			crash_wipe_memzero(payload->data, payload->length);
+	}
+}
+
 /*
  * Register key type
  */

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:57 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:57 +0200
Subject: [PATCH v2 08/13] security/keys: implement wipe op for trusted and
 encrypted keys
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-8-9561d13c2340@jaseg.de>

Wipe the decrypted key material and the sealed blob held in the payload.

Used by CONFIG_CRASH_WIPE_SECRETS.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 security/keys/encrypted-keys/encrypted.c  | 13 +++++++++++++
 security/keys/trusted-keys/trusted_core.c | 15 +++++++++++++++
 2 files changed, 28 insertions(+)

diff --git a/security/keys/encrypted-keys/encrypted.c b/security/keys/encrypted-keys/encrypted.c
index 59cb77b237b3..9d6fe08ecb7a 100644
--- a/security/keys/encrypted-keys/encrypted.c
+++ b/security/keys/encrypted-keys/encrypted.c
@@ -12,6 +12,7 @@
  */
 
 #include <linux/uaccess.h>
+#include <linux/crash_core.h>
 #include <linux/module.h>
 #include <linux/hex.h>
 #include <linux/init.h>
@@ -970,11 +971,23 @@ static void encrypted_destroy(struct key *key)
 	kfree_sensitive(key->payload.data[0]);
 }
 
+static void encrypted_wipe(struct key *key)
+{
+	struct encrypted_key_payload *epayload = key->payload.data[0];
+
+	if (!epayload)
+		return;
+
+	crash_wipe_memzero(epayload->payload_data,
+			   epayload->payload_datalen + epayload->datablob_len);
+}
+
 struct key_type key_type_encrypted = {
 	.name = "encrypted",
 	.instantiate = encrypted_instantiate,
 	.update = encrypted_update,
 	.destroy = encrypted_destroy,
+	.wipe = encrypted_wipe,
 	.describe = user_describe,
 	.read = encrypted_read,
 };
diff --git a/security/keys/trusted-keys/trusted_core.c b/security/keys/trusted-keys/trusted_core.c
index 0509d9955f2a..7b220c6381b5 100644
--- a/security/keys/trusted-keys/trusted_core.c
+++ b/security/keys/trusted-keys/trusted_core.c
@@ -14,6 +14,7 @@
 #include <keys/trusted_tpm.h>
 #include <keys/trusted_pkwm.h>
 #include <linux/capability.h>
+#include <linux/crash_core.h>
 #include <linux/err.h>
 #include <linux/hex.h>
 #include <linux/init.h>
@@ -325,11 +326,25 @@ static void trusted_destroy(struct key *key)
 	kfree_sensitive(key->payload.data[0]);
 }
 
+static void trusted_wipe(struct key *key)
+{
+	struct trusted_key_payload *p = key->payload.data[0];
+
+	if (!p)
+		return;
+
+	crash_wipe_memzero(p->key, sizeof(p->key));
+	crash_wipe_memzero(p->blob, sizeof(p->blob));
+	p->key_len = 0;
+	p->blob_len = 0;
+}
+
 struct key_type key_type_trusted = {
 	.name = "trusted",
 	.instantiate = trusted_instantiate,
 	.update = trusted_update,
 	.destroy = trusted_destroy,
+	.wipe = trusted_wipe,
 	.describe = user_describe,
 	.read = trusted_read,
 };

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:58 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:58 +0200
Subject: [PATCH v2 09/13] security/keys: implement wipe op for asymmetric
 keys
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-9-9561d13c2340@jaseg.de>

Add wipe op to the subtype and implement it for public_key.

Used by CONFIG_CRASH_WIPE_SECRETS.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 crypto/asymmetric_keys/asymmetric_type.c | 11 +++++++++++
 crypto/asymmetric_keys/public_key.c      | 15 +++++++++++++++
 include/keys/asymmetric-subtype.h        |  5 +++++
 3 files changed, 31 insertions(+)

diff --git a/crypto/asymmetric_keys/asymmetric_type.c b/crypto/asymmetric_keys/asymmetric_type.c
index 16a7ae16593c..a0a68ebd0746 100644
--- a/crypto/asymmetric_keys/asymmetric_type.c
+++ b/crypto/asymmetric_keys/asymmetric_type.c
@@ -481,6 +481,16 @@ static void asymmetric_key_destroy(struct key *key)
 	asymmetric_key_free_kids(kids);
 }
 
+/* wipe the key without freeing. used by CONFIG_CRASH_WIPE_SECRETS. */
+static void asymmetric_key_wipe(struct key *key)
+{
+	struct asymmetric_key_subtype *subtype = asymmetric_key_subtype(key);
+
+	if (subtype && subtype->wipe)
+		subtype->wipe(key->payload.data[asym_crypto],
+				key->payload.data[asym_auth]);
+}
+
 static struct key_restriction *asymmetric_restriction_alloc(
 	key_restrict_link_func_t check,
 	struct key *key)
@@ -612,6 +622,7 @@ struct key_type key_type_asymmetric = {
 	.match_preparse		= asymmetric_key_match_preparse,
 	.match_free		= asymmetric_key_match_free,
 	.destroy		= asymmetric_key_destroy,
+	.wipe			= asymmetric_key_wipe,
 	.describe		= asymmetric_key_describe,
 	.lookup_restriction	= asymmetric_lookup_restriction,
 	.asym_query		= query_asymmetric_key,
diff --git a/crypto/asymmetric_keys/public_key.c b/crypto/asymmetric_keys/public_key.c
index 09a0b83d5d77..9931d56337e5 100644
--- a/crypto/asymmetric_keys/public_key.c
+++ b/crypto/asymmetric_keys/public_key.c
@@ -13,6 +13,7 @@
 #include <crypto/sig.h>
 #include <keys/asymmetric-subtype.h>
 #include <linux/asn1.h>
+#include <linux/crash_core.h>
 #include <linux/err.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
@@ -58,6 +59,19 @@ static void public_key_destroy(void *payload0, void *payload3)
 	public_key_signature_free(payload3);
 }
 
+/* wipe the key without freeing. used by CONFIG_CRASH_WIPE_SECRETS. */
+static void public_key_wipe(void *payload0, void *payload3)
+{
+	struct public_key *key = payload0;
+
+	if (key) {
+		if (key->key)
+			crash_wipe_memzero(key->key, key->keylen);
+		if (key->params)
+			crash_wipe_memzero(key->params, key->paramlen);
+	}
+}
+
 /*
  * Given a public_key, and an encoding and hash_algo to be used for signing
  * and/or verification with that key, determine the name of the corresponding
@@ -464,6 +478,7 @@ struct asymmetric_key_subtype public_key_subtype = {
 	.name_len		= sizeof("public_key") - 1,
 	.describe		= public_key_describe,
 	.destroy		= public_key_destroy,
+	.wipe			= public_key_wipe,
 	.query			= software_key_query,
 	.eds_op			= software_key_eds_op,
 	.verify_signature	= public_key_verify_signature_2,
diff --git a/include/keys/asymmetric-subtype.h b/include/keys/asymmetric-subtype.h
index d55171f640a0..17c74070c101 100644
--- a/include/keys/asymmetric-subtype.h
+++ b/include/keys/asymmetric-subtype.h
@@ -32,6 +32,11 @@ struct asymmetric_key_subtype {
 	/* Destroy a key of this subtype */
 	void (*destroy)(void *payload_crypto, void *payload_auth);
 
+	/* Wipe a key of this subtype without freeing it (optional).  Used by
+	 * CONFIG_CRASH_WIPE_SECRETS from the panic path.
+	 */
+	void (*wipe)(void *payload_crypto, void *payload_auth);
+
 	int (*query)(const struct kernel_pkey_params *params,
 		     struct kernel_pkey_query *info);
 

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:52:59 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:52:59 +0200
Subject: [PATCH v2 10/13] rxrpc: implement wipe op for rxrpc keys
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-10-9561d13c2340@jaseg.de>

Wipe session keys and tickets of client key tokens, and add a server key
wipe op implemented by rxkad and rxgk.

Used by CONFIG_CRASH_WIPE_SECRETS.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 net/rxrpc/ar-internal.h |  5 +++++
 net/rxrpc/key.c         | 34 ++++++++++++++++++++++++++++++++++
 net/rxrpc/rxgk.c        | 11 +++++++++++
 net/rxrpc/rxkad.c       | 14 ++++++++++++++
 net/rxrpc/server_key.c  | 11 +++++++++++
 5 files changed, 75 insertions(+)

diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 865f05fe37ab..6466eb5a9929 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -282,6 +282,11 @@ struct rxrpc_security {
 	/* Destroy the payload of a server key */
 	void (*destroy_server_key)(struct key *);
 
+	/* Wipe the payload of a server key without freeing it.  Used by
+	 * CONFIG_CRASH_WIPE_SECRETS from the panic path.
+	 */
+	void (*wipe_server_key)(struct key *);
+
 	/* Describe a server key */
 	void (*describe_server_key)(const struct key *, struct seq_file *);
 
diff --git a/net/rxrpc/key.c b/net/rxrpc/key.c
index a0aa78d89289..e612b7a8df13 100644
--- a/net/rxrpc/key.c
+++ b/net/rxrpc/key.c
@@ -11,6 +11,7 @@
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
 #include <linux/module.h>
+#include <linux/crash_core.h>
 #include <linux/net.h>
 #include <linux/overflow.h>
 #include <linux/skbuff.h>
@@ -26,6 +27,7 @@
 static int rxrpc_preparse(struct key_preparsed_payload *);
 static void rxrpc_free_preparse(struct key_preparsed_payload *);
 static void rxrpc_destroy(struct key *);
+static void rxrpc_wipe(struct key *);
 static void rxrpc_describe(const struct key *, struct seq_file *);
 static long rxrpc_read(const struct key *, char *, size_t);
 
@@ -40,6 +42,7 @@ struct key_type key_type_rxrpc = {
 	.free_preparse	= rxrpc_free_preparse,
 	.instantiate	= generic_key_instantiate,
 	.destroy	= rxrpc_destroy,
+	.wipe		= rxrpc_wipe,
 	.describe	= rxrpc_describe,
 	.read		= rxrpc_read,
 };
@@ -570,6 +573,31 @@ static void rxrpc_free_token_list(struct rxrpc_key_token *token)
 	}
 }
 
+static void rxrpc_wipe_token_list(struct rxrpc_key_token *token)
+{
+	struct rxrpc_key_token *next;
+
+	for (; token; token = next) {
+		next = token->next;
+		switch (token->security_index) {
+		case RXRPC_SECURITY_RXKAD:
+			crash_wipe_memzero(token->kad->session_key,
+					   sizeof(token->kad->session_key));
+			crash_wipe_memzero(token->kad->ticket,
+					   token->kad->ticket_len);
+			break;
+		case RXRPC_SECURITY_YFS_RXGK:
+			crash_wipe_memzero(token->rxgk->key.data,
+					   token->rxgk->key.len);
+			crash_wipe_memzero(token->rxgk->ticket.data,
+					   token->rxgk->ticket.len);
+			break;
+		default:
+			break;
+		}
+	}
+}
+
 /*
  * Clean up preparse data.
  */
@@ -586,6 +614,12 @@ static void rxrpc_destroy(struct key *key)
 	rxrpc_free_token_list(key->payload.data[0]);
 }
 
+/* wipe the key without freeing. used by CONFIG_CRASH_WIPE_SECRETS. */
+static void rxrpc_wipe(struct key *key)
+{
+	rxrpc_wipe_token_list(key->payload.data[0]);
+}
+
 /*
  * describe the rxrpc key
  */
diff --git a/net/rxrpc/rxgk.c b/net/rxrpc/rxgk.c
index 77a67ace1d24..51c428fb3a60 100644
--- a/net/rxrpc/rxgk.c
+++ b/net/rxrpc/rxgk.c
@@ -8,6 +8,7 @@
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
 #include <linux/net.h>
+#include <linux/crash_core.h>
 #include <linux/skbuff.h>
 #include <linux/slab.h>
 #include <linux/key-type.h>
@@ -68,6 +69,15 @@ static void rxgk_destroy_server_key(struct key *key)
 	rxgk_free_server_key(&key->payload);
 }
 
+/* wipe the key without freeing. used by CONFIG_CRASH_WIPE_SECRETS. */
+static void rxgk_wipe_server_key(struct key *key)
+{
+	struct krb5_buffer *server_key = (void *)&key->payload.data[2];
+
+	if (server_key->data)
+		crash_wipe_memzero(server_key->data, server_key->len);
+}
+
 static void rxgk_describe_server_key(const struct key *key, struct seq_file *m)
 {
 	const struct krb5_enctype *krb5 = key->payload.data[0];
@@ -1338,6 +1348,7 @@ const struct rxrpc_security rxgk_yfs = {
 	.preparse_server_key		= rxgk_preparse_server_key,
 	.free_preparse_server_key	= rxgk_free_preparse_server_key,
 	.destroy_server_key		= rxgk_destroy_server_key,
+	.wipe_server_key		= rxgk_wipe_server_key,
 	.describe_server_key		= rxgk_describe_server_key,
 	.init_connection_security	= rxgk_init_connection_security,
 	.alloc_txbuf			= rxgk_alloc_txbuf,
diff --git a/net/rxrpc/rxkad.c b/net/rxrpc/rxkad.c
index ca9f0e82cb9a..054ab67aca32 100644
--- a/net/rxrpc/rxkad.c
+++ b/net/rxrpc/rxkad.c
@@ -10,6 +10,7 @@
 #include <crypto/des.h>
 #include <kunit/visibility.h>
 #include <linux/export.h>
+#include <linux/crash_core.h>
 #include <linux/fips.h>
 #include <linux/module.h>
 #include <linux/net.h>
@@ -91,6 +92,18 @@ static void rxkad_destroy_server_key(struct key *key)
 	key->payload.data[0] = NULL;
 }
 
+/* wipe the key without freeing. used by CONFIG_CRASH_WIPE_SECRETS. */
+static void rxkad_wipe_server_key(struct key *key)
+{
+	struct des_ctx *des_key = key->payload.data[0];
+
+	if (des_key)
+		crash_wipe_memzero(des_key, sizeof(*des_key));
+
+	/* the raw 8-byte key is kept inline in the payload union */
+	crash_wipe_memzero(&key->payload.data[2], 8);
+}
+
 /*
  * initialise connection security
  */
@@ -1127,6 +1140,7 @@ const struct rxrpc_security rxkad = {
 	.preparse_server_key		= rxkad_preparse_server_key,
 	.free_preparse_server_key	= rxkad_free_preparse_server_key,
 	.destroy_server_key		= rxkad_destroy_server_key,
+	.wipe_server_key		= rxkad_wipe_server_key,
 	.init_connection_security	= rxkad_init_connection_security,
 	.alloc_txbuf			= rxkad_alloc_txbuf,
 	.secure_packet			= rxkad_secure_packet,
diff --git a/net/rxrpc/server_key.c b/net/rxrpc/server_key.c
index 3efe104b1930..a8e736d58793 100644
--- a/net/rxrpc/server_key.c
+++ b/net/rxrpc/server_key.c
@@ -26,6 +26,7 @@ static int rxrpc_vet_description_s(const char *);
 static int rxrpc_preparse_s(struct key_preparsed_payload *);
 static void rxrpc_free_preparse_s(struct key_preparsed_payload *);
 static void rxrpc_destroy_s(struct key *);
+static void rxrpc_wipe_s(struct key *);
 static void rxrpc_describe_s(const struct key *, struct seq_file *);
 
 /*
@@ -40,6 +41,7 @@ struct key_type key_type_rxrpc_s = {
 	.free_preparse	= rxrpc_free_preparse_s,
 	.instantiate	= generic_key_instantiate,
 	.destroy	= rxrpc_destroy_s,
+	.wipe		= rxrpc_wipe_s,
 	.describe	= rxrpc_describe_s,
 };
 
@@ -105,6 +107,15 @@ static void rxrpc_destroy_s(struct key *key)
 		sec->destroy_server_key(key);
 }
 
+/* wipe the key without freeing. used by CONFIG_CRASH_WIPE_SECRETS. */
+static void rxrpc_wipe_s(struct key *key)
+{
+	const struct rxrpc_security *sec = key->payload.data[1];
+
+	if (sec && sec->wipe_server_key)
+		sec->wipe_server_key(key);
+}
+
 static void rxrpc_describe_s(const struct key *key, struct seq_file *m)
 {
 	const struct rxrpc_security *sec = key->payload.data[1];

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:53:00 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:53:00 +0200
Subject: [PATCH v2 11/13] fscrypt: wipe master keys before kdump
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-11-9561d13c2340@jaseg.de>

fscrypt master keys hang off the superblock rather than a keyring of their
own key type, so register a separate notifier that walks the live
superblocks via the new super_blocks_crash_wipe() helper.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 fs/crypto/block.c           | 10 +++++++
 fs/crypto/fscrypt_private.h | 15 ++++++++++
 fs/crypto/keyring.c         | 68 ++++++++++++++++++++++++++++++++++++++++++++-
 fs/crypto/keysetup_v1.c     | 15 ++++++++++
 fs/super.c                  | 29 +++++++++++++++++++
 include/linux/fs.h          |  4 +++
 6 files changed, 140 insertions(+), 1 deletion(-)

diff --git a/fs/crypto/block.c b/fs/crypto/block.c
index 5193f8ba3ee0..a2fa1469b3c4 100644
--- a/fs/crypto/block.c
+++ b/fs/crypto/block.c
@@ -15,6 +15,7 @@
 
 #include <linux/blk-crypto.h>
 #include <linux/blkdev.h>
+#include <linux/crash_core.h>
 #include <linux/export.h>
 #include <linux/sched/mm.h>
 #include <linux/slab.h>
@@ -144,6 +145,15 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key,
 	return err;
 }
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+void fscrypt_crash_wipe_inline_crypt_key(struct fscrypt_prepared_key *prep_key)
+{
+	if (prep_key->blk_key)
+		crash_wipe_memzero(prep_key->blk_key->bytes,
+				   sizeof(prep_key->blk_key->bytes));
+}
+#endif /* CONFIG_CRASH_WIPE_SECRETS */
+
 void fscrypt_destroy_inline_crypt_key(struct super_block *sb,
 				      struct fscrypt_prepared_key *prep_key)
 {
diff --git a/fs/crypto/fscrypt_private.h b/fs/crypto/fscrypt_private.h
index 74329e0953d1..95cb50e5cbcd 100644
--- a/fs/crypto/fscrypt_private.h
+++ b/fs/crypto/fscrypt_private.h
@@ -413,6 +413,10 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key,
 void fscrypt_destroy_inline_crypt_key(struct super_block *sb,
 				      struct fscrypt_prepared_key *prep_key);
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS /* && CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
+void fscrypt_crash_wipe_inline_crypt_key(struct fscrypt_prepared_key *prep_key);
+#endif
+
 int fscrypt_derive_sw_secret(struct super_block *sb,
 			     const u8 *wrapped_key, size_t wrapped_key_size,
 			     u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE]);
@@ -454,6 +458,13 @@ fscrypt_destroy_inline_crypt_key(struct super_block *sb,
 {
 }
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS /* && !CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
+static inline void
+fscrypt_crash_wipe_inline_crypt_key(struct fscrypt_prepared_key *prep_key)
+{
+}
+#endif
+
 static inline int
 fscrypt_derive_sw_secret(struct super_block *sb,
 			 const u8 *wrapped_key, size_t wrapped_key_size,
@@ -760,6 +771,10 @@ static inline int fscrypt_require_key(struct inode *inode)
 
 void fscrypt_put_direct_key(struct fscrypt_direct_key *dk);
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+void fscrypt_crash_wipe_direct_keys(void);
+#endif
+
 int fscrypt_setup_v1_file_key(struct fscrypt_inode_info *ci,
 			      const u8 *raw_master_key);
 
diff --git a/fs/crypto/keyring.c b/fs/crypto/keyring.c
index 76e28d1e0064..5ac302146eb2 100644
--- a/fs/crypto/keyring.c
+++ b/fs/crypto/keyring.c
@@ -19,6 +19,7 @@
  */
 
 #include <crypto/skcipher.h>
+#include <linux/crash_core.h>
 #include <linux/export.h>
 #include <linux/key-type.h>
 #include <linux/once.h>
@@ -239,8 +240,9 @@ void fscrypt_destroy_keyring(struct super_block *sb)
 			fscrypt_initiate_key_removal(sb, mk);
 		}
 	}
+	/* Stop panic-time walkers from finding @keyring before it is freed. */
+	smp_store_release(&sb->s_master_keys, NULL);
 	kfree_sensitive(keyring);
-	sb->s_master_keys = NULL;
 }
 
 static struct hlist_head *
@@ -640,6 +642,13 @@ static void fscrypt_provisioning_key_destroy(struct key *key)
 	kfree_sensitive(key->payload.data[0]);
 }
 
+/* wipe the key without freeing. used by CONFIG_CRASH_WIPE_SECRETS. */
+static void fscrypt_provisioning_key_wipe(struct key *key)
+{
+	if (key->payload.data[0])
+		crash_wipe_memzero(key->payload.data[0], key->datalen);
+}
+
 static struct key_type key_type_fscrypt_provisioning = {
 	.name			= "fscrypt-provisioning",
 	.preparse		= fscrypt_provisioning_key_preparse,
@@ -647,6 +656,7 @@ static struct key_type key_type_fscrypt_provisioning = {
 	.instantiate		= generic_key_instantiate,
 	.describe		= fscrypt_provisioning_key_describe,
 	.destroy		= fscrypt_provisioning_key_destroy,
+	.wipe			= fscrypt_provisioning_key_wipe,
 };
 
 /*
@@ -1220,6 +1230,58 @@ int fscrypt_ioctl_get_key_status(struct file *filp, void __user *uarg)
 }
 EXPORT_SYMBOL_GPL(fscrypt_ioctl_get_key_status);
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+/*
+ * Wipe the master keys of one superblock.  The master keys don't live on a
+ * keyring of their own key_type, so the keyrings core can't reach them.
+ */
+static void fscrypt_crash_wipe_sb(struct super_block *sb)
+{
+	struct fscrypt_keyring *keyring = sb->s_master_keys;
+	size_t i;
+
+	if (!keyring)
+		return;
+
+	for (i = 0; i < ARRAY_SIZE(keyring->key_hashtable); i++) {
+		struct fscrypt_master_key *mk;
+
+		hlist_for_each_entry(mk, &keyring->key_hashtable[i], mk_node) {
+			struct fscrypt_inode_info *ci;
+			struct fscrypt_mode_key *node;
+
+			crash_wipe_memzero(&mk->mk_secret,
+					   sizeof(mk->mk_secret));
+
+			list_for_each_entry(node, &mk->mk_mode_keys, link)
+				fscrypt_crash_wipe_inline_crypt_key(&node->key);
+
+			list_for_each_entry(ci, &mk->mk_decrypted_inodes,
+					    ci_master_key_link)
+				fscrypt_crash_wipe_inline_crypt_key(&ci->ci_enc_key);
+		}
+	}
+}
+
+/* Called far into vpanic from crash_core.c with other CPUs stopped and
+ * preemption disabled
+ */
+static int fscrypt_crash_wipe(struct notifier_block *nb, unsigned long action,
+			      void *data)
+{
+	if (!super_blocks_crash_wipe(fscrypt_crash_wipe_sb))
+		pr_crit("crash_wipe_secrets: can't acquire sb_lock. skipping fscrypt keys.\n");
+
+	fscrypt_crash_wipe_direct_keys();
+
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block fscrypt_crash_wipe_nb = {
+	.notifier_call = fscrypt_crash_wipe
+};
+#endif /* CONFIG_CRASH_WIPE_SECRETS */
+
 void __init fscrypt_init_keyring(void)
 {
 	int err;
@@ -1235,4 +1297,8 @@ void __init fscrypt_init_keyring(void)
 	if (err)
 		panic("failed to register fscrypt-provisioning key type (%d)",
 		      err);
+
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+	crash_wipe_secrets_register(&fscrypt_crash_wipe_nb);
+#endif
 }
diff --git a/fs/crypto/keysetup_v1.c b/fs/crypto/keysetup_v1.c
index 87fe13ccb253..08dd0a682a8f 100644
--- a/fs/crypto/keysetup_v1.c
+++ b/fs/crypto/keysetup_v1.c
@@ -23,6 +23,7 @@
 #include <crypto/aes.h>
 #include <crypto/utils.h>
 #include <keys/user-type.h>
+#include <linux/crash_core.h>
 #include <linux/hashtable.h>
 
 #include "fscrypt_private.h"
@@ -118,6 +119,20 @@ void fscrypt_put_direct_key(struct fscrypt_direct_key *dk)
 	free_direct_key(dk);
 }
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+void fscrypt_crash_wipe_direct_keys(void)
+{
+	struct fscrypt_direct_key *dk;
+	unsigned int i;
+
+	/* No locking: all other CPUs are stopped, so nothing can race with us. */
+	hash_for_each(fscrypt_direct_keys, i, dk, dk_node) {
+		crash_wipe_memzero(dk->dk_raw, sizeof(dk->dk_raw));
+		fscrypt_crash_wipe_inline_crypt_key(&dk->dk_key);
+	}
+}
+#endif /* CONFIG_CRASH_WIPE_SECRETS */
+
 /*
  * Find/insert the given key into the fscrypt_direct_keys table.  If found, it
  * is returned with elevated refcount, and 'to_insert' is freed if non-NULL.  If
diff --git a/fs/super.c b/fs/super.c
index 5feecf5d9038..cd51e9cfe93b 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -2463,3 +2463,32 @@ int sb_init_dio_done_wq(struct super_block *sb)
 	return 0;
 }
 EXPORT_SYMBOL_GPL(sb_init_dio_done_wq);
+
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+/**
+ * super_blocks_crash_wipe - run @wipe against every live superblock
+ * @wipe: callback to invoke for each superblock
+ *
+ * Called from the panic path with other CPUs stopped and preemption disabled.
+ * The callback must not sleep, allocate, free or take locks.
+ *
+ * Returns false without doing anything if @sb_lock could not be acquired, in
+ * which case the list may be inconsistent and walking it is unsafe.
+ */
+bool super_blocks_crash_wipe(void (*wipe)(struct super_block *sb))
+{
+	struct super_block *sb;
+
+	/* There is no point in waiting for a lock that will never be released
+	 * at this stage.
+	 */
+	if (!spin_trylock(&sb_lock))
+		return false;
+
+	list_for_each_entry(sb, &super_blocks, s_list)
+		wipe(sb);
+
+	spin_unlock(&sb_lock);
+	return true;
+}
+#endif
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 072d8cd09a0b..7832a77908d2 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2378,6 +2378,10 @@ extern __printf(2, 3)
 int super_setup_bdi_name(struct super_block *sb, char *fmt, ...);
 extern int super_setup_bdi(struct super_block *sb);
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+bool super_blocks_crash_wipe(void (*wipe)(struct super_block *sb));
+#endif
+
 static inline void super_set_uuid(struct super_block *sb, const u8 *uuid, unsigned len)
 {
 	if (WARN_ON(len > sizeof(sb->s_uuid)))

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:53:01 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:53:01 +0200
Subject: [PATCH v2 12/13] crypto: api - wipe tfm contexts before kdump
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-12-9561d13c2340@jaseg.de>

tfm structs contain key material like expanded key schedules. Under
CONFIG_CRASH_WIPE_SECRETS, wipe all tfms before kdump to prevent
leakage. Copies outside the tfm context, e.g. on the stack or in
hardware key registers, are not covered.

Also change two kfree() calls to kfree_sensitive() for good measure.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 crypto/api.c           | 78 ++++++++++++++++++++++++++++++++++++++++++++------
 include/linux/crypto.h | 11 +++++++
 2 files changed, 81 insertions(+), 8 deletions(-)

diff --git a/crypto/api.c b/crypto/api.c
index 24227582cfcf..f919f52e8b5f 100644
--- a/crypto/api.c
+++ b/crypto/api.c
@@ -10,13 +10,16 @@
  * and Nettle, by Niels M?ller.
  */
 
+#include <linux/crash_core.h>
 #include <linux/err.h>
 #include <linux/errno.h>
 #include <linux/jump_label.h>
 #include <linux/kernel.h>
 #include <linux/kmod.h>
 #include <linux/module.h>
+#include <linux/notifier.h>
 #include <linux/param.h>
+#include <linux/rculist.h>
 #include <linux/sched/signal.h>
 #include <linux/slab.h>
 #include <linux/string.h>
@@ -397,6 +400,60 @@ static unsigned int crypto_ctxsize(struct crypto_alg *alg, u32 type, u32 mask)
 	return len;
 }
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+/*
+ * tfm allocations are tracked on crypto_tfm_list so that they can be wiped
+ * before kdump.
+ */
+static HLIST_HEAD(crypto_tfm_list);
+static DEFINE_SPINLOCK(crypto_tfm_list_lock);
+
+static int crypto_crash_wipe(struct notifier_block *nb, unsigned long action,
+		void *data)
+{
+	struct crypto_tfm *tfm;
+
+	rcu_read_lock();
+	hlist_for_each_entry_rcu(tfm, &crypto_tfm_list, wipe_list)
+		crash_wipe_memzero(tfm->__crt_ctx, tfm->wipe_size);
+	rcu_read_unlock();
+
+	/* off to kexec()! */
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block crypto_wipe_nb = {
+	.notifier_call = crypto_crash_wipe
+};
+
+static int __init crypto_tfm_track_init(void)
+{
+	crash_wipe_secrets_register(&crypto_wipe_nb);
+	return 0;
+}
+core_initcall(crypto_tfm_track_init);
+
+/* @size is the size of __crt_ctx */
+static void crypto_track_tfm(struct crypto_tfm *tfm, size_t size)
+{
+	tfm->wipe_size = size;
+
+	spin_lock(&crypto_tfm_list_lock);
+	hlist_add_head_rcu(&tfm->wipe_list, &crypto_tfm_list);
+	spin_unlock(&crypto_tfm_list_lock);
+}
+
+static void crypto_untrack_tfm(struct crypto_tfm *tfm)
+{
+	spin_lock(&crypto_tfm_list_lock);
+	hlist_del_rcu(&tfm->wipe_list);
+	spin_unlock(&crypto_tfm_list_lock);
+}
+#else
+static void crypto_track_tfm(struct crypto_tfm *tfm, size_t size) { }
+static void crypto_untrack_tfm(struct crypto_tfm *tfm) { }
+#endif /* CONFIG_CRASH_WIPE_SECRETS */
+
 void crypto_shoot_alg(struct crypto_alg *alg)
 {
 	down_write(&crypto_alg_sem);
@@ -409,15 +466,16 @@ struct crypto_tfm *__crypto_alloc_tfm(struct crypto_alg *alg, u32 type,
 				      u32 mask)
 {
 	struct crypto_tfm *tfm;
-	unsigned int tfm_size;
+	unsigned int ctx_size;
 	int err = -ENOMEM;
 
-	tfm_size = sizeof(*tfm) + crypto_ctxsize(alg, type, mask);
-	tfm = kzalloc(tfm_size, GFP_KERNEL);
+	ctx_size = crypto_ctxsize(alg, type, mask);
+	tfm = kzalloc(sizeof(*tfm) + ctx_size, GFP_KERNEL);
 	if (tfm == NULL)
 		goto out_err;
 
 	tfm->__crt_alg = alg;
+	crypto_track_tfm(tfm, ctx_size);
 
 	if (!tfm->exit && alg->cra_init && (err = alg->cra_init(tfm)))
 		goto cra_init_failed;
@@ -428,7 +486,8 @@ struct crypto_tfm *__crypto_alloc_tfm(struct crypto_alg *alg, u32 type,
 	crypto_exit_ops(tfm);
 	if (err == -EAGAIN)
 		crypto_shoot_alg(alg);
-	kfree(tfm);
+	crypto_untrack_tfm(tfm);
+	kfree_sensitive(tfm);
 out_err:
 	tfm = ERR_PTR(err);
 out:
@@ -497,12 +556,12 @@ void *crypto_create_tfm_node(struct crypto_alg *alg,
 			     int node)
 {
 	struct crypto_tfm *tfm;
-	size_t size;
 	char *mem;
 	int err;
+	size_t ctx_size = frontend->extsize(alg);
 
-	size = frontend->tfmsize + sizeof(*tfm) + frontend->extsize(alg);
-	mem = kzalloc_node(size, GFP_KERNEL, node);
+	mem = kzalloc_node(frontend->tfmsize + sizeof(*tfm) + ctx_size,
+			GFP_KERNEL, node);
 	if (!mem)
 		return ERR_PTR(-ENOMEM);
 
@@ -510,6 +569,7 @@ void *crypto_create_tfm_node(struct crypto_alg *alg,
 	tfm->__crt_alg = alg;
 	tfm->node = node;
 	tfm->fb = tfm;
+	crypto_track_tfm(tfm, ctx_size);
 
 	err = frontend->init_tfm(tfm);
 	if (err)
@@ -525,7 +585,8 @@ void *crypto_create_tfm_node(struct crypto_alg *alg,
 out_free_tfm:
 	if (err == -EAGAIN)
 		crypto_shoot_alg(alg);
-	kfree(mem);
+	crypto_untrack_tfm(tfm);
+	kfree_sensitive(mem);
 	mem = ERR_PTR(err);
 out:
 	return mem;
@@ -627,6 +688,7 @@ void crypto_destroy_tfm(void *mem, struct crypto_tfm *tfm)
 		alg->cra_exit(tfm);
 	crypto_exit_ops(tfm);
 	crypto_mod_put(alg);
+	crypto_untrack_tfm(tfm);
 	kfree_sensitive(mem);
 }
 EXPORT_SYMBOL_GPL(crypto_destroy_tfm);
diff --git a/include/linux/crypto.h b/include/linux/crypto.h
index b7c97f1c47c9..59d6d62180fd 100644
--- a/include/linux/crypto.h
+++ b/include/linux/crypto.h
@@ -14,6 +14,7 @@
 
 #include <linux/completion.h>
 #include <linux/errno.h>
+#include <linux/list.h>
 #include <linux/refcount_types.h>
 #include <linux/slab.h>
 #include <linux/types.h>
@@ -420,6 +421,16 @@ struct crypto_tfm {
 
 	struct crypto_alg *__crt_alg;
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+	/*
+	 * Link on crypto_tfm_list, used to wipe the expanded key schedule in
+	 * __crt_ctx before kdump. wipe_size covers __crt_ctx only, excluding
+	 * this struct and any frontend preceding it.
+	 */
+	struct hlist_node wipe_list;
+	size_t wipe_size;
+#endif
+
 	void *__crt_ctx[] CRYPTO_MINALIGN_ATTR;
 };
 

-- 
2.53.0



From linux at jaseg.de  Tue Aug 11 10:53:02 2026
From: linux at jaseg.de (=?utf-8?q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 19:53:02 +0200
Subject: [PATCH v2 13/13] dm crypt: wipe key material before kdump
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811-crash-zeroize-rework-v2-13-9561d13c2340@jaseg.de>

Wipe volume key/iv copies kept by dm-crypt with
CONFIG_CRASH_WIPE_SECRETS. The backend tfms are already handled
separately.

Add a list tracking struct crypt_config instances when
CONFIG_CRASH_WIPE_SECRETS is set. Structs are tracked here to avoid
having to enumerate them through some roundabout way before kdump, when
we can't safely take locks anymore.

Use custom wipe handlers even for things like ivs that have existing
wipe functions elsewhere because we need to use crash_wipe_memzero
instead of memzero_explicit. The crash_wipe helper memzero_explicit's
the target buffers and flushes data caches. On ARM64, missing that cache
flush could lead to the zeros not being written to DRAM before the kdump
code turns off the data caches moments later.

Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
---
 drivers/md/dm-crypt.c | 151 ++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 134 insertions(+), 17 deletions(-)

diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 608b617fb817..86adb2f9b94c 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -9,12 +9,14 @@
  */
 
 #include <linux/completion.h>
+#include <linux/crash_core.h>
 #include <linux/err.h>
 #include <linux/module.h>
 #include <linux/hex.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
 #include <linux/key.h>
+#include <linux/list.h>
 #include <linux/bio.h>
 #include <linux/blkdev.h>
 #include <linux/blk-integrity.h>
@@ -234,6 +236,11 @@ struct crypt_config {
 	struct mutex bio_alloc_lock;
 
 	u8 *authenc_key; /* space for keys in authenc() format (if used) */
+
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+	struct list_head wipe_list;
+#endif
+
 	u8 key[] __counted_by(key_size);
 };
 
@@ -243,6 +250,12 @@ struct crypt_config {
 
 static DEFINE_SPINLOCK(dm_crypt_clients_lock);
 static unsigned int dm_crypt_clients_n;
+
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+static LIST_HEAD(dm_crypt_wipe_list);
+static DEFINE_SPINLOCK(dm_crypt_wipe_list_lock);
+#endif
+
 static volatile unsigned long dm_crypt_pages_per_client;
 #define DM_CRYPT_MEMORY_PERCENT			2
 #define DM_CRYPT_MIN_PAGES_PER_CLIENT		(BIO_MAX_VECS * 16)
@@ -460,8 +473,7 @@ static void crypt_iv_lmk_dtr(struct crypt_config *cc)
 {
 	struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
 
-	kfree_sensitive(lmk->seed);
-	lmk->seed = NULL;
+	kfree_sensitive(xchg(&lmk->seed, NULL));
 }
 
 static int crypt_iv_lmk_ctr(struct crypt_config *cc, struct dm_target *ti,
@@ -582,10 +594,8 @@ static void crypt_iv_tcw_dtr(struct crypt_config *cc)
 {
 	struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
 
-	kfree_sensitive(tcw->iv_seed);
-	tcw->iv_seed = NULL;
-	kfree_sensitive(tcw->whitening);
-	tcw->whitening = NULL;
+	kfree_sensitive(xchg(&tcw->iv_seed, NULL));
+	kfree_sensitive(xchg(&tcw->whitening, NULL));
 }
 
 static int crypt_iv_tcw_ctr(struct crypt_config *cc, struct dm_target *ti,
@@ -761,8 +771,7 @@ static void crypt_iv_elephant_dtr(struct crypt_config *cc)
 {
 	struct iv_elephant_private *elephant = &cc->iv_gen_private.elephant;
 
-	kfree_sensitive(elephant->key);
-	elephant->key = NULL;
+	kfree_sensitive(xchg(&elephant->key, NULL));
 }
 
 static int crypt_iv_elephant_ctr(struct crypt_config *cc, struct dm_target *ti,
@@ -2547,8 +2556,7 @@ static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string
 		goto free_new_key_string;
 
 	set_bit(DM_CRYPT_KEY_VALID, &cc->flags);
-	kfree_sensitive(cc->key_string);
-	cc->key_string = new_key_string;
+	kfree_sensitive(xchg(&cc->key_string, new_key_string));
 	return 0;
 
 free_new_key_string:
@@ -2612,8 +2620,7 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
 	clear_bit(DM_CRYPT_KEY_VALID, &cc->flags);
 
 	/* wipe references to any kernel keyring key */
-	kfree_sensitive(cc->key_string);
-	cc->key_string = NULL;
+	kfree_sensitive(xchg(&cc->key_string, NULL));
 
 	/* Decode key from its hex representation. */
 	if (cc->key_size && hex2bin(cc->key, key, cc->key_size) < 0)
@@ -2641,14 +2648,97 @@ static int crypt_wipe_key(struct crypt_config *cc)
 	if (cc->iv_gen_ops && cc->iv_gen_ops->wipe)
 		cc->iv_gen_ops->wipe(cc);
 
-	kfree_sensitive(cc->key_string);
-	cc->key_string = NULL;
+	kfree_sensitive(xchg(&cc->key_string, NULL));
 	r = crypt_setkey(cc);
 	memset(&cc->key, 0, cc->key_size * sizeof(u8));
 
 	return r;
 }
 
+#ifdef CONFIG_CRASH_WIPE_SECRETS
+static void crypt_crash_wipe_iv(struct crypt_config *cc)
+{
+	if (cc->iv_gen_ops == &crypt_iv_lmk_ops) {
+		struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
+
+		if (lmk->seed)
+			crash_wipe_memzero(lmk->seed, LMK_SEED_SIZE);
+	} else if (cc->iv_gen_ops == &crypt_iv_tcw_ops) {
+		struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
+
+		if (tcw->iv_seed)
+			crash_wipe_memzero(tcw->iv_seed, cc->iv_size);
+		if (tcw->whitening)
+			crash_wipe_memzero(tcw->whitening, TCW_WHITENING_SIZE);
+	} else if (cc->iv_gen_ops == &crypt_iv_elephant_ops) {
+		struct iv_elephant_private *elephant =
+			&cc->iv_gen_private.elephant;
+
+		if (elephant->key)
+			crash_wipe_memzero(elephant->key,
+					   sizeof(*elephant->key));
+	}
+}
+
+static int crypt_crash_wipe(struct notifier_block *nb, unsigned long action,
+			    void *data)
+{
+	struct crypt_config *cc;
+
+	/* No locking: all other CPUs are stopped, and a cc is unlinked before
+	 * it is freed, so the forward walk can't reach freed memory.
+	 */
+	list_for_each_entry(cc, &dm_crypt_wipe_list, wipe_list) {
+		crash_wipe_memzero(cc->key, cc->key_size);
+
+		if (cc->authenc_key)
+			crash_wipe_memzero(cc->authenc_key,
+					   crypt_authenckey_size(cc));
+
+		if (cc->key_string)
+			crash_wipe_memzero(cc->key_string,
+					   strlen(cc->key_string));
+
+		crypt_crash_wipe_iv(cc);
+	}
+
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block crypt_crash_wipe_nb = {
+	.notifier_call = crypt_crash_wipe
+};
+
+static void crypt_track_cc(struct crypt_config *cc)
+{
+	spin_lock(&dm_crypt_wipe_list_lock);
+	list_add(&cc->wipe_list, &dm_crypt_wipe_list);
+	spin_unlock(&dm_crypt_wipe_list_lock);
+}
+
+static void crypt_untrack_cc(struct crypt_config *cc)
+{
+	spin_lock(&dm_crypt_wipe_list_lock);
+	list_del(&cc->wipe_list);
+	spin_unlock(&dm_crypt_wipe_list_lock);
+}
+
+static void crypt_crash_wipe_init(void)
+{
+	crash_wipe_secrets_register(&crypt_crash_wipe_nb);
+}
+
+static void crypt_crash_wipe_exit(void)
+{
+	crash_wipe_secrets_unregister(&crypt_crash_wipe_nb);
+}
+#else
+static void crypt_track_cc(struct crypt_config *cc) { }
+static void crypt_untrack_cc(struct crypt_config *cc) { }
+static void crypt_crash_wipe_init(void) { }
+static void crypt_crash_wipe_exit(void) { }
+#endif /* CONFIG_CRASH_WIPE_SECRETS */
+
 static void crypt_calculate_pages_per_client(void)
 {
 	unsigned long pages = (totalram_pages() - totalhigh_pages()) * DM_CRYPT_MEMORY_PERCENT / 100;
@@ -2729,12 +2819,15 @@ static void crypt_dtr(struct dm_target *ti)
 		dm_put_device(ti, cc->dev);
 
 	kfree_sensitive(cc->cipher_string);
-	kfree_sensitive(cc->key_string);
+	kfree_sensitive(xchg(&cc->key_string, NULL));
 	kfree_sensitive(cc->cipher_auth);
-	kfree_sensitive(cc->authenc_key);
+	kfree_sensitive(xchg(&cc->authenc_key, NULL));
 
 	mutex_destroy(&cc->bio_alloc_lock);
 
+	memzero_explicit(cc->key, cc->key_size);
+	crypt_untrack_cc(cc);
+
 	/* Must zero key material before freeing */
 	kfree_sensitive(cc);
 
@@ -3210,6 +3303,8 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 
 	ti->private = cc;
 
+	crypt_track_cc(cc);
+
 	spin_lock(&dm_crypt_clients_lock);
 	dm_crypt_clients_n++;
 	crypt_calculate_pages_per_client();
@@ -3716,7 +3811,29 @@ static struct target_type crypt_target = {
 	.iterate_devices = crypt_iterate_devices,
 	.io_hints = crypt_io_hints,
 };
-module_dm(crypt);
+
+static int __init dm_crypt_init(void)
+{
+	int r;
+
+	crypt_crash_wipe_init();
+
+	r = dm_register_target(&crypt_target);
+	if (r) {
+		crypt_crash_wipe_exit();
+		return r;
+	}
+
+	return 0;
+}
+module_init(dm_crypt_init);
+
+static void __exit dm_crypt_exit(void)
+{
+	crypt_crash_wipe_exit();
+	dm_unregister_target(&crypt_target);
+}
+module_exit(dm_crypt_exit);
 
 MODULE_AUTHOR("Jana Saout <jana at saout.de>");
 MODULE_DESCRIPTION(DM_NAME " target for transparent encryption / decryption");

-- 
2.53.0



From ebiggers at kernel.org  Tue Aug 11 11:20:12 2026
From: ebiggers at kernel.org (Eric Biggers)
Date: Tue, 11 Aug 2026 18:20:12 +0000
Subject: [PATCH v2 00/13] CRASH_WIPE_SECRETS: Wipe secrets before kdump
 (was: CRASH_ZEROIZE)
In-Reply-To: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
Message-ID: <20260811182012.GB2895176@google.com>

On Tue, Aug 11, 2026 at 07:52:49PM +0200, Jan Sebastian G?tte wrote:
> I'm using linux on an embedded target in a Hardware Security Module-like
> application. One requirement is that I want the system to be able to
> quickly erase its memory when it detects physical tampering. I'm
> approaching that by using kdump to load into a small payload that
> instead of dumping RAM, erases RAM from start to end. However, writing
> all of RAM, especially on an embedded target, is rather slow. For this
> reason, I propose the mechanism in this patch series:
> 
> Add CONFIG_CRASH_ZEROIZE (default off), which when enabled makes various
> subsystems handling secret data do a quick, targeted wipe of these
> secrets before kdump. This behavior might also be interesting in cases
> where you run a normal kdump kernel but you still want to keep things
> like fde crypto keys out of these dumps.
> 
> CONFIG_CRASH_ZEROIZE is a best effort, defense in depth solution. There
> are circumstances, such as when a panic is triggered after memory
> corruption, or when a panic interrupts some operation that mutates data
> structures under locks, when the kernel cannot safely wipe some memory
> areas. The handlers proposed in this series will just print a warning
> and skip the affected areas in this case.
> 
> This series introduces handlers for the major locations I found where
> having this sort of thing makes sense. Notable omissions right now are
> the Ceph and CIFS subsystems. I have WIP patches for these, but since I
> can't easily test them right now, I omitted them from this patch set for
> now. Currently included locations are:
> 
> * various key types in security/keys
> * rxrpc
> * fscrypt
> * dm-crypt
> * crypto tfm instances
> * secretmem (which I'm going to start using in my application)
> 
> I've verified this patch series on an ARM64 target using the helper code
> at https://codeberg.org/yasec/crash-wipe-test . This code stuffs the
> affected kernel subsystems with keys and secret data, then crashes the
> system, takes a RAM dump and verifies the dump is clean of secrets. Note
> that the helper code is partially LLM-generated, so read with care. It
> passes a positive control test with the config option disabled.
> 
> The patch series applies on top of linux-next but should work on 7.0.0,
> too. I've tested the patches on a Arduino uno Q (Qualcomm QRB2210,
> ARM64) embedded target.

Can we not?  It's already hard enough to zeroize keys and data at the
normal end of their lifetime in the kernel: that's something that is
always a struggle, with fix patches regularly going by for many years
and many subsystems never fixed at all.  If we can barely even do that,
we aren't going to be able to correctly and completely implement and
maintain separate zeroization code for every kernel subsystem that runs
only on kernel panics and has special constraints, like not being able
to take locks.

This is never going to be done either, with the scope always wanting to
grow to include other keys and data.  You may think this series covers
"almost everything" but it's actually not even close.

Could you perhaps narrow the scope to one or two things that actually
are useful and can reasonably be supported for the application?  For
example, secretmem seems to be the only thing you mentioned that you're
actually planning to use.

- Eric


From contact at jaseg.de  Tue Aug 11 12:38:20 2026
From: contact at jaseg.de (=?UTF-8?Q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 11 Aug 2026 21:38:20 +0200
Subject: [PATCH v2 00/13] CRASH_WIPE_SECRETS: Wipe secrets before kdump
 (was: CRASH_ZEROIZE)
In-Reply-To: <20260811182012.GB2895176@google.com>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811182012.GB2895176@google.com>
Message-ID: <d242ab53-a39e-421d-ac39-4bee49be6ac0@jaseg.de>

On 8/11/26 20:20, Eric Biggers wrote:
> On Tue, Aug 11, 2026 at 07:52:49PM +0200, Jan Sebastian G?tte wrote:
>> I'm using linux on an embedded target in a Hardware Security Module-like
>> application. One requirement is that I want the system to be able to
>> quickly erase its memory when it detects physical tampering. I'm
>> approaching that by using kdump to load into a small payload that
>> instead of dumping RAM, erases RAM from start to end. However, writing
>> all of RAM, especially on an embedded target, is rather slow. For this
>> reason, I propose the mechanism in this patch series:
>>
>> Add CONFIG_CRASH_ZEROIZE (default off), which when enabled makes various
>> subsystems handling secret data do a quick, targeted wipe of these
>> secrets before kdump. This behavior might also be interesting in cases
>> where you run a normal kdump kernel but you still want to keep things
>> like fde crypto keys out of these dumps.
>>
>> CONFIG_CRASH_ZEROIZE is a best effort, defense in depth solution. There
>> are circumstances, such as when a panic is triggered after memory
>> corruption, or when a panic interrupts some operation that mutates data
>> structures under locks, when the kernel cannot safely wipe some memory
>> areas. The handlers proposed in this series will just print a warning
>> and skip the affected areas in this case.
>>
>> This series introduces handlers for the major locations I found where
>> having this sort of thing makes sense. Notable omissions right now are
>> the Ceph and CIFS subsystems. I have WIP patches for these, but since I
>> can't easily test them right now, I omitted them from this patch set for
>> now. Currently included locations are:
>>
>> * various key types in security/keys
>> * rxrpc
>> * fscrypt
>> * dm-crypt
>> * crypto tfm instances
>> * secretmem (which I'm going to start using in my application)
>>
>> I've verified this patch series on an ARM64 target using the helper code
>> at https://codeberg.org/yasec/crash-wipe-test . This code stuffs the
>> affected kernel subsystems with keys and secret data, then crashes the
>> system, takes a RAM dump and verifies the dump is clean of secrets. Note
>> that the helper code is partially LLM-generated, so read with care. It
>> passes a positive control test with the config option disabled.
>>
>> The patch series applies on top of linux-next but should work on 7.0.0,
>> too. I've tested the patches on a Arduino uno Q (Qualcomm QRB2210,
>> ARM64) embedded target.
> 
> Can we not?  It's already hard enough to zeroize keys and data at the
> normal end of their lifetime in the kernel: that's something that is
> always a struggle, with fix patches regularly going by for many years
> and many subsystems never fixed at all.  If we can barely even do that,
> we aren't going to be able to correctly and completely implement and
> maintain separate zeroization code for every kernel subsystem that runs
> only on kernel panics and has special constraints, like not being able
> to take locks.

These crash wipe code paths are less critical than places where you'd 
use memzero_explicit or kfree_sensitive. They are "only" a defense in 
depth / hardening measure. For most of these, if the crash happens at 
the exact time when data structures are being modified or when things 
are being intialized/deinitialized, they can skip things so they can 
never be 100% reliable anyway.

It's not realistic to clean *all* sensitive data in the first place, 
since that would include plaintext as well, which would mean cleaning 
the entire page cache which would be too complex, take too long, and 
destroy forensic evidence that people might want to keep intact for 
regular kdump uses.

> This is never going to be done either, with the scope always wanting to
> grow to include other keys and data.  You may think this series covers
> "almost everything" but it's actually not even close.

I'm sure there's always more. In my particular application, I'm using 
this not for complete coverage (for that I have another full memory wipe 
afterwards), but for speed - I want the obviously sensitive stuff like 
keys gone as fast as possible, with the full wipe cleaning up whatever 
the fast wipe here missed.

> Could you perhaps narrow the scope to one or two things that actually
> are useful and can reasonably be supported for the application?  For
> example, secretmem seems to be the only thing you mentioned that you're
> actually planning to use.

I'm open to whittling down the scope of this series. In my use case I 
need at least secretmem as well as dm-crypt, which then pulls in the 
crypto tfm stuff. Personally, I could live without the other bits, I 
just thought they'd be easy enough to include.

- Jan


From gmazyland at gmail.com  Tue Aug 11 13:18:27 2026
From: gmazyland at gmail.com (Milan Broz)
Date: Tue, 11 Aug 2026 22:18:27 +0200
Subject: [PATCH v2 13/13] dm crypt: wipe key material before kdump
In-Reply-To: <20260811-crash-zeroize-rework-v2-13-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811-crash-zeroize-rework-v2-13-9561d13c2340@jaseg.de>
Message-ID: <8654c2d4-8d9d-43f5-9256-adf2915209e9@gmail.com>

On 8/11/26 7:53 PM, Jan Sebastian G??tte wrote:
> Wipe volume key/iv copies kept by dm-crypt with
> CONFIG_CRASH_WIPE_SECRETS. The backend tfms are already handled
> separately.
> 
> Add a list tracking struct crypt_config instances when
> CONFIG_CRASH_WIPE_SECRETS is set. Structs are tracked here to avoid
> having to enumerate them through some roundabout way before kdump, when
> we can't safely take locks anymore.

Well, dm-crypt has crypt_wipe_key(), which can be called through a device-mapper message.
It also sets keys to zero in the crypto API.

Why do we need yet another way to wipe keys here, reimplementing everything twice?

I can imagine an emergency wrapper callback that will suspend dm-crypt and call existing code.

> Use custom wipe handlers even for things like ivs that have existing
> wipe functions elsewhere because we need to use crash_wipe_memzero
> instead of memzero_explicit. The crash_wipe helper memzero_explicit's
> the target buffers and flushes data caches. On ARM64, missing that cache
> flush could lead to the zeros not being written to DRAM before the kdump
> code turns off the data caches moments later.

Please no. It looks to me like you are trying to fix this on the wrong layer.
This way everyone will need their own memzero...

Dunno, but I really do not like dm-crypt becoming completely bloated with code
that has nothing to do with the original purpose of this driver.

Milan


> 
> Signed-off-by: Jan Sebastian G?tte <linux at jaseg.de>
> ---
>   drivers/md/dm-crypt.c | 151 ++++++++++++++++++++++++++++++++++++++++++++------
>   1 file changed, 134 insertions(+), 17 deletions(-)
> 


From ebiggers at kernel.org  Tue Aug 11 13:41:01 2026
From: ebiggers at kernel.org (Eric Biggers)
Date: Tue, 11 Aug 2026 13:41:01 -0700
Subject: [PATCH v2 00/13] CRASH_WIPE_SECRETS: Wipe secrets before kdump
 (was: CRASH_ZEROIZE)
In-Reply-To: <d242ab53-a39e-421d-ac39-4bee49be6ac0@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811182012.GB2895176@google.com>
 <d242ab53-a39e-421d-ac39-4bee49be6ac0@jaseg.de>
Message-ID: <20260811204101.GA1905@sol>

On Tue, Aug 11, 2026 at 09:38:20PM +0200, Jan Sebastian G?tte wrote:
> > Could you perhaps narrow the scope to one or two things that actually
> > are useful and can reasonably be supported for the application?  For
> > example, secretmem seems to be the only thing you mentioned that you're
> > actually planning to use.
> 
> I'm open to whittling down the scope of this series. In my use case I need
> at least secretmem as well as dm-crypt, which then pulls in the crypto tfm
> stuff. Personally, I could live without the other bits, I just thought
> they'd be easy enough to include.

Please do reduce the scope.

Note that even if dm-crypt is in scope, "crypto: api - wipe tfm contexts
before kdump" is neither required nor sufficient.  For one, dm-crypt has
its own copy of the key.  The wiping should be done by the kernel
feature(s) that need it, as they know what actually needs to be wiped,
and they can do so without the bottleneck of a new global list.

- Eric


From baoquan.he at linux.dev  Wed Aug 12 02:30:08 2026
From: baoquan.he at linux.dev (Baoquan He)
Date: Wed, 12 Aug 2026 17:30:08 +0800
Subject: [PATCH v2 01/13] kexec: add CRASH_WIPE_SECRETS to wipe secrets
 before kdump
In-Reply-To: <20260811-crash-zeroize-rework-v2-1-9561d13c2340@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811-crash-zeroize-rework-v2-1-9561d13c2340@jaseg.de>
Message-ID: <anw9ILJiO5xIjP4f@MiWiFi-R3L-srv>

On 08/11/26 at 07:52pm, Jan Sebastian G?tte wrote:
.....snip...
> @@ -142,6 +170,7 @@ void __noclone __crash_kexec(struct pt_regs *regs)
>  			crash_save_vmcoreinfo();
>  			machine_crash_shutdown(&fixed_regs);
>  			crash_cma_clear_pending_dma();
> +			crash_wipe_secrets();

I remember when crash_cma added this, we argued a lot about the risk,
finally it's added in with an added note to tell it's an opt-in feature
and risky. I don't know if it opens a Pandora box, I think anything
wanted being here need prove anything else has been tried, and the thing
is very very important. If we don't guard the door well, one day one guy
cold come and said I need ask Oracle database to do some transactions
before juming into kdump kernel because the actions will save billions
of dollar. 

And what does security people say about this wiping? Is it very
important information and wiping is mandotary? There's no any other way
to wipe it or pollute it, no way in kdump kernel? I noticed it's
happening on en embedded target, is it possible you do this on your
embedded kernel tree, but not mainline tree?

Anyone who wants to do thing like this, please prepare answers for them,
and be ready to accept failure.

Thanks
Baoquan

>  			machine_kexec(kexec_crash_image);
>  		}
>  		kexec_unlock();
> 
> -- 
> 2.53.0
> 


From contact at jaseg.de  Wed Aug 12 02:56:59 2026
From: contact at jaseg.de (=?UTF-8?Q?Jan_Sebastian_G=C3=B6tte?=)
Date: Wed, 12 Aug 2026 11:56:59 +0200
Subject: [PATCH v2 13/13] dm crypt: wipe key material before kdump
In-Reply-To: <8654c2d4-8d9d-43f5-9256-adf2915209e9@gmail.com>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811-crash-zeroize-rework-v2-13-9561d13c2340@jaseg.de>
 <8654c2d4-8d9d-43f5-9256-adf2915209e9@gmail.com>
Message-ID: <c5376b88-cc68-420a-ad8f-d494db9c7b1b@jaseg.de>

On 8/11/26 22:18, Milan Broz wrote:
> On 8/11/26 7:53 PM, Jan Sebastian G??tte wrote:
>> Wipe volume key/iv copies kept by dm-crypt with
>> CONFIG_CRASH_WIPE_SECRETS. The backend tfms are already handled
>> separately.
>>
>> Add a list tracking struct crypt_config instances when
>> CONFIG_CRASH_WIPE_SECRETS is set. Structs are tracked here to avoid
>> having to enumerate them through some roundabout way before kdump, when
>> we can't safely take locks anymore.
> 
> Well, dm-crypt has crypt_wipe_key(), which can be called through a 
> device-mapper message.
> It also sets keys to zero in the crypto API.
> 
> Why do we need yet another way to wipe keys here, reimplementing 
> everything twice?
> 
> I can imagine an emergency wrapper callback that will suspend dm-crypt 
> and call existing code.

I originally decided I'd keep these function separate since you can't 
rely on memory allocation/freeing to work during panic. crypt_wipe_key 
currently calls kfree_sensitive, and inside crypto_*_setkey there's also 
kalloc/kfree calls hiding.

I could rework the patch to call into crypt_wipe_key, but I'd have to 
make that avoid memory allocation/freeing. The direct kfree_sensitive 
call can be replaced with a memzero_explicit, but I think I'd have to 
add a dedicated "wipe without allocations" function to the crypto 
backends as an alternative to setkey with a zero key.

>> Use custom wipe handlers even for things like ivs that have existing
>> wipe functions elsewhere because we need to use crash_wipe_memzero
>> instead of memzero_explicit. The crash_wipe helper memzero_explicit's
>> the target buffers and flushes data caches. On ARM64, missing that cache
>> flush could lead to the zeros not being written to DRAM before the kdump
>> code turns off the data caches moments later.
> 
> Please no. It looks to me like you are trying to fix this on the wrong 
> layer.
> This way everyone will need their own memzero...

You're probably right. I'll remove this from the next version and make 
sure the caches are flushed properly during kexec instead.

 > Dunno, but I really do not like dm-crypt becoming completely bloated 
 > with code
> that has nothing to do with the original purpose of this driver.

I feel like "delete key quick" is a pretty normal function for crypto code.


From contact at jaseg.de  Wed Aug 12 03:08:39 2026
From: contact at jaseg.de (=?UTF-8?Q?Jan_Sebastian_G=C3=B6tte?=)
Date: Wed, 12 Aug 2026 12:08:39 +0200
Subject: [PATCH v2 01/13] kexec: add CRASH_WIPE_SECRETS to wipe secrets
 before kdump
In-Reply-To: <anw9ILJiO5xIjP4f@MiWiFi-R3L-srv>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811-crash-zeroize-rework-v2-1-9561d13c2340@jaseg.de>
 <anw9ILJiO5xIjP4f@MiWiFi-R3L-srv>
Message-ID: <9765f7ea-36bb-4c7b-a19b-7facec97b762@jaseg.de>

On 8/12/26 11:30, Baoquan He wrote:
> On 08/11/26 at 07:52pm, Jan Sebastian G?tte wrote:
> .....snip...
>> @@ -142,6 +170,7 @@ void __noclone __crash_kexec(struct pt_regs *regs)
>>   			crash_save_vmcoreinfo();
>>   			machine_crash_shutdown(&fixed_regs);
>>   			crash_cma_clear_pending_dma();
>> +			crash_wipe_secrets();
> 
> I remember when crash_cma added this, we argued a lot about the risk,
> finally it's added in with an added note to tell it's an opt-in feature
> and risky. I don't know if it opens a Pandora box, I think anything
> wanted being here need prove anything else has been tried, and the thing
> is very very important. If we don't guard the door well, one day one guy
> cold come and said I need ask Oracle database to do some transactions
> before juming into kdump kernel because the actions will save billions
> of dollar.

I see your point.

> And what does security people say about this wiping? Is it very
> important information and wiping is mandotary? There's no any other way
> to wipe it or pollute it, no way in kdump kernel?

I'm doing both - my kdump kernel does a linear memzero of all of DRAM. 
However, that's too slow - it takes several hundred milliseconds, and I 
want these cryptographic keys gone within no more than 10 ms.

Doing this in the original kernel is cleanest I think, because to do a 
targeted erase in the kdump kernel, that kernel would have to know all 
the original kernel's symbols, and it would have to forensically 
reconstruct it's memory content similar to the crash utility. I think 
that's too complex and error-prone to do in this use case.

I can imagine one alternative way to approach this, tell me what you 
think: Instead of registering wipe handlers that memzero places on 
panic, I could put an optional registry of (addr, len) descriptors into 
crashkernel memory that the original kernel populates with the PAs of 
buffers to clear ahead of time. Then the kdump kernel could do the 
actual memzero. This would remove all code from the actual kdump path 
here. The registry could be made per-core to avoid locks.

> I noticed it's
> happening on en embedded target, is it possible you do this on your
> embedded kernel tree, but not mainline tree?

I could keep it in a vendor tree, but I think the functionality is 
useful beyond my particular use case.


From dhowells at redhat.com  Wed Aug 12 04:01:16 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:16 +0100
Subject: [PATCH net v7 01/11] rxrpc: Fix sendmsg to not return an error if last packet queued
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-2-dhowells@redhat.com>

Fix AF_RXRPC sendmsg() so that it doesn't return an error if it has
successfully queued the last packet of a call, but the call has seen to
have completed after it did that.  Rather, leave it to recvmsg() to report
the completion (which it will do anyway).

The problem with trying to report the error twice is that the caller may
try to clean up the dead call twice.

Fixes: d41b3f5b9688 ("rxrpc: Wrap accesses to get call state to put the barrier in one place")
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
---
 net/rxrpc/sendmsg.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index ed2c9a51005a..4c754f78ece9 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -453,9 +453,6 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 
 success:
 	ret = copied;
-	if (rxrpc_call_is_complete(call) &&
-	    call->error < 0)
-		ret = call->error;
 out:
 	call->tx_pending = txb;
 	_leave(" = %d", ret);
@@ -467,8 +464,14 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	return call->error;
 
 maybe_error:
-	if (copied)
+	if (copied) {
+		if (rxrpc_call_is_complete(call) &&
+		    call->error < 0) {
+			ret = call->error;
+			goto out;
+		}
 		goto success;
+	}
 	goto out;
 
 efault:



From dhowells at redhat.com  Wed Aug 12 04:01:15 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:15 +0100
Subject: [PATCH net v7 00/11] rxrpc: Fix CHALLENGE packet handling
Message-ID: <20260812110129.979970-1-dhowells@redhat.com>

Here's a fix for AF_RXRPC's CHALLENGE packet handling, addressing an issue
raised by Sashiko[1], plus some miscellaneous fixes found in the process of
fixing this, plus a number of things raised by Sashiko[2][3][4][5][6][7].

Firstly, the miscellaneous patches:

 (1) Fix rxrpc_sendmsg so that it doesn't return an error if it queued the
     last packet of a call.  After that point, the error will be returned
     by recvmsg() and returned it twice in two different places may
     complicate userspace cleaning up its own structures.

 (2) Fix error handling in rxrpc_send_data() for if ->secure_packet()
     returns an error.

 (3) Fix the update of call->pending in rxrpc_send_data() in paths when the
     call lock has been dropped.

 (4) Fix the generation of notifications from rxrpc after call completion.

And then there are the patches to fix CHALLENGE packet overqueuing and
simplify RESPONSE packet generation by pre-creating the RxGK application
data up front and passing it in a user key (thereby allowing userspace to
partake).  This is split into five patches:

 (5) Expand the abort trace enum to be larger than a signed char as the
     number of elements will exceed 128.

 (6) Add a refcount to the user key payload.

 (7) Make the AFS filesystem generate per-server appdata keys.

 (8) Pass the appdata from AFS (or userspace) to rxrpc.

 (9) Change over to using the appdata key to supply the appdata.

(10) Remove all the OOB stuff.

[!] Note that this entails a significant change in the UAPI for AF_RXRPC,
with the CMSG types and sockopt to support the OOB queuing being removed
and replaced with a new single CMSG type that conveys the user key ID.  I
don't think it likely anyone is using this outside of my kafs-utils
package.

This also involves a change to the user-defined key type, making the
payload refcounted so that it can be accessed and the length read, then a
buffer allocated that will hold it and other data, and then the content
copied.  The problem is that the user is perfectly at liberty to change the
content of a user-defined key (which will RCU-replace the content of the
key), so the length might change when we drop the RCU read lock in order to
allocate.  This could be got around by locking the key->rwsem sharedly, but
that might be able to deadlock part of the rxrpc protocol engine if memory
reclaim occurs.

David

The patches can be found here also:

	http://git.kernel.org/cgit/linux/kernel/git/dhowells/linux-fs.git/log/?h=rxrpc-fixes

Changes
=======
ver #7)
- Rebased on latest net/main.
- Added a patch to change rxrpc_send_data() to use len rather than msg_iter
  count to be consistent about the amount to send so as to do the LAST flag
  determination correctly.
- Fixed more Sashiko-reported bugs[7]:
  - Made the loops in afs_make_call() that call rxrpc_kernel_send_data()
    pass the amount left in the iterator rather than an unreducing size.
  - Made the second loop in afs_make_call() check to see if
    rxrpc_kernel_send_data() returned an error.
  - Removed yet more OOB references, two in linux/af_rxrpc.h and one in
    rxrpc.rst.

ver #6)
- Rebased on latest net/main.
- Fixed more Sashiko-reported bugs[6]:
  - Fixed rxrpc_send_data() to redo the RXRPC_CALL_TX_ERROR and the
    RXRPC_CALL_TX_NO_MORE checks after having dropped the call mutex.
  - Altered rxrpc_send_data(), as discussed with Paulo Abeni, to rewind as
    much as possible on retryable crypto error (e.g. ENOMEM) rather than
    rewinding just one byte.
  - Fixed afs_make_call() to keep trying rxrpc_kernel_send_data() until the
    iterator is drained unless an error occurs.
  - Fixed afs_create_yfs_rxgk_cm_appdata() to use kfree_sensitive().
  - Removed remaining OOB trace constants.

ver #5)
- Rebased on latest net/main.
- Fixed more Sashiko-reported bugs[5]:
  - Removed dropped_lock from rxrpc_do_sendmsg() as it's now always false.
  - Fix rxrpc_send_data() to not leak a txbuf from the "maybe_error:" path
    by reattaching it to call->tx_pending.
  - Increased the size of the rxrpc_abort_reason enum value by removing the
    __mode(byte) specifier.

ver #4)
- Rebased on latest net/main.
- Split out non-relevant AFS patches.
- Allow logon-type key as well as user-type key as they're basically the
  same thing internally.
- Fixed more Sashiko-reported bugs[4]:
  - Fixed rxrpc_send_data() to wind the transmitted data back by 1 byte if
    ENOMEM is hit when encrypting the final packet so that the caller can
    retry.
  - Fixed afs_create_yfs_rxgk_cm_appdata() to add the 4 bytes for the level
    into toksize.
  - Changed bundle code to include the appdata key as part of the client
    connection bundle lookup criteria (don't share connections with
    different appdata keys).
  - Fixed rxrpc_sendmsg_cmsg() to allow only, not disallow user-type keys
    for RXRPC_RESPONSE_APPDATA.
  - Reversed the removal of the rejection of MSG_OOB passed to
    rxrpc_recvmsg().
  - Fixed rxgk_construct_response() to use xdr_object_len() to calculate
    the space needed for the appdata and also the space needed for the
    token and the authenticator.
  - Added a check into rxgk_respond_to_challenge() to make sure the key
    type is user or login before we access its payload.
  - Remove ->sendmsg_respond_to_challenge() too.

ver #3)
- Rebased on latest net/main.
  - Removed two obsoleted patches.

ver #2)
- Split the CHALLENGE/RESPONSE fix into smaller patches.
- Fixed more Sashiko-reported bugs[2][3]:
  - Added some more patches to fix some more bugs.
  - Get rid of the AFS_SERVER_FL_APPDATA flag and check the pointer to the
    appdata instead.
  - Rename the appdata key pointer in the AFS_SERVER to reflect this one is
    only for the YFS-RxGK security class.
  - Use barriers when reading or writing the server appdata key pointer.
  - Ignore the appdata for RxNULL, RxKAD and OpenAFS's RxGK for now.
  - Check that sendmsg() with RXRPC_RESPONSE_APPDATA is passed a user key.
  - Check that the appdata key's payload isn't NULL, for instance if it
    gets revoked.
  - Add some error path key_put()s in rxrpc_do_sendmsg().

[1] https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
[2] https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
[3] https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
[4] https://sashiko.dev/#/patchset/20260713081022.2186481-1-dhowells%40redhat.com
[5] https://sashiko.dev/#/patchset/20260723100309.530157-1-dhowells%40redhat.com
[6] https://sashiko.dev/#/patchset/20260729160108.2031453-1-dhowells%40redhat.com
[7] https://sashiko.dev/#/patchset/20260804172639.2844491-1-dhowells%40redhat.com

David Howells (11):
  rxrpc: Fix sendmsg to not return an error if last packet queued
  rxrpc: Fix sendmsg length
  rxrpc: Fix packet encryption error handling
  rxrpc: Fix update of call->tx_pending without holding lock
  rxrpc: Fix generation of notifications after call completion
  rxrpc: Expand abort trace enum
  keys: Add refcounting to user-defined key type payload
  afs: Create a server appdata key
  rxrpc: Pass appdata key to rxrpc_call and thence to rxrpc_bundle
  rxrpc: Fix CHALLENGE packet overqueuing and simplify RESPONSE
    generation
  rxrpc: Remove OOB challenge/response code

 Documentation/networking/rxrpc.rst |   1 -
 fs/afs/cm_security.c               | 315 +++++++++++------------
 fs/afs/fs_probe.c                  |   5 +
 fs/afs/internal.h                  |  38 +--
 fs/afs/main.c                      |   1 -
 fs/afs/rxrpc.c                     |  64 ++---
 fs/afs/server.c                    |   2 +-
 include/keys/user-type.h           |   2 +
 include/net/af_rxrpc.h             |  26 +-
 include/trace/events/afs.h         |   1 +
 include/trace/events/rxrpc.h       |  15 +-
 include/uapi/linux/rxrpc.h         |   6 +-
 net/dns_resolver/dns_key.c         |   1 +
 net/rxrpc/Makefile                 |   1 -
 net/rxrpc/af_rxrpc.c               |  49 +---
 net/rxrpc/ar-internal.h            |  27 +-
 net/rxrpc/call_object.c            |   2 +
 net/rxrpc/call_state.c             |  57 ++++-
 net/rxrpc/conn_client.c            |   4 +
 net/rxrpc/conn_event.c             |  70 +-----
 net/rxrpc/insecure.c               |   7 -
 net/rxrpc/key.c                    |  37 +++
 net/rxrpc/oob.c                    | 387 -----------------------------
 net/rxrpc/proc.c                   |   5 +-
 net/rxrpc/recvmsg.c                | 124 ++-------
 net/rxrpc/rxgk.c                   | 138 ++++------
 net/rxrpc/rxkad.c                  |  27 --
 net/rxrpc/sendmsg.c                | 157 ++++++++----
 net/rxrpc/server_key.c             |  40 ---
 security/keys/user_defined.c       |  23 +-
 30 files changed, 518 insertions(+), 1114 deletions(-)
 delete mode 100644 net/rxrpc/oob.c



From dhowells at redhat.com  Wed Aug 12 04:01:17 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:17 +0100
Subject: [PATCH net v7 02/11] rxrpc: Fix sendmsg length
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-3-dhowells@redhat.com>

rxrpc_send_data() is given two data lengths (len and msg->msg_iter.count)
and is inconsistent about how it uses them.  Fix this by using len in
preference to msg->msg_iter.count.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
Fixes: 382d7974de31 ("RxRPC: Use iov_iter_count() in rxrpc_send_data() instead of the len argument")
---
 net/rxrpc/sendmsg.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index 4c754f78ece9..626ef5778691 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -379,9 +379,9 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 
 	ret = -EMSGSIZE;
 	if (call->tx_total_len != -1) {
-		if (len - copied > call->tx_total_len)
+		if (len > call->tx_total_len)
 			goto maybe_error;
-		if (!more && len - copied != call->tx_total_len)
+		if (!more && len != call->tx_total_len)
 			goto maybe_error;
 	}
 
@@ -405,7 +405,7 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 			 * the security header is going to be in the padded
 			 * region (enc blocksize), but the trailer is not.
 			 */
-			remain = more ? INT_MAX : msg_data_left(msg);
+			remain = more ? INT_MAX : len;
 			txb = call->conn->security->alloc_txbuf(call, remain, sk->sk_allocation);
 			if (!txb) {
 				ret = -ENOMEM;
@@ -416,8 +416,8 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 		_debug("append");
 
 		/* append next segment of data to the current buffer */
-		if (msg_data_left(msg) > 0) {
-			size_t copy = umin(txb->space, msg_data_left(msg));
+		if (len > 0) {
+			size_t copy = umin(txb->space, len);
 
 			_debug("add %zu", copy);
 			if (!copy_from_iter_full(txb->data + txb->offset,
@@ -428,6 +428,7 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 			txb->len += copy;
 			txb->offset += copy;
 			copied += copy;
+			len -= copy;
 			if (call->tx_total_len != -1)
 				call->tx_total_len -= copy;
 		}
@@ -439,8 +440,8 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 
 		/* add the packet to the send queue if it's now full */
 		if (!txb->space ||
-		    (msg_data_left(msg) == 0 && !more)) {
-			if (msg_data_left(msg) == 0 && !more)
+		    (len == 0 && !more)) {
+			if (len == 0 && !more)
 				txb->flags |= RXRPC_LAST_PACKET;
 
 			ret = call->security->secure_packet(call, txb);
@@ -449,7 +450,7 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 			rxrpc_queue_packet(rx, call, txb, notify_end_tx);
 			txb = NULL;
 		}
-	} while (msg_data_left(msg) > 0);
+	} while (len > 0);
 
 success:
 	ret = copied;



From dhowells at redhat.com  Wed Aug 12 04:01:18 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:18 +0100
Subject: [PATCH net v7 03/11] rxrpc: Fix packet encryption error handling
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-4-dhowells@redhat.com>

In rxrpc_send_data(), if ->secure_packet() returns an error, the code
currently just jumps to out: and returns the error to the app on the
assumption that any error returned by this is automatically fatal for the
call, and may even have corrupted the transmission queue - but leaving it
to userspace to deal with.  Nothing stops the application from retrying the
sendmsg(), which will try to encrypt the buffer again, and might succeed
with a corrupt buffer.

Fix rxrpc_send_data() in the following ways:

 (1) If -ENOMEM is returned, assume we never got as far as the encryption
     and that the operation is retryable.  In which case, jump to
     maybe_error_rewind and, if we've copied data into the last packet,
     remove a byte from it so that we don't tell the caller that we've
     completed the transmission phase.

 (2) If any other error occurs, set the TX_ERROR flag on the call and
     return that error directly; on all subsequent attempts to add data to
     the call, return -EIO.  The app must then abort the call to get rid of
     it (this allows the app to choose the abort code to use).

Fixes: 17926a79320a ("[AF_RXRPC]: Provide secure RxRPC sockets for use by userspace and kernel both")
Closes: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
---
 fs/afs/rxrpc.c               | 24 ++++++++++------
 include/trace/events/rxrpc.h |  1 +
 net/rxrpc/ar-internal.h      |  1 +
 net/rxrpc/sendmsg.c          | 53 ++++++++++++++++++++++++++++--------
 4 files changed, 59 insertions(+), 20 deletions(-)

diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index d82916657a3d..f3552a1bbc03 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -412,21 +412,27 @@ void afs_make_call(struct afs_call *call, gfp_t gfp)
 	msg.msg_controllen	= 0;
 	msg.msg_flags		= MSG_WAITALL | (call->write_iter ? MSG_MORE : 0);
 
-	ret = rxrpc_kernel_send_data(call->net->socket, rxcall,
-				     &msg, call->request_size,
-				     afs_notify_end_request_tx);
-	if (ret < 0)
-		goto error_do_abort;
+	do {
+		ret = rxrpc_kernel_send_data(call->net->socket, rxcall, &msg,
+					     msg_data_left(&msg),
+					     afs_notify_end_request_tx);
+		if (ret < 0)
+			goto error_do_abort;
+	} while (msg_data_left(&msg) > 0);
 
 	if (call->write_iter) {
 		msg.msg_iter = *call->write_iter;
 		msg.msg_flags &= ~MSG_MORE;
 		trace_afs_send_data(call, &msg);
 
-		ret = rxrpc_kernel_send_data(call->net->socket,
-					     call->rxcall, &msg,
-					     iov_iter_count(&msg.msg_iter),
-					     afs_notify_end_request_tx);
+		do {
+			ret = rxrpc_kernel_send_data(call->net->socket,
+						     call->rxcall, &msg,
+						     msg_data_left(&msg),
+						     afs_notify_end_request_tx);
+			if (ret < 0)
+				goto error_do_abort;
+		} while (msg_data_left(&msg) > 0);
 		*call->write_iter = msg.msg_iter;
 
 		trace_afs_sent_data(call, &msg, ret);
diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index 704a10de6670..8f3e3967885a 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -148,6 +148,7 @@
 	EM(rxrpc_eproto_wrong_security,		"wrong-sec")		\
 	EM(rxrpc_recvmsg_excess_data,		"recvmsg-excess")	\
 	EM(rxrpc_recvmsg_short_data,		"recvmsg-short")	\
+	EM(rxrpc_sendmsg_tx_error,		"tx-error")		\
 	E_(rxrpc_sendmsg_late_send,		"sendmsg-late")
 
 #define rxrpc_call_poke_traces \
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 865f05fe37ab..a6f830c1621f 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -642,6 +642,7 @@ enum rxrpc_call_flag {
 	RXRPC_CALL_TX_LAST,		/* Last packet in Tx buffer (at rxtx_top) */
 	RXRPC_CALL_TX_ALL_ACKED,	/* Last packet has been hard-acked */
 	RXRPC_CALL_TX_NO_MORE,		/* No more data to transmit (MSG_MORE deasserted) */
+	RXRPC_CALL_TX_ERROR,		/* Terminal error; call needs abort */
 	RXRPC_CALL_SEND_PING,		/* A ping will need to be sent */
 	RXRPC_CALL_RETRANS_TIMEOUT,	/* Retransmission due to timeout occurred */
 	RXRPC_CALL_BEGAN_RX_TIMER,	/* We began the expect_rx_by timer */
diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index 626ef5778691..7932442f33fb 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -330,13 +330,6 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	bool more = msg->msg_flags & MSG_MORE;
 	int ret, copied = 0;
 
-	if (test_bit(RXRPC_CALL_TX_NO_MORE, &call->flags)) {
-		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_late_send,
-				  call->cid, call->call_id, call->rx_consumed,
-				  0, -EPROTO);
-		return -EPROTO;
-	}
-
 	timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
 
 	ret = rxrpc_wait_to_be_connected(call, &timeo);
@@ -353,6 +346,19 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk);
 
 reload:
+	if (unlikely(test_bit(RXRPC_CALL_TX_NO_MORE, &call->flags))) {
+		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_late_send,
+				  call->cid, call->call_id, call->rx_consumed,
+				  0, -EPROTO);
+		return -EPROTO;
+	}
+	if (unlikely(test_bit(RXRPC_CALL_TX_ERROR, &call->flags))) {
+		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_tx_error,
+				  call->cid, call->call_id, call->rx_consumed,
+				  0, -EIO);
+		return -EIO;
+	}
+
 	txb = call->tx_pending;
 	call->tx_pending = NULL;
 	if (txb)
@@ -441,12 +447,21 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 		/* add the packet to the send queue if it's now full */
 		if (!txb->space ||
 		    (len == 0 && !more)) {
-			if (len == 0 && !more)
-				txb->flags |= RXRPC_LAST_PACKET;
-
+			/* Do any required crypto.  If this fails, it could
+			 * have corrupted the txbuf content with a partial
+			 * encrypt.  Assume that ENOMEM is retryable, but
+			 * everything else is terminal.
+			 */
 			ret = call->security->secure_packet(call, txb);
-			if (ret < 0)
+			if (ret < 0) {
+				if (ret == -ENOMEM)
+					goto maybe_error_rewind;
+				set_bit(RXRPC_CALL_TX_ERROR, &call->flags);
 				goto out;
+			}
+
+			if (len == 0 && !more)
+				txb->flags |= RXRPC_LAST_PACKET;
 			rxrpc_queue_packet(rx, call, txb, notify_end_tx);
 			txb = NULL;
 		}
@@ -464,6 +479,22 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 	_leave(" = %d", call->error);
 	return call->error;
 
+maybe_error_rewind:
+	/* If we got a retryable error after copying all the supplied data into
+	 * the last packet, we need to rewind as much as we can so the caller
+	 * knows they need to retry the sendmsg.
+	 */
+	if (copied && !more && !len) {
+		unsigned int rewind_by = umin(copied, txb->len);
+
+		txb->space  += rewind_by;
+		txb->len    -= rewind_by;
+		txb->offset -= rewind_by;
+		copied      -= rewind_by;
+		if (call->tx_total_len != -1)
+			call->tx_total_len += rewind_by;
+		iov_iter_revert(&msg->msg_iter, rewind_by);
+	}
 maybe_error:
 	if (copied) {
 		if (rxrpc_call_is_complete(call) &&



From dhowells at redhat.com  Wed Aug 12 04:01:19 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:19 +0100
Subject: [PATCH net v7 04/11] rxrpc: Fix update of call->tx_pending without holding lock
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-5-dhowells@redhat.com>

Currently, rxrpc_send_data() updates call->tx_pending just before it
returns - but it won't be holding the call->lock when it does this if a
wait was interrupted by a signal.  This would allow a parallel sendmsg() to
race.

Further, both the callers of rxrpc_send_data() call it with the lock held,
and then it returns an indication through the parameter list to say whether
it has dropped the lock or not - after which the callers both just drop the
lock if it's still held.

Fix this by:

 (1) Moving the release of call->lock down into rxrpc_send_data() and get
     rid of the indicator parameter.  This makes it easier to see where the
     lock is held.

 (2) Make the wait_for_space path move the value in txb back into
     call->tx_pending before dropping the lock prior to the wait.

 (3) After waiting, if the attempt to reacquire the mutex is interrupted,
     just return directly there rather than going to out_unlock

Fixes: b0f571ecd794 ("rxrpc: Fix locking in rxrpc's sendmsg")
Closes: https://sashiko.dev/#/patchset/20260702144919.172295-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
---
 net/rxrpc/sendmsg.c | 56 ++++++++++++++++++++++++---------------------
 1 file changed, 30 insertions(+), 26 deletions(-)

diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index 7932442f33fb..1cfc03db78f9 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -320,8 +320,8 @@ static int rxrpc_alloc_txqueue(struct sock *sk, struct rxrpc_call *call)
 static int rxrpc_send_data(struct rxrpc_sock *rx,
 			   struct rxrpc_call *call,
 			   struct msghdr *msg, size_t len,
-			   rxrpc_notify_end_tx_t notify_end_tx,
-			   bool *_dropped_lock)
+			   rxrpc_notify_end_tx_t notify_end_tx)
+	__releases(&call->user_mutex)
 {
 	struct rxrpc_txbuf *txb;
 	struct sock *sk = &rx->sk;
@@ -334,12 +334,12 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 
 	ret = rxrpc_wait_to_be_connected(call, &timeo);
 	if (ret < 0)
-		return ret;
+		goto out_unlock;
 
 	if (call->conn->state == RXRPC_CONN_CLIENT_UNSECURED) {
 		ret = rxrpc_init_client_conn_security(call->conn);
 		if (ret < 0)
-			return ret;
+			goto out_unlock;
 	}
 
 	/* this should be in poll */
@@ -350,13 +350,15 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_late_send,
 				  call->cid, call->call_id, call->rx_consumed,
 				  0, -EPROTO);
-		return -EPROTO;
+		ret = -EPROTO;
+		goto out_unlock;
 	}
 	if (unlikely(test_bit(RXRPC_CALL_TX_ERROR, &call->flags))) {
 		trace_rxrpc_abort(call->debug_id, rxrpc_sendmsg_tx_error,
 				  call->cid, call->call_id, call->rx_consumed,
 				  0, -EIO);
-		return -EIO;
+		ret = -EIO;
+		goto out_unlock;
 	}
 
 	txb = call->tx_pending;
@@ -457,7 +459,7 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 				if (ret == -ENOMEM)
 					goto maybe_error_rewind;
 				set_bit(RXRPC_CALL_TX_ERROR, &call->flags);
-				goto out;
+				goto out_txb;
 			}
 
 			if (len == 0 && !more)
@@ -469,15 +471,18 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 
 success:
 	ret = copied;
-out:
+out_txb:
 	call->tx_pending = txb;
+out_unlock:
+	mutex_unlock(&call->user_mutex);
 	_leave(" = %d", ret);
 	return ret;
 
 call_terminated:
 	rxrpc_put_txbuf(txb, rxrpc_txbuf_put_send_aborted);
-	_leave(" = %d", call->error);
-	return call->error;
+	call->tx_pending = NULL;
+	ret = call->error;
+	goto out_unlock;
 
 maybe_error_rewind:
 	/* If we got a retryable error after copying all the supplied data into
@@ -500,36 +505,40 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 		if (rxrpc_call_is_complete(call) &&
 		    call->error < 0) {
 			ret = call->error;
-			goto out;
+			goto out_txb;
 		}
 		goto success;
 	}
-	goto out;
+	goto out_txb;
 
 efault:
 	ret = -EFAULT;
-	goto out;
+	goto out_txb;
 
 wait_for_space:
 	ret = -EAGAIN;
 	if (msg->msg_flags & MSG_DONTWAIT)
 		goto maybe_error;
+	call->tx_pending = txb;
+	txb = NULL;
 	mutex_unlock(&call->user_mutex);
-	*_dropped_lock = true;
+
 	ret = rxrpc_wait_for_tx_window(rx, call, &timeo,
 				       msg->msg_flags & MSG_WAITALL);
 	if (ret < 0)
-		goto maybe_error;
+		goto out_nolock;
 	if (call->interruptibility == RXRPC_INTERRUPTIBLE) {
 		if (mutex_lock_interruptible(&call->user_mutex) < 0) {
 			ret = sock_intr_errno(timeo);
-			goto maybe_error;
+			goto out_nolock;
 		}
 	} else {
 		mutex_lock(&call->user_mutex);
 	}
-	*_dropped_lock = false;
 	goto reload;
+out_nolock:
+	_leave(" = %d [intr]", ret);
+	return copied ?: ret;
 }
 
 /*
@@ -695,7 +704,6 @@ rxrpc_new_client_call_for_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg,
 int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
 {
 	struct rxrpc_call *call;
-	bool dropped_lock = false;
 	int ret;
 
 	struct rxrpc_send_params p = {
@@ -804,16 +812,15 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
 		ret = 0;
 		break;
 	case RXRPC_CMD_SEND_DATA:
-		ret = rxrpc_send_data(rx, call, msg, len, NULL, &dropped_lock);
-		break;
+		ret = rxrpc_send_data(rx, call, msg, len, NULL);
+		goto error_put;
 	default:
 		ret = -EINVAL;
 		break;
 	}
 
 out_put_unlock:
-	if (!dropped_lock)
-		mutex_unlock(&call->user_mutex);
+	mutex_unlock(&call->user_mutex);
 error_put:
 	rxrpc_put_call(call, rxrpc_call_put_sendmsg);
 	_leave(" = %d", ret);
@@ -843,7 +850,6 @@ int rxrpc_kernel_send_data(struct socket *sock, struct rxrpc_call *call,
 			   struct msghdr *msg, size_t len,
 			   rxrpc_notify_end_tx_t notify_end_tx)
 {
-	bool dropped_lock = false;
 	int ret;
 
 	_enter("{%d},", call->debug_id);
@@ -854,12 +860,10 @@ int rxrpc_kernel_send_data(struct socket *sock, struct rxrpc_call *call,
 	mutex_lock(&call->user_mutex);
 
 	ret = rxrpc_send_data(rxrpc_sk(sock->sk), call, msg, len,
-			      notify_end_tx, &dropped_lock);
+			      notify_end_tx);
 	if (ret == -ESHUTDOWN)
 		ret = call->error;
 
-	if (!dropped_lock)
-		mutex_unlock(&call->user_mutex);
 	_leave(" = %d", ret);
 	return ret;
 }



From dhowells at redhat.com  Wed Aug 12 04:01:20 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:20 +0100
Subject: [PATCH net v7 05/11] rxrpc: Fix generation of notifications after call completion
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-6-dhowells@redhat.com>

AF_RXRPC may generate a notification to the application after a call has
completed because it generates one notification when
rxrpc_input_split_jumbo() queues the final packet and completes the call
and then generates another when rxrpc_input_split_jumbo() does the
aggregated data receive notification at the end of the function.

This might cause the AFS filesystem to malfunction because it tries to
queue the afs_call for processing an extra time.  Most of the time this
happens quickly enough that the second queue_work skips, but sometimes this
means that the call work may happen a second time with implications for
afs_call lifetime management.

Fix this by:

 (1) Create a lighter version of rxrpc_notify_socket() that's just used to
     requeue a call for rxrpc_recvmsg() without creating another
     notification.

 (2) Move rxrpc_notify_socket() to call_state.c and rename it to
     __rxrpc_notify_socket().

 (3) Create a wrapper called rxrpc_notify_socket() that skips the
     notification if a call is completed.

 (4) Make rxrpc_set_call_completion() call __rxrpc_notify_socket() to avoid
     the skip-if-completed check.

Fixes: 2d1faf7a0ca3 ("rxrpc: Simplify skbuff accounting in receive path")
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: linux-afs at lists.infradead.org
cc: stable at kernel.org
---
 include/trace/events/rxrpc.h |  1 +
 net/rxrpc/call_state.c       | 57 +++++++++++++++++++++++++++++++++++-
 net/rxrpc/recvmsg.c          | 42 +++++++++-----------------
 3 files changed, 71 insertions(+), 29 deletions(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index 8f3e3967885a..d7c7b04d69fc 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -343,6 +343,7 @@
 	EM(rxrpc_call_see_distribute_error,	"SEE dist-err") \
 	EM(rxrpc_call_see_input,		"SEE input   ") \
 	EM(rxrpc_call_see_notify_released,	"SEE nfy-rlsd") \
+	EM(rxrpc_call_see_notify_skipped,	"SEE nfy-skip") \
 	EM(rxrpc_call_see_recvmsg,		"SEE recvmsg ") \
 	EM(rxrpc_call_see_recvmsg_requeue,	"SEE recv-rqu") \
 	EM(rxrpc_call_see_recvmsg_requeue_first, "SEE recv-rqF") \
diff --git a/net/rxrpc/call_state.c b/net/rxrpc/call_state.c
index 6afb54373ebb..6e402312e145 100644
--- a/net/rxrpc/call_state.c
+++ b/net/rxrpc/call_state.c
@@ -7,6 +7,61 @@
 
 #include "ar-internal.h"
 
+/*
+ * Post a call for attention by the socket or kernel service.  Further
+ * notifications are suppressed by putting recvmsg_link on a dummy queue.
+ */
+static void __rxrpc_notify_socket(struct rxrpc_call *call)
+{
+	struct rxrpc_sock *rx;
+	struct sock *sk;
+
+	if (test_bit(RXRPC_CALL_RELEASED, &call->flags)) {
+		rxrpc_see_call(call, rxrpc_call_see_notify_released);
+		return;
+	}
+
+	rcu_read_lock();
+
+	rx = rcu_dereference(call->socket);
+	sk = &rx->sk;
+	if (rx && sk->sk_state < RXRPC_CLOSE) {
+		if (call->notify_rx) {
+			spin_lock_irq(&call->notify_lock);
+			call->notify_rx(sk, call, call->user_call_ID);
+			spin_unlock_irq(&call->notify_lock);
+		} else {
+			spin_lock_irq(&rx->recvmsg_lock);
+			if (list_empty(&call->recvmsg_link)) {
+				rxrpc_get_call(call, rxrpc_call_get_notify_socket);
+				list_add_tail(&call->recvmsg_link, &rx->recvmsg_q);
+			}
+			spin_unlock_irq(&rx->recvmsg_lock);
+
+			if (!sock_flag(sk, SOCK_DEAD)) {
+				_debug("call %ps", sk->sk_data_ready);
+				sk->sk_data_ready(sk);
+			}
+		}
+	}
+
+	rcu_read_unlock();
+}
+
+/*
+ * Post a call for attention by the socket or kernel service.  Further
+ * notifications are suppressed by putting recvmsg_link on a dummy queue.
+ */
+void rxrpc_notify_socket(struct rxrpc_call *call)
+{
+	if (rxrpc_call_is_complete(call)) {
+		rxrpc_see_call(call, rxrpc_call_see_notify_skipped);
+		return;
+	}
+
+	__rxrpc_notify_socket(call);
+}
+
 /*
  * Transition a call to the complete state.
  */
@@ -25,7 +80,7 @@ bool rxrpc_set_call_completion(struct rxrpc_call *call,
 	rxrpc_set_call_state(call, RXRPC_CALL_COMPLETE);
 	trace_rxrpc_call_complete(call);
 	wake_up(&call->waitq);
-	rxrpc_notify_socket(call);
+	__rxrpc_notify_socket(call);
 	return true;
 }
 
diff --git a/net/rxrpc/recvmsg.c b/net/rxrpc/recvmsg.c
index efcba4b2e74f..28b2148b5693 100644
--- a/net/rxrpc/recvmsg.c
+++ b/net/rxrpc/recvmsg.c
@@ -17,13 +17,12 @@
 #include "ar-internal.h"
 
 /*
- * Post a call for attention by the socket or kernel service.  Further
- * notifications are suppressed by putting recvmsg_link on a dummy queue.
+ * Requeue a call for recvmsg() to pick up.
  */
-void rxrpc_notify_socket(struct rxrpc_call *call)
+static void rxrpc_requeue_call(struct socket *sock, struct rxrpc_call *call)
 {
-	struct rxrpc_sock *rx;
-	struct sock *sk;
+	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
+	struct sock *sk = &rx->sk;
 
 	_enter("%d", call->debug_id);
 
@@ -32,31 +31,18 @@ void rxrpc_notify_socket(struct rxrpc_call *call)
 		return;
 	}
 
-	rcu_read_lock();
-
-	rx = rcu_dereference(call->socket);
-	sk = &rx->sk;
-	if (rx && sk->sk_state < RXRPC_CLOSE) {
-		if (call->notify_rx) {
-			spin_lock_irq(&call->notify_lock);
-			call->notify_rx(sk, call, call->user_call_ID);
-			spin_unlock_irq(&call->notify_lock);
-		} else {
-			spin_lock_irq(&rx->recvmsg_lock);
-			if (list_empty(&call->recvmsg_link)) {
-				rxrpc_get_call(call, rxrpc_call_get_notify_socket);
-				list_add_tail(&call->recvmsg_link, &rx->recvmsg_q);
-			}
-			spin_unlock_irq(&rx->recvmsg_lock);
+	spin_lock_irq(&rx->recvmsg_lock);
+	if (list_empty(&call->recvmsg_link)) {
+		rxrpc_get_call(call, rxrpc_call_get_notify_socket);
+		list_add_tail(&call->recvmsg_link, &rx->recvmsg_q);
+	}
+	spin_unlock_irq(&rx->recvmsg_lock);
 
-			if (!sock_flag(sk, SOCK_DEAD)) {
-				_debug("call %ps", sk->sk_data_ready);
-				sk->sk_data_ready(sk);
-			}
-		}
+	if (!sock_flag(sk, SOCK_DEAD)) {
+		_debug("call %ps", sk->sk_data_ready);
+		sk->sk_data_ready(sk);
 	}
 
-	rcu_read_unlock();
 	_leave("");
 }
 
@@ -561,7 +547,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 
 	if (!(flags & MSG_PEEK) &&
 	    !skb_queue_empty(&call->recvmsg_queue))
-		rxrpc_notify_socket(call);
+		rxrpc_requeue_call(sock, call);
 	goto not_yet_complete;
 
 call_failed:



From dhowells at redhat.com  Wed Aug 12 04:01:21 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:21 +0100
Subject: [PATCH net v7 06/11] rxrpc: Expand abort trace enum
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-7-dhowells@redhat.com>

The rxrpc_abort_reason enum is marked __mode(byte), but there are currently
exactly 128 in the enum (0-127), but this is about to be increased and the
enum is represented by a signed byte and an increase causes compilation
errors.

Remove the mode marking to return it to a 32-bit int.

Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 include/trace/events/rxrpc.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index d7c7b04d69fc..ba2c6e3c78a6 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -602,7 +602,7 @@
 #define EM(a, b) a,
 #define E_(a, b) a
 
-enum rxrpc_abort_reason		{ rxrpc_abort_reasons } __mode(byte);
+enum rxrpc_abort_reason		{ rxrpc_abort_reasons };
 enum rxrpc_bundle_trace		{ rxrpc_bundle_traces } __mode(byte);
 enum rxrpc_call_poke_trace	{ rxrpc_call_poke_traces } __mode(byte);
 enum rxrpc_call_trace		{ rxrpc_call_traces } __mode(byte);



From dhowells at redhat.com  Wed Aug 12 04:01:22 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:22 +0100
Subject: [PATCH net v7 07/11] keys: Add refcounting to user-defined key type payload
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-8-dhowells@redhat.com>

Add refcounting to user-defined key type payload so that a kernel service
wanting to use such a key can hold onto the payload without the RCU read
lock held in order that it can do an allocation without having to be
concerned with the key getting updated.

This is the first part of the fix for the AF_RXRPC challenge response
generation code.

Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 include/keys/user-type.h     |  2 ++
 net/dns_resolver/dns_key.c   |  1 +
 security/keys/user_defined.c | 23 ++++++++++++++++-------
 3 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/include/keys/user-type.h b/include/keys/user-type.h
index 386c31432789..7002a993a472 100644
--- a/include/keys/user-type.h
+++ b/include/keys/user-type.h
@@ -26,6 +26,7 @@
  */
 struct user_key_payload {
 	struct rcu_head	rcu;		/* RCU destructor */
+	refcount_t	ref;
 	unsigned short	datalen;	/* length of this data */
 	char		data[] __aligned(__alignof__(u64)); /* actual data */
 };
@@ -37,6 +38,7 @@ struct key_preparsed_payload;
 
 extern int user_preparse(struct key_preparsed_payload *prep);
 extern void user_free_preparse(struct key_preparsed_payload *prep);
+void put_user_key_payload(struct user_key_payload *payload);
 extern int user_update(struct key *key, struct key_preparsed_payload *prep);
 extern void user_revoke(struct key *key);
 extern void user_destroy(struct key *key);
diff --git a/net/dns_resolver/dns_key.c b/net/dns_resolver/dns_key.c
index c3c8c3240ef9..aa3c058f4095 100644
--- a/net/dns_resolver/dns_key.c
+++ b/net/dns_resolver/dns_key.c
@@ -208,6 +208,7 @@ dns_resolver_preparse(struct key_preparsed_payload *prep)
 		kleave(" = -ENOMEM");
 		return -ENOMEM;
 	}
+	refcount_set(&upayload->ref, 1);
 
 	upayload->datalen = result_len;
 	memcpy(upayload->data, data, result_len);
diff --git a/security/keys/user_defined.c b/security/keys/user_defined.c
index 6f88b507f927..90c1bd5d7dfe 100644
--- a/security/keys/user_defined.c
+++ b/security/keys/user_defined.c
@@ -67,6 +67,7 @@ int user_preparse(struct key_preparsed_payload *prep)
 	upayload = kmalloc_flex(*upayload, data, datalen);
 	if (!upayload)
 		return -ENOMEM;
+	refcount_set(&upayload->ref, 1);
 
 	/* attach the data */
 	prep->quotalen = datalen;
@@ -88,12 +89,22 @@ EXPORT_SYMBOL_GPL(user_free_preparse);
 
 static void user_free_payload_rcu(struct rcu_head *head)
 {
-	struct user_key_payload *payload;
+	struct user_key_payload *payload =
+		container_of(head, struct user_key_payload, rcu);
 
-	payload = container_of(head, struct user_key_payload, rcu);
 	kfree_sensitive(payload);
 }
 
+/*
+ * Free a user defined key payload.
+ */
+void put_user_key_payload(struct user_key_payload *payload)
+{
+	if (payload && refcount_dec_and_test(&payload->ref))
+		call_rcu(&payload->rcu, user_free_payload_rcu);
+}
+EXPORT_SYMBOL_GPL(put_user_key_payload);
+
 /*
  * update a user defined key
  * - the key's semaphore is write-locked
@@ -115,8 +126,7 @@ int user_update(struct key *key, struct key_preparsed_payload *prep)
 	rcu_assign_keypointer(key, prep->payload.data[0]);
 	prep->payload.data[0] = NULL;
 
-	if (zap)
-		call_rcu(&zap->rcu, user_free_payload_rcu);
+	put_user_key_payload(zap);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(user_update);
@@ -134,7 +144,7 @@ void user_revoke(struct key *key)
 
 	if (upayload) {
 		rcu_assign_keypointer(key, NULL);
-		call_rcu(&upayload->rcu, user_free_payload_rcu);
+		put_user_key_payload(upayload);
 	}
 }
 
@@ -147,9 +157,8 @@ void user_destroy(struct key *key)
 {
 	struct user_key_payload *upayload = key->payload.data[0];
 
-	kfree_sensitive(upayload);
+	put_user_key_payload(upayload);
 }
-
 EXPORT_SYMBOL_GPL(user_destroy);
 
 /*



From dhowells at redhat.com  Wed Aug 12 04:01:23 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:23 +0100
Subject: [PATCH net v7 08/11] afs: Create a server appdata key
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-9-dhowells@redhat.com>

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

This is a part of the fix.  With this, the AFS filesystem creates an
appdata key for each fileserver it talks to with RxGK and attaches it to
the afs_server record.

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 fs/afs/cm_security.c   | 232 +++++++++++++++++++++++++++++++++++++++++
 fs/afs/fs_probe.c      |   5 +
 fs/afs/internal.h      |   2 +
 fs/afs/server.c        |   1 +
 include/net/af_rxrpc.h |   2 +
 net/rxrpc/key.c        |  37 +++++++
 6 files changed, 279 insertions(+)

diff --git a/fs/afs/cm_security.c b/fs/afs/cm_security.c
index 103168c70dd4..8f04fec20a2a 100644
--- a/fs/afs/cm_security.c
+++ b/fs/afs/cm_security.c
@@ -6,7 +6,9 @@
  */
 
 #include <linux/slab.h>
+#include <linux/key-type.h>
 #include <crypto/krb5.h>
+#include <keys/user-type.h>
 #include "internal.h"
 #include "afs_cm.h"
 #include "afs_fs.h"
@@ -23,6 +25,236 @@ static int afs_create_yfs_cm_token(struct sk_buff *challenge,
 				   struct afs_server *server);
 #endif
 
+#ifdef CONFIG_RXGK
+/*
+ * As the YFS RxGK appdata to be passed in the YFS.FS-service RESPONSE packet,
+ * create a GSS token to use as a ticket to the specified fileserver.
+ */
+static int afs_create_yfs_rxgk_cm_appdata(struct afs_server *server, u32 enctype)
+{
+	const struct krb5_enctype *conn_krb5, *token_krb5;
+	const struct krb5_buffer *token_key;
+	struct crypto_aead *aead;
+	struct scatterlist sg;
+	struct afs_net *net = server->cell->net;
+	const struct key *cm_key = net->fs_cm_token_key;
+	struct key *appdata_key = NULL;
+	size_t keysize, uuidsize, authsize, toksize, encsize, contsize;
+	size_t adatasize, offset;
+	__be32 caps[1] = {
+		[0] = htonl(AFS_CAP_ERROR_TRANSLATION),
+	};
+	__be32 *xdr;
+	void *appdata, *K0, *encbase;
+	int ret;
+
+	if (!cm_key)
+		return -ENOKEY;
+
+	/* Assume that the fileserver is happy to use the same encoding type as
+	 * we were told to use by the token obtained by the user.
+	 */
+	conn_krb5 = crypto_krb5_find_enctype(enctype);
+	if (!conn_krb5)
+		return -ENOPKG;
+	token_krb5 = cm_key->payload.data[0];
+	token_key = (const struct krb5_buffer *)&cm_key->payload.data[2];
+
+	/* struct rxgk_key {
+	 *	afs_uint32	enctype;
+	 *	opaque		key<>;
+	 * };
+	 */
+	keysize = 4 + xdr_len_object(conn_krb5->key_len);
+
+	/* struct RXGK_AuthName {
+	 *	afs_int32	kind;
+	 *	opaque		data<AUTHDATAMAX>;
+	 *	opaque		display<AUTHPRINTABLEMAX>;
+	 * };
+	 */
+	uuidsize = sizeof(server->uuid);
+	authsize = 4 + xdr_len_object(uuidsize) + xdr_len_object(0);
+
+	/* struct RXGK_Token {
+	 *	rxgk_key		K0;
+	 *	RXGK_Level		level;
+	 *	rxgkTime		starttime;
+	 *	afs_int32		lifetime;
+	 *	afs_int32		bytelife;
+	 *	rxgkTime		expirationtime;
+	 *	struct RXGK_AuthName	identities<>;
+	 * };
+	 */
+	toksize = keysize + 4 + 8 + 4 + 4 + 8 + xdr_len_object(authsize);
+
+	offset = 0;
+	encsize = crypto_krb5_how_much_buffer(token_krb5, KRB5_ENCRYPT_MODE, toksize, &offset);
+
+	/* struct RXGK_TokenContainer {
+	 *	afs_int32	kvno;
+	 *	afs_int32	enctype;
+	 *	opaque		encrypted_token<>;
+	 * };
+	 */
+	contsize = 4 + 4 + xdr_len_object(encsize);
+
+	/* struct YFSAppData {
+	 *	opr_uuid	initiatorUuid;
+	 *	opr_uuid	acceptorUuid;
+	 *	Capabilities	caps;
+	 *	afs_int32	enctype;
+	 *	opaque		callbackKey<>;
+	 *	opaque		callbackToken<>;
+	 * };
+	 */
+	adatasize = 16 + 16 +
+		xdr_len_object(sizeof(caps)) +
+		4 +
+		xdr_len_object(conn_krb5->key_len) +
+		xdr_len_object(contsize);
+
+	ret = -ENOMEM;
+	appdata = kzalloc(adatasize, GFP_KERNEL);
+	if (!appdata)
+		goto out;
+	xdr = appdata;
+
+	memcpy(xdr, &net->uuid, 16);		/* appdata.initiatorUuid */
+	xdr += 16 / 4;
+	memcpy(xdr, &server->uuid, 16);		/* appdata.acceptorUuid */
+	xdr += 16 / 4;
+	*xdr++ = htonl(ARRAY_SIZE(caps));	/* appdata.caps.len */
+	memcpy(xdr, &caps, sizeof(caps));	/* appdata.caps */
+	xdr += ARRAY_SIZE(caps);
+	*xdr++ = htonl(conn_krb5->etype);	/* appdata.enctype */
+
+	*xdr++ = htonl(conn_krb5->key_len);	/* appdata.callbackKey.len */
+	K0 = xdr;
+	get_random_bytes(K0, conn_krb5->key_len); /* appdata.callbackKey.data */
+	xdr += xdr_round_up(conn_krb5->key_len) / 4;
+
+	*xdr++ = htonl(contsize);		/* appdata.callbackToken.len */
+	*xdr++ = htonl(1);			/* cont.kvno */
+	*xdr++ = htonl(token_krb5->etype);	/* cont.enctype */
+	*xdr++ = htonl(encsize);		/* cont.encrypted_token.len */
+
+	encbase = xdr;
+	xdr += offset / 4;
+	*xdr++ = htonl(conn_krb5->etype);	/* token.K0.enctype */
+	*xdr++ = htonl(conn_krb5->key_len);	/* token.K0.key.len */
+	memcpy(xdr, K0, conn_krb5->key_len);	/* token.K0.key.data */
+	xdr += xdr_round_up(conn_krb5->key_len) / 4;
+
+	*xdr++ = htonl(RXRPC_SECURITY_ENCRYPT);	/* token.level */
+	*xdr++ = htonl(0);			/* token.starttime */
+	*xdr++ = htonl(0);			/* " */
+	*xdr++ = htonl(0);			/* token.lifetime */
+	*xdr++ = htonl(0);			/* token.bytelife */
+	*xdr++ = htonl(0);			/* token.expirationtime */
+	*xdr++ = htonl(0);			/* " */
+	*xdr++ = htonl(1);			/* token.identities.count */
+	*xdr++ = htonl(0);			/* token.identities[0].kind */
+	*xdr++ = htonl(uuidsize);		/* token.identities[0].data.len */
+	memcpy(xdr, &server->uuid, uuidsize);
+	xdr += xdr_round_up(uuidsize) / 4;
+	*xdr++ = htonl(0);			/* token.identities[0].display.len */
+
+	xdr = encbase + xdr_round_up(encsize);
+
+	if ((unsigned long)xdr - (unsigned long)appdata != adatasize)
+		pr_err("Appdata size incorrect %lx != %zx\n",
+		       (unsigned long)xdr - (unsigned long)appdata, adatasize);
+
+	aead = crypto_krb5_prepare_encryption(token_krb5, token_key, RXGK_SERVER_ENC_TOKEN,
+					      GFP_KERNEL);
+	if (IS_ERR(aead)) {
+		ret = PTR_ERR(aead);
+		goto out_token;
+	}
+
+	sg_init_one(&sg, encbase, encsize);
+	ret = crypto_krb5_encrypt(token_krb5, aead, &sg, 1, encsize, offset, toksize, false);
+	if (ret < 0)
+		goto out_aead;
+
+	appdata_key = key_alloc(&key_type_user, "rxrpc: afs rxgk appdata",
+				GLOBAL_ROOT_UID, GLOBAL_ROOT_GID, current_cred(),
+				KEY_POS_VIEW | KEY_POS_SEARCH | KEY_USR_VIEW,
+				KEY_ALLOC_NOT_IN_QUOTA, NULL);
+	if (IS_ERR(appdata_key)) {
+		ret = PTR_ERR(appdata_key);
+		goto out_aead;
+	}
+
+	ret = key_instantiate_and_link(appdata_key, appdata, adatasize, NULL, NULL);
+	if (ret < 0) {
+		key_put(appdata_key);
+		goto out_aead;
+	}
+
+	/* Store the appdata before the key pointer */
+	smp_store_release(&server->yfs_rxgk_appdata, appdata_key);
+
+out_aead:
+	crypto_free_aead(aead);
+out_token:
+	kfree_sensitive(appdata);
+out:
+	return ret;
+}
+#endif /* CONFIG_RXGK */
+
+/*
+ * Create the application data to go in a RESPONSE packet a server's CHALLENGE
+ * from the parameters contained in a key.  The key specifies the security
+ * index and other appropriate parameters such as the encoding type for RxGK.
+ */
+int afs_create_server_appdata(struct afs_server *server, struct key *key)
+{
+	u32 krb5_enctype;
+	int ret;
+	u8 security_index;
+
+	if (!key)
+		return 0;
+
+	rxrpc_kernel_query_key(key, &security_index, &krb5_enctype);
+
+	_enter("%u,%u", security_index, krb5_enctype);
+
+	switch (security_index) {
+#ifdef CONFIG_RXGK
+	case RXRPC_SECURITY_YFS_RXGK:
+		/* Read the key pointer before the appdata */
+		if (smp_load_acquire(&server->yfs_rxgk_appdata))
+			return 0;
+		break;
+#endif
+	default:
+		return 0;
+	}
+
+	ret = 0;
+	mutex_lock(&server->cm_token_lock);
+
+	switch (security_index) {
+#ifdef CONFIG_RXGK
+	case RXRPC_SECURITY_YFS_RXGK:
+		/* Read the key pointer before the appdata */
+		if (smp_load_acquire(&server->yfs_rxgk_appdata))
+			break;
+		ret = afs_create_yfs_rxgk_cm_appdata(server, krb5_enctype);
+		break;
+#endif
+	default:
+		break;
+	}
+
+	mutex_unlock(&server->cm_token_lock);
+	return ret;
+}
+
 /*
  * Respond to an RxGK challenge, adding appdata.
  */
diff --git a/fs/afs/fs_probe.c b/fs/afs/fs_probe.c
index a91ad1938d07..e26f7f1e30a2 100644
--- a/fs/afs/fs_probe.c
+++ b/fs/afs/fs_probe.c
@@ -241,9 +241,14 @@ int afs_fs_probe_fileserver(struct afs_net *net, struct afs_server *server,
 	struct afs_endpoint_state *estate, *old;
 	struct afs_addr_list *old_alist = NULL, *alist;
 	unsigned long unprobed;
+	int ret;
 
 	_enter("%pU", &server->uuid);
 
+	ret = afs_create_server_appdata(server, key);
+	if (ret < 0)
+		return ret;
+
 	estate = kzalloc_obj(*estate);
 	if (!estate)
 		return -ENOMEM;
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 290873bac89b..320c9535832a 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -549,6 +549,7 @@ struct afs_server {
 	struct timer_list	timer;		/* Management timer */
 	struct mutex		cm_token_lock;	/* Lock governing creation of appdata */
 	struct krb5_buffer	cm_rxgk_appdata; /* Appdata to be included in RESPONSE packet */
+	struct key		*yfs_rxgk_appdata; /* Appdata to be included in RESPONSE packet */
 	time64_t		unuse_time;	/* Time at which last unused */
 	unsigned long		flags;
 #define AFS_SERVER_FL_RESPONDING 0		/* The server is responding */
@@ -1089,6 +1090,7 @@ extern bool afs_cm_incoming_call(struct afs_call *);
 /*
  * cm_security.c
  */
+int afs_create_server_appdata(struct afs_server *server, struct key *key);
 void afs_process_oob_queue(struct work_struct *work);
 #ifdef CONFIG_RXGK
 int afs_create_token_key(struct afs_net *net, struct socket *socket);
diff --git a/fs/afs/server.c b/fs/afs/server.c
index 0fe162ea2a36..b08d9080b0c5 100644
--- a/fs/afs/server.c
+++ b/fs/afs/server.c
@@ -399,6 +399,7 @@ static void afs_server_rcu(struct rcu_head *rcu)
 			       afs_estate_trace_put_server);
 	afs_put_cell(server->cell, afs_cell_trace_put_server);
 	kfree(server->cm_rxgk_appdata.data);
+	key_put(server->yfs_rxgk_appdata);
 	kfree(server);
 }
 
diff --git a/include/net/af_rxrpc.h b/include/net/af_rxrpc.h
index 0fb4c41c9bbf..c4b68049c06f 100644
--- a/include/net/af_rxrpc.h
+++ b/include/net/af_rxrpc.h
@@ -109,6 +109,8 @@ int rxkad_kernel_respond_to_challenge(struct sk_buff *challenge);
 u32 rxgk_kernel_query_challenge(struct sk_buff *challenge);
 int rxgk_kernel_respond_to_challenge(struct sk_buff *challenge,
 				     struct krb5_buffer *appdata);
+void rxrpc_kernel_query_key(const struct key *key, u8 *_security_index,
+			    u32 *_krb5_enctype);
 u8 rxrpc_kernel_query_call_security(struct rxrpc_call *call,
 				    u16 *_service_id, u32 *_enctype);
 
diff --git a/net/rxrpc/key.c b/net/rxrpc/key.c
index a0aa78d89289..16bab72e6f07 100644
--- a/net/rxrpc/key.c
+++ b/net/rxrpc/key.c
@@ -893,3 +893,40 @@ static long rxrpc_read(const struct key *key,
 	_leave(" = %zu", size);
 	return size;
 }
+
+/**
+ * rxrpc_kernel_query_key - Query parameters from an rxrpc key
+ * @key: The key to query
+ * @_security_index: Where to return the security index
+ * @_krb5_enctype: Where to return the krb5 encryption type if applicable
+ *
+ * Query an rxrpc authentication key, extracting the security index from the
+ * first token therein.
+ */
+void rxrpc_kernel_query_key(const struct key *key, u8 *_security_index,
+			    u32 *_krb5_enctype)
+{
+	const struct rxrpc_key_token *token;
+
+	token = key->payload.data[0];
+	if (!token) {
+		*_security_index = 0;
+		*_krb5_enctype = 0;
+		return;
+	}
+
+	*_security_index = token->security_index;
+	switch (token->security_index) {
+	case RXRPC_SECURITY_RXKAD:
+		*_krb5_enctype = 0;
+		break;
+	case RXRPC_SECURITY_YFS_RXGK:
+		*_krb5_enctype = token->rxgk->enctype;
+		break;
+	default:
+		WARN_ON_ONCE(1);
+		*_krb5_enctype = 0;
+		break;
+	}
+}
+EXPORT_SYMBOL(rxrpc_kernel_query_key);



From dhowells at redhat.com  Wed Aug 12 04:01:24 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:24 +0100
Subject: [PATCH net v7 09/11] rxrpc: Pass appdata key to rxrpc_call and thence to rxrpc_bundle
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-10-dhowells@redhat.com>

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

This is a part of the fix.  With this, the appdata key created by the AFS
filesystem or passed in via sendmsg CMSG to a user AF_RXRPC socket is added
to an rxrpc_call struct and will then be added to an rxrpc_bundle struct.

Note that afs_make_op_call() has to be moved so that it can call
afs_use_server().  afs_operation-based calls did not heretofore 'use' the
server and server will be 'un-used' by afs_free_call().

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 fs/afs/internal.h          | 33 +++++++++++++++++----------------
 fs/afs/rxrpc.c             | 21 ++++++++++++++++++++-
 include/net/af_rxrpc.h     |  1 +
 include/trace/events/afs.h |  1 +
 include/uapi/linux/rxrpc.h |  1 +
 net/rxrpc/af_rxrpc.c       |  3 +++
 net/rxrpc/ar-internal.h    |  3 +++
 net/rxrpc/call_object.c    |  2 ++
 net/rxrpc/conn_client.c    |  4 ++++
 net/rxrpc/proc.c           |  5 +++--
 net/rxrpc/sendmsg.c        | 26 +++++++++++++++++++++++++-
 11 files changed, 80 insertions(+), 20 deletions(-)

diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index 320c9535832a..ce05595b7d3c 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -1418,22 +1418,6 @@ static inline void afs_see_call(struct afs_call *call, enum afs_call_trace why)
 		       __builtin_return_address(0));
 }
 
-static inline void afs_make_op_call(struct afs_operation *op, struct afs_call *call,
-				    gfp_t gfp)
-{
-	struct afs_addr_list *alist = op->estate->addresses;
-
-	op->call	= afs_get_call(call, afs_call_trace_get);
-	op->type	= call->type;
-	call->op	= op;
-	call->key	= op->key;
-	call->intr	= !(op->flags & AFS_OPERATION_UNINTR);
-	call->peer	= rxrpc_kernel_get_peer(alist->addrs[op->addr_index].peer);
-	call->service_id = op->server->service_id;
-	afs_make_call(call, gfp);
-	afs_put_call(call);
-}
-
 static inline void afs_extract_begin(struct afs_call *call, void *buf, size_t size)
 {
 	call->iov_len = size;
@@ -1765,6 +1749,23 @@ static inline struct inode *AFS_VNODE_TO_I(struct afs_vnode *vnode)
 	return &vnode->netfs.inode;
 }
 
+static inline void afs_make_op_call(struct afs_operation *op, struct afs_call *call,
+				    gfp_t gfp)
+{
+	struct afs_addr_list *alist = op->estate->addresses;
+
+	op->call	= afs_get_call(call, afs_call_trace_get);
+	op->type	= call->type;
+	call->op	= op;
+	call->server	= afs_use_server(op->server, false, afs_server_trace_use_call);
+	call->key	= op->key;
+	call->intr	= !(op->flags & AFS_OPERATION_UNINTR);
+	call->peer	= rxrpc_kernel_get_peer(alist->addrs[op->addr_index].peer);
+	call->service_id = op->server->service_id;
+	afs_make_call(call, gfp);
+	afs_put_call(call);
+}
+
 /*
  * Note that a dentry got changed.  We need to set d_fsdata to the data version
  * number derived from the result of the operation.  It doesn't matter if
diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index f3552a1bbc03..7aee2814a9e9 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -347,6 +347,7 @@ void afs_make_call(struct afs_call *call, gfp_t gfp)
 	struct rxrpc_call *rxcall;
 	struct msghdr msg;
 	struct kvec iov[1];
+	struct key *app_data = NULL;
 	size_t len;
 	s64 tx_total_len;
 	int ret;
@@ -378,8 +379,26 @@ void afs_make_call(struct afs_call *call, gfp_t gfp)
 		call->drop_ref = true;
 	}
 
+	if (call->key && call->server) {
+		u32 krb5_enctype = 0;
+		u8 security_index = 0;
+
+		rxrpc_kernel_query_key(call->key, &security_index, &krb5_enctype);
+		switch (security_index) {
+#ifdef CONFIG_RXGK
+		case RXRPC_SECURITY_YFS_RXGK:
+			/* Read the key pointer before the appdata */
+			app_data = smp_load_acquire(&call->server->yfs_rxgk_appdata);
+			break;
+#endif
+		default:
+			break;
+		}
+	}
+
 	/* create a call */
-	rxcall = rxrpc_kernel_begin_call(call->net->socket, call->peer, call->key,
+	rxcall = rxrpc_kernel_begin_call(call->net->socket, call->peer,
+					 call->key, app_data,
 					 (unsigned long)call,
 					 tx_total_len,
 					 call->max_lifespan,
diff --git a/include/net/af_rxrpc.h b/include/net/af_rxrpc.h
index c4b68049c06f..19c61a2f5af3 100644
--- a/include/net/af_rxrpc.h
+++ b/include/net/af_rxrpc.h
@@ -55,6 +55,7 @@ void rxrpc_kernel_set_notifications(struct socket *sock,
 struct rxrpc_call *rxrpc_kernel_begin_call(struct socket *sock,
 					   struct rxrpc_peer *peer,
 					   struct key *key,
+					   struct key *app_data,
 					   unsigned long user_call_ID,
 					   s64 tx_total_len,
 					   u32 hard_timeout,
diff --git a/include/trace/events/afs.h b/include/trace/events/afs.h
index 1b3c48b5591d..04b0bb682b81 100644
--- a/include/trace/events/afs.h
+++ b/include/trace/events/afs.h
@@ -148,6 +148,7 @@ enum yfs_cm_operation {
 	EM(afs_server_trace_unuse_slist_isort,	"UNU isort") \
 	EM(afs_server_trace_update,		"UPDATE   ") \
 	EM(afs_server_trace_use_by_uuid,	"USE uuid ") \
+	EM(afs_server_trace_use_call,		"USE call ") \
 	EM(afs_server_trace_use_cm_call,	"USE cm-cl") \
 	EM(afs_server_trace_use_get_caps,	"USE gcaps") \
 	EM(afs_server_trace_use_give_up_cb,	"USE gvupc") \
diff --git a/include/uapi/linux/rxrpc.h b/include/uapi/linux/rxrpc.h
index d9735abd4c79..bcdfdf9c67a1 100644
--- a/include/uapi/linux/rxrpc.h
+++ b/include/uapi/linux/rxrpc.h
@@ -63,6 +63,7 @@ enum rxrpc_cmsg_type {
 	RXRPC_RESPOND		= 17,	/* Cs-: Respond to a challenge */
 	RXRPC_RESPONDED		= 18,	/* S-r: Data received in RESPONSE */
 	RXRPC_RESP_RXGK_APPDATA	= 19,	/* Cs-: RESPONSE: RxGK app data to include */
+	RXRPC_RESPONSE_APPDATA	= 20,	/* Cs-: User key holding app data for RESPONSE */
 	RXRPC__SUPPORTED
 };
 
diff --git a/net/rxrpc/af_rxrpc.c b/net/rxrpc/af_rxrpc.c
index 9ab0f22c881e..a19c0fd3c51a 100644
--- a/net/rxrpc/af_rxrpc.c
+++ b/net/rxrpc/af_rxrpc.c
@@ -318,6 +318,7 @@ EXPORT_SYMBOL(rxrpc_kernel_put_peer);
  * @sock: The socket on which to make the call
  * @peer: The peer to contact
  * @key: The security context to use (defaults to socket setting)
+ * @app_data: The security response application data (or NULL)
  * @user_call_ID: The ID to use
  * @tx_total_len: Total length of data to transmit during the call (or -1)
  * @hard_timeout: The maximum lifespan of the call in sec
@@ -340,6 +341,7 @@ EXPORT_SYMBOL(rxrpc_kernel_put_peer);
 struct rxrpc_call *rxrpc_kernel_begin_call(struct socket *sock,
 					   struct rxrpc_peer *peer,
 					   struct key *key,
+					   struct key *app_data,
 					   unsigned long user_call_ID,
 					   s64 tx_total_len,
 					   u32 hard_timeout,
@@ -368,6 +370,7 @@ struct rxrpc_call *rxrpc_kernel_begin_call(struct socket *sock,
 		key = NULL; /* a no-security key */
 
 	memset(&p, 0, sizeof(p));
+	p.app_data		= app_data;
 	p.user_call_ID		= user_call_ID;
 	p.tx_total_len		= tx_total_len;
 	p.interruptibility	= interruptibility;
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index a6f830c1621f..6e16df7173d3 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -516,6 +516,7 @@ struct rxrpc_bundle {
 	struct rxrpc_local	*local;		/* Representation of local endpoint */
 	struct rxrpc_peer	*peer;		/* Remote endpoint */
 	struct key		*key;		/* Security details */
+	struct key		*app_data;	/* Security response app data */
 	struct list_head	proc_link;	/* Link in net->bundle_proc_list */
 	const struct rxrpc_security *security;	/* applied security module */
 	refcount_t		ref;
@@ -720,6 +721,7 @@ struct rxrpc_call {
 	struct rxrpc_sock __rcu	*socket;	/* socket responsible */
 	struct rxrpc_net	*rxnet;		/* Network namespace to which call belongs */
 	struct key		*key;		/* Security details */
+	struct key		*app_data;	/* Security response app data */
 	const struct rxrpc_security *security;	/* applied security module */
 	struct mutex		user_mutex;	/* User access mutex */
 	struct sockaddr_rxrpc	dest_srx;	/* Destination address */
@@ -914,6 +916,7 @@ enum rxrpc_command {
 };
 
 struct rxrpc_call_params {
+	struct key		*app_data;	/* Security response app data */
 	s64			tx_total_len;	/* Total Tx data length (if send data) */
 	unsigned long		user_call_ID;	/* User's call ID */
 	struct {
diff --git a/net/rxrpc/call_object.c b/net/rxrpc/call_object.c
index 817ed9acb91e..9f6130d90c4c 100644
--- a/net/rxrpc/call_object.c
+++ b/net/rxrpc/call_object.c
@@ -211,6 +211,7 @@ static struct rxrpc_call *rxrpc_alloc_client_call(struct rxrpc_sock *rx,
 	call->interruptibility	= p->interruptibility;
 	call->tx_total_len	= p->tx_total_len;
 	call->key		= key_get(cp->key);
+	call->app_data		= key_get(p->app_data);
 	call->peer		= rxrpc_get_peer(cp->peer, rxrpc_peer_get_call);
 	call->local		= rxrpc_get_local(cp->local, rxrpc_local_get_call);
 	call->security_level	= cp->security_level;
@@ -697,6 +698,7 @@ static void rxrpc_destroy_call(struct work_struct *work)
 	rxrpc_put_peer(call->peer, rxrpc_peer_put_call);
 	rxrpc_put_local(call->local, rxrpc_local_put_call);
 	key_put(call->key);
+	key_put(call->app_data);
 	call_rcu(&call->rcu, rxrpc_rcu_free_call);
 }
 
diff --git a/net/rxrpc/conn_client.c b/net/rxrpc/conn_client.c
index 48519f0de185..dd79630d3066 100644
--- a/net/rxrpc/conn_client.c
+++ b/net/rxrpc/conn_client.c
@@ -81,6 +81,7 @@ static struct rxrpc_bundle *rxrpc_alloc_bundle(struct rxrpc_call *call,
 		bundle->local		= call->local;
 		bundle->peer		= rxrpc_get_peer(call->peer, rxrpc_peer_get_bundle);
 		bundle->key		= key_get(call->key);
+		bundle->app_data	= key_get(call->app_data);
 		bundle->security	= call->security;
 		bundle->exclusive	= test_bit(RXRPC_CALL_EXCLUSIVE, &call->flags);
 		bundle->upgrade		= test_bit(RXRPC_CALL_UPGRADE, &call->flags);
@@ -118,6 +119,7 @@ static void rxrpc_free_bundle(struct rxrpc_bundle *bundle)
 	write_unlock(&bundle->local->rxnet->conn_lock);
 	rxrpc_put_peer(bundle->peer, rxrpc_peer_put_bundle);
 	key_put(bundle->key);
+	key_put(bundle->app_data);
 	kfree(bundle);
 }
 
@@ -274,6 +276,7 @@ int rxrpc_look_up_bundle(struct rxrpc_call *call, gfp_t gfp)
 #define cmp(X, Y) ((long)(X) - (long)(Y))
 		diff = (cmp(bundle->peer, call->peer) ?:
 			cmp(bundle->key, call->key) ?:
+			cmp(bundle->app_data, call->app_data) ?:
 			cmp(bundle->security_level, call->security_level) ?:
 			cmp(bundle->upgrade, upgrade));
 #undef cmp
@@ -303,6 +306,7 @@ int rxrpc_look_up_bundle(struct rxrpc_call *call, gfp_t gfp)
 #define cmp(X, Y) ((long)(X) - (long)(Y))
 		diff = (cmp(bundle->peer, call->peer) ?:
 			cmp(bundle->key, call->key) ?:
+			cmp(bundle->app_data, call->app_data) ?:
 			cmp(bundle->security_level, call->security_level) ?:
 			cmp(bundle->upgrade, upgrade));
 #undef cmp
diff --git a/net/rxrpc/proc.c b/net/rxrpc/proc.c
index e9a27fa7b25d..f300491b06bd 100644
--- a/net/rxrpc/proc.c
+++ b/net/rxrpc/proc.c
@@ -238,7 +238,7 @@ static int rxrpc_bundle_seq_show(struct seq_file *seq, void *v)
 		seq_puts(seq,
 			 "Proto Local                                          "
 			 " Remote                                         "
-			 " SvID Ref Act Flg Key      |"
+			 " SvID Ref Act Flg Key      AppData  |"
 			 " Bundle   Conn_0   Conn_1   Conn_2   Conn_3\n"
 			 );
 		return 0;
@@ -250,7 +250,7 @@ static int rxrpc_bundle_seq_show(struct seq_file *seq, void *v)
 	scnprintf(rbuff, sizeof(rbuff), "%pISpc", &bundle->peer->srx.transport);
 	seq_printf(seq,
 		   "UDP   %-47.47s %-47.47s %4x %3u %3d"
-		   " %c%c%c %08x | %08x %08x %08x %08x %08x\n",
+		   " %c%c%c %08x %08x | %08x %08x %08x %08x %08x\n",
 		   lbuff,
 		   rbuff,
 		   bundle->service_id,
@@ -260,6 +260,7 @@ static int rxrpc_bundle_seq_show(struct seq_file *seq, void *v)
 		   bundle->exclusive ? 'e' : '-',
 		   bundle->upgrade ? 'u' : '-',
 		   key_serial(bundle->key),
+		   key_serial(bundle->app_data),
 		   bundle->debug_id,
 		   bundle->conn_ids[0],
 		   bundle->conn_ids[1],
diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index 1cfc03db78f9..4755fc76d8f3 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -12,6 +12,7 @@
 #include <linux/skbuff.h>
 #include <linux/export.h>
 #include <linux/sched/signal.h>
+#include <keys/user-type.h>
 
 #include <net/sock.h>
 #include <net/af_rxrpc.h>
@@ -547,6 +548,8 @@ static int rxrpc_send_data(struct rxrpc_sock *rx,
 static int rxrpc_sendmsg_cmsg(struct msghdr *msg, struct rxrpc_send_params *p)
 {
 	struct cmsghdr *cmsg;
+	key_serial_t key_id;
+	key_ref_t key;
 	bool got_user_ID = false;
 	int len;
 
@@ -631,6 +634,23 @@ static int rxrpc_sendmsg_cmsg(struct msghdr *msg, struct rxrpc_send_params *p)
 				return -ERANGE;
 			break;
 
+		case RXRPC_RESPONSE_APPDATA:
+			if (len != sizeof(key_serial_t))
+				return -EINVAL;
+			if (p->call.app_data)
+				return -EINVAL;
+			key_id = *(key_serial_t *)CMSG_DATA(cmsg);
+			key = lookup_user_key(key_id, 0, KEY_NEED_SEARCH);
+			if (IS_ERR(key))
+				return PTR_ERR(key);
+			if (key_ref_to_ptr(key)->type != &key_type_user &&
+			    key_ref_to_ptr(key)->type != &key_type_logon) {
+				key_ref_put(key);
+				return -EINVAL;
+			}
+			p->call.app_data = key_ref_to_ptr(key);
+			break;
+
 		default:
 			return -EINVAL;
 		}
@@ -738,8 +758,10 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
 			goto error_release_sock;
 		call = rxrpc_new_client_call_for_sendmsg(rx, msg, &p);
 		/* The socket is now unlocked... */
-		if (IS_ERR(call))
+		if (IS_ERR(call)) {
+			key_put(p.call.app_data);
 			return PTR_ERR(call);
+		}
 		/* ... and we have the call lock. */
 		p.call.nr_timeouts = 0;
 		ret = 0;
@@ -823,11 +845,13 @@ int rxrpc_do_sendmsg(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
 	mutex_unlock(&call->user_mutex);
 error_put:
 	rxrpc_put_call(call, rxrpc_call_put_sendmsg);
+	key_put(p.call.app_data);
 	_leave(" = %d", ret);
 	return ret;
 
 error_release_sock:
 	release_sock(&rx->sk);
+	key_put(p.call.app_data);
 	return ret;
 }
 



From dhowells at redhat.com  Wed Aug 12 04:01:25 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:25 +0100
Subject: [PATCH net v7 10/11] rxrpc: Fix CHALLENGE packet overqueuing and simplify RESPONSE generation
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-11-dhowells@redhat.com>

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

This is the main part of the fix.  It switches the code over from using the
OOB communication mechanism to get the appdata on the fly from the
application (either the AFS filesystem or userspace) to using the appdata
key preemptively provided by the application.

The following changes are made:

 (1) Revert to making the connection event processor work item parse the
     CHALLENGE and generate the RESPONSE directly.

 (2) Add another (optional) parameter that is passed in when an rxrpc
     client call is created and ends up attached to the connection bundle
     and is a user-type or logon-type key containing the application data.

 (3) RESPONSE generation looks at the bundle and if the app-data is there,
     it will include it (if the security class is YFS-RxGK; RxKAD ignores
     it).

The AFS filesystem driver creates an app-data key when it probes a
fileserver and attaches it to the afs_server struct.  This is picked up
when a call is made to that server and thence passed to rxrpc.

Direct userspace users of AF_RXRPC can partake by creating a user-type or
logon-type key containing the app-data they want to use and passing its
serial ID in a CMSG of type RXRPC_RESPONSE_APPDATA in the initial sendmsg()
of a call.

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 include/trace/events/rxrpc.h |  1 +
 net/rxrpc/af_rxrpc.c         |  5 +--
 net/rxrpc/conn_event.c       | 68 ++----------------------------------
 net/rxrpc/rxgk.c             | 67 ++++++++++++++++++++++-------------
 4 files changed, 48 insertions(+), 93 deletions(-)

diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index ba2c6e3c78a6..48e774034195 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -78,6 +78,7 @@
 	EM(rxgk_abort_2_short_encdata,		"rxgk2-short-encdata")	\
 	EM(rxgk_abort_2_short_header,		"rxgk2-short-hdr")	\
 	EM(rxgk_abort_bad_key_number,		"rxgk-bad-key-num")	\
+	EM(rxgk_abort_chall_bad_app_key,	"rxgk-chall-bad-app-key")\
 	EM(rxgk_abort_chall_key_expired,	"rxgk-chall-key-exp")	\
 	EM(rxgk_abort_chall_no_key,		"rxgk-chall-nokey")	\
 	EM(rxgk_abort_chall_short,		"rxgk-chall-short")	\
diff --git a/net/rxrpc/af_rxrpc.c b/net/rxrpc/af_rxrpc.c
index a19c0fd3c51a..9eb6ca3c5ebf 100644
--- a/net/rxrpc/af_rxrpc.c
+++ b/net/rxrpc/af_rxrpc.c
@@ -598,10 +598,7 @@ static int rxrpc_sendmsg(struct socket *sock, struct msghdr *m, size_t len)
 		fallthrough;
 	case RXRPC_SERVER_BOUND:
 	case RXRPC_SERVER_LISTENING:
-		if (m->msg_flags & MSG_OOB)
-			ret = rxrpc_sendmsg_oob(rx, m, len);
-		else
-			ret = rxrpc_do_sendmsg(rx, m, len);
+		ret = rxrpc_do_sendmsg(rx, m, len);
 		/* The socket has been unlocked */
 		goto out;
 	default:
diff --git a/net/rxrpc/conn_event.c b/net/rxrpc/conn_event.c
index 611c790bc6d0..9146b9d4c2ac 100644
--- a/net/rxrpc/conn_event.c
+++ b/net/rxrpc/conn_event.c
@@ -279,10 +279,7 @@ static int rxrpc_process_event(struct rxrpc_connection *conn,
 
 	switch (sp->hdr.type) {
 	case RXRPC_PACKET_TYPE_CHALLENGE:
-		ret = conn->security->respond_to_challenge(conn, skb);
-		sp->chall.conn = NULL;
-		rxrpc_put_connection(conn, rxrpc_conn_put_challenge_input);
-		return ret;
+		return conn->security->respond_to_challenge(conn, skb);
 
 	case RXRPC_PACKET_TYPE_RESPONSE:
 		spin_lock_irq(&conn->state_lock);
@@ -425,66 +422,6 @@ static void rxrpc_post_packet_to_conn(struct rxrpc_connection *conn,
 	rxrpc_queue_conn(conn, rxrpc_conn_queue_rx_work);
 }
 
-/*
- * Post a CHALLENGE packet to the socket of one of a connection's calls so that
- * it can get application data to include in the packet, possibly querying
- * userspace.
- */
-static bool rxrpc_post_challenge(struct rxrpc_connection *conn,
-				 struct sk_buff *skb)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
-	struct rxrpc_call *call = NULL;
-	struct rxrpc_sock *rx;
-	bool respond = false, queued = false;
-
-	sp->chall.conn =
-		rxrpc_get_connection(conn, rxrpc_conn_get_challenge_input);
-
-	if (!conn->security->challenge_to_recvmsg) {
-		rxrpc_post_packet_to_conn(conn, skb);
-		return true;
-	}
-
-	rcu_read_lock();
-
-	for (int i = 0; i < ARRAY_SIZE(conn->channels); i++) {
-		if (conn->channels[i].call) {
-			call = conn->channels[i].call;
-			rx = rcu_dereference(call->socket);
-			if (!rx) {
-				call = NULL;
-				continue;
-			}
-
-			respond = true;
-			if (test_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags))
-				break;
-			call = NULL;
-		}
-	}
-
-	if (!respond) {
-		rcu_read_unlock();
-		rxrpc_put_connection(conn, rxrpc_conn_put_challenge_input);
-		sp->chall.conn = NULL;
-		return false;
-	}
-
-	if (call)
-		queued = rxrpc_notify_socket_oob(call, skb);
-	rcu_read_unlock();
-	if (call && !queued) {
-		rxrpc_put_connection(conn, rxrpc_conn_put_challenge_input);
-		sp->chall.conn = NULL;
-		return false;
-	}
-
-	if (!call)
-		rxrpc_post_packet_to_conn(conn, skb);
-	return true;
-}
-
 /*
  * Input a connection-level packet.
  */
@@ -513,7 +450,8 @@ bool rxrpc_input_conn_packet(struct rxrpc_connection *conn, struct sk_buff *skb)
 		}
 		if (!conn->security->validate_challenge(conn, skb))
 			return false;
-		return rxrpc_post_challenge(conn, skb);
+		rxrpc_post_packet_to_conn(conn, skb);
+		return true;
 
 	case RXRPC_PACKET_TYPE_RESPONSE:
 		if (rxrpc_is_conn_aborted(conn)) {
diff --git a/net/rxrpc/rxgk.c b/net/rxrpc/rxgk.c
index 77a67ace1d24..031c6719d33e 100644
--- a/net/rxrpc/rxgk.c
+++ b/net/rxrpc/rxgk.c
@@ -11,6 +11,7 @@
 #include <linux/skbuff.h>
 #include <linux/slab.h>
 #include <linux/key-type.h>
+#include <keys/user-type.h>
 #include "ar-internal.h"
 #include "rxgk_common.h"
 
@@ -837,7 +838,7 @@ static noinline ssize_t rxgk_insert_response_header(struct rxrpc_connection *con
  */
 static ssize_t rxgk_construct_authenticator(struct rxrpc_connection *conn,
 					    struct sk_buff *challenge,
-					    const struct krb5_buffer *appdata,
+					    const struct user_key_payload *appdata,
 					    struct sk_buff *response,
 					    size_t offset)
 {
@@ -859,20 +860,20 @@ static ssize_t rxgk_construct_authenticator(struct rxrpc_connection *conn,
 	if (ret < 0)
 		return -EPROTO;
 
-	a.appdata_len = htonl(appdata->len);
+	a.appdata_len = htonl(appdata->datalen);
 
 	ret = skb_store_bits(response, offset, &a, sizeof(a));
 	if (ret < 0)
 		return ret;
 	offset += sizeof(a);
 
-	if (appdata->len) {
-		ret = skb_store_bits(response, offset, appdata->data, appdata->len);
+	if (appdata->datalen) {
+		ret = skb_store_bits(response, offset, appdata->data, appdata->datalen);
 		if (ret < 0)
 			return ret;
-		offset += appdata->len;
+		offset += appdata->datalen;
 
-		ret = rxgk_pad_out(response, appdata->len, offset);
+		ret = rxgk_pad_out(response, appdata->datalen, offset);
 		if (ret < 0)
 			return ret;
 		offset += ret;
@@ -890,7 +891,7 @@ static ssize_t rxgk_construct_authenticator(struct rxrpc_connection *conn,
 	ret = skb_store_bits(response, offset, &b, sizeof(b));
 	if (ret < 0)
 		return ret;
-	return sizeof(a) + xdr_round_up(appdata->len) + sizeof(b);
+	return sizeof(a) + xdr_round_up(appdata->datalen) + sizeof(b);
 }
 
 static ssize_t rxgk_encrypt_authenticator(struct rxrpc_connection *conn,
@@ -923,7 +924,7 @@ static ssize_t rxgk_encrypt_authenticator(struct rxrpc_connection *conn,
  */
 static int rxgk_construct_response(struct rxrpc_connection *conn,
 				   struct sk_buff *challenge,
-				   struct krb5_buffer *appdata)
+				   const struct user_key_payload *appdata)
 {
 	struct rxrpc_skb_priv *csp, *rsp;
 	struct rxgk_context *gk;
@@ -936,11 +937,11 @@ static int rxgk_construct_response(struct rxrpc_connection *conn,
 	if (IS_ERR(gk))
 		return PTR_ERR(gk);
 
-	auth_len = 20 + (4 + appdata->len) + 12 + (1 + 4) * 4;
+	auth_len = 20 + xdr_object_len(appdata->datalen) + 12 + (1 + 4) * 4;
 	authx_len = crypto_krb5_how_much_buffer(gk->krb5, KRB5_ENCRYPT_MODE,
 						auth_len, &auth_offset);
 	len = sizeof(struct rxrpc_wire_header) +
-		8 + (4 + xdr_round_up(gk->key->ticket.len)) + (4 + authx_len);
+		8 + xdr_object_len(gk->key->ticket.len) + xdr_object_len(authx_len);
 
 	response = alloc_skb_with_frags(0, len, 0, &ret, GFP_NOFS);
 	if (!response)
@@ -1011,24 +1012,44 @@ static int rxgk_construct_response(struct rxrpc_connection *conn,
  * Respond to a challenge packet.
  */
 static int rxgk_respond_to_challenge(struct rxrpc_connection *conn,
-				     struct sk_buff *challenge,
-				     struct krb5_buffer *appdata)
+				     struct sk_buff *challenge)
 {
-	_enter("{%d,%x}", conn->debug_id, key_serial(conn->key));
+	struct user_key_payload dummy = {}, *appdata = &dummy;
+	int ret;
+
+	_enter("{%d,%u,%x,%x}",
+	       conn->debug_id, conn->service_id,
+	       key_serial(conn->key), key_serial(conn->bundle->app_data));
 
 	if (key_validate(conn->key) < 0)
 		return rxrpc_abort_conn(conn, NULL, RXGK_EXPIRED, -EPROTO,
 					rxgk_abort_chall_key_expired);
 
-	return rxgk_construct_response(conn, challenge, appdata);
-}
+	if (conn->bundle->app_data) {
+		struct key *app_key = conn->bundle->app_data;
+
+		/* The wrong sort of key shouldn't get this far. */
+		if (WARN_ON_ONCE(app_key->type != &key_type_user &&
+				 app_key->type != &key_type_logon))
+			return rxrpc_abort_conn(conn, NULL, RXGK_EXPIRED, -EKEYREJECTED,
+						rxgk_abort_chall_bad_app_key);
+
+		rcu_read_lock();
+		appdata = (struct user_key_payload *)
+			user_key_payload_rcu(conn->bundle->app_data);
+		if (appdata && !refcount_inc_not_zero(&appdata->ref))
+			appdata = NULL;
+		rcu_read_unlock();
+		if (!appdata)
+			return rxrpc_abort_conn(conn, NULL, RXGK_EXPIRED, -EKEYREVOKED,
+						rxgk_abort_chall_key_expired);
+	}
 
-static int rxgk_respond_to_challenge_no_appdata(struct rxrpc_connection *conn,
-						struct sk_buff *challenge)
-{
-	struct krb5_buffer appdata = {};
+	ret = rxgk_construct_response(conn, challenge, appdata);
 
-	return rxgk_respond_to_challenge(conn, challenge, &appdata);
+	if (appdata != &dummy)
+		put_user_key_payload(appdata);
+	return ret;
 }
 
 /**
@@ -1044,9 +1065,7 @@ static int rxgk_respond_to_challenge_no_appdata(struct rxrpc_connection *conn,
 int rxgk_kernel_respond_to_challenge(struct sk_buff *challenge,
 				     struct krb5_buffer *appdata)
 {
-	struct rxrpc_skb_priv *csp = rxrpc_skb(challenge);
-
-	return rxgk_respond_to_challenge(csp->chall.conn, challenge, appdata);
+	return -EINVAL;
 }
 EXPORT_SYMBOL(rxgk_kernel_respond_to_challenge);
 
@@ -1348,7 +1367,7 @@ const struct rxrpc_security rxgk_yfs = {
 	.validate_challenge		= rxgk_validate_challenge,
 	.challenge_to_recvmsg		= rxgk_challenge_to_recvmsg,
 	.sendmsg_respond_to_challenge	= rxgk_sendmsg_respond_to_challenge,
-	.respond_to_challenge		= rxgk_respond_to_challenge_no_appdata,
+	.respond_to_challenge		= rxgk_respond_to_challenge,
 	.verify_response		= rxgk_verify_response,
 	.clear				= rxgk_clear,
 	.default_decode_ticket		= rxgk_yfs_decode_ticket,



From dhowells at redhat.com  Wed Aug 12 04:01:26 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 12:01:26 +0100
Subject: [PATCH net v7 11/11] rxrpc: Remove OOB challenge/response code
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260812110129.979970-12-dhowells@redhat.com>

Currently, when a CHALLENGE packet comes in, it's queued in an OOB queue on
the AF_RXRPC socket that generated one of the calls on that connection for
the application (which might be in userspace) to service.  The application
then picks up the CHALLENGE and requests a RESPONSE packet be generated,
allowing the app to include app-specific data in it if appropriate.  There
is, however, no actual limit on the capacity of the CHALLENGE queue, and
this could be abused remotely - and also getting the OOB mechanism right
has proven tricky.

Further, by analogy with other AFS codebases, it's not actually necessary
to generate the application data in response to the CHALLENGE.  The reason
I did this was to set the encryption on the app-data to be the same as that
specified in the CHALLENGE as the server must be able to handle that.
However, it's sufficient to use the encoding type set in the token that is
going to be sent to the server; presumably the kerberos server knows that
the fileserver can handle that type - otherwise why tell the client to use
it?

This is the final part of the fix.  It removes all the OOB communication
stuff along with the UAPI elements related to them.

Fixes: 5800b1cf3fd8 ("rxrpc: Allow CHALLENGEs to the passed to the app for a RESPONSE")
Link: https://sashiko.dev/#/patchset/20260624163819.3017002-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Marc Dionne <marc.dionne at auristor.com>
cc: Jeffrey Altman <jaltman at auristor.com>
cc: Eric Dumazet <edumazet at google.com>
cc: "David S. Miller" <davem at davemloft.net>
cc: Jakub Kicinski <kuba at kernel.org>
cc: Paolo Abeni <pabeni at redhat.com>
cc: Simon Horman <horms at kernel.org>
cc: Jarkko Sakkinen <jarkko at kernel.org>
cc: linux-afs at lists.infradead.org
cc: keyrings at vger.kernel.org
cc: stable at kernel.org
---
 Documentation/networking/rxrpc.rst |   1 -
 fs/afs/cm_security.c               | 259 -------------------
 fs/afs/internal.h                  |   3 -
 fs/afs/main.c                      |   1 -
 fs/afs/rxrpc.c                     |  19 --
 fs/afs/server.c                    |   1 -
 include/net/af_rxrpc.h             |  23 --
 include/trace/events/rxrpc.h       |  10 +-
 include/uapi/linux/rxrpc.h         |   5 -
 net/rxrpc/Makefile                 |   1 -
 net/rxrpc/af_rxrpc.c               |  41 +--
 net/rxrpc/ar-internal.h            |  23 --
 net/rxrpc/conn_event.c             |   2 +-
 net/rxrpc/insecure.c               |   7 -
 net/rxrpc/oob.c                    | 387 -----------------------------
 net/rxrpc/recvmsg.c                |  82 +-----
 net/rxrpc/rxgk.c                   |  73 ------
 net/rxrpc/rxkad.c                  |  27 --
 net/rxrpc/server_key.c             |  40 ---
 19 files changed, 7 insertions(+), 998 deletions(-)
 delete mode 100644 net/rxrpc/oob.c

diff --git a/Documentation/networking/rxrpc.rst b/Documentation/networking/rxrpc.rst
index 8926dab8e2e6..1018cc9ff284 100644
--- a/Documentation/networking/rxrpc.rst
+++ b/Documentation/networking/rxrpc.rst
@@ -1153,7 +1153,6 @@ API Function Reference
 .. kernel-doc:: net/rxrpc/af_rxrpc.c
 .. kernel-doc:: net/rxrpc/call_object.c
 .. kernel-doc:: net/rxrpc/key.c
-.. kernel-doc:: net/rxrpc/oob.c
 .. kernel-doc:: net/rxrpc/peer_object.c
 .. kernel-doc:: net/rxrpc/recvmsg.c
 .. kernel-doc:: net/rxrpc/rxgk.c
diff --git a/fs/afs/cm_security.c b/fs/afs/cm_security.c
index 8f04fec20a2a..0d4700b062f6 100644
--- a/fs/afs/cm_security.c
+++ b/fs/afs/cm_security.c
@@ -20,11 +20,6 @@
 #define xdr_round_up(x) (round_up((x), sizeof(__be32)))
 #define xdr_len_object(x) (4 + round_up((x), sizeof(__be32)))
 
-#ifdef CONFIG_RXGK
-static int afs_create_yfs_cm_token(struct sk_buff *challenge,
-				   struct afs_server *server);
-#endif
-
 #ifdef CONFIG_RXGK
 /*
  * As the YFS RxGK appdata to be passed in the YFS.FS-service RESPONSE packet,
@@ -255,95 +250,6 @@ int afs_create_server_appdata(struct afs_server *server, struct key *key)
 	return ret;
 }
 
-/*
- * Respond to an RxGK challenge, adding appdata.
- */
-static int afs_respond_to_challenge(struct sk_buff *challenge)
-{
-#ifdef CONFIG_RXGK
-	struct krb5_buffer appdata = {};
-	struct afs_server *server;
-#endif
-	struct rxrpc_peer *peer;
-	unsigned long peer_data;
-	u16 service_id;
-	u8 security_index;
-
-	rxrpc_kernel_query_challenge(challenge, &peer, &peer_data,
-				     &service_id, &security_index);
-
-	_enter("%u,%u", service_id, security_index);
-
-	switch (service_id) {
-		/* We don't send CM_SERVICE RPCs, so don't expect a challenge
-		 * therefrom.
-		 */
-	case FS_SERVICE:
-	case VL_SERVICE:
-	case YFS_FS_SERVICE:
-	case YFS_VL_SERVICE:
-		break;
-	default:
-		pr_warn("Can't respond to unknown challenge %u:%u",
-			service_id, security_index);
-		return rxrpc_kernel_reject_challenge(challenge, RX_USER_ABORT, -EPROTO,
-						     afs_abort_unsupported_sec_class);
-	}
-
-	switch (security_index) {
-#ifdef CONFIG_RXKAD
-	case RXRPC_SECURITY_RXKAD:
-		return rxkad_kernel_respond_to_challenge(challenge);
-#endif
-
-#ifdef CONFIG_RXGK
-	case RXRPC_SECURITY_RXGK:
-		return rxgk_kernel_respond_to_challenge(challenge, &appdata);
-
-	case RXRPC_SECURITY_YFS_RXGK:
-		switch (service_id) {
-		case FS_SERVICE:
-		case YFS_FS_SERVICE:
-			server = (struct afs_server *)peer_data;
-			if (!server->cm_rxgk_appdata.data) {
-				mutex_lock(&server->cm_token_lock);
-				if (!server->cm_rxgk_appdata.data)
-					afs_create_yfs_cm_token(challenge, server);
-				mutex_unlock(&server->cm_token_lock);
-			}
-			if (server->cm_rxgk_appdata.data)
-				appdata = server->cm_rxgk_appdata;
-			break;
-		}
-		return rxgk_kernel_respond_to_challenge(challenge, &appdata);
-#endif
-
-	default:
-		return rxrpc_kernel_reject_challenge(challenge, RX_USER_ABORT, -EPROTO,
-						     afs_abort_unsupported_sec_class);
-	}
-}
-
-/*
- * Process the OOB message queue, processing challenge packets.
- */
-void afs_process_oob_queue(struct work_struct *work)
-{
-	struct afs_net *net = container_of(work, struct afs_net, rx_oob_work);
-	struct sk_buff *oob;
-	enum rxrpc_oob_type type;
-
-	while (READ_ONCE(net->live) &&
-	       (oob = rxrpc_kernel_dequeue_oob(net->socket, &type))) {
-		switch (type) {
-		case RXRPC_OOB_CHALLENGE:
-			afs_respond_to_challenge(oob);
-			break;
-		}
-		rxrpc_kernel_free_oob(oob);
-	}
-}
-
 #ifdef CONFIG_RXGK
 /*
  * Create a securities keyring for the cache manager and attach a key to it for
@@ -405,169 +311,4 @@ int afs_create_token_key(struct afs_net *net, struct socket *socket)
 	key_put(ring);
 	return ret;
 }
-
-/*
- * Create an YFS RxGK GSS token to use as a ticket to the specified fileserver.
- */
-static int afs_create_yfs_cm_token(struct sk_buff *challenge,
-				   struct afs_server *server)
-{
-	const struct krb5_enctype *conn_krb5, *token_krb5;
-	const struct krb5_buffer *token_key;
-	struct crypto_aead *aead;
-	struct scatterlist sg;
-	struct afs_net *net = server->cell->net;
-	const struct key *key = net->fs_cm_token_key;
-	size_t keysize, uuidsize, authsize, toksize, encsize, contsize, adatasize, offset;
-	__be32 caps[1] = {
-		[0] = htonl(AFS_CAP_ERROR_TRANSLATION),
-	};
-	__be32 *xdr;
-	void *appdata, *K0, *encbase;
-	u32 enctype;
-	int ret;
-
-	if (!key)
-		return -ENOKEY;
-
-	/* Assume that the fileserver is happy to use the same encoding type as
-	 * we were told to use by the token obtained by the user.
-	 */
-	enctype = rxgk_kernel_query_challenge(challenge);
-
-	conn_krb5 = crypto_krb5_find_enctype(enctype);
-	if (!conn_krb5)
-		return -ENOPKG;
-	token_krb5 = key->payload.data[0];
-	token_key = (const struct krb5_buffer *)&key->payload.data[2];
-
-	/* struct rxgk_key {
-	 *	afs_uint32	enctype;
-	 *	opaque		key<>;
-	 * };
-	 */
-	keysize = 4 + xdr_len_object(conn_krb5->key_len);
-
-	/* struct RXGK_AuthName {
-	 *	afs_int32	kind;
-	 *	opaque		data<AUTHDATAMAX>;
-	 *	opaque		display<AUTHPRINTABLEMAX>;
-	 * };
-	 */
-	uuidsize = sizeof(server->uuid);
-	authsize = 4 + xdr_len_object(uuidsize) + xdr_len_object(0);
-
-	/* struct RXGK_Token {
-	 *	rxgk_key		K0;
-	 *	RXGK_Level		level;
-	 *	rxgkTime		starttime;
-	 *	afs_int32		lifetime;
-	 *	afs_int32		bytelife;
-	 *	rxgkTime		expirationtime;
-	 *	struct RXGK_AuthName	identities<>;
-	 * };
-	 */
-	toksize = keysize + 8 + 4 + 4 + 8 + xdr_len_object(authsize);
-
-	offset = 0;
-	encsize = crypto_krb5_how_much_buffer(token_krb5, KRB5_ENCRYPT_MODE, toksize, &offset);
-
-	/* struct RXGK_TokenContainer {
-	 *	afs_int32	kvno;
-	 *	afs_int32	enctype;
-	 *	opaque		encrypted_token<>;
-	 * };
-	 */
-	contsize = 4 + 4 + xdr_len_object(encsize);
-
-	/* struct YFSAppData {
-	 *	opr_uuid	initiatorUuid;
-	 *	opr_uuid	acceptorUuid;
-	 *	Capabilities	caps;
-	 *	afs_int32	enctype;
-	 *	opaque		callbackKey<>;
-	 *	opaque		callbackToken<>;
-	 * };
-	 */
-	adatasize = 16 + 16 +
-		xdr_len_object(sizeof(caps)) +
-		4 +
-		xdr_len_object(conn_krb5->key_len) +
-		xdr_len_object(contsize);
-
-	ret = -ENOMEM;
-	appdata = kzalloc(adatasize, GFP_KERNEL);
-	if (!appdata)
-		goto out;
-	xdr = appdata;
-
-	memcpy(xdr, &net->uuid, 16);		/* appdata.initiatorUuid */
-	xdr += 16 / 4;
-	memcpy(xdr, &server->uuid, 16);		/* appdata.acceptorUuid */
-	xdr += 16 / 4;
-	*xdr++ = htonl(ARRAY_SIZE(caps));	/* appdata.caps.len */
-	memcpy(xdr, &caps, sizeof(caps));	/* appdata.caps */
-	xdr += ARRAY_SIZE(caps);
-	*xdr++ = htonl(conn_krb5->etype);	/* appdata.enctype */
-
-	*xdr++ = htonl(conn_krb5->key_len);	/* appdata.callbackKey.len */
-	K0 = xdr;
-	get_random_bytes(K0, conn_krb5->key_len); /* appdata.callbackKey.data */
-	xdr += xdr_round_up(conn_krb5->key_len) / 4;
-
-	*xdr++ = htonl(contsize);		/* appdata.callbackToken.len */
-	*xdr++ = htonl(1);			/* cont.kvno */
-	*xdr++ = htonl(token_krb5->etype);	/* cont.enctype */
-	*xdr++ = htonl(encsize);		/* cont.encrypted_token.len */
-
-	encbase = xdr;
-	xdr += offset / 4;
-	*xdr++ = htonl(conn_krb5->etype);	/* token.K0.enctype */
-	*xdr++ = htonl(conn_krb5->key_len);	/* token.K0.key.len */
-	memcpy(xdr, K0, conn_krb5->key_len);	/* token.K0.key.data */
-	xdr += xdr_round_up(conn_krb5->key_len) / 4;
-
-	*xdr++ = htonl(RXRPC_SECURITY_ENCRYPT);	/* token.level */
-	*xdr++ = htonl(0);			/* token.starttime */
-	*xdr++ = htonl(0);			/* " */
-	*xdr++ = htonl(0);			/* token.lifetime */
-	*xdr++ = htonl(0);			/* token.bytelife */
-	*xdr++ = htonl(0);			/* token.expirationtime */
-	*xdr++ = htonl(0);			/* " */
-	*xdr++ = htonl(1);			/* token.identities.count */
-	*xdr++ = htonl(0);			/* token.identities[0].kind */
-	*xdr++ = htonl(uuidsize);		/* token.identities[0].data.len */
-	memcpy(xdr, &server->uuid, uuidsize);
-	xdr += xdr_round_up(uuidsize) / 4;
-	*xdr++ = htonl(0);			/* token.identities[0].display.len */
-
-	xdr = encbase + xdr_round_up(encsize);
-
-	if ((unsigned long)xdr - (unsigned long)appdata != adatasize)
-		pr_err("Appdata size incorrect %lx != %zx\n",
-		       (unsigned long)xdr - (unsigned long)appdata, adatasize);
-
-	aead = crypto_krb5_prepare_encryption(token_krb5, token_key, RXGK_SERVER_ENC_TOKEN,
-					      GFP_KERNEL);
-	if (IS_ERR(aead)) {
-		ret = PTR_ERR(aead);
-		goto out_token;
-	}
-
-	sg_init_one(&sg, encbase, encsize);
-	ret = crypto_krb5_encrypt(token_krb5, aead, &sg, 1, encsize, offset, toksize, false);
-	if (ret < 0)
-		goto out_aead;
-
-	server->cm_rxgk_appdata.len  = adatasize;
-	server->cm_rxgk_appdata.data = appdata;
-	appdata = NULL;
-
-out_aead:
-	crypto_free_aead(aead);
-out_token:
-	kfree(appdata);
-out:
-	return ret;
-}
 #endif /* CONFIG_RXGK */
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index ce05595b7d3c..d2bcb567c31b 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -283,7 +283,6 @@ struct afs_net {
 	struct socket		*socket;
 	struct afs_call		*spare_incoming_call;
 	struct work_struct	charge_preallocation_work;
-	struct work_struct	rx_oob_work;
 	struct mutex		socket_mutex;
 	atomic_t		nr_outstanding_calls;
 	atomic_t		nr_superblocks;
@@ -548,7 +547,6 @@ struct afs_server {
 	struct work_struct	destroyer;	/* Work item to try and destroy a server */
 	struct timer_list	timer;		/* Management timer */
 	struct mutex		cm_token_lock;	/* Lock governing creation of appdata */
-	struct krb5_buffer	cm_rxgk_appdata; /* Appdata to be included in RESPONSE packet */
 	struct key		*yfs_rxgk_appdata; /* Appdata to be included in RESPONSE packet */
 	time64_t		unuse_time;	/* Time at which last unused */
 	unsigned long		flags;
@@ -1091,7 +1089,6 @@ extern bool afs_cm_incoming_call(struct afs_call *);
  * cm_security.c
  */
 int afs_create_server_appdata(struct afs_server *server, struct key *key);
-void afs_process_oob_queue(struct work_struct *work);
 #ifdef CONFIG_RXGK
 int afs_create_token_key(struct afs_net *net, struct socket *socket);
 #else
diff --git a/fs/afs/main.c b/fs/afs/main.c
index 7a883c59976f..6353feaa5fb4 100644
--- a/fs/afs/main.c
+++ b/fs/afs/main.c
@@ -73,7 +73,6 @@ static int __net_init afs_net_init(struct net *net_ns)
 	generate_random_uuid((unsigned char *)&net->uuid);
 
 	INIT_WORK(&net->charge_preallocation_work, afs_charge_preallocation);
-	INIT_WORK(&net->rx_oob_work, afs_process_oob_queue);
 	mutex_init(&net->socket_mutex);
 
 	net->cells = RB_ROOT;
diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c
index 7aee2814a9e9..6efcc4cda2bc 100644
--- a/fs/afs/rxrpc.c
+++ b/fs/afs/rxrpc.c
@@ -25,14 +25,12 @@ static void afs_process_async_call(struct work_struct *);
 static void afs_rx_new_call(struct sock *, struct rxrpc_call *, unsigned long);
 static void afs_rx_discard_new_call(struct rxrpc_call *, unsigned long);
 static void afs_rx_attach(struct rxrpc_call *rxcall, unsigned long user_call_ID);
-static void afs_rx_notify_oob(struct sock *sk, struct sk_buff *oob);
 static int afs_deliver_cm_op_id(struct afs_call *);
 
 static const struct rxrpc_kernel_ops afs_rxrpc_callback_ops = {
 	.notify_new_call	= afs_rx_new_call,
 	.discard_new_call	= afs_rx_discard_new_call,
 	.user_attach_call	= afs_rx_attach,
-	.notify_oob		= afs_rx_notify_oob,
 };
 
 /* asynchronous incoming call initial processing */
@@ -74,10 +72,6 @@ int afs_open_socket(struct afs_net *net)
 	if (ret < 0)
 		goto error_2;
 
-	ret = rxrpc_sock_set_manage_response(socket->sk, true);
-	if (ret < 0)
-		goto error_2;
-
 	ret = afs_create_token_key(net, socket);
 	if (ret < 0)
 		pr_err("Couldn't create RxGK CM key: %d\n", ret);
@@ -128,7 +122,6 @@ void afs_close_socket(struct afs_net *net)
 	_enter("");
 
 	cancel_work_sync(&net->charge_preallocation_work);
-	cancel_work_sync(&net->rx_oob_work);
 	/* Future work items should now see ->live is false. */
 
 	kernel_listen(net->socket, 0);
@@ -149,7 +142,6 @@ void afs_close_socket(struct afs_net *net)
 
 	kernel_sock_shutdown(net->socket, SHUT_RDWR);
 	flush_workqueue(afs_async_calls);
-	cancel_work_sync(&net->rx_oob_work);
 	net->socket->sk->sk_user_data = NULL;
 	sock_release(net->socket);
 	key_put(net->fs_cm_token_key);
@@ -1008,14 +1000,3 @@ noinline int afs_protocol_error(struct afs_call *call,
 		call->unmarshalling_error = true;
 	return -EBADMSG;
 }
-
-/*
- * Wake up OOB notification processing.
- */
-static void afs_rx_notify_oob(struct sock *sk, struct sk_buff *oob)
-{
-	struct afs_net *net = sk->sk_user_data;
-
-	if (READ_ONCE(net->live))
-		queue_work(afs_wq, &net->rx_oob_work);
-}
diff --git a/fs/afs/server.c b/fs/afs/server.c
index b08d9080b0c5..4c3a7bd0f7d8 100644
--- a/fs/afs/server.c
+++ b/fs/afs/server.c
@@ -398,7 +398,6 @@ static void afs_server_rcu(struct rcu_head *rcu)
 	afs_put_endpoint_state(rcu_access_pointer(server->endpoint_state),
 			       afs_estate_trace_put_server);
 	afs_put_cell(server->cell, afs_cell_trace_put_server);
-	kfree(server->cm_rxgk_appdata.data);
 	key_put(server->yfs_rxgk_appdata);
 	kfree(server);
 }
diff --git a/include/net/af_rxrpc.h b/include/net/af_rxrpc.h
index 19c61a2f5af3..809ef95b92a1 100644
--- a/include/net/af_rxrpc.h
+++ b/include/net/af_rxrpc.h
@@ -25,10 +25,6 @@ enum rxrpc_interruptibility {
 	RXRPC_UNINTERRUPTIBLE,	/* Call should not be interruptible at all */
 };
 
-enum rxrpc_oob_type {
-	RXRPC_OOB_CHALLENGE,	/* Security challenge for a connection */
-};
-
 /*
  * Debug ID counter for tracing.
  */
@@ -42,7 +38,6 @@ struct rxrpc_kernel_ops {
 				unsigned long user_call_ID);
 	void (*discard_new_call)(struct rxrpc_call *call, unsigned long user_call_ID);
 	void (*user_attach_call)(struct rxrpc_call *call, unsigned long user_call_ID);
-	void (*notify_oob)(struct sock *sk, struct sk_buff *oob);
 };
 
 typedef void (*rxrpc_notify_rx_t)(struct sock *, struct rxrpc_call *,
@@ -92,24 +87,6 @@ bool rxrpc_kernel_check_life(const struct socket *, const struct rxrpc_call *);
 
 int rxrpc_sock_set_min_security_level(struct sock *sk, unsigned int val);
 int rxrpc_sock_set_security_keyring(struct sock *, struct key *);
-int rxrpc_sock_set_manage_response(struct sock *sk, bool set);
-
-enum rxrpc_oob_type rxrpc_kernel_query_oob(struct sk_buff *oob,
-					   struct rxrpc_peer **_peer,
-					   unsigned long *_peer_appdata);
-struct sk_buff *rxrpc_kernel_dequeue_oob(struct socket *sock,
-					 enum rxrpc_oob_type *_type);
-void rxrpc_kernel_free_oob(struct sk_buff *oob);
-void rxrpc_kernel_query_challenge(struct sk_buff *challenge,
-				  struct rxrpc_peer **_peer,
-				  unsigned long *_peer_appdata,
-				  u16 *_service_id, u8 *_security_index);
-int rxrpc_kernel_reject_challenge(struct sk_buff *challenge, u32 abort_code,
-				  int error, enum rxrpc_abort_reason why);
-int rxkad_kernel_respond_to_challenge(struct sk_buff *challenge);
-u32 rxgk_kernel_query_challenge(struct sk_buff *challenge);
-int rxgk_kernel_respond_to_challenge(struct sk_buff *challenge,
-				     struct krb5_buffer *appdata);
 void rxrpc_kernel_query_key(const struct key *key, u8 *_security_index,
 			    u32 *_krb5_enctype);
 u8 rxrpc_kernel_query_call_security(struct rxrpc_call *call,
diff --git a/include/trace/events/rxrpc.h b/include/trace/events/rxrpc.h
index 48e774034195..0eb5158f4e3d 100644
--- a/include/trace/events/rxrpc.h
+++ b/include/trace/events/rxrpc.h
@@ -169,7 +169,6 @@
 	EM(rxrpc_skb_get_conn_secured,		"GET conn-secd") \
 	EM(rxrpc_skb_get_conn_work,		"GET conn-work") \
 	EM(rxrpc_skb_get_local_work,		"GET locl-work") \
-	EM(rxrpc_skb_get_post_oob,		"GET post-oob ") \
 	EM(rxrpc_skb_get_reject_work,		"GET rej-work ") \
 	EM(rxrpc_skb_get_to_recvmsg,		"GET to-recv  ") \
 	EM(rxrpc_skb_get_to_recvmsg_oos,	"GET to-recv-o") \
@@ -186,18 +185,15 @@
 	EM(rxrpc_skb_put_error_report,		"PUT error-rep") \
 	EM(rxrpc_skb_put_input,			"PUT input    ") \
 	EM(rxrpc_skb_put_jumbo_subpacket,	"PUT jumbo-sub") \
-	EM(rxrpc_skb_put_oob,			"PUT oob      ") \
 	EM(rxrpc_skb_put_old_response,		"PUT old-resp ") \
 	EM(rxrpc_skb_put_purge,			"PUT purge    ") \
-	EM(rxrpc_skb_put_purge_oob,		"PUT purge-oob") \
 	EM(rxrpc_skb_put_response,		"PUT response ") \
 	EM(rxrpc_skb_put_response_copy,		"PUT resp-cpy ") \
 	EM(rxrpc_skb_put_rotate,		"PUT rotate   ") \
 	EM(rxrpc_skb_put_unknown,		"PUT unknown  ") \
 	EM(rxrpc_skb_see_conn_work,		"SEE conn-work") \
-	EM(rxrpc_skb_see_oob_challenge,		"SEE oob-chall") \
+	EM(rxrpc_skb_see_challenge,		"SEE challenge") \
 	EM(rxrpc_skb_see_recvmsg,		"SEE recvmsg  ") \
-	EM(rxrpc_skb_see_recvmsg_oob,		"SEE recvm-oob") \
 	EM(rxrpc_skb_see_reject,		"SEE reject   ") \
 	EM(rxrpc_skb_see_rotate,		"SEE rotate   ") \
 	EM(rxrpc_skb_see_unshare_nomem,		"SEE unshar-nm") \
@@ -264,7 +260,6 @@
 	EM(rxrpc_conn_free,			"FREE        ") \
 	EM(rxrpc_conn_get_activate_call,	"GET act-call") \
 	EM(rxrpc_conn_get_call_input,		"GET inp-call") \
-	EM(rxrpc_conn_get_challenge_input,	"GET inp-chal") \
 	EM(rxrpc_conn_get_conn_input,		"GET inp-conn") \
 	EM(rxrpc_conn_get_idle,			"GET idle    ") \
 	EM(rxrpc_conn_get_poke_abort,		"GET pk-abort") \
@@ -276,12 +271,10 @@
 	EM(rxrpc_conn_new_service,		"NEW service ") \
 	EM(rxrpc_conn_put_call,			"PUT call    ") \
 	EM(rxrpc_conn_put_call_input,		"PUT inp-call") \
-	EM(rxrpc_conn_put_challenge_input,	"PUT inp-chal") \
 	EM(rxrpc_conn_put_conn_input,		"PUT inp-conn") \
 	EM(rxrpc_conn_put_discard_idle,		"PUT disc-idl") \
 	EM(rxrpc_conn_put_local_dead,		"PUT loc-dead") \
 	EM(rxrpc_conn_put_noreuse,		"PUT noreuse ") \
-	EM(rxrpc_conn_put_oob,			"PUT oob     ") \
 	EM(rxrpc_conn_put_poke,			"PUT poke    ") \
 	EM(rxrpc_conn_put_service_reaped,	"PUT svc-reap") \
 	EM(rxrpc_conn_put_unbundle,		"PUT unbundle") \
@@ -391,7 +384,6 @@
 	EM(rxrpc_recvmsg_full,			"FULL") \
 	EM(rxrpc_recvmsg_hole,			"HOLE") \
 	EM(rxrpc_recvmsg_next,			"NEXT") \
-	EM(rxrpc_recvmsg_oobq,			"OOBQ") \
 	EM(rxrpc_recvmsg_requeue,		"REQU") \
 	EM(rxrpc_recvmsg_return,		"RETN") \
 	EM(rxrpc_recvmsg_terminal,		"TERM") \
diff --git a/include/uapi/linux/rxrpc.h b/include/uapi/linux/rxrpc.h
index bcdfdf9c67a1..7d6a50151931 100644
--- a/include/uapi/linux/rxrpc.h
+++ b/include/uapi/linux/rxrpc.h
@@ -58,11 +58,6 @@ enum rxrpc_cmsg_type {
 	RXRPC_TX_LENGTH		= 12,	/* -s-: Total length of Tx data */
 	RXRPC_SET_CALL_TIMEOUT	= 13,	/* -s-: Set one or more call timeouts */
 	RXRPC_CHARGE_ACCEPT	= 14,	/* Ss-: Charge the accept pool with a user call ID */
-	RXRPC_OOB_ID		= 15,	/* -sr: OOB message ID */
-	RXRPC_CHALLENGED	= 16,	/* C-r: Info on a received CHALLENGE */
-	RXRPC_RESPOND		= 17,	/* Cs-: Respond to a challenge */
-	RXRPC_RESPONDED		= 18,	/* S-r: Data received in RESPONSE */
-	RXRPC_RESP_RXGK_APPDATA	= 19,	/* Cs-: RESPONSE: RxGK app data to include */
 	RXRPC_RESPONSE_APPDATA	= 20,	/* Cs-: User key holding app data for RESPONSE */
 	RXRPC__SUPPORTED
 };
diff --git a/net/rxrpc/Makefile b/net/rxrpc/Makefile
index f994f9f30a29..f7a5e0a592ee 100644
--- a/net/rxrpc/Makefile
+++ b/net/rxrpc/Makefile
@@ -24,7 +24,6 @@ rxrpc-y := \
 	local_object.o \
 	misc.o \
 	net_ns.o \
-	oob.o \
 	output.o \
 	peer_event.o \
 	peer_object.o \
diff --git a/net/rxrpc/af_rxrpc.c b/net/rxrpc/af_rxrpc.c
index 9eb6ca3c5ebf..9d5ce1c7ad0d 100644
--- a/net/rxrpc/af_rxrpc.c
+++ b/net/rxrpc/af_rxrpc.c
@@ -633,7 +633,7 @@ static int rxrpc_setsockopt(struct socket *sock, int level, int optname,
 			    sockptr_t optval, unsigned int optlen)
 {
 	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
-	unsigned int min_sec_level, val;
+	unsigned int min_sec_level;
 	u16 service_upgrade[2];
 	int ret;
 
@@ -710,23 +710,7 @@ static int rxrpc_setsockopt(struct socket *sock, int level, int optname,
 
 		case RXRPC_MANAGE_RESPONSE:
 			ret = -EINVAL;
-			if (optlen != sizeof(unsigned int))
-				goto error;
-			ret = -EISCONN;
-			if (rx->sk.sk_state != RXRPC_UNBOUND)
-				goto error;
-			ret = copy_safe_from_sockptr(&val, sizeof(val),
-						     optval, optlen);
-			if (ret)
-				goto error;
-			ret = -EINVAL;
-			if (val > 1)
-				goto error;
-			if (val)
-				set_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-			else
-				clear_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-			goto success;
+			goto error;
 
 		default:
 			break;
@@ -835,8 +819,6 @@ static int rxrpc_create(struct net *net, struct socket *sock, int protocol,
 	rx->calls = RB_ROOT;
 
 	spin_lock_init(&rx->incoming_lock);
-	skb_queue_head_init(&rx->recvmsg_oobq);
-	rx->pending_oobq = RB_ROOT;
 	INIT_LIST_HEAD(&rx->sock_calls);
 	INIT_LIST_HEAD(&rx->to_be_accepted);
 	INIT_LIST_HEAD(&rx->recvmsg_q);
@@ -884,23 +866,6 @@ static int rxrpc_shutdown(struct socket *sock, int flags)
 	return ret;
 }
 
-/*
- * Purge the out-of-band queue.
- */
-static void rxrpc_purge_oob_queue(struct sock *sk)
-{
-	struct rxrpc_sock *rx = rxrpc_sk(sk);
-	struct sk_buff *skb;
-
-	while ((skb = skb_dequeue(&rx->recvmsg_oobq)))
-		rxrpc_kernel_free_oob(skb);
-	while (!RB_EMPTY_ROOT(&rx->pending_oobq)) {
-		skb = rb_entry(rx->pending_oobq.rb_node, struct sk_buff, rbnode);
-		rb_erase(&skb->rbnode, &rx->pending_oobq);
-		rxrpc_kernel_free_oob(skb);
-	}
-}
-
 /*
  * RxRPC socket destructor
  */
@@ -908,7 +873,6 @@ static void rxrpc_sock_destructor(struct sock *sk)
 {
 	_enter("%p", sk);
 
-	rxrpc_purge_oob_queue(sk);
 	rxrpc_purge_queue(&sk->sk_receive_queue);
 
 	WARN_ON(refcount_read(&sk->sk_wmem_alloc));
@@ -961,7 +925,6 @@ static int rxrpc_release_sock(struct sock *sk)
 	rxrpc_discard_prealloc(rx);
 	rxrpc_release_calls_on_socket(rx);
 	flush_workqueue(rxrpc_workqueue);
-	rxrpc_purge_oob_queue(sk);
 	rxrpc_purge_queue(&sk->sk_receive_queue);
 
 	rxrpc_unuse_local(rx->local, rxrpc_local_unuse_release_sock);
diff --git a/net/rxrpc/ar-internal.h b/net/rxrpc/ar-internal.h
index 6e16df7173d3..83858c45fa4a 100644
--- a/net/rxrpc/ar-internal.h
+++ b/net/rxrpc/ar-internal.h
@@ -171,9 +171,6 @@ struct rxrpc_sock {
 	const struct rxrpc_kernel_ops *app_ops;	/* Table of kernel app notification funcs */
 	struct rxrpc_local	*local;		/* local endpoint */
 	struct rxrpc_backlog	*backlog;	/* Preallocation for services */
-	struct sk_buff_head	recvmsg_oobq;	/* OOB messages for recvmsg to pick up */
-	struct rb_root		pending_oobq;	/* OOB messages awaiting userspace to respond to */
-	u64			oob_id_counter;	/* OOB message ID counter */
 	spinlock_t		incoming_lock;	/* Incoming call vs service shutdown lock */
 	struct list_head	sock_calls;	/* List of calls owned by this socket */
 	struct list_head	to_be_accepted;	/* calls awaiting acceptance */
@@ -184,7 +181,6 @@ struct rxrpc_sock {
 	struct rb_root		calls;		/* User ID -> call mapping */
 	unsigned long		flags;
 #define RXRPC_SOCK_CONNECTED		0	/* connect_srx is set */
-#define RXRPC_SOCK_MANAGE_RESPONSE	1	/* User wants to manage RESPONSE packets */
 	rwlock_t		call_lock;	/* lock for calls */
 	u32			min_sec_level;	/* minimum security level */
 #define RXRPC_SECURITY_MAX	RXRPC_SECURITY_ENCRYPT
@@ -241,7 +237,6 @@ struct rxrpc_skb_priv {
 			u8		reason;		/* Reason for ack */
 		} ack;
 		struct {
-			struct rxrpc_connection *conn;	/* Connection referred to */
 			union {
 				u32 rxkad_nonce;
 			};
@@ -310,17 +305,6 @@ struct rxrpc_security {
 	bool (*validate_challenge)(struct rxrpc_connection *conn,
 				   struct sk_buff *skb);
 
-	/* Fill out the cmsg for recvmsg() to pass on a challenge to userspace.
-	 * The security class gets to add additional information.
-	 */
-	int (*challenge_to_recvmsg)(struct rxrpc_connection *conn,
-				    struct sk_buff *challenge,
-				    struct msghdr *msg);
-
-	/* Parse sendmsg() control message and respond to challenge. */
-	int (*sendmsg_respond_to_challenge)(struct sk_buff *challenge,
-					    struct msghdr *msg);
-
 	/* respond to a challenge */
 	int (*respond_to_challenge)(struct rxrpc_connection *conn,
 				    struct sk_buff *challenge);
@@ -1381,13 +1365,6 @@ static inline struct rxrpc_net *rxrpc_net(struct net *net)
 	return net_generic(net, rxrpc_net_id);
 }
 
-/*
- * oob.c
- */
-bool rxrpc_notify_socket_oob(struct rxrpc_call *call, struct sk_buff *skb);
-void rxrpc_add_pending_oob(struct rxrpc_sock *rx, struct sk_buff *skb);
-int rxrpc_sendmsg_oob(struct rxrpc_sock *rx, struct msghdr *msg, size_t len);
-
 /*
  * output.c
  */
diff --git a/net/rxrpc/conn_event.c b/net/rxrpc/conn_event.c
index 9146b9d4c2ac..267f2d0f883d 100644
--- a/net/rxrpc/conn_event.c
+++ b/net/rxrpc/conn_event.c
@@ -442,7 +442,7 @@ bool rxrpc_input_conn_packet(struct rxrpc_connection *conn, struct sk_buff *skb)
 		return true;
 
 	case RXRPC_PACKET_TYPE_CHALLENGE:
-		rxrpc_see_skb(skb, rxrpc_skb_see_oob_challenge);
+		rxrpc_see_skb(skb, rxrpc_skb_see_challenge);
 		if (rxrpc_is_conn_aborted(conn)) {
 			if (conn->completion == RXRPC_CALL_LOCALLY_ABORTED)
 				rxrpc_send_conn_abort(conn);
diff --git a/net/rxrpc/insecure.c b/net/rxrpc/insecure.c
index 0b39046bdc61..c083c82c4e7f 100644
--- a/net/rxrpc/insecure.c
+++ b/net/rxrpc/insecure.c
@@ -47,12 +47,6 @@ static bool none_validate_challenge(struct rxrpc_connection *conn,
 	return true;
 }
 
-static int none_sendmsg_respond_to_challenge(struct sk_buff *challenge,
-					     struct msghdr *msg)
-{
-	return -EINVAL;
-}
-
 static int none_verify_response(struct rxrpc_connection *conn,
 				struct sk_buff *response_skb,
 				void *response, unsigned int len)
@@ -88,7 +82,6 @@ const struct rxrpc_security rxrpc_no_security = {
 	.secure_packet			= none_secure_packet,
 	.verify_packet			= none_verify_packet,
 	.validate_challenge		= none_validate_challenge,
-	.sendmsg_respond_to_challenge	= none_sendmsg_respond_to_challenge,
 	.verify_response		= none_verify_response,
 	.clear				= none_clear,
 };
diff --git a/net/rxrpc/oob.c b/net/rxrpc/oob.c
deleted file mode 100644
index c80ee2487d09..000000000000
--- a/net/rxrpc/oob.c
+++ /dev/null
@@ -1,387 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/* Out of band message handling (e.g. challenge-response)
- *
- * Copyright (C) 2025 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells at redhat.com)
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/net.h>
-#include <linux/gfp.h>
-#include <linux/skbuff.h>
-#include <linux/export.h>
-#include <linux/sched/signal.h>
-#include <net/sock.h>
-#include <net/af_rxrpc.h>
-#include "ar-internal.h"
-
-enum rxrpc_oob_command {
-	RXRPC_OOB_CMD_UNSET,
-	RXRPC_OOB_CMD_RESPOND,
-} __mode(byte);
-
-struct rxrpc_oob_params {
-	u64			oob_id;		/* ID number of message if reply */
-	s32			abort_code;
-	enum rxrpc_oob_command	command;
-	bool			have_oob_id:1;
-};
-
-/*
- * Post an out-of-band message for attention by the socket or kernel service
- * associated with a reference call.
- */
-bool rxrpc_notify_socket_oob(struct rxrpc_call *call, struct sk_buff *skb)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(skb);
-	struct rxrpc_sock *rx;
-	struct sock *sk;
-	bool queued = false;
-
-	rcu_read_lock();
-
-	rx = rcu_dereference(call->socket);
-	if (rx) {
-		sk = &rx->sk;
-		spin_lock_irq(&rx->recvmsg_lock);
-
-		if (sk->sk_state < RXRPC_CLOSE) {
-			skb->skb_mstamp_ns = rx->oob_id_counter++;
-			rxrpc_get_skb(skb, rxrpc_skb_get_post_oob);
-			skb_queue_tail(&rx->recvmsg_oobq, skb);
-			queued = true;
-
-			trace_rxrpc_notify_socket(call->debug_id, sp->hdr.serial);
-			if (rx->app_ops)
-				rx->app_ops->notify_oob(sk, skb);
-		}
-
-		spin_unlock_irq(&rx->recvmsg_lock);
-		if (queued && !rx->app_ops && !sock_flag(sk, SOCK_DEAD))
-			sk->sk_data_ready(sk);
-	}
-
-	rcu_read_unlock();
-	return queued;
-}
-
-/*
- * Locate the OOB message to respond to by its ID.
- */
-static struct sk_buff *rxrpc_find_pending_oob(struct rxrpc_sock *rx, u64 oob_id)
-{
-	struct rb_node *p;
-	struct sk_buff *skb;
-
-	p = rx->pending_oobq.rb_node;
-	while (p) {
-		skb = rb_entry(p, struct sk_buff, rbnode);
-
-		if (oob_id < skb->skb_mstamp_ns)
-			p = p->rb_left;
-		else if (oob_id > skb->skb_mstamp_ns)
-			p = p->rb_right;
-		else
-			return skb;
-	}
-
-	return NULL;
-}
-
-/*
- * Add an OOB message into the pending-response set.  We always assign the next
- * value from a 64-bit counter to the oob_id, so just assume we're always going
- * to be on the right-hand edge of the tree and that the counter won't wrap.
- * The tree is also given a ref to the message.
- */
-void rxrpc_add_pending_oob(struct rxrpc_sock *rx, struct sk_buff *skb)
-{
-	struct rb_node **pp = &rx->pending_oobq.rb_node, *p = NULL;
-
-	while (*pp) {
-		p = *pp;
-		pp = &(*pp)->rb_right;
-	}
-
-	rb_link_node(&skb->rbnode, p, pp);
-	rb_insert_color(&skb->rbnode, &rx->pending_oobq);
-}
-
-/*
- * Extract control messages from the sendmsg() control buffer.
- */
-static int rxrpc_sendmsg_oob_cmsg(struct msghdr *msg, struct rxrpc_oob_params *p)
-{
-	struct cmsghdr *cmsg;
-	int len;
-
-	if (msg->msg_controllen == 0)
-		return -EINVAL;
-
-	for_each_cmsghdr(cmsg, msg) {
-		if (!CMSG_OK(msg, cmsg))
-			return -EINVAL;
-
-		len = cmsg->cmsg_len - sizeof(struct cmsghdr);
-		_debug("CMSG %d, %d, %d",
-		       cmsg->cmsg_level, cmsg->cmsg_type, len);
-
-		if (cmsg->cmsg_level != SOL_RXRPC)
-			continue;
-
-		switch (cmsg->cmsg_type) {
-		case RXRPC_OOB_ID:
-			if (len != sizeof(p->oob_id) || p->have_oob_id)
-				return -EINVAL;
-			memcpy(&p->oob_id, CMSG_DATA(cmsg), sizeof(p->oob_id));
-			p->have_oob_id = true;
-			break;
-		case RXRPC_RESPOND:
-			if (p->command != RXRPC_OOB_CMD_UNSET)
-				return -EINVAL;
-			p->command = RXRPC_OOB_CMD_RESPOND;
-			break;
-		case RXRPC_ABORT:
-			if (len != sizeof(p->abort_code) || p->abort_code)
-				return -EINVAL;
-			memcpy(&p->abort_code, CMSG_DATA(cmsg), sizeof(p->abort_code));
-			if (p->abort_code == 0)
-				return -EINVAL;
-			break;
-		case RXRPC_RESP_RXGK_APPDATA:
-			if (p->command != RXRPC_OOB_CMD_RESPOND)
-				return -EINVAL;
-			break;
-		default:
-			return -EINVAL;
-		}
-	}
-
-	switch (p->command) {
-	case RXRPC_OOB_CMD_RESPOND:
-		if (!p->have_oob_id)
-			return -EBADSLT;
-		break;
-	default:
-		return -EINVAL;
-	}
-
-	return 0;
-}
-
-/*
- * Allow userspace to respond to an OOB using sendmsg().
- */
-static int rxrpc_respond_to_oob(struct rxrpc_sock *rx,
-				struct rxrpc_oob_params *p,
-				struct msghdr *msg)
-{
-	struct rxrpc_connection *conn;
-	struct rxrpc_skb_priv *sp;
-	struct sk_buff *skb;
-	int ret;
-
-	skb = rxrpc_find_pending_oob(rx, p->oob_id);
-	if (skb)
-		rb_erase(&skb->rbnode, &rx->pending_oobq);
-	release_sock(&rx->sk);
-	if (!skb)
-		return -EBADSLT;
-
-	sp = rxrpc_skb(skb);
-
-	switch (p->command) {
-	case RXRPC_OOB_CMD_RESPOND:
-		ret = -EPROTO;
-		if (skb->mark != RXRPC_OOB_CHALLENGE)
-			break;
-		conn = sp->chall.conn;
-		ret = -EOPNOTSUPP;
-		if (!conn->security->sendmsg_respond_to_challenge)
-			break;
-		if (p->abort_code) {
-			rxrpc_abort_conn(conn, NULL, p->abort_code, -ECONNABORTED,
-					 rxrpc_abort_response_sendmsg);
-			ret = 0;
-		} else {
-			ret = conn->security->sendmsg_respond_to_challenge(skb, msg);
-		}
-		break;
-	default:
-		ret = -EINVAL;
-		break;
-	}
-
-	switch (skb->mark) {
-	case RXRPC_OOB_CHALLENGE:
-		rxrpc_put_connection(sp->chall.conn, rxrpc_conn_put_oob);
-		break;
-	}
-	rxrpc_free_skb(skb, rxrpc_skb_put_oob);
-	return ret;
-}
-
-/*
- * Send an out-of-band message or respond to a received out-of-band message.
- * - caller gives us the socket lock
- * - the socket may be either a client socket or a server socket
- */
-int rxrpc_sendmsg_oob(struct rxrpc_sock *rx, struct msghdr *msg, size_t len)
-{
-	struct rxrpc_oob_params p = {};
-	int ret;
-
-	_enter("");
-
-	ret = rxrpc_sendmsg_oob_cmsg(msg, &p);
-	if (ret < 0)
-		goto error_release_sock;
-
-	if (p.have_oob_id)
-		return rxrpc_respond_to_oob(rx, &p, msg);
-
-	release_sock(&rx->sk);
-
-	switch (p.command) {
-	default:
-		ret = -EINVAL;
-		break;
-	}
-
-	_leave(" = %d", ret);
-	return ret;
-
-error_release_sock:
-	release_sock(&rx->sk);
-	return ret;
-}
-
-/**
- * rxrpc_kernel_query_oob - Query the parameters of an out-of-band message
- * @oob: The message to query
- * @_peer: Where to return the peer record
- * @_peer_appdata: The application data attached to a peer record
- *
- * Extract useful parameters from an out-of-band message.  The source peer
- * parameters are returned through the argument list and the message type is
- * returned.
- *
- * Return:
- * * %RXRPC_OOB_CHALLENGE - Challenge wanting a response.
- */
-enum rxrpc_oob_type rxrpc_kernel_query_oob(struct sk_buff *oob,
-					   struct rxrpc_peer **_peer,
-					   unsigned long *_peer_appdata)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(oob);
-	enum rxrpc_oob_type type = oob->mark;
-
-	switch (type) {
-	case RXRPC_OOB_CHALLENGE:
-		*_peer		= sp->chall.conn->peer;
-		*_peer_appdata	= sp->chall.conn->peer->app_data;
-		break;
-	default:
-		WARN_ON_ONCE(1);
-		*_peer		= NULL;
-		*_peer_appdata	= 0;
-		break;
-	}
-
-	return type;
-}
-EXPORT_SYMBOL(rxrpc_kernel_query_oob);
-
-/**
- * rxrpc_kernel_dequeue_oob - Dequeue and return the front OOB message
- * @sock: The socket to query
- * @_type: Where to return the message type
- *
- * Dequeue the front OOB message, if there is one, and return it and
- * its type.
- *
- * Return: The sk_buff representing the OOB message or %NULL if the queue was
- * empty.
- */
-struct sk_buff *rxrpc_kernel_dequeue_oob(struct socket *sock,
-					 enum rxrpc_oob_type *_type)
-{
-	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
-	struct sk_buff *oob;
-
-	oob = skb_dequeue(&rx->recvmsg_oobq);
-	if (oob)
-		*_type = oob->mark;
-	return oob;
-}
-EXPORT_SYMBOL(rxrpc_kernel_dequeue_oob);
-
-/**
- * rxrpc_kernel_free_oob - Free an out-of-band message
- * @oob: The OOB message to free
- *
- * Free an OOB message along with any resources it holds.
- */
-void rxrpc_kernel_free_oob(struct sk_buff *oob)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(oob);
-
-	switch (oob->mark) {
-	case RXRPC_OOB_CHALLENGE:
-		rxrpc_put_connection(sp->chall.conn, rxrpc_conn_put_oob);
-		break;
-	}
-
-	rxrpc_free_skb(oob, rxrpc_skb_put_purge_oob);
-}
-EXPORT_SYMBOL(rxrpc_kernel_free_oob);
-
-/**
- * rxrpc_kernel_query_challenge - Query the parameters of a challenge
- * @challenge: The challenge to query
- * @_peer: Where to return the peer record
- * @_peer_appdata: The application data attached to a peer record
- * @_service_id: Where to return the connection service ID
- * @_security_index: Where to return the connection security index
- *
- * Extract useful parameters from a CHALLENGE message.
- */
-void rxrpc_kernel_query_challenge(struct sk_buff *challenge,
-				  struct rxrpc_peer **_peer,
-				  unsigned long *_peer_appdata,
-				  u16 *_service_id, u8 *_security_index)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(challenge);
-
-	*_peer		= sp->chall.conn->peer;
-	*_peer_appdata	= sp->chall.conn->peer->app_data;
-	*_service_id	= sp->hdr.serviceId;
-	*_security_index = sp->hdr.securityIndex;
-}
-EXPORT_SYMBOL(rxrpc_kernel_query_challenge);
-
-/**
- * rxrpc_kernel_reject_challenge - Allow a kernel service to reject a challenge
- * @challenge: The challenge to be rejected
- * @abort_code: The abort code to stick into the ABORT packet
- * @error: Local error value
- * @why: Indication as to why.
- *
- * Allow a kernel service to reject a challenge by aborting the connection if
- * it's still in an abortable state.  The error is returned so this function
- * can be used with a return statement.
- *
- * Return: The %error parameter.
- */
-int rxrpc_kernel_reject_challenge(struct sk_buff *challenge, u32 abort_code,
-				  int error, enum rxrpc_abort_reason why)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(challenge);
-
-	_enter("{%x},%d,%d,%u", sp->hdr.serial, abort_code, error, why);
-
-	rxrpc_abort_conn(sp->chall.conn, NULL, abort_code, error, why);
-	return error;
-}
-EXPORT_SYMBOL(rxrpc_kernel_reject_challenge);
diff --git a/net/rxrpc/recvmsg.c b/net/rxrpc/recvmsg.c
index 28b2148b5693..8c4571982edc 100644
--- a/net/rxrpc/recvmsg.c
+++ b/net/rxrpc/recvmsg.c
@@ -201,61 +201,6 @@ static int rxrpc_recvmsg_user_id(struct rxrpc_call *call, struct msghdr *msg,
 	}
 }
 
-/*
- * Deal with a CHALLENGE packet.
- */
-static int rxrpc_recvmsg_challenge(struct socket *sock, struct msghdr *msg,
-				   struct sk_buff *challenge, unsigned int flags)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(challenge);
-	struct rxrpc_connection *conn = sp->chall.conn;
-
-	return conn->security->challenge_to_recvmsg(conn, challenge, msg);
-}
-
-/*
- * Process OOB packets.  Called with the socket locked.
- */
-static int rxrpc_recvmsg_oob(struct socket *sock, struct msghdr *msg,
-			     unsigned int flags)
-{
-	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
-	struct sk_buff *skb;
-	bool need_response = false;
-	int ret;
-
-	skb = skb_peek(&rx->recvmsg_oobq);
-	if (!skb)
-		return -EAGAIN;
-	rxrpc_see_skb(skb, rxrpc_skb_see_recvmsg);
-
-	ret = put_cmsg(msg, SOL_RXRPC, RXRPC_OOB_ID, sizeof(u64),
-		       &skb->skb_mstamp_ns);
-	if (ret < 0)
-		return ret;
-
-	switch ((enum rxrpc_oob_type)skb->mark) {
-	case RXRPC_OOB_CHALLENGE:
-		need_response = true;
-		ret = rxrpc_recvmsg_challenge(sock, msg, skb, flags);
-		break;
-	default:
-		WARN_ONCE(1, "recvmsg() can't process unknown OOB type %u\n",
-			  skb->mark);
-		ret = -EIO;
-		break;
-	}
-
-	if (!(flags & MSG_PEEK)) {
-		skb_unlink(skb, &rx->recvmsg_oobq);
-		if (need_response)
-			rxrpc_add_pending_oob(rx, skb);
-		else
-			rxrpc_free_skb(skb, rxrpc_skb_put_oob);
-	}
-	return ret;
-}
-
 /*
  * Deliver messages to a call.  This keeps processing packets until the buffer
  * is filled and we find either more DATA (returns 0) or the end of the DATA
@@ -267,7 +212,6 @@ static int rxrpc_recvmsg_data(struct socket *sock, struct rxrpc_call *call,
 			      size_t len, int flags, size_t *_offset)
 {
 	struct rxrpc_skb_priv *sp;
-	struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
 	struct sk_buff *skb;
 	rxrpc_seq_t seq = 0;
 	size_t remain;
@@ -362,13 +306,6 @@ static int rxrpc_recvmsg_data(struct socket *sock, struct rxrpc_call *call,
 
 		if (!(flags & MSG_PEEK))
 			rxrpc_rotate_rx_window(call);
-
-		if (!rx->app_ops &&
-		    !skb_queue_empty_lockless(&rx->recvmsg_oobq)) {
-			trace_rxrpc_recvdata(call, rxrpc_recvmsg_oobq, seq,
-					     rx_pkt_offset, rx_pkt_len, ret);
-			break;
-		}
 	}
 
 out:
@@ -416,14 +353,12 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 	/* Return immediately if a client socket has no outstanding calls */
 	if (RB_EMPTY_ROOT(&rx->calls) &&
 	    list_empty(&rx->recvmsg_q) &&
-	    skb_queue_empty_lockless(&rx->recvmsg_oobq) &&
 	    rx->sk.sk_state != RXRPC_SERVER_LISTENING) {
 		release_sock(&rx->sk);
 		return -EAGAIN;
 	}
 
-	if (list_empty(&rx->recvmsg_q) &&
-	    skb_queue_empty_lockless(&rx->recvmsg_oobq)) {
+	if (list_empty(&rx->recvmsg_q)) {
 		ret = -EWOULDBLOCK;
 		if (timeo == 0) {
 			call = NULL;
@@ -439,8 +374,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 		if (ret)
 			goto wait_error;
 
-		if (list_empty(&rx->recvmsg_q) &&
-		    skb_queue_empty_lockless(&rx->recvmsg_oobq)) {
+		if (list_empty(&rx->recvmsg_q)) {
 			if (signal_pending(current))
 				goto wait_interrupted;
 			trace_rxrpc_recvmsg(0, rxrpc_recvmsg_wait, 0);
@@ -450,15 +384,6 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 		goto try_again;
 	}
 
-	/* Deal with OOB messages before we consider getting normal data. */
-	if (!skb_queue_empty_lockless(&rx->recvmsg_oobq)) {
-		ret = rxrpc_recvmsg_oob(sock, msg, flags);
-		release_sock(&rx->sk);
-		if (ret == -EAGAIN)
-			goto try_again;
-		goto error_trace;
-	}
-
 	/* Find the next call and dequeue it if we're not just peeking.  If we
 	 * do dequeue it, that comes with a ref that we will need to release.
 	 * We also want to weed out calls that got requeued whilst we were
@@ -469,8 +394,7 @@ int rxrpc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 	call = list_entry(l, struct rxrpc_call, recvmsg_link);
 
 	if (!rxrpc_call_is_complete(call) &&
-	    skb_queue_empty(&call->recvmsg_queue) &&
-	    skb_queue_empty(&rx->recvmsg_oobq)) {
+	    skb_queue_empty(&call->recvmsg_queue)) {
 		list_del_init(&call->recvmsg_link);
 		spin_unlock_irq(&rx->recvmsg_lock);
 		release_sock(&rx->sk);
diff --git a/net/rxrpc/rxgk.c b/net/rxrpc/rxgk.c
index 031c6719d33e..c64ef96763ae 100644
--- a/net/rxrpc/rxgk.c
+++ b/net/rxrpc/rxgk.c
@@ -735,37 +735,6 @@ static bool rxgk_validate_challenge(struct rxrpc_connection *conn,
 	return true;
 }
 
-/**
- * rxgk_kernel_query_challenge - Query RxGK-specific challenge parameters
- * @challenge: The challenge packet to query
- *
- * Return: The Kerberos 5 encoding type for the challenged connection.
- */
-u32 rxgk_kernel_query_challenge(struct sk_buff *challenge)
-{
-	struct rxrpc_skb_priv *sp = rxrpc_skb(challenge);
-
-	return sp->chall.conn->rxgk.enctype;
-}
-EXPORT_SYMBOL(rxgk_kernel_query_challenge);
-
-/*
- * Fill out the control message to pass to userspace to inform about the
- * challenge.
- */
-static int rxgk_challenge_to_recvmsg(struct rxrpc_connection *conn,
-				     struct sk_buff *challenge,
-				     struct msghdr *msg)
-{
-	struct rxgk_challenge chall;
-
-	chall.base.service_id		= conn->service_id;
-	chall.base.security_index	= conn->security_ix;
-	chall.enctype			= conn->rxgk.enctype;
-
-	return put_cmsg(msg, SOL_RXRPC, RXRPC_CHALLENGED, sizeof(chall), &chall);
-}
-
 /*
  * Insert the requisite amount of XDR padding for the length given.
  */
@@ -1052,46 +1021,6 @@ static int rxgk_respond_to_challenge(struct rxrpc_connection *conn,
 	return ret;
 }
 
-/**
- * rxgk_kernel_respond_to_challenge - Respond to a challenge with appdata
- * @challenge: The challenge to respond to
- * @appdata: The application data to include in the RESPONSE authenticator
- *
- * Allow a kernel application to respond to a CHALLENGE with application data
- * to be included in the RxGK RESPONSE Authenticator.
- *
- * Return: %0 if successful and a negative error code otherwise.
- */
-int rxgk_kernel_respond_to_challenge(struct sk_buff *challenge,
-				     struct krb5_buffer *appdata)
-{
-	return -EINVAL;
-}
-EXPORT_SYMBOL(rxgk_kernel_respond_to_challenge);
-
-/*
- * Parse sendmsg() control message and respond to challenge.  We need to see if
- * there's an appdata to fish out.
- */
-static int rxgk_sendmsg_respond_to_challenge(struct sk_buff *challenge,
-					     struct msghdr *msg)
-{
-	struct krb5_buffer appdata = {};
-	struct cmsghdr *cmsg;
-
-	for_each_cmsghdr(cmsg, msg) {
-		if (cmsg->cmsg_level != SOL_RXRPC ||
-		    cmsg->cmsg_type != RXRPC_RESP_RXGK_APPDATA)
-			continue;
-		if (appdata.data)
-			return -EINVAL;
-		appdata.data = CMSG_DATA(cmsg);
-		appdata.len = cmsg->cmsg_len - sizeof(struct cmsghdr);
-	}
-
-	return rxgk_kernel_respond_to_challenge(challenge, &appdata);
-}
-
 /*
  * Verify the authenticator.
  *
@@ -1365,8 +1294,6 @@ const struct rxrpc_security rxgk_yfs = {
 	.free_call_crypto		= rxgk_free_call_crypto,
 	.issue_challenge		= rxgk_issue_challenge,
 	.validate_challenge		= rxgk_validate_challenge,
-	.challenge_to_recvmsg		= rxgk_challenge_to_recvmsg,
-	.sendmsg_respond_to_challenge	= rxgk_sendmsg_respond_to_challenge,
 	.respond_to_challenge		= rxgk_respond_to_challenge,
 	.verify_response		= rxgk_verify_response,
 	.clear				= rxgk_clear,
diff --git a/net/rxrpc/rxkad.c b/net/rxrpc/rxkad.c
index ca9f0e82cb9a..e55bfca10e1a 100644
--- a/net/rxrpc/rxkad.c
+++ b/net/rxrpc/rxkad.c
@@ -748,32 +748,6 @@ static int rxkad_respond_to_challenge(struct rxrpc_connection *conn,
 	return ret;
 }
 
-/*
- * RxKAD does automatic response only as there's nothing to manage that isn't
- * already in the key.
- */
-static int rxkad_sendmsg_respond_to_challenge(struct sk_buff *challenge,
-					      struct msghdr *msg)
-{
-	return -EINVAL;
-}
-
-/**
- * rxkad_kernel_respond_to_challenge - Respond to a challenge with appdata
- * @challenge: The challenge to respond to
- *
- * Allow a kernel application to respond to a CHALLENGE.
- *
- * Return: %0 if successful and a negative error code otherwise.
- */
-int rxkad_kernel_respond_to_challenge(struct sk_buff *challenge)
-{
-	struct rxrpc_skb_priv *csp = rxrpc_skb(challenge);
-
-	return rxkad_respond_to_challenge(csp->chall.conn, challenge);
-}
-EXPORT_SYMBOL(rxkad_kernel_respond_to_challenge);
-
 /* Decrypt data in-place using DES-PCBC.  @len must be a multiple of 8. */
 VISIBLE_IF_KUNIT void des_pcbc_decrypt_inplace(const struct des_ctx *key,
 					       __le64 iv, u8 *data, size_t len)
@@ -1134,7 +1108,6 @@ const struct rxrpc_security rxkad = {
 	.free_call_crypto		= rxkad_free_call_crypto,
 	.issue_challenge		= rxkad_issue_challenge,
 	.validate_challenge		= rxkad_validate_challenge,
-	.sendmsg_respond_to_challenge	= rxkad_sendmsg_respond_to_challenge,
 	.respond_to_challenge		= rxkad_respond_to_challenge,
 	.verify_response		= rxkad_verify_response,
 	.clear				= rxkad_clear,
diff --git a/net/rxrpc/server_key.c b/net/rxrpc/server_key.c
index 3efe104b1930..fc6f478e1762 100644
--- a/net/rxrpc/server_key.c
+++ b/net/rxrpc/server_key.c
@@ -173,43 +173,3 @@ int rxrpc_sock_set_security_keyring(struct sock *sk, struct key *keyring)
 	return ret;
 }
 EXPORT_SYMBOL(rxrpc_sock_set_security_keyring);
-
-/**
- * rxrpc_sock_set_manage_response - Set the manage-response flag for a kernel service
- * @sk: The socket to set the keyring on
- * @set: True to set, false to clear the flag
- *
- * Set the flag on an rxrpc socket to say that the caller wants to manage the
- * RESPONSE packet and the user-defined data it may contain.  Setting this
- * means that recvmsg() will return messages with RXRPC_CHALLENGED in the
- * control message buffer containing information about the challenge.
- *
- * The user should respond to the challenge by passing RXRPC_RESPOND or
- * RXRPC_RESPOND_ABORT control messages with sendmsg() to the same call.
- * Supplementary control messages, such as RXRPC_RESP_RXGK_APPDATA, may be
- * included to indicate the parts the user wants to supply.
- *
- * The server will be passed the response data with a RXRPC_RESPONDED control
- * message when it gets the first data from each call.
- *
- * Note that this is only honoured by security classes that need auxiliary data
- * (e.g. RxGK).  Those that don't offer the facility (e.g. RxKAD) respond
- * without consulting userspace.
- *
- * Return: The previous setting.
- */
-int rxrpc_sock_set_manage_response(struct sock *sk, bool set)
-{
-	struct rxrpc_sock *rx = rxrpc_sk(sk);
-	int ret;
-
-	lock_sock(sk);
-	ret = !!test_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-	if (set)
-		set_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-	else
-		clear_bit(RXRPC_SOCK_MANAGE_RESPONSE, &rx->flags);
-	release_sock(sk);
-	return ret;
-}
-EXPORT_SYMBOL(rxrpc_sock_set_manage_response);



From dhowells at redhat.com  Thu Aug 13 06:45:30 2026
From: dhowells at redhat.com (David Howells)
Date: Thu, 13 Aug 2026 14:45:30 +0100
Subject: [PATCH 0/2] netfs: Miscellaneous fixes
Message-ID: <20260813134533.1174132-1-dhowells@redhat.com>

Hi Christian,

Here are a couple of miscellaneous fixes for netfslib.

 (1) Fix an uninitialised return value from netfs_unbuffered_write().

 (2) Fix read progress reporting to avoid 64-bit tearing on a 32-bit
     machine.  This has been modified from the previous submission as part
     of a different series to take account of a sashiko reported issue[1].

The patches can also be found here:

	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-fixes

Thanks,
David

[1] https://sashiko.dev/#/patchset/20260810144746.574036-1-dhowells%40redhat.com

David Howells (1):
  netfs: Fix read progress reporting

Karl Mehltretter (1):
  netfs: Fix uninitialized return value in netfs_unbuffered_write()

 fs/netfs/buffered_read.c     |  4 +++
 fs/netfs/direct_write.c      |  2 +-
 fs/netfs/internal.h          |  1 +
 fs/netfs/objects.c           | 32 ++++++++++++--------
 fs/netfs/read_collect.c      | 58 +++++++++++++++++++++++++++---------
 fs/netfs/read_single.c       |  2 ++
 include/linux/netfs.h        |  2 +-
 include/trace/events/netfs.h | 21 +++++++++++++
 8 files changed, 94 insertions(+), 28 deletions(-)



From dhowells at redhat.com  Thu Aug 13 06:45:32 2026
From: dhowells at redhat.com (David Howells)
Date: Thu, 13 Aug 2026 14:45:32 +0100
Subject: [PATCH 2/2] netfs: Fix read progress reporting
In-Reply-To: <20260813134533.1174132-1-dhowells@redhat.com>
References: <20260813134533.1174132-1-dhowells@redhat.com>
Message-ID: <20260813134533.1174132-3-dhowells@redhat.com>

For really big read RPC ops that span multiple folios, netfslib allows the
filesystem to give progress notifications to wake up the collector thread
to do a collection of folios that have now been fetched, even if the RPC is
still ongoing, thereby allowing the application to make progress.

This works by taking the current rreq->cleaned_to value (which indicates
which folios have been unlocked) and adding the stashed size of the next
folio to it.  cleaned_to, however, is subject to 64-bit tearing on a 32-bit
arch.

Fix this by stashing the next progress notification point as a size_t
(which won't tear) to be added to rreq->start (which won't change), with
the collector thread calculating that from cleaned_to plus the next folio
size.

Further, however, if the folios are small, the collector thread gets
constantly woken up - which has a negative performance impact on the
system.

Fix that too by setting a minimum trigger of 256KiB or the size of the
folio at the front of the queue, whichever is larger.

Also, make sure rreq->cleaned_to is initialised up front, along with
rreq->collected_to and stream->collected_to.

Fixes: e2d46f2ec332 ("netfs: Change the read result collector to only use one work item")
Link: https://sashiko.dev/#/patchset/20260804100224.2748935-1-dhowells%40redhat.com
Signed-off-by: David Howells <dhowells at redhat.com>
cc: Paulo Alcantara <pc at manguebit.org>
cc: netfs at lists.linux.dev
cc: linux-fsdevel at vger.kernel.org
---
 fs/netfs/buffered_read.c     |  4 +++
 fs/netfs/internal.h          |  1 +
 fs/netfs/objects.c           | 32 ++++++++++++--------
 fs/netfs/read_collect.c      | 58 +++++++++++++++++++++++++++---------
 fs/netfs/read_single.c       |  2 ++
 include/linux/netfs.h        |  2 +-
 include/trace/events/netfs.h | 21 +++++++++++++
 7 files changed, 93 insertions(+), 27 deletions(-)

diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 7fdfa4f27e34..3c32ef41a27f 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -106,6 +106,9 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq,
 				folio_batch_release(&put_batch);
 				return added;
 			}
+
+			if (!rreq->progress_at)
+				netfs_read_set_unlock_at(rreq);
 			rreq->submitted += added;
 		}
 		folio_batch_release(&put_batch);
@@ -387,6 +390,7 @@ static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct fo
 	if (added < 0)
 		return added;
 	rreq->submitted = rreq->start + added;
+	rreq->progress_at = added;
 	return 0;
 }
 
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index 420ee7b26580..f92281a611de 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -109,6 +109,7 @@ static inline void netfs_see_subrequest(struct netfs_io_subrequest *subreq,
 /*
  * read_collect.c
  */
+void netfs_read_set_unlock_at(struct netfs_io_request *rreq);
 bool netfs_read_collection(struct netfs_io_request *rreq);
 void netfs_read_collection_worker(struct work_struct *work);
 void netfs_cancel_read(struct netfs_io_subrequest *subreq, int error);
diff --git a/fs/netfs/objects.c b/fs/netfs/objects.c
index 01461a74642d..7f6a3e912602 100644
--- a/fs/netfs/objects.c
+++ b/fs/netfs/objects.c
@@ -41,24 +41,32 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
 
 	memset(rreq, 0, kmem_cache_size(cache));
 	INIT_WORK(&rreq->cleanup_work, netfs_free_request);
-	rreq->gfp	= gfp;
-	rreq->start	= start;
-	rreq->len	= len;
-	rreq->origin	= origin;
-	rreq->netfs_ops	= ctx->ops;
-	rreq->mapping	= mapping;
-	rreq->inode	= inode;
-	rreq->i_size	= i_size_read(inode);
-	rreq->debug_id	= atomic_inc_return(&debug_ids);
-	rreq->wsize	= INT_MAX;
+	rreq->gfp		= gfp;
+	rreq->start		= start;
+	rreq->collected_to	= start;
+	rreq->cleaned_to	= start;
+	rreq->len		= len;
+	rreq->progress_at	= 0;
+	rreq->origin		= origin;
+	rreq->netfs_ops		= ctx->ops;
+	rreq->mapping		= mapping;
+	rreq->inode		= inode;
+	rreq->i_size		= i_size_read(inode);
+	rreq->debug_id		= atomic_inc_return(&debug_ids);
+	rreq->wsize		= INT_MAX;
 	rreq->io_streams[0].sreq_max_len = ULONG_MAX;
 	rreq->io_streams[0].sreq_max_segs = 0;
 	spin_lock_init(&rreq->lock);
-	INIT_LIST_HEAD(&rreq->io_streams[0].subrequests);
-	INIT_LIST_HEAD(&rreq->io_streams[1].subrequests);
 	init_waitqueue_head(&rreq->waitq);
 	refcount_set(&rreq->ref, 2);
 
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_stream *stream = &rreq->io_streams[s];
+
+		INIT_LIST_HEAD(&stream->subrequests);
+		stream->collected_to = rreq->start;
+	}
+
 	if (origin == NETFS_READAHEAD ||
 	    origin == NETFS_READPAGE ||
 	    origin == NETFS_READ_GAPS ||
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 23660a590124..723b479ef606 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -94,6 +94,35 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
 	folioq_clear(folioq, slot);
 }
 
+/*
+ * Determine how much to gather before unlocking more folios.
+ */
+void netfs_read_set_unlock_at(struct netfs_io_request *rreq)
+{
+	struct folio_queue *folioq = rreq->buffer.tail;
+	unsigned int slot = rreq->buffer.first_tail_slot;
+	size_t cleaned_to = rreq->cleaned_to - rreq->start;
+	size_t progress_at = cleaned_to;
+	size_t minimum = 256 * 1024;
+
+	while (progress_at < rreq->len) {
+		if (slot >= folioq_nr_slots(folioq)) {
+			folioq = folioq->next;
+			if (!folioq)
+				break;
+			slot = 0;
+		}
+
+		progress_at += folioq_folio_size(folioq, slot);
+		if (progress_at - cleaned_to >= minimum)
+			break;
+		slot++;
+	}
+
+	WRITE_ONCE(rreq->progress_at, progress_at);
+	trace_netfs_read_progress_at(rreq);
+}
+
 /*
  * Unlock any folios we've finished with.
  */
@@ -112,7 +141,7 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 	if (slot >= folioq_nr_slots(folioq)) {
 		folioq = rolling_buffer_delete_spent(&rreq->buffer);
 		if (!folioq) {
-			rreq->front_folio_order = 0;
+			WRITE_ONCE(rreq->progress_at, ULONG_MAX);
 			return;
 		}
 		slot = 0;
@@ -120,8 +149,7 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 
 	for (;;) {
 		struct folio *folio;
-		unsigned long long fpos, fend;
-		unsigned int order;
+		unsigned long long fpos = rreq->cleaned_to, fend;
 		size_t fsize;
 
 		if (*notes & COPY_TO_CACHE)
@@ -133,9 +161,7 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 			      rreq->debug_id, folio->index))
 			trace_netfs_folio(folio, netfs_folio_trace_not_locked);
 
-		order = folioq_folio_order(folioq, slot);
-		rreq->front_folio_order = order;
-		fsize = PAGE_SIZE << order;
+		fsize = folioq_folio_size(folioq, slot);
 		fpos = folio_pos(folio);
 		fend = fpos + fsize;
 
@@ -146,7 +172,7 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 			break;
 
 		netfs_unlock_read_folio(rreq, folioq, slot);
-		WRITE_ONCE(rreq->cleaned_to, fpos + fsize);
+		WRITE_ONCE(rreq->cleaned_to, fend);
 		*notes |= MADE_PROGRESS;
 
 		clear_bit(NETFS_RREQ_FOLIO_COPY_TO_CACHE, &rreq->flags);
@@ -172,6 +198,8 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
 	rreq->buffer.tail = folioq;
 done:
 	rreq->buffer.first_tail_slot = slot;
+
+	netfs_read_set_unlock_at(rreq);
 }
 
 /*
@@ -232,7 +260,7 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 		 * subreqs.
 		 */
 		if (notes & BUFFERED) {
-			size_t fsize = PAGE_SIZE << rreq->front_folio_order;
+			uoff_t unlock_at = rreq->start + rreq->progress_at;
 
 			/* Clear the tail of a short read. */
 			if (!(notes & HIT_PENDING) &&
@@ -257,7 +285,7 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)
 				transferred = front->len;
 				trace_netfs_rreq(rreq, netfs_rreq_trace_set_abandon);
 			}
-			if (front->start + transferred >= rreq->cleaned_to + fsize ||
+			if (front->start + transferred >= unlock_at ||
 			    test_bit(NETFS_SREQ_HIT_EOF, &front->flags))
 				netfs_read_unlock_folios(rreq, &notes);
 		} else {
@@ -477,20 +505,22 @@ void netfs_read_collection_worker(struct work_struct *work)
 void netfs_read_subreq_progress(struct netfs_io_subrequest *subreq)
 {
 	struct netfs_io_request *rreq = subreq->rreq;
-	struct netfs_io_stream *stream = &rreq->io_streams[0];
-	size_t fsize = PAGE_SIZE << rreq->front_folio_order;
-
-	trace_netfs_sreq(subreq, netfs_sreq_trace_progress);
+	struct netfs_io_stream *stream = &rreq->io_streams[subreq->stream_nr];
+	size_t progress_at = READ_ONCE(rreq->progress_at);
+	uoff_t update_at = rreq->start + progress_at;
+	uoff_t transferred_to = subreq->start + subreq->transferred;
 
 	/* If we are at the head of the queue, wake up the collector,
 	 * getting a ref to it if we were the ones to do so.
 	 */
-	if (subreq->start + subreq->transferred > rreq->cleaned_to + fsize &&
+	if (progress_at != ULONG_MAX &&
+	    transferred_to >= update_at &&
 	    (rreq->origin == NETFS_READAHEAD ||
 	     rreq->origin == NETFS_READPAGE ||
 	     rreq->origin == NETFS_READ_FOR_WRITE) &&
 	    list_is_first(&subreq->rreq_link, &stream->subrequests)
 	    ) {
+		trace_netfs_sreq(subreq, netfs_sreq_trace_progress);
 		__set_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
 		netfs_wake_collector(rreq);
 	}
diff --git a/fs/netfs/read_single.c b/fs/netfs/read_single.c
index 8833550d2eb6..de67ac41548d 100644
--- a/fs/netfs/read_single.c
+++ b/fs/netfs/read_single.c
@@ -170,6 +170,8 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_ite
 	if (IS_ERR(rreq))
 		return PTR_ERR(rreq);
 
+	rreq->progress_at = rreq->len;
+
 	ret = netfs_single_begin_cache_read(rreq, ictx);
 	if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
 		goto cleanup_free;
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index d0b62d53eea9..cc8d6500d059 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -247,6 +247,7 @@ struct netfs_io_request {
 	unsigned long long	submitted;	/* Amount submitted for I/O so far */
 	unsigned long long	len;		/* Length of the request */
 	size_t			transferred;	/* Amount to be indicated as transferred */
+	size_t			progress_at;	/* Report read progress when hit this much read */
 	long			error;		/* 0 or error that occurred */
 	unsigned long long	i_size;		/* Size of the file */
 	unsigned long long	start;		/* Start position */
@@ -263,7 +264,6 @@ struct netfs_io_request {
 	atomic_t		subreq_counter;	/* Next subreq->debug_index */
 	unsigned int		nr_group_rel;	/* Number of refs to release on ->group */
 	spinlock_t		lock;		/* Lock for queuing subreqs */
-	unsigned char		front_folio_order; /* Order (size) of front folio */
 	enum netfs_io_origin	origin;		/* Origin of the request */
 	bool			direct_bv_unpin; /* T if direct_bv[] must be unpinned */
 	refcount_t		ref;
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index 082cb03c6131..8ec10c076875 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -786,6 +786,27 @@ TRACE_EVENT(netfs_folioq,
 		      __print_symbolic(__entry->trace, netfs_folioq_traces))
 	    );
 
+TRACE_EVENT(netfs_read_progress_at,
+	    TP_PROTO(const struct netfs_io_request *rreq),
+
+	    TP_ARGS(rreq),
+
+	    TP_STRUCT__entry(
+		    __field(unsigned int,	rreq)
+		    __field(size_t,		progress_at)
+		    __field(size_t,		cleaned_to)
+			     ),
+
+	    TP_fast_assign(
+		    __entry->rreq	= rreq->debug_id;
+		    __entry->cleaned_to	= rreq->cleaned_to - rreq->start;
+		    __entry->progress_at = rreq->progress_at;
+			   ),
+
+	    TP_printk("R=%08x cln=%zx prg=%zx",
+		      __entry->rreq, __entry->cleaned_to, __entry->progress_at)
+	    );
+
 #undef EM
 #undef E_
 #endif /* _TRACE_NETFS_H */



From syzbot+listbcafced09a5a1a002383 at syzkaller.appspotmail.com  Mon Aug 17 05:32:31 2026
From: syzbot+listbcafced09a5a1a002383 at syzkaller.appspotmail.com (syzbot)
Date: Mon, 17 Aug 2026 05:32:31 -0700
Subject: [syzbot] Monthly afs report (Aug 2026)
Message-ID: <6a82ff5f.dbb3a75c.20434b.0058.GAE@google.com>

Hello afs maintainers/developers,

This is a 31-day syzbot report for the afs subsystem.
All related reports/information can be found at:
https://syzkaller.appspot.com/upstream/s/afs

During the period, 3 new issues were detected and 0 were fixed.
In total, 7 issues are still open and 52 have already been fixed.

Some of the still happening issues:

Ref Crashes Repro Title
<1> 118     No    INFO: task hung in rxrpc_destroy_all_calls (4)
                  https://syzkaller.appspot.com/bug?extid=1c21ab751f6531a595d6
<2> 15      Yes   KASAN: slab-use-after-free Read in rxrpc_destroy_all_peers
                  https://syzkaller.appspot.com/bug?extid=c876adfab6362679008c
<3> 5       No    kernel BUG in rxrpc_destroy_all_connections (2)
                  https://syzkaller.appspot.com/bug?extid=89001660f14fb7684c83

---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller at googlegroups.com.

To disable reminders for individual bugs, reply with the following command:
#syz set <Ref> no-reminders

To change bug's subsystems, reply with:
#syz set <Ref> subsystems: new-subsystem

You may send multiple commands in a single email message.


From kuba at kernel.org  Mon Aug 17 14:06:50 2026
From: kuba at kernel.org (Jakub Kicinski)
Date: Mon, 17 Aug 2026 14:06:50 -0700
Subject: [PATCH net v6 00/10] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <1843918.1786997961@warthog.procyon.org.uk>
References: <20260807180518.0287b58e@kernel.org>
	<20260804172639.2844491-1-dhowells@redhat.com>
	<20260806091511.46a2fabb@kernel.org>
	<1843918.1786997961@warthog.procyon.org.uk>
Message-ID: <20260817140650.1253230c@kernel.org>

On Mon, 17 Aug 2026 21:19:21 +0100 David Howells wrote:
> Jakub Kicinski <kuba at kernel.org> wrote:
> 
> > Blah, doesn't apply or pull cleanly now to either tree :(  
> 
> I posted a v7:
> 
> https://patchwork.kernel.org/project/netdevbpf/list/?series=1144661

Ack, hopefully we will get to it in time for the PR tomorrow.
I already spent 3 hours reviewing stuff today and the queue is
at 481 patches. Sigh.


From kuba at kernel.org  Mon Aug 17 18:58:02 2026
From: kuba at kernel.org (Jakub Kicinski)
Date: Mon, 17 Aug 2026 18:58:02 -0700
Subject: [PATCH net v7 00/11] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <20260812110129.979970-1-dhowells@redhat.com>
References: <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <20260817185802.0aa4c9ee@kernel.org>

On Wed, 12 Aug 2026 12:01:15 +0100 David Howells wrote:
> - Fixed more Sashiko-reported bugs[7]:
>   - Made the loops in afs_make_call() that call rxrpc_kernel_send_data()
>     pass the amount left in the iterator rather than an unreducing size.
>   - Made the second loop in afs_make_call() check to see if
>     rxrpc_kernel_send_data() returned an error.
>   - Removed yet more OOB references, two in linux/af_rxrpc.h and one in
>     rxrpc.rst.

FWIW I had to kick off clashiko manually because sparse false-positives
on one of the patches. It should finish in 30min or so. I'm signing off
for the day, so please TAL if you can:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260812110129.979970-6-dhowells at redhat.com
and let us know if we should ship v7 as is or you want to tweak..


From kuba at kernel.org  Tue Aug 18 07:49:10 2026
From: kuba at kernel.org (Jakub Kicinski)
Date: Tue, 18 Aug 2026 07:49:10 -0700
Subject: [PATCH net v7 00/11] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <2038699.1787058233@warthog.procyon.org.uk>
References: <20260817185802.0aa4c9ee@kernel.org>
	<20260812110129.979970-1-dhowells@redhat.com>
	<2038699.1787058233@warthog.procyon.org.uk>
Message-ID: <20260818074910.5618ecb9@kernel.org>

On Tue, 18 Aug 2026 14:03:53 +0100 David Howells wrote:
> Okay, there are only a couple of things that might merit producing a v8:
> 
>  (1) There's a missing key_put() in afs_open_socket()'s error path.
> 
>  (2) I should probably require READ permission on the key holding the appdata
>      provided by usespace through RXRPC_RESPONSE_APPDATA rather than SEARCH
>      permission to prevent this being used to pull the data out of keys that
>      can't otherwise read directly with keyctl().

Let's do a full v8 if you don't mind.
I gotta wrap things up for the -next PR and still ~260 patches 
in the queue ??


From dhowells at redhat.com  Wed Aug 12 06:07:56 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 14:07:56 +0100
Subject: [PATCH v3 0/2] vfs, afs, bash: Fix miscomparison of foreign user IDs in the VFS
In-Reply-To: <20260810-jazzfest-unglaubhaft-miene-4d1cf4cf42d3@brauner>
References: <20260810-jazzfest-unglaubhaft-miene-4d1cf4cf42d3@brauner> <20260728141344.1811120-1-dhowells@redhat.com>
Message-ID: <986590.1786540076@warthog.procyon.org.uk>

Christian Brauner <brauner at kernel.org> wrote:

> > Note that chown may also need modifying in some way - but that can't
> > necessarily supply the information required (for instance, an AuriStor YFS
> > ID is 64 bits, but chown can only handle a 32-bit integer; CIFS might use
> > a GUID).
> 
> Seems that some other changes are required for this. I'll take a look
> once a new version is out.

If you're referring to the chown thing, can we ignore that for now?  It would
probably need some sort of new system call, but I don't know how that should
look for now.

David



From dhowells at redhat.com  Wed Aug 12 06:12:54 2026
From: dhowells at redhat.com (David Howells)
Date: Wed, 12 Aug 2026 14:12:54 +0100
Subject: [PATCH v3 1/2] vfs: Allow filesystems with foreign owner IDs to override UID checks
In-Reply-To: <4bdf58bd593d9a54aa20760fd49aaa1df57ef216.camel@linux.ibm.com>
References: <4bdf58bd593d9a54aa20760fd49aaa1df57ef216.camel@linux.ibm.com> <20260728141344.1811120-1-dhowells@redhat.com> <20260728141344.1811120-2-dhowells@redhat.com>
Message-ID: <986908.1786540374@warthog.procyon.org.uk>

Mimi Zohar <zohar at linux.ibm.com> wrote:

> On Tue, 2026-07-28 at 15:13 +0100, David Howells wrote:
> > 
> >  (*) IMA policy ops.
> > 
> >      Unchanged for now.  I'm not sure what the best way to deal with this
> >      is - if, indeed, it needs any changes.
> 
> IMA policy rules may be written in terms of the uid, euid and fowner.  Any
> changes, if necessary, would need to be made to ima_match_rules().

I think I can probably leave this for now as using uids mapped through the
namespace mapping mechanism.  To do otherwise would, I think, require the
possibility of encoding foreign IDs in IMA rules.  Something to consider in
the future, maybe, but I don't think I need to deal with that here.

Possibly, we could make a general rule that "for any otherwise unmapped
foreign ID, do X" sort of thing.

David



From dhowells at redhat.com  Thu Aug 13 06:45:31 2026
From: dhowells at redhat.com (David Howells)
Date: Thu, 13 Aug 2026 14:45:31 +0100
Subject: [PATCH 1/2] netfs: Fix uninitialized return value in netfs_unbuffered_write()
In-Reply-To: <20260813134533.1174132-1-dhowells@redhat.com>
References: <20260813134533.1174132-1-dhowells@redhat.com>
Message-ID: <20260813134533.1174132-2-dhowells@redhat.com>

From: Karl Mehltretter <kmehltretter at gmail.com>

If preparation of the first subrequest fails,
netfs_unbuffered_write() exits its loop before ret is initialized. The
empty-iterator check can do the same.

For synchronous writes, netfs_unbuffered_write_iter_locked() may then
return an unrelated error instead of wreq->error. This is reachable
through CIFS if cifs_prepare_write() fails to reopen the file or obtain
credits.

Initialize ret to 0 so the caller returns wreq->error if no data was
written, or the number of bytes already written otherwise.

Found with Clang's -Wconditional-uninitialized.

Fixes: a0b4c7a49137e ("netfs: Fix unbuffered/DIO writes to dispatch subrequests in strict sequence")
Cc: stable at vger.kernel.org
Assisted-by: Claude:claude-fable-5
Signed-off-by: Karl Mehltretter <kmehltretter at gmail.com>
Signed-off-by: David Howells <dhowells at redhat.com>
---
 fs/netfs/direct_write.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/netfs/direct_write.c b/fs/netfs/direct_write.c
index c16fbad286a1..b04019097ab8 100644
--- a/fs/netfs/direct_write.c
+++ b/fs/netfs/direct_write.c
@@ -95,7 +95,7 @@ static int netfs_unbuffered_write(struct netfs_io_request *wreq)
 {
 	struct netfs_io_subrequest *subreq = NULL;
 	struct netfs_io_stream *stream = &wreq->io_streams[0];
-	int ret;
+	int ret = 0;
 
 	_enter("%llx", wreq->len);
 



From lukas at wunner.de  Sun Aug 16 05:13:37 2026
From: lukas at wunner.de (Lukas Wunner)
Date: Sun, 16 Aug 2026 14:13:37 +0200
Subject: [PATCH v2 01/13] kexec: add CRASH_WIPE_SECRETS to wipe secrets
 before kdump
In-Reply-To: <9765f7ea-36bb-4c7b-a19b-7facec97b762@jaseg.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811-crash-zeroize-rework-v2-1-9561d13c2340@jaseg.de>
 <anw9ILJiO5xIjP4f@MiWiFi-R3L-srv>
 <9765f7ea-36bb-4c7b-a19b-7facec97b762@jaseg.de>
Message-ID: <aoGpcczo5GkBsPwt@wunner.de>

On Wed, Aug 12, 2026 at 12:08:39PM +0200, Jan Sebastian G?tte wrote:
> I can imagine one alternative way to approach this, tell me what you think:
> Instead of registering wipe handlers that memzero places on panic, I could
> put an optional registry of (addr, len) descriptors into crashkernel memory
> that the original kernel populates with the PAs of buffers to clear ahead of
> time. Then the kdump kernel could do the actual memzero. This would remove
> all code from the actual kdump path here. The registry could be made
> per-core to avoid locks.

Or create a dedicated mempool from which all sensitive data is allocated,
then call mempool_free_bulk() on kexec to poison everything in that pool?

Thanks,

Lukas


From baoquan.he at linux.dev  Mon Aug 17 00:38:53 2026
From: baoquan.he at linux.dev (Baoquan He)
Date: Mon, 17 Aug 2026 15:38:53 +0800
Subject: [PATCH v2 01/13] kexec: add CRASH_WIPE_SECRETS to wipe secrets
 before kdump
In-Reply-To: <aoGpcczo5GkBsPwt@wunner.de>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811-crash-zeroize-rework-v2-1-9561d13c2340@jaseg.de>
 <anw9ILJiO5xIjP4f@MiWiFi-R3L-srv>
 <9765f7ea-36bb-4c7b-a19b-7facec97b762@jaseg.de>
 <aoGpcczo5GkBsPwt@wunner.de>
Message-ID: <aoK6jXA-tF9qfKOZ@MiWiFi-R3L-srv>

On 08/16/26 at 02:13pm, Lukas Wunner wrote:
> On Wed, Aug 12, 2026 at 12:08:39PM +0200, Jan Sebastian G?tte wrote:
> > I can imagine one alternative way to approach this, tell me what you think:
> > Instead of registering wipe handlers that memzero places on panic, I could
> > put an optional registry of (addr, len) descriptors into crashkernel memory
> > that the original kernel populates with the PAs of buffers to clear ahead of
> > time. Then the kdump kernel could do the actual memzero. This would remove
> > all code from the actual kdump path here. The registry could be made
> > per-core to avoid locks.

Since there are a lot of security expert in CC, I am wondering what's
the risk if we wipe secrets during dumping. System is located in
user's place, and vmcore is dumped and collected in user's place. If we
filter out secrets memory when we dumping, what could be leaked out?

> 
> Or create a dedicated mempool from which all sensitive data is allocated,
> then call mempool_free_bulk() on kexec to poison everything in that pool?
> 
> Thanks,
> 
> Lukas


From baoquan.he at linux.dev  Mon Aug 17 01:25:14 2026
From: baoquan.he at linux.dev (Baoquan He)
Date: Mon, 17 Aug 2026 16:25:14 +0800
Subject: [PATCH v2 01/13] kexec: add CRASH_WIPE_SECRETS to wipe secrets
 before kdump
In-Reply-To: <aoK6jXA-tF9qfKOZ@MiWiFi-R3L-srv>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811-crash-zeroize-rework-v2-1-9561d13c2340@jaseg.de>
 <anw9ILJiO5xIjP4f@MiWiFi-R3L-srv>
 <9765f7ea-36bb-4c7b-a19b-7facec97b762@jaseg.de>
 <aoGpcczo5GkBsPwt@wunner.de>
 <aoK6jXA-tF9qfKOZ@MiWiFi-R3L-srv>
Message-ID: <aoLFai0gzzH2bGgy@MiWiFi-R3L-srv>

On 08/17/26 at 03:38pm, Baoquan He wrote:
> On 08/16/26 at 02:13pm, Lukas Wunner wrote:
> > On Wed, Aug 12, 2026 at 12:08:39PM +0200, Jan Sebastian G?tte wrote:
> > > I can imagine one alternative way to approach this, tell me what you think:
> > > Instead of registering wipe handlers that memzero places on panic, I could
> > > put an optional registry of (addr, len) descriptors into crashkernel memory
> > > that the original kernel populates with the PAs of buffers to clear ahead of
> > > time. Then the kdump kernel could do the actual memzero. This would remove
> > > all code from the actual kdump path here. The registry could be made
> > > per-core to avoid locks.

If this has to be done in kernel, your (addr, len) registry idea is a
better direction, and we can make it a bit more general.

Instead of a secrets-only wipe registry, consider a small "crash memory
region" table with an action per entry:

enum crash_region_action {
    CRASH_REG_WIPE,   /* memzero, then dump normally (leave zeros) */
    CRASH_REG_SKIP,   /* do not include in the vmcore at all */
};

struct crash_region {
    phys_addr_t addr;
    size_t      len;
    enum crash_region_action action;
};

void crash_reg_add(phys_addr_t addr, size_t len,
                   enum crash_region_action action);
void crash_reg_del(phys_addr_t addr, size_t len);

Two actions because the use cases differ:

- WIPE is what you need: zero the secret, 
- SKIP covers things people do not want in the dump at all
   ? Registering as SKIP reduce dump size. I have been considering how
     to shrink vmcore size on some big system with huge memory, the
     vmcore size usually is several or tens of GB.

If this is OK to you, you can start a regiestry table
infrastructure with wiping secrets, we can add SKIP later and plus some
driver used memory.

I am not sure if it's satisfactory, but you can give it a shot to show
what it looks like.

Thanks
Baoquan

> 
> Since there are a lot of security expert in CC, I am wondering what's
> the risk if we wipe secrets during dumping. System is located in
> user's place, and vmcore is dumped and collected in user's place. If we
> filter out secrets memory when we dumping, what could be leaked out?
> 
> > 
> > Or create a dedicated mempool from which all sensitive data is allocated,
> > then call mempool_free_bulk() on kexec to poison everything in that pool?
> > 
> > Thanks,
> > 
> > Lukas


From dhowells at redhat.com  Mon Aug 17 13:19:21 2026
From: dhowells at redhat.com (David Howells)
Date: Mon, 17 Aug 2026 21:19:21 +0100
Subject: [PATCH net v6 00/10] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <20260807180518.0287b58e@kernel.org>
References: <20260807180518.0287b58e@kernel.org> <20260804172639.2844491-1-dhowells@redhat.com> <20260806091511.46a2fabb@kernel.org>
Message-ID: <1843918.1786997961@warthog.procyon.org.uk>

Jakub Kicinski <kuba at kernel.org> wrote:

> Blah, doesn't apply or pull cleanly now to either tree :(

I posted a v7:

https://patchwork.kernel.org/project/netdevbpf/list/?series=1144661

David



From contact at jaseg.de  Tue Aug 18 02:03:19 2026
From: contact at jaseg.de (=?UTF-8?Q?Jan_Sebastian_G=C3=B6tte?=)
Date: Tue, 18 Aug 2026 11:03:19 +0200
Subject: [PATCH v2 01/13] kexec: add CRASH_WIPE_SECRETS to wipe secrets
 before kdump
In-Reply-To: <aoLFai0gzzH2bGgy@MiWiFi-R3L-srv>
References: <20260811-crash-zeroize-rework-v2-0-9561d13c2340@jaseg.de>
 <20260811-crash-zeroize-rework-v2-1-9561d13c2340@jaseg.de>
 <anw9ILJiO5xIjP4f@MiWiFi-R3L-srv>
 <9765f7ea-36bb-4c7b-a19b-7facec97b762@jaseg.de> <aoGpcczo5GkBsPwt@wunner.de>
 <aoK6jXA-tF9qfKOZ@MiWiFi-R3L-srv> <aoLFai0gzzH2bGgy@MiWiFi-R3L-srv>
Message-ID: <9071e03b-a633-43e4-ac77-3b05291c1f66@jaseg.de>

On 8/17/26 10:25, Baoquan He wrote:
> On 08/17/26 at 03:38pm, Baoquan He wrote:
>> On 08/16/26 at 02:13pm, Lukas Wunner wrote:
>>> On Wed, Aug 12, 2026 at 12:08:39PM +0200, Jan Sebastian G?tte wrote:
>>>> I can imagine one alternative way to approach this, tell me what you think:
>>>> Instead of registering wipe handlers that memzero places on panic, I could
>>>> put an optional registry of (addr, len) descriptors into crashkernel memory
>>>> that the original kernel populates with the PAs of buffers to clear ahead of
>>>> time. Then the kdump kernel could do the actual memzero. This would remove
>>>> all code from the actual kdump path here. The registry could be made
>>>> per-core to avoid locks.
> 
> If this has to be done in kernel, your (addr, len) registry idea is a
> better direction, and we can make it a bit more general.
> 
> Instead of a secrets-only wipe registry, consider a small "crash memory
> region" table with an action per entry:
> 
> enum crash_region_action {
>      CRASH_REG_WIPE,   /* memzero, then dump normally (leave zeros) */
>      CRASH_REG_SKIP,   /* do not include in the vmcore at all */
> };
> 
> struct crash_region {
>      phys_addr_t addr;
>      size_t      len;
>      enum crash_region_action action;
> };
> 
> void crash_reg_add(phys_addr_t addr, size_t len,
>                     enum crash_region_action action);
> void crash_reg_del(phys_addr_t addr, size_t len);
> 
> Two actions because the use cases differ:
> 
> - WIPE is what you need: zero the secret,
> - SKIP covers things people do not want in the dump at all
>     ? Registering as SKIP reduce dump size. I have been considering how
>       to shrink vmcore size on some big system with huge memory, the
>       vmcore size usually is several or tens of GB.
> 
> If this is OK to you, you can start a regiestry table
> infrastructure with wiping secrets, we can add SKIP later and plus some
> driver used memory.
> 
> I am not sure if it's satisfactory, but you can give it a shot to show
> what it looks like.
I think that's a good idea. I'll post a patch adding that registry this 
or next week.

Thanks,
Jan



From dhowells at redhat.com  Tue Aug 18 06:03:53 2026
From: dhowells at redhat.com (David Howells)
Date: Tue, 18 Aug 2026 14:03:53 +0100
Subject: [PATCH net v7 00/11] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <20260817185802.0aa4c9ee@kernel.org>
References: <20260817185802.0aa4c9ee@kernel.org> <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <2038699.1787058233@warthog.procyon.org.uk>

Jakub Kicinski <kuba at kernel.org> wrote:

> FWIW I had to kick off clashiko manually because sparse false-positives
> on one of the patches. It should finish in 30min or so. I'm signing off
> for the day, so please TAL if you can:
> https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260812110129.979970-6-dhowells at redhat.com
> and let us know if we should ship v7 as is or you want to tweak..

Okay, there are only a couple of things that might merit producing a v8:

 (1) There's a missing key_put() in afs_open_socket()'s error path.

 (2) I should probably require READ permission on the key holding the appdata
     provided by usespace through RXRPC_RESPONSE_APPDATA rather than SEARCH
     permission to prevent this being used to pull the data out of keys that
     can't otherwise read directly with keyctl().

I can fix both of these with follow-up single line fix patches or (2) could
be fixed in place at the point of application:

--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -640,7 +640,7 @@ static int rxrpc_sendmsg_cmsg(struct msghdr *msg, struct rxrpc_send_params *p)
 			if (p->call.app_data)
 				return -EINVAL;
 			key_id = *(key_serial_t *)CMSG_DATA(cmsg);
-			key = lookup_user_key(key_id, 0, KEY_NEED_SEARCH);
+			key = lookup_user_key(key_id, 0, KEY_NEED_READ);
 			if (IS_ERR(key))
 				return PTR_ERR(key);
 			if (key_ref_to_ptr(key)->type != &key_type_user &&

Everything else, I think, can be safely deferred.  I've discussed the points
raised below.

David
---
=== Patch 1

"This isn't a bug introduced by this patch, but should the kernel-doc for the
exported rxrpc_kernel_send_data() be updated in the same change?" - I can
address doc updates in an additonal patch.

"The two in-tree callers already disagree on how success is encoded:
fs/afs/rxrpc.c:afs_send_empty_reply() switches on "case 0:" while
afs_send_simple_reply() tests "n >= 0"." - That's probably worth fixing, but
it'd be an AFS patch and isn't relevant to this patchset.

=== Patch 2

"Is there a reachable case where len differs from
iov_iter_count(&msg->msg_iter) on entry to rxrpc_send_data()?" - This really
applies to all implementations of ->sendmsg(), not just rxrpc.  In theory,
maybe; but in practice I don't think so.  I did spend some time looking if we
could eliminate the len argument entirely, but that doesn't need dealing with
here.  I've made the assumption that "len" controls how much we want to write,
particularly for the purpose of marking the last packet, and if msg_iter is
short, then we return a short send (if we've already filled a txbuf) or
-EFAULT (if we haven't) and don't mark last packet.  If there's more in
msg_iter, we just ignore the excess.

I can create an additional patch to fix the doc.

=== Patch 3

"The changelog only describes rxrpc_send_data(), and doesn't mention this
fs/afs/rxrpc.c hunk at all." - That's not actually true; it even quotes the
mention about converting to retry loops.

"Are the two lines immediately following now stale?" - True, but I can remove
the stale lines with a followup patch.

"Can -ENOMEM here leave a partially encrypted txbuf that is later encrypted a
second time?" - The assumption is that if ENOMEM occurs, we haven't tried to
encrypt the buffer yet.  Even if a confounder has been inserted, that's not a
problem as it's overwriting the specific bit of buffer reserved for it.  A new
confounder can just be written over it.  skcipher shouldn't go down the slow
path as the buffer should be correctly aligned and encryption is done in
place.

"The second half of the concern is that the retry does not always re-copy the
plaintext." - This shouldn't matter.  If the packet is successfully encrypted,
then txb/call->tx_pending should be cleared until we go back to the top of the
loop and allocate a new txbuf.  As previously mentioned, the assumption is
that once we actually start encrypting, we should not fail with ENOMEM.

"Does this condition depend on an earlier patch in the series that isn't cc'd
to stable?" - An oversight, but I'm not sure it matters enough to respin.

=== Patch 4

"Can this store ever move a non-NULL txb?" - Good point; txb must be NULL,
otherwise we wouldn't come down the wait_for_space branch.  But it doesn't
really need fixing; it just writes NULL twice to the same place under lock.

"The new out_nolock does "return copied ?: ret", so with copied > 0 the
negative value from the wait is discarded." - This change is correct.
Possibly it should have been mentioned in the changelog as a change of
behaviour.

"Also, the trace prints ret while the function returns "copied ?: ret"" - It
doesn't really matter, but I probably want to see the error that caused the
return there.

=== Patch 5

"This isn't a bug introduced by this patch, but since the code is being
relocated here it may be worth a look: should these nested sections use
spin_lock_irqsave()/spin_unlock_irqrestore() instead?" - No point as IRQs are
known to be enabled.

"I could not find an IRQ-context acquirer of either call->recvmsg_queue.lock
or rx->recvmsg_lock, so this looks like a fragility rather than a demonstrable
deadlock today." - The problem is that the app thread can otherwise hold up
the I/O thread, particularly if realtime is involved.

"Further notifications are suppressed by putting recvmsg_link on a dummy
queue." - The comment needs updating, but that can be done with a follow-up
patch.

"Should the prototype move too?" - Yeah.  A follow-up patch can do that.

=== Patch 6

Nothing mentioned.

=== Patch 7

"Should this patch carry a Fixes: tag?  It is cc'd to stable but does not name
the defect it fixes, nor the user-visible symptom in the AF_RXRPC challenge
response path." - With regard to the keys patch, that's not technically a fix,
but a prerequisite.

"Does the comment block just above struct user_key_payload need updating?" -
Yeah, but that can be a follow-up patch.

"Is the comment on put_user_key_payload() accurate?" - Ditto.

=== Patch 8

"This isn't a bug introduced by this patch, but the new length calculation
here differs from the pre-existing one in afs_create_yfs_cm_token() in the
same file, which reads:" - Yeah, and also, as noted, the code is removed.

"Is anything reading server->yfs_rxgk_appdata as of this commit?" - See the
next patch.

"Can a cached fileserver record end up being used for RxGK calls without ever
getting appdata created?" - Yes, this can happen to probe calls for the
moment.  That can be fixed, but I think separately as it's purely work in
fs/afs/.

"and -ENOPKG is user-influenced, since rxrpc_preparse_xdr_yfs_rxgk() only
range-checks the token enctype rather than looking it up with
crypto_krb5_find_enctype()." - But it creates a callback key of the same type
as the key the user provides to make FS calls.  We don't get a CHALLENGE
packet until we have encrypted a DATA packet and sent it, in which case
-ENOPKG would have already happened.

If afs_create_token_key() fails, we can still do unencrypted Cache Manager
service comms, which is why I don't make it a fatal error.

"Does the address-update path silently swallow these new failures?" - Um,
that's really about talking to the VL server, not the FS server.  The VL
server doesn't involve interaction eith the CM service (that's just the
3rd-party change notification channel from the FS server).

Possibly there is a further bug here, but it's an AFS bug, not an AF_RXRPC
bug if so.

"Should this pick the same token that rxrpc will actually use for the
connection, rather than always the first one?" - This is actually a
pre-existing issue and needs more work.

=== Patch 9

"Should Documentation/networking/rxrpc.rst be updated alongside this?" - That
actually needs a more comprehensive update and can be done separately.

"RXRPC_SUPPORTED_CMSG sees type 20 as available with nothing in the
documentation describing it, its key_serial_t payload, or the restriction to
key_type_user/key_type_logon keys." - Yeah.  I need to add that, but it can be
done in a follow-up patch.

"Should this case also validate p->command, or that the sendmsg() is going to
create a new call?" - The parameter is just ignored if not creating a call.  I
suppose I could add a check that a call is being created - but that can be
done in a separate patch.

=== Patch 10

"Can this path disclose the payload of a key the caller is not permitted to
read?" - That's a good point and needs READ permission, not SEARCH permission.
That can be followed up with a quick patch, I think, but might need the patch
respinning.

=== Patch 11

"does afs_open_socket() leak the key created by afs_create_token_key()?" -
Yeah, that needs a fix.

"Should the rxrpc_abort_reason enum entry go too, in the same way as these?
EM(rxrpc_abort_response_sendmsg, "resp-sendmsg")" - That can be done in a
follow-up patch.

"With RXRPC_CHALLENGED gone from this enum, are the structs describing it
meant to stay in the same header?" - Those can be removed in a follow-up
patch.

"#define RXRPC_MANAGE_RESPONSE" - Ditto.

"setsockopt(SOL_RXRPC, RXRPC_MANAGE_RESPONSE)" - The stub handler for that can
be removed in a follow up patch.



From dhowells at redhat.com  Tue Aug 18 07:49:08 2026
From: dhowells at redhat.com (David Howells)
Date: Tue, 18 Aug 2026 15:49:08 +0100
Subject: [PATCH net v7 00/11] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <2038699.1787058233@warthog.procyon.org.uk>
References: <2038699.1787058233@warthog.procyon.org.uk> <20260817185802.0aa4c9ee@kernel.org> <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <2088510.1787064548@warthog.procyon.org.uk>

David Howells <dhowells at redhat.com> wrote:

> 
>  (2) I should probably require READ permission on the key holding the appdata
>      provided by usespace through RXRPC_RESPONSE_APPDATA rather than SEARCH
>      permission to prevent this being used to pull the data out of keys that
>      can't otherwise read directly with keyctl().
> 
> I can fix both of these with follow-up single line fix patches or (2) could
> be fixed in place at the point of application:
> 
> --- a/net/rxrpc/sendmsg.c
> +++ b/net/rxrpc/sendmsg.c
> @@ -640,7 +640,7 @@ static int rxrpc_sendmsg_cmsg(struct msghdr *msg, struct rxrpc_send_params *p)
>  			if (p->call.app_data)
>  				return -EINVAL;
>  			key_id = *(key_serial_t *)CMSG_DATA(cmsg);
> -			key = lookup_user_key(key_id, 0, KEY_NEED_SEARCH);
> +			key = lookup_user_key(key_id, 0, KEY_NEED_READ);
>  			if (IS_ERR(key))
>  				return PTR_ERR(key);
>  			if (key_ref_to_ptr(key)->type != &key_type_user &&

Actually, there's a better way to do this, and that's to check the prefix on
the key description.  See attached patch.

David
---
commit 6d456b373d5c7cf2c1c9f5ead0e563c75e739442
Author: David Howells <dhowells at redhat.com>
Date:   Tue Aug 18 14:30:37 2026 +0100

    rxrpc: Fix user appdata key check
    
    The check made by rxrpc_sendmsg_cmsg() for RXRPC_RESPONSE_APPDATA on the
    key it retrieves allows keys to be accessed by generating
    CHALLENGE/RESPONSE exchange.  Currently, any user or logon key can be
    accessed in this manner.  Fix this by restricting the patch description to
    require a prefix of "rxrpc-appdata:".
    
    Fixes: xxxxxxxxxxxx ("rxrpc: Fix CHALLENGE packet overqueuing and simplify RESPONSE generation")
    Link: https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260812110129.979970-6-dhowells at redhat.com
    Signed-off-by: David Howells <dhowells at redhat.com>
    cc: Marc Dionne <marc.dionne at auristor.com>
    cc: Jeffrey Altman <jaltman at auristor.com>
    cc: Eric Dumazet <edumazet at google.com>
    cc: "David S. Miller" <davem at davemloft.net>
    cc: Jakub Kicinski <kuba at kernel.org>
    cc: Paolo Abeni <pabeni at redhat.com>
    cc: Simon Horman <horms at kernel.org>
    cc: Jarkko Sakkinen <jarkko at kernel.org>
    cc: linux-afs at lists.infradead.org
    cc: keyrings at vger.kernel.org
    cc: stable at kernel.org

diff --git a/net/rxrpc/sendmsg.c b/net/rxrpc/sendmsg.c
index 4755fc76d8f3..eb3dc352684e 100644
--- a/net/rxrpc/sendmsg.c
+++ b/net/rxrpc/sendmsg.c
@@ -648,6 +648,13 @@ static int rxrpc_sendmsg_cmsg(struct msghdr *msg, struct rxrpc_send_params *p)
 				key_ref_put(key);
 				return -EINVAL;
 			}
+			if (!key_ref_to_ptr(key)->description ||
+			    strncmp(key_ref_to_ptr(key)->description,
+				    "rxrpc-appdata:", 14) != 0) {
+				key_ref_put(key);
+				return -EINVAL;
+			}
+				
 			p->call.app_data = key_ref_to_ptr(key);
 			break;
 



From dhowells at redhat.com  Tue Aug 18 07:58:06 2026
From: dhowells at redhat.com (David Howells)
Date: Tue, 18 Aug 2026 15:58:06 +0100
Subject: [PATCH net v7 00/11] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <2038699.1787058233@warthog.procyon.org.uk>
References: <2038699.1787058233@warthog.procyon.org.uk> <20260817185802.0aa4c9ee@kernel.org> <20260812110129.979970-1-dhowells@redhat.com>
Message-ID: <2089213.1787065086@warthog.procyon.org.uk>

David Howells <dhowells at redhat.com> wrote:

>  (1) There's a missing key_put() in afs_open_socket()'s error path.

Actually, this is preexisting, so it needs fixing, but isn't due to this
patchset (it was added in d98c317fd9aa78dfa45e47deb6536cd35783afd1).

David



From dhowells at redhat.com  Tue Aug 18 08:05:24 2026
From: dhowells at redhat.com (David Howells)
Date: Tue, 18 Aug 2026 16:05:24 +0100
Subject: [PATCH net v7 00/11] rxrpc: Fix CHALLENGE packet handling
In-Reply-To: <20260818074910.5618ecb9@kernel.org>
References: <20260818074910.5618ecb9@kernel.org> <20260817185802.0aa4c9ee@kernel.org> <20260812110129.979970-1-dhowells@redhat.com> <2038699.1787058233@warthog.procyon.org.uk>
Message-ID: <2090401.1787065524@warthog.procyon.org.uk>

Jakub Kicinski <kuba at kernel.org> wrote:

> Let's do a full v8 if you don't mind.

Ok.

David



