[Lsf-pc] [LSF/MM/BPF TOPIC] Improving Block Layer Tracepoints for Next-Generation Backup Systems

[Vishnu ks]

Thanks everyone for the detailed technical feedback and clarifications
- they've been extremely valuable in understanding the fundamental
challenges and existing solutions.

I appreciate the points about md-cluster and DRBD's network RAID
capabilities. While these are robust solutions for network-based
replication, I'm particularly interested in the point-in-time recovery
capability for scenarios like ransomware recovery, where being able to
roll back to a specific point before encryption occurred would be
valuable.

Regarding blk_filter - I've been exploring it since it was mentioned,
and it indeed seems to be the right approach for what we're trying to
achieve. However, I've found that many of our current requirements can
actually be implemented using eBPF without additional kernel modules.
I plan to create a detailed demonstration video to share my findings
with this thread. Additionally, I'll be cleaning up and open-sourcing
our replicator utility implementation for community feedback.

I would very much like to attend the LSF/MM/BPF summit to discuss
these ideas in person and learn more about blk_filter and proper block
layer fundamentals. Would it be possible for someone to help me with
an invitation?

Thanks again to everyone who took the time to explain the intricacies
of write caching, sector tracking limitations, and data persistence
guarantees. This discussion has been incredibly educational.

Thanks and regards,
Vishnu KS

On Tue, 7 Jan 2025 at 03:48, Dan Williams <dan.j.williams at intel.com> wrote:
>
> Song Liu via Lsf-pc wrote:
> > On Sat, Jan 4, 2025 at 9:52 AM Vishnu ks <ksvishnu56 at gmail.com> wrote:
> > >
> > [...]
> > >
> > > @Song: Our approach fundamentally differs from md/raid in several ways:
> > >
> > > 1. Network-based vs Local:
> > >    - Our system operates over network, allowing replication across
> > > geographically distributed systems
> > >    - md/raid works only with locally attached storage devices
> >
> > md-cluster (https://docs.kernel.org/driver-api/md/md-cluster.html)
> > does support RAID in a cluster.
>
> Also,
>
> https://docs.kernel.org/admin-guide/blockdev/drbd/index.html

-- 
Vishnu KS,
Opensource contributor and researcher,
https://iamvishnuks.com