Roadmap for netfslib and local caching (cachefiles)

[David Howells]

David Howells <dhowells at redhat.com> wrote:

> Disconnected Operation
> ======================
> 
> I'm working towards providing support for disconnected operation, so that,
> provided you've got your working set pinned in the cache, you can continue to
> work on your network-provided files when the network goes away and resync the
> changes later.
> 
> This is going to require a number of things:
> 
>  (1) A user API by which files can be preloaded into the cache and pinned.
> 
>  (2) The ability to track changes in the cache.
> 
>  (3) A way to synchronise changes on reconnection.
> 
>  (4) A way to communicate to the user when there's a conflict with a third
>      party change on reconnect.  This might involve communicating via systemd
>      to the desktop environment to ask the user to indicate how they'd like
>      conflicts recolved.
> 
>  (5) A way to prompt the user to re-enter their authentication/crypto keys.
> 
>  (6) A way to ask the user how to handle a process that wants to access data
>      we don't have (error/wait) - and how to handle the DE getting stuck in
>      this fashion.

Some further thoughts stemming from a discussion with Willy:

 - Would need to store the pre-disconnection metadata as well as any updated
   metadata.  When performing conflict resolution, userspace would need to be
   able to access these in addition to the current state (local) and current
   state (server).

 - Would need the ability to include extra stats, such as the AFS data
   version, that are used for cache coherency management.

 - Would need to provide an API by which userspace can access both states of
   the data, possibly including the original data if we still have it in the
   cache.  That could be a number of ioctls on the target file.

 - Would need a range of resolution options in userspace, not limited to keep
   local, keep remote, but also the option to stash one/both somewhere.  May
   also need to provide app-specific resolvers - merging git trees for
   example, but also what do you do about sqlite databases, say?

 - There may be bulk changes that the user would want to resolve in bulk,
   perhaps by "everything in the subtree" or pattern matching rules,
   e.g. "disard all .o files" or "take the .o file matching the newest .c file
   in the same directory".

 - May need to change how expired keys are handled so that they aren't already
   garbage collected, but can continue to be used as a token off which to hang
   cached access rights.

David