UBIFS: Is it possible to get the compressed size of a file?

[Andreas Dilger]

On Feb 2, 2015, at 2:33 AM, Artem Bityutskiy <dedekind1 at gmail.com> wrote:
> Hi Markus,
> 
> On Thu, 2014-12-18 at 12:08 +0100, markus.heininger at online.de wrote:
>> in our system we´re using ring buffers where it is necessary to
>> remove old files from certain directories when the physical usage
>> of each directory is above a certain level which is different for each 
>> folder.
>> 
>> Evaluating the output of "df" after write access might be difficult
>> since there are several concurrent writing processes.
>> 
>> But many thanks for your answer, it seems that there is no easy
>> way to get the information needed and we must investigate further on
>> our own.
> 
> Yes, no easy way, but I think implementing what you need is possible. I
> do not have plans and time to work on this, but I can help by giving
> advises and review.
> 
> The question has 2 major parts.
> 
> 1. The interface
> 2. The implementation
> 
> For the former, one need to carefully investigate if there is something
> like this already implemented for other file-systems. I think btrfs may
> have it. If it is, then UBIFS should use similar interface, probably.
> 
> And whatever is the interface choice, it should be discussed in the
> linux-fsdevel at vger.kernel.org mailing list, which I am CCing.

One option that was discussed for btrfs was to use the first fe_reserved
field for the FIEMAP ioctl struct fiemap_extent to fe_phys_length to
hold the compressed size of each extent in the file.

http://lwn.net/Articles/607552/
http://thread.gmane.org/gmane.comp.file-systems.btrfs/37312

I'm not sure what happened to that patch series - I was looking forward
to it landing, and it was in very good shape I think.

I think FIEMAP would be a good userspace API to get this kind of
low-level allocation information about the compressed size of data.
For most files it would probably only need a single fiemap_extent
to return the allocation data.

Cheers, Andreas

> For the latter, I'd suggest to try this.
> 
> a. 'struct ubifs_ino_node' has unused space, use it to add the
> compressed size field.
> b. maintain this field
> c. this field will only be correct for the part of the file which are on
> the media. The dirty data in the page cache has not yet been compressed,
> so we do not know its compressed size yet.
> e. when user asks for the compressed size, you have to sync the inode
> first, in order to make sure the compressed size is correct.
> 
> And the implementation should be backward-compatible. That is, if new
> driver mounts the old media, you return something predicatable. I guess
> uncompressed size could be it.
> 
> 
> Hope this helps,
> Artem.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Cheers, Andreas