Synchronization in UBIFS (zero length files)

[Tanya Brokhman]

Hi Artem,

On 12/18/2014 5:12 PM, Richard Weinberger wrote:
> Tanya,
>
> Am 18.12.2014 um 15:53 schrieb Tanya Brokhman:
>> Hi Richard,
>>
>> On 12/18/2014 1:12 PM, Richard Weinberger wrote:
>>> Tanya,
>>>
>>> Am 18.12.2014 um 09:34 schrieb Tanya Brokhman:
>>>> Hi Artem/Richard
>>>>
>>>> I've been looking into the "zero length files" issue in UBIFS and came across "Synchronization exceptions for buggy applications" @
>>>> http://www.linux-mtd.infradead.org/doc/ubifs.html#L_sync_semantics. This section concludes with:
>>>>
>>>> "We have plans to implement these features in UBIFS, but this has not been done yet. The problem is that UBI/MTD are fully synchronous and we cannot initiate asynchronous
>>>> write-out, so we'd have to synchronously write files on close/rename, which is slow. So implementing these features would require implementing asynchronous I/O in UBI, which is a
>>>> big job. But feel free to do this :-)."
>>>>
>>>> So two questions:
>>>> 1. was anything done in ubifs to handle file truncation and rename similar to ext4? Didn't find anything but afraid I might be missing something
>>>
>>> I don't think so.
>>>
>>>> 2. Not sure I understand why "implementing these features would require implementing asynchronous I/O in UBI". Could you please elaborate on this?
>>>
>>> If we'd do it synchronous it would horrible slow down UBIFS. That's why ext4 is doing it async.
>>
>> Regarding EXT4. In Theodore Tso article about this (http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/) he mentioned 3 ext4 patches that
>> are handling this issue but I can't find them by their git id:
>> "These three patches (with git id’s bf1b69c0, f32b730a, and 8411e347) will cause a file to have any delayed allocation blocks to be allocated immediately when a file is replaced."
>> Are you familiar with the patches in question?
>
> Nope.
>
>>> Having async IO for UBI is not trivial. Especially because of powercut safety.
>>
>> Is it even doable? It seems to me that if we add async functionality to ubi it will become not power-cut tolerant, unless we implement a journal there (really don't want to do
>> that....)
>
> In software you can do everything but I fear it will be messy.
>
>> Another idea: from what I've read on the subject so far (this is new to me so be patient), zero-length files can result due to several use cases one of them is renaming files in
>> the following scheme:
>> 1. write to temp file A - written to cache
>> 2. rename A to B - written directly to disk
>> 3. eventually data is flashed to disk
>> If power cut occurs between #2 and #3 we end up with empty file B. But what if we call fdatasync on A as part of the rename operation?
>> More specifically: I'm looking @ fs/ubifs/dir.c-ubifs_rename(). What if we force sync of old_inode by:
>> 1. err = old_inode->i_sb->s_op->write_inode(old_inode, NULL);
>> 2. err = ubifs_sync_wbufs_by_inode(c, old_inode);
>> If I understand the code correctly it is basically the same as adding fsync() in userspace after #2 above.
>>
>> What do you think?
>
> Let's wait what Artem says.
> I bet here are more dragons. :)

Sorry to nag, this is a bit urgent for us. Could you please share your 
inputs on the above?


Thanks,
Tanya Brokhman
-- 
Qualcomm Israel, on behalf of Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora 
Forum, a Linux Foundation Collaborative Project