Jffs2 and big file = very slow jffs2_garbage_collect_pass

[Glenn Henshaw]

On 17-Jan-08, at 12:43 PM, Josh Boyer wrote:

> On Thu, 17 Jan 2008 17:26:01 +0100
> Jörn Engel <joern at logfs.org> wrote:
>
>> On Thu, 17 January 2008 17:12:29 +0100, Matthieu CASTET wrote:
>>>
>>> we have a 240 MB jffs2 partition with summary enabled and no
>>> compression. We use 2ad8ee713566671875216ebcec64f2eda47bd19d git  
>>> jffs2
>>> version
>>> (http://git.infradead.org/?p=mtd-2.6.git;a=commit;h=2ad8ee713566671875216ebcec64f2eda47bd19d 
>>> )
>>>
>>>
>>> On this partition we have several file (less than 1 MB) and a big  
>>> file
>>> in the root (200 MB).
>>>
>>> The big file is a FAT image that is exported with usb-storage  
>>> (with usb
>>> device) or mounted on a loopback device.
>>>
>>> After some FAT operations, we manage to get in a situation were the
>>> jffs2_garbage_collect_pass take 12 minutes.
>>>
>>> jffs2_lookup for the big file (triggered with a ls in the root)  
>>> take 12
>>> minutes.
>>>
>>> If we do a ls without waiting that jffs2_garbage_collect_pass  
>>> finish, ls
>>> takes 12 minutes to complete.
>>
>> Impressive!  JFFS2 may be slow, but it shouldn't be _that_ slow.  Not
>
> How do you know?  A 200MiB file will likely have around 50,000 nodes.
> If the summary stuff is incorrect, and since we have no idea what kind
> of platform is being used here, it may well be within reason.

   I found a similar problem on an older 2.4.27 based system. We have  
a 64k JFFS2 partition (1024 blocks of 4kbytes). As the file system  
fills up, the time for any operation increases exponentially. When it  
reaches 90% full, it takes minutes to write a file. After a cursory  
inspection, it seems to block doing garbage collection and compressing  
blocks.

   We gave up and limited the capacity to 60% full at the application  
level.

   I'd appreciate any pointer to fix this, as migrating to a 2.6  
kernel is not an option.


-- 
Glenn Henshaw                     Logical Outcome Ltd.
e: thraxisp at logicaloutcome.ca     w: www.logicaloutcome.ca