Scheduler latency problems when using NAND

[Iwo Mergler]

Mark Mason wrote:
> Hi all,
> 
> I hope this is the right place for this question.  I'm having some
> problems with scheduler latency when using UBIFS, and I'm hoping for
> some suggestions.
<snip>
> The application is storing streaming video, almost entirely large
> sequential files, roughly 250K to 15M, to a 1.6G filesystem.  There's
> no seeking or rewriting, just creat, write, close, repeat.  No
> compression is used on the filesystem.
> 
> The problem I'm seeing is excessively large scheduler latency when
> data is flushed to NAND.
<snip>
> Does anyone have any suggestions, ideas, hints, advice, etc?

The Linux block cache is optimised for mechanical hard drives,
to minimise seek times. Some of the assumptions don't make much
sense with FLASH and streaming storage.

Maybe try to flush data whenever you have written a few blocks'
worth. Or have a look at the O_DIRECT flag (or madvise), although
I don't know how it interacts with UBIFS.

Have you considered using a different file system? UBIFS is not
exactly optimised for your use case. JFFS2 would be better, but
won't stomach your flash size.

Especially if you only deal with one stream at a time, you could
create a very simple circular file system yourself. Like JFFS2
without the memory cache.

You could use a real filesystem to store the metadata for your
circular storage partition (file name, length, offset).

Maybe use raw UBI so you don't have to worry about bad blocks.

Either way, the time to erase a block and write a single page
is predictable and you can do it as soon as you get the data.


Best regards,

Iwo