Some information about direct-io support on ubifs

[Michael Nazzareno Trimarchi]

Hi

On Thu, Nov 14, 2024 at 3:10 PM Zhihao Cheng <chengzhihao1 at huawei.com> wrote:
>
> 在 2024/11/11 23:08, Michael Nazzareno Trimarchi 写道:
> > Hi
> >
> > On Mon, Nov 11, 2024 at 3:15 PM Zhihao Cheng <chengzhihao1 at huawei.com> wrote:
> >>
> >> 在 2024/11/11 17:37, Michael Nazzareno Trimarchi 写道:
> >>> Hi all
> >>>

First of all, thank you very much.

> >>> I would like to have some information on how to implement direct-io
> >>> support in ubifs. I have tried to mount squashfs image as file in
> >>> ubifs using loopback and direct-io but it seems that loop can not
> >>> handle when this is on top of ubifs. From what I understand from the
> >>> code it seems that the requirement is that the direct-io is
> >>> implemented in the filesystem where the file is stored. I have
> >>> simulated using nand sim. The idea is to reduce the buffering. I'm
> >>> open to have other suggestions regarding this too.
> >>>
> >>> Michael
> >>> .
> >>>
> >> Hi Michael,
> >> There is a discussion(9 years ago) about ubifs dio[1], of course we
> >> cannot implement it as a fake interface now.
> >> Before talking about adding dio support for UBIFS, let me know some
> >> background information. What's the purpose of using dio for opening
> >> squashfs image file? What's the goal of reducing buffer you mentioned
> >> above, Better performance? Less memory usage?
> >>
> >>
> >> [1]
> >> https://lore.kernel.org/lkml/1440016553-26481-2-git-send-email-richard@nod.at/
> >
> >
> > The goal is less memory usage. Each squashfs is mounted using loopback
> > device, so according
> > to some thread on direct-io, I understand that we then not have
> > buffering two times on ubifs level and loop
> > level, but the thread was not clear to me. The device has very limited
> > capability and memory. I have done some
> > test with fuse  but I did not have numbers to compare them
> >
>
> Hi Michael,
>
> I think it is possible to support DIO for UBIFS in theory, but it is not
> an easy task. We need to imeplement read/write functions, and we should
> iterate user data bytes by UBIFS_BLOCK_SIZE unit.
> 1. Things should be done in write function:
>     1) Do space reservation, refer to ubifs_write_begin ->
> allocate_budget, of cource the origin 'req.new_page' cannot be used and
> we should define a new budget type based on UBIFS_BLOCK_SIZE.

I have seen that allocate a budget using an allocated request. Here you mean
to change the interface a bit to require the allocated budget that
must be multiple of BLOCK_SIZE?

If the question looks silly, sorry for that.

>     2) record user data in data node, refer to do_writepage ->
> ubifs_jnl_write_data
>     3) invalidate page cache after io complete, refer to dio_complete ->
> kiocb_invalidate_post_direct_write
>     4) release budget, refer to do_writepage ->
> release_new_page_budget/release_existing_page_budget

Ok. Let me give myself a few days to study.

> 2. Things should be done in read function:
>     1) sync page cache data to disk, make sure that dio reading get the
> latest user data, refer to __blockdev_direct_IO ->
> filemap_write_and_wait_range
>     2) read data from UHIFS, refer to do_readpage -> read_block
> Maybe there are many trivial details during developing.

Here the nice thing is to know how you test at the moment the ubifs and easy way
to development it.

Michael

> >
> > Michael
> > .
> >
>


-- 
Michael Nazzareno Trimarchi
Co-Founder & Chief Executive Officer
M. +39 347 913 2170
michael at amarulasolutions.com
__________________________________

Amarula Solutions BV
Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
T. +31 (0)85 111 9172
info at amarulasolutions.com
www.amarulasolutions.com