[RFC PATCH v2 0/10] makedumpfile: cyclic processing to keep memory consumption.

[Atsushi Kumagai]

Hello HATAYAMA-san,

On Fri, 13 Jul 2012 14:18:01 +0900 (JST)
HATAYAMA Daisuke <d.hatayama at jp.fujitsu.com> wrote:

> From: Atsushi Kumagai <kumagai-atsushi at mxc.nes.nec.co.jp>
> Subject: Re: [RFC PATCH v2 0/10] makedumpfile: cyclic processing to keep memory consumption.
> Date: Fri, 13 Jul 2012 09:36:11 +0900
> 
> > Hello HATAYAMA-san,
> > 
> > On Wed, 11 Jul 2012 14:23:59 +0900 (JST)
> > HATAYAMA Daisuke <d.hatayama at jp.fujitsu.com> wrote:
> > 
> > [...]
> >> >>> Hi Atushi san,
> >> >>> 
> >> >>> Good to see this work making progress. I have few queries.
> >> >>> 
> >> >>> - Do you have some numbers for bigger machines like 1TB or higher memory. 
> >> >>>   I am curious to know how bad is the time penalty.
> >> >> 
> >> >> I'm afraid that I don't have such a large machine, so I need someone who can
> >> >> measure execution time in large machine.
> >> >> 
> >> > 
> >> > I can prepare such machine but I cannot say I can use the machine on
> >> > this day precisely for example. But I think it must be until the end
> >> > of this month at most. So, I would like to fix content of benchmark
> >> > first.
> >> > 
> >> 
> >> Hello Kumagai-san,
> >> 
> >> I'm now trying to reserve the machine with big memory, but I'm not
> >> accurately sure when I can use the machine; expecting within this
> >> month? In advance, I want to make sure what I measure in this
> >> benchmark in more detail.
> > 
> > OK, I'm glad for your help.
> > 
> >> I'm going to collect at least what you showed in your benchmark: RSS
> >> size for no option, -cd31 and -Ed31. Is there another to collect?
> > 
> > Would you collect RSS also for --split ?
> > While I expect the RSS will be increase based on number of child processes,
> > I want to see measured data.
> > 
> 
> Yes, I'll collect RSS information. But due to COW, child processes
> share parent process's memory until they modify their memory, so
> simply looking value of RSS is not enough. We need to measure size of
> shared part e.g. by looking at /proc/<PID>/smaps.

You're right.  Would you choose the way which can correct effective
memory consumption ?

> And, what version of makedumpfile should I use for this bench? If you
> have additonal changes locally, it might be better to use the version.

Please use v1.4.4 + v2 patch of cyclic + performance improvement patch which
I sent today, it's the latest version internally.


Thanks
Atsushi Kumagai