[RFC PATCH 2/4] makedumpfile: Add partial bitmap for cyclic.

[HATAYAMA Daisuke]

From: Atsushi Kumagai <kumagai-atsushi at mxc.nes.nec.co.jp>
Subject: Re: [RFC PATCH 2/4] makedumpfile: Add partial bitmap for cyclic.
Date: Tue, 12 Jun 2012 17:30:04 +0900

> Hello HATAYAMA-san,
> 
> On Thu, 7 Jun 2012 15:04:36 +0900
> Atsushi Kumagai <kumagai-atsushi at mxc.nes.nec.co.jp> wrote:
> 
>> On Mon, 04 Jun 2012 12:19:57 +0900 (JST)
>> HATAYAMA Daisuke <d.hatayama at jp.fujitsu.com> wrote:
>> 
>> > > + */
>> > > +#define BUFSIZE_CYCLIC	        (8192)
>> > > +#define PFN_CYCLIC		(BUFSIZE_CYCLIC * BITPERBYTE)
>> > > +
>> > > +
>> > > +/*
>> > >   * Minimam vmcore has 2 ProgramHeaderTables(PT_NOTE and PT_LOAD).
>> > 
>> > How did you choose this buffer size? Best performance? I think the
>> > performacne varies according to this buffer size, and to evaluate it,
>> > it's useful to be able to specify the size through command-line.
>>
>> Honestly, the buffer size has no means, I defined it without a specific reason
>> at first.
>> 
>> However, it's understandable that the size is important key for performance and
>> I agree with your opinion that enabling to specify the size is useful.
>> 
>> But, I noticed just now, prototype with increased buffer size isn't work correctly.
>> 
>>  - BUFSIZE_CYCLIC = 8192:
>> 
>>     $ makedumpfile -Kcd31 vmcore testdump.Kcd31
>>     ...
>>     The dumpfile is saved to testdump.Kcd31.
>> 
>>     makedumpfile Completed.
>>     $ ls -l testdump.*
>>     -rw------- 1 kumagai kumagai 21167309  6月  7 13:55 testdump.Kcd31
>>     -rw------- 1 kumagai kumagai 21167309  6月  7 13:28 testdump.cd31
>>     $
>> 
>>  - BUFSIZE_CYCLIC = 16384:
>> 
>>     $ makedumpfile -Kcd31 vmcore testdump.Kcd31
>>     ...
>>     The dumpfile is saved to testdump.Kcd31.
>> 
>>     makedumpfile Completed.
>>     $ ls -l testdump.*
>>     -rw------- 1 kumagai kumagai 18836948  6月  7 13:59 testdump.Kcd31
>>     -rw------- 1 kumagai kumagai 21167309  6月  7 13:28 testdump.cd31
>>     $
>> 
>> 
>> I must review my code quickly.
> 
> I found my mistake in the process of incraesing target region and
> show the modification at the end of this email.
> 
> On Mon, 4 Jun 2012 16:54:49 +0900
> Atsushi Kumagai <kumagai-atsushi at mxc.nes.nec.co.jp> wrote:
> [...]
>> After received your mail, I measured executing time with the patch below.
>> The result below was measured in 5GB memory machine.
>> 
>> Result:
>>   a. makedumpfile -Kcd31 
>>     excluding time:   6.55 [sec]
>>     writing time:     5.89 [sec]
> 
> I measured executing time with BUFSIZE=8192 on 4 June as above.
> Additionally, I measured executing time with BUFSIZE=16384, 24576 and 32768
> in the same environment.
> 
> Result:
>     ------------------------------------------------------------------------
>     buffer size [byte]	     excluding time [sec]        writing time [sec]
>     ------------------------------------------------------------------------
>     	  8192		 	    6.55			 5.89
>     	 16384		 	    3.49			 5.91
>     	 24576		 	    2.39			 5.96
>     	 32768		 	    1.83			 5.96
>     ------------------------------------------------------------------------
> 
> It seems that the buffer size is really effective in performance.
> Even if we choose the way without tracing free_list, the overhead of cyclic processing
> will be reduced, so I expect some speedup.

Performacne of the excluding time looks improved propoertional to
buffer size. I guess this result means part of processing to switch to
next cycle is so much heavy. I'll follow something for further
improvement around there.

> 
> If enabling to specify the buffer size, I think the information for decision of 
> buffer size is needed. How do you decide the buffer size ?
> 

Proper buffer size depends on machine spec. I think it natural to
choose the largest common divisor as default. OTOH, on the environment
where users need maximal speed definitely, such as on the machine
where large memory where a little bit time difference can reach a big
difference in total, and such as when used in kdump, they would want
to know the proper values by actually running makedumpfile for a
variety of parameters relevant to speed. For such tool that make it
easy to do this, I imagine a kind of tool to search for fatest mirror
site.

Thanks.
HATAYAMA, Daisuke