[RFC PATCH 0/4] makedumpfile: cyclic processing to keep memory consumption.

[HATAYAMA Daisuke]

From: Maxim Uvarov <muvarov at gmail.com>
Subject: Re: [RFC PATCH 0/4] makedumpfile: cyclic processing to keep memory consumption.
Date: Mon, 4 Jun 2012 12:29:07 +0400

> 2012/6/4 Atsushi Kumagai <kumagai-atsushi at mxc.nes.nec.co.jp>
> 
>> Hello Maxim,
>>
>> On Thu, 31 May 2012 10:50:39 +0400
>> Maxim Uvarov <muvarov at gmail.com> wrote:
>>

>>
>> After received your mail, I measured executing time with the patch below.
>> The result below was measured in 5GB memory machine.
>>
>> Result:
>>  a. makedumpfile -Kcd31
>>    excluding time:   6.55 [sec]
>>    writing time:     5.89 [sec]
>>
>>  b. makedumpfile -cd31
>>    excluding time:   0.21 [sec]
>>    writing time:     5.82 [sec]
>>
>> I don't think this result is good, the prototype has some points to be
>> improved for performance.
>> (e.g. buffer size, the way to exclude free pages, etc...)
>>
>> However, I think getting a bit of speed down can't be helped to keep
>> memory consumption.
>>
>>
> I'm asking because I have machines with several TBs of ram. And filtering
> and saving data from vmcore takes long time. (more then 30 minutes). I
> wonder if somebody already work on performance. Probably we can do SMP
> optimizations.
> 
> Maxim.

Hello Maxim,

I'm now benchmarking for three kinds of pralell compressions: no
compression, lzo and snappy. I'll post its result after completing it,
by Friday? if possible.

Also I'll soon post a patch for filtering free pages in constant time
based on mem_map, just a small extension of the patch Kumagai-san
posted previously, by which filtering score for -Kcd31 above would be
improved so much; it now repeats tracing free_list (memory size /
cyclic_size) times.

Thanks.
HATAYAMA, Daisuke