[RFC PATCH v2 0/10] makedumpfile: cyclic processing to keep memory consumption.

[Atsushi Kumagai]

Hello,

I improved prototype of cyclic processing as version 2.
If there is no objection to basic idea, I want to consider the things
related to performance as next step. (Concretely, buffer size and the patch set
HATAYAMA-san sent a short time ago.) 


Version 1:
 
  http://lists.infradead.org/pipermail/kexec/2012-May/006363.html

Introduction:

 - The purpose of cyclic processing is to fix memory consumption.
 - cyclic processing doesn't use temporary bitmap file, store partial bitmap data
   in memory only for each cycle instead.
 - The prototype was passed the regression test which is used by every release.

How to use:
  
  Specify '--cyclic' option, then makedumpfile works cyclically.

Example:

    $ makedumpfile --cyclic -cd31 vmcore testdump.cd31
    Copying data                       : [  5 %]
    Excluding free pages               : [100 %]
    ...
    Excluding free pages               : [100 %]
    Copying data                       : [100 %]

    The dumpfile is saved to testdump.cd31.

    makedumpfile Completed.

Changelog:

  v1 =>  v2:

    - Fix the process of increasing target region.
    - Change method for kdump-compressed format.
    - Add support for ELF format.
    - Add support for --split option.
 
Memory consumption:

  I measured the RSS of makedumpfile with ps(1) as memory consumption.

    a. working for 5G memory:
                 |                 RSS [KB]                
                 |  no option  |    -cd31    |    -Ed31
      -----------+-------------+-------------+-------------
        v1.4.4   |     1108    |     1184    |      868
        cyclic   |     2976    |     3252    |     2984

        
    b. working for 8G memory:
                 |                 RSS [KB]                
                 |  no option  |    -cd31    |    -Ed31
      -----------+-------------+-------------+-------------
        v1.4.4   |     1108    |     1180    |      864
        cyclic   |     2972    |     3256    |     2984


  This result seems to say that v1.4.4 is better than cyclic, but the size
  of temporary bitmap file grows based on memory size. The increasing rate 
  can be represented as (memory size / 4K / 8 ) * 2.

     memory size [GB] |  bitmap size [KB]
    ------------------+------------------
            5         |       320
            8         |       512
           ...        |       ...
          1,024       |      65,536

  Even above size will be counted as memory consumption, if the system 
  doesn't mount rootfs. This is the cause of the memory consumption issue
  we discussed.

  On the other hand, cyclic processing doesn't use temporary bitmap files,
  all memory consumption will be appeared in RSS.
  The memory consumption to store bitmap will be kept around 2MB(BUFSIZE_CYCLIC * 2).
  

Thanks
Atsushi Kumagai