[RFC] makedumpfile-1.5.1 RC

[Hatayama, Daisuke]

> -----Original Message-----
> From: kexec-bounces at lists.infradead.org
> [mailto:kexec-bounces at lists.infradead.org] On Behalf Of Vivek Goyal
> Sent: Wednesday, November 21, 2012 10:54 PM
> To: Lisa Mitchell
> Cc: kexec at lists.infradead.org; Atsushi Kumagai; Hoemann, Jerry; Cliff
> Wickman
> Subject: Re: [RFC] makedumpfile-1.5.1 RC
[...]
> > The changes proposed by Ciff Wickman in
> > http://lists.infradead.org/pipermail/kexec/2012-November/007178.html
> > sound like they could bring big improvements in performance, so these
> > should be investigated.  I would like to try a version of them built on
> > top of makedumpfile v1.5.1-rc, to try on our 4 TB system, to see what
> > performance gains we can get, as an experiment.
> 
> I am wondering if it is time to look into parallel processing. Somebody
> was working on bringing up more cpus in kdump kernel. If that works, the
> probably multiple makedumpfile threads can try to filter out different
> sections of physical memory.
> 

Makedumpfile has already had such parallel processing feature:

$ ./makedumpfile --help
...
  [--split]:
      Split the dump data to multiple DUMPFILEs in parallel. If specifying
      DUMPFILEs on different storage devices, a device can share I/O load with
      other devices and it reduces time for saving the dump data. The file size
      of each DUMPFILE is smaller than the system memory size which is divided
      by the number of DUMPFILEs.
      This feature supports only the kdump-compressed format.

Doing makedumpfile like:

  $ makedumpfile --split dumpfile vmcore1 vmcore2 [vmcore3 ... vmcore_n]

original dumpfile are splitted into n vmcores of kdump-compressed formats, each of
which has the same size basically and where n processes are used, not threads.
(The author told me the reason why process was chosen that he didn't want to put
relatively large libc library in the 2nd kernel at that time. But recently, libc library is
present on the 2nd kernel as scp needs to use it. This might no longer pointless.)

I think Cliff's idea works orthogonally to parallel processing. I'll also test it on our
machine.

Also, sorry for delaying the work on multiple cpus in the 2nd kernel. Posting new
version of the patch set is delayed a few weeks more. But it's possible to wake up
AP cpus in the 2nd kernel safely if BIOS always assigns 0 lapicid to BSP since
then if kexec enteres 2nd kernel with some AP lcpu, kernel always assigns 1 lcpu
number to BSP lcpu. So, maxcpus=1 and waking up cpus except for 1 lcpu works
as a workaround. If anyone wants to bench with parallel processing, please do it
like this.

Thanks.
HATAYAMA, Daisuke