makedumpfile-1.5.9: Introduce parallel compression processing.

Atsushi Kumagai ats-kumagai at
Tue Oct 27 23:31:50 PDT 2015


makedumpfile version 1.5.9 is released.
Your comments/patches are welcome.

Main new feature:
o Parallel compression
 - Add the new option "--num-threads" to enable parallel compression
   for faster processing. This option intends parallel compression
   while --split intends parallel i/o, they try to solve different

o Change the way to choose the non-cyclic mode
 - Add the new option "--work-dir" to specify the directory used to
   store the bitmap. This is the alternative to the combination of
   --non-cyclic and TMPDIR.

o Support new kernels
 - The supported kernel is updated to 4.1 in this version.
   At first I'm going to extend the supported version to 4.2, but
   I found an issue that makedumpfile seems to exclude necessary pages
   by mistake on linux 4.2. When crash-7.1.3 reads that filtered dump file,
   the following message is shown.

      crash: page excluded: kernel virtual address: f3fe0000  type: "fill_task_struct"

   This will be fixed in the next version.

o New feature
    - [PATCH] Support ARM64. (by Pratyush Anand) 94d47d1
    - [PATCH 1/2] Enable compressed dump formats for Xen. (by Petr Tesarik) 349a0ed
    - [PATCH 2/2] Remove notes about ELF being the only available format for Xen dumps.
      (by Petr Tesarik) 5cab480
    - [PATCH v5 01/10] Add readpage_kdump_compressed_parallel. (by Qiao Nuohan) 5fab021
    - [PATCH v5 02/10] Add mappage_elf_parallel. (by Qiao Nuohan) edca232
    - [PATCH v5 03/10] Add readpage_elf_parallel. (by Qiao Nuohan) 7f15b41
    - [PATCH v5 04/10] Add read_pfn_parallel. (by Qiao Nuohan) a7243fe
    - [PATCH v5 05/10] Add function to initial bitmap for parallel use. (by Qiao Nuohan) 56a6bc4
    - [PATCH v5 06/10] Add filter_data_buffer_parallel. (by Qiao Nuohan) 2a7cb9f
    - [PATCH v5 07/10] Add write_kdump_pages_parallel to allow parallel process. (by Qiao Nuohan) c2595d9
    - [PATCH v5 08/10] Initial and free data used for parallel process. (by Qiao Nuohan) 4538848
    - [PATCH v5 09/10] Make makedumpfile available to read and compress pages parallelly.
      (by Qiao Nuohan) b265102
    - [PATCH v5 10/10] Add usage and manual about multiple threads process. (by Qiao Nuohan) b18a8a6

o Bugfix
    - [PATCH] Correct vmap_area_list support for i386 and ppc32. (by Atsushi Kumagai) 4194d7b
    - [PATCH] Free pages aren't removed in non-cyclic mode. (by Atsushi Kumagai) d64cf7f
    - [PATCH v2] Fix a bug in generating incomplete kdump core. (by Zhou Wenjian) 9cee495
    - [PATCH] Remove the double free of sph. (by Chao Fan) 72dec41
    - [PATCH 1/2] sadump: Change bit order. (by HATAYAMA Daisuke) 5f15256
    - [PATCH 2/2] sadump: Perform explicit zero page filtering. (by HATAYAMA Daisuke) 19b3a91

o Cleanup
    - [PATCH 01/13] Organize bitmap structure for cyclic logic. (by Atsushi Kumagai) 5fc24bf
    - [PATCH 02/13] Add option to specify working directory for the bitmap. (by Atsushi Kumagai) 5c5a8b8
    - [PATCH 03/13] Integrate the entry point of is_dumpable(). (by Atsushi Kumagai) 5c8c680
    - [PATCH 04/13] Integrate the main logic of writing kdump file. (by Atsushi Kumagai) d18796d
    - [PATCH 05/13] Communalize the function for creating 1st bitmap. (by Atsushi Kumagai) b00a7b0
    - [PATCH 06/13] Remove the old logic of writing kdump pages. (by Atsushi Kumagai) 4d55542
    - [PATCH 07/13] Integrate filtering process for ELF path. (by Atsushi Kumagai) 679a818
    - [PATCH 08/13] Remove the old logic of writing ELF pages. (by Atsushi Kumagai) 672354b
    - [PATCH 09/13] Adjust --mem-usage path to the new code. (by Atsushi Kumagai) 57153b0
    - [PATCH 10/13] Adjust --split/--reassemble path to the new code. (by Atsushi Kumagai) 95435de
    - [PATCH 11/13] Adjust refiltering path to the new code. (by Atsushi Kumagai) c7fb41c
    - [PATCH 12/13] Adjust sadump path to the new code. (by Atsushi Kumagai) edd2bf2
    - [PATCH 13/13] Remove --non-cyclic option. (by Atsushi Kumagai) a117ae9
    - [PATCH] Optimise the function reserve_diskspace. (by Zhou Wenjian) 3605073
    - [PATCH v2] Add description of pages lost by ENOSPACE in IMPLEMENTATION. (by Zhou Wenjian) 0e20f33
    - [PATCH 1/2] Enable compressed dump formats for Xen. (by Petr Tesarik) 349a0ed
    - [PATCH 2/2] Remove notes about ELF being the only available format for Xen dumps.
      (by Petr Tesarik) 5cab480
    - [PATCH v2] Code changes to satisfy the coverity scan. (by Chao Fan) 0c65e38
    - [PATCH] Improve performance for parallel compression with zlib. (by Atsushi Kumagai) 3ef6629

Explanation of makedumpfile:
  To shorten the size of the dumpfile and the time of creating the
  dumpfile, makedumpfile copies only the necessary pages for analysis
  to the dumpfile from /proc/vmcore. You can specify the kind of
  unnecessary pages with dump_level. If you want to shorten the size
  further, enable the compression of the page data.

  You can download the latest makedumpfile from the following URL.
  Details of the change are written on the git page of the following site.

Method of installation:
  You can compile the makedumpfile command as follows;
  1. "tar -zxvf makedumpfile-x.y.z.tar.gz"
  2. "cd makedumpfile-x.y.z"
  3. "make; make install"

  makedumpfile [-c] [-E] [-d dump_level] [-x vmlinux] dump_mem dump_file

  If you want to exclude pages filled by zero, cache pages, user pages
  and free pages and to enable compression, please execute the following

  # makedumpfile -c -d 31 /proc/vmcore dumpfile

Atsushi Kumagai

More information about the kexec mailing list