[PATCH 2/2 V2] exclude unused vmemmap pages

[Dave Anderson]

----- Original Message -----
>
> provides some requested changes:
> - remove the automatic exclusion of page structures for memories over 1TB; it
>   will only be done by explicit request (-e)
> - remove the -N option; no need to explicitly include unused vmemmap pages
>   as they will be included by default
> - add DUMP_DH_EXCLUDED_VMEMMAP to the dump header; to warn crash users that
>   these page structures are excluded
> - fix the making of the filename for pfn file (in init_save_control())
> but still is only tested on a disk dump of an x86_64 system.
> 
> Exclude kernel pages that contain nothing but page structures for pages
> that are not being included in the dump.
> These can amount to 3.67 million pages per terabyte of system memory!
> 
> The kernel's page table, starting at virtual address 0xffffea0000000000, is
> searched to find the actual pages containing the vmemmap page structures.
> 
> Bitmap1 is a map of dumpable (i.e existing) pages. Bitmap2 is a map
> of pages not to be excluded.
> To speed the search of bitmaps only whole 64-bit words of 1's in
> bitmap1 and 0's in bitmap2 are tested to see if they are vmemmap pages.
> 
> The list of vmemmap pfn's to be excluded is written to a small file in order
> to conserve crash kernel memory.
> 
> In practice, this whole procedure only takes about 10 seconds on a
> 16TB machine.
> 
> Signed-off-by: Cliff Wickman <cpw at sgi.com>

... [ cut ] ...

> +#define OPT_EXCLUDEVM           'e'
>  #define OPT_ELF_DUMPFILE        'E'
>  #define OPT_FLATTEN             'F'
>  #define OPT_FORCE               'f'
> Index: makedumpfile-1.5.5/print_info.c
> ===================================================================
> --- makedumpfile-1.5.5.orig/print_info.c
> +++ makedumpfile-1.5.5/print_info.c
> @@ -48,7 +48,7 @@ print_usage(void)
>  	MSG("\n");
>  	MSG("Usage:\n");
>  	MSG("  Creating DUMPFILE:\n");
> -	MSG("  # makedumpfile    [-c|-l|-E] [-d DL] [-j] [-J] [-x VMLINUX|-i VMCOREINFO] VMCORE\n");
> +	MSG("  # makedumpfile    [-c|-l|-E|-j|-J|-e|-N] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE\n");
>  	MSG("    DUMPFILE\n");
>  	MSG("\n");
>  	MSG("  Creating DUMPFILE with filtered kernel data specified through filter config\n");
> @@ -101,6 +101,10 @@ print_usage(void)
>  	MSG("  [-J]:\n");
>  	MSG("      Use raw (O_DIRECT) i/o on bitmap file to avoid expanding kernel pagecache.\n");
>  	MSG("\n");
> +	MSG("  [-e]:\n");
> +	MSG("      Exclude page structures (vmemmap) for unused pages.\n");
> +	MSG("      (the default is to capture them all, which amounts to 3.67M pages per terabyte)\n");
> +	MSG("\n");
>  	MSG("  [-d DL]:\n");
>  	MSG("      Specify the type of unnecessary page for analysis.\n");
>  	MSG("      Pages of the specified type are not copied to DUMPFILE. The page
>  	type\n");

Perhaps there should be a warning above concerning the potential for crash analysis
problems?  The description above makes it sound like -e is similar in nature (and
therefore as harmless) as the -d <level> option.

It could be argued that there's no such warning for eppic/erasures, but with eppic
it's far less likely to result in crash analysis failures.  With -e, the user is
virtually guaranteed to have issues.

I realize that this is beating a dead horse, but again, this is an option
that should very rarely be used -- and if applied, the user/administrator should
be well aware of the ramifications. 

Dave