From olaf at aepfle.de  Mon May  2 06:22:30 2011
From: olaf at aepfle.de (Olaf Hering)
Date: Mon, 2 May 2011 12:22:30 +0200
Subject: dynamic oldmem in kdump kernel
In-Reply-To: <20110407095646.GA30788@aepfle.de>
References: <20110407095646.GA30788@aepfle.de>
Message-ID: <20110502102230.GA9497@aepfle.de>

On Thu, Apr 07, Olaf Hering wrote:

> Recently kdump for pv-on-hvm Xen guests was implemented by me.
> 
> One issue remains:
> The xen_balloon driver in the guest frees guest pages and gives them
> back to the hypervisor. These pages are marked as mmio in the
> hypervisor. During a read of such a page via the /proc/vmcore interface
> the hypervisor calls the qemu-dm process. qemu-dm tries to map the page,
> this attempt fails because the page is not backed by ram and 0xff is
> returned. All this generates high load in dom0 because the reads come
> as 8byte requests.
> 
> There seems to be no way to make the crash kernel aware of the state of
> individual pages in the crashed kernel, it is not aware of memory
> ballooning. And doing that from within the "kernel to crash" seems error
> prone. Since over time the fragmentation will increase, it would be best
> if the crash kernel itself queries the state of oldmem pages. 


Here is a version that works for me.

A hook is called for each pfn. If the hook returns 0 the pfn is not ram,
and the range is cleared, for sparse vmcore files.
Otherwise the pfn is handled as ram. In the worst case each read attempt
will be handled by qemu-dm process which causes just some load in dom0.


This patch still lacks locking.
How can I make sure unregister_oldmem_pfn_is_ram() is not called while
the loop in read_from_oldmem() is still active?
Is there an example of similar code already in the kernel?

Olaf

---
 fs/proc/vmcore.c           |   29 ++++++++++++++++++++++++++---
 include/linux/crash_dump.h |    5 +++++
 2 files changed, 31 insertions(+), 3 deletions(-)

Index: linux-2.6.39-rc5/fs/proc/vmcore.c
===================================================================
--- linux-2.6.39-rc5.orig/fs/proc/vmcore.c
+++ linux-2.6.39-rc5/fs/proc/vmcore.c
@@ -35,6 +35,22 @@ static u64 vmcore_size;
 
 static struct proc_dir_entry *proc_vmcore = NULL;
 
+/* returns > 0 for RAM pages, 0 for non-RAM pages, < 0 on error */
+static int (*oldmem_pfn_is_ram)(unsigned long pfn);
+
+void register_oldmem_pfn_is_ram(int (*fn)(unsigned long))
+{
+	oldmem_pfn_is_ram = fn;
+}
+
+void unregister_oldmem_pfn_is_ram(void)
+{
+	oldmem_pfn_is_ram = NULL;
+}
+
+EXPORT_SYMBOL_GPL(register_oldmem_pfn_is_ram);
+EXPORT_SYMBOL_GPL(unregister_oldmem_pfn_is_ram);
+
 /* Reads a page from the oldmem device from given offset. */
 static ssize_t read_from_oldmem(char *buf, size_t count,
 				u64 *ppos, int userbuf)
@@ -42,6 +58,7 @@ static ssize_t read_from_oldmem(char *bu
 	unsigned long pfn, offset;
 	size_t nr_bytes;
 	ssize_t read = 0, tmp;
+	int (*fn)(unsigned long);
 
 	if (!count)
 		return 0;
@@ -55,9 +72,15 @@ static ssize_t read_from_oldmem(char *bu
 		else
 			nr_bytes = count;
 
-		tmp = copy_oldmem_page(pfn, buf, nr_bytes, offset, userbuf);
-		if (tmp < 0)
-			return tmp;
+		fn = oldmem_pfn_is_ram;
+		/* if pfn is not ram, return zeros for spares dump files */
+		if (fn && fn(pfn) == 0)
+			memset(buf, 0, nr_bytes);
+		else {
+			tmp = copy_oldmem_page(pfn, buf, nr_bytes, offset, userbuf);
+			if (tmp < 0)
+				return tmp;
+		}
 		*ppos += nr_bytes;
 		count -= nr_bytes;
 		buf += nr_bytes;
Index: linux-2.6.39-rc5/include/linux/crash_dump.h
===================================================================
--- linux-2.6.39-rc5.orig/include/linux/crash_dump.h
+++ linux-2.6.39-rc5/include/linux/crash_dump.h
@@ -66,6 +66,11 @@ static inline void vmcore_unusable(void)
 	if (is_kdump_kernel())
 		elfcorehdr_addr = ELFCORE_ADDR_ERR;
 }
+
+#define HAVE_OLDMEM_PFN_IS_RAM 1
+extern void register_oldmem_pfn_is_ram(int (*fn)(unsigned long));
+extern void unregister_oldmem_pfn_is_ram(void);
+
 #else /* !CONFIG_CRASH_DUMP */
 static inline int is_kdump_kernel(void) { return 0; }
 #endif /* CONFIG_CRASH_DUMP */


From olaf at aepfle.de  Tue May  3 15:08:06 2011
From: olaf at aepfle.de (Olaf Hering)
Date: Tue, 3 May 2011 21:08:06 +0200
Subject: [PATCH] add hook to read_from_oldmem() to check for non-ram pages
In-Reply-To: <20110407095646.GA30788@aepfle.de>
References: <20110407095646.GA30788@aepfle.de>
Message-ID: <20110503190806.GA12485@aepfle.de>


The balloon driver in a Xen guest frees guest pages and marks them as
mmio. When the kernel crashes and the crash kernel attempts to read the
oldmem via /proc/vmcore a read from ballooned pages will generate 100%
load in dom0 because Xen asks qemu-dm for the page content. Since the
reads come in as 8byte requests each ballooned page is tried 512 times.

With this change a hook can be registered which checks wether the given
pfn is really ram. The hook has to return a value > 0 for ram pages, a
value < 0 on error (because the hypercall is not known) and 0 for
non-ram pages.

This will reduce the time to read /proc/vmcore.  Without this change a
512M guest with 128M crashkernel region needs 200 seconds to read it,
with this change it takes just 2 seconds.


Signed-off-by: Olaf Hering <olaf at aepfle.de>

---
 fs/proc/vmcore.c           |   50 ++++++++++++++++++++++++++++++++++++++++++---
 include/linux/crash_dump.h |    5 ++++
 2 files changed, 52 insertions(+), 3 deletions(-)

Index: linux-2.6.39-rc5/fs/proc/vmcore.c
===================================================================
--- linux-2.6.39-rc5.orig/fs/proc/vmcore.c
+++ linux-2.6.39-rc5/fs/proc/vmcore.c
@@ -18,6 +18,7 @@
 #include <linux/init.h>
 #include <linux/crash_dump.h>
 #include <linux/list.h>
+#include <linux/wait.h>
 #include <asm/uaccess.h>
 #include <asm/io.h>
 
@@ -35,6 +36,44 @@ static u64 vmcore_size;
 
 static struct proc_dir_entry *proc_vmcore = NULL;
 
+/* returns > 0 for RAM pages, 0 for non-RAM pages, < 0 on error */
+static int (*oldmem_pfn_is_ram)(unsigned long pfn);
+static DECLARE_WAIT_QUEUE_HEAD(oldmem_fn_waitq);
+static atomic_t oldmem_fn_refcount = ATOMIC_INIT(0);
+
+void register_oldmem_pfn_is_ram(int (*fn)(unsigned long))
+{
+	if (oldmem_pfn_is_ram == NULL)
+		oldmem_pfn_is_ram = fn;
+}
+
+void unregister_oldmem_pfn_is_ram(void)
+{
+	wait_event(oldmem_fn_waitq, atomic_read(&oldmem_fn_refcount) == 0);
+	oldmem_pfn_is_ram = NULL;
+	wmb();
+}
+
+static int pfn_is_ram(unsigned long pfn)
+{
+	int (*fn)(unsigned long);
+	/* pfn is ram unless fn() checks pagetype */
+	int ret = 1;
+
+	atomic_inc(&oldmem_fn_refcount);
+	smp_mb__after_atomic_inc();
+	fn = oldmem_pfn_is_ram;
+	if (fn)
+		ret = fn(pfn);
+	if (atomic_dec_and_test(&oldmem_fn_refcount))
+		wake_up(&oldmem_fn_waitq);
+
+	return ret;
+}
+
+EXPORT_SYMBOL_GPL(register_oldmem_pfn_is_ram);
+EXPORT_SYMBOL_GPL(unregister_oldmem_pfn_is_ram);
+
 /* Reads a page from the oldmem device from given offset. */
 static ssize_t read_from_oldmem(char *buf, size_t count,
 				u64 *ppos, int userbuf)
@@ -55,9 +94,14 @@ static ssize_t read_from_oldmem(char *bu
 		else
 			nr_bytes = count;
 
-		tmp = copy_oldmem_page(pfn, buf, nr_bytes, offset, userbuf);
-		if (tmp < 0)
-			return tmp;
+		/* if pfn is not ram, return zeros for spares dump files */
+		if (pfn_is_ram(pfn) == 0)
+			memset(buf, 0, nr_bytes);
+		else {
+			tmp = copy_oldmem_page(pfn, buf, nr_bytes, offset, userbuf);
+			if (tmp < 0)
+				return tmp;
+		}
 		*ppos += nr_bytes;
 		count -= nr_bytes;
 		buf += nr_bytes;
Index: linux-2.6.39-rc5/include/linux/crash_dump.h
===================================================================
--- linux-2.6.39-rc5.orig/include/linux/crash_dump.h
+++ linux-2.6.39-rc5/include/linux/crash_dump.h
@@ -66,6 +66,11 @@ static inline void vmcore_unusable(void)
 	if (is_kdump_kernel())
 		elfcorehdr_addr = ELFCORE_ADDR_ERR;
 }
+
+#define HAVE_OLDMEM_PFN_IS_RAM 1
+extern void register_oldmem_pfn_is_ram(int (*fn)(unsigned long));
+extern void unregister_oldmem_pfn_is_ram(void);
+
 #else /* !CONFIG_CRASH_DUMP */
 static inline int is_kdump_kernel(void) { return 0; }
 #endif /* CONFIG_CRASH_DUMP */


From olaf at aepfle.de  Wed May  4 12:32:32 2011
From: olaf at aepfle.de (Olaf Hering)
Date: Wed, 4 May 2011 18:32:32 +0200
Subject: [PATCH] detect Xen dom0 properly to avoid conflict with pv_on_hvm domU
Message-ID: <20110504163232.GA24669@aepfle.de>

A Xen HVM guest with PV drivers loaded has also a /proc/xen directory.
But such a guest is an ordinary PC and the special handling for dom0
breaks kdump in this environment.
Test for /proc/xen/capabilities instead and cache the result.

Also make two variables static, they are only used in this file.

Signed-off-by: Olaf Hering <olaf at aepfle.de>

---
 kexec/crashdump-xen.c |   15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

Index: kexec-tools-0.8.15/kexec/crashdump-xen.c
===================================================================
--- kexec-tools-0.8.15.orig/kexec/crashdump-xen.c
+++ kexec-tools-0.8.15/kexec/crashdump-xen.c
@@ -25,14 +25,19 @@ struct crash_note_info {
 	unsigned long length;
 };
 
-int xen_phys_cpus = 0;
-struct crash_note_info *xen_phys_notes;
+static int xen_phys_cpus;
+static struct crash_note_info *xen_phys_notes;
+static int is_dom0;
 
 int xen_present(void)
 {
-	struct stat buf;
-
-	return stat("/proc/xen", &buf) == 0;
+	if (!is_dom0) {
+		if (access("/proc/xen/capabilities", F_OK) == 0)
+			is_dom0 = 1;
+		else
+			is_dom0 = -1;
+	}
+	return is_dom0 > 0;
 }
 
 unsigned long xen_architecture(struct crash_elf_info *elf_info)


From sujit.linux at gmail.com  Wed May  4 14:35:46 2011
From: sujit.linux at gmail.com (Sujit V)
Date: Wed, 4 May 2011 11:35:46 -0700
Subject: Kexec & Memory Zones question
Message-ID: <BANLkTinOd+kmJKMdtW7saW0eWHN9xbO_3w@mail.gmail.com>

On our x86_64 NUMA hardware running linux 2.6.23 with two memory nodes
 have the following zone layout
DMA   0 - 16MB
DMA32   16MB to 4GB
NORMAL 4GB to 96GB

We had the crashkernel boot param as 128M at 16M.
I am using kexec-tools-2.0.0

We observed that the reserved crashkernel memory was getting used by
the system.
We found out by kdb  that around 17MB of physical memory the memory
contents were changing which proved that the system was
using the memory which is actually reserved for crash kernel.
[ If I load crash kernel using kexec then the system would crash. The
back trace was always in the megasas driver. ]

My Question is
1) Should the crashkernel memory be located past the DMA32 zone.?

I have tried the following
(1) crashkernel=128M at 4GB  ( So the memory reservation is past DMA32)
In this scenario the kexec tools gave an error "Could not find a free
area of memory of xyz bytes"

(2) I changed the max of DMA32 to 1GB
     crashkernel=128M at 1G
Still kexec gave the same error "Could not find a free area of memory
of xyz bytes"

Is there any specific restriction on where the crashkernel memory
could be located.?

Is it ok to be in the DMA32 region OR it should be beyond the DMA32 region.

Thanks


From prasad at linux.vnet.ibm.com  Wed May  4 15:35:09 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 5 May 2011 01:05:09 +0530
Subject: [RFC] Kdump and memory error handling
Message-ID: <20110504193509.GA5342@in.ibm.com>

Hi All,
        We've been trying to study and improve the kdump behaviour when
a panic is triggered due to an unrecoverable memory error causing a
machine check exception (MCE) followed by a kernel panic.

In this context we foresee a few issues in capturing kdump and would
like to receive comments about the ways to handle them.

Probable Issues when capturing coredump through kdump following a memory
error
---------------------------
- First, a coredump of the memory from the crashing kernel isn't really
  helpful in debugging the crash that was caused due to a faulty memory.
  Collecting the same has some of the problems illustrated below. It should
  therefore suffice to let the user know the reason of the crash
  rather than provide a complete dump of the memory.

  For this, a 'slim' yet crash-tool readable coredump containing:
  - message about the cause (such as crash due to unrecoverable memory error)
    in the coredump's elf-note section.
  - and no data from the memory of the 'crashing' kernel (their elf
    sections can be reduced to zero length).
  may be suitable.

- Alternatively, if the kdump kernel decides to capture the coredump,
  its attempts to read the faulty memory location may lead to subsequent 
  faults in the context of kdump kernel with fatal consequences. This
  may either be avoided by:

  a) Pass the address of the corrupt memory location to the kdump kernel
  and skip reading that location while creating the vmcore. This needs
  an instance of 'struct mce' (from the 'crashing' kernel), which
  already contains the faulty memory address (in the physical address
  form, which should be confirmed using the IA32_MCi_MISC[8:6] bits stored
  in 'misc' field of 'struct mce') to be populated inside the elf
  (-notes?) section.

  b) Use modified copy applications (such as a modified 'cp' command)
  that can map the /dev/oldmem into user-space and then initiate the
  creation of vmcore. In this method, the user-space process performing
  the copy will receive a SIGBUS while consuming the faulty memory (through
  INT18 -> do_machine_check) but it must be modified to be resilient to the
  signal, while intelligently skipping to the subsequent memory location
  for further copying. Meanwhile the data for the faulty memory location
  can be represented using 'zero-ed' data and the vmcore enhanced to
  indicate the cause of the crash as one resulting from a fatal MCE.

Any thoughts/suggestions?

Thanks,
K.Prasad



From tony.luck at intel.com  Wed May  4 16:02:25 2011
From: tony.luck at intel.com (Luck, Tony)
Date: Wed, 4 May 2011 13:02:25 -0700
Subject: [RFC] Kdump and memory error handling
In-Reply-To: <20110504193509.GA5342@in.ibm.com>
References: <20110504193509.GA5342@in.ibm.com>
Message-ID: <987664A83D2D224EAE907B061CE93D5301C53C9966@orsmsx505.amr.corp.intel.com>

Your first suggestion of a "slim" dump makes the most sense. The
purpose of a crash dump is a research resource to find out why
the system crashed - but in the case of a machine check, we already
have the reasons for the crash captured by the machine check handler.

Perhaps you could include __log_buf[] in the slim crash dump? Assuming
that the machine check is not a result of an uncorrectable error
in this memory range.

-Tony


From andi at firstfloor.org  Wed May  4 16:39:14 2011
From: andi at firstfloor.org (Andi Kleen)
Date: Wed, 4 May 2011 22:39:14 +0200
Subject: [RFC] Kdump and memory error handling
In-Reply-To: <20110504193509.GA5342@in.ibm.com>
References: <20110504193509.GA5342@in.ibm.com>
Message-ID: <20110504203914.GC1737@one.firstfloor.org>

> Any thoughts/suggestions?

My old attempts to solve this are

Don't dump on MCE:

http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/xpanic

Handle dumps of corrupted memory regresions:

http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/crashdump

IMHO these patches are still the right solutions for this.

-Andi


From vgoyal at redhat.com  Wed May  4 23:02:56 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Wed, 4 May 2011 23:02:56 -0400
Subject: [RFC] Kdump and memory error handling
In-Reply-To: <20110504203914.GC1737@one.firstfloor.org>
References: <20110504193509.GA5342@in.ibm.com>
	<20110504203914.GC1737@one.firstfloor.org>
Message-ID: <20110505030256.GA11823@redhat.com>

On Wed, May 04, 2011 at 10:39:14PM +0200, Andi Kleen wrote:
> > Any thoughts/suggestions?
> 
> My old attempts to solve this are
> 
> Don't dump on MCE:
> 
> http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/xpanic
> 
> Handle dumps of corrupted memory regresions:
> 
> http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/crashdump
> 

This idea of disabling mce temporarily sounds interesting. 

The slim dump giving access to log buffers makes most sense to me. Why
not leave it to user space to filter out only log buffers. So if a 
crash happens due to MCE, we can probably append an ELF note section
to vmcore and may be user space filtering utitliy (makedumpfile) can
extract and save only log portion of dump if it is an MCE triggered crash.

Of course this needs to be coupled with Andi's patch of disabling mce
temporarily so that makedumpfile does not induce another crash.

On a side note, can we just save log buf in NVRAM area and access later
using pstore (by tony luck) and if we can detect that system has that
NVRAM capability then skip kdump or something like that.

Thanks
Vivek


From vatsa at linux.vnet.ibm.com  Thu May  5 05:25:55 2011
From: vatsa at linux.vnet.ibm.com (Srivatsa Vaddagiri)
Date: Thu, 5 May 2011 14:55:55 +0530
Subject: [RFC] Kdump and memory error handling
In-Reply-To: <20110504203914.GC1737@one.firstfloor.org>
References: <20110504193509.GA5342@in.ibm.com>
	<20110504203914.GC1737@one.firstfloor.org>
Message-ID: <20110505092555.GA17414@linux.vnet.ibm.com>

On Wed, May 04, 2011 at 10:39:14PM +0200, Andi Kleen wrote:
> Handle dumps of corrupted memory regresions:
> 
> http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/crashdump

What happens when mce is disabled and capture kernel reads corrupted memory? 
Does that result in dump having erroneous data in dump?

- vatsa


From horms at verge.net.au  Thu May  5 08:20:03 2011
From: horms at verge.net.au (Simon Horman)
Date: Thu, 5 May 2011 21:20:03 +0900
Subject: [PATCH] detect Xen dom0 properly to avoid conflict with
	pv_on_hvm domU
In-Reply-To: <20110504163232.GA24669@aepfle.de>
References: <20110504163232.GA24669@aepfle.de>
Message-ID: <20110505122003.GM22010@verge.net.au>

On Wed, May 04, 2011 at 06:32:32PM +0200, Olaf Hering wrote:
> A Xen HVM guest with PV drivers loaded has also a /proc/xen directory.
> But such a guest is an ordinary PC and the special handling for dom0
> breaks kdump in this environment.
> Test for /proc/xen/capabilities instead and cache the result.
> 
> Also make two variables static, they are only used in this file.
> 
> Signed-off-by: Olaf Hering <olaf at aepfle.de>
> 
> ---
>  kexec/crashdump-xen.c |   15 ++++++++++-----
>  1 file changed, 10 insertions(+), 5 deletions(-)
> 
> Index: kexec-tools-0.8.15/kexec/crashdump-xen.c
> ===================================================================
> --- kexec-tools-0.8.15.orig/kexec/crashdump-xen.c
> +++ kexec-tools-0.8.15/kexec/crashdump-xen.c
> @@ -25,14 +25,19 @@ struct crash_note_info {
>  	unsigned long length;
>  };
>  
> -int xen_phys_cpus = 0;
> -struct crash_note_info *xen_phys_notes;
> +static int xen_phys_cpus;
> +static struct crash_note_info *xen_phys_notes;
> +static int is_dom0;

I think you need to explicitly initialise is_dom0 to 0.

Alternatively, is it really such a burden to call access()
each time xen_present() is called?  (I'm ambivalent with regards to this).

>  
>  int xen_present(void)
>  {
> -	struct stat buf;
> -
> -	return stat("/proc/xen", &buf) == 0;
> +	if (!is_dom0) {
> +		if (access("/proc/xen/capabilities", F_OK) == 0)
> +			is_dom0 = 1;
> +		else
> +			is_dom0 = -1;
> +	}
> +	return is_dom0 > 0;
>  }
>  
>  unsigned long xen_architecture(struct crash_elf_info *elf_info)
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
> 


From olaf at aepfle.de  Thu May  5 08:34:46 2011
From: olaf at aepfle.de (Olaf Hering)
Date: Thu, 5 May 2011 14:34:46 +0200
Subject: [PATCH] detect Xen dom0 properly to avoid conflict with
	pv_on_hvm domU
In-Reply-To: <20110505122003.GM22010@verge.net.au>
References: <20110504163232.GA24669@aepfle.de>
	<20110505122003.GM22010@verge.net.au>
Message-ID: <20110505123446.GA4214@aepfle.de>

On Thu, May 05, Simon Horman wrote:

> On Wed, May 04, 2011 at 06:32:32PM +0200, Olaf Hering wrote:

> > +static int is_dom0;
> 
> I think you need to explicitly initialise is_dom0 to 0.

Globals are always initialized to zero, its a C language thing.

> Alternatively, is it really such a burden to call access()
> each time xen_present() is called?  (I'm ambivalent with regards to this).

Probably not, its called a dozen times in my testing.

Olaf


From akpm at linux-foundation.org  Thu May  5 17:25:51 2011
From: akpm at linux-foundation.org (Andrew Morton)
Date: Thu, 5 May 2011 14:25:51 -0700
Subject: [PATCH] add hook to read_from_oldmem() to check for non-ram pages
In-Reply-To: <20110503190806.GA12485@aepfle.de>
References: <20110407095646.GA30788@aepfle.de>
	<20110503190806.GA12485@aepfle.de>
Message-ID: <20110505142551.b4d2d95a.akpm@linux-foundation.org>

On Tue, 3 May 2011 21:08:06 +0200
Olaf Hering <olaf at aepfle.de> wrote:

> 
> The balloon driver in a Xen guest frees guest pages and marks them as
> mmio. When the kernel crashes and the crash kernel attempts to read the
> oldmem via /proc/vmcore a read from ballooned pages will generate 100%
> load in dom0 because Xen asks qemu-dm for the page content. Since the
> reads come in as 8byte requests each ballooned page is tried 512 times.
> 
> With this change a hook can be registered which checks wether the given
> pfn is really ram. The hook has to return a value > 0 for ram pages, a
> value < 0 on error (because the hypercall is not known) and 0 for
> non-ram pages.
> 
> This will reduce the time to read /proc/vmcore.  Without this change a
> 512M guest with 128M crashkernel region needs 200 seconds to read it,
> with this change it takes just 2 seconds.

Seems reasonable, I suppose.

Is there some suitable ifdef we can put around this stuff to avoid
adding it to kernel builds which will never use it?

> ...
>
> --- linux-2.6.39-rc5.orig/fs/proc/vmcore.c
> +++ linux-2.6.39-rc5/fs/proc/vmcore.c
> @@ -18,6 +18,7 @@
>  #include <linux/init.h>
>  #include <linux/crash_dump.h>
>  #include <linux/list.h>
> +#include <linux/wait.h>
>  #include <asm/uaccess.h>
>  #include <asm/io.h>
>  
> @@ -35,6 +36,44 @@ static u64 vmcore_size;
>  
>  static struct proc_dir_entry *proc_vmcore = NULL;
>  
> +/* returns > 0 for RAM pages, 0 for non-RAM pages, < 0 on error */
> +static int (*oldmem_pfn_is_ram)(unsigned long pfn);
> +static DECLARE_WAIT_QUEUE_HEAD(oldmem_fn_waitq);
> +static atomic_t oldmem_fn_refcount = ATOMIC_INIT(0);
> +
> +void register_oldmem_pfn_is_ram(int (*fn)(unsigned long))
> +{
> +	if (oldmem_pfn_is_ram == NULL)
> +		oldmem_pfn_is_ram = fn;
> +}

This is racy, and it should return a success code.  And we may as well
mark it __must_check to prevent people from cheating.

> +void unregister_oldmem_pfn_is_ram(void)
> +{
> +	wait_event(oldmem_fn_waitq, atomic_read(&oldmem_fn_refcount) == 0);
> +	oldmem_pfn_is_ram = NULL;
> +	wmb();
> +}

I'd say we should do away with the (also racy) refcount thing. 
Instead, require that callers not be using the thing when they
unregister.  ie: that callers not be buggy.

> +static int pfn_is_ram(unsigned long pfn)
> +{
> +	int (*fn)(unsigned long);
> +	/* pfn is ram unless fn() checks pagetype */
> +	int ret = 1;
> +
> +	atomic_inc(&oldmem_fn_refcount);
> +	smp_mb__after_atomic_inc();
> +	fn = oldmem_pfn_is_ram;
> +	if (fn)
> +		ret = fn(pfn);
> +	if (atomic_dec_and_test(&oldmem_fn_refcount))
> +		wake_up(&oldmem_fn_waitq);
> +
> +	return ret;
> +}

This function would have been a suitable place at which to document the
entire feature.  As it stands, anyone who is reading this code won't
have any clue why it exists.

> +EXPORT_SYMBOL_GPL(register_oldmem_pfn_is_ram);
> +EXPORT_SYMBOL_GPL(unregister_oldmem_pfn_is_ram);

Each export should be placed immediately after the function which is
being exported, please.  Checkpatch reports this.  Please use checkpatch.


>  /* Reads a page from the oldmem device from given offset. */
>  static ssize_t read_from_oldmem(char *buf, size_t count,
>  				u64 *ppos, int userbuf)
> @@ -55,9 +94,14 @@ static ssize_t read_from_oldmem(char *bu
>  		else
>  			nr_bytes = count;
>  
> -		tmp = copy_oldmem_page(pfn, buf, nr_bytes, offset, userbuf);
> -		if (tmp < 0)
> -			return tmp;
> +		/* if pfn is not ram, return zeros for spares dump files */

typo.

Also, sentences start with capital letters!

> +		if (pfn_is_ram(pfn) == 0)
> +			memset(buf, 0, nr_bytes);
> +		else {
> +			tmp = copy_oldmem_page(pfn, buf, nr_bytes, offset, userbuf);
> +			if (tmp < 0)
> +				return tmp;
> +		}
>  		*ppos += nr_bytes;
>  		count -= nr_bytes;
>  		buf += nr_bytes;
> Index: linux-2.6.39-rc5/include/linux/crash_dump.h
> ===================================================================
> --- linux-2.6.39-rc5.orig/include/linux/crash_dump.h
> +++ linux-2.6.39-rc5/include/linux/crash_dump.h
> @@ -66,6 +66,11 @@ static inline void vmcore_unusable(void)
>  	if (is_kdump_kernel())
>  		elfcorehdr_addr = ELFCORE_ADDR_ERR;
>  }
> +
> +#define HAVE_OLDMEM_PFN_IS_RAM 1

What's this for?

> +extern void register_oldmem_pfn_is_ram(int (*fn)(unsigned long));

"unsigned long pfn" in the declaration, please.  This has good
documentation value.

> +extern void unregister_oldmem_pfn_is_ram(void);




From horms at verge.net.au  Thu May  5 23:29:23 2011
From: horms at verge.net.au (Simon Horman)
Date: Fri, 6 May 2011 12:29:23 +0900
Subject: [PATCH] detect Xen dom0 properly to avoid conflict with
	pv_on_hvm domU
In-Reply-To: <20110505123446.GA4214@aepfle.de>
References: <20110504163232.GA24669@aepfle.de>
	<20110505122003.GM22010@verge.net.au>
	<20110505123446.GA4214@aepfle.de>
Message-ID: <20110506032923.GA4079@verge.net.au>

On Thu, May 05, 2011 at 02:34:46PM +0200, Olaf Hering wrote:
> On Thu, May 05, Simon Horman wrote:
> 
> > On Wed, May 04, 2011 at 06:32:32PM +0200, Olaf Hering wrote:
> 
> > > +static int is_dom0;
> > 
> > I think you need to explicitly initialise is_dom0 to 0.
> 
> Globals are always initialized to zero, its a C language thing.

Point taken :^)

> > Alternatively, is it really such a burden to call access()
> > each time xen_present() is called?  (I'm ambivalent with regards to this).
> 
> Probably not, its called a dozen times in my testing.

As we already have the code it seems that we may as well use it.

I've merged your patch as-is.


From olaf at aepfle.de  Fri May  6 06:55:46 2011
From: olaf at aepfle.de (Olaf Hering)
Date: Fri, 6 May 2011 12:55:46 +0200
Subject: [PATCH] add hook to read_from_oldmem() to check for non-ram pages
In-Reply-To: <20110505142551.b4d2d95a.akpm@linux-foundation.org>
References: <20110407095646.GA30788@aepfle.de>
	<20110503190806.GA12485@aepfle.de>
	<20110505142551.b4d2d95a.akpm@linux-foundation.org>
Message-ID: <20110506105545.GA16945@aepfle.de>

On Thu, May 05, Andrew Morton wrote:

> On Tue, 3 May 2011 21:08:06 +0200
> Olaf Hering <olaf at aepfle.de> wrote:

> > This will reduce the time to read /proc/vmcore.  Without this change a
> > 512M guest with 128M crashkernel region needs 200 seconds to read it,
> > with this change it takes just 2 seconds.
> 
> Seems reasonable, I suppose.

Andrew,
Thanks for your feedback.

> Is there some suitable ifdef we can put around this stuff to avoid
> adding it to kernel builds which will never use it?

The change is for pv-on-hvm guests. In this setup the (out-of-tree)
paravirtualized drivers shutdown the emulated hardware, then they
communicate directly with the backend.
There is no ifdef right now.  I guess at some point, when xen is fully
merged, this hook can be put into some CONFIG_XEN_PV_ON_HVM thing.

> > +void register_oldmem_pfn_is_ram(int (*fn)(unsigned long))
> > +{
> > +	if (oldmem_pfn_is_ram == NULL)
> > +		oldmem_pfn_is_ram = fn;
> > +}
> 
> This is racy, and it should return a success code.  And we may as well
> mark it __must_check to prevent people from cheating.

I will update that part.

> > +void unregister_oldmem_pfn_is_ram(void)
> > +{
> > +	wait_event(oldmem_fn_waitq, atomic_read(&oldmem_fn_refcount) == 0);
> > +	oldmem_pfn_is_ram = NULL;
> > +	wmb();
> > +}
> 
> I'd say we should do away with the (also racy) refcount thing. 
> Instead, require that callers not be using the thing when they
> unregister.  ie: that callers not be buggy.

I think oldmem_pfn_is_ram can be cleared unconditionally, the NULL check
in pfn_is_ram() below will prevent a crash.
This whole refcount thing is to prevent a module unload while
pfn_is_ram() is calling the hook, in other words the called code should
not go away until the hook returns to pfn_is_ram().
Should the called code increase/decrease the modules refcount instead?
I remember there was some MODULE_INC/MODULE_DEC macro (cant remember the
exact name) at some point. What needs to be done inside the module to
prevent an unload while its still in use? Is it __module_get/module_put
for each call of fn()? 

The called function which will go into the xen source at some point
looks like shown below. HVMOP_get_mem_type was just merged into xen-unstable.

xen-unstable.hg/unmodified_drivers/linux-2.6/platform-pci/platform-pci.c

#ifdef HAVE_OLDMEM_PFN_IS_RAM
static int xen_oldmem_pfn_is_ram(unsigned long pfn)
{
	struct xen_hvm_get_mem_type a;
	int ret;

	a.domid = DOMID_SELF;
	a.pfn = pfn;
	if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a))
		return -ENXIO;

	switch (a.mem_type) {
		case HVMMEM_mmio_dm:
			ret = 0;
			break;
		case HVMMEM_ram_rw:
		case HVMMEM_ram_ro:
		default:
			ret = 1;
			break;
	}

	return ret;
}
#endif

static int __devinit platform_pci_init(...)
{
        /* other init stuff */
#ifdef HAVE_OLDMEM_PFN_IS_RAM
        register_oldmem_pfn_is_ram(&xen_oldmem_pfn_is_ram);
#endif
        /* other init stuff */
}


Also, this xen driver has no module_exit. So for xen the unregister is
not an issue. I havent looked at the to-be-merged pv-on-hvm drivers,
maybe they can be properly unloaded.

> > +static int pfn_is_ram(unsigned long pfn)
> > +{
> > +	int (*fn)(unsigned long);
> > +	/* pfn is ram unless fn() checks pagetype */
> > +	int ret = 1;
> > +
> > +	atomic_inc(&oldmem_fn_refcount);
> > +	smp_mb__after_atomic_inc();
> > +	fn = oldmem_pfn_is_ram;
> > +	if (fn)
> > +		ret = fn(pfn);
> > +	if (atomic_dec_and_test(&oldmem_fn_refcount))
> > +		wake_up(&oldmem_fn_waitq);
> > +
> > +	return ret;
> > +}
> 
> This function would have been a suitable place at which to document the
> entire feature.  As it stands, anyone who is reading this code won't
> have any clue why it exists.

I will add a comment.

> > +EXPORT_SYMBOL_GPL(register_oldmem_pfn_is_ram);
> > +EXPORT_SYMBOL_GPL(unregister_oldmem_pfn_is_ram);
> 
> Each export should be placed immediately after the function which is
> being exported, please.  Checkpatch reports this.  Please use checkpatch.

Will do.

> > +++ linux-2.6.39-rc5/include/linux/crash_dump.h
> > @@ -66,6 +66,11 @@ static inline void vmcore_unusable(void)
> >  	if (is_kdump_kernel())
> >  		elfcorehdr_addr = ELFCORE_ADDR_ERR;
> >  }
> > +
> > +#define HAVE_OLDMEM_PFN_IS_RAM 1
> 
> What's this for?

So that out-of-tree drivers dont fail to compile when they call that
hook unconditionally. Perhaps they could use the kernel version.

Olaf


From olaf at aepfle.de  Fri May  6 09:20:24 2011
From: olaf at aepfle.de (Olaf Hering)
Date: Fri, 6 May 2011 15:20:24 +0200
Subject: [PATCH v2] add hook to read_from_oldmem() to check for non-ram pages
In-Reply-To: <20110503190806.GA12485@aepfle.de>
References: <20110407095646.GA30788@aepfle.de>
	<20110503190806.GA12485@aepfle.de>
Message-ID: <20110506132024.GA19233@aepfle.de>


The balloon driver in a Xen guest frees guest pages and marks them as
mmio. When the kernel crashes and the crash kernel attempts to read the
oldmem via /proc/vmcore a read from ballooned pages will generate 100%
load in dom0 because Xen asks qemu-dm for the page content. Since the
reads come in as 8byte requests each ballooned page is tried 512 times.

With this change a hook can be registered which checks wether the given
pfn is really ram. The hook has to return a value > 0 for ram pages, a
value < 0 on error (because the hypercall is not known) and 0 for
non-ram pages.

This will reduce the time to read /proc/vmcore.  Without this change a
512M guest with 128M crashkernel region needs 200 seconds to read it,
with this change it takes just 2 seconds.


Signed-off-by: Olaf Hering <olaf at aepfle.de>

---
v2:
 remove refcounting, called function has to take care of module refcounting
 add comments to pfn_is_ram()
 register_oldmem_pfn_is_ram() returns -EBUSY if a function is already registerd
 move exports close to exported function
 update prototypes to include 'pfn'

 fs/proc/vmcore.c           |   52 ++++++++++++++++++++++++++++++++++++++++++---
 include/linux/crash_dump.h |    5 ++++
 2 files changed, 54 insertions(+), 3 deletions(-)

Index: linux-2.6.39-rc6/fs/proc/vmcore.c
===================================================================
--- linux-2.6.39-rc6.orig/fs/proc/vmcore.c
+++ linux-2.6.39-rc6/fs/proc/vmcore.c
@@ -35,6 +35,46 @@ static u64 vmcore_size;
 
 static struct proc_dir_entry *proc_vmcore = NULL;
 
+/*
+ * Returns > 0 for RAM pages, 0 for non-RAM pages, < 0 on error
+ * The called function has to take care of module refcounting.
+ */
+static int (*oldmem_pfn_is_ram)(unsigned long pfn);
+
+int register_oldmem_pfn_is_ram(int (*fn)(unsigned long pfn))
+{
+	if (oldmem_pfn_is_ram)
+		return -EBUSY;
+	oldmem_pfn_is_ram = fn;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(register_oldmem_pfn_is_ram);
+
+void unregister_oldmem_pfn_is_ram(void)
+{
+	oldmem_pfn_is_ram = NULL;
+	wmb();
+}
+EXPORT_SYMBOL_GPL(unregister_oldmem_pfn_is_ram);
+
+static int pfn_is_ram(unsigned long pfn)
+{
+	int (*fn)(unsigned long pfn);
+	/* pfn is ram unless fn() checks pagetype */
+	int ret = 1;
+
+	/*
+	 * Ask hypervisor if the pfn is really ram.
+	 * A ballooned page contains no data and reading from such a page
+	 * will cause high load in the hypervisor.
+	 */
+	fn = oldmem_pfn_is_ram;
+	if (fn)
+		ret = fn(pfn);
+
+	return ret;
+}
+
 /* Reads a page from the oldmem device from given offset. */
 static ssize_t read_from_oldmem(char *buf, size_t count,
 				u64 *ppos, int userbuf)
@@ -55,9 +95,15 @@ static ssize_t read_from_oldmem(char *bu
 		else
 			nr_bytes = count;
 
-		tmp = copy_oldmem_page(pfn, buf, nr_bytes, offset, userbuf);
-		if (tmp < 0)
-			return tmp;
+		/* If pfn is not ram, return zeros for sparse dump files */
+		if (pfn_is_ram(pfn) == 0)
+			memset(buf, 0, nr_bytes);
+		else {
+			tmp = copy_oldmem_page(pfn, buf, nr_bytes,
+						offset, userbuf);
+			if (tmp < 0)
+				return tmp;
+		}
 		*ppos += nr_bytes;
 		count -= nr_bytes;
 		buf += nr_bytes;
Index: linux-2.6.39-rc6/include/linux/crash_dump.h
===================================================================
--- linux-2.6.39-rc6.orig/include/linux/crash_dump.h
+++ linux-2.6.39-rc6/include/linux/crash_dump.h
@@ -66,6 +66,11 @@ static inline void vmcore_unusable(void)
 	if (is_kdump_kernel())
 		elfcorehdr_addr = ELFCORE_ADDR_ERR;
 }
+
+#define HAVE_OLDMEM_PFN_IS_RAM 1
+extern int register_oldmem_pfn_is_ram(int (*fn)(unsigned long pfn));
+extern void unregister_oldmem_pfn_is_ram(void);
+
 #else /* !CONFIG_CRASH_DUMP */
 static inline int is_kdump_kernel(void) { return 0; }
 #endif /* CONFIG_CRASH_DUMP */


From adrian.wenl at gmail.com  Fri May  6 10:33:45 2011
From: adrian.wenl at gmail.com (Lei Wen)
Date: Fri, 6 May 2011 22:33:45 +0800
Subject: kdump without the kexec
Message-ID: <BANLkTi=Jt=Q9Q9kM4ogtpW0PN7k=WSLtTQ@mail.gmail.com>

Hi,

Is there any existed solution that could make the kdump without the kexec?
For some kind of system hang, always hardware bug, or software deadlock, which
could not trigger the kernel oops, so that the first kernel cannot
load the second kernel
with kexec tool. The only way here is to directly press the reset key.

We could be assure that the ddr memory would not be corrupted by this
way of reset
(for we don't shutdown the ddr power during the reset). So my question
is that whether
we could take use of bootloader, like the u-boot, to pretend we are
loading the second kernel
after the reset behavior and then perform the kdump process?

Best regards,
Lei


From prasad at linux.vnet.ibm.com  Fri May  6 12:54:12 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Fri, 6 May 2011 22:24:12 +0530
Subject: [Bug] Kdump does not work when panic triggered due to MCE
Message-ID: <20110506165412.GB2719@in.ibm.com>

Hi All,
	I wanted to test the behaviour of kdump when panic is triggered
due to MCE on x86 and found that kdump is not captured.

While the kdump service is configured and running and non-MCE panics
(such as those triggered through to /proc/sysrq-trigger) successfully
capture a kdump, any fatal MCE error injected through the mce-inject
tool causes a reboot of the machine.

The code has been traced (using early_serial_putc()) to enter the kexec
path i.e. panic()->crash_kexec()->machine_kexec()->relocate_kernel()
but is untraceable further.

Kdump works fine when the same the similar test is carried out inside a
KVM guest.

Has anybody tested this before? Or have found kdump working when fatal
MCEs have actually occurred?

Thanks,
K.Prasad

Relevant Screen logs
---------------------

login: root
Password: 
Last login: Fri May  6 11:16:52 from 9.77.122.190

# uname -a
Linux elm3a97.beaverton.ibm.com 2.6.39-rc6.prasad_kdump+ #1 SMP Fri May 6 07:47:31 EDT 2011 i686 i686 i386 GNU/Linux
# lsmod | grep mce
mce_inject              2355  0 [permanent]
# service kdump status
Kdump is operational
# mce-inject /home/prasadkr/mce/mce-test/cases/soft-inj/panic_ucr/data/srar_over
Triggering MCE exception on CPU 0
Disabling lock debugging due to kernel taint
[Hardware Error]: CPU 0: Machine Check Exception: 6 Bank 2: f580000000000000
[Hardware Error]: RIP 73:<000000001eadbabe> 
[Hardware Error]: TSC 21dde8717030 ADDR 1234 
[Hardware Error]: PROCESSOR 0:106a5 TIME 1304696989 SOCKET 0 APIC 0
[Hardware Error]: No human readable MCE decoding support on this CPU type.
[Hardware Error]: Run the message through 'mcelog --ascii' to decode.
[Hardware Error]: Machine check: Overflowed uncorrected
Kernel panic - not syncing: Fatal Machine check
Pid: 0, comm: kworker/0:0 Tainted: G   M    W   2.6.39-rc6.prasad_kdump+ #1
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/traps.c:436!
invalid opcode: 0000 [#1] SMP 
last sysfs file: /sys/devices/system/cpu/cpu15/cache/index2/shared_cpu_map
Modules linked in: mce_inject autofs4 cpufreq_ondemand acpi_cpufreq mperf ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log dm_mod cdc_ether usbnet mii microcode sg i2c_i801 serio_raw pcspkr iTCO_wdt iTCO_vendor_support bnx2x libcrc32c mdio ioatdma dca i7core_edac edac_core bnx2 ext4 jbd2 sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]

Pid: 0, comm: kworker/0:1 Tainted: G   M    W   2.6.39-rc6.prasad_kdump+ #1 IBM IBM System x -[7839AC1]-/46C7890     
EIP: 0060:[<c0860da9>] EFLAGS: 00010006 CPU: 12
EIP is at do_nmi+0x89/0xa0
EAX: e9ba9c9c EBX: e9ba9c9c ECX: 04010000 EDX: e9ba8000
ESI: 0000000c EDI: 00000af0 EBP: e9ba9c94 ESP: e9ba9c90
 DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Process kworker/0:1 (pid: 0, ti=e9ba8000 task=e9ba70d0 task.ti=e9ba8000)
Stack:
 0000000c e9ba9ce8 c0860678 0000000c 00000010 000021dd 0000000c 00000af0
 e9ba9ce8 f0eca855 c0aa007b 0000007b e9ba00d8 c08600e0 f0eca855 c06048fd
 00000060 00000246 f0eca5dd 004b9910 0000000c 0000000c e9ba9cf0 c06048be
Call Trace:
 [<c0860678>] nmi_stack_correct+0x2f/0x34
 [<c08600e0>] ? invalidate_interrupt23+0x3c/0x3c
 [<c06048fd>] ? delay_tsc+0x3d/0x70
 [<c06048be>] __const_udelay+0x1e/0x20
 [<c041c9b5>] wait_for_panic+0x25/0x50
 [<c041cd78>] mce_timed_out+0x48/0x90
 [<c041cf39>] mce_end+0x59/0x100
 [<c041db2b>] do_machine_check+0x3db/0x6a0
 [<c0476600>] ? __hrtimer_start_range_ns+0xa0/0x470
 [<f7e15064>] raise_exception+0x34/0xa0 [mce_inject]
 [<c040f978>] ? sched_clock+0x8/0x10
 [<c0478735>] ? sched_clock_cpu+0x145/0x190
 [<c0489cc0>] ? __lock_acquire+0x2c0/0x490
 [<f7e151a1>] mce_raise_notify+0x61/0x70 [mce_inject]
 [<c0863393>] notifier_call_chain+0x43/0x60
 [<c086340b>] __atomic_notifier_call_chain+0x5b/0x80
 [<c08633b0>] ? notifier_call_chain+0x60/0x60
 [<c086344a>] atomic_notifier_call_chain+0x1a/0x20
 [<c086347d>] notify_die+0x2d/0x30
 [<c0860ac2>] default_do_nmi+0x32/0x290
 [<c048a312>] ? __lock_release+0x72/0x180
 [<c04809ea>] ? clockevents_notify+0x3a/0xf0
 [<c0860da7>] do_nmi+0x87/0xa0
 [<c0860678>] nmi_stack_correct+0x2f/0x34
 [<c048007b>] ? leaps_between+0x3b/0x90
 [<c065521c>] ? intel_idle+0x8c/0x100
 [<c077a32d>] cpuidle_idle_call+0x8d/0x210
 [<c0409b0b>] cpu_idle+0x9b/0xd0
 [<c0858714>] start_secondary+0xdd/0xe3
Code: 5e 2f c2 ff 89 e0 25 00 e0 ff ff 8b 50 14 f7 c2 00 00 00 04 74 1e 81 ea 00 00 01 04 89 50 14 5b 5d c3 89 d8 e8 e9 fc ff ff eb c2 <0f> 0b 90 8d 74 26 00 eb f9 0f 0b eb fe 8d 76 00 8d bc 27 00 00 
EIP: [<c0860da9>] do_nmi+0x89/0xa0 SS:ESP 0068:e9ba9c90
Call Trace:
 [<c085c638>] panic+0x57/0x165
 [<c041cd10>] mce_panic+0x1c0/0x1e0
 [<c041ced0>] mce_reign+0x110/0x120
 [<c041cfca>] mce_end+0xea/0x100
 [<c041db2b>] do_machine_check+0x3db/0x6a0
 [<c0476600>] ? __hrtimer_start_range_ns+0xa0/0x470
 [<f7e15064>] raise_exception+0x34/0xa0 [mce_inject]
 [<c040f978>] ? sched_clock+0x8/0x10
 [<c0478735>] ? sched_clock_cpu+0x145/0x190
 [<c0489cc0>] ? __lock_acquire+0x2c0/0x490
 [<f7e151a1>] mce_raise_notify+0x61/0x70 [mce_inject]
 [<c0863393>] nx30
 [<c0860ac2>] default_do_nmi+0x32/0x290
 [<c048a312>] ? __lock_release+0x72/0x180
 [<c04809ea>] ? clockevents_notify+0x3a/0xf0
 [<c0860da7>] do_nmi+0x87/0xa0
 [<c0860678>] nmi_stack_correct+0x2f/0x34
 [<c048007b>] ? leaps_between+0x3b/0x90
 [<c065521c>] ? intel_idle+0x8c/0x100
 [<c077a32d>] cpuidle_idle_call+0x8d/0x210
 [<c0409b0b>] cpu_idle+0x9b/0xd0
 [<c0858714>] start_secondary+0xdd/0xe3
Rebooting in 1 seconds..



From andi at firstfloor.org  Fri May  6 13:38:25 2011
From: andi at firstfloor.org (Andi Kleen)
Date: Fri, 6 May 2011 19:38:25 +0200
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110506165412.GB2719@in.ibm.com>
References: <20110506165412.GB2719@in.ibm.com>
Message-ID: <20110506173825.GK11636@one.firstfloor.org>

> Has anybody tested this before? Or have found kdump working when fatal
> MCEs have actually occurred?

Ying did some testing. mce-test has test cases for kdump.

My guess is you injected the error into some area used by the kexec
code or boot up path of the kexec kernel.

-Andi


From akpm at linux-foundation.org  Fri May  6 15:30:19 2011
From: akpm at linux-foundation.org (Andrew Morton)
Date: Fri, 6 May 2011 12:30:19 -0700
Subject: [PATCH] add hook to read_from_oldmem() to check for non-ram pages
In-Reply-To: <20110506105545.GA16945@aepfle.de>
References: <20110407095646.GA30788@aepfle.de>
	<20110503190806.GA12485@aepfle.de>
	<20110505142551.b4d2d95a.akpm@linux-foundation.org>
	<20110506105545.GA16945@aepfle.de>
Message-ID: <20110506123019.80461d5d.akpm@linux-foundation.org>

On Fri, 6 May 2011 12:55:46 +0200
Olaf Hering <olaf at aepfle.de> wrote:

> Should the called code increase/decrease the modules refcount instead?
> I remember there was some MODULE_INC/MODULE_DEC macro (cant remember the
> exact name) at some point. What needs to be done inside the module to
> prevent an unload while its still in use? Is it __module_get/module_put
> for each call of fn()? 

A try_module_get(THIS_MODULE) in the register function will do the trick.
However it's unneeded.  Documentation/DocBook/kernel-hacking.tmpl tells us

try_module_get() module_put()

    These manipulate the module usage count, to protect against
    removal (a module also can't be removed if another module uses one
    of its exported symbols: see below).  Before calling into module
    code, you should call <function>try_module_get()</function> on
    that module: if it fails, then the module is being removed and you
    should act as if it wasn't there.  Otherwise, you can safely enter
    the module, and call <function>module_put()</function> when you're
    finished.

So as your module will have a reference to vmcore.c's register and unregister
functions, nothing needs to be done: the presence of the client module alone
will pin the vmcore.c module.

However it's all moot, because the fs/proc/vmcore.c code cannot
presently be built as a module and it's rather unlikely that it ever
will be.



From olaf at aepfle.de  Fri May  6 15:39:16 2011
From: olaf at aepfle.de (Olaf Hering)
Date: Fri, 6 May 2011 21:39:16 +0200
Subject: [PATCH] add hook to read_from_oldmem() to check for non-ram pages
In-Reply-To: <20110506123019.80461d5d.akpm@linux-foundation.org>
References: <20110407095646.GA30788@aepfle.de>
	<20110503190806.GA12485@aepfle.de>
	<20110505142551.b4d2d95a.akpm@linux-foundation.org>
	<20110506105545.GA16945@aepfle.de>
	<20110506123019.80461d5d.akpm@linux-foundation.org>
Message-ID: <20110506193916.GA26129@aepfle.de>

On Fri, May 06, Andrew Morton wrote:

> So as your module will have a reference to vmcore.c's register and unregister
> functions, nothing needs to be done: the presence of the client module alone
> will pin the vmcore.c module.

I meant the other way around. Keep /proc/vmcore open and read from it,
then try to rmmod foo.ko which provides fn().

Olaf


From akpm at linux-foundation.org  Fri May  6 15:55:46 2011
From: akpm at linux-foundation.org (Andrew Morton)
Date: Fri, 6 May 2011 12:55:46 -0700
Subject: [PATCH] add hook to read_from_oldmem() to check for non-ram pages
In-Reply-To: <20110506193916.GA26129@aepfle.de>
References: <20110407095646.GA30788@aepfle.de>
	<20110503190806.GA12485@aepfle.de>
	<20110505142551.b4d2d95a.akpm@linux-foundation.org>
	<20110506105545.GA16945@aepfle.de>
	<20110506123019.80461d5d.akpm@linux-foundation.org>
	<20110506193916.GA26129@aepfle.de>
Message-ID: <20110506125546.72d138ed.akpm@linux-foundation.org>

On Fri, 6 May 2011 21:39:16 +0200
Olaf Hering <olaf at aepfle.de> wrote:

> On Fri, May 06, Andrew Morton wrote:
> 
> > So as your module will have a reference to vmcore.c's register and unregister
> > functions, nothing needs to be done: the presence of the client module alone
> > will pin the vmcore.c module.
> 
> I meant the other way around. Keep /proc/vmcore open and read from it,
> then try to rmmod foo.ko which provides fn().
> 

The client foo.ko will need to prevent itself from being unloaded while
it's actively doing stuff, yes.  Typically that would be done in its
module_exit() function - wait for current activity to complete and
block new activity.  The "block new activity" thing should be automatic
because nobody has any more references to anything in the module.



From bernhard at bwalle.de  Sat May  7 03:55:00 2011
From: bernhard at bwalle.de (Bernhard Walle)
Date: Sat, 7 May 2011 09:55:00 +0200
Subject: kdump without the kexec
In-Reply-To: <BANLkTi=Jt=Q9Q9kM4ogtpW0PN7k=WSLtTQ@mail.gmail.com>
References: <BANLkTi=Jt=Q9Q9kM4ogtpW0PN7k=WSLtTQ@mail.gmail.com>
Message-ID: <20110507075500.GB29941@regiomontanus.bwalle.de>

* Lei Wen <adrian.wenl at gmail.com> [2011-05-06 16:33]:
> 
> Is there any existed solution that could make the kdump without the kexec?
> For some kind of system hang, always hardware bug, or software deadlock, which
> could not trigger the kernel oops, so that the first kernel cannot
> load the second kernel
> with kexec tool. The only way here is to directly press the reset key.
> 
> We could be assure that the ddr memory would not be corrupted by this
> way of reset
> (for we don't shutdown the ddr power during the reset). So my question
> is that whether
> we could take use of bootloader, like the u-boot, to pretend we are
> loading the second kernel
> after the reset behavior and then perform the kdump process?

Did you try sysrq-c? Which platform? Most of them have the concept of a
NMI that also can trigger kdump.


Regards,
Bernhard


From adrian.wenl at gmail.com  Sun May  8 11:31:30 2011
From: adrian.wenl at gmail.com (Lei Wen)
Date: Sun, 8 May 2011 23:31:30 +0800
Subject: kdump without the kexec
In-Reply-To: <20110507075500.GB29941@regiomontanus.bwalle.de>
References: <BANLkTi=Jt=Q9Q9kM4ogtpW0PN7k=WSLtTQ@mail.gmail.com>
	<20110507075500.GB29941@regiomontanus.bwalle.de>
Message-ID: <BANLkTikgb4f-OTj1zdw9OkOWb-OZpwJS7g@mail.gmail.com>

Hi Bernhard,

On Sat, May 7, 2011 at 3:55 PM, Bernhard Walle <bernhard at bwalle.de> wrote:
> * Lei Wen <adrian.wenl at gmail.com> [2011-05-06 16:33]:
>>
>> Is there any existed solution that could make the kdump without the kexec?
>> For some kind of system hang, always hardware bug, or software deadlock, which
>> could not trigger the kernel oops, so that the first kernel cannot
>> load the second kernel
>> with kexec tool. The only way here is to directly press the reset key.
>>
>> We could be assure that the ddr memory would not be corrupted by this
>> way of reset
>> (for we don't shutdown the ddr power during the reset). So my question
>> is that whether
>> we could take use of bootloader, like the u-boot, to pretend we are
>> loading the second kernel
>> after the reset behavior and then perform the kdump process?
>
> Did you try sysrq-c? Which platform? Most of them have the concept of a
> NMI that also can trigger kdump.
>
>
For some case, the sysrq-c also cannot help, as the cpu is not
responding anymore for some
hw bug. I am running on arm board, there is no NMI concept on it...
So the only help is to put the hw reset and wish the bootloader do the
task which is expected
done by the kexec...

Best regards,
Lei


From xiyou.wangcong at gmail.com  Mon May  9 02:42:37 2011
From: xiyou.wangcong at gmail.com (WANG Cong)
Date: Mon, 9 May 2011 06:42:37 +0000 (UTC)
Subject: kdump without the kexec
References: <BANLkTi=Jt=Q9Q9kM4ogtpW0PN7k=WSLtTQ@mail.gmail.com>
	<20110507075500.GB29941@regiomontanus.bwalle.de>
	<BANLkTikgb4f-OTj1zdw9OkOWb-OZpwJS7g@mail.gmail.com>
Message-ID: <iq82ct$bs6$1@dough.gmane.org>

On Sun, 08 May 2011 23:31:30 +0800, Lei Wen wrote:

> Hi Bernhard,
> 
> On Sat, May 7, 2011 at 3:55 PM, Bernhard Walle
> <bernhard at bwalle.de> wrote:
>> * Lei Wen <adrian.wenl at gmail.com>
>> [2011-05-06 16:33]:
>>>
>>> Is there any existed solution that could make the kdump without the
>>> kexec? For some kind of system hang, always hardware bug, or software
>>> deadlock, which could not trigger the kernel oops, so that the first
>>> kernel cannot load the second kernel
>>> with kexec tool. The only way here is to directly press the reset key.
>>>
>>> We could be assure that the ddr memory would not be corrupted by this
>>> way of reset
>>> (for we don't shutdown the ddr power during the reset). So my question
>>> is that whether
>>> we could take use of bootloader, like the u-boot, to pretend we are
>>> loading the second kernel
>>> after the reset behavior and then perform the kdump process?
>>
>> Did you try sysrq-c? Which platform? Most of them have the concept of a
>> NMI that also can trigger kdump.
>>
>>
> For some case, the sysrq-c also cannot help, as the cpu is not
> responding anymore for some
> hw bug. I am running on arm board, there is no NMI concept on it... So
> the only help is to put the hw reset and wish the bootloader do the task
> which is expected
> done by the kexec...

Then kdump can't help in this case.

And I am afraid you can't use the normal bootloader (e.g. grub, uboot), 
because we load the second kernel *while* running the first kernel, which 
means, for example, on x86 we are in protect mode after the kernel boots, 
not in the real mode in which grub starts.

What's more, kdump will not hw-reset the devices unless you add 
"reset_devices" to the second kernel.



From adrian.wenl at gmail.com  Mon May  9 03:04:42 2011
From: adrian.wenl at gmail.com (Lei Wen)
Date: Mon, 9 May 2011 15:04:42 +0800
Subject: kdump without the kexec
In-Reply-To: <iq82ct$bs6$1@dough.gmane.org>
References: <BANLkTi=Jt=Q9Q9kM4ogtpW0PN7k=WSLtTQ@mail.gmail.com>
	<20110507075500.GB29941@regiomontanus.bwalle.de>
	<BANLkTikgb4f-OTj1zdw9OkOWb-OZpwJS7g@mail.gmail.com>
	<iq82ct$bs6$1@dough.gmane.org>
Message-ID: <BANLkTin=0D6dZANXtKKDR=JMFaB5wHzr2A@mail.gmail.com>

Hi Cong,

On Mon, May 9, 2011 at 2:42 PM, WANG Cong <xiyou.wangcong at gmail.com> wrote:
> On Sun, 08 May 2011 23:31:30 +0800, Lei Wen wrote:
>
>> Hi Bernhard,
>>
>> On Sat, May 7, 2011 at 3:55 PM, Bernhard Walle
>> <bernhard at bwalle.de> wrote:
>>> * Lei Wen <adrian.wenl at gmail.com>
>>> [2011-05-06 16:33]:
>>>>
>>>> Is there any existed solution that could make the kdump without the
>>>> kexec? For some kind of system hang, always hardware bug, or software
>>>> deadlock, which could not trigger the kernel oops, so that the first
>>>> kernel cannot load the second kernel
>>>> with kexec tool. The only way here is to directly press the reset key.
>>>>
>>>> We could be assure that the ddr memory would not be corrupted by this
>>>> way of reset
>>>> (for we don't shutdown the ddr power during the reset). So my question
>>>> is that whether
>>>> we could take use of bootloader, like the u-boot, to pretend we are
>>>> loading the second kernel
>>>> after the reset behavior and then perform the kdump process?
>>>
>>> Did you try sysrq-c? Which platform? Most of them have the concept of a
>>> NMI that also can trigger kdump.
>>>
>>>
>> For some case, the sysrq-c also cannot help, as the cpu is not
>> responding anymore for some
>> hw bug. I am running on arm board, there is no NMI concept on it... So
>> the only help is to put the hw reset and wish the bootloader do the task
>> which is expected
>> done by the kexec...
>
> Then kdump can't help in this case.
>
> And I am afraid you can't use the normal bootloader (e.g. grub, uboot),
> because we load the second kernel *while* running the first kernel, which
> means, for example, on x86 we are in protect mode after the kernel boots,
> not in the real mode in which grub starts.
>
> What's more, kdump will not hw-reset the devices unless you add
> "reset_devices" to the second kernel.
>
I see...
But what I am care is the memory dump done by the kdump. If I could
boot the second kernel without break the first kernel's space, that by put the
second kernel just in the space that has been reserved by the first kernel.
Is that possible to use the bootloader to perform such task?

The devices state is not really cared by us, for the only interesting
point is the
memory context left by the first kernel.

Thanks,
Lei


From vgoyal at redhat.com  Mon May  9 08:39:02 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 9 May 2011 08:39:02 -0400
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110506165412.GB2719@in.ibm.com>
References: <20110506165412.GB2719@in.ibm.com>
Message-ID: <20110509123902.GA5975@redhat.com>

On Fri, May 06, 2011 at 10:24:12PM +0530, K.Prasad wrote:
> Hi All,
> 	I wanted to test the behaviour of kdump when panic is triggered
> due to MCE on x86 and found that kdump is not captured.
> 
> While the kdump service is configured and running and non-MCE panics
> (such as those triggered through to /proc/sysrq-trigger) successfully
> capture a kdump, any fatal MCE error injected through the mce-inject
> tool causes a reboot of the machine.
> 
> The code has been traced (using early_serial_putc()) to enter the kexec
> path i.e. panic()->crash_kexec()->machine_kexec()->relocate_kernel()
> but is untraceable further.
> 
> Kdump works fine when the same the similar test is carried out inside a
> KVM guest.
> 
> Has anybody tested this before? Or have found kdump working when fatal
> MCEs have actually occurred?

Prasad,

I have never tried taking dump in MCE situation. Does kdump work on this
machine with normal panic()?

Use --debug and --serial option in kexec-tools to print some debug message
and look for "I am in purgatory". This will tell you whether you hanged
in first kernel or second kernel.

Then put "outb()" messages in the kernel to trace what happened. 

Thanks
Vivek


From louis.bouchard at canonical.com  Mon May  9 11:21:06 2011
From: louis.bouchard at canonical.com (Bouchard Louis)
Date: Mon, 09 May 2011 17:21:06 +0200
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110509123902.GA5975@redhat.com>
References: <20110506165412.GB2719@in.ibm.com>
	<20110509123902.GA5975@redhat.com>
Message-ID: <4DC80662.9000703@canonical.com>

Hello,

Le 09/05/2011 14:39, Vivek Goyal a ?crit :
>
> Prasad,
>
> I have never tried taking dump in MCE situation. Does kdump work on this
> machine with normal panic()?
>
> Use --debug and --serial option in kexec-tools to print some debug message
> and look for "I am in purgatory". This will tell you whether you hanged
> in first kernel or second kernel.
>
> Then put "outb()" messages in the kernel to trace what happened. 
>
> Thanks
> Vivek
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
I have seen numerous occurrences of MCE triggered kernel panics on both
RHEL & SLES environment used on IA32 architecture. Both in contexts
where kexec/kdump was being used.

 Matter of fact, MCE triggered panic are part of the reason that pushed
me to work on crashdc : only one crash command is required to get the
MCE trace out of the kernel ring buffer. This avoids transfering massive
amount of vmcore file over the net.

crashdc does well on those, mcelog can be applied on the data gathered.

HTH,

...Louis

-- 
Louis Bouchard
Server Support Analyst
Canonical Ltd
Ubuntu support: http://landscape.canonical.com



From vgoyal at redhat.com  Mon May  9 11:46:06 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 9 May 2011 11:46:06 -0400
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <4DC80662.9000703@canonical.com>
References: <20110506165412.GB2719@in.ibm.com>
	<20110509123902.GA5975@redhat.com> <4DC80662.9000703@canonical.com>
Message-ID: <20110509154606.GE5975@redhat.com>

On Mon, May 09, 2011 at 05:21:06PM +0200, Bouchard Louis wrote:
> Hello,
> 
> Le 09/05/2011 14:39, Vivek Goyal a ?crit :
> >
> > Prasad,
> >
> > I have never tried taking dump in MCE situation. Does kdump work on this
> > machine with normal panic()?
> >
> > Use --debug and --serial option in kexec-tools to print some debug message
> > and look for "I am in purgatory". This will tell you whether you hanged
> > in first kernel or second kernel.
> >
> > Then put "outb()" messages in the kernel to trace what happened. 
> >
> > Thanks
> > Vivek
> >
> > _______________________________________________
> > kexec mailing list
> > kexec at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kexec
> I have seen numerous occurrences of MCE triggered kernel panics on both
> RHEL & SLES environment used on IA32 architecture. Both in contexts
> where kexec/kdump was being used.
> 
>  Matter of fact, MCE triggered panic are part of the reason that pushed
> me to work on crashdc : only one crash command is required to get the
> MCE trace out of the kernel ring buffer. This avoids transfering massive
> amount of vmcore file over the net.
> 
> crashdc does well on those, mcelog can be applied on the data gathered.
> 

Louis, is this "crashdc" an independent executable. Where is it packaged?
What is the significance of name "crashdc" (Crash data collecor ?) Will it
make sense to merge it with makedumpfile which does kernel filtering
already and this could be one of the additional features.

Thanks
Vivek

> HTH,
> 
> ...Louis
> 
> -- 
> Louis Bouchard
> Server Support Analyst
> Canonical Ltd
> Ubuntu support: http://landscape.canonical.com


From prasad at linux.vnet.ibm.com  Mon May  9 12:35:40 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Mon, 9 May 2011 22:05:40 +0530
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110506173825.GK11636@one.firstfloor.org>
References: <20110506165412.GB2719@in.ibm.com>
	<20110506173825.GK11636@one.firstfloor.org>
Message-ID: <20110509163540.GA1963@in.ibm.com>

On Fri, May 06, 2011 at 07:38:25PM +0200, Andi Kleen wrote:
> > Has anybody tested this before? Or have found kdump working when fatal
> > MCEs have actually occurred?
> 
> Ying did some testing. mce-test has test cases for kdump.
> 

We'd be glad to hear about any successful testcases with recent kernels.
My manual testing was quite similar to what the LTP kdump testcase would
do i.e. configure kdump service, trigger crash through
/proc/sysrq-trigger and watchout for kdump....but as you could see in
the logs, that did not happen.

> My guess is you injected the error into some area used by the kexec
> code or boot up path of the kexec kernel.
> 
> -Andi

The logs did not suggest that the second kernel was booted into. The
"Rebooting in ... seconds" message appeared from the first kernel. I
tried the kdump testcase in atleast two dissimilar machines but with
the same results, so it is not clear if the kexec code was affected by
the MCE injection in both the cases.

Thanks,
K.Prasad



From prasad at linux.vnet.ibm.com  Mon May  9 12:53:57 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Mon, 9 May 2011 22:23:57 +0530
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110509123902.GA5975@redhat.com>
References: <20110506165412.GB2719@in.ibm.com>
	<20110509123902.GA5975@redhat.com>
Message-ID: <20110509165357.GB1963@in.ibm.com>

On Mon, May 09, 2011 at 08:39:02AM -0400, Vivek Goyal wrote:
> On Fri, May 06, 2011 at 10:24:12PM +0530, K.Prasad wrote:
> > Hi All,
> > 	I wanted to test the behaviour of kdump when panic is triggered
> > due to MCE on x86 and found that kdump is not captured.
> > 
> > While the kdump service is configured and running and non-MCE panics
> > (such as those triggered through to /proc/sysrq-trigger) successfully
> > capture a kdump, any fatal MCE error injected through the mce-inject
> > tool causes a reboot of the machine.
> > 
> > The code has been traced (using early_serial_putc()) to enter the kexec
> > path i.e. panic()->crash_kexec()->machine_kexec()->relocate_kernel()
> > but is untraceable further.
> > 
> > Kdump works fine when the same the similar test is carried out inside a
> > KVM guest.
> > 
> > Has anybody tested this before? Or have found kdump working when fatal
> > MCEs have actually occurred?
> 
> Prasad,
> 
> I have never tried taking dump in MCE situation. Does kdump work on this
> machine with normal panic()?
> 

Hi Vivek,
	kdump worked fine on this machine for non-MCE triggered panic
calls (the /proc/sysrq-trigger initiated crashes got the kdump fine).

> Use --debug and --serial option in kexec-tools to print some debug message
> and look for "I am in purgatory". This will tell you whether you hanged
> in first kernel or second kernel.
> 

There were no boot logs from the second kernel while the "Rebooting in X
seconds..." message had appeared before the system rebooted, suggesting
that the second kernel did not boot at all.

> Then put "outb()" messages in the kernel to trace what happened. 
> 

The outb logs showed that the system entered machine_kexec function (traceable
upto relocate_kernel) but then rebooted from inside the panic() function.

Thanks,
K.Prasad



From prasad at linux.vnet.ibm.com  Mon May  9 13:03:36 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Mon, 9 May 2011 22:33:36 +0530
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <4DC80662.9000703@canonical.com>
References: <20110506165412.GB2719@in.ibm.com>
	<20110509123902.GA5975@redhat.com> <4DC80662.9000703@canonical.com>
Message-ID: <20110509170336.GC1963@in.ibm.com>

On Mon, May 09, 2011 at 05:21:06PM +0200, Bouchard Louis wrote:
> Hello,
> 
> Le 09/05/2011 14:39, Vivek Goyal a ?crit :
> >
> > Prasad,
> >
> > I have never tried taking dump in MCE situation. Does kdump work on this
> > machine with normal panic()?
> >
> > Use --debug and --serial option in kexec-tools to print some debug message
> > and look for "I am in purgatory". This will tell you whether you hanged
> > in first kernel or second kernel.
> >
> > Then put "outb()" messages in the kernel to trace what happened. 
> >
> > Thanks
> > Vivek
> >
> > _______________________________________________
> > kexec mailing list
> > kexec at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kexec
> I have seen numerous occurrences of MCE triggered kernel panics on both
> RHEL & SLES environment used on IA32 architecture. Both in contexts
> where kexec/kdump was being used.
>

That's interesting! Assuming that these are not software induced MCEs
but panic() calls invoked due to unrecoverable memory errors in a
physical machine, did you experience any situation where the kdump
kernel hung/rebooted due to a second MCE (triggered while reading the
faulty memory location belonging to the first kernel)?
 
>  Matter of fact, MCE triggered panic are part of the reason that pushed
> me to work on crashdc : only one crash command is required to get the
> MCE trace out of the kernel ring buffer. This avoids transfering massive
> amount of vmcore file over the net.
> 

What is the data that is contained in the faulty memory location (whose
I/O triggered an MCE in the first place)? Basically we'd like to
understand what a 'read' operation on the corrupted memory location
would result in.

> crashdc does well on those, mcelog can be applied on the data gathered.
>

We're contemplating a solution on the similar lines (refer the
description of 'slim' kdump at https://lkml.org/lkml/2011/5/4/396) to
create a 'crash tool readable coredump containing a message that
indicates the cause of the crash as MCE (and not any data from the old
memory).

I'll take a look at the crashdc code and see if there are ideas that we
can borrow from there.

Thanks,
K.Prasad
 


From vgoyal at redhat.com  Mon May  9 13:05:28 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 9 May 2011 13:05:28 -0400
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110509165357.GB1963@in.ibm.com>
References: <20110506165412.GB2719@in.ibm.com>
	<20110509123902.GA5975@redhat.com>
	<20110509165357.GB1963@in.ibm.com>
Message-ID: <20110509170528.GG5975@redhat.com>

On Mon, May 09, 2011 at 10:23:57PM +0530, K.Prasad wrote:
> On Mon, May 09, 2011 at 08:39:02AM -0400, Vivek Goyal wrote:
> > On Fri, May 06, 2011 at 10:24:12PM +0530, K.Prasad wrote:
> > > Hi All,
> > > 	I wanted to test the behaviour of kdump when panic is triggered
> > > due to MCE on x86 and found that kdump is not captured.
> > > 
> > > While the kdump service is configured and running and non-MCE panics
> > > (such as those triggered through to /proc/sysrq-trigger) successfully
> > > capture a kdump, any fatal MCE error injected through the mce-inject
> > > tool causes a reboot of the machine.
> > > 
> > > The code has been traced (using early_serial_putc()) to enter the kexec
> > > path i.e. panic()->crash_kexec()->machine_kexec()->relocate_kernel()
> > > but is untraceable further.
> > > 
> > > Kdump works fine when the same the similar test is carried out inside a
> > > KVM guest.
> > > 
> > > Has anybody tested this before? Or have found kdump working when fatal
> > > MCEs have actually occurred?
> > 
> > Prasad,
> > 
> > I have never tried taking dump in MCE situation. Does kdump work on this
> > machine with normal panic()?
> > 
> 
> Hi Vivek,
> 	kdump worked fine on this machine for non-MCE triggered panic
> calls (the /proc/sysrq-trigger initiated crashes got the kdump fine).
> 
> > Use --debug and --serial option in kexec-tools to print some debug message
> > and look for "I am in purgatory". This will tell you whether you hanged
> > in first kernel or second kernel.
> > 
> 
> There were no boot logs from the second kernel while the "Rebooting in X
> seconds..." message had appeared before the system rebooted, suggesting
> that the second kernel did not boot at all.
> 
> > Then put "outb()" messages in the kernel to trace what happened. 
> > 
> 
> The outb logs showed that the system entered machine_kexec function (traceable
> upto relocate_kernel) but then rebooted from inside the panic() function.

Ok, that means that we returned from crash_kexec() function instead of
transitioning into second kernel. This is strange. machine_kexec() is not
supposed to return until and unless it finds that there is no crash
kernel loaded. As per your mail you can trace it to relocate_kernel()
being entered. So only thing I can suggest is debug relocate_kernel()
code now to see why it is returning.

Thanks
Vivek



From prasad at linux.vnet.ibm.com  Mon May  9 13:29:35 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Mon, 9 May 2011 22:59:35 +0530
Subject: [RFC] Kdump and memory error handling
In-Reply-To: <20110504203914.GC1737@one.firstfloor.org>
References: <20110504193509.GA5342@in.ibm.com>
	<20110504203914.GC1737@one.firstfloor.org>
Message-ID: <20110509172935.GD1963@in.ibm.com>

On Wed, May 04, 2011 at 10:39:14PM +0200, Andi Kleen wrote:
> > Any thoughts/suggestions?
> 
> My old attempts to solve this are
> 
> Don't dump on MCE:
> 
> http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/xpanic
> 

The problem we seen in avoiding a panic->crash_kexec->[coredump capture] is
that the user may not have a means to know the reason for crash, unless
the serial console is connected to capture and store the panic string.

Alternatively a 'slim' kdump (as described here:
https://lkml.org/lkml/2011/5/4/396) would not contain meaningless data from
the old memory, but inform the user about the cause of the crash. I'm
intending to post some patches with a quick implementation of it soon.

> Handle dumps of corrupted memory regresions:
> 
> http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/crashdump
> 

> IMHO these patches are still the right solutions for this.
> 

Like Vatsa had raised, the processor's behaviour upon reading (or any I/O
operation) the faulty memory location isn't clearly defined (to the
extent I read through System Programming Guide Part 1, Volume 3A,
Chapter 15). In such a scenario, disabling MCE for the kdump kernel (which can
potentially read the faulty memory) is making things hazy.

Thanks,
K.Prasad



From vgoyal at redhat.com  Mon May  9 13:40:13 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Mon, 9 May 2011 13:40:13 -0400
Subject: [RFC] Kdump and memory error handling
In-Reply-To: <20110509172935.GD1963@in.ibm.com>
References: <20110504193509.GA5342@in.ibm.com>
	<20110504203914.GC1737@one.firstfloor.org>
	<20110509172935.GD1963@in.ibm.com>
Message-ID: <20110509174013.GH5975@redhat.com>

On Mon, May 09, 2011 at 10:59:35PM +0530, K.Prasad wrote:
> On Wed, May 04, 2011 at 10:39:14PM +0200, Andi Kleen wrote:
> > > Any thoughts/suggestions?
> > 
> > My old attempts to solve this are
> > 
> > Don't dump on MCE:
> > 
> > http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/xpanic
> > 
> 
> The problem we seen in avoiding a panic->crash_kexec->[coredump capture] is
> that the user may not have a means to know the reason for crash, unless
> the serial console is connected to capture and store the panic string.
> 
> Alternatively a 'slim' kdump (as described here:
> https://lkml.org/lkml/2011/5/4/396) would not contain meaningless data from
> the old memory, but inform the user about the cause of the crash. I'm
> intending to post some patches with a quick implementation of it soon.
> 
> > Handle dumps of corrupted memory regresions:
> > 
> > http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=shortlog;h=refs/heads/mce/crashdump
> > 
> 
> > IMHO these patches are still the right solutions for this.
> > 
> 
> Like Vatsa had raised, the processor's behaviour upon reading (or any I/O
> operation) the faulty memory location isn't clearly defined (to the
> extent I read through System Programming Guide Part 1, Volume 3A,
> Chapter 15). In such a scenario, disabling MCE for the kdump kernel (which can
> potentially read the faulty memory) is making things hazy.

How would a slim dump make that any better? And why leaving it to user
space to filter out the relevant pieces is not a good idea? 

I agree that it can lead to failure in case the memory we are dependent
on extracting the right information is corrupted but then slim dump
should have similar issues too (until and unless we do something smart
of determining the safe reason and putting all the inforamtion regarding
dump there from inside the kernel after the fault).

Thanks
Vivek


From ying.huang at intel.com  Mon May  9 21:28:00 2011
From: ying.huang at intel.com (Huang Ying)
Date: Tue, 10 May 2011 09:28:00 +0800
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110509163540.GA1963@in.ibm.com>
References: <20110506165412.GB2719@in.ibm.com>
	<20110506173825.GK11636@one.firstfloor.org>
	<20110509163540.GA1963@in.ibm.com>
Message-ID: <4DC894A0.5080303@intel.com>

Hi, Prasad,

On 05/10/2011 12:35 AM, K.Prasad wrote:
> On Fri, May 06, 2011 at 07:38:25PM +0200, Andi Kleen wrote:
>>> Has anybody tested this before? Or have found kdump working when fatal
>>> MCEs have actually occurred?
>>
>> Ying did some testing. mce-test has test cases for kdump.
>>
> 
> We'd be glad to hear about any successful testcases with recent kernels.
> My manual testing was quite similar to what the LTP kdump testcase would
> do i.e. configure kdump service, trigger crash through
> /proc/sysrq-trigger and watchout for kdump....but as you could see in
> the logs, that did not happen.
> 
>> My guess is you injected the error into some area used by the kexec
>> code or boot up path of the kexec kernel.
>>
>> -Andi
> 
> The logs did not suggest that the second kernel was booted into. The
> "Rebooting in ... seconds" message appeared from the first kernel. I
> tried the kdump testcase in atleast two dissimilar machines but with
> the same results, so it is not clear if the kexec code was affected by
> the MCE injection in both the cases.


From louis.bouchard at canonical.com  Tue May 10 03:19:49 2011
From: louis.bouchard at canonical.com (Bouchard Louis)
Date: Tue, 10 May 2011 09:19:49 +0200
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110509170336.GC1963@in.ibm.com>
References: <20110506165412.GB2719@in.ibm.com>	<20110509123902.GA5975@redhat.com>
	<4DC80662.9000703@canonical.com> <20110509170336.GC1963@in.ibm.com>
Message-ID: <4DC8E715.2050102@canonical.com>

Hello,

Le 09/05/2011 19:03, K.Prasad a ?crit :
> On Mon, May 09, 2011 at 05:21:06PM +0200, Bouchard Louis wrote:
>> I have seen numerous occurrences of MCE triggered kernel panics on both
>> RHEL & SLES environment used on IA32 architecture. Both in contexts
>> where kexec/kdump was being used.
>>
> That's interesting! Assuming that these are not software induced MCEs
> but panic() calls invoked due to unrecoverable memory errors in a
> physical machine, did you experience any situation where the kdump
> kernel hung/rebooted due to a second MCE (triggered while reading the
> faulty memory location belonging to the first kernel)? 
What I'm relating is issues I have seen in the past, while I was
employed somewhere else. Unfortunately, I no longer have access to any
of those dumps.

Kind regards,

...Louis

-- 
Louis Bouchard
Server Support Analyst
Canonical Ltd
Ubuntu support: http://landscape.canonical.com



From louis.bouchard at canonical.com  Tue May 10 03:31:14 2011
From: louis.bouchard at canonical.com (Bouchard Louis)
Date: Tue, 10 May 2011 09:31:14 +0200
Subject: [Bug] Kdump does not work when panic triggered due to MCE
In-Reply-To: <20110509154606.GE5975@redhat.com>
References: <20110506165412.GB2719@in.ibm.com>	<20110509123902.GA5975@redhat.com>
	<4DC80662.9000703@canonical.com> <20110509154606.GE5975@redhat.com>
Message-ID: <4DC8E9C2.3030801@canonical.com>

Hello,

Le 09/05/2011 17:46, Vivek Goyal a ?crit :
> Louis, is this "crashdc" an independent executable. Where is it packaged?
> What is the significance of name "crashdc" (Crash data collecor ?) Will it
> make sense to merge it with makedumpfile which does kernel filtering
> already and this could be one of the additional features.
>
> Thanks
> Vivek

crashdc is hosted no SF.  You can have a look at it here :

http://crashdc.svn.sourceforge.net/viewvc/crashdc/trunk/and

Most of the work happens in /usr/bin/crashdc and the
run-crashdc-{distro}.sh scripts.

It does indeed means crash data collector.  It basically is a set of
shell script that send a set of predefined commands to the crash utility
and gather the result in a text file that can be easily sent over email.

In one implementation, it makes use of the KDUMP_POST mechanism to run.
Another way to use it is through an init script upon reboot.

As for merging it with makedumpfile, it could be an interesting choice. 
But that might add requirements to makedumpfile that you might not want
to have (dependencies on crash & the kernel-debuginfo packages which are
needed).

I have recently changed jobs, so I had to suspend my work on crashdc a
bit. I'm hoping to find time to work on it and adapt it to debian based
distro in the near future.

Kind regards,

...Louis

-- 
Louis Bouchard
Server Support Analyst
Canonical Ltd
Ubuntu support: http://landscape.canonical.com



From xiyou.wangcong at gmail.com  Tue May 10 03:48:57 2011
From: xiyou.wangcong at gmail.com (WANG Cong)
Date: Tue, 10 May 2011 07:48:57 +0000 (UTC)
Subject: kdump without the kexec
References: <BANLkTi=Jt=Q9Q9kM4ogtpW0PN7k=WSLtTQ@mail.gmail.com>
	<20110507075500.GB29941@regiomontanus.bwalle.de>
	<BANLkTikgb4f-OTj1zdw9OkOWb-OZpwJS7g@mail.gmail.com>
	<iq82ct$bs6$1@dough.gmane.org>
	<BANLkTin=0D6dZANXtKKDR=JMFaB5wHzr2A@mail.gmail.com>
Message-ID: <iqaql8$tt2$1@dough.gmane.org>

On Mon, 09 May 2011 15:04:42 +0800, Lei Wen wrote:

> Hi Cong,
> 
> On Mon, May 9, 2011 at 2:42 PM, WANG Cong
> <xiyou.wangcong at gmail.com> wrote:
>> On Sun, 08 May 2011 23:31:30 +0800, Lei Wen wrote:
>>
>>> Hi Bernhard,
>>>
>>> On Sat, May 7, 2011 at 3:55 PM, Bernhard Walle
>>> <bernhard at bwalle.de> wrote:
>>>> * Lei Wen <adrian.wenl at gmail.com>
>>>> [2011-05-06 16:33]:
>>>>>
>>>>> Is there any existed solution that could make the kdump without the
>>>>> kexec? For some kind of system hang, always hardware bug, or
>>>>> software deadlock, which could not trigger the kernel oops, so that
>>>>> the first kernel cannot load the second kernel
>>>>> with kexec tool. The only way here is to directly press the reset
>>>>> key.
>>>>>
>>>>> We could be assure that the ddr memory would not be corrupted by
>>>>> this way of reset
>>>>> (for we don't shutdown the ddr power during the reset). So my
>>>>> question is that whether
>>>>> we could take use of bootloader, like the u-boot, to pretend we are
>>>>> loading the second kernel
>>>>> after the reset behavior and then perform the kdump process?
>>>>
>>>> Did you try sysrq-c? Which platform? Most of them have the concept of
>>>> a NMI that also can trigger kdump.
>>>>
>>>>
>>> For some case, the sysrq-c also cannot help, as the cpu is not
>>> responding anymore for some
>>> hw bug. I am running on arm board, there is no NMI concept on it... So
>>> the only help is to put the hw reset and wish the bootloader do the
>>> task which is expected
>>> done by the kexec...
>>
>> Then kdump can't help in this case.
>>
>> And I am afraid you can't use the normal bootloader (e.g. grub, uboot),
>> because we load the second kernel *while* running the first kernel,
>> which means, for example, on x86 we are in protect mode after the
>> kernel boots, not in the real mode in which grub starts.
>>
>> What's more, kdump will not hw-reset the devices unless you add
>> "reset_devices" to the second kernel.
>>
> I see...
> But what I am care is the memory dump done by the kdump. If I could boot
> the second kernel without break the first kernel's space, that by put
> the second kernel just in the space that has been reserved by the first
> kernel. Is that possible to use the bootloader to perform such task?
> 
> The devices state is not really cared by us, for the only interesting
> point is the
> memory context left by the first kernel.

You need to pass correct kernel parameters to the second kernel to make 
this happen, kexec uses "memmap=exactmap memmap=XXX ..." to describe the 
memory map used by the second kernel.

This is _not_ hard for normal bootloaders, the problem is still that you 
need to load the second kernel before the first kernel crashes and during 
the first kernel is running.



From xiyou.wangcong at gmail.com  Tue May 10 03:51:55 2011
From: xiyou.wangcong at gmail.com (WANG Cong)
Date: Tue, 10 May 2011 07:51:55 +0000 (UTC)
Subject: how to use kexec on android?
References: <4DAD7E60.8090009@Thunderst.com>
Message-ID: <iqaqqr$tt2$2@dough.gmane.org>

On Tue, 19 Apr 2011 20:21:52 +0800, zhangchenyu wrote:

> hi:
>      I want to use kexec on android, could somebody tell me how to use
>      it?
> 

I am wondering why you want to use it? For kdump or fast reboot?

And does the android kernel support kexec?



From xiyou.wangcong at gmail.com  Tue May 10 05:42:11 2011
From: xiyou.wangcong at gmail.com (WANG Cong)
Date: Tue, 10 May 2011 09:42:11 +0000 (UTC)
Subject: Kexec & Memory Zones question
References: <BANLkTinOd+kmJKMdtW7saW0eWHN9xbO_3w@mail.gmail.com>
Message-ID: <iqb19j$e10$1@dough.gmane.org>

On Wed, 04 May 2011 11:35:46 -0700, Sujit V wrote:

> On our x86_64 NUMA hardware running linux 2.6.23 with two memory nodes
>  have the following zone layout
> DMA   0 - 16MB
> DMA32   16MB to 4GB
> NORMAL 4GB to 96GB
> 
> We had the crashkernel boot param as 128M at 16M. I am using
> kexec-tools-2.0.0
> 
> We observed that the reserved crashkernel memory was getting used by the
> system.
> We found out by kdb  that around 17MB of physical memory the memory
> contents were changing which proved that the system was using the memory
> which is actually reserved for crash kernel. [ If I load crash kernel
> using kexec then the system would crash. The back trace was always in
> the megasas driver. ]
> 
> My Question is
> 1) Should the crashkernel memory be located past the DMA32 zone.?


Your DMA32 zone looks extremely large, probably due to you enabled NUMA? 
What does dmesg say if possible? Also, what is your kernel version?

> 
> I have tried the following
> (1) crashkernel=128M at 4GB  ( So the memory reservation is past DMA32) In
> this scenario the kexec tools gave an error "Could not find a free area
> of memory of xyz bytes"
> 
> (2) I changed the max of DMA32 to 1GB
>      crashkernel=128M at 1G
> Still kexec gave the same error "Could not find a free area of memory of
> xyz bytes"
> 
> Is there any specific restriction on where the crashkernel memory could
> be located.?

No, have you tried "crashkernel=128M at 0"? Can that reserve the memory
successfully?

> 
> Is it ok to be in the DMA32 region OR it should be beyond the DMA32
> region.
> 


The reserve memory is unseen by the first kernel nor it should be used,
even if it is in DMA32 zone.




From xiyou.wangcong at gmail.com  Tue May 10 06:21:08 2011
From: xiyou.wangcong at gmail.com (WANG Cong)
Date: Tue, 10 May 2011 10:21:08 +0000 (UTC)
Subject: [Bug] Kdump does not work when panic triggered due to MCE
References: <20110506165412.GB2719@in.ibm.com>
	<20110509123902.GA5975@redhat.com> <4DC80662.9000703@canonical.com>
	<20110509170336.GC1963@in.ibm.com>
Message-ID: <iqb3ij$e10$2@dough.gmane.org>

On Mon, 09 May 2011 22:33:36 +0530, K.Prasad wrote:

> That's interesting! Assuming that these are not software induced MCEs
> but panic() calls invoked due to unrecoverable memory errors in a
> physical machine, did you experience any situation where the kdump
> kernel hung/rebooted due to a second MCE (triggered while reading the
> faulty memory location belonging to the first kernel)?


Someone discussed this with me inside RH, is disabling MCE
checking in the second kernel an acceptable solution?

> We're contemplating a solution on the similar lines (refer the
> description of 'slim' kdump at https://lkml.org/lkml/2011/5/4/396) to
> create a 'crash tool readable coredump containing a message that
> indicates the cause of the crash as MCE (and not any data from the old
> memory).
> 

You might want to try use other way instead of memory to store
the log of the first kernel, for example, mtdoops.



From vgoyal at redhat.com  Wed May 11 11:09:08 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Wed, 11 May 2011 11:09:08 -0400
Subject: Kexec & Memory Zones question
In-Reply-To: <BANLkTinOd+kmJKMdtW7saW0eWHN9xbO_3w@mail.gmail.com>
References: <BANLkTinOd+kmJKMdtW7saW0eWHN9xbO_3w@mail.gmail.com>
Message-ID: <20110511150908.GG31633@redhat.com>

On Wed, May 04, 2011 at 11:35:46AM -0700, Sujit V wrote:
> On our x86_64 NUMA hardware running linux 2.6.23 with two memory nodes
>  have the following zone layout
> DMA   0 - 16MB
> DMA32   16MB to 4GB
> NORMAL 4GB to 96GB
> 
> We had the crashkernel boot param as 128M at 16M.
> I am using kexec-tools-2.0.0
> 
> We observed that the reserved crashkernel memory was getting used by
> the system.
> We found out by kdb  that around 17MB of physical memory the memory
> contents were changing which proved that the system was
> using the memory which is actually reserved for crash kernel.
> [ If I load crash kernel using kexec then the system would crash. The
> back trace was always in the megasas driver. ]

This is problematic. This memory has been reserved by kernel in early
boot and nobody else should be using this memory. This sounds like a
but somewhere.

> 
> My Question is
> 1) Should the crashkernel memory be located past the DMA32 zone.?
> 
> I have tried the following
> (1) crashkernel=128M at 4GB  ( So the memory reservation is past DMA32)
> In this scenario the kexec tools gave an error "Could not find a free
> area of memory of xyz bytes"
> 
> (2) I changed the max of DMA32 to 1GB
>      crashkernel=128M at 1G
> Still kexec gave the same error "Could not find a free area of memory
> of xyz bytes"

We have discussed this in the past and due to various reasons the max
amount of RAM you can boot your kernel from seems to be 896MB for
x86_64 and 512MB for 32bit. I shall have to open a previous thread
with hpa to get exact numbers. So loading kernel even higher is not
the solution.

We need to figure out who is using this memory and where is the bug
and fix it.

Thanks
Vivek


From xiyou.wangcong at gmail.com  Thu May 12 06:03:03 2011
From: xiyou.wangcong at gmail.com (WANG Cong)
Date: Thu, 12 May 2011 10:03:03 +0000 (UTC)
Subject: Kexec & Memory Zones question
References: <BANLkTinOd+kmJKMdtW7saW0eWHN9xbO_3w@mail.gmail.com>
	<20110511150908.GG31633@redhat.com>
Message-ID: <iqgb8m$ruq$1@dough.gmane.org>

On Wed, 11 May 2011 11:09:08 -0400, Vivek Goyal wrote:

> We have discussed this in the past and due to various reasons the max
> amount of RAM you can boot your kernel from seems to be 896MB for x86_64
> and 512MB for 32bit. I shall have to open a previous thread with hpa to
> get exact numbers. So loading kernel even higher is not the solution.
> 

On the kexec-tools side, I think the limit is hard-coded,

./include/x86/x86-linux.h:250:#define DEFAULT_INITRD_ADDR_MAX 0x37FFFFFF

but we have,

        initrd_addr_max = DEFAULT_INITRD_ADDR_MAX;
        if (real_mode->protocol_version >= 0x0203) {
                initrd_addr_max = real_mode->initrd_addr_max;
                dbgprintf("initrd_addr_max is 0x%lx\n", initrd_addr_max);
        }


so, from the code, initrd_addr_max can be provided by the bootloader.

I remember on the kernel side there's also such a limit, but I can't
find where it is. I am wondering what prevents us from increasing this 
limit to 4G on i386 and even higher on x86_64.

Thanks.



From ebiederm at xmission.com  Thu May 12 18:22:44 2011
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Thu, 12 May 2011 15:22:44 -0700
Subject: [RFC] Kdump and memory error handling
In-Reply-To: <20110504193509.GA5342@in.ibm.com> (K. Prasad's message of "Thu, 
	5 May 2011 01:05:09 +0530")
References: <20110504193509.GA5342@in.ibm.com>
Message-ID: <m1hb8zmudn.fsf@fess.ebiederm.org>

"K.Prasad" <prasad at linux.vnet.ibm.com> writes:

> Hi All,
>         We've been trying to study and improve the kdump behaviour when
> a panic is triggered due to an unrecoverable memory error causing a
> machine check exception (MCE) followed by a kernel panic.
>
> In this context we foresee a few issues in capturing kdump and would
> like to receive comments about the ways to handle them.
>
> Probable Issues when capturing coredump through kdump following a memory
> error
> ---------------------------
> - First, a coredump of the memory from the crashing kernel isn't really
>   helpful in debugging the crash that was caused due to a faulty memory.
>   Collecting the same has some of the problems illustrated below. It should
>   therefore suffice to let the user know the reason of the crash
>   rather than provide a complete dump of the memory.
>
>   For this, a 'slim' yet crash-tool readable coredump containing:
>   - message about the cause (such as crash due to unrecoverable memory error)
>     in the coredump's elf-note section.
>   - and no data from the memory of the 'crashing' kernel (their elf
>     sections can be reduced to zero length).
>   may be suitable.
>
> - Alternatively, if the kdump kernel decides to capture the coredump,
>   its attempts to read the faulty memory location may lead to subsequent 
>   faults in the context of kdump kernel with fatal consequences. This
>   may either be avoided by:
>
>   a) Pass the address of the corrupt memory location to the kdump kernel
>   and skip reading that location while creating the vmcore. This needs
>   an instance of 'struct mce' (from the 'crashing' kernel), which
>   already contains the faulty memory address (in the physical address
>   form, which should be confirmed using the IA32_MCi_MISC[8:6] bits stored
>   in 'misc' field of 'struct mce') to be populated inside the elf
>   (-notes?) section.
>
>   b) Use modified copy applications (such as a modified 'cp' command)
>   that can map the /dev/oldmem into user-space and then initiate the
>   creation of vmcore. In this method, the user-space process performing
>   the copy will receive a SIGBUS while consuming the faulty memory (through
>   INT18 -> do_machine_check) but it must be modified to be resilient to the
>   signal, while intelligently skipping to the subsequent memory location
>   for further copying. Meanwhile the data for the faulty memory location
>   can be represented using 'zero-ed' data and the vmcore enhanced to
>   indicate the cause of the crash as one resulting from a fatal MCE.
>
> Any thoughts/suggestions?

In practice this all works for me.

I have received several crash dumps where there was an mce error.

I admit I have my userspace configured to just grab the dmesg from the
kernel log and not do a full crash dump.  So in that sense I am already
a slim crash dump.

But in practice with real hardware errors it is working today without
kernel changes.

Eric


From adrian.wenl at gmail.com  Tue May 17 10:49:45 2011
From: adrian.wenl at gmail.com (Lei Wen)
Date: Tue, 17 May 2011 22:49:45 +0800
Subject: kdump without the kexec
In-Reply-To: <iqaql8$tt2$1@dough.gmane.org>
References: <BANLkTi=Jt=Q9Q9kM4ogtpW0PN7k=WSLtTQ@mail.gmail.com>
	<20110507075500.GB29941@regiomontanus.bwalle.de>
	<BANLkTikgb4f-OTj1zdw9OkOWb-OZpwJS7g@mail.gmail.com>
	<iq82ct$bs6$1@dough.gmane.org>
	<BANLkTin=0D6dZANXtKKDR=JMFaB5wHzr2A@mail.gmail.com>
	<iqaql8$tt2$1@dough.gmane.org>
Message-ID: <BANLkTintTcL+WdthGRcbOnEBZdLucuO30g@mail.gmail.com>

On Tue, May 10, 2011 at 3:48 PM, WANG Cong <xiyou.wangcong at gmail.com> wrote:
> On Mon, 09 May 2011 15:04:42 +0800, Lei Wen wrote:
>
>> Hi Cong,
>>
>> On Mon, May 9, 2011 at 2:42 PM, WANG Cong
>> <xiyou.wangcong at gmail.com> wrote:
>>> On Sun, 08 May 2011 23:31:30 +0800, Lei Wen wrote:
>>>
>>>> Hi Bernhard,
>>>>
>>>> On Sat, May 7, 2011 at 3:55 PM, Bernhard Walle
>>>> <bernhard at bwalle.de> wrote:
>>>>> * Lei Wen <adrian.wenl at gmail.com>
>>>>> [2011-05-06 16:33]:
>>>>>>
>>>>>> Is there any existed solution that could make the kdump without the
>>>>>> kexec? For some kind of system hang, always hardware bug, or
>>>>>> software deadlock, which could not trigger the kernel oops, so that
>>>>>> the first kernel cannot load the second kernel
>>>>>> with kexec tool. The only way here is to directly press the reset
>>>>>> key.
>>>>>>
>>>>>> We could be assure that the ddr memory would not be corrupted by
>>>>>> this way of reset
>>>>>> (for we don't shutdown the ddr power during the reset). So my
>>>>>> question is that whether
>>>>>> we could take use of bootloader, like the u-boot, to pretend we are
>>>>>> loading the second kernel
>>>>>> after the reset behavior and then perform the kdump process?
>>>>>
>>>>> Did you try sysrq-c? Which platform? Most of them have the concept of
>>>>> a NMI that also can trigger kdump.
>>>>>
>>>>>
>>>> For some case, the sysrq-c also cannot help, as the cpu is not
>>>> responding anymore for some
>>>> hw bug. I am running on arm board, there is no NMI concept on it... So
>>>> the only help is to put the hw reset and wish the bootloader do the
>>>> task which is expected
>>>> done by the kexec...
>>>
>>> Then kdump can't help in this case.
>>>
>>> And I am afraid you can't use the normal bootloader (e.g. grub, uboot),
>>> because we load the second kernel *while* running the first kernel,
>>> which means, for example, on x86 we are in protect mode after the
>>> kernel boots, not in the real mode in which grub starts.
>>>
>>> What's more, kdump will not hw-reset the devices unless you add
>>> "reset_devices" to the second kernel.
>>>
>> I see...
>> But what I am care is the memory dump done by the kdump. If I could boot
>> the second kernel without break the first kernel's space, that by put
>> the second kernel just in the space that has been reserved by the first
>> kernel. Is that possible to use the bootloader to perform such task?
>>
>> The devices state is not really cared by us, for the only interesting
>> point is the
>> memory context left by the first kernel.
>
> You need to pass correct kernel parameters to the second kernel to make
> this happen, kexec uses "memmap=exactmap memmap=XXX ..." to describe the
> memory map used by the second kernel.
>
> This is _not_ hard for normal bootloaders, the problem is still that you
> need to load the second kernel before the first kernel crashes and during
> the first kernel is running.
>
Thanks for this hint. :) I would have it a try.

Best regards,
Lei


From prasad at linux.vnet.ibm.com  Tue May 17 13:24:50 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Tue, 17 May 2011 22:54:50 +0530
Subject: [RFC] Kdump and memory error handling
In-Reply-To: <m1hb8zmudn.fsf@fess.ebiederm.org>
References: <20110504193509.GA5342@in.ibm.com>
	<m1hb8zmudn.fsf@fess.ebiederm.org>
Message-ID: <20110517172450.GC24131@in.ibm.com>

On Thu, May 12, 2011 at 03:22:44PM -0700, Eric W. Biederman wrote:
> "K.Prasad" <prasad at linux.vnet.ibm.com> writes:
> 
> > Hi All,
> >         We've been trying to study and improve the kdump behaviour when
> > a panic is triggered due to an unrecoverable memory error causing a
> > machine check exception (MCE) followed by a kernel panic.
> >
> > In this context we foresee a few issues in capturing kdump and would
> > like to receive comments about the ways to handle them.
> >
> > Probable Issues when capturing coredump through kdump following a memory
> > error
> > ---------------------------
> > - First, a coredump of the memory from the crashing kernel isn't really
> >   helpful in debugging the crash that was caused due to a faulty memory.
> >   Collecting the same has some of the problems illustrated below. It should
> >   therefore suffice to let the user know the reason of the crash
> >   rather than provide a complete dump of the memory.
> >
> >   For this, a 'slim' yet crash-tool readable coredump containing:
> >   - message about the cause (such as crash due to unrecoverable memory error)
> >     in the coredump's elf-note section.
> >   - and no data from the memory of the 'crashing' kernel (their elf
> >     sections can be reduced to zero length).
> >   may be suitable.
> >
> > - Alternatively, if the kdump kernel decides to capture the coredump,
> >   its attempts to read the faulty memory location may lead to subsequent 
> >   faults in the context of kdump kernel with fatal consequences. This
> >   may either be avoided by:
> >
> >   a) Pass the address of the corrupt memory location to the kdump kernel
> >   and skip reading that location while creating the vmcore. This needs
> >   an instance of 'struct mce' (from the 'crashing' kernel), which
> >   already contains the faulty memory address (in the physical address
> >   form, which should be confirmed using the IA32_MCi_MISC[8:6] bits stored
> >   in 'misc' field of 'struct mce') to be populated inside the elf
> >   (-notes?) section.
> >
> >   b) Use modified copy applications (such as a modified 'cp' command)
> >   that can map the /dev/oldmem into user-space and then initiate the
> >   creation of vmcore. In this method, the user-space process performing
> >   the copy will receive a SIGBUS while consuming the faulty memory (through
> >   INT18 -> do_machine_check) but it must be modified to be resilient to the
> >   signal, while intelligently skipping to the subsequent memory location
> >   for further copying. Meanwhile the data for the faulty memory location
> >   can be represented using 'zero-ed' data and the vmcore enhanced to
> >   indicate the cause of the crash as one resulting from a fatal MCE.
> >
> > Any thoughts/suggestions?
> 
> In practice this all works for me.
> 
> I have received several crash dumps where there was an mce error.
> 
> I admit I have my userspace configured to just grab the dmesg from the
> kernel log and not do a full crash dump.  So in that sense I am already
> a slim crash dump.
> 
> But in practice with real hardware errors it is working today without
> kernel changes.
>

The problem with the existing kernel code is that it allows for the old
kernel's memory regions to be read (through read_vmcore function),
although intelligent userspace tools may avoid such a possibility (like
the one you mentioned).

Given that the system can experience recursive MCE faults while reading
the corrupt memory region, a 'slim' vmcore region presented by the kernel
to the user-space would be a safe option. We could also use such a dump to
include more relevant information such as address of corrupt memory,
type of memory error, etc.

Thanks,
K.Prasad



From mahesh at linux.vnet.ibm.com  Tue May 17 15:59:06 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:29:06 +0530
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter out
	kernel data from vmcore
Message-ID: <20110517195751.12740.97565.stgit@mars.in.ibm.com>

Hi All,

Please find the version 2 of makedumpfile enhancement patchset that introduces
a data filtering feature which enables makedumpfile to filter out desired kernel
symbol data and it's members from the specified VMCORE file. The data to be
filtered out is poisoned with character 'X' (58 in Hex).

This feature will be very useful for the customers who wants to erase the
customer sensitive data like security keys and other confidential data, in
DUMPFILE before sending it to support team for analysis.

This feature introduces a filter config file where, using filter commands,
user can specify desired kernel data symbols and it's members that need to be
filtered out while creating o/p DUMPFILE. The Syntax for filter commands are
provided in the makedumpfile.conf(8) man page.

The first 4 patches prepares the base work for filtering framework. The
patches 5 and 6 implements the generic filtering framework to erase desired
kernel data. The last two patches implements inclusion of eraseinfo data into
o/p dumpfile (kdump as well as ELF). The eraseinfo data contains the
effective lines from configuration files.

I have tested these patches on x86_64 and s390x architecture against RHEL6 GA
kernel. The feature supports filtering data from ELF as well as
kdump-compressed formatted dump.

Change in v2:
- Changes in patch 5 and inclusion of new patches 7 & 8
- Re-named filter.conf file to makedumpfile.conf and some cleanup work.
- Introduced last two patches that implements inclusion of eraseinfo data
  into kdump compressed as well as ELF formatted dumpfile.

Please review the patchset and let me know your comments.

Thanks,
-Mahesh.
---

Mahesh Salgaonkar (8):
      makedumpfile: Add '--config' option to specify filter config file.
      makedumpfile: Apply relocation while loading module debuginfo.
      makedumpfile: Load the module symbol data from vmcore.
      makedumpfile: Introduce routines to get type name from debuginfo.
      makedumpfile: Read and process filter commands from config file.
      makedumpfile: Read and process 'for' command from config file.
      makedumpfile: Add erased information in compressed kdump file
      makedumpfile: Add erase information in ELF formatted dumpfile


 IMPLEMENTATION      |    6 
 Makefile            |    8 
 diskdump_mod.h      |    2 
 makedumpfile.8      |   20 
 makedumpfile.c      | 2747 ++++++++++++++++++++++++++++++++++++++++++++++++---
 makedumpfile.conf   |  149 +++
 makedumpfile.conf.8 |  419 ++++++++
 makedumpfile.h      |  199 ++++
 8 files changed, 3394 insertions(+), 156 deletions(-)
 create mode 100644 makedumpfile.conf
 create mode 100644 makedumpfile.conf.8

-- 
Signature


From mahesh at linux.vnet.ibm.com  Tue May 17 16:00:07 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:30:07 +0530
Subject: [PATCH v2 1/8] makedumpfile: Add '--config' option to specify filter
	config file.
Message-ID: <20110517195951.12740.25514.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

This patch adds '--config' option to specify config file that contains filter
commands to filter out desired kernel data and it's members.

Updated makedumpfile man page with '--config' option info.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
---

 makedumpfile.8 |   20 +++++++++++++++++++-
 makedumpfile.c |   17 +++++++++++++++++
 makedumpfile.h |    7 +++++++
 3 files changed, 43 insertions(+), 1 deletions(-)

diff --git a/makedumpfile.8 b/makedumpfile.8
index 4a0271d..a1aaa3e 100644
--- a/makedumpfile.8
+++ b/makedumpfile.8
@@ -6,6 +6,8 @@ makedumpfile \- make a small dumpfile of kdump
 .br
 \fBmakedumpfile\fR \-F [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR
 .br
+\fBmakedumpfile\fR   [\fIOPTION\fR] \-x \fIVMLINUX\fR \-\-config \fIFILTERCONFIGFILE\fR \fIVMCORE\fR \fIDUMPFILE\fR
+.br
 \fBmakedumpfile\fR \-R \fIDUMPFILE\fR
 .br
 \fBmakedumpfile\fR \-\-split [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE1\fR \fIDUMPFILE2\fR [\fIDUMPFILE3\fR ..]
@@ -55,6 +57,15 @@ compression support. The ELF format is readable with GDB and the crash utility.
 If a user wants to use GDB, \fIDUMPFILE\fR format has to be explicitly
 specified to be the ELF format.
 .PP
+Apart from the exclusion of unnecessary pages mentioned above, makedumpfile
+allows user to filter out targeted kernel data. The filter config file can
+be used to specify kernel/module symbols and its members that need to be
+filtered out through the erase command syntax. makedumpfile reads the filter
+config and builds the list of memory addresses and its sizes after processing
+filter commands. The memory locations that require to be filtered out are
+then poisoned with character 'X' (58 in Hex). Refer to \fBfilter.conf(8)\fR
+for file format.
+.PP
 To analyze the first kernel's memory usage, makedumpfile can refer to
 \fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the first
 kernel's information (structure size, field offset, etc.), and \fIVMCOREINFO\fR
@@ -232,6 +243,13 @@ specified.
 # makedumpfile \-g vmcoreinfo \-x vmlinux
 
 .TP
+\fB\-\-config\fR \fIFILTERCONFIGFILE\fR
+Used in conjunction with \-x \fIVMLINUX\fR option, to specify the filter
+config file \fIFILTERCONFIGFILE\fR that contains erase commands to filter out
+desired kernel data from vmcore while creating \fIDUMPFILE\fR. For filter
+command syntax please refer to \fBfilter.conf(8)\fR.
+
+.TP
 \fB\-F\fR
 Output the dump data in the flattened format to the standard output for
 transporting the dump data by SSH.
@@ -482,5 +500,5 @@ Written by Masaki Tachibana, and Ken'ichi Ohmichi.
 
 .SH SEE ALSO
 .PP
-crash(8), gdb(1), kexec(8)
+crash(8), gdb(1), kexec(8), filter.conf(8)
 
diff --git a/makedumpfile.c b/makedumpfile.c
index 7d0c663..f136eba 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -636,6 +636,11 @@ print_usage(void)
 	MSG("  Creating DUMPFILE:\n");
 	MSG("  # makedumpfile    [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE DUMPFILE\n");
 	MSG("\n");
+	MSG("  Creating DUMPFILE with filtered kernel data specified through filter config\n");
+	MSG("  file:\n");
+	MSG("  # makedumpfile    [-c|-E] [-d DL] -x VMLINUX --config FILTERCONFIGFILE VMCORE\n");
+	MSG("    DUMPFILE\n");
+	MSG("\n");
 	MSG("  Outputting the dump data in the flattened format to the standard output:\n");
 	MSG("  # makedumpfile -F [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE\n");
 	MSG("\n");
@@ -714,6 +719,11 @@ print_usage(void)
 	MSG("      other system that is running the same first kernel. [-x VMLINUX] must\n");
 	MSG("      be specified.\n");
 	MSG("\n");
+	MSG("  [--config FILTERCONFIGFILE]:\n");
+	MSG("      Used in conjunction with -x VMLINUX option, to specify the filter config\n");
+	MSG("      file that contains filter commands to filter out desired kernel data\n");
+	MSG("      from vmcore while creating DUMPFILE.\n");
+	MSG("\n");
 	MSG("  [-F]:\n");
 	MSG("      Output the dump data in the flattened format to the standard output\n");
 	MSG("      for transporting the dump data by SSH.\n");
@@ -7729,6 +7739,9 @@ check_param_for_creating_dumpfile(int argc, char *argv[])
 	if (info->flag_flatten && info->flag_split)
 		return FALSE;
 
+	if (info->name_filterconfig && !info->name_vmlinux)
+		return FALSE;
+
 	if ((argc == optind + 2) && !info->flag_flatten
 				 && !info->flag_split) {
 		/*
@@ -7821,6 +7834,7 @@ static struct option longopts[] = {
 	{"message-level", required_argument, NULL, 'm'},
 	{"vtop", required_argument, NULL, 'V'},
 	{"dump-dmesg", no_argument, NULL, 'M'}, 
+	{"config", required_argument, NULL, 'C'},
 	{"help", no_argument, NULL, 'h'},
 	{0, 0, 0, 0}
 };
@@ -7851,6 +7865,9 @@ main(int argc, char *argv[])
 		case 'b':
 			info->block_order = atoi(optarg);
 			break;
+		case 'C':
+			info->name_filterconfig = optarg;
+			break;
 		case 'c':
 			info->flag_compress = 1;
 			break;
diff --git a/makedumpfile.h b/makedumpfile.h
index 4ec77b7..e037f12 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -888,6 +888,13 @@ struct DumpInfo {
 	unsigned long	vmemmap_end;
 
 	/*
+	 * Filter config file containing filter commands to filter out kernel
+	 * data from vmcore.
+	 */
+	char		*name_filterconfig;
+	FILE		*file_filterconfig;
+
+	/*
 	 * diskdimp info:
 	 */
 	int		block_order;



From mahesh at linux.vnet.ibm.com  Tue May 17 16:01:01 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:31:01 +0530
Subject: [PATCH v2 2/8] makedumpfile: Apply relocation while loading module
	debuginfo.
Message-ID: <20110517200052.12740.36169.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

So far makedumpfile implementation does not have to deal with module
debuginfo files. In order to support filtering of kernel module data,
it needs to read dwarf information from module debuginfo files. The linux
kernel module debuginfo are of ET_REL type and requires relocation to be
applied in order to get the dwarf information.

This patch uses dwfl_* API's to make the code relocation-aware while
loading module debuginfo files. It also uses dwfl_* API to search module
debuginfo files installed under default debuginfo path.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
---

 makedumpfile.c |  261 +++++++++++++++++++++++++++++++++++++++++++++-----------
 makedumpfile.h |    7 ++
 2 files changed, 216 insertions(+), 52 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index f136eba..2274b18 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -1372,6 +1372,146 @@ get_elf_info(void)
 	return TRUE;
 }
 
+static int
+process_module (Dwfl_Module *dwflmod,
+		void **userdata __attribute__ ((unused)),
+		const char *name __attribute__ ((unused)),
+		Dwarf_Addr base __attribute__ ((unused)),
+		void *arg)
+{
+	const char *fname, *mod_name, *debugfile;
+	Dwarf_Addr dwbias;
+
+	/* get a debug context descriptor.*/
+	dwarf_info.dwarfd = dwfl_module_getdwarf (dwflmod, &dwbias);
+	dwarf_info.elfd = dwarf_getelf(dwarf_info.dwarfd);
+
+	mod_name = dwfl_module_info(dwflmod, NULL, NULL, NULL, NULL, NULL,
+							&fname, &debugfile);
+
+	if (!strcmp(dwarf_info.module_name, mod_name) &&
+		!dwarf_info.name_debuginfo && debugfile) {
+		/*
+		 * Store the debuginfo filename. Next time we will
+		 * open debuginfo file direclty instead of searching
+		 * for it again.
+		 */
+		dwarf_info.name_debuginfo = strdup(debugfile);
+	}
+
+	return DWARF_CB_OK;
+}
+
+static int
+dwfl_report_module_p(const char *modname, const char *filename)
+{
+	if (filename && !strcmp(modname, dwarf_info.module_name))
+		return 1;
+	return 0;
+}
+
+static void
+clean_dwfl_info(void)
+{
+	if (dwarf_info.dwfl)
+		dwfl_end(dwarf_info.dwfl);
+
+	dwarf_info.dwfl = NULL;
+	dwarf_info.dwarfd = NULL;
+	dwarf_info.elfd = NULL;
+}
+
+/*
+ * Intitialize the dwarf info.
+ * Linux kernel module debuginfo are of ET_REL (relocatable) type. The old
+ * implementation of get_debug_info() function that reads the debuginfo was
+ * not relocation-aware and hence could not read the dwarf info properly
+ * from module debuginfo.
+ *
+ * This function uses dwfl API's to apply relocation before reading the
+ * dwarf information from module debuginfo.
+ * On success, this function sets the dwarf_info.elfd and dwarf_info.dwarfd
+ * after applying relocation to module debuginfo.
+ */
+static int
+init_dwarf_info(void)
+{
+	Dwfl *dwfl = NULL;
+	int dwfl_fd = -1;
+	static char *debuginfo_path = DEFAULT_DEBUGINFO_PATH;
+	static const Dwfl_Callbacks callbacks = {
+		.section_address = dwfl_offline_section_address,
+		.find_debuginfo = dwfl_standard_find_debuginfo,
+		.debuginfo_path = &debuginfo_path,
+	};
+
+	dwarf_info.elfd = NULL;
+	dwarf_info.dwarfd = NULL;
+
+	if ((dwfl = dwfl_begin(&callbacks)) == NULL) {
+		ERRMSG("Can't create a handle for a new dwfl session.\n");
+		return FALSE;
+	}
+
+	if (dwarf_info.name_debuginfo) {
+		/* We have absolute path for debuginfo file, use it directly
+		 * instead of searching it through
+		 * dwfl_linux_kernel_report_offline() call.
+		 *
+		 * Open the debuginfo file if it is not already open.
+		 */
+		if (dwarf_info.fd_debuginfo < 0)
+			dwarf_info.fd_debuginfo =
+				open(dwarf_info.name_debuginfo, O_RDONLY);
+
+		dwfl_fd = dup(dwarf_info.fd_debuginfo);
+		if (dwfl_fd < 0) {
+			ERRMSG("Failed to get a duplicate handle for"
+				" debuginfo.\n");
+			goto err_out;
+		}
+	}
+	if (dwarf_info.fd_debuginfo > 0) {
+		if (dwfl_report_offline(dwfl, dwarf_info.module_name,
+				dwarf_info.name_debuginfo, dwfl_fd) == NULL) {
+			ERRMSG("Failed reading %s: %s\n",
+				dwarf_info.name_debuginfo, dwfl_errmsg (-1));
+			/* dwfl_fd is consumed on success, not on failure */
+			close(dwfl_fd);
+			goto err_out;
+		}
+	}
+	else if (dwfl_linux_kernel_report_offline(dwfl,
+						info->system_utsname.release,
+						&dwfl_report_module_p)) {
+		ERRMSG("Can't get Module debuginfo for module '%s'\n",
+					dwarf_info.module_name);
+		goto err_out;
+	}
+	dwfl_report_end(dwfl, NULL, NULL);
+
+	dwfl_getmodules(dwfl, &process_module, NULL, 0);
+
+	if (dwarf_info.elfd == NULL) {
+		ERRMSG("Can't get first elf header of %s.\n",
+		    dwarf_info.name_debuginfo);
+		goto err_out;
+	}
+
+	if (dwarf_info.dwarfd == NULL) {
+		ERRMSG("Can't get debug context descriptor for %s.\n",
+		    dwarf_info.name_debuginfo);
+		goto err_out;
+	}
+	dwarf_info.dwfl = dwfl;
+	return TRUE;
+err_out:
+	if (dwfl)
+		dwfl_end(dwfl);
+
+	return FALSE;
+}
+
 unsigned long long
 get_symbol_addr(char *symname)
 {
@@ -1383,18 +1523,12 @@ get_symbol_addr(char *symname)
 	Elf_Data *data = NULL;
 	Elf_Scn *scn = NULL;
 	char *sym_name = NULL;
-	const off_t failed = (off_t)-1;
 
-	if (lseek(dwarf_info.fd_debuginfo, 0, SEEK_SET) == failed) {
-		ERRMSG("Can't seek the kernel file(%s). %s\n",
-		    dwarf_info.name_debuginfo, strerror(errno));
-		return NOT_FOUND_SYMBOL;
-	}
-	if (!(elfd = elf_begin(dwarf_info.fd_debuginfo, ELF_C_READ, NULL))) {
-		ERRMSG("Can't get first elf header of %s.\n",
-		    dwarf_info.name_debuginfo);
-		return NOT_FOUND_SYMBOL;
-	}
+	if (!init_dwarf_info())
+		return 0;
+
+	elfd = dwarf_info.elfd;
+
 	while ((scn = elf_nextscn(elfd, scn)) != NULL) {
 		if (gelf_getshdr(scn, &shdr) == NULL) {
 			ERRMSG("Can't get section header.\n");
@@ -1431,8 +1565,7 @@ get_symbol_addr(char *symname)
 		}
 	}
 out:
-	if (elfd != NULL)
-		elf_end(elfd);
+	clean_dwfl_info();
 
 	return symbol;
 }
@@ -1449,18 +1582,12 @@ get_next_symbol_addr(char *symname)
 	Elf_Data *data = NULL;
 	Elf_Scn *scn = NULL;
 	char *sym_name = NULL;
-	const off_t failed = (off_t)-1;
 
-	if (lseek(dwarf_info.fd_debuginfo, 0, SEEK_SET) == failed) {
-		ERRMSG("Can't seek the kernel file(%s). %s\n",
-		    dwarf_info.name_debuginfo, strerror(errno));
-		return NOT_FOUND_SYMBOL;
-	}
-	if (!(elfd = elf_begin(dwarf_info.fd_debuginfo, ELF_C_READ, NULL))) {
-		ERRMSG("Can't get first elf header of %s.\n",
-		    dwarf_info.name_debuginfo);
-		return NOT_FOUND_SYMBOL;
-	}
+	if (!init_dwarf_info())
+		return 0;
+
+	elfd = dwarf_info.elfd;
+
 	while ((scn = elf_nextscn(elfd, scn)) != NULL) {
 		if (gelf_getshdr(scn, &shdr) == NULL) {
 			ERRMSG("Can't get section header.\n");
@@ -1522,8 +1649,7 @@ get_next_symbol_addr(char *symname)
 		}
 	}
 out:
-	if (elfd != NULL)
-		elf_end(elfd);
+	clean_dwfl_info();
 
 	return next_symbol;
 }
@@ -2034,24 +2160,15 @@ get_debug_info(void)
 	Elf_Scn *scn = NULL;
 	GElf_Shdr scnhdr_mem, *scnhdr = NULL;
 	Dwarf_Die cu_die;
-	const off_t failed = (off_t)-1;
 
 	int ret = FALSE;
 
-	if (lseek(dwarf_info.fd_debuginfo, 0, SEEK_SET) == failed) {
-		ERRMSG("Can't seek the kernel file(%s). %s\n",
-		    dwarf_info.name_debuginfo, strerror(errno));
-		return FALSE;
-	}
-	if (!(elfd = elf_begin(dwarf_info.fd_debuginfo, ELF_C_READ_MMAP, NULL))) {
-		ERRMSG("Can't get first elf header of %s.\n",
-		    dwarf_info.name_debuginfo);
+	if (!init_dwarf_info())
 		return FALSE;
-	}
-	if (!(dwarfd = dwarf_begin_elf(elfd, DWARF_C_READ, NULL))) {
-		ERRMSG("Can't create a handle for a new debug session.\n");
-		goto out;
-	}
+
+	elfd = dwarf_info.elfd;
+	dwarfd = dwarf_info.dwarfd;
+
 	if (elf_getshstrndx(elfd, &shstrndx) < 0) {
 		ERRMSG("Can't get the section index of the string table.\n");
 		goto out;
@@ -2088,10 +2205,7 @@ get_debug_info(void)
 	}
 	ret = TRUE;
 out:
-	if (dwarfd != NULL)
-		dwarf_end(dwarfd);
-	if (elfd != NULL)
-		elf_end(elfd);
+	clean_dwfl_info();
 
 	return ret;
 }
@@ -2448,14 +2562,58 @@ get_mem_type(void)
 	return ret;
 }
 
+/*
+ * Set the dwarf_info with kernel/module debuginfo file information.
+ */
+int
+set_dwarf_debuginfo(char *mod_name, char *name_debuginfo, int fd_debuginfo)
+{
+	if (!mod_name)
+		return FALSE;
+	if (dwarf_info.module_name && !strcmp(dwarf_info.module_name, mod_name))
+		return TRUE;
+
+	/* Switching to different module.
+	 *
+	 * Close the file descriptor if previous module is != kernel and
+	 * xen-syms. The reason is, vmlinux file will always be supplied
+	 * by user and code to open/close kernel debuginfo file already
+	 * in place. The module debuginfo files are opened only if '--config'
+	 * option is used. This helps not to break the existing functionlity
+	 * if called without '--config' option.
+	 */
+
+	if (dwarf_info.module_name
+			&& strcmp(dwarf_info.module_name, "vmlinux")
+			&& strcmp(dwarf_info.module_name, "xen-syms")) {
+		if (dwarf_info.fd_debuginfo > 0)
+			close(dwarf_info.fd_debuginfo);
+		if (dwarf_info.name_debuginfo)
+			free(dwarf_info.name_debuginfo);
+	}
+	dwarf_info.fd_debuginfo = fd_debuginfo;
+	dwarf_info.name_debuginfo = name_debuginfo;
+	dwarf_info.module_name = mod_name;
+
+	if (!strcmp(dwarf_info.module_name, "vmlinux") ||
+		!strcmp(dwarf_info.module_name, "xen-syms"))
+		return TRUE;
+
+	/* check to see whether module debuginfo is available */
+	if (!init_dwarf_info())
+		return FALSE;
+	else
+		clean_dwfl_info();
+	return TRUE;
+}
+
 int
 generate_vmcoreinfo(void)
 {
 	if (!set_page_size(sysconf(_SC_PAGE_SIZE)))
 		return FALSE;
 
-	dwarf_info.fd_debuginfo   = info->fd_vmlinux;
-	dwarf_info.name_debuginfo = info->name_vmlinux;
+	set_dwarf_debuginfo("vmlinux", info->name_vmlinux, info->fd_vmlinux);
 
 	if (!get_symbol_info())
 		return FALSE;
@@ -3888,8 +4046,8 @@ initial(void)
 	 * Get the debug information for analysis from the kernel file
 	 */
 	} else if (info->name_vmlinux) {
-		dwarf_info.fd_debuginfo   = info->fd_vmlinux;
-		dwarf_info.name_debuginfo = info->name_vmlinux;
+		set_dwarf_debuginfo("vmlinux", info->name_vmlinux,
+						info->fd_vmlinux);
 
 		if (!get_symbol_info())
 			return FALSE;
@@ -6552,8 +6710,7 @@ generate_vmcoreinfo_xen(void)
 		ERRMSG("Can't get the size of page.\n");
 		return FALSE;
 	}
-	dwarf_info.fd_debuginfo   = info->fd_xen_syms;
-	dwarf_info.name_debuginfo = info->name_xen_syms;
+	set_dwarf_debuginfo("xen-syms", info->name_xen_syms, info->fd_xen_syms);
 
 	if (!get_symbol_info_xen())
 		return FALSE;
@@ -6820,8 +6977,8 @@ initial_xen(void)
 	 * Get the debug information for analysis from the xen-syms file
 	 */
 	} else if (info->name_xen_syms) {
-		dwarf_info.fd_debuginfo   = info->fd_xen_syms;
-		dwarf_info.name_debuginfo = info->name_xen_syms;
+		set_dwarf_debuginfo("xen-syms", info->name_xen_syms,
+							info->fd_xen_syms);
 
 		if (!get_symbol_info_xen())
 			return FALSE;
diff --git a/makedumpfile.h b/makedumpfile.h
index e037f12..3fda754 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -27,6 +27,7 @@
 #include <sys/wait.h>
 #include <zlib.h>
 #include <elfutils/libdw.h>
+#include <elfutils/libdwfl.h>
 #include <libelf.h>
 #include <dwarf.h>
 #include <byteswap.h>
@@ -1177,6 +1178,8 @@ extern struct srcfile_table	srcfile_table;
 /*
  * Debugging information
  */
+#define DEFAULT_DEBUGINFO_PATH	"/usr/lib/debug"
+
 enum {
 	DWARF_INFO_GET_STRUCT_SIZE,
 	DWARF_INFO_GET_MEMBER_OFFSET,
@@ -1194,10 +1197,14 @@ struct dwarf_info {
 	unsigned int	cmd;		/* IN */
 	int	fd_debuginfo;		/* IN */
 	char	*name_debuginfo;	/* IN */
+	char	*module_name;		/* IN */
 	char	*struct_name;		/* IN */
 	char	*symbol_name;		/* IN */
 	char	*member_name;		/* IN */
 	char	*enum_name;		/* IN */
+	Elf	*elfd;			/* OUT */
+	Dwarf	*dwarfd;		/* OUT */
+	Dwfl	*dwfl;			/* OUT */
 	long	struct_size;		/* OUT */
 	long	member_offset;		/* OUT */
 	long	array_length;		/* OUT */



From mahesh at linux.vnet.ibm.com  Tue May 17 16:01:53 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:31:53 +0530
Subject: [PATCH v2 3/8] makedumpfile: Load the module symbol data from vmcore.
Message-ID: <20110517200143.12740.52724.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

This patch enables makedumpfile to load module symbol data from vmcore. This
info is required during kernel module data filtering process. Traverse the
modules list and load all module's symbol info data in the memory for fast
lookup.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
---

 makedumpfile.c |  305 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.h |   52 ++++++++++
 2 files changed, 357 insertions(+), 0 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index 2274b18..7601e3d 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -26,6 +26,7 @@ struct srcfile_table	srcfile_table;
 struct dwarf_info	dwarf_info;
 struct vm_table		vt = { 0 };
 struct DumpInfo		*info = NULL;
+struct module_sym_table	mod_st = { 0 };
 
 char filename_stdout[] = FILENAME_STDOUT;
 int message_level;
@@ -1512,6 +1513,54 @@ err_out:
 	return FALSE;
 }
 
+static struct module_info *
+get_loaded_module(char *mod_name)
+{
+	unsigned int i;
+	struct module_info *modules;
+
+	modules = mod_st.modules;
+	if (strcmp(mod_name, modules[mod_st.current_mod].name)) {
+		for (i = 0; i < mod_st.num_modules; i++) {
+			if (!strcmp(mod_name, modules[i].name))
+				break;
+		}
+		if (i == mod_st.num_modules)
+			return NULL;
+		/* set the current_mod for fast lookup next time */
+		mod_st.current_mod = i;
+	}
+
+	return &modules[mod_st.current_mod];
+}
+
+int
+sym_in_module(char *symname, unsigned long long *symbol_addr)
+{
+	int i;
+	struct module_info *module_ptr;
+	struct symbol_info *sym_info;
+
+	if (!mod_st.num_modules
+		|| !strcmp(dwarf_info.module_name, "vmlinux")
+		|| !strcmp(dwarf_info.module_name, "xen-syms"))
+		return FALSE;
+
+	module_ptr = get_loaded_module(dwarf_info.module_name);
+	if (!module_ptr)
+		return FALSE;
+	sym_info = module_ptr->sym_info;
+	if (!sym_info)
+		return FALSE;
+	for (i = 1; i < module_ptr->num_syms; i++) {
+		if (sym_info[i].name && !strcmp(sym_info[i].name, symname)) {
+			*symbol_addr = sym_info[i].value;
+			return TRUE;
+		}
+	}
+	return FALSE;
+}
+
 unsigned long long
 get_symbol_addr(char *symname)
 {
@@ -1524,6 +1573,13 @@ get_symbol_addr(char *symname)
 	Elf_Scn *scn = NULL;
 	char *sym_name = NULL;
 
+	/*
+	 * If we are looking for module symbol then traverse through
+	 * mod_st.modules for symbol lookup
+	 */
+	if (sym_in_module(symname, &symbol))
+		return symbol;
+
 	if (!init_dwarf_info())
 		return 0;
 
@@ -2335,6 +2391,7 @@ get_symbol_info(void)
 	SYMBOL_INIT(log_buf_len, "log_buf_len");
 	SYMBOL_INIT(log_end, "log_end");
 	SYMBOL_INIT(max_pfn, "max_pfn");
+	SYMBOL_INIT(modules, "modules");
 
 	if (SYMBOL(node_data) != NOT_FOUND_SYMBOL)
 		SYMBOL_ARRAY_TYPE_INIT(node_data, "node_data");
@@ -2437,6 +2494,20 @@ get_structure_info(void)
 
 	OFFSET_INIT(vm_struct.addr, "vm_struct", "addr");
 
+	/*
+	 * Get offset of the module members.
+	 */
+	SIZE_INIT(module, "module");
+	OFFSET_INIT(module.strtab, "module", "strtab");
+	OFFSET_INIT(module.symtab, "module", "symtab");
+	OFFSET_INIT(module.num_symtab, "module", "num_symtab");
+	OFFSET_INIT(module.list, "module", "list");
+	OFFSET_INIT(module.name, "module", "name");
+	OFFSET_INIT(module.module_core, "module", "module_core");
+	OFFSET_INIT(module.core_size, "module", "core_size");
+	OFFSET_INIT(module.module_init, "module", "module_init");
+	OFFSET_INIT(module.init_size, "module", "init_size");
+
 	ENUM_NUMBER_INIT(NR_FREE_PAGES, "NR_FREE_PAGES");
 	ENUM_NUMBER_INIT(N_ONLINE, "N_ONLINE");
 
@@ -7254,6 +7325,240 @@ writeout_multiple_dumpfiles(void)
 	return ret;
 }
 
+static unsigned int
+get_num_modules(unsigned long head, unsigned int *num)
+{
+	unsigned long cur;
+	unsigned int num_modules = 0;
+
+	if (!num)
+		return FALSE;
+
+	if (!readmem(VADDR, head + OFFSET(list_head.next), &cur, sizeof cur)) {
+		ERRMSG("Can't get next list_head.\n");
+		return FALSE;
+	}
+	while (cur != head) {
+		num_modules++;
+		if (!readmem(VADDR, cur + OFFSET(list_head.next),
+					&cur, sizeof cur)) {
+			ERRMSG("Can't get next list_head.\n");
+			return FALSE;
+		}
+	}
+	*num = num_modules;
+	return TRUE;
+}
+
+static void
+free_symbol_info(struct module_info *module)
+{
+	int i;
+
+	if (module->num_syms) {
+		for (i = 1; i < module->num_syms; i++)
+			if (module->sym_info[i].name)
+				free(module->sym_info[i].name);
+		free(module->sym_info);
+	}
+}
+
+static void
+clean_module_symbols(void)
+{
+	int i;
+
+	for (i = 0; i < mod_st.num_modules; i++)
+		free_symbol_info(&mod_st.modules[i]);
+
+	if (mod_st.num_modules) {
+		free(mod_st.modules);
+		mod_st.modules = NULL;
+		mod_st.num_modules = 0;
+	}
+}
+
+static int
+load_module_symbols(void)
+{
+	unsigned long head, cur, cur_module;
+	unsigned long symtab, strtab;
+	unsigned long mod_base, mod_init;
+	unsigned int mod_size, mod_init_size;
+	unsigned char *module_struct_mem, *module_core_mem;
+	unsigned char *module_init_mem = NULL;
+	unsigned char *symtab_mem;
+	char *module_name, *strtab_mem, *nameptr;
+	struct module_info *modules = NULL;
+	struct symbol_info *sym_info;
+	unsigned int num_symtab;
+	unsigned int i = 0, nsym;
+
+	head = SYMBOL(modules);
+	if (!get_num_modules(head, &mod_st.num_modules)) {
+		ERRMSG("Can't get module count\n");
+		return FALSE;
+	}
+	if (!mod_st.num_modules) {
+		return FALSE;
+	}
+	mod_st.modules = calloc(mod_st.num_modules,
+					sizeof(struct module_info));
+	if (!mod_st.modules) {
+		ERRMSG("Can't allocate memory for module info\n");
+		return FALSE;
+	}
+	modules = mod_st.modules;
+
+	/* Allocate buffer to read struct module data from vmcore. */
+	if ((module_struct_mem = calloc(1, SIZE(module))) == NULL) {
+		ERRMSG("Failed to allocate buffer for module\n");
+		return FALSE;
+	}
+
+	if (!readmem(VADDR, head + OFFSET(list_head.next), &cur, sizeof cur)) {
+		ERRMSG("Can't get next list_head.\n");
+		return FALSE;
+	}
+
+	/* Travese the list and read module symbols */
+	while (cur != head) {
+		cur_module = cur - OFFSET(module.list);
+		if (!readmem(VADDR, cur_module, module_struct_mem,
+							SIZE(module))) {
+			ERRMSG("Can't get module info.\n");
+			return FALSE;
+		}
+
+		module_name = (char *)(module_struct_mem + OFFSET(module.name));
+		if (strlen(module_name) < MOD_NAME_LEN)
+			strcpy(modules[i].name, module_name);
+		else
+			strncpy(modules[i].name, module_name, MOD_NAME_LEN-1);
+
+		mod_init = ULONG(module_struct_mem +
+						OFFSET(module.module_init));
+		mod_init_size = UINT(module_struct_mem +
+						OFFSET(module.init_size));
+		mod_base = ULONG(module_struct_mem +
+						OFFSET(module.module_core));
+		mod_size = UINT(module_struct_mem +
+						OFFSET(module.core_size));
+
+		DEBUG_MSG("Module: %s, Base: 0x%lx, Size: %u\n",
+				module_name, mod_base, mod_size);
+		if (mod_init_size > 0) {
+			module_init_mem = calloc(1, mod_init_size);
+			if (module_init_mem == NULL) {
+				ERRMSG("Can't allocate memory for module "
+								"init\n");
+				return FALSE;
+			}
+			if (!readmem(VADDR, mod_init, module_init_mem,
+							mod_init_size)) {
+				ERRMSG("Can't access module init in memory.\n");
+				return FALSE;
+			}
+		}
+
+		if ((module_core_mem = calloc(1, mod_size)) == NULL) {
+			ERRMSG("Can't allocate memory for module\n");
+			return FALSE;
+		}
+		if (!readmem(VADDR, mod_base, module_core_mem, mod_size)) {
+			ERRMSG("Can't access module in memory.\n");
+			return FALSE;
+		}
+
+		num_symtab = UINT(module_struct_mem +
+						OFFSET(module.num_symtab));
+		if (!num_symtab) {
+			ERRMSG("%s: Symbol info not available\n", module_name);
+			return FALSE;
+		}
+		modules[i].num_syms = num_symtab;
+		DEBUG_MSG("num_sym: %d\n", num_symtab);
+
+		symtab = ULONG(module_struct_mem + OFFSET(module.symtab));
+		strtab = ULONG(module_struct_mem + OFFSET(module.strtab));
+
+		/* check if symtab and strtab are inside the module space. */
+		if (!IN_RANGE(symtab, mod_base, mod_size) &&
+			!IN_RANGE(symtab, mod_init, mod_init_size)) {
+			ERRMSG("%s: module symtab is outseide of module "
+				"address space\n", module_name);
+			return FALSE;
+		}
+		if (IN_RANGE(symtab, mod_base, mod_size))
+			symtab_mem = module_core_mem + (symtab - mod_base);
+		else
+			symtab_mem = module_init_mem + (symtab - mod_init);
+
+		if (!IN_RANGE(strtab, mod_base, mod_size) &&
+			!IN_RANGE(strtab, mod_init, mod_init_size)) {
+			ERRMSG("%s: module strtab is outseide of module "
+				"address space\n", module_name);
+			return FALSE;
+		}
+		if (IN_RANGE(strtab, mod_base, mod_size))
+			strtab_mem = (char *)(module_core_mem
+						+ (strtab - mod_base));
+		else
+			strtab_mem = (char *)(module_init_mem
+						+ (strtab - mod_init));
+
+		modules[i].sym_info = calloc(num_symtab,
+						sizeof(struct symbol_info));
+		if (modules[i].sym_info == NULL) {
+			ERRMSG("Can't allocate memory to store sym info\n");
+			return FALSE;
+		}
+		sym_info = modules[i].sym_info;
+
+		/* symbols starts from 1 */
+		for (nsym = 1; nsym < num_symtab; nsym++) {
+			Elf32_Sym *sym32;
+			Elf64_Sym *sym64;
+			/*
+			 * We can not depend on info->flag_elf64_memory if
+			 * the input vmcore file is kdump-compressed format.
+			 * We will fix this in the next patch and remove the
+			 * dependancy on info->flag_elf64_memory.
+			 */
+			if (info->flag_elf64_memory) {
+				sym64 = (Elf64_Sym *) (symtab_mem
+						+ (nsym * sizeof(Elf64_Sym)));
+				sym_info[nsym].value =
+					(unsigned long long) sym64->st_value;
+				nameptr = strtab_mem + sym64->st_name;
+			}
+			else {
+				sym32 = (Elf32_Sym *) (symtab_mem
+						+ (nsym * sizeof(Elf32_Sym)));
+				sym_info[nsym].value =
+					(unsigned long long) sym32->st_value;
+				nameptr = strtab_mem + sym32->st_name;
+			}
+			if (strlen(nameptr))
+				sym_info[nsym].name = strdup(nameptr);
+			DEBUG_MSG("\t[%d] %llx %s\n", nsym,
+						sym_info[nsym].value, nameptr);
+		}
+
+		if (!readmem(VADDR, cur + OFFSET(list_head.next),
+					&cur, sizeof cur)) {
+			ERRMSG("Can't get next list_head.\n");
+			return FALSE;
+		}
+		i++;
+		free(module_core_mem);
+		if (mod_init_size > 0)
+			free(module_init_mem);
+	} while (cur != head);
+	free(module_struct_mem);
+	return TRUE;
+}
+
 int
 create_dumpfile(void)
 {
diff --git a/makedumpfile.h b/makedumpfile.h
index 3fda754..8ad1287 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1012,6 +1012,33 @@ struct vm_table {
 };
 extern struct vm_table		vt;
 
+/*
+ * Loaded module symbols info.
+ */
+#define MOD_NAME_LEN	64
+#define IN_RANGE(addr, mbase, sz) \
+	(((unsigned long)(addr) >= (unsigned long)mbase) \
+	&& ((unsigned long)addr < (unsigned long)(mbase + sz)))
+
+struct symbol_info {
+	char			*name;
+	unsigned long long	value;
+};
+
+struct module_info {
+	char			name[MOD_NAME_LEN];
+	unsigned int		num_syms;
+	struct symbol_info	*sym_info;
+};
+
+struct module_sym_table {
+	unsigned int		num_modules;
+	unsigned int		current_mod;
+	struct module_info	*modules;
+};
+
+extern struct module_sym_table	mod_st;
+
 struct symbol_table {
 	unsigned long long	mem_map;
 	unsigned long long	vmem_map;
@@ -1052,6 +1079,12 @@ struct symbol_table {
 	unsigned long long	frametable_pg_dir;
 	unsigned long long	max_page;
 	unsigned long long	alloc_bitmap;
+
+	/*
+	 * for loading module symbol data
+	 */
+
+	unsigned long long	modules;
 };
 
 struct size_table {
@@ -1069,6 +1102,11 @@ struct size_table {
 	 */
 	long	page_info;
 	long	domain;
+
+	/*
+	 * for loading module symbol data
+	 */
+	long	module;
 };
 
 struct offset_table {
@@ -1123,6 +1161,20 @@ struct offset_table {
 		long	next_in_list;
 	} domain;
 
+	/*
+	 * for loading module symbol data
+	 */
+	struct module {
+		long	list;
+		long	name;
+		long	module_core;
+		long	core_size;
+		long	module_init;
+		long	init_size;
+		long	num_symtab;
+		long	symtab;
+		long	strtab;
+	} module;
 };
 
 /*



From mahesh at linux.vnet.ibm.com  Tue May 17 16:03:22 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:33:22 +0530
Subject: [PATCH v2 4/8] makedumpfile: Introduce routines to get type name from
	debuginfo.
Message-ID: <20110517200247.12740.45844.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

This patch implements following routines that returns type, type name and
size of kernel symbol/member using dwarf info:

	char *get_symbol_type_name();
	char *get_member_type_name();
	long get_pointer_size();

These routines will be used to resolve symbol/member information specified
through filter config commands.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
---

 makedumpfile.c |  142 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 makedumpfile.h |   13 +++++
 2 files changed, 148 insertions(+), 7 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index 7601e3d..f7d37cc 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -30,6 +30,7 @@ struct module_sym_table	mod_st = { 0 };
 
 char filename_stdout[] = FILENAME_STDOUT;
 int message_level;
+long pointer_size;
 
 /*
  * Forward declarations
@@ -1822,6 +1823,41 @@ check_array_type(Dwarf *dwarfd, Dwarf_Die *die)
 	return TRUE;
 }
 
+static int
+get_dwarf_base_type(Dwarf *dwarfd, Dwarf_Die *die)
+{
+	int tag;
+	const char *name;
+
+	while (get_die_type(dwarfd, die, die)) {
+		tag = dwarf_tag(die);
+		switch (tag) {
+		case DW_TAG_array_type:
+			dwarf_info.type_flag |= TYPE_ARRAY;
+			break;
+		case DW_TAG_pointer_type:
+			dwarf_info.type_flag |= TYPE_PTR;
+			break;
+		case DW_TAG_structure_type:
+			dwarf_info.type_flag |= TYPE_STRUCT;
+			break;
+		case DW_TAG_base_type:
+			dwarf_info.type_flag |= TYPE_BASE;
+			break;
+		}
+	}
+
+	name = dwarf_diename(die);
+	if (name)
+		dwarf_info.type_name = strdup(name);
+	else if (dwarf_info.type_flag == TYPE_PTR)
+		dwarf_info.type_name = strdup("void");
+
+	dwarf_info.struct_size = dwarf_bytesize(die);
+
+	return TRUE;
+}
+
 /*
  * Function for searching struct page.union.struct.mapping.
  */
@@ -1902,6 +1938,15 @@ search_member(Dwarf *dwarfd, Dwarf_Die *die)
 			continue;
 
 		switch (dwarf_info.cmd) {
+		case DWARF_INFO_GET_MEMBER_TYPE:
+			if ((!name) || strcmp(name, dwarf_info.member_name))
+				continue;
+			/*
+			 * Get the member offset.
+			 */
+			if (!get_dwarf_base_type(dwarfd, walker))
+				continue;
+			return;
 		case DWARF_INFO_GET_MEMBER_OFFSET:
 			if ((!name) || strcmp(name, dwarf_info.member_name))
 				continue;
@@ -1964,6 +2009,7 @@ is_search_structure(int cmd)
 {
 	if ((cmd == DWARF_INFO_GET_STRUCT_SIZE)
 	    || (cmd == DWARF_INFO_GET_MEMBER_OFFSET)
+	    || (cmd == DWARF_INFO_GET_MEMBER_TYPE)
 	    || (cmd == DWARF_INFO_GET_MEMBER_OFFSET_IN_UNION)
 	    || (cmd == DWARF_INFO_GET_MEMBER_OFFSET_1ST_UNION)
 	    || (cmd == DWARF_INFO_GET_MEMBER_ARRAY_LENGTH))
@@ -1985,6 +2031,7 @@ int
 is_search_symbol(int cmd)
 {
 	if ((cmd == DWARF_INFO_GET_SYMBOL_ARRAY_LENGTH)
+	    || (cmd == DWARF_INFO_GET_SYMBOL_TYPE)
 	    || (cmd == DWARF_INFO_CHECK_SYMBOL_ARRAY_TYPE))
 		return TRUE;
 	else
@@ -2041,6 +2088,7 @@ search_structure(Dwarf *dwarfd, Dwarf_Die *die, int *found)
 	switch (dwarf_info.cmd) {
 	case DWARF_INFO_GET_STRUCT_SIZE:
 		break;
+	case DWARF_INFO_GET_MEMBER_TYPE:
 	case DWARF_INFO_GET_MEMBER_OFFSET:
 	case DWARF_INFO_GET_MEMBER_OFFSET_IN_UNION:
 	case DWARF_INFO_GET_MEMBER_OFFSET_1ST_UNION:
@@ -2173,6 +2221,9 @@ search_symbol(Dwarf *dwarfd, Dwarf_Die *die, int *found)
 	case DWARF_INFO_CHECK_SYMBOL_ARRAY_TYPE:
 		check_array_type(dwarfd, die);
 		break;
+	case DWARF_INFO_GET_SYMBOL_TYPE:
+		get_dwarf_base_type(dwarfd, die);
+		break;
 	}
 }
 
@@ -2249,6 +2300,10 @@ get_debug_info(void)
 	 */
 	while (dwarf_nextcu(dwarfd, off, &next_off, &header_size,
 	    &abbrev_offset, &address_size, &offset_size) == 0) {
+		if (dwarf_info.cmd == DWARF_INFO_GET_PTR_SIZE) {
+			dwarf_info.struct_size = address_size;
+			break;
+		}
 		off += header_size;
 		if (dwarf_offdie(dwarfd, off, &cu_die) == NULL) {
 			ERRMSG("Can't get CU die.\n");
@@ -2287,6 +2342,47 @@ get_structure_size(char *structname, int flag_typedef)
 }
 
 /*
+ * Get the size of pointer.
+ */
+long
+get_pointer_size()
+{
+	dwarf_info.cmd = DWARF_INFO_GET_PTR_SIZE;
+	/* reuse struct_size member to report pointer size */
+	dwarf_info.struct_size = NOT_FOUND_STRUCTURE;
+
+	if (!get_debug_info())
+		return FAILED_DWARFINFO;
+
+	return dwarf_info.struct_size;
+}
+
+/*
+ * Get the type of given symbol.
+ */
+char *
+get_symbol_type_name(char *symname, int cmd, long *size,
+					unsigned long *flag)
+{
+	dwarf_info.cmd = cmd;
+	dwarf_info.symbol_name = symname;
+	dwarf_info.type_name = NULL;
+	dwarf_info.struct_size = NOT_FOUND_STRUCTURE;
+	dwarf_info.type_flag = 0;
+
+	if (!get_debug_info())
+		return NULL;
+
+	if (size)
+		*size = dwarf_info.struct_size;
+
+	if (flag)
+		*flag = dwarf_info.type_flag;
+
+	return dwarf_info.type_name;
+}
+
+/*
  * Get the offset of member.
  */
 long
@@ -2305,6 +2401,35 @@ get_member_offset(char *structname, char *membername, int cmd)
 }
 
 /*
+ * Get the type name and size of member.
+ */
+char *
+get_member_type_name(char *structname, char *membername, int cmd, long *size,
+						unsigned long *flag)
+{
+	dwarf_info.cmd = cmd;
+	dwarf_info.struct_name = structname;
+	dwarf_info.struct_size = NOT_FOUND_STRUCTURE;
+	dwarf_info.member_name = membername;
+	dwarf_info.type_name = NULL;
+	dwarf_info.type_flag = 0;
+
+	if (!get_debug_info())
+		return NULL;
+
+	if (dwarf_info.struct_size == NOT_FOUND_STRUCTURE)
+		return NULL;
+
+	if (size)
+		*size = dwarf_info.struct_size;
+
+	if (flag)
+		*flag = dwarf_info.type_flag;
+
+	return dwarf_info.type_name;
+}
+
+/*
  * Get the length of array.
  */
 long
@@ -2402,6 +2527,8 @@ get_symbol_info(void)
 	if (SYMBOL(node_memblk) != NOT_FOUND_SYMBOL)
 		SYMBOL_ARRAY_LENGTH_INIT(node_memblk, "node_memblk");
 
+	pointer_size = get_pointer_size();
+
 	return TRUE;
 }
 
@@ -7519,13 +7646,16 @@ load_module_symbols(void)
 		for (nsym = 1; nsym < num_symtab; nsym++) {
 			Elf32_Sym *sym32;
 			Elf64_Sym *sym64;
-			/*
-			 * We can not depend on info->flag_elf64_memory if
-			 * the input vmcore file is kdump-compressed format.
-			 * We will fix this in the next patch and remove the
-			 * dependancy on info->flag_elf64_memory.
+			/* If case of ELF vmcore then the word size can be
+			 * determined using info->flag_elf64_memory flag.
+			 * But in case of kdump-compressed dump, kdump header
+			 * does not carry word size info. May be in future
+			 * this info will be available in kdump header.
+			 * Until then, in order to make this logic work on both
+			 * situation we depend on pointer_size that is
+			 * extracted from vmlinux dwarf information.
 			 */
-			if (info->flag_elf64_memory) {
+			if ((pointer_size * 8) == 64) {
 				sym64 = (Elf64_Sym *) (symtab_mem
 						+ (nsym * sizeof(Elf64_Sym)));
 				sym_info[nsym].value =
diff --git a/makedumpfile.h b/makedumpfile.h
index 8ad1287..62c7ff5 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1242,7 +1242,10 @@ enum {
 	DWARF_INFO_GET_TYPEDEF_SIZE,
 	DWARF_INFO_GET_TYPEDEF_SRCNAME,
 	DWARF_INFO_GET_ENUM_NUMBER,
-	DWARF_INFO_CHECK_SYMBOL_ARRAY_TYPE
+	DWARF_INFO_CHECK_SYMBOL_ARRAY_TYPE,
+	DWARF_INFO_GET_SYMBOL_TYPE,
+	DWARF_INFO_GET_MEMBER_TYPE,
+	DWARF_INFO_GET_PTR_SIZE,
 };
 
 struct dwarf_info {
@@ -1257,13 +1260,21 @@ struct dwarf_info {
 	Elf	*elfd;			/* OUT */
 	Dwarf	*dwarfd;		/* OUT */
 	Dwfl	*dwfl;			/* OUT */
+	char	*type_name;		/* OUT */
 	long	struct_size;		/* OUT */
 	long	member_offset;		/* OUT */
 	long	array_length;		/* OUT */
 	long	enum_number;		/* OUT */
+	unsigned char	type_flag;	/* OUT */
 	char	src_name[LEN_SRCFILE];	/* OUT */
 };
 
+/* flags for dwarf_info.type_flag */
+#define TYPE_BASE	0x01
+#define TYPE_ARRAY	0x02
+#define TYPE_PTR	0x04
+#define TYPE_STRUCT	0x08
+
 extern struct dwarf_info	dwarf_info;
 
 int readmem(int type_addr, unsigned long long addr, void *bufptr, size_t size);



From mahesh at linux.vnet.ibm.com  Tue May 17 16:04:15 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:34:15 +0530
Subject: [PATCH v2 5/8] makedumpfile: Read and process filter commands from
	config file.
Message-ID: <20110517200407.12740.95780.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

This patch enables to makedumpfile to read and process filter commands from
specified config file. It builds a list of filter info consisting of memory
address (paddr) and size to be filtered out. This list is used during
read_pfn() function to filter out user specified kernel data from vmcore
before writing it to compressed DUMPFILE. The filtered memory locations are
filled with 'X' (0x58) character.

The filter command syntax is:
	erase <Symbol>[.member[...]] [size <SizeValue>[K|M]]
	erase <Symbol>[.member[...]] [size <SizeSymbol>]
	erase <Symbol>[.member[...]] [nullify]

Below are the examples how filter commands in config file look like:

	erase modules
	erase cred_jar.name size 10
	erase vmlist.addr nullify

This is useful feature for the customers who wants to erase the customer
sensitive data like security keys, in DUMPFILE before sending it to
support team for analysis.

The man page for makedumpfile.conf(8) has been introduced that describes the
config file format and erase command syntax.

Change in v2:
- Renamed filter.conf file to makedumpfile.conf
- Added cleanup routine clear_filter_info() to freeup memory allocated during
  gather_filter_info().
- Modified gather_filter_info() function to return error if
  process_config_file() can't open configuration file.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
---

 Makefile            |    8 
 makedumpfile.c      |  863 +++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.conf   |   82 +++++
 makedumpfile.conf.8 |  213 +++++++++++++
 makedumpfile.h      |   60 ++++
 5 files changed, 1225 insertions(+), 1 deletions(-)
 create mode 100644 makedumpfile.conf
 create mode 100644 makedumpfile.conf.8

diff --git a/Makefile b/Makefile
index 47467e3..240b0f4 100644
--- a/Makefile
+++ b/Makefile
@@ -37,12 +37,18 @@ makedumpfile: $(SRC) $(OBJ_ARCH)
 	grep -v "^.TH MAKEDUMPFILE 8" makedumpfile.8 >> temp.8
 	mv temp.8 makedumpfile.8
 	gzip -c ./makedumpfile.8 > ./makedumpfile.8.gz
+	echo .TH FILTER.CONF 8 \"$(DATE)\" \"makedumpfile v$(VERSION)\" \"Linux System Administrator\'s Manual\" > temp.8
+	grep -v "^.TH FILTER.CONF 8" makedumpfile.conf.8 >> temp.8
+	mv temp.8 makedumpfile.conf.8
+	gzip -c ./makedumpfile.conf.8 > ./makedumpfile.conf.8.gz
 
 clean:
-	rm -f $(OBJ) $(OBJ_ARCH) makedumpfile makedumpfile.8.gz
+	rm -f $(OBJ) $(OBJ_ARCH) makedumpfile makedumpfile.8.gz makedumpfile.conf.8.gz
 
 install:
 	cp makedumpfile ${DESTDIR}/bin
 	cp makedumpfile-R.pl ${DESTDIR}/bin
 	cp makedumpfile.8.gz ${DESTDIR}/usr/share/man/man8
+	cp makedumpfile.conf.8.gz ${DESTDIR}/usr/share/man/man8
+	cp makedumpfile.conf ${DESTDIR}/etc/makedumpfile.conf.sample
 
diff --git a/makedumpfile.c b/makedumpfile.c
index f7d37cc..1c7237a 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -27,10 +27,13 @@ struct dwarf_info	dwarf_info;
 struct vm_table		vt = { 0 };
 struct DumpInfo		*info = NULL;
 struct module_sym_table	mod_st = { 0 };
+struct filter_info	*filter_info = NULL;
+struct filter_config	filter_config;
 
 char filename_stdout[] = FILENAME_STDOUT;
 int message_level;
 long pointer_size;
+char config_buf[BUFSIZE_FGETS];
 
 /*
  * Forward declarations
@@ -6047,6 +6050,78 @@ get_num_dumpable(void)
 	return num_dumpable;
 }
 
+void
+split_filter_info(struct filter_info *prev, unsigned long long next_paddr,
+						size_t size)
+{
+	struct filter_info *new;
+
+	if ((new = calloc(1, sizeof(struct filter_info))) == NULL) {
+		ERRMSG("Can't allocate memory to split filter info\n");
+		return;
+	}
+	new->nullify = prev->nullify;
+	new->paddr = next_paddr;
+	new->size = size;
+	new->next = prev->next;
+	prev->next = new;
+}
+
+int
+extract_filter_info(unsigned long long start_paddr,
+			unsigned long long end_paddr,
+			struct filter_info *fl_info)
+{
+	struct filter_info *fi = filter_info;
+	struct filter_info *prev = NULL;
+	size_t size1, size2;
+
+	if (!fl_info)
+		return FALSE;
+
+	while (fi) {
+		if ((fi->paddr >= start_paddr) && (fi->paddr < end_paddr)) {
+			size1 = end_paddr - fi->paddr;
+			if (fi->size <= size1)
+				break;
+			size2 = fi->size - size1;
+			fi->size = size1;
+			split_filter_info(fi, fi->paddr + size1, size2);
+			break;
+		}
+		prev = fi;
+		fi = fi->next;
+	}
+	if (fi) {
+		*fl_info = *fi;
+		fl_info->next = NULL;
+		/* Delete this node */
+		if (!prev)
+			filter_info = fi->next;
+		else
+			prev->next = fi->next;
+		free(fi);
+		return TRUE;
+	}
+	return FALSE;
+}
+
+void
+filter_data_buffer(unsigned char *buf, unsigned long long paddr,
+					size_t size)
+{
+	struct filter_info fl_info;
+	unsigned char *buf_ptr;
+
+	while (extract_filter_info(paddr, paddr + size, &fl_info)) {
+		buf_ptr = buf + (fl_info.paddr - paddr);
+		if (fl_info.nullify)
+			memset(buf_ptr, 0, fl_info.size);
+		else
+			memset(buf_ptr, 'X', fl_info.size);
+	}
+}
+
 int
 write_elf_load_segment(struct cache_data *cd_page, unsigned long long paddr,
 		       off_t off_memory, long long size)
@@ -6078,6 +6153,8 @@ write_elf_load_segment(struct cache_data *cd_page, unsigned long long paddr,
 			    info->name_memory, strerror(errno));
 			return FALSE;
 		}
+		filter_data_buffer((unsigned char *)buf, paddr, bufsz_write);
+		paddr += bufsz_write;
 		if (!write_cache(cd_page, buf, bufsz_write))
 			return FALSE;
 
@@ -6296,6 +6373,7 @@ read_pfn(unsigned long long pfn, unsigned char *buf)
 			ERRMSG("Can't get the page data.\n");
 			return FALSE;
 		}
+		filter_data_buffer(buf, paddr, info->page_size);
 		return TRUE;
 	}
 
@@ -6318,6 +6396,7 @@ read_pfn(unsigned long long pfn, unsigned char *buf)
 		ERRMSG("Can't get the page data.\n");
 		return FALSE;
 	}
+	filter_data_buffer(buf, paddr, size1);
 	if (size1 != info->page_size) {
 		size2 = info->page_size - size1;
 		if (!offset2) {
@@ -6327,6 +6406,7 @@ read_pfn(unsigned long long pfn, unsigned char *buf)
 				ERRMSG("Can't get the page data.\n");
 				return FALSE;
 			}
+			filter_data_buffer(buf + size1, paddr + size1, size2);
 		}
 	}
 	return TRUE;
@@ -7689,6 +7769,785 @@ load_module_symbols(void)
 	return TRUE;
 }
 
+void
+free_config_entry(struct config_entry *ce)
+{
+	struct config_entry *p;
+
+	while(ce) {
+		p = ce;
+		ce = p->next;
+		if (p->name)
+			free(p->name);
+		if (p->type_name)
+			free(p->type_name);
+		free(p);
+	}
+}
+
+void
+free_config(struct config *config)
+{
+	if (config) {
+		if (config->module_name)
+			free(config->module_name);
+		if (config->filter_symbol)
+			free_config_entry(config->filter_symbol);
+		if (config->size_symbol)
+			free_config_entry(config->size_symbol);
+		free(config);
+	}
+}
+
+void
+print_config_entry(struct config_entry *ce)
+{
+	while (ce) {
+		DEBUG_MSG("Name: %s\n", ce->name);
+		DEBUG_MSG("Type Name: %s, ", ce->type_name);
+		DEBUG_MSG("flag: %x, ", ce->flag);
+		DEBUG_MSG("Type flag: %lx, ", ce->type_flag);
+		DEBUG_MSG("sym_addr: %llx, ", ce->sym_addr);
+		DEBUG_MSG("addr: %llx, ", ce->addr);
+		DEBUG_MSG("offset: %lx, ", ce->offset);
+		DEBUG_MSG("size: %zd\n", ce->size);
+
+		ce = ce->next;
+	}
+}
+
+/*
+ * Read the non-terminal's which are in the form of <Symbol>[.member[...]]
+ */
+struct config_entry *
+create_config_entry(const char *token, unsigned short flag, int line)
+{
+	struct config_entry *ce = NULL, *ptr, *prev_ce;
+	char *str, *cur, *next;
+	long len;
+	int depth = 0;
+
+	if (!token)
+		return NULL;
+
+	cur = str = strdup(token);
+	prev_ce = ptr = NULL;
+	while (cur != NULL) {
+		if ((next = strchr(cur, '.')) != NULL) {
+			*next++ = '\0';
+		}
+		if (!strlen(cur)) {
+			cur = next;
+			continue;
+		}
+
+		if ((ptr = calloc(1, sizeof(struct config_entry))) == NULL) {
+			ERRMSG("Can't allocate memory for config_entry\n");
+			goto err_out;
+		}
+		ptr->line = line;
+		ptr->flag |= flag;
+		if (depth == 0) {
+			/* First node is always a symbol name */
+			ptr->flag |= SYMBOL_ENTRY;
+		}
+		if (flag & FILTER_ENTRY) {
+			ptr->name = strdup(cur);
+		}
+		if (flag & SIZE_ENTRY) {
+			char ch = '\0';
+			int n = 0;
+			/* See if absolute length is provided */
+			if ((depth == 0) &&
+				((n = sscanf(cur, "%zd%c", &len, &ch)) > 0)) {
+				if (len < 0) {
+					ERRMSG("Config error at %d: size "
+						"value must be positive.\n",
+						line);
+					goto err_out;
+				}
+				ptr->size = len;
+				ptr->flag |= ENTRY_RESOLVED;
+				if (n == 2) {
+					/* Handle suffix.
+					 * K = Kilobytes
+					 * M = Megabytes
+					 */
+					switch (ch) {
+					case 'M':
+					case 'm':
+						ptr->size *= 1024;
+					case 'K':
+					case 'k':
+						ptr->size *= 1024;
+						break;
+					}
+				}
+			}
+			else
+				ptr->name = strdup(cur);
+		}
+		if (prev_ce) {
+			prev_ce->next = ptr;
+			prev_ce = ptr;
+		}
+		else
+			ce = prev_ce = ptr;
+		cur = next;
+		depth++;
+		ptr = NULL;
+	}
+	free(str);
+	return ce;
+
+err_out:
+	if (ce)
+		free_config_entry(ce);
+	if (ptr)
+		free_config_entry(ptr);
+	return NULL;
+}
+
+int
+is_module_loaded(char *mod_name)
+{
+	if (!strcmp(mod_name, "vmlinux") || get_loaded_module(mod_name))
+		return TRUE;
+	return FALSE;
+}
+
+/*
+ * read filter config file and return each string token. If the parameter
+ * expected_token is non-NULL, then return the current token if it matches
+ * with expected_token otherwise save the current token and return NULL.
+ * At start of every module section filter_config.new_section is set to 1 and
+ * subsequent function invocations return NULL untill filter_config.new_section
+ * is reset to 0 by passing @flag = CONFIG_NEW_CMD (0x02).
+ *
+ * Parameters:
+ * @expected_token	INPUT
+ *	Token string to match with currnet token.
+ *	=NULL - return the current available token.
+ *
+ * @flag		INPUT
+ *	=0x01 - Skip to next module section.
+ *	=0x02 - Treat the next token as next filter command and reset.
+ *
+ * @line		OUTPUT
+ *	Line number of current token in filter config file.
+ *
+ * @cur_mod		OUTPUT
+ *	Points to current module section name on non-NULL return value.
+ *
+ * @eof			OUTPUT
+ *	set to -1 when end of file is reached.
+ *	set to -2 when end of section is reached.
+ */
+static char *
+get_config_token(char *expected_token, unsigned char flag, int *line,
+			char **cur_mod, int *eof)
+{
+	char *p;
+	struct filter_config *fc = &filter_config;
+	int skip = flag & CONFIG_SKIP_SECTION;
+
+	if (!fc->file_filterconfig)
+		return NULL;
+
+	if (eof)
+		*eof = 0;
+
+	/*
+	 * set token and saved_token to NULL if skip module section is set
+	 * to 1.
+	 */
+	if (skip) {
+		fc->token = NULL;
+		fc->saved_token = NULL;
+	}
+
+	if (fc->saved_token) {
+		fc->token = fc->saved_token;
+		fc->saved_token = NULL;
+	}
+	else if (fc->token)
+		fc->token = strtok(NULL, " ");
+
+	/* Read next line if we are done all tokens from previous line */
+	while (!fc->token && fgets(config_buf, sizeof(config_buf),
+					fc->file_filterconfig)) {
+		if ((p = strchr(config_buf, '\n'))) {
+			*p = '\0';
+			fc->line_count++;
+		}
+		if ((p = strchr(config_buf, '#'))) {
+			*p = '\0';
+		}
+		/* replace all tabs with spaces */
+		for (p = config_buf; *p != '\0'; p++)
+			if (*p == '\t')
+				*p = ' ';
+		if (config_buf[0] == '[') {
+			/* module section entry */
+			p = strchr(config_buf, ']');
+			if (!p) {
+				ERRMSG("Config error at %d: Invalid module "
+					"section entry.\n", fc->line_count);
+				/* skip to next valid module section */
+				skip = 1;
+			}
+			else {
+				/*
+				 * Found the valid module section. Reset the
+				 * skip flag.
+				 */
+				*p = '\0';
+				if (fc->cur_module)
+					free(fc->cur_module);
+				fc->cur_module = strdup(&config_buf[1]);
+				skip = 0;
+				fc->new_section = 1;
+			}
+			continue;
+		}
+		/*
+		 * If symbol info for current module is not loaded then
+		 * skip to next module section.
+		 */
+		if (skip ||
+			(fc->cur_module && !is_module_loaded(fc->cur_module)))
+			continue;
+
+		fc->token = strtok(config_buf, " ");
+	}
+	if (!fc->token) {
+		if (eof)
+			*eof = -1;
+		return NULL;
+	}
+	if (fc->new_section && !(flag & CONFIG_NEW_CMD)) {
+		fc->saved_token = fc->token;
+		if (eof)
+			*eof = -2;
+		return NULL;
+	}
+	else
+		fc->new_section = 0;
+
+	if (cur_mod)
+		*cur_mod = fc->cur_module;
+
+	if (line)
+		*line = fc->line_count;
+
+	if (expected_token && strcmp(fc->token, expected_token)) {
+		fc->saved_token = fc->token;
+		return NULL;
+	}
+	return fc->token;
+}
+
+static int
+read_size_entry(struct config *config, int line)
+{
+	char *token = get_config_token(NULL, 0, &line, NULL, NULL);
+
+	if (!token || IS_KEYWORD(token)) {
+		ERRMSG("Config error at %d: expected size symbol after"
+		" 'size' keyword.\n", line);
+		return FALSE;
+	}
+	config->size_symbol = create_config_entry(token, SIZE_ENTRY, line);
+	if (!config->size_symbol) {
+		ERRMSG("Error at line %d: Failed to read size symbol\n",
+									line);
+		return FALSE;
+	}
+	return TRUE;
+}
+
+/*
+ * Read erase command entry. The erase command syntax is:
+ *
+ *	erase <Symbol>[.member[...]] [size <SizeValue>[K|M]]
+ *	erase <Symbol>[.member[...]] [size <SizeSymbol>]
+ *	erase <Symbol>[.member[...]] [nullify]
+ */
+static int
+read_filter_entry(struct config *config, int line)
+{
+	char *token = get_config_token(NULL, 0, &line, NULL, NULL);
+
+	if (!token || IS_KEYWORD(token)) {
+		ERRMSG("Config error at %d: expected kernel symbol after"
+		" 'erase' command.\n", line);
+		return FALSE;
+	}
+	config->filter_symbol =
+			create_config_entry(token, FILTER_ENTRY, line);
+	if (!config->filter_symbol) {
+		ERRMSG("Error at line %d: Failed to read filter symbol\n",
+									line);
+		return FALSE;
+	}
+	if (get_config_token("nullify", 0, &line, NULL, NULL)) {
+		config->filter_symbol->nullify = 1;
+	}
+	else if (get_config_token("size", 0, &line, NULL, NULL)) {
+		if (!read_size_entry(config, line))
+			return FALSE;
+	}
+	return TRUE;
+}
+
+/*
+ * Configuration file 'makedumpfile.conf' contains filter commands.
+ * Every individual filter command is considered as a config entry. A config
+ * entry can be provided on a single line or multiple lines.
+ */
+struct config *
+get_config(int skip)
+{
+	struct config *config;
+	char *token = NULL;
+	static int line_count = 0;
+	char *cur_module = NULL;
+	int eof = 0;
+	unsigned char flag = CONFIG_NEW_CMD | skip;
+
+	if ((config = calloc(1, sizeof(struct config))) == NULL)
+		return NULL;
+
+	if (get_config_token("erase", flag, &line_count, &cur_module, &eof)) {
+		if (cur_module)
+			config->module_name = strdup(cur_module);
+
+		if (!read_filter_entry(config, line_count))
+			goto err_out;
+	}
+	else {
+		if (!eof) {
+			token = get_config_token(NULL, 0, &line_count,
+								NULL, NULL);
+			ERRMSG("Config error at %d: Invalid token '%s'.\n",
+							line_count, token);
+		}
+		goto err_out;
+	}
+	return config;
+err_out:
+	if (config)
+		free_config(config);
+	return NULL;
+}
+
+static unsigned long long
+read_pointer_value(unsigned long long addr)
+{
+	uint32_t val_32;
+	uint64_t val_64;
+
+	switch (pointer_size) {
+	case 4:
+		if (!readmem(VADDR, addr, &val_32, sizeof(val_32))) {
+			ERRMSG("Can't read pointer value\n");
+			return 0;
+		}
+		return (unsigned long long)val_32;
+		break;
+	case 8:
+		if (!readmem(VADDR, addr, &val_64, sizeof(val_64))) {
+			ERRMSG("Can't read pointer value\n");
+			return 0;
+		}
+		return (unsigned long long)val_64;
+		break;
+	}
+	return 0;
+}
+
+int
+resolve_config_entry(struct config_entry *ce, unsigned long long base_addr,
+						char *base_struct_name)
+{
+	char buf[BUFSIZE + 1];
+
+	if (ce->flag & SYMBOL_ENTRY) {
+		/* find the symbol info */
+		if (!ce->name)
+			return FALSE;
+
+		ce->sym_addr = get_symbol_addr(ce->name);
+		if (!ce->sym_addr) {
+			ERRMSG("Config error at %d: Can't find symbol '%s'.\n",
+							ce->line, ce->name);
+			return FALSE;
+		}
+		ce->type_name = get_symbol_type_name(ce->name,
+					DWARF_INFO_GET_SYMBOL_TYPE,
+					&ce->size, &ce->type_flag);
+		if (ce->type_flag & TYPE_ARRAY) {
+			ce->array_length = get_array_length(ce->name, NULL,
+					DWARF_INFO_GET_SYMBOL_ARRAY_LENGTH);
+			if (ce->array_length < 0)
+				ce->array_length = 0;
+		}
+	}
+	else {
+		/* find the member offset */
+		ce->offset = get_member_offset(base_struct_name,
+				ce->name, DWARF_INFO_GET_MEMBER_OFFSET);
+		ce->sym_addr = base_addr + ce->offset;
+		ce->type_name = get_member_type_name(base_struct_name,
+				ce->name, DWARF_INFO_GET_MEMBER_TYPE,
+				&ce->size, &ce->type_flag);
+		if (ce->type_flag & TYPE_ARRAY) {
+			ce->array_length = get_array_length(base_struct_name,
+					ce->name,
+					DWARF_INFO_GET_MEMBER_ARRAY_LENGTH);
+			if (ce->array_length < 0)
+				ce->array_length = 0;
+		}
+	}
+	if (ce->type_name == NULL) {
+		if (!(ce->flag & SYMBOL_ENTRY))
+			ERRMSG("Config error at %d: struct '%s' has no member"
+				" with name '%s'.\n",
+				ce->line, base_struct_name, ce->name);
+		return FALSE;
+	}
+	ce->addr = ce->sym_addr;
+	if (ce->size < 0)
+		ce->size = 0;
+	if ((ce->type_flag & (TYPE_ARRAY | TYPE_PTR)) == TYPE_PTR) {
+		/* If it's a pointer variable (not array) then read the
+		 * pointer value. */
+		ce->addr = read_pointer_value(ce->sym_addr);
+
+		/*
+		 * if it is a void pointer then reset the size to 0
+		 * User need to provide a size to filter data referenced
+		 * by 'void *' pointer.
+		 */
+		if (!strcmp(ce->type_name, "void"))
+			ce->size = 0;
+
+	}
+	if ((ce->type_flag & TYPE_BASE) && (ce->type_flag & TYPE_PTR)) {
+		/*
+		 * Determine the string length for 'char' pointer.
+		 * BUFSIZE(1024) is the upper limit for string length.
+		 */
+		if (!strcmp(ce->type_name, "char")) {
+			if (readmem(VADDR, ce->addr, buf, BUFSIZE)) {
+				buf[BUFSIZE] = '\0';
+				ce->size = strlen(buf);
+			}
+		}
+	}
+	if (!ce->next && (ce->flag & SIZE_ENTRY)) {
+		void *val;
+
+		/* leaf node of size entry */
+		/* If it is size argument then update the size with data
+		 * value of this symbol/member.
+		 * Check if current symbol/member is of base data type.
+		 */
+
+		if (((ce->type_flag & (TYPE_ARRAY | TYPE_BASE)) != TYPE_BASE)
+				|| (ce->size > sizeof(long))) {
+			ERRMSG("Config error at %d: size symbol/member '%s' "
+				"is not of base type.\n", ce->line, ce->name);
+			return FALSE;
+		}
+		if ((val = calloc(1, ce->size)) == NULL) {
+			ERRMSG("Can't get memory for size parameter\n");
+			return FALSE;
+		}
+
+		if (!readmem(VADDR, ce->addr, val, ce->size)) {
+			ERRMSG("Can't read symbol/member data value\n");
+			return FALSE;
+		}
+		switch (ce->size) {
+		case 1:
+			ce->size = (long)(*((uint8_t *)val));
+			break;
+		case 2:
+			ce->size = (long)(*((uint16_t *)val));
+			break;
+		case 4:
+			ce->size = (long)(*((uint32_t *)val));
+			break;
+		case 8:
+			ce->size = (long)(*((uint64_t *)val));
+			break;
+		}
+		free(val);
+	}
+	ce->flag |= ENTRY_RESOLVED;
+	return TRUE;
+}
+
+unsigned long long
+get_config_symbol_addr(struct config_entry *ce,
+			unsigned long long base_addr,
+			char *base_struct_name)
+{
+	if (!(ce->flag & ENTRY_RESOLVED)) {
+		if (!resolve_config_entry(ce, base_addr, base_struct_name))
+			return 0;
+	}
+
+	if (ce->next && ce->addr) {
+		/* Populate nullify flag down the list */
+		ce->next->nullify = ce->nullify;
+		return get_config_symbol_addr(ce->next, ce->addr,
+							ce->type_name);
+	}
+	else if (ce->nullify) {
+		/* nullify is applicable to pointer type */
+		if (ce->type_flag & TYPE_PTR)
+			return ce->sym_addr;
+		else
+			return 0;
+	}
+	else
+		return ce->addr;
+}
+
+long
+get_config_symbol_size(struct config_entry *ce,
+			unsigned long long base_addr,
+			char *base_struct_name)
+{
+	if (!(ce->flag & ENTRY_RESOLVED)) {
+		if (!resolve_config_entry(ce, base_addr, base_struct_name))
+			return 0;
+	}
+
+	if (ce->next && ce->addr)
+		return get_config_symbol_size(ce->next, ce->addr,
+							ce->type_name);
+	else {
+		if (ce->type_flag & TYPE_ARRAY) {
+			if (ce->type_flag & TYPE_PTR)
+				return ce->array_length * pointer_size;
+			else
+				return ce->array_length * ce->size;
+		}
+		return ce->size;
+	}
+}
+
+/*
+ * Insert the filter info node using insertion sort.
+ * If filter node for a given paddr is aready present then update the size
+ * and delete the fl_info node passed.
+ */
+void
+insert_filter_info(struct filter_info *fl_info)
+{
+	struct filter_info *prev = NULL;
+	struct filter_info *ptr = filter_info;
+
+	if (!ptr) {
+		filter_info = fl_info;
+		return;
+	}
+
+	while (ptr) {
+		if (fl_info->paddr <= ptr->paddr)
+			break;
+		prev = ptr;
+		ptr = ptr->next;
+	}
+	if (ptr && (fl_info->paddr == ptr->paddr)) {
+		if (fl_info->size > ptr->size)
+			ptr->size = fl_info->size;
+		free(fl_info);
+		return;
+	}
+
+	if (prev) {
+		fl_info->next = ptr;
+		prev->next = fl_info;
+	}
+	else {
+		fl_info->next = filter_info;
+		filter_info = fl_info;
+	}
+	return;
+}
+
+int
+update_filter_info(struct config_entry *filter_symbol,
+			struct config_entry *size_symbol)
+{
+	unsigned long long addr;
+	long size;
+	struct filter_info *fl_info;
+
+	addr = get_config_symbol_addr(filter_symbol, 0, NULL);
+	if (message_level & ML_PRINT_DEBUG_MSG)
+		print_config_entry(filter_symbol);
+	if (!addr)
+		return FALSE;
+
+	if (filter_symbol->nullify)
+		size = pointer_size;
+	else if (size_symbol) {
+		size = get_config_symbol_size(size_symbol, 0, NULL);
+		if (message_level & ML_PRINT_DEBUG_MSG)
+			print_config_entry(size_symbol);
+	}
+	else
+		size = get_config_symbol_size(filter_symbol, 0, NULL);
+
+	if (size <= 0)
+		return FALSE;
+
+	if ((fl_info = calloc(1, sizeof(struct filter_info))) == NULL) {
+		ERRMSG("Can't allocate filter info\n");
+		return FALSE;
+	}
+	fl_info->address = addr;
+	fl_info->paddr = vaddr_to_paddr(addr);
+	fl_info->size = size;
+	fl_info->nullify = filter_symbol->nullify;
+
+	insert_filter_info(fl_info);
+	return TRUE;
+}
+
+/*
+ * Process the config entry that has been read by get_config.
+ * return TRUE on success
+ */
+int
+process_config(struct config *config)
+{
+	update_filter_info(config->filter_symbol, config->size_symbol);
+
+	return TRUE;
+}
+
+void
+print_filter_info()
+{
+	struct filter_info *fl_info = filter_info;
+
+	DEBUG_MSG("\n");
+	while (fl_info) {
+		DEBUG_MSG("filter address: paddr (%llx), sym_addr (%llx),"
+			" Size (%ld)\n",
+			fl_info->paddr, fl_info->address, fl_info->size);
+		fl_info = fl_info->next;
+	}
+}
+
+void
+init_filter_config()
+{
+	filter_config.name_filterconfig = info->name_filterconfig;
+	filter_config.file_filterconfig = info->file_filterconfig;
+	filter_config.saved_token = NULL;
+	filter_config.token = NULL;
+	filter_config.cur_module = NULL;
+	filter_config.new_section = 0;
+	filter_config.line_count = 0;
+}
+
+/*
+ * Read and process each config entry (filter commands) from filter config
+ * file. If no module debuginfo found for specified module section then skip
+ * to next module section.
+ */
+int
+process_config_file(const char *name_config)
+{
+	struct config *config;
+	int skip_section = 0;
+
+	if (!name_config)
+		return FALSE;
+
+	if ((info->file_filterconfig = fopen(name_config, "r")) == NULL) {
+		ERRMSG("Can't open config file(%s). %s\n",
+		    name_config, strerror(errno));
+		return FALSE;
+	}
+
+	init_filter_config();
+
+	while((config = get_config(skip_section)) != NULL) {
+		skip_section = 0;
+		if (config->module_name &&
+				strcmp(config->module_name, "vmlinux")) {
+			/*
+			 * if Module debuginfo is not available, then skip to
+			 * next module section.
+			 */
+			if (!set_dwarf_debuginfo(config->module_name,
+								NULL, -1)) {
+				ERRMSG("Skipping to next Module section\n");
+				skip_section = 1;
+				free_config(config);
+				continue;
+			}
+		}
+		else {
+			set_dwarf_debuginfo("vmlinux", info->name_vmlinux,
+							info->fd_vmlinux);
+		}
+		process_config(config);
+		free_config(config);
+	}
+
+	fclose(info->file_filterconfig);
+	print_filter_info();
+	return TRUE;
+}
+
+int
+gather_filter_info()
+{
+	int ret;
+
+	/*
+	 * Before processing filter config file, load the symbol data of
+	 * loaded modules from vmcore.
+	 */
+	set_dwarf_debuginfo("vmlinux", info->name_vmlinux, info->fd_vmlinux);
+	if (!load_module_symbols())
+		return FALSE;
+
+	ret = process_config_file(info->name_filterconfig);
+
+	/*
+	 * Remove modules symbol information, we dont need now.
+	 * Reset the dwarf debuginfo to vmlinux to close open file
+	 * descripter of module debuginfo file, if any.
+	 */
+	clean_module_symbols();
+	set_dwarf_debuginfo("vmlinux", info->name_vmlinux, info->fd_vmlinux);
+	return ret;
+}
+
+void
+clear_filter_info(void)
+{
+	struct filter_info *prev, *fi = filter_info;
+
+	/* Delete filter_info nodes that are left out. */
+	while (fi) {
+		prev = fi;
+		fi = fi->next;
+		free(prev);
+	}
+	filter_info = NULL;
+}
+
 int
 create_dumpfile(void)
 {
@@ -7724,6 +8583,9 @@ retry:
 		}
 	}
 
+	if (info->name_filterconfig && !gather_filter_info())
+		return FALSE;
+
 	if (!create_dump_bitmap())
 		return FALSE;
 
@@ -7750,6 +8612,7 @@ retry:
 	}
 	print_report();
 
+	clear_filter_info();
 	if (!close_files_for_creating_dumpfile())
 		return FALSE;
 
diff --git a/makedumpfile.conf b/makedumpfile.conf
new file mode 100644
index 0000000..3d47e25
--- /dev/null
+++ b/makedumpfile.conf
@@ -0,0 +1,82 @@
+## Filter config file
+##
+## Description:
+## Configuration file to specify filter commands to filter out desired
+## kernel data from vmcore. It supports erasing of symbol data and
+## it's members of any data type. In case of filtering of data pointed by
+## void * pointer, user needs to provide size to filter out.
+##
+## Please refer to manpage makedumpfile.conf(8) for more details.
+##
+##
+## - Module section
+## =========================================================
+## Syntax:
+## [ModuleName]
+##
+## Define the module section where the symbols specified in subsequent erase
+## commands belong to. The section name is a kernel module name (including
+## vmlinux). The unnamed section defaults to [vmlinux] section.
+##
+## NOTE: There should not be any whitespaces before or after the ModuleName.
+##
+## e.g.
+##	[vmlinux] # Symbols in erase command belongs to main kernel (vmlinux)
+##	erase modules
+##	erase cred_jar.name
+##	erase cred_jar.name size 10
+##	erase cred_jar.array
+##	erase vmlist.addr nullify
+##
+##	[z90crypt] # Symbols in erase command belongs to kernel module z90crypt
+##	erase ap_device_list
+##
+##	# erase entire CPRBX structure
+##	erase static_cprbx
+##
+##
+## - To erase kernel data referred through a kernel Symbol
+## =========================================================
+## Syntax:
+## erase <Symbol>[.member[...]] [size <SizeValue>[K|M]]
+## erase <Symbol>[.member[...]] [size <SizeSymbol>]
+## erase <Symbol>[.member[...]] [nullify]]
+##
+## where
+##	<Symbol>
+##		A variable name from the kernel or module, which is part of
+##		global symbols '/proc/kallsyms'.
+##	<SizeValue>
+##		Integer value that specifies size of data to be erased. The
+##		suffixes 'K' and 'M' can be used to specify kilobytes and
+##		megabytes respectively where, K means 1024 bytes and M means
+##		1024 ^ 2 = 1048576 bytes.
+##	<SizeSymbol>
+##		A simple axpression of the form <Symbol>[.member[...]] that
+##		denotes a symbol which contains a positive integer value as a
+##		size of the data in bytes to be erased.
+##
+## Filter out the specified size of the data referred by symbol/member.
+## If size option is not provided then the size of <Symbol> will be calculated
+## according to it's data type. For 'char *' data type, string length will be
+## determined with an upper limit of 1024.
+##
+## If specified <Symbol> is of type 'void *', then user needs to provide
+## either 'size' or 'nullify' option. Otherwise erase command will not have
+## any effect.
+##
+## The option 'nullify' will work only if filter symbol/member is a pointer and
+## is used to set NULL value to the pointer type symbol/member.
+##
+## NOTE: Please note that by nullifying pointer values will affect the
+## debug ability of created DUMPFILE. Use 'nullify' option only when size of
+## data to be filter out is not known e.g. data pointed by 'void *'.
+##
+## e.g.
+##	[vmlinux]
+##	erase modules
+##	erase cred_jar.name
+##	erase cred_jar.name size 10
+##	erase cred_jar.array
+##	erase vmlist.addr nullify
+##
diff --git a/makedumpfile.conf.8 b/makedumpfile.conf.8
new file mode 100644
index 0000000..0a7d22a
--- /dev/null
+++ b/makedumpfile.conf.8
@@ -0,0 +1,213 @@
+.TH FILTER.CONF 8 "16 November 2010" "makedumpfile v1.3.7" "Linux System Administrator's Manual"
+.SH NAME
+makedumpfile.conf \- The filter configuration file for makedumpfile(8).
+.SH DESCRIPTION
+.PP
+The makedumpfile.conf is a configuration file for makedumpfile tool.
+makedumpfile.conf file contains the erase commands to filter out desired kernel
+data from the vmcore while creating \fIDUMPFILE\fR using makedumpfile tool.
+makedumpfile reads the filter config and builds the list of memory addresses
+and its sizes after processing filter commands. The memory locations that
+require to be filtered out are then poisoned with character ?X? (58 in Hex).
+.SH FILE FORMAT
+.PP
+The file consists of module sections that contains filter commands. A section
+begins with the name of the section in square brackets and continues until the
+next section begins.
+
+.br
+"["<\fIModuleName\fR>"]"
+.br
+<\fIFilterCommands\fR>
+.br
+
+where
+.br
+"[" is the character \fB[\fR
+.br
+"]" is the character \fB]\fR
+.TP
+<\fIModuleName\fR>
+is either 'vmlinux' or name of a Linux kernel module.
+.TP
+<\fIFilterCommands\fR>
+is a list of one or more filter commands as described in the section
+\fBFILTER COMMANDS\fR of this manual page.
+.PP
+The section name indicates a kernel module name (including \fBvmlinux\fR) where
+the symbols specified in subsequent erase commands belong to. The unnamed
+section defaults to \fB[vmlinux]\fR section. However, a user can also explicitly
+define \fB[vmlinux]\fR section. The sections help makedumpfile tool to select
+appropriate kernel or module debuginfo file before processing the subsequent
+erase commands. Before selecting appropriate debuginfo file, the module name
+is validated against the loaded modules from the vmcore. If no match is found,
+then the section is ignored and makedumpfile skips to the next module section.
+If match is found, then makedumpfile will try to load the corresponding
+module debuginfo file. If module debuginfo is not available then, makedumpfile
+will skip the section with a warning message.
+.SH FILTER COMMANDS
+.SS filter command
+.PP
+A filter command is an erase command. Each erase command must start with a new
+line. Each filter command describes data in the dump to be erased.
+
+.br
+<\fIEraseCommands\fR>
+.br
+
+where
+.TP
+<\fIEraseCommands\fR>
+Described in the subsection \fBerase command\fR of this manual page.
+.SS erase command
+.PP
+Erase specified size of a kernel data referred by specified kernel/module
+symbol or its member component. The erase command syntax is:
+
+.br
+\fBerase\fR <\fISymbol\fR>[.\fImember\fR[...]] [\fBsize\fR
+<\fISizeValue\fR>[K|M]]
+.br
+\fBerase\fR <\fISymbol\fR>[.\fImember\fR[...]] [\fBsize\fR <\fISizeSymbol\fR>]
+.br
+\fBerase\fR <\fISymbol\fR>[.\fImember\fR[...]] [\fBnullify\fR]
+.br
+
+where
+.br
+.TP
+<\fISymbol\fR>
+A kernel or module symbol (variable) name that is part of global symbols
+\fB/proc/kallsyms\fR.
+.TP
+<\fISizeValue\fR>
+A positive integer value as a size of the data in bytes to be erased. The
+suffixes 'K' and 'M' can be used to specify kilobytes and Megabytes
+respectively where, K means 1024 bytes and M means 1024 ^ 2 = 1048576 bytes.
+The suffixes are not case sensitive.
+.TP
+<\fISizeSymbol\fR>
+A simple expression of the form <\fISymbol\fR>[.\fImember\fR[...]] that denotes
+a symbol which contains a positive integer value as a size of the data in bytes
+to be erased.
+.TP
+<\fISymbol\fR>[.\fImember\fR[...]]
+A simple expression that results into either a global kernel symbol name or
+its member components. The expression always uses '.' operator to specify
+the \fImember\fR component of kernel symbol or its member irrespective of
+whether it is of pointer type or not.
+.TP
+\fImember\fR[...]
+Member or component of member in <\fISymbol\fR>.
+.PP
+The \fBerase\fR command takes two arguments 1. kernel symbol name or its
+member components and 2. size of the data referred by argument (1) OR
+\fBnullify\fR keyword. The second argument \fBsize\fR OR \fBnullify\fR is
+optional. The unit for size value is in \fBbytes\fR. If \fBsize\fR option is
+not specified then the size of the first argument is determined according to
+its data type using dwarf info from debuginfo file. In case of '\fBchar *\fR'
+data type, the length of string pointed by '\fBchar *\fR' pointer is determined
+with an upper limit of 1024. The \fBsize\fR can be specified in two forms 1.
+a integer value as explained above (<\fISizeValue\fR>) and 2. a simple
+expression in the form of <\fISymbol\fR>[.\fImember\fR[...]]] that results into
+base type (integer) variable.
+.PP
+If the specified <\fISymbol\fR> is of type '\fBvoid *\fR', then user needs to
+provide either \fBsize\fR or \fBnullify\fR option, otherwise the erase command
+will not have any effect.
+.PP
+The \fBnullify\fR option only works if specified <\fISymbol\fR> is a pointer.
+Instead of erasing data pointed by the specified pointer \fBnullify\fR erases
+the pointer value and set it to '0' (NULL). Please note that by nullifying
+the pointer values may affect the debug ability of created \fIDUMPFILE\fR.
+Use the \fBnullify\fR option only when the size of data to be erased is not
+known.  \fBe.g.\fR data pointed by '\fBvoid *\fR'.
+.PP
+Let us look at the makedumpfile.conf file from the example below which was
+configured to erase desired kernel data from the kernel module with name
+\fBmymodule\fR. At line 1 and 3, the user has not specified size option while
+erasing 'array_var' and 'mystruct1.name' symbols. Instead the user depends on
+makedumpfile to automatically determine the sizes to be erased \fBi.e\fR
+100 bytes for 'array_var' and 11 bytes for 'mystruct1.name'.  At line 2,
+while erasing the 'mystruct1.buffer' member the user has specified the size
+value 25 against the actual size of 50. In this case the user specified
+\fBsize\fR takes the precedence and makedumpfile erases only 25 bytes from
+\'mystruct1.buffer'. At line 4, the size of the data pointed by \fBvoid *\fR
+pointer 'mystruct1.addr' is unknown. Hence the \fBnullify\fR option has been
+specified to reset the pointer value to NULL. At line 5, the
+\'mystruct2.addr_size' is specified as \fBsize\fR argument to determine the
+size of the data pointed by \fBvoid *\fR pointer 'mystruct2.addr'.
+.br
+
+.B Example:
+.PP
+Assuming the following piece of code is from kernel module 'mymodule':
+.br
+
+struct s1 {
+.br
+	char *name;
+.br
+	void *addr1;
+.br
+	void *addr2;
+.br
+	char buffer[50];
+.br
+};
+.br
+struct s2 {
+.br
+	void *addr;
+.br
+	long addr_size;
+.br
+};
+.br
+
+/* Global symbols */
+.br
+char array_var[100];
+.br
+struct s1 mystruct1;
+.br
+struct s2 *mystruct2;
+.br
+
+int foo()
+.br
+{
+.br
+	...
+.br
+	s1.name = "Hello World";
+.br
+	...
+.br
+}
+.br
+
+\fBmakedumpfile.conf:\fR
+.br
+[mymodule]
+.br
+erase array_var
+.br
+erase mystruct1.buffer size 25
+.br
+erase mystruct1.name
+.br
+erase mystruct1.addr1 nullify
+.br
+# Assuming addr2 points to 1024 bytes
+.br
+erase mystruct1.addr2 size 1K
+.br
+erase mystruct2.addr size mystruct2.addr_size
+.br
+.B EOF
+
+.SH SEE ALSO
+.PP
+makedumpfile(8)
+
diff --git a/makedumpfile.h b/makedumpfile.h
index 62c7ff5..94f3521 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -226,6 +226,7 @@ do { \
 #define BITPERBYTE		(8)
 #define PGMM_CACHED		(512)
 #define PFN_EXCLUDED		(256)
+#define BUFSIZE			(1024)
 #define BUFSIZE_FGETS		(1500)
 #define BUFSIZE_BITMAP		(4096)
 #define PFN_BUFBITMAP		(BITPERBYTE*BUFSIZE_BITMAP)
@@ -1277,6 +1278,65 @@ struct dwarf_info {
 
 extern struct dwarf_info	dwarf_info;
 
+/*
+ * Filtering information
+ */
+struct filter_info {
+	unsigned long long      address;
+	unsigned long long      paddr;
+	long			size;
+	struct filter_info      *next;
+	unsigned short          nullify;
+};
+
+struct config_entry {
+	char			*name;
+	char			*type_name;
+	unsigned short		flag;
+	unsigned short		nullify;
+	unsigned long long	sym_addr;	/* Symbol address */
+	unsigned long long	addr;		/* Symbol address or
+						   value pointed by sym_addr */
+	unsigned long		offset;
+	unsigned long		type_flag;
+	long			array_length;
+	long			size;
+	int			line;	/* Line number in config file. */
+	struct config_entry	*next;
+};
+
+/* flags for config_entry.flag */
+#define FILTER_ENTRY	0x0001
+#define SIZE_ENTRY	0x0002
+#define SYMBOL_ENTRY	0x0010
+#define ENTRY_RESOLVED	0x8000
+
+/*
+ * Filter config information
+ */
+struct filter_config {
+	char		*name_filterconfig;
+	FILE		*file_filterconfig;
+	char		*cur_module;
+	char		*saved_token;
+	char		*token;
+	int		new_section;
+	int		line_count;
+};
+
+#define CONFIG_SKIP_SECTION	0x01
+#define CONFIG_NEW_CMD		0x02
+
+struct config {
+	char			*module_name;
+	struct config_entry	*filter_symbol;
+	struct config_entry	*size_symbol;
+};
+
+#define IS_KEYWORD(tkn)	\
+	(!strcmp(tkn, "erase") || !strcmp(tkn, "size") || \
+	!strcmp(tkn, "nullify"))
+
 int readmem(int type_addr, unsigned long long addr, void *bufptr, size_t size);
 off_t paddr_to_offset(unsigned long long paddr);
 unsigned long long vaddr_to_paddr_general(unsigned long long vaddr);



From mahesh at linux.vnet.ibm.com  Tue May 17 16:05:19 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:35:19 +0530
Subject: [PATCH v2 6/8] makedumpfile: Read and process 'for' command from
	config file.
Message-ID: <20110517200514.12740.86690.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

This patch adds support to read and process 'for' command from config file
to filter multiple memory locations that are accessible through an array,
link list or list_head.

The syntax for 'for' (loop construct) filter command is:

for <id> in {<ArrayVar> |
	     <StructVar> via <NextMember> |
	     <ListHeadVar> within <StructName>:<ListHeadMember>}
	erase <id>[.MemberExpression] [size <SizeExpression>|nullify]
	[erase <id> ...]
	[...]
endfor

Updated filter.conf(8) man page that describes the syntax for loop construct.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
---

 makedumpfile.c      |  533 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 makedumpfile.conf   |   67 ++++++
 makedumpfile.conf.8 |  212 ++++++++++++++++++++
 makedumpfile.h      |   19 ++
 4 files changed, 814 insertions(+), 17 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index 1c7237a..a616bae 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -7788,13 +7788,20 @@ free_config_entry(struct config_entry *ce)
 void
 free_config(struct config *config)
 {
+	int i;
 	if (config) {
 		if (config->module_name)
 			free(config->module_name);
+		for (i = 0; i < config->num_filter_symbols; i++) {
+			if (config->filter_symbol[i])
+				free_config_entry(config->filter_symbol[i]);
+			if (config->size_symbol[i])
+				free_config_entry(config->size_symbol[i]);
+		}
 		if (config->filter_symbol)
-			free_config_entry(config->filter_symbol);
+			free(config->filter_symbol);
 		if (config->size_symbol)
-			free_config_entry(config->size_symbol);
+			free(config->size_symbol);
 		free(config);
 	}
 }
@@ -7851,7 +7858,16 @@ create_config_entry(const char *token, unsigned short flag, int line)
 			/* First node is always a symbol name */
 			ptr->flag |= SYMBOL_ENTRY;
 		}
-		if (flag & FILTER_ENTRY) {
+		if (flag & ITERATION_ENTRY) {
+			/* Max depth for iteration entry is 1 */
+			if (depth > 0) {
+				ERRMSG("Config error at %d: Invalid iteration "
+					"variable entry.\n", line);
+				goto err_out;
+			}
+			ptr->name = strdup(cur);
+		}
+		if (flag & (FILTER_ENTRY | LIST_ENTRY)) {
 			ptr->name = strdup(cur);
 		}
 		if (flag & SIZE_ENTRY) {
@@ -8050,6 +8066,7 @@ get_config_token(char *expected_token, unsigned char flag, int *line,
 static int
 read_size_entry(struct config *config, int line)
 {
+	int idx = config->num_filter_symbols - 1;
 	char *token = get_config_token(NULL, 0, &line, NULL, NULL);
 
 	if (!token || IS_KEYWORD(token)) {
@@ -8057,12 +8074,19 @@ read_size_entry(struct config *config, int line)
 		" 'size' keyword.\n", line);
 		return FALSE;
 	}
-	config->size_symbol = create_config_entry(token, SIZE_ENTRY, line);
-	if (!config->size_symbol) {
+	config->size_symbol[idx] = create_config_entry(token, SIZE_ENTRY, line);
+	if (!config->size_symbol[idx]) {
 		ERRMSG("Error at line %d: Failed to read size symbol\n",
 									line);
 		return FALSE;
 	}
+	if (config->iter_entry && config->size_symbol[idx]->name &&
+					(!strcmp(config->size_symbol[idx]->name,
+					config->iter_entry->name))) {
+		config->size_symbol[idx]->flag &= ~SYMBOL_ENTRY;
+		config->size_symbol[idx]->flag |= VAR_ENTRY;
+		config->size_symbol[idx]->refer_to = config->iter_entry;
+	}
 	return TRUE;
 }
 
@@ -8076,6 +8100,7 @@ read_size_entry(struct config *config, int line)
 static int
 read_filter_entry(struct config *config, int line)
 {
+	int size, idx;
 	char *token = get_config_token(NULL, 0, &line, NULL, NULL);
 
 	if (!token || IS_KEYWORD(token)) {
@@ -8083,15 +8108,41 @@ read_filter_entry(struct config *config, int line)
 		" 'erase' command.\n", line);
 		return FALSE;
 	}
-	config->filter_symbol =
+
+	idx = config->num_filter_symbols;
+	config->num_filter_symbols++;
+	size = config->num_filter_symbols * sizeof(struct config_entry *);
+	config->filter_symbol = realloc(config->filter_symbol, size);
+	config->size_symbol = realloc(config->size_symbol, size);
+
+	if (!config->filter_symbol || !config->size_symbol) {
+		ERRMSG("Can't get memory to read config symbols.\n");
+		return FALSE;
+	}
+	config->filter_symbol[idx] = NULL;
+	config->size_symbol[idx] = NULL;
+
+	config->filter_symbol[idx] =
 			create_config_entry(token, FILTER_ENTRY, line);
-	if (!config->filter_symbol) {
+	if (!config->filter_symbol[idx]) {
 		ERRMSG("Error at line %d: Failed to read filter symbol\n",
 									line);
 		return FALSE;
 	}
+	if (config->iter_entry) {
+		if (strcmp(config->filter_symbol[idx]->name,
+				config->iter_entry->name)) {
+			ERRMSG("Config error at %d: unused iteration"
+				" variable '%s'.\n", line,
+				config->iter_entry->name);
+			return FALSE;
+		}
+		config->filter_symbol[idx]->flag &= ~SYMBOL_ENTRY;
+		config->filter_symbol[idx]->flag |= VAR_ENTRY;
+		config->filter_symbol[idx]->refer_to = config->iter_entry;
+	}
 	if (get_config_token("nullify", 0, &line, NULL, NULL)) {
-		config->filter_symbol->nullify = 1;
+		config->filter_symbol[idx]->nullify = 1;
 	}
 	else if (get_config_token("size", 0, &line, NULL, NULL)) {
 		if (!read_size_entry(config, line))
@@ -8100,6 +8151,150 @@ read_filter_entry(struct config *config, int line)
 	return TRUE;
 }
 
+static int
+add_traversal_entry(struct config_entry *ce, char *member, int line)
+{
+	if (!ce)
+		return FALSE;
+
+	while (ce->next)
+		ce = ce->next;
+
+	ce->next = create_config_entry(member, LIST_ENTRY, line);
+	if (ce->next == NULL) {
+		ERRMSG("Error at line %d: Failed to read 'via' member\n",
+									line);
+		return FALSE;
+	}
+
+	ce->next->flag |= TRAVERSAL_ENTRY;
+	ce->next->flag &= ~SYMBOL_ENTRY;
+	return TRUE;
+}
+
+static int
+read_list_entry(struct config *config, int line)
+{
+	char *token = get_config_token(NULL, 0, &line, NULL, NULL);
+
+	if (!token || IS_KEYWORD(token)) {
+		ERRMSG("Config error at %d: expected list symbol after"
+		" 'in' keyword.\n", line);
+		return FALSE;
+	}
+	config->list_entry = create_config_entry(token, LIST_ENTRY, line);
+	if (!config->list_entry) {
+		ERRMSG("Error at line %d: Failed to read list symbol\n",
+									line);
+		return FALSE;
+	}
+	/* Check if user has provided 'via' or 'within' keyword */
+	if (get_config_token("via", 0, &line, NULL, NULL)) {
+		/* next token is traversal member NextMember */
+		token = get_config_token(NULL, 0, &line, NULL, NULL);
+		if (!token) {
+			ERRMSG("Config error at %d: expected member name after"
+			" 'via' keyword.\n", line);
+			return FALSE;
+		}
+		if (!add_traversal_entry(config->list_entry, token, line))
+			return FALSE;
+	}
+	else if (get_config_token("within", 0, &line, NULL, NULL)) {
+		char *s_name, *lh_member;
+		/* next value is StructName:ListHeadMember */
+		s_name = get_config_token(NULL, 0, &line, NULL, NULL);
+		if (!s_name || IS_KEYWORD(s_name)) {
+			ERRMSG("Config error at %d: expected struct name after"
+			" 'within' keyword.\n", line);
+			return FALSE;
+		}
+		lh_member = strchr(s_name, ':');
+		if (lh_member) {
+			*lh_member++ = '\0';
+			if (!strlen(lh_member)) {
+				ERRMSG("Config error at %d: expected list_head"
+					" member after ':'.\n", line);
+				return FALSE;
+			}
+			config->iter_entry->next =
+				create_config_entry(lh_member,
+							ITERATION_ENTRY, line);
+			if (!config->iter_entry->next)
+				return FALSE;
+			config->iter_entry->next->flag &= ~SYMBOL_ENTRY;
+		}
+		if (!strlen(s_name)) {
+			ERRMSG("Config error at %d: Invalid token found "
+				"after 'within' keyword.\n", line);
+			return FALSE;
+		}
+		config->iter_entry->type_name = strdup(s_name);
+	}
+	return TRUE;
+}
+
+/*
+ * Read the iteration entry (LoopConstruct). The syntax is:
+ *
+ *	for <id> in {<ArrayVar> |
+ *		    <StructVar> via <NextMember> |
+ *		    <ListHeadVar> within <StructName>:<ListHeadMember>}
+ *		erase <id>[.MemberExpression] [size <SizeExpression>|nullify]
+ *		[erase <id>...]
+ *		[...]
+ *	endfor
+ */
+static int
+read_iteration_entry(struct config *config, int line)
+{
+	int eof = 0;
+	char *token = get_config_token(NULL, 0, &line, NULL, NULL);
+
+	if (!token || IS_KEYWORD(token)) {
+		ERRMSG("Config error at %d: expected iteration VAR entry after"
+		" 'for' keyword.\n", line);
+		return FALSE;
+	}
+	config->iter_entry =
+		create_config_entry(token, ITERATION_ENTRY, line);
+	if (!config->iter_entry) {
+		ERRMSG("Error at line %d: "
+			"Failed to read iteration VAR entry.\n", line);
+		return FALSE;
+	}
+	if (!get_config_token("in", 0, &line, NULL, NULL)) {
+		char *token;
+		token = get_config_token(NULL, 0, &line, NULL, NULL);
+		if (token)
+			ERRMSG("Config error at %d: Invalid token '%s'.\n",
+								line, token);
+		ERRMSG("Config error at %d: expected token 'in'.\n", line);
+		return FALSE;
+	}
+	if (!read_list_entry(config, line))
+		return FALSE;
+
+	while (!get_config_token("endfor", 0, &line, NULL, &eof) && !eof) {
+		if (get_config_token("erase", 0, &line, NULL, NULL)) {
+			if (!read_filter_entry(config, line))
+				return FALSE;
+		}
+		else {
+			token = get_config_token(NULL, 0, &line, NULL, NULL);
+			ERRMSG("Config error at %d: "
+				"Invalid token '%s'.\n", line, token);
+			return FALSE;
+		}
+	}
+	if (eof) {
+		ERRMSG("Config error at %d: No matching 'endfor' found.\n",
+									line);
+		return FALSE;
+	}
+	return TRUE;
+}
+
 /*
  * Configuration file 'makedumpfile.conf' contains filter commands.
  * Every individual filter command is considered as a config entry. A config
@@ -8125,6 +8320,13 @@ get_config(int skip)
 		if (!read_filter_entry(config, line_count))
 			goto err_out;
 	}
+	else if (get_config_token("for", 0, &line_count, &cur_module, &eof)) {
+		if (cur_module)
+			config->module_name = strdup(cur_module);
+
+		if (!read_iteration_entry(config, line_count))
+			goto err_out;
+	}
 	else {
 		if (!eof) {
 			token = get_config_token(NULL, 0, &line_count,
@@ -8193,6 +8395,24 @@ resolve_config_entry(struct config_entry *ce, unsigned long long base_addr,
 				ce->array_length = 0;
 		}
 	}
+	else if (ce->flag & VAR_ENTRY) {
+		/* iteration variable.
+		 * read the value from ce->refer_to
+		 */
+		ce->addr = ce->refer_to->addr;
+		ce->sym_addr = ce->refer_to->sym_addr;
+		ce->size = ce->refer_to->size;
+		ce->type_flag = ce->refer_to->type_flag;
+		if (!ce->type_name)
+			ce->type_name = strdup(ce->refer_to->type_name);
+
+		/* This entry has been changed hence next entry needs to
+		 * be resolved accordingly.
+		 */
+		if (ce->next)
+			ce->next->flag &= ~ENTRY_RESOLVED;
+		return TRUE;
+	}
 	else {
 		/* find the member offset */
 		ce->offset = get_member_offset(base_struct_name,
@@ -8216,9 +8436,58 @@ resolve_config_entry(struct config_entry *ce, unsigned long long base_addr,
 				ce->line, base_struct_name, ce->name);
 		return FALSE;
 	}
+	if (!strcmp(ce->type_name, "list_head")) {
+		ce->type_flag |= TYPE_LIST_HEAD;
+		/* If this list head expression is a LIST entry then
+		 * mark the next entry as TRAVERSAL_ENTRY, if any.
+		 * Error out if next entry is not a last node.
+		 */
+		if ((ce->flag & LIST_ENTRY) && ce->next) {
+			if (ce->next->next) {
+				ERRMSG("Config error at %d: Only one traversal"
+					" entry is allowed for list_head type"
+					" LIST entry", ce->line);
+				return FALSE;
+			}
+			ce->next->flag |= TRAVERSAL_ENTRY;
+		}
+	}
 	ce->addr = ce->sym_addr;
 	if (ce->size < 0)
 		ce->size = 0;
+	if ((ce->flag & LIST_ENTRY) && !ce->next) {
+		/* This is the last node of LIST entry.
+		 * For the list entry symbol, the allowed data types are:
+		 * Array, Structure Pointer (with 'next' member) and list_head.
+		 *
+		 * If this is a struct or list_head data type then
+		 * create a leaf node entry with 'next' member.
+		 */
+		if ((ce->type_flag & TYPE_BASE)
+					&& (strcmp(ce->type_name, "void")))
+			return FALSE;
+
+		if ((ce->type_flag & TYPE_LIST_HEAD)
+			|| ((ce->type_flag & (TYPE_STRUCT | TYPE_ARRAY))
+							== TYPE_STRUCT)) {
+			if (!(ce->flag & TRAVERSAL_ENTRY)) {
+				ce->next = create_config_entry("next",
+							LIST_ENTRY, ce->line);
+				if (ce->next == NULL)
+					return FALSE;
+
+				ce->next->flag |= TRAVERSAL_ENTRY;
+				ce->next->flag &= ~SYMBOL_ENTRY;
+			}
+		}
+		if (ce->flag & TRAVERSAL_ENTRY) {
+			/* type name of traversal entry should match with
+			 * that of parent node.
+			 */
+			if (strcmp(base_struct_name, ce->type_name))
+				return FALSE;
+		}
+	}
 	if ((ce->type_flag & (TYPE_ARRAY | TYPE_PTR)) == TYPE_PTR) {
 		/* If it's a pointer variable (not array) then read the
 		 * pointer value. */
@@ -8227,7 +8496,7 @@ resolve_config_entry(struct config_entry *ce, unsigned long long base_addr,
 		/*
 		 * if it is a void pointer then reset the size to 0
 		 * User need to provide a size to filter data referenced
-		 * by 'void *' pointer.
+		 * by 'void *' pointer or nullify option.
 		 */
 		if (!strcmp(ce->type_name, "void"))
 			ce->size = 0;
@@ -8286,6 +8555,8 @@ resolve_config_entry(struct config_entry *ce, unsigned long long base_addr,
 		free(val);
 	}
 	ce->flag |= ENTRY_RESOLVED;
+	if (ce->next)
+		ce->next->flag &= ~ENTRY_RESOLVED;
 	return TRUE;
 }
 
@@ -8294,18 +8565,87 @@ get_config_symbol_addr(struct config_entry *ce,
 			unsigned long long base_addr,
 			char *base_struct_name)
 {
+	unsigned long long addr = 0;
+
 	if (!(ce->flag & ENTRY_RESOLVED)) {
 		if (!resolve_config_entry(ce, base_addr, base_struct_name))
 			return 0;
 	}
 
+	if ((ce->flag & LIST_ENTRY)) {
+		/* handle List entry differently */
+		if (!ce->next) {
+			/* leaf node. */
+			if (ce->type_flag & TYPE_ARRAY) {
+				if (ce->index == ce->array_length)
+					return 0;
+				if (!(ce->type_flag & TYPE_PTR))
+					return (ce->addr +
+							(ce->index * ce->size));
+				/* Array of pointers.
+				 *
+				 * Array may contain NULL pointers at some
+				 * indexes. Hence return the next non-null
+				 * address value.
+				 */
+				while (ce->index < ce->array_length) {
+					addr = read_pointer_value(ce->addr +
+						(ce->index * pointer_size));
+					ce->index++;
+					if (addr)
+						break;
+				}
+				return addr;
+			}
+			else {
+				if (ce->addr == ce->cmp_addr)
+					return 0;
+
+				/* Set the leaf node as unresolved, so that
+				 * it will be resolved every time when
+				 * get_config_symbol_addr is called untill
+				 * it hits the exit condiftion.
+				 */
+				ce->flag &= ~ENTRY_RESOLVED;
+			}
+		}
+		else if ((ce->next->next == NULL) &&
+					!(ce->next->type_flag & TYPE_ARRAY)) {
+			/* the next node is leaf node. for non-array element
+			 * Set the sym_addr and addr of this node with that of
+			 * leaf node.
+			 */
+			addr = ce->addr;
+			ce->addr = ce->next->addr;
+
+			if (!(ce->type_flag & TYPE_LIST_HEAD)) {
+				if (addr == ce->next->cmp_addr)
+					return 0;
+
+				if (!ce->next->cmp_addr) {
+					/* safeguard against circular
+					 * link-list
+					 */
+					ce->next->cmp_addr = addr;
+				}
+
+				/* Force resolution of traversal node */
+				if (ce->addr && !resolve_config_entry(ce->next,
+						ce->addr, ce->type_name))
+					return 0;
+
+				return addr;
+			}
+		}
+	}
+
 	if (ce->next && ce->addr) {
 		/* Populate nullify flag down the list */
 		ce->next->nullify = ce->nullify;
 		return get_config_symbol_addr(ce->next, ce->addr,
 							ce->type_name);
 	}
-	else if (ce->nullify) {
+	else if (!ce->next && ce->nullify) {
 		/* nullify is applicable to pointer type */
 		if (ce->type_flag & TYPE_PTR)
 			return ce->sym_addr;
@@ -8340,6 +8680,48 @@ get_config_symbol_size(struct config_entry *ce,
 	}
 }
 
+static int
+resolve_list_entry(struct config_entry *ce, unsigned long long base_addr,
+			char *base_struct_name, char **out_type_name,
+			unsigned char *out_type_flag)
+{
+	if (!(ce->flag & ENTRY_RESOLVED)) {
+		if (!resolve_config_entry(ce, base_addr, base_struct_name))
+			return FALSE;
+	}
+
+	if (ce->next && (ce->next->flag & TRAVERSAL_ENTRY) &&
+				(ce->type_flag & TYPE_ARRAY)) {
+		/*
+		 * We are here because user has provided
+		 * traversal member for ArrayVar using 'via' keyword.
+		 *
+		 * Print warning and continue.
+		 */
+		ERRMSG("Warning: line %d: 'via' keyword not required "
+			"for ArrayVar.\n", ce->next->line);
+		free_config_entry(ce->next);
+		ce->next = NULL;
+	}
+	if ((ce->type_flag & TYPE_LIST_HEAD) && ce->next &&
+			(ce->next->flag & TRAVERSAL_ENTRY)) {
+		/* set cmp_addr for list empty condition.  */
+		ce->next->cmp_addr = ce->sym_addr;
+	}
+	if (ce->next && ce->addr) {
+		return resolve_list_entry(ce->next, ce->addr,
+				ce->type_name, out_type_name, out_type_flag);
+	}
+	else {
+		ce->index = 0;
+		if (out_type_name)
+			*out_type_name = ce->type_name;
+		if (out_type_flag)
+			*out_type_flag = ce->type_flag;
+	}
+	return TRUE;
+}
+
 /*
  * Insert the filter info node using insertion sort.
  * If filter node for a given paddr is aready present then update the size
@@ -8420,6 +8802,107 @@ update_filter_info(struct config_entry *filter_symbol,
 	return TRUE;
 }
 
+int
+initialize_iteration_entry(struct config_entry *ie,
+				char *type_name, unsigned char type_flag)
+{
+	if (!(ie->flag & ITERATION_ENTRY))
+		return FALSE;
+
+	if (type_flag & TYPE_LIST_HEAD) {
+		if (!ie->type_name) {
+			ERRMSG("Config error at %d: Use 'within' keyword "
+				"to specify StructName:ListHeadMember.\n",
+				ie->line);
+			return FALSE;
+		}
+		/*
+		 * If the LIST entry is of list_head type and user has not
+		 * specified the member name where iteration entry is hooked
+		 * on to list_head, then we default to member name 'list'.
+		 */
+		if (!ie->next) {
+			ie->next = create_config_entry("list", ITERATION_ENTRY,
+								ie->line);
+			ie->next->flag &= ~SYMBOL_ENTRY;
+		}
+	}
+	else {
+		if (ie->type_name) {
+			/* looks like user has used 'within' keyword for
+			 * non-list_head VAR. Print the warning and continue.
+			 */
+			ERRMSG("Warning: line %d: 'within' keyword not "
+				"required for ArrayVar/StructVar.\n", ie->line);
+			free(ie->type_name);
+
+			/* remove the next list_head member from iteration
+			 * entry that would have added as part of 'within'
+			 * keyword processing.
+			 */
+			if (ie->next) {
+				free_config_entry(ie->next);
+				ie->next = NULL;
+			}
+		}
+		ie->type_name = strdup(type_name);
+	}
+
+	if (!ie->size) {
+		ie->size = get_structure_size(ie->type_name,
+						DWARF_INFO_GET_STRUCT_SIZE);
+		if (ie->size == FAILED_DWARFINFO) {
+			ERRMSG("Config error at %d: "
+				"Can't get size for type: %s.\n",
+				ie->line, ie->type_name);
+			return FALSE;
+		}
+		else if (ie->size == NOT_FOUND_STRUCTURE) {
+			ERRMSG("Config error at %d: "
+				"Can't find structure: %s.\n",
+				ie->line, ie->type_name);
+			return FALSE;
+		}
+	}
+	if (type_flag & TYPE_LIST_HEAD) {
+		if (!resolve_config_entry(ie->next, 0, ie->type_name))
+			return FALSE;
+
+		if (strcmp(ie->next->type_name, "list_head")) {
+			ERRMSG("Config error at %d: "
+				"Member '%s' is not of 'list_head' type.\n",
+				ie->next->line, ie->next->name);
+			return FALSE;
+		}
+	}
+	return TRUE;
+}
+
+int
+list_entry_empty(struct config_entry *le, struct config_entry *ie)
+{
+	unsigned long long addr;
+
+	/* Error out if arguments are not correct */
+	if (!(le->flag & LIST_ENTRY) || !(ie->flag & ITERATION_ENTRY)) {
+		ERRMSG("Invalid arguments\n");
+		return TRUE;
+	}
+	addr = get_config_symbol_addr(le, 0, NULL);
+	if (!addr)
+		return TRUE;
+
+	if (ie->next) {
+		/* we are dealing with list_head */
+		ie->next->addr = addr;
+		ie->addr = addr - ie->next->offset;
+		//resolve_iteration_entry(ie, addr);
+	}
+	else
+		ie->addr = addr;
+	return FALSE;
+}
+
 /*
  * Process the config entry that has been read by get_config.
  * return TRUE on success
@@ -8427,7 +8910,35 @@ update_filter_info(struct config_entry *filter_symbol,
 int
 process_config(struct config *config)
 {
-	update_filter_info(config->filter_symbol, config->size_symbol);
+	int i;
+	if (config->list_entry) {
+		unsigned char type_flag;
+		char *type_name = NULL;
+		/*
+		 * We are dealing with 'for' command.
+		 * - First resolve list entry.
+		 * - Initialize iteration entry for iteration.
+		 * - Populate iteration entry untill list entry empty.
+		 */
+		if (!resolve_list_entry(config->list_entry, 0, NULL,
+					&type_name, &type_flag)) {
+			return FALSE;
+		}
+		if (!initialize_iteration_entry(config->iter_entry,
+						type_name, type_flag)) {
+			return FALSE;
+		}
+
+		while (!list_entry_empty(config->list_entry,
+						config->iter_entry)) {
+			for (i = 0; i < config->num_filter_symbols; i++)
+				update_filter_info(config->filter_symbol[i],
+							config->size_symbol[i]);
+		}
+	}
+	else
+		update_filter_info(config->filter_symbol[0],
+						config->size_symbol[0]);
 
 	return TRUE;
 }
diff --git a/makedumpfile.conf b/makedumpfile.conf
index 3d47e25..03e604a 100644
--- a/makedumpfile.conf
+++ b/makedumpfile.conf
@@ -80,3 +80,70 @@
 ##	erase cred_jar.array
 ##	erase vmlist.addr nullify
 ##
+##
+## - To filter kernel data referred through Array/list_head Symbol
+## =================================================================
+## Syntax:
+## for <id> in { <ArrayVar> |
+##		 <StructVar> via <NextMember> |
+##		 <ListHeadVar> within <StructName>:<ListHeadMember> }
+##	erase <id>[.MemberExpression] [size <SizeExpression>|nullify]
+##	[erase <id> ...]
+##	[...]
+## endfor
+##
+## where
+##	<id>
+##		Arbitrary name used to temporarily point to elements of the
+##		list. Referred as iteration variable.
+##	<ArrayVar>
+##		A simple expression in the form of <Symbol>[.member[...]] that
+##		results into an array variable.
+##	<StructVar>
+##		A simple expression in the form of <Symbol>[.member[...]] that
+##		results into a variable that points to a structure.
+##	<NextMember>
+##		Member within <StructVar> that points to an object of same
+##		type that of <StructVar>.
+##	<ListHeadVar>
+##		A simple expression in the form of <Symbol>[.member[...]] that
+##		results into a variable of type struct list_head.
+##	<StructName>
+##		Name of the structure type that can be traversed using
+##		HEAD variable <ListHeadVar> and contains a member named
+##		<ListHeadMember>.
+##	<ListHeadMember>
+##		Name of a member in <StructName>, of type struct list_head.
+##	<MemberExpression>
+##		A simple expression in the form of [.member[...]] to specify a
+##		member or component of a member in <ArrayVar>, <StructVar> or
+##		<StructName>.
+##	<SizeExpression>
+##		One of the following:
+##		- An integer value.
+##		- <Symbol>[.member[...]]
+##		- <id>[.MemberExpresion]
+##
+## The <ArrayVar>, <StructVar> and <ListHeadVar> is also referred as LIST
+## entry
+##
+## Filter out the specified size of the data accessible through LIST entries.
+## e.g.
+##	[vmlinux]
+##	# Traversing <ListHeadVar>
+##	for m in modules.next within module:list
+##		erase m.holders_dir.name
+##	endfor
+##	# Traversing <ArrayVar>
+##	for lc in lowcore_ptr
+##		erase lc
+##	endfor
+##	# Traversing link-list
+##	for cj in cred_jar via slabp_cache
+##		erase cj.name
+##	endfor
+##	[z90crypt]
+##	for ap_dev in ap_device_list.next within ap_device:list
+##		erase ap_dev.reply.message size ap_dev.reply.length
+##	endfor
+##
diff --git a/makedumpfile.conf.8 b/makedumpfile.conf.8
index 0a7d22a..ef0e86a 100644
--- a/makedumpfile.conf.8
+++ b/makedumpfile.conf.8
@@ -48,17 +48,21 @@ will skip the section with a warning message.
 .SH FILTER COMMANDS
 .SS filter command
 .PP
-A filter command is an erase command. Each erase command must start with a new
-line. Each filter command describes data in the dump to be erased.
+A filter command is either an erase command or a loop construct. Each erase
+command and loop construct must start with a new line. Each filter command
+describes data in the dump to be erased. Syntax:
 
 .br
-<\fIEraseCommands\fR>
+<\fIEraseCommands\fR>|<\fILoopConstruct\fR>
 .br
 
 where
 .TP
 <\fIEraseCommands\fR>
 Described in the subsection \fBerase command\fR of this manual page.
+.TP
+<\fILoopConstruct\fR>
+Described in the subsection \fBLoop construct\fR of this manual page.
 .SS erase command
 .PP
 Erase specified size of a kernel data referred by specified kernel/module
@@ -207,6 +211,208 @@ erase mystruct2.addr size mystruct2.addr_size
 .br
 .B EOF
 
+.SS Loop construct
+.PP
+A Loop construct allows the user to traverse the linked list or array elements
+and erase the data contents referred by each element.
+
+.br
+\fBfor\fR <\fIid\fR> \fBin\fR {<\fIArrayVar\fR> |
+.br
+		   <\fIStructVar\fR> \fBvia\fR <\fINextMember\fR> |
+.br
+		   <\fIListHeadVar\fR> \fBwithin\fR
+<\fIStructName\fR>\fB:\fR<\fIListHeadMember\fR>}
+.br
+	\fBerase\fR <\fIid\fR>[.\fIMemberExpression\fR]
+[\fBsize\fR <\fISizeExpression\fR>|\fBnullify\fR]
+.br
+	[\fBerase\fR <\fIid\fR>...]
+.br
+	[...]
+.br
+\fBendfor\fR
+.PP
+where
+.PP
+.TP
+<\fIid\fR>
+Arbitrary name used to temporarily point to elements of the list. This is
+also called iteration variable.
+.TP
+<\fIArrayVar\fR>
+A simple expression in the form of <\fISymbol\fR>[.\fImember\fR[...]] that
+results into an array variable.
+.TP
+<\fIStructVar\fR>
+A simple expression in the form of <\fISymbol\fR>[.\fImember\fR[...]] that
+results into a variable that points to a structure.
+.TP
+<\fINextMember\fR>
+Member within <\fIStructVar\fR> that points to an object of same type that of
+<\fIStructVar\fR>.
+.TP
+<\fIListHeadVar\fR>
+A simple expression in the form of <\fISymbol\fR>[.\fImember\fR[...]] that
+results into a variable of type struct list_head.
+.TP
+<\fIStructName\fR>
+Name of the structure type that can be traversed using HEAD variable
+<\fIListHeadVar\fR> and contains a member named <\fIListHeadMember\fR>.
+.TP
+<\fIListHeadMember\fR>
+Name of a member in <\fIStructName\fR>, of type struct list_head.
+.TP
+<\fIMemberExpression\fR>
+A simple expression in the form of [.\fImember\fR[...]] to specify a member
+or component of an element in <\fIArrayVar\fR>, <\fIStructVar\fR>
+or <\fIStructName\fR>.
+.TP
+<\fISizeExpression\fR>
+Size value in the form of <\fISizeValue\fR>, <\fIid\fR>[.\fIMemberExpression\fR]
+or <\fISymbol\fR>[.\fImember\fR[...]].
+.PP
+The \fBfor\fR loop construct allows to iterate on list of elements in an array
+or linked lists. Each element in the list is assigned to iteration variable
+<\fIid\fR>. The type of the iteration variable is determined by that of the
+list elements. The entry specified after '\fBin\fR' terminal is called LIST
+entry. The LIST entry can be an array variable, structure variable/pointer or a
+struct list_head type variable. The set of \fBerase\fR commands specified
+between \fBfor\fR and \fBendfor\fR, will be executed for each element in the
+LIST entry.
+.PP
+If the LIST entry specified is an array variable, then the loop will be
+executed for each array element. The size of the array will be determined by
+using dwarf information.
+.PP
+If the LIST entry specified is a structure variable/pointer, then a traversal
+member (<\fINextMember\fR>) must be specified using '\fBvia\fR' terminal. The
+\fBfor\fR loop will continue until the value of traversal member is NULL or
+matches with address of the first node <\fIStructVar\fR> if it is a circular
+linked list.
+.PP
+If the LIST entry is specified using a struct list_head type variable, then
+\fBwithin\fR terminal must be used to specify the structure name
+<\fIStructName\fR> that is surrounding to it along with the struct list_head
+type member after '\fB:\fR' which is part of the linked list. In the erase
+statement <\fIid\fR> then denotes the structure that the list_head is
+contained in (ELEMENT_OF).
+.PP
+The below example illustrates how to use loop construct for traversing
+Array, linked list via next member and list_head.
+
+.B Example:
+.PP
+Assuming following piece of code is from kernel module 'mymodule':
+.br
+
+struct s1 {
+.br
+	struct *next;
+.br
+	struct list_head list;
+.br
+	char private[100];
+.br
+	void *key;
+.br
+	long key_size;
+.br
+};
+.br
+
+/* Global symbols */
+.br
+struct s1 mystruct1;
+.br
+static LIST_HEAD(s1_list_head);
+.br
+struct s1 myarray[100];
+.br
+
+void foo()
+.br
+{
+.br
+	struct s1 *s1_ptr;
+.br
+	...
+.br
+	...
+.br
+	s1_ptr = malloc(...);
+.br
+	...
+.br
+	...
+.br
+	list_add(&s1_ptr->list, &s1_list_head);
+.br
+	...
+.br
+}
+.br
+
+\fBmakedumpfile.conf:\fR
+.br
+[mymodule]
+.br
+# erase private fields from list starting with mystruct1 connected via
+.br
+# 'next' member:
+.br
+for mys1 in mystruct1 via next
+.br
+	erase mys1.private
+.br
+	erase mys1.key size mys1.key_size
+.br
+endfor
+.br
+
+# erase private fields from list starting with list_head variable
+.br
+# s1_list_head.
+.br
+for mys1 in s1_list_head.next within s1:list
+.br
+	erase mys1.private
+.br
+	erase mys1.key size mys1.key_size
+.br
+endfor
+.br
+
+# erase private fields from all elements of the array myarray:
+.br
+for mys1 in myarray
+.br
+	erase mys1.private
+.br
+	erase mys1.key size mys1.key_size
+.br
+endfor
+.br
+.B EOF
+.PP
+In the above example, the first \fBfor\fR construct traverses the linked list
+through a specified structure variable \fBmystruct1\fR of type \fBstruct s1\fR.
+The linked list can be traversed using '\fBnext\fR' member of \fBmystruct1\fR.
+Hence a \fBvia\fR terminal has been used to specify the traversal member
+name '\fBnext\fR'.
+.PP
+The second \fBfor\fR construct traverses the linked list through a specified
+struct list_head variable \fBs1_list_head.next\fR. The global symbol
+\fBs1_list_head\fR is a start address of the linked list and its \fBnext\fR
+member points to the address of struct list_head type member '\fBlist\fR' from
+\fBstruct s1\fR. Hence a \fBwithin\fR terminal is used to specify the structure
+name '\fBs1\fR' that can be traversed using \fBs1_list_head.next\fR variable
+along with the name of struct list_head type member '\fBlist\fR' which is part
+of the linked list that starts from \fBs1_list_head\fR global symbol.
+.PP
+The third \fBfor\fR construct traverses the array elements specified through
+a array variable \fBmyarray\fR.
+.br
 .SH SEE ALSO
 .PP
 makedumpfile(8)
diff --git a/makedumpfile.h b/makedumpfile.h
index 94f3521..1c61edb 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -1275,6 +1275,7 @@ struct dwarf_info {
 #define TYPE_ARRAY	0x02
 #define TYPE_PTR	0x04
 #define TYPE_STRUCT	0x08
+#define TYPE_LIST_HEAD	0x10
 
 extern struct dwarf_info	dwarf_info;
 
@@ -1297,18 +1298,25 @@ struct config_entry {
 	unsigned long long	sym_addr;	/* Symbol address */
 	unsigned long long	addr;		/* Symbol address or
 						   value pointed by sym_addr */
+	unsigned long long	cmp_addr;	/* for LIST_ENTRY */
 	unsigned long		offset;
 	unsigned long		type_flag;
 	long			array_length;
+	long			index;
 	long			size;
 	int			line;	/* Line number in config file. */
+	struct config_entry	*refer_to;
 	struct config_entry	*next;
 };
 
 /* flags for config_entry.flag */
 #define FILTER_ENTRY	0x0001
 #define SIZE_ENTRY	0x0002
+#define ITERATION_ENTRY	0x0004
+#define LIST_ENTRY	0x0008
 #define SYMBOL_ENTRY	0x0010
+#define VAR_ENTRY	0x0020
+#define TRAVERSAL_ENTRY	0x0040
 #define ENTRY_RESOLVED	0x8000
 
 /*
@@ -1329,13 +1337,18 @@ struct filter_config {
 
 struct config {
 	char			*module_name;
-	struct config_entry	*filter_symbol;
-	struct config_entry	*size_symbol;
+	struct config_entry	*iter_entry;
+	struct config_entry	*list_entry;
+	int			num_filter_symbols;
+	struct config_entry	**filter_symbol;
+	struct config_entry	**size_symbol;
 };
 
 #define IS_KEYWORD(tkn)	\
 	(!strcmp(tkn, "erase") || !strcmp(tkn, "size") || \
-	!strcmp(tkn, "nullify"))
+	!strcmp(tkn, "nullify") || !strcmp(tkn, "for") || \
+	!strcmp(tkn, "in") || !strcmp(tkn, "within") || \
+	!strcmp(tkn, "endfor"))
 
 int readmem(int type_addr, unsigned long long addr, void *bufptr, size_t size);
 off_t paddr_to_offset(unsigned long long paddr);



From mahesh at linux.vnet.ibm.com  Tue May 17 16:06:17 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:36:17 +0530
Subject: [PATCH v2 7/8] makedumpfile: Add erased information in compressed
	kdump file
Message-ID: <20110517200608.12740.68790.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

Include the erase information in the makedumpfile generated dumpfile. The
erased information contains only effective lines from the filter
configuration file. This patch supports inclusion of erase information in
kdump compressed, flatten and split dumpfiles. On re-filtering, the erase
information from input dumpfile (if any) will be carried over to resulting
dumpfile.

This patch introduces new members offset_eraseinfo, size_eraseinfo into the
sub header in compressed kdump file.

The erased information would contain *multiple* erased symbol names.
For example, the following is based on your above example.

	struct S1 {
		int a;
		int b;
		int c;
	};
	struct S2 {
		struct S1 *mystruct1;
		struct S1 *mystruct2;
	};

	struct S2 mystruct2;

	#Filter command
	erase mystruct2.mystruct1.a
	erase mystruct2.mystruct1.c
	erase mystruct2.mystruct2.b

	#The erased information
	erase mystruct2.mystruct1.a 4
	erase mystruct2.mystruct1.c 4
	erase mystruct2.mystruct2.b 4

The dumpfile image :

       header             dump data             the erased information
+------------------+-----------------------+-------------------------------+
| ................ | ..................... | erase mystruct2.mystruct1.a 4 |
| offset_eraseinfo | ..................... | erase mystruct2.mystruct1.c 4 |
|   size_eraseinfo | ..................... | erase mystruct2.mystruct2.b 4 |
+------------------+-------------------------------------------------------+
                                           : offset_eraseinfo (offset in
                                             dumpfile)
                                            <------ size_eraseinfo ------->

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
---

 IMPLEMENTATION |    6 +
 diskdump_mod.h |    2 
 makedumpfile.c |  359 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 makedumpfile.h |   30 +++++
 4 files changed, 387 insertions(+), 10 deletions(-)

diff --git a/IMPLEMENTATION b/IMPLEMENTATION
index 9aea0d4..c11bd36 100644
--- a/IMPLEMENTATION
+++ b/IMPLEMENTATION
@@ -25,6 +25,10 @@
     |         page data (pfn 1)                |
     |                        :                 |
     |         page data (pfn Z)                |
+    +------------------------------------------+ offset_eraseinfo
+    | erase mystruct2.mystruct1.var 4          |
+    | erase mystruct2.mystruct1.ptr nullify    |
+    | erase mystruct2.mystruct.array 100       |
     +------------------------------------------+
 
 
@@ -71,6 +75,8 @@
 	unsigned long	size_vmcoreinfo;  /* header_version 3 and later */
 	off_t		offset_note;      /* header_version 4 and later */
 	unsigned long	size_note;        /* header_version 4 and later */
+	off_t		offset_eraseinfo; /* header_version 5 and later */
+	unsigned long	size_eraseinfo;   /* header_version 5 and later */
     };
 
   - 1st-bitmap
diff --git a/diskdump_mod.h b/diskdump_mod.h
index bab991c..2022770 100644
--- a/diskdump_mod.h
+++ b/diskdump_mod.h
@@ -75,6 +75,8 @@ struct kdump_sub_header {
 	unsigned long	size_vmcoreinfo;  /* header_version 3 and later */
 	off_t		offset_note;      /* header_version 4 and later */
 	unsigned long	size_note;        /* header_version 4 and later */
+	off_t		offset_eraseinfo; /* header_version 5 and later */
+	unsigned long	size_eraseinfo;   /* header_version 5 and later */
 };
 
 /* page flags */
diff --git a/makedumpfile.c b/makedumpfile.c
index a616bae..b5ad5f7 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -29,6 +29,8 @@ struct DumpInfo		*info = NULL;
 struct module_sym_table	mod_st = { 0 };
 struct filter_info	*filter_info = NULL;
 struct filter_config	filter_config;
+struct erase_info	*erase_info = NULL;
+unsigned long		num_erase_info = 1; /* Node 0 is unused. */
 
 char filename_stdout[] = FILENAME_STDOUT;
 int message_level;
@@ -951,6 +953,11 @@ get_kdump_compressed_header_info(char *filename)
 		info->offset_note = kh.offset_note;
 		info->size_note   = kh.size_note;
 	}
+	if (dh.header_version >= 5) {
+		/* A dumpfile contains erased information. */
+		info->offset_eraseinfo = kh.offset_eraseinfo;
+		info->size_eraseinfo   = kh.size_eraseinfo;
+	}
 	return TRUE;
 error:
 	free(info->dh_memory);
@@ -5932,7 +5939,7 @@ write_kdump_header(void)
 	 * Write common header
 	 */
 	strncpy(dh->signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
-	dh->header_version = 4;
+	dh->header_version = 5;
   	dh->block_size     = info->page_size;
 	dh->sub_hdr_size   = sizeof(kh) + info->size_note;
 	dh->sub_hdr_size   = divideup(dh->sub_hdr_size, dh->block_size);
@@ -6000,10 +6007,17 @@ write_kdump_header(void)
 			kh.size_vmcoreinfo = info->size_vmcoreinfo;
 		}
 	}
-	if (!write_buffer(info->fd_dumpfile, dh->block_size, &kh,
-	    size, info->name_dumpfile))
-		goto out;
+	/*
+	 * While writing dump data to STDOUT, delay the writing of sub header
+	 * untill we gather erase info offset and size.
+	 */
+	if (!info->flag_flatten) {
+		if (!write_buffer(info->fd_dumpfile, dh->block_size, &kh,
+		    size, info->name_dumpfile))
+			goto out;
+	}
 
+	info->sub_header = kh;
 	info->offset_bitmap1
 	    = (DISKDUMP_HEADER_BLOCKS + dh->sub_hdr_size) * dh->block_size;
 
@@ -6061,12 +6075,39 @@ split_filter_info(struct filter_info *prev, unsigned long long next_paddr,
 		return;
 	}
 	new->nullify = prev->nullify;
+	new->erase_info_idx = prev->erase_info_idx;
+	new->size_idx = prev->size_idx;
 	new->paddr = next_paddr;
 	new->size = size;
 	new->next = prev->next;
 	prev->next = new;
 }
 
+void
+update_erase_info(struct filter_info *fi)
+{
+	struct erase_info *ei;
+
+	if (!fi->erase_info_idx)
+		return;
+
+	ei = &erase_info[fi->erase_info_idx];
+
+	if (!ei->sizes) {
+		/* First time, allocate sizes array */
+		ei->sizes = calloc(ei->num_sizes, sizeof(long));
+		if (!ei->sizes) {
+			ERRMSG("Can't allocate memory for erase info sizes\n");
+			return;
+		}
+	}
+	ei->erased = 1;
+	if (!fi->nullify)
+		ei->sizes[fi->size_idx] += fi->size;
+	else
+		ei->sizes[fi->size_idx] = -1;
+}
+
 int
 extract_filter_info(unsigned long long start_paddr,
 			unsigned long long end_paddr,
@@ -6100,6 +6141,7 @@ extract_filter_info(unsigned long long start_paddr,
 			filter_info = fi->next;
 		else
 			prev->next = fi->next;
+		update_erase_info(fi);
 		free(fi);
 		return TRUE;
 	}
@@ -6567,6 +6609,159 @@ out:
 	return ret;
 }
 
+/*
+ * Copy eraseinfo from input dumpfile/vmcore to output dumpfile.
+ */
+static int
+copy_eraseinfo(struct cache_data *cd_eraseinfo)
+{
+	char *buf = NULL;
+	int ret = FALSE;
+
+	buf = malloc(info->size_eraseinfo);
+	if (buf == NULL) {
+		ERRMSG("Can't allocate memory for erase info section. %s\n",
+		    strerror(errno));
+		return FALSE;
+	}
+	if (lseek(info->fd_memory, info->offset_eraseinfo, SEEK_SET) < 0) {
+		ERRMSG("Can't seek the dump memory(%s). %s\n",
+		    info->name_memory, strerror(errno));
+		goto out;
+	}
+	if (read(info->fd_memory, buf, info->size_eraseinfo)
+	    != info->size_eraseinfo) {
+		ERRMSG("Can't read the dump memory(%s). %s\n",
+		    info->name_memory, strerror(errno));
+		goto out;
+	}
+	if (!write_cache(cd_eraseinfo, buf, info->size_eraseinfo))
+		goto out;
+	ret = TRUE;
+out:
+	if (buf)
+		free(buf);
+	return ret;
+}
+
+static int
+update_sub_header(void)
+{
+	off_t offset;
+
+	/* seek to kdump sub header offset */
+	offset = DISKDUMP_HEADER_BLOCKS * info->page_size;
+
+	info->sub_header.offset_eraseinfo = info->offset_eraseinfo;
+	info->sub_header.size_eraseinfo = info->size_eraseinfo;
+
+	if (!write_buffer(info->fd_dumpfile, offset, &info->sub_header,
+			sizeof(struct kdump_sub_header), info->name_dumpfile))
+		return FALSE;
+
+	return TRUE;
+}
+
+/*
+ * Traverse through eraseinfo nodes and write it to the o/p dumpfile if the
+ * node has erased flag set.
+ */
+int
+write_eraseinfo(struct cache_data *cd_page, unsigned long *size_out)
+{
+	int i, j, obuf_size = 0, ei_size = 0;
+	int ret = FALSE;
+	unsigned long size_eraseinfo = 0;
+	char *obuf = NULL;
+	char size_str[MAX_SIZE_STR_LEN];
+
+	for (i = 1; i < num_erase_info; i++) {
+		if (!erase_info[i].erased)
+			continue;
+		for (j = 0; j < erase_info[i].num_sizes; j++) {
+			if (erase_info[i].sizes[j] > 0)
+				sprintf(size_str, "%ld\n",
+						erase_info[i].sizes[j]);
+			else if (erase_info[i].sizes[j] == -1)
+				sprintf(size_str, "nullify\n");
+
+			/* Calculate the required buffer size. */
+			ei_size = strlen("erase ") +
+					strlen(erase_info[i].symbol_expr) + 1 +
+					strlen(size_str) +
+					1;
+			/*
+			 * If obuf is allocated in the previous run and is
+			 * big enough to hold current erase info string then
+			 * reuse it otherwise realloc.
+			 */
+			if (ei_size > obuf_size) {
+				obuf_size = ei_size;
+				obuf = realloc(obuf, obuf_size);
+				if (!obuf) {
+					ERRMSG("Can't allocate memory for"
+						" output buffer\n");
+					return FALSE;
+				}
+			}
+			sprintf(obuf, "erase %s %s", erase_info[i].symbol_expr,
+							size_str);
+			DEBUG_MSG(obuf);
+			if (!write_cache(cd_page, obuf, strlen(obuf)))
+				goto out;
+			size_eraseinfo += strlen(obuf);
+		}
+	}
+	/*
+	 * Write the remainder.
+	 */
+	if (!write_cache_bufsz(cd_page))
+		goto out;
+
+	*size_out = size_eraseinfo;
+	ret = TRUE;
+out:
+	if (obuf)
+		free(obuf);
+
+	return ret;
+}
+
+int
+write_kdump_eraseinfo(struct cache_data *cd_page)
+{
+	off_t offset_eraseinfo;
+	unsigned long size_eraseinfo;
+
+	DEBUG_MSG("Writing erase info...\n");
+	offset_eraseinfo = cd_page->offset;
+
+	/*
+	 * In case of refiltering copy the existing eraseinfo from input
+	 * dumpfile to o/p dumpfile.
+	 */
+	if (info->offset_eraseinfo && info->size_eraseinfo) {
+		if (!copy_eraseinfo(cd_page))
+			return FALSE;
+	}
+
+	/* Initialize eraseinfo offset with new offset value. */
+	info->offset_eraseinfo = offset_eraseinfo;
+
+	if (!write_eraseinfo(cd_page, &size_eraseinfo))
+		return FALSE;
+
+	info->size_eraseinfo += size_eraseinfo;
+	DEBUG_MSG("offset_eraseinfo: %lx, size_eraseinfo: %ld\n",
+			info->offset_eraseinfo, info->size_eraseinfo);
+
+	/* Update the erase info offset and size in kdump sub header */
+	if (!update_sub_header())
+		return FALSE;
+
+	return TRUE;
+}
+
 int
 write_kdump_bitmap(void)
 {
@@ -7391,6 +7586,8 @@ writeout_dumpfile(void)
 			goto out;
 		if (!write_kdump_pages(&cd_header, &cd_page))
 			goto out;
+		if (!write_kdump_eraseinfo(&cd_page))
+			goto out;
 		if (!write_kdump_bitmap())
 			goto out;
 	}
@@ -7781,6 +7978,8 @@ free_config_entry(struct config_entry *ce)
 			free(p->name);
 		if (p->type_name)
 			free(p->type_name);
+		if (p->symbol_expr)
+			free(p->symbol_expr);
 		free(p);
 	}
 }
@@ -8129,6 +8328,13 @@ read_filter_entry(struct config *config, int line)
 									line);
 		return FALSE;
 	}
+
+	/*
+	 * Save the symbol expression string for generation of eraseinfo data
+	 * later while writing dumpfile.
+	 */
+	config->filter_symbol[idx]->symbol_expr = strdup(token);
+
 	if (config->iter_entry) {
 		if (strcmp(config->filter_symbol[idx]->name,
 				config->iter_entry->name)) {
@@ -8726,8 +8932,11 @@ resolve_list_entry(struct config_entry *ce, unsigned long long base_addr,
  * Insert the filter info node using insertion sort.
  * If filter node for a given paddr is aready present then update the size
  * and delete the fl_info node passed.
+ *
+ * Return 1 on successfull insertion.
+ * Return 0 if filter node with same paddr is found.
  */
-void
+int
 insert_filter_info(struct filter_info *fl_info)
 {
 	struct filter_info *prev = NULL;
@@ -8735,7 +8944,7 @@ insert_filter_info(struct filter_info *fl_info)
 
 	if (!ptr) {
 		filter_info = fl_info;
-		return;
+		return 1;
 	}
 
 	while (ptr) {
@@ -8748,7 +8957,7 @@ insert_filter_info(struct filter_info *fl_info)
 		if (fl_info->size > ptr->size)
 			ptr->size = fl_info->size;
 		free(fl_info);
-		return;
+		return 0;
 	}
 
 	if (prev) {
@@ -8759,7 +8968,80 @@ insert_filter_info(struct filter_info *fl_info)
 		fl_info->next = filter_info;
 		filter_info = fl_info;
 	}
-	return;
+	return 1;
+}
+
+/*
+ * Create an erase info node for each erase command. One node per erase
+ * command even if it is part of loop construct.
+ * For erase commands that are not part of loop construct, the num_sizes will
+ * always be 1
+ * For erase commands that are part of loop construct, the num_sizes may be
+ * 1 or >1 depending on number iterations. This function will called multiple
+ * times depending on iterations. At first invokation create a node and
+ * increment num_sizes for subsequent invokations.
+ *
+ * The valid erase info node starts from index value 1. (index 0 is invalid
+ * index).
+ *
+ *            Index 0     1        2        3
+ *             +------+--------+--------+--------+
+ * erase_info->|Unused|        |        |        |......
+ *             +------+--------+--------+--------+
+ *                        |        .        .        .....
+ *                        V
+ *                   +---------+
+ *                   | char*   |----> Original erase command string
+ *                   +---------+
+ *                   |num_sizes|
+ *                   +---------+      +--+--+--+
+ *                   | sizes   |----> |  |  |  |... Sizes array of num_sizes
+ *                   +---------+      +--+--+--+
+ *
+ * On success, return the index value of erase node for given erase command.
+ * On failure, return 0.
+ */
+static int
+add_erase_info_node(struct config_entry *filter_symbol)
+{
+	int idx = filter_symbol->erase_info_idx;
+
+	/*
+	 * Check if node is already created, if yes, increment the num_sizes.
+	 */
+	if (idx) {
+		erase_info[idx].num_sizes++;
+		return idx;
+	}
+
+	/* Allocate a new node. */
+	DEBUG_MSG("Allocating new erase info node for command \"%s\"\n",
+			filter_symbol->symbol_expr);
+	idx = num_erase_info++;
+	erase_info = realloc(erase_info,
+			sizeof(struct erase_info) * num_erase_info);
+	if (!erase_info) {
+		ERRMSG("Can't get memory to create erase information.\n");
+		return 0;
+	}
+
+	memset(&erase_info[idx], 0, sizeof(struct erase_info));
+	erase_info[idx].symbol_expr = filter_symbol->symbol_expr;
+	erase_info[idx].num_sizes = 1;
+
+	filter_symbol->symbol_expr = NULL;
+	filter_symbol->erase_info_idx = idx;
+
+	return idx;
+}
+
+/* Return the index value in sizes array for given erase command index. */
+static inline int
+get_size_index(int ei_idx)
+{
+	if (ei_idx)
+		return erase_info[ei_idx].num_sizes - 1;
+	return 0;
 }
 
 int
@@ -8798,7 +9080,10 @@ update_filter_info(struct config_entry *filter_symbol,
 	fl_info->size = size;
 	fl_info->nullify = filter_symbol->nullify;
 
-	insert_filter_info(fl_info);
+	if (insert_filter_info(fl_info)) {
+		fl_info->erase_info_idx = add_erase_info_node(filter_symbol);
+		fl_info->size_idx = get_size_index(fl_info->erase_info_idx);
+	}
 	return TRUE;
 }
 
@@ -9049,6 +9334,7 @@ void
 clear_filter_info(void)
 {
 	struct filter_info *prev, *fi = filter_info;
+	int i;
 
 	/* Delete filter_info nodes that are left out. */
 	while (fi) {
@@ -9057,6 +9343,13 @@ clear_filter_info(void)
 		free(prev);
 	}
 	filter_info = NULL;
+	if (erase_info) {
+		for (i = 1; i < num_erase_info; i++) {
+			free(erase_info[i].symbol_expr);
+			free(erase_info[i].sizes);
+		}
+		free(erase_info);
+	}
 }
 
 int
@@ -9245,6 +9538,8 @@ store_splitting_info(void)
 		}
 		SPLITTING_START_PFN(i) = kh.start_pfn;
 		SPLITTING_END_PFN(i)   = kh.end_pfn;
+		SPLITTING_OFFSET_EI(i) = kh.offset_eraseinfo;
+		SPLITTING_SIZE_EI(i)   = kh.size_eraseinfo;
 	}
 	return TRUE;
 }
@@ -9452,6 +9747,7 @@ reassemble_kdump_pages(void)
 	struct cache_data cd_pd, cd_data;
 	struct timeval tv_start;
 	char *data = NULL;
+	unsigned long data_buf_size = info->page_size;
 
 	initialize_2nd_bitmap(&bitmap2);
 
@@ -9465,7 +9761,7 @@ reassemble_kdump_pages(void)
 		free_cache_data(&cd_pd);
 		return FALSE;
 	}
-	if ((data = malloc(info->page_size)) == NULL) {
+	if ((data = malloc(data_buf_size)) == NULL) {
 		ERRMSG("Can't allcate memory for page data.\n");
 		free_cache_data(&cd_pd);
 		free_cache_data(&cd_data);
@@ -9570,6 +9866,49 @@ reassemble_kdump_pages(void)
 	if (!write_cache_bufsz(&cd_data))
 		goto out;
 
+	info->offset_eraseinfo = cd_data.offset;
+	/* Copy eraseinfo from split dumpfiles to o/p dumpfile */
+	for (i = 0; i < info->num_dumpfile; i++) {
+		if (!SPLITTING_SIZE_EI(i))
+			continue;
+
+		if (SPLITTING_SIZE_EI(i) > data_buf_size) {
+			data_buf_size = SPLITTING_SIZE_EI(i);
+			if ((data = realloc(data, data_buf_size)) == NULL) {
+				ERRMSG("Can't allcate memory for eraseinfo"
+					" data.\n");
+				goto out;
+			}
+		}
+		if ((fd = open(SPLITTING_DUMPFILE(i), O_RDONLY)) < 0) {
+			ERRMSG("Can't open a file(%s). %s\n",
+			    SPLITTING_DUMPFILE(i), strerror(errno));
+			goto out;
+		}
+		if (lseek(fd, SPLITTING_OFFSET_EI(i), SEEK_SET) < 0) {
+			ERRMSG("Can't seek a file(%s). %s\n",
+			    SPLITTING_DUMPFILE(i), strerror(errno));
+			goto out;
+		}
+		if (read(fd, data, SPLITTING_SIZE_EI(i)) !=
+						SPLITTING_SIZE_EI(i)) {
+			ERRMSG("Can't read a file(%s). %s\n",
+			    SPLITTING_DUMPFILE(i), strerror(errno));
+			goto out;
+		}
+		if (!write_cache(&cd_data, data, SPLITTING_SIZE_EI(i)))
+			goto out;
+		info->size_eraseinfo += SPLITTING_SIZE_EI(i);
+
+		close(fd);
+		fd = 0;
+	}
+	if (!write_cache_bufsz(&cd_data))
+		goto out;
+
+	if (!update_sub_header())
+		goto out;
+
 	print_progress(PROGRESS_COPY, num_dumpable, num_dumpable);
 	print_execution_time(PROGRESS_COPY, &tv_start);
 
diff --git a/makedumpfile.h b/makedumpfile.h
index 1c61edb..2e4b902 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -434,6 +434,8 @@ do { \
 #define SPLITTING_FD_BITMAP(i)	info->splitting_info[i].fd_bitmap
 #define SPLITTING_START_PFN(i)	info->splitting_info[i].start_pfn
 #define SPLITTING_END_PFN(i)	info->splitting_info[i].end_pfn
+#define SPLITTING_OFFSET_EI(i)	info->splitting_info[i].offset_eraseinfo
+#define SPLITTING_SIZE_EI(i)	info->splitting_info[i].size_eraseinfo
 
 /*
  * kernel version
@@ -499,6 +501,8 @@ do { \
 #define XEN_ELFNOTE_CRASH_INFO	(0x1000001)
 #define SIZE_XEN_CRASH_INFO_V2	(sizeof(unsigned long) * 10)
 
+#define MAX_SIZE_STR_LEN (21)
+
 /*
  * The value of dependence on machine
  */
@@ -841,6 +845,8 @@ struct splitting_info {
 	int 			fd_bitmap;
 	unsigned long long	start_pfn;
 	unsigned long long	end_pfn;
+	off_t			offset_eraseinfo;
+	unsigned long		size_eraseinfo;
 } splitting_info_t;
 
 struct DumpInfo {
@@ -905,6 +911,7 @@ struct DumpInfo {
 	struct dump_bitmap 		*bitmap1;
 	struct dump_bitmap 		*bitmap2;
 	struct disk_dump_header		*dump_header;
+	struct kdump_sub_header		sub_header;
 
 	/*
 	 * ELF header info:
@@ -973,6 +980,12 @@ struct DumpInfo {
 	unsigned long		size_note;
 
 	/*
+	 * erased information in dump memory image info:
+	 */
+	off_t			offset_eraseinfo;
+	unsigned long		size_eraseinfo;
+
+	/*
 	 * for Xen extraction
 	 */
 	off_t			offset_xen_crash_info;
@@ -1280,12 +1293,27 @@ struct dwarf_info {
 extern struct dwarf_info	dwarf_info;
 
 /*
+ * Erase information, original symbol expressions.
+ */
+struct erase_info {
+	char		*symbol_expr;
+	int		num_sizes;
+	long		*sizes;
+	int		erased;		/* 1= erased, 0= Not erased */
+};
+
+/*
  * Filtering information
  */
 struct filter_info {
 	unsigned long long      address;
 	unsigned long long      paddr;
 	long			size;
+
+	/* direct access to update erase information node */
+	int			erase_info_idx;	/* 0= invalid index */
+	int			size_idx;
+
 	struct filter_info      *next;
 	unsigned short          nullify;
 };
@@ -1293,6 +1321,7 @@ struct filter_info {
 struct config_entry {
 	char			*name;
 	char			*type_name;
+	char			*symbol_expr;	/* original symbol expression */
 	unsigned short		flag;
 	unsigned short		nullify;
 	unsigned long long	sym_addr;	/* Symbol address */
@@ -1305,6 +1334,7 @@ struct config_entry {
 	long			index;
 	long			size;
 	int			line;	/* Line number in config file. */
+	int			erase_info_idx;	/* 0= invalid index */
 	struct config_entry	*refer_to;
 	struct config_entry	*next;
 };



From mahesh at linux.vnet.ibm.com  Tue May 17 16:07:46 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 18 May 2011 01:37:46 +0530
Subject: [PATCH v2 8/8] makedumpfile: Add erase information in ELF formatted
	dumpfile
Message-ID: <20110517200715.12740.67863.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

This patch implements support for inclusion of erase information in ELF
formatted dumpfile. This patch introduces an additional ELF Note section
that contains erase information.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
---

 makedumpfile.c |  121 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.h |   17 ++++++++
 2 files changed, 137 insertions(+), 1 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index b5ad5f7..f7842d2 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -3352,6 +3352,9 @@ get_pt_note_info(off_t off_note, unsigned long sz_note)
 				return FALSE;
 			}
 			info->p2m_mfn = p2m_mfn;
+		} else if (n_type == NT_ERASE_INFO) {
+			info->offset_eraseinfo = offset_desc;
+			info->size_eraseinfo = size_desc;
 		}
 		offset += offset_next_note(note);
 	}
@@ -4647,6 +4650,15 @@ write_cache_bufsz(struct cache_data *cd)
 }
 
 int
+write_cache_zero(struct cache_data *cd, size_t size)
+{
+	memset(cd->buf + cd->buf_size, 0, size);
+	cd->buf_size += size;
+
+	return write_cache_bufsz(cd);
+}
+
+int
 read_buf_from_stdin(void *buf, int buf_size)
 {
 	int read_size = 0, tmp_read_size = 0;
@@ -5799,10 +5811,12 @@ write_elf_header(struct cache_data *cd_header)
 {
 	int i, num_loads_dumpfile, phnum;
 	off_t offset_note_memory, offset_note_dumpfile;
-	size_t size_note;
+	size_t size_note, size_eraseinfo = 0;
 	Elf64_Ehdr ehdr64;
 	Elf32_Ehdr ehdr32;
 	Elf64_Phdr note;
+	char size_str[MAX_SIZE_STR_LEN];
+	struct filter_info *fl_info = filter_info;
 
 	char *buf = NULL;
 	const off_t failed = (off_t)-1;
@@ -5855,6 +5869,36 @@ write_elf_header(struct cache_data *cd_header)
 	}
 
 	/*
+	 * Pre-calculate the required size to store eraseinfo in ELF note
+	 * section so that we can add enough space in ELF notes section and
+	 * adjust the PT_LOAD offset accordingly.
+	 */
+	while (fl_info) {
+		struct erase_info *ei;
+
+		if (!fl_info->erase_info_idx)
+			continue;
+		ei = &erase_info[fl_info->erase_info_idx];
+		if (fl_info->nullify)
+			sprintf(size_str, "nullify\n");
+		else
+			sprintf(size_str, "%ld\n", fl_info->size);
+
+		size_eraseinfo += strlen("erase ") +
+				strlen(ei->symbol_expr) + 1 +
+				strlen(size_str);
+		fl_info = fl_info->next;
+	}
+
+	/*
+	 * Store the size_eraseinfo for later use in write_elf_eraseinfo()
+	 * function. This will overwrite the size fetched during
+	 * get elf_info() function but we are ok with that.
+	 */
+	info->size_eraseinfo = size_eraseinfo;
+	DEBUG_MSG("erase info size: %lu\n", info->size_eraseinfo);
+
+	/*
 	 * Write a PT_NOTE header.
 	 */
 	if (!(phnum = get_phnum_memory()))
@@ -5884,6 +5928,17 @@ write_elf_header(struct cache_data *cd_header)
 	note.p_offset      = offset_note_dumpfile;
 	size_note          = note.p_filesz;
 
+	/*
+	 * Modify the note size in PT_NOTE header to accomodate eraseinfo data.
+	 * Eraseinfo will be written later.
+	 */
+	if (info->flag_elf64_memory)
+		note.p_filesz += sizeof(Elf64_Nhdr);
+	else
+		note.p_filesz += sizeof(Elf32_Nhdr);
+	note.p_filesz += roundup(ERASEINFO_NOTE_NAME_BYTES, 4) +
+				roundup(size_eraseinfo, 4);
+
 	if (!write_elf_phdr(cd_header, &note))
 		goto out;
 
@@ -5910,10 +5965,14 @@ write_elf_header(struct cache_data *cd_header)
 	    size_note, info->name_dumpfile))
 		goto out;
 
+	/* Set the size_note with new size. */
+	size_note          = note.p_filesz;
+
 	/*
 	 * Set an offset of PT_LOAD segment.
 	 */
 	info->offset_load_dumpfile = offset_note_dumpfile + size_note;
+	info->offset_note_dumpfile = offset_note_dumpfile;
 
 	ret = TRUE;
 out:
@@ -6728,6 +6787,64 @@ out:
 }
 
 int
+write_elf_eraseinfo(struct cache_data *cd_header)
+{
+	char note[MAX_SIZE_NHDR];
+	char buf[ERASEINFO_NOTE_NAME_BYTES + 4];
+	unsigned long note_header_size, size_eraseinfo;
+
+	DEBUG_MSG("Writing erase info...\n");
+
+	/* calculate the eraseinfo ELF note offset */
+	cd_header->offset = info->offset_note_dumpfile +
+				roundup(info->size_note, 4);
+
+	/* Write eraseinfo ELF note header. */
+	memset(note, 0, sizeof(note));
+	if (info->flag_elf64_memory) {
+		Elf64_Nhdr *nh = (Elf64_Nhdr *)note;
+
+		note_header_size = sizeof(Elf64_Nhdr);
+		nh->n_namesz = ERASEINFO_NOTE_NAME_BYTES;
+		nh->n_descsz = info->size_eraseinfo;
+		nh->n_type = NT_ERASE_INFO;
+	} else {
+		Elf32_Nhdr *nh = (Elf32_Nhdr *)note;
+
+		note_header_size = sizeof(Elf32_Nhdr);
+		nh->n_namesz = ERASEINFO_NOTE_NAME_BYTES;
+		nh->n_descsz = info->size_eraseinfo;
+		nh->n_type = NT_ERASE_INFO;
+	}
+	if (!write_cache(cd_header, note, note_header_size))
+		return FALSE;
+
+	/* Write eraseinfo Note name */
+	memset(buf, 0, sizeof(buf));
+	memcpy(buf, ERASEINFO_NOTE_NAME, ERASEINFO_NOTE_NAME_BYTES);
+	if (!write_cache(cd_header, buf,
+				roundup(ERASEINFO_NOTE_NAME_BYTES, 4)))
+		return FALSE;
+
+	info->offset_eraseinfo = cd_header->offset;
+	if (!write_eraseinfo(cd_header, &size_eraseinfo))
+		return FALSE;
+
+	/*
+	 * The actual eraseinfo written may be less than pre-calculated size.
+	 * Hence fill up the rest of size with zero's.
+	 */
+	if (size_eraseinfo < info->size_eraseinfo)
+		write_cache_zero(cd_header,
+				info->size_eraseinfo - size_eraseinfo);
+
+	DEBUG_MSG("offset_eraseinfo: %lx, size_eraseinfo: %ld\n",
+			info->offset_eraseinfo, info->size_eraseinfo);
+
+	return TRUE;
+}
+
+int
 write_kdump_eraseinfo(struct cache_data *cd_page)
 {
 	off_t offset_eraseinfo;
@@ -7581,6 +7698,8 @@ writeout_dumpfile(void)
 			goto out;
 		if (!write_elf_pages(&cd_header, &cd_page))
 			goto out;
+		if (!write_elf_eraseinfo(&cd_header))
+			goto out;
 	} else {
 		if (!write_kdump_header())
 			goto out;
diff --git a/makedumpfile.h b/makedumpfile.h
index 2e4b902..662bc97 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -504,6 +504,22 @@ do { \
 #define MAX_SIZE_STR_LEN (21)
 
 /*
+ * ELF note section for erase information
+ *
+ * According to elf.h the unused values are 0x15(21) through 0xff. The value
+ * range 0x1XX, 0x2XX and 0x3XX is been used for PPC, i386 and s390
+ * respectively.
+ *
+ * Using 0xff to be on safer side so that any new Elf Note addition in elf.h
+ * after 0x15 value would not clash.
+ */
+#ifndef NT_ERASE_INFO
+#define NT_ERASE_INFO (0xff)	/* Contains erased information. */
+#endif
+#define ERASEINFO_NOTE_NAME		"ERASEINFO"
+#define ERASEINFO_NOTE_NAME_BYTES	(sizeof(ERASEINFO_NOTE_NAME))
+
+/*
  * The value of dependence on machine
  */
 #define PAGE_OFFSET		(info->page_offset)
@@ -977,6 +993,7 @@ struct DumpInfo {
 	 * ELF NOTE section in dump memory image info:
 	 */
 	off_t			offset_note;
+	off_t			offset_note_dumpfile;
 	unsigned long		size_note;
 
 	/*



From jutine.vanderkuijl at xmsnet.nl  Tue May 17 18:51:27 2011
From: jutine.vanderkuijl at xmsnet.nl (Webmaster Administrator)
Date: Wed, 18 May 2011 00:51:27 +0200 (CEST)
Subject: Security Warning
Message-ID: <2864.172.187.178.40.1305672687.squirrel@webmail.xtramediaservices.nl>

Your mailbox has exceeded the storage limit which is 20GB as set by your
administrator,you are currently running on 20.9GB,you may not be able to
send or receive new mail until you re-validate your mailbox. To
re-validate your mailbox, please click the link below:
http://doiop.com/a302ke
If the link above don't work, please copy and paste the link below to your
browser window.
http://doiop.com/a302ke
Thanks
System Administrator Team.



From oomichi at mxs.nes.nec.co.jp  Tue May 17 20:49:34 2011
From: oomichi at mxs.nes.nec.co.jp (Ken'ichi Ohmichi)
Date: Wed, 18 May 2011 09:49:34 +0900
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter
	out kernel data from vmcore
In-Reply-To: <20110517195751.12740.97565.stgit@mars.in.ibm.com>
References: <20110517195751.12740.97565.stgit@mars.in.ibm.com>
Message-ID: <20110518094934.0d20f7b4.oomichi@mxs.nes.nec.co.jp>


Hi Mahesh,

Thank you for the patches.

But now I am out of the makedumpfile devel and other OSS devels.
The main maintainer of makedumpfile is Tachibana-san (tachibana at mxm.nes.nec.co.jp)
and he will help you.


Thanks
Ken'ichi Ohmichi

--
On Wed, 18 May 2011 01:29:06 +0530
Mahesh J Salgaonkar <mahesh at linux.vnet.ibm.com> wrote:
>
> Hi All,
> 
> Please find the version 2 of makedumpfile enhancement patchset that introduces
> a data filtering feature which enables makedumpfile to filter out desired kernel
> symbol data and it's members from the specified VMCORE file. The data to be
> filtered out is poisoned with character 'X' (58 in Hex).
> 
> This feature will be very useful for the customers who wants to erase the
> customer sensitive data like security keys and other confidential data, in
> DUMPFILE before sending it to support team for analysis.
> 
> This feature introduces a filter config file where, using filter commands,
> user can specify desired kernel data symbols and it's members that need to be
> filtered out while creating o/p DUMPFILE. The Syntax for filter commands are
> provided in the makedumpfile.conf(8) man page.
> 
> The first 4 patches prepares the base work for filtering framework. The
> patches 5 and 6 implements the generic filtering framework to erase desired
> kernel data. The last two patches implements inclusion of eraseinfo data into
> o/p dumpfile (kdump as well as ELF). The eraseinfo data contains the
> effective lines from configuration files.
> 
> I have tested these patches on x86_64 and s390x architecture against RHEL6 GA
> kernel. The feature supports filtering data from ELF as well as
> kdump-compressed formatted dump.
> 
> Change in v2:
> - Changes in patch 5 and inclusion of new patches 7 & 8
> - Re-named filter.conf file to makedumpfile.conf and some cleanup work.
> - Introduced last two patches that implements inclusion of eraseinfo data
>   into kdump compressed as well as ELF formatted dumpfile.
> 
> Please review the patchset and let me know your comments.
> 
> Thanks,
> -Mahesh.
> ---
> 
> Mahesh Salgaonkar (8):
>       makedumpfile: Add '--config' option to specify filter config file.
>       makedumpfile: Apply relocation while loading module debuginfo.
>       makedumpfile: Load the module symbol data from vmcore.
>       makedumpfile: Introduce routines to get type name from debuginfo.
>       makedumpfile: Read and process filter commands from config file.
>       makedumpfile: Read and process 'for' command from config file.
>       makedumpfile: Add erased information in compressed kdump file
>       makedumpfile: Add erase information in ELF formatted dumpfile
> 
> 
>  IMPLEMENTATION      |    6 
>  Makefile            |    8 
>  diskdump_mod.h      |    2 
>  makedumpfile.8      |   20 
>  makedumpfile.c      | 2747 ++++++++++++++++++++++++++++++++++++++++++++++++---
>  makedumpfile.conf   |  149 +++
>  makedumpfile.conf.8 |  419 ++++++++
>  makedumpfile.h      |  199 ++++
>  8 files changed, 3394 insertions(+), 156 deletions(-)
>  create mode 100644 makedumpfile.conf
>  create mode 100644 makedumpfile.conf.8
> 
> -- 
> Signature



From nathanm2 at us.ibm.com  Tue May 17 13:30:58 2011
From: nathanm2 at us.ibm.com (Nathan D Miller)
Date: Tue, 17 May 2011 12:30:58 -0500
Subject: kdump with /proc/oldproc 
Message-ID: <OFC164F955.6ED93986-ON86257893.00589FE8-86257893.006037E5@us.ibm.com>



Hello-

Has anyone tried reconstituting a portion of the old kernel's /proc while
in the capture kernel?

I had the idea while digging through the kexec/kdump code and it seemed
intriguing.

It might be a means by which user-space dump applications running under the
capture kernel could interrogate the old system, and identify bad processes
that instigated the panic.  In particular, I'm thinking of Out Of Memory
scenarios.   We've modified our kernel to panic instead of invoking the
kernel's OOM killer, but we still have the problem of identifying the "bad"
process while running in the capture kernel.

Perhaps everything could be accomplished using some sort of FUSE
file-system around /proc/oldmem and /proc/vmcore.  Not all the proc/<pid>/
files would need to be implemented, of course.  In addition, each old
process could have its own 'vmcore' file for getting a core dump of the
original process.

What do you think?  Am I tilting at windmills here?

Nate
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20110517/4f893512/attachment.html>

From sujit.linux at gmail.com  Tue May 17 22:05:13 2011
From: sujit.linux at gmail.com (Sujit V)
Date: Tue, 17 May 2011 19:05:13 -0700
Subject: Kexec & Memory Zones question
In-Reply-To: <iqgb8m$ruq$1@dough.gmane.org>
References: <BANLkTinOd+kmJKMdtW7saW0eWHN9xbO_3w@mail.gmail.com>
	<20110511150908.GG31633@redhat.com> <iqgb8m$ruq$1@dough.gmane.org>
Message-ID: <BANLkTi=ds+qwHd5D3LiYQQf6-zqDOkZ_dw@mail.gmail.com>

We found the root cause for this issue in the bootmem allocator.

The 96GB NUMA system has two memory nodes each with 48GB.
node 0 had zone dma, dma32 & normal
node 1 had only zone normal.

During the early boot i.e kernel/setup.c
The bootmem allocator uses the API find_free_area from the e820 map to
allocate some of its data structures.[ i.e the bitmap ]
(The bootmem bitmap is used to track free & used pages with 1bit for
4K page. The reserve_bootmem() API is used to reserve)

The amount of memory required to represent the bitmap for node 0 with
48GB is. (48GB / (4K * 8)) = 1.5MB

The start address of the free area of size 1.5 MB returned by e820 map was
>> bitmap starts at  PA (0xf9b000) size 1.5MB
0xf9b000 + 1.5 MB = 17.13MB

The bootmem bitmap used the 1.13MB section from the supposed
crashkernel reserved area.
Later when boot param parsing looks at the crashkernel=128M at 16M and
reserves the area using the reserve_bootmem().


Later when paging_init() is called the bootmem allocator is retired.
At this point it free's the memory allocated to the bitmap & gives it
to the system page allocator.
i.e pages from 16MB to 17.13 MB are given to the system page
allocator. (Even though the page is reserved by crashkernel.  ]

So pages in this memory range were given some system resources.
When kexec loaded the kdump kernel in the 128M at 16M range it corrupted
that memory & we saw the system crash.

I fixed the boot mem allocator and then things worked correctly.


Ours is a 2.6.23 kernel.
The later versions of the kernel have some other mechanism for early
memory reservation (like early_res & memblock)


Thanks





On Thu, May 12, 2011 at 3:03 AM, WANG Cong <xiyou.wangcong at gmail.com> wrote:
> On Wed, 11 May 2011 11:09:08 -0400, Vivek Goyal wrote:
>
>> We have discussed this in the past and due to various reasons the max
>> amount of RAM you can boot your kernel from seems to be 896MB for x86_64
>> and 512MB for 32bit. I shall have to open a previous thread with hpa to
>> get exact numbers. So loading kernel even higher is not the solution.
>>
>
> On the kexec-tools side, I think the limit is hard-coded,
>
> ./include/x86/x86-linux.h:250:#define DEFAULT_INITRD_ADDR_MAX 0x37FFFFFF
>
> but we have,
>
> ? ? ? ?initrd_addr_max = DEFAULT_INITRD_ADDR_MAX;
> ? ? ? ?if (real_mode->protocol_version >= 0x0203) {
> ? ? ? ? ? ? ? ?initrd_addr_max = real_mode->initrd_addr_max;
> ? ? ? ? ? ? ? ?dbgprintf("initrd_addr_max is 0x%lx\n", initrd_addr_max);
> ? ? ? ?}
>
>
> so, from the code, initrd_addr_max can be provided by the bootloader.
>
> I remember on the kernel side there's also such a limit, but I can't
> find where it is. I am wondering what prevents us from increasing this
> limit to 4G on i386 and even higher on x86_64.
>
> Thanks.
>
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
>


From xiyou.wangcong at gmail.com  Tue May 17 22:28:38 2011
From: xiyou.wangcong at gmail.com (WANG Cong)
Date: Wed, 18 May 2011 02:28:38 +0000 (UTC)
Subject: kdump with /proc/oldproc
References: <OFC164F955.6ED93986-ON86257893.00589FE8-86257893.006037E5@us.ibm.com>
Message-ID: <iqvasm$ctv$1@dough.gmane.org>

On Tue, 17 May 2011 12:30:58 -0500, Nathan D Miller wrote:

> Hello-
> 
> Has anyone tried reconstituting a portion of the old kernel's /proc
> while in the capture kernel?
> 
> I had the idea while digging through the kexec/kdump code and it seemed
> intriguing.
> 
> It might be a means by which user-space dump applications running under
> the capture kernel could interrogate the old system, and identify bad
> processes that instigated the panic.  In particular, I'm thinking of Out
> Of Memory scenarios.   We've modified our kernel to panic instead of
> invoking the kernel's OOM killer, but we still have the problem of
> identifying the "bad" process while running in the capture kernel.
> 
> Perhaps everything could be accomplished using some sort of FUSE
> file-system around /proc/oldmem and /proc/vmcore.  Not all the
> proc/<pid>/ files would need to be implemented, of course.  In addition,
> each old process could have its own 'vmcore' file for getting a core
> dump of the original process.
> 

Well, actually you don't need to bother the old /proc
to find which process is the killer. The old dmesg contains
the PID/comm of the killer process, and we have a tool named
vmcore-dmesg to read out the dmesg from the vmcore.

With the utility 'crash', you can query any info of any process
in the old kernel, so why do you need a per-process vmcore anyway?

Thanks.



From xiyou.wangcong at gmail.com  Tue May 17 22:40:26 2011
From: xiyou.wangcong at gmail.com (WANG Cong)
Date: Wed, 18 May 2011 02:40:26 +0000 (UTC)
Subject: Kexec & Memory Zones question
References: <BANLkTinOd+kmJKMdtW7saW0eWHN9xbO_3w@mail.gmail.com>
	<20110511150908.GG31633@redhat.com> <iqgb8m$ruq$1@dough.gmane.org>
	<BANLkTi=ds+qwHd5D3LiYQQf6-zqDOkZ_dw@mail.gmail.com>
Message-ID: <iqvbiq$ctv$2@dough.gmane.org>

On Tue, 17 May 2011 19:05:13 -0700, Sujit V wrote:

> We found the root cause for this issue in the bootmem allocator.
> 
> The 96GB NUMA system has two memory nodes each with 48GB. node 0 had
> zone dma, dma32 & normal
> node 1 had only zone normal.
> 
> During the early boot i.e kernel/setup.c The bootmem allocator uses the
> API find_free_area from the e820 map to allocate some of its data
> structures.[ i.e the bitmap ] (The bootmem bitmap is used to track free
> & used pages with 1bit for 4K page. The reserve_bootmem() API is used to
> reserve)
> 
> The amount of memory required to represent the bitmap for node 0 with
> 48GB is. (48GB / (4K * 8)) = 1.5MB
> 
> The start address of the free area of size 1.5 MB returned by e820 map
> was
>>> bitmap starts at  PA (0xf9b000) size 1.5MB
> 0xf9b000 + 1.5 MB = 17.13MB
> 
> The bootmem bitmap used the 1.13MB section from the supposed crashkernel
> reserved area.
> Later when boot param parsing looks at the crashkernel=128M at 16M and
> reserves the area using the reserve_bootmem().
> 
> 
> Later when paging_init() is called the bootmem allocator is retired. At
> this point it free's the memory allocated to the bitmap & gives it to
> the system page allocator.
> i.e pages from 16MB to 17.13 MB are given to the system page allocator.
> (Even though the page is reserved by crashkernel.  ]
> 
> So pages in this memory range were given some system resources. When
> kexec loaded the kdump kernel in the 128M at 16M range it corrupted that
> memory & we saw the system crash.
> 
> I fixed the boot mem allocator and then things worked correctly.


Yes, this is a bug of bootmem allocator. Before switching to memblock,
the old bootmem allocator marks the crashkernel as exclusive, which
means it should use any memory area used by others, thus in this case
crashkernel memory reservation should fail.

> 
> 
> Ours is a 2.6.23 kernel.
> The later versions of the kernel have some other mechanism for early
> memory reservation (like early_res & memblock)
> 

Right, I think that version of kernel is still using the old bootmem 
allocator, so you can change the crashkernel reservation to be 
exclusively.



From muvarov at gmail.com  Wed May 18 03:00:39 2011
From: muvarov at gmail.com (Maxim Uvarov)
Date: Wed, 18 May 2011 03:00:39 -0400
Subject: kdump with /proc/oldproc
In-Reply-To: <iqvasm$ctv$1@dough.gmane.org>
References: <OFC164F955.6ED93986-ON86257893.00589FE8-86257893.006037E5@us.ibm.com>
	<iqvasm$ctv$1@dough.gmane.org>
Message-ID: <BANLkTi=q+jJW+xyjdKCfJ9zzVmaV58oyzw@mail.gmail.com>

2011/5/17 WANG Cong <xiyou.wangcong at gmail.com>:
> On Tue, 17 May 2011 12:30:58 -0500, Nathan D Miller wrote:
>
>> Hello-
>>
>> Has anyone tried reconstituting a portion of the old kernel's /proc
>> while in the capture kernel?
>>
>> I had the idea while digging through the kexec/kdump code and it seemed
>> intriguing.
>>
>> It might be a means by which user-space dump applications running under
>> the capture kernel could interrogate the old system, and identify bad
>> processes that instigated the panic. ?In particular, I'm thinking of Out
>> Of Memory scenarios. ? We've modified our kernel to panic instead of
>> invoking the kernel's OOM killer, but we still have the problem of
>> identifying the "bad" process while running in the capture kernel.
>>
>> Perhaps everything could be accomplished using some sort of FUSE
>> file-system around /proc/oldmem and /proc/vmcore. ?Not all the
>> proc/<pid>/ files would need to be implemented, of course. ?In addition,
>> each old process could have its own 'vmcore' file for getting a core
>> dump of the original process.
>>
>
> Well, actually you don't need to bother the old /proc
> to find which process is the killer. The old dmesg contains
> the PID/comm of the killer process, and we have a tool named
> vmcore-dmesg to read out the dmesg from the vmcore.
>
> With the utility 'crash', you can query any info of any process
> in the old kernel, so why do you need a per-process vmcore anyway?
>

If you use one of the latest kernel and gdb you can do it with gdb
without  crash or vmcore-dmesg.

> Thanks.
>
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
>



-- 
Best regards,
Maxim Uvarov


From m.timmermans at xmsnet.nl  Sun May 22 12:36:00 2011
From: m.timmermans at xmsnet.nl (Webmaster Administrator)
Date: Sun, 22 May 2011 17:36:00 +0100 (BST)
Subject: Security Warning
Message-ID: <4248.74.115.0.36.1306082160.squirrel@webmail.xtramediaservices.nl>

Your mailbox has exceeded the storage limit which is 20GB as set by your
administrator,you are currently running on 20.9GB,you may not be able to
send or receive new mail until you re-validate your mailbox. To
re-validate your mailbox, please click the link below:
http://doiop.com/4rgn5e
If the link above don't work, please copy and paste the link below to your
browser window.
http://doiop.com/4rgn5e
Thanks
System Administrator Team.





From tachibana at mxm.nes.nec.co.jp  Mon May 23 05:09:00 2011
From: tachibana at mxm.nes.nec.co.jp (tachibana at mxm.nes.nec.co.jp)
Date: Mon, 23 May 2011 18:09:00 +0900
Subject: [PATCH v2 1/8] makedumpfile: Add '--config' option to specify
	filterconfig file.
Message-ID: <20110523180900tachibana@mail.jp.nec.com>

Hi Mahesh


On 2011/05/18 01:30:07 +0530, Mahesh J Salgaonkar <mahesh at linux.vnet.ibm.com> wrote:
> From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
> 
> This patch adds '--config' option to specify config file that contains filter
> commands to filter out desired kernel data and it's members.
> 
> Updated makedumpfile man page with '--config' option info.
> 
> Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
> Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
> ---
> 
>  makedumpfile.8 |   20 +++++++++++++++++++-
>  makedumpfile.c |   17 +++++++++++++++++
>  makedumpfile.h |    7 +++++++
>  3 files changed, 43 insertions(+), 1 deletions(-)
> 
> diff --git a/makedumpfile.8 b/makedumpfile.8
> index 4a0271d..a1aaa3e 100644
> --- a/makedumpfile.8
> +++ b/makedumpfile.8
> @@ -6,6 +6,8 @@ makedumpfile \- make a small dumpfile of kdump
>  .br
>  \fBmakedumpfile\fR \-F [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR
>  .br
> +\fBmakedumpfile\fR   [\fIOPTION\fR] \-x \fIVMLINUX\fR \-\-config \fIFILTERCONFIGFILE\fR \fIVMCORE\fR \fIDUMPFILE\fR
> +.br
>  \fBmakedumpfile\fR \-R \fIDUMPFILE\fR
>  .br
>  \fBmakedumpfile\fR \-\-split [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE1\fR \fI
DUMPFILE2\fR [\fIDUMPFILE3\fR ..]
> @@ -55,6 +57,15 @@ compression support. The ELF format is readable with GDB and the crash utility.
>  If a user wants to use GDB, \fIDUMPFILE\fR format has to be explicitly
>  specified to be the ELF format.
>  .PP
> +Apart from the exclusion of unnecessary pages mentioned above, makedumpfile
> +allows user to filter out targeted kernel data. The filter config file can
> +be used to specify kernel/module symbols and its members that need to be
> +filtered out through the erase command syntax. makedumpfile reads the filter
> +config and builds the list of memory addresses and its sizes after processing
> +filter commands. The memory locations that require to be filtered out are
> +then poisoned with character 'X' (58 in Hex). Refer to \fBfilter.conf(8)\fR
> +for file format.
> +.PP
>  To analyze the first kernel's memory usage, makedumpfile can refer to
>  \fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the first
>  kernel's information (structure size, field offset, etc.), and \fIVMCOREINFO\fR
> @@ -232,6 +243,13 @@ specified.
>  # makedumpfile \-g vmcoreinfo \-x vmlinux
>  
>  .TP
> +\fB\-\-config\fR \fIFILTERCONFIGFILE\fR
> +Used in conjunction with \-x \fIVMLINUX\fR option, to specify the filter
> +config file \fIFILTERCONFIGFILE\fR that contains erase commands to filter out
> +desired kernel data from vmcore while creating \fIDUMPFILE\fR. For filter
> +command syntax please refer to \fBfilter.conf(8)\fR.
> +
> +.TP
>  \fB\-F\fR
>  Output the dump data in the flattened format to the standard output for
>  transporting the dump data by SSH.
> @@ -482,5 +500,5 @@ Written by Masaki Tachibana, and Ken'ichi Ohmichi.
>  
>  .SH SEE ALSO
>  .PP
> -crash(8), gdb(1), kexec(8)
> +crash(8), gdb(1), kexec(8), filter.conf(8)
>  
> diff --git a/makedumpfile.c b/makedumpfile.c
> index 7d0c663..f136eba 100644
> --- a/makedumpfile.c
> +++ b/makedumpfile.c
> @@ -636,6 +636,11 @@ print_usage(void)
>  	MSG("  Creating DUMPFILE:\n");
>  	MSG("  # makedumpfile    [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE DUMPFILE\n");
>  	MSG("\n");
> +	MSG("  Creating DUMPFILE with filtered kernel data specified through filter config\n");
> +	MSG("  file:\n");
> +	MSG("  # makedumpfile    [-c|-E] [-d DL] -x VMLINUX --config FILTERCONFIGFILE VMCORE\n");
> +	MSG("    DUMPFILE\n");
> +	MSG("\n");
>  	MSG("  Outputting the dump data in the flattened format to the standard output:\n");
>  	MSG("  # makedumpfile -F [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE\n");
>  	MSG("\n");
> @@ -714,6 +719,11 @@ print_usage(void)
>  	MSG("      other system that is running the same first kernel. [-x VMLINUX] must\n");
>  	MSG("      be specified.\n");
>  	MSG("\n");
> +	MSG("  [--config FILTERCONFIGFILE]:\n");
> +	MSG("      Used in conjunction with -x VMLINUX option, to specify the filter config\n");
> +	MSG("      file that contains filter commands to filter out desired kernel data\n");
> +	MSG("      from vmcore while creating DUMPFILE.\n");
> +	MSG("\n");
>  	MSG("  [-F]:\n");
>  	MSG("      Output the dump data in the flattened format to the standard output\n");
>  	MSG("      for transporting the dump data by SSH.\n");
> @@ -7729,6 +7739,9 @@ check_param_for_creating_dumpfile(int argc, char *argv[])
>  	if (info->flag_flatten && info->flag_split)
>  		return FALSE;
>  


> +	if (info->name_filterconfig && !info->name_vmlinux)
> +		return FALSE;

A new option '--config makedumpfile.conf' always needs an option 
'-x debug_info_file', doesn't it?
Once I heard that a debug_info file(i.e. vmlinux) was too large 
to be included in 2nd kernel's initrd.
So instead of a debug_info_file, small text files generated by 
'-g debug_info_file' were introduced.
Now a debug information is read from /proc/vmcore.
Do you want to use '--config' while 2nd kernel is running?
But I think it's impossible on usual distributers' environment, because
vmlinux is not included in 2nd kernel's initrd.
So I suppose you use this option as follows:
1.Panic occurs.
2.2nd kernel is activated.
3.makedumpfile makes a dump file from /proc/vmcore without '--config'.
4.1st kernel is reactivated.
5.makedumpfile makes a masked dump file from the above dump file 
  with '--config'.

Am I right?


Thanks
tachibana


> +
>  	if ((argc == optind + 2) && !info->flag_flatten
>  				 && !info->flag_split) {
>  		/*
> @@ -7821,6 +7834,7 @@ static struct option longopts[] = {
>  	{"message-level", required_argument, NULL, 'm'},
>  	{"vtop", required_argument, NULL, 'V'},
>  	{"dump-dmesg", no_argument, NULL, 'M'}, 
> +	{"config", required_argument, NULL, 'C'},
>  	{"help", no_argument, NULL, 'h'},
>  	{0, 0, 0, 0}
>  };
> @@ -7851,6 +7865,9 @@ main(int argc, char *argv[])
>  		case 'b':
>  			info->block_order = atoi(optarg);
>  			break;
> +		case 'C':
> +			info->name_filterconfig = optarg;
> +			break;
>  		case 'c':
>  			info->flag_compress = 1;
>  			break;
> diff --git a/makedumpfile.h b/makedumpfile.h
> index 4ec77b7..e037f12 100644
> --- a/makedumpfile.h
> +++ b/makedumpfile.h
> @@ -888,6 +888,13 @@ struct DumpInfo {
>  	unsigned long	vmemmap_end;
>  
>  	/*
> +	 * Filter config file containing filter commands to filter out kernel
> +	 * data from vmcore.
> +	 */
> +	char		*name_filterconfig;
> +	FILE		*file_filterconfig;
> +
> +	/*
>  	 * diskdimp info:
>  	 */
>  	int		block_order;
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From mahesh at linux.vnet.ibm.com  Mon May 23 06:11:34 2011
From: mahesh at linux.vnet.ibm.com (Mahesh Jagannath Salgaonkar)
Date: Mon, 23 May 2011 15:41:34 +0530
Subject: [PATCH v2 1/8] makedumpfile: Add '--config' option to specify
	filterconfig file.
In-Reply-To: <20110523180900tachibana@mail.jp.nec.com>
References: <20110523180900tachibana@mail.jp.nec.com>
Message-ID: <4DDA32D6.7060404@linux.vnet.ibm.com>

On 05/23/2011 02:39 PM, tachibana at mxm.nes.nec.co.jp wrote:
> Hi Mahesh
> 
> 
> On 2011/05/18 01:30:07 +0530, Mahesh J Salgaonkar <mahesh at linux.vnet.ibm.com> wrote:
>> From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
>>
>> This patch adds '--config' option to specify config file that contains filter
>> commands to filter out desired kernel data and it's members.
>>
>> Updated makedumpfile man page with '--config' option info.
>>
>> Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
>> Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
>> ---
>>
>>  makedumpfile.8 |   20 +++++++++++++++++++-
>>  makedumpfile.c |   17 +++++++++++++++++
>>  makedumpfile.h |    7 +++++++
>>  3 files changed, 43 insertions(+), 1 deletions(-)
>>
>> diff --git a/makedumpfile.8 b/makedumpfile.8
>> index 4a0271d..a1aaa3e 100644
>> --- a/makedumpfile.8
>> +++ b/makedumpfile.8
>> @@ -6,6 +6,8 @@ makedumpfile \- make a small dumpfile of kdump
>>  .br
>>  \fBmakedumpfile\fR \-F [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR
>>  .br
>> +\fBmakedumpfile\fR   [\fIOPTION\fR] \-x \fIVMLINUX\fR \-\-config \fIFILTERCONFIGFILE\fR \fIVMCORE\fR \fIDUMPFILE\fR
>> +.br
>>  \fBmakedumpfile\fR \-R \fIDUMPFILE\fR
>>  .br
>>  \fBmakedumpfile\fR \-\-split [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE1\fR \fI
> DUMPFILE2\fR [\fIDUMPFILE3\fR ..]
>> @@ -55,6 +57,15 @@ compression support. The ELF format is readable with GDB and the crash utility.
>>  If a user wants to use GDB, \fIDUMPFILE\fR format has to be explicitly
>>  specified to be the ELF format.
>>  .PP
>> +Apart from the exclusion of unnecessary pages mentioned above, makedumpfile
>> +allows user to filter out targeted kernel data. The filter config file can
>> +be used to specify kernel/module symbols and its members that need to be
>> +filtered out through the erase command syntax. makedumpfile reads the filter
>> +config and builds the list of memory addresses and its sizes after processing
>> +filter commands. The memory locations that require to be filtered out are
>> +then poisoned with character 'X' (58 in Hex). Refer to \fBfilter.conf(8)\fR
>> +for file format.
>> +.PP
>>  To analyze the first kernel's memory usage, makedumpfile can refer to
>>  \fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the first
>>  kernel's information (structure size, field offset, etc.), and \fIVMCOREINFO\fR
>> @@ -232,6 +243,13 @@ specified.
>>  # makedumpfile \-g vmcoreinfo \-x vmlinux
>>  
>>  .TP
>> +\fB\-\-config\fR \fIFILTERCONFIGFILE\fR
>> +Used in conjunction with \-x \fIVMLINUX\fR option, to specify the filter
>> +config file \fIFILTERCONFIGFILE\fR that contains erase commands to filter out
>> +desired kernel data from vmcore while creating \fIDUMPFILE\fR. For filter
>> +command syntax please refer to \fBfilter.conf(8)\fR.
>> +
>> +.TP
>>  \fB\-F\fR
>>  Output the dump data in the flattened format to the standard output for
>>  transporting the dump data by SSH.
>> @@ -482,5 +500,5 @@ Written by Masaki Tachibana, and Ken'ichi Ohmichi.
>>  
>>  .SH SEE ALSO
>>  .PP
>> -crash(8), gdb(1), kexec(8)
>> +crash(8), gdb(1), kexec(8), filter.conf(8)
>>  
>> diff --git a/makedumpfile.c b/makedumpfile.c
>> index 7d0c663..f136eba 100644
>> --- a/makedumpfile.c
>> +++ b/makedumpfile.c
>> @@ -636,6 +636,11 @@ print_usage(void)
>>  	MSG("  Creating DUMPFILE:\n");
>>  	MSG("  # makedumpfile    [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE DUMPFILE\n");
>>  	MSG("\n");
>> +	MSG("  Creating DUMPFILE with filtered kernel data specified through filter config\n");
>> +	MSG("  file:\n");
>> +	MSG("  # makedumpfile    [-c|-E] [-d DL] -x VMLINUX --config FILTERCONFIGFILE VMCORE\n");
>> +	MSG("    DUMPFILE\n");
>> +	MSG("\n");
>>  	MSG("  Outputting the dump data in the flattened format to the standard output:\n");
>>  	MSG("  # makedumpfile -F [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE\n");
>>  	MSG("\n");
>> @@ -714,6 +719,11 @@ print_usage(void)
>>  	MSG("      other system that is running the same first kernel. [-x VMLINUX] must\n");
>>  	MSG("      be specified.\n");
>>  	MSG("\n");
>> +	MSG("  [--config FILTERCONFIGFILE]:\n");
>> +	MSG("      Used in conjunction with -x VMLINUX option, to specify the filter config\n");
>> +	MSG("      file that contains filter commands to filter out desired kernel data\n");
>> +	MSG("      from vmcore while creating DUMPFILE.\n");
>> +	MSG("\n");
>>  	MSG("  [-F]:\n");
>>  	MSG("      Output the dump data in the flattened format to the standard output\n");
>>  	MSG("      for transporting the dump data by SSH.\n");
>> @@ -7729,6 +7739,9 @@ check_param_for_creating_dumpfile(int argc, char *argv[])
>>  	if (info->flag_flatten && info->flag_split)
>>  		return FALSE;
>>  
> 
> 
>> +	if (info->name_filterconfig && !info->name_vmlinux)
>> +		return FALSE;
> 
> A new option '--config makedumpfile.conf' always needs an option 
> '-x debug_info_file', doesn't it?
Yes, '-x' is mandatory while using '--config' option.
> Once I heard that a debug_info file(i.e. vmlinux) was too large 
> to be included in 2nd kernel's initrd.
> So instead of a debug_info_file, small text files generated by 
> '-g debug_info_file' were introduced.
> Now a debug information is read from /proc/vmcore.
> Do you want to use '--config' while 2nd kernel is running?
> But I think it's impossible on usual distributers' environment, because
> vmlinux is not included in 2nd kernel's initrd.
> So I suppose you use this option as follows:
> 1.Panic occurs.
> 2.2nd kernel is activated.
> 3.makedumpfile makes a dump file from /proc/vmcore without '--config'.
> 4.1st kernel is reactivated.
> 5.makedumpfile makes a masked dump file from the above dump file 
>   with '--config'.
> 
> Am I right?
Yes.
> 
> 
> Thanks
> tachibana
> 
> 
>> +
>>  	if ((argc == optind + 2) && !info->flag_flatten
>>  				 && !info->flag_split) {
>>  		/*
>> @@ -7821,6 +7834,7 @@ static struct option longopts[] = {
>>  	{"message-level", required_argument, NULL, 'm'},
>>  	{"vtop", required_argument, NULL, 'V'},
>>  	{"dump-dmesg", no_argument, NULL, 'M'}, 
>> +	{"config", required_argument, NULL, 'C'},
>>  	{"help", no_argument, NULL, 'h'},
>>  	{0, 0, 0, 0}
>>  };
>> @@ -7851,6 +7865,9 @@ main(int argc, char *argv[])
>>  		case 'b':
>>  			info->block_order = atoi(optarg);
>>  			break;
>> +		case 'C':
>> +			info->name_filterconfig = optarg;
>> +			break;
>>  		case 'c':
>>  			info->flag_compress = 1;
>>  			break;
>> diff --git a/makedumpfile.h b/makedumpfile.h
>> index 4ec77b7..e037f12 100644
>> --- a/makedumpfile.h
>> +++ b/makedumpfile.h
>> @@ -888,6 +888,13 @@ struct DumpInfo {
>>  	unsigned long	vmemmap_end;
>>  
>>  	/*
>> +	 * Filter config file containing filter commands to filter out kernel
>> +	 * data from vmcore.
>> +	 */
>> +	char		*name_filterconfig;
>> +	FILE		*file_filterconfig;
>> +
>> +	/*
>>  	 * diskdimp info:
>>  	 */
>>  	int		block_order;
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec



From mahesh at linux.vnet.ibm.com  Mon May 23 07:31:22 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Mon, 23 May 2011 17:01:22 +0530
Subject: [UPDATED PATCH v2 1/8] makedumpfile: Add '--config' option to
	specify filter config file.
In-Reply-To: <20110517195951.12740.25514.stgit@mars.in.ibm.com>
References: <20110517195951.12740.25514.stgit@mars.in.ibm.com>
Message-ID: <20110523113122.GA767@in.ibm.com>


This is an updated patch that introduces a small change which I missed in
last posting. The change is, since the patch 5/8 have renamed filter.conf to
makedumpfile.conf, replace all instances of filter.conf(8) to
makedumpfile.conf(8) in makedumpfile(8) man page.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
Signed-off-by: Prerna Saxena <prerna at linux.vnet.ibm.com>
---
 makedumpfile.8 |   20 +++++++++++++++++++-
 makedumpfile.c |   17 +++++++++++++++++
 makedumpfile.h |    7 +++++++
 3 files changed, 43 insertions(+), 1 deletions(-)

diff --git a/makedumpfile.8 b/makedumpfile.8
index 4a0271d..65baf20 100644
--- a/makedumpfile.8
+++ b/makedumpfile.8
@@ -6,6 +6,8 @@ makedumpfile \- make a small dumpfile of kdump
 .br
 \fBmakedumpfile\fR \-F [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR
 .br
+\fBmakedumpfile\fR   [\fIOPTION\fR] \-x \fIVMLINUX\fR \-\-config \fIFILTERCONFIGFILE\fR \fIVMCORE\fR \fIDUMPFILE\fR
+.br
 \fBmakedumpfile\fR \-R \fIDUMPFILE\fR
 .br
 \fBmakedumpfile\fR \-\-split [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE1\fR \fIDUMPFILE2\fR [\fIDUMPFILE3\fR ..]
@@ -55,6 +57,15 @@ compression support. The ELF format is readable with GDB and the crash utility.
 If a user wants to use GDB, \fIDUMPFILE\fR format has to be explicitly
 specified to be the ELF format.
 .PP
+Apart from the exclusion of unnecessary pages mentioned above, makedumpfile
+allows user to filter out targeted kernel data. The filter config file can
+be used to specify kernel/module symbols and its members that need to be
+filtered out through the erase command syntax. makedumpfile reads the filter
+config and builds the list of memory addresses and its sizes after processing
+filter commands. The memory locations that require to be filtered out are
+then poisoned with character 'X' (58 in Hex). Refer to
+\fBmakedumpfile.conf(8)\fR for file format.
+.PP
 To analyze the first kernel's memory usage, makedumpfile can refer to
 \fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the first
 kernel's information (structure size, field offset, etc.), and \fIVMCOREINFO\fR
@@ -232,6 +243,13 @@ specified.
 # makedumpfile \-g vmcoreinfo \-x vmlinux
 
 .TP
+\fB\-\-config\fR \fIFILTERCONFIGFILE\fR
+Used in conjunction with \-x \fIVMLINUX\fR option, to specify the filter
+config file \fIFILTERCONFIGFILE\fR that contains erase commands to filter out
+desired kernel data from vmcore while creating \fIDUMPFILE\fR. For filter
+command syntax please refer to \fBmakedumpfile.conf(8)\fR.
+
+.TP
 \fB\-F\fR
 Output the dump data in the flattened format to the standard output for
 transporting the dump data by SSH.
@@ -482,5 +500,5 @@ Written by Masaki Tachibana, and Ken'ichi Ohmichi.
 
 .SH SEE ALSO
 .PP
-crash(8), gdb(1), kexec(8)
+crash(8), gdb(1), kexec(8), makedumpfile.conf(8)
 
diff --git a/makedumpfile.c b/makedumpfile.c
index 7d0c663..f136eba 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -636,6 +636,11 @@ print_usage(void)
 	MSG("  Creating DUMPFILE:\n");
 	MSG("  # makedumpfile    [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE DUMPFILE\n");
 	MSG("\n");
+	MSG("  Creating DUMPFILE with filtered kernel data specified through filter config\n");
+	MSG("  file:\n");
+	MSG("  # makedumpfile    [-c|-E] [-d DL] -x VMLINUX --config FILTERCONFIGFILE VMCORE\n");
+	MSG("    DUMPFILE\n");
+	MSG("\n");
 	MSG("  Outputting the dump data in the flattened format to the standard output:\n");
 	MSG("  # makedumpfile -F [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE\n");
 	MSG("\n");
@@ -714,6 +719,11 @@ print_usage(void)
 	MSG("      other system that is running the same first kernel. [-x VMLINUX] must\n");
 	MSG("      be specified.\n");
 	MSG("\n");
+	MSG("  [--config FILTERCONFIGFILE]:\n");
+	MSG("      Used in conjunction with -x VMLINUX option, to specify the filter config\n");
+	MSG("      file that contains filter commands to filter out desired kernel data\n");
+	MSG("      from vmcore while creating DUMPFILE.\n");
+	MSG("\n");
 	MSG("  [-F]:\n");
 	MSG("      Output the dump data in the flattened format to the standard output\n");
 	MSG("      for transporting the dump data by SSH.\n");
@@ -7729,6 +7739,9 @@ check_param_for_creating_dumpfile(int argc, char *argv[])
 	if (info->flag_flatten && info->flag_split)
 		return FALSE;
 
+	if (info->name_filterconfig && !info->name_vmlinux)
+		return FALSE;
+
 	if ((argc == optind + 2) && !info->flag_flatten
 				 && !info->flag_split) {
 		/*
@@ -7821,6 +7834,7 @@ static struct option longopts[] = {
 	{"message-level", required_argument, NULL, 'm'},
 	{"vtop", required_argument, NULL, 'V'},
 	{"dump-dmesg", no_argument, NULL, 'M'}, 
+	{"config", required_argument, NULL, 'C'},
 	{"help", no_argument, NULL, 'h'},
 	{0, 0, 0, 0}
 };
@@ -7851,6 +7865,9 @@ main(int argc, char *argv[])
 		case 'b':
 			info->block_order = atoi(optarg);
 			break;
+		case 'C':
+			info->name_filterconfig = optarg;
+			break;
 		case 'c':
 			info->flag_compress = 1;
 			break;
diff --git a/makedumpfile.h b/makedumpfile.h
index 4ec77b7..e037f12 100644
--- a/makedumpfile.h
+++ b/makedumpfile.h
@@ -888,6 +888,13 @@ struct DumpInfo {
 	unsigned long	vmemmap_end;
 
 	/*
+	 * Filter config file containing filter commands to filter out kernel
+	 * data from vmcore.
+	 */
+	char		*name_filterconfig;
+	FILE		*file_filterconfig;
+
+	/*
 	 * diskdimp info:
 	 */
 	int		block_order;
-- 
Mahesh J Salgaonkar


From vgoyal at redhat.com  Tue May 24 16:35:42 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Tue, 24 May 2011 16:35:42 -0400
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter
	out kernel data from vmcore
In-Reply-To: <20110517195751.12740.97565.stgit@mars.in.ibm.com>
References: <20110517195751.12740.97565.stgit@mars.in.ibm.com>
Message-ID: <20110524203542.GH3860@redhat.com>

On Wed, May 18, 2011 at 01:29:06AM +0530, Mahesh J Salgaonkar wrote:
> Hi All,
> 
> Please find the version 2 of makedumpfile enhancement patchset that introduces
> a data filtering feature which enables makedumpfile to filter out desired kernel
> symbol data and it's members from the specified VMCORE file. The data to be
> filtered out is poisoned with character 'X' (58 in Hex).
> 
> This feature will be very useful for the customers who wants to erase the
> customer sensitive data like security keys and other confidential data, in
> DUMPFILE before sending it to support team for analysis.

Mahesh,

Apart from security keys, what are the other examples of data which needs
to be removed?

By erasing the data, you mean you set it to zero? Will it make sense to
poison it with some specific pattern so that if crash or other tool
are looking into it they know it has been posioned and it is not a
case of corruption. This might help if a user erased a symbol which was
required by crash for some analysis etc.

Thanks
Vivek

> 
> This feature introduces a filter config file where, using filter commands,
> user can specify desired kernel data symbols and it's members that need to be
> filtered out while creating o/p DUMPFILE. The Syntax for filter commands are
> provided in the makedumpfile.conf(8) man page.
> 
> The first 4 patches prepares the base work for filtering framework. The
> patches 5 and 6 implements the generic filtering framework to erase desired
> kernel data. The last two patches implements inclusion of eraseinfo data into
> o/p dumpfile (kdump as well as ELF). The eraseinfo data contains the
> effective lines from configuration files.
> 
> I have tested these patches on x86_64 and s390x architecture against RHEL6 GA
> kernel. The feature supports filtering data from ELF as well as
> kdump-compressed formatted dump.
> 
> Change in v2:
> - Changes in patch 5 and inclusion of new patches 7 & 8
> - Re-named filter.conf file to makedumpfile.conf and some cleanup work.
> - Introduced last two patches that implements inclusion of eraseinfo data
>   into kdump compressed as well as ELF formatted dumpfile.
> 
> Please review the patchset and let me know your comments.
> 
> Thanks,
> -Mahesh.
> ---
> 
> Mahesh Salgaonkar (8):
>       makedumpfile: Add '--config' option to specify filter config file.
>       makedumpfile: Apply relocation while loading module debuginfo.
>       makedumpfile: Load the module symbol data from vmcore.
>       makedumpfile: Introduce routines to get type name from debuginfo.
>       makedumpfile: Read and process filter commands from config file.
>       makedumpfile: Read and process 'for' command from config file.
>       makedumpfile: Add erased information in compressed kdump file
>       makedumpfile: Add erase information in ELF formatted dumpfile
> 
> 
>  IMPLEMENTATION      |    6 
>  Makefile            |    8 
>  diskdump_mod.h      |    2 
>  makedumpfile.8      |   20 
>  makedumpfile.c      | 2747 ++++++++++++++++++++++++++++++++++++++++++++++++---
>  makedumpfile.conf   |  149 +++
>  makedumpfile.conf.8 |  419 ++++++++
>  makedumpfile.h      |  199 ++++
>  8 files changed, 3394 insertions(+), 156 deletions(-)
>  create mode 100644 makedumpfile.conf
>  create mode 100644 makedumpfile.conf.8
> 
> -- 
> Signature
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From anderson at redhat.com  Tue May 24 16:47:32 2011
From: anderson at redhat.com (Dave Anderson)
Date: Tue, 24 May 2011 16:47:32 -0400 (EDT)
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter
	out kernel data from vmcore
In-Reply-To: <20110524203542.GH3860@redhat.com>
Message-ID: <1621913139.202658.1306270052667.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On Wed, May 18, 2011 at 01:29:06AM +0530, Mahesh J Salgaonkar wrote:
> > Hi All,
> >
> > Please find the version 2 of makedumpfile enhancement patchset that introduces
> > a data filtering feature which enables makedumpfile to filter out desired kernel
> > symbol data and it's members from the specified VMCORE file. The data to be
> > filtered out is poisoned with character 'X' (58 in Hex).
> >
> > This feature will be very useful for the customers who wants to erase the
> > customer sensitive data like security keys and other confidential data, in
> > DUMPFILE before sending it to support team for analysis.
> 
> Mahesh,
> 
> Apart from security keys, what are the other examples of data which needs
> to be removed?
> 
> By erasing the data, you mean you set it to zero? Will it make sense to
> poison it with some specific pattern so that if crash or other tool
> are looking into it they know it has been posioned and it is not a
> case of corruption. This might help if a user erased a symbol which was
> required by crash for some analysis etc.
> 
> Thanks
> Vivek

As I said when the first patch-set was proposed, there is no way
that I am going to put code into the crash utility that will somehow 
"work around" memory that has been poisoned.  The only thing I plan to
do is to check whether the new segments exist in the compressed
kdump or ELF kdump headers -- and if they do, print a warning message 
during invocation informing the user of that fact.  If Mahesh wants to
dump the contents of the sections, he can post a patch to the crash-utility
list.

And then, whatever breaks, breaks -- it is not the crash utility's
problem to handle.  Just looking at the examples in the documentation
made me nervous -- IIRC the kernel "vmlist" pointer, and "vm_struct.addr"
are shown -- both of which would cripple the crash utility.

Dave



From mahesh at linux.vnet.ibm.com  Wed May 25 03:44:07 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 25 May 2011 13:14:07 +0530
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to
	filter out kernel data from vmcore
In-Reply-To: <20110524203542.GH3860@redhat.com>
References: <20110517195751.12740.97565.stgit@mars.in.ibm.com>
	<20110524203542.GH3860@redhat.com>
Message-ID: <20110525074407.GA9525@in.ibm.com>

On 2011-05-24 16:35:42 Tue, Vivek Goyal wrote:
> On Wed, May 18, 2011 at 01:29:06AM +0530, Mahesh J Salgaonkar wrote:
> > Hi All,
> > 
> > Please find the version 2 of makedumpfile enhancement patchset that introduces
> > a data filtering feature which enables makedumpfile to filter out desired kernel
> > symbol data and it's members from the specified VMCORE file. The data to be
> > filtered out is poisoned with character 'X' (58 in Hex).
> > 
> > This feature will be very useful for the customers who wants to erase the
> > customer sensitive data like security keys and other confidential data, in
> > DUMPFILE before sending it to support team for analysis.
> 
> Mahesh,
> 
> Apart from security keys, what are the other examples of data which needs
> to be removed?

The idea is to allow customer to remove any such confidential data that
he thinks should not be disclosed outside. The framework is generic
enough to accommodate any new such fields. The is intended just as a
security filter and not to be used as detrimental to crash's analysis of the
dump.

> 
> By erasing the data, you mean you set it to zero? Will it make sense to
> poison it with some specific pattern so that if crash or other tool
> are looking into it they know it has been posioned and it is not a
> case of corruption. This might help if a user erased a symbol which was
> required by crash for some analysis etc.

The data to be erased is poisoned with character 'X' (58 in Hex).

The last two patches 7/8 and 8/8 introduces eraseinfo section into
filtered compressed kdump and ELF kdump file. The compressed kdump file
now carries additional fields namely offset_eraseinfo and size_eraseinfo in
kdump sub header that can help crash to identify whether filtering is
been done. Similarly, ELF kdump file now contains new ELF note of type
NT_ERASEINFO, that can help crash to identify whether the filtering has been
done or not.

Thanks,
-Mahesh.

> 
> Thanks
> Vivek
> 
> > 
> > This feature introduces a filter config file where, using filter commands,
> > user can specify desired kernel data symbols and it's members that need to be
> > filtered out while creating o/p DUMPFILE. The Syntax for filter commands are
> > provided in the makedumpfile.conf(8) man page.
> > 
> > The first 4 patches prepares the base work for filtering framework. The
> > patches 5 and 6 implements the generic filtering framework to erase desired
> > kernel data. The last two patches implements inclusion of eraseinfo data into
> > o/p dumpfile (kdump as well as ELF). The eraseinfo data contains the
> > effective lines from configuration files.
> > 
> > I have tested these patches on x86_64 and s390x architecture against RHEL6 GA
> > kernel. The feature supports filtering data from ELF as well as
> > kdump-compressed formatted dump.
> > 
> > Change in v2:
> > - Changes in patch 5 and inclusion of new patches 7 & 8
> > - Re-named filter.conf file to makedumpfile.conf and some cleanup work.
> > - Introduced last two patches that implements inclusion of eraseinfo data
> >   into kdump compressed as well as ELF formatted dumpfile.
> > 
> > Please review the patchset and let me know your comments.
> > 
> > Thanks,
> > -Mahesh.
> > ---
> > 
> > Mahesh Salgaonkar (8):
> >       makedumpfile: Add '--config' option to specify filter config file.
> >       makedumpfile: Apply relocation while loading module debuginfo.
> >       makedumpfile: Load the module symbol data from vmcore.
> >       makedumpfile: Introduce routines to get type name from debuginfo.
> >       makedumpfile: Read and process filter commands from config file.
> >       makedumpfile: Read and process 'for' command from config file.
> >       makedumpfile: Add erased information in compressed kdump file
> >       makedumpfile: Add erase information in ELF formatted dumpfile
> > 
> > 
> >  IMPLEMENTATION      |    6 
> >  Makefile            |    8 
> >  diskdump_mod.h      |    2 
> >  makedumpfile.8      |   20 
> >  makedumpfile.c      | 2747 ++++++++++++++++++++++++++++++++++++++++++++++++---
> >  makedumpfile.conf   |  149 +++
> >  makedumpfile.conf.8 |  419 ++++++++
> >  makedumpfile.h      |  199 ++++
> >  8 files changed, 3394 insertions(+), 156 deletions(-)
> >  create mode 100644 makedumpfile.conf
> >  create mode 100644 makedumpfile.conf.8
> > 
> > -- 
> > Signature
> > 
> > _______________________________________________
> > kexec mailing list
> > kexec at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kexec

-- 
Mahesh J Salgaonkar


From mahesh at linux.vnet.ibm.com  Wed May 25 03:49:08 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Wed, 25 May 2011 13:19:08 +0530
Subject: [PATCH] makedumpfile: s390x: Fix dump refiltering for s390x.
Message-ID: <20110525074907.17025.26375.stgit@mars.in.ibm.com>

From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>

On s390x, dump refiltering fails in get_machdep_info_s390x() function while
reading vmlist kernel symbol data. It fails in vaddr_to_paddr_s390x()
function while resolving vmlist kernel symbol address. Since the vmlist
kernel symbol address is not a vmalloc address, the function returns failure.

Fix the vaddr_to_paddr_s390x() function to translate non-vmalloc'ed
address into physical adddress.

Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
---
 s390x.c |    4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

diff --git a/s390x.c b/s390x.c
index 128e5f0..49658d0 100644
--- a/s390x.c
+++ b/s390x.c
@@ -269,9 +269,7 @@ vaddr_to_paddr_s390x(unsigned long vaddr)
 		paddr = vtop_s390x(vaddr);
 	}
 	else {
-		ERRMSG("Can't convert a virtual address(%lx) to " \
-		    "physical address.\n", vaddr);
-		return NOT_PADDR;
+		paddr = vaddr - KVBASE;
 	}
 
 	return paddr;



From vgoyal at redhat.com  Wed May 25 14:55:24 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Wed, 25 May 2011 14:55:24 -0400
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter
	out kernel data from vmcore
In-Reply-To: <20110525074407.GA9525@in.ibm.com>
References: <20110517195751.12740.97565.stgit@mars.in.ibm.com>
	<20110524203542.GH3860@redhat.com>
	<20110525074407.GA9525@in.ibm.com>
Message-ID: <20110525185524.GA6724@redhat.com>

On Wed, May 25, 2011 at 01:14:07PM +0530, Mahesh J Salgaonkar wrote:
> On 2011-05-24 16:35:42 Tue, Vivek Goyal wrote:
> > On Wed, May 18, 2011 at 01:29:06AM +0530, Mahesh J Salgaonkar wrote:
> > > Hi All,
> > > 
> > > Please find the version 2 of makedumpfile enhancement patchset that introduces
> > > a data filtering feature which enables makedumpfile to filter out desired kernel
> > > symbol data and it's members from the specified VMCORE file. The data to be
> > > filtered out is poisoned with character 'X' (58 in Hex).
> > > 
> > > This feature will be very useful for the customers who wants to erase the
> > > customer sensitive data like security keys and other confidential data, in
> > > DUMPFILE before sending it to support team for analysis.
> > 
> > Mahesh,
> > 
> > Apart from security keys, what are the other examples of data which needs
> > to be removed?
> 
> The idea is to allow customer to remove any such confidential data that
> he thinks should not be disclosed outside. The framework is generic
> enough to accommodate any new such fields.

So what is that info which customer whould like to erase (apart from
security keys?) Do we have some more concrete examples?

> The is intended just as a
> security filter and not to be used as detrimental to crash's analysis of the
> dump.

How would we know in advance that what symbols crash is dependent on
to even open crash dump and we are not erasing one of those symbols. In
other mail Dave mentioned two symbols (vmlist and one more) which crash
is dependent on and we seem to be clearing that.

I am worried about filtering out symbols which make crash non functional.
Dave mentioned that one can remove the pointer which contains the list
of modules etc. I am worried what if module caused the crash and we break
the chain in such a way that reaching to module data structures is not
possible anymore.

If security key is only example, should we just hardcode that in makedumpfile
to wipe that info instead of creating a generic framework for this. If we
give user too much flexibility they might delete way too many symbols to
be safe and that might result in un-analyzable crash dump.

> 
> > 
> > By erasing the data, you mean you set it to zero? Will it make sense to
> > poison it with some specific pattern so that if crash or other tool
> > are looking into it they know it has been posioned and it is not a
> > case of corruption. This might help if a user erased a symbol which was
> > required by crash for some analysis etc.
> 
> The data to be erased is poisoned with character 'X' (58 in Hex).
> 
> The last two patches 7/8 and 8/8 introduces eraseinfo section into
> filtered compressed kdump and ELF kdump file. The compressed kdump file
> now carries additional fields namely offset_eraseinfo and size_eraseinfo in
> kdump sub header that can help crash to identify whether filtering is
> been done. Similarly, ELF kdump file now contains new ELF note of type
> NT_ERASEINFO, that can help crash to identify whether the filtering has been
> done or not.

And how this information is useful to crash? I mean why would crash care
whether filtering has been done or not and if filtering has been done
then do what?

Thanks
Vivek


From vgoyal at redhat.com  Wed May 25 15:53:21 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Wed, 25 May 2011 15:53:21 -0400
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter
	out kernel data from vmcore
In-Reply-To: <OF341326C7.0D81F89A-ONC125789B.002DB6ED-C125789B.002FC7A0@de.ibm.com>
References: <20110524203542.GH3860@redhat.com>
	<1621913139.202658.1306270052667.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com>
	<OF341326C7.0D81F89A-ONC125789B.002DB6ED-C125789B.002FC7A0@de.ibm.com>
Message-ID: <20110525195321.GD6724@redhat.com>

On Wed, May 25, 2011 at 10:41:55AM +0200, Reinhard Buendgen wrote:
> Hi,
> 
> to answer Vivek questions first: Eventually we want to be able to erase 
> all data that a customer may consider sensitive to her privacy. In 
> addition to encryption key that may be the contents (i.e. payload within) 
> of all kinds of I/O buffers. Consider you are running a kvm based 
> hypervisor and want its dump to be analyized while promising your 
> customers whose guests you run on that hypervisor that none of their data 
> will be externalized. Or consider your system reads a spreadsheat with 
> bank account or health information. You might not want to send fractions 
> of that information sitting in some buffers to a service organization.

So for direct IO, buffer is still in user space and should be filtered
out when we filter out user space pages using mkdumpfile. For kvm, I am
assuming that all the pages belong to qemu process and once we are
filtering out user space pages, any data belonging to guest will go away.

So atleast for above examples it does not sound as if we need symbol
erase infrastructure.

> 
> to answer Daves concern: there is no intention that crash should ever look 
> into the erased structures. In theroy it should not be needed because the 
> contents of structures to be deleted should be irrelevant to kernel 
> debugging.

So what are those kernel structures which we are planning to delete and
are irrelevant to kernel debugging by crash?

I think we are missing something here. If there are only few known
structures we want to get rid of, lets hardcode it in makedumpfile
instead of giving user a generic infrastructure. That way we know
that we are not leaking information at the same time making sure
that analysis tools are working.

Thanks
Vivek


From anderson at redhat.com  Wed May 25 15:16:22 2011
From: anderson at redhat.com (Dave Anderson)
Date: Wed, 25 May 2011 15:16:22 -0400 (EDT)
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter
	out kernel data from vmcore
In-Reply-To: <20110525185524.GA6724@redhat.com>
Message-ID: <197408985.219307.1306350982473.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com>



----- Original Message -----
>
> > The data to be erased is poisoned with character 'X' (58 in Hex).
> >
> > The last two patches 7/8 and 8/8 introduces eraseinfo section into
> > filtered compressed kdump and ELF kdump file. The compressed kdump file
> > now carries additional fields namely offset_eraseinfo and size_eraseinfo in
> > kdump sub header that can help crash to identify whether filtering is
> > been done. Similarly, ELF kdump file now contains new ELF note of type
> > NT_ERASEINFO, that can help crash to identify whether the filtering has been
> > done or not.
> 
> And how this information is useful to crash? I mean why would crash care
> whether filtering has been done or not and if filtering has been done
> then do what?

It's not particularly useful to crash per se -- but rather it would be 
useful to a user trying to analyze the dumpfile.  If a prominent warning
message is displayed early on during invocation, then it won't be a big
surprise if/when things are subsequently broken/crippled.

Dave
 
> Thanks
> Vivek


From vgoyal at redhat.com  Thu May 26 09:39:23 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Thu, 26 May 2011 09:39:23 -0400
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter
	out kernel data from vmcore
In-Reply-To: <OF05104FA0.A922963A-ONC125789C.003BDAA9-C125789C.003DD130@de.ibm.com>
References: <20110524203542.GH3860@redhat.com>
	<1621913139.202658.1306270052667.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com>
	<OF341326C7.0D81F89A-ONC125789B.002DB6ED-C125789B.002FC7A0@de.ibm.com>
	<20110525195321.GD6724@redhat.com>
	<OF05104FA0.A922963A-ONC125789C.003BDAA9-C125789C.003DD130@de.ibm.com>
Message-ID: <20110526133923.GA29496@redhat.com>

On Thu, May 26, 2011 at 01:15:14PM +0200, Reinhard Buendgen wrote:
> Vivek,
> 
> I/O is not restricted to disk I/O (it may be network I/O, data sent to 
> crtypto cards etc.) and it is not always direct, Device drivers may have 
> buffers to which such data is copied. 
> 
> So it is more than just keys, and it may change over time.
> I do not think hardwiring a filter in makedumpfile is a good idea because 
> you would need a new makedumpfile release for every Distribution 
> (release).

Ok, so we are worried about data being in slub/slab caches or driver's
kmalloced buffers etc.

When do I need access to debuginfo files. I am assuming that makedumpfile
reads them in first kernel sometime and stores relevant info in initramfs.
Otherwise, it is not possible to get to it in second kernel after crash.

> 
> Allowing to configure makedumpfile allows each distribution and each 
> platform to provide appropriate filters.
> 

I was thinking that there are very few symbols and these will not change
frequently. makdumpfile is already dependent on some kenrel data strcutres
and symbols for doing filtering. 

But looks like you are planning to filter out lots of symbols. So in that
case agreed that hardcoding is not a good idea.

Thanks
Vivek

> Mit freundlichen Gr??en/Best Regards/Cordialement 
> 
> Reinhard
> 
> Dr. Reinhard B?ndgen 
> RAS & Crypto Architect for Linux on System z 
> Virtualization and Systems Management 
>  
> Mail:buendgen at de.ibm.com
> Phone: ++49-(0)7031-16-1130
> Fax: ++49-(0)7031-16-3456 
>  
> IBM Deutschland Research & Development GmbH
> Vorsitzender des Aufsichtsrats: Martin Jetter
> Gesch?ftsf?hrung: Dirk Wittkopp 
> Sitz der Gesellschaft: B?blingen
> Registergericht: Amtsgericht Stuttgart, HRB 243294
> 
> 
> 
> 
> 
> From:   Vivek Goyal <vgoyal at redhat.com>
> To:     Reinhard Buendgen/Germany/IBM at IBMDE
> Cc:     Dave Anderson <anderson at redhat.com>, Ananth N Mavinakayanahalli 
> <ananth at in.ibm.com>, kexec at lists.infradead.org, Mahesh J Salgaonkar 
> <mahesh at linux.vnet.ibm.com>, "Ken'ichi Ohmichi" 
> <oomichi at mxs.nes.nec.co.jp>, V Srivatsa <vsrivatsa at in.ibm.com>
> Date:   25.05.2011 21:53
> Subject:        Re: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement 
> to filter out kernel data from vmcore
> 
> 
> 
> On Wed, May 25, 2011 at 10:41:55AM +0200, Reinhard Buendgen wrote:
> > Hi,
> > 
> > to answer Vivek questions first: Eventually we want to be able to erase 
> > all data that a customer may consider sensitive to her privacy. In 
> > addition to encryption key that may be the contents (i.e. payload 
> within) 
> > of all kinds of I/O buffers. Consider you are running a kvm based 
> > hypervisor and want its dump to be analyized while promising your 
> > customers whose guests you run on that hypervisor that none of their 
> data 
> > will be externalized. Or consider your system reads a spreadsheat with 
> > bank account or health information. You might not want to send fractions 
> 
> > of that information sitting in some buffers to a service organization.
> 
> So for direct IO, buffer is still in user space and should be filtered
> out when we filter out user space pages using mkdumpfile. For kvm, I am
> assuming that all the pages belong to qemu process and once we are
> filtering out user space pages, any data belonging to guest will go away.
> 
> So atleast for above examples it does not sound as if we need symbol
> erase infrastructure.
> 
> > 
> > to answer Daves concern: there is no intention that crash should ever 
> look 
> > into the erased structures. In theroy it should not be needed because 
> the 
> > contents of structures to be deleted should be irrelevant to kernel 
> > debugging.
> 
> So what are those kernel structures which we are planning to delete and
> are irrelevant to kernel debugging by crash?
> 
> I think we are missing something here. If there are only few known
> structures we want to get rid of, lets hardcode it in makedumpfile
> instead of giving user a generic infrastructure. That way we know
> that we are not leaking information at the same time making sure
> that analysis tools are working.
> 
> Thanks
> Vivek
> 


From vgoyal at redhat.com  Thu May 26 15:10:08 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Thu, 26 May 2011 15:10:08 -0400
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526185838.GG4065@one.firstfloor.org>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
	<20110526174447.GC29496@redhat.com>
	<20110526180931.GF4065@one.firstfloor.org>
	<20110526182618.GD29496@redhat.com>
	<20110526185838.GG4065@one.firstfloor.org>
Message-ID: <20110526191008.GF29496@redhat.com>

On Thu, May 26, 2011 at 08:58:38PM +0200, Andi Kleen wrote:
> > If we are just extracting and saving MCE registers from vmcore, then
> > reboot time does not increase. It increases only if user decides to
> > extract and save extra data from vmcore.
> 
> Hmm I was thinking of user space usually saving the dump first
> before analyzing it. But yes it could probably do some minimal
> analysis first.

In RHEL, now we filter out the dump by default until and unless user
decides to no filter the dump with the help of config options. 

So I think in this case we can just introduce an extra filtering
option in "makedumpfile" and ask it to save only MCE registers
if it notices that there is NT_MCE type of ELF note in vmcore.

I can very well imaging that extracting dmesg along with MCE registers
can be useful. If nothing else, it can give us useful information about
the system configuration.

Zeroing out all the other ELF headers in vmcore takes away the
capability to extract log buffers in case of MCE and I think
it probably is not the best idea.

Thanks
Vivek


From andi at firstfloor.org  Thu May 26 14:58:38 2011
From: andi at firstfloor.org (Andi Kleen)
Date: Thu, 26 May 2011 20:58:38 +0200
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526182618.GD29496@redhat.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
	<20110526174447.GC29496@redhat.com>
	<20110526180931.GF4065@one.firstfloor.org>
	<20110526182618.GD29496@redhat.com>
Message-ID: <20110526185838.GG4065@one.firstfloor.org>

> If we are just extracting and saving MCE registers from vmcore, then
> reboot time does not increase. It increases only if user decides to
> extract and save extra data from vmcore.

Hmm I was thinking of user space usually saving the dump first
before analyzing it. But yes it could probably do some minimal
analysis first.

I guess most user spaces just won't change and will just continue
to DTWT though.

-Andi

-- 
ak at linux.intel.com -- Speaking for myself only.


From vgoyal at redhat.com  Thu May 26 14:43:50 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Thu, 26 May 2011 14:43:50 -0400
Subject: [RFC Patch 4/6] PANIC_MCE: Introduce a new panic flag for fatal
	MCE, capture related information
In-Reply-To: <20110526171521.GD17988@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171521.GD17988@in.ibm.com>
Message-ID: <20110526184350.GE29496@redhat.com>

On Thu, May 26, 2011 at 10:45:21PM +0530, K.Prasad wrote:

[..]
> Index: linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
> ===================================================================
> --- linux-2.6.slim_kdump.orig/arch/x86/kernel/cpu/mcheck/mce.c
> +++ linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
> @@ -258,8 +258,7 @@ static void wait_for_panic(void)
>  	local_irq_enable();
>  	while (timeout-- > 0)
>  		udelay(1);
> -	xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, 0,
> -		"Panicing machine check CPU died");
> +	xpanic(PANIC_MCE, 0, NULL, 0, "Panicing machine check CPU died");
>  }
>  
>  static void mce_panic(char *msg, struct mce *final, char *exp)
> @@ -315,8 +314,8 @@ static void mce_panic(char *msg, struct
>  	if (exp)
>  		pr_emerg(HW_ERR "Machine check: %s\n", exp);
>  	if (!fake_panic) {
> -		xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, mce_panic_timeout,
> -			msg);
> +		xpanic(PANIC_MCE, mce_panic_timeout, final,
> +			sizeof(struct mce), msg);
>  	} else
>  		pr_emerg(HW_ERR "Fake kernel panic: %s\n", msg);
>  }

In previous patches you introduce PANIC_NO_KEXEC and PANIC_NO_BACKTRACE. 
Now in this patch you got rid of those. Are there any other users left
of PANIC_NO_BACKTRACE and PANIC_NO_EXEC? If not, then why to introduce
these to begin with.

Thanks
Vivek


From vgoyal at redhat.com  Thu May 26 14:26:18 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Thu, 26 May 2011 14:26:18 -0400
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526180931.GF4065@one.firstfloor.org>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
	<20110526174447.GC29496@redhat.com>
	<20110526180931.GF4065@one.firstfloor.org>
Message-ID: <20110526182618.GD29496@redhat.com>

On Thu, May 26, 2011 at 08:09:31PM +0200, Andi Kleen wrote:
> On Thu, May 26, 2011 at 01:44:47PM -0400, Vivek Goyal wrote:
> > On Thu, May 26, 2011 at 10:53:05PM +0530, K.Prasad wrote:
> > > 
> > > slimdump: Capture slimdump for fatal MCE generated crashes
> > > 
> > > System crashes resulting from fatal hardware errors (such as MCE) don't need
> > > all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
> > > retains only essential information while discarding the old memory.
> > > 
> > 
> > Why to enforce zeroing out of rest of the vmcore data in kernel. Why not
> > leave it to user space. 
> 
> I think it's a good default to not do a full dump on MCE.
> It's very unlikely to be useful for anything, and will just waste
> reboot time (aka nines).

If we are just extracting and saving MCE registers from vmcore, then
reboot time does not increase. It increases only if user decides to
extract and save extra data from vmcore.

> 
> That said including the dmesg too may be a good idea.

dmesg is already part of vmcore and user space tools can easily find
it. 

I can easily imagine a default policy of a distro in user space where
in case of MCE crash, we just extract dmesg and MCE registers (from vmcore
notes section) reboot. This will be fast and will reduce the amount of code
in kernel.

IMHO, we should not introduce any additional notion of slimdump as such in
kernel. A better thing would be to just read MCE registers and export to
user space through ELF notes and then let user space automate the rest of
it.

Thanks
Vivek


From andi at firstfloor.org  Thu May 26 14:09:31 2011
From: andi at firstfloor.org (Andi Kleen)
Date: Thu, 26 May 2011 20:09:31 +0200
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526174447.GC29496@redhat.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
	<20110526174447.GC29496@redhat.com>
Message-ID: <20110526180931.GF4065@one.firstfloor.org>

On Thu, May 26, 2011 at 01:44:47PM -0400, Vivek Goyal wrote:
> On Thu, May 26, 2011 at 10:53:05PM +0530, K.Prasad wrote:
> > 
> > slimdump: Capture slimdump for fatal MCE generated crashes
> > 
> > System crashes resulting from fatal hardware errors (such as MCE) don't need
> > all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
> > retains only essential information while discarding the old memory.
> > 
> 
> Why to enforce zeroing out of rest of the vmcore data in kernel. Why not
> leave it to user space. 

I think it's a good default to not do a full dump on MCE.
It's very unlikely to be useful for anything, and will just waste
reboot time (aka nines).

That said including the dmesg too may be a good idea.

On the other hand I think the slim dump should be probably generalized
and used for more situations. I could well imagine enabling it by
default on various systems.
 
-Andi



From vgoyal at redhat.com  Thu May 26 13:44:47 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Thu, 26 May 2011 13:44:47 -0400
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526172305.GA18295@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
Message-ID: <20110526174447.GC29496@redhat.com>

On Thu, May 26, 2011 at 10:53:05PM +0530, K.Prasad wrote:
> 
> slimdump: Capture slimdump for fatal MCE generated crashes
> 
> System crashes resulting from fatal hardware errors (such as MCE) don't need
> all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
> retains only essential information while discarding the old memory.
> 

Why to enforce zeroing out of rest of the vmcore data in kernel. Why not
leave it to user space. 

As Andi said, you anyway will require disabling MCE temporarily in second
kernel. So that should allay the concern that we might run into second
MCE while trying to read the vmcore.

A user might want to extract just the dmesg buffers also after MCE from
vmcore.

What I am saying is that this sounds more like a policy. Don't enforce
it in kernel. Probably give user MCE registers as part of ELF note of
vmcore. Now leave it up to user what data he wants to extract from vmcore.
Just the MCE registers or something more then that.

Thanks
Vivek

> Signed-off-by: K.Prasad <prasad at linux.vnet.ibm.com>
> ---
>  fs/proc/vmcore.c |  130 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 128 insertions(+), 2 deletions(-)
> 
> Index: linux-2.6.slim_kdump/fs/proc/vmcore.c
> ===================================================================
> --- linux-2.6.slim_kdump.orig/fs/proc/vmcore.c
> +++ linux-2.6.slim_kdump/fs/proc/vmcore.c
> @@ -483,9 +483,60 @@ static void __init set_vmcore_list_offse
>  	}
>  }
>  
> +/*
> + * Check if the crash was due to a fatal Memory Check Exception
> + */
> +static int is_mce_crash64(void)
> +{
> +	int i, j, len = 0, rc;
> +	Elf64_Ehdr *ehdr_ptr;
> +	Elf64_Phdr *phdr_ptr;
> +	Elf64_Nhdr *nhdr_ptr;
> +
> +	ehdr_ptr = (Elf64_Ehdr *)elfcorebuf;
> +	phdr_ptr = (Elf64_Phdr *)(elfcorebuf + sizeof(Elf64_Ehdr));
> +
> +	for (i = 0; i < ehdr_ptr->e_phnum; i++, phdr_ptr++) {
> +		void *notes_section;
> +		u64 offset, max_sz;
> +		if (phdr_ptr->p_type != PT_NOTE)
> +			continue;
> +		max_sz = phdr_ptr->p_memsz;
> +		offset = phdr_ptr->p_offset;
> +		notes_section = kmalloc(max_sz, GFP_KERNEL);
> +		if (!notes_section)
> +			return -ENOMEM;
> +		rc = read_from_oldmem(notes_section, max_sz, &offset, 0);
> +		if (rc < 0) {
> +			kfree(notes_section);
> +			return rc;
> +		}
> +
> +		for (j = 0; j < phdr_ptr->p_filesz; j += len) {
> +			nhdr_ptr = notes_section + j;
> +			if (nhdr_ptr->n_type == NT_MCE)
> +			{
> +				kfree(notes_section);
> +				return 1;
> +			}
> +			/*
> +			 * The elf-64 standard specifies 8-byte alignment while
> +			 * append_elf_note function does only 4-byte roundup.
> +			 * Hence this code also does a 4-byte roundup.
> +			 */
> +			len = sizeof(Elf64_Nhdr);
> +			len = roundup(len + nhdr_ptr->n_namesz, 4);
> +			len = roundup(len + nhdr_ptr->n_descsz, 4);
> +		}
> +		kfree(notes_section);
> +	}
> +	return 0;
> +}
> +
>  static int __init parse_crash_elf64_headers(void)
>  {
> -	int rc=0;
> +	int i, rc = 0;
> +	Elf64_Phdr *phdr_ptr;
>  	Elf64_Ehdr ehdr;
>  	u64 addr;
>  
> @@ -523,6 +574,18 @@ static int __init parse_crash_elf64_head
>  		return rc;
>  	}
>  
> +	phdr_ptr = (Elf64_Phdr *)(elfcorebuf + sizeof(Elf64_Ehdr));
> +	if (is_mce_crash64() > 0) {
> +		/*
> +		 * If crash is due to Machine Check exception, don't populate
> +		 * sections other than elf-notes. Mark their sizes as zero.
> +		 */
> +		for (i = 0; i < ehdr.e_phnum; i++, phdr_ptr++) {
> +			if (phdr_ptr->p_type != PT_NOTE)
> +				phdr_ptr->p_memsz = phdr_ptr->p_filesz = 0;
> +		}
> +	}
> +
>  	/* Merge all PT_NOTE headers into one. */
>  	rc = merge_note_headers_elf64(elfcorebuf, &elfcorebuf_sz, &vmcore_list);
>  	if (rc) {
> @@ -539,9 +602,60 @@ static int __init parse_crash_elf64_head
>  	return 0;
>  }
>  
> +/*
> + * Check if the crash was due to a fatal Memory Check Exception
> + */
> +static int is_mce_crash32(void)
> +{
> +	int i, j, len = 0, rc;
> +	Elf32_Ehdr *ehdr_ptr;
> +	Elf32_Phdr *phdr_ptr;
> +	Elf32_Nhdr *nhdr_ptr;
> +
> +	ehdr_ptr = (Elf32_Ehdr *)elfcorebuf;
> +	phdr_ptr = (Elf32_Phdr *)(elfcorebuf + sizeof(Elf32_Ehdr));
> +
> +	for (i = 0; i < ehdr_ptr->e_phnum; i++, phdr_ptr++) {
> +		void *notes_section;
> +		u64 offset, max_sz;
> +		if (phdr_ptr->p_type != PT_NOTE)
> +			continue;
> +		max_sz = phdr_ptr->p_memsz;
> +		offset = phdr_ptr->p_offset;
> +		notes_section = kmalloc(max_sz, GFP_KERNEL);
> +		if (!notes_section)
> +			return -ENOMEM;
> +		rc = read_from_oldmem(notes_section, max_sz, &offset, 0);
> +		if (rc < 0) {
> +			kfree(notes_section);
> +			return rc;
> +		}
> +
> +		for (j = 0; j < phdr_ptr->p_filesz; j += len) {
> +			nhdr_ptr = notes_section + j;
> +			if (nhdr_ptr->n_type == NT_MCE)
> +			{
> +				kfree(notes_section);
> +				return 1;
> +			}
> +			/*
> +			 * The elf-64 standard specifies 8-byte alignment while
> +			 * append_elf_note function does only 4-byte roundup.
> +			 * Hence this code also does a 4-byte roundup.
> +			 */
> +			len = sizeof(Elf64_Nhdr);
> +			len = roundup(len + nhdr_ptr->n_namesz, 4);
> +			len = roundup(len + nhdr_ptr->n_descsz, 4);
> +		}
> +		kfree(notes_section);
> +	}
> +	return 0;
> +}
> +
>  static int __init parse_crash_elf32_headers(void)
>  {
> -	int rc=0;
> +	int i, rc = 0;
> +	Elf32_Phdr *phdr_ptr;
>  	Elf32_Ehdr ehdr;
>  	u64 addr;
>  
> @@ -579,6 +693,18 @@ static int __init parse_crash_elf32_head
>  		return rc;
>  	}
>  
> +	phdr_ptr = (Elf32_Phdr *)(elfcorebuf + sizeof(Elf32_Ehdr));
> +	if (is_mce_crash32() > 0) {
> +		/*
> +		 * If crash is due to Machine Check exception, don't populate
> +		 * sections other than elf-notes. Mark their sizes as zero.
> +		 */
> +		for (i = 0; i < ehdr.e_phnum; i++, phdr_ptr++) {
> +			if (phdr_ptr->p_type != PT_NOTE)
> +				phdr_ptr->p_memsz = phdr_ptr->p_filesz = 0;
> +		}
> +	}
> +
>  	/* Merge all PT_NOTE headers into one. */
>  	rc = merge_note_headers_elf32(elfcorebuf, &elfcorebuf_sz, &vmcore_list);
>  	if (rc) {


From richard.weinberger at gmail.com  Thu May 26 13:38:19 2011
From: richard.weinberger at gmail.com (richard -rw- weinberger)
Date: Thu, 26 May 2011 19:38:19 +0200
Subject: [Patch 1/6] XPANIC: Add extended panic interface
In-Reply-To: <20110526171209.GA17988@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171209.GA17988@in.ibm.com>
Message-ID: <BANLkTi=ZrPkRwFSmdkAB60-MKS3fwZOjpg@mail.gmail.com>

On Thu, May 26, 2011 at 7:12 PM, K.Prasad <prasad at linux.vnet.ibm.com> wrote:
>
> commit e668fa1aea7844ac4c7ea09030a2f3e647a4adb1
> Author: Andi Kleen <ak at linux.intel.com>
> Date: ? Fri Nov 19 18:36:44 2010 +0100
>
> ? ?XPANIC: Add extended panic interface
>
> ? ?One panic size doesn't fit all.
>
> ? ?Machine check has some special requirements on panic, like:
> ? ?- It wants to do a reboot timeout by default so that the machine
> ? ?check event can be logged to disk after warm reboot.
> ? ?- For memory errors it usually doesn't want to do crash dumps because
> ? ?that can lead to crash loops (dumping corrupted memory would
> ? ?lead to another machine check)
> ? ?- It doesn't want to do backtraces because machine checks
> ? ?are not a kernel bug.
>
> ? ?In a earlier patch this was done with various adhoc hacks,
> ? ?but it's cleaner to extend panic to a 'xpanic' that directly
> ? ?gets a flag and timeout argument.
>
> ? ?The only user right now will be x86 machine checks, but I consider
> ? ?it likely that other users will switch to this too.
>
> ? ?For example one obvious candidate would be the "no root
> ? ?found" panic which doesn't really deserve a backtrace.
>
> ? ?I exported a vpanic() interface too as a global. That's not
> ? ?needed by the current user, but the interface has to exist
> ? ?internally anyways and I could see how other code would
> ? ?find a v* variant of panic useful.
>
> ? ?Originally based on a suggestion by H. Peter Anvin.
> ? ?Signed-off-by: Andi Kleen <ak at linux.intel.com>
>
> ---
> ?include/linux/kernel.h | ? 11 +++++++++++
> ?kernel/panic.c ? ? ? ? | ? 41 +++++++++++++++++++++++++++++++----------
> ?2 files changed, 42 insertions(+), 10 deletions(-)
>
> Index: linux-2.6.slim_kdump/include/linux/kernel.h
> ===================================================================
> --- linux-2.6.slim_kdump.orig/include/linux/kernel.h
> +++ linux-2.6.slim_kdump/include/linux/kernel.h
> @@ -175,10 +175,21 @@ static inline void might_fault(void)
> ?}
> ?#endif
>
> +enum panic_flags {
> + ? ? ? PANIC_NO_KEXEC ? ? = (1 << 0),
> + ? ? ? PANIC_NO_BACKTRACE = (1 << 1),
> +};
> +
> ?extern struct atomic_notifier_head panic_notifier_list;
> ?extern long (*panic_blink)(int state);
> ?NORET_TYPE void panic(const char * fmt, ...)
> ? ? ? ?__attribute__ ((NORET_AND format (printf, 1, 2))) __cold;
> +NORET_TYPE void xpanic(enum panic_flags flags, int timeout,
> + ? ? ? ? ? ? ? ? ? ? ?const char *fmt, ...)
> + ? ? ? __attribute__ ((NORET_AND format (printf, 3, 4))) __cold;
> +NORET_TYPE void vpanic(enum panic_flags flags, int timeout,
> + ? ? ? ? ? ? ? ? ? ? ?const char *fmt,
> + ? ? ? ? ? ? ? ? ? ? ?va_list ap) __noreturn __cold;
> ?extern void oops_enter(void);
> ?extern void oops_exit(void);
> ?void print_oops_end_marker(void);
> Index: linux-2.6.slim_kdump/kernel/panic.c
> ===================================================================
> --- linux-2.6.slim_kdump.orig/kernel/panic.c
> +++ linux-2.6.slim_kdump/kernel/panic.c
> @@ -57,14 +57,37 @@ EXPORT_SYMBOL(panic_blink);
> ?*
> ?* ? ? This function never returns.
> ?*/
> -NORET_TYPE void panic(const char * fmt, ...)
> +NORET_TYPE void panic(const char *fmt, ...)
> +{
> + ? ? ? va_list ap;
> + ? ? ? va_start(ap, fmt);
> + ? ? ? vpanic(0, 0, fmt, ap);
> +}
> +EXPORT_SYMBOL(panic);
> +
> +NORET_TYPE void xpanic(enum panic_flags flags, int timeout,
> + ? ? ? ? ? ? ? ? ? ? ? const char *fmt, ...)
> +{
> + ? ? ? va_list ap;
> + ? ? ? va_start(ap, fmt);
> + ? ? ? xpanic(flags, timeout, fmt, ap);

Why are you calling xpanic() here again?
I guess you meant vpanic().

> +}
> +EXPORT_SYMBOL(xpanic);
> +
> +NORET_TYPE void vpanic(enum panic_flags flags, int timeout,
> + ? ? ? ? ? ? ? ? ? ? ?const char * fmt, va_list args)
> ?{
> ? ? ? ?static char buf[1024];
> - ? ? ? va_list args;
> ? ? ? ?long i, i_next = 0;
> ? ? ? ?int state = 0;
>
> ? ? ? ?/*
> + ? ? ? ?* Let user always override panic_timeout.
> + ? ? ? ?*/
> + ? ? ? if (panic_timeout > 0)
> + ? ? ? ? ? ? ? timeout = panic_timeout;
> +
> + ? ? ? /*
> ? ? ? ? * It's possible to come here directly from a panic-assertion and
> ? ? ? ? * not have preempt disabled. Some functions called from here want
> ? ? ? ? * preempt to be disabled. No point enabling it later though...
> @@ -73,12 +96,11 @@ NORET_TYPE void panic(const char * fmt,
>
> ? ? ? ?console_verbose();
> ? ? ? ?bust_spinlocks(1);
> - ? ? ? va_start(args, fmt);
> ? ? ? ?vsnprintf(buf, sizeof(buf), fmt, args);
> - ? ? ? va_end(args);
> ? ? ? ?printk(KERN_EMERG "Kernel panic - not syncing: %s\n",buf);
> ?#ifdef CONFIG_DEBUG_BUGVERBOSE
> - ? ? ? dump_stack();
> + ? ? ? if (!(flags & PANIC_NO_BACKTRACE))
> + ? ? ? ? ? ? ? dump_stack();
> ?#endif
>
> ? ? ? ?/*
> @@ -86,7 +108,8 @@ NORET_TYPE void panic(const char * fmt,
> ? ? ? ? * everything else.
> ? ? ? ? * Do we want to call this before we try to display a message?
> ? ? ? ? */
> - ? ? ? crash_kexec(NULL);
> + ? ? ? if (!(flags & PANIC_NO_KEXEC))
> + ? ? ? ? ? ? ? crash_kexec(NULL);
>
> ? ? ? ?kmsg_dump(KMSG_DUMP_PANIC);
>
> @@ -104,7 +127,7 @@ NORET_TYPE void panic(const char * fmt,
> ? ? ? ?if (!panic_blink)
> ? ? ? ? ? ? ? ?panic_blink = no_blink;
>
> - ? ? ? if (panic_timeout > 0) {
> + ? ? ? if (timeout > 0) {
> ? ? ? ? ? ? ? ?/*
> ? ? ? ? ? ? ? ? * Delay timeout seconds before rebooting the machine.
> ? ? ? ? ? ? ? ? * We can't use the "normal" timers since we just panicked.
> @@ -152,9 +175,7 @@ NORET_TYPE void panic(const char * fmt,
> ? ? ? ? ? ? ? ?mdelay(PANIC_TIMER_STEP);
> ? ? ? ?}
> ?}
> -
> -EXPORT_SYMBOL(panic);
> -
> +EXPORT_SYMBOL(vpanic);
>
> ?struct tnt {
> ? ? ? ?u8 ? ? ?bit;
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at ?http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at ?http://www.tux.org/lkml/
>

-- 
Thanks,
//richard


From andi at firstfloor.org  Thu May 26 13:32:57 2011
From: andi at firstfloor.org (Andi Kleen)
Date: Thu, 26 May 2011 19:32:57 +0200
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526172305.GA18295@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
Message-ID: <20110526173257.GC4065@one.firstfloor.org>

On Thu, May 26, 2011 at 10:53:05PM +0530, K.Prasad wrote:
> 
> slimdump: Capture slimdump for fatal MCE generated crashes
> 
> System crashes resulting from fatal hardware errors (such as MCE) don't need
> all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
> retains only essential information while discarding the old memory.

While this is a good idea, note there may be still poisoned lines
in memory that haven't resulted in a machine check yet, but could
still be fatal when read after a full crash dump for some other
reason.

So you still need 

http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=commit;h=fe61906edce9e70d02481a77a617ba1397573dce
and
http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=commit;h=cb58f049ae6709ddbab71be199390dc6852018cd

in addition.

-Andi


From prasad at linux.vnet.ibm.com  Thu May 26 13:31:56 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 26 May 2011 23:01:56 +0530
Subject: [RFC Patch 0/6] slimdump: Enable slimdump if crashing kernel
	memory is not required
In-Reply-To: <20110526170722.GB23266@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
Message-ID: <20110526173156.GC18295@in.ibm.com>

On Thu, May 26, 2011 at 10:37:22PM +0530, K.Prasad wrote:
> Hi All,

A small correction related to patch description...

> 
> Summary of the patches
> ----
> Patch 1, 2 - Patches from Andi Kleen's tree
> git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6.git which
> introduce an enhanced panic function - xpanic().
> 

Patch 3 - A small bugfix for patch 1,2

Patch 4 - Introduces a new PANIC_MCE flag to signal the need for slimdump
for MCE initiated panic calls. Appends a new elf-note inside coredump
containing MCE related registers.

Thanks,
K.Prasad



From prasad at linux.vnet.ibm.com  Thu May 26 13:26:18 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 26 May 2011 22:56:18 +0530
Subject: [RFC Patch 6/6] Crash: Recognise slim coredumps and process new
	elf-note sections
In-Reply-To: <20110526170722.GB23266@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
Message-ID: <20110526172618.GB18295@in.ibm.com>


Crash: Recognise slim coredumps and process new elf-note sections

The Linux kernel will begin to support SlimDump for certain types of crashes
and the 'crash' tool needs to recognise them. For these types of coredumps, it
need not lookout for usual elf-structures and start gdb. Also process new
elf-note sections that contain additional information about the crash.

Signed-off-by: K.Prasad <prasad at linux.vnet.ibm.com>
---
 diskdump.c |   84 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 netdump.c  |    8 +++++
 x86.h      |   91 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 183 insertions(+)

Index: crash-5.1.5.slim_kdump/x86.h
===================================================================
--- /dev/null
+++ crash-5.1.5.slim_kdump/x86.h
@@ -0,0 +1,91 @@
+/*
+ * x86.h - x86 Architecture specific definitions
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Copyright (C) IBM Corporation, 2011
+ *
+ * Author: K.Prasad <prasad at linux.vnet.ibm.com>
+ */
+
+typedef unsigned long long u64;
+typedef unsigned int u32;
+typedef unsigned short u16;
+typedef unsigned char u8;
+
+#define __u64 u64
+#define __u32 u32
+#define __u16 u16
+#define __u8  u8
+
+/* Mask for finding the address mode in IA32_MCi_MISC[8:6] register */
+#define MCI_MISC_ADDR_MODE	0X1C0
+/* Number of bits to shift the IA32_MCi_MISC to read the address-mode bits */
+#define MISC_ADDR_MODE_POS	6
+
+/* Address Modes in IA32_MCi_MISC[8:6] */
+#define MCM_ADDR_SEGOFF  0      /* segment offset */
+#define MCM_ADDR_LINEAR  1      /* linear address */
+#define MCM_ADDR_PHYS    2      /* physical address */
+#define MCM_ADDR_MEM     3      /* memory address */
+#define MCM_ADDR_GENERIC 7      /* generic */
+
+#define MCI_STATUS_MISCV (1ULL<<59)  /* misc error reg. valid */
+#define MCI_STATUS_ADDRV (1ULL<<58)  /* addr reg. valid */
+
+#define PAGE_SHIFT 12
+
+static const char *mce_addr_mode[] =
+{
+	"Segment offset",	/* MCM_ADDR_SEGOFF */
+	"Linear address",	/* MCM_ADDR_LINEAR */
+	"Physical address",	/* MCM_ADDR_PHYS */
+	"Memory address",	/* MCM_ADDR_MEM */
+	"",			/* reserved */
+	"",			/* reserved */
+	"",			/* reserved */
+	"Generic"		/* MCM_ADDR_GENERIC */
+};
+
+/*
+ * kernel structure: Keep this in sync with the definition in
+ * arch/x86/include/asm/mce.h of linux source code.
+ *
+ * Fields are zero when not available
+ *
+ */
+struct mce {
+	__u64 status;
+	__u64 misc;
+	__u64 addr;
+	__u64 mcgstatus;
+	__u64 ip;
+	__u64 tsc;	/* cpu time stamp counter */
+	__u64 time;	/* wall time_t when error was detected */
+	__u8  cpuvendor;	/* cpu vendor as encoded in system.h */
+	__u8  pad1;
+	__u16 pad2;
+	__u32 cpuid;	/* CPUID 1 EAX */
+	__u8  cs;		/* code segment */
+	__u8  bank;	/* machine check bank */
+	__u8  cpu;	/* cpu number; obsolete; use extcpu now */
+	__u8  finished;   /* entry is valid */
+	__u32 extcpu;	/* linux cpu number that detected the error */
+	__u32 socketid;	/* CPU socket ID */
+	__u32 apicid;	/* CPU initial apic ID */
+	__u64 mcgcap;	/* MCGCAP MSR: machine check capabilities of CPU */
+	__u64 aux0;
+	__u64 aux1;
+};
Index: crash-5.1.5.slim_kdump/netdump.c
===================================================================
--- crash-5.1.5.slim_kdump.orig/netdump.c
+++ crash-5.1.5.slim_kdump/netdump.c
@@ -331,6 +331,10 @@ is_netdump(char *file, ulong source_quer
 		}
         	nd->notes32 = (Elf32_Phdr *)
 		    &nd->elf_header[sizeof(Elf32_Ehdr)];
+		if (machdep->process_elf_notes)
+			machdep->process_elf_notes((char *)nd->elf32 +
+							nd->notes32->p_offset,
+							nd->notes32->p_filesz);
         	nd->load32 = (Elf32_Phdr *)
 		    &nd->elf_header[sizeof(Elf32_Ehdr)+sizeof(Elf32_Phdr)];
 		if (DUMPFILE_FORMAT(nd->flags) == NETDUMP_ELF32)
@@ -360,6 +364,10 @@ is_netdump(char *file, ulong source_quer
                 }
                 nd->notes64 = (Elf64_Phdr *)
                     &nd->elf_header[sizeof(Elf64_Ehdr)];
+		if (machdep->process_elf_notes)
+			machdep->process_elf_notes((char *)nd->elf64 +
+							nd->notes64->p_offset,
+							nd->notes64->p_filesz);
                 nd->load64 = (Elf64_Phdr *)
                     &nd->elf_header[sizeof(Elf64_Ehdr)+sizeof(Elf64_Phdr)];
 		if (DUMPFILE_FORMAT(nd->flags) == NETDUMP_ELF64)
Index: crash-5.1.5.slim_kdump/diskdump.c
===================================================================
--- crash-5.1.5.slim_kdump.orig/diskdump.c
+++ crash-5.1.5.slim_kdump/diskdump.c
@@ -231,6 +231,27 @@ open_dump_file(char *file)
 	dd->dfd = fd;
 	return TRUE;
 }
+#if defined(X86_64) || defined(X86)
+#include "x86.h"
+
+/*
+ * Check if the address reported by the CPU is in a format we can parse.
+ * It would be possible to add code for most other cases, but all would
+ * be somewhat complicated (e.g. segment offset would require an instruction
+ * parser). So only support physical addresses up to page granuality for now.
+ *
+ * Function derived from arch/x86/kernel/cpu/mcheck/mce.c in Linux source
+ *
+ */
+static int mce_usable_address(struct mce *m)
+{
+	if (!(m->status & MCI_STATUS_MISCV) || !(m->status & MCI_STATUS_ADDRV))
+		return 0;
+	if ((m->misc & 0x3f) > PAGE_SHIFT)
+		return 0;
+	return 1;
+}
+#endif /* defined(X86_64) || defined(X86) */
 
 void 
 x86_process_elf_notes(void *note_ptr, unsigned long size_note)
@@ -239,10 +260,43 @@ x86_process_elf_notes(void *note_ptr, un
 	Elf64_Nhdr *note64 = NULL;
 	size_t tot, len = 0;
 	int num = 0;
+#if defined(X86_64) || defined(X86)
+	struct mce *mce;
+	ushort addr_mode;
+#endif /* defined(X86_64) || defined(X86) */
 
 	for (tot = 0; tot < size_note; tot += len) {
 		if (machine_type("X86_64")) {
 			note64 = note_ptr + tot;
+#ifdef X86_64
+			/*
+			 * If vmcore is generated due to fatal Machine Check
+			 * Exception, we only have a 'slim' crashdump. Don't
+			 * analyse further, inform the user about it and exit.
+			 */
+			if (note64->n_type == NT_MCE) {
+				fprintf(fp, "\"System crashed due to a hardware"
+					" memory error. No coredump"
+					" available.\"\n");
+
+				/* Do we have a copy of 'struct mce'? */
+				if (note64->n_descsz == 0)
+					goto exit;
+
+				mce = (struct mce *)((char *)note64 +
+					sizeof(Elf64_Nhdr) + note64->n_namesz);
+				if (!mce_usable_address(mce))
+					goto exit;
+
+				addr_mode = (mce->misc >> MISC_ADDR_MODE_POS) &
+						MCI_MISC_ADDR_MODE;
+				fprintf(fp, "Memory error occured at %llx "
+					"(address type: %s\n)", mce->addr,
+					mce_addr_mode[addr_mode]);
+exit:
+				clean_exit(0);
+			}
+#endif /* X86_64 */
 
 			if (note64->n_type == NT_PRSTATUS) {
 				dd->nt_prstatus_percpu[num] = note64;
@@ -255,6 +309,36 @@ x86_process_elf_notes(void *note_ptr, un
 		} else if (machine_type("X86")) {
 			note32 = note_ptr + tot;
 
+#ifdef X86
+			/*
+			 * If vmcore is generated due to fatal Machine Check
+			 * Exception, we only have a 'slim' crashdump. Don't
+			 * analyse further, inform the user about it and exit.
+			 */
+			if (note32->n_type == NT_MCE) {
+				fprintf(fp, "\"System crashed due to a hardware"
+					" memory error. No coredump"
+					" available.\"\n");
+
+				/* Do we have a copy of 'struct mce'? */
+				if (note32->n_descsz == 0)
+					goto exit;
+
+				mce = (struct mce *)((char *)note32 +
+					sizeof(Elf32_Nhdr) + note32->n_namesz);
+				if (!mce_usable_address(mce))
+					goto exit;
+
+				addr_mode = (mce->misc >> MISC_ADDR_MODE_POS) &
+						MCI_MISC_ADDR_MODE;
+				fprintf(fp, "Memory error occured at %llx "
+					"(address type: %s\n)", mce->addr,
+					mce_addr_mode[addr_mode]);
+exit:
+				clean_exit(0);
+			}
+#endif /* X86 */
+
 			if (note32->n_type == NT_PRSTATUS) {
 				dd->nt_prstatus_percpu[num] = note32;
 				num++;


From prasad at linux.vnet.ibm.com  Thu May 26 13:23:05 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 26 May 2011 22:53:05 +0530
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE generated
	crashes
In-Reply-To: <20110526170722.GB23266@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
Message-ID: <20110526172305.GA18295@in.ibm.com>


slimdump: Capture slimdump for fatal MCE generated crashes

System crashes resulting from fatal hardware errors (such as MCE) don't need
all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
retains only essential information while discarding the old memory.

Signed-off-by: K.Prasad <prasad at linux.vnet.ibm.com>
---
 fs/proc/vmcore.c |  130 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 128 insertions(+), 2 deletions(-)

Index: linux-2.6.slim_kdump/fs/proc/vmcore.c
===================================================================
--- linux-2.6.slim_kdump.orig/fs/proc/vmcore.c
+++ linux-2.6.slim_kdump/fs/proc/vmcore.c
@@ -483,9 +483,60 @@ static void __init set_vmcore_list_offse
 	}
 }
 
+/*
+ * Check if the crash was due to a fatal Memory Check Exception
+ */
+static int is_mce_crash64(void)
+{
+	int i, j, len = 0, rc;
+	Elf64_Ehdr *ehdr_ptr;
+	Elf64_Phdr *phdr_ptr;
+	Elf64_Nhdr *nhdr_ptr;
+
+	ehdr_ptr = (Elf64_Ehdr *)elfcorebuf;
+	phdr_ptr = (Elf64_Phdr *)(elfcorebuf + sizeof(Elf64_Ehdr));
+
+	for (i = 0; i < ehdr_ptr->e_phnum; i++, phdr_ptr++) {
+		void *notes_section;
+		u64 offset, max_sz;
+		if (phdr_ptr->p_type != PT_NOTE)
+			continue;
+		max_sz = phdr_ptr->p_memsz;
+		offset = phdr_ptr->p_offset;
+		notes_section = kmalloc(max_sz, GFP_KERNEL);
+		if (!notes_section)
+			return -ENOMEM;
+		rc = read_from_oldmem(notes_section, max_sz, &offset, 0);
+		if (rc < 0) {
+			kfree(notes_section);
+			return rc;
+		}
+
+		for (j = 0; j < phdr_ptr->p_filesz; j += len) {
+			nhdr_ptr = notes_section + j;
+			if (nhdr_ptr->n_type == NT_MCE)
+			{
+				kfree(notes_section);
+				return 1;
+			}
+			/*
+			 * The elf-64 standard specifies 8-byte alignment while
+			 * append_elf_note function does only 4-byte roundup.
+			 * Hence this code also does a 4-byte roundup.
+			 */
+			len = sizeof(Elf64_Nhdr);
+			len = roundup(len + nhdr_ptr->n_namesz, 4);
+			len = roundup(len + nhdr_ptr->n_descsz, 4);
+		}
+		kfree(notes_section);
+	}
+	return 0;
+}
+
 static int __init parse_crash_elf64_headers(void)
 {
-	int rc=0;
+	int i, rc = 0;
+	Elf64_Phdr *phdr_ptr;
 	Elf64_Ehdr ehdr;
 	u64 addr;
 
@@ -523,6 +574,18 @@ static int __init parse_crash_elf64_head
 		return rc;
 	}
 
+	phdr_ptr = (Elf64_Phdr *)(elfcorebuf + sizeof(Elf64_Ehdr));
+	if (is_mce_crash64() > 0) {
+		/*
+		 * If crash is due to Machine Check exception, don't populate
+		 * sections other than elf-notes. Mark their sizes as zero.
+		 */
+		for (i = 0; i < ehdr.e_phnum; i++, phdr_ptr++) {
+			if (phdr_ptr->p_type != PT_NOTE)
+				phdr_ptr->p_memsz = phdr_ptr->p_filesz = 0;
+		}
+	}
+
 	/* Merge all PT_NOTE headers into one. */
 	rc = merge_note_headers_elf64(elfcorebuf, &elfcorebuf_sz, &vmcore_list);
 	if (rc) {
@@ -539,9 +602,60 @@ static int __init parse_crash_elf64_head
 	return 0;
 }
 
+/*
+ * Check if the crash was due to a fatal Memory Check Exception
+ */
+static int is_mce_crash32(void)
+{
+	int i, j, len = 0, rc;
+	Elf32_Ehdr *ehdr_ptr;
+	Elf32_Phdr *phdr_ptr;
+	Elf32_Nhdr *nhdr_ptr;
+
+	ehdr_ptr = (Elf32_Ehdr *)elfcorebuf;
+	phdr_ptr = (Elf32_Phdr *)(elfcorebuf + sizeof(Elf32_Ehdr));
+
+	for (i = 0; i < ehdr_ptr->e_phnum; i++, phdr_ptr++) {
+		void *notes_section;
+		u64 offset, max_sz;
+		if (phdr_ptr->p_type != PT_NOTE)
+			continue;
+		max_sz = phdr_ptr->p_memsz;
+		offset = phdr_ptr->p_offset;
+		notes_section = kmalloc(max_sz, GFP_KERNEL);
+		if (!notes_section)
+			return -ENOMEM;
+		rc = read_from_oldmem(notes_section, max_sz, &offset, 0);
+		if (rc < 0) {
+			kfree(notes_section);
+			return rc;
+		}
+
+		for (j = 0; j < phdr_ptr->p_filesz; j += len) {
+			nhdr_ptr = notes_section + j;
+			if (nhdr_ptr->n_type == NT_MCE)
+			{
+				kfree(notes_section);
+				return 1;
+			}
+			/*
+			 * The elf-64 standard specifies 8-byte alignment while
+			 * append_elf_note function does only 4-byte roundup.
+			 * Hence this code also does a 4-byte roundup.
+			 */
+			len = sizeof(Elf64_Nhdr);
+			len = roundup(len + nhdr_ptr->n_namesz, 4);
+			len = roundup(len + nhdr_ptr->n_descsz, 4);
+		}
+		kfree(notes_section);
+	}
+	return 0;
+}
+
 static int __init parse_crash_elf32_headers(void)
 {
-	int rc=0;
+	int i, rc = 0;
+	Elf32_Phdr *phdr_ptr;
 	Elf32_Ehdr ehdr;
 	u64 addr;
 
@@ -579,6 +693,18 @@ static int __init parse_crash_elf32_head
 		return rc;
 	}
 
+	phdr_ptr = (Elf32_Phdr *)(elfcorebuf + sizeof(Elf32_Ehdr));
+	if (is_mce_crash32() > 0) {
+		/*
+		 * If crash is due to Machine Check exception, don't populate
+		 * sections other than elf-notes. Mark their sizes as zero.
+		 */
+		for (i = 0; i < ehdr.e_phnum; i++, phdr_ptr++) {
+			if (phdr_ptr->p_type != PT_NOTE)
+				phdr_ptr->p_memsz = phdr_ptr->p_filesz = 0;
+		}
+	}
+
 	/* Merge all PT_NOTE headers into one. */
 	rc = merge_note_headers_elf32(elfcorebuf, &elfcorebuf_sz, &vmcore_list);
 	if (rc) {


From prasad at linux.vnet.ibm.com  Thu May 26 13:15:21 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 26 May 2011 22:45:21 +0530
Subject: [RFC Patch 4/6] PANIC_MCE: Introduce a new panic flag for fatal MCE, 
	capture related information
In-Reply-To: <20110526170722.GB23266@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
Message-ID: <20110526171521.GD17988@in.ibm.com>


PANIC_MCE: Introduce a new panic flag for fatal MCE, capture related information

Fatal machine check exceptions (caused due to hardware memory errors) will now
result in a 'slim' coredump that captures vital information about the MCE. This
patch introduces a new panic flag, and new parameters to *panic functions
that can capture more information pertaining to the cause of crash.

Enable a new elf-notes section to store additional information about the crash.
For MCE, enable a new notes section that captures relevant register status
(struct mce) to be later read during coredump analysis.

Signed-off-by: K.Prasad <prasad at linux.vnet.ibm.com>
---
 arch/arm/kernel/traps.c          |    2 +-
 arch/powerpc/kernel/traps.c      |    2 +-
 arch/sh/kernel/traps_32.c        |    2 +-
 arch/x86/kernel/cpu/mcheck/mce.c |    7 +++----
 arch/x86/kernel/dumpstack.c      |    2 +-
 include/linux/elf.h              |    5 +++++
 include/linux/kernel.h           |    9 +++++----
 include/linux/kexec.h            |    9 ++++++---
 kernel/kexec.c                   |   17 +++++++++++------
 kernel/panic.c                   |   16 ++++++++--------
 10 files changed, 42 insertions(+), 29 deletions(-)

Index: linux-2.6.slim_kdump/include/linux/kernel.h
===================================================================
--- linux-2.6.slim_kdump.orig/include/linux/kernel.h
+++ linux-2.6.slim_kdump/include/linux/kernel.h
@@ -178,17 +178,18 @@ static inline void might_fault(void)
 enum panic_flags {
 	PANIC_NO_KEXEC     = (1 << 0),
 	PANIC_NO_BACKTRACE = (1 << 1),
+	PANIC_MCE          = (1 << 2),
 };
 
 extern struct atomic_notifier_head panic_notifier_list;
 extern long (*panic_blink)(int state);
 NORET_TYPE void panic(const char * fmt, ...)
 	__attribute__ ((NORET_AND format (printf, 1, 2))) __cold;
-NORET_TYPE void xpanic(enum panic_flags flags, int timeout,
-		       const char *fmt, ...)
-	__attribute__ ((NORET_AND format (printf, 3, 4))) __cold;
+NORET_TYPE void xpanic(enum panic_flags flags, int timeout, void *arch_info,
+			size_t arch_info_size, const char *fmt, ...)
+	__attribute__ ((NORET_AND format (printf, 5, 6))) __cold;
 NORET_TYPE void vpanic(enum panic_flags flags, int timeout,
-		       const char *fmt,
+		       void *arch_info, size_t arch_info_size, const char *fmt,
 		       va_list ap) __noreturn __cold;
 extern void oops_enter(void);
 extern void oops_exit(void);
Index: linux-2.6.slim_kdump/kernel/panic.c
===================================================================
--- linux-2.6.slim_kdump.orig/kernel/panic.c
+++ linux-2.6.slim_kdump/kernel/panic.c
@@ -61,21 +61,21 @@ NORET_TYPE void panic(const char *fmt, .
 {
 	va_list ap;
 	va_start(ap, fmt);
-	vpanic(0, 0, fmt, ap);
+	vpanic(0, 0, NULL, 0, fmt, ap);
 }
 EXPORT_SYMBOL(panic);
 
-NORET_TYPE void xpanic(enum panic_flags flags, int timeout,
-                       const char *fmt, ...)
+NORET_TYPE void xpanic(enum panic_flags flags, int timeout, void *arch_info,
+			size_t arch_info_size, const char *fmt, ...)
 {
 	va_list ap;
 	va_start(ap, fmt);
-	vpanic(flags, timeout, fmt, ap);
+	vpanic(flags, timeout, arch_info, arch_info_size, fmt, ap);
 }
 EXPORT_SYMBOL(xpanic);
 
-NORET_TYPE void vpanic(enum panic_flags flags, int timeout,
-		       const char * fmt, va_list args)
+NORET_TYPE void vpanic(enum panic_flags flags, int timeout, void *arch_info,
+			size_t arch_info_size, const char * fmt, va_list args)
 {
 	static char buf[1024];
 	long i, i_next = 0;
@@ -99,7 +99,7 @@ NORET_TYPE void vpanic(enum panic_flags
 	vsnprintf(buf, sizeof(buf), fmt, args);
 	printk(KERN_EMERG "Kernel panic - not syncing: %s\n",buf);
 #ifdef CONFIG_DEBUG_BUGVERBOSE
-	if (!(flags & PANIC_NO_BACKTRACE))
+	if (!(flags & (PANIC_NO_BACKTRACE | PANIC_MCE)))
 		dump_stack();
 #endif
 
@@ -109,7 +109,7 @@ NORET_TYPE void vpanic(enum panic_flags
 	 * Do we want to call this before we try to display a message?
 	 */
 	if (!(flags & PANIC_NO_KEXEC))
-		crash_kexec(NULL);
+		crash_kexec(NULL, arch_info, arch_info_size, flags);
 
 	kmsg_dump(KMSG_DUMP_PANIC);
 
Index: linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
===================================================================
--- linux-2.6.slim_kdump.orig/arch/x86/kernel/cpu/mcheck/mce.c
+++ linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
@@ -258,8 +258,7 @@ static void wait_for_panic(void)
 	local_irq_enable();
 	while (timeout-- > 0)
 		udelay(1);
-	xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, 0,
-		"Panicing machine check CPU died");
+	xpanic(PANIC_MCE, 0, NULL, 0, "Panicing machine check CPU died");
 }
 
 static void mce_panic(char *msg, struct mce *final, char *exp)
@@ -315,8 +314,8 @@ static void mce_panic(char *msg, struct
 	if (exp)
 		pr_emerg(HW_ERR "Machine check: %s\n", exp);
 	if (!fake_panic) {
-		xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, mce_panic_timeout,
-			msg);
+		xpanic(PANIC_MCE, mce_panic_timeout, final,
+			sizeof(struct mce), msg);
 	} else
 		pr_emerg(HW_ERR "Fake kernel panic: %s\n", msg);
 }
Index: linux-2.6.slim_kdump/arch/arm/kernel/traps.c
===================================================================
--- linux-2.6.slim_kdump.orig/arch/arm/kernel/traps.c
+++ linux-2.6.slim_kdump/arch/arm/kernel/traps.c
@@ -274,7 +274,7 @@ void die(const char *str, struct pt_regs
 	ret = __die(str, err, thread, regs);
 
 	if (regs && kexec_should_crash(thread->task))
-		crash_kexec(regs);
+		crash_kexec(regs, NULL, 0, 0);
 
 	bust_spinlocks(0);
 	add_taint(TAINT_DIE);
Index: linux-2.6.slim_kdump/arch/powerpc/kernel/traps.c
===================================================================
--- linux-2.6.slim_kdump.orig/arch/powerpc/kernel/traps.c
+++ linux-2.6.slim_kdump/arch/powerpc/kernel/traps.c
@@ -161,7 +161,7 @@ int die(const char *str, struct pt_regs
 
 	if (kexec_should_crash(current) ||
 		kexec_sr_activated(smp_processor_id()))
-		crash_kexec(regs);
+		crash_kexec(regs, NULL, 0, 0);
 	crash_kexec_secondary(regs);
 
 	if (in_interrupt())
Index: linux-2.6.slim_kdump/arch/sh/kernel/traps_32.c
===================================================================
--- linux-2.6.slim_kdump.orig/arch/sh/kernel/traps_32.c
+++ linux-2.6.slim_kdump/arch/sh/kernel/traps_32.c
@@ -106,7 +106,7 @@ void die(const char * str, struct pt_reg
 	oops_exit();
 
 	if (kexec_should_crash(current))
-		crash_kexec(regs);
+		crash_kexec(regs, NULL, 0, 0);
 
 	if (in_interrupt())
 		panic("Fatal exception in interrupt");
Index: linux-2.6.slim_kdump/arch/x86/kernel/dumpstack.c
===================================================================
--- linux-2.6.slim_kdump.orig/arch/x86/kernel/dumpstack.c
+++ linux-2.6.slim_kdump/arch/x86/kernel/dumpstack.c
@@ -241,7 +241,7 @@ EXPORT_SYMBOL_GPL(oops_begin);
 void __kprobes oops_end(unsigned long flags, struct pt_regs *regs, int signr)
 {
 	if (regs && kexec_should_crash(current))
-		crash_kexec(regs);
+		crash_kexec(regs, NULL, 0, 0);
 
 	bust_spinlocks(0);
 	die_owner = -1;
Index: linux-2.6.slim_kdump/include/linux/elf.h
===================================================================
--- linux-2.6.slim_kdump.orig/include/linux/elf.h
+++ linux-2.6.slim_kdump/include/linux/elf.h
@@ -381,6 +381,11 @@ typedef struct elf64_shdr {
 #define NT_PRPSINFO	3
 #define NT_TASKSTRUCT	4
 #define NT_AUXV		6
+/*
+ * Although numbers 1 - 6 have been defined here, the user-space include files
+ * have numbers 1 - 20 taken up. Hence defining NT_MCE as 21.
+ */
+#define NT_MCE		21		/* Machine Check Exception related data */
 #define NT_PRXFPREG     0x46e62b7f      /* copied from gdb5.1/include/elf/common.h */
 #define NT_PPC_VMX	0x100		/* PowerPC Altivec/VMX registers */
 #define NT_PPC_SPE	0x101		/* PowerPC SPE/EVR registers */
Index: linux-2.6.slim_kdump/include/linux/kexec.h
===================================================================
--- linux-2.6.slim_kdump.orig/include/linux/kexec.h
+++ linux-2.6.slim_kdump/include/linux/kexec.h
@@ -125,10 +125,12 @@ extern asmlinkage long compat_sys_kexec_
 #endif
 extern struct page *kimage_alloc_control_pages(struct kimage *image,
 						unsigned int order);
-extern void crash_kexec(struct pt_regs *);
+extern void crash_kexec(struct pt_regs *, void *arch_info,
+			size_t arch_info_size, enum panic_flags);
 int kexec_should_crash(struct task_struct *);
 void crash_save_cpu(struct pt_regs *regs, int cpu);
-void crash_save_vmcoreinfo(void);
+void crash_save_vmcoreinfo(void *arch_info, size_t arch_info_size,
+						enum panic_flags);
 void arch_crash_save_vmcoreinfo(void);
 void vmcoreinfo_append_str(const char *fmt, ...)
 	__attribute__ ((format (printf, 1, 2)));
@@ -213,7 +215,8 @@ void crash_free_reserved_phys_range(unsi
 #else /* !CONFIG_KEXEC */
 struct pt_regs;
 struct task_struct;
-static inline void crash_kexec(struct pt_regs *regs) { }
+static inline void crash_kexec(struct pt_regs *regs, void *arch_info,
+				size_t arch_info_size, enum panic_flags) { }
 static inline int kexec_should_crash(struct task_struct *p) { return 0; }
 #endif /* CONFIG_KEXEC */
 #endif /* LINUX_KEXEC_H */
Index: linux-2.6.slim_kdump/kernel/kexec.c
===================================================================
--- linux-2.6.slim_kdump.orig/kernel/kexec.c
+++ linux-2.6.slim_kdump/kernel/kexec.c
@@ -1065,7 +1065,8 @@ asmlinkage long compat_sys_kexec_load(un
 }
 #endif
 
-void crash_kexec(struct pt_regs *regs)
+void crash_kexec(struct pt_regs *regs, void *arch_info, size_t arch_info_size,
+		 enum panic_flags flags)
 {
 	/* Take the kexec_mutex here to prevent sys_kexec_load
 	 * running on one cpu from replacing the crash kernel
@@ -1082,7 +1083,7 @@ void crash_kexec(struct pt_regs *regs)
 			kmsg_dump(KMSG_DUMP_KEXEC);
 
 			crash_setup_regs(&fixed_regs, regs);
-			crash_save_vmcoreinfo();
+			crash_save_vmcoreinfo(arch_info, arch_info_size, flags);
 			machine_crash_shutdown(&fixed_regs);
 			machine_kexec(kexec_crash_image);
 		}
@@ -1381,7 +1382,8 @@ int __init parse_crashkernel(char 		 *cm
 
 
 
-void crash_save_vmcoreinfo(void)
+void crash_save_vmcoreinfo(void *arch_info, size_t arch_info_size,
+						enum panic_flags flags)
 {
 	u32 *buf;
 
@@ -1392,9 +1394,12 @@ void crash_save_vmcoreinfo(void)
 
 	buf = (u32 *)vmcoreinfo_note;
 
-	buf = append_elf_note(buf, VMCOREINFO_NOTE_NAME, 0, vmcoreinfo_data,
-			      vmcoreinfo_size);
-
+	if (flags & PANIC_MCE)
+		buf = append_elf_note(buf, "PANIC_MCE", NT_MCE, arch_info,
+					arch_info_size);
+	else
+		buf = append_elf_note(buf, VMCOREINFO_NOTE_NAME, 0,
+				vmcoreinfo_data, vmcoreinfo_size);
 	final_note(buf);
 }
 


From prasad at linux.vnet.ibm.com  Thu May 26 13:14:22 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 26 May 2011 22:44:22 +0530
Subject: [Bugfix][Patch 3/3] Invoke vpanic inside xpanic function
In-Reply-To: <20110526170722.GB23266@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
Message-ID: <20110526171422.GC17988@in.ibm.com>


Invoke vpanic inside xpanic function

xpanic must invoke the worker routine vpanic to perform the erstwhile panic
related function. Instead it erroneously invokes xpanic (recursively) which
needs to be fixed.

Signed-off-by: K.Prasad <prasad at linux.vnet.ibm.com>
---
 kernel/panic.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-2.6.slim_kdump/kernel/panic.c
===================================================================
--- linux-2.6.slim_kdump.orig/kernel/panic.c
+++ linux-2.6.slim_kdump/kernel/panic.c
@@ -70,7 +70,7 @@ NORET_TYPE void xpanic(enum panic_flags
 {
 	va_list ap;
 	va_start(ap, fmt);
-	xpanic(flags, timeout, fmt, ap);
+	vpanic(flags, timeout, fmt, ap);
 }
 EXPORT_SYMBOL(xpanic);
 


From prasad at linux.vnet.ibm.com  Thu May 26 13:12:51 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 26 May 2011 22:42:51 +0530
Subject: [Patch 2/6] x86: mce: Convert mce code to xpanic
In-Reply-To: <20110526170722.GB23266@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
Message-ID: <20110526171251.GB17988@in.ibm.com>


commit aea29c4a9324f24b5c61e7c3919a8137a53be935
Author: Andi Kleen <ak at linux.intel.com>
Date:   Fri Nov 19 18:42:02 2010 +0100

    x86: mce: Convert mce code to xpanic
    
    - Pass in the panic timeout directly instead of
    abusing global variable.
    - Disable backtraces and kexecs on machine check panics
    because they don't do anything useful.
    
    Signed-off-by: Andi Kleen <ak at linux.intel.com>

---
 arch/x86/kernel/cpu/mcheck/mce.c |   10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

Index: linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
===================================================================
--- linux-2.6.slim_kdump.orig/arch/x86/kernel/cpu/mcheck/mce.c
+++ linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
@@ -258,9 +258,8 @@ static void wait_for_panic(void)
 	local_irq_enable();
 	while (timeout-- > 0)
 		udelay(1);
-	if (panic_timeout == 0)
-		panic_timeout = mce_panic_timeout;
-	panic("Panicing machine check CPU died");
+	xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, 0,
+		"Panicing machine check CPU died");
 }
 
 static void mce_panic(char *msg, struct mce *final, char *exp)
@@ -316,9 +315,8 @@ static void mce_panic(char *msg, struct
 	if (exp)
 		pr_emerg(HW_ERR "Machine check: %s\n", exp);
 	if (!fake_panic) {
-		if (panic_timeout == 0)
-			panic_timeout = mce_panic_timeout;
-		panic(msg);
+		xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, mce_panic_timeout,
+			msg);
 	} else
 		pr_emerg(HW_ERR "Fake kernel panic: %s\n", msg);
 }


From prasad at linux.vnet.ibm.com  Thu May 26 13:12:09 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 26 May 2011 22:42:09 +0530
Subject: [Patch 1/6] XPANIC: Add extended panic interface
In-Reply-To: <20110526170722.GB23266@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
Message-ID: <20110526171209.GA17988@in.ibm.com>


commit e668fa1aea7844ac4c7ea09030a2f3e647a4adb1
Author: Andi Kleen <ak at linux.intel.com>
Date:   Fri Nov 19 18:36:44 2010 +0100

    XPANIC: Add extended panic interface
    
    One panic size doesn't fit all.
    
    Machine check has some special requirements on panic, like:
    - It wants to do a reboot timeout by default so that the machine
    check event can be logged to disk after warm reboot.
    - For memory errors it usually doesn't want to do crash dumps because
    that can lead to crash loops (dumping corrupted memory would
    lead to another machine check)
    - It doesn't want to do backtraces because machine checks
    are not a kernel bug.
    
    In a earlier patch this was done with various adhoc hacks,
    but it's cleaner to extend panic to a 'xpanic' that directly
    gets a flag and timeout argument.
    
    The only user right now will be x86 machine checks, but I consider
    it likely that other users will switch to this too.
    
    For example one obvious candidate would be the "no root
    found" panic which doesn't really deserve a backtrace.
    
    I exported a vpanic() interface too as a global. That's not
    needed by the current user, but the interface has to exist
    internally anyways and I could see how other code would
    find a v* variant of panic useful.
    
    Originally based on a suggestion by H. Peter Anvin.
    Signed-off-by: Andi Kleen <ak at linux.intel.com>

---
 include/linux/kernel.h |   11 +++++++++++
 kernel/panic.c         |   41 +++++++++++++++++++++++++++++++----------
 2 files changed, 42 insertions(+), 10 deletions(-)

Index: linux-2.6.slim_kdump/include/linux/kernel.h
===================================================================
--- linux-2.6.slim_kdump.orig/include/linux/kernel.h
+++ linux-2.6.slim_kdump/include/linux/kernel.h
@@ -175,10 +175,21 @@ static inline void might_fault(void)
 }
 #endif
 
+enum panic_flags {
+	PANIC_NO_KEXEC     = (1 << 0),
+	PANIC_NO_BACKTRACE = (1 << 1),
+};
+
 extern struct atomic_notifier_head panic_notifier_list;
 extern long (*panic_blink)(int state);
 NORET_TYPE void panic(const char * fmt, ...)
 	__attribute__ ((NORET_AND format (printf, 1, 2))) __cold;
+NORET_TYPE void xpanic(enum panic_flags flags, int timeout,
+		       const char *fmt, ...)
+	__attribute__ ((NORET_AND format (printf, 3, 4))) __cold;
+NORET_TYPE void vpanic(enum panic_flags flags, int timeout,
+		       const char *fmt,
+		       va_list ap) __noreturn __cold;
 extern void oops_enter(void);
 extern void oops_exit(void);
 void print_oops_end_marker(void);
Index: linux-2.6.slim_kdump/kernel/panic.c
===================================================================
--- linux-2.6.slim_kdump.orig/kernel/panic.c
+++ linux-2.6.slim_kdump/kernel/panic.c
@@ -57,14 +57,37 @@ EXPORT_SYMBOL(panic_blink);
  *
  *	This function never returns.
  */
-NORET_TYPE void panic(const char * fmt, ...)
+NORET_TYPE void panic(const char *fmt, ...)
+{
+	va_list ap;
+	va_start(ap, fmt);
+	vpanic(0, 0, fmt, ap);
+}
+EXPORT_SYMBOL(panic);
+
+NORET_TYPE void xpanic(enum panic_flags flags, int timeout,
+                       const char *fmt, ...)
+{
+	va_list ap;
+	va_start(ap, fmt);
+	xpanic(flags, timeout, fmt, ap);
+}
+EXPORT_SYMBOL(xpanic);
+
+NORET_TYPE void vpanic(enum panic_flags flags, int timeout,
+		       const char * fmt, va_list args)
 {
 	static char buf[1024];
-	va_list args;
 	long i, i_next = 0;
 	int state = 0;
 
 	/*
+	 * Let user always override panic_timeout.
+	 */
+	if (panic_timeout > 0)
+		timeout = panic_timeout;
+
+	/*
 	 * It's possible to come here directly from a panic-assertion and
 	 * not have preempt disabled. Some functions called from here want
 	 * preempt to be disabled. No point enabling it later though...
@@ -73,12 +96,11 @@ NORET_TYPE void panic(const char * fmt,
 
 	console_verbose();
 	bust_spinlocks(1);
-	va_start(args, fmt);
 	vsnprintf(buf, sizeof(buf), fmt, args);
-	va_end(args);
 	printk(KERN_EMERG "Kernel panic - not syncing: %s\n",buf);
 #ifdef CONFIG_DEBUG_BUGVERBOSE
-	dump_stack();
+	if (!(flags & PANIC_NO_BACKTRACE))
+		dump_stack();
 #endif
 
 	/*
@@ -86,7 +108,8 @@ NORET_TYPE void panic(const char * fmt,
 	 * everything else.
 	 * Do we want to call this before we try to display a message?
 	 */
-	crash_kexec(NULL);
+	if (!(flags & PANIC_NO_KEXEC))
+		crash_kexec(NULL);
 
 	kmsg_dump(KMSG_DUMP_PANIC);
 
@@ -104,7 +127,7 @@ NORET_TYPE void panic(const char * fmt,
 	if (!panic_blink)
 		panic_blink = no_blink;
 
-	if (panic_timeout > 0) {
+	if (timeout > 0) {
 		/*
 		 * Delay timeout seconds before rebooting the machine.
 		 * We can't use the "normal" timers since we just panicked.
@@ -152,9 +175,7 @@ NORET_TYPE void panic(const char * fmt,
 		mdelay(PANIC_TIMER_STEP);
 	}
 }
-
-EXPORT_SYMBOL(panic);
-
+EXPORT_SYMBOL(vpanic);
 
 struct tnt {
 	u8	bit;


From prasad at linux.vnet.ibm.com  Thu May 26 13:07:22 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Thu, 26 May 2011 22:37:22 +0530
Subject: [RFC Patch 0/6] slimdump: Enable slimdump if crashing kernel memory
	is not required
Message-ID: <20110526170722.GB23266@in.ibm.com>

Hi All,
	Please find a patchset that implements 'slimdump' - a new type
of coredump (captured through kdump) which will retain only essential
debugging information but will discard old kernel memory (which is mostly
irrelevant) in case of system crashes triggered through fatal hardware errors.

When a system crashes due to an unrecoverable memory error, kdump would
ordinarily attempt to read/capture the crashing kernel's memory which has
the following undesirable consequences:
- The old kernel memory is irrelevant for debugging crashes initiated due
  to hardware errors (such as physical memory corruption causing fatal
  machine check exceptions - MCE).
- The kdump capture kernel might experience a second MCE when attempting
  to read the corrupt memory area having fatal results (and failure to
  capture the coredump).

'slimdump' is particularly useful in avoiding the above hazards. It is a
light-weight dump and increases the reliability of the system by avoiding
unsafe operations.

In essence slimdump enables light-weight coredumps and increases the
reliability of the system by avoiding unsafe operations.

Summary of the patches
----
Patch 1, 2 - Patches from Andi Kleen's tree
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6.git which
introduce an enhanced panic function - xpanic().

Patch 3 - Introduces a new PANIC_MCE flag to signal the need for
SlimDump for
MCE initiated panic calls.

Patch 4 - Append a new elf-note inside coredump containing MCE related
registers

Patch 5 - Create a SlimDump by discarding old kernel's memory

Patch 6 - Patch for 'crash' tool
(http://people.redhat.com/anderson/crash-5.1.5.tar.gz) to recognise the
new
elf-notes section.

The patches have been tested from inside a VM and have been found to
work fine. The user-space tool used for copying the coredump is 'cp'
(makedumpfile testing will be done soon).

Example
--------
Normal vmcore
-----------
# ls -lh /home/prasadkr/vmcore.usual
-r-------- 1 root root 1.6G May 17 01:07 /home/prasadkr/vmcore.usual

slimdump
-----------
# ls -lh /home/prasadkr/vmcore.SlimDump
-r-------- 1 root root 1.8K May 18 23:28 /home/prasadkr/vmcore.SlimDump
# ./crash vmlinux vmcore.SlimDump

crash 5.1.5
Copyright (C) 2002-2011  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public
License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for
details.

"System crashed due to a hardware memory error. No coredump available."
#
#

Thanks,
K.Prasad
_______________________________________________________
ltcras mailing list ltcras at lists.linux.ibm.com
To unsubscribe from the list, change your list options
or if you have forgotten your list password visit:
http://lists.linux.ibm.com/mailman/listinfo/ltcras


From horms at verge.net.au  Thu May 26 19:44:01 2011
From: horms at verge.net.au (Simon Horman)
Date: Fri, 27 May 2011 08:44:01 +0900
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526191008.GF29496@redhat.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
	<20110526174447.GC29496@redhat.com>
	<20110526180931.GF4065@one.firstfloor.org>
	<20110526182618.GD29496@redhat.com>
	<20110526185838.GG4065@one.firstfloor.org>
	<20110526191008.GF29496@redhat.com>
Message-ID: <20110526234401.GE2815@verge.net.au>

On Thu, May 26, 2011 at 03:10:08PM -0400, Vivek Goyal wrote:
> On Thu, May 26, 2011 at 08:58:38PM +0200, Andi Kleen wrote:
> > > If we are just extracting and saving MCE registers from vmcore, then
> > > reboot time does not increase. It increases only if user decides to
> > > extract and save extra data from vmcore.
> > 
> > Hmm I was thinking of user space usually saving the dump first
> > before analyzing it. But yes it could probably do some minimal
> > analysis first.
> 
> In RHEL, now we filter out the dump by default until and unless user
> decides to no filter the dump with the help of config options. 
> 
> So I think in this case we can just introduce an extra filtering
> option in "makedumpfile" and ask it to save only MCE registers
> if it notices that there is NT_MCE type of ELF note in vmcore.
> 
> I can very well imaging that extracting dmesg along with MCE registers
> can be useful. If nothing else, it can give us useful information about
> the system configuration.
> 
> Zeroing out all the other ELF headers in vmcore takes away the
> capability to extract log buffers in case of MCE and I think
> it probably is not the best idea.
> 

FWIW, my initial thoughts are inline with Vivek's.
I think that the bulk of this, if not all of it, can
be done in user space. And that user space is the preferable
location for such logic.



From BUENDGEN at de.ibm.com  Thu May 26 07:15:14 2011
From: BUENDGEN at de.ibm.com (Reinhard Buendgen)
Date: Thu, 26 May 2011 13:15:14 +0200
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to filter
	out kernel data from vmcore
In-Reply-To: <20110525195321.GD6724@redhat.com>
References: <20110524203542.GH3860@redhat.com>
	<1621913139.202658.1306270052667.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com>
	<OF341326C7.0D81F89A-ONC125789B.002DB6ED-C125789B.002FC7A0@de.ibm.com>
	<20110525195321.GD6724@redhat.com>
Message-ID: <OF05104FA0.A922963A-ONC125789C.003BDAA9-C125789C.003DD130@de.ibm.com>

Vivek,

I/O is not restricted to disk I/O (it may be network I/O, data sent to 
crtypto cards etc.) and it is not always direct, Device drivers may have 
buffers to which such data is copied. 

So it is more than just keys, and it may change over time.
I do not think hardwiring a filter in makedumpfile is a good idea because 
you would need a new makedumpfile release for every Distribution 
(release).

Allowing to configure makedumpfile allows each distribution and each 
platform to provide appropriate filters.

Mit freundlichen Gr??en/Best Regards/Cordialement 

Reinhard

Dr. Reinhard B?ndgen 
RAS & Crypto Architect for Linux on System z 
Virtualization and Systems Management 
 
Mail:buendgen at de.ibm.com
Phone: ++49-(0)7031-16-1130
Fax: ++49-(0)7031-16-3456 
 
IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Martin Jetter
Gesch?ftsf?hrung: Dirk Wittkopp 
Sitz der Gesellschaft: B?blingen
Registergericht: Amtsgericht Stuttgart, HRB 243294





From:   Vivek Goyal <vgoyal at redhat.com>
To:     Reinhard Buendgen/Germany/IBM at IBMDE
Cc:     Dave Anderson <anderson at redhat.com>, Ananth N Mavinakayanahalli 
<ananth at in.ibm.com>, kexec at lists.infradead.org, Mahesh J Salgaonkar 
<mahesh at linux.vnet.ibm.com>, "Ken'ichi Ohmichi" 
<oomichi at mxs.nes.nec.co.jp>, V Srivatsa <vsrivatsa at in.ibm.com>
Date:   25.05.2011 21:53
Subject:        Re: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement 
to filter out kernel data from vmcore



On Wed, May 25, 2011 at 10:41:55AM +0200, Reinhard Buendgen wrote:
> Hi,
> 
> to answer Vivek questions first: Eventually we want to be able to erase 
> all data that a customer may consider sensitive to her privacy. In 
> addition to encryption key that may be the contents (i.e. payload 
within) 
> of all kinds of I/O buffers. Consider you are running a kvm based 
> hypervisor and want its dump to be analyized while promising your 
> customers whose guests you run on that hypervisor that none of their 
data 
> will be externalized. Or consider your system reads a spreadsheat with 
> bank account or health information. You might not want to send fractions 

> of that information sitting in some buffers to a service organization.

So for direct IO, buffer is still in user space and should be filtered
out when we filter out user space pages using mkdumpfile. For kvm, I am
assuming that all the pages belong to qemu process and once we are
filtering out user space pages, any data belonging to guest will go away.

So atleast for above examples it does not sound as if we need symbol
erase infrastructure.

> 
> to answer Daves concern: there is no intention that crash should ever 
look 
> into the erased structures. In theroy it should not be needed because 
the 
> contents of structures to be deleted should be irrelevant to kernel 
> debugging.

So what are those kernel structures which we are planning to delete and
are irrelevant to kernel debugging by crash?

I think we are missing something here. If there are only few known
structures we want to get rid of, lets hardcode it in makedumpfile
instead of giving user a generic infrastructure. That way we know
that we are not leaking information at the same time making sure
that analysis tools are working.

Thanks
Vivek

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20110526/f8451157/attachment-0001.html>

From mahesh at linux.vnet.ibm.com  Fri May 27 02:56:21 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Fri, 27 May 2011 12:26:21 +0530
Subject: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement to
	filter out kernel data from vmcore
In-Reply-To: <20110526133923.GA29496@redhat.com>
References: <20110524203542.GH3860@redhat.com>
	<1621913139.202658.1306270052667.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com>
	<OF341326C7.0D81F89A-ONC125789B.002DB6ED-C125789B.002FC7A0@de.ibm.com>
	<20110525195321.GD6724@redhat.com>
	<OF05104FA0.A922963A-ONC125789C.003BDAA9-C125789C.003DD130@de.ibm.com>
	<20110526133923.GA29496@redhat.com>
Message-ID: <20110527065621.GA26119@in.ibm.com>

On 2011-05-26 09:39:23 Thu, Vivek Goyal wrote:
> On Thu, May 26, 2011 at 01:15:14PM +0200, Reinhard Buendgen wrote:
> > Vivek,
> > 
> > I/O is not restricted to disk I/O (it may be network I/O, data sent to 
> > crtypto cards etc.) and it is not always direct, Device drivers may have 
> > buffers to which such data is copied. 
> > 
> > So it is more than just keys, and it may change over time.
> > I do not think hardwiring a filter in makedumpfile is a good idea because 
> > you would need a new makedumpfile release for every Distribution 
> > (release).
> 
> Ok, so we are worried about data being in slub/slab caches or driver's
> kmalloced buffers etc.
> 
> When do I need access to debuginfo files. I am assuming that makedumpfile
> reads them in first kernel sometime and stores relevant info in initramfs.
> Otherwise, it is not possible to get to it in second kernel after crash.
> 

The current approach is to re-run the makedumpfile on the crash dump
(generated in the second kernel) when we are back into production kernel
(1st kernel).

> > 
> > Allowing to configure makedumpfile allows each distribution and each 
> > platform to provide appropriate filters.
> > 
> 
> I was thinking that there are very few symbols and these will not change
> frequently. makdumpfile is already dependent on some kenrel data strcutres
> and symbols for doing filtering. 
> 
> But looks like you are planning to filter out lots of symbols. So in that
> case agreed that hardcoding is not a good idea.
> 
> Thanks
> Vivek
> 
> > Mit freundlichen Gr??en/Best Regards/Cordialement 
> > 
> > Reinhard
> > 
> > Dr. Reinhard B?ndgen 
> > RAS & Crypto Architect for Linux on System z 
> > Virtualization and Systems Management 
> >  
> > Mail:buendgen at de.ibm.com
> > Phone: ++49-(0)7031-16-1130
> > Fax: ++49-(0)7031-16-3456 
> >  
> > IBM Deutschland Research & Development GmbH
> > Vorsitzender des Aufsichtsrats: Martin Jetter
> > Gesch?ftsf?hrung: Dirk Wittkopp 
> > Sitz der Gesellschaft: B?blingen
> > Registergericht: Amtsgericht Stuttgart, HRB 243294
> > 
> > 
> > 
> > 
> > 
> > From:   Vivek Goyal <vgoyal at redhat.com>
> > To:     Reinhard Buendgen/Germany/IBM at IBMDE
> > Cc:     Dave Anderson <anderson at redhat.com>, Ananth N Mavinakayanahalli 
> > <ananth at in.ibm.com>, kexec at lists.infradead.org, Mahesh J Salgaonkar 
> > <mahesh at linux.vnet.ibm.com>, "Ken'ichi Ohmichi" 
> > <oomichi at mxs.nes.nec.co.jp>, V Srivatsa <vsrivatsa at in.ibm.com>
> > Date:   25.05.2011 21:53
> > Subject:        Re: [PATCH v2 0/8] makedumpfile: makedumpfile enhancement 
> > to filter out kernel data from vmcore
> > 
> > 
> > 
> > On Wed, May 25, 2011 at 10:41:55AM +0200, Reinhard Buendgen wrote:
> > > Hi,
> > > 
> > > to answer Vivek questions first: Eventually we want to be able to erase 
> > > all data that a customer may consider sensitive to her privacy. In 
> > > addition to encryption key that may be the contents (i.e. payload 
> > within) 
> > > of all kinds of I/O buffers. Consider you are running a kvm based 
> > > hypervisor and want its dump to be analyized while promising your 
> > > customers whose guests you run on that hypervisor that none of their 
> > data 
> > > will be externalized. Or consider your system reads a spreadsheat with 
> > > bank account or health information. You might not want to send fractions 
> > 
> > > of that information sitting in some buffers to a service organization.
> > 
> > So for direct IO, buffer is still in user space and should be filtered
> > out when we filter out user space pages using mkdumpfile. For kvm, I am
> > assuming that all the pages belong to qemu process and once we are
> > filtering out user space pages, any data belonging to guest will go away.
> > 
> > So atleast for above examples it does not sound as if we need symbol
> > erase infrastructure.
> > 
> > > 
> > > to answer Daves concern: there is no intention that crash should ever 
> > look 
> > > into the erased structures. In theroy it should not be needed because 
> > the 
> > > contents of structures to be deleted should be irrelevant to kernel 
> > > debugging.
> > 
> > So what are those kernel structures which we are planning to delete and
> > are irrelevant to kernel debugging by crash?
> > 
> > I think we are missing something here. If there are only few known
> > structures we want to get rid of, lets hardcode it in makedumpfile
> > instead of giving user a generic infrastructure. That way we know
> > that we are not leaking information at the same time making sure
> > that analysis tools are working.
> > 
> > Thanks
> > Vivek
> > 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec

-- 
Mahesh J Salgaonkar


From mahesh at linux.vnet.ibm.com  Fri May 27 11:37:00 2011
From: mahesh at linux.vnet.ibm.com (Mahesh J Salgaonkar)
Date: Fri, 27 May 2011 21:07:00 +0530
Subject: [RFC Patch 6/6] Crash: Recognise slim coredumps and process
	new elf-note sections
In-Reply-To: <20110526172618.GB18295@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172618.GB18295@in.ibm.com>
Message-ID: <20110527153700.GA13007@in.ibm.com>

On 2011-05-26 22:56:18 Thu, K.Prasad wrote:
> 
> Crash: Recognise slim coredumps and process new elf-note sections
> 
> The Linux kernel will begin to support SlimDump for certain types of crashes
> and the 'crash' tool needs to recognise them. For these types of coredumps, it
> need not lookout for usual elf-structures and start gdb. Also process new
> elf-note sections that contain additional information about the crash.
> 
> Signed-off-by: K.Prasad <prasad at linux.vnet.ibm.com>
> ---
>  diskdump.c |   84 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  netdump.c  |    8 +++++
>  x86.h      |   91 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 183 insertions(+)
> 
> Index: crash-5.1.5.slim_kdump/x86.h
> ===================================================================
> --- /dev/null
> +++ crash-5.1.5.slim_kdump/x86.h
> @@ -0,0 +1,91 @@
> +/*
> + * x86.h - x86 Architecture specific definitions
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, write to the Free Software
> + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
> + *
> + * Copyright (C) IBM Corporation, 2011
> + *
> + * Author: K.Prasad <prasad at linux.vnet.ibm.com>
> + */
> +
> +typedef unsigned long long u64;
> +typedef unsigned int u32;
> +typedef unsigned short u16;
> +typedef unsigned char u8;
> +
> +#define __u64 u64
> +#define __u32 u32
> +#define __u16 u16
> +#define __u8  u8
> +
> +/* Mask for finding the address mode in IA32_MCi_MISC[8:6] register */
> +#define MCI_MISC_ADDR_MODE	0X1C0
> +/* Number of bits to shift the IA32_MCi_MISC to read the address-mode bits */
> +#define MISC_ADDR_MODE_POS	6
> +
> +/* Address Modes in IA32_MCi_MISC[8:6] */
> +#define MCM_ADDR_SEGOFF  0      /* segment offset */
> +#define MCM_ADDR_LINEAR  1      /* linear address */
> +#define MCM_ADDR_PHYS    2      /* physical address */
> +#define MCM_ADDR_MEM     3      /* memory address */
> +#define MCM_ADDR_GENERIC 7      /* generic */
> +
> +#define MCI_STATUS_MISCV (1ULL<<59)  /* misc error reg. valid */
> +#define MCI_STATUS_ADDRV (1ULL<<58)  /* addr reg. valid */
> +
> +#define PAGE_SHIFT 12
> +
> +static const char *mce_addr_mode[] =
> +{
> +	"Segment offset",	/* MCM_ADDR_SEGOFF */
> +	"Linear address",	/* MCM_ADDR_LINEAR */
> +	"Physical address",	/* MCM_ADDR_PHYS */
> +	"Memory address",	/* MCM_ADDR_MEM */
> +	"",			/* reserved */
> +	"",			/* reserved */
> +	"",			/* reserved */
> +	"Generic"		/* MCM_ADDR_GENERIC */
> +};
> +
> +/*
> + * kernel structure: Keep this in sync with the definition in
> + * arch/x86/include/asm/mce.h of linux source code.
> + *
> + * Fields are zero when not available
> + *
> + */
> +struct mce {
> +	__u64 status;
> +	__u64 misc;
> +	__u64 addr;
> +	__u64 mcgstatus;
> +	__u64 ip;
> +	__u64 tsc;	/* cpu time stamp counter */
> +	__u64 time;	/* wall time_t when error was detected */
> +	__u8  cpuvendor;	/* cpu vendor as encoded in system.h */
> +	__u8  pad1;
> +	__u16 pad2;
> +	__u32 cpuid;	/* CPUID 1 EAX */
> +	__u8  cs;		/* code segment */
> +	__u8  bank;	/* machine check bank */
> +	__u8  cpu;	/* cpu number; obsolete; use extcpu now */
> +	__u8  finished;   /* entry is valid */
> +	__u32 extcpu;	/* linux cpu number that detected the error */
> +	__u32 socketid;	/* CPU socket ID */
> +	__u32 apicid;	/* CPU initial apic ID */
> +	__u64 mcgcap;	/* MCGCAP MSR: machine check capabilities of CPU */
> +	__u64 aux0;
> +	__u64 aux1;
> +};
> Index: crash-5.1.5.slim_kdump/netdump.c
> ===================================================================
> --- crash-5.1.5.slim_kdump.orig/netdump.c
> +++ crash-5.1.5.slim_kdump/netdump.c
> @@ -331,6 +331,10 @@ is_netdump(char *file, ulong source_quer
>  		}
>          	nd->notes32 = (Elf32_Phdr *)
>  		    &nd->elf_header[sizeof(Elf32_Ehdr)];
> +		if (machdep->process_elf_notes)
> +			machdep->process_elf_notes((char *)nd->elf32 +
> +							nd->notes32->p_offset,
> +							nd->notes32->p_filesz);
>          	nd->load32 = (Elf32_Phdr *)
>  		    &nd->elf_header[sizeof(Elf32_Ehdr)+sizeof(Elf32_Phdr)];
>  		if (DUMPFILE_FORMAT(nd->flags) == NETDUMP_ELF32)
> @@ -360,6 +364,10 @@ is_netdump(char *file, ulong source_quer
>                  }
>                  nd->notes64 = (Elf64_Phdr *)
>                      &nd->elf_header[sizeof(Elf64_Ehdr)];
> +		if (machdep->process_elf_notes)
> +			machdep->process_elf_notes((char *)nd->elf64 +
> +							nd->notes64->p_offset,
> +							nd->notes64->p_filesz);

Now that machdep->process_elf_notes() is invoked in generic KDUMP
processing code path, please remove the separate invocation of
machdep->dumpfile_init() which was introduced for s390x architecture in
dump_Elf64_Nhdr() function. The reason is, machdep->process_elf_notes()
on s390x internally invokes machdep->dumpfile_init(). Hence we can
safely remove it.

>                  nd->load64 = (Elf64_Phdr *)
>                      &nd->elf_header[sizeof(Elf64_Ehdr)+sizeof(Elf64_Phdr)];
>  		if (DUMPFILE_FORMAT(nd->flags) == NETDUMP_ELF64)
> Index: crash-5.1.5.slim_kdump/diskdump.c
> ===================================================================
> --- crash-5.1.5.slim_kdump.orig/diskdump.c
> +++ crash-5.1.5.slim_kdump/diskdump.c
> @@ -231,6 +231,27 @@ open_dump_file(char *file)
>  	dd->dfd = fd;
>  	return TRUE;
>  }
> +#if defined(X86_64) || defined(X86)
> +#include "x86.h"
> +
> +/*
> + * Check if the address reported by the CPU is in a format we can parse.
> + * It would be possible to add code for most other cases, but all would
> + * be somewhat complicated (e.g. segment offset would require an instruction
> + * parser). So only support physical addresses up to page granuality for now.
> + *
> + * Function derived from arch/x86/kernel/cpu/mcheck/mce.c in Linux source
> + *
> + */
> +static int mce_usable_address(struct mce *m)
> +{
> +	if (!(m->status & MCI_STATUS_MISCV) || !(m->status & MCI_STATUS_ADDRV))
> +		return 0;
> +	if ((m->misc & 0x3f) > PAGE_SHIFT)
> +		return 0;
> +	return 1;
> +}
> +#endif /* defined(X86_64) || defined(X86) */
> 
>  void 
>  x86_process_elf_notes(void *note_ptr, unsigned long size_note)
> @@ -239,10 +260,43 @@ x86_process_elf_notes(void *note_ptr, un
>  	Elf64_Nhdr *note64 = NULL;
>  	size_t tot, len = 0;
>  	int num = 0;
> +#if defined(X86_64) || defined(X86)
> +	struct mce *mce;
> +	ushort addr_mode;
> +#endif /* defined(X86_64) || defined(X86) */
> 
>  	for (tot = 0; tot < size_note; tot += len) {
>  		if (machine_type("X86_64")) {
>  			note64 = note_ptr + tot;
> +#ifdef X86_64
> +			/*
> +			 * If vmcore is generated due to fatal Machine Check
> +			 * Exception, we only have a 'slim' crashdump. Don't
> +			 * analyse further, inform the user about it and exit.
> +			 */
> +			if (note64->n_type == NT_MCE) {
> +				fprintf(fp, "\"System crashed due to a hardware"
> +					" memory error. No coredump"
> +					" available.\"\n");
> +
> +				/* Do we have a copy of 'struct mce'? */
> +				if (note64->n_descsz == 0)
> +					goto exit;
> +
> +				mce = (struct mce *)((char *)note64 +
> +					sizeof(Elf64_Nhdr) + note64->n_namesz);
> +				if (!mce_usable_address(mce))
> +					goto exit;
> +
> +				addr_mode = (mce->misc >> MISC_ADDR_MODE_POS) &
> +						MCI_MISC_ADDR_MODE;
> +				fprintf(fp, "Memory error occured at %llx "
> +					"(address type: %s\n)", mce->addr,
> +					mce_addr_mode[addr_mode]);
> +exit:
> +				clean_exit(0);
> +			}
> +#endif /* X86_64 */

The function x86_process_elf_notes() is invoked through
machdep->process_elf_notes function pointer, which makes it arch
dependent code. How about moving this function (x86_process_elf_notes)
to an arch dependent file say x86_common.c ? By doing so we can get rid of
all "#ifdefs" here.

Hi Dave, what do you say?

> 
>  			if (note64->n_type == NT_PRSTATUS) {
>  				dd->nt_prstatus_percpu[num] = note64;
> @@ -255,6 +309,36 @@ x86_process_elf_notes(void *note_ptr, un
>  		} else if (machine_type("X86")) {
>  			note32 = note_ptr + tot;
> 
> +#ifdef X86
> +			/*
> +			 * If vmcore is generated due to fatal Machine Check
> +			 * Exception, we only have a 'slim' crashdump. Don't
> +			 * analyse further, inform the user about it and exit.
> +			 */
> +			if (note32->n_type == NT_MCE) {
> +				fprintf(fp, "\"System crashed due to a hardware"
> +					" memory error. No coredump"
> +					" available.\"\n");
> +
> +				/* Do we have a copy of 'struct mce'? */
> +				if (note32->n_descsz == 0)
> +					goto exit;
> +
> +				mce = (struct mce *)((char *)note32 +
> +					sizeof(Elf32_Nhdr) + note32->n_namesz);
> +				if (!mce_usable_address(mce))
> +					goto exit;
> +
> +				addr_mode = (mce->misc >> MISC_ADDR_MODE_POS) &
> +						MCI_MISC_ADDR_MODE;
> +				fprintf(fp, "Memory error occured at %llx "
> +					"(address type: %s\n)", mce->addr,
> +					mce_addr_mode[addr_mode]);
> +exit:
> +				clean_exit(0);
> +			}
> +#endif /* X86 */
> +
>  			if (note32->n_type == NT_PRSTATUS) {
>  				dd->nt_prstatus_percpu[num] = note32;
>  				num++;

-- 
Mahesh J Salgaonkar


From prasad at linux.vnet.ibm.com  Fri May 27 11:53:46 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Fri, 27 May 2011 21:23:46 +0530
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526173257.GC4065@one.firstfloor.org>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
	<20110526173257.GC4065@one.firstfloor.org>
Message-ID: <20110527155346.GA2384@in.ibm.com>

On Thu, May 26, 2011 at 07:32:57PM +0200, Andi Kleen wrote:
> On Thu, May 26, 2011 at 10:53:05PM +0530, K.Prasad wrote:
> > 
> > slimdump: Capture slimdump for fatal MCE generated crashes
> > 
> > System crashes resulting from fatal hardware errors (such as MCE) don't need
> > all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
> > retains only essential information while discarding the old memory.
> 
> While this is a good idea, note there may be still poisoned lines
> in memory that haven't resulted in a machine check yet, but could
> still be fatal when read after a full crash dump for some other
> reason.
>

True, this patch does not handle the discovery of old poisoned lines/new
memory errors that may occur when inside the kdump kernel.
 
> So you still need 
> 
> http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=commit;h=fe61906edce9e70d02481a77a617ba1397573dce
> and
> http://git.kernel.org/?p=linux/kernel/git/ak/linux-mce-2.6.git;a=commit;h=cb58f049ae6709ddbab71be199390dc6852018cd
> 
> in addition.
> 
> -Andi

So, there could be (atleast) two ways to handle fatal MCEs in kdump
kernel:

- To disable MCE exceptions as done by the patches cited above. However
  the result of a read operation on corrupted memory is unknown and the
  system behaviour is undefined. We're unsure if this is a safe thing to
  do.

- To disable capture of kdump (when panic is invoked from) inside kdump
  kernel and simply reboot the system. Since the chance of memory error
  inside kdump kernel (which runs for a very short duration) is rare, I
  think this solution is preferrable.

Let me know your thoughts on this.

Thanks,
K.Prasad




From prasad at linux.vnet.ibm.com  Fri May 27 11:56:56 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Fri, 27 May 2011 21:26:56 +0530
Subject: [Patch 1/6] XPANIC: Add extended panic interface
In-Reply-To: <BANLkTi=ZrPkRwFSmdkAB60-MKS3fwZOjpg@mail.gmail.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171209.GA17988@in.ibm.com>
	<BANLkTi=ZrPkRwFSmdkAB60-MKS3fwZOjpg@mail.gmail.com>
Message-ID: <20110527155656.GB2384@in.ibm.com>

On Thu, May 26, 2011 at 07:38:19PM +0200, richard -rw- weinberger wrote:
> On Thu, May 26, 2011 at 7:12 PM, K.Prasad <prasad at linux.vnet.ibm.com> wrote:
> >
> > commit e668fa1aea7844ac4c7ea09030a2f3e647a4adb1
> > Author: Andi Kleen <ak at linux.intel.com>
> > Date: ? Fri Nov 19 18:36:44 2010 +0100
> >
> > Index: linux-2.6.slim_kdump/kernel/panic.c
> > ===================================================================
> > --- linux-2.6.slim_kdump.orig/kernel/panic.c
> > +++ linux-2.6.slim_kdump/kernel/panic.c
> > @@ -57,14 +57,37 @@ EXPORT_SYMBOL(panic_blink);
> > ?*
> > ?* ? ? This function never returns.
> > ?*/
> > -NORET_TYPE void panic(const char * fmt, ...)
> > +NORET_TYPE void panic(const char *fmt, ...)
> > +{
> > + ? ? ? va_list ap;
> > + ? ? ? va_start(ap, fmt);
> > + ? ? ? vpanic(0, 0, fmt, ap);
> > +}
> > +EXPORT_SYMBOL(panic);
> > +
> > +NORET_TYPE void xpanic(enum panic_flags flags, int timeout,
> > + ? ? ? ? ? ? ? ? ? ? ? const char *fmt, ...)
> > +{
> > + ? ? ? va_list ap;
> > + ? ? ? va_start(ap, fmt);
> > + ? ? ? xpanic(flags, timeout, fmt, ap);
> 
> Why are you calling xpanic() here again?
> I guess you meant vpanic().
>

True, this is a bug in the original patch found in the git tree
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6.git, for
which I have proposed a patch here: https://lkml.org/lkml/2011/5/26/337.

Thanks,
K.Prasad
 


From prasad at linux.vnet.ibm.com  Fri May 27 12:57:36 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Fri, 27 May 2011 22:27:36 +0530
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526174447.GC29496@redhat.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
	<20110526174447.GC29496@redhat.com>
Message-ID: <20110527165736.GC2384@in.ibm.com>

On Thu, May 26, 2011 at 01:44:47PM -0400, Vivek Goyal wrote:
> On Thu, May 26, 2011 at 10:53:05PM +0530, K.Prasad wrote:
> > 
> > slimdump: Capture slimdump for fatal MCE generated crashes
> > 
> > System crashes resulting from fatal hardware errors (such as MCE) don't need
> > all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
> > retains only essential information while discarding the old memory.
> > 
> 
> Why to enforce zeroing out of rest of the vmcore data in kernel. Why not
> leave it to user space. 
> 

Our concern comes from the fact that it is unsafe for the OS to read
any part of the corrupt memory region, so the kernel does not have to make
that address space available for read/write.

> As Andi said, you anyway will require disabling MCE temporarily in second
> kernel. So that should allay the concern that we might run into second
> MCE while trying to read the vmcore.
> 

The processor manuals don't define the behaviour for a read operation
upon a corrupt memory location. So the consequences for a read after
disabling MCE is unknown (as I've mentioned here:
https://lkml.org/lkml/2011/5/27/258). 

> A user might want to extract just the dmesg buffers also after MCE from
> vmcore.
>
> What I am saying is that this sounds more like a policy. Don't enforce
> it in kernel. Probably give user MCE registers as part of ELF note of
> vmcore. Now leave it up to user what data he wants to extract from vmcore.
> Just the MCE registers or something more then that.

I'm unsure, at the moment, as to what it actually entails to extract
dmesg buffers from the old kernel's memory; but given that there exists
a corrupt memory region with unknown consequences for read operations
upon it, I doubt if it is a safe idea to allow user-space application to
navigate through the kernel data-structures to extract the dmesg.

Alternatively, we might leave MCE enabled for the kdump kernel and
modify the user-space application to turn resilient to SIGBUS signal. So
if it reads a corrupt memory region, it will receive a signal from
do_machine_check() upon which it can skip that particular address and/or
fill it with zeroes (or somesuch). We haven't gone through the
details....but just some loud thinking

Thanks,
K.Prasad


From prasad at linux.vnet.ibm.com  Fri May 27 13:03:31 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Fri, 27 May 2011 22:33:31 +0530
Subject: [RFC Patch 4/6] PANIC_MCE: Introduce a new panic flag for fatal
	MCE, capture related information
In-Reply-To: <20110526184350.GE29496@redhat.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171521.GD17988@in.ibm.com>
	<20110526184350.GE29496@redhat.com>
Message-ID: <20110527170331.GD2384@in.ibm.com>

On Thu, May 26, 2011 at 02:43:50PM -0400, Vivek Goyal wrote:
> On Thu, May 26, 2011 at 10:45:21PM +0530, K.Prasad wrote:
> 
> [..]
> > Index: linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
> > ===================================================================
> > --- linux-2.6.slim_kdump.orig/arch/x86/kernel/cpu/mcheck/mce.c
> > +++ linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
> > @@ -258,8 +258,7 @@ static void wait_for_panic(void)
> >  	local_irq_enable();
> >  	while (timeout-- > 0)
> >  		udelay(1);
> > -	xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, 0,
> > -		"Panicing machine check CPU died");
> > +	xpanic(PANIC_MCE, 0, NULL, 0, "Panicing machine check CPU died");
> >  }
> >  
> >  static void mce_panic(char *msg, struct mce *final, char *exp)
> > @@ -315,8 +314,8 @@ static void mce_panic(char *msg, struct
> >  	if (exp)
> >  		pr_emerg(HW_ERR "Machine check: %s\n", exp);
> >  	if (!fake_panic) {
> > -		xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, mce_panic_timeout,
> > -			msg);
> > +		xpanic(PANIC_MCE, mce_panic_timeout, final,
> > +			sizeof(struct mce), msg);
> >  	} else
> >  		pr_emerg(HW_ERR "Fake kernel panic: %s\n", msg);
> >  }
> 
> In previous patches you introduce PANIC_NO_KEXEC and PANIC_NO_BACKTRACE. 
> Now in this patch you got rid of those. Are there any other users left
> of PANIC_NO_BACKTRACE and PANIC_NO_EXEC? If not, then why to introduce
> these to begin with.
>

The previous patch also converts panic to xpanic and is taken from
Andi's
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6.git tree.
The changes are kept as two separate patches to identify their origin.

Thanks,
K.Prasad

 


From ebiederm at xmission.com  Fri May 27 13:59:11 2011
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Fri, 27 May 2011 10:59:11 -0700
Subject: [Patch 1/6] XPANIC: Add extended panic interface
In-Reply-To: <20110526171209.GA17988@in.ibm.com> (K. Prasad's message of "Thu, 
	26 May 2011 22:42:09 +0530")
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171209.GA17988@in.ibm.com>
Message-ID: <m1tycg9g9c.fsf@fess.ebiederm.org>

"K.Prasad" <prasad at linux.vnet.ibm.com> writes:

> commit e668fa1aea7844ac4c7ea09030a2f3e647a4adb1
> Author: Andi Kleen <ak at linux.intel.com>
> Date:   Fri Nov 19 18:36:44 2010 +0100
>
>     XPANIC: Add extended panic interface
>     
>     One panic size doesn't fit all.
>     
>     Machine check has some special requirements on panic, like:
>     - It wants to do a reboot timeout by default so that the machine
>     check event can be logged to disk after warm reboot.

There is certainly not comment to that effect and I have not see a
mechanism that does that.

>     - For memory errors it usually doesn't want to do crash dumps because
>     that can lead to crash loops (dumping corrupted memory would
>     lead to another machine check)

It usually doesn't, and it would not lead to a crash loop just a failure
to capture the cause of the crash.

>     - It doesn't want to do backtraces because machine checks
>     are not a kernel bug.

But it sure is nice to know where it happened anyway, so you can guess
the damage.  This sounds like more of the Andi thing where he doesn't
want to hear about memory errors, because it is not a software problem.

That is definitely not sufficient reason to turn off backtraces.

>     In a earlier patch this was done with various adhoc hacks,
>     but it's cleaner to extend panic to a 'xpanic' that directly
>     gets a flag and timeout argument.
>     
>     The only user right now will be x86 machine checks, but I consider
>     it likely that other users will switch to this too.
>     
>     For example one obvious candidate would be the "no root
>     found" panic which doesn't really deserve a backtrace.

That does seem like a good fit for that case.  However it doesn't seem
like a good fit for your case.

>     I exported a vpanic() interface too as a global. That's not
>     needed by the current user, but the interface has to exist
>     internally anyways and I could see how other code would
>     find a v* variant of panic useful.
>     
>     Originally based on a suggestion by H. Peter Anvin.
>     Signed-off-by: Andi Kleen <ak at linux.intel.com>

Nacked-by: "Eric W. Biederman" <ebiederm at xmission.com>

This entire chunk of two patches appears to be complete non-sense.

mce_panic_timeout is hard coded policy that userspace does not get
to control, that overrides userspace policy.

It looks to me like the right solution is just to delete the 
mce_panic_timeout.

As for the extra flags that you add the panic path.  They can
only make the code more fragile.

Eric


From vgoyal at redhat.com  Fri May 27 13:59:49 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Fri, 27 May 2011 13:59:49 -0400
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110527165736.GC2384@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
	<20110526174447.GC29496@redhat.com>
	<20110527165736.GC2384@in.ibm.com>
Message-ID: <20110527175949.GG8053@redhat.com>

On Fri, May 27, 2011 at 10:27:36PM +0530, K.Prasad wrote:
> On Thu, May 26, 2011 at 01:44:47PM -0400, Vivek Goyal wrote:
> > On Thu, May 26, 2011 at 10:53:05PM +0530, K.Prasad wrote:
> > > 
> > > slimdump: Capture slimdump for fatal MCE generated crashes
> > > 
> > > System crashes resulting from fatal hardware errors (such as MCE) don't need
> > > all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
> > > retains only essential information while discarding the old memory.
> > > 
> > 
> > Why to enforce zeroing out of rest of the vmcore data in kernel. Why not
> > leave it to user space. 
> > 
> 
> Our concern comes from the fact that it is unsafe for the OS to read
> any part of the corrupt memory region, so the kernel does not have to make
> that address space available for read/write.

The very fact you are booting into second kernel you are reading lots
of address space already. The assumption is that whole of the reserved
region is fine otherwise kernel will not even boot. Then even in older
kernel you are accessing memory used for saving the elf headers and
mce registers.

> 
> > As Andi said, you anyway will require disabling MCE temporarily in second
> > kernel. So that should allay the concern that we might run into second
> > MCE while trying to read the vmcore.
> > 
> 
> The processor manuals don't define the behaviour for a read operation
> upon a corrupt memory location. So the consequences for a read after
> disabling MCE is unknown (as I've mentioned here:
> https://lkml.org/lkml/2011/5/27/258). 
> 

First of all a user can simply have a configuration to extract MCE
registers and your concern is addressed. Eric W. Biederman said that
he had got bunch of MCE triggered dumps also and things were fine. I think
we are over engineering this. In the first step lets just keep it simple
and just export MCE registers as a note. If accessing rest of regions
becomes a issue in real life, then we can have a look at this again.


> > A user might want to extract just the dmesg buffers also after MCE from
> > vmcore.
> >
> > What I am saying is that this sounds more like a policy. Don't enforce
> > it in kernel. Probably give user MCE registers as part of ELF note of
> > vmcore. Now leave it up to user what data he wants to extract from vmcore.
> > Just the MCE registers or something more then that.
> 
> I'm unsure, at the moment, as to what it actually entails to extract
> dmesg buffers from the old kernel's memory; but given that there exists
> a corrupt memory region with unknown consequences for read operations
> upon it, I doubt if it is a safe idea to allow user-space application to
> navigate through the kernel data-structures to extract the dmesg.
> 
> Alternatively, we might leave MCE enabled for the kdump kernel and
> modify the user-space application to turn resilient to SIGBUS signal. So
> if it reads a corrupt memory region, it will receive a signal from
> do_machine_check() upon which it can skip that particular address and/or
> fill it with zeroes (or somesuch). We haven't gone through the
> details....but just some loud thinking

Again, I think simplest would be to disable MCE while we are accessing
previous kernel's memory and configure user space to either just extract
MCE registers. If you are too paranoid, you can do it in two steps. First
save MCE registes and make sure these are on disk and then go for
extracting rest of the info like dmesg and in the process if you go down,
anyway it was best effort thing.

Thanks
Vivek


From ebiederm at xmission.com  Fri May 27 14:01:29 2011
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Fri, 27 May 2011 11:01:29 -0700
Subject: [Patch 2/6] x86: mce: Convert mce code to xpanic
In-Reply-To: <20110526171251.GB17988@in.ibm.com> (K. Prasad's message of "Thu, 
	26 May 2011 22:42:51 +0530")
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171251.GB17988@in.ibm.com>
Message-ID: <m1lixs9g5i.fsf@fess.ebiederm.org>

"K.Prasad" <prasad at linux.vnet.ibm.com> writes:

> commit aea29c4a9324f24b5c61e7c3919a8137a53be935
> Author: Andi Kleen <ak at linux.intel.com>
> Date:   Fri Nov 19 18:42:02 2010 +0100
>
>     x86: mce: Convert mce code to xpanic
>     
>     - Pass in the panic timeout directly instead of
>     abusing global variable.
>     - Disable backtraces and kexecs on machine check panics
>     because they don't do anything useful.

Nacked-by: "Eric W. Biederman" <ebiederm at xmission.com>

Kexec on panic does not fact do something useful, and usable
today.

The mce_panic_timeout looks like a horrible hack that should
be killed.  Why do we think we know better than the sysadmin
that sets up the system.

Eric


From ebiederm at xmission.com  Fri May 27 14:04:06 2011
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Fri, 27 May 2011 11:04:06 -0700
Subject: [RFC Patch 4/6] PANIC_MCE: Introduce a new panic flag for fatal
	MCE, capture related information
In-Reply-To: <20110526171521.GD17988@in.ibm.com> (K. Prasad's message of "Thu, 
	26 May 2011 22:45:21 +0530")
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171521.GD17988@in.ibm.com>
Message-ID: <m1fwo09g15.fsf@fess.ebiederm.org>

"K.Prasad" <prasad at linux.vnet.ibm.com> writes:

> PANIC_MCE: Introduce a new panic flag for fatal MCE, capture related information
>
> Fatal machine check exceptions (caused due to hardware memory errors) will now
> result in a 'slim' coredump that captures vital information about the MCE. This
> patch introduces a new panic flag, and new parameters to *panic functions
> that can capture more information pertaining to the cause of crash.
>
> Enable a new elf-notes section to store additional information about the crash.
> For MCE, enable a new notes section that captures relevant register status
> (struct mce) to be later read during coredump analysis.

There may be a reason to pass everything struct mce through 5 layers of
code but right now it looks like it just makes everything uglier to no
real purpose.

Eric


From ebiederm at xmission.com  Fri May 27 14:09:44 2011
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Fri, 27 May 2011 11:09:44 -0700
Subject: [RFC Patch 4/6] PANIC_MCE: Introduce a new panic flag for fatal
	MCE, capture related information
In-Reply-To: <20110526171521.GD17988@in.ibm.com> (K. Prasad's message of "Thu, 
	26 May 2011 22:45:21 +0530")
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171521.GD17988@in.ibm.com>
Message-ID: <m1vcww817b.fsf@fess.ebiederm.org>

"K.Prasad" <prasad at linux.vnet.ibm.com> writes:

> PANIC_MCE: Introduce a new panic flag for fatal MCE, capture related information
>
> Fatal machine check exceptions (caused due to hardware memory errors) will now
> result in a 'slim' coredump that captures vital information about the MCE. This
> patch introduces a new panic flag, and new parameters to *panic functions
> that can capture more information pertaining to the cause of crash.
>
> Enable a new elf-notes section to store additional information about the crash.
> For MCE, enable a new notes section that captures relevant register status
> (struct mce) to be later read during coredump analysis.
>
> Signed-off-by: K.Prasad <prasad at linux.vnet.ibm.com>

Nacked-by: "Eric W. Biederman" <ebiederm at xmission.com>


> -void crash_save_vmcoreinfo(void)
> +void crash_save_vmcoreinfo(void *arch_info, size_t arch_info_size,
> +						enum panic_flags flags)
>  {
>  	u32 *buf;
>  
> @@ -1392,9 +1394,12 @@ void crash_save_vmcoreinfo(void)
>  
>  	buf = (u32 *)vmcoreinfo_note;
>  
> -	buf = append_elf_note(buf, VMCOREINFO_NOTE_NAME, 0, vmcoreinfo_data,
> -			      vmcoreinfo_size);
> -
> +	if (flags & PANIC_MCE)
> +		buf = append_elf_note(buf, "PANIC_MCE", NT_MCE, arch_info,
> +					arch_info_size);

Ok.  This is uglier than I thought.  You have this terribly generic
interface than when you go to use it you assume it is an mce.  Ugh
no.

Furthermore you short circuit out other pieces of this code.

This patch is a total hack that is massively over engineered.

Saving NT_MCE is fine, in and of itself.  The rest of the information
is totally nonsense.

Eric


> +	else
> +		buf = append_elf_note(buf, VMCOREINFO_NOTE_NAME, 0,
> +				vmcoreinfo_data, vmcoreinfo_size);
>  	final_note(buf);
>  }
>  


From ebiederm at xmission.com  Fri May 27 14:14:00 2011
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Fri, 27 May 2011 11:14:00 -0700
Subject: [RFC Patch 5/6] slimdump: Capture slimdump for fatal MCE
	generated crashes
In-Reply-To: <20110526172305.GA18295@in.ibm.com> (K. Prasad's message of "Thu, 
	26 May 2011 22:53:05 +0530")
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172305.GA18295@in.ibm.com>
Message-ID: <m1fwo08107.fsf@fess.ebiederm.org>

"K.Prasad" <prasad at linux.vnet.ibm.com> writes:

> slimdump: Capture slimdump for fatal MCE generated crashes
>
> System crashes resulting from fatal hardware errors (such as MCE) don't need
> all the contents from crashing-kernel's memory. Generate a new 'slimdump' that
> retains only essential information while discarding the old memory.

Ugh.  You are changing the kernel to change userspace policy?

And because of how you have previous populated this we won't get the
notes to let us extract dmesg easily.

No thanks.  Let's change this in userspace instead.

You are going to an awful lot of work to avoid a problem that doesn't
happen to in practice.

Patching makedumpfile to key in on an mce note would be more appropriate.

Nacked-by: "Eric W. Biederman" <ebiederm at xmission.com>


From ebiederm at xmission.com  Fri May 27 14:16:34 2011
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Fri, 27 May 2011 11:16:34 -0700
Subject: [RFC Patch 6/6] Crash: Recognise slim coredumps and process new
	elf-note sections
In-Reply-To: <20110526172618.GB18295@in.ibm.com> (K. Prasad's message of "Thu, 
	26 May 2011 22:56:18 +0530")
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172618.GB18295@in.ibm.com>
Message-ID: <m11uzk80vx.fsf@fess.ebiederm.org>

"K.Prasad" <prasad at linux.vnet.ibm.com> writes:

> Crash: Recognise slim coredumps and process new elf-note sections
>
> The Linux kernel will begin to support SlimDump for certain types of crashes
> and the 'crash' tool needs to recognise them. For these types of coredumps, it
> need not lookout for usual elf-structures and start gdb. Also process new
> elf-note sections that contain additional information about the crash.

I suppose patching crash make sense.

Unfortunately crash doesn't work on  99% of the  kernels I run so, so I
stopped caring a while ago.

Eric


From vgoyal at redhat.com  Fri May 27 14:22:10 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Fri, 27 May 2011 14:22:10 -0400
Subject: [RFC Patch 6/6] Crash: Recognise slim coredumps and process new
	elf-note sections
In-Reply-To: <m11uzk80vx.fsf@fess.ebiederm.org>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172618.GB18295@in.ibm.com>
	<m11uzk80vx.fsf@fess.ebiederm.org>
Message-ID: <20110527182210.GH8053@redhat.com>

On Fri, May 27, 2011 at 11:16:34AM -0700, Eric W. Biederman wrote:
> "K.Prasad" <prasad at linux.vnet.ibm.com> writes:
> 
> > Crash: Recognise slim coredumps and process new elf-note sections
> >
> > The Linux kernel will begin to support SlimDump for certain types of crashes
> > and the 'crash' tool needs to recognise them. For these types of coredumps, it
> > need not lookout for usual elf-structures and start gdb. Also process new
> > elf-note sections that contain additional information about the crash.
> 
> I suppose patching crash make sense.
> 
> Unfortunately crash doesn't work on  99% of the  kernels I run so, so I
> stopped caring a while ago.

Are these patched kernels? Otherwise Dave Anderson is pretty good about
keeping crash running across multiple kernels.

For me it works just great. I recently analyzed a crash dump with
2.6.39-rc7 kernel. Dave is generally takes patches for crash for latest
upstream kernel changes.

Thanks
Vivek


From vgoyal at redhat.com  Fri May 27 14:29:07 2011
From: vgoyal at redhat.com (Vivek Goyal)
Date: Fri, 27 May 2011 14:29:07 -0400
Subject: [RFC Patch 4/6] PANIC_MCE: Introduce a new panic flag for fatal
	MCE, capture related information
In-Reply-To: <20110527170331.GD2384@in.ibm.com>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171521.GD17988@in.ibm.com>
	<20110526184350.GE29496@redhat.com>
	<20110527170331.GD2384@in.ibm.com>
Message-ID: <20110527182907.GI8053@redhat.com>

On Fri, May 27, 2011 at 10:33:31PM +0530, K.Prasad wrote:
> On Thu, May 26, 2011 at 02:43:50PM -0400, Vivek Goyal wrote:
> > On Thu, May 26, 2011 at 10:45:21PM +0530, K.Prasad wrote:
> > 
> > [..]
> > > Index: linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
> > > ===================================================================
> > > --- linux-2.6.slim_kdump.orig/arch/x86/kernel/cpu/mcheck/mce.c
> > > +++ linux-2.6.slim_kdump/arch/x86/kernel/cpu/mcheck/mce.c
> > > @@ -258,8 +258,7 @@ static void wait_for_panic(void)
> > >  	local_irq_enable();
> > >  	while (timeout-- > 0)
> > >  		udelay(1);
> > > -	xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, 0,
> > > -		"Panicing machine check CPU died");
> > > +	xpanic(PANIC_MCE, 0, NULL, 0, "Panicing machine check CPU died");
> > >  }
> > >  
> > >  static void mce_panic(char *msg, struct mce *final, char *exp)
> > > @@ -315,8 +314,8 @@ static void mce_panic(char *msg, struct
> > >  	if (exp)
> > >  		pr_emerg(HW_ERR "Machine check: %s\n", exp);
> > >  	if (!fake_panic) {
> > > -		xpanic(PANIC_NO_KEXEC|PANIC_NO_BACKTRACE, mce_panic_timeout,
> > > -			msg);
> > > +		xpanic(PANIC_MCE, mce_panic_timeout, final,
> > > +			sizeof(struct mce), msg);
> > >  	} else
> > >  		pr_emerg(HW_ERR "Fake kernel panic: %s\n", msg);
> > >  }
> > 
> > In previous patches you introduce PANIC_NO_KEXEC and PANIC_NO_BACKTRACE. 
> > Now in this patch you got rid of those. Are there any other users left
> > of PANIC_NO_BACKTRACE and PANIC_NO_EXEC? If not, then why to introduce
> > these to begin with.
> >
> 
> The previous patch also converts panic to xpanic and is taken from
> Andi's
> git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6.git tree.
> The changes are kept as two separate patches to identify their origin.

If you have to remove dead piece of code from original patch, I think
you can always take original patch, modify it and give the credit to
original author by explicitly mentioning it in your changelog.

Thanks
Vivek


From ebiederm at xmission.com  Fri May 27 14:35:53 2011
From: ebiederm at xmission.com (Eric W. Biederman)
Date: Fri, 27 May 2011 11:35:53 -0700
Subject: [RFC Patch 6/6] Crash: Recognise slim coredumps and process new
	elf-note sections
In-Reply-To: <20110527182210.GH8053@redhat.com> (Vivek Goyal's message of
	"Fri, 27 May 2011 14:22:10 -0400")
References: <20110526170722.GB23266@in.ibm.com>
	<20110526172618.GB18295@in.ibm.com> <m11uzk80vx.fsf@fess.ebiederm.org>
	<20110527182210.GH8053@redhat.com>
Message-ID: <m1hb8g6lfa.fsf@fess.ebiederm.org>

Vivek Goyal <vgoyal at redhat.com> writes:

> On Fri, May 27, 2011 at 11:16:34AM -0700, Eric W. Biederman wrote:
>> "K.Prasad" <prasad at linux.vnet.ibm.com> writes:
>> 
>> > Crash: Recognise slim coredumps and process new elf-note sections
>> >
>> > The Linux kernel will begin to support SlimDump for certain types of crashes
>> > and the 'crash' tool needs to recognise them. For these types of coredumps, it
>> > need not lookout for usual elf-structures and start gdb. Also process new
>> > elf-note sections that contain additional information about the crash.
>> 
>> I suppose patching crash make sense.
>> 
>> Unfortunately crash doesn't work on  99% of the  kernels I run so, so I
>> stopped caring a while ago.
>
> Are these patched kernels? Otherwise Dave Anderson is pretty good about
> keeping crash running across multiple kernels.

Only slightly patched.  But they are definitely kernels I have built
myself.  It might just be a logistics problem where if I want to use
crash I need to upgrade it when I upgrade my kernels.

I got frustrated a while ago trying to get something out of crash
(and I'm afraid it shows), and I have just gone to extracting the
dmesg.  Which gives me all of the information I generally have time
to process.

Eric


From tachibana at mxm.nes.nec.co.jp  Mon May 30 01:12:22 2011
From: tachibana at mxm.nes.nec.co.jp (tachibana at mxm.nes.nec.co.jp)
Date: Mon, 30 May 2011 14:12:22 +0900
Subject: [PATCH] makedumpfile: s390x: Fix dump refiltering for s390x.
In-Reply-To: <20110525074907.17025.26375.stgit@mars.in.ibm.com>
References: <20110525074907.17025.26375.stgit@mars.in.ibm.com>
Message-ID: <20110530141222tachibana@mail.jp.nec.com>

Hi Mahesh,

Thank you for the patch.
I will take it into a next makedumpfile.


Thanks
tachibana

On 2011/05/25 13:19:08 +0530, Mahesh J Salgaonkar <mahesh at linux.vnet.ibm.com> wrote:
> From: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
> 
> On s390x, dump refiltering fails in get_machdep_info_s390x() function while
> reading vmlist kernel symbol data. It fails in vaddr_to_paddr_s390x()
> function while resolving vmlist kernel symbol address. Since the vmlist
> kernel symbol address is not a vmalloc address, the function returns failure.
> 
> Fix the vaddr_to_paddr_s390x() function to translate non-vmalloc'ed
> address into physical adddress.
> 
> Signed-off-by: Mahesh Salgaonkar <mahesh at linux.vnet.ibm.com>
> ---
>  s390x.c |    4 +---
>  1 files changed, 1 insertions(+), 3 deletions(-)
> 
> diff --git a/s390x.c b/s390x.c
> index 128e5f0..49658d0 100644
> --- a/s390x.c
> +++ b/s390x.c
> @@ -269,9 +269,7 @@ vaddr_to_paddr_s390x(unsigned long vaddr)
>  		paddr = vtop_s390x(vaddr);
>  	}
>  	else {
> -		ERRMSG("Can't convert a virtual address(%lx) to " \
> -		    "physical address.\n", vaddr);
> -		return NOT_PADDR;
> +		paddr = vaddr - KVBASE;
>  	}
>  
>  	return paddr;
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From suzuki at in.ibm.com  Mon May 30 02:30:06 2011
From: suzuki at in.ibm.com (Suzuki Poulose)
Date: Mon, 30 May 2011 12:00:06 +0530
Subject: [RFC][PATCH] powerpc: Use the #address-cells information to parse
	memory/reg
In-Reply-To: <4DCA285D.2080006@in.ibm.com>
References: <4DC923C4.6080707@in.ibm.com> <4DC92784.2040001@in.ibm.com>
	<20110510134152.GB18537@in.ibm.com> <4DCA285D.2080006@in.ibm.com>
Message-ID: <4DE3396E.4090801@in.ibm.com>

Use the #address-cells, #size-cells information to parse the memory/reg info
from  device tree.

The format of memory/reg is based on the #address-cells,#size-cells. Currently,
the kexec-tools doesn't use the above values in parsing the memory/reg values.
Hence the kexec cannot handle cases where #address-cells, #size-cells are
different, (for e.g, PPC440X ).

This patch introduces a read_memory_region_limits(), which parses the
memory/reg contents based on the values of #address-cells and #size-cells.

Changed the add_usable_mem_property() to accept FILE* fp instead of int fd,
as most of the other users of read_memory_region_limits() deals with FILE*.

Signed-off-by: Suzuki K. Poulose <suzuki at in.ibm.com>

---
  kexec/arch/ppc/crashdump-powerpc.c |   23 ------
  kexec/arch/ppc/fs2dt.c             |   31 ++------
  kexec/arch/ppc/kexec-ppc.c         |  136 ++++++++++++++++++++++++++-----------
  kexec/arch/ppc/kexec-ppc.h         |    6 +
  4 files changed, 118 insertions(+), 78 deletions(-)

Index: kexec-tools-2.0.4/kexec/arch/ppc/kexec-ppc.c
===================================================================
--- kexec-tools-2.0.4.orig/kexec/arch/ppc/kexec-ppc.c
+++ kexec-tools-2.0.4/kexec/arch/ppc/kexec-ppc.c
@@ -26,6 +26,7 @@
  
  #include "config.h"
  
+unsigned long dt_address_cells = 0, dt_size_cells = 0;
  uint64_t rmo_top;
  unsigned long long crash_base = 0, crash_size = 0;
  unsigned long long initrd_base = 0, initrd_size = 0;
@@ -34,6 +35,92 @@ unsigned int rtas_base, rtas_size;
  int max_memory_ranges;
  const char *ramdisk;
  
+/*
+ * Reads the #address-cells and #size-cells on this platform.
+ * This is used to parse the memory/reg info from the device-tree
+ */
+int init_memory_region_info()
+{
+	size_t res = 0;
+	FILE *fp;
+	char *file;
+
+	file = "/proc/device-tree/#address-cells";
+	fp = fopen(file, "r");
+	if (!fp) {
+		fprintf(stderr,"Unable to open %s\n", file);
+		return -1;
+	}
+
+	res = fread(&dt_address_cells,sizeof(unsigned long),1,fp);
+	if (res != 1) {
+		fprintf(stderr,"Error reading %s\n", file);
+		return -1;
+	}
+	fclose(fp);
+	dt_address_cells *= sizeof(unsigned long);
+
+	file = "/proc/device-tree/#size-cells";
+	fp = fopen(file, "r");
+	if (!fp) {
+		fprintf(stderr,"Unable to open %s\n", file);
+		return -1;
+	}
+
+	res = fread(&dt_size_cells,sizeof(unsigned long),1,fp);
+	if (res != 1) {
+		fprintf(stderr,"Error reading %s\n", file);
+		return -1;
+	}
+	fclose(fp);
+	dt_size_cells *= sizeof(unsigned long);
+
+	return 0;
+}
+
+#define MAXBYTES 128
+/*
+ * Reads the memory region info from the device-tree node pointed
+ * by @fp and fills the *start, *end with the boundaries of the region
+ */
+int read_memory_region_limits(FILE* fp, unsigned long long *start,
+				unsigned long long *end)
+{
+	char buf[MAXBYTES];
+	unsigned long *p;
+	unsigned long nbytes = dt_address_cells + dt_size_cells;
+
+	if (fread(buf, 1, MAXBYTES, fp) != nbytes) {
+		fprintf(stderr, "Error reading the memory region info\n");
+		return -1;
+	}
+
+	p = (unsigned long*)buf;
+	if (dt_address_cells == sizeof(unsigned long)) {
+		*start = p[0];
+		p++;
+	} else if (dt_address_cells == sizeof(unsigned long long)) {
+		*start = ((unsigned long long *)p)[0];
+		p = (unsigned long long *)p + 1;
+	} else {
+		fprintf(stderr,"Unsupported value for #address-cells : %ld\n",
+					dt_address_cells);
+		return -1;
+	}
+
+	if (dt_size_cells == sizeof(unsigned long))
+		*end = *start + p[0];
+	else if (dt_size_cells == sizeof(unsigned long long))
+		*end = *start + ((unsigned long long *)p)[0];
+	else {
+		fprintf(stderr,"Unsupported value for #size-cells : %ld\n",
+					dt_size_cells);
+		return -1;
+	}
+
+	return 0;
+}
+
  void arch_reuse_initrd(void)
  {
  	reuse_initrd = 1;
@@ -182,9 +269,6 @@ static int sort_base_ranges(void)
  	return 0;
  }
  
-
-#define MAXBYTES 128
-
  static int realloc_memory_ranges(void)
  {
  	size_t memory_range_len;
@@ -248,6 +332,8 @@ static int get_base_ranges(void)
  			return -1;
  		}
  		while ((mentry = readdir(dmem)) != NULL) {
+			unsigned long long start, end;
+
  			if (strcmp(mentry->d_name, "reg"))
  				continue;
  			strcat(fname, "/reg");
@@ -257,8 +343,7 @@ static int get_base_ranges(void)
  				closedir(dir);
  				return -1;
  			}
-			if ((n = fread(buf, 1, MAXBYTES, file)) < 0) {
-				perror(fname);
+			if (read_memory_region_limits(file, &start, &end) != 0) {
  				fclose(file);
  				closedir(dmem);
  				closedir(dir);
@@ -271,24 +356,8 @@ static int get_base_ranges(void)
  				}
  			}
  
-			if (n == sizeof(uint32_t) * 2) {
-				base_memory_range[local_memory_ranges].start =
-					((uint32_t *)buf)[0];
-				base_memory_range[local_memory_ranges].end  =
-					base_memory_range[local_memory_ranges].start +
-					((uint32_t *)buf)[1];
-			}
-			else if (n == sizeof(uint64_t) * 2) {
-				base_memory_range[local_memory_ranges].start =
-                                        ((uint64_t *)buf)[0];
-                                base_memory_range[local_memory_ranges].end  =
-                                        base_memory_range[local_memory_ranges].start +
-                                        ((uint64_t *)buf)[1];
-			}
-			else {
-				fprintf(stderr, "Mem node has invalid size: %d\n", n);
-				return -1;
-			}
+			base_memory_range[local_memory_ranges].start = start;
+			base_memory_range[local_memory_ranges].end  = end;
  			base_memory_range[local_memory_ranges].type = RANGE_RAM;
  			local_memory_ranges++;
  			dbgprintf("%016llx-%016llx : %x\n",
@@ -577,20 +646,10 @@ static int get_devtree_details(unsigned
  				perror(fname);
  				goto error_opencdir;
  			}
-			if ((n = fread(buf, 1, MAXBYTES, file)) < 0) {
-				perror(fname);
-				goto error_openfile;
-			}
-			if (n == sizeof(uint64_t)) {
-				rmo_base = ((uint32_t *)buf)[0];
-				rmo_top = rmo_base + ((uint32_t *)buf)[1];
-			} else if (n == 16) {
-				rmo_base = ((uint64_t *)buf)[0];
-				rmo_top = rmo_base + ((uint64_t *)buf)[1];
-			} else {
-				fprintf(stderr, "Mem node has invalid size: %d\n", n);
+			if (read_memory_region_limits(file, &rmo_base, &rmo_top) != 0) {
  				goto error_openfile;
  			}
+
  			if (rmo_top > 0x30000000UL)
  				rmo_top = 0x30000000UL;
  
@@ -664,7 +723,6 @@ error_opendir:
  	return -1;
  }
  
-
  /* Setup a sorted list of memory ranges. */
  static int setup_memory_ranges(unsigned long kexec_flags)
  {
@@ -756,7 +814,6 @@ out:
  	return -1;
  }
  
-
  /* Return a list of valid memory ranges */
  int get_memory_ranges_dt(struct memory_range **range, int *ranges,
  		unsigned long kexec_flags)
@@ -778,6 +835,11 @@ int get_memory_ranges_dt(struct memory_r
  int get_memory_ranges(struct memory_range **range, int *ranges,
  					unsigned long kexec_flags)
  {
+	int res = 0;
+
+	res = init_memory_region_info();
+	if (res != 0)
+		return res;
  #ifdef WITH_GAMECUBE
  	return get_memory_ranges_gc(range, ranges, kexec_flags);
  #else
Index: kexec-tools-2.0.4/kexec/arch/ppc/kexec-ppc.h
===================================================================
--- kexec-tools-2.0.4.orig/kexec/arch/ppc/kexec-ppc.h
+++ kexec-tools-2.0.4/kexec/arch/ppc/kexec-ppc.h
@@ -69,6 +69,12 @@ extern unsigned long long initrd_base, i
  extern unsigned long long ramdisk_base, ramdisk_size;
  extern unsigned char reuse_initrd;
  extern const char *ramdisk;
+
+/* Method to parse the memory/reg nodes in device-tree */
+extern unsigned long dt_address_cells, dt_size_cells;
+extern int init_memory_region_info(void);
+extern int read_memory_region_limits(FILE *fp, unsigned long long *start,
+					unsigned long long *end);
  #define COMMAND_LINE_SIZE	512 /* from kernel */
  /*fs2dt*/
  void reserve(unsigned long long where, unsigned long long length);
Index: kexec-tools-2.0.4/kexec/arch/ppc/crashdump-powerpc.c
===================================================================
--- kexec-tools-2.0.4.orig/kexec/arch/ppc/crashdump-powerpc.c
+++ kexec-tools-2.0.4/kexec/arch/ppc/crashdump-powerpc.c
@@ -130,9 +130,8 @@ static int get_crash_memory_ranges(struc
  				closedir(dir);
  				goto err;
  			}
-			n = fread(buf, 1, MAXBYTES, file);
-			if (n < 0) {
-				perror(fname);
+			n = read_memory_region_limits(file, &start, &end);
+			if (n != 0) {
  				fclose(file);
  				closedir(dmem);
  				closedir(dir);
@@ -146,24 +145,6 @@ static int get_crash_memory_ranges(struc
  				goto err;
  			}
  
-			/*
-			 * FIXME: This code fails on platforms that
-			 * have more than one memory range specified
-			 * in the device-tree's /memory/reg property.
-			 * or where the #address-cells and #size-cells
-			 * are not identical.
-			 *
-			 * We should interpret the /memory/reg property
-			 * based on the values of the #address-cells and
-			 * #size-cells properites.
-			 */
-			if (n == (sizeof(unsigned long) * 2)) {
-				start = ((unsigned long *)buf)[0];
-				end = start + ((unsigned long *)buf)[1];
-			} else {
-				start = ((unsigned long long *)buf)[0];
-				end = start + ((unsigned long long *)buf)[1];
-			}
  			if (start == 0 && end >= (BACKUP_SRC_END + 1))
  				start = BACKUP_SRC_END + 1;
  
Index: kexec-tools-2.0.4/kexec/arch/ppc/fs2dt.c
===================================================================
--- kexec-tools-2.0.4.orig/kexec/arch/ppc/fs2dt.c
+++ kexec-tools-2.0.4/kexec/arch/ppc/fs2dt.c
@@ -122,7 +122,7 @@ static unsigned propnum(const char *name
  	return offset;
  }
  
-static void add_usable_mem_property(int fd, int len)
+static void add_usable_mem_property(FILE* fp, int len)
  {
  	char fname[MAXPATH], *bname;
  	unsigned long buf[2];
@@ -137,21 +137,11 @@ static void add_usable_mem_property(int
  	if (strncmp(bname, "/memory@", 8) && strcmp(bname, "/memory"))
  		return;
  
-	if (len < 2 * sizeof(unsigned long))
-		die("unrecoverable error: not enough data for mem property\n");
-	len = 2 * sizeof(unsigned long);
-
-	if (lseek(fd, 0, SEEK_SET) < 0)
+	if (fseek(fp, 0, SEEK_SET) < 0)
  		die("unrecoverable error: error seeking in \"%s\": %s\n",
  		    pathname, strerror(errno));
-	if (read(fd, buf, len) != len)
-		die("unrecoverable error: error reading \"%s\": %s\n",
-		    pathname, strerror(errno));
-
-	if (~0ULL - buf[0] < buf[1])
-		die("unrecoverable error: mem property overflow\n");
-	base = buf[0];
-	end = base + buf[1];
+	if (read_memory_region_limits(fp, &base, &end) != 0)
+		die("unrecoverable error: error parsing memory/reg limits\n");
  
  	for (range = 0; range < usablemem_rgns.size; range++) {
  		loc_base = usablemem_rgns.ranges[range].start;
@@ -194,8 +184,9 @@ static void add_usable_mem_property(int
  static void putprops(char *fn, struct dirent **nlist, int numlist)
  {
  	struct dirent *dp;
-	int i = 0, fd, len;
+	int i = 0, len;
  	struct stat statbuf;
+	FILE *fp;
  
  	for (i = 0; i < numlist; i++) {
  		dp = nlist[i];
@@ -243,12 +234,12 @@ static void putprops(char *fn, struct di
  		*dt++ = len;
  		*dt++ = propnum(fn);
  
-		fd = open(pathname, O_RDONLY);
-		if (fd == -1)
+		fp = fopen(pathname, "r");
+		if (fp == NULL)
  			die("unrecoverable error: could not open \"%s\": %s\n",
  			    pathname, strerror(errno));
  
-		if (read(fd, dt, len) != len)
+		if (fread(dt, 1, len, fp) != len)
  			die("unrecoverable error: could not read \"%s\": %s\n",
  			    pathname, strerror(errno));
  
@@ -290,8 +281,8 @@ static void putprops(char *fn, struct di
  
  		dt += (len + 3)/4;
  		if (!strcmp(dp->d_name, "reg") && usablemem_rgns.size)
-			add_usable_mem_property(fd, len);
-		close(fd);
+			add_usable_mem_property(fp, len);
+		fclose(fp);
  	}
  
  	fn[0] = '\0';


From suzuki at in.ibm.com  Mon May 30 03:22:24 2011
From: suzuki at in.ibm.com (Suzuki Poulose)
Date: Mon, 30 May 2011 12:52:24 +0530
Subject: [RFC][PATCH] Kexec support for PPC440x
Message-ID: <4DE345B0.8020505@in.ibm.com>

KEXEC Support for ppc440X

The patch adds kexec support for PPC440x based boards. This work is based
on the KEXEC patches for FSL BookE.

The FSL BookE patch and the code flow could be found at the link below:

http://patchwork.ozlabs.org/patch/49359/

We invalidate all the TLB entries except the one this code is run from,
and create a 1:1 mapping for 0-2GiB in blocks of 256M. Later we jump to
the 1:1 mapping.

I have tested this patches on Ebony and Sequoia boards. It would be great
if somebody could test this on the other boards.

Signed-off-by: Suzuki K. Poulose <suzuki at in.ibm.com>
---
  arch/powerpc/Kconfig                    |    2
  arch/powerpc/include/asm/kexec.h        |    2
  arch/powerpc/kernel/44x_kexec_mapping.S |   66 ++++++++++++++++++++++++++++++++
  arch/powerpc/kernel/misc_32.S           |   22 ++++++++++
  4 files changed, 90 insertions(+), 2 deletions(-)

Index: powerpc/arch/powerpc/Kconfig
===================================================================
--- powerpc.orig/arch/powerpc/Kconfig
+++ powerpc/arch/powerpc/Kconfig
@@ -349,7 +349,7 @@ config ARCH_ENABLE_MEMORY_HOTREMOVE
  
  config KEXEC
  	bool "kexec system call (EXPERIMENTAL)"
-	depends on (PPC_BOOK3S || FSL_BOOKE) && EXPERIMENTAL
+	depends on (PPC_BOOK3S || FSL_BOOKE || (44x && !SMP && !47x)) && EXPERIMENTAL
  	help
  	  kexec is a system call that implements the ability to shutdown your
  	  current kernel, and to start another kernel.  It is like a reboot
Index: powerpc/arch/powerpc/include/asm/kexec.h
===================================================================
--- powerpc.orig/arch/powerpc/include/asm/kexec.h
+++ powerpc/arch/powerpc/include/asm/kexec.h
@@ -2,7 +2,7 @@
  #define _ASM_POWERPC_KEXEC_H
  #ifdef __KERNEL__
  
-#ifdef CONFIG_FSL_BOOKE
+#if defined(CONFIG_FSL_BOOKE) || defined(CONFIG_44x)
  
  /*
   * On FSL-BookE we setup a 1:1 mapping which covers the first 2GiB of memory
Index: powerpc/arch/powerpc/kernel/44x_kexec_mapping.S
===================================================================
--- /dev/null
+++ powerpc/arch/powerpc/kernel/44x_kexec_mapping.S
@@ -0,0 +1,66 @@
+/*
+ * Copyright (C) 2011 IBM
+ *
+ * Author : Suzuki K. Poulose <suzuki at in.ibm.com>
+ *
+ * This source code is licensed under the GNU General Public License,
+ * Version 2.  See the file COPYING for more details.
+ *
+ * Code for setting up 1:1 mapping for PPC440x for KEXEC
+ *
+ * We cannot switch off the MMU on PPC44x.
+ * So we:
+ * 1) Invalidate all the mappings except the one we are
+ * running from.
+ * 2) Create a 1:1 mapping for 0-2GiB
+ *
+ * - Based on the kexec support code for FSL BookE
+ * - Based on the boot up code for ppc44x from head_44x.S
+ * - Doesn't support 47x yet.
+ * - Doesn't support SMP.
+ *
+ */
+	bl	nxtins				/* Find our address */
+nxtins:	mflr	r5				/* Make it accessible */
+	tlbsx	r23,0,r5			/* Find entry we are in */
+	li	r4,0				/* Start at TLB entry 0 */
+	li	r3,0				/* Set PAGEID inval value */
+1:	cmpw	r23,r4				/* Is this our entry? */
+	beq	skip				/* If so, skip the inval */
+	tlbwe	r3,r4,PPC44x_TLB_PAGEID		/* If not, inval the entry */
+skip:	addi	r4,r4,1				/* Increment */
+	cmpwi	r4,64				/* Are we done? */
+	bne	1b				/* If not, repeat */
+	isync
+
+/*
+ * Setup 1:1 mapping  for 0-2GiB in blocks of 256M.
+ * r23 is the index to the mapping of this code.
+ * Skip it.
+ */
+	/* attribute fields. rwx for SUPERVISOR mode */
+	li	r5, 0
+	ori	r5, r5, (PPC44x_TLB_SW | PPC44x_TLB_SR | PPC44x_TLB_SX | PPC44x_TLB_G)
+
+	li	r11, 0				/* PageNumber */
+	li	r6, 0				/* TLB Index  */
+
+next_tlb:
+	rotlwi	r3, r11, 28			/* Create EPN (bits 0-3) */
+	mr	r4, r3				/* RPN = EPN  */
+	ori	r3, r3, PPC44x_TLB_VALID | PPC44x_TLB_256M /* SIZE = 256M, Valid */
+	cmpw	r6, r23				/* Is this our TLB entry ? */
+	bne	write_tlb
+	addi	r6, r6, 1			/* Skip our entry */
+
+write_tlb:
+	tlbwe	r3, r6, PPC44x_TLB_PAGEID	/* PageID field : EPN, V, SIZE */
+	tlbwe	r4, r6, PPC44x_TLB_XLAT		/* Address translation : RPN   */
+	tlbwe	r5, r6, PPC44x_TLB_ATTRIB	/* Attributes */
+
+	addi	r11, r11, 1			/* Increment PN */
+	addi	r6, r6, 1			/* Increment TLB Index */
+	cmpwi	r11, 8				/* Are we done ? */
+	bne	next_tlb
+	isync
+
Index: powerpc/arch/powerpc/kernel/misc_32.S
===================================================================
--- powerpc.orig/arch/powerpc/kernel/misc_32.S
+++ powerpc/arch/powerpc/kernel/misc_32.S
@@ -736,6 +736,28 @@ relocate_new_kernel:
  	mr      r5, r31
  
  	li	r0, 0
+#elif defined(CONFIG_44x)  && !defined(CONFIG_47x)
+
+	mr	r29, r3
+	mr	r30, r4
+	mr	r31, r5
+
+#include "44x_kexec_mapping.S"
+
+	mr	r3, r29
+	mr	r4, r30
+	mr	r5, r31
+
+	li	r0, 0
+
+	/* Jump back to the 1:1 address */
+	mr	r8, r0
+	mtspr	SPRN_SRR1, r8
+	addi	r8, r4, 1f - relocate_new_kernel
+	mtspr	SPRN_SRR0, r8
+	sync
+	rfi
+1:
  #else
  	li	r0, 0
  


From bigeasy at linutronix.de  Tue May 31 11:15:35 2011
From: bigeasy at linutronix.de (Sebastian Andrzej Siewior)
Date: Tue, 31 May 2011 17:15:35 +0200
Subject: [RFC][PATCH] Kexec support for PPC440x
In-Reply-To: <4DE345B0.8020505@in.ibm.com>
References: <4DE345B0.8020505@in.ibm.com>
Message-ID: <4DE50617.7090509@linutronix.de>


Suzuki Poulose wrote:
> Index: powerpc/arch/powerpc/kernel/44x_kexec_mapping.S
> ===================================================================
> --- /dev/null
> +++ powerpc/arch/powerpc/kernel/44x_kexec_mapping.S
....

> + *
> + */
> +    bl    nxtins                /* Find our address */
> +nxtins:    mflr    r5                /* Make it accessible */

Please don't mix labels and instructions.

> +    tlbsx    r23,0,r5            /* Find entry we are in */

using tabs instead of spaces would make it look nice. Please also separate 
the arguments of the instruction i.e.
	tlbsx	r23, 0, r5


> Index: powerpc/arch/powerpc/kernel/misc_32.S
> ===================================================================
> --- powerpc.orig/arch/powerpc/kernel/misc_32.S
> +++ powerpc/arch/powerpc/kernel/misc_32.S
> @@ -736,6 +736,28 @@ relocate_new_kernel:
>      mr      r5, r31
>  
>      li    r0, 0
> +#elif defined(CONFIG_44x)  && !defined(CONFIG_47x)
> +
> +    mr    r29, r3
> +    mr    r30, r4
> +    mr    r31, r5
> +
> +#include "44x_kexec_mapping.S"

The way you setup the 1:1 mapping should be close to what you are doing on
kernel entry. Isn't it possible to include the file here and in the entry
code?

Sebastian


From prasad at linux.vnet.ibm.com  Tue May 31 13:40:43 2011
From: prasad at linux.vnet.ibm.com (K.Prasad)
Date: Tue, 31 May 2011 23:10:43 +0530
Subject: [RFC Patch 4/6] PANIC_MCE: Introduce a new panic flag for fatal
	MCE, capture related information
In-Reply-To: <m1fwo09g15.fsf@fess.ebiederm.org>
References: <20110526170722.GB23266@in.ibm.com>
	<20110526171521.GD17988@in.ibm.com>
	<m1fwo09g15.fsf@fess.ebiederm.org>
Message-ID: <20110531174043.GA2000@in.ibm.com>

On Fri, May 27, 2011 at 11:04:06AM -0700, Eric W. Biederman wrote:
> "K.Prasad" <prasad at linux.vnet.ibm.com> writes:
> 
> > PANIC_MCE: Introduce a new panic flag for fatal MCE, capture related information
> >
> > Fatal machine check exceptions (caused due to hardware memory errors) will now
> > result in a 'slim' coredump that captures vital information about the MCE. This
> > patch introduces a new panic flag, and new parameters to *panic functions
> > that can capture more information pertaining to the cause of crash.
> >
> > Enable a new elf-notes section to store additional information about the crash.
> > For MCE, enable a new notes section that captures relevant register status
> > (struct mce) to be later read during coredump analysis.
> 
> There may be a reason to pass everything struct mce through 5 layers of
> code but right now it looks like it just makes everything uglier to no
> real purpose.

We could have stopped with just a blank elf-note of type NT_MCE
indicating an MCE triggered panic, but dumping 'struct mce' in it will
help gather more useful information about the error - especially the
memory address that experienced unrecoverable error (stored in
mce->addr).

The patch 6/6 for the 'crash' tool enabled decoding of 'struct
mce' to show this information (although the sample log in patch 0/6)
didn't show these benefits because 'mce-inject' tool used to soft-inject
these errors doesn't populate all registers with valid contents.

The idea was that when mce->addr contains physical address is shown
while decoding coredump, the corresponding memory DIMM could be identified
for replacement/isolation.

Given that 'struct mce' isn't placed in a user-space visible file its
duplicate copies have to be maintained in 'crash' (like it is done in
'mcelog' tool), and that's one disadvantage.

If you think that this complicates the patch, I'll start with a much
'slimmer' version (!) of the slimdump and the improvements may be
contemplated iteratively.

Thanks,
K.Prasad


From root at ns2.teleproffen.no  Tue May 31 23:16:07 2011
From: root at ns2.teleproffen.no (Noel class)
Date: Wed, 1 Jun 2011 05:16:07 +0200
Subject: [ bseseuum9jzw9ujl] : marketing list for businesses in the UK
Message-ID: <ec9ebabc-72eb-406c-b026-232da4a9cb49@teleproffen.teleproffen.no>

Healthcare List Package: $295 for all 21 complete lists below
 - Also available, Business Lists, Foregin Business Lists and Consumer data


Physicians (34 specialties)
Database	:	 788k records, 17k emails, 200k fax numbers
 
Chiropractors
Database	:	 108,421 total records * 3,414 emails * 6,553 fax numbers
 
Alternative Medicine
Database	:	 1,141,602 total records with 36,320 emails and 38.935 fax numbers
 
Dentists
Database	:	 164k records, 45k emails, 77k fax numbers
 
Dentists with Specialties
Database	:	 30k records all with emails
 
	
Veterinarians
Database	:	 78,986 total records with 1,438  emails and 1,050  fax numbers
 
Hospitals
Database	:	 10,661 Emails for hospitals in the USA
 
Nursing Homes
Database	:	 31,589 Senior Administrators, 11,288 Nursing Directors in over 14,706 Nursing Homes (full contact info no emails)
 
Pharmaceutical Companies
Database	:	 47,000 emails of pharma company employees
 
Physical Therapists
Database	:	 125,460 total records with 5,483 emails and 4,405 fax numbers
 
	
Oncology Doctors
Database	:	 2,200 records all with emails
 
US Surgery Centers
Database	:	 85k records and 14k emails
 
Acupuncturists
Database	:	 23,988 records 1,826 emails
 
Massage Therapists
Database	:	 76,701 records and 8,305 emails
 
Medical Equipment Suppliers
Database	:	 67,425 total records with 6,940 emails and 5,812 fax numbers
 
	
Mental Health Counselors
Database	:	 283,184 records 7,206 emails
 
Visiting Nurses & RN's
Database	:	 91,386 total records with 2,788 emails and 2,390 fax numbers
 
Optometrists
Database	:	 63,837 records 2,015 emails
 
Psychologists
Database	:	 272,188 records and 9,874 emails
Healthcare Recruiters
Database	:	 11,370 emails for active healthcare recruiters in the USA
Plastic Surgeons
Database	:	 23,109 emails with full data

Send me an email here mark.robinson1 at gmx.us for counts and sample spreadsheets. We also sell lists individually at reduced prices.




to terminate please send a blank message to takeitoff at gmx.com
  






