[RFC] autokdump - automated kdump testsuite

[Vivek Goyal]

On Mon, Sep 22, 2014 at 09:00:00AM -0400, CAI Qian wrote:
> 
> 
> ----- Original Message -----
> > From: "Vivek Goyal" <vgoyal at redhat.com>
> > To: "CAI Qian" <caiqian at redhat.com>
> > Cc: "linux-kernel" <linux-kernel at vger.kernel.org>, "ltp-list" <ltp-list at lists.sourceforge.net>, "crash-utility"
> > <crash-utility at redhat.com>, "kexec" <kexec at lists.infradead.org>, "kexec kdump redhat mailing list"
> > <kexec-kdump-list at redhat.com>
> > Sent: Friday, September 19, 2014 9:22:36 PM
> > Subject: Re: [RFC] autokdump - automated kdump testsuite
> > 
> > On Fri, Sep 19, 2014 at 05:52:25AM -0400, CAI Qian wrote:
> > > I plan to release an automated kdump testsuite that will be
> > 
> > So will this be a standalone test suit? Can it be merged with
> > something already existing say, LTP.
> Yes, it is likely to be standalone. It won't make use of the LTP
> API, and the LTP kdump test suite is outdated, so there is no
> benefit to continue working over there.

So why make it standalone and not replace the old LTP kdump test suite
with this new one?

> > 
> > > focus on testing kernel and the crash utility. It should work
> > > for all major distros since it will use none of distro-specific
> > > stuff, and also support different arches including x86, ARM,
> > > PPC64 and s390x.
> > > 
> > > It does the following:
> > > 1) check if there is a memory reserved for kdump. If not,
> > >    reserve the memory and reboot the system.
> > > 2) once the system is back, load kexec on panic and
> > >    prepare a separate initramfs that including needed
> > >    modules to load a local filesystem and necessary utilities
> > 
> > So you will write logic to prepare custom initramfs or will rely
> > on dracut or some other utility for that.
> I'll probably prepare custom initramfs for the sake of simplicity.

Well, preparing custom initramfs will become very tricky. We used
to do that and finally we switched to dracut.

Why not simply let the respective service on the host do this job and
test only makes sure that kdump service is running. It feels little
out of place that a test is generating custom initramfs. 

> > 
> > >    in order to analyse /proc/vmcore in the 2nd kernel.
> > > 3) trigger the system crash using methods like sysrq-c, NMI,
> > >    and panic_on_hung_task etc.
> > > 4) in the 2nd kernel, mount a filesystem and use the crash
> > >    utility to analyse /proc/vmcore. Then, gather the analyse
> > >    logs, serial console output, dmesg etc into the filesystem.
> > 
> > Why not save core and boot back in first kernel and then analyze.
> > 
> > Trying to work directly with /proc/vmcore does not test makedumfile
> > which everybody uses. Also it will require more memory to be reserved
> > and packing crash and debug vmlinux into initramfs.
> The additional memory for vmlinux and the crash utility is predictable
> and manageable, so it can just ask 256M memory reserved before running
> the program. On the other hand, it is not usually feasible to ask
> the systems under testing has enough available disk spaces bigger than
> the memory size.

makedumpfile will reduce the vmcore file size to few hundreds of mega
bytes on most of the systems. Especially, this is just a test, so 
system will be lightly loaded and vmcore will be small after filtering.

If there is not enough space, test fails, period. I don't think there
is any need to try to circumvent that and try to run crash in initramfs.
And in the process we don't test makedumpfile which is very imporatnt
component of this whole process.

IMHO, just rely on "systemctl start kdump" to generate and load custom
initramfs and save filtered vmcore to root fs by default and alanyze
vmcore post reboot. That will keep things simple.

Thanks
Vivek