Capturing crash with 4.6.0 and above kernel does not work

[Himanshu Madhani]

On 8/26/16, 5:15 PM, "Baoquan He" <bhe at redhat.com> wrote:

>On 08/26/16 at 04:47pm, Himanshu Madhani wrote:
>> 
>> 
>> On 8/25/16, 10:39 PM, "Baoquan He" <bhe at redhat.com> wrote:
>> 
>> >On 08/25/16 at 05:45pm, Himanshu Madhani wrote:
>> >> 
>> >
>> >crashkernel has been reserved successfully.
>> >Aug 25 10:36:44 dut4062 kernel: Reserving 512MB of memory at 368MB for crashkernel (System RAM: 130847MB)
>> >
>> >So what's the result of executing "grep Crash /porc/iomem" on your 
>> >system?
>> 
>> root at dut4062:~# grep Crash /proc/iomem
>>   17000000-36ffffff : Crash kernel
>> 
>> Which does indicate that 512MB is set aside for crash dump
>
>Please get a kexec-tools code and debug it. The message is printed in
>main() of kexec.
>
>git://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git
>
>It should be easy to add debug printing or gdb. Now the latest is
>2.0.13. What's your version?

Sorry got distracted today so could not get to my server at all. 

I’ll try to get to this one in next couple days. 

Here’s what I have on my server for kexec version.

root at dut4062:/home/hmadhani/wspace/kexec-tools(master)# kexec --version
kexec-tools 2.0.0 released 19th July 2008


>
>> 
>> 
>> >
>> >> 
>> >> On 8/25/16, 1:10 AM, "Michal Hocko" <mhocko at kernel.org> wrote:
>> >> 
>> >> >[Let's add kdump people]
>> >> >
>> >> >On Wed 24-08-16 16:38:56, Himanshu Madhani wrote:
>> >> >> Hello list,
>> >> >> 
>> >> >> I am wondering if anybody has issue capturing crash dump with the 4.6.0 and above kenrel.
>> >> >> 
>> >> >> I have a system, when booted in 4.5.7 kernel is able to capture crash dump.
>> >> >> However, when I boot this system in 4.6.4 and 4.7.2 kernel, crash dump is not 
>> >> >> able to capture any crash. 
>> >> >> 
>> >> >> I am still facing same issue with 4.8.0-rc2+ kernel and from the error at the command prompt, 
>> >> >> it seems like kexec is ignoring “crashkenrel” parameter. 
>> >> >> 
>> >> >> I added below information in https://bugzilla.kernel.org/show_bug.cgi?id=119291. 
>> >> >> 
>> >> >> # uname -r
>> >> >> 4.8.0-rc2+
>> >> >> 
>> >> >> # cat /proc/cmdline 
>> >> >> ro root=/dev/mapper/vg_dut4110-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=512M rd_LVM_LV=vg_dut4110/lv_swap rd_LVM_LV=vg_dut4110/lv_root rd_NO_DM rhgb quiet
>> >> >> 
>> >> >> # service kdump status
>> >> >> Kdump is not operational
>> >> >> 
>> >> >> # service kdump start
>> >> >> Memory for crashkernel is not reserved
>> >> >> Please reserve memory by passing "crashkernel=X at Y" parameter to the kernel
>> >> >> Starting kdump:                                            [FAILED]
>> >> >
>> >> >It smells like the crash kernel reservation has failed. Could you
>> >> >provide the full kernel log?
>> >> 
>> >> Attached is kernel log from fresh kernel compile of 4.6.5 (linux-stable) tree
>> >> 
>> >> root at dut4062:~# cat /proc/cmdline 
>> >> ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 crashkernel=512M rd_LVM_LV=VolGroup/lv_root  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb console=ttyS0 console=ttyS0,115200n8
>> >> 
>> >> root at dut4062:~# service kdump status
>> >> 
>> >> Kdump is not operational
>> >> 
>> >> root at dut4062:~# service kdump start
>> >> Memory for crashkernel is not reserved
>> >> Please reserve memory by passing "crashkernel=X at Y" parameter to the kernel
>> >> Starting kdump:                                            [FAILED]
>> >> root at dut4062:~# 
>> >> 
>> >> Messages in log fil
>> >> 
>> >> Aug 25 10:42:16 dut4062 kdump: kexec: failed to load kdump kernel
>> >> Aug 25 10:42:16 dut4062 kdump: failed to start up
>> >> 
>> >> 
>> >> root at dut4062:~# kexec --ver
>> >> kexec-tools 2.0.0 released 19th July 2008
>> >> root at dut4062:~# 
>> >> 
>> >> root at dut4062:~# crash --version
>> >> 
>> >> crash 7.1.5++
>> >> Copyright (C) 2002-2016  Red Hat, Inc.
>> >> Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
>> >> Copyright (C) 1999-2006  Hewlett-Packard Co
>> >> Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
>> >> Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
>> >> Copyright (C) 2005, 2011  NEC Corporation
>> >> Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
>> >> Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
>> >> This program is free software, covered by the GNU General Public License,
>> >> and you are welcome to change it and/or distribute copies of it under
>> >> certain conditions.  Enter "help copying" to see the conditions.
>> >> This program has absolutely no warranty.  Enter "help warranty" for details.
>> >>  
>> >> GNU gdb (GDB) 7.6
>> >> Copyright (C) 2013 Free Software Foundation, Inc.
>> >> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>> >> This is free software: you are free to change and redistribute it.
>> >> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> >> and "show warranty" for details.
>> >> This GDB was configured as "x86_64-unknown-linux-gnu".
>> >> 
>> >> Let me know if any other details are needed. I have multiple servers which exhibit same issues.
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> >
>> >> >> message file confirms that kexec was not able to start the service 
>> >> >> 
>> >> >> Aug 15 10:41:17 dut4110 kdump: kexec: failed to load kdump kernel
>> >> >> Aug 15 10:41:17 dut4110 kdump: failed to start up
>> >> >> 
>> >> >> Note, that same option is able to load kdump service for 4.5.7 kernel.  
>> >> >> 
>> >> >> I can provide any details needed to help resolve this issue. 
>> >> >> 
>> >> >> Thanks,
>> >> >> - Himanshu
>> >> >> 
>> >> >
>> >> >-- 
>> >> >Michal Hocko
>> >> >SUSE Labs
>> >
>> >