896MB address limit
cpw at sgi.com
Thu Sep 27 19:07:12 EDT 2012
On Tue, Sep 25, 2012 at 08:10:04AM -0700, Eric W. Biederman wrote:
> Cliff Wickman <cpw at sgi.com> writes:
> > Hi Eric, and all,
> > On Mon, Sep 24, 2012 at 08:11:12PM -0700, Eric W. Biederman wrote:
> >> Cliff Wickman <cpw at sgi.com> writes:
> >> > Gentlemen,
> >> >
> >> > In dumping very large memories we are running up against the 896MB
> >> > limit in SLES11SP2 (3.0.38 kernel).
> >> Odd. That limit should be the maximum address in memory to load the
> >> crash kernel. Tha limit should have nothing to do with the dump process
> >> itself.
> >> Are you saying you need more that 512MiB reserved for the crash kernel
> >> to be able to dump all of the memory in your system?
> >> Eric
> > As I noted to Eric privately, yes we need to bump up to crashkernel=1G
> > or more for some very large memories.
> > As an experiment I bumped
> > +++ linux/arch/x86/kernel/setup.c
> > @@ -528,7 +528,7 @@ static inline unsigned long long get_tot
> > #ifdef CONFIG_X86_32
> > # define CRASH_KERNEL_ADDR_MAX (512 << 20)
> > #else
> > -# define CRASH_KERNEL_ADDR_MAX (896 << 20)
> > +# define CRASH_KERNEL_ADDR_MAX (1700 << 20)
> > And that seems to work. i.e. I'm currently dumping a system where
> > crashkernel=1G and it seems to be working.
> > Am I just living dangerously?
> So fundamentally this should work. However there have been a lot of
> kinks and silly limitations in the x86 boot protocol.
> So it used to be that the bootloader protocol variable ramdisk_max was
> set to 896M for 32bit kernels. Because the ramdisk could not be located
> in high memory.
> Looking today it appears that ramdisk_max has been upped to 4G.
> I will let you look through the /sbin/kexec source code.
> As for testing I would up the limit to 4G on x86_64 and see how far
> you get.
> The practical question does the system still work with crashkernel=32M
> when you have raised the limit much higher.
> So I would test with crashkernel=1G at 2G and see if that works. If that
> works I figure that in practice all of the bugs are historical and we
> can forget them. But a sweep through the /sbin/kexec code for the magic
> number 896 might not be out of order.
I did try setting the limit to 8G. The crashkernel did get loaded there
but it would not execute there.
It works fine on a UV to set the limit to 4G and use a
crashkernel=1280M. We have a hole of almost 2G there.
The memory at 2G is already in use so I can't explicitly place it there.
The kernel patch looks like this:
@@ -522,13 +522,12 @@ static inline unsigned long long get_tot
* Keep the crash kernel below this limit. On 32 bits earlier kernels
* would limit the kernel to the low 512 MiB due to mapping
- * On 64 bits, kexec-tools currently limits us to 896 MiB; increase
- * limit once kexec-tools are fixed.
+ * On 64 bits, the boot protocol limits us to 4G.
# define CRASH_KERNEL_ADDR_MAX (512 << 20)
-# define CRASH_KERNEL_ADDR_MAX (896 << 20)
+# define CRASH_KERNEL_ADDR_MAX (1UL << 32)
static void __init reserve_crashkernel(void)
cpw at sgi.com
More information about the kexec