[PATCH] makedumpfile: change the wrong code to calculate bufsize_cyclic for elf dump
vgoyal at redhat.com
Fri May 9 13:49:24 PDT 2014
On Fri, May 09, 2014 at 05:36:13AM +0000, Atsushi Kumagai wrote:
> I tried to reproduce OOM in my environment.
> Unfortunately, I couldn't get a chance to use a large memory machine,
> so I controlled the bitmap buffer size with --cyclic-buffer like below:
> / # free
> total used free shared buffers
> Mem: 37544 19796 17748 0 56
> Swap: 0 0 0
> Total: 37544 19796 17748
> / # /mnt/usr/sbin/makedumpfile_static -E --cyclic-buffer=8000 /proc/vmcore /mnt/tmp/dumpfile.E
> Copying data : [100.0 %] |
> The dumpfile is saved to /mnt/tmp/dumpfile.E.
> makedumpfile Completed.
> VmHWM: 16456 kB
> / #
> As above, OOM didn't happen even when makedumpfile consumed most of the
> available memory (the remains were only 1MB).
> Of course, OOM happened when the memory usage exceeded the limit:
I think this is the key point. How did makedumpfile exceed the limit. So
if you don't specify --cyclic-buffer=X, then makedumpfile will allocate
80% of available memory. That would be roughly 16MB of cyclic buffer
(instead of 8MB).
In your test case looks like makeudmpfile used 16MB of memory. Out of
which 8MB must have been used for bitmaps and rest 8MB for other purposes.
So clearly our calculation of using 80% of available memory for bitmaps
is not right. Rest of the 20% memory might not be enough for fulfilling
the needs of makeudmpfile.
We don't have a good way to determine how much *non-bitmap* memory
allocation makedumpfile requires. I am wondering how about making
some assumtions and modify the formula for bitmap memory.
- Assume makedumpfile requires 16MB of free memory for various purposes.
- Subtract 16MB from available memory and take 80% of remaining memory
to calculate the bitmap memory size.
- Keep 4MB as minimum for bitmap buffer size.
One more person has reported makedumpfile OOM (even without -E option),
on a machine. There also 30MB of memory is free. I think it has the
same issue that 24MB must have been allocated in bitmaps and remainig
6MB is not sufficient to satisfy makedumpfile's needs.
What do you think?
Also, does anybody know what's the impact of bitmap buffer size on dump
speed. Does dump speed change significantly with bitmap buffer size?
More information about the kexec