makedumpfile issues many readpage_elf: Attempt to read non-existent page

Louis Bouchard louis.bouchard at canonical.com
Mon Oct 24 07:54:02 PDT 2016


Hello,

Le 29/09/2016 à 15:34, Louis Bouchard a écrit :
> Hello,
>
> Le 23/09/2016 14:04, Atsushi Kumagai a écrit :
>>> Does someone have an idea of where this could come from ?
>>
>> Thanks for your report, but I'm afraid that I'm going on
>> vacation till Oct 2.
>> I hope someone helps you until I'm back.
>>
>>
>> Regards,
>> Atsushi Kumagai
>>
>
> As a follow up to my question, I've been investigating the issue. The source of
> the problem seems to come from readpage_elf() which tries to read to an
> inexistant page :
>
> readpage_elf: Attempt to read non-existent page at 0x1a7b7fff5000.
>
> The error happens when readpage_elf() is called from section_mem_map_addr with
> pgaddr, which is derived from paddr. This one comes from :
>
>    paddr = vaddr_to_paddr(addr)
>
> When tracing vaddr_to_paddr, I see a difference in the calculation. on a 4.4
> kernel :
>
>   info->phys_base = 0
>
> On 4.8 :
>   info->phys_base = 0xffffffffe3800000
>
> So the paddr values used to define pgaddr are :
>
> for 4.4 kernels : 0x3fff4000
> for 4.8 kernels : 0x1a7b7fff5000
>
> Running makedumpfile --dump-dmesg in debug mode leads to :
>
> 4.4 kernel :
>
> sadump: does not have partition header
> sadump: read dump device as unknown format
> sadump: unknown format
> LOAD (0)
>   phys_start : 1000000
>   phys_end   : 2224000
>   virt_start : ffffffff81000000
>   virt_end   : ffffffff82224000
> LOAD (1)
>   phys_start : 1000
>   phys_end   : 9fc00
>   virt_start : ffff880000001000
>   virt_end   : ffff88000009fc00
> LOAD (2)
>   phys_start : 100000
>   phys_end   : 2b000000
>   virt_start : ffff880000100000
>   virt_end   : ffff88002b000000
> LOAD (3)
>   phys_start : 33000000
>   phys_end   : 3fffe000
>   virt_start : ffff880033000000
>   virt_end   : ffff88003fffe000
> Linux kdump
> page_size    : 4096
>
> max_mapnr    : 3fffe
>
> Buffer size for the cyclic mode: 65535
>
> num of NODEs : 1
>
>
> Memory type  : SPARSEMEM_EX
>
> mem_map (0)
>   mem_map    : ffffea0000000000
>   pfn_start  : 0
>   pfn_end    : 8000
> mem_map (1)
>   mem_map    : ffffea0000200000
>   pfn_start  : 8000
>   pfn_end    : 10000
> mem_map (2)
>   mem_map    : ffffea0000400000
>   pfn_start  : 10000
>   pfn_end    : 18000
> mem_map (3)
>   mem_map    : ffffea0000600000
>   pfn_start  : 18000
>   pfn_end    : 20000
> mem_map (4)
>   mem_map    : ffffea0000800000
>   pfn_start  : 20000
>   pfn_end    : 28000
> mem_map (5)
>   mem_map    : ffffea0000a00000
>   pfn_start  : 28000
>   pfn_end    : 30000
> mem_map (6)
>   mem_map    : ffffea0000c00000
>   pfn_start  : 30000
>   pfn_end    : 38000
> mem_map (7)
>   mem_map    : ffffea0000e00000
>   pfn_start  : 38000
>   pfn_end    : 3fffe
> mmap() is available on the kernel.
>
> log_buf       : ffffffff8210521c
> log_end       : 0
> log_buf_len   : 262144
> log_first_idx : 0
> log_next_idx  : 141176
>
> The dmesg log is saved to /tmp/titi.
>
> makedumpfile Completed.
>
>
> For 4.8 kernels :
> sadump: does not have partition header
> sadump: read dump device as unknown format
> sadump: unknown format
> LOAD (0)
>   phys_start : 7200000
>   phys_end   : 847d000
>   virt_start : ffffffffa3a00000
>   virt_end   : ffffffffa4c7d000
> LOAD (1)
>   phys_start : 1000
>   phys_end   : 9fc00
>   virt_start : ffff880000001000
>   virt_end   : ffff88000009fc00
> LOAD (2)
>   phys_start : 100000
>   phys_end   : 2b000000
>   virt_start : ffff880000100000
>   virt_end   : ffff88002b000000
> LOAD (3)
>   phys_start : 33000000
>   phys_end   : 3fffe000
>   virt_start : ffff880033000000
>   virt_end   : ffff88003fffe000
> Linux kdump
> page_size    : 4096
>
> max_mapnr    : 3fffe
>
> Buffer size for the cyclic mode: 65535
> The kernel version is not supported.
> The makedumpfile operation may be incomplete.
>
> num of NODEs : 1
>
>
> Memory type  : SPARSEMEM_EX
>
> mem_map (0)
>   mem_map    : 0
>   pfn_start  : 0
>   pfn_end    : 8000
> mem_map (1)
>   mem_map    : 0
>   pfn_start  : 8000
>   pfn_end    : 10000
> mem_map (2)
>   mem_map    : 0
>   pfn_start  : 10000
>   pfn_end    : 18000
> mem_map (3)
>   mem_map    : 0
>   pfn_start  : 18000
>   pfn_end    : 20000
> mem_map (4)
>   mem_map    : 0
>   pfn_start  : 20000
>   pfn_end    : 28000
> mem_map (5)
>   mem_map    : 0
>   pfn_start  : 28000
>   pfn_end    : 30000
> mem_map (6)
>   mem_map    : 0
>   pfn_start  : 30000
>   pfn_end    : 38000
> mem_map (7)
>   mem_map    : 0
>   pfn_start  : 38000
>   pfn_end    : 3fffe
> mmap() is available on the kernel.
>
> log_buf       : ffffffffa4b3c19c
> log_end       : 0
> log_buf_len   : 262144
> log_first_idx : 0
> log_next_idx  : 43160
>
> The dmesg log is saved to /tmp/toto.
>
> makedumpfile Completed.
>
>
> I don't know if it is to be expected with kernels 4.8 to have all mem_map
> addresses set to 0, unlike with k4.4 :
>
> mem_map (0)
> mem_map : 0
> pfn_start : 0
> pfn_end : 8000
>
> Any comment appreciated while I continue to investigate.
>
> Kind regards,
>
> ...Louis
>

I have now completed the kernel bisection between 4.7.8 and 4.8-rc1 and
identified the kernel modification that triggers the errors cited above :

> commit 021182e52fe01c1f7b126f97fd6ba048dc4234fd
> Author: Thomas Garnier <thgarnie at google.com>
> Date:   Tue Jun 21 17:47:03 2016 -0700
>
>     x86/mm: Enable KASLR for physical mapping memory regions
>
>     Add the physical mapping in the list of randomized memory regions.
>
>     The physical memory mapping holds most allocations from boot and heap
>     allocators. Knowing the base address and physical memory size, an attacker
>     can deduce the PDE virtual address for the vDSO memory page. This attack
>     was demonstrated at CanSecWest 2016, in the following presentation:
>
>       "Getting Physical: Extreme Abuse of Intel Based Paged Systems":
>       https://github.com/n3k/CansecWest2016_Getting_Physical_Extreme_Abuse_of_Intel_Based_Paging_Systems/blob/master/Presentation/CanSec2016_Presentation.pdf
>
>     (See second part of the presentation).
>
>     The exploits used against Linux worked successfully against 4.6+ but
>     fail with KASLR memory enabled:
>
>       https://github.com/n3k/CansecWest2016_Getting_Physical_Extreme_Abuse_of_Intel_Based_Paging_Systems/tree/master/Demos/Linux/exploits
>
>     Similar research was done at Google leading to this patch proposal.
>
>     Variants exists to overwrite /proc or /sys objects ACLs leading to
>     elevation of privileges. These variants were tested against 4.6+.
>
>     The page offset used by the compressed kernel retains the static value
>     since it is not yet randomized during this boot stage.
>
>     Signed-off-by: Thomas Garnier <thgarnie at google.com>
>     Signed-off-by: Kees Cook <keescook at chromium.org>
>     Cc: Alexander Kuleshov <kuleshovmail at gmail.com>
<truncated>

The interesting change seems to be :

> -#define __PAGE_OFFSET           _AC(0xffff880000000000, UL)
> +#define __PAGE_OFFSET_BASE      _AC(0xffff880000000000, UL)
> +#ifdef CONFIG_RANDOMIZE_MEMORY
> +#define __PAGE_OFFSET           page_offset_base
> +#else
> +#define __PAGE_OFFSET           __PAGE_OFFSET_BASE
> +#endif /* CONFIG_RANDOMIZE_MEMORY */

I'll try to see if I can fix that.

Kind regards,

...Louis




-- 
Louis Bouchard
Software engineer, Cloud & Sustaining eng.
Canonical Ltd
Ubuntu developer                       Debian Maintainer
GPG : 429D 7A3B DD05 B6F8 AF63  B9C4 8B3D 867C 823E 7A61



More information about the kexec mailing list