[RFC] arm64: extra entries in /proc/iomem for kexec
Bhupesh Sharma
bhsharma at redhat.com
Mon Mar 19 12:48:34 PDT 2018
On 03/14/2018 01:59 PM, AKASHI Takahiro wrote:
> In the last couples of months, there were some problems reported [1],[2]
> around arm64 kexec/kdump. Where those phenomenon look different,
> the root cause would be that kexec/kdump doesn't take into account
> crucial "reserved" regions of system memory and unintentionally corrupts
> them.
>
> Given that kexec-tools looks for all the information by seeking the file,
> /proc/iomem, the first step to address said problems is to expand this file's
> format so that it will have enough information about system memory and
> its usage.
>
> Attached is my experimental code: With this patch applied, /proc/iomem sees
> something like the below:
>
> (format A)
> 40000000-5871ffff : System RAM
> 40080000-40f1ffff : Kernel code
> 41040000-411e8fff : Kernel data
> 54400000-583fffff : Crash kernel
> 58590000-585effff : EFI Resources
> 58700000-5871ffff : EFI Resources
> 58720000-58b5ffff : System RAM
> 58720000-58b5ffff : EFI Resources
> 58b60000-5be3ffff : System RAM
> 58b61018-58b61947 : EFI Memory Map
> 59a7b118-59a7b667 : EFI Configuration Tables
> 5be40000-5becffff : System RAM <== (A-1)
> 5be40000-5becffff : EFI Resources
> 5bed0000-5bedffff : System RAM
> 5bee0000-5bffffff : System RAM
> 5bee0000-5bffffff : EFI Resources
> 5c000000-5fffffff : System RAM
> 8000000000-ffffffffff : PCI Bus 0000:00
>
> Meanwhile, the workaround I suggested in [3] gave us a simpler view:
>
> (format B)
> 40000000-5871ffff : System RAM
> 40080000-40f1ffff : Kernel code
> 41040000-411e9fff : Kernel data
> 54400000-583fffff : Crash kernel
> 58590000-585effff : reserved
> 58700000-5871ffff : reserved
> 58720000-58b5ffff : reserved
> 58b60000-5be3ffff : System RAM
> 58b61000-58b61fff : reserved
> 59a7b318-59a7b867 : reserved
> 5be40000-5becffff : reserved <== (B-1)
> 5bed0000-5bedffff : System RAM
> 5bee0000-5bffffff : reserved
> 5c000000-5fffffff : System RAM
> 5ec00000-5edfffff : reserved
> 8000000000-ffffffffff : PCI Bus 0000:00
>
> Here all the regions to be protected are named just "reserved" whether
> they are NOMAP regions or simply-memblock_reserve'd.
Personally, I like this format over the other two proposed.
However, I would suggest adding "reserved" regions as reserved (NOMAP)
regions and reserved (MAP'ed) regions (or a similar meaning wording for
the same).
> They are not very
> useful for anything but kexec/kdump which knows what they mean.
I disagree. I have found the naming does help in debugging issues
in the crashkernel itself which cause an early panic in the crashkernel.
Knowing the type of entry in '/proc/iomem' really helps in understanding
what the kexec-tools might have picked up and sent as a part of the
"linux,usable-memory" range property to the crashkernel.
> Alternatively, we may want to give them more specific names, based on
> related efi memory map descriptors and else, that will characterize
> their contents:
>
> (format C)
> 40000000-5871ffff : System RAM
> 40080000-40f1ffff : Kernel code
> 41040000-411e9fff : Kernel data
> 54400000-583fffff : Crash kernel
> 58590000-585effff : ACPI Reclaim Memory
> 58700000-5871ffff : ACPI Reclaim Memory
> 58720000-58b5ffff : System RAM
> 58720000-5878ffff : Runtime Data
> 58790000-587dffff : Runtime Code
> 587e0000-5882ffff : Runtime Data
> 58830000-5887ffff : Runtime Code
> 58880000-588cffff : Runtime Data
> 588d0000-5891ffff : Runtime Code
> 58920000-5896ffff : Runtime Data
> 58970000-589bffff : Runtime Code
> 589c0000-58a5ffff : Runtime Data
> 58a60000-58abffff : Runtime Code
> 58ac0000-58b0ffff : Runtime Data
> 58b10000-58b5ffff : Runtime Code
> 58b60000-5be3ffff : System RAM
> 58b61000-58b61fff : EFI Memory Map
> 59a7b118-59a7b667 : EFI Memory Attributes Table
> 5be40000-5becffff : System RAM
> 5be40000-5becffff : Runtime Code
> 5bed0000-5bedffff : System RAM
> 5bee0000-5bffffff : System RAM
> 5bee0000-5bffffff : Runtime Data
> 5c000000-5fffffff : System RAM
> 8000000000-ffffffffff : PCI Bus 0000:00
>
> I once created a patch for this format, but it looks quite noisy and
> names are a sort of mixture of memory attributes( ACPI Reclaim memory,
> Conventional Memory, Persistent Memory etc.) vs.
> function/usages ([Loader|Boot Service|Runtime] Code/Data).
> (As a matter of fact, (C-1) consists of various ACPI tables.)
> Anyhow, they seem not so useful for most of other applications.
>
> Those observations lead to format A, where some entries with the same
> attributes are squeezed into a single entry under a simple name if they
> are neighbouring.
>
>
> So my questions here are:
>
> 1. Which format, A, B, or C, is the most appropriate for the moment?
> or any other suggestions?
>
> Currently, there is a inconsistent view between (A) and the mainline's:
> see (A-1) and (B-1). If this is really a matter, I can fix it.
> Kexec-tools can be easily modified to accept both formats, though.
>
>
> 2. How should we determine which regions be exported in /proc/iomem?
>
> a. Trust all the memblock_reserve'd regions as my previous patch [3] does.
>
> As I said, it's a kind of "overkill." Some of regions, say fdt, are
> not required to be preserved across kexec.
I think we should preserve all the memblock_reserve'd regions. So +1 on
this approach from my side. I believe it might help avoid issues we have
seen in the past with 'kexec-tools' _incorrectly_ determining which
regions to pick from the '/proc/iomem'.
If every memblock_reserve'd region is exported in /proc/iomem', its
easier to debug issues in the 'kexec-tools' which might have cause the
early crashkernel to panic and we can exclude primary kernel as a
potential suspect for causing the same.
Regards,
Bhupesh
>
> b. List regions separately and exhaustively later on at a single point
> as my patch attached does.
>
> I'm afraid of any possibility that some regions may be doubly counted,
> one from efi memory map search and another from other efi/acpi code
> (various type of "tables" in most cases).
>
> c. Expand efi_mem_reserve() with an argument of "resource descriptor" and
> replace memblock_reserve() in efi code wherever necessary so as to
> maintain an export list.
>
> efi_mem_reserve() was first introduced for specific needs at kexec
> on x86, but I believe that its coverage over efi code is far from perfect.
>
> [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2017-November/541831.html
> [2] http://lkml.iu.edu//hypermail/linux/kernel/1802.2/06745.html
> [3] http://lkml.iu.edu//hypermail/linux/kernel/1803.0/04658.html
> http://lkml.iu.edu//hypermail/linux/kernel/1803.0/04659.html
>
> -Takahiro AKASHI
>
> ===8<===
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index 30ad2f085d1f..feda5cbdc6bf 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -214,13 +214,8 @@ static void __init request_standard_resources(void)
>
> for_each_memblock(memory, region) {
> res = alloc_bootmem_low(sizeof(*res));
> - if (memblock_is_nomap(region)) {
> - res->name = "reserved";
> - res->flags = IORESOURCE_MEM;
> - } else {
> - res->name = "System RAM";
> - res->flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
> - }
> + res->name = "System RAM";
> + res->flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
> res->start = __pfn_to_phys(memblock_region_memory_base_pfn(region));
> res->end = __pfn_to_phys(memblock_region_memory_end_pfn(region)) - 1;
>
> @@ -239,6 +234,9 @@ static void __init request_standard_resources(void)
> request_resource(res, &crashk_res);
> #endif
> }
> +
> + /* Add firmware-reserved memory */
> + efi_arch_request_resources();
> }
>
> u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID };
> diff --git a/drivers/firmware/efi/arm-init.c b/drivers/firmware/efi/arm-init.c
> index 80d1a885def5..308143e69db4 100644
> --- a/drivers/firmware/efi/arm-init.c
> +++ b/drivers/firmware/efi/arm-init.c
> @@ -13,8 +13,10 @@
>
> #define pr_fmt(fmt) "efi: " fmt
>
> +#include <linux/bootmem.h>
> #include <linux/efi.h>
> #include <linux/init.h>
> +#include <linux/ioport.h>
> #include <linux/memblock.h>
> #include <linux/mm_types.h>
> #include <linux/of.h>
> @@ -280,3 +282,97 @@ static int __init register_gop_device(void)
> return PTR_ERR_OR_ZERO(pd);
> }
> subsys_initcall(register_gop_device);
> +
> +static unsigned long __init efi_memattr_to_iores_type(u64 addr)
> +{
> + if (efi_mem_attributes(addr) &
> + (EFI_MEMORY_WB|EFI_MEMORY_WT|EFI_MEMORY_WC))
> + return IORESOURCE_SYSTEM_RAM;
> + else
> + return IORESOURCE_MEM;
> +}
> +
> +static unsigned long __init efi_type_to_iores_desc(u64 addr)
> +{
> + /* TODO */
> + return IORES_DESC_NONE;
> +}
> +
> +static struct resource * __init _efi_arch_request_resource(u64 start, u64 end,
> + char *name, struct resource *prev)
> +{
> + struct resource *conflict, *res;
> +
> + res = alloc_bootmem_low(sizeof(*res));
> +
> + res->start = start;
> + res->end = end;
> + res->flags = efi_memattr_to_iores_type(res->start);
> + res->desc = efi_type_to_iores_desc(res->start);
> + res->name = name;
> +
> + conflict = request_resource_conflict(&iomem_resource, res);
> + if (conflict) {
> + if (prev && (prev->parent == conflict) &&
> + ((prev->end + 1) == start)) {
> + /* merge consecutive regions */
> + adjust_resource(prev, prev->start,
> + end - prev->start + 1);
> + free_bootmem((unsigned long)res, sizeof(*res));
> + res = prev;
> + } else
> + insert_resource(conflict, res);
> + }
> +
> + return res;
> +}
> +
> +/* Kexec expects those resources to be preserved */
> +void __init efi_arch_request_resources(void)
> +{
> + struct resource *res = NULL;
> + efi_memory_desc_t *md;
> + u64 paddr, npages, size;
> +
> + /* EFI Memory Map */
> + /* FIXME */
> + _efi_arch_request_resource(efi.memmap.phys_map,
> + efi.memmap.phys_map
> + + efi.memmap.desc_size * efi.memmap.nr_map - 1,
> + "EFI Memory Map", res);
> +
> + /* generic EFI Configuration Tables */
> + efi_request_config_table_resources(_efi_arch_request_resource);
> +
> + /* architecture-specifc Configuration Tables */
> + if (screen_info.lfb_size)
> + _efi_arch_request_resource(screen_info.lfb_base,
> + screen_info.lfb_base + screen_info.lfb_size - 1,
> + "EFI Screen Info Table", res);
> +
> + /* architecture-specific EFI resources */
> + /* FIXME */
> + efi_memmap_install(efi.memmap.phys_map, efi.memmap.nr_map);
> +
> + res = NULL;
> + for_each_efi_memory_desc(md) {
> + paddr = md->phys_addr;
> + npages = md->num_pages;
> +
> + memrange_efi_to_native(&paddr, &npages);
> + size = npages << PAGE_SHIFT;
> +
> + if (is_memory(md)) {
> + if (!is_usable_memory(md))
> + res = _efi_arch_request_resource(paddr,
> + paddr + size - 1,
> + "EFI Resources", res);
> +
> + if (md->type == EFI_ACPI_RECLAIM_MEMORY)
> + res = _efi_arch_request_resource(paddr,
> + paddr + size - 1,
> + "EFI Resources", res);
> + }
> + }
> + efi_memmap_unmap();
> +}
> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
> index cd42f66a7c85..b13c9461278b 100644
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -603,6 +603,33 @@ int __init efi_config_init(efi_config_table_type_t *arch_tables)
> return ret;
> }
>
> +void __init efi_request_config_table_resources(struct resource *(*fn)(u64 start,
> + u64 end, char *name, struct resource *prev))
> +{
> + struct resource *prev = NULL;
> + char *name = "EFI Configuration Tables";
> +
> + if (efi.config_table_size)
> + prev = fn(efi.config_table,
> + efi.config_table + efi.config_table_size - 1, name,
> + prev);
> +
> + if (efi.mem_attr_table_size)
> + prev = fn(efi.mem_attr_table,
> + efi.mem_attr_table + efi.mem_attr_table_size - 1, name,
> + prev);
> +
> + if (efi.esrt_size)
> + prev = fn(efi.esrt, efi.esrt + efi.esrt_size - 1, name, prev);
> +
> + if (efi.tpm_log_size)
> + prev = fn(efi.tpm_log, efi.tpm_log + efi.tpm_log_size - 1, name,
> + prev);
> +
> +
> + /* TODO: BGRT */
> +}
> +
> #ifdef CONFIG_EFI_VARS_MODULE
> static int __init efi_load_efivars(void)
> {
> diff --git a/drivers/firmware/efi/esrt.c b/drivers/firmware/efi/esrt.c
> index c47e0c6ec00f..61f66c139afb 100644
> --- a/drivers/firmware/efi/esrt.c
> +++ b/drivers/firmware/efi/esrt.c
> @@ -330,6 +330,7 @@ void __init efi_esrt_init(void)
>
> esrt_data = (phys_addr_t)efi.esrt;
> esrt_data_size = size;
> + efi.esrt_size = size;
>
> end = esrt_data + size;
> pr_info("Reserving ESRT space from %pa to %pa.\n", &esrt_data, &end);
> diff --git a/drivers/firmware/efi/memattr.c b/drivers/firmware/efi/memattr.c
> index 8986757eafaf..dc2c7608793a 100644
> --- a/drivers/firmware/efi/memattr.c
> +++ b/drivers/firmware/efi/memattr.c
> @@ -42,6 +42,7 @@ int __init efi_memattr_init(void)
> }
>
> tbl_size = sizeof(*tbl) + tbl->num_entries * tbl->desc_size;
> + efi.mem_attr_table_size = tbl_size;
> memblock_reserve(efi.mem_attr_table, tbl_size);
> set_bit(EFI_MEM_ATTR, &efi.flags);
>
> diff --git a/drivers/firmware/efi/tpm.c b/drivers/firmware/efi/tpm.c
> index 0cbeb3d46b18..53cfb12513fa 100644
> --- a/drivers/firmware/efi/tpm.c
> +++ b/drivers/firmware/efi/tpm.c
> @@ -33,6 +33,7 @@ int __init efi_tpm_eventlog_init(void)
> }
>
> tbl_size = sizeof(*log_tbl) + log_tbl->size;
> + efi.tpm_log_size = tbl_size;
> memblock_reserve(efi.tpm_log, tbl_size);
> early_memunmap(log_tbl, sizeof(*log_tbl));
> return 0;
> diff --git a/include/linux/efi.h b/include/linux/efi.h
> index f5083aa72eae..9c3f8d284b36 100644
> --- a/include/linux/efi.h
> +++ b/include/linux/efi.h
> @@ -942,11 +942,15 @@ extern struct efi {
> unsigned long fw_vendor; /* fw_vendor */
> unsigned long runtime; /* runtime table */
> unsigned long config_table; /* config tables */
> + unsigned long config_table_size;
> unsigned long esrt; /* ESRT table */
> + unsigned long esrt_size;
> unsigned long properties_table; /* properties table */
> unsigned long mem_attr_table; /* memory attributes table */
> + unsigned long mem_attr_table_size;
> unsigned long rng_seed; /* UEFI firmware random seed */
> unsigned long tpm_log; /* TPM2 Event Log table */
> + unsigned long tpm_log_size;
> efi_get_time_t *get_time;
> efi_set_time_t *set_time;
> efi_get_wakeup_time_t *get_wakeup_time;
> @@ -980,6 +984,9 @@ efi_guid_to_str(efi_guid_t *guid, char *out)
> }
>
> extern void efi_init (void);
> +extern void efi_arch_request_resources(void);
> +extern void efi_request_config_table_resources(struct resource *
> + (*fn)(u64 start, u64 end, char *name, struct resource *prev));
> extern void *efi_get_pal_addr (void);
> extern void efi_map_pal_code (void);
> extern void efi_memmap_walk (efi_freemem_callback_t callback, void *arg);
>
More information about the kexec
mailing list