From horms at kernel.org  Tue Aug  1 00:00:41 2023
From: horms at kernel.org (Simon Horman)
Date: Tue, 1 Aug 2023 09:00:41 +0200
Subject: [PATCHv6 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <20230724022142.6425-4-piliu@redhat.com>
References: <20230724022142.6425-1-piliu@redhat.com>
 <20230724022142.6425-4-piliu@redhat.com>
Message-ID: <ZMitmSZPqnH+/70s@kernel.org>

On Mon, Jul 24, 2023 at 10:21:40AM +0800, Pingfan Liu wrote:
> From: Jeremy Linton <jeremy.linton at arm.com>
> 
> The linux kernel CONFIG_ZBOOT option creates
> self decompressing PE kernel images. So this means
> that kexec should have a generic understanding of
> the format which may be used by multiple arches.
> 
> So lets add an arch independent validation
> and decompression routine.
> 
> Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> [Modified by Pingfan to export kernel fd]
> Signed-off-by: Pingfan Liu <piliu at redhat.com>

Hi Pingfan,

unfortunately this causes a build failure on hppa.

../../kexec/kexec-pe-zboot.c:31:10: fatal error: kexec-pe-zboot.h: No such file or directory
   31 | #include <kexec-pe-zboot.h>
      |          ^~~~~~~~~~~~~~~~~~

Link: https://github.com/horms/kexec-tools/actions/runs/5723580523/job/15508425790


From petrtesarik at huaweicloud.com  Tue Aug  1 02:27:35 2023
From: petrtesarik at huaweicloud.com (Petr Tesarik)
Date: Tue, 1 Aug 2023 11:27:35 +0200
Subject: [PATCH v2 0/3] RISC-V: Fix a few kexec_file_load(2) failures
In-Reply-To: <cover.1690365011.git.petr.tesarik.ext@huawei.com>
References: <cover.1690365011.git.petr.tesarik.ext@huawei.com>
Message-ID: <c1b125d6-7031-d0aa-bcaa-8fb3875ac64f@huaweicloud.com>

Hi,

On 7/26/2023 11:53 AM, Petr Tesarik wrote:
> From: Petr Tesarik <petr.tesarik.ext at huawei.com>
> 
> The kexec_file_load(2) syscall does not work at least in some kernel
> builds. For details see the relevant section in this blog post:
> 
> https://sigillatum.tesarici.cz/2023-07-21-state-of-riscv64-kdump.html
> 
> This patch series handles an additional relocation types, removes the need
> to implement a Global Offset Table (GOT) for the purgatory and fixes the
> placement of initrd.

It seems there are no objections, but what is the plan here? Take it
into 6.5 as a fix, or let it go through for-next?

Petr T

> Changelog
> =========
> 
> Changes from v1:
> - Replace memcmp() with a for loop.
> - Drop handling of 16-bit add/subtract relocations. They were used only
>   by alternatives in strcmp(), which was referenced only by string.o.
> - Add the initrd placement fix.
> 
> Petr Tesarik (1):
>   riscv/purgatory: do not link with string.o and its dependencies
> 
> Torsten Duwe (2):
>   riscv/kexec: handle R_RISCV_CALL_PLT relocation type
>   riscv/kexec: load initrd high in available memory
> 
>  arch/riscv/kernel/elf_kexec.c    |  3 ++-
>  arch/riscv/purgatory/Makefile    | 26 +-------------------------
>  arch/riscv/purgatory/purgatory.c |  6 ++++--
>  3 files changed, 7 insertions(+), 28 deletions(-)
> 



From tusharsu at linux.microsoft.com  Tue Aug  1 11:19:11 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Tue,  1 Aug 2023 11:19:11 -0700
Subject: [PATCH 0/6] Measuring TPM update counter in IMA
Message-ID: <20230801181917.8535-1-tusharsu@linux.microsoft.com>

Entries in IMA log may be lost due to code bugs, certain error conditions
being met etc.  This can result in TPM PCRs getting out of sync with the
IMA log.  One such example is events between kexec 'load' and kexec
'execute' getting lost from the IMA log when the system soft-boots into
the new Kernel using kexec[1].  The remote attestation service does not
have any information if the PCR mismatch with IMA log is because of loss
of entries in the IMA log or something else.  TPM 2.0 provides an update
counter which is incremented each time a PCR is updated [2]. Measuring the
TPM PCR update counter in IMA subsystem will help the remote attestation
service to validate if there are any missing entries in the IMA log, when
the system goes through certain important state changes (e.g. kexec soft
boot, IMA log snapshotting etc.)

This patch series provides the required functionality to measure the
update counter through IMA subsystem by -
 - introducing a function to retrieve PCR update counter in the TPM
   subsystem.
 - IMA functionality to acquire the update counter from the TPM subsystem.
 - Measuring the update counter at system boot and at kexec Kernel
   load.


This patch series would be a prerequisite for the next version of kexec
load/execute series[1] and the future IMA log snapshotting patch series.

[1] https://lore.kernel.org/all/20230703215709.1195644-1-tusharsu at linux.microsoft.com/
    ima: measure events between kexec load and execute

[2] https://trustedcomputinggroup.org/wp-content/uploads/TCG_TPM2_r1p59_Part3_Commands_pub.pdf
    Section 22.4.2, Page 206.

Tushar Sugandhi (6):
  tpm: implement TPM2 function to get update counter
  tpm: provide functionality to get update counter
  ima: get TPM update counter
  ima: implement functionality to measure TPM update counter
  ima: measure TPM update counter at ima_init
  kexec: measure TPM update counter in ima log at kexec load

 drivers/char/tpm/tpm-interface.c   | 28 +++++++++++++++++
 drivers/char/tpm/tpm.h             |  3 ++
 drivers/char/tpm/tpm2-cmd.c        | 48 ++++++++++++++++++++++++++++++
 include/linux/ima.h                |  1 +
 include/linux/tpm.h                |  8 +++++
 kernel/kexec_file.c                |  3 ++
 security/integrity/ima/ima.h       |  2 ++
 security/integrity/ima/ima_init.c  |  3 ++
 security/integrity/ima/ima_main.c  | 29 ++++++++++++++++++
 security/integrity/ima/ima_queue.c | 16 ++++++++++
 10 files changed, 141 insertions(+)

-- 
2.25.1



From tusharsu at linux.microsoft.com  Tue Aug  1 11:19:12 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Tue,  1 Aug 2023 11:19:12 -0700
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
Message-ID: <20230801181917.8535-2-tusharsu@linux.microsoft.com>

The TPM2_PCR_Read command returns TPM2_PCR_Read Response struct[1].  It
contains pcrUpdateCounter member which contains the current value of TPM
PCR update counter.  The update counter provides the number of times the
PCRs are updated, which is essential for tracking changes and verifying
system integrity.  Thus, subsystems (like IMA) should measure
pcrUpdateCounter value.  Although tpm2_pcr_read_out struct is returned
by tpm2_pcr_read(), it is not used by it's caller function tpm_pcr_read().
Further, TPM2_PCR_Read Response struct and pcrUpdateCounter is not
available in tpm1_pcr_read().

PcrUpdateCounter is only needed in a specific case (IMA for measurements).
Changing tpm_pcr_read() and tpm2_pcr_read() function signature to return
tpm2_pcr_read_out struct would be a more disruptive change, since these
functions are used elsewhere too.  Creating separate functions to get
pcrUpdateCounter when needed would be a cleaner approach.

Add a function, 'tpm2_pcr_get_update_counter()' to retrieve
the update counter for a given PCR index and algorithm ID on a TPM2 chip.

This function complements existing TPM functionalities such as reading
and extending PCRs, and enhances the ability to monitor PCR status
in the Linux Kernel. 

[1] https://trustedcomputinggroup.org/wp-content/uploads/TCG_TPM2_r1p59_Part3_Commands_pub.pdf
Section 22.4.2, Page 206.

Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
---
 drivers/char/tpm/tpm.h      |  3 +++
 drivers/char/tpm/tpm2-cmd.c | 48 +++++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+)

diff --git a/drivers/char/tpm/tpm.h b/drivers/char/tpm/tpm.h
index 830014a26609..60489f21d3bd 100644
--- a/drivers/char/tpm/tpm.h
+++ b/drivers/char/tpm/tpm.h
@@ -288,6 +288,9 @@ static inline void tpm_add_ppi(struct tpm_chip *chip)
 int tpm2_get_timeouts(struct tpm_chip *chip);
 int tpm2_pcr_read(struct tpm_chip *chip, u32 pcr_idx,
 		  struct tpm_digest *digest, u16 *digest_size_ptr);
+int tpm2_pcr_get_update_counter(struct tpm_chip *chip,
+				u32 pcr_idx, u16 alg_id,
+				u32 *update_counter);
 int tpm2_pcr_extend(struct tpm_chip *chip, u32 pcr_idx,
 		    struct tpm_digest *digests);
 int tpm2_get_random(struct tpm_chip *chip, u8 *dest, size_t max);
diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c
index 93545be190a5..55f4e102289a 100644
--- a/drivers/char/tpm/tpm2-cmd.c
+++ b/drivers/char/tpm/tpm2-cmd.c
@@ -216,6 +216,54 @@ int tpm2_pcr_read(struct tpm_chip *chip, u32 pcr_idx,
 	return rc;
 }
 
+/**
+ * tpm2_pcr_get_update_counter() - gets an update counter value for a PCR bank
+ * @chip: TPM chip to use
+ * @pcr_idx: PCR index used to retrieve the update counter
+ * @alg_id:	alg id used to retrieve the update counter
+ * @update_counter: output update counter value
+ *
+ * Return: Same as with tpm_transmit_cmd.
+ */
+int tpm2_pcr_get_update_counter(struct tpm_chip *chip,
+				u32 pcr_idx, u16 alg_id, u32 *update_counter)
+{
+	int rc;
+	struct tpm_buf buf;
+	struct tpm2_pcr_read_out *read_out;
+	u8 pcr_select[TPM2_PCR_SELECT_MIN] = {0};
+
+	if (pcr_idx >= TPM2_PLATFORM_PCR)
+		return -EINVAL;
+
+	if (!update_counter)
+		return -EINVAL;
+
+	rc = tpm_buf_init(&buf, TPM2_ST_NO_SESSIONS, TPM2_CC_PCR_READ);
+	if (rc)
+		return rc;
+
+	pcr_select[pcr_idx >> 3] = 1 << (pcr_idx & 0x7);
+
+	tpm_buf_append_u32(&buf, 1);
+	tpm_buf_append_u16(&buf, alg_id);
+	tpm_buf_append_u8(&buf, TPM2_PCR_SELECT_MIN);
+	tpm_buf_append(&buf, (const unsigned char *)pcr_select,
+		       sizeof(pcr_select));
+
+	rc = tpm_transmit_cmd(chip, &buf, 0, "attempting to read a pcr value");
+	if (rc)
+		goto out;
+
+	read_out = (struct tpm2_pcr_read_out *)&buf.data[TPM_HEADER_SIZE];
+
+	*update_counter = be32_to_cpu(read_out->update_cnt);
+
+out:
+	tpm_buf_destroy(&buf);
+	return rc;
+}
+
 struct tpm2_null_auth_area {
 	__be32  handle;
 	__be16  nonce_size;
-- 
2.25.1



From tusharsu at linux.microsoft.com  Tue Aug  1 11:19:14 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Tue,  1 Aug 2023 11:19:14 -0700
Subject: [PATCH 3/6] ima: get TPM update counter
In-Reply-To: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
Message-ID: <20230801181917.8535-4-tusharsu@linux.microsoft.com>

Measuring the TPM PCR update counter will help the remote attestation
service to validate if there are any missing entries in the IMA log, when
the system goes through certain important state changes (e.g. kexec soft
boot, IMA log snapshotting etc.).  Detecting such missing entries would
help the remote attestation service functionality to be more robust.
It should also help the system administrators with manual investigations
when TPM PCR quotes go out of sync with IMA measurements.

Implement a new function, 'ima_tpm_get_update_counter()', which uses
the 'tpm_pcr_get_update_counter()' function from the TPM driver interface
to retrieve the PCR update counter of the TPM chip in use.

Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
---
 security/integrity/ima/ima.h       |  1 +
 security/integrity/ima/ima_queue.c | 16 ++++++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/security/integrity/ima/ima.h b/security/integrity/ima/ima.h
index c29db699c996..4acd0e5a830f 100644
--- a/security/integrity/ima/ima.h
+++ b/security/integrity/ima/ima.h
@@ -167,6 +167,7 @@ void ima_init_template_list(void);
 int __init ima_init_digests(void);
 int ima_lsm_policy_change(struct notifier_block *nb, unsigned long event,
 			  void *lsm_data);
+int ima_tpm_get_update_counter(u32 *cpu_update_counter);
 
 /*
  * used to protect h_table and sha_table
diff --git a/security/integrity/ima/ima_queue.c b/security/integrity/ima/ima_queue.c
index 532da87ce519..38f5c35b23b2 100644
--- a/security/integrity/ima/ima_queue.c
+++ b/security/integrity/ima/ima_queue.c
@@ -135,6 +135,22 @@ unsigned long ima_get_binary_runtime_size(void)
 		return binary_runtime_size + sizeof(struct ima_kexec_hdr);
 }
 
+int ima_tpm_get_update_counter(u32 *update_counter)
+{
+	int result;
+
+	if (!update_counter)
+		return -EINVAL;
+
+	result = tpm_pcr_get_update_counter(ima_tpm_chip,
+				CONFIG_IMA_MEASURE_PCR_IDX, TPM_ALG_SHA1, update_counter);
+
+	if (result != 0)
+		pr_err("Failed to get TPM PCR update counter, result: %d\n", result);
+
+	return result;
+}
+
 static int ima_pcr_extend(struct tpm_digest *digests_arg, int pcr)
 {
 	int result = 0;
-- 
2.25.1



From tusharsu at linux.microsoft.com  Tue Aug  1 11:19:15 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Tue,  1 Aug 2023 11:19:15 -0700
Subject: [PATCH 4/6] ima: implement functionality to measure TPM update counter
In-Reply-To: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
Message-ID: <20230801181917.8535-5-tusharsu@linux.microsoft.com>

Currently TPM update counter is not available external to the system,
for instance, a remote attestation service.  It is a problem because
the service cannot easily determine if the IMA log entries are missing.
The IMA functionality needs to be extended to measure the TPM update
counter from various subsystems in Linux kernel to help detect if
the IMA log entries are missing.

Implement a function, 'ima_measure_update_counter()' which would retrieve
the TPM update counter using the previously defined function
'ima_tpm_get_update_counter()'.  Format it as a string with the value 
"update_counter=<N>;", and measure it using the function
'ima_measure_critical_data()'.

The function takes an event name as input, and the update counter value
is measured as part of this event.

Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
---
 include/linux/ima.h               |  1 +
 security/integrity/ima/ima.h      |  1 +
 security/integrity/ima/ima_main.c | 28 ++++++++++++++++++++++++++++
 3 files changed, 30 insertions(+)

diff --git a/include/linux/ima.h b/include/linux/ima.h
index 86b57757c7b1..f15f3a6a4c72 100644
--- a/include/linux/ima.h
+++ b/include/linux/ima.h
@@ -40,6 +40,7 @@ extern int ima_measure_critical_data(const char *event_label,
 				     const char *event_name,
 				     const void *buf, size_t buf_len,
 				     bool hash, u8 *digest, size_t digest_len);
+int ima_measure_update_counter(const char *event_name);
 
 #ifdef CONFIG_IMA_APPRAISE_BOOTPARAM
 extern void ima_appraise_parse_cmdline(void);
diff --git a/security/integrity/ima/ima.h b/security/integrity/ima/ima.h
index 4acd0e5a830f..5484bd362237 100644
--- a/security/integrity/ima/ima.h
+++ b/security/integrity/ima/ima.h
@@ -168,6 +168,7 @@ int __init ima_init_digests(void);
 int ima_lsm_policy_change(struct notifier_block *nb, unsigned long event,
 			  void *lsm_data);
 int ima_tpm_get_update_counter(u32 *cpu_update_counter);
+int ima_measure_update_counter(const char *event_name);
 
 /*
  * used to protect h_table and sha_table
diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
index d66a0a36415e..1bcd45cc5a6a 100644
--- a/security/integrity/ima/ima_main.c
+++ b/security/integrity/ima/ima_main.c
@@ -1071,6 +1071,34 @@ int ima_measure_critical_data(const char *event_label,
 }
 EXPORT_SYMBOL_GPL(ima_measure_critical_data);
 
+#define IMA_TPM_UPDATE_CTR_BUF_SIZE 128
+int ima_measure_update_counter(const char *event_name)
+{
+	int result;
+	u32 update_counter = 0;
+	char buf[IMA_TPM_UPDATE_CTR_BUF_SIZE];
+	int buf_len;
+
+	if (!event_name)
+		return -ENOPARAM;
+
+	result = ima_tpm_get_update_counter(&update_counter);
+
+	if (result != 0)
+		return result;
+
+	scnprintf(buf, IMA_TPM_UPDATE_CTR_BUF_SIZE, "update_counter=%u;",
+			  update_counter);
+
+	buf_len = strlen(buf);
+
+	result = ima_measure_critical_data("tpm_pcr_update_counter", event_name,
+				  buf, buf_len, false, NULL, 0);
+
+	return result;
+}
+EXPORT_SYMBOL_GPL(ima_measure_update_counter);
+
 static int __init init_ima(void)
 {
 	int error;
-- 
2.25.1



From tusharsu at linux.microsoft.com  Tue Aug  1 11:19:13 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Tue,  1 Aug 2023 11:19:13 -0700
Subject: [PATCH 2/6] tpm: provide functionality to get update counter
In-Reply-To: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
Message-ID: <20230801181917.8535-3-tusharsu@linux.microsoft.com>

The IMA subsystem needs to measure pcrUpdateCounter value present in 
TPM2_PCR_Read Response struct [1].  However,the  pcrUpdateCounter value
is not exposed outside of the TPM subsystem by any of the existing
functions. 

Implement a new function 'tpm_pcr_get_update_counter()',  which provides
a way to retrieve the PCR update counter values from subsystems outside 
of TPM.  If the input tpm_chip is not a TPM2 chip, return an error as
the functionality is currently only implemented for TPM2 chips.

This function improves TPM capabilities in the Linux kernel by
facilitating access to the PCR update counter.

[1] https://trustedcomputinggroup.org/wp-content/uploads/TCG_TPM2_r1p59_Part3_Commands_pub.pdf
Section 22.4.2, Page 206.

Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
---
 drivers/char/tpm/tpm-interface.c | 28 ++++++++++++++++++++++++++++
 include/linux/tpm.h              |  8 ++++++++
 2 files changed, 36 insertions(+)

diff --git a/drivers/char/tpm/tpm-interface.c b/drivers/char/tpm/tpm-interface.c
index 7e513b771832..9a1088914487 100644
--- a/drivers/char/tpm/tpm-interface.c
+++ b/drivers/char/tpm/tpm-interface.c
@@ -300,6 +300,34 @@ int tpm_pcr_read(struct tpm_chip *chip, u32 pcr_idx,
 }
 EXPORT_SYMBOL_GPL(tpm_pcr_read);
 
+/**
+ * tpm_pcr_get_update_counter - gets an update counter value for a PCR bank
+ * @chip: a &struct tpm_chip instance, %NULL for the default chip
+ * @pcr_idx: PCR index used to retrieve the update counter
+ * @alg_id:	alg id used to retrieve the update counter
+ * @update_counter: output update counter value
+ *
+ * Return: same as with tpm_transmit_cmd()
+ */
+int tpm_pcr_get_update_counter(struct tpm_chip *chip, u32 pcr_idx,
+		 u16 alg_id, u32 *update_counter)
+{
+	int rc;
+
+	chip = tpm_find_get_ops(chip);
+	if (!chip)
+		return -ENODEV;
+
+	if (chip->flags & TPM_CHIP_FLAG_TPM2)
+		rc = tpm2_pcr_get_update_counter(chip, pcr_idx, alg_id,
+						 update_counter);
+	else
+		rc = -ENODEV;
+
+	tpm_put_ops(chip);
+	return rc;
+}
+EXPORT_SYMBOL_GPL(tpm_pcr_get_update_counter);
 /**
  * tpm_pcr_extend - extend a PCR value in SHA1 bank.
  * @chip:	a &struct tpm_chip instance, %NULL for the default chip
diff --git a/include/linux/tpm.h b/include/linux/tpm.h
index 4dc97b9f65fb..3b55218b70fa 100644
--- a/include/linux/tpm.h
+++ b/include/linux/tpm.h
@@ -424,6 +424,8 @@ extern ssize_t tpm_transmit_cmd(struct tpm_chip *chip, struct tpm_buf *buf,
 				size_t min_rsp_body_length, const char *desc);
 extern int tpm_pcr_read(struct tpm_chip *chip, u32 pcr_idx,
 			struct tpm_digest *digest);
+extern int tpm_pcr_get_update_counter(struct tpm_chip *chip, u32 pcr_idx,
+		 u16 alg_id, u32 *update_counter);
 extern int tpm_pcr_extend(struct tpm_chip *chip, u32 pcr_idx,
 			  struct tpm_digest *digests);
 extern int tpm_send(struct tpm_chip *chip, void *cmd, size_t buflen);
@@ -440,6 +442,12 @@ static inline int tpm_pcr_read(struct tpm_chip *chip, int pcr_idx,
 {
 	return -ENODEV;
 }
+static inline int tpm_pcr_get_update_counter(struct tpm_chip *chip,
+					     u32 pcr_idx, u16 alg_id,
+					     u32 *update_counter)
+{
+	return -ENODEV;
+}
 
 static inline int tpm_pcr_extend(struct tpm_chip *chip, u32 pcr_idx,
 				 struct tpm_digest *digests)
-- 
2.25.1



From tusharsu at linux.microsoft.com  Tue Aug  1 11:19:17 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Tue,  1 Aug 2023 11:19:17 -0700
Subject: [PATCH 6/6] kexec: measure TPM update counter in ima log at kexec load
In-Reply-To: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
Message-ID: <20230801181917.8535-7-tusharsu@linux.microsoft.com>

IMA measurements snapshot occurs at kexec 'load', but any additional
measurements between 'load' and kexec 'execute' aren't carried over
post kexec soft-reboot.[1] This may lead to TPM PCRs extending with
events that are not reflected in the new Kernel's IMA log.  By measuring
the TPM update counter at kexec 'load' and at ima_init after the kexec
soft-reboot, the remote attestation service can identify potentially
lost events by comparing the log event count with the counter difference.

Measure the TPM update counter at kexec image load.

[1] https://lore.kernel.org/all/20230703215709.1195644-1-tusharsu at linux.microsoft.com/
    ima: measure events between kexec load and execute

Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
---
 kernel/kexec_file.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index f1a0e4e3fb5c..4b6391b02c5a 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -246,6 +246,9 @@ kimage_file_prepare_segments(struct kimage *image, int kernel_fd, int initrd_fd,
 				  image->cmdline_buf_len - 1);
 	}
 
+	/* Measures TPM update counter at kexec load. */
+	ima_measure_update_counter("kexec_load_tpm_update_counter");
+
 	/* IMA needs to pass the measurement list to the next kernel. */
 	ima_add_kexec_buffer(image);
 
-- 
2.25.1



From tusharsu at linux.microsoft.com  Tue Aug  1 11:19:16 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Tue,  1 Aug 2023 11:19:16 -0700
Subject: [PATCH 5/6] ima: measure TPM update counter at ima_init
In-Reply-To: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
Message-ID: <20230801181917.8535-6-tusharsu@linux.microsoft.com>

IMA log entries can be lost due to a variety of causes, such as code bugs
or error conditions, leading to a mismatch between TPM PCRs and
the IMA log.  Measuring TPM PCR update counter during ima_init would
provide a baseline counter for the number of times the TPM PCRs are
updated.  The remote attestation service can compare this baseline
counter with a subsequent measured one (e.g., post-kexec soft-boot) to
identify if there are any lost IMA log events.

Measure the TPM update counter at ima init.

Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
---
 security/integrity/ima/ima_init.c | 3 +++
 security/integrity/ima/ima_main.c | 1 +
 2 files changed, 4 insertions(+)

diff --git a/security/integrity/ima/ima_init.c b/security/integrity/ima/ima_init.c
index 63979aefc95f..9bb18d6c2fd6 100644
--- a/security/integrity/ima/ima_init.c
+++ b/security/integrity/ima/ima_init.c
@@ -154,5 +154,8 @@ int __init ima_init(void)
 				  UTS_RELEASE, strlen(UTS_RELEASE), false,
 				  NULL, 0);
 
+	/* Measures TPM update counter at ima_init */
+	ima_measure_update_counter("ima_init_tpm_update_counter");
+
 	return rc;
 }
diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
index 1bcd45cc5a6a..93357c245e82 100644
--- a/security/integrity/ima/ima_main.c
+++ b/security/integrity/ima/ima_main.c
@@ -1035,6 +1035,7 @@ void ima_kexec_cmdline(int kernel_fd, const void *buf, int size)
 				   buf, size, "kexec-cmdline", KEXEC_CMDLINE, 0,
 				   NULL, false, NULL, 0);
 	fdput(f);
+
 }
 
 /**
-- 
2.25.1



From jarkko at kernel.org  Tue Aug  1 12:02:45 2023
From: jarkko at kernel.org (Jarkko Sakkinen)
Date: Tue, 01 Aug 2023 22:02:45 +0300
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <20230801181917.8535-2-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
Message-ID: <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>

The short summary is cryptic to say the least.

"update counter" does not map it to have anything to do with PCRs.

Why not "tpm: Read pcrUpdateCounter field from TPM2_PCR_Read"?

On Tue Aug 1, 2023 at 9:19 PM EEST, Tushar Sugandhi wrote:
> The TPM2_PCR_Read command returns TPM2_PCR_Read Response struct[1].  It
> contains pcrUpdateCounter member which contains the current value of TPM
> PCR update counter.  The update counter provides the number of times the
> PCRs are updated, which is essential for tracking changes and verifying
> system integrity.  Thus, subsystems (like IMA) should measure
> pcrUpdateCounter value.  Although tpm2_pcr_read_out struct is returned
> by tpm2_pcr_read(), it is not used by it's caller function tpm_pcr_read().
> Further, TPM2_PCR_Read Response struct and pcrUpdateCounter is not
> available in tpm1_pcr_read().
>
> PcrUpdateCounter is only needed in a specific case (IMA for measurements).
> Changing tpm_pcr_read() and tpm2_pcr_read() function signature to return
> tpm2_pcr_read_out struct would be a more disruptive change, since these
> functions are used elsewhere too.  Creating separate functions to get
> pcrUpdateCounter when needed would be a cleaner approach.
>
> Add a function, 'tpm2_pcr_get_update_counter()' to retrieve
> the update counter for a given PCR index and algorithm ID on a TPM2 chip.
>
> This function complements existing TPM functionalities such as reading
> and extending PCRs, and enhances the ability to monitor PCR status
> in the Linux Kernel. 
>
> [1] https://trustedcomputinggroup.org/wp-content/uploads/TCG_TPM2_r1p59_Part3_Commands_pub.pdf
> Section 22.4.2, Page 206.
>
> Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
> ---
>  drivers/char/tpm/tpm.h      |  3 +++
>  drivers/char/tpm/tpm2-cmd.c | 48 +++++++++++++++++++++++++++++++++++++
>  2 files changed, 51 insertions(+)
>
> diff --git a/drivers/char/tpm/tpm.h b/drivers/char/tpm/tpm.h
> index 830014a26609..60489f21d3bd 100644
> --- a/drivers/char/tpm/tpm.h
> +++ b/drivers/char/tpm/tpm.h
> @@ -288,6 +288,9 @@ static inline void tpm_add_ppi(struct tpm_chip *chip)
>  int tpm2_get_timeouts(struct tpm_chip *chip);
>  int tpm2_pcr_read(struct tpm_chip *chip, u32 pcr_idx,
>  		  struct tpm_digest *digest, u16 *digest_size_ptr);
> +int tpm2_pcr_get_update_counter(struct tpm_chip *chip,
> +				u32 pcr_idx, u16 alg_id,
> +				u32 *update_counter);

tpm_pcr_read_update_cnt()

>  int tpm2_pcr_extend(struct tpm_chip *chip, u32 pcr_idx,
>  		    struct tpm_digest *digests);
>  int tpm2_get_random(struct tpm_chip *chip, u8 *dest, size_t max);
> diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c
> index 93545be190a5..55f4e102289a 100644
> --- a/drivers/char/tpm/tpm2-cmd.c
> +++ b/drivers/char/tpm/tpm2-cmd.c
> @@ -216,6 +216,54 @@ int tpm2_pcr_read(struct tpm_chip *chip, u32 pcr_idx,
>  	return rc;
>  }
>  
> +/**
> + * tpm2_pcr_get_update_counter() - gets an update counter value for a PCR bank
> + * @chip: TPM chip to use
> + * @pcr_idx: PCR index used to retrieve the update counter
> + * @alg_id:	alg id used to retrieve the update counter
> + * @update_counter: output update counter value
> + *
> + * Return: Same as with tpm_transmit_cmd.
> + */
> +int tpm2_pcr_get_update_counter(struct tpm_chip *chip,
> +				u32 pcr_idx, u16 alg_id, u32 *update_counter)
> +{
> +	int rc;
> +	struct tpm_buf buf;
> +	struct tpm2_pcr_read_out *read_out;
> +	u8 pcr_select[TPM2_PCR_SELECT_MIN] = {0};
> +
> +	if (pcr_idx >= TPM2_PLATFORM_PCR)
> +		return -EINVAL;
> +
> +	if (!update_counter)
> +		return -EINVAL;
> +
> +	rc = tpm_buf_init(&buf, TPM2_ST_NO_SESSIONS, TPM2_CC_PCR_READ);
> +	if (rc)
> +		return rc;
> +
> +	pcr_select[pcr_idx >> 3] = 1 << (pcr_idx & 0x7);
> +
> +	tpm_buf_append_u32(&buf, 1);
> +	tpm_buf_append_u16(&buf, alg_id);
> +	tpm_buf_append_u8(&buf, TPM2_PCR_SELECT_MIN);
> +	tpm_buf_append(&buf, (const unsigned char *)pcr_select,
> +		       sizeof(pcr_select));
> +
> +	rc = tpm_transmit_cmd(chip, &buf, 0, "attempting to read a pcr value");
> +	if (rc)
> +		goto out;
> +
> +	read_out = (struct tpm2_pcr_read_out *)&buf.data[TPM_HEADER_SIZE];
> +
> +	*update_counter = be32_to_cpu(read_out->update_cnt);
> +
> +out:
> +	tpm_buf_destroy(&buf);
> +	return rc;
> +}
> +
>  struct tpm2_null_auth_area {
>  	__be32  handle;
>  	__be16  nonce_size;
> -- 
> 2.25.1

BR, Jarkko


From sushring at linux.microsoft.com  Tue Aug  1 12:12:44 2023
From: sushring at linux.microsoft.com (Sush Shringarputale)
Date: Tue, 1 Aug 2023 12:12:44 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
Message-ID: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>

================================================
| A. Problem Statement???????????????????????? |
================================================
Depending on the IMA policy, the IMA log can consume a lot of Kernel 
memory on
the device.? For instance, the events for the following IMA policy 
entries may
need to be measured in certain scenarios, but they can also lead to a 
verbose
IMA log when the device is running for a long period of time.
?????????????????????????????????????????
?# PROC_SUPER_MAGIC???????????????????? ?
?measure fsmagic=0x9fa0???????????????? ?
?# SYSFS_MAGIC????????????????????????? ?
?measure fsmagic=0x62656572???????????? ?
?# DEBUGFS_MAGIC??????????????????????? ?
?measure fsmagic=0x64626720???????????? ?
?# TMPFS_MAGIC????????????????????????? ?
?measure fsmagic=0x01021994???????????? ?
?# RAMFS_MAGIC????????????????????????? ?
?measure fsmagic=0x858458f6???????????? ?
?# SECURITYFS_MAGIC???????????????????? ?
?measure fsmagic=0x73636673???????????? ?
?# OVERLAYFS_MAGIC????????????????????? ?
?measure fsmagic=0x794c7630???????????? ?
?# log, audit or tmp files????????????? ?
?measure obj_type=var_log_t???????????? ?
?measure obj_type=auditd_log_t????????? ?
?measure obj_type=tmp_t???????????????? ?
?????????????????????????????????????????

Secondly, certain devices are configured to take Kernel updates using Kexec
soft-boot.? The IMA log from the previous Kernel gets carried over and the
Kernel memory consumption problem worsens when such devices undergo multiple
Kexec soft-boots over a long period of time.

The above two scenarios can cause IMA log to grow and consume Kernel memory.

In addition, a large IMA log can add pressure on the network bandwidth when
the attestation client sends it to remote-attestation-service.

Truncating IMA log to reclaim memory is not feasible, since it makes the 
log go
out of sync with the TPM PCR quote making remote attestation fail.

A sophisticated solution is required which will help relieve the memory
pressure on the device and continue supporting remote attestation without
disruptions.

-------------------------------------------------------------------------------
================================================
| B. Proposed Solution???????????????????????? |
================================================
In this document, we propose an enhancement to the IMA subsystem to improve
the long-running performance by snapshotting the IMA log, while still
providing mechanisms to verify its integrity using the PCR quotes.

The remainder of the document describes details of the proposed solution 
in the
following sub-sections.
 ?- High-level Work-flow
 ?- Snapshot Triggering Mechanism
 ?- Design Choices for Storing Snapshots
 ?- Attestation-Client and Remote-Attestation-Service Side Changes
 ?- Example Walk-through
 ?- Open Questions

-------------------------------------------------------------------------------
================================================
| B.1 High-level Work-flow???????????????????? |
================================================
Pre-requisites:
- IMA Integrity guarantees are maintained.

The proposed high level work-flow of IMA log snapshotting is as follows:
- A user-mode process will trigger the snapshot by opening a file in SysFS
 ? say /sys/kernel/security/ima/snapshot (referred to as 
sysk_ima_snapshot_file
 ? here onwards).
- The Kernel will get the current TPM PCR values and PCR update counter [2]
 ? and store them as template data in a new IMA event "snapshot_aggregate".
 ? This event will be measured by IMA using critical data measurement
 ? functionality [1].? Recording regular IMA events will be paused while
 ? "snapshot_aggregate" is being computed using the existing IMA mutex lock.
- Once the "snapshot_aggregate" is computed and measured in IMA log, the 
prior
 ? IMA events will be made available in the sysk_ima_snapshot_file.
- The UM process will copy those IMA events from sysk_ima_snapshot_file to a
 ? snapshot file on disk chosen by UM (referred to as UM_snapshot_file here
 ? onwards).? The location, file-system type, access permissions etc. of the
 ? UM_snapshot_file would be controlled by UM process itself.
- Once UM is done copying the IMA events from sysk_ima_snapshot_file to
 ? UM_snapshot_file, it will indicate to the Kernel that the snapshot can be
 ? finalized by triggering a write with any data to the 
sysk_ima_snapshot_file.
 ? UM process cannot prevent the IMA log purge operation after this point.
- The Kernel will truncate the current IMA log and and clear HTable up 
to the
 ? "snapshot_aggregate" marker.
- The Kernel will measure the PCR update counter as part of measuring
 ? snapshot_aggregate, so that it can be used by the remote attestation 
service
 ? for detecting missing events.
- UM can prevent the IMA log purge by closing the sysk_ima_snapshot_file
 ? without performing a write operation on it.? In this case, while the
 ? "snapshot_aggregate" marker may still be in the log, the event can be 
ignored
 ? since the previous entries in the IMA log will not be purged.

Note:
- This work-flow should work when interleaved with Kexec 'load' and 
'execute'
 ? events and should not cause IMA log + snapshot to go out of sync with PCR
 ? quotes. The implementation details are omitted from this document for
 ? brevity.
-------------------------------------------------------------------------------
================================================
| B.2 Snapshot Triggering Mechanism??????????? |
================================================
The IMA subsystem will create a new file: /sys/kernel/security/ima/snapshot
(a.k.a. sysk_ima_snapshot_file). A UM process opening this file will trigger
the snapshot.? The file will be opened exclusively, so only one UM 
process can
trigger the snapshot at a time.
Once the kernel has written the snapshot_aggregate marker to the IMA 
log, the
IMA log prior to that can be read by UM on the same FD.? When UM writes some
data to sysk_ima_snapshot_file, the kernel will finalize the snapshot.? 
If UM
closes sysk_ima_snapshot_file without writing to it, the Kernel will not
purge the IMA log.

If a system administrator requires that only a specific client process 
should
trigger the snapshot, this capability can be set as an SeLinux policy.

-------------------------------------------------------------------------------
================================================
| B.3 Design Choices for Storing Snapshots???? |
================================================
Choosing the snapshot location is handled by the UM process.? The Kernel 
will
wait for the UM process to indicate that the current IMA log has been 
written to
UM_snapshot_file and then the Kernel will truncate the IMA log. UM is
responsible to clear any existing UM_snapshot_file(s) on system start or 
on a
hard reboot/power cycle.

-------------------------------------------------------------------------------
======================================================================
| B.4 Attestation-Client and Remote-Attestation-Service Side Changes |
======================================================================
A remote attestation client and service will need to be aware of the
snapshot_aggregate marker and how it should be handled.? A typical 
attestation
path on the device will remain the same - send the current IMA log along 
with
the signed TPM PCR quotes to the remote attestation-service.? But the
attestation clients and remote services need to be aware that they need 
to use
UM_snapshot_file(s) too, along with PCR quotes and the data in
"snapshot_aggregate" event in IMA log to re-establish the chain of trust if
the feature is enabled and a snapshot is taken.

-------------------------------------------------------------------------------
================================================
| B.4.1 Attestation-Client side changes??????? |
================================================
To use snapshot feature, the attestation client is required to know
 ??? - if snapshotting is supported/enabled by the Kernel
 ??? - if snapshotting is supported/enabled by the 
remote-attestation-service
 ????? it interacts with
 ??? - the correct order of UM_snapshot_file(s) for backtracking through the
 ????? IMA log

The protocol between the attestation-client and remote-attestation-service
needs to be updated to send previous snapshots to the service as requested.

For service transition, clients connected to a service unaware of
snapshot_aggregate should not write any data to the sysk_ima_snapshot_file
before closing it.? This will add the snapshot_aggregate marker to the IMA
log without purging the log.? Once the service implements snapshot_aggregate
parsing, the client can implement the functionality to? write to the
sysk_ima_snapshot_file each time it triggers the snapshot.? This would 
indicate
to the Kernel that the log should be purged.

-------------------------------------------------------------------------------
================================================
| B.4.2 Remote-Attestation-Service side changes|
================================================
For remote attestation to work, the service will need to know how to 
validate
the snapshot_aggregate entry in the IMA log.? It will have to read the PCR
values present in the template data of snapshot_aggregate event in the 
latest
IMA log, and ensure that the PCR quotes align with the contents of the past
UM_snapshot_file(s).? This will re-establish the chain of trust needed 
for the
device to pass remote attestation.? This will also maintain the ability of
the remote-attestation-service to seal the secrets, if the client-server use
TPM unseal mechanism to attest the state of the device.

The client-service protocol will need an implementation for requesting
previous snapshots of the IMA logs.? There may be various scenarios when
such a request is made such as:
 ??? - the service requires all the data since boot based on admin decisions
 ??? - linked IMA events are split across a snapshot boundary
 ??? - any other protocol level decision

The service will need to request the client to send the old 
UM_snapshot_file(s)
and ensure that the log replay still generates the expected PCR values 
provided
in the quote.

To avoid asking for previous snapshot chunks from the device, the service
may maintain the past attestation status of the device at a given snapshot
checkpoint.? This would require the service to be stateful, but it will
relieve the storage pressure on the device as well as the network bandwidth
once the attestation-client starts taking snapshots.
-------------------------------------------------------------------------------
================================================
| B.5 Example Walk-through???????????????????? |
================================================
For illustration, we provide an example of the snapshot scenario. Assume
the below is the IMA log on a system before a snapshot is taken.
???????????????????????????????????????????????????????????????????????????
?PCR?Template Hash?Template??? Data Hash?? ???? Data File ?TemplateData?
???????????????????????????????????????????????????????????????????????????
?10 ?322a847385...?ima-sig ?sha256:309c... ?boot_aggregate ???????????? ?
???????????????????????????????????????????????????????????????????????????
?10 ?92dbf55061...?ima-buf ?sha256:b93e... ?kernel_version...?352342347...?
???????????????????????????????????????????????????????????????????????????
?11 ?e8e12d9532...?ima-buf ?sha256:cd00... ?dm_table_load....?568956899...?
???????????????????????????????????????????????????????????????????????????
?12 ?e8e12d9532...?ima-sig ?sha256:5601... ?/usr/lib/modul...???????????? ?
???????????????????????????????????????????????????????????????????????????

For this example, the PCR values in sha256 bank are as follows:

 ??? sha256:
 ??????? 0 : 
0xDA009CB9DDBF2DF2FD5982A8E44492718D3CAE516A99847262790479368F82B6
 ??????? ...
 ??????? ...
 ??????? 10: 
0xB93EBF68FC66C6B6F5E2D0E34E113D84303BC0D5DA0B419F059DE27EAE3BAA29
 ??????? 11: 
0xCD00ABB3D84DB0F0419F2064ECD8AD4C01A1ADCDCADB15DEED47BA6FCE40D420
 ??????? 12: 
0x560153FB6A0CC6033475994ECA726765892BB48772682F692E44A0393281DB45
 ??????? ...
 ??????? ...
 ??????? 23: 
0x0000000000000000000000000000000000000000000000000000000000000000

If a snapshot is taken at this point, the current IMA log will be written to
disk.? No additional events will be measured in the log until the 
snapshot is
complete.

The state of the IMA log after the snapshot:
???????????????????????????????????????????????????????????????????????????????
?PCR?Template ?Template? Data Hash?? ????? Data File??? ? Template????? ?
??? ? Hash??? ???????? ????????????? ????? Data File??? ? Data??????? ?
???????????????????????????????????????????????????????????????????????????????
?10 ?e55cba...?ima-buf ?Sha256:30a...?snapshot_aggregate? 
<TEMPLATE_DATA>?? ?
??? ????????? ???????? ????????????? ?????????????????? ??? see the 
grammar?? ?
??? ????????? ???????? ????????????? ?????????????????? ? below????????? ?
???????????????????????????????????????????????????????????????????????????????
 ?where the Data Hash of the log line is -
 ?Append(H(snapshot_file),PCR0,PCR1,...PCR23).? All available PCR banks 
will be
 ?included in the template data in a known structure.

The Template Data will follow the grammar below:
 ??? TEMPLATE_DATA := 
<PCR_Update_Counter>";"<Snapshot_Counter>";"<PCR_Banks>";"
 ??? PCR_Update_Counter := "PCR_Counter="<PCR update counter value from TPM>
 ??? Snapshot_Counter := "Snapshot_Attempt_Count=" <num. of snapshots 
attempted>
 ??? PCR_Banks????????? := <PCR_Bank>";"|<PCR_Banks> "," <PCR_Bank>";"
 ??? PCR_Bank?????????? := <PCR_Hash Type> ":" <PCRn>
 ??? PCR_Hash_Type????? := "sha1"|"sha256"|"sha384"
 ??? PCRn?????????????? := "PCR"<N>":"<PCR_Hash>
 ??? N????????????????? := [0-23]
 ??? PCR_Hash?????????? := <hash_of_the_pcr_N>

The state of the PCRs after the snapshot:
 ??? (Only PCR 10 will change, since snapshot_aggregate is being extended
 ???? in that PCR.)
 ??? sha256:
 ??????? 0 : 
0xDA009CB9DDBF2DF2FD5982A8E44492718D3CAE516A99847262790479368F82B6
 ??????? ...
 ??????? ...
 ??????? 10 : 
0x30A1B69F09A9599ED45FFEA5C94000852518394A8E1A3B8D343D4458E6FB0B04
 ??????? 11 : 
0xCD00ABB3D84DB0F0419F2064ECD8AD4C01A1ADCDCADB15DEED47BA6FCE40D420
 ??????? 12 : 
0x560153FB6A0CC6033475994ECA726765892BB48772682F692E44A0393281DB45
 ??????? ...
 ??????? ...
 ??????? 23: 
0x0000000000000000000000000000000000000000000000000000000000000000

An example of "snapshot_aggregate" template data is given below.
10 e55cba... ima-buf? Sha256:30a... snapshot_aggregate
 ??? PCR_Counter=11;
 ??? Snapshot_Attempt_Count=7;
sha256:PCR0:0xDA009CB9DDBF2DF2FD5982A8E44492718D3CAE516A99847262790479368F82B6,
sha256:PCR1:0x30A1B69F09A9599ED45FFEA5C94000852518394A8E1A3B8D343D4458E6FB0B04,
 ??? ...
sha256:PCR23:0x30A1B69F09A9599ED45FFEA5C94000852518394A8E1A3B8D343D4458E6FB0B04;

Future logs can then continue to extend the assigned PCRs.? Since IMA
measurements extend the TPM-PCRs and computing snapshot_aggregate involves
reading TPM PCR banks, IMA measurements must be suspended until the
snapshot_aggregate is computed and measured.? Otherwise, these two 
operations
may interfere with each other compromising the integrity of the system.

The remote-attestation-service can verify the contents of the past (N-1)
UM_snapshot_file(s) by replaying the events in them and comparing them
with the PCR values stored in template data of the first 
"snapshot_aggregate"
event of subsequent IMA log.

-------------------------------------------------------------------------------
================================================
| C. Open Questions??????????????????????????? |
================================================
1. There is a known issue with PCRs being reset after hibernate [3].? Is 
there
 ? any proposed solution for the hibernate problem? Can we reuse parts 
of it for
 ? IMA log snapshotting?

-------------------------------------------------------------------------------
We appreciate your comments on this proposal as well as pointers to any
prior efforts made in addressing this issue.

Thanks!
Sush & Tushar

-------------------------------------------------------------------------------
================================================
| D. References??????????????????????????????? |
================================================
[1] IMA: support for measuring kernel integrity critical data
https://patchwork.kernel.org/project/linux-integrity/cover/20210108040708.8389-1-tusharsu at linux.microsoft.com/

[2] Measuring TPM update counter in IMA
https://patchwork.kernel.org/project/linux-integrity/cover/20230801181917.8535-1-tusharsu at linux.microsoft.com/

[3] TPM: Hibernate with IMA PCR 10
https://lore.kernel.org/linux-integrity/2bc656bf67af52e0b9a68e91c5b574e0ab4ffa8e.camel at linux.ibm.com/



From tusharsu at linux.microsoft.com  Tue Aug  1 14:01:12 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Tue, 1 Aug 2023 14:01:12 -0700
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
Message-ID: <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>

Thanks for the response Jarkko.

On 8/1/23 12:02, Jarkko Sakkinen wrote:
> The short summary is cryptic to say the least.
Do you mean the patch subject line, or the description below?
> "update counter" does not map it to have anything to do with PCRs.
Agreed.? I noticed that when I was testing the patches.
The update counter is same for all PCRs.? It was also the same for
the two hash algo's I tested it for (SHA1 and SHA256). But the spec
description and Kernel implementation requires to pass the
pcr_idx and hash algo to PCR_Read command to get the update counter.
> Why not "tpm: Read pcrUpdateCounter field from TPM2_PCR_Read"?
As I said in the patch description below, update counter is only
needed for IMA measurements.? None of the other code that calls
tpm2_pcr_read() use the update counter.

I was not sure if you were okay changing the function signature and
implementation of tpm2_pcr_read(). It felt disruptive.

But I can update tpm2_pcr_read() if you are ok with it.
Please let me know.

I also have a few more thoughts on this in the comment below.
> On Tue Aug 1, 2023 at 9:19 PM EEST, Tushar Sugandhi wrote:
>> The TPM2_PCR_Read command returns TPM2_PCR_Read Response struct[1].  It
>> contains pcrUpdateCounter member which contains the current value of TPM
>> PCR update counter.  The update counter provides the number of times the
>> PCRs are updated, which is essential for tracking changes and verifying
>> system integrity.  Thus, subsystems (like IMA) should measure
>> pcrUpdateCounter value.  Although tpm2_pcr_read_out struct is returned
>> by tpm2_pcr_read(), it is not used by it's caller function tpm_pcr_read().
>> Further, TPM2_PCR_Read Response struct and pcrUpdateCounter is not
>> available in tpm1_pcr_read().
>>
>> PcrUpdateCounter is only needed in a specific case (IMA for measurements).
>> Changing tpm_pcr_read() and tpm2_pcr_read() function signature to return
>> tpm2_pcr_read_out struct would be a more disruptive change, since these
>> functions are used elsewhere too.  Creating separate functions to get
>> pcrUpdateCounter when needed would be a cleaner approach.
>>
>> Add a function, 'tpm2_pcr_get_update_counter()' to retrieve
>> the update counter for a given PCR index and algorithm ID on a TPM2 chip.
>>
>> This function complements existing TPM functionalities such as reading
>> and extending PCRs, and enhances the ability to monitor PCR status
>> in the Linux Kernel.
>>
>> [1] https://trustedcomputinggroup.org/wp-content/uploads/TCG_TPM2_r1p59_Part3_Commands_pub.pdf
>> Section 22.4.2, Page 206.
>>
>> Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
>> ---
>>   drivers/char/tpm/tpm.h      |  3 +++
>>   drivers/char/tpm/tpm2-cmd.c | 48 +++++++++++++++++++++++++++++++++++++
>>   2 files changed, 51 insertions(+)
>>
>> diff --git a/drivers/char/tpm/tpm.h b/drivers/char/tpm/tpm.h
>> index 830014a26609..60489f21d3bd 100644
>> --- a/drivers/char/tpm/tpm.h
>> +++ b/drivers/char/tpm/tpm.h
>> @@ -288,6 +288,9 @@ static inline void tpm_add_ppi(struct tpm_chip *chip)
>>   int tpm2_get_timeouts(struct tpm_chip *chip);
>>   int tpm2_pcr_read(struct tpm_chip *chip, u32 pcr_idx,
>>   		  struct tpm_digest *digest, u16 *digest_size_ptr);
>> +int tpm2_pcr_get_update_counter(struct tpm_chip *chip,
>> +				u32 pcr_idx, u16 alg_id,
>> +				u32 *update_counter);
> tpm_pcr_read_update_cnt()
I can rename 'get' -> 'read'

About 'tpm2' -> 'tpm':
I already have tpm_pcr_get_update_counter() defined in patch 2.

I was following the existing pattern here in patch 1 and 2 i. e.
 ?- Implementing the tpm1/tpm2 specific functionality in
 ?? drivers/char/tpm/tpm.h, drivers/char/tpm/tpm2-cmd.c,
 ?? drivers/char/tpm/tpm1-cmd.c.

 ?- And combining that functionality in drivers/char/tpm/tpm-interface.c
 ?? and exposing it to other subsystems (like IMA) through 
include/linux/tpm.h
 ?? (patch 2 of this series)

BTW, if I understand correctly, the update counter is not available in 
TPM 1.2.

Please let me know if you want me to expose the functionality directly
from drivers/char/tpm/tpm2-cmd.c and getting rid of patch #2 of this series.

~Tushar

>>   int tpm2_pcr_extend(struct tpm_chip *chip, u32 pcr_idx,
>>   		    struct tpm_digest *digests);
>>   int tpm2_get_random(struct tpm_chip *chip, u8 *dest, size_t max);
>> diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c
>> index 93545be190a5..55f4e102289a 100644
>> --- a/drivers/char/tpm/tpm2-cmd.c
>> +++ b/drivers/char/tpm/tpm2-cmd.c
>> @@ -216,6 +216,54 @@ int tpm2_pcr_read(struct tpm_chip *chip, u32 pcr_idx,
>>   	return rc;
>>   }
>>   
>> +/**
>> + * tpm2_pcr_get_update_counter() - gets an update counter value for a PCR bank
>> + * @chip: TPM chip to use
>> + * @pcr_idx: PCR index used to retrieve the update counter
>> + * @alg_id:	alg id used to retrieve the update counter
>> + * @update_counter: output update counter value
>> + *
>> + * Return: Same as with tpm_transmit_cmd.
>> + */
>> +int tpm2_pcr_get_update_counter(struct tpm_chip *chip,
>> +				u32 pcr_idx, u16 alg_id, u32 *update_counter)
>> +{
>> +	int rc;
>> +	struct tpm_buf buf;
>> +	struct tpm2_pcr_read_out *read_out;
>> +	u8 pcr_select[TPM2_PCR_SELECT_MIN] = {0};
>> +
>> +	if (pcr_idx >= TPM2_PLATFORM_PCR)
>> +		return -EINVAL;
>> +
>> +	if (!update_counter)
>> +		return -EINVAL;
>> +
>> +	rc = tpm_buf_init(&buf, TPM2_ST_NO_SESSIONS, TPM2_CC_PCR_READ);
>> +	if (rc)
>> +		return rc;
>> +
>> +	pcr_select[pcr_idx >> 3] = 1 << (pcr_idx & 0x7);
>> +
>> +	tpm_buf_append_u32(&buf, 1);
>> +	tpm_buf_append_u16(&buf, alg_id);
>> +	tpm_buf_append_u8(&buf, TPM2_PCR_SELECT_MIN);
>> +	tpm_buf_append(&buf, (const unsigned char *)pcr_select,
>> +		       sizeof(pcr_select));
>> +
>> +	rc = tpm_transmit_cmd(chip, &buf, 0, "attempting to read a pcr value");
>> +	if (rc)
>> +		goto out;
>> +
>> +	read_out = (struct tpm2_pcr_read_out *)&buf.data[TPM_HEADER_SIZE];
>> +
>> +	*update_counter = be32_to_cpu(read_out->update_cnt);
>> +
>> +out:
>> +	tpm_buf_destroy(&buf);
>> +	return rc;
>> +}
>> +
>>   struct tpm2_null_auth_area {
>>   	__be32  handle;
>>   	__be16  nonce_size;
>> -- 
>> 2.25.1
> BR, Jarkko


From James.Bottomley at HansenPartnership.com  Tue Aug  1 14:21:55 2023
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Tue, 01 Aug 2023 17:21:55 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
Message-ID: <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>

On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
[...]
> Truncating IMA log to reclaim memory is not feasible, since it makes
> the log go out of sync with the TPM PCR quote making remote
> attestation fail.

This assumption isn't entirely true.  It's perfectly possible to shard
an IMA log using two TPM2_Quote's for the beginning and end PCR values
to validate the shard.  The IMA log could be truncated in the same way
(replace the removed part of the log with a TPM2_Quote and AK, so the
log still validates from the beginning quote to the end).

If you use a TPM2_Quote mechanism to save the log, all you need to do
is have the kernel generate the quote with an internal AK.  You can
keep a record of the quote and the AK at the beginning of the truncated
kernel log.  If the truncated entries are saved in a file shard it
should have a beginning and end quote and a record of the AK used. 
Since verifiers like Keylime are already using this beginning and end
quote for sharded logs, it's the most natural format to feed to
something externally for verification and it means you don't have to
invent a new format to do the same thing.

Regards,

James



From jarkko at kernel.org  Tue Aug  1 20:58:18 2023
From: jarkko at kernel.org (Jarkko Sakkinen)
Date: Wed, 02 Aug 2023 06:58:18 +0300
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
 <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
Message-ID: <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>

On Wed Aug 2, 2023 at 12:01 AM EEST, Tushar Sugandhi wrote:
> Thanks for the response Jarkko.
>
> On 8/1/23 12:02, Jarkko Sakkinen wrote:
> > The short summary is cryptic to say the least.
> Do you mean the patch subject line, or the description below?

It is in the process documentation:

https://www.kernel.org/doc/html/v6.3/process/submitting-patches.html#the-canonical-patch-format

> > "update counter" does not map it to have anything to do with PCRs.
> Agreed.? I noticed that when I was testing the patches.
> The update counter is same for all PCRs.? It was also the same for
> the two hash algo's I tested it for (SHA1 and SHA256). But the spec
> description and Kernel implementation requires to pass the
> pcr_idx and hash algo to PCR_Read command to get the update counter.

I was referring to the fact that TPM2_PCR_Read does not have a field
called "update counter" in its response but it has a field called
"pcrUpdateCounter". Please refer to thigs that actually exist.

In the long description you are in some occasions referring to the same
object as:

1. "update counter"
2. "pcrUpdateCounter"
3. "PcrUpdateCounter"

This is ambiguous and wrong.

>From long description I see zero motivation to ack this change, except
some heresay about IMA requiring it. Why does IMA need update_cnt and
why this is not documented to the long description?

> But I can update tpm2_pcr_read() if you are ok with it.
> Please let me know.

You can add "u32 *update_cnt".

BR, Jarkko


From ltao at redhat.com  Wed Aug  2 01:22:54 2023
From: ltao at redhat.com (Tao Liu)
Date: Wed, 2 Aug 2023 16:22:54 +0800
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local>
References: <20230601072043.24439-1-ltao@redhat.com> <20230713100459.GEZK/MS69XbphJa+tN@fat_crate.local>
 <CAO7dBbVMNKTSDi5eP4BseEUexsk0Mo0GWJpyHfOcp+tHs6cSUw@mail.gmail.com>
 <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local> <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local>
Message-ID: <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>

Hi Borislav,

On Sat, Jul 29, 2023 at 12:56?AM Borislav Petkov <bp at alien8.de> wrote:
>
> On Thu, Jul 27, 2023 at 07:03:26PM +0800, Tao Liu wrote:
> > Hi Borislav,
> >
> > Sorry for the late response. I spent some time retesting your patch
> > against 6.5.0-rc1 and 6.5.0-rc3, and it is OK. So
> >
> > Reported-and-tested-by: Tao Liu <ltao at redhat.com>
> >
> > And will we use this patch as a workaround or will we wait for a
> > better solution as proposed by Michael?
>
> First of all, please do not top-post.
>

OK, thanks for the reminder.

> And yes, here's a better one. I'd appreciate it you testing it.
>

Thanks for the patch! I have tested it on the lenovo machine in the
past few days, no issue found, so the patch tests OK.

Thanks,
Tao Liu

> Thx.
>
> ---
>  arch/x86/boot/compressed/idt_64.c |  5 ++++-
>  arch/x86/boot/compressed/sev.c    | 37 +++++++++++++++++++++++++++++--
>  2 files changed, 39 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/boot/compressed/idt_64.c b/arch/x86/boot/compressed/idt_64.c
> index 6debb816e83d..0f03ac12e2a6 100644
> --- a/arch/x86/boot/compressed/idt_64.c
> +++ b/arch/x86/boot/compressed/idt_64.c
> @@ -63,7 +63,10 @@ void load_stage2_idt(void)
>         set_idt_entry(X86_TRAP_PF, boot_page_fault);
>
>  #ifdef CONFIG_AMD_MEM_ENCRYPT
> -       set_idt_entry(X86_TRAP_VC, boot_stage2_vc);
> +       if (sev_status & BIT(1))
> +               set_idt_entry(X86_TRAP_VC, boot_stage2_vc);
> +       else
> +               set_idt_entry(X86_TRAP_VC, NULL);
>  #endif
>
>         load_boot_idt(&boot_idt_desc);
> diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
> index 09dc8c187b3c..c3e343bd4760 100644
> --- a/arch/x86/boot/compressed/sev.c
> +++ b/arch/x86/boot/compressed/sev.c
> @@ -404,13 +404,46 @@ void sev_enable(struct boot_params *bp)
>         if (bp)
>                 bp->cc_blob_address = 0;
>
> +       /*
> +        * Do an initial SEV capability check before snp_init() which
> +        * loads the CPUID page and the same checks afterwards are done
> +        * without the hypervisor and are trustworthy.
> +        *
> +        * If the HV fakes SEV support, the guest will crash'n'burn
> +        * which is good enough.
> +        */
> +
> +       /* Check for the SME/SEV support leaf */
> +       eax = 0x80000000;
> +       ecx = 0;
> +       native_cpuid(&eax, &ebx, &ecx, &edx);
> +       if (eax < 0x8000001f)
> +               return;
> +
> +       /*
> +        * Check for the SME/SEV feature:
> +        *   CPUID Fn8000_001F[EAX]
> +        *   - Bit 0 - Secure Memory Encryption support
> +        *   - Bit 1 - Secure Encrypted Virtualization support
> +        *   CPUID Fn8000_001F[EBX]
> +        *   - Bits 5:0 - Pagetable bit position used to indicate encryption
> +        */
> +       eax = 0x8000001f;
> +       ecx = 0;
> +       native_cpuid(&eax, &ebx, &ecx, &edx);
> +       /* Check whether SEV is supported */
> +       if (!(eax & BIT(1)))
> +               return;
> +
>         /*
>          * Setup/preliminary detection of SNP. This will be sanity-checked
>          * against CPUID/MSR values later.
>          */
>         snp = snp_init(bp);
>
> -       /* Check for the SME/SEV support leaf */
> +       /* Now repeat the checks with the SNP CPUID table. */
> +
> +       /* Recheck the SME/SEV support leaf */
>         eax = 0x80000000;
>         ecx = 0;
>         native_cpuid(&eax, &ebx, &ecx, &edx);
> @@ -418,7 +451,7 @@ void sev_enable(struct boot_params *bp)
>                 return;
>
>         /*
> -        * Check for the SME/SEV feature:
> +        * Recheck for the SME/SEV feature:
>          *   CPUID Fn8000_001F[EAX]
>          *   - Bit 0 - Secure Memory Encryption support
>          *   - Bit 1 - Secure Encrypted Virtualization support
> --
> 2.41.0
>
> --
> Regards/Gruss,
>     Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette
>



From bp at alien8.de  Wed Aug  2 02:39:27 2023
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 2 Aug 2023 11:39:27 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
References: <20230601072043.24439-1-ltao@redhat.com>
 <20230713100459.GEZK/MS69XbphJa+tN@fat_crate.local>
 <CAO7dBbVMNKTSDi5eP4BseEUexsk0Mo0GWJpyHfOcp+tHs6cSUw@mail.gmail.com>
 <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local>
 <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local>
 <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
Message-ID: <20230802093927.GAZMokT57anC5jBISK@fat_crate.local>

On Wed, Aug 02, 2023 at 04:22:54PM +0800, Tao Liu wrote:
> Thanks for the patch! I have tested it on the lenovo machine in the
> past few days, no issue found, so the patch tests OK.

Thanks for testing!

Mike, Tom, the below ok this way?

---
From: "Borislav Petkov (AMD)" <bp at alien8.de>
Date: Sun, 16 Jul 2023 20:22:20 +0200
Subject: [PATCH] x86/sev: Do not try to parse for the CC blob on non-AMD
 hardware

Tao Liu reported a boot hang on an Intel Atom machine due to an unmapped
EFI config table. The reason being that the CC blob which contains the
CPUID page for AMD SNP guests is parsed for before even checking
whether the machine runs on AMD hardware.

Usually that's not a problem on !AMD hw - it simply won't find the CC
blob's GUID and return. However, if any parts of the config table
pointers array is not mapped, the kernel will #PF very early in the
decompressor stage without any opportunity to recover.

Therefore, do a superficial CPUID check before poking for the CC blob.
This will fix the current issue on real hardware. It would also work as
a guest on a non-lying hypervisor.

For the lying hypervisor, the check is done again, *after* parsing the
CC blob as the real CPUID page will be present then.

Clear the #VC handler in case SEV-{ES,SNP} hasn't been detected, as
a precaution.

Fixes: c01fce9cef84 ("x86/compressed: Add SEV-SNP feature detection/setup")
Reported-by: Tao Liu <ltao at redhat.com>
Signed-off-by: Borislav Petkov (AMD) <bp at alien8.de>
Tested-by: Tao Liu <ltao at redhat.com>
Cc: <stable at kernel.org>
Link: https://lore.kernel.org/r/20230601072043.24439-1-ltao at redhat.com
---
 arch/x86/boot/compressed/idt_64.c |  9 +++++++-
 arch/x86/boot/compressed/sev.c    | 37 +++++++++++++++++++++++++++++--
 2 files changed, 43 insertions(+), 3 deletions(-)

diff --git a/arch/x86/boot/compressed/idt_64.c b/arch/x86/boot/compressed/idt_64.c
index 6debb816e83d..3cdf94b41456 100644
--- a/arch/x86/boot/compressed/idt_64.c
+++ b/arch/x86/boot/compressed/idt_64.c
@@ -63,7 +63,14 @@ void load_stage2_idt(void)
 	set_idt_entry(X86_TRAP_PF, boot_page_fault);
 
 #ifdef CONFIG_AMD_MEM_ENCRYPT
-	set_idt_entry(X86_TRAP_VC, boot_stage2_vc);
+	/*
+	 * Clear the second stage #VC handler in case guest types
+	 * needing #VC have not been detected.
+	 */
+	if (sev_status & BIT(1))
+		set_idt_entry(X86_TRAP_VC, boot_stage2_vc);
+	else
+		set_idt_entry(X86_TRAP_VC, NULL);
 #endif
 
 	load_boot_idt(&boot_idt_desc);
diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
index 09dc8c187b3c..c3e343bd4760 100644
--- a/arch/x86/boot/compressed/sev.c
+++ b/arch/x86/boot/compressed/sev.c
@@ -404,13 +404,46 @@ void sev_enable(struct boot_params *bp)
 	if (bp)
 		bp->cc_blob_address = 0;
 
+	/*
+	 * Do an initial SEV capability check before snp_init() which
+	 * loads the CPUID page and the same checks afterwards are done
+	 * without the hypervisor and are trustworthy.
+	 *
+	 * If the HV fakes SEV support, the guest will crash'n'burn
+	 * which is good enough.
+	 */
+
+	/* Check for the SME/SEV support leaf */
+	eax = 0x80000000;
+	ecx = 0;
+	native_cpuid(&eax, &ebx, &ecx, &edx);
+	if (eax < 0x8000001f)
+		return;
+
+	/*
+	 * Check for the SME/SEV feature:
+	 *   CPUID Fn8000_001F[EAX]
+	 *   - Bit 0 - Secure Memory Encryption support
+	 *   - Bit 1 - Secure Encrypted Virtualization support
+	 *   CPUID Fn8000_001F[EBX]
+	 *   - Bits 5:0 - Pagetable bit position used to indicate encryption
+	 */
+	eax = 0x8000001f;
+	ecx = 0;
+	native_cpuid(&eax, &ebx, &ecx, &edx);
+	/* Check whether SEV is supported */
+	if (!(eax & BIT(1)))
+		return;
+
 	/*
 	 * Setup/preliminary detection of SNP. This will be sanity-checked
 	 * against CPUID/MSR values later.
 	 */
 	snp = snp_init(bp);
 
-	/* Check for the SME/SEV support leaf */
+	/* Now repeat the checks with the SNP CPUID table. */
+
+	/* Recheck the SME/SEV support leaf */
 	eax = 0x80000000;
 	ecx = 0;
 	native_cpuid(&eax, &ebx, &ecx, &edx);
@@ -418,7 +451,7 @@ void sev_enable(struct boot_params *bp)
 		return;
 
 	/*
-	 * Check for the SME/SEV feature:
+	 * Recheck for the SME/SEV feature:
 	 *   CPUID Fn8000_001F[EAX]
 	 *   - Bit 0 - Secure Memory Encryption support
 	 *   - Bit 1 - Secure Encrypted Virtualization support
-- 
2.41.0

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette


From piliu at redhat.com  Wed Aug  2 02:53:59 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Wed, 2 Aug 2023 17:53:59 +0800
Subject: [PATCHv6 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <ZMitmSZPqnH+/70s@kernel.org>
References: <20230724022142.6425-1-piliu@redhat.com> <20230724022142.6425-4-piliu@redhat.com>
 <ZMitmSZPqnH+/70s@kernel.org>
Message-ID: <CAF+s44QBMgsT7Q2ro9obVvHYCrKYTQoeCnOVOMzhfexQS+_6Sw@mail.gmail.com>

Hi Simon,

Thanks for the try. Please see the comment below.

On Tue, Aug 1, 2023 at 3:00?PM Simon Horman <horms at kernel.org> wrote:
>
> On Mon, Jul 24, 2023 at 10:21:40AM +0800, Pingfan Liu wrote:
> > From: Jeremy Linton <jeremy.linton at arm.com>
> >
> > The linux kernel CONFIG_ZBOOT option creates
> > self decompressing PE kernel images. So this means
> > that kexec should have a generic understanding of
> > the format which may be used by multiple arches.
> >
> > So lets add an arch independent validation
> > and decompression routine.
> >
> > Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> > [Modified by Pingfan to export kernel fd]
> > Signed-off-by: Pingfan Liu <piliu at redhat.com>
>
> Hi Pingfan,
>
> unfortunately this causes a build failure on hppa.
>
> ../../kexec/kexec-pe-zboot.c:31:10: fatal error: kexec-pe-zboot.h: No such file or directory
>    31 | #include <kexec-pe-zboot.h>
>       |          ^~~~~~~~~~~~~~~~~~
>
> Link: https://github.com/horms/kexec-tools/actions/runs/5723580523/job/15508425790
>

It is not related to cross-compiling. Actually, I have tried to
simplify the test matrix, which limits the compilation only on x86_64.
And I got the similar error [1]

The workflow control file is [2], which clips out all arches except
x86_64.  But I can successfully build it on the Fedora system with the
following bash script, which is based on the github's build log.  So
maybe it is a bug with the compiling tools?

kexec_tools_dir="./"

mkdir $kexec_tools_dir/_build \
         $kexec_tools_dir/_build/sub \
         $kexec_tools_dir/_inst \
         $kexec_tools_dir/_dest
chmod a-w $kexec_tools_dir
test -d $kexec_tools_dir/_build
INSTALL_BASE=$(cd $kexec_tools_dir/_inst && pwd | sed -e
's,^[^:\\/]:[\\/],/,') &&\
        DESTDIR="$kexec_tools_dir/_dest" && \
        cd $kexec_tools_dir/_build/sub && \
                ../../configure \
                 \
                --srcdir=../.. --prefix="$INSTALL_BASE" && \
        make  -j8


[1]: https://github.com/pfliu/kexec-tools/actions/runs/5737254109/job/15548520863
[2]: https://github.com/pfliu/kexec-tools/blob/zbootV6/.github/workflows/main.yml


Thanks,

Pingfan



From horms at kernel.org  Wed Aug  2 05:16:33 2023
From: horms at kernel.org (Simon Horman)
Date: Wed, 2 Aug 2023 14:16:33 +0200
Subject: [PATCHv6 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <CAF+s44QBMgsT7Q2ro9obVvHYCrKYTQoeCnOVOMzhfexQS+_6Sw@mail.gmail.com>
References: <20230724022142.6425-1-piliu@redhat.com>
 <20230724022142.6425-4-piliu@redhat.com>
 <ZMitmSZPqnH+/70s@kernel.org>
 <CAF+s44QBMgsT7Q2ro9obVvHYCrKYTQoeCnOVOMzhfexQS+_6Sw@mail.gmail.com>
Message-ID: <ZMpJIZ18ynQLln8W@kernel.org>

On Wed, Aug 02, 2023 at 05:53:59PM +0800, Pingfan Liu wrote:
> Hi Simon,
> 
> Thanks for the try. Please see the comment below.
> 
> On Tue, Aug 1, 2023 at 3:00?PM Simon Horman <horms at kernel.org> wrote:
> >
> > On Mon, Jul 24, 2023 at 10:21:40AM +0800, Pingfan Liu wrote:
> > > From: Jeremy Linton <jeremy.linton at arm.com>
> > >
> > > The linux kernel CONFIG_ZBOOT option creates
> > > self decompressing PE kernel images. So this means
> > > that kexec should have a generic understanding of
> > > the format which may be used by multiple arches.
> > >
> > > So lets add an arch independent validation
> > > and decompression routine.
> > >
> > > Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> > > [Modified by Pingfan to export kernel fd]
> > > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> >
> > Hi Pingfan,
> >
> > unfortunately this causes a build failure on hppa.
> >
> > ../../kexec/kexec-pe-zboot.c:31:10: fatal error: kexec-pe-zboot.h: No such file or directory
> >    31 | #include <kexec-pe-zboot.h>
> >       |          ^~~~~~~~~~~~~~~~~~
> >
> > Link: https://github.com/horms/kexec-tools/actions/runs/5723580523/job/15508425790
> >
> 
> It is not related to cross-compiling. Actually, I have tried to
> simplify the test matrix, which limits the compilation only on x86_64.
> And I got the similar error [1]
> 
> The workflow control file is [2], which clips out all arches except
> x86_64.  But I can successfully build it on the Fedora system with the
> following bash script, which is based on the github's build log.  So
> maybe it is a bug with the compiling tools?
> 
> kexec_tools_dir="./"
> 
> mkdir $kexec_tools_dir/_build \
>          $kexec_tools_dir/_build/sub \
>          $kexec_tools_dir/_inst \
>          $kexec_tools_dir/_dest
> chmod a-w $kexec_tools_dir
> test -d $kexec_tools_dir/_build
> INSTALL_BASE=$(cd $kexec_tools_dir/_inst && pwd | sed -e
> 's,^[^:\\/]:[\\/],/,') &&\
>         DESTDIR="$kexec_tools_dir/_dest" && \
>         cd $kexec_tools_dir/_build/sub && \
>                 ../../configure \
>                  \
>                 --srcdir=../.. --prefix="$INSTALL_BASE" && \
>         make  -j8
> 
> 
> [1]: https://github.com/pfliu/kexec-tools/actions/runs/5737254109/job/15548520863
> [2]: https://github.com/pfliu/kexec-tools/blob/zbootV6/.github/workflows/main.yml

Thanks,

I guess that kexec-pe-zboot.h is missing in the build environment for the
GitHub actions, but present in your Fedora environment.

Could you take a look and see where your copy of kexec-pe-zboot.h
came from?


From horms at kernel.org  Wed Aug  2 05:17:54 2023
From: horms at kernel.org (Simon Horman)
Date: Wed, 2 Aug 2023 14:17:54 +0200
Subject: [PATCHv6 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <ZMpJIZ18ynQLln8W@kernel.org>
References: <20230724022142.6425-1-piliu@redhat.com>
 <20230724022142.6425-4-piliu@redhat.com>
 <ZMitmSZPqnH+/70s@kernel.org>
 <CAF+s44QBMgsT7Q2ro9obVvHYCrKYTQoeCnOVOMzhfexQS+_6Sw@mail.gmail.com>
 <ZMpJIZ18ynQLln8W@kernel.org>
Message-ID: <ZMpJcgeGeYhcuGfe@kernel.org>

On Wed, Aug 02, 2023 at 02:16:33PM +0200, Simon Horman wrote:
> On Wed, Aug 02, 2023 at 05:53:59PM +0800, Pingfan Liu wrote:
> > Hi Simon,
> > 
> > Thanks for the try. Please see the comment below.
> > 
> > On Tue, Aug 1, 2023 at 3:00?PM Simon Horman <horms at kernel.org> wrote:
> > >
> > > On Mon, Jul 24, 2023 at 10:21:40AM +0800, Pingfan Liu wrote:
> > > > From: Jeremy Linton <jeremy.linton at arm.com>
> > > >
> > > > The linux kernel CONFIG_ZBOOT option creates
> > > > self decompressing PE kernel images. So this means
> > > > that kexec should have a generic understanding of
> > > > the format which may be used by multiple arches.
> > > >
> > > > So lets add an arch independent validation
> > > > and decompression routine.
> > > >
> > > > Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> > > > [Modified by Pingfan to export kernel fd]
> > > > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > >
> > > Hi Pingfan,
> > >
> > > unfortunately this causes a build failure on hppa.
> > >
> > > ../../kexec/kexec-pe-zboot.c:31:10: fatal error: kexec-pe-zboot.h: No such file or directory
> > >    31 | #include <kexec-pe-zboot.h>
> > >       |          ^~~~~~~~~~~~~~~~~~
> > >
> > > Link: https://github.com/horms/kexec-tools/actions/runs/5723580523/job/15508425790
> > >
> > 
> > It is not related to cross-compiling. Actually, I have tried to
> > simplify the test matrix, which limits the compilation only on x86_64.
> > And I got the similar error [1]
> > 
> > The workflow control file is [2], which clips out all arches except
> > x86_64.  But I can successfully build it on the Fedora system with the
> > following bash script, which is based on the github's build log.  So
> > maybe it is a bug with the compiling tools?
> > 
> > kexec_tools_dir="./"
> > 
> > mkdir $kexec_tools_dir/_build \
> >          $kexec_tools_dir/_build/sub \
> >          $kexec_tools_dir/_inst \
> >          $kexec_tools_dir/_dest
> > chmod a-w $kexec_tools_dir
> > test -d $kexec_tools_dir/_build
> > INSTALL_BASE=$(cd $kexec_tools_dir/_inst && pwd | sed -e
> > 's,^[^:\\/]:[\\/],/,') &&\
> >         DESTDIR="$kexec_tools_dir/_dest" && \
> >         cd $kexec_tools_dir/_build/sub && \
> >                 ../../configure \
> >                  \
> >                 --srcdir=../.. --prefix="$INSTALL_BASE" && \
> >         make  -j8
> > 
> > 
> > [1]: https://github.com/pfliu/kexec-tools/actions/runs/5737254109/job/15548520863
> > [2]: https://github.com/pfliu/kexec-tools/blob/zbootV6/.github/workflows/main.yml
> 
> Thanks,
> 
> I guess that kexec-pe-zboot.h is missing in the build environment for the
> GitHub actions, but present in your Fedora environment.
> 
> Could you take a look and see where your copy of kexec-pe-zboot.h
> came from?

Actually it seems to be added by this patch (sorry for not noticing)!
So I guess it is an include path problem.


From horms at kernel.org  Wed Aug  2 05:33:48 2023
From: horms at kernel.org (Simon Horman)
Date: Wed, 2 Aug 2023 14:33:48 +0200
Subject: [PATCHv6 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <ZMpJcgeGeYhcuGfe@kernel.org>
References: <20230724022142.6425-1-piliu@redhat.com>
 <20230724022142.6425-4-piliu@redhat.com>
 <ZMitmSZPqnH+/70s@kernel.org>
 <CAF+s44QBMgsT7Q2ro9obVvHYCrKYTQoeCnOVOMzhfexQS+_6Sw@mail.gmail.com>
 <ZMpJIZ18ynQLln8W@kernel.org>
 <ZMpJcgeGeYhcuGfe@kernel.org>
Message-ID: <ZMpNLGt6RtxalSBb@kernel.org>

On Wed, Aug 02, 2023 at 02:17:57PM +0200, Simon Horman wrote:
> On Wed, Aug 02, 2023 at 02:16:33PM +0200, Simon Horman wrote:
> > On Wed, Aug 02, 2023 at 05:53:59PM +0800, Pingfan Liu wrote:
> > > Hi Simon,
> > > 
> > > Thanks for the try. Please see the comment below.
> > > 
> > > On Tue, Aug 1, 2023 at 3:00?PM Simon Horman <horms at kernel.org> wrote:
> > > >
> > > > On Mon, Jul 24, 2023 at 10:21:40AM +0800, Pingfan Liu wrote:
> > > > > From: Jeremy Linton <jeremy.linton at arm.com>
> > > > >
> > > > > The linux kernel CONFIG_ZBOOT option creates
> > > > > self decompressing PE kernel images. So this means
> > > > > that kexec should have a generic understanding of
> > > > > the format which may be used by multiple arches.
> > > > >
> > > > > So lets add an arch independent validation
> > > > > and decompression routine.
> > > > >
> > > > > Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> > > > > [Modified by Pingfan to export kernel fd]
> > > > > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > > >
> > > > Hi Pingfan,
> > > >
> > > > unfortunately this causes a build failure on hppa.
> > > >
> > > > ../../kexec/kexec-pe-zboot.c:31:10: fatal error: kexec-pe-zboot.h: No such file or directory
> > > >    31 | #include <kexec-pe-zboot.h>
> > > >       |          ^~~~~~~~~~~~~~~~~~
> > > >
> > > > Link: https://github.com/horms/kexec-tools/actions/runs/5723580523/job/15508425790
> > > >
> > > 
> > > It is not related to cross-compiling. Actually, I have tried to
> > > simplify the test matrix, which limits the compilation only on x86_64.
> > > And I got the similar error [1]
> > > 
> > > The workflow control file is [2], which clips out all arches except
> > > x86_64.  But I can successfully build it on the Fedora system with the
> > > following bash script, which is based on the github's build log.  So
> > > maybe it is a bug with the compiling tools?
> > > 
> > > kexec_tools_dir="./"
> > > 
> > > mkdir $kexec_tools_dir/_build \
> > >          $kexec_tools_dir/_build/sub \
> > >          $kexec_tools_dir/_inst \
> > >          $kexec_tools_dir/_dest
> > > chmod a-w $kexec_tools_dir
> > > test -d $kexec_tools_dir/_build
> > > INSTALL_BASE=$(cd $kexec_tools_dir/_inst && pwd | sed -e
> > > 's,^[^:\\/]:[\\/],/,') &&\
> > >         DESTDIR="$kexec_tools_dir/_dest" && \
> > >         cd $kexec_tools_dir/_build/sub && \
> > >                 ../../configure \
> > >                  \
> > >                 --srcdir=../.. --prefix="$INSTALL_BASE" && \
> > >         make  -j8
> > > 
> > > 
> > > [1]: https://github.com/pfliu/kexec-tools/actions/runs/5737254109/job/15548520863
> > > [2]: https://github.com/pfliu/kexec-tools/blob/zbootV6/.github/workflows/main.yml
> > 
> > Thanks,
> > 
> > I guess that kexec-pe-zboot.h is missing in the build environment for the
> > GitHub actions, but present in your Fedora environment.
> > 
> > Could you take a look and see where your copy of kexec-pe-zboot.h
> > came from?
> 
> Actually it seems to be added by this patch (sorry for not noticing)!
> So I guess it is an include path problem.

I think I have found the problem.
The kexec-tools build system is a bit unusual,
and the new file, kexec-pe-zboot.h, was not included in distribution
tarballs. Thus the build failures.

I think you can resolve that by squashing the following into this patch.

diff --git a/include/Makefile b/include/Makefile
--- a/include/Makefile
+++ b/include/Makefile
@@ -1,6 +1,7 @@
 dist += include/Makefile		\
 	include/config.h		\
 	include/config.h.in		\
+	include/kexec-pe-zboot.h	\
 	include/kexec-uImage.h		\
 	include/x86/x86-linux.h		\
 	include/x86/mb_info.h		\


From thomas.lendacky at amd.com  Wed Aug  2 06:40:36 2023
From: thomas.lendacky at amd.com (Tom Lendacky)
Date: Wed, 2 Aug 2023 08:40:36 -0500
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <20230802093927.GAZMokT57anC5jBISK@fat_crate.local>
References: <20230601072043.24439-1-ltao@redhat.com>
 <20230713100459.GEZK/MS69XbphJa+tN@fat_crate.local>
 <CAO7dBbVMNKTSDi5eP4BseEUexsk0Mo0GWJpyHfOcp+tHs6cSUw@mail.gmail.com>
 <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local>
 <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local>
 <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local>
Message-ID: <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>

On 8/2/23 04:39, Borislav Petkov wrote:
> On Wed, Aug 02, 2023 at 04:22:54PM +0800, Tao Liu wrote:
>> Thanks for the patch! I have tested it on the lenovo machine in the
>> past few days, no issue found, so the patch tests OK.
> 
> Thanks for testing!
> 
> Mike, Tom, the below ok this way?

Short of figuring out how to map page accesses earlier through the 
boot_page_fault IDT routine, this seems reasonable.

Acked-by: Tom Lendacky <thomas.lendacky at amd.com>

> 
> ---
> From: "Borislav Petkov (AMD)" <bp at alien8.de>
> Date: Sun, 16 Jul 2023 20:22:20 +0200
> Subject: [PATCH] x86/sev: Do not try to parse for the CC blob on non-AMD
>   hardware
> 
> Tao Liu reported a boot hang on an Intel Atom machine due to an unmapped
> EFI config table. The reason being that the CC blob which contains the
> CPUID page for AMD SNP guests is parsed for before even checking
> whether the machine runs on AMD hardware.
> 
> Usually that's not a problem on !AMD hw - it simply won't find the CC
> blob's GUID and return. However, if any parts of the config table
> pointers array is not mapped, the kernel will #PF very early in the
> decompressor stage without any opportunity to recover.
> 
> Therefore, do a superficial CPUID check before poking for the CC blob.
> This will fix the current issue on real hardware. It would also work as
> a guest on a non-lying hypervisor.
> 
> For the lying hypervisor, the check is done again, *after* parsing the
> CC blob as the real CPUID page will be present then.
> 
> Clear the #VC handler in case SEV-{ES,SNP} hasn't been detected, as
> a precaution.
> 
> Fixes: c01fce9cef84 ("x86/compressed: Add SEV-SNP feature detection/setup")
> Reported-by: Tao Liu <ltao at redhat.com>
> Signed-off-by: Borislav Petkov (AMD) <bp at alien8.de>
> Tested-by: Tao Liu <ltao at redhat.com>
> Cc: <stable at kernel.org>
> Link: https://lore.kernel.org/r/20230601072043.24439-1-ltao at redhat.com
> ---
>   arch/x86/boot/compressed/idt_64.c |  9 +++++++-
>   arch/x86/boot/compressed/sev.c    | 37 +++++++++++++++++++++++++++++--
>   2 files changed, 43 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/boot/compressed/idt_64.c b/arch/x86/boot/compressed/idt_64.c
> index 6debb816e83d..3cdf94b41456 100644
> --- a/arch/x86/boot/compressed/idt_64.c
> +++ b/arch/x86/boot/compressed/idt_64.c
> @@ -63,7 +63,14 @@ void load_stage2_idt(void)
>   	set_idt_entry(X86_TRAP_PF, boot_page_fault);
>   
>   #ifdef CONFIG_AMD_MEM_ENCRYPT
> -	set_idt_entry(X86_TRAP_VC, boot_stage2_vc);
> +	/*
> +	 * Clear the second stage #VC handler in case guest types
> +	 * needing #VC have not been detected.
> +	 */
> +	if (sev_status & BIT(1))
> +		set_idt_entry(X86_TRAP_VC, boot_stage2_vc);
> +	else
> +		set_idt_entry(X86_TRAP_VC, NULL);
>   #endif
>   
>   	load_boot_idt(&boot_idt_desc);
> diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
> index 09dc8c187b3c..c3e343bd4760 100644
> --- a/arch/x86/boot/compressed/sev.c
> +++ b/arch/x86/boot/compressed/sev.c
> @@ -404,13 +404,46 @@ void sev_enable(struct boot_params *bp)
>   	if (bp)
>   		bp->cc_blob_address = 0;
>   
> +	/*
> +	 * Do an initial SEV capability check before snp_init() which
> +	 * loads the CPUID page and the same checks afterwards are done
> +	 * without the hypervisor and are trustworthy.
> +	 *
> +	 * If the HV fakes SEV support, the guest will crash'n'burn
> +	 * which is good enough.
> +	 */
> +
> +	/* Check for the SME/SEV support leaf */
> +	eax = 0x80000000;
> +	ecx = 0;
> +	native_cpuid(&eax, &ebx, &ecx, &edx);
> +	if (eax < 0x8000001f)
> +		return;
> +
> +	/*
> +	 * Check for the SME/SEV feature:
> +	 *   CPUID Fn8000_001F[EAX]
> +	 *   - Bit 0 - Secure Memory Encryption support
> +	 *   - Bit 1 - Secure Encrypted Virtualization support
> +	 *   CPUID Fn8000_001F[EBX]
> +	 *   - Bits 5:0 - Pagetable bit position used to indicate encryption
> +	 */
> +	eax = 0x8000001f;
> +	ecx = 0;
> +	native_cpuid(&eax, &ebx, &ecx, &edx);
> +	/* Check whether SEV is supported */
> +	if (!(eax & BIT(1)))
> +		return;
> +
>   	/*
>   	 * Setup/preliminary detection of SNP. This will be sanity-checked
>   	 * against CPUID/MSR values later.
>   	 */
>   	snp = snp_init(bp);
>   
> -	/* Check for the SME/SEV support leaf */
> +	/* Now repeat the checks with the SNP CPUID table. */
> +
> +	/* Recheck the SME/SEV support leaf */
>   	eax = 0x80000000;
>   	ecx = 0;
>   	native_cpuid(&eax, &ebx, &ecx, &edx);
> @@ -418,7 +451,7 @@ void sev_enable(struct boot_params *bp)
>   		return;
>   
>   	/*
> -	 * Check for the SME/SEV feature:
> +	 * Recheck for the SME/SEV feature:
>   	 *   CPUID Fn8000_001F[EAX]
>   	 *   - Bit 0 - Secure Memory Encryption support
>   	 *   - Bit 1 - Secure Encrypted Virtualization support


From bp at alien8.de  Wed Aug  2 06:58:56 2023
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 2 Aug 2023 15:58:56 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
References: <20230601072043.24439-1-ltao@redhat.com>
 <20230713100459.GEZK/MS69XbphJa+tN@fat_crate.local>
 <CAO7dBbVMNKTSDi5eP4BseEUexsk0Mo0GWJpyHfOcp+tHs6cSUw@mail.gmail.com>
 <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local>
 <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local>
 <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local>
 <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
Message-ID: <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local>

On Wed, Aug 02, 2023 at 08:40:36AM -0500, Tom Lendacky wrote:
> Short of figuring out how to map page accesses earlier through the
> boot_page_fault IDT routine

And you want to do that because?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette


From ardb at kernel.org  Wed Aug  2 07:55:27 2023
From: ardb at kernel.org (Ard Biesheuvel)
Date: Wed, 2 Aug 2023 16:55:27 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local>
References: <20230601072043.24439-1-ltao@redhat.com> <20230713100459.GEZK/MS69XbphJa+tN@fat_crate.local>
 <CAO7dBbVMNKTSDi5eP4BseEUexsk0Mo0GWJpyHfOcp+tHs6cSUw@mail.gmail.com>
 <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local> <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local> <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local> <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
 <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local>
Message-ID: <CAMj1kXEM5hGknVGwHh_w99D4L8yrYrTFycwGHZ0CQun70CLipw@mail.gmail.com>

On Wed, 2 Aug 2023 at 15:59, Borislav Petkov <bp at alien8.de> wrote:
>
> On Wed, Aug 02, 2023 at 08:40:36AM -0500, Tom Lendacky wrote:
> > Short of figuring out how to map page accesses earlier through the
> > boot_page_fault IDT routine
>
> And you want to do that because?
>

... because now, entering via startup_32 is broken, given that it only
maps the kernel image itself and relies on the #PF handling for
everything else it accesses, including firmware tables.

AFAICT this also means that entering via startup_32 is broken entirely
for any configuration that enables the cc blob config table check,
regardless of the platform.


From bp at alien8.de  Wed Aug  2 08:51:46 2023
From: bp at alien8.de (Borislav Petkov)
Date: Wed, 2 Aug 2023 17:51:46 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <CAMj1kXEM5hGknVGwHh_w99D4L8yrYrTFycwGHZ0CQun70CLipw@mail.gmail.com>
References: <20230713100459.GEZK/MS69XbphJa+tN@fat_crate.local>
 <CAO7dBbVMNKTSDi5eP4BseEUexsk0Mo0GWJpyHfOcp+tHs6cSUw@mail.gmail.com>
 <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local>
 <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local>
 <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local>
 <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
 <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local>
 <CAMj1kXEM5hGknVGwHh_w99D4L8yrYrTFycwGHZ0CQun70CLipw@mail.gmail.com>
Message-ID: <20230802155146.GCZMp7ksDdN2ETVzKV@fat_crate.local>

On Wed, Aug 02, 2023 at 04:55:27PM +0200, Ard Biesheuvel wrote:
> ... because now, entering via startup_32 is broken, given that it only
> maps the kernel image itself and relies on the #PF handling for
> everything else it accesses, including firmware tables.
> 
> AFAICT this also means that entering via startup_32 is broken entirely
> for any configuration that enables the cc blob config table check,
> regardless of the platform.

Lemme brain-dump what Tom and I just talked on IRC.

That startup_32 entry path for SNP guests was used with old grubs which
used to enter through there and not anymore, reportedly. Which means,
that must've worked at some point but Joerg would know. CCed.

Newer grubs enter through the 64-bit entry point and thus are fine
- otherwise we would be seeing explosions left and right.

So dependent on what we wanna do, if we kill the 32-bit path, we can
kill the 32-bit C-bit verif code. But that's for later and an item on my
TODO list.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette


From tusharsu at linux.microsoft.com  Wed Aug  2 14:04:57 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Wed, 2 Aug 2023 14:04:57 -0700
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
 <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
 <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
Message-ID: <dd126081-3216-c15e-124e-ef137b376cdb@linux.microsoft.com>


On 8/1/23 20:58, Jarkko Sakkinen wrote:
> On Wed Aug 2, 2023 at 12:01 AM EEST, Tushar Sugandhi wrote:
>> Thanks for the response Jarkko.
>>
>> On 8/1/23 12:02, Jarkko Sakkinen wrote:
>>> The short summary is cryptic to say the least.
>> Do you mean the patch subject line, or the description below?
> It is in the process documentation:
>
> https://www.kernel.org/doc/html/v6.3/process/submitting-patches.html#the-canonical-patch-format
Sounds good.? I will cleanup both the summary phrase and the patch 
description.
>>> "update counter" does not map it to have anything to do with PCRs.
>> Agreed.? I noticed that when I was testing the patches.
>> The update counter is same for all PCRs.? It was also the same for
>> the two hash algo's I tested it for (SHA1 and SHA256). But the spec
>> description and Kernel implementation requires to pass the
>> pcr_idx and hash algo to PCR_Read command to get the update counter.
> I was referring to the fact that TPM2_PCR_Read does not have a field
> called "update counter" in its response but it has a field called
> "pcrUpdateCounter". Please refer to thigs that actually exist.
>
> In the long description you are in some occasions referring to the same
> object as:
>
> 1. "update counter"
> 2. "pcrUpdateCounter"
> 3. "PcrUpdateCounter"
>
> This is ambiguous and wrong.
Thanks. I will consistently use pcrUpdateCounter going forward.
> >From long description I see zero motivation to ack this change, except
> some heresay about IMA requiring it. Why does IMA need update_cnt and
> why this is not documented to the long description?
Since patch 2 of this series exposes the functionality to IMA,
it is described in the long description of patch 2.

But I can add the description here as well for completeness.
>> But I can update tpm2_pcr_read() if you are ok with it.
>> Please let me know.
> You can add "u32 *update_cnt".
Sounds good.? Will do.

Btw, the function tpm2_pcr_read is not exposed directly to the other
subsystems (like IMA).? It is exposed via tpm_pcr_read.

Do you want to expose tpm2_pcr_read directly,
or do you want me to update the function signature of tpm_pcr_read as well?

Updating the function signature of tpm_pcr_read as well -
to return "u32 *update_cnt" seems like the right approach.
In that case, I can set *update_cnt to say 0 or -1 for TPM1
(because pcrUpdateCounter is not available for TPM1).

Please let me know what do you think.

I will make the changes accordingly.

I will also wait for IMA/Kexec maintainers to take a look at the 
remaining patches
in this series, incorporate their feedback, and send the V2 of this series.

Thanks again for your feedback. Really appreciate it.

~Tushar
>
> BR, Jarkko


From zohar at linux.ibm.com  Wed Aug  2 18:22:06 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Wed, 02 Aug 2023 21:22:06 -0400
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
	 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
	 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
	 <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
	 <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
Message-ID: <8c4257fb07a8c9fd26b70ce716665a9535e6af54.camel@linux.ibm.com>

On Wed, 2023-08-02 at 06:58 +0300, Jarkko Sakkinen wrote:
> 
> From long description I see zero motivation to ack this change, except
> some heresay about IMA requiring it. Why does IMA need update_cnt and
> why this is not documented to the long description?

The motivation is to detect whether the IMA measurement list has been
truncated, for whatever reason.  A new IMA record should be defined
containing the "pcrCounter" value.  (I have not had a chance to review
this patch set.)

This new record would be a pre-req for both Tushar's "ima: measure
events between kexec load and execute" patch set and Sush's proposal to
trim the measurement list.  (I haven't looked at it yet either.)

-- 
thanks,

Mimi



From piliu at redhat.com  Wed Aug  2 19:37:10 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Thu, 3 Aug 2023 10:37:10 +0800
Subject: [PATCHv6 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <ZMpNLGt6RtxalSBb@kernel.org>
References: <20230724022142.6425-1-piliu@redhat.com> <20230724022142.6425-4-piliu@redhat.com>
 <ZMitmSZPqnH+/70s@kernel.org> <CAF+s44QBMgsT7Q2ro9obVvHYCrKYTQoeCnOVOMzhfexQS+_6Sw@mail.gmail.com>
 <ZMpJIZ18ynQLln8W@kernel.org> <ZMpJcgeGeYhcuGfe@kernel.org> <ZMpNLGt6RtxalSBb@kernel.org>
Message-ID: <CAF+s44QFUonTW1DJjW-gGtN6Yi+tdw=rMSyr+jGgZvuHmGOpZA@mail.gmail.com>

On Wed, Aug 2, 2023 at 8:33?PM Simon Horman <horms at kernel.org> wrote:
>
> On Wed, Aug 02, 2023 at 02:17:57PM +0200, Simon Horman wrote:
> > On Wed, Aug 02, 2023 at 02:16:33PM +0200, Simon Horman wrote:
> > > On Wed, Aug 02, 2023 at 05:53:59PM +0800, Pingfan Liu wrote:
> > > > Hi Simon,
> > > >
> > > > Thanks for the try. Please see the comment below.
> > > >
> > > > On Tue, Aug 1, 2023 at 3:00?PM Simon Horman <horms at kernel.org> wrote:
> > > > >
> > > > > On Mon, Jul 24, 2023 at 10:21:40AM +0800, Pingfan Liu wrote:
> > > > > > From: Jeremy Linton <jeremy.linton at arm.com>
> > > > > >
> > > > > > The linux kernel CONFIG_ZBOOT option creates
> > > > > > self decompressing PE kernel images. So this means
> > > > > > that kexec should have a generic understanding of
> > > > > > the format which may be used by multiple arches.
> > > > > >
> > > > > > So lets add an arch independent validation
> > > > > > and decompression routine.
> > > > > >
> > > > > > Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> > > > > > [Modified by Pingfan to export kernel fd]
> > > > > > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > > > >
> > > > > Hi Pingfan,
> > > > >
> > > > > unfortunately this causes a build failure on hppa.
> > > > >
> > > > > ../../kexec/kexec-pe-zboot.c:31:10: fatal error: kexec-pe-zboot.h: No such file or directory
> > > > >    31 | #include <kexec-pe-zboot.h>
> > > > >       |          ^~~~~~~~~~~~~~~~~~
> > > > >
> > > > > Link: https://github.com/horms/kexec-tools/actions/runs/5723580523/job/15508425790
> > > > >
> > > >
> > > > It is not related to cross-compiling. Actually, I have tried to
> > > > simplify the test matrix, which limits the compilation only on x86_64.
> > > > And I got the similar error [1]
> > > >
> > > > The workflow control file is [2], which clips out all arches except
> > > > x86_64.  But I can successfully build it on the Fedora system with the
> > > > following bash script, which is based on the github's build log.  So
> > > > maybe it is a bug with the compiling tools?
> > > >
> > > > kexec_tools_dir="./"
> > > >
> > > > mkdir $kexec_tools_dir/_build \
> > > >          $kexec_tools_dir/_build/sub \
> > > >          $kexec_tools_dir/_inst \
> > > >          $kexec_tools_dir/_dest
> > > > chmod a-w $kexec_tools_dir
> > > > test -d $kexec_tools_dir/_build
> > > > INSTALL_BASE=$(cd $kexec_tools_dir/_inst && pwd | sed -e
> > > > 's,^[^:\\/]:[\\/],/,') &&\
> > > >         DESTDIR="$kexec_tools_dir/_dest" && \
> > > >         cd $kexec_tools_dir/_build/sub && \
> > > >                 ../../configure \
> > > >                  \
> > > >                 --srcdir=../.. --prefix="$INSTALL_BASE" && \
> > > >         make  -j8
> > > >
> > > >
> > > > [1]: https://github.com/pfliu/kexec-tools/actions/runs/5737254109/job/15548520863
> > > > [2]: https://github.com/pfliu/kexec-tools/blob/zbootV6/.github/workflows/main.yml
> > >
> > > Thanks,
> > >
> > > I guess that kexec-pe-zboot.h is missing in the build environment for the
> > > GitHub actions, but present in your Fedora environment.
> > >
> > > Could you take a look and see where your copy of kexec-pe-zboot.h
> > > came from?
> >
> > Actually it seems to be added by this patch (sorry for not noticing)!
> > So I guess it is an include path problem.
>
> I think I have found the problem.
> The kexec-tools build system is a bit unusual,
> and the new file, kexec-pe-zboot.h, was not included in distribution
> tarballs. Thus the build failures.
>

Unexpectedly, and thank for your insight.

> I think you can resolve that by squashing the following into this patch.
>
> diff --git a/include/Makefile b/include/Makefile
> --- a/include/Makefile
> +++ b/include/Makefile
> @@ -1,6 +1,7 @@
>  dist += include/Makefile               \
>         include/config.h                \
>         include/config.h.in             \
> +       include/kexec-pe-zboot.h        \
>         include/kexec-uImage.h          \
>         include/x86/x86-linux.h         \
>         include/x86/mb_info.h           \
>

After applying this patch, the github workflow successfully finished.
I will send out V7 immediately.

Appreciate for your kind help again.

Thanks,

Pingfan



From piliu at redhat.com  Wed Aug  2 19:41:47 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Thu,  3 Aug 2023 10:41:47 +0800
Subject: [PATCHv7 0/5] arm64: zboot support
Message-ID: <20230803024152.11663-1-piliu@redhat.com>

From: root <root at piliu.users.ipa.redhat.com>

As more complicated capsule kernel format occurs like zboot, where the
compressed kernel is stored as a payload. The straight forward
decompression can not meet the demand.
  
As the first step, on aarch64, reading in the kernel file in a probe
method and decide how to unfold the content by the method itself.

This series consists of two parts
[1/5], simplify the current aarch64 image probe
[2-5/5], return the kernel fd by the image load interface, and let the
handling of zboot image built on it. (Thanks for Dave Young, who
contributes the original idea and the code)
 
 
To ease the review, a branch is also available at https://github.com/pfliu/kexec-tools.git
branch zbootV7
 
To: kexec at lists.infradead.org
Cc: Dave Young <dyoung at redhat.com>
Cc: horms at verge.net.au
Cc: ardb at kernel.org
Cc: jeremy.linton at arm.com
---
v6 -> v7
  Fix the included file issue in dist

v5 -> v6
  introduce kexec_info.kernel_fd and return the fd through image load
interface.


Jeremy Linton (3):
  kexec/zboot: Add arch independent zboot support
  arm64: Add ZBOOT PE containing compressed image support
  arm64: Hook up the ZBOOT support as vmlinuz

Pingfan Liu (2):
  kexec/arm64: Simplify the code for zImage
  kexec: Introduce a member kernel_fd in kexec_info

 include/Makefile                       |   1 +
 include/kexec-pe-zboot.h               |  15 ++
 kexec/Makefile                         |   1 +
 kexec/arch/arm64/Makefile              |   2 +-
 kexec/arch/arm64/image-header.h        |   1 +
 kexec/arch/arm64/kexec-arm64.c         |   2 +-
 kexec/arch/arm64/kexec-arm64.h         |   8 +-
 kexec/arch/arm64/kexec-image-arm64.c   |   2 +-
 kexec/arch/arm64/kexec-vmlinuz-arm64.c | 110 ++++++++++++
 kexec/arch/arm64/kexec-zImage-arm64.c  | 226 -------------------------
 kexec/kexec-pe-zboot.c                 | 131 ++++++++++++++
 kexec/kexec.c                          |  50 ++++--
 kexec/kexec.h                          |   1 +
 13 files changed, 299 insertions(+), 251 deletions(-)
 create mode 100644 include/kexec-pe-zboot.h
 create mode 100644 kexec/arch/arm64/kexec-vmlinuz-arm64.c
 delete mode 100644 kexec/arch/arm64/kexec-zImage-arm64.c
 create mode 100644 kexec/kexec-pe-zboot.c

-- 
2.31.1



From piliu at redhat.com  Wed Aug  2 19:41:48 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Thu,  3 Aug 2023 10:41:48 +0800
Subject: [PATCHv7 1/5] kexec/arm64: Simplify the code for zImage
In-Reply-To: <20230803024152.11663-1-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com>
Message-ID: <20230803024152.11663-2-piliu@redhat.com>

Inside zimage_probe(), it uncompresses the kernel and performs some
check, similar to image_probe(). Taking a close look, the uncompressing
has already executed before the image probe is called. What is missing
here is to provide a fd, pointing to an uncompressed kernel image.

This patch creates a memfd based on the result produced by
slurp_decompress_file(), and finally simplify the logical of the probe
for aarch64.

The credit goes to the Dave Young, who contributes the original code.

Signed-off-by: Pingfan Liu <piliu at redhat.com>
Co-authored-by: Dave Young <dyoung at redhat.com>
To: kexec at lists.infradead.org
Cc: horms at verge.net.au
Cc: ardb at kernel.org
Cc: jeremy.linton at arm.com
---
 kexec/arch/arm64/Makefile             |   3 +-
 kexec/arch/arm64/kexec-arm64.c        |   1 -
 kexec/arch/arm64/kexec-arm64.h        |   6 -
 kexec/arch/arm64/kexec-image-arm64.c  |   2 +-
 kexec/arch/arm64/kexec-zImage-arm64.c | 226 --------------------------
 kexec/kexec.c                         |  42 +++--
 6 files changed, 26 insertions(+), 254 deletions(-)
 delete mode 100644 kexec/arch/arm64/kexec-zImage-arm64.c

diff --git a/kexec/arch/arm64/Makefile b/kexec/arch/arm64/Makefile
index d27c8ee..9d9111c 100644
--- a/kexec/arch/arm64/Makefile
+++ b/kexec/arch/arm64/Makefile
@@ -15,8 +15,7 @@ arm64_KEXEC_SRCS += \
 	kexec/arch/arm64/kexec-arm64.c \
 	kexec/arch/arm64/kexec-elf-arm64.c \
 	kexec/arch/arm64/kexec-uImage-arm64.c \
-	kexec/arch/arm64/kexec-image-arm64.c \
-	kexec/arch/arm64/kexec-zImage-arm64.c
+	kexec/arch/arm64/kexec-image-arm64.c
 
 arm64_UIMAGE = kexec/kexec-uImage.c
 
diff --git a/kexec/arch/arm64/kexec-arm64.c b/kexec/arch/arm64/kexec-arm64.c
index ec6df4b..a830ec7 100644
--- a/kexec/arch/arm64/kexec-arm64.c
+++ b/kexec/arch/arm64/kexec-arm64.c
@@ -74,7 +74,6 @@ struct file_type file_type[] = {
 	{"vmlinux", elf_arm64_probe, elf_arm64_load, elf_arm64_usage},
 	{"Image", image_arm64_probe, image_arm64_load, image_arm64_usage},
 	{"uImage", uImage_arm64_probe, uImage_arm64_load, uImage_arm64_usage},
-	{"zImage", zImage_arm64_probe, zImage_arm64_load, zImage_arm64_usage},
 };
 
 int file_types = sizeof(file_type) / sizeof(file_type[0]);
diff --git a/kexec/arch/arm64/kexec-arm64.h b/kexec/arch/arm64/kexec-arm64.h
index 5eb9fc0..d29b1b3 100644
--- a/kexec/arch/arm64/kexec-arm64.h
+++ b/kexec/arch/arm64/kexec-arm64.h
@@ -44,12 +44,6 @@ int uImage_arm64_load(int argc, char **argv, const char *buf, off_t len,
 		      struct kexec_info *info);
 void uImage_arm64_usage(void);
 
-int zImage_arm64_probe(const char *kernel_buf, off_t kernel_size);
-int zImage_arm64_load(int argc, char **argv, const char *kernel_buf,
-	off_t kernel_size, struct kexec_info *info);
-void zImage_arm64_usage(void);
-
-
 extern off_t initrd_base;
 extern off_t initrd_size;
 
diff --git a/kexec/arch/arm64/kexec-image-arm64.c b/kexec/arch/arm64/kexec-image-arm64.c
index aa8f2e2..a196747 100644
--- a/kexec/arch/arm64/kexec-image-arm64.c
+++ b/kexec/arch/arm64/kexec-image-arm64.c
@@ -114,6 +114,6 @@ exit:
 void image_arm64_usage(void)
 {
 	printf(
-"     An ARM64 binary image, uncompressed, big or little endian.\n"
+"     An ARM64 binary image, compressed or not, big or little endian.\n"
 "     Typically an Image file.\n\n");
 }
diff --git a/kexec/arch/arm64/kexec-zImage-arm64.c b/kexec/arch/arm64/kexec-zImage-arm64.c
deleted file mode 100644
index 6ee82ff..0000000
--- a/kexec/arch/arm64/kexec-zImage-arm64.c
+++ /dev/null
@@ -1,226 +0,0 @@
-/*
- * ARM64 kexec zImage (Image.gz) support.
- *
- * Several distros use 'make zinstall' rule inside
- * 'arch/arm64/boot/Makefile' to install the arm64
- * Image.gz compressed file inside the boot destination
- * directory (for e.g. /boot).
- *
- * Currently we cannot use kexec_file_load() to load vmlinuz
- * (or Image.gz).
- *
- * To support Image.gz, we should:
- * a). Copy the contents of Image.gz to a temporary file.
- * b). Decompress (gunzip-decompress) the contents inside the
- *     temporary file.
- * c). Pass the 'fd' of the temporary file to the kernel space.
- *
- * So basically the kernel space still gets a decompressed
- * kernel image to load via kexec-tools.
- */
-
-#define _GNU_SOURCE
-
-#include <errno.h>
-#include <fcntl.h>
-#include <limits.h>
-#include <stdlib.h>
-#include "crashdump-arm64.h"
-#include "image-header.h"
-#include "kexec.h"
-#include "kexec-arm64.h"
-#include "kexec-syscall.h"
-#include "kexec-zlib.h"
-#include "arch/options.h"
-
-#define FILENAME_IMAGE		"/tmp/ImageXXXXXX"
-
-/* Returns:
- * -1 : in case of error/invalid format (not a valid Image.gz format.
- * fd : File descriptor of the temp file containing the decompressed
- *      Image.
- */
-int zImage_arm64_probe(const char *kernel_buf, off_t kernel_size)
-{
-	int ret = -1;
-	int fd = 0;
-	int kernel_fd = 0;
-	char *fname = NULL;
-	char *kernel_uncompressed_buf = NULL;
-	const struct arm64_image_header *h;
-
-	if (!is_zlib_file(kernel_buf, &kernel_size)) {
-		dbgprintf("%s: Not an zImage file (Image.gz).\n", __func__);
-		return -1;
-	}
-
-	if (!(fname = strdup(FILENAME_IMAGE))) {
-		dbgprintf("%s: Can't duplicate strings %s\n", __func__,
-				fname);
-		return -1;
-	}
-
-	if ((fd = mkstemp(fname)) < 0) {
-		dbgprintf("%s: Can't open file %s\n", __func__,
-				fname);
-		ret = -1;
-		goto fail_mkstemp;
-	}
-
-	kernel_uncompressed_buf =
-		(char *) calloc(kernel_size, sizeof(off_t));
-	if (!kernel_uncompressed_buf) {
-		dbgprintf("%s: Can't calloc %ld bytes\n",
-				__func__, kernel_size);
-		ret= -ENOMEM;
-		goto fail_calloc;
-	}
-
-	/* slurp in the input kernel */
-	dbgprintf("%s: ", __func__);
-	kernel_uncompressed_buf = slurp_decompress_file(kernel_buf,
-							&kernel_size);
-
-	/* check for correct header magic */
-	if (kernel_size < sizeof(struct arm64_image_header)) {
-		dbgprintf("%s: No arm64 image header.\n", __func__);
-		ret = -1;
-		goto fail_bad_header;
-	}
-
-	h = (const struct arm64_image_header *)(kernel_uncompressed_buf);
-
-	if (!arm64_header_check_magic(h)) {
-		dbgprintf("%s: Bad arm64 image header.\n", __func__);
-		ret = -1;
-		goto fail_bad_header;
-	}
-
-	if (write(fd, kernel_uncompressed_buf,
-				kernel_size) != kernel_size) {
-		dbgprintf("%s: Can't write the uncompressed file %s\n",
-				__func__, fname);
-		ret = -1;
-		goto fail_bad_header;
-	}
-
-	close(fd);
-
-	/* Open the tmp file again, this time in O_RDONLY mode, as
-	 * opening the file in O_RDWR and calling kexec_file_load()
-	 * causes the kernel to return -ETXTBSY
-	 */
-	kernel_fd = open(fname, O_RDONLY);
-	if (kernel_fd == -1) {
-		dbgprintf("%s: Failed to open file %s\n",
-				__func__, fname);
-		ret = -1;
-		goto fail_bad_header;
-	}
-
-	unlink(fname);
-
-	free(kernel_uncompressed_buf);
-	free(fname);
-
-	return kernel_fd;
-
-fail_bad_header:
-	free(kernel_uncompressed_buf);
-
-fail_calloc:
-	if (fd >= 0)
-		close(fd);
-
-	unlink(fname);
-
-fail_mkstemp:
-	free(fname);
-
-	return ret;
-}
-
-int zImage_arm64_load(int argc, char **argv, const char *kernel_buf,
-	off_t kernel_size, struct kexec_info *info)
-{
-	const struct arm64_image_header *header;
-	unsigned long kernel_segment;
-	int result;
-
-	if (info->file_mode) {
-		if (arm64_opts.initrd) {
-			info->initrd_fd = open(arm64_opts.initrd, O_RDONLY);
-			if (info->initrd_fd == -1) {
-				fprintf(stderr,
-					"Could not open initrd file %s:%s\n",
-					arm64_opts.initrd, strerror(errno));
-				result = EFAILED;
-				goto exit;
-			}
-		}
-
-		if (arm64_opts.command_line) {
-			info->command_line = (char *)arm64_opts.command_line;
-			info->command_line_len =
-					strlen(arm64_opts.command_line) + 1;
-		}
-
-		return 0;
-	}
-
-	header = (const struct arm64_image_header *)(kernel_buf);
-
-	if (arm64_process_image_header(header))
-		return EFAILED;
-
-	kernel_segment = arm64_locate_kernel_segment(info);
-
-	if (kernel_segment == ULONG_MAX) {
-		dbgprintf("%s: Kernel segment is not allocated\n", __func__);
-		result = EFAILED;
-		goto exit;
-	}
-
-	dbgprintf("%s: kernel_segment: %016lx\n", __func__, kernel_segment);
-	dbgprintf("%s: text_offset:    %016lx\n", __func__,
-		arm64_mem.text_offset);
-	dbgprintf("%s: image_size:     %016lx\n", __func__,
-		arm64_mem.image_size);
-	dbgprintf("%s: phys_offset:    %016lx\n", __func__,
-		arm64_mem.phys_offset);
-	dbgprintf("%s: vp_offset:      %016lx\n", __func__,
-		arm64_mem.vp_offset);
-	dbgprintf("%s: PE format:      %s\n", __func__,
-		(arm64_header_check_pe_sig(header) ? "yes" : "no"));
-
-	/* create and initialize elf core header segment */
-	if (info->kexec_flags & KEXEC_ON_CRASH) {
-		result = load_crashdump_segments(info);
-		if (result) {
-			dbgprintf("%s: Creating eflcorehdr failed.\n",
-								__func__);
-			goto exit;
-		}
-	}
-
-	/* load the kernel */
-	add_segment_phys_virt(info, kernel_buf, kernel_size,
-			kernel_segment + arm64_mem.text_offset,
-			arm64_mem.image_size, 0);
-
-	/* load additional data */
-	result = arm64_load_other_segments(info, kernel_segment
-		+ arm64_mem.text_offset);
-
-exit:
-	if (result)
-		fprintf(stderr, "kexec: load failed.\n");
-	return result;
-}
-
-void zImage_arm64_usage(void)
-{
-	printf(
-"     An ARM64 zImage, compressed, big or little endian.\n"
-"     Typically an Image.gz or Image.lzma file.\n\n");
-}
diff --git a/kexec/kexec.c b/kexec/kexec.c
index 36bb2ad..d132eb5 100644
--- a/kexec/kexec.c
+++ b/kexec/kexec.c
@@ -638,6 +638,21 @@ char *slurp_decompress_file(const char *filename, off_t *r_size)
 	return kernel_buf;
 }
 
+static int copybuf_memfd(const char *kernel_buf, size_t size)
+{
+	int fd, count;
+
+	fd = memfd_create("kernel", MFD_ALLOW_SEALING);
+	if (fd == -1)
+		return fd;
+
+	count = write(fd, kernel_buf, size);
+	if (count < 0)
+		return -1;
+
+	return fd;
+}
+
 static void update_purgatory(struct kexec_info *info)
 {
 	static const uint8_t null_buf[256];
@@ -1290,31 +1305,22 @@ static int do_kexec_file_load(int fileind, int argc, char **argv,
 
 	kernel = argv[fileind];
 
-	kernel_fd = open(kernel, O_RDONLY);
-	if (kernel_fd == -1) {
-		fprintf(stderr, "Failed to open file %s:%s\n", kernel,
+	/* slurp in the input kernel */
+	kernel_buf = slurp_decompress_file(kernel, &kernel_size);
+	if (!kernel_buf) {
+		fprintf(stderr, "Failed to decompress file %s:%s\n", kernel,
 				strerror(errno));
 		return EFAILED;
 	}
-
-	/* slurp in the input kernel */
-	kernel_buf = slurp_decompress_file(kernel, &kernel_size);
+	kernel_fd = copybuf_memfd(kernel_buf, kernel_size);
+	if (kernel_fd < 0) {
+		fprintf(stderr, "Failed to copy decompressed buf\n");
+		return EFAILED;
+	}
 
 	for (i = 0; i < file_types; i++) {
-#ifdef __aarch64__
-		/* handle Image.gz like cases */
-		if (is_zlib_file(kernel, &kernel_size)) {
-			if ((ret = file_type[i].probe(kernel, kernel_size)) >= 0) {
-				kernel_fd = ret;
-				break;
-			}
-		} else
-			if (file_type[i].probe(kernel_buf, kernel_size) >= 0)
-				break;
-#else
 		if (file_type[i].probe(kernel_buf, kernel_size) >= 0)
 			break;
-#endif
 	}
 
 	if (i == file_types) {
-- 
2.31.1



From piliu at redhat.com  Wed Aug  2 19:41:49 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Thu,  3 Aug 2023 10:41:49 +0800
Subject: [PATCHv7 2/5] kexec: Introduce a member kernel_fd in kexec_info
In-Reply-To: <20230803024152.11663-1-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com>
Message-ID: <20230803024152.11663-3-piliu@redhat.com>

Utilize the image load interface to export the kernel fd, which points
to the uncompressed kernel and will be passed to kexec_file_load.

The credit goes to the Dave Young, who contributes the original code.

Signed-off-by: Pingfan Liu <piliu at redhat.com>
Co-authored-by: Dave Young <dyoung at redhat.com>
To: kexec at lists.infradead.org
Cc: horms at verge.net.au
Cc: ardb at kernel.org
Cc: jeremy.linton at arm.com
---
 kexec/kexec.c | 8 ++++++++
 kexec/kexec.h | 1 +
 2 files changed, 9 insertions(+)

diff --git a/kexec/kexec.c b/kexec/kexec.c
index d132eb5..c3b182e 100644
--- a/kexec/kexec.c
+++ b/kexec/kexec.c
@@ -1292,6 +1292,7 @@ static int do_kexec_file_load(int fileind, int argc, char **argv,
 	info.kexec_flags = flags;
 
 	info.file_mode = 1;
+	info.kernel_fd = -1;
 	info.initrd_fd = -1;
 
 	if (!is_kexec_file_load_implemented())
@@ -1337,6 +1338,13 @@ static int do_kexec_file_load(int fileind, int argc, char **argv,
 		return ret;
 	}
 
+       /*
+	* image type specific load functioin detect the capsule kernel type
+	* and create another fd for file load. For example the zboot kernel.
+	*/
+	if (info.kernel_fd != -1)
+		kernel_fd = info.kernel_fd;
+
 	/*
 	 * If there is no initramfs, set KEXEC_FILE_NO_INITRAMFS flag so that
 	 * kernel does not return error with negative initrd_fd.
diff --git a/kexec/kexec.h b/kexec/kexec.h
index 0d820ad..ed3b499 100644
--- a/kexec/kexec.h
+++ b/kexec/kexec.h
@@ -164,6 +164,7 @@ struct kexec_info {
 	unsigned long file_mode :1;
 
 	/* Filled by kernel image processing code */
+	int kernel_fd;
 	int initrd_fd;
 	char *command_line;
 	int command_line_len;
-- 
2.31.1



From piliu at redhat.com  Wed Aug  2 19:41:50 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Thu,  3 Aug 2023 10:41:50 +0800
Subject: [PATCHv7 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <20230803024152.11663-1-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com>
Message-ID: <20230803024152.11663-4-piliu@redhat.com>

From: Jeremy Linton <jeremy.linton at arm.com>

The linux kernel CONFIG_ZBOOT option creates
self decompressing PE kernel images. So this means
that kexec should have a generic understanding of
the format which may be used by multiple arches.

So lets add an arch independent validation
and decompression routine.

Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
[Modified by Pingfan to export kernel fd]
Signed-off-by: Pingfan Liu <piliu at redhat.com>
To: kexec at lists.infradead.org
Cc: horms at verge.net.au
Cc: ardb at kernel.org
Cc: jeremy.linton at arm.com
---
 include/Makefile         |   1 +
 include/kexec-pe-zboot.h |  15 +++++
 kexec/Makefile           |   1 +
 kexec/kexec-pe-zboot.c   | 131 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 148 insertions(+)
 create mode 100644 include/kexec-pe-zboot.h
 create mode 100644 kexec/kexec-pe-zboot.c

diff --git a/include/Makefile b/include/Makefile
index 621ce9f..cd88a26 100644
--- a/include/Makefile
+++ b/include/Makefile
@@ -1,6 +1,7 @@
 dist += include/Makefile		\
 	include/config.h		\
 	include/config.h.in		\
+	include/kexec-pe-zboot.h	\
 	include/kexec-uImage.h		\
 	include/x86/x86-linux.h		\
 	include/x86/mb_info.h		\
diff --git a/include/kexec-pe-zboot.h b/include/kexec-pe-zboot.h
new file mode 100644
index 0000000..c588ca2
--- /dev/null
+++ b/include/kexec-pe-zboot.h
@@ -0,0 +1,15 @@
+#ifndef __KEXEC_PE_ZBOOT_H__
+#define __KEXEC_PE_ZBOOT_H__
+
+/* see drivers/firmware/efi/libstub/zboot-header.S */
+struct linux_pe_zboot_header {
+	uint32_t mz_magic;
+        uint32_t image_type;
+        uint32_t payload_offset;
+        uint32_t payload_size;
+        uint32_t reserved[2];
+        uint32_t compress_type;
+};
+
+int pez_prepare(const char *crude_buf, off_t buf_sz, int *kernel_fd);
+#endif
diff --git a/kexec/Makefile b/kexec/Makefile
index 8a52e8d..11682bf 100644
--- a/kexec/Makefile
+++ b/kexec/Makefile
@@ -17,6 +17,7 @@ KEXEC_SRCS_base += kexec/kexec-elf-exec.c
 KEXEC_SRCS_base += kexec/kexec-elf-core.c
 KEXEC_SRCS_base += kexec/kexec-elf-rel.c
 KEXEC_SRCS_base += kexec/kexec-elf-boot.c
+KEXEC_SRCS_base += kexec/kexec-pe-zboot.c
 KEXEC_SRCS_base += kexec/kexec-iomem.c
 KEXEC_SRCS_base += kexec/firmware_memmap.c
 KEXEC_SRCS_base += kexec/crashdump.c
diff --git a/kexec/kexec-pe-zboot.c b/kexec/kexec-pe-zboot.c
new file mode 100644
index 0000000..2f2e052
--- /dev/null
+++ b/kexec/kexec-pe-zboot.c
@@ -0,0 +1,131 @@
+/*
+ * Generic PE compressed Image (vmlinuz, ZBOOT) support.
+ *
+ * Several distros use 'make zinstall' with CONFIG_ZBOOT
+ * enabled to create UEFI PE images that contain
+ * a decompressor and a compressed kernel image.
+ *
+ * Currently we cannot use kexec_file_load() to load vmlinuz
+ * PE images that self decompress.
+ *
+ * To support ZBOOT, we should:
+ * a). Copy the compressed contents of vmlinuz to a temporary file.
+ * b). Decompress (gunzip-decompress) the contents inside the
+ *     temporary file.
+ * c). Validate the resulting image and write it back to the
+ *     temporary file.
+ * d). Pass the 'fd' of the temporary file to the kernel space.
+ *
+ * This module contains the arch independent code for the above,
+ * arch specific PE and image checks should wrap calls
+ * to functions in this module.
+ */
+
+#define _GNU_SOURCE
+#include <stdlib.h>
+#include <stdint.h>
+#include <sys/types.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include "kexec.h"
+#include <kexec-pe-zboot.h>
+
+#define FILENAME_IMAGE		"/tmp/ImageXXXXXX"
+
+/*
+ * Returns -1 : in case of error/invalid format (not a valid PE+compressed ZBOOT format.
+ *
+ * crude_buf: the content, which is read from the kernel file without any processing
+ */
+int pez_prepare(const char *crude_buf, off_t buf_sz, int *kernel_fd)
+{
+	int ret = -1;
+	int fd = 0;
+	char *fname = NULL;
+	char *kernel_uncompressed_buf = NULL;
+	off_t decompressed_size = 0;
+	const struct linux_pe_zboot_header *z;
+
+	z = (const struct linux_pe_zboot_header *)(crude_buf);
+
+	if (memcmp(&z->image_type, "zimg", sizeof(z->image_type))) {
+		dbgprintf("%s: PE doesn't contain a compressed kernel.\n", __func__);
+		return -1;
+	}
+
+	/*
+	 * At the moment its possible to create images with more compression
+	 * algorithms than are supported here, error out if we detect that.
+	 */
+	if (memcmp(&z->compress_type, "gzip", 4) &&
+	    memcmp(&z->compress_type, "lzma", 4)) {
+		dbgprintf("%s: kexec can only decompress gziped and lzma images.\n", __func__);
+		return -1;
+	}
+
+	if (buf_sz < z->payload_offset + z->payload_size) {
+		dbgprintf("%s: PE too small to contain complete payload.\n", __func__);
+		return -1;
+	}
+
+	if (!(fname = strdup(FILENAME_IMAGE))) {
+		dbgprintf("%s: Can't duplicate strings\n", __func__);
+		return -1;
+	}
+
+	if ((fd = mkstemp(fname)) < 0) {
+		dbgprintf("%s: Can't open file %s\n", __func__,	fname);
+		ret = -1;
+		goto fail_mkstemp;
+	}
+
+	if (write(fd, &crude_buf[z->payload_offset],
+		  z->payload_size) != z->payload_size) {
+		dbgprintf("%s: Can't write the compressed file %s\n",
+				__func__, fname);
+		ret = -1;
+		goto fail_write;
+	}
+
+	kernel_uncompressed_buf = slurp_decompress_file(fname,
+							&decompressed_size);
+
+	dbgprintf("%s: decompressed size %ld\n", __func__, decompressed_size);
+
+	lseek(fd, 0, SEEK_SET);
+
+	if (write(fd,  kernel_uncompressed_buf,
+		  decompressed_size) != decompressed_size) {
+		dbgprintf("%s: Can't write the decompressed file %s\n",
+				__func__, fname);
+		ret = -1;
+		goto fail_bad_header;
+	}
+
+	*kernel_fd = open(fname, O_RDONLY);
+	if (*kernel_fd == -1) {
+		dbgprintf("%s: Failed to open file %s\n",
+				__func__, fname);
+		ret = -1;
+		goto fail_bad_header;
+	}
+
+	dbgprintf("%s: done\n", __func__);
+
+	ret = 0;
+	goto fail_write;
+
+fail_bad_header:
+	free(kernel_uncompressed_buf);
+
+fail_write:
+	if (fd >= 0)
+		close(fd);
+
+	unlink(fname);
+
+fail_mkstemp:
+	free(fname);
+
+	return ret;
+}
-- 
2.31.1



From piliu at redhat.com  Wed Aug  2 19:41:51 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Thu,  3 Aug 2023 10:41:51 +0800
Subject: [PATCHv7 4/5] arm64: Add ZBOOT PE containing compressed image support
In-Reply-To: <20230803024152.11663-1-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com>
Message-ID: <20230803024152.11663-5-piliu@redhat.com>

From: Jeremy Linton <jeremy.linton at arm.com>

The kernel EFI stub ZBOOT feature creates a PE that
contains a compressed linux kernel image. The stub
when run in a valid UEFI environment then decompresses
the resulting image and executes it.

Support these image formats with kexec as well to avoid
having to keep an alternate kernel image around.

This patch adds a the _probe(), _load() and usage() routines needed for
kexec to understand this format.

Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
[Modified by Pingfan to export kernel fd with load method]
Signed-off-by: Pingfan Liu <piliu at redhat.com>
To: kexec at lists.infradead.org
Cc: horms at verge.net.au
Cc: ardb at kernel.org
Cc: jeremy.linton at arm.com
---
 kexec/arch/arm64/image-header.h        |   1 +
 kexec/arch/arm64/kexec-vmlinuz-arm64.c | 110 +++++++++++++++++++++++++
 2 files changed, 111 insertions(+)
 create mode 100644 kexec/arch/arm64/kexec-vmlinuz-arm64.c

diff --git a/kexec/arch/arm64/image-header.h b/kexec/arch/arm64/image-header.h
index 158d411..26bb02f 100644
--- a/kexec/arch/arm64/image-header.h
+++ b/kexec/arch/arm64/image-header.h
@@ -37,6 +37,7 @@ struct arm64_image_header {
 
 static const uint8_t arm64_image_magic[4] = {'A', 'R', 'M', 0x64U};
 static const uint8_t arm64_image_pe_sig[2] = {'M', 'Z'};
+static const uint8_t arm64_pe_machtype[6] = {'P','E', 0x0, 0x0, 0x64, 0xAA};
 static const uint64_t arm64_image_flag_be = (1UL << 0);
 static const uint64_t arm64_image_flag_page_size = (3UL << 1);
 static const uint64_t arm64_image_flag_placement = (1UL << 3);
diff --git a/kexec/arch/arm64/kexec-vmlinuz-arm64.c b/kexec/arch/arm64/kexec-vmlinuz-arm64.c
new file mode 100644
index 0000000..c0ee47c
--- /dev/null
+++ b/kexec/arch/arm64/kexec-vmlinuz-arm64.c
@@ -0,0 +1,110 @@
+/*
+ * ARM64 PE compressed Image (vmlinuz, ZBOOT) support.
+ *
+ * Several distros use 'make zinstall' rule inside
+ * 'arch/arm64/boot/Makefile' to install the arm64
+ * ZBOOT compressed file inside the boot destination
+ * directory (for e.g. /boot).
+ *
+ * Currently we cannot use kexec_file_load() to load vmlinuz
+ * PE images that self decompress.
+ *
+ * To support ZBOOT, we should:
+ * a). Copy the compressed contents of vmlinuz to a temporary file.
+ * b). Decompress (gunzip-decompress) the contents inside the
+ *     temporary file.
+ * c). Validate the resulting image and write it back to the
+ *     temporary file.
+ * d). Pass the 'fd' of the temporary file to the kernel space.
+ *
+ * Note this, module doesn't provide a _load() function instead
+ * relying on image_arm64_load() to load the resulting decompressed
+ * image.
+ *
+ * So basically the kernel space still gets a decompressed
+ * kernel image to load via kexec-tools.
+ */
+
+#define _GNU_SOURCE
+#include <fcntl.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include "kexec-arm64.h"
+#include <kexec-pe-zboot.h>
+#include "arch/options.h"
+
+static int kernel_fd = -1;
+
+/* Returns:
+ * -1 : in case of error/invalid format (not a valid PE+compressed ZBOOT format.
+ */
+int pez_arm64_probe(const char *kernel_buf, off_t kernel_size)
+{
+	int ret = -1;
+	const struct arm64_image_header *h;
+	char *buf;
+	off_t buf_sz;
+
+	buf = (char *)kernel_buf;
+	buf_sz = kernel_size;
+	if (!buf)
+		return -1;
+	h = (const struct arm64_image_header *)buf;
+
+	dbgprintf("%s: PROBE.\n", __func__);
+	if (buf_sz < sizeof(struct arm64_image_header)) {
+		dbgprintf("%s: Not large enough to be a PE image.\n", __func__);
+		return -1;
+	}
+	if (!arm64_header_check_pe_sig(h)) {
+		dbgprintf("%s: Not an PE image.\n", __func__);
+		return -1;
+	}
+
+	if (buf_sz < sizeof(struct arm64_image_header) + h->pe_header) {
+		dbgprintf("%s: PE image offset larger than image.\n", __func__);
+		return -1;
+	}
+
+	if (memcmp(&buf[h->pe_header],
+		   arm64_pe_machtype, sizeof(arm64_pe_machtype))) {
+		dbgprintf("%s: PE header doesn't match machine type.\n", __func__);
+		return -1;
+	}
+
+	ret = pez_prepare(buf, buf_sz, &kernel_fd);
+
+	if (!ret) {
+	    /* validate the arm64 specific header */
+	    struct arm64_image_header hdr_check;
+	    if (read(kernel_fd, &hdr_check, sizeof(hdr_check)) != sizeof(hdr_check))
+		goto bad_header;
+
+	    lseek(kernel_fd, 0, SEEK_SET);
+
+	    if (!arm64_header_check_magic(&hdr_check)) {
+		dbgprintf("%s: Bad arm64 image header.\n", __func__);
+		goto bad_header;
+	    }
+	}
+
+	return ret;
+bad_header:
+	close(kernel_fd);
+	free(buf);
+	return -1;
+}
+
+int pez_arm64_load(int argc, char **argv, const char *buf, off_t len,
+			struct kexec_info *info)
+{
+	info->kernel_fd = kernel_fd;
+	return image_arm64_load(argc, argv, buf, len, info);
+}
+
+void pez_arm64_usage(void)
+{
+	printf(
+"     An ARM64 vmlinuz, PE image of a compressed, little endian.\n"
+"     kernel, built with ZBOOT enabled.\n\n");
+}
-- 
2.31.1



From piliu at redhat.com  Wed Aug  2 19:41:52 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Thu,  3 Aug 2023 10:41:52 +0800
Subject: [PATCHv7 5/5] arm64: Hook up the ZBOOT support as vmlinuz
In-Reply-To: <20230803024152.11663-1-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com>
Message-ID: <20230803024152.11663-6-piliu@redhat.com>

From: Jeremy Linton <jeremy.linton at arm.com>

Add the previously defined _probe() and _usage() routines
to the kexec file types table, and build the new module.

It should be noted that this "vmlinuz" support reuses the
"Image" support to actually load the resulting image after
it has been decompressed to a temporary file.

Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
Signed-off-by: Pingfan Liu <piliu at redhat.com>
To: kexec at lists.infradead.org
Cc: horms at verge.net.au
Cc: ardb at kernel.org
Cc: jeremy.linton at arm.com
---
 kexec/arch/arm64/Makefile      | 3 ++-
 kexec/arch/arm64/kexec-arm64.c | 1 +
 kexec/arch/arm64/kexec-arm64.h | 6 ++++++
 3 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/kexec/arch/arm64/Makefile b/kexec/arch/arm64/Makefile
index 9d9111c..59212f1 100644
--- a/kexec/arch/arm64/Makefile
+++ b/kexec/arch/arm64/Makefile
@@ -15,7 +15,8 @@ arm64_KEXEC_SRCS += \
 	kexec/arch/arm64/kexec-arm64.c \
 	kexec/arch/arm64/kexec-elf-arm64.c \
 	kexec/arch/arm64/kexec-uImage-arm64.c \
-	kexec/arch/arm64/kexec-image-arm64.c
+	kexec/arch/arm64/kexec-image-arm64.c \
+	kexec/arch/arm64/kexec-vmlinuz-arm64.c
 
 arm64_UIMAGE = kexec/kexec-uImage.c
 
diff --git a/kexec/arch/arm64/kexec-arm64.c b/kexec/arch/arm64/kexec-arm64.c
index a830ec7..4a67b0d 100644
--- a/kexec/arch/arm64/kexec-arm64.c
+++ b/kexec/arch/arm64/kexec-arm64.c
@@ -74,6 +74,7 @@ struct file_type file_type[] = {
 	{"vmlinux", elf_arm64_probe, elf_arm64_load, elf_arm64_usage},
 	{"Image", image_arm64_probe, image_arm64_load, image_arm64_usage},
 	{"uImage", uImage_arm64_probe, uImage_arm64_load, uImage_arm64_usage},
+	{"vmlinuz", pez_arm64_probe, pez_arm64_load, pez_arm64_usage},
 };
 
 int file_types = sizeof(file_type) / sizeof(file_type[0]);
diff --git a/kexec/arch/arm64/kexec-arm64.h b/kexec/arch/arm64/kexec-arm64.h
index d29b1b3..95fb5c2 100644
--- a/kexec/arch/arm64/kexec-arm64.h
+++ b/kexec/arch/arm64/kexec-arm64.h
@@ -44,6 +44,12 @@ int uImage_arm64_load(int argc, char **argv, const char *buf, off_t len,
 		      struct kexec_info *info);
 void uImage_arm64_usage(void);
 
+int pez_arm64_probe(const char *kernel_buf, off_t kernel_size);
+int pez_arm64_load(int argc, char **argv, const char *buf, off_t len,
+			struct kexec_info *info);
+void pez_arm64_usage(void);
+
+
 extern off_t initrd_base;
 extern off_t initrd_size;
 
-- 
2.31.1



From horms at kernel.org  Thu Aug  3 01:04:53 2023
From: horms at kernel.org (Simon Horman)
Date: Thu, 3 Aug 2023 10:04:53 +0200
Subject: [PATCHv6 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <CAF+s44QFUonTW1DJjW-gGtN6Yi+tdw=rMSyr+jGgZvuHmGOpZA@mail.gmail.com>
References: <20230724022142.6425-1-piliu@redhat.com>
 <20230724022142.6425-4-piliu@redhat.com>
 <ZMitmSZPqnH+/70s@kernel.org>
 <CAF+s44QBMgsT7Q2ro9obVvHYCrKYTQoeCnOVOMzhfexQS+_6Sw@mail.gmail.com>
 <ZMpJIZ18ynQLln8W@kernel.org>
 <ZMpJcgeGeYhcuGfe@kernel.org>
 <ZMpNLGt6RtxalSBb@kernel.org>
 <CAF+s44QFUonTW1DJjW-gGtN6Yi+tdw=rMSyr+jGgZvuHmGOpZA@mail.gmail.com>
Message-ID: <ZMtfpX5EF5TqzODM@kernel.org>

On Thu, Aug 03, 2023 at 10:37:10AM +0800, Pingfan Liu wrote:
> On Wed, Aug 2, 2023 at 8:33?PM Simon Horman <horms at kernel.org> wrote:
> >
> > On Wed, Aug 02, 2023 at 02:17:57PM +0200, Simon Horman wrote:
> > > On Wed, Aug 02, 2023 at 02:16:33PM +0200, Simon Horman wrote:
> > > > On Wed, Aug 02, 2023 at 05:53:59PM +0800, Pingfan Liu wrote:
> > > > > Hi Simon,
> > > > >
> > > > > Thanks for the try. Please see the comment below.
> > > > >
> > > > > On Tue, Aug 1, 2023 at 3:00?PM Simon Horman <horms at kernel.org> wrote:
> > > > > >
> > > > > > On Mon, Jul 24, 2023 at 10:21:40AM +0800, Pingfan Liu wrote:
> > > > > > > From: Jeremy Linton <jeremy.linton at arm.com>
> > > > > > >
> > > > > > > The linux kernel CONFIG_ZBOOT option creates
> > > > > > > self decompressing PE kernel images. So this means
> > > > > > > that kexec should have a generic understanding of
> > > > > > > the format which may be used by multiple arches.
> > > > > > >
> > > > > > > So lets add an arch independent validation
> > > > > > > and decompression routine.
> > > > > > >
> > > > > > > Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> > > > > > > [Modified by Pingfan to export kernel fd]
> > > > > > > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > > > > >
> > > > > > Hi Pingfan,
> > > > > >
> > > > > > unfortunately this causes a build failure on hppa.
> > > > > >
> > > > > > ../../kexec/kexec-pe-zboot.c:31:10: fatal error: kexec-pe-zboot.h: No such file or directory
> > > > > >    31 | #include <kexec-pe-zboot.h>
> > > > > >       |          ^~~~~~~~~~~~~~~~~~
> > > > > >
> > > > > > Link: https://github.com/horms/kexec-tools/actions/runs/5723580523/job/15508425790
> > > > > >
> > > > >
> > > > > It is not related to cross-compiling. Actually, I have tried to
> > > > > simplify the test matrix, which limits the compilation only on x86_64.
> > > > > And I got the similar error [1]
> > > > >
> > > > > The workflow control file is [2], which clips out all arches except
> > > > > x86_64.  But I can successfully build it on the Fedora system with the
> > > > > following bash script, which is based on the github's build log.  So
> > > > > maybe it is a bug with the compiling tools?
> > > > >
> > > > > kexec_tools_dir="./"
> > > > >
> > > > > mkdir $kexec_tools_dir/_build \
> > > > >          $kexec_tools_dir/_build/sub \
> > > > >          $kexec_tools_dir/_inst \
> > > > >          $kexec_tools_dir/_dest
> > > > > chmod a-w $kexec_tools_dir
> > > > > test -d $kexec_tools_dir/_build
> > > > > INSTALL_BASE=$(cd $kexec_tools_dir/_inst && pwd | sed -e
> > > > > 's,^[^:\\/]:[\\/],/,') &&\
> > > > >         DESTDIR="$kexec_tools_dir/_dest" && \
> > > > >         cd $kexec_tools_dir/_build/sub && \
> > > > >                 ../../configure \
> > > > >                  \
> > > > >                 --srcdir=../.. --prefix="$INSTALL_BASE" && \
> > > > >         make  -j8
> > > > >
> > > > >
> > > > > [1]: https://github.com/pfliu/kexec-tools/actions/runs/5737254109/job/15548520863
> > > > > [2]: https://github.com/pfliu/kexec-tools/blob/zbootV6/.github/workflows/main.yml
> > > >
> > > > Thanks,
> > > >
> > > > I guess that kexec-pe-zboot.h is missing in the build environment for the
> > > > GitHub actions, but present in your Fedora environment.
> > > >
> > > > Could you take a look and see where your copy of kexec-pe-zboot.h
> > > > came from?
> > >
> > > Actually it seems to be added by this patch (sorry for not noticing)!
> > > So I guess it is an include path problem.
> >
> > I think I have found the problem.
> > The kexec-tools build system is a bit unusual,
> > and the new file, kexec-pe-zboot.h, was not included in distribution
> > tarballs. Thus the build failures.
> >
> 
> Unexpectedly, and thank for your insight.
> 
> > I think you can resolve that by squashing the following into this patch.
> >
> > diff --git a/include/Makefile b/include/Makefile
> > --- a/include/Makefile
> > +++ b/include/Makefile
> > @@ -1,6 +1,7 @@
> >  dist += include/Makefile               \
> >         include/config.h                \
> >         include/config.h.in             \
> > +       include/kexec-pe-zboot.h        \
> >         include/kexec-uImage.h          \
> >         include/x86/x86-linux.h         \
> >         include/x86/mb_info.h           \
> >
> 
> After applying this patch, the github workflow successfully finished.
> I will send out V7 immediately.
> 
> Appreciate for your kind help again.

Thanks, I see v7 now.


From jarkko at kernel.org  Thu Aug  3 01:43:37 2023
From: jarkko at kernel.org (Jarkko Sakkinen)
Date: Thu, 03 Aug 2023 11:43:37 +0300
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <dd126081-3216-c15e-124e-ef137b376cdb@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
 <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
 <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
 <dd126081-3216-c15e-124e-ef137b376cdb@linux.microsoft.com>
Message-ID: <CUIRWHEMXL38.29Z2XR564KV33@suppilovahvero>

On Thu Aug 3, 2023 at 12:04 AM EEST, Tushar Sugandhi wrote:
> Btw, the function tpm2_pcr_read is not exposed directly to the other
> subsystems (like IMA).? It is exposed via tpm_pcr_read.
>
> Do you want to expose tpm2_pcr_read directly,
> or do you want me to update the function signature of tpm_pcr_read as well?

As long as you mention that 'update_cnt' causes divegence in the
in-kernel API, and therefore tpm[12]_pcr_read() cannnot be under the
same orchestrator.

If you take this path, please implement a precursory patch, which
replace the existing call sites with some sequence of tpm[12]_pcr_read()
and tpm_is_tpm2() calls.

BR, Jarkko


From ltao at redhat.com  Thu Aug  3 01:45:04 2023
From: ltao at redhat.com (Tao Liu)
Date: Thu, 3 Aug 2023 16:45:04 +0800
Subject: [RFC][PATCH] iwlwifi: Add pci .shutdown() hook for iwlwifi driver
In-Reply-To: <20230803023257.9260-1-ltao@redhat.com>
References: <20230803023257.9260-1-ltao@redhat.com>
Message-ID: <CAO7dBbV=D3N31L-VPS=2Vtreqc-4drKYHT1xWrKphT3J_G5Ndw@mail.gmail.com>

Add kexec to the CC list so kexec people can know this.

On Thu, Aug 3, 2023 at 10:55?AM Tao Liu <ltao at redhat.com> wrote:
>
> Previously no .shutdown() hook is implemented for iwlwifi driver, a
> ETIMEDOUT error will occur during the kexec kernel bootup. As a
> consequence, wifi is unusable after kexec into the new kernel.
>
> This issue is observed and patch tested on the following wireless cards:
>
> 1) Network controller: Intel Corporation Comet Lake PCH-LP CNVi WiFi,
>    Subsystem: Intel Corporation Wi-Fi 6 AX201 160MHz
> 2) Network controller: Intel Corporation Wireless-AC 9260,
>    Subsystem: Intel Corporation Device e014
>
> Signed-off-by: Tao Liu <ltao at redhat.com>
> ---
>
> Hi folks,
>
> This is a RFC patch and I'm not sure about the correctness of the code,
> especially about the pci_clear_master() part. What I want is to stop any
> ongoing DMA access, in case if the memory overwritting during kexec
> kernel bootup. But there is already pci_clear_master(pci_dev) in
> drivers/pci/pci-driver.c:pci_device_shutdown(), so I'm not sure if it is
> still needed in the driver side. And I only tested the patch against the
> above 2 wireless cards and worked OK, not sure if it can work for others.
> Please review the patch, thanks in advance!
>
> Thanks,
> Tao Liu
>
> ---
>  drivers/net/wireless/intel/iwlwifi/pcie/drv.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
> index 73c1fb3c0c5e..24c4c2dd7cb0 100644
> --- a/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
> +++ b/drivers/net/wireless/intel/iwlwifi/pcie/drv.c
> @@ -1513,6 +1513,18 @@ static void iwl_pci_remove(struct pci_dev *pdev)
>         iwl_trans_pcie_free(trans);
>  }
>
> +static void iwl_pci_shutdown(struct pci_dev *pdev)
> +{
> +       struct iwl_trans *trans = pci_get_drvdata(pdev);
> +
> +       if (!trans)
> +               return;
> +
> +       iwl_drv_stop(trans->drv);
> +
> +       pci_clear_master(pdev);
> +}
> +
>  #ifdef CONFIG_PM_SLEEP
>
>  static int iwl_pci_suspend(struct device *device)
> @@ -1583,6 +1595,7 @@ static struct pci_driver iwl_pci_driver = {
>         .id_table = iwl_hw_card_ids,
>         .probe = iwl_pci_probe,
>         .remove = iwl_pci_remove,
> +       .shutdown = iwl_pci_shutdown,
>         .driver.pm = IWL_PM_OPS,
>  };
>
> --
> 2.40.1
>



From jarkko at kernel.org  Thu Aug  3 01:57:11 2023
From: jarkko at kernel.org (Jarkko Sakkinen)
Date: Thu, 03 Aug 2023 11:57:11 +0300
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <8c4257fb07a8c9fd26b70ce716665a9535e6af54.camel@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
 <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
 <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
 <8c4257fb07a8c9fd26b70ce716665a9535e6af54.camel@linux.ibm.com>
Message-ID: <CUIS9Q2RKN2Q.4V9YYAYHE2BD@suppilovahvero>

On Thu Aug 3, 2023 at 4:22 AM EEST, Mimi Zohar wrote:
> On Wed, 2023-08-02 at 06:58 +0300, Jarkko Sakkinen wrote:
> > 
> > From long description I see zero motivation to ack this change, except
> > some heresay about IMA requiring it. Why does IMA need update_cnt and
> > why this is not documented to the long description?
>
> The motivation is to detect whether the IMA measurement list has been
> truncated, for whatever reason.  A new IMA record should be defined
> containing the "pcrCounter" value.  (I have not had a chance to review
> this patch set.)
>
> This new record would be a pre-req for both Tushar's "ima: measure
> events between kexec load and execute" patch set and Sush's proposal to
> trim the measurement list.  (I haven't looked at it yet either.)

Please describe the story in a bit more understandable form. In the
commit messages it is not good to have some redundancy in patch sets.

BR, Jarkko


From ardb at kernel.org  Thu Aug  3 04:11:54 2023
From: ardb at kernel.org (Ard Biesheuvel)
Date: Thu, 3 Aug 2023 13:11:54 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <20230802155146.GCZMp7ksDdN2ETVzKV@fat_crate.local>
References: <20230713100459.GEZK/MS69XbphJa+tN@fat_crate.local>
 <CAO7dBbVMNKTSDi5eP4BseEUexsk0Mo0GWJpyHfOcp+tHs6cSUw@mail.gmail.com>
 <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local> <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local> <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local> <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
 <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local> <CAMj1kXEM5hGknVGwHh_w99D4L8yrYrTFycwGHZ0CQun70CLipw@mail.gmail.com>
 <20230802155146.GCZMp7ksDdN2ETVzKV@fat_crate.local>
Message-ID: <CAMj1kXHnSzdQw5CMAVXU7EzpnrdRvAqKZVgA+EV35kHmwVULgQ@mail.gmail.com>

On Wed, 2 Aug 2023 at 17:52, Borislav Petkov <bp at alien8.de> wrote:
>
> On Wed, Aug 02, 2023 at 04:55:27PM +0200, Ard Biesheuvel wrote:
> > ... because now, entering via startup_32 is broken, given that it only
> > maps the kernel image itself and relies on the #PF handling for
> > everything else it accesses, including firmware tables.
> >
> > AFAICT this also means that entering via startup_32 is broken entirely
> > for any configuration that enables the cc blob config table check,
> > regardless of the platform.
>
> Lemme brain-dump what Tom and I just talked on IRC.
>
> That startup_32 entry path for SNP guests was used with old grubs which
> used to enter through there and not anymore, reportedly. Which means,
> that must've worked at some point but Joerg would know. CCed.
>

Sadly, not only 'old' grubs - GRUB mainline only recently added
support for booting Linux/x86 via the EFI stub (because I wrote the
code for them), but it will still fall back to the previous mode for
kernels that are built without EFI stub support, or which are older
than ~v5.8 (because their EFI stub does not implement the generic EFI
initrd loading mechanism)

This fallback still appears to enter via startup_32, even when GRUB
itself runs in long mode in the context of EFI.

> Newer grubs enter through the 64-bit entry point and thus are fine
> - otherwise we would be seeing explosions left and right.
>

Yeah. what seems to be saving our ass here is that startup_32 maps the
first 1G of physical address space 4 times, and x86_64 EFI usually
puts firmware tables below 4G. This means the cc blob check doesn't
fault, but it may dereference bogus memory traversing the config table
array looking for the cc blob GUID. However, the system table field
holding the size of the array may also appear as bogus so this may
still break in weird ways.

> So dependent on what we wanna do, if we kill the 32-bit path, we can
> kill the 32-bit C-bit verif code. But that's for later and an item on my
> TODO list.
>

I don't think we can kill it yet, but it would be nice if we could
avoid the need to support SNP boot when entering that way.


From stefanb at linux.ibm.com  Thu Aug  3 06:37:38 2023
From: stefanb at linux.ibm.com (Stefan Berger)
Date: Thu, 3 Aug 2023 09:37:38 -0400
Subject: [PATCH 0/6] Measuring TPM update counter in IMA
In-Reply-To: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
Message-ID: <dbfb545b-9399-d505-6739-c2940930ef48@linux.ibm.com>



On 8/1/23 14:19, Tushar Sugandhi wrote:
> Entries in IMA log may be lost due to code bugs, certain error conditions

I hope we don't have such bugs. And I guess the most critical ones would be
between logging and PCR extensions

> being met etc.  This can result in TPM PCRs getting out of sync with the
> IMA log.  One such example is events between kexec 'load' and kexec
> 'execute' getting lost from the IMA log when the system soft-boots into
> the new Kernel using kexec[1].  The remote attestation service does not

Though this particular condition I thought would go away with your kexec series.

The other conditions would be an out-of-memory or TPM failure. The OOM would
probably be more critical since something that was supposed to be logged
couldn't be logged and now you cannot show this anymore and presumably not even
an error condition could be logged.

https://elixir.bootlin.com/linux/latest/source/security/integrity/ima/ima_queue.c#L179

> have any information if the PCR mismatch with IMA log is because of loss
> of entries in the IMA log or something else.  TPM 2.0 provides an update
> counter which is incremented each time a PCR is updated [2]. Measuring the
> TPM PCR update counter in IMA subsystem will help the remote attestation
> service to validate if there are any missing entries in the IMA log, when



> the system goes through certain important state changes (e.g. kexec soft
> boot, IMA log snapshotting etc.)
> 
> This patch series provides the required functionality to measure the
> update counter through IMA subsystem by -
>   - introducing a function to retrieve PCR update counter in the TPM
>     subsystem.
>   - IMA functionality to acquire the update counter from the TPM subsystem.
>   - Measuring the update counter at system boot and at kexec Kernel
>     load.

Then the bugs you mentioned above that may happen between system boot and kexec
load are still going to confuse anyone looking at the log and quote. I don't
think you should mention them. I thought you would provide a way to sync
up on every step...

Also, I thought you had a variable in your kexec series that would prevent all further
logging and measuring once the log had been marshalled during kexec 'exec' stage
and this wasn't necessary.

    Stefan

> 
> 
> This patch series would be a prerequisite for the next version of kexec
> load/execute series[1] and the future IMA log snapshotting patch series.
> 
> [1] https://lore.kernel.org/all/20230703215709.1195644-1-tusharsu at linux.microsoft.com/
>      ima: measure events between kexec load and execute
> 
> [2] https://trustedcomputinggroup.org/wp-content/uploads/TCG_TPM2_r1p59_Part3_Commands_pub.pdf
>      Section 22.4.2, Page 206.
> 
> Tushar Sugandhi (6):
>    tpm: implement TPM2 function to get update counter
>    tpm: provide functionality to get update counter
>    ima: get TPM update counter
>    ima: implement functionality to measure TPM update counter
>    ima: measure TPM update counter at ima_init
>    kexec: measure TPM update counter in ima log at kexec load
> 
>   drivers/char/tpm/tpm-interface.c   | 28 +++++++++++++++++
>   drivers/char/tpm/tpm.h             |  3 ++
>   drivers/char/tpm/tpm2-cmd.c        | 48 ++++++++++++++++++++++++++++++
>   include/linux/ima.h                |  1 +
>   include/linux/tpm.h                |  8 +++++
>   kernel/kexec_file.c                |  3 ++
>   security/integrity/ima/ima.h       |  2 ++
>   security/integrity/ima/ima_init.c  |  3 ++
>   security/integrity/ima/ima_main.c  | 29 ++++++++++++++++++
>   security/integrity/ima/ima_queue.c | 16 ++++++++++
>   10 files changed, 141 insertions(+)
> 


From ardb at kernel.org  Thu Aug  3 07:27:41 2023
From: ardb at kernel.org (Ard Biesheuvel)
Date: Thu, 3 Aug 2023 16:27:41 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <CAMj1kXHnSzdQw5CMAVXU7EzpnrdRvAqKZVgA+EV35kHmwVULgQ@mail.gmail.com>
References: <20230713100459.GEZK/MS69XbphJa+tN@fat_crate.local>
 <CAO7dBbVMNKTSDi5eP4BseEUexsk0Mo0GWJpyHfOcp+tHs6cSUw@mail.gmail.com>
 <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local> <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local> <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local> <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
 <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local> <CAMj1kXEM5hGknVGwHh_w99D4L8yrYrTFycwGHZ0CQun70CLipw@mail.gmail.com>
 <20230802155146.GCZMp7ksDdN2ETVzKV@fat_crate.local> <CAMj1kXHnSzdQw5CMAVXU7EzpnrdRvAqKZVgA+EV35kHmwVULgQ@mail.gmail.com>
Message-ID: <CAMj1kXESuCScMLLAS4tSDcYxA3JTb24RuF7ipcKGd65tBvOBWQ@mail.gmail.com>

On Thu, 3 Aug 2023 at 13:11, Ard Biesheuvel <ardb at kernel.org> wrote:
>
> On Wed, 2 Aug 2023 at 17:52, Borislav Petkov <bp at alien8.de> wrote:
> >
> > On Wed, Aug 02, 2023 at 04:55:27PM +0200, Ard Biesheuvel wrote:
> > > ... because now, entering via startup_32 is broken, given that it only
> > > maps the kernel image itself and relies on the #PF handling for
> > > everything else it accesses, including firmware tables.
> > >
> > > AFAICT this also means that entering via startup_32 is broken entirely
> > > for any configuration that enables the cc blob config table check,
> > > regardless of the platform.
> >
> > Lemme brain-dump what Tom and I just talked on IRC.
> >
> > That startup_32 entry path for SNP guests was used with old grubs which
> > used to enter through there and not anymore, reportedly. Which means,
> > that must've worked at some point but Joerg would know. CCed.
> >
>
> Sadly, not only 'old' grubs - GRUB mainline only recently added
> support for booting Linux/x86 via the EFI stub (because I wrote the
> code for them), but it will still fall back to the previous mode for
> kernels that are built without EFI stub support, or which are older
> than ~v5.8 (because their EFI stub does not implement the generic EFI
> initrd loading mechanism)
>
> This fallback still appears to enter via startup_32, even when GRUB
> itself runs in long mode in the context of EFI.
>
> > Newer grubs enter through the 64-bit entry point and thus are fine
> > - otherwise we would be seeing explosions left and right.
> >
>
> Yeah. what seems to be saving our ass here is that startup_32 maps the
> first 1G of physical address space 4 times, and x86_64 EFI usually
> puts firmware tables below 4G. This means the cc blob check doesn't
> fault, but it may dereference bogus memory traversing the config table
> array looking for the cc blob GUID. However, the system table field
> holding the size of the array may also appear as bogus so this may
> still break in weird ways.
>
> > So dependent on what we wanna do, if we kill the 32-bit path, we can
> > kill the 32-bit C-bit verif code. But that's for later and an item on my
> > TODO list.
> >
>
> I don't think we can kill it yet, but it would be nice if we could
> avoid the need to support SNP boot when entering that way.

https://lists.gnu.org/archive/html/grub-devel/2023-08/msg00005.html

Coming to your distro any decade now!


From patchwork-bot+linux-riscv at kernel.org  Thu Aug  3 08:10:23 2023
From: patchwork-bot+linux-riscv at kernel.org (patchwork-bot+linux-riscv at kernel.org)
Date: Thu, 03 Aug 2023 15:10:23 +0000
Subject: [RESEND PATCH -fixes 1/2] riscv: Export va_kernel_pa_offset in
 vmcoreinfo
In-Reply-To: <20230724100917.309061-1-suagrfillet@gmail.com>
References: <20230724100917.309061-1-suagrfillet@gmail.com>
Message-ID: <169107542303.19238.12982353248491606719.git-patchwork-notify@kernel.org>

Hello:

This series was applied to riscv/linux.git (fixes)
by Palmer Dabbelt <palmer at rivosinc.com>:

On Mon, 24 Jul 2023 18:09:16 +0800 you wrote:
> Since RISC-V Linux v6.4, the commit 3335068f8721 ("riscv: Use
> PUD/P4D/PGD pages for the linear mapping") changes phys_ram_base
> from the physical start of the kernel to the actual start of the DRAM.
> 
> The Crash-utility's VTOP() still uses phys_ram_base and kernel_map.virt_addr
> to translate kernel virtual address, that failed the Crash with Linux v6.4 [1].
> 
> [...]

Here is the summary with links:
  - [RESEND,-fixes,1/2] riscv: Export va_kernel_pa_offset in vmcoreinfo
    https://git.kernel.org/riscv/c/fbe7d19d2b7f
  - [RESEND,-fixes,2/2] Documentation: kdump: Add va_kernel_pa_offset for RISCV64
    https://git.kernel.org/riscv/c/640c503d7dbd

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html




From patchwork-bot+linux-riscv at kernel.org  Thu Aug  3 08:10:22 2023
From: patchwork-bot+linux-riscv at kernel.org (patchwork-bot+linux-riscv at kernel.org)
Date: Thu, 03 Aug 2023 15:10:22 +0000
Subject: [PATCH v2 0/3] RISC-V: Fix a few kexec_file_load(2) failures
In-Reply-To: <cover.1690365011.git.petr.tesarik.ext@huawei.com>
References: <cover.1690365011.git.petr.tesarik.ext@huawei.com>
Message-ID: <169107542294.19238.10753003765050685366.git-patchwork-notify@kernel.org>

Hello:

This series was applied to riscv/linux.git (fixes)
by Palmer Dabbelt <palmer at rivosinc.com>:

On Wed, 26 Jul 2023 11:53:58 +0200 you wrote:
> From: Petr Tesarik <petr.tesarik.ext at huawei.com>
> 
> The kexec_file_load(2) syscall does not work at least in some kernel
> builds. For details see the relevant section in this blog post:
> 
> https://sigillatum.tesarici.cz/2023-07-21-state-of-riscv64-kdump.html
> 
> [...]

Here is the summary with links:
  - [v2,1/3] riscv/kexec: handle R_RISCV_CALL_PLT relocation type
    https://git.kernel.org/riscv/c/1be0b05b3a80
  - [v2,2/3] riscv/purgatory: do not link with string.o and its dependencies
    (no matching commit)
  - [v2,3/3] riscv/kexec: load initrd high in available memory
    https://git.kernel.org/riscv/c/0ccd2e803745

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html




From palmer at dabbelt.com  Thu Aug  3 08:13:48 2023
From: palmer at dabbelt.com (Palmer Dabbelt)
Date: Thu, 03 Aug 2023 08:13:48 -0700 (PDT)
Subject: [PATCH v2 2/3] riscv/purgatory: do not link with string.o and its dependencies
In-Reply-To: <20230726-excavate-swoosh-424ede98f63d@spud>
Message-ID: <mhng-45e67c5d-6969-43da-9147-b53dc9a78612@palmer-ri-x1c9>

On Wed, 26 Jul 2023 09:33:49 PDT (-0700), Conor Dooley wrote:
> On Wed, Jul 26, 2023 at 11:54:00AM +0200, Petr Tesarik wrote:
>> From: Petr Tesarik <petr.tesarik.ext at huawei.com>
>> 
>> Linking with this object file makes kexec_file_load(2) fail because of
>> multiple unknown relocation types:
>> 
>> - R_RISCV_ADD16, R_RISCV_SUB16: used by alternatives in strcmp()
>> - R_RISCV_GOT_HI20: used to resolve _ctype in strcasecmp()
>> 
>> All this hassle is needed for one single call to memcmp() from
>> verify_sha256_digest() to compare 32 bytes of SHA256 checksum.
>> 
>> Simply replace this memcmp() call with an explicit loop over those 32 bytes
>> and remove the need to link with string.o and all the other object files
>> that provide undefined symbols from that object file.
>> 
>> Fixes: 838b3e28488f ("RISC-V: Load purgatory in kexec_file")
>> Signed-off-by: Petr Tesarik <petr.tesarik.ext at huawei.com>
>
> This version keeps the automation happy,
> Acked-by: Conor Dooley <conor.dooley at microchip.com>

Oddly enough, this breaks my builds.  I tried fixing the first one with
something like

    From 41c5a952f77e53bf4201296abff0132725aa19e6 Mon Sep 17 00:00:00 2001
    From: Palmer Dabbelt <palmer at rivosinc.com>
    Date: Wed, 2 Aug 2023 20:22:33 -0700
    Subject: [PATCH] RISC-V: Include io from timex
    
    Without this I get some implicit declarations.
    
      CC      arch/riscv/kernel/asm-offsets.s
    In file included from linux/include/linux/timex.h:67,
                     from linux/include/linux/time32.h:13,
                     from linux/include/linux/time.h:60,
                     from linux/include/linux/ktime.h:24,
                     from linux/include/linux/timer.h:6,
                     from linux/include/linux/workqueue.h:9,
                     from linux/include/linux/mm_types.h:19,
                     from linux/include/linux/mmzone.h:22,
                     from linux/include/linux/gfp.h:7,
                     from linux/include/linux/mm.h:7,
                     from linux/arch/riscv/kernel/asm-offsets.c:10:
    linux/arch/riscv/include/asm/timex.h: In function 'get_cycles':
    linux/arch/riscv/include/asm/timex.h:25:16: error: implicit declaration of function 'readl_relaxed' [-Werror=implicit-function-declaration]
       25 |         return readl_relaxed(((u32 *)clint_time_val));
          |
    
    Signed-off-by: Palmer Dabbelt <palmer at rivosinc.com>
    ---
     arch/riscv/include/asm/timex.h | 1 +
     1 file changed, 1 insertion(+)
    
    diff --git a/arch/riscv/include/asm/timex.h b/arch/riscv/include/asm/timex.h
    index a06697846e69..1a4d181193e0 100644
    --- a/arch/riscv/include/asm/timex.h
    +++ b/arch/riscv/include/asm/timex.h
    @@ -7,6 +7,7 @@
     #define _ASM_RISCV_TIMEX_H
     
     #include <asm/csr.h>
    +#include <asm/io.h>
     
     typedef unsigned long cycles_t;
 
    -- 
    2.41.0

The other two look fine and are somewhat independent, so I've picked those up
for fixes.

Thanks!

>
> Thanks,
> Conor.


From palmer at rivosinc.com  Thu Aug  3 07:45:25 2023
From: palmer at rivosinc.com (Palmer Dabbelt)
Date: Thu, 03 Aug 2023 07:45:25 -0700
Subject: [RESEND PATCH -fixes 1/2] riscv: Export va_kernel_pa_offset in
 vmcoreinfo
In-Reply-To: <20230724100917.309061-1-suagrfillet@gmail.com>
References: <20230724100917.309061-1-suagrfillet@gmail.com>
Message-ID: <169107392509.27633.2884019260902221664.b4-ty@rivosinc.com>


On Mon, 24 Jul 2023 18:09:16 +0800, Song Shuai wrote:
> Since RISC-V Linux v6.4, the commit 3335068f8721 ("riscv: Use
> PUD/P4D/PGD pages for the linear mapping") changes phys_ram_base
> from the physical start of the kernel to the actual start of the DRAM.
> 
> The Crash-utility's VTOP() still uses phys_ram_base and kernel_map.virt_addr
> to translate kernel virtual address, that failed the Crash with Linux v6.4 [1].
> 
> [...]

Applied, thanks!

[1/2] riscv: Export va_kernel_pa_offset in vmcoreinfo
      https://git.kernel.org/palmer/c/fbe7d19d2b7f
[2/2] Documentation: kdump: Add va_kernel_pa_offset for RISCV64
      https://git.kernel.org/palmer/c/640c503d7dbd

Best regards,
-- 
Palmer Dabbelt <palmer at rivosinc.com>



From conor at kernel.org  Thu Aug  3 08:17:26 2023
From: conor at kernel.org (Conor Dooley)
Date: Thu, 3 Aug 2023 16:17:26 +0100
Subject: [PATCH v2 2/3] riscv/purgatory: do not link with string.o and
 its dependencies
In-Reply-To: <mhng-45e67c5d-6969-43da-9147-b53dc9a78612@palmer-ri-x1c9>
References: <20230726-excavate-swoosh-424ede98f63d@spud>
 <mhng-45e67c5d-6969-43da-9147-b53dc9a78612@palmer-ri-x1c9>
Message-ID: <20230803-buddhism-hardwood-eea45fdedd6e@spud>

On Thu, Aug 03, 2023 at 08:13:48AM -0700, Palmer Dabbelt wrote:
> On Wed, 26 Jul 2023 09:33:49 PDT (-0700), Conor Dooley wrote:
> > On Wed, Jul 26, 2023 at 11:54:00AM +0200, Petr Tesarik wrote:
> > > From: Petr Tesarik <petr.tesarik.ext at huawei.com>
> > > 
> > > Linking with this object file makes kexec_file_load(2) fail because of
> > > multiple unknown relocation types:
> > > 
> > > - R_RISCV_ADD16, R_RISCV_SUB16: used by alternatives in strcmp()
> > > - R_RISCV_GOT_HI20: used to resolve _ctype in strcasecmp()
> > > 
> > > All this hassle is needed for one single call to memcmp() from
> > > verify_sha256_digest() to compare 32 bytes of SHA256 checksum.
> > > 
> > > Simply replace this memcmp() call with an explicit loop over those 32 bytes
> > > and remove the need to link with string.o and all the other object files
> > > that provide undefined symbols from that object file.
> > > 
> > > Fixes: 838b3e28488f ("RISC-V: Load purgatory in kexec_file")
> > > Signed-off-by: Petr Tesarik <petr.tesarik.ext at huawei.com>
> > 
> > This version keeps the automation happy,
> > Acked-by: Conor Dooley <conor.dooley at microchip.com>
> 
> Oddly enough, this breaks my builds.  I tried fixing the first one with
> something like

You say "tried", does that mean it also didn't work?
What's the config?

Cheers,
Conor.

> 
>    From 41c5a952f77e53bf4201296abff0132725aa19e6 Mon Sep 17 00:00:00 2001
>    From: Palmer Dabbelt <palmer at rivosinc.com>
>    Date: Wed, 2 Aug 2023 20:22:33 -0700
>    Subject: [PATCH] RISC-V: Include io from timex
>    Without this I get some implicit declarations.
>      CC      arch/riscv/kernel/asm-offsets.s
>    In file included from linux/include/linux/timex.h:67,
>                     from linux/include/linux/time32.h:13,
>                     from linux/include/linux/time.h:60,
>                     from linux/include/linux/ktime.h:24,
>                     from linux/include/linux/timer.h:6,
>                     from linux/include/linux/workqueue.h:9,
>                     from linux/include/linux/mm_types.h:19,
>                     from linux/include/linux/mmzone.h:22,
>                     from linux/include/linux/gfp.h:7,
>                     from linux/include/linux/mm.h:7,
>                     from linux/arch/riscv/kernel/asm-offsets.c:10:
>    linux/arch/riscv/include/asm/timex.h: In function 'get_cycles':
>    linux/arch/riscv/include/asm/timex.h:25:16: error: implicit declaration of function 'readl_relaxed' [-Werror=implicit-function-declaration]
>       25 |         return readl_relaxed(((u32 *)clint_time_val));
>          |
>    Signed-off-by: Palmer Dabbelt <palmer at rivosinc.com>
>    ---
>     arch/riscv/include/asm/timex.h | 1 +
>     1 file changed, 1 insertion(+)
>    diff --git a/arch/riscv/include/asm/timex.h b/arch/riscv/include/asm/timex.h
>    index a06697846e69..1a4d181193e0 100644
>    --- a/arch/riscv/include/asm/timex.h
>    +++ b/arch/riscv/include/asm/timex.h
>    @@ -7,6 +7,7 @@
>     #define _ASM_RISCV_TIMEX_H
>     #include <asm/csr.h>
>    +#include <asm/io.h>
>     typedef unsigned long cycles_t;
> 
>    --    2.41.0
> 
> The other two look fine and are somewhat independent, so I've picked those up
> for fixes.
> 
> Thanks!
> 
> > 
> > Thanks,
> > Conor.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20230803/d08af415/attachment.sig>

From petrtesarik at huaweicloud.com  Thu Aug  3 08:33:37 2023
From: petrtesarik at huaweicloud.com (Petr Tesarik)
Date: Thu, 3 Aug 2023 17:33:37 +0200
Subject: [PATCH v2 2/3] riscv/purgatory: do not link with string.o and its
 dependencies
In-Reply-To: <mhng-45e67c5d-6969-43da-9147-b53dc9a78612@palmer-ri-x1c9>
References: <mhng-45e67c5d-6969-43da-9147-b53dc9a78612@palmer-ri-x1c9>
Message-ID: <f4d8f17e-384d-866b-5905-94339d41e8ef@huaweicloud.com>

On 8/3/2023 5:13 PM, Palmer Dabbelt wrote:
> On Wed, 26 Jul 2023 09:33:49 PDT (-0700), Conor Dooley wrote:
>> On Wed, Jul 26, 2023 at 11:54:00AM +0200, Petr Tesarik wrote:
>>> From: Petr Tesarik <petr.tesarik.ext at huawei.com>
>>>
>>> Linking with this object file makes kexec_file_load(2) fail because of
>>> multiple unknown relocation types:
>>>
>>> - R_RISCV_ADD16, R_RISCV_SUB16: used by alternatives in strcmp()
>>> - R_RISCV_GOT_HI20: used to resolve _ctype in strcasecmp()
>>>
>>> All this hassle is needed for one single call to memcmp() from
>>> verify_sha256_digest() to compare 32 bytes of SHA256 checksum.
>>>
>>> Simply replace this memcmp() call with an explicit loop over those 32
>>> bytes
>>> and remove the need to link with string.o and all the other object files
>>> that provide undefined symbols from that object file.
>>>
>>> Fixes: 838b3e28488f ("RISC-V: Load purgatory in kexec_file")
>>> Signed-off-by: Petr Tesarik <petr.tesarik.ext at huawei.com>
>>
>> This version keeps the automation happy,
>> Acked-by: Conor Dooley <conor.dooley at microchip.com>
> 
> Oddly enough, this breaks my builds.? I tried fixing the first one with
> something like

FTR I have no idea how a patch to the purgatory code and its Makefile
can have any effect on the compilation of asm-offsets.s in another
directory.

Petr T

> ?? From 41c5a952f77e53bf4201296abff0132725aa19e6 Mon Sep 17 00:00:00 2001
> ?? From: Palmer Dabbelt <palmer at rivosinc.com>
> ?? Date: Wed, 2 Aug 2023 20:22:33 -0700
> ?? Subject: [PATCH] RISC-V: Include io from timex
> ?? ?? Without this I get some implicit declarations.
> ?? ???? CC????? arch/riscv/kernel/asm-offsets.s
> ?? In file included from linux/include/linux/timex.h:67,
> ??????????????????? from linux/include/linux/time32.h:13,
> ??????????????????? from linux/include/linux/time.h:60,
> ??????????????????? from linux/include/linux/ktime.h:24,
> ??????????????????? from linux/include/linux/timer.h:6,
> ??????????????????? from linux/include/linux/workqueue.h:9,
> ??????????????????? from linux/include/linux/mm_types.h:19,
> ??????????????????? from linux/include/linux/mmzone.h:22,
> ??????????????????? from linux/include/linux/gfp.h:7,
> ??????????????????? from linux/include/linux/mm.h:7,
> ??????????????????? from linux/arch/riscv/kernel/asm-offsets.c:10:
> ?? linux/arch/riscv/include/asm/timex.h: In function 'get_cycles':
> ?? linux/arch/riscv/include/asm/timex.h:25:16: error: implicit
> declaration of function 'readl_relaxed'
> [-Werror=implicit-function-declaration]
> ????? 25 |???????? return readl_relaxed(((u32 *)clint_time_val));
> ???????? |
> ?? ?? Signed-off-by: Palmer Dabbelt <palmer at rivosinc.com>
> ?? ---
> ??? arch/riscv/include/asm/timex.h | 1 +
> ??? 1 file changed, 1 insertion(+)
> ?? ?? diff --git a/arch/riscv/include/asm/timex.h
> b/arch/riscv/include/asm/timex.h
> ?? index a06697846e69..1a4d181193e0 100644
> ?? --- a/arch/riscv/include/asm/timex.h
> ?? +++ b/arch/riscv/include/asm/timex.h
> ?? @@ -7,6 +7,7 @@
> ??? #define _ASM_RISCV_TIMEX_H
> ??? ??? #include <asm/csr.h>
> ?? +#include <asm/io.h>
> ??? ??? typedef unsigned long cycles_t;
> 
> ?? -- ?? 2.41.0
> 
> The other two look fine and are somewhat independent, so I've picked
> those up
> for fixes.
> 
> Thanks!
> 
>>
>> Thanks,
>> Conor.



From eric.devolder at oracle.com  Thu Aug  3 11:20:28 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Thu, 3 Aug 2023 13:20:28 -0500
Subject: [PATCH v25 01/10] drivers/base: refactor cpu.c to use
 .is_visible()
In-Reply-To: <6d2811f5-a5ee-a49d-012d-b519b2c6ee26@oracle.com>
References: <20230629192119.6613-1-eric.devolder@oracle.com>
 <20230629192119.6613-2-eric.devolder@oracle.com>
 <2023070342-human-spill-a62c@gregkh>
 <31c1393d-4285-0032-7675-737737d21f71@oracle.com>
 <6d2811f5-a5ee-a49d-012d-b519b2c6ee26@oracle.com>
Message-ID: <a6dd6e27-e8a7-09ac-f33d-03318dd5695c@oracle.com>



On 7/21/23 11:32, Eric DeVolder wrote:
> 
> 
> On 7/3/23 11:53, Eric DeVolder wrote:
>>
>>
>> On 7/3/23 08:05, Greg KH wrote:
>>> On Thu, Jun 29, 2023 at 03:21:10PM -0400, Eric DeVolder wrote:
>>>> ? - the function body of the callback functions are now wrapped with
>>>> ??? IS_ENABLED(); as the callback function must exist now that the
>>>> ??? attribute is always compiled-in (though not necessarily visible).
>>>
>>> Why do you need to do this last thing?? Is it a code savings goal?? Or
>>> something else?? The file will not be present in the system if the
>>> option is not enabled, so it should be safe to not do this unless you
>>> feel it's necessary for some reason?
>>
>> To accommodate the request, all DEVICE_ATTR() must be unconditionally present in this file. The 
>> DEVICE_ATTR() requires the .show() callback. As the callback is referenced from a data structure, 
>> the callback has to be present for link. All the callbacks for these attributes are in this file.
>>
>> I have two basic choices for gutting the function body if the config feature is not enabled. I can 
>> either use #ifdef or IS_ENABLED(). Thomas has made it clear I need to use IS_ENABLED(). I can 
>> certainly use #ifdef (which is what I did in v24).
>>
>>>
>>> Not doing this would make the diff easier to read :)
>>
>> I agree this is messy. I'm not really sure what this request/effort achieves as these attributes 
>> are not strongly related (unlike cacheinfo) and the way the file was before results in less code.
>>
>> At any rate, please indicate if you'd rather I use #ifdef.
>> Thanks for your time!
>> eric
>>
>>>
>>> thanks,
>>>
>>> greg k-h
> 
> Hi Greg,
> I was wondering if you might weigh-in so that I can proceed.
> 
> I think there are three options on the table:
> - use #ifdef to comment out these function bodies, which keeps the diff much more readable
> - use IS_ENABLED() as Thomas has requested I do, but makes the diff more difficult to read
> - remove this refactor altogether, perhaps post-poning until after this crash hotplug series merges, 
> as this refactor is largely unrelated to crash hotplug.
> 
> Thank you for your time on this topic!
> eric

Hi Greg,
If you have an opinion on how to proceed, please provide.
Thanks,
eric


From gregkh at linuxfoundation.org  Thu Aug  3 11:36:10 2023
From: gregkh at linuxfoundation.org (Greg KH)
Date: Thu, 3 Aug 2023 20:36:10 +0200
Subject: [PATCH v25 01/10] drivers/base: refactor cpu.c to use
 .is_visible()
In-Reply-To: <a6dd6e27-e8a7-09ac-f33d-03318dd5695c@oracle.com>
References: <20230629192119.6613-1-eric.devolder@oracle.com>
 <20230629192119.6613-2-eric.devolder@oracle.com>
 <2023070342-human-spill-a62c@gregkh>
 <31c1393d-4285-0032-7675-737737d21f71@oracle.com>
 <6d2811f5-a5ee-a49d-012d-b519b2c6ee26@oracle.com>
 <a6dd6e27-e8a7-09ac-f33d-03318dd5695c@oracle.com>
Message-ID: <2023080320-dinner-ravine-7d18@gregkh>

On Thu, Aug 03, 2023 at 01:20:28PM -0500, Eric DeVolder wrote:
> 
> 
> On 7/21/23 11:32, Eric DeVolder wrote:
> > 
> > 
> > On 7/3/23 11:53, Eric DeVolder wrote:
> > > 
> > > 
> > > On 7/3/23 08:05, Greg KH wrote:
> > > > On Thu, Jun 29, 2023 at 03:21:10PM -0400, Eric DeVolder wrote:
> > > > > ? - the function body of the callback functions are now wrapped with
> > > > > ??? IS_ENABLED(); as the callback function must exist now that the
> > > > > ??? attribute is always compiled-in (though not necessarily visible).
> > > > 
> > > > Why do you need to do this last thing?? Is it a code savings goal?? Or
> > > > something else?? The file will not be present in the system if the
> > > > option is not enabled, so it should be safe to not do this unless you
> > > > feel it's necessary for some reason?
> > > 
> > > To accommodate the request, all DEVICE_ATTR() must be
> > > unconditionally present in this file. The DEVICE_ATTR() requires the
> > > .show() callback. As the callback is referenced from a data
> > > structure, the callback has to be present for link. All the
> > > callbacks for these attributes are in this file.
> > > 
> > > I have two basic choices for gutting the function body if the config
> > > feature is not enabled. I can either use #ifdef or IS_ENABLED().
> > > Thomas has made it clear I need to use IS_ENABLED(). I can certainly
> > > use #ifdef (which is what I did in v24).
> > > 
> > > > 
> > > > Not doing this would make the diff easier to read :)
> > > 
> > > I agree this is messy. I'm not really sure what this request/effort
> > > achieves as these attributes are not strongly related (unlike
> > > cacheinfo) and the way the file was before results in less code.
> > > 
> > > At any rate, please indicate if you'd rather I use #ifdef.
> > > Thanks for your time!
> > > eric
> > > 
> > > > 
> > > > thanks,
> > > > 
> > > > greg k-h
> > 
> > Hi Greg,
> > I was wondering if you might weigh-in so that I can proceed.
> > 
> > I think there are three options on the table:
> > - use #ifdef to comment out these function bodies, which keeps the diff much more readable
> > - use IS_ENABLED() as Thomas has requested I do, but makes the diff more difficult to read
> > - remove this refactor altogether, perhaps post-poning until after this
> > crash hotplug series merges, as this refactor is largely unrelated to
> > crash hotplug.
> > 
> > Thank you for your time on this topic!
> > eric
> 
> Hi Greg,
> If you have an opinion on how to proceed, please provide.

Sorry, totally swamped by "stuff".  I don't know, use your judgement
here and send a new version, don't wait for me to weigh in on design
decisions for longer than a week.

thanks,

greg k-h


From tusharsu at linux.microsoft.com  Thu Aug  3 12:30:20 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Thu, 3 Aug 2023 12:30:20 -0700
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <CUIRWHEMXL38.29Z2XR564KV33@suppilovahvero>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
 <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
 <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
 <dd126081-3216-c15e-124e-ef137b376cdb@linux.microsoft.com>
 <CUIRWHEMXL38.29Z2XR564KV33@suppilovahvero>
Message-ID: <7913a162-3da3-61b4-49ce-8e762da47966@linux.microsoft.com>


On 8/3/23 01:43, Jarkko Sakkinen wrote:
> On Thu Aug 3, 2023 at 12:04 AM EEST, Tushar Sugandhi wrote:
>> Btw, the function tpm2_pcr_read is not exposed directly to the other
>> subsystems (like IMA).? It is exposed via tpm_pcr_read.
>>
>> Do you want to expose tpm2_pcr_read directly,
>> or do you want me to update the function signature of tpm_pcr_read as well?
> As long as you mention that 'update_cnt' causes divegence in the
> in-kernel API, and therefore tpm[12]_pcr_read() cannnot be under the
> same orchestrator.
Yup. I will mention that in the description/comment.
>
> If you take this path, please implement a precursory patch, which
> replace the existing call sites with some sequence of tpm[12]_pcr_read()
> and tpm_is_tpm2() calls.
Sure.? I will add a precursory patch which will replace the existing
call sites.

Thanks for confirming the approach.

~Tushar
> BR, Jarkko


From tusharsu at linux.microsoft.com  Thu Aug  3 12:31:24 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Thu, 3 Aug 2023 12:31:24 -0700
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <8c4257fb07a8c9fd26b70ce716665a9535e6af54.camel@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
 <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
 <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
 <8c4257fb07a8c9fd26b70ce716665a9535e6af54.camel@linux.ibm.com>
Message-ID: <a057a6c2-92c7-33f0-f03d-2f4e255ce04c@linux.microsoft.com>


On 8/2/23 18:22, Mimi Zohar wrote:
> On Wed, 2023-08-02 at 06:58 +0300, Jarkko Sakkinen wrote:
>>  From long description I see zero motivation to ack this change, except
>> some heresay about IMA requiring it. Why does IMA need update_cnt and
>> why this is not documented to the long description?
> The motivation is to detect whether the IMA measurement list has been
> truncated, for whatever reason.  A new IMA record should be defined
> containing the "pcrCounter" value.  (I have not had a chance to review
> this patch set.)
>
> This new record would be a pre-req for both Tushar's "ima: measure
> events between kexec load and execute" patch set and Sush's proposal to
> trim the measurement list.  (I haven't looked at it yet either.)
>
 ?Thanks Mimi. Take your time.


From tusharsu at linux.microsoft.com  Thu Aug  3 12:33:53 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Thu, 3 Aug 2023 12:33:53 -0700
Subject: [PATCH 1/6] tpm: implement TPM2 function to get update counter
In-Reply-To: <CUIS9Q2RKN2Q.4V9YYAYHE2BD@suppilovahvero>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-2-tusharsu@linux.microsoft.com>
 <CUHFWAAKMKJN.2EA3ZHLOOPPGB@suppilovahvero>
 <1d592ca1-1f6c-9eef-ce71-b07a837372b0@linux.microsoft.com>
 <CUHRAC79EW66.1VQRXS97GB0UO@suppilovahvero>
 <8c4257fb07a8c9fd26b70ce716665a9535e6af54.camel@linux.ibm.com>
 <CUIS9Q2RKN2Q.4V9YYAYHE2BD@suppilovahvero>
Message-ID: <d9e7297a-cda7-74e5-70a8-dd128f68f726@linux.microsoft.com>


On 8/3/23 01:57, Jarkko Sakkinen wrote:
> On Thu Aug 3, 2023 at 4:22 AM EEST, Mimi Zohar wrote:
>> On Wed, 2023-08-02 at 06:58 +0300, Jarkko Sakkinen wrote:
>>>  From long description I see zero motivation to ack this change, except
>>> some heresay about IMA requiring it. Why does IMA need update_cnt and
>>> why this is not documented to the long description?
>> The motivation is to detect whether the IMA measurement list has been
>> truncated, for whatever reason.  A new IMA record should be defined
>> containing the "pcrCounter" value.  (I have not had a chance to review
>> this patch set.)
>>
>> This new record would be a pre-req for both Tushar's "ima: measure
>> events between kexec load and execute" patch set and Sush's proposal to
>> trim the measurement list.  (I haven't looked at it yet either.)
> Please describe the story in a bit more understandable form. In the
> commit messages it is not good to have some redundancy in patch sets.
>
> BR, Jarkko
Thanks Jarkko.? I had provided the overall context in the cover letter.
But I understand the cover letter will be lost when the patches are
merged. I will describe the story in the patch descriptions as well,
in the next version of this series.

~Tushar


From zohar at linux.ibm.com  Thu Aug  3 14:42:16 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Thu, 03 Aug 2023 17:42:16 -0400
Subject: [PATCH 4/6] ima: implement functionality to measure TPM update
 counter
In-Reply-To: <20230801181917.8535-5-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
	 <20230801181917.8535-5-tusharsu@linux.microsoft.com>
Message-ID: <a2d9af7f54f364da9aeaef3dc6e03b7412957c23.camel@linux.ibm.com>

On Tue, 2023-08-01 at 11:19 -0700, Tushar Sugandhi wrote:
> Currently TPM update counter is not available external to the system,
> for instance, a remote attestation service.  It is a problem because
> the service cannot easily determine if the IMA log entries are missing.
> The IMA functionality needs to be extended to measure the TPM update
> counter from various subsystems in Linux kernel to help detect if
> the IMA log entries are missing.
> 
> Implement a function, 'ima_measure_update_counter()' which would retrieve
> the TPM update counter using the previously defined function
> 'ima_tpm_get_update_counter()'.  Format it as a string with the value 
> "update_counter=<N>;", and measure it using the function
> 'ima_measure_critical_data()'.
> 
> The function takes an event name as input, and the update counter value
> is measured as part of this event.
> 
> Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>

Explicit TPM2 quote commands do not return the quoted PCR values or the
pcrCounter value.  Define and include a new IMA measurement record
containing the pcrCounter, other TPM info, and IMA info in the IMA
measurement list to help simplify detecting a trimmed/truncated
measurement list.

> ---
>  include/linux/ima.h               |  1 +
>  security/integrity/ima/ima.h      |  1 +
>  security/integrity/ima/ima_main.c | 28 ++++++++++++++++++++++++++++
>  3 files changed, 30 insertions(+)
> 
> diff --git a/include/linux/ima.h b/include/linux/ima.h
> index 86b57757c7b1..f15f3a6a4c72 100644
> --- a/include/linux/ima.h
> +++ b/include/linux/ima.h
> @@ -40,6 +40,7 @@ extern int ima_measure_critical_data(const char *event_label,
>  				     const char *event_name,
>  				     const void *buf, size_t buf_len,
>  				     bool hash, u8 *digest, size_t digest_len);
> +int ima_measure_update_counter(const char *event_name);
>  
>  #ifdef CONFIG_IMA_APPRAISE_BOOTPARAM
>  extern void ima_appraise_parse_cmdline(void);
> diff --git a/security/integrity/ima/ima.h b/security/integrity/ima/ima.h
> index 4acd0e5a830f..5484bd362237 100644
> --- a/security/integrity/ima/ima.h
> +++ b/security/integrity/ima/ima.h
> @@ -168,6 +168,7 @@ int __init ima_init_digests(void);
>  int ima_lsm_policy_change(struct notifier_block *nb, unsigned long event,
>  			  void *lsm_data);
>  int ima_tpm_get_update_counter(u32 *cpu_update_counter);
> +int ima_measure_update_counter(const char *event_name);
>  
>  /*
>   * used to protect h_table and sha_table
> diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
> index d66a0a36415e..1bcd45cc5a6a 100644
> --- a/security/integrity/ima/ima_main.c
> +++ b/security/integrity/ima/ima_main.c
> @@ -1071,6 +1071,34 @@ int ima_measure_critical_data(const char *event_label,
>  }
>  EXPORT_SYMBOL_GPL(ima_measure_critical_data);
>  
> +#define IMA_TPM_UPDATE_CTR_BUF_SIZE 128
> +int ima_measure_update_counter(const char *event_name)
> +{
> +	int result;
> +	u32 update_counter = 0;
> +	char buf[IMA_TPM_UPDATE_CTR_BUF_SIZE];
> +	int buf_len;
> +
> +	if (!event_name)
> +		return -ENOPARAM;
> +
> +	result = ima_tpm_get_update_counter(&update_counter);
> +
> +	if (result != 0)
> +		return result;
> +
> +	scnprintf(buf, IMA_TPM_UPDATE_CTR_BUF_SIZE, "update_counter=%u;",
> +			  update_counter);
> +
> +	buf_len = strlen(buf);
> +
> +	result = ima_measure_critical_data("tpm_pcr_update_counter", event_name,
> +				  buf, buf_len, false, NULL, 0);
> 

The new record should contain everything needed to verify the
pcrCounter.  For example, each IMA measurement record updates the
pcrCounter for each TPM bank enabled.  So the number of enabled TPM
banks and number of IMA measurements should also be included in this
record.

Perhaps include a version number as well, so that if we ever want to
include other information, we could.

Mimi


> +
> +	return result;
> +}
> +EXPORT_SYMBOL_GPL(ima_measure_update_counter);
> +
>  static int __init init_ima(void)
>  {
>  	int error;




From tusharsu at linux.microsoft.com  Thu Aug  3 14:45:54 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Thu, 3 Aug 2023 14:45:54 -0700
Subject: [PATCH 0/6] Measuring TPM update counter in IMA
In-Reply-To: <dbfb545b-9399-d505-6739-c2940930ef48@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <dbfb545b-9399-d505-6739-c2940930ef48@linux.ibm.com>
Message-ID: <65bc533d-8775-579b-aae7-7ae0a936bf7e@linux.microsoft.com>

Thanks Stefan for reviewing this series. Appreciate it.

Re-sending this email. I accidentally had some HTML content, the email
bounced back from integrity mailing list.

On 8/3/23 06:37, Stefan Berger wrote:
>
>
> On 8/1/23 14:19, Tushar Sugandhi wrote:
>> Entries in IMA log may be lost due to code bugs, certain error 
>> conditions
>
> I hope we don't have such bugs. And I guess the most critical ones 
> would be
> between logging and PCR extensions
I hope so too, but in general, it?s hard to prove a negative.
That?s why this patch series. :)
>
>
>> being met etc. This can result in TPM PCRs getting out of sync with the
>> IMA log.? One such example is events between kexec 'load' and kexec
>> 'execute' getting lost from the IMA log when the system soft-boots into
>> the new Kernel using kexec[1].? The remote attestation service does not
>
> Though this particular condition I thought would go away with your 
> kexec series.
Currently the events in-between kexec ?load? and ?execute? are always
lost ? because IMA log is captured at ?load?.? My kexec series addresses
this scenario. But as you said, there is always a possibility that the
events will still be lost during kexec because of error conditions, OOM
etc.
>
> The other conditions would be an out-of-memory or TPM failure. The OOM 
> would
> probably be more critical since something that was supposed to be logged
> couldn't be logged and now you cannot show this anymore and presumably 
> not even
> an error condition could be logged.
>
Precisely. As you can see in patch 5 of this series, I am logging the
counter at ima_init (ima_init_tpm_update_counter).? So we will get the
baseline counter at the system boot, where memory pressure is hopefully
low.? Even if we are not able to log the counter later due to OOM, this
baseline counter along with the number of events in the IMA log should
help detect if IMA log is missing events.
> https://elixir.bootlin.com/linux/latest/source/security/integrity/ima/ima_queue.c#L179 
>
>
>> have any information if the PCR mismatch with IMA log is because of loss
>> of entries in the IMA log or something else.? TPM 2.0 provides an update
>> counter which is incremented each time a PCR is updated [2]. 
>> Measuring the
>> TPM PCR update counter in IMA subsystem will help the remote attestation
>> service to validate if there are any missing entries in the IMA log, 
>> when
>
>
>
>> the system goes through certain important state changes (e.g. kexec soft
>> boot, IMA log snapshotting etc.)
>>
>> This patch series provides the required functionality to measure the
>> update counter through IMA subsystem by -
>> ? - introducing a function to retrieve PCR update counter in the TPM
>> ??? subsystem.
>> ? - IMA functionality to acquire the update counter from the TPM 
>> subsystem.
>> ? - Measuring the update counter at system boot and at kexec Kernel
>> ??? load.
>
> Then the bugs you mentioned above that may happen between system boot 
> and kexec
> load are still going to confuse anyone looking at the log and quote. I 
> don't
> think you should mention them. I thought you would provide a way to sync
I used the kexec load-execute bug as an example to demonstrate the value of
measuring update counter.? There could be other examples which I am not
aware of.? As we discussed above, even when I fix the kexec bug ? there is
still a possibility that events may go missing in error/OOM cases.

I can remove the kexec example if it is causing confusion.
Please let me know.
> up on every step...
I don?t fully understand what you mean by ?provide a way to sync up
on every step?.? Could you please elaborate?
>
> Also, I thought you had a variable in your kexec series that would 
> prevent all further
> logging and measuring once the log had been marshalled during kexec 
> 'exec' stage
> and this wasn't necessary.
>
No, the variable suspend_ima_measurements[1] suspends the measurements
while copying the kexec buffer during kexec execute to ensure consistency
of the IMA measurements.? It doesn?t prevent all future logging. The
variable is reset and the IMA measurements resume when the system boots
into the new Kernel image.

[1] 
https://patchwork.kernel.org/project/linux-integrity/patch/20230703215709.1195644-10-tusharsu at linux.microsoft.com/

~Tushar
> ?? Stefan
>
>>
>>
>> This patch series would be a prerequisite for the next version of kexec
>> load/execute series[1] and the future IMA log snapshotting patch series.
>>
>> [1] 
>> https://lore.kernel.org/all/20230703215709.1195644-1-tusharsu at linux.microsoft.com/
>> ???? ima: measure events between kexec load and execute
>>
>> [2] 
>> https://trustedcomputinggroup.org/wp-content/uploads/TCG_TPM2_r1p59_Part3_Commands_pub.pdf
>> ???? Section 22.4.2, Page 206.
>>
>> Tushar Sugandhi (6):
>> ?? tpm: implement TPM2 function to get update counter
>> ?? tpm: provide functionality to get update counter
>> ?? ima: get TPM update counter
>> ?? ima: implement functionality to measure TPM update counter
>> ?? ima: measure TPM update counter at ima_init
>> ?? kexec: measure TPM update counter in ima log at kexec load
>>
>> ? drivers/char/tpm/tpm-interface.c?? | 28 +++++++++++++++++
>> ? drivers/char/tpm/tpm.h???????????? |? 3 ++
>> ? drivers/char/tpm/tpm2-cmd.c??????? | 48 ++++++++++++++++++++++++++++++
>> ? include/linux/ima.h??????????????? |? 1 +
>> ? include/linux/tpm.h??????????????? |? 8 +++++
>> ? kernel/kexec_file.c??????????????? |? 3 ++
>> ? security/integrity/ima/ima.h?????? |? 2 ++
>> ? security/integrity/ima/ima_init.c? |? 3 ++
>> ? security/integrity/ima/ima_main.c? | 29 ++++++++++++++++++
>> ? security/integrity/ima/ima_queue.c | 16 ++++++++++
>> ? 10 files changed, 141 insertions(+)
>>



From stefanb at linux.ibm.com  Thu Aug  3 15:09:59 2023
From: stefanb at linux.ibm.com (Stefan Berger)
Date: Thu, 3 Aug 2023 18:09:59 -0400
Subject: [PATCH 0/6] Measuring TPM update counter in IMA
In-Reply-To: <cb2029b8-d585-1c06-a0ac-15624cf70e28@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <dbfb545b-9399-d505-6739-c2940930ef48@linux.ibm.com>
 <cb2029b8-d585-1c06-a0ac-15624cf70e28@linux.microsoft.com>
Message-ID: <a4a5e40b-abc1-27fa-3984-cee18fb4522c@linux.ibm.com>



On 8/3/23 17:30, Tushar Sugandhi wrote:

> 
> Thanks Stefan for reviewing this series. Appreciate it.
> 
> On 8/3/23 06:37, Stefan Berger wrote:
>>
>>
>> On 8/1/23 14:19, Tushar Sugandhi wrote:
>>> Entries in IMA log may be lost due to code bugs, certain error conditions
>>
>> I hope we don't have such bugs. And I guess the most critical ones would be
>> between logging and PCR extensions
>>
> I hope so too, but in general, it?s hard to prove a negative.
> That?s why this patch series. :)
>>> being met etc.? This can result in TPM PCRs getting out of sync with the
>>> IMA log.? One such example is events between kexec 'load' and kexec
>>> 'execute' getting lost from the IMA log when the system soft-boots into
>>> the new Kernel using kexec[1].? The remote attestation service does not
>>
>> Though this particular condition I thought would go away with your kexec series.
>>
> Currently the events in-between kexec ?load? and ?execute? are always
> lost ? because IMA log is captured at ?load?.? My kexec series addresses
> this scenario. But as you said, there is always a possibility that the
> events will still be lost during kexec because of error conditions, OOM
> etc.
>> The other conditions would be an out-of-memory or TPM failure. The OOM would
>> probably be more critical since something that was supposed to be logged
>> couldn't be logged and now you cannot show this anymore and presumably not even
>> an error condition could be logged.
>>
> Precisely. As you can see in patch 5 of this series, I am logging the
> counter at ima_init (ima_init_tpm_update_counter).? So we will get the
> baseline counter at the system boot, where memory pressure is hopefully
> low.? Even if we are not able to log the counter later due to OOM, this
> baseline counter along with the number of events in the IMA log should
> help detect if IMA log is missing events.

How do you account for updates to other PCRs than PCR 10 that a user may use for whatever reason?
I think you would have to have the TPM driver count the updates for PCR 10.
Form what I can see there's one global PCR update counter for all PCRs and
all banks.

Also, if we hit an oom condition when logging then the PCR is not extended, which
is good since otherwise we would be hopelessly out-of-sync.


> 
>> https://elixir.bootlin.com/linux/latest/source/security/integrity/ima/ima_queue.c#L179
>>
>>> have any information if the PCR mismatch with IMA log is because of loss
>>> of entries in the IMA log or something else.? TPM 2.0 provides an update
>>> counter which is incremented each time a PCR is updated [2]. Measuring the
>>> TPM PCR update counter in IMA subsystem will help the remote attestation
>>> service to validate if there are any missing entries in the IMA log, when
>>
>>
>>
>>> the system goes through certain important state changes (e.g. kexec soft
>>> boot, IMA log snapshotting etc.)
>>>
>>> This patch series provides the required functionality to measure the
>>> update counter through IMA subsystem by -
>>> ? - introducing a function to retrieve PCR update counter in the TPM
>>> ??? subsystem.
>>> ? - IMA functionality to acquire the update counter from the TPM subsystem.
>>> ? - Measuring the update counter at system boot and at kexec Kernel
>>> ??? load.
>>
>> Then the bugs you mentioned above that may happen between system boot and kexec
>> load are still going to confuse anyone looking at the log and quote. I don't
>> think you should mention them. I thought you would provide a way to sync
> I used the kexec load-execute bug as an example to demonstrate the value of
> measuring update counter.? There could be other examples which I am not
> aware of.? As we discussed above, even when I fix the kexec bug ? there is
> still a possibility that events may go missing in error/OOM cases.

Yes, and we're not extending the PCRs then either, which would be catastrophic
if we were.

> 
> I can remove the kexec example if it is causing confusion.> Please let me know.

I am not convinced we need this series  ... :-( Your kexec series prevents
further logging and especially PCR extensions after the frozen measurement log
has been created and in ima_add_template_entry(), if we hit an oom condition,
then we luckily do not extend the PCR either. If either the log was to have one
more entry than number PCR extensions occurred or vice versa, then the remote
attestation service will see this mismatch no matter what and all the PCR update
counter won't help (and is generally not a good indicator for this purpose imo)
for it to recover from this. It's better to declare the system as un-trusted/
corrupted in this case then.

    Stefan


>> up on every step...
>>
> I don?t fully understand what you mean by ?provide a way to sync up
> on every step?.? Could you please elaborate?
> 
>> Also, I thought you had a variable in your kexec series that would prevent all further
>> logging and measuring once the log had been marshalled during kexec 'exec' stage
>> and this wasn't necessary.
>>
> No, the variable suspend_ima_measurements[1] suspends the measurements
> while copying the kexec buffer during kexec execute to ensure consistency
> of the IMA measurements.? It doesn?t prevent all future logging. The
> variable is reset and the IMA measurements resume when the system boots
> into the new Kernel image.
> 
> [1] https://patchwork.kernel.org/project/linux-integrity/patch/20230703215709.1195644-10-tusharsu at linux.microsoft.com/
> 
> ~Tushar
> 
>> Stefan
>>
>>>
>>>
>>> This patch series would be a prerequisite for the next version of kexec
>>> load/execute series[1] and the future IMA log snapshotting patch series.
>>>
>>> [1] https://lore.kernel.org/all/20230703215709.1195644-1-tusharsu at linux.microsoft.com/
>>> ???? ima: measure events between kexec load and execute
>>>
>>> [2] https://trustedcomputinggroup.org/wp-content/uploads/TCG_TPM2_r1p59_Part3_Commands_pub.pdf
>>> ???? Section 22.4.2, Page 206.
>>>
>>> Tushar Sugandhi (6):
>>> ?? tpm: implement TPM2 function to get update counter
>>> ?? tpm: provide functionality to get update counter
>>> ?? ima: get TPM update counter
>>> ?? ima: implement functionality to measure TPM update counter
>>> ?? ima: measure TPM update counter at ima_init
>>> ?? kexec: measure TPM update counter in ima log at kexec load
>>>
>>> ? drivers/char/tpm/tpm-interface.c?? | 28 +++++++++++++++++
>>> ? drivers/char/tpm/tpm.h???????????? |? 3 ++
>>> ? drivers/char/tpm/tpm2-cmd.c??????? | 48 ++++++++++++++++++++++++++++++
>>> ? include/linux/ima.h??????????????? |? 1 +
>>> ? include/linux/tpm.h??????????????? |? 8 +++++
>>> ? kernel/kexec_file.c??????????????? |? 3 ++
>>> ? security/integrity/ima/ima.h?????? |? 2 ++
>>> ? security/integrity/ima/ima_init.c? |? 3 ++
>>> ? security/integrity/ima/ima_main.c? | 29 ++++++++++++++++++
>>> ? security/integrity/ima/ima_queue.c | 16 ++++++++++
>>> ? 10 files changed, 141 insertions(+)
>>>


From zohar at linux.ibm.com  Thu Aug  3 15:15:55 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Thu, 03 Aug 2023 18:15:55 -0400
Subject: [PATCH 5/6] ima: measure TPM update counter at ima_init
In-Reply-To: <20230801181917.8535-6-tusharsu@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
	 <20230801181917.8535-6-tusharsu@linux.microsoft.com>
Message-ID: <223d5c5a655fec263895c991550e4291d1bbd4c9.camel@linux.ibm.com>

On Tue, 2023-08-01 at 11:19 -0700, Tushar Sugandhi wrote:
> IMA log entries can be lost due to a variety of causes, such as code bugs
> or error conditions, leading to a mismatch between TPM PCRs and
> the IMA log.  Measuring TPM PCR update counter during ima_init would
> provide a baseline counter for the number of times the TPM PCRs are
> updated.  The remote attestation service can compare this baseline
> counter with a subsequent measured one (e.g., post-kexec soft-boot) to
> identify if there are any lost IMA log events.
> 
> Measure the TPM update counter at ima init.

No need for separate patches for one line changes like this.  Either
merge patches 5/6 and 6/6 or all three 4/6, 5/6, 6/6 together.

> 
> Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
> ---
>  security/integrity/ima/ima_init.c | 3 +++
>  security/integrity/ima/ima_main.c | 1 +
>  2 files changed, 4 insertions(+)
> 
> diff --git a/security/integrity/ima/ima_init.c b/security/integrity/ima/ima_init.c
> index 63979aefc95f..9bb18d6c2fd6 100644
> --- a/security/integrity/ima/ima_init.c
> +++ b/security/integrity/ima/ima_init.c
> @@ -154,5 +154,8 @@ int __init ima_init(void)
>  				  UTS_RELEASE, strlen(UTS_RELEASE), false,
>  				  NULL, 0);
>  
> +	/* Measures TPM update counter at ima_init */
> +	ima_measure_update_counter("ima_init_tpm_update_counter");
> +

With "ima_policy=critical_data" on the boot command line, the IMA
measurement list record looks like:

6e190cc643ff0b718485966a0300473baedface735 ima_init_tpm_update_counter 7570646174655f636f756e7465723d3330383b

Please change the "ima_init_tpm_update_counter" to something shorter
and the hex encoded ascii string and pcr counter to something readable.
Perhaps name this critical-data "tpm" and "tpm-info", similar to the
SELinux "selinux" and "selinux-state".  Then again, if this is TPM
critical-data we should rethink what other info should be included.

>  	return rc;
>  }
> diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
> index 1bcd45cc5a6a..93357c245e82 100644
> --- a/security/integrity/ima/ima_main.c
> +++ b/security/integrity/ima/ima_main.c
> @@ -1035,6 +1035,7 @@ void ima_kexec_cmdline(int kernel_fd, const void *buf, int size)
>  				   buf, size, "kexec-cmdline", KEXEC_CMDLINE, 0,
>  				   NULL, false, NULL, 0);
>  	fdput(f);
> +
>  }
>  
>  /**

Unnecessary change.

-- 
thanks,

Mimi



From zohar at linux.ibm.com  Thu Aug  3 15:36:53 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Thu, 03 Aug 2023 18:36:53 -0400
Subject: [PATCH 0/6] Measuring TPM update counter in IMA
In-Reply-To: <a4a5e40b-abc1-27fa-3984-cee18fb4522c@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
	 <dbfb545b-9399-d505-6739-c2940930ef48@linux.ibm.com>
	 <cb2029b8-d585-1c06-a0ac-15624cf70e28@linux.microsoft.com>
	 <a4a5e40b-abc1-27fa-3984-cee18fb4522c@linux.ibm.com>
Message-ID: <d51cd6959472885a59fc13b863b71f2157fc8f65.camel@linux.ibm.com>

On Thu, 2023-08-03 at 18:09 -0400, Stefan Berger wrote:
> > I can remove the kexec example if it is causing confusion.> Please let me know.
> 
> I am not convinced we need this series  ... :-( Your kexec series prevents
> further logging and especially PCR extensions after the frozen measurement log
> has been created and in ima_add_template_entry(), if we hit an oom condition,
> then we luckily do not extend the PCR either. If either the log was to have one
> more entry than number PCR extensions occurred or vice versa, then the remote
> attestation service will see this mismatch no matter what and all the PCR update
> counter won't help (and is generally not a good indicator for this purpose imo)
> for it to recover from this. It's better to declare the system as un-trusted/
> corrupted in this case then.

As previously mentioned, there is a patch set that doesn't carry any
records across kexec, if the the measurement list is too large, and
another proposal to trim the measurement list.

In both of these cases including a new IMA mesaurement record, at least
after the boot_aggregate, would help simplify detecting whether the
measurement list has been trimmed/truncated.

-- 
thanks,

Mimi



From stefanb at linux.ibm.com  Thu Aug  3 15:55:43 2023
From: stefanb at linux.ibm.com (Stefan Berger)
Date: Thu, 3 Aug 2023 18:55:43 -0400
Subject: [PATCH 0/6] Measuring TPM update counter in IMA
In-Reply-To: <d51cd6959472885a59fc13b863b71f2157fc8f65.camel@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <dbfb545b-9399-d505-6739-c2940930ef48@linux.ibm.com>
 <cb2029b8-d585-1c06-a0ac-15624cf70e28@linux.microsoft.com>
 <a4a5e40b-abc1-27fa-3984-cee18fb4522c@linux.ibm.com>
 <d51cd6959472885a59fc13b863b71f2157fc8f65.camel@linux.ibm.com>
Message-ID: <dc883eeb-729d-f951-ba1b-9c030eed8b93@linux.ibm.com>



On 8/3/23 18:36, Mimi Zohar wrote:
> On Thu, 2023-08-03 at 18:09 -0400, Stefan Berger wrote:
>>> I can remove the kexec example if it is causing confusion.> Please let me know.
>>
>> I am not convinced we need this series  ... :-( Your kexec series prevents
>> further logging and especially PCR extensions after the frozen measurement log
>> has been created and in ima_add_template_entry(), if we hit an oom condition,
>> then we luckily do not extend the PCR either. If either the log was to have one
>> more entry than number PCR extensions occurred or vice versa, then the remote
>> attestation service will see this mismatch no matter what and all the PCR update
>> counter won't help (and is generally not a good indicator for this purpose imo)
>> for it to recover from this. It's better to declare the system as un-trusted/
>> corrupted in this case then.
> 
> As previously mentioned, there is a patch set that doesn't carry any
> records across kexec, if the the measurement list is too large, and
> another proposal to trim the measurement list.
> 
> In both of these cases including a new IMA mesaurement record, at least
> after the boot_aggregate, would help simplify detecting whether the
> measurement list has been trimmed/truncated.
> 

And if you can detect that I would log an event but not using the PCR update counter.
Unless the state of PCRs is also logged, it's going to be unrecoverable for a log+quote
verifier from there.

    Stefan


From tusharsu at linux.microsoft.com  Thu Aug  3 16:01:56 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Thu, 3 Aug 2023 16:01:56 -0700
Subject: [PATCH 4/6] ima: implement functionality to measure TPM update
 counter
In-Reply-To: <a2d9af7f54f364da9aeaef3dc6e03b7412957c23.camel@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-5-tusharsu@linux.microsoft.com>
 <a2d9af7f54f364da9aeaef3dc6e03b7412957c23.camel@linux.ibm.com>
Message-ID: <b9a77076-d6ac-a1ec-5e75-ab066dfade69@linux.microsoft.com>

Thanks for the review Mimi.

On 8/3/23 14:42, Mimi Zohar wrote:
> On Tue, 2023-08-01 at 11:19 -0700, Tushar Sugandhi wrote:
>> Currently TPM update counter is not available external to the system,
>> for instance, a remote attestation service.  It is a problem because
>> the service cannot easily determine if the IMA log entries are missing.
>> The IMA functionality needs to be extended to measure the TPM update
>> counter from various subsystems in Linux kernel to help detect if
>> the IMA log entries are missing.
>>
>> Implement a function, 'ima_measure_update_counter()' which would retrieve
>> the TPM update counter using the previously defined function
>> 'ima_tpm_get_update_counter()'.  Format it as a string with the value
>> "update_counter=<N>;", and measure it using the function
>> 'ima_measure_critical_data()'.
>>
>> The function takes an event name as input, and the update counter value
>> is measured as part of this event.
>>
>> Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
> Explicit TPM2 quote commands do not return the quoted PCR values or the
> pcrCounter value.  Define and include a new IMA measurement record
> containing the pcrCounter, other TPM info, and IMA info in the IMA
> measurement list to help simplify detecting a trimmed/truncated
> measurement list.
Sounds good.
>> ---
>>   include/linux/ima.h               |  1 +
>>   security/integrity/ima/ima.h      |  1 +
>>   security/integrity/ima/ima_main.c | 28 ++++++++++++++++++++++++++++
>>   3 files changed, 30 insertions(+)
>>
>> diff --git a/include/linux/ima.h b/include/linux/ima.h
>> index 86b57757c7b1..f15f3a6a4c72 100644
>> --- a/include/linux/ima.h
>> +++ b/include/linux/ima.h
>> @@ -40,6 +40,7 @@ extern int ima_measure_critical_data(const char *event_label,
>>   				     const char *event_name,
>>   				     const void *buf, size_t buf_len,
>>   				     bool hash, u8 *digest, size_t digest_len);
>> +int ima_measure_update_counter(const char *event_name);
>>   
>>   #ifdef CONFIG_IMA_APPRAISE_BOOTPARAM
>>   extern void ima_appraise_parse_cmdline(void);
>> diff --git a/security/integrity/ima/ima.h b/security/integrity/ima/ima.h
>> index 4acd0e5a830f..5484bd362237 100644
>> --- a/security/integrity/ima/ima.h
>> +++ b/security/integrity/ima/ima.h
>> @@ -168,6 +168,7 @@ int __init ima_init_digests(void);
>>   int ima_lsm_policy_change(struct notifier_block *nb, unsigned long event,
>>   			  void *lsm_data);
>>   int ima_tpm_get_update_counter(u32 *cpu_update_counter);
>> +int ima_measure_update_counter(const char *event_name);
>>   
>>   /*
>>    * used to protect h_table and sha_table
>> diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
>> index d66a0a36415e..1bcd45cc5a6a 100644
>> --- a/security/integrity/ima/ima_main.c
>> +++ b/security/integrity/ima/ima_main.c
>> @@ -1071,6 +1071,34 @@ int ima_measure_critical_data(const char *event_label,
>>   }
>>   EXPORT_SYMBOL_GPL(ima_measure_critical_data);
>>   
>> +#define IMA_TPM_UPDATE_CTR_BUF_SIZE 128
>> +int ima_measure_update_counter(const char *event_name)
>> +{
>> +	int result;
>> +	u32 update_counter = 0;
>> +	char buf[IMA_TPM_UPDATE_CTR_BUF_SIZE];
>> +	int buf_len;
>> +
>> +	if (!event_name)
>> +		return -ENOPARAM;
>> +
>> +	result = ima_tpm_get_update_counter(&update_counter);
>> +
>> +	if (result != 0)
>> +		return result;
>> +
>> +	scnprintf(buf, IMA_TPM_UPDATE_CTR_BUF_SIZE, "update_counter=%u;",
>> +			  update_counter);
>> +
>> +	buf_len = strlen(buf);
>> +
>> +	result = ima_measure_critical_data("tpm_pcr_update_counter", event_name,
>> +				  buf, buf_len, false, NULL, 0);
>>
> The new record should contain everything needed to verify the
> pcrCounter.  For example, each IMA measurement record updates the
> pcrCounter for each TPM bank enabled.  So the number of enabled TPM
> banks and number of IMA measurements should also be included in this
> record.
Agreed. That should be valuable information.
How does the below format look like for the buf above?

version=<N>.<N>.<N>;num_enabled_pcr_banks=<N>;pcrUpdateCounter=<N>;num_ima_measurements=<N>;

>
> Perhaps include a version number as well, so that if we ever want to
> include other information, we could.
By version number, do you mean kernel_version, or a new version
number specific to this record? Or something else?

~Tushar
> Mimi
>
>
>> +
>> +	return result;
>> +}
>> +EXPORT_SYMBOL_GPL(ima_measure_update_counter);
>> +
>>   static int __init init_ima(void)
>>   {
>>   	int error;



From tusharsu at linux.microsoft.com  Thu Aug  3 16:34:56 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Thu, 3 Aug 2023 16:34:56 -0700
Subject: [PATCH 5/6] ima: measure TPM update counter at ima_init
In-Reply-To: <223d5c5a655fec263895c991550e4291d1bbd4c9.camel@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-6-tusharsu@linux.microsoft.com>
 <223d5c5a655fec263895c991550e4291d1bbd4c9.camel@linux.ibm.com>
Message-ID: <da91b56a-83c8-d12e-71f8-887b6701f021@linux.microsoft.com>



On 8/3/23 15:15, Mimi Zohar wrote:
> On Tue, 2023-08-01 at 11:19 -0700, Tushar Sugandhi wrote:
>> IMA log entries can be lost due to a variety of causes, such as code bugs
>> or error conditions, leading to a mismatch between TPM PCRs and
>> the IMA log.  Measuring TPM PCR update counter during ima_init would
>> provide a baseline counter for the number of times the TPM PCRs are
>> updated.  The remote attestation service can compare this baseline
>> counter with a subsequent measured one (e.g., post-kexec soft-boot) to
>> identify if there are any lost IMA log events.
>>
>> Measure the TPM update counter at ima init.
> No need for separate patches for one line changes like this.  Either
> merge patches 5/6 and 6/6 or all three 4/6, 5/6, 6/6 together.
>
Sounds good.
I will merge 4/6, 5/6, 6/6 together.
>> Signed-off-by: Tushar Sugandhi <tusharsu at linux.microsoft.com>
>> ---
>>   security/integrity/ima/ima_init.c | 3 +++
>>   security/integrity/ima/ima_main.c | 1 +
>>   2 files changed, 4 insertions(+)
>>
>> diff --git a/security/integrity/ima/ima_init.c b/security/integrity/ima/ima_init.c
>> index 63979aefc95f..9bb18d6c2fd6 100644
>> --- a/security/integrity/ima/ima_init.c
>> +++ b/security/integrity/ima/ima_init.c
>> @@ -154,5 +154,8 @@ int __init ima_init(void)
>>   				  UTS_RELEASE, strlen(UTS_RELEASE), false,
>>   				  NULL, 0);
>>   
>> +	/* Measures TPM update counter at ima_init */
>> +	ima_measure_update_counter("ima_init_tpm_update_counter");
>> +
> With "ima_policy=critical_data" on the boot command line, the IMA
> measurement list record looks like:
>
> 6e190cc643ff0b718485966a0300473baedface735 ima_init_tpm_update_counter 7570646174655f636f756e7465723d3330383b
>
> Please change the "ima_init_tpm_update_counter" to something shorter
> and the hex encoded ascii string and pcr counter to something readable.
I believe you are seeing the above line in ascill_runtime_measurements log.

The ascii logging format is consistent with other event data for 
critical_data event e.g. kernel_version.
10 8f449175bbf88bc55fc1127466628c39a3957d15 ima-buf 
sha1:4acab4fbb08db663b7b7b4528e8729187d726782 kernel_version 
362e332e302d7263332b
10 f10678b63c4b2529339dff02282e63d9c6bb0385 ima-buf 
sha1:d8c187524412f74a961f2051a9529c009e700337 
ima_init_tpm_update_counter 7570646174655f636f756e7465723d3133303b

Entries in the binary runtime measurements look readable to me.

ima_init_tpm_update_counter update_counter=130;
...
kexec_load_tpm_update_counte rupdate_counter=133;

Please let me know if you still want me to change the format.

> Perhaps name this critical-data "tpm" and "tpm-info", similar to the
 From patch 4/6:
+??? result = ima_measure_critical_data("tpm_pcr_update_counter", 
event_name,
+????????????????? buf, buf_len, false, NULL, 0);

The critical_data event_label value is currently set to 
"tpm_pcr_update_counter".
I can rename event_label to "tpm-info", so that the admins can filter the
event in IMA policy based on the label if needed.

As you know, event_label doesn't appear in IMA log, it can appear in IMA 
policy.
Whereas event_name appears in IMA log.

I was thinking of using event_name to identify when was the info captured.
(e.g. ima_init, kexec_load, or at some other event in future).

We can either do
(a)
event_label = "tpm-info" event_name = "tpm-info-ima-init" | 
"tpm-info-kexec-load" | ...

-or-

(b)
event_label = "tpm" event_name = "tpm-info"
and event_data to describe the where/when this info was captured.
e.g.
version=<N>.<N>.<N>;num_enabled_pcr_banks=<N>;pcrUpdateCounter=<N>;num_ima_measurements=<N>;event=kexec_load;

Let me know if you would prefer option (a) or (b) or something else.


> SELinux "selinux" and "selinux-state".  Then again, if this is TPM
> critical-data we should rethink what other info should be included.
As you suggested in Patch 4/6, I will add version, number of enabled pcr 
banks,
pcrUpdateCounter, and num_ima_measurements. I think we should include 
the TPM
version as well (1 v/s 2).

Please let me know if you think of any other attribute to record.

>>   	return rc;
>>   }
>> diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
>> index 1bcd45cc5a6a..93357c245e82 100644
>> --- a/security/integrity/ima/ima_main.c
>> +++ b/security/integrity/ima/ima_main.c
>> @@ -1035,6 +1035,7 @@ void ima_kexec_cmdline(int kernel_fd, const void *buf, int size)
>>   				   buf, size, "kexec-cmdline", KEXEC_CMDLINE, 0,
>>   				   NULL, false, NULL, 0);
>>   	fdput(f);
>> +
>>   }
>>   
>>   /**
> Unnecessary change.
>
oops. Thanks for catching. Will fix.


~Tushar



From zohar at linux.ibm.com  Thu Aug  3 18:18:57 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Thu, 03 Aug 2023 21:18:57 -0400
Subject: [PATCH 5/6] ima: measure TPM update counter at ima_init
In-Reply-To: <da91b56a-83c8-d12e-71f8-887b6701f021@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
	 <20230801181917.8535-6-tusharsu@linux.microsoft.com>
	 <223d5c5a655fec263895c991550e4291d1bbd4c9.camel@linux.ibm.com>
	 <da91b56a-83c8-d12e-71f8-887b6701f021@linux.microsoft.com>
Message-ID: <e9ab98a823c2e6c536fb2acbe86229a0085bd34a.camel@linux.ibm.com>

On Thu, 2023-08-03 at 16:34 -0700, Tushar Sugandhi wrote:

> >> +++ b/security/integrity/ima/ima_init.c
> >> @@ -154,5 +154,8 @@ int __init ima_init(void)
> >>   				  UTS_RELEASE, strlen(UTS_RELEASE), false,
> >>   				  NULL, 0);
> >>   
> >> +	/* Measures TPM update counter at ima_init */
> >> +	ima_measure_update_counter("ima_init_tpm_update_counter");
> >> +
> > With "ima_policy=critical_data" on the boot command line, the IMA
> > measurement list record looks like:
> >
> > 6e190cc643ff0b718485966a0300473baedface735 ima_init_tpm_update_counter 7570646174655f636f756e7465723d3330383b
> >
> > Please change the "ima_init_tpm_update_counter" to something shorter
> > and the hex encoded ascii string and pcr counter to something readable.
> I believe you are seeing the above line in ascill_runtime_measurements log.

Yes, the ascii_runtime_measurements are suppose to be readable to the
end user.

> The ascii logging format is consistent with other event data for 
> critical_data event e.g. kernel_version.

Then you got it wrong.

> 10 8f449175bbf88bc55fc1127466628c39a3957d15 ima-buf 
> sha1:4acab4fbb08db663b7b7b4528e8729187d726782 kernel_version 
> 362e332e302d7263332b
> 10 f10678b63c4b2529339dff02282e63d9c6bb0385 ima-buf 
> sha1:d8c187524412f74a961f2051a9529c009e700337 
> ima_init_tpm_update_counter 7570646174655f636f756e7465723d3133303b
> 
> Entries in the binary runtime measurements look readable to me.

You've inverted the meaning of the ascii and binary runtime measurement
lists.  For comparison look at the ima-ng/ima-sig records.

> ima_init_tpm_update_counter update_counter=130;
> ...
> kexec_load_tpm_update_counte rupdate_counter=133;
> 
> Please let me know if you still want me to change the format.

OI course the ascii measurement list should be human readable.

> > Perhaps name this critical-data "tpm" and "tpm-info", similar to the
>  From patch 4/6:
> +    result = ima_measure_critical_data("tpm_pcr_update_counter", 
> event_name,
> +                  buf, buf_len, false, NULL, 0);
> 
> The critical_data event_label value is currently set to 
> "tpm_pcr_update_counter".

Why is the string so long?   Long strings or variables don't make the
code or logs more understandable.  Please shorten both the strings and
variables.

> I can rename event_label to "tpm-info", so that the admins can filter the
> event in IMA policy based on the label if needed.

The new record needs to be self containd and verifiable.  The
additional info I suggested were just examples.  Please take the time
to consider what needs to be included in the new record.  Decide
whether this is a TPM security critical data record.  Only then, decide
on the naming.

-- 
thanks,

Mimi




From zohar at linux.ibm.com  Thu Aug  3 18:22:35 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Thu, 03 Aug 2023 21:22:35 -0400
Subject: [PATCH 4/6] ima: implement functionality to measure TPM update
 counter
In-Reply-To: <b9a77076-d6ac-a1ec-5e75-ab066dfade69@linux.microsoft.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
	 <20230801181917.8535-5-tusharsu@linux.microsoft.com>
	 <a2d9af7f54f364da9aeaef3dc6e03b7412957c23.camel@linux.ibm.com>
	 <b9a77076-d6ac-a1ec-5e75-ab066dfade69@linux.microsoft.com>
Message-ID: <7da13140fd4ab43e797214f08bf7bca07640c238.camel@linux.ibm.com>

On Thu, 2023-08-03 at 16:01 -0700, Tushar Sugandhi wrote:
> >> +    scnprintf(buf, IMA_TPM_UPDATE_CTR_BUF_SIZE, "update_counter=%u;",
> >> +                      update_counter);
> >> +
> >> +    buf_len = strlen(buf);
> >> +
> >> +    result = ima_measure_critical_data("tpm_pcr_update_counter", event_name,
> >> +                              buf, buf_len, false, NULL, 0);
> >>
> > The new record should contain everything needed to verify the
> > pcrCounter.  For example, each IMA measurement record updates the
> > pcrCounter for each TPM bank enabled.  So the number of enabled TPM
> > banks and number of IMA measurements should also be included in this
> > record.
> Agreed. That should be valuable information.
> How does the below format look like for the buf above?
> 
> version=<N>.<N>.<N>;num_enabled_pcr_banks=<N>;pcrUpdateCounter=<N>;num_ima_measurements=<N>;

Refer to comment in 5/6.

> > Perhaps include a version number as well, so that if we ever want to
> > include other information, we could.
> By version number, do you mean kernel_version, or a new version
> number specific to this record? Or something else?

This is a record version type number.  The record format shouldn't
change, but we should be prepared for it changing.  A single number
should be fine.

-- 
thanks,

Mimi



From k-hagio-ab at nec.com  Thu Aug  3 21:42:35 2023
From: k-hagio-ab at nec.com (=?utf-8?B?SEFHSU8gS0FaVUhJVE8o6JCp5bC+44CA5LiA5LuBKQ==?=)
Date: Fri, 4 Aug 2023 04:42:35 +0000
Subject: [Crash-utility] RISCV64: Use va_kernel_pa_offset in VTOP()
In-Reply-To: <20230724040649.220279-1-suagrfillet@gmail.com>
References: <20230724040649.220279-1-suagrfillet@gmail.com>
Message-ID: <378ea919-5149-fb1a-02ec-3c0b41ebecc3@nec.com>

On 2023/07/24 13:06, Song Shuai wrote:
> Since RISC-V Linux v6.4, the commit 3335068f8721 ("riscv: Use
> PUD/P4D/PGD pages for the linear mapping") changes the
> phys_ram_base from the kernel_map.phys_addr to the start of DRAM.
> 
> The Crash's VTOP() still uses phys_ram_base and kernel_map.virt_addr
> to translate kernel virtual address, that made Crash boot failed with
> Linux v6.4 and later version.
> 
> Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can
> use "va_kernel_pa_offset" to translate the kernel virtual address in
> VTOP() correctly.
> 
> Signed-off-by: Song Shuai <suagrfillet at gmail.com>
> ---
> You can check/test the Linux changes from this link:
> https://github.com/sugarfillet/linux/commits/6.5-rc3-crash
> 
> And I'll send the Linux changes to riscv/for-next If you're ok with this patch.
> ---
>   defs.h    |  4 ++--
>   riscv64.c | 22 ++++++++++++++++++++++
>   2 files changed, 24 insertions(+), 2 deletions(-)
> 
> diff --git a/defs.h b/defs.h
> index 358f365..46b9857 100644
> --- a/defs.h
> +++ b/defs.h
> @@ -3662,8 +3662,7 @@ typedef signed int s32;
>   	ulong _X = X;									\
>   	(THIS_KERNEL_VERSION >= LINUX(5,13,0) &&					\
>   		(_X) >= machdep->machspec->kernel_link_addr) ?				\
> -		(((unsigned long)(_X)-(machdep->machspec->kernel_link_addr)) +		\
> -		 machdep->machspec->phys_base):						\
> +		((unsigned long)(_X)-(machdep->machspec->va_kernel_pa_offset)): 	\
>   		(((unsigned long)(_X)-(machdep->kvbase)) +				\
>   		 machdep->machspec->phys_base);						\
>   	})
> @@ -7021,6 +7020,7 @@ struct machine_specific {
>   	ulong modules_vaddr;
>   	ulong modules_end;
>   	ulong kernel_link_addr;
> +	ulong va_kernel_pa_offset;
>   
>   	ulong _page_present;
>   	ulong _page_read;
> diff --git a/riscv64.c b/riscv64.c
> index 6b9a688..b9e50b4 100644
> --- a/riscv64.c
> +++ b/riscv64.c
> @@ -418,6 +418,27 @@ error:
>   	error(FATAL, "cannot get vm layout\n");
>   }
>   
> +static void
> +riscv64_get_va_kernel_pa_offset(struct machine_specific *ms)
> +{
> +	unsigned long kernel_version = riscv64_get_kernel_version();
> +
> +	/*
> +	 * va_kernel_pa_offset is defined in Linux kernel since 6.5.
> +	 */
> +	if (kernel_version >= LINUX(6,5,0)) {

The kernel patches look accepted, so for the crash patch detail,

I think this first version check is not necessary, we can just use the 
vmcoreinfo entry if available.  With it, backporting the kernel patches 
to e.g. 6.4.0 will also be supported.

Thanks,
Kazu

> +		char *string;
> +		if ((string = pc->read_vmcoreinfo("NUMBER(va_kernel_pa_offset)"))) {
> +			ms->va_kernel_pa_offset = htol(string, QUIET, NULL);
> +			free(string);
> +		} else
> +			error(FATAL, "cannot read va_kernel_pa_offset\n");
> +	} else if (kernel_version >= LINUX(6,4,0))
> +		error(FATAL, "cannot determine va_kernel_pa_offset since Linux 6.4\n");
> +	else
> +		ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base;
> +}
> +
>   static int
>   riscv64_is_kvaddr(ulong vaddr)
>   {
> @@ -1352,6 +1373,7 @@ riscv64_init(int when)
>   		riscv64_get_struct_page_size(machdep->machspec);
>   		riscv64_get_va_bits(machdep->machspec);
>   		riscv64_get_va_range(machdep->machspec);
> +		riscv64_get_va_kernel_pa_offset(machdep->machspec);
>   
>   		pt_level_alloc(&machdep->pgd, "cannot malloc pgd space.");
>   		pt_level_alloc(&machdep->machspec->p4d, "cannot malloc p4d space.");

From suagrfillet at gmail.com  Fri Aug  4 02:15:59 2023
From: suagrfillet at gmail.com (Song Shuai)
Date: Fri,  4 Aug 2023 17:15:59 +0800
Subject: [Crash-utility PATCH V2] RISCV64: Use va_kernel_pa_offset in VTOP()
Message-ID: <20230804091559.3005820-1-suagrfillet@gmail.com>

Since RISC-V Linux v6.4, the commit 3335068f8721 ("riscv: Use
PUD/P4D/PGD pages for the linear mapping") changes phys_ram_base from
the physical start of the kernel to the actual start of the DRAM.

The Crash's VTOP() still uses phys_ram_base and kernel_map.virt_addr
to translate kernel virtual address, that made Crash boot failed with
Linux v6.4 and later version.

Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and backported
v6.4.0 stable, so Crash can use "va_kernel_pa_offset" to translate the
kernel virtual address in VTOP() correctly.

Signed-off-by: Song Shuai <suagrfillet at gmail.com>
---
Changes since V1:
  - remove unnecessary first kernel version check as Kazu suggested
  - amend the commit-msg as Alex suggested
---
 defs.h    |  4 ++--
 riscv64.c | 23 +++++++++++++++++++++++
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/defs.h b/defs.h
index 5ee60f1..c07e6d7 100644
--- a/defs.h
+++ b/defs.h
@@ -3662,8 +3662,7 @@ typedef signed int s32;
 	ulong _X = X;									\
 	(THIS_KERNEL_VERSION >= LINUX(5,13,0) &&					\
 		(_X) >= machdep->machspec->kernel_link_addr) ?				\
-		(((unsigned long)(_X)-(machdep->machspec->kernel_link_addr)) +		\
-		 machdep->machspec->phys_base):						\
+		((unsigned long)(_X)-(machdep->machspec->va_kernel_pa_offset)): 	\
 		(((unsigned long)(_X)-(machdep->kvbase)) +				\
 		 machdep->machspec->phys_base);						\
 	})
@@ -7021,6 +7020,7 @@ struct machine_specific {
 	ulong modules_vaddr;
 	ulong modules_end;
 	ulong kernel_link_addr;
+	ulong va_kernel_pa_offset;
 
 	ulong _page_present;
 	ulong _page_read;
diff --git a/riscv64.c b/riscv64.c
index 6b9a688..7b5dd3d 100644
--- a/riscv64.c
+++ b/riscv64.c
@@ -418,6 +418,28 @@ error:
 	error(FATAL, "cannot get vm layout\n");
 }
 
+static void
+riscv64_get_va_kernel_pa_offset(struct machine_specific *ms)
+{
+	unsigned long kernel_version = riscv64_get_kernel_version();
+
+	/*
+	 * Since Linux v6.4 phys_base is not the physical start of the kernel,
+	 * trying to use "va_kernel_pa_offset" to determine the offset between
+	 * kernel virtual and physical addresses.
+	 */
+	if (kernel_version >= LINUX(6,4,0)) {
+		char *string;
+		if ((string = pc->read_vmcoreinfo("NUMBER(va_kernel_pa_offset)"))) {
+			ms->va_kernel_pa_offset = htol(string, QUIET, NULL);
+			free(string);
+		} else
+			error(FATAL, "cannot read va_kernel_pa_offset\n");
+	}
+	else
+		ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base;
+}
+
 static int
 riscv64_is_kvaddr(ulong vaddr)
 {
@@ -1352,6 +1374,7 @@ riscv64_init(int when)
 		riscv64_get_struct_page_size(machdep->machspec);
 		riscv64_get_va_bits(machdep->machspec);
 		riscv64_get_va_range(machdep->machspec);
+		riscv64_get_va_kernel_pa_offset(machdep->machspec);
 
 		pt_level_alloc(&machdep->pgd, "cannot malloc pgd space.");
 		pt_level_alloc(&machdep->machspec->p4d, "cannot malloc p4d space.");
-- 
2.20.1



From suagrfillet at gmail.com  Fri Aug  4 02:20:55 2023
From: suagrfillet at gmail.com (Song Shuai)
Date: Fri, 4 Aug 2023 17:20:55 +0800
Subject: [Crash-utility] RISCV64: Use va_kernel_pa_offset in VTOP()
In-Reply-To: <378ea919-5149-fb1a-02ec-3c0b41ebecc3@nec.com>
References: <20230724040649.220279-1-suagrfillet@gmail.com>
 <378ea919-5149-fb1a-02ec-3c0b41ebecc3@nec.com>
Message-ID: <bd8be47f-4c46-6a47-970b-86a5d36e22c2@gmail.com>



? 2023/8/4 12:42, HAGIO KAZUHITO(?? ??) ??:
> On 2023/07/24 13:06, Song Shuai wrote:
>> Since RISC-V Linux v6.4, the commit 3335068f8721 ("riscv: Use
>> PUD/P4D/PGD pages for the linear mapping") changes the
>> phys_ram_base from the kernel_map.phys_addr to the start of DRAM.
>>
>> The Crash's VTOP() still uses phys_ram_base and kernel_map.virt_addr
>> to translate kernel virtual address, that made Crash boot failed with
>> Linux v6.4 and later version.
>>
>> Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and Crash can
>> use "va_kernel_pa_offset" to translate the kernel virtual address in
>> VTOP() correctly.
>>
>> Signed-off-by: Song Shuai <suagrfillet at gmail.com>
>> ---
>> You can check/test the Linux changes from this link:
>> https://github.com/sugarfillet/linux/commits/6.5-rc3-crash
>>
>> And I'll send the Linux changes to riscv/for-next If you're ok with this patch.
>> ---
>>    defs.h    |  4 ++--
>>    riscv64.c | 22 ++++++++++++++++++++++
>>    2 files changed, 24 insertions(+), 2 deletions(-)
>>
>> diff --git a/defs.h b/defs.h
>> index 358f365..46b9857 100644
>> --- a/defs.h
>> +++ b/defs.h
>> @@ -3662,8 +3662,7 @@ typedef signed int s32;
>>    	ulong _X = X;									\
>>    	(THIS_KERNEL_VERSION >= LINUX(5,13,0) &&					\
>>    		(_X) >= machdep->machspec->kernel_link_addr) ?				\
>> -		(((unsigned long)(_X)-(machdep->machspec->kernel_link_addr)) +		\
>> -		 machdep->machspec->phys_base):						\
>> +		((unsigned long)(_X)-(machdep->machspec->va_kernel_pa_offset)): 	\
>>    		(((unsigned long)(_X)-(machdep->kvbase)) +				\
>>    		 machdep->machspec->phys_base);						\
>>    	})
>> @@ -7021,6 +7020,7 @@ struct machine_specific {
>>    	ulong modules_vaddr;
>>    	ulong modules_end;
>>    	ulong kernel_link_addr;
>> +	ulong va_kernel_pa_offset;
>>    
>>    	ulong _page_present;
>>    	ulong _page_read;
>> diff --git a/riscv64.c b/riscv64.c
>> index 6b9a688..b9e50b4 100644
>> --- a/riscv64.c
>> +++ b/riscv64.c
>> @@ -418,6 +418,27 @@ error:
>>    	error(FATAL, "cannot get vm layout\n");
>>    }
>>    
>> +static void
>> +riscv64_get_va_kernel_pa_offset(struct machine_specific *ms)
>> +{
>> +	unsigned long kernel_version = riscv64_get_kernel_version();
>> +
>> +	/*
>> +	 * va_kernel_pa_offset is defined in Linux kernel since 6.5.
>> +	 */
>> +	if (kernel_version >= LINUX(6,5,0)) {
> 
> The kernel patches look accepted, so for the crash patch detail,
> 
> I think this first version check is not necessary, we can just use the
> vmcoreinfo entry if available.  With it, backporting the kernel patches
> to e.g. 6.4.0 will also be supported.
> 
ok, V2 is here:

https://lore.kernel.org/linux-riscv/20230804091559.3005820-1-suagrfillet at gmail.com/T/#u

> Thanks,
> Kazu
> 
>> +		char *string;
>> +		if ((string = pc->read_vmcoreinfo("NUMBER(va_kernel_pa_offset)"))) {
>> +			ms->va_kernel_pa_offset = htol(string, QUIET, NULL);
>> +			free(string);
>> +		} else
>> +			error(FATAL, "cannot read va_kernel_pa_offset\n");
>> +	} else if (kernel_version >= LINUX(6,4,0))
>> +		error(FATAL, "cannot determine va_kernel_pa_offset since Linux 6.4\n");
>> +	else
>> +		ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base;
>> +}
>> +
>>    static int
>>    riscv64_is_kvaddr(ulong vaddr)
>>    {
>> @@ -1352,6 +1373,7 @@ riscv64_init(int when)
>>    		riscv64_get_struct_page_size(machdep->machspec);
>>    		riscv64_get_va_bits(machdep->machspec);
>>    		riscv64_get_va_range(machdep->machspec);
>> +		riscv64_get_va_kernel_pa_offset(machdep->machspec);
>>    
>>    		pt_level_alloc(&machdep->pgd, "cannot malloc pgd space.");
>>    		pt_level_alloc(&machdep->machspec->p4d, "cannot malloc p4d space.");

-- 
Thanks
Song Shuai


From tusharsu at linux.microsoft.com  Fri Aug  4 10:11:57 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Fri, 4 Aug 2023 10:11:57 -0700
Subject: [PATCH 5/6] ima: measure TPM update counter at ima_init
In-Reply-To: <e9ab98a823c2e6c536fb2acbe86229a0085bd34a.camel@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-6-tusharsu@linux.microsoft.com>
 <223d5c5a655fec263895c991550e4291d1bbd4c9.camel@linux.ibm.com>
 <da91b56a-83c8-d12e-71f8-887b6701f021@linux.microsoft.com>
 <e9ab98a823c2e6c536fb2acbe86229a0085bd34a.camel@linux.ibm.com>
Message-ID: <299c6b9d-a9d6-b5c9-697f-3c320fad91c6@linux.microsoft.com>



On 8/3/23 18:18, Mimi Zohar wrote:
> On Thu, 2023-08-03 at 16:34 -0700, Tushar Sugandhi wrote:
>
>>>> +++ b/security/integrity/ima/ima_init.c
>>>> @@ -154,5 +154,8 @@ int __init ima_init(void)
>>>>    				  UTS_RELEASE, strlen(UTS_RELEASE), false,
>>>>    				  NULL, 0);
>>>>    
>>>> +	/* Measures TPM update counter at ima_init */
>>>> +	ima_measure_update_counter("ima_init_tpm_update_counter");
>>>> +
>>> With "ima_policy=critical_data" on the boot command line, the IMA
>>> measurement list record looks like:
>>>
>>> 6e190cc643ff0b718485966a0300473baedface735 ima_init_tpm_update_counter 7570646174655f636f756e7465723d3330383b
>>>
>>> Please change the "ima_init_tpm_update_counter" to something shorter
>>> and the hex encoded ascii string and pcr counter to something readable.
>> I believe you are seeing the above line in ascill_runtime_measurements log.
> Yes, the ascii_runtime_measurements are suppose to be readable to the
> end user.
We were passing string literals to 'buf' param in 
ima_measure_critical_data().
I believe we need to convert them first.

>> The ascii logging format is consistent with other event data for
>> critical_data event e.g. kernel_version.
> Then you got it wrong.
I see.? I will fix it for tpm in this patch series.
I think I should spin up another series to fix it for
selinux, kernel info, DM etc.
>> 10 8f449175bbf88bc55fc1127466628c39a3957d15 ima-buf
>> sha1:4acab4fbb08db663b7b7b4528e8729187d726782 kernel_version
>> 362e332e302d7263332b
>> 10 f10678b63c4b2529339dff02282e63d9c6bb0385 ima-buf
>> sha1:d8c187524412f74a961f2051a9529c009e700337
>> ima_init_tpm_update_counter 7570646174655f636f756e7465723d3133303b
>>
>> Entries in the binary runtime measurements look readable to me.
> You've inverted the meaning of the ascii and binary runtime measurement
> lists.  For comparison look at the ima-ng/ima-sig records.
Yup.? Agreed.
>> ima_init_tpm_update_counter update_counter=130;
>> ...
>> kexec_load_tpm_update_counte rupdate_counter=133;
>>
>> Please let me know if you still want me to change the format.
> OI course the ascii measurement list should be human readable.
Yup.? I will make the changes as I mentioned above.
>
>>> Perhaps name this critical-data "tpm" and "tpm-info", similar to the
>>   From patch 4/6:
>> +    result = ima_measure_critical_data("tpm_pcr_update_counter",
>> event_name,
>> +                  buf, buf_len, false, NULL, 0);
>>
>> The critical_data event_label value is currently set to
>> "tpm_pcr_update_counter".
> Why is the string so long?   Long strings or variables don't make the
> code or logs more understandable.  Please shorten both the strings and
> variables.
Agreed.? I will name this "tpm" and "tpm-info" or something similar.
>> I can rename event_label to "tpm-info", so that the admins can filter the
>> event in IMA policy based on the label if needed.
> The new record needs to be self containd and verifiable.  The
> additional info I suggested were just examples.  Please take the time
> to consider what needs to be included in the new record.  Decide
> whether this is a TPM security critical data record.  Only then, decide
> on the naming.
Yes.? I will explore what other potential attributes can be added to 
this record.
And I'll share them here for the community feedback.


From tusharsu at linux.microsoft.com  Fri Aug  4 10:13:26 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Fri, 4 Aug 2023 10:13:26 -0700
Subject: [PATCH 4/6] ima: implement functionality to measure TPM update
 counter
In-Reply-To: <7da13140fd4ab43e797214f08bf7bca07640c238.camel@linux.ibm.com>
References: <20230801181917.8535-1-tusharsu@linux.microsoft.com>
 <20230801181917.8535-5-tusharsu@linux.microsoft.com>
 <a2d9af7f54f364da9aeaef3dc6e03b7412957c23.camel@linux.ibm.com>
 <b9a77076-d6ac-a1ec-5e75-ab066dfade69@linux.microsoft.com>
 <7da13140fd4ab43e797214f08bf7bca07640c238.camel@linux.ibm.com>
Message-ID: <dbe58fc4-80f6-664f-e834-c27d64a8c60a@linux.microsoft.com>



On 8/3/23 18:22, Mimi Zohar wrote:
> On Thu, 2023-08-03 at 16:01 -0700, Tushar Sugandhi wrote:
>>>> +    scnprintf(buf, IMA_TPM_UPDATE_CTR_BUF_SIZE, "update_counter=%u;",
>>>> +                      update_counter);
>>>> +
>>>> +    buf_len = strlen(buf);
>>>> +
>>>> +    result = ima_measure_critical_data("tpm_pcr_update_counter", event_name,
>>>> +                              buf, buf_len, false, NULL, 0);
>>>>
>>> The new record should contain everything needed to verify the
>>> pcrCounter.  For example, each IMA measurement record updates the
>>> pcrCounter for each TPM bank enabled.  So the number of enabled TPM
>>> banks and number of IMA measurements should also be included in this
>>> record.
>> Agreed. That should be valuable information.
>> How does the below format look like for the buf above?
>>
>> version=<N>.<N>.<N>;num_enabled_pcr_banks=<N>;pcrUpdateCounter=<N>;num_ima_measurements=<N>;
> Refer to comment in 5/6.
Responded.
>>> Perhaps include a version number as well, so that if we ever want to
>>> include other information, we could.
>> By version number, do you mean kernel_version, or a new version
>> number specific to this record? Or something else?
> This is a record version type number.  The record format shouldn't
> change, but we should be prepared for it changing.  A single number
> should be fine.
>
Sounds good.



From eric.devolder at oracle.com  Fri Aug  4 14:03:56 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:56 -0400
Subject: [PATCH v26 5/8] x86/crash: add x86 crash hotplug support
In-Reply-To: <20230804210359.8321-1-eric.devolder@oracle.com>
References: <20230804210359.8321-1-eric.devolder@oracle.com>
Message-ID: <20230804210359.8321-6-eric.devolder@oracle.com>

When CPU or memory is hot un/plugged, or off/onlined, the crash
elfcorehdr, which describes the CPUs and memory in the system,
must also be updated.

A new elfcorehdr is generated from the available CPUs and memory
and replaces the existing elfcorehdr. The segment containing the
elfcorehdr is identified at run-time in
crash_core:crash_handle_hotplug_event().

No modifications to purgatory (see 'kexec: exclude elfcorehdr
from the segment digest') or boot_params (as the elfcorehdr=
capture kernel command line parameter pointer remains unchanged
and correct) are needed, just elfcorehdr.

For kexec_file_load(), the elfcorehdr segment size is based on
NR_CPUS and CRASH_MAX_MEMORY_RANGES in order to accommodate a
growing number of CPU and memory resources.

For kexec_load(), the userspace kexec utility needs to size the
elfcorehdr segment in the same/similar manner.

To accommodate kexec_load() syscall in the absence of
kexec_file_load() syscall support, prepare_elf_headers() and
dependents are moved outside of CONFIG_KEXEC_FILE.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/Kconfig             |   3 +
 arch/x86/include/asm/kexec.h |  15 +++++
 arch/x86/kernel/crash.c      | 103 ++++++++++++++++++++++++++++++++---
 3 files changed, 114 insertions(+), 7 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index fedc6dddd743..d9fc80b9ef84 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2069,6 +2069,9 @@ config ARCH_SUPPORTS_KEXEC_JUMP
 config ARCH_SUPPORTS_CRASH_DUMP
 	def_bool X86_64 || (X86_32 && HIGHMEM)
 
+config ARCH_SUPPORTS_CRASH_HOTPLUG
+	def_bool y
+
 config PHYSICAL_START
 	hex "Physical address where the kernel is loaded" if (EXPERT || CRASH_DUMP)
 	default "0x1000000"
diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 5b77bbc28f96..9143100ea3ea 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -209,6 +209,21 @@ typedef void crash_vmclear_fn(void);
 extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss;
 extern void kdump_nmi_shootdown_cpus(void);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+void arch_crash_handle_hotplug_event(struct kimage *image);
+#define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event
+
+#ifdef CONFIG_HOTPLUG_CPU
+static inline int crash_hotplug_cpu_support(void) { return 1; }
+#define crash_hotplug_cpu_support crash_hotplug_cpu_support
+#endif
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+static inline int crash_hotplug_memory_support(void) { return 1; }
+#define crash_hotplug_memory_support crash_hotplug_memory_support
+#endif
+#endif
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_X86_KEXEC_H */
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index cdd92ab43cda..c70a111c44fa 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -158,8 +158,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 	crash_save_cpu(regs, safe_smp_processor_id());
 }
 
-#ifdef CONFIG_KEXEC_FILE
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -231,7 +229,7 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 
 /* Prepare elf headers. Return addr and size */
 static int prepare_elf_headers(struct kimage *image, void **addr,
-					unsigned long *sz)
+					unsigned long *sz, unsigned long *nr_mem_ranges)
 {
 	struct crash_mem *cmem;
 	int ret;
@@ -249,6 +247,9 @@ static int prepare_elf_headers(struct kimage *image, void **addr,
 	if (ret)
 		goto out;
 
+	/* Return the computed number of memory ranges, for hotplug usage */
+	*nr_mem_ranges = cmem->nr_ranges;
+
 	/* By default prepare 64bit headers */
 	ret =  crash_prepare_elf64_headers(cmem, IS_ENABLED(CONFIG_X86_64), addr, sz);
 
@@ -257,6 +258,7 @@ static int prepare_elf_headers(struct kimage *image, void **addr,
 	return ret;
 }
 
+#ifdef CONFIG_KEXEC_FILE
 static int add_e820_entry(struct boot_params *params, struct e820_entry *entry)
 {
 	unsigned int nr_e820_entries;
@@ -371,18 +373,42 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 int crash_load_segments(struct kimage *image)
 {
 	int ret;
+	unsigned long pnum = 0;
 	struct kexec_buf kbuf = { .image = image, .buf_min = 0,
 				  .buf_max = ULONG_MAX, .top_down = false };
 
 	/* Prepare elf headers and add a segment */
-	ret = prepare_elf_headers(image, &kbuf.buffer, &kbuf.bufsz);
+	ret = prepare_elf_headers(image, &kbuf.buffer, &kbuf.bufsz, &pnum);
 	if (ret)
 		return ret;
 
-	image->elf_headers = kbuf.buffer;
-	image->elf_headers_sz = kbuf.bufsz;
+	image->elf_headers	= kbuf.buffer;
+	image->elf_headers_sz	= kbuf.bufsz;
+	kbuf.memsz		= kbuf.bufsz;
+
+#ifdef CONFIG_CRASH_HOTPLUG
+	/*
+	 * The elfcorehdr segment size accounts for VMCOREINFO, kernel_map,
+	 * maximum CPUs and maximum memory ranges.
+	 */
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		pnum = 2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES;
+	else
+		pnum += 2 + CONFIG_NR_CPUS_DEFAULT;
+
+	if (pnum < (unsigned long)PN_XNUM) {
+		kbuf.memsz = pnum * sizeof(Elf64_Phdr);
+		kbuf.memsz += sizeof(Elf64_Ehdr);
+
+		image->elfcorehdr_index = image->nr_segments;
+
+		/* Mark as usable to crash kernel, else crash kernel fails on boot */
+		image->elf_headers_sz = kbuf.memsz;
+	} else {
+		pr_err("number of Phdrs %lu exceeds max\n", pnum);
+	}
+#endif
 
-	kbuf.memsz = kbuf.bufsz;
 	kbuf.buf_align = ELF_CORE_HEADER_ALIGN;
 	kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
 	ret = kexec_add_buffer(&kbuf);
@@ -395,3 +421,66 @@ int crash_load_segments(struct kimage *image)
 	return ret;
 }
 #endif /* CONFIG_KEXEC_FILE */
+
+#ifdef CONFIG_CRASH_HOTPLUG
+
+#undef pr_fmt
+#define pr_fmt(fmt) "crash hp: " fmt
+
+/**
+ * arch_crash_handle_hotplug_event() - Handle hotplug elfcorehdr changes
+ * @image: a pointer to kexec_crash_image
+ *
+ * Prepare the new elfcorehdr and replace the existing elfcorehdr.
+ */
+void arch_crash_handle_hotplug_event(struct kimage *image)
+{
+	void *elfbuf = NULL, *old_elfcorehdr;
+	unsigned long nr_mem_ranges;
+	unsigned long mem, memsz;
+	unsigned long elfsz = 0;
+
+	/*
+	 * Create the new elfcorehdr reflecting the changes to CPU and/or
+	 * memory resources.
+	 */
+	if (prepare_elf_headers(image, &elfbuf, &elfsz, &nr_mem_ranges)) {
+		pr_err("unable to create new elfcorehdr");
+		goto out;
+	}
+
+	/*
+	 * Obtain address and size of the elfcorehdr segment, and
+	 * check it against the new elfcorehdr buffer.
+	 */
+	mem = image->segment[image->elfcorehdr_index].mem;
+	memsz = image->segment[image->elfcorehdr_index].memsz;
+	if (elfsz > memsz) {
+		pr_err("update elfcorehdr elfsz %lu > memsz %lu",
+			elfsz, memsz);
+		goto out;
+	}
+
+	/*
+	 * Copy new elfcorehdr over the old elfcorehdr at destination.
+	 */
+	old_elfcorehdr = kmap_local_page(pfn_to_page(mem >> PAGE_SHIFT));
+	if (!old_elfcorehdr) {
+		pr_err("mapping elfcorehdr segment failed\n");
+		goto out;
+	}
+
+	/*
+	 * Temporarily invalidate the crash image while the
+	 * elfcorehdr is updated.
+	 */
+	xchg(&kexec_crash_image, NULL);
+	memcpy_flushcache(old_elfcorehdr, elfbuf, elfsz);
+	xchg(&kexec_crash_image, image);
+	kunmap_local(old_elfcorehdr);
+	pr_debug("updated elfcorehdr\n");
+
+out:
+	vfree(elfbuf);
+}
+#endif
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug  4 14:03:55 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:55 -0400
Subject: [PATCH v26 4/8] crash: memory and CPU hotplug sysfs attributes
In-Reply-To: <20230804210359.8321-1-eric.devolder@oracle.com>
References: <20230804210359.8321-1-eric.devolder@oracle.com>
Message-ID: <20230804210359.8321-5-eric.devolder@oracle.com>

Introduce the crash_hotplug attribute for memory and CPUs for
use by userspace.  These attributes directly facilitate the udev
rule for managing userspace re-loading of the crash kernel upon
hot un/plug changes.

For memory, expose the crash_hotplug attribute to the
/sys/devices/system/memory directory. For example:

 # udevadm info --attribute-walk /sys/devices/system/memory/memory81
  looking at device '/devices/system/memory/memory81':
    KERNEL=="memory81"
    SUBSYSTEM=="memory"
    DRIVER==""
    ATTR{online}=="1"
    ATTR{phys_device}=="0"
    ATTR{phys_index}=="00000051"
    ATTR{removable}=="1"
    ATTR{state}=="online"
    ATTR{valid_zones}=="Movable"

  looking at parent device '/devices/system/memory':
    KERNELS=="memory"
    SUBSYSTEMS==""
    DRIVERS==""
    ATTRS{auto_online_blocks}=="offline"
    ATTRS{block_size_bytes}=="8000000"
    ATTRS{crash_hotplug}=="1"

For CPUs, expose the crash_hotplug attribute to the
/sys/devices/system/cpu directory. For example:

 # udevadm info --attribute-walk /sys/devices/system/cpu/cpu0
  looking at device '/devices/system/cpu/cpu0':
    KERNEL=="cpu0"
    SUBSYSTEM=="cpu"
    DRIVER=="processor"
    ATTR{crash_notes}=="277c38600"
    ATTR{crash_notes_size}=="368"
    ATTR{online}=="1"

  looking at parent device '/devices/system/cpu':
    KERNELS=="cpu"
    SUBSYSTEMS==""
    DRIVERS==""
    ATTRS{crash_hotplug}=="1"
    ATTRS{isolated}==""
    ATTRS{kernel_max}=="8191"
    ATTRS{nohz_full}=="  (null)"
    ATTRS{offline}=="4-7"
    ATTRS{online}=="0-3"
    ATTRS{possible}=="0-7"
    ATTRS{present}=="0-3"

With these sysfs attributes in place, it is possible to efficiently
instruct the udev rule to skip crash kernel reloading for kernels
configured with crash hotplug support.

For example, the following is the proposed udev rule change for RHEL
system 98-kexec.rules (as the first lines of the rule file):

 # The kernel updates the crash elfcorehdr for CPU and memory changes
 SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
 SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

When examined in the context of 98-kexec.rules, the above rules
test if crash_hotplug is set, and if so, the userspace initiated
unload-then-reload of the crash kernel is skipped.

CPU and memory checks are separated in accordance with
CONFIG_HOTPLUG_CPU and CONFIG_MEMORY_HOTPLUG kernel config options.
If an architecture supports, for example, memory hotplug but not
CPU hotplug, then the /sys/devices/system/memory/crash_hotplug
attribute file is present, but the /sys/devices/system/cpu/crash_hotplug
attribute file will NOT be present. Thus the udev rule skips
userspace processing of memory hot un/plug events, but the udev
rule will evaluate false for CPU events, thus allowing userspace to
process CPU hot un/plug events (ie the unload-then-reload of the kdump
capture kernel).

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 Documentation/ABI/testing/sysfs-devices-memory |  8 ++++++++
 .../ABI/testing/sysfs-devices-system-cpu       |  8 ++++++++
 .../admin-guide/mm/memory-hotplug.rst          |  8 ++++++++
 Documentation/core-api/cpu_hotplug.rst         | 18 ++++++++++++++++++
 drivers/base/cpu.c                             | 13 +++++++++++++
 drivers/base/memory.c                          | 13 +++++++++++++
 include/linux/kexec.h                          |  8 ++++++++
 7 files changed, 76 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-devices-memory b/Documentation/ABI/testing/sysfs-devices-memory
index d8b0f80b9e33..a95e0f17c35a 100644
--- a/Documentation/ABI/testing/sysfs-devices-memory
+++ b/Documentation/ABI/testing/sysfs-devices-memory
@@ -110,3 +110,11 @@ Description:
 		link is created for memory section 9 on node0.
 
 		/sys/devices/system/node/node0/memory9 -> ../../memory/memory9
+
+What:		/sys/devices/system/memory/crash_hotplug
+Date:		Aug 2023
+Contact:	Linux kernel mailing list <linux-kernel at vger.kernel.org>
+Description:
+		(RO) indicates whether or not the kernel directly supports
+		modifying the crash elfcorehdr for memory hot un/plug and/or
+		on/offline changes.
diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
index ecd585ca2d50..31189da7ef57 100644
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -686,3 +686,11 @@ Description:
 		(RO) the list of CPUs that are isolated and don't
 		participate in load balancing. These CPUs are set by
 		boot parameter "isolcpus=".
+
+What:		/sys/devices/system/cpu/crash_hotplug
+Date:		Aug 2023
+Contact:	Linux kernel mailing list <linux-kernel at vger.kernel.org>
+Description:
+		(RO) indicates whether or not the kernel directly supports
+		modifying the crash elfcorehdr for CPU hot un/plug and/or
+		on/offline changes.
diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst
index 1b02fe5807cc..eb99d79223a3 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -291,6 +291,14 @@ The following files are currently defined:
 		       Availability depends on the CONFIG_ARCH_MEMORY_PROBE
 		       kernel configuration option.
 ``uevent``	       read-write: generic udev file for device subsystems.
+``crash_hotplug``      read-only: when changes to the system memory map
+		       occur due to hot un/plug of memory, this file contains
+		       '1' if the kernel updates the kdump capture kernel memory
+		       map itself (via elfcorehdr), or '0' if userspace must update
+		       the kdump capture kernel memory map.
+
+		       Availability depends on the CONFIG_MEMORY_HOTPLUG kernel
+		       configuration option.
 ====================== =========================================================
 
 .. note::
diff --git a/Documentation/core-api/cpu_hotplug.rst b/Documentation/core-api/cpu_hotplug.rst
index e6f5bc39cf5c..d6d470d7dda0 100644
--- a/Documentation/core-api/cpu_hotplug.rst
+++ b/Documentation/core-api/cpu_hotplug.rst
@@ -741,6 +741,24 @@ will receive all events. A script like::
 
 can process the event further.
 
+When changes to the CPUs in the system occur, the sysfs file
+/sys/devices/system/cpu/crash_hotplug contains '1' if the kernel
+updates the kdump capture kernel list of CPUs itself (via elfcorehdr),
+or '0' if userspace must update the kdump capture kernel list of CPUs.
+
+The availability depends on the CONFIG_HOTPLUG_CPU kernel configuration
+option.
+
+To skip userspace processing of CPU hot un/plug events for kdump
+(i.e. the unload-then-reload to obtain a current list of CPUs), this sysfs
+file can be used in a udev rule as follows:
+
+ SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
+
+For a CPU hot un/plug event, if the architecture supports kernel updates
+of the elfcorehdr (which contains the list of CPUs), then the rule skips
+the unload-then-reload of the kdump capture kernel.
+
 Kernel Inline Documentations Reference
 ======================================
 
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index c1815b9dae68..c9204d69e616 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -282,6 +282,16 @@ static ssize_t print_cpus_nohz_full(struct device *dev,
 static DEVICE_ATTR(nohz_full, 0444, print_cpus_nohz_full, NULL);
 #endif
 
+#ifdef CONFIG_CRASH_HOTPLUG
+static ssize_t crash_hotplug_show(struct device *dev,
+				     struct device_attribute *attr,
+				     char *buf)
+{
+	return sysfs_emit(buf, "%d\n", crash_hotplug_cpu_support());
+}
+static DEVICE_ATTR_ADMIN_RO(crash_hotplug);
+#endif
+
 static void cpu_device_release(struct device *dev)
 {
 	/*
@@ -469,6 +479,9 @@ static struct attribute *cpu_root_attrs[] = {
 #ifdef CONFIG_NO_HZ_FULL
 	&dev_attr_nohz_full.attr,
 #endif
+#ifdef CONFIG_CRASH_HOTPLUG
+	&dev_attr_crash_hotplug.attr,
+#endif
 #ifdef CONFIG_GENERIC_CPU_AUTOPROBE
 	&dev_attr_modalias.attr,
 #endif
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index b456ac213610..15bb416e58ce 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -490,6 +490,16 @@ static ssize_t auto_online_blocks_store(struct device *dev,
 
 static DEVICE_ATTR_RW(auto_online_blocks);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+#include <linux/kexec.h>
+static ssize_t crash_hotplug_show(struct device *dev,
+				       struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%d\n", crash_hotplug_memory_support());
+}
+static DEVICE_ATTR_RO(crash_hotplug);
+#endif
+
 /*
  * Some architectures will have custom drivers to do this, and
  * will not need to do it from userspace.  The fake hot-add code
@@ -889,6 +899,9 @@ static struct attribute *memory_root_attrs[] = {
 
 	&dev_attr_block_size_bytes.attr,
 	&dev_attr_auto_online_blocks.attr,
+#ifdef CONFIG_CRASH_HOTPLUG
+	&dev_attr_crash_hotplug.attr,
+#endif
 	NULL
 };
 
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index b9903dd48e24..6a8a724ac638 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -501,6 +501,14 @@ static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) {
 static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
 #endif
 
+#ifndef crash_hotplug_cpu_support
+static inline int crash_hotplug_cpu_support(void) { return 0; }
+#endif
+
+#ifndef crash_hotplug_memory_support
+static inline int crash_hotplug_memory_support(void) { return 0; }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug  4 14:03:57 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:57 -0400
Subject: [PATCH v26 6/8] crash: hotplug support for kexec_load()
In-Reply-To: <20230804210359.8321-1-eric.devolder@oracle.com>
References: <20230804210359.8321-1-eric.devolder@oracle.com>
Message-ID: <20230804210359.8321-7-eric.devolder@oracle.com>

The hotplug support for kexec_load() requires changes to the
userspace kexec-tools and a little extra help from the kernel.

Given a kdump capture kernel loaded via kexec_load(), and a
subsequent hotplug event, the crash hotplug handler finds the
elfcorehdr and rewrites it to reflect the hotplug change.
That is the desired outcome, however, at kernel panic time,
the purgatory integrity check fails (because the elfcorehdr
changed), and the capture kernel does not boot and no vmcore
is generated.

Therefore, the userspace kexec-tools/kexec must indicate to the
kernel that the elfcorehdr can be modified (because the kexec
excluded the elfcorehdr from the digest, and sized the elfcorehdr
memory buffer appropriately).

To facilitate hotplug support with kexec_load():
 - a new kexec flag KEXEC_UPATE_ELFCOREHDR indicates that it is
   safe for the kernel to modify the kexec_load()'d elfcorehdr
 - the /sys/kernel/crash_elfcorehdr_size node communicates the
   preferred size of the elfcorehdr memory buffer
 - The sysfs crash_hotplug nodes (ie.
   /sys/devices/system/[cpu|memory]/crash_hotplug) dynamically
   take into account kexec_file_load() vs kexec_load() and
   KEXEC_UPDATE_ELFCOREHDR.
   This is critical so that the udev rule processing of crash_hotplug
   is all that is needed to determine if the userspace unload-then-load
   of the kdump image is to be skipped, or not. The proposed udev
   rule change looks like:
   # The kernel updates the crash elfcorehdr for CPU and memory changes
   SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
   SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

The table below indicates the behavior of kexec_load()'d kdump image
updates (with the new udev crash_hotplug rule in place):

 Kernel |Kexec
 -------+-----+----
 Old    |Old  |New
        |  a  | a
 -------+-----+----
 New    |  a  | b
 -------+-----+----

where kexec 'old' and 'new' delineate kexec-tools has the needed
modifications for the crash hotplug feature, and kernel 'old' and
'new' delineate the kernel supports this crash hotplug feature.

Behavior 'a' indicates the unload-then-reload of the entire kdump
image. For the kexec 'old' column, the unload-then-reload occurs
due to the missing flag KEXEC_UPDATE_ELFCOREHDR. An 'old' kernel
(with 'new' kexec) does not present the crash_hotplug sysfs node,
which leads to the unload-then-reload of the kdump image.

Behavior 'b' indicates the desired optimized behavior of the kernel
directly modifying the elfcorehdr and avoiding the unload-then-reload
of the kdump image.

If the udev rule is not updated with crash_hotplug node check, then
no matter any combination of kernel or kexec is new or old, the
kdump image continues to be unload-then-reload on hotplug changes.

To fully support crash hotplug feature, there needs to be a rollout
of kernel, kexec-tools and udev rule changes. However, the order of
the rollout of these pieces does not matter; kexec_load()'d kdump
images still function for hotplug as-is.

Suggested-by: Hari Bathini <hbathini at linux.ibm.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/include/asm/kexec.h | 11 +++++++----
 arch/x86/kernel/crash.c      | 27 +++++++++++++++++++++++++++
 include/linux/kexec.h        | 14 ++++++++++++--
 include/uapi/linux/kexec.h   |  1 +
 kernel/Kconfig.kexec         |  4 ++++
 kernel/crash_core.c          | 31 +++++++++++++++++++++++++++++++
 kernel/kexec.c               |  5 +++++
 kernel/ksysfs.c              | 15 +++++++++++++++
 8 files changed, 102 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 9143100ea3ea..3be6a98751f0 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -214,14 +214,17 @@ void arch_crash_handle_hotplug_event(struct kimage *image);
 #define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event
 
 #ifdef CONFIG_HOTPLUG_CPU
-static inline int crash_hotplug_cpu_support(void) { return 1; }
-#define crash_hotplug_cpu_support crash_hotplug_cpu_support
+int arch_crash_hotplug_cpu_support(void);
+#define crash_hotplug_cpu_support arch_crash_hotplug_cpu_support
 #endif
 
 #ifdef CONFIG_MEMORY_HOTPLUG
-static inline int crash_hotplug_memory_support(void) { return 1; }
-#define crash_hotplug_memory_support crash_hotplug_memory_support
+int arch_crash_hotplug_memory_support(void);
+#define crash_hotplug_memory_support arch_crash_hotplug_memory_support
 #endif
+
+unsigned int arch_crash_get_elfcorehdr_size(void);
+#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size
 #endif
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index c70a111c44fa..caf22bcb61af 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -427,6 +427,33 @@ int crash_load_segments(struct kimage *image)
 #undef pr_fmt
 #define pr_fmt(fmt) "crash hp: " fmt
 
+/* These functions provide the value for the sysfs crash_hotplug nodes */
+#ifdef CONFIG_HOTPLUG_CPU
+int arch_crash_hotplug_cpu_support(void)
+{
+	return crash_check_update_elfcorehdr();
+}
+#endif
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int arch_crash_hotplug_memory_support(void)
+{
+	return crash_check_update_elfcorehdr();
+}
+#endif
+
+unsigned int arch_crash_get_elfcorehdr_size(void)
+{
+	unsigned int sz;
+
+	/* kernel_map, VMCOREINFO and maximum CPUs */
+	sz = 2 + CONFIG_NR_CPUS_DEFAULT;
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		sz += CONFIG_CRASH_MAX_MEMORY_RANGES;
+	sz *= sizeof(Elf64_Phdr);
+	return sz;
+}
+
 /**
  * arch_crash_handle_hotplug_event() - Handle hotplug elfcorehdr changes
  * @image: a pointer to kexec_crash_image
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 6a8a724ac638..bb0e614f2a05 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -335,6 +335,10 @@ struct kimage {
 	unsigned int preserve_context : 1;
 	/* If set, we are using file mode kexec syscall */
 	unsigned int file_mode:1;
+#ifdef CONFIG_CRASH_HOTPLUG
+	/* If set, allow changes to elfcorehdr of kexec_load'd image */
+	unsigned int update_elfcorehdr:1;
+#endif
 
 #ifdef ARCH_HAS_KIMAGE_ARCH
 	struct kimage_arch arch;
@@ -411,9 +415,9 @@ bool kexec_load_permitted(int kexec_image_type);
 
 /* List of defined/legal kexec flags */
 #ifndef CONFIG_KEXEC_JUMP
-#define KEXEC_FLAGS    KEXEC_ON_CRASH
+#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_UPDATE_ELFCOREHDR)
 #else
-#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT)
+#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT | KEXEC_UPDATE_ELFCOREHDR)
 #endif
 
 /* List of defined/legal kexec file flags */
@@ -501,6 +505,8 @@ static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) {
 static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
 #endif
 
+int crash_check_update_elfcorehdr(void);
+
 #ifndef crash_hotplug_cpu_support
 static inline int crash_hotplug_cpu_support(void) { return 0; }
 #endif
@@ -509,6 +515,10 @@ static inline int crash_hotplug_cpu_support(void) { return 0; }
 static inline int crash_hotplug_memory_support(void) { return 0; }
 #endif
 
+#ifndef crash_get_elfcorehdr_size
+static inline unsigned int crash_get_elfcorehdr_size(void) { return 0; }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/include/uapi/linux/kexec.h b/include/uapi/linux/kexec.h
index 981016e05cfa..01766dd839b0 100644
--- a/include/uapi/linux/kexec.h
+++ b/include/uapi/linux/kexec.h
@@ -12,6 +12,7 @@
 /* kexec flags for different usage scenarios */
 #define KEXEC_ON_CRASH		0x00000001
 #define KEXEC_PRESERVE_CONTEXT	0x00000002
+#define KEXEC_UPDATE_ELFCOREHDR	0x00000004
 #define KEXEC_ARCH_MASK		0xffff0000
 
 /*
diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
index d0a9a5392035..6e5350d2d7a6 100644
--- a/kernel/Kconfig.kexec
+++ b/kernel/Kconfig.kexec
@@ -144,4 +144,8 @@ config CRASH_MAX_MEMORY_RANGES
 	  memory buffer/segment size under 1MiB. This represents a sane choice
 	  to accommodate both baremetal and virtual machine configurations.
 
+	  For the kexec_load() syscall path, CRASH_MAX_MEMORY_RANGES is part of
+	  the computation behind the value provided through the
+	  /sys/kernel/crash_elfcorehdr_size attribute.
+
 endmenu
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 53d211c690a1..fa918176d46d 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -704,6 +704,33 @@ subsys_initcall(crash_save_vmcoreinfo_init);
 #ifdef CONFIG_CRASH_HOTPLUG
 #undef pr_fmt
 #define pr_fmt(fmt) "crash hp: " fmt
+
+/*
+ * This routine utilized when the crash_hotplug sysfs node is read.
+ * It reflects the kernel's ability/permission to update the crash
+ * elfcorehdr directly.
+ */
+int crash_check_update_elfcorehdr(void)
+{
+	int rc = 0;
+
+	/* Obtain lock while reading crash information */
+	if (!kexec_trylock()) {
+		pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n");
+		return 0;
+	}
+	if (kexec_crash_image) {
+		if (kexec_crash_image->file_mode)
+			rc = 1;
+		else
+			rc = kexec_crash_image->update_elfcorehdr;
+	}
+	/* Release lock now that update complete */
+	kexec_unlock();
+
+	return rc;
+}
+
 /*
  * To accurately reflect hot un/plug changes of cpu and memory resources
  * (including onling and offlining of those resources), the elfcorehdr
@@ -734,6 +761,10 @@ static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
 
 	image = kexec_crash_image;
 
+	/* Check that updating elfcorehdr is permitted */
+	if (!(image->file_mode || image->update_elfcorehdr))
+		goto out;
+
 	if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
 		hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
 		pr_debug("hp_action %u, cpu %u\n", hp_action, cpu);
diff --git a/kernel/kexec.c b/kernel/kexec.c
index 92d301f98776..107f355eac10 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -129,6 +129,11 @@ static int do_kexec_load(unsigned long entry, unsigned long nr_segments,
 	if (flags & KEXEC_PRESERVE_CONTEXT)
 		image->preserve_context = 1;
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	if (flags & KEXEC_UPDATE_ELFCOREHDR)
+		image->update_elfcorehdr = 1;
+#endif
+
 	ret = machine_kexec_prepare(image);
 	if (ret)
 		goto out;
diff --git a/kernel/ksysfs.c b/kernel/ksysfs.c
index aad7a3bfd846..1d4bc493b2f4 100644
--- a/kernel/ksysfs.c
+++ b/kernel/ksysfs.c
@@ -165,6 +165,18 @@ static ssize_t vmcoreinfo_show(struct kobject *kobj,
 }
 KERNEL_ATTR_RO(vmcoreinfo);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+static ssize_t crash_elfcorehdr_size_show(struct kobject *kobj,
+			       struct kobj_attribute *attr, char *buf)
+{
+	unsigned int sz = crash_get_elfcorehdr_size();
+
+	return sysfs_emit(buf, "%u\n", sz);
+}
+KERNEL_ATTR_RO(crash_elfcorehdr_size);
+
+#endif
+
 #endif /* CONFIG_CRASH_CORE */
 
 /* whether file capabilities are enabled */
@@ -255,6 +267,9 @@ static struct attribute * kernel_attrs[] = {
 #endif
 #ifdef CONFIG_CRASH_CORE
 	&vmcoreinfo_attr.attr,
+#ifdef CONFIG_CRASH_HOTPLUG
+	&crash_elfcorehdr_size_attr.attr,
+#endif
 #endif
 #ifndef CONFIG_TINY_RCU
 	&rcu_expedited_attr.attr,
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug  4 14:03:53 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:53 -0400
Subject: [PATCH v26 2/8] crash: add generic infrastructure for crash hotplug support
In-Reply-To: <20230804210359.8321-1-eric.devolder@oracle.com>
References: <20230804210359.8321-1-eric.devolder@oracle.com>
Message-ID: <20230804210359.8321-3-eric.devolder@oracle.com>

To support crash hotplug, a mechanism is needed to update the crash
elfcorehdr upon CPU or memory changes (eg. hot un/plug or off/
onlining). The crash elfcorehdr describes the CPUs and memory to
be written into the vmcore.

To track CPU changes, callbacks are registered with the cpuhp
mechanism via cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN). The
crash hotplug elfcorehdr update has no explicit ordering requirement
(relative to other cpuhp states), so meets the criteria for
utilizing CPUHP_BP_PREPARE_DYN. CPUHP_BP_PREPARE_DYN is a dynamic
state and avoids the need to introduce a new state for crash
hotplug. Also, CPUHP_BP_PREPARE_DYN is the last state in the PREPARE
group, just prior to the STARTING group, which is very close to the
CPU starting up in a plug/online situation, or stopping in a unplug/
offline situation. This minimizes the window of time during an
actual plug/online or unplug/offline situation in which the
elfcorehdr would be inaccurate. Note that for a CPU being unplugged
or offlined, the CPU will still be present in the list of CPUs
generated by crash_prepare_elf64_headers(). However, there is no
need to explicitly omit the CPU, see justification in
'crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()'.

To track memory changes, a notifier is registered to capture the
memblock MEM_ONLINE and MEM_OFFLINE events via register_memory_notifier().

The CPU callbacks and memory notifiers invoke crash_handle_hotplug_event()
which performs needed tasks and then dispatches the event to the
architecture specific arch_crash_handle_hotplug_event() to update the
elfcorehdr with the current state of CPUs and memory. During the
process, the kexec_lock is held.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |   9 +++
 include/linux/kexec.h      |  11 +++
 kernel/Kconfig.kexec       |  31 ++++++++
 kernel/crash_core.c        | 142 +++++++++++++++++++++++++++++++++++++
 kernel/kexec_core.c        |   6 ++
 5 files changed, 199 insertions(+)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index de62a722431e..e14345cc7a22 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -84,4 +84,13 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
 int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
 
+#define KEXEC_CRASH_HP_NONE			0
+#define KEXEC_CRASH_HP_ADD_CPU			1
+#define KEXEC_CRASH_HP_REMOVE_CPU		2
+#define KEXEC_CRASH_HP_ADD_MEMORY		3
+#define KEXEC_CRASH_HP_REMOVE_MEMORY		4
+#define KEXEC_CRASH_HP_INVALID_CPU		-1U
+
+struct kimage;
+
 #endif /* LINUX_CRASH_CORE_H */
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 811a90e09698..b9903dd48e24 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -33,6 +33,7 @@ extern note_buf_t __percpu *crash_notes;
 #include <linux/compat.h>
 #include <linux/ioport.h>
 #include <linux/module.h>
+#include <linux/highmem.h>
 #include <asm/kexec.h>
 
 /* Verify architecture specific macros are defined */
@@ -360,6 +361,12 @@ struct kimage {
 	struct purgatory_info purgatory_info;
 #endif
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	int hp_action;
+	int elfcorehdr_index;
+	bool elfcorehdr_updated;
+#endif
+
 #ifdef CONFIG_IMA_KEXEC
 	/* Virtual address of IMA measurement buffer for kexec syscall */
 	void *ima_buffer;
@@ -490,6 +497,10 @@ static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, g
 static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) { }
 #endif
 
+#ifndef arch_crash_handle_hotplug_event
+static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
index ff72e45cfaef..d0a9a5392035 100644
--- a/kernel/Kconfig.kexec
+++ b/kernel/Kconfig.kexec
@@ -113,4 +113,35 @@ config CRASH_DUMP
 	  For s390, this option also enables zfcpdump.
 	  See also <file:Documentation/s390/zfcpdump.rst>
 
+config CRASH_HOTPLUG
+	bool "Update the crash elfcorehdr on system configuration changes"
+	default y
+	depends on CRASH_DUMP && (HOTPLUG_CPU || MEMORY_HOTPLUG)
+	depends on ARCH_SUPPORTS_CRASH_HOTPLUG
+	help
+	  Enable direct update to the crash elfcorehdr (which contains
+	  the list of CPUs and memory regions to be dumped upon a crash)
+	  in response to hot plug/unplug or online/offline of CPUs or
+	  memory. This is a much more advanced approach than userspace
+	  attempting that.
+
+	  If unsure, say Y.
+
+config CRASH_MAX_MEMORY_RANGES
+	int "Specify the maximum number of memory regions for the elfcorehdr"
+	default 8192
+	depends on CRASH_HOTPLUG
+	help
+	  For the kexec_file_load() syscall path, specify the maximum number of
+	  memory regions that the elfcorehdr buffer/segment can accommodate.
+	  These regions are obtained via walk_system_ram_res(); eg. the
+	  'System RAM' entries in /proc/iomem.
+	  This value is combined with NR_CPUS_DEFAULT and multiplied by
+	  sizeof(Elf64_Phdr) to determine the final elfcorehdr memory buffer/
+	  segment size.
+	  The value 8192, for example, covers a (sparsely populated) 1TiB system
+	  consisting of 128MiB memblocks, while resulting in an elfcorehdr
+	  memory buffer/segment size under 1MiB. This represents a sane choice
+	  to accommodate both baremetal and virtual machine configurations.
+
 endmenu
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index b7c30b748a16..53d211c690a1 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -11,6 +11,8 @@
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
 #include <linux/kexec.h>
+#include <linux/memory.h>
+#include <linux/cpuhotplug.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -18,6 +20,7 @@
 #include <crypto/sha1.h>
 
 #include "kallsyms_internal.h"
+#include "kexec_internal.h"
 
 /* vmcoreinfo stuff */
 unsigned char *vmcoreinfo_data;
@@ -697,3 +700,142 @@ static int __init crash_save_vmcoreinfo_init(void)
 }
 
 subsys_initcall(crash_save_vmcoreinfo_init);
+
+#ifdef CONFIG_CRASH_HOTPLUG
+#undef pr_fmt
+#define pr_fmt(fmt) "crash hp: " fmt
+/*
+ * To accurately reflect hot un/plug changes of cpu and memory resources
+ * (including onling and offlining of those resources), the elfcorehdr
+ * (which is passed to the crash kernel via the elfcorehdr= parameter)
+ * must be updated with the new list of CPUs and memories.
+ *
+ * In order to make changes to elfcorehdr, two conditions are needed:
+ * First, the segment containing the elfcorehdr must be large enough
+ * to permit a growing number of resources; the elfcorehdr memory size
+ * is based on NR_CPUS_DEFAULT and CRASH_MAX_MEMORY_RANGES.
+ * Second, purgatory must explicitly exclude the elfcorehdr from the
+ * list of segments it checks (since the elfcorehdr changes and thus
+ * would require an update to purgatory itself to update the digest).
+ */
+static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
+{
+	struct kimage *image;
+
+	/* Obtain lock while changing crash information */
+	if (!kexec_trylock()) {
+		pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n");
+		return;
+	}
+
+	/* Check kdump is not loaded */
+	if (!kexec_crash_image)
+		goto out;
+
+	image = kexec_crash_image;
+
+	if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
+		hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
+		pr_debug("hp_action %u, cpu %u\n", hp_action, cpu);
+	else
+		pr_debug("hp_action %u\n", hp_action);
+
+	/*
+	 * The elfcorehdr_index is set to -1 when the struct kimage
+	 * is allocated. Find the segment containing the elfcorehdr,
+	 * if not already found.
+	 */
+	if (image->elfcorehdr_index < 0) {
+		unsigned long mem;
+		unsigned char *ptr;
+		unsigned int n;
+
+		for (n = 0; n < image->nr_segments; n++) {
+			mem = image->segment[n].mem;
+			ptr = kmap_local_page(pfn_to_page(mem >> PAGE_SHIFT));
+			if (ptr) {
+				/* The segment containing elfcorehdr */
+				if (memcmp(ptr, ELFMAG, SELFMAG) == 0)
+					image->elfcorehdr_index = (int)n;
+				kunmap_local(ptr);
+			}
+		}
+	}
+
+	if (image->elfcorehdr_index < 0) {
+		pr_err("unable to locate elfcorehdr segment");
+		goto out;
+	}
+
+	/* Needed in order for the segments to be updated */
+	arch_kexec_unprotect_crashkres();
+
+	/* Differentiate between normal load and hotplug update */
+	image->hp_action = hp_action;
+
+	/* Now invoke arch-specific update handler */
+	arch_crash_handle_hotplug_event(image);
+
+	/* No longer handling a hotplug event */
+	image->hp_action = KEXEC_CRASH_HP_NONE;
+	image->elfcorehdr_updated = true;
+
+	/* Change back to read-only */
+	arch_kexec_protect_crashkres();
+
+	/* Errors in the callback is not a reason to rollback state */
+out:
+	/* Release lock now that update complete */
+	kexec_unlock();
+}
+
+static int crash_memhp_notifier(struct notifier_block *nb, unsigned long val, void *v)
+{
+	switch (val) {
+	case MEM_ONLINE:
+		crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_MEMORY,
+			KEXEC_CRASH_HP_INVALID_CPU);
+		break;
+
+	case MEM_OFFLINE:
+		crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_MEMORY,
+			KEXEC_CRASH_HP_INVALID_CPU);
+		break;
+	}
+	return NOTIFY_OK;
+}
+
+static struct notifier_block crash_memhp_nb = {
+	.notifier_call = crash_memhp_notifier,
+	.priority = 0
+};
+
+static int crash_cpuhp_online(unsigned int cpu)
+{
+	crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_CPU, cpu);
+	return 0;
+}
+
+static int crash_cpuhp_offline(unsigned int cpu)
+{
+	crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_CPU, cpu);
+	return 0;
+}
+
+static int __init crash_hotplug_init(void)
+{
+	int result = 0;
+
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		register_memory_notifier(&crash_memhp_nb);
+
+	if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
+		result = cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN,
+			"crash/cpuhp", crash_cpuhp_online, crash_cpuhp_offline);
+	}
+
+	return result;
+}
+
+subsys_initcall(crash_hotplug_init);
+#endif
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index e2f2574d8b74..5d323255862a 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -277,6 +277,12 @@ struct kimage *do_kimage_alloc_init(void)
 	/* Initialize the list of unusable pages */
 	INIT_LIST_HEAD(&image->unusable_pages);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	image->hp_action = KEXEC_CRASH_HP_NONE;
+	image->elfcorehdr_index = -1;
+	image->elfcorehdr_updated = false;
+#endif
+
 	return image;
 }
 
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug  4 14:03:51 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:51 -0400
Subject: [PATCH v26 0/8] crash: Kernel handling of CPU and memory hot un/plug
Message-ID: <20230804210359.8321-1-eric.devolder@oracle.com>

This series is dependent upon "refactor Kconfig to consolidate
KEXEC and CRASH options".
 https://lore.kernel.org/lkml/20230712161545.87870-1-eric.devolder at oracle.com/

Once the kdump service is loaded, if changes to CPUs or memory occur,
either by hot un/plug or off/onlining, the crash elfcorehdr must also
be updated.

The elfcorehdr describes to kdump the CPUs and memory in the system,
and any inaccuracies can result in a vmcore with missing CPU context
or memory regions.

The current solution utilizes udev to initiate an unload-then-reload
of the kdump image (eg. kernel, initrd, boot_params, purgatory and
elfcorehdr) by the userspace kexec utility. In the original post I
outlined the significant performance problems related to offloading
this activity to userspace.

This patchset introduces a generic crash handler that registers with
the CPU and memory notifiers. Upon CPU or memory changes, from either
hot un/plug or off/onlining, this generic handler is invoked and
performs important housekeeping, for example obtaining the appropriate
lock, and then invokes an architecture specific handler to do the
appropriate elfcorehdr update.

Note the description in patch 'crash: change crash_prepare_elf64_headers()
to for_each_possible_cpu()' and 'x86/crash: optimize CPU changes' that
enables further optimizations related to CPU plug/unplug/online/offline
performance of elfcorehdr updates.

In the case of x86_64, the arch specific handler generates a new
elfcorehdr, and overwrites the old one in memory; thus no involvement
with userspace needed.

To realize the benefits/test this patchset, one must make a couple
of minor changes to userspace:

 - Prevent udev from updating kdump crash kernel on hot un/plug changes.
   Add the following as the first lines to the RHEL udev rule file
   /usr/lib/udev/rules.d/98-kexec.rules:

   # The kernel updates the crash elfcorehdr for CPU and memory changes
   SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
   SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

   With this changeset applied, the two rules evaluate to false for
   CPU and memory change events and thus skip the userspace
   unload-then-reload of kdump.

 - Change to the kexec_file_load for loading the kdump kernel:
   Eg. on RHEL: in /usr/bin/kdumpctl, change to:
    standard_kexec_args="-p -d -s"
   which adds the -s to select kexec_file_load() syscall.

This kernel patchset also supports kexec_load() with a modified kexec
userspace utility. A working changeset to the kexec userspace utility
is posted to the kexec-tools mailing list here:

 http://lists.infradead.org/pipermail/kexec/2023-May/027049.html

To use the kexec-tools patch, apply, build and install kexec-tools,
then change the kdumpctl's standard_kexec_args to replace the -s with
--hotplug. The removal of -s reverts to the kexec_load syscall and
the addition of --hotplug invokes the changes put forth in the
kexec-tools patch.

Regards,
eric
---
v26: 4aug2023
 - Rebased onto 6.5.0-rc4
 - Dropped the refactor of files drivers/base/cpu|memory.c as unrelated
   to this series.
 - Minor corrections to documentation, per Randy Dunlap and GregKH.

v25: 29jun2023
 https://lore.kernel.org/lkml/20230629192119.6613-1-eric.devolder at oracle.com/
 - Properly applied IS_ENABLED() to the function bodies of callbacks
   in drivers/base/cpu|memory.c.
 - Re-ran compile and run-time testing of the impacted attributes for
   both enabled and not enabled config settings.

v24: 28jun2023
 https://lore.kernel.org/lkml/20230628185215.40707-1-eric.devolder at oracle.com/
 - Rebased onto 6.4.0
 - Included Documentation/ABI/testing entries for the new sysfs
   crash_hotplug attributes, per Greg Kroah-Hartman.
 - Refactored drivers/base/cpu|memory.c to use the .is_visible()
   method for attributes, per Greg Kroah-Hartman.
 - Retained all existing Acks and RBs as the few changes as a result
   of Greg's requests were trivial.

v23: 12jun2023
 https://lore.kernel.org/lkml/20230612210712.683175-1-eric.devolder at oracle.com/
 - Rebased onto 6.4.0-rc6
 - Refactored Kconfig, per Thomas. See series:
   https://lore.kernel.org/lkml/20230612172805.681179-1-eric.devolder at oracle.com/
 - Reworked commit messages to conform to style, per Thomas.
 - Applied Baoquan He Acked-by to kexec_load() patch.
 - Applied Hari Bathini Acked-by for the series.
 - No code changes.

v22: 3may2023
 https://lore.kernel.org/lkml/20230503224145.7405-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0
 - Improved support for kexec_load(), per Hari Bathini. See
   "crash: hotplug support for kexec_load()" which is the only
   change to this series.
 - Applied Baoquan He's Acked-by for all other patches.

v21: 4apr2023
 https://lkml.org/lkml/2023/4/4/1136
 https://lore.kernel.org/lkml/20230404180326.6890-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0-rc5
 - Additional simplification of indentation in crash_handle_hotplug_event(),
   per Baoquan.

v20: 17mar2023
 https://lkml.org/lkml/2023/3/17/1169
 https://lore.kernel.org/lkml/20230317212128.21424-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0-rc2
 - Defaulting CRASH_HOTPLUG for x86 to Y, per Sourabh.
 - Explicitly initializing image->hp_action, per Baoquan.
 - Simplified kexec_trylock() in crash_handle_hotplug_event(),
   per Baoquan.
 - Applied Sourabh's Reviewed-by to the series.

v19: 6mar2023
 https://lkml.org/lkml/2023/3/6/1358
 https://lore.kernel.org/lkml/20230306162228.8277-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0
 - Did away with offlinecpu, per Thomas Gleixner.
 - Changed to CPUHP_BP_PREPARE_DYN instead of CPUHP_AP_ONLINE_DYN.
 - Did away with elfcorehdr_index_valid, per Sourabh.
 - Convert to for_each_possible_cpu() in crash_prepare_elf64_headers()
   per Sourabh.
 - Small optimization for x86 cpu changes.

v18: 31jan2023
 https://lkml.org/lkml/2023/1/31/1356
 https://lore.kernel.org/lkml/20230131224236.122805-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc6
 - Renamed struct kimage member hotplug_event to hp_action, and
   re-enumerated the KEXEC_CRASH_HP_x items, adding _NONE at 0.
 - Moved to cpuhp state CPUHP_BP_PREPARE_DYN instead of
   CPUHP_AP_ONLINE_DYN in order to minimize window of time CPU
   is not reflected in elfcorehdr.
 - Reworked some of the comments and commit messages to offer
   more of the why, than what, per Thomas Gleixner.

v17: 18jan2023
 https://lkml.org/lkml/2023/1/18/1420
 https://lore.kernel.org/lkml/20230118213544.2128-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc4
 - Moved a bit of code around so that kexec_load()-only builds
   work, per Sourabh.
 - Corrected computation of number of memory region Phdrs needed
   when x86 memory hotplug is not enabled, per Baoquan.

v16: 5jan2023
 https://lkml.org/lkml/2023/1/5/673
 https://lore.kernel.org/lkml/20230105151709.1845-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc2
 - Corrected error identified by Baoquan.

v15: 9dec2022
 https://lkml.org/lkml/2022/12/9/520
 https://lore.kernel.org/lkml/20221209153656.3284-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc8
 - Replaced arch_un/map_crash_pages() with direct use of
   kun/map_local_pages(), per Boris.
 - Some x86 changes, per Boris.

v14: 16nov2022
 https://lkml.org/lkml/2022/11/16/1645
 https://lore.kernel.org/lkml/20221116214643.6384-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc5
 - Introduced CRASH_HOTPLUG Kconfig item to better fine tune
   compilation of feature components, per Boris.
 - Removed hp_action parameter to arch_crash_handle_hotplug_event()
   as it is unused.

v13: 31oct2022
 https://lkml.org/lkml/2022/10/31/854
 https://lore.kernel.org/lkml/20221031193604.28779-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc3, which means converting to use the new
   kexec_trylock() away from mutex_lock(kexec_mutex).
 - Moved arch_un/map_crash_pages() into kexec.h and default
   implementation using k/unmap_local_pages().
 - Changed more #ifdef's into IS_ENABLED()
 - Changed CRASH_MAX_MEMORY_RANGES to 8192 from 32768, and it moved
   into x86 crash.c as #define rather Kconfig item, per Boris.
 - Check number of Phdrs against PN_XNUM, max possible.

v12: 9sep2022
 https://lkml.org/lkml/2022/9/9/1358
 https://lore.kernel.org/lkml/20220909210509.6286-1-eric.devolder at oracle.com/
 - Rebased onto 6.0-rc4
 - Addressed some minor formatting items, per Baoquan

v11: 26aug2022
 https://lkml.org/lkml/2022/8/26/963
 https://lore.kernel.org/lkml/20220826173704.1895-1-eric.devolder at oracle.com/
 - Rebased onto 6.0-rc2
 - Redid the rework of __weak to use asm/kexec.h, per Baoquan
 - Reworked some comments and minor items, per Baoquan

v10: 21jul2022
 https://lkml.org/lkml/2022/7/21/1007
 https://lore.kernel.org/lkml/20220721181747.1640-1-eric.devolder at oracle.com/
 - Rebased to 5.19.0-rc7
 - Per Sourabh, corrected build issue with arch_un/map_crash_pages()
   for architectures not supporting this feature.
 - Per David Hildebrand, removed the WARN_ONCE() altogether.
 - Per David Hansen, converted to use of kmap_local_page().
 - Per Baoquan He, replaced use of __weak with the kexec technique.

v9: 13jun2022
 https://lkml.org/lkml/2022/6/13/3382
 https://lore.kernel.org/lkml/20220613224240.79400-1-eric.devolder at oracle.com/
 - Rebased to 5.18.0
 - Per Sourabh, moved crash_prepare_elf64_headers() into common
   crash_core.c to avoid compile issues with kexec_load only path.
 - Per David Hildebrand, replaced mutex_trylock() with mutex_lock().
 - Changed the __weak arch_crash_handle_hotplug_event() to utilize
   WARN_ONCE() instead of WARN(). Fix some formatting issues.
 - Per Sourabh, introduced sysfs attribute crash_hotplug for memory
   and CPUs; for use by userspace (udev) to determine if the kernel
   performs crash hot un/plug support.
 - Per Sourabh, moved the code detecting the elfcorehdr segment from
   arch/x86 into crash_core:handle_hotplug_event() so both kexec_load
   and kexec_file_load can benefit.
 - Updated userspace kexec-tools kexec utility to reflect change to
   using CRASH_MAX_MEMORY_RANGES and get_nr_cpus().
 - Updated the new proposed udev rules to reflect using the sysfs
   attributes crash_hotplug.

v8: 5may2022
 https://lkml.org/lkml/2022/5/5/1133
 https://lore.kernel.org/lkml/20220505184603.1548-1-eric.devolder at oracle.com/
 - Per Borislav Petkov, eliminated CONFIG_CRASH_HOTPLUG in favor
   of CONFIG_HOTPLUG_CPU || CONFIG_MEMORY_HOTPLUG, ie a new define
   is not needed. Also use of IS_ENABLED() rather than #ifdef's.
   Renamed crash_hotplug_handler() to handle_hotplug_event().
   And other corrections.
 - Per Baoquan, minimized the parameters to the arch_crash_
   handle_hotplug_event() to hp_action and cpu.
 - Introduce KEXEC_CRASH_HP_INVALID_CPU definition, per Baoquan.
 - Per Sourabh Jain, renamed and repurposed CRASH_HOTPLUG_ELFCOREHDR_SZ
   to CONFIG_CRASH_MAX_MEMORY_RANGES, mirroring kexec-tools change
   by David Hildebrand. Folded this patch into the x86
   kexec_file_load support patch.

v7: 13apr2022
 https://lkml.org/lkml/2022/4/13/850
 https://lore.kernel.org/lkml/20220413164237.20845-1-eric.devolder at oracle.com/
 - Resolved parameter usage to crash_hotplug_handler(), per Baoquan.

v6: 1apr2022
 https://lkml.org/lkml/2022/4/1/1203
 https://lore.kernel.org/lkml/20220401183040.1624-1-eric.devolder at oracle.com/
 - Reword commit messages and some comment cleanup per Baoquan.
 - Changed elf_index to elfcorehdr_index for clarity.
 - Minor code changes per Baoquan.

v5: 3mar2022
 https://lkml.org/lkml/2022/3/3/674
 https://lore.kernel.org/lkml/20220303162725.49640-1-eric.devolder at oracle.com/
 - Reworded description of CRASH_HOTPLUG_ELFCOREHDR_SZ, per
   David Hildenbrand.
 - Refactored slightly a few patches per Baoquan recommendation.

v4: 9feb2022
 https://lkml.org/lkml/2022/2/9/1406
 https://lore.kernel.org/lkml/20220209195706.51522-1-eric.devolder at oracle.com/
 - Refactored patches per Baoquan suggestsions.
 - A few corrections, per Baoquan.

v3: 10jan2022
 https://lkml.org/lkml/2022/1/10/1212
 https://lore.kernel.org/lkml/20220110195727.1682-1-eric.devolder at oracle.com/
 - Rebasing per Baoquan He request.
 - Changed memory notifier per David Hildenbrand.
 - Providing example kexec userspace change in cover letter.

RFC v2: 7dec2021
 https://lkml.org/lkml/2021/12/7/1088
 https://lore.kernel.org/lkml/20211207195204.1582-1-eric.devolder at oracle.com/
 - Acting upon Baoquan He suggestion of removing elfcorehdr from
   the purgatory list of segments, removed purgatory code from
   patchset, and it is signficiantly simpler now.

RFC v1: 18nov2021
 https://lkml.org/lkml/2021/11/18/845
 https://lore.kernel.org/lkml/20211118174948.37435-1-eric.devolder at oracle.com/
 - working patchset demonstrating kernel handling of hotplug
   updates to x86 elfcorehdr for kexec_file_load

RFC: 14dec2020
 https://lkml.org/lkml/2020/12/14/532
 https://lore.kernel.org/lkml/b04ed259-dc5f-7f30-6661-c26f92d9096a at oracle.com/
 NOTE: s/vmcoreinfo/elfcorehdr/g
 - proposed concept of allowing kernel to handle hotplug update
   of elfcorehdr
---

Eric DeVolder (8):
  crash: move a few code bits to setup support of crash hotplug
  crash: add generic infrastructure for crash hotplug support
  kexec: exclude elfcorehdr from the segment digest
  crash: memory and CPU hotplug sysfs attributes
  x86/crash: add x86 crash hotplug support
  crash: hotplug support for kexec_load()
  crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()
  x86/crash: optimize CPU changes

 .../ABI/testing/sysfs-devices-memory          |   8 +
 .../ABI/testing/sysfs-devices-system-cpu      |   8 +
 .../admin-guide/mm/memory-hotplug.rst         |   8 +
 Documentation/core-api/cpu_hotplug.rst        |  18 +
 arch/x86/Kconfig                              |   3 +
 arch/x86/include/asm/kexec.h                  |  18 +
 arch/x86/kernel/crash.c                       | 140 ++++++-
 drivers/base/cpu.c                            |  13 +
 drivers/base/memory.c                         |  13 +
 include/linux/crash_core.h                    |   9 +
 include/linux/kexec.h                         |  63 +++-
 include/uapi/linux/kexec.h                    |   1 +
 kernel/Kconfig.kexec                          |  35 ++
 kernel/crash_core.c                           | 355 ++++++++++++++++++
 kernel/kexec.c                                |   5 +
 kernel/kexec_core.c                           |   6 +
 kernel/kexec_file.c                           | 187 +--------
 kernel/ksysfs.c                               |  15 +
 18 files changed, 700 insertions(+), 205 deletions(-)

-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug  4 14:03:52 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:52 -0400
Subject: [PATCH v26 1/8] crash: move a few code bits to setup support of crash hotplug
In-Reply-To: <20230804210359.8321-1-eric.devolder@oracle.com>
References: <20230804210359.8321-1-eric.devolder@oracle.com>
Message-ID: <20230804210359.8321-2-eric.devolder@oracle.com>

The crash hotplug support leans on the work for the kexec_file_load()
syscall. To also support the kexec_load() syscall, a few bits of code
need to be move outside of CONFIG_KEXEC_FILE. As such, these bits are
moved out of kexec_file.c and into a common location crash_core.c.

No functionality change intended.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 include/linux/kexec.h |  30 +++----
 kernel/crash_core.c   | 182 ++++++++++++++++++++++++++++++++++++++++++
 kernel/kexec_file.c   | 181 -----------------------------------------
 3 files changed, 197 insertions(+), 196 deletions(-)

diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 22b5cd24f581..811a90e09698 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -105,6 +105,21 @@ struct compat_kexec_segment {
 };
 #endif
 
+/* Alignment required for elf header segment */
+#define ELF_CORE_HEADER_ALIGN   4096
+
+struct crash_mem {
+	unsigned int max_nr_ranges;
+	unsigned int nr_ranges;
+	struct range ranges[];
+};
+
+extern int crash_exclude_mem_range(struct crash_mem *mem,
+				   unsigned long long mstart,
+				   unsigned long long mend);
+extern int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
+				       void **addr, unsigned long *sz);
+
 #ifdef CONFIG_KEXEC_FILE
 struct purgatory_info {
 	/*
@@ -230,21 +245,6 @@ static inline int arch_kexec_locate_mem_hole(struct kexec_buf *kbuf)
 }
 #endif
 
-/* Alignment required for elf header segment */
-#define ELF_CORE_HEADER_ALIGN   4096
-
-struct crash_mem {
-	unsigned int max_nr_ranges;
-	unsigned int nr_ranges;
-	struct range ranges[];
-};
-
-extern int crash_exclude_mem_range(struct crash_mem *mem,
-				   unsigned long long mstart,
-				   unsigned long long mend);
-extern int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
-				       void **addr, unsigned long *sz);
-
 #ifndef arch_kexec_apply_relocations_add
 /*
  * arch_kexec_apply_relocations_add - apply relocations of type RELA
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 90ce1dfd591c..b7c30b748a16 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -10,6 +10,7 @@
 #include <linux/utsname.h>
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
+#include <linux/kexec.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -314,6 +315,187 @@ static int __init parse_crashkernel_dummy(char *arg)
 }
 early_param("crashkernel", parse_crashkernel_dummy);
 
+int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
+			  void **addr, unsigned long *sz)
+{
+	Elf64_Ehdr *ehdr;
+	Elf64_Phdr *phdr;
+	unsigned long nr_cpus = num_possible_cpus(), nr_phdr, elf_sz;
+	unsigned char *buf;
+	unsigned int cpu, i;
+	unsigned long long notes_addr;
+	unsigned long mstart, mend;
+
+	/* extra phdr for vmcoreinfo ELF note */
+	nr_phdr = nr_cpus + 1;
+	nr_phdr += mem->nr_ranges;
+
+	/*
+	 * kexec-tools creates an extra PT_LOAD phdr for kernel text mapping
+	 * area (for example, ffffffff80000000 - ffffffffa0000000 on x86_64).
+	 * I think this is required by tools like gdb. So same physical
+	 * memory will be mapped in two ELF headers. One will contain kernel
+	 * text virtual addresses and other will have __va(physical) addresses.
+	 */
+
+	nr_phdr++;
+	elf_sz = sizeof(Elf64_Ehdr) + nr_phdr * sizeof(Elf64_Phdr);
+	elf_sz = ALIGN(elf_sz, ELF_CORE_HEADER_ALIGN);
+
+	buf = vzalloc(elf_sz);
+	if (!buf)
+		return -ENOMEM;
+
+	ehdr = (Elf64_Ehdr *)buf;
+	phdr = (Elf64_Phdr *)(ehdr + 1);
+	memcpy(ehdr->e_ident, ELFMAG, SELFMAG);
+	ehdr->e_ident[EI_CLASS] = ELFCLASS64;
+	ehdr->e_ident[EI_DATA] = ELFDATA2LSB;
+	ehdr->e_ident[EI_VERSION] = EV_CURRENT;
+	ehdr->e_ident[EI_OSABI] = ELF_OSABI;
+	memset(ehdr->e_ident + EI_PAD, 0, EI_NIDENT - EI_PAD);
+	ehdr->e_type = ET_CORE;
+	ehdr->e_machine = ELF_ARCH;
+	ehdr->e_version = EV_CURRENT;
+	ehdr->e_phoff = sizeof(Elf64_Ehdr);
+	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
+	ehdr->e_phentsize = sizeof(Elf64_Phdr);
+
+	/* Prepare one phdr of type PT_NOTE for each present CPU */
+	for_each_present_cpu(cpu) {
+		phdr->p_type = PT_NOTE;
+		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
+		phdr->p_offset = phdr->p_paddr = notes_addr;
+		phdr->p_filesz = phdr->p_memsz = sizeof(note_buf_t);
+		(ehdr->e_phnum)++;
+		phdr++;
+	}
+
+	/* Prepare one PT_NOTE header for vmcoreinfo */
+	phdr->p_type = PT_NOTE;
+	phdr->p_offset = phdr->p_paddr = paddr_vmcoreinfo_note();
+	phdr->p_filesz = phdr->p_memsz = VMCOREINFO_NOTE_SIZE;
+	(ehdr->e_phnum)++;
+	phdr++;
+
+	/* Prepare PT_LOAD type program header for kernel text region */
+	if (need_kernel_map) {
+		phdr->p_type = PT_LOAD;
+		phdr->p_flags = PF_R|PF_W|PF_X;
+		phdr->p_vaddr = (unsigned long) _text;
+		phdr->p_filesz = phdr->p_memsz = _end - _text;
+		phdr->p_offset = phdr->p_paddr = __pa_symbol(_text);
+		ehdr->e_phnum++;
+		phdr++;
+	}
+
+	/* Go through all the ranges in mem->ranges[] and prepare phdr */
+	for (i = 0; i < mem->nr_ranges; i++) {
+		mstart = mem->ranges[i].start;
+		mend = mem->ranges[i].end;
+
+		phdr->p_type = PT_LOAD;
+		phdr->p_flags = PF_R|PF_W|PF_X;
+		phdr->p_offset  = mstart;
+
+		phdr->p_paddr = mstart;
+		phdr->p_vaddr = (unsigned long) __va(mstart);
+		phdr->p_filesz = phdr->p_memsz = mend - mstart + 1;
+		phdr->p_align = 0;
+		ehdr->e_phnum++;
+		pr_debug("Crash PT_LOAD ELF header. phdr=%p vaddr=0x%llx, paddr=0x%llx, sz=0x%llx e_phnum=%d p_offset=0x%llx\n",
+			phdr, phdr->p_vaddr, phdr->p_paddr, phdr->p_filesz,
+			ehdr->e_phnum, phdr->p_offset);
+		phdr++;
+	}
+
+	*addr = buf;
+	*sz = elf_sz;
+	return 0;
+}
+
+int crash_exclude_mem_range(struct crash_mem *mem,
+			    unsigned long long mstart, unsigned long long mend)
+{
+	int i, j;
+	unsigned long long start, end, p_start, p_end;
+	struct range temp_range = {0, 0};
+
+	for (i = 0; i < mem->nr_ranges; i++) {
+		start = mem->ranges[i].start;
+		end = mem->ranges[i].end;
+		p_start = mstart;
+		p_end = mend;
+
+		if (mstart > end || mend < start)
+			continue;
+
+		/* Truncate any area outside of range */
+		if (mstart < start)
+			p_start = start;
+		if (mend > end)
+			p_end = end;
+
+		/* Found completely overlapping range */
+		if (p_start == start && p_end == end) {
+			mem->ranges[i].start = 0;
+			mem->ranges[i].end = 0;
+			if (i < mem->nr_ranges - 1) {
+				/* Shift rest of the ranges to left */
+				for (j = i; j < mem->nr_ranges - 1; j++) {
+					mem->ranges[j].start =
+						mem->ranges[j+1].start;
+					mem->ranges[j].end =
+							mem->ranges[j+1].end;
+				}
+
+				/*
+				 * Continue to check if there are another overlapping ranges
+				 * from the current position because of shifting the above
+				 * mem ranges.
+				 */
+				i--;
+				mem->nr_ranges--;
+				continue;
+			}
+			mem->nr_ranges--;
+			return 0;
+		}
+
+		if (p_start > start && p_end < end) {
+			/* Split original range */
+			mem->ranges[i].end = p_start - 1;
+			temp_range.start = p_end + 1;
+			temp_range.end = end;
+		} else if (p_start != start)
+			mem->ranges[i].end = p_start - 1;
+		else
+			mem->ranges[i].start = p_end + 1;
+		break;
+	}
+
+	/* If a split happened, add the split to array */
+	if (!temp_range.end)
+		return 0;
+
+	/* Split happened */
+	if (i == mem->max_nr_ranges - 1)
+		return -ENOMEM;
+
+	/* Location where new range should go */
+	j = i + 1;
+	if (j < mem->nr_ranges) {
+		/* Move over all ranges one slot towards the end */
+		for (i = mem->nr_ranges - 1; i >= j; i--)
+			mem->ranges[i + 1] = mem->ranges[i];
+	}
+
+	mem->ranges[j].start = temp_range.start;
+	mem->ranges[j].end = temp_range.end;
+	mem->nr_ranges++;
+	return 0;
+}
+
 Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len)
 {
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index dd115571f8fc..453b7a513540 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -1151,184 +1151,3 @@ int kexec_purgatory_get_set_symbol(struct kimage *image, const char *name,
 	return 0;
 }
 #endif /* CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY */
-
-int crash_exclude_mem_range(struct crash_mem *mem,
-			    unsigned long long mstart, unsigned long long mend)
-{
-	int i, j;
-	unsigned long long start, end, p_start, p_end;
-	struct range temp_range = {0, 0};
-
-	for (i = 0; i < mem->nr_ranges; i++) {
-		start = mem->ranges[i].start;
-		end = mem->ranges[i].end;
-		p_start = mstart;
-		p_end = mend;
-
-		if (mstart > end || mend < start)
-			continue;
-
-		/* Truncate any area outside of range */
-		if (mstart < start)
-			p_start = start;
-		if (mend > end)
-			p_end = end;
-
-		/* Found completely overlapping range */
-		if (p_start == start && p_end == end) {
-			mem->ranges[i].start = 0;
-			mem->ranges[i].end = 0;
-			if (i < mem->nr_ranges - 1) {
-				/* Shift rest of the ranges to left */
-				for (j = i; j < mem->nr_ranges - 1; j++) {
-					mem->ranges[j].start =
-						mem->ranges[j+1].start;
-					mem->ranges[j].end =
-							mem->ranges[j+1].end;
-				}
-
-				/*
-				 * Continue to check if there are another overlapping ranges
-				 * from the current position because of shifting the above
-				 * mem ranges.
-				 */
-				i--;
-				mem->nr_ranges--;
-				continue;
-			}
-			mem->nr_ranges--;
-			return 0;
-		}
-
-		if (p_start > start && p_end < end) {
-			/* Split original range */
-			mem->ranges[i].end = p_start - 1;
-			temp_range.start = p_end + 1;
-			temp_range.end = end;
-		} else if (p_start != start)
-			mem->ranges[i].end = p_start - 1;
-		else
-			mem->ranges[i].start = p_end + 1;
-		break;
-	}
-
-	/* If a split happened, add the split to array */
-	if (!temp_range.end)
-		return 0;
-
-	/* Split happened */
-	if (i == mem->max_nr_ranges - 1)
-		return -ENOMEM;
-
-	/* Location where new range should go */
-	j = i + 1;
-	if (j < mem->nr_ranges) {
-		/* Move over all ranges one slot towards the end */
-		for (i = mem->nr_ranges - 1; i >= j; i--)
-			mem->ranges[i + 1] = mem->ranges[i];
-	}
-
-	mem->ranges[j].start = temp_range.start;
-	mem->ranges[j].end = temp_range.end;
-	mem->nr_ranges++;
-	return 0;
-}
-
-int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
-			  void **addr, unsigned long *sz)
-{
-	Elf64_Ehdr *ehdr;
-	Elf64_Phdr *phdr;
-	unsigned long nr_cpus = num_possible_cpus(), nr_phdr, elf_sz;
-	unsigned char *buf;
-	unsigned int cpu, i;
-	unsigned long long notes_addr;
-	unsigned long mstart, mend;
-
-	/* extra phdr for vmcoreinfo ELF note */
-	nr_phdr = nr_cpus + 1;
-	nr_phdr += mem->nr_ranges;
-
-	/*
-	 * kexec-tools creates an extra PT_LOAD phdr for kernel text mapping
-	 * area (for example, ffffffff80000000 - ffffffffa0000000 on x86_64).
-	 * I think this is required by tools like gdb. So same physical
-	 * memory will be mapped in two ELF headers. One will contain kernel
-	 * text virtual addresses and other will have __va(physical) addresses.
-	 */
-
-	nr_phdr++;
-	elf_sz = sizeof(Elf64_Ehdr) + nr_phdr * sizeof(Elf64_Phdr);
-	elf_sz = ALIGN(elf_sz, ELF_CORE_HEADER_ALIGN);
-
-	buf = vzalloc(elf_sz);
-	if (!buf)
-		return -ENOMEM;
-
-	ehdr = (Elf64_Ehdr *)buf;
-	phdr = (Elf64_Phdr *)(ehdr + 1);
-	memcpy(ehdr->e_ident, ELFMAG, SELFMAG);
-	ehdr->e_ident[EI_CLASS] = ELFCLASS64;
-	ehdr->e_ident[EI_DATA] = ELFDATA2LSB;
-	ehdr->e_ident[EI_VERSION] = EV_CURRENT;
-	ehdr->e_ident[EI_OSABI] = ELF_OSABI;
-	memset(ehdr->e_ident + EI_PAD, 0, EI_NIDENT - EI_PAD);
-	ehdr->e_type = ET_CORE;
-	ehdr->e_machine = ELF_ARCH;
-	ehdr->e_version = EV_CURRENT;
-	ehdr->e_phoff = sizeof(Elf64_Ehdr);
-	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
-	ehdr->e_phentsize = sizeof(Elf64_Phdr);
-
-	/* Prepare one phdr of type PT_NOTE for each present CPU */
-	for_each_present_cpu(cpu) {
-		phdr->p_type = PT_NOTE;
-		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
-		phdr->p_offset = phdr->p_paddr = notes_addr;
-		phdr->p_filesz = phdr->p_memsz = sizeof(note_buf_t);
-		(ehdr->e_phnum)++;
-		phdr++;
-	}
-
-	/* Prepare one PT_NOTE header for vmcoreinfo */
-	phdr->p_type = PT_NOTE;
-	phdr->p_offset = phdr->p_paddr = paddr_vmcoreinfo_note();
-	phdr->p_filesz = phdr->p_memsz = VMCOREINFO_NOTE_SIZE;
-	(ehdr->e_phnum)++;
-	phdr++;
-
-	/* Prepare PT_LOAD type program header for kernel text region */
-	if (need_kernel_map) {
-		phdr->p_type = PT_LOAD;
-		phdr->p_flags = PF_R|PF_W|PF_X;
-		phdr->p_vaddr = (unsigned long) _text;
-		phdr->p_filesz = phdr->p_memsz = _end - _text;
-		phdr->p_offset = phdr->p_paddr = __pa_symbol(_text);
-		ehdr->e_phnum++;
-		phdr++;
-	}
-
-	/* Go through all the ranges in mem->ranges[] and prepare phdr */
-	for (i = 0; i < mem->nr_ranges; i++) {
-		mstart = mem->ranges[i].start;
-		mend = mem->ranges[i].end;
-
-		phdr->p_type = PT_LOAD;
-		phdr->p_flags = PF_R|PF_W|PF_X;
-		phdr->p_offset  = mstart;
-
-		phdr->p_paddr = mstart;
-		phdr->p_vaddr = (unsigned long) __va(mstart);
-		phdr->p_filesz = phdr->p_memsz = mend - mstart + 1;
-		phdr->p_align = 0;
-		ehdr->e_phnum++;
-		pr_debug("Crash PT_LOAD ELF header. phdr=%p vaddr=0x%llx, paddr=0x%llx, sz=0x%llx e_phnum=%d p_offset=0x%llx\n",
-			phdr, phdr->p_vaddr, phdr->p_paddr, phdr->p_filesz,
-			ehdr->e_phnum, phdr->p_offset);
-		phdr++;
-	}
-
-	*addr = buf;
-	*sz = elf_sz;
-	return 0;
-}
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug  4 14:03:59 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:59 -0400
Subject: [PATCH v26 8/8] x86/crash: optimize CPU changes
In-Reply-To: <20230804210359.8321-1-eric.devolder@oracle.com>
References: <20230804210359.8321-1-eric.devolder@oracle.com>
Message-ID: <20230804210359.8321-9-eric.devolder@oracle.com>

crash_prepare_elf64_headers() writes into the elfcorehdr an ELF
PT_NOTE for all possible CPUs. As such, subsequent changes to CPUs
(ie. hot un/plug, online/offline) do not need to rewrite the elfcorehdr.

The kimage->file_mode term covers kdump images loaded via the
kexec_file_load() syscall. Since crash_prepare_elf64_headers()
wrote the initial elfcorehdr, no update to the elfcorehdr is
needed for CPU changes.

The kimage->elfcorehdr_updated term covers kdump images loaded via
the kexec_load() syscall. At least one memory or CPU change must occur
to cause crash_prepare_elf64_headers() to rewrite the elfcorehdr.
Afterwards, no update to the elfcorehdr is needed for CPU changes.

This code is intentionally *NOT* hoisted into
crash_handle_hotplug_event() as it would prevent the arch-specific
handler from running for CPU changes. This would break PPC, for
example, which needs to update other information besides the
elfcorehdr, on CPU changes.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/kernel/crash.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index caf22bcb61af..18d2a18d1073 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -467,6 +467,16 @@ void arch_crash_handle_hotplug_event(struct kimage *image)
 	unsigned long mem, memsz;
 	unsigned long elfsz = 0;
 
+	/*
+	 * As crash_prepare_elf64_headers() has already described all
+	 * possible CPUs, there is no need to update the elfcorehdr
+	 * for additional CPU changes.
+	 */
+	if ((image->file_mode || image->elfcorehdr_updated) &&
+		((image->hp_action == KEXEC_CRASH_HP_ADD_CPU) ||
+		(image->hp_action == KEXEC_CRASH_HP_REMOVE_CPU)))
+		return;
+
 	/*
 	 * Create the new elfcorehdr reflecting the changes to CPU and/or
 	 * memory resources.
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug  4 14:03:54 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:54 -0400
Subject: [PATCH v26 3/8] kexec: exclude elfcorehdr from the segment digest
In-Reply-To: <20230804210359.8321-1-eric.devolder@oracle.com>
References: <20230804210359.8321-1-eric.devolder@oracle.com>
Message-ID: <20230804210359.8321-4-eric.devolder@oracle.com>

When a crash kernel is loaded via the kexec_file_load() syscall, the
kernel places the various segments (ie crash kernel, crash initrd,
boot_params, elfcorehdr, purgatory, etc) in memory. For those
architectures that utilize purgatory, a hash digest of the segments
is calculated for integrity checking. The digest is embedded into
the purgatory image prior to placing in memory.

Updates to the elfcorehdr in response to CPU and memory changes
would cause the purgatory integrity checking to fail (at crash time,
and no vmcore created). Therefore, the elfcorehdr segment is
explicitly excluded from the purgatory digest, enabling updates to
the elfcorehdr while also avoiding the need to recompute the hash
digest and reload purgatory.

Suggested-by: Baoquan He <bhe at redhat.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 kernel/kexec_file.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index 453b7a513540..e2ec9d7b9a1f 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -726,6 +726,12 @@ static int kexec_calculate_store_digests(struct kimage *image)
 	for (j = i = 0; i < image->nr_segments; i++) {
 		struct kexec_segment *ksegment;
 
+#ifdef CONFIG_CRASH_HOTPLUG
+		/* Exclude elfcorehdr segment to allow future changes via hotplug */
+		if (j == image->elfcorehdr_index)
+			continue;
+#endif
+
 		ksegment = &image->segment[i];
 		/*
 		 * Skip purgatory as it will be modified once we put digest
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug  4 14:03:58 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri,  4 Aug 2023 17:03:58 -0400
Subject: [PATCH v26 7/8] crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()
In-Reply-To: <20230804210359.8321-1-eric.devolder@oracle.com>
References: <20230804210359.8321-1-eric.devolder@oracle.com>
Message-ID: <20230804210359.8321-8-eric.devolder@oracle.com>

The function crash_prepare_elf64_headers() generates the elfcorehdr
which describes the CPUs and memory in the system for the crash kernel.
In particular, it writes out ELF PT_NOTEs for memory regions and the
CPUs in the system.

With respect to the CPUs, the current implementation utilizes
for_each_present_cpu() which means that as CPUs are added and removed,
the elfcorehdr must again be updated to reflect the new set of CPUs.

The reasoning behind the move to use for_each_possible_cpu(), is:

- At kernel boot time, all percpu crash_notes are allocated for all
  possible CPUs; that is, crash_notes are not allocated dynamically
  when CPUs are plugged/unplugged. Thus the crash_notes for each
  possible CPU are always available.

- The crash_prepare_elf64_headers() creates an ELF PT_NOTE per CPU.
  Changing to for_each_possible_cpu() is valid as the crash_notes
  pointed to by each CPU PT_NOTE are present and always valid.

Furthermore, examining a common crash processing path of:

 kernel panic -> crash kernel -> makedumpfile -> 'crash' analyzer
           elfcorehdr      /proc/vmcore     vmcore

reveals how the ELF CPU PT_NOTEs are utilized:

- Upon panic, each CPU is sent an IPI and shuts itself down, recording
 its state in its crash_notes. When all CPUs are shutdown, the
 crash kernel is launched with a pointer to the elfcorehdr.

- The crash kernel via linux/fs/proc/vmcore.c does not examine or
 use the contents of the PT_NOTEs, it exposes them via /proc/vmcore.

- The makedumpfile utility uses /proc/vmcore and reads the CPU
 PT_NOTEs to craft a nr_cpus variable, which is reported in a
 header but otherwise generally unused. Makedumpfile creates the
 vmcore.

- The 'crash' dump analyzer does not appear to reference the CPU
 PT_NOTEs. Instead it looks-up the cpu_[possible|present|onlin]_mask
 symbols and directly examines those structure contents from vmcore
 memory. From that information it is able to determine which CPUs
 are present and online, and locate the corresponding crash_notes.
 Said differently, it appears that 'crash' analyzer does not rely
 on the ELF PT_NOTEs for CPUs; rather it obtains the information
 directly via kernel symbols and the memory within the vmcore.

(There maybe other vmcore generating and analysis tools that do use
these PT_NOTEs, but 'makedumpfile' and 'crash' seems to be the most
common solution.)

This results in the benefit of having all CPUs described in the
elfcorehdr, and therefore reducing the need to re-generate the
elfcorehdr on CPU changes, at the small expense of an additional
56 bytes per PT_NOTE for not-present-but-possible CPUs.

On systems where kexec_file_load() syscall is utilized, all the above
is valid. On systems where kexec_load() syscall is utilized, there
may be the need for the elfcorehdr to be regenerated once. The reason
being that some archs only populate the 'present' CPUs from the
/sys/devices/system/cpus entries, which the userspace 'kexec' utility
uses to generate the userspace-supplied elfcorehdr. In this situation,
one memory or CPU change will rewrite the elfcorehdr via the
crash_prepare_elf64_headers() function and now all possible CPUs will
be described, just as with kexec_file_load() syscall.

Suggested-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 kernel/crash_core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index fa918176d46d..7378b501fada 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -364,8 +364,8 @@ int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
 	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
 	ehdr->e_phentsize = sizeof(Elf64_Phdr);
 
-	/* Prepare one phdr of type PT_NOTE for each present CPU */
-	for_each_present_cpu(cpu) {
+	/* Prepare one phdr of type PT_NOTE for each possible CPU */
+	for_each_possible_cpu(cpu) {
 		phdr->p_type = PT_NOTE;
 		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
 		phdr->p_offset = phdr->p_paddr = notes_addr;
-- 
2.31.1



From bimmtu at gmail.com  Fri Aug  4 23:38:10 2023
From: bimmtu at gmail.com (Bintu Felicia)
Date: Sat, 5 Aug 2023 07:38:10 +0100
Subject: HELLO...,
Message-ID: <CAAF5RuyAstj_aNt4=9isuh=YJOE4tHO7zqgAKyrdD3b0X1u+hw@mail.gmail.com>

How are you today? I hope you are fine. My name is Miss
Bintu Felicia . l am single looking for honest and nice
person whom i can partner with . I don't care about
your color, ethnicity, Status or Sex. Upon your reply to
this mail I will tell you more about myself and send you
more of my picture .I am sending you this beautiful mail
with a wish for much happiness


From bp at alien8.de  Sat Aug  5 02:17:38 2023
From: bp at alien8.de (Borislav Petkov)
Date: Sat, 5 Aug 2023 11:17:38 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <CAMj1kXHnSzdQw5CMAVXU7EzpnrdRvAqKZVgA+EV35kHmwVULgQ@mail.gmail.com>
References: <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local>
 <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local>
 <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local>
 <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
 <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local>
 <CAMj1kXEM5hGknVGwHh_w99D4L8yrYrTFycwGHZ0CQun70CLipw@mail.gmail.com>
 <20230802155146.GCZMp7ksDdN2ETVzKV@fat_crate.local>
 <CAMj1kXHnSzdQw5CMAVXU7EzpnrdRvAqKZVgA+EV35kHmwVULgQ@mail.gmail.com>
Message-ID: <20230805091738.GAZM4TslVWR3HHKeUC@fat_crate.local>

On Thu, Aug 03, 2023 at 01:11:54PM +0200, Ard Biesheuvel wrote:
> Sadly, not only 'old' grubs - GRUB mainline only recently added
> support for booting Linux/x86 via the EFI stub (because I wrote the
> code for them),

haha.

> but it will still fall back to the previous mode for kernels that are
> built without EFI stub support, or which are older than ~v5.8 (because
> their EFI stub does not implement the generic EFI initrd loading
> mechanism)

The thing is, those SNP kernels pretty much use the EFI boot mechanism.
I mean, don't take my word for it as I run SNP guests only from time to
time but that's what everyone uses AFAIK.

> Yeah. what seems to be saving our ass here is that startup_32 maps the
> first 1G of physical address space 4 times, and x86_64 EFI usually
> puts firmware tables below 4G. This means the cc blob check doesn't
> fault, but it may dereference bogus memory traversing the config table
> array looking for the cc blob GUID. However, the system table field
> holding the size of the array may also appear as bogus so this may
> still break in weird ways.

Oh fun.

> I don't think we can kill it yet, but it would be nice if we could
> avoid the need to support SNP boot when entering that way.

That's what I meant - not boot SNP guests through the 32-bit entry path.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette


From bp at alien8.de  Sat Aug  5 02:19:32 2023
From: bp at alien8.de (Borislav Petkov)
Date: Sat, 5 Aug 2023 11:19:32 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <CAMj1kXESuCScMLLAS4tSDcYxA3JTb24RuF7ipcKGd65tBvOBWQ@mail.gmail.com>
References: <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local>
 <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local>
 <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
 <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local>
 <CAMj1kXEM5hGknVGwHh_w99D4L8yrYrTFycwGHZ0CQun70CLipw@mail.gmail.com>
 <20230802155146.GCZMp7ksDdN2ETVzKV@fat_crate.local>
 <CAMj1kXHnSzdQw5CMAVXU7EzpnrdRvAqKZVgA+EV35kHmwVULgQ@mail.gmail.com>
 <CAMj1kXESuCScMLLAS4tSDcYxA3JTb24RuF7ipcKGd65tBvOBWQ@mail.gmail.com>
Message-ID: <20230805091932.GBZM4UJC2wL4S2y2Jc@fat_crate.local>

On Thu, Aug 03, 2023 at 04:27:41PM +0200, Ard Biesheuvel wrote:
> https://lists.gnu.org/archive/html/grub-devel/2023-08/msg00005.html
> 
> Coming to your distro any decade now!

Cool. The less 32-bit crap we have to deal with, the better.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette


From ardb at kernel.org  Sun Aug  6 02:00:11 2023
From: ardb at kernel.org (Ard Biesheuvel)
Date: Sun, 6 Aug 2023 11:00:11 +0200
Subject: [PATCH v2] x86/kexec: Add EFI config table identity mapping for
 kexec kernel
In-Reply-To: <20230805091738.GAZM4TslVWR3HHKeUC@fat_crate.local>
References: <20230717141409.GGZLVMsU6d/9mpJvMO@fat_crate.local>
 <CAO7dBbXJv9JzDbSa-DLT03+osYCQXNUXFwz63gbq=NGDxEVyEA@mail.gmail.com>
 <20230728165535.GDZMPzB/ek5QM+xJqA@fat_crate.local> <CAO7dBbVyuLHH6RfdVQkU5ThXaJ-F4yvFAYD1PDNGkOpph9xvnA@mail.gmail.com>
 <20230802093927.GAZMokT57anC5jBISK@fat_crate.local> <99cb3813-1737-9d10-1f24-77565e460c55@amd.com>
 <20230802135856.GBZMphIHHLa3dXRRVe@fat_crate.local> <CAMj1kXEM5hGknVGwHh_w99D4L8yrYrTFycwGHZ0CQun70CLipw@mail.gmail.com>
 <20230802155146.GCZMp7ksDdN2ETVzKV@fat_crate.local> <CAMj1kXHnSzdQw5CMAVXU7EzpnrdRvAqKZVgA+EV35kHmwVULgQ@mail.gmail.com>
 <20230805091738.GAZM4TslVWR3HHKeUC@fat_crate.local>
Message-ID: <CAMj1kXFf1QuiYo+5gm_1X3OHStHqO=OtmumvQM=D75uNP=mpTQ@mail.gmail.com>

On Sat, 5 Aug 2023 at 11:18, Borislav Petkov <bp at alien8.de> wrote:
>
> On Thu, Aug 03, 2023 at 01:11:54PM +0200, Ard Biesheuvel wrote:
> > Sadly, not only 'old' grubs - GRUB mainline only recently added
> > support for booting Linux/x86 via the EFI stub (because I wrote the
> > code for them),
>
> haha.
>
> > but it will still fall back to the previous mode for kernels that are
> > built without EFI stub support, or which are older than ~v5.8 (because
> > their EFI stub does not implement the generic EFI initrd loading
> > mechanism)
>
> The thing is, those SNP kernels pretty much use the EFI boot mechanism.
> I mean, don't take my word for it as I run SNP guests only from time to
> time but that's what everyone uses AFAIK.
>
> > Yeah. what seems to be saving our ass here is that startup_32 maps the
> > first 1G of physical address space 4 times, and x86_64 EFI usually
> > puts firmware tables below 4G. This means the cc blob check doesn't
> > fault, but it may dereference bogus memory traversing the config table
> > array looking for the cc blob GUID. However, the system table field
> > holding the size of the array may also appear as bogus so this may
> > still break in weird ways.
>
> Oh fun.
>

This is not actually true, I misread the code.

The initial mapping is 1:1 for the lower 4G of system memory, so
anything that lives there is accessible before the demand paging stuff
is up and running.

IOW, your change should be sufficient to fix this even when entering
via the 32-bit entry point.


From liuwenyu7 at huawei.com  Sun Aug  6 19:52:06 2023
From: liuwenyu7 at huawei.com (Wenyu Liu)
Date: Mon, 7 Aug 2023 10:52:06 +0800
Subject: [PATCH v2 RESEND] kexec_lock: Replace kexec_mutex() by kexec_lock() in two comments
Message-ID: <20230807025206.3682381-1-liuwenyu7@huawei.com>

kexec_mutex is replaced by an atomic variable
in 05c6257433b (panic, kexec: make __crash_kexec() NMI safe).

But there are still two comments that referenced kexec_mutex,
replace them by kexec_lock.

Signed-off-by: Wenyu Liu <liuwenyu7 at huawei.com>
Acked-by: Baoquan He <bhe at redhat.com>
Acked-by: Paul Menzel <pmenzel at molgen.mpg.de>
---
v1 -> v2
 - fixed some mistakes in the submission information

 kernel/kexec_file.c                | 2 +-
 security/integrity/ima/ima_kexec.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index 881ba0d1714c..b5bbb2fe0668 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -624,7 +624,7 @@ int kexec_locate_mem_hole(struct kexec_buf *kbuf)
  * kexec_add_buffer - place a buffer in a kexec segment
  * @kbuf:	Buffer contents and memory parameters.
  *
- * This function assumes that kexec_mutex is held.
+ * This function assumes that kexec_lock is held.
  * On successful return, @kbuf->mem will have the physical address of
  * the buffer in memory.
  *
diff --git a/security/integrity/ima/ima_kexec.c b/security/integrity/ima/ima_kexec.c
index 419dc405c831..ad133fe120db 100644
--- a/security/integrity/ima/ima_kexec.c
+++ b/security/integrity/ima/ima_kexec.c
@@ -77,7 +77,7 @@ static int ima_dump_measurement_list(unsigned long *buffer_size, void **buffer,
  * Called during kexec_file_load so that IMA can add a segment to the kexec
  * image for the measurement list for the next kernel.
  *
- * This function assumes that kexec_mutex is held.
+ * This function assumes that kexec_lock is held.
  */
 void ima_add_kexec_buffer(struct kimage *image)
 {
-- 
2.33.0



From horms at kernel.org  Mon Aug  7 06:21:48 2023
From: horms at kernel.org (Simon Horman)
Date: Mon, 7 Aug 2023 15:21:48 +0200
Subject: [PATCHv7 1/5] kexec/arm64: Simplify the code for zImage
In-Reply-To: <20230803024152.11663-2-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com>
 <20230803024152.11663-2-piliu@redhat.com>
Message-ID: <ZNDv7L3lWXZxrmWT@vergenet.net>

On Thu, Aug 03, 2023 at 10:41:48AM +0800, Pingfan Liu wrote:
> Inside zimage_probe(), it uncompresses the kernel and performs some
> check, similar to image_probe(). Taking a close look, the uncompressing
> has already executed before the image probe is called. What is missing
> here is to provide a fd, pointing to an uncompressed kernel image.
> 
> This patch creates a memfd based on the result produced by
> slurp_decompress_file(), and finally simplify the logical of the probe
> for aarch64.
> 
> The credit goes to the Dave Young, who contributes the original code.
> 
> Signed-off-by: Pingfan Liu <piliu at redhat.com>
> Co-authored-by: Dave Young <dyoung at redhat.com>

Dave, could I get a Signed-off-by line from you?
Likewise for patch 2/5.

I think simply replying to the relevant emails should be sufficient.


From horms at kernel.org  Mon Aug  7 06:22:59 2023
From: horms at kernel.org (Simon Horman)
Date: Mon, 7 Aug 2023 15:22:59 +0200
Subject: [PATCHv7 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <20230803024152.11663-4-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com>
 <20230803024152.11663-4-piliu@redhat.com>
Message-ID: <ZNDwM0YfAHT7PDef@vergenet.net>

On Thu, Aug 03, 2023 at 10:41:50AM +0800, Pingfan Liu wrote:
> From: Jeremy Linton <jeremy.linton at arm.com>
> 
> The linux kernel CONFIG_ZBOOT option creates
> self decompressing PE kernel images. So this means
> that kexec should have a generic understanding of
> the format which may be used by multiple arches.
> 
> So lets add an arch independent validation
> and decompression routine.
> 
> Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> [Modified by Pingfan to export kernel fd]
> Signed-off-by: Pingfan Liu <piliu at redhat.com>
> To: kexec at lists.infradead.org
> Cc: horms at verge.net.au
> Cc: ardb at kernel.org
> Cc: jeremy.linton at arm.com
> ---
>  include/Makefile         |   1 +
>  include/kexec-pe-zboot.h |  15 +++++
>  kexec/Makefile           |   1 +
>  kexec/kexec-pe-zboot.c   | 131 +++++++++++++++++++++++++++++++++++++++
>  4 files changed, 148 insertions(+)
>  create mode 100644 include/kexec-pe-zboot.h
>  create mode 100644 kexec/kexec-pe-zboot.c
> 
> diff --git a/include/Makefile b/include/Makefile
> index 621ce9f..cd88a26 100644
> --- a/include/Makefile
> +++ b/include/Makefile
> @@ -1,6 +1,7 @@
>  dist += include/Makefile		\
>  	include/config.h		\
>  	include/config.h.in		\
> +	include/kexec-pe-zboot.h	\
>  	include/kexec-uImage.h		\
>  	include/x86/x86-linux.h		\
>  	include/x86/mb_info.h		\
> diff --git a/include/kexec-pe-zboot.h b/include/kexec-pe-zboot.h
> new file mode 100644
> index 0000000..c588ca2
> --- /dev/null
> +++ b/include/kexec-pe-zboot.h
> @@ -0,0 +1,15 @@
> +#ifndef __KEXEC_PE_ZBOOT_H__
> +#define __KEXEC_PE_ZBOOT_H__
> +
> +/* see drivers/firmware/efi/libstub/zboot-header.S */
> +struct linux_pe_zboot_header {
> +	uint32_t mz_magic;
> +        uint32_t image_type;
> +        uint32_t payload_offset;
> +        uint32_t payload_size;
> +        uint32_t reserved[2];
> +        uint32_t compress_type;
> +};

nit: the above should consistently use tabs for indentation.
     I can try and fix this when applying if there is no v8.


From dyoung at redhat.com  Mon Aug  7 06:26:49 2023
From: dyoung at redhat.com (Dave Young)
Date: Mon, 7 Aug 2023 21:26:49 +0800
Subject: [PATCHv7 1/5] kexec/arm64: Simplify the code for zImage
In-Reply-To: <ZNDv7L3lWXZxrmWT@vergenet.net>
References: <20230803024152.11663-1-piliu@redhat.com> <20230803024152.11663-2-piliu@redhat.com>
 <ZNDv7L3lWXZxrmWT@vergenet.net>
Message-ID: <CALu+AoS-Df2zrAAHnjHnjc-tZPqGfnme2O-xj28Qofg5JdaR9Q@mail.gmail.com>

On Mon, 7 Aug 2023 at 21:23, Simon Horman <horms at kernel.org> wrote:
>
> On Thu, Aug 03, 2023 at 10:41:48AM +0800, Pingfan Liu wrote:
> > Inside zimage_probe(), it uncompresses the kernel and performs some
> > check, similar to image_probe(). Taking a close look, the uncompressing
> > has already executed before the image probe is called. What is missing
> > here is to provide a fd, pointing to an uncompressed kernel image.
> >
> > This patch creates a memfd based on the result produced by
> > slurp_decompress_file(), and finally simplify the logical of the probe
> > for aarch64.
> >
> > The credit goes to the Dave Young, who contributes the original code.
> >
> > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > Co-authored-by: Dave Young <dyoung at redhat.com>
>
> Dave, could I get a Signed-off-by line from you?
> Likewise for patch 2/5.

I Simon, sounds good to me, thanks!

>
> I think simply replying to the relevant emails should be sufficient.
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
>



From zohar at linux.ibm.com  Mon Aug  7 07:12:16 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Mon, 07 Aug 2023 10:12:16 -0400
Subject: [PATCH v2 RESEND] kexec_lock: Replace kexec_mutex() by
 kexec_lock() in two comments
In-Reply-To: <20230807025206.3682381-1-liuwenyu7@huawei.com>
References: <20230807025206.3682381-1-liuwenyu7@huawei.com>
Message-ID: <53b8559db650886f013d5e91704ce8f4df77d815.camel@linux.ibm.com>

On Mon, 2023-08-07 at 10:52 +0800, Wenyu Liu wrote:
> kexec_mutex is replaced by an atomic variable
> in 05c6257433b (panic, kexec: make __crash_kexec() NMI safe).
> 
> But there are still two comments that referenced kexec_mutex,
> replace them by kexec_lock.
> 
> Signed-off-by: Wenyu Liu <liuwenyu7 at huawei.com>
> Acked-by: Baoquan He <bhe at redhat.com>
> Acked-by: Paul Menzel <pmenzel at molgen.mpg.de>

Thank you. It's now queued 
https://git.kernel.org/pub/scm/linux/kernel/git/zohar/linux-integrity.git/log/?h=next-integrity
.

--
thanks,

Mimi



From piliu at redhat.com  Mon Aug  7 08:22:08 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Mon, 7 Aug 2023 23:22:08 +0800
Subject: [PATCHv7 3/5] kexec/zboot: Add arch independent zboot support
In-Reply-To: <ZNDwM0YfAHT7PDef@vergenet.net>
References: <20230803024152.11663-1-piliu@redhat.com> <20230803024152.11663-4-piliu@redhat.com>
 <ZNDwM0YfAHT7PDef@vergenet.net>
Message-ID: <CAF+s44TK6O3psWco=+cQn4MFMLSCy2s7S=PAwfiYCkuPZEM-iA@mail.gmail.com>

On Mon, Aug 7, 2023 at 9:23?PM Simon Horman <horms at kernel.org> wrote:
>
> On Thu, Aug 03, 2023 at 10:41:50AM +0800, Pingfan Liu wrote:
> > From: Jeremy Linton <jeremy.linton at arm.com>
> >
> > The linux kernel CONFIG_ZBOOT option creates
> > self decompressing PE kernel images. So this means
> > that kexec should have a generic understanding of
> > the format which may be used by multiple arches.
> >
> > So lets add an arch independent validation
> > and decompression routine.
> >
> > Signed-off-by: Jeremy Linton <jeremy.linton at arm.com>
> > [Modified by Pingfan to export kernel fd]
> > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > To: kexec at lists.infradead.org
> > Cc: horms at verge.net.au
> > Cc: ardb at kernel.org
> > Cc: jeremy.linton at arm.com
> > ---
> >  include/Makefile         |   1 +
> >  include/kexec-pe-zboot.h |  15 +++++
> >  kexec/Makefile           |   1 +
> >  kexec/kexec-pe-zboot.c   | 131 +++++++++++++++++++++++++++++++++++++++
> >  4 files changed, 148 insertions(+)
> >  create mode 100644 include/kexec-pe-zboot.h
> >  create mode 100644 kexec/kexec-pe-zboot.c
> >
> > diff --git a/include/Makefile b/include/Makefile
> > index 621ce9f..cd88a26 100644
> > --- a/include/Makefile
> > +++ b/include/Makefile
> > @@ -1,6 +1,7 @@
> >  dist += include/Makefile             \
> >       include/config.h                \
> >       include/config.h.in             \
> > +     include/kexec-pe-zboot.h        \
> >       include/kexec-uImage.h          \
> >       include/x86/x86-linux.h         \
> >       include/x86/mb_info.h           \
> > diff --git a/include/kexec-pe-zboot.h b/include/kexec-pe-zboot.h
> > new file mode 100644
> > index 0000000..c588ca2
> > --- /dev/null
> > +++ b/include/kexec-pe-zboot.h
> > @@ -0,0 +1,15 @@
> > +#ifndef __KEXEC_PE_ZBOOT_H__
> > +#define __KEXEC_PE_ZBOOT_H__
> > +
> > +/* see drivers/firmware/efi/libstub/zboot-header.S */
> > +struct linux_pe_zboot_header {
> > +     uint32_t mz_magic;
> > +        uint32_t image_type;
> > +        uint32_t payload_offset;
> > +        uint32_t payload_size;
> > +        uint32_t reserved[2];
> > +        uint32_t compress_type;
> > +};
>
> nit: the above should consistently use tabs for indentation.
>      I can try and fix this when applying if there is no v8.
>

Appreciate it if you do it. I think that no v8 if no other guys have
opinions on this series.

Thanks,

Pingfan



From stefanb at linux.ibm.com  Mon Aug  7 15:49:25 2023
From: stefanb at linux.ibm.com (Stefan Berger)
Date: Mon, 7 Aug 2023 18:49:25 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
Message-ID: <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>



On 8/1/23 17:21, James Bottomley wrote:
> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
> [...]
>> Truncating IMA log to reclaim memory is not feasible, since it makes
>> the log go out of sync with the TPM PCR quote making remote
>> attestation fail.
> 
> This assumption isn't entirely true.  It's perfectly possible to shard
> an IMA log using two TPM2_Quote's for the beginning and end PCR values
> to validate the shard.  The IMA log could be truncated in the same way
> (replace the removed part of the log with a TPM2_Quote and AK, so the
> log still validates from the beginning quote to the end).
> 
> If you use a TPM2_Quote mechanism to save the log, all you need to do
> is have the kernel generate the quote with an internal AK.  You can
> keep a record of the quote and the AK at the beginning of the truncated
> kernel log.  If the truncated entries are saved in a file shard it

The truncation seems dangerous to me. Maybe not all the scenarios with an attestation
client (client = reading logs and quoting) are possible then anymore, such as starting
an attestation client only after truncation but a verifier must have witnessed the
system's PCRs and log state before the truncation occurred.

I think an ima-buf (or similar) log entry in IMA log would have to appear at the beginning of the
truncated log stating the value of all PCRs that IMA touched (typically only PCR 10
but it can be others). The needs to be done since the quote itself doesn't
provide the state of the individual PCRs. This would at least allow an attestation
client to re-read the log from the beginning (when it is re-start or started for the
first time after the truncation). However, this alone (without the
internal AK quoting the old state) could lead to abuse where I could create totally
fake IMA logs stating the state of the PCRs at the beginning (so the verifier
syncs its internal PCR state to this state). Further, even with the AK-quote that
you propose I may be able to create fake logs and trick a verifier into
trusting the machine IFF it doesn't know what kernel this system was booted with
that I may have hacked to provide a fake AK-quote that just happens to match the
PCR state presented at the beginning of the log.

=> Can a truncated log be made safe for attestation when the attestation starts
only after the truncation occurred?

=> Even if attestation was occurring 'what' state does an attestation server
need to carry around for an attested-to system so that the truncation is 'safe'
and I cannot create fake AK-quotes and fake IMA logs with initial PCR states?
Can I ever restart the client and have it read the truncated log from the
beginning and what type of verification needs to happen on the server then?
It seems like the server would have to remember the state of the IMA PCRs upon
last truncation to detect a possible attack. This would make staring to monitor
a system after truncation impossible -- would be good to know these details.




> should have a beginning and end quote and a record of the AK used.
> Since verifiers like Keylime are already using this beginning and end
> quote for sharded logs, it's the most natural format to feed to
> something externally for verification and it means you don't have to
> invent a new format to do the same thing.
> 
> Regards,
> 
> James
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From James.Bottomley at HansenPartnership.com  Tue Aug  8 05:35:35 2023
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Tue, 08 Aug 2023 08:35:35 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
	 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
Message-ID: <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>

On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
> 
> 
> On 8/1/23 17:21, James Bottomley wrote:
> > On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
> > [...]
> > > Truncating IMA log to reclaim memory is not feasible, since it
> > > makes the log go out of sync with the TPM PCR quote making remote
> > > attestation fail.
> > 
> > This assumption isn't entirely true.? It's perfectly possible to
> > shard an IMA log using two TPM2_Quote's for the beginning and end
> > PCR values to validate the shard.? The IMA log could be truncated
> > in the same way (replace the removed part of the log with a
> > TPM2_Quote and AK, so the log still validates from the beginning
> > quote to the end).
> > 
> > If you use a TPM2_Quote mechanism to save the log, all you need to
> > do is have the kernel generate the quote with an internal AK.? You
> > can keep a record of the quote and the AK at the beginning of the
> > truncated kernel log.? If the truncated entries are saved in a file
> > shard it
> 
> The truncation seems dangerous to me. Maybe not all the scenarios
> with an attestation client (client = reading logs and quoting) are
> possible then anymore, such as starting an attestation client only
> after truncation but a verifier must have witnessed the system's PCRs
> and log state before the truncation occurred.

That's not exactly correct.  Nothing needs to have "witnessed" the
starting PCR value because the quote vouches for it (and can vouch for
it after the fact).  The only thing you need to verify the quote is the
attestation key and the only thing you need to do to trust the
attestation key is ensure it was TPM created.  All of that can be
verified after the fact as well.  The only thing that can be done to
disrupt this is to destroy the TPM (or re-own it).

Remember the assumption is you *also* have the removed log shard to
present.  From that the PCR state of the starting quote can be
calculated and checked for matching the quote.  If you lose that, it's
equivalent to the log being tampered with and all bets are off.  The
assumption is that because of the impossibility of engineering TPM
extensions, it should be impossible to come up with a fake log that
produces the PCRs of the real one.  If that's violated, then IMA itself
becomes useless.

James



From stefanb at linux.ibm.com  Tue Aug  8 06:31:01 2023
From: stefanb at linux.ibm.com (Stefan Berger)
Date: Tue, 8 Aug 2023 09:31:01 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
Message-ID: <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>



On 8/8/23 08:35, James Bottomley wrote:
> On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
>>
>>
>> On 8/1/23 17:21, James Bottomley wrote:
>>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>>> [...]
>>>> Truncating IMA log to reclaim memory is not feasible, since it
>>>> makes the log go out of sync with the TPM PCR quote making remote
>>>> attestation fail.
>>>
>>> This assumption isn't entirely true.? It's perfectly possible to
>>> shard an IMA log using two TPM2_Quote's for the beginning and end
>>> PCR values to validate the shard.? The IMA log could be truncated
>>> in the same way (replace the removed part of the log with a
>>> TPM2_Quote and AK, so the log still validates from the beginning
>>> quote to the end).
>>>
>>> If you use a TPM2_Quote mechanism to save the log, all you need to
>>> do is have the kernel generate the quote with an internal AK.? You
>>> can keep a record of the quote and the AK at the beginning of the
>>> truncated kernel log.? If the truncated entries are saved in a file
>>> shard it
>>
>> The truncation seems dangerous to me. Maybe not all the scenarios
>> with an attestation client (client = reading logs and quoting) are
>> possible then anymore, such as starting an attestation client only
>> after truncation but a verifier must have witnessed the system's PCRs
>> and log state before the truncation occurred.
> 
> That's not exactly correct.  Nothing needs to have "witnessed" the
> starting PCR value because the quote vouches for it (and can vouch for
> it after the fact).  The only thing you need to verify the quote is the
> attestation key and the only thing you need to do to trust the> attestation key is ensure it was TPM created.  All of that can be
> verified after the fact as well.  The only thing that can be done to
> disrupt this is to destroy the TPM (or re-own it).> 
> Remember the assumption is you *also* have the removed log shard to
> present.  From that the PCR state of the starting quote can be

Yes, the whole sequence of old logs needs to be available. IF that's the
case and the logs can be stitched together seamlessly, who then looks at the
kernel AK quote and under what circumstances?

> calculated and checked for matching the quote.  If you lose that, it's
> equivalent to the log being tampered with and all bets are off.  The
> assumption is that because of the impossibility of engineering TPM
> extensions, it should be impossible to come up with a fake log that
> produces the PCRs of the real one.  If that's violated, then IMA itself
> becomes useless.
> 
> James
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From James.Bottomley at HansenPartnership.com  Tue Aug  8 11:26:51 2023
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Tue, 08 Aug 2023 14:26:51 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
	 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
	 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
	 <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
Message-ID: <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>

On Tue, 2023-08-08 at 09:31 -0400, Stefan Berger wrote:
> 
> 
> On 8/8/23 08:35, James Bottomley wrote:
> > On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
> > > 
> > > 
> > > On 8/1/23 17:21, James Bottomley wrote:
> > > > On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
> > > > [...]
> > > > > Truncating IMA log to reclaim memory is not feasible, since
> > > > > it makes the log go out of sync with the TPM PCR quote making
> > > > > remote attestation fail.
> > > > 
> > > > This assumption isn't entirely true.? It's perfectly possible
> > > > to shard an IMA log using two TPM2_Quote's for the beginning
> > > > and end PCR values to validate the shard.? The IMA log could be
> > > > truncated in the same way (replace the removed part of the log
> > > > with a TPM2_Quote and AK, so the log still validates from the
> > > > beginning quote to the end).
> > > > 
> > > > If you use a TPM2_Quote mechanism to save the log, all you need
> > > > to do is have the kernel generate the quote with an internal
> > > > AK.? You can keep a record of the quote and the AK at the
> > > > beginning of the truncated kernel log.? If the truncated
> > > > entries are saved in a file shard it
> > > 
> > > The truncation seems dangerous to me. Maybe not all the scenarios
> > > with an attestation client (client = reading logs and quoting)
> > > are possible then anymore, such as starting an attestation client
> > > only after truncation but a verifier must have witnessed the
> > > system's PCRs and log state before the truncation occurred.
> > 
> > That's not exactly correct.? Nothing needs to have "witnessed" the
> > starting PCR value because the quote vouches for it (and can vouch
> > for it after the fact).? The only thing you need to verify the
> > quote is the attestation key and the only thing you need to do to
> > trust the attestation key is ensure it was TPM created.? All of
> > that can be verified after the fact as well.? The only thing that
> > can be done to disrupt this is to destroy the TPM (or re-own it).> 
> > Remember the assumption is you *also* have the removed log shard to
> > present.? From that the PCR state of the starting quote can be
> 
> Yes, the whole sequence of old logs needs to be available.

Yes and no.  If the person relying on the logs is happy they've
extracted all the evidentiary value from the log itself then they can
reduce the preceding log shard to simply the PCR values that match the
quote and discard the rest.

>  IF that's the case and the logs can be stitched together seamlessly,
> who then looks at the kernel AK quote and under what circumstances?

For incremental attestation.  Each log shard can be verified using the
base PCR values corresponding to the bottom quote then replayed and the
top quote verified.  This means that logs that aren't needed anymore
can be discarded, which, I recall, was the base reason for this
proposal: reducing IMA memory consumption.  Although all you need to do
is extract the shards from kernel memory to file space and free the
kernel memory.  Since that scheme can keep all logs intact, there's no
reason to further reduce them unless the filesystem is running out of
space.

James



From stefanb at linux.ibm.com  Tue Aug  8 13:09:45 2023
From: stefanb at linux.ibm.com (Stefan Berger)
Date: Tue, 8 Aug 2023 16:09:45 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
 <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
 <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>
Message-ID: <04fb2fe5-9ebe-b35f-bdde-6ef22786438f@linux.ibm.com>



On 8/8/23 14:26, James Bottomley wrote:
> On Tue, 2023-08-08 at 09:31 -0400, Stefan Berger wrote:
>>
>>
>> On 8/8/23 08:35, James Bottomley wrote:
>>> On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
>>>>
>>>>
>>>> On 8/1/23 17:21, James Bottomley wrote:
>>>>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>>>>> [...]
>>>>>> Truncating IMA log to reclaim memory is not feasible, since
>>>>>> it makes the log go out of sync with the TPM PCR quote making
>>>>>> remote attestation fail.
>>>>>
>>>>> This assumption isn't entirely true.? It's perfectly possible
>>>>> to shard an IMA log using two TPM2_Quote's for the beginning
>>>>> and end PCR values to validate the shard.? The IMA log could be
>>>>> truncated in the same way (replace the removed part of the log
>>>>> with a TPM2_Quote and AK, so the log still validates from the
>>>>> beginning quote to the end).
>>>>>
>>>>> If you use a TPM2_Quote mechanism to save the log, all you need
>>>>> to do is have the kernel generate the quote with an internal
>>>>> AK.? You can keep a record of the quote and the AK at the
>>>>> beginning of the truncated kernel log.? If the truncated
>>>>> entries are saved in a file shard it
>>>>
>>>> The truncation seems dangerous to me. Maybe not all the scenarios
>>>> with an attestation client (client = reading logs and quoting)
>>>> are possible then anymore, such as starting an attestation client
>>>> only after truncation but a verifier must have witnessed the
>>>> system's PCRs and log state before the truncation occurred.
>>>
>>> That's not exactly correct.? Nothing needs to have "witnessed" the
>>> starting PCR value because the quote vouches for it (and can vouch
>>> for it after the fact).? The only thing you need to verify the
>>> quote is the attestation key and the only thing you need to do to
>>> trust the attestation key is ensure it was TPM created.? All of
>>> that can be verified after the fact as well.? The only thing that
>>> can be done to disrupt this is to destroy the TPM (or re-own it).>
>>> Remember the assumption is you *also* have the removed log shard to
>>> present.? From that the PCR state of the starting quote can be
>>
>> Yes, the whole sequence of old logs needs to be available.
> 
> Yes and no.  If the person relying on the logs is happy they've
> extracted all the evidentiary value from the log itself then they can
> reduce the preceding log shard to simply the PCR values that match the
> quote and discard the rest.
> 
>>   IF that's the case and the logs can be stitched together seamlessly,
>> who then looks at the kernel AK quote and under what circumstances?
> 
> For incremental attestation.  Each log shard can be verified using the
> base PCR values corresponding to the bottom quote then replayed and the


Somehow you have to tell a verifier to take a snapshot of the current state
of the PCRs when it replays the logs to be able to truncate the log. Whether
the state of the PCRs is in the log itself or it's just some sort of entry in
the log indicating a truncation probably doesn't matter for as long as the
verifying side keeps state of the PCRs at point of truncatiokn.

Also, the verifying side needs to take notice of the trustworthiness of the
system at the time the log was truncated in case the attestation client is
restarted and starts out sending the log with the first entry. The PCR state
shown at the beginning of the truncated log (when restarting the attestation
client) must then match when the 'notice' was taken and that determines its
trustworthiness at this point in the log.

That there's a kernel AK signature at this point doesn't seem necessary since one
presumably can verify the log and PCR states at the end with the 'regular' quote.
Nobody should ever trust a system by starting to look at the beginning of a
truncated log. You have to have evaluated all the entries in the log before and
determined whether the system was trustworthy. I don't think the kernel AK
quote buys much - at least not from what I can see.


> top quote verified.  This means that logs that aren't needed anymore
> can be discarded, which, I recall, was the base reason for this
> proposal: reducing IMA memory consumption.  Although all you need to do
> is extract the shards from kernel memory to file space and free the
> kernel memory.  Since that scheme can keep all logs intact, there's no
> reason to further reduce them unless the filesystem is running out of
> space.
> 
> James
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From James.Bottomley at HansenPartnership.com  Tue Aug  8 14:41:28 2023
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Tue, 08 Aug 2023 17:41:28 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <04fb2fe5-9ebe-b35f-bdde-6ef22786438f@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
	 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
	 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
	 <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
	 <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>
	 <04fb2fe5-9ebe-b35f-bdde-6ef22786438f@linux.ibm.com>
Message-ID: <a522b17a536ea87a6a4c2faf95583ae3b7b74a26.camel@HansenPartnership.com>

On Tue, 2023-08-08 at 16:09 -0400, Stefan Berger wrote:
> 
> 
> On 8/8/23 14:26, James Bottomley wrote:
> > On Tue, 2023-08-08 at 09:31 -0400, Stefan Berger wrote:
> > > 
> > > 
> > > On 8/8/23 08:35, James Bottomley wrote:
> > > > On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
> > > > > 
> > > > > 
> > > > > On 8/1/23 17:21, James Bottomley wrote:
> > > > > > On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale
> > > > > > wrote:
> > > > > > [...]
> > > > > > > Truncating IMA log to reclaim memory is not feasible,
> > > > > > > since
> > > > > > > it makes the log go out of sync with the TPM PCR quote
> > > > > > > making
> > > > > > > remote attestation fail.
> > > > > > 
> > > > > > This assumption isn't entirely true.? It's perfectly
> > > > > > possible
> > > > > > to shard an IMA log using two TPM2_Quote's for the
> > > > > > beginning
> > > > > > and end PCR values to validate the shard.? The IMA log
> > > > > > could be
> > > > > > truncated in the same way (replace the removed part of the
> > > > > > log
> > > > > > with a TPM2_Quote and AK, so the log still validates from
> > > > > > the
> > > > > > beginning quote to the end).
> > > > > > 
> > > > > > If you use a TPM2_Quote mechanism to save the log, all you
> > > > > > need
> > > > > > to do is have the kernel generate the quote with an
> > > > > > internal
> > > > > > AK.? You can keep a record of the quote and the AK at the
> > > > > > beginning of the truncated kernel log.? If the truncated
> > > > > > entries are saved in a file shard it
> > > > > 
> > > > > The truncation seems dangerous to me. Maybe not all the
> > > > > scenarios
> > > > > with an attestation client (client = reading logs and
> > > > > quoting)
> > > > > are possible then anymore, such as starting an attestation
> > > > > client
> > > > > only after truncation but a verifier must have witnessed the
> > > > > system's PCRs and log state before the truncation occurred.
> > > > 
> > > > That's not exactly correct.? Nothing needs to have "witnessed"
> > > > the
> > > > starting PCR value because the quote vouches for it (and can
> > > > vouch
> > > > for it after the fact).? The only thing you need to verify the
> > > > quote is the attestation key and the only thing you need to do
> > > > to
> > > > trust the attestation key is ensure it was TPM created.? All of
> > > > that can be verified after the fact as well.? The only thing
> > > > that
> > > > can be done to disrupt this is to destroy the TPM (or re-own
> > > > it).>
> > > > Remember the assumption is you *also* have the removed log
> > > > shard to
> > > > present.? From that the PCR state of the starting quote can be
> > > 
> > > Yes, the whole sequence of old logs needs to be available.
> > 
> > Yes and no.? If the person relying on the logs is happy they've
> > extracted all the evidentiary value from the log itself then they
> > can
> > reduce the preceding log shard to simply the PCR values that match
> > the
> > quote and discard the rest.
> > 
> > > ? IF that's the case and the logs can be stitched together
> > > seamlessly, who then looks at the kernel AK quote and under what
> > > circumstances?
> > 
> > For incremental attestation.? Each log shard can be verified using
> > the base PCR values corresponding to the bottom quote then replayed
> > and the
> 
> 
> Somehow you have to tell a verifier to take a snapshot of the current
> state of the PCRs when it replays the logs to be able to truncate the
> log.

No, the verifier is server side.  It would be the agent or the kernel,
client side, which gets the quote and shards the log.  That way the
operation can be done in such a way as to make sure the quote and the
shard point match.  I'd imagine the verifier can provide some sort of
guide as to how big it wants the shards to be and the client complies.

> Whether the state of the PCRs is in the log itself or it's just some
> sort of entry in the log indicating a truncation probably doesn't
> matter for as long as the verifying side keeps state of the PCRs at
> point of truncatiokn.

The idea would be the log shard would be self attesting, that's why
quote at beginning and end (and probably PCR state at beginning), so no
verifier required until someone wants to check the log.

> Also, the verifying side needs to take notice of the trustworthiness
> of the system at the time the log was truncated in case the
> attestation client is restarted and starts out sending the log with
> the first entry.

That's usually what a runtime verification system is actually
verifying, yes.

>  The PCR state shown at the beginning of the truncated log (when
> restarting the attestation client) must then match when the 'notice'
> was taken and that determines its trustworthiness at this point in
> the log.
> 
> That there's a kernel AK signature

A quote is a signature over PCR state signed by an AK, yes.

>  at this point doesn't seem necessary since one presumably can verify
> the log and PCR states at the end with the 'regular' quote.

I don't understand this.  A regular quote is a signature over PCR state
by an AK.  The point about saving the AK in the log for the original is
that if the *kernel* truncates the log and saves it to a file, it needs
to generate both the AK and the quote for the top of the file shard. 
That means the AK/EK binding is unverified, but can be verified by
loading the AK and running the usual tests, which can only be done if
you have the loadable AK, which is why you need it as part of the log
saving proposal.

> Nobody should ever trust a system by starting to look at the
> beginning of a truncated log. You have to have evaluated all the
> entries in the log before and determined whether the system was
> trustworthy. I don't think the kernel AK quote buys much - at least
> not from what I can see.

You have a log shard with PCR state at the beginning and calculated at
the end, both of which you can verify.  How you establish trust in the
starting PCR values of the shard is a policy decision, but the policy
doesn't have to be you see every shard up to boot.  You could take the
word of the system owner on a handoff of responsibility for instance.

James



From bhe at redhat.com  Tue Aug  8 19:11:46 2023
From: bhe at redhat.com (bhe at redhat.com)
Date: Wed, 9 Aug 2023 10:11:46 +0800
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64
 platform
In-Reply-To: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
Message-ID: <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>

On 08/08/23 at 07:17pm, Pandey, Radhey Shyam wrote:
> Hi,
> 
> I am trying to bring up kdump on arm64 platform[1]. But I get "Cannot get kernel _text symbol address".
> 
> Is there some Dump-capture kernel config options that I am missing?
> 
> FYI, copied below complete kexec debug log.
> 
> [1]: https://www.xilinx.com/products/boards-and-kits/vck190.html

Your description isn't clear. You saw the printing, then your kdump
kernel loading succeeded or not?

If no, have you tried applying Pingfan's patchset and still saw the issue?

[PATCHv7 0/5] arm64: zboot support
https://lore.kernel.org/all/20230803024152.11663-1-piliu at redhat.com/T/#u

Thanks
Baoquan



From palmer at rivosinc.com  Wed Aug  9 07:35:42 2023
From: palmer at rivosinc.com (Palmer Dabbelt)
Date: Wed, 09 Aug 2023 07:35:42 -0700
Subject: (subset) [PATCH v2 0/3] RISC-V: Fix a few kexec_file_load(2)
 failures
In-Reply-To: <cover.1690365011.git.petr.tesarik.ext@huawei.com>
References: <cover.1690365011.git.petr.tesarik.ext@huawei.com>
Message-ID: <169159174235.20984.13290949121654917312.b4-ty@rivosinc.com>


On Wed, 26 Jul 2023 11:53:58 +0200, Petr Tesarik wrote:
> From: Petr Tesarik <petr.tesarik.ext at huawei.com>
> 
> The kexec_file_load(2) syscall does not work at least in some kernel
> builds. For details see the relevant section in this blog post:
> 
> https://sigillatum.tesarici.cz/2023-07-21-state-of-riscv64-kdump.html
> 
> [...]

Applied, thanks!

[1/3] riscv/kexec: handle R_RISCV_CALL_PLT relocation type
      https://git.kernel.org/palmer/c/d0b4f95a5103
[3/3] riscv/kexec: load initrd high in available memory
      https://git.kernel.org/palmer/c/49af7a2cd5f6

Best regards,
-- 
Palmer Dabbelt <palmer at rivosinc.com>



From ming.lei at redhat.com  Wed Aug  9 17:09:27 2023
From: ming.lei at redhat.com (Ming Lei)
Date: Thu, 10 Aug 2023 08:09:27 +0800
Subject: [PATCH V3 01/14] blk-mq: add blk_mq_max_nr_hw_queues()
In-Reply-To: <20230809134401.GA31852@lst.de>
References: <20230808104239.146085-1-ming.lei@redhat.com>
 <20230808104239.146085-2-ming.lei@redhat.com>
 <20230809134401.GA31852@lst.de>
Message-ID: <ZNQqt1C0pXspGl3d@fedora>

On Wed, Aug 09, 2023 at 03:44:01PM +0200, Christoph Hellwig wrote:
> I'm starting to sound like a broken record, but we can't just do random
> is_kdump checks, and it's not going to get better by resending it again and
> again.  If kdump kernels limit the number of possible CPUs, it needs to
> reflected in cpu_possible_map and we need to use that information.
> 

Can you look at previous kdump/arch guys' comment about kdump usage &
num_possible_cpus?

    https://lore.kernel.org/linux-block/CAF+s44RuqswbosY9kMDx35crviQnxOeuvgNsuE75Bb0Y2Jg2uw at mail.gmail.com/
    https://lore.kernel.org/linux-block/ZKz912KyFQ7q9qwL at MiWiFi-R3L-srv/

The point is that kdump kernels does not limit the number of possible CPUs.

1) some archs support 'nr_cpus=1' for kdump kernel, which is fine, since
num_possible_cpus becomes 1.

2) some archs do not support 'nr_cpus=1', and have to rely on
'max_cpus=1', so num_possible_cpus isn't changed, and kernel just boots
with single online cpu. That causes trouble because blk-mq limits single
queue.

Documentation/admin-guide/kdump/kdump.rst

Thanks, 
Ming



From tusharsu at linux.microsoft.com  Wed Aug  9 18:03:20 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Wed, 9 Aug 2023 18:03:20 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
Message-ID: <e9b67fd8-3a8f-7024-5940-a093a704358e@linux.microsoft.com>

Thanks a lot James for looking at this proposal,
and sharing your thoughts. Really appreciate it.

On 8/1/23 14:21, James Bottomley wrote:
> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
> [...]
>> Truncating IMA log to reclaim memory is not feasible, since it makes
>> the log go out of sync with the TPM PCR quote making remote
>> attestation fail.
> This assumption isn't entirely true.  It's perfectly possible to shard
> an IMA log using two TPM2_Quote's for the beginning and end PCR values
> to validate the shard.  The IMA log could be truncated in the same way
> (replace the removed part of the log with a TPM2_Quote and AK, so the
> log still validates from the beginning quote to the end).
Here we meant just truncating IMA log is not a complete
solution in itself. As you said, we have to take additional steps
like logging TPM2_Quotes etc. Logging AK is an interesting proposal
which we didn?t consider earlier. I am not sure if embedding AK to IMA
log/snapshot is needed. If the client sends them separately with "signed
PCR quotes" + "IMA log" + snapshots, it should still serve the purpose,
right?

>
> If you use a TPM2_Quote mechanism to save the log, all you need to do
> is have the kernel generate the quote with an internal AK.  You can
> keep a record of the quote and the AK at the beginning of the truncated
> kernel log.  If the truncated entries are saved in a file shard it
> should have a beginning and end quote and a record of the AK used.
A new IMA log snapshot file (or shard as you call it) will have
the TPM2_Quote record (plus some additional metadata) at the beginning.
I don't believe it needs to be logged at the end of the snapshot (since 
it can
be computed by replaying the remaining entries in the snapshot).

See the snapshot_aggregate field in section B.5 in the original RFC mail 
[1].
> Since verifiers like Keylime are already using this beginning and end
> quote for sharded logs, it's the most natural format to feed to
> something externally for verification and it means you don't have to
> invent a new format to do the same thing.
Could you please point to the Keylime source and/or documentation
which explains the use of beginning and end quotes? We would like to
understand how the verifiers are addressing this problem currently.


[1] 
https://lore.kernel.org/all/c5737141-7827-1c83-ab38-0119dcfea485 at linux.microsoft.com/#t

~Tushar
>
> Regards,
>
> James



From tusharsu at linux.microsoft.com  Wed Aug  9 18:15:47 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Wed, 9 Aug 2023 18:15:47 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
Message-ID: <b538f7d2-5a04-46d0-3792-a18653230a95@linux.microsoft.com>

Thanks a lot Stefan for looking into this proposal,
and providing your feedback. We really appreciate it.

On 8/7/23 15:49, Stefan Berger wrote:
>
>
> On 8/1/23 17:21, James Bottomley wrote:
>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>> [...]
>>> Truncating IMA log to reclaim memory is not feasible, since it makes
>>> the log go out of sync with the TPM PCR quote making remote
>>> attestation fail.
>>
>> This assumption isn't entirely true.? It's perfectly possible to shard
>> an IMA log using two TPM2_Quote's for the beginning and end PCR values
>> to validate the shard.? The IMA log could be truncated in the same way
>> (replace the removed part of the log with a TPM2_Quote and AK, so the
>> log still validates from the beginning quote to the end).
>>
>> If you use a TPM2_Quote mechanism to save the log, all you need to do
>> is have the kernel generate the quote with an internal AK.? You can
>> keep a record of the quote and the AK at the beginning of the truncated
>> kernel log.? If the truncated entries are saved in a file shard it
>
> The truncation seems dangerous to me. Maybe not all the scenarios with 
> an attestation
> client (client = reading logs and quoting) are possible then anymore, 
> such as starting
> an attestation client only after truncation but a verifier must have 
> witnessed the
> system's PCRs and log state before the truncation occurred.
You are correct that truncation on it?s own is dangerous. It needs to be
accompanied by (a) saving the IMA log data to disk as snapshots, (b) 
adding the
necessary TPM PCR quotes to the current IMA log (as James mentioned above),
(c) attestation clients having an ability to send the past snapshots to the
remote-attestation-service (verifiers), (d) and verifiers having an ability
to use the snapshots along with current IMA logs for the purpose of 
attestation.
All these points are explained in the original RFC email in sections B.1 
through B.5 [1].
>
> I think an ima-buf (or similar) log entry in IMA log would have to 
> appear at the beginning of the
> truncated log stating the value of all PCRs that IMA touched 
> (typically only PCR 10
> but it can be others). The needs to be done since the quote itself 
> doesn't
> provide the state of the individual PCRs. This would at least allow an 
> attestation
> client to re-read the log from the beginning (when it is re-start or 
> started for the
> first time after the truncation). 
 ?Agreed. See the description of snapshot_aggregate in Section B.5 in the
original RFC email [1].
> However, this alone (without the
> internal AK quoting the old state) could lead to abuse where I could 
> create totally
> fake IMA logs stating the state of the PCRs at the beginning (so the 
> verifier
> syncs its internal PCR state to this state). 
Yes, the PCR quotes sent to the verifier must be signed by the AK that
is trusted by the verifier. That assumption is true regardless of IMA log
snapshotting feature.
> Further, even with the AK-quote that
> you propose I may be able to create fake logs and trick a verifier into
> trusting the machine IFF it doesn't know what kernel this system was 
> booted with
> that I may have hacked to provide a fake AK-quote that just happens to 
> match the
> PCR state presented at the beginning of the log.
>
If the Kernel is compromised, then all-bets are off.
(Regardless of IMA log snapshotting feature.)
> => Can a truncated log be made safe for attestation when the 
> attestation starts
> only after the truncation occurred?
>
Yes. If the ?PCR quotes in the snapshot_aggregate event in IMA log?
+ "replay of rest of the events in IMA log" results in the ?final PCR 
quotes?
that matches with the ?AK signed PCR quotes? sent by the client, then 
the truncated
IMA log can be trusted. The verifier can either ?trust? the ?PCR quotes 
in the
snapshot_aggregate event in IMA log? or it can ask for the (n-1)th 
snapshot shard
to check the past events.

> => Even if attestation was occurring 'what' state does an attestation 
> server
> need to carry around for an attested-to system so that the truncation 
> is 'safe'
> and I cannot create fake AK-quotes and fake IMA logs with initial PCR 
> states?
Assuming most of the client devices take a snapshot at specific checkpoints,
the ?PCR quotes in the snapshot_aggregate event in IMA log? will be the 
same for them.
The remote attestation server will have to remember these golden PCR quotes.
It doesn't have to remember the state of each client device.

> Can I ever restart the client and have it read the truncated log from the
> beginning and what type of verification needs to happen on the server 
> then?
>
Yes, restarting the client should be possible.
> It seems like the server would have to remember the state of the IMA 
> PCRs upon
> last truncation to detect a possible attack. This would make staring 
> to monitor
> a system after truncation impossible -- would be good to know these 
> details.
>
The server is not forced to remember the state of IMA PCRs. It can
always ask for the last n snapshot files (shards) and replay the events. 
Even
though the data is truncated from the IMA log, it is not totally lost. It is
simply being transferred to the disk. It is saved by UM as snapshot 
files/shards.
The goal of IMA snapshotting is to reduce the Kernel memory pressure on the
client devices - to save them from out-of-memory errors which are harder 
to manage
on long running clients. It comes with a cost of additional work on the 
server
side to attest those clients.


Being said that, in the current proposal, taking a snapshots is totally 
optional
and controlled by UM attestation clients. If the 
attestation-clients/services are
not-ready/don?t-want to take advantage of IMA log snapshotting, they 
don?t have to.

No snapshot will be taken, and the client-service can process the 
monolithic IMA
log just like they do today.

[1] 
https://lore.kernel.org/all/c5737141-7827-1c83-ab38-0119dcfea485 at linux.microsoft.com/#t 


>
>
>
>> should have a beginning and end quote and a record of the AK used.
>> Since verifiers like Keylime are already using this beginning and end
>> quote for sharded logs, it's the most natural format to feed to
>> something externally for verification and it means you don't have to
>> invent a new format to do the same thing.
>>
>> Regards,
>>
>> James
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec



From bhe at redhat.com  Wed Aug  9 18:18:27 2023
From: bhe at redhat.com (Baoquan He)
Date: Thu, 10 Aug 2023 09:18:27 +0800
Subject: [PATCH V3 01/14] blk-mq: add blk_mq_max_nr_hw_queues()
In-Reply-To: <ZNQqt1C0pXspGl3d@fedora>
References: <20230808104239.146085-1-ming.lei@redhat.com>
 <20230808104239.146085-2-ming.lei@redhat.com>
 <20230809134401.GA31852@lst.de>
 <ZNQqt1C0pXspGl3d@fedora>
Message-ID: <ZNQ64xhCIBU6XM/5@MiWiFi-R3L-srv>

On 08/10/23 at 08:09am, Ming Lei wrote:
> On Wed, Aug 09, 2023 at 03:44:01PM +0200, Christoph Hellwig wrote:
> > I'm starting to sound like a broken record, but we can't just do random
> > is_kdump checks, and it's not going to get better by resending it again and
> > again.  If kdump kernels limit the number of possible CPUs, it needs to
> > reflected in cpu_possible_map and we need to use that information.
> > 
> 
> Can you look at previous kdump/arch guys' comment about kdump usage &
> num_possible_cpus?
> 
>     https://lore.kernel.org/linux-block/CAF+s44RuqswbosY9kMDx35crviQnxOeuvgNsuE75Bb0Y2Jg2uw at mail.gmail.com/
>     https://lore.kernel.org/linux-block/ZKz912KyFQ7q9qwL at MiWiFi-R3L-srv/
> 
> The point is that kdump kernels does not limit the number of possible CPUs.
> 
> 1) some archs support 'nr_cpus=1' for kdump kernel, which is fine, since
> num_possible_cpus becomes 1.

Yes, "nr_cpus=" is strongly suggested in kdump kernel because "nr_cpus="
limits the possible cpu numbers, while "maxcpuss=" only limits the cpu
number which can be brought up during bootup. We noticed this diference
because a large number of possible cpus will cost more memory in kdump
kernel. e.g percpu initialization, even though kdump kernel have set
"maxcpus=1". 

Currently x86 and arm64 all support "nr_cpus=". Pingfan ever spent much
effort to make patches to add "nr_cpus=" support to ppc64, seems ppc64
dev and maintainers do not care about it. Finally the patches are not
accepted, and the work is not continued.

Now, I am wondering what is the barrier to add "nr_cpus=" to power ach.
Can we reconsider adding 'nr_cpus=' to power arch since real issue
occurred in kdump kernel?

As for this patchset, it can be accpeted so that no failure in kdump
kernel is seen on ARCHes w/o "nr_cpus=" support? My personal opinion.

> 
> 2) some archs do not support 'nr_cpus=1', and have to rely on
> 'max_cpus=1', so num_possible_cpus isn't changed, and kernel just boots
> with single online cpu. That causes trouble because blk-mq limits single
> queue.
> 
> Documentation/admin-guide/kdump/kdump.rst
> 
> Thanks, 
> Ming
> 



From tusharsu at linux.microsoft.com  Wed Aug  9 18:23:01 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Wed, 9 Aug 2023 18:23:01 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
Message-ID: <72f195f3-d96e-c516-d9e6-983ad7f86cdc@linux.microsoft.com>

hi James,
Thanks for addressing Stefan's concerns.

On 8/8/23 05:35, James Bottomley wrote:
> On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
>>
>> On 8/1/23 17:21, James Bottomley wrote:
>>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>>> [...]
>>>> Truncating IMA log to reclaim memory is not feasible, since it
>>>> makes the log go out of sync with the TPM PCR quote making remote
>>>> attestation fail.
>>> This assumption isn't entirely true.? It's perfectly possible to
>>> shard an IMA log using two TPM2_Quote's for the beginning and end
>>> PCR values to validate the shard.? The IMA log could be truncated
>>> in the same way (replace the removed part of the log with a
>>> TPM2_Quote and AK, so the log still validates from the beginning
>>> quote to the end).
>>>
>>> If you use a TPM2_Quote mechanism to save the log, all you need to
>>> do is have the kernel generate the quote with an internal AK.? You
>>> can keep a record of the quote and the AK at the beginning of the
>>> truncated kernel log.? If the truncated entries are saved in a file
>>> shard it
>> The truncation seems dangerous to me. Maybe not all the scenarios
>> with an attestation client (client = reading logs and quoting) are
>> possible then anymore, such as starting an attestation client only
>> after truncation but a verifier must have witnessed the system's PCRs
>> and log state before the truncation occurred.
> That's not exactly correct.  Nothing needs to have "witnessed" the
> starting PCR value because the quote vouches for it (and can vouch for
> it after the fact).  The only thing you need to verify the quote is the
> attestation key and the only thing you need to do to trust the
> attestation key is ensure it was TPM created.  All of that can be
> verified after the fact as well.  The only thing that can be done to
> disrupt this is to destroy the TPM (or re-own it).
Agreed.
>
> Remember the assumption is you *also* have the removed log shard to
> present.  From that the PCR state of the starting quote can be
> calculated and checked for matching the quote.  If you lose that, it's
> equivalent to the log being tampered with and all bets are off.  The
> assumption is that because of the impossibility of engineering TPM
> extensions, it should be impossible to come up with a fake log that
> produces the PCRs of the real one.  If that's violated, then IMA itself
> becomes useless.
Totally agreed. I believe our proposal is consistent with your
comments. Please let us know if you see any inconsistencies or technical
blockers.

~Tushar

>
> James



From ming.lei at redhat.com  Wed Aug  9 19:06:46 2023
From: ming.lei at redhat.com (Ming Lei)
Date: Thu, 10 Aug 2023 10:06:46 +0800
Subject: [PATCH V3 01/14] blk-mq: add blk_mq_max_nr_hw_queues()
In-Reply-To: <ZNQ64xhCIBU6XM/5@MiWiFi-R3L-srv>
References: <20230808104239.146085-1-ming.lei@redhat.com>
 <20230808104239.146085-2-ming.lei@redhat.com>
 <20230809134401.GA31852@lst.de>
 <ZNQqt1C0pXspGl3d@fedora>
 <ZNQ64xhCIBU6XM/5@MiWiFi-R3L-srv>
Message-ID: <ZNRGNsRzEJfzUEzH@fedora>

On Thu, Aug 10, 2023 at 09:18:27AM +0800, Baoquan He wrote:
> On 08/10/23 at 08:09am, Ming Lei wrote:
> > On Wed, Aug 09, 2023 at 03:44:01PM +0200, Christoph Hellwig wrote:
> > > I'm starting to sound like a broken record, but we can't just do random
> > > is_kdump checks, and it's not going to get better by resending it again and
> > > again.  If kdump kernels limit the number of possible CPUs, it needs to
> > > reflected in cpu_possible_map and we need to use that information.
> > > 
> > 
> > Can you look at previous kdump/arch guys' comment about kdump usage &
> > num_possible_cpus?
> > 
> >     https://lore.kernel.org/linux-block/CAF+s44RuqswbosY9kMDx35crviQnxOeuvgNsuE75Bb0Y2Jg2uw at mail.gmail.com/
> >     https://lore.kernel.org/linux-block/ZKz912KyFQ7q9qwL at MiWiFi-R3L-srv/
> > 
> > The point is that kdump kernels does not limit the number of possible CPUs.
> > 
> > 1) some archs support 'nr_cpus=1' for kdump kernel, which is fine, since
> > num_possible_cpus becomes 1.
> 
> Yes, "nr_cpus=" is strongly suggested in kdump kernel because "nr_cpus="
> limits the possible cpu numbers, while "maxcpuss=" only limits the cpu
> number which can be brought up during bootup. We noticed this diference
> because a large number of possible cpus will cost more memory in kdump
> kernel. e.g percpu initialization, even though kdump kernel have set
> "maxcpus=1". 
> 
> Currently x86 and arm64 all support "nr_cpus=". Pingfan ever spent much
> effort to make patches to add "nr_cpus=" support to ppc64, seems ppc64
> dev and maintainers do not care about it. Finally the patches are not
> accepted, and the work is not continued.
> 
> Now, I am wondering what is the barrier to add "nr_cpus=" to power ach.
> Can we reconsider adding 'nr_cpus=' to power arch since real issue
> occurred in kdump kernel?

If 'nr_cpus=' can be supported on ppc64, this patchset isn't needed.

> 
> As for this patchset, it can be accpeted so that no failure in kdump
> kernel is seen on ARCHes w/o "nr_cpus=" support? My personal opinion.

IMO 'nr_cpus=' support should be preferred, given it is annoying to
maintain two kinds of implementation for kdump kernel from driver
viewpoint. I guess kdump things can be simplified too with supporting
'nr_cpus=' only.

thanks,
Ming



From bhe at redhat.com  Wed Aug  9 20:01:46 2023
From: bhe at redhat.com (Baoquan He)
Date: Thu, 10 Aug 2023 11:01:46 +0800
Subject: [PATCH V3 01/14] blk-mq: add blk_mq_max_nr_hw_queues()
In-Reply-To: <ZNRGNsRzEJfzUEzH@fedora>
References: <20230808104239.146085-1-ming.lei@redhat.com>
 <20230808104239.146085-2-ming.lei@redhat.com>
 <20230809134401.GA31852@lst.de>
 <ZNQqt1C0pXspGl3d@fedora>
 <ZNQ64xhCIBU6XM/5@MiWiFi-R3L-srv>
 <ZNRGNsRzEJfzUEzH@fedora>
Message-ID: <ZNRTGrRuwf69EgnE@MiWiFi-R3L-srv>

On 08/10/23 at 10:06am, Ming Lei wrote:
> On Thu, Aug 10, 2023 at 09:18:27AM +0800, Baoquan He wrote:
> > On 08/10/23 at 08:09am, Ming Lei wrote:
> > > On Wed, Aug 09, 2023 at 03:44:01PM +0200, Christoph Hellwig wrote:
> > > > I'm starting to sound like a broken record, but we can't just do random
> > > > is_kdump checks, and it's not going to get better by resending it again and
> > > > again.  If kdump kernels limit the number of possible CPUs, it needs to
> > > > reflected in cpu_possible_map and we need to use that information.
> > > > 
> > > 
> > > Can you look at previous kdump/arch guys' comment about kdump usage &
> > > num_possible_cpus?
> > > 
> > >     https://lore.kernel.org/linux-block/CAF+s44RuqswbosY9kMDx35crviQnxOeuvgNsuE75Bb0Y2Jg2uw at mail.gmail.com/
> > >     https://lore.kernel.org/linux-block/ZKz912KyFQ7q9qwL at MiWiFi-R3L-srv/
> > > 
> > > The point is that kdump kernels does not limit the number of possible CPUs.
> > > 
> > > 1) some archs support 'nr_cpus=1' for kdump kernel, which is fine, since
> > > num_possible_cpus becomes 1.
> > 
> > Yes, "nr_cpus=" is strongly suggested in kdump kernel because "nr_cpus="
> > limits the possible cpu numbers, while "maxcpuss=" only limits the cpu
> > number which can be brought up during bootup. We noticed this diference
> > because a large number of possible cpus will cost more memory in kdump
> > kernel. e.g percpu initialization, even though kdump kernel have set
> > "maxcpus=1". 
> > 
> > Currently x86 and arm64 all support "nr_cpus=". Pingfan ever spent much
> > effort to make patches to add "nr_cpus=" support to ppc64, seems ppc64
> > dev and maintainers do not care about it. Finally the patches are not
> > accepted, and the work is not continued.
> > 
> > Now, I am wondering what is the barrier to add "nr_cpus=" to power ach.
> > Can we reconsider adding 'nr_cpus=' to power arch since real issue
> > occurred in kdump kernel?
> 
> If 'nr_cpus=' can be supported on ppc64, this patchset isn't needed.
> 
> > 
> > As for this patchset, it can be accpeted so that no failure in kdump
> > kernel is seen on ARCHes w/o "nr_cpus=" support? My personal opinion.
> 
> IMO 'nr_cpus=' support should be preferred, given it is annoying to
> maintain two kinds of implementation for kdump kernel from driver
> viewpoint. I guess kdump things can be simplified too with supporting
> 'nr_cpus=' only.

Yes, 'nr_cpus=' is ideal. Not sure if there's some underlying concerns so
that power people decided to not support it.



From tusharsu at linux.microsoft.com  Wed Aug  9 21:29:59 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Wed, 9 Aug 2023 21:29:59 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
 <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
Message-ID: <480fb280-90ef-73c9-42cf-d72c76267259@linux.microsoft.com>



On 8/8/23 06:31, Stefan Berger wrote:
>
>
> On 8/8/23 08:35, James Bottomley wrote:
>> On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
>>>
>>>
>>> On 8/1/23 17:21, James Bottomley wrote:
>>>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>>>> [...]
>>>>> Truncating IMA log to reclaim memory is not feasible, since it
>>>>> makes the log go out of sync with the TPM PCR quote making remote
>>>>> attestation fail.
>>>>
>>>> This assumption isn't entirely true.? It's perfectly possible to
>>>> shard an IMA log using two TPM2_Quote's for the beginning and end
>>>> PCR values to validate the shard.? The IMA log could be truncated
>>>> in the same way (replace the removed part of the log with a
>>>> TPM2_Quote and AK, so the log still validates from the beginning
>>>> quote to the end).
>>>>
>>>> If you use a TPM2_Quote mechanism to save the log, all you need to
>>>> do is have the kernel generate the quote with an internal AK.? You
>>>> can keep a record of the quote and the AK at the beginning of the
>>>> truncated kernel log.? If the truncated entries are saved in a file
>>>> shard it
>>>
>>> The truncation seems dangerous to me. Maybe not all the scenarios
>>> with an attestation client (client = reading logs and quoting) are
>>> possible then anymore, such as starting an attestation client only
>>> after truncation but a verifier must have witnessed the system's PCRs
>>> and log state before the truncation occurred.
>>
>> That's not exactly correct.? Nothing needs to have "witnessed" the
>> starting PCR value because the quote vouches for it (and can vouch for
>> it after the fact).? The only thing you need to verify the quote is the
>> attestation key and the only thing you need to do to trust the> 
>> attestation key is ensure it was TPM created.? All of that can be
>> verified after the fact as well.? The only thing that can be done to
>> disrupt this is to destroy the TPM (or re-own it).> Remember the 
>> assumption is you *also* have the removed log shard to
>> present.? From that the PCR state of the starting quote can be
>
> Yes, the whole sequence of old logs needs to be available. IF that's the
> case and the logs can be stitched together seamlessly, who then looks 
> at the
> kernel AK quote and under what circumstances?
>
James has addressed this question in his response to this message [1].

Thanks James.

[1] 
https://lore.kernel.org/all/350ecdcbf7796f488807fcd7983414a02dd71be4.camel at HansenPartnership.com/#r 


~Tushar

>> calculated and checked for matching the quote.? If you lose that, it's
>> equivalent to the log being tampered with and all bets are off. The
>> assumption is that because of the impossibility of engineering TPM
>> extensions, it should be impossible to come up with a fake log that
>> produces the PCRs of the real one.? If that's violated, then IMA itself
>> becomes useless.
>>
>> James
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec



From tusharsu at linux.microsoft.com  Wed Aug  9 21:31:35 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Wed, 9 Aug 2023 21:31:35 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
 <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
 <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>
Message-ID: <2f60bc38-6a9e-6a2d-038d-cd3daf46faf2@linux.microsoft.com>



On 8/8/23 11:26, James Bottomley wrote:
> On Tue, 2023-08-08 at 09:31 -0400, Stefan Berger wrote:
>>
>> On 8/8/23 08:35, James Bottomley wrote:
>>> On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
>>>>
>>>> On 8/1/23 17:21, James Bottomley wrote:
>>>>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>>>>> [...]
>>>>>> Truncating IMA log to reclaim memory is not feasible, since
>>>>>> it makes the log go out of sync with the TPM PCR quote making
>>>>>> remote attestation fail.
>>>>> This assumption isn't entirely true.? It's perfectly possible
>>>>> to shard an IMA log using two TPM2_Quote's for the beginning
>>>>> and end PCR values to validate the shard.? The IMA log could be
>>>>> truncated in the same way (replace the removed part of the log
>>>>> with a TPM2_Quote and AK, so the log still validates from the
>>>>> beginning quote to the end).
>>>>>
>>>>> If you use a TPM2_Quote mechanism to save the log, all you need
>>>>> to do is have the kernel generate the quote with an internal
>>>>> AK.? You can keep a record of the quote and the AK at the
>>>>> beginning of the truncated kernel log.? If the truncated
>>>>> entries are saved in a file shard it
>>>> The truncation seems dangerous to me. Maybe not all the scenarios
>>>> with an attestation client (client = reading logs and quoting)
>>>> are possible then anymore, such as starting an attestation client
>>>> only after truncation but a verifier must have witnessed the
>>>> system's PCRs and log state before the truncation occurred.
>>> That's not exactly correct.? Nothing needs to have "witnessed" the
>>> starting PCR value because the quote vouches for it (and can vouch
>>> for it after the fact).? The only thing you need to verify the
>>> quote is the attestation key and the only thing you need to do to
>>> trust the attestation key is ensure it was TPM created.? All of
>>> that can be verified after the fact as well.? The only thing that
>>> can be done to disrupt this is to destroy the TPM (or re-own it).>
>>> Remember the assumption is you *also* have the removed log shard to
>>> present.? From that the PCR state of the starting quote can be
>> Yes, the whole sequence of old logs needs to be available.
> Yes and no.  If the person relying on the logs is happy they've
> extracted all the evidentiary value from the log itself then they can
> reduce the preceding log shard to simply the PCR values that match the
> quote and discard the rest.
>
>>   IF that's the case and the logs can be stitched together seamlessly,
>> who then looks at the kernel AK quote and under what circumstances?
> For incremental attestation.  Each log shard can be verified using the
> base PCR values corresponding to the bottom quote then replayed and the
> top quote verified.  This means that logs that aren't needed anymore
> can be discarded, which, I recall, was the base reason for this
> proposal: reducing IMA memory consumption.  Although all you need to do
> is extract the shards from kernel memory to file space and free the
> kernel memory.  Since that scheme can keep all logs intact, there's no
> reason to further reduce them unless the filesystem is running out of
> space.
>
> James
Thank you James for addressing Stefan?s concerns here.
Appreciate it.


~Tushar


From tusharsu at linux.microsoft.com  Wed Aug  9 21:43:22 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Wed, 9 Aug 2023 21:43:22 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <a522b17a536ea87a6a4c2faf95583ae3b7b74a26.camel@HansenPartnership.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
 <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
 <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>
 <04fb2fe5-9ebe-b35f-bdde-6ef22786438f@linux.ibm.com>
 <a522b17a536ea87a6a4c2faf95583ae3b7b74a26.camel@HansenPartnership.com>
Message-ID: <5cb03349-7a32-8f74-f2a1-ff3c6247c1ef@linux.microsoft.com>



On 8/8/23 14:41, James Bottomley wrote:
> On Tue, 2023-08-08 at 16:09 -0400, Stefan Berger wrote:
>>
>> On 8/8/23 14:26, James Bottomley wrote:
>>> On Tue, 2023-08-08 at 09:31 -0400, Stefan Berger wrote:
>>>>
>>>> On 8/8/23 08:35, James Bottomley wrote:
>>>>> On Mon, 2023-08-07 at 18:49 -0400, Stefan Berger wrote:
>>>>>>
>>>>>> On 8/1/23 17:21, James Bottomley wrote:
>>>>>>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale
>>>>>>> wrote:
>>>>>>> [...]
>>>>>>>> Truncating IMA log to reclaim memory is not feasible,
>>>>>>>> since
>>>>>>>> it makes the log go out of sync with the TPM PCR quote
>>>>>>>> making
>>>>>>>> remote attestation fail.
>>>>>>> This assumption isn't entirely true.? It's perfectly
>>>>>>> possible
>>>>>>> to shard an IMA log using two TPM2_Quote's for the
>>>>>>> beginning
>>>>>>> and end PCR values to validate the shard.? The IMA log
>>>>>>> could be
>>>>>>> truncated in the same way (replace the removed part of the
>>>>>>> log
>>>>>>> with a TPM2_Quote and AK, so the log still validates from
>>>>>>> the
>>>>>>> beginning quote to the end).
>>>>>>>
>>>>>>> If you use a TPM2_Quote mechanism to save the log, all you
>>>>>>> need
>>>>>>> to do is have the kernel generate the quote with an
>>>>>>> internal
>>>>>>> AK.? You can keep a record of the quote and the AK at the
>>>>>>> beginning of the truncated kernel log.? If the truncated
>>>>>>> entries are saved in a file shard it
>>>>>> The truncation seems dangerous to me. Maybe not all the
>>>>>> scenarios
>>>>>> with an attestation client (client = reading logs and
>>>>>> quoting)
>>>>>> are possible then anymore, such as starting an attestation
>>>>>> client
>>>>>> only after truncation but a verifier must have witnessed the
>>>>>> system's PCRs and log state before the truncation occurred.
>>>>> That's not exactly correct.? Nothing needs to have "witnessed"
>>>>> the
>>>>> starting PCR value because the quote vouches for it (and can
>>>>> vouch
>>>>> for it after the fact).? The only thing you need to verify the
>>>>> quote is the attestation key and the only thing you need to do
>>>>> to
>>>>> trust the attestation key is ensure it was TPM created.? All of
>>>>> that can be verified after the fact as well.? The only thing
>>>>> that
>>>>> can be done to disrupt this is to destroy the TPM (or re-own
>>>>> it).>
>>>>> Remember the assumption is you *also* have the removed log
>>>>> shard to
>>>>> present.? From that the PCR state of the starting quote can be
>>>> Yes, the whole sequence of old logs needs to be available.
>>> Yes and no.? If the person relying on the logs is happy they've
>>> extracted all the evidentiary value from the log itself then they
>>> can
>>> reduce the preceding log shard to simply the PCR values that match
>>> the
>>> quote and discard the rest.
>>>
>>>>  ? IF that's the case and the logs can be stitched together
>>>> seamlessly, who then looks at the kernel AK quote and under what
>>>> circumstances?
>>> For incremental attestation.? Each log shard can be verified using
>>> the base PCR values corresponding to the bottom quote then replayed
>>> and the
>>
>> Somehow you have to tell a verifier to take a snapshot of the current
>> state of the PCRs when it replays the logs to be able to truncate the
>> log.
> No, the verifier is server side.  It would be the agent or the kernel,
> client side, which gets the quote and shards the log.  That way the
> operation can be done in such a way as to make sure the quote and the
> shard point match.  I'd imagine the verifier can provide some sort of
> guide as to how big it wants the shards to be and the client complies.
Yup. Agreed.
>> Whether the state of the PCRs is in the log itself or it's just some
>> sort of entry in the log indicating a truncation probably doesn't
>> matter for as long as the verifying side keeps state of the PCRs at
>> point of truncatiokn.
> The idea would be the log shard would be self attesting, that's why
> quote at beginning and end (and probably PCR state at beginning), so no
> verifier required until someone wants to check the log.
>
>> Also, the verifying side needs to take notice of the trustworthiness
>> of the system at the time the log was truncated in case the
>> attestation client is restarted and starts out sending the log with
>> the first entry.
> That's usually what a runtime verification system is actually
> verifying, yes.
>
>>   The PCR state shown at the beginning of the truncated log (when
>> restarting the attestation client) must then match when the 'notice'
>> was taken and that determines its trustworthiness at this point in
>> the log.
>>
>> That there's a kernel AK signature
> A quote is a signature over PCR state signed by an AK, yes.
>
>>   at this point doesn't seem necessary since one presumably can verify
>> the log and PCR states at the end with the 'regular' quote.
> I don't understand this.  A regular quote is a signature over PCR state
> by an AK.  The point about saving the AK in the log for the original is
> that if the *kernel* truncates the log and saves it to a file, it needs
> to generate both the AK and the quote for the top of the file shard.
> That means the AK/EK binding is unverified, but can be verified by
> loading the AK and running the usual tests, which can only be done if
> you have the loadable AK, which is why you need it as part of the log
> saving proposal.
I had this question about the usability of AK/EK in this
context. Although AK/EK + PCR quote is needed to verify the snapshot
shards / IMA logs are not tampered with, I am still not sure why AK/EK
needs to be part of the shard/IMA log. The client sending AK/EK to
attestation service separately would still serve the purpose, right?

For instance, PCR quotes will be signed by AK. So as long as the verifier
trusts the AK/EK, it can verify the quotes are not tampered with.
Replaying IMA log/snapshot can produce the PCR quotes which can be
matched with signed PCR quotes. If they match, then the verifier can
conclude that the IMA log is not tampered with. So AK doesn't need to
be part of the log/snapshot.

BTW, in this proposal, kernel is truncating the log and passing the
truncated buffer to UM.? UM client need to save it to the disk location
of it's choice.


>> Nobody should ever trust a system by starting to look at the
>> beginning of a truncated log. You have to have evaluated all the
>> entries in the log before and determined whether the system was
>> trustworthy. I don't think the kernel AK quote buys much - at least
>> not from what I can see.
> You have a log shard with PCR state at the beginning and calculated at
> the end, both of which you can verify.  How you establish trust in the
> starting PCR values of the shard is a policy decision, but the policy
> doesn't have to be you see every shard up to boot.  You could take the
> word of the system owner on a handoff of responsibility for instance.
Agreed. You don?t have to go all the way to the boot snapshot
(shard).? It is up to the attestation service which last-known-good PCR
quotes (sometimes referred as golden PCR quotes) it trusts, and verify
the snapshots / logs after that.

~Tushar

> James



From James.Bottomley at HansenPartnership.com  Thu Aug 10 04:43:25 2023
From: James.Bottomley at HansenPartnership.com (James Bottomley)
Date: Thu, 10 Aug 2023 07:43:25 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <5cb03349-7a32-8f74-f2a1-ff3c6247c1ef@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
	 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
	 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
	 <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
	 <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>
	 <04fb2fe5-9ebe-b35f-bdde-6ef22786438f@linux.ibm.com>
	 <a522b17a536ea87a6a4c2faf95583ae3b7b74a26.camel@HansenPartnership.com>
	 <5cb03349-7a32-8f74-f2a1-ff3c6247c1ef@linux.microsoft.com>
Message-ID: <8ccaec30bf85cfbf4415bbafa22646a62e753840.camel@HansenPartnership.com>

On Wed, 2023-08-09 at 21:43 -0700, Tushar Sugandhi wrote:
> On 8/8/23 14:41, James Bottomley wrote:
> > On Tue, 2023-08-08 at 16:09 -0400, Stefan Berger wrote:
[...]
> > > ? at this point doesn't seem necessary since one presumably can
> > > verify the log and PCR states at the end with the 'regular'
> > > quote.
> > ?
> > I don't understand this.? A regular quote is a signature over PCR
> > state by an AK.? The point about saving the AK in the log for the
> > original is that if the *kernel* truncates the log and saves it to
> > a file, it needs to generate both the AK and the quote for the top
> > of the file shard. That means the AK/EK binding is unverified, but
> > can be verified by loading the AK and running the usual tests,
> > which can only be done if you have the loadable AK, which is why
> > you need it as part of the log saving proposal.
> ?
> I had this question about the usability of AK/EK in this
> context. Although AK/EK + PCR quote is needed to verify the snapshot
> shards / IMA logs are not tampered with, I am still not sure why
> AK/EK needs to be part of the shard/IMA log. The client sending AK/EK
> to attestation service separately would still serve the purpose,
> right?

Well, the EK doesn't need to be part of the log: it's just a permanent
part of the TPM identity.  To verify the log, you need access to the
TPM that was used to create it, so that's the point at which you get
the EK.

An AK is simply a TPM generated signing key (meaning the private part
of the key is secured by the TPM and known to no-one else).  In the
literature a TPM generated signing key doesn't become an Attestation
Key until it's been verified using an EK property (either a certify for
a signing EK or a make/activate credential round trip for the more
usual encryption EK.

So the proposal is for each quote that's used to verify a log shard is
that the TPM simply generate a random signing key and use that to sign
the quote.  You need to save the TPM form of the generated key so it
can be loaded later and the reason for that is you can do the EK
verification at any time after the quote was given by loading the saved
key and running the verification protocol.  In the normal attestation
you do the EK verification of the AK *before* the quote, but there's no
property of the quote that depends on this precedence provided you do
the quote with a TPM generated signing key.

The underlying point is that the usual way an EK verifies an AK
requires a remote observer, which the kernel won't have, so the kernel
must do all its stuff locally (generate key, get quote) and then at
some point later the system can become remote connected and prove to
whatever external entity that the log shard is valid.  So we have to
have all the components necessary for that proof: the log shard, the
quote and the TPM form of the AK.

> For instance, PCR quotes will be signed by AK. So as long as the
> verifier trusts the AK/EK,

Right, but if you're sharding a log, the kernel doesn't know if a
verifier has been in contact yet.  The point of the protocol above is
to make that not matter.  The verifier can contact the system after the
log has been saved and the verification will still work.

>  it can verify the quotes are not tampered with.
> Replaying IMA log/snapshot can produce the PCR quotes which can be
> matched with signed PCR quotes. If they match, then the verifier can
> conclude that the IMA log is not tampered with. So AK doesn't need to
> be part of the log/snapshot.

Only if the system is currently in contact with the verifier and the
verifier has created the AK.  That may not have happened.

> BTW, in this proposal, kernel is truncating the log and passing the
> truncated buffer to UM.? UM client need to save it to the disk
> location of it's choice.

Yes, but I was assuming tampering with or discarding the log file would
be treated in exactly the same way as an in-kernel IMA log tamper.

James



From horms at kernel.org  Thu Aug 10 05:11:47 2023
From: horms at kernel.org (Simon Horman)
Date: Thu, 10 Aug 2023 14:11:47 +0200
Subject: [PATCHv7 1/5] kexec/arm64: Simplify the code for zImage
In-Reply-To: <CALu+AoS-Df2zrAAHnjHnjc-tZPqGfnme2O-xj28Qofg5JdaR9Q@mail.gmail.com>
References: <20230803024152.11663-1-piliu@redhat.com>
 <20230803024152.11663-2-piliu@redhat.com>
 <ZNDv7L3lWXZxrmWT@vergenet.net>
 <CALu+AoS-Df2zrAAHnjHnjc-tZPqGfnme2O-xj28Qofg5JdaR9Q@mail.gmail.com>
Message-ID: <ZNTUA/+Q2BVkhxHU@vergenet.net>

On Mon, Aug 07, 2023 at 09:26:49PM +0800, Dave Young wrote:
> On Mon, 7 Aug 2023 at 21:23, Simon Horman <horms at kernel.org> wrote:
> >
> > On Thu, Aug 03, 2023 at 10:41:48AM +0800, Pingfan Liu wrote:
> > > Inside zimage_probe(), it uncompresses the kernel and performs some
> > > check, similar to image_probe(). Taking a close look, the uncompressing
> > > has already executed before the image probe is called. What is missing
> > > here is to provide a fd, pointing to an uncompressed kernel image.
> > >
> > > This patch creates a memfd based on the result produced by
> > > slurp_decompress_file(), and finally simplify the logical of the probe
> > > for aarch64.
> > >
> > > The credit goes to the Dave Young, who contributes the original code.
> > >
> > > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > > Co-authored-by: Dave Young <dyoung at redhat.com>
> >
> > Dave, could I get a Signed-off-by line from you?
> > Likewise for patch 2/5.
> 
> I Simon, sounds good to me, thanks!

Thanks, could your respond to the two patches with the following?

Signed-off by: <<your details go here>>

> 
> >
> > I think simply replying to the relevant emails should be sufficient.
> >
> > _______________________________________________
> > kexec mailing list
> > kexec at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kexec
> >
> 


From stefanb at linux.ibm.com  Thu Aug 10 07:12:42 2023
From: stefanb at linux.ibm.com (Stefan Berger)
Date: Thu, 10 Aug 2023 10:12:42 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <b538f7d2-5a04-46d0-3792-a18653230a95@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <b538f7d2-5a04-46d0-3792-a18653230a95@linux.microsoft.com>
Message-ID: <011d8a79-236f-dc20-08fc-b5da7dd1d5a7@linux.ibm.com>



On 8/9/23 21:15, Tushar Sugandhi wrote:
> Thanks a lot Stefan for looking into this proposal,
> and providing your feedback. We really appreciate it.
> 
> On 8/7/23 15:49, Stefan Berger wrote:
>>
>>
>> On 8/1/23 17:21, James Bottomley wrote:
>>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>>> [...]
>>>> Truncating IMA log to reclaim memory is not feasible, since it makes
>>>> the log go out of sync with the TPM PCR quote making remote
>>>> attestation fail.
>>>
>>> This assumption isn't entirely true.? It's perfectly possible to shard
>>> an IMA log using two TPM2_Quote's for the beginning and end PCR values
>>> to validate the shard.? The IMA log could be truncated in the same way
>>> (replace the removed part of the log with a TPM2_Quote and AK, so the
>>> log still validates from the beginning quote to the end).
>>>
>>> If you use a TPM2_Quote mechanism to save the log, all you need to do
>>> is have the kernel generate the quote with an internal AK.? You can
>>> keep a record of the quote and the AK at the beginning of the truncated
>>> kernel log.? If the truncated entries are saved in a file shard it
>>
>> The truncation seems dangerous to me. Maybe not all the scenarios with an attestation
>> client (client = reading logs and quoting) are possible then anymore, such as starting
>> an attestation client only after truncation but a verifier must have witnessed the
>> system's PCRs and log state before the truncation occurred.
> You are correct that truncation on it?s own is dangerous. It needs to be
> accompanied by (a) saving the IMA log data to disk as snapshots, (b) adding the
> necessary TPM PCR quotes to the current IMA log (as James mentioned above),
> (c) attestation clients having an ability to send the past snapshots to the
> remote-attestation-service (verifiers), (d) and verifiers having an ability
> to use the snapshots along with current IMA logs for the purpose of attestation.
> All these points are explained in the original RFC email in sections B.1 through B.5 [1].

I read it.

Maybe you have dismissed the PCR update counter already...
I am not sure what the PCR update counter is supposed to help with. It won't allow you to detect
missing log events but rather will confuse anyone looking at it when my application extends PCR 12
for example, which also affects the update counter. It's a global counter that increases with every
PCR extension (except PCR 16, 21, 22, 23) and if used as proposed would prevent any application from
extending PCRs.

https://github.com/stefanberger/libtpms/blob/master/src/tpm2/PCR.c#L667
https://github.com/stefanberger/libtpms/blob/master/src/tpm2/PCR.c#L629
https://github.com/stefanberger/libtpms/blob/master/src/tpm2/PCR.c#L161


The shards should will need to be written into some sort of standard location or a config file needs to
be defined, so that everyone knows where to find them and how they are named.


>>
>> I think an ima-buf (or similar) log entry in IMA log would have to appear at the beginning of the
>> truncated log stating the value of all PCRs that IMA touched (typically only PCR 10
>> but it can be others). The needs to be done since the quote itself doesn't
>> provide the state of the individual PCRs. This would at least allow an attestation
>> client to re-read the log from the beginning (when it is re-start or started for the
>> first time after the truncation). 
>  ?Agreed. See the description of snapshot_aggregate in Section B.5 in the
> original RFC email [1].
>> However, this alone (without the
>> internal AK quoting the old state) could lead to abuse where I could create totally
>> fake IMA logs stating the state of the PCRs at the beginning (so the verifier
>> syncs its internal PCR state to this state). 
> Yes, the PCR quotes sent to the verifier must be signed by the AK that
> is trusted by the verifier. That assumption is true regardless of IMA log
> snapshotting feature.
>> Further, even with the AK-quote that
>> you propose I may be able to create fake logs and trick a verifier into
>> trusting the machine IFF it doesn't know what kernel this system was booted with
>> that I may have hacked to provide a fake AK-quote that just happens to match the
>> PCR state presented at the beginning of the log.
>>
> If the Kernel is compromised, then all-bets are off.
> (Regardless of IMA log snapshotting feature.)
>> => Can a truncated log be made safe for attestation when the attestation starts
>> only after the truncation occurred?
>>
> Yes. If the ?PCR quotes in the snapshot_aggregate event in IMA log?

PCR quote or 'quotes'? Why multiple?

Form your proposal but you may have changed your opinion  following what I see in other messages:
"- The Kernel will get the current TPM PCR values and PCR update counter [2]
    and store them as template data in a new IMA event "snapshot_aggregate"."

Afaik TPM quote's don't give you the state of the individual PCR values, therefore
I would expect to at least find the 'PCR values' of all the PCRs that IMA touched to
be in the snapshot_aggregate so I can replay all the following events on top of these
PCR values and come up with the values that were used in the "final PCR quote". This
is unless you expect the server to take an automatic snapshot of the values of the
PCRs  that it computed while evaluating the log in case it ever needs to go back.

> + "replay of rest of the events in IMA log" results in the ?final PCR quotes?
> that matches with the ?AK signed PCR quotes? sent by the client, then the truncated
> IMA log can be trusted. The verifier can either ?trust? the ?PCR quotes in the
> snapshot_aggregate event in IMA log? or it can ask for the (n-1)th snapshot shard
> to check the past events.

For anything regarding determining the 'trustworthiness of a system' one would have to
be able to go back to the very beginning of the log *or* remember in what state a
system was when the latest snapshot was taken so that if a restart happens it can resume
with that assumption about state of trustworthiness and know what the values of the PCRs
were at that time so it can resume replaying the log (or the server would get these
values from the log).

The AK quotes by the kernel (which adds a 2nd AK key) that James is proposing
could be useful if the entire log, consisting of multiple shards, is very large and
cannot be transferred from the client to the server in one go so that the server could
evaluate the 'final PCR quote' immediately . However, if a client can indicated 'I will
send more the next time and I have this much more to transfer' and the server allows
this multiple times (until all the 1MB shards of the 20MB log are transferred) then that
kernel AK key would not be necessary since presumably the "final PCR quote", created
by a user space client, would resolve whether the entire log is trustworthy.

> 
>> => Even if attestation was occurring 'what' state does an attestation server
>> need to carry around for an attested-to system so that the truncation is 'safe'
>> and I cannot create fake AK-quotes and fake IMA logs with initial PCR states?
> Assuming most of the client devices take a snapshot at specific checkpoints,
> the ?PCR quotes in the snapshot_aggregate event in IMA log? will be the same for them.
> The remote attestation server will have to remember these golden PCR quotes.

I thought maybe 'golden PCR values'... because those let me replay PCR extensions from
a previous point.

> It doesn't have to remember the state of each client device.

Can you give a reason for this? You mean the state doesn't need to be remembered for client
devices whose log hasn't been truncated?

  
>> Can I ever restart the client and have it read the truncated log from the
>> beginning and what type of verification needs to happen on the server then?
>>
> Yes, restarting the client should be possible.

Yes, this must be possible.

>> It seems like the server would have to remember the state of the IMA PCRs upon
>> last truncation to detect a possible attack. This would make staring to monitor
>> a system after truncation impossible -- would be good to know these details.
>>
> The server is not forced to remember the state of IMA PCRs. It can
> always ask for the last n snapshot files (shards) and replay the events. Even
> though the data is truncated from the IMA log, it is not totally lost. It is
> simply being transferred to the disk. It is saved by UM as snapshot files/shards.
> The goal of IMA snapshotting is to reduce the Kernel memory pressure on the
> client devices - to save them from out-of-memory errors which are harder to manage
> on long running clients. It comes with a cost of additional work on the server
> side to attest those clients.

Agreed.
> 
> 
> Being said that, in the current proposal, taking a snapshots is totally optional
> and controlled by UM attestation clients. If the attestation-clients/services are
> not-ready/don?t-want to take advantage of IMA log snapshotting, they don?t have to.

Agreed.

> 
> No snapshot will be taken, and the client-service can process the monolithic IMA
> log just like they do today.
> 

Agreed.

> [1] https://lore.kernel.org/all/c5737141-7827-1c83-ab38-0119dcfea485 at linux.microsoft.com/#t
> 
>>
>>
>>
>>> should have a beginning and end quote and a record of the AK used.
>>> Since verifiers like Keylime are already using this beginning and end
>>> quote for sharded logs, it's the most natural format to feed to
>>> something externally for verification and it means you don't have to
>>> invent a new format to do the same thing.
>>>
>>> Regards,
>>>
>>> James
>>>
>>>
>>> _______________________________________________
>>> kexec mailing list
>>> kexec at lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/kexec
> 


From dyoung at redhat.com  Thu Aug 10 17:31:44 2023
From: dyoung at redhat.com (Dave Young)
Date: Fri, 11 Aug 2023 08:31:44 +0800
Subject: [PATCHv7 1/5] kexec/arm64: Simplify the code for zImage
In-Reply-To: <ZNTUA/+Q2BVkhxHU@vergenet.net>
References: <20230803024152.11663-1-piliu@redhat.com> <20230803024152.11663-2-piliu@redhat.com>
 <ZNDv7L3lWXZxrmWT@vergenet.net> <CALu+AoS-Df2zrAAHnjHnjc-tZPqGfnme2O-xj28Qofg5JdaR9Q@mail.gmail.com>
 <ZNTUA/+Q2BVkhxHU@vergenet.net>
Message-ID: <CALu+AoQNCAb=tm-naV=8jr6yaT3wTyobQBr0t-2aWb2pyCbkpw@mail.gmail.com>

On Thu, 10 Aug 2023 at 20:11, Simon Horman <horms at kernel.org> wrote:
>
> On Mon, Aug 07, 2023 at 09:26:49PM +0800, Dave Young wrote:
> > On Mon, 7 Aug 2023 at 21:23, Simon Horman <horms at kernel.org> wrote:
> > >
> > > On Thu, Aug 03, 2023 at 10:41:48AM +0800, Pingfan Liu wrote:
> > > > Inside zimage_probe(), it uncompresses the kernel and performs some
> > > > check, similar to image_probe(). Taking a close look, the uncompressing
> > > > has already executed before the image probe is called. What is missing
> > > > here is to provide a fd, pointing to an uncompressed kernel image.
> > > >
> > > > This patch creates a memfd based on the result produced by
> > > > slurp_decompress_file(), and finally simplify the logical of the probe
> > > > for aarch64.
> > > >
> > > > The credit goes to the Dave Young, who contributes the original code.
> > > >
> > > > Signed-off-by: Pingfan Liu <piliu at redhat.com>
> > > > Co-authored-by: Dave Young <dyoung at redhat.com>
> > >
> > > Dave, could I get a Signed-off-by line from you?
> > > Likewise for patch 2/5.
> >
> > I Simon, sounds good to me, thanks!
>
> Thanks, could your respond to the two patches with the following?
>
> Signed-off by: <<your details go here>>

Sure, please feel free to add:

Signed-off-by: Dave Young <dyoung at redhat.com>

>
> >
> > >
> > > I think simply replying to the relevant emails should be sufficient.
> > >
> > > _______________________________________________
> > > kexec mailing list
> > > kexec at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/kexec
> > >
> >
>



From dyoung at redhat.com  Thu Aug 10 18:23:05 2023
From: dyoung at redhat.com (Dave Young)
Date: Fri, 11 Aug 2023 09:23:05 +0800
Subject: [PATCHv7 2/5] kexec: Introduce a member kernel_fd in kexec_info
In-Reply-To: <20230803024152.11663-3-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com> <20230803024152.11663-3-piliu@redhat.com>
Message-ID: <CALu+AoSj+cc4zrWYcwNHBV__FYfH+bST+D_BOdCix2RCq80REQ@mail.gmail.com>

Hi,

On Thu, 3 Aug 2023 at 10:42, Pingfan Liu <piliu at redhat.com> wrote:
>
> Utilize the image load interface to export the kernel fd, which points
> to the uncompressed kernel and will be passed to kexec_file_load.
>
> The credit goes to the Dave Young, who contributes the original code.
>
> Signed-off-by: Pingfan Liu <piliu at redhat.com>
> Co-authored-by: Dave Young <dyoung at redhat.com>

Signed-off-by: Dave Young <dyoung at redhat.com>

> To: kexec at lists.infradead.org
> Cc: horms at verge.net.au
> Cc: ardb at kernel.org
> Cc: jeremy.linton at arm.com
> ---
>  kexec/kexec.c | 8 ++++++++
>  kexec/kexec.h | 1 +
>  2 files changed, 9 insertions(+)
>
> diff --git a/kexec/kexec.c b/kexec/kexec.c
> index d132eb5..c3b182e 100644
> --- a/kexec/kexec.c
> +++ b/kexec/kexec.c
> @@ -1292,6 +1292,7 @@ static int do_kexec_file_load(int fileind, int argc, char **argv,
>         info.kexec_flags = flags;
>
>         info.file_mode = 1;
> +       info.kernel_fd = -1;
>         info.initrd_fd = -1;
>
>         if (!is_kexec_file_load_implemented())
> @@ -1337,6 +1338,13 @@ static int do_kexec_file_load(int fileind, int argc, char **argv,
>                 return ret;
>         }
>
> +       /*
> +       * image type specific load functioin detect the capsule kernel type
> +       * and create another fd for file load. For example the zboot kernel.
> +       */
> +       if (info.kernel_fd != -1)
> +               kernel_fd = info.kernel_fd;
> +
>         /*
>          * If there is no initramfs, set KEXEC_FILE_NO_INITRAMFS flag so that
>          * kernel does not return error with negative initrd_fd.
> diff --git a/kexec/kexec.h b/kexec/kexec.h
> index 0d820ad..ed3b499 100644
> --- a/kexec/kexec.h
> +++ b/kexec/kexec.h
> @@ -164,6 +164,7 @@ struct kexec_info {
>         unsigned long file_mode :1;
>
>         /* Filled by kernel image processing code */
> +       int kernel_fd;
>         int initrd_fd;
>         char *command_line;
>         int command_line_len;
> --
> 2.31.1
>



From horms at kernel.org  Fri Aug 11 00:29:19 2023
From: horms at kernel.org (Simon Horman)
Date: Fri, 11 Aug 2023 09:29:19 +0200
Subject: [PATCHv7 0/5] arm64: zboot support
In-Reply-To: <20230803024152.11663-1-piliu@redhat.com>
References: <20230803024152.11663-1-piliu@redhat.com>
Message-ID: <ZNXjT8J2UCktknRa@vergenet.net>

On Thu, Aug 03, 2023 at 10:41:47AM +0800, Pingfan Liu wrote:
> From: root <root at piliu.users.ipa.redhat.com>
> 
> As more complicated capsule kernel format occurs like zboot, where the
> compressed kernel is stored as a payload. The straight forward
> decompression can not meet the demand.
>   
> As the first step, on aarch64, reading in the kernel file in a probe
> method and decide how to unfold the content by the method itself.
> 
> This series consists of two parts
> [1/5], simplify the current aarch64 image probe
> [2-5/5], return the kernel fd by the image load interface, and let the
> handling of zboot image built on it. (Thanks for Dave Young, who
> contributes the original idea and the code)
>  
>  
> To ease the review, a branch is also available at https://github.com/pfliu/kexec-tools.git
> branch zbootV7
>  
> To: kexec at lists.infradead.org
> Cc: Dave Young <dyoung at redhat.com>
> Cc: horms at verge.net.au
> Cc: ardb at kernel.org
> Cc: jeremy.linton at arm.com

Thanks everyone,

applied.


From horms at kernel.org  Fri Aug 11 00:30:59 2023
From: horms at kernel.org (Simon Horman)
Date: Fri, 11 Aug 2023 09:30:59 +0200
Subject: [ANNOUNCE] kexec-tools v2.0.27 preparation
Message-ID: <ZNXjs19UOvGEl/5K@vergenet.net>

Hi all,

I am planning to release kexec-tools v2.0.27 in the next two weeks
to roughly coincide with the release of the v6.5 kernel.

I would like to ask interested parties to send any patches they would like
included in v2.0.27 within one week so they can be considered for inclusion
in an rc release.

For reference the patches queued up since v2.0.26 are as follows.

Thanks to everyone who has contributed to kexec-tools!

f67c4146d7b5 arm64: Hook up the ZBOOT support as vmlinuz
fc7b83bdf734 arm64: Add ZBOOT PE containing compressed image support
f41c4182b0c4 kexec/zboot: Add arch independent zboot support
1572b91da7c4 kexec: Introduce a member kernel_fd in kexec_info
714fa11590fe kexec/arm64: Simplify the code for zImage
a8de94e5f033 LoongArch: kdump: Set up kernel image segment
4203eaccfa92 kexec: __NR_kexec_file_load is set to undefined on LoongArch
63e9a012112e ppc64: Add elf-ppc64 file types/options and an arch specific flag to man page
806711fca9e9 x86: add devicetree support
29fe5067ed07 kexec: make -a the default
e63fefd4fc35 ppc64: add --reuse-cmdline parameter support
8fc55927f700 kexec-tools 2.0.26.git


From hbathini at linux.ibm.com  Fri Aug 11 00:53:05 2023
From: hbathini at linux.ibm.com (Hari Bathini)
Date: Fri, 11 Aug 2023 13:23:05 +0530
Subject: [PATCH V3 01/14] blk-mq: add blk_mq_max_nr_hw_queues()
In-Reply-To: <ZNRTGrRuwf69EgnE@MiWiFi-R3L-srv>
References: <20230808104239.146085-1-ming.lei@redhat.com>
 <20230808104239.146085-2-ming.lei@redhat.com> <20230809134401.GA31852@lst.de>
 <ZNQqt1C0pXspGl3d@fedora> <ZNQ64xhCIBU6XM/5@MiWiFi-R3L-srv>
 <ZNRGNsRzEJfzUEzH@fedora> <ZNRTGrRuwf69EgnE@MiWiFi-R3L-srv>
Message-ID: <772c4140-3035-16d8-0253-f5893c3698e2@linux.ibm.com>



On 10/08/23 8:31 am, Baoquan He wrote:
> On 08/10/23 at 10:06am, Ming Lei wrote:
>> On Thu, Aug 10, 2023 at 09:18:27AM +0800, Baoquan He wrote:
>>> On 08/10/23 at 08:09am, Ming Lei wrote:
>>>> On Wed, Aug 09, 2023 at 03:44:01PM +0200, Christoph Hellwig wrote:
>>>>> I'm starting to sound like a broken record, but we can't just do random
>>>>> is_kdump checks, and it's not going to get better by resending it again and
>>>>> again.  If kdump kernels limit the number of possible CPUs, it needs to
>>>>> reflected in cpu_possible_map and we need to use that information.
>>>>>
>>>>
>>>> Can you look at previous kdump/arch guys' comment about kdump usage &
>>>> num_possible_cpus?
>>>>
>>>>      https://lore.kernel.org/linux-block/CAF+s44RuqswbosY9kMDx35crviQnxOeuvgNsuE75Bb0Y2Jg2uw at mail.gmail.com/
>>>>      https://lore.kernel.org/linux-block/ZKz912KyFQ7q9qwL at MiWiFi-R3L-srv/
>>>>
>>>> The point is that kdump kernels does not limit the number of possible CPUs.
>>>>
>>>> 1) some archs support 'nr_cpus=1' for kdump kernel, which is fine, since
>>>> num_possible_cpus becomes 1.
>>>
>>> Yes, "nr_cpus=" is strongly suggested in kdump kernel because "nr_cpus="
>>> limits the possible cpu numbers, while "maxcpuss=" only limits the cpu
>>> number which can be brought up during bootup. We noticed this diference
>>> because a large number of possible cpus will cost more memory in kdump
>>> kernel. e.g percpu initialization, even though kdump kernel have set
>>> "maxcpus=1".
>>>
>>> Currently x86 and arm64 all support "nr_cpus=". Pingfan ever spent much
>>> effort to make patches to add "nr_cpus=" support to ppc64, seems ppc64
>>> dev and maintainers do not care about it. Finally the patches are not
>>> accepted, and the work is not continued.
>>>
>>> Now, I am wondering what is the barrier to add "nr_cpus=" to power ach.
>>> Can we reconsider adding 'nr_cpus=' to power arch since real issue
>>> occurred in kdump kernel?
>>
>> If 'nr_cpus=' can be supported on ppc64, this patchset isn't needed.
>>
>>>
>>> As for this patchset, it can be accpeted so that no failure in kdump
>>> kernel is seen on ARCHes w/o "nr_cpus=" support? My personal opinion.
>>
>> IMO 'nr_cpus=' support should be preferred, given it is annoying to
>> maintain two kinds of implementation for kdump kernel from driver
>> viewpoint. I guess kdump things can be simplified too with supporting
>> 'nr_cpus=' only.
> 
> Yes, 'nr_cpus=' is ideal. Not sure if there's some underlying concerns so
> that power people decided to not support it.

Though "nr_cpus=1" is an ideal solution, maintainer was not happy with
the patch as the code changes have impact for regular boot path and
it is likely to cause breakages. So, even if "nr_cpus=1" support for
ppc64 is revived, the change is going to take time to be accepted
upstream.

Also, I see is_kdump_kernel() being used irrespective of "nr_cpus=1"
support for other optimizations in the driver for the special dump
capture environment kdump is.

If there is no other downside for driver code, to use is_kdump_kernel(),
other than the maintainability aspect, I think the above changes are
worth considering.

Thanks
Hari


From songshuaishuai at tinylab.org  Fri Aug 11 03:41:17 2023
From: songshuaishuai at tinylab.org (Song Shuai)
Date: Fri, 11 Aug 2023 18:41:17 +0800
Subject: [PATCH 2/2] =?UTF-8?q?RISC-V:=20Fix=20the=20undeclared=20?= =?UTF-8?q?=E2=80=98EM=5FRISCV=E2=80=99=20build=20failure?=
In-Reply-To: <20230811104117.91406-1-songshuaishuai@tinylab.org>
References: <20230811104117.91406-1-songshuaishuai@tinylab.org>
Message-ID: <20230811104117.91406-2-songshuaishuai@tinylab.org>

Use local `elf.h` instead of `linux/elf.h` to fix this build error:

```
kexec/arch/riscv/crashdump-riscv.c:17:13: error: ?EM_RISCV? undeclared here (not in a function); did you mean ?EM_CRIS??
  .machine = EM_RISCV,
             ^~~~~~~~
             EM_CRIS
```

Signed-off-by: Song Shuai <songshuaishuai at tinylab.org>
---
 kexec/arch/riscv/crashdump-riscv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kexec/arch/riscv/crashdump-riscv.c b/kexec/arch/riscv/crashdump-riscv.c
index 3ed4fe3..336d7a7 100644
--- a/kexec/arch/riscv/crashdump-riscv.c
+++ b/kexec/arch/riscv/crashdump-riscv.c
@@ -1,5 +1,5 @@
 #include <errno.h>
-#include <linux/elf.h>
+#include <elf.h>
 #include <unistd.h>
 
 #include "kexec.h"
-- 
2.20.1



From songshuaishuai at tinylab.org  Fri Aug 11 03:41:16 2023
From: songshuaishuai at tinylab.org (Song Shuai)
Date: Fri, 11 Aug 2023 18:41:16 +0800
Subject: [PATCH 1/2] RISC-V: Use linux,usable-memory-range for crash kernel
Message-ID: <20230811104117.91406-1-songshuaishuai@tinylab.org>

Now we use "memeory::linux,usable-memory" to indicate the available
memory for the crash kernel.

While booting with UEFI, the crash kernel would use efi.memmap to
re-populate memblock and then first kernel's memory would be corrputed.
Consequently, the /proc/vmcore file failed to create in my local test.

And according to "chosen" dtschema [1], the available memory for the
crash kernel should be held via "chosen::linux,usable-memory-range"
property which will re-cap memblock even after UEFI's re-population.

[1]:
https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/chosen.yaml

Signed-off-by: Song Shuai <songshuaishuai at tinylab.org>
---
 kexec/arch/riscv/kexec-riscv.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/kexec/arch/riscv/kexec-riscv.c b/kexec/arch/riscv/kexec-riscv.c
index fe5dd2d..5aea035 100644
--- a/kexec/arch/riscv/kexec-riscv.c
+++ b/kexec/arch/riscv/kexec-riscv.c
@@ -79,20 +79,20 @@ int load_extra_segments(struct kexec_info *info, uint64_t kernel_base,
 		}
 
 		ret = dtb_add_range_property(&fdt->buf, &fdt->size, start, end,
-					     "memory", "linux,usable-memory");
+					     "chosen", "linux,usable-memory-range");
 		if (ret) {
-			fprintf(stderr, "Couldn't add usable-memory to fdt\n");
+			fprintf(stderr, "Couldn't add usable-memory-range to fdt\n");
 			return ret;
 		}
 
 		max_usable = end;
 	} else {
 		/*
-		 * Make sure we remove elfcorehdr and usable-memory
+		 * Make sure we remove elfcorehdr and usable-memory-range
 		 * when switching from crash kernel to a normal one.
 		 */
 		dtb_delete_property(fdt->buf, "chosen", "linux,elfcorehdr");
-		dtb_delete_property(fdt->buf, "memory", "linux,usable-memory");
+		dtb_delete_property(fdt->buf, "chosen", "linux,usable-memory-range");
 	}
 
 	/* Do we need to include an initrd image ? */
-- 
2.20.1



From hch at lst.de  Fri Aug 11 06:10:49 2023
From: hch at lst.de (Christoph Hellwig)
Date: Fri, 11 Aug 2023 15:10:49 +0200
Subject: [PATCH V3 01/14] blk-mq: add blk_mq_max_nr_hw_queues()
In-Reply-To: <ZNQqt1C0pXspGl3d@fedora>
References: <20230808104239.146085-1-ming.lei@redhat.com> <20230808104239.146085-2-ming.lei@redhat.com> <20230809134401.GA31852@lst.de> <ZNQqt1C0pXspGl3d@fedora>
Message-ID: <20230811131049.GA20991@lst.de>

On Thu, Aug 10, 2023 at 08:09:27AM +0800, Ming Lei wrote:
> 1) some archs support 'nr_cpus=1' for kdump kernel, which is fine, since
> num_possible_cpus becomes 1.
> 
> 2) some archs do not support 'nr_cpus=1', and have to rely on
> 'max_cpus=1', so num_possible_cpus isn't changed, and kernel just boots
> with single online cpu. That causes trouble because blk-mq limits single
> queue.

And we need to fix case 2.  We need to drop the is_kdump support, and
if they want to force less cpus they need to make nr_cpus=1 work.



From zohar at linux.ibm.com  Fri Aug 11 06:14:16 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Fri, 11 Aug 2023 09:14:16 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
Message-ID: <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>

Hi Sush, Tushar,

On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
> ================================================
> | A. Problem Statement                         |
> ================================================
> Depending on the IMA policy, the IMA log can consume a lot of Kernel 
> memory on
> the device.  For instance, the events for the following IMA policy 
> entries may
> need to be measured in certain scenarios, but they can also lead to a 
> verbose
> IMA log when the device is running for a long period of time.
> ?????????????????????????????????????????
> ?# PROC_SUPER_MAGIC                     ?
> ?measure fsmagic=0x9fa0                 ?
> ?# SYSFS_MAGIC                          ?
> ?measure fsmagic=0x62656572             ?
> ?# DEBUGFS_MAGIC                        ?
> ?measure fsmagic=0x64626720             ?
> ?# TMPFS_MAGIC                          ?
> ?measure fsmagic=0x01021994             ?
> ?# RAMFS_MAGIC                          ?
> ?measure fsmagic=0x858458f6             ?
> ?# SECURITYFS_MAGIC                     ?
> ?measure fsmagic=0x73636673             ?
> ?# OVERLAYFS_MAGIC                      ?
> ?measure fsmagic=0x794c7630             ?
> ?# log, audit or tmp files              ?
> ?measure obj_type=var_log_t             ?
> ?measure obj_type=auditd_log_t          ?
> ?measure obj_type=tmp_t                 ?
> ?????????????????????????????????????????
> 
> Secondly, certain devices are configured to take Kernel updates using Kexec
> soft-boot.  The IMA log from the previous Kernel gets carried over and the
> Kernel memory consumption problem worsens when such devices undergo multiple
> Kexec soft-boots over a long period of time.
> 
> The above two scenarios can cause IMA log to grow and consume Kernel memory.
> 
> In addition, a large IMA log can add pressure on the network bandwidth when
> the attestation client sends it to remote-attestation-service.
> 
> Truncating IMA log to reclaim memory is not feasible, since it makes the 
> log go
> out of sync with the TPM PCR quote making remote attestation fail.
> 
> A sophisticated solution is required which will help relieve the memory
> pressure on the device and continue supporting remote attestation without
> disruptions.

If the problem is kernel memory, then using a single tmpfs file has
already been proposed [1].  As entries are added to the measurement
list, they are copied to the tmpfs file and removed from kernel memory.
Userspace would still access the measurement list via the existing
securityfs file.

The IMA measurement list is a sequential file, allowing it to be read
from an offset.  How much or how little of the measuremnt list is read
by the attestation client and sent to the attestation server is up to
the attestation client/server.

If the problem is not kernel memory, but memory pressure in general,
then instead of a tmpfs file, the measurement list could similarly be
copied to a single persistent file [1].

> 
> -------------------------------------------------------------------------------
> ================================================
> | B. Proposed Solution                         |
> ================================================
> In this document, we propose an enhancement to the IMA subsystem to improve
> the long-running performance by snapshotting the IMA log, while still
> providing mechanisms to verify its integrity using the PCR quotes.
> 
> The remainder of the document describes details of the proposed solution 
> in the
> following sub-sections.
>   - High-level Work-flow
>   - Snapshot Triggering Mechanism
>   - Design Choices for Storing Snapshots
>   - Attestation-Client and Remote-Attestation-Service Side Changes
>   - Example Walk-through
>   - Open Questions
> -------------------------------------------------------------------------------
> ================================================
> | B.1 High-level Work-flow                     |
> ================================================
> Pre-requisites:
> - IMA Integrity guarantees are maintained.
> 
> The proposed high level work-flow of IMA log snapshotting is as follows:
> - A user-mode process will trigger the snapshot by opening a file in SysFS
>    say /sys/kernel/security/ima/snapshot (referred to as 
> sysk_ima_snapshot_file
>    here onwards).

Please fix the mailer so that it doesn't wrap sentences.   Adding blank
lines between bullets would improve readability.

> - The Kernel will get the current TPM PCR values and PCR update counter [2]
>    and store them as template data in a new IMA event "snapshot_aggregate".
>    This event will be measured by IMA using critical data measurement
>    functionality [1].  Recording regular IMA events will be paused while
>    "snapshot_aggregate" is being computed using the existing IMA mutex lock.

> - Once the "snapshot_aggregate" is computed and measured in IMA log, the 
> prior
>    IMA events will be made available in the sysk_ima_snapshot_file.

> - The UM process will copy those IMA events from sysk_ima_snapshot_file to a
>    snapshot file on disk chosen by UM (referred to as UM_snapshot_file here
>    onwards).  The location, file-system type, access permissions etc. of the
>    UM_snapshot_file would be controlled by UM process itself.

> - Once UM is done copying the IMA events from sysk_ima_snapshot_file to
>    UM_snapshot_file, it will indicate to the Kernel that the snapshot can be
>    finalized by triggering a write with any data to the 
> sysk_ima_snapshot_file.

>    UM process cannot prevent the IMA log purge operation after this point.
> - The Kernel will truncate the current IMA log and and clear HTable up 
> to the
>    "snapshot_aggregate" marker.

> - The Kernel will measure the PCR update counter as part of measuring
>    snapshot_aggregate, so that it can be used by the remote attestation 
> service
>    for detecting missing events.

> - UM can prevent the IMA log purge by closing the sysk_ima_snapshot_file
>    without performing a write operation on it.  In this case, while the
>    "snapshot_aggregate" marker may still be in the log, the event can be 
> ignored
>    since the previous entries in the IMA log will not be purged.
> 
> Note:
> - This work-flow should work when interleaved with Kexec 'load' and 
> 'execute'
>    events and should not cause IMA log + snapshot to go out of sync with PCR
>    quotes. The implementation details are omitted from this document for
>    brevity.

This design seems overly complex and requires synchronization between
the "snapshot" record and exporting the records from the measurement
list.  None of this would be necessary if the measurements were copied
from kernel memory to a backing file (e.g. tmpfs), as described in [1].

What is the real problem - kernel memory pressure, memory pressure in
general, or disk space?  Is the intention to remove or offload the
exported measurements?

Concerns:
- Pausing extending the measurement list.

[1] 
https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t

-- 
thanks,

Mimi



From radhey.shyam.pandey at amd.com  Fri Aug 11 06:27:11 2023
From: radhey.shyam.pandey at amd.com (Pandey, Radhey Shyam)
Date: Fri, 11 Aug 2023 13:27:11 +0000
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64
 platform
In-Reply-To: <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>
Message-ID: <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>

> -----Original Message-----
> From: bhe at redhat.com <bhe at redhat.com>
> Sent: Wednesday, August 9, 2023 7:42 AM
> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>;
> piliu at redhat.com
> Cc: kexec at lists.infradead.org; linux-kernel at vger.kernel.org
> Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> arm64 platform
> 
> On 08/08/23 at 07:17pm, Pandey, Radhey Shyam wrote:
> > Hi,
> >
> > I am trying to bring up kdump on arm64 platform[1]. But I get "Cannot get
> kernel _text symbol address".
> >
> > Is there some Dump-capture kernel config options that I am missing?
> >
> > FYI, copied below complete kexec debug log.
> >
> > [1]: https://www.xilinx.com/products/boards-and-kits/vck190.html
> 
> Your description isn't clear. You saw the printing, then your kdump kernel
> loading succeeded or not?
> 
> If no, have you tried applying Pingfan's patchset and still saw the issue?
> 
> [PATCHv7 0/5] arm64: zboot support
> https://lore.kernel.org/all/20230803024152.11663-1-piliu at redhat.com/T/#u

I was able to proceed further with loading with crash kernel on triggering system crash.
echo c > /proc/sysrq-trigger

But when I copy /proc/vmcore it throws memory abort. Also I see size of /proc/vmcore really huge (18446603353488633856).
Any possible guess on what could be wrong?


[   80.733523] Starting crashdump kernel...
[   80.737435] Bye!
[    0.000000] Booting Linux on physical CPU 0x0000000001 [0x410fd083]
[    0.000000] Linux version 6.5.0-rc4-ge28001fb4e07 (radheys at xhdradheys41) (aarch64-xilinx-linux-gcc.real (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39.0.20220819) #23 SMP Fri Aug 11 16:25:34 IST 2023
<snip>



xilinx-vck190-20232:/run/media/mmcblk0p1# cat /proc/meminfo | head
MemTotal:        2092876 kB
MemFree:         1219928 kB
MemAvailable:    1166004 kB
Buffers:              32 kB
Cached:           756952 kB
SwapCached:            0 kB
Active:             1480 kB
Inactive:          24164 kB
Active(anon):       1452 kB
Inactive(anon):    24160 kB
xilinx-vck190-20232:/run/media/mmcblk0p1# cp /proc/vmcore dump     
[  975.284865] Unable to handle kernel level 3 address size fault at virtual address ffff80008d7cf000
[  975.293871] Mem abort info:
[  975.296669]   ESR = 0x0000000096000003
[  975.300425]   EC = 0x25: DABT (current EL), IL = 32 bits
[  975.305738]   SET = 0, FnV = 0
[  975.308788]   EA = 0, S1PTW = 0
[  975.311925]   FSC = 0x03: level 3 address size fault
[  975.316888] Data abort info:
[  975.319763]   ISV = 0, ISS = 0x00000003, ISS2 = 0x00000000
[  975.325245]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[  975.330292]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[  975.335599] swapper pgtable: 4k pages, 48-bit VAs, pgdp=000005016ef6b000
[  975.342297] [ffff80008d7cf000] pgd=10000501eddfe003, p4d=10000501eddfe003, pud=10000501eddfd003, pmd=100005017b695003, pte=00687fff84000703
[  975.354827] Internal error: Oops: 0000000096000003 [#4] SMP
[  975.360392] Modules linked in:
3  975.
63440] CBPrUo:a d0c aPID: 664 Comm: cp Tainted: G      D            6.5.0-rc4-ge28001fb4e07 #23
[  975.372822] Hardware name: Xilinx Versal vck190 Eval board revA (DT)
[  975.379165] pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  975.386119] pc : __memcpy+0x110/0x230
[  975.389783] lr : _copy_to_iter+0x3d8/0x4d0
[  975.393874] sp : ffff80008dc939a0
[  975.397178] x29: ffff80008dc939a0 x28: ffff05013c1bea30 x27: 0000000000001000
[  975.404309] x26: 0000000000001000 x25: 0000000000001000 x24: ffff80008d7cf000
[  975.411440] x23: 0000040000000000 x22: ffff80008dc93ba0 x21: 0000000000001000
[  975.418570] x20: ffff000000000000 x19: 0000000000001000 x18: 0000000000000000
[  975.425699] x17: 0000000000000000 x16: 0000000000000000 x15: 0140000000000000
[  975.432829] x14: ffff8500a9919000 x13: 0040000000000001 x12: 0000fffef6831000
[  975.439958] x11: ffff80008d9cf000 x10: 0000000000000000 x9 : 0000000000000000
[  975.447088] x8 : ffff80008d7d0000 x7 : ffff0501addfd358 x6 : 0400000000000001
[  975.454217] x5 : ffff0501370e9000 x4 : ffff80008d7d0000 x3 : 0000000000000000
[  975.461346] x2 : 0000000000001000 x1 : ffff80008d7cf000 x0 : ffff0501370e8000
[  975.468476] Call trace:
[  975.470912]  __memcpy+0x110/0x230
[  975.474221]  copy_oldmem_page+0x70/0xac
[  975.478050]  read_from_oldmem.part.0+0x120/0x188
[  975.482663]  read_vmcore+0x14c/0x238
[  975.486231]  proc_reg_read_iter+0x84/0xd8
[  975.490233]  copy_splice_read+0x160/0x288
[  975.494236]  vfs_splice_read+0xac/0x10c
[  975.498063]  splice_direct_to_actor+0xa4/0x26c
[  975.502498]  do_splice_direct+0x90/0xdc
[  975.506325]  do_sendfile+0x344/0x454
[  975.509892]  __arm64_sys_sendfile64+0x134/0x140
[  975.514415]  invoke_syscall+0x54/0x124
[  975.518157]  el0_svc_common.constprop.0+0xc4/0xe4
[  975.522854]  do_el0_svc+0x38/0x98
[  975.526162]  el0_svc+0x2c/0x84
[  975.529211]  el0t_64_sync_handler+0x100/0x12c
[  975.533562]  el0t_64_sync+0x190/0x194
[  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3 (a940342c) 
[  975.543302] ---[ end trace 0000000000000000 ]---
t message from systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):

kernel[539]: [  975.354827] Internal error: Oops: 0000000096000003 [#4] SMP


Broadcast message from systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):

kernel[539]: [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3 (a940342c)

Segmentation fault
xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lrth /proc/vmcore 
-r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lh /proc/vmcore
-r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
xilinx-vck190-20232:/run/media/mmcblk0p1# ls -l /proc/vmcore
-r--------    1 root     root     18446603353488633856 Nov  8 14:05 /proc/vmcore

Thanks,
Radhey


From tusharsu at linux.microsoft.com  Fri Aug 11 08:48:25 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Fri, 11 Aug 2023 08:48:25 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <8ccaec30bf85cfbf4415bbafa22646a62e753840.camel@HansenPartnership.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <8ad131f35c33cf10788344be6c981473971f9c1c.camel@HansenPartnership.com>
 <abe53dde-9a83-81fd-422d-babf4587c545@linux.ibm.com>
 <350ecdcbf7796f488807fcd7983414a02dd71be4.camel@HansenPartnership.com>
 <04fb2fe5-9ebe-b35f-bdde-6ef22786438f@linux.ibm.com>
 <a522b17a536ea87a6a4c2faf95583ae3b7b74a26.camel@HansenPartnership.com>
 <5cb03349-7a32-8f74-f2a1-ff3c6247c1ef@linux.microsoft.com>
 <8ccaec30bf85cfbf4415bbafa22646a62e753840.camel@HansenPartnership.com>
Message-ID: <72e39852-1ff1-c7f6-ac7e-593e8142dbe8@linux.microsoft.com>



On 8/10/23 04:43, James Bottomley wrote:
> On Wed, 2023-08-09 at 21:43 -0700, Tushar Sugandhi wrote:
>> On 8/8/23 14:41, James Bottomley wrote:
>>> On Tue, 2023-08-08 at 16:09 -0400, Stefan Berger wrote:
> [...]
>>>>  ? at this point doesn't seem necessary since one presumably can
>>>> verify the log and PCR states at the end with the 'regular'
>>>> quote.
>>>   
>>> I don't understand this.? A regular quote is a signature over PCR
>>> state by an AK.? The point about saving the AK in the log for the
>>> original is that if the *kernel* truncates the log and saves it to
>>> a file, it needs to generate both the AK and the quote for the top
>>> of the file shard. That means the AK/EK binding is unverified, but
>>> can be verified by loading the AK and running the usual tests,
>>> which can only be done if you have the loadable AK, which is why
>>> you need it as part of the log saving proposal.
>>   
>> I had this question about the usability of AK/EK in this
>> context. Although AK/EK + PCR quote is needed to verify the snapshot
>> shards / IMA logs are not tampered with, I am still not sure why
>> AK/EK needs to be part of the shard/IMA log. The client sending AK/EK
>> to attestation service separately would still serve the purpose,
>> right?
> 
> Well, the EK doesn't need to be part of the log: it's just a permanent
> part of the TPM identity.  To verify the log, you need access to the
> TPM that was used to create it, so that's the point at which you get
> the EK.
> 
Agreed. EK is part of TPM identity. But to verify the log,
you don?t need to have physical access to the TPM. You need to have
access to just public part of EK and AK/AIK certs (TPM on the system
would sign the quote using the private AK).
I believe you already know this, just stating for the sake of
completing the conversation. :)
> An AK is simply a TPM generated signing key (meaning the private part
> of the key is secured by the TPM and known to no-one else).  In the
> literature a TPM generated signing key doesn't become an Attestation
> Key until it's been verified using an EK property (either a certify for
> a signing EK or a make/activate credential round trip for the more
> usual encryption EK.
> 
Yes. That aligns with my understanding of EK/AK in general.
Thanks for describing.

> So the proposal is for each quote that's used to verify a log shard is
> that the TPM simply generate a random signing key and use that to sign
I believe you are suggesting creating a new AK each time you
want to sign a PCR quote. It is doable in TPM 2.0, and it provides
benefits like privacy and untraceability. But it comes with it?s own
costs ? cost of generating new AK each time you want to sign,
maintaining mapping of AK and it?s signed quotes, maintaining
multiple public AK certs etc.

> the quote.  You need to save the TPM form of the generated key so it
> can be loaded later and the reason for that is you can do the EK
> verification at any time after the quote was given by loading the saved
> key and running the verification protocol.  In the normal attestation
> you do the EK verification of the AK *before* the quote, but there's no
> property of the quote that depends on this precedence provided you do
> the quote with a TPM generated signing key.
Yes.

> 
> The underlying point is that the usual way an EK verifies an AK
> requires a remote observer, which the kernel won't have, so the kernel
Agreed.

> must do all its stuff locally (generate key, get quote) and then at
I believe the Kernel doesn?t have to generate key while
taking the snapshot. In the current proposal, Kernel can simply get
the (unsigned) PCR quote and log it in IMA log as part of the
snapshot_aggregate event. We don?t need to sign the quote while
logging it in the IMA log as snapshot_aggregate. And the act of
logging that event in IMA log extends the PCR bank. Sometime later,
when a remote observer wants to validate the log ? it can do it by
comparing against the PCR quote that was signed at that point.

> some point later the system can become remote connected and prove to
> whatever external entity that the log shard is valid.  So we have to
> have all the components necessary for that proof: the log shard, the
> quote and the TPM form of the AK.
> 
>> For instance, PCR quotes will be signed by AK. So as long as the
>> verifier trusts the AK/EK,
> 
> Right, but if you're sharding a log, the kernel doesn't know if a
> verifier has been in contact yet.  The point of the protocol above is
> to make that not matter.  The verifier can contact the system after the
> log has been saved and the verification will still work.
> 
The Kernel doesn?t need to know. And it still doesn?t matter.
The benefit of our approach is the PCR values that represent the
previous snapshot(shard) is now logged in the IMA log as
snapshot_aggregate, and the PCRs are extended again as part of
logging that event in IMA log.

>>   it can verify the quotes are not tampered with.
>> Replaying IMA log/snapshot can produce the PCR quotes which can be
>> matched with signed PCR quotes. If they match, then the verifier can
>> conclude that the IMA log is not tampered with. So AK doesn't need to
>> be part of the log/snapshot.
> 
> Only if the system is currently in contact with the verifier and the
> verifier has created the AK.  That may not have happened.
> 
Hope my above explanation addresses this point.

>> BTW, in this proposal, kernel is truncating the log and passing the
>> truncated buffer to UM.? UM client need to save it to the disk
>> location of it's choice.
> 
> Yes, but I was assuming tampering with or discarding the log file would
> be treated in exactly the same way as an in-kernel IMA log tamper.
> 
Hope my above explanation addresses this point.

~Tushar

> James


From tusharsu at linux.microsoft.com  Fri Aug 11 08:57:24 2023
From: tusharsu at linux.microsoft.com (Tushar Sugandhi)
Date: Fri, 11 Aug 2023 08:57:24 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <011d8a79-236f-dc20-08fc-b5da7dd1d5a7@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <b538f7d2-5a04-46d0-3792-a18653230a95@linux.microsoft.com>
 <011d8a79-236f-dc20-08fc-b5da7dd1d5a7@linux.ibm.com>
Message-ID: <aba709f8-a808-9aa4-8322-430e3997f686@linux.microsoft.com>



On 8/10/23 07:12, Stefan Berger wrote:
> 
> 
> On 8/9/23 21:15, Tushar Sugandhi wrote:
>> Thanks a lot Stefan for looking into this proposal,
>> and providing your feedback. We really appreciate it.
>>
>> On 8/7/23 15:49, Stefan Berger wrote:
>>>
>>>
>>> On 8/1/23 17:21, James Bottomley wrote:
>>>> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>>>> [...]
>>>>> Truncating IMA log to reclaim memory is not feasible, since it makes
>>>>> the log go out of sync with the TPM PCR quote making remote
>>>>> attestation fail.
>>>>
>>>> This assumption isn't entirely true.? It's perfectly possible to shard
>>>> an IMA log using two TPM2_Quote's for the beginning and end PCR values
>>>> to validate the shard.? The IMA log could be truncated in the same way
>>>> (replace the removed part of the log with a TPM2_Quote and AK, so the
>>>> log still validates from the beginning quote to the end).
>>>>
>>>> If you use a TPM2_Quote mechanism to save the log, all you need to do
>>>> is have the kernel generate the quote with an internal AK.? You can
>>>> keep a record of the quote and the AK at the beginning of the truncated
>>>> kernel log.? If the truncated entries are saved in a file shard it
>>>
>>> The truncation seems dangerous to me. Maybe not all the scenarios 
>>> with an attestation
>>> client (client = reading logs and quoting) are possible then anymore, 
>>> such as starting
>>> an attestation client only after truncation but a verifier must have 
>>> witnessed the
>>> system's PCRs and log state before the truncation occurred.
>> You are correct that truncation on it?s own is dangerous. It needs to be
>> accompanied by (a) saving the IMA log data to disk as snapshots, (b) 
>> adding the
>> necessary TPM PCR quotes to the current IMA log (as James mentioned 
>> above),
>> (c) attestation clients having an ability to send the past snapshots 
>> to the
>> remote-attestation-service (verifiers), (d) and verifiers having an 
>> ability
>> to use the snapshots along with current IMA logs for the purpose of 
>> attestation.
>> All these points are explained in the original RFC email in sections 
>> B.1 through B.5 [1].
> 
> I read it.
> 
> Maybe you have dismissed the PCR update counter already...
> I am not sure what the PCR update counter is supposed to help with. It 
> won't allow you to detect
> missing log events but rather will confuse anyone looking at it when my 
> application extends PCR 12
> for example, which also affects the update counter. It's a global 
> counter that increases with every
> PCR extension (except PCR 16, 21, 22, 23) and if used as proposed would 
> prevent any application from
> extending PCRs.
> 
> https://github.com/stefanberger/libtpms/blob/master/src/tpm2/PCR.c#L667
> https://github.com/stefanberger/libtpms/blob/master/src/tpm2/PCR.c#L629
> https://github.com/stefanberger/libtpms/blob/master/src/tpm2/PCR.c#L161
> 
> 
Agree with your point about TPM PCR update counter Stefan.
I will bring it up in the update counter patch series discussion [1].

[1] 
https://patchwork.kernel.org/project/linux-integrity/cover/20230801181917.8535-1-tusharsu at linux.microsoft.com/ 


> The shards should will need to be written into some sort of standard 
> location or a config file needs to
> be defined, so that everyone knows where to find them and how they are 
> named.
> 
We thought about well known standard location earlier.
Letting the Kernel choose the name/location of the snapshot
file comes with its own complexity. Our initial stance is we don?t
want to handle that at Kernel level, and let the UM client choose
the location/naming of the snapshot files. But we are happy to
reconsider if the community requests it.
> 
>>>
>>> I think an ima-buf (or similar) log entry in IMA log would have to 
>>> appear at the beginning of the
>>> truncated log stating the value of all PCRs that IMA touched 
>>> (typically only PCR 10
>>> but it can be others). The needs to be done since the quote itself 
>>> doesn't
>>> provide the state of the individual PCRs. This would at least allow 
>>> an attestation
>>> client to re-read the log from the beginning (when it is re-start or 
>>> started for the
>>> first time after the truncation). 
>> ??Agreed. See the description of snapshot_aggregate in Section B.5 in the
>> original RFC email [1].
>>> However, this alone (without the
>>> internal AK quoting the old state) could lead to abuse where I could 
>>> create totally
>>> fake IMA logs stating the state of the PCRs at the beginning (so the 
>>> verifier
>>> syncs its internal PCR state to this state). 
>> Yes, the PCR quotes sent to the verifier must be signed by the AK that
>> is trusted by the verifier. That assumption is true regardless of IMA log
>> snapshotting feature.
>>> Further, even with the AK-quote that
>>> you propose I may be able to create fake logs and trick a verifier into
>>> trusting the machine IFF it doesn't know what kernel this system was 
>>> booted with
>>> that I may have hacked to provide a fake AK-quote that just happens 
>>> to match the
>>> PCR state presented at the beginning of the log.
>>>
>> If the Kernel is compromised, then all-bets are off.
>> (Regardless of IMA log snapshotting feature.)
>>> => Can a truncated log be made safe for attestation when the 
>>> attestation starts
>>> only after the truncation occurred?
>>>
>> Yes. If the ?PCR quotes in the snapshot_aggregate event in IMA log?
> 
> PCR quote or 'quotes'? Why multiple?
> 
> Form your proposal but you may have changed your opinion? following what 
> I see in other messages:
> "- The Kernel will get the current TPM PCR values and PCR update counter 
> [2]
>  ?? and store them as template data in a new IMA event 
> "snapshot_aggregate"."
> 
> Afaik TPM quote's don't give you the state of the individual PCR values, 
> therefore
> I would expect to at least find the 'PCR values' of all the PCRs that 
> IMA touched to
> be in the snapshot_aggregate so I can replay all the following events on 
> top of these
> PCR values and come up with the values that were used in the "final PCR 
> quote". This
> is unless you expect the server to take an automatic snapshot of the 
> values of the
> PCRs? that it computed while evaluating the log in case it ever needs to 
> go back.
> 
I meant a single set of PCR values captured when snapshot_aggregate
is logged. Sorry for the confusion.

>> + "replay of rest of the events in IMA log" results in the ?final PCR 
>> quotes?
>> that matches with the ?AK signed PCR quotes? sent by the client, then 
>> the truncated
>> IMA log can be trusted. The verifier can either ?trust? the ?PCR 
>> quotes in the
>> snapshot_aggregate event in IMA log? or it can ask for the (n-1)th 
>> snapshot shard
>> to check the past events.
> 
> For anything regarding determining the 'trustworthiness of a system' one 
> would have to
> be able to go back to the very beginning of the log *or* remember in 
> what state a
> system was when the latest snapshot was taken so that if a restart 
> happens it can resume
> with that assumption about state of trustworthiness and know what the 
> values of the PCRs
> were at that time so it can resume replaying the log (or the server 
> would get these
> values from the log).
> 
Correct. We intend to support the above. I hope our proposal
description captures it. BTW, when you say ?restart?, you mean the UM
process restart, right? Because in case of a Kernel restart
(i.e. cold-boot) the past IMA log (and the TPM state) is lost,
and old snapshots (if any) are useless.

> The AK quotes by the kernel (which adds a 2nd AK key) that James is 
> proposing
> could be useful if the entire log, consisting of multiple shards, is 
> very large and
> cannot be transferred from the client to the server in one go so that 
> the server could
> evaluate the 'final PCR quote' immediately . However, if a client can 
> indicated 'I will
> send more the next time and I have this much more to transfer' and the 
> server allows
> this multiple times (until all the 1MB shards of the 20MB log are 
> transferred) then that
> kernel AK key would not be necessary since presumably the "final PCR 
> quote", created
> by a user space client, would resolve whether the entire log is 
> trustworthy.
> 
See my responses to James today [2]

[2] 
https://lore.kernel.org/all/72e39852-1ff1-c7f6-ac7e-593e8142dbe8 at linux.microsoft.com/
>>
>>> => Even if attestation was occurring 'what' state does an attestation 
>>> server
>>> need to carry around for an attested-to system so that the truncation 
>>> is 'safe'
>>> and I cannot create fake AK-quotes and fake IMA logs with initial PCR 
>>> states?
>> Assuming most of the client devices take a snapshot at specific 
>> checkpoints,
>> the ?PCR quotes in the snapshot_aggregate event in IMA log? will be 
>> the same for them.
>> The remote attestation server will have to remember these golden PCR 
>> quotes.
> 
> I thought maybe 'golden PCR values'... because those let me replay PCR 
> extensions from
> a previous point.
> 
>> It doesn't have to remember the state of each client device.
> 
> Can you give a reason for this? You mean the state doesn't need to be 
> remembered for client
> devices whose log hasn't been truncated?
> 
I meant it doesn?t have to be remembered for each individual
client device. Majority of the client devices will be in one of the few
golden-PCR-states when the snapshots are captured.

~ Tushar

> 
>>> Can I ever restart the client and have it read the truncated log from 
>>> the
>>> beginning and what type of verification needs to happen on the server 
>>> then?
>>>
>> Yes, restarting the client should be possible.
> 
> Yes, this must be possible.
> 
>>> It seems like the server would have to remember the state of the IMA 
>>> PCRs upon
>>> last truncation to detect a possible attack. This would make staring 
>>> to monitor
>>> a system after truncation impossible -- would be good to know these 
>>> details.
>>>
>> The server is not forced to remember the state of IMA PCRs. It can
>> always ask for the last n snapshot files (shards) and replay the 
>> events. Even
>> though the data is truncated from the IMA log, it is not totally lost. 
>> It is
>> simply being transferred to the disk. It is saved by UM as snapshot 
>> files/shards.
>> The goal of IMA snapshotting is to reduce the Kernel memory pressure 
>> on the
>> client devices - to save them from out-of-memory errors which are 
>> harder to manage
>> on long running clients. It comes with a cost of additional work on 
>> the server
>> side to attest those clients.
> 
> Agreed.
>>
>>
>> Being said that, in the current proposal, taking a snapshots is 
>> totally optional
>> and controlled by UM attestation clients. If the 
>> attestation-clients/services are
>> not-ready/don?t-want to take advantage of IMA log snapshotting, they 
>> don?t have to.
> 
> Agreed.
> 
>>
>> No snapshot will be taken, and the client-service can process the 
>> monolithic IMA
>> log just like they do today.
>>
> 
> Agreed.
> 
>> [1] 
>> https://lore.kernel.org/all/c5737141-7827-1c83-ab38-0119dcfea485 at linux.microsoft.com/#t
>>
>>>
>>>
>>>
>>>> should have a beginning and end quote and a record of the AK used.
>>>> Since verifiers like Keylime are already using this beginning and end
>>>> quote for sharded logs, it's the most natural format to feed to
>>>> something externally for verification and it means you don't have to
>>>> invent a new format to do the same thing.
>>>>
>>>> Regards,
>>>>
>>>> James
>>>>
>>>>
>>>> _______________________________________________
>>>> kexec mailing list
>>>> kexec at lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/kexec
>>


From eric.devolder at oracle.com  Fri Aug 11 10:06:37 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:37 -0400
Subject: [PATCH v27 3/8] kexec: exclude elfcorehdr from the segment digest
In-Reply-To: <20230811170642.6696-1-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
Message-ID: <20230811170642.6696-4-eric.devolder@oracle.com>

When a crash kernel is loaded via the kexec_file_load() syscall, the
kernel places the various segments (ie crash kernel, crash initrd,
boot_params, elfcorehdr, purgatory, etc) in memory. For those
architectures that utilize purgatory, a hash digest of the segments
is calculated for integrity checking. The digest is embedded into
the purgatory image prior to placing in memory.

Updates to the elfcorehdr in response to CPU and memory changes
would cause the purgatory integrity checking to fail (at crash time,
and no vmcore created). Therefore, the elfcorehdr segment is
explicitly excluded from the purgatory digest, enabling updates to
the elfcorehdr while also avoiding the need to recompute the hash
digest and reload purgatory.

Suggested-by: Baoquan He <bhe at redhat.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 kernel/kexec_file.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index 453b7a513540..e2ec9d7b9a1f 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -726,6 +726,12 @@ static int kexec_calculate_store_digests(struct kimage *image)
 	for (j = i = 0; i < image->nr_segments; i++) {
 		struct kexec_segment *ksegment;
 
+#ifdef CONFIG_CRASH_HOTPLUG
+		/* Exclude elfcorehdr segment to allow future changes via hotplug */
+		if (j == image->elfcorehdr_index)
+			continue;
+#endif
+
 		ksegment = &image->segment[i];
 		/*
 		 * Skip purgatory as it will be modified once we put digest
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug 11 10:06:38 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:38 -0400
Subject: [PATCH v27 4/8] crash: memory and CPU hotplug sysfs attributes
In-Reply-To: <20230811170642.6696-1-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
Message-ID: <20230811170642.6696-5-eric.devolder@oracle.com>

Introduce the crash_hotplug attribute for memory and CPUs for
use by userspace.  These attributes directly facilitate the udev
rule for managing userspace re-loading of the crash kernel upon
hot un/plug changes.

For memory, expose the crash_hotplug attribute to the
/sys/devices/system/memory directory. For example:

 # udevadm info --attribute-walk /sys/devices/system/memory/memory81
  looking at device '/devices/system/memory/memory81':
    KERNEL=="memory81"
    SUBSYSTEM=="memory"
    DRIVER==""
    ATTR{online}=="1"
    ATTR{phys_device}=="0"
    ATTR{phys_index}=="00000051"
    ATTR{removable}=="1"
    ATTR{state}=="online"
    ATTR{valid_zones}=="Movable"

  looking at parent device '/devices/system/memory':
    KERNELS=="memory"
    SUBSYSTEMS==""
    DRIVERS==""
    ATTRS{auto_online_blocks}=="offline"
    ATTRS{block_size_bytes}=="8000000"
    ATTRS{crash_hotplug}=="1"

For CPUs, expose the crash_hotplug attribute to the
/sys/devices/system/cpu directory. For example:

 # udevadm info --attribute-walk /sys/devices/system/cpu/cpu0
  looking at device '/devices/system/cpu/cpu0':
    KERNEL=="cpu0"
    SUBSYSTEM=="cpu"
    DRIVER=="processor"
    ATTR{crash_notes}=="277c38600"
    ATTR{crash_notes_size}=="368"
    ATTR{online}=="1"

  looking at parent device '/devices/system/cpu':
    KERNELS=="cpu"
    SUBSYSTEMS==""
    DRIVERS==""
    ATTRS{crash_hotplug}=="1"
    ATTRS{isolated}==""
    ATTRS{kernel_max}=="8191"
    ATTRS{nohz_full}=="  (null)"
    ATTRS{offline}=="4-7"
    ATTRS{online}=="0-3"
    ATTRS{possible}=="0-7"
    ATTRS{present}=="0-3"

With these sysfs attributes in place, it is possible to efficiently
instruct the udev rule to skip crash kernel reloading for kernels
configured with crash hotplug support.

For example, the following is the proposed udev rule change for RHEL
system 98-kexec.rules (as the first lines of the rule file):

 # The kernel updates the crash elfcorehdr for CPU and memory changes
 SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
 SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

When examined in the context of 98-kexec.rules, the above rules
test if crash_hotplug is set, and if so, the userspace initiated
unload-then-reload of the crash kernel is skipped.

CPU and memory checks are separated in accordance with
CONFIG_HOTPLUG_CPU and CONFIG_MEMORY_HOTPLUG kernel config options.
If an architecture supports, for example, memory hotplug but not
CPU hotplug, then the /sys/devices/system/memory/crash_hotplug
attribute file is present, but the /sys/devices/system/cpu/crash_hotplug
attribute file will NOT be present. Thus the udev rule skips
userspace processing of memory hot un/plug events, but the udev
rule will evaluate false for CPU events, thus allowing userspace to
process CPU hot un/plug events (ie the unload-then-reload of the kdump
capture kernel).

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 Documentation/ABI/testing/sysfs-devices-memory |  8 ++++++++
 .../ABI/testing/sysfs-devices-system-cpu       |  8 ++++++++
 .../admin-guide/mm/memory-hotplug.rst          |  8 ++++++++
 Documentation/core-api/cpu_hotplug.rst         | 18 ++++++++++++++++++
 drivers/base/cpu.c                             | 13 +++++++++++++
 drivers/base/memory.c                          | 13 +++++++++++++
 include/linux/kexec.h                          |  8 ++++++++
 7 files changed, 76 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-devices-memory b/Documentation/ABI/testing/sysfs-devices-memory
index d8b0f80b9e33..a95e0f17c35a 100644
--- a/Documentation/ABI/testing/sysfs-devices-memory
+++ b/Documentation/ABI/testing/sysfs-devices-memory
@@ -110,3 +110,11 @@ Description:
 		link is created for memory section 9 on node0.
 
 		/sys/devices/system/node/node0/memory9 -> ../../memory/memory9
+
+What:		/sys/devices/system/memory/crash_hotplug
+Date:		Aug 2023
+Contact:	Linux kernel mailing list <linux-kernel at vger.kernel.org>
+Description:
+		(RO) indicates whether or not the kernel directly supports
+		modifying the crash elfcorehdr for memory hot un/plug and/or
+		on/offline changes.
diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
index 77942eedf4f6..b52564de2b18 100644
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -687,3 +687,11 @@ Description:
 		(RO) the list of CPUs that are isolated and don't
 		participate in load balancing. These CPUs are set by
 		boot parameter "isolcpus=".
+
+What:		/sys/devices/system/cpu/crash_hotplug
+Date:		Aug 2023
+Contact:	Linux kernel mailing list <linux-kernel at vger.kernel.org>
+Description:
+		(RO) indicates whether or not the kernel directly supports
+		modifying the crash elfcorehdr for CPU hot un/plug and/or
+		on/offline changes.
diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst
index 1b02fe5807cc..eb99d79223a3 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -291,6 +291,14 @@ The following files are currently defined:
 		       Availability depends on the CONFIG_ARCH_MEMORY_PROBE
 		       kernel configuration option.
 ``uevent``	       read-write: generic udev file for device subsystems.
+``crash_hotplug``      read-only: when changes to the system memory map
+		       occur due to hot un/plug of memory, this file contains
+		       '1' if the kernel updates the kdump capture kernel memory
+		       map itself (via elfcorehdr), or '0' if userspace must update
+		       the kdump capture kernel memory map.
+
+		       Availability depends on the CONFIG_MEMORY_HOTPLUG kernel
+		       configuration option.
 ====================== =========================================================
 
 .. note::
diff --git a/Documentation/core-api/cpu_hotplug.rst b/Documentation/core-api/cpu_hotplug.rst
index e6f5bc39cf5c..d6d470d7dda0 100644
--- a/Documentation/core-api/cpu_hotplug.rst
+++ b/Documentation/core-api/cpu_hotplug.rst
@@ -741,6 +741,24 @@ will receive all events. A script like::
 
 can process the event further.
 
+When changes to the CPUs in the system occur, the sysfs file
+/sys/devices/system/cpu/crash_hotplug contains '1' if the kernel
+updates the kdump capture kernel list of CPUs itself (via elfcorehdr),
+or '0' if userspace must update the kdump capture kernel list of CPUs.
+
+The availability depends on the CONFIG_HOTPLUG_CPU kernel configuration
+option.
+
+To skip userspace processing of CPU hot un/plug events for kdump
+(i.e. the unload-then-reload to obtain a current list of CPUs), this sysfs
+file can be used in a udev rule as follows:
+
+ SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
+
+For a CPU hot un/plug event, if the architecture supports kernel updates
+of the elfcorehdr (which contains the list of CPUs), then the rule skips
+the unload-then-reload of the kdump capture kernel.
+
 Kernel Inline Documentations Reference
 ======================================
 
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 52df435eecf8..5fb0c89e1ad5 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -282,6 +282,16 @@ static ssize_t print_cpus_nohz_full(struct device *dev,
 static DEVICE_ATTR(nohz_full, 0444, print_cpus_nohz_full, NULL);
 #endif
 
+#ifdef CONFIG_CRASH_HOTPLUG
+static ssize_t crash_hotplug_show(struct device *dev,
+				     struct device_attribute *attr,
+				     char *buf)
+{
+	return sysfs_emit(buf, "%d\n", crash_hotplug_cpu_support());
+}
+static DEVICE_ATTR_ADMIN_RO(crash_hotplug);
+#endif
+
 static void cpu_device_release(struct device *dev)
 {
 	/*
@@ -469,6 +479,9 @@ static struct attribute *cpu_root_attrs[] = {
 #ifdef CONFIG_NO_HZ_FULL
 	&dev_attr_nohz_full.attr,
 #endif
+#ifdef CONFIG_CRASH_HOTPLUG
+	&dev_attr_crash_hotplug.attr,
+#endif
 #ifdef CONFIG_GENERIC_CPU_AUTOPROBE
 	&dev_attr_modalias.attr,
 #endif
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index b456ac213610..15bb416e58ce 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -490,6 +490,16 @@ static ssize_t auto_online_blocks_store(struct device *dev,
 
 static DEVICE_ATTR_RW(auto_online_blocks);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+#include <linux/kexec.h>
+static ssize_t crash_hotplug_show(struct device *dev,
+				       struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%d\n", crash_hotplug_memory_support());
+}
+static DEVICE_ATTR_RO(crash_hotplug);
+#endif
+
 /*
  * Some architectures will have custom drivers to do this, and
  * will not need to do it from userspace.  The fake hot-add code
@@ -889,6 +899,9 @@ static struct attribute *memory_root_attrs[] = {
 
 	&dev_attr_block_size_bytes.attr,
 	&dev_attr_auto_online_blocks.attr,
+#ifdef CONFIG_CRASH_HOTPLUG
+	&dev_attr_crash_hotplug.attr,
+#endif
 	NULL
 };
 
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index b9903dd48e24..6a8a724ac638 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -501,6 +501,14 @@ static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) {
 static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
 #endif
 
+#ifndef crash_hotplug_cpu_support
+static inline int crash_hotplug_cpu_support(void) { return 0; }
+#endif
+
+#ifndef crash_hotplug_memory_support
+static inline int crash_hotplug_memory_support(void) { return 0; }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug 11 10:06:35 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:35 -0400
Subject: [PATCH v27 1/8] crash: move a few code bits to setup support of crash hotplug
In-Reply-To: <20230811170642.6696-1-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
Message-ID: <20230811170642.6696-2-eric.devolder@oracle.com>

The crash hotplug support leans on the work for the kexec_file_load()
syscall. To also support the kexec_load() syscall, a few bits of code
need to be move outside of CONFIG_KEXEC_FILE. As such, these bits are
moved out of kexec_file.c and into a common location crash_core.c.

No functionality change intended.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 include/linux/kexec.h |  30 +++----
 kernel/crash_core.c   | 182 ++++++++++++++++++++++++++++++++++++++++++
 kernel/kexec_file.c   | 181 -----------------------------------------
 3 files changed, 197 insertions(+), 196 deletions(-)

diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 22b5cd24f581..811a90e09698 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -105,6 +105,21 @@ struct compat_kexec_segment {
 };
 #endif
 
+/* Alignment required for elf header segment */
+#define ELF_CORE_HEADER_ALIGN   4096
+
+struct crash_mem {
+	unsigned int max_nr_ranges;
+	unsigned int nr_ranges;
+	struct range ranges[];
+};
+
+extern int crash_exclude_mem_range(struct crash_mem *mem,
+				   unsigned long long mstart,
+				   unsigned long long mend);
+extern int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
+				       void **addr, unsigned long *sz);
+
 #ifdef CONFIG_KEXEC_FILE
 struct purgatory_info {
 	/*
@@ -230,21 +245,6 @@ static inline int arch_kexec_locate_mem_hole(struct kexec_buf *kbuf)
 }
 #endif
 
-/* Alignment required for elf header segment */
-#define ELF_CORE_HEADER_ALIGN   4096
-
-struct crash_mem {
-	unsigned int max_nr_ranges;
-	unsigned int nr_ranges;
-	struct range ranges[];
-};
-
-extern int crash_exclude_mem_range(struct crash_mem *mem,
-				   unsigned long long mstart,
-				   unsigned long long mend);
-extern int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
-				       void **addr, unsigned long *sz);
-
 #ifndef arch_kexec_apply_relocations_add
 /*
  * arch_kexec_apply_relocations_add - apply relocations of type RELA
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 90ce1dfd591c..b7c30b748a16 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -10,6 +10,7 @@
 #include <linux/utsname.h>
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
+#include <linux/kexec.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -314,6 +315,187 @@ static int __init parse_crashkernel_dummy(char *arg)
 }
 early_param("crashkernel", parse_crashkernel_dummy);
 
+int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
+			  void **addr, unsigned long *sz)
+{
+	Elf64_Ehdr *ehdr;
+	Elf64_Phdr *phdr;
+	unsigned long nr_cpus = num_possible_cpus(), nr_phdr, elf_sz;
+	unsigned char *buf;
+	unsigned int cpu, i;
+	unsigned long long notes_addr;
+	unsigned long mstart, mend;
+
+	/* extra phdr for vmcoreinfo ELF note */
+	nr_phdr = nr_cpus + 1;
+	nr_phdr += mem->nr_ranges;
+
+	/*
+	 * kexec-tools creates an extra PT_LOAD phdr for kernel text mapping
+	 * area (for example, ffffffff80000000 - ffffffffa0000000 on x86_64).
+	 * I think this is required by tools like gdb. So same physical
+	 * memory will be mapped in two ELF headers. One will contain kernel
+	 * text virtual addresses and other will have __va(physical) addresses.
+	 */
+
+	nr_phdr++;
+	elf_sz = sizeof(Elf64_Ehdr) + nr_phdr * sizeof(Elf64_Phdr);
+	elf_sz = ALIGN(elf_sz, ELF_CORE_HEADER_ALIGN);
+
+	buf = vzalloc(elf_sz);
+	if (!buf)
+		return -ENOMEM;
+
+	ehdr = (Elf64_Ehdr *)buf;
+	phdr = (Elf64_Phdr *)(ehdr + 1);
+	memcpy(ehdr->e_ident, ELFMAG, SELFMAG);
+	ehdr->e_ident[EI_CLASS] = ELFCLASS64;
+	ehdr->e_ident[EI_DATA] = ELFDATA2LSB;
+	ehdr->e_ident[EI_VERSION] = EV_CURRENT;
+	ehdr->e_ident[EI_OSABI] = ELF_OSABI;
+	memset(ehdr->e_ident + EI_PAD, 0, EI_NIDENT - EI_PAD);
+	ehdr->e_type = ET_CORE;
+	ehdr->e_machine = ELF_ARCH;
+	ehdr->e_version = EV_CURRENT;
+	ehdr->e_phoff = sizeof(Elf64_Ehdr);
+	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
+	ehdr->e_phentsize = sizeof(Elf64_Phdr);
+
+	/* Prepare one phdr of type PT_NOTE for each present CPU */
+	for_each_present_cpu(cpu) {
+		phdr->p_type = PT_NOTE;
+		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
+		phdr->p_offset = phdr->p_paddr = notes_addr;
+		phdr->p_filesz = phdr->p_memsz = sizeof(note_buf_t);
+		(ehdr->e_phnum)++;
+		phdr++;
+	}
+
+	/* Prepare one PT_NOTE header for vmcoreinfo */
+	phdr->p_type = PT_NOTE;
+	phdr->p_offset = phdr->p_paddr = paddr_vmcoreinfo_note();
+	phdr->p_filesz = phdr->p_memsz = VMCOREINFO_NOTE_SIZE;
+	(ehdr->e_phnum)++;
+	phdr++;
+
+	/* Prepare PT_LOAD type program header for kernel text region */
+	if (need_kernel_map) {
+		phdr->p_type = PT_LOAD;
+		phdr->p_flags = PF_R|PF_W|PF_X;
+		phdr->p_vaddr = (unsigned long) _text;
+		phdr->p_filesz = phdr->p_memsz = _end - _text;
+		phdr->p_offset = phdr->p_paddr = __pa_symbol(_text);
+		ehdr->e_phnum++;
+		phdr++;
+	}
+
+	/* Go through all the ranges in mem->ranges[] and prepare phdr */
+	for (i = 0; i < mem->nr_ranges; i++) {
+		mstart = mem->ranges[i].start;
+		mend = mem->ranges[i].end;
+
+		phdr->p_type = PT_LOAD;
+		phdr->p_flags = PF_R|PF_W|PF_X;
+		phdr->p_offset  = mstart;
+
+		phdr->p_paddr = mstart;
+		phdr->p_vaddr = (unsigned long) __va(mstart);
+		phdr->p_filesz = phdr->p_memsz = mend - mstart + 1;
+		phdr->p_align = 0;
+		ehdr->e_phnum++;
+		pr_debug("Crash PT_LOAD ELF header. phdr=%p vaddr=0x%llx, paddr=0x%llx, sz=0x%llx e_phnum=%d p_offset=0x%llx\n",
+			phdr, phdr->p_vaddr, phdr->p_paddr, phdr->p_filesz,
+			ehdr->e_phnum, phdr->p_offset);
+		phdr++;
+	}
+
+	*addr = buf;
+	*sz = elf_sz;
+	return 0;
+}
+
+int crash_exclude_mem_range(struct crash_mem *mem,
+			    unsigned long long mstart, unsigned long long mend)
+{
+	int i, j;
+	unsigned long long start, end, p_start, p_end;
+	struct range temp_range = {0, 0};
+
+	for (i = 0; i < mem->nr_ranges; i++) {
+		start = mem->ranges[i].start;
+		end = mem->ranges[i].end;
+		p_start = mstart;
+		p_end = mend;
+
+		if (mstart > end || mend < start)
+			continue;
+
+		/* Truncate any area outside of range */
+		if (mstart < start)
+			p_start = start;
+		if (mend > end)
+			p_end = end;
+
+		/* Found completely overlapping range */
+		if (p_start == start && p_end == end) {
+			mem->ranges[i].start = 0;
+			mem->ranges[i].end = 0;
+			if (i < mem->nr_ranges - 1) {
+				/* Shift rest of the ranges to left */
+				for (j = i; j < mem->nr_ranges - 1; j++) {
+					mem->ranges[j].start =
+						mem->ranges[j+1].start;
+					mem->ranges[j].end =
+							mem->ranges[j+1].end;
+				}
+
+				/*
+				 * Continue to check if there are another overlapping ranges
+				 * from the current position because of shifting the above
+				 * mem ranges.
+				 */
+				i--;
+				mem->nr_ranges--;
+				continue;
+			}
+			mem->nr_ranges--;
+			return 0;
+		}
+
+		if (p_start > start && p_end < end) {
+			/* Split original range */
+			mem->ranges[i].end = p_start - 1;
+			temp_range.start = p_end + 1;
+			temp_range.end = end;
+		} else if (p_start != start)
+			mem->ranges[i].end = p_start - 1;
+		else
+			mem->ranges[i].start = p_end + 1;
+		break;
+	}
+
+	/* If a split happened, add the split to array */
+	if (!temp_range.end)
+		return 0;
+
+	/* Split happened */
+	if (i == mem->max_nr_ranges - 1)
+		return -ENOMEM;
+
+	/* Location where new range should go */
+	j = i + 1;
+	if (j < mem->nr_ranges) {
+		/* Move over all ranges one slot towards the end */
+		for (i = mem->nr_ranges - 1; i >= j; i--)
+			mem->ranges[i + 1] = mem->ranges[i];
+	}
+
+	mem->ranges[j].start = temp_range.start;
+	mem->ranges[j].end = temp_range.end;
+	mem->nr_ranges++;
+	return 0;
+}
+
 Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len)
 {
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index dd115571f8fc..453b7a513540 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -1151,184 +1151,3 @@ int kexec_purgatory_get_set_symbol(struct kimage *image, const char *name,
 	return 0;
 }
 #endif /* CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY */
-
-int crash_exclude_mem_range(struct crash_mem *mem,
-			    unsigned long long mstart, unsigned long long mend)
-{
-	int i, j;
-	unsigned long long start, end, p_start, p_end;
-	struct range temp_range = {0, 0};
-
-	for (i = 0; i < mem->nr_ranges; i++) {
-		start = mem->ranges[i].start;
-		end = mem->ranges[i].end;
-		p_start = mstart;
-		p_end = mend;
-
-		if (mstart > end || mend < start)
-			continue;
-
-		/* Truncate any area outside of range */
-		if (mstart < start)
-			p_start = start;
-		if (mend > end)
-			p_end = end;
-
-		/* Found completely overlapping range */
-		if (p_start == start && p_end == end) {
-			mem->ranges[i].start = 0;
-			mem->ranges[i].end = 0;
-			if (i < mem->nr_ranges - 1) {
-				/* Shift rest of the ranges to left */
-				for (j = i; j < mem->nr_ranges - 1; j++) {
-					mem->ranges[j].start =
-						mem->ranges[j+1].start;
-					mem->ranges[j].end =
-							mem->ranges[j+1].end;
-				}
-
-				/*
-				 * Continue to check if there are another overlapping ranges
-				 * from the current position because of shifting the above
-				 * mem ranges.
-				 */
-				i--;
-				mem->nr_ranges--;
-				continue;
-			}
-			mem->nr_ranges--;
-			return 0;
-		}
-
-		if (p_start > start && p_end < end) {
-			/* Split original range */
-			mem->ranges[i].end = p_start - 1;
-			temp_range.start = p_end + 1;
-			temp_range.end = end;
-		} else if (p_start != start)
-			mem->ranges[i].end = p_start - 1;
-		else
-			mem->ranges[i].start = p_end + 1;
-		break;
-	}
-
-	/* If a split happened, add the split to array */
-	if (!temp_range.end)
-		return 0;
-
-	/* Split happened */
-	if (i == mem->max_nr_ranges - 1)
-		return -ENOMEM;
-
-	/* Location where new range should go */
-	j = i + 1;
-	if (j < mem->nr_ranges) {
-		/* Move over all ranges one slot towards the end */
-		for (i = mem->nr_ranges - 1; i >= j; i--)
-			mem->ranges[i + 1] = mem->ranges[i];
-	}
-
-	mem->ranges[j].start = temp_range.start;
-	mem->ranges[j].end = temp_range.end;
-	mem->nr_ranges++;
-	return 0;
-}
-
-int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
-			  void **addr, unsigned long *sz)
-{
-	Elf64_Ehdr *ehdr;
-	Elf64_Phdr *phdr;
-	unsigned long nr_cpus = num_possible_cpus(), nr_phdr, elf_sz;
-	unsigned char *buf;
-	unsigned int cpu, i;
-	unsigned long long notes_addr;
-	unsigned long mstart, mend;
-
-	/* extra phdr for vmcoreinfo ELF note */
-	nr_phdr = nr_cpus + 1;
-	nr_phdr += mem->nr_ranges;
-
-	/*
-	 * kexec-tools creates an extra PT_LOAD phdr for kernel text mapping
-	 * area (for example, ffffffff80000000 - ffffffffa0000000 on x86_64).
-	 * I think this is required by tools like gdb. So same physical
-	 * memory will be mapped in two ELF headers. One will contain kernel
-	 * text virtual addresses and other will have __va(physical) addresses.
-	 */
-
-	nr_phdr++;
-	elf_sz = sizeof(Elf64_Ehdr) + nr_phdr * sizeof(Elf64_Phdr);
-	elf_sz = ALIGN(elf_sz, ELF_CORE_HEADER_ALIGN);
-
-	buf = vzalloc(elf_sz);
-	if (!buf)
-		return -ENOMEM;
-
-	ehdr = (Elf64_Ehdr *)buf;
-	phdr = (Elf64_Phdr *)(ehdr + 1);
-	memcpy(ehdr->e_ident, ELFMAG, SELFMAG);
-	ehdr->e_ident[EI_CLASS] = ELFCLASS64;
-	ehdr->e_ident[EI_DATA] = ELFDATA2LSB;
-	ehdr->e_ident[EI_VERSION] = EV_CURRENT;
-	ehdr->e_ident[EI_OSABI] = ELF_OSABI;
-	memset(ehdr->e_ident + EI_PAD, 0, EI_NIDENT - EI_PAD);
-	ehdr->e_type = ET_CORE;
-	ehdr->e_machine = ELF_ARCH;
-	ehdr->e_version = EV_CURRENT;
-	ehdr->e_phoff = sizeof(Elf64_Ehdr);
-	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
-	ehdr->e_phentsize = sizeof(Elf64_Phdr);
-
-	/* Prepare one phdr of type PT_NOTE for each present CPU */
-	for_each_present_cpu(cpu) {
-		phdr->p_type = PT_NOTE;
-		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
-		phdr->p_offset = phdr->p_paddr = notes_addr;
-		phdr->p_filesz = phdr->p_memsz = sizeof(note_buf_t);
-		(ehdr->e_phnum)++;
-		phdr++;
-	}
-
-	/* Prepare one PT_NOTE header for vmcoreinfo */
-	phdr->p_type = PT_NOTE;
-	phdr->p_offset = phdr->p_paddr = paddr_vmcoreinfo_note();
-	phdr->p_filesz = phdr->p_memsz = VMCOREINFO_NOTE_SIZE;
-	(ehdr->e_phnum)++;
-	phdr++;
-
-	/* Prepare PT_LOAD type program header for kernel text region */
-	if (need_kernel_map) {
-		phdr->p_type = PT_LOAD;
-		phdr->p_flags = PF_R|PF_W|PF_X;
-		phdr->p_vaddr = (unsigned long) _text;
-		phdr->p_filesz = phdr->p_memsz = _end - _text;
-		phdr->p_offset = phdr->p_paddr = __pa_symbol(_text);
-		ehdr->e_phnum++;
-		phdr++;
-	}
-
-	/* Go through all the ranges in mem->ranges[] and prepare phdr */
-	for (i = 0; i < mem->nr_ranges; i++) {
-		mstart = mem->ranges[i].start;
-		mend = mem->ranges[i].end;
-
-		phdr->p_type = PT_LOAD;
-		phdr->p_flags = PF_R|PF_W|PF_X;
-		phdr->p_offset  = mstart;
-
-		phdr->p_paddr = mstart;
-		phdr->p_vaddr = (unsigned long) __va(mstart);
-		phdr->p_filesz = phdr->p_memsz = mend - mstart + 1;
-		phdr->p_align = 0;
-		ehdr->e_phnum++;
-		pr_debug("Crash PT_LOAD ELF header. phdr=%p vaddr=0x%llx, paddr=0x%llx, sz=0x%llx e_phnum=%d p_offset=0x%llx\n",
-			phdr, phdr->p_vaddr, phdr->p_paddr, phdr->p_filesz,
-			ehdr->e_phnum, phdr->p_offset);
-		phdr++;
-	}
-
-	*addr = buf;
-	*sz = elf_sz;
-	return 0;
-}
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug 11 10:06:34 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:34 -0400
Subject: [PATCH v27 0/8] crash: Kernel handling of CPU and memory hot un/plug
Message-ID: <20230811170642.6696-1-eric.devolder@oracle.com>

This series is dependent upon "refactor Kconfig to consolidate
KEXEC and CRASH options".
 https://lore.kernel.org/lkml/20230712161545.87870-1-eric.devolder at oracle.com/

Once the kdump service is loaded, if changes to CPUs or memory occur,
either by hot un/plug or off/onlining, the crash elfcorehdr must also
be updated.

The elfcorehdr describes to kdump the CPUs and memory in the system,
and any inaccuracies can result in a vmcore with missing CPU context
or memory regions.

The current solution utilizes udev to initiate an unload-then-reload
of the kdump image (eg. kernel, initrd, boot_params, purgatory and
elfcorehdr) by the userspace kexec utility. In the original post I
outlined the significant performance problems related to offloading
this activity to userspace.

This patchset introduces a generic crash handler that registers with
the CPU and memory notifiers. Upon CPU or memory changes, from either
hot un/plug or off/onlining, this generic handler is invoked and
performs important housekeeping, for example obtaining the appropriate
lock, and then invokes an architecture specific handler to do the
appropriate elfcorehdr update.

Note the description in patch 'crash: change crash_prepare_elf64_headers()
to for_each_possible_cpu()' and 'x86/crash: optimize CPU changes' that
enables further optimizations related to CPU plug/unplug/online/offline
performance of elfcorehdr updates.

In the case of x86_64, the arch specific handler generates a new
elfcorehdr, and overwrites the old one in memory; thus no involvement
with userspace needed.

To realize the benefits/test this patchset, one must make a couple
of minor changes to userspace:

 - Prevent udev from updating kdump crash kernel on hot un/plug changes.
   Add the following as the first lines to the RHEL udev rule file
   /usr/lib/udev/rules.d/98-kexec.rules:

   # The kernel updates the crash elfcorehdr for CPU and memory changes
   SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
   SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

   With this changeset applied, the two rules evaluate to false for
   CPU and memory change events and thus skip the userspace
   unload-then-reload of kdump.

 - Change to the kexec_file_load for loading the kdump kernel:
   Eg. on RHEL: in /usr/bin/kdumpctl, change to:
    standard_kexec_args="-p -d -s"
   which adds the -s to select kexec_file_load() syscall.

This kernel patchset also supports kexec_load() with a modified kexec
userspace utility. A working changeset to the kexec userspace utility
is posted to the kexec-tools mailing list here:

 http://lists.infradead.org/pipermail/kexec/2023-May/027049.html

To use the kexec-tools patch, apply, build and install kexec-tools,
then change the kdumpctl's standard_kexec_args to replace the -s with
--hotplug. The removal of -s reverts to the kexec_load syscall and
the addition of --hotplug invokes the changes put forth in the
kexec-tools patch.

Regards,
eric
---
v27: 11aug2023
 - Rebased onto 6.5.0-rc5
 - The linux-next and akpm test bots found a build error when just
   PROC_KCORE is configured (with no KEXEC or CRASH), which resulted
   in CRASH_CORE enabled by itself. To solve, the struct crash_mem
   moved from include/linux/kexec.h to include/linux/crash_core.h.
   Similarly, the crash_notes also moved from kernel/kexec.c to
   kernel/crash_core.c.
 - Minor adjustment to arch/x86/kernel/crash.c was also needed to
   avoid unused function build errors for just PROC_KCORE.
 - Spot testing of several architectures did not reveal any further
   build problems (PROC_KCORE, KEXEC, CRASH_DUMP, CRASH_HOTPLUG).

v26: 4aug2023
 https://lore.kernel.org/lkml/20230804210359.8321-1-eric.devolder at oracle.com/
 - Rebased onto 6.5.0-rc4
 - Dropped the refactor of files drivers/base/cpu|memory.c as unrelated
   to this series.
 - Minor corrections to documentation, per Randy Dunlap and GregKH.

v25: 29jun2023
 https://lore.kernel.org/lkml/20230629192119.6613-1-eric.devolder at oracle.com/
 - Properly applied IS_ENABLED() to the function bodies of callbacks
   in drivers/base/cpu|memory.c.
 - Re-ran compile and run-time testing of the impacted attributes for
   both enabled and not enabled config settings.

v24: 28jun2023
 https://lore.kernel.org/lkml/20230628185215.40707-1-eric.devolder at oracle.com/
 - Rebased onto 6.4.0
 - Included Documentation/ABI/testing entries for the new sysfs
   crash_hotplug attributes, per Greg Kroah-Hartman.
 - Refactored drivers/base/cpu|memory.c to use the .is_visible()
   method for attributes, per Greg Kroah-Hartman.
 - Retained all existing Acks and RBs as the few changes as a result
   of Greg's requests were trivial.

v23: 12jun2023
 https://lore.kernel.org/lkml/20230612210712.683175-1-eric.devolder at oracle.com/
 - Rebased onto 6.4.0-rc6
 - Refactored Kconfig, per Thomas. See series:
   https://lore.kernel.org/lkml/20230612172805.681179-1-eric.devolder at oracle.com/
 - Reworked commit messages to conform to style, per Thomas.
 - Applied Baoquan He Acked-by to kexec_load() patch.
 - Applied Hari Bathini Acked-by for the series.
 - No code changes.

v22: 3may2023
 https://lore.kernel.org/lkml/20230503224145.7405-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0
 - Improved support for kexec_load(), per Hari Bathini. See
   "crash: hotplug support for kexec_load()" which is the only
   change to this series.
 - Applied Baoquan He's Acked-by for all other patches.

v21: 4apr2023
 https://lkml.org/lkml/2023/4/4/1136
 https://lore.kernel.org/lkml/20230404180326.6890-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0-rc5
 - Additional simplification of indentation in crash_handle_hotplug_event(),
   per Baoquan.

v20: 17mar2023
 https://lkml.org/lkml/2023/3/17/1169
 https://lore.kernel.org/lkml/20230317212128.21424-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0-rc2
 - Defaulting CRASH_HOTPLUG for x86 to Y, per Sourabh.
 - Explicitly initializing image->hp_action, per Baoquan.
 - Simplified kexec_trylock() in crash_handle_hotplug_event(),
   per Baoquan.
 - Applied Sourabh's Reviewed-by to the series.

v19: 6mar2023
 https://lkml.org/lkml/2023/3/6/1358
 https://lore.kernel.org/lkml/20230306162228.8277-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0
 - Did away with offlinecpu, per Thomas Gleixner.
 - Changed to CPUHP_BP_PREPARE_DYN instead of CPUHP_AP_ONLINE_DYN.
 - Did away with elfcorehdr_index_valid, per Sourabh.
 - Convert to for_each_possible_cpu() in crash_prepare_elf64_headers()
   per Sourabh.
 - Small optimization for x86 cpu changes.

v18: 31jan2023
 https://lkml.org/lkml/2023/1/31/1356
 https://lore.kernel.org/lkml/20230131224236.122805-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc6
 - Renamed struct kimage member hotplug_event to hp_action, and
   re-enumerated the KEXEC_CRASH_HP_x items, adding _NONE at 0.
 - Moved to cpuhp state CPUHP_BP_PREPARE_DYN instead of
   CPUHP_AP_ONLINE_DYN in order to minimize window of time CPU
   is not reflected in elfcorehdr.
 - Reworked some of the comments and commit messages to offer
   more of the why, than what, per Thomas Gleixner.

v17: 18jan2023
 https://lkml.org/lkml/2023/1/18/1420
 https://lore.kernel.org/lkml/20230118213544.2128-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc4
 - Moved a bit of code around so that kexec_load()-only builds
   work, per Sourabh.
 - Corrected computation of number of memory region Phdrs needed
   when x86 memory hotplug is not enabled, per Baoquan.

v16: 5jan2023
 https://lkml.org/lkml/2023/1/5/673
 https://lore.kernel.org/lkml/20230105151709.1845-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc2
 - Corrected error identified by Baoquan.

v15: 9dec2022
 https://lkml.org/lkml/2022/12/9/520
 https://lore.kernel.org/lkml/20221209153656.3284-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc8
 - Replaced arch_un/map_crash_pages() with direct use of
   kun/map_local_pages(), per Boris.
 - Some x86 changes, per Boris.

v14: 16nov2022
 https://lkml.org/lkml/2022/11/16/1645
 https://lore.kernel.org/lkml/20221116214643.6384-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc5
 - Introduced CRASH_HOTPLUG Kconfig item to better fine tune
   compilation of feature components, per Boris.
 - Removed hp_action parameter to arch_crash_handle_hotplug_event()
   as it is unused.

v13: 31oct2022
 https://lkml.org/lkml/2022/10/31/854
 https://lore.kernel.org/lkml/20221031193604.28779-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc3, which means converting to use the new
   kexec_trylock() away from mutex_lock(kexec_mutex).
 - Moved arch_un/map_crash_pages() into kexec.h and default
   implementation using k/unmap_local_pages().
 - Changed more #ifdef's into IS_ENABLED()
 - Changed CRASH_MAX_MEMORY_RANGES to 8192 from 32768, and it moved
   into x86 crash.c as #define rather Kconfig item, per Boris.
 - Check number of Phdrs against PN_XNUM, max possible.

v12: 9sep2022
 https://lkml.org/lkml/2022/9/9/1358
 https://lore.kernel.org/lkml/20220909210509.6286-1-eric.devolder at oracle.com/
 - Rebased onto 6.0-rc4
 - Addressed some minor formatting items, per Baoquan

v11: 26aug2022
 https://lkml.org/lkml/2022/8/26/963
 https://lore.kernel.org/lkml/20220826173704.1895-1-eric.devolder at oracle.com/
 - Rebased onto 6.0-rc2
 - Redid the rework of __weak to use asm/kexec.h, per Baoquan
 - Reworked some comments and minor items, per Baoquan

v10: 21jul2022
 https://lkml.org/lkml/2022/7/21/1007
 https://lore.kernel.org/lkml/20220721181747.1640-1-eric.devolder at oracle.com/
 - Rebased to 5.19.0-rc7
 - Per Sourabh, corrected build issue with arch_un/map_crash_pages()
   for architectures not supporting this feature.
 - Per David Hildebrand, removed the WARN_ONCE() altogether.
 - Per David Hansen, converted to use of kmap_local_page().
 - Per Baoquan He, replaced use of __weak with the kexec technique.

v9: 13jun2022
 https://lkml.org/lkml/2022/6/13/3382
 https://lore.kernel.org/lkml/20220613224240.79400-1-eric.devolder at oracle.com/
 - Rebased to 5.18.0
 - Per Sourabh, moved crash_prepare_elf64_headers() into common
   crash_core.c to avoid compile issues with kexec_load only path.
 - Per David Hildebrand, replaced mutex_trylock() with mutex_lock().
 - Changed the __weak arch_crash_handle_hotplug_event() to utilize
   WARN_ONCE() instead of WARN(). Fix some formatting issues.
 - Per Sourabh, introduced sysfs attribute crash_hotplug for memory
   and CPUs; for use by userspace (udev) to determine if the kernel
   performs crash hot un/plug support.
 - Per Sourabh, moved the code detecting the elfcorehdr segment from
   arch/x86 into crash_core:handle_hotplug_event() so both kexec_load
   and kexec_file_load can benefit.
 - Updated userspace kexec-tools kexec utility to reflect change to
   using CRASH_MAX_MEMORY_RANGES and get_nr_cpus().
 - Updated the new proposed udev rules to reflect using the sysfs
   attributes crash_hotplug.

v8: 5may2022
 https://lkml.org/lkml/2022/5/5/1133
 https://lore.kernel.org/lkml/20220505184603.1548-1-eric.devolder at oracle.com/
 - Per Borislav Petkov, eliminated CONFIG_CRASH_HOTPLUG in favor
   of CONFIG_HOTPLUG_CPU || CONFIG_MEMORY_HOTPLUG, ie a new define
   is not needed. Also use of IS_ENABLED() rather than #ifdef's.
   Renamed crash_hotplug_handler() to handle_hotplug_event().
   And other corrections.
 - Per Baoquan, minimized the parameters to the arch_crash_
   handle_hotplug_event() to hp_action and cpu.
 - Introduce KEXEC_CRASH_HP_INVALID_CPU definition, per Baoquan.
 - Per Sourabh Jain, renamed and repurposed CRASH_HOTPLUG_ELFCOREHDR_SZ
   to CONFIG_CRASH_MAX_MEMORY_RANGES, mirroring kexec-tools change
   by David Hildebrand. Folded this patch into the x86
   kexec_file_load support patch.

v7: 13apr2022
 https://lkml.org/lkml/2022/4/13/850
 https://lore.kernel.org/lkml/20220413164237.20845-1-eric.devolder at oracle.com/
 - Resolved parameter usage to crash_hotplug_handler(), per Baoquan.

v6: 1apr2022
 https://lkml.org/lkml/2022/4/1/1203
 https://lore.kernel.org/lkml/20220401183040.1624-1-eric.devolder at oracle.com/
 - Reword commit messages and some comment cleanup per Baoquan.
 - Changed elf_index to elfcorehdr_index for clarity.
 - Minor code changes per Baoquan.

v5: 3mar2022
 https://lkml.org/lkml/2022/3/3/674
 https://lore.kernel.org/lkml/20220303162725.49640-1-eric.devolder at oracle.com/
 - Reworded description of CRASH_HOTPLUG_ELFCOREHDR_SZ, per
   David Hildenbrand.
 - Refactored slightly a few patches per Baoquan recommendation.

v4: 9feb2022
 https://lkml.org/lkml/2022/2/9/1406
 https://lore.kernel.org/lkml/20220209195706.51522-1-eric.devolder at oracle.com/
 - Refactored patches per Baoquan suggestsions.
 - A few corrections, per Baoquan.

v3: 10jan2022
 https://lkml.org/lkml/2022/1/10/1212
 https://lore.kernel.org/lkml/20220110195727.1682-1-eric.devolder at oracle.com/
 - Rebasing per Baoquan He request.
 - Changed memory notifier per David Hildenbrand.
 - Providing example kexec userspace change in cover letter.

RFC v2: 7dec2021
 https://lkml.org/lkml/2021/12/7/1088
 https://lore.kernel.org/lkml/20211207195204.1582-1-eric.devolder at oracle.com/
 - Acting upon Baoquan He suggestion of removing elfcorehdr from
   the purgatory list of segments, removed purgatory code from
   patchset, and it is signficiantly simpler now.

RFC v1: 18nov2021
 https://lkml.org/lkml/2021/11/18/845
 https://lore.kernel.org/lkml/20211118174948.37435-1-eric.devolder at oracle.com/
 - working patchset demonstrating kernel handling of hotplug
   updates to x86 elfcorehdr for kexec_file_load

RFC: 14dec2020
 https://lkml.org/lkml/2020/12/14/532
 https://lore.kernel.org/lkml/b04ed259-dc5f-7f30-6661-c26f92d9096a at oracle.com/
 NOTE: s/vmcoreinfo/elfcorehdr/g
 - proposed concept of allowing kernel to handle hotplug update
   of elfcorehdr
---


Eric DeVolder (8):
  crash: move a few code bits to setup support of crash hotplug
  crash: add generic infrastructure for crash hotplug support
  kexec: exclude elfcorehdr from the segment digest
  crash: memory and CPU hotplug sysfs attributes
  x86/crash: add x86 crash hotplug support
  crash: hotplug support for kexec_load()
  crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()
  x86/crash: optimize CPU changes

 .../ABI/testing/sysfs-devices-memory          |   8 +
 .../ABI/testing/sysfs-devices-system-cpu      |   8 +
 .../admin-guide/mm/memory-hotplug.rst         |   8 +
 Documentation/core-api/cpu_hotplug.rst        |  18 +
 arch/x86/Kconfig                              |   3 +
 arch/x86/include/asm/kexec.h                  |  18 +
 arch/x86/kernel/crash.c                       | 140 ++++++-
 drivers/base/cpu.c                            |  13 +
 drivers/base/memory.c                         |  13 +
 include/linux/crash_core.h                    |   9 +
 include/linux/kexec.h                         |  63 +++-
 include/uapi/linux/kexec.h                    |   1 +
 kernel/Kconfig.kexec                          |  35 ++
 kernel/crash_core.c                           | 355 ++++++++++++++++++
 kernel/kexec.c                                |   5 +
 kernel/kexec_core.c                           |   6 +
 kernel/kexec_file.c                           | 187 +--------
 kernel/ksysfs.c                               |  15 +
 18 files changed, 700 insertions(+), 205 deletions(-)

-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug 11 10:06:39 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:39 -0400
Subject: [PATCH v27 5/8] x86/crash: add x86 crash hotplug support
In-Reply-To: <20230811170642.6696-1-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
Message-ID: <20230811170642.6696-6-eric.devolder@oracle.com>

When CPU or memory is hot un/plugged, or off/onlined, the crash
elfcorehdr, which describes the CPUs and memory in the system,
must also be updated.

A new elfcorehdr is generated from the available CPUs and memory
and replaces the existing elfcorehdr. The segment containing the
elfcorehdr is identified at run-time in
crash_core:crash_handle_hotplug_event().

No modifications to purgatory (see 'kexec: exclude elfcorehdr
from the segment digest') or boot_params (as the elfcorehdr=
capture kernel command line parameter pointer remains unchanged
and correct) are needed, just elfcorehdr.

For kexec_file_load(), the elfcorehdr segment size is based on
NR_CPUS and CRASH_MAX_MEMORY_RANGES in order to accommodate a
growing number of CPU and memory resources.

For kexec_load(), the userspace kexec utility needs to size the
elfcorehdr segment in the same/similar manner.

To accommodate kexec_load() syscall in the absence of
kexec_file_load() syscall support, prepare_elf_headers() and
dependents are moved outside of CONFIG_KEXEC_FILE.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/Kconfig             |   3 +
 arch/x86/include/asm/kexec.h |  15 +++++
 arch/x86/kernel/crash.c      | 103 ++++++++++++++++++++++++++++++++---
 3 files changed, 114 insertions(+), 7 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 7082fc10b346..ffc95c3d6abd 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2069,6 +2069,9 @@ config ARCH_SUPPORTS_KEXEC_JUMP
 config ARCH_SUPPORTS_CRASH_DUMP
 	def_bool X86_64 || (X86_32 && HIGHMEM)
 
+config ARCH_SUPPORTS_CRASH_HOTPLUG
+	def_bool y
+
 config PHYSICAL_START
 	hex "Physical address where the kernel is loaded" if (EXPERT || CRASH_DUMP)
 	default "0x1000000"
diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 5b77bbc28f96..9143100ea3ea 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -209,6 +209,21 @@ typedef void crash_vmclear_fn(void);
 extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss;
 extern void kdump_nmi_shootdown_cpus(void);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+void arch_crash_handle_hotplug_event(struct kimage *image);
+#define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event
+
+#ifdef CONFIG_HOTPLUG_CPU
+static inline int crash_hotplug_cpu_support(void) { return 1; }
+#define crash_hotplug_cpu_support crash_hotplug_cpu_support
+#endif
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+static inline int crash_hotplug_memory_support(void) { return 1; }
+#define crash_hotplug_memory_support crash_hotplug_memory_support
+#endif
+#endif
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_X86_KEXEC_H */
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index cdd92ab43cda..c70a111c44fa 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -158,8 +158,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 	crash_save_cpu(regs, safe_smp_processor_id());
 }
 
-#ifdef CONFIG_KEXEC_FILE
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -231,7 +229,7 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 
 /* Prepare elf headers. Return addr and size */
 static int prepare_elf_headers(struct kimage *image, void **addr,
-					unsigned long *sz)
+					unsigned long *sz, unsigned long *nr_mem_ranges)
 {
 	struct crash_mem *cmem;
 	int ret;
@@ -249,6 +247,9 @@ static int prepare_elf_headers(struct kimage *image, void **addr,
 	if (ret)
 		goto out;
 
+	/* Return the computed number of memory ranges, for hotplug usage */
+	*nr_mem_ranges = cmem->nr_ranges;
+
 	/* By default prepare 64bit headers */
 	ret =  crash_prepare_elf64_headers(cmem, IS_ENABLED(CONFIG_X86_64), addr, sz);
 
@@ -257,6 +258,7 @@ static int prepare_elf_headers(struct kimage *image, void **addr,
 	return ret;
 }
 
+#ifdef CONFIG_KEXEC_FILE
 static int add_e820_entry(struct boot_params *params, struct e820_entry *entry)
 {
 	unsigned int nr_e820_entries;
@@ -371,18 +373,42 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 int crash_load_segments(struct kimage *image)
 {
 	int ret;
+	unsigned long pnum = 0;
 	struct kexec_buf kbuf = { .image = image, .buf_min = 0,
 				  .buf_max = ULONG_MAX, .top_down = false };
 
 	/* Prepare elf headers and add a segment */
-	ret = prepare_elf_headers(image, &kbuf.buffer, &kbuf.bufsz);
+	ret = prepare_elf_headers(image, &kbuf.buffer, &kbuf.bufsz, &pnum);
 	if (ret)
 		return ret;
 
-	image->elf_headers = kbuf.buffer;
-	image->elf_headers_sz = kbuf.bufsz;
+	image->elf_headers	= kbuf.buffer;
+	image->elf_headers_sz	= kbuf.bufsz;
+	kbuf.memsz		= kbuf.bufsz;
+
+#ifdef CONFIG_CRASH_HOTPLUG
+	/*
+	 * The elfcorehdr segment size accounts for VMCOREINFO, kernel_map,
+	 * maximum CPUs and maximum memory ranges.
+	 */
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		pnum = 2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES;
+	else
+		pnum += 2 + CONFIG_NR_CPUS_DEFAULT;
+
+	if (pnum < (unsigned long)PN_XNUM) {
+		kbuf.memsz = pnum * sizeof(Elf64_Phdr);
+		kbuf.memsz += sizeof(Elf64_Ehdr);
+
+		image->elfcorehdr_index = image->nr_segments;
+
+		/* Mark as usable to crash kernel, else crash kernel fails on boot */
+		image->elf_headers_sz = kbuf.memsz;
+	} else {
+		pr_err("number of Phdrs %lu exceeds max\n", pnum);
+	}
+#endif
 
-	kbuf.memsz = kbuf.bufsz;
 	kbuf.buf_align = ELF_CORE_HEADER_ALIGN;
 	kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
 	ret = kexec_add_buffer(&kbuf);
@@ -395,3 +421,66 @@ int crash_load_segments(struct kimage *image)
 	return ret;
 }
 #endif /* CONFIG_KEXEC_FILE */
+
+#ifdef CONFIG_CRASH_HOTPLUG
+
+#undef pr_fmt
+#define pr_fmt(fmt) "crash hp: " fmt
+
+/**
+ * arch_crash_handle_hotplug_event() - Handle hotplug elfcorehdr changes
+ * @image: a pointer to kexec_crash_image
+ *
+ * Prepare the new elfcorehdr and replace the existing elfcorehdr.
+ */
+void arch_crash_handle_hotplug_event(struct kimage *image)
+{
+	void *elfbuf = NULL, *old_elfcorehdr;
+	unsigned long nr_mem_ranges;
+	unsigned long mem, memsz;
+	unsigned long elfsz = 0;
+
+	/*
+	 * Create the new elfcorehdr reflecting the changes to CPU and/or
+	 * memory resources.
+	 */
+	if (prepare_elf_headers(image, &elfbuf, &elfsz, &nr_mem_ranges)) {
+		pr_err("unable to create new elfcorehdr");
+		goto out;
+	}
+
+	/*
+	 * Obtain address and size of the elfcorehdr segment, and
+	 * check it against the new elfcorehdr buffer.
+	 */
+	mem = image->segment[image->elfcorehdr_index].mem;
+	memsz = image->segment[image->elfcorehdr_index].memsz;
+	if (elfsz > memsz) {
+		pr_err("update elfcorehdr elfsz %lu > memsz %lu",
+			elfsz, memsz);
+		goto out;
+	}
+
+	/*
+	 * Copy new elfcorehdr over the old elfcorehdr at destination.
+	 */
+	old_elfcorehdr = kmap_local_page(pfn_to_page(mem >> PAGE_SHIFT));
+	if (!old_elfcorehdr) {
+		pr_err("mapping elfcorehdr segment failed\n");
+		goto out;
+	}
+
+	/*
+	 * Temporarily invalidate the crash image while the
+	 * elfcorehdr is updated.
+	 */
+	xchg(&kexec_crash_image, NULL);
+	memcpy_flushcache(old_elfcorehdr, elfbuf, elfsz);
+	xchg(&kexec_crash_image, image);
+	kunmap_local(old_elfcorehdr);
+	pr_debug("updated elfcorehdr\n");
+
+out:
+	vfree(elfbuf);
+}
+#endif
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug 11 10:06:36 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:36 -0400
Subject: [PATCH v27 2/8] crash: add generic infrastructure for crash hotplug support
In-Reply-To: <20230811170642.6696-1-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
Message-ID: <20230811170642.6696-3-eric.devolder@oracle.com>

To support crash hotplug, a mechanism is needed to update the crash
elfcorehdr upon CPU or memory changes (eg. hot un/plug or off/
onlining). The crash elfcorehdr describes the CPUs and memory to
be written into the vmcore.

To track CPU changes, callbacks are registered with the cpuhp
mechanism via cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN). The
crash hotplug elfcorehdr update has no explicit ordering requirement
(relative to other cpuhp states), so meets the criteria for
utilizing CPUHP_BP_PREPARE_DYN. CPUHP_BP_PREPARE_DYN is a dynamic
state and avoids the need to introduce a new state for crash
hotplug. Also, CPUHP_BP_PREPARE_DYN is the last state in the PREPARE
group, just prior to the STARTING group, which is very close to the
CPU starting up in a plug/online situation, or stopping in a unplug/
offline situation. This minimizes the window of time during an
actual plug/online or unplug/offline situation in which the
elfcorehdr would be inaccurate. Note that for a CPU being unplugged
or offlined, the CPU will still be present in the list of CPUs
generated by crash_prepare_elf64_headers(). However, there is no
need to explicitly omit the CPU, see justification in
'crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()'.

To track memory changes, a notifier is registered to capture the
memblock MEM_ONLINE and MEM_OFFLINE events via register_memory_notifier().

The CPU callbacks and memory notifiers invoke crash_handle_hotplug_event()
which performs needed tasks and then dispatches the event to the
architecture specific arch_crash_handle_hotplug_event() to update the
elfcorehdr with the current state of CPUs and memory. During the
process, the kexec_lock is held.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |   9 +++
 include/linux/kexec.h      |  11 +++
 kernel/Kconfig.kexec       |  31 ++++++++
 kernel/crash_core.c        | 142 +++++++++++++++++++++++++++++++++++++
 kernel/kexec_core.c        |   6 ++
 5 files changed, 199 insertions(+)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index de62a722431e..e14345cc7a22 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -84,4 +84,13 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
 int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
 
+#define KEXEC_CRASH_HP_NONE			0
+#define KEXEC_CRASH_HP_ADD_CPU			1
+#define KEXEC_CRASH_HP_REMOVE_CPU		2
+#define KEXEC_CRASH_HP_ADD_MEMORY		3
+#define KEXEC_CRASH_HP_REMOVE_MEMORY		4
+#define KEXEC_CRASH_HP_INVALID_CPU		-1U
+
+struct kimage;
+
 #endif /* LINUX_CRASH_CORE_H */
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 811a90e09698..b9903dd48e24 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -33,6 +33,7 @@ extern note_buf_t __percpu *crash_notes;
 #include <linux/compat.h>
 #include <linux/ioport.h>
 #include <linux/module.h>
+#include <linux/highmem.h>
 #include <asm/kexec.h>
 
 /* Verify architecture specific macros are defined */
@@ -360,6 +361,12 @@ struct kimage {
 	struct purgatory_info purgatory_info;
 #endif
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	int hp_action;
+	int elfcorehdr_index;
+	bool elfcorehdr_updated;
+#endif
+
 #ifdef CONFIG_IMA_KEXEC
 	/* Virtual address of IMA measurement buffer for kexec syscall */
 	void *ima_buffer;
@@ -490,6 +497,10 @@ static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, g
 static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) { }
 #endif
 
+#ifndef arch_crash_handle_hotplug_event
+static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
index ff72e45cfaef..d0a9a5392035 100644
--- a/kernel/Kconfig.kexec
+++ b/kernel/Kconfig.kexec
@@ -113,4 +113,35 @@ config CRASH_DUMP
 	  For s390, this option also enables zfcpdump.
 	  See also <file:Documentation/s390/zfcpdump.rst>
 
+config CRASH_HOTPLUG
+	bool "Update the crash elfcorehdr on system configuration changes"
+	default y
+	depends on CRASH_DUMP && (HOTPLUG_CPU || MEMORY_HOTPLUG)
+	depends on ARCH_SUPPORTS_CRASH_HOTPLUG
+	help
+	  Enable direct update to the crash elfcorehdr (which contains
+	  the list of CPUs and memory regions to be dumped upon a crash)
+	  in response to hot plug/unplug or online/offline of CPUs or
+	  memory. This is a much more advanced approach than userspace
+	  attempting that.
+
+	  If unsure, say Y.
+
+config CRASH_MAX_MEMORY_RANGES
+	int "Specify the maximum number of memory regions for the elfcorehdr"
+	default 8192
+	depends on CRASH_HOTPLUG
+	help
+	  For the kexec_file_load() syscall path, specify the maximum number of
+	  memory regions that the elfcorehdr buffer/segment can accommodate.
+	  These regions are obtained via walk_system_ram_res(); eg. the
+	  'System RAM' entries in /proc/iomem.
+	  This value is combined with NR_CPUS_DEFAULT and multiplied by
+	  sizeof(Elf64_Phdr) to determine the final elfcorehdr memory buffer/
+	  segment size.
+	  The value 8192, for example, covers a (sparsely populated) 1TiB system
+	  consisting of 128MiB memblocks, while resulting in an elfcorehdr
+	  memory buffer/segment size under 1MiB. This represents a sane choice
+	  to accommodate both baremetal and virtual machine configurations.
+
 endmenu
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index b7c30b748a16..53d211c690a1 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -11,6 +11,8 @@
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
 #include <linux/kexec.h>
+#include <linux/memory.h>
+#include <linux/cpuhotplug.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -18,6 +20,7 @@
 #include <crypto/sha1.h>
 
 #include "kallsyms_internal.h"
+#include "kexec_internal.h"
 
 /* vmcoreinfo stuff */
 unsigned char *vmcoreinfo_data;
@@ -697,3 +700,142 @@ static int __init crash_save_vmcoreinfo_init(void)
 }
 
 subsys_initcall(crash_save_vmcoreinfo_init);
+
+#ifdef CONFIG_CRASH_HOTPLUG
+#undef pr_fmt
+#define pr_fmt(fmt) "crash hp: " fmt
+/*
+ * To accurately reflect hot un/plug changes of cpu and memory resources
+ * (including onling and offlining of those resources), the elfcorehdr
+ * (which is passed to the crash kernel via the elfcorehdr= parameter)
+ * must be updated with the new list of CPUs and memories.
+ *
+ * In order to make changes to elfcorehdr, two conditions are needed:
+ * First, the segment containing the elfcorehdr must be large enough
+ * to permit a growing number of resources; the elfcorehdr memory size
+ * is based on NR_CPUS_DEFAULT and CRASH_MAX_MEMORY_RANGES.
+ * Second, purgatory must explicitly exclude the elfcorehdr from the
+ * list of segments it checks (since the elfcorehdr changes and thus
+ * would require an update to purgatory itself to update the digest).
+ */
+static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
+{
+	struct kimage *image;
+
+	/* Obtain lock while changing crash information */
+	if (!kexec_trylock()) {
+		pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n");
+		return;
+	}
+
+	/* Check kdump is not loaded */
+	if (!kexec_crash_image)
+		goto out;
+
+	image = kexec_crash_image;
+
+	if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
+		hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
+		pr_debug("hp_action %u, cpu %u\n", hp_action, cpu);
+	else
+		pr_debug("hp_action %u\n", hp_action);
+
+	/*
+	 * The elfcorehdr_index is set to -1 when the struct kimage
+	 * is allocated. Find the segment containing the elfcorehdr,
+	 * if not already found.
+	 */
+	if (image->elfcorehdr_index < 0) {
+		unsigned long mem;
+		unsigned char *ptr;
+		unsigned int n;
+
+		for (n = 0; n < image->nr_segments; n++) {
+			mem = image->segment[n].mem;
+			ptr = kmap_local_page(pfn_to_page(mem >> PAGE_SHIFT));
+			if (ptr) {
+				/* The segment containing elfcorehdr */
+				if (memcmp(ptr, ELFMAG, SELFMAG) == 0)
+					image->elfcorehdr_index = (int)n;
+				kunmap_local(ptr);
+			}
+		}
+	}
+
+	if (image->elfcorehdr_index < 0) {
+		pr_err("unable to locate elfcorehdr segment");
+		goto out;
+	}
+
+	/* Needed in order for the segments to be updated */
+	arch_kexec_unprotect_crashkres();
+
+	/* Differentiate between normal load and hotplug update */
+	image->hp_action = hp_action;
+
+	/* Now invoke arch-specific update handler */
+	arch_crash_handle_hotplug_event(image);
+
+	/* No longer handling a hotplug event */
+	image->hp_action = KEXEC_CRASH_HP_NONE;
+	image->elfcorehdr_updated = true;
+
+	/* Change back to read-only */
+	arch_kexec_protect_crashkres();
+
+	/* Errors in the callback is not a reason to rollback state */
+out:
+	/* Release lock now that update complete */
+	kexec_unlock();
+}
+
+static int crash_memhp_notifier(struct notifier_block *nb, unsigned long val, void *v)
+{
+	switch (val) {
+	case MEM_ONLINE:
+		crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_MEMORY,
+			KEXEC_CRASH_HP_INVALID_CPU);
+		break;
+
+	case MEM_OFFLINE:
+		crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_MEMORY,
+			KEXEC_CRASH_HP_INVALID_CPU);
+		break;
+	}
+	return NOTIFY_OK;
+}
+
+static struct notifier_block crash_memhp_nb = {
+	.notifier_call = crash_memhp_notifier,
+	.priority = 0
+};
+
+static int crash_cpuhp_online(unsigned int cpu)
+{
+	crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_CPU, cpu);
+	return 0;
+}
+
+static int crash_cpuhp_offline(unsigned int cpu)
+{
+	crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_CPU, cpu);
+	return 0;
+}
+
+static int __init crash_hotplug_init(void)
+{
+	int result = 0;
+
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		register_memory_notifier(&crash_memhp_nb);
+
+	if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
+		result = cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN,
+			"crash/cpuhp", crash_cpuhp_online, crash_cpuhp_offline);
+	}
+
+	return result;
+}
+
+subsys_initcall(crash_hotplug_init);
+#endif
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index e2f2574d8b74..5d323255862a 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -277,6 +277,12 @@ struct kimage *do_kimage_alloc_init(void)
 	/* Initialize the list of unusable pages */
 	INIT_LIST_HEAD(&image->unusable_pages);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	image->hp_action = KEXEC_CRASH_HP_NONE;
+	image->elfcorehdr_index = -1;
+	image->elfcorehdr_updated = false;
+#endif
+
 	return image;
 }
 
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug 11 10:06:42 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:42 -0400
Subject: [PATCH v27 8/8] x86/crash: optimize CPU changes
In-Reply-To: <20230811170642.6696-1-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
Message-ID: <20230811170642.6696-9-eric.devolder@oracle.com>

crash_prepare_elf64_headers() writes into the elfcorehdr an ELF
PT_NOTE for all possible CPUs. As such, subsequent changes to CPUs
(ie. hot un/plug, online/offline) do not need to rewrite the elfcorehdr.

The kimage->file_mode term covers kdump images loaded via the
kexec_file_load() syscall. Since crash_prepare_elf64_headers()
wrote the initial elfcorehdr, no update to the elfcorehdr is
needed for CPU changes.

The kimage->elfcorehdr_updated term covers kdump images loaded via
the kexec_load() syscall. At least one memory or CPU change must occur
to cause crash_prepare_elf64_headers() to rewrite the elfcorehdr.
Afterwards, no update to the elfcorehdr is needed for CPU changes.

This code is intentionally *NOT* hoisted into
crash_handle_hotplug_event() as it would prevent the arch-specific
handler from running for CPU changes. This would break PPC, for
example, which needs to update other information besides the
elfcorehdr, on CPU changes.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/kernel/crash.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index caf22bcb61af..18d2a18d1073 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -467,6 +467,16 @@ void arch_crash_handle_hotplug_event(struct kimage *image)
 	unsigned long mem, memsz;
 	unsigned long elfsz = 0;
 
+	/*
+	 * As crash_prepare_elf64_headers() has already described all
+	 * possible CPUs, there is no need to update the elfcorehdr
+	 * for additional CPU changes.
+	 */
+	if ((image->file_mode || image->elfcorehdr_updated) &&
+		((image->hp_action == KEXEC_CRASH_HP_ADD_CPU) ||
+		(image->hp_action == KEXEC_CRASH_HP_REMOVE_CPU)))
+		return;
+
 	/*
 	 * Create the new elfcorehdr reflecting the changes to CPU and/or
 	 * memory resources.
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug 11 10:06:40 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:40 -0400
Subject: [PATCH v27 6/8] crash: hotplug support for kexec_load()
In-Reply-To: <20230811170642.6696-1-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
Message-ID: <20230811170642.6696-7-eric.devolder@oracle.com>

The hotplug support for kexec_load() requires changes to the
userspace kexec-tools and a little extra help from the kernel.

Given a kdump capture kernel loaded via kexec_load(), and a
subsequent hotplug event, the crash hotplug handler finds the
elfcorehdr and rewrites it to reflect the hotplug change.
That is the desired outcome, however, at kernel panic time,
the purgatory integrity check fails (because the elfcorehdr
changed), and the capture kernel does not boot and no vmcore
is generated.

Therefore, the userspace kexec-tools/kexec must indicate to the
kernel that the elfcorehdr can be modified (because the kexec
excluded the elfcorehdr from the digest, and sized the elfcorehdr
memory buffer appropriately).

To facilitate hotplug support with kexec_load():
 - a new kexec flag KEXEC_UPATE_ELFCOREHDR indicates that it is
   safe for the kernel to modify the kexec_load()'d elfcorehdr
 - the /sys/kernel/crash_elfcorehdr_size node communicates the
   preferred size of the elfcorehdr memory buffer
 - The sysfs crash_hotplug nodes (ie.
   /sys/devices/system/[cpu|memory]/crash_hotplug) dynamically
   take into account kexec_file_load() vs kexec_load() and
   KEXEC_UPDATE_ELFCOREHDR.
   This is critical so that the udev rule processing of crash_hotplug
   is all that is needed to determine if the userspace unload-then-load
   of the kdump image is to be skipped, or not. The proposed udev
   rule change looks like:
   # The kernel updates the crash elfcorehdr for CPU and memory changes
   SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
   SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

The table below indicates the behavior of kexec_load()'d kdump image
updates (with the new udev crash_hotplug rule in place):

 Kernel |Kexec
 -------+-----+----
 Old    |Old  |New
        |  a  | a
 -------+-----+----
 New    |  a  | b
 -------+-----+----

where kexec 'old' and 'new' delineate kexec-tools has the needed
modifications for the crash hotplug feature, and kernel 'old' and
'new' delineate the kernel supports this crash hotplug feature.

Behavior 'a' indicates the unload-then-reload of the entire kdump
image. For the kexec 'old' column, the unload-then-reload occurs
due to the missing flag KEXEC_UPDATE_ELFCOREHDR. An 'old' kernel
(with 'new' kexec) does not present the crash_hotplug sysfs node,
which leads to the unload-then-reload of the kdump image.

Behavior 'b' indicates the desired optimized behavior of the kernel
directly modifying the elfcorehdr and avoiding the unload-then-reload
of the kdump image.

If the udev rule is not updated with crash_hotplug node check, then
no matter any combination of kernel or kexec is new or old, the
kdump image continues to be unload-then-reload on hotplug changes.

To fully support crash hotplug feature, there needs to be a rollout
of kernel, kexec-tools and udev rule changes. However, the order of
the rollout of these pieces does not matter; kexec_load()'d kdump
images still function for hotplug as-is.

Suggested-by: Hari Bathini <hbathini at linux.ibm.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/include/asm/kexec.h | 11 +++++++----
 arch/x86/kernel/crash.c      | 27 +++++++++++++++++++++++++++
 include/linux/kexec.h        | 14 ++++++++++++--
 include/uapi/linux/kexec.h   |  1 +
 kernel/Kconfig.kexec         |  4 ++++
 kernel/crash_core.c          | 31 +++++++++++++++++++++++++++++++
 kernel/kexec.c               |  5 +++++
 kernel/ksysfs.c              | 15 +++++++++++++++
 8 files changed, 102 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 9143100ea3ea..3be6a98751f0 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -214,14 +214,17 @@ void arch_crash_handle_hotplug_event(struct kimage *image);
 #define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event
 
 #ifdef CONFIG_HOTPLUG_CPU
-static inline int crash_hotplug_cpu_support(void) { return 1; }
-#define crash_hotplug_cpu_support crash_hotplug_cpu_support
+int arch_crash_hotplug_cpu_support(void);
+#define crash_hotplug_cpu_support arch_crash_hotplug_cpu_support
 #endif
 
 #ifdef CONFIG_MEMORY_HOTPLUG
-static inline int crash_hotplug_memory_support(void) { return 1; }
-#define crash_hotplug_memory_support crash_hotplug_memory_support
+int arch_crash_hotplug_memory_support(void);
+#define crash_hotplug_memory_support arch_crash_hotplug_memory_support
 #endif
+
+unsigned int arch_crash_get_elfcorehdr_size(void);
+#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size
 #endif
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index c70a111c44fa..caf22bcb61af 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -427,6 +427,33 @@ int crash_load_segments(struct kimage *image)
 #undef pr_fmt
 #define pr_fmt(fmt) "crash hp: " fmt
 
+/* These functions provide the value for the sysfs crash_hotplug nodes */
+#ifdef CONFIG_HOTPLUG_CPU
+int arch_crash_hotplug_cpu_support(void)
+{
+	return crash_check_update_elfcorehdr();
+}
+#endif
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int arch_crash_hotplug_memory_support(void)
+{
+	return crash_check_update_elfcorehdr();
+}
+#endif
+
+unsigned int arch_crash_get_elfcorehdr_size(void)
+{
+	unsigned int sz;
+
+	/* kernel_map, VMCOREINFO and maximum CPUs */
+	sz = 2 + CONFIG_NR_CPUS_DEFAULT;
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		sz += CONFIG_CRASH_MAX_MEMORY_RANGES;
+	sz *= sizeof(Elf64_Phdr);
+	return sz;
+}
+
 /**
  * arch_crash_handle_hotplug_event() - Handle hotplug elfcorehdr changes
  * @image: a pointer to kexec_crash_image
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 6a8a724ac638..bb0e614f2a05 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -335,6 +335,10 @@ struct kimage {
 	unsigned int preserve_context : 1;
 	/* If set, we are using file mode kexec syscall */
 	unsigned int file_mode:1;
+#ifdef CONFIG_CRASH_HOTPLUG
+	/* If set, allow changes to elfcorehdr of kexec_load'd image */
+	unsigned int update_elfcorehdr:1;
+#endif
 
 #ifdef ARCH_HAS_KIMAGE_ARCH
 	struct kimage_arch arch;
@@ -411,9 +415,9 @@ bool kexec_load_permitted(int kexec_image_type);
 
 /* List of defined/legal kexec flags */
 #ifndef CONFIG_KEXEC_JUMP
-#define KEXEC_FLAGS    KEXEC_ON_CRASH
+#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_UPDATE_ELFCOREHDR)
 #else
-#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT)
+#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT | KEXEC_UPDATE_ELFCOREHDR)
 #endif
 
 /* List of defined/legal kexec file flags */
@@ -501,6 +505,8 @@ static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) {
 static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
 #endif
 
+int crash_check_update_elfcorehdr(void);
+
 #ifndef crash_hotplug_cpu_support
 static inline int crash_hotplug_cpu_support(void) { return 0; }
 #endif
@@ -509,6 +515,10 @@ static inline int crash_hotplug_cpu_support(void) { return 0; }
 static inline int crash_hotplug_memory_support(void) { return 0; }
 #endif
 
+#ifndef crash_get_elfcorehdr_size
+static inline unsigned int crash_get_elfcorehdr_size(void) { return 0; }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/include/uapi/linux/kexec.h b/include/uapi/linux/kexec.h
index 981016e05cfa..01766dd839b0 100644
--- a/include/uapi/linux/kexec.h
+++ b/include/uapi/linux/kexec.h
@@ -12,6 +12,7 @@
 /* kexec flags for different usage scenarios */
 #define KEXEC_ON_CRASH		0x00000001
 #define KEXEC_PRESERVE_CONTEXT	0x00000002
+#define KEXEC_UPDATE_ELFCOREHDR	0x00000004
 #define KEXEC_ARCH_MASK		0xffff0000
 
 /*
diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
index d0a9a5392035..6e5350d2d7a6 100644
--- a/kernel/Kconfig.kexec
+++ b/kernel/Kconfig.kexec
@@ -144,4 +144,8 @@ config CRASH_MAX_MEMORY_RANGES
 	  memory buffer/segment size under 1MiB. This represents a sane choice
 	  to accommodate both baremetal and virtual machine configurations.
 
+	  For the kexec_load() syscall path, CRASH_MAX_MEMORY_RANGES is part of
+	  the computation behind the value provided through the
+	  /sys/kernel/crash_elfcorehdr_size attribute.
+
 endmenu
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 53d211c690a1..fa918176d46d 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -704,6 +704,33 @@ subsys_initcall(crash_save_vmcoreinfo_init);
 #ifdef CONFIG_CRASH_HOTPLUG
 #undef pr_fmt
 #define pr_fmt(fmt) "crash hp: " fmt
+
+/*
+ * This routine utilized when the crash_hotplug sysfs node is read.
+ * It reflects the kernel's ability/permission to update the crash
+ * elfcorehdr directly.
+ */
+int crash_check_update_elfcorehdr(void)
+{
+	int rc = 0;
+
+	/* Obtain lock while reading crash information */
+	if (!kexec_trylock()) {
+		pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n");
+		return 0;
+	}
+	if (kexec_crash_image) {
+		if (kexec_crash_image->file_mode)
+			rc = 1;
+		else
+			rc = kexec_crash_image->update_elfcorehdr;
+	}
+	/* Release lock now that update complete */
+	kexec_unlock();
+
+	return rc;
+}
+
 /*
  * To accurately reflect hot un/plug changes of cpu and memory resources
  * (including onling and offlining of those resources), the elfcorehdr
@@ -734,6 +761,10 @@ static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
 
 	image = kexec_crash_image;
 
+	/* Check that updating elfcorehdr is permitted */
+	if (!(image->file_mode || image->update_elfcorehdr))
+		goto out;
+
 	if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
 		hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
 		pr_debug("hp_action %u, cpu %u\n", hp_action, cpu);
diff --git a/kernel/kexec.c b/kernel/kexec.c
index 92d301f98776..107f355eac10 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -129,6 +129,11 @@ static int do_kexec_load(unsigned long entry, unsigned long nr_segments,
 	if (flags & KEXEC_PRESERVE_CONTEXT)
 		image->preserve_context = 1;
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	if (flags & KEXEC_UPDATE_ELFCOREHDR)
+		image->update_elfcorehdr = 1;
+#endif
+
 	ret = machine_kexec_prepare(image);
 	if (ret)
 		goto out;
diff --git a/kernel/ksysfs.c b/kernel/ksysfs.c
index aad7a3bfd846..1d4bc493b2f4 100644
--- a/kernel/ksysfs.c
+++ b/kernel/ksysfs.c
@@ -165,6 +165,18 @@ static ssize_t vmcoreinfo_show(struct kobject *kobj,
 }
 KERNEL_ATTR_RO(vmcoreinfo);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+static ssize_t crash_elfcorehdr_size_show(struct kobject *kobj,
+			       struct kobj_attribute *attr, char *buf)
+{
+	unsigned int sz = crash_get_elfcorehdr_size();
+
+	return sysfs_emit(buf, "%u\n", sz);
+}
+KERNEL_ATTR_RO(crash_elfcorehdr_size);
+
+#endif
+
 #endif /* CONFIG_CRASH_CORE */
 
 /* whether file capabilities are enabled */
@@ -255,6 +267,9 @@ static struct attribute * kernel_attrs[] = {
 #endif
 #ifdef CONFIG_CRASH_CORE
 	&vmcoreinfo_attr.attr,
+#ifdef CONFIG_CRASH_HOTPLUG
+	&crash_elfcorehdr_size_attr.attr,
+#endif
 #endif
 #ifndef CONFIG_TINY_RCU
 	&rcu_expedited_attr.attr,
-- 
2.31.1



From eric.devolder at oracle.com  Fri Aug 11 10:06:41 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Fri, 11 Aug 2023 13:06:41 -0400
Subject: [PATCH v27 7/8] crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()
In-Reply-To: <20230811170642.6696-1-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
Message-ID: <20230811170642.6696-8-eric.devolder@oracle.com>

The function crash_prepare_elf64_headers() generates the elfcorehdr
which describes the CPUs and memory in the system for the crash kernel.
In particular, it writes out ELF PT_NOTEs for memory regions and the
CPUs in the system.

With respect to the CPUs, the current implementation utilizes
for_each_present_cpu() which means that as CPUs are added and removed,
the elfcorehdr must again be updated to reflect the new set of CPUs.

The reasoning behind the move to use for_each_possible_cpu(), is:

- At kernel boot time, all percpu crash_notes are allocated for all
  possible CPUs; that is, crash_notes are not allocated dynamically
  when CPUs are plugged/unplugged. Thus the crash_notes for each
  possible CPU are always available.

- The crash_prepare_elf64_headers() creates an ELF PT_NOTE per CPU.
  Changing to for_each_possible_cpu() is valid as the crash_notes
  pointed to by each CPU PT_NOTE are present and always valid.

Furthermore, examining a common crash processing path of:

 kernel panic -> crash kernel -> makedumpfile -> 'crash' analyzer
           elfcorehdr      /proc/vmcore     vmcore

reveals how the ELF CPU PT_NOTEs are utilized:

- Upon panic, each CPU is sent an IPI and shuts itself down, recording
 its state in its crash_notes. When all CPUs are shutdown, the
 crash kernel is launched with a pointer to the elfcorehdr.

- The crash kernel via linux/fs/proc/vmcore.c does not examine or
 use the contents of the PT_NOTEs, it exposes them via /proc/vmcore.

- The makedumpfile utility uses /proc/vmcore and reads the CPU
 PT_NOTEs to craft a nr_cpus variable, which is reported in a
 header but otherwise generally unused. Makedumpfile creates the
 vmcore.

- The 'crash' dump analyzer does not appear to reference the CPU
 PT_NOTEs. Instead it looks-up the cpu_[possible|present|onlin]_mask
 symbols and directly examines those structure contents from vmcore
 memory. From that information it is able to determine which CPUs
 are present and online, and locate the corresponding crash_notes.
 Said differently, it appears that 'crash' analyzer does not rely
 on the ELF PT_NOTEs for CPUs; rather it obtains the information
 directly via kernel symbols and the memory within the vmcore.

(There maybe other vmcore generating and analysis tools that do use
these PT_NOTEs, but 'makedumpfile' and 'crash' seems to be the most
common solution.)

This results in the benefit of having all CPUs described in the
elfcorehdr, and therefore reducing the need to re-generate the
elfcorehdr on CPU changes, at the small expense of an additional
56 bytes per PT_NOTE for not-present-but-possible CPUs.

On systems where kexec_file_load() syscall is utilized, all the above
is valid. On systems where kexec_load() syscall is utilized, there
may be the need for the elfcorehdr to be regenerated once. The reason
being that some archs only populate the 'present' CPUs from the
/sys/devices/system/cpus entries, which the userspace 'kexec' utility
uses to generate the userspace-supplied elfcorehdr. In this situation,
one memory or CPU change will rewrite the elfcorehdr via the
crash_prepare_elf64_headers() function and now all possible CPUs will
be described, just as with kexec_file_load() syscall.

Suggested-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 kernel/crash_core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index fa918176d46d..7378b501fada 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -364,8 +364,8 @@ int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
 	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
 	ehdr->e_phentsize = sizeof(Elf64_Phdr);
 
-	/* Prepare one phdr of type PT_NOTE for each present CPU */
-	for_each_present_cpu(cpu) {
+	/* Prepare one phdr of type PT_NOTE for each possible CPU */
+	for_each_possible_cpu(cpu) {
 		phdr->p_type = PT_NOTE;
 		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
 		phdr->p_offset = phdr->p_paddr = notes_addr;
-- 
2.31.1



From stefanb at linux.ibm.com  Fri Aug 11 11:16:25 2023
From: stefanb at linux.ibm.com (Stefan Berger)
Date: Fri, 11 Aug 2023 14:16:25 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <aba709f8-a808-9aa4-8322-430e3997f686@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <b748230c8ee291288afcf48898507556c3aa7c71.camel@HansenPartnership.com>
 <5d21276a-daac-fc9b-add9-62e7c04bbdcd@linux.ibm.com>
 <b538f7d2-5a04-46d0-3792-a18653230a95@linux.microsoft.com>
 <011d8a79-236f-dc20-08fc-b5da7dd1d5a7@linux.ibm.com>
 <aba709f8-a808-9aa4-8322-430e3997f686@linux.microsoft.com>
Message-ID: <b9094b2f-d349-e029-80d6-bab92f6eaea6@linux.ibm.com>



On 8/11/23 11:57, Tushar Sugandhi wrote:
> 
> 
> 
> [1] https://patchwork.kernel.org/project/linux-integrity/cover/20230801181917.8535-1-tusharsu at linux.microsoft.com/
> 
>> The shards should will need to be written into some sort of standard location or a config file needs to
>> be defined, so that everyone knows where to find them and how they are named.
>>
> We thought about well known standard location earlier.
> Letting the Kernel choose the name/location of the snapshot
> file comes with its own complexity. Our initial stance is we don?t
> want to handle that at Kernel level, and let the UM client choose
> the location/naming of the snapshot files. But we are happy to
> reconsider if the community requests it.

I would also let user space do the snapshotting but all applications
relying on shards should know where they are located on the system
and what the naming scheme is so they can be  process in proper order.
evmctl for example would have to know where the shards are if keylime
agent had taken snapshots.



>>> Yes. If the ?PCR quotes in the snapshot_aggregate event in IMA log?
>>
>> PCR quote or 'quotes'? Why multiple?
>>
>> Form your proposal but you may have changed your opinion? following what I see in other messages:
>> "- The Kernel will get the current TPM PCR values and PCR update counter [2]
>> ??? and store them as template data in a new IMA event "snapshot_aggregate"."
>>
>> Afaik TPM quote's don't give you the state of the individual PCR values, therefore
>> I would expect to at least find the 'PCR values' of all the PCRs that IMA touched to
>> be in the snapshot_aggregate so I can replay all the following events on top of these
>> PCR values and come up with the values that were used in the "final PCR quote". This
>> is unless you expect the server to take an automatic snapshot of the values of the
>> PCRs? that it computed while evaluating the log in case it ever needs to go back.
>>
> I meant a single set of PCR values captured when snapshot_aggregate
> is logged. Sorry for the confusion.

Ok.

> 
>>> + "replay of rest of the events in IMA log" results in the ?final PCR quotes?
>>> that matches with the ?AK signed PCR quotes? sent by the client, then the truncated
>>> IMA log can be trusted. The verifier can either ?trust? the ?PCR quotes in the
>>> snapshot_aggregate event in IMA log? or it can ask for the (n-1)th snapshot shard
>>> to check the past events.
>>
>> For anything regarding determining the 'trustworthiness of a system' one would have to
>> be able to go back to the very beginning of the log *or* remember in what state a
>> system was when the latest snapshot was taken so that if a restart happens it can resume
>> with that assumption about state of trustworthiness and know what the values of the PCRs
>> were at that time so it can resume replaying the log (or the server would get these
>> values from the log).
>>
> Correct. We intend to support the above. I hope our proposal
> description captures it. BTW, when you say ?restart?, you mean the UM
> process restart, right? Because in case of a Kernel restart

Yes, client restart not reboot.

> (i.e. cold-boot) the past IMA log (and the TPM state) is lost,
> and old snapshots (if any) are useless.

Right. Some script should run on boot and delete all contents of the directory where the log
shards are.

> 
>> The AK quotes by the kernel (which adds a 2nd AK key) that James is proposing
>> could be useful if the entire log, consisting of multiple shards, is very large and
>> cannot be transferred from the client to the server in one go so that the server could
>> evaluate the 'final PCR quote' immediately . However, if a client can indicated 'I will
>> send more the next time and I have this much more to transfer' and the server allows
>> this multiple times (until all the 1MB shards of the 20MB log are transferred) then that
>> kernel AK key would not be necessary since presumably the "final PCR quote", created
>> by a user space client, would resolve whether the entire log is trustworthy.
>>
> See my responses to James today [2]
> 
> [2] https://lore.kernel.org/all/72e39852-1ff1-c7f6-ac7e-593e8142dbe8 at linux.microsoft.com/

I think James was proposing one AK, possibly persisted in the TPM's NVRAM. Still, the less keys
that are involved in this the better...

    Stefan



From bhe at redhat.com  Fri Aug 11 16:11:34 2023
From: bhe at redhat.com (bhe at redhat.com)
Date: Sat, 12 Aug 2023 07:11:34 +0800
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64
 platform
In-Reply-To: <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>
 <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
Message-ID: <ZNbAC+nq2+CJSQX8@MiWiFi-R3L-srv>

On 08/11/23 at 01:27pm, Pandey, Radhey Shyam wrote:
> > -----Original Message-----
> > From: bhe at redhat.com <bhe at redhat.com>
> > Sent: Wednesday, August 9, 2023 7:42 AM
> > To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>;
> > piliu at redhat.com
> > Cc: kexec at lists.infradead.org; linux-kernel at vger.kernel.org
> > Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> > arm64 platform
> > 
> > On 08/08/23 at 07:17pm, Pandey, Radhey Shyam wrote:
> > > Hi,
> > >
> > > I am trying to bring up kdump on arm64 platform[1]. But I get "Cannot get
> > kernel _text symbol address".
> > >
> > > Is there some Dump-capture kernel config options that I am missing?
> > >
> > > FYI, copied below complete kexec debug log.
> > >
> > > [1]: https://www.xilinx.com/products/boards-and-kits/vck190.html
> > 
> > Your description isn't clear. You saw the printing, then your kdump kernel
> > loading succeeded or not?
> > 
> > If no, have you tried applying Pingfan's patchset and still saw the issue?
> > 
> > [PATCHv7 0/5] arm64: zboot support
> > https://lore.kernel.org/all/20230803024152.11663-1-piliu at redhat.com/T/#u
> 
> I was able to proceed further with loading with crash kernel on triggering system crash.
> echo c > /proc/sysrq-trigger
> 
> But when I copy /proc/vmcore it throws memory abort. Also I see size of /proc/vmcore really huge (18446603353488633856).

This is a better symptom description.

It's very similar with a solved issue even though the calltrace is not
completely same, can you try below patch to see if it fix your problem?

[PATCH] fs/proc/kcore: reinstate bounce buffer for KCORE_TEXT regions
https://lore.kernel.org/all/20230731215021.70911-1-lstoakes at gmail.com/T/#u

> Any possible guess on what could be wrong?
> 
> 
> [   80.733523] Starting crashdump kernel...
> [   80.737435] Bye!
> [    0.000000] Booting Linux on physical CPU 0x0000000001 [0x410fd083]
> [    0.000000] Linux version 6.5.0-rc4-ge28001fb4e07 (radheys at xhdradheys41) (aarch64-xilinx-linux-gcc.real (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39.0.20220819) #23 SMP Fri Aug 11 16:25:34 IST 2023
> <snip>
> 
> 
> 
> xilinx-vck190-20232:/run/media/mmcblk0p1# cat /proc/meminfo | head
> MemTotal:        2092876 kB
> MemFree:         1219928 kB
> MemAvailable:    1166004 kB
> Buffers:              32 kB
> Cached:           756952 kB
> SwapCached:            0 kB
> Active:             1480 kB
> Inactive:          24164 kB
> Active(anon):       1452 kB
> Inactive(anon):    24160 kB
> xilinx-vck190-20232:/run/media/mmcblk0p1# cp /proc/vmcore dump     
> [  975.284865] Unable to handle kernel level 3 address size fault at virtual address ffff80008d7cf000
> [  975.293871] Mem abort info:
> [  975.296669]   ESR = 0x0000000096000003
> [  975.300425]   EC = 0x25: DABT (current EL), IL = 32 bits
> [  975.305738]   SET = 0, FnV = 0
> [  975.308788]   EA = 0, S1PTW = 0
> [  975.311925]   FSC = 0x03: level 3 address size fault
> [  975.316888] Data abort info:
> [  975.319763]   ISV = 0, ISS = 0x00000003, ISS2 = 0x00000000
> [  975.325245]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> [  975.330292]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> [  975.335599] swapper pgtable: 4k pages, 48-bit VAs, pgdp=000005016ef6b000
> [  975.342297] [ffff80008d7cf000] pgd=10000501eddfe003, p4d=10000501eddfe003, pud=10000501eddfd003, pmd=100005017b695003, pte=00687fff84000703
> [  975.354827] Internal error: Oops: 0000000096000003 [#4] SMP
> [  975.360392] Modules linked in:
> 3  975.
> 63440] CBPrUo:a d0c aPID: 664 Comm: cp Tainted: G      D            6.5.0-rc4-ge28001fb4e07 #23
> [  975.372822] Hardware name: Xilinx Versal vck190 Eval board revA (DT)
> [  975.379165] pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [  975.386119] pc : __memcpy+0x110/0x230
> [  975.389783] lr : _copy_to_iter+0x3d8/0x4d0
> [  975.393874] sp : ffff80008dc939a0
> [  975.397178] x29: ffff80008dc939a0 x28: ffff05013c1bea30 x27: 0000000000001000
> [  975.404309] x26: 0000000000001000 x25: 0000000000001000 x24: ffff80008d7cf000
> [  975.411440] x23: 0000040000000000 x22: ffff80008dc93ba0 x21: 0000000000001000
> [  975.418570] x20: ffff000000000000 x19: 0000000000001000 x18: 0000000000000000
> [  975.425699] x17: 0000000000000000 x16: 0000000000000000 x15: 0140000000000000
> [  975.432829] x14: ffff8500a9919000 x13: 0040000000000001 x12: 0000fffef6831000
> [  975.439958] x11: ffff80008d9cf000 x10: 0000000000000000 x9 : 0000000000000000
> [  975.447088] x8 : ffff80008d7d0000 x7 : ffff0501addfd358 x6 : 0400000000000001
> [  975.454217] x5 : ffff0501370e9000 x4 : ffff80008d7d0000 x3 : 0000000000000000
> [  975.461346] x2 : 0000000000001000 x1 : ffff80008d7cf000 x0 : ffff0501370e8000
> [  975.468476] Call trace:
> [  975.470912]  __memcpy+0x110/0x230
> [  975.474221]  copy_oldmem_page+0x70/0xac
> [  975.478050]  read_from_oldmem.part.0+0x120/0x188
> [  975.482663]  read_vmcore+0x14c/0x238
> [  975.486231]  proc_reg_read_iter+0x84/0xd8
> [  975.490233]  copy_splice_read+0x160/0x288
> [  975.494236]  vfs_splice_read+0xac/0x10c
> [  975.498063]  splice_direct_to_actor+0xa4/0x26c
> [  975.502498]  do_splice_direct+0x90/0xdc
> [  975.506325]  do_sendfile+0x344/0x454
> [  975.509892]  __arm64_sys_sendfile64+0x134/0x140
> [  975.514415]  invoke_syscall+0x54/0x124
> [  975.518157]  el0_svc_common.constprop.0+0xc4/0xe4
> [  975.522854]  do_el0_svc+0x38/0x98
> [  975.526162]  el0_svc+0x2c/0x84
> [  975.529211]  el0t_64_sync_handler+0x100/0x12c
> [  975.533562]  el0t_64_sync+0x190/0x194
> [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3 (a940342c) 
> [  975.543302] ---[ end trace 0000000000000000 ]---
> t message from systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):
> 
> kernel[539]: [  975.354827] Internal error: Oops: 0000000096000003 [#4] SMP
> 
> 
> Broadcast message from systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):
> 
> kernel[539]: [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3 (a940342c)
> 
> Segmentation fault
> xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lrth /proc/vmcore 
> -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lh /proc/vmcore
> -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> xilinx-vck190-20232:/run/media/mmcblk0p1# ls -l /proc/vmcore
> -r--------    1 root     root     18446603353488633856 Nov  8 14:05 /proc/vmcore
> 
> Thanks,
> Radhey
> 



From bhe at redhat.com  Fri Aug 11 20:41:16 2023
From: bhe at redhat.com (bhe at redhat.com)
Date: Sat, 12 Aug 2023 11:41:16 +0800
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64
 platform
In-Reply-To: <ZNbAC+nq2+CJSQX8@MiWiFi-R3L-srv>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>
 <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNbAC+nq2+CJSQX8@MiWiFi-R3L-srv>
Message-ID: <ZNb/XKryxy4VHvWF@MiWiFi-R3L-srv>

On 08/12/23 at 07:11am, Baoquan He wrote:
> On 08/11/23 at 01:27pm, Pandey, Radhey Shyam wrote:
> > > -----Original Message-----
> > > From: bhe at redhat.com <bhe at redhat.com>
> > > Sent: Wednesday, August 9, 2023 7:42 AM
> > > To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>;
> > > piliu at redhat.com
> > > Cc: kexec at lists.infradead.org; linux-kernel at vger.kernel.org
> > > Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> > > arm64 platform
> > > 
> > > On 08/08/23 at 07:17pm, Pandey, Radhey Shyam wrote:
> > > > Hi,
> > > >
> > > > I am trying to bring up kdump on arm64 platform[1]. But I get "Cannot get
> > > kernel _text symbol address".
> > > >
> > > > Is there some Dump-capture kernel config options that I am missing?
> > > >
> > > > FYI, copied below complete kexec debug log.
> > > >
> > > > [1]: https://www.xilinx.com/products/boards-and-kits/vck190.html
> > > 
> > > Your description isn't clear. You saw the printing, then your kdump kernel
> > > loading succeeded or not?
> > > 
> > > If no, have you tried applying Pingfan's patchset and still saw the issue?
> > > 
> > > [PATCHv7 0/5] arm64: zboot support
> > > https://lore.kernel.org/all/20230803024152.11663-1-piliu at redhat.com/T/#u
> > 
> > I was able to proceed further with loading with crash kernel on triggering system crash.
> > echo c > /proc/sysrq-trigger
> > 
> > But when I copy /proc/vmcore it throws memory abort. Also I see size of /proc/vmcore really huge (18446603353488633856).
> 
> This is a better symptom description.
> 
> It's very similar with a solved issue even though the calltrace is not
> completely same, can you try below patch to see if it fix your problem?

Oops, I was wrong. Below patch is irrelevant because it's a kcore issue,
you met a vmcore issue, please ignore this. We need investigate to see
what is happening.

> 
> [PATCH] fs/proc/kcore: reinstate bounce buffer for KCORE_TEXT regions
> https://lore.kernel.org/all/20230731215021.70911-1-lstoakes at gmail.com/T/#u
> 
> > Any possible guess on what could be wrong?
> > 
> > 
> > [   80.733523] Starting crashdump kernel...
> > [   80.737435] Bye!
> > [    0.000000] Booting Linux on physical CPU 0x0000000001 [0x410fd083]
> > [    0.000000] Linux version 6.5.0-rc4-ge28001fb4e07 (radheys at xhdradheys41) (aarch64-xilinx-linux-gcc.real (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39.0.20220819) #23 SMP Fri Aug 11 16:25:34 IST 2023
> > <snip>
> > 
> > 
> > 
> > xilinx-vck190-20232:/run/media/mmcblk0p1# cat /proc/meminfo | head
> > MemTotal:        2092876 kB
> > MemFree:         1219928 kB
> > MemAvailable:    1166004 kB
> > Buffers:              32 kB
> > Cached:           756952 kB
> > SwapCached:            0 kB
> > Active:             1480 kB
> > Inactive:          24164 kB
> > Active(anon):       1452 kB
> > Inactive(anon):    24160 kB
> > xilinx-vck190-20232:/run/media/mmcblk0p1# cp /proc/vmcore dump     
> > [  975.284865] Unable to handle kernel level 3 address size fault at virtual address ffff80008d7cf000
> > [  975.293871] Mem abort info:
> > [  975.296669]   ESR = 0x0000000096000003
> > [  975.300425]   EC = 0x25: DABT (current EL), IL = 32 bits
> > [  975.305738]   SET = 0, FnV = 0
> > [  975.308788]   EA = 0, S1PTW = 0
> > [  975.311925]   FSC = 0x03: level 3 address size fault
> > [  975.316888] Data abort info:
> > [  975.319763]   ISV = 0, ISS = 0x00000003, ISS2 = 0x00000000
> > [  975.325245]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > [  975.330292]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> > [  975.335599] swapper pgtable: 4k pages, 48-bit VAs, pgdp=000005016ef6b000
> > [  975.342297] [ffff80008d7cf000] pgd=10000501eddfe003, p4d=10000501eddfe003, pud=10000501eddfd003, pmd=100005017b695003, pte=00687fff84000703
> > [  975.354827] Internal error: Oops: 0000000096000003 [#4] SMP
> > [  975.360392] Modules linked in:
> > 3  975.
> > 63440] CBPrUo:a d0c aPID: 664 Comm: cp Tainted: G      D            6.5.0-rc4-ge28001fb4e07 #23
> > [  975.372822] Hardware name: Xilinx Versal vck190 Eval board revA (DT)
> > [  975.379165] pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > [  975.386119] pc : __memcpy+0x110/0x230
> > [  975.389783] lr : _copy_to_iter+0x3d8/0x4d0
> > [  975.393874] sp : ffff80008dc939a0
> > [  975.397178] x29: ffff80008dc939a0 x28: ffff05013c1bea30 x27: 0000000000001000
> > [  975.404309] x26: 0000000000001000 x25: 0000000000001000 x24: ffff80008d7cf000
> > [  975.411440] x23: 0000040000000000 x22: ffff80008dc93ba0 x21: 0000000000001000
> > [  975.418570] x20: ffff000000000000 x19: 0000000000001000 x18: 0000000000000000
> > [  975.425699] x17: 0000000000000000 x16: 0000000000000000 x15: 0140000000000000
> > [  975.432829] x14: ffff8500a9919000 x13: 0040000000000001 x12: 0000fffef6831000
> > [  975.439958] x11: ffff80008d9cf000 x10: 0000000000000000 x9 : 0000000000000000
> > [  975.447088] x8 : ffff80008d7d0000 x7 : ffff0501addfd358 x6 : 0400000000000001
> > [  975.454217] x5 : ffff0501370e9000 x4 : ffff80008d7d0000 x3 : 0000000000000000
> > [  975.461346] x2 : 0000000000001000 x1 : ffff80008d7cf000 x0 : ffff0501370e8000
> > [  975.468476] Call trace:
> > [  975.470912]  __memcpy+0x110/0x230
> > [  975.474221]  copy_oldmem_page+0x70/0xac
> > [  975.478050]  read_from_oldmem.part.0+0x120/0x188
> > [  975.482663]  read_vmcore+0x14c/0x238
> > [  975.486231]  proc_reg_read_iter+0x84/0xd8
> > [  975.490233]  copy_splice_read+0x160/0x288
> > [  975.494236]  vfs_splice_read+0xac/0x10c
> > [  975.498063]  splice_direct_to_actor+0xa4/0x26c
> > [  975.502498]  do_splice_direct+0x90/0xdc
> > [  975.506325]  do_sendfile+0x344/0x454
> > [  975.509892]  __arm64_sys_sendfile64+0x134/0x140
> > [  975.514415]  invoke_syscall+0x54/0x124
> > [  975.518157]  el0_svc_common.constprop.0+0xc4/0xe4
> > [  975.522854]  do_el0_svc+0x38/0x98
> > [  975.526162]  el0_svc+0x2c/0x84
> > [  975.529211]  el0t_64_sync_handler+0x100/0x12c
> > [  975.533562]  el0t_64_sync+0x190/0x194
> > [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3 (a940342c) 
> > [  975.543302] ---[ end trace 0000000000000000 ]---
> > t message from systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):
> > 
> > kernel[539]: [  975.354827] Internal error: Oops: 0000000096000003 [#4] SMP
> > 
> > 
> > Broadcast message from systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):
> > 
> > kernel[539]: [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3 (a940342c)
> > 
> > Segmentation fault
> > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lrth /proc/vmcore 
> > -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lh /proc/vmcore
> > -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -l /proc/vmcore
> > -r--------    1 root     root     18446603353488633856 Nov  8 14:05 /proc/vmcore
> > 
> > Thanks,
> > Radhey
> > 
> 



From sourabhjain at linux.ibm.com  Sat Aug 12 03:47:52 2023
From: sourabhjain at linux.ibm.com (Sourabh Jain)
Date: Sat, 12 Aug 2023 16:17:52 +0530
Subject: [PATCH v27 2/8] crash: add generic infrastructure for crash
 hotplug support
In-Reply-To: <20230811170642.6696-3-eric.devolder@oracle.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
 <20230811170642.6696-3-eric.devolder@oracle.com>
Message-ID: <188cfb39-7d10-967b-3150-22dac6e95baa@linux.ibm.com>

Hello Eric,

On 11/08/23 22:36, Eric DeVolder wrote:
> To support crash hotplug, a mechanism is needed to update the crash
> elfcorehdr upon CPU or memory changes (eg. hot un/plug or off/
> onlining). The crash elfcorehdr describes the CPUs and memory to
> be written into the vmcore.
>
> To track CPU changes, callbacks are registered with the cpuhp
> mechanism via cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN). The
> crash hotplug elfcorehdr update has no explicit ordering requirement
> (relative to other cpuhp states), so meets the criteria for
> utilizing CPUHP_BP_PREPARE_DYN. CPUHP_BP_PREPARE_DYN is a dynamic
> state and avoids the need to introduce a new state for crash
> hotplug. Also, CPUHP_BP_PREPARE_DYN is the last state in the PREPARE
> group, just prior to the STARTING group, which is very close to the
> CPU starting up in a plug/online situation, or stopping in a unplug/
> offline situation. This minimizes the window of time during an
> actual plug/online or unplug/offline situation in which the
> elfcorehdr would be inaccurate. Note that for a CPU being unplugged
> or offlined, the CPU will still be present in the list of CPUs
> generated by crash_prepare_elf64_headers(). However, there is no
> need to explicitly omit the CPU, see justification in
> 'crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()'.
>
> To track memory changes, a notifier is registered to capture the
> memblock MEM_ONLINE and MEM_OFFLINE events via register_memory_notifier().
>
> The CPU callbacks and memory notifiers invoke crash_handle_hotplug_event()
> which performs needed tasks and then dispatches the event to the
> architecture specific arch_crash_handle_hotplug_event() to update the
> elfcorehdr with the current state of CPUs and memory. During the
> process, the kexec_lock is held.
>
> Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
> Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
> Acked-by: Hari Bathini <hbathini at linux.ibm.com>
> Acked-by: Baoquan He <bhe at redhat.com>
> ---
>   include/linux/crash_core.h |   9 +++
>   include/linux/kexec.h      |  11 +++
>   kernel/Kconfig.kexec       |  31 ++++++++
>   kernel/crash_core.c        | 142 +++++++++++++++++++++++++++++++++++++
>   kernel/kexec_core.c        |   6 ++
>   5 files changed, 199 insertions(+)
>
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index de62a722431e..e14345cc7a22 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -84,4 +84,13 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
>   int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
>   		unsigned long long *crash_size, unsigned long long *crash_base);
>   
> +#define KEXEC_CRASH_HP_NONE			0
> +#define KEXEC_CRASH_HP_ADD_CPU			1
> +#define KEXEC_CRASH_HP_REMOVE_CPU		2
> +#define KEXEC_CRASH_HP_ADD_MEMORY		3
> +#define KEXEC_CRASH_HP_REMOVE_MEMORY		4
> +#define KEXEC_CRASH_HP_INVALID_CPU		-1U
> +
> +struct kimage;
> +
>   #endif /* LINUX_CRASH_CORE_H */
> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
> index 811a90e09698..b9903dd48e24 100644
> --- a/include/linux/kexec.h
> +++ b/include/linux/kexec.h
> @@ -33,6 +33,7 @@ extern note_buf_t __percpu *crash_notes;
>   #include <linux/compat.h>
>   #include <linux/ioport.h>
>   #include <linux/module.h>
> +#include <linux/highmem.h>
>   #include <asm/kexec.h>
>   
>   /* Verify architecture specific macros are defined */
> @@ -360,6 +361,12 @@ struct kimage {
>   	struct purgatory_info purgatory_info;
>   #endif
>   
> +#ifdef CONFIG_CRASH_HOTPLUG
> +	int hp_action;
> +	int elfcorehdr_index;
> +	bool elfcorehdr_updated;
> +#endif
> +
>   #ifdef CONFIG_IMA_KEXEC
>   	/* Virtual address of IMA measurement buffer for kexec syscall */
>   	void *ima_buffer;
> @@ -490,6 +497,10 @@ static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, g
>   static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) { }
>   #endif
>   
> +#ifndef arch_crash_handle_hotplug_event
> +static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
> +#endif
> +

Isn't the above function should be declare under CONFIG_CRASH_HOTPLUG?

Thanks,
Sourabh

>   #else /* !CONFIG_KEXEC_CORE */
>   struct pt_regs;
>   struct task_struct;
> diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
> index ff72e45cfaef..d0a9a5392035 100644
> --- a/kernel/Kconfig.kexec
> +++ b/kernel/Kconfig.kexec
> @@ -113,4 +113,35 @@ config CRASH_DUMP
>   	  For s390, this option also enables zfcpdump.
>   	  See also <file:Documentation/s390/zfcpdump.rst>
>   
> +config CRASH_HOTPLUG
> +	bool "Update the crash elfcorehdr on system configuration changes"
> +	default y
> +	depends on CRASH_DUMP && (HOTPLUG_CPU || MEMORY_HOTPLUG)
> +	depends on ARCH_SUPPORTS_CRASH_HOTPLUG
> +	help
> +	  Enable direct update to the crash elfcorehdr (which contains
> +	  the list of CPUs and memory regions to be dumped upon a crash)
> +	  in response to hot plug/unplug or online/offline of CPUs or
> +	  memory. This is a much more advanced approach than userspace
> +	  attempting that.
> +
> +	  If unsure, say Y.
> +
> +config CRASH_MAX_MEMORY_RANGES
> +	int "Specify the maximum number of memory regions for the elfcorehdr"
> +	default 8192
> +	depends on CRASH_HOTPLUG
> +	help
> +	  For the kexec_file_load() syscall path, specify the maximum number of
> +	  memory regions that the elfcorehdr buffer/segment can accommodate.
> +	  These regions are obtained via walk_system_ram_res(); eg. the
> +	  'System RAM' entries in /proc/iomem.
> +	  This value is combined with NR_CPUS_DEFAULT and multiplied by
> +	  sizeof(Elf64_Phdr) to determine the final elfcorehdr memory buffer/
> +	  segment size.
> +	  The value 8192, for example, covers a (sparsely populated) 1TiB system
> +	  consisting of 128MiB memblocks, while resulting in an elfcorehdr
> +	  memory buffer/segment size under 1MiB. This represents a sane choice
> +	  to accommodate both baremetal and virtual machine configurations.
> +
>   endmenu
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index b7c30b748a16..53d211c690a1 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -11,6 +11,8 @@
>   #include <linux/vmalloc.h>
>   #include <linux/sizes.h>
>   #include <linux/kexec.h>
> +#include <linux/memory.h>
> +#include <linux/cpuhotplug.h>
>   
>   #include <asm/page.h>
>   #include <asm/sections.h>
> @@ -18,6 +20,7 @@
>   #include <crypto/sha1.h>
>   
>   #include "kallsyms_internal.h"
> +#include "kexec_internal.h"
>   
>   /* vmcoreinfo stuff */
>   unsigned char *vmcoreinfo_data;
> @@ -697,3 +700,142 @@ static int __init crash_save_vmcoreinfo_init(void)
>   }
>   
>   subsys_initcall(crash_save_vmcoreinfo_init);
> +
> +#ifdef CONFIG_CRASH_HOTPLUG
> +#undef pr_fmt
> +#define pr_fmt(fmt) "crash hp: " fmt
> +/*
> + * To accurately reflect hot un/plug changes of cpu and memory resources
> + * (including onling and offlining of those resources), the elfcorehdr
> + * (which is passed to the crash kernel via the elfcorehdr= parameter)
> + * must be updated with the new list of CPUs and memories.
> + *
> + * In order to make changes to elfcorehdr, two conditions are needed:
> + * First, the segment containing the elfcorehdr must be large enough
> + * to permit a growing number of resources; the elfcorehdr memory size
> + * is based on NR_CPUS_DEFAULT and CRASH_MAX_MEMORY_RANGES.
> + * Second, purgatory must explicitly exclude the elfcorehdr from the
> + * list of segments it checks (since the elfcorehdr changes and thus
> + * would require an update to purgatory itself to update the digest).
> + */
> +static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
> +{
> +	struct kimage *image;
> +
> +	/* Obtain lock while changing crash information */
> +	if (!kexec_trylock()) {
> +		pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n");
> +		return;
> +	}
> +
> +	/* Check kdump is not loaded */
> +	if (!kexec_crash_image)
> +		goto out;
> +
> +	image = kexec_crash_image;
> +
> +	if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
> +		hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
> +		pr_debug("hp_action %u, cpu %u\n", hp_action, cpu);
> +	else
> +		pr_debug("hp_action %u\n", hp_action);
> +
> +	/*
> +	 * The elfcorehdr_index is set to -1 when the struct kimage
> +	 * is allocated. Find the segment containing the elfcorehdr,
> +	 * if not already found.
> +	 */
> +	if (image->elfcorehdr_index < 0) {
> +		unsigned long mem;
> +		unsigned char *ptr;
> +		unsigned int n;
> +
> +		for (n = 0; n < image->nr_segments; n++) {
> +			mem = image->segment[n].mem;
> +			ptr = kmap_local_page(pfn_to_page(mem >> PAGE_SHIFT));
> +			if (ptr) {
> +				/* The segment containing elfcorehdr */
> +				if (memcmp(ptr, ELFMAG, SELFMAG) == 0)
> +					image->elfcorehdr_index = (int)n;
> +				kunmap_local(ptr);
> +			}
> +		}
> +	}
> +
> +	if (image->elfcorehdr_index < 0) {
> +		pr_err("unable to locate elfcorehdr segment");
> +		goto out;
> +	}
> +
> +	/* Needed in order for the segments to be updated */
> +	arch_kexec_unprotect_crashkres();
> +
> +	/* Differentiate between normal load and hotplug update */
> +	image->hp_action = hp_action;
> +
> +	/* Now invoke arch-specific update handler */
> +	arch_crash_handle_hotplug_event(image);
> +
> +	/* No longer handling a hotplug event */
> +	image->hp_action = KEXEC_CRASH_HP_NONE;
> +	image->elfcorehdr_updated = true;
> +
> +	/* Change back to read-only */
> +	arch_kexec_protect_crashkres();
> +
> +	/* Errors in the callback is not a reason to rollback state */
> +out:
> +	/* Release lock now that update complete */
> +	kexec_unlock();
> +}
> +
> +static int crash_memhp_notifier(struct notifier_block *nb, unsigned long val, void *v)
> +{
> +	switch (val) {
> +	case MEM_ONLINE:
> +		crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_MEMORY,
> +			KEXEC_CRASH_HP_INVALID_CPU);
> +		break;
> +
> +	case MEM_OFFLINE:
> +		crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_MEMORY,
> +			KEXEC_CRASH_HP_INVALID_CPU);
> +		break;
> +	}
> +	return NOTIFY_OK;
> +}
> +
> +static struct notifier_block crash_memhp_nb = {
> +	.notifier_call = crash_memhp_notifier,
> +	.priority = 0
> +};
> +
> +static int crash_cpuhp_online(unsigned int cpu)
> +{
> +	crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_CPU, cpu);
> +	return 0;
> +}
> +
> +static int crash_cpuhp_offline(unsigned int cpu)
> +{
> +	crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_CPU, cpu);
> +	return 0;
> +}
> +
> +static int __init crash_hotplug_init(void)
> +{
> +	int result = 0;
> +
> +	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
> +		register_memory_notifier(&crash_memhp_nb);
> +
> +	if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
> +		result = cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN,
> +			"crash/cpuhp", crash_cpuhp_online, crash_cpuhp_offline);
> +	}
> +
> +	return result;
> +}
> +
> +subsys_initcall(crash_hotplug_init);
> +#endif
> diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
> index e2f2574d8b74..5d323255862a 100644
> --- a/kernel/kexec_core.c
> +++ b/kernel/kexec_core.c
> @@ -277,6 +277,12 @@ struct kimage *do_kimage_alloc_init(void)
>   	/* Initialize the list of unusable pages */
>   	INIT_LIST_HEAD(&image->unusable_pages);
>   
> +#ifdef CONFIG_CRASH_HOTPLUG
> +	image->hp_action = KEXEC_CRASH_HP_NONE;
> +	image->elfcorehdr_index = -1;
> +	image->elfcorehdr_updated = false;
> +#endif
> +
>   	return image;
>   }
>   



From sourabhjain at linux.ibm.com  Sun Aug 13 19:47:34 2023
From: sourabhjain at linux.ibm.com (Sourabh Jain)
Date: Mon, 14 Aug 2023 08:17:34 +0530
Subject: [ANNOUNCE] kexec-tools v2.0.27 preparation
In-Reply-To: <ZNXjs19UOvGEl/5K@vergenet.net>
References: <ZNXjs19UOvGEl/5K@vergenet.net>
Message-ID: <d92decbb-768c-27e7-8c0b-eea662a86670@linux.ibm.com>

Hello Simon,

Can we include below fix?
https://lists.fedoraproject.org/archives/list/kexec at lists.fedoraproject.org/thread/O3W4SLVVFT6J25U5YSXNZ3URZ4QY6SRU/

Thanks,
Sourabh

On 11/08/23 13:00, Simon Horman wrote:
> Hi all,
>
> I am planning to release kexec-tools v2.0.27 in the next two weeks
> to roughly coincide with the release of the v6.5 kernel.
>
> I would like to ask interested parties to send any patches they would like
> included in v2.0.27 within one week so they can be considered for inclusion
> in an rc release.
>
> For reference the patches queued up since v2.0.26 are as follows.
>
> Thanks to everyone who has contributed to kexec-tools!
>
> f67c4146d7b5 arm64: Hook up the ZBOOT support as vmlinuz
> fc7b83bdf734 arm64: Add ZBOOT PE containing compressed image support
> f41c4182b0c4 kexec/zboot: Add arch independent zboot support
> 1572b91da7c4 kexec: Introduce a member kernel_fd in kexec_info
> 714fa11590fe kexec/arm64: Simplify the code for zImage
> a8de94e5f033 LoongArch: kdump: Set up kernel image segment
> 4203eaccfa92 kexec: __NR_kexec_file_load is set to undefined on LoongArch
> 63e9a012112e ppc64: Add elf-ppc64 file types/options and an arch specific flag to man page
> 806711fca9e9 x86: add devicetree support
> 29fe5067ed07 kexec: make -a the default
> e63fefd4fc35 ppc64: add --reuse-cmdline parameter support
> 8fc55927f700 kexec-tools 2.0.26.git
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec



From coxu at redhat.com  Sun Aug 13 20:19:07 2023
From: coxu at redhat.com (Coiby Xu)
Date: Mon, 14 Aug 2023 11:19:07 +0800
Subject: [ANNOUNCE] kexec-tools v2.0.27 preparation
In-Reply-To: <d92decbb-768c-27e7-8c0b-eea662a86670@linux.ibm.com>
References: <ZNXjs19UOvGEl/5K@vergenet.net>
 <d92decbb-768c-27e7-8c0b-eea662a86670@linux.ibm.com>
Message-ID: <ymdn754ihzovgpgoikjnpwueqx6zzmgcempvlr6bsn5rx3mwrw@62dmgvygv4eo>

Hi Sourabh,

On Mon, Aug 14, 2023 at 08:17:34AM +0530, Sourabh Jain wrote:
>Hello Simon,
>
>Can we include below fix?
>https://lists.fedoraproject.org/archives/list/kexec at lists.fedoraproject.org/thread/O3W4SLVVFT6J25U5YSXNZ3URZ4QY6SRU/

I'll review your patch! Note the patch is against Fedora's kexec-tools
so it doesn't apply to this upstream kexec-tools. The relationship
between Fedora's kexec-tools and this upstream kexec-tools can be
described as follows,
1. Fedora's kexec-tools packages the kexec binary which is provides by upstream
kexec-tools; 
2. The kexec binary is used for loading the kdump kernel and initrd but
it's not responsible for building the kdump initrd. Fedora's kexec-tools
is used to build the kdump initrd instead. 

Currrently it's WIP to split current Fedora kexec-tools into
kexec-tools, kdump-utils, makedumpfile and eppic. Hopefully after it's
done, you and other uses will no longer be confused by the relationship
between Fedora's kexec-tools and this upstream kexec-tools.

-- 
Best regards,
Coiby



From sourabhjain at linux.ibm.com  Sun Aug 13 20:50:09 2023
From: sourabhjain at linux.ibm.com (Sourabh Jain)
Date: Mon, 14 Aug 2023 09:20:09 +0530
Subject: [ANNOUNCE] kexec-tools v2.0.27 preparation
In-Reply-To: <ymdn754ihzovgpgoikjnpwueqx6zzmgcempvlr6bsn5rx3mwrw@62dmgvygv4eo>
References: <ZNXjs19UOvGEl/5K@vergenet.net>
 <d92decbb-768c-27e7-8c0b-eea662a86670@linux.ibm.com>
 <ymdn754ihzovgpgoikjnpwueqx6zzmgcempvlr6bsn5rx3mwrw@62dmgvygv4eo>
Message-ID: <a35b5b18-de90-5570-a296-44d110709a29@linux.ibm.com>

Hello Coiby,

On 14/08/23 08:49, Coiby Xu wrote:
> Hi Sourabh,
>
> On Mon, Aug 14, 2023 at 08:17:34AM +0530, Sourabh Jain wrote:
>> Hello Simon,
>>
>> Can we include below fix?
>> https://lists.fedoraproject.org/archives/list/kexec at lists.fedoraproject.org/thread/O3W4SLVVFT6J25U5YSXNZ3URZ4QY6SRU/ 
>>
>
> I'll review your patch! Note the patch is against Fedora's kexec-tools
> so it doesn't apply to this upstream kexec-tools. The relationship
> between Fedora's kexec-tools and this upstream kexec-tools can be
> described as follows,
> 1. Fedora's kexec-tools packages the kexec binary which is provides by 
> upstream
> kexec-tools; 2. The kexec binary is used for loading the kdump kernel 
> and initrd but
> it's not responsible for building the kdump initrd. Fedora's kexec-tools
> is used to build the kdump initrd instead.
> Currrently it's WIP to split current Fedora kexec-tools into
> kexec-tools, kdump-utils, makedumpfile and eppic. Hopefully after it's
> done, you and other uses will no longer be confused by the relationship
> between Fedora's kexec-tools and this upstream kexec-tools.

My Thunderbird filters mail from kexec at lists.fedoraproject.org and 
kexec at lists.infradead.org
to the same folder so I got confused. I should update my Thunderbird filter.

Thanks for clarification.
Sourabh



From conor.dooley at microchip.com  Mon Aug 14 04:10:58 2023
From: conor.dooley at microchip.com (Conor Dooley)
Date: Mon, 14 Aug 2023 12:10:58 +0100
Subject: [PATCH 1/2] RISC-V: Use linux,usable-memory-range for crash
 kernel
In-Reply-To: <20230811104117.91406-1-songshuaishuai@tinylab.org>
References: <20230811104117.91406-1-songshuaishuai@tinylab.org>
Message-ID: <20230814-hefty-reformist-ac79d3c6b84b@wendy>

On Fri, Aug 11, 2023 at 06:41:16PM +0800, Song Shuai wrote:
> Now we use "memeory::linux,usable-memory" to indicate the available
> memory for the crash kernel.
> 
> While booting with UEFI, the crash kernel would use efi.memmap to
> re-populate memblock and then first kernel's memory would be corrputed.
> Consequently, the /proc/vmcore file failed to create in my local test.
> 
> And according to "chosen" dtschema [1], the available memory for the
> crash kernel should be held via "chosen::linux,usable-memory-range"
> property which will re-cap memblock even after UEFI's re-population.
> 
> [1]:
> https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/chosen.yaml

nit: Please use Link: tags for links rather than free-form text.

This doesn't apply to fixes, for-next or master. What is your base?

> 
> Signed-off-by: Song Shuai <songshuaishuai at tinylab.org>
> ---
>  kexec/arch/riscv/kexec-riscv.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/kexec/arch/riscv/kexec-riscv.c b/kexec/arch/riscv/kexec-riscv.c
> index fe5dd2d..5aea035 100644
> --- a/kexec/arch/riscv/kexec-riscv.c
> +++ b/kexec/arch/riscv/kexec-riscv.c
> @@ -79,20 +79,20 @@ int load_extra_segments(struct kexec_info *info, uint64_t kernel_base,
>  		}
>  
>  		ret = dtb_add_range_property(&fdt->buf, &fdt->size, start, end,
> -					     "memory", "linux,usable-memory");
> +					     "chosen", "linux,usable-memory-range");
>  		if (ret) {
> -			fprintf(stderr, "Couldn't add usable-memory to fdt\n");
> +			fprintf(stderr, "Couldn't add usable-memory-range to fdt\n");
>  			return ret;
>  		}
>  
>  		max_usable = end;
>  	} else {
>  		/*
> -		 * Make sure we remove elfcorehdr and usable-memory
> +		 * Make sure we remove elfcorehdr and usable-memory-range
>  		 * when switching from crash kernel to a normal one.
>  		 */
>  		dtb_delete_property(fdt->buf, "chosen", "linux,elfcorehdr");
> -		dtb_delete_property(fdt->buf, "memory", "linux,usable-memory");
> +		dtb_delete_property(fdt->buf, "chosen", "linux,usable-memory-range");
>  	}
>  
>  	/* Do we need to include an initrd image ? */
> -- 
> 2.20.1
> 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20230814/7bb611fb/attachment.sig>

From bhe at redhat.com  Mon Aug 14 05:38:00 2023
From: bhe at redhat.com (bhe at redhat.com)
Date: Mon, 14 Aug 2023 20:38:00 +0800
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64
 platform
In-Reply-To: <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>
 <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
Message-ID: <ZNogKKZCp12Q92jh@MiWiFi-R3L-srv>

On 08/11/23 at 01:27pm, Pandey, Radhey Shyam wrote:
> > -----Original Message-----
> > From: bhe at redhat.com <bhe at redhat.com>
> > Sent: Wednesday, August 9, 2023 7:42 AM
> > To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>;
> > piliu at redhat.com
> > Cc: kexec at lists.infradead.org; linux-kernel at vger.kernel.org
> > Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> > arm64 platform
> > 
> > On 08/08/23 at 07:17pm, Pandey, Radhey Shyam wrote:
> > > Hi,
> > >
> > > I am trying to bring up kdump on arm64 platform[1]. But I get "Cannot get
> > kernel _text symbol address".
> > >
> > > Is there some Dump-capture kernel config options that I am missing?
> > >
> > > FYI, copied below complete kexec debug log.
> > >
> > > [1]: https://www.xilinx.com/products/boards-and-kits/vck190.html
> > 
> > Your description isn't clear. You saw the printing, then your kdump kernel
> > loading succeeded or not?
> > 
> > If no, have you tried applying Pingfan's patchset and still saw the issue?
> > 
> > [PATCHv7 0/5] arm64: zboot support
> > https://lore.kernel.org/all/20230803024152.11663-1-piliu at redhat.com/T/#u
> 
> I was able to proceed further with loading with crash kernel on triggering system crash.
> echo c > /proc/sysrq-trigger
> 
> But when I copy /proc/vmcore it throws memory abort. Also I see size of /proc/vmcore really huge (18446603353488633856).
> Any possible guess on what could be wrong?

I didn't reproduce this issue on a arm64 baremetal system with the
latest kernel. From the log, It could be the iov_iter convertion
patch which caused this. Can you revert below patch to see if it works?

5d8de293c224 vmcore: convert copy_oldmem_page() to take an iov_iter

> 
> 
> [   80.733523] Starting crashdump kernel...
> [   80.737435] Bye!
> [    0.000000] Booting Linux on physical CPU 0x0000000001 [0x410fd083]
> [    0.000000] Linux version 6.5.0-rc4-ge28001fb4e07 (radheys at xhdradheys41) (aarch64-xilinx-linux-gcc.real (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39.0.20220819) #23 SMP Fri Aug 11 16:25:34 IST 2023
> <snip>
> 
> 
> 
> xilinx-vck190-20232:/run/media/mmcblk0p1# cat /proc/meminfo | head
> MemTotal:        2092876 kB
> MemFree:         1219928 kB
> MemAvailable:    1166004 kB
> Buffers:              32 kB
> Cached:           756952 kB
> SwapCached:            0 kB
> Active:             1480 kB
> Inactive:          24164 kB
> Active(anon):       1452 kB
> Inactive(anon):    24160 kB
> xilinx-vck190-20232:/run/media/mmcblk0p1# cp /proc/vmcore dump     
> [  975.284865] Unable to handle kernel level 3 address size fault at virtual address ffff80008d7cf000
> [  975.293871] Mem abort info:
> [  975.296669]   ESR = 0x0000000096000003
> [  975.300425]   EC = 0x25: DABT (current EL), IL = 32 bits
> [  975.305738]   SET = 0, FnV = 0
> [  975.308788]   EA = 0, S1PTW = 0
> [  975.311925]   FSC = 0x03: level 3 address size fault
> [  975.316888] Data abort info:
> [  975.319763]   ISV = 0, ISS = 0x00000003, ISS2 = 0x00000000
> [  975.325245]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> [  975.330292]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> [  975.335599] swapper pgtable: 4k pages, 48-bit VAs, pgdp=000005016ef6b000
> [  975.342297] [ffff80008d7cf000] pgd=10000501eddfe003, p4d=10000501eddfe003, pud=10000501eddfd003, pmd=100005017b695003, pte=00687fff84000703
> [  975.354827] Internal error: Oops: 0000000096000003 [#4] SMP
> [  975.360392] Modules linked in:
> 3  975.
> 63440] CBPrUo:a d0c aPID: 664 Comm: cp Tainted: G      D            6.5.0-rc4-ge28001fb4e07 #23
> [  975.372822] Hardware name: Xilinx Versal vck190 Eval board revA (DT)
> [  975.379165] pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [  975.386119] pc : __memcpy+0x110/0x230
> [  975.389783] lr : _copy_to_iter+0x3d8/0x4d0
> [  975.393874] sp : ffff80008dc939a0
> [  975.397178] x29: ffff80008dc939a0 x28: ffff05013c1bea30 x27: 0000000000001000
> [  975.404309] x26: 0000000000001000 x25: 0000000000001000 x24: ffff80008d7cf000
> [  975.411440] x23: 0000040000000000 x22: ffff80008dc93ba0 x21: 0000000000001000
> [  975.418570] x20: ffff000000000000 x19: 0000000000001000 x18: 0000000000000000
> [  975.425699] x17: 0000000000000000 x16: 0000000000000000 x15: 0140000000000000
> [  975.432829] x14: ffff8500a9919000 x13: 0040000000000001 x12: 0000fffef6831000
> [  975.439958] x11: ffff80008d9cf000 x10: 0000000000000000 x9 : 0000000000000000
> [  975.447088] x8 : ffff80008d7d0000 x7 : ffff0501addfd358 x6 : 0400000000000001
> [  975.454217] x5 : ffff0501370e9000 x4 : ffff80008d7d0000 x3 : 0000000000000000
> [  975.461346] x2 : 0000000000001000 x1 : ffff80008d7cf000 x0 : ffff0501370e8000
> [  975.468476] Call trace:
> [  975.470912]  __memcpy+0x110/0x230
> [  975.474221]  copy_oldmem_page+0x70/0xac
> [  975.478050]  read_from_oldmem.part.0+0x120/0x188
> [  975.482663]  read_vmcore+0x14c/0x238
> [  975.486231]  proc_reg_read_iter+0x84/0xd8
> [  975.490233]  copy_splice_read+0x160/0x288
> [  975.494236]  vfs_splice_read+0xac/0x10c
> [  975.498063]  splice_direct_to_actor+0xa4/0x26c
> [  975.502498]  do_splice_direct+0x90/0xdc
> [  975.506325]  do_sendfile+0x344/0x454
> [  975.509892]  __arm64_sys_sendfile64+0x134/0x140
> [  975.514415]  invoke_syscall+0x54/0x124
> [  975.518157]  el0_svc_common.constprop.0+0xc4/0xe4
> [  975.522854]  do_el0_svc+0x38/0x98
> [  975.526162]  el0_svc+0x2c/0x84
> [  975.529211]  el0t_64_sync_handler+0x100/0x12c
> [  975.533562]  el0t_64_sync+0x190/0x194
> [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3 (a940342c) 
> [  975.543302] ---[ end trace 0000000000000000 ]---
> t message from systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):
> 
> kernel[539]: [  975.354827] Internal error: Oops: 0000000096000003 [#4] SMP
> 
> 
> Broadcast message from systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):
> 
> kernel[539]: [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3 (a940342c)
> 
> Segmentation fault
> xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lrth /proc/vmcore 
> -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lh /proc/vmcore
> -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> xilinx-vck190-20232:/run/media/mmcblk0p1# ls -l /proc/vmcore
> -r--------    1 root     root     18446603353488633856 Nov  8 14:05 /proc/vmcore
> 
> Thanks,
> Radhey
> 



From sushring at linux.microsoft.com  Mon Aug 14 14:42:46 2023
From: sushring at linux.microsoft.com (Sush Shringarputale)
Date: Mon, 14 Aug 2023 14:42:46 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
Message-ID: <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>

Hello Mimi,

Thanks for your feedback on this.


On 8/11/2023 6:14 AM, Mimi Zohar wrote:
> Hi Sush, Tushar,
>
> On Tue, 2023-08-01 at 12:12 -0700, Sush Shringarputale wrote:
>> ================================================
>> | A. Problem Statement                         |
>> ================================================
>> Depending on the IMA policy, the IMA log can consume a lot of Kernel
>> memory on
>> the device.  For instance, the events for the following IMA policy
>> entries may
>> need to be measured in certain scenarios, but they can also lead to a
>> verbose
>> IMA log when the device is running for a long period of time.
>> ?????????????????????????????????????????
>> ?# PROC_SUPER_MAGIC                     ?
>> ?measure fsmagic=0x9fa0                 ?
>> ?# SYSFS_MAGIC                          ?
>> ?measure fsmagic=0x62656572             ?
>> ?# DEBUGFS_MAGIC                        ?
>> ?measure fsmagic=0x64626720             ?
>> ?# TMPFS_MAGIC                          ?
>> ?measure fsmagic=0x01021994             ?
>> ?# RAMFS_MAGIC                          ?
>> ?measure fsmagic=0x858458f6             ?
>> ?# SECURITYFS_MAGIC                     ?
>> ?measure fsmagic=0x73636673             ?
>> ?# OVERLAYFS_MAGIC                      ?
>> ?measure fsmagic=0x794c7630             ?
>> ?# log, audit or tmp files              ?
>> ?measure obj_type=var_log_t             ?
>> ?measure obj_type=auditd_log_t          ?
>> ?measure obj_type=tmp_t                 ?
>> ?????????????????????????????????????????
>>
>> Secondly, certain devices are configured to take Kernel updates using Kexec
>> soft-boot.  The IMA log from the previous Kernel gets carried over and the
>> Kernel memory consumption problem worsens when such devices undergo multiple
>> Kexec soft-boots over a long period of time.
>>
>> The above two scenarios can cause IMA log to grow and consume Kernel memory.
>>
>> In addition, a large IMA log can add pressure on the network bandwidth when
>> the attestation client sends it to remote-attestation-service.
>>
>> Truncating IMA log to reclaim memory is not feasible, since it makes the
>> log go
>> out of sync with the TPM PCR quote making remote attestation fail.
>>
>> A sophisticated solution is required which will help relieve the memory
>> pressure on the device and continue supporting remote attestation without
>> disruptions.
> If the problem is kernel memory, then using a single tmpfs file has
> already been proposed [1].  As entries are added to the measurement
> list, they are copied to the tmpfs file and removed from kernel memory.
> Userspace would still access the measurement list via the existing
> securityfs file.
>
> The IMA measurement list is a sequential file, allowing it to be read
> from an offset.  How much or how little of the measuremnt list is read
> by the attestation client and sent to the attestation server is up to
> the attestation client/server.
>
> If the problem is not kernel memory, but memory pressure in general,
> then instead of a tmpfs file, the measurement list could similarly be
> copied to a single persistent file [1].
The suggested approach in this RFC discussion using a vfs_tmpfile was
only discussed but no prototype was created back then.? We are
discussing the approach internally now and will respond with more
details about it.
>> -------------------------------------------------------------------------------
>> ================================================
>> | B. Proposed Solution                         |
>> ================================================
>> In this document, we propose an enhancement to the IMA subsystem to improve
>> the long-running performance by snapshotting the IMA log, while still
>> providing mechanisms to verify its integrity using the PCR quotes.
>>
>> The remainder of the document describes details of the proposed solution
>> in the
>> following sub-sections.
>>    - High-level Work-flow
>>    - Snapshot Triggering Mechanism
>>    - Design Choices for Storing Snapshots
>>    - Attestation-Client and Remote-Attestation-Service Side Changes
>>    - Example Walk-through
>>    - Open Questions
>> -------------------------------------------------------------------------------
>> ================================================
>> | B.1 High-level Work-flow                     |
>> ================================================
>> Pre-requisites:
>> - IMA Integrity guarantees are maintained.
>>
>> The proposed high level work-flow of IMA log snapshotting is as follows:
>> - A user-mode process will trigger the snapshot by opening a file in SysFS
>>     say /sys/kernel/security/ima/snapshot (referred to as
>> sysk_ima_snapshot_file
>>     here onwards).
> Please fix the mailer so that it doesn't wrap sentences.   Adding blank
> lines between bullets would improve readability.
Noted, will do.
>> - The Kernel will get the current TPM PCR values and PCR update counter [2]
>>     and store them as template data in a new IMA event "snapshot_aggregate".
>>     This event will be measured by IMA using critical data measurement
>>     functionality [1].  Recording regular IMA events will be paused while
>>     "snapshot_aggregate" is being computed using the existing IMA mutex lock.
>> - Once the "snapshot_aggregate" is computed and measured in IMA log, the
>> prior
>>     IMA events will be made available in the sysk_ima_snapshot_file.
>> - The UM process will copy those IMA events from sysk_ima_snapshot_file to a
>>     snapshot file on disk chosen by UM (referred to as UM_snapshot_file here
>>     onwards).  The location, file-system type, access permissions etc. of the
>>     UM_snapshot_file would be controlled by UM process itself.
>> - Once UM is done copying the IMA events from sysk_ima_snapshot_file to
>>     UM_snapshot_file, it will indicate to the Kernel that the snapshot can be
>>     finalized by triggering a write with any data to the
>> sysk_ima_snapshot_file.
>>     UM process cannot prevent the IMA log purge operation after this point.
>> - The Kernel will truncate the current IMA log and and clear HTable up
>> to the
>>     "snapshot_aggregate" marker.
>> - The Kernel will measure the PCR update counter as part of measuring
>>     snapshot_aggregate, so that it can be used by the remote attestation
>> service
>>     for detecting missing events.
>> - UM can prevent the IMA log purge by closing the sysk_ima_snapshot_file
>>     without performing a write operation on it.  In this case, while the
>>     "snapshot_aggregate" marker may still be in the log, the event can be
>> ignored
>>     since the previous entries in the IMA log will not be purged.
>>
>> Note:
>> - This work-flow should work when interleaved with Kexec 'load' and
>> 'execute'
>>     events and should not cause IMA log + snapshot to go out of sync with PCR
>>     quotes. The implementation details are omitted from this document for
>>     brevity.
> This design seems overly complex and requires synchronization between
> the "snapshot" record and exporting the records from the measurement
> list.  None of this would be necessary if the measurements were copied
> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
>
> What is the real problem - kernel memory pressure, memory pressure in
> general, or disk space?  Is the intention to remove or offload the
> exported measurements?
The main concern is the memory pressure on both the kernel and the 
attestation client
when it sends the request.? The concern you bring up is valid and we are 
working on
creating a prototype.? There is no intention to remove the exported 
measurements.
- Sush
> Concerns:
> - Pausing extending the measurement list.
>
> [1]
> https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t
>


From eric.devolder at oracle.com  Mon Aug 14 14:44:41 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:41 -0400
Subject: [PATCH v28 3/8] kexec: exclude elfcorehdr from the segment digest
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814214446.6659-4-eric.devolder@oracle.com>

When a crash kernel is loaded via the kexec_file_load() syscall, the
kernel places the various segments (ie crash kernel, crash initrd,
boot_params, elfcorehdr, purgatory, etc) in memory. For those
architectures that utilize purgatory, a hash digest of the segments
is calculated for integrity checking. The digest is embedded into
the purgatory image prior to placing in memory.

Updates to the elfcorehdr in response to CPU and memory changes
would cause the purgatory integrity checking to fail (at crash time,
and no vmcore created). Therefore, the elfcorehdr segment is
explicitly excluded from the purgatory digest, enabling updates to
the elfcorehdr while also avoiding the need to recompute the hash
digest and reload purgatory.

Suggested-by: Baoquan He <bhe at redhat.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 kernel/kexec_file.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index 453b7a513540..e2ec9d7b9a1f 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -726,6 +726,12 @@ static int kexec_calculate_store_digests(struct kimage *image)
 	for (j = i = 0; i < image->nr_segments; i++) {
 		struct kexec_segment *ksegment;
 
+#ifdef CONFIG_CRASH_HOTPLUG
+		/* Exclude elfcorehdr segment to allow future changes via hotplug */
+		if (j == image->elfcorehdr_index)
+			continue;
+#endif
+
 		ksegment = &image->segment[i];
 		/*
 		 * Skip purgatory as it will be modified once we put digest
-- 
2.31.1



From eric.devolder at oracle.com  Mon Aug 14 14:44:45 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:45 -0400
Subject: [PATCH v28 7/8] crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814214446.6659-8-eric.devolder@oracle.com>

The function crash_prepare_elf64_headers() generates the elfcorehdr
which describes the CPUs and memory in the system for the crash kernel.
In particular, it writes out ELF PT_NOTEs for memory regions and the
CPUs in the system.

With respect to the CPUs, the current implementation utilizes
for_each_present_cpu() which means that as CPUs are added and removed,
the elfcorehdr must again be updated to reflect the new set of CPUs.

The reasoning behind the move to use for_each_possible_cpu(), is:

- At kernel boot time, all percpu crash_notes are allocated for all
  possible CPUs; that is, crash_notes are not allocated dynamically
  when CPUs are plugged/unplugged. Thus the crash_notes for each
  possible CPU are always available.

- The crash_prepare_elf64_headers() creates an ELF PT_NOTE per CPU.
  Changing to for_each_possible_cpu() is valid as the crash_notes
  pointed to by each CPU PT_NOTE are present and always valid.

Furthermore, examining a common crash processing path of:

 kernel panic -> crash kernel -> makedumpfile -> 'crash' analyzer
           elfcorehdr      /proc/vmcore     vmcore

reveals how the ELF CPU PT_NOTEs are utilized:

- Upon panic, each CPU is sent an IPI and shuts itself down, recording
 its state in its crash_notes. When all CPUs are shutdown, the
 crash kernel is launched with a pointer to the elfcorehdr.

- The crash kernel via linux/fs/proc/vmcore.c does not examine or
 use the contents of the PT_NOTEs, it exposes them via /proc/vmcore.

- The makedumpfile utility uses /proc/vmcore and reads the CPU
 PT_NOTEs to craft a nr_cpus variable, which is reported in a
 header but otherwise generally unused. Makedumpfile creates the
 vmcore.

- The 'crash' dump analyzer does not appear to reference the CPU
 PT_NOTEs. Instead it looks-up the cpu_[possible|present|onlin]_mask
 symbols and directly examines those structure contents from vmcore
 memory. From that information it is able to determine which CPUs
 are present and online, and locate the corresponding crash_notes.
 Said differently, it appears that 'crash' analyzer does not rely
 on the ELF PT_NOTEs for CPUs; rather it obtains the information
 directly via kernel symbols and the memory within the vmcore.

(There maybe other vmcore generating and analysis tools that do use
these PT_NOTEs, but 'makedumpfile' and 'crash' seems to be the most
common solution.)

This results in the benefit of having all CPUs described in the
elfcorehdr, and therefore reducing the need to re-generate the
elfcorehdr on CPU changes, at the small expense of an additional
56 bytes per PT_NOTE for not-present-but-possible CPUs.

On systems where kexec_file_load() syscall is utilized, all the above
is valid. On systems where kexec_load() syscall is utilized, there
may be the need for the elfcorehdr to be regenerated once. The reason
being that some archs only populate the 'present' CPUs from the
/sys/devices/system/cpus entries, which the userspace 'kexec' utility
uses to generate the userspace-supplied elfcorehdr. In this situation,
one memory or CPU change will rewrite the elfcorehdr via the
crash_prepare_elf64_headers() function and now all possible CPUs will
be described, just as with kexec_file_load() syscall.

Suggested-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 kernel/crash_core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 34dc7bddfd77..7b87db9973a5 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -367,8 +367,8 @@ int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
 	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
 	ehdr->e_phentsize = sizeof(Elf64_Phdr);
 
-	/* Prepare one phdr of type PT_NOTE for each present CPU */
-	for_each_present_cpu(cpu) {
+	/* Prepare one phdr of type PT_NOTE for each possible CPU */
+	for_each_possible_cpu(cpu) {
 		phdr->p_type = PT_NOTE;
 		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
 		phdr->p_offset = phdr->p_paddr = notes_addr;
-- 
2.31.1



From eric.devolder at oracle.com  Mon Aug 14 14:44:40 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:40 -0400
Subject: [PATCH v28 2/8] crash: add generic infrastructure for crash hotplug support
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814214446.6659-3-eric.devolder@oracle.com>

To support crash hotplug, a mechanism is needed to update the crash
elfcorehdr upon CPU or memory changes (eg. hot un/plug or off/
onlining). The crash elfcorehdr describes the CPUs and memory to
be written into the vmcore.

To track CPU changes, callbacks are registered with the cpuhp
mechanism via cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN). The
crash hotplug elfcorehdr update has no explicit ordering requirement
(relative to other cpuhp states), so meets the criteria for
utilizing CPUHP_BP_PREPARE_DYN. CPUHP_BP_PREPARE_DYN is a dynamic
state and avoids the need to introduce a new state for crash
hotplug. Also, CPUHP_BP_PREPARE_DYN is the last state in the PREPARE
group, just prior to the STARTING group, which is very close to the
CPU starting up in a plug/online situation, or stopping in a unplug/
offline situation. This minimizes the window of time during an
actual plug/online or unplug/offline situation in which the
elfcorehdr would be inaccurate. Note that for a CPU being unplugged
or offlined, the CPU will still be present in the list of CPUs
generated by crash_prepare_elf64_headers(). However, there is no
need to explicitly omit the CPU, see justification in
'crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()'.

To track memory changes, a notifier is registered to capture the
memblock MEM_ONLINE and MEM_OFFLINE events via register_memory_notifier().

The CPU callbacks and memory notifiers invoke crash_handle_hotplug_event()
which performs needed tasks and then dispatches the event to the
architecture specific arch_crash_handle_hotplug_event() to update the
elfcorehdr with the current state of CPUs and memory. During the
process, the kexec_lock is held.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |   7 ++
 include/linux/kexec.h      |  11 +++
 kernel/Kconfig.kexec       |  31 ++++++++
 kernel/crash_core.c        | 142 +++++++++++++++++++++++++++++++++++++
 kernel/kexec_core.c        |   6 ++
 5 files changed, 197 insertions(+)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 1e48b1d96404..0c06561bf5ff 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -104,4 +104,11 @@ extern int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_ma
 struct kimage;
 struct kexec_segment;
 
+#define KEXEC_CRASH_HP_NONE			0
+#define KEXEC_CRASH_HP_ADD_CPU			1
+#define KEXEC_CRASH_HP_REMOVE_CPU		2
+#define KEXEC_CRASH_HP_ADD_MEMORY		3
+#define KEXEC_CRASH_HP_REMOVE_MEMORY		4
+#define KEXEC_CRASH_HP_INVALID_CPU		-1U
+
 #endif /* LINUX_CRASH_CORE_H */
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index fb4350db33ff..df395f888915 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -33,6 +33,7 @@ extern note_buf_t __percpu *crash_notes;
 #include <linux/compat.h>
 #include <linux/ioport.h>
 #include <linux/module.h>
+#include <linux/highmem.h>
 #include <asm/kexec.h>
 
 /* Verify architecture specific macros are defined */
@@ -345,6 +346,12 @@ struct kimage {
 	struct purgatory_info purgatory_info;
 #endif
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	int hp_action;
+	int elfcorehdr_index;
+	bool elfcorehdr_updated;
+#endif
+
 #ifdef CONFIG_IMA_KEXEC
 	/* Virtual address of IMA measurement buffer for kexec syscall */
 	void *ima_buffer;
@@ -475,6 +482,10 @@ static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, g
 static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) { }
 #endif
 
+#ifndef arch_crash_handle_hotplug_event
+static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
index ff72e45cfaef..d0a9a5392035 100644
--- a/kernel/Kconfig.kexec
+++ b/kernel/Kconfig.kexec
@@ -113,4 +113,35 @@ config CRASH_DUMP
 	  For s390, this option also enables zfcpdump.
 	  See also <file:Documentation/s390/zfcpdump.rst>
 
+config CRASH_HOTPLUG
+	bool "Update the crash elfcorehdr on system configuration changes"
+	default y
+	depends on CRASH_DUMP && (HOTPLUG_CPU || MEMORY_HOTPLUG)
+	depends on ARCH_SUPPORTS_CRASH_HOTPLUG
+	help
+	  Enable direct update to the crash elfcorehdr (which contains
+	  the list of CPUs and memory regions to be dumped upon a crash)
+	  in response to hot plug/unplug or online/offline of CPUs or
+	  memory. This is a much more advanced approach than userspace
+	  attempting that.
+
+	  If unsure, say Y.
+
+config CRASH_MAX_MEMORY_RANGES
+	int "Specify the maximum number of memory regions for the elfcorehdr"
+	default 8192
+	depends on CRASH_HOTPLUG
+	help
+	  For the kexec_file_load() syscall path, specify the maximum number of
+	  memory regions that the elfcorehdr buffer/segment can accommodate.
+	  These regions are obtained via walk_system_ram_res(); eg. the
+	  'System RAM' entries in /proc/iomem.
+	  This value is combined with NR_CPUS_DEFAULT and multiplied by
+	  sizeof(Elf64_Phdr) to determine the final elfcorehdr memory buffer/
+	  segment size.
+	  The value 8192, for example, covers a (sparsely populated) 1TiB system
+	  consisting of 128MiB memblocks, while resulting in an elfcorehdr
+	  memory buffer/segment size under 1MiB. This represents a sane choice
+	  to accommodate both baremetal and virtual machine configurations.
+
 endmenu
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 336083fba623..40388ad51034 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -11,6 +11,8 @@
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
 #include <linux/kexec.h>
+#include <linux/memory.h>
+#include <linux/cpuhotplug.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -18,6 +20,7 @@
 #include <crypto/sha1.h>
 
 #include "kallsyms_internal.h"
+#include "kexec_internal.h"
 
 /* Per cpu memory for storing cpu states in case of system crash. */
 note_buf_t __percpu *crash_notes;
@@ -733,3 +736,142 @@ static int __init crash_notes_memory_init(void)
 	return 0;
 }
 subsys_initcall(crash_notes_memory_init);
+
+#ifdef CONFIG_CRASH_HOTPLUG
+#undef pr_fmt
+#define pr_fmt(fmt) "crash hp: " fmt
+/*
+ * To accurately reflect hot un/plug changes of cpu and memory resources
+ * (including onling and offlining of those resources), the elfcorehdr
+ * (which is passed to the crash kernel via the elfcorehdr= parameter)
+ * must be updated with the new list of CPUs and memories.
+ *
+ * In order to make changes to elfcorehdr, two conditions are needed:
+ * First, the segment containing the elfcorehdr must be large enough
+ * to permit a growing number of resources; the elfcorehdr memory size
+ * is based on NR_CPUS_DEFAULT and CRASH_MAX_MEMORY_RANGES.
+ * Second, purgatory must explicitly exclude the elfcorehdr from the
+ * list of segments it checks (since the elfcorehdr changes and thus
+ * would require an update to purgatory itself to update the digest).
+ */
+static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
+{
+	struct kimage *image;
+
+	/* Obtain lock while changing crash information */
+	if (!kexec_trylock()) {
+		pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n");
+		return;
+	}
+
+	/* Check kdump is not loaded */
+	if (!kexec_crash_image)
+		goto out;
+
+	image = kexec_crash_image;
+
+	if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
+		hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
+		pr_debug("hp_action %u, cpu %u\n", hp_action, cpu);
+	else
+		pr_debug("hp_action %u\n", hp_action);
+
+	/*
+	 * The elfcorehdr_index is set to -1 when the struct kimage
+	 * is allocated. Find the segment containing the elfcorehdr,
+	 * if not already found.
+	 */
+	if (image->elfcorehdr_index < 0) {
+		unsigned long mem;
+		unsigned char *ptr;
+		unsigned int n;
+
+		for (n = 0; n < image->nr_segments; n++) {
+			mem = image->segment[n].mem;
+			ptr = kmap_local_page(pfn_to_page(mem >> PAGE_SHIFT));
+			if (ptr) {
+				/* The segment containing elfcorehdr */
+				if (memcmp(ptr, ELFMAG, SELFMAG) == 0)
+					image->elfcorehdr_index = (int)n;
+				kunmap_local(ptr);
+			}
+		}
+	}
+
+	if (image->elfcorehdr_index < 0) {
+		pr_err("unable to locate elfcorehdr segment");
+		goto out;
+	}
+
+	/* Needed in order for the segments to be updated */
+	arch_kexec_unprotect_crashkres();
+
+	/* Differentiate between normal load and hotplug update */
+	image->hp_action = hp_action;
+
+	/* Now invoke arch-specific update handler */
+	arch_crash_handle_hotplug_event(image);
+
+	/* No longer handling a hotplug event */
+	image->hp_action = KEXEC_CRASH_HP_NONE;
+	image->elfcorehdr_updated = true;
+
+	/* Change back to read-only */
+	arch_kexec_protect_crashkres();
+
+	/* Errors in the callback is not a reason to rollback state */
+out:
+	/* Release lock now that update complete */
+	kexec_unlock();
+}
+
+static int crash_memhp_notifier(struct notifier_block *nb, unsigned long val, void *v)
+{
+	switch (val) {
+	case MEM_ONLINE:
+		crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_MEMORY,
+			KEXEC_CRASH_HP_INVALID_CPU);
+		break;
+
+	case MEM_OFFLINE:
+		crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_MEMORY,
+			KEXEC_CRASH_HP_INVALID_CPU);
+		break;
+	}
+	return NOTIFY_OK;
+}
+
+static struct notifier_block crash_memhp_nb = {
+	.notifier_call = crash_memhp_notifier,
+	.priority = 0
+};
+
+static int crash_cpuhp_online(unsigned int cpu)
+{
+	crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_CPU, cpu);
+	return 0;
+}
+
+static int crash_cpuhp_offline(unsigned int cpu)
+{
+	crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_CPU, cpu);
+	return 0;
+}
+
+static int __init crash_hotplug_init(void)
+{
+	int result = 0;
+
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		register_memory_notifier(&crash_memhp_nb);
+
+	if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
+		result = cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN,
+			"crash/cpuhp", crash_cpuhp_online, crash_cpuhp_offline);
+	}
+
+	return result;
+}
+
+subsys_initcall(crash_hotplug_init);
+#endif
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index 4ff564bbd8d8..9dc728982d79 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -274,6 +274,12 @@ struct kimage *do_kimage_alloc_init(void)
 	/* Initialize the list of unusable pages */
 	INIT_LIST_HEAD(&image->unusable_pages);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	image->hp_action = KEXEC_CRASH_HP_NONE;
+	image->elfcorehdr_index = -1;
+	image->elfcorehdr_updated = false;
+#endif
+
 	return image;
 }
 
-- 
2.31.1



From eric.devolder at oracle.com  Mon Aug 14 14:44:42 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:42 -0400
Subject: [PATCH v28 4/8] crash: memory and CPU hotplug sysfs attributes
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814214446.6659-5-eric.devolder@oracle.com>

Introduce the crash_hotplug attribute for memory and CPUs for
use by userspace.  These attributes directly facilitate the udev
rule for managing userspace re-loading of the crash kernel upon
hot un/plug changes.

For memory, expose the crash_hotplug attribute to the
/sys/devices/system/memory directory. For example:

 # udevadm info --attribute-walk /sys/devices/system/memory/memory81
  looking at device '/devices/system/memory/memory81':
    KERNEL=="memory81"
    SUBSYSTEM=="memory"
    DRIVER==""
    ATTR{online}=="1"
    ATTR{phys_device}=="0"
    ATTR{phys_index}=="00000051"
    ATTR{removable}=="1"
    ATTR{state}=="online"
    ATTR{valid_zones}=="Movable"

  looking at parent device '/devices/system/memory':
    KERNELS=="memory"
    SUBSYSTEMS==""
    DRIVERS==""
    ATTRS{auto_online_blocks}=="offline"
    ATTRS{block_size_bytes}=="8000000"
    ATTRS{crash_hotplug}=="1"

For CPUs, expose the crash_hotplug attribute to the
/sys/devices/system/cpu directory. For example:

 # udevadm info --attribute-walk /sys/devices/system/cpu/cpu0
  looking at device '/devices/system/cpu/cpu0':
    KERNEL=="cpu0"
    SUBSYSTEM=="cpu"
    DRIVER=="processor"
    ATTR{crash_notes}=="277c38600"
    ATTR{crash_notes_size}=="368"
    ATTR{online}=="1"

  looking at parent device '/devices/system/cpu':
    KERNELS=="cpu"
    SUBSYSTEMS==""
    DRIVERS==""
    ATTRS{crash_hotplug}=="1"
    ATTRS{isolated}==""
    ATTRS{kernel_max}=="8191"
    ATTRS{nohz_full}=="  (null)"
    ATTRS{offline}=="4-7"
    ATTRS{online}=="0-3"
    ATTRS{possible}=="0-7"
    ATTRS{present}=="0-3"

With these sysfs attributes in place, it is possible to efficiently
instruct the udev rule to skip crash kernel reloading for kernels
configured with crash hotplug support.

For example, the following is the proposed udev rule change for RHEL
system 98-kexec.rules (as the first lines of the rule file):

 # The kernel updates the crash elfcorehdr for CPU and memory changes
 SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
 SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

When examined in the context of 98-kexec.rules, the above rules
test if crash_hotplug is set, and if so, the userspace initiated
unload-then-reload of the crash kernel is skipped.

CPU and memory checks are separated in accordance with
CONFIG_HOTPLUG_CPU and CONFIG_MEMORY_HOTPLUG kernel config options.
If an architecture supports, for example, memory hotplug but not
CPU hotplug, then the /sys/devices/system/memory/crash_hotplug
attribute file is present, but the /sys/devices/system/cpu/crash_hotplug
attribute file will NOT be present. Thus the udev rule skips
userspace processing of memory hot un/plug events, but the udev
rule will evaluate false for CPU events, thus allowing userspace to
process CPU hot un/plug events (ie the unload-then-reload of the kdump
capture kernel).

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 Documentation/ABI/testing/sysfs-devices-memory |  8 ++++++++
 .../ABI/testing/sysfs-devices-system-cpu       |  8 ++++++++
 .../admin-guide/mm/memory-hotplug.rst          |  8 ++++++++
 Documentation/core-api/cpu_hotplug.rst         | 18 ++++++++++++++++++
 drivers/base/cpu.c                             | 13 +++++++++++++
 drivers/base/memory.c                          | 13 +++++++++++++
 include/linux/kexec.h                          |  8 ++++++++
 7 files changed, 76 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-devices-memory b/Documentation/ABI/testing/sysfs-devices-memory
index d8b0f80b9e33..a95e0f17c35a 100644
--- a/Documentation/ABI/testing/sysfs-devices-memory
+++ b/Documentation/ABI/testing/sysfs-devices-memory
@@ -110,3 +110,11 @@ Description:
 		link is created for memory section 9 on node0.
 
 		/sys/devices/system/node/node0/memory9 -> ../../memory/memory9
+
+What:		/sys/devices/system/memory/crash_hotplug
+Date:		Aug 2023
+Contact:	Linux kernel mailing list <linux-kernel at vger.kernel.org>
+Description:
+		(RO) indicates whether or not the kernel directly supports
+		modifying the crash elfcorehdr for memory hot un/plug and/or
+		on/offline changes.
diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
index 77942eedf4f6..b52564de2b18 100644
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -687,3 +687,11 @@ Description:
 		(RO) the list of CPUs that are isolated and don't
 		participate in load balancing. These CPUs are set by
 		boot parameter "isolcpus=".
+
+What:		/sys/devices/system/cpu/crash_hotplug
+Date:		Aug 2023
+Contact:	Linux kernel mailing list <linux-kernel at vger.kernel.org>
+Description:
+		(RO) indicates whether or not the kernel directly supports
+		modifying the crash elfcorehdr for CPU hot un/plug and/or
+		on/offline changes.
diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst
index 1b02fe5807cc..eb99d79223a3 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -291,6 +291,14 @@ The following files are currently defined:
 		       Availability depends on the CONFIG_ARCH_MEMORY_PROBE
 		       kernel configuration option.
 ``uevent``	       read-write: generic udev file for device subsystems.
+``crash_hotplug``      read-only: when changes to the system memory map
+		       occur due to hot un/plug of memory, this file contains
+		       '1' if the kernel updates the kdump capture kernel memory
+		       map itself (via elfcorehdr), or '0' if userspace must update
+		       the kdump capture kernel memory map.
+
+		       Availability depends on the CONFIG_MEMORY_HOTPLUG kernel
+		       configuration option.
 ====================== =========================================================
 
 .. note::
diff --git a/Documentation/core-api/cpu_hotplug.rst b/Documentation/core-api/cpu_hotplug.rst
index e6f5bc39cf5c..d6d470d7dda0 100644
--- a/Documentation/core-api/cpu_hotplug.rst
+++ b/Documentation/core-api/cpu_hotplug.rst
@@ -741,6 +741,24 @@ will receive all events. A script like::
 
 can process the event further.
 
+When changes to the CPUs in the system occur, the sysfs file
+/sys/devices/system/cpu/crash_hotplug contains '1' if the kernel
+updates the kdump capture kernel list of CPUs itself (via elfcorehdr),
+or '0' if userspace must update the kdump capture kernel list of CPUs.
+
+The availability depends on the CONFIG_HOTPLUG_CPU kernel configuration
+option.
+
+To skip userspace processing of CPU hot un/plug events for kdump
+(i.e. the unload-then-reload to obtain a current list of CPUs), this sysfs
+file can be used in a udev rule as follows:
+
+ SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
+
+For a CPU hot un/plug event, if the architecture supports kernel updates
+of the elfcorehdr (which contains the list of CPUs), then the rule skips
+the unload-then-reload of the kdump capture kernel.
+
 Kernel Inline Documentations Reference
 ======================================
 
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index fe6690ecf563..43dab03958f1 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -282,6 +282,16 @@ static ssize_t print_cpus_nohz_full(struct device *dev,
 static DEVICE_ATTR(nohz_full, 0444, print_cpus_nohz_full, NULL);
 #endif
 
+#ifdef CONFIG_CRASH_HOTPLUG
+static ssize_t crash_hotplug_show(struct device *dev,
+				     struct device_attribute *attr,
+				     char *buf)
+{
+	return sysfs_emit(buf, "%d\n", crash_hotplug_cpu_support());
+}
+static DEVICE_ATTR_ADMIN_RO(crash_hotplug);
+#endif
+
 static void cpu_device_release(struct device *dev)
 {
 	/*
@@ -469,6 +479,9 @@ static struct attribute *cpu_root_attrs[] = {
 #ifdef CONFIG_NO_HZ_FULL
 	&dev_attr_nohz_full.attr,
 #endif
+#ifdef CONFIG_CRASH_HOTPLUG
+	&dev_attr_crash_hotplug.attr,
+#endif
 #ifdef CONFIG_GENERIC_CPU_AUTOPROBE
 	&dev_attr_modalias.attr,
 #endif
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index b456ac213610..15bb416e58ce 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -490,6 +490,16 @@ static ssize_t auto_online_blocks_store(struct device *dev,
 
 static DEVICE_ATTR_RW(auto_online_blocks);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+#include <linux/kexec.h>
+static ssize_t crash_hotplug_show(struct device *dev,
+				       struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%d\n", crash_hotplug_memory_support());
+}
+static DEVICE_ATTR_RO(crash_hotplug);
+#endif
+
 /*
  * Some architectures will have custom drivers to do this, and
  * will not need to do it from userspace.  The fake hot-add code
@@ -889,6 +899,9 @@ static struct attribute *memory_root_attrs[] = {
 
 	&dev_attr_block_size_bytes.attr,
 	&dev_attr_auto_online_blocks.attr,
+#ifdef CONFIG_CRASH_HOTPLUG
+	&dev_attr_crash_hotplug.attr,
+#endif
 	NULL
 };
 
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index df395f888915..172e9a544928 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -486,6 +486,14 @@ static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) {
 static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
 #endif
 
+#ifndef crash_hotplug_cpu_support
+static inline int crash_hotplug_cpu_support(void) { return 0; }
+#endif
+
+#ifndef crash_hotplug_memory_support
+static inline int crash_hotplug_memory_support(void) { return 0; }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
-- 
2.31.1



From eric.devolder at oracle.com  Mon Aug 14 14:44:46 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:46 -0400
Subject: [PATCH v28 8/8] x86/crash: optimize CPU changes
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814214446.6659-9-eric.devolder@oracle.com>

crash_prepare_elf64_headers() writes into the elfcorehdr an ELF
PT_NOTE for all possible CPUs. As such, subsequent changes to CPUs
(ie. hot un/plug, online/offline) do not need to rewrite the elfcorehdr.

The kimage->file_mode term covers kdump images loaded via the
kexec_file_load() syscall. Since crash_prepare_elf64_headers()
wrote the initial elfcorehdr, no update to the elfcorehdr is
needed for CPU changes.

The kimage->elfcorehdr_updated term covers kdump images loaded via
the kexec_load() syscall. At least one memory or CPU change must occur
to cause crash_prepare_elf64_headers() to rewrite the elfcorehdr.
Afterwards, no update to the elfcorehdr is needed for CPU changes.

This code is intentionally *NOT* hoisted into
crash_handle_hotplug_event() as it would prevent the arch-specific
handler from running for CPU changes. This would break PPC, for
example, which needs to update other information besides the
elfcorehdr, on CPU changes.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/kernel/crash.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 1900efcdf1bc..86d2ca80b9b2 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -469,6 +469,16 @@ void arch_crash_handle_hotplug_event(struct kimage *image)
 	unsigned long mem, memsz;
 	unsigned long elfsz = 0;
 
+	/*
+	 * As crash_prepare_elf64_headers() has already described all
+	 * possible CPUs, there is no need to update the elfcorehdr
+	 * for additional CPU changes.
+	 */
+	if ((image->file_mode || image->elfcorehdr_updated) &&
+		((image->hp_action == KEXEC_CRASH_HP_ADD_CPU) ||
+		(image->hp_action == KEXEC_CRASH_HP_REMOVE_CPU)))
+		return;
+
 	/*
 	 * Create the new elfcorehdr reflecting the changes to CPU and/or
 	 * memory resources.
-- 
2.31.1



From eric.devolder at oracle.com  Mon Aug 14 14:44:38 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:38 -0400
Subject: [PATCH v28 0/8] crash: Kernel handling of CPU and memory hot un/plug
Message-ID: <20230814214446.6659-1-eric.devolder@oracle.com>

This series is dependent upon "refactor Kconfig to consolidate
KEXEC and CRASH options".
 https://lore.kernel.org/lkml/20230712161545.87870-1-eric.devolder at oracle.com/

Once the kdump service is loaded, if changes to CPUs or memory occur,
either by hot un/plug or off/onlining, the crash elfcorehdr must also
be updated.

The elfcorehdr describes to kdump the CPUs and memory in the system,
and any inaccuracies can result in a vmcore with missing CPU context
or memory regions.

The current solution utilizes udev to initiate an unload-then-reload
of the kdump image (eg. kernel, initrd, boot_params, purgatory and
elfcorehdr) by the userspace kexec utility. In the original post I
outlined the significant performance problems related to offloading
this activity to userspace.

This patchset introduces a generic crash handler that registers with
the CPU and memory notifiers. Upon CPU or memory changes, from either
hot un/plug or off/onlining, this generic handler is invoked and
performs important housekeeping, for example obtaining the appropriate
lock, and then invokes an architecture specific handler to do the
appropriate elfcorehdr update.

Note the description in patch 'crash: change crash_prepare_elf64_headers()
to for_each_possible_cpu()' and 'x86/crash: optimize CPU changes' that
enables further optimizations related to CPU plug/unplug/online/offline
performance of elfcorehdr updates.

In the case of x86_64, the arch specific handler generates a new
elfcorehdr, and overwrites the old one in memory; thus no involvement
with userspace needed.

To realize the benefits/test this patchset, one must make a couple
of minor changes to userspace:

 - Prevent udev from updating kdump crash kernel on hot un/plug changes.
   Add the following as the first lines to the RHEL udev rule file
   /usr/lib/udev/rules.d/98-kexec.rules:

   # The kernel updates the crash elfcorehdr for CPU and memory changes
   SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
   SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

   With this changeset applied, the two rules evaluate to false for
   CPU and memory change events and thus skip the userspace
   unload-then-reload of kdump.

 - Change to the kexec_file_load for loading the kdump kernel:
   Eg. on RHEL: in /usr/bin/kdumpctl, change to:
    standard_kexec_args="-p -d -s"
   which adds the -s to select kexec_file_load() syscall.

This kernel patchset also supports kexec_load() with a modified kexec
userspace utility. A working changeset to the kexec userspace utility
is posted to the kexec-tools mailing list here:

 http://lists.infradead.org/pipermail/kexec/2023-May/027049.html

To use the kexec-tools patch, apply, build and install kexec-tools,
then change the kdumpctl's standard_kexec_args to replace the -s with
--hotplug. The removal of -s reverts to the kexec_load syscall and
the addition of --hotplug invokes the changes put forth in the
kexec-tools patch.

Regards,
eric
---
v28: 14aug2023
 - Rebased onto 6.5.0-rc5
 - Erroneously posted v26 as v27, so posting corrects that mistake.

v27: 11aug2023
 https://lore.kernel.org/lkml/20230811170642.6696-1-eric.devolder at oracle.com/
 - Rebased onto 6.5.0-rc5
 - The linux-next and akpm test bots found a build error when just
   PROC_KCORE is configured (with no KEXEC or CRASH), which resulted
   in CRASH_CORE enabled by itself. To solve, the struct crash_mem
   moved from include/linux/kexec.h to include/linux/crash_core.h.
   Similarly, the crash_notes also moved from kernel/kexec.c to
   kernel/crash_core.c.
 - Minor adjustment to arch/x86/kernel/crash.c was also needed to
   avoid unused function build errors for just PROC_KCORE.
 - Spot testing of several architectures did not reveal any further
   build problems (PROC_KCORE, KEXEC, CRASH_DUMP, CRASH_HOTPLUG).

v26: 4aug2023
 https://lore.kernel.org/lkml/20230804210359.8321-1-eric.devolder at oracle.com/
 - Rebased onto 6.5.0-rc4
 - Dropped the refactor of files drivers/base/cpu|memory.c as unrelated
   to this series.
 - Minor corrections to documentation, per Randy Dunlap and GregKH.

v25: 29jun2023
 https://lore.kernel.org/lkml/20230629192119.6613-1-eric.devolder at oracle.com/
 - Properly applied IS_ENABLED() to the function bodies of callbacks
   in drivers/base/cpu|memory.c.
 - Re-ran compile and run-time testing of the impacted attributes for
   both enabled and not enabled config settings.

v24: 28jun2023
 https://lore.kernel.org/lkml/20230628185215.40707-1-eric.devolder at oracle.com/
 - Rebased onto 6.4.0
 - Included Documentation/ABI/testing entries for the new sysfs
   crash_hotplug attributes, per Greg Kroah-Hartman.
 - Refactored drivers/base/cpu|memory.c to use the .is_visible()
   method for attributes, per Greg Kroah-Hartman.
 - Retained all existing Acks and RBs as the few changes as a result
   of Greg's requests were trivial.

v23: 12jun2023
 https://lore.kernel.org/lkml/20230612210712.683175-1-eric.devolder at oracle.com/
 - Rebased onto 6.4.0-rc6
 - Refactored Kconfig, per Thomas. See series:
   https://lore.kernel.org/lkml/20230612172805.681179-1-eric.devolder at oracle.com/
 - Reworked commit messages to conform to style, per Thomas.
 - Applied Baoquan He Acked-by to kexec_load() patch.
 - Applied Hari Bathini Acked-by for the series.
 - No code changes.

v22: 3may2023
 https://lore.kernel.org/lkml/20230503224145.7405-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0
 - Improved support for kexec_load(), per Hari Bathini. See
   "crash: hotplug support for kexec_load()" which is the only
   change to this series.
 - Applied Baoquan He's Acked-by for all other patches.

v21: 4apr2023
 https://lkml.org/lkml/2023/4/4/1136
 https://lore.kernel.org/lkml/20230404180326.6890-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0-rc5
 - Additional simplification of indentation in crash_handle_hotplug_event(),
   per Baoquan.

v20: 17mar2023
 https://lkml.org/lkml/2023/3/17/1169
 https://lore.kernel.org/lkml/20230317212128.21424-1-eric.devolder at oracle.com/
 - Rebased onto 6.3.0-rc2
 - Defaulting CRASH_HOTPLUG for x86 to Y, per Sourabh.
 - Explicitly initializing image->hp_action, per Baoquan.
 - Simplified kexec_trylock() in crash_handle_hotplug_event(),
   per Baoquan.
 - Applied Sourabh's Reviewed-by to the series.

v19: 6mar2023
 https://lkml.org/lkml/2023/3/6/1358
 https://lore.kernel.org/lkml/20230306162228.8277-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0
 - Did away with offlinecpu, per Thomas Gleixner.
 - Changed to CPUHP_BP_PREPARE_DYN instead of CPUHP_AP_ONLINE_DYN.
 - Did away with elfcorehdr_index_valid, per Sourabh.
 - Convert to for_each_possible_cpu() in crash_prepare_elf64_headers()
   per Sourabh.
 - Small optimization for x86 cpu changes.

v18: 31jan2023
 https://lkml.org/lkml/2023/1/31/1356
 https://lore.kernel.org/lkml/20230131224236.122805-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc6
 - Renamed struct kimage member hotplug_event to hp_action, and
   re-enumerated the KEXEC_CRASH_HP_x items, adding _NONE at 0.
 - Moved to cpuhp state CPUHP_BP_PREPARE_DYN instead of
   CPUHP_AP_ONLINE_DYN in order to minimize window of time CPU
   is not reflected in elfcorehdr.
 - Reworked some of the comments and commit messages to offer
   more of the why, than what, per Thomas Gleixner.

v17: 18jan2023
 https://lkml.org/lkml/2023/1/18/1420
 https://lore.kernel.org/lkml/20230118213544.2128-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc4
 - Moved a bit of code around so that kexec_load()-only builds
   work, per Sourabh.
 - Corrected computation of number of memory region Phdrs needed
   when x86 memory hotplug is not enabled, per Baoquan.

v16: 5jan2023
 https://lkml.org/lkml/2023/1/5/673
 https://lore.kernel.org/lkml/20230105151709.1845-1-eric.devolder at oracle.com/
 - Rebased onto 6.2.0-rc2
 - Corrected error identified by Baoquan.

v15: 9dec2022
 https://lkml.org/lkml/2022/12/9/520
 https://lore.kernel.org/lkml/20221209153656.3284-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc8
 - Replaced arch_un/map_crash_pages() with direct use of
   kun/map_local_pages(), per Boris.
 - Some x86 changes, per Boris.

v14: 16nov2022
 https://lkml.org/lkml/2022/11/16/1645
 https://lore.kernel.org/lkml/20221116214643.6384-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc5
 - Introduced CRASH_HOTPLUG Kconfig item to better fine tune
   compilation of feature components, per Boris.
 - Removed hp_action parameter to arch_crash_handle_hotplug_event()
   as it is unused.

v13: 31oct2022
 https://lkml.org/lkml/2022/10/31/854
 https://lore.kernel.org/lkml/20221031193604.28779-1-eric.devolder at oracle.com/
 - Rebased onto 6.1.0-rc3, which means converting to use the new
   kexec_trylock() away from mutex_lock(kexec_mutex).
 - Moved arch_un/map_crash_pages() into kexec.h and default
   implementation using k/unmap_local_pages().
 - Changed more #ifdef's into IS_ENABLED()
 - Changed CRASH_MAX_MEMORY_RANGES to 8192 from 32768, and it moved
   into x86 crash.c as #define rather Kconfig item, per Boris.
 - Check number of Phdrs against PN_XNUM, max possible.

v12: 9sep2022
 https://lkml.org/lkml/2022/9/9/1358
 https://lore.kernel.org/lkml/20220909210509.6286-1-eric.devolder at oracle.com/
 - Rebased onto 6.0-rc4
 - Addressed some minor formatting items, per Baoquan

v11: 26aug2022
 https://lkml.org/lkml/2022/8/26/963
 https://lore.kernel.org/lkml/20220826173704.1895-1-eric.devolder at oracle.com/
 - Rebased onto 6.0-rc2
 - Redid the rework of __weak to use asm/kexec.h, per Baoquan
 - Reworked some comments and minor items, per Baoquan

v10: 21jul2022
 https://lkml.org/lkml/2022/7/21/1007
 https://lore.kernel.org/lkml/20220721181747.1640-1-eric.devolder at oracle.com/
 - Rebased to 5.19.0-rc7
 - Per Sourabh, corrected build issue with arch_un/map_crash_pages()
   for architectures not supporting this feature.
 - Per David Hildebrand, removed the WARN_ONCE() altogether.
 - Per David Hansen, converted to use of kmap_local_page().
 - Per Baoquan He, replaced use of __weak with the kexec technique.

v9: 13jun2022
 https://lkml.org/lkml/2022/6/13/3382
 https://lore.kernel.org/lkml/20220613224240.79400-1-eric.devolder at oracle.com/
 - Rebased to 5.18.0
 - Per Sourabh, moved crash_prepare_elf64_headers() into common
   crash_core.c to avoid compile issues with kexec_load only path.
 - Per David Hildebrand, replaced mutex_trylock() with mutex_lock().
 - Changed the __weak arch_crash_handle_hotplug_event() to utilize
   WARN_ONCE() instead of WARN(). Fix some formatting issues.
 - Per Sourabh, introduced sysfs attribute crash_hotplug for memory
   and CPUs; for use by userspace (udev) to determine if the kernel
   performs crash hot un/plug support.
 - Per Sourabh, moved the code detecting the elfcorehdr segment from
   arch/x86 into crash_core:handle_hotplug_event() so both kexec_load
   and kexec_file_load can benefit.
 - Updated userspace kexec-tools kexec utility to reflect change to
   using CRASH_MAX_MEMORY_RANGES and get_nr_cpus().
 - Updated the new proposed udev rules to reflect using the sysfs
   attributes crash_hotplug.

v8: 5may2022
 https://lkml.org/lkml/2022/5/5/1133
 https://lore.kernel.org/lkml/20220505184603.1548-1-eric.devolder at oracle.com/
 - Per Borislav Petkov, eliminated CONFIG_CRASH_HOTPLUG in favor
   of CONFIG_HOTPLUG_CPU || CONFIG_MEMORY_HOTPLUG, ie a new define
   is not needed. Also use of IS_ENABLED() rather than #ifdef's.
   Renamed crash_hotplug_handler() to handle_hotplug_event().
   And other corrections.
 - Per Baoquan, minimized the parameters to the arch_crash_
   handle_hotplug_event() to hp_action and cpu.
 - Introduce KEXEC_CRASH_HP_INVALID_CPU definition, per Baoquan.
 - Per Sourabh Jain, renamed and repurposed CRASH_HOTPLUG_ELFCOREHDR_SZ
   to CONFIG_CRASH_MAX_MEMORY_RANGES, mirroring kexec-tools change
   by David Hildebrand. Folded this patch into the x86
   kexec_file_load support patch.

v7: 13apr2022
 https://lkml.org/lkml/2022/4/13/850
 https://lore.kernel.org/lkml/20220413164237.20845-1-eric.devolder at oracle.com/
 - Resolved parameter usage to crash_hotplug_handler(), per Baoquan.

v6: 1apr2022
 https://lkml.org/lkml/2022/4/1/1203
 https://lore.kernel.org/lkml/20220401183040.1624-1-eric.devolder at oracle.com/
 - Reword commit messages and some comment cleanup per Baoquan.
 - Changed elf_index to elfcorehdr_index for clarity.
 - Minor code changes per Baoquan.

v5: 3mar2022
 https://lkml.org/lkml/2022/3/3/674
 https://lore.kernel.org/lkml/20220303162725.49640-1-eric.devolder at oracle.com/
 - Reworded description of CRASH_HOTPLUG_ELFCOREHDR_SZ, per
   David Hildenbrand.
 - Refactored slightly a few patches per Baoquan recommendation.

v4: 9feb2022
 https://lkml.org/lkml/2022/2/9/1406
 https://lore.kernel.org/lkml/20220209195706.51522-1-eric.devolder at oracle.com/
 - Refactored patches per Baoquan suggestsions.
 - A few corrections, per Baoquan.

v3: 10jan2022
 https://lkml.org/lkml/2022/1/10/1212
 https://lore.kernel.org/lkml/20220110195727.1682-1-eric.devolder at oracle.com/
 - Rebasing per Baoquan He request.
 - Changed memory notifier per David Hildenbrand.
 - Providing example kexec userspace change in cover letter.

RFC v2: 7dec2021
 https://lkml.org/lkml/2021/12/7/1088
 https://lore.kernel.org/lkml/20211207195204.1582-1-eric.devolder at oracle.com/
 - Acting upon Baoquan He suggestion of removing elfcorehdr from
   the purgatory list of segments, removed purgatory code from
   patchset, and it is signficiantly simpler now.

RFC v1: 18nov2021
 https://lkml.org/lkml/2021/11/18/845
 https://lore.kernel.org/lkml/20211118174948.37435-1-eric.devolder at oracle.com/
 - working patchset demonstrating kernel handling of hotplug
   updates to x86 elfcorehdr for kexec_file_load

RFC: 14dec2020
 https://lkml.org/lkml/2020/12/14/532
 https://lore.kernel.org/lkml/b04ed259-dc5f-7f30-6661-c26f92d9096a at oracle.com/
 NOTE: s/vmcoreinfo/elfcorehdr/g
 - proposed concept of allowing kernel to handle hotplug update
   of elfcorehdr
---


Eric DeVolder (8):
  crash: move a few code bits to setup support of crash hotplug
  crash: add generic infrastructure for crash hotplug support
  kexec: exclude elfcorehdr from the segment digest
  crash: memory and CPU hotplug sysfs attributes
  x86/crash: add x86 crash hotplug support
  crash: hotplug support for kexec_load()
  crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()
  x86/crash: optimize CPU changes

 .../ABI/testing/sysfs-devices-memory          |   8 +
 .../ABI/testing/sysfs-devices-system-cpu      |   8 +
 .../admin-guide/mm/memory-hotplug.rst         |   8 +
 Documentation/core-api/cpu_hotplug.rst        |  18 +
 arch/x86/Kconfig                              |   3 +
 arch/x86/include/asm/kexec.h                  |  18 +
 arch/x86/kernel/crash.c                       | 142 ++++++-
 drivers/base/cpu.c                            |  13 +
 drivers/base/memory.c                         |  13 +
 include/linux/crash_core.h                    |  27 ++
 include/linux/kexec.h                         |  48 ++-
 include/uapi/linux/kexec.h                    |   1 +
 kernel/Kconfig.kexec                          |  35 ++
 kernel/crash_core.c                           | 391 ++++++++++++++++++
 kernel/kexec.c                                |   5 +
 kernel/kexec_core.c                           |  43 +-
 kernel/kexec_file.c                           | 187 +--------
 kernel/ksysfs.c                               |  15 +
 18 files changed, 741 insertions(+), 242 deletions(-)

-- 
2.31.1



From eric.devolder at oracle.com  Mon Aug 14 14:44:44 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:44 -0400
Subject: [PATCH v28 6/8] crash: hotplug support for kexec_load()
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814214446.6659-7-eric.devolder@oracle.com>

The hotplug support for kexec_load() requires changes to the
userspace kexec-tools and a little extra help from the kernel.

Given a kdump capture kernel loaded via kexec_load(), and a
subsequent hotplug event, the crash hotplug handler finds the
elfcorehdr and rewrites it to reflect the hotplug change.
That is the desired outcome, however, at kernel panic time,
the purgatory integrity check fails (because the elfcorehdr
changed), and the capture kernel does not boot and no vmcore
is generated.

Therefore, the userspace kexec-tools/kexec must indicate to the
kernel that the elfcorehdr can be modified (because the kexec
excluded the elfcorehdr from the digest, and sized the elfcorehdr
memory buffer appropriately).

To facilitate hotplug support with kexec_load():
 - a new kexec flag KEXEC_UPATE_ELFCOREHDR indicates that it is
   safe for the kernel to modify the kexec_load()'d elfcorehdr
 - the /sys/kernel/crash_elfcorehdr_size node communicates the
   preferred size of the elfcorehdr memory buffer
 - The sysfs crash_hotplug nodes (ie.
   /sys/devices/system/[cpu|memory]/crash_hotplug) dynamically
   take into account kexec_file_load() vs kexec_load() and
   KEXEC_UPDATE_ELFCOREHDR.
   This is critical so that the udev rule processing of crash_hotplug
   is all that is needed to determine if the userspace unload-then-load
   of the kdump image is to be skipped, or not. The proposed udev
   rule change looks like:
   # The kernel updates the crash elfcorehdr for CPU and memory changes
   SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
   SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"

The table below indicates the behavior of kexec_load()'d kdump image
updates (with the new udev crash_hotplug rule in place):

 Kernel |Kexec
 -------+-----+----
 Old    |Old  |New
        |  a  | a
 -------+-----+----
 New    |  a  | b
 -------+-----+----

where kexec 'old' and 'new' delineate kexec-tools has the needed
modifications for the crash hotplug feature, and kernel 'old' and
'new' delineate the kernel supports this crash hotplug feature.

Behavior 'a' indicates the unload-then-reload of the entire kdump
image. For the kexec 'old' column, the unload-then-reload occurs
due to the missing flag KEXEC_UPDATE_ELFCOREHDR. An 'old' kernel
(with 'new' kexec) does not present the crash_hotplug sysfs node,
which leads to the unload-then-reload of the kdump image.

Behavior 'b' indicates the desired optimized behavior of the kernel
directly modifying the elfcorehdr and avoiding the unload-then-reload
of the kdump image.

If the udev rule is not updated with crash_hotplug node check, then
no matter any combination of kernel or kexec is new or old, the
kdump image continues to be unload-then-reload on hotplug changes.

To fully support crash hotplug feature, there needs to be a rollout
of kernel, kexec-tools and udev rule changes. However, the order of
the rollout of these pieces does not matter; kexec_load()'d kdump
images still function for hotplug as-is.

Suggested-by: Hari Bathini <hbathini at linux.ibm.com>
Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/include/asm/kexec.h | 11 +++++++----
 arch/x86/kernel/crash.c      | 27 +++++++++++++++++++++++++++
 include/linux/kexec.h        | 14 ++++++++++++--
 include/uapi/linux/kexec.h   |  1 +
 kernel/Kconfig.kexec         |  4 ++++
 kernel/crash_core.c          | 31 +++++++++++++++++++++++++++++++
 kernel/kexec.c               |  5 +++++
 kernel/ksysfs.c              | 15 +++++++++++++++
 8 files changed, 102 insertions(+), 6 deletions(-)

diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 9143100ea3ea..3be6a98751f0 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -214,14 +214,17 @@ void arch_crash_handle_hotplug_event(struct kimage *image);
 #define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event
 
 #ifdef CONFIG_HOTPLUG_CPU
-static inline int crash_hotplug_cpu_support(void) { return 1; }
-#define crash_hotplug_cpu_support crash_hotplug_cpu_support
+int arch_crash_hotplug_cpu_support(void);
+#define crash_hotplug_cpu_support arch_crash_hotplug_cpu_support
 #endif
 
 #ifdef CONFIG_MEMORY_HOTPLUG
-static inline int crash_hotplug_memory_support(void) { return 1; }
-#define crash_hotplug_memory_support crash_hotplug_memory_support
+int arch_crash_hotplug_memory_support(void);
+#define crash_hotplug_memory_support arch_crash_hotplug_memory_support
 #endif
+
+unsigned int arch_crash_get_elfcorehdr_size(void);
+#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size
 #endif
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 4b6cebceec68..1900efcdf1bc 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -429,6 +429,33 @@ int crash_load_segments(struct kimage *image)
 #undef pr_fmt
 #define pr_fmt(fmt) "crash hp: " fmt
 
+/* These functions provide the value for the sysfs crash_hotplug nodes */
+#ifdef CONFIG_HOTPLUG_CPU
+int arch_crash_hotplug_cpu_support(void)
+{
+	return crash_check_update_elfcorehdr();
+}
+#endif
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+int arch_crash_hotplug_memory_support(void)
+{
+	return crash_check_update_elfcorehdr();
+}
+#endif
+
+unsigned int arch_crash_get_elfcorehdr_size(void)
+{
+	unsigned int sz;
+
+	/* kernel_map, VMCOREINFO and maximum CPUs */
+	sz = 2 + CONFIG_NR_CPUS_DEFAULT;
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		sz += CONFIG_CRASH_MAX_MEMORY_RANGES;
+	sz *= sizeof(Elf64_Phdr);
+	return sz;
+}
+
 /**
  * arch_crash_handle_hotplug_event() - Handle hotplug elfcorehdr changes
  * @image: a pointer to kexec_crash_image
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 172e9a544928..32c78078552c 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -320,6 +320,10 @@ struct kimage {
 	unsigned int preserve_context : 1;
 	/* If set, we are using file mode kexec syscall */
 	unsigned int file_mode:1;
+#ifdef CONFIG_CRASH_HOTPLUG
+	/* If set, allow changes to elfcorehdr of kexec_load'd image */
+	unsigned int update_elfcorehdr:1;
+#endif
 
 #ifdef ARCH_HAS_KIMAGE_ARCH
 	struct kimage_arch arch;
@@ -396,9 +400,9 @@ bool kexec_load_permitted(int kexec_image_type);
 
 /* List of defined/legal kexec flags */
 #ifndef CONFIG_KEXEC_JUMP
-#define KEXEC_FLAGS    KEXEC_ON_CRASH
+#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_UPDATE_ELFCOREHDR)
 #else
-#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT)
+#define KEXEC_FLAGS    (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT | KEXEC_UPDATE_ELFCOREHDR)
 #endif
 
 /* List of defined/legal kexec file flags */
@@ -486,6 +490,8 @@ static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) {
 static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
 #endif
 
+int crash_check_update_elfcorehdr(void);
+
 #ifndef crash_hotplug_cpu_support
 static inline int crash_hotplug_cpu_support(void) { return 0; }
 #endif
@@ -494,6 +500,10 @@ static inline int crash_hotplug_cpu_support(void) { return 0; }
 static inline int crash_hotplug_memory_support(void) { return 0; }
 #endif
 
+#ifndef crash_get_elfcorehdr_size
+static inline unsigned int crash_get_elfcorehdr_size(void) { return 0; }
+#endif
+
 #else /* !CONFIG_KEXEC_CORE */
 struct pt_regs;
 struct task_struct;
diff --git a/include/uapi/linux/kexec.h b/include/uapi/linux/kexec.h
index 981016e05cfa..01766dd839b0 100644
--- a/include/uapi/linux/kexec.h
+++ b/include/uapi/linux/kexec.h
@@ -12,6 +12,7 @@
 /* kexec flags for different usage scenarios */
 #define KEXEC_ON_CRASH		0x00000001
 #define KEXEC_PRESERVE_CONTEXT	0x00000002
+#define KEXEC_UPDATE_ELFCOREHDR	0x00000004
 #define KEXEC_ARCH_MASK		0xffff0000
 
 /*
diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
index d0a9a5392035..6e5350d2d7a6 100644
--- a/kernel/Kconfig.kexec
+++ b/kernel/Kconfig.kexec
@@ -144,4 +144,8 @@ config CRASH_MAX_MEMORY_RANGES
 	  memory buffer/segment size under 1MiB. This represents a sane choice
 	  to accommodate both baremetal and virtual machine configurations.
 
+	  For the kexec_load() syscall path, CRASH_MAX_MEMORY_RANGES is part of
+	  the computation behind the value provided through the
+	  /sys/kernel/crash_elfcorehdr_size attribute.
+
 endmenu
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 40388ad51034..34dc7bddfd77 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -740,6 +740,33 @@ subsys_initcall(crash_notes_memory_init);
 #ifdef CONFIG_CRASH_HOTPLUG
 #undef pr_fmt
 #define pr_fmt(fmt) "crash hp: " fmt
+
+/*
+ * This routine utilized when the crash_hotplug sysfs node is read.
+ * It reflects the kernel's ability/permission to update the crash
+ * elfcorehdr directly.
+ */
+int crash_check_update_elfcorehdr(void)
+{
+	int rc = 0;
+
+	/* Obtain lock while reading crash information */
+	if (!kexec_trylock()) {
+		pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n");
+		return 0;
+	}
+	if (kexec_crash_image) {
+		if (kexec_crash_image->file_mode)
+			rc = 1;
+		else
+			rc = kexec_crash_image->update_elfcorehdr;
+	}
+	/* Release lock now that update complete */
+	kexec_unlock();
+
+	return rc;
+}
+
 /*
  * To accurately reflect hot un/plug changes of cpu and memory resources
  * (including onling and offlining of those resources), the elfcorehdr
@@ -770,6 +797,10 @@ static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
 
 	image = kexec_crash_image;
 
+	/* Check that updating elfcorehdr is permitted */
+	if (!(image->file_mode || image->update_elfcorehdr))
+		goto out;
+
 	if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
 		hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
 		pr_debug("hp_action %u, cpu %u\n", hp_action, cpu);
diff --git a/kernel/kexec.c b/kernel/kexec.c
index 92d301f98776..107f355eac10 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -129,6 +129,11 @@ static int do_kexec_load(unsigned long entry, unsigned long nr_segments,
 	if (flags & KEXEC_PRESERVE_CONTEXT)
 		image->preserve_context = 1;
 
+#ifdef CONFIG_CRASH_HOTPLUG
+	if (flags & KEXEC_UPDATE_ELFCOREHDR)
+		image->update_elfcorehdr = 1;
+#endif
+
 	ret = machine_kexec_prepare(image);
 	if (ret)
 		goto out;
diff --git a/kernel/ksysfs.c b/kernel/ksysfs.c
index aad7a3bfd846..1d4bc493b2f4 100644
--- a/kernel/ksysfs.c
+++ b/kernel/ksysfs.c
@@ -165,6 +165,18 @@ static ssize_t vmcoreinfo_show(struct kobject *kobj,
 }
 KERNEL_ATTR_RO(vmcoreinfo);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+static ssize_t crash_elfcorehdr_size_show(struct kobject *kobj,
+			       struct kobj_attribute *attr, char *buf)
+{
+	unsigned int sz = crash_get_elfcorehdr_size();
+
+	return sysfs_emit(buf, "%u\n", sz);
+}
+KERNEL_ATTR_RO(crash_elfcorehdr_size);
+
+#endif
+
 #endif /* CONFIG_CRASH_CORE */
 
 /* whether file capabilities are enabled */
@@ -255,6 +267,9 @@ static struct attribute * kernel_attrs[] = {
 #endif
 #ifdef CONFIG_CRASH_CORE
 	&vmcoreinfo_attr.attr,
+#ifdef CONFIG_CRASH_HOTPLUG
+	&crash_elfcorehdr_size_attr.attr,
+#endif
 #endif
 #ifndef CONFIG_TINY_RCU
 	&rcu_expedited_attr.attr,
-- 
2.31.1



From eric.devolder at oracle.com  Mon Aug 14 14:44:43 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:43 -0400
Subject: [PATCH v28 5/8] x86/crash: add x86 crash hotplug support
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814214446.6659-6-eric.devolder@oracle.com>

When CPU or memory is hot un/plugged, or off/onlined, the crash
elfcorehdr, which describes the CPUs and memory in the system,
must also be updated.

A new elfcorehdr is generated from the available CPUs and memory
and replaces the existing elfcorehdr. The segment containing the
elfcorehdr is identified at run-time in
crash_core:crash_handle_hotplug_event().

No modifications to purgatory (see 'kexec: exclude elfcorehdr
from the segment digest') or boot_params (as the elfcorehdr=
capture kernel command line parameter pointer remains unchanged
and correct) are needed, just elfcorehdr.

For kexec_file_load(), the elfcorehdr segment size is based on
NR_CPUS and CRASH_MAX_MEMORY_RANGES in order to accommodate a
growing number of CPU and memory resources.

For kexec_load(), the userspace kexec utility needs to size the
elfcorehdr segment in the same/similar manner.

To accommodate kexec_load() syscall in the absence of
kexec_file_load() syscall support, prepare_elf_headers() and
dependents are moved outside of CONFIG_KEXEC_FILE.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/Kconfig             |   3 +
 arch/x86/include/asm/kexec.h |  15 +++++
 arch/x86/kernel/crash.c      | 105 ++++++++++++++++++++++++++++++++---
 3 files changed, 116 insertions(+), 7 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 7082fc10b346..ffc95c3d6abd 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2069,6 +2069,9 @@ config ARCH_SUPPORTS_KEXEC_JUMP
 config ARCH_SUPPORTS_CRASH_DUMP
 	def_bool X86_64 || (X86_32 && HIGHMEM)
 
+config ARCH_SUPPORTS_CRASH_HOTPLUG
+	def_bool y
+
 config PHYSICAL_START
 	hex "Physical address where the kernel is loaded" if (EXPERT || CRASH_DUMP)
 	default "0x1000000"
diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 5b77bbc28f96..9143100ea3ea 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -209,6 +209,21 @@ typedef void crash_vmclear_fn(void);
 extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss;
 extern void kdump_nmi_shootdown_cpus(void);
 
+#ifdef CONFIG_CRASH_HOTPLUG
+void arch_crash_handle_hotplug_event(struct kimage *image);
+#define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event
+
+#ifdef CONFIG_HOTPLUG_CPU
+static inline int crash_hotplug_cpu_support(void) { return 1; }
+#define crash_hotplug_cpu_support crash_hotplug_cpu_support
+#endif
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+static inline int crash_hotplug_memory_support(void) { return 1; }
+#define crash_hotplug_memory_support crash_hotplug_memory_support
+#endif
+#endif
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_X86_KEXEC_H */
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index cdd92ab43cda..4b6cebceec68 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -158,8 +158,7 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 	crash_save_cpu(regs, safe_smp_processor_id());
 }
 
-#ifdef CONFIG_KEXEC_FILE
-
+#if defined(CONFIG_KEXEC_FILE) || defined(CONFIG_CRASH_DUMP)
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -231,7 +230,7 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 
 /* Prepare elf headers. Return addr and size */
 static int prepare_elf_headers(struct kimage *image, void **addr,
-					unsigned long *sz)
+					unsigned long *sz, unsigned long *nr_mem_ranges)
 {
 	struct crash_mem *cmem;
 	int ret;
@@ -249,6 +248,9 @@ static int prepare_elf_headers(struct kimage *image, void **addr,
 	if (ret)
 		goto out;
 
+	/* Return the computed number of memory ranges, for hotplug usage */
+	*nr_mem_ranges = cmem->nr_ranges;
+
 	/* By default prepare 64bit headers */
 	ret =  crash_prepare_elf64_headers(cmem, IS_ENABLED(CONFIG_X86_64), addr, sz);
 
@@ -256,7 +258,9 @@ static int prepare_elf_headers(struct kimage *image, void **addr,
 	vfree(cmem);
 	return ret;
 }
+#endif
 
+#ifdef CONFIG_KEXEC_FILE
 static int add_e820_entry(struct boot_params *params, struct e820_entry *entry)
 {
 	unsigned int nr_e820_entries;
@@ -371,18 +375,42 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 int crash_load_segments(struct kimage *image)
 {
 	int ret;
+	unsigned long pnum = 0;
 	struct kexec_buf kbuf = { .image = image, .buf_min = 0,
 				  .buf_max = ULONG_MAX, .top_down = false };
 
 	/* Prepare elf headers and add a segment */
-	ret = prepare_elf_headers(image, &kbuf.buffer, &kbuf.bufsz);
+	ret = prepare_elf_headers(image, &kbuf.buffer, &kbuf.bufsz, &pnum);
 	if (ret)
 		return ret;
 
-	image->elf_headers = kbuf.buffer;
-	image->elf_headers_sz = kbuf.bufsz;
+	image->elf_headers	= kbuf.buffer;
+	image->elf_headers_sz	= kbuf.bufsz;
+	kbuf.memsz		= kbuf.bufsz;
+
+#ifdef CONFIG_CRASH_HOTPLUG
+	/*
+	 * The elfcorehdr segment size accounts for VMCOREINFO, kernel_map,
+	 * maximum CPUs and maximum memory ranges.
+	 */
+	if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+		pnum = 2 + CONFIG_NR_CPUS_DEFAULT + CONFIG_CRASH_MAX_MEMORY_RANGES;
+	else
+		pnum += 2 + CONFIG_NR_CPUS_DEFAULT;
+
+	if (pnum < (unsigned long)PN_XNUM) {
+		kbuf.memsz = pnum * sizeof(Elf64_Phdr);
+		kbuf.memsz += sizeof(Elf64_Ehdr);
+
+		image->elfcorehdr_index = image->nr_segments;
+
+		/* Mark as usable to crash kernel, else crash kernel fails on boot */
+		image->elf_headers_sz = kbuf.memsz;
+	} else {
+		pr_err("number of Phdrs %lu exceeds max\n", pnum);
+	}
+#endif
 
-	kbuf.memsz = kbuf.bufsz;
 	kbuf.buf_align = ELF_CORE_HEADER_ALIGN;
 	kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
 	ret = kexec_add_buffer(&kbuf);
@@ -395,3 +423,66 @@ int crash_load_segments(struct kimage *image)
 	return ret;
 }
 #endif /* CONFIG_KEXEC_FILE */
+
+#ifdef CONFIG_CRASH_HOTPLUG
+
+#undef pr_fmt
+#define pr_fmt(fmt) "crash hp: " fmt
+
+/**
+ * arch_crash_handle_hotplug_event() - Handle hotplug elfcorehdr changes
+ * @image: a pointer to kexec_crash_image
+ *
+ * Prepare the new elfcorehdr and replace the existing elfcorehdr.
+ */
+void arch_crash_handle_hotplug_event(struct kimage *image)
+{
+	void *elfbuf = NULL, *old_elfcorehdr;
+	unsigned long nr_mem_ranges;
+	unsigned long mem, memsz;
+	unsigned long elfsz = 0;
+
+	/*
+	 * Create the new elfcorehdr reflecting the changes to CPU and/or
+	 * memory resources.
+	 */
+	if (prepare_elf_headers(image, &elfbuf, &elfsz, &nr_mem_ranges)) {
+		pr_err("unable to create new elfcorehdr");
+		goto out;
+	}
+
+	/*
+	 * Obtain address and size of the elfcorehdr segment, and
+	 * check it against the new elfcorehdr buffer.
+	 */
+	mem = image->segment[image->elfcorehdr_index].mem;
+	memsz = image->segment[image->elfcorehdr_index].memsz;
+	if (elfsz > memsz) {
+		pr_err("update elfcorehdr elfsz %lu > memsz %lu",
+			elfsz, memsz);
+		goto out;
+	}
+
+	/*
+	 * Copy new elfcorehdr over the old elfcorehdr at destination.
+	 */
+	old_elfcorehdr = kmap_local_page(pfn_to_page(mem >> PAGE_SHIFT));
+	if (!old_elfcorehdr) {
+		pr_err("mapping elfcorehdr segment failed\n");
+		goto out;
+	}
+
+	/*
+	 * Temporarily invalidate the crash image while the
+	 * elfcorehdr is updated.
+	 */
+	xchg(&kexec_crash_image, NULL);
+	memcpy_flushcache(old_elfcorehdr, elfbuf, elfsz);
+	xchg(&kexec_crash_image, image);
+	kunmap_local(old_elfcorehdr);
+	pr_debug("updated elfcorehdr\n");
+
+out:
+	vfree(elfbuf);
+}
+#endif
-- 
2.31.1



From eric.devolder at oracle.com  Mon Aug 14 14:44:39 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Mon, 14 Aug 2023 17:44:39 -0400
Subject: [PATCH v28 1/8] crash: move a few code bits to setup support of crash hotplug
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814214446.6659-2-eric.devolder@oracle.com>

The crash hotplug support leans on the work for the kexec_file_load()
syscall. To also support the kexec_load() syscall, a few bits of code
need to be move outside of CONFIG_KEXEC_FILE. As such, these bits are
moved out of kexec_file.c and into a common location crash_core.c.

In addition, struct crash_mem and crash_notes were moved to new
locales so that PROC_KCORE, which sets CRASH_CORE alone, builds
correctly.

No functionality change intended.

Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
Acked-by: Hari Bathini <hbathini at linux.ibm.com>
Acked-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |  20 ++++
 include/linux/kexec.h      |  15 ---
 kernel/crash_core.c        | 218 +++++++++++++++++++++++++++++++++++++
 kernel/kexec_core.c        |  37 -------
 kernel/kexec_file.c        | 181 ------------------------------
 5 files changed, 238 insertions(+), 233 deletions(-)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index de62a722431e..1e48b1d96404 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -28,6 +28,8 @@
 				     VMCOREINFO_BYTES)
 
 typedef u32 note_buf_t[CRASH_CORE_NOTE_BYTES/4];
+/* Per cpu memory for storing cpu states in case of system crash. */
+extern note_buf_t __percpu *crash_notes;
 
 void crash_update_vmcoreinfo_safecopy(void *ptr);
 void crash_save_vmcoreinfo(void);
@@ -84,4 +86,22 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
 int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
 
+/* Alignment required for elf header segment */
+#define ELF_CORE_HEADER_ALIGN   4096
+
+struct crash_mem {
+	unsigned int max_nr_ranges;
+	unsigned int nr_ranges;
+	struct range ranges[];
+};
+
+extern int crash_exclude_mem_range(struct crash_mem *mem,
+				   unsigned long long mstart,
+				   unsigned long long mend);
+extern int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
+				       void **addr, unsigned long *sz);
+
+struct kimage;
+struct kexec_segment;
+
 #endif /* LINUX_CRASH_CORE_H */
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 22b5cd24f581..fb4350db33ff 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -230,21 +230,6 @@ static inline int arch_kexec_locate_mem_hole(struct kexec_buf *kbuf)
 }
 #endif
 
-/* Alignment required for elf header segment */
-#define ELF_CORE_HEADER_ALIGN   4096
-
-struct crash_mem {
-	unsigned int max_nr_ranges;
-	unsigned int nr_ranges;
-	struct range ranges[];
-};
-
-extern int crash_exclude_mem_range(struct crash_mem *mem,
-				   unsigned long long mstart,
-				   unsigned long long mend);
-extern int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
-				       void **addr, unsigned long *sz);
-
 #ifndef arch_kexec_apply_relocations_add
 /*
  * arch_kexec_apply_relocations_add - apply relocations of type RELA
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 90ce1dfd591c..336083fba623 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -10,6 +10,7 @@
 #include <linux/utsname.h>
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
+#include <linux/kexec.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -18,6 +19,9 @@
 
 #include "kallsyms_internal.h"
 
+/* Per cpu memory for storing cpu states in case of system crash. */
+note_buf_t __percpu *crash_notes;
+
 /* vmcoreinfo stuff */
 unsigned char *vmcoreinfo_data;
 size_t vmcoreinfo_size;
@@ -314,6 +318,187 @@ static int __init parse_crashkernel_dummy(char *arg)
 }
 early_param("crashkernel", parse_crashkernel_dummy);
 
+int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
+			  void **addr, unsigned long *sz)
+{
+	Elf64_Ehdr *ehdr;
+	Elf64_Phdr *phdr;
+	unsigned long nr_cpus = num_possible_cpus(), nr_phdr, elf_sz;
+	unsigned char *buf;
+	unsigned int cpu, i;
+	unsigned long long notes_addr;
+	unsigned long mstart, mend;
+
+	/* extra phdr for vmcoreinfo ELF note */
+	nr_phdr = nr_cpus + 1;
+	nr_phdr += mem->nr_ranges;
+
+	/*
+	 * kexec-tools creates an extra PT_LOAD phdr for kernel text mapping
+	 * area (for example, ffffffff80000000 - ffffffffa0000000 on x86_64).
+	 * I think this is required by tools like gdb. So same physical
+	 * memory will be mapped in two ELF headers. One will contain kernel
+	 * text virtual addresses and other will have __va(physical) addresses.
+	 */
+
+	nr_phdr++;
+	elf_sz = sizeof(Elf64_Ehdr) + nr_phdr * sizeof(Elf64_Phdr);
+	elf_sz = ALIGN(elf_sz, ELF_CORE_HEADER_ALIGN);
+
+	buf = vzalloc(elf_sz);
+	if (!buf)
+		return -ENOMEM;
+
+	ehdr = (Elf64_Ehdr *)buf;
+	phdr = (Elf64_Phdr *)(ehdr + 1);
+	memcpy(ehdr->e_ident, ELFMAG, SELFMAG);
+	ehdr->e_ident[EI_CLASS] = ELFCLASS64;
+	ehdr->e_ident[EI_DATA] = ELFDATA2LSB;
+	ehdr->e_ident[EI_VERSION] = EV_CURRENT;
+	ehdr->e_ident[EI_OSABI] = ELF_OSABI;
+	memset(ehdr->e_ident + EI_PAD, 0, EI_NIDENT - EI_PAD);
+	ehdr->e_type = ET_CORE;
+	ehdr->e_machine = ELF_ARCH;
+	ehdr->e_version = EV_CURRENT;
+	ehdr->e_phoff = sizeof(Elf64_Ehdr);
+	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
+	ehdr->e_phentsize = sizeof(Elf64_Phdr);
+
+	/* Prepare one phdr of type PT_NOTE for each present CPU */
+	for_each_present_cpu(cpu) {
+		phdr->p_type = PT_NOTE;
+		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
+		phdr->p_offset = phdr->p_paddr = notes_addr;
+		phdr->p_filesz = phdr->p_memsz = sizeof(note_buf_t);
+		(ehdr->e_phnum)++;
+		phdr++;
+	}
+
+	/* Prepare one PT_NOTE header for vmcoreinfo */
+	phdr->p_type = PT_NOTE;
+	phdr->p_offset = phdr->p_paddr = paddr_vmcoreinfo_note();
+	phdr->p_filesz = phdr->p_memsz = VMCOREINFO_NOTE_SIZE;
+	(ehdr->e_phnum)++;
+	phdr++;
+
+	/* Prepare PT_LOAD type program header for kernel text region */
+	if (need_kernel_map) {
+		phdr->p_type = PT_LOAD;
+		phdr->p_flags = PF_R|PF_W|PF_X;
+		phdr->p_vaddr = (unsigned long) _text;
+		phdr->p_filesz = phdr->p_memsz = _end - _text;
+		phdr->p_offset = phdr->p_paddr = __pa_symbol(_text);
+		ehdr->e_phnum++;
+		phdr++;
+	}
+
+	/* Go through all the ranges in mem->ranges[] and prepare phdr */
+	for (i = 0; i < mem->nr_ranges; i++) {
+		mstart = mem->ranges[i].start;
+		mend = mem->ranges[i].end;
+
+		phdr->p_type = PT_LOAD;
+		phdr->p_flags = PF_R|PF_W|PF_X;
+		phdr->p_offset  = mstart;
+
+		phdr->p_paddr = mstart;
+		phdr->p_vaddr = (unsigned long) __va(mstart);
+		phdr->p_filesz = phdr->p_memsz = mend - mstart + 1;
+		phdr->p_align = 0;
+		ehdr->e_phnum++;
+		pr_debug("Crash PT_LOAD ELF header. phdr=%p vaddr=0x%llx, paddr=0x%llx, sz=0x%llx e_phnum=%d p_offset=0x%llx\n",
+			phdr, phdr->p_vaddr, phdr->p_paddr, phdr->p_filesz,
+			ehdr->e_phnum, phdr->p_offset);
+		phdr++;
+	}
+
+	*addr = buf;
+	*sz = elf_sz;
+	return 0;
+}
+
+int crash_exclude_mem_range(struct crash_mem *mem,
+			    unsigned long long mstart, unsigned long long mend)
+{
+	int i, j;
+	unsigned long long start, end, p_start, p_end;
+	struct range temp_range = {0, 0};
+
+	for (i = 0; i < mem->nr_ranges; i++) {
+		start = mem->ranges[i].start;
+		end = mem->ranges[i].end;
+		p_start = mstart;
+		p_end = mend;
+
+		if (mstart > end || mend < start)
+			continue;
+
+		/* Truncate any area outside of range */
+		if (mstart < start)
+			p_start = start;
+		if (mend > end)
+			p_end = end;
+
+		/* Found completely overlapping range */
+		if (p_start == start && p_end == end) {
+			mem->ranges[i].start = 0;
+			mem->ranges[i].end = 0;
+			if (i < mem->nr_ranges - 1) {
+				/* Shift rest of the ranges to left */
+				for (j = i; j < mem->nr_ranges - 1; j++) {
+					mem->ranges[j].start =
+						mem->ranges[j+1].start;
+					mem->ranges[j].end =
+							mem->ranges[j+1].end;
+				}
+
+				/*
+				 * Continue to check if there are another overlapping ranges
+				 * from the current position because of shifting the above
+				 * mem ranges.
+				 */
+				i--;
+				mem->nr_ranges--;
+				continue;
+			}
+			mem->nr_ranges--;
+			return 0;
+		}
+
+		if (p_start > start && p_end < end) {
+			/* Split original range */
+			mem->ranges[i].end = p_start - 1;
+			temp_range.start = p_end + 1;
+			temp_range.end = end;
+		} else if (p_start != start)
+			mem->ranges[i].end = p_start - 1;
+		else
+			mem->ranges[i].start = p_end + 1;
+		break;
+	}
+
+	/* If a split happened, add the split to array */
+	if (!temp_range.end)
+		return 0;
+
+	/* Split happened */
+	if (i == mem->max_nr_ranges - 1)
+		return -ENOMEM;
+
+	/* Location where new range should go */
+	j = i + 1;
+	if (j < mem->nr_ranges) {
+		/* Move over all ranges one slot towards the end */
+		for (i = mem->nr_ranges - 1; i >= j; i--)
+			mem->ranges[i + 1] = mem->ranges[i];
+	}
+
+	mem->ranges[j].start = temp_range.start;
+	mem->ranges[j].end = temp_range.end;
+	mem->nr_ranges++;
+	return 0;
+}
+
 Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len)
 {
@@ -515,3 +700,36 @@ static int __init crash_save_vmcoreinfo_init(void)
 }
 
 subsys_initcall(crash_save_vmcoreinfo_init);
+
+static int __init crash_notes_memory_init(void)
+{
+	/* Allocate memory for saving cpu registers. */
+	size_t size, align;
+
+	/*
+	 * crash_notes could be allocated across 2 vmalloc pages when percpu
+	 * is vmalloc based . vmalloc doesn't guarantee 2 continuous vmalloc
+	 * pages are also on 2 continuous physical pages. In this case the
+	 * 2nd part of crash_notes in 2nd page could be lost since only the
+	 * starting address and size of crash_notes are exported through sysfs.
+	 * Here round up the size of crash_notes to the nearest power of two
+	 * and pass it to __alloc_percpu as align value. This can make sure
+	 * crash_notes is allocated inside one physical page.
+	 */
+	size = sizeof(note_buf_t);
+	align = min(roundup_pow_of_two(sizeof(note_buf_t)), PAGE_SIZE);
+
+	/*
+	 * Break compile if size is bigger than PAGE_SIZE since crash_notes
+	 * definitely will be in 2 pages with that.
+	 */
+	BUILD_BUG_ON(size > PAGE_SIZE);
+
+	crash_notes = __alloc_percpu(size, align);
+	if (!crash_notes) {
+		pr_warn("Memory allocation for saving cpu register states failed\n");
+		return -ENOMEM;
+	}
+	return 0;
+}
+subsys_initcall(crash_notes_memory_init);
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index e2f2574d8b74..4ff564bbd8d8 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -49,9 +49,6 @@
 
 atomic_t __kexec_lock = ATOMIC_INIT(0);
 
-/* Per cpu memory for storing cpu states in case of system crash. */
-note_buf_t __percpu *crash_notes;
-
 /* Flag to indicate we are going to kexec a new kernel */
 bool kexec_in_progress = false;
 
@@ -1218,40 +1215,6 @@ void crash_save_cpu(struct pt_regs *regs, int cpu)
 	final_note(buf);
 }
 
-static int __init crash_notes_memory_init(void)
-{
-	/* Allocate memory for saving cpu registers. */
-	size_t size, align;
-
-	/*
-	 * crash_notes could be allocated across 2 vmalloc pages when percpu
-	 * is vmalloc based . vmalloc doesn't guarantee 2 continuous vmalloc
-	 * pages are also on 2 continuous physical pages. In this case the
-	 * 2nd part of crash_notes in 2nd page could be lost since only the
-	 * starting address and size of crash_notes are exported through sysfs.
-	 * Here round up the size of crash_notes to the nearest power of two
-	 * and pass it to __alloc_percpu as align value. This can make sure
-	 * crash_notes is allocated inside one physical page.
-	 */
-	size = sizeof(note_buf_t);
-	align = min(roundup_pow_of_two(sizeof(note_buf_t)), PAGE_SIZE);
-
-	/*
-	 * Break compile if size is bigger than PAGE_SIZE since crash_notes
-	 * definitely will be in 2 pages with that.
-	 */
-	BUILD_BUG_ON(size > PAGE_SIZE);
-
-	crash_notes = __alloc_percpu(size, align);
-	if (!crash_notes) {
-		pr_warn("Memory allocation for saving cpu register states failed\n");
-		return -ENOMEM;
-	}
-	return 0;
-}
-subsys_initcall(crash_notes_memory_init);
-
-
 /*
  * Move into place and start executing a preloaded standalone
  * executable.  If nothing was preloaded return an error.
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index dd115571f8fc..453b7a513540 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -1151,184 +1151,3 @@ int kexec_purgatory_get_set_symbol(struct kimage *image, const char *name,
 	return 0;
 }
 #endif /* CONFIG_ARCH_SUPPORTS_KEXEC_PURGATORY */
-
-int crash_exclude_mem_range(struct crash_mem *mem,
-			    unsigned long long mstart, unsigned long long mend)
-{
-	int i, j;
-	unsigned long long start, end, p_start, p_end;
-	struct range temp_range = {0, 0};
-
-	for (i = 0; i < mem->nr_ranges; i++) {
-		start = mem->ranges[i].start;
-		end = mem->ranges[i].end;
-		p_start = mstart;
-		p_end = mend;
-
-		if (mstart > end || mend < start)
-			continue;
-
-		/* Truncate any area outside of range */
-		if (mstart < start)
-			p_start = start;
-		if (mend > end)
-			p_end = end;
-
-		/* Found completely overlapping range */
-		if (p_start == start && p_end == end) {
-			mem->ranges[i].start = 0;
-			mem->ranges[i].end = 0;
-			if (i < mem->nr_ranges - 1) {
-				/* Shift rest of the ranges to left */
-				for (j = i; j < mem->nr_ranges - 1; j++) {
-					mem->ranges[j].start =
-						mem->ranges[j+1].start;
-					mem->ranges[j].end =
-							mem->ranges[j+1].end;
-				}
-
-				/*
-				 * Continue to check if there are another overlapping ranges
-				 * from the current position because of shifting the above
-				 * mem ranges.
-				 */
-				i--;
-				mem->nr_ranges--;
-				continue;
-			}
-			mem->nr_ranges--;
-			return 0;
-		}
-
-		if (p_start > start && p_end < end) {
-			/* Split original range */
-			mem->ranges[i].end = p_start - 1;
-			temp_range.start = p_end + 1;
-			temp_range.end = end;
-		} else if (p_start != start)
-			mem->ranges[i].end = p_start - 1;
-		else
-			mem->ranges[i].start = p_end + 1;
-		break;
-	}
-
-	/* If a split happened, add the split to array */
-	if (!temp_range.end)
-		return 0;
-
-	/* Split happened */
-	if (i == mem->max_nr_ranges - 1)
-		return -ENOMEM;
-
-	/* Location where new range should go */
-	j = i + 1;
-	if (j < mem->nr_ranges) {
-		/* Move over all ranges one slot towards the end */
-		for (i = mem->nr_ranges - 1; i >= j; i--)
-			mem->ranges[i + 1] = mem->ranges[i];
-	}
-
-	mem->ranges[j].start = temp_range.start;
-	mem->ranges[j].end = temp_range.end;
-	mem->nr_ranges++;
-	return 0;
-}
-
-int crash_prepare_elf64_headers(struct crash_mem *mem, int need_kernel_map,
-			  void **addr, unsigned long *sz)
-{
-	Elf64_Ehdr *ehdr;
-	Elf64_Phdr *phdr;
-	unsigned long nr_cpus = num_possible_cpus(), nr_phdr, elf_sz;
-	unsigned char *buf;
-	unsigned int cpu, i;
-	unsigned long long notes_addr;
-	unsigned long mstart, mend;
-
-	/* extra phdr for vmcoreinfo ELF note */
-	nr_phdr = nr_cpus + 1;
-	nr_phdr += mem->nr_ranges;
-
-	/*
-	 * kexec-tools creates an extra PT_LOAD phdr for kernel text mapping
-	 * area (for example, ffffffff80000000 - ffffffffa0000000 on x86_64).
-	 * I think this is required by tools like gdb. So same physical
-	 * memory will be mapped in two ELF headers. One will contain kernel
-	 * text virtual addresses and other will have __va(physical) addresses.
-	 */
-
-	nr_phdr++;
-	elf_sz = sizeof(Elf64_Ehdr) + nr_phdr * sizeof(Elf64_Phdr);
-	elf_sz = ALIGN(elf_sz, ELF_CORE_HEADER_ALIGN);
-
-	buf = vzalloc(elf_sz);
-	if (!buf)
-		return -ENOMEM;
-
-	ehdr = (Elf64_Ehdr *)buf;
-	phdr = (Elf64_Phdr *)(ehdr + 1);
-	memcpy(ehdr->e_ident, ELFMAG, SELFMAG);
-	ehdr->e_ident[EI_CLASS] = ELFCLASS64;
-	ehdr->e_ident[EI_DATA] = ELFDATA2LSB;
-	ehdr->e_ident[EI_VERSION] = EV_CURRENT;
-	ehdr->e_ident[EI_OSABI] = ELF_OSABI;
-	memset(ehdr->e_ident + EI_PAD, 0, EI_NIDENT - EI_PAD);
-	ehdr->e_type = ET_CORE;
-	ehdr->e_machine = ELF_ARCH;
-	ehdr->e_version = EV_CURRENT;
-	ehdr->e_phoff = sizeof(Elf64_Ehdr);
-	ehdr->e_ehsize = sizeof(Elf64_Ehdr);
-	ehdr->e_phentsize = sizeof(Elf64_Phdr);
-
-	/* Prepare one phdr of type PT_NOTE for each present CPU */
-	for_each_present_cpu(cpu) {
-		phdr->p_type = PT_NOTE;
-		notes_addr = per_cpu_ptr_to_phys(per_cpu_ptr(crash_notes, cpu));
-		phdr->p_offset = phdr->p_paddr = notes_addr;
-		phdr->p_filesz = phdr->p_memsz = sizeof(note_buf_t);
-		(ehdr->e_phnum)++;
-		phdr++;
-	}
-
-	/* Prepare one PT_NOTE header for vmcoreinfo */
-	phdr->p_type = PT_NOTE;
-	phdr->p_offset = phdr->p_paddr = paddr_vmcoreinfo_note();
-	phdr->p_filesz = phdr->p_memsz = VMCOREINFO_NOTE_SIZE;
-	(ehdr->e_phnum)++;
-	phdr++;
-
-	/* Prepare PT_LOAD type program header for kernel text region */
-	if (need_kernel_map) {
-		phdr->p_type = PT_LOAD;
-		phdr->p_flags = PF_R|PF_W|PF_X;
-		phdr->p_vaddr = (unsigned long) _text;
-		phdr->p_filesz = phdr->p_memsz = _end - _text;
-		phdr->p_offset = phdr->p_paddr = __pa_symbol(_text);
-		ehdr->e_phnum++;
-		phdr++;
-	}
-
-	/* Go through all the ranges in mem->ranges[] and prepare phdr */
-	for (i = 0; i < mem->nr_ranges; i++) {
-		mstart = mem->ranges[i].start;
-		mend = mem->ranges[i].end;
-
-		phdr->p_type = PT_LOAD;
-		phdr->p_flags = PF_R|PF_W|PF_X;
-		phdr->p_offset  = mstart;
-
-		phdr->p_paddr = mstart;
-		phdr->p_vaddr = (unsigned long) __va(mstart);
-		phdr->p_filesz = phdr->p_memsz = mend - mstart + 1;
-		phdr->p_align = 0;
-		ehdr->e_phnum++;
-		pr_debug("Crash PT_LOAD ELF header. phdr=%p vaddr=0x%llx, paddr=0x%llx, sz=0x%llx e_phnum=%d p_offset=0x%llx\n",
-			phdr, phdr->p_vaddr, phdr->p_paddr, phdr->p_filesz,
-			ehdr->e_phnum, phdr->p_offset);
-		phdr++;
-	}
-
-	*addr = buf;
-	*sz = elf_sz;
-	return 0;
-}
-- 
2.31.1



From zohar at linux.ibm.com  Mon Aug 14 15:02:23 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Mon, 14 Aug 2023 18:02:23 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
	 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
Message-ID: <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>

On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
> > This design seems overly complex and requires synchronization between
> > the "snapshot" record and exporting the records from the measurement
> > list.  None of this would be necessary if the measurements were copied
> > from kernel memory to a backing file (e.g. tmpfs), as described in [1].
> >
> > What is the real problem - kernel memory pressure, memory pressure in
> > general, or disk space?  Is the intention to remove or offload the
> > exported measurements?

> The main concern is the memory pressure on both the kernel and the 
> attestation client
> when it sends the request.  The concern you bring up is valid and we are 
> working on
> creating a prototype.  There is no intention to remove the exported 
> measurements.

Glad to hear that you're not intending to remove the exported
measurements.

Defining and including a new record in the measurement list measurement
is fine, if it helps with attestation and doesn't require pausing the
measurements.

-- 
thanks,

Mimi



From akpm at linux-foundation.org  Mon Aug 14 15:33:38 2023
From: akpm at linux-foundation.org (Andrew Morton)
Date: Mon, 14 Aug 2023 15:33:38 -0700
Subject: [PATCH v28 0/8] crash: Kernel handling of CPU and memory hot
 un/plug
In-Reply-To: <20230814214446.6659-1-eric.devolder@oracle.com>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
Message-ID: <20230814153338.79164562f1d2810a18f495dd@linux-foundation.org>

On Mon, 14 Aug 2023 17:44:38 -0400 Eric DeVolder <eric.devolder at oracle.com> wrote:

> This series is dependent upon "refactor Kconfig to consolidate
> KEXEC and CRASH options".
>  https://lore.kernel.org/lkml/20230712161545.87870-1-eric.devolder at oracle.com/
> 
> Once the kdump service is loaded, if changes to CPUs or memory occur,
> either by hot un/plug or off/onlining, the crash elfcorehdr must also
> be updated.

Thanks, I updated branch mm-nonmm-unstable to this version.


From songshuaishuai at tinylab.org  Mon Aug 14 19:49:12 2023
From: songshuaishuai at tinylab.org (Song Shuai)
Date: Tue, 15 Aug 2023 10:49:12 +0800
Subject: [PATCH 1/2] RISC-V: Use linux,usable-memory-range for crash
 kernel
In-Reply-To: <20230814-hefty-reformist-ac79d3c6b84b@wendy>
References: <20230811104117.91406-1-songshuaishuai@tinylab.org>
 <20230814-hefty-reformist-ac79d3c6b84b@wendy>
Message-ID: <1815D8529965B5FE+f6e31844-3c4e-6290-cf28-6822fb7d34ea@tinylab.org>



? 2023/8/14 19:10, Conor Dooley ??:
> On Fri, Aug 11, 2023 at 06:41:16PM +0800, Song Shuai wrote:
>> Now we use "memeory::linux,usable-memory" to indicate the available
>> memory for the crash kernel.
>>
>> While booting with UEFI, the crash kernel would use efi.memmap to
>> re-populate memblock and then first kernel's memory would be corrputed.
>> Consequently, the /proc/vmcore file failed to create in my local test.
>>
>> And according to "chosen" dtschema [1], the available memory for the
>> crash kernel should be held via "chosen::linux,usable-memory-range"
>> property which will re-cap memblock even after UEFI's re-population.
>>
>> [1]:
>> https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/chosen.yaml
> 
> nit: Please use Link: tags for links rather than free-form text.
> 
> This doesn't apply to fixes, for-next or master. What is your base?
> 
Actually, I sent these patches to kexec-tools/build-test-riscv-v2,
perhaps I shouldn't cc the linux-riscv mailing list.
Sorry for bothering you.
>>
>> Signed-off-by: Song Shuai <songshuaishuai at tinylab.org>
>> ---
>>   kexec/arch/riscv/kexec-riscv.c | 8 ++++----
>>   1 file changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/kexec/arch/riscv/kexec-riscv.c b/kexec/arch/riscv/kexec-riscv.c
>> index fe5dd2d..5aea035 100644
>> --- a/kexec/arch/riscv/kexec-riscv.c
>> +++ b/kexec/arch/riscv/kexec-riscv.c
>> @@ -79,20 +79,20 @@ int load_extra_segments(struct kexec_info *info, uint64_t kernel_base,
>>   		}
>>   
>>   		ret = dtb_add_range_property(&fdt->buf, &fdt->size, start, end,
>> -					     "memory", "linux,usable-memory");
>> +					     "chosen", "linux,usable-memory-range");
>>   		if (ret) {
>> -			fprintf(stderr, "Couldn't add usable-memory to fdt\n");
>> +			fprintf(stderr, "Couldn't add usable-memory-range to fdt\n");
>>   			return ret;
>>   		}
>>   
>>   		max_usable = end;
>>   	} else {
>>   		/*
>> -		 * Make sure we remove elfcorehdr and usable-memory
>> +		 * Make sure we remove elfcorehdr and usable-memory-range
>>   		 * when switching from crash kernel to a normal one.
>>   		 */
>>   		dtb_delete_property(fdt->buf, "chosen", "linux,elfcorehdr");
>> -		dtb_delete_property(fdt->buf, "memory", "linux,usable-memory");
>> +		dtb_delete_property(fdt->buf, "chosen", "linux,usable-memory-range");
>>   	}
>>   
>>   	/* Do we need to include an initrd image ? */
>> -- 
>> 2.20.1
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec

-- 
Thanks
Song Shuai


From k-hagio-ab at nec.com  Mon Aug 14 22:07:38 2023
From: k-hagio-ab at nec.com (=?utf-8?B?SEFHSU8gS0FaVUhJVE8o6JCp5bC+44CA5LiA5LuBKQ==?=)
Date: Tue, 15 Aug 2023 05:07:38 +0000
Subject: [Crash-utility PATCH V2] RISCV64: Use va_kernel_pa_offset in
 VTOP()
In-Reply-To: <20230804091559.3005820-1-suagrfillet@gmail.com>
References: <20230804091559.3005820-1-suagrfillet@gmail.com>
Message-ID: <89ee81ef-2702-15b0-6246-aae92c14da1e@nec.com>

On 2023/08/04 18:15, Song Shuai wrote:
> Since RISC-V Linux v6.4, the commit 3335068f8721 ("riscv: Use
> PUD/P4D/PGD pages for the linear mapping") changes phys_ram_base from
> the physical start of the kernel to the actual start of the DRAM.
> 
> The Crash's VTOP() still uses phys_ram_base and kernel_map.virt_addr
> to translate kernel virtual address, that made Crash boot failed with
> Linux v6.4 and later version.
> 
> Let Linux export kernel_map.va_kernel_pa_offset in v6.5 and backported
> v6.4.0 stable, so Crash can use "va_kernel_pa_offset" to translate the
> kernel virtual address in VTOP() correctly.
> 
> Signed-off-by: Song Shuai <suagrfillet at gmail.com>
> ---
> Changes since V1:
>    - remove unnecessary first kernel version check as Kazu suggested
>    - amend the commit-msg as Alex suggested

Thanks for the v2.

Acked-by: Kazuhito Hagio <k-hagio-ab at nec.com>

Another way is that checking the vmcoreinfo entry before the kernel 
version, but either is fine to me.

Thanks,
Kazu

> ---
>   defs.h    |  4 ++--
>   riscv64.c | 23 +++++++++++++++++++++++
>   2 files changed, 25 insertions(+), 2 deletions(-)
> 
> diff --git a/defs.h b/defs.h
> index 5ee60f1..c07e6d7 100644
> --- a/defs.h
> +++ b/defs.h
> @@ -3662,8 +3662,7 @@ typedef signed int s32;
>   	ulong _X = X;									\
>   	(THIS_KERNEL_VERSION >= LINUX(5,13,0) &&					\
>   		(_X) >= machdep->machspec->kernel_link_addr) ?				\
> -		(((unsigned long)(_X)-(machdep->machspec->kernel_link_addr)) +		\
> -		 machdep->machspec->phys_base):						\
> +		((unsigned long)(_X)-(machdep->machspec->va_kernel_pa_offset)): 	\
>   		(((unsigned long)(_X)-(machdep->kvbase)) +				\
>   		 machdep->machspec->phys_base);						\
>   	})
> @@ -7021,6 +7020,7 @@ struct machine_specific {
>   	ulong modules_vaddr;
>   	ulong modules_end;
>   	ulong kernel_link_addr;
> +	ulong va_kernel_pa_offset;
>   
>   	ulong _page_present;
>   	ulong _page_read;
> diff --git a/riscv64.c b/riscv64.c
> index 6b9a688..7b5dd3d 100644
> --- a/riscv64.c
> +++ b/riscv64.c
> @@ -418,6 +418,28 @@ error:
>   	error(FATAL, "cannot get vm layout\n");
>   }
>   
> +static void
> +riscv64_get_va_kernel_pa_offset(struct machine_specific *ms)
> +{
> +	unsigned long kernel_version = riscv64_get_kernel_version();
> +
> +	/*
> +	 * Since Linux v6.4 phys_base is not the physical start of the kernel,
> +	 * trying to use "va_kernel_pa_offset" to determine the offset between
> +	 * kernel virtual and physical addresses.
> +	 */
> +	if (kernel_version >= LINUX(6,4,0)) {
> +		char *string;
> +		if ((string = pc->read_vmcoreinfo("NUMBER(va_kernel_pa_offset)"))) {
> +			ms->va_kernel_pa_offset = htol(string, QUIET, NULL);
> +			free(string);
> +		} else
> +			error(FATAL, "cannot read va_kernel_pa_offset\n");
> +	}
> +	else
> +		ms->va_kernel_pa_offset = ms->kernel_link_addr - ms->phys_base;
> +}
> +
>   static int
>   riscv64_is_kvaddr(ulong vaddr)
>   {
> @@ -1352,6 +1374,7 @@ riscv64_init(int when)
>   		riscv64_get_struct_page_size(machdep->machspec);
>   		riscv64_get_va_bits(machdep->machspec);
>   		riscv64_get_va_range(machdep->machspec);
> +		riscv64_get_va_kernel_pa_offset(machdep->machspec);
>   
>   		pt_level_alloc(&machdep->pgd, "cannot malloc pgd space.");
>   		pt_level_alloc(&machdep->machspec->p4d, "cannot malloc p4d space.");

From conor at kernel.org  Tue Aug 15 00:07:41 2023
From: conor at kernel.org (Conor Dooley)
Date: Tue, 15 Aug 2023 08:07:41 +0100
Subject: [PATCH 1/2] RISC-V: Use linux,usable-memory-range for crash
 kernel
In-Reply-To: <1815D8529965B5FE+f6e31844-3c4e-6290-cf28-6822fb7d34ea@tinylab.org>
References: <20230811104117.91406-1-songshuaishuai@tinylab.org>
 <20230814-hefty-reformist-ac79d3c6b84b@wendy>
 <1815D8529965B5FE+f6e31844-3c4e-6290-cf28-6822fb7d34ea@tinylab.org>
Message-ID: <20230815-irritably-resident-7b717f79a98e@spud>

On Tue, Aug 15, 2023 at 10:49:12AM +0800, Song Shuai wrote:
> 
> 
> ? 2023/8/14 19:10, Conor Dooley ??:
> > On Fri, Aug 11, 2023 at 06:41:16PM +0800, Song Shuai wrote:
> > > Now we use "memeory::linux,usable-memory" to indicate the available
> > > memory for the crash kernel.
> > > 
> > > While booting with UEFI, the crash kernel would use efi.memmap to
> > > re-populate memblock and then first kernel's memory would be corrputed.
> > > Consequently, the /proc/vmcore file failed to create in my local test.
> > > 
> > > And according to "chosen" dtschema [1], the available memory for the
> > > crash kernel should be held via "chosen::linux,usable-memory-range"
> > > property which will re-cap memblock even after UEFI's re-population.
> > > 
> > > [1]:
> > > https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/chosen.yaml
> > 
> > nit: Please use Link: tags for links rather than free-form text.
> > 
> > This doesn't apply to fixes, for-next or master. What is your base?
> > 
> Actually, I sent these patches to kexec-tools/build-test-riscv-v2,
> perhaps I shouldn't cc the linux-riscv mailing list.

Oh right, sorry about that!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kexec/attachments/20230815/f11af40a/attachment.sig>

From horms at kernel.org  Tue Aug 15 03:39:56 2023
From: horms at kernel.org (Simon Horman)
Date: Tue, 15 Aug 2023 12:39:56 +0200
Subject: [ANNOUNCE] kexec-tools v2.0.27 preparation
In-Reply-To: <ymdn754ihzovgpgoikjnpwueqx6zzmgcempvlr6bsn5rx3mwrw@62dmgvygv4eo>
References: <ZNXjs19UOvGEl/5K@vergenet.net>
 <d92decbb-768c-27e7-8c0b-eea662a86670@linux.ibm.com>
 <ymdn754ihzovgpgoikjnpwueqx6zzmgcempvlr6bsn5rx3mwrw@62dmgvygv4eo>
Message-ID: <ZNtV/P4fPCrMevQZ@vergenet.net>

On Mon, Aug 14, 2023 at 11:19:07AM +0800, Coiby Xu wrote:
> Hi Sourabh,
> 
> On Mon, Aug 14, 2023 at 08:17:34AM +0530, Sourabh Jain wrote:
> > Hello Simon,
> > 
> > Can we include below fix?
> > https://lists.fedoraproject.org/archives/list/kexec at lists.fedoraproject.org/thread/O3W4SLVVFT6J25U5YSXNZ3URZ4QY6SRU/
> 
> I'll review your patch! Note the patch is against Fedora's kexec-tools
> so it doesn't apply to this upstream kexec-tools. The relationship
> between Fedora's kexec-tools and this upstream kexec-tools can be
> described as follows,
> 1. Fedora's kexec-tools packages the kexec binary which is provides by upstream
> kexec-tools; 2. The kexec binary is used for loading the kdump kernel and
> initrd but
> it's not responsible for building the kdump initrd. Fedora's kexec-tools
> is used to build the kdump initrd instead.
> 
> Currrently it's WIP to split current Fedora kexec-tools into
> kexec-tools, kdump-utils, makedumpfile and eppic. Hopefully after it's
> done, you and other uses will no longer be confused by the relationship
> between Fedora's kexec-tools and this upstream kexec-tools.

Thanks for the clarification, I'm assuming no action is needed on my part
at this time. Let me know if I am mistaken.


From songshuaishuai at tinylab.org  Tue Aug 15 03:48:00 2023
From: songshuaishuai at tinylab.org (Song Shuai)
Date: Tue, 15 Aug 2023 18:48:00 +0800
Subject: [Crash-utility PATCH] RISCV64: Add KASLR support
Message-ID: <20230815104800.705753-1-songshuaishuai@tinylab.org>

From: Song Shuai <suagrfillet at gmail.com>

This patch adds KASLR support for Crash to analyze KASLR-ed vmcore
since RISC-V Linux is already sufficiently prepared for KASLR [1].

With this patch, even if the Crash '--kaslr' option is not set or Linux
CONFIG_RANDOMIZE_BASE is not configured, the 'derive_kaslr_offset()'
function will always work?to calculate 'kt->relocate' which serves to
update the kernel virtual address.

[1]: https://lore.kernel.org/linux-riscv/20230722123850.634544-1-alexghiti at rivosinc.com/

Signed-off-by: Song Shuai <suagrfillet at gmail.com>
---
 main.c    |  2 +-
 riscv64.c | 11 +++++++++++
 symbols.c |  4 ++--
 3 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/main.c b/main.c
index b278c22..0c6e595 100644
--- a/main.c
+++ b/main.c
@@ -228,7 +228,7 @@ main(int argc, char **argv)
 			} else if (STREQ(long_options[option_index].name, "kaslr")) {
 				if (!machine_type("X86_64") &&
 				    !machine_type("ARM64") && !machine_type("X86") &&
-				    !machine_type("S390X"))
+				    !machine_type("S390X") && !machine_type("RISCV64"))
 					error(INFO, "--kaslr not valid "
 						"with this machine type.\n");
 				else if (STREQ(optarg, "auto"))
diff --git a/riscv64.c b/riscv64.c
index a02f75a..288c7ae 100644
--- a/riscv64.c
+++ b/riscv64.c
@@ -378,6 +378,9 @@ static void riscv64_get_va_range(struct machine_specific *ms)
 	} else
 		goto error;
 
+	if ((kt->flags2 & KASLR) && (kt->flags & RELOC_SET))
+		ms->kernel_link_addr += (kt->relocate * -1);
+
 	/*
 	 * From Linux 5.13, the kernel mapping is moved to the last 2GB
 	 * of the address space, modules use the 2GB memory range right
@@ -1360,6 +1363,14 @@ riscv64_init(int when)
 
 		machdep->verify_paddr = generic_verify_paddr;
 		machdep->ptrs_per_pgd = PTRS_PER_PGD;
+
+		/*
+		 * Even if CONFIG_RANDOMIZE_BASE is not configured,
+		 * derive_kaslr_offset() should work and set
+		 * kt->relocate to 0
+		 */
+		if (!kt->relocate && !(kt->flags2 & (RELOC_AUTO|KASLR)))
+			kt->flags2 |= (RELOC_AUTO|KASLR);
 		break;
 
 	case PRE_GDB:
diff --git a/symbols.c b/symbols.c
index 876be7a..8e8b4c3 100644
--- a/symbols.c
+++ b/symbols.c
@@ -629,7 +629,7 @@ kaslr_init(void)
 	char *string;
 
 	if ((!machine_type("X86_64") && !machine_type("ARM64") && !machine_type("X86") &&
-	    !machine_type("S390X")) || (kt->flags & RELOC_SET))
+	    !machine_type("S390X") && !machine_type("RISCV64")) || (kt->flags & RELOC_SET))
 		return;
 
 	if (!kt->vmcoreinfo._stext_SYMBOL &&
@@ -795,7 +795,7 @@ store_symbols(bfd *abfd, int dynamic, void *minisyms, long symcount,
 		} else if (!(kt->flags & RELOC_SET))
 			kt->flags |= RELOC_FORCE;
 	} else if (machine_type("X86_64") || machine_type("ARM64") ||
-		   machine_type("S390X")) {
+		   machine_type("S390X") || machine_type("RISCV64")) {
 		if ((kt->flags2 & RELOC_AUTO) && !(kt->flags & RELOC_SET))
 			derive_kaslr_offset(abfd, dynamic, from,
 				fromend, size, store);
-- 
2.20.1



From eric.devolder at oracle.com  Tue Aug 15 07:40:32 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Tue, 15 Aug 2023 09:40:32 -0500
Subject: [PATCH v27 2/8] crash: add generic infrastructure for crash
 hotplug support
In-Reply-To: <188cfb39-7d10-967b-3150-22dac6e95baa@linux.ibm.com>
References: <20230811170642.6696-1-eric.devolder@oracle.com>
 <20230811170642.6696-3-eric.devolder@oracle.com>
 <188cfb39-7d10-967b-3150-22dac6e95baa@linux.ibm.com>
Message-ID: <36c4a86d-7b91-24c7-bdad-0f3ee1c2b7e7@oracle.com>



On 8/12/23 05:47, Sourabh Jain wrote:
> Hello Eric,
> 
> On 11/08/23 22:36, Eric DeVolder wrote:
>> To support crash hotplug, a mechanism is needed to update the crash
>> elfcorehdr upon CPU or memory changes (eg. hot un/plug or off/
>> onlining). The crash elfcorehdr describes the CPUs and memory to
>> be written into the vmcore.
>>
>> To track CPU changes, callbacks are registered with the cpuhp
>> mechanism via cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN). The
>> crash hotplug elfcorehdr update has no explicit ordering requirement
>> (relative to other cpuhp states), so meets the criteria for
>> utilizing CPUHP_BP_PREPARE_DYN. CPUHP_BP_PREPARE_DYN is a dynamic
>> state and avoids the need to introduce a new state for crash
>> hotplug. Also, CPUHP_BP_PREPARE_DYN is the last state in the PREPARE
>> group, just prior to the STARTING group, which is very close to the
>> CPU starting up in a plug/online situation, or stopping in a unplug/
>> offline situation. This minimizes the window of time during an
>> actual plug/online or unplug/offline situation in which the
>> elfcorehdr would be inaccurate. Note that for a CPU being unplugged
>> or offlined, the CPU will still be present in the list of CPUs
>> generated by crash_prepare_elf64_headers(). However, there is no
>> need to explicitly omit the CPU, see justification in
>> 'crash: change crash_prepare_elf64_headers() to for_each_possible_cpu()'.
>>
>> To track memory changes, a notifier is registered to capture the
>> memblock MEM_ONLINE and MEM_OFFLINE events via register_memory_notifier().
>>
>> The CPU callbacks and memory notifiers invoke crash_handle_hotplug_event()
>> which performs needed tasks and then dispatches the event to the
>> architecture specific arch_crash_handle_hotplug_event() to update the
>> elfcorehdr with the current state of CPUs and memory. During the
>> process, the kexec_lock is held.
>>
>> Signed-off-by: Eric DeVolder <eric.devolder at oracle.com>
>> Reviewed-by: Sourabh Jain <sourabhjain at linux.ibm.com>
>> Acked-by: Hari Bathini <hbathini at linux.ibm.com>
>> Acked-by: Baoquan He <bhe at redhat.com>
>> ---
>> ? include/linux/crash_core.h |?? 9 +++
>> ? include/linux/kexec.h????? |? 11 +++
>> ? kernel/Kconfig.kexec?????? |? 31 ++++++++
>> ? kernel/crash_core.c??????? | 142 +++++++++++++++++++++++++++++++++++++
>> ? kernel/kexec_core.c??????? |?? 6 ++
>> ? 5 files changed, 199 insertions(+)
>>
>> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
>> index de62a722431e..e14345cc7a22 100644
>> --- a/include/linux/crash_core.h
>> +++ b/include/linux/crash_core.h
>> @@ -84,4 +84,13 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
>> ? int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
>> ????????? unsigned long long *crash_size, unsigned long long *crash_base);
>> +#define KEXEC_CRASH_HP_NONE??????????? 0
>> +#define KEXEC_CRASH_HP_ADD_CPU??????????? 1
>> +#define KEXEC_CRASH_HP_REMOVE_CPU??????? 2
>> +#define KEXEC_CRASH_HP_ADD_MEMORY??????? 3
>> +#define KEXEC_CRASH_HP_REMOVE_MEMORY??????? 4
>> +#define KEXEC_CRASH_HP_INVALID_CPU??????? -1U
>> +
>> +struct kimage;
>> +
>> ? #endif /* LINUX_CRASH_CORE_H */
>> diff --git a/include/linux/kexec.h b/include/linux/kexec.h
>> index 811a90e09698..b9903dd48e24 100644
>> --- a/include/linux/kexec.h
>> +++ b/include/linux/kexec.h
>> @@ -33,6 +33,7 @@ extern note_buf_t __percpu *crash_notes;
>> ? #include <linux/compat.h>
>> ? #include <linux/ioport.h>
>> ? #include <linux/module.h>
>> +#include <linux/highmem.h>
>> ? #include <asm/kexec.h>
>> ? /* Verify architecture specific macros are defined */
>> @@ -360,6 +361,12 @@ struct kimage {
>> ????? struct purgatory_info purgatory_info;
>> ? #endif
>> +#ifdef CONFIG_CRASH_HOTPLUG
>> +??? int hp_action;
>> +??? int elfcorehdr_index;
>> +??? bool elfcorehdr_updated;
>> +#endif
>> +
>> ? #ifdef CONFIG_IMA_KEXEC
>> ????? /* Virtual address of IMA measurement buffer for kexec syscall */
>> ????? void *ima_buffer;
>> @@ -490,6 +497,10 @@ static inline int arch_kexec_post_alloc_pages(void *vaddr, unsigned int pages, g
>> ? static inline void arch_kexec_pre_free_pages(void *vaddr, unsigned int pages) { }
>> ? #endif
>> +#ifndef arch_crash_handle_hotplug_event
>> +static inline void arch_crash_handle_hotplug_event(struct kimage *image) { }
>> +#endif
>> +
> 
> Isn't the above function should be declare under CONFIG_CRASH_HOTPLUG?
> 
> Thanks,
> Sourabh
> 
There are no compiler warnings/errors, due to the nature of being declared static inline.
And most of the other functions defined in a similar way in this file are not guard banded
by CONFIG ifdefs. I'm inclined to leave it this way.
Thanks!
eric

>> ? #else /* !CONFIG_KEXEC_CORE */
>> ? struct pt_regs;
>> ? struct task_struct;
>> diff --git a/kernel/Kconfig.kexec b/kernel/Kconfig.kexec
>> index ff72e45cfaef..d0a9a5392035 100644
>> --- a/kernel/Kconfig.kexec
>> +++ b/kernel/Kconfig.kexec
>> @@ -113,4 +113,35 @@ config CRASH_DUMP
>> ??????? For s390, this option also enables zfcpdump.
>> ??????? See also <file:Documentation/s390/zfcpdump.rst>
>> +config CRASH_HOTPLUG
>> +??? bool "Update the crash elfcorehdr on system configuration changes"
>> +??? default y
>> +??? depends on CRASH_DUMP && (HOTPLUG_CPU || MEMORY_HOTPLUG)
>> +??? depends on ARCH_SUPPORTS_CRASH_HOTPLUG
>> +??? help
>> +????? Enable direct update to the crash elfcorehdr (which contains
>> +????? the list of CPUs and memory regions to be dumped upon a crash)
>> +????? in response to hot plug/unplug or online/offline of CPUs or
>> +????? memory. This is a much more advanced approach than userspace
>> +????? attempting that.
>> +
>> +????? If unsure, say Y.
>> +
>> +config CRASH_MAX_MEMORY_RANGES
>> +??? int "Specify the maximum number of memory regions for the elfcorehdr"
>> +??? default 8192
>> +??? depends on CRASH_HOTPLUG
>> +??? help
>> +????? For the kexec_file_load() syscall path, specify the maximum number of
>> +????? memory regions that the elfcorehdr buffer/segment can accommodate.
>> +????? These regions are obtained via walk_system_ram_res(); eg. the
>> +????? 'System RAM' entries in /proc/iomem.
>> +????? This value is combined with NR_CPUS_DEFAULT and multiplied by
>> +????? sizeof(Elf64_Phdr) to determine the final elfcorehdr memory buffer/
>> +????? segment size.
>> +????? The value 8192, for example, covers a (sparsely populated) 1TiB system
>> +????? consisting of 128MiB memblocks, while resulting in an elfcorehdr
>> +????? memory buffer/segment size under 1MiB. This represents a sane choice
>> +????? to accommodate both baremetal and virtual machine configurations.
>> +
>> ? endmenu
>> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>> index b7c30b748a16..53d211c690a1 100644
>> --- a/kernel/crash_core.c
>> +++ b/kernel/crash_core.c
>> @@ -11,6 +11,8 @@
>> ? #include <linux/vmalloc.h>
>> ? #include <linux/sizes.h>
>> ? #include <linux/kexec.h>
>> +#include <linux/memory.h>
>> +#include <linux/cpuhotplug.h>
>> ? #include <asm/page.h>
>> ? #include <asm/sections.h>
>> @@ -18,6 +20,7 @@
>> ? #include <crypto/sha1.h>
>> ? #include "kallsyms_internal.h"
>> +#include "kexec_internal.h"
>> ? /* vmcoreinfo stuff */
>> ? unsigned char *vmcoreinfo_data;
>> @@ -697,3 +700,142 @@ static int __init crash_save_vmcoreinfo_init(void)
>> ? }
>> ? subsys_initcall(crash_save_vmcoreinfo_init);
>> +
>> +#ifdef CONFIG_CRASH_HOTPLUG
>> +#undef pr_fmt
>> +#define pr_fmt(fmt) "crash hp: " fmt
>> +/*
>> + * To accurately reflect hot un/plug changes of cpu and memory resources
>> + * (including onling and offlining of those resources), the elfcorehdr
>> + * (which is passed to the crash kernel via the elfcorehdr= parameter)
>> + * must be updated with the new list of CPUs and memories.
>> + *
>> + * In order to make changes to elfcorehdr, two conditions are needed:
>> + * First, the segment containing the elfcorehdr must be large enough
>> + * to permit a growing number of resources; the elfcorehdr memory size
>> + * is based on NR_CPUS_DEFAULT and CRASH_MAX_MEMORY_RANGES.
>> + * Second, purgatory must explicitly exclude the elfcorehdr from the
>> + * list of segments it checks (since the elfcorehdr changes and thus
>> + * would require an update to purgatory itself to update the digest).
>> + */
>> +static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)
>> +{
>> +??? struct kimage *image;
>> +
>> +??? /* Obtain lock while changing crash information */
>> +??? if (!kexec_trylock()) {
>> +??????? pr_info("kexec_trylock() failed, elfcorehdr may be inaccurate\n");
>> +??????? return;
>> +??? }
>> +
>> +??? /* Check kdump is not loaded */
>> +??? if (!kexec_crash_image)
>> +??????? goto out;
>> +
>> +??? image = kexec_crash_image;
>> +
>> +??? if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
>> +??????? hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
>> +??????? pr_debug("hp_action %u, cpu %u\n", hp_action, cpu);
>> +??? else
>> +??????? pr_debug("hp_action %u\n", hp_action);
>> +
>> +??? /*
>> +???? * The elfcorehdr_index is set to -1 when the struct kimage
>> +???? * is allocated. Find the segment containing the elfcorehdr,
>> +???? * if not already found.
>> +???? */
>> +??? if (image->elfcorehdr_index < 0) {
>> +??????? unsigned long mem;
>> +??????? unsigned char *ptr;
>> +??????? unsigned int n;
>> +
>> +??????? for (n = 0; n < image->nr_segments; n++) {
>> +??????????? mem = image->segment[n].mem;
>> +??????????? ptr = kmap_local_page(pfn_to_page(mem >> PAGE_SHIFT));
>> +??????????? if (ptr) {
>> +??????????????? /* The segment containing elfcorehdr */
>> +??????????????? if (memcmp(ptr, ELFMAG, SELFMAG) == 0)
>> +??????????????????? image->elfcorehdr_index = (int)n;
>> +??????????????? kunmap_local(ptr);
>> +??????????? }
>> +??????? }
>> +??? }
>> +
>> +??? if (image->elfcorehdr_index < 0) {
>> +??????? pr_err("unable to locate elfcorehdr segment");
>> +??????? goto out;
>> +??? }
>> +
>> +??? /* Needed in order for the segments to be updated */
>> +??? arch_kexec_unprotect_crashkres();
>> +
>> +??? /* Differentiate between normal load and hotplug update */
>> +??? image->hp_action = hp_action;
>> +
>> +??? /* Now invoke arch-specific update handler */
>> +??? arch_crash_handle_hotplug_event(image);
>> +
>> +??? /* No longer handling a hotplug event */
>> +??? image->hp_action = KEXEC_CRASH_HP_NONE;
>> +??? image->elfcorehdr_updated = true;
>> +
>> +??? /* Change back to read-only */
>> +??? arch_kexec_protect_crashkres();
>> +
>> +??? /* Errors in the callback is not a reason to rollback state */
>> +out:
>> +??? /* Release lock now that update complete */
>> +??? kexec_unlock();
>> +}
>> +
>> +static int crash_memhp_notifier(struct notifier_block *nb, unsigned long val, void *v)
>> +{
>> +??? switch (val) {
>> +??? case MEM_ONLINE:
>> +??????? crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_MEMORY,
>> +??????????? KEXEC_CRASH_HP_INVALID_CPU);
>> +??????? break;
>> +
>> +??? case MEM_OFFLINE:
>> +??????? crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_MEMORY,
>> +??????????? KEXEC_CRASH_HP_INVALID_CPU);
>> +??????? break;
>> +??? }
>> +??? return NOTIFY_OK;
>> +}
>> +
>> +static struct notifier_block crash_memhp_nb = {
>> +??? .notifier_call = crash_memhp_notifier,
>> +??? .priority = 0
>> +};
>> +
>> +static int crash_cpuhp_online(unsigned int cpu)
>> +{
>> +??? crash_handle_hotplug_event(KEXEC_CRASH_HP_ADD_CPU, cpu);
>> +??? return 0;
>> +}
>> +
>> +static int crash_cpuhp_offline(unsigned int cpu)
>> +{
>> +??? crash_handle_hotplug_event(KEXEC_CRASH_HP_REMOVE_CPU, cpu);
>> +??? return 0;
>> +}
>> +
>> +static int __init crash_hotplug_init(void)
>> +{
>> +??? int result = 0;
>> +
>> +??? if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
>> +??????? register_memory_notifier(&crash_memhp_nb);
>> +
>> +??? if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
>> +??????? result = cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN,
>> +??????????? "crash/cpuhp", crash_cpuhp_online, crash_cpuhp_offline);
>> +??? }
>> +
>> +??? return result;
>> +}
>> +
>> +subsys_initcall(crash_hotplug_init);
>> +#endif
>> diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
>> index e2f2574d8b74..5d323255862a 100644
>> --- a/kernel/kexec_core.c
>> +++ b/kernel/kexec_core.c
>> @@ -277,6 +277,12 @@ struct kimage *do_kimage_alloc_init(void)
>> ????? /* Initialize the list of unusable pages */
>> ????? INIT_LIST_HEAD(&image->unusable_pages);
>> +#ifdef CONFIG_CRASH_HOTPLUG
>> +??? image->hp_action = KEXEC_CRASH_HP_NONE;
>> +??? image->elfcorehdr_index = -1;
>> +??? image->elfcorehdr_updated = false;
>> +#endif
>> +
>> ????? return image;
>> ? }
> 


From urszula.wasiak at melynllyn.pl  Thu Aug 17 00:30:17 2023
From: urszula.wasiak at melynllyn.pl (Urszula Wasiak)
Date: Thu, 17 Aug 2023 07:30:17 GMT
Subject: Faktoring
Message-ID: <20230817064500-0.1.cs.1isq2.0.8c7kh82y4c@melynllyn.pl>

Dzie? dobry,

rozwa?ali Pa?stwo wyb?r finansowania, kt?re spe?ni potrzeby firmy, zapewniaj?c natychmiastowy dost?p do got?wki, bez zb?dnych przestoj?w? 

Przygotowali?my rozwi?zania faktoringowe dopasowane do Pa?stwa bran?y i wielko?ci firmy, dzi?ki kt?rym, nie musz? Pa?stwo martwi? si? o niewyp?acalno?? kontrahent?w, poniewa? transakcje s? zabezpieczone i posiadaj? gwarancj? sp?aty. 

Chc? Pa?stwo przeanalizowa? dost?pne opcje?


Pozdrawiam
Urszula Wasiak


From guoren at kernel.org  Thu Aug 17 17:45:44 2023
From: guoren at kernel.org (Guo Ren)
Date: Fri, 18 Aug 2023 08:45:44 +0800
Subject: [Crash-utility PATCH] RISCV64: Add KASLR support
In-Reply-To: <20230815104800.705753-1-songshuaishuai@tinylab.org>
References: <20230815104800.705753-1-songshuaishuai@tinylab.org>
Message-ID: <CAJF2gTTJh7gQXacVOyx2h7pU8Tqq6tOWvuWaFS09PrX_1+CK8g@mail.gmail.com>

On Tue, Aug 15, 2023 at 6:50?PM Song Shuai <songshuaishuai at tinylab.org> wrote:
>
> From: Song Shuai <suagrfillet at gmail.com>
>
> This patch adds KASLR support for Crash to analyze KASLR-ed vmcore
> since RISC-V Linux is already sufficiently prepared for KASLR [1].
>
> With this patch, even if the Crash '--kaslr' option is not set or Linux
> CONFIG_RANDOMIZE_BASE is not configured, the 'derive_kaslr_offset()'
> function will always work to calculate 'kt->relocate' which serves to
> update the kernel virtual address.
>
> [1]: https://lore.kernel.org/linux-riscv/20230722123850.634544-1-alexghiti at rivosinc.com/
>
> Signed-off-by: Song Shuai <suagrfillet at gmail.com>
> ---
>  main.c    |  2 +-
>  riscv64.c | 11 +++++++++++
>  symbols.c |  4 ++--
>  3 files changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/main.c b/main.c
> index b278c22..0c6e595 100644
> --- a/main.c
> +++ b/main.c
> @@ -228,7 +228,7 @@ main(int argc, char **argv)
>                         } else if (STREQ(long_options[option_index].name, "kaslr")) {
>                                 if (!machine_type("X86_64") &&
>                                     !machine_type("ARM64") && !machine_type("X86") &&
> -                                   !machine_type("S390X"))
> +                                   !machine_type("S390X") && !machine_type("RISCV64"))
>                                         error(INFO, "--kaslr not valid "
>                                                 "with this machine type.\n");
>                                 else if (STREQ(optarg, "auto"))
> diff --git a/riscv64.c b/riscv64.c
> index a02f75a..288c7ae 100644
> --- a/riscv64.c
> +++ b/riscv64.c
> @@ -378,6 +378,9 @@ static void riscv64_get_va_range(struct machine_specific *ms)
>         } else
>                 goto error;
>
> +       if ((kt->flags2 & KASLR) && (kt->flags & RELOC_SET))
> +               ms->kernel_link_addr += (kt->relocate * -1);
> +
>         /*
>          * From Linux 5.13, the kernel mapping is moved to the last 2GB
>          * of the address space, modules use the 2GB memory range right
> @@ -1360,6 +1363,14 @@ riscv64_init(int when)
>
>                 machdep->verify_paddr = generic_verify_paddr;
>                 machdep->ptrs_per_pgd = PTRS_PER_PGD;
> +
> +               /*
> +                * Even if CONFIG_RANDOMIZE_BASE is not configured,
> +                * derive_kaslr_offset() should work and set
> +                * kt->relocate to 0
> +                */
> +               if (!kt->relocate && !(kt->flags2 & (RELOC_AUTO|KASLR)))
> +                       kt->flags2 |= (RELOC_AUTO|KASLR);
>                 break;
>
>         case PRE_GDB:
> diff --git a/symbols.c b/symbols.c
> index 876be7a..8e8b4c3 100644
> --- a/symbols.c
> +++ b/symbols.c
> @@ -629,7 +629,7 @@ kaslr_init(void)
>         char *string;
>
>         if ((!machine_type("X86_64") && !machine_type("ARM64") && !machine_type("X86") &&
> -           !machine_type("S390X")) || (kt->flags & RELOC_SET))
> +           !machine_type("S390X") && !machine_type("RISCV64")) || (kt->flags & RELOC_SET))
>                 return;
>
>         if (!kt->vmcoreinfo._stext_SYMBOL &&
> @@ -795,7 +795,7 @@ store_symbols(bfd *abfd, int dynamic, void *minisyms, long symcount,
>                 } else if (!(kt->flags & RELOC_SET))
>                         kt->flags |= RELOC_FORCE;
>         } else if (machine_type("X86_64") || machine_type("ARM64") ||
> -                  machine_type("S390X")) {
> +                  machine_type("S390X") || machine_type("RISCV64")) {
>                 if ((kt->flags2 & RELOC_AUTO) && !(kt->flags & RELOC_SET))
>                         derive_kaslr_offset(abfd, dynamic, from,
>                                 fromend, size, store);
> --
> 2.20.1
>
Reviewed-by: Guo Ren <guoren at kernel.org>

Could you put some output in the commit log, to let other turst this
patch could work.

-- 
Best Regards
 Guo Ren


From songshuaishuai at tinylab.org  Fri Aug 18 02:50:28 2023
From: songshuaishuai at tinylab.org (Song Shuai)
Date: Fri, 18 Aug 2023 17:50:28 +0800
Subject: [Crash-utility PATCH V2] RISCV64: Add KASLR support
Message-ID: <20230818095028.1012639-1-songshuaishuai@tinylab.org>

From: Song Shuai <suagrfillet at gmail.com>

This patch adds KASLR support for Crash to analyze KASLR-ed vmcore
since RISC-V Linux is already sufficiently prepared for KASLR [1].

With this patch, even if the Crash '--kaslr' option is not set or Linux
CONFIG_RANDOMIZE_BASE is not configured, the 'derive_kaslr_offset()'
function will always work?to calculate 'kt->relocate' which serves to
update the kernel virtual address.

Testing in Qemu rv64 virt, kernel log outputed the kernel offset:

[  121.214447] SMP: stopping secondary CPUs
[  121.215445] Kernel Offset: 0x37c00000 from 0xffffffff80000000
[  121.216312] Starting crashdump kernel...
[  121.216585] Will call new kernel at 94800000 from hart id 0
[  121.216834] FDT image at 9c7fd000
[  121.216982] Bye...

Running crash with `-d 1` option and without `--kaslr` option,
we get the right `kt->relocate` and kernel link addr:

$ ../crash/crash -d 1 vmlinux vmcore_kaslr_0815
...
KASLR:
  _stext from vmlinux: ffffffff80002000
  _stext from vmcoreinfo: ffffffffb7c02000
  relocate: 37c00000 (892MB)
vmemmap : 0xff1c000000000000 - 0xff20000000000000
vmalloc : 0xff20000000000000 - 0xff60000000000000
mudules : 0xffffffff3952f000 - 0xffffffffb7c00000
lowmem  : 0xff60000000000000 -
kernel link addr        : 0xffffffffb7c00000
...
      KERNEL: /home/song/9_linux/linux/00_rv_kaslr/vmlinux
    DUMPFILE: /tmp/hello/vmcore_kaslr_0815
        CPUS: 2
        DATE: Tue Aug 15 16:36:15 CST 2023
      UPTIME: 00:02:01
LOAD AVERAGE: 0.40, 0.23, 0.09
       TASKS: 63
    NODENAME: stage4.fedoraproject.org
     RELEASE: 6.5.0-rc3-00008-gad18dee423ac
     VERSION: #17 SMP Tue Aug 15 14:41:12 CST 2023
     MACHINE: riscv64  (unknown Mhz)
      MEMORY: 511.8 MB
       PANIC: "Kernel panic - not syncing: sysrq triggered crash"
         PID: 160
     COMMAND: "bash"
        TASK: ff6000000152bac0  [THREAD_INFO: ff6000000152bac0]
         CPU: 1
       STATE: TASK_RUNNING (PANIC)
crash>

[1]: https://lore.kernel.org/linux-riscv/20230722123850.634544-1-alexghiti at rivosinc.com/

Signed-off-by: Song Shuai <suagrfillet at gmail.com>
Reviewed-by: Guo Ren <guoren at kernel.org>

---
Changes since V1:
https://lore.kernel.org/linux-riscv/20230815104800.705753-1-songshuaishuai at tinylab.org/
  - supplement the output of my Crash test in the commit-msg 
  - add the Reviewed-by from Guo

---
 main.c    |  2 +-
 riscv64.c | 11 +++++++++++
 symbols.c |  4 ++--
 3 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/main.c b/main.c
index b278c22..0c6e595 100644
--- a/main.c
+++ b/main.c
@@ -228,7 +228,7 @@ main(int argc, char **argv)
 			} else if (STREQ(long_options[option_index].name, "kaslr")) {
 				if (!machine_type("X86_64") &&
 				    !machine_type("ARM64") && !machine_type("X86") &&
-				    !machine_type("S390X"))
+				    !machine_type("S390X") && !machine_type("RISCV64"))
 					error(INFO, "--kaslr not valid "
 						"with this machine type.\n");
 				else if (STREQ(optarg, "auto"))
diff --git a/riscv64.c b/riscv64.c
index a02f75a..288c7ae 100644
--- a/riscv64.c
+++ b/riscv64.c
@@ -378,6 +378,9 @@ static void riscv64_get_va_range(struct machine_specific *ms)
 	} else
 		goto error;
 
+	if ((kt->flags2 & KASLR) && (kt->flags & RELOC_SET))
+		ms->kernel_link_addr += (kt->relocate * -1);
+
 	/*
 	 * From Linux 5.13, the kernel mapping is moved to the last 2GB
 	 * of the address space, modules use the 2GB memory range right
@@ -1360,6 +1363,14 @@ riscv64_init(int when)
 
 		machdep->verify_paddr = generic_verify_paddr;
 		machdep->ptrs_per_pgd = PTRS_PER_PGD;
+
+		/*
+		 * Even if CONFIG_RANDOMIZE_BASE is not configured,
+		 * derive_kaslr_offset() should work and set
+		 * kt->relocate to 0
+		 */
+		if (!kt->relocate && !(kt->flags2 & (RELOC_AUTO|KASLR)))
+			kt->flags2 |= (RELOC_AUTO|KASLR);
 		break;
 
 	case PRE_GDB:
diff --git a/symbols.c b/symbols.c
index 876be7a..8e8b4c3 100644
--- a/symbols.c
+++ b/symbols.c
@@ -629,7 +629,7 @@ kaslr_init(void)
 	char *string;
 
 	if ((!machine_type("X86_64") && !machine_type("ARM64") && !machine_type("X86") &&
-	    !machine_type("S390X")) || (kt->flags & RELOC_SET))
+	    !machine_type("S390X") && !machine_type("RISCV64")) || (kt->flags & RELOC_SET))
 		return;
 
 	if (!kt->vmcoreinfo._stext_SYMBOL &&
@@ -795,7 +795,7 @@ store_symbols(bfd *abfd, int dynamic, void *minisyms, long symcount,
 		} else if (!(kt->flags & RELOC_SET))
 			kt->flags |= RELOC_FORCE;
 	} else if (machine_type("X86_64") || machine_type("ARM64") ||
-		   machine_type("S390X")) {
+		   machine_type("S390X") || machine_type("RISCV64")) {
 		if ((kt->flags2 & RELOC_AUTO) && !(kt->flags & RELOC_SET))
 			derive_kaslr_offset(abfd, dynamic, from,
 				fromend, size, store);
-- 
2.20.1



From horms at kernel.org  Fri Aug 18 03:49:02 2023
From: horms at kernel.org (Simon Horman)
Date: Fri, 18 Aug 2023 12:49:02 +0200
Subject: [ANNOUNCE] kexec-tools 2.0.27-rc1
Message-ID: <ZN9MnhFyaAbcVzo3@vergenet.net>

Hi all,

I am happy to announce the release of kexec-tools 2.0.27-rc1.

This is an incremental feature pre-release.

So long as no serious problems arise I intend to release kexec-tools 2.0.27
in a weeks time. Testing of 2.0.27-rc1 would be greatly appreciated.

I do not have any outstanding changes for 2.0.27 at this time.
And I would like to only accept bug fixes at this time and take
features patches once 2.0.27 has been released.

The pre-release can be downloaded from kernel.org:

	https://kernel.org/pub/linux/utils/kernel/kexec/kexec-tools-2.0.27-rc1.tar.xz
	https://kernel.org/pub/linux/utils/kernel/kexec/

I have also tagged it in git:

	https://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git

Thanks to everyone who has contributed to kexec.


For reference the changes since v2.0.26 are:

777ca453ca69 kexec-tools 2.0.27-rc1
f67c4146d7b5 arm64: Hook up the ZBOOT support as vmlinuz
fc7b83bdf734 arm64: Add ZBOOT PE containing compressed image support
f41c4182b0c4 kexec/zboot: Add arch independent zboot support
1572b91da7c4 kexec: Introduce a member kernel_fd in kexec_info
714fa11590fe kexec/arm64: Simplify the code for zImage
a8de94e5f033 LoongArch: kdump: Set up kernel image segment
4203eaccfa92 kexec: __NR_kexec_file_load is set to undefined on LoongArch
63e9a012112e ppc64: Add elf-ppc64 file types/options and an arch specific flag to man page
806711fca9e9 x86: add devicetree support
29fe5067ed07 kexec: make -a the default
e63fefd4fc35 ppc64: add --reuse-cmdline parameter support
8fc55927f700 kexec-tools 2.0.26.git


From alexander.kamensky42 at gmail.com  Sat Aug 19 12:11:19 2023
From: alexander.kamensky42 at gmail.com (Alexander Kamensky)
Date: Sat, 19 Aug 2023 12:11:19 -0700
Subject: [PATCH] aarch64: kern_paddr_start calculation makes more accurate legacy check
Message-ID: <20230819191119.975299-1-alexander.kamensky42@gmail.com>

In the latest openembedded with aarch64 image that uses 6.4.3 kernel on qemu I
tried to run makedumpfile in secondary kernel with /proc/vmcore. It failed as
follows:

> root at qemuarm64:~# makedumpfile -c -F /proc/vmcore > /dev/null
> read_from_vmcore: Can't seek the dump memory(/proc/vmcore). (offset: ffffffc0123fa000) Invalid argument
> readpage_elf: Can't read the dump memory(/proc/vmcore).
> <snip>

It turns out that not all parts for /proc/vmcore are readable:

> root at qemuarm64:~# cat /proc/vmcore > /dev/null
> [  136.394931] Internal error: synchronous external abort: 0000000096000010 [#1] PREEMPT SMP
> [  136.422434] Modules linked in:
> <snip>

Binary search of different kernel versions I found that the issue goes back to
5.11. It works fine with 5.10 version of the kernel.

After running secondary kernel under qemu gdb, I've found that the primary kernel
main memory segment in elfcorehdr has wrong address and size.

Looking at kexec output with debug enabled at the time when it loads secondary
crash kernel I saw the following:

> Kernel text Elf header: p_type = 1, p_offset = 0x4030200000 p_paddr = 0x4030200000 p_vaddr = 0xffffffffffffffff p_filesz = 0xffffffc011410000 p_memsz = 0xffffffc011410000

These values come from elf_info variable and are set in iomem_range_callback
function. The function gets the following input on my system:

root at qemuarm64:~# cat /proc/iomem | grep "Kernel code"
  40210000-40feffff : Kernel code
root at qemuarm64:~# cat /proc/kallsyms | grep -e ' _text$'
root at qemuarm64:~# cat /proc/kallsyms | grep -e ' _stext$'
ffffffc010010000 T _stext
root at qemuarm64:~# cat /proc/kallsyms | grep -e ' __init_begin$'
ffffffc010df0000 T __init_begin

the function calculates elf_info.kern_paddr_start address similar to
these manual steps:

base = 0x40210000
length = 0x40feffff - 0x40210000 + 0x1 = 0xde0000
kva_text_end - kva_stext = 0xffffffc010df0000 - 0xffffffc010010000 = 0xde0000

elf_info.kern_paddr_start = base - (kva_stext - kva_text) =
                      0x40210000 - 0xffffffc010010000 = 0x4030200000

since 'length' and 'kva_text_end - kva_stext' are equal the function calculate
elf_info.kern_paddr_start as in legacy case:

  if (kva_text_end - kva_stext == length)
     elf_info.kern_paddr_start = base - (kva_stext - kva_text); // <---------
  else
     elf_info.kern_paddr_start = base;

but my kernel is not legacy and kva_text is zero.

The fix is just to add a check for kva_text not being zero before following the
legacy case.

Signed-off-by: Alexander Kamensky <alexander.kamensky42 at gmail.com>
---
 kexec/arch/arm64/crashdump-arm64.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kexec/arch/arm64/crashdump-arm64.c b/kexec/arch/arm64/crashdump-arm64.c
index 3098315..639c82a 100644
--- a/kexec/arch/arm64/crashdump-arm64.c
+++ b/kexec/arch/arm64/crashdump-arm64.c
@@ -82,7 +82,7 @@ static int iomem_range_callback(void *UNUSED(data), int UNUSED(nr),
 		 * For compatibility, deduce by comparing the gap "__init_begin - _stext"
 		 * and the res size of "Kernel code" in /proc/iomem
 		 */
-		if (kva_text_end - kva_stext == length)
+		if ((kva_text_end - kva_stext == length) && (kva_text != 0))
 			elf_info.kern_paddr_start = base - (kva_stext - kva_text);
 		else
 			elf_info.kern_paddr_start = base;
-- 
2.41.0



From eric.devolder at oracle.com  Sun Aug 20 14:56:22 2023
From: eric.devolder at oracle.com (Eric DeVolder)
Date: Sun, 20 Aug 2023 16:56:22 -0500
Subject: [PATCH v28 0/8] crash: Kernel handling of CPU and memory hot
 un/plug
In-Reply-To: <20230814153338.79164562f1d2810a18f495dd@linux-foundation.org>
References: <20230814214446.6659-1-eric.devolder@oracle.com>
 <20230814153338.79164562f1d2810a18f495dd@linux-foundation.org>
Message-ID: <2dac3bf6-dc57-4e60-ba82-f3eab73e1d5c@oracle.com>


On 8/14/23 17:33, Andrew Morton wrote:
> On Mon, 14 Aug 2023 17:44:38 -0400 Eric DeVolder <eric.devolder at oracle.com> wrote:
> 
>> This series is dependent upon "refactor Kconfig to consolidate
>> KEXEC and CRASH options".
>>   https://lore.kernel.org/lkml/20230712161545.87870-1-eric.devolder at oracle.com/
>>
>> Once the kdump service is loaded, if changes to CPUs or memory occur,
>> either by hot un/plug or off/onlining, the crash elfcorehdr must also
>> be updated.
> 
> Thanks, I updated branch mm-nonmm-unstable to this version.

Andrew,
So far only one issue has popped up. I've posted the following patch to akpm to solve
that issue. Please apply this patch on-top/with this v28 series.

  [PATCH] x86/crash: correct unused function build error

The thread on this issue is here:
https://lore.kernel.org/lkml/08fc20ef-854d-404a-b2f2-75941eeeccf8 at paulmck-laptop/

If you'd rather I post a v29, I'll happily do so.
Thank you!
eric


From k-hagio-ab at nec.com  Sun Aug 20 17:43:47 2023
From: k-hagio-ab at nec.com (=?utf-8?B?SEFHSU8gS0FaVUhJVE8o6JCp5bC+44CA5LiA5LuBKQ==?=)
Date: Mon, 21 Aug 2023 00:43:47 +0000
Subject: [Crash-utility PATCH V2] RISCV64: Add KASLR support
In-Reply-To: <20230818095028.1012639-1-songshuaishuai@tinylab.org>
References: <20230818095028.1012639-1-songshuaishuai@tinylab.org>
Message-ID: <158d3047-179a-4b1c-03a6-b9edfdbb5c0b@nec.com>

On 2023/08/18 18:50, Song Shuai wrote:
> From: Song Shuai <suagrfillet at gmail.com>
> 
> This patch adds KASLR support for Crash to analyze KASLR-ed vmcore
> since RISC-V Linux is already sufficiently prepared for KASLR [1].
> 
> With this patch, even if the Crash '--kaslr' option is not set or Linux
> CONFIG_RANDOMIZE_BASE is not configured, the 'derive_kaslr_offset()'
> function will always work?to calculate 'kt->relocate' which serves to
> update the kernel virtual address.
> 
> Testing in Qemu rv64 virt, kernel log outputed the kernel offset:
> 
> [  121.214447] SMP: stopping secondary CPUs
> [  121.215445] Kernel Offset: 0x37c00000 from 0xffffffff80000000
> [  121.216312] Starting crashdump kernel...
> [  121.216585] Will call new kernel at 94800000 from hart id 0
> [  121.216834] FDT image at 9c7fd000
> [  121.216982] Bye...
> 
> Running crash with `-d 1` option and without `--kaslr` option,
> we get the right `kt->relocate` and kernel link addr:
> 
> $ ../crash/crash -d 1 vmlinux vmcore_kaslr_0815
> ...
> KASLR:
>    _stext from vmlinux: ffffffff80002000
>    _stext from vmcoreinfo: ffffffffb7c02000
>    relocate: 37c00000 (892MB)
> vmemmap : 0xff1c000000000000 - 0xff20000000000000
> vmalloc : 0xff20000000000000 - 0xff60000000000000
> mudules : 0xffffffff3952f000 - 0xffffffffb7c00000
> lowmem  : 0xff60000000000000 -
> kernel link addr        : 0xffffffffb7c00000
> ...
>        KERNEL: /home/song/9_linux/linux/00_rv_kaslr/vmlinux
>      DUMPFILE: /tmp/hello/vmcore_kaslr_0815
>          CPUS: 2
>          DATE: Tue Aug 15 16:36:15 CST 2023
>        UPTIME: 00:02:01
> LOAD AVERAGE: 0.40, 0.23, 0.09
>         TASKS: 63
>      NODENAME: stage4.fedoraproject.org
>       RELEASE: 6.5.0-rc3-00008-gad18dee423ac
>       VERSION: #17 SMP Tue Aug 15 14:41:12 CST 2023
>       MACHINE: riscv64  (unknown Mhz)
>        MEMORY: 511.8 MB
>         PANIC: "Kernel panic - not syncing: sysrq triggered crash"
>           PID: 160
>       COMMAND: "bash"
>          TASK: ff6000000152bac0  [THREAD_INFO: ff6000000152bac0]
>           CPU: 1
>         STATE: TASK_RUNNING (PANIC)
> crash>
> 
> [1]: https://lore.kernel.org/linux-riscv/20230722123850.634544-1-alexghiti at rivosinc.com/
> 
> Signed-off-by: Song Shuai <suagrfillet at gmail.com>
> Reviewed-by: Guo Ren <guoren at kernel.org>
> 
> ---
> Changes since V1:
> https://lore.kernel.org/linux-riscv/20230815104800.705753-1-songshuaishuai at tinylab.org/
>    - supplement the output of my Crash test in the commit-msg
>    - add the Reviewed-by from Guo
> 
> ---
>   main.c    |  2 +-
>   riscv64.c | 11 +++++++++++
>   symbols.c |  4 ++--
>   3 files changed, 14 insertions(+), 3 deletions(-)
> 
> diff --git a/main.c b/main.c
> index b278c22..0c6e595 100644
> --- a/main.c
> +++ b/main.c
> @@ -228,7 +228,7 @@ main(int argc, char **argv)
>   			} else if (STREQ(long_options[option_index].name, "kaslr")) {
>   				if (!machine_type("X86_64") &&
>   				    !machine_type("ARM64") && !machine_type("X86") &&
> -				    !machine_type("S390X"))
> +				    !machine_type("S390X") && !machine_type("RISCV64"))
>   					error(INFO, "--kaslr not valid "
>   						"with this machine type.\n");
>   				else if (STREQ(optarg, "auto"))
> diff --git a/riscv64.c b/riscv64.c
> index a02f75a..288c7ae 100644
> --- a/riscv64.c
> +++ b/riscv64.c
> @@ -378,6 +378,9 @@ static void riscv64_get_va_range(struct machine_specific *ms)
>   	} else
>   		goto error;
>   
> +	if ((kt->flags2 & KASLR) && (kt->flags & RELOC_SET))
> +		ms->kernel_link_addr += (kt->relocate * -1);
> +
>   	/*
>   	 * From Linux 5.13, the kernel mapping is moved to the last 2GB
>   	 * of the address space, modules use the 2GB memory range right
> @@ -1360,6 +1363,14 @@ riscv64_init(int when)
>   
>   		machdep->verify_paddr = generic_verify_paddr;
>   		machdep->ptrs_per_pgd = PTRS_PER_PGD;
> +
> +		/*
> +		 * Even if CONFIG_RANDOMIZE_BASE is not configured,
> +		 * derive_kaslr_offset() should work and set
> +		 * kt->relocate to 0
> +		 */
> +		if (!kt->relocate && !(kt->flags2 & (RELOC_AUTO|KASLR)))
> +			kt->flags2 |= (RELOC_AUTO|KASLR);
>   		break;
>   
>   	case PRE_GDB:
> diff --git a/symbols.c b/symbols.c
> index 876be7a..8e8b4c3 100644
> --- a/symbols.c
> +++ b/symbols.c
> @@ -629,7 +629,7 @@ kaslr_init(void)
>   	char *string;
>   
>   	if ((!machine_type("X86_64") && !machine_type("ARM64") && !machine_type("X86") &&
> -	    !machine_type("S390X")) || (kt->flags & RELOC_SET))
> +	    !machine_type("S390X") && !machine_type("RISCV64")) || (kt->flags & RELOC_SET))
>   		return;
>   
>   	if (!kt->vmcoreinfo._stext_SYMBOL &&
> @@ -795,7 +795,7 @@ store_symbols(bfd *abfd, int dynamic, void *minisyms, long symcount,
>   		} else if (!(kt->flags & RELOC_SET))
>   			kt->flags |= RELOC_FORCE;
>   	} else if (machine_type("X86_64") || machine_type("ARM64") ||
> -		   machine_type("S390X")) {
> +		   machine_type("S390X") || machine_type("RISCV64")) {
>   		if ((kt->flags2 & RELOC_AUTO) && !(kt->flags & RELOC_SET))
>   			derive_kaslr_offset(abfd, dynamic, from,
>   				fromend, size, store);

Thanks, applied.
https://github.com/crash-utility/crash/commit/1aa93cd33fa11f9d9bc9dc7e6a698d690fdd1bb3

Kazu

From radhey.shyam.pandey at amd.com  Mon Aug 21 05:22:13 2023
From: radhey.shyam.pandey at amd.com (Pandey, Radhey Shyam)
Date: Mon, 21 Aug 2023 12:22:13 +0000
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64
 platform
In-Reply-To: <ZNogKKZCp12Q92jh@MiWiFi-R3L-srv>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>
 <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNogKKZCp12Q92jh@MiWiFi-R3L-srv>
Message-ID: <MN0PR12MB5953FFB655A208BD4FA32EE4B71EA@MN0PR12MB5953.namprd12.prod.outlook.com>

> -----Original Message-----
> From: bhe at redhat.com <bhe at redhat.com>
> Sent: Monday, August 14, 2023 6:08 PM
> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
> Cc: piliu at redhat.com; kexec at lists.infradead.org; linux-
> kernel at vger.kernel.org; Sarangi, Anirudha <anirudha.sarangi at amd.com>
> Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> arm64 platform
> 
> On 08/11/23 at 01:27pm, Pandey, Radhey Shyam wrote:
> > > -----Original Message-----
> > > From: bhe at redhat.com <bhe at redhat.com>
> > > Sent: Wednesday, August 9, 2023 7:42 AM
> > > To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>;
> > > piliu at redhat.com
> > > Cc: kexec at lists.infradead.org; linux-kernel at vger.kernel.org
> > > Subject: Re: kexec reports "Cannot get kernel _text symbol address"
> > > on
> > > arm64 platform
> > >
> > > On 08/08/23 at 07:17pm, Pandey, Radhey Shyam wrote:
> > > > Hi,
> > > >
> > > > I am trying to bring up kdump on arm64 platform[1]. But I get
> > > > "Cannot get
> > > kernel _text symbol address".
> > > >
> > > > Is there some Dump-capture kernel config options that I am missing?
> > > >
> > > > FYI, copied below complete kexec debug log.
> > > >
> > > > [1]: https://www.xilinx.com/products/boards-and-kits/vck190.html
> > >
> > > Your description isn't clear. You saw the printing, then your kdump
> > > kernel loading succeeded or not?
> > >
> > > If no, have you tried applying Pingfan's patchset and still saw the issue?
> > >
> > > [PATCHv7 0/5] arm64: zboot support
> > > https://lore.kernel.org/all/20230803024152.11663-1-piliu at redhat.com/
> > > T/#u
> >
> > I was able to proceed further with loading with crash kernel on triggering
> system crash.
> > echo c > /proc/sysrq-trigger
> >
> > But when I copy /proc/vmcore it throws memory abort. Also I see size of
> /proc/vmcore really huge (18446603353488633856).
> > Any possible guess on what could be wrong?
> 
> I didn't reproduce this issue on a arm64 baremetal system with the latest
> kernel. From the log, It could be the iov_iter convertion patch which caused
> this. Can you revert below patch to see if it works?
> 
> 5d8de293c224 vmcore: convert copy_oldmem_page() to take an iov_iter

Revert of this commit resulted in lot of conflicts. So as a safer side I checkout
v5.18 kernel version before above commit. Still I see the same issue.

/ # ls -lrth /proc/vmcore 
-r--------    1 root     root       16.0E Aug 21 12:16 /proc/vmcore
/ # dmesg | grep -i 5.18
[    0.000000] Linux version 5.18.0-00001-g689fdf110e63-dirty (radheys at xhdradheys41) (aarch64-xilinx-linux-gcc.real (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39.0.20220819) #37 SMP Mon Aug 21 17:38:24 IST 2023
[    2.494578] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.18
[    2.514941] usb usb1: Manufacturer: Linux 5.18.0-00001-g689fdf110e63-dirty xhci-hcd
[    2.555265] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.18
[    2.575621] usb usb2: Manufacturer: Linux 5.18.0-00001-g689fdf110e63-dirty xhci-hcd
[    3.152182] usb 1-1: new high-speed USB device number 2 using xhci-hcd
/ # cp /proc/vmcore dump
[   86.204704] Unable to handle kernel level 3 address size fault at virtual address ffff800009b75000
[   86.213677] Mem abort info:
[   86.216464]   ESR = 0x96000003
[   86.219508]   EC = 0x25: DABT (current EL), IL = 32 bits
[   86.224812]   SET = 0, FnV = 0
[   86.227856]   EA = 0, S1PTW = 0
[   86.230989]   FSC = 0x03: level 3 address size fault
[   86.235945] Data abort info:
[   86.238819]   ISV = 0, ISS = 0x00000003
[   86.242646]   CM = 0, WnR = 0
[   86.245608] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000043049000
[   86.252304] [ffff800009b75000] pgd=1000000061ffe003, p4d=1000000061ffe003, pud=1000000061ffd003, pmd=1000000043c12003, pte=00687ffff8200703
[   86.264828] Internal error: Oops: 96000003 [#1] SMP
[   86.269696] Modules linked in:
[   86.272741] CPU: 1 PID: 298 Comm: cp Not tainted 5.18.0-00001-g689fdf110e63-dirty #37
[   86.280562] Hardware name: Xilinx Versal vck190 Eval board revA (DT)
[   86.286905] pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   86.293857] pc : __arch_copy_to_user+0x180/0x240
[   86.298475] lr : copy_oldmem_page+0xa8/0x110
[   86.302738] sp : ffff80000af6bc50
[   86.306041] x29: ffff80000af6bc50 x28: ffff8000097de3b0 x27: ffff8000097de228
[   86.313170] x26: 0000000000000000 x25: ffff80000af6bd60 x24: 0000000000000000
[   86.320299] x23: ffff800009b75000 x22: 0000000000000001 x21: 0000ffffffa1e5a8
[   86.327427] x20: 0000000000000000 x19: 0000000000001000 x18: 0000000000000000
[   86.334555] x17: 0000000000000000 x16: 0000000000000000 x15: ffff800009b75000
[   86.341682] x14: ffff800009863568 x13: 0000000000000000 x12: ffff800008000000
[   86.348810] x11: 00007ffff8201000 x10: ffff800009b75fff x9 : 0000000000000000
[   86.355937] x8 : ffff800009b76000 x7 : 0400000000000001 x6 : 0000ffffffa1e5a8
[   86.363065] x5 : 0000ffffffa1f5a8 x4 : 0000000000000000 x3 : 0000ffffffffffff
[   86.370192] x2 : 0000000000000f80 x1 : ffff800009b75000 x0 : 0000ffffffa1e5a8
[   86.377320] Call trace:
[   86.379755]  __arch_copy_to_user+0x180/0x240
[   86.384018]  read_from_oldmem.part.0+0x160/0x1f4
[   86.388629]  read_vmcore+0xe4/0x214
[   86.392109]  proc_reg_read+0xb0/0x100
[   86.395763]  vfs_read+0x90/0x1dc
[   86.398981]  ksys_read+0x70/0x10c
[   86.402286]  __arm64_sys_read+0x20/0x30
[   86.406111]  invoke_syscall+0x54/0x124
[   86.409852]  el0_svc_common.constprop.0+0x44/0xec
[   86.414547]  do_el0_svc+0x70/0x90
[   86.417853]  el0_svc+0x50/0xa4
[   86.420899]  el0t_64_sync_handler+0x10c/0x140
[   86.425247]  el0t_64_sync+0x18c/0x190
[   86.428902] Code: d503201f d503201f d503201f d503201f (a8c12027) 
[   86.434984] ---[ end trace 0000000000000000 ]---
Segmentation fault


> 
> >
> >
> > [   80.733523] Starting crashdump kernel...
> > [   80.737435] Bye!
> > [    0.000000] Booting Linux on physical CPU 0x0000000001 [0x410fd083]
> > [    0.000000] Linux version 6.5.0-rc4-ge28001fb4e07
> (radheys at xhdradheys41) (aarch64-xilinx-linux-gcc.real (GCC) 12.2.0, GNU ld
> (GNU Binutils) 2.39.0.20220819) #23 SMP Fri Aug 11 16:25:34 IST 2023
> > <snip>
> >
> >
> >
> > xilinx-vck190-20232:/run/media/mmcblk0p1# cat /proc/meminfo | head
> > MemTotal:        2092876 kB
> > MemFree:         1219928 kB
> > MemAvailable:    1166004 kB
> > Buffers:              32 kB
> > Cached:           756952 kB
> > SwapCached:            0 kB
> > Active:             1480 kB
> > Inactive:          24164 kB
> > Active(anon):       1452 kB
> > Inactive(anon):    24160 kB
> > xilinx-vck190-20232:/run/media/mmcblk0p1# cp /proc/vmcore dump
> > [  975.284865] Unable to handle kernel level 3 address size fault at
> > virtual address ffff80008d7cf000 [  975.293871] Mem abort info:
> > [  975.296669]   ESR = 0x0000000096000003
> > [  975.300425]   EC = 0x25: DABT (current EL), IL = 32 bits
> > [  975.305738]   SET = 0, FnV = 0
> > [  975.308788]   EA = 0, S1PTW = 0
> > [  975.311925]   FSC = 0x03: level 3 address size fault
> > [  975.316888] Data abort info:
> > [  975.319763]   ISV = 0, ISS = 0x00000003, ISS2 = 0x00000000
> > [  975.325245]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > [  975.330292]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> > [  975.335599] swapper pgtable: 4k pages, 48-bit VAs,
> > pgdp=000005016ef6b000 [  975.342297] [ffff80008d7cf000]
> > pgd=10000501eddfe003, p4d=10000501eddfe003, pud=10000501eddfd003,
> > pmd=100005017b695003, pte=00687fff84000703 [  975.354827] Internal
> error: Oops: 0000000096000003 [#4] SMP [  975.360392] Modules linked in:
> > 3  975.
> > 63440] CBPrUo:a d0c aPID: 664 Comm: cp Tainted: G      D            6.5.0-rc4-
> ge28001fb4e07 #23
> > [  975.372822] Hardware name: Xilinx Versal vck190 Eval board revA
> > (DT) [  975.379165] pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT
> > -SSBS BTYPE=--) [  975.386119] pc : __memcpy+0x110/0x230 [
> > 975.389783] lr : _copy_to_iter+0x3d8/0x4d0 [  975.393874] sp :
> > ffff80008dc939a0 [  975.397178] x29: ffff80008dc939a0 x28:
> > ffff05013c1bea30 x27: 0000000000001000 [  975.404309] x26:
> > 0000000000001000 x25: 0000000000001000 x24: ffff80008d7cf000 [
> > 975.411440] x23: 0000040000000000 x22: ffff80008dc93ba0 x21:
> > 0000000000001000 [  975.418570] x20: ffff000000000000 x19:
> > 0000000000001000 x18: 0000000000000000 [  975.425699] x17:
> > 0000000000000000 x16: 0000000000000000 x15: 0140000000000000 [
> > 975.432829] x14: ffff8500a9919000 x13: 0040000000000001 x12:
> > 0000fffef6831000 [  975.439958] x11: ffff80008d9cf000 x10:
> > 0000000000000000 x9 : 0000000000000000 [  975.447088] x8 :
> > ffff80008d7d0000 x7 : ffff0501addfd358 x6 : 0400000000000001 [
> > 975.454217] x5 : ffff0501370e9000 x4 : ffff80008d7d0000 x3 :
> 0000000000000000 [  975.461346] x2 : 0000000000001000 x1 :
> ffff80008d7cf000 x0 : ffff0501370e8000 [  975.468476] Call trace:
> > [  975.470912]  __memcpy+0x110/0x230
> > [  975.474221]  copy_oldmem_page+0x70/0xac [  975.478050]
> > read_from_oldmem.part.0+0x120/0x188
> > [  975.482663]  read_vmcore+0x14c/0x238 [  975.486231]
> > proc_reg_read_iter+0x84/0xd8 [  975.490233]
> > copy_splice_read+0x160/0x288 [  975.494236]
> > vfs_splice_read+0xac/0x10c [  975.498063]
> > splice_direct_to_actor+0xa4/0x26c [  975.502498]
> > do_splice_direct+0x90/0xdc [  975.506325]  do_sendfile+0x344/0x454 [
> > 975.509892]  __arm64_sys_sendfile64+0x134/0x140
> > [  975.514415]  invoke_syscall+0x54/0x124 [  975.518157]
> > el0_svc_common.constprop.0+0xc4/0xe4
> > [  975.522854]  do_el0_svc+0x38/0x98
> > [  975.526162]  el0_svc+0x2c/0x84
> > [  975.529211]  el0t_64_sync_handler+0x100/0x12c [  975.533562]
> > el0t_64_sync+0x190/0x194 [  975.537218] Code: cb01000e b4fffc2e
> > eb0201df 540004a3 (a940342c) [  975.543302] ---[ end trace
> > 0000000000000000 ]--- t message from
> > systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):
> >
> > kernel[539]: [  975.354827] Internal error: Oops: 0000000096000003
> > [#4] SMP
> >
> >
> > Broadcast message from systemd-journald at xilinx-vck190-20232 (Tue
> 2022-11-08 14:16:20 UTC):
> >
> > kernel[539]: [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3
> > (a940342c)
> >
> > Segmentation fault
> > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lrth /proc/vmcore
> > -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lh /proc/vmcore
> > -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -l /proc/vmcore
> > -r--------    1 root     root     18446603353488633856 Nov  8 14:05
> /proc/vmcore
> >
> > Thanks,
> > Radhey
> >



From sushring at linux.microsoft.com  Mon Aug 21 15:05:17 2023
From: sushring at linux.microsoft.com (Sush Shringarputale)
Date: Mon, 21 Aug 2023 15:05:17 -0700
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
Message-ID: <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>



On 8/14/2023 3:02 PM, Mimi Zohar wrote:
> On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
>>> This design seems overly complex and requires synchronization between
>>> the "snapshot" record and exporting the records from the measurement
>>> list.  None of this would be necessary if the measurements were copied
>>> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
Even if the Kernel maintains the link between a tmpfs exported and an
in-memory IMA log - it still has to copy the tmpfs portion to the
Kernel memory during kexec soft boot.? tmpfs is cleared during kexec,
so this copying of tmpfs back to kernel memory is necessary to preserve
the integrity of the log during kexec.? But the copying would add back
the memory pressure on the node during kexec (which may result in
out-of-memory), defeating the purpose of the overall effort/feature.
Copying to a regular *persistent* protected file seems a cleaner
approach, compared to tmpfs.? We prototyped this solution, however it
does not seem to be a common pattern within the Kernel to write state
directly to files on disk file systems.? We considered two potential
options:

Option (A): (RECOMMENDED)
 ??????????? Let Kernel write the file using KM file APIs
--------------------------------------------
 ??? Use Kernel mode file APIs such as _file_open_root_, _vfs_llseek_,
 ??? and _vfs_write_ to use a persistent file on disk.? There is not
 ??? sufficient precedent for this pattern in the Kernel currently, so
 ??? we need guidance from area experts on the best mechanism to
 ??? implement this.

 ??? As for the location of the file, we suggest setting this in
 ??? KConfig. The file will be created by the Kernel, so it should be
 ??? protected from UM access.? Additionally, on a full boot, the
 ??? file should be cleared by the Kernel.

 ? POTENTIAL ISSUES AND MITIGATIONS
 ??? - handling IO errors from KM

 ????? A potential mitigation for this is to retry the failed write.
 ????? This assumes that the vfs_write Kernel method can handle any
 ????? failures gracefully without causing crashes.


 ??? - using file system paths to resolve the file from KM

 ????? Using the file_open_root seems to provide sufficient resiliency
 ????? against this.? The file can be located at a well known location
 ????? to minimize potential concerns.? However, any guidance in
 ????? minimizing squatting risks would be greatly appreciated.

 ??? - the file could be tampered by UM

 ????? There needs to be a lock/kernel-only-permission on the file
 ????? so that a UM process cannot tamper with the file.? A description
 ????? on how this file would be protected was provided at [2].


Option (B): (NOT RECOMMENDED)
 ??????????? Hand over the file writing to UM
--------------------------------------------
 ??? Alternately, UM could write the file contents and decide the
 ??? location of the file on disk.? If we want to preserve the Kernel
 ??? behavior of rendering the IMA log as a single monolithic log on
 ??? client (as ascii/binary_runtime_measurements file), the
 ??? Kernel will have to read from a file written by UM and combine it
 ??? with the remaining in-kernel-memory IMA log events.

 ??? This approach of a Kernel reading from a UM written file as an
 ??? IMA log violates the call stack, and thus it is not recommended.

If UM is to handle writing the IMA log to disk, the Kernel cannot read
those snapshotted events again.? Integrity of the IMA log can still be
maintained - a solution implementing this was in the original proposal
we submitted [1].

We appreciate the community's feedback on helping mold this feature to a
suitable implementation.

Thanks,
Sush and Tushar.


References:
[1]
https://lore.kernel.org/linux-integrity/c5737141-7827-1c83-ab38-0119dcfea485 at linux.microsoft.com/ 


[2]
https://lore.kernel.org/linux-integrity/CAOQ4uxiBAGKco1BKgyLOMY54r_Ck2jnvz8RCFODD-V87CGqLEw at mail.gmail.com/ 

>>>
>>> What is the real problem - kernel memory pressure, memory pressure in
>>> general, or disk space?  Is the intention to remove or offload the
>>> exported measurements?
>> The main concern is the memory pressure on both the kernel and the
>> attestation client
>> when it sends the request.  The concern you bring up is valid and we are
>> working on
>> creating a prototype.  There is no intention to remove the exported
>> measurements.
> Glad to hear that you're not intending to remove the exported
> measurements.
>
> Defining and including a new record in the measurement list measurement
> is fine, if it helps with attestation and doesn't require pausing the
> measurements.
>



From zohar at linux.ibm.com  Mon Aug 21 16:07:41 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Mon, 21 Aug 2023 19:07:41 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
	 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
	 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
	 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
Message-ID: <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>

On Mon, 2023-08-21 at 15:05 -0700, Sush Shringarputale wrote:
> 
> On 8/14/2023 3:02 PM, Mimi Zohar wrote:
> > On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
> >>> This design seems overly complex and requires synchronization between
> >>> the "snapshot" record and exporting the records from the measurement
> >>> list.  None of this would be necessary if the measurements were copied
> >>> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
> Even if the Kernel maintains the link between a tmpfs exported and an
> in-memory IMA log - it still has to copy the tmpfs portion to the
> Kernel memory during kexec soft boot.  tmpfs is cleared during kexec,
> so this copying of tmpfs back to kernel memory is necessary to preserve
> the integrity of the log during kexec.  But the copying would add back
> the memory pressure on the node during kexec (which may result in
> out-of-memory), defeating the purpose of the overall effort/feature.
> Copying to a regular *persistent* protected file seems a cleaner
> approach, compared to tmpfs.

>From a kernel perspective, it doesn't make a difference if userspace
provides a tmpfs or persistent file.  As per the discussion 
https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t
, userspace provides the kernel with the file descriptor of the opened
file.

> We prototyped this solution, however it
> does not seem to be a common pattern within the Kernel to write state
> directly to files on disk file systems.  We considered two potential
> options:

If no file descriptor is provided, then the measurements aren't copied
and removed from the securityfs file.  If there are write errors, the
measurements aren't removed from the securityfs file until the write
errors are resolved.

Mimi

> 
> Option (A): (RECOMMENDED)
>              Let Kernel write the file using KM file APIs
> --------------------------------------------
>      Use Kernel mode file APIs such as _file_open_root_, _vfs_llseek_,
>      and _vfs_write_ to use a persistent file on disk.  There is not
>      sufficient precedent for this pattern in the Kernel currently, so
>      we need guidance from area experts on the best mechanism to
>      implement this.
> 
>      As for the location of the file, we suggest setting this in
>      KConfig. The file will be created by the Kernel, so it should be
>      protected from UM access.  Additionally, on a full boot, the
>      file should be cleared by the Kernel.
> 
>    POTENTIAL ISSUES AND MITIGATIONS
>      - handling IO errors from KM
> 
>        A potential mitigation for this is to retry the failed write.
>        This assumes that the vfs_write Kernel method can handle any
>        failures gracefully without causing crashes.
> 
> 
>      - using file system paths to resolve the file from KM
> 
>        Using the file_open_root seems to provide sufficient resiliency
>        against this.  The file can be located at a well known location
>        to minimize potential concerns.  However, any guidance in
>        minimizing squatting risks would be greatly appreciated.
> 
>      - the file could be tampered by UM
> 
>        There needs to be a lock/kernel-only-permission on the file
>        so that a UM process cannot tamper with the file.  A description
>        on how this file would be protected was provided at [2].
> 
> Option (B): (NOT RECOMMENDED)
>              Hand over the file writing to UM
> --------------------------------------------
>      Alternately, UM could write the file contents and decide the
>      location of the file on disk.  If we want to preserve the Kernel
>      behavior of rendering the IMA log as a single monolithic log on
>      client (as ascii/binary_runtime_measurements file), the
>      Kernel will have to read from a file written by UM and combine it
>      with the remaining in-kernel-memory IMA log events.
> 
>      This approach of a Kernel reading from a UM written file as an
>      IMA log violates the call stack, and thus it is not recommended.
> 
> If UM is to handle writing the IMA log to disk, the Kernel cannot read
> those snapshotted events again.  Integrity of the IMA log can still be
> maintained - a solution implementing this was in the original proposal
> we submitted [1].
> 
> We appreciate the community's feedback on helping mold this feature to a
> suitable implementation.
> 
> Thanks,
> Sush and Tushar.
> 
> 
> References:
> [1]
> https://lore.kernel.org/linux-integrity/c5737141-7827-1c83-ab38-0119dcfea485 at linux.microsoft.com/ 
> 
> 
> [2]
> https://lore.kernel.org/linux-integrity/CAOQ4uxiBAGKco1BKgyLOMY54r_Ck2jnvz8RCFODD-V87CGqLEw at mail.gmail.com/ 
> 
> >>>
> >>> What is the real problem - kernel memory pressure, memory pressure in
> >>> general, or disk space?  Is the intention to remove or offload the
> >>> exported measurements?
> >> The main concern is the memory pressure on both the kernel and the
> >> attestation client
> >> when it sends the request.  The concern you bring up is valid and we are
> >> working on
> >> creating a prototype.  There is no intention to remove the exported
> >> measurements.
> > Glad to hear that you're not intending to remove the exported
> > measurements.
> >
> > Defining and including a new record in the measurement list measurement
> > is fine, if it helps with attestation and doesn't require pausing the
> > measurements.
> >
> 




From alexander.kamensky42 at gmail.com  Mon Aug 21 20:54:28 2023
From: alexander.kamensky42 at gmail.com (Alexander Kamensky)
Date: Mon, 21 Aug 2023 20:54:28 -0700
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64 platform
In-Reply-To: <MN0PR12MB5953FFB655A208BD4FA32EE4B71EA@MN0PR12MB5953.namprd12.prod.outlook.com>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv> <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNogKKZCp12Q92jh@MiWiFi-R3L-srv> <MN0PR12MB5953FFB655A208BD4FA32EE4B71EA@MN0PR12MB5953.namprd12.prod.outlook.com>
Message-ID: <CACBYKvRJL_WmZUN7BH_TjxX+TC2O1gjsudkh1GtRyB3WifjMJA@mail.gmail.com>

Hi All,

Sorry for the top post, but I believe that I might have posted a fix
for this issue a couple days ago.

Please check and see if it helps
https://lore.kernel.org/kexec/20230819191119.975299-1-alexander.kamensky42 at gmail.com/T/#u

Explanation for the issue I observed with a similar secondary kernel
traceback on arm64 is in the commit message.

Best Regards,
Alexander Kamensky

On Mon, Aug 21, 2023 at 5:25?AM Pandey, Radhey Shyam
<radhey.shyam.pandey at amd.com> wrote:
>
> > -----Original Message-----
> > From: bhe at redhat.com <bhe at redhat.com>
> > Sent: Monday, August 14, 2023 6:08 PM
> > To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
> > Cc: piliu at redhat.com; kexec at lists.infradead.org; linux-
> > kernel at vger.kernel.org; Sarangi, Anirudha <anirudha.sarangi at amd.com>
> > Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> > arm64 platform
> >
> > On 08/11/23 at 01:27pm, Pandey, Radhey Shyam wrote:
> > > > -----Original Message-----
> > > > From: bhe at redhat.com <bhe at redhat.com>
> > > > Sent: Wednesday, August 9, 2023 7:42 AM
> > > > To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>;
> > > > piliu at redhat.com
> > > > Cc: kexec at lists.infradead.org; linux-kernel at vger.kernel.org
> > > > Subject: Re: kexec reports "Cannot get kernel _text symbol address"
> > > > on
> > > > arm64 platform
> > > >
> > > > On 08/08/23 at 07:17pm, Pandey, Radhey Shyam wrote:
> > > > > Hi,
> > > > >
> > > > > I am trying to bring up kdump on arm64 platform[1]. But I get
> > > > > "Cannot get
> > > > kernel _text symbol address".
> > > > >
> > > > > Is there some Dump-capture kernel config options that I am missing?
> > > > >
> > > > > FYI, copied below complete kexec debug log.
> > > > >
> > > > > [1]: https://www.xilinx.com/products/boards-and-kits/vck190.html
> > > >
> > > > Your description isn't clear. You saw the printing, then your kdump
> > > > kernel loading succeeded or not?
> > > >
> > > > If no, have you tried applying Pingfan's patchset and still saw the issue?
> > > >
> > > > [PATCHv7 0/5] arm64: zboot support
> > > > https://lore.kernel.org/all/20230803024152.11663-1-piliu at redhat.com/
> > > > T/#u
> > >
> > > I was able to proceed further with loading with crash kernel on triggering
> > system crash.
> > > echo c > /proc/sysrq-trigger
> > >
> > > But when I copy /proc/vmcore it throws memory abort. Also I see size of
> > /proc/vmcore really huge (18446603353488633856).
> > > Any possible guess on what could be wrong?
> >
> > I didn't reproduce this issue on a arm64 baremetal system with the latest
> > kernel. From the log, It could be the iov_iter convertion patch which caused
> > this. Can you revert below patch to see if it works?
> >
> > 5d8de293c224 vmcore: convert copy_oldmem_page() to take an iov_iter
>
> Revert of this commit resulted in lot of conflicts. So as a safer side I checkout
> v5.18 kernel version before above commit. Still I see the same issue.
>
> / # ls -lrth /proc/vmcore
> -r--------    1 root     root       16.0E Aug 21 12:16 /proc/vmcore
> / # dmesg | grep -i 5.18
> [    0.000000] Linux version 5.18.0-00001-g689fdf110e63-dirty (radheys at xhdradheys41) (aarch64-xilinx-linux-gcc.real (GCC) 12.2.0, GNU ld (GNU Binutils) 2.39.0.20220819) #37 SMP Mon Aug 21 17:38:24 IST 2023
> [    2.494578] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.18
> [    2.514941] usb usb1: Manufacturer: Linux 5.18.0-00001-g689fdf110e63-dirty xhci-hcd
> [    2.555265] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.18
> [    2.575621] usb usb2: Manufacturer: Linux 5.18.0-00001-g689fdf110e63-dirty xhci-hcd
> [    3.152182] usb 1-1: new high-speed USB device number 2 using xhci-hcd
> / # cp /proc/vmcore dump
> [   86.204704] Unable to handle kernel level 3 address size fault at virtual address ffff800009b75000
> [   86.213677] Mem abort info:
> [   86.216464]   ESR = 0x96000003
> [   86.219508]   EC = 0x25: DABT (current EL), IL = 32 bits
> [   86.224812]   SET = 0, FnV = 0
> [   86.227856]   EA = 0, S1PTW = 0
> [   86.230989]   FSC = 0x03: level 3 address size fault
> [   86.235945] Data abort info:
> [   86.238819]   ISV = 0, ISS = 0x00000003
> [   86.242646]   CM = 0, WnR = 0
> [   86.245608] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000043049000
> [   86.252304] [ffff800009b75000] pgd=1000000061ffe003, p4d=1000000061ffe003, pud=1000000061ffd003, pmd=1000000043c12003, pte=00687ffff8200703
> [   86.264828] Internal error: Oops: 96000003 [#1] SMP
> [   86.269696] Modules linked in:
> [   86.272741] CPU: 1 PID: 298 Comm: cp Not tainted 5.18.0-00001-g689fdf110e63-dirty #37
> [   86.280562] Hardware name: Xilinx Versal vck190 Eval board revA (DT)
> [   86.286905] pstate: 20000005 (nzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [   86.293857] pc : __arch_copy_to_user+0x180/0x240
> [   86.298475] lr : copy_oldmem_page+0xa8/0x110
> [   86.302738] sp : ffff80000af6bc50
> [   86.306041] x29: ffff80000af6bc50 x28: ffff8000097de3b0 x27: ffff8000097de228
> [   86.313170] x26: 0000000000000000 x25: ffff80000af6bd60 x24: 0000000000000000
> [   86.320299] x23: ffff800009b75000 x22: 0000000000000001 x21: 0000ffffffa1e5a8
> [   86.327427] x20: 0000000000000000 x19: 0000000000001000 x18: 0000000000000000
> [   86.334555] x17: 0000000000000000 x16: 0000000000000000 x15: ffff800009b75000
> [   86.341682] x14: ffff800009863568 x13: 0000000000000000 x12: ffff800008000000
> [   86.348810] x11: 00007ffff8201000 x10: ffff800009b75fff x9 : 0000000000000000
> [   86.355937] x8 : ffff800009b76000 x7 : 0400000000000001 x6 : 0000ffffffa1e5a8
> [   86.363065] x5 : 0000ffffffa1f5a8 x4 : 0000000000000000 x3 : 0000ffffffffffff
> [   86.370192] x2 : 0000000000000f80 x1 : ffff800009b75000 x0 : 0000ffffffa1e5a8
> [   86.377320] Call trace:
> [   86.379755]  __arch_copy_to_user+0x180/0x240
> [   86.384018]  read_from_oldmem.part.0+0x160/0x1f4
> [   86.388629]  read_vmcore+0xe4/0x214
> [   86.392109]  proc_reg_read+0xb0/0x100
> [   86.395763]  vfs_read+0x90/0x1dc
> [   86.398981]  ksys_read+0x70/0x10c
> [   86.402286]  __arm64_sys_read+0x20/0x30
> [   86.406111]  invoke_syscall+0x54/0x124
> [   86.409852]  el0_svc_common.constprop.0+0x44/0xec
> [   86.414547]  do_el0_svc+0x70/0x90
> [   86.417853]  el0_svc+0x50/0xa4
> [   86.420899]  el0t_64_sync_handler+0x10c/0x140
> [   86.425247]  el0t_64_sync+0x18c/0x190
> [   86.428902] Code: d503201f d503201f d503201f d503201f (a8c12027)
> [   86.434984] ---[ end trace 0000000000000000 ]---
> Segmentation fault
>
>
> >
> > >
> > >
> > > [   80.733523] Starting crashdump kernel...
> > > [   80.737435] Bye!
> > > [    0.000000] Booting Linux on physical CPU 0x0000000001 [0x410fd083]
> > > [    0.000000] Linux version 6.5.0-rc4-ge28001fb4e07
> > (radheys at xhdradheys41) (aarch64-xilinx-linux-gcc.real (GCC) 12.2.0, GNU ld
> > (GNU Binutils) 2.39.0.20220819) #23 SMP Fri Aug 11 16:25:34 IST 2023
> > > <snip>
> > >
> > >
> > >
> > > xilinx-vck190-20232:/run/media/mmcblk0p1# cat /proc/meminfo | head
> > > MemTotal:        2092876 kB
> > > MemFree:         1219928 kB
> > > MemAvailable:    1166004 kB
> > > Buffers:              32 kB
> > > Cached:           756952 kB
> > > SwapCached:            0 kB
> > > Active:             1480 kB
> > > Inactive:          24164 kB
> > > Active(anon):       1452 kB
> > > Inactive(anon):    24160 kB
> > > xilinx-vck190-20232:/run/media/mmcblk0p1# cp /proc/vmcore dump
> > > [  975.284865] Unable to handle kernel level 3 address size fault at
> > > virtual address ffff80008d7cf000 [  975.293871] Mem abort info:
> > > [  975.296669]   ESR = 0x0000000096000003
> > > [  975.300425]   EC = 0x25: DABT (current EL), IL = 32 bits
> > > [  975.305738]   SET = 0, FnV = 0
> > > [  975.308788]   EA = 0, S1PTW = 0
> > > [  975.311925]   FSC = 0x03: level 3 address size fault
> > > [  975.316888] Data abort info:
> > > [  975.319763]   ISV = 0, ISS = 0x00000003, ISS2 = 0x00000000
> > > [  975.325245]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > > [  975.330292]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> > > [  975.335599] swapper pgtable: 4k pages, 48-bit VAs,
> > > pgdp=000005016ef6b000 [  975.342297] [ffff80008d7cf000]
> > > pgd=10000501eddfe003, p4d=10000501eddfe003, pud=10000501eddfd003,
> > > pmd=100005017b695003, pte=00687fff84000703 [  975.354827] Internal
> > error: Oops: 0000000096000003 [#4] SMP [  975.360392] Modules linked in:
> > > 3  975.
> > > 63440] CBPrUo:a d0c aPID: 664 Comm: cp Tainted: G      D            6.5.0-rc4-
> > ge28001fb4e07 #23
> > > [  975.372822] Hardware name: Xilinx Versal vck190 Eval board revA
> > > (DT) [  975.379165] pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT
> > > -SSBS BTYPE=--) [  975.386119] pc : __memcpy+0x110/0x230 [
> > > 975.389783] lr : _copy_to_iter+0x3d8/0x4d0 [  975.393874] sp :
> > > ffff80008dc939a0 [  975.397178] x29: ffff80008dc939a0 x28:
> > > ffff05013c1bea30 x27: 0000000000001000 [  975.404309] x26:
> > > 0000000000001000 x25: 0000000000001000 x24: ffff80008d7cf000 [
> > > 975.411440] x23: 0000040000000000 x22: ffff80008dc93ba0 x21:
> > > 0000000000001000 [  975.418570] x20: ffff000000000000 x19:
> > > 0000000000001000 x18: 0000000000000000 [  975.425699] x17:
> > > 0000000000000000 x16: 0000000000000000 x15: 0140000000000000 [
> > > 975.432829] x14: ffff8500a9919000 x13: 0040000000000001 x12:
> > > 0000fffef6831000 [  975.439958] x11: ffff80008d9cf000 x10:
> > > 0000000000000000 x9 : 0000000000000000 [  975.447088] x8 :
> > > ffff80008d7d0000 x7 : ffff0501addfd358 x6 : 0400000000000001 [
> > > 975.454217] x5 : ffff0501370e9000 x4 : ffff80008d7d0000 x3 :
> > 0000000000000000 [  975.461346] x2 : 0000000000001000 x1 :
> > ffff80008d7cf000 x0 : ffff0501370e8000 [  975.468476] Call trace:
> > > [  975.470912]  __memcpy+0x110/0x230
> > > [  975.474221]  copy_oldmem_page+0x70/0xac [  975.478050]
> > > read_from_oldmem.part.0+0x120/0x188
> > > [  975.482663]  read_vmcore+0x14c/0x238 [  975.486231]
> > > proc_reg_read_iter+0x84/0xd8 [  975.490233]
> > > copy_splice_read+0x160/0x288 [  975.494236]
> > > vfs_splice_read+0xac/0x10c [  975.498063]
> > > splice_direct_to_actor+0xa4/0x26c [  975.502498]
> > > do_splice_direct+0x90/0xdc [  975.506325]  do_sendfile+0x344/0x454 [
> > > 975.509892]  __arm64_sys_sendfile64+0x134/0x140
> > > [  975.514415]  invoke_syscall+0x54/0x124 [  975.518157]
> > > el0_svc_common.constprop.0+0xc4/0xe4
> > > [  975.522854]  do_el0_svc+0x38/0x98
> > > [  975.526162]  el0_svc+0x2c/0x84
> > > [  975.529211]  el0t_64_sync_handler+0x100/0x12c [  975.533562]
> > > el0t_64_sync+0x190/0x194 [  975.537218] Code: cb01000e b4fffc2e
> > > eb0201df 540004a3 (a940342c) [  975.543302] ---[ end trace
> > > 0000000000000000 ]--- t message from
> > > systemd-journald at xilinx-vck190-20232 (Tue 2022-11-08 14:16:20 UTC):
> > >
> > > kernel[539]: [  975.354827] Internal error: Oops: 0000000096000003
> > > [#4] SMP
> > >
> > >
> > > Broadcast message from systemd-journald at xilinx-vck190-20232 (Tue
> > 2022-11-08 14:16:20 UTC):
> > >
> > > kernel[539]: [  975.537218] Code: cb01000e b4fffc2e eb0201df 540004a3
> > > (a940342c)
> > >
> > > Segmentation fault
> > > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lrth /proc/vmcore
> > > -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> > > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -lh /proc/vmcore
> > > -r--------    1 root     root       16.0E Nov  8 14:05 /proc/vmcore
> > > xilinx-vck190-20232:/run/media/mmcblk0p1# ls -l /proc/vmcore
> > > -r--------    1 root     root     18446603353488633856 Nov  8 14:05
> > /proc/vmcore
> > >
> > > Thanks,
> > > Radhey
> > >
>
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec


From radhey.shyam.pandey at amd.com  Wed Aug 23 12:09:16 2023
From: radhey.shyam.pandey at amd.com (Pandey, Radhey Shyam)
Date: Wed, 23 Aug 2023 19:09:16 +0000
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64
 platform
In-Reply-To: <CACBYKvRJL_WmZUN7BH_TjxX+TC2O1gjsudkh1GtRyB3WifjMJA@mail.gmail.com>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>
 <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNogKKZCp12Q92jh@MiWiFi-R3L-srv>
 <MN0PR12MB5953FFB655A208BD4FA32EE4B71EA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <CACBYKvRJL_WmZUN7BH_TjxX+TC2O1gjsudkh1GtRyB3WifjMJA@mail.gmail.com>
Message-ID: <MN0PR12MB59532ABFAF131AC84F36B73FB71CA@MN0PR12MB5953.namprd12.prod.outlook.com>

> -----Original Message-----
> From: Alexander Kamensky <alexander.kamensky42 at gmail.com>
> Sent: Tuesday, August 22, 2023 9:24 AM
> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
> Cc: bhe at redhat.com; piliu at redhat.com; kexec at lists.infradead.org; linux-
> kernel at vger.kernel.org; Sarangi, Anirudha <anirudha.sarangi at amd.com>
> Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> arm64 platform
> 
> Hi All,
> 
> Sorry for the top post, but I believe that I might have posted a fix for this
> issue a couple days ago.
> 
> Please check and see if it helps
> https://lore.kernel.org/kexec/20230819191119.975299-1-
> alexander.kamensky42 at gmail.com/T/#u
> 
> Explanation for the issue I observed with a similar secondary kernel
> traceback on arm64 is in the commit message.

I fetched latest kexec sources (777ca453ca69e46f7376b07ba6727bd261ec97ef)
and applied above patch, a bit improved but still vmcore size is huge.

/ # ls -lrth /proc/vmcore 
-r--------    1 root     root       15.5G Aug 23 18:55 /proc/vmcore

Thanks,
Radhey

From alexander.kamensky42 at gmail.com  Thu Aug 24 08:05:08 2023
From: alexander.kamensky42 at gmail.com (Alexander Kamensky)
Date: Thu, 24 Aug 2023 08:05:08 -0700
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64 platform
In-Reply-To: <MN0PR12MB59532ABFAF131AC84F36B73FB71CA@MN0PR12MB5953.namprd12.prod.outlook.com>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv> <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNogKKZCp12Q92jh@MiWiFi-R3L-srv> <MN0PR12MB5953FFB655A208BD4FA32EE4B71EA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <CACBYKvRJL_WmZUN7BH_TjxX+TC2O1gjsudkh1GtRyB3WifjMJA@mail.gmail.com> <MN0PR12MB59532ABFAF131AC84F36B73FB71CA@MN0PR12MB5953.namprd12.prod.outlook.com>
Message-ID: <CACBYKvT6AzupfrHZ58BYM8xi8TDLeppqD+Lt=rf9hfmSew+nmQ@mail.gmail.com>

On Wed, Aug 23, 2023 at 12:09?PM Pandey, Radhey Shyam
<radhey.shyam.pandey at amd.com> wrote:
>
> > -----Original Message-----
> > From: Alexander Kamensky <alexander.kamensky42 at gmail.com>
> > Sent: Tuesday, August 22, 2023 9:24 AM
> > To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
> > Cc: bhe at redhat.com; piliu at redhat.com; kexec at lists.infradead.org; linux-
> > kernel at vger.kernel.org; Sarangi, Anirudha <anirudha.sarangi at amd.com>
> > Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> > arm64 platform
> >
> > Hi All,
> >
> > Sorry for the top post, but I believe that I might have posted a fix for this
> > issue a couple days ago.
> >
> > Please check and see if it helps
> > https://lore.kernel.org/kexec/20230819191119.975299-1-
> > alexander.kamensky42 at gmail.com/T/#u
> >
> > Explanation for the issue I observed with a similar secondary kernel
> > traceback on arm64 is in the commit message.
>
> I fetched latest kexec sources (777ca453ca69e46f7376b07ba6727bd261ec97ef)
> and applied above patch, a bit improved but still vmcore size is huge.
>
> / # ls -lrth /proc/vmcore
> -r--------    1 root     root       15.5G Aug 23 18:55 /proc/vmcore
>
How big is your system memory? If it is 16G then the above is normal.

The most important thing is can your secondary kernel read it? For example

cat /proc/vmcore > /dev/null

If you want to capture only kernel core out of /proc/vmcore you need
to use makedumpfile
to filter out the rest of the memory.

Thanks,
Alexander

> Thanks,
> Radhey


From piliu at redhat.com  Thu Aug 24 23:16:53 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Fri, 25 Aug 2023 14:16:53 +0800
Subject: [PATCH] aarch64: kern_paddr_start calculation makes more accurate
 legacy check
In-Reply-To: <20230819191119.975299-1-alexander.kamensky42@gmail.com>
References: <20230819191119.975299-1-alexander.kamensky42@gmail.com>
Message-ID: <CAF+s44THuTneZr3ny1_FeB8b1+rytySadU4HMLEthnfxzF_F1A@mail.gmail.com>

On Sun, Aug 20, 2023 at 3:16?AM Alexander Kamensky
<alexander.kamensky42 at gmail.com> wrote:
>
> In the latest openembedded with aarch64 image that uses 6.4.3 kernel on qemu I
> tried to run makedumpfile in secondary kernel with /proc/vmcore. It failed as
> follows:
>
> > root at qemuarm64:~# makedumpfile -c -F /proc/vmcore > /dev/null
> > read_from_vmcore: Can't seek the dump memory(/proc/vmcore). (offset: ffffffc0123fa000) Invalid argument
> > readpage_elf: Can't read the dump memory(/proc/vmcore).
> > <snip>
>
> It turns out that not all parts for /proc/vmcore are readable:
>
> > root at qemuarm64:~# cat /proc/vmcore > /dev/null
> > [  136.394931] Internal error: synchronous external abort: 0000000096000010 [#1] PREEMPT SMP
> > [  136.422434] Modules linked in:
> > <snip>
>
> Binary search of different kernel versions I found that the issue goes back to
> 5.11. It works fine with 5.10 version of the kernel.
>
> After running secondary kernel under qemu gdb, I've found that the primary kernel
> main memory segment in elfcorehdr has wrong address and size.
>
> Looking at kexec output with debug enabled at the time when it loads secondary
> crash kernel I saw the following:
>
> > Kernel text Elf header: p_type = 1, p_offset = 0x4030200000 p_paddr = 0x4030200000 p_vaddr = 0xffffffffffffffff p_filesz = 0xffffffc011410000 p_memsz = 0xffffffc011410000
>
> These values come from elf_info variable and are set in iomem_range_callback
> function. The function gets the following input on my system:
>
> root at qemuarm64:~# cat /proc/iomem | grep "Kernel code"
>   40210000-40feffff : Kernel code
> root at qemuarm64:~# cat /proc/kallsyms | grep -e ' _text$'

This is the crux of the problem. Could you say something about how it
disappeared?

On my side, I can see this info on 6.2.14-300.fc38.aarch64

grep -w _text /proc/kallsyms
ffffc705d8500000 T _text

Thanks,

Pingfan
> root at qemuarm64:~# cat /proc/kallsyms | grep -e ' _stext$'
> ffffffc010010000 T _stext
> root at qemuarm64:~# cat /proc/kallsyms | grep -e ' __init_begin$'
> ffffffc010df0000 T __init_begin
>
> the function calculates elf_info.kern_paddr_start address similar to
> these manual steps:
>
> base = 0x40210000
> length = 0x40feffff - 0x40210000 + 0x1 = 0xde0000
> kva_text_end - kva_stext = 0xffffffc010df0000 - 0xffffffc010010000 = 0xde0000
>
> elf_info.kern_paddr_start = base - (kva_stext - kva_text) =
>                       0x40210000 - 0xffffffc010010000 = 0x4030200000
>
> since 'length' and 'kva_text_end - kva_stext' are equal the function calculate
> elf_info.kern_paddr_start as in legacy case:
>
>   if (kva_text_end - kva_stext == length)
>      elf_info.kern_paddr_start = base - (kva_stext - kva_text); // <---------
>   else
>      elf_info.kern_paddr_start = base;
>
> but my kernel is not legacy and kva_text is zero.
>
> The fix is just to add a check for kva_text not being zero before following the
> legacy case.
>
> Signed-off-by: Alexander Kamensky <alexander.kamensky42 at gmail.com>
> ---
>  kexec/arch/arm64/crashdump-arm64.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kexec/arch/arm64/crashdump-arm64.c b/kexec/arch/arm64/crashdump-arm64.c
> index 3098315..639c82a 100644
> --- a/kexec/arch/arm64/crashdump-arm64.c
> +++ b/kexec/arch/arm64/crashdump-arm64.c
> @@ -82,7 +82,7 @@ static int iomem_range_callback(void *UNUSED(data), int UNUSED(nr),
>                  * For compatibility, deduce by comparing the gap "__init_begin - _stext"
>                  * and the res size of "Kernel code" in /proc/iomem
>                  */
> -               if (kva_text_end - kva_stext == length)
> +               if ((kva_text_end - kva_stext == length) && (kva_text != 0))
>                         elf_info.kern_paddr_start = base - (kva_stext - kva_text);
>                 else
>                         elf_info.kern_paddr_start = base;
> --
> 2.41.0
>
>
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
>



From alexander.kamensky42 at gmail.com  Sat Aug 26 13:23:00 2023
From: alexander.kamensky42 at gmail.com (Alexander Kamensky)
Date: Sat, 26 Aug 2023 13:23:00 -0700
Subject: [PATCH] aarch64: kern_paddr_start calculation makes more accurate
 legacy check
In-Reply-To: <CAF+s44THuTneZr3ny1_FeB8b1+rytySadU4HMLEthnfxzF_F1A@mail.gmail.com>
References: <20230819191119.975299-1-alexander.kamensky42@gmail.com> <CAF+s44THuTneZr3ny1_FeB8b1+rytySadU4HMLEthnfxzF_F1A@mail.gmail.com>
Message-ID: <CACBYKvSs0P2MAfx-oeaRKi_V2m8iKFd_Ko7wBD6C3AQUtUUaYQ@mail.gmail.com>

Hi Pingfan,

Please see inline

On Thu, Aug 24, 2023 at 11:17?PM Pingfan Liu <piliu at redhat.com> wrote:
>
> On Sun, Aug 20, 2023 at 3:16?AM Alexander Kamensky
> <alexander.kamensky42 at gmail.com> wrote:
> >
> > In the latest openembedded with aarch64 image that uses 6.4.3 kernel on qemu I
> > tried to run makedumpfile in secondary kernel with /proc/vmcore. It failed as
> > follows:
> >
> > > root at qemuarm64:~# makedumpfile -c -F /proc/vmcore > /dev/null
> > > read_from_vmcore: Can't seek the dump memory(/proc/vmcore). (offset: ffffffc0123fa000) Invalid argument
> > > readpage_elf: Can't read the dump memory(/proc/vmcore).
> > > <snip>
> >
> > It turns out that not all parts for /proc/vmcore are readable:
> >
> > > root at qemuarm64:~# cat /proc/vmcore > /dev/null
> > > [  136.394931] Internal error: synchronous external abort: 0000000096000010 [#1] PREEMPT SMP
> > > [  136.422434] Modules linked in:
> > > <snip>
> >
> > Binary search of different kernel versions I found that the issue goes back to
> > 5.11. It works fine with 5.10 version of the kernel.
> >
> > After running secondary kernel under qemu gdb, I've found that the primary kernel
> > main memory segment in elfcorehdr has wrong address and size.
> >
> > Looking at kexec output with debug enabled at the time when it loads secondary
> > crash kernel I saw the following:
> >
> > > Kernel text Elf header: p_type = 1, p_offset = 0x4030200000 p_paddr = 0x4030200000 p_vaddr = 0xffffffffffffffff p_filesz = 0xffffffc011410000 p_memsz = 0xffffffc011410000
> >
> > These values come from elf_info variable and are set in iomem_range_callback
> > function. The function gets the following input on my system:
> >
> > root at qemuarm64:~# cat /proc/iomem | grep "Kernel code"
> >   40210000-40feffff : Kernel code
> > root at qemuarm64:~# cat /proc/kallsyms | grep -e ' _text$'
>
> This is the crux of the problem. Could you say something about how it
> disappeared?

Here is what I found:

* in kallsyms .tmp_vmlinux.kallsyms1.syms _text I have this:

ffffffc008000000 T _text
ffffffc008010000 T __irqentry_text_start
ffffffc008010000 T _stext

* in scripts/kallsyms.c ranges defined as

static struct addr_range text_ranges[] = {
    { "_stext",     "_etext"     },
    { "_sinittext", "_einittext" },
};

* as a result _text fails check_symbol_range function its not included in
kallsyms

* from 'gdb vmlinux'

(gdb) p _text
$1 = 0xffffffc008000000 <_text> "MZ@\372'd>\024"
(gdb) p _stext
$2 = 0xffffffc008010000 <_stext> "\037 \003\325\037 \003\325\037
\003\325\037 \003\325?#\003\325\375{\273\251\375\003"

* from kernel linker script ./arch/arm64/kernel/vmlinux.lds, note
_stext follows _text and aligned by 0x00010000

OUTPUT_ARCH(aarch64)
ENTRY(_text)
jiffies = jiffies_64;
PECOFF_FILE_ALIGNMENT = 0x200;
SECTIONS
{
 /DISCARD/ : { *(.exitcall.exit) *(.discard) *(.discard.*) *(.modinfo)
*(.gnu.version*) }
 /DISCARD/ : {
  *(.interp .dynamic)
  *(.dynsym .dynstr .hash .gnu.hash)
 }
 . = ((((-(((1)) << ((((39))) - 1)))) + (0x08000000)));
 .head.text : {
  _text = .; // <------------------------------------------------
  KEEP(*(.head.text))
 }
 .text : ALIGN(0x00010000) { // <--------------------------------
  _stext = .; // <-----------------------------------------------
   . = ALIGN(8); __irqentry_text_start = .; *(.irqentry.text)
__irqentry_text_end = .;
   . = ALIGN(8); __softirqentry_text_start = .; *(.softirqentry.text)
__softirqentry_text_end = .;

* ./arch/arm64/kernel/vmlinux.lds is preproccessed file from
arch/arm64/kernel/vmlinux.lds.S, here are relevant snippets:

    .head.text : {
        _text = .;
        HEAD_TEXT
    }
    .text : ALIGN(SEGMENT_ALIGN) {    /* Real text segment        */
        _stext = .;        /* Text and read-only data    */
            IRQENTRY_TEXT
            SOFTIRQENTRY_TEXT

* and SEGMENT_ALIGN defined in

arch/arm64/include/asm/memory.h:#define SEGMENT_ALIGN           SZ_64K

* symbol_valid function from scripts/kallsyms.c will ignore range
check if --all-symbols option is passed

* from scripts/link-vmlinux.sh one can see that --all-symbols will be
added only if CONFIG_KALLSYMS_ALL is enabled

kallsyms()
{
        local kallsymopt;

        if is_enabled CONFIG_KALLSYMS_ALL; then
                kallsymopt="${kallsymopt} --all-symbols"
        fi

* in my kernel CONFIG_KALLSYMS_ALL is not set

* I've checked aarch64 fedora 38 kernel config in there
CONFIG_KALLSYMS_ALL is enabled that explains why you see this _text in
kallsyms of fedora

* so far it seems that _text address cannot be part of kallsyms unless
CONFIG_KALLSYMS_ALL is enabled or original _text is 64k aligned so it
will coincide with _stext

* in all my testing even with 5.10 kernel _text was missing in
kallsyms, that it why I thought it presence constituted legacy case,
in 5.10 case last it worked for me length check was failing and code
was taking else part using _stext anyway

* I believe my patch is correct and it should work for both cases when
CONFIG_KALLSYMS_ALL is set or not.

* Maybe we can remove whole 'kva_text_end - kva_stext == length' check
and set 'elf_info.kern_paddr_start = base' unconditionally, but I am
concerned that some older kernels will trip on it

    if (kva_text_end - kva_stext == length)
        elf_info.kern_paddr_start = base - (kva_stext - kva_text);
    else
        elf_info.kern_paddr_start = base;

Thanks,
Alexander Kamensky

> On my side, I can see this info on 6.2.14-300.fc38.aarch64
>
> grep -w _text /proc/kallsyms
> ffffc705d8500000 T _text
>
> Thanks,
>
> Pingfan
> > root at qemuarm64:~# cat /proc/kallsyms | grep -e ' _stext$'
> > ffffffc010010000 T _stext
> > root at qemuarm64:~# cat /proc/kallsyms | grep -e ' __init_begin$'
> > ffffffc010df0000 T __init_begin
> >
> > the function calculates elf_info.kern_paddr_start address similar to
> > these manual steps:
> >
> > base = 0x40210000
> > length = 0x40feffff - 0x40210000 + 0x1 = 0xde0000
> > kva_text_end - kva_stext = 0xffffffc010df0000 - 0xffffffc010010000 = 0xde0000
> >
> > elf_info.kern_paddr_start = base - (kva_stext - kva_text) =
> >                       0x40210000 - 0xffffffc010010000 = 0x4030200000
> >
> > since 'length' and 'kva_text_end - kva_stext' are equal the function calculate
> > elf_info.kern_paddr_start as in legacy case:
> >
> >   if (kva_text_end - kva_stext == length)
> >      elf_info.kern_paddr_start = base - (kva_stext - kva_text); // <---------
> >   else
> >      elf_info.kern_paddr_start = base;
> >
> > but my kernel is not legacy and kva_text is zero.
> >
> > The fix is just to add a check for kva_text not being zero before following the
> > legacy case.
> >
> > Signed-off-by: Alexander Kamensky <alexander.kamensky42 at gmail.com>
> > ---
> >  kexec/arch/arm64/crashdump-arm64.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/kexec/arch/arm64/crashdump-arm64.c b/kexec/arch/arm64/crashdump-arm64.c
> > index 3098315..639c82a 100644
> > --- a/kexec/arch/arm64/crashdump-arm64.c
> > +++ b/kexec/arch/arm64/crashdump-arm64.c
> > @@ -82,7 +82,7 @@ static int iomem_range_callback(void *UNUSED(data), int UNUSED(nr),
> >                  * For compatibility, deduce by comparing the gap "__init_begin - _stext"
> >                  * and the res size of "Kernel code" in /proc/iomem
> >                  */
> > -               if (kva_text_end - kva_stext == length)
> > +               if ((kva_text_end - kva_stext == length) && (kva_text != 0))
> >                         elf_info.kern_paddr_start = base - (kva_stext - kva_text);
> >                 else
> >                         elf_info.kern_paddr_start = base;
> > --
> > 2.41.0
> >
> >
> > _______________________________________________
> > kexec mailing list
> > kexec at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kexec
> >
>


From bhe at redhat.com  Sun Aug 27 03:11:20 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:20 +0800
Subject: [PATCH 1/8] crash_core.c: remove unnecessary parameter of function
In-Reply-To: <20230827101128.70931-1-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
Message-ID: <20230827101128.70931-2-bhe@redhat.com>

In all call sites of __parse_crashkernel(), the parameter 'name' is
hardcoded as "crashkernel=". So remove the unnecessary parameter 'name',
add local varibale 'name' inside __parse_crashkernel() instead.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 kernel/crash_core.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 90ce1dfd591c..f27b4e45d410 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -241,11 +241,11 @@ static int __init __parse_crashkernel(char *cmdline,
 			     unsigned long long system_ram,
 			     unsigned long long *crash_size,
 			     unsigned long long *crash_base,
-			     const char *name,
 			     const char *suffix)
 {
 	char	*first_colon, *first_space;
 	char	*ck_cmdline;
+	char	*name = "crashkernel=";
 
 	BUG_ON(!crash_size || !crash_base);
 	*crash_size = 0;
@@ -283,7 +283,7 @@ int __init parse_crashkernel(char *cmdline,
 			     unsigned long long *crash_base)
 {
 	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-					"crashkernel=", NULL);
+				NULL);
 }
 
 int __init parse_crashkernel_high(char *cmdline,
@@ -292,7 +292,7 @@ int __init parse_crashkernel_high(char *cmdline,
 			     unsigned long long *crash_base)
 {
 	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				"crashkernel=", suffix_tbl[SUFFIX_HIGH]);
+				suffix_tbl[SUFFIX_HIGH]);
 }
 
 int __init parse_crashkernel_low(char *cmdline,
@@ -301,7 +301,7 @@ int __init parse_crashkernel_low(char *cmdline,
 			     unsigned long long *crash_base)
 {
 	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				"crashkernel=", suffix_tbl[SUFFIX_LOW]);
+				suffix_tbl[SUFFIX_LOW]);
 }
 
 /*
-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:11:19 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:19 +0800
Subject: [PATCH 0/8] kdump: use generic functions to simplify crashkernel reservation in architectures
Message-ID: <20230827101128.70931-1-bhe@redhat.com>

In the current arm64, crashkernel=,high support has been finished after
several rounds of posting and careful reviewing. The code in arm64 which
parses crashkernel kernel parameters firstly, then reserve memory can be
a good example for other ARCH to refer to.

Whereas in x86_64, the code mixing crashkernel parameter parsing and
memory reserving is twisted, and looks messy. Refactoring the code to
make it more readable maintainable is necessary.

Here, firstly abstract the crashkernel parameter parsing code into
parse_crashkernel() to make it be able to parse crashkernel=,high|low.
Then abstract the crashkernel memory reserving code into a generic
function reserve_crashkernel_generic(). Finally, in ARCH which
crashkernel=,high support is needed, a simple arch_reserve_crashkernel()
can be added to call above two functions. This can remove the duplicated
implmentation code in each ARCH, like arm64, x86_64.

I only change the arm64 and x86_64 implementation to make use of the
generic functions to simplify code. Risc-v can be done very easily refer
to the steps in arm64 and x86_64.

History:
- RFC patchset:
  https://lore.kernel.org/all/20230619055951.45620-1-bhe at redhat.com/T/#u
  [RFC PATCH 0/4] kdump: add generic functions to simplify crashkernel crashkernel in architecture
  Dave and Philipp commented the old parse_crashkernel_common() and
  parse_crashkernel_generic() are quite confusing. In this formal post,
  I made change to address the concern by unifying all crashkernel
  parsing into parse_crashkernel().

Baoquan He (8):
  crash_core.c: remove unnecessary parameter of function
  crash_core: change the prototype of function parse_crashkernel()
  include/linux/kexec.h: move down crash_core.h including
  crash_core: change parse_crashkernel() to support
    crashkernel=,high|low parsing
  crash_core: add generic function to do reservation
  arm64: kdump: use generic interface to simplify crashkernel
    reservation
  x86: kdump: use generic interface to simplify crashkernel reservation
    code
  crash_core.c: remove unneeded functions

 arch/arm/kernel/setup.c              |   3 +-
 arch/arm64/Kconfig                   |   3 +
 arch/arm64/include/asm/kexec.h       |   5 +
 arch/arm64/mm/init.c                 | 140 ++--------------------
 arch/ia64/kernel/setup.c             |   2 +-
 arch/loongarch/kernel/setup.c        |   4 +-
 arch/mips/kernel/setup.c             |   3 +-
 arch/powerpc/kernel/fadump.c         |   2 +-
 arch/powerpc/kexec/core.c            |   2 +-
 arch/powerpc/mm/nohash/kaslr_booke.c |   2 +-
 arch/riscv/mm/init.c                 |   2 +-
 arch/s390/kernel/setup.c             |   4 +-
 arch/sh/kernel/machine_kexec.c       |   2 +-
 arch/x86/Kconfig                     |   3 +
 arch/x86/include/asm/kexec.h         |  32 +++++
 arch/x86/kernel/setup.c              | 143 +++-------------------
 include/linux/crash_core.h           |  35 +++++-
 include/linux/kexec.h                |  14 +--
 kernel/crash_core.c                  | 170 +++++++++++++++++++++++----
 19 files changed, 266 insertions(+), 305 deletions(-)

-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:11:21 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:21 +0800
Subject: [PATCH 2/8] crash_core: change the prototype of function parse_crashkernel()
In-Reply-To: <20230827101128.70931-1-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
Message-ID: <20230827101128.70931-3-bhe@redhat.com>

Add two parameters 'low_size' and 'high' to function parse_crashkernel(),
later crashkernel=,high|low parsing will be added. Make adjustments in all
call sites of parse_crashkernel() in arch.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 arch/arm/kernel/setup.c              |  3 ++-
 arch/arm64/mm/init.c                 |  2 +-
 arch/ia64/kernel/setup.c             |  2 +-
 arch/loongarch/kernel/setup.c        |  4 +++-
 arch/mips/kernel/setup.c             |  3 ++-
 arch/powerpc/kernel/fadump.c         |  2 +-
 arch/powerpc/kexec/core.c            |  2 +-
 arch/powerpc/mm/nohash/kaslr_booke.c |  2 +-
 arch/riscv/mm/init.c                 |  2 +-
 arch/s390/kernel/setup.c             |  4 ++--
 arch/sh/kernel/machine_kexec.c       |  2 +-
 arch/x86/kernel/setup.c              |  3 ++-
 include/linux/crash_core.h           |  3 ++-
 kernel/crash_core.c                  | 15 ++++++++++++---
 14 files changed, 32 insertions(+), 17 deletions(-)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index c66b560562b3..e2bb7afd0683 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -1010,7 +1010,8 @@ static void __init reserve_crashkernel(void)
 
 	total_mem = get_total_mem();
 	ret = parse_crashkernel(boot_command_line, total_mem,
-				&crash_size, &crash_base);
+				&crash_size, &crash_base,
+				NULL, NULL);
 	/* invalid value specified or crashkernel=0 */
 	if (ret || !crash_size)
 		return;
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index d31c3a9290c5..36967b82c150 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -115,7 +115,7 @@ static void __init reserve_crashkernel(void)
 
 	/* crashkernel=X[@offset] */
 	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
-				&crash_size, &crash_base);
+				&crash_size, &crash_base, NULL, NULL);
 	if (ret == -ENOENT) {
 		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
 		if (ret || !crash_size)
diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
index 5a55ac82c13a..4faea2d2cf07 100644
--- a/arch/ia64/kernel/setup.c
+++ b/arch/ia64/kernel/setup.c
@@ -277,7 +277,7 @@ static void __init setup_crashkernel(unsigned long total, int *n)
 	int ret;
 
 	ret = parse_crashkernel(boot_command_line, total,
-			&size, &base);
+			&size, &base, NULL, NULL);
 	if (ret == 0 && size > 0) {
 		if (!base) {
 			sort_regions(rsvd_region, *n);
diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c
index 9d830ab4e302..776a068d8718 100644
--- a/arch/loongarch/kernel/setup.c
+++ b/arch/loongarch/kernel/setup.c
@@ -267,7 +267,9 @@ static void __init arch_parse_crashkernel(void)
 	unsigned long long crash_base, crash_size;
 
 	total_mem = memblock_phys_mem_size();
-	ret = parse_crashkernel(boot_command_line, total_mem, &crash_size, &crash_base);
+	ret = parse_crashkernel(boot_command_line, total_mem,
+				&crash_size, &crash_base,
+				NULL, NULL);
 	if (ret < 0 || crash_size <= 0)
 		return;
 
diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
index cb871eb784a7..08321c945ac4 100644
--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -460,7 +460,8 @@ static void __init mips_parse_crashkernel(void)
 
 	total_mem = memblock_phys_mem_size();
 	ret = parse_crashkernel(boot_command_line, total_mem,
-				&crash_size, &crash_base);
+				&crash_size, &crash_base,
+				NULL, NULL);
 	if (ret != 0 || crash_size <= 0)
 		return;
 
diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c
index ea0a073abd96..7dbdeba56e74 100644
--- a/arch/powerpc/kernel/fadump.c
+++ b/arch/powerpc/kernel/fadump.c
@@ -313,7 +313,7 @@ static __init u64 fadump_calculate_reserve_size(void)
 	 * memory at a predefined offset.
 	 */
 	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
-				&size, &base);
+				&size, &base, NULL, NULL);
 	if (ret == 0 && size > 0) {
 		unsigned long max_size;
 
diff --git a/arch/powerpc/kexec/core.c b/arch/powerpc/kexec/core.c
index de64c7962991..9346c960b296 100644
--- a/arch/powerpc/kexec/core.c
+++ b/arch/powerpc/kexec/core.c
@@ -109,7 +109,7 @@ void __init reserve_crashkernel(void)
 	total_mem_sz = memory_limit ? memory_limit : memblock_phys_mem_size();
 	/* use common parsing */
 	ret = parse_crashkernel(boot_command_line, total_mem_sz,
-			&crash_size, &crash_base);
+			&crash_size, &crash_base, NULL, NULL);
 	if (ret == 0 && crash_size > 0) {
 		crashk_res.start = crash_base;
 		crashk_res.end = crash_base + crash_size - 1;
diff --git a/arch/powerpc/mm/nohash/kaslr_booke.c b/arch/powerpc/mm/nohash/kaslr_booke.c
index 2fb3edafe9ab..b4f2786a7d2b 100644
--- a/arch/powerpc/mm/nohash/kaslr_booke.c
+++ b/arch/powerpc/mm/nohash/kaslr_booke.c
@@ -178,7 +178,7 @@ static void __init get_crash_kernel(void *fdt, unsigned long size)
 	int ret;
 
 	ret = parse_crashkernel(boot_command_line, size, &crash_size,
-				&crash_base);
+				&crash_base, NULL, NULL);
 	if (ret != 0 || crash_size == 0)
 		return;
 	if (crash_base == 0)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index e4c35ac2357f..a9ef0824f905 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -1332,7 +1332,7 @@ static void __init reserve_crashkernel(void)
 	}
 
 	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
-				&crash_size, &crash_base);
+				&crash_size, &crash_base, NULL, NULL);
 	if (ret || !crash_size)
 		return;
 
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index 00d76448319d..386a36181ceb 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -625,8 +625,8 @@ static void __init reserve_crashkernel(void)
 	phys_addr_t low, high;
 	int rc;
 
-	rc = parse_crashkernel(boot_command_line, ident_map_size, &crash_size,
-			       &crash_base);
+	rc = parse_crashkernel(boot_command_line, ident_map_size,
+			       &crash_size, &crash_base, NULL, NULL);
 
 	crash_base = ALIGN(crash_base, KEXEC_CRASH_MEM_ALIGN);
 	crash_size = ALIGN(crash_size, KEXEC_CRASH_MEM_ALIGN);
diff --git a/arch/sh/kernel/machine_kexec.c b/arch/sh/kernel/machine_kexec.c
index 223c14f44af7..fa3a7b36190a 100644
--- a/arch/sh/kernel/machine_kexec.c
+++ b/arch/sh/kernel/machine_kexec.c
@@ -154,7 +154,7 @@ void __init reserve_crashkernel(void)
 	int ret;
 
 	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
-			&crash_size, &crash_base);
+			&crash_size, &crash_base, NULL, NULL);
 	if (ret == 0 && crash_size > 0) {
 		crashk_res.start = crash_base;
 		crashk_res.end = crash_base + crash_size - 1;
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index fd975a4a5200..382c66d2cf71 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -558,7 +558,8 @@ static void __init reserve_crashkernel(void)
 	total_mem = memblock_phys_mem_size();
 
 	/* crashkernel=XM */
-	ret = parse_crashkernel(boot_command_line, total_mem, &crash_size, &crash_base);
+	ret = parse_crashkernel(boot_command_line, total_mem,
+				&crash_size, &crash_base, NULL, NULL);
 	if (ret != 0 || crash_size <= 0) {
 		/* crashkernel=X,high */
 		ret = parse_crashkernel_high(boot_command_line, total_mem,
diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index de62a722431e..2e76289699ff 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -78,7 +78,8 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 void final_note(Elf_Word *buf);
 
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
-		unsigned long long *crash_size, unsigned long long *crash_base);
+		unsigned long long *crash_size, unsigned long long *crash_base,
+		unsigned long long *low_size, bool *high);
 int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
 int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index f27b4e45d410..f6a5c219e2e1 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -280,10 +280,19 @@ static int __init __parse_crashkernel(char *cmdline,
 int __init parse_crashkernel(char *cmdline,
 			     unsigned long long system_ram,
 			     unsigned long long *crash_size,
-			     unsigned long long *crash_base)
+			     unsigned long long *crash_base,
+			     unsigned long long *low_size,
+			     bool *high)
 {
-	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				NULL);
+	int ret;
+
+	/* crashkernel=X[@offset] */
+	ret = __parse_crashkernel(cmdline, system_ram, crash_size,
+				crash_base, NULL);
+	if (!high)
+		return ret;
+
+	return 0;
 }
 
 int __init parse_crashkernel_high(char *cmdline,
-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:11:22 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:22 +0800
Subject: [PATCH 3/8] include/linux/kexec.h: move down crash_core.h including
In-Reply-To: <20230827101128.70931-1-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
Message-ID: <20230827101128.70931-4-bhe@redhat.com>

Later generic macros related to crash_core will be added into
<linux/crash_core.h>, and the corresponding arch specific macros will be
added to override them if needed. And Kconfig option KEXEC_CORE selects
CRASH_CORE. So move <linux/crash_core.h> including after <asm/kexec.h>.

And also move the crash_res|low_res and crash_notes delcarations after
<linux/crash_core.h> including because they are all defined in
kernel/kexec_core.c, and note_buf_t is definied in crash_core.h.

This is a preparation patch.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/kexec.h | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 22b5cd24f581..8768fd9e2a66 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -15,25 +15,25 @@
 
 #if !defined(__ASSEMBLY__)
 
-#include <linux/crash_core.h>
 #include <asm/io.h>
 #include <linux/range.h>
 
 #include <uapi/linux/kexec.h>
 #include <linux/verification.h>
 
-/* Location of a reserved region to hold the crash kernel.
- */
-extern struct resource crashk_res;
-extern struct resource crashk_low_res;
-extern note_buf_t __percpu *crash_notes;
-
 #ifdef CONFIG_KEXEC_CORE
 #include <linux/list.h>
 #include <linux/compat.h>
 #include <linux/ioport.h>
 #include <linux/module.h>
 #include <asm/kexec.h>
+#include <linux/crash_core.h>
+
+/* Location of a reserved region to hold the crash kernel.
+ */
+extern struct resource crashk_res;
+extern struct resource crashk_low_res;
+extern note_buf_t __percpu *crash_notes;
 
 /* Verify architecture specific macros are defined */
 
-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:11:23 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:23 +0800
Subject: [PATCH 4/8] crash_core: change parse_crashkernel() to support crashkernel=,high|low parsing
In-Reply-To: <20230827101128.70931-1-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
Message-ID: <20230827101128.70931-5-bhe@redhat.com>

Now parse_crashkernel() is a real entry point for all kinds of
crahskernel parsing on any architecture.

And wrap the crahskernel=,high|low handling inside
CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION ifdeffery scope.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |  6 ++++++
 kernel/crash_core.c        | 28 +++++++++++++++++++++++++++-
 2 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 2e76289699ff..85260bf4a734 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -77,6 +77,12 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len);
 void final_note(Elf_Word *buf);
 
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
+#define DEFAULT_CRASH_KERNEL_LOW_SIZE  (128UL << 20)
+#endif
+#endif
+
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base,
 		unsigned long long *low_size, bool *high);
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index f6a5c219e2e1..355b0ab5189c 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -276,6 +276,9 @@ static int __init __parse_crashkernel(char *cmdline,
 /*
  * That function is the entry point for command line parsing and should be
  * called from the arch-specific code.
+ *
+ * If crashkernel=,high|low is supported on architecture, non-NULL values
+ * should be passed to parameters 'low_size' and 'high'.
  */
 int __init parse_crashkernel(char *cmdline,
 			     unsigned long long system_ram,
@@ -291,7 +294,30 @@ int __init parse_crashkernel(char *cmdline,
 				crash_base, NULL);
 	if (!high)
 		return ret;
-
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+	else if (ret == -ENOENT) {
+		ret = __parse_crashkernel(cmdline, 0, crash_size,
+				crash_base, suffix_tbl[SUFFIX_HIGH]);
+		if (ret || !*crash_size)
+			return -1;
+
+		/*
+		 * crashkernel=Y,low can be specified or not, but invalid value
+		 * is not allowed.
+		 */
+		ret = __parse_crashkernel(cmdline, 0, low_size,
+				crash_base, suffix_tbl[SUFFIX_LOW]);
+		if (ret == -ENOENT)
+			*low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
+		else if (ret)
+			return -1;
+
+		*high = true;
+	} else if (ret || !*crash_size) {
+		/* The specified value is invalid */
+		return -1;
+	}
+#endif
 	return 0;
 }
 
-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:11:24 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:24 +0800
Subject: [PATCH 5/8] crash_core: add generic function to do reservation
In-Reply-To: <20230827101128.70931-1-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
Message-ID: <20230827101128.70931-6-bhe@redhat.com>

In architecture like x86_64, arm64 and riscv, they have vast virtual
address space and usually have huge physical memory RAM. Their
crashkernel reservation doesn't have to be limited under 4G RAM,
but can be extended to the whole physical memory via crashkernel=,high
support.

Now add function reserve_crashkernel_generic() to reserve crashkernel
memory if users specify any case of kernel pamameters, like
crashkernel=xM[@offset] or crashkernel=,high|low.

This is preparation to simplify code of crashkernel=,high support
in architecutures.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |  34 ++++++++++--
 kernel/crash_core.c        | 109 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 136 insertions(+), 7 deletions(-)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 85260bf4a734..2f732493e922 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -77,12 +77,6 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len);
 void final_note(Elf_Word *buf);
 
-#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
-#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
-#define DEFAULT_CRASH_KERNEL_LOW_SIZE  (128UL << 20)
-#endif
-#endif
-
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base,
 		unsigned long long *low_size, bool *high);
@@ -91,4 +85,32 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
 int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
 
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
+#define DEFAULT_CRASH_KERNEL_LOW_SIZE	(128UL << 20)
+#endif
+#ifndef CRASH_ALIGN
+#define CRASH_ALIGN			SZ_2M
+#endif
+#ifndef CRASH_ADDR_LOW_MAX
+#define CRASH_ADDR_LOW_MAX		SZ_4G
+#endif
+#ifndef CRASH_ADDR_HIGH_MAX
+#define CRASH_ADDR_HIGH_MAX		memblock_end_of_DRAM()
+#endif
+
+void __init reserve_crashkernel_generic(char *cmdline,
+		unsigned long long crash_size,
+		unsigned long long crash_base,
+		unsigned long long crash_low_size,
+		bool high);
+#else
+static inline void __init reserve_crashkernel_generic(char *cmdline,
+		unsigned long long crash_size,
+		unsigned long long crash_base,
+		unsigned long long crash_low_size,
+		bool high)
+{}
+#endif
+
 #endif /* LINUX_CRASH_CORE_H */
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 355b0ab5189c..6bc00cc390b5 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -5,11 +5,13 @@
  */
 
 #include <linux/buildid.h>
-#include <linux/crash_core.h>
 #include <linux/init.h>
 #include <linux/utsname.h>
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
+#include <linux/memblock.h>
+#include <linux/kexec.h>
+#include <linux/kmemleak.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -349,6 +351,111 @@ static int __init parse_crashkernel_dummy(char *arg)
 }
 early_param("crashkernel", parse_crashkernel_dummy);
 
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+static int __init reserve_crashkernel_low(unsigned long long low_size)
+{
+#ifdef CONFIG_64BIT
+	unsigned long long low_base;
+
+	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
+	if (!low_base) {
+		pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
+		return -ENOMEM;
+	}
+
+	pr_info("crashkernel low memory reserved: 0x%08llx - 0x%08llx (%lld MB)\n",
+		low_base, low_base + low_size, low_size >> 20);
+
+	crashk_low_res.start = low_base;
+	crashk_low_res.end   = low_base + low_size - 1;
+	insert_resource(&iomem_resource, &crashk_low_res);
+#endif
+	return 0;
+}
+
+void __init reserve_crashkernel_generic(char *cmdline,
+			     unsigned long long crash_size,
+			     unsigned long long crash_base,
+			     unsigned long long crash_low_size,
+			     bool high)
+{
+	unsigned long long search_end = CRASH_ADDR_LOW_MAX, search_base = 0;
+	bool fixed_base = false;
+
+	/* User specifies base address explicitly. */
+	if (crash_base) {
+		fixed_base = true;
+		search_base = crash_base;
+		search_end = crash_base + crash_size;
+	}
+
+	if (high) {
+		search_base = CRASH_ADDR_LOW_MAX;
+		search_end = CRASH_ADDR_HIGH_MAX;
+	}
+
+retry:
+	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
+					       search_base, search_end);
+	if (!crash_base) {
+		/*
+		 * For crashkernel=size[KMG]@offset[KMG], print out failure
+		 * message if can't reserve the specified region.
+		 */
+		if (fixed_base) {
+			pr_warn("crashkernel reservation failed - memory is in use.\n");
+			return;
+		}
+
+		/*
+		 * For crashkernel=size[KMG], if the first attempt was for
+		 * low memory, fall back to high memory, the minimum required
+		 * low memory will be reserved later.
+		 */
+		if (!high && search_end == CRASH_ADDR_LOW_MAX) {
+			search_end = CRASH_ADDR_HIGH_MAX;
+			search_base = CRASH_ADDR_LOW_MAX;
+			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
+			goto retry;
+		}
+
+		/*
+		 * For crashkernel=size[KMG],high, if the first attempt was
+		 * for high memory, fall back to low memory.
+		 */
+		if (high && search_end == CRASH_ADDR_HIGH_MAX) {
+			search_end = CRASH_ADDR_LOW_MAX;
+			search_base = 0;
+			goto retry;
+		}
+		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
+			crash_size);
+		return;
+	}
+
+	if ((crash_base > CRASH_ADDR_LOW_MAX) &&
+	     crash_low_size && reserve_crashkernel_low(crash_low_size)) {
+		memblock_phys_free(crash_base, crash_size);
+		return;
+	}
+
+	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
+		crash_base, crash_base + crash_size, crash_size >> 20);
+
+	/*
+	 * The crashkernel memory will be removed from the kernel linear
+	 * map. Inform kmemleak so that it won't try to access it.
+	 */
+	kmemleak_ignore_phys(crash_base);
+	if (crashk_low_res.end)
+		kmemleak_ignore_phys(crashk_low_res.start);
+
+	crashk_res.start = crash_base;
+	crashk_res.end = crash_base + crash_size - 1;
+	insert_resource(&iomem_resource, &crashk_res);
+}
+#endif
+
 Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len)
 {
-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:11:25 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:25 +0800
Subject: [PATCH 6/8] arm64: kdump: use generic interface to simplify crashkernel reservation
In-Reply-To: <20230827101128.70931-1-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
Message-ID: <20230827101128.70931-7-bhe@redhat.com>

With the help of newly changed function parse_crashkernel() and
generic reserve_crashkernel_generic(), crashkernel reservation can be
simplified by steps:

1) Provide CRASH_ALIGN, CRASH_ADDR_LOW_MAX, CRASH_ADDR_HIGH_MAX and
   DEFAULT_CRASH_KERNEL_LOW_SIZE in <asm/kexec.h>;

2) Add arch_reserve_crashkernel() to call parse_crashkernel() and
   reserve_crashkernel_generic();

3) Add ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION Kconfig in
   arch/arm64/Kconfig.

The old reserve_crashkernel_low() and reserve_crashkernel() can be
removed.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 arch/arm64/Kconfig             |   3 +
 arch/arm64/include/asm/kexec.h |   5 ++
 arch/arm64/mm/init.c           | 140 ++-------------------------------
 3 files changed, 16 insertions(+), 132 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index a2511b30d0f6..6739e7b456b9 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1481,6 +1481,9 @@ config KEXEC_FILE
 	  for kernel and initramfs as opposed to list of segments as
 	  accepted by previous system call.
 
+config ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+	def_bool CRASH_CORE
+
 config KEXEC_SIG
 	bool "Verify kernel signature during kexec_file_load() syscall"
 	depends on KEXEC_FILE
diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h
index 9ac9572a3bbe..442fc13588a2 100644
--- a/arch/arm64/include/asm/kexec.h
+++ b/arch/arm64/include/asm/kexec.h
@@ -132,4 +132,9 @@ extern int load_other_segments(struct kimage *image,
 
 #endif /* __ASSEMBLY__ */
 
+/* Current arm64 boot protocol requires 2MB alignment */
+#define CRASH_ALIGN                     SZ_2M
+
+#define CRASH_ADDR_LOW_MAX              arm64_dma_phys_limit
+#define CRASH_ADDR_HIGH_MAX             (PHYS_MASK + 1)
 #endif
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 36967b82c150..2e63954a2574 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -64,149 +64,25 @@ EXPORT_SYMBOL(memstart_addr);
  */
 phys_addr_t __ro_after_init arm64_dma_phys_limit;
 
-/* Current arm64 boot protocol requires 2MB alignment */
-#define CRASH_ALIGN			SZ_2M
-
-#define CRASH_ADDR_LOW_MAX		arm64_dma_phys_limit
-#define CRASH_ADDR_HIGH_MAX		(PHYS_MASK + 1)
-#define CRASH_HIGH_SEARCH_BASE		SZ_4G
-
-#define DEFAULT_CRASH_KERNEL_LOW_SIZE	(128UL << 20)
-
-static int __init reserve_crashkernel_low(unsigned long long low_size)
-{
-	unsigned long long low_base;
-
-	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
-	if (!low_base) {
-		pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
-		return -ENOMEM;
-	}
-
-	pr_info("crashkernel low memory reserved: 0x%08llx - 0x%08llx (%lld MB)\n",
-		low_base, low_base + low_size, low_size >> 20);
-
-	crashk_low_res.start = low_base;
-	crashk_low_res.end   = low_base + low_size - 1;
-	insert_resource(&iomem_resource, &crashk_low_res);
-
-	return 0;
-}
-
-/*
- * reserve_crashkernel() - reserves memory for crash kernel
- *
- * This function reserves memory area given in "crashkernel=" kernel command
- * line parameter. The memory reserved is used by dump capture kernel when
- * primary kernel is crashing.
- */
-static void __init reserve_crashkernel(void)
+static void __init arch_reserve_crashkernel(void)
 {
-	unsigned long long crash_low_size = 0, search_base = 0;
-	unsigned long long crash_max = CRASH_ADDR_LOW_MAX;
+	unsigned long long low_size = 0;
 	unsigned long long crash_base, crash_size;
 	char *cmdline = boot_command_line;
-	bool fixed_base = false;
 	bool high = false;
 	int ret;
 
 	if (!IS_ENABLED(CONFIG_KEXEC_CORE))
 		return;
 
-	/* crashkernel=X[@offset] */
 	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
-				&crash_size, &crash_base, NULL, NULL);
-	if (ret == -ENOENT) {
-		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
-		if (ret || !crash_size)
-			return;
-
-		/*
-		 * crashkernel=Y,low can be specified or not, but invalid value
-		 * is not allowed.
-		 */
-		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
-		if (ret == -ENOENT)
-			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
-		else if (ret)
-			return;
-
-		search_base = CRASH_HIGH_SEARCH_BASE;
-		crash_max = CRASH_ADDR_HIGH_MAX;
-		high = true;
-	} else if (ret || !crash_size) {
-		/* The specified value is invalid */
+				&crash_size, &crash_base,
+				&low_size, &high);
+	if (ret)
 		return;
-	}
-
-	crash_size = PAGE_ALIGN(crash_size);
-
-	/* User specifies base address explicitly. */
-	if (crash_base) {
-		fixed_base = true;
-		search_base = crash_base;
-		crash_max = crash_base + crash_size;
-	}
-
-retry:
-	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
-					       search_base, crash_max);
-	if (!crash_base) {
-		/*
-		 * For crashkernel=size[KMG]@offset[KMG], print out failure
-		 * message if can't reserve the specified region.
-		 */
-		if (fixed_base) {
-			pr_warn("crashkernel reservation failed - memory is in use.\n");
-			return;
-		}
-
-		/*
-		 * For crashkernel=size[KMG], if the first attempt was for
-		 * low memory, fall back to high memory, the minimum required
-		 * low memory will be reserved later.
-		 */
-		if (!high && crash_max == CRASH_ADDR_LOW_MAX) {
-			crash_max = CRASH_ADDR_HIGH_MAX;
-			search_base = CRASH_ADDR_LOW_MAX;
-			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
-			goto retry;
-		}
-
-		/*
-		 * For crashkernel=size[KMG],high, if the first attempt was
-		 * for high memory, fall back to low memory.
-		 */
-		if (high && crash_max == CRASH_ADDR_HIGH_MAX) {
-			crash_max = CRASH_ADDR_LOW_MAX;
-			search_base = 0;
-			goto retry;
-		}
-		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
-			crash_size);
-		return;
-	}
-
-	if ((crash_base >= CRASH_ADDR_LOW_MAX) && crash_low_size &&
-	     reserve_crashkernel_low(crash_low_size)) {
-		memblock_phys_free(crash_base, crash_size);
-		return;
-	}
-
-	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
-		crash_base, crash_base + crash_size, crash_size >> 20);
-
-	/*
-	 * The crashkernel memory will be removed from the kernel linear
-	 * map. Inform kmemleak so that it won't try to access it.
-	 */
-	kmemleak_ignore_phys(crash_base);
-	if (crashk_low_res.end)
-		kmemleak_ignore_phys(crashk_low_res.start);
 
-	crashk_res.start = crash_base;
-	crashk_res.end = crash_base + crash_size - 1;
-	insert_resource(&iomem_resource, &crashk_res);
+	reserve_crashkernel_generic(cmdline, crash_size, crash_base,
+				    low_size, high);
 }
 
 /*
@@ -454,7 +330,7 @@ void __init bootmem_init(void)
 	 * request_standard_resources() depends on crashkernel's memory being
 	 * reserved, so do it here.
 	 */
-	reserve_crashkernel();
+	arch_reserve_crashkernel();
 
 	memblock_dump_all();
 }
-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:11:26 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:26 +0800
Subject: [PATCH 7/8] x86: kdump: use generic interface to simplify crashkernel reservation code
In-Reply-To: <20230827101128.70931-1-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
Message-ID: <20230827101128.70931-8-bhe@redhat.com>

With the help of newly changed function parse_crashkernel() and
generic reserve_crashkernel_generic(), crashkernel reservation can be
simplified by steps:

1) Provide CRASH_ALIGN, CRASH_ADDR_LOW_MAX, CRASH_ADDR_HIGH_MAX and
   DEFAULT_CRASH_KERNEL_LOW_SIZE in <asm/kexec.h>;

2) Add arch_reserve_crashkernel() to call parse_crashkernel() and
   reserve_crashkernel_generic(), and do the ARCH specific work if
   needed.

3) Add ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION Kconfig in
   arch/x86/Kconfig.

When adding DEFAULT_CRASH_KERNEL_LOW_SIZE, add crash_low_size_default()
to calculate crashkernel low memory because x86_64 has special
requirement.

The old reserve_crashkernel_low() and reserve_crashkernel() can be
removed.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/Kconfig             |   3 +
 arch/x86/include/asm/kexec.h |  32 ++++++++
 arch/x86/kernel/setup.c      | 144 ++++-------------------------------
 3 files changed, 51 insertions(+), 128 deletions(-)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index e36261b4ea14..31515b3ef55b 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2073,6 +2073,9 @@ config KEXEC_FILE
 config ARCH_HAS_KEXEC_PURGATORY
 	def_bool KEXEC_FILE
 
+config ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+	def_bool CRASH_CORE
+
 config KEXEC_SIG
 	bool "Verify kernel signature during kexec_file_load() syscall"
 	depends on KEXEC_FILE
diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index 5b77bbc28f96..84a7d1f6f153 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -66,6 +66,37 @@ struct kimage;
 # define KEXEC_ARCH KEXEC_ARCH_X86_64
 #endif
 
+/*
+ * --------- Crashkernel reservation ------------------------------
+ */
+
+/* 16M alignment for crash kernel regions */
+#define CRASH_ALIGN             SZ_16M
+
+/*
+ * Keep the crash kernel below this limit.
+ *
+ * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
+ * due to mapping restrictions.
+ *
+ * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
+ * the upper limit of system RAM in 4-level paging mode. Since the kdump
+ * jump could be from 5-level paging to 4-level paging, the jump will fail if
+ * the kernel is put above 64 TB, and during the 1st kernel bootup there's
+ * no good way to detect the paging mode of the target kernel which will be
+ * loaded for dumping.
+ */
+
+#ifdef CONFIG_X86_32
+# define CRASH_ADDR_LOW_MAX     SZ_512M
+# define CRASH_ADDR_HIGH_MAX    SZ_512M
+#else
+# define CRASH_ADDR_LOW_MAX     SZ_4G
+# define CRASH_ADDR_HIGH_MAX    SZ_64T
+#endif
+
+# define DEFAULT_CRASH_KERNEL_LOW_SIZE crash_low_size_default()
+
 /*
  * This function is responsible for capturing register states if coming
  * via panic otherwise just fix up the ss and sp if coming via kernel
@@ -209,6 +240,7 @@ typedef void crash_vmclear_fn(void);
 extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss;
 extern void kdump_nmi_shootdown_cpus(void);
 
+extern unsigned long crash_low_size_default(void);
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_X86_KEXEC_H */
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 382c66d2cf71..559a5c4141db 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -474,152 +474,40 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 /*
  * --------- Crashkernel reservation ------------------------------
  */
-
-/* 16M alignment for crash kernel regions */
-#define CRASH_ALIGN		SZ_16M
-
-/*
- * Keep the crash kernel below this limit.
- *
- * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
- * due to mapping restrictions.
- *
- * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
- * the upper limit of system RAM in 4-level paging mode. Since the kdump
- * jump could be from 5-level paging to 4-level paging, the jump will fail if
- * the kernel is put above 64 TB, and during the 1st kernel bootup there's
- * no good way to detect the paging mode of the target kernel which will be
- * loaded for dumping.
- */
-#ifdef CONFIG_X86_32
-# define CRASH_ADDR_LOW_MAX	SZ_512M
-# define CRASH_ADDR_HIGH_MAX	SZ_512M
-#else
-# define CRASH_ADDR_LOW_MAX	SZ_4G
-# define CRASH_ADDR_HIGH_MAX	SZ_64T
-#endif
-
-static int __init reserve_crashkernel_low(void)
+unsigned long crash_low_size_default(void)
 {
 #ifdef CONFIG_X86_64
-	unsigned long long base, low_base = 0, low_size = 0;
-	unsigned long low_mem_limit;
-	int ret;
-
-	low_mem_limit = min(memblock_phys_mem_size(), CRASH_ADDR_LOW_MAX);
-
-	/* crashkernel=Y,low */
-	ret = parse_crashkernel_low(boot_command_line, low_mem_limit, &low_size, &base);
-	if (ret) {
-		/*
-		 * two parts from kernel/dma/swiotlb.c:
-		 * -swiotlb size: user-specified with swiotlb= or default.
-		 *
-		 * -swiotlb overflow buffer: now hardcoded to 32k. We round it
-		 * to 8M for other buffers that may need to stay low too. Also
-		 * make sure we allocate enough extra low memory so that we
-		 * don't run out of DMA buffers for 32-bit devices.
-		 */
-		low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);
-	} else {
-		/* passed with crashkernel=0,low ? */
-		if (!low_size)
-			return 0;
-	}
-
-	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
-	if (!low_base) {
-		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
-		       (unsigned long)(low_size >> 20));
-		return -ENOMEM;
-	}
-
-	pr_info("Reserving %ldMB of low memory at %ldMB for crashkernel (low RAM limit: %ldMB)\n",
-		(unsigned long)(low_size >> 20),
-		(unsigned long)(low_base >> 20),
-		(unsigned long)(low_mem_limit >> 20));
-
-	crashk_low_res.start = low_base;
-	crashk_low_res.end   = low_base + low_size - 1;
-	insert_resource(&iomem_resource, &crashk_low_res);
-#endif
+	return max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);
+#else
 	return 0;
+#endif
 }
 
-static void __init reserve_crashkernel(void)
+static void __init arch_reserve_crashkernel(void)
 {
-	unsigned long long crash_size, crash_base, total_mem;
+	unsigned long long crash_base, crash_size, low_size = 0;
+	char *cmdline = boot_command_line;
 	bool high = false;
 	int ret;
 
 	if (!IS_ENABLED(CONFIG_KEXEC_CORE))
 		return;
 
-	total_mem = memblock_phys_mem_size();
-
-	/* crashkernel=XM */
-	ret = parse_crashkernel(boot_command_line, total_mem,
-				&crash_size, &crash_base, NULL, NULL);
-	if (ret != 0 || crash_size <= 0) {
-		/* crashkernel=X,high */
-		ret = parse_crashkernel_high(boot_command_line, total_mem,
-					     &crash_size, &crash_base);
-		if (ret != 0 || crash_size <= 0)
-			return;
-		high = true;
-	}
+	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
+				&crash_size, &crash_base,
+				&low_size, &high);
+	if (ret)
+		return;
 
 	if (xen_pv_domain()) {
 		pr_info("Ignoring crashkernel for a Xen PV domain\n");
 		return;
 	}
 
-	/* 0 means: find the address automatically */
-	if (!crash_base) {
-		/*
-		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * crashkernel=x,high reserves memory over 4G, also allocates
-		 * 256M extra low memory for DMA buffers and swiotlb.
-		 * But the extra memory is not required for all machines.
-		 * So try low memory first and fall back to high memory
-		 * unless "crashkernel=size[KMG],high" is specified.
-		 */
-		if (!high)
-			crash_base = memblock_phys_alloc_range(crash_size,
-						CRASH_ALIGN, CRASH_ALIGN,
-						CRASH_ADDR_LOW_MAX);
-		if (!crash_base)
-			crash_base = memblock_phys_alloc_range(crash_size,
-						CRASH_ALIGN, CRASH_ALIGN,
-						CRASH_ADDR_HIGH_MAX);
-		if (!crash_base) {
-			pr_info("crashkernel reservation failed - No suitable area found.\n");
-			return;
-		}
-	} else {
-		unsigned long long start;
-
-		start = memblock_phys_alloc_range(crash_size, SZ_1M, crash_base,
-						  crash_base + crash_size);
-		if (start != crash_base) {
-			pr_info("crashkernel reservation failed - memory is in use.\n");
-			return;
-		}
-	}
-
-	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
-		memblock_phys_free(crash_base, crash_size);
-		return;
-	}
-
-	pr_info("Reserving %ldMB of memory at %ldMB for crashkernel (System RAM: %ldMB)\n",
-		(unsigned long)(crash_size >> 20),
-		(unsigned long)(crash_base >> 20),
-		(unsigned long)(total_mem >> 20));
+	reserve_crashkernel_generic(cmdline, crash_size, crash_base,
+				    low_size, high);
 
-	crashk_res.start = crash_base;
-	crashk_res.end   = crash_base + crash_size - 1;
-	insert_resource(&iomem_resource, &crashk_res);
+	return;
 }
 
 static struct resource standard_io_resources[] = {
@@ -1231,7 +1119,7 @@ void __init setup_arch(char **cmdline_p)
 	 * Reserve memory for crash kernel after SRAT is parsed so that it
 	 * won't consume hotpluggable memory.
 	 */
-	reserve_crashkernel();
+	arch_reserve_crashkernel();
 
 	memblock_find_dma_reserve();
 
-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:11:27 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:11:27 +0800
Subject: [PATCH 8/8] crash_core.c: remove unneeded functions
In-Reply-To: <20230827101128.70931-1-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
Message-ID: <20230827101128.70931-9-bhe@redhat.com>

So far, nobody calls functions parse_crashkernel_high() and
parse_crashkernel_high(), remove both of them.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |  4 ----
 kernel/crash_core.c        | 18 ------------------
 2 files changed, 22 deletions(-)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 2f732493e922..0c512a80a768 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -80,10 +80,6 @@ void final_note(Elf_Word *buf);
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base,
 		unsigned long long *low_size, bool *high);
-int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
-		unsigned long long *crash_size, unsigned long long *crash_base);
-int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
-		unsigned long long *crash_size, unsigned long long *crash_base);
 
 #ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
 #ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 6bc00cc390b5..61a8ea3b23a2 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -323,24 +323,6 @@ int __init parse_crashkernel(char *cmdline,
 	return 0;
 }
 
-int __init parse_crashkernel_high(char *cmdline,
-			     unsigned long long system_ram,
-			     unsigned long long *crash_size,
-			     unsigned long long *crash_base)
-{
-	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				suffix_tbl[SUFFIX_HIGH]);
-}
-
-int __init parse_crashkernel_low(char *cmdline,
-			     unsigned long long system_ram,
-			     unsigned long long *crash_size,
-			     unsigned long long *crash_base)
-{
-	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				suffix_tbl[SUFFIX_LOW]);
-}
-
 /*
  * Add a dummy early_param handler to mark crashkernel= as a known command line
  * parameter and suppress incorrect warnings in init/main.c.
-- 
2.41.0



From bhe at redhat.com  Sun Aug 27 03:43:55 2023
From: bhe at redhat.com (Baoquan He)
Date: Sun, 27 Aug 2023 18:43:55 +0800
Subject: [RFC PATCH 0/4] kdump: add generic functions to simplify
 crashkernel crashkernel in architecture
In-Reply-To: <20230710192018.031c9912@rotkaeppchen>
References: <20230619055951.45620-1-bhe@redhat.com>
 <ZKjG2R8RhWmJex53@darkstar.users.ipa.redhat.com>
 <20230710192018.031c9912@rotkaeppchen>
Message-ID: <ZOso66U0DIR9rfMW@MiWiFi-R3L-srv>

Hi Dave, Philipp

On 07/10/23 at 07:20pm, Philipp Rudo wrote:
> Hi Baoquan,
> Hi Dave,
> 
> On Sat, 8 Jul 2023 10:15:53 +0800
> Dave Young <dyoung at redhat.com> wrote:
> 
> > On 06/19/23 at 01:59pm, Baoquan He wrote:
> > > In the current arm64, crashkernel=,high support has been finished after
> > > several rounds of posting and careful reviewing. The code in arm64 which
> > > parses crashkernel kernel parameters firstly, then reserve memory can be
> > > a good example for other ARCH to refer to.
> > > 
> > > Whereas in x86_64, the code mixing crashkernel parameter parsing and
> > > memory reserving is twisted, and looks messy. Refactoring the code to
> > > make it more readable maintainable is necessary.
> > > 
> > > Here, try to abstract the crashkernel parameter parsing code into a
> > > generic function parse_crashkernel_generic(), and the crashkernel memory
> > > reserving code into a generic function reserve_crashkernel_generic().
> > > Then, in ARCH which crashkernel=,high support is needed, a simple
> > > arch_reserve_crashkernel() can be added to call above two generic
> > > functions. This can remove the duplicated implmentation code in each
> > > ARCH, like arm64, x86_64.  
> > 
> > Hi Baoquan, the parse_crashkernel_common and parse_crashkernel_generic
> > are confusion to me.  Thanks for the effort though.
> 
> I agree, having both parse_crashkernel_common and
> parse_crashkernel_generic is pretty confusing.

Sorry for being late to respond, and thank both a lot for reviewing
and valuable suggestions on this patchset, and

I have made a new patchset to address your concern that the old 
parse_crashkernel_common() and parse_crashkernel_generic() are
confusing. Please see the new formal post which can be accessed from
below link. Within the new post, I integrated all kinds of crashkernel
parsing into parse_crashkernel().

https://lore.kernel.org/all/20230827101128.70931-1-bhe at redhat.com/T/#u 
[PATCH 0/8] kdump: use generic functions to simplify crashkernel reservation in architectures

As for introducing a new structure struct crashkernel and enum
crashkernel_type, after careful thinking, I think it may not be
appropriate in this place. The reason is that even though we have
several different grammers of crashkernel= specification, in fact
there's one being able to set in kernel parameters. Crashkernel=,high
support is special because it needs too, while we can ignore the
crashernel=,low since crashkernel=,low is not an independent one.
So a structure wrapper isn't helping much and will cause many code
change churn in all architectures where crashkernel=,high is not
supported. Besides, we have fallback mechanism for crashkernel=xM and
crashkernel=xM,high. So the switch case handling Phlipp suggested
doesn't fit in this case.

As you can see, with the v1 patchset, the code change is few and not
hard to understand.

> 
> > I'm not sure if it will be easy or not, but ideally I think the parse
> > function can be arch independent, something like a general funtion
> > parse_crashkernel() which can return the whole necessary infomation of
> > crashkenrel for arch code to use, for example return like
> > below pseudo stucture(just a concept, may need to think more):
> > 
> > structure crashkernel_range {
> > 	size,
> > 	range,
> > 	struct list_head list;
> > }
> > 
> > structure crashkernel{
> >   structure crashkernel_range *range_list;
> >   union {
> >   	offset,
> >   	low_high
> >   }
> > }
> > 
> > So the arch code can just get the data of crashkernel and then check
> > about the details, if it does not support low and high reservation then
> > it can just ignore the option.
> > 
> > Thoughts?
> 
> Sounds reasonable. The only thing I would make different is for the
> parser to take the systems ram into account and simply return the size
> that needs to be allocated in case multiple ranges are given.
> 
> I've played around a little on how this might look like (probably
> wasting way too much time on it) and came up with the patch below. With
> that patch parse_crashkernel_{common,high,low} and friends could be
> removed once all architectures are ported to the new parse_crashkernel
> function.
> 
> Please note that I never tested the patch. So it probably doesn't even
> compile. Nevertheless I hope it helps to get an idea on what I think
> should work :-)
> 
> Thanks
> Philipp
> 
> ----
> 
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index fb904cc57ab1..fd5d01872c53 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -66,22 +66,12 @@ phys_addr_t __ro_after_init arm64_dma_phys_limit;
>  
>  static void __init arch_reserve_crashkernel(void)
>  {
> -	unsigned long long low_size = 0;
> -	unsigned long long crash_base, crash_size;
>  	char *cmdline = boot_command_line;
> -	bool high = false;
> -	int ret;
>  
>  	if (!IS_ENABLED(CONFIG_KEXEC_CORE))
>  		return;
>  
> -	ret = parse_crashkernel_generic(cmdline, &crash_size, &crash_base,
> -					&low_size, &high);
> -	if (ret)
> -		return;
> -
> -	reserve_crashkernel_generic(cmdline, crash_size, crash_base,
> -				    low_size, high);
> +	reserve_crashkernel_generic(cmdline);
>  }
>  
>  /*
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index b26221022283..4a78bf8ad494 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -486,28 +486,17 @@ unsigned long crash_low_size_default(void)
>  
>  static void __init arch_reserve_crashkernel(void)
>  {
> -	unsigned long long crash_base, crash_size, low_size = 0;
>  	char *cmdline = boot_command_line;
> -	bool high = false;
> -	int ret;
>  
>  	if (!IS_ENABLED(CONFIG_KEXEC_CORE))
>  		return;
>  
> -	ret = parse_crashkernel_generic(cmdline, &crash_size, &crash_base,
> -					&low_size, &high);
> -	if (ret)
> -		return;
> -
>  	if (xen_pv_domain()) {
>  		pr_info("Ignoring crashkernel for a Xen PV domain\n");
>  		return;
>  	}
>  
> -	reserve_crashkernel_generic(cmdline, crash_size, crash_base,
> -				    low_size, high);
> -
> -	return;
> +	reserve_crashkernel_generic(cmdline);
>  }
>  
>  static struct resource standard_io_resources[] = {
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 1b12868cad1b..a1ebd6c47467 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -84,35 +84,25 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
>  int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
>  		unsigned long long *crash_size, unsigned long long *crash_base);
>  
> -#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> -int __init parse_crashkernel_generic(char *cmdline,
> -				 unsigned long long *crash_size,
> -				 unsigned long long *crash_base,
> -				 unsigned long long *low_size,
> -				 bool *high);
> -
> -void __init reserve_crashkernel_generic(char *cmdline,
> -		unsigned long long crash_size,
> -		unsigned long long crash_base,
> -		unsigned long long crash_low_size,
> -		bool high);
> -#else
> -
> -static inline int __init parse_crashkernel_generic(char *cmdline,
> -		unsigned long long *crash_size,
> -		unsigned long long *crash_base,
> -		unsigned long long *low_size,
> -		bool *high)
> -{
> -	return 0;
> +enum crashkernel_type {
> +	CRASHKERNEL_NORMAL,
> +	CRASHKERNEL_FIXED_OFFSET,
> +	CRASHKERNEL_HIGH,
> +	CRASHKERNEL_LOW
>  }
>  
> -static inline void __init reserve_crashkernel_generic(char *cmdline,
> -		unsigned long long crash_size,
> -		unsigned long long crash_base,
> -		unsigned long long crash_low_size,
> -		bool high)
> -{}
> +struct crashkernl {
> +	enum crashkernel_type type;
> +	unsigned long long size;
> +	unsigned long long offet;
> +};
> +
> +int __init parse_crashkernel(char *cmdline, struct crashkernel *ck);
> +
> +#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> +void __init reserve_crashkernel_generic(char *cmdline);
> +#else
> +void __init reserve_crashkernel_generic(char *cmdline) {}
>  #endif
>  
>  #endif /* LINUX_CRASH_CORE_H */
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index b82dc8c970de..c04a8675446b 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -203,8 +203,7 @@ static int __init parse_crashkernel_suffix(char *cmdline,
>  }
>  
>  static __init char *get_last_crashkernel(char *cmdline,
> -			     const char *name,
> -			     const char *suffix)
> +					 const char *name)
>  {
>  	char *p = cmdline, *ck_cmdline = NULL;
>  
> @@ -217,23 +216,6 @@ static __init char *get_last_crashkernel(char *cmdline,
>  		if (!end_p)
>  			end_p = p + strlen(p);
>  
> -		if (!suffix) {
> -			int i;
> -
> -			/* skip the one with any known suffix */
> -			for (i = 0; suffix_tbl[i]; i++) {
> -				q = end_p - strlen(suffix_tbl[i]);
> -				if (!strncmp(q, suffix_tbl[i],
> -					     strlen(suffix_tbl[i])))
> -					goto next;
> -			}
> -			ck_cmdline = p;
> -		} else {
> -			q = end_p - strlen(suffix);
> -			if (!strncmp(q, suffix, strlen(suffix)))
> -				ck_cmdline = p;
> -		}
> -next:
>  		p = strstr(p+1, name);
>  	}
>  
> @@ -314,42 +296,144 @@ static int __init parse_crashkernel_dummy(char *arg)
>  early_param("crashkernel", parse_crashkernel_dummy);
>  
>  
> -#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> -int __init parse_crashkernel_generic(char *cmdline,
> -			     unsigned long long *crash_size,
> -			     unsigned long long *crash_base,
> -			     unsigned long long *low_size,
> -			     bool *high)
> +/*
> + * This function parses command lines in the format
> + *
> + *   crashkernel=[start1-end1:]size1[,...][@offset|,high|,low]
> + *
> + * The function returns 0 on success and -EINVAL on failure.
> + */
> +static int __init _parse_crashkernel(char *cmdline, struct crashkernel *ck)
>  {
> -	int ret;
> +	unsigned long long start, end, size, offset;
> +	unsigned long long system_ram;
> +	char *next, *cur = cmdline;
>  
> -	/* crashkernel=X[@offset] */
> -	ret = parse_crashkernel_common(cmdline, memblock_phys_mem_size(),
> -				crash_size, crash_base);
> -	if (ret == -ENOENT) {
> -		ret = parse_crashkernel_high(cmdline, 0, crash_size, crash_base);
> -		if (ret || !*crash_size)
> -			return -1;
> -
> -		/*
> -		 * crashkernel=Y,low can be specified or not, but invalid value
> -		 * is not allowed.
> -		 */
> -		ret = parse_crashkernel_low(cmdline, 0, low_size, crash_base);
> -		if (ret == -ENOENT)
> -			*low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> -		else if (ret)
> -			return -1;
> -
> -		*high = true;
> -	} else if (ret || !*crash_size) {
> -		/* The specified value is invalid */
> -		return -1;
> +	/*
> +	 * Firmware sometimes reserves some memory regions for its own use,
> +	 * so the system memory size is less than the actual physical memory
> +	 * size. Work around this by rounding up the total size to 128M,
> +	 * which is enough for most test cases.
> +	 */
> +	system_ram = memblock_phys_mem_size()
> +	system_ram = roundup(system_mem, SZ_128M);
> +
> +	start = 0;
> +	end = ULLONG_MAX;
> +	size = memparse(cur, &next);
> +	if (cur == next) {
> +		pr_warn("crashkernel: Memory value expected\n");
> +		return -EINVAL;
> +	}
> +	cur = next;
> +	ck->type=CRASHKERNEL_NORMAL;
> +
> +	/* case crashkerne=size[@offset|,high|,low] */
> +	if (!strchr(cmdline, '-')) {
> +		ck->size = size;
> +	}
> +
> +	while (*cur != ' ' && *cur != '\0') {
> +		switch (*cur) {
> +		case '@':
> +			offset = memparse(++cur, &next);
> +			if (*next != ' ' && *next != '\0') {
> +				pr_warn("crashkernel: Offset must be at the end\n");
> +				return -EINVAL;
> +			}
> +			/* allow corner cases with @0 */
> +			ck->type=CRASHKERNEL_FIXED_OFFSET;
> +			ck->offset = offset;
> +			break;
> +
> +		case '-':
> +			start = size;
> +			end = memparse(++cur, &next);
> +			/* allow <start>-:<size> */
> +			if (cur == next) {
> +				end = system_ram;
> +				next++;
> +			}
> +
> +			if (*next != ':') {
> +				pr_warn("crashkernel: ':' expected\n");
> +				return -EINVAL;
> +			}
> +
> +			cur = next + 1;
> +			size = memparse(cur, &next);
> +			if (cur == next) {
> +				pr_warn("crashkernel: No size provided\n");
> +				return -EINVAL;
> +			}
> +
> +			if (end <= start) {
> +				pr_warn("crashkernel: end <= start\n");
> +				return -EINVAL;
> +			}
> +
> +			if (start <= system_ram && end > system_ram)
> +				ck->size = size;
> +
> +
> +			cur = next + 1;
> +			break;
> +
> +		case ',':
> +			cur++;
> +
> +			/* check for ,high, ,low */
> +			if (strncmp(cur, "high", strlen("high"))) {
> +				ck->type=CRASHKERNEL_HIGH;
> +				cur+=strlen("high");
> +				if (*cur != ' ' || *cur != '\0') {
> +					pr_warn("crashkernel: ,high must be at the end\n");
> +					return -EINVAL;
> +				}
> +				break;
> +			} else if (strncmp(cur, "low". strlen("low"))) {
> +				ck->type=CRASHKERNEL_LOW;
> +				cur+=strlen("low");
> +				if (*cur != ' ' || *cur != '\0') {
> +					pr_warn("crashkernel: ,high must be at the end\n");
> +					return -EINVAL;
> +				}
> +				break;
> +			}
> +
> +			/* start with next entry */
> +			size = memparse(cur, &next);
> +			if (cur == next) {
> +				pr_warn("crashkernel: Memory value expected\n");
> +				return -EINVAL;
> +			}
> +			cur = next;
> +			break;
> +
> +		default:
> +			pr_warn("crashkernel: Invalid character '%c' provided\n", *cur);
> +			return -EINVAL;
> +		}
>  	}
>  
>  	return 0;
>  }
>  
> +#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> +int __init parse_crashkernel(char *cmdline, struct crashkernel *ck)
> +{
> +	const char *name="crashkernel=";
> +	char *ck_cmdline;
> +
> +	BUG_ON(!ck);
> +
> +	ck_cmdline = get_last_crashkernel(cmdline, name);
> +	if (!ck_cmdline)
> +		return -ENOENT;
> +	ck_cmdline += strlen(name);
> +	return _parse_crashkernel(ck_cmdline, ck);
> +}
> +
>  static int __init reserve_crashkernel_low(unsigned long long low_size)
>  {
>  #ifdef CONFIG_64BIT
> @@ -371,70 +455,57 @@ static int __init reserve_crashkernel_low(unsigned long long low_size)
>  	return 0;
>  }
>  
> -void __init reserve_crashkernel_generic(char *cmdline,
> -			     unsigned long long crash_size,
> -			     unsigned long long crash_base,
> -			     unsigned long long crash_low_size,
> -			     bool high)
> +void __init reserve_crashkernel_generic(char *cmdline)
>  {
> -	unsigned long long search_end = CRASH_ADDR_LOW_MAX, search_base = 0;
> -	bool fixed_base = false;
> -
> -	/* User specifies base address explicitly. */
> -	if (crash_base) {
> -		fixed_base = true;
> -		search_base = crash_base;
> -		search_end = crash_base + crash_size;
> -	}
> +	struct ck = { 0 };
>  
> -	if (high) {
> -		search_base = CRASH_ADDR_LOW_MAX;
> -		search_end = CRASH_ADDR_HIGH_MAX;
> -	}
> +	parse_crashkernel(cmdline, &ck);
>  
> -retry:
> -	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
> -					       search_base, search_end);
> -	if (!crash_base) {
> -		/*
> -		 * For crashkernel=size[KMG]@offset[KMG], print out failure
> -		 * message if can't reserve the specified region.
> -		 */
> -		if (fixed_base) {
> -			pr_warn("crashkernel reservation failed - memory is in use.\n");
> -			return;
> -		}
> +	if (!ck.size)
> +		return;
>  
> -		/*
> -		 * For crashkernel=size[KMG], if the first attempt was for
> -		 * low memory, fall back to high memory, the minimum required
> -		 * low memory will be reserved later.
> -		 */
> -		if (!high && search_end == CRASH_ADDR_LOW_MAX) {
> -			search_end = CRASH_ADDR_HIGH_MAX;
> -			search_base = CRASH_ADDR_LOW_MAX;
> -			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> -			goto retry;
> +	switch (ck.type) {
> +	CRASHKERNEL_FIXED_OFFSET:
> +		crash_base = memblock_phys_alloc_range(ck.size, CRASH_ALIGN,
> +						       ck.offset,
> +						       ck.offset + ck.size);
> +		break;
> +
> +	CRASHKERNEL_NORMAL:
> +		if (DEFAULT_CRASH_KERNEL_LOW_SIZE) {
> +			/* Simply continue in case we fail to allocate the low
> +			 * memory */
> +			if (!reserve_crashkernel_low(DEFAULT_CRASH_KERNEL_LOW_SIZE))
> +				ck.size -= DEFAULT_CRASH_KERNEL_LOW_SIZE;
>  		}
>  
> -		/*
> -		 * For crashkernel=size[KMG],high, if the first attempt was
> -		 * for high memory, fall back to low memory.
> -		 */
> -		if (high && search_end == CRASH_ADDR_HIGH_MAX) {
> -			search_end = CRASH_ADDR_LOW_MAX;
> -			search_base = 0;
> -			goto retry;
> -		}
> -		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
> -			crash_size);
> +		fallthrough;
> +	CRASHKERNEL_HIGH:
> +		crash_base = memblock_phys_alloc_range(ck.size, CRASH_ALIGN,
> +						       CRASH_ADDR_LOW_MAX,
> +						       CRASH_ADDR_HIGH_MAX);
> +		if (crash_base)
> +			break;
> +
> +		fallthrough;
> +	CRASHKERNEL_LOW:
> +		crash_base = memblock_phys_alloc_range(ck.size, CRASH_ALIGN, 0,
> +						       CRASH_ADDR_LOW_MAX);
> +		break;
> +
> +	default:
> +		pr_warn("Invalid crashkernel type %i\n", ck.type);
>  		return;
>  	}
>  
> -	if ((crash_base > CRASH_ADDR_LOW_MAX) &&
> -	     crash_low_size && reserve_crashkernel_low(crash_low_size)) {
> -		memblock_phys_free(crash_base, crash_size);
> -		return;
> +	if (!crash_base) {
> +		pr_warn("could not allocate crashkernel (size:0x%llx)\n",
> +			ck.size);
> +		if (crashk_low_res.end) {
> +			memblock_phys_free(crashk_low_res.start,
> +					   crashk_low_res.end - crashk_low_res.start + 1);
> +		}
> +		return
>  	}
>  
>  	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
> @@ -449,7 +520,7 @@ void __init reserve_crashkernel_generic(char *cmdline,
>  		kmemleak_ignore_phys(crashk_low_res.start);
>  
>  	crashk_res.start = crash_base;
> -	crashk_res.end = crash_base + crash_size - 1;
> +	crashk_res.end = crash_base + ck.size - 1;
>  	insert_resource(&iomem_resource, &crashk_res);
>  }
>  #endif
> 



From lkp at intel.com  Sun Aug 27 06:53:02 2023
From: lkp at intel.com (kernel test robot)
Date: Sun, 27 Aug 2023 21:53:02 +0800
Subject: [PATCH 5/8] crash_core: add generic function to do reservation
In-Reply-To: <20230827101128.70931-6-bhe@redhat.com>
References: <20230827101128.70931-6-bhe@redhat.com>
Message-ID: <202308272150.p3kRkMoF-lkp@intel.com>

Hi Baoquan,

kernel test robot noticed the following build errors:

[auto build test ERROR on tip/x86/core]
[also build test ERROR on powerpc/next powerpc/fixes linus/master v6.5-rc7]
[cannot apply to arm64/for-next/core next-20230825]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Baoquan-He/crash_core-c-remove-unnecessary-parameter-of-function/20230827-181555
base:   tip/x86/core
patch link:    https://lore.kernel.org/r/20230827101128.70931-6-bhe%40redhat.com
patch subject: [PATCH 5/8] crash_core: add generic function to do reservation
config: csky-defconfig (https://download.01.org/0day-ci/archive/20230827/202308272150.p3kRkMoF-lkp at intel.com/config)
compiler: csky-linux-gcc (GCC) 13.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230827/202308272150.p3kRkMoF-lkp at intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp at intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202308272150.p3kRkMoF-lkp at intel.com/

All errors (new ones prefixed by >>):

     537 | void __weak arch_crash_save_vmcoreinfo(void)
         |             ^~~~~~~~~~~~~~~~~~~~~~~~~~
   kernel/crash_core.c:540:20: warning: no previous prototype for 'paddr_vmcoreinfo_note' [-Wmissing-prototypes]
     540 | phys_addr_t __weak paddr_vmcoreinfo_note(void)
         |                    ^~~~~~~~~~~~~~~~~~~~~
   kernel/crash_core.c: In function 'crash_save_vmcoreinfo_init':
   kernel/crash_core.c:554:45: error: 'VMCOREINFO_NOTE_SIZE' undeclared (first use in this function)
     554 |         vmcoreinfo_note = alloc_pages_exact(VMCOREINFO_NOTE_SIZE,
         |                                             ^~~~~~~~~~~~~~~~~~~~
   kernel/crash_core.c:563:9: error: implicit declaration of function 'VMCOREINFO_OSRELEASE' [-Werror=implicit-function-declaration]
     563 |         VMCOREINFO_OSRELEASE(init_uts_ns.name.release);
         |         ^~~~~~~~~~~~~~~~~~~~
   kernel/crash_core.c:564:9: error: implicit declaration of function 'VMCOREINFO_BUILD_ID' [-Werror=implicit-function-declaration]
     564 |         VMCOREINFO_BUILD_ID();
         |         ^~~~~~~~~~~~~~~~~~~
   kernel/crash_core.c:565:9: error: implicit declaration of function 'VMCOREINFO_PAGESIZE' [-Werror=implicit-function-declaration]
     565 |         VMCOREINFO_PAGESIZE(PAGE_SIZE);
         |         ^~~~~~~~~~~~~~~~~~~
   kernel/crash_core.c:567:9: error: implicit declaration of function 'VMCOREINFO_SYMBOL' [-Werror=implicit-function-declaration]
     567 |         VMCOREINFO_SYMBOL(init_uts_ns);
         |         ^~~~~~~~~~~~~~~~~
   kernel/crash_core.c:568:9: error: implicit declaration of function 'VMCOREINFO_OFFSET' [-Werror=implicit-function-declaration]
     568 |         VMCOREINFO_OFFSET(uts_namespace, name);
         |         ^~~~~~~~~~~~~~~~~
   kernel/crash_core.c:568:27: error: 'uts_namespace' undeclared (first use in this function)
     568 |         VMCOREINFO_OFFSET(uts_namespace, name);
         |                           ^~~~~~~~~~~~~
   kernel/crash_core.c:568:42: error: 'name' undeclared (first use in this function)
     568 |         VMCOREINFO_OFFSET(uts_namespace, name);
         |                                          ^~~~
   kernel/crash_core.c:571:9: error: implicit declaration of function 'VMCOREINFO_SYMBOL_ARRAY' [-Werror=implicit-function-declaration]
     571 |         VMCOREINFO_SYMBOL_ARRAY(swapper_pg_dir);
         |         ^~~~~~~~~~~~~~~~~~~~~~~
   kernel/crash_core.c:588:9: error: implicit declaration of function 'VMCOREINFO_STRUCT_SIZE' [-Werror=implicit-function-declaration]
     588 |         VMCOREINFO_STRUCT_SIZE(page);
         |         ^~~~~~~~~~~~~~~~~~~~~~
   kernel/crash_core.c:588:32: error: 'page' undeclared (first use in this function)
     588 |         VMCOREINFO_STRUCT_SIZE(page);
         |                                ^~~~
   kernel/crash_core.c:589:32: error: 'pglist_data' undeclared (first use in this function)
     589 |         VMCOREINFO_STRUCT_SIZE(pglist_data);
         |                                ^~~~~~~~~~~
   kernel/crash_core.c:590:32: error: 'zone' undeclared (first use in this function)
     590 |         VMCOREINFO_STRUCT_SIZE(zone);
         |                                ^~~~
   kernel/crash_core.c:591:32: error: 'free_area' undeclared (first use in this function)
     591 |         VMCOREINFO_STRUCT_SIZE(free_area);
         |                                ^~~~~~~~~
   kernel/crash_core.c:592:32: error: 'list_head' undeclared (first use in this function)
     592 |         VMCOREINFO_STRUCT_SIZE(list_head);
         |                                ^~~~~~~~~
   kernel/crash_core.c:593:9: error: implicit declaration of function 'VMCOREINFO_SIZE' [-Werror=implicit-function-declaration]
     593 |         VMCOREINFO_SIZE(nodemask_t);
         |         ^~~~~~~~~~~~~~~
   kernel/crash_core.c:593:25: error: expected expression before 'nodemask_t'
     593 |         VMCOREINFO_SIZE(nodemask_t);
         |                         ^~~~~~~~~~
   kernel/crash_core.c:594:33: error: 'flags' undeclared (first use in this function); did you mean 'fls'?
     594 |         VMCOREINFO_OFFSET(page, flags);
         |                                 ^~~~~
         |                                 fls
   kernel/crash_core.c:595:33: error: '_refcount' undeclared (first use in this function); did you mean 'seqcount'?
     595 |         VMCOREINFO_OFFSET(page, _refcount);
         |                                 ^~~~~~~~~
         |                                 seqcount
   kernel/crash_core.c:596:33: error: 'mapping' undeclared (first use in this function)
     596 |         VMCOREINFO_OFFSET(page, mapping);
         |                                 ^~~~~~~
   kernel/crash_core.c:597:33: error: 'lru' undeclared (first use in this function)
     597 |         VMCOREINFO_OFFSET(page, lru);
         |                                 ^~~
   kernel/crash_core.c:598:33: error: '_mapcount' undeclared (first use in this function); did you mean 'nmi_count'?
     598 |         VMCOREINFO_OFFSET(page, _mapcount);
         |                                 ^~~~~~~~~
         |                                 nmi_count
   kernel/crash_core.c:599:33: error: 'private' undeclared (first use in this function); did you mean 'PG_private'?
     599 |         VMCOREINFO_OFFSET(page, private);
         |                                 ^~~~~~~
         |                                 PG_private
   kernel/crash_core.c:600:27: error: 'folio' undeclared (first use in this function)
     600 |         VMCOREINFO_OFFSET(folio, _folio_dtor);
         |                           ^~~~~
   kernel/crash_core.c:600:34: error: '_folio_dtor' undeclared (first use in this function)
     600 |         VMCOREINFO_OFFSET(folio, _folio_dtor);
         |                                  ^~~~~~~~~~~
   kernel/crash_core.c:601:34: error: '_folio_order' undeclared (first use in this function); did you mean 'folio_order'?
     601 |         VMCOREINFO_OFFSET(folio, _folio_order);
         |                                  ^~~~~~~~~~~~
         |                                  folio_order
   kernel/crash_core.c:602:33: error: 'compound_head' undeclared (first use in this function); did you mean '_compound_head'?
     602 |         VMCOREINFO_OFFSET(page, compound_head);
         |                                 ^~~~~~~~~~~~~
         |                                 _compound_head
   kernel/crash_core.c:603:40: error: 'node_zones' undeclared (first use in this function); did you mean 'node_zonelist'?
     603 |         VMCOREINFO_OFFSET(pglist_data, node_zones);
         |                                        ^~~~~~~~~~
         |                                        node_zonelist
   kernel/crash_core.c:604:40: error: 'nr_zones' undeclared (first use in this function)
     604 |         VMCOREINFO_OFFSET(pglist_data, nr_zones);
         |                                        ^~~~~~~~
>> kernel/crash_core.c:606:40: error: 'node_mem_map' undeclared (first use in this function); did you mean 'node_remap'?
     606 |         VMCOREINFO_OFFSET(pglist_data, node_mem_map);
         |                                        ^~~~~~~~~~~~
         |                                        node_remap
   kernel/crash_core.c:608:40: error: 'node_start_pfn' undeclared (first use in this function)
     608 |         VMCOREINFO_OFFSET(pglist_data, node_start_pfn);
         |                                        ^~~~~~~~~~~~~~
   kernel/crash_core.c:609:40: error: 'node_spanned_pages' undeclared (first use in this function); did you mean 'zone_managed_pages'?
     609 |         VMCOREINFO_OFFSET(pglist_data, node_spanned_pages);
         |                                        ^~~~~~~~~~~~~~~~~~
         |                                        zone_managed_pages
   kernel/crash_core.c:610:40: error: 'node_id' undeclared (first use in this function)
     610 |         VMCOREINFO_OFFSET(pglist_data, node_id);
         |                                        ^~~~~~~
   kernel/crash_core.c:612:33: error: 'vm_stat' undeclared (first use in this function); did you mean 'vfs_stat'?
     612 |         VMCOREINFO_OFFSET(zone, vm_stat);
         |                                 ^~~~~~~
         |                                 vfs_stat
   kernel/crash_core.c:613:33: error: 'spanned_pages' undeclared (first use in this function); did you mean 'shake_page'?
     613 |         VMCOREINFO_OFFSET(zone, spanned_pages);
         |                                 ^~~~~~~~~~~~~
         |                                 shake_page
   kernel/crash_core.c:614:38: error: 'free_list' undeclared (first use in this function); did you mean 'kfree_link'?
     614 |         VMCOREINFO_OFFSET(free_area, free_list);
         |                                      ^~~~~~~~~
         |                                      kfree_link
   kernel/crash_core.c:615:38: error: 'next' undeclared (first use in this function); did you mean 'net'?
     615 |         VMCOREINFO_OFFSET(list_head, next);
         |                                      ^~~~
         |                                      net
   kernel/crash_core.c:616:38: error: 'prev' undeclared (first use in this function)
     616 |         VMCOREINFO_OFFSET(list_head, prev);
         |                                      ^~~~
   kernel/crash_core.c:617:27: error: 'vmap_area' undeclared (first use in this function)
     617 |         VMCOREINFO_OFFSET(vmap_area, va_start);
         |                           ^~~~~~~~~
   kernel/crash_core.c:617:38: error: 'va_start' undeclared (first use in this function); did you mean '_start'?
     617 |         VMCOREINFO_OFFSET(vmap_area, va_start);
         |                                      ^~~~~~~~
         |                                      _start
   kernel/crash_core.c:618:38: error: 'list' undeclared (first use in this function)
     618 |         VMCOREINFO_OFFSET(vmap_area, list);
         |                                      ^~~~
   kernel/crash_core.c:619:9: error: implicit declaration of function 'VMCOREINFO_LENGTH' [-Werror=implicit-function-declaration]
     619 |         VMCOREINFO_LENGTH(zone.free_area, MAX_ORDER + 1);
         |         ^~~~~~~~~~~~~~~~~
   kernel/crash_core.c:622:9: error: implicit declaration of function 'VMCOREINFO_NUMBER' [-Werror=implicit-function-declaration]
     622 |         VMCOREINFO_NUMBER(NR_FREE_PAGES);
         |         ^~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors


vim +606 kernel/crash_core.c

692f66f26a4c19d Hari Bathini            2017-05-08  545  
692f66f26a4c19d Hari Bathini            2017-05-08  546  static int __init crash_save_vmcoreinfo_init(void)
692f66f26a4c19d Hari Bathini            2017-05-08  547  {
203e9e41219b4e7 Xunlei Pang             2017-07-12  548  	vmcoreinfo_data = (unsigned char *)get_zeroed_page(GFP_KERNEL);
203e9e41219b4e7 Xunlei Pang             2017-07-12  549  	if (!vmcoreinfo_data) {
203e9e41219b4e7 Xunlei Pang             2017-07-12  550  		pr_warn("Memory allocation for vmcoreinfo_data failed\n");
203e9e41219b4e7 Xunlei Pang             2017-07-12  551  		return -ENOMEM;
203e9e41219b4e7 Xunlei Pang             2017-07-12  552  	}
203e9e41219b4e7 Xunlei Pang             2017-07-12  553  
203e9e41219b4e7 Xunlei Pang             2017-07-12  554  	vmcoreinfo_note = alloc_pages_exact(VMCOREINFO_NOTE_SIZE,
203e9e41219b4e7 Xunlei Pang             2017-07-12  555  						GFP_KERNEL | __GFP_ZERO);
203e9e41219b4e7 Xunlei Pang             2017-07-12  556  	if (!vmcoreinfo_note) {
203e9e41219b4e7 Xunlei Pang             2017-07-12  557  		free_page((unsigned long)vmcoreinfo_data);
203e9e41219b4e7 Xunlei Pang             2017-07-12  558  		vmcoreinfo_data = NULL;
203e9e41219b4e7 Xunlei Pang             2017-07-12  559  		pr_warn("Memory allocation for vmcoreinfo_note failed\n");
203e9e41219b4e7 Xunlei Pang             2017-07-12  560  		return -ENOMEM;
203e9e41219b4e7 Xunlei Pang             2017-07-12  561  	}
203e9e41219b4e7 Xunlei Pang             2017-07-12  562  
692f66f26a4c19d Hari Bathini            2017-05-08  563  	VMCOREINFO_OSRELEASE(init_uts_ns.name.release);
44e8a5e9120bf4f Stephen Boyd            2021-07-07  564  	VMCOREINFO_BUILD_ID();
692f66f26a4c19d Hari Bathini            2017-05-08  565  	VMCOREINFO_PAGESIZE(PAGE_SIZE);
692f66f26a4c19d Hari Bathini            2017-05-08  566  
692f66f26a4c19d Hari Bathini            2017-05-08  567  	VMCOREINFO_SYMBOL(init_uts_ns);
ca4a9241cc5e718 Alexander Egorenkov     2020-12-15  568  	VMCOREINFO_OFFSET(uts_namespace, name);
692f66f26a4c19d Hari Bathini            2017-05-08  569  	VMCOREINFO_SYMBOL(node_online_map);
692f66f26a4c19d Hari Bathini            2017-05-08  570  #ifdef CONFIG_MMU
eff4345e7fba0aa Omar Sandoval           2018-08-21  571  	VMCOREINFO_SYMBOL_ARRAY(swapper_pg_dir);
692f66f26a4c19d Hari Bathini            2017-05-08  572  #endif
692f66f26a4c19d Hari Bathini            2017-05-08  573  	VMCOREINFO_SYMBOL(_stext);
692f66f26a4c19d Hari Bathini            2017-05-08  574  	VMCOREINFO_SYMBOL(vmap_area_list);
692f66f26a4c19d Hari Bathini            2017-05-08  575  
a9ee6cf5c60ed10 Mike Rapoport           2021-06-28  576  #ifndef CONFIG_NUMA
692f66f26a4c19d Hari Bathini            2017-05-08  577  	VMCOREINFO_SYMBOL(mem_map);
692f66f26a4c19d Hari Bathini            2017-05-08  578  	VMCOREINFO_SYMBOL(contig_page_data);
692f66f26a4c19d Hari Bathini            2017-05-08  579  #endif
692f66f26a4c19d Hari Bathini            2017-05-08  580  #ifdef CONFIG_SPARSEMEM
a0b1280368d1e91 Kirill A. Shutemov      2018-01-12  581  	VMCOREINFO_SYMBOL_ARRAY(mem_section);
692f66f26a4c19d Hari Bathini            2017-05-08  582  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
692f66f26a4c19d Hari Bathini            2017-05-08  583  	VMCOREINFO_STRUCT_SIZE(mem_section);
692f66f26a4c19d Hari Bathini            2017-05-08  584  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
4f5aecdff25f59f Pingfan Liu             2021-06-15  585  	VMCOREINFO_NUMBER(SECTION_SIZE_BITS);
1d50e5d0c505244 Bhupesh Sharma          2020-05-14  586  	VMCOREINFO_NUMBER(MAX_PHYSMEM_BITS);
692f66f26a4c19d Hari Bathini            2017-05-08  587  #endif
692f66f26a4c19d Hari Bathini            2017-05-08  588  	VMCOREINFO_STRUCT_SIZE(page);
692f66f26a4c19d Hari Bathini            2017-05-08  589  	VMCOREINFO_STRUCT_SIZE(pglist_data);
692f66f26a4c19d Hari Bathini            2017-05-08  590  	VMCOREINFO_STRUCT_SIZE(zone);
692f66f26a4c19d Hari Bathini            2017-05-08  591  	VMCOREINFO_STRUCT_SIZE(free_area);
692f66f26a4c19d Hari Bathini            2017-05-08  592  	VMCOREINFO_STRUCT_SIZE(list_head);
692f66f26a4c19d Hari Bathini            2017-05-08  593  	VMCOREINFO_SIZE(nodemask_t);
692f66f26a4c19d Hari Bathini            2017-05-08  594  	VMCOREINFO_OFFSET(page, flags);
692f66f26a4c19d Hari Bathini            2017-05-08  595  	VMCOREINFO_OFFSET(page, _refcount);
692f66f26a4c19d Hari Bathini            2017-05-08  596  	VMCOREINFO_OFFSET(page, mapping);
692f66f26a4c19d Hari Bathini            2017-05-08  597  	VMCOREINFO_OFFSET(page, lru);
692f66f26a4c19d Hari Bathini            2017-05-08  598  	VMCOREINFO_OFFSET(page, _mapcount);
692f66f26a4c19d Hari Bathini            2017-05-08  599  	VMCOREINFO_OFFSET(page, private);
1c5509be58f636a Matthew Wilcox (Oracle  2023-01-11  600) 	VMCOREINFO_OFFSET(folio, _folio_dtor);
1c5509be58f636a Matthew Wilcox (Oracle  2023-01-11  601) 	VMCOREINFO_OFFSET(folio, _folio_order);
692f66f26a4c19d Hari Bathini            2017-05-08  602  	VMCOREINFO_OFFSET(page, compound_head);
692f66f26a4c19d Hari Bathini            2017-05-08  603  	VMCOREINFO_OFFSET(pglist_data, node_zones);
692f66f26a4c19d Hari Bathini            2017-05-08  604  	VMCOREINFO_OFFSET(pglist_data, nr_zones);
43b02ba93b25b1c Mike Rapoport           2021-06-28  605  #ifdef CONFIG_FLATMEM
692f66f26a4c19d Hari Bathini            2017-05-08 @606  	VMCOREINFO_OFFSET(pglist_data, node_mem_map);
692f66f26a4c19d Hari Bathini            2017-05-08  607  #endif
692f66f26a4c19d Hari Bathini            2017-05-08  608  	VMCOREINFO_OFFSET(pglist_data, node_start_pfn);
692f66f26a4c19d Hari Bathini            2017-05-08  609  	VMCOREINFO_OFFSET(pglist_data, node_spanned_pages);
692f66f26a4c19d Hari Bathini            2017-05-08  610  	VMCOREINFO_OFFSET(pglist_data, node_id);
692f66f26a4c19d Hari Bathini            2017-05-08  611  	VMCOREINFO_OFFSET(zone, free_area);
692f66f26a4c19d Hari Bathini            2017-05-08  612  	VMCOREINFO_OFFSET(zone, vm_stat);
692f66f26a4c19d Hari Bathini            2017-05-08  613  	VMCOREINFO_OFFSET(zone, spanned_pages);
692f66f26a4c19d Hari Bathini            2017-05-08  614  	VMCOREINFO_OFFSET(free_area, free_list);
692f66f26a4c19d Hari Bathini            2017-05-08  615  	VMCOREINFO_OFFSET(list_head, next);
692f66f26a4c19d Hari Bathini            2017-05-08  616  	VMCOREINFO_OFFSET(list_head, prev);
692f66f26a4c19d Hari Bathini            2017-05-08  617  	VMCOREINFO_OFFSET(vmap_area, va_start);
692f66f26a4c19d Hari Bathini            2017-05-08  618  	VMCOREINFO_OFFSET(vmap_area, list);
23baf831a32c04f Kirill A. Shutemov      2023-03-15  619  	VMCOREINFO_LENGTH(zone.free_area, MAX_ORDER + 1);
692f66f26a4c19d Hari Bathini            2017-05-08  620  	log_buf_vmcoreinfo_setup();
692f66f26a4c19d Hari Bathini            2017-05-08  621  	VMCOREINFO_LENGTH(free_area.free_list, MIGRATE_TYPES);
692f66f26a4c19d Hari Bathini            2017-05-08  622  	VMCOREINFO_NUMBER(NR_FREE_PAGES);
692f66f26a4c19d Hari Bathini            2017-05-08  623  	VMCOREINFO_NUMBER(PG_lru);
692f66f26a4c19d Hari Bathini            2017-05-08  624  	VMCOREINFO_NUMBER(PG_private);
692f66f26a4c19d Hari Bathini            2017-05-08  625  	VMCOREINFO_NUMBER(PG_swapcache);
1cbf29da3628b66 Petr Tesarik            2018-04-13  626  	VMCOREINFO_NUMBER(PG_swapbacked);
692f66f26a4c19d Hari Bathini            2017-05-08  627  	VMCOREINFO_NUMBER(PG_slab);
692f66f26a4c19d Hari Bathini            2017-05-08  628  #ifdef CONFIG_MEMORY_FAILURE
692f66f26a4c19d Hari Bathini            2017-05-08  629  	VMCOREINFO_NUMBER(PG_hwpoison);
692f66f26a4c19d Hari Bathini            2017-05-08  630  #endif
692f66f26a4c19d Hari Bathini            2017-05-08  631  	VMCOREINFO_NUMBER(PG_head_mask);
6e292b9be7f4358 Matthew Wilcox          2018-06-07  632  #define PAGE_BUDDY_MAPCOUNT_VALUE	(~PG_buddy)
692f66f26a4c19d Hari Bathini            2017-05-08  633  	VMCOREINFO_NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE);
692f66f26a4c19d Hari Bathini            2017-05-08  634  #ifdef CONFIG_HUGETLB_PAGE
692f66f26a4c19d Hari Bathini            2017-05-08  635  	VMCOREINFO_NUMBER(HUGETLB_PAGE_DTOR);
e04b742f74c2362 David Hildenbrand       2019-03-05  636  #define PAGE_OFFLINE_MAPCOUNT_VALUE	(~PG_offline)
e04b742f74c2362 David Hildenbrand       2019-03-05  637  	VMCOREINFO_NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE);
692f66f26a4c19d Hari Bathini            2017-05-08  638  #endif
692f66f26a4c19d Hari Bathini            2017-05-08  639  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki


From samuel.holland at sifive.com  Sun Aug 27 22:26:29 2023
From: samuel.holland at sifive.com (Samuel Holland)
Date: Mon, 28 Aug 2023 00:26:29 -0500
Subject: [PATCH 8/8] crash_core.c: remove unneeded functions
In-Reply-To: <20230827101128.70931-9-bhe@redhat.com>
References: <20230827101128.70931-1-bhe@redhat.com>
 <20230827101128.70931-9-bhe@redhat.com>
Message-ID: <15058508-9e21-0d10-4eca-2c8a5562d36c@sifive.com>

Hi Baoquan,

On 2023-08-27 5:11 AM, Baoquan He wrote:
> So far, nobody calls functions parse_crashkernel_high() and
> parse_crashkernel_high(), remove both of them.

minor typo: should be parse_crashkernel_low().

Regards,
Samuel

> Signed-off-by: Baoquan He <bhe at redhat.com>
> ---
>  include/linux/crash_core.h |  4 ----
>  kernel/crash_core.c        | 18 ------------------
>  2 files changed, 22 deletions(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 2f732493e922..0c512a80a768 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -80,10 +80,6 @@ void final_note(Elf_Word *buf);
>  int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
>  		unsigned long long *crash_size, unsigned long long *crash_base,
>  		unsigned long long *low_size, bool *high);
> -int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
> -		unsigned long long *crash_size, unsigned long long *crash_base);
> -int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
> -		unsigned long long *crash_size, unsigned long long *crash_base);
>  
>  #ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
>  #ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index 6bc00cc390b5..61a8ea3b23a2 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -323,24 +323,6 @@ int __init parse_crashkernel(char *cmdline,
>  	return 0;
>  }
>  
> -int __init parse_crashkernel_high(char *cmdline,
> -			     unsigned long long system_ram,
> -			     unsigned long long *crash_size,
> -			     unsigned long long *crash_base)
> -{
> -	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> -				suffix_tbl[SUFFIX_HIGH]);
> -}
> -
> -int __init parse_crashkernel_low(char *cmdline,
> -			     unsigned long long system_ram,
> -			     unsigned long long *crash_size,
> -			     unsigned long long *crash_base)
> -{
> -	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> -				suffix_tbl[SUFFIX_LOW]);
> -}
> -
>  /*
>   * Add a dummy early_param handler to mark crashkernel= as a known command line
>   * parameter and suppress incorrect warnings in init/main.c.



From hbathini at linux.ibm.com  Sun Aug 27 23:16:14 2023
From: hbathini at linux.ibm.com (Hari Bathini)
Date: Mon, 28 Aug 2023 11:46:14 +0530
Subject: [PATCH] kexec: update manpage with explicit mention of clean
 kexec
In-Reply-To: <20230707052929.427625-1-hbathini@linux.ibm.com>
References: <20230707052929.427625-1-hbathini@linux.ibm.com>
Message-ID: <63e65749-4ec7-c618-72d3-1568992fff7a@linux.ibm.com>

Any comments on this, please..

On 07/07/23 10:59 am, Hari Bathini wrote:
> While the manpage does mention about kexec boot with a clean shutdown,
> it is not explicit about it. Make it explicit.
> 
> Signed-off-by: Hari Bathini <hbathini at linux.ibm.com>
> ---
>   kexec/kexec.8 | 12 ++++++++++--
>   1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/kexec/kexec.8 b/kexec/kexec.8
> index 3a344c5..87e4ab5 100644
> --- a/kexec/kexec.8
> +++ b/kexec/kexec.8
> @@ -95,8 +95,16 @@ then you would use the following command to load the kernel:
>   .RB "\-\-append=" "root=/dev/hda1" "\ \-\-initrd=" /boot/initrd
>   .RE
>   .PP
> -After this kernel is loaded, it can be booted to at any time using the
> -command:
> +After this kernel is loaded, assuming your Linux distribution supports
> +kexec-based rebooting, it can be booted to, with a clean shutdown, using
> +the command:
> +
> +.RS
> +.BR reboot
> +.RE
> +.PP
> +Alternatively, it can also be booted to, without calling shutdown(8), with
> +the command:
> 
>   .RS
>   .BR kexec \ \-e


From horms at kernel.org  Mon Aug 28 00:48:10 2023
From: horms at kernel.org (Simon Horman)
Date: Mon, 28 Aug 2023 09:48:10 +0200
Subject: [PATCH] kexec: update manpage with explicit mention of clean
 kexec
In-Reply-To: <63e65749-4ec7-c618-72d3-1568992fff7a@linux.ibm.com>
References: <20230707052929.427625-1-hbathini@linux.ibm.com>
 <63e65749-4ec7-c618-72d3-1568992fff7a@linux.ibm.com>
Message-ID: <20230828074810.GY3523530@kernel.org>

On Mon, Aug 28, 2023 at 11:46:14AM +0530, Hari Bathini wrote:
> Any comments on this, please..

Hi Hari,

Sorry, I had missed this.

> On 07/07/23 10:59 am, Hari Bathini wrote:
> > While the manpage does mention about kexec boot with a clean shutdown,
> > it is not explicit about it. Make it explicit.
> > 
> > Signed-off-by: Hari Bathini <hbathini at linux.ibm.com>
> > ---
> >   kexec/kexec.8 | 12 ++++++++++--
> >   1 file changed, 10 insertions(+), 2 deletions(-)
> > 
> > diff --git a/kexec/kexec.8 b/kexec/kexec.8
> > index 3a344c5..87e4ab5 100644
> > --- a/kexec/kexec.8
> > +++ b/kexec/kexec.8
> > @@ -95,8 +95,16 @@ then you would use the following command to load the kernel:
> >   .RB "\-\-append=" "root=/dev/hda1" "\ \-\-initrd=" /boot/initrd
> >   .RE
> >   .PP
> > -After this kernel is loaded, it can be booted to at any time using the
> > -command:
> > +After this kernel is loaded, assuming your Linux distribution supports
> > +kexec-based rebooting, it can be booted to, with a clean shutdown, using
> > +the command:

Perhaps user-space would be better than "your Linux distribution".
But perhaps that is just kernel developer speak and makes less
sense to others.

> > +
> > +.RS
> > +.BR reboot
> > +.RE
> > +.PP
> > +Alternatively, it can also be booted to, without calling shutdown(8), with
> > +the command:
> > 
> >   .RS
> >   .BR kexec \ \-e
> 


From horms at kernel.org  Mon Aug 28 01:00:51 2023
From: horms at kernel.org (Simon Horman)
Date: Mon, 28 Aug 2023 10:00:51 +0200
Subject: [ANNOUNCE] kexec-tools 2.0.27
Message-ID: <20230828080051.GA3806727@kernel.org>

Hi all,

I am happy to announce the release of kexec-tools 2.0.27.

This is a feature release coinciding with the release of v6.5
of the Linux Kernel.

This release can be downloaded from kernel.org:

	http://kernel.org/pub/linux/utils/kernel/kexec/kexec-tools-2.0.27.tar.xz
	http://kernel.org/pub/linux/utils/kernel/kexec/

It is also tagged it in git:

	git://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git

Thanks to everyone who has contributed to kexec!

Commits since v2.0.26:

17590eedff6d kexec-tools 2.0.27
777ca453ca69 kexec-tools 2.0.27-rc1
f67c4146d7b5 arm64: Hook up the ZBOOT support as vmlinuz
fc7b83bdf734 arm64: Add ZBOOT PE containing compressed image support
f41c4182b0c4 kexec/zboot: Add arch independent zboot support
1572b91da7c4 kexec: Introduce a member kernel_fd in kexec_info
714fa11590fe kexec/arm64: Simplify the code for zImage
a8de94e5f033 LoongArch: kdump: Set up kernel image segment
4203eaccfa92 kexec: __NR_kexec_file_load is set to undefined on LoongArch
63e9a012112e ppc64: Add elf-ppc64 file types/options and an arch specific flag to man page
806711fca9e9 x86: add devicetree support
29fe5067ed07 kexec: make -a the default
e63fefd4fc35 ppc64: add --reuse-cmdline parameter support
8fc55927f700 kexec-tools 2.0.26.git


From bhe at redhat.com  Mon Aug 28 02:42:05 2023
From: bhe at redhat.com (Baoquan He)
Date: Mon, 28 Aug 2023 17:42:05 +0800
Subject: [PATCH] kexec: update manpage with explicit mention of clean
 kexec
In-Reply-To: <20230828074810.GY3523530@kernel.org>
References: <20230707052929.427625-1-hbathini@linux.ibm.com>
 <63e65749-4ec7-c618-72d3-1568992fff7a@linux.ibm.com>
 <20230828074810.GY3523530@kernel.org>
Message-ID: <ZOxr7VjBhBe6WW91@MiWiFi-R3L-srv>

On 08/28/23 at 09:48am, Simon Horman wrote:
> On Mon, Aug 28, 2023 at 11:46:14AM +0530, Hari Bathini wrote:
> > Any comments on this, please..
> 
> Hi Hari,
> 
> Sorry, I had missed this.
> 
> > On 07/07/23 10:59 am, Hari Bathini wrote:
> > > While the manpage does mention about kexec boot with a clean shutdown,
> > > it is not explicit about it. Make it explicit.
> > > 
> > > Signed-off-by: Hari Bathini <hbathini at linux.ibm.com>
> > > ---
> > >   kexec/kexec.8 | 12 ++++++++++--
> > >   1 file changed, 10 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/kexec/kexec.8 b/kexec/kexec.8
> > > index 3a344c5..87e4ab5 100644
> > > --- a/kexec/kexec.8
> > > +++ b/kexec/kexec.8
> > > @@ -95,8 +95,16 @@ then you would use the following command to load the kernel:
> > >   .RB "\-\-append=" "root=/dev/hda1" "\ \-\-initrd=" /boot/initrd
> > >   .RE
> > >   .PP
> > > -After this kernel is loaded, it can be booted to at any time using the
> > > -command:
> > > +After this kernel is loaded, assuming your Linux distribution supports
> > > +kexec-based rebooting, it can be booted to, with a clean shutdown, using
> > > +the command:
> 
> Perhaps user-space would be better than "your Linux distribution".
> But perhaps that is just kernel developer speak and makes less
> sense to others.

Agree, linux distrobution doesn't make sense here. And since this is a
kexec man page, does it really need to make this change? We know
kexec-tools loaded a kernel, we will use kexec-tools to trigger the
reboot. Wondering why we need to stress distros or kernel or kexec-tools
supports the kexec-based rebooting. 'with a clean shutdown' seems a
little helpful though.

> 
> > > +
> > > +.RS
> > > +.BR reboot
> > > +.RE
> > > +.PP
> > > +Alternatively, it can also be booted to, without calling shutdown(8), with
> > > +the command:
> > > 
> > >   .RS
> > >   .BR kexec \ \-e
> > 
> 
> _______________________________________________
> kexec mailing list
> kexec at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
> 



From bhe at redhat.com  Mon Aug 28 02:42:48 2023
From: bhe at redhat.com (Baoquan He)
Date: Mon, 28 Aug 2023 17:42:48 +0800
Subject: [PATCH 8/8] crash_core.c: remove unneeded functions
In-Reply-To: <15058508-9e21-0d10-4eca-2c8a5562d36c@sifive.com>
References: <20230827101128.70931-1-bhe@redhat.com>
 <20230827101128.70931-9-bhe@redhat.com>
 <15058508-9e21-0d10-4eca-2c8a5562d36c@sifive.com>
Message-ID: <ZOxsGIeYYvypf9op@MiWiFi-R3L-srv>

On 08/28/23 at 12:26am, Samuel Holland wrote:
> Hi Baoquan,
> 
> On 2023-08-27 5:11 AM, Baoquan He wrote:
> > So far, nobody calls functions parse_crashkernel_high() and
> > parse_crashkernel_high(), remove both of them.
> 
> minor typo: should be parse_crashkernel_low().

Good catch, will update. Thanks.

> 
> > Signed-off-by: Baoquan He <bhe at redhat.com>
> > ---
> >  include/linux/crash_core.h |  4 ----
> >  kernel/crash_core.c        | 18 ------------------
> >  2 files changed, 22 deletions(-)
> > 
> > diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> > index 2f732493e922..0c512a80a768 100644
> > --- a/include/linux/crash_core.h
> > +++ b/include/linux/crash_core.h
> > @@ -80,10 +80,6 @@ void final_note(Elf_Word *buf);
> >  int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
> >  		unsigned long long *crash_size, unsigned long long *crash_base,
> >  		unsigned long long *low_size, bool *high);
> > -int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
> > -		unsigned long long *crash_size, unsigned long long *crash_base);
> > -int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
> > -		unsigned long long *crash_size, unsigned long long *crash_base);
> >  
> >  #ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> >  #ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
> > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> > index 6bc00cc390b5..61a8ea3b23a2 100644
> > --- a/kernel/crash_core.c
> > +++ b/kernel/crash_core.c
> > @@ -323,24 +323,6 @@ int __init parse_crashkernel(char *cmdline,
> >  	return 0;
> >  }
> >  
> > -int __init parse_crashkernel_high(char *cmdline,
> > -			     unsigned long long system_ram,
> > -			     unsigned long long *crash_size,
> > -			     unsigned long long *crash_base)
> > -{
> > -	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> > -				suffix_tbl[SUFFIX_HIGH]);
> > -}
> > -
> > -int __init parse_crashkernel_low(char *cmdline,
> > -			     unsigned long long system_ram,
> > -			     unsigned long long *crash_size,
> > -			     unsigned long long *crash_base)
> > -{
> > -	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> > -				suffix_tbl[SUFFIX_LOW]);
> > -}
> > -
> >  /*
> >   * Add a dummy early_param handler to mark crashkernel= as a known command line
> >   * parameter and suppress incorrect warnings in init/main.c.
> 



From leah at vuxu.org  Mon Aug 28 02:58:15 2023
From: leah at vuxu.org (Leah Neukirchen)
Date: Mon, 28 Aug 2023 11:58:15 +0200
Subject: [kexec-tools PATCH] build: fix tarball creation
Message-ID: <20230828095840.595-1-leah@vuxu.org>

The fix in 41b77edac did not work, bsdtar still complains about
"hardlink pointing to itself".

Simplify the code instead: since the staging directory contains
exactly the files we want, just package it as a whole.

Signed-off-by: Leah Neukirchen <leah at vuxu.org>
---
 Makefile.in | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index 09bbd5c..3cad22a 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -179,8 +179,6 @@ GENERATED_SRCS:= $(SPEC)
 TARBALL=$(PACKAGE_NAME)-$(PACKAGE_VERSION).tar
 TARBALL.gz=$(TARBALL).gz
 SRCS:= $(dist)
-PSRCS:=$(foreach s, $(SRCS), $(PACKAGE_NAME)-$(PACKAGE_VERSION)/$(s))
-PGSRCS:=$(foreach s, $(GENERATED_SRCS), $(PACKAGE_NAME)-$(PACKAGE_VERSION)/$(s))
 
 MAN_PAGES:=$(KEXEC_MANPAGE) $(VMCORE_DMESG_MANPAGE)
 BINARIES_i386:=$(KEXEC_TEST)
@@ -223,8 +221,7 @@ $(TARBALL): $(SRCS) $(GENERATED_SRCS)
 	$(MKDIR) $(PACKAGE_NAME)-$(PACKAGE_VERSION)
 	$(TAR) -c $(SRCS) $(GENERATED_SRCS) | \
 	        $(TAR) -C $(PACKAGE_NAME)-$(PACKAGE_VERSION) -x
-	$(TAR) -cf $@ $(PSRCS)
-	$(TAR) -rf $@ $(PGSRCS)
+	$(TAR) -cf $@ $(PACKAGE_NAME)-$(PACKAGE_VERSION)
 	$(RM) -rf $(PACKAGE_NAME)-$(PACKAGE_VERSION)
 
 $(TARBALL.gz): $(TARBALL)
-- 
2.42.0



From hbathini at linux.ibm.com  Mon Aug 28 03:27:11 2023
From: hbathini at linux.ibm.com (Hari Bathini)
Date: Mon, 28 Aug 2023 15:57:11 +0530
Subject: [PATCH] kexec: update manpage with explicit mention of clean
 kexec
In-Reply-To: <20230828074810.GY3523530@kernel.org>
References: <20230707052929.427625-1-hbathini@linux.ibm.com>
 <63e65749-4ec7-c618-72d3-1568992fff7a@linux.ibm.com>
 <20230828074810.GY3523530@kernel.org>
Message-ID: <4f81d535-ff2c-fba3-8285-7e68bd6493bb@linux.ibm.com>



On 28/08/23 1:18 pm, Simon Horman wrote:
> On Mon, Aug 28, 2023 at 11:46:14AM +0530, Hari Bathini wrote:
>> Any comments on this, please..
> 
> Hi Hari,
> 
> Sorry, I had missed this.

No worries. Thanks for the response.

> 
>> On 07/07/23 10:59 am, Hari Bathini wrote:
>>> While the manpage does mention about kexec boot with a clean shutdown,
>>> it is not explicit about it. Make it explicit.
>>>
>>> Signed-off-by: Hari Bathini <hbathini at linux.ibm.com>
>>> ---
>>>    kexec/kexec.8 | 12 ++++++++++--
>>>    1 file changed, 10 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/kexec/kexec.8 b/kexec/kexec.8
>>> index 3a344c5..87e4ab5 100644
>>> --- a/kexec/kexec.8
>>> +++ b/kexec/kexec.8
>>> @@ -95,8 +95,16 @@ then you would use the following command to load the kernel:
>>>    .RB "\-\-append=" "root=/dev/hda1" "\ \-\-initrd=" /boot/initrd
>>>    .RE
>>>    .PP
>>> -After this kernel is loaded, it can be booted to at any time using the
>>> -command:
>>> +After this kernel is loaded, assuming your Linux distribution supports
>>> +kexec-based rebooting, it can be booted to, with a clean shutdown, using
>>> +the command:
> 
> Perhaps user-space would be better than "your Linux distribution".
> But perhaps that is just kernel developer speak and makes less
> sense to others.

No strong inclination to putting it either way. Would leave it to you :)

- Hari


From bhe at redhat.com  Mon Aug 28 05:57:33 2023
From: bhe at redhat.com (Baoquan He)
Date: Mon, 28 Aug 2023 20:57:33 +0800
Subject: [PATCH 5/8] crash_core: add generic function to do reservation
In-Reply-To: <202308272150.p3kRkMoF-lkp@intel.com>
References: <20230827101128.70931-6-bhe@redhat.com>
 <202308272150.p3kRkMoF-lkp@intel.com>
Message-ID: <ZOyZvawxNOSUJcsa@MiWiFi-R3L-srv>

On 08/27/23 at 09:53pm, kernel test robot wrote:
> Hi Baoquan,
> 
> kernel test robot noticed the following build errors:
> 
> [auto build test ERROR on tip/x86/core]
> [also build test ERROR on powerpc/next powerpc/fixes linus/master v6.5-rc7]
> [cannot apply to arm64/for-next/core next-20230825]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch#_base_tree_information]
> 
> url:    https://github.com/intel-lab-lkp/linux/commits/Baoquan-He/crash_core-c-remove-unnecessary-parameter-of-function/20230827-181555
> base:   tip/x86/core
> patch link:    https://lore.kernel.org/r/20230827101128.70931-6-bhe%40redhat.com
> patch subject: [PATCH 5/8] crash_core: add generic function to do reservation
> config: csky-defconfig (https://download.01.org/0day-ci/archive/20230827/202308272150.p3kRkMoF-lkp at intel.com/config)
> compiler: csky-linux-gcc (GCC) 13.2.0
> reproduce: (https://download.01.org/0day-ci/archive/20230827/202308272150.p3kRkMoF-lkp at intel.com/reproduce)
> 
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <lkp at intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202308272150.p3kRkMoF-lkp at intel.com/

Thanks for reporting this. The error is caused by patch 3:
[PATCH 3/8] include/linux/kexec.h: move down crash_core.h including

In this lkp's config, CONFIG_CRASH_CORE=y and CONFIG_KEXEC_CORE=n, the
moving down of crash_core.h including will triger the error. I will
think of another way to fix this.

> 
> All errors (new ones prefixed by >>):
> 
>      537 | void __weak arch_crash_save_vmcoreinfo(void)
>          |             ^~~~~~~~~~~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:540:20: warning: no previous prototype for 'paddr_vmcoreinfo_note' [-Wmissing-prototypes]
>      540 | phys_addr_t __weak paddr_vmcoreinfo_note(void)
>          |                    ^~~~~~~~~~~~~~~~~~~~~
>    kernel/crash_core.c: In function 'crash_save_vmcoreinfo_init':
>    kernel/crash_core.c:554:45: error: 'VMCOREINFO_NOTE_SIZE' undeclared (first use in this function)
>      554 |         vmcoreinfo_note = alloc_pages_exact(VMCOREINFO_NOTE_SIZE,
>          |                                             ^~~~~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:563:9: error: implicit declaration of function 'VMCOREINFO_OSRELEASE' [-Werror=implicit-function-declaration]
>      563 |         VMCOREINFO_OSRELEASE(init_uts_ns.name.release);
>          |         ^~~~~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:564:9: error: implicit declaration of function 'VMCOREINFO_BUILD_ID' [-Werror=implicit-function-declaration]
>      564 |         VMCOREINFO_BUILD_ID();
>          |         ^~~~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:565:9: error: implicit declaration of function 'VMCOREINFO_PAGESIZE' [-Werror=implicit-function-declaration]
>      565 |         VMCOREINFO_PAGESIZE(PAGE_SIZE);
>          |         ^~~~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:567:9: error: implicit declaration of function 'VMCOREINFO_SYMBOL' [-Werror=implicit-function-declaration]
>      567 |         VMCOREINFO_SYMBOL(init_uts_ns);
>          |         ^~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:568:9: error: implicit declaration of function 'VMCOREINFO_OFFSET' [-Werror=implicit-function-declaration]
>      568 |         VMCOREINFO_OFFSET(uts_namespace, name);
>          |         ^~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:568:27: error: 'uts_namespace' undeclared (first use in this function)
>      568 |         VMCOREINFO_OFFSET(uts_namespace, name);
>          |                           ^~~~~~~~~~~~~
>    kernel/crash_core.c:568:42: error: 'name' undeclared (first use in this function)
>      568 |         VMCOREINFO_OFFSET(uts_namespace, name);
>          |                                          ^~~~
>    kernel/crash_core.c:571:9: error: implicit declaration of function 'VMCOREINFO_SYMBOL_ARRAY' [-Werror=implicit-function-declaration]
>      571 |         VMCOREINFO_SYMBOL_ARRAY(swapper_pg_dir);
>          |         ^~~~~~~~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:588:9: error: implicit declaration of function 'VMCOREINFO_STRUCT_SIZE' [-Werror=implicit-function-declaration]
>      588 |         VMCOREINFO_STRUCT_SIZE(page);
>          |         ^~~~~~~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:588:32: error: 'page' undeclared (first use in this function)
>      588 |         VMCOREINFO_STRUCT_SIZE(page);
>          |                                ^~~~
>    kernel/crash_core.c:589:32: error: 'pglist_data' undeclared (first use in this function)
>      589 |         VMCOREINFO_STRUCT_SIZE(pglist_data);
>          |                                ^~~~~~~~~~~
>    kernel/crash_core.c:590:32: error: 'zone' undeclared (first use in this function)
>      590 |         VMCOREINFO_STRUCT_SIZE(zone);
>          |                                ^~~~
>    kernel/crash_core.c:591:32: error: 'free_area' undeclared (first use in this function)
>      591 |         VMCOREINFO_STRUCT_SIZE(free_area);
>          |                                ^~~~~~~~~
>    kernel/crash_core.c:592:32: error: 'list_head' undeclared (first use in this function)
>      592 |         VMCOREINFO_STRUCT_SIZE(list_head);
>          |                                ^~~~~~~~~
>    kernel/crash_core.c:593:9: error: implicit declaration of function 'VMCOREINFO_SIZE' [-Werror=implicit-function-declaration]
>      593 |         VMCOREINFO_SIZE(nodemask_t);
>          |         ^~~~~~~~~~~~~~~
>    kernel/crash_core.c:593:25: error: expected expression before 'nodemask_t'
>      593 |         VMCOREINFO_SIZE(nodemask_t);
>          |                         ^~~~~~~~~~
>    kernel/crash_core.c:594:33: error: 'flags' undeclared (first use in this function); did you mean 'fls'?
>      594 |         VMCOREINFO_OFFSET(page, flags);
>          |                                 ^~~~~
>          |                                 fls
>    kernel/crash_core.c:595:33: error: '_refcount' undeclared (first use in this function); did you mean 'seqcount'?
>      595 |         VMCOREINFO_OFFSET(page, _refcount);
>          |                                 ^~~~~~~~~
>          |                                 seqcount
>    kernel/crash_core.c:596:33: error: 'mapping' undeclared (first use in this function)
>      596 |         VMCOREINFO_OFFSET(page, mapping);
>          |                                 ^~~~~~~
>    kernel/crash_core.c:597:33: error: 'lru' undeclared (first use in this function)
>      597 |         VMCOREINFO_OFFSET(page, lru);
>          |                                 ^~~
>    kernel/crash_core.c:598:33: error: '_mapcount' undeclared (first use in this function); did you mean 'nmi_count'?
>      598 |         VMCOREINFO_OFFSET(page, _mapcount);
>          |                                 ^~~~~~~~~
>          |                                 nmi_count
>    kernel/crash_core.c:599:33: error: 'private' undeclared (first use in this function); did you mean 'PG_private'?
>      599 |         VMCOREINFO_OFFSET(page, private);
>          |                                 ^~~~~~~
>          |                                 PG_private
>    kernel/crash_core.c:600:27: error: 'folio' undeclared (first use in this function)
>      600 |         VMCOREINFO_OFFSET(folio, _folio_dtor);
>          |                           ^~~~~
>    kernel/crash_core.c:600:34: error: '_folio_dtor' undeclared (first use in this function)
>      600 |         VMCOREINFO_OFFSET(folio, _folio_dtor);
>          |                                  ^~~~~~~~~~~
>    kernel/crash_core.c:601:34: error: '_folio_order' undeclared (first use in this function); did you mean 'folio_order'?
>      601 |         VMCOREINFO_OFFSET(folio, _folio_order);
>          |                                  ^~~~~~~~~~~~
>          |                                  folio_order
>    kernel/crash_core.c:602:33: error: 'compound_head' undeclared (first use in this function); did you mean '_compound_head'?
>      602 |         VMCOREINFO_OFFSET(page, compound_head);
>          |                                 ^~~~~~~~~~~~~
>          |                                 _compound_head
>    kernel/crash_core.c:603:40: error: 'node_zones' undeclared (first use in this function); did you mean 'node_zonelist'?
>      603 |         VMCOREINFO_OFFSET(pglist_data, node_zones);
>          |                                        ^~~~~~~~~~
>          |                                        node_zonelist
>    kernel/crash_core.c:604:40: error: 'nr_zones' undeclared (first use in this function)
>      604 |         VMCOREINFO_OFFSET(pglist_data, nr_zones);
>          |                                        ^~~~~~~~
> >> kernel/crash_core.c:606:40: error: 'node_mem_map' undeclared (first use in this function); did you mean 'node_remap'?
>      606 |         VMCOREINFO_OFFSET(pglist_data, node_mem_map);
>          |                                        ^~~~~~~~~~~~
>          |                                        node_remap
>    kernel/crash_core.c:608:40: error: 'node_start_pfn' undeclared (first use in this function)
>      608 |         VMCOREINFO_OFFSET(pglist_data, node_start_pfn);
>          |                                        ^~~~~~~~~~~~~~
>    kernel/crash_core.c:609:40: error: 'node_spanned_pages' undeclared (first use in this function); did you mean 'zone_managed_pages'?
>      609 |         VMCOREINFO_OFFSET(pglist_data, node_spanned_pages);
>          |                                        ^~~~~~~~~~~~~~~~~~
>          |                                        zone_managed_pages
>    kernel/crash_core.c:610:40: error: 'node_id' undeclared (first use in this function)
>      610 |         VMCOREINFO_OFFSET(pglist_data, node_id);
>          |                                        ^~~~~~~
>    kernel/crash_core.c:612:33: error: 'vm_stat' undeclared (first use in this function); did you mean 'vfs_stat'?
>      612 |         VMCOREINFO_OFFSET(zone, vm_stat);
>          |                                 ^~~~~~~
>          |                                 vfs_stat
>    kernel/crash_core.c:613:33: error: 'spanned_pages' undeclared (first use in this function); did you mean 'shake_page'?
>      613 |         VMCOREINFO_OFFSET(zone, spanned_pages);
>          |                                 ^~~~~~~~~~~~~
>          |                                 shake_page
>    kernel/crash_core.c:614:38: error: 'free_list' undeclared (first use in this function); did you mean 'kfree_link'?
>      614 |         VMCOREINFO_OFFSET(free_area, free_list);
>          |                                      ^~~~~~~~~
>          |                                      kfree_link
>    kernel/crash_core.c:615:38: error: 'next' undeclared (first use in this function); did you mean 'net'?
>      615 |         VMCOREINFO_OFFSET(list_head, next);
>          |                                      ^~~~
>          |                                      net
>    kernel/crash_core.c:616:38: error: 'prev' undeclared (first use in this function)
>      616 |         VMCOREINFO_OFFSET(list_head, prev);
>          |                                      ^~~~
>    kernel/crash_core.c:617:27: error: 'vmap_area' undeclared (first use in this function)
>      617 |         VMCOREINFO_OFFSET(vmap_area, va_start);
>          |                           ^~~~~~~~~
>    kernel/crash_core.c:617:38: error: 'va_start' undeclared (first use in this function); did you mean '_start'?
>      617 |         VMCOREINFO_OFFSET(vmap_area, va_start);
>          |                                      ^~~~~~~~
>          |                                      _start
>    kernel/crash_core.c:618:38: error: 'list' undeclared (first use in this function)
>      618 |         VMCOREINFO_OFFSET(vmap_area, list);
>          |                                      ^~~~
>    kernel/crash_core.c:619:9: error: implicit declaration of function 'VMCOREINFO_LENGTH' [-Werror=implicit-function-declaration]
>      619 |         VMCOREINFO_LENGTH(zone.free_area, MAX_ORDER + 1);
>          |         ^~~~~~~~~~~~~~~~~
>    kernel/crash_core.c:622:9: error: implicit declaration of function 'VMCOREINFO_NUMBER' [-Werror=implicit-function-declaration]
>      622 |         VMCOREINFO_NUMBER(NR_FREE_PAGES);
>          |         ^~~~~~~~~~~~~~~~~
>    cc1: some warnings being treated as errors
> 
> 
> vim +606 kernel/crash_core.c
> 
> 692f66f26a4c19d Hari Bathini            2017-05-08  545  
> 692f66f26a4c19d Hari Bathini            2017-05-08  546  static int __init crash_save_vmcoreinfo_init(void)
> 692f66f26a4c19d Hari Bathini            2017-05-08  547  {
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  548  	vmcoreinfo_data = (unsigned char *)get_zeroed_page(GFP_KERNEL);
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  549  	if (!vmcoreinfo_data) {
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  550  		pr_warn("Memory allocation for vmcoreinfo_data failed\n");
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  551  		return -ENOMEM;
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  552  	}
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  553  
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  554  	vmcoreinfo_note = alloc_pages_exact(VMCOREINFO_NOTE_SIZE,
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  555  						GFP_KERNEL | __GFP_ZERO);
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  556  	if (!vmcoreinfo_note) {
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  557  		free_page((unsigned long)vmcoreinfo_data);
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  558  		vmcoreinfo_data = NULL;
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  559  		pr_warn("Memory allocation for vmcoreinfo_note failed\n");
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  560  		return -ENOMEM;
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  561  	}
> 203e9e41219b4e7 Xunlei Pang             2017-07-12  562  
> 692f66f26a4c19d Hari Bathini            2017-05-08  563  	VMCOREINFO_OSRELEASE(init_uts_ns.name.release);
> 44e8a5e9120bf4f Stephen Boyd            2021-07-07  564  	VMCOREINFO_BUILD_ID();
> 692f66f26a4c19d Hari Bathini            2017-05-08  565  	VMCOREINFO_PAGESIZE(PAGE_SIZE);
> 692f66f26a4c19d Hari Bathini            2017-05-08  566  
> 692f66f26a4c19d Hari Bathini            2017-05-08  567  	VMCOREINFO_SYMBOL(init_uts_ns);
> ca4a9241cc5e718 Alexander Egorenkov     2020-12-15  568  	VMCOREINFO_OFFSET(uts_namespace, name);
> 692f66f26a4c19d Hari Bathini            2017-05-08  569  	VMCOREINFO_SYMBOL(node_online_map);
> 692f66f26a4c19d Hari Bathini            2017-05-08  570  #ifdef CONFIG_MMU
> eff4345e7fba0aa Omar Sandoval           2018-08-21  571  	VMCOREINFO_SYMBOL_ARRAY(swapper_pg_dir);
> 692f66f26a4c19d Hari Bathini            2017-05-08  572  #endif
> 692f66f26a4c19d Hari Bathini            2017-05-08  573  	VMCOREINFO_SYMBOL(_stext);
> 692f66f26a4c19d Hari Bathini            2017-05-08  574  	VMCOREINFO_SYMBOL(vmap_area_list);
> 692f66f26a4c19d Hari Bathini            2017-05-08  575  
> a9ee6cf5c60ed10 Mike Rapoport           2021-06-28  576  #ifndef CONFIG_NUMA
> 692f66f26a4c19d Hari Bathini            2017-05-08  577  	VMCOREINFO_SYMBOL(mem_map);
> 692f66f26a4c19d Hari Bathini            2017-05-08  578  	VMCOREINFO_SYMBOL(contig_page_data);
> 692f66f26a4c19d Hari Bathini            2017-05-08  579  #endif
> 692f66f26a4c19d Hari Bathini            2017-05-08  580  #ifdef CONFIG_SPARSEMEM
> a0b1280368d1e91 Kirill A. Shutemov      2018-01-12  581  	VMCOREINFO_SYMBOL_ARRAY(mem_section);
> 692f66f26a4c19d Hari Bathini            2017-05-08  582  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
> 692f66f26a4c19d Hari Bathini            2017-05-08  583  	VMCOREINFO_STRUCT_SIZE(mem_section);
> 692f66f26a4c19d Hari Bathini            2017-05-08  584  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> 4f5aecdff25f59f Pingfan Liu             2021-06-15  585  	VMCOREINFO_NUMBER(SECTION_SIZE_BITS);
> 1d50e5d0c505244 Bhupesh Sharma          2020-05-14  586  	VMCOREINFO_NUMBER(MAX_PHYSMEM_BITS);
> 692f66f26a4c19d Hari Bathini            2017-05-08  587  #endif
> 692f66f26a4c19d Hari Bathini            2017-05-08  588  	VMCOREINFO_STRUCT_SIZE(page);
> 692f66f26a4c19d Hari Bathini            2017-05-08  589  	VMCOREINFO_STRUCT_SIZE(pglist_data);
> 692f66f26a4c19d Hari Bathini            2017-05-08  590  	VMCOREINFO_STRUCT_SIZE(zone);
> 692f66f26a4c19d Hari Bathini            2017-05-08  591  	VMCOREINFO_STRUCT_SIZE(free_area);
> 692f66f26a4c19d Hari Bathini            2017-05-08  592  	VMCOREINFO_STRUCT_SIZE(list_head);
> 692f66f26a4c19d Hari Bathini            2017-05-08  593  	VMCOREINFO_SIZE(nodemask_t);
> 692f66f26a4c19d Hari Bathini            2017-05-08  594  	VMCOREINFO_OFFSET(page, flags);
> 692f66f26a4c19d Hari Bathini            2017-05-08  595  	VMCOREINFO_OFFSET(page, _refcount);
> 692f66f26a4c19d Hari Bathini            2017-05-08  596  	VMCOREINFO_OFFSET(page, mapping);
> 692f66f26a4c19d Hari Bathini            2017-05-08  597  	VMCOREINFO_OFFSET(page, lru);
> 692f66f26a4c19d Hari Bathini            2017-05-08  598  	VMCOREINFO_OFFSET(page, _mapcount);
> 692f66f26a4c19d Hari Bathini            2017-05-08  599  	VMCOREINFO_OFFSET(page, private);
> 1c5509be58f636a Matthew Wilcox (Oracle  2023-01-11  600) 	VMCOREINFO_OFFSET(folio, _folio_dtor);
> 1c5509be58f636a Matthew Wilcox (Oracle  2023-01-11  601) 	VMCOREINFO_OFFSET(folio, _folio_order);
> 692f66f26a4c19d Hari Bathini            2017-05-08  602  	VMCOREINFO_OFFSET(page, compound_head);
> 692f66f26a4c19d Hari Bathini            2017-05-08  603  	VMCOREINFO_OFFSET(pglist_data, node_zones);
> 692f66f26a4c19d Hari Bathini            2017-05-08  604  	VMCOREINFO_OFFSET(pglist_data, nr_zones);
> 43b02ba93b25b1c Mike Rapoport           2021-06-28  605  #ifdef CONFIG_FLATMEM
> 692f66f26a4c19d Hari Bathini            2017-05-08 @606  	VMCOREINFO_OFFSET(pglist_data, node_mem_map);
> 692f66f26a4c19d Hari Bathini            2017-05-08  607  #endif
> 692f66f26a4c19d Hari Bathini            2017-05-08  608  	VMCOREINFO_OFFSET(pglist_data, node_start_pfn);
> 692f66f26a4c19d Hari Bathini            2017-05-08  609  	VMCOREINFO_OFFSET(pglist_data, node_spanned_pages);
> 692f66f26a4c19d Hari Bathini            2017-05-08  610  	VMCOREINFO_OFFSET(pglist_data, node_id);
> 692f66f26a4c19d Hari Bathini            2017-05-08  611  	VMCOREINFO_OFFSET(zone, free_area);
> 692f66f26a4c19d Hari Bathini            2017-05-08  612  	VMCOREINFO_OFFSET(zone, vm_stat);
> 692f66f26a4c19d Hari Bathini            2017-05-08  613  	VMCOREINFO_OFFSET(zone, spanned_pages);
> 692f66f26a4c19d Hari Bathini            2017-05-08  614  	VMCOREINFO_OFFSET(free_area, free_list);
> 692f66f26a4c19d Hari Bathini            2017-05-08  615  	VMCOREINFO_OFFSET(list_head, next);
> 692f66f26a4c19d Hari Bathini            2017-05-08  616  	VMCOREINFO_OFFSET(list_head, prev);
> 692f66f26a4c19d Hari Bathini            2017-05-08  617  	VMCOREINFO_OFFSET(vmap_area, va_start);
> 692f66f26a4c19d Hari Bathini            2017-05-08  618  	VMCOREINFO_OFFSET(vmap_area, list);
> 23baf831a32c04f Kirill A. Shutemov      2023-03-15  619  	VMCOREINFO_LENGTH(zone.free_area, MAX_ORDER + 1);
> 692f66f26a4c19d Hari Bathini            2017-05-08  620  	log_buf_vmcoreinfo_setup();
> 692f66f26a4c19d Hari Bathini            2017-05-08  621  	VMCOREINFO_LENGTH(free_area.free_list, MIGRATE_TYPES);
> 692f66f26a4c19d Hari Bathini            2017-05-08  622  	VMCOREINFO_NUMBER(NR_FREE_PAGES);
> 692f66f26a4c19d Hari Bathini            2017-05-08  623  	VMCOREINFO_NUMBER(PG_lru);
> 692f66f26a4c19d Hari Bathini            2017-05-08  624  	VMCOREINFO_NUMBER(PG_private);
> 692f66f26a4c19d Hari Bathini            2017-05-08  625  	VMCOREINFO_NUMBER(PG_swapcache);
> 1cbf29da3628b66 Petr Tesarik            2018-04-13  626  	VMCOREINFO_NUMBER(PG_swapbacked);
> 692f66f26a4c19d Hari Bathini            2017-05-08  627  	VMCOREINFO_NUMBER(PG_slab);
> 692f66f26a4c19d Hari Bathini            2017-05-08  628  #ifdef CONFIG_MEMORY_FAILURE
> 692f66f26a4c19d Hari Bathini            2017-05-08  629  	VMCOREINFO_NUMBER(PG_hwpoison);
> 692f66f26a4c19d Hari Bathini            2017-05-08  630  #endif
> 692f66f26a4c19d Hari Bathini            2017-05-08  631  	VMCOREINFO_NUMBER(PG_head_mask);
> 6e292b9be7f4358 Matthew Wilcox          2018-06-07  632  #define PAGE_BUDDY_MAPCOUNT_VALUE	(~PG_buddy)
> 692f66f26a4c19d Hari Bathini            2017-05-08  633  	VMCOREINFO_NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE);
> 692f66f26a4c19d Hari Bathini            2017-05-08  634  #ifdef CONFIG_HUGETLB_PAGE
> 692f66f26a4c19d Hari Bathini            2017-05-08  635  	VMCOREINFO_NUMBER(HUGETLB_PAGE_DTOR);
> e04b742f74c2362 David Hildenbrand       2019-03-05  636  #define PAGE_OFFLINE_MAPCOUNT_VALUE	(~PG_offline)
> e04b742f74c2362 David Hildenbrand       2019-03-05  637  	VMCOREINFO_NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE);
> 692f66f26a4c19d Hari Bathini            2017-05-08  638  #endif
> 692f66f26a4c19d Hari Bathini            2017-05-08  639  
> 
> -- 
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki
> 



From radhey.shyam.pandey at amd.com  Mon Aug 28 10:39:27 2023
From: radhey.shyam.pandey at amd.com (Pandey, Radhey Shyam)
Date: Mon, 28 Aug 2023 17:39:27 +0000
Subject: kexec reports "Cannot get kernel _text symbol address" on arm64
 platform
In-Reply-To: <CACBYKvT6AzupfrHZ58BYM8xi8TDLeppqD+Lt=rf9hfmSew+nmQ@mail.gmail.com>
References: <MN0PR12MB59538822AA264031D0CAE468B70DA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNL14lnrHvzbpRQu@MiWiFi-R3L-srv>
 <MN0PR12MB59535434E071499DDE439B08B710A@MN0PR12MB5953.namprd12.prod.outlook.com>
 <ZNogKKZCp12Q92jh@MiWiFi-R3L-srv>
 <MN0PR12MB5953FFB655A208BD4FA32EE4B71EA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <CACBYKvRJL_WmZUN7BH_TjxX+TC2O1gjsudkh1GtRyB3WifjMJA@mail.gmail.com>
 <MN0PR12MB59532ABFAF131AC84F36B73FB71CA@MN0PR12MB5953.namprd12.prod.outlook.com>
 <CACBYKvT6AzupfrHZ58BYM8xi8TDLeppqD+Lt=rf9hfmSew+nmQ@mail.gmail.com>
Message-ID: <MN0PR12MB5953557EB8909B66B4D09B28B7E0A@MN0PR12MB5953.namprd12.prod.outlook.com>

> -----Original Message-----
> From: Alexander Kamensky <alexander.kamensky42 at gmail.com>
> Sent: Thursday, August 24, 2023 8:35 PM
> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
> Cc: bhe at redhat.com; piliu at redhat.com; kexec at lists.infradead.org; linux-
> kernel at vger.kernel.org; Sarangi, Anirudha <anirudha.sarangi at amd.com>
> Subject: Re: kexec reports "Cannot get kernel _text symbol address" on
> arm64 platform
> 
> On Wed, Aug 23, 2023 at 12:09?PM Pandey, Radhey Shyam
> <radhey.shyam.pandey at amd.com> wrote:
> >
> > > -----Original Message-----
> > > From: Alexander Kamensky <alexander.kamensky42 at gmail.com>
> > > Sent: Tuesday, August 22, 2023 9:24 AM
> > > To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
> > > Cc: bhe at redhat.com; piliu at redhat.com; kexec at lists.infradead.org;
> > > linux- kernel at vger.kernel.org; Sarangi, Anirudha
> > > <anirudha.sarangi at amd.com>
> > > Subject: Re: kexec reports "Cannot get kernel _text symbol address"
> > > on
> > > arm64 platform
> > >
> > > Hi All,
> > >
> > > Sorry for the top post, but I believe that I might have posted a fix
> > > for this issue a couple days ago.
> > >
> > > Please check and see if it helps
> > > https://lore.kernel.org/kexec/20230819191119.975299-1-
> > > alexander.kamensky42 at gmail.com/T/#u
> > >
> > > Explanation for the issue I observed with a similar secondary kernel
> > > traceback on arm64 is in the commit message.
> >
> > I fetched latest kexec sources
> > (777ca453ca69e46f7376b07ba6727bd261ec97ef)
> > and applied above patch, a bit improved but still vmcore size is huge.
> >
> > / # ls -lrth /proc/vmcore
> > -r--------    1 root     root       15.5G Aug 23 18:55 /proc/vmcore
> >
> How big is your system memory? If it is 16G then the above is normal.

Yes, it's 16G.

> 
> The most important thing is can your secondary kernel read it? For example
> 
> cat /proc/vmcore > /dev/null

This runs fine and I can read /proc/vmcore

> 
> If you want to capture only kernel core out of /proc/vmcore you need to use
> makedumpfile to filter out the rest of the memory.

Thanks for the pointers. I could use makedumpfile to extract dmesg dump 
and it pointed to system Kernel dump correctly also compress dump data.

Further exploring on it and will report how the analysis goes.

xilinx-vck190-20232:/run/media/mmcblk0p1# makedumpfile -c --split -d 3   /proc/vmcore dumpfile1 dumpfile2
The kernel version is not supported.
The makedumpfile operation may be incomplete.
Checking for memory holes                         : [100.0 %] |                  Checking for memory holes                         : [100.0 %] |             
Checking for memory holes                         : [100.0 %] /                  Checking for memory holes                         : [100.0 %] /             
Checking for memory holes                         : [100.0 %] |                  Checking for memory holes                         : [100.0 %] |             
Checking for memory holes                         : [100.0 %] \                  Checking for memory holes                         : [100.0 %] \             
Checking for memory holes                         : [100.0 %] |                  Checking for memory holes                         : [100.0 %] |             
Checking for memory holes                         : [100.0 %] -                  Checking for memory holes                         : [100.0 %] -             
Copying data                                      : [ 45.7 %] -        eta: 6m13s
Copying data                                      : [ 54.1 %] -           eta: 1s

The dumpfiles are saved to dumpfile1, and dumpfile2.

makedumpfile Completed. 

> 
> Thanks,
> Alexander
> 
> > Thanks,
> > Radhey

From graf at amazon.de  Mon Aug 28 13:50:19 2023
From: graf at amazon.de (Alexander Graf)
Date: Mon, 28 Aug 2023 22:50:19 +0200
Subject: [RFC PATCH] Introduce persistent memory pool
In-Reply-To: <20230823024500.GA25462@skinsburskii.>
References: <64e7cbf7.050a0220.114c7.b70dSMTPIN_ADDED_BROKEN@mx.google.com>
 <2023082506-enchanted-tripping-d1d5@gregkh>
 <c26ad989dcc6737dd295e980c78ef53740098810.camel@amazon.com>
 <20230823024500.GA25462@skinsburskii.>
Message-ID: <e0ed9fb9-8e7a-44ad-976a-27362f6e537a@amazon.de>

+kexec, iommu, kvm

On 23.08.23 04:45, Stanislav Kinsburskii wrote:
>
> +akpm, +linux-mm
>
> On Fri, Aug 25, 2023 at 01:32:40PM +0000, Gowans, James wrote:
>> On Fri, 2023-08-25 at 10:05 +0200, Greg Kroah-Hartman wrote:
>>
>> Thanks for adding me to this thread Greg!
>>
>>> On Tue, Aug 22, 2023 at 11:34:34AM -0700, Stanislav Kinsburskii wrote:
>>>> This patch addresses the need for a memory allocator dedicated to
>>>> persistent memory within the kernel. This allocator will preserve
>>>> kernel-specific states like DMA passthrough device states, IOMMU state, and
>>>> more across kexec.
>>>> The proposed solution offers a foundational implementation for potential
>>>> custom solutions that might follow. Though the implementation is
>>>> intentionally kept concise and straightforward to foster discussion and
>>>> feedback, it's fully functional in its current state.
>> Hi Stanislav, it looks like we're working on similar things. I'm looking
>> to develop a mechanism to support hypervisor live update for when KVM is
>> running VMs with PCI device passthrough. VMs with device passthrough
>> also necessitates passing and re-hydrating IOMMU state so that DMA can
>> continue during live update.
>>
>> Planning on having an LPC session on this topic:
>> https://lpc.events/event/17/abstracts/1629/ (currently it's only a
>> submitted abstract so not sure if visible, hopefully it will be soon).
>>
>> We are looking at implementing persistence across kexec via an in-memory
>> filesystem on top of reserved memory. This would have files for anything
>> that needs to be persisted. That includes files for IOMMU pgtables, for
>> guest memory or userspace-accessible memory.
>>
>> It may be nice to solve all kexec persistence requirements with one
>> solution, but we can consider IOMMU separately. There are at least three
>> ways that this can be done:
>> a) carving out reserved memory for pgtables. This is done by your
>> proposal here, as well as my suggestion of a filesystem.
>> b) pre/post kexec hooks for drivers to serialise state and pass it
>> across in a structured format from old to new kernel.
>> c) Reconstructing IOMMU state in the new kernel by starting at the
>> hardware registers and walking the page tables. No state passing needed.
>>
>> Have you considered option (b) and (c) here? One of the implications of
>> (b) and (c) are that they would need to hook into the buddy allocator
>> really early to be able to carve out the reconstructed page tables
>> before the allocator is used. Similar to how pkram [0] hooks in early to
>> carve out pages used for its filesystem.
>>
> Hi James,
>
> We are indeed working on similar things, so thanks for chiming in.
> I've seen pkram proposal as well as your comments there.
>
> I think (b) will need some persistent-over-kexec memory to pass the
> state across kexec as well as some key-value store persisted as well.
> And the proposed persistent memory pool is aimed exactly for this
> purpose.
> Or do you imply some other way to pass driver's data accross kexec?


If I had to build this, I'd probably do it just like device tree passing 
on ARM. It's a single, physically contiguous blob of data whose entry 
point you pass to the target kernel. IIRC ACPI passing works similarly. 
This would just be one more opaque data structure that then needs very 
strict versioning and forward/backward compat guarantees.


> I dind't consider (c) yet, thanks for for the pointer.
>
> I have a question in this scope: how is PCI devices registers state is persisted
> across kexec with the files system you are working on? I.e. how does
> driver know, that the device shouldn't not be reinitialized?


The easiest way to do it initially would be kernel command line options 
that hack up the drivers. But I suppose depending on the option we go 
with, you can also use the respective "natural" path:

(a) A special metadata file that explains the state to the driver
(b) An entry in the structured file format that explains the state to 
the target driver
(c) Compatible target drivers try to enumerate state from the target 
device's register file


>
>>>> Potential applications include:
>>>>
>>>>    1. Allowing various in-kernel entities to allocate persistent pages from
>>>>       a singular memory pool, eliminating the need for multiple region
>>>>       reservations.
>>>>
>>>>    2. For in-kernel components that require the allocation address to be
>>>>       available on kernel kexec, this address can be exposed to user space and
>>>>       then passed via the command line.
>> Do you have specific examples of other state that needs to be passed
>> across? Trying to see whether tailoring specifically to the IOMMU case
>> is okay. Conceptually IOMMU state can be reconstructed starting with
>> hardware registers, not needing reserved memory. Other use-cases may not
>> have this option.
>>
> Well, basically it's IOMMU state and PCI devices to skip/avoid
> initializing.
> I bet there can be other misc (and unrelated things) like persistent
> filesystems, block devices, etc. But I don't have a solid set of use
> cases to present.


Would be great if you could think through the problem space until LPC so 
we can have a solid conversation there :)


>
>>> As you have no in-kernel users of this, it's not something we can even
>>> consider at the moment for obvious reasons (neither would you want us
>>> to.)
>>>
>>> Can you make this part of a patch series that actually adds a user,
>>> probably more than one, so that we can see if any of this even makes
>>> sense?
>> I'm very keen to see this as well. The way that the IOMMU drivers are
>> enlightened to hook into your memory pool will likely be similar to how
>> they would hook into my proposal of an in-memory filesystem.
>> Do you have code available showing the IOMMU integration?
>>
> No, I don't have such a code yet.
> But I was thinking that using such a allocator in the mempool allows
> to hide this implementation under the hood of an existent generic
> mechanism, which is then can be used to create persistent objects (file
> system, for example) on top of it.


Unfortunately it's practically impossible to have a solid conversation 
on generic mechanisms without actual users to see how they fit in with 
the real world. That's Greg's answer to your patch set and I tend to 
agree. What if (b) or (c) turn out much more viable? Then we've wasted a 
lot of effort in shaping up the allocator for no good reason.


>
>>>>   drivers/misc/Kconfig   |    7 +
>>>>   drivers/misc/Makefile  |    1
>>>>   drivers/misc/pmpool.c  |  270 ++++++++++++++++++++++++++++++++++++++++++++++++
>>>>   include/linux/pmpool.h |   20 ++++
>>>>   4 files changed, 298 insertions(+)
>>>>   create mode 100644 drivers/misc/pmpool.c
>>>>   create mode 100644 include/linux/pmpool.h
>>> misc is not for memory pools, as this is not a driver.  please put this
>>> in the properly location instead of trying to hide it from the mm
>>> maintainers and subsystem :)
>> One of the reasons I thought a proper filesystem would be a better way
>> of exposing this functionality.
>>
> Yes, I see the point of having a file system for the goals you are
> targeting. It looks like the right way forward.
>
> What I'm not sure about is that persistent-over-kexec memory management and
> the actual preservation mechanism should be an embedded part of this file
> system.
>
> What I'm trying to propose is a part of a generic mechanism to provide
> state persistence across kexec, which can then be used by your file
> system or something else.
>
> What do you think about this approach?


IMHO we need to at least prototype each of the paths outlined above to 
be able to create a sense for what works the best. I can see problems 
with all of them - and I'm personally not convinced there will be a 
one-size-fits-all solution yet.

That said, I am happy to see you pursue similar paths to what we have in 
mind. It means there is a real gap in functionality in Linux kexec that 
we need to overcome sooner or later.


Alex





Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879



From piliu at redhat.com  Mon Aug 28 19:35:24 2023
From: piliu at redhat.com (Pingfan Liu)
Date: Tue, 29 Aug 2023 10:35:24 +0800
Subject: [PATCH] aarch64: kern_paddr_start calculation makes more accurate
 legacy check
In-Reply-To: <CACBYKvSs0P2MAfx-oeaRKi_V2m8iKFd_Ko7wBD6C3AQUtUUaYQ@mail.gmail.com>
References: <20230819191119.975299-1-alexander.kamensky42@gmail.com>
 <CAF+s44THuTneZr3ny1_FeB8b1+rytySadU4HMLEthnfxzF_F1A@mail.gmail.com> <CACBYKvSs0P2MAfx-oeaRKi_V2m8iKFd_Ko7wBD6C3AQUtUUaYQ@mail.gmail.com>
Message-ID: <CAF+s44SZ4sU74xUqURcEaZj9YQoNvEe-B_u5-Pkd+b-ue2=6xg@mail.gmail.com>

On Sun, Aug 27, 2023 at 4:23?AM Alexander Kamensky
<alexander.kamensky42 at gmail.com> wrote:
>
> Hi Pingfan,
>
> Please see inline
>
> On Thu, Aug 24, 2023 at 11:17?PM Pingfan Liu <piliu at redhat.com> wrote:
> >
> > On Sun, Aug 20, 2023 at 3:16?AM Alexander Kamensky
> > <alexander.kamensky42 at gmail.com> wrote:
> > >
> > > In the latest openembedded with aarch64 image that uses 6.4.3 kernel on qemu I
> > > tried to run makedumpfile in secondary kernel with /proc/vmcore. It failed as
> > > follows:
> > >
> > > > root at qemuarm64:~# makedumpfile -c -F /proc/vmcore > /dev/null
> > > > read_from_vmcore: Can't seek the dump memory(/proc/vmcore). (offset: ffffffc0123fa000) Invalid argument
> > > > readpage_elf: Can't read the dump memory(/proc/vmcore).
> > > > <snip>
> > >
> > > It turns out that not all parts for /proc/vmcore are readable:
> > >
> > > > root at qemuarm64:~# cat /proc/vmcore > /dev/null
> > > > [  136.394931] Internal error: synchronous external abort: 0000000096000010 [#1] PREEMPT SMP
> > > > [  136.422434] Modules linked in:
> > > > <snip>
> > >
> > > Binary search of different kernel versions I found that the issue goes back to
> > > 5.11. It works fine with 5.10 version of the kernel.
> > >
> > > After running secondary kernel under qemu gdb, I've found that the primary kernel
> > > main memory segment in elfcorehdr has wrong address and size.
> > >
> > > Looking at kexec output with debug enabled at the time when it loads secondary
> > > crash kernel I saw the following:
> > >
> > > > Kernel text Elf header: p_type = 1, p_offset = 0x4030200000 p_paddr = 0x4030200000 p_vaddr = 0xffffffffffffffff p_filesz = 0xffffffc011410000 p_memsz = 0xffffffc011410000
> > >
> > > These values come from elf_info variable and are set in iomem_range_callback
> > > function. The function gets the following input on my system:
> > >
> > > root at qemuarm64:~# cat /proc/iomem | grep "Kernel code"
> > >   40210000-40feffff : Kernel code
> > > root at qemuarm64:~# cat /proc/kallsyms | grep -e ' _text$'
> >
> > This is the crux of the problem. Could you say something about how it
> > disappeared?
>
> Here is what I found:
>
> * in kallsyms .tmp_vmlinux.kallsyms1.syms _text I have this:
>
> ffffffc008000000 T _text
> ffffffc008010000 T __irqentry_text_start
> ffffffc008010000 T _stext
>
> * in scripts/kallsyms.c ranges defined as
>
> static struct addr_range text_ranges[] = {
>     { "_stext",     "_etext"     },
>     { "_sinittext", "_einittext" },
> };
>
> * as a result _text fails check_symbol_range function its not included in
> kallsyms
>
> * from 'gdb vmlinux'
>
> (gdb) p _text
> $1 = 0xffffffc008000000 <_text> "MZ@\372'd>\024"
> (gdb) p _stext
> $2 = 0xffffffc008010000 <_stext> "\037 \003\325\037 \003\325\037
> \003\325\037 \003\325?#\003\325\375{\273\251\375\003"
>
> * from kernel linker script ./arch/arm64/kernel/vmlinux.lds, note
> _stext follows _text and aligned by 0x00010000
>
> OUTPUT_ARCH(aarch64)
> ENTRY(_text)
> jiffies = jiffies_64;
> PECOFF_FILE_ALIGNMENT = 0x200;
> SECTIONS
> {
>  /DISCARD/ : { *(.exitcall.exit) *(.discard) *(.discard.*) *(.modinfo)
> *(.gnu.version*) }
>  /DISCARD/ : {
>   *(.interp .dynamic)
>   *(.dynsym .dynstr .hash .gnu.hash)
>  }
>  . = ((((-(((1)) << ((((39))) - 1)))) + (0x08000000)));
>  .head.text : {
>   _text = .; // <------------------------------------------------
>   KEEP(*(.head.text))
>  }
>  .text : ALIGN(0x00010000) { // <--------------------------------
>   _stext = .; // <-----------------------------------------------
>    . = ALIGN(8); __irqentry_text_start = .; *(.irqentry.text)
> __irqentry_text_end = .;
>    . = ALIGN(8); __softirqentry_text_start = .; *(.softirqentry.text)
> __softirqentry_text_end = .;
>
> * ./arch/arm64/kernel/vmlinux.lds is preproccessed file from
> arch/arm64/kernel/vmlinux.lds.S, here are relevant snippets:
>
>     .head.text : {
>         _text = .;
>         HEAD_TEXT
>     }
>     .text : ALIGN(SEGMENT_ALIGN) {    /* Real text segment        */
>         _stext = .;        /* Text and read-only data    */
>             IRQENTRY_TEXT
>             SOFTIRQENTRY_TEXT
>
> * and SEGMENT_ALIGN defined in
>
> arch/arm64/include/asm/memory.h:#define SEGMENT_ALIGN           SZ_64K
>
> * symbol_valid function from scripts/kallsyms.c will ignore range
> check if --all-symbols option is passed
>
> * from scripts/link-vmlinux.sh one can see that --all-symbols will be
> added only if CONFIG_KALLSYMS_ALL is enabled
>
> kallsyms()
> {
>         local kallsymopt;
>
>         if is_enabled CONFIG_KALLSYMS_ALL; then
>                 kallsymopt="${kallsymopt} --all-symbols"
>         fi
>
> * in my kernel CONFIG_KALLSYMS_ALL is not set
>
> * I've checked aarch64 fedora 38 kernel config in there
> CONFIG_KALLSYMS_ALL is enabled that explains why you see this _text in
> kallsyms of fedora
>
> * so far it seems that _text address cannot be part of kallsyms unless
> CONFIG_KALLSYMS_ALL is enabled or original _text is 64k aligned so it
> will coincide with _stext
>
> * in all my testing even with 5.10 kernel _text was missing in
> kallsyms, that it why I thought it presence constituted legacy case,
> in 5.10 case last it worked for me length check was failing and code
> was taking else part using _stext anyway
>
> * I believe my patch is correct and it should work for both cases when
> CONFIG_KALLSYMS_ALL is set or not.
>

Now, the crux is clear. It is introduced by the filtering of
CONFIG_KALLSYMS_ALL. And the kernel layout has not changed.

> * Maybe we can remove whole 'kva_text_end - kva_stext == length' check
> and set 'elf_info.kern_paddr_start = base' unconditionally, but I am
> concerned that some older kernels will trip on it
>

Yes, compatibility is a challenge, that is why the code looks like this.

Furthermore, since the kernel layout does not change, the code logic
stands. And I think you should consider about changing
"elf_info.kern_vaddr_start = get_kernel_sym("_text");"  to ("_stext").

Finally you can unify the changes there and the following code snippet.

Thanks,

Pingfan

>     if (kva_text_end - kva_stext == length)
>         elf_info.kern_paddr_start = base - (kva_stext - kva_text);
>     else
>         elf_info.kern_paddr_start = base;
>
> Thanks,
> Alexander Kamensky
>
> > On my side, I can see this info on 6.2.14-300.fc38.aarch64
> >
> > grep -w _text /proc/kallsyms
> > ffffc705d8500000 T _text
> >
> > Thanks,
> >
> > Pingfan
> > > root at qemuarm64:~# cat /proc/kallsyms | grep -e ' _stext$'
> > > ffffffc010010000 T _stext
> > > root at qemuarm64:~# cat /proc/kallsyms | grep -e ' __init_begin$'
> > > ffffffc010df0000 T __init_begin
> > >
> > > the function calculates elf_info.kern_paddr_start address similar to
> > > these manual steps:
> > >
> > > base = 0x40210000
> > > length = 0x40feffff - 0x40210000 + 0x1 = 0xde0000
> > > kva_text_end - kva_stext = 0xffffffc010df0000 - 0xffffffc010010000 = 0xde0000
> > >
> > > elf_info.kern_paddr_start = base - (kva_stext - kva_text) =
> > >                       0x40210000 - 0xffffffc010010000 = 0x4030200000
> > >
> > > since 'length' and 'kva_text_end - kva_stext' are equal the function calculate
> > > elf_info.kern_paddr_start as in legacy case:
> > >
> > >   if (kva_text_end - kva_stext == length)
> > >      elf_info.kern_paddr_start = base - (kva_stext - kva_text); // <---------
> > >   else
> > >      elf_info.kern_paddr_start = base;
> > >
> > > but my kernel is not legacy and kva_text is zero.
> > >
> > > The fix is just to add a check for kva_text not being zero before following the
> > > legacy case.
> > >
> > > Signed-off-by: Alexander Kamensky <alexander.kamensky42 at gmail.com>
> > > ---
> > >  kexec/arch/arm64/crashdump-arm64.c | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/kexec/arch/arm64/crashdump-arm64.c b/kexec/arch/arm64/crashdump-arm64.c
> > > index 3098315..639c82a 100644
> > > --- a/kexec/arch/arm64/crashdump-arm64.c
> > > +++ b/kexec/arch/arm64/crashdump-arm64.c
> > > @@ -82,7 +82,7 @@ static int iomem_range_callback(void *UNUSED(data), int UNUSED(nr),
> > >                  * For compatibility, deduce by comparing the gap "__init_begin - _stext"
> > >                  * and the res size of "Kernel code" in /proc/iomem
> > >                  */
> > > -               if (kva_text_end - kva_stext == length)
> > > +               if ((kva_text_end - kva_stext == length) && (kva_text != 0))
> > >                         elf_info.kern_paddr_start = base - (kva_stext - kva_text);
> > >                 else
> > >                         elf_info.kern_paddr_start = base;
> > > --
> > > 2.41.0
> > >
> > >
> > > _______________________________________________
> > > kexec mailing list
> > > kexec at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/kexec
> > >
> >
>



From chenjiahao16 at huawei.com  Tue Aug 29 02:54:35 2023
From: chenjiahao16 at huawei.com (chenjiahao (C))
Date: Tue, 29 Aug 2023 17:54:35 +0800
Subject: [PATCH -next v9 0/2] support allocating crashkernel above 4G
 explicitly on riscv
In-Reply-To: <20230726175000.2536220-1-chenjiahao16@huawei.com>
References: <20230726175000.2536220-1-chenjiahao16@huawei.com>
Message-ID: <4b4063c2-dfd3-474b-6827-8dbd1257af2d@huawei.com>

Hi folks,

This is a friendly ping. Anyone has additional comments on this patch set?

Best regards,
Jiahao

On 2023/7/27 1:49, Chen Jiahao wrote:
> On riscv, the current crash kernel allocation logic is trying to
> allocate within 32bit addressible memory region by default, if
> failed, try to allocate without 4G restriction.
>
> In need of saving DMA zone memory while allocating a relatively large
> crash kernel region, allocating the reserved memory top down in
> high memory, without overlapping the DMA zone, is a mature solution.
> Hence this patchset introduces the parameter option crashkernel=X,[high,low].
>
> One can reserve the crash kernel from high memory above DMA zone range
> by explicitly passing "crashkernel=X,high"; or reserve a memory range
> below 4G with "crashkernel=X,low". Besides, there are few rules need
> to take notice:
> 1. "crashkernel=X,[high,low]" will be ignored if "crashkernel=size"
>     is specified.
> 2. "crashkernel=X,low" is valid only when "crashkernel=X,high" is passed
>     and there is enough memory to be allocated under 4G.
> 3. When allocating crashkernel above 4G and no "crashkernel=X,low" is
>     specified, a 128M low memory will be allocated automatically for
>     swiotlb bounce buffer.
> See Documentation/admin-guide/kernel-parameters.txt for more information.
>
> To verify loading the crashkernel, adapted kexec-tools is attached below:
> https://github.com/chenjh005/kexec-tools/tree/build-test-riscv-v2
>
> Following test cases have been performed as expected:
> 1) crashkernel=256M                          //low=256M
> 2) crashkernel=1G                            //low=1G
> 3) crashkernel=4G                            //high=4G, low=128M(default)
> 4) crashkernel=4G crashkernel=256M,high      //high=4G, low=128M(default), high is ignored
> 5) crashkernel=4G crashkernel=256M,low       //high=4G, low=128M(default), low is ignored
> 6) crashkernel=4G,high                       //high=4G, low=128M(default)
> 7) crashkernel=256M,low                      //low=0M, invalid
> 8) crashkernel=4G,high crashkernel=256M,low  //high=4G, low=256M
> 9) crashkernel=4G,high crashkernel=4G,low    //high=0M, low=0M, invalid
> 10) crashkernel=512M at 0xd0000000              //low=512M
> 11) crashkernel=1G,high crashkernel=0M,low   //high=1G, low=0M
>
> Changes since [v9]:
> 1. As per Conor's comment, rebase to correct base on riscv/for-next
>     branch. No code logic changed.
>
> Changes since [v8]:
> 1. Rebase to newest mainline head, not modifying any code logic.
>
> Changes since [v7]:
> 1. Minor refactor: move crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE
>     into the !high branch when the first allocation fails. Not changing
>     the result but further align with Arm64 logic, refer to Baoquan's
>     comment.
> 2. Add test case "crashkernel=1G,high crashkernel=0M,low", the result
>     also matches our expectation.
>
> Changes since [v6]:
> 1. Introduce the "high" flag to mark whether "crashkernel=X,high"
>     is passed. Fix the retrying logic between "crashkernel=X,high"
>     case and others when the first allocation attempt fails.
>
> Changes since [v5]:
> 1. Update the crashkernel allocation logic when crashkernel=X,high
>     is specified. In this case, region above 4G will directly get
>     reserved as crashkernel, rather than trying lower 32bit allocation
>     first.
>
> Changes since [v4]:
> 1. Update some imprecise code comments for cmdline parsing.
>
> Changes since [v3]:
> 1. Update to print warning and return explicitly on failure when
>     crashkernel=size at offset is specified. Not changing the result
>     in this case but making the logic more straightforward.
> 2. Some minor cleanup.
>
> Changes since [v2]:
> 1. Update the allocation logic to ensure the high crashkernel
>     region is reserved strictly above dma32_phys_limit.
> 2. Clean up some minor format problems.
>
> Chen Jiahao (2):
>    riscv: kdump: Implement crashkernel=X,[high,low]
>    docs: kdump: Update the crashkernel description for riscv
>
>   .../admin-guide/kernel-parameters.txt         | 15 +--
>   arch/riscv/kernel/setup.c                     |  5 +
>   arch/riscv/mm/init.c                          | 93 +++++++++++++++++--
>   3 files changed, 99 insertions(+), 14 deletions(-)
>


From bhe at redhat.com  Tue Aug 29 05:16:02 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:02 +0800
Subject: [PATCH v2 0/8] kdump: use generic functions to simplify crashkernel reservation in arch
Message-ID: <20230829121610.138107-1-bhe@redhat.com>

In the current arm64, crashkernel=,high support has been finished after
several rounds of posting and careful reviewing. The code in arm64 which
parses crashkernel kernel parameters firstly, then reserve memory can be
a good example for other ARCH to refer to.

Whereas in x86_64, the code mixing crashkernel parameter parsing and
memory reserving is twisted, and looks messy. Refactoring the code to
make it more readable maintainable is necessary.

Here, firstly abstract the crashkernel parameter parsing code into
parse_crashkernel() to make it be able to parse crashkernel=,high|low.
Then abstract the crashkernel memory reserving code into a generic
function reserve_crashkernel_generic(). Finally, in ARCH which
crashkernel=,high support is needed, a simple arch_reserve_crashkernel()
can be added to call above two functions. This can remove the duplicated
implmentation code in each ARCH, like arm64, x86_64.

I only change the arm64 and x86_64 implementation to make use of the
generic functions to simplify code. Risc-v can be done very easily refer
to the steps in arm64 and x86_64.

History:
- RFC patchset:
  https://lore.kernel.org/all/20230619055951.45620-1-bhe at redhat.com/T/#u
  [RFC PATCH 0/4] kdump: add generic functions to simplify crashkernel crashkernel in architecture
  Dave and Philipp commented the old parse_crashkernel_common() and
  parse_crashkernel_generic() are quite confusing. In this formal post,
  I made change to address the concern by unifying all crashkernel
  parsing into parse_crashkernel().

v1->v2:
- Change to add asm/crash_core.h into x86 and arm64 to contain those
  arch specific macro definitions for generic reservaton. This fixes the
  compiling error reported by LKP in v1.
  https://lore.kernel.org/all/202308272150.p3kRkMoF-lkp at intel.com/T/#u
- Fix a log typo in patch 8, thanks to Samuel.

Baoquan He (8):
  crash_core.c: remove unnecessary parameter of function
  crash_core: change the prototype of function parse_crashkernel()
  crash_core: change parse_crashkernel() to support
    crashkernel=,high|low parsing
  crash_core: add generic function to do reservation
  crash_core.h: include <asm/crash_core.h> if generic reservation is
    needed
  x86: kdump: use generic interface to simplify crashkernel reservation
    code
  arm64: kdump: use generic interface to simplify crashkernel
    reservation
  crash_core.c: remove unneeded functions

 arch/arm/kernel/setup.c              |   3 +-
 arch/arm64/Kconfig                   |   3 +
 arch/arm64/include/asm/crash_core.h  |  10 ++
 arch/arm64/mm/init.c                 | 140 ++--------------------
 arch/ia64/kernel/setup.c             |   2 +-
 arch/loongarch/kernel/setup.c        |   4 +-
 arch/mips/kernel/setup.c             |   3 +-
 arch/powerpc/kernel/fadump.c         |   2 +-
 arch/powerpc/kexec/core.c            |   2 +-
 arch/powerpc/mm/nohash/kaslr_booke.c |   2 +-
 arch/riscv/mm/init.c                 |   2 +-
 arch/s390/kernel/setup.c             |   4 +-
 arch/sh/kernel/machine_kexec.c       |   2 +-
 arch/x86/Kconfig                     |   3 +
 arch/x86/include/asm/crash_core.h    |  34 ++++++
 arch/x86/kernel/setup.c              | 143 +++-------------------
 include/linux/crash_core.h           |  38 +++++-
 kernel/crash_core.c                  | 170 +++++++++++++++++++++++----
 18 files changed, 269 insertions(+), 298 deletions(-)
 create mode 100644 arch/arm64/include/asm/crash_core.h
 create mode 100644 arch/x86/include/asm/crash_core.h

-- 
2.41.0



From bhe at redhat.com  Tue Aug 29 05:16:03 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:03 +0800
Subject: [PATCH v2 1/8] crash_core.c: remove unnecessary parameter of function
In-Reply-To: <20230829121610.138107-1-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
Message-ID: <20230829121610.138107-2-bhe@redhat.com>

In all call sites of __parse_crashkernel(), the parameter 'name' is
hardcoded as "crashkernel=". So remove the unnecessary parameter 'name',
add local varibale 'name' inside __parse_crashkernel() instead.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 kernel/crash_core.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 90ce1dfd591c..f27b4e45d410 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -241,11 +241,11 @@ static int __init __parse_crashkernel(char *cmdline,
 			     unsigned long long system_ram,
 			     unsigned long long *crash_size,
 			     unsigned long long *crash_base,
-			     const char *name,
 			     const char *suffix)
 {
 	char	*first_colon, *first_space;
 	char	*ck_cmdline;
+	char	*name = "crashkernel=";
 
 	BUG_ON(!crash_size || !crash_base);
 	*crash_size = 0;
@@ -283,7 +283,7 @@ int __init parse_crashkernel(char *cmdline,
 			     unsigned long long *crash_base)
 {
 	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-					"crashkernel=", NULL);
+				NULL);
 }
 
 int __init parse_crashkernel_high(char *cmdline,
@@ -292,7 +292,7 @@ int __init parse_crashkernel_high(char *cmdline,
 			     unsigned long long *crash_base)
 {
 	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				"crashkernel=", suffix_tbl[SUFFIX_HIGH]);
+				suffix_tbl[SUFFIX_HIGH]);
 }
 
 int __init parse_crashkernel_low(char *cmdline,
@@ -301,7 +301,7 @@ int __init parse_crashkernel_low(char *cmdline,
 			     unsigned long long *crash_base)
 {
 	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				"crashkernel=", suffix_tbl[SUFFIX_LOW]);
+				suffix_tbl[SUFFIX_LOW]);
 }
 
 /*
-- 
2.41.0



From bhe at redhat.com  Tue Aug 29 05:16:04 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:04 +0800
Subject: [PATCH v2 2/8] crash_core: change the prototype of function parse_crashkernel()
In-Reply-To: <20230829121610.138107-1-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
Message-ID: <20230829121610.138107-3-bhe@redhat.com>

Add two parameters 'low_size' and 'high' to function parse_crashkernel(),
later crashkernel=,high|low parsing will be added. Make adjustments in all
call sites of parse_crashkernel() in arch.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 arch/arm/kernel/setup.c              |  3 ++-
 arch/arm64/mm/init.c                 |  2 +-
 arch/ia64/kernel/setup.c             |  2 +-
 arch/loongarch/kernel/setup.c        |  4 +++-
 arch/mips/kernel/setup.c             |  3 ++-
 arch/powerpc/kernel/fadump.c         |  2 +-
 arch/powerpc/kexec/core.c            |  2 +-
 arch/powerpc/mm/nohash/kaslr_booke.c |  2 +-
 arch/riscv/mm/init.c                 |  2 +-
 arch/s390/kernel/setup.c             |  4 ++--
 arch/sh/kernel/machine_kexec.c       |  2 +-
 arch/x86/kernel/setup.c              |  3 ++-
 include/linux/crash_core.h           |  3 ++-
 kernel/crash_core.c                  | 15 ++++++++++++---
 14 files changed, 32 insertions(+), 17 deletions(-)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index c66b560562b3..e2bb7afd0683 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -1010,7 +1010,8 @@ static void __init reserve_crashkernel(void)
 
 	total_mem = get_total_mem();
 	ret = parse_crashkernel(boot_command_line, total_mem,
-				&crash_size, &crash_base);
+				&crash_size, &crash_base,
+				NULL, NULL);
 	/* invalid value specified or crashkernel=0 */
 	if (ret || !crash_size)
 		return;
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 4fcb88a445ef..4ad637508b75 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -142,7 +142,7 @@ static void __init reserve_crashkernel(void)
 
 	/* crashkernel=X[@offset] */
 	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
-				&crash_size, &crash_base);
+				&crash_size, &crash_base, NULL, NULL);
 	if (ret == -ENOENT) {
 		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
 		if (ret || !crash_size)
diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
index 5a55ac82c13a..4faea2d2cf07 100644
--- a/arch/ia64/kernel/setup.c
+++ b/arch/ia64/kernel/setup.c
@@ -277,7 +277,7 @@ static void __init setup_crashkernel(unsigned long total, int *n)
 	int ret;
 
 	ret = parse_crashkernel(boot_command_line, total,
-			&size, &base);
+			&size, &base, NULL, NULL);
 	if (ret == 0 && size > 0) {
 		if (!base) {
 			sort_regions(rsvd_region, *n);
diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c
index 9d830ab4e302..776a068d8718 100644
--- a/arch/loongarch/kernel/setup.c
+++ b/arch/loongarch/kernel/setup.c
@@ -267,7 +267,9 @@ static void __init arch_parse_crashkernel(void)
 	unsigned long long crash_base, crash_size;
 
 	total_mem = memblock_phys_mem_size();
-	ret = parse_crashkernel(boot_command_line, total_mem, &crash_size, &crash_base);
+	ret = parse_crashkernel(boot_command_line, total_mem,
+				&crash_size, &crash_base,
+				NULL, NULL);
 	if (ret < 0 || crash_size <= 0)
 		return;
 
diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
index cb871eb784a7..08321c945ac4 100644
--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -460,7 +460,8 @@ static void __init mips_parse_crashkernel(void)
 
 	total_mem = memblock_phys_mem_size();
 	ret = parse_crashkernel(boot_command_line, total_mem,
-				&crash_size, &crash_base);
+				&crash_size, &crash_base,
+				NULL, NULL);
 	if (ret != 0 || crash_size <= 0)
 		return;
 
diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c
index ea0a073abd96..7dbdeba56e74 100644
--- a/arch/powerpc/kernel/fadump.c
+++ b/arch/powerpc/kernel/fadump.c
@@ -313,7 +313,7 @@ static __init u64 fadump_calculate_reserve_size(void)
 	 * memory at a predefined offset.
 	 */
 	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
-				&size, &base);
+				&size, &base, NULL, NULL);
 	if (ret == 0 && size > 0) {
 		unsigned long max_size;
 
diff --git a/arch/powerpc/kexec/core.c b/arch/powerpc/kexec/core.c
index de64c7962991..9346c960b296 100644
--- a/arch/powerpc/kexec/core.c
+++ b/arch/powerpc/kexec/core.c
@@ -109,7 +109,7 @@ void __init reserve_crashkernel(void)
 	total_mem_sz = memory_limit ? memory_limit : memblock_phys_mem_size();
 	/* use common parsing */
 	ret = parse_crashkernel(boot_command_line, total_mem_sz,
-			&crash_size, &crash_base);
+			&crash_size, &crash_base, NULL, NULL);
 	if (ret == 0 && crash_size > 0) {
 		crashk_res.start = crash_base;
 		crashk_res.end = crash_base + crash_size - 1;
diff --git a/arch/powerpc/mm/nohash/kaslr_booke.c b/arch/powerpc/mm/nohash/kaslr_booke.c
index 2fb3edafe9ab..b4f2786a7d2b 100644
--- a/arch/powerpc/mm/nohash/kaslr_booke.c
+++ b/arch/powerpc/mm/nohash/kaslr_booke.c
@@ -178,7 +178,7 @@ static void __init get_crash_kernel(void *fdt, unsigned long size)
 	int ret;
 
 	ret = parse_crashkernel(boot_command_line, size, &crash_size,
-				&crash_base);
+				&crash_base, NULL, NULL);
 	if (ret != 0 || crash_size == 0)
 		return;
 	if (crash_base == 0)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index e4c35ac2357f..a9ef0824f905 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -1332,7 +1332,7 @@ static void __init reserve_crashkernel(void)
 	}
 
 	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
-				&crash_size, &crash_base);
+				&crash_size, &crash_base, NULL, NULL);
 	if (ret || !crash_size)
 		return;
 
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index c744104e4a9c..98204a5f62b1 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -626,8 +626,8 @@ static void __init reserve_crashkernel(void)
 	phys_addr_t low, high;
 	int rc;
 
-	rc = parse_crashkernel(boot_command_line, ident_map_size, &crash_size,
-			       &crash_base);
+	rc = parse_crashkernel(boot_command_line, ident_map_size,
+			       &crash_size, &crash_base, NULL, NULL);
 
 	crash_base = ALIGN(crash_base, KEXEC_CRASH_MEM_ALIGN);
 	crash_size = ALIGN(crash_size, KEXEC_CRASH_MEM_ALIGN);
diff --git a/arch/sh/kernel/machine_kexec.c b/arch/sh/kernel/machine_kexec.c
index 223c14f44af7..fa3a7b36190a 100644
--- a/arch/sh/kernel/machine_kexec.c
+++ b/arch/sh/kernel/machine_kexec.c
@@ -154,7 +154,7 @@ void __init reserve_crashkernel(void)
 	int ret;
 
 	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
-			&crash_size, &crash_base);
+			&crash_size, &crash_base, NULL, NULL);
 	if (ret == 0 && crash_size > 0) {
 		crashk_res.start = crash_base;
 		crashk_res.end = crash_base + crash_size - 1;
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index fd975a4a5200..382c66d2cf71 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -558,7 +558,8 @@ static void __init reserve_crashkernel(void)
 	total_mem = memblock_phys_mem_size();
 
 	/* crashkernel=XM */
-	ret = parse_crashkernel(boot_command_line, total_mem, &crash_size, &crash_base);
+	ret = parse_crashkernel(boot_command_line, total_mem,
+				&crash_size, &crash_base, NULL, NULL);
 	if (ret != 0 || crash_size <= 0) {
 		/* crashkernel=X,high */
 		ret = parse_crashkernel_high(boot_command_line, total_mem,
diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index de62a722431e..2e76289699ff 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -78,7 +78,8 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 void final_note(Elf_Word *buf);
 
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
-		unsigned long long *crash_size, unsigned long long *crash_base);
+		unsigned long long *crash_size, unsigned long long *crash_base,
+		unsigned long long *low_size, bool *high);
 int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
 int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index f27b4e45d410..f6a5c219e2e1 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -280,10 +280,19 @@ static int __init __parse_crashkernel(char *cmdline,
 int __init parse_crashkernel(char *cmdline,
 			     unsigned long long system_ram,
 			     unsigned long long *crash_size,
-			     unsigned long long *crash_base)
+			     unsigned long long *crash_base,
+			     unsigned long long *low_size,
+			     bool *high)
 {
-	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				NULL);
+	int ret;
+
+	/* crashkernel=X[@offset] */
+	ret = __parse_crashkernel(cmdline, system_ram, crash_size,
+				crash_base, NULL);
+	if (!high)
+		return ret;
+
+	return 0;
 }
 
 int __init parse_crashkernel_high(char *cmdline,
-- 
2.41.0



From bhe at redhat.com  Tue Aug 29 05:16:05 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:05 +0800
Subject: [PATCH v2 3/8] crash_core: change parse_crashkernel() to support crashkernel=,high|low parsing
In-Reply-To: <20230829121610.138107-1-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
Message-ID: <20230829121610.138107-4-bhe@redhat.com>

Now parse_crashkernel() is a real entry point for all kinds of
crahskernel parsing on any architecture.

And wrap the crahskernel=,high|low handling inside
CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION ifdeffery scope.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |  6 ++++++
 kernel/crash_core.c        | 28 +++++++++++++++++++++++++++-
 2 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 2e76289699ff..85260bf4a734 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -77,6 +77,12 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len);
 void final_note(Elf_Word *buf);
 
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
+#define DEFAULT_CRASH_KERNEL_LOW_SIZE  (128UL << 20)
+#endif
+#endif
+
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base,
 		unsigned long long *low_size, bool *high);
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index f6a5c219e2e1..355b0ab5189c 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -276,6 +276,9 @@ static int __init __parse_crashkernel(char *cmdline,
 /*
  * That function is the entry point for command line parsing and should be
  * called from the arch-specific code.
+ *
+ * If crashkernel=,high|low is supported on architecture, non-NULL values
+ * should be passed to parameters 'low_size' and 'high'.
  */
 int __init parse_crashkernel(char *cmdline,
 			     unsigned long long system_ram,
@@ -291,7 +294,30 @@ int __init parse_crashkernel(char *cmdline,
 				crash_base, NULL);
 	if (!high)
 		return ret;
-
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+	else if (ret == -ENOENT) {
+		ret = __parse_crashkernel(cmdline, 0, crash_size,
+				crash_base, suffix_tbl[SUFFIX_HIGH]);
+		if (ret || !*crash_size)
+			return -1;
+
+		/*
+		 * crashkernel=Y,low can be specified or not, but invalid value
+		 * is not allowed.
+		 */
+		ret = __parse_crashkernel(cmdline, 0, low_size,
+				crash_base, suffix_tbl[SUFFIX_LOW]);
+		if (ret == -ENOENT)
+			*low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
+		else if (ret)
+			return -1;
+
+		*high = true;
+	} else if (ret || !*crash_size) {
+		/* The specified value is invalid */
+		return -1;
+	}
+#endif
 	return 0;
 }
 
-- 
2.41.0



From bhe at redhat.com  Tue Aug 29 05:16:06 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:06 +0800
Subject: [PATCH v2 4/8] crash_core: add generic function to do reservation
In-Reply-To: <20230829121610.138107-1-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
Message-ID: <20230829121610.138107-5-bhe@redhat.com>

In architecture like x86_64, arm64 and riscv, they have vast virtual
address space and usually have huge physical memory RAM. Their
crashkernel reservation doesn't have to be limited under 4G RAM,
but can be extended to the whole physical memory via crashkernel=,high
support.

Now add function reserve_crashkernel_generic() to reserve crashkernel
memory if users specify any case of kernel pamameters, like
crashkernel=xM[@offset] or crashkernel=,high|low.

This is preparation to simplify code of crashkernel=,high support
in architecutures.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |  34 ++++++++++--
 kernel/crash_core.c        | 109 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 136 insertions(+), 7 deletions(-)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 85260bf4a734..2f732493e922 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -77,12 +77,6 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len);
 void final_note(Elf_Word *buf);
 
-#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
-#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
-#define DEFAULT_CRASH_KERNEL_LOW_SIZE  (128UL << 20)
-#endif
-#endif
-
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base,
 		unsigned long long *low_size, bool *high);
@@ -91,4 +85,32 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
 int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
 
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
+#define DEFAULT_CRASH_KERNEL_LOW_SIZE	(128UL << 20)
+#endif
+#ifndef CRASH_ALIGN
+#define CRASH_ALIGN			SZ_2M
+#endif
+#ifndef CRASH_ADDR_LOW_MAX
+#define CRASH_ADDR_LOW_MAX		SZ_4G
+#endif
+#ifndef CRASH_ADDR_HIGH_MAX
+#define CRASH_ADDR_HIGH_MAX		memblock_end_of_DRAM()
+#endif
+
+void __init reserve_crashkernel_generic(char *cmdline,
+		unsigned long long crash_size,
+		unsigned long long crash_base,
+		unsigned long long crash_low_size,
+		bool high);
+#else
+static inline void __init reserve_crashkernel_generic(char *cmdline,
+		unsigned long long crash_size,
+		unsigned long long crash_base,
+		unsigned long long crash_low_size,
+		bool high)
+{}
+#endif
+
 #endif /* LINUX_CRASH_CORE_H */
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 355b0ab5189c..6bc00cc390b5 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -5,11 +5,13 @@
  */
 
 #include <linux/buildid.h>
-#include <linux/crash_core.h>
 #include <linux/init.h>
 #include <linux/utsname.h>
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
+#include <linux/memblock.h>
+#include <linux/kexec.h>
+#include <linux/kmemleak.h>
 
 #include <asm/page.h>
 #include <asm/sections.h>
@@ -349,6 +351,111 @@ static int __init parse_crashkernel_dummy(char *arg)
 }
 early_param("crashkernel", parse_crashkernel_dummy);
 
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+static int __init reserve_crashkernel_low(unsigned long long low_size)
+{
+#ifdef CONFIG_64BIT
+	unsigned long long low_base;
+
+	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
+	if (!low_base) {
+		pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
+		return -ENOMEM;
+	}
+
+	pr_info("crashkernel low memory reserved: 0x%08llx - 0x%08llx (%lld MB)\n",
+		low_base, low_base + low_size, low_size >> 20);
+
+	crashk_low_res.start = low_base;
+	crashk_low_res.end   = low_base + low_size - 1;
+	insert_resource(&iomem_resource, &crashk_low_res);
+#endif
+	return 0;
+}
+
+void __init reserve_crashkernel_generic(char *cmdline,
+			     unsigned long long crash_size,
+			     unsigned long long crash_base,
+			     unsigned long long crash_low_size,
+			     bool high)
+{
+	unsigned long long search_end = CRASH_ADDR_LOW_MAX, search_base = 0;
+	bool fixed_base = false;
+
+	/* User specifies base address explicitly. */
+	if (crash_base) {
+		fixed_base = true;
+		search_base = crash_base;
+		search_end = crash_base + crash_size;
+	}
+
+	if (high) {
+		search_base = CRASH_ADDR_LOW_MAX;
+		search_end = CRASH_ADDR_HIGH_MAX;
+	}
+
+retry:
+	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
+					       search_base, search_end);
+	if (!crash_base) {
+		/*
+		 * For crashkernel=size[KMG]@offset[KMG], print out failure
+		 * message if can't reserve the specified region.
+		 */
+		if (fixed_base) {
+			pr_warn("crashkernel reservation failed - memory is in use.\n");
+			return;
+		}
+
+		/*
+		 * For crashkernel=size[KMG], if the first attempt was for
+		 * low memory, fall back to high memory, the minimum required
+		 * low memory will be reserved later.
+		 */
+		if (!high && search_end == CRASH_ADDR_LOW_MAX) {
+			search_end = CRASH_ADDR_HIGH_MAX;
+			search_base = CRASH_ADDR_LOW_MAX;
+			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
+			goto retry;
+		}
+
+		/*
+		 * For crashkernel=size[KMG],high, if the first attempt was
+		 * for high memory, fall back to low memory.
+		 */
+		if (high && search_end == CRASH_ADDR_HIGH_MAX) {
+			search_end = CRASH_ADDR_LOW_MAX;
+			search_base = 0;
+			goto retry;
+		}
+		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
+			crash_size);
+		return;
+	}
+
+	if ((crash_base > CRASH_ADDR_LOW_MAX) &&
+	     crash_low_size && reserve_crashkernel_low(crash_low_size)) {
+		memblock_phys_free(crash_base, crash_size);
+		return;
+	}
+
+	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
+		crash_base, crash_base + crash_size, crash_size >> 20);
+
+	/*
+	 * The crashkernel memory will be removed from the kernel linear
+	 * map. Inform kmemleak so that it won't try to access it.
+	 */
+	kmemleak_ignore_phys(crash_base);
+	if (crashk_low_res.end)
+		kmemleak_ignore_phys(crashk_low_res.start);
+
+	crashk_res.start = crash_base;
+	crashk_res.end = crash_base + crash_size - 1;
+	insert_resource(&iomem_resource, &crashk_res);
+}
+#endif
+
 Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
 			  void *data, size_t data_len)
 {
-- 
2.41.0



From bhe at redhat.com  Tue Aug 29 05:16:07 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:07 +0800
Subject: [PATCH v2 5/8] crash_core.h: include <asm/crash_core.h> if generic reservation is needed
In-Reply-To: <20230829121610.138107-1-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
Message-ID: <20230829121610.138107-6-bhe@redhat.com>

In asm/crash_core.h, ARCH can provide its own macro definitions to
override DEFAULT_CRASH_KERNEL_LOW_SIZE, CRASH_ALIGN, CRASH_ADDR_LOW_MAX,
CRASH_ADDR_HIGH_MAX in <linux/crash_core.h> if needed.

Later, x86 and arm64 wil support the generic reservaton, so wrap the
including into CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION ifdeffery
scope to avoid compiling error in other ARCH-es which doesn't take the
generic reservation.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 2f732493e922..86e22e6a039f 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -5,6 +5,9 @@
 #include <linux/linkage.h>
 #include <linux/elfcore.h>
 #include <linux/elf.h>
+#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+#include <asm/crash_core.h>
+#endif
 
 #define CRASH_CORE_NOTE_NAME	   "CORE"
 #define CRASH_CORE_NOTE_HEAD_BYTES ALIGN(sizeof(struct elf_note), 4)
-- 
2.41.0



From bhe at redhat.com  Tue Aug 29 05:16:08 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:08 +0800
Subject: [PATCH v2 6/8] x86: kdump: use generic interface to simplify crashkernel reservation code
In-Reply-To: <20230829121610.138107-1-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
Message-ID: <20230829121610.138107-7-bhe@redhat.com>

With the help of newly changed function parse_crashkernel() and
generic reserve_crashkernel_generic(), crashkernel reservation can be
simplified by steps:

1) Add a new header file <asm/crash_core.h>, and define CRASH_ALIGN,
   CRASH_ADDR_LOW_MAX, CRASH_ADDR_HIGH_MAX and
   DEFAULT_CRASH_KERNEL_LOW_SIZE in <asm/crash_core.h>;

2) Add arch_reserve_crashkernel() to call parse_crashkernel() and
   reserve_crashkernel_generic(), and do the ARCH specific work if
   needed.

3) Add ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION Kconfig in
   arch/x86/Kconfig.

When adding DEFAULT_CRASH_KERNEL_LOW_SIZE, add crash_low_size_default()
to calculate crashkernel low memory because x86_64 has special
requirement.

The old reserve_crashkernel_low() and reserve_crashkernel() can be
removed.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 arch/x86/Kconfig                  |   3 +
 arch/x86/include/asm/crash_core.h |  34 +++++++
 arch/x86/kernel/setup.c           | 144 ++++--------------------------
 3 files changed, 53 insertions(+), 128 deletions(-)
 create mode 100644 arch/x86/include/asm/crash_core.h

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 8d9e4b362572..c4539dc35985 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2037,6 +2037,9 @@ config KEXEC_FILE
 config ARCH_HAS_KEXEC_PURGATORY
 	def_bool KEXEC_FILE
 
+config ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+	def_bool CRASH_CORE
+
 config KEXEC_SIG
 	bool "Verify kernel signature during kexec_file_load() syscall"
 	depends on KEXEC_FILE
diff --git a/arch/x86/include/asm/crash_core.h b/arch/x86/include/asm/crash_core.h
new file mode 100644
index 000000000000..5fc5e4f94521
--- /dev/null
+++ b/arch/x86/include/asm/crash_core.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _X86_CRASH_CORE_H
+#define _X86_CRASH_CORE_H
+
+/* 16M alignment for crash kernel regions */
+#define CRASH_ALIGN             SZ_16M
+
+/*
+ * Keep the crash kernel below this limit.
+ *
+ * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
+ * due to mapping restrictions.
+ *
+ * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
+ * the upper limit of system RAM in 4-level paging mode. Since the kdump
+ * jump could be from 5-level paging to 4-level paging, the jump will fail if
+ * the kernel is put above 64 TB, and during the 1st kernel bootup there's
+ * no good way to detect the paging mode of the target kernel which will be
+ * loaded for dumping.
+ */
+
+#ifdef CONFIG_X86_32
+# define CRASH_ADDR_LOW_MAX     SZ_512M
+# define CRASH_ADDR_HIGH_MAX    SZ_512M
+#else
+# define CRASH_ADDR_LOW_MAX     SZ_4G
+# define CRASH_ADDR_HIGH_MAX    SZ_64T
+#endif
+
+# define DEFAULT_CRASH_KERNEL_LOW_SIZE crash_low_size_default()
+
+extern unsigned long crash_low_size_default(void);
+
+#endif /* _X86_CRASH_CORE_H */
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 382c66d2cf71..559a5c4141db 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -474,152 +474,40 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 /*
  * --------- Crashkernel reservation ------------------------------
  */
-
-/* 16M alignment for crash kernel regions */
-#define CRASH_ALIGN		SZ_16M
-
-/*
- * Keep the crash kernel below this limit.
- *
- * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
- * due to mapping restrictions.
- *
- * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
- * the upper limit of system RAM in 4-level paging mode. Since the kdump
- * jump could be from 5-level paging to 4-level paging, the jump will fail if
- * the kernel is put above 64 TB, and during the 1st kernel bootup there's
- * no good way to detect the paging mode of the target kernel which will be
- * loaded for dumping.
- */
-#ifdef CONFIG_X86_32
-# define CRASH_ADDR_LOW_MAX	SZ_512M
-# define CRASH_ADDR_HIGH_MAX	SZ_512M
-#else
-# define CRASH_ADDR_LOW_MAX	SZ_4G
-# define CRASH_ADDR_HIGH_MAX	SZ_64T
-#endif
-
-static int __init reserve_crashkernel_low(void)
+unsigned long crash_low_size_default(void)
 {
 #ifdef CONFIG_X86_64
-	unsigned long long base, low_base = 0, low_size = 0;
-	unsigned long low_mem_limit;
-	int ret;
-
-	low_mem_limit = min(memblock_phys_mem_size(), CRASH_ADDR_LOW_MAX);
-
-	/* crashkernel=Y,low */
-	ret = parse_crashkernel_low(boot_command_line, low_mem_limit, &low_size, &base);
-	if (ret) {
-		/*
-		 * two parts from kernel/dma/swiotlb.c:
-		 * -swiotlb size: user-specified with swiotlb= or default.
-		 *
-		 * -swiotlb overflow buffer: now hardcoded to 32k. We round it
-		 * to 8M for other buffers that may need to stay low too. Also
-		 * make sure we allocate enough extra low memory so that we
-		 * don't run out of DMA buffers for 32-bit devices.
-		 */
-		low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);
-	} else {
-		/* passed with crashkernel=0,low ? */
-		if (!low_size)
-			return 0;
-	}
-
-	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
-	if (!low_base) {
-		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
-		       (unsigned long)(low_size >> 20));
-		return -ENOMEM;
-	}
-
-	pr_info("Reserving %ldMB of low memory at %ldMB for crashkernel (low RAM limit: %ldMB)\n",
-		(unsigned long)(low_size >> 20),
-		(unsigned long)(low_base >> 20),
-		(unsigned long)(low_mem_limit >> 20));
-
-	crashk_low_res.start = low_base;
-	crashk_low_res.end   = low_base + low_size - 1;
-	insert_resource(&iomem_resource, &crashk_low_res);
-#endif
+	return max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);
+#else
 	return 0;
+#endif
 }
 
-static void __init reserve_crashkernel(void)
+static void __init arch_reserve_crashkernel(void)
 {
-	unsigned long long crash_size, crash_base, total_mem;
+	unsigned long long crash_base, crash_size, low_size = 0;
+	char *cmdline = boot_command_line;
 	bool high = false;
 	int ret;
 
 	if (!IS_ENABLED(CONFIG_KEXEC_CORE))
 		return;
 
-	total_mem = memblock_phys_mem_size();
-
-	/* crashkernel=XM */
-	ret = parse_crashkernel(boot_command_line, total_mem,
-				&crash_size, &crash_base, NULL, NULL);
-	if (ret != 0 || crash_size <= 0) {
-		/* crashkernel=X,high */
-		ret = parse_crashkernel_high(boot_command_line, total_mem,
-					     &crash_size, &crash_base);
-		if (ret != 0 || crash_size <= 0)
-			return;
-		high = true;
-	}
+	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
+				&crash_size, &crash_base,
+				&low_size, &high);
+	if (ret)
+		return;
 
 	if (xen_pv_domain()) {
 		pr_info("Ignoring crashkernel for a Xen PV domain\n");
 		return;
 	}
 
-	/* 0 means: find the address automatically */
-	if (!crash_base) {
-		/*
-		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * crashkernel=x,high reserves memory over 4G, also allocates
-		 * 256M extra low memory for DMA buffers and swiotlb.
-		 * But the extra memory is not required for all machines.
-		 * So try low memory first and fall back to high memory
-		 * unless "crashkernel=size[KMG],high" is specified.
-		 */
-		if (!high)
-			crash_base = memblock_phys_alloc_range(crash_size,
-						CRASH_ALIGN, CRASH_ALIGN,
-						CRASH_ADDR_LOW_MAX);
-		if (!crash_base)
-			crash_base = memblock_phys_alloc_range(crash_size,
-						CRASH_ALIGN, CRASH_ALIGN,
-						CRASH_ADDR_HIGH_MAX);
-		if (!crash_base) {
-			pr_info("crashkernel reservation failed - No suitable area found.\n");
-			return;
-		}
-	} else {
-		unsigned long long start;
-
-		start = memblock_phys_alloc_range(crash_size, SZ_1M, crash_base,
-						  crash_base + crash_size);
-		if (start != crash_base) {
-			pr_info("crashkernel reservation failed - memory is in use.\n");
-			return;
-		}
-	}
-
-	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
-		memblock_phys_free(crash_base, crash_size);
-		return;
-	}
-
-	pr_info("Reserving %ldMB of memory at %ldMB for crashkernel (System RAM: %ldMB)\n",
-		(unsigned long)(crash_size >> 20),
-		(unsigned long)(crash_base >> 20),
-		(unsigned long)(total_mem >> 20));
+	reserve_crashkernel_generic(cmdline, crash_size, crash_base,
+				    low_size, high);
 
-	crashk_res.start = crash_base;
-	crashk_res.end   = crash_base + crash_size - 1;
-	insert_resource(&iomem_resource, &crashk_res);
+	return;
 }
 
 static struct resource standard_io_resources[] = {
@@ -1231,7 +1119,7 @@ void __init setup_arch(char **cmdline_p)
 	 * Reserve memory for crash kernel after SRAT is parsed so that it
 	 * won't consume hotpluggable memory.
 	 */
-	reserve_crashkernel();
+	arch_reserve_crashkernel();
 
 	memblock_find_dma_reserve();
 
-- 
2.41.0



From bhe at redhat.com  Tue Aug 29 05:16:10 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:10 +0800
Subject: [PATCH v2 8/8] crash_core.c: remove unneeded functions
In-Reply-To: <20230829121610.138107-1-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
Message-ID: <20230829121610.138107-9-bhe@redhat.com>

So far, nobody calls functions parse_crashkernel_high() and
parse_crashkernel_low(), remove both of them.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |  4 ----
 kernel/crash_core.c        | 18 ------------------
 2 files changed, 22 deletions(-)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index 86e22e6a039f..d64006c4bd43 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -83,10 +83,6 @@ void final_note(Elf_Word *buf);
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base,
 		unsigned long long *low_size, bool *high);
-int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
-		unsigned long long *crash_size, unsigned long long *crash_base);
-int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
-		unsigned long long *crash_size, unsigned long long *crash_base);
 
 #ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
 #ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 6bc00cc390b5..61a8ea3b23a2 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -323,24 +323,6 @@ int __init parse_crashkernel(char *cmdline,
 	return 0;
 }
 
-int __init parse_crashkernel_high(char *cmdline,
-			     unsigned long long system_ram,
-			     unsigned long long *crash_size,
-			     unsigned long long *crash_base)
-{
-	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				suffix_tbl[SUFFIX_HIGH]);
-}
-
-int __init parse_crashkernel_low(char *cmdline,
-			     unsigned long long system_ram,
-			     unsigned long long *crash_size,
-			     unsigned long long *crash_base)
-{
-	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
-				suffix_tbl[SUFFIX_LOW]);
-}
-
 /*
  * Add a dummy early_param handler to mark crashkernel= as a known command line
  * parameter and suppress incorrect warnings in init/main.c.
-- 
2.41.0



From bhe at redhat.com  Tue Aug 29 05:16:09 2023
From: bhe at redhat.com (Baoquan He)
Date: Tue, 29 Aug 2023 20:16:09 +0800
Subject: [PATCH v2 7/8] arm64: kdump: use generic interface to simplify crashkernel reservation
In-Reply-To: <20230829121610.138107-1-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
Message-ID: <20230829121610.138107-8-bhe@redhat.com>

With the help of newly changed function parse_crashkernel() and
generic reserve_crashkernel_generic(), crashkernel reservation can be
simplified by steps:

1) Add a new header file <asm/crash_core.h>, and define CRASH_ALIGN,
   CRASH_ADDR_LOW_MAX, CRASH_ADDR_HIGH_MAX and
   DEFAULT_CRASH_KERNEL_LOW_SIZE in <asm/crash_core.h>;

2) Add arch_reserve_crashkernel() to call parse_crashkernel() and
   reserve_crashkernel_generic();

3) Add ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION Kconfig in
   arch/arm64/Kconfig.

The old reserve_crashkernel_low() and reserve_crashkernel() can be
removed.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 arch/arm64/Kconfig                  |   3 +
 arch/arm64/include/asm/crash_core.h |  10 ++
 arch/arm64/mm/init.c                | 140 ++--------------------------
 3 files changed, 21 insertions(+), 132 deletions(-)
 create mode 100644 arch/arm64/include/asm/crash_core.h

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 29db061db9bb..07fb8c71339d 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1481,6 +1481,9 @@ config KEXEC_FILE
 	  for kernel and initramfs as opposed to list of segments as
 	  accepted by previous system call.
 
+config ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+	def_bool CRASH_CORE
+
 config KEXEC_SIG
 	bool "Verify kernel signature during kexec_file_load() syscall"
 	depends on KEXEC_FILE
diff --git a/arch/arm64/include/asm/crash_core.h b/arch/arm64/include/asm/crash_core.h
new file mode 100644
index 000000000000..9f5c8d339f44
--- /dev/null
+++ b/arch/arm64/include/asm/crash_core.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ARM64_CRASH_CORE_H
+#define _ARM64_CRASH_CORE_H
+
+/* Current arm64 boot protocol requires 2MB alignment */
+#define CRASH_ALIGN                     SZ_2M
+
+#define CRASH_ADDR_LOW_MAX              arm64_dma_phys_limit
+#define CRASH_ADDR_HIGH_MAX             (PHYS_MASK + 1)
+#endif
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 4ad637508b75..48ab23531bb6 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -64,15 +64,6 @@ EXPORT_SYMBOL(memstart_addr);
  */
 phys_addr_t __ro_after_init arm64_dma_phys_limit;
 
-/* Current arm64 boot protocol requires 2MB alignment */
-#define CRASH_ALIGN			SZ_2M
-
-#define CRASH_ADDR_LOW_MAX		arm64_dma_phys_limit
-#define CRASH_ADDR_HIGH_MAX		(PHYS_MASK + 1)
-#define CRASH_HIGH_SEARCH_BASE		SZ_4G
-
-#define DEFAULT_CRASH_KERNEL_LOW_SIZE	(128UL << 20)
-
 /*
  * To make optimal use of block mappings when laying out the linear
  * mapping, round down the base of physical memory to a size that can
@@ -100,140 +91,25 @@ phys_addr_t __ro_after_init arm64_dma_phys_limit;
 #define ARM64_MEMSTART_ALIGN	(1UL << ARM64_MEMSTART_SHIFT)
 #endif
 
-static int __init reserve_crashkernel_low(unsigned long long low_size)
-{
-	unsigned long long low_base;
-
-	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
-	if (!low_base) {
-		pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
-		return -ENOMEM;
-	}
-
-	pr_info("crashkernel low memory reserved: 0x%08llx - 0x%08llx (%lld MB)\n",
-		low_base, low_base + low_size, low_size >> 20);
-
-	crashk_low_res.start = low_base;
-	crashk_low_res.end   = low_base + low_size - 1;
-	insert_resource(&iomem_resource, &crashk_low_res);
-
-	return 0;
-}
-
-/*
- * reserve_crashkernel() - reserves memory for crash kernel
- *
- * This function reserves memory area given in "crashkernel=" kernel command
- * line parameter. The memory reserved is used by dump capture kernel when
- * primary kernel is crashing.
- */
-static void __init reserve_crashkernel(void)
+static void __init arch_reserve_crashkernel(void)
 {
-	unsigned long long crash_low_size = 0, search_base = 0;
-	unsigned long long crash_max = CRASH_ADDR_LOW_MAX;
+	unsigned long long low_size = 0;
 	unsigned long long crash_base, crash_size;
 	char *cmdline = boot_command_line;
-	bool fixed_base = false;
 	bool high = false;
 	int ret;
 
 	if (!IS_ENABLED(CONFIG_KEXEC_CORE))
 		return;
 
-	/* crashkernel=X[@offset] */
 	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
-				&crash_size, &crash_base, NULL, NULL);
-	if (ret == -ENOENT) {
-		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
-		if (ret || !crash_size)
-			return;
-
-		/*
-		 * crashkernel=Y,low can be specified or not, but invalid value
-		 * is not allowed.
-		 */
-		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
-		if (ret == -ENOENT)
-			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
-		else if (ret)
-			return;
-
-		search_base = CRASH_HIGH_SEARCH_BASE;
-		crash_max = CRASH_ADDR_HIGH_MAX;
-		high = true;
-	} else if (ret || !crash_size) {
-		/* The specified value is invalid */
+				&crash_size, &crash_base,
+				&low_size, &high);
+	if (ret)
 		return;
-	}
-
-	crash_size = PAGE_ALIGN(crash_size);
-
-	/* User specifies base address explicitly. */
-	if (crash_base) {
-		fixed_base = true;
-		search_base = crash_base;
-		crash_max = crash_base + crash_size;
-	}
-
-retry:
-	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
-					       search_base, crash_max);
-	if (!crash_base) {
-		/*
-		 * For crashkernel=size[KMG]@offset[KMG], print out failure
-		 * message if can't reserve the specified region.
-		 */
-		if (fixed_base) {
-			pr_warn("crashkernel reservation failed - memory is in use.\n");
-			return;
-		}
-
-		/*
-		 * For crashkernel=size[KMG], if the first attempt was for
-		 * low memory, fall back to high memory, the minimum required
-		 * low memory will be reserved later.
-		 */
-		if (!high && crash_max == CRASH_ADDR_LOW_MAX) {
-			crash_max = CRASH_ADDR_HIGH_MAX;
-			search_base = CRASH_ADDR_LOW_MAX;
-			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
-			goto retry;
-		}
-
-		/*
-		 * For crashkernel=size[KMG],high, if the first attempt was
-		 * for high memory, fall back to low memory.
-		 */
-		if (high && crash_max == CRASH_ADDR_HIGH_MAX) {
-			crash_max = CRASH_ADDR_LOW_MAX;
-			search_base = 0;
-			goto retry;
-		}
-		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
-			crash_size);
-		return;
-	}
-
-	if ((crash_base >= CRASH_ADDR_LOW_MAX) && crash_low_size &&
-	     reserve_crashkernel_low(crash_low_size)) {
-		memblock_phys_free(crash_base, crash_size);
-		return;
-	}
-
-	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
-		crash_base, crash_base + crash_size, crash_size >> 20);
-
-	/*
-	 * The crashkernel memory will be removed from the kernel linear
-	 * map. Inform kmemleak so that it won't try to access it.
-	 */
-	kmemleak_ignore_phys(crash_base);
-	if (crashk_low_res.end)
-		kmemleak_ignore_phys(crashk_low_res.start);
 
-	crashk_res.start = crash_base;
-	crashk_res.end = crash_base + crash_size - 1;
-	insert_resource(&iomem_resource, &crashk_res);
+	reserve_crashkernel_generic(cmdline, crash_size, crash_base,
+				    low_size, high);
 }
 
 /*
@@ -481,7 +357,7 @@ void __init bootmem_init(void)
 	 * request_standard_resources() depends on crashkernel's memory being
 	 * reserved, so do it here.
 	 */
-	reserve_crashkernel();
+	arch_reserve_crashkernel();
 
 	memblock_dump_all();
 }
-- 
2.41.0



From paul at paul-moore.com  Tue Aug 29 12:34:10 2023
From: paul at paul-moore.com (Paul Moore)
Date: Tue, 29 Aug 2023 15:34:10 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com> <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
Message-ID: <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>

On Mon, Aug 21, 2023 at 7:08?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> On Mon, 2023-08-21 at 15:05 -0700, Sush Shringarputale wrote:
> > On 8/14/2023 3:02 PM, Mimi Zohar wrote:
> > > On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
> > >>> This design seems overly complex and requires synchronization between
> > >>> the "snapshot" record and exporting the records from the measurement
> > >>> list.  None of this would be necessary if the measurements were copied
> > >>> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
> > Even if the Kernel maintains the link between a tmpfs exported and an
> > in-memory IMA log - it still has to copy the tmpfs portion to the
> > Kernel memory during kexec soft boot.  tmpfs is cleared during kexec,
> > so this copying of tmpfs back to kernel memory is necessary to preserve
> > the integrity of the log during kexec.  But the copying would add back
> > the memory pressure on the node during kexec (which may result in
> > out-of-memory), defeating the purpose of the overall effort/feature.
> > Copying to a regular *persistent* protected file seems a cleaner
> > approach, compared to tmpfs.
>
> From a kernel perspective, it doesn't make a difference if userspace
> provides a tmpfs or persistent file.  As per the discussion
> https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t
> , userspace provides the kernel with the file descriptor of the opened
> file.
>
> > We prototyped this solution, however it
> > does not seem to be a common pattern within the Kernel to write state
> > directly to files on disk file systems.  We considered two potential
> > options:
>
> If no file descriptor is provided, then the measurements aren't copied
> and removed from the securityfs file.  If there are write errors, the
> measurements aren't removed from the securityfs file until the write
> errors are resolved.

It sounds like this approach would require the file/filesystem to be
continuously available for the life of the system once the log was
snapshotted/overflowed to persistent storage, yes?  Assuming that is
the case, what happens if the file/filesystem becomes inaccessible at
some point and an attestation client attempts to read the entire log?

-- 
paul-moore.com


From zohar at linux.ibm.com  Tue Aug 29 14:03:23 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Tue, 29 Aug 2023 17:03:23 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
	 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
	 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
	 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
	 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
	 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
Message-ID: <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>

On Tue, 2023-08-29 at 15:34 -0400, Paul Moore wrote:
> On Mon, Aug 21, 2023 at 7:08?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > On Mon, 2023-08-21 at 15:05 -0700, Sush Shringarputale wrote:
> > > On 8/14/2023 3:02 PM, Mimi Zohar wrote:
> > > > On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
> > > >>> This design seems overly complex and requires synchronization between
> > > >>> the "snapshot" record and exporting the records from the measurement
> > > >>> list.  None of this would be necessary if the measurements were copied
> > > >>> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
> > > Even if the Kernel maintains the link between a tmpfs exported and an
> > > in-memory IMA log - it still has to copy the tmpfs portion to the
> > > Kernel memory during kexec soft boot.  tmpfs is cleared during kexec,
> > > so this copying of tmpfs back to kernel memory is necessary to preserve
> > > the integrity of the log during kexec.  But the copying would add back
> > > the memory pressure on the node during kexec (which may result in
> > > out-of-memory), defeating the purpose of the overall effort/feature.
> > > Copying to a regular *persistent* protected file seems a cleaner
> > > approach, compared to tmpfs.
> >
> > From a kernel perspective, it doesn't make a difference if userspace
> > provides a tmpfs or persistent file.  As per the discussion
> > https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t
> > , userspace provides the kernel with the file descriptor of the opened
> > file.
> >
> > > We prototyped this solution, however it
> > > does not seem to be a common pattern within the Kernel to write state
> > > directly to files on disk file systems.  We considered two potential
> > > options:
> >
> > If no file descriptor is provided, then the measurements aren't copied
> > and removed from the securityfs file.  If there are write errors, the
> > measurements aren't removed from the securityfs file until the write
> > errors are resolved.
> 
> It sounds like this approach would require the file/filesystem to be
> continuously available for the life of the system once the log was
> snapshotted/overflowed to persistent storage, yes?  Assuming that is
> the case, what happens if the file/filesystem becomes inaccessible at
> some point and an attestation client attempts to read the entire log?

The main purpose of the change is to addres kernel memory pressure. 
Two designs are being discussed: Sush's "snapshotting" design and
Amir's original suggestion of continously exporting the measurement
records to a tmpfs or regular file.  Both designs require verifying the
initial attestation quote by walking the entire measurement list,
calculating the expected TPM PCR value(s).  That doesn't change.

-- 
thanks,

Mimi



From paul at paul-moore.com  Tue Aug 29 14:30:20 2023
From: paul at paul-moore.com (Paul Moore)
Date: Tue, 29 Aug 2023 17:30:20 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com> <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
Message-ID: <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>

On Tue, Aug 29, 2023 at 5:05?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> On Tue, 2023-08-29 at 15:34 -0400, Paul Moore wrote:
> > On Mon, Aug 21, 2023 at 7:08?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > On Mon, 2023-08-21 at 15:05 -0700, Sush Shringarputale wrote:
> > > > On 8/14/2023 3:02 PM, Mimi Zohar wrote:
> > > > > On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
> > > > >>> This design seems overly complex and requires synchronization between
> > > > >>> the "snapshot" record and exporting the records from the measurement
> > > > >>> list.  None of this would be necessary if the measurements were copied
> > > > >>> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
> > > > Even if the Kernel maintains the link between a tmpfs exported and an
> > > > in-memory IMA log - it still has to copy the tmpfs portion to the
> > > > Kernel memory during kexec soft boot.  tmpfs is cleared during kexec,
> > > > so this copying of tmpfs back to kernel memory is necessary to preserve
> > > > the integrity of the log during kexec.  But the copying would add back
> > > > the memory pressure on the node during kexec (which may result in
> > > > out-of-memory), defeating the purpose of the overall effort/feature.
> > > > Copying to a regular *persistent* protected file seems a cleaner
> > > > approach, compared to tmpfs.
> > >
> > > From a kernel perspective, it doesn't make a difference if userspace
> > > provides a tmpfs or persistent file.  As per the discussion
> > > https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t
> > > , userspace provides the kernel with the file descriptor of the opened
> > > file.
> > >
> > > > We prototyped this solution, however it
> > > > does not seem to be a common pattern within the Kernel to write state
> > > > directly to files on disk file systems.  We considered two potential
> > > > options:
> > >
> > > If no file descriptor is provided, then the measurements aren't copied
> > > and removed from the securityfs file.  If there are write errors, the
> > > measurements aren't removed from the securityfs file until the write
> > > errors are resolved.
> >
> > It sounds like this approach would require the file/filesystem to be
> > continuously available for the life of the system once the log was
> > snapshotted/overflowed to persistent storage, yes?  Assuming that is
> > the case, what happens if the file/filesystem becomes inaccessible at
> > some point and an attestation client attempts to read the entire log?
>
> The main purpose of the change is to addres kernel memory pressure.
> Two designs are being discussed: Sush's "snapshotting" design and
> Amir's original suggestion of continously exporting the measurement
> records to a tmpfs or regular file.  Both designs require verifying the
> initial attestation quote by walking the entire measurement list,
> calculating the expected TPM PCR value(s).  That doesn't change.

Sure, but my question is about what happens if portions of the
measurement list disappear due to file/filesystem problems?  How is
that handled?

-- 
paul-moore.com


From lkp at intel.com  Tue Aug 29 14:37:18 2023
From: lkp at intel.com (kernel test robot)
Date: Wed, 30 Aug 2023 05:37:18 +0800
Subject: [PATCH v2 6/8] x86: kdump: use generic interface to simplify
 crashkernel reservation code
In-Reply-To: <20230829121610.138107-7-bhe@redhat.com>
References: <20230829121610.138107-7-bhe@redhat.com>
Message-ID: <202308300522.oOG0V3U3-lkp@intel.com>

Hi Baoquan,

kernel test robot noticed the following build errors:

[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on tip/x86/core powerpc/next powerpc/fixes linus/master v6.5]
[cannot apply to next-20230829]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Baoquan-He/crash_core-c-remove-unnecessary-parameter-of-function/20230829-201942
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
patch link:    https://lore.kernel.org/r/20230829121610.138107-7-bhe%40redhat.com
patch subject: [PATCH v2 6/8] x86: kdump: use generic interface to simplify crashkernel reservation code
config: i386-randconfig-r026-20230830 (https://download.01.org/0day-ci/archive/20230830/202308300522.oOG0V3U3-lkp at intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20230830/202308300522.oOG0V3U3-lkp at intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp at intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202308300522.oOG0V3U3-lkp at intel.com/

All errors (new ones prefixed by >>):

   ld: kernel/crash_core.o: in function `reserve_crashkernel_generic':
>> kernel/crash_core.c:450: undefined reference to `crashk_low_res'
>> ld: kernel/crash_core.c:451: undefined reference to `crashk_low_res'
>> ld: kernel/crash_core.c:455: undefined reference to `crashk_res'
   ld: kernel/crash_core.c:453: undefined reference to `crashk_res'
   ld: kernel/crash_core.c:454: undefined reference to `crashk_res'


vim +450 kernel/crash_core.c

6bee83d29d2e09 Baoquan He 2023-08-29  375  
6bee83d29d2e09 Baoquan He 2023-08-29  376  void __init reserve_crashkernel_generic(char *cmdline,
6bee83d29d2e09 Baoquan He 2023-08-29  377  			     unsigned long long crash_size,
6bee83d29d2e09 Baoquan He 2023-08-29  378  			     unsigned long long crash_base,
6bee83d29d2e09 Baoquan He 2023-08-29  379  			     unsigned long long crash_low_size,
6bee83d29d2e09 Baoquan He 2023-08-29  380  			     bool high)
6bee83d29d2e09 Baoquan He 2023-08-29  381  {
6bee83d29d2e09 Baoquan He 2023-08-29  382  	unsigned long long search_end = CRASH_ADDR_LOW_MAX, search_base = 0;
6bee83d29d2e09 Baoquan He 2023-08-29  383  	bool fixed_base = false;
6bee83d29d2e09 Baoquan He 2023-08-29  384  
6bee83d29d2e09 Baoquan He 2023-08-29  385  	/* User specifies base address explicitly. */
6bee83d29d2e09 Baoquan He 2023-08-29  386  	if (crash_base) {
6bee83d29d2e09 Baoquan He 2023-08-29  387  		fixed_base = true;
6bee83d29d2e09 Baoquan He 2023-08-29  388  		search_base = crash_base;
6bee83d29d2e09 Baoquan He 2023-08-29  389  		search_end = crash_base + crash_size;
6bee83d29d2e09 Baoquan He 2023-08-29  390  	}
6bee83d29d2e09 Baoquan He 2023-08-29  391  
6bee83d29d2e09 Baoquan He 2023-08-29  392  	if (high) {
6bee83d29d2e09 Baoquan He 2023-08-29  393  		search_base = CRASH_ADDR_LOW_MAX;
6bee83d29d2e09 Baoquan He 2023-08-29  394  		search_end = CRASH_ADDR_HIGH_MAX;
6bee83d29d2e09 Baoquan He 2023-08-29  395  	}
6bee83d29d2e09 Baoquan He 2023-08-29  396  
6bee83d29d2e09 Baoquan He 2023-08-29  397  retry:
6bee83d29d2e09 Baoquan He 2023-08-29  398  	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
6bee83d29d2e09 Baoquan He 2023-08-29  399  					       search_base, search_end);
6bee83d29d2e09 Baoquan He 2023-08-29  400  	if (!crash_base) {
6bee83d29d2e09 Baoquan He 2023-08-29  401  		/*
6bee83d29d2e09 Baoquan He 2023-08-29  402  		 * For crashkernel=size[KMG]@offset[KMG], print out failure
6bee83d29d2e09 Baoquan He 2023-08-29  403  		 * message if can't reserve the specified region.
6bee83d29d2e09 Baoquan He 2023-08-29  404  		 */
6bee83d29d2e09 Baoquan He 2023-08-29  405  		if (fixed_base) {
6bee83d29d2e09 Baoquan He 2023-08-29  406  			pr_warn("crashkernel reservation failed - memory is in use.\n");
6bee83d29d2e09 Baoquan He 2023-08-29  407  			return;
6bee83d29d2e09 Baoquan He 2023-08-29  408  		}
6bee83d29d2e09 Baoquan He 2023-08-29  409  
6bee83d29d2e09 Baoquan He 2023-08-29  410  		/*
6bee83d29d2e09 Baoquan He 2023-08-29  411  		 * For crashkernel=size[KMG], if the first attempt was for
6bee83d29d2e09 Baoquan He 2023-08-29  412  		 * low memory, fall back to high memory, the minimum required
6bee83d29d2e09 Baoquan He 2023-08-29  413  		 * low memory will be reserved later.
6bee83d29d2e09 Baoquan He 2023-08-29  414  		 */
6bee83d29d2e09 Baoquan He 2023-08-29  415  		if (!high && search_end == CRASH_ADDR_LOW_MAX) {
6bee83d29d2e09 Baoquan He 2023-08-29  416  			search_end = CRASH_ADDR_HIGH_MAX;
6bee83d29d2e09 Baoquan He 2023-08-29  417  			search_base = CRASH_ADDR_LOW_MAX;
6bee83d29d2e09 Baoquan He 2023-08-29  418  			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
6bee83d29d2e09 Baoquan He 2023-08-29  419  			goto retry;
6bee83d29d2e09 Baoquan He 2023-08-29  420  		}
6bee83d29d2e09 Baoquan He 2023-08-29  421  
6bee83d29d2e09 Baoquan He 2023-08-29  422  		/*
6bee83d29d2e09 Baoquan He 2023-08-29  423  		 * For crashkernel=size[KMG],high, if the first attempt was
6bee83d29d2e09 Baoquan He 2023-08-29  424  		 * for high memory, fall back to low memory.
6bee83d29d2e09 Baoquan He 2023-08-29  425  		 */
6bee83d29d2e09 Baoquan He 2023-08-29  426  		if (high && search_end == CRASH_ADDR_HIGH_MAX) {
6bee83d29d2e09 Baoquan He 2023-08-29  427  			search_end = CRASH_ADDR_LOW_MAX;
6bee83d29d2e09 Baoquan He 2023-08-29  428  			search_base = 0;
6bee83d29d2e09 Baoquan He 2023-08-29  429  			goto retry;
6bee83d29d2e09 Baoquan He 2023-08-29  430  		}
6bee83d29d2e09 Baoquan He 2023-08-29  431  		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
6bee83d29d2e09 Baoquan He 2023-08-29  432  			crash_size);
6bee83d29d2e09 Baoquan He 2023-08-29  433  		return;
6bee83d29d2e09 Baoquan He 2023-08-29  434  	}
6bee83d29d2e09 Baoquan He 2023-08-29  435  
6bee83d29d2e09 Baoquan He 2023-08-29  436  	if ((crash_base > CRASH_ADDR_LOW_MAX) &&
6bee83d29d2e09 Baoquan He 2023-08-29  437  	     crash_low_size && reserve_crashkernel_low(crash_low_size)) {
6bee83d29d2e09 Baoquan He 2023-08-29  438  		memblock_phys_free(crash_base, crash_size);
6bee83d29d2e09 Baoquan He 2023-08-29  439  		return;
6bee83d29d2e09 Baoquan He 2023-08-29  440  	}
6bee83d29d2e09 Baoquan He 2023-08-29  441  
6bee83d29d2e09 Baoquan He 2023-08-29  442  	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
6bee83d29d2e09 Baoquan He 2023-08-29  443  		crash_base, crash_base + crash_size, crash_size >> 20);
6bee83d29d2e09 Baoquan He 2023-08-29  444  
6bee83d29d2e09 Baoquan He 2023-08-29  445  	/*
6bee83d29d2e09 Baoquan He 2023-08-29  446  	 * The crashkernel memory will be removed from the kernel linear
6bee83d29d2e09 Baoquan He 2023-08-29  447  	 * map. Inform kmemleak so that it won't try to access it.
6bee83d29d2e09 Baoquan He 2023-08-29  448  	 */
6bee83d29d2e09 Baoquan He 2023-08-29  449  	kmemleak_ignore_phys(crash_base);
6bee83d29d2e09 Baoquan He 2023-08-29 @450  	if (crashk_low_res.end)
6bee83d29d2e09 Baoquan He 2023-08-29 @451  		kmemleak_ignore_phys(crashk_low_res.start);
6bee83d29d2e09 Baoquan He 2023-08-29  452  
6bee83d29d2e09 Baoquan He 2023-08-29  453  	crashk_res.start = crash_base;
6bee83d29d2e09 Baoquan He 2023-08-29  454  	crashk_res.end = crash_base + crash_size - 1;
6bee83d29d2e09 Baoquan He 2023-08-29 @455  	insert_resource(&iomem_resource, &crashk_res);
6bee83d29d2e09 Baoquan He 2023-08-29  456  }
6bee83d29d2e09 Baoquan He 2023-08-29  457  #endif
6bee83d29d2e09 Baoquan He 2023-08-29  458  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki


From zohar at linux.ibm.com  Tue Aug 29 14:54:10 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Tue, 29 Aug 2023 17:54:10 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
	 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
	 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
	 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
	 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
	 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
	 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
	 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
Message-ID: <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>

On Tue, 2023-08-29 at 17:30 -0400, Paul Moore wrote:
> On Tue, Aug 29, 2023 at 5:05?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > On Tue, 2023-08-29 at 15:34 -0400, Paul Moore wrote:
> > > On Mon, Aug 21, 2023 at 7:08?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > On Mon, 2023-08-21 at 15:05 -0700, Sush Shringarputale wrote:
> > > > > On 8/14/2023 3:02 PM, Mimi Zohar wrote:
> > > > > > On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
> > > > > >>> This design seems overly complex and requires synchronization between
> > > > > >>> the "snapshot" record and exporting the records from the measurement
> > > > > >>> list.  None of this would be necessary if the measurements were copied
> > > > > >>> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
> > > > > Even if the Kernel maintains the link between a tmpfs exported and an
> > > > > in-memory IMA log - it still has to copy the tmpfs portion to the
> > > > > Kernel memory during kexec soft boot.  tmpfs is cleared during kexec,
> > > > > so this copying of tmpfs back to kernel memory is necessary to preserve
> > > > > the integrity of the log during kexec.  But the copying would add back
> > > > > the memory pressure on the node during kexec (which may result in
> > > > > out-of-memory), defeating the purpose of the overall effort/feature.
> > > > > Copying to a regular *persistent* protected file seems a cleaner
> > > > > approach, compared to tmpfs.
> > > >
> > > > From a kernel perspective, it doesn't make a difference if userspace
> > > > provides a tmpfs or persistent file.  As per the discussion
> > > > https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t
> > > > , userspace provides the kernel with the file descriptor of the opened
> > > > file.
> > > >
> > > > > We prototyped this solution, however it
> > > > > does not seem to be a common pattern within the Kernel to write state
> > > > > directly to files on disk file systems.  We considered two potential
> > > > > options:
> > > >
> > > > If no file descriptor is provided, then the measurements aren't copied
> > > > and removed from the securityfs file.  If there are write errors, the
> > > > measurements aren't removed from the securityfs file until the write
> > > > errors are resolved.
> > >
> > > It sounds like this approach would require the file/filesystem to be
> > > continuously available for the life of the system once the log was
> > > snapshotted/overflowed to persistent storage, yes?  Assuming that is
> > > the case, what happens if the file/filesystem becomes inaccessible at
> > > some point and an attestation client attempts to read the entire log?
> >
> > The main purpose of the change is to addres kernel memory pressure.
> > Two designs are being discussed: Sush's "snapshotting" design and
> > Amir's original suggestion of continously exporting the measurement
> > records to a tmpfs or regular file.  Both designs require verifying the
> > initial attestation quote by walking the entire measurement list,
> > calculating the expected TPM PCR value(s).  That doesn't change.
> 
> Sure, but my question is about what happens if portions of the
> measurement list disappear due to file/filesystem problems?  How is
> that handled?

With the "snapshotting" solution there could be multiple files, so
portions could be missing.  The other solution, the preferred solution,
would be one file.

Any suggestions?

-- 
thanks,

Mimi




From skinsburskii at linux.microsoft.com  Tue Aug 29 15:07:40 2023
From: skinsburskii at linux.microsoft.com (Stanislav Kinsburskii)
Date: Tue, 29 Aug 2023 15:07:40 -0700
Subject: [RFC PATCH] Introduce persistent memory pool
In-Reply-To: <e0ed9fb9-8e7a-44ad-976a-27362f6e537a@amazon.de>
References: <64e7cbf7.050a0220.114c7.b70dSMTPIN_ADDED_BROKEN@mx.google.com>
 <2023082506-enchanted-tripping-d1d5@gregkh>
 <c26ad989dcc6737dd295e980c78ef53740098810.camel@amazon.com>
 <20230823024500.GA25462@skinsburskii.>
 <e0ed9fb9-8e7a-44ad-976a-27362f6e537a@amazon.de>
Message-ID: <20230829220740.GA26605@skinsburskii.>

On Mon, Aug 28, 2023 at 10:50:19PM +0200, Alexander Graf wrote:
> +kexec, iommu, kvm
> 
> On 23.08.23 04:45, Stanislav Kinsburskii wrote:
> > 
> > +akpm, +linux-mm
> > 
> > On Fri, Aug 25, 2023 at 01:32:40PM +0000, Gowans, James wrote:
> > > On Fri, 2023-08-25 at 10:05 +0200, Greg Kroah-Hartman wrote:
> > > 
> > > Thanks for adding me to this thread Greg!
> > > 
> > > > On Tue, Aug 22, 2023 at 11:34:34AM -0700, Stanislav Kinsburskii wrote:
> > > > > This patch addresses the need for a memory allocator dedicated to
> > > > > persistent memory within the kernel. This allocator will preserve
> > > > > kernel-specific states like DMA passthrough device states, IOMMU state, and
> > > > > more across kexec.
> > > > > The proposed solution offers a foundational implementation for potential
> > > > > custom solutions that might follow. Though the implementation is
> > > > > intentionally kept concise and straightforward to foster discussion and
> > > > > feedback, it's fully functional in its current state.
> > > Hi Stanislav, it looks like we're working on similar things. I'm looking
> > > to develop a mechanism to support hypervisor live update for when KVM is
> > > running VMs with PCI device passthrough. VMs with device passthrough
> > > also necessitates passing and re-hydrating IOMMU state so that DMA can
> > > continue during live update.
> > > 
> > > Planning on having an LPC session on this topic:
> > > https://lpc.events/event/17/abstracts/1629/ (currently it's only a
> > > submitted abstract so not sure if visible, hopefully it will be soon).
> > > 
> > > We are looking at implementing persistence across kexec via an in-memory
> > > filesystem on top of reserved memory. This would have files for anything
> > > that needs to be persisted. That includes files for IOMMU pgtables, for
> > > guest memory or userspace-accessible memory.
> > > 
> > > It may be nice to solve all kexec persistence requirements with one
> > > solution, but we can consider IOMMU separately. There are at least three
> > > ways that this can be done:
> > > a) carving out reserved memory for pgtables. This is done by your
> > > proposal here, as well as my suggestion of a filesystem.
> > > b) pre/post kexec hooks for drivers to serialise state and pass it
> > > across in a structured format from old to new kernel.
> > > c) Reconstructing IOMMU state in the new kernel by starting at the
> > > hardware registers and walking the page tables. No state passing needed.
> > > 
> > > Have you considered option (b) and (c) here? One of the implications of
> > > (b) and (c) are that they would need to hook into the buddy allocator
> > > really early to be able to carve out the reconstructed page tables
> > > before the allocator is used. Similar to how pkram [0] hooks in early to
> > > carve out pages used for its filesystem.
> > > 
> > Hi James,
> > 
> > We are indeed working on similar things, so thanks for chiming in.
> > I've seen pkram proposal as well as your comments there.
> > 
> > I think (b) will need some persistent-over-kexec memory to pass the
> > state across kexec as well as some key-value store persisted as well.
> > And the proposed persistent memory pool is aimed exactly for this
> > purpose.
> > Or do you imply some other way to pass driver's data accross kexec?
> 
> 
> If I had to build this, I'd probably do it just like device tree passing on
> ARM. It's a single, physically contiguous blob of data whose entry point you
> pass to the target kernel. IIRC ACPI passing works similarly. This would
> just be one more opaque data structure that then needs very strict
> versioning and forward/backward compat guarantees.
> 

Device tree or ACPI are options indeed. However AFAIU in case of DT user
space has to involved into the picture to modify and complie it, while
ACPI isn't flexible or easily extendable.
Also, AFAIU both these standards were designed with passing
hardware-specific data in mind from bootstrap software to an OS kernel
and thus were never really intended to be used for creating a persistent
state accross kexec.
To me, an attempt to use either of them to pass kernel-specific data looks
like an abuse (or misuse) excused by the simplicity of implementation.

> 
> > I dind't consider (c) yet, thanks for for the pointer.
> > 
> > I have a question in this scope: how is PCI devices registers state is persisted
> > across kexec with the files system you are working on? I.e. how does
> > driver know, that the device shouldn't not be reinitialized?
> 
> 
> The easiest way to do it initially would be kernel command line options that
> hack up the drivers. But I suppose depending on the option we go with, you
> can also use the respective "natural" path:
> 
> (a) A special metadata file that explains the state to the driver
> (b) An entry in the structured file format that explains the state to the
> target driver
> (c) Compatible target drivers try to enumerate state from the target
> device's register file
> 

Command line option is the simplest way to go indeed, but from my POV
it's good only for pointing to a particualr object, which is persisted
somehow else. But it we have a persistence mechanism, then I think we
can make another step forward and don't use command line at all (which
is a bit cumbersome and errorprone due to it's human-readable and
serialized nature).

I'm leaning towards some kind of "natural" path you mentioned... I guess
I'm a bit confused with the word "file" here, as it sounds line it
implies a file system driver, and I'm not sure that's what we want for
driver specific data.

> 
> > 
> > > > > Potential applications include:
> > > > > 
> > > > >    1. Allowing various in-kernel entities to allocate persistent pages from
> > > > >       a singular memory pool, eliminating the need for multiple region
> > > > >       reservations.
> > > > > 
> > > > >    2. For in-kernel components that require the allocation address to be
> > > > >       available on kernel kexec, this address can be exposed to user space and
> > > > >       then passed via the command line.
> > > Do you have specific examples of other state that needs to be passed
> > > across? Trying to see whether tailoring specifically to the IOMMU case
> > > is okay. Conceptually IOMMU state can be reconstructed starting with
> > > hardware registers, not needing reserved memory. Other use-cases may not
> > > have this option.
> > > 
> > Well, basically it's IOMMU state and PCI devices to skip/avoid
> > initializing.
> > I bet there can be other misc (and unrelated things) like persistent
> > filesystems, block devices, etc. But I don't have a solid set of use
> > cases to present.
> 
> 
> Would be great if you could think through the problem space until LPC so we
> can have a solid conversation there :)
> 

Yeah, I have a few ideas I'll try to implement and share before LPC.
Unfortunatelly I'm not planning to attend it this year, so this
conversation will be without me.
But I'll do my best to provide as much content to discuss as I can.

> 
> > 
> > > > As you have no in-kernel users of this, it's not something we can even
> > > > consider at the moment for obvious reasons (neither would you want us
> > > > to.)
> > > > 
> > > > Can you make this part of a patch series that actually adds a user,
> > > > probably more than one, so that we can see if any of this even makes
> > > > sense?
> > > I'm very keen to see this as well. The way that the IOMMU drivers are
> > > enlightened to hook into your memory pool will likely be similar to how
> > > they would hook into my proposal of an in-memory filesystem.
> > > Do you have code available showing the IOMMU integration?
> > > 
> > No, I don't have such a code yet.
> > But I was thinking that using such a allocator in the mempool allows
> > to hide this implementation under the hood of an existent generic
> > mechanism, which is then can be used to create persistent objects (file
> > system, for example) on top of it.
> 
> 
> Unfortunately it's practically impossible to have a solid conversation on
> generic mechanisms without actual users to see how they fit in with the real
> world. That's Greg's answer to your patch set and I tend to agree. What if
> (b) or (c) turn out much more viable? Then we've wasted a lot of effort in
> shaping up the allocator for no good reason.
> 

This is fair.
I sent such a small piece, because I wanted to get some opinions about
the approach in general without too much investement into.
Thanks to you and Greg, I now have the feadeback I was looking for, so
I'm planning to come up with a series including in-kernel users, like
Greg suggested.

Thanks,
Stanislav



From paul at paul-moore.com  Tue Aug 29 16:15:15 2023
From: paul at paul-moore.com (Paul Moore)
Date: Tue, 29 Aug 2023 19:15:15 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com> <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
Message-ID: <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>

On Tue, Aug 29, 2023 at 5:54?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> On Tue, 2023-08-29 at 17:30 -0400, Paul Moore wrote:
> > On Tue, Aug 29, 2023 at 5:05?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > On Tue, 2023-08-29 at 15:34 -0400, Paul Moore wrote:
> > > > On Mon, Aug 21, 2023 at 7:08?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > On Mon, 2023-08-21 at 15:05 -0700, Sush Shringarputale wrote:
> > > > > > On 8/14/2023 3:02 PM, Mimi Zohar wrote:
> > > > > > > On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
> > > > > > >>> This design seems overly complex and requires synchronization between
> > > > > > >>> the "snapshot" record and exporting the records from the measurement
> > > > > > >>> list.  None of this would be necessary if the measurements were copied
> > > > > > >>> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
> > > > > > Even if the Kernel maintains the link between a tmpfs exported and an
> > > > > > in-memory IMA log - it still has to copy the tmpfs portion to the
> > > > > > Kernel memory during kexec soft boot.  tmpfs is cleared during kexec,
> > > > > > so this copying of tmpfs back to kernel memory is necessary to preserve
> > > > > > the integrity of the log during kexec.  But the copying would add back
> > > > > > the memory pressure on the node during kexec (which may result in
> > > > > > out-of-memory), defeating the purpose of the overall effort/feature.
> > > > > > Copying to a regular *persistent* protected file seems a cleaner
> > > > > > approach, compared to tmpfs.
> > > > >
> > > > > From a kernel perspective, it doesn't make a difference if userspace
> > > > > provides a tmpfs or persistent file.  As per the discussion
> > > > > https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t
> > > > > , userspace provides the kernel with the file descriptor of the opened
> > > > > file.
> > > > >
> > > > > > We prototyped this solution, however it
> > > > > > does not seem to be a common pattern within the Kernel to write state
> > > > > > directly to files on disk file systems.  We considered two potential
> > > > > > options:
> > > > >
> > > > > If no file descriptor is provided, then the measurements aren't copied
> > > > > and removed from the securityfs file.  If there are write errors, the
> > > > > measurements aren't removed from the securityfs file until the write
> > > > > errors are resolved.
> > > >
> > > > It sounds like this approach would require the file/filesystem to be
> > > > continuously available for the life of the system once the log was
> > > > snapshotted/overflowed to persistent storage, yes?  Assuming that is
> > > > the case, what happens if the file/filesystem becomes inaccessible at
> > > > some point and an attestation client attempts to read the entire log?
> > >
> > > The main purpose of the change is to addres kernel memory pressure.
> > > Two designs are being discussed: Sush's "snapshotting" design and
> > > Amir's original suggestion of continously exporting the measurement
> > > records to a tmpfs or regular file.  Both designs require verifying the
> > > initial attestation quote by walking the entire measurement list,
> > > calculating the expected TPM PCR value(s).  That doesn't change.
> >
> > Sure, but my question is about what happens if portions of the
> > measurement list disappear due to file/filesystem problems?  How is
> > that handled?
>
> With the "snapshotting" solution there could be multiple files, so
> portions could be missing.  The other solution, the preferred solution,
> would be one file.

With the snapshotting approach the kernel doesn't need to maintain
ongoing access to a file, that is left up to the user process
performing the attestation (or simply inspecting the logs).  I have to
ask, for the third time now in as many hours, how does the proposed
kernel-holds-an-fd-open solution handle the case where the
file/filesystem is no longer accessible?  The snapshotting approach
should be much more resilient here as the error recovery paths can be
much more involved than what we would have available in the kernel,
not to mention the flexibility of allowing a user process to determine
how to store and manage the snapshotted log.

Considering that the snapshotting approach is opt-in (userspace has to
initiate the snapshot), I'm not sure the concern over log offsets is a
significant objection: existing userspace will never trigger a
snapshot, and new userspace that could potentially trigger a snapshot
should be written to take that into account.

-- 
paul-moore.com


From lkp at intel.com  Tue Aug 29 18:49:27 2023
From: lkp at intel.com (kernel test robot)
Date: Wed, 30 Aug 2023 09:49:27 +0800
Subject: [PATCH v2 6/8] x86: kdump: use generic interface to simplify
 crashkernel reservation code
In-Reply-To: <20230829121610.138107-7-bhe@redhat.com>
References: <20230829121610.138107-7-bhe@redhat.com>
Message-ID: <202308300910.e0i4piJT-lkp@intel.com>

Hi Baoquan,

kernel test robot noticed the following build errors:

[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on tip/x86/core powerpc/next powerpc/fixes v6.5]
[cannot apply to linus/master next-20230829]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Baoquan-He/crash_core-c-remove-unnecessary-parameter-of-function/20230829-201942
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
patch link:    https://lore.kernel.org/r/20230829121610.138107-7-bhe%40redhat.com
patch subject: [PATCH v2 6/8] x86: kdump: use generic interface to simplify crashkernel reservation code
config: x86_64-randconfig-r022-20230830 (https://download.01.org/0day-ci/archive/20230830/202308300910.e0i4piJT-lkp at intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20230830/202308300910.e0i4piJT-lkp at intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp at intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202308300910.e0i4piJT-lkp at intel.com/

All errors (new ones prefixed by >>):

   ld: vmlinux.o: in function `reserve_crashkernel_low':
   kernel/crash_core.c:369: undefined reference to `crashk_low_res'
   ld: kernel/crash_core.c:369: undefined reference to `crashk_low_res'
   ld: kernel/crash_core.c:370: undefined reference to `crashk_low_res'
   ld: kernel/crash_core.c:369: undefined reference to `crashk_low_res'
   ld: kernel/crash_core.c:370: undefined reference to `crashk_low_res'
   ld: vmlinux.o:kernel/crash_core.c:371: more undefined references to `crashk_low_res' follow
   ld: vmlinux.o: in function `reserve_crashkernel_generic':
>> kernel/crash_core.c:453: undefined reference to `crashk_res'
>> ld: kernel/crash_core.c:453: undefined reference to `crashk_res'
   ld: kernel/crash_core.c:454: undefined reference to `crashk_res'
>> ld: kernel/crash_core.c:453: undefined reference to `crashk_res'
   ld: kernel/crash_core.c:454: undefined reference to `crashk_res'
   ld: vmlinux.o:kernel/crash_core.c:454: more undefined references to `crashk_res' follow


vim +453 kernel/crash_core.c

71d2bcec2d4d69 Philipp Rudo 2021-12-24  353  
6bee83d29d2e09 Baoquan He   2023-08-29  354  #ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
6bee83d29d2e09 Baoquan He   2023-08-29  355  static int __init reserve_crashkernel_low(unsigned long long low_size)
6bee83d29d2e09 Baoquan He   2023-08-29  356  {
6bee83d29d2e09 Baoquan He   2023-08-29  357  #ifdef CONFIG_64BIT
6bee83d29d2e09 Baoquan He   2023-08-29  358  	unsigned long long low_base;
6bee83d29d2e09 Baoquan He   2023-08-29  359  
6bee83d29d2e09 Baoquan He   2023-08-29  360  	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
6bee83d29d2e09 Baoquan He   2023-08-29  361  	if (!low_base) {
6bee83d29d2e09 Baoquan He   2023-08-29  362  		pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
6bee83d29d2e09 Baoquan He   2023-08-29  363  		return -ENOMEM;
6bee83d29d2e09 Baoquan He   2023-08-29  364  	}
6bee83d29d2e09 Baoquan He   2023-08-29  365  
6bee83d29d2e09 Baoquan He   2023-08-29  366  	pr_info("crashkernel low memory reserved: 0x%08llx - 0x%08llx (%lld MB)\n",
6bee83d29d2e09 Baoquan He   2023-08-29  367  		low_base, low_base + low_size, low_size >> 20);
6bee83d29d2e09 Baoquan He   2023-08-29  368  
6bee83d29d2e09 Baoquan He   2023-08-29 @369  	crashk_low_res.start = low_base;
6bee83d29d2e09 Baoquan He   2023-08-29  370  	crashk_low_res.end   = low_base + low_size - 1;
6bee83d29d2e09 Baoquan He   2023-08-29  371  	insert_resource(&iomem_resource, &crashk_low_res);
6bee83d29d2e09 Baoquan He   2023-08-29  372  #endif
6bee83d29d2e09 Baoquan He   2023-08-29  373  	return 0;
6bee83d29d2e09 Baoquan He   2023-08-29  374  }
6bee83d29d2e09 Baoquan He   2023-08-29  375  
6bee83d29d2e09 Baoquan He   2023-08-29  376  void __init reserve_crashkernel_generic(char *cmdline,
6bee83d29d2e09 Baoquan He   2023-08-29  377  			     unsigned long long crash_size,
6bee83d29d2e09 Baoquan He   2023-08-29  378  			     unsigned long long crash_base,
6bee83d29d2e09 Baoquan He   2023-08-29  379  			     unsigned long long crash_low_size,
6bee83d29d2e09 Baoquan He   2023-08-29  380  			     bool high)
6bee83d29d2e09 Baoquan He   2023-08-29  381  {
6bee83d29d2e09 Baoquan He   2023-08-29  382  	unsigned long long search_end = CRASH_ADDR_LOW_MAX, search_base = 0;
6bee83d29d2e09 Baoquan He   2023-08-29  383  	bool fixed_base = false;
6bee83d29d2e09 Baoquan He   2023-08-29  384  
6bee83d29d2e09 Baoquan He   2023-08-29  385  	/* User specifies base address explicitly. */
6bee83d29d2e09 Baoquan He   2023-08-29  386  	if (crash_base) {
6bee83d29d2e09 Baoquan He   2023-08-29  387  		fixed_base = true;
6bee83d29d2e09 Baoquan He   2023-08-29  388  		search_base = crash_base;
6bee83d29d2e09 Baoquan He   2023-08-29  389  		search_end = crash_base + crash_size;
6bee83d29d2e09 Baoquan He   2023-08-29  390  	}
6bee83d29d2e09 Baoquan He   2023-08-29  391  
6bee83d29d2e09 Baoquan He   2023-08-29  392  	if (high) {
6bee83d29d2e09 Baoquan He   2023-08-29  393  		search_base = CRASH_ADDR_LOW_MAX;
6bee83d29d2e09 Baoquan He   2023-08-29  394  		search_end = CRASH_ADDR_HIGH_MAX;
6bee83d29d2e09 Baoquan He   2023-08-29  395  	}
6bee83d29d2e09 Baoquan He   2023-08-29  396  
6bee83d29d2e09 Baoquan He   2023-08-29  397  retry:
6bee83d29d2e09 Baoquan He   2023-08-29  398  	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
6bee83d29d2e09 Baoquan He   2023-08-29  399  					       search_base, search_end);
6bee83d29d2e09 Baoquan He   2023-08-29  400  	if (!crash_base) {
6bee83d29d2e09 Baoquan He   2023-08-29  401  		/*
6bee83d29d2e09 Baoquan He   2023-08-29  402  		 * For crashkernel=size[KMG]@offset[KMG], print out failure
6bee83d29d2e09 Baoquan He   2023-08-29  403  		 * message if can't reserve the specified region.
6bee83d29d2e09 Baoquan He   2023-08-29  404  		 */
6bee83d29d2e09 Baoquan He   2023-08-29  405  		if (fixed_base) {
6bee83d29d2e09 Baoquan He   2023-08-29  406  			pr_warn("crashkernel reservation failed - memory is in use.\n");
6bee83d29d2e09 Baoquan He   2023-08-29  407  			return;
6bee83d29d2e09 Baoquan He   2023-08-29  408  		}
6bee83d29d2e09 Baoquan He   2023-08-29  409  
6bee83d29d2e09 Baoquan He   2023-08-29  410  		/*
6bee83d29d2e09 Baoquan He   2023-08-29  411  		 * For crashkernel=size[KMG], if the first attempt was for
6bee83d29d2e09 Baoquan He   2023-08-29  412  		 * low memory, fall back to high memory, the minimum required
6bee83d29d2e09 Baoquan He   2023-08-29  413  		 * low memory will be reserved later.
6bee83d29d2e09 Baoquan He   2023-08-29  414  		 */
6bee83d29d2e09 Baoquan He   2023-08-29  415  		if (!high && search_end == CRASH_ADDR_LOW_MAX) {
6bee83d29d2e09 Baoquan He   2023-08-29  416  			search_end = CRASH_ADDR_HIGH_MAX;
6bee83d29d2e09 Baoquan He   2023-08-29  417  			search_base = CRASH_ADDR_LOW_MAX;
6bee83d29d2e09 Baoquan He   2023-08-29  418  			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
6bee83d29d2e09 Baoquan He   2023-08-29  419  			goto retry;
6bee83d29d2e09 Baoquan He   2023-08-29  420  		}
6bee83d29d2e09 Baoquan He   2023-08-29  421  
6bee83d29d2e09 Baoquan He   2023-08-29  422  		/*
6bee83d29d2e09 Baoquan He   2023-08-29  423  		 * For crashkernel=size[KMG],high, if the first attempt was
6bee83d29d2e09 Baoquan He   2023-08-29  424  		 * for high memory, fall back to low memory.
6bee83d29d2e09 Baoquan He   2023-08-29  425  		 */
6bee83d29d2e09 Baoquan He   2023-08-29  426  		if (high && search_end == CRASH_ADDR_HIGH_MAX) {
6bee83d29d2e09 Baoquan He   2023-08-29  427  			search_end = CRASH_ADDR_LOW_MAX;
6bee83d29d2e09 Baoquan He   2023-08-29  428  			search_base = 0;
6bee83d29d2e09 Baoquan He   2023-08-29  429  			goto retry;
6bee83d29d2e09 Baoquan He   2023-08-29  430  		}
6bee83d29d2e09 Baoquan He   2023-08-29  431  		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
6bee83d29d2e09 Baoquan He   2023-08-29  432  			crash_size);
6bee83d29d2e09 Baoquan He   2023-08-29  433  		return;
6bee83d29d2e09 Baoquan He   2023-08-29  434  	}
6bee83d29d2e09 Baoquan He   2023-08-29  435  
6bee83d29d2e09 Baoquan He   2023-08-29  436  	if ((crash_base > CRASH_ADDR_LOW_MAX) &&
6bee83d29d2e09 Baoquan He   2023-08-29  437  	     crash_low_size && reserve_crashkernel_low(crash_low_size)) {
6bee83d29d2e09 Baoquan He   2023-08-29  438  		memblock_phys_free(crash_base, crash_size);
6bee83d29d2e09 Baoquan He   2023-08-29  439  		return;
6bee83d29d2e09 Baoquan He   2023-08-29  440  	}
6bee83d29d2e09 Baoquan He   2023-08-29  441  
6bee83d29d2e09 Baoquan He   2023-08-29  442  	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
6bee83d29d2e09 Baoquan He   2023-08-29  443  		crash_base, crash_base + crash_size, crash_size >> 20);
6bee83d29d2e09 Baoquan He   2023-08-29  444  
6bee83d29d2e09 Baoquan He   2023-08-29  445  	/*
6bee83d29d2e09 Baoquan He   2023-08-29  446  	 * The crashkernel memory will be removed from the kernel linear
6bee83d29d2e09 Baoquan He   2023-08-29  447  	 * map. Inform kmemleak so that it won't try to access it.
6bee83d29d2e09 Baoquan He   2023-08-29  448  	 */
6bee83d29d2e09 Baoquan He   2023-08-29  449  	kmemleak_ignore_phys(crash_base);
6bee83d29d2e09 Baoquan He   2023-08-29  450  	if (crashk_low_res.end)
6bee83d29d2e09 Baoquan He   2023-08-29  451  		kmemleak_ignore_phys(crashk_low_res.start);
6bee83d29d2e09 Baoquan He   2023-08-29  452  
6bee83d29d2e09 Baoquan He   2023-08-29 @453  	crashk_res.start = crash_base;
6bee83d29d2e09 Baoquan He   2023-08-29  454  	crashk_res.end = crash_base + crash_size - 1;
6bee83d29d2e09 Baoquan He   2023-08-29  455  	insert_resource(&iomem_resource, &crashk_res);
6bee83d29d2e09 Baoquan He   2023-08-29  456  }
6bee83d29d2e09 Baoquan He   2023-08-29  457  #endif
6bee83d29d2e09 Baoquan He   2023-08-29  458  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki


From graf at amazon.de  Wed Aug 30 00:20:34 2023
From: graf at amazon.de (Alexander Graf)
Date: Wed, 30 Aug 2023 09:20:34 +0200
Subject: [RFC PATCH] Introduce persistent memory pool
In-Reply-To: <20230829220740.GA26605@skinsburskii.>
References: <64e7cbf7.050a0220.114c7.b70dSMTPIN_ADDED_BROKEN@mx.google.com>
 <2023082506-enchanted-tripping-d1d5@gregkh>
 <c26ad989dcc6737dd295e980c78ef53740098810.camel@amazon.com>
 <20230823024500.GA25462@skinsburskii.>
 <e0ed9fb9-8e7a-44ad-976a-27362f6e537a@amazon.de>
 <20230829220740.GA26605@skinsburskii.>
Message-ID: <82416797-01aa-471a-a737-471297e37c4c@amazon.de>

Hi Stanislav,

On 30.08.23 00:07, Stanislav Kinsburskii wrote:
>
> On Mon, Aug 28, 2023 at 10:50:19PM +0200, Alexander Graf wrote:
>> +kexec, iommu, kvm
>>
>> On 23.08.23 04:45, Stanislav Kinsburskii wrote:
>>> +akpm, +linux-mm
>>>
>>> On Fri, Aug 25, 2023 at 01:32:40PM +0000, Gowans, James wrote:
>>>> On Fri, 2023-08-25 at 10:05 +0200, Greg Kroah-Hartman wrote:
>>>>
>>>> Thanks for adding me to this thread Greg!
>>>>
>>>>> On Tue, Aug 22, 2023 at 11:34:34AM -0700, Stanislav Kinsburskii wrote:
>>>>>> This patch addresses the need for a memory allocator dedicated to
>>>>>> persistent memory within the kernel. This allocator will preserve
>>>>>> kernel-specific states like DMA passthrough device states, IOMMU state, and
>>>>>> more across kexec.
>>>>>> The proposed solution offers a foundational implementation for potential
>>>>>> custom solutions that might follow. Though the implementation is
>>>>>> intentionally kept concise and straightforward to foster discussion and
>>>>>> feedback, it's fully functional in its current state.
>>>> Hi Stanislav, it looks like we're working on similar things. I'm looking
>>>> to develop a mechanism to support hypervisor live update for when KVM is
>>>> running VMs with PCI device passthrough. VMs with device passthrough
>>>> also necessitates passing and re-hydrating IOMMU state so that DMA can
>>>> continue during live update.
>>>>
>>>> Planning on having an LPC session on this topic:
>>>> https://lpc.events/event/17/abstracts/1629/ (currently it's only a
>>>> submitted abstract so not sure if visible, hopefully it will be soon).
>>>>
>>>> We are looking at implementing persistence across kexec via an in-memory
>>>> filesystem on top of reserved memory. This would have files for anything
>>>> that needs to be persisted. That includes files for IOMMU pgtables, for
>>>> guest memory or userspace-accessible memory.
>>>>
>>>> It may be nice to solve all kexec persistence requirements with one
>>>> solution, but we can consider IOMMU separately. There are at least three
>>>> ways that this can be done:
>>>> a) carving out reserved memory for pgtables. This is done by your
>>>> proposal here, as well as my suggestion of a filesystem.
>>>> b) pre/post kexec hooks for drivers to serialise state and pass it
>>>> across in a structured format from old to new kernel.
>>>> c) Reconstructing IOMMU state in the new kernel by starting at the
>>>> hardware registers and walking the page tables. No state passing needed.
>>>>
>>>> Have you considered option (b) and (c) here? One of the implications of
>>>> (b) and (c) are that they would need to hook into the buddy allocator
>>>> really early to be able to carve out the reconstructed page tables
>>>> before the allocator is used. Similar to how pkram [0] hooks in early to
>>>> carve out pages used for its filesystem.
>>>>
>>> Hi James,
>>>
>>> We are indeed working on similar things, so thanks for chiming in.
>>> I've seen pkram proposal as well as your comments there.
>>>
>>> I think (b) will need some persistent-over-kexec memory to pass the
>>> state across kexec as well as some key-value store persisted as well.
>>> And the proposed persistent memory pool is aimed exactly for this
>>> purpose.
>>> Or do you imply some other way to pass driver's data accross kexec?
>>
>> If I had to build this, I'd probably do it just like device tree passing on
>> ARM. It's a single, physically contiguous blob of data whose entry point you
>> pass to the target kernel. IIRC ACPI passing works similarly. This would
>> just be one more opaque data structure that then needs very strict
>> versioning and forward/backward compat guarantees.
>>
> Device tree or ACPI are options indeed. However AFAIU in case of DT user
> space has to involved into the picture to modify and complie it, while
> ACPI isn't flexible or easily extendable.
> Also, AFAIU both these standards were designed with passing
> hardware-specific data in mind from bootstrap software to an OS kernel
> and thus were never really intended to be used for creating a persistent
> state accross kexec.
> To me, an attempt to use either of them to pass kernel-specific data looks
> like an abuse (or misuse) excused by the simplicity of implementation.


What I was describing above is that the Linux boot protocol already has 
natural ways to pass a DT (arm) or set of ACPI tables (x86) to the 
target kernel. Whatever we do here should either piggy back on top of 
those natural mechanisms (e.g. /chosen node in DT) or be on the same 
level (e.g. pass DT in one register, pass metadata structure in another 
register).

When it comes to the actual content of the metadata, I'm personally also 
leaning towards DT. We already have libfdt inside the kernel. It gives 
is a very simple, well understood structured file format that you can 
extend, version, etc etc. And the kernel has mechanisms to modify fdt 
contents.


>
>>> I dind't consider (c) yet, thanks for for the pointer.
>>>
>>> I have a question in this scope: how is PCI devices registers state is persisted
>>> across kexec with the files system you are working on? I.e. how does
>>> driver know, that the device shouldn't not be reinitialized?
>>
>> The easiest way to do it initially would be kernel command line options that
>> hack up the drivers. But I suppose depending on the option we go with, you
>> can also use the respective "natural" path:
>>
>> (a) A special metadata file that explains the state to the driver
>> (b) An entry in the structured file format that explains the state to the
>> target driver
>> (c) Compatible target drivers try to enumerate state from the target
>> device's register file
>>
> Command line option is the simplest way to go indeed, but from my POV
> it's good only for pointing to a particualr object, which is persisted
> somehow else. But it we have a persistence mechanism, then I think we
> can make another step forward and don't use command line at all (which
> is a bit cumbersome and errorprone due to it's human-readable and
> serialized nature).
>
> I'm leaning towards some kind of "natural" path you mentioned... I guess
> I'm a bit confused with the word "file" here, as it sounds line it
> implies a file system driver, and I'm not sure that's what we want for
> driver specific data.


Oh, I was just referring to the set of registers a device exposes :).


>
>>>>>> Potential applications include:
>>>>>>
>>>>>>     1. Allowing various in-kernel entities to allocate persistent pages from
>>>>>>        a singular memory pool, eliminating the need for multiple region
>>>>>>        reservations.
>>>>>>
>>>>>>     2. For in-kernel components that require the allocation address to be
>>>>>>        available on kernel kexec, this address can be exposed to user space and
>>>>>>        then passed via the command line.
>>>> Do you have specific examples of other state that needs to be passed
>>>> across? Trying to see whether tailoring specifically to the IOMMU case
>>>> is okay. Conceptually IOMMU state can be reconstructed starting with
>>>> hardware registers, not needing reserved memory. Other use-cases may not
>>>> have this option.
>>>>
>>> Well, basically it's IOMMU state and PCI devices to skip/avoid
>>> initializing.
>>> I bet there can be other misc (and unrelated things) like persistent
>>> filesystems, block devices, etc. But I don't have a solid set of use
>>> cases to present.
>>
>> Would be great if you could think through the problem space until LPC so we
>> can have a solid conversation there :)
>>
> Yeah, I have a few ideas I'll try to implement and share before LPC.
> Unfortunatelly I'm not planning to attend it this year, so this
> conversation will be without me.
> But I'll do my best to provide as much content to discuss as I can.


Thanks, looking forward to it! :)


Alex




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879



From bhe at redhat.com  Wed Aug 30 04:39:05 2023
From: bhe at redhat.com (Baoquan He)
Date: Wed, 30 Aug 2023 19:39:05 +0800
Subject: [PATCH v2 6/8] x86: kdump: use generic interface to simplify
 crashkernel reservation code
In-Reply-To: <202308300910.e0i4piJT-lkp@intel.com>
References: <20230829121610.138107-7-bhe@redhat.com>
 <202308300910.e0i4piJT-lkp@intel.com>
Message-ID: <ZO8qWQg23JT2lWNg@MiWiFi-R3L-srv>

On 08/30/23 at 09:49am, kernel test robot wrote:
> Hi Baoquan,
> 
> kernel test robot noticed the following build errors:
> 
> [auto build test ERROR on arm64/for-next/core]
> [also build test ERROR on tip/x86/core powerpc/next powerpc/fixes v6.5]
> [cannot apply to linus/master next-20230829]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch#_base_tree_information]
> 
> url:    https://github.com/intel-lab-lkp/linux/commits/Baoquan-He/crash_core-c-remove-unnecessary-parameter-of-function/20230829-201942
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
> patch link:    https://lore.kernel.org/r/20230829121610.138107-7-bhe%40redhat.com
> patch subject: [PATCH v2 6/8] x86: kdump: use generic interface to simplify crashkernel reservation code
> config: x86_64-randconfig-r022-20230830 (https://download.01.org/0day-ci/archive/20230830/202308300910.e0i4piJT-lkp at intel.com/config)
> compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
> reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20230830/202308300910.e0i4piJT-lkp at intel.com/reproduce)
> 
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <lkp at intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202308300910.e0i4piJT-lkp at intel.com/
> 
> All errors (new ones prefixed by >>):
> 
>    ld: vmlinux.o: in function `reserve_crashkernel_low':
>    kernel/crash_core.c:369: undefined reference to `crashk_low_res'
>    ld: kernel/crash_core.c:369: undefined reference to `crashk_low_res'
>    ld: kernel/crash_core.c:370: undefined reference to `crashk_low_res'
>    ld: kernel/crash_core.c:369: undefined reference to `crashk_low_res'
>    ld: kernel/crash_core.c:370: undefined reference to `crashk_low_res'
>    ld: vmlinux.o:kernel/crash_core.c:371: more undefined references to `crashk_low_res' follow
>    ld: vmlinux.o: in function `reserve_crashkernel_generic':
> >> kernel/crash_core.c:453: undefined reference to `crashk_res'
> >> ld: kernel/crash_core.c:453: undefined reference to `crashk_res'
>    ld: kernel/crash_core.c:454: undefined reference to `crashk_res'
> >> ld: kernel/crash_core.c:453: undefined reference to `crashk_res'
>    ld: kernel/crash_core.c:454: undefined reference to `crashk_res'
>    ld: vmlinux.o:kernel/crash_core.c:454: more undefined references to `crashk_res' follow

Thanks for reporting. This one has the same root cause as the i386-randconfig
building. It's because crashk_res|crashk_low_res are defined in
kernel/kexec_core.c, but referenced in generic reservation code in
crash_core.c, while in this lkp's randconfig, CONFIG_KEXEC_CORE is
unset, CONFIG_CRASH_CORE=on. Then undefined reference to `crashk_res'
and `crashk_low_res' are reported.

Below patch can fix it. I will post v3 to contain this.

>From 5a5bda3b5b1e370b789489d87516c8f1fb966c46 Mon Sep 17 00:00:00 2001
From: Baoquan He <bhe at redhat.com>
Date: Tue, 29 Aug 2023 22:38:15 -0400
Subject: [PATCH] crash_core: move crashk_res and crashk_low_res definition
 into crash_core.c
Content-type: text/plain

Both crashk_res and crashk_low_res are used to mark the reserved
crashkernel regions in iomem_resource tree. And later the generic
crashkernel resrvation which references them will be added into
crash_core.c. So move crashk_res and crashk_low_res definition into
crash_core.c to avoid compiling error when CONFIG_CRASH_CORE=on while
CONFIG_KEXEC_CORE is unset.

Signed-off-by: Baoquan He <bhe at redhat.com>
---
 include/linux/crash_core.h |  5 +++++
 include/linux/kexec.h      |  4 ----
 kernel/crash_core.c        | 16 ++++++++++++++++
 kernel/kexec_core.c        | 17 -----------------
 4 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
index d64006c4bd43..daa87b8730d3 100644
--- a/include/linux/crash_core.h
+++ b/include/linux/crash_core.h
@@ -9,6 +9,11 @@
 #include <asm/crash_core.h>
 #endif
 
+/* Location of a reserved region to hold the crash kernel.
+ */
+extern struct resource crashk_res;
+extern struct resource crashk_low_res;
+
 #define CRASH_CORE_NOTE_NAME	   "CORE"
 #define CRASH_CORE_NOTE_HEAD_BYTES ALIGN(sizeof(struct elf_note), 4)
 #define CRASH_CORE_NOTE_NAME_BYTES ALIGN(sizeof(CRASH_CORE_NOTE_NAME), 4)
diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index 22b5cd24f581..e762b0435c39 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -22,10 +22,6 @@
 #include <uapi/linux/kexec.h>
 #include <linux/verification.h>
 
-/* Location of a reserved region to hold the crash kernel.
- */
-extern struct resource crashk_res;
-extern struct resource crashk_low_res;
 extern note_buf_t __percpu *crash_notes;
 
 #ifdef CONFIG_KEXEC_CORE
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index 61a8ea3b23a2..d76e7280c651 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -28,6 +28,22 @@ u32 *vmcoreinfo_note;
 /* trusted vmcoreinfo, e.g. we can make a copy in the crash memory */
 static unsigned char *vmcoreinfo_data_safecopy;
 
+/* Location of the reserved area for the crash kernel */
+struct resource crashk_res = {
+	.name  = "Crash kernel",
+	.start = 0,
+	.end   = 0,
+	.flags = IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM,
+	.desc  = IORES_DESC_CRASH_KERNEL
+};
+struct resource crashk_low_res = {
+	.name  = "Crash kernel",
+	.start = 0,
+	.end   = 0,
+	.flags = IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM,
+	.desc  = IORES_DESC_CRASH_KERNEL
+};
+
 /*
  * parsing the "crashkernel" commandline
  *
diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index e2f2574d8b74..03ee65546df6 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -55,23 +55,6 @@ note_buf_t __percpu *crash_notes;
 /* Flag to indicate we are going to kexec a new kernel */
 bool kexec_in_progress = false;
 
-
-/* Location of the reserved area for the crash kernel */
-struct resource crashk_res = {
-	.name  = "Crash kernel",
-	.start = 0,
-	.end   = 0,
-	.flags = IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM,
-	.desc  = IORES_DESC_CRASH_KERNEL
-};
-struct resource crashk_low_res = {
-	.name  = "Crash kernel",
-	.start = 0,
-	.end   = 0,
-	.flags = IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM,
-	.desc  = IORES_DESC_CRASH_KERNEL
-};
-
 int kexec_should_crash(struct task_struct *p)
 {
 	/*
-- 
2.41.0



From patchwork-bot+linux-riscv at kernel.org  Wed Aug 30 06:20:34 2023
From: patchwork-bot+linux-riscv at kernel.org (patchwork-bot+linux-riscv at kernel.org)
Date: Wed, 30 Aug 2023 13:20:34 +0000
Subject: [PATCH -next v9 0/2] support allocating crashkernel above 4G
 explicitly on riscv
In-Reply-To: <20230726175000.2536220-1-chenjiahao16@huawei.com>
References: <20230726175000.2536220-1-chenjiahao16@huawei.com>
Message-ID: <169340163484.19859.603263749901327950.git-patchwork-notify@kernel.org>

Hello:

This series was applied to riscv/linux.git (for-next)
by Palmer Dabbelt <palmer at rivosinc.com>:

On Wed, 26 Jul 2023 17:49:58 +0000 you wrote:
> On riscv, the current crash kernel allocation logic is trying to
> allocate within 32bit addressible memory region by default, if
> failed, try to allocate without 4G restriction.
> 
> In need of saving DMA zone memory while allocating a relatively large
> crash kernel region, allocating the reserved memory top down in
> high memory, without overlapping the DMA zone, is a mature solution.
> Hence this patchset introduces the parameter option crashkernel=X,[high,low].
> 
> [...]

Here is the summary with links:
  - [-next,v9,1/2] riscv: kdump: Implement crashkernel=X,[high,low]
    https://git.kernel.org/riscv/c/5882e5acf18d
  - [-next,v9,2/2] docs: kdump: Update the crashkernel description for riscv
    https://git.kernel.org/riscv/c/33f0dd973d4e

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html




From pstanner at redhat.com  Wed Aug 30 06:45:51 2023
From: pstanner at redhat.com (Philipp Stanner)
Date: Wed, 30 Aug 2023 15:45:51 +0200
Subject: [PATCH 0/5] Introduce new wrappers to copy user-arrays
Message-ID: <cover.1693386602.git.pstanner@redhat.com>

Hi!

David Airlie suggested that we could implement new wrappers around
(v)memdup_user() for duplicating user arrays.

This small patch series first implements the two new wrapper functions
memdup_array_user() and vmemdup_array_user(). They calculate the
array-sizes safely, i.e., they return an error in case of an overflow.

It then implements the new wrappers in two components in kernel/ and two
in the drm-subsystem.

In total, there are 18 files in the kernel that use (v)memdup_user() to
duplicate arrays. My plan is to provide patches for the other 14
successively once this series has been merged.

P.

Philipp Stanner (5):
  string.h: add array-wrappers for (v)memdup_user()
  kernel: kexec: copy user-array safely
  kernel: watch_queue: copy user-array safely
  drm_lease.c: copy user-array safely
  drm: vmgfx_surface.c: copy user-array safely

 drivers/gpu/drm/drm_lease.c             |  4 +--
 drivers/gpu/drm/vmwgfx/vmwgfx_surface.c |  4 +--
 include/linux/string.h                  | 42 +++++++++++++++++++++++++
 kernel/kexec.c                          |  2 +-
 kernel/watch_queue.c                    |  2 +-
 5 files changed, 48 insertions(+), 6 deletions(-)

-- 
2.41.0



From pstanner at redhat.com  Wed Aug 30 06:45:52 2023
From: pstanner at redhat.com (Philipp Stanner)
Date: Wed, 30 Aug 2023 15:45:52 +0200
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <cover.1693386602.git.pstanner@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com>
Message-ID: <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>

Currently, user array duplications are sometimes done without an
overflow check. Sometimes the checks are done manually; sometimes the
array size is calculated with array_size() and sometimes by calculating
n * size directly in code.

Introduce wrappers for arrays for memdup_user() and vmemdup_user() to
provide a standardized and safe way for duplicating user arrays.

This is both for new code as well as replacing usage of (v)memdup_user()
in existing code that uses, e.g., n * size to calculate array sizes.

Suggested-by: David Airlie <airlied at redhat.com>
Signed-off-by: Philipp Stanner <pstanner at redhat.com>
---
 include/linux/string.h | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/include/linux/string.h b/include/linux/string.h
index dbfc66400050..0e8e7a40bae7 100644
--- a/include/linux/string.h
+++ b/include/linux/string.h
@@ -6,6 +6,8 @@
 #include <linux/types.h>	/* for size_t */
 #include <linux/stddef.h>	/* for NULL */
 #include <linux/errno.h>	/* for E2BIG */
+#include <linux/overflow.h>	/* for check_mul_overflow() */
+#include <linux/err.h>		/* for ERR_PTR() */
 #include <linux/stdarg.h>
 #include <uapi/linux/string.h>
 
@@ -14,6 +16,46 @@ extern void *memdup_user(const void __user *, size_t);
 extern void *vmemdup_user(const void __user *, size_t);
 extern void *memdup_user_nul(const void __user *, size_t);
 
+/**
+ * memdup_array_user - duplicate array from user space
+ *
+ * @src: source address in user space
+ * @n: number of array members to copy
+ * @size: size of one array member
+ *
+ * Return: an ERR_PTR() on failure.  Result is physically
+ * contiguous, to be freed by kfree().
+ */
+static inline void *memdup_array_user(const void __user *src, size_t n, size_t size)
+{
+	size_t nbytes;
+
+	if (unlikely(check_mul_overflow(n, size, &nbytes)))
+		return ERR_PTR(-EINVAL);
+
+	return memdup_user(src, nbytes);
+}
+
+/**
+ * vmemdup_array_user - duplicate array from user space
+ *
+ * @src: source address in user space
+ * @n: number of array members to copy
+ * @size: size of one array member
+ *
+ * Return: an ERR_PTR() on failure.  Result may be not
+ * physically contiguous.  Use kvfree() to free.
+ */
+static inline void *vmemdup_array_user(const void __user *src, size_t n, size_t size)
+{
+	size_t nbytes;
+
+	if (unlikely(check_mul_overflow(n, size, &nbytes)))
+		return ERR_PTR(-EINVAL);
+
+	return vmemdup_user(src, nbytes);
+}
+
 /*
  * Include machine specific inline routines
  */
-- 
2.41.0



From pstanner at redhat.com  Wed Aug 30 06:45:53 2023
From: pstanner at redhat.com (Philipp Stanner)
Date: Wed, 30 Aug 2023 15:45:53 +0200
Subject: [PATCH 2/5] kernel: kexec: copy user-array safely
In-Reply-To: <cover.1693386602.git.pstanner@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com>
Message-ID: <0a3f1bca273792edbdc41b64c454a6f7f499c9a8.1693386602.git.pstanner@redhat.com>

Currently, there is no overflow-check with memdup_user().

Use the new function memdup_array_user() instead of memdup_user() for
duplicating the user-space array safely.

Suggested-by: David Airlie <airlied at redhat.com>
Signed-off-by: Philipp Stanner <pstanner at redhat.com>
---
 kernel/kexec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/kexec.c b/kernel/kexec.c
index 92d301f98776..f6067c1bb089 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -242,7 +242,7 @@ SYSCALL_DEFINE4(kexec_load, unsigned long, entry, unsigned long, nr_segments,
 		((flags & KEXEC_ARCH_MASK) != KEXEC_ARCH_DEFAULT))
 		return -EINVAL;
 
-	ksegments = memdup_user(segments, nr_segments * sizeof(ksegments[0]));
+	ksegments = memdup_array_user(segments, nr_segments, sizeof(ksegments[0]));
 	if (IS_ERR(ksegments))
 		return PTR_ERR(ksegments);
 
-- 
2.41.0



From pstanner at redhat.com  Wed Aug 30 06:45:54 2023
From: pstanner at redhat.com (Philipp Stanner)
Date: Wed, 30 Aug 2023 15:45:54 +0200
Subject: [PATCH 3/5] kernel: watch_queue: copy user-array safely
In-Reply-To: <cover.1693386602.git.pstanner@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com>
Message-ID: <4168bc2d9313d5d1a1a2b9ca0b9fc2990d499506.1693386602.git.pstanner@redhat.com>

Currently, there is no overflow-check with memdup_user().

Use the new function memdup_array_user() instead of memdup_user() for
duplicating the user-space array safely.

Suggested-by: David Airlie <airlied at redhat.com>
Signed-off-by: Philipp Stanner <pstanner at redhat.com>
---
 kernel/watch_queue.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/watch_queue.c b/kernel/watch_queue.c
index d0b6b390ee42..778b4056700f 100644
--- a/kernel/watch_queue.c
+++ b/kernel/watch_queue.c
@@ -331,7 +331,7 @@ long watch_queue_set_filter(struct pipe_inode_info *pipe,
 	    filter.__reserved != 0)
 		return -EINVAL;
 
-	tf = memdup_user(_filter->filters, filter.nr_filters * sizeof(*tf));
+	tf = memdup_array_user(_filter->filters, filter.nr_filters, sizeof(*tf));
 	if (IS_ERR(tf))
 		return PTR_ERR(tf);
 
-- 
2.41.0



From pstanner at redhat.com  Wed Aug 30 06:45:55 2023
From: pstanner at redhat.com (Philipp Stanner)
Date: Wed, 30 Aug 2023 15:45:55 +0200
Subject: [PATCH 4/5] drm_lease.c: copy user-array safely
In-Reply-To: <cover.1693386602.git.pstanner@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com>
Message-ID: <34e14523c9cea13e2f63afabf2938c89d2e84776.1693386602.git.pstanner@redhat.com>

Currently, there is no overflow-check with memdup_user().

Use the new function memdup_array_user() instead of memdup_user() for
duplicating the user-space array safely.

Suggested-by: David Airlie <airlied at redhat.com>
Signed-off-by: Philipp Stanner <pstanner at redhat.com>
---
 drivers/gpu/drm/drm_lease.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_lease.c b/drivers/gpu/drm/drm_lease.c
index 150fe1555068..94375c6a5425 100644
--- a/drivers/gpu/drm/drm_lease.c
+++ b/drivers/gpu/drm/drm_lease.c
@@ -510,8 +510,8 @@ int drm_mode_create_lease_ioctl(struct drm_device *dev,
 	/* Handle leased objects, if any */
 	idr_init(&leases);
 	if (object_count != 0) {
-		object_ids = memdup_user(u64_to_user_ptr(cl->object_ids),
-					 array_size(object_count, sizeof(__u32)));
+		object_ids = memdup_array_user(u64_to_user_ptr(cl->object_ids),
+					       object_count, sizeof(__u32));
 		if (IS_ERR(object_ids)) {
 			ret = PTR_ERR(object_ids);
 			idr_destroy(&leases);
-- 
2.41.0



From pstanner at redhat.com  Wed Aug 30 06:45:56 2023
From: pstanner at redhat.com (Philipp Stanner)
Date: Wed, 30 Aug 2023 15:45:56 +0200
Subject: [PATCH 5/5] drm: vmgfx_surface.c: copy user-array safely
In-Reply-To: <cover.1693386602.git.pstanner@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com>
Message-ID: <6f9397d6af919b7e8325c2945385c13aaa3dc5f1.1693386602.git.pstanner@redhat.com>

Currently, there is no overflow-check with memdup_user().

Use the new function memdup_array_user() instead of memdup_user() for
duplicating the user-space array safely.

Suggested-by: David Airlie <airlied at redhat.com>
Signed-off-by: Philipp Stanner <pstanner at redhat.com>
---
 drivers/gpu/drm/vmwgfx/vmwgfx_surface.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
index 5db403ee8261..9be185b094cb 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c
@@ -777,9 +777,9 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data,
 	       sizeof(metadata->mip_levels));
 	metadata->num_sizes = num_sizes;
 	metadata->sizes =
-		memdup_user((struct drm_vmw_size __user *)(unsigned long)
+		memdup_array_user((struct drm_vmw_size __user *)(unsigned long)
 			    req->size_addr,
-			    sizeof(*metadata->sizes) * metadata->num_sizes);
+			    metadata->num_sizes, sizeof(*metadata->sizes));
 	if (IS_ERR(metadata->sizes)) {
 		ret = PTR_ERR(metadata->sizes);
 		goto out_no_sizes;
-- 
2.41.0



From andy.shevchenko at gmail.com  Wed Aug 30 07:11:21 2023
From: andy.shevchenko at gmail.com (Andy Shevchenko)
Date: Wed, 30 Aug 2023 17:11:21 +0300
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com> <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
Message-ID: <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>

On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner <pstanner at redhat.com> wrote:
>
> Currently, user array duplications are sometimes done without an
> overflow check. Sometimes the checks are done manually; sometimes the
> array size is calculated with array_size() and sometimes by calculating
> n * size directly in code.
>
> Introduce wrappers for arrays for memdup_user() and vmemdup_user() to
> provide a standardized and safe way for duplicating user arrays.
>
> This is both for new code as well as replacing usage of (v)memdup_user()
> in existing code that uses, e.g., n * size to calculate array sizes.

...

> --- a/include/linux/string.h
> +++ b/include/linux/string.h

I'm wondering if this has no side-effects as string.h/string.c IIRC is
used also for early stages where some of the APIs are not available.

> @@ -6,6 +6,8 @@
>  #include <linux/types.h>       /* for size_t */
>  #include <linux/stddef.h>      /* for NULL */
>  #include <linux/errno.h>       /* for E2BIG */
> +#include <linux/overflow.h>    /* for check_mul_overflow() */
> +#include <linux/err.h>         /* for ERR_PTR() */

Can we preserve order (to some extent)?

>  #include <linux/stdarg.h>
>  #include <uapi/linux/string.h>

...

> +/**
> + * memdup_array_user - duplicate array from user space

> + *

Do we need this blank line?

> + * @src: source address in user space
> + * @n: number of array members to copy
> + * @size: size of one array member
> + *
> + * Return: an ERR_PTR() on failure.  Result is physically
> + * contiguous, to be freed by kfree().
> + */

...

> +/**
> + * vmemdup_array_user - duplicate array from user space

> + *

Redundant?

> + * @src: source address in user space
> + * @n: number of array members to copy
> + * @size: size of one array member
> + *
> + * Return: an ERR_PTR() on failure.  Result may be not
> + * physically contiguous.  Use kvfree() to free.
> + */

-- 
With Best Regards,
Andy Shevchenko


From andy.shevchenko at gmail.com  Wed Aug 30 07:15:02 2023
From: andy.shevchenko at gmail.com (Andy Shevchenko)
Date: Wed, 30 Aug 2023 17:15:02 +0300
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com> <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
Message-ID: <CAHp75Vc72uuNvFxq4TGO=nXoGhbqVxyqvQ67ioL8ajtZwMUBgA@mail.gmail.com>

On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner <pstanner at redhat.com> wrote:

> +       if (unlikely(check_mul_overflow(n, size, &nbytes)))
> +               return ERR_PTR(-EINVAL);

> +       if (unlikely(check_mul_overflow(n, size, &nbytes)))
> +               return ERR_PTR(-EINVAL);

Btw, why not -EOVERFLOW ?

-- 
With Best Regards,
Andy Shevchenko


From pstanner at redhat.com  Wed Aug 30 07:19:32 2023
From: pstanner at redhat.com (pstanner at redhat.com)
Date: Wed, 30 Aug 2023 16:19:32 +0200
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>
References: <cover.1693386602.git.pstanner@redhat.com>
	 <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
	 <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>
Message-ID: <721a70c347d82931d12e5b75b19d132f82ee5ed2.camel@redhat.com>

On Wed, 2023-08-30 at 17:11 +0300, Andy Shevchenko wrote:
> On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner <pstanner at redhat.com>
> wrote:
> > 
> > Currently, user array duplications are sometimes done without an
> > overflow check. Sometimes the checks are done manually; sometimes
> > the
> > array size is calculated with array_size() and sometimes by
> > calculating
> > n * size directly in code.
> > 
> > Introduce wrappers for arrays for memdup_user() and vmemdup_user()
> > to
> > provide a standardized and safe way for duplicating user arrays.
> > 
> > This is both for new code as well as replacing usage of
> > (v)memdup_user()
> > in existing code that uses, e.g., n * size to calculate array
> > sizes.
> 
> ...
> 
> > --- a/include/linux/string.h
> > +++ b/include/linux/string.h
> 
> I'm wondering if this has no side-effects as string.h/string.c IIRC
> is
> used also for early stages where some of the APIs are not available.
> 
> > @@ -6,6 +6,8 @@
> > ?#include <linux/types.h>?????? /* for size_t */
> > ?#include <linux/stddef.h>????? /* for NULL */
> > ?#include <linux/errno.h>?????? /* for E2BIG */
> > +#include <linux/overflow.h>??? /* for check_mul_overflow() */
> > +#include <linux/err.h>???????? /* for ERR_PTR() */
> 
> Can we preserve order (to some extent)?

Sure. I just put it there so the comments build a congruent block.
Which order would you prefer?

> 
> > ?#include <linux/stdarg.h>
> > ?#include <uapi/linux/string.h>
> 
> ...
> 
> > +/**
> > + * memdup_array_user - duplicate array from user space
> 
> > + *
> 
> Do we need this blank line?

I more or less directly copied the docstring format from the original
functions (v)memdup_user() in mm/util.c
I guess this is common style?

> 
> > + * @src: source address in user space
> > + * @n: number of array members to copy
> > + * @size: size of one array member
> > + *
> > + * Return: an ERR_PTR() on failure.? Result is physically
> > + * contiguous, to be freed by kfree().
> > + */
> 
> ...
> 
> > +/**
> > + * vmemdup_array_user - duplicate array from user space
> 
> > + *
> 
> Redundant?

No, there are two functions:
 * memdup_array_user()
 * vmemdup_array_user()

On the deeper layers they utilize kmalloc() or kvmalloc(),
respectively.


Greetings,
P.

> 
> > + * @src: source address in user space
> > + * @n: number of array members to copy
> > + * @size: size of one array member
> > + *
> > + * Return: an ERR_PTR() on failure.? Result may be not
> > + * physically contiguous.? Use kvfree() to free.
> > + */
> 



From pstanner at redhat.com  Wed Aug 30 07:23:10 2023
From: pstanner at redhat.com (pstanner at redhat.com)
Date: Wed, 30 Aug 2023 16:23:10 +0200
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <CAHp75Vc72uuNvFxq4TGO=nXoGhbqVxyqvQ67ioL8ajtZwMUBgA@mail.gmail.com>
References: <cover.1693386602.git.pstanner@redhat.com>
	 <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
	 <CAHp75Vc72uuNvFxq4TGO=nXoGhbqVxyqvQ67ioL8ajtZwMUBgA@mail.gmail.com>
Message-ID: <a79b3c34a12abc51bc5e840ae3a87ab9a47f7742.camel@redhat.com>

On Wed, 2023-08-30 at 17:15 +0300, Andy Shevchenko wrote:
> On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner <pstanner at redhat.com>
> wrote:
> 
> > +?????? if (unlikely(check_mul_overflow(n, size, &nbytes)))
> > +?????????????? return ERR_PTR(-EINVAL);
> 
> > +?????? if (unlikely(check_mul_overflow(n, size, &nbytes)))
> > +?????????????? return ERR_PTR(-EINVAL);
> 
> Btw, why not -EOVERFLOW ?
> 

Good question, actually.
To be honest I wasn't quite sure which code to pick (-E2BIG was also
once I candidate).

-EINVAL was picked because the idea was that a request overflowing a
size_t could surely be expected to contain an invalid parameter,
because no one would ever request an array _that_ large

?



From andy.shevchenko at gmail.com  Wed Aug 30 07:29:09 2023
From: andy.shevchenko at gmail.com (Andy Shevchenko)
Date: Wed, 30 Aug 2023 17:29:09 +0300
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <721a70c347d82931d12e5b75b19d132f82ee5ed2.camel@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com> <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
 <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com> <721a70c347d82931d12e5b75b19d132f82ee5ed2.camel@redhat.com>
Message-ID: <CAHp75VdS=kSWnz8FzHcdrZPaeZKsQNbzjE9mNN7BDvZA_nQpaA@mail.gmail.com>

On Wed, Aug 30, 2023 at 5:19?PM <pstanner at redhat.com> wrote:
> On Wed, 2023-08-30 at 17:11 +0300, Andy Shevchenko wrote:
> > On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner <pstanner at redhat.com>
> > wrote:

> > > --- a/include/linux/string.h
> > > +++ b/include/linux/string.h
> >
> > I'm wondering if this has no side-effects as string.h/string.c IIRC
> > is
> > used also for early stages where some of the APIs are not available.
> >
> > > @@ -6,6 +6,8 @@
> > >  #include <linux/types.h>       /* for size_t */
> > >  #include <linux/stddef.h>      /* for NULL */
> > >  #include <linux/errno.h>       /* for E2BIG */
> > > +#include <linux/overflow.h>    /* for check_mul_overflow() */
> > > +#include <linux/err.h>         /* for ERR_PTR() */
> >
> > Can we preserve order (to some extent)?
>
> Sure. I just put it there so the comments build a congruent block.
> Which order would you prefer?

Alphabetical.

compiler.h
err.h
overflow.h
...the rest that is a bit unordered...

> > >  #include <linux/stdarg.h>
> > >  #include <uapi/linux/string.h>

...

> > > +/**
> > > + * memdup_array_user - duplicate array from user space
> >
> > > + *
> >
> > Do we need this blank line?
>
> I more or less directly copied the docstring format from the original
> functions (v)memdup_user() in mm/util.c
> I guess this is common style?

I think it's not. But you may grep kernel source tree and tell which
one occurs more often with or without this (unneeded) blank line.

> > > + * @src: source address in user space
> > > + * @n: number of array members to copy
> > > + * @size: size of one array member
> > > + *
> > > + * Return: an ERR_PTR() on failure.  Result is physically
> > > + * contiguous, to be freed by kfree().
> > > + */

...

> > > +/**
> > > + * vmemdup_array_user - duplicate array from user space
> >
> > > + *
> >
> > Redundant?
>
> No, there are two functions:
>  * memdup_array_user()
>  * vmemdup_array_user()
>
> On the deeper layers they utilize kmalloc() or kvmalloc(),
> respectively.

I guess you misunderstood my comment. I was talking about kernel doc
(as in the previous function).

> > > + * @src: source address in user space
> > > + * @n: number of array members to copy
> > > + * @size: size of one array member
> > > + *
> > > + * Return: an ERR_PTR() on failure.  Result may be not
> > > + * physically contiguous.  Use kvfree() to free.
> > > + */


-- 
With Best Regards,
Andy Shevchenko


From kgold at linux.ibm.com  Wed Aug 30 11:06:23 2023
From: kgold at linux.ibm.com (Ken Goldman)
Date: Wed, 30 Aug 2023 14:06:23 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal - network bandwidth
In-Reply-To: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
Message-ID: <5c323243-e22e-dd61-f808-2875654936a6@linux.ibm.com>



On 8/1/2023 3:12 PM, Sush Shringarputale wrote:
> In addition, a large IMA log can add pressure on the network bandwidth when
> the attestation client sends it to remote-attestation-service.

I would not worry too much about network bandwidth.

1. Every solution eventually realizes that sending the entire log each 
time hurts performance.  The verifier will ask the attestor, "give me 
everything since record n", and the number of new entries approaches zero.

2. My benchmarks show that

On the client, the TPM quote time swamps everything else.
On the server, verifying the IMA entry signatures swamps everything else.

The network transfer time is negligible.


From kgold at linux.ibm.com  Wed Aug 30 11:12:27 2023
From: kgold at linux.ibm.com (Ken Goldman)
Date: Wed, 30 Aug 2023 14:12:27 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal - aggregate
In-Reply-To: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
Message-ID: <598fdd62-f4c3-a6dc-ae22-8f5a9e18f570@linux.ibm.com>

On 8/1/2023 3:12 PM, Sush Shringarputale wrote:
> - A user-mode process will trigger the snapshot by opening a file in SysFS
>  ? say /sys/kernel/security/ima/snapshot (referred to as 
> sysk_ima_snapshot_file
>  ? here onwards).
> - The Kernel will get the current TPM PCR values and PCR update counter [2]
>  ? and store them as template data in a new IMA event "snapshot_aggregate".

If this is relying on a user-mode process, is there a concern that the 
process doesn't run. Might it be safer to have the kernel trigger the
snapshot.

PCR reads are not atomic, with each other and with event log appends. 
Is this an issue?

The PCR update counter can change between PCR reads.  What is its purpose?

What is the purpose of the snapshot aggregate?  Since the entire event 
log has to be retained and sent to the verifier, is the aggregate redundant?



From kgold at linux.ibm.com  Wed Aug 30 12:12:39 2023
From: kgold at linux.ibm.com (Ken Goldman)
Date: Wed, 30 Aug 2023 15:12:39 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal - unseal
In-Reply-To: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
Message-ID: <1ef45099-da24-b73f-b33f-6a299c0b1696@linux.ibm.com>

On 8/1/2023 3:12 PM, Sush Shringarputale wrote:

> For remote attestation to work, the service will need to know how to
>  validate the snapshot_aggregate entry in the IMA log.  It will have
> to read the PCR values present in the template data of
> snapshot_aggregate event in the latest IMA log, and ensure that the
> PCR quotes align with the contents of the past UM_snapshot_file(s).
> This will re-establish the chain of trust needed for the device to
> pass remote attestation.  This will also maintain the ability of the
> remote-attestation-service to seal the secrets, if the client-server
>  use TPM unseal mechanism to attest the state of the device.

I think that seal/unseal to IMA PCRs is futile.  Since boot is
multi-threaded, the IMA PCR is unpredictable even when valid.



From pstanner at redhat.com  Wed Aug 30 12:15:28 2023
From: pstanner at redhat.com (pstanner at redhat.com)
Date: Wed, 30 Aug 2023 21:15:28 +0200
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <CAHp75VdS=kSWnz8FzHcdrZPaeZKsQNbzjE9mNN7BDvZA_nQpaA@mail.gmail.com>
References: <cover.1693386602.git.pstanner@redhat.com>
	 <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
	 <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>
	 <721a70c347d82931d12e5b75b19d132f82ee5ed2.camel@redhat.com>
	 <CAHp75VdS=kSWnz8FzHcdrZPaeZKsQNbzjE9mNN7BDvZA_nQpaA@mail.gmail.com>
Message-ID: <a4ba0493965ad32fcf95cd6e2439096668a9ed46.camel@redhat.com>

On Wed, 2023-08-30 at 17:29 +0300, Andy Shevchenko wrote:
> On Wed, Aug 30, 2023 at 5:19?PM <pstanner at redhat.com> wrote:
> > On Wed, 2023-08-30 at 17:11 +0300, Andy Shevchenko wrote:
> > > On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner
> > > <pstanner at redhat.com>
> > > wrote:
> 
> > > > --- a/include/linux/string.h
> > > > +++ b/include/linux/string.h
> > > 
> > > I'm wondering if this has no side-effects as string.h/string.c
> > > IIRC
> > > is
> > > used also for early stages where some of the APIs are not
> > > available.
> > > 
> > > > @@ -6,6 +6,8 @@
> > > > ?#include <linux/types.h>?????? /* for size_t */
> > > > ?#include <linux/stddef.h>????? /* for NULL */
> > > > ?#include <linux/errno.h>?????? /* for E2BIG */
> > > > +#include <linux/overflow.h>??? /* for check_mul_overflow() */
> > > > +#include <linux/err.h>???????? /* for ERR_PTR() */
> > > 
> > > Can we preserve order (to some extent)?
> > 
> > Sure. I just put it there so the comments build a congruent block.
> > Which order would you prefer?
> 
> Alphabetical.
> 
> compiler.h
> err.h
> overflow.h
> ...the rest that is a bit unordered...
> 
> > > > ?#include <linux/stdarg.h>
> > > > ?#include <uapi/linux/string.h>
> 
> ...

I mean I could include my own in a sorted manner ? but the existing
ones are not sorted:

/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _LINUX_STRING_H_
#define _LINUX_STRING_H_

#include <linux/compiler.h>	/* for inline */
#include <linux/types.h>	/* for size_t */
#include <linux/stddef.h>	/* for NULL */
#include <linux/errno.h>	/* for E2BIG */
#include <linux/stdarg.h>
#include <uapi/linux/string.h>

extern char *strndup_user(const char __user *, long);

We could sort them all, but I'd prefer to do that in a separate patch
so that this commit does not make the impression of doing anything else
than including the two new headers

Such a separate patch could also unify the docstring style, see below

> 
> > > > +/**
> > > > + * memdup_array_user - duplicate array from user space
> > > 
> > > > + *
> > > 
> > > Do we need this blank line?
> > 
> > I more or less directly copied the docstring format from the
> > original
> > functions (v)memdup_user() in mm/util.c
> > I guess this is common style?
> 
> I think it's not. But you may grep kernel source tree and tell which
> one occurs more often with or without this (unneeded) blank line.


It seems to be very much mixed. string.h itself is mixed.
When you look at the bottom of string.h, you'll find functions such as
kbasename() that have the extra line.

That's not really a super decisive point for me, though. We can remove
the line I guess


P.


> 
> > > > + * @src: source address in user space
> > > > + * @n: number of array members to copy
> > > > + * @size: size of one array member
> > > > + *
> > > > + * Return: an ERR_PTR() on failure.? Result is physically
> > > > + * contiguous, to be freed by kfree().
> > > > + */
> 
> ...
> 
> > > > +/**
> > > > + * vmemdup_array_user - duplicate array from user space
> > > 
> > > > + *
> > > 
> > > Redundant?
> > 
> > No, there are two functions:
> > ?* memdup_array_user()
> > ?* vmemdup_array_user()
> > 
> > On the deeper layers they utilize kmalloc() or kvmalloc(),
> > respectively.
> 
> I guess you misunderstood my comment. I was talking about kernel doc
> (as in the previous function).
> 
> > > > + * @src: source address in user space
> > > > + * @n: number of array members to copy
> > > > + * @size: size of one array member
> > > > + *
> > > > + * Return: an ERR_PTR() on failure.? Result may be not
> > > > + * physically contiguous.? Use kvfree() to free.
> > > > + */
> 
> 



From zohar at linux.ibm.com  Wed Aug 30 13:25:05 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Wed, 30 Aug 2023 16:25:05 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
	 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
	 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
	 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
	 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
	 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
	 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
	 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
	 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
	 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
Message-ID: <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>

On Tue, 2023-08-29 at 19:15 -0400, Paul Moore wrote:
> On Tue, Aug 29, 2023 at 5:54?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > On Tue, 2023-08-29 at 17:30 -0400, Paul Moore wrote:
> > > On Tue, Aug 29, 2023 at 5:05?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > On Tue, 2023-08-29 at 15:34 -0400, Paul Moore wrote:
> > > > > On Mon, Aug 21, 2023 at 7:08?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > > On Mon, 2023-08-21 at 15:05 -0700, Sush Shringarputale wrote:
> > > > > > > On 8/14/2023 3:02 PM, Mimi Zohar wrote:
> > > > > > > > On Mon, 2023-08-14 at 14:42 -0700, Sush Shringarputale wrote:
> > > > > > > >>> This design seems overly complex and requires synchronization between
> > > > > > > >>> the "snapshot" record and exporting the records from the measurement
> > > > > > > >>> list.  None of this would be necessary if the measurements were copied
> > > > > > > >>> from kernel memory to a backing file (e.g. tmpfs), as described in [1].
> > > > > > > Even if the Kernel maintains the link between a tmpfs exported and an
> > > > > > > in-memory IMA log - it still has to copy the tmpfs portion to the
> > > > > > > Kernel memory during kexec soft boot.  tmpfs is cleared during kexec,
> > > > > > > so this copying of tmpfs back to kernel memory is necessary to preserve
> > > > > > > the integrity of the log during kexec.  But the copying would add back
> > > > > > > the memory pressure on the node during kexec (which may result in
> > > > > > > out-of-memory), defeating the purpose of the overall effort/feature.
> > > > > > > Copying to a regular *persistent* protected file seems a cleaner
> > > > > > > approach, compared to tmpfs.
> > > > > >
> > > > > > From a kernel perspective, it doesn't make a difference if userspace
> > > > > > provides a tmpfs or persistent file.  As per the discussion
> > > > > > https://lore.kernel.org/linux-integrity/CAOQ4uxj4Pv2Wr1wgvBCDR-tnA5dsZT3rvdDzKgAH1aEV_-r9Qg at mail.gmail.com/#t
> > > > > > , userspace provides the kernel with the file descriptor of the opened
> > > > > > file.
> > > > > >
> > > > > > > We prototyped this solution, however it
> > > > > > > does not seem to be a common pattern within the Kernel to write state
> > > > > > > directly to files on disk file systems.  We considered two potential
> > > > > > > options:
> > > > > >
> > > > > > If no file descriptor is provided, then the measurements aren't copied
> > > > > > and removed from the securityfs file.  If there are write errors, the
> > > > > > measurements aren't removed from the securityfs file until the write
> > > > > > errors are resolved.
> > > > >
> > > > > It sounds like this approach would require the file/filesystem to be
> > > > > continuously available for the life of the system once the log was
> > > > > snapshotted/overflowed to persistent storage, yes?  Assuming that is
> > > > > the case, what happens if the file/filesystem becomes inaccessible at
> > > > > some point and an attestation client attempts to read the entire log?
> > > >
> > > > The main purpose of the change is to addres kernel memory pressure.
> > > > Two designs are being discussed: Sush's "snapshotting" design and
> > > > Amir's original suggestion of continously exporting the measurement
> > > > records to a tmpfs or regular file.  Both designs require verifying the
> > > > initial attestation quote by walking the entire measurement list,
> > > > calculating the expected TPM PCR value(s).  That doesn't change.
> > >
> > > Sure, but my question is about what happens if portions of the
> > > measurement list disappear due to file/filesystem problems?  How is
> > > that handled?
> >
> > With the "snapshotting" solution there could be multiple files, so
> > portions could be missing.  The other solution, the preferred solution,
> > would be one file.
> 
> With the snapshotting approach the kernel doesn't need to maintain
> ongoing access to a file, that is left up to the user process
> performing the attestation (or simply inspecting the logs).  I have to
> ask, for the third time now in as many hours, how does the proposed
> kernel-holds-an-fd-open solution handle the case where the
> file/filesystem is no longer accessible?  The snapshotting approach
> should be much more resilient here as the error recovery paths can be
> much more involved than what we would have available in the kernel,
> not to mention the flexibility of allowing a user process to determine
> how to store and manage the snapshotted log.
> 
> Considering that the snapshotting approach is opt-in (userspace has to
> initiate the snapshot), I'm not sure the concern over log offsets is a
> significant objection: existing userspace will never trigger a
> snapshot, and new userspace that could potentially trigger a snapshot
> should be written to take that into account.

Your initial question was "what happens if the file/filesystem becomes
inaccessible at some point and an attestation client attempts to read
the entire log?".  For what reason would it be inaccessible?  For the
original single tmpfs file, what would make it inaccessible?  In the
"snapshotting" design this problem becomes a userspace issue.

The first sentence of the cover letter is "Depending on the IMA policy,
the IMA log can consume a lot of Kernel memory on the device." 
Copying/exporting the measurements from kernel memory to a single
(tmpfs) file resolves the kernel memory pressure issue.  The entire
measurement list would remain accessible via the securityfs file.  As
previously mentioned in this thread, it wouldn't resolve the kexec
issue of allocating a buffer large enough for really large measurement
lists.

If the real motivation for this patch set is trimming the measurement
list, the cover letter needs to say that.  The existing "snapshotting"
design requires locking of the entire measurement list while copying
and trimming it.  The "snapshotting" design needs to be simplified. 
Perhaps a trusted userspace process could copy the measurements and
then trigger trimming of the in kernel memory/tmpfs measurement list.

-- 
thanks,

Mimi



From paul at paul-moore.com  Wed Aug 30 13:47:40 2023
From: paul at paul-moore.com (Paul Moore)
Date: Wed, 30 Aug 2023 16:47:40 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com> <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
Message-ID: <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>

On Wed, Aug 30, 2023 at 4:25?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> Your initial question was "what happens if the file/filesystem becomes
> inaccessible at some point and an attestation client attempts to read
> the entire log?".  For what reason would it be inaccessible?  For the
> original single tmpfs file, what would make it inaccessible?

In your reply that I had responded to you had mentioned that the
kernel was simply being passed a fd and taking ownership of it, the fd
could either be a tmpfs backed file or some form of persistent storage
as both were discussed in this thread.  I imagine a tmpfs filesystem
could still be forcibly unmounted, resulting in problems, but I can't
say that for certain.  However, there are definitely cases where a fd
backed against an arbitrary filesystem could run into problems:
storage device issues for local filesystems, networking issues for
network filesystems, and good old fashioned user/admin intervention in
both cases.

> In the
> "snapshotting" design this problem becomes a userspace issue.

Yes, exactly.  Userspace is almost always going to have an easier time
recovering from these types of errors ... or at least I believe so,
perhaps you have a clever solution for how the kernel can handle the
file/filesystem disappearing from under the fd?

> The first sentence of the cover letter is "Depending on the IMA policy,
> the IMA log can consume a lot of Kernel memory on the device." ...

As I'm still looking for an answer to my question, let's stay focused
on that before we start worrying too much about the phrasing of the
design proposal that was submitted.

-- 
paul-moore.com


From zohar at linux.ibm.com  Wed Aug 30 14:50:12 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Wed, 30 Aug 2023 17:50:12 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
	 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
	 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
	 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
	 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
	 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
	 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
	 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
	 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
	 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
	 <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
	 <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>
Message-ID: <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com>

On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> On Wed, Aug 30, 2023 at 4:25?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > Your initial question was "what happens if the file/filesystem becomes
> > inaccessible at some point and an attestation client attempts to read
> > the entire log?".  For what reason would it be inaccessible?  For the
> > original single tmpfs file, what would make it inaccessible?
> 
> In your reply that I had responded to you had mentioned that the
> kernel was simply being passed a fd and taking ownership of it, the fd
> could either be a tmpfs backed file or some form of persistent storage
> as both were discussed in this thread.  I imagine a tmpfs filesystem
> could still be forcibly unmounted, resulting in problems, but I can't
> say that for certain.  However, there are definitely cases where a fd
> backed against an arbitrary filesystem could run into problems:
> storage device issues for local filesystems, networking issues for
> network filesystems, and good old fashioned user/admin intervention in
> both cases.

"I imagine tmpfs filesystem could still be forcibly unmounted" sounds
like an attack. Not being able to verify the measurement list against a
quote is probably a good thing.

> 
> > In the
> > "snapshotting" design this problem becomes a userspace issue.
> 
> Yes, exactly.  Userspace is almost always going to have an easier time
> recovering from these types of errors ... or at least I believe so,
> perhaps you have a clever solution for how the kernel can handle the
> file/filesystem disappearing from under the fd?

Nothing changes.  New measurements are initially stored in kernel
memory, until they are successfully copied.

> > The first sentence of the cover letter is "Depending on the IMA policy,
> > the IMA log can consume a lot of Kernel memory on the device." ...
> 
> As I'm still looking for an answer to my question, let's stay focused
> on that before we start worrying too much about the phrasing of the
> design proposal that was submitted.

It's more than just phrasing, but the purpose/motivation of the
proposed changes.

--  
thanks,

Mimi



From paul at paul-moore.com  Wed Aug 30 15:21:02 2023
From: paul at paul-moore.com (Paul Moore)
Date: Wed, 30 Aug 2023 18:21:02 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
 <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
 <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com> <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com>
Message-ID: <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com>

On Wed, Aug 30, 2023 at 5:50?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> > On Wed, Aug 30, 2023 at 4:25?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > Your initial question was "what happens if the file/filesystem becomes
> > > inaccessible at some point and an attestation client attempts to read
> > > the entire log?".  For what reason would it be inaccessible?  For the
> > > original single tmpfs file, what would make it inaccessible?
> >
> > In your reply that I had responded to you had mentioned that the
> > kernel was simply being passed a fd and taking ownership of it, the fd
> > could either be a tmpfs backed file or some form of persistent storage
> > as both were discussed in this thread.  I imagine a tmpfs filesystem
> > could still be forcibly unmounted, resulting in problems, but I can't
> > say that for certain.  However, there are definitely cases where a fd
> > backed against an arbitrary filesystem could run into problems:
> > storage device issues for local filesystems, networking issues for
> > network filesystems, and good old fashioned user/admin intervention in
> > both cases.
>
> "I imagine tmpfs filesystem could still be forcibly unmounted" sounds
> like an attack. Not being able to verify the measurement list against a
> quote is probably a good thing.

Okay, can you answer the question for an arbitrary persistent
filesystem?  That was always the more important question, and your
continued avoidance is getting me increasingly annoyed.

-- 
paul-moore.com


From paul at paul-moore.com  Wed Aug 30 15:23:48 2023
From: paul at paul-moore.com (Paul Moore)
Date: Wed, 30 Aug 2023 18:23:48 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
 <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
 <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>
 <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com> <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com>
Message-ID: <CAHC9VhR1s+p-q35xqHL5Jj2WuzE=bhMrK6PBBThaUBiMRxwvuQ@mail.gmail.com>

On Wed, Aug 30, 2023 at 6:21?PM Paul Moore <paul at paul-moore.com> wrote:
> On Wed, Aug 30, 2023 at 5:50?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> > > On Wed, Aug 30, 2023 at 4:25?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > Your initial question was "what happens if the file/filesystem becomes
> > > > inaccessible at some point and an attestation client attempts to read
> > > > the entire log?".  For what reason would it be inaccessible?  For the
> > > > original single tmpfs file, what would make it inaccessible?
> > >
> > > In your reply that I had responded to you had mentioned that the
> > > kernel was simply being passed a fd and taking ownership of it, the fd
> > > could either be a tmpfs backed file or some form of persistent storage
> > > as both were discussed in this thread.  I imagine a tmpfs filesystem
> > > could still be forcibly unmounted, resulting in problems, but I can't
> > > say that for certain.  However, there are definitely cases where a fd
> > > backed against an arbitrary filesystem could run into problems:
> > > storage device issues for local filesystems, networking issues for
> > > network filesystems, and good old fashioned user/admin intervention in
> > > both cases.
> >
> > "I imagine tmpfs filesystem could still be forcibly unmounted" sounds
> > like an attack. Not being able to verify the measurement list against a
> > quote is probably a good thing.
>
> Okay, can you answer the question for an arbitrary persistent
> filesystem?  That was always the more important question, and your
> continued avoidance is getting me increasingly annoyed.

Speaking of being annoyed, I'm fixing Tushar's email as the bounces
are also driving me nuts.

-- 
paul-moore.com


From zohar at linux.ibm.com  Wed Aug 30 16:06:32 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Wed, 30 Aug 2023 19:06:32 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <CAHC9VhR1s+p-q35xqHL5Jj2WuzE=bhMrK6PBBThaUBiMRxwvuQ@mail.gmail.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
	 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
	 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
	 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
	 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
	 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
	 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
	 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
	 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
	 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
	 <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
	 <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>
	 <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com>
	 <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com>
	 <CAHC9VhR1s+p-q35xqHL5Jj2WuzE=bhMrK6PBBThaUBiMRxwvuQ@mail.gmail.com>
Message-ID: <0835ea7fd96df23419431301dfbc0d190e6c36f4.camel@linux.ibm.com>

On Wed, 2023-08-30 at 18:23 -0400, Paul Moore wrote:
> On Wed, Aug 30, 2023 at 6:21?PM Paul Moore <paul at paul-moore.com> wrote:
> > On Wed, Aug 30, 2023 at 5:50?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> > > > On Wed, Aug 30, 2023 at 4:25?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > Your initial question was "what happens if the file/filesystem becomes
> > > > > inaccessible at some point and an attestation client attempts to read
> > > > > the entire log?".  For what reason would it be inaccessible?  For the
> > > > > original single tmpfs file, what would make it inaccessible?
> > > >
> > > > In your reply that I had responded to you had mentioned that the
> > > > kernel was simply being passed a fd and taking ownership of it, the fd
> > > > could either be a tmpfs backed file or some form of persistent storage
> > > > as both were discussed in this thread.  I imagine a tmpfs filesystem
> > > > could still be forcibly unmounted, resulting in problems, but I can't
> > > > say that for certain.  However, there are definitely cases where a fd
> > > > backed against an arbitrary filesystem could run into problems:
> > > > storage device issues for local filesystems, networking issues for
> > > > network filesystems, and good old fashioned user/admin intervention in
> > > > both cases.
> > >
> > > "I imagine tmpfs filesystem could still be forcibly unmounted" sounds
> > > like an attack. Not being able to verify the measurement list against a
> > > quote is probably a good thing.
> >
> > Okay, can you answer the question for an arbitrary persistent
> > filesystem?  That was always the more important question, ...

The original proposal, not mine, suggested using a tmpfs file.   The
idea of writing the measurements to a file on a persistent filesystem
wasn't mine either.  Sush/Tushar were pushing for writing to a
persistent file(s).  No argument from me that writing to a file on an
arbitrary persistent filesystem is not a good idea.

-- 
thanks,

Mimi






From paul at paul-moore.com  Wed Aug 30 16:22:27 2023
From: paul at paul-moore.com (Paul Moore)
Date: Wed, 30 Aug 2023 19:22:27 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <0835ea7fd96df23419431301dfbc0d190e6c36f4.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
 <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
 <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>
 <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com>
 <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com>
 <CAHC9VhR1s+p-q35xqHL5Jj2WuzE=bhMrK6PBBThaUBiMRxwvuQ@mail.gmail.com> <0835ea7fd96df23419431301dfbc0d190e6c36f4.camel@linux.ibm.com>
Message-ID: <CAHC9VhTbmmVrZCHCeZe-=s9fCGmEEDdb4p-skKJ=0wmX_K3arQ@mail.gmail.com>

On Wed, Aug 30, 2023 at 7:07?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> On Wed, 2023-08-30 at 18:23 -0400, Paul Moore wrote:
> > On Wed, Aug 30, 2023 at 6:21?PM Paul Moore <paul at paul-moore.com> wrote:
> > > On Wed, Aug 30, 2023 at 5:50?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> > > > > On Wed, Aug 30, 2023 at 4:25?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > > Your initial question was "what happens if the file/filesystem becomes
> > > > > > inaccessible at some point and an attestation client attempts to read
> > > > > > the entire log?".  For what reason would it be inaccessible?  For the
> > > > > > original single tmpfs file, what would make it inaccessible?
> > > > >
> > > > > In your reply that I had responded to you had mentioned that the
> > > > > kernel was simply being passed a fd and taking ownership of it, the fd
> > > > > could either be a tmpfs backed file or some form of persistent storage
> > > > > as both were discussed in this thread.  I imagine a tmpfs filesystem
> > > > > could still be forcibly unmounted, resulting in problems, but I can't
> > > > > say that for certain.  However, there are definitely cases where a fd
> > > > > backed against an arbitrary filesystem could run into problems:
> > > > > storage device issues for local filesystems, networking issues for
> > > > > network filesystems, and good old fashioned user/admin intervention in
> > > > > both cases.
> > > >
> > > > "I imagine tmpfs filesystem could still be forcibly unmounted" sounds
> > > > like an attack. Not being able to verify the measurement list against a
> > > > quote is probably a good thing.
> > >
> > > Okay, can you answer the question for an arbitrary persistent
> > > filesystem?  That was always the more important question, ...
>
> The original proposal, not mine, suggested using a tmpfs file.   The
> idea of writing the measurements to a file on a persistent filesystem
> wasn't mine either.  Sush/Tushar were pushing for writing to a
> persistent file(s).  No argument from me that writing to a file on an
> arbitrary persistent filesystem is not a good idea.

Progress.  Okay, so we all now agree that the kernel writing to an
arbitrary filesystem is not a good idea, and using tmpfs doesn't solve
the problem of general memory pressure (from previously in this
thread), that's all helpful and good to clarify.

Assuming Sush and Tushar rework the document to clarify the
motivation/purpose for the work, as you suggested earlier, I'm
assuming we can revisit this problem and solutions?

-- 
paul-moore.com


From arnd at arndb.de  Wed Aug 30 16:39:18 2023
From: arnd at arndb.de (Arnd Bergmann)
Date: Wed, 30 Aug 2023 19:39:18 -0400
Subject: [RFC PATCH] Introduce persistent memory pool
In-Reply-To: <82416797-01aa-471a-a737-471297e37c4c@amazon.de>
References: <64e7cbf7.050a0220.114c7.b70dSMTPIN_ADDED_BROKEN@mx.google.com>
 <2023082506-enchanted-tripping-d1d5@gregkh>
 <c26ad989dcc6737dd295e980c78ef53740098810.camel@amazon.com>
 <20230823024500.GA25462@skinsburskii.>
 <e0ed9fb9-8e7a-44ad-976a-27362f6e537a@amazon.de>
 <20230829220740.GA26605@skinsburskii.>
 <82416797-01aa-471a-a737-471297e37c4c@amazon.de>
Message-ID: <1697efce-665a-43d5-b0be-7c03c0a4d850@app.fastmail.com>

On Wed, Aug 30, 2023, at 03:20, Alexander Graf wrote:
> On 30.08.23 00:07, Stanislav Kinsburskii wrote:
>> On Mon, Aug 28, 2023 at 10:50:19PM +0200, Alexander Graf wrote:

>> Device tree or ACPI are options indeed. However AFAIU in case of DT user
>> space has to involved into the picture to modify and complie it, while
>> ACPI isn't flexible or easily extendable.
>> Also, AFAIU both these standards were designed with passing
>> hardware-specific data in mind from bootstrap software to an OS kernel
>> and thus were never really intended to be used for creating a persistent
>> state accross kexec.
>> To me, an attempt to use either of them to pass kernel-specific data looks
>> like an abuse (or misuse) excused by the simplicity of implementation.
>
>
> What I was describing above is that the Linux boot protocol already has 
> natural ways to pass a DT (arm) or set of ACPI tables (x86) to the 
> target kernel. Whatever we do here should either piggy back on top of 
> those natural mechanisms (e.g. /chosen node in DT) or be on the same 
> level (e.g. pass DT in one register, pass metadata structure in another 
> register).
>
> When it comes to the actual content of the metadata, I'm personally also 
> leaning towards DT. We already have libfdt inside the kernel. It gives 
> is a very simple, well understood structured file format that you can 
> extend, version, etc etc. And the kernel has mechanisms to modify fdt 
> contents.

Agreed. This also makes a lot of sense since the fdt format was
originally introduced for this exact purpose, to be a key-value
store to pass data from the running kernel to the next one after
kexec when the original source of the data (originally open
firmware) is gone. It only turned into the generic way to
describe embedded systems later on, but both the fdt binary
format and the kexec infrastructure for manipulating and
passing the blob should be easy to reuse for additional purposes
as long as the contents are put into appropriate namespaces that
don't clash with existing usage.

       Arnd


From thunder.leizhen at huaweicloud.com  Wed Aug 30 18:25:59 2023
From: thunder.leizhen at huaweicloud.com (Leizhen (ThunderTown))
Date: Thu, 31 Aug 2023 09:25:59 +0800
Subject: [PATCH v2 1/8] crash_core.c: remove unnecessary parameter of
 function
In-Reply-To: <20230829121610.138107-2-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
 <20230829121610.138107-2-bhe@redhat.com>
Message-ID: <d00bf2e8-3022-64a1-0458-56497179d5aa@huaweicloud.com>



On 2023/8/29 20:16, Baoquan He wrote:
> In all call sites of __parse_crashkernel(), the parameter 'name' is
> hardcoded as "crashkernel=". So remove the unnecessary parameter 'name',
> add local varibale 'name' inside __parse_crashkernel() instead.

Reviewed-by: Zhen Lei <thunder.leizhen at huawei.com>

> 
> Signed-off-by: Baoquan He <bhe at redhat.com>
> ---
>  kernel/crash_core.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index 90ce1dfd591c..f27b4e45d410 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -241,11 +241,11 @@ static int __init __parse_crashkernel(char *cmdline,
>  			     unsigned long long system_ram,
>  			     unsigned long long *crash_size,
>  			     unsigned long long *crash_base,
> -			     const char *name,
>  			     const char *suffix)
>  {
>  	char	*first_colon, *first_space;
>  	char	*ck_cmdline;
> +	char	*name = "crashkernel=";
>  
>  	BUG_ON(!crash_size || !crash_base);
>  	*crash_size = 0;
> @@ -283,7 +283,7 @@ int __init parse_crashkernel(char *cmdline,
>  			     unsigned long long *crash_base)
>  {
>  	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> -					"crashkernel=", NULL);
> +				NULL);
>  }
>  
>  int __init parse_crashkernel_high(char *cmdline,
> @@ -292,7 +292,7 @@ int __init parse_crashkernel_high(char *cmdline,
>  			     unsigned long long *crash_base)
>  {
>  	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> -				"crashkernel=", suffix_tbl[SUFFIX_HIGH]);
> +				suffix_tbl[SUFFIX_HIGH]);
>  }
>  
>  int __init parse_crashkernel_low(char *cmdline,
> @@ -301,7 +301,7 @@ int __init parse_crashkernel_low(char *cmdline,
>  			     unsigned long long *crash_base)
>  {
>  	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> -				"crashkernel=", suffix_tbl[SUFFIX_LOW]);
> +				suffix_tbl[SUFFIX_LOW]);
>  }
>  
>  /*
> 

-- 
Regards,
  Zhen Lei



From thunder.leizhen at huaweicloud.com  Wed Aug 30 19:30:20 2023
From: thunder.leizhen at huaweicloud.com (Leizhen (ThunderTown))
Date: Thu, 31 Aug 2023 10:30:20 +0800
Subject: [PATCH v2 2/8] crash_core: change the prototype of function
 parse_crashkernel()
In-Reply-To: <20230829121610.138107-3-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
 <20230829121610.138107-3-bhe@redhat.com>
Message-ID: <77c34c98-f6b1-75e1-dbf3-a148691e7f9d@huaweicloud.com>



On 2023/8/29 20:16, Baoquan He wrote:
> Add two parameters 'low_size' and 'high' to function parse_crashkernel(),
> later crashkernel=,high|low parsing will be added. Make adjustments in all
> call sites of parse_crashkernel() in arch.

Reviewed-by: Zhen Lei <thunder.leizhen at huawei.com>

> 
> Signed-off-by: Baoquan He <bhe at redhat.com>
> ---
>  arch/arm/kernel/setup.c              |  3 ++-
>  arch/arm64/mm/init.c                 |  2 +-
>  arch/ia64/kernel/setup.c             |  2 +-
>  arch/loongarch/kernel/setup.c        |  4 +++-
>  arch/mips/kernel/setup.c             |  3 ++-
>  arch/powerpc/kernel/fadump.c         |  2 +-
>  arch/powerpc/kexec/core.c            |  2 +-
>  arch/powerpc/mm/nohash/kaslr_booke.c |  2 +-
>  arch/riscv/mm/init.c                 |  2 +-
>  arch/s390/kernel/setup.c             |  4 ++--
>  arch/sh/kernel/machine_kexec.c       |  2 +-
>  arch/x86/kernel/setup.c              |  3 ++-
>  include/linux/crash_core.h           |  3 ++-
>  kernel/crash_core.c                  | 15 ++++++++++++---
>  14 files changed, 32 insertions(+), 17 deletions(-)
> 
> diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
> index c66b560562b3..e2bb7afd0683 100644
> --- a/arch/arm/kernel/setup.c
> +++ b/arch/arm/kernel/setup.c
> @@ -1010,7 +1010,8 @@ static void __init reserve_crashkernel(void)
>  
>  	total_mem = get_total_mem();
>  	ret = parse_crashkernel(boot_command_line, total_mem,
> -				&crash_size, &crash_base);
> +				&crash_size, &crash_base,
> +				NULL, NULL);
>  	/* invalid value specified or crashkernel=0 */
>  	if (ret || !crash_size)
>  		return;
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 4fcb88a445ef..4ad637508b75 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -142,7 +142,7 @@ static void __init reserve_crashkernel(void)
>  
>  	/* crashkernel=X[@offset] */
>  	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
> -				&crash_size, &crash_base);
> +				&crash_size, &crash_base, NULL, NULL);
>  	if (ret == -ENOENT) {
>  		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
>  		if (ret || !crash_size)
> diff --git a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
> index 5a55ac82c13a..4faea2d2cf07 100644
> --- a/arch/ia64/kernel/setup.c
> +++ b/arch/ia64/kernel/setup.c
> @@ -277,7 +277,7 @@ static void __init setup_crashkernel(unsigned long total, int *n)
>  	int ret;
>  
>  	ret = parse_crashkernel(boot_command_line, total,
> -			&size, &base);
> +			&size, &base, NULL, NULL);
>  	if (ret == 0 && size > 0) {
>  		if (!base) {
>  			sort_regions(rsvd_region, *n);
> diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c
> index 9d830ab4e302..776a068d8718 100644
> --- a/arch/loongarch/kernel/setup.c
> +++ b/arch/loongarch/kernel/setup.c
> @@ -267,7 +267,9 @@ static void __init arch_parse_crashkernel(void)
>  	unsigned long long crash_base, crash_size;
>  
>  	total_mem = memblock_phys_mem_size();
> -	ret = parse_crashkernel(boot_command_line, total_mem, &crash_size, &crash_base);
> +	ret = parse_crashkernel(boot_command_line, total_mem,
> +				&crash_size, &crash_base,
> +				NULL, NULL);
>  	if (ret < 0 || crash_size <= 0)
>  		return;
>  
> diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
> index cb871eb784a7..08321c945ac4 100644
> --- a/arch/mips/kernel/setup.c
> +++ b/arch/mips/kernel/setup.c
> @@ -460,7 +460,8 @@ static void __init mips_parse_crashkernel(void)
>  
>  	total_mem = memblock_phys_mem_size();
>  	ret = parse_crashkernel(boot_command_line, total_mem,
> -				&crash_size, &crash_base);
> +				&crash_size, &crash_base,
> +				NULL, NULL);
>  	if (ret != 0 || crash_size <= 0)
>  		return;
>  
> diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c
> index ea0a073abd96..7dbdeba56e74 100644
> --- a/arch/powerpc/kernel/fadump.c
> +++ b/arch/powerpc/kernel/fadump.c
> @@ -313,7 +313,7 @@ static __init u64 fadump_calculate_reserve_size(void)
>  	 * memory at a predefined offset.
>  	 */
>  	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
> -				&size, &base);
> +				&size, &base, NULL, NULL);
>  	if (ret == 0 && size > 0) {
>  		unsigned long max_size;
>  
> diff --git a/arch/powerpc/kexec/core.c b/arch/powerpc/kexec/core.c
> index de64c7962991..9346c960b296 100644
> --- a/arch/powerpc/kexec/core.c
> +++ b/arch/powerpc/kexec/core.c
> @@ -109,7 +109,7 @@ void __init reserve_crashkernel(void)
>  	total_mem_sz = memory_limit ? memory_limit : memblock_phys_mem_size();
>  	/* use common parsing */
>  	ret = parse_crashkernel(boot_command_line, total_mem_sz,
> -			&crash_size, &crash_base);
> +			&crash_size, &crash_base, NULL, NULL);
>  	if (ret == 0 && crash_size > 0) {
>  		crashk_res.start = crash_base;
>  		crashk_res.end = crash_base + crash_size - 1;
> diff --git a/arch/powerpc/mm/nohash/kaslr_booke.c b/arch/powerpc/mm/nohash/kaslr_booke.c
> index 2fb3edafe9ab..b4f2786a7d2b 100644
> --- a/arch/powerpc/mm/nohash/kaslr_booke.c
> +++ b/arch/powerpc/mm/nohash/kaslr_booke.c
> @@ -178,7 +178,7 @@ static void __init get_crash_kernel(void *fdt, unsigned long size)
>  	int ret;
>  
>  	ret = parse_crashkernel(boot_command_line, size, &crash_size,
> -				&crash_base);
> +				&crash_base, NULL, NULL);
>  	if (ret != 0 || crash_size == 0)
>  		return;
>  	if (crash_base == 0)
> diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
> index e4c35ac2357f..a9ef0824f905 100644
> --- a/arch/riscv/mm/init.c
> +++ b/arch/riscv/mm/init.c
> @@ -1332,7 +1332,7 @@ static void __init reserve_crashkernel(void)
>  	}
>  
>  	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
> -				&crash_size, &crash_base);
> +				&crash_size, &crash_base, NULL, NULL);
>  	if (ret || !crash_size)
>  		return;
>  
> diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
> index c744104e4a9c..98204a5f62b1 100644
> --- a/arch/s390/kernel/setup.c
> +++ b/arch/s390/kernel/setup.c
> @@ -626,8 +626,8 @@ static void __init reserve_crashkernel(void)
>  	phys_addr_t low, high;
>  	int rc;
>  
> -	rc = parse_crashkernel(boot_command_line, ident_map_size, &crash_size,
> -			       &crash_base);
> +	rc = parse_crashkernel(boot_command_line, ident_map_size,
> +			       &crash_size, &crash_base, NULL, NULL);
>  
>  	crash_base = ALIGN(crash_base, KEXEC_CRASH_MEM_ALIGN);
>  	crash_size = ALIGN(crash_size, KEXEC_CRASH_MEM_ALIGN);
> diff --git a/arch/sh/kernel/machine_kexec.c b/arch/sh/kernel/machine_kexec.c
> index 223c14f44af7..fa3a7b36190a 100644
> --- a/arch/sh/kernel/machine_kexec.c
> +++ b/arch/sh/kernel/machine_kexec.c
> @@ -154,7 +154,7 @@ void __init reserve_crashkernel(void)
>  	int ret;
>  
>  	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
> -			&crash_size, &crash_base);
> +			&crash_size, &crash_base, NULL, NULL);
>  	if (ret == 0 && crash_size > 0) {
>  		crashk_res.start = crash_base;
>  		crashk_res.end = crash_base + crash_size - 1;
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index fd975a4a5200..382c66d2cf71 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -558,7 +558,8 @@ static void __init reserve_crashkernel(void)
>  	total_mem = memblock_phys_mem_size();
>  
>  	/* crashkernel=XM */
> -	ret = parse_crashkernel(boot_command_line, total_mem, &crash_size, &crash_base);
> +	ret = parse_crashkernel(boot_command_line, total_mem,
> +				&crash_size, &crash_base, NULL, NULL);
>  	if (ret != 0 || crash_size <= 0) {
>  		/* crashkernel=X,high */
>  		ret = parse_crashkernel_high(boot_command_line, total_mem,
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index de62a722431e..2e76289699ff 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -78,7 +78,8 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
>  void final_note(Elf_Word *buf);
>  
>  int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
> -		unsigned long long *crash_size, unsigned long long *crash_base);
> +		unsigned long long *crash_size, unsigned long long *crash_base,
> +		unsigned long long *low_size, bool *high);
>  int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
>  		unsigned long long *crash_size, unsigned long long *crash_base);
>  int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index f27b4e45d410..f6a5c219e2e1 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -280,10 +280,19 @@ static int __init __parse_crashkernel(char *cmdline,
>  int __init parse_crashkernel(char *cmdline,
>  			     unsigned long long system_ram,
>  			     unsigned long long *crash_size,
> -			     unsigned long long *crash_base)
> +			     unsigned long long *crash_base,
> +			     unsigned long long *low_size,
> +			     bool *high)
>  {
> -	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> -				NULL);
> +	int ret;
> +
> +	/* crashkernel=X[@offset] */
> +	ret = __parse_crashkernel(cmdline, system_ram, crash_size,
> +				crash_base, NULL);
> +	if (!high)
> +		return ret;
> +
> +	return 0;
>  }
>  
>  int __init parse_crashkernel_high(char *cmdline,
> 

-- 
Regards,
  Zhen Lei



From thunder.leizhen at huaweicloud.com  Wed Aug 30 19:56:58 2023
From: thunder.leizhen at huaweicloud.com (Leizhen (ThunderTown))
Date: Thu, 31 Aug 2023 10:56:58 +0800
Subject: [PATCH v2 3/8] crash_core: change parse_crashkernel() to support
 crashkernel=,high|low parsing
In-Reply-To: <20230829121610.138107-4-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
 <20230829121610.138107-4-bhe@redhat.com>
Message-ID: <c4a1f96e-2c8d-a496-61f3-0299d77c5c84@huaweicloud.com>



On 2023/8/29 20:16, Baoquan He wrote:
> Now parse_crashkernel() is a real entry point for all kinds of
> crahskernel parsing on any architecture.
> 
> And wrap the crahskernel=,high|low handling inside
> CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION ifdeffery scope.
> 
> Signed-off-by: Baoquan He <bhe at redhat.com>
> ---
>  include/linux/crash_core.h |  6 ++++++
>  kernel/crash_core.c        | 28 +++++++++++++++++++++++++++-
>  2 files changed, 33 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 2e76289699ff..85260bf4a734 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -77,6 +77,12 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
>  			  void *data, size_t data_len);
>  void final_note(Elf_Word *buf);
>  
> +#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> +#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
> +#define DEFAULT_CRASH_KERNEL_LOW_SIZE  (128UL << 20)
> +#endif
> +#endif
> +
>  int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
>  		unsigned long long *crash_size, unsigned long long *crash_base,
>  		unsigned long long *low_size, bool *high);
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index f6a5c219e2e1..355b0ab5189c 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -276,6 +276,9 @@ static int __init __parse_crashkernel(char *cmdline,
>  /*
>   * That function is the entry point for command line parsing and should be
>   * called from the arch-specific code.
> + *
> + * If crashkernel=,high|low is supported on architecture, non-NULL values
> + * should be passed to parameters 'low_size' and 'high'.
>   */
>  int __init parse_crashkernel(char *cmdline,
>  			     unsigned long long system_ram,
> @@ -291,7 +294,30 @@ int __init parse_crashkernel(char *cmdline,
>  				crash_base, NULL);
>  	if (!high)
>  		return ret;
> -
> +#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> +	else if (ret == -ENOENT) {
> +		ret = __parse_crashkernel(cmdline, 0, crash_size,
> +				crash_base, suffix_tbl[SUFFIX_HIGH]);
> +		if (ret || !*crash_size)
> +			return -1;

Change -1 to -EINVAL?

> +
> +		/*
> +		 * crashkernel=Y,low can be specified or not, but invalid value
> +		 * is not allowed.
> +		 */
> +		ret = __parse_crashkernel(cmdline, 0, low_size,
> +				crash_base, suffix_tbl[SUFFIX_LOW]);
> +		if (ret == -ENOENT)
> +			*low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> +		else if (ret)
> +			return -1;

return ret;

> +
> +		*high = true;
> +	} else if (ret || !*crash_size) {

This check can be moved outside of #ifdef. Because even '!high', it's completely
applicable. The overall adjustment is as follows:

-  	if (!high)
-  		return ret;

#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
	if (high && ret == -ENOENT) {
		... ...
		if (ret || !*crash_size)	//parse HIGH
		... ...
	}

	//At this point, *crash_size is not 0 and ret is 0.
	//We can also delete if (!*crash_size) above because it will be checked later.
#endif

	if (!*crash_size)
		ret = -EINVAL;

	return ret;

-  	return 0;

> +		/* The specified value is invalid */
> +		return -1;
> +	}
> +#endif
>  	return 0;
>  }
>  
> 

-- 
Regards,
  Zhen Lei



From thunder.leizhen at huaweicloud.com  Wed Aug 30 20:23:34 2023
From: thunder.leizhen at huaweicloud.com (Leizhen (ThunderTown))
Date: Thu, 31 Aug 2023 11:23:34 +0800
Subject: [PATCH v2 4/8] crash_core: add generic function to do reservation
In-Reply-To: <20230829121610.138107-5-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
 <20230829121610.138107-5-bhe@redhat.com>
Message-ID: <df395f20-55e9-169b-846b-f55d21d00861@huaweicloud.com>



On 2023/8/29 20:16, Baoquan He wrote:
> In architecture like x86_64, arm64 and riscv, they have vast virtual
> address space and usually have huge physical memory RAM. Their
> crashkernel reservation doesn't have to be limited under 4G RAM,
> but can be extended to the whole physical memory via crashkernel=,high
> support.
> 
> Now add function reserve_crashkernel_generic() to reserve crashkernel
> memory if users specify any case of kernel pamameters, like
> crashkernel=xM[@offset] or crashkernel=,high|low.
> 
> This is preparation to simplify code of crashkernel=,high support
> in architecutures.
> 
> Signed-off-by: Baoquan He <bhe at redhat.com>
> ---
>  include/linux/crash_core.h |  34 ++++++++++--
>  kernel/crash_core.c        | 109 ++++++++++++++++++++++++++++++++++++-
>  2 files changed, 136 insertions(+), 7 deletions(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 85260bf4a734..2f732493e922 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -77,12 +77,6 @@ Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
>  			  void *data, size_t data_len);
>  void final_note(Elf_Word *buf);
>  
> -#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> -#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
> -#define DEFAULT_CRASH_KERNEL_LOW_SIZE  (128UL << 20)
> -#endif
> -#endif
> -
>  int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
>  		unsigned long long *crash_size, unsigned long long *crash_base,
>  		unsigned long long *low_size, bool *high);
> @@ -91,4 +85,32 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
>  int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
>  		unsigned long long *crash_size, unsigned long long *crash_base);
>  
> +#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> +#ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
> +#define DEFAULT_CRASH_KERNEL_LOW_SIZE	(128UL << 20)
> +#endif
> +#ifndef CRASH_ALIGN
> +#define CRASH_ALIGN			SZ_2M
> +#endif
> +#ifndef CRASH_ADDR_LOW_MAX
> +#define CRASH_ADDR_LOW_MAX		SZ_4G
> +#endif
> +#ifndef CRASH_ADDR_HIGH_MAX
> +#define CRASH_ADDR_HIGH_MAX		memblock_end_of_DRAM()
> +#endif
> +
> +void __init reserve_crashkernel_generic(char *cmdline,
> +		unsigned long long crash_size,
> +		unsigned long long crash_base,
> +		unsigned long long crash_low_size,
> +		bool high);
> +#else
> +static inline void __init reserve_crashkernel_generic(char *cmdline,
> +		unsigned long long crash_size,
> +		unsigned long long crash_base,
> +		unsigned long long crash_low_size,
> +		bool high)
> +{}
> +#endif
> +
>  #endif /* LINUX_CRASH_CORE_H */
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index 355b0ab5189c..6bc00cc390b5 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -5,11 +5,13 @@
>   */
>  
>  #include <linux/buildid.h>
> -#include <linux/crash_core.h>
>  #include <linux/init.h>
>  #include <linux/utsname.h>
>  #include <linux/vmalloc.h>
>  #include <linux/sizes.h>
> +#include <linux/memblock.h>
> +#include <linux/kexec.h>
> +#include <linux/kmemleak.h>
>  
>  #include <asm/page.h>
>  #include <asm/sections.h>
> @@ -349,6 +351,111 @@ static int __init parse_crashkernel_dummy(char *arg)
>  }
>  early_param("crashkernel", parse_crashkernel_dummy);
>  
> +#ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> +static int __init reserve_crashkernel_low(unsigned long long low_size)
> +{
> +#ifdef CONFIG_64BIT
> +	unsigned long long low_base;
> +
> +	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
> +	if (!low_base) {
> +		pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
> +		return -ENOMEM;
> +	}
> +
> +	pr_info("crashkernel low memory reserved: 0x%08llx - 0x%08llx (%lld MB)\n",
> +		low_base, low_base + low_size, low_size >> 20);
> +
> +	crashk_low_res.start = low_base;
> +	crashk_low_res.end   = low_base + low_size - 1;
> +	insert_resource(&iomem_resource, &crashk_low_res);
> +#endif
> +	return 0;
> +}
> +
> +void __init reserve_crashkernel_generic(char *cmdline,
> +			     unsigned long long crash_size,
> +			     unsigned long long crash_base,
> +			     unsigned long long crash_low_size,
> +			     bool high)
> +{
> +	unsigned long long search_end = CRASH_ADDR_LOW_MAX, search_base = 0;
> +	bool fixed_base = false;
> +
> +	/* User specifies base address explicitly. */
> +	if (crash_base) {
> +		fixed_base = true;
> +		search_base = crash_base;
> +		search_end = crash_base + crash_size;
> +	}
> +
> +	if (high) {

It might be a little clearer to use "else if (high) {"

> +		search_base = CRASH_ADDR_LOW_MAX;
> +		search_end = CRASH_ADDR_HIGH_MAX;
> +	}
> +
> +retry:
> +	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
> +					       search_base, search_end);
> +	if (!crash_base) {
> +		/*
> +		 * For crashkernel=size[KMG]@offset[KMG], print out failure
> +		 * message if can't reserve the specified region.
> +		 */
> +		if (fixed_base) {
> +			pr_warn("crashkernel reservation failed - memory is in use.\n");
> +			return;
> +		}
> +
> +		/*
> +		 * For crashkernel=size[KMG], if the first attempt was for
> +		 * low memory, fall back to high memory, the minimum required
> +		 * low memory will be reserved later.
> +		 */
> +		if (!high && search_end == CRASH_ADDR_LOW_MAX) {
> +			search_end = CRASH_ADDR_HIGH_MAX;
> +			search_base = CRASH_ADDR_LOW_MAX;
> +			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> +			goto retry;
> +		}
> +
> +		/*
> +		 * For crashkernel=size[KMG],high, if the first attempt was
> +		 * for high memory, fall back to low memory.
> +		 */
> +		if (high && search_end == CRASH_ADDR_HIGH_MAX) {
> +			search_end = CRASH_ADDR_LOW_MAX;
> +			search_base = 0;
> +			goto retry;
> +		}
> +		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
> +			crash_size);
> +		return;
> +	}
> +
> +	if ((crash_base > CRASH_ADDR_LOW_MAX) &&
> +	     crash_low_size && reserve_crashkernel_low(crash_low_size)) {
> +		memblock_phys_free(crash_base, crash_size);
> +		return;
> +	}
> +
> +	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
> +		crash_base, crash_base + crash_size, crash_size >> 20);
> +
> +	/*
> +	 * The crashkernel memory will be removed from the kernel linear
> +	 * map. Inform kmemleak so that it won't try to access it.
> +	 */
> +	kmemleak_ignore_phys(crash_base);
> +	if (crashk_low_res.end)
> +		kmemleak_ignore_phys(crashk_low_res.start);
> +
> +	crashk_res.start = crash_base;
> +	crashk_res.end = crash_base + crash_size - 1;
> +	insert_resource(&iomem_resource, &crashk_res);
> +}
> +#endif
> +
>  Elf_Word *append_elf_note(Elf_Word *buf, char *name, unsigned int type,
>  			  void *data, size_t data_len)
>  {
> 

-- 
Regards,
  Zhen Lei



From thunder.leizhen at huaweicloud.com  Wed Aug 30 20:43:53 2023
From: thunder.leizhen at huaweicloud.com (Leizhen (ThunderTown))
Date: Thu, 31 Aug 2023 11:43:53 +0800
Subject: [PATCH v2 6/8] x86: kdump: use generic interface to simplify
 crashkernel reservation code
In-Reply-To: <20230829121610.138107-7-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
 <20230829121610.138107-7-bhe@redhat.com>
Message-ID: <5b53d460-7134-e292-77ec-967016e89003@huaweicloud.com>



On 2023/8/29 20:16, Baoquan He wrote:
> With the help of newly changed function parse_crashkernel() and
> generic reserve_crashkernel_generic(), crashkernel reservation can be
> simplified by steps:
> 
> 1) Add a new header file <asm/crash_core.h>, and define CRASH_ALIGN,
>    CRASH_ADDR_LOW_MAX, CRASH_ADDR_HIGH_MAX and
>    DEFAULT_CRASH_KERNEL_LOW_SIZE in <asm/crash_core.h>;
> 
> 2) Add arch_reserve_crashkernel() to call parse_crashkernel() and
>    reserve_crashkernel_generic(), and do the ARCH specific work if
>    needed.
> 
> 3) Add ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION Kconfig in
>    arch/x86/Kconfig.
> 
> When adding DEFAULT_CRASH_KERNEL_LOW_SIZE, add crash_low_size_default()
> to calculate crashkernel low memory because x86_64 has special
> requirement.
> 
> The old reserve_crashkernel_low() and reserve_crashkernel() can be
> removed.
> 
> Signed-off-by: Baoquan He <bhe at redhat.com>
> ---
>  arch/x86/Kconfig                  |   3 +
>  arch/x86/include/asm/crash_core.h |  34 +++++++
>  arch/x86/kernel/setup.c           | 144 ++++--------------------------
>  3 files changed, 53 insertions(+), 128 deletions(-)
>  create mode 100644 arch/x86/include/asm/crash_core.h
> 
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 8d9e4b362572..c4539dc35985 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -2037,6 +2037,9 @@ config KEXEC_FILE
>  config ARCH_HAS_KEXEC_PURGATORY
>  	def_bool KEXEC_FILE
>  
> +config ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> +	def_bool CRASH_CORE
> +
>  config KEXEC_SIG
>  	bool "Verify kernel signature during kexec_file_load() syscall"
>  	depends on KEXEC_FILE
> diff --git a/arch/x86/include/asm/crash_core.h b/arch/x86/include/asm/crash_core.h
> new file mode 100644
> index 000000000000..5fc5e4f94521
> --- /dev/null
> +++ b/arch/x86/include/asm/crash_core.h
> @@ -0,0 +1,34 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _X86_CRASH_CORE_H
> +#define _X86_CRASH_CORE_H
> +
> +/* 16M alignment for crash kernel regions */
> +#define CRASH_ALIGN             SZ_16M
> +
> +/*
> + * Keep the crash kernel below this limit.
> + *
> + * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
> + * due to mapping restrictions.
> + *
> + * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
> + * the upper limit of system RAM in 4-level paging mode. Since the kdump
> + * jump could be from 5-level paging to 4-level paging, the jump will fail if
> + * the kernel is put above 64 TB, and during the 1st kernel bootup there's
> + * no good way to detect the paging mode of the target kernel which will be
> + * loaded for dumping.
> + */
> +
> +#ifdef CONFIG_X86_32
> +# define CRASH_ADDR_LOW_MAX     SZ_512M
> +# define CRASH_ADDR_HIGH_MAX    SZ_512M
> +#else
> +# define CRASH_ADDR_LOW_MAX     SZ_4G
> +# define CRASH_ADDR_HIGH_MAX    SZ_64T
> +#endif
> +
> +# define DEFAULT_CRASH_KERNEL_LOW_SIZE crash_low_size_default()
> +
> +extern unsigned long crash_low_size_default(void);
> +
> +#endif /* _X86_CRASH_CORE_H */
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 382c66d2cf71..559a5c4141db 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -474,152 +474,40 @@ static void __init memblock_x86_reserve_range_setup_data(void)
>  /*
>   * --------- Crashkernel reservation ------------------------------
>   */
> -
> -/* 16M alignment for crash kernel regions */
> -#define CRASH_ALIGN		SZ_16M
> -
> -/*
> - * Keep the crash kernel below this limit.
> - *
> - * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
> - * due to mapping restrictions.
> - *
> - * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
> - * the upper limit of system RAM in 4-level paging mode. Since the kdump
> - * jump could be from 5-level paging to 4-level paging, the jump will fail if
> - * the kernel is put above 64 TB, and during the 1st kernel bootup there's
> - * no good way to detect the paging mode of the target kernel which will be
> - * loaded for dumping.
> - */
> -#ifdef CONFIG_X86_32
> -# define CRASH_ADDR_LOW_MAX	SZ_512M
> -# define CRASH_ADDR_HIGH_MAX	SZ_512M
> -#else
> -# define CRASH_ADDR_LOW_MAX	SZ_4G
> -# define CRASH_ADDR_HIGH_MAX	SZ_64T
> -#endif
> -
> -static int __init reserve_crashkernel_low(void)
> +unsigned long crash_low_size_default(void)
>  {
>  #ifdef CONFIG_X86_64
> -	unsigned long long base, low_base = 0, low_size = 0;
> -	unsigned long low_mem_limit;
> -	int ret;
> -
> -	low_mem_limit = min(memblock_phys_mem_size(), CRASH_ADDR_LOW_MAX);
> -
> -	/* crashkernel=Y,low */
> -	ret = parse_crashkernel_low(boot_command_line, low_mem_limit, &low_size, &base);
> -	if (ret) {
> -		/*
> -		 * two parts from kernel/dma/swiotlb.c:
> -		 * -swiotlb size: user-specified with swiotlb= or default.
> -		 *
> -		 * -swiotlb overflow buffer: now hardcoded to 32k. We round it
> -		 * to 8M for other buffers that may need to stay low too. Also
> -		 * make sure we allocate enough extra low memory so that we
> -		 * don't run out of DMA buffers for 32-bit devices.
> -		 */
> -		low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);
> -	} else {
> -		/* passed with crashkernel=0,low ? */
> -		if (!low_size)
> -			return 0;
> -	}
> -
> -	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
> -	if (!low_base) {
> -		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
> -		       (unsigned long)(low_size >> 20));
> -		return -ENOMEM;
> -	}
> -
> -	pr_info("Reserving %ldMB of low memory at %ldMB for crashkernel (low RAM limit: %ldMB)\n",
> -		(unsigned long)(low_size >> 20),
> -		(unsigned long)(low_base >> 20),
> -		(unsigned long)(low_mem_limit >> 20));
> -
> -	crashk_low_res.start = low_base;
> -	crashk_low_res.end   = low_base + low_size - 1;
> -	insert_resource(&iomem_resource, &crashk_low_res);
> -#endif
> +	return max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);
> +#else
>  	return 0;
> +#endif
>  }
>  
> -static void __init reserve_crashkernel(void)
> +static void __init arch_reserve_crashkernel(void)
>  {
> -	unsigned long long crash_size, crash_base, total_mem;
> +	unsigned long long crash_base, crash_size, low_size = 0;
> +	char *cmdline = boot_command_line;
>  	bool high = false;
>  	int ret;
>  
>  	if (!IS_ENABLED(CONFIG_KEXEC_CORE))
>  		return;
>  
> -	total_mem = memblock_phys_mem_size();
> -
> -	/* crashkernel=XM */
> -	ret = parse_crashkernel(boot_command_line, total_mem,
> -				&crash_size, &crash_base, NULL, NULL);
> -	if (ret != 0 || crash_size <= 0) {
> -		/* crashkernel=X,high */
> -		ret = parse_crashkernel_high(boot_command_line, total_mem,
> -					     &crash_size, &crash_base);
> -		if (ret != 0 || crash_size <= 0)
> -			return;
> -		high = true;
> -	}
> +	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
> +				&crash_size, &crash_base,
> +				&low_size, &high);
> +	if (ret)
> +		return;
>  
>  	if (xen_pv_domain()) {
>  		pr_info("Ignoring crashkernel for a Xen PV domain\n");
>  		return;
>  	}
>  
> -	/* 0 means: find the address automatically */
> -	if (!crash_base) {
> -		/*
> -		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
> -		 * crashkernel=x,high reserves memory over 4G, also allocates
> -		 * 256M extra low memory for DMA buffers and swiotlb.
> -		 * But the extra memory is not required for all machines.
> -		 * So try low memory first and fall back to high memory
> -		 * unless "crashkernel=size[KMG],high" is specified.
> -		 */
> -		if (!high)
> -			crash_base = memblock_phys_alloc_range(crash_size,
> -						CRASH_ALIGN, CRASH_ALIGN,
> -						CRASH_ADDR_LOW_MAX);
> -		if (!crash_base)
> -			crash_base = memblock_phys_alloc_range(crash_size,
> -						CRASH_ALIGN, CRASH_ALIGN,
> -						CRASH_ADDR_HIGH_MAX);
> -		if (!crash_base) {
> -			pr_info("crashkernel reservation failed - No suitable area found.\n");
> -			return;
> -		}
> -	} else {
> -		unsigned long long start;
> -
> -		start = memblock_phys_alloc_range(crash_size, SZ_1M, crash_base,
> -						  crash_base + crash_size);
> -		if (start != crash_base) {
> -			pr_info("crashkernel reservation failed - memory is in use.\n");
> -			return;
> -		}
> -	}
> -
> -	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
> -		memblock_phys_free(crash_base, crash_size);
> -		return;
> -	}
> -
> -	pr_info("Reserving %ldMB of memory at %ldMB for crashkernel (System RAM: %ldMB)\n",
> -		(unsigned long)(crash_size >> 20),
> -		(unsigned long)(crash_base >> 20),
> -		(unsigned long)(total_mem >> 20));
> +	reserve_crashkernel_generic(cmdline, crash_size, crash_base,
> +				    low_size, high);
>  
> -	crashk_res.start = crash_base;
> -	crashk_res.end   = crash_base + crash_size - 1;
> -	insert_resource(&iomem_resource, &crashk_res);
> +	return;

This can be omitted.

>  }
>  
>  static struct resource standard_io_resources[] = {
> @@ -1231,7 +1119,7 @@ void __init setup_arch(char **cmdline_p)
>  	 * Reserve memory for crash kernel after SRAT is parsed so that it
>  	 * won't consume hotpluggable memory.
>  	 */
> -	reserve_crashkernel();
> +	arch_reserve_crashkernel();
>  
>  	memblock_find_dma_reserve();
>  
> 

-- 
Regards,
  Zhen Lei



From thunder.leizhen at huaweicloud.com  Wed Aug 30 20:51:06 2023
From: thunder.leizhen at huaweicloud.com (Leizhen (ThunderTown))
Date: Thu, 31 Aug 2023 11:51:06 +0800
Subject: [PATCH v2 7/8] arm64: kdump: use generic interface to simplify
 crashkernel reservation
In-Reply-To: <20230829121610.138107-8-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
 <20230829121610.138107-8-bhe@redhat.com>
Message-ID: <54bb7826-ef3b-0ab2-68b3-e73c7e0d6b35@huaweicloud.com>



On 2023/8/29 20:16, Baoquan He wrote:
> With the help of newly changed function parse_crashkernel() and
> generic reserve_crashkernel_generic(), crashkernel reservation can be
> simplified by steps:
> 
> 1) Add a new header file <asm/crash_core.h>, and define CRASH_ALIGN,
>    CRASH_ADDR_LOW_MAX, CRASH_ADDR_HIGH_MAX and
>    DEFAULT_CRASH_KERNEL_LOW_SIZE in <asm/crash_core.h>;
> 
> 2) Add arch_reserve_crashkernel() to call parse_crashkernel() and
>    reserve_crashkernel_generic();
> 
> 3) Add ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION Kconfig in
>    arch/arm64/Kconfig.
> 
> The old reserve_crashkernel_low() and reserve_crashkernel() can be
> removed.

Reviewed-by: Zhen Lei <thunder.leizhen at huawei.com>

> 
> Signed-off-by: Baoquan He <bhe at redhat.com>
> ---
>  arch/arm64/Kconfig                  |   3 +
>  arch/arm64/include/asm/crash_core.h |  10 ++
>  arch/arm64/mm/init.c                | 140 ++--------------------------
>  3 files changed, 21 insertions(+), 132 deletions(-)
>  create mode 100644 arch/arm64/include/asm/crash_core.h
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 29db061db9bb..07fb8c71339d 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1481,6 +1481,9 @@ config KEXEC_FILE
>  	  for kernel and initramfs as opposed to list of segments as
>  	  accepted by previous system call.
>  
> +config ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
> +	def_bool CRASH_CORE
> +
>  config KEXEC_SIG
>  	bool "Verify kernel signature during kexec_file_load() syscall"
>  	depends on KEXEC_FILE
> diff --git a/arch/arm64/include/asm/crash_core.h b/arch/arm64/include/asm/crash_core.h
> new file mode 100644
> index 000000000000..9f5c8d339f44
> --- /dev/null
> +++ b/arch/arm64/include/asm/crash_core.h
> @@ -0,0 +1,10 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _ARM64_CRASH_CORE_H
> +#define _ARM64_CRASH_CORE_H
> +
> +/* Current arm64 boot protocol requires 2MB alignment */
> +#define CRASH_ALIGN                     SZ_2M
> +
> +#define CRASH_ADDR_LOW_MAX              arm64_dma_phys_limit
> +#define CRASH_ADDR_HIGH_MAX             (PHYS_MASK + 1)
> +#endif
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 4ad637508b75..48ab23531bb6 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -64,15 +64,6 @@ EXPORT_SYMBOL(memstart_addr);
>   */
>  phys_addr_t __ro_after_init arm64_dma_phys_limit;
>  
> -/* Current arm64 boot protocol requires 2MB alignment */
> -#define CRASH_ALIGN			SZ_2M
> -
> -#define CRASH_ADDR_LOW_MAX		arm64_dma_phys_limit
> -#define CRASH_ADDR_HIGH_MAX		(PHYS_MASK + 1)
> -#define CRASH_HIGH_SEARCH_BASE		SZ_4G
> -
> -#define DEFAULT_CRASH_KERNEL_LOW_SIZE	(128UL << 20)
> -
>  /*
>   * To make optimal use of block mappings when laying out the linear
>   * mapping, round down the base of physical memory to a size that can
> @@ -100,140 +91,25 @@ phys_addr_t __ro_after_init arm64_dma_phys_limit;
>  #define ARM64_MEMSTART_ALIGN	(1UL << ARM64_MEMSTART_SHIFT)
>  #endif
>  
> -static int __init reserve_crashkernel_low(unsigned long long low_size)
> -{
> -	unsigned long long low_base;
> -
> -	low_base = memblock_phys_alloc_range(low_size, CRASH_ALIGN, 0, CRASH_ADDR_LOW_MAX);
> -	if (!low_base) {
> -		pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
> -		return -ENOMEM;
> -	}
> -
> -	pr_info("crashkernel low memory reserved: 0x%08llx - 0x%08llx (%lld MB)\n",
> -		low_base, low_base + low_size, low_size >> 20);
> -
> -	crashk_low_res.start = low_base;
> -	crashk_low_res.end   = low_base + low_size - 1;
> -	insert_resource(&iomem_resource, &crashk_low_res);
> -
> -	return 0;
> -}
> -
> -/*
> - * reserve_crashkernel() - reserves memory for crash kernel
> - *
> - * This function reserves memory area given in "crashkernel=" kernel command
> - * line parameter. The memory reserved is used by dump capture kernel when
> - * primary kernel is crashing.
> - */
> -static void __init reserve_crashkernel(void)
> +static void __init arch_reserve_crashkernel(void)
>  {
> -	unsigned long long crash_low_size = 0, search_base = 0;
> -	unsigned long long crash_max = CRASH_ADDR_LOW_MAX;
> +	unsigned long long low_size = 0;
>  	unsigned long long crash_base, crash_size;
>  	char *cmdline = boot_command_line;
> -	bool fixed_base = false;
>  	bool high = false;
>  	int ret;
>  
>  	if (!IS_ENABLED(CONFIG_KEXEC_CORE))
>  		return;
>  
> -	/* crashkernel=X[@offset] */
>  	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
> -				&crash_size, &crash_base, NULL, NULL);
> -	if (ret == -ENOENT) {
> -		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
> -		if (ret || !crash_size)
> -			return;
> -
> -		/*
> -		 * crashkernel=Y,low can be specified or not, but invalid value
> -		 * is not allowed.
> -		 */
> -		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
> -		if (ret == -ENOENT)
> -			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> -		else if (ret)
> -			return;
> -
> -		search_base = CRASH_HIGH_SEARCH_BASE;
> -		crash_max = CRASH_ADDR_HIGH_MAX;
> -		high = true;
> -	} else if (ret || !crash_size) {
> -		/* The specified value is invalid */
> +				&crash_size, &crash_base,
> +				&low_size, &high);
> +	if (ret)
>  		return;
> -	}
> -
> -	crash_size = PAGE_ALIGN(crash_size);
> -
> -	/* User specifies base address explicitly. */
> -	if (crash_base) {
> -		fixed_base = true;
> -		search_base = crash_base;
> -		crash_max = crash_base + crash_size;
> -	}
> -
> -retry:
> -	crash_base = memblock_phys_alloc_range(crash_size, CRASH_ALIGN,
> -					       search_base, crash_max);
> -	if (!crash_base) {
> -		/*
> -		 * For crashkernel=size[KMG]@offset[KMG], print out failure
> -		 * message if can't reserve the specified region.
> -		 */
> -		if (fixed_base) {
> -			pr_warn("crashkernel reservation failed - memory is in use.\n");
> -			return;
> -		}
> -
> -		/*
> -		 * For crashkernel=size[KMG], if the first attempt was for
> -		 * low memory, fall back to high memory, the minimum required
> -		 * low memory will be reserved later.
> -		 */
> -		if (!high && crash_max == CRASH_ADDR_LOW_MAX) {
> -			crash_max = CRASH_ADDR_HIGH_MAX;
> -			search_base = CRASH_ADDR_LOW_MAX;
> -			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> -			goto retry;
> -		}
> -
> -		/*
> -		 * For crashkernel=size[KMG],high, if the first attempt was
> -		 * for high memory, fall back to low memory.
> -		 */
> -		if (high && crash_max == CRASH_ADDR_HIGH_MAX) {
> -			crash_max = CRASH_ADDR_LOW_MAX;
> -			search_base = 0;
> -			goto retry;
> -		}
> -		pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
> -			crash_size);
> -		return;
> -	}
> -
> -	if ((crash_base >= CRASH_ADDR_LOW_MAX) && crash_low_size &&
> -	     reserve_crashkernel_low(crash_low_size)) {
> -		memblock_phys_free(crash_base, crash_size);
> -		return;
> -	}
> -
> -	pr_info("crashkernel reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
> -		crash_base, crash_base + crash_size, crash_size >> 20);
> -
> -	/*
> -	 * The crashkernel memory will be removed from the kernel linear
> -	 * map. Inform kmemleak so that it won't try to access it.
> -	 */
> -	kmemleak_ignore_phys(crash_base);
> -	if (crashk_low_res.end)
> -		kmemleak_ignore_phys(crashk_low_res.start);
>  
> -	crashk_res.start = crash_base;
> -	crashk_res.end = crash_base + crash_size - 1;
> -	insert_resource(&iomem_resource, &crashk_res);
> +	reserve_crashkernel_generic(cmdline, crash_size, crash_base,
> +				    low_size, high);
>  }
>  
>  /*
> @@ -481,7 +357,7 @@ void __init bootmem_init(void)
>  	 * request_standard_resources() depends on crashkernel's memory being
>  	 * reserved, so do it here.
>  	 */
> -	reserve_crashkernel();
> +	arch_reserve_crashkernel();
>  
>  	memblock_dump_all();
>  }
> 

-- 
Regards,
  Zhen Lei



From thunder.leizhen at huaweicloud.com  Wed Aug 30 20:51:44 2023
From: thunder.leizhen at huaweicloud.com (Leizhen (ThunderTown))
Date: Thu, 31 Aug 2023 11:51:44 +0800
Subject: [PATCH v2 8/8] crash_core.c: remove unneeded functions
In-Reply-To: <20230829121610.138107-9-bhe@redhat.com>
References: <20230829121610.138107-1-bhe@redhat.com>
 <20230829121610.138107-9-bhe@redhat.com>
Message-ID: <720d11fc-aad9-41de-54db-8ebc1a06e336@huaweicloud.com>



On 2023/8/29 20:16, Baoquan He wrote:
> So far, nobody calls functions parse_crashkernel_high() and
> parse_crashkernel_low(), remove both of them.

Reviewed-by: Zhen Lei <thunder.leizhen at huawei.com>

> 
> Signed-off-by: Baoquan He <bhe at redhat.com>
> ---
>  include/linux/crash_core.h |  4 ----
>  kernel/crash_core.c        | 18 ------------------
>  2 files changed, 22 deletions(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 86e22e6a039f..d64006c4bd43 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -83,10 +83,6 @@ void final_note(Elf_Word *buf);
>  int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
>  		unsigned long long *crash_size, unsigned long long *crash_base,
>  		unsigned long long *low_size, bool *high);
> -int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
> -		unsigned long long *crash_size, unsigned long long *crash_base);
> -int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
> -		unsigned long long *crash_size, unsigned long long *crash_base);
>  
>  #ifdef CONFIG_ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
>  #ifndef DEFAULT_CRASH_KERNEL_LOW_SIZE
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index 6bc00cc390b5..61a8ea3b23a2 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -323,24 +323,6 @@ int __init parse_crashkernel(char *cmdline,
>  	return 0;
>  }
>  
> -int __init parse_crashkernel_high(char *cmdline,
> -			     unsigned long long system_ram,
> -			     unsigned long long *crash_size,
> -			     unsigned long long *crash_base)
> -{
> -	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> -				suffix_tbl[SUFFIX_HIGH]);
> -}
> -
> -int __init parse_crashkernel_low(char *cmdline,
> -			     unsigned long long system_ram,
> -			     unsigned long long *crash_size,
> -			     unsigned long long *crash_base)
> -{
> -	return __parse_crashkernel(cmdline, system_ram, crash_size, crash_base,
> -				suffix_tbl[SUFFIX_LOW]);
> -}
> -
>  /*
>   * Add a dummy early_param handler to mark crashkernel= as a known command line
>   * parameter and suppress incorrect warnings in init/main.c.
> 

-- 
Regards,
  Zhen Lei



From andy.shevchenko at gmail.com  Thu Aug 31 01:59:14 2023
From: andy.shevchenko at gmail.com (Andy Shevchenko)
Date: Thu, 31 Aug 2023 11:59:14 +0300
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <a4ba0493965ad32fcf95cd6e2439096668a9ed46.camel@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com> <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
 <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>
 <721a70c347d82931d12e5b75b19d132f82ee5ed2.camel@redhat.com>
 <CAHp75VdS=kSWnz8FzHcdrZPaeZKsQNbzjE9mNN7BDvZA_nQpaA@mail.gmail.com> <a4ba0493965ad32fcf95cd6e2439096668a9ed46.camel@redhat.com>
Message-ID: <CAHp75VdqFPJAnCm9A-S+TCChr9v9_s=Wfnb5wO2=vuwp1v+JvA@mail.gmail.com>

On Wed, Aug 30, 2023 at 10:15?PM <pstanner at redhat.com> wrote:
> On Wed, 2023-08-30 at 17:29 +0300, Andy Shevchenko wrote:
> > On Wed, Aug 30, 2023 at 5:19?PM <pstanner at redhat.com> wrote:
> > > On Wed, 2023-08-30 at 17:11 +0300, Andy Shevchenko wrote:
> > > > On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner
> > > > <pstanner at redhat.com>
> > > > wrote:

...

> > > > >  #include <linux/types.h>       /* for size_t */
> > > > >  #include <linux/stddef.h>      /* for NULL */
> > > > >  #include <linux/errno.h>       /* for E2BIG */
> > > > > +#include <linux/overflow.h>    /* for check_mul_overflow() */
> > > > > +#include <linux/err.h>         /* for ERR_PTR() */
> > > >
> > > > Can we preserve order (to some extent)?
> > >
> > > Sure. I just put it there so the comments build a congruent block.
> > > Which order would you prefer?
> >
> > Alphabetical.
> >
> > compiler.h
> > err.h
> > overflow.h
> > ...the rest that is a bit unordered...
> >
> > > > >  #include <linux/stdarg.h>
> > > > >  #include <uapi/linux/string.h>
>
> I mean I could include my own in a sorted manner ? but the existing
> ones are not sorted:

I know, that's why I put "(to some extent)" in my initial comment.

> We could sort them all, but I'd prefer to do that in a separate patch
> so that this commit does not make the impression of doing anything else
> than including the two new headers

But you can do your stuff first with better ordering than you proposed
initially, so there will be less churn for any additional change
(either that simply unifies the thing or something else).

> Such a separate patch could also unify the docstring style, see below

Sure, patches are welcome!

> > > > > +/**
> > > > > + * memdup_array_user - duplicate array from user space
> > > >
> > > > > + *
> > > >
> > > > Do we need this blank line?
> > >
> > > I more or less directly copied the docstring format from the
> > > original
> > > functions (v)memdup_user() in mm/util.c
> > > I guess this is common style?
> >
> > I think it's not. But you may grep kernel source tree and tell which
> > one occurs more often with or without this (unneeded) blank line.
>
>
> It seems to be very much mixed. string.h itself is mixed.
> When you look at the bottom of string.h, you'll find functions such as
> kbasename() that have the extra line.
>
> That's not really a super decisive point for me, though. We can remove
> the line I guess

I think the less LoCs the better generally speaking. So, if we don't
need that line and it doesn't make the readability worse, why to keep
it?

> > > > > + * @src: source address in user space
> > > > > + * @n: number of array members to copy
> > > > > + * @size: size of one array member
> > > > > + *
> > > > > + * Return: an ERR_PTR() on failure.  Result is physically
> > > > > + * contiguous, to be freed by kfree().
> > > > > + */

--
With Best Regards,
Andy Shevchenko


From pstanner at redhat.com  Thu Aug 31 05:16:54 2023
From: pstanner at redhat.com (Philipp Stanner)
Date: Thu, 31 Aug 2023 14:16:54 +0200
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <CAHp75VdqFPJAnCm9A-S+TCChr9v9_s=Wfnb5wO2=vuwp1v+JvA@mail.gmail.com>
References: <cover.1693386602.git.pstanner@redhat.com>
	 <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
	 <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>
	 <721a70c347d82931d12e5b75b19d132f82ee5ed2.camel@redhat.com>
	 <CAHp75VdS=kSWnz8FzHcdrZPaeZKsQNbzjE9mNN7BDvZA_nQpaA@mail.gmail.com>
	 <a4ba0493965ad32fcf95cd6e2439096668a9ed46.camel@redhat.com>
	 <CAHp75VdqFPJAnCm9A-S+TCChr9v9_s=Wfnb5wO2=vuwp1v+JvA@mail.gmail.com>
Message-ID: <98bf616420e4ef3c7c18a1f52fda7cd410137f7d.camel@redhat.com>

On Thu, 2023-08-31 at 11:59 +0300, Andy Shevchenko wrote:
> On Wed, Aug 30, 2023 at 10:15?PM <pstanner at redhat.com> wrote:
> > On Wed, 2023-08-30 at 17:29 +0300, Andy Shevchenko wrote:
> > > On Wed, Aug 30, 2023 at 5:19?PM <pstanner at redhat.com> wrote:
> > > > On Wed, 2023-08-30 at 17:11 +0300, Andy Shevchenko wrote:
> > > > > On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner
> > > > > <pstanner at redhat.com>
> > > Alphabetical.
> > > 
> > > compiler.h
> > > err.h
> > > overflow.h
> > > ...the rest that is a bit unordered...
> > > 
> > > > > > ?#include <linux/stdarg.h>
> > > > > > ?#include <uapi/linux/string.h>
> > 
> > I mean I could include my own in a sorted manner ? but the existing
> > ones are not sorted:
> 
> I know, that's why I put "(to some extent)" in my initial comment.
> 
> > We could sort them all, but I'd prefer to do that in a separate
> > patch
> > so that this commit does not make the impression of doing anything
> > else
> > than including the two new headers
> 
> But you can do your stuff first with better ordering than you
> proposed
> initially, so there will be less churn for any additional change
> (either that simply unifies the thing or something else).
> 
> 

I'll take care of those points in a v2 once I gathered some more
feedback from the other parties.

P.



From pstanner at redhat.com  Thu Aug 31 05:22:08 2023
From: pstanner at redhat.com (Philipp Stanner)
Date: Thu, 31 Aug 2023 14:22:08 +0200
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>
References: <cover.1693386602.git.pstanner@redhat.com>
	 <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
	 <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>
Message-ID: <94b893ef37fe0cc0bcc52e8b54c62be3853a7bbb.camel@redhat.com>

On Wed, 2023-08-30 at 17:11 +0300, Andy Shevchenko wrote:
> On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner <pstanner at redhat.com>
> wrote:
> > 
> > Currently, user array duplications are sometimes done without an
> > overflow check. Sometimes the checks are done manually; sometimes
> > the
> > array size is calculated with array_size() and sometimes by
> > calculating
> > n * size directly in code.
> > 
> > Introduce wrappers for arrays for memdup_user() and vmemdup_user()
> > to
> > provide a standardized and safe way for duplicating user arrays.
> > 
> > This is both for new code as well as replacing usage of
> > (v)memdup_user()
> > in existing code that uses, e.g., n * size to calculate array
> > sizes.
> 
> ...
> 
> > --- a/include/linux/string.h
> > +++ b/include/linux/string.h
> 
> I'm wondering if this has no side-effects as string.h/string.c IIRC
> is
> used also for early stages where some of the APIs are not available.
> 

I forgot to address this point in my previous reply.

Who's going to decide whether this is a problem or not?


My personal guess is that this is unlikely to be a problem because

   A. either (v)memdup_user() is available, in which case
      (v)memdup_array_user() will always work ??
   B. or (v)memdup_user() is not available, which would cause the code
      that currently uses (v)memdup_user() for copying arrays to fail
      anyways.


P.



From andy.shevchenko at gmail.com  Thu Aug 31 06:16:17 2023
From: andy.shevchenko at gmail.com (Andy Shevchenko)
Date: Thu, 31 Aug 2023 16:16:17 +0300
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <94b893ef37fe0cc0bcc52e8b54c62be3853a7bbb.camel@redhat.com>
References: <cover.1693386602.git.pstanner@redhat.com> <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
 <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com> <94b893ef37fe0cc0bcc52e8b54c62be3853a7bbb.camel@redhat.com>
Message-ID: <CAHp75VcNyE9uJu8-v2eVWRdZ3-nm2r-kdxE-naQB4VN6zxPFxg@mail.gmail.com>

On Thu, Aug 31, 2023 at 3:22?PM Philipp Stanner <pstanner at redhat.com> wrote:
> On Wed, 2023-08-30 at 17:11 +0300, Andy Shevchenko wrote:
> > On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner <pstanner at redhat.com>
> > wrote:

...

> > I'm wondering if this has no side-effects as string.h/string.c IIRC
> > is used also for early stages where some of the APIs are not available.
>
> I forgot to address this point in my previous reply.
>
> Who's going to decide whether this is a problem or not?
>
> My personal guess is that this is unlikely to be a problem because
>
>    A. either (v)memdup_user() is available, in which case
>       (v)memdup_array_user() will always work ?
>    B. or (v)memdup_user() is not available, which would cause the code
>       that currently uses (v)memdup_user() for copying arrays to fail
>       anyways.

It also uses something from overflow.h. I don't remember if that
header was ever been used in early stage modules (like
boot/decompressor).

-- 
With Best Regards,
Andy Shevchenko


From andy.shevchenko at gmail.com  Thu Aug 31 06:17:54 2023
From: andy.shevchenko at gmail.com (Andy Shevchenko)
Date: Thu, 31 Aug 2023 16:17:54 +0300
Subject: [PATCH 1/5] string.h: add array-wrappers for (v)memdup_user()
In-Reply-To: <CAHp75VcNyE9uJu8-v2eVWRdZ3-nm2r-kdxE-naQB4VN6zxPFxg@mail.gmail.com>
References: <cover.1693386602.git.pstanner@redhat.com> <46f667e154393a930a97d2218d8e90286d93a062.1693386602.git.pstanner@redhat.com>
 <CAHp75VfkzV-=XuEZwipYzfHNu4EXuwzbu6vfEKh1Uueseo2=wA@mail.gmail.com>
 <94b893ef37fe0cc0bcc52e8b54c62be3853a7bbb.camel@redhat.com> <CAHp75VcNyE9uJu8-v2eVWRdZ3-nm2r-kdxE-naQB4VN6zxPFxg@mail.gmail.com>
Message-ID: <CAHp75Vc6Bj87V5HVZeNXJRmN1qBYP8_eDDSj5tLnfL7d81o9+w@mail.gmail.com>

On Thu, Aug 31, 2023 at 4:16?PM Andy Shevchenko
<andy.shevchenko at gmail.com> wrote:
> On Thu, Aug 31, 2023 at 3:22?PM Philipp Stanner <pstanner at redhat.com> wrote:
> > On Wed, 2023-08-30 at 17:11 +0300, Andy Shevchenko wrote:
> > > On Wed, Aug 30, 2023 at 4:46?PM Philipp Stanner <pstanner at redhat.com>
> > > wrote:

...

> > > I'm wondering if this has no side-effects as string.h/string.c IIRC
> > > is used also for early stages where some of the APIs are not available.
> >
> > I forgot to address this point in my previous reply.
> >
> > Who's going to decide whether this is a problem or not?
> >
> > My personal guess is that this is unlikely to be a problem because
> >
> >    A. either (v)memdup_user() is available, in which case
> >       (v)memdup_array_user() will always work ?
> >    B. or (v)memdup_user() is not available, which would cause the code
> >       that currently uses (v)memdup_user() for copying arrays to fail
> >       anyways.
>
> It also uses something from overflow.h. I don't remember if that
> header was ever been used in early stage modules (like
> boot/decompressor).

Also we need to be sure UML is still buildable. Dunno if they are
using anything from this, but it appeared to me recently that someone
tried to optimize something using (internal) kernel headers and broke
the build in some cases.

-- 
With Best Regards,
Andy Shevchenko


From zohar at linux.ibm.com  Thu Aug 31 07:01:52 2023
From: zohar at linux.ibm.com (Mimi Zohar)
Date: Thu, 31 Aug 2023 10:01:52 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <CAHC9VhTbmmVrZCHCeZe-=s9fCGmEEDdb4p-skKJ=0wmX_K3arQ@mail.gmail.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
	 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
	 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
	 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
	 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
	 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
	 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
	 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
	 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
	 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
	 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
	 <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
	 <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>
	 <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com>
	 <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com>
	 <CAHC9VhR1s+p-q35xqHL5Jj2WuzE=bhMrK6PBBThaUBiMRxwvuQ@mail.gmail.com>
	 <0835ea7fd96df23419431301dfbc0d190e6c36f4.camel@linux.ibm.com>
	 <CAHC9VhTbmmVrZCHCeZe-=s9fCGmEEDdb4p-skKJ=0wmX_K3arQ@mail.gmail.com>
Message-ID: <99ff7c12dec73209a0a20464f3010c4c61d6c5f5.camel@linux.ibm.com>

On Wed, 2023-08-30 at 19:22 -0400, Paul Moore wrote:
> On Wed, Aug 30, 2023 at 7:07?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > On Wed, 2023-08-30 at 18:23 -0400, Paul Moore wrote:
> > > On Wed, Aug 30, 2023 at 6:21?PM Paul Moore <paul at paul-moore.com> wrote:
> > > > On Wed, Aug 30, 2023 at 5:50?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> > > > > > On Wed, Aug 30, 2023 at 4:25?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > > > Your initial question was "what happens if the file/filesystem becomes
> > > > > > > inaccessible at some point and an attestation client attempts to read
> > > > > > > the entire log?".  For what reason would it be inaccessible?  For the
> > > > > > > original single tmpfs file, what would make it inaccessible?
> > > > > >
> > > > > > In your reply that I had responded to you had mentioned that the
> > > > > > kernel was simply being passed a fd and taking ownership of it, the fd
> > > > > > could either be a tmpfs backed file or some form of persistent storage
> > > > > > as both were discussed in this thread.  I imagine a tmpfs filesystem
> > > > > > could still be forcibly unmounted, resulting in problems, but I can't
> > > > > > say that for certain.  However, there are definitely cases where a fd
> > > > > > backed against an arbitrary filesystem could run into problems:
> > > > > > storage device issues for local filesystems, networking issues for
> > > > > > network filesystems, and good old fashioned user/admin intervention in
> > > > > > both cases.
> > > > >
> > > > > "I imagine tmpfs filesystem could still be forcibly unmounted" sounds
> > > > > like an attack. Not being able to verify the measurement list against a
> > > > > quote is probably a good thing.
> > > >
> > > > Okay, can you answer the question for an arbitrary persistent
> > > > filesystem?  That was always the more important question, ...
> >
> > The original proposal, not mine, suggested using a tmpfs file.   The
> > idea of writing the measurements to a file on a persistent filesystem
> > wasn't mine either.  Sush/Tushar were pushing for writing to a
> > persistent file(s).  No argument from me that writing to a file on an
> > arbitrary persistent filesystem is not a good idea.
> 
> ...  Okay, so we all now agree that the kernel writing to an
> arbitrary filesystem is not a good idea, and using tmpfs doesn't solve
> the problem of general memory pressure (from previously in this
> thread), that's all helpful and good to clarify.

Do we also agree that the "tmpfs" solution would address the existing
design motivation - kernel memory pressure?

> 
> Assuming Sush and Tushar rework the document to clarify the
> motivation/purpose for the work, as you suggested earlier, I'm
> assuming we can revisit this problem and solutions?

In addition to the mismatch between the motivation and the design, the
design has some major flaws that first need to be addressed.

Assuming the design issues are addressed, please make sure the document
is written clearly and concisely.

-- 
thanks,

Mimi



From paul at paul-moore.com  Thu Aug 31 07:43:31 2023
From: paul at paul-moore.com (Paul Moore)
Date: Thu, 31 Aug 2023 10:43:31 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <99ff7c12dec73209a0a20464f3010c4c61d6c5f5.camel@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com>
 <277db5491460d5fd607785f2bcc733de39022a35.camel@linux.ibm.com>
 <bf794136-703a-0d33-e245-7e723007b5c0@linux.microsoft.com>
 <0e1511e8819b24ab8a34a7b15821f06eff688f29.camel@linux.ibm.com>
 <8bc0f024-fc12-cb32-7af0-e500948cc6db@linux.microsoft.com>
 <7e32afa2596b9d8cfdc275614575b2023cd1d673.camel@linux.ibm.com>
 <CAHC9VhSVO9t=9e9JmniXKoqqvMv42E4dVeYtQTWZ4Eih3Sfr0Q@mail.gmail.com>
 <ed16f899485cbd559f9e6d7fbe17fa0d92335c20.camel@linux.ibm.com>
 <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
 <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
 <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>
 <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com>
 <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com>
 <CAHC9VhR1s+p-q35xqHL5Jj2WuzE=bhMrK6PBBThaUBiMRxwvuQ@mail.gmail.com>
 <0835ea7fd96df23419431301dfbc0d190e6c36f4.camel@linux.ibm.com>
 <CAHC9VhTbmmVrZCHCeZe-=s9fCGmEEDdb4p-skKJ=0wmX_K3arQ@mail.gmail.com> <99ff7c12dec73209a0a20464f3010c4c61d6c5f5.camel@linux.ibm.com>
Message-ID: <CAHC9VhQbnyd2nvmL-t=3kXppsm985ps+NPJ5QDvM1WSS-Hd_Ew@mail.gmail.com>

On Thu, Aug 31, 2023 at 10:07?AM Mimi Zohar <zohar at linux.ibm.com> wrote:
> On Wed, 2023-08-30 at 19:22 -0400, Paul Moore wrote:
> > On Wed, Aug 30, 2023 at 7:07?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > On Wed, 2023-08-30 at 18:23 -0400, Paul Moore wrote:
> > > > On Wed, Aug 30, 2023 at 6:21?PM Paul Moore <paul at paul-moore.com> wrote:
> > > > > On Wed, Aug 30, 2023 at 5:50?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > > On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> > > > > > > On Wed, Aug 30, 2023 at 4:25?PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > > > > Your initial question was "what happens if the file/filesystem becomes
> > > > > > > > inaccessible at some point and an attestation client attempts to read
> > > > > > > > the entire log?".  For what reason would it be inaccessible?  For the
> > > > > > > > original single tmpfs file, what would make it inaccessible?
> > > > > > >
> > > > > > > In your reply that I had responded to you had mentioned that the
> > > > > > > kernel was simply being passed a fd and taking ownership of it, the fd
> > > > > > > could either be a tmpfs backed file or some form of persistent storage
> > > > > > > as both were discussed in this thread.  I imagine a tmpfs filesystem
> > > > > > > could still be forcibly unmounted, resulting in problems, but I can't
> > > > > > > say that for certain.  However, there are definitely cases where a fd
> > > > > > > backed against an arbitrary filesystem could run into problems:
> > > > > > > storage device issues for local filesystems, networking issues for
> > > > > > > network filesystems, and good old fashioned user/admin intervention in
> > > > > > > both cases.
> > > > > >
> > > > > > "I imagine tmpfs filesystem could still be forcibly unmounted" sounds
> > > > > > like an attack. Not being able to verify the measurement list against a
> > > > > > quote is probably a good thing.
> > > > >
> > > > > Okay, can you answer the question for an arbitrary persistent
> > > > > filesystem?  That was always the more important question, ...
> > >
> > > The original proposal, not mine, suggested using a tmpfs file.   The
> > > idea of writing the measurements to a file on a persistent filesystem
> > > wasn't mine either.  Sush/Tushar were pushing for writing to a
> > > persistent file(s).  No argument from me that writing to a file on an
> > > arbitrary persistent filesystem is not a good idea.
> >
> > ...  Okay, so we all now agree that the kernel writing to an
> > arbitrary filesystem is not a good idea, and using tmpfs doesn't solve
> > the problem of general memory pressure (from previously in this
> > thread), that's all helpful and good to clarify.
>
> Do we also agree that the "tmpfs" solution would address the existing
> design motivation - kernel memory pressure?

I haven't really looked into that so I'm not comfortable commenting on
that.  I can say that in off-list discussions with Sush, Tushar, and
others the issue we are facing is general memory pressure so using any
type of memory backed filesystem doesn't really resolve our problem.

> > Assuming Sush and Tushar rework the document to clarify the
> > motivation/purpose for the work, as you suggested earlier, I'm
> > assuming we can revisit this problem and solutions?
>
> In addition to the mismatch between the motivation and the design, the
> design has some major flaws that first need to be addressed.
>
> Assuming the design issues are addressed, please make sure the document
> is written clearly and concisely.

That feels like a normal part of the design, review, and iterate cycle
to me.  Sush and Tushar are following along and I'm sure they will
take all of that into account in the next revision of the design
document.

-- 
paul-moore.com


From greg at enjellic.com  Thu Aug 31 08:54:24 2023
From: greg at enjellic.com (Dr. Greg)
Date: Thu, 31 Aug 2023 10:54:24 -0500
Subject: [RFC] IMA Log Snapshotting Design Proposal - unseal
In-Reply-To: <1ef45099-da24-b73f-b33f-6a299c0b1696@linux.ibm.com>
References: <c5737141-7827-1c83-ab38-0119dcfea485@linux.microsoft.com> <1ef45099-da24-b73f-b33f-6a299c0b1696@linux.ibm.com>
Message-ID: <20230831155423.GA3820@wind.enjellic.com>

On Wed, Aug 30, 2023 at 03:12:39PM -0400, Ken Goldman wrote:

Good morning, I hope the day is going well for everyone.

> On 8/1/2023 3:12 PM, Sush Shringarputale wrote:
> 
> >For remote attestation to work, the service will need to know how to
> > validate the snapshot_aggregate entry in the IMA log.  It will have
> >to read the PCR values present in the template data of
> >snapshot_aggregate event in the latest IMA log, and ensure that the
> >PCR quotes align with the contents of the past UM_snapshot_file(s).
> >This will re-establish the chain of trust needed for the device to
> >pass remote attestation.  This will also maintain the ability of the
> >remote-attestation-service to seal the secrets, if the client-server
> > use TPM unseal mechanism to attest the state of the device.

> I think that seal/unseal to IMA PCRs is futile.  Since boot is
> multi-threaded, the IMA PCR is unpredictable even when valid.

Yes, unbiased observation calls into question the relevancy of the
2000-2002/Palladium model for TPM based integrity attestation, both
from a hardware and software perspective.  In addition to interference
by standard scheduling artifacts, the notion of everything being SMP
makes PCR invariancy a hopeless issue.

In fact, our trust orchestrators demonstrate that the very act of
modeling, ie computing file digests, causes an event ordering that
will be different from subsequent invocations when the digest value is
cached.  A concept that should be of no surprise to anyone fluent in
the writings of Heisenberg... :-)

We discuss this at some length, along with our proposed solution, in
the documentation that we provide with our TSEM LSM:

https://lore.kernel.org/linux-security-module/20230710102319.19716-1-greg at enjellic.com/T/#t

TSEM is a superset of IMA, given that file checksums are just one of
the components in the security state coefficients that our security
modeling is based on, but the issue is the same.

Having an invariant 'security state' value also simplifies the
attestation processw, since the relying party only needs to evaluate a
single signed value in order to verify that the workload has been
consistent with its desired security model.

It is also becoming increasingly apparent that advancing the art of
integrity and trusted systems will become problematic without the
ability to 'namespace' or partition integrity on a workload by
workload basis.  We also introduce support for that with our security
modeling namespaces.

I missed copying the LSM list on my reply, but I discussed some of the
challenges that TDX, and COCO in general, raises to current integrity
infrastructures in the following thread, which should be easily
locatable on the LKML or COCO lists:

tsm: Introduce a shared ABI for attestation reports

Given our experiences, it is difficult to understand how the notion of
'legitimate' confidential computing is going to grow from 7 billion
dollars to 55 billion dollars over the next five years without the
concept of better 'trust orchestration'.

Have a good remainder of the week.

As always,
Dr. Greg

The Quixote Project - Flailing at the Travails of Cybersecurity


From greg at enjellic.com  Thu Aug 31 09:46:27 2023
From: greg at enjellic.com (Dr. Greg)
Date: Thu, 31 Aug 2023 11:46:27 -0500
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <CAHC9VhTbmmVrZCHCeZe-=s9fCGmEEDdb4p-skKJ=0wmX_K3arQ@mail.gmail.com>
References: <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com> <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com> <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com> <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com> <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com> <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com> <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com> <CAHC9VhR1s+p-q35xqHL5Jj2WuzE=bhMrK6PBBThaUBiMRxwvuQ@mail.gmail.com> <0835ea7fd96df23419431301dfbc0d190e6c36f4.camel@linux.ibm.com> <CAHC9VhTbmmVrZCHCeZe-=s9fCGmEEDdb4p-skKJ=0wmX_K3arQ@mail.gmail.com>
Message-ID: <20230831164627.GA4371@wind.enjellic.com>

On Wed, Aug 30, 2023 at 07:22:27PM -0400, Paul Moore wrote:

Good morning.

> On Wed, Aug 30, 2023 at 7:07???PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > On Wed, 2023-08-30 at 18:23 -0400, Paul Moore wrote:
> > > On Wed, Aug 30, 2023 at 6:21???PM Paul Moore <paul at paul-moore.com> wrote:
> > > > On Wed, Aug 30, 2023 at 5:50???PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> > > > > > On Wed, Aug 30, 2023 at 4:25???PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > > > Your initial question was "what happens if the file/filesystem becomes
> > > > > > > inaccessible at some point and an attestation client attempts to read
> > > > > > > the entire log?".  For what reason would it be inaccessible?  For the
> > > > > > > original single tmpfs file, what would make it inaccessible?
> > > > > >
> > > > > > In your reply that I had responded to you had mentioned that the
> > > > > > kernel was simply being passed a fd and taking ownership of it, the fd
> > > > > > could either be a tmpfs backed file or some form of persistent storage
> > > > > > as both were discussed in this thread.  I imagine a tmpfs filesystem
> > > > > > could still be forcibly unmounted, resulting in problems, but I can't
> > > > > > say that for certain.  However, there are definitely cases where a fd
> > > > > > backed against an arbitrary filesystem could run into problems:
> > > > > > storage device issues for local filesystems, networking issues for
> > > > > > network filesystems, and good old fashioned user/admin intervention in
> > > > > > both cases.
> > > > >
> > > > > "I imagine tmpfs filesystem could still be forcibly unmounted" sounds
> > > > > like an attack. Not being able to verify the measurement list against a
> > > > > quote is probably a good thing.
> > > >
> > > > Okay, can you answer the question for an arbitrary persistent
> > > > filesystem?  That was always the more important question, ...
> >
> > The original proposal, not mine, suggested using a tmpfs file.   The
> > idea of writing the measurements to a file on a persistent filesystem
> > wasn't mine either.  Sush/Tushar were pushing for writing to a
> > persistent file(s).  No argument from me that writing to a file on an
> > arbitrary persistent filesystem is not a good idea.

> Progress.  Okay, so we all now agree that the kernel writing to an
> arbitrary filesystem is not a good idea, and using tmpfs doesn't
> solve the problem of general memory pressure (from previously in
> this thread), that's all helpful and good to clarify.
>
> Assuming Sush and Tushar rework the document to clarify the
> motivation/purpose for the work, as you suggested earlier, I'm
> assuming we can revisit this problem and solutions?

IMA will obviously go, with our blessings, in its own direction.

I would only call out, as I indicated in my reply to Ken Goldman, that
our 20+ year old integrity and attestation architectures and models
are now arguably challenged, given emerging technologies and their
requirements for integrity and confidentiality.

This thread highlights a lot of the issues that caused us to bring
TSEM to the table.

If we, for the moment, isolate discussion to the growth and/or
management of the event log.  The architecture of externally modeled
namespaces gets the events immediately out of the kernel for a
userspace Trusted Modeling Agent (TMA) to deal with, however it
chooses.

The TMA doesn't even need to be running on the machine itself.  We are
currently feeding security event streams to entirely different
platforms, some hypervisor based, particularly for the training and
enforcement of machine learning models.

There are obviously concessions that need to be made for LSM event
handlers running in atomic context, but that isn't an issue for a
security model based simply on file integrity.  IMA in atomic context
isn't possible, not only from the perspective of doing the file I/O
but the need to sleep while the TPM transaction runs.

Once again, no criticism of IMA, it is proven and has its own camp and
following.  From our perspective, and our collaborators, there is an
opportunity, and need, to enhance the options available.

> paul-moore.com

Best wishes for an end of summer holiday weekend.

As always,
Dr. Greg

The Quixote Project - Flailing at the Travails of Cybersecurity


From skinsburskii at linux.microsoft.com  Wed Aug 30 19:24:43 2023
From: skinsburskii at linux.microsoft.com (Stanislav Kinsburskii)
Date: Wed, 30 Aug 2023 19:24:43 -0700
Subject: [RFC PATCH] Introduce persistent memory pool
In-Reply-To: <1697efce-665a-43d5-b0be-7c03c0a4d850@app.fastmail.com>
References: <64e7cbf7.050a0220.114c7.b70dSMTPIN_ADDED_BROKEN@mx.google.com>
 <2023082506-enchanted-tripping-d1d5@gregkh>
 <c26ad989dcc6737dd295e980c78ef53740098810.camel@amazon.com>
 <20230823024500.GA25462@skinsburskii.>
 <e0ed9fb9-8e7a-44ad-976a-27362f6e537a@amazon.de>
 <20230829220740.GA26605@skinsburskii.>
 <82416797-01aa-471a-a737-471297e37c4c@amazon.de>
 <1697efce-665a-43d5-b0be-7c03c0a4d850@app.fastmail.com>
Message-ID: <20230831022443.GA10414@skinsburskii.>

On Wed, Aug 30, 2023 at 07:39:18PM -0400, Arnd Bergmann wrote:
> On Wed, Aug 30, 2023, at 03:20, Alexander Graf wrote:
> > On 30.08.23 00:07, Stanislav Kinsburskii wrote:
> >> On Mon, Aug 28, 2023 at 10:50:19PM +0200, Alexander Graf wrote:
> 
> >> Device tree or ACPI are options indeed. However AFAIU in case of DT user
> >> space has to involved into the picture to modify and complie it, while
> >> ACPI isn't flexible or easily extendable.
> >> Also, AFAIU both these standards were designed with passing
> >> hardware-specific data in mind from bootstrap software to an OS kernel
> >> and thus were never really intended to be used for creating a persistent
> >> state accross kexec.
> >> To me, an attempt to use either of them to pass kernel-specific data looks
> >> like an abuse (or misuse) excused by the simplicity of implementation.
> >
> >
> > What I was describing above is that the Linux boot protocol already has 
> > natural ways to pass a DT (arm) or set of ACPI tables (x86) to the 
> > target kernel. Whatever we do here should either piggy back on top of 
> > those natural mechanisms (e.g. /chosen node in DT) or be on the same 
> > level (e.g. pass DT in one register, pass metadata structure in another 
> > register).
> >
> > When it comes to the actual content of the metadata, I'm personally also 
> > leaning towards DT. We already have libfdt inside the kernel. It gives 
> > is a very simple, well understood structured file format that you can 
> > extend, version, etc etc. And the kernel has mechanisms to modify fdt 
> > contents.
> 
> Agreed. This also makes a lot of sense since the fdt format was
> originally introduced for this exact purpose, to be a key-value
> store to pass data from the running kernel to the next one after
> kexec when the original source of the data (originally open
> firmware) is gone. It only turned into the generic way to
> describe embedded systems later on, but both the fdt binary
> format and the kexec infrastructure for manipulating and
> passing the blob should be easy to reuse for additional purposes
> as long as the contents are put into appropriate namespaces that
> don't clash with existing usage.
> 

I see. Using DT instead of introducing another format to pass metadata
is quite tempting.
The only problem I see here is that it enforces a user of x86 host to
use device tree.
I guess such a device tree can be fully constructred in kernel and then
added to kexec parameters by kernel itself thus keeping the such
matadata management independent from user setup.
But then how to combine this in-kernel device tree with the one,
provided by user?
Also, AFAIK user can override device tree via kexec paramaters, so I
guess placing kernel-specific data into a DT, controlled by user, isn't
very reliable.
Is there a way to pass multiple device trees over kexec?
Or should there be an implicit extention to patch user provided DT
with kernel metadata before kexec?

Thanks,
Stanislav

>        Arnd


From paul at paul-moore.com  Thu Aug 31 10:56:52 2023
From: paul at paul-moore.com (Paul Moore)
Date: Thu, 31 Aug 2023 13:56:52 -0400
Subject: [RFC] IMA Log Snapshotting Design Proposal
In-Reply-To: <20230831164627.GA4371@wind.enjellic.com>
References: <CAHC9VhTjej-GFjvEQhnL-HWnnkvJKA_DuOA_Md1KkV24Tx5haA@mail.gmail.com>
 <077249ac2bf2cb6d34347514e921720bb0f30b66.camel@linux.ibm.com>
 <CAHC9VhTvK=sJUgCUS0H9BWWXPnj3e0XkfE-4vB3-oxyt2_Wj9w@mail.gmail.com>
 <930e517c724fac27f7a1a4165af51dbc37cce4a0.camel@linux.ibm.com>
 <CAHC9VhStr3BAzb3tyHzHVPXzzuxyXjPQ4vmi+SrJqbTWio04+Q@mail.gmail.com>
 <2d800c3c0b6b4908843b490c36ef9df0cb4da134.camel@linux.ibm.com>
 <CAHC9VhSRepx+Q5pwhD3+AYPLXJAeL84FykLDnABfAwCUWXaWJA@mail.gmail.com>
 <CAHC9VhR1s+p-q35xqHL5Jj2WuzE=bhMrK6PBBThaUBiMRxwvuQ@mail.gmail.com>
 <0835ea7fd96df23419431301dfbc0d190e6c36f4.camel@linux.ibm.com>
 <CAHC9VhTbmmVrZCHCeZe-=s9fCGmEEDdb4p-skKJ=0wmX_K3arQ@mail.gmail.com> <20230831164627.GA4371@wind.enjellic.com>
Message-ID: <CAHC9VhQx6LxSGEj_quZtVnf6VXfiBwazqpEqHdJT9Ro7BWTYHg@mail.gmail.com>

On Thu, Aug 31, 2023 at 12:47?PM Dr. Greg <greg at enjellic.com> wrote:
> On Wed, Aug 30, 2023 at 07:22:27PM -0400, Paul Moore wrote:
> > On Wed, Aug 30, 2023 at 7:07???PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > On Wed, 2023-08-30 at 18:23 -0400, Paul Moore wrote:
> > > > On Wed, Aug 30, 2023 at 6:21???PM Paul Moore <paul at paul-moore.com> wrote:
> > > > > On Wed, Aug 30, 2023 at 5:50???PM Mimi Zohar <zohar at linux.ibm.com> wrote:
> > > > > > On Wed, 2023-08-30 at 16:47 -0400, Paul Moore wrote:
> > > > > > > On Wed, Aug 30, 2023 at 4:25???PM Mimi Zohar <zohar at linux.ibm.com> wrote:

...

> > Assuming Sush and Tushar rework the document to clarify the
> > motivation/purpose for the work, as you suggested earlier, I'm
> > assuming we can revisit this problem and solutions?
>
> IMA will obviously go, with our blessings, in its own direction.
>
> I would only call out, as I indicated in my reply to Ken Goldman, that
> our 20+ year old integrity and attestation architectures and models
> are now arguably challenged, given emerging technologies and their
> requirements for integrity and confidentiality.
>
> This thread highlights a lot of the issues that caused us to bring
> TSEM to the table ...

It would be helpful if you could refrain from hijacking barely
tangentially related threads as a marketing pitch for TSEM.

Everyone is welcome, and encouraged (!), to join the upstream
discussion threads, especially when one has something relevant to
contribute which hasn't been previously discussed.  However, when
every other contribution an individual makes follows the pattern of
"we could solve all of this if we only had feature 'X' which I
helpfully posted here <lore link>" then those contributions grow very
tiring and I tend to start ignoring that individual.

-- 
paul-moore.com


