[PATCH v7 0/4] kexec: Fix kexec_file_load for llvm16 with PGO

Song Liu song at kernel.org
Thu Sep 7 16:08:11 PDT 2023


Hi Ricardo and folks,

On Fri, May 19, 2023 at 7:48 AM Ricardo Ribalda <ribalda at chromium.org> wrote:
>
> When upreving llvm I realised that kexec stopped working on my test
> platform.
>
> The reason seems to be that due to PGO there are multiple .text sections
> on the purgatory, and kexec does not supports that.
>
> Signed-off-by: Ricardo Ribalda <ribalda at chromium.org>

We are seeing WARNINGs like the following while kexec'ing a PGO and
LTO enabled kernel:

WARNING: CPU: 26 PID: 110894 at kernel/kexec_file.c:919
kexec_load_purgatory+0x37f/0x390

AFAICT, the warning was added by this set, and it was triggered when
we have many .text sections
in purgatory.ro. The kexec was actually successful. So I wonder
whether we really need the
WARNING here. If we disable LTO (PGO is still enabled), we don't see
the WARNING any more.

I also tested an older kernel (5.19 based), where we also see many
.text sections with LTO. It
kexec()'ed fine. (It doesn't have the WARN_ON() in
kexec_purgatory_setup_sechdrs).

Please help us fix this properly (as I really don't know much about kexec).

Thanks in advance,
Song


Here is readelf -S output on purgatory.ro.

With LTO:

readelf -W -S purgatory.ro
There are 48 section headers, starting at offset 0x4a10:

Section Headers:
  [Nr] Name              Type            Address          Off    Size
 ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000
000000 00      0   0  0
  [ 1] .text             PROGBITS        0000000000000000 000040
0000d0 00  AX  0   0 16
  [ 2] .data             PROGBITS        0000000000000000 001000
001000 00  WA  0   0 4096
  [ 3] .rela.text        RELA            0000000000000000 003788
000228 18   I 45   1  8
  [ 4] .rodata           PROGBITS        0000000000000000 002000
0000e0 00   A  0   0 16
  [ 5] .rela.rodata      RELA            0000000000000000 0039b0
000030 18   I 45   4  8
  [ 6] .bss              NOBITS          0000000000000000 0020e0
001000 00  WA  0   0 4096
  [ 7] .text.purgatory   PROGBITS        0000000000000000 0020e0
0000df 00  AX  0   0 16
  [ 8] .rela.text.purgatory RELA            0000000000000000 0039e0
000060 18   I 45   7  8
  [ 9] .text.warn        PROGBITS        0000000000000000 0021c0
000001 00  AX  0   0 16
  [10] .kexec-purgatory  PROGBITS        0000000000000000 0021d0
000120 00  WA  0   0 16
  [11] .comment          PROGBITS        0000000000000000 003a40
000046 01  MS  0   0  1
  [12] .llvm_addrsig     LOOS+0xfff4c03  0000000000000000 003a86
000005 00   E  0   0  1
  [13] .text.sha256_update PROGBITS        0000000000000000 0022f0
0008eb 00  AX  0   0 16
  [14] .rela.text.sha256_update RELA            0000000000000000
003a90 000060 18   I 45  13  8
  [15] .text.sha224_update PROGBITS        0000000000000000 002be0
00000c 00  AX  0   0 16
  [16] .rela.text.sha224_update RELA            0000000000000000
003af0 000018 18   I 45  15  8
  [17] .text.sha256_final PROGBITS        0000000000000000 002bf0
0000cd 00  AX  0   0 16
  [18] .rela.text.sha256_final RELA            0000000000000000 003b08
000030 18   I 45  17  8
  [19] .text.sha224_final PROGBITS        0000000000000000 002cc0
0000bd 00  AX  0   0 16
  [20] .rela.text.sha224_final RELA            0000000000000000 003b38
000030 18   I 45  19  8
  [21] .text.sha256      PROGBITS        0000000000000000 002d80
00011d 00  AX  0   0 16
  [22] .rela.text.sha256 RELA            0000000000000000 003b68
000030 18   I 45  21  8
  [23] .modinfo          PROGBITS        0000000000000000 002e9d
000039 00   A  0   0  1
  [24] .rodata.SHA256_K  PROGBITS        0000000000000000 002ee0
000100 00   A  0   0 16
  [25] .rodata.__sha256_final.padding PROGBITS        0000000000000000
002fe0 000040 00   A  0   0 16
  [26] .text.memcmp      PROGBITS        0000000000000000 003020
00000b 00  AX  0   0 16
  [27] .text.bcmp        PROGBITS        0000000000000000 003030
00000b 00  AX  0   0 16
  [28] .text.strcmp      PROGBITS        0000000000000000 003040
000041 00  AX  0   0 16
  [29] .text.strncmp     PROGBITS        0000000000000000 003090
00003a 00  AX  0   0 16
  [30] .text.strnlen     PROGBITS        0000000000000000 0030d0
000039 00  AX  0   0 16
  [31] .text.atou        PROGBITS        0000000000000000 003110
000035 00  AX  0   0 16
  [32] .text.simple_strtoull PROGBITS        0000000000000000 003150
0000b6 00  AX  0   0 16
  [33] .text.simple_strtol PROGBITS        0000000000000000 003210
0001b6 00  AX  0   0 16
  [34] .text.strlen      PROGBITS        0000000000000000 0033d0
00001c 00  AX  0   0 16
  [35] .text.strstr      PROGBITS        0000000000000000 0033f0
00005f 00  AX  0   0 16
  [36] .text.strchr      PROGBITS        0000000000000000 003450
000022 00  AX  0   0 16
  [37] .text.kstrtoull   PROGBITS        0000000000000000 003480
000142 00  AX  0   0 16
  [38] .text.boot_kstrtoul PROGBITS        0000000000000000 0035d0
00000c 00  AX  0   0 16
  [39] .rela.text.boot_kstrtoul RELA            0000000000000000
003b98 000018 18   I 45  38  8
  [40] .text.memset      PROGBITS        0000000000000000 0035e0
00001f 00  AX  0   0 16
  [41] .text.memmove     PROGBITS        0000000000000000 003600
0000a6 00  AX  0   0 16
  [42] .text.memcpy      PROGBITS        0000000000000000 0036b0
0000a6 00  AX  0   0 16
  [43] .rodata.str1.1    PROGBITS        0000000000000000 003756
000032 01 AMS  0   0  1
  [44] .note.GNU-stack   PROGBITS        0000000000000000 003bb0
000000 00      0   0  1
  [45] .symtab           SYMTAB          0000000000000000 003bb0
000948 18     47  68  8
  [46] .shstrtab         STRTAB          0000000000000000 0044f8
0002cd 00      0   0  1
  [47] .strtab           STRTAB          0000000000000000 0047c5
000248 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  l (large), p (processor specific)


Without LTO:

readelf -W -S purgatory.ro
There are 16 section headers, starting at offset 0x4130:

Section Headers:
  [Nr] Name              Type            Address          Off    Size
 ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000
000000 00      0   0  0
  [ 1] .text             PROGBITS        0000000000000000 000040
00131b 00  AX  0   0 16
  [ 2] .rela.text        RELA            0000000000000000 003290
000480 18   I 13   1  8
  [ 3] .kexec-purgatory  PROGBITS        0000000000000000 001360
000120 00  WA  0   0 16
  [ 4] .comment          PROGBITS        0000000000000000 003710
000046 01  MS  0   0  1
  [ 5] .llvm_addrsig     LOOS+0xfff4c03  0000000000000000 003756
000005 00   E  0   0  1
  [ 6] .data             PROGBITS        0000000000000000 002000
001000 00  WA  0   0 4096
  [ 7] .rodata           PROGBITS        0000000000000000 003000
000220 00   A  0   0 16
  [ 8] .rela.rodata      RELA            0000000000000000 003760
000030 18   I 13   7  8
  [ 9] .bss              NOBITS          0000000000000000 003220
001000 00  WA  0   0 4096
  [10] .modinfo          PROGBITS        0000000000000000 003220
000039 00   A  0   0  1
  [11] .rodata.str1.1    PROGBITS        0000000000000000 003259
000032 01 AMS  0   0  1
  [12] .note.GNU-stack   PROGBITS        0000000000000000 003790
000000 00      0   0  1
  [13] .symtab           SYMTAB          0000000000000000 003790
0006d8 18     15  43  8
  [14] .shstrtab         STRTAB          0000000000000000 003e68
00009c 00      0   0  1
  [15] .strtab           STRTAB          0000000000000000 003f04
00022b 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  l (large), p (processor specific)

[...]

> Ricardo Ribalda (4):
>       kexec: Support purgatories with .text.hot sections
>       x86/purgatory: Remove PGO flags
>       powerpc/purgatory: Remove PGO flags
>       riscv/purgatory: Remove PGO flags
>
>  arch/powerpc/purgatory/Makefile |  5 +++++
>  arch/riscv/purgatory/Makefile   |  5 +++++
>  arch/x86/purgatory/Makefile     |  5 +++++
>  kernel/kexec_file.c             | 14 +++++++++++++-
>  4 files changed, 28 insertions(+), 1 deletion(-)
> ---
> base-commit: 58390c8ce1bddb6c623f62e7ed36383e7fa5c02f
> change-id: 20230321-kexec_clang16-4510c23d129c
>
> Best regards,
> --
> Ricardo Ribalda Delgado <ribalda at chromium.org>
>



More information about the linux-riscv mailing list