[PATCH v3] RISC-V: build: Allow LTO to be selected

[Nathan Chancellor]

From: Wende Tan <twd2.me at gmail.com>

Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an
issue [1] in prior LLD versions that prevents LLD to generate proper
machine code for RISC-V when writing `nop`s.

I have tested enabling LTO for `defconfig`. The LLD took ~2m21s and ~3GiB
on our Intel Xeon Gold 6140 server and produced an 18MiB Image. The image
can boot to shell using an archriscv rootfs on QEMU.

I have also tested it for `allyesconfig` without COMPILE_TEST, FTRACE,
KASAN, and GCOV. The LLD took ~7h03m and ~335GiB on the server,
successfully producing a 1.7GiB Image. Unfortunately, we cannot boot this
image because the `create_kernel_page_table()` -> `alloc_pmd_early()` ->
`BUG_ON()` logic limits the image to be < 1GiB. Maybe we can fix it in a
separate patch further.

Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF
object file when manipulating the files in that subfolder, rather than
LLVM bitcode.

[1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM
    commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops")

Tested-by: Wende Tan <twd2.me at gmail.com>
Signed-off-by: Wende Tan <twd2.me at gmail.com>
Co-developed-by: Nathan Chancellor <nathan at kernel.org>
Signed-off-by: Nathan Chancellor <nathan at kernel.org>
---
NOTE: I tested LLVM 14 through 18 with defconfig + full/thin LTO and
allmodconfig + thin LTO. allmodconfig + thin LTO with LLVM 15 and 16
shows

  ld.lld: error: section size decrease is too large

when linking vmlinux. This appears to be resolved in LLVM 17 with
https://github.com/llvm/llvm-project/commit/9d37ea95df1b84cca9b5e954d8964c976a5e303e
(I did not bisect but the commit message lines up with the issue). I
kept the existing version check because defconfig worked fine but we may
want to bump it to 17.0.0 if randconfigs trip over this.

Changes in v3:
- Disable LTO in arch/riscv/kernel/pi/Makefile, which was added to the
  kernel after the submission of v2. This change matches arm64.
- Link to v2: https://lore.kernel.org/r/20220512205545.992288-1-twd2.me@gmail.com/

Changes in v2:
- Some textual changes suggested by Nick.
- Drop the changes to `arch/riscv/Makefile`, since the LLVM issue is
  filed and resolved.
- Drop the unnecessary changes to `arch/riscv/kernel/vdso/Makefile`.
- Link to v1: https://lore.kernel.org/r/20210719205208.1023221-1-twd2.me@gmail.com/
---
 arch/riscv/Kconfig            | 3 +++
 arch/riscv/kernel/pi/Makefile | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index d607ab0f7c6d..523640f7441e 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -46,6 +46,9 @@ config RISCV
 	select ARCH_SUPPORTS_CFI_CLANG
 	select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU
 	select ARCH_SUPPORTS_HUGETLBFS if MMU
+	# LLD >= 14: https://github.com/llvm/llvm-project/issues/50505
+	select ARCH_SUPPORTS_LTO_CLANG if LLD_VERSION >= 140000
+	select ARCH_SUPPORTS_LTO_CLANG_THIN if LLD_VERSION >= 140000
 	select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
 	select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
 	select ARCH_USE_MEMTEST
diff --git a/arch/riscv/kernel/pi/Makefile b/arch/riscv/kernel/pi/Makefile
index 07915dc9279e..b75f150b923d 100644
--- a/arch/riscv/kernel/pi/Makefile
+++ b/arch/riscv/kernel/pi/Makefile
@@ -9,6 +9,9 @@ KBUILD_CFLAGS	:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
 		   -fno-asynchronous-unwind-tables -fno-unwind-tables \
 		   $(call cc-option,-fno-addrsig)
 
+# Disable LTO
+KBUILD_CFLAGS	:= $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
+
 KBUILD_CFLAGS	+= -mcmodel=medany
 
 CFLAGS_cmdline_early.o += -D__NO_FORTIFY

---
base-commit: 8a749fd1a8720d4619c91c8b6e7528c0a355c0aa
change-id: 20231003-riscv-lto-f013beed8587

Best regards,
-- 
Nathan Chancellor <nathan at kernel.org>