[PATCH v7 00/33] arm64: robustify boot sequence and add support for WXN

[Ard Biesheuvel]

The purpose of this series is to make the boot sequence more robust, and
more efficient, by trying to adhere to the following principles:
- do as little as possible before enabling the MMU and caches [1]
- do everything only once
- do everything from C code
- don't rely on mappings that are executable and writable at the same
  time [2]
- avoid handling external input while the kernel text is mapped
  writable.

[1] has been addressed already with the exception of some EFI specific
tweaks that have been sent out separately:
https://lore.kernel.org/all/20221108182204.2447664-1-ardb@kernel.org/

[2] this is a useful principle in general, but also a prerequisite for
allowing WXN to be enabled, which was the inspiration for this work.

The size of the series has ballooned a bit, but I think the first ~8
patches could be picked up piecemeal, while the remainder is being
discussed and maybe respun again (or rejected).

The bulk of the series covers changes that are needed so that the kernel
mapping is created only once, and before it is used to execute the
kernel. All C code that implements this and its dependencies is moved
into the early mini C runtime that was added recently for the early
KASLR init code.

The most intrusive changes of the series are the ones that modify the
early CPU feature override detection in idreg-override.c. This is needed
to get rid of absolute symbol references (which is not trivial in this
code, which is highly dependent on statically allocated data structures
containing pointers to functions and other global objects). It is also
needed to remove the dependency on kernel infrastructure for command
line parsing etc.

The resulting code can execute from the initial ID map, and populate the
feature override variables that have been moved into BSS (which is the
only part of the kernel image that is mapped writable in the initial ID
map)

Also, a new C implementation is provided to create the permanent kernel
mapping without the need to pivot through a temporary one based on block
mappings. This permanent mapping is created with all the necessary
attributes that support the optional features detected (and potentially
overridden) beforehand.

This means the kernel mapping code in mm/mmu.c can be retired, along
with the handling of the switch between the two and the copying of Kasan
shadow mappings and fixmap intermediate page table levels.

WXN
===
The final two patches cover the remaining changes that are needed to
enable WXN support, which is desirable for robustness, given that
writable, executable mappings of memory are too easy to subvert, and in
the kernel, we rarely rely on such mappings anyway.

Setting SCTLR_ELx.WXN makes all writable mappings implicitly
non-executable, and when set at EL1, it affects EL0 as well as EL1. This
means we need some diligence on the part of user space, but fortunately,
most JITs and other user space components that actually need to
manipulate the contents of their own executable code use split views on
the same memory, or switch between RW- and R-X and back. (One notable
exception is V8, which recently switched back to a full RWX mappings
based JIT)

So on the user space side, we need a couple of minor tweaks to validate
the mmap()/mprotect() arguments when WXN is in effect, and to handle any
faults that might occur on such mappings.

Changes since v6:
- radical overhaul of the boot code to move the feature override
  detection before the permanent mapping of the kernel is created
- rebase onto arm64/dynamic-scs and incorporate the initial dynamic SCS
  patching into the early C map_kernel() routine as well
- remove some false dependences on the SWAPPER_BLOCK_xxx constants, and
  rename them now that they no longer apply to swapper_pg_dir, but
  only to the initial ID map

(v5 was a subset of v4 without the WXN specific pieces)

Changes since v4: [0]
- don't move __ro_after_init section now that we no longer need to,
- don't complicate the asm kernel mapping routines further, but instead,
  merge the two existing passes into one implemented in C,
- deal with rodata=off on WXN enabled builds (i.e., turn off WXN as
  well),
- add some acks from Kees

Cc: Marc Zyngier <maz at kernel.org>
Cc: Will Deacon <will at kernel.org>
Cc: Mark Rutland <mark.rutland at arm.com>
Cc: Kees Cook <keescook at chromium.org>
Cc: Catalin Marinas <catalin.marinas at arm.com>
Cc: Mark Brown <broonie at kernel.org>
Cc: Anshuman Khandual <anshuman.khandual at arm.com>

Ard Biesheuvel (32):
  arm64: mm: Avoid SWAPPER_BLOCK_xxx constants in FDT fixmap logic
  arm64: mm: Avoid swapper block size when choosing vmemmap granularity
  arm64: kaslr: don't pretend KASLR is enabled if offset <
    MIN_KIMG_ALIGN
  arm64: kaslr: drop special case for ThunderX in kaslr_requires_kpti()
  arm64: kernel: Disable latent_entropy GCC plugin in early C runtime
  arm64: kernel: Add relocation check to code built under pi/
  arm64: kernel: Don't rely on objcopy to make code under pi/ __init
  arm64: head: move relocation handling to C code
  arm64: idreg-override: Omit non-NULL checks for override pointer
  arm64: idreg-override: Use relative references to override variables
  arm64: idreg-override: Use relative references to filter routines
  arm64: idreg-override: Avoid parameq() and parameqn()
  arm64: idreg-override: avoid strlen() to check for empty strings
  arm64: idreg-override: Avoid sprintf() for simple string concatenation
  arm64: idreg_override: Avoid kstrtou64() to parse a single hex digit
  arm64: idreg-override: Move to early mini C runtime
  arm64: kernel: Remove early fdt remap code
  arm64: head: Clear BSS and the kernel page tables in one go
  arm64: Move feature overrides into the BSS section
  arm64: head: Run feature override detection before mapping the kernel
  arm64: head: move dynamic shadow call stack patching into early C
    runtime
  arm64: kaslr: Use feature override instead of parsing the cmdline
    again
  arm64: idreg-override: Create a pseudo feature for rodata=off
  arm64: head: allocate more pages for the kernel mapping
  arm64: head: move memstart_offset_seed handling to C code
  arm64: head: Move early kernel mapping routines into C code
  arm64: mm: avoid fixmap for early swapper_pg_dir updates
  arm64: mm: omit redundant remap of kernel image
  arm64: Revert "mm: provide idmap pointer to cpu_replace_ttbr1()"
  arm64: mmu: Retire SWAPPER_BLOCK_xxx and related constants
  mm: add arch hook to validate mmap() prot flags
  arm64: mm: add support for WXN memory translation attribute

Marc Zyngier (1):
  arm64: Turn kaslr_feature_override into a generic SW feature override

 arch/arm64/Kconfig                      |  11 +
 arch/arm64/include/asm/cpufeature.h     |  15 +
 arch/arm64/include/asm/fixmap.h         |   5 +-
 arch/arm64/include/asm/kasan.h          |   2 -
 arch/arm64/include/asm/kernel-pgtable.h |  69 ++--
 arch/arm64/include/asm/memory.h         |  11 +
 arch/arm64/include/asm/mman.h           |  36 ++
 arch/arm64/include/asm/mmu.h            |   2 +-
 arch/arm64/include/asm/mmu_context.h    |  43 ++-
 arch/arm64/include/asm/scs.h            |  34 +-
 arch/arm64/include/asm/setup.h          |   3 -
 arch/arm64/kernel/Makefile              |   7 +-
 arch/arm64/kernel/cpufeature.c          |  32 +-
 arch/arm64/kernel/head.S                | 208 ++----------
 arch/arm64/kernel/idreg-override.c      | 321 ------------------
 arch/arm64/kernel/image-vars.h          |  28 ++
 arch/arm64/kernel/kaslr.c               |  10 +-
 arch/arm64/kernel/module.c              |   2 +-
 arch/arm64/kernel/pi/Makefile           |  24 +-
 arch/arm64/kernel/pi/idreg-override.c   | 356 ++++++++++++++++++++
 arch/arm64/kernel/pi/kaslr_early.c      |  70 +---
 arch/arm64/kernel/pi/map_kernel.c       | 306 +++++++++++++++++
 arch/arm64/kernel/{ => pi}/patch-scs.c  |  34 +-
 arch/arm64/kernel/pi/pi.h               |  12 +
 arch/arm64/kernel/pi/relacheck.c        | 104 ++++++
 arch/arm64/kernel/pi/relocate.c         |  63 ++++
 arch/arm64/kernel/setup.c               |  22 --
 arch/arm64/kernel/suspend.c             |   2 +-
 arch/arm64/kernel/vmlinux.lds.S         |  16 +-
 arch/arm64/mm/kasan_init.c              |  19 +-
 arch/arm64/mm/mmu.c                     | 160 +++------
 arch/arm64/mm/proc.S                    |   9 +-
 include/linux/mman.h                    |  15 +
 mm/mmap.c                               |   3 +
 34 files changed, 1188 insertions(+), 866 deletions(-)
 delete mode 100644 arch/arm64/kernel/idreg-override.c
 create mode 100644 arch/arm64/kernel/pi/idreg-override.c
 create mode 100644 arch/arm64/kernel/pi/map_kernel.c
 rename arch/arm64/kernel/{ => pi}/patch-scs.c (90%)
 create mode 100644 arch/arm64/kernel/pi/pi.h
 create mode 100644 arch/arm64/kernel/pi/relacheck.c
 create mode 100644 arch/arm64/kernel/pi/relocate.c

-- 
2.35.1