[PATCH v4 00/20] arm64: entry: migrate more code to C

[Mark Rutland]

This series (based on v5.13-rc2) migrates most of the remaining
exception triage assembly to C. All the exception vectors are given C
handlers, so that we can defer all decision making to C code, and the
assembly code can be made simpler and more uniform. At the same time,
I've tried to consolidate all the entry sequencing (e.g. reading
exception registers and calling accounting code) in entry-common.c so
that this is easier to maintain.

I was recently informed that `noinstr` wasn't protecting entry sequences
from KCOV instrumentation, so I've refactored things so that we can
avoid this by preventing KCOV instrumentation for the entirety of
entry-common.c. I've done likewise for the low-level idle sequences
which have the same problems with instrumentation when RCU isn't
watching, etc.

I've stopped short of converting the ret_to_user / work_pending loop.
Converting this cleanly will probably need something like the wrappers
generated by SYSCALL_DEFINE() to handle the common entry/exit logic, and
this is easier to build once all the handlers have been converted to C.
Similar is true for portions of kernel_entry and kernel_exit that could
be converted to C.

It should also be possible to generate the vectors and their associated
assembly handlers in one go by placing these in separate sections and
using .pushsection and .popsection. I've held off doing this for now as
this probably requires some changes to the linker script, and regardless
it should be easier to make that change atop this series.

So far this has seen some light boot testing, and a day's worth of fuzzing
under Syzkaller, which I intend to leave to soak for a while.

I've pushed the series to my arm64/entry/rework branch on kernel.org:

  https://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git/log/?h=arm64/entry/rework
  git://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git arm64/entry/rework

I've tagged this version as arm64-entry-rework-20210525.

Since v1 [1]
* Rebase to v5.13-rc2
* Fold NMI entry/exit sequencing into entry-common.c
* Make NMI entry/exit helpers private to entry-comomn.c
* Prevent KCOV instrumentation of entry-common.c
* Prevent KCOV instrumentation of idle code

Since v2 [2]:
* Correct commit message description in patch 2
* Use `el0t` prefix for EL0 handlers for consistency with `el1h` and `el1t`
* Remove `user_exit_irqoff` macro
* Remove leftover bad_mode() prototype
* Add patch to replace el1_inv() with a direct call to __panic_unhandled()
* Accumulate Reviewed-by tags
* Typo fixes

Since v3 [3]:
* Fix preemption check logic to check thread_info::{count,need_resched}
* Clean up alignment of assembly handlers using SYM_CODE_START_LOCAL()
* Make `ht` a required asm macro argument in `kernel_ventry` and `entry_handler`
* Add comment for KCOV opt-out
* Typo and grammar fixes
* Accumulate tags

[1] https://lore.kernel.org/r/20210510155621.52811-1-mark.rutland@arm.com
[2] https://lore.kernel.org/r/20210519123902.2452-1-mark.rutland@arm.com
[3] https://lore.kernel.org/r/20210525183302.56293-1-mark.rutland@arm.com

Thanks,
Mark.

Mark Rutland (20):
  arm64: remove redundant local_daif_mask() in bad_mode()
  arm64: entry: unmask IRQ+FIQ after EL0 handling
  arm64: entry: convert SError handlers to C
  arm64: entry: move arm64_preempt_schedule_irq to entry-common.c
  arm64: entry: move NMI preempt logic to C
  arm64: entry: add a call_on_irq_stack helper
  arm64: entry: convert IRQ+FIQ handlers to C
  arm64: entry: organise entry handlers consistently
  arm64: entry: organise entry vectors consistently
  arm64: entry: consolidate EL1 exception returns
  arm64: entry: move bad_mode() to entry-common.c
  arm64: entry: improve bad_mode()
  arm64: entry: template the entry asm functions
  arm64: entry: handle all vectors with C
  arm64: entry: fold el1_inv() into el1h_64_sync_handler()
  arm64: entry: split bad stack entry
  arm64: entry: split SDEI entry
  arm64: entry: make NMI entry/exit functions static
  arm64: entry: don't instrument entry code with KCOV
  arm64: idle: don't instrument idle code with KCOV

 arch/arm64/include/asm/exception.h |  34 +++-
 arch/arm64/include/asm/processor.h |   2 -
 arch/arm64/include/asm/sdei.h      |   3 +
 arch/arm64/kernel/Makefile         |   8 +-
 arch/arm64/kernel/entry-common.c   | 256 ++++++++++++++++++++++++--
 arch/arm64/kernel/entry.S          | 359 ++++++++++---------------------------
 arch/arm64/kernel/idle.c           |  69 +++++++
 arch/arm64/kernel/process.c        |  74 --------
 arch/arm64/kernel/sdei.c           |  48 +----
 arch/arm64/kernel/traps.c          |  40 +----
 arch/arm64/mm/fault.c              |   7 -
 11 files changed, 440 insertions(+), 460 deletions(-)
 create mode 100644 arch/arm64/kernel/idle.c

-- 
2.11.0