[PATCH 8/9] x86/kexec: Cope with relocate_kernel() not being at the start of the page

[Ard Biesheuvel]

On Tue, 17 Dec 2024 at 00:37, David Woodhouse <dwmw2 at infradead.org> wrote:
>
> From: David Woodhouse <dwmw at amazon.co.uk>
>
> A few places in the kexec control code page make the assumption that the
> first instruction of relocate_kernel is at the very start of the page.
>
> To allow for Clang CFI information to be added to relocate_kernel(), as
> well as the general principle of removing unwarranted assumptions, fix
> them to use the external __relocate_kernel_start symbol that the linker
> adds. This means using a separate addq and subq for calculating offsets,
> as the assembler can no longer calculate the delta directly for itself
> and relocations aren't that versatile.
>

You can still avoid the absolute relocations though, ...

> Turn the jump from relocate_kernel() to identity_mapped() into a real
> indirect 'jmp *%rsi' too, while touching it. There was no real reason
> for it to be a push+ret in the first place, and adding Clang CFI info
> will also give objtool enough visibility to start complaining 'return
> with modified stack frame' about it.
>
> Signed-off-by: David Woodhouse <dwmw at amazon.co.uk>
> ---
>  arch/x86/kernel/relocate_kernel_64.S | 18 +++++++++++-------
>  1 file changed, 11 insertions(+), 7 deletions(-)
>
> diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocate_kernel_64.S
> index 1996cea909ff..d74798d78263 100644
> --- a/arch/x86/kernel/relocate_kernel_64.S
> +++ b/arch/x86/kernel/relocate_kernel_64.S
> @@ -95,11 +95,10 @@ SYM_CODE_START_NOALIGN(relocate_kernel)
>         lea     PAGE_SIZE(%rsi), %rsp
>
>         /* jump to identity mapped page */
> -       addq    $(identity_mapped - relocate_kernel), %rsi
> -       pushq   %rsi
> -       ANNOTATE_UNRET_SAFE
> -       ret
> -       int3
> +       addq    $identity_mapped, %rsi
> +       subq    $__relocate_kernel_start, %rsi

... if you turn this into

0:     addq    $identity_mapped - 0b, %rsi
       subq    $__relocate_kernel_start - 0b, %rsi


> +       ANNOTATE_RETPOLINE_SAFE
> +       jmp     *%rsi
>  SYM_CODE_END(relocate_kernel)
>
>  SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
> @@ -219,16 +218,21 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
>
>         /* get the re-entry point of the peer system */
>         popq    %rbp
> -       leaq    relocate_kernel(%rip), %r8
>         movq    kexec_pa_swap_page(%rip), %r10
>         movq    pa_backup_pages_map(%rip), %rdi
>         movq    kexec_pa_table_page(%rip), %rax
>         movq    %rax, %cr3
> +
> +       /* Find start (and end) of this physical mapping of control page */
> +       leaq    (%rip), %r8
> +       ANNOTATE_NOENDBR
> +       andq    $PAGE_MASK, %r8
>         lea     PAGE_SIZE(%r8), %rsp
>         movq    $1, %r11        /* Ensure preserve_context flag is set */
>         call    swap_pages
>         movq    kexec_va_control_page(%rip), %rax
> -       addq    $(virtual_mapped - relocate_kernel), %rax
> +       addq    $virtual_mapped, %rax
> +       subq    $__relocate_kernel_start, %rax
>         pushq   %rax
>         ANNOTATE_UNRET_SAFE
>         ret
> --
> 2.47.0
>