[PATCH] KVM: arm64: Disable LTO in hyp

[Marc Zyngier]

On Thu, 04 Mar 2021 21:25:41 +0000,
Sami Tolvanen <samitolvanen at google.com> wrote:
> 
> On Thu, Mar 4, 2021 at 11:15 AM Marc Zyngier <maz at kernel.org> wrote:
> >
> > On Thu, 04 Mar 2021 18:45:44 +0000,
> > Sami Tolvanen <samitolvanen at google.com> wrote:
> > >
> > > allmodconfig + CONFIG_LTO_CLANG_THIN=y fails to build due to following
> > > linker errors:
> > >
> > >   ld.lld: error: irqbypass.c:(function __guest_enter: .text+0x21CC):
> >
> > I assume this message is only an oddity, right? Because
> > __guest_enter() is as far as you can imagine from irqbypass.c...
> 
> I'm not sure what's up with the filename in the error message. Fangrui
> or Nick probably have a better idea.
> 
> > >   relocation R_AARCH64_CONDBR19 out of range: 2031220 is not in
> > >   [-1048576, 1048575]; references hyp_panic
> > >   >>> defined in vmlinux.o
> > >
> > >   ld.lld: error: irqbypass.c:(function __guest_enter: .text+0x21E0):
> > >   relocation R_AARCH64_ADR_PREL_LO21 out of range: 2031200 is not in
> > >   [-1048576, 1048575]; references hyp_panic
> > >   >>> defined in vmlinux.o
> > >
> > > As LTO is not really necessary for the hypervisor code, disable it for
> > > the hyp directory to fix the build.
> >
> > Can you shed some light on what the problem is exactly?
> 
> I assume hyp_panic() ends up being placed too far from __guest_enter()
> when the kernel is large enough. Possibly something to do with LLVM
> always splitting functions into separate sections with LTO. I'm not
> sure why the linker cannot shuffle things around to make everyone
> happy in this case, but I confirmed that this patch also fixes the
> build issue for me:
> 
> diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c
> index af8e940d0f03..128197b7c794 100644
> --- a/arch/arm64/kvm/hyp/vhe/switch.c
> +++ b/arch/arm64/kvm/hyp/vhe/switch.c
> @@ -214,7 +214,7 @@ static void __hyp_call_panic(u64 spsr, u64 elr, u64 par)
>  }
>  NOKPROBE_SYMBOL(__hyp_call_panic);
> 
> -void __noreturn hyp_panic(void)
> +void __noreturn hyp_panic(void) __section(".text")
>  {
>         u64 spsr = read_sysreg_el2(SYS_SPSR);
>         u64 elr = read_sysreg_el2(SYS_ELR);
>

We're getting into black-magic territory here. Why wouldn't hyp_panic
be in the .text section already?

> > >
> > > Link: https://github.com/ClangBuiltLinux/linux/issues/1317
> > > Reported-by: Nathan Chancellor <nathan at kernel.org>
> > > Tested-by: Nathan Chancellor <nathan at kernel.org>
> > > Signed-off-by: Sami Tolvanen <samitolvanen at google.com>
> > > ---
> > >  arch/arm64/kvm/hyp/Makefile | 3 +++
> > >  1 file changed, 3 insertions(+)
> > >
> > > diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> > > index 687598e41b21..e8116016e6a8 100644
> > > --- a/arch/arm64/kvm/hyp/Makefile
> > > +++ b/arch/arm64/kvm/hyp/Makefile
> > > @@ -11,3 +11,6 @@ subdir-ccflags-y := -I$(incdir)                             \
> > >                   $(DISABLE_STACKLEAK_PLUGIN)
> > >
> > >  obj-$(CONFIG_KVM) += vhe/ nvhe/ pgtable.o
> > > +
> > > +# Disable LTO for the files in this directory
> > > +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
> > >
> > > base-commit: f69d02e37a85645aa90d18cacfff36dba370f797
> >
> > Can this be reduced to the nvhe part of the tree? The rest of the
> > hypervisor should support being built with LTO, I'd expect. Or am I
> > missing something more significant?
> 
> No, this error appears to be about hyp_panic() in the vhe code. While
> I'm not sure how beneficial LTO is in hypervisor code, there shouldn't
> be any other reason we can't use it there.

The VHE part of the hypervisor is the kernel itself, and not any
different from the rest of the code. It should be able to benefit from
LTO. On the contrary, the nVHE part is what could do without LTO,
given that it is essentially a separate object that happens to be
bundled with the kernel.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.