[PATCH v2 09/28] kbuild: add support for Clang LTO

[Sami Tolvanen]

On Tue, Sep 08, 2020 at 12:30:14AM +0900, Masahiro Yamada wrote:
> On Fri, Sep 4, 2020 at 5:31 AM Sami Tolvanen <samitolvanen at google.com> wrote:
> >
> > This change adds build system support for Clang's Link Time
> > Optimization (LTO). With -flto, instead of ELF object files, Clang
> > produces LLVM bitcode, which is compiled into native code at link
> > time, allowing the final binary to be optimized globally. For more
> > details, see:
> >
> >   https://llvm.org/docs/LinkTimeOptimization.html
> >
> > The Kconfig option CONFIG_LTO_CLANG is implemented as a choice,
> > which defaults to LTO being disabled. To use LTO, the architecture
> > must select ARCH_SUPPORTS_LTO_CLANG and support:
> >
> >   - compiling with Clang,
> >   - compiling inline assembly with Clang's integrated assembler,
> >   - and linking with LLD.
> >
> > While using full LTO results in the best runtime performance, the
> > compilation is not scalable in time or memory. CONFIG_THINLTO
> > enables ThinLTO, which allows parallel optimization and faster
> > incremental builds. ThinLTO is used by default if the architecture
> > also selects ARCH_SUPPORTS_THINLTO:
> >
> >   https://clang.llvm.org/docs/ThinLTO.html
> >
> > To enable LTO, LLVM tools must be used to handle bitcode files. The
> > easiest way is to pass the LLVM=1 option to make:
> >
> >   $ make LLVM=1 defconfig
> >   $ scripts/config -e LTO_CLANG
> >   $ make LLVM=1
> >
> > Alternatively, at least the following LLVM tools must be used:
> >
> >   CC=clang LD=ld.lld AR=llvm-ar NM=llvm-nm
> >
> > To prepare for LTO support with other compilers, common parts are
> > gated behind the CONFIG_LTO option, and LTO can be disabled for
> > specific files by filtering out CC_FLAGS_LTO.
> >
> > Note that support for DYNAMIC_FTRACE and MODVERSIONS are added in
> > follow-up patches.
> >
> > Signed-off-by: Sami Tolvanen <samitolvanen at google.com>
> > ---
> >  Makefile                          | 18 +++++++-
> >  arch/Kconfig                      | 68 +++++++++++++++++++++++++++++++
> >  include/asm-generic/vmlinux.lds.h | 11 +++--
> >  scripts/Makefile.build            |  9 +++-
> >  scripts/Makefile.modfinal         |  9 +++-
> >  scripts/Makefile.modpost          | 24 ++++++++++-
> >  scripts/link-vmlinux.sh           | 32 +++++++++++----
> >  7 files changed, 154 insertions(+), 17 deletions(-)
> 
> 
> 
> >  #define TEXT_MAIN .text
> > diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> > index 6ecf30c70ced..a5f4b5d407e6 100644
> > --- a/scripts/Makefile.build
> > +++ b/scripts/Makefile.build
> > @@ -111,7 +111,7 @@ endif
> >  # ---------------------------------------------------------------------------
> >
> >  quiet_cmd_cc_s_c = CC $(quiet_modtag)  $@
> > -      cmd_cc_s_c = $(CC) $(filter-out $(DEBUG_CFLAGS), $(c_flags)) $(DISABLE_LTO) -fverbose-asm -S -o $@ $<
> > +      cmd_cc_s_c = $(CC) $(filter-out $(DEBUG_CFLAGS) $(CC_FLAGS_LTO), $(c_flags)) -fverbose-asm -S -o $@ $<
> >
> >  $(obj)/%.s: $(src)/%.c FORCE
> >         $(call if_changed_dep,cc_s_c)
> > @@ -428,8 +428,15 @@ $(obj)/lib.a: $(lib-y) FORCE
> >  # Do not replace $(filter %.o,^) with $(real-prereqs). When a single object
> >  # module is turned into a multi object module, $^ will contain header file
> >  # dependencies recorded in the .*.cmd file.
> > +ifdef CONFIG_LTO_CLANG
> > +quiet_cmd_link_multi-m = AR [M]  $@
> > +cmd_link_multi-m =                                             \
> > +       rm -f $@;                                               \
> > +       $(AR) rcsTP$(KBUILD_ARFLAGS) $@ $(filter %.o,$^)
> 
> 
> KBUILD_ARFLAGS no longer exists in the mainline.
> (commit 13dc8c029cabf52ba95f60c56eb104d4d95d5889)

Thanks, I'll drop this in the next version.

> > +ifdef CONFIG_LTO_CLANG
> > +# With CONFIG_LTO_CLANG, .o files might be LLVM bitcode,
> 
> or, .o files might be even thin archives.

Right, and with LTO the thin archive might also point to a mix of bitcode
and ELF to further complicate things.

> For example,
> 
> $ file net/ipv6/netfilter/nf_defrag_ipv6.o
> net/ipv6/netfilter/nf_defrag_ipv6.o: thin archive with 6 symbol entries
> 
> 
> Now we have 3 possibilities for .o files:
> 
>   - ELF  (real .o)
>   - LLVM bitcode (.bc)
>   - Thin archive (.a)
> 
> 
> Let me discuss how to proceed with this...

Did you have something in mind to make this cleaner?

Sami