[PATCH 15/56] KVM: arm64: Add build rules for separate VHE/nVHE object files

[Auger Eric]

Hi Marc,

On 5/5/21 8:03 PM, Marc Zyngier wrote:
> Hi Eric,
> 
> On Tue, 04 May 2021 15:47:36 +0100,
> Auger Eric <eric.auger at redhat.com> wrote:
>>
>> Hi David, Marc,
>>
>> On 8/5/20 7:56 PM, Marc Zyngier wrote:
>>> From: David Brazdil <dbrazdil at google.com>
>>>
>>> Add new folders arch/arm64/kvm/hyp/{vhe,nvhe} and Makefiles for building code
>>> that runs in EL2 under VHE/nVHE KVM, repsectivelly. Add an include folder for
>>> hyp-specific header files which will include code common to VHE/nVHE.
>>>
>>> Build nVHE code with -D__KVM_NVHE_HYPERVISOR__, VHE code with
>>> -D__KVM_VHE_HYPERVISOR__.
>>>
>>> Under nVHE compile each source file into a `.hyp.tmp.o` object first, then
>>> prefix all its symbols with "__kvm_nvhe_" using `objcopy` and produce
>>> a `.hyp.o`. Suffixes were chosen so that it would be possible for VHE and nVHE
>>> to share some source files, but compiled with different CFLAGS.
>>>
>>> The nVHE ELF symbol prefix is added to kallsyms.c as ignored. EL2-only symbols
>>> will never appear in EL1 stack traces.
>>>
>>> Due to symbol prefixing, add a section in image-vars.h for aliases of symbols
>>> that are defined in nVHE EL2 and accessed by kernel in EL1 or vice versa.
>>>
>>> Signed-off-by: David Brazdil <dbrazdil at google.com>
>>> Signed-off-by: Marc Zyngier <maz at kernel.org>
>>> Link: https://lore.kernel.org/r/20200625131420.71444-4-dbrazdil@google.com
>>> ---
>>>  arch/arm64/kernel/image-vars.h   | 14 +++++++++++++
>>>  arch/arm64/kvm/hyp/Makefile      | 10 +++++++---
>>>  arch/arm64/kvm/hyp/nvhe/Makefile | 34 ++++++++++++++++++++++++++++++++
>>>  arch/arm64/kvm/hyp/vhe/Makefile  | 17 ++++++++++++++++
>>>  scripts/kallsyms.c               |  1 +
>>>  5 files changed, 73 insertions(+), 3 deletions(-)
>>>  create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefile
>>>  create mode 100644 arch/arm64/kvm/hyp/vhe/Makefile
> 
> [...]
> 
>>> diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
>>> index 6dc3078649fa..0096cd965332 100644
>>> --- a/scripts/kallsyms.c
>>> +++ b/scripts/kallsyms.c
>>> @@ -109,6 +109,7 @@ static bool is_ignored_symbol(const char *name, char type)
>>>  		".LASANPC",		/* s390 kasan local symbols */
>>>  		"__crc_",		/* modversions */
>>>  		"__efistub_",		/* arm64 EFI stub namespace */
>>> +		"__kvm_nvhe_",		/* arm64 non-VHE KVM namespace */
>> The addition of this line seems to have introduced errors on the
>> 'vmlinux symtab matches kallsyms' perf test (perf test -v 1) which fails
>> on aarch64 for all __kvm_nvhe_ prefixed symbols, like
>>
>> ERR : <addr> : __kvm_nvhe___invalid not on kallsyms
>> ERR : <addr> : __kvm_nvhe___do_hyp_init not on kallsyms
>> ERR : <addr> : __kvm_nvhe___kvm_handle_stub_hvc not on kallsyms
>> ERR : <addr> : __kvm_nvhe_reset not on kallsyms
>> ../..
>>
>> I understand we willingly hided those symbols from /proc/kallsyms. Do
>> you confirm the right fix is to upgrade the perf test suite accordingly?
> 
> Hmmm. This test always fail here, no matter whether I have this line
> or not:

sorry Marc I missed your reply,
> 
> <quote>
> maz at big-leg-emma:~$ sudo perf_5.10 test -v 1
>  1: vmlinux symtab matches kallsyms                                 :
> --- start ---
> test child forked, pid 664
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> Looking at the vmlinux_path (8 entries long)
> symsrc__init: cannot get elf header.
> symsrc__init: cannot get elf header.
> Couldn't find a vmlinux that matches the kernel running on this machine, skipping test
> test child finished with -2
> ---- end ----
> vmlinux symtab matches kallsyms: Skip
> </quote>
> 
> Rookie question: How do you provide a kernel to the test framework?
On my end I compiled and installed a kernel and compiled perf from that
same kernel. Should work. I don't know how to point to a specific vmlinux

Thanks

Eric
> 
> Thanks,
> 
> 	M.
>