Linux coresight arm64 : Incorrect data in cstrace.bin
James Clark
james.clark at arm.com
Fri Jul 7 08:50:59 PDT 2023
On 07/07/2023 09:49, James Clark wrote:
>
>
> On 06/07/2023 13:47, Pandey, Radhey Shyam wrote:
>>> -----Original Message-----
>>> From: Pandey, Radhey Shyam
>>> Sent: Thursday, July 6, 2023 5:40 PM
>>> To: James Clark <james.clark at arm.com>
>>> Cc: coresight at lists.linaro.org; linux-arm-kernel at lists.infradead.org; Suzuki K
>>> Poulose <suzuki.poulose at arm.com>; mathieu.poirier at linaro.org;
>>> mike.leach at linaro.org; leo.yan at linaro.org;
>>> alexander.shishkin at linux.intel.com; Sarangi, Anirudha
>>> <anirudha.sarangi at amd.com>
>>> Subject: RE: Linux coresight arm64 : Incorrect data in cstrace.bin
>>>
>>>> -----Original Message-----
>>>> From: James Clark <james.clark at arm.com>
>>>> Sent: Wednesday, July 5, 2023 5:44 PM
>>>> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
>>>> Cc: coresight at lists.linaro.org; linux-arm-kernel at lists.infradead.org;
>>>> Suzuki K Poulose <suzuki.poulose at arm.com>; mathieu.poirier at linaro.org;
>>>> mike.leach at linaro.org; leo.yan at linaro.org;
>>>> alexander.shishkin at linux.intel.com; Sarangi, Anirudha
>>>> <anirudha.sarangi at amd.com>
>>>> Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
>>>>
>>>>
>>>>
>>>> On 05/07/2023 12:48, Pandey, Radhey Shyam wrote:
>>>>>> -----Original Message-----
>>>>>> From: James Clark <james.clark at arm.com>
>>>>>> Sent: Wednesday, July 5, 2023 4:29 PM
>>>>>> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
>>>>>> Cc: coresight at lists.linaro.org;
>>>>>> linux-arm-kernel at lists.infradead.org;
>>>>>> Suzuki K Poulose <suzuki.poulose at arm.com>;
>>>>>> mathieu.poirier at linaro.org; mike.leach at linaro.org;
>>>>>> leo.yan at linaro.org; alexander.shishkin at linux.intel.com
>>>>>> Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 05/07/2023 10:56, Pandey, Radhey Shyam wrote:
>>>>>>>> -----Original Message-----
>>>>>>>> From: Suzuki K Poulose <suzuki.poulose at arm.com>
>>>>>>>> Sent: Wednesday, July 5, 2023 1:57 PM
>>>>>>>> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>;
>>>>>>>> mathieu.poirier at linaro.org; mike.leach at linaro.org;
>>>>>>>> leo.yan at linaro.org; alexander.shishkin at linux.intel.com
>>>>>>>> Cc: coresight at lists.linaro.org;
>>>>>>>> linux-arm-kernel at lists.infradead.org
>>>>>>>> Subject: Re: Linux coresight arm64 : Incorrect data in
>>>>>>>> cstrace.bin
>>>>>>>>
>>>>>>>> Hi Radhe Shyam
>>>>>>>>
>>>>>>>> On 05/07/2023 05:38, Pandey, Radhey Shyam wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I am using linux 6.1 kernel coresight framework to capture ETM
>>> trace.
>>>>>>>>> Enabled coresight driver and added coresight component device
>>>>>>>>> node in DTS. With that, we could probe all coresight components.
>>>>>>>>>
>>>>>>>>> But when we do capture using sysfs and then read the trace.bin
>>>>>>>>> using ptm2human we see invalid trace data. Trace content
>>>>>>>>> changes on each capture. Any pointers to help narrow down the
>>> issue?
>>>>>>>>>
>>>>>>>>> Tried: sysfs capture and perf (with OpenCSD enabled)
>>>>>>>>>
>>>>>>>>> Development board: VCK190 :
>>>>>>>>> https://docs.xilinx.com/r/en-US/ug1366-vck190-eval-bd
>>>>>>>>> https://www.xilinx.com/support/documents/architecture-
>>>>>>>> manuals/am011-ve
>>>>>>>>> rsal-acap-trm.pdf
>>>>>>>>>
>>>>>>>>> xilinx-vck190-20231:/home/petalinux# dmesg | grep -i amba
>>>>>>>>> [ 0.301054] Serial: AMBA PL011 UART driver
>>>>>>>>> [ 0.306226] amba f0c20000.funnel: Fixing up cyclic dependency
>>> with
>>>>>>>> f0d70000.etm
>>>>>>>>> [ 0.313624] amba f0c20000.funnel: Fixing up cyclic dependency
>>> with
>>>>>>>> f0d30000.etm
>>>>>>>>> [ 0.321110] amba f0c30000.etf: Fixing up cyclic dependency with
>>>>>>>> f0c20000.funnel
>>>>>>>>> [ 1.470247] Serial: AMBA driver
>>>>>>>>> xilinx-vck190-20231:/home/petalinux# dmesg | grep -i coresight
>>>>>>>>> [ 1.865006] cs_system_cfg: CoreSight Configuration manager
>>>> initialised
>>>>>>>>> [ 1.876858] coresight etm0: CPU0: etm v4.0 initialized
>>>>>>>>> [ 1.885666] coresight etm1: CPU1: etm v4.0 initialized
>>>>>>>>> [ 1.894435] coresight-cpu-debug f0d00000.debug1: Coresight
>>> debug-
>>>>>> CPU0
>>>>>>>> initialized
>>>>>>>>> [ 1.902072] coresight-cpu-debug f0d40000.debug1: Coresight
>>> debug-
>>>>>> CPU1
>>>>>>>> initialized
>>>>>>>>>
>>>>>>>>> xilinx-vck190-20231:/home/petalinux# ls
>>> /sys/bus/coresight/devices/
>>>>>>>>> etm0 etm1 funnel0 tmc_etf0
>>>>>>>>>
>>>>>>>>> cd /sys/bus/coresight/devices/ echo 1 > tmc_etf0/enable_sink
>>>>>>>>> echo 1 > etm0/enable_source echo 0 > etm0/enable_source echo 0 >
>>>>>>>>> tmc_etf0/enable_sink cd /root/ dd if=/dev/tmc_etf0
>>>>>>>>> of=cstrace_28Jun.bin
>>>>>>>>>
>>>>>>>>> ./ptm2human/ptm2human -e -i cstrace.bin -d
>>>>>>>>
>>>>>>>> Please note that ptm2human is for PTM trace decoding and
>>>>>>>> ETMv4 uses a different format and thus is not compatible.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I also tried OpenCSD integration with PERF.
>>>>>>>>
>>>>>>>> Have you made sure the perf is "linked" to the opencsd ?
>>>>>>>
>>>>>>> Earlier I statically linked but after adding CORESIGHT=1.
>>>>>>>
>>>>>>>
>>>>>>> linux-xlnx$ make ARCH=arm64 NO_LIBELF=1 NO_JVMTI=1 VF=1
>>>>>> CORESIGHT=1 -C
>>>>>>> tools/perf/
>>>>>>> make: Entering directory 'linux-xlnx/tools/perf'
>>>>>>> BUILD: Doing 'make -j24' parallel build
>>>>>>>
>>>>>>> Makefile.config:520: *** Error: No libopencsd library found or the
>>>>>>> version
>>>>>> is not up-to-date.
>>>>>>> Please install recent libopencsd to build with CORESIGHT=1. Stop.
>>>>>>>
>>>>>>>
>>>>>>> echo $CSINCLUDES
>>>>>>> /scratch/development/coresight/my-opencsd/decoder/include
>>>>>>> radheys at xhdradheys41:/scratch/development/linux-xlnx$ ls
>>>>>>> /scratch/development/coresight/my-opencsd/decoder/include
>>>>>>> common i_dec interfaces mem_acc opencsd opencsd.h
>>>>>>> pkt_printers
>>>>>>>
>>>>>>> echo $CSLIBS
>>>>>>> /scratch/development/coresight/my-opencsd/decoder/lib/builddir
>>>>>>> radheys at xhdradheys41:/scratch/development/linux-xlnx$ ls
>>>>>> /scratch/development/coresight/my-opencsd/decoder/lib/builddir
>>>>>>> libopencsd.a libopencsd_c_api.so libopencsd_c_api.so.1.4.0
>>>>>> libopencsd.so.1
>>>>>>> libopencsd_c_api.a libopencsd_c_api.so.1 libopencsd.so
>>>>>> libopencsd.so.1.4.0
>>>>>>>
>>>>>>> Anything I am missing to fix this opencsd lib not found?
>>>>>>
>>>>>> There is a minimum version of 1.1.1 but it looks like you are using
>>>>>> higher than that so that part should be fine.
>>>>>>
>>>>>> Personally I "make install" OpenCSD to the system path. I had a
>>>>>> play around with $CSLIBS it and maybe there is also some stickyness
>>>>>> to the feature detection so a make clean might help to make sure
>>>>>> the errors you are seeing are real.
>>>>>>
>>>>>> Lastly you can print the output of why the feature test compilation
>>>>>> failed which might help. I see some linking issues in there if I
>>>>>> delete my system installed version of OpenCSD, so there might be a
>>>>>> bug with just using CSINCLUDES and CSLIBS:
>>>>>>
>>>>>> $ cat tools/build/feature/test-libopencsd.make.output
>>>>>>
>>>>>> /usr/bin/ld: warning: libopencsd.so.1, needed by opencsd-
>>>>>> local/decoder/lib/builddir//libopencsd_c_api.so, not found (try
>>>>>> using -rpath or -rpath-link)
>>>>>> /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so:
>>>>>> undefined reference to `vtable for StmTrcPacket'
>>>>>> /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so:
>>>>>> undefined reference to `DecodeTree::getDecoderStats(unsigned char,
>>>>>> _ocsd_decode_stats**)'
>>>>>> /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so:
>>>>>> undefined reference to
>>>>>> `DecodeTree::addRawFramePrinter(RawFramePrinter**, unsigned int)'
>>>>>>
>>>>>> Can you try the make install and see if that works? If that works
>>>>>> but the other way doesn't I can try looking into why its not and make a
>>> fix.
>>>>>
>>>>> I tried make install but still it errors out. Looking at make.output
>>>>> I am seeing multiple undefined references.
>>>>>
>>>>> NOTE I have to still set CSINCLUDES and CSLIBS as we are
>>>>> cross-compiling Perf on a x86 host machine.
>>>>
>>>> This should still work if you make install them to the arch specific
>>>> folder for example /usr/lib/aarch64-linux-gnu/. If do 'apt install
>>> libopencsd-dev:arm64'
>>>> I would get the arm ones in there.
>>>>
>>>> Then if I cross build like this I get openCSD linked:
>>>>
>>>> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/perf
>>>> CORESIGHT=1
>>>>
>>>> So for now you should be able to work around the issue in this way.
>>>
>>> I tried copying opencsd libraries to /usr/lib/aarch64-linux-gnu but still I see
>>> undefined reference. Is it because I am using cross-compiler from server
>>> (toolchain path set to $PATH variable)?
>>>
>>> /proj/<snip>/Vitis/2023.2/gnu/aarch64/lin/aarch64-linux/bin/aarch64-linux-
>>> gnu-gcc
>>> I assume CSLIBS and CSINCLUDES should be sufficient to cross compile perf
>>> with coresight support?
>>
>> FYI, I tried with static linking and could proceed to perf compilation.
>>
>
> Static compilation works because the link line is slightly different (it
> includes -lopencsd rather than just -lopencsd_c_api:
>
> ifeq ($(findstring -static,${LDFLAGS}),-static)
> OPENCSDLIBS += -lopencsd -lstdc++
>
> I think that this is related to the issue and there have been some
> recent changes around here. So yes I'll look into the fix for the non
> static and $CSLIBS mode.
There was an issue, thanks for the report. The fix should be here:
https://lore.kernel.org/all/20230707154546.456720-1-james.clark@arm.com/T/#u
Please give it a go and reply with the outcome on that thread.
More information about the linux-arm-kernel
mailing list