Linux coresight arm64 : Incorrect data in cstrace.bin
Pandey, Radhey Shyam
radhey.shyam.pandey at amd.com
Thu Jul 6 05:09:46 PDT 2023
> -----Original Message-----
> From: James Clark <james.clark at arm.com>
> Sent: Wednesday, July 5, 2023 5:44 PM
> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
> Cc: coresight at lists.linaro.org; linux-arm-kernel at lists.infradead.org; Suzuki K
> Poulose <suzuki.poulose at arm.com>; mathieu.poirier at linaro.org;
> mike.leach at linaro.org; leo.yan at linaro.org;
> alexander.shishkin at linux.intel.com; Sarangi, Anirudha
> <anirudha.sarangi at amd.com>
> Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
>
>
>
> On 05/07/2023 12:48, Pandey, Radhey Shyam wrote:
> >> -----Original Message-----
> >> From: James Clark <james.clark at arm.com>
> >> Sent: Wednesday, July 5, 2023 4:29 PM
> >> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>
> >> Cc: coresight at lists.linaro.org; linux-arm-kernel at lists.infradead.org;
> >> Suzuki K Poulose <suzuki.poulose at arm.com>;
> >> mathieu.poirier at linaro.org; mike.leach at linaro.org;
> >> leo.yan at linaro.org; alexander.shishkin at linux.intel.com
> >> Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
> >>
> >>
> >>
> >> On 05/07/2023 10:56, Pandey, Radhey Shyam wrote:
> >>>> -----Original Message-----
> >>>> From: Suzuki K Poulose <suzuki.poulose at arm.com>
> >>>> Sent: Wednesday, July 5, 2023 1:57 PM
> >>>> To: Pandey, Radhey Shyam <radhey.shyam.pandey at amd.com>;
> >>>> mathieu.poirier at linaro.org; mike.leach at linaro.org;
> >>>> leo.yan at linaro.org; alexander.shishkin at linux.intel.com
> >>>> Cc: coresight at lists.linaro.org;
> >>>> linux-arm-kernel at lists.infradead.org
> >>>> Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
> >>>>
> >>>> Hi Radhe Shyam
> >>>>
> >>>> On 05/07/2023 05:38, Pandey, Radhey Shyam wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I am using linux 6.1 kernel coresight framework to capture ETM trace.
> >>>>> Enabled coresight driver and added coresight component device node
> >>>>> in DTS. With that, we could probe all coresight components.
> >>>>>
> >>>>> But when we do capture using sysfs and then read the trace.bin
> >>>>> using ptm2human we see invalid trace data. Trace content changes
> >>>>> on each capture. Any pointers to help narrow down the issue?
> >>>>>
> >>>>> Tried: sysfs capture and perf (with OpenCSD enabled)
> >>>>>
> >>>>> Development board: VCK190 :
> >>>>> https://docs.xilinx.com/r/en-US/ug1366-vck190-eval-bd
> >>>>> https://www.xilinx.com/support/documents/architecture-
> >>>> manuals/am011-ve
> >>>>> rsal-acap-trm.pdf
> >>>>>
> >>>>> xilinx-vck190-20231:/home/petalinux# dmesg | grep -i amba
> >>>>> [ 0.301054] Serial: AMBA PL011 UART driver
> >>>>> [ 0.306226] amba f0c20000.funnel: Fixing up cyclic dependency with
> >>>> f0d70000.etm
> >>>>> [ 0.313624] amba f0c20000.funnel: Fixing up cyclic dependency with
> >>>> f0d30000.etm
> >>>>> [ 0.321110] amba f0c30000.etf: Fixing up cyclic dependency with
> >>>> f0c20000.funnel
> >>>>> [ 1.470247] Serial: AMBA driver
> >>>>> xilinx-vck190-20231:/home/petalinux# dmesg | grep -i coresight
> >>>>> [ 1.865006] cs_system_cfg: CoreSight Configuration manager
> initialised
> >>>>> [ 1.876858] coresight etm0: CPU0: etm v4.0 initialized
> >>>>> [ 1.885666] coresight etm1: CPU1: etm v4.0 initialized
> >>>>> [ 1.894435] coresight-cpu-debug f0d00000.debug1: Coresight debug-
> >> CPU0
> >>>> initialized
> >>>>> [ 1.902072] coresight-cpu-debug f0d40000.debug1: Coresight debug-
> >> CPU1
> >>>> initialized
> >>>>>
> >>>>> xilinx-vck190-20231:/home/petalinux# ls /sys/bus/coresight/devices/
> >>>>> etm0 etm1 funnel0 tmc_etf0
> >>>>>
> >>>>> cd /sys/bus/coresight/devices/
> >>>>> echo 1 > tmc_etf0/enable_sink
> >>>>> echo 1 > etm0/enable_source
> >>>>> echo 0 > etm0/enable_source
> >>>>> echo 0 > tmc_etf0/enable_sink
> >>>>> cd /root/ dd if=/dev/tmc_etf0 of=cstrace_28Jun.bin
> >>>>>
> >>>>> ./ptm2human/ptm2human -e -i cstrace.bin -d
> >>>>
> >>>> Please note that ptm2human is for PTM trace decoding and
> >>>> ETMv4 uses a different format and thus is not compatible.
> >>>>
> >>>>>
> >>>>> I also tried OpenCSD integration with PERF.
> >>>>
> >>>> Have you made sure the perf is "linked" to the opencsd ?
> >>>
> >>> Earlier I statically linked but after adding CORESIGHT=1.
> >>>
> >>>
> >>> linux-xlnx$ make ARCH=arm64 NO_LIBELF=1 NO_JVMTI=1 VF=1
> >> CORESIGHT=1 -C
> >>> tools/perf/
> >>> make: Entering directory 'linux-xlnx/tools/perf'
> >>> BUILD: Doing 'make -j24' parallel build
> >>>
> >>> Makefile.config:520: *** Error: No libopencsd library found or the
> >>> version
> >> is not up-to-date.
> >>> Please install recent libopencsd to build with CORESIGHT=1. Stop.
> >>>
> >>>
> >>> echo $CSINCLUDES
> >>> /scratch/development/coresight/my-opencsd/decoder/include
> >>> radheys at xhdradheys41:/scratch/development/linux-xlnx$ ls
> >>> /scratch/development/coresight/my-opencsd/decoder/include
> >>> common i_dec interfaces mem_acc opencsd opencsd.h pkt_printers
> >>>
> >>> echo $CSLIBS
> >>> /scratch/development/coresight/my-opencsd/decoder/lib/builddir
> >>> radheys at xhdradheys41:/scratch/development/linux-xlnx$ ls
> >> /scratch/development/coresight/my-opencsd/decoder/lib/builddir
> >>> libopencsd.a libopencsd_c_api.so libopencsd_c_api.so.1.4.0
> >> libopencsd.so.1
> >>> libopencsd_c_api.a libopencsd_c_api.so.1 libopencsd.so
> >> libopencsd.so.1.4.0
> >>>
> >>> Anything I am missing to fix this opencsd lib not found?
> >>
> >> There is a minimum version of 1.1.1 but it looks like you are using
> >> higher than that so that part should be fine.
> >>
> >> Personally I "make install" OpenCSD to the system path. I had a play
> >> around with $CSLIBS it and maybe there is also some stickyness to the
> >> feature detection so a make clean might help to make sure the errors
> >> you are seeing are real.
> >>
> >> Lastly you can print the output of why the feature test compilation
> >> failed which might help. I see some linking issues in there if I
> >> delete my system installed version of OpenCSD, so there might be a
> >> bug with just using CSINCLUDES and CSLIBS:
> >>
> >> $ cat tools/build/feature/test-libopencsd.make.output
> >>
> >> /usr/bin/ld: warning: libopencsd.so.1, needed by opencsd-
> >> local/decoder/lib/builddir//libopencsd_c_api.so, not found (try using
> >> -rpath or -rpath-link)
> >> /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so:
> >> undefined reference to `vtable for StmTrcPacket'
> >> /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so:
> >> undefined reference to `DecodeTree::getDecoderStats(unsigned char,
> >> _ocsd_decode_stats**)'
> >> /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so:
> >> undefined reference to
> >> `DecodeTree::addRawFramePrinter(RawFramePrinter**, unsigned int)'
> >>
> >> Can you try the make install and see if that works? If that works but
> >> the other way doesn't I can try looking into why its not and make a fix.
> >
> > I tried make install but still it errors out. Looking at make.output I
> > am seeing multiple undefined references.
> >
> > NOTE I have to still set CSINCLUDES and CSLIBS as we are
> > cross-compiling Perf on a x86 host machine.
>
> This should still work if you make install them to the arch specific folder for
> example /usr/lib/aarch64-linux-gnu/. If do 'apt install libopencsd-dev:arm64'
> I would get the arm ones in there.
>
> Then if I cross build like this I get openCSD linked:
>
> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/perf
> CORESIGHT=1
>
> So for now you should be able to work around the issue in this way.
I tried copying opencsd libraries to /usr/lib/aarch64-linux-gnu but still I see
undefined reference. Is it because I am using cross-compiler from server
(toolchain path set to $PATH variable)?
/proj/<snip>/Vitis/2023.2/gnu/aarch64/lin/aarch64-linux/bin/aarch64-linux-gnu-gcc
I assume CSLIBS and CSINCLUDES should be sufficient to cross compile perf with
coresight support?
>
> >
> > cat tools/build/feature/test-libopencsd.make.output <snip library
> > paths>
> >
> > aarch64-xilinx-linux-ld.real: warning: libopencsd.so.1, needed by
> > libopencsd_c_api.so, not found (try using -rpath or -rpath-link)
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `vtable for StmTrcPacket'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::getDecoderStats(unsigned char, _ocsd_decode_stats**)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::addRawFramePrinter(RawFramePrinter**, unsigned int)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::logMappedRanges()'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `OcsdLibDcdRegister::deregisterAllDecoders()'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `PtmTrcPacket::toString(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >&) const'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::s_error_logger'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::addBinFileRegionMemAcc(_ocsd_file_mem_region const*, int,
> _ocsd_mem_space_acc_t, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `EtmV4ITrcPacket::~EtmV4ITrcPacket()'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `OcsdLibDcdRegister::getDecoderRegister()'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `typeinfo for TraceComponent'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `EtmV3TrcPacket::~EtmV3TrcPacket()'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `OcsdTraceElement::toString(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >&) const'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::addGenElemPrinter(TrcGenericElementPrinter**)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::DestroyDecodeTree(DecodeTree*)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `ocsdError::getErrorString[abi:cxx11](ocsdError const&)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::setGenTraceElemOutI(ITrcGenElemIn*)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `ocsdError::ocsdError(ocsdError const*)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `ocsdError::ocsdError(_ocsd_err_severity_t, _ocsd_err_t, unsigned int,
> unsigned char, std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> > const&)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `EtmV4ITrcPacket::toString(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >&) const'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `DecodeTree::createDecoder(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, int, CSConfig const*)'
> > aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to
> `StmTrcPacket::StmTrcPacket()'
> >
>
> Yes this looks like what I see as well. There is probably some issue with
> CSLIBS not being propagated to all parts of the build. I would recommend
> using the system library path for now and I will look into a fix next week.
> Unless you are happy to look into it and send a fix that would be great too.
>
> James
>
> > Thanks,
> > Radhey
> >
> >>
> >> Thanks
> >> James
> >
More information about the linux-arm-kernel
mailing list