[Question] How to enable CoreSight ETMv4 on rk3399?

[Robin Murphy]

On 2023-06-05 14:21, Seonghyun Park wrote:
> Hi, community.
> 
> I am trying to reproduce coresight-trace, a CoreSight ETMv4 trace decoder
> utility, (https://github.com/RICSecLab/coresight-trace) on RockPro64 with
> vanilla 6.3.1 Linux kernel and vanilla dts.
> 
> The utility depends on a library called CSAL
> (https://github.com/RICSecLab/CSAL/tree/fc8c493) which is used to configure
> CoreSight registers.
> 
> I found that the helper script, csscan.py
> (https://github.com/ARM-software/CSAL/blob/fc8c493/csscan.py), which is
> supposed to print the topology of CoreSignt components given the address
> of the ROM table, but it ends up halting the whole machine when trying to
> access CoreSight components within Big core cluster. Here is the failing log:
> 
> ```
> sudo python3 coresight-tools/csscan.py --topology 0xfe400000
> @0xfe400000 0x000 0x000 r0.0 ROM table
> @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6
> @0xfe403000 0x23b 0x906 r4.0 CS CTI CTI <no arch> channels:4 triggers:8
> @0xfe404000 0x23b 0x101 r1.0 TM101 Timestamp CoreSight timestamp generator
> @0xfe405000 0x23b 0x912 r4.0 CS TPIU port <no arch> TPIU
> @0xfe420000 0x23b 0x4a3 r4.0 ROM table
> @0xfe430000 - device excluded from scan
> @0xfe431000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3 rev0
> aff=0x80000000 not acessing
> @0xfe432000 - device excluded from scan
> @0xfe433000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3 rev0
> aff=0x80000001 not acessing
> @0xfe434000 - device excluded from scan
> @0xfe435000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3 rev0
> aff=0x80000002 not acessing
> @0xfe436000 - device excluded from scan
> @0xfe437000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3 rev0
> aff=0x80000003 not acessing
> @0xfe438000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI rev0
> aff=0x80000000 channels:4 triggers:8 gate
> @0xfe439000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI rev0
> aff=0x80000001 channels:4 triggers:8 gate
> @0xfe43a000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI rev0
> aff=0x80000002 channels:4 triggers:8 gate
> @0xfe43b000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI rev0
> aff=0x80000003 channels:4 triggers:8 gate
> @0xfe43c000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4 rev0
> aff=0x80000000 pdsr=0x00000023 ETMv4.0 ts:64 bb cc min-ccit:4 retstack
> stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2
> seqstates:4 extin:30 extinsel:4
> @0xfe43d000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4 rev0
> aff=0x80000001 pdsr=0x00000023 ETMv4.0 ts:64 bb cc min-ccit:4 retstack
> stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2
> seqstates:4 extin:30 extinsel:4
> @0xfe43e000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4 rev0
> aff=0x80000002 pdsr=0x00000023 ETMv4.0 ts:64 bb cc min-ccit:4 retstack
> stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2
> seqstates:4 extin:30 extinsel:4
> @0xfe43f000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4 rev0
> aff=0x80000003 pdsr=0x00000023 ETMv4.0 ts:64 bb cc min-ccit:4 retstack
> stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2
> seqstates:4 extin:30 extinsel:4
> @0xfe600000 0x23b 0x4a4 r0.0 ROM table
> @0xfe610000 - device excluded from scan
> @0xfe620000 0x23b 0x906 r4.0 CS CTI CTI <no arch> channels:4 triggers:8
> 
> (halt)
> ```
> 
> Based on the TRM, the address it starts to fail seems to be around the
> CLUSTERB_CTI0 or CLUSTERB_PMU0, implying that there are some issues with
> accessing CoreSight components on the Big core cluster.

Most likely because the big cores are idle such that the cluster is 
clock-gated and/or powered off. This is one of the perils of poking 
around in /dev/mem.

> I'd like to ask if you have any recommendations on what I should do to resolve
> this issue.

You could muck about with sysfs and/or kernel parameters trying to 
disable as many power management features as possible, but the better 
option would be to add the CoreSight components to the devicetree and 
use the kernel drivers which can handle clocks and power domains 
properly. See [1] for the overview.

Robin.

[1] https://www.kernel.org/doc/html/latest/trace/coresight/index.html