[Question] How to enable CoreSight ETMv4 on rk3399?
Seonghyun Park
shp4rk at gmail.com
Thu Jan 9 23:40:22 PST 2025
Hi Bo, thank you for the information!
Perhaps on rk3399 the trace unit core power domain is connected to the
PE core power domain, too as explained in the section 3.5.6 figure 3-2
of the ETMv4 specification.
Best Regards
Seonghyun
On Fri, Jan 10, 2025 at 4:11 PM Bo Gan <ganboing at gmail.com> wrote:
>
> Hi Seonghyun,
>
> Hope you are still around. I revived my rockpro64 for some coresight debugging.
> Robin is right. We need to keep all those components clocked/enabled for a
> successful scan. Easiest way is to disable cpu-sleep/cluster-sleep through
> cpupower:
>
> $ cpupower idle-set -D 2
> $ cpupower idle-info
> CPUidle driver: psci_idle
> CPUidle governor: menu
> analyzing CPU 0:
>
> Number of idle states: 3
> Available idle states: WFI cpu-sleep cluster-sleep
> WFI:
> Flags/Description: ARM WFI
> Latency: 1
> Usage: 50336
> Duration: 4990394200
> cpu-sleep (DISABLED) :
> Flags/Description: cpu-sleep
> Latency: 370
> Usage: 3226
> Duration: 3412630
> cluster-sleep (DISABLED) :
> Flags/Description: cluster-sleep
> Latency: 900
> Usage: 13599
> Duration: 120658311
>
> $ python3 coresight-tools/csscan.py --topology 0xfe400000
>
> @0xfe400000 0x000 0x000 r0.0 ROM table
> @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6 locked
> @0xfe403000 0x23b 0x906 r4.0 CS CTI CTI <no arch> channels:4 triggers:8 locked
> @0xfe404000 0x23b 0x101 r1.0 TM101 Timestamp CoreSight timestamp generator
> @0xfe405000 0x23b 0x912 r4.0 CS TPIU port <no arch> TPIU locked
> @0xfe420000 0x23b 0x4a3 r4.0 ROM table
> @0xfe430000 0x23b 0xd03 r4.0 Cortex-A53 debug core-debug Arm v8.0-A aff=0x80000000 midr=0x410fd034 pfr=0x1002222 dfr=0x10305106 bkpt:6(2) wpt:4 PMUv3 pc-sampling:3 locked
> @0xfe431000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3.0 aff=0x80000000 counters:6 64-bit prescale exportable id: 67ffbfff 00000000 00000000 00000000 locked
> @0xfe432000 0x23b 0xd03 r4.0 Cortex-A53 debug core-debug Arm v8.0-A aff=0x80000001 midr=0x410fd034 pfr=0x1002222 dfr=0x10305106 bkpt:6(2) wpt:4 PMUv3 pc-sampling:3 locked
> @0xfe433000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3.0 aff=0x80000001 counters:6 64-bit prescale exportable id: 67ffbfff 00000000 00000000 00000000 locked
> @0xfe434000 0x23b 0xd03 r4.0 Cortex-A53 debug core-debug Arm v8.0-A aff=0x80000002 midr=0x410fd034 pfr=0x1002222 dfr=0x10305106 bkpt:6(2) wpt:4 PMUv3 pc-sampling:3 locked
> @0xfe435000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3.0 aff=0x80000002 counters:6 64-bit prescale exportable id: 67ffbfff 00000000 00000000 00000000 locked
> @0xfe436000 0x23b 0xd03 r4.0 Cortex-A53 debug core-debug Arm v8.0-A aff=0x80000003 midr=0x410fd034 pfr=0x1002222 dfr=0x10305106 bkpt:6(2) wpt:4 PMUv3 pc-sampling:3 locked
> @0xfe437000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3.0 aff=0x80000003 counters:6 64-bit prescale exportable id: 67ffbfff 00000000 00000000 00000000 locked
> @0xfe438000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI aff=0x80000000 channels:4 triggers:8 gate-inputs locked
> @0xfe439000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI aff=0x80000001 channels:4 triggers:8 gate-inputs locked
> @0xfe43a000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI aff=0x80000002 channels:4 triggers:8 gate-inputs locked
> @0xfe43b000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI aff=0x80000003 channels:4 triggers:8 gate-inputs locked
> @0xfe43c000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4.0 aff=0x80000000 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:30 extinsel:4 locked
> @0xfe43d000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4.0 aff=0x80000001 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:30 extinsel:4 locked
> @0xfe43e000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4.0 aff=0x80000002 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:30 extinsel:4 locked
> @0xfe43f000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4.0 aff=0x80000003 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:30 extinsel:4 locked
> @0xfe600000 0x23b 0x4a4 r0.0 Cortex-A72 ROM ROM table
> @0xfe610000 0x23b 0xd08 r0.0 Cortex-A72 debug core-debug Arm v8.0-A aff=0x80000100 midr=0x410fd082 pfr=0x1002222 dfr=0x10305106 bkpt:6(2) wpt:4 PMUv3 pc-sampling:3 locked
> @0xfe620000 0x23b 0x906 r4.0 CS CTI CTI <no arch> channels:4 triggers:8 locked
> @0xfe630000 0x23b 0x9d8 r0.0 Cortex-A72 PMU PMU (core) Arm PMUv3.0 aff=0x80000100 counters:6 64-bit prescale exportable id: 7fff0f3f 00000000 00000000 00000000 locked
> @0xfe640000 0x23b 0x95a r0.0 Cortex-A72 ETM ETM Arm ETMv4.0 aff=0x80000100 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:110 extinsel:4 locked
> @0xfe710000 0x23b 0xd08 r0.0 Cortex-A72 debug core-debug Arm v8.0-A aff=0x80000101 midr=0x410fd082 pfr=0x1002222 dfr=0x10305106 bkpt:6(2) wpt:4 PMUv3 pc-sampling:3 locked
> @0xfe720000 0x23b 0x906 r4.0 CS CTI CTI <no arch> channels:4 triggers:8 locked
> @0xfe730000 0x23b 0x9d8 r0.0 Cortex-A72 PMU PMU (core) Arm PMUv3.0 aff=0x80000101 counters:6 64-bit prescale exportable id: 7fff0f3f 00000000 00000000 00000000 locked
> @0xfe740000 0x23b 0x95a r0.0 Cortex-A72 ETM ETM Arm ETMv4.0 aff=0x80000101 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:110 extinsel:4 locked
>
> ATB topology detection
> ATB scan... @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6 unlocked integration
> 0->0 @0xfe405000 0x23b 0x912 r4.0 CS TPIU port <no arch> TPIU unlocked integration
> ATB scan... @0xfe43c000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4.0 aff=0x80000000 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:30 extinsel:4 unlocked integration
> 0->0 @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6 unlocked integration
> ATB scan... @0xfe43d000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4.0 aff=0x80000001 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:30 extinsel:4 unlocked integration
> 0->1 @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6 unlocked integration
> ATB scan... @0xfe43e000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4.0 aff=0x80000002 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:30 extinsel:4 unlocked integration
> 0->2 @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6 unlocked integration
> ATB scan... @0xfe43f000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4.0 aff=0x80000003 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:30 extinsel:4 unlocked integration
> 0->3 @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6 unlocked integration
> ATB scan... @0xfe640000 0x23b 0x95a r0.0 Cortex-A72 ETM ETM Arm ETMv4.0 aff=0x80000100 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:110 extinsel:4 unlocked integration
> 0->4 @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6 unlocked integration
> ATB scan... @0xfe740000 0x23b 0x95a r0.0 Cortex-A72 ETM ETM Arm ETMv4.0 aff=0x80000101 pdsr=0x00000021 ETMv4.0 ts:64 bb cc min-ccit:4 retstack events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2 seqstates:4 extin:110 extinsel:4 unlocked integration
> 0->5 @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6 unlocked integration
> ATB topology detection complete.
>
> On 6/7/23 03:42, Robin Murphy wrote:
> > On 2023-06-05 14:21, Seonghyun Park wrote:
> >> Hi, community.
> >>
> >> I am trying to reproduce coresight-trace, a CoreSight ETMv4 trace decoder
> >> utility, (https://github.com/RICSecLab/coresight-trace) on RockPro64 with
> >> vanilla 6.3.1 Linux kernel and vanilla dts.
> >>
> >> The utility depends on a library called CSAL
> >> (https://github.com/RICSecLab/CSAL/tree/fc8c493) which is used to configure
> >> CoreSight registers.
> >>
> >> I found that the helper script, csscan.py
> >> (https://github.com/ARM-software/CSAL/blob/fc8c493/csscan.py), which is
> >> supposed to print the topology of CoreSignt components given the address
> >> of the ROM table, but it ends up halting the whole machine when trying to
> >> access CoreSight components within Big core cluster. Here is the failing log:
> >>
> >> ```
> >> sudo python3 coresight-tools/csscan.py --topology 0xfe400000
> >> @0xfe400000 0x000 0x000 r0.0 ROM table
> >> @0xfe401000 0x23b 0x908 r2.0 CS Funnel funnel <no arch> in-ports:6
> >> @0xfe403000 0x23b 0x906 r4.0 CS CTI CTI <no arch> channels:4 triggers:8
> >> @0xfe404000 0x23b 0x101 r1.0 TM101 Timestamp CoreSight timestamp generator
> >> @0xfe405000 0x23b 0x912 r4.0 CS TPIU port <no arch> TPIU
> >> @0xfe420000 0x23b 0x4a3 r4.0 ROM table
> >> @0xfe430000 - device excluded from scan
> >> @0xfe431000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3 rev0
> >> aff=0x80000000 not acessing
> >> @0xfe432000 - device excluded from scan
> >> @0xfe433000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3 rev0
> >> aff=0x80000001 not acessing
> >> @0xfe434000 - device excluded from scan
> >> @0xfe435000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3 rev0
> >> aff=0x80000002 not acessing
> >> @0xfe436000 - device excluded from scan
> >> @0xfe437000 0x23b 0x9d3 r4.0 Cortex-A53 PMU PMU (core) Arm PMUv3 rev0
> >> aff=0x80000003 not acessing
> >> @0xfe438000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI rev0
> >> aff=0x80000000 channels:4 triggers:8 gate
> >> @0xfe439000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI rev0
> >> aff=0x80000001 channels:4 triggers:8 gate
> >> @0xfe43a000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI rev0
> >> aff=0x80000002 channels:4 triggers:8 gate
> >> @0xfe43b000 0x23b 0x9a8 r4.0 Cortex-A53 CTI CTI Arm CTI rev0
> >> aff=0x80000003 channels:4 triggers:8 gate
> >> @0xfe43c000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4 rev0
> >> aff=0x80000000 pdsr=0x00000023 ETMv4.0 ts:64 bb cc min-ccit:4 retstack
> >> stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2
> >> seqstates:4 extin:30 extinsel:4
> >> @0xfe43d000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4 rev0
> >> aff=0x80000001 pdsr=0x00000023 ETMv4.0 ts:64 bb cc min-ccit:4 retstack
> >> stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2
> >> seqstates:4 extin:30 extinsel:4
> >> @0xfe43e000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4 rev0
> >> aff=0x80000002 pdsr=0x00000023 ETMv4.0 ts:64 bb cc min-ccit:4 retstack
> >> stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2
> >> seqstates:4 extin:30 extinsel:4
> >> @0xfe43f000 0x23b 0x95d r4.0 Cortex-A53 ETM ETM Arm ETMv4 rev0
> >> aff=0x80000003 pdsr=0x00000023 ETMv4.0 ts:64 bb cc min-ccit:4 retstack
> >> stall events:4 resources:16 addrcomp:4 ssc:1 pecomp:0 counters:2
> >> seqstates:4 extin:30 extinsel:4
> >> @0xfe600000 0x23b 0x4a4 r0.0 ROM table
> >> @0xfe610000 - device excluded from scan
> >> @0xfe620000 0x23b 0x906 r4.0 CS CTI CTI <no arch> channels:4 triggers:8
> >>
> >> (halt)
> >> ```
> >>
> >> Based on the TRM, the address it starts to fail seems to be around the
> >> CLUSTERB_CTI0 or CLUSTERB_PMU0, implying that there are some issues with
> >> accessing CoreSight components on the Big core cluster.
> >
> > Most likely because the big cores are idle such that the cluster is clock-gated and/or powered off. This is one of the perils of poking around in /dev/mem.
> >
> >> I'd like to ask if you have any recommendations on what I should do to resolve
> >> this issue.
> >
> > You could muck about with sysfs and/or kernel parameters trying to disable as many power management features as possible, but the better option would be to add the CoreSight components to the devicetree and use the kernel drivers which can handle clocks and power domains properly. See [1] for the overview.
> >
> > Robin.
> >
> > [1] https://www.kernel.org/doc/html/latest/trace/coresight/index.html
> >
> > _______________________________________________
> > Linux-rockchip mailing list
> > Linux-rockchip at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-rockchip
>
> Bo
More information about the Linux-rockchip
mailing list