[PATCH V3 11/14] coresight: sink: Add TRBE driver
Mathieu Poirier
mathieu.poirier at linaro.org
Fri Feb 12 12:02:32 EST 2021
On Fri, Feb 12, 2021 at 11:13:01AM +0530, Anshuman Khandual wrote:
>
>
> On 2/11/21 12:30 AM, Mathieu Poirier wrote:
> > On Wed, Jan 27, 2021 at 02:25:35PM +0530, Anshuman Khandual wrote:
> >> Trace Buffer Extension (TRBE) implements a trace buffer per CPU which is
> >> accessible via the system registers. The TRBE supports different addressing
> >> modes including CPU virtual address and buffer modes including the circular
> >> buffer mode. The TRBE buffer is addressed by a base pointer (TRBBASER_EL1),
> >> an write pointer (TRBPTR_EL1) and a limit pointer (TRBLIMITR_EL1). But the
> >> access to the trace buffer could be prohibited by a higher exception level
> >> (EL3 or EL2), indicated by TRBIDR_EL1.P. The TRBE can also generate a CPU
> >> private interrupt (PPI) on address translation errors and when the buffer
> >> is full. Overall implementation here is inspired from the Arm SPE driver.
> >>
> >> Cc: Mathieu Poirier <mathieu.poirier at linaro.org>
> >> Cc: Mike Leach <mike.leach at linaro.org>
> >> Cc: Suzuki K Poulose <suzuki.poulose at arm.com>
> >> Signed-off-by: Anshuman Khandual <anshuman.khandual at arm.com>
> >> ---
> >> Changes in V3:
> >>
> >> - Added new DT bindings document TRBE.yaml
> >> - Changed TRBLIMITR_TRIG_MODE_SHIFT from 2 to 3
> >> - Dropped isb() from trbe_reset_local()
> >> - Dropped gap between (void *) and buf->trbe_base
> >> - Changed 'int' to 'unsigned int' in is_trbe_available()
> >> - Dropped unused function set_trbe_running(), set_trbe_virtual_mode(),
> >> set_trbe_enabled() and set_trbe_limit_pointer()
> >> - Changed get_trbe_flag_update(), is_trbe_programmable() and
> >> get_trbe_address_align() to accept TRBIDR value
> >> - Changed is_trbe_running(), is_trbe_abort(), is_trbe_wrap(), is_trbe_trg(),
> >> is_trbe_irq(), get_trbe_bsc() and get_trbe_ec() to accept TRBSR value
> >> - Dropped snapshot mode condition in arm_trbe_alloc_buffer()
> >> - Exit arm_trbe_init() when arm64_kernel_unmapped_at_el0() is enabled
> >> - Compute trbe_limit before trbe_write to get the updated handle
> >> - Added trbe_stop_and_truncate_event()
> >> - Dropped trbe_handle_fatal()
> >>
> >> Documentation/trace/coresight/coresight-trbe.rst | 39 +
> >> arch/arm64/include/asm/sysreg.h | 1 +
> >> drivers/hwtracing/coresight/Kconfig | 11 +
> >> drivers/hwtracing/coresight/Makefile | 1 +
> >> drivers/hwtracing/coresight/coresight-trbe.c | 1023 ++++++++++++++++++++++
> >> drivers/hwtracing/coresight/coresight-trbe.h | 160 ++++
> >> 6 files changed, 1235 insertions(+)
> >> create mode 100644 Documentation/trace/coresight/coresight-trbe.rst
> >> create mode 100644 drivers/hwtracing/coresight/coresight-trbe.c
> >> create mode 100644 drivers/hwtracing/coresight/coresight-trbe.h
> >> +
> >
> > [...]
> >
> >> +static void arm_trbe_probe_coresight_cpu(void *info)
> >> +{
> >> + struct trbe_drvdata *drvdata = info;
> >> + struct coresight_desc desc = { 0 };
> >> + int cpu = smp_processor_id();
> >> + struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu);
> >> + struct coresight_device *trbe_csdev = per_cpu(csdev_sink, cpu);
> >> + u64 trbidr = read_sysreg_s(SYS_TRBIDR_EL1);
> >> + struct device *dev;
> >> +
> >> + if (WARN_ON(!cpudata))
> >> + goto cpu_clear;
> >
> > There is already a check for this in arm_trbe_probe_coresight(), we couldn't be
> > here if there was a problem with the allocation.
>
> Right but just to be extra cautious. Do you really want this to be dropped ?
I don't think it is necessary but there is no harm in keeping it if you are keen
on it.
>
> >
> >> +
> >> + if (trbe_csdev)
> >> + return;
> >
> > Now that's a reason to have a WARN_ON(). If we are probing and a sink is
> > already present in this cpu's slot, something went seriously wrong and we should
> > be clear about it.
>
> Right, will add an WARN_ON().
>
> >
> >> +
> >> + cpudata->cpu = smp_processor_id();
> >> + cpudata->drvdata = drvdata;
> >> + dev = &cpudata->drvdata->pdev->dev;
> >> +
> >> + if (!is_trbe_available()) {
> >> + pr_err("TRBE is not implemented on cpu %d\n", cpudata->cpu);
> >> + goto cpu_clear;
> >> + }
> >> +
> >> + if (!is_trbe_programmable(trbidr)) {
> >> + pr_err("TRBE is owned in higher exception level on cpu %d\n", cpudata->cpu);
> >> + goto cpu_clear;
> >> + }
> >> + desc.name = devm_kasprintf(dev, GFP_KERNEL, "%s%d", DRVNAME, smp_processor_id());
> >
> > We will end up with "arm_trbe0", "arm_trbe1" and so on in sysfs... Is the
> > "arm_" part absolutely needed? I think this should be like what we do for etmv3
> > and etmv4 where only "etmX" shows up in sysfs.
>
> Okay, will drop arm_ here. IIRC this was originally trbeX where X is the cpu number
> but then ended up using DRVNAME as prefix.
>
> >
> >> + if (IS_ERR(desc.name))
> >> + goto cpu_clear;
> >> +
> >> + desc.type = CORESIGHT_DEV_TYPE_SINK;
> >> + desc.subtype.sink_subtype = CORESIGHT_DEV_SUBTYPE_SINK_PERCPU_SYSMEM;
> >> + desc.ops = &arm_trbe_cs_ops;
> >> + desc.pdata = dev_get_platdata(dev);
> >> + desc.groups = arm_trbe_groups;
> >> + desc.dev = dev;
> >> + trbe_csdev = coresight_register(&desc);
> >> + if (IS_ERR(trbe_csdev))
> >> + goto cpu_clear;
> >> +
> >> + dev_set_drvdata(&trbe_csdev->dev, cpudata);
> >> + cpudata->trbe_dbm = get_trbe_flag_update(trbidr);
> >> + cpudata->trbe_align = 1ULL << get_trbe_address_align(trbidr);
> >> + if (cpudata->trbe_align > SZ_2K) {
> >> + pr_err("Unsupported alignment on cpu %d\n", cpudata->cpu);
> >> + goto cpu_clear;
> >
> > Here coresight_unregister() should be called. The other option is to call
> > coresight_register() when everything else is known to be fine, which is the
> > favoured approach.
>
> Okay, will change accordingly.
>
> >
> >> + }
> >> + per_cpu(csdev_sink, cpu) = trbe_csdev;
> >> + trbe_reset_local();
> >> + enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE);
> >> + return;
> >> +cpu_clear:
> >> + cpumask_clear_cpu(cpudata->cpu, &cpudata->drvdata->supported_cpus);
> >> +}
> >> +
> >> +static void arm_trbe_remove_coresight_cpu(void *info)
> >> +{
> >> + int cpu = smp_processor_id();
> >> + struct trbe_drvdata *drvdata = info;
> >> + struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu);
> >> + struct coresight_device *trbe_csdev = per_cpu(csdev_sink, cpu);
> >> +
> >> + if (trbe_csdev) {
> >
> > In what scenario do you see not having a trbe_csdev and still needing to disable
> > IRQs for the HW? If there is a such a case then a few lines of comment is
> > needed.
> >
> >> + coresight_unregister(trbe_csdev);
> >> + cpudata->drvdata = NULL;
> >> + per_cpu(csdev_sink, cpu) = NULL;
> >> + }
> >> + disable_percpu_irq(drvdata->irq);
> >> + trbe_reset_local();
> >
> > Theoretically this code shouldn't run when the TRBE is enabled, because the CS
> > core will prevent that from happening. As sush disabling interrupts after
> > coresight_unregister() has been called and setting cpudata->drvdata to NULL
> > should be fine. But from an outsider's point of view it will look very bizarre.
> > Either write a comment to explain all that or call the above two before doing
> > the cleanup.
>
> Okay, will move them before the cleanup.
>
> >
> >> +}
> >> +
> >> +static int arm_trbe_probe_coresight(struct trbe_drvdata *drvdata)
> >> +{
> >> + drvdata->cpudata = alloc_percpu(typeof(*drvdata->cpudata));
> >> + if (IS_ERR(drvdata->cpudata))
> >> + return PTR_ERR(drvdata->cpudata);
> >
> > As far as I can tell alloc_percpu() returns NULL on failure and nothing else.
>
> Sure, will change the return code as -ENOMEM when alloc_percpu() returns NULL.
>
> >
> >> +
> >> + arm_trbe_probe_coresight_cpu(drvdata);
> >> + smp_call_function_many(&drvdata->supported_cpus, arm_trbe_probe_coresight_cpu, drvdata, 1);
> >
> > The above two calls look racy to me. The executing process could be moved to
> > another CPU between the call to arm_trbe_probe_coresight_cpu() and
> > smp_call_function_many(), which would prevent the initialisation of the TRBE on
> > the new CPU to be done. I suggest using a for_each_cpu() loop where
> > smp_call_function_single() would be used. That way we are guaranteed all the
> > TRBEs will be initialised.
>
> Okay, will change.
>
> >
> >> + return 0;
> >> +}
> >> +
> >> +static int arm_trbe_remove_coresight(struct trbe_drvdata *drvdata)
> >> +{
> >> + arm_trbe_remove_coresight_cpu(drvdata);
> >> + smp_call_function_many(&drvdata->supported_cpus, arm_trbe_remove_coresight_cpu, drvdata, 1);
> >
> > Same as above.
>
> Okay, will do.
>
> >
> > I'm out of time for today, more to come tomorrow.
>
> Okay.
>
> >
> > Mathieu
>
> - Anshuman
More information about the linux-arm-kernel
mailing list