[EXT] Re: [PATCH RESEND] perf/marvell_cn10k: Add MPAM support for TAD PMU

[Amit Singh Tomar]

Hi James,

-----Original Message-----
From: James Morse <james.morse at arm.com> 
Sent: Friday, July 1, 2022 10:50 PM
To: Tanmay Jagdale <tanmay at marvell.com>; Robin Murphy <robin.murphy at arm.com>; Will Deacon <will at kernel.org>
Cc: mark.rutland at arm.com; robh at kernel.org; linux-arm-kernel at lists.infradead.org; linux-kernel at vger.kernel.org; Sunil Kovvuri Goutham <sgoutham at marvell.com>; Linu Cherian <lcherian at marvell.com>; Bharat Bhushan <bbhushan2 at marvell.com>; Amit Singh Tomar <amitsinght at marvell.com>
Subject: [EXT] Re: [PATCH RESEND] perf/marvell_cn10k: Add MPAM support for TAD PMU

External Email

----------------------------------------------------------------------
Hi Tanmay,

On 27/06/2022 14:18, Tanmay Jagdale wrote:
>> On 2022-06-24 13:14, Will Deacon wrote:
>>> On Sat, May 28, 2022 at 12:26:47AM +0530, Tanmay Jagdale wrote:
>>>> The TAD PMU supports following counters that can be filtered by 
>>>> MPAM partition id.
>>>>      - (0x1a) tad_alloc_dtg : Allocations to DTG.
>>>>      - (0x1b) tad_alloc_ltg : Allocations to LTG.
>>>>      - (0x1c) tad_alloc_any : Total allocations to DTG/LTG.
>>>>      - (0x1d) tad_hit_dtg   : DTG hits.
>>>>      - (0x1e) tad_hit_ltg   : LTG hits.
>>>>      - (0x1f) tad_hit_any   : Hit in LTG/DTG.
>>>>      - (0x20) tad_tag_rd    : Total tag reads.
>>>>
>>>> Add a new 'partid' attribute of 16-bits to get the partition id 
>>>> passed from perf tool. This value would be stored in config1 field 
>>>> of perf_event_attr structure.
>>>>
>>>> Example:
>>>> perf stat -e tad/tad_alloc_any,partid=0x12/ <program>
>>>>
>>>> - Drop read of TAD_PRF since we don't have to preserve any
>>>>    bit fields and always write an updated value.
>>>> - Update register offsets of TAD_PRF and TAD_PFC.
>>>
>>> It would be great if you could document some of this under 
>>> Documentation/admin-guide/perf like many of the other PMU drivers 
>>> have done.
>>
>> Especially documenting how the user obtains the required partid value 
>> to pass.

> We created MPAM partitions using the resctrl filesystem interface.
> Example:
>         $ cd /sys/fs/resctrl
>         $ mkdir p1
>         $ echo "L3:0=f" > p1/schemata (configure 4 L3 cache ways)
>         $ mkdir p2
>         $ echo "L3:1=ff0" > p2/schemata (configure 8 L3 cache ways)
> 
> Here directory name 'p1' creates a MPAM partid 0x1 and 'p2' creates
> 0x2 and so on.

You can't rely on this.

See the KNOWN_ISSUES file in the the mpam tree: PARTID 0 should be reserved for unknown hardware. In fact any number of PARTID may be reserved for in-kernel users. You can't guess what the offset might be from user-space.


> Right now, there is no file which exposes the partid to userspace.
> We must rely on the sequential order in which we create partitions via 
> resctrl and use that to derive the partid.

If you dig in the MPAM tree you'll find how I intend to solve this for exposing the MPAM counters via perf. But this is a user-space visible change to resctrl, so it will need to wait until all the refactoring is done and the bulk of the MPAM driver is upstream.

[>>] But these are non-standard (HAS_MSMON=0) monitoring counters that just happen to use MPAM partID. Therefore, IMHO we 
       should expose them via Marvell specific PMU driver, and exposing the MPAM partID to user space (may be under resctrl)
       will help this cause.
      or
      resctrl based PMU driver can still support such counters that are not comply with MPAM standard?


Thanks,
 -Amit