From wxjstz at 126.com  Tue Nov  7 06:32:01 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 07 Nov 2023 22:32:01 +0800
Subject: issue about lr/sc and __riscv_atomic
Message-ID: <f93027a6dd48ed1a0432750c4dd4b24ebeaa1117.camel@126.com>

There are some conditional compilation related to __riscv_atomic in
lib/sbi/riscv_atomic.c. Atomic operations are implemented through lr/sc when
the __riscv_atomic macro is not defined. But lr/sc is also part of the A
extension. If the A extension is not supported, lr/sc should not be used.

We should remove these conditional compilations, opensbi is strongly dependent
on the A extension.

The following is the compilation log when I turned off the A extension.

make CROSS_COMPILE=riscv64-unknown-elf- PLATFORM_RISCV_ISA=rv64imfdc_zicsr_zifencei
 CC        lib/sbi/riscv_asm.o
 CC        lib/sbi/riscv_atomic.o
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c: Assembler messages:
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:37: Error: unrecognized opcode `amoadd.d.aqrl a5,a1,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:37: Error: unrecognized opcode `amoadd.d.aqrl a5,a5,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:173: Error: unrecognized opcode `lr.d a3,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:175: Error: unrecognized opcode `sc.d.rl a2,a4,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:183: Error: unrecognized opcode `lr.d a4,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:184: Error: unrecognized opcode `sc.d.rl a3,a5,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:194: Error: unrecognized opcode `lr.w a4,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:195: Error: unrecognized opcode `sc.w.rl a3,a5,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:205: Error: unrecognized opcode `lr.d a4,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:206: Error: unrecognized opcode `sc.d.rl a3,a5,0(a0)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:237: Error: unrecognized opcode `amoor.d.aqrl a0,a0,0(a1)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:242: Error: unrecognized opcode `amoand.d.aqrl a0,a0,0(a1)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:237: Error: unrecognized opcode `amoor.d.aqrl a0,a0,0(a1)', extension `a' required
/home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:242: Error: unrecognized opcode `amoand.d.aqrl a0,a0,0(a1)', extension `a' required
make: *** [Makefile:504: /home/merle/workspaces/opensbi/build/lib/sbi/riscv_atomic.o] Error 1



From apatel at ventanamicro.com  Tue Nov  7 07:10:03 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Tue, 7 Nov 2023 20:40:03 +0530
Subject: issue about lr/sc and __riscv_atomic
In-Reply-To: <f93027a6dd48ed1a0432750c4dd4b24ebeaa1117.camel@126.com>
References: <f93027a6dd48ed1a0432750c4dd4b24ebeaa1117.camel@126.com>
Message-ID: <CAK9=C2VJGfT_R5LrE_ZkdfJeh_0AKFUYTbbE=EmdRpe8p3jAZA@mail.gmail.com>

On Tue, Nov 7, 2023 at 8:03?PM Xiang W <wxjstz at 126.com> wrote:
>
> There are some conditional compilation related to __riscv_atomic in
> lib/sbi/riscv_atomic.c. Atomic operations are implemented through lr/sc when
> the __riscv_atomic macro is not defined. But lr/sc is also part of the A
> extension. If the A extension is not supported, lr/sc should not be used.
>
> We should remove these conditional compilations, opensbi is strongly dependent
> on the A extension.
>
> The following is the compilation log when I turned off the A extension.
>
> make CROSS_COMPILE=riscv64-unknown-elf- PLATFORM_RISCV_ISA=rv64imfdc_zicsr_zifencei
>  CC        lib/sbi/riscv_asm.o
>  CC        lib/sbi/riscv_atomic.o
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c: Assembler messages:
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:37: Error: unrecognized opcode `amoadd.d.aqrl a5,a1,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:37: Error: unrecognized opcode `amoadd.d.aqrl a5,a5,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:173: Error: unrecognized opcode `lr.d a3,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:175: Error: unrecognized opcode `sc.d.rl a2,a4,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:183: Error: unrecognized opcode `lr.d a4,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:184: Error: unrecognized opcode `sc.d.rl a3,a5,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:194: Error: unrecognized opcode `lr.w a4,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:195: Error: unrecognized opcode `sc.w.rl a3,a5,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:205: Error: unrecognized opcode `lr.d a4,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:206: Error: unrecognized opcode `sc.d.rl a3,a5,0(a0)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:237: Error: unrecognized opcode `amoor.d.aqrl a0,a0,0(a1)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:242: Error: unrecognized opcode `amoand.d.aqrl a0,a0,0(a1)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:237: Error: unrecognized opcode `amoor.d.aqrl a0,a0,0(a1)', extension `a' required
> /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:242: Error: unrecognized opcode `amoand.d.aqrl a0,a0,0(a1)', extension `a' required
> make: *** [Makefile:504: /home/merle/workspaces/opensbi/build/lib/sbi/riscv_atomic.o] Error 1

We did not add "#if" checks because IMA is a minimal requirement for OpenSBI.

Refer, https://github.com/riscv-software-src/opensbi/blob/master/docs/platform_requirements.md

Regards,
Anup


From wxjstz at 126.com  Tue Nov  7 07:41:01 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 07 Nov 2023 23:41:01 +0800
Subject: issue about lr/sc and __riscv_atomic
In-Reply-To: <CAK9=C2VJGfT_R5LrE_ZkdfJeh_0AKFUYTbbE=EmdRpe8p3jAZA@mail.gmail.com>
References: <f93027a6dd48ed1a0432750c4dd4b24ebeaa1117.camel@126.com>
	 <CAK9=C2VJGfT_R5LrE_ZkdfJeh_0AKFUYTbbE=EmdRpe8p3jAZA@mail.gmail.com>
Message-ID: <b9467b1d1907891c5d1b3b2c565207f3330415c7.camel@126.com>

? 2023-11-07???? 20:40 +0530?Anup Patel???
> On Tue, Nov 7, 2023 at 8:03?PM Xiang W <wxjstz at 126.com> wrote:
> > 
> > There are some conditional compilation related to __riscv_atomic in
> > lib/sbi/riscv_atomic.c. Atomic operations are implemented through lr/sc when
> > the __riscv_atomic macro is not defined. But lr/sc is also part of the A
> > extension. If the A extension is not supported, lr/sc should not be used.
> > 
> > We should remove these conditional compilations, opensbi is strongly dependent
> > on the A extension.
> > 
> > The following is the compilation log when I turned off the A extension.
> > 
> > make CROSS_COMPILE=riscv64-unknown-elf- PLATFORM_RISCV_ISA=rv64imfdc_zicsr_zifencei
> > ?CC??????? lib/sbi/riscv_asm.o
> > ?CC??????? lib/sbi/riscv_atomic.o
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c: Assembler messages:
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:37: Error: unrecognized opcode `amoadd.d.aqrl a5,a1,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:37: Error: unrecognized opcode `amoadd.d.aqrl a5,a5,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:173: Error: unrecognized opcode `lr.d a3,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:175: Error: unrecognized opcode `sc.d.rl a2,a4,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:183: Error: unrecognized opcode `lr.d a4,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:184: Error: unrecognized opcode `sc.d.rl a3,a5,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:194: Error: unrecognized opcode `lr.w a4,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:195: Error: unrecognized opcode `sc.w.rl a3,a5,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:205: Error: unrecognized opcode `lr.d a4,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:206: Error: unrecognized opcode `sc.d.rl a3,a5,0(a0)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:237: Error: unrecognized opcode `amoor.d.aqrl a0,a0,0(a1)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:242: Error: unrecognized opcode `amoand.d.aqrl a0,a0,0(a1)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:237: Error: unrecognized opcode `amoor.d.aqrl a0,a0,0(a1)', extension `a' required
> > /home/merle/workspaces/opensbi/lib/sbi/riscv_atomic.c:242: Error: unrecognized opcode `amoand.d.aqrl a0,a0,0(a1)', extension `a' required
> > make: *** [Makefile:504: /home/merle/workspaces/opensbi/build/lib/sbi/riscv_atomic.o] Error 1
> 
> We did not add "#if" checks because IMA is a minimal requirement for OpenSBI.
I mean remove these
https://github.com/riscv-software-src/opensbi/blob/master/lib/sbi/riscv_atomic.c#L170
https://github.com/riscv-software-src/opensbi/blob/master/lib/sbi/riscv_atomic.c#L180
https://github.com/riscv-software-src/opensbi/blob/master/lib/sbi/riscv_atomic.c#L191
https://github.com/riscv-software-src/opensbi/blob/master/lib/sbi/riscv_atomic.c#L202

Regards,
Xiang W

> 
> Refer, https://github.com/riscv-software-src/opensbi/blob/master/docs/platform_requirements.md
> 
> Regards,
> Anup



From wxjstz at 126.com  Tue Nov  7 19:28:16 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  8 Nov 2023 11:28:16 +0800
Subject: [PATCH] lib: sbi: Remove xchg/cmpxchg implemented via lr/sc
Message-ID: <20231108032859.159198-1-wxjstz@126.com>

lr/sc is part of the A extension. If the A extension is not supported,
lr/sc cannot be used. So remove xchg/cmpxchg.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/riscv_atomic.c | 106 ++---------------------------------------
 1 file changed, 4 insertions(+), 102 deletions(-)

diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
index 528686f..a143218 100644
--- a/lib/sbi/riscv_atomic.c
+++ b/lib/sbi/riscv_atomic.c
@@ -12,6 +12,10 @@
 #include <sbi/riscv_atomic.h>
 #include <sbi/riscv_barrier.h>
 
+#ifndef __riscv_atomic
+#error "opensbi strongly relies on the A extension of RISC-V"
+#endif
+
 long atomic_read(atomic_t *atom)
 {
 	long ret = atom->counter;
@@ -79,131 +83,29 @@ long atomic_sub_return(atomic_t *atom, long value)
 		(__typeof__(*(ptr))) __axchg((ptr), _x_, sizeof(*(ptr)));	\
 	})
 
-
-#define __xchg(ptr, new, size)                                            \
-	({                                                                \
-		__typeof__(ptr) __ptr	 = (ptr);                         \
-		__typeof__(*(ptr)) __new = (new);                         \
-		__typeof__(*(ptr)) __ret;                                 \
-		register unsigned int __rc;                               \
-		switch (size) {                                           \
-		case 4:                                                   \
-			__asm__ __volatile__("0:	lr.w %0, %2\n"    \
-					     "	sc.w.rl %1, %z3, %2\n"    \
-					     "	bnez %1, 0b\n"            \
-					     "	fence rw, rw\n"           \
-					     : "=&r"(__ret), "=&r"(__rc), \
-					       "+A"(*__ptr)               \
-					     : "rJ"(__new)                \
-					     : "memory");                 \
-			break;                                            \
-		case 8:                                                   \
-			__asm__ __volatile__("0:	lr.d %0, %2\n"    \
-					     "	sc.d.rl %1, %z3, %2\n"    \
-					     "	bnez %1, 0b\n"            \
-					     "	fence rw, rw\n"           \
-					     : "=&r"(__ret), "=&r"(__rc), \
-					       "+A"(*__ptr)               \
-					     : "rJ"(__new)                \
-					     : "memory");                 \
-			break;                                            \
-		default:                                                  \
-			break;                                            \
-		}                                                         \
-		__ret;                                                    \
-	})
-
-#define xchg(ptr, n)                                                     \
-	({                                                               \
-		__typeof__(*(ptr)) _n_ = (n);                            \
-		(__typeof__(*(ptr))) __xchg((ptr), _n_, sizeof(*(ptr))); \
-	})
-
-#define __cmpxchg(ptr, old, new, size)                                    \
-	({                                                                \
-		__typeof__(ptr) __ptr	 = (ptr);                         \
-		__typeof__(*(ptr)) __old = (old);                         \
-		__typeof__(*(ptr)) __new = (new);                         \
-		__typeof__(*(ptr)) __ret;                                 \
-		register unsigned int __rc;                               \
-		switch (size) {                                           \
-		case 4:                                                   \
-			__asm__ __volatile__("0:	lr.w %0, %2\n"    \
-					     "	bne  %0, %z3, 1f\n"       \
-					     "	sc.w.rl %1, %z4, %2\n"    \
-					     "	bnez %1, 0b\n"            \
-					     "	fence rw, rw\n"           \
-					     "1:\n"                       \
-					     : "=&r"(__ret), "=&r"(__rc), \
-					       "+A"(*__ptr)               \
-					     : "rJ"(__old), "rJ"(__new)   \
-					     : "memory");                 \
-			break;                                            \
-		case 8:                                                   \
-			__asm__ __volatile__("0:	lr.d %0, %2\n"    \
-					     "	bne %0, %z3, 1f\n"        \
-					     "	sc.d.rl %1, %z4, %2\n"    \
-					     "	bnez %1, 0b\n"            \
-					     "	fence rw, rw\n"           \
-					     "1:\n"                       \
-					     : "=&r"(__ret), "=&r"(__rc), \
-					       "+A"(*__ptr)               \
-					     : "rJ"(__old), "rJ"(__new)   \
-					     : "memory");                 \
-			break;                                            \
-		default:                                                  \
-			break;                                            \
-		}                                                         \
-		__ret;                                                    \
-	})
-
-#define cmpxchg(ptr, o, n)                                          \
-	({                                                          \
-		__typeof__(*(ptr)) _o_ = (o);                       \
-		__typeof__(*(ptr)) _n_ = (n);                       \
-		(__typeof__(*(ptr)))                                \
-			__cmpxchg((ptr), _o_, _n_, sizeof(*(ptr))); \
-	})
-
 long atomic_cmpxchg(atomic_t *atom, long oldval, long newval)
 {
-#ifdef __riscv_atomic
 	return __sync_val_compare_and_swap(&atom->counter, oldval, newval);
-#else
-	return cmpxchg(&atom->counter, oldval, newval);
-#endif
 }
 
 long atomic_xchg(atomic_t *atom, long newval)
 {
 	/* Atomically set new value and return old value. */
-#ifdef __riscv_atomic
 	return axchg(&atom->counter, newval);
-#else
-	return xchg(&atom->counter, newval);
-#endif
 }
 
 unsigned int atomic_raw_xchg_uint(volatile unsigned int *ptr,
 				  unsigned int newval)
 {
 	/* Atomically set new value and return old value. */
-#ifdef __riscv_atomic
 	return axchg(ptr, newval);
-#else
-	return xchg(ptr, newval);
-#endif
 }
 
 unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
 				    unsigned long newval)
 {
 	/* Atomically set new value and return old value. */
-#ifdef __riscv_atomic
 	return axchg(ptr, newval);
-#else
-	return xchg(ptr, newval);
-#endif
 }
 
 #if (__SIZEOF_POINTER__ == 8)
-- 
2.42.0



From ganboing at gmail.com  Wed Nov  8 00:09:52 2023
From: ganboing at gmail.com (Bo Gan)
Date: Wed, 8 Nov 2023 00:09:52 -0800
Subject: [PATCH] lib: sbi: Remove xchg/cmpxchg implemented via lr/sc
In-Reply-To: <20231108032859.159198-1-wxjstz@126.com>
References: <20231108032859.159198-1-wxjstz@126.com>
Message-ID: <d50c455f-fb4c-4e63-c256-61af74cf8c9b@gmail.com>

On 11/7/23 7:28 PM, Xiang W wrote:
> lr/sc is part of the A extension. If the A extension is not supported,
> lr/sc cannot be used. So remove xchg/cmpxchg.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> ---
>   lib/sbi/riscv_atomic.c | 106 ++---------------------------------------
>   1 file changed, 4 insertions(+), 102 deletions(-)
>
> diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
> index 528686f..a143218 100644
> --- a/lib/sbi/riscv_atomic.c
> +++ b/lib/sbi/riscv_atomic.c
> @@ -12,6 +12,10 @@
>   #include <sbi/riscv_atomic.h>
>   #include <sbi/riscv_barrier.h>
>   +#ifndef __riscv_atomic
> +#error "opensbi strongly relies on the A extension of RISC-V"
> +#endif
> +
>   long atomic_read(atomic_t *atom)
>   {
Hi Xiang,

I like this patch. I'm new to OpenSBI mailing list, but I've been studying JH7110 (U74) for a while.
I just want to add one more thing. Even if A extension is supported, we might still not able to use
lr/sc. The U74MC has 1 S7 (M/U nommu) core and 4 U7 core (M/S/U). In the U74MC core complex manual
https://starfivetech.com/uploads/u74mc_core_complex_manual_21G1.pdf, Chapter 3.6 states that the S7
core does not have data cache, and LR/SC will generate a precise access exception, thus the only way
for S7 and U7 to synchronize is amo.

My advise is, if for any reason, that we decided to keep lr/sc, perhaps we can consider make it an
optional feature that by default disable, but can be enabled at build time -- to make everyone happy.

Bo


From wxjstz at 126.com  Wed Nov  8 01:26:44 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 08 Nov 2023 17:26:44 +0800
Subject: [PATCH] lib: sbi: Remove xchg/cmpxchg implemented via lr/sc
In-Reply-To: <d50c455f-fb4c-4e63-c256-61af74cf8c9b@gmail.com>
References: <20231108032859.159198-1-wxjstz@126.com>
	 <d50c455f-fb4c-4e63-c256-61af74cf8c9b@gmail.com>
Message-ID: <0569501d1669fee1dfcceae0fb456c76b13c0b80.camel@126.com>

? 2023-11-08???? 00:09 -0800?Bo Gan???
> On 11/7/23 7:28 PM, Xiang W wrote:
> > lr/sc is part of the A extension. If the A extension is not supported,
> > lr/sc cannot be used. So remove xchg/cmpxchg.
> > 
> > Signed-off-by: Xiang W <wxjstz at 126.com>
> > ---
> > ? lib/sbi/riscv_atomic.c | 106 ++---------------------------------------
> > ? 1 file changed, 4 insertions(+), 102 deletions(-)
> > 
> > diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
> > index 528686f..a143218 100644
> > --- a/lib/sbi/riscv_atomic.c
> > +++ b/lib/sbi/riscv_atomic.c
> > @@ -12,6 +12,10 @@
> > ? #include <sbi/riscv_atomic.h>
> > ? #include <sbi/riscv_barrier.h>
> > ? +#ifndef __riscv_atomic
> > +#error "opensbi strongly relies on the A extension of RISC-V"
> > +#endif
> > +
> > ? long atomic_read(atomic_t *atom)
> > ? {
> Hi Xiang,
> 
> I like this patch. I'm new to OpenSBI mailing list, but I've been studying JH7110 (U74) for a while.
> I just want to add one more thing. Even if A extension is supported, we might still not able to use
> lr/sc. The U74MC has 1 S7 (M/U nommu) core and 4 U7 core (M/S/U). In the U74MC core complex manual
> https://starfivetech.com/uploads/u74mc_core_complex_manual_21G1.pdf, Chapter 3.6 states that the S7
> core does not have data cache, and LR/SC will generate a precise access exception, thus the only way
> for S7 and U7 to synchronize is amo.
Thanks for the feedback.

I'm a bit confused: doesn't your example prove that lr/sc doesn't work on a specific platform?
Why do you advise keeping?

Regards,
Xiang W
> 
> My advise is, if for any reason, that we decided to keep lr/sc, perhaps we can consider make it an
> optional feature that by default disable, but can be enabled at build time -- to make everyone happy.
> 
> Bo
> 



From ganboing at gmail.com  Wed Nov  8 02:26:14 2023
From: ganboing at gmail.com (Bo Gan)
Date: Wed, 8 Nov 2023 02:26:14 -0800
Subject: [PATCH] lib: sbi: Remove xchg/cmpxchg implemented via lr/sc
In-Reply-To: <0569501d1669fee1dfcceae0fb456c76b13c0b80.camel@126.com>
References: <20231108032859.159198-1-wxjstz@126.com>
 <d50c455f-fb4c-4e63-c256-61af74cf8c9b@gmail.com>
 <0569501d1669fee1dfcceae0fb456c76b13c0b80.camel@126.com>
Message-ID: <f21d8661-43c5-55c5-b887-c20931f8e839@gmail.com>

On 11/8/23 1:26 AM, Xiang W wrote:
> Thanks for the feedback.
> 
> I'm a bit confused: doesn't your example prove that lr/sc doesn't work on a specific platform?
> Why do you advise keeping?
> 
> Regards,
> Xiang W

I have no preference on whether it should be removed. If there's no objection, go ahead.
If other people find it might be useful and against the removal, then I'd prefer lr/sc to
be made optional, and by default use amo.


From bjorn at kernel.org  Wed Nov  8 11:13:10 2023
From: bjorn at kernel.org (=?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?=)
Date: Wed,  8 Nov 2023 20:13:10 +0100
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13 instruction
Message-ID: <20231108191310.49950-1-bjorn@kernel.org>

From: Bj?rn T?pel <bjorn at rivosinc.com>

The Linux kernel RISC-V image format allows that UEFI Images can also
be booted by non-UEFI firmware. However for that to work, the PE/Image
combo requires that 'MZ' is a valid instruction. On RISC-V, 'MZ' is
only a valid instruction if the hardware is C capable [1].

Emulate "c.li s4,-13" for non-C capable hardware.

Link: https://lore.kernel.org/linux-riscv/20231024192648.25527-1-bjorn at kernel.org/ # [1]
Signed-off-by: Bj?rn T?pel <bjorn at rivosinc.com>
---
 lib/sbi/sbi_illegal_insn.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
index 2be47575a365..a3ade71295ce 100644
--- a/lib/sbi/sbi_illegal_insn.c
+++ b/lib/sbi/sbi_illegal_insn.c
@@ -102,6 +102,19 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
 	return 0;
 }
 
+static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
+{
+	/* Only handle 'MZ'/c.li s4,-13/0x5a3d */
+	if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
+		regs->s4 = -13;
+		regs->mepc += 4;
+
+		return 0;
+	}
+
+	return truly_illegal_insn(insn, regs);
+}
+
 static const illegal_insn_func illegal_insn_table[32] = {
 	truly_illegal_insn, /* 0 */
 	truly_illegal_insn, /* 1 */
@@ -140,6 +153,7 @@ static const illegal_insn_func illegal_insn_table[32] = {
 int sbi_illegal_insn_handler(ulong insn, struct sbi_trap_regs *regs)
 {
 	struct sbi_trap_info uptrap;
+	ulong tmp;
 
 	/*
 	 * We only deal with 32-bit (or longer) illegal instructions. If we
@@ -159,7 +173,11 @@ int sbi_illegal_insn_handler(ulong insn, struct sbi_trap_regs *regs)
 			uptrap.epc = regs->mepc;
 			return sbi_trap_redirect(regs, &uptrap);
 		}
-		if ((insn & 3) != 3)
+
+		tmp = insn & 3;
+		if (tmp == 1)
+			return compressed_insn(insn, regs);
+		else if (tmp != 3)
 			return truly_illegal_insn(insn, regs);
 	}
 

base-commit: cbdd86973901b6be2a1a2d3d6b54f3184fdf9a44
-- 
2.40.1



From schwab at linux-m68k.org  Wed Nov  8 13:55:18 2023
From: schwab at linux-m68k.org (Andreas Schwab)
Date: Wed, 08 Nov 2023 22:55:18 +0100
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13
 instruction
In-Reply-To: <20231108191310.49950-1-bjorn@kernel.org> (=?utf-8?Q?=22Bj?=
 =?utf-8?Q?=C3=B6rn_T=C3=B6pel=22's?=
	message of "Wed, 8 Nov 2023 20:13:10 +0100")
References: <20231108191310.49950-1-bjorn@kernel.org>
Message-ID: <874jhvc3ll.fsf@igel.home>

On Nov 08 2023, Bj?rn T?pel wrote:

> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
> +{
> +	/* Only handle 'MZ'/c.li s4,-13/0x5a3d */
> +	if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
> +		regs->s4 = -13;
> +		regs->mepc += 4;

By skipping 4 bytes execution will resume in the middle of the next insn
(the jump around the header).

-- 
Andreas Schwab, schwab at linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."


From bjorn at kernel.org  Wed Nov  8 16:02:51 2023
From: bjorn at kernel.org (=?UTF-8?B?QmrDtnJuIFTDtnBlbA==?=)
Date: Thu, 9 Nov 2023 01:02:51 +0100
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13 instruction
In-Reply-To: <874jhvc3ll.fsf@igel.home>
References: <20231108191310.49950-1-bjorn@kernel.org> <874jhvc3ll.fsf@igel.home>
Message-ID: <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>

Hi Andreas!

On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
>
> On Nov 08 2023, Bj?rn T?pel wrote:
>
> > +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
> > +{
> > +     /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
> > +     if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
> > +             regs->s4 = -13;
> > +             regs->mepc += 4;
>
> By skipping 4 bytes execution will resume in the middle of the next insn
> (the jump around the header).

This is in a non-C environment -- "!misa_extension('C')", so we're not
jumping into the middle of the next insn.

Note that the Linux kernel needs the change pointed out in this patch,
to build w/o C.


Bj?rn


From jrtc27 at jrtc27.com  Wed Nov  8 16:11:19 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Thu, 9 Nov 2023 00:11:19 +0000
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13
 instruction
In-Reply-To: <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>
References: <20231108191310.49950-1-bjorn@kernel.org>
 <874jhvc3ll.fsf@igel.home>
 <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>
Message-ID: <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>

On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
> 
> Hi Andreas!
> 
> On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
>> 
>> On Nov 08 2023, Bj?rn T?pel wrote:
>> 
>>> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
>>> +{
>>> +     /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
>>> +     if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
>>> +             regs->s4 = -13;
>>> +             regs->mepc += 4;
>> 
>> By skipping 4 bytes execution will resume in the middle of the next insn
>> (the jump around the header).
> 
> This is in a non-C environment -- "!misa_extension('C')", so we're not
> jumping into the middle of the next insn.

But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
whose first 16-bit parcel is the same as c.li and whose second 16-bit
parcel is ignored. I really do not think this is a good idea.

> Note that the Linux kernel needs the change pointed out in this patch,
> to build w/o C.

That sounds like a Linux problem. Other OSes cope just fine. Fixing in
firmware is not the right approach; fix your software instead. Which
may mean abandoning an image format that isn?t fit for purpose. But the
solution isn?t to hack in whatever random crud Linux wants in firmware,
and other non-firmware SBI implementations like hypervisors.

So this is a resounding NAK from me.

Jess

> Bj?rn
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



From bjorn at kernel.org  Wed Nov  8 16:22:17 2023
From: bjorn at kernel.org (=?UTF-8?B?QmrDtnJuIFTDtnBlbA==?=)
Date: Thu, 9 Nov 2023 01:22:17 +0100
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13 instruction
In-Reply-To: <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>
References: <20231108191310.49950-1-bjorn@kernel.org> <874jhvc3ll.fsf@igel.home>
 <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com> <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>
Message-ID: <CAJ+HfNh_bYn6rCdCyveZybyQFDa7CU3nPCZzBKW-rvf4J7ft6g@mail.gmail.com>

On Thu, 9 Nov 2023 at 01:11, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
>
> On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
> >
> > Hi Andreas!
> >
> > On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
> >>
> >> On Nov 08 2023, Bj?rn T?pel wrote:
> >>
> >>> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
> >>> +{
> >>> +     /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
> >>> +     if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
> >>> +             regs->s4 = -13;
> >>> +             regs->mepc += 4;
> >>
> >> By skipping 4 bytes execution will resume in the middle of the next insn
> >> (the jump around the header).
> >
> > This is in a non-C environment -- "!misa_extension('C')", so we're not
> > jumping into the middle of the next insn.
>
> But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
                                                 ^^^^ My very own! :-D

> whose first 16-bit parcel is the same as c.li and whose second 16-bit
> parcel is ignored. I really do not think this is a good idea.
>
> > Note that the Linux kernel needs the change pointed out in this patch,
> > to build w/o C.
>
> That sounds like a Linux problem. Other OSes cope just fine. Fixing in
> firmware is not the right approach; fix your software instead. Which
> may mean abandoning an image format that isn?t fit for purpose. But the
> solution isn?t to hack in whatever random crud Linux wants in firmware,
> and other non-firmware SBI implementations like hypervisors.

There are other kernels that people care about?

> So this is a resounding NAK from me.

Jokes aside; Fair enough! I'll go back to the drawing board.

Thank you for the input!


Bj?rn


From wxjstz at 126.com  Wed Nov  8 20:04:20 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 09 Nov 2023 12:04:20 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13
 instruction
In-Reply-To: <CAJ+HfNh_bYn6rCdCyveZybyQFDa7CU3nPCZzBKW-rvf4J7ft6g@mail.gmail.com>
References: <20231108191310.49950-1-bjorn@kernel.org>
	 <874jhvc3ll.fsf@igel.home>
	 <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>
	 <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>
	 <CAJ+HfNh_bYn6rCdCyveZybyQFDa7CU3nPCZzBKW-rvf4J7ft6g@mail.gmail.com>
Message-ID: <9411269e9975e9ececae2755eaf74271de45b8c0.camel@126.com>

? 2023-11-09???? 01:22 +0100?Bj?rn T?pel???
> On Thu, 9 Nov 2023 at 01:11, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
> > 
> > On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
> > > 
> > > Hi Andreas!
> > > 
> > > On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
> > > > 
> > > > On Nov 08 2023, Bj?rn T?pel wrote:
> > > > 
> > > > > +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
> > > > > +{
> > > > > +???? /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
> > > > > +???? if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
> > > > > +???????????? regs->s4 = -13;
> > > > > +???????????? regs->mepc += 4;
> > > > 
> > > > By skipping 4 bytes execution will resume in the middle of the next insn
> > > > (the jump around the header).
> > > 
> > > This is in a non-C environment -- "!misa_extension('C')", so we're not
> > > jumping into the middle of the next insn.
> > 
> > But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
> ???????????????????????????????????????????????? ^^^^ My very own! :-D
> 
> > whose first 16-bit parcel is the same as c.li and whose second 16-bit
> > parcel is ignored. I really do not think this is a good idea.
> > 
> > > Note that the Linux kernel needs the change pointed out in this patch,
> > > to build w/o C.
> > 
> > That sounds like a Linux problem. Other OSes cope just fine. Fixing in
> > firmware is not the right approach; fix your software instead. Which
> > may mean abandoning an image format that isn?t fit for purpose. But the
> > solution isn?t to hack in whatever random crud Linux wants in firmware,
> > and other non-firmware SBI implementations like hypervisors.
> 
> There are other kernels that people care about?
> 
> > So this is a resounding NAK from me.
> 
> Jokes aside; Fair enough! I'll go back to the drawing board.

The kernel cannot handle this problem because the kernel has not initialized
stvec at this time. And the MZ cannot be changed because it is part of the PE
header. The best way is to skip the MZ of the PE header through the upper
bootloader. Therefore, when the kernel is used as the payload of opensbi,
skipped operations also need to be added. sbi_hart_switch_mode needs to add
some code to detect whether the current core supports C extension, whether
there is 0x5a4d5a4d at next_addr, and modify next_addr.

Regards,
Xiang W
> 
> Thank you for the input!
> 
> 
> Bj?rn
> 



From jrtc27 at jrtc27.com  Wed Nov  8 22:24:14 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Thu, 9 Nov 2023 06:24:14 +0000
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13
 instruction
In-Reply-To: <9411269e9975e9ececae2755eaf74271de45b8c0.camel@126.com>
References: <20231108191310.49950-1-bjorn@kernel.org>
 <874jhvc3ll.fsf@igel.home>
 <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>
 <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>
 <CAJ+HfNh_bYn6rCdCyveZybyQFDa7CU3nPCZzBKW-rvf4J7ft6g@mail.gmail.com>
 <9411269e9975e9ececae2755eaf74271de45b8c0.camel@126.com>
Message-ID: <8C375D8E-811D-45F2-B879-FA508A34AB54@jrtc27.com>

On 9 Nov 2023, at 04:04, Xiang W <wxjstz at 126.com> wrote:
> 
> ? 2023-11-09???? 01:22 +0100?Bj?rn T?pel???
>> On Thu, 9 Nov 2023 at 01:11, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
>>> 
>>> On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
>>>> 
>>>> Hi Andreas!
>>>> 
>>>> On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
>>>>> 
>>>>> On Nov 08 2023, Bj?rn T?pel wrote:
>>>>> 
>>>>>> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
>>>>>> +{
>>>>>> +     /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
>>>>>> +     if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
>>>>>> +             regs->s4 = -13;
>>>>>> +             regs->mepc += 4;
>>>>> 
>>>>> By skipping 4 bytes execution will resume in the middle of the next insn
>>>>> (the jump around the header).
>>>> 
>>>> This is in a non-C environment -- "!misa_extension('C')", so we're not
>>>> jumping into the middle of the next insn.
>>> 
>>> But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
>>                                                  ^^^^ My very own! :-D
>> 
>>> whose first 16-bit parcel is the same as c.li and whose second 16-bit
>>> parcel is ignored. I really do not think this is a good idea.
>>> 
>>>> Note that the Linux kernel needs the change pointed out in this patch,
>>>> to build w/o C.
>>> 
>>> That sounds like a Linux problem. Other OSes cope just fine. Fixing in
>>> firmware is not the right approach; fix your software instead. Which
>>> may mean abandoning an image format that isn?t fit for purpose. But the
>>> solution isn?t to hack in whatever random crud Linux wants in firmware,
>>> and other non-firmware SBI implementations like hypervisors.
>> 
>> There are other kernels that people care about?
>> 
>>> So this is a resounding NAK from me.
>> 
>> Jokes aside; Fair enough! I'll go back to the drawing board.
> 
> The kernel cannot handle this problem because the kernel has not initialized
> stvec at this time. And the MZ cannot be changed because it is part of the PE
> header. The best way is to skip the MZ of the PE header through the upper
> bootloader. Therefore, when the kernel is used as the payload of opensbi,
> skipped operations also need to be added. sbi_hart_switch_mode needs to add
> some code to detect whether the current core supports C extension, whether
> there is 0x5a4d5a4d at next_addr, and modify next_addr.

That?s also extremely ugly; now you?re just skipping instructions in
the payload rather than trapping like the current de-facto spec would
mandate. And what would you skip? Again, you can?t just skip 2 bytes,
you would have to skip a whole 4 bytes, which is more than just the
c.li instruction. And what happens if a vendor, not implementing C,
wants to use that encoding space? For example, Qualcomm, who are
proposing to ditch C entirely. It may not conform to a standard
profile, but it is a thing that SBI must support.

Linux is abusing file formats and trying to create a polyglot where
it?s not possible. That?s Linux?s problem, and it needs to stop
assuming things that aren?t true. This grotesque approach is not
present in other OSes.

Jess



From wxjstz at 126.com  Wed Nov  8 22:55:20 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 09 Nov 2023 14:55:20 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13
 instruction
In-Reply-To: <8C375D8E-811D-45F2-B879-FA508A34AB54@jrtc27.com>
References: <20231108191310.49950-1-bjorn@kernel.org>
	 <874jhvc3ll.fsf@igel.home>
	 <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>
	 <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>
	 <CAJ+HfNh_bYn6rCdCyveZybyQFDa7CU3nPCZzBKW-rvf4J7ft6g@mail.gmail.com>
	 <9411269e9975e9ececae2755eaf74271de45b8c0.camel@126.com>
	 <8C375D8E-811D-45F2-B879-FA508A34AB54@jrtc27.com>
Message-ID: <fd13341f086cf26f4c1cd4fb2b681365914e7781.camel@126.com>

? 2023-11-09???? 06:24 +0000?Jessica Clarke???
> On 9 Nov 2023, at 04:04, Xiang W <wxjstz at 126.com> wrote:
> > 
> > ? 2023-11-09???? 01:22 +0100?Bj?rn T?pel???
> > > On Thu, 9 Nov 2023 at 01:11, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
> > > > 
> > > > On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
> > > > > 
> > > > > Hi Andreas!
> > > > > 
> > > > > On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
> > > > > > 
> > > > > > On Nov 08 2023, Bj?rn T?pel wrote:
> > > > > > 
> > > > > > > +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
> > > > > > > +{
> > > > > > > +???? /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
> > > > > > > +???? if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
> > > > > > > +???????????? regs->s4 = -13;
> > > > > > > +???????????? regs->mepc += 4;
> > > > > > 
> > > > > > By skipping 4 bytes execution will resume in the middle of the next insn
> > > > > > (the jump around the header).
> > > > > 
> > > > > This is in a non-C environment -- "!misa_extension('C')", so we're not
> > > > > jumping into the middle of the next insn.
> > > > 
> > > > But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
> > > ???????????????????????????????????????????????? ^^^^ My very own! :-D
> > > 
> > > > whose first 16-bit parcel is the same as c.li and whose second 16-bit
> > > > parcel is ignored. I really do not think this is a good idea.
> > > > 
> > > > > Note that the Linux kernel needs the change pointed out in this patch,
> > > > > to build w/o C.
> > > > 
> > > > That sounds like a Linux problem. Other OSes cope just fine. Fixing in
> > > > firmware is not the right approach; fix your software instead. Which
> > > > may mean abandoning an image format that isn?t fit for purpose. But the
> > > > solution isn?t to hack in whatever random crud Linux wants in firmware,
> > > > and other non-firmware SBI implementations like hypervisors.
> > > 
> > > There are other kernels that people care about?
> > > 
> > > > So this is a resounding NAK from me.
> > > 
> > > Jokes aside; Fair enough! I'll go back to the drawing board.
> > 
> > The kernel cannot handle this problem because the kernel has not initialized
> > stvec at this time. And the MZ cannot be changed because it is part of the PE
> > header. The best way is to skip the MZ of the PE header through the upper
> > bootloader. Therefore, when the kernel is used as the payload of opensbi,
> > skipped operations also need to be added. sbi_hart_switch_mode needs to add
> > some code to detect whether the current core supports C extension, whether
> > there is 0x5a4d5a4d at next_addr, and modify next_addr.
> 
> That?s also extremely ugly; now you?re just skipping instructions in
> the payload rather than trapping like the current de-facto spec would
> mandate. And what would you skip? Again, you can?t just skip 2 bytes,
> you would have to skip a whole 4 bytes, which is more than just the
> c.li instruction. And what happens if a vendor, not implementing C,
> wants to use that encoding space? For example, Qualcomm, who are
> proposing to ditch C entirely. It may not conform to a standard
> profile, but it is a thing that SBI must support.
> 
> Linux is abusing file formats and trying to create a polyglot where
> it?s not possible. That?s Linux?s problem, and it needs to stop
> assuming things that aren?t true. This grotesque approach is not
> present in other OSes.
How do other operating systems solve it? This is a problem with the uefi
image file format. Can we remove dos header?

Regards,
Xiang W
> 
> Jess



From jrtc27 at jrtc27.com  Wed Nov  8 23:00:24 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Thu, 9 Nov 2023 07:00:24 +0000
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13
 instruction
In-Reply-To: <fd13341f086cf26f4c1cd4fb2b681365914e7781.camel@126.com>
References: <20231108191310.49950-1-bjorn@kernel.org>
 <874jhvc3ll.fsf@igel.home>
 <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>
 <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>
 <CAJ+HfNh_bYn6rCdCyveZybyQFDa7CU3nPCZzBKW-rvf4J7ft6g@mail.gmail.com>
 <9411269e9975e9ececae2755eaf74271de45b8c0.camel@126.com>
 <8C375D8E-811D-45F2-B879-FA508A34AB54@jrtc27.com>
 <fd13341f086cf26f4c1cd4fb2b681365914e7781.camel@126.com>
Message-ID: <A1818050-BD38-4B8C-97D8-129AC737A7D7@jrtc27.com>

On 9 Nov 2023, at 06:55, Xiang W <wxjstz at 126.com> wrote:
> ? 2023-11-09???? 06:24 +0000?Jessica Clarke???
>> On 9 Nov 2023, at 04:04, Xiang W <wxjstz at 126.com> wrote:
>>> 
>>> ? 2023-11-09???? 01:22 +0100?Bj?rn T?pel???
>>>> On Thu, 9 Nov 2023 at 01:11, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
>>>>> 
>>>>> On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
>>>>>> 
>>>>>> Hi Andreas!
>>>>>> 
>>>>>> On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
>>>>>>> 
>>>>>>> On Nov 08 2023, Bj?rn T?pel wrote:
>>>>>>> 
>>>>>>>> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
>>>>>>>> +{
>>>>>>>> +     /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
>>>>>>>> +     if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
>>>>>>>> +             regs->s4 = -13;
>>>>>>>> +             regs->mepc += 4;
>>>>>>> 
>>>>>>> By skipping 4 bytes execution will resume in the middle of the next insn
>>>>>>> (the jump around the header).
>>>>>> 
>>>>>> This is in a non-C environment -- "!misa_extension('C')", so we're not
>>>>>> jumping into the middle of the next insn.
>>>>> 
>>>>> But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
>>>>                                                  ^^^^ My very own! :-D
>>>> 
>>>>> whose first 16-bit parcel is the same as c.li and whose second 16-bit
>>>>> parcel is ignored. I really do not think this is a good idea.
>>>>> 
>>>>>> Note that the Linux kernel needs the change pointed out in this patch,
>>>>>> to build w/o C.
>>>>> 
>>>>> That sounds like a Linux problem. Other OSes cope just fine. Fixing in
>>>>> firmware is not the right approach; fix your software instead. Which
>>>>> may mean abandoning an image format that isn?t fit for purpose. But the
>>>>> solution isn?t to hack in whatever random crud Linux wants in firmware,
>>>>> and other non-firmware SBI implementations like hypervisors.
>>>> 
>>>> There are other kernels that people care about?
>>>> 
>>>>> So this is a resounding NAK from me.
>>>> 
>>>> Jokes aside; Fair enough! I'll go back to the drawing board.
>>> 
>>> The kernel cannot handle this problem because the kernel has not initialized
>>> stvec at this time. And the MZ cannot be changed because it is part of the PE
>>> header. The best way is to skip the MZ of the PE header through the upper
>>> bootloader. Therefore, when the kernel is used as the payload of opensbi,
>>> skipped operations also need to be added. sbi_hart_switch_mode needs to add
>>> some code to detect whether the current core supports C extension, whether
>>> there is 0x5a4d5a4d at next_addr, and modify next_addr.
>> 
>> That?s also extremely ugly; now you?re just skipping instructions in
>> the payload rather than trapping like the current de-facto spec would
>> mandate. And what would you skip? Again, you can?t just skip 2 bytes,
>> you would have to skip a whole 4 bytes, which is more than just the
>> c.li instruction. And what happens if a vendor, not implementing C,
>> wants to use that encoding space? For example, Qualcomm, who are
>> proposing to ditch C entirely. It may not conform to a standard
>> profile, but it is a thing that SBI must support.
>> 
>> Linux is abusing file formats and trying to create a polyglot where
>> it?s not possible. That?s Linux?s problem, and it needs to stop
>> assuming things that aren?t true. This grotesque approach is not
>> present in other OSes.
> How do other operating systems solve it? This is a problem with the uefi
> image file format. Can we remove dos header?

They don?t delude themselves that you can just jump to the start of a
UEFI PE/COFF executable and interpret it as machine code.

Jess



From wxjstz at 126.com  Wed Nov  8 23:10:30 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 09 Nov 2023 15:10:30 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13
 instruction
In-Reply-To: <A1818050-BD38-4B8C-97D8-129AC737A7D7@jrtc27.com>
References: <20231108191310.49950-1-bjorn@kernel.org>
	 <874jhvc3ll.fsf@igel.home>
	 <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>
	 <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>
	 <CAJ+HfNh_bYn6rCdCyveZybyQFDa7CU3nPCZzBKW-rvf4J7ft6g@mail.gmail.com>
	 <9411269e9975e9ececae2755eaf74271de45b8c0.camel@126.com>
	 <8C375D8E-811D-45F2-B879-FA508A34AB54@jrtc27.com>
	 <fd13341f086cf26f4c1cd4fb2b681365914e7781.camel@126.com>
	 <A1818050-BD38-4B8C-97D8-129AC737A7D7@jrtc27.com>
Message-ID: <808d6234494a600a1f6a042da1da64ff0fc45ec8.camel@126.com>

? 2023-11-09???? 07:00 +0000?Jessica Clarke???
> On 9 Nov 2023, at 06:55, Xiang W <wxjstz at 126.com> wrote:
> > ? 2023-11-09???? 06:24 +0000?Jessica Clarke???
> > > On 9 Nov 2023, at 04:04, Xiang W <wxjstz at 126.com> wrote:
> > > > 
> > > > ? 2023-11-09???? 01:22 +0100?Bj?rn T?pel???
> > > > > On Thu, 9 Nov 2023 at 01:11, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
> > > > > > 
> > > > > > On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
> > > > > > > 
> > > > > > > Hi Andreas!
> > > > > > > 
> > > > > > > On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
> > > > > > > > 
> > > > > > > > On Nov 08 2023, Bj?rn T?pel wrote:
> > > > > > > > 
> > > > > > > > > +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
> > > > > > > > > +{
> > > > > > > > > +???? /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
> > > > > > > > > +???? if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
> > > > > > > > > +???????????? regs->s4 = -13;
> > > > > > > > > +???????????? regs->mepc += 4;
> > > > > > > > 
> > > > > > > > By skipping 4 bytes execution will resume in the middle of the next insn
> > > > > > > > (the jump around the header).
> > > > > > > 
> > > > > > > This is in a non-C environment -- "!misa_extension('C')", so we're not
> > > > > > > jumping into the middle of the next insn.
> > > > > > 
> > > > > > But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
> > > > > ???????????????????????????????????????????????? ^^^^ My very own! :-D
> > > > > 
> > > > > > whose first 16-bit parcel is the same as c.li and whose second 16-bit
> > > > > > parcel is ignored. I really do not think this is a good idea.
> > > > > > 
> > > > > > > Note that the Linux kernel needs the change pointed out in this patch,
> > > > > > > to build w/o C.
> > > > > > 
> > > > > > That sounds like a Linux problem. Other OSes cope just fine. Fixing in
> > > > > > firmware is not the right approach; fix your software instead. Which
> > > > > > may mean abandoning an image format that isn?t fit for purpose. But the
> > > > > > solution isn?t to hack in whatever random crud Linux wants in firmware,
> > > > > > and other non-firmware SBI implementations like hypervisors.
> > > > > 
> > > > > There are other kernels that people care about?
> > > > > 
> > > > > > So this is a resounding NAK from me.
> > > > > 
> > > > > Jokes aside; Fair enough! I'll go back to the drawing board.
> > > > 
> > > > The kernel cannot handle this problem because the kernel has not initialized
> > > > stvec at this time. And the MZ cannot be changed because it is part of the PE
> > > > header. The best way is to skip the MZ of the PE header through the upper
> > > > bootloader. Therefore, when the kernel is used as the payload of opensbi,
> > > > skipped operations also need to be added. sbi_hart_switch_mode needs to add
> > > > some code to detect whether the current core supports C extension, whether
> > > > there is 0x5a4d5a4d at next_addr, and modify next_addr.
> > > 
> > > That?s also extremely ugly; now you?re just skipping instructions in
> > > the payload rather than trapping like the current de-facto spec would
> > > mandate. And what would you skip? Again, you can?t just skip 2 bytes,
> > > you would have to skip a whole 4 bytes, which is more than just the
> > > c.li instruction. And what happens if a vendor, not implementing C,
> > > wants to use that encoding space? For example, Qualcomm, who are
> > > proposing to ditch C entirely. It may not conform to a standard
> > > profile, but it is a thing that SBI must support.
> > > 
> > > Linux is abusing file formats and trying to create a polyglot where
> > > it?s not possible. That?s Linux?s problem, and it needs to stop
> > > assuming things that aren?t true. This grotesque approach is not
> > > present in other OSes.
> > How do other operating systems solve it? This is a problem with the uefi
> > image file format. Can we remove dos header?
> 
> They don?t delude themselves that you can just jump to the start of a
> UEFI PE/COFF executable and interpret it as machine code.
Show me code.

Thx,
Xiang W
> 
> Jess



From jrtc27 at jrtc27.com  Wed Nov  8 23:19:19 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Thu, 9 Nov 2023 07:19:19 +0000
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13
 instruction
In-Reply-To: <808d6234494a600a1f6a042da1da64ff0fc45ec8.camel@126.com>
References: <20231108191310.49950-1-bjorn@kernel.org>
 <874jhvc3ll.fsf@igel.home>
 <CAJ+HfNgzOErURt34Orzn+8RafgsvHRpptPsJ+F6zx-6mufDSGQ@mail.gmail.com>
 <5F1BB663-A00C-4855-95D9-0F0C99E75C5D@jrtc27.com>
 <CAJ+HfNh_bYn6rCdCyveZybyQFDa7CU3nPCZzBKW-rvf4J7ft6g@mail.gmail.com>
 <9411269e9975e9ececae2755eaf74271de45b8c0.camel@126.com>
 <8C375D8E-811D-45F2-B879-FA508A34AB54@jrtc27.com>
 <fd13341f086cf26f4c1cd4fb2b681365914e7781.camel@126.com>
 <A1818050-BD38-4B8C-97D8-129AC737A7D7@jrtc27.com>
 <808d6234494a600a1f6a042da1da64ff0fc45ec8.camel@126.com>
Message-ID: <DADA653C-2A45-47AA-AB4F-BFFD3954FF6C@jrtc27.com>

On 9 Nov 2023, at 07:10, Xiang W <wxjstz at 126.com> wrote:
> ? 2023-11-09???? 07:00 +0000?Jessica Clarke???
>> On 9 Nov 2023, at 06:55, Xiang W <wxjstz at 126.com> wrote:
>>> ? 2023-11-09???? 06:24 +0000?Jessica Clarke???
>>>> On 9 Nov 2023, at 04:04, Xiang W <wxjstz at 126.com> wrote:
>>>>> 
>>>>> ? 2023-11-09???? 01:22 +0100?Bj?rn T?pel???
>>>>>> On Thu, 9 Nov 2023 at 01:11, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
>>>>>>> 
>>>>>>> On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
>>>>>>>> 
>>>>>>>> Hi Andreas!
>>>>>>>> 
>>>>>>>> On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
>>>>>>>>> 
>>>>>>>>> On Nov 08 2023, Bj?rn T?pel wrote:
>>>>>>>>> 
>>>>>>>>>> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
>>>>>>>>>> +{
>>>>>>>>>> +     /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
>>>>>>>>>> +     if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
>>>>>>>>>> +             regs->s4 = -13;
>>>>>>>>>> +             regs->mepc += 4;
>>>>>>>>> 
>>>>>>>>> By skipping 4 bytes execution will resume in the middle of the next insn
>>>>>>>>> (the jump around the header).
>>>>>>>> 
>>>>>>>> This is in a non-C environment -- "!misa_extension('C')", so we're not
>>>>>>>> jumping into the middle of the next insn.
>>>>>>> 
>>>>>>> But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
>>>>>>                                                  ^^^^ My very own! :-D
>>>>>> 
>>>>>>> whose first 16-bit parcel is the same as c.li and whose second 16-bit
>>>>>>> parcel is ignored. I really do not think this is a good idea.
>>>>>>> 
>>>>>>>> Note that the Linux kernel needs the change pointed out in this patch,
>>>>>>>> to build w/o C.
>>>>>>> 
>>>>>>> That sounds like a Linux problem. Other OSes cope just fine. Fixing in
>>>>>>> firmware is not the right approach; fix your software instead. Which
>>>>>>> may mean abandoning an image format that isn?t fit for purpose. But the
>>>>>>> solution isn?t to hack in whatever random crud Linux wants in firmware,
>>>>>>> and other non-firmware SBI implementations like hypervisors.
>>>>>> 
>>>>>> There are other kernels that people care about?
>>>>>> 
>>>>>>> So this is a resounding NAK from me.
>>>>>> 
>>>>>> Jokes aside; Fair enough! I'll go back to the drawing board.
>>>>> 
>>>>> The kernel cannot handle this problem because the kernel has not initialized
>>>>> stvec at this time. And the MZ cannot be changed because it is part of the PE
>>>>> header. The best way is to skip the MZ of the PE header through the upper
>>>>> bootloader. Therefore, when the kernel is used as the payload of opensbi,
>>>>> skipped operations also need to be added. sbi_hart_switch_mode needs to add
>>>>> some code to detect whether the current core supports C extension, whether
>>>>> there is 0x5a4d5a4d at next_addr, and modify next_addr.
>>>> 
>>>> That?s also extremely ugly; now you?re just skipping instructions in
>>>> the payload rather than trapping like the current de-facto spec would
>>>> mandate. And what would you skip? Again, you can?t just skip 2 bytes,
>>>> you would have to skip a whole 4 bytes, which is more than just the
>>>> c.li instruction. And what happens if a vendor, not implementing C,
>>>> wants to use that encoding space? For example, Qualcomm, who are
>>>> proposing to ditch C entirely. It may not conform to a standard
>>>> profile, but it is a thing that SBI must support.
>>>> 
>>>> Linux is abusing file formats and trying to create a polyglot where
>>>> it?s not possible. That?s Linux?s problem, and it needs to stop
>>>> assuming things that aren?t true. This grotesque approach is not
>>>> present in other OSes.
>>> How do other operating systems solve it? This is a problem with the uefi
>>> image file format. Can we remove dos header?
>> 
>> They don?t delude themselves that you can just jump to the start of a
>> UEFI PE/COFF executable and interpret it as machine code.
> Show me code.

They are separate binaries, one for UEFI PE/COFF and one for direct SBI
booting if you want it. Unless you introduce a new image format for SBI
payloads there is no way around that without inappropriate hacks like
these suggestions. So there is no magic code. The answer is, like I?ve
been saying, to just not, because it?s not possible.

Jess



From inochiama at outlook.com  Wed Nov  8 23:54:21 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu,  9 Nov 2023 15:54:21 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate 'MZ'/c.li s4,-13 instruction
In-Reply-To: <DADA653C-2A45-47AA-AB4F-BFFD3954FF6C@jrtc27.com>
References: <DADA653C-2A45-47AA-AB4F-BFFD3954FF6C@jrtc27.com>
Message-ID: <IA1PR20MB4953F7E940FFC38799B7C0E8BBAFA@IA1PR20MB4953.namprd20.prod.outlook.com>

>On 9 Nov 2023, at 07:10, Xiang W <wxjstz at 126.com> wrote:
>> ? 2023-11-09???? 07:00 +0000?Jessica Clarke???
>>> On 9 Nov 2023, at 06:55, Xiang W <wxjstz at 126.com> wrote:
>>>> ? 2023-11-09???? 06:24 +0000?Jessica Clarke???
>>>>> On 9 Nov 2023, at 04:04, Xiang W <wxjstz at 126.com> wrote:
>>>>>>
>>>>>> ? 2023-11-09???? 01:22 +0100?Bj?rn T?pel???
>>>>>>> On Thu, 9 Nov 2023 at 01:11, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
>>>>>>>>
>>>>>>>> On 9 Nov 2023, at 00:02, Bj?rn T?pel <bjorn at kernel.org> wrote:
>>>>>>>>>
>>>>>>>>> Hi Andreas!
>>>>>>>>>
>>>>>>>>> On Wed, 8 Nov 2023 at 22:55, Andreas Schwab <schwab at linux-m68k.org> wrote:
>>>>>>>>>>
>>>>>>>>>> On Nov 08 2023, Bj?rn T?pel wrote:
>>>>>>>>>>
>>>>>>>>>>> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs)
>>>>>>>>>>> +{
>>>>>>>>>>> +     /* Only handle 'MZ'/c.li s4,-13/0x5a3d */
>>>>>>>>>>> +     if (!misa_extension('C') && (insn & 0xffff) == 0x5a4d) {
>>>>>>>>>>> +             regs->s4 = -13;
>>>>>>>>>>> +             regs->mepc += 4;
>>>>>>>>>>
>>>>>>>>>> By skipping 4 bytes execution will resume in the middle of the next insn
>>>>>>>>>> (the jump around the header).
>>>>>>>>>
>>>>>>>>> This is in a non-C environment -- "!misa_extension('C')", so we're not
>>>>>>>>> jumping into the middle of the next insn.
>>>>>>>>
>>>>>>>> But that?s not c.li, is it. That?s bjorn.c.li, a 32-bit instruction
>>>>>>>                                                  ^^^^ My very own! ?
>>>>>>>
>>>>>>>> whose first 16-bit parcel is the same as c.li and whose second 16-bit
>>>>>>>> parcel is ignored. I really do not think this is a good idea.
>>>>>>>>
>>>>>>>>> Note that the Linux kernel needs the change pointed out in this patch,
>>>>>>>>> to build w/o C.
>>>>>>>>
>>>>>>>> That sounds like a Linux problem. Other OSes cope just fine. Fixing in
>>>>>>>> firmware is not the right approach; fix your software instead. Which
>>>>>>>> may mean abandoning an image format that isn?t fit for purpose. But the
>>>>>>>> solution isn?t to hack in whatever random crud Linux wants in firmware,
>>>>>>>> and other non-firmware SBI implementations like hypervisors.
>>>>>>>
>>>>>>> There are other kernels that people care about?
>>>>>>>
>>>>>>>> So this is a resounding NAK from me.
>>>>>>>
>>>>>>> Jokes aside; Fair enough! I'll go back to the drawing board.
>>>>>>
>>>>>> The kernel cannot handle this problem because the kernel has not initialized
>>>>>> stvec at this time. And the MZ cannot be changed because it is part of the PE
>>>>>> header. The best way is to skip the MZ of the PE header through the upper
>>>>>> bootloader. Therefore, when the kernel is used as the payload of opensbi,
>>>>>> skipped operations also need to be added. sbi_hart_switch_mode needs to add
>>>>>> some code to detect whether the current core supports C extension, whether
>>>>>> there is 0x5a4d5a4d at next_addr, and modify next_addr.
>>>>>
>>>>> That?s also extremely ugly; now you?re just skipping instructions in
>>>>> the payload rather than trapping like the current de-facto spec would
>>>>> mandate. And what would you skip? Again, you can?t just skip 2 bytes,
>>>>> you would have to skip a whole 4 bytes, which is more than just the
>>>>> c.li instruction. And what happens if a vendor, not implementing C,
>>>>> wants to use that encoding space? For example, Qualcomm, who are
>>>>> proposing to ditch C entirely. It may not conform to a standard
>>>>> profile, but it is a thing that SBI must support.
>>>>>
>>>>> Linux is abusing file formats and trying to create a polyglot where
>>>>> it?s not possible. That?s Linux?s problem, and it needs to stop
>>>>> assuming things that aren?t true. This grotesque approach is not
>>>>> present in other OSes.
>>>> How do other operating systems solve it? This is a problem with the uefi
>>>> image file format. Can we remove dos header?
>>>
>>> They don?t delude themselves that you can just jump to the start of a
>>> UEFI PE/COFF executable and interpret it as machine code.
>> Show me code.
>
>They are separate binaries, one for UEFI PE/COFF and one for direct SBI
>booting if you want it. Unless you introduce a new image format for SBI
>payloads there is no way around that without inappropriate hacks like
>these suggestions. So there is no magic code. The answer is, like I?ve
>been saying, to just not, because it?s not possible.
>
>Jess
>

As the UEFI header can be vaild code, it can be the first inst of bare
program. Skipping it may cause some bare program not work.
In fact, this is a problem releated to the UEFI, not SBI.


From wxjstz at 126.com  Thu Nov  9 00:25:17 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu,  9 Nov 2023 16:25:17 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate c.li
Message-ID: <20231109082540.52144-1-wxjstz@126.com>

The Linux kernel RISC-V image format allows that UEFI Images can also
be booted by non-UEFI firmware. However for that to work, the PE/Image
combo requires that 'MZ' is a valid instruction. On RISC-V, 'MZ' is
only a valid instruction if the hardware is C capable [1]. So add
Emulate c.li

Signed-off-by: Bj?rn T?pel <bjorn at rivosinc.com>
Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_illegal_insn.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
index 2be4757..4ab10f4 100644
--- a/lib/sbi/sbi_illegal_insn.c
+++ b/lib/sbi/sbi_illegal_insn.c
@@ -102,6 +102,22 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
 	return 0;
 }
 
+static int compressed_insn(ulong insn, struct sbi_trap_regs *regs) {
+	unsigned long imm, rd;
+	unsigned long *regs_p = (unsigned long *)regs;
+
+	if ((insn & 0xe003) == 0x4001) { /* c.li */
+		imm = (insn >> 2) & 0x1f;
+		imm |= ((insn >> 12) & 1) ? -32 : 0;
+		rd = (insn >> 7) & 0x1f;
+		if (rd)
+			regs_p[rd] = imm;
+		return 0;
+	}
+
+	return truly_illegal_insn(insn, regs);
+}
+
 static const illegal_insn_func illegal_insn_table[32] = {
 	truly_illegal_insn, /* 0 */
 	truly_illegal_insn, /* 1 */
@@ -160,7 +176,7 @@ int sbi_illegal_insn_handler(ulong insn, struct sbi_trap_regs *regs)
 			return sbi_trap_redirect(regs, &uptrap);
 		}
 		if ((insn & 3) != 3)
-			return truly_illegal_insn(insn, regs);
+			return compressed_insn(insn, regs);
 	}
 
 	return illegal_insn_table[(insn & 0x7c) >> 2](insn, regs);
-- 
2.42.0



From inochiama at outlook.com  Thu Nov  9 01:35:18 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu,  9 Nov 2023 17:35:18 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate c.li
In-Reply-To: <20231109082540.52144-1-wxjstz@126.com>
References: <20231109082540.52144-1-wxjstz@126.com>
Message-ID: <IA1PR20MB4953CD29AD157825B54D1014BBAFA@IA1PR20MB4953.namprd20.prod.outlook.com>

>The Linux kernel RISC-V image format allows that UEFI Images can also
>be booted by non-UEFI firmware. However for that to work, the PE/Image
>combo requires that 'MZ' is a valid instruction. On RISC-V, 'MZ' is
>only a valid instruction if the hardware is C capable [1]. So add
>Emulate c.li
>
>Signed-off-by: Bj?rn T?pel <bjorn at rivosinc.com>
>Signed-off-by: Xiang W <wxjstz at 126.com>
>---
> lib/sbi/sbi_illegal_insn.c | 18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
>
>diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
>index 2be4757..4ab10f4 100644
>--- a/lib/sbi/sbi_illegal_insn.c
>+++ b/lib/sbi/sbi_illegal_insn.c
>@@ -102,6 +102,22 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
> 	return 0;
> }
>
>+static int compressed_insn(ulong insn, struct sbi_trap_regs *regs) {
>+	unsigned long imm, rd;
>+	unsigned long *regs_p = (unsigned long *)regs;
>+
>+	if ((insn & 0xe003) == 0x4001) { /* c.li */
>+		imm = (insn >> 2) & 0x1f;
>+		imm |= ((insn >> 12) & 1) ? -32 : 0;
>+		rd = (insn >> 7) & 0x1f;
>+		if (rd)
>+			regs_p[rd] = imm;
>+		return 0;
>+	}
>+
>+	return truly_illegal_insn(insn, regs);
>+}
>+
> static const illegal_insn_func illegal_insn_table[32] = {
> 	truly_illegal_insn, /* 0 */
> 	truly_illegal_insn, /* 1 */
>@@ -160,7 +176,7 @@ int sbi_illegal_insn_handler(ulong insn, struct sbi_trap_regs *regs)
> 			return sbi_trap_redirect(regs, &uptrap);
> 		}
> 		if ((insn & 3) != 3)
>-			return truly_illegal_insn(insn, regs);
>+			return compressed_insn(insn, regs);
> 	}
>
> 	return illegal_insn_table[(insn & 0x7c) >> 2](insn, regs);
>--
>2.42.0
>

I think this patch may be unnecessary.

After searching the boot flow of UEFI, I only found two implementations:

For QEMU RISC-V virt platform and SG2042 platform:
SBI -> (SEC -> PEI -> DXE) -> bootloader -> OS
               UEFI

For SiFive U540 platform:
(SEC -> OpenSBI -> PEI -> DXE) -> bootloader -> OS
             UEFI

Both invoke SBI at a very early stage and use raw vaild address, and
handling the UEFI header is more like bootloader's job. I have no idea
about why this should be handled by SBI. FYI, Bj?rn T?pel.


From wxjstz at 126.com  Thu Nov  9 02:04:56 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 09 Nov 2023 18:04:56 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate c.li
In-Reply-To: <IA1PR20MB4953CD29AD157825B54D1014BBAFA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <20231109082540.52144-1-wxjstz@126.com>
	 <IA1PR20MB4953CD29AD157825B54D1014BBAFA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <ab3b8e8cb1c2bef8f75bf0becfd7effffd372b2b.camel@126.com>

? 2023-11-09???? 17:35 +0800?Inochi Amaoto???
> > The Linux kernel RISC-V image format allows that UEFI Images can also
> > be booted by non-UEFI firmware. However for that to work, the PE/Image
> > combo requires that 'MZ' is a valid instruction. On RISC-V, 'MZ' is
> > only a valid instruction if the hardware is C capable [1]. So add
> > Emulate c.li
> > 
> > Signed-off-by: Bj?rn T?pel <bjorn at rivosinc.com>
> > Signed-off-by: Xiang W <wxjstz at 126.com>
> > ---
> > lib/sbi/sbi_illegal_insn.c | 18 +++++++++++++++++-
> > 1 file changed, 17 insertions(+), 1 deletion(-)
> > 
> > diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
> > index 2be4757..4ab10f4 100644
> > --- a/lib/sbi/sbi_illegal_insn.c
> > +++ b/lib/sbi/sbi_illegal_insn.c
> > @@ -102,6 +102,22 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
> > 	return 0;
> > }
> > 
> > +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs) {
> > +	unsigned long imm, rd;
> > +	unsigned long *regs_p = (unsigned long *)regs;
> > +
> > +	if ((insn & 0xe003) == 0x4001) { /* c.li */
> > +		imm = (insn >> 2) & 0x1f;
> > +		imm |= ((insn >> 12) & 1) ? -32 : 0;
> > +		rd = (insn >> 7) & 0x1f;
> > +		if (rd)
> > +			regs_p[rd] = imm;
> > +		return 0;
> > +	}
> > +
> > +	return truly_illegal_insn(insn, regs);
> > +}
> > +
> > static const illegal_insn_func illegal_insn_table[32] = {
> > 	truly_illegal_insn, /* 0 */
> > 	truly_illegal_insn, /* 1 */
> > @@ -160,7 +176,7 @@ int sbi_illegal_insn_handler(ulong insn, struct sbi_trap_regs *regs)
> > 			return sbi_trap_redirect(regs, &uptrap);
> > 		}
> > 		if ((insn & 3) != 3)
> > -			return truly_illegal_insn(insn, regs);
> > +			return compressed_insn(insn, regs);
> > 	}
> > 
> > 	return illegal_insn_table[(insn & 0x7c) >> 2](insn, regs);
> > --
> > 2.42.0
> > 
> 
> I think this patch may be unnecessary.
> 
> After searching the boot flow of UEFI, I only found two implementations:
> 
> For QEMU RISC-V virt platform and SG2042 platform:
> SBI -> (SEC -> PEI -> DXE) -> bootloader -> OS
> ?????????????? UEFI
> 
> For SiFive U540 platform:
> (SEC -> OpenSBI -> PEI -> DXE) -> bootloader -> OS
> ???????????? UEFI
> 
> Both invoke SBI at a very early stage and use raw vaild address, and
> handling the UEFI header is more like bootloader's job. I have no idea
> about why this should be handled by SBI. FYI, Bj?rn T?pel.
This patch is used to solve the problem of non-UEFI firmware booting the
kernel. The UEFI boot kernel should parse the PE file header, so this
patch is not needed. But non-UEFI probably treats the kernel as a
binary image.

Regards,
Xiang W



From inochiama at outlook.com  Thu Nov  9 03:20:58 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu,  9 Nov 2023 19:20:58 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate c.li
In-Reply-To: <ab3b8e8cb1c2bef8f75bf0becfd7effffd372b2b.camel@126.com>
References: <ab3b8e8cb1c2bef8f75bf0becfd7effffd372b2b.camel@126.com>
Message-ID: <IA1PR20MB495332C748AA9B8063D47F6BBBAFA@IA1PR20MB4953.namprd20.prod.outlook.com>

>? 2023-11-09???? 17:35 +0800?Inochi Amaoto???
>>> The Linux kernel RISC-V image format allows that UEFI Images can also
>>> be booted by non-UEFI firmware. However for that to work, the PE/Image
>>> combo requires that 'MZ' is a valid instruction. On RISC-V, 'MZ' is
>>> only a valid instruction if the hardware is C capable [1]. So add
>>> Emulate c.li
>>>
>>> Signed-off-by: Bj?rn T?pel <bjorn at rivosinc.com>
>>> Signed-off-by: Xiang W <wxjstz at 126.com>
>>> ---
>>> lib/sbi/sbi_illegal_insn.c | 18 +++++++++++++++++-
>>> 1 file changed, 17 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
>>> index 2be4757..4ab10f4 100644
>>> --- a/lib/sbi/sbi_illegal_insn.c
>>> +++ b/lib/sbi/sbi_illegal_insn.c
>>> @@ -102,6 +102,22 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
>>> 	return 0;
>>> }
>>>
>>> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs) {
>>> +	unsigned long imm, rd;
>>> +	unsigned long *regs_p = (unsigned long *)regs;
>>> +
>>> +	if ((insn & 0xe003) == 0x4001) { /* c.li */
>>> +		imm = (insn >> 2) & 0x1f;
>>> +		imm |= ((insn >> 12) & 1) ? -32 : 0;
>>> +		rd = (insn >> 7) & 0x1f;
>>> +		if (rd)
>>> +			regs_p[rd] = imm;
>>> +		return 0;
>>> +	}
>>> +
>>> +	return truly_illegal_insn(insn, regs);
>>> +}
>>> +
>>> static const illegal_insn_func illegal_insn_table[32] = {
>>> 	truly_illegal_insn, /* 0 */
>>> 	truly_illegal_insn, /* 1 */
>>> @@ -160,7 +176,7 @@ int sbi_illegal_insn_handler(ulong insn, struct sbi_trap_regs *regs)
>>> 			return sbi_trap_redirect(regs, &uptrap);
>>> 		}
>>> 		if ((insn & 3) != 3)
>>> -			return truly_illegal_insn(insn, regs);
>>> +			return compressed_insn(insn, regs);
>>> 	}
>>>
>>> 	return illegal_insn_table[(insn & 0x7c) >> 2](insn, regs);
>>> --
>>> 2.42.0
>>>
>>
>> I think this patch may be unnecessary.
>>
>> After searching the boot flow of UEFI, I only found two implementations:
>>
>> For QEMU RISC-V virt platform and SG2042 platform:
>> SBI -> (SEC -> PEI -> DXE) -> bootloader -> OS
>>                UEFI
>>
>> For SiFive U540 platform:
>> (SEC -> OpenSBI -> PEI -> DXE) -> bootloader -> OS
>>              UEFI
>>
>> Both invoke SBI at a very early stage and use raw vaild address, and
>> handling the UEFI header is more like bootloader's job. I have no idea
>> about why this should be handled by SBI. FYI, Bj?rn T?pel.
>This patch is used to solve the problem of non-UEFI firmware booting the
>kernel. The UEFI boot kernel should parse the PE file header, so this
>patch is not needed. But non-UEFI probably treats the kernel as a
>binary image.

This is not a vaild scenario for UEFI binary. IIRC, even the aarch64 arch
does not support using UEFI binary in this way. I agree with Jessica about
the fact that they are different. If you insist, I suggest using a custom
bootloader to handle this.

>
>Regards,
>Xiang W
>
>


From wxjstz at 126.com  Thu Nov  9 03:53:23 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 09 Nov 2023 19:53:23 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate c.li
In-Reply-To: <IA1PR20MB495332C748AA9B8063D47F6BBBAFA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <ab3b8e8cb1c2bef8f75bf0becfd7effffd372b2b.camel@126.com>
	 <IA1PR20MB495332C748AA9B8063D47F6BBBAFA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <a02f7b6c392e48e905e701f635c0899addf94409.camel@126.com>

? 2023-11-09???? 19:20 +0800?Inochi Amaoto???
> > ? 2023-11-09???? 17:35 +0800?Inochi Amaoto???
> > > > The Linux kernel RISC-V image format allows that UEFI Images can also
> > > > be booted by non-UEFI firmware. However for that to work, the PE/Image
> > > > combo requires that 'MZ' is a valid instruction. On RISC-V, 'MZ' is
> > > > only a valid instruction if the hardware is C capable [1]. So add
> > > > Emulate c.li
> > > > 
> > > > Signed-off-by: Bj?rn T?pel <bjorn at rivosinc.com>
> > > > Signed-off-by: Xiang W <wxjstz at 126.com>
> > > > ---
> > > > lib/sbi/sbi_illegal_insn.c | 18 +++++++++++++++++-
> > > > 1 file changed, 17 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
> > > > index 2be4757..4ab10f4 100644
> > > > --- a/lib/sbi/sbi_illegal_insn.c
> > > > +++ b/lib/sbi/sbi_illegal_insn.c
> > > > @@ -102,6 +102,22 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
> > > > 	return 0;
> > > > }
> > > > 
> > > > +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs) {
> > > > +	unsigned long imm, rd;
> > > > +	unsigned long *regs_p = (unsigned long *)regs;
> > > > +
> > > > +	if ((insn & 0xe003) == 0x4001) { /* c.li */
> > > > +		imm = (insn >> 2) & 0x1f;
> > > > +		imm |= ((insn >> 12) & 1) ? -32 : 0;
> > > > +		rd = (insn >> 7) & 0x1f;
> > > > +		if (rd)
> > > > +			regs_p[rd] = imm;
> > > > +		return 0;
> > > > +	}
> > > > +
> > > > +	return truly_illegal_insn(insn, regs);
> > > > +}
> > > > +
> > > > static const illegal_insn_func illegal_insn_table[32] = {
> > > > 	truly_illegal_insn, /* 0 */
> > > > 	truly_illegal_insn, /* 1 */
> > > > @@ -160,7 +176,7 @@ int sbi_illegal_insn_handler(ulong insn, struct sbi_trap_regs *regs)
> > > > 			return sbi_trap_redirect(regs, &uptrap);
> > > > 		}
> > > > 		if ((insn & 3) != 3)
> > > > -			return truly_illegal_insn(insn, regs);
> > > > +			return compressed_insn(insn, regs);
> > > > 	}
> > > > 
> > > > 	return illegal_insn_table[(insn & 0x7c) >> 2](insn, regs);
> > > > --
> > > > 2.42.0
> > > > 
> > > 
> > > I think this patch may be unnecessary.
> > > 
> > > After searching the boot flow of UEFI, I only found two implementations:
> > > 
> > > For QEMU RISC-V virt platform and SG2042 platform:
> > > SBI -> (SEC -> PEI -> DXE) -> bootloader -> OS
> > > ?????????????? UEFI
> > > 
> > > For SiFive U540 platform:
> > > (SEC -> OpenSBI -> PEI -> DXE) -> bootloader -> OS
> > > ???????????? UEFI
> > > 
> > > Both invoke SBI at a very early stage and use raw vaild address, and
> > > handling the UEFI header is more like bootloader's job. I have no idea
> > > about why this should be handled by SBI. FYI, Bj?rn T?pel.
> > This patch is used to solve the problem of non-UEFI firmware booting the
> > kernel. The UEFI boot kernel should parse the PE file header, so this
> > patch is not needed. But non-UEFI probably treats the kernel as a
> > binary image.
> 
> This is not a vaild scenario for UEFI binary. IIRC, even the aarch64 arch
> does not support using UEFI binary in this way. I agree with Jessica about
> the fact that they are different. If you insist, I suggest using a custom
> bootloader to handle this.
Now this modification only adds an instruction to simulate and will not affect
the other programs.

Regards,
Xiang W
> 
> > 
> > Regards,
> > Xiang W
> > 
> > 



From bjorn at kernel.org  Thu Nov  9 06:48:47 2023
From: bjorn at kernel.org (=?UTF-8?B?QmrDtnJuIFTDtnBlbA==?=)
Date: Thu, 9 Nov 2023 15:48:47 +0100
Subject: [PATCH] lib: sbi_illegal_insn: Emulate c.li
In-Reply-To: <20231109082540.52144-1-wxjstz@126.com>
References: <20231109082540.52144-1-wxjstz@126.com>
Message-ID: <CAJ+HfNj3s+jkBdVC=9H8W7NZwvUiROL0Miyj5xXumLam861VbA@mail.gmail.com>

On Thu, 9 Nov 2023 at 09:26, Xiang W <wxjstz at 126.com> wrote:
>
> The Linux kernel RISC-V image format allows that UEFI Images can also
> be booted by non-UEFI firmware. However for that to work, the PE/Image
> combo requires that 'MZ' is a valid instruction. On RISC-V, 'MZ' is
> only a valid instruction if the hardware is C capable [1]. So add
> Emulate c.li
>
> Signed-off-by: Bj?rn T?pel <bjorn at rivosinc.com>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> ---
>  lib/sbi/sbi_illegal_insn.c | 18 +++++++++++++++++-
>  1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
> index 2be4757..4ab10f4 100644
> --- a/lib/sbi/sbi_illegal_insn.c
> +++ b/lib/sbi/sbi_illegal_insn.c
> @@ -102,6 +102,22 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
>         return 0;
>  }
>
> +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs) {
> +       unsigned long imm, rd;
> +       unsigned long *regs_p = (unsigned long *)regs;
> +
> +       if ((insn & 0xe003) == 0x4001) { /* c.li */
> +               imm = (insn >> 2) & 0x1f;
> +               imm |= ((insn >> 12) & 1) ? -32 : 0;
> +               rd = (insn >> 7) & 0x1f;
> +               if (rd)
> +                       regs_p[rd] = imm;
> +               return 0;
> +       }

The mepc update is missing, so this patch will not work.

That aside, what Jess pointed out is that on a machine *NOT*
supporting C, we're not emulating anything. The 16b instruction
parcels do not exist here, so we cannot really emulate that. Instead,
what the firmware gets is a 32b bogus/nonexisting instruction.
Emulating that is... weird. A valid concern! ;-)

The Linux Image spec says that code0 can be 'MZ' (and will be for UEFI
images). One correct fix (I think) is changing the non-UEFI loader,
fixing up code0 if it's MZ, making sure not to execute that (well, you
could on C capable machines).  That, or changing specs.


Bj?rn


From wxjstz at 126.com  Thu Nov  9 07:22:46 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 09 Nov 2023 23:22:46 +0800
Subject: [PATCH] lib: sbi_illegal_insn: Emulate c.li
In-Reply-To: <CAJ+HfNj3s+jkBdVC=9H8W7NZwvUiROL0Miyj5xXumLam861VbA@mail.gmail.com>
References: <20231109082540.52144-1-wxjstz@126.com>
	 <CAJ+HfNj3s+jkBdVC=9H8W7NZwvUiROL0Miyj5xXumLam861VbA@mail.gmail.com>
Message-ID: <8a13f19a91fcf56616ea9e36eac387ca0ea04d44.camel@126.com>

? 2023-11-09???? 15:48 +0100?Bj?rn T?pel???
> On Thu, 9 Nov 2023 at 09:26, Xiang W <wxjstz at 126.com> wrote:
> > 
> > The Linux kernel RISC-V image format allows that UEFI Images can also
> > be booted by non-UEFI firmware. However for that to work, the PE/Image
> > combo requires that 'MZ' is a valid instruction. On RISC-V, 'MZ' is
> > only a valid instruction if the hardware is C capable [1]. So add
> > Emulate c.li
> > 
> > Signed-off-by: Bj?rn T?pel <bjorn at rivosinc.com>
> > Signed-off-by: Xiang W <wxjstz at 126.com>
> > ---
> > ?lib/sbi/sbi_illegal_insn.c | 18 +++++++++++++++++-
> > ?1 file changed, 17 insertions(+), 1 deletion(-)
> > 
> > diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
> > index 2be4757..4ab10f4 100644
> > --- a/lib/sbi/sbi_illegal_insn.c
> > +++ b/lib/sbi/sbi_illegal_insn.c
> > @@ -102,6 +102,22 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
> > ??????? return 0;
> > ?}
> > 
> > +static int compressed_insn(ulong insn, struct sbi_trap_regs *regs) {
> > +?????? unsigned long imm, rd;
> > +?????? unsigned long *regs_p = (unsigned long *)regs;
> > +
> > +?????? if ((insn & 0xe003) == 0x4001) { /* c.li */
> > +?????????????? imm = (insn >> 2) & 0x1f;
> > +?????????????? imm |= ((insn >> 12) & 1) ? -32 : 0;
> > +?????????????? rd = (insn >> 7) & 0x1f;
> > +?????????????? if (rd)
> > +?????????????????????? regs_p[rd] = imm;
> > +?????????????? return 0;
> > +?????? }
> 
> The mepc update is missing, so this patch will not work.
> 
> That aside, what Jess pointed out is that on a machine *NOT*
> supporting C, we're not emulating anything. The 16b instruction
> parcels do not exist here, so we cannot really emulate that. Instead,
> what the firmware gets is a 32b bogus/nonexisting instruction.
> Emulating that is... weird. A valid concern! ;-)
> 
> The Linux Image spec says that code0 can be 'MZ' (and will be for UEFI
> images). One correct fix (I think) is changing the non-UEFI loader,
> fixing up code0 if it's MZ, making sure not to execute that (well, you
> could on C capable machines).? That, or changing specs.
Ok. let's abandon.

Regards,
Xiang W

> 
> 
> Bj?rn



From heinrich.schuchardt at canonical.com  Fri Nov 10 19:26:40 2023
From: heinrich.schuchardt at canonical.com (Heinrich Schuchardt)
Date: Sat, 11 Nov 2023 04:26:40 +0100
Subject: [PATCH 1/1] lib: fix access to hfeatures->extensions
Message-ID: <20231111032640.38175-1-heinrich.schuchardt@canonical.com>

We may in future have more than xlen ISA extensions supported by OpenSBI.
Commit 6259b2ec2d09 ("lib: utils/fdt: Fix fdt_parse_isa_extensions()
implementation") therefore assumes that hfeatures->extensions is an array.
We have to make this change in the rest of the code too.

Fixes: 6259b2ec2d09 ("lib: utils/fdt: Fix fdt_parse_isa_extensions() implementation")
Addresses-Coverity-ID: 1568357 Out-of-bounds access
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>
---
 include/sbi/sbi_hart.h      |  3 ++-
 lib/sbi/sbi_hart.c          | 46 ++++++++++++++++++++-----------------
 platform/generic/platform.c |  2 +-
 3 files changed, 28 insertions(+), 23 deletions(-)

diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
index e60f415..9a5335a 100644
--- a/include/sbi/sbi_hart.h
+++ b/include/sbi/sbi_hart.h
@@ -10,6 +10,7 @@
 #ifndef __SBI_HART_H__
 #define __SBI_HART_H__
 
+#include <sbi/sbi_bitops.h>
 #include <sbi/sbi_types.h>
 
 /** Possible privileged specification versions of a hart */
@@ -65,7 +66,7 @@ enum sbi_hart_extensions {
 struct sbi_hart_features {
 	bool detected;
 	int priv_version;
-	unsigned long extensions;
+	unsigned long extensions[BITS_TO_LONGS(SBI_HART_EXT_MAX)];
 	unsigned int pmp_count;
 	unsigned int pmp_addr_bits;
 	unsigned long pmp_gran;
diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index 29d6481..c3fccb3 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -614,9 +614,9 @@ static inline void __sbi_hart_update_extension(
 					bool enable)
 {
 	if (enable)
-		hfeatures->extensions |= BIT(ext);
+		hfeatures->extensions[BIT_WORD(ext)] |= BIT_MASK(ext);
 	else
-		hfeatures->extensions &= ~BIT(ext);
+		hfeatures->extensions[BIT_WORD(ext)] &= ~BIT_MASK(ext);
 }
 
 /**
@@ -649,7 +649,7 @@ bool sbi_hart_has_extension(struct sbi_scratch *scratch,
 	struct sbi_hart_features *hfeatures =
 			sbi_scratch_offset_ptr(scratch, hart_features_offset);
 
-	if (hfeatures->extensions & BIT(ext))
+	if (hfeatures->extensions[BIT_WORD(ext)] & BIT_MASK(ext))
 		return true;
 	else
 		return false;
@@ -705,31 +705,35 @@ void sbi_hart_get_extensions_str(struct sbi_scratch *scratch,
 {
 	struct sbi_hart_features *hfeatures =
 			sbi_scratch_offset_ptr(scratch, hart_features_offset);
-	int offset = 0, ext = 0;
-	char *temp;
+	int offset = 0;
 
 	if (!extensions_str || nestr <= 0)
 		return;
 	sbi_memset(extensions_str, 0, nestr);
 
-	if (!hfeatures->extensions)
-		goto done;
-
-	do {
-		if (hfeatures->extensions & BIT(ext)) {
-			temp = sbi_hart_extension_id2string(ext);
-			if (temp) {
-				sbi_snprintf(extensions_str + offset,
-					     nestr - offset,
-					     "%s,", temp);
-				offset = offset + sbi_strlen(temp) + 1;
+	for (int word = 0; word < BITS_TO_LONGS(SBI_HART_EXT_MAX); ++word) {
+		int ext;
+
+		if (!hfeatures->extensions[word])
+			continue;
+
+		ext = word * BITS_PER_LONG;
+		for (int bit = 0; bit < BITS_PER_LONG && ext < SBI_HART_EXT_MAX;
+		     ++bit, ++ext) {
+			char *temp;
+
+			if (hfeatures->extensions[word] & BIT(bit)) {
+				temp = sbi_hart_extension_id2string(ext);
+				if (temp) {
+					sbi_snprintf(extensions_str + offset,
+						     nestr - offset,
+						     "%s,", temp);
+					offset += sbi_strlen(temp) + 1;
+				}
 			}
 		}
+	}
 
-		ext++;
-	} while (ext < SBI_HART_EXT_MAX);
-
-done:
 	if (offset)
 		extensions_str[offset - 1] = '\0';
 	else
@@ -799,7 +803,7 @@ static int hart_detect_features(struct sbi_scratch *scratch)
 		return 0;
 
 	/* Clear hart features */
-	hfeatures->extensions = 0;
+	sbi_memset(hfeatures->extensions, 0, sizeof(hfeatures->extensions));
 	hfeatures->pmp_count = 0;
 	hfeatures->mhpm_mask = 0;
 
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index 66a0b77..9182faa 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -215,7 +215,7 @@ static int generic_extensions_init(struct sbi_hart_features *hfeatures)
 
 	/* Parse the ISA string from FDT and enable the listed extensions */
 	rc = fdt_parse_isa_extensions(fdt_get_address(), current_hartid(),
-				      &hfeatures->extensions);
+				      hfeatures->extensions);
 
 	if (rc)
 		return rc;
-- 
2.40.1



From inochiama at outlook.com  Fri Nov 10 20:16:40 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Sat, 11 Nov 2023 12:16:40 +0800
Subject: [PATCH] lib: sbi_pmu: disable IR and CY counter in SBI initialization
Message-ID: <IA1PR20MB49535075F058FF74E83CD41BBBADA@IA1PR20MB4953.namprd20.prod.outlook.com>

The default overflow behavior of INSTRET and CYCLE counter remains
unclear when platform has a custom pmu_device with irq callback.
For example, enabling the IR and CY events in the startup without
setting related CSRs may cause missing interrupt and wrong event
report for D1 soc.

Disable IR and CY counter in the startup and let OS manage them.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
---
 lib/sbi/sbi_hart.c | 2 +-
 lib/sbi/sbi_pmu.c  | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index 29d6481..8a908d6 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -64,7 +64,7 @@ static void mstatus_init(struct sbi_scratch *scratch)
 
 	/* All programmable counters will start running at runtime after S-mode request */
 	if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_11)
-		csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFF8);
+		csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFFD);
 
 	/**
 	 * The mhpmeventn[h] CSR should be initialized with interrupt disabled
diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index f4c8fc4..1e2fc01 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -944,7 +944,7 @@ void sbi_pmu_set_device(const struct sbi_pmu_device *dev)
 void sbi_pmu_exit(struct sbi_scratch *scratch)
 {
 	if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_11)
-		csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFF8);
+		csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFFD);
 
 	if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_10)
 		csr_write(CSR_MCOUNTEREN, -1);
-- 
2.42.1



From ganboing at gmail.com  Fri Nov 10 22:05:58 2023
From: ganboing at gmail.com (Bo Gan)
Date: Fri, 10 Nov 2023 22:05:58 -0800
Subject: Question about potential missed IPI events
Message-ID: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>

Hi OpenSBI community,

Current IPI send/recv implementation uses ipi_data->ipi_type to indicate
an IPI event pending. In sbi_ipi_send, I see have the following:

> /*
>  * Set IPI type on remote hart's scratch area and
>  * trigger the interrupt
>  */
> atomic_raw_set_bit(event, &ipi_data->ipi_type);
> smp_wmb();
> 
> if (ipi_dev && ipi_dev->ipi_send)
> 	ipi_dev->ipi_send(remote_hartindex)

And in sbi_ipi_process, the following:

> if (ipi_dev && ipi_dev->ipi_clear)
> 	ipi_dev->ipi_clear(hartindex);
> 
> ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
> ipi_event = 0;
> while (ipi_type) {

I'm think of if there's any possibility of missed IPIs. Assuming we are
talking about sifive/clint (aclint_mswi.c), I know it means that there're
both smp_wmb (fence w,w) and __io_bw (fence w,o) between atomic_raw_set_bit
and the write to remote hart's msip register in clint. Consider the following

Right before Hart A ipi_dev->ipi_clear() for the msip write-1 issued by hart B,
Hart C did an ipi_dev->ipi_send to hart A. Hart A then cleared the msip bit,
resulting in both IPI (from B and C) coalesced to a single one. In this case,
A will observe the ipi_type set by B, due to the smp_wmb + __io_bw, and the IPI
was indeed generated by B's write to msip. However will the barrier ensure A
also observe the write to ipi_type from hart C? For the coalescing to happen,
clint will observe C's write-1 before A's write-0, does it imply A'll observe
C's write to ipi_type from xchg?

I doubt it mainly due to 2 reasons:

1. ipi_dev->ipi_clear is a writel(msip), which has __io_bw (fence w,o) I don't
    think it helps preventing the writel to happen before xchg. If that happens,
    it's possible that A missed C's write to ipi_type.

2. Suppose the ipi_clear() happened before xchg, from A's standpoint, it's a
    write, not a read. A wasn't reading from msip, but only writing to it. I'm
    not sure if A's write to msip after C's write (observed by clint) implies
    A's observing the same order. A is not reading from msip, so what helps to
    establish this order? Will the RVWMO global memory order help? Can we think
    of clint as another hart that follows RVWMO?

Bo


From jrtc27 at jrtc27.com  Fri Nov 10 23:48:20 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Sat, 11 Nov 2023 07:48:20 +0000
Subject: [PATCH] lib: sbi_pmu: disable IR and CY counter in SBI
 initialization
In-Reply-To: <IA1PR20MB49535075F058FF74E83CD41BBBADA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49535075F058FF74E83CD41BBBADA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <D1256FAE-0648-4776-A8EC-BC42066386BF@jrtc27.com>

On 11 Nov 2023, at 04:16, Inochi Amaoto <inochiama at outlook.com> wrote:
> 
> The default overflow behavior of INSTRET and CYCLE counter remains
> unclear when platform has a custom pmu_device with irq callback.
> For example, enabling the IR and CY events in the startup without
> setting related CSRs may cause missing interrupt and wrong event
> report for D1 soc.
> 
> Disable IR and CY counter in the startup and let OS manage them.

Is this not an ABI break? Maybe Linux doesn?t care, but other OSes like
FreeBSD assume the counters are working and make them usable by U-mode.
If they no longer count up then you will break that for all existing
FreeBSD kernels.

Please don?t change existing semantics.

Jess

> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> ---
> lib/sbi/sbi_hart.c | 2 +-
> lib/sbi/sbi_pmu.c  | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 29d6481..8a908d6 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -64,7 +64,7 @@ static void mstatus_init(struct sbi_scratch *scratch)
> 
> /* All programmable counters will start running at runtime after S-mode request */
> if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_11)
> - csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFF8);
> + csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFFD);
> 
> /**
> * The mhpmeventn[h] CSR should be initialized with interrupt disabled
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index f4c8fc4..1e2fc01 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -944,7 +944,7 @@ void sbi_pmu_set_device(const struct sbi_pmu_device *dev)
> void sbi_pmu_exit(struct sbi_scratch *scratch)
> {
> if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_11)
> - csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFF8);
> + csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFFD);
> 
> if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_10)
> csr_write(CSR_MCOUNTEREN, -1);
> -- 
> 2.42.1
> 
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



From inochiama at outlook.com  Sat Nov 11 00:21:15 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Sat, 11 Nov 2023 16:21:15 +0800
Subject: [PATCH] lib: sbi_pmu: disable IR and CY counter in SBI initialization
In-Reply-To: <D1256FAE-0648-4776-A8EC-BC42066386BF@jrtc27.com>
References: <D1256FAE-0648-4776-A8EC-BC42066386BF@jrtc27.com>
Message-ID: <IA1PR20MB4953A812CB161F4CAEEA09B4BBADA@IA1PR20MB4953.namprd20.prod.outlook.com>

>
>On 11 Nov 2023, at 04:16, Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> The default overflow behavior of INSTRET and CYCLE counter remains
>> unclear when platform has a custom pmu_device with irq callback.
>> For example, enabling the IR and CY events in the startup without
>> setting related CSRs may cause missing interrupt and wrong event
>> report for D1 soc.
>>
>> Disable IR and CY counter in the startup and let OS manage them.
>
>Is this not an ABI break? Maybe Linux doesn?t care, but other OSes like
>FreeBSD assume the counters are working and make them usable by U-mode.
>If they no longer count up then you will break that for all existing
>FreeBSD kernels.
>
>Please don?t change existing semantics.
>

Thanks for your reminder, this does introduct a ABI break for the existing
counter. I will explore a new way to resolve this problem.

>Jess
>
>> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
>> ---
>> lib/sbi/sbi_hart.c | 2 +-
>> lib/sbi/sbi_pmu.c  | 2 +-
>> 2 files changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
>> index 29d6481..8a908d6 100644
>> --- a/lib/sbi/sbi_hart.c
>> +++ b/lib/sbi/sbi_hart.c
>> @@ -64,7 +64,7 @@ static void mstatus_init(struct sbi_scratch *scratch)
>>
>> /* All programmable counters will start running at runtime after S-mode request */
>> if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_11)
>> - csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFF8);
>> + csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFFD);
>>
>> /**
>> * The mhpmeventn[h] CSR should be initialized with interrupt disabled
>> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
>> index f4c8fc4..1e2fc01 100644
>> --- a/lib/sbi/sbi_pmu.c
>> +++ b/lib/sbi/sbi_pmu.c
>> @@ -944,7 +944,7 @@ void sbi_pmu_set_device(const struct sbi_pmu_device *dev)
>> void sbi_pmu_exit(struct sbi_scratch *scratch)
>> {
>> if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_11)
>> - csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFF8);
>> + csr_write(CSR_MCOUNTINHIBIT, 0xFFFFFFFD);
>>
>> if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_10)
>> csr_write(CSR_MCOUNTEREN, -1);
>> --
>> 2.42.1
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi
>
>


From wxjstz at 126.com  Sat Nov 11 04:47:13 2023
From: wxjstz at 126.com (Xiang W)
Date: Sat, 11 Nov 2023 20:47:13 +0800
Subject: [PATCH 1/1] lib: fix access to hfeatures->extensions
In-Reply-To: <20231111032640.38175-1-heinrich.schuchardt@canonical.com>
References: <20231111032640.38175-1-heinrich.schuchardt@canonical.com>
Message-ID: <9602b0abd8d4b99bae77290b3c86252f167a5c2d.camel@126.com>

? 2023-11-11???? 04:26 +0100?Heinrich Schuchardt???
> We may in future have more than xlen ISA extensions supported by OpenSBI.
> Commit 6259b2ec2d09 ("lib: utils/fdt: Fix fdt_parse_isa_extensions()
> implementation") therefore assumes that hfeatures->extensions is an array.
> We have to make this change in the rest of the code too.
> 
> Fixes: 6259b2ec2d09 ("lib: utils/fdt: Fix fdt_parse_isa_extensions() implementation")
> Addresses-Coverity-ID: 1568357 Out-of-bounds access
> Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>
Similar to this:
https://lists.infradead.org/pipermail/opensbi/2023-October/005787.html

Regards,
Xiang W

> ---
> ?include/sbi/sbi_hart.h????? |? 3 ++-
> ?lib/sbi/sbi_hart.c????????? | 46 ++++++++++++++++++++-----------------
> ?platform/generic/platform.c |? 2 +-
> ?3 files changed, 28 insertions(+), 23 deletions(-)
> 
> diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
> index e60f415..9a5335a 100644
> --- a/include/sbi/sbi_hart.h
> +++ b/include/sbi/sbi_hart.h
> @@ -10,6 +10,7 @@
> ?#ifndef __SBI_HART_H__
> ?#define __SBI_HART_H__
> ?
> +#include <sbi/sbi_bitops.h>
> ?#include <sbi/sbi_types.h>
> ?
> ?/** Possible privileged specification versions of a hart */
> @@ -65,7 +66,7 @@ enum sbi_hart_extensions {
> ?struct sbi_hart_features {
> ?	bool detected;
> ?	int priv_version;
> -	unsigned long extensions;
> +	unsigned long extensions[BITS_TO_LONGS(SBI_HART_EXT_MAX)];
> ?	unsigned int pmp_count;
> ?	unsigned int pmp_addr_bits;
> ?	unsigned long pmp_gran;
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 29d6481..c3fccb3 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -614,9 +614,9 @@ static inline void __sbi_hart_update_extension(
> ?					bool enable)
> ?{
> ?	if (enable)
> -		hfeatures->extensions |= BIT(ext);
> +		hfeatures->extensions[BIT_WORD(ext)] |= BIT_MASK(ext);
> ?	else
> -		hfeatures->extensions &= ~BIT(ext);
> +		hfeatures->extensions[BIT_WORD(ext)] &= ~BIT_MASK(ext);
> ?}
> ?
> ?/**
> @@ -649,7 +649,7 @@ bool sbi_hart_has_extension(struct sbi_scratch *scratch,
> ?	struct sbi_hart_features *hfeatures =
> ?			sbi_scratch_offset_ptr(scratch, hart_features_offset);
> ?
> -	if (hfeatures->extensions & BIT(ext))
> +	if (hfeatures->extensions[BIT_WORD(ext)] & BIT_MASK(ext))
> ?		return true;
> ?	else
> ?		return false;
> @@ -705,31 +705,35 @@ void sbi_hart_get_extensions_str(struct sbi_scratch *scratch,
> ?{
> ?	struct sbi_hart_features *hfeatures =
> ?			sbi_scratch_offset_ptr(scratch, hart_features_offset);
> -	int offset = 0, ext = 0;
> -	char *temp;
> +	int offset = 0;
> ?
> ?	if (!extensions_str || nestr <= 0)
> ?		return;
> ?	sbi_memset(extensions_str, 0, nestr);
> ?
> -	if (!hfeatures->extensions)
> -		goto done;
> -
> -	do {
> -		if (hfeatures->extensions & BIT(ext)) {
> -			temp = sbi_hart_extension_id2string(ext);
> -			if (temp) {
> -				sbi_snprintf(extensions_str + offset,
> -					???? nestr - offset,
> -					???? "%s,", temp);
> -				offset = offset + sbi_strlen(temp) + 1;
> +	for (int word = 0; word < BITS_TO_LONGS(SBI_HART_EXT_MAX); ++word) {
> +		int ext;
> +
> +		if (!hfeatures->extensions[word])
> +			continue;
> +
> +		ext = word * BITS_PER_LONG;
> +		for (int bit = 0; bit < BITS_PER_LONG && ext < SBI_HART_EXT_MAX;
> +		???? ++bit, ++ext) {
> +			char *temp;
> +
> +			if (hfeatures->extensions[word] & BIT(bit)) {
> +				temp = sbi_hart_extension_id2string(ext);
> +				if (temp) {
> +					sbi_snprintf(extensions_str + offset,
> +						???? nestr - offset,
> +						???? "%s,", temp);
> +					offset += sbi_strlen(temp) + 1;
> +				}
> ?			}
> ?		}
> +	}
> ?
> -		ext++;
> -	} while (ext < SBI_HART_EXT_MAX);
> -
> -done:
> ?	if (offset)
> ?		extensions_str[offset - 1] = '\0';
> ?	else
> @@ -799,7 +803,7 @@ static int hart_detect_features(struct sbi_scratch *scratch)
> ?		return 0;
> ?
> ?	/* Clear hart features */
> -	hfeatures->extensions = 0;
> +	sbi_memset(hfeatures->extensions, 0, sizeof(hfeatures->extensions));
> ?	hfeatures->pmp_count = 0;
> ?	hfeatures->mhpm_mask = 0;
> ?
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 66a0b77..9182faa 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -215,7 +215,7 @@ static int generic_extensions_init(struct sbi_hart_features *hfeatures)
> ?
> ?	/* Parse the ISA string from FDT and enable the listed extensions */
> ?	rc = fdt_parse_isa_extensions(fdt_get_address(), current_hartid(),
> -				????? &hfeatures->extensions);
> +				????? hfeatures->extensions);
> ?
> ?	if (rc)
> ?		return rc;
> -- 
> 2.40.1
> 
> 



From heinrich.schuchardt at canonical.com  Sat Nov 11 07:10:34 2023
From: heinrich.schuchardt at canonical.com (Heinrich Schuchardt)
Date: Sat, 11 Nov 2023 16:10:34 +0100
Subject: [PATCH 1/1] lib: fix access to hfeatures->extensions
In-Reply-To: <9602b0abd8d4b99bae77290b3c86252f167a5c2d.camel@126.com>
References: <20231111032640.38175-1-heinrich.schuchardt@canonical.com>
 <9602b0abd8d4b99bae77290b3c86252f167a5c2d.camel@126.com>
Message-ID: <f213c08d-ca0f-4c27-b83e-fc0469c17780@canonical.com>

On 11/11/23 13:47, Xiang W wrote:
> ? 2023-11-11???? 04:26 +0100?Heinrich Schuchardt???
>> We may in future have more than xlen ISA extensions supported by OpenSBI.
>> Commit 6259b2ec2d09 ("lib: utils/fdt: Fix fdt_parse_isa_extensions()
>> implementation") therefore assumes that hfeatures->extensions is an array.
>> We have to make this change in the rest of the code too.
>>
>> Fixes: 6259b2ec2d09 ("lib: utils/fdt: Fix fdt_parse_isa_extensions() implementation")
>> Addresses-Coverity-ID: 1568357 Out-of-bounds access
>> Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>
> Similar to this:
> https://lists.infradead.org/pipermail/opensbi/2023-October/005787.html
> 
> Regards,
> Xiang W

I should have scanned the mailing list. Us two coming up with nearly 
identical solution confirms that we are on the right path.

Your usage of bitops macros looks nicer.

Best regards

Heinrich

> 
>> ---
>>  ?include/sbi/sbi_hart.h????? |? 3 ++-
>>  ?lib/sbi/sbi_hart.c????????? | 46 ++++++++++++++++++++-----------------
>>  ?platform/generic/platform.c |? 2 +-
>>  ?3 files changed, 28 insertions(+), 23 deletions(-)
>>
>> diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
>> index e60f415..9a5335a 100644
>> --- a/include/sbi/sbi_hart.h
>> +++ b/include/sbi/sbi_hart.h
>> @@ -10,6 +10,7 @@
>>  ?#ifndef __SBI_HART_H__
>>  ?#define __SBI_HART_H__
>>   
>> +#include <sbi/sbi_bitops.h>
>>  ?#include <sbi/sbi_types.h>
>>   
>>  ?/** Possible privileged specification versions of a hart */
>> @@ -65,7 +66,7 @@ enum sbi_hart_extensions {
>>  ?struct sbi_hart_features {
>>  ?	bool detected;
>>  ?	int priv_version;
>> -	unsigned long extensions;
>> +	unsigned long extensions[BITS_TO_LONGS(SBI_HART_EXT_MAX)];
>>  ?	unsigned int pmp_count;
>>  ?	unsigned int pmp_addr_bits;
>>  ?	unsigned long pmp_gran;
>> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
>> index 29d6481..c3fccb3 100644
>> --- a/lib/sbi/sbi_hart.c
>> +++ b/lib/sbi/sbi_hart.c
>> @@ -614,9 +614,9 @@ static inline void __sbi_hart_update_extension(
>>  ?					bool enable)
>>  ?{
>>  ?	if (enable)
>> -		hfeatures->extensions |= BIT(ext);
>> +		hfeatures->extensions[BIT_WORD(ext)] |= BIT_MASK(ext);
>>  ?	else
>> -		hfeatures->extensions &= ~BIT(ext);
>> +		hfeatures->extensions[BIT_WORD(ext)] &= ~BIT_MASK(ext);
>>  ?}
>>   
>>  ?/**
>> @@ -649,7 +649,7 @@ bool sbi_hart_has_extension(struct sbi_scratch *scratch,
>>  ?	struct sbi_hart_features *hfeatures =
>>  ?			sbi_scratch_offset_ptr(scratch, hart_features_offset);
>>   
>> -	if (hfeatures->extensions & BIT(ext))
>> +	if (hfeatures->extensions[BIT_WORD(ext)] & BIT_MASK(ext))
>>  ?		return true;
>>  ?	else
>>  ?		return false;
>> @@ -705,31 +705,35 @@ void sbi_hart_get_extensions_str(struct sbi_scratch *scratch,
>>  ?{
>>  ?	struct sbi_hart_features *hfeatures =
>>  ?			sbi_scratch_offset_ptr(scratch, hart_features_offset);
>> -	int offset = 0, ext = 0;
>> -	char *temp;
>> +	int offset = 0;
>>   
>>  ?	if (!extensions_str || nestr <= 0)
>>  ?		return;
>>  ?	sbi_memset(extensions_str, 0, nestr);
>>   
>> -	if (!hfeatures->extensions)
>> -		goto done;
>> -
>> -	do {
>> -		if (hfeatures->extensions & BIT(ext)) {
>> -			temp = sbi_hart_extension_id2string(ext);
>> -			if (temp) {
>> -				sbi_snprintf(extensions_str + offset,
>> -					???? nestr - offset,
>> -					???? "%s,", temp);
>> -				offset = offset + sbi_strlen(temp) + 1;
>> +	for (int word = 0; word < BITS_TO_LONGS(SBI_HART_EXT_MAX); ++word) {
>> +		int ext;
>> +
>> +		if (!hfeatures->extensions[word])
>> +			continue;
>> +
>> +		ext = word * BITS_PER_LONG;
>> +		for (int bit = 0; bit < BITS_PER_LONG && ext < SBI_HART_EXT_MAX;
>> +		???? ++bit, ++ext) {
>> +			char *temp;
>> +
>> +			if (hfeatures->extensions[word] & BIT(bit)) {
>> +				temp = sbi_hart_extension_id2string(ext);
>> +				if (temp) {
>> +					sbi_snprintf(extensions_str + offset,
>> +						???? nestr - offset,
>> +						???? "%s,", temp);
>> +					offset += sbi_strlen(temp) + 1;
>> +				}
>>  ?			}
>>  ?		}
>> +	}
>>   
>> -		ext++;
>> -	} while (ext < SBI_HART_EXT_MAX);
>> -
>> -done:
>>  ?	if (offset)
>>  ?		extensions_str[offset - 1] = '\0';
>>  ?	else
>> @@ -799,7 +803,7 @@ static int hart_detect_features(struct sbi_scratch *scratch)
>>  ?		return 0;
>>   
>>  ?	/* Clear hart features */
>> -	hfeatures->extensions = 0;
>> +	sbi_memset(hfeatures->extensions, 0, sizeof(hfeatures->extensions));
>>  ?	hfeatures->pmp_count = 0;
>>  ?	hfeatures->mhpm_mask = 0;
>>   
>> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
>> index 66a0b77..9182faa 100644
>> --- a/platform/generic/platform.c
>> +++ b/platform/generic/platform.c
>> @@ -215,7 +215,7 @@ static int generic_extensions_init(struct sbi_hart_features *hfeatures)
>>   
>>  ?	/* Parse the ISA string from FDT and enable the listed extensions */
>>  ?	rc = fdt_parse_isa_extensions(fdt_get_address(), current_hartid(),
>> -				????? &hfeatures->extensions);
>> +				????? hfeatures->extensions);
>>   
>>  ?	if (rc)
>>  ?		return rc;
>> -- 
>> 2.40.1
>>
>>
> 



From heinrich.schuchardt at canonical.com  Sat Nov 11 07:14:07 2023
From: heinrich.schuchardt at canonical.com (Heinrich Schuchardt)
Date: Sat, 11 Nov 2023 16:14:07 +0100
Subject: [PATCH] lib: sbi: change sbi_hart_features.extensions as an array
In-Reply-To: <20231030164046.25108-1-wxjstz@126.com>
References: <20231030164046.25108-1-wxjstz@126.com>
Message-ID: <8e77c232-8fec-46ea-88a5-a72928930c61@canonical.com>

On 10/30/23 17:39, Xiang W wrote:
> In the future there may be a lot of ISA extensions, a 'long' may not
> be able to accommodate, changed to an array for the future.
> 

Fixes: 6259b2ec2d09 ("lib: utils/fdt: Fix fdt_parse_isa_extensions() 
implementation")
Addresses-Coverity-ID: 1568357 Out-of-bounds access

> Signed-off-by: Xiang W <wxjstz at 126.com>

Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>

> ---
>   include/sbi/sbi_hart.h      |  3 ++-
>   lib/sbi/sbi_hart.c          | 32 ++++++++++++--------------------
>   platform/generic/platform.c |  2 +-
>   3 files changed, 15 insertions(+), 22 deletions(-)
> 
> diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
> index e60f415..3b73f63 100644
> --- a/include/sbi/sbi_hart.h
> +++ b/include/sbi/sbi_hart.h
> @@ -11,6 +11,7 @@
>   #define __SBI_HART_H__
>   
>   #include <sbi/sbi_types.h>
> +#include <sbi/sbi_bitops.h>
>   
>   /** Possible privileged specification versions of a hart */
>   enum sbi_hart_priv_versions {
> @@ -65,7 +66,7 @@ enum sbi_hart_extensions {
>   struct sbi_hart_features {
>   	bool detected;
>   	int priv_version;
> -	unsigned long extensions;
> +	unsigned long extensions[BITS_TO_LONGS(SBI_HART_EXT_MAX)];
>   	unsigned int pmp_count;
>   	unsigned int pmp_addr_bits;
>   	unsigned long pmp_gran;
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 29d6481..f94eba5 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -614,9 +614,9 @@ static inline void __sbi_hart_update_extension(
>   					bool enable)
>   {
>   	if (enable)
> -		hfeatures->extensions |= BIT(ext);
> +		__set_bit(ext, hfeatures->extensions);
>   	else
> -		hfeatures->extensions &= ~BIT(ext);
> +		__clear_bit(ext, hfeatures->extensions);
>   }
>   
>   /**
> @@ -649,7 +649,7 @@ bool sbi_hart_has_extension(struct sbi_scratch *scratch,
>   	struct sbi_hart_features *hfeatures =
>   			sbi_scratch_offset_ptr(scratch, hart_features_offset);
>   
> -	if (hfeatures->extensions & BIT(ext))
> +	if (__test_bit(ext, hfeatures->extensions))
>   		return true;
>   	else
>   		return false;
> @@ -712,24 +712,16 @@ void sbi_hart_get_extensions_str(struct sbi_scratch *scratch,
>   		return;
>   	sbi_memset(extensions_str, 0, nestr);
>   
> -	if (!hfeatures->extensions)
> -		goto done;
> -
> -	do {
> -		if (hfeatures->extensions & BIT(ext)) {
> -			temp = sbi_hart_extension_id2string(ext);
> -			if (temp) {
> -				sbi_snprintf(extensions_str + offset,
> -					     nestr - offset,
> -					     "%s,", temp);
> -				offset = offset + sbi_strlen(temp) + 1;
> -			}
> +	for_each_set_bit(ext, hfeatures->extensions, SBI_HART_EXT_MAX) {
> +		temp = sbi_hart_extension_id2string(ext);
> +		if (temp) {
> +			sbi_snprintf(extensions_str + offset,
> +				     nestr - offset,
> +				     "%s,", temp);
> +			offset = offset + sbi_strlen(temp) + 1;
>   		}
> +	}
>   
> -		ext++;
> -	} while (ext < SBI_HART_EXT_MAX);
> -
> -done:
>   	if (offset)
>   		extensions_str[offset - 1] = '\0';
>   	else
> @@ -799,7 +791,7 @@ static int hart_detect_features(struct sbi_scratch *scratch)
>   		return 0;
>   
>   	/* Clear hart features */
> -	hfeatures->extensions = 0;
> +	sbi_memset(hfeatures->extensions, 0, sizeof(hfeatures->extensions));
>   	hfeatures->pmp_count = 0;
>   	hfeatures->mhpm_mask = 0;
>   
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 66a0b77..9182faa 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -215,7 +215,7 @@ static int generic_extensions_init(struct sbi_hart_features *hfeatures)
>   
>   	/* Parse the ISA string from FDT and enable the listed extensions */
>   	rc = fdt_parse_isa_extensions(fdt_get_address(), current_hartid(),
> -				      &hfeatures->extensions);
> +				      hfeatures->extensions);
>   
>   	if (rc)
>   		return rc;



From ganboing at gmail.com  Mon Nov 13 04:31:42 2023
From: ganboing at gmail.com (Bo Gan)
Date: Mon, 13 Nov 2023 04:31:42 -0800
Subject: Security and correctness issues around captured FDT pointers
Message-ID: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>

Hi OpenSBI community again,

It's a little bit unclear to me from design point of view how are we going to
protect the flattened device tree shared with lower privileged modes. I ask
because in several places we have the following

1. generic/platform.c:
> static void fw_platform_lookup_special(void *fdt, int root_offset)
> {
> ...
> 	match = fdt_match_node(fdt, root_offset, plat->match_table);
> 	if (!match)
> 		continue;
> 
> 	generic_plat = plat;
> 	generic_plat_match = match;

`generic_plat_match` captures a data blob inside fdt.

2. generic/starfive/jh7110.c
> static int starfive_jh7110_final_init(bool cold_boot,
> 				      const struct fdt_match *match)
> {
> 	void *fdt = fdt_get_address();
> 
> 	if (cold_boot) {
> 		fdt_reset_driver_init(fdt, &fdt_reset_pmic);
> 		return starfive_jh7110_inst_init(fdt);
> 	}
> 
> static int pm_reset_init(void *fdt, int nodeoff,
> 			 const struct fdt_match *match)
> {
> ...
> 	rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
> 	if (rc)
> 		return rc;
> 
> 	pmic_inst.dev_addr = addr;
> 	pmic_inst.compatible = match->compatible;

`fdt_reset_pmic` will capture a compatible string in `pmic.compatible`

For 1, it's probably safe unless we are using the same fdt from previous boot-
loader and pass it to the next stage. (Could happen if using fw_jump and not
defining FW_JUMP_FDT_ADDR). For 2, it's very likely unsafe. `fdt_get_address`
will return `root.next_arg1`, which IS the fdt we pass to next stage. I don't
think we can ever made the fdt to next stage as read-only, because they, such
as U-boot or Linux kernel running in S-mode, usually will relocate it and reuse
the memory for something else, or at least probably modify it in-place. When it
happens, the harts, at runtime, regardless of cold/warm booted, will access the
corrupted data. This can also be exploited by malicious actors.

In my opinion, the only place we access fdt should be from code boot hart, and
we should stop accessing it and zero-out pointers captured before it kicks warm
boot harts. If the fdt data is still need afterwards, either parse it or make a
copy (perhaps on the heap). fdt_get_address() can also be enhanced to prevent
it from being called in the wrong places. Please let me know your thoughts, and
I'm looking forward to your ideas. Thanks!

Bo




From apatel at ventanamicro.com  Mon Nov 13 09:29:45 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 13 Nov 2023 22:59:45 +0530
Subject: Security and correctness issues around captured FDT pointers
In-Reply-To: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>
References: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>
Message-ID: <CAK9=C2W3d1t+83kNoWhkA81M59Wen2gGa_EcgOJA1U19WJP4mQ@mail.gmail.com>

On Mon, Nov 13, 2023 at 6:02?PM Bo Gan <ganboing at gmail.com> wrote:
>
> Hi OpenSBI community again,
>
> It's a little bit unclear to me from design point of view how are we going to
> protect the flattened device tree shared with lower privileged modes. I ask
> because in several places we have the following
>
> 1. generic/platform.c:
> > static void fw_platform_lookup_special(void *fdt, int root_offset)
> > {
> > ...
> >       match = fdt_match_node(fdt, root_offset, plat->match_table);
> >       if (!match)
> >               continue;
> >
> >       generic_plat = plat;
> >       generic_plat_match = match;
>
> `generic_plat_match` captures a data blob inside fdt.
>
> 2. generic/starfive/jh7110.c
> > static int starfive_jh7110_final_init(bool cold_boot,
> >                                     const struct fdt_match *match)
> > {
> >       void *fdt = fdt_get_address();
> >
> >       if (cold_boot) {
> >               fdt_reset_driver_init(fdt, &fdt_reset_pmic);
> >               return starfive_jh7110_inst_init(fdt);
> >       }
> >
> > static int pm_reset_init(void *fdt, int nodeoff,
> >                        const struct fdt_match *match)
> > {
> > ...
> >       rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
> >       if (rc)
> >               return rc;
> >
> >       pmic_inst.dev_addr = addr;
> >       pmic_inst.compatible = match->compatible;
>
> `fdt_reset_pmic` will capture a compatible string in `pmic.compatible`
>
> For 1, it's probably safe unless we are using the same fdt from previous boot-
> loader and pass it to the next stage. (Could happen if using fw_jump and not
> defining FW_JUMP_FDT_ADDR). For 2, it's very likely unsafe. `fdt_get_address`
> will return `root.next_arg1`, which IS the fdt we pass to next stage. I don't
> think we can ever made the fdt to next stage as read-only, because they, such
> as U-boot or Linux kernel running in S-mode, usually will relocate it and reuse
> the memory for something else, or at least probably modify it in-place. When it
> happens, the harts, at runtime, regardless of cold/warm booted, will access the
> corrupted data. This can also be exploited by malicious actors.

Yes, #1 is mostly safe.

The #2 needs to be fixed in the generic/starfive/jh7110.c and all other places
where it exists.

>
> In my opinion, the only place we access fdt should be from code boot hart, and
> we should stop accessing it and zero-out pointers captured before it kicks warm
> boot harts. If the fdt data is still need afterwards, either parse it or make a
> copy (perhaps on the heap). fdt_get_address() can also be enhanced to prevent
> it from being called in the wrong places. Please let me know your thoughts, and
> I'm looking forward to your ideas. Thanks!

We should certainly stop accessing FDT after cold boot hard is done but I don't
think we should zero-out pointers as well.

Regards,
Anup

>
> Bo
>
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 07:29:42 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 20:59:42 +0530
Subject: [PATCH 1/1] lib: sbi: simplify sanitize_domain()
In-Reply-To: <20231006205926.26599-1-heinrich.schuchardt@canonical.com>
References: <20231006205926.26599-1-heinrich.schuchardt@canonical.com>
Message-ID: <CAAhSdy03qDHYRkVdweV819D=P3bE_199Td_d=A3ka42=dnz4qA@mail.gmail.com>

On Sat, Oct 7, 2023 at 2:30?AM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> Since commit 112daa2e647c ("lib: sbi: Maximize the use of HART index in
> sbi_domain") the platform parameter is unused.
>
> Fixes: 112daa2e647c ("lib: sbi: Maximize the use of HART index in sbi_domain")
> Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_domain.c | 9 +++------
>  1 file changed, 3 insertions(+), 6 deletions(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index fd4a296..4d248d0 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -256,8 +256,7 @@ static const struct sbi_domain_memregion *find_next_subset_region(
>         return ret;
>  }
>
> -static int sanitize_domain(const struct sbi_platform *plat,
> -                          struct sbi_domain *dom)
> +static int sanitize_domain(struct sbi_domain *dom)
>  {
>         u32 i, j, count;
>         struct sbi_domain_memregion treg, *reg, *reg1;
> @@ -494,7 +493,6 @@ int sbi_domain_register(struct sbi_domain *dom,
>         int rc;
>         struct sbi_domain *tdom;
>         u32 cold_hartid = current_hartid();
> -       const struct sbi_platform *plat = sbi_platform_thishart_ptr();
>
>         /* Sanity checks */
>         if (!dom || !assign_mask || domain_finalized)
> @@ -517,7 +515,7 @@ int sbi_domain_register(struct sbi_domain *dom,
>         }
>
>         /* Sanitize discovered domain */
> -       rc = sanitize_domain(plat, dom);
> +       rc = sanitize_domain(dom);
>         if (rc) {
>                 sbi_printf("%s: sanity checks failed for"
>                            " %s (error %d)\n", __func__,
> @@ -564,7 +562,6 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
>         int rc;
>         bool reg_merged;
>         struct sbi_domain_memregion *nreg, *nreg1, *nreg2;
> -       const struct sbi_platform *plat = sbi_platform_thishart_ptr();
>
>         /* Sanity checks */
>         if (!reg || domain_finalized || !root.regions ||
> @@ -590,7 +587,7 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
>         /* Sort and optimize root regions */
>         do {
>                 /* Sanitize the root domain so that memregions are sorted */
> -               rc = sanitize_domain(plat, &root);
> +               rc = sanitize_domain(&root);
>                 if (rc) {
>                         sbi_printf("%s: sanity checks failed for"
>                                    " %s (error %d)\n", __func__,
> --
> 2.40.1
>


From anup at brainfault.org  Tue Nov 14 07:45:24 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 21:15:24 +0530
Subject: [PATCH v2 1/4] platform: generic: thead: move to thead c9xx
 header to vendor specific postion
In-Reply-To: <IA1PR20MB495315D187896E99EF4A8747BBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953287B7688E0F5058984BDBBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB495315D187896E99EF4A8747BBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy329OVshQD_ciLx85fzFpwabJ6dcfEnkPsTxBS940JYZA@mail.gmail.com>

On Thu, Oct 5, 2023 at 12:28?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> The CSR encoding for t-head c9xx cores is shared across all the
> platforms with these cores. So move header thead_c9xx.h to the
> thead subdir.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/allwinner/sun20i-d1.c                        | 2 +-
>  .../generic/include/{thead_c9xx.h => thead/c9xx_encoding.h}   | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
>  rename platform/generic/include/{thead_c9xx.h => thead/c9xx_encoding.h} (98%)
>
> diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
> index a9a7e47..b960284 100644
> --- a/platform/generic/allwinner/sun20i-d1.c
> +++ b/platform/generic/allwinner/sun20i-d1.c
> @@ -5,7 +5,7 @@
>   */
>
>  #include <platform_override.h>
> -#include <thead_c9xx.h>
> +#include <thead/c9xx_encoding.h>
>  #include <sbi/riscv_asm.h>
>  #include <sbi/riscv_io.h>
>  #include <sbi/sbi_bitops.h>
> diff --git a/platform/generic/include/thead_c9xx.h b/platform/generic/include/thead/c9xx_encoding.h
> similarity index 98%
> rename from platform/generic/include/thead_c9xx.h
> rename to platform/generic/include/thead/c9xx_encoding.h
> index bab0408..58adbef 100644
> --- a/platform/generic/include/thead_c9xx.h
> +++ b/platform/generic/include/thead/c9xx_encoding.h
> @@ -1,5 +1,5 @@
> -#ifndef __RISCV_THEAD_C9XX_H____
> -#define __RISCV_THEAD_C9XX_H____
> +#ifndef __RISCV_THEAD_C9XX_ENCODING_H____
> +#define __RISCV_THEAD_C9XX_ENCODING_H____
>
>  /* T-HEAD C9xx M mode CSR.  */
>  #define THEAD_C9XX_CSR_MXSTATUS                0x7c0
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 07:45:39 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 21:15:39 +0530
Subject: [PATCH v2 2/4] platform: generic: thead: separate implement of
 T-HEAD c9xx pmu
In-Reply-To: <IA1PR20MB4953F08770A3C489E3413567BBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953287B7688E0F5058984BDBBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
 <20231005065843.239501-1-inochiama@outlook.com> <IA1PR20MB4953F08770A3C489E3413567BBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy0iRTjfYz82u1KS+rm8ri8atifRWPEVNCOOHwU3BSfSTg@mail.gmail.com>

On Thu, Oct 5, 2023 at 12:29?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Separate the implement of T-HEAD c9xx pmu to allow any platform with
> c9xx cores can use it.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/Kconfig                  |  2 +
>  platform/generic/allwinner/sun20i-d1.c    | 51 +----------------
>  platform/generic/include/thead/c9xx_pmu.h |  7 +++
>  platform/generic/thead/Kconfig            |  5 ++
>  platform/generic/thead/objects.mk         |  2 +
>  platform/generic/thead/thead_c9xx_pmu.c   | 69 +++++++++++++++++++++++
>  6 files changed, 87 insertions(+), 49 deletions(-)
>  create mode 100644 platform/generic/include/thead/c9xx_pmu.h
>  create mode 100644 platform/generic/thead/Kconfig
>  create mode 100644 platform/generic/thead/thead_c9xx_pmu.c
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index e7bd94e..ea3b5cf 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -26,6 +26,7 @@ config PLATFORM_GENERIC_MINOR_VER
>  config PLATFORM_ALLWINNER_D1
>         bool "Allwinner D1 support"
>         depends on FDT_IRQCHIP_PLIC
> +       select THEAD_C9XX_PMU
>         default n
>
>  config PLATFORM_ANDES_AE350
> @@ -57,5 +58,6 @@ config PLATFORM_THEAD
>         default n
>
>  source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
> +source "$(OPENSBI_SRC_DIR)/platform/generic/thead/Kconfig"
>
>  endif
> diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
> index b960284..e9388db 100644
> --- a/platform/generic/allwinner/sun20i-d1.c
> +++ b/platform/generic/allwinner/sun20i-d1.c
> @@ -6,6 +6,7 @@
>
>  #include <platform_override.h>
>  #include <thead/c9xx_encoding.h>
> +#include <thead/c9xx_pmu.h>
>  #include <sbi/riscv_asm.h>
>  #include <sbi/riscv_io.h>
>  #include <sbi/sbi_bitops.h>
> @@ -223,58 +224,10 @@ static int sun20i_d1_fdt_fixup(void *fdt, const struct fdt_match *match)
>         return fdt_add_cpu_idle_states(fdt, sun20i_d1_cpu_idle_states);
>  }
>
> -static void thead_c9xx_pmu_ctr_enable_irq(uint32_t ctr_idx)
> -{
> -       if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
> -               return;
> -
> -       /**
> -        * Clear out the OF bit so that next interrupt can be enabled.
> -        * This should be done before starting interrupt to avoid unexcepted
> -        * overflow interrupt.
> -        */
> -       csr_clear(THEAD_C9XX_CSR_MCOUNTEROF, BIT(ctr_idx));
> -
> -       /**
> -        * This register is described in C9xx document as the control register
> -        * for enabling writes to the superuser state counter. However, if the
> -        * corresponding bit is not set to 1, scounterof will always read as 0
> -        * when the counter register overflows.
> -        */
> -       csr_set(THEAD_C9XX_CSR_MCOUNTERWEN, BIT(ctr_idx));
> -
> -       /**
> -        * SSCOFPMF uses the OF bit for enabling/disabling the interrupt,
> -        * while the C9XX has designated enable bits.
> -        * So enable per-counter interrupt on C9xx here.
> -        */
> -       csr_set(THEAD_C9XX_CSR_MCOUNTERINTEN, BIT(ctr_idx));
> -}
> -
> -static void thead_c9xx_pmu_ctr_disable_irq(uint32_t ctr_idx)
> -{
> -       /**
> -        * There is no need to clear the bit of mcounterwen, it will expire
> -        * after setting the csr mcountinhibit.
> -        */
> -       csr_clear(THEAD_C9XX_CSR_MCOUNTERINTEN, BIT(ctr_idx));
> -}
> -
> -static int thead_c9xx_pmu_irq_bit(void)
> -{
> -       return THEAD_C9XX_MIP_MOIP;
> -}
> -
> -const struct sbi_pmu_device thead_c9xx_pmu_device = {
> -       .hw_counter_enable_irq = thead_c9xx_pmu_ctr_enable_irq,
> -       .hw_counter_disable_irq = thead_c9xx_pmu_ctr_disable_irq,
> -       .hw_counter_irq_bit = thead_c9xx_pmu_irq_bit,
> -};
> -
>  static int sun20i_d1_extensions_init(const struct fdt_match *match,
>                                      struct sbi_hart_features *hfeatures)
>  {
> -       sbi_pmu_set_device(&thead_c9xx_pmu_device);
> +       thead_c9xx_register_pmu_device();
>
>         /* auto-detection doesn't work on t-head c9xx cores */
>         /* D1 has 29 mhpmevent csrs, but only 3-9,13-17 have valid value */
> diff --git a/platform/generic/include/thead/c9xx_pmu.h b/platform/generic/include/thead/c9xx_pmu.h
> new file mode 100644
> index 0000000..bb1d831
> --- /dev/null
> +++ b/platform/generic/include/thead/c9xx_pmu.h
> @@ -0,0 +1,7 @@
> +
> +#ifndef __RISCV_THEAD_C9XX_PMU_H____
> +#define __RISCV_THEAD_C9XX_PMU_H____
> +
> +void thead_c9xx_register_pmu_device(void);
> +
> +#endif // __RISCV_THEAD_C9XX_PMU_H____
> diff --git a/platform/generic/thead/Kconfig b/platform/generic/thead/Kconfig
> new file mode 100644
> index 0000000..e54e621
> --- /dev/null
> +++ b/platform/generic/thead/Kconfig
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: BSD-2-Clause
> +
> +config THEAD_C9XX_PMU
> +       bool "T-HEAD c9xx M-mode PMU support"
> +       default n
> diff --git a/platform/generic/thead/objects.mk b/platform/generic/thead/objects.mk
> index 854bfbd..df2c61b 100644
> --- a/platform/generic/thead/objects.mk
> +++ b/platform/generic/thead/objects.mk
> @@ -5,6 +5,8 @@
>  # Copyright (C) 2023 Alibaba Group Holding Limited.
>  #
>
> +platform-objs-$(CONFIG_THEAD_C9XX_PMU) += thead/thead_c9xx_pmu.o
> +
>  carray-platform_override_modules-$(CONFIG_PLATFORM_THEAD) += thead_generic
>  platform-objs-$(CONFIG_PLATFORM_THEAD) += thead/thead-generic.o
>  platform-objs-$(CONFIG_PLATFORM_THEAD) += thead/thead-trap-handler.o
> diff --git a/platform/generic/thead/thead_c9xx_pmu.c b/platform/generic/thead/thead_c9xx_pmu.c
> new file mode 100644
> index 0000000..d205802
> --- /dev/null
> +++ b/platform/generic/thead/thead_c9xx_pmu.c
> @@ -0,0 +1,69 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Authors:
> + *   Inochi Amaoto <inochiama at outlook.com>
> + *   Haijiao Liu <haijiao.liu at sophgo.com>
> + *
> + */
> +
> +#include <thead/c9xx_encoding.h>
> +#include <thead/c9xx_pmu.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/sbi_bitops.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi/sbi_pmu.h>
> +
> +static void thead_c9xx_pmu_ctr_enable_irq(uint32_t ctr_idx)
> +{
> +       if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
> +               return;
> +
> +       /**
> +        * Clear out the OF bit so that next interrupt can be enabled.
> +        * This should be done before starting interrupt to avoid unexcepted
> +        * overflow interrupt.
> +        */
> +       csr_clear(THEAD_C9XX_CSR_MCOUNTEROF, BIT(ctr_idx));
> +
> +       /**
> +        * This register is described in C9xx document as the control register
> +        * for enabling writes to the superuser state counter. However, if the
> +        * corresponding bit is not set to 1, scounterof will always read as 0
> +        * when the counter register overflows.
> +        */
> +       csr_set(THEAD_C9XX_CSR_MCOUNTERWEN, BIT(ctr_idx));
> +
> +       /**
> +        * SSCOFPMF uses the OF bit for enabling/disabling the interrupt,
> +        * while the C9XX has designated enable bits.
> +        * So enable per-counter interrupt on C9xx here.
> +        */
> +       csr_set(THEAD_C9XX_CSR_MCOUNTERINTEN, BIT(ctr_idx));
> +}
> +
> +static void thead_c9xx_pmu_ctr_disable_irq(uint32_t ctr_idx)
> +{
> +       /**
> +        * There is no need to clear the bit of mcounterwen, it will expire
> +        * after setting the csr mcountinhibit.
> +        */
> +       csr_clear(THEAD_C9XX_CSR_MCOUNTERINTEN, BIT(ctr_idx));
> +}
> +
> +static int thead_c9xx_pmu_irq_bit(void)
> +{
> +       return THEAD_C9XX_MIP_MOIP;
> +}
> +
> +static const struct sbi_pmu_device thead_c9xx_pmu_device = {
> +       .name = "thead,c900-pmu",
> +       .hw_counter_enable_irq = thead_c9xx_pmu_ctr_enable_irq,
> +       .hw_counter_disable_irq = thead_c9xx_pmu_ctr_disable_irq,
> +       .hw_counter_irq_bit = thead_c9xx_pmu_irq_bit,
> +};
> +
> +void thead_c9xx_register_pmu_device(void)
> +{
> +       sbi_pmu_set_device(&thead_c9xx_pmu_device);
> +}
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 07:45:55 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 21:15:55 +0530
Subject: [PATCH v2 3/4] platform: generic: thead: separate implement of
 T-HEAD c9xx errata
In-Reply-To: <IA1PR20MB495386DC04E588A3869ADD69BBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953287B7688E0F5058984BDBBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
 <20231005065843.239501-1-inochiama@outlook.com> <IA1PR20MB495386DC04E588A3869ADD69BBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy3O1hfmBKe4NK1L8ECHEitSSp05pX_L14_P-pm-epuReA@mail.gmail.com>

On Thu, Oct 5, 2023 at 12:29?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Separate the implement of T-HEAD c9xx errata to allow any platform
> with bug related to c9xx cores can use it.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/Kconfig                      |  1 +
>  platform/generic/include/thead/c9xx_errata.h  | 14 ++++++++++
>  platform/generic/thead/Kconfig                |  4 +++
>  platform/generic/thead/objects.mk             |  4 ++-
>  platform/generic/thead/thead-generic.c        | 28 ++++++++-----------
>  .../thead/thead_c9xx_errata_tlb_flush.c       | 17 +++++++++++
>  ...andler.S => thead_c9xx_tlb_trap_handler.S} |  0
>  7 files changed, 50 insertions(+), 18 deletions(-)
>  create mode 100644 platform/generic/include/thead/c9xx_errata.h
>  create mode 100644 platform/generic/thead/thead_c9xx_errata_tlb_flush.c
>  rename platform/generic/thead/{thead-trap-handler.S => thead_c9xx_tlb_trap_handler.S} (100%)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index ea3b5cf..3be75cf 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -55,6 +55,7 @@ config PLATFORM_STARFIVE_JH7110
>
>  config PLATFORM_THEAD
>         bool "THEAD C9xx support"
> +       select THEAD_C9XX_ERRATA
>         default n
>
>  source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
> diff --git a/platform/generic/include/thead/c9xx_errata.h b/platform/generic/include/thead/c9xx_errata.h
> new file mode 100644
> index 0000000..b617762
> --- /dev/null
> +++ b/platform/generic/include/thead/c9xx_errata.h
> @@ -0,0 +1,14 @@
> +
> +#ifndef __RISCV_THEAD_C9XX_ERRATA_H____
> +#define __RISCV_THEAD_C9XX_ERRATA_H____
> +
> +/**
> + * T-HEAD board with this quirk need to execute sfence.vma to flush
> + * stale entrie avoid incorrect memory access.
> + */
> +#define THEAD_QUIRK_ERRATA_TLB_FLUSH           BIT(0)
> +
> +void thead_register_tlb_flush_trap_handler(void);
> +
> +#endif // __RISCV_THEAD_C9XX_ERRATA_H____
> +
> diff --git a/platform/generic/thead/Kconfig b/platform/generic/thead/Kconfig
> index e54e621..c50d38e 100644
> --- a/platform/generic/thead/Kconfig
> +++ b/platform/generic/thead/Kconfig
> @@ -3,3 +3,7 @@
>  config THEAD_C9XX_PMU
>         bool "T-HEAD c9xx M-mode PMU support"
>         default n
> +
> +config THEAD_C9XX_ERRATA
> +       bool "T-HEAD c9xx errata support"
> +       default n
> diff --git a/platform/generic/thead/objects.mk b/platform/generic/thead/objects.mk
> index df2c61b..84f9faf 100644
> --- a/platform/generic/thead/objects.mk
> +++ b/platform/generic/thead/objects.mk
> @@ -7,6 +7,8 @@
>
>  platform-objs-$(CONFIG_THEAD_C9XX_PMU) += thead/thead_c9xx_pmu.o
>
> +platform-objs-$(CONFIG_THEAD_C9XX_ERRATA) += thead/thead_c9xx_tlb_trap_handler.o
> +platform-objs-$(CONFIG_THEAD_C9XX_ERRATA) += thead/thead_c9xx_errata_tlb_flush.o
> +
>  carray-platform_override_modules-$(CONFIG_PLATFORM_THEAD) += thead_generic
>  platform-objs-$(CONFIG_PLATFORM_THEAD) += thead/thead-generic.o
> -platform-objs-$(CONFIG_PLATFORM_THEAD) += thead/thead-trap-handler.o
> diff --git a/platform/generic/thead/thead-generic.c b/platform/generic/thead/thead-generic.c
> index 8120f6f..c77140c 100644
> --- a/platform/generic/thead/thead-generic.c
> +++ b/platform/generic/thead/thead-generic.c
> @@ -7,40 +7,34 @@
>   */
>
>  #include <platform_override.h>
> -#include <sbi/riscv_barrier.h>
> +#include <thead/c9xx_errata.h>
>  #include <sbi/sbi_const.h>
>  #include <sbi/sbi_platform.h>
>  #include <sbi/sbi_scratch.h>
>  #include <sbi/sbi_string.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
> -/**
> - * T-HEAD board with this quirk need to execute sfence.vma to flush
> - * stale entrie avoid incorrect memory access.
> - */
> -#define THEAD_QUIRK_TLB_FLUSH_FIXUP            BIT(0)
> -
> -void _thead_tlb_flush_fixup_trap_handler(void);
> -
> -void thead_register_tlb_flush_trap_handler(void)
> -{
> -       csr_write(CSR_MTVEC, &_thead_tlb_flush_fixup_trap_handler);
> -}
> +struct thead_generic_quirks {
> +       u64     errata;
> +};
>
>  static int thead_generic_early_init(bool cold_boot,
>                                     const struct fdt_match *match)
>  {
> -       unsigned long quirks = (unsigned long)match->data;
> +       struct thead_generic_quirks *quirks = (void *)match->data;
>
> -       if (quirks & THEAD_QUIRK_TLB_FLUSH_FIXUP)
> +       if (quirks->errata & THEAD_QUIRK_ERRATA_TLB_FLUSH)
>                 thead_register_tlb_flush_trap_handler();
>
>         return 0;
>  }
>
> +static struct thead_generic_quirks thead_th1520_quirks = {
> +       .errata = THEAD_QUIRK_ERRATA_TLB_FLUSH,
> +};
> +
>  static const struct fdt_match thead_generic_match[] = {
> -       { .compatible = "thead,th1520",
> -         .data = (void*)THEAD_QUIRK_TLB_FLUSH_FIXUP },
> +       { .compatible = "thead,th1520", .data = &thead_th1520_quirks },
>         { },
>  };
>
> diff --git a/platform/generic/thead/thead_c9xx_errata_tlb_flush.c b/platform/generic/thead/thead_c9xx_errata_tlb_flush.c
> new file mode 100644
> index 0000000..755899a
> --- /dev/null
> +++ b/platform/generic/thead/thead_c9xx_errata_tlb_flush.c
> @@ -0,0 +1,17 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Authors:
> + *   Inochi Amaoto <inochiama at outlook.com>
> + *
> + */
> +
> +#include <sbi/riscv_encoding.h>
> +#include <sbi/riscv_asm.h>
> +
> +void _thead_tlb_flush_fixup_trap_handler(void);
> +
> +void thead_register_tlb_flush_trap_handler(void)
> +{
> +       csr_write(CSR_MTVEC, &_thead_tlb_flush_fixup_trap_handler);
> +}
> diff --git a/platform/generic/thead/thead-trap-handler.S b/platform/generic/thead/thead_c9xx_tlb_trap_handler.S
> similarity index 100%
> rename from platform/generic/thead/thead-trap-handler.S
> rename to platform/generic/thead/thead_c9xx_tlb_trap_handler.S
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 07:46:09 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 21:16:09 +0530
Subject: [PATCH v2 4/4] platform: generic: thead: initialize PMU by
 default in thead generic platform
In-Reply-To: <IA1PR20MB49536C4C0A19D97FA8E288CBBBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953287B7688E0F5058984BDBBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
 <20231005065843.239501-1-inochiama@outlook.com> <IA1PR20MB49536C4C0A19D97FA8E288CBBBCAA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy2fuwtgq8SUmxCu6i8nOFv=SbWOvT0cESCq4LmYZRSRqQ@mail.gmail.com>

On Thu, Oct 5, 2023 at 12:29?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Since all the SoC with thead c9xx cores need this initialization at now,
> initialize the c9xx pmu in the thead generic platform by default.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/Kconfig               |  1 +
>  platform/generic/thead/thead-generic.c | 13 +++++++++++--
>  2 files changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index 3be75cf..f9e7ed2 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -56,6 +56,7 @@ config PLATFORM_STARFIVE_JH7110
>  config PLATFORM_THEAD
>         bool "THEAD C9xx support"
>         select THEAD_C9XX_ERRATA
> +       select THEAD_C9XX_PMU
>         default n
>
>  source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
> diff --git a/platform/generic/thead/thead-generic.c b/platform/generic/thead/thead-generic.c
> index c77140c..72f4aaa 100644
> --- a/platform/generic/thead/thead-generic.c
> +++ b/platform/generic/thead/thead-generic.c
> @@ -8,6 +8,7 @@
>
>  #include <platform_override.h>
>  #include <thead/c9xx_errata.h>
> +#include <thead/c9xx_pmu.h>
>  #include <sbi/sbi_const.h>
>  #include <sbi/sbi_platform.h>
>  #include <sbi/sbi_scratch.h>
> @@ -29,6 +30,13 @@ static int thead_generic_early_init(bool cold_boot,
>         return 0;
>  }
>
> +static int thead_generic_extensions_init(const struct fdt_match *match,
> +                                        struct sbi_hart_features *hfeatures)
> +{
> +       thead_c9xx_register_pmu_device();
> +       return 0;
> +}
> +
>  static struct thead_generic_quirks thead_th1520_quirks = {
>         .errata = THEAD_QUIRK_ERRATA_TLB_FLUSH,
>  };
> @@ -39,6 +47,7 @@ static const struct fdt_match thead_generic_match[] = {
>  };
>
>  const struct platform_override thead_generic = {
> -       .match_table    = thead_generic_match,
> -       .early_init     = thead_generic_early_init,
> +       .match_table            = thead_generic_match,
> +       .early_init             = thead_generic_early_init,
> +       .extensions_init        = thead_generic_extensions_init,
>  };
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 08:24:51 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 21:54:51 +0530
Subject: [PATCH v3 1/4] platform: generic: thead: move to thead c9xx
 header to vendor specific postion
In-Reply-To: <IA1PR20MB4953402F2CEAEA2038B7A8F7BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49534E329372B295663B2153BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB4953402F2CEAEA2038B7A8F7BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy0be2eMdMfb5s6NPZES1C3UCDCcaS6h96V+CUY3yh7aHg@mail.gmail.com>

On Sat, Oct 7, 2023 at 8:36?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> The CSR encoding for t-head c9xx cores is shared across all the
> platforms with these cores. So move header thead_c9xx.h to the
> thead subdir.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> Acked-by: Guo Ren <guoren at kernel.org>

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/allwinner/sun20i-d1.c                        | 2 +-
>  .../generic/include/{thead_c9xx.h => thead/c9xx_encoding.h}   | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
>  rename platform/generic/include/{thead_c9xx.h => thead/c9xx_encoding.h} (98%)
>
> diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
> index a9a7e47..b960284 100644
> --- a/platform/generic/allwinner/sun20i-d1.c
> +++ b/platform/generic/allwinner/sun20i-d1.c
> @@ -5,7 +5,7 @@
>   */
>
>  #include <platform_override.h>
> -#include <thead_c9xx.h>
> +#include <thead/c9xx_encoding.h>
>  #include <sbi/riscv_asm.h>
>  #include <sbi/riscv_io.h>
>  #include <sbi/sbi_bitops.h>
> diff --git a/platform/generic/include/thead_c9xx.h b/platform/generic/include/thead/c9xx_encoding.h
> similarity index 98%
> rename from platform/generic/include/thead_c9xx.h
> rename to platform/generic/include/thead/c9xx_encoding.h
> index bab0408..58adbef 100644
> --- a/platform/generic/include/thead_c9xx.h
> +++ b/platform/generic/include/thead/c9xx_encoding.h
> @@ -1,5 +1,5 @@
> -#ifndef __RISCV_THEAD_C9XX_H____
> -#define __RISCV_THEAD_C9XX_H____
> +#ifndef __RISCV_THEAD_C9XX_ENCODING_H____
> +#define __RISCV_THEAD_C9XX_ENCODING_H____
>
>  /* T-HEAD C9xx M mode CSR.  */
>  #define THEAD_C9XX_CSR_MXSTATUS                0x7c0
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 08:25:06 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 21:55:06 +0530
Subject: [PATCH v3 2/4] platform: generic: thead: separate implement of
 T-HEAD c9xx pmu
In-Reply-To: <IA1PR20MB49531384A849F05A47A17539BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49534E329372B295663B2153BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <20231007030631.101073-1-inochiama@outlook.com> <IA1PR20MB49531384A849F05A47A17539BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy1trNQkNXMm8sW_JuWeWz+0MHUvanVGp-eNLvSwDFNhvQ@mail.gmail.com>

On Sat, Oct 7, 2023 at 8:37?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Separate the implement of T-HEAD c9xx pmu to allow any platform with
> c9xx cores can use it.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/Kconfig                  |  2 +
>  platform/generic/allwinner/sun20i-d1.c    | 51 +----------------
>  platform/generic/include/thead/c9xx_pmu.h |  7 +++
>  platform/generic/thead/Kconfig            |  5 ++
>  platform/generic/thead/objects.mk         |  2 +
>  platform/generic/thead/thead_c9xx_pmu.c   | 69 +++++++++++++++++++++++
>  6 files changed, 87 insertions(+), 49 deletions(-)
>  create mode 100644 platform/generic/include/thead/c9xx_pmu.h
>  create mode 100644 platform/generic/thead/Kconfig
>  create mode 100644 platform/generic/thead/thead_c9xx_pmu.c
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index e7bd94e..ea3b5cf 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -26,6 +26,7 @@ config PLATFORM_GENERIC_MINOR_VER
>  config PLATFORM_ALLWINNER_D1
>         bool "Allwinner D1 support"
>         depends on FDT_IRQCHIP_PLIC
> +       select THEAD_C9XX_PMU
>         default n
>
>  config PLATFORM_ANDES_AE350
> @@ -57,5 +58,6 @@ config PLATFORM_THEAD
>         default n
>
>  source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
> +source "$(OPENSBI_SRC_DIR)/platform/generic/thead/Kconfig"
>
>  endif
> diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
> index b960284..e9388db 100644
> --- a/platform/generic/allwinner/sun20i-d1.c
> +++ b/platform/generic/allwinner/sun20i-d1.c
> @@ -6,6 +6,7 @@
>
>  #include <platform_override.h>
>  #include <thead/c9xx_encoding.h>
> +#include <thead/c9xx_pmu.h>
>  #include <sbi/riscv_asm.h>
>  #include <sbi/riscv_io.h>
>  #include <sbi/sbi_bitops.h>
> @@ -223,58 +224,10 @@ static int sun20i_d1_fdt_fixup(void *fdt, const struct fdt_match *match)
>         return fdt_add_cpu_idle_states(fdt, sun20i_d1_cpu_idle_states);
>  }
>
> -static void thead_c9xx_pmu_ctr_enable_irq(uint32_t ctr_idx)
> -{
> -       if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
> -               return;
> -
> -       /**
> -        * Clear out the OF bit so that next interrupt can be enabled.
> -        * This should be done before starting interrupt to avoid unexcepted
> -        * overflow interrupt.
> -        */
> -       csr_clear(THEAD_C9XX_CSR_MCOUNTEROF, BIT(ctr_idx));
> -
> -       /**
> -        * This register is described in C9xx document as the control register
> -        * for enabling writes to the superuser state counter. However, if the
> -        * corresponding bit is not set to 1, scounterof will always read as 0
> -        * when the counter register overflows.
> -        */
> -       csr_set(THEAD_C9XX_CSR_MCOUNTERWEN, BIT(ctr_idx));
> -
> -       /**
> -        * SSCOFPMF uses the OF bit for enabling/disabling the interrupt,
> -        * while the C9XX has designated enable bits.
> -        * So enable per-counter interrupt on C9xx here.
> -        */
> -       csr_set(THEAD_C9XX_CSR_MCOUNTERINTEN, BIT(ctr_idx));
> -}
> -
> -static void thead_c9xx_pmu_ctr_disable_irq(uint32_t ctr_idx)
> -{
> -       /**
> -        * There is no need to clear the bit of mcounterwen, it will expire
> -        * after setting the csr mcountinhibit.
> -        */
> -       csr_clear(THEAD_C9XX_CSR_MCOUNTERINTEN, BIT(ctr_idx));
> -}
> -
> -static int thead_c9xx_pmu_irq_bit(void)
> -{
> -       return THEAD_C9XX_MIP_MOIP;
> -}
> -
> -const struct sbi_pmu_device thead_c9xx_pmu_device = {
> -       .hw_counter_enable_irq = thead_c9xx_pmu_ctr_enable_irq,
> -       .hw_counter_disable_irq = thead_c9xx_pmu_ctr_disable_irq,
> -       .hw_counter_irq_bit = thead_c9xx_pmu_irq_bit,
> -};
> -
>  static int sun20i_d1_extensions_init(const struct fdt_match *match,
>                                      struct sbi_hart_features *hfeatures)
>  {
> -       sbi_pmu_set_device(&thead_c9xx_pmu_device);
> +       thead_c9xx_register_pmu_device();
>
>         /* auto-detection doesn't work on t-head c9xx cores */
>         /* D1 has 29 mhpmevent csrs, but only 3-9,13-17 have valid value */
> diff --git a/platform/generic/include/thead/c9xx_pmu.h b/platform/generic/include/thead/c9xx_pmu.h
> new file mode 100644
> index 0000000..bb1d831
> --- /dev/null
> +++ b/platform/generic/include/thead/c9xx_pmu.h
> @@ -0,0 +1,7 @@
> +
> +#ifndef __RISCV_THEAD_C9XX_PMU_H____
> +#define __RISCV_THEAD_C9XX_PMU_H____
> +
> +void thead_c9xx_register_pmu_device(void);
> +
> +#endif // __RISCV_THEAD_C9XX_PMU_H____
> diff --git a/platform/generic/thead/Kconfig b/platform/generic/thead/Kconfig
> new file mode 100644
> index 0000000..e54e621
> --- /dev/null
> +++ b/platform/generic/thead/Kconfig
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: BSD-2-Clause
> +
> +config THEAD_C9XX_PMU
> +       bool "T-HEAD c9xx M-mode PMU support"
> +       default n
> diff --git a/platform/generic/thead/objects.mk b/platform/generic/thead/objects.mk
> index 854bfbd..df2c61b 100644
> --- a/platform/generic/thead/objects.mk
> +++ b/platform/generic/thead/objects.mk
> @@ -5,6 +5,8 @@
>  # Copyright (C) 2023 Alibaba Group Holding Limited.
>  #
>
> +platform-objs-$(CONFIG_THEAD_C9XX_PMU) += thead/thead_c9xx_pmu.o
> +
>  carray-platform_override_modules-$(CONFIG_PLATFORM_THEAD) += thead_generic
>  platform-objs-$(CONFIG_PLATFORM_THEAD) += thead/thead-generic.o
>  platform-objs-$(CONFIG_PLATFORM_THEAD) += thead/thead-trap-handler.o
> diff --git a/platform/generic/thead/thead_c9xx_pmu.c b/platform/generic/thead/thead_c9xx_pmu.c
> new file mode 100644
> index 0000000..d205802
> --- /dev/null
> +++ b/platform/generic/thead/thead_c9xx_pmu.c
> @@ -0,0 +1,69 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Authors:
> + *   Inochi Amaoto <inochiama at outlook.com>
> + *   Haijiao Liu <haijiao.liu at sophgo.com>
> + *
> + */
> +
> +#include <thead/c9xx_encoding.h>
> +#include <thead/c9xx_pmu.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/sbi_bitops.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi/sbi_pmu.h>
> +
> +static void thead_c9xx_pmu_ctr_enable_irq(uint32_t ctr_idx)
> +{
> +       if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
> +               return;
> +
> +       /**
> +        * Clear out the OF bit so that next interrupt can be enabled.
> +        * This should be done before starting interrupt to avoid unexcepted
> +        * overflow interrupt.
> +        */
> +       csr_clear(THEAD_C9XX_CSR_MCOUNTEROF, BIT(ctr_idx));
> +
> +       /**
> +        * This register is described in C9xx document as the control register
> +        * for enabling writes to the superuser state counter. However, if the
> +        * corresponding bit is not set to 1, scounterof will always read as 0
> +        * when the counter register overflows.
> +        */
> +       csr_set(THEAD_C9XX_CSR_MCOUNTERWEN, BIT(ctr_idx));
> +
> +       /**
> +        * SSCOFPMF uses the OF bit for enabling/disabling the interrupt,
> +        * while the C9XX has designated enable bits.
> +        * So enable per-counter interrupt on C9xx here.
> +        */
> +       csr_set(THEAD_C9XX_CSR_MCOUNTERINTEN, BIT(ctr_idx));
> +}
> +
> +static void thead_c9xx_pmu_ctr_disable_irq(uint32_t ctr_idx)
> +{
> +       /**
> +        * There is no need to clear the bit of mcounterwen, it will expire
> +        * after setting the csr mcountinhibit.
> +        */
> +       csr_clear(THEAD_C9XX_CSR_MCOUNTERINTEN, BIT(ctr_idx));
> +}
> +
> +static int thead_c9xx_pmu_irq_bit(void)
> +{
> +       return THEAD_C9XX_MIP_MOIP;
> +}
> +
> +static const struct sbi_pmu_device thead_c9xx_pmu_device = {
> +       .name = "thead,c900-pmu",
> +       .hw_counter_enable_irq = thead_c9xx_pmu_ctr_enable_irq,
> +       .hw_counter_disable_irq = thead_c9xx_pmu_ctr_disable_irq,
> +       .hw_counter_irq_bit = thead_c9xx_pmu_irq_bit,
> +};
> +
> +void thead_c9xx_register_pmu_device(void)
> +{
> +       sbi_pmu_set_device(&thead_c9xx_pmu_device);
> +}
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 08:25:20 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 21:55:20 +0530
Subject: [PATCH v3 3/4] platform: generic: thead: separate implement of
 T-HEAD c9xx errata
In-Reply-To: <IA1PR20MB495399EC057D782C694A6A27BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49534E329372B295663B2153BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <20231007030631.101073-1-inochiama@outlook.com> <IA1PR20MB495399EC057D782C694A6A27BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy00amz8g88cQhbDMkRNntVK18UPxU0NgtiVoqN=mc289A@mail.gmail.com>

On Sat, Oct 7, 2023 at 8:37?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Separate the implement of T-HEAD c9xx errata to allow any platform
> with bug related to c9xx cores can use it.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/Kconfig                      |  1 +
>  platform/generic/include/thead/c9xx_errata.h  | 13 +++++++++
>  platform/generic/thead/Kconfig                |  4 +++
>  platform/generic/thead/objects.mk             |  4 ++-
>  platform/generic/thead/thead-generic.c        | 28 ++++++++-----------
>  .../thead/thead_c9xx_errata_tlb_flush.c       | 17 +++++++++++
>  ...andler.S => thead_c9xx_tlb_trap_handler.S} |  0
>  7 files changed, 49 insertions(+), 18 deletions(-)
>  create mode 100644 platform/generic/include/thead/c9xx_errata.h
>  create mode 100644 platform/generic/thead/thead_c9xx_errata_tlb_flush.c
>  rename platform/generic/thead/{thead-trap-handler.S => thead_c9xx_tlb_trap_handler.S} (100%)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index ea3b5cf..3be75cf 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -55,6 +55,7 @@ config PLATFORM_STARFIVE_JH7110
>
>  config PLATFORM_THEAD
>         bool "THEAD C9xx support"
> +       select THEAD_C9XX_ERRATA
>         default n
>
>  source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
> diff --git a/platform/generic/include/thead/c9xx_errata.h b/platform/generic/include/thead/c9xx_errata.h
> new file mode 100644
> index 0000000..bf97938
> --- /dev/null
> +++ b/platform/generic/include/thead/c9xx_errata.h
> @@ -0,0 +1,13 @@
> +
> +#ifndef __RISCV_THEAD_C9XX_ERRATA_H____
> +#define __RISCV_THEAD_C9XX_ERRATA_H____
> +
> +/**
> + * T-HEAD board with this quirk need to execute sfence.vma to flush
> + * stale entrie avoid incorrect memory access.
> + */
> +#define THEAD_QUIRK_ERRATA_TLB_FLUSH           BIT(0)
> +
> +void thead_register_tlb_flush_trap_handler(void);
> +
> +#endif // __RISCV_THEAD_C9XX_ERRATA_H____
> diff --git a/platform/generic/thead/Kconfig b/platform/generic/thead/Kconfig
> index e54e621..c50d38e 100644
> --- a/platform/generic/thead/Kconfig
> +++ b/platform/generic/thead/Kconfig
> @@ -3,3 +3,7 @@
>  config THEAD_C9XX_PMU
>         bool "T-HEAD c9xx M-mode PMU support"
>         default n
> +
> +config THEAD_C9XX_ERRATA
> +       bool "T-HEAD c9xx errata support"
> +       default n
> diff --git a/platform/generic/thead/objects.mk b/platform/generic/thead/objects.mk
> index df2c61b..84f9faf 100644
> --- a/platform/generic/thead/objects.mk
> +++ b/platform/generic/thead/objects.mk
> @@ -7,6 +7,8 @@
>
>  platform-objs-$(CONFIG_THEAD_C9XX_PMU) += thead/thead_c9xx_pmu.o
>
> +platform-objs-$(CONFIG_THEAD_C9XX_ERRATA) += thead/thead_c9xx_tlb_trap_handler.o
> +platform-objs-$(CONFIG_THEAD_C9XX_ERRATA) += thead/thead_c9xx_errata_tlb_flush.o
> +
>  carray-platform_override_modules-$(CONFIG_PLATFORM_THEAD) += thead_generic
>  platform-objs-$(CONFIG_PLATFORM_THEAD) += thead/thead-generic.o
> -platform-objs-$(CONFIG_PLATFORM_THEAD) += thead/thead-trap-handler.o
> diff --git a/platform/generic/thead/thead-generic.c b/platform/generic/thead/thead-generic.c
> index 8120f6f..c77140c 100644
> --- a/platform/generic/thead/thead-generic.c
> +++ b/platform/generic/thead/thead-generic.c
> @@ -7,40 +7,34 @@
>   */
>
>  #include <platform_override.h>
> -#include <sbi/riscv_barrier.h>
> +#include <thead/c9xx_errata.h>
>  #include <sbi/sbi_const.h>
>  #include <sbi/sbi_platform.h>
>  #include <sbi/sbi_scratch.h>
>  #include <sbi/sbi_string.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
> -/**
> - * T-HEAD board with this quirk need to execute sfence.vma to flush
> - * stale entrie avoid incorrect memory access.
> - */
> -#define THEAD_QUIRK_TLB_FLUSH_FIXUP            BIT(0)
> -
> -void _thead_tlb_flush_fixup_trap_handler(void);
> -
> -void thead_register_tlb_flush_trap_handler(void)
> -{
> -       csr_write(CSR_MTVEC, &_thead_tlb_flush_fixup_trap_handler);
> -}
> +struct thead_generic_quirks {
> +       u64     errata;
> +};
>
>  static int thead_generic_early_init(bool cold_boot,
>                                     const struct fdt_match *match)
>  {
> -       unsigned long quirks = (unsigned long)match->data;
> +       struct thead_generic_quirks *quirks = (void *)match->data;
>
> -       if (quirks & THEAD_QUIRK_TLB_FLUSH_FIXUP)
> +       if (quirks->errata & THEAD_QUIRK_ERRATA_TLB_FLUSH)
>                 thead_register_tlb_flush_trap_handler();
>
>         return 0;
>  }
>
> +static struct thead_generic_quirks thead_th1520_quirks = {
> +       .errata = THEAD_QUIRK_ERRATA_TLB_FLUSH,
> +};
> +
>  static const struct fdt_match thead_generic_match[] = {
> -       { .compatible = "thead,th1520",
> -         .data = (void*)THEAD_QUIRK_TLB_FLUSH_FIXUP },
> +       { .compatible = "thead,th1520", .data = &thead_th1520_quirks },
>         { },
>  };
>
> diff --git a/platform/generic/thead/thead_c9xx_errata_tlb_flush.c b/platform/generic/thead/thead_c9xx_errata_tlb_flush.c
> new file mode 100644
> index 0000000..755899a
> --- /dev/null
> +++ b/platform/generic/thead/thead_c9xx_errata_tlb_flush.c
> @@ -0,0 +1,17 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Authors:
> + *   Inochi Amaoto <inochiama at outlook.com>
> + *
> + */
> +
> +#include <sbi/riscv_encoding.h>
> +#include <sbi/riscv_asm.h>
> +
> +void _thead_tlb_flush_fixup_trap_handler(void);
> +
> +void thead_register_tlb_flush_trap_handler(void)
> +{
> +       csr_write(CSR_MTVEC, &_thead_tlb_flush_fixup_trap_handler);
> +}
> diff --git a/platform/generic/thead/thead-trap-handler.S b/platform/generic/thead/thead_c9xx_tlb_trap_handler.S
> similarity index 100%
> rename from platform/generic/thead/thead-trap-handler.S
> rename to platform/generic/thead/thead_c9xx_tlb_trap_handler.S
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 08:25:35 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 21:55:35 +0530
Subject: [PATCH v3 4/4] platform: generic: thead: initialize PMU by
 default in thead generic platform
In-Reply-To: <IA1PR20MB49536E28A3FC24C353A8DC04BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49534E329372B295663B2153BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <20231007030631.101073-1-inochiama@outlook.com> <IA1PR20MB49536E28A3FC24C353A8DC04BBC8A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy2wUoS6EBFiATeZfWjft9NdSrWzn9r+Z3p8-XJyAhuw=A@mail.gmail.com>

On Sat, Oct 7, 2023 at 8:37?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Since all the SoC with thead c9xx cores need this initialization at now,
> initialize the c9xx pmu in the thead generic platform by default.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/Kconfig               |  1 +
>  platform/generic/thead/thead-generic.c | 13 +++++++++++--
>  2 files changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index 3be75cf..f9e7ed2 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -56,6 +56,7 @@ config PLATFORM_STARFIVE_JH7110
>  config PLATFORM_THEAD
>         bool "THEAD C9xx support"
>         select THEAD_C9XX_ERRATA
> +       select THEAD_C9XX_PMU
>         default n
>
>  source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
> diff --git a/platform/generic/thead/thead-generic.c b/platform/generic/thead/thead-generic.c
> index c77140c..72f4aaa 100644
> --- a/platform/generic/thead/thead-generic.c
> +++ b/platform/generic/thead/thead-generic.c
> @@ -8,6 +8,7 @@
>
>  #include <platform_override.h>
>  #include <thead/c9xx_errata.h>
> +#include <thead/c9xx_pmu.h>
>  #include <sbi/sbi_const.h>
>  #include <sbi/sbi_platform.h>
>  #include <sbi/sbi_scratch.h>
> @@ -29,6 +30,13 @@ static int thead_generic_early_init(bool cold_boot,
>         return 0;
>  }
>
> +static int thead_generic_extensions_init(const struct fdt_match *match,
> +                                        struct sbi_hart_features *hfeatures)
> +{
> +       thead_c9xx_register_pmu_device();
> +       return 0;
> +}
> +
>  static struct thead_generic_quirks thead_th1520_quirks = {
>         .errata = THEAD_QUIRK_ERRATA_TLB_FLUSH,
>  };
> @@ -39,6 +47,7 @@ static const struct fdt_match thead_generic_match[] = {
>  };
>
>  const struct platform_override thead_generic = {
> -       .match_table    = thead_generic_match,
> -       .early_init     = thead_generic_early_init,
> +       .match_table            = thead_generic_match,
> +       .early_init             = thead_generic_early_init,
> +       .extensions_init        = thead_generic_extensions_init,
>  };
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 08:36:01 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 22:06:01 +0530
Subject: [PATCH v2 0/4] platform: generic: Add Sophgo sg2042 support
In-Reply-To: <IA1PR20MB4953CAA2C31F07F9A1157EF7BBCFA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953CAA2C31F07F9A1157EF7BBCFA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy1HRNAE6g1cTi_i3fRf1hJjp4oPfF-KnjPg77qq-yzivA@mail.gmail.com>

On Sun, Oct 8, 2023 at 5:46?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> As the dts of sg2042 in the mainline kernel is considered as stable, it is
> time to port sg2042 soc.
>
> This patch depends other two series:
> 1. platform: generic: add T-HEAD th1520 soc support
> 2. lib: sbi: Add sub-regions check for sanitizing domain
>
> Link: https://lore.kernel.org/linux-riscv/cover.1696433229.git.unicorn_wang at outlook.com/
>
> Changed from v1:
> 1. make number of entries in tlb fifo auto deteted.
>
> Inochi Amaoto (4):
>   lib: utils/timer: mtimer: add separate T-Head C9xx CLINT mtimer
>     compatible
>   lib: utils/ipi: mswi: add separate T-Head C9xx CLINT mswi compatible
>   platform: Allow platform uses hart count as the size of tlb info
>   platform: generic: Add Sophgo sg2042 platform support

There are unaddressed comments in this series. I assume there will
be a v3.

Regards,
Anup

>
>  lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
>  lib/utils/timer/fdt_timer_mtimer.c |  2 +
>  platform/generic/Kconfig           |  6 +++
>  platform/generic/configs/defconfig |  1 +
>  platform/generic/platform.c        |  7 ++++
>  platform/generic/sophgo/objects.mk |  9 +++++
>  platform/generic/sophgo/sg2042.c   | 64 ++++++++++++++++++++++++++++++
>  7 files changed, 90 insertions(+)
>  create mode 100644 platform/generic/sophgo/objects.mk
>  create mode 100644 platform/generic/sophgo/sg2042.c
>
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 08:39:23 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 22:09:23 +0530
Subject: [PATCH] lib: sbi: Improve hartindex_to_hartid_table
In-Reply-To: <20231009092202.267633-1-wxjstz@126.com>
References: <20231009092202.267633-1-wxjstz@126.com>
Message-ID: <CAAhSdy3rEn0FKSyH1HL0VEx2Se5o62U9cYFFWeh_vGtS20B_mA@mail.gmail.com>

On Mon, Oct 9, 2023 at 2:52?PM Xiang W <wxjstz at 126.com> wrote:
>
> Modify hartindex_to_hartid_table point to plat->hart_index2id to
> save memory. This slows down sbi_hartindex_to_hartid slightly
> because it needs to determine whether hartindex_to_hartid_table is
> NULL. This will also improve the speed of sbi_hartid_to_hartindex
> when plat->hart_index2id is NULL, since there is no need to iterate
> through the array.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> ---
>  include/sbi/sbi_scratch.h | 13 ++++++++-----
>  lib/sbi/sbi_scratch.c     |  7 +++++--
>  2 files changed, 13 insertions(+), 7 deletions(-)
>
> diff --git a/include/sbi/sbi_scratch.h b/include/sbi/sbi_scratch.h
> index e6a33ba..02db224 100644
> --- a/include/sbi/sbi_scratch.h
> +++ b/include/sbi/sbi_scratch.h
> @@ -213,13 +213,16 @@ extern u32 last_hartindex_having_scratch;
>  (((__hartindex) <= sbi_scratch_last_hartindex()) ? true : false)
>
>  /** HART index to HART id table */
> -extern u32 hartindex_to_hartid_table[];
> +extern const u32* hartindex_to_hartid_table;
>
>  /** Get sbi_scratch from HART index */
> -#define sbi_hartindex_to_hartid(__hartindex)           \
> -({                                                     \
> -       ((__hartindex) <= sbi_scratch_last_hartindex()) ?\
> -       hartindex_to_hartid_table[__hartindex] : -1U;   \
> +#define sbi_hartindex_to_hartid(__hartindex)                   \
> +({                                                             \
> +       ((__hartindex) <= sbi_scratch_last_hartindex()) ?       \
> +               (hartindex_to_hartid_table ?                    \
> +                       hartindex_to_hartid_table[__hartindex] :\
> +                       __hartindex) :                          \
> +               -1U;                                            \

Now, we have one more unnecessary "if ()" check in the
hot-path.

This patch deteriorates sbi_hartindex_to_hartid() instead
of improving it.

Regards,
Anup

>  })
>
>  /** HART index to scratch table */
> diff --git a/lib/sbi/sbi_scratch.c b/lib/sbi/sbi_scratch.c
> index ccbbc68..e097077 100644
> --- a/lib/sbi/sbi_scratch.c
> +++ b/lib/sbi/sbi_scratch.c
> @@ -15,7 +15,7 @@
>  #include <sbi/sbi_string.h>
>
>  u32 last_hartindex_having_scratch = 0;
> -u32 hartindex_to_hartid_table[SBI_HARTMASK_MAX_BITS + 1] = { -1U };
> +const u32 *hartindex_to_hartid_table;
>  struct sbi_scratch *hartindex_to_scratch_table[SBI_HARTMASK_MAX_BITS + 1] = { 0 };
>
>  static spinlock_t extra_lock = SPIN_LOCK_INITIALIZER;
> @@ -25,6 +25,9 @@ u32 sbi_hartid_to_hartindex(u32 hartid)
>  {
>         u32 i;
>
> +       if (hartindex_to_hartid_table == NULL)
> +               return hartid;
> +
>         for (i = 0; i <= last_hartindex_having_scratch; i++)
>                 if (hartindex_to_hartid_table[i] == hartid)
>                         return i;
> @@ -41,11 +44,11 @@ int sbi_scratch_init(struct sbi_scratch *scratch)
>
>         for (i = 0; i < plat->hart_count; i++) {
>                 h = (plat->hart_index2id) ? plat->hart_index2id[i] : i;
> -               hartindex_to_hartid_table[i] = h;
>                 hartindex_to_scratch_table[i] =
>                         ((hartid2scratch)scratch->hartid_to_scratch)(h, i);
>         }
>
> +       hartindex_to_hartid_table = plat->hart_index2id;
>         last_hartindex_having_scratch = plat->hart_count - 1;
>
>         return 0;
> --
> 2.40.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 08:45:58 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 22:15:58 +0530
Subject: [PATCH v2 0/5] Add Svadu extension support
In-Reply-To: <20231024101145.32000-1-yongxuan.wang@sifive.com>
References: <20231024101145.32000-1-yongxuan.wang@sifive.com>
Message-ID: <CAAhSdy2ALrnjXMUoA0AondR1-pEPyLCafsZX=N=d2gN7WL3nGQ@mail.gmail.com>

On Tue, Oct 24, 2023 at 3:42?PM Yong-Xuan Wang <yongxuan.wang at sifive.com> wrote:
>
> This series enables Svadu extension support by configuring the menvcfg
> CSR and, if available, displays the Svadu extension in the boot log.
>
> Additionally, we've made some programming improvements in
> lib/sbi/sbi_hart.c and lib/utils/fdt/fdt_helper.c.
>
> ---
> v2:
> - Rearrange the patches to do the code refactoring first before adding
>   new features
> - Suggested by Anup and Atish, detect extensions from DT instead of
>   menvcfg CSR
> - Enable access to some extensions through menvcfg CSR if they are
>   present in the device tree.
>
> Yong-Xuan Wang (5):
>   lib: sbi: Improve the code of privilege mode and extensions detection
>   lib: sbi: Refactor the code for enable extensions in menvfg CSR
>   lib: sbi: Using one array to define the name of extensions
>   lib: sbi: Detect extensions from the ISA string in DT
>   lib: sbi: Add support for Svadu extension

For backward compatibility with existing OSes, it is better to have
supervisor OS explicitly enable Svadu using the upcoming SBI
FWFT extension instead of enabling it unconditionally whenever
Svadu extension is available.

Regards,
Anup

>
>  include/sbi/riscv_encoding.h |   6 +-
>  include/sbi/sbi_hart.h       |  15 +++
>  lib/sbi/sbi_hart.c           | 226 +++++++++++++----------------------
>  lib/utils/fdt/fdt_helper.c   |   5 +-
>  4 files changed, 104 insertions(+), 148 deletions(-)
>
> --
> 2.17.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Nov 14 08:56:07 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 22:26:07 +0530
Subject: [PATCH v2 1/3] lib: sbi: separate the swap operation of domain region
In-Reply-To: <IA1PR20MB49533CD1EAD7784957CF4361BBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49533A7AB5888E0117F7799BBBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB49533CD1EAD7784957CF4361BBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy3f2gnRPbKy0i1QSRo8xht5EwMaj60yQ9h+GLhNQX4B-w@mail.gmail.com>

On Tue, Oct 10, 2023 at 6:30?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Swapping domain region is a common operation when sorting domain region,
> so separate it as a function to make code clean.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Thanks,
Anup

> ---
>  lib/sbi/sbi_domain.c | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index fd4a296..283b598 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -256,11 +256,21 @@ static const struct sbi_domain_memregion *find_next_subset_region(
>         return ret;
>  }
>
> +static void swap_region(struct sbi_domain_memregion* reg1,
> +                       struct sbi_domain_memregion* reg2)
> +{
> +       struct sbi_domain_memregion treg;
> +
> +       sbi_memcpy(&treg, reg1, sizeof(treg));
> +       sbi_memcpy(reg1, reg2, sizeof(treg));
> +       sbi_memcpy(reg2, &treg, sizeof(treg));
> +}
> +
>  static int sanitize_domain(const struct sbi_platform *plat,
>                            struct sbi_domain *dom)
>  {
>         u32 i, j, count;
> -       struct sbi_domain_memregion treg, *reg, *reg1;
> +       struct sbi_domain_memregion *reg, *reg1;
>
>         /* Check possible HARTs */
>         if (!dom->possible_harts) {
> @@ -324,9 +334,7 @@ static int sanitize_domain(const struct sbi_platform *plat,
>                         if (!is_region_before(reg1, reg))
>                                 continue;
>
> -                       sbi_memcpy(&treg, reg1, sizeof(treg));
> -                       sbi_memcpy(reg1, reg, sizeof(treg));
> -                       sbi_memcpy(reg, &treg, sizeof(treg));
> +                       swap_region(reg, reg1);
>                 }
>         }
>
> --
> 2.42.0
>


From anup at brainfault.org  Tue Nov 14 09:13:01 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 14 Nov 2023 22:43:01 +0530
Subject: [PATCH v2 2/3] lib: sbi: add more restriction to domain region
 subset check
In-Reply-To: <IA1PR20MB495351E19560D833EAC11BC8BBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49533A7AB5888E0117F7799BBBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB495351E19560D833EAC11BC8BBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy0fDAUA3yM2faj=0ObfKz-Nx7ioi=cySvNKjD1aRqpnQg@mail.gmail.com>

On Tue, Oct 10, 2023 at 6:30?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> make is_region_subset() check the bound of the domain region, so it can
> handle the domain region with the same base and length.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Important fact here is that regB_start < regB_end and
regA_start < regA_end because order == 0 region is of
size 1 byte.

When regB_start == regA_start and regB_end == regA_end,
the following is the outcome of the expressions in the existing
"if ()" check.

(regB_start <= regA_start) == true
(regA_start < regB_end) == true
(regB_start < regA_end) == true
(regA_end <= regB_end) == true

Based on the above, I don't see how this patch helps.

Regards,
Anup



> ---
>  lib/sbi/sbi_domain.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index 283b598..2b9d16f 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -185,8 +185,8 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
>         ulong regB_end = regB->base + (BIT(regB->order) - 1);
>
>         if ((regB_start <= regA_start) &&
> -           (regA_start < regB_end) &&
> -           (regB_start < regA_end) &&
> +           (regB_start <= regA_end) &&
> +           (regA_start <= regB_end) &&
>             (regA_end <= regB_end))
>                 return true;
>
> --
> 2.42.0
>


From inochiama at outlook.com  Tue Nov 14 15:21:57 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Wed, 15 Nov 2023 07:21:57 +0800
Subject: [PATCH v2 0/4] platform: generic: Add Sophgo sg2042 support
In-Reply-To: <CAAhSdy1HRNAE6g1cTi_i3fRf1hJjp4oPfF-KnjPg77qq-yzivA@mail.gmail.com>
References: <CAAhSdy1HRNAE6g1cTi_i3fRf1hJjp4oPfF-KnjPg77qq-yzivA@mail.gmail.com>
Message-ID: <IA1PR20MB4953161DEC2C623255ECD357BBB2A@IA1PR20MB4953.namprd20.prod.outlook.com>

>On Sun, Oct 8, 2023 at 5:46?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> As the dts of sg2042 in the mainline kernel is considered as stable, it is
>> time to port sg2042 soc.
>>
>> This patch depends other two series:
>> 1. platform: generic: add T-HEAD th1520 soc support
>> 2. lib: sbi: Add sub-regions check for sanitizing domain
>>
>> Link: https://lore.kernel.org/linux-riscv/cover.1696433229.git.unicorn_wang at outlook.com/
>>
>> Changed from v1:
>> 1. make number of entries in tlb fifo auto deteted.
>>
>> Inochi Amaoto (4):
>>   lib: utils/timer: mtimer: add separate T-Head C9xx CLINT mtimer
>>     compatible
>>   lib: utils/ipi: mswi: add separate T-Head C9xx CLINT mswi compatible
>>   platform: Allow platform uses hart count as the size of tlb info
>>   platform: generic: Add Sophgo sg2042 platform support
>
>There are unaddressed comments in this series. I assume there will
>be a v3.
>

There is already a v3 version. but the patch is splitted since v3, as the
aclint driver may evolve a lot.

the latest version of each patch is:
1. the patch 1-2
https://lists.infradead.org/pipermail/opensbi/2023-October/005736.html
2. the patch 3
https://lists.infradead.org/pipermail/opensbi/2023-October/005719.html
3. the patch 4
https://lists.infradead.org/pipermail/opensbi/2023-October/005760.html

>Regards,
>Anup
>
>>
>>  lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
>>  lib/utils/timer/fdt_timer_mtimer.c |  2 +
>>  platform/generic/Kconfig           |  6 +++
>>  platform/generic/configs/defconfig |  1 +
>>  platform/generic/platform.c        |  7 ++++
>>  platform/generic/sophgo/objects.mk |  9 +++++
>>  platform/generic/sophgo/sg2042.c   | 64 ++++++++++++++++++++++++++++++
>>  7 files changed, 90 insertions(+)
>>  create mode 100644 platform/generic/sophgo/objects.mk
>>  create mode 100644 platform/generic/sophgo/sg2042.c
>>
>> --
>> 2.42.0
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi
>


From inochiama at outlook.com  Tue Nov 14 15:30:17 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Wed, 15 Nov 2023 07:30:17 +0800
Subject: [PATCH v2] lib: utils/fdt: Allow zero length mtime when parsing ACLINT
In-Reply-To: <IA1PR20MB49534DA3EEF05A947D5EF6EABBD3A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49534DA3EEF05A947D5EF6EABBD3A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB4953C2F395E322FEDA0BAF34BBB2A@IA1PR20MB4953.namprd20.prod.outlook.com>

>Currently, the fdt_parse_aclint_node() does not allow the first reg to
>be zero length. This may cause the parser to fail when the ACLINT does
>not support mtime.
>
>To be robust, delay the check of first reg in fdt_parse_aclint_node()
>so the parser can handle ACLINT without mtime support.
>
>Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
>---
> lib/utils/fdt/fdt_helper.c | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
>diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
>index 9ae7f09..972e529 100644
>--- a/lib/utils/fdt/fdt_helper.c
>+++ b/lib/utils/fdt/fdt_helper.c
>@@ -891,8 +891,10 @@ int fdt_parse_aclint_node(void *fdt, int nodeoffset, bool for_timer,
>
> 	rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
> 				    &reg_addr, &reg_size);
>-	if (rc < 0 || !reg_size)
>+	if (rc < 0)
> 		return SBI_ENODEV;
>+	if (!reg_size)
>+		reg_addr = reg_size = 0;
> 	*out_addr1 = reg_addr;
> 	*out_size1 = reg_size;
>
>@@ -900,6 +902,8 @@ int fdt_parse_aclint_node(void *fdt, int nodeoffset, bool for_timer,
> 				    &reg_addr, &reg_size);
> 	if (rc < 0 || !reg_size)
> 		reg_addr = reg_size = 0;
>+	if (!reg_size && *out_size1 == 0)
>+		return SBI_ENODEV;
> 	if (out_addr2)
> 		*out_addr2 = reg_addr;
> 	if (out_size2)
>--
>2.42.0
>

As Conor point out, Using "reg-name" to identify the reg of ACLINT will be
a better idea and zero-length reg is unacceptably. This patch should
be abandoned and favor the new way.

[1] https://lore.kernel.org/all/20231114-skedaddle-precinct-66c8897227bb at squawk/


From inochiama at outlook.com  Tue Nov 14 18:24:00 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Wed, 15 Nov 2023 10:24:00 +0800
Subject: [PATCH v2 2/3] lib: sbi: add more restriction to domain region subset check
In-Reply-To: <CAAhSdy0fDAUA3yM2faj=0ObfKz-Nx7ioi=cySvNKjD1aRqpnQg@mail.gmail.com>
References: <CAAhSdy0fDAUA3yM2faj=0ObfKz-Nx7ioi=cySvNKjD1aRqpnQg@mail.gmail.com>
Message-ID: <IA1PR20MB49531CD6FEBC229D79204A7ABBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>

>On Tue, Oct 10, 2023 at 6:30?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> make is_region_subset() check the bound of the domain region, so it can
>> handle the domain region with the same base and length.
>>
>> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
>
>Important fact here is that regB_start < regB_end and
>regA_start < regA_end because order == 0 region is of
>size 1 byte.
>
>When regB_start == regA_start and regB_end == regA_end,
>the following is the outcome of the expressions in the existing
>"if ()" check.
>
>(regB_start <= regA_start) == true
>(regA_start < regB_end) == true
>(regB_start < regA_end) == true
>(regA_end <= regB_end) == true
>
>Based on the above, I don't see how this patch helps.
>

Yes, this is my mistake, drop this patch is just OK.

>Regards,
>Anup
>
>
>
>> ---
>>  lib/sbi/sbi_domain.c | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
>> index 283b598..2b9d16f 100644
>> --- a/lib/sbi/sbi_domain.c
>> +++ b/lib/sbi/sbi_domain.c
>> @@ -185,8 +185,8 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
>>         ulong regB_end = regB->base + (BIT(regB->order) - 1);
>>
>>         if ((regB_start <= regA_start) &&
>> -           (regA_start < regB_end) &&
>> -           (regB_start < regA_end) &&
>> +           (regB_start <= regA_end) &&
>> +           (regA_start <= regB_end) &&
>>             (regA_end <= regB_end))
>>                 return true;
>>
>> --
>> 2.42.0
>>
>


From anup at brainfault.org  Tue Nov 14 19:33:44 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 15 Nov 2023 09:03:44 +0530
Subject: [PATCH v2 0/4] platform: generic: Add Sophgo sg2042 support
In-Reply-To: <IA1PR20MB4953161DEC2C623255ECD357BBB2A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <CAAhSdy1HRNAE6g1cTi_i3fRf1hJjp4oPfF-KnjPg77qq-yzivA@mail.gmail.com>
 <IA1PR20MB4953161DEC2C623255ECD357BBB2A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy31_S5NHTAWQTWPjxfzWZGZHQsBgOLHqGdYHQ_TUB35tQ@mail.gmail.com>

On Wed, Nov 15, 2023 at 4:51?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> >On Sun, Oct 8, 2023 at 5:46?PM Inochi Amaoto <inochiama at outlook.com> wrote:
> >>
> >> As the dts of sg2042 in the mainline kernel is considered as stable, it is
> >> time to port sg2042 soc.
> >>
> >> This patch depends other two series:
> >> 1. platform: generic: add T-HEAD th1520 soc support
> >> 2. lib: sbi: Add sub-regions check for sanitizing domain
> >>
> >> Link: https://lore.kernel.org/linux-riscv/cover.1696433229.git.unicorn_wang at outlook.com/
> >>
> >> Changed from v1:
> >> 1. make number of entries in tlb fifo auto deteted.
> >>
> >> Inochi Amaoto (4):
> >>   lib: utils/timer: mtimer: add separate T-Head C9xx CLINT mtimer
> >>     compatible
> >>   lib: utils/ipi: mswi: add separate T-Head C9xx CLINT mswi compatible
> >>   platform: Allow platform uses hart count as the size of tlb info
> >>   platform: generic: Add Sophgo sg2042 platform support
> >
> >There are unaddressed comments in this series. I assume there will
> >be a v3.
> >
>
> There is already a v3 version. but the patch is splitted since v3, as the
> aclint driver may evolve a lot.
>
> the latest version of each patch is:
> 1. the patch 1-2
> https://lists.infradead.org/pipermail/opensbi/2023-October/005736.html
> 2. the patch 3
> https://lists.infradead.org/pipermail/opensbi/2023-October/005719.html
> 3. the patch 4
> https://lists.infradead.org/pipermail/opensbi/2023-October/005760.html

I had applied v3 patches but accidentally replied to v2 series. Sorry
for the noise.

Regards,
Anup

>
> >Regards,
> >Anup
> >
> >>
> >>  lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
> >>  lib/utils/timer/fdt_timer_mtimer.c |  2 +
> >>  platform/generic/Kconfig           |  6 +++
> >>  platform/generic/configs/defconfig |  1 +
> >>  platform/generic/platform.c        |  7 ++++
> >>  platform/generic/sophgo/objects.mk |  9 +++++
> >>  platform/generic/sophgo/sg2042.c   | 64 ++++++++++++++++++++++++++++++
> >>  7 files changed, 90 insertions(+)
> >>  create mode 100644 platform/generic/sophgo/objects.mk
> >>  create mode 100644 platform/generic/sophgo/sg2042.c
> >>
> >> --
> >> 2.42.0
> >>
> >>
> >> --
> >> opensbi mailing list
> >> opensbi at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/opensbi
> >


From anup at brainfault.org  Tue Nov 14 22:51:31 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 15 Nov 2023 12:21:31 +0530
Subject: [PATCH v2 3/3] lib: sbi: Add regions merging when sanitizing
 domain region
In-Reply-To: <IA1PR20MB4953272F8B5C9000CDA69F02BBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49533A7AB5888E0117F7799BBBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB4953272F8B5C9000CDA69F02BBCDA@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy0xVowOfN3xeNMB-XT+kcjnKpitTUckvMeNSwDCNb4-BQ@mail.gmail.com>

On Tue, Oct 10, 2023 at 6:30?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> As the domain will reject a new memory region which has a sub-regions
> already in the domain, even the new region is bigger and has the same
> flags. This problem can be solved by relaxing region restriction and
> rechecking when adding and sanitizing domains.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> ---
>  lib/sbi/sbi_domain.c | 66 ++++++++++++++++++++++++++++++--------------
>  1 file changed, 45 insertions(+), 21 deletions(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index 2b9d16f..9e9cdad 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
>         return false;
>  }
>
> -/** Check if regionA conflicts regionB */
> -static bool is_region_conflict(const struct sbi_domain_memregion *regA,
> -                               const struct sbi_domain_memregion *regB)
> +/** Check if regionA can be replaced by regionB */
> +static bool is_region_compatible(const struct sbi_domain_memregion *regA,
> +                                const struct sbi_domain_memregion *regB)
>  {
> -       if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
> -           regA->flags == regB->flags)
> +       if (is_region_subset(regA, regB) && regA->flags == regB->flags)
>                 return true;
>
>         return false;
> @@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
>         sbi_memcpy(reg2, &treg, sizeof(treg));
>  }
>
> +static void clear_region(struct sbi_domain_memregion* reg)
> +{
> +       sbi_memset(reg, 0x0, sizeof(*reg));
> +}
> +
>  static int sanitize_domain(const struct sbi_platform *plat,
>                            struct sbi_domain *dom)
>  {
> @@ -315,22 +319,46 @@ static int sanitize_domain(const struct sbi_platform *plat,
>                 return SBI_EINVAL;
>         }
>
> +       /* Remove covered regions */
> +       while(i < (count - 1)) {
> +               reg = &dom->regions[i];
> +               j = i + 1;
> +               while (j < count) {
> +                       reg1 = &dom->regions[j];
> +
> +                       /* reg1 is sub-region of reg, remove reg1 */
> +                       if (is_region_compatible(reg1, reg)) {
> +                               swap_region(reg1, &dom->regions[count - 1]);
> +                               clear_region(&dom->regions[count - 1]);
> +                               count--;
> +                               continue;
> +                       }
> +
> +                       /*
> +                        * reg is sub-region of reg1, replace reg with reg1
> +                        * and recheck all other regions.
> +                        */
> +                       if (is_region_compatible(reg, reg1)) {
> +                               swap_region(reg, reg1);
> +                               swap_region(reg1, &dom->regions[count - 1]);
> +                               clear_region(&dom->regions[count - 1]);
> +                               count--;
> +                               goto regions_recheck;
> +                       }
> +
> +                       j++;
> +               }
> +
> +               i++;
> +regions_recheck:
> +       }
> +
>         /* Sort the memory regions */
>         for (i = 0; i < (count - 1); i++) {
>                 reg = &dom->regions[i];
>                 for (j = i + 1; j < count; j++) {
>                         reg1 = &dom->regions[j];
>
> -                       if (is_region_conflict(reg1, reg)) {
> -                               sbi_printf("%s: %s conflict between regions "
> -                                       "(base=0x%lx order=%lu flags=0x%lx) and "
> -                                       "(base=0x%lx order=%lu flags=0x%lx)\n",
> -                                       __func__, dom->name,
> -                                       reg->base, reg->order, reg->flags,
> -                                       reg1->base, reg1->order, reg1->flags);
> -                               return SBI_EINVAL;
> -                       }
> -
>                         if (!is_region_before(reg1, reg))
>                                 continue;
>
> @@ -581,12 +609,8 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
>
>         /* Check for conflicts */

This comment needs to change.

>         sbi_domain_for_each_memregion(&root, nreg) {
> -               if (is_region_conflict(reg, nreg)) {
> -                       sbi_printf("%s: is_region_conflict check failed"
> -                       " 0x%lx conflicts existing 0x%lx\n", __func__,
> -                                  reg->base, nreg->base);
> -                       return SBI_EALREADY;
> -               }
> +               if (is_region_compatible(reg, nreg))
> +                       return 0;
>         }
>
>         /* Append the memregion to root memregions */
> --
> 2.42.0
>

Otherwise, looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Thanks,
Anup


From wxjstz at 126.com  Tue Nov 14 23:15:25 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 15 Nov 2023 15:15:25 +0800
Subject: [PATCH] lib: sbi: Improved atomic bit operations
Message-ID: <20231115071622.465416-1-wxjstz@126.com>

The previous atomic operation returns the new value. After setting and
clearing, the new value is determined. The value before modification
should be returned. The previous code actually returns the word before
modification, not the bit value. This patch improves these issues.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/riscv_atomic.h |  8 ++++----
 lib/sbi/riscv_atomic.c     | 36 ++++++++----------------------------
 2 files changed, 12 insertions(+), 32 deletions(-)

diff --git a/include/sbi/riscv_atomic.h b/include/sbi/riscv_atomic.h
index 3972e0b..31dc673 100644
--- a/include/sbi/riscv_atomic.h
+++ b/include/sbi/riscv_atomic.h
@@ -39,14 +39,14 @@ unsigned int atomic_raw_xchg_uint(volatile unsigned int *ptr,
 unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
 				    unsigned long newval);
 /**
- * Set a bit in an atomic variable and return the new value.
+ * Set a bit in an atomic variable and return the value of bit before modify.
  * @nr : Bit to set.
  * @atom: atomic variable to modify
  */
 int atomic_set_bit(int nr, atomic_t *atom);
 
 /**
- * Clear a bit in an atomic variable and return the new value.
+ * Clear a bit in an atomic variable and return the value of bit before modify.
  * @nr : Bit to set.
  * @atom: atomic variable to modify
  */
@@ -54,14 +54,14 @@ int atomic_set_bit(int nr, atomic_t *atom);
 int atomic_clear_bit(int nr, atomic_t *atom);
 
 /**
- * Set a bit in any address and return the new value .
+ * Set a bit in any address and return the the value of bit before modify.
  * @nr : Bit to set.
  * @addr: Address to modify
  */
 int atomic_raw_set_bit(int nr, volatile unsigned long *addr);
 
 /**
- * Clear a bit in any address and return the new value .
+ * Clear a bit in any address and return the the value of bit before modify.
  * @nr : Bit to set.
  * @addr: Address to modify
  */
diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
index 528686f..95c6ff7 100644
--- a/lib/sbi/riscv_atomic.c
+++ b/lib/sbi/riscv_atomic.c
@@ -206,40 +206,20 @@ unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
 #endif
 }
 
-#if (__SIZEOF_POINTER__ == 8)
-#define __AMO(op) "amo" #op ".d"
-#elif (__SIZEOF_POINTER__ == 4)
-#define __AMO(op) "amo" #op ".w"
-#else
-#error "Unexpected __SIZEOF_POINTER__"
-#endif
-
-#define __atomic_op_bit_ord(op, mod, nr, addr, ord)                          \
-	({                                                                   \
-		unsigned long __res, __mask;                                 \
-		__mask = BIT_MASK(nr);                                       \
-		__asm__ __volatile__(__AMO(op) #ord " %0, %2, %1"            \
-				     : "=r"(__res), "+A"(addr[BIT_WORD(nr)]) \
-				     : "r"(mod(__mask))                      \
-				     : "memory");                            \
-		__res;                                                       \
-	})
-
-#define __atomic_op_bit(op, mod, nr, addr) \
-	__atomic_op_bit_ord(op, mod, nr, addr, .aqrl)
-
-/* Bitmask modifiers */
-#define __NOP(x) (x)
-#define __NOT(x) (~(x))
-
 inline int atomic_raw_set_bit(int nr, volatile unsigned long *addr)
 {
-	return __atomic_op_bit(or, __NOP, nr, addr);
+	unsigned long res, mask = BIT_MASK(nr);
+	volatile unsigned long *ptr = &addr[BIT_WORD(nr)];
+	res = __atomic_fetch_and(ptr, ~mask, __ATOMIC_ACQ_REL);
+	return res & mask ? 1 : 0;
 }
 
 inline int atomic_raw_clear_bit(int nr, volatile unsigned long *addr)
 {
-	return __atomic_op_bit(and, __NOT, nr, addr);
+	unsigned long res, mask = BIT_MASK(nr);
+	volatile unsigned long *ptr = &addr[BIT_WORD(nr)];
+	res = __atomic_fetch_or(ptr, mask, __ATOMIC_ACQ_REL);
+	return res & mask ? 1 : 0;
 }
 
 inline int atomic_set_bit(int nr, atomic_t *atom)
-- 
2.42.0



From inochiama at outlook.com  Tue Nov 14 23:28:54 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Wed, 15 Nov 2023 15:28:54 +0800
Subject: [PATCH v2 3/3] lib: sbi: Add regions merging when sanitizing domain region
In-Reply-To: <CAAhSdy0xVowOfN3xeNMB-XT+kcjnKpitTUckvMeNSwDCNb4-BQ@mail.gmail.com>
References: <CAAhSdy0xVowOfN3xeNMB-XT+kcjnKpitTUckvMeNSwDCNb4-BQ@mail.gmail.com>
Message-ID: <IA1PR20MB4953DB28D9CA7F8C074DC63FBBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>

>On Tue, Oct 10, 2023 at 6:30?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> As the domain will reject a new memory region which has a sub-regions
>> already in the domain, even the new region is bigger and has the same
>> flags. This problem can be solved by relaxing region restriction and
>> rechecking when adding and sanitizing domains.
>>
>> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
>> ---
>>  lib/sbi/sbi_domain.c | 66 ++++++++++++++++++++++++++++++--------------
>>  1 file changed, 45 insertions(+), 21 deletions(-)
>>
>> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
>> index 2b9d16f..9e9cdad 100644
>> --- a/lib/sbi/sbi_domain.c
>> +++ b/lib/sbi/sbi_domain.c
>> @@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
>>         return false;
>>  }
>>
>> -/** Check if regionA conflicts regionB */
>> -static bool is_region_conflict(const struct sbi_domain_memregion *regA,
>> -                               const struct sbi_domain_memregion *regB)
>> +/** Check if regionA can be replaced by regionB */
>> +static bool is_region_compatible(const struct sbi_domain_memregion *regA,
>> +                                const struct sbi_domain_memregion *regB)
>>  {
>> -       if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
>> -           regA->flags == regB->flags)
>> +       if (is_region_subset(regA, regB) && regA->flags == regB->flags)
>>                 return true;
>>
>>         return false;
>> @@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
>>         sbi_memcpy(reg2, &treg, sizeof(treg));
>>  }
>>
>> +static void clear_region(struct sbi_domain_memregion* reg)
>> +{
>> +       sbi_memset(reg, 0x0, sizeof(*reg));
>> +}
>> +
>>  static int sanitize_domain(const struct sbi_platform *plat,
>>                            struct sbi_domain *dom)
>>  {
>> @@ -315,22 +319,46 @@ static int sanitize_domain(const struct sbi_platform *plat,
>>                 return SBI_EINVAL;
>>         }
>>
>> +       /* Remove covered regions */
>> +       while(i < (count - 1)) {
>> +               reg = &dom->regions[i];
>> +               j = i + 1;
>> +               while (j < count) {
>> +                       reg1 = &dom->regions[j];
>> +
>> +                       /* reg1 is sub-region of reg, remove reg1 */
>> +                       if (is_region_compatible(reg1, reg)) {
>> +                               swap_region(reg1, &dom->regions[count - 1]);
>> +                               clear_region(&dom->regions[count - 1]);
>> +                               count--;
>> +                               continue;
>> +                       }
>> +
>> +                       /*
>> +                        * reg is sub-region of reg1, replace reg with reg1
>> +                        * and recheck all other regions.
>> +                        */
>> +                       if (is_region_compatible(reg, reg1)) {
>> +                               swap_region(reg, reg1);
>> +                               swap_region(reg1, &dom->regions[count - 1]);
>> +                               clear_region(&dom->regions[count - 1]);
>> +                               count--;
>> +                               goto regions_recheck;
>> +                       }
>> +
>> +                       j++;
>> +               }
>> +
>> +               i++;
>> +regions_recheck:
>> +       }
>> +
>>         /* Sort the memory regions */
>>         for (i = 0; i < (count - 1); i++) {
>>                 reg = &dom->regions[i];
>>                 for (j = i + 1; j < count; j++) {
>>                         reg1 = &dom->regions[j];
>>
>> -                       if (is_region_conflict(reg1, reg)) {
>> -                               sbi_printf("%s: %s conflict between regions "
>> -                                       "(base=0x%lx order=%lu flags=0x%lx) and "
>> -                                       "(base=0x%lx order=%lu flags=0x%lx)\n",
>> -                                       __func__, dom->name,
>> -                                       reg->base, reg->order, reg->flags,
>> -                                       reg1->base, reg1->order, reg1->flags);
>> -                               return SBI_EINVAL;
>> -                       }
>> -
>>                         if (!is_region_before(reg1, reg))
>>                                 continue;
>>
>> @@ -581,12 +609,8 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
>>
>>         /* Check for conflicts */
>
>This comment needs to change.

OK, Thanks

>
>>         sbi_domain_for_each_memregion(&root, nreg) {
>> -               if (is_region_conflict(reg, nreg)) {
>> -                       sbi_printf("%s: is_region_conflict check failed"
>> -                       " 0x%lx conflicts existing 0x%lx\n", __func__,
>> -                                  reg->base, nreg->base);
>> -                       return SBI_EALREADY;
>> -               }
>> +               if (is_region_compatible(reg, nreg))
>> +                       return 0;
>>         }
>>
>>         /* Append the memregion to root memregions */
>> --
>> 2.42.0
>>
>
>Otherwise, looks good to me.
>
>Reviewed-by: Anup Patel <anup at brainfault.org>
>
>Thanks,
>Anup
>
>


From inochiama at outlook.com  Wed Nov 15 00:04:06 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Wed, 15 Nov 2023 16:04:06 +0800
Subject: [PATCH v3 0/2] lib: sbi: Add sub-regions check for sanitizing domain
Message-ID: <IA1PR20MB4953AF8946C1164D11CA494ABBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>

As the domain will reject a new memory region which has a sub-regions
already in the domain, even the new region is bigger and has the same
flags.

To resolve this problem, just relax restriction of memory region when
adding it, and merge the region into the existed domain regions when
sorting regions.

Changed from v2:
1. rebase to master
2. removed the useless changed for conflict check.
3. improve the commit for the patch 3.

Changed from v1:
1. separate the patch into three parts.

Inochi Amaoto (2):
  lib: sbi: separate the swap operation of domain region
  lib: sbi: Add regions merging when sanitizing domain region

 lib/sbi/sbi_domain.c | 84 ++++++++++++++++++++++++++++++--------------
 1 file changed, 58 insertions(+), 26 deletions(-)

--
2.42.1



From inochiama at outlook.com  Wed Nov 15 00:05:25 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Wed, 15 Nov 2023 16:05:25 +0800
Subject: [PATCH v3 1/2] lib: sbi: separate the swap operation of domain region
In-Reply-To: <IA1PR20MB4953AF8946C1164D11CA494ABBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953AF8946C1164D11CA494ABBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB495376ECA68E0D085198806EBBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>

Swapping domain region is a common operation when sorting domain region,
so separate it as a function to make code clean.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_domain.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
index 4d248d0..71cb381 100644
--- a/lib/sbi/sbi_domain.c
+++ b/lib/sbi/sbi_domain.c
@@ -256,10 +256,20 @@ static const struct sbi_domain_memregion *find_next_subset_region(
 	return ret;
 }
 
+static void swap_region(struct sbi_domain_memregion* reg1,
+			struct sbi_domain_memregion* reg2)
+{
+	struct sbi_domain_memregion treg;
+
+	sbi_memcpy(&treg, reg1, sizeof(treg));
+	sbi_memcpy(reg1, reg2, sizeof(treg));
+	sbi_memcpy(reg2, &treg, sizeof(treg));
+}
+
 static int sanitize_domain(struct sbi_domain *dom)
 {
 	u32 i, j, count;
-	struct sbi_domain_memregion treg, *reg, *reg1;
+	struct sbi_domain_memregion *reg, *reg1;
 
 	/* Check possible HARTs */
 	if (!dom->possible_harts) {
@@ -323,9 +333,7 @@ static int sanitize_domain(struct sbi_domain *dom)
 			if (!is_region_before(reg1, reg))
 				continue;
 
-			sbi_memcpy(&treg, reg1, sizeof(treg));
-			sbi_memcpy(reg1, reg, sizeof(treg));
-			sbi_memcpy(reg, &treg, sizeof(treg));
+			swap_region(reg, reg1);
 		}
 	}
 
-- 
2.42.1



From inochiama at outlook.com  Wed Nov 15 00:05:26 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Wed, 15 Nov 2023 16:05:26 +0800
Subject: [PATCH v3 2/2] lib: sbi: Add regions merging when sanitizing domain region
In-Reply-To: <IA1PR20MB4953AF8946C1164D11CA494ABBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953AF8946C1164D11CA494ABBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB4953E3C9BCE80A02F5ADD010BBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>

As the domain will reject a new memory region which has a sub-regions
already in the domain, even the new region is bigger and has the same
flags. This problem can be solved by relaxing region restriction and
rechecking when adding and sanitizing domains.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_domain.c | 68 ++++++++++++++++++++++++++++++--------------
 1 file changed, 46 insertions(+), 22 deletions(-)

diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
index 71cb381..ba8ce44 100644
--- a/lib/sbi/sbi_domain.c
+++ b/lib/sbi/sbi_domain.c
@@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
 	return false;
 }
 
-/** Check if regionA conflicts regionB */
-static bool is_region_conflict(const struct sbi_domain_memregion *regA,
-				const struct sbi_domain_memregion *regB)
+/** Check if regionA can be replaced by regionB */
+static bool is_region_compatible(const struct sbi_domain_memregion *regA,
+				 const struct sbi_domain_memregion *regB)
 {
-	if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
-	    regA->flags == regB->flags)
+	if (is_region_subset(regA, regB) && regA->flags == regB->flags)
 		return true;
 
 	return false;
@@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
 	sbi_memcpy(reg2, &treg, sizeof(treg));
 }
 
+static void clear_region(struct sbi_domain_memregion* reg)
+{
+	sbi_memset(reg, 0x0, sizeof(*reg));
+}
+
 static int sanitize_domain(struct sbi_domain *dom)
 {
 	u32 i, j, count;
@@ -314,22 +318,46 @@ static int sanitize_domain(struct sbi_domain *dom)
 		return SBI_EINVAL;
 	}
 
+	/* Remove covered regions */
+	while(i < (count - 1)) {
+		reg = &dom->regions[i];
+		j = i + 1;
+		while (j < count) {
+			reg1 = &dom->regions[j];
+
+			/* reg1 is sub-region of reg, remove reg1 */
+			if (is_region_compatible(reg1, reg)) {
+				swap_region(reg1, &dom->regions[count - 1]);
+				clear_region(&dom->regions[count - 1]);
+				count--;
+				continue;
+			}
+
+			/*
+			 * reg is sub-region of reg1, replace reg with reg1
+			 * and recheck all other regions.
+			 */
+			if (is_region_compatible(reg, reg1)) {
+				swap_region(reg, reg1);
+				swap_region(reg1, &dom->regions[count - 1]);
+				clear_region(&dom->regions[count - 1]);
+				count--;
+				goto regions_recheck;
+			}
+
+			j++;
+		}
+
+		i++;
+regions_recheck:
+	}
+
 	/* Sort the memory regions */
 	for (i = 0; i < (count - 1); i++) {
 		reg = &dom->regions[i];
 		for (j = i + 1; j < count; j++) {
 			reg1 = &dom->regions[j];
 
-			if (is_region_conflict(reg1, reg)) {
-				sbi_printf("%s: %s conflict between regions "
-					"(base=0x%lx order=%lu flags=0x%lx) and "
-					"(base=0x%lx order=%lu flags=0x%lx)\n",
-					__func__, dom->name,
-					reg->base, reg->order, reg->flags,
-					reg1->base, reg1->order, reg1->flags);
-				return SBI_EINVAL;
-			}
-
 			if (!is_region_before(reg1, reg))
 				continue;
 
@@ -576,14 +604,10 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
 	    (ROOT_REGION_MAX <= root_memregs_count))
 		return SBI_EINVAL;
 
-	/* Check for conflicts */
+	/* Check whether compatible region exists for the new one */
 	sbi_domain_for_each_memregion(&root, nreg) {
-		if (is_region_conflict(reg, nreg)) {
-			sbi_printf("%s: is_region_conflict check failed"
-			" 0x%lx conflicts existing 0x%lx\n", __func__,
-				   reg->base, nreg->base);
-			return SBI_EALREADY;
-		}
+		if (is_region_compatible(reg, nreg))
+			return 0;
 	}
 
 	/* Append the memregion to root memregions */
-- 
2.42.1



From jrtc27 at jrtc27.com  Wed Nov 15 00:05:17 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Wed, 15 Nov 2023 08:05:17 +0000
Subject: [PATCH] lib: sbi: Improved atomic bit operations
In-Reply-To: <20231115071622.465416-1-wxjstz@126.com>
References: <20231115071622.465416-1-wxjstz@126.com>
Message-ID: <E77C3F09-3F8F-4196-AAC3-97ECE3B25124@jrtc27.com>

On 15 Nov 2023, at 07:15, Xiang W <wxjstz at 126.com> wrote:
> 
> The previous atomic operation returns the new value.

Looks to me like they returned the old value? That is, this patch:

(a) Fixes the comments for the functions to document the actual
semantics
(b) Replaces the implementations with __atomic intrinsics to avoid the
need for inline assembly

These should be separate commits, and the commit message should be
correct. Unless I?m misunderstanding. Otherwise it sounds like you?re
Changing the behaviour of existing functions...

Jess

> After setting and
> clearing, the new value is determined. The value before modification
> should be returned. The previous code actually returns the word before
> modification, not the bit value. This patch improves these issues.
> 
> Signed-off-by: Xiang W <wxjstz at 126.com>
> ---
> include/sbi/riscv_atomic.h |  8 ++++----
> lib/sbi/riscv_atomic.c     | 36 ++++++++----------------------------
> 2 files changed, 12 insertions(+), 32 deletions(-)
> 
> diff --git a/include/sbi/riscv_atomic.h b/include/sbi/riscv_atomic.h
> index 3972e0b..31dc673 100644
> --- a/include/sbi/riscv_atomic.h
> +++ b/include/sbi/riscv_atomic.h
> @@ -39,14 +39,14 @@ unsigned int atomic_raw_xchg_uint(volatile unsigned int *ptr,
> unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
>    unsigned long newval);
> /**
> - * Set a bit in an atomic variable and return the new value.
> + * Set a bit in an atomic variable and return the value of bit before modify.
>  * @nr : Bit to set.
>  * @atom: atomic variable to modify
>  */
> int atomic_set_bit(int nr, atomic_t *atom);
> 
> /**
> - * Clear a bit in an atomic variable and return the new value.
> + * Clear a bit in an atomic variable and return the value of bit before modify.
>  * @nr : Bit to set.
>  * @atom: atomic variable to modify
>  */
> @@ -54,14 +54,14 @@ int atomic_set_bit(int nr, atomic_t *atom);
> int atomic_clear_bit(int nr, atomic_t *atom);
> 
> /**
> - * Set a bit in any address and return the new value .
> + * Set a bit in any address and return the the value of bit before modify.
>  * @nr : Bit to set.
>  * @addr: Address to modify
>  */
> int atomic_raw_set_bit(int nr, volatile unsigned long *addr);
> 
> /**
> - * Clear a bit in any address and return the new value .
> + * Clear a bit in any address and return the the value of bit before modify.
>  * @nr : Bit to set.
>  * @addr: Address to modify
>  */
> diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
> index 528686f..95c6ff7 100644
> --- a/lib/sbi/riscv_atomic.c
> +++ b/lib/sbi/riscv_atomic.c
> @@ -206,40 +206,20 @@ unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
> #endif
> }
> 
> -#if (__SIZEOF_POINTER__ == 8)
> -#define __AMO(op) "amo" #op ".d"
> -#elif (__SIZEOF_POINTER__ == 4)
> -#define __AMO(op) "amo" #op ".w"
> -#else
> -#error "Unexpected __SIZEOF_POINTER__"
> -#endif
> -
> -#define __atomic_op_bit_ord(op, mod, nr, addr, ord)                          \
> - ({                                                                   \
> - unsigned long __res, __mask;                                 \
> - __mask = BIT_MASK(nr);                                       \
> - __asm__ __volatile__(__AMO(op) #ord " %0, %2, %1"            \
> -     : "=r"(__res), "+A"(addr[BIT_WORD(nr)]) \
> -     : "r"(mod(__mask))                      \
> -     : "memory");                            \
> - __res;                                                       \
> - })
> -
> -#define __atomic_op_bit(op, mod, nr, addr) \
> - __atomic_op_bit_ord(op, mod, nr, addr, .aqrl)
> -
> -/* Bitmask modifiers */
> -#define __NOP(x) (x)
> -#define __NOT(x) (~(x))
> -
> inline int atomic_raw_set_bit(int nr, volatile unsigned long *addr)
> {
> - return __atomic_op_bit(or, __NOP, nr, addr);
> + unsigned long res, mask = BIT_MASK(nr);
> + volatile unsigned long *ptr = &addr[BIT_WORD(nr)];
> + res = __atomic_fetch_and(ptr, ~mask, __ATOMIC_ACQ_REL);
> + return res & mask ? 1 : 0;
> }
> 
> inline int atomic_raw_clear_bit(int nr, volatile unsigned long *addr)
> {
> - return __atomic_op_bit(and, __NOT, nr, addr);
> + unsigned long res, mask = BIT_MASK(nr);
> + volatile unsigned long *ptr = &addr[BIT_WORD(nr)];
> + res = __atomic_fetch_or(ptr, mask, __ATOMIC_ACQ_REL);
> + return res & mask ? 1 : 0;
> }
> 
> inline int atomic_set_bit(int nr, atomic_t *atom)
> -- 
> 2.42.0
> 
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



From anup at brainfault.org  Wed Nov 15 05:24:25 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 15 Nov 2023 18:54:25 +0530
Subject: [PATCH v3 2/2] lib: sbi: Add regions merging when sanitizing
 domain region
In-Reply-To: <IA1PR20MB4953E3C9BCE80A02F5ADD010BBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953AF8946C1164D11CA494ABBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB4953E3C9BCE80A02F5ADD010BBB1A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy3dcSXBcC2BXk+vBhyHMqos0Uf8C-n26RPXP3aTsY+yzg@mail.gmail.com>

On Wed, Nov 15, 2023 at 1:35?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> As the domain will reject a new memory region which has a sub-regions
> already in the domain, even the new region is bigger and has the same
> flags. This problem can be solved by relaxing region restriction and
> rechecking when adding and sanitizing domains.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
>  lib/sbi/sbi_domain.c | 68 ++++++++++++++++++++++++++++++--------------
>  1 file changed, 46 insertions(+), 22 deletions(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index 71cb381..ba8ce44 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
>         return false;
>  }
>
> -/** Check if regionA conflicts regionB */
> -static bool is_region_conflict(const struct sbi_domain_memregion *regA,
> -                               const struct sbi_domain_memregion *regB)
> +/** Check if regionA can be replaced by regionB */
> +static bool is_region_compatible(const struct sbi_domain_memregion *regA,
> +                                const struct sbi_domain_memregion *regB)
>  {
> -       if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
> -           regA->flags == regB->flags)
> +       if (is_region_subset(regA, regB) && regA->flags == regB->flags)
>                 return true;
>
>         return false;
> @@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
>         sbi_memcpy(reg2, &treg, sizeof(treg));
>  }
>
> +static void clear_region(struct sbi_domain_memregion* reg)
> +{
> +       sbi_memset(reg, 0x0, sizeof(*reg));
> +}
> +
>  static int sanitize_domain(struct sbi_domain *dom)
>  {
>         u32 i, j, count;
> @@ -314,22 +318,46 @@ static int sanitize_domain(struct sbi_domain *dom)
>                 return SBI_EINVAL;
>         }
>
> +       /* Remove covered regions */
> +       while(i < (count - 1)) {
> +               reg = &dom->regions[i];
> +               j = i + 1;
> +               while (j < count) {
> +                       reg1 = &dom->regions[j];
> +
> +                       /* reg1 is sub-region of reg, remove reg1 */
> +                       if (is_region_compatible(reg1, reg)) {
> +                               swap_region(reg1, &dom->regions[count - 1]);
> +                               clear_region(&dom->regions[count - 1]);
> +                               count--;
> +                               continue;
> +                       }
> +
> +                       /*
> +                        * reg is sub-region of reg1, replace reg with reg1
> +                        * and recheck all other regions.
> +                        */
> +                       if (is_region_compatible(reg, reg1)) {
> +                               swap_region(reg, reg1);
> +                               swap_region(reg1, &dom->regions[count - 1]);
> +                               clear_region(&dom->regions[count - 1]);

This does not seem right.

We should disturb the sorted order of regions so all regions
from i + 1 to count - 1 should be shifted up.

> +                               count--;
> +                               goto regions_recheck;
> +                       }
> +
> +                       j++;
> +               }
> +
> +               i++;
> +regions_recheck:
> +       }
> +
>         /* Sort the memory regions */
>         for (i = 0; i < (count - 1); i++) {
>                 reg = &dom->regions[i];
>                 for (j = i + 1; j < count; j++) {
>                         reg1 = &dom->regions[j];
>
> -                       if (is_region_conflict(reg1, reg)) {
> -                               sbi_printf("%s: %s conflict between regions "
> -                                       "(base=0x%lx order=%lu flags=0x%lx) and "
> -                                       "(base=0x%lx order=%lu flags=0x%lx)\n",
> -                                       __func__, dom->name,
> -                                       reg->base, reg->order, reg->flags,
> -                                       reg1->base, reg1->order, reg1->flags);
> -                               return SBI_EINVAL;
> -                       }
> -
>                         if (!is_region_before(reg1, reg))
>                                 continue;
>
> @@ -576,14 +604,10 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
>             (ROOT_REGION_MAX <= root_memregs_count))
>                 return SBI_EINVAL;
>
> -       /* Check for conflicts */
> +       /* Check whether compatible region exists for the new one */
>         sbi_domain_for_each_memregion(&root, nreg) {
> -               if (is_region_conflict(reg, nreg)) {
> -                       sbi_printf("%s: is_region_conflict check failed"
> -                       " 0x%lx conflicts existing 0x%lx\n", __func__,
> -                                  reg->base, nreg->base);
> -                       return SBI_EALREADY;
> -               }
> +               if (is_region_compatible(reg, nreg))
> +                       return 0;
>         }
>
>         /* Append the memregion to root memregions */
> --
> 2.42.1
>

Regards,
Anup


From wxjstz at 126.com  Wed Nov 15 06:23:54 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 15 Nov 2023 22:23:54 +0800
Subject: [PATCH] lib: sbi: Improved atomic bit operations
In-Reply-To: <E77C3F09-3F8F-4196-AAC3-97ECE3B25124@jrtc27.com>
References: <20231115071622.465416-1-wxjstz@126.com>
	 <E77C3F09-3F8F-4196-AAC3-97ECE3B25124@jrtc27.com>
Message-ID: <13f41a66f1e6c220c772298613c65f9633723b8d.camel@126.com>

? 2023-11-15???? 08:05 +0000?Jessica Clarke???
> On 15 Nov 2023, at 07:15, Xiang W <wxjstz at 126.com> wrote:
> > 
> > The previous atomic operation returns the new value.
> 
> Looks to me like they returned the old value? That is, this patch:
The original code return value is old. The function description is
to return a new value. My description here is rather vague.
> 
> (a) Fixes the comments for the functions to document the actual
> semantics
> (b) Replaces the implementations with __atomic intrinsics to avoid the
> need for inline assembly
> 
> These should be separate commits, and the commit message should be
> correct. Unless I?m misunderstanding. Otherwise it sounds like you?re
> Changing the behaviour of existing functions...
Ok!

Regards,
Xiang W
> 
> Jess
> 
> > After setting and
> > clearing, the new value is determined. The value before modification
> > should be returned. The previous code actually returns the word before
> > modification, not the bit value. This patch improves these issues.
> > 
> > Signed-off-by: Xiang W <wxjstz at 126.com>
> > ---
> > include/sbi/riscv_atomic.h |? 8 ++++----
> > lib/sbi/riscv_atomic.c???? | 36 ++++++++----------------------------
> > 2 files changed, 12 insertions(+), 32 deletions(-)
> > 
> > diff --git a/include/sbi/riscv_atomic.h b/include/sbi/riscv_atomic.h
> > index 3972e0b..31dc673 100644
> > --- a/include/sbi/riscv_atomic.h
> > +++ b/include/sbi/riscv_atomic.h
> > @@ -39,14 +39,14 @@ unsigned int atomic_raw_xchg_uint(volatile unsigned int *ptr,
> > unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
> > ?? unsigned long newval);
> > /**
> > - * Set a bit in an atomic variable and return the new value.
> > + * Set a bit in an atomic variable and return the value of bit before modify.
> > ?* @nr : Bit to set.
> > ?* @atom: atomic variable to modify
> > ?*/
> > int atomic_set_bit(int nr, atomic_t *atom);
> > 
> > /**
> > - * Clear a bit in an atomic variable and return the new value.
> > + * Clear a bit in an atomic variable and return the value of bit before modify.
> > ?* @nr : Bit to set.
> > ?* @atom: atomic variable to modify
> > ?*/
> > @@ -54,14 +54,14 @@ int atomic_set_bit(int nr, atomic_t *atom);
> > int atomic_clear_bit(int nr, atomic_t *atom);
> > 
> > /**
> > - * Set a bit in any address and return the new value .
> > + * Set a bit in any address and return the the value of bit before modify.
> > ?* @nr : Bit to set.
> > ?* @addr: Address to modify
> > ?*/
> > int atomic_raw_set_bit(int nr, volatile unsigned long *addr);
> > 
> > /**
> > - * Clear a bit in any address and return the new value .
> > + * Clear a bit in any address and return the the value of bit before modify.
> > ?* @nr : Bit to set.
> > ?* @addr: Address to modify
> > ?*/
> > diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
> > index 528686f..95c6ff7 100644
> > --- a/lib/sbi/riscv_atomic.c
> > +++ b/lib/sbi/riscv_atomic.c
> > @@ -206,40 +206,20 @@ unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
> > #endif
> > }
> > 
> > -#if (__SIZEOF_POINTER__ == 8)
> > -#define __AMO(op) "amo" #op ".d"
> > -#elif (__SIZEOF_POINTER__ == 4)
> > -#define __AMO(op) "amo" #op ".w"
> > -#else
> > -#error "Unexpected __SIZEOF_POINTER__"
> > -#endif
> > -
> > -#define __atomic_op_bit_ord(op, mod, nr, addr, ord)????????????????????????? \
> > - ({?????????????????????????????????????????????????????????????????? \
> > - unsigned long __res, __mask;???????????????????????????????? \
> > - __mask = BIT_MASK(nr);?????????????????????????????????????? \
> > - __asm__ __volatile__(__AMO(op) #ord " %0, %2, %1"??????????? \
> > -???? : "=r"(__res), "+A"(addr[BIT_WORD(nr)]) \
> > -???? : "r"(mod(__mask))????????????????????? \
> > -???? : "memory");??????????????????????????? \
> > - __res;?????????????????????????????????????????????????????? \
> > - })
> > -
> > -#define __atomic_op_bit(op, mod, nr, addr) \
> > - __atomic_op_bit_ord(op, mod, nr, addr, .aqrl)
> > -
> > -/* Bitmask modifiers */
> > -#define __NOP(x) (x)
> > -#define __NOT(x) (~(x))
> > -
> > inline int atomic_raw_set_bit(int nr, volatile unsigned long *addr)
> > {
> > - return __atomic_op_bit(or, __NOP, nr, addr);
> > + unsigned long res, mask = BIT_MASK(nr);
> > + volatile unsigned long *ptr = &addr[BIT_WORD(nr)];
> > + res = __atomic_fetch_and(ptr, ~mask, __ATOMIC_ACQ_REL);
> > + return res & mask ? 1 : 0;
> > }
> > 
> > inline int atomic_raw_clear_bit(int nr, volatile unsigned long *addr)
> > {
> > - return __atomic_op_bit(and, __NOT, nr, addr);
> > + unsigned long res, mask = BIT_MASK(nr);
> > + volatile unsigned long *ptr = &addr[BIT_WORD(nr)];
> > + res = __atomic_fetch_or(ptr, mask, __ATOMIC_ACQ_REL);
> > + return res & mask ? 1 : 0;
> > }
> > 
> > inline int atomic_set_bit(int nr, atomic_t *atom)
> > -- 
> > 2.42.0
> > 
> > 
> > -- 
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi



From wxjstz at 126.com  Wed Nov 15 06:59:20 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 15 Nov 2023 22:59:20 +0800
Subject: [PATCH v2 0/2] lib: sbi: Improved atomic bit operations
Message-ID: <20231115145929.488427-1-wxjstz@126.com>

The original atomic bit operation returns the word before modification,
but under RV64, int is 32 bits and cannot accommodate one machine word.
Corrected to return the value of the bit. And correct the comments of
the function.

Changed from v1:
- Split patch
- fix comments

Xiang W (2):
  lib: sbi: Fix __atomic_op_bit_ord and comments
  lib: sbi: Replace __atomic_op_bit_ord with __atomic intrinsics

 include/sbi/riscv_atomic.h |  8 ++++----
 lib/sbi/riscv_atomic.c     | 40 ++++++++++++--------------------------
 2 files changed, 16 insertions(+), 32 deletions(-)

-- 
2.42.0



From wxjstz at 126.com  Wed Nov 15 06:59:22 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 15 Nov 2023 22:59:22 +0800
Subject: [PATCH v2 2/2] lib: sbi: Replace __atomic_op_bit_ord with __atomic intrinsics
In-Reply-To: <20231115145929.488427-1-wxjstz@126.com>
References: <20231115145929.488427-1-wxjstz@126.com>
Message-ID: <20231115145929.488427-3-wxjstz@126.com>

Simplify atomic-related bit operations through __atomic intrinsics.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/riscv_atomic.c | 40 ++++++++++++----------------------------
 1 file changed, 12 insertions(+), 28 deletions(-)

diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
index 477f709..1c852a1 100644
--- a/lib/sbi/riscv_atomic.c
+++ b/lib/sbi/riscv_atomic.c
@@ -206,40 +206,24 @@ unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
 #endif
 }
 
-#if (__SIZEOF_POINTER__ == 8)
-#define __AMO(op) "amo" #op ".d"
-#elif (__SIZEOF_POINTER__ == 4)
-#define __AMO(op) "amo" #op ".w"
-#else
-#error "Unexpected __SIZEOF_POINTER__"
-#endif
-
-#define __atomic_op_bit_ord(op, mod, nr, addr, ord)                          \
-	({                                                                   \
-		unsigned long __res, __mask;                                 \
-		__mask = BIT_MASK(nr);                                       \
-		__asm__ __volatile__(__AMO(op) #ord " %0, %2, %1"            \
-				     : "=r"(__res), "+A"(addr[BIT_WORD(nr)]) \
-				     : "r"(mod(__mask))                      \
-				     : "memory");                            \
-		__res & mask ? 1 : 0;                                        \
-	})
-
-#define __atomic_op_bit(op, mod, nr, addr) \
-	__atomic_op_bit_ord(op, mod, nr, addr, .aqrl)
-
-/* Bitmask modifiers */
-#define __NOP(x) (x)
-#define __NOT(x) (~(x))
-
 inline int atomic_raw_set_bit(int nr, volatile unsigned long *addr)
 {
-	return __atomic_op_bit(or, __NOP, nr, addr);
+	volatile unsigned long *ptr;
+	unsigned long res, mask;
+	mask = BIT_MASK(nr);
+	ptr = &addr[BIT_WORD(nr)];
+	res = __atomic_fetch_or(ptr, mask, __ATOMIC_ACQ_REL);
+	return res & mask ? 1 : 0;
 }
 
 inline int atomic_raw_clear_bit(int nr, volatile unsigned long *addr)
 {
-	return __atomic_op_bit(and, __NOT, nr, addr);
+	volatile unsigned long *ptr;
+	unsigned long res, mask;
+	mask = BIT_MASK(nr);
+	ptr = &addr[BIT_WORD(nr)];
+	res = __atomic_fetch_and(ptr, ~mask, __ATOMIC_ACQ_REL);
+	return res & mask ? 1 : 0;
 }
 
 inline int atomic_set_bit(int nr, atomic_t *atom)
-- 
2.42.0



From wxjstz at 126.com  Wed Nov 15 06:59:21 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 15 Nov 2023 22:59:21 +0800
Subject: [PATCH v2 1/2] lib: sbi: Fix __atomic_op_bit_ord and comments
In-Reply-To: <20231115145929.488427-1-wxjstz@126.com>
References: <20231115145929.488427-1-wxjstz@126.com>
Message-ID: <20231115145929.488427-2-wxjstz@126.com>

The original code returns the value of the word before modification.
When modifying the upper 32 bits under RV64, the value returned via
int return will have no meaning. Corrected to return the value of the
bit. And modify the function description.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/riscv_atomic.h | 8 ++++----
 lib/sbi/riscv_atomic.c     | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/include/sbi/riscv_atomic.h b/include/sbi/riscv_atomic.h
index 3972e0b..c5aa05e 100644
--- a/include/sbi/riscv_atomic.h
+++ b/include/sbi/riscv_atomic.h
@@ -39,14 +39,14 @@ unsigned int atomic_raw_xchg_uint(volatile unsigned int *ptr,
 unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
 				    unsigned long newval);
 /**
- * Set a bit in an atomic variable and return the new value.
+ * Set a bit in an atomic variable and return the value of bit before modify.
  * @nr : Bit to set.
  * @atom: atomic variable to modify
  */
 int atomic_set_bit(int nr, atomic_t *atom);
 
 /**
- * Clear a bit in an atomic variable and return the new value.
+ * Clear a bit in an atomic variable and return the value of bit before modify.
  * @nr : Bit to set.
  * @atom: atomic variable to modify
  */
@@ -54,14 +54,14 @@ int atomic_set_bit(int nr, atomic_t *atom);
 int atomic_clear_bit(int nr, atomic_t *atom);
 
 /**
- * Set a bit in any address and return the new value .
+ * Set a bit in any address and return the value of bit before modify.
  * @nr : Bit to set.
  * @addr: Address to modify
  */
 int atomic_raw_set_bit(int nr, volatile unsigned long *addr);
 
 /**
- * Clear a bit in any address and return the new value .
+ * Clear a bit in any address and return the value of bit before modify.
  * @nr : Bit to set.
  * @addr: Address to modify
  */
diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
index 528686f..477f709 100644
--- a/lib/sbi/riscv_atomic.c
+++ b/lib/sbi/riscv_atomic.c
@@ -222,7 +222,7 @@ unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
 				     : "=r"(__res), "+A"(addr[BIT_WORD(nr)]) \
 				     : "r"(mod(__mask))                      \
 				     : "memory");                            \
-		__res;                                                       \
+		__res & mask ? 1 : 0;                                        \
 	})
 
 #define __atomic_op_bit(op, mod, nr, addr) \
-- 
2.42.0



From ganboing at gmail.com  Wed Nov 15 16:25:48 2023
From: ganboing at gmail.com (Bo Gan)
Date: Wed, 15 Nov 2023 16:25:48 -0800
Subject: Security and correctness issues around captured FDT pointers
In-Reply-To: <CAK9=C2W3d1t+83kNoWhkA81M59Wen2gGa_EcgOJA1U19WJP4mQ@mail.gmail.com>
References: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>
 <CAK9=C2W3d1t+83kNoWhkA81M59Wen2gGa_EcgOJA1U19WJP4mQ@mail.gmail.com>
Message-ID: <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>

On 11/13/23 9:29 AM, Anup Patel wrote:
> 
> Yes, #1 is mostly safe.
> 
> The #2 needs to be fixed in the generic/starfive/jh7110.c and all other places
> where it exists.
> 
> We should certainly stop accessing FDT after cold boot hard is done but I don't
> think we should zero-out pointers as well.
> 
Thanks for the reply, Anup. I did a little bit more digging on this and I think
#1 is only safe when OpenSBI's carrying the FDT itself (through FW_FDT_PATH).
Otherwise, we have to make sure FDT will never be accessed after cold boot done.
By zeroing-out the pointers, we can avoid speculative access as well. This helps
to defense against speculative timing attacks. (Maybe I'm too paranoid?).

Given OpenSBI starts at very early stage, it's probably uncommon to not have
FW_FDT_PATH defined. I have a JH7110 on hand, and the OpenSBI (loaded by U-Boot
SPL) will carry the FDT built from U-Boot. I'd imagine other boards/vendors do
it similarly. Just in case folks really don't have FW_FDT_PATH, can we emit a
warning, and suggest the FDT to be protected through other mechanism?

For #2, can @Minda Chen or @Wei Liang help take a look? I see no reason to have
the `const char *compatible` field in `struct pmic`. The compatible string is
always going to be "stf,axp15060-regulator" in the current implementation, so
what's point of strcmp it again in `pmic_ops`? If it's for future extension,
can you strcmp it inside `pm_reset_init`, and use an enum to indicate the model?

Bo


From ganboing at gmail.com  Wed Nov 15 16:33:29 2023
From: ganboing at gmail.com (Bo Gan)
Date: Wed, 15 Nov 2023 16:33:29 -0800
Subject: Question about potential missed IPI events
In-Reply-To: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
Message-ID: <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>

On 11/10/23 10:05 PM, Bo Gan wrote:
> Hi OpenSBI community,
> 
> Current IPI send/recv implementation uses ipi_data->ipi_type to indicate
> an IPI event pending. In sbi_ipi_send, I see have the following:
> 
>> /*
>> ?* Set IPI type on remote hart's scratch area and
>> ?* trigger the interrupt
>> ?*/
>> atomic_raw_set_bit(event, &ipi_data->ipi_type);
>> smp_wmb();
>>
>> if (ipi_dev && ipi_dev->ipi_send)
>> ????ipi_dev->ipi_send(remote_hartindex)
> 
> And in sbi_ipi_process, the following:
> 
>> if (ipi_dev && ipi_dev->ipi_clear)
>> ????ipi_dev->ipi_clear(hartindex);
>>
>> ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
>> ipi_event = 0;
>> while (ipi_type) {
> 
> I'm think of if there's any possibility of missed IPIs. Assuming we are
> talking about sifive/clint (aclint_mswi.c), I know it means that there're
> both smp_wmb (fence w,w) and __io_bw (fence w,o) between atomic_raw_set_bit
> and the write to remote hart's msip register in clint. Consider the following
> 
> Right before Hart A ipi_dev->ipi_clear() for the msip write-1 issued by hart B,
> Hart C did an ipi_dev->ipi_send to hart A. Hart A then cleared the msip bit,
> resulting in both IPI (from B and C) coalesced to a single one. In this case,
> A will observe the ipi_type set by B, due to the smp_wmb + __io_bw, and the IPI
> was indeed generated by B's write to msip. However will the barrier ensure A
> also observe the write to ipi_type from hart C? For the coalescing to happen,
> clint will observe C's write-1 before A's write-0, does it imply A'll observe
> C's write to ipi_type from xchg?
> 
> I doubt it mainly due to 2 reasons:
> 
> 1. ipi_dev->ipi_clear is a writel(msip), which has __io_bw (fence w,o) I don't
>  ?? think it helps preventing the writel to happen before xchg. If that happens,
>  ?? it's possible that A missed C's write to ipi_type.
> 
> 2. Suppose the ipi_clear() happened before xchg, from A's standpoint, it's a
>  ?? write, not a read. A wasn't reading from msip, but only writing to it. I'm
>  ?? not sure if A's write to msip after C's write (observed by clint) implies
>  ?? A's observing the same order. A is not reading from msip, so what helps to
>  ?? establish this order? Will the RVWMO global memory order help? Can we think
>  ?? of clint as another hart that follows RVWMO?
> 
> Bo
> 

Hi @Atish @Anup and other folks, any thoughts on this?


From inochiama at outlook.com  Wed Nov 15 16:37:06 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 08:37:06 +0800
Subject: [PATCH v3 2/2] lib: sbi: Add regions merging when sanitizing domain region
In-Reply-To: <CAAhSdy3dcSXBcC2BXk+vBhyHMqos0Uf8C-n26RPXP3aTsY+yzg@mail.gmail.com>
References: <CAAhSdy3dcSXBcC2BXk+vBhyHMqos0Uf8C-n26RPXP3aTsY+yzg@mail.gmail.com>
Message-ID: <IA1PR20MB49531646A00D16AC4CA87764BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

>On Wed, Nov 15, 2023 at 1:35?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> As the domain will reject a new memory region which has a sub-regions
>> already in the domain, even the new region is bigger and has the same
>> flags. This problem can be solved by relaxing region restriction and
>> rechecking when adding and sanitizing domains.
>>
>> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
>> Reviewed-by: Anup Patel <anup at brainfault.org>
>> ---
>>  lib/sbi/sbi_domain.c | 68 ++++++++++++++++++++++++++++++--------------
>>  1 file changed, 46 insertions(+), 22 deletions(-)
>>
>> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
>> index 71cb381..ba8ce44 100644
>> --- a/lib/sbi/sbi_domain.c
>> +++ b/lib/sbi/sbi_domain.c
>> @@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
>>         return false;
>>  }
>>
>> -/** Check if regionA conflicts regionB */
>> -static bool is_region_conflict(const struct sbi_domain_memregion *regA,
>> -                               const struct sbi_domain_memregion *regB)
>> +/** Check if regionA can be replaced by regionB */
>> +static bool is_region_compatible(const struct sbi_domain_memregion *regA,
>> +                                const struct sbi_domain_memregion *regB)
>>  {
>> -       if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
>> -           regA->flags == regB->flags)
>> +       if (is_region_subset(regA, regB) && regA->flags == regB->flags)
>>                 return true;
>>
>>         return false;
>> @@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
>>         sbi_memcpy(reg2, &treg, sizeof(treg));
>>  }
>>
>> +static void clear_region(struct sbi_domain_memregion* reg)
>> +{
>> +       sbi_memset(reg, 0x0, sizeof(*reg));
>> +}
>> +
>>  static int sanitize_domain(struct sbi_domain *dom)
>>  {
>>         u32 i, j, count;
>> @@ -314,22 +318,46 @@ static int sanitize_domain(struct sbi_domain *dom)
>>                 return SBI_EINVAL;
>>         }
>>
>> +       /* Remove covered regions */
>> +       while(i < (count - 1)) {
>> +               reg = &dom->regions[i];
>> +               j = i + 1;
>> +               while (j < count) {
>> +                       reg1 = &dom->regions[j];
>> +
>> +                       /* reg1 is sub-region of reg, remove reg1 */
>> +                       if (is_region_compatible(reg1, reg)) {
>> +                               swap_region(reg1, &dom->regions[count - 1]);
>> +                               clear_region(&dom->regions[count - 1]);
>> +                               count--;
>> +                               continue;
>> +                       }
>> +
>> +                       /*
>> +                        * reg is sub-region of reg1, replace reg with reg1
>> +                        * and recheck all other regions.
>> +                        */
>> +                       if (is_region_compatible(reg, reg1)) {
>> +                               swap_region(reg, reg1);
>> +                               swap_region(reg1, &dom->regions[count - 1]);
>> +                               clear_region(&dom->regions[count - 1]);
>
>This does not seem right.
>
>We should disturb the sorted order of regions so all regions
>from i + 1 to count - 1 should be shifted up.
>

It already shift up the region from i + 1 to count - 1.
In fact, the process can be described as follows:

R[i] -> ... -> R[j] -> ... -> R[c-2] -> R[c-1]        (1)
  i              j                      count

R[j] -> ... -> R[i] -> ... -> R[c-2] -> R[c-1]        (2)
  i              j                      count

R[j] -> ... -> R[c-1] -> ... -> R[c-2] -> R[i]        (3)
  i              j                        count

R[j] -> ... -> R[c-1] -> ... -> R[c-2]                (4)
  i              j              count

It just replaces pos i with j, the replace pos j with last. This make no
change for the whole check. It just needs a recheck at pos i with new
count - 1 boundary after swapping.

The only thing needed to be changed is just replacing pos i with the last,
without replacing pos i and j. Because we must recheck all the left regions.
So swapping pos i and j gives no help.

Please correct me if there is something I miss. Thanks.

>> +                               count--;
>> +                               goto regions_recheck;
>> +                       }
>> +
>> +                       j++;
>> +               }
>> +
>> +               i++;
>> +regions_recheck:
>> +       }
>> +
>>         /* Sort the memory regions */
>>         for (i = 0; i < (count - 1); i++) {
>>                 reg = &dom->regions[i];
>>                 for (j = i + 1; j < count; j++) {
>>                         reg1 = &dom->regions[j];
>>
>> -                       if (is_region_conflict(reg1, reg)) {
>> -                               sbi_printf("%s: %s conflict between regions "
>> -                                       "(base=0x%lx order=%lu flags=0x%lx) and "
>> -                                       "(base=0x%lx order=%lu flags=0x%lx)\n",
>> -                                       __func__, dom->name,
>> -                                       reg->base, reg->order, reg->flags,
>> -                                       reg1->base, reg1->order, reg1->flags);
>> -                               return SBI_EINVAL;
>> -                       }
>> -
>>                         if (!is_region_before(reg1, reg))
>>                                 continue;
>>
>> @@ -576,14 +604,10 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
>>             (ROOT_REGION_MAX <= root_memregs_count))
>>                 return SBI_EINVAL;
>>
>> -       /* Check for conflicts */
>> +       /* Check whether compatible region exists for the new one */
>>         sbi_domain_for_each_memregion(&root, nreg) {
>> -               if (is_region_conflict(reg, nreg)) {
>> -                       sbi_printf("%s: is_region_conflict check failed"
>> -                       " 0x%lx conflicts existing 0x%lx\n", __func__,
>> -                                  reg->base, nreg->base);
>> -                       return SBI_EALREADY;
>> -               }
>> +               if (is_region_compatible(reg, nreg))
>> +                       return 0;
>>         }
>>
>>         /* Append the memregion to root memregions */
>> --
>> 2.42.1
>>
>
>Regards,
>Anup
>
>


From samuel.holland at sifive.com  Wed Nov 15 17:03:00 2023
From: samuel.holland at sifive.com (Samuel Holland)
Date: Wed, 15 Nov 2023 20:03:00 -0500
Subject: Security and correctness issues around captured FDT pointers
In-Reply-To: <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>
References: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>
 <CAK9=C2W3d1t+83kNoWhkA81M59Wen2gGa_EcgOJA1U19WJP4mQ@mail.gmail.com>
 <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>
Message-ID: <43407cfa-5d82-4337-ae99-8ddac8f23bf6@sifive.com>

Hi Bo,

On 2023-11-15 6:25 PM, Bo Gan wrote:
> On 11/13/23 9:29 AM, Anup Patel wrote:
>>
>> Yes, #1 is mostly safe.
>>
>> The #2 needs to be fixed in the generic/starfive/jh7110.c and all other places
>> where it exists.
>>
>> We should certainly stop accessing FDT after cold boot hard is done but I don't
>> think we should zero-out pointers as well.
>>
> Thanks for the reply, Anup. I did a little bit more digging on this and I think
> #1 is only safe when OpenSBI's carrying the FDT itself (through FW_FDT_PATH).
> Otherwise, we have to make sure FDT will never be accessed after cold boot done.
> By zeroing-out the pointers, we can avoid speculative access as well. This helps
> to defense against speculative timing attacks. (Maybe I'm too paranoid?).
> 
> Given OpenSBI starts at very early stage, it's probably uncommon to not have
> FW_FDT_PATH defined. I have a JH7110 on hand, and the OpenSBI (loaded by U-Boot
> SPL) will carry the FDT built from U-Boot. I'd imagine other boards/vendors do
> it similarly. Just in case folks really don't have FW_FDT_PATH, can we emit a
> warning, and suggest the FDT to be protected through other mechanism?

If the FDT comes from U-Boot (the common case) or another prior stage (e.g.
QEMU), that means the FDT address was provided in a register, and FW_FDT_PATH is
*not* defined. So we should not assume FW_FDT_PATH is set.

Regards,
Samuel



From ganboing at gmail.com  Wed Nov 15 17:29:20 2023
From: ganboing at gmail.com (Bo Gan)
Date: Wed, 15 Nov 2023 17:29:20 -0800
Subject: Security and correctness issues around captured FDT pointers
In-Reply-To: <43407cfa-5d82-4337-ae99-8ddac8f23bf6@sifive.com>
References: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>
 <CAK9=C2W3d1t+83kNoWhkA81M59Wen2gGa_EcgOJA1U19WJP4mQ@mail.gmail.com>
 <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>
 <43407cfa-5d82-4337-ae99-8ddac8f23bf6@sifive.com>
Message-ID: <9db84767-3623-1ac5-ed56-b11a6e052fa2@gmail.com>

On 11/15/23 5:03 PM, Samuel Holland wrote:
> Hi Bo,
> 
> On 2023-11-15 6:25 PM, Bo Gan wrote:
>> On 11/13/23 9:29 AM, Anup Patel wrote:
>>>
>>> Yes, #1 is mostly safe.
>>>
>>> The #2 needs to be fixed in the generic/starfive/jh7110.c and all other places
>>> where it exists.
>>>
>>> We should certainly stop accessing FDT after cold boot hard is done but I don't
>>> think we should zero-out pointers as well.
>>>
>> Thanks for the reply, Anup. I did a little bit more digging on this and I think
>> #1 is only safe when OpenSBI's carrying the FDT itself (through FW_FDT_PATH).
>> Otherwise, we have to make sure FDT will never be accessed after cold boot done.
>> By zeroing-out the pointers, we can avoid speculative access as well. This helps
>> to defense against speculative timing attacks. (Maybe I'm too paranoid?).
>>
>> Given OpenSBI starts at very early stage, it's probably uncommon to not have
>> FW_FDT_PATH defined. I have a JH7110 on hand, and the OpenSBI (loaded by U-Boot
>> SPL) will carry the FDT built from U-Boot. I'd imagine other boards/vendors do
>> it similarly. Just in case folks really don't have FW_FDT_PATH, can we emit a
>> warning, and suggest the FDT to be protected through other mechanism?
> 
> If the FDT comes from U-Boot (the common case) or another prior stage (e.g.
> QEMU), that means the FDT address was provided in a register, and FW_FDT_PATH is
> *not* defined. So we should not assume FW_FDT_PATH is set.
> 
> Regards,
> Samuel
> 

Hi Samuel, thanks for the reply. Yes, we should not assume FW_FDT_PATH, and if
it's pretty common for real world/production setup to use the FDT from previous
boot stage, I'd suggest we ensure there's never global variables or heap variables
that can capture any pointer to FDT, and all access to FDT must happen before cold-
boot done. Otherwise, the memory region contains FDT passed from previous bootstage
must be protected, but surely it'll complicate things and could potentially involve
platform specific handling.

Bo


From wxjstz at 126.com  Wed Nov 15 19:12:58 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 16 Nov 2023 11:12:58 +0800
Subject: Question about potential missed IPI events
In-Reply-To: <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
	 <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
Message-ID: <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>

? 2023-11-15???? 16:33 -0800?Bo Gan???
> On 11/10/23 10:05 PM, Bo Gan wrote:
> > Hi OpenSBI community,
> > 
> > Current IPI send/recv implementation uses ipi_data->ipi_type to indicate
> > an IPI event pending. In sbi_ipi_send, I see have the following:
> > 
> > > /*
> > > ?* Set IPI type on remote hart's scratch area and
> > > ?* trigger the interrupt
> > > ?*/
> > > atomic_raw_set_bit(event, &ipi_data->ipi_type);
> > > smp_wmb();
> > > 
> > > if (ipi_dev && ipi_dev->ipi_send)
> > > ????ipi_dev->ipi_send(remote_hartindex)
> > 
> > And in sbi_ipi_process, the following:
> > 
> > > if (ipi_dev && ipi_dev->ipi_clear)
> > > ????ipi_dev->ipi_clear(hartindex);
> > > 
> > > ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
> > > ipi_event = 0;
> > > while (ipi_type) {
> > 
> > I'm think of if there's any possibility of missed IPIs. Assuming we are
> > talking about sifive/clint (aclint_mswi.c), I know it means that there're
> > both smp_wmb (fence w,w) and __io_bw (fence w,o) between atomic_raw_set_bit
> > and the write to remote hart's msip register in clint. Consider the following
> > 
> > Right before Hart A ipi_dev->ipi_clear() for the msip write-1 issued by hart B,
> > Hart C did an ipi_dev->ipi_send to hart A. Hart A then cleared the msip bit,
> > resulting in both IPI (from B and C) coalesced to a single one. In this case,
> > A will observe the ipi_type set by B, due to the smp_wmb + __io_bw, and the IPI
> > was indeed generated by B's write to msip. However will the barrier ensure A
> > also observe the write to ipi_type from hart C? For the coalescing to happen,
> > clint will observe C's write-1 before A's write-0, does it imply A'll observe
> > C's write to ipi_type from xchg?
Does the picture below represent what you mean?

 B             A             C
 |             |             |
 | xchg        |             |
 | send------->|             |   
 |             | xchg        |
 |             |             | xchg
 |             |<------------| send
 |             | clear       |
> 

In fact, A will not lose C's ipi message and will be deleted when it receives
a new ipi interrupt, as shown below
 B             A             C             D
 |             |             |             |
 | xchg        |             |             |
 | send------->|             |             |
 |             | xchg        |             |
 |             |             | xchg        |
 |             |<------------| send        |
 |             | clear       |             |
               |                           |
               |<--------------------------| send

There is another situation that will cause the efficiency to drop. Please refer to
the figure below.
 B             A             C
 |             |             |
 | xchg        |             |
 | send------->|             | xchg  
 |             | clear       |
 |             | xchg        |
 |             |<------------| send
 |             |             |

A may enter exception handling one more time.


I think we need to protect xchg/send and clear/xchg with spinlock.


Regards,
Xiang W
> > I doubt it mainly due to 2 reasons:
> > 
> > 1. ipi_dev->ipi_clear is a writel(msip), which has __io_bw (fence w,o) I don't
> > ??? think it helps preventing the writel to happen before xchg. If that happens,
> > ??? it's possible that A missed C's write to ipi_type.
> > 
> > 2. Suppose the ipi_clear() happened before xchg, from A's standpoint, it's a
> > ??? write, not a read. A wasn't reading from msip, but only writing to it. I'm
> > ??? not sure if A's write to msip after C's write (observed by clint) implies
> > ??? A's observing the same order. A is not reading from msip, so what helps to
> > ??? establish this order? Will the RVWMO global memory order help? Can we think
> > ??? of clint as another hart that follows RVWMO?
> > 
> > Bo
> > 
> 
> Hi @Atish @Anup and other folks, any thoughts on this?
> 



From anup at brainfault.org  Wed Nov 15 19:39:13 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 09:09:13 +0530
Subject: [PATCH v3 2/2] lib: sbi: Add regions merging when sanitizing
 domain region
In-Reply-To: <IA1PR20MB49531646A00D16AC4CA87764BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <CAAhSdy3dcSXBcC2BXk+vBhyHMqos0Uf8C-n26RPXP3aTsY+yzg@mail.gmail.com>
 <IA1PR20MB49531646A00D16AC4CA87764BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy2Dw6r77DS7ZDZk6fhjRKNTCJN-nOC-_9uHJPg05-qr8w@mail.gmail.com>

On Thu, Nov 16, 2023 at 6:06?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> >On Wed, Nov 15, 2023 at 1:35?PM Inochi Amaoto <inochiama at outlook.com> wrote:
> >>
> >> As the domain will reject a new memory region which has a sub-regions
> >> already in the domain, even the new region is bigger and has the same
> >> flags. This problem can be solved by relaxing region restriction and
> >> rechecking when adding and sanitizing domains.
> >>
> >> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> >> Reviewed-by: Anup Patel <anup at brainfault.org>
> >> ---
> >>  lib/sbi/sbi_domain.c | 68 ++++++++++++++++++++++++++++++--------------
> >>  1 file changed, 46 insertions(+), 22 deletions(-)
> >>
> >> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> >> index 71cb381..ba8ce44 100644
> >> --- a/lib/sbi/sbi_domain.c
> >> +++ b/lib/sbi/sbi_domain.c
> >> @@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
> >>         return false;
> >>  }
> >>
> >> -/** Check if regionA conflicts regionB */
> >> -static bool is_region_conflict(const struct sbi_domain_memregion *regA,
> >> -                               const struct sbi_domain_memregion *regB)
> >> +/** Check if regionA can be replaced by regionB */
> >> +static bool is_region_compatible(const struct sbi_domain_memregion *regA,
> >> +                                const struct sbi_domain_memregion *regB)
> >>  {
> >> -       if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
> >> -           regA->flags == regB->flags)
> >> +       if (is_region_subset(regA, regB) && regA->flags == regB->flags)
> >>                 return true;
> >>
> >>         return false;
> >> @@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
> >>         sbi_memcpy(reg2, &treg, sizeof(treg));
> >>  }
> >>
> >> +static void clear_region(struct sbi_domain_memregion* reg)
> >> +{
> >> +       sbi_memset(reg, 0x0, sizeof(*reg));
> >> +}
> >> +
> >>  static int sanitize_domain(struct sbi_domain *dom)
> >>  {
> >>         u32 i, j, count;
> >> @@ -314,22 +318,46 @@ static int sanitize_domain(struct sbi_domain *dom)
> >>                 return SBI_EINVAL;
> >>         }
> >>
> >> +       /* Remove covered regions */
> >> +       while(i < (count - 1)) {
> >> +               reg = &dom->regions[i];
> >> +               j = i + 1;
> >> +               while (j < count) {
> >> +                       reg1 = &dom->regions[j];
> >> +
> >> +                       /* reg1 is sub-region of reg, remove reg1 */
> >> +                       if (is_region_compatible(reg1, reg)) {
> >> +                               swap_region(reg1, &dom->regions[count - 1]);
> >> +                               clear_region(&dom->regions[count - 1]);
> >> +                               count--;
> >> +                               continue;
> >> +                       }
> >> +
> >> +                       /*
> >> +                        * reg is sub-region of reg1, replace reg with reg1
> >> +                        * and recheck all other regions.
> >> +                        */
> >> +                       if (is_region_compatible(reg, reg1)) {
> >> +                               swap_region(reg, reg1);
> >> +                               swap_region(reg1, &dom->regions[count - 1]);
> >> +                               clear_region(&dom->regions[count - 1]);
> >
> >This does not seem right.
> >
> >We should disturb the sorted order of regions so all regions
> >from i + 1 to count - 1 should be shifted up.
> >
>
> It already shift up the region from i + 1 to count - 1.
> In fact, the process can be described as follows:
>
> R[i] -> ... -> R[j] -> ... -> R[c-2] -> R[c-1]        (1)
>   i              j                      count
>
> R[j] -> ... -> R[i] -> ... -> R[c-2] -> R[c-1]        (2)
>   i              j                      count
>
> R[j] -> ... -> R[c-1] -> ... -> R[c-2] -> R[i]        (3)
>   i              j                        count
>
> R[j] -> ... -> R[c-1] -> ... -> R[c-2]                (4)
>   i              j              count
>
> It just replaces pos i with j, the replace pos j with last. This make no
> change for the whole check. It just needs a recheck at pos i with new
> count - 1 boundary after swapping.
>
> The only thing needed to be changed is just replacing pos i with the last,
> without replacing pos i and j. Because we must recheck all the left regions.
> So swapping pos i and j gives no help.

This is exactly the problem. You are not maintaining the sorted
order of the regions.

..., R[i-1], R[i], R[i+1], ..., R[j-1], R[j], R[j+1], ....,
R[count-2], R[count-1]

to preserve the sorted order of regions after removal of R[i], this will be

..., R[i-1], R[i+1], ..., R[j-1], R[j], R[j+1], ...., R[count-2], R[count-1]

whereas you patch changes this to

..., R[i-1], R[j], R[i+1], ..., R[j-1], R[count-1], R[j+1], ..., R[count-2]

Further, the checking "reg1 is sub-region of reg, remove reg1" is
totally redundant if the "Remove covered regions" is moved after
"Sort the memory regions".

This patch needs to be re-written.

Regards,
Anup

>
> Please correct me if there is something I miss. Thanks.
>
> >> +                               count--;
> >> +                               goto regions_recheck;
> >> +                       }
> >> +
> >> +                       j++;
> >> +               }
> >> +
> >> +               i++;
> >> +regions_recheck:
> >> +       }
> >> +
> >>         /* Sort the memory regions */
> >>         for (i = 0; i < (count - 1); i++) {
> >>                 reg = &dom->regions[i];
> >>                 for (j = i + 1; j < count; j++) {
> >>                         reg1 = &dom->regions[j];
> >>
> >> -                       if (is_region_conflict(reg1, reg)) {
> >> -                               sbi_printf("%s: %s conflict between regions "
> >> -                                       "(base=0x%lx order=%lu flags=0x%lx) and "
> >> -                                       "(base=0x%lx order=%lu flags=0x%lx)\n",
> >> -                                       __func__, dom->name,
> >> -                                       reg->base, reg->order, reg->flags,
> >> -                                       reg1->base, reg1->order, reg1->flags);
> >> -                               return SBI_EINVAL;
> >> -                       }
> >> -
> >>                         if (!is_region_before(reg1, reg))
> >>                                 continue;
> >>
> >> @@ -576,14 +604,10 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
> >>             (ROOT_REGION_MAX <= root_memregs_count))
> >>                 return SBI_EINVAL;
> >>
> >> -       /* Check for conflicts */
> >> +       /* Check whether compatible region exists for the new one */
> >>         sbi_domain_for_each_memregion(&root, nreg) {
> >> -               if (is_region_conflict(reg, nreg)) {
> >> -                       sbi_printf("%s: is_region_conflict check failed"
> >> -                       " 0x%lx conflicts existing 0x%lx\n", __func__,
> >> -                                  reg->base, nreg->base);
> >> -                       return SBI_EALREADY;
> >> -               }
> >> +               if (is_region_compatible(reg, nreg))
> >> +                       return 0;
> >>         }
> >>
> >>         /* Append the memregion to root memregions */
> >> --
> >> 2.42.1
> >>
> >
> >Regards,
> >Anup
> >
> >


From anup at brainfault.org  Wed Nov 15 19:42:54 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 09:12:54 +0530
Subject: Security and correctness issues around captured FDT pointers
In-Reply-To: <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>
References: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>
 <CAK9=C2W3d1t+83kNoWhkA81M59Wen2gGa_EcgOJA1U19WJP4mQ@mail.gmail.com> <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>
Message-ID: <CAAhSdy3xDs54CCNzQFBHZsyV+iEMTEf0jw-yXsoi4u9osRYJQA@mail.gmail.com>

On Thu, Nov 16, 2023 at 5:56?AM Bo Gan <ganboing at gmail.com> wrote:
>
> On 11/13/23 9:29 AM, Anup Patel wrote:
> >
> > Yes, #1 is mostly safe.
> >
> > The #2 needs to be fixed in the generic/starfive/jh7110.c and all other places
> > where it exists.
> >
> > We should certainly stop accessing FDT after cold boot hard is done but I don't
> > think we should zero-out pointers as well.
> >
> Thanks for the reply, Anup. I did a little bit more digging on this and I think
> #1 is only safe when OpenSBI's carrying the FDT itself (through FW_FDT_PATH).
> Otherwise, we have to make sure FDT will never be accessed after cold boot done.
> By zeroing-out the pointers, we can avoid speculative access as well. This helps
> to defense against speculative timing attacks. (Maybe I'm too paranoid?).
>
> Given OpenSBI starts at very early stage, it's probably uncommon to not have
> FW_FDT_PATH defined. I have a JH7110 on hand, and the OpenSBI (loaded by U-Boot
> SPL) will carry the FDT built from U-Boot. I'd imagine other boards/vendors do
> it similarly. Just in case folks really don't have FW_FDT_PATH, can we emit a
> warning, and suggest the FDT to be protected through other mechanism?

That's an incorrect understanding of the FDT match table in OpenSBI.

The "generic_plat_match" points to a matching entry in a FDT match table
provided by one of the platform override instances. It does not point to any
part of the FDT. Please look at the fdt_match_node() implementation.

>
> For #2, can @Minda Chen or @Wei Liang help take a look? I see no reason to have
> the `const char *compatible` field in `struct pmic`. The compatible string is
> always going to be "stf,axp15060-regulator" in the current implementation, so
> what's point of strcmp it again in `pmic_ops`? If it's for future extension,
> can you strcmp it inside `pm_reset_init`, and use an enum to indicate the model?
>
> Bo
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From anup at brainfault.org  Wed Nov 15 20:03:51 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 09:33:51 +0530
Subject: [PATCH] lib: sbi: Correctly limit flushes to a single ASID/VMID
In-Reply-To: <20231017222027.3668468-1-samuel.holland@sifive.com>
References: <20231017222027.3668468-1-samuel.holland@sifive.com>
Message-ID: <CAAhSdy1tW2YNQzkDqsRWhSw0J7Vfg9DX5hSHarnw246QGL3Zrg@mail.gmail.com>

On Wed, Oct 18, 2023 at 3:50?AM Samuel Holland
<samuel.holland at sifive.com> wrote:
>
> Per the SBI specification, the effects of these functions are limited to
> a specific ASID and/or VMID. This applies even when flushing the entire
> address space.
>
> Signed-off-by: Samuel Holland <samuel.holland at sifive.com>

This patch looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

> ---
> The SBI spec doesn't say anything about this `start == 0 && size == 0`
> exception, so I wonder if it should be removed entirely.

The SBI spec does define behaviour for `start == 0 && size == 0`.

At the beginning of "Chapter 8. RFENCE Extension" we have the
following text which describes this behavior:

"
The remote fence function acts as a full TLB flush if
* start_addr and size are both 0
* size is equal to 2^XLEN-1
"

>
>  lib/sbi/sbi_tlb.c | 21 +++------------------
>  1 file changed, 3 insertions(+), 18 deletions(-)
>
> diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
> index ced2854..dad9508 100644
> --- a/lib/sbi/sbi_tlb.c
> +++ b/lib/sbi/sbi_tlb.c
> @@ -111,12 +111,7 @@ void sbi_tlb_local_hfence_vvma_asid(struct sbi_tlb_info *tinfo)
>         hgatp = csr_swap(CSR_HGATP,
>                          (vmid << HGATP_VMID_SHIFT) & HGATP_VMID_MASK);
>
> -       if (start == 0 && size == 0) {
> -               __sbi_hfence_vvma_all();
> -               goto done;
> -       }
> -
> -       if (size == SBI_TLB_FLUSH_ALL) {
> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
>                 __sbi_hfence_vvma_asid(asid);
>                 goto done;
>         }
> @@ -138,12 +133,7 @@ void sbi_tlb_local_hfence_gvma_vmid(struct sbi_tlb_info *tinfo)
>
>         sbi_pmu_ctr_incr_fw(SBI_PMU_FW_HFENCE_GVMA_VMID_RCVD);
>
> -       if (start == 0 && size == 0) {
> -               __sbi_hfence_gvma_all();
> -               return;
> -       }
> -
> -       if (size == SBI_TLB_FLUSH_ALL) {
> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
>                 __sbi_hfence_gvma_vmid(vmid);
>                 return;
>         }
> @@ -162,13 +152,8 @@ void sbi_tlb_local_sfence_vma_asid(struct sbi_tlb_info *tinfo)
>
>         sbi_pmu_ctr_incr_fw(SBI_PMU_FW_SFENCE_VMA_ASID_RCVD);
>
> -       if (start == 0 && size == 0) {
> -               tlb_flush_all();
> -               return;
> -       }
> -
>         /* Flush entire MM context for a given ASID */
> -       if (size == SBI_TLB_FLUSH_ALL) {
> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
>                 __asm__ __volatile__("sfence.vma x0, %0"
>                                      :
>                                      : "r"(asid)
> --
> 2.41.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Thanks,
Anup


From anup at brainfault.org  Wed Nov 15 20:15:41 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 09:45:41 +0530
Subject: [PATCH] platform: generic: thead: improve tlb flush errata
In-Reply-To: <IA1PR20MB49530A123B23DFFED0CC3E06BBD3A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49530A123B23DFFED0CC3E06BBD3A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy1QsY3skTBFj+kAifnb5B_HSKmMgerjNL76uCAUJQ2xhg@mail.gmail.com>

On Thu, Oct 12, 2023 at 7:24?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Flushing the tlb entries can solve the thead tlb problem, but flushing
> it by address will miss something and lead to a exception in some rare
> cases, and this is more common for sg2042.
>
> To solve this problem, flush the tlb entries by asid in the custom trap
> handler to ensure it is refreshed.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
> This patch based on the th1520 series patch.
> ---
>  platform/generic/thead/thead_c9xx_tlb_trap_handler.S | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/platform/generic/thead/thead_c9xx_tlb_trap_handler.S b/platform/generic/thead/thead_c9xx_tlb_trap_handler.S
> index 861c029..120daa7 100644
> --- a/platform/generic/thead/thead_c9xx_tlb_trap_handler.S
> +++ b/platform/generic/thead/thead_c9xx_tlb_trap_handler.S
> @@ -9,5 +9,5 @@
>         .align 3
>         .globl _thead_tlb_flush_fixup_trap_handler
>  _thead_tlb_flush_fixup_trap_handler:
> -       sfence.vma t0, zero
> +       sfence.vma zero, t0
>         j _trap_handler
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Nov 15 21:26:59 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 10:56:59 +0530
Subject: [PATCH v2] platform: Uses hart count as the default size of tlb info
In-Reply-To: <IA1PR20MB495379709E66871C21333296BBD3A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB495379709E66871C21333296BBD3A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy0uPpBQMoZ8xx=Q3ozeLhPqjC4QNN6cTeuf5=kHp=yXfg@mail.gmail.com>

On Thu, Oct 12, 2023 at 2:04?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> For platform with high number of harts, it is better to auto detect a
> suitable number of entries in tlb fifo. Since allocating tlb entry for
> all online harts can reduce the wait time significantly, using the
> number of the online harts can make most platforms happy. This auto
> detection can avoid most duplicate code for setting tlb fifo size.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
> Changed from v1:
> 1. replace SBI_PLATFORM_TLB_FIFO_NUM_ENTRIES with hart count in all cases.
>
> ---
>  include/sbi/sbi_platform.h  | 3 +--
>  platform/generic/platform.c | 2 +-
>  2 files changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/include/sbi/sbi_platform.h b/include/sbi/sbi_platform.h
> index 389203b..58b9069 100644
> --- a/include/sbi/sbi_platform.h
> +++ b/include/sbi/sbi_platform.h
> @@ -41,7 +41,6 @@
>  #define SBI_PLATFORM_HART_INDEX2ID_OFFSET (0x60 + (__SIZEOF_POINTER__ * 2))
>
>  #define SBI_PLATFORM_TLB_RANGE_FLUSH_LIMIT_DEFAULT             (1UL << 12)
> -#define SBI_PLATFORM_TLB_FIFO_NUM_ENTRIES                              8
>
>  #ifndef __ASSEMBLER__
>
> @@ -330,7 +329,7 @@ static inline u32 sbi_platform_tlb_fifo_num_entries(const struct sbi_platform *p
>  {
>         if (plat && sbi_platform_ops(plat)->get_tlb_num_entries)
>                 return sbi_platform_ops(plat)->get_tlb_num_entries();
> -       return SBI_PLATFORM_TLB_FIFO_NUM_ENTRIES;
> +       return sbi_scratch_last_hartindex() + 1;
>  }
>
>  /**
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 66a0b77..4246efd 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -260,7 +260,7 @@ static u32 generic_tlb_num_entries(void)
>  {
>         if (generic_plat && generic_plat->tlb_num_entries)
>                 return generic_plat->tlb_num_entries(generic_plat_match);
> -       return SBI_PLATFORM_TLB_FIFO_NUM_ENTRIES;
> +       return sbi_scratch_last_hartindex() + 1;
>  }
>
>  static int generic_pmu_init(void)
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Nov 15 21:41:45 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 11:11:45 +0530
Subject: [PATCH v2 1/3] lib: utils/timer: Allow ACLINT MTIMER driver to
 setup quirks
In-Reply-To: <IA1PR20MB4953107FD516BD3BF45704C6BBD2A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49533BA0170E1F1A025EBF99BBD2A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB4953107FD516BD3BF45704C6BBD2A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy15pM9S95joEberBR7jOuJ2pTWcns2Pb0vn0WXS8GeiPw@mail.gmail.com>

On Fri, Oct 13, 2023 at 12:17?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> The quirks checking will cause ACLINT step into a CLINT code path, this
> is not expected when ACLINT needs a quirks.
>
> Add more check to ensure the timer is CLINT before applying quirks.
> And now apply the general quirks after applying CLINT specific quirks.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> ---
>  lib/utils/timer/fdt_timer_mtimer.c | 17 ++++++++++++++---
>  1 file changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> index 9eaa11d..fb5917f 100644
> --- a/lib/utils/timer/fdt_timer_mtimer.c
> +++ b/lib/utils/timer/fdt_timer_mtimer.c
> @@ -29,6 +29,12 @@ static SBI_LIST_HEAD(mtn_list);
>
>  static struct aclint_mtimer_data *mt_reference = NULL;
>
> +static int clint_contains_timer(const void *data)
> +{
> +       const struct timer_mtimer_quirks *quirks = data;
> +       return quirks && quirks->mtime_offset;

This is already broken because you are not allowing mtime_offset
to be zero in quirks.

Drop clint_contains_timer() function.

> +}
> +
>  static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                                   const struct fdt_match *match)
>  {
> @@ -58,7 +64,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                 return rc;
>         }
>
> -       if (match->data) { /* SiFive CLINT */
> +       if (clint_contains_timer(match->data)) {
>                 const struct timer_mtimer_quirks *quirks = match->data;
>
>                 /* Set CLINT addresses */
> @@ -74,8 +80,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                         mt->mtime_addr = mt->mtime_size = 0;
>                 }
>                 mt->mtimecmp_addr += quirks->mtime_offset;
> -               /* Apply additional CLINT quirks */
> -               mt->has_64bit_mmio = quirks->has_64bit_mmio;
>         } else { /* RISC-V ACLINT MTIMER */
>                 /* Set ACLINT MTIMER addresses */
>                 mt->mtime_addr = addr[0];
> @@ -84,6 +88,13 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                 mt->mtimecmp_size = size[1];
>         }
>
> +       /* Apply additional quirks for CLINT and ACLINT */
> +       if (match->data) {
> +               const struct timer_mtimer_quirks *quirks = match->data;
> +
> +               mt->has_64bit_mmio = quirks->has_64bit_mmio;
> +       }
> +
>         /* Check if MTIMER device has shared MTIME address */
>         if (mt->mtime_size) {
>                 mt->has_shared_mtime = false;
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Instead of above, I suggest to do as follows:

diff --git a/lib/utils/timer/fdt_timer_mtimer.c
b/lib/utils/timer/fdt_timer_mtimer.c
index 9eaa11d..0187272 100644
--- a/lib/utils/timer/fdt_timer_mtimer.c
+++ b/lib/utils/timer/fdt_timer_mtimer.c
@@ -16,6 +16,7 @@
 #include <sbi_utils/timer/aclint_mtimer.h>

 struct timer_mtimer_quirks {
+    bool        custom_aclint;
     unsigned int    mtime_offset;
     bool        has_64bit_mmio;
     bool        without_mtime;
@@ -36,6 +37,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
     unsigned long addr[2], size[2];
     struct timer_mtimer_node *mtn, *n;
     struct aclint_mtimer_data *mt;
+    const struct timer_mtimer_quirks *quirks = match->data;

     mtn = sbi_zalloc(sizeof(*mtn));
     if (!mtn)
@@ -58,9 +60,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
         return rc;
     }

-    if (match->data) { /* SiFive CLINT */
-        const struct timer_mtimer_quirks *quirks = match->data;
-
+    if (quirks && !quirks->custom_aclint) { /* SiFive CLINT */
         /* Set CLINT addresses */
         mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
         mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
@@ -74,8 +74,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
             mt->mtime_addr = mt->mtime_size = 0;
         }
         mt->mtimecmp_addr += quirks->mtime_offset;
-        /* Apply additional CLINT quirks */
-        mt->has_64bit_mmio = quirks->has_64bit_mmio;
     } else { /* RISC-V ACLINT MTIMER */
         /* Set ACLINT MTIMER addresses */
         mt->mtime_addr = addr[0];
@@ -84,6 +82,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
         mt->mtimecmp_size = size[1];
     }

+    if (quirks) {
+        /* Apply additional quirks */
+        mt->has_64bit_mmio = quirks->has_64bit_mmio;
+    }
+
     /* Check if MTIMER device has shared MTIME address */
     if (mt->mtime_size) {
         mt->has_shared_mtime = false;

Regards,
Anup


From anup at brainfault.org  Wed Nov 15 22:12:53 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 11:42:53 +0530
Subject: [PATCH v2] lib: utils/serial: Optimize semihosting_putc implementation
In-Reply-To: <20231018024205.270-1-cp0613@linux.alibaba.com>
References: <20231018024205.270-1-cp0613@linux.alibaba.com>
Message-ID: <CAAhSdy0BphFUER+8kiza-Cw1qG3TD7xMZW9KkJeyAjqgxyqk4g@mail.gmail.com>

On Wed, Oct 18, 2023 at 8:12?AM <cp0613 at linux.alibaba.com> wrote:
>
> From: Chen Pei <cp0613 at linux.alibaba.com>
>
> For some debuggers that do not implement SYSWRITEC and SYSREADC
> operations, we can use SYSWRITE and SYSREAD instead like the
> implementation of semihosting_getc().
>
> Signed-off-by: Chen Pei <cp0613 at linux.alibaba.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
> Changed from v1:
> 1. simplified by calling semihosting_puts directly.
>
> ---
>  lib/utils/serial/semihosting.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/lib/utils/serial/semihosting.c b/lib/utils/serial/semihosting.c
> index ce65887..a27c69e 100644
> --- a/lib/utils/serial/semihosting.c
> +++ b/lib/utils/serial/semihosting.c
> @@ -160,11 +160,6 @@ static long semihosting_write(long fd, const void *memp, size_t len)
>
>  /* clang-format on */
>
> -static void semihosting_putc(char ch)
> -{
> -       semihosting_trap(SYSWRITEC, &ch);
> -}
> -
>  static unsigned long semihosting_puts(const char *str, unsigned long len)
>  {
>         char ch;
> @@ -183,6 +178,11 @@ static unsigned long semihosting_puts(const char *str, unsigned long len)
>         return (ret < 0) ? 0 : ret;
>  }
>
> +static void semihosting_putc(char ch)
> +{
> +       semihosting_puts(&ch, 1);
> +}
> +
>  static int semihosting_getc(void)
>  {
>         char ch = 0;
> --
> 2.25.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Nov 15 22:32:01 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:02:01 +0530
Subject: [PATCH v2 01/11] sbi: sbi_pmu: Improve sbi_pmu_init() error handling
In-Reply-To: <20231019113713.3508153-2-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-2-peterlin@andestech.com>
Message-ID: <CAAhSdy0iroZcQAgFoHtaeRo9mAWTuF6t9FZSF70fF=2wP=L+6g@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:10?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> This patch makes the following changes:
>
> - As sbi_platform_pmu_init() returns a negative error code on
>   failure, let sbi_pmu_init() to hang by propagating the error
>   code.
>
> - In order to distinguish the SBI_EFAIL error returned by
>   sbi_pmu_add_*_counter_map(), return SBI_ENOENT to indicate
>   that fdt_pmu_setup() failed to locate "riscv,pmu" node, and
>   generic_pmu_init() ignores such case.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
> Changes v1 -> v2:
>   - New patch
> ---
>  lib/sbi/sbi_pmu.c           | 5 ++++-
>  lib/utils/fdt/fdt_pmu.c     | 2 +-
>  platform/generic/platform.c | 8 +++++++-
>  3 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index f4c8fc4..3cbd4ff 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -957,6 +957,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
>         int hpm_count = sbi_fls(sbi_hart_mhpm_mask(scratch));
>         struct sbi_pmu_hart_state *phs;
>         const struct sbi_platform *plat;
> +       int rc;
>
>         if (cold_boot) {
>                 hw_event_map = sbi_calloc(sizeof(*hw_event_map),
> @@ -972,7 +973,9 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
>
>                 plat = sbi_platform_ptr(scratch);
>                 /* Initialize hw pmu events */
> -               sbi_platform_pmu_init(plat);
> +               rc = sbi_platform_pmu_init(plat);
> +               if (rc)
> +                       return rc;
>
>                 /* mcycle & minstret is available always */
>                 if (!hpm_count)
> diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
> index 83301bb..a8d7648 100644
> --- a/lib/utils/fdt/fdt_pmu.c
> +++ b/lib/utils/fdt/fdt_pmu.c
> @@ -74,7 +74,7 @@ int fdt_pmu_setup(void *fdt)
>
>         pmu_offset = fdt_node_offset_by_compatible(fdt, -1, "riscv,pmu");
>         if (pmu_offset < 0)
> -               return SBI_EFAIL;
> +               return SBI_ENOENT;
>
>         event_ctr_map = fdt_getprop(fdt, pmu_offset,
>                                     "riscv,event-to-mhpmcounters", &len);
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 66a0b77..cb9270d 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -265,7 +265,13 @@ static u32 generic_tlb_num_entries(void)
>
>  static int generic_pmu_init(void)
>  {
> -       return fdt_pmu_setup(fdt_get_address());
> +       int rc;
> +
> +       rc = fdt_pmu_setup(fdt_get_address());
> +       if (rc && rc != SBI_ENOENT)
> +               return rc;
> +
> +       return 0;
>  }
>
>  static uint64_t generic_pmu_xlate_to_mhpmevent(uint32_t event_idx,
> --
> 2.34.1
>


From anup at brainfault.org  Wed Nov 15 22:36:53 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:06:53 +0530
Subject: [PATCH v2 02/11] sbi: sbi_pmu: Add hw_counter_filter_mode() to
 pmu device
In-Reply-To: <20231019113713.3508153-3-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-3-peterlin@andestech.com>
Message-ID: <CAAhSdy1CjqgChq1ZUkBiHG0zFCh=hsSh2Y2jbdspX2_TzygRNw@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:10?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add support for custom PMU extensions to set inhibit bits
> on custom CSRs by introducing the PMU device callback
> hw_counter_filter_mode(). This allows the perf tool to
> restrict event counting under a specified privileged
> mode by appending a modifier, e.g. perf record -e event:k
> to count events only happening in kernel mode.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - No change
> ---
>  include/sbi/sbi_pmu.h | 6 ++++++
>  lib/sbi/sbi_pmu.c     | 5 ++++-
>  2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
> index 16f6877..d63149c 100644
> --- a/include/sbi/sbi_pmu.h
> +++ b/include/sbi/sbi_pmu.h
> @@ -89,6 +89,12 @@ struct sbi_pmu_device {
>          * Custom function returning the machine-specific irq-bit.
>          */
>         int (*hw_counter_irq_bit)(void);
> +
> +       /**
> +        * Custom function to inhibit counting of events while in
> +        * specified mode.
> +        */
> +       void (*hw_counter_filter_mode)(unsigned long flags, int counter_index);
>  };
>
>  /** Get the PMU platform device */
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index 3cbd4ff..7e3723f 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -599,7 +599,10 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,
>                 pmu_dev->hw_counter_disable_irq(ctr_idx);
>
>         /* Update the inhibit flags based on inhibit flags received from supervisor */
> -       pmu_update_inhibit_flags(flags, &mhpmevent_val);
> +       if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
> +               pmu_update_inhibit_flags(flags, &mhpmevent_val);


pmu_update_inhibit_flags() is called from other places as well.

Looks like this patch needs to be rebased on the latest OpenSBI.

> +       if (pmu_dev && pmu_dev->hw_counter_filter_mode)
> +               pmu_dev->hw_counter_filter_mode(flags, ctr_idx);
>
>  #if __riscv_xlen == 32
>         csr_write_num(CSR_MHPMEVENT3 + ctr_idx - 3, mhpmevent_val & 0xFFFFFFFF);
> --
> 2.34.1
>

Regards,
Anup


From anup at brainfault.org  Wed Nov 15 22:39:53 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:09:53 +0530
Subject: [PATCH v2 03/11] platform: include: andes45: Add PMU related CSR defines
In-Reply-To: <20231019113713.3508153-4-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-4-peterlin@andestech.com>
Message-ID: <CAAhSdy0bW9KxQ-H9H0wgt6-m1L2uc2m1MK96JkN6jyLPw-=Sfw@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:10?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> This patch adds CSR for Andes PMU extension.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
> Changes v1 -> v2:
>   - Rename andes_pmu() -> has_andes_pmu()
> ---
>  platform/generic/include/andes/andes45.h | 26 ++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
>
> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index f570994..ce31617 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -12,6 +12,17 @@
>  #define CSR_MDCM_CFG           0xfc1
>  #define CSR_MMSC_CFG           0xfc2
>
> +/* Machine Trap Related Registers */
> +#define CSR_MSLIDELEG          0x7d5
> +
> +/* Counter Related Registers */
> +#define CSR_MCOUNTERWEN                0x7ce
> +#define CSR_MCOUNTERINTEN      0x7cf
> +#define CSR_MCOUNTERMASK_M     0x7d1
> +#define CSR_MCOUNTERMASK_S     0x7d2
> +#define CSR_MCOUNTERMASK_U     0x7d3
> +#define CSR_MCOUNTEROVF                0x7d4
> +
>  #define MICM_CFG_ISZ_OFFSET            6
>  #define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
>
> @@ -26,4 +37,19 @@
>  #define MCACHE_CTL_CCTL_SUEN_OFFSET    8
>  #define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
>
> +/* Performance monitor */
> +#define MMSC_CFG_PMNDS_MASK            (1 << 15)
> +#define MIP_PMOVI                      (1 << 18)
> +
> +#ifndef __ASSEMBLER__
> +
> +#define has_andes_pmu()                                        \
> +({                                                     \
> +       (((csr_read(CSR_MMSC_CFG) &                     \
> +          MMSC_CFG_PMNDS_MASK)                         \
> +         && misa_extension('S')) ? true : false);      \
> +})
> +
> +#endif /* __ASSEMBLER__ */
> +
>  #endif /* _RISCV_ANDES45_H */
> --
> 2.34.1
>


From anup at brainfault.org  Wed Nov 15 22:41:59 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:11:59 +0530
Subject: [PATCH v2 04/11] platform: andes: Add Andes custom PMU support
In-Reply-To: <20231019113713.3508153-5-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-5-peterlin@andestech.com>
Message-ID: <CAAhSdy37QCVw7brmkRC4yxqHfekVpnEm3GqCC4cTYDgfEatABg@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:10?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Before the ratification of Sscofpmf, the Andes PMU extension
> was designed to support the sampling and filtering with hardware
> performance counters, it works with the current SBI PMU extension
> and Linux SBI PMU driver.
>
> This patch implements the PMU device callbacks that update the
> corresponding bits on custom CSRs.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
> Changes v1 -> v2:
>   - Fix mode filtering in andes_hw_counter_filter_mode()
>   - Return early if pmu is not supported in andes_pmu_init() (suggested by Prabhakar)
>   - Don't grant write permissions via CSR_MCOUNTERWEN as not needed
> ---
>  platform/generic/andes/Kconfig             |  4 ++
>  platform/generic/andes/andes_pmu.c         | 81 ++++++++++++++++++++++
>  platform/generic/andes/objects.mk          |  1 +
>  platform/generic/include/andes/andes_pmu.h |  8 +++
>  4 files changed, 94 insertions(+)
>  create mode 100644 platform/generic/andes/andes_pmu.c
>  create mode 100644 platform/generic/include/andes/andes_pmu.h
>
> diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
> index a91fb9c..555e4fe 100644
> --- a/platform/generic/andes/Kconfig
> +++ b/platform/generic/andes/Kconfig
> @@ -7,3 +7,7 @@ config ANDES45_PMA
>  config ANDES_SBI
>         bool "Andes SBI support"
>         default n
> +
> +config ANDES_PMU
> +       bool "Andes custom PMU extension support"
> +       default n
> diff --git a/platform/generic/andes/andes_pmu.c b/platform/generic/andes/andes_pmu.c
> new file mode 100644
> index 0000000..0f6ecc0
> --- /dev/null
> +++ b/platform/generic/andes/andes_pmu.c
> @@ -0,0 +1,81 @@
> +// SPDX-License-Identifier: BSD-2-Clause
> +/*
> + * Copyright (C) 2023 Andes Technology Corporation
> + */
> +#include <andes/andes45.h>
> +#include <andes/andes_pmu.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/sbi_bitops.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_hart.h>
> +#include <sbi/sbi_pmu.h>
> +#include <sbi/sbi_scratch.h>
> +
> +static void andes_hw_counter_enable_irq(uint32_t ctr_idx)
> +{
> +       unsigned long mip_val;
> +
> +       if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
> +               return;
> +
> +       mip_val = csr_read(CSR_MIP);
> +       if (!(mip_val & MIP_PMOVI))
> +               csr_clear(CSR_MCOUNTEROVF, BIT(ctr_idx));
> +
> +       csr_set(CSR_MCOUNTERINTEN, BIT(ctr_idx));
> +}
> +
> +static void andes_hw_counter_disable_irq(uint32_t ctr_idx)
> +{
> +       csr_clear(CSR_MCOUNTERINTEN, BIT(ctr_idx));
> +}
> +
> +static void andes_hw_counter_filter_mode(unsigned long flags, int ctr_idx)
> +{
> +       if (flags & SBI_PMU_CFG_FLAG_SET_UINH)
> +               csr_set(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
> +       else
> +               csr_clear(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
> +
> +       if (flags & SBI_PMU_CFG_FLAG_SET_SINH)
> +               csr_set(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
> +       else
> +               csr_clear(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
> +}
> +
> +static struct sbi_pmu_device andes_pmu = {
> +       .name = "andes_pmu",
> +       .hw_counter_enable_irq  = andes_hw_counter_enable_irq,
> +       .hw_counter_disable_irq = andes_hw_counter_disable_irq,
> +       /*
> +        * We set delegation of supervisor local interrupts via
> +        * 18th bit on mslideleg instead of mideleg, so leave
> +        * hw_counter_irq_bit() callback unimplemented.
> +        */
> +       .hw_counter_irq_bit     = NULL,
> +       .hw_counter_filter_mode = andes_hw_counter_filter_mode
> +};
> +
> +int andes_pmu_init(void)
> +{
> +       if (!has_andes_pmu())
> +               return SBI_ENOTSUPP;
> +
> +       /*
> +        * It is not reasonable for an Andes CPU to support
> +        * both Andes PMU and standard Sscofpmf, as they
> +        * serve the same purpose.
> +        */
> +       if (sbi_hart_has_extension(sbi_scratch_thishart_ptr(),
> +                                  SBI_HART_EXT_SSCOFPMF))
> +               sbi_hart_hang();
> +
> +       /* Inhibit HPM counter in M-mode */
> +       csr_write(CSR_MCOUNTERMASK_M, 0xfffffffd);
> +       /* Delegate S-mode local interrupt to S-mode */
> +       csr_write(CSR_MSLIDELEG, MIP_PMOVI);
> +
> +       sbi_pmu_set_device(&andes_pmu);
> +
> +       return 0;
> +}
> diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
> index e8f86ea..6a8c66c 100644
> --- a/platform/generic/andes/objects.mk
> +++ b/platform/generic/andes/objects.mk
> @@ -7,3 +7,4 @@ platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
>
>  platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
>  platform-objs-$(CONFIG_ANDES_SBI) += andes/andes_sbi.o
> +platform-objs-$(CONFIG_ANDES_PMU) += andes/andes_pmu.o
> diff --git a/platform/generic/include/andes/andes_pmu.h b/platform/generic/include/andes/andes_pmu.h
> new file mode 100644
> index 0000000..70b3a12
> --- /dev/null
> +++ b/platform/generic/include/andes/andes_pmu.h
> @@ -0,0 +1,8 @@
> +// SPDX-License-Identifier: BSD-2-Clause
> +
> +#ifndef _RISCV_ANDES_PMU_H
> +#define _RISCV_ANDES_PMU_H
> +
> +int andes_pmu_init(void);
> +
> +#endif /* _RISCV_ANDES_PMU_H */
> --
> 2.34.1
>


From anup at brainfault.org  Wed Nov 15 22:45:20 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:15:20 +0530
Subject: [PATCH v2 05/11] platform: andes: Enable Andes PMU for AE350
In-Reply-To: <20231019113713.3508153-6-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-6-peterlin@andestech.com>
Message-ID: <CAAhSdy13q-i0xiEEqJxE3a7SguUHuW+Sp2SPiVPGLVC7CEM9rQ@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:10?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Enable Andes PMU extension support for AE350 platforms.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - Implement ae350_extensions_init()
> ---
>  platform/generic/Kconfig       |  1 +
>  platform/generic/andes/ae350.c | 13 +++++++++++++
>  2 files changed, 14 insertions(+)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index e7bd94e..9b2f9c7 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -31,6 +31,7 @@ config PLATFORM_ALLWINNER_D1
>  config PLATFORM_ANDES_AE350
>         bool "Andes AE350 support"
>         select SYS_ATCSMU
> +       select ANDES_PMU
>         default n
>
>  config PLATFORM_RENESAS_RZFIVE
> diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
> index 80eca05..c3f280d 100644
> --- a/platform/generic/andes/ae350.c
> +++ b/platform/generic/andes/ae350.c
> @@ -8,6 +8,7 @@
>   */
>
>  #include <platform_override.h>
> +#include <andes/andes_pmu.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>  #include <sbi_utils/fdt/fdt_fixup.h>
>  #include <sbi_utils/sys/atcsmu.h>
> @@ -110,6 +111,17 @@ static int ae350_final_init(bool cold_boot, const struct fdt_match *match)
>         return 0;
>  }
>
> +static int ae350_extensions_init(const struct fdt_match *match,
> +                         struct sbi_hart_features *hfeatures)
> +{
> +       int rc;
> +       rc = andes_pmu_init();
> +       if (rc && rc != SBI_ENOTSUPP)
> +               return rc;
> +
> +       return 0;
> +}
> +
>  static const struct fdt_match andes_ae350_match[] = {
>         { .compatible = "andestech,ae350" },
>         { },
> @@ -118,4 +130,5 @@ static const struct fdt_match andes_ae350_match[] = {
>  const struct platform_override andes_ae350 = {
>         .match_table = andes_ae350_match,
>         .final_init  = ae350_final_init,
> +       .extensions_init = ae350_extensions_init,

This is not the right way.

Add a new callback pmu_init() in "struct platform_override" and
implement that here.

The generic_pmu_init() will call pmu_init() in "struct platform_override"
if it is available otherwise it will call fdt_pmu_setup().

>  };
> --
> 2.34.1
>

Regards,
Anup


From anup at brainfault.org  Wed Nov 15 22:45:59 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:15:59 +0530
Subject: [PATCH v2 06/11] platform: rzfive: Enable Andes PMU for RZ/Five
In-Reply-To: <20231019113713.3508153-7-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-7-peterlin@andestech.com>
Message-ID: <CAAhSdy28SqsNwqHacff-LEgpU7bGM5mBTqHMfgK4aAGt+sK0Ng@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:10?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Enable Andes PMU extension support for RZ/Five.
> This patch also staticize renesas_rzfive_early_init() as
> it is not used outside of this unit.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - Implement renesas_rzfive_extensions_init()
>   - staticize renesas_rzfive_early_init()
> ---
>  platform/generic/Kconfig                 |  1 +
>  platform/generic/renesas/rzfive/rzfive.c | 16 +++++++++++++++-
>  2 files changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index 9b2f9c7..d6dafef 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -38,6 +38,7 @@ config PLATFORM_RENESAS_RZFIVE
>         bool "Renesas RZ/Five support"
>         select ANDES45_PMA
>         select ANDES_SBI
> +       select ANDES_PMU
>         default n
>
>  config PLATFORM_SIFIVE_FU540
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index a69797b..2f772c8 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -5,9 +5,11 @@
>   */
>
>  #include <andes/andes45_pma.h>
> +#include <andes/andes_pmu.h>
>  #include <andes/andes_sbi.h>
>  #include <platform_override.h>
>  #include <sbi/sbi_domain.h>
> +#include <sbi/sbi_error.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
>  static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
> @@ -29,7 +31,7 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
>                                          array_size(renesas_rzfive_pma_regions));
>  }
>
> -int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
> +static int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>  {
>         /*
>          * Renesas RZ/Five RISC-V SoC has Instruction local memory and
> @@ -47,6 +49,17 @@ int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>                                             SBI_DOMAIN_MEMREGION_M_RWX);
>  }
>
> +static int renesas_rzfive_extensions_init(const struct fdt_match *match,
> +                         struct sbi_hart_features *hfeatures)
> +{
> +       int rc;
> +       rc = andes_pmu_init();
> +       if (rc && rc != SBI_ENOTSUPP)
> +               return rc;
> +
> +       return 0;
> +}
> +
>  static const struct fdt_match renesas_rzfive_match[] = {
>         { .compatible = "renesas,r9a07g043f01" },
>         { /* sentinel */ }
> @@ -57,4 +70,5 @@ const struct platform_override renesas_rzfive = {
>         .early_init = renesas_rzfive_early_init,
>         .final_init = renesas_rzfive_final_init,
>         .vendor_ext_provider = andes_sbi_vendor_ext_provider,
> +       .extensions_init = renesas_rzfive_extensions_init,

Same comment as PATCH5.

>  };
> --
> 2.34.1
>

Regards,
Anup


From anup at brainfault.org  Wed Nov 15 22:48:15 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:18:15 +0530
Subject: [PATCH v2 08/11] lib: utils: fdt_fixup: Allow preserving PMU properties
In-Reply-To: <20231019113713.3508153-9-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-9-peterlin@andestech.com>
Message-ID: <CAAhSdy18NUx61ZGnawEin7Jm_YkfriDYn97iOPq3O-G4rzOy9Q@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:11?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add a scratch option to control PMU fixup, so the next
> stage software can dump the PMU node including event
> mapping information for debugging purposes.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> Changes v1 -> v2:
>   - New patch
> ---
>  include/sbi/sbi_scratch.h | 2 ++
>  lib/utils/fdt/fdt_fixup.c | 6 +++++-
>  2 files changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/include/sbi/sbi_scratch.h b/include/sbi/sbi_scratch.h
> index e6a33ba..7914dfb 100644
> --- a/include/sbi/sbi_scratch.h
> +++ b/include/sbi/sbi_scratch.h
> @@ -151,6 +151,8 @@ enum sbi_scratch_options {
>         SBI_SCRATCH_NO_BOOT_PRINTS = (1 << 0),
>         /** Enable runtime debug prints */
>         SBI_SCRATCH_DEBUG_PRINTS = (1 << 1),
> +       /** Preserve PMU node properties */
> +       SBI_SCRATCH_PRESERVE_PMU_PROP = (1 << 2),

s/SBI_SCRATCH_PRESERVE_PMU_PROP/SBI_SCRATCH_PRESERVE_PMU_NODE/

>  };
>
>  /** Get pointer to sbi_scratch for current HART */
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 67e2e2e..b0c885f 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -482,6 +482,8 @@ int fdt_reserved_memory_fixup(void *fdt)
>
>  void fdt_fixups(void *fdt)
>  {
> +       struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> +
>         fdt_aplic_fixup(fdt);
>
>         fdt_imsic_fixup(fdt);
> @@ -489,5 +491,7 @@ void fdt_fixups(void *fdt)
>         fdt_plic_fixup(fdt);
>
>         fdt_reserved_memory_fixup(fdt);
> -       fdt_pmu_fixup(fdt);
> +
> +       if (!(scratch->options & SBI_SCRATCH_PRESERVE_PMU_PROP))
> +               fdt_pmu_fixup(fdt);
>  }
> --
> 2.34.1
>

Otherwise, it looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup


From anup at brainfault.org  Wed Nov 15 22:49:31 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:19:31 +0530
Subject: [PATCH v2 09/11] platform: andes: Factor out is_andes() helper
In-Reply-To: <20231019113713.3508153-10-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-10-peterlin@andestech.com>
Message-ID: <CAAhSdy1z1He4pB+FzcC-XDUU=aT6XxvAgnccktYxYV+45gW2TQ@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:11?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> We will need is_andes(45) in the following patch,
> so factor out the code that parses marchid to make
> it reusable for checking any Andes CPU variants.
>
> Also improves the comment in ae350_hart_start().
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
> Changes v1 -> v2:
>   - New patch
> ---
>  platform/generic/andes/ae350.c           | 16 +++++++---------
>  platform/generic/include/andes/andes45.h |  6 ++++++
>  2 files changed, 13 insertions(+), 9 deletions(-)
>
> diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
> index c3f280d..80cd294 100644
> --- a/platform/generic/andes/ae350.c
> +++ b/platform/generic/andes/ae350.c
> @@ -24,16 +24,14 @@ static struct smu_data smu = { 0 };
>  extern void __ae350_enable_coherency_warmboot(void);
>  extern void __ae350_disable_coherency(void);
>
> -static __always_inline bool is_andes25(void)
> -{
> -       ulong marchid = csr_read(CSR_MARCHID);
> -       return !!(EXTRACT_FIELD(marchid, CSR_MARCHID_MICROID) == 0xa25);
> -}
> -
>  static int ae350_hart_start(u32 hartid, ulong saddr)
>  {
> -       /* Don't send wakeup command at boot-time */
> -       if (!sbi_init_count(hartid) || (is_andes25() && hartid == 0))
> +       /*
> +        * Don't send wakeup command
> +        * 1) at boot-time
> +        * 2) the target hart is non-sleepable 25-series hart0
> +        */
> +       if (!sbi_init_count(hartid) || (is_andes(25) && hartid == 0))
>                 return sbi_ipi_raw_send(sbi_hartid_to_hartindex(hartid));
>
>         /* Write wakeup command to the sleep hart */
> @@ -52,7 +50,7 @@ static int ae350_hart_stop(void)
>          * L2-cache, instead of turning it off, it should fall
>          * through and jump to warmboot_addr.
>          */
> -       if (is_andes25() && hartid == 0)
> +       if (is_andes(25) && hartid == 0)
>                 return SBI_ENOTSUPP;
>
>         if (!smu_support_sleep_mode(&smu, DEEPSLEEP_MODE, hartid))
> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index ce31617..01f63d4 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -43,6 +43,12 @@
>
>  #ifndef __ASSEMBLER__
>
> +#define is_andes(series)                               \
> +({                                                     \
> +       char value = csr_read(CSR_MARCHID) & 0xff;      \
> +       (series) == (value >> 4) * 10 + (value & 0x0f); \
> +})
> +
>  #define has_andes_pmu()                                        \
>  ({                                                     \
>         (((csr_read(CSR_MMSC_CFG) &                     \
> --
> 2.34.1
>


From anup at brainfault.org  Wed Nov 15 22:58:30 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:28:30 +0530
Subject: [PATCH v2 10/11] platform: andes: Implement andes_fdt_add_pmu_mappings
 platform override
In-Reply-To: <20231019113713.3508153-11-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-11-peterlin@andestech.com>
Message-ID: <CAAhSdy3gVJj9n6+Suiq2=GBB1TO-kOFdYb3G+O1tM7UDdSuhfg@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:11?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Implement the andes_fdt_add_pmu_mappings() callback, which creates PMU
> node properties that will later be populated with mapping tables.
>
> Currently we only support 45-series event mappings.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> Changes v1 -> v2:
>   - New patch
> ---
>  include/sbi/sbi_ecall_interface.h          |   5 +
>  platform/generic/Kconfig                   |   2 +
>  platform/generic/andes/Kconfig             |   7 +
>  platform/generic/andes/ae350.c             |   2 +
>  platform/generic/andes/andes_hpm.c         | 381 +++++++++++++++++++++
>  platform/generic/andes/objects.mk          |   1 +
>  platform/generic/include/andes/andes_hpm.h |  83 +++++
>  platform/generic/renesas/rzfive/rzfive.c   |   2 +
>  8 files changed, 483 insertions(+)
>  create mode 100644 platform/generic/andes/andes_hpm.c
>  create mode 100644 platform/generic/include/andes/andes_hpm.h
>
> diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
> index 1fe469e..89187e7 100644
> --- a/include/sbi/sbi_ecall_interface.h
> +++ b/include/sbi/sbi_ecall_interface.h
> @@ -155,6 +155,11 @@ enum sbi_pmu_hw_cache_op_result_id {
>         SBI_PMU_HW_CACHE_RESULT_MAX,
>  };
>
> +#define SBI_PMU_HW_CACHE_EVENT_IDX(id, op, res) \
> +       (SBI_PMU_EVENT_TYPE_HW_CACHE << SBI_PMU_EVENT_IDX_TYPE_OFFSET | \
> +       SBI_PMU_HW_CACHE_##id << 3 | SBI_PMU_HW_CACHE_OP_##op << 1 | \
> +       SBI_PMU_HW_CACHE_RESULT_##res)
> +
>  /**
>   * Special "firmware" events provided by the OpenSBI, even if the hardware
>   * does not support performance events. These events are encoded as a raw
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index d6dafef..fbcd870 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -31,6 +31,7 @@ config PLATFORM_ALLWINNER_D1
>  config PLATFORM_ANDES_AE350
>         bool "Andes AE350 support"
>         select SYS_ATCSMU
> +       select ANDES_HPM
>         select ANDES_PMU
>         default n
>
> @@ -38,6 +39,7 @@ config PLATFORM_RENESAS_RZFIVE
>         bool "Renesas RZ/Five support"
>         select ANDES45_PMA
>         select ANDES_SBI
> +       select ANDES_HPM
>         select ANDES_PMU
>         default n
>
> diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
> index 555e4fe..8af2704 100644
> --- a/platform/generic/andes/Kconfig
> +++ b/platform/generic/andes/Kconfig
> @@ -8,6 +8,13 @@ config ANDES_SBI
>         bool "Andes SBI support"
>         default n
>
> +config ANDES_HPM
> +       bool "Andes HPM support"
> +       default n
> +       help
> +         This provides Andes platform override for creating
> +         event to counter mappings in pmu node.
> +
>  config ANDES_PMU
>         bool "Andes custom PMU extension support"
>         default n
> diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
> index 80cd294..28c187e 100644
> --- a/platform/generic/andes/ae350.c
> +++ b/platform/generic/andes/ae350.c
> @@ -19,6 +19,7 @@
>  #include <sbi/sbi_ipi.h>
>  #include <sbi/sbi_init.h>
>  #include <andes/andes45.h>
> +#include <andes/andes_hpm.h>
>
>  static struct smu_data smu = { 0 };
>  extern void __ae350_enable_coherency_warmboot(void);
> @@ -128,5 +129,6 @@ static const struct fdt_match andes_ae350_match[] = {
>  const struct platform_override andes_ae350 = {
>         .match_table = andes_ae350_match,
>         .final_init  = ae350_final_init,
> +       .fdt_add_pmu_mappings = andes_fdt_add_pmu_mappings,
>         .extensions_init = ae350_extensions_init,
>  };
> diff --git a/platform/generic/andes/andes_hpm.c b/platform/generic/andes/andes_hpm.c
> new file mode 100644
> index 0000000..f9e6d2e
> --- /dev/null
> +++ b/platform/generic/andes/andes_hpm.c
> @@ -0,0 +1,381 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2023 Andes Technology Corporation
> + */
> +
> +#include <andes/andes45.h>
> +#include <andes/andes_hpm.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi_utils/fdt/fdt_fixup.h>
> +
> +static const struct sbi_pmu_event_select_map andes45_hw_evt_selects[] = {
> +       /* Hardware general events (Type #0) */
> +       {
> +               /* perf: cycles (eidx: 0x1) */
> +               .eidx = SBI_PMU_HW_CPU_CYCLES,
> +               .select = ANDES_CYCLES
> +       },
> +       {
> +               /* perf: instructions (eidx: 0x2) */
> +               .eidx = SBI_PMU_HW_INSTRUCTIONS,
> +               .select = ANDES_INSTRET
> +       },
> +       {
> +               /* perf: cache-references (eidx: 0x3) */
> +               .eidx = SBI_PMU_HW_CACHE_REFERENCES,
> +               .select = ANDES_DCACHE_ACCESS
> +       },
> +       {
> +               /* perf: cache-misses (eidx: 0x4) */
> +               .eidx = SBI_PMU_HW_CACHE_MISSES,
> +               .select = ANDES_DCACHE_MISS
> +       },
> +       {
> +               /* perf: branches (eidx: 0x5) */
> +               .eidx = SBI_PMU_HW_BRANCH_INSTRUCTIONS,
> +               .select = ANDES_CONDITION_BR,
> +       },
> +       {
> +               /* perf: branch-misses (eidx: 0x6) */
> +               .eidx = SBI_PMU_HW_BRANCH_MISSES,
> +               .select = ANDES_MISPREDICT_CONDITION_BR,
> +       },
> +       /* Hardware cache events (Type #1) */
> +       {
> +               /* perf: L1-dcache-loads (eidx: 0x10000) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, ACCESS),
> +               .select = ANDES_DCACHE_LOAD_ACCESS
> +       },
> +       {
> +               /* perf: L1-dcache-loads-misses (eidx: 0x10001) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, MISS),
> +               .select = ANDES_DCACHE_LOAD_MISS
> +       },
> +       {
> +               /* perf: L1-dcache-stores (eidx: 0x10002) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, ACCESS),
> +               .select = ANDES_DCACHE_STORE_ACCESS
> +       },
> +       {
> +               /* perf: L1-dcache-store-misses (eidx: 0x10003) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, MISS),
> +               .select = ANDES_DCACHE_STORE_MISS
> +       },
> +       {
> +               /* perf: L1-icache-load (eidx: 0x10008) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, ACCESS),
> +               .select = ANDES_ICACHE_ACCESS
> +       },
> +       {
> +               /* perf: L1-icache-load-misses (eidx: 0x10009) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, MISS),
> +               .select = ANDES_ICACHE_MISS
> +       },
> +       { /* sentinel */ }
> +};
> +
> +static const struct sbi_pmu_event_counter_map andes45_hw_evt_counters[] = {
> +       {
> +               /* perf: cycles (eidx: 0x1) */
> +               .eidx_start = SBI_PMU_HW_CPU_CYCLES,
> +               /* perf: branch-misses (eidx: 0x6) */
> +               .eidx_end = SBI_PMU_HW_BRANCH_MISSES,
> +               .ctr_map = ANDES_MHPM_MAP,
> +       },
> +       {
> +               /* perf: L1-dcache-loads (eidx: 0x10000) */
> +               .eidx_start = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, ACCESS),
> +               /* perf: L1-dcache-store-misses (eidx: 0x10003) */
> +               .eidx_end = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, MISS),
> +               .ctr_map = ANDES_MHPM_MAP,
> +       },
> +       {
> +               /* perf: L1-icache-load (eidx: 0x10008) */
> +               .eidx_start = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, ACCESS),
> +               /* perf: L1-icache-load-misses (eidx: 0x10009) */
> +               .eidx_end = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, MISS),
> +               .ctr_map = ANDES_MHPM_MAP,
> +       },
> +       { /* sentinel */ }
> +};
> +
> +static const struct sbi_pmu_raw_event_counter_map andes45_raw_evt_counters[] = {
> +       {
> +               .select = ANDES_CYCLES,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INSTRET,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_LOAD_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_STORE_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_ATOMIC_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_SYS_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_COMPUTE_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CONDITION_BR,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_TAKEN_CONDITION_BR,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_JAL_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_JALR_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_RET_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CONTROL_TRANS_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_EX9_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_MUL_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_DIV_REMAINDER_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_LOAD_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_STORE_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_ADD_SUB_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_MUL_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_FUSED_MULADD_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_DIV_SQUARE_ROOT_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_OTHER_FLOAT_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_MUL_AND_SUB_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_RETIRED_OP,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_ILM_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DLM_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_ICACHE_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_ICACHE_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_LOAD_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_LOAD_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_STORE_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_STORE_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_WB,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_ICACHE_FILL,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_DCACHE_FILL,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_UNCACHED_IFETCH_FROM_BUS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_UNCACHED_LOAD_FROM_BUS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_UNCACHED_IFETCH,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_UNCACHED_LOAD,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MAIN_ITLB_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MAIN_ITLB_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MAIN_DTLB_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MAIN_DTLB_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_ITLB_FILL,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_PIPE_STALL_CYCLE_DTLB_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_HW_PREFETCH_BUS_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MISPREDICT_CONDITION_BR,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MISPREDICT_TAKE_CONDITION_BR,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MISPREDICT_TARGET_RET_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       { /* sentinel */ }
> +};
> +
> +int andes_fdt_add_pmu_mappings(void *fdt, const struct fdt_match *match)
> +{
> +       /*
> +        * At the moment, simply create mapping for any 45-series core
> +        * based on marchid, we may check and differentiate the mapping
> +        * by mimpid.
> +        */
> +       if (is_andes(45))
> +               return fdt_add_pmu_mappings(fdt, andes45_hw_evt_selects,
> +                                           andes45_hw_evt_counters,
> +                                           andes45_raw_evt_counters);

This approach is an overkill because you are creating an FDT node at
boot-time which is later parsed by fdt_pmu_setup().

Instead, I would suggest to:
1) Drop PATCH7
2) Directly populate HW counters and event mappings over here
    in the same way as fdt_pmu_setup() for your andes 45-series

> +
> +       return 0;
> +}
> diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
> index 6a8c66c..57caaf6 100644
> --- a/platform/generic/andes/objects.mk
> +++ b/platform/generic/andes/objects.mk
> @@ -8,3 +8,4 @@ platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
>  platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
>  platform-objs-$(CONFIG_ANDES_SBI) += andes/andes_sbi.o
>  platform-objs-$(CONFIG_ANDES_PMU) += andes/andes_pmu.o
> +platform-objs-$(CONFIG_ANDES_HPM) += andes/andes_hpm.o
> diff --git a/platform/generic/include/andes/andes_hpm.h b/platform/generic/include/andes/andes_hpm.h
> new file mode 100644
> index 0000000..0f301db
> --- /dev/null
> +++ b/platform/generic/include/andes/andes_hpm.h
> @@ -0,0 +1,83 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2023 Andes Technology Corporation
> + */
> +
> +#ifndef _ANDES_HPM_H_
> +#define _ANDES_HPM_H_
> +
> +#include <sbi_utils/fdt/fdt_helper.h>
> +
> +#define ANDES_MHPM_MAP         0x78
> +#define ANDES_RAW_EVENT_MASK   ~0ULL
> +
> +/* Event code for instruction commit events */
> +#define ANDES_CYCLES                           0x10
> +#define ANDES_INSTRET                          0x20
> +#define ANDES_INT_LOAD_INST                    0x30
> +#define ANDES_INT_STORE_INST                   0x40
> +#define ANDES_ATOMIC_INST                      0x50
> +#define ANDES_SYS_INST                         0x60
> +#define ANDES_INT_COMPUTE_INST                 0x70
> +#define ANDES_CONDITION_BR                     0x80
> +#define ANDES_TAKEN_CONDITION_BR               0x90
> +#define ANDES_JAL_INST                         0xA0
> +#define ANDES_JALR_INST                                0xB0
> +#define ANDES_RET_INST                         0xC0
> +#define ANDES_CONTROL_TRANS_INST               0xD0
> +#define ANDES_EX9_INST                         0xE0
> +#define ANDES_INT_MUL_INST                     0xF0
> +#define ANDES_INT_DIV_REMAINDER_INST           0x100
> +#define ANDES_FLOAT_LOAD_INST                  0x110
> +#define ANDES_FLOAT_STORE_INST                 0x120
> +#define ANDES_FLOAT_ADD_SUB_INST               0x130
> +#define ANDES_FLOAT_MUL_INST                   0x140
> +#define ANDES_FLOAT_FUSED_MULADD_INST          0x150
> +#define ANDES_FLOAT_DIV_SQUARE_ROOT_INST       0x160
> +#define ANDES_OTHER_FLOAT_INST                 0x170
> +#define ANDES_INT_MUL_AND_SUB_INST             0x180
> +#define ANDES_RETIRED_OP                       0x190
> +
> +/* Event code for memory system events */
> +#define ANDES_ILM_ACCESS                       0x01
> +#define ANDES_DLM_ACCESS                       0x11
> +#define ANDES_ICACHE_ACCESS                    0x21
> +#define ANDES_ICACHE_MISS                      0x31
> +#define ANDES_DCACHE_ACCESS                    0x41
> +#define ANDES_DCACHE_MISS                      0x51
> +#define ANDES_DCACHE_LOAD_ACCESS               0x61
> +#define ANDES_DCACHE_LOAD_MISS                 0x71
> +#define ANDES_DCACHE_STORE_ACCESS              0x81
> +#define ANDES_DCACHE_STORE_MISS                        0x91
> +#define ANDES_DCACHE_WB                                0xA1
> +#define ANDES_CYCLE_WAIT_ICACHE_FILL           0xB1
> +#define ANDES_CYCLE_WAIT_DCACHE_FILL           0xC1
> +#define ANDES_UNCACHED_IFETCH_FROM_BUS         0xD1
> +#define ANDES_UNCACHED_LOAD_FROM_BUS           0xE1
> +#define ANDES_CYCLE_WAIT_UNCACHED_IFETCH       0xF1
> +#define ANDES_CYCLE_WAIT_UNCACHED_LOAD         0x101
> +#define ANDES_MAIN_ITLB_ACCESS                 0x111
> +#define ANDES_MAIN_ITLB_MISS                   0x121
> +#define ANDES_MAIN_DTLB_ACCESS                 0x131
> +#define ANDES_MAIN_DTLB_MISS                   0x141
> +#define ANDES_CYCLE_WAIT_ITLB_FILL             0x151
> +#define ANDES_PIPE_STALL_CYCLE_DTLB_MISS       0x161
> +#define ANDES_HW_PREFETCH_BUS_ACCESS           0x171
> +
> +/* Event code for microarchitecture events */
> +#define ANDES_MISPREDICT_CONDITION_BR          0x02
> +#define ANDES_MISPREDICT_TAKE_CONDITION_BR     0x12
> +#define ANDES_MISPREDICT_TARGET_RET_INST       0x22
> +
> +#ifdef CONFIG_ANDES_HPM
> +
> +int andes_fdt_add_pmu_mappings(void *fdt, const struct fdt_match *match);
> +
> +#else
> +
> +int andes_fdt_add_pmu_mappings(void *fdt, const struct fdt_match *match) { return 0; }
> +
> +#endif /* CONFIG_ANDES_HPM */
> +
> +#endif /* _ANDES_HPM_H_ */
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index 2f772c8..fafbff0 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -5,6 +5,7 @@
>   */
>
>  #include <andes/andes45_pma.h>
> +#include <andes/andes_hpm.h>
>  #include <andes/andes_pmu.h>
>  #include <andes/andes_sbi.h>
>  #include <platform_override.h>
> @@ -69,6 +70,7 @@ const struct platform_override renesas_rzfive = {
>         .match_table = renesas_rzfive_match,
>         .early_init = renesas_rzfive_early_init,
>         .final_init = renesas_rzfive_final_init,
> +       .fdt_add_pmu_mappings = andes_fdt_add_pmu_mappings,
>         .vendor_ext_provider = andes_sbi_vendor_ext_provider,
>         .extensions_init = renesas_rzfive_extensions_init,
>  };
> --
> 2.34.1
>

Regards,
Anup


From anup at brainfault.org  Wed Nov 15 22:59:16 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:29:16 +0530
Subject: [PATCH v2 07/11] lib: utils: fdt_fixup: Add fdt_add_pmu_mappings()
 helper function
In-Reply-To: <20231019113713.3508153-8-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-8-peterlin@andestech.com>
Message-ID: <CAAhSdy02NNmjqVVr3420=MF5Tbd_DjQpr_kYNJn5N8Z6OvZH6w@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:11?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add fdt_add_pmu_mappings() that creates entries of riscv,*event-to-mhpm*
> property from arrays right before fdt_pmu_setup() populating the mapping
> tables (i.e. hw_event_map[] and fdt_pmu_evt_select[]).
>
> The helper function will skip the creation of those properties if a
> "/pmu" node with "riscv,pmu" compatible string is provided.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>

This patch can be dropped. See my comment in PATCH10.

Regards,
Anup

> ---
> Changes v1 -> v2:
>   - New patch
> ---
>  include/sbi_utils/fdt/fdt_fixup.h            | 48 ++++++++++
>  lib/utils/fdt/fdt_fixup.c                    | 95 ++++++++++++++++++++
>  platform/generic/include/platform_override.h |  1 +
>  platform/generic/platform.c                  | 10 ++-
>  4 files changed, 153 insertions(+), 1 deletion(-)
>
> diff --git a/include/sbi_utils/fdt/fdt_fixup.h b/include/sbi_utils/fdt/fdt_fixup.h
> index ecd55a7..9b72df8 100644
> --- a/include/sbi_utils/fdt/fdt_fixup.h
> +++ b/include/sbi_utils/fdt/fdt_fixup.h
> @@ -19,6 +19,54 @@ struct sbi_cpu_idle_state {
>         uint32_t wakeup_latency_us;
>  };
>
> +struct sbi_pmu_event_select_map {
> +       /**
> +        * The description of an entry in
> +        * riscv,event-to-mhpmevent property
> +        */
> +       uint32_t eidx;
> +       uint64_t select;
> +};
> +
> +struct sbi_pmu_event_counter_map {
> +       /**
> +        * The description of an entry in
> +        * riscv,event-to-mhpmcounters property
> +        */
> +       uint32_t eidx_start;
> +       uint32_t eidx_end;
> +       uint32_t ctr_map;
> +};
> +
> +struct sbi_pmu_raw_event_counter_map {
> +       /**
> +        * The description of an entry in
> +        * riscv,raw-event-to-mhpmcounters property
> +        */
> +       uint64_t select;
> +       uint64_t select_mask;
> +       uint32_t ctr_map;
> +};
> +
> +/**
> + * Add PMU properties in the DT
> + *
> + * Add information about event to selector/counter mappings to the
> + * devicetree.
> + *
> + * @param fdt: device tree blob
> + * @param selects: array of event index to selector value mapping
> + *                 descriptions, ending with empty element
> + * @param counters: array of event indexes to counters mapping
> + *                  descriptions, ending with empty element
> + * @param rcounters: array of raw events to counters mapping
> + *                   descriptions, ending with empty element
> + * @return zero on success and -ve on failure
> + */
> +int fdt_add_pmu_mappings(void *fdt, const struct sbi_pmu_event_select_map *selects,
> +               const struct sbi_pmu_event_counter_map *counters,
> +               const struct sbi_pmu_raw_event_counter_map *rcounters);
> +
>  /**
>   * Add CPU idle states to cpu nodes in the DT
>   *
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index e213ded..67e2e2e 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -20,6 +20,101 @@
>  #include <sbi_utils/fdt/fdt_pmu.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
> +int fdt_add_pmu_mappings(void *fdt, const struct sbi_pmu_event_select_map *selects,
> +               const struct sbi_pmu_event_counter_map *counters,
> +               const struct sbi_pmu_raw_event_counter_map *rcounters)
> +{
> +       int i, err, pmu_noff, root_noff;
> +       const char *comp;
> +       fdt32_t evt_to_mhpmevent[3];
> +       fdt32_t evt_to_mhpmcounters[3];
> +       fdt32_t raw_evt_to_mhpmcounters[5];
> +
> +       /* Try to locate pmu node */
> +       pmu_noff = fdt_path_offset(fdt, "/pmu");
> +
> +       if (pmu_noff > 0) {
> +               /*
> +                * If compatible string is "riscv,pmu",
> +                * we assume a valid pmu node has been
> +                * provided.
> +                */
> +               comp = fdt_getprop(fdt, pmu_noff, "compatible", NULL);
> +               if (comp && !strcmp(comp, "riscv,pmu"))
> +                       return 0;
> +               else
> +                       return -FDT_ERR_BADVALUE;
> +       }
> +
> +       if (pmu_noff < 0 && pmu_noff != -FDT_ERR_NOTFOUND)
> +               return pmu_noff;
> +
> +       /*
> +        * If "riscv,event-to-mhpmevent" is present, "riscv,event-to-mhpmcounters"
> +        * must be provided as well, but not vice versa (OpenSBI will direct mapping
> +        * event_idx as selector value).
> +        */
> +       if (selects && !counters) {
> +               sbi_printf("%s: ERR: riscv,event-to-mhpmcounters is mandatory if"
> +                       " riscv,event-to-mhpmevent is present.", __func__);
> +               return SBI_EINVAL;
> +       }
> +
> +       /*
> +        * Create pmu node based on given @selects, @counters
> +        * and @rcounters.
> +        */
> +       root_noff = fdt_path_offset(fdt, "/");
> +       pmu_noff = fdt_add_subnode(fdt, root_noff, "pmu");
> +       if  (pmu_noff < 0)
> +               return pmu_noff;
> +
> +       err = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + 1024);
> +       if (err < 0)
> +               return err;
> +
> +       err = fdt_setprop_string(fdt, pmu_noff, "compatible", "riscv,pmu");
> +       if (err)
> +               return err;
> +
> +       /* Add riscv,event-to-mhpmevent */
> +       for (i = 0; selects && selects[i].eidx; i++) {
> +               evt_to_mhpmevent[0] = cpu_to_fdt32(selects[i].eidx);
> +               evt_to_mhpmevent[1] = cpu_to_fdt32(selects[i].select >> 32);
> +               evt_to_mhpmevent[2] = cpu_to_fdt32(selects[i].select & ~0UL);
> +               err = fdt_appendprop(fdt, pmu_noff, "riscv,event-to-mhpmevent",
> +                               evt_to_mhpmevent, 3 * sizeof(fdt32_t));
> +               if (err)
> +                       return err;
> +       }
> +
> +       /* Add riscv,event-to-mhpmcounters */
> +       for (i = 0; counters && counters[i].eidx_start; i++) {
> +               evt_to_mhpmcounters[0] = cpu_to_fdt32(counters[i].eidx_start);
> +               evt_to_mhpmcounters[1] = cpu_to_fdt32(counters[i].eidx_end);
> +               evt_to_mhpmcounters[2] = cpu_to_fdt32(counters[i].ctr_map);
> +               err = fdt_appendprop(fdt, pmu_noff, "riscv,event-to-mhpmcounters",
> +                               evt_to_mhpmcounters, 3 * sizeof(fdt32_t));
> +               if (err)
> +                       return err;
> +       }
> +
> +       /* Add riscv,raw-event-to-mhpmcounters */
> +       for (i = 0; rcounters && rcounters[i].select; i++) {
> +               raw_evt_to_mhpmcounters[0] = cpu_to_fdt32(rcounters[i].select >> 32);
> +               raw_evt_to_mhpmcounters[1] = cpu_to_fdt32(rcounters[i].select & ~0UL);
> +               raw_evt_to_mhpmcounters[2] = cpu_to_fdt32(rcounters[i].select_mask >> 32);
> +               raw_evt_to_mhpmcounters[3] = cpu_to_fdt32(rcounters[i].select_mask & ~0UL);
> +               raw_evt_to_mhpmcounters[4] = cpu_to_fdt32(rcounters[i].ctr_map);
> +               err = fdt_appendprop(fdt, pmu_noff, "riscv,raw-event-to-mhpmcounters",
> +                               raw_evt_to_mhpmcounters, 5 * sizeof(fdt32_t));
> +               if (err)
> +                       return err;
> +       }
> +
> +       return 0;
> +}
> +
>  int fdt_add_cpu_idle_states(void *fdt, const struct sbi_cpu_idle_state *state)
>  {
>         int cpu_node, cpus_node, err, idle_states_node;
> diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
> index bf4b112..bd34d2a 100644
> --- a/platform/generic/include/platform_override.h
> +++ b/platform/generic/include/platform_override.h
> @@ -19,6 +19,7 @@ struct platform_override {
>         u64 (*features)(const struct fdt_match *match);
>         u64 (*tlbr_flush_limit)(const struct fdt_match *match);
>         u32 (*tlb_num_entries)(const struct fdt_match *match);
> +       int (*fdt_add_pmu_mappings)(void *fdt, const struct fdt_match *match);
>         bool (*cold_boot_allowed)(u32 hartid, const struct fdt_match *match);
>         int (*early_init)(bool cold_boot, const struct fdt_match *match);
>         int (*final_init)(bool cold_boot, const struct fdt_match *match);
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index cb9270d..a48a8b7 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -265,9 +265,17 @@ static u32 generic_tlb_num_entries(void)
>
>  static int generic_pmu_init(void)
>  {
> +       void *fdt = fdt_get_address();
>         int rc;
>
> -       rc = fdt_pmu_setup(fdt_get_address());
> +       if (generic_plat && generic_plat->fdt_add_pmu_mappings) {
> +               rc = generic_plat->fdt_add_pmu_mappings(fdt,
> +                                                       generic_plat_match);
> +               if (rc)
> +                       return rc;
> +       }
> +
> +       rc = fdt_pmu_setup(fdt);
>         if (rc && rc != SBI_ENOENT)
>                 return rc;
>
> --
> 2.34.1
>


From anup at brainfault.org  Wed Nov 15 23:00:06 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:30:06 +0530
Subject: [PATCH v2 11/11] docs: pmu: Add Andes PMU node example
In-Reply-To: <20231019113713.3508153-12-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com> <20231019113713.3508153-12-peterlin@andestech.com>
Message-ID: <CAAhSdy2fsQVjigCocb2LWZNucwXkigZqnMd6yrB0g3Mqu1=CRA@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:11?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add PMU node example for event index to counter index mapping
> and selector value translation of Andes' CPUs.
>
> Currently, there are 4 HPM counters that can be used to monitor
> all of the events for each hart.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Locus Wei-Han Chen <locus84 at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
> Changes v1 -> v2:
>   - sync up with datasheet
> ---
>  docs/pmu_support.md | 82 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 82 insertions(+)
>
> diff --git a/docs/pmu_support.md b/docs/pmu_support.md
> index 8cfa08c..9b48f1e 100644
> --- a/docs/pmu_support.md
> +++ b/docs/pmu_support.md
> @@ -125,3 +125,85 @@ pmu {
>                                           <0x0 0x2 0xffffffff 0xffffe0ff 0x18>;
>  };
>  ```
> +
> +### Example 3
> +
> +```
> +/*
> + * For Andes 45-series platforms. The encodings can be found in the
> + * "Machine Performance Monitoring Event Selector" section
> + * http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
> + */
> +pmu {
> +       compatible                      = "riscv,pmu";
> +       riscv,event-to-mhpmevent =
> +                                        <0x1 0x0000 0x10>, /* CPU_CYCLES          -> Cycle count */
> +                                        <0x2 0x0000 0x20>, /* INSTRUCTIONS        -> Retired instruction count */
> +                                        <0x3 0x0000 0x41>, /* CACHE_REFERENCES    -> D-Cache access */
> +                                        <0x4 0x0000 0x51>, /* CACHE_MISSES        -> D-Cache miss */
> +                                        <0x5 0x0000 0x80>, /* BRANCH_INSTRUCTIONS -> Conditional branch instruction count */
> +                                        <0x6 0x0000 0x02>, /* BRANCH_MISSES       -> Misprediction of conditional branches */
> +                                        <0x10000 0x0000 0x61>,  /* L1D_READ_ACCESS  -> D-Cache load access */
> +                                        <0x10001 0x0000 0x71>,  /* L1D_READ_MISS    -> D-Cache load miss */
> +                                        <0x10002 0x0000 0x81>,  /* L1D_WRITE_ACCESS -> D-Cache store access */
> +                                        <0x10003 0x0000 0x91>,  /* L1D_WRITE_MISS   -> D-Cache store miss */
> +                                        <0x10008 0x0000 0x21>,  /* L1I_READ_ACCESS  -> I-Cache access */
> +                                        <0x10009 0x0000 0x31>;  /* L1I_READ_MISS    -> I-Cache miss */
> +       riscv,event-to-mhpmcounters = <0x1 0x6 0x78>,
> +                                                       <0x10000 0x10003 0x78>,
> +                                                       <0x10008 0x10009 0x78>;
> +       riscv,raw-event-to-mhpmcounters =
> +                                               <0x0 0x10 0xffffffff 0xffffffff 0x78>, /* Cycle count */
> +                                               <0x0 0x20 0xffffffff 0xffffffff 0x78>, /* Retired instruction count */
> +                                               <0x0 0x30 0xffffffff 0xffffffff 0x78>, /* Integer load instruction count */
> +                                               <0x0 0x40 0xffffffff 0xffffffff 0x78>, /* Integer store instruction count */
> +                                               <0x0 0x50 0xffffffff 0xffffffff 0x78>, /* Atomic instruction count */
> +                                               <0x0 0x60 0xffffffff 0xffffffff 0x78>, /* System instruction count */
> +                                               <0x0 0x70 0xffffffff 0xffffffff 0x78>, /* Integer computational instruction count */
> +                                               <0x0 0x80 0xffffffff 0xffffffff 0x78>, /* Conditional branch instruction count */
> +                                               <0x0 0x90 0xffffffff 0xffffffff 0x78>, /* Taken conditional branch instruction count */
> +                                               <0x0 0xA0 0xffffffff 0xffffffff 0x78>, /* JAL instruction count */
> +                                               <0x0 0xB0 0xffffffff 0xffffffff 0x78>, /* JALR instruction count */
> +                                               <0x0 0xC0 0xffffffff 0xffffffff 0x78>, /* Return instruction count */
> +                                               <0x0 0xD0 0xffffffff 0xffffffff 0x78>, /* Control transfer instruction count */
> +                                               <0x0 0xE0 0xffffffff 0xffffffff 0x78>, /* EXEC.IT instruction count */
> +                                               <0x0 0xF0 0xffffffff 0xffffffff 0x78>, /* Integer multiplication instruction count */
> +                                               <0x0 0x100 0xffffffff 0xffffffff 0x78>, /* Integer division instruction count */
> +                                               <0x0 0x110 0xffffffff 0xffffffff 0x78>, /* Floating-point load instruction count */
> +                                               <0x0 0x120 0xffffffff 0xffffffff 0x78>, /* Floating-point store instruction count */
> +                                               <0x0 0x130 0xffffffff 0xffffffff 0x78>, /* Floating-point addition/subtraction instruction count */
> +                                               <0x0 0x140 0xffffffff 0xffffffff 0x78>, /* Floating-point multiplication instruction count */
> +                                               <0x0 0x150 0xffffffff 0xffffffff 0x78>, /* Floating-point fused multiply-add instruction count */
> +                                               <0x0 0x160 0xffffffff 0xffffffff 0x78>, /* Floating-point division or square-root instruction count */
> +                                               <0x0 0x170 0xffffffff 0xffffffff 0x78>, /* Other floating-point instruction count */
> +                                               <0x0 0x180 0xffffffff 0xffffffff 0x78>, /* Integer multiplication and add/sub instruction count */
> +                                               <0x0 0x190 0xffffffff 0xffffffff 0x78>, /* Retired operation count */
> +                                               <0x0 0x01 0xffffffff 0xffffffff 0x78>, /* ILM access */
> +                                               <0x0 0x11 0xffffffff 0xffffffff 0x78>, /* DLM access */
> +                                               <0x0 0x21 0xffffffff 0xffffffff 0x78>, /* I-Cache access */
> +                                               <0x0 0x31 0xffffffff 0xffffffff 0x78>, /* I-Cache miss */
> +                                               <0x0 0x41 0xffffffff 0xffffffff 0x78>, /* D-Cache access */
> +                                               <0x0 0x51 0xffffffff 0xffffffff 0x78>, /* D-Cache miss */
> +                                               <0x0 0x61 0xffffffff 0xffffffff 0x78>, /* D-Cache load access */
> +                                               <0x0 0x71 0xffffffff 0xffffffff 0x78>, /* D-Cache load miss */
> +                                               <0x0 0x81 0xffffffff 0xffffffff 0x78>, /* D-Cache store access */
> +                                               <0x0 0x91 0xffffffff 0xffffffff 0x78>, /* D-Cache store miss */
> +                                               <0x0 0xA1 0xffffffff 0xffffffff 0x78>, /* D-Cache writeback */
> +                                               <0x0 0xB1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for I-Cache fill data */
> +                                               <0x0 0xC1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for D-Cache fill data */
> +                                               <0x0 0xD1 0xffffffff 0xffffffff 0x78>, /* Uncached fetch data access from bus */
> +                                               <0x0 0xE1 0xffffffff 0xffffffff 0x78>, /* Uncached load data access from bus */
> +                                               <0x0 0xF1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for uncached fetch data from bus */
> +                                               <0x0 0x101 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for uncached load data from bus */
> +                                               <0x0 0x111 0xffffffff 0xffffffff 0x78>, /* Main ITLB access */
> +                                               <0x0 0x121 0xffffffff 0xffffffff 0x78>, /* Main ITLB miss */
> +                                               <0x0 0x131 0xffffffff 0xffffffff 0x78>, /* Main DTLB access */
> +                                               <0x0 0x141 0xffffffff 0xffffffff 0x78>, /* Main DTLB miss */
> +                                               <0x0 0x151 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for Main ITLB fill data */
> +                                               <0x0 0x161 0xffffffff 0xffffffff 0x78>, /* Pipeline stall cycles caused by Main DTLB miss */
> +                                               <0x0 0x171 0xffffffff 0xffffffff 0x78>, /* Hardware prefetch bus access */
> +                                               <0x0 0x02 0xffffffff 0xffffffff 0x78>, /* Misprediction of conditional branches */
> +                                               <0x0 0x12 0xffffffff 0xffffffff 0x78>, /* Misprediction of taken conditional branches */
> +                                               <0x0 0x22 0xffffffff 0xffffffff 0x78>; /* Misprediction of targets of Return instructions */
> +};
> +```
> --
> 2.34.1
>


From anup at brainfault.org  Wed Nov 15 23:01:03 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 12:31:03 +0530
Subject: [PATCH v2 00/11] Add Andes PMU extension support
In-Reply-To: <20231019113713.3508153-1-peterlin@andestech.com>
References: <20231019113713.3508153-1-peterlin@andestech.com>
Message-ID: <CAAhSdy3TRicu-kE=JUesndHwGXmHfCBvAMBcH=Sv8bdqXtOV5g@mail.gmail.com>

On Thu, Oct 19, 2023 at 5:10?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> This patch series enables perf tool to utilize Andes PMU
> extension via PMU SBI calls, provides PMU device callbacks
> to achieve event sampling and mode filtering.
>
> This version also introduces a platform override fdt_add_pmu_mappings()
> to create event mappings before PMU setup if a valid PMU node
> is missing.
>
> The last patch provides a PMU node example used on AX45MP cores.
>
> The OpenSBI and Linux patches can be found on Andes Technology GitHub
> - https://github.com/andestech/opensbi/commits/andes-pmu-support-v2
> - https://github.com/andestech/linux/commits/andes-pmu-support-v2
>
> Yu Chien Peter Lin (11):
>   sbi: sbi_pmu: Improve sbi_pmu_init() error handling
>   sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
>   platform: include: andes45: Add PMU related CSR defines
>   platform: andes: Add Andes custom PMU support
>   platform: andes: Enable Andes PMU for AE350
>   platform: rzfive: Enable Andes PMU for RZ/Five
>   lib: utils: fdt_fixup: Add fdt_add_pmu_mappings() helper function
>   lib: utils: fdt_fixup: Allow preserving PMU properties
>   platform: andes: Factor out is_andes() helper
>   platform: andes: Implement andes_fdt_add_pmu_mappings platform
>     override
>   docs: pmu: Add Andes PMU node example

It would be great to have v3 sooner because next month we
have OpenSBI v1.4 release.

Regards,
Anup

>
>  docs/pmu_support.md                          |  82 ++++
>  include/sbi/sbi_ecall_interface.h            |   5 +
>  include/sbi/sbi_pmu.h                        |   6 +
>  include/sbi/sbi_scratch.h                    |   2 +
>  include/sbi_utils/fdt/fdt_fixup.h            |  48 +++
>  lib/sbi/sbi_pmu.c                            |  10 +-
>  lib/utils/fdt/fdt_fixup.c                    | 101 ++++-
>  lib/utils/fdt/fdt_pmu.c                      |   2 +-
>  platform/generic/Kconfig                     |   4 +
>  platform/generic/andes/Kconfig               |  11 +
>  platform/generic/andes/ae350.c               |  31 +-
>  platform/generic/andes/andes_hpm.c           | 381 +++++++++++++++++++
>  platform/generic/andes/andes_pmu.c           |  81 ++++
>  platform/generic/andes/objects.mk            |   2 +
>  platform/generic/include/andes/andes45.h     |  32 ++
>  platform/generic/include/andes/andes_hpm.h   |  83 ++++
>  platform/generic/include/andes/andes_pmu.h   |   8 +
>  platform/generic/include/platform_override.h |   1 +
>  platform/generic/platform.c                  |  16 +-
>  platform/generic/renesas/rzfive/rzfive.c     |  18 +-
>  20 files changed, 909 insertions(+), 15 deletions(-)
>  create mode 100644 platform/generic/andes/andes_hpm.c
>  create mode 100644 platform/generic/andes/andes_pmu.c
>  create mode 100644 platform/generic/include/andes/andes_hpm.h
>  create mode 100644 platform/generic/include/andes/andes_pmu.h
>
> --
> 2.34.1
>


From ganboing at gmail.com  Thu Nov 16 00:14:07 2023
From: ganboing at gmail.com (Bo Gan)
Date: Thu, 16 Nov 2023 00:14:07 -0800
Subject: Security and correctness issues around captured FDT pointers
In-Reply-To: <CAAhSdy3xDs54CCNzQFBHZsyV+iEMTEf0jw-yXsoi4u9osRYJQA@mail.gmail.com>
References: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>
 <CAK9=C2W3d1t+83kNoWhkA81M59Wen2gGa_EcgOJA1U19WJP4mQ@mail.gmail.com>
 <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>
 <CAAhSdy3xDs54CCNzQFBHZsyV+iEMTEf0jw-yXsoi4u9osRYJQA@mail.gmail.com>
Message-ID: <142bd9bf-1175-7c3c-50b0-0b1b30bff97a@gmail.com>

On 11/15/23 7:42 PM, Anup Patel wrote:
> On Thu, Nov 16, 2023 at 5:56?AM Bo Gan <ganboing at gmail.com> wrote:
>> Thanks for the reply, Anup. I did a little bit more digging on this and I think
>> #1 is only safe when OpenSBI's carrying the FDT itself (through FW_FDT_PATH).
>> Otherwise, we have to make sure FDT will never be accessed after cold boot done.
>> By zeroing-out the pointers, we can avoid speculative access as well. This helps
>> to defense against speculative timing attacks. (Maybe I'm too paranoid?).
>>
>> Given OpenSBI starts at very early stage, it's probably uncommon to not have
>> FW_FDT_PATH defined. I have a JH7110 on hand, and the OpenSBI (loaded by U-Boot
>> SPL) will carry the FDT built from U-Boot. I'd imagine other boards/vendors do
>> it similarly. Just in case folks really don't have FW_FDT_PATH, can we emit a
>> warning, and suggest the FDT to be protected through other mechanism?
> 
> That's an incorrect understanding of the FDT match table in OpenSBI.
> 
> The "generic_plat_match" points to a matching entry in a FDT match table
> provided by one of the platform override instances. It does not point to any
> part of the FDT. Please look at the fdt_match_node() implementation.
> 
Hi Anup, you are right, `generic_plat_match` won't capture the FDT. Thanks for
clarifying this.

Bo


From inochiama at outlook.com  Thu Nov 16 00:19:36 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 16:19:36 +0800
Subject: [PATCH v3 2/2] lib: sbi: Add regions merging when sanitizing domain region
In-Reply-To: <CAAhSdy2Dw6r77DS7ZDZk6fhjRKNTCJN-nOC-_9uHJPg05-qr8w@mail.gmail.com>
References: <CAAhSdy2Dw6r77DS7ZDZk6fhjRKNTCJN-nOC-_9uHJPg05-qr8w@mail.gmail.com>
Message-ID: <IA1PR20MB49531C1AB0DB4BBBF590227EBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

>
>On Thu, Nov 16, 2023 at 6:06?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>>> On Wed, Nov 15, 2023 at 1:35?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>>>>
>>>> As the domain will reject a new memory region which has a sub-regions
>>>> already in the domain, even the new region is bigger and has the same
>>>> flags. This problem can be solved by relaxing region restriction and
>>>> rechecking when adding and sanitizing domains.
>>>>
>>>> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
>>>> Reviewed-by: Anup Patel <anup at brainfault.org>
>>>> ---
>>>>  lib/sbi/sbi_domain.c | 68 ++++++++++++++++++++++++++++++--------------
>>>>  1 file changed, 46 insertions(+), 22 deletions(-)
>>>>
>>>> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
>>>> index 71cb381..ba8ce44 100644
>>>> --- a/lib/sbi/sbi_domain.c
>>>> +++ b/lib/sbi/sbi_domain.c
>>>> @@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
>>>>         return false;
>>>>  }
>>>>
>>>> -/** Check if regionA conflicts regionB */
>>>> -static bool is_region_conflict(const struct sbi_domain_memregion *regA,
>>>> -                               const struct sbi_domain_memregion *regB)
>>>> +/** Check if regionA can be replaced by regionB */
>>>> +static bool is_region_compatible(const struct sbi_domain_memregion *regA,
>>>> +                                const struct sbi_domain_memregion *regB)
>>>>  {
>>>> -       if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
>>>> -           regA->flags == regB->flags)
>>>> +       if (is_region_subset(regA, regB) && regA->flags == regB->flags)
>>>>                 return true;
>>>>
>>>>         return false;
>>>> @@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
>>>>         sbi_memcpy(reg2, &treg, sizeof(treg));
>>>>  }
>>>>
>>>> +static void clear_region(struct sbi_domain_memregion* reg)
>>>> +{
>>>> +       sbi_memset(reg, 0x0, sizeof(*reg));
>>>> +}
>>>> +
>>>>  static int sanitize_domain(struct sbi_domain *dom)
>>>>  {
>>>>         u32 i, j, count;
>>>> @@ -314,22 +318,46 @@ static int sanitize_domain(struct sbi_domain *dom)
>>>>                 return SBI_EINVAL;
>>>>         }
>>>>
>>>> +       /* Remove covered regions */
>>>> +       while(i < (count - 1)) {
>>>> +               reg = &dom->regions[i];
>>>> +               j = i + 1;
>>>> +               while (j < count) {
>>>> +                       reg1 = &dom->regions[j];
>>>> +
>>>> +                       /* reg1 is sub-region of reg, remove reg1 */
>>>> +                       if (is_region_compatible(reg1, reg)) {
>>>> +                               swap_region(reg1, &dom->regions[count - 1]);
>>>> +                               clear_region(&dom->regions[count - 1]);
>>>> +                               count--;
>>>> +                               continue;
>>>> +                       }
>>>> +
>>>> +                       /*
>>>> +                        * reg is sub-region of reg1, replace reg with reg1
>>>> +                        * and recheck all other regions.
>>>> +                        */
>>>> +                       if (is_region_compatible(reg, reg1)) {
>>>> +                               swap_region(reg, reg1);
>>>> +                               swap_region(reg1, &dom->regions[count - 1]);
>>>> +                               clear_region(&dom->regions[count - 1]);
>>>
>>> This does not seem right.
>>>
>>> We should disturb the sorted order of regions so all regions
>> >from i + 1 to count - 1 should be shifted up.
>>>
>>
>> It already shift up the region from i + 1 to count - 1.
>> In fact, the process can be described as follows:
>>
>> R[i] -> ... -> R[j] -> ... -> R[c-2] -> R[c-1]        (1)
>>   i              j                      count
>>
>> R[j] -> ... -> R[i] -> ... -> R[c-2] -> R[c-1]        (2)
>>   i              j                      count
>>
>> R[j] -> ... -> R[c-1] -> ... -> R[c-2] -> R[i]        (3)
>>   i              j                        count
>>
>> R[j] -> ... -> R[c-1] -> ... -> R[c-2]                (4)
>>   i              j              count
>>
>> It just replaces pos i with j, the replace pos j with last. This make no
>> change for the whole check. It just needs a recheck at pos i with new
>> count - 1 boundary after swapping.
>>
>> The only thing needed to be changed is just replacing pos i with the last,
>> without replacing pos i and j. Because we must recheck all the left regions.
>> So swapping pos i and j gives no help.
>
>This is exactly the problem. You are not maintaining the sorted
>order of the regions.
>
>..., R[i-1], R[i], R[i+1], ..., R[j-1], R[j], R[j+1], ....,
>R[count-2], R[count-1]
>
>to preserve the sorted order of regions after removal of R[i], this will be
>
>..., R[i-1], R[i+1], ..., R[j-1], R[j], R[j+1], ...., R[count-2], R[count-1]
>
>whereas you patch changes this to
>
>..., R[i-1], R[j], R[i+1], ..., R[j-1], R[count-1], R[j+1], ..., R[count-2]
>
>Further, the checking "reg1 is sub-region of reg, remove reg1" is
>totally redundant if the "Remove covered regions" is moved after
>"Sort the memory regions".
>
>This patch needs to be re-written.
>

OK, I will take it and prepare a new patch for this.
Thanks for your advice.

>Regards,
>Anup
>
>>
>> Please correct me if there is something I miss. Thanks.
>>
>>>> +                               count--;
>>>> +                               goto regions_recheck;
>>>> +                       }
>>>> +
>>>> +                       j++;
>>>> +               }
>>>> +
>>>> +               i++;
>>>> +regions_recheck:
>>>> +       }
>>>> +
>>>>         /* Sort the memory regions */
>>>>         for (i = 0; i < (count - 1); i++) {
>>>>                 reg = &dom->regions[i];
>>>>                 for (j = i + 1; j < count; j++) {
>>>>                         reg1 = &dom->regions[j];
>>>>
>>>> -                       if (is_region_conflict(reg1, reg)) {
>>>> -                               sbi_printf("%s: %s conflict between regions "
>>>> -                                       "(base=0x%lx order=%lu flags=0x%lx) and "
>>>> -                                       "(base=0x%lx order=%lu flags=0x%lx)\n",
>>>> -                                       __func__, dom->name,
>>>> -                                       reg->base, reg->order, reg->flags,
>>>> -                                       reg1->base, reg1->order, reg1->flags);
>>>> -                               return SBI_EINVAL;
>>>> -                       }
>>>> -
>>>>                         if (!is_region_before(reg1, reg))
>>>>                                 continue;
>>>>
>>>> @@ -576,14 +604,10 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
>>>>             (ROOT_REGION_MAX <= root_memregs_count))
>>>>                 return SBI_EINVAL;
>>>>
>>>> -       /* Check for conflicts */
>>>> +       /* Check whether compatible region exists for the new one */
>>>>         sbi_domain_for_each_memregion(&root, nreg) {
>>>> -               if (is_region_conflict(reg, nreg)) {
>>>> -                       sbi_printf("%s: is_region_conflict check failed"
>>>> -                       " 0x%lx conflicts existing 0x%lx\n", __func__,
>>>> -                                  reg->base, nreg->base);
>>>> -                       return SBI_EALREADY;
>>>> -               }
>>>> +               if (is_region_compatible(reg, nreg))
>>>> +                       return 0;
>>>>         }
>>>>
>>>>         /* Append the memregion to root memregions */
>>>> --
>>>> 2.42.1
>>>>
>>>
>>> Regards,
>>> Anup
>>>
>>>
>


From inochiama at outlook.com  Thu Nov 16 00:32:33 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 16:32:33 +0800
Subject: [PATCH v2 1/3] lib: utils/timer: Allow ACLINT MTIMER driver to setup quirks
In-Reply-To: <CAAhSdy15pM9S95joEberBR7jOuJ2pTWcns2Pb0vn0WXS8GeiPw@mail.gmail.com>
References: <CAAhSdy15pM9S95joEberBR7jOuJ2pTWcns2Pb0vn0WXS8GeiPw@mail.gmail.com>
Message-ID: <IA1PR20MB49532FE3429208601E4ADF45BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

>On Fri, Oct 13, 2023 at 12:17?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> The quirks checking will cause ACLINT step into a CLINT code path, this
>> is not expected when ACLINT needs a quirks.
>>
>> Add more check to ensure the timer is CLINT before applying quirks.
>> And now apply the general quirks after applying CLINT specific quirks.
>>
>> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
>> ---
>>  lib/utils/timer/fdt_timer_mtimer.c | 17 ++++++++++++++---
>>  1 file changed, 14 insertions(+), 3 deletions(-)
>>
>> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
>> index 9eaa11d..fb5917f 100644
>> --- a/lib/utils/timer/fdt_timer_mtimer.c
>> +++ b/lib/utils/timer/fdt_timer_mtimer.c
>> @@ -29,6 +29,12 @@ static SBI_LIST_HEAD(mtn_list);
>>
>>  static struct aclint_mtimer_data *mt_reference = NULL;
>>
>> +static int clint_contains_timer(const void *data)
>> +{
>> +       const struct timer_mtimer_quirks *quirks = data;
>> +       return quirks && quirks->mtime_offset;
>
>This is already broken because you are not allowing mtime_offset
>to be zero in quirks.
>
>Drop clint_contains_timer() function.
>

Drop this is OK for me, since it does break things.

>> +}
>> +
>>  static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>>                                   const struct fdt_match *match)
>>  {
>> @@ -58,7 +64,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>>                 return rc;
>>         }
>>
>> -       if (match->data) { /* SiFive CLINT */
>> +       if (clint_contains_timer(match->data)) {
>>                 const struct timer_mtimer_quirks *quirks = match->data;
>>
>>                 /* Set CLINT addresses */
>> @@ -74,8 +80,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>>                         mt->mtime_addr = mt->mtime_size = 0;
>>                 }
>>                 mt->mtimecmp_addr += quirks->mtime_offset;
>> -               /* Apply additional CLINT quirks */
>> -               mt->has_64bit_mmio = quirks->has_64bit_mmio;
>>         } else { /* RISC-V ACLINT MTIMER */
>>                 /* Set ACLINT MTIMER addresses */
>>                 mt->mtime_addr = addr[0];
>> @@ -84,6 +88,13 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>>                 mt->mtimecmp_size = size[1];
>>         }
>>
>> +       /* Apply additional quirks for CLINT and ACLINT */
>> +       if (match->data) {
>> +               const struct timer_mtimer_quirks *quirks = match->data;
>> +
>> +               mt->has_64bit_mmio = quirks->has_64bit_mmio;
>> +       }
>> +
>>         /* Check if MTIMER device has shared MTIME address */
>>         if (mt->mtime_size) {
>>                 mt->has_shared_mtime = false;
>> --
>> 2.42.0
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi
>
>Instead of above, I suggest to do as follows:
>
>diff --git a/lib/utils/timer/fdt_timer_mtimer.c
>b/lib/utils/timer/fdt_timer_mtimer.c
>index 9eaa11d..0187272 100644
>--- a/lib/utils/timer/fdt_timer_mtimer.c
>+++ b/lib/utils/timer/fdt_timer_mtimer.c
>@@ -16,6 +16,7 @@
> #include <sbi_utils/timer/aclint_mtimer.h>
>
> struct timer_mtimer_quirks {
>+    bool        custom_aclint;
>     unsigned int    mtime_offset;
>     bool        has_64bit_mmio;
>     bool        without_mtime;
>@@ -36,6 +37,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>     unsigned long addr[2], size[2];
>     struct timer_mtimer_node *mtn, *n;
>     struct aclint_mtimer_data *mt;
>+    const struct timer_mtimer_quirks *quirks = match->data;
>
>     mtn = sbi_zalloc(sizeof(*mtn));
>     if (!mtn)
>@@ -58,9 +60,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>         return rc;
>     }
>
>-    if (match->data) { /* SiFive CLINT */
>-        const struct timer_mtimer_quirks *quirks = match->data;
>-
>+    if (quirks && !quirks->custom_aclint) { /* SiFive CLINT */
>         /* Set CLINT addresses */
>         mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
>         mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
>@@ -74,8 +74,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>             mt->mtime_addr = mt->mtime_size = 0;
>         }
>         mt->mtimecmp_addr += quirks->mtime_offset;
>-        /* Apply additional CLINT quirks */
>-        mt->has_64bit_mmio = quirks->has_64bit_mmio;
>     } else { /* RISC-V ACLINT MTIMER */
>         /* Set ACLINT MTIMER addresses */
>         mt->mtime_addr = addr[0];
>@@ -84,6 +82,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>         mt->mtimecmp_size = size[1];
>     }
>
>+    if (quirks) {
>+        /* Apply additional quirks */
>+        mt->has_64bit_mmio = quirks->has_64bit_mmio;
>+    }
>+
>     /* Check if MTIMER device has shared MTIME address */
>     if (mt->mtime_size) {
>         mt->has_shared_mtime = false;
>
>Regards,
>Anup
>
>

Yes, this is my alternative options, I will take this advice and rebase
these patches. Thanks for your advice.


From ganboing at gmail.com  Thu Nov 16 00:41:09 2023
From: ganboing at gmail.com (Bo Gan)
Date: Thu, 16 Nov 2023 00:41:09 -0800
Subject: Question about potential missed IPI events
In-Reply-To: <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
 <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
 <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>
Message-ID: <af55316c-3647-f720-4000-86639e23a4d3@gmail.com>

On 11/15/23 7:12 PM, Xiang W wrote:
> ? 2023-11-15???? 16:33 -0800?Bo Gan???
>> On 11/10/23 10:05 PM, Bo Gan wrote:
>>> Right before Hart A ipi_dev->ipi_clear() for the msip write-1 issued by hart B,
>>> Hart C did an ipi_dev->ipi_send to hart A. Hart A then cleared the msip bit,
>>> resulting in both IPI (from B and C) coalesced to a single one. In this case,
>>> A will observe the ipi_type set by B, due to the smp_wmb + __io_bw, and the IPI
>>> was indeed generated by B's write to msip. However will the barrier ensure A
>>> also observe the write to ipi_type from hart C? For the coalescing to happen,
>>> clint will observe C's write-1 before A's write-0, does it imply A'll observe
>>> C's write to ipi_type from xchg?
> Does the picture below represent what you mean?
> 
>   B             A             C
>   |             |             |
>   | xchg        |             |
>   | send------->|             |
>   |             | xchg        |
>   |             |             | xchg
>   |             |<------------| send
>   |             | clear       |

Hi Xiang, Thank you so much for replying. I'd like to revise your chart a little bit:

Case 1, xchg_ulong in `ipi_process` gets reordered before ipi_clear:

   B                 A                C                 CLINT
   |                 |                |                  |
   | bit_set         |                |                  |
   | send -----------|----------------|----------------> |
   |                 | xchg           |                  |
   |                 |                | bit_set          |
   |                 |                | send ----------> |
   |                 | clear----------|----------------> |

In this case, A would observe the ipi_data->ipi_type *eventually*, but when?
A won't process C's IPI request until another IPI comes in the future, which
might cause the IPI request to wait indefinitely. This is not an efficiency
problem, but a correctness problem. I think the fix should be adding specific
fence (need to reason about the correct one) between `ipi_dev->ipi_clear` and
`atomic_raw_xchg_ulong` in `ipi_process`


Case 2, xchg_ulong doesn't get reordered, however:

   B                 A                C                 CLINT
   |                 |                |                  |
   | bit_set         |                |                  |
   | send -----------|----------------|----------------> |
   |                 | xchg           |            /---> |
   |                 | clear----------|-----------/----> |
   |                 |                | bit_set  /       |
   |                 |                | send ----        |

In this case, the same thing can happen. The request from C can be delayed
indefinitely. The cause is due to the clint observes memory order differently
than harts. I'm wondering if the clint can be viewed as another hart that
follows RVWMO? If yes, then this case is likely invalid.

Bo


From wxjstz at 126.com  Thu Nov 16 00:45:25 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 16 Nov 2023 16:45:25 +0800
Subject: [PATCH v2] lib: utils/serial: Optimize semihosting_putc
 implementation
In-Reply-To: <CAAhSdy0BphFUER+8kiza-Cw1qG3TD7xMZW9KkJeyAjqgxyqk4g@mail.gmail.com>
References: <20231018024205.270-1-cp0613@linux.alibaba.com>
	 <CAAhSdy0BphFUER+8kiza-Cw1qG3TD7xMZW9KkJeyAjqgxyqk4g@mail.gmail.com>
Message-ID: <4d7d9b69c833a7de7806019f734576a9998b7f76.camel@126.com>

? 2023-11-16???? 11:42 +0530?Anup Patel???
> On Wed, Oct 18, 2023 at 8:12?AM <cp0613 at linux.alibaba.com> wrote:
> > 
> > From: Chen Pei <cp0613 at linux.alibaba.com>
> > 
> > For some debuggers that do not implement SYSWRITEC and SYSREADC
> > operations, we can use SYSWRITE and SYSREAD instead like the
> > implementation of semihosting_getc().
> > 
> > Signed-off-by: Chen Pei <cp0613 at linux.alibaba.com>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
> 
> Looks good to me.
> 
> Reviewed-by: Anup Patel <anup at brainfault.org>
> 
> Applied this patch to the riscv/opensbi repo.
I submitted a new workaround.

https://patchwork.ozlabs.org/project/opensbi/patch/20231025043104.748104-1-wxjstz at 126.com/
https://patchwork.ozlabs.org/project/opensbi/patch/20231025043104.748104-2-wxjstz at 126.com/

Regards,
Xiang W
> 
> Thanks,
> Anup
> 
> > 
> > Changed from v1:
> > 1. simplified by calling semihosting_puts directly.
> > 
> > ---
> > ?lib/utils/serial/semihosting.c | 10 +++++-----
> > ?1 file changed, 5 insertions(+), 5 deletions(-)
> > 
> > diff --git a/lib/utils/serial/semihosting.c b/lib/utils/serial/semihosting.c
> > index ce65887..a27c69e 100644
> > --- a/lib/utils/serial/semihosting.c
> > +++ b/lib/utils/serial/semihosting.c
> > @@ -160,11 +160,6 @@ static long semihosting_write(long fd, const void *memp, size_t len)
> > 
> > ?/* clang-format on */
> > 
> > -static void semihosting_putc(char ch)
> > -{
> > -?????? semihosting_trap(SYSWRITEC, &ch);
> > -}
> > -
> > ?static unsigned long semihosting_puts(const char *str, unsigned long len)
> > ?{
> > ??????? char ch;
> > @@ -183,6 +178,11 @@ static unsigned long semihosting_puts(const char *str, unsigned long len)
> > ??????? return (ret < 0) ? 0 : ret;
> > ?}
> > 
> > +static void semihosting_putc(char ch)
> > +{
> > +?????? semihosting_puts(&ch, 1);
> > +}
> > +
> > ?static int semihosting_getc(void)
> > ?{
> > ??????? char ch = 0;
> > --
> > 2.25.1
> > 
> > 
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi



From inochiama at outlook.com  Thu Nov 16 01:05:57 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 17:05:57 +0800
Subject: [PATCH v3 0/3] lib: utils: Add T-HEAD C900 ACLINT
Message-ID: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

Add T-HEAD C900 ACLINT driver for sg2042

This patch need a new "reg-names" based timer register parsing to get
worked register. But this patch can be tested by manually setting regs
in the DT and temporarily setting without_mtimer quirks.

Link: https://lore.kernel.org/linux-riscv/6e48cbe5e60f9ada2fd1fe58e803e127f1a678e5.1696433229.git.unicorn_wang at outlook.com/
Link: https://lore.kernel.org/linux-riscv/1f6b82a1864477a51db33d3f295889ff985b497b.1696433229.git.unicorn_wang at outlook.com/

Changed from v2:
1. use new quirk to identify custom ACLINT

Changed from v1:
1. function fix and comment cleanup.

Changed from sg2042 v2:
1. Allow ACLINT has its own quirks
2. add new quirks for thead,c900-aclint-mtimer

This patch is moved from the patch series of supporting sg2042 as it
became complex.
The original patch: platform: generic: Add Sophgo sg2042 support

Inochi Amaoto (3):
  lib: utils/timer: Allow ACLINT MTIMER driver to setup quirks
  lib: utils/timer: mtimer: add separate T-Head C9xx CLINT mtimer
    compatible
  lib: utils/ipi: mswi: add separate T-Head C9xx CLINT mswi compatible

 lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
 lib/utils/timer/fdt_timer_mtimer.c | 20 +++++++++++++++-----
 2 files changed, 16 insertions(+), 5 deletions(-)

--
2.42.1



From inochiama at outlook.com  Thu Nov 16 01:07:13 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 17:07:13 +0800
Subject: [PATCH v3 1/3] lib: utils/timer: Allow ACLINT MTIMER driver to setup quirks
In-Reply-To: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB495301D1E80A07F9C6F0CB4FBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

The quirks checking will cause ACLINT step into a CLINT code path, this
is not expected when ACLINT needs custom quirks.

Add a new quirk to identify custom ACLINT, and apply the general quirks
after applying CLINT specific quirks.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
---
 lib/utils/timer/fdt_timer_mtimer.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
index 9eaa11d..29560fb 100644
--- a/lib/utils/timer/fdt_timer_mtimer.c
+++ b/lib/utils/timer/fdt_timer_mtimer.c
@@ -16,6 +16,7 @@
 #include <sbi_utils/timer/aclint_mtimer.h>
 
 struct timer_mtimer_quirks {
+	bool		custom_aclint;
 	unsigned int	mtime_offset;
 	bool		has_64bit_mmio;
 	bool		without_mtime;
@@ -36,6 +37,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 	unsigned long addr[2], size[2];
 	struct timer_mtimer_node *mtn, *n;
 	struct aclint_mtimer_data *mt;
+	const struct timer_mtimer_quirks *quirks = match->data;
 
 	mtn = sbi_zalloc(sizeof(*mtn));
 	if (!mtn)
@@ -58,9 +60,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 		return rc;
 	}
 
-	if (match->data) { /* SiFive CLINT */
-		const struct timer_mtimer_quirks *quirks = match->data;
-
+	if (quirks && !quirks->custom_aclint) { /* SiFive CLINT */
 		/* Set CLINT addresses */
 		mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
 		mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
@@ -74,8 +74,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 			mt->mtime_addr = mt->mtime_size = 0;
 		}
 		mt->mtimecmp_addr += quirks->mtime_offset;
-		/* Apply additional CLINT quirks */
-		mt->has_64bit_mmio = quirks->has_64bit_mmio;
 	} else { /* RISC-V ACLINT MTIMER */
 		/* Set ACLINT MTIMER addresses */
 		mt->mtime_addr = addr[0];
@@ -84,6 +82,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 		mt->mtimecmp_size = size[1];
 	}
 
+	/* Apply additional quirks */
+	if (quirks) {
+		mt->has_64bit_mmio = quirks->has_64bit_mmio;
+	}
+
 	/* Check if MTIMER device has shared MTIME address */
 	if (mt->mtime_size) {
 		mt->has_shared_mtime = false;
-- 
2.42.1



From inochiama at outlook.com  Thu Nov 16 01:07:14 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 17:07:14 +0800
Subject: [PATCH v3 2/3] lib: utils/timer: mtimer: add separate T-Head C9xx CLINT mtimer compatible
In-Reply-To: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB4953CBA138C8AE2410A21C33BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

T-HEAD allows soc vendor to map the mtimer and mswi of C9xx clint on
different address, which may cause a misuse if use the same compatible
string, add a new timer compatible string to avoid this.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
Link: https://lore.kernel.org/linux-riscv/6e48cbe5e60f9ada2fd1fe58e803e127f1a678e5.1696433229.git.unicorn_wang at outlook.com/
Reviewed-by: Guo Ren <guoren at kernel.org>
---
 lib/utils/timer/fdt_timer_mtimer.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
index 29560fb..86ad1aa 100644
--- a/lib/utils/timer/fdt_timer_mtimer.c
+++ b/lib/utils/timer/fdt_timer_mtimer.c
@@ -145,10 +145,17 @@ static const struct timer_mtimer_quirks thead_clint_quirks = {
 	.without_mtime  = true,
 };
 
+static const struct timer_mtimer_quirks thead_aclint_quirks = {
+	.has_64bit_mmio	= false,
+	.custom_aclint	= true,
+};
+
 static const struct fdt_match timer_mtimer_match[] = {
 	{ .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
 	{ .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
 	{ .compatible = "thead,c900-clint", .data = &thead_clint_quirks },
+	{ .compatible = "thead,c900-aclint-mtimer",
+	  .data = &thead_aclint_quirks },
 	{ .compatible = "riscv,aclint-mtimer" },
 	{ },
 };
-- 
2.42.1



From inochiama at outlook.com  Thu Nov 16 01:07:15 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 17:07:15 +0800
Subject: [PATCH v3 3/3] lib: utils/ipi: mswi: add separate T-Head C9xx CLINT mswi compatible
In-Reply-To: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB49535C88C1F556B2AD341566BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

Like the mtimer of T-HEAD C9xx clint, the mswi also needs new compatible
string to avoid misuse.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
Link: https://lore.kernel.org/linux-riscv/1f6b82a1864477a51db33d3f295889ff985b497b.1696433229.git.unicorn_wang at outlook.com/
Reviewed-by: Guo Ren <guoren at kernel.org>
---
 lib/utils/ipi/fdt_ipi_mswi.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/utils/ipi/fdt_ipi_mswi.c b/lib/utils/ipi/fdt_ipi_mswi.c
index e3a6378..5432db6 100644
--- a/lib/utils/ipi/fdt_ipi_mswi.c
+++ b/lib/utils/ipi/fdt_ipi_mswi.c
@@ -56,6 +56,7 @@ static const struct fdt_match ipi_mswi_match[] = {
 	{ .compatible = "riscv,clint0", .data = &clint_offset },
 	{ .compatible = "sifive,clint0", .data = &clint_offset },
 	{ .compatible = "thead,c900-clint", .data = &clint_offset },
+	{ .compatible = "thead,c900-aclint-mswi" },
 	{ .compatible = "riscv,aclint-mswi" },
 	{ },
 };
-- 
2.42.1



From wxjstz at 126.com  Thu Nov 16 01:19:06 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 16 Nov 2023 17:19:06 +0800
Subject: Question about potential missed IPI events
In-Reply-To: <af55316c-3647-f720-4000-86639e23a4d3@gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
	 <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
	 <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>
	 <af55316c-3647-f720-4000-86639e23a4d3@gmail.com>
Message-ID: <ec3855f1c0a00c8fe271bf4d170ae33eab55669c.camel@126.com>

? 2023-11-16???? 00:41 -0800?Bo Gan???
> On 11/15/23 7:12 PM, Xiang W wrote:
> > ? 2023-11-15???? 16:33 -0800?Bo Gan???
> > > On 11/10/23 10:05 PM, Bo Gan wrote:
> > > > Right before Hart A ipi_dev->ipi_clear() for the msip write-1 issued by hart B,
> > > > Hart C did an ipi_dev->ipi_send to hart A. Hart A then cleared the msip bit,
> > > > resulting in both IPI (from B and C) coalesced to a single one. In this case,
> > > > A will observe the ipi_type set by B, due to the smp_wmb + __io_bw, and the IPI
> > > > was indeed generated by B's write to msip. However will the barrier ensure A
> > > > also observe the write to ipi_type from hart C? For the coalescing to happen,
> > > > clint will observe C's write-1 before A's write-0, does it imply A'll observe
> > > > C's write to ipi_type from xchg?
> > Does the picture below represent what you mean?
> > 
> > ? B???????????? A???????????? C
> > ? |???????????? |???????????? |
> > ? | xchg??????? |???????????? |
> > ? | send------->|???????????? |
> > ? |???????????? | xchg??????? |
> > ? |???????????? |???????????? | xchg
> > ? |???????????? |<------------| send
> > ? |???????????? | clear?????? |
> 
> Hi Xiang, Thank you so much for replying. I'd like to revise your chart a little bit:
> 
> Case 1, xchg_ulong in `ipi_process` gets reordered before ipi_clear:
> 
> ?? B???????????????? A??????????????? C???????????????? CLINT
> ?? |???????????????? |??????????????? |????????????????? |
> ?? | bit_set???????? |??????????????? |????????????????? |
> ?? | send -----------|----------------|----------------> |
> ?? |???????????????? | xchg?????????? |????????????????? |
> ?? |???????????????? |??????????????? | bit_set????????? |
> ?? |???????????????? |??????????????? | send ----------> |
> ?? |???????????????? | clear----------|----------------> |
> 
> In this case, A would observe the ipi_data->ipi_type *eventually*, but when?
> A won't process C's IPI request until another IPI comes in the future, which
> might cause the IPI request to wait indefinitely. This is not an efficiency
> problem, but a correctness problem. I think the fix should be adding specific
> fence (need to reason about the correct one) between `ipi_dev->ipi_clear` and
> `atomic_raw_xchg_ulong` in `ipi_process`
Your statement is more correct.

> 
> 
> Case 2, xchg_ulong doesn't get reordered, however:
> 
> ?? B???????????????? A??????????????? C???????????????? CLINT
> ?? |???????????????? |??????????????? |????????????????? |
> ?? | bit_set???????? |??????????????? |????????????????? |
> ?? | send -----------|----------------|----------------> |
> ?? |???????????????? | xchg?????????? |??????????? /---> |
> ?? |???????????????? | clear----------|-----------/----> |
> ?? |???????????????? |??????????????? | bit_set? /?????? |
> ?? |???????????????? |??????????????? | send ----??????? |
 B             A             C
 |             |             |
 | bit_set     |             |
 | send------->|             | bit_set  
 |             | clear       |
 |             | xchg        |
 |             |<------------| send
 |             |             |

What I want to say about this picture is that due to different running
speeds, A may get C's bit_set befor C send ipi. This will cause A to
enter exception handling one more time.

Why is there an upward-sloping arrow in your picture? Multiple cores
observing atomic operations at the same address should not be
inconsistent.

Regards,
Xiang W

> 
> In this case, the same thing can happen. The request from C can be delayed
> indefinitely. The cause is due to the clint observes memory order differently
> than harts. I'm wondering if the clint can be viewed as another hart that
> follows RVWMO? If yes, then this case is likely invalid.
> 
> Bo


From ganboing at gmail.com  Thu Nov 16 01:44:11 2023
From: ganboing at gmail.com (Bo Gan)
Date: Thu, 16 Nov 2023 01:44:11 -0800
Subject: Question about potential missed IPI events
In-Reply-To: <ec3855f1c0a00c8fe271bf4d170ae33eab55669c.camel@126.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
 <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
 <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>
 <af55316c-3647-f720-4000-86639e23a4d3@gmail.com>
 <ec3855f1c0a00c8fe271bf4d170ae33eab55669c.camel@126.com>
Message-ID: <3113ff3f-f3fa-67e7-3b04-ce775eed5891@gmail.com>

On 11/16/23 1:19 AM, Xiang W wrote:
> ? 2023-11-16???? 00:41 -0800?Bo Gan???
>>
>> Hi Xiang, Thank you so much for replying. I'd like to revise your chart a little bit:
>>
>> Case 1, xchg_ulong in `ipi_process` gets reordered before ipi_clear:
>>
>>  ?? B???????????????? A??????????????? C???????????????? CLINT
>>  ?? |???????????????? |??????????????? |????????????????? |
>>  ?? | bit_set???????? |??????????????? |????????????????? |
>>  ?? | send -----------|----------------|----------------> |
>>  ?? |???????????????? | xchg?????????? |????????????????? |
>>  ?? |???????????????? |??????????????? | bit_set????????? |
>>  ?? |???????????????? |??????????????? | send ----------> |
>>  ?? |???????????????? | clear----------|----------------> |
>>
>> In this case, A would observe the ipi_data->ipi_type *eventually*, but when?
>> A won't process C's IPI request until another IPI comes in the future, which
>> might cause the IPI request to wait indefinitely. This is not an efficiency
>> problem, but a correctness problem. I think the fix should be adding specific
>> fence (need to reason about the correct one) between `ipi_dev->ipi_clear` and
>> `atomic_raw_xchg_ulong` in `ipi_process`
> Your statement is more correct.
> 

For case 1, can we agree there's an issue in the current code, and it needs to
be fixed?

>>
>>
>> Case 2, xchg_ulong doesn't get reordered, however:
>>
>>  ?? B???????????????? A??????????????? C???????????????? CLINT
>>  ?? |???????????????? |??????????????? |????????????????? |
>>  ?? | bit_set???????? |??????????????? |????????????????? |
>>  ?? | send -----------|----------------|----------------> |
>>  ?? |???????????????? | xchg?????????? |??????????? /---> |
>>  ?? |???????????????? | clear----------|-----------/----> |
>>  ?? |???????????????? |??????????????? | bit_set? /?????? |
>>  ?? |???????????????? |??????????????? | send ----??????? |
> Why is there an upward-sloping arrow in your picture? Multiple cores
> observing atomic operations at the same address should not be
> inconsistent.
> 

Let me clarify case 2 a little bit more. Let me use a downward-sloping
arrow. In this case the clear sent from A took so long that it arrived
at CLINT after C's send.

     B                 A                C                 CLINT
     |                 |                |                  |
     | bit_set ------> |                |                  |
     | send -----------|----------------|----------------> |
     |                 | xchg           |                  |
     |                 | clear----------|----------------> |
     |                 |          \     |                  |
     |                 | <---------\----|-bit_set          |
     |                 |            \   | send ----------> |
     |                 |             \  |                  |
     |                 |              --|----------------> |
                                 
Back to your question. Here we have two memory/IO locations, not one.
First is the `ipi_data->ipi_type`, second is the msip MMIO for hart A.
Essentially, can the following happen?

  i.  Assume there's proper fence to ensure xchg orders before clear in A.

  ii. From A's point of view, the xchg observes bit_set from B,
      but not bit_set from C.

  iii.From CLINT's point of view, it observes send from B, then send from C,
      then clear from A.

I don't know how to reason about this. The only thing I can think of is the
global memory order defined by RVWMO. Can we apply the global memory order
to the interaction between hart and clint?

Bo


From wxjstz at 126.com  Thu Nov 16 02:11:39 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 16 Nov 2023 18:11:39 +0800
Subject: Question about potential missed IPI events
In-Reply-To: <3113ff3f-f3fa-67e7-3b04-ce775eed5891@gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
	 <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
	 <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>
	 <af55316c-3647-f720-4000-86639e23a4d3@gmail.com>
	 <ec3855f1c0a00c8fe271bf4d170ae33eab55669c.camel@126.com>
	 <3113ff3f-f3fa-67e7-3b04-ce775eed5891@gmail.com>
Message-ID: <3e26f02e75e820d88ce98098d879975a8a4657a6.camel@126.com>

? 2023-11-16???? 01:44 -0800?Bo Gan???
> On 11/16/23 1:19 AM, Xiang W wrote:
> > ? 2023-11-16???? 00:41 -0800?Bo Gan???
> > > 
> > > Hi Xiang, Thank you so much for replying. I'd like to revise your chart a little bit:
> > > 
> > > Case 1, xchg_ulong in `ipi_process` gets reordered before ipi_clear:
> > > 
> > > ??? B???????????????? A??????????????? C???????????????? CLINT
> > > ??? |???????????????? |??????????????? |????????????????? |
> > > ??? | bit_set???????? |??????????????? |????????????????? |
> > > ??? | send -----------|----------------|----------------> |
> > > ??? |???????????????? | xchg?????????? |????????????????? |
> > > ??? |???????????????? |??????????????? | bit_set????????? |
> > > ??? |???????????????? |??????????????? | send ----------> |
> > > ??? |???????????????? | clear----------|----------------> |
> > > 
> > > In this case, A would observe the ipi_data->ipi_type *eventually*, but when?
> > > A won't process C's IPI request until another IPI comes in the future, which
> > > might cause the IPI request to wait indefinitely. This is not an efficiency
> > > problem, but a correctness problem. I think the fix should be adding specific
> > > fence (need to reason about the correct one) between `ipi_dev->ipi_clear` and
> > > `atomic_raw_xchg_ulong` in `ipi_process`
> > Your statement is more correct.
> > 
> 
> For case 1, can we agree there's an issue in the current code, and it needs to
> be fixed?
Yes!
> 
> > > 
> > > 
> > > Case 2, xchg_ulong doesn't get reordered, however:
> > > 
> > > ??? B???????????????? A??????????????? C???????????????? CLINT
> > > ??? |???????????????? |??????????????? |????????????????? |
> > > ??? | bit_set???????? |??????????????? |????????????????? |
> > > ??? | send -----------|----------------|----------------> |
> > > ??? |???????????????? | xchg?????????? |??????????? /---> |
> > > ??? |???????????????? | clear----------|-----------/----> |
> > > ??? |???????????????? |??????????????? | bit_set? /?????? |
> > > ??? |???????????????? |??????????????? | send ----??????? |
> > Why is there an upward-sloping arrow in your picture? Multiple cores
> > observing atomic operations at the same address should not be
> > inconsistent.
> > 
> 
> Let me clarify case 2 a little bit more. Let me use a downward-sloping
> arrow. In this case the clear sent from A took so long that it arrived
> at CLINT after C's send.
> 
> ???? B???????????????? A??????????????? C???????????????? CLINT
> ???? |???????????????? |??????????????? |????????????????? |
> ???? | bit_set ------> |??????????????? |????????????????? |
> ???? | send -----------|----------------|----------------> |
> ???? |???????????????? | xchg?????????? |????????????????? |
> ???? |???????????????? | clear----------|----------------> |
> ???? |???????????????? |????????? \???? |????????????????? |
> ???? |???????????????? | <---------\----|-bit_set????????? |
> ???? |???????????????? |??????????? \?? | send ----------> |
> ???? |???????????????? |???????????? \? |????????????????? |
> ???? |???????????????? |????????????? --|----------------> |
> ???????????????????????????????? 
> Back to your question. Here we have two memory/IO locations, not one.
> First is the `ipi_data->ipi_type`, second is the msip MMIO for hart A.
> Essentially, can the following happen?
> 
> ? i.? Assume there's proper fence to ensure xchg orders before clear in A.
> 
> ? ii. From A's point of view, the xchg observes bit_set from B,
> ????? but not bit_set from C.
> 
> ? iii.From CLINT's point of view, it observes send from B, then send from C,
> ????? then clear from A.
> 
> I don't know how to reason about this. The only thing I can think of is the
> global memory order defined by RVWMO. Can we apply the global memory order
> to the interaction between hart and clint?

I understand what you mean. The one I proposed is caused by inconsistent core
speeds and has nothing to do with memory order. Do you think the memory order
observed by hart and clint is inconsistent?

How to solve this? fence should only restrict memory ordering on the same core.

Regards,
Xiang W
> 
> Bo


From anup at brainfault.org  Thu Nov 16 02:16:41 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 15:46:41 +0530
Subject: [PATCH v3 1/3] lib: utils/timer: Allow ACLINT MTIMER driver to
 setup quirks
In-Reply-To: <IA1PR20MB495301D1E80A07F9C6F0CB4FBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB495301D1E80A07F9C6F0CB4FBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy3oZW=N0PzSAibRj53K-njkaHWkLDe8-kQ8E5hBkp9aJA@mail.gmail.com>

On Thu, Nov 16, 2023 at 2:36?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> The quirks checking will cause ACLINT step into a CLINT code path, this
> is not expected when ACLINT needs custom quirks.
>
> Add a new quirk to identify custom ACLINT, and apply the general quirks
> after applying CLINT specific quirks.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/utils/timer/fdt_timer_mtimer.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> index 9eaa11d..29560fb 100644
> --- a/lib/utils/timer/fdt_timer_mtimer.c
> +++ b/lib/utils/timer/fdt_timer_mtimer.c
> @@ -16,6 +16,7 @@
>  #include <sbi_utils/timer/aclint_mtimer.h>
>
>  struct timer_mtimer_quirks {
> +       bool            custom_aclint;
>         unsigned int    mtime_offset;
>         bool            has_64bit_mmio;
>         bool            without_mtime;
> @@ -36,6 +37,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>         unsigned long addr[2], size[2];
>         struct timer_mtimer_node *mtn, *n;
>         struct aclint_mtimer_data *mt;
> +       const struct timer_mtimer_quirks *quirks = match->data;
>
>         mtn = sbi_zalloc(sizeof(*mtn));
>         if (!mtn)
> @@ -58,9 +60,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                 return rc;
>         }
>
> -       if (match->data) { /* SiFive CLINT */
> -               const struct timer_mtimer_quirks *quirks = match->data;
> -
> +       if (quirks && !quirks->custom_aclint) { /* SiFive CLINT */
>                 /* Set CLINT addresses */
>                 mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
>                 mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
> @@ -74,8 +74,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                         mt->mtime_addr = mt->mtime_size = 0;
>                 }
>                 mt->mtimecmp_addr += quirks->mtime_offset;
> -               /* Apply additional CLINT quirks */
> -               mt->has_64bit_mmio = quirks->has_64bit_mmio;
>         } else { /* RISC-V ACLINT MTIMER */
>                 /* Set ACLINT MTIMER addresses */
>                 mt->mtime_addr = addr[0];
> @@ -84,6 +82,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                 mt->mtimecmp_size = size[1];
>         }
>
> +       /* Apply additional quirks */
> +       if (quirks) {
> +               mt->has_64bit_mmio = quirks->has_64bit_mmio;
> +       }
> +
>         /* Check if MTIMER device has shared MTIME address */
>         if (mt->mtime_size) {
>                 mt->has_shared_mtime = false;
> --
> 2.42.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Thu Nov 16 02:39:39 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 16:09:39 +0530
Subject: [PATCH v3 2/3] lib: utils/timer: mtimer: add separate T-Head C9xx
 CLINT mtimer compatible
In-Reply-To: <IA1PR20MB4953CBA138C8AE2410A21C33BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB4953CBA138C8AE2410A21C33BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy34OexruH0=V0qLho+HGzUs06uEVSHJe5Q=aHaAfnBxww@mail.gmail.com>

On Thu, Nov 16, 2023 at 2:36?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> T-HEAD allows soc vendor to map the mtimer and mswi of C9xx clint on
> different address, which may cause a misuse if use the same compatible
> string, add a new timer compatible string to avoid this.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> Link: https://lore.kernel.org/linux-riscv/6e48cbe5e60f9ada2fd1fe58e803e127f1a678e5.1696433229.git.unicorn_wang at outlook.com/
> Reviewed-by: Guo Ren <guoren at kernel.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/utils/timer/fdt_timer_mtimer.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> index 29560fb..86ad1aa 100644
> --- a/lib/utils/timer/fdt_timer_mtimer.c
> +++ b/lib/utils/timer/fdt_timer_mtimer.c
> @@ -145,10 +145,17 @@ static const struct timer_mtimer_quirks thead_clint_quirks = {
>         .without_mtime  = true,
>  };
>
> +static const struct timer_mtimer_quirks thead_aclint_quirks = {
> +       .has_64bit_mmio = false,
> +       .custom_aclint  = true,
> +};
> +
>  static const struct fdt_match timer_mtimer_match[] = {
>         { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
>         { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
>         { .compatible = "thead,c900-clint", .data = &thead_clint_quirks },
> +       { .compatible = "thead,c900-aclint-mtimer",
> +         .data = &thead_aclint_quirks },
>         { .compatible = "riscv,aclint-mtimer" },
>         { },
>  };
> --
> 2.42.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Thu Nov 16 02:44:51 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 16:14:51 +0530
Subject: [PATCH v3 3/3] lib: utils/ipi: mswi: add separate T-Head C9xx
 CLINT mswi compatible
In-Reply-To: <IA1PR20MB49535C88C1F556B2AD341566BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB49535C88C1F556B2AD341566BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy2v-4zxjPftKUyyuEwDtnHcBu7h227D6cKmzUkxUriUog@mail.gmail.com>

On Thu, Nov 16, 2023 at 2:36?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Like the mtimer of T-HEAD C9xx clint, the mswi also needs new compatible
> string to avoid misuse.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> Link: https://lore.kernel.org/linux-riscv/1f6b82a1864477a51db33d3f295889ff985b497b.1696433229.git.unicorn_wang at outlook.com/
> Reviewed-by: Guo Ren <guoren at kernel.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/utils/ipi/fdt_ipi_mswi.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/lib/utils/ipi/fdt_ipi_mswi.c b/lib/utils/ipi/fdt_ipi_mswi.c
> index e3a6378..5432db6 100644
> --- a/lib/utils/ipi/fdt_ipi_mswi.c
> +++ b/lib/utils/ipi/fdt_ipi_mswi.c
> @@ -56,6 +56,7 @@ static const struct fdt_match ipi_mswi_match[] = {
>         { .compatible = "riscv,clint0", .data = &clint_offset },
>         { .compatible = "sifive,clint0", .data = &clint_offset },
>         { .compatible = "thead,c900-clint", .data = &clint_offset },
> +       { .compatible = "thead,c900-aclint-mswi" },
>         { .compatible = "riscv,aclint-mswi" },
>         { },
>  };
> --
> 2.42.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From inochiama at outlook.com  Thu Nov 16 02:48:50 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 18:48:50 +0800
Subject: [PATCH v4 0/2] lib: sbi: Add sub-regions check for sanitizing domain
Message-ID: <IA1PR20MB49535E12C4609D9816C7C824BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

As the domain will reject a new memory region which has a sub-regions
already in the domain, even the new region is bigger and has the same
flags.

To resolve this problem, just relax restriction of memory region when
adding it, and merge the region into the existed domain regions when
sorting regions.

Changed from v3:
1. rearrange the code to avoid unnecessary check.
2. use a stable check to preserve region orders.

Changed from v2:
1. rebase to master
2. removed the useless changed for conflict check.
3. improve the commit for the patch 3.

Changed from v1:
1. separate the patch into three parts.

Inochi Amaoto (2):
  lib: sbi: separate the swap operation of domain region
  lib: sbi: Add regions merging when sanitizing domain region

 lib/sbi/sbi_domain.c | 75 +++++++++++++++++++++++++++++---------------
 1 file changed, 49 insertions(+), 26 deletions(-)

--
2.42.1



From inochiama at outlook.com  Thu Nov 16 02:49:38 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 18:49:38 +0800
Subject: [PATCH v4 1/2] lib: sbi: separate the swap operation of domain region
In-Reply-To: <IA1PR20MB49535E12C4609D9816C7C824BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49535E12C4609D9816C7C824BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB495392B05FE034A393C7C539BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

Swapping domain region is a common operation when sorting domain region,
so separate it as a function to make code clean.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_domain.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
index 4d248d0..71cb381 100644
--- a/lib/sbi/sbi_domain.c
+++ b/lib/sbi/sbi_domain.c
@@ -256,10 +256,20 @@ static const struct sbi_domain_memregion *find_next_subset_region(
 	return ret;
 }
 
+static void swap_region(struct sbi_domain_memregion* reg1,
+			struct sbi_domain_memregion* reg2)
+{
+	struct sbi_domain_memregion treg;
+
+	sbi_memcpy(&treg, reg1, sizeof(treg));
+	sbi_memcpy(reg1, reg2, sizeof(treg));
+	sbi_memcpy(reg2, &treg, sizeof(treg));
+}
+
 static int sanitize_domain(struct sbi_domain *dom)
 {
 	u32 i, j, count;
-	struct sbi_domain_memregion treg, *reg, *reg1;
+	struct sbi_domain_memregion *reg, *reg1;
 
 	/* Check possible HARTs */
 	if (!dom->possible_harts) {
@@ -323,9 +333,7 @@ static int sanitize_domain(struct sbi_domain *dom)
 			if (!is_region_before(reg1, reg))
 				continue;
 
-			sbi_memcpy(&treg, reg1, sizeof(treg));
-			sbi_memcpy(reg1, reg, sizeof(treg));
-			sbi_memcpy(reg, &treg, sizeof(treg));
+			swap_region(reg, reg1);
 		}
 	}
 
-- 
2.42.1



From inochiama at outlook.com  Thu Nov 16 02:49:39 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 18:49:39 +0800
Subject: [PATCH v4 2/2] lib: sbi: Add regions merging when sanitizing domain region
In-Reply-To: <IA1PR20MB49535E12C4609D9816C7C824BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49535E12C4609D9816C7C824BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB4953FEB382550CB12FA24E2FBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

As the domain will reject a new memory region which has a sub-regions
already in the domain, even the new region is bigger and has the same
flags. This problem can be solved by relaxing region restriction and
rechecking when adding and sanitizing domains.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
---
 lib/sbi/sbi_domain.c | 59 +++++++++++++++++++++++++++-----------------
 1 file changed, 37 insertions(+), 22 deletions(-)

diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
index 71cb381..bc60562 100644
--- a/lib/sbi/sbi_domain.c
+++ b/lib/sbi/sbi_domain.c
@@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
 	return false;
 }
 
-/** Check if regionA conflicts regionB */
-static bool is_region_conflict(const struct sbi_domain_memregion *regA,
-				const struct sbi_domain_memregion *regB)
+/** Check if regionA can be replaced by regionB */
+static bool is_region_compatible(const struct sbi_domain_memregion *regA,
+				 const struct sbi_domain_memregion *regB)
 {
-	if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
-	    regA->flags == regB->flags)
+	if (is_region_subset(regA, regB) && regA->flags == regB->flags)
 		return true;
 
 	return false;
@@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
 	sbi_memcpy(reg2, &treg, sizeof(treg));
 }
 
+static void clear_region(struct sbi_domain_memregion* reg)
+{
+	sbi_memset(reg, 0x0, sizeof(*reg));
+}
+
 static int sanitize_domain(struct sbi_domain *dom)
 {
 	u32 i, j, count;
@@ -320,16 +324,6 @@ static int sanitize_domain(struct sbi_domain *dom)
 		for (j = i + 1; j < count; j++) {
 			reg1 = &dom->regions[j];
 
-			if (is_region_conflict(reg1, reg)) {
-				sbi_printf("%s: %s conflict between regions "
-					"(base=0x%lx order=%lu flags=0x%lx) and "
-					"(base=0x%lx order=%lu flags=0x%lx)\n",
-					__func__, dom->name,
-					reg->base, reg->order, reg->flags,
-					reg1->base, reg1->order, reg1->flags);
-				return SBI_EINVAL;
-			}
-
 			if (!is_region_before(reg1, reg))
 				continue;
 
@@ -337,6 +331,31 @@ static int sanitize_domain(struct sbi_domain *dom)
 		}
 	}
 
+	/* Remove covered regions */
+	while(i < (count - 1)) {
+		bool need_removed = false;
+		reg = &dom->regions[i];
+
+		for (j = i + 1; j < count; j++) {
+			reg1 = &dom->regions[j];
+
+			if (is_region_compatible(reg, reg1)) {
+				need_removed = true;
+				break;
+			}
+		}
+
+		/* find a region is superset of reg, remove reg */
+		if (need_removed) {
+			for (j = i; j < (count - 1); j++)
+				swap_region(&dom->regions[j],
+					    &dom->regions[j + 1]);
+			clear_region(&dom->regions[count - 1]);
+			count--;
+		} else
+			i++;
+	}
+
 	/*
 	 * We don't need to check boot HART id of domain because if boot
 	 * HART id is not possible/assigned to this domain then it won't
@@ -576,14 +595,10 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
 	    (ROOT_REGION_MAX <= root_memregs_count))
 		return SBI_EINVAL;
 
-	/* Check for conflicts */
+	/* Check whether compatible region exists for the new one */
 	sbi_domain_for_each_memregion(&root, nreg) {
-		if (is_region_conflict(reg, nreg)) {
-			sbi_printf("%s: is_region_conflict check failed"
-			" 0x%lx conflicts existing 0x%lx\n", __func__,
-				   reg->base, nreg->base);
-			return SBI_EALREADY;
-		}
+		if (is_region_compatible(reg, nreg))
+			return 0;
 	}
 
 	/* Append the memregion to root memregions */
-- 
2.42.1



From anup at brainfault.org  Thu Nov 16 02:52:12 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 16:22:12 +0530
Subject: [PATCH] fw_base.S: Fix boot hart status synchronization
In-Reply-To: <20231019092734.2554798-1-guoren@kernel.org>
References: <20231019092734.2554798-1-guoren@kernel.org>
Message-ID: <CAAhSdy1g=4s17Pd6neLkgsAx=ApAN7b2HCkoUtLNh4p+0KG6Uw@mail.gmail.com>

On Thu, Oct 19, 2023 at 2:57?PM <guoren at kernel.org> wrote:
>
> From: Guo Ren <guoren at linux.alibaba.com>
>
> It's wrong to put the fence after setting the boot status flag because
> all relocation operations must be finished before setting the status
> flag. So, this fence must be put before the setting status flag, and
> there is no use in putting a fence between _start_warm and setting
> status flag.
>
> Also, nop can't delay other harts too much, so use div instead, just
> like Linux cpu_relax. Current opensbi force enables ?M? Standard
> Extension, and mul instructions have been used in the fw_base.S.
>
> After the above two fixes, the boot hart index param of the
> fw_dynamic_info could be guaranteed properly for all platforms.
>
> Signed-off-by: Guo Ren <guoren at linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren at kernel.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  firmware/fw_base.S | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/firmware/fw_base.S b/firmware/fw_base.S
> index 42b03fe890f0..f7763f4d9e56 100644
> --- a/firmware/fw_base.S
> +++ b/firmware/fw_base.S
> @@ -399,8 +399,8 @@ _fdt_reloc_done:
>         /* mark boot hart done */
>         li      t0, BOOT_STATUS_BOOT_HART_DONE
>         lla     t1, _boot_status
> -       REG_S   t0, 0(t1)
>         fence   rw, rw
> +       REG_S   t0, 0(t1)
>         j       _start_warm
>
>         /* waiting for boot hart to be done (_boot_status == 2) */
> @@ -409,9 +409,9 @@ _wait_for_boot_hart:
>         lla     t1, _boot_status
>         REG_L   t1, 0(t1)
>         /* Reduce the bus traffic so that boot hart may proceed faster */
> -       nop
> -       nop
> -       nop
> +       div     t2, t2, zero
> +       div     t2, t2, zero
> +       div     t2, t2, zero
>         bne     t0, t1, _wait_for_boot_hart
>
>  _start_warm:
> --
> 2.36.1
>


From anup at brainfault.org  Thu Nov 16 03:02:22 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 16:32:22 +0530
Subject: [PATCH] utils/reset: Remove fdt_reset_thead
In-Reply-To: <20231012085913.1679067-1-guoren@kernel.org>
References: <20231012085913.1679067-1-guoren@kernel.org>
Message-ID: <CAAhSdy3a9337LWWA0pftvb4FhU-vv68Cz_L5wtorjKr5KXyBaA@mail.gmail.com>

On Thu, Oct 12, 2023 at 2:29?PM <guoren at kernel.org> wrote:
>
> From: Guo Ren <guoren at linux.alibaba.com>
>
> In the past, we used fdt_reset_thead to help customers with prototype
> verification. However, with the emergence of the Big-little SoC system,
> it can no longer meet the demand. Therefore, we use zero_stage_boot
> instead of fdt_reset_thead. It cleans up the opensbi code and ends the
> disputation of reset_sample's dts.
>
> This patch removes the fdt_reset_thead component and updates the related
> doc.
>
> Signed-off-by: Guo Ren <guoren at linux.alibaba.com>
> Signed-off-by: Guo Ren <guoren at kernel.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  docs/platform/thead-c9xx.md           | 183 +-------------------------
>  lib/utils/reset/Kconfig               |   4 -
>  lib/utils/reset/fdt_reset_thead.c     | 134 -------------------
>  lib/utils/reset/fdt_reset_thead.h     |  23 ----
>  lib/utils/reset/fdt_reset_thead_asm.S |  47 -------
>  lib/utils/reset/objects.mk            |   4 -
>  platform/generic/configs/defconfig    |   1 -
>  7 files changed, 7 insertions(+), 389 deletions(-)
>  delete mode 100644 lib/utils/reset/fdt_reset_thead.c
>  delete mode 100644 lib/utils/reset/fdt_reset_thead.h
>  delete mode 100644 lib/utils/reset/fdt_reset_thead_asm.S
>
> diff --git a/docs/platform/thead-c9xx.md b/docs/platform/thead-c9xx.md
> index 8bb9e91f1a9b..cc05b58ca6b6 100644
> --- a/docs/platform/thead-c9xx.md
> +++ b/docs/platform/thead-c9xx.md
> @@ -1,7 +1,7 @@
>  T-HEAD C9xx Series Processors
>  =============================
>
> -The **C9xx** series processors are high-performance RISC-V architecture
> +The C9xx series processors are high-performance RISC-V architecture
>  multi-core processors with AI vector acceleration engine.
>
>  For more details, refer [T-HEAD.CN](https://www.t-head.cn/)
> @@ -12,185 +12,16 @@ To build the platform-specific library and firmware images, provide the
>  Platform Options
>  ----------------
>
> -The *T-HEAD C9xx* does not have any platform-specific compile options
> +The T-HEAD C9xx does not have any platform-specific compile options
>  because it uses generic platform.
>
>  ```
> -CROSS_COMPILE=riscv64-linux-gnu- PLATFORM=generic /usr/bin/make
> +CROSS_COMPILE=riscv64-linux-gnu- PLATFORM=generic make
>  ```
>
> -The *T-HEAD C9xx* DTB provided to OpenSBI generic firmwares will usually have
> -"riscv,clint0", "riscv,plic0", "thead,reset-sample" compatible strings.
> +Here is the simplest boot flow for a fpga prototype:
>
> -DTS Example1: (Single core, eg: Allwinner D1 - c906)
> -----------------------------------------------------
> + (Jtag gdbinit) -> (zsb) -> (opensbi) -> (linux)
>
> -```
> -       cpus {
> -               #address-cells = <1>;
> -               #size-cells = <0>;
> -               timebase-frequency = <3000000>;
> -               cpu at 0 {
> -                       device_type = "cpu";
> -                       reg = <0>;
> -                       status = "okay";
> -                       compatible = "riscv";
> -                       riscv,isa = "rv64imafdcv";
> -                       mmu-type = "riscv,sv39";
> -                       cpu0_intc: interrupt-controller {
> -                               #interrupt-cells = <1>;
> -                               compatible = "riscv,cpu-intc";
> -                               interrupt-controller;
> -                       };
> -               };
> -       };
> -
> -       soc {
> -               #address-cells = <2>;
> -               #size-cells = <2>;
> -               compatible = "simple-bus";
> -               ranges;
> -
> -               clint0: clint at 14000000 {
> -                       compatible = "allwinner,sun20i-d1-clint";
> -                       interrupts-extended = <
> -                               &cpu0_intc  3 &cpu0_intc  7
> -                               >;
> -                       reg = <0x0 0x14000000 0x0 0x04000000>;
> -               };
> -
> -               intc: interrupt-controller at 10000000 {
> -                       #interrupt-cells = <1>;
> -                       compatible = "allwinner,sun20i-d1-plic",
> -                                    "thead,c900-plic";
> -                       interrupt-controller;
> -                       interrupts-extended = <
> -                               &cpu0_intc  0xffffffff &cpu0_intc  9
> -                               >;
> -                       reg = <0x0 0x10000000 0x0 0x04000000>;
> -                       reg-names = "control";
> -                       riscv,max-priority = <7>;
> -                       riscv,ndev = <200>;
> -               };
> -       }
> -```
> -
> -DTS Example2: (Multi cores with soc reset-regs)
> ------------------------------------------------
> -
> -```
> -       cpus {
> -               #address-cells = <1>;
> -               #size-cells = <0>;
> -               timebase-frequency = <3000000>;
> -               cpu at 0 {
> -                       device_type = "cpu";
> -                       reg = <0>;
> -                       status = "okay";
> -                       compatible = "riscv";
> -                       riscv,isa = "rv64imafdc";
> -                       mmu-type = "riscv,sv39";
> -                       cpu0_intc: interrupt-controller {
> -                               #interrupt-cells = <1>;
> -                               compatible = "riscv,cpu-intc";
> -                               interrupt-controller;
> -                       };
> -               };
> -               cpu at 1 {
> -                       device_type = "cpu";
> -                       reg = <1>;
> -                       status = "fail";
> -                       compatible = "riscv";
> -                       riscv,isa = "rv64imafdc";
> -                       mmu-type = "riscv,sv39";
> -                       cpu1_intc: interrupt-controller {
> -                               #interrupt-cells = <1>;
> -                               compatible = "riscv,cpu-intc";
> -                               interrupt-controller;
> -                       };
> -               };
> -               cpu at 2 {
> -                       device_type = "cpu";
> -                       reg = <2>;
> -                       status = "fail";
> -                       compatible = "riscv";
> -                       riscv,isa = "rv64imafdc";
> -                       mmu-type = "riscv,sv39";
> -                       cpu2_intc: interrupt-controller {
> -                               #interrupt-cells = <1>;
> -                               compatible = "riscv,cpu-intc";
> -                               interrupt-controller;
> -                       };
> -               };
> -               cpu at 3 {
> -                       device_type = "cpu";
> -                       reg = <3>;
> -                       status = "fail";
> -                       compatible = "riscv";
> -                       riscv,isa = "rv64imafdc";
> -                       mmu-type = "riscv,sv39";
> -                       cpu3_intc: interrupt-controller {
> -                               #interrupt-cells = <1>;
> -                               compatible = "riscv,cpu-intc";
> -                               interrupt-controller;
> -                       };
> -               };
> -       };
> -
> -       soc {
> -               #address-cells = <2>;
> -               #size-cells = <2>;
> -               compatible = "simple-bus";
> -               ranges;
> -
> -               reset: reset-sample {
> -                       compatible = "thead,reset-sample";
> -                       entry-reg = <0xff 0xff019050>;
> -                       entry-cnt = <4>;
> -                       control-reg = <0xff 0xff015004>;
> -                       control-val = <0x1c>;
> -                       csr-copy = <0x7f3 0x7c0 0x7c1 0x7c2 0x7c3 0x7c5 0x7cc>;
> -               };
> -
> -               clint0: clint at ffdc000000 {
> -                       compatible = "riscv,clint0";
> -                       interrupts-extended = <
> -                               &cpu0_intc  3 &cpu0_intc  7
> -                               &cpu1_intc  3 &cpu1_intc  7
> -                               &cpu2_intc  3 &cpu2_intc  7
> -                               &cpu3_intc  3 &cpu3_intc  7
> -                               &cpu4_intc  3 &cpu4_intc  7
> -                               >;
> -                       reg = <0xff 0xdc000000 0x0 0x04000000>;
> -               };
> -
> -               intc: interrupt-controller at ffd8000000 {
> -                       #interrupt-cells = <1>;
> -                       compatible = "thead,c900-plic";
> -                       interrupt-controller;
> -                       interrupts-extended = <
> -                               &cpu0_intc  0xffffffff &cpu0_intc  9
> -                               &cpu1_intc  0xffffffff &cpu1_intc  9
> -                               &cpu2_intc  0xffffffff &cpu2_intc  9
> -                               &cpu3_intc  0xffffffff &cpu3_intc  9
> -                               >;
> -                       reg = <0xff 0xd8000000 0x0 0x04000000>;
> -                       reg-names = "control";
> -                       riscv,max-priority = <7>;
> -                       riscv,ndev = <80>;
> -               };
> -       }
> -```
> -
> -DTS Example2: (Multi cores with old reset csrs)
> ------------------------------------------------
> -```
> -       reset: reset-sample {
> -               compatible = "thead,reset-sample";
> -               using-csr-reset;
> -               csr-copy = <0x7c0 0x7c1 0x7c2 0x7c3 0x7c5 0x7cc
> -                           0x3b0 0x3b1 0x3b2 0x3b3
> -                           0x3b4 0x3b5 0x3b6 0x3b7
> -                           0x3a0>;
> -       };
> -```
> +For more details, refer:
> + [zero stage boot](https://github.com/c-sky/zero_stage_boot)
> diff --git a/lib/utils/reset/Kconfig b/lib/utils/reset/Kconfig
> index 23f50c4dfe40..b26260db1678 100644
> --- a/lib/utils/reset/Kconfig
> +++ b/lib/utils/reset/Kconfig
> @@ -33,10 +33,6 @@ config FDT_RESET_SYSCON
>         depends on FDT_REGMAP
>         default n
>
> -config FDT_RESET_THEAD
> -       bool "T-HEAD FDT reset driver"
> -       default n
> -
>  endif
>
>  endmenu
> diff --git a/lib/utils/reset/fdt_reset_thead.c b/lib/utils/reset/fdt_reset_thead.c
> deleted file mode 100644
> index e1d6885debae..000000000000
> --- a/lib/utils/reset/fdt_reset_thead.c
> +++ /dev/null
> @@ -1,134 +0,0 @@
> -/*
> - * SPDX-License-Identifier: BSD-2-Clause
> - */
> -
> -#include <libfdt.h>
> -#include <sbi/riscv_io.h>
> -#include <sbi/sbi_bitops.h>
> -#include <sbi/sbi_hart.h>
> -#include <sbi/sbi_scratch.h>
> -#include <sbi/sbi_system.h>
> -#include <sbi_utils/fdt/fdt_helper.h>
> -#include <sbi_utils/reset/fdt_reset.h>
> -
> -#include "fdt_reset_thead.h"
> -
> -struct custom_csr custom_csr[MAX_CUSTOM_CSR];
> -
> -#define CSR_OPCODE 0x39073
> -static void clone_csrs(int cnt)
> -{
> -       unsigned long i;
> -
> -       for (i = 0; i < cnt; i++) {
> -               /* Write csr BIT[31 - 20] to stub */
> -               __reset_thead_csr_stub[3*i + 1] =
> -                               CSR_OPCODE | (custom_csr[i].index << 20);
> -
> -               /* Mask csr BIT[31 - 20] */
> -               *(u32 *)&__fdt_reset_thead_csrr &= BIT(20) - 1;
> -               smp_mb();
> -
> -               /* Write csr BIT[31 - 20] to __fdt_reset_thead_csrr */
> -               *(u32 *)&__fdt_reset_thead_csrr |= custom_csr[i].index << 20;
> -               smp_mb();
> -
> -               RISCV_FENCE_I;
> -
> -               custom_csr[i].value = __fdt_reset_thead_csrr();
> -       }
> -}
> -
> -static int thead_system_reset_check(u32 type, u32 reason)
> -{
> -       return 1;
> -}
> -
> -static void thead_system_reset(u32 type, u32 reason)
> -{
> -       ebreak();
> -}
> -
> -static struct sbi_system_reset_device thead_reset = {
> -       .name = "thead_reset",
> -       .system_reset_check = thead_system_reset_check,
> -       .system_reset = thead_system_reset
> -};
> -
> -extern void __thead_pre_start_warm(void);
> -static int thead_reset_init(void *fdt, int nodeoff,
> -                                const struct fdt_match *match)
> -{
> -       char *p;
> -       const fdt64_t *val;
> -       const fdt32_t *val_w;
> -       int len, i;
> -       u32 t, tmp = 0;
> -
> -       /* Prepare clone csrs */
> -       val_w = fdt_getprop(fdt, nodeoff, "csr-copy", &len);
> -       if (len > 0 && val_w) {
> -               int cnt;
> -
> -               cnt = len / sizeof(fdt32_t);
> -               if (cnt > MAX_CUSTOM_CSR)
> -                       sbi_hart_hang();
> -
> -               for (i = 0; i < cnt; i++) {
> -                       custom_csr[i].index = fdt32_to_cpu(val_w[i]);
> -               }
> -
> -               if (cnt)
> -                       clone_csrs(cnt);
> -       }
> -
> -       /* Old reset method for secondary harts */
> -       if (fdt_getprop(fdt, nodeoff, "using-csr-reset", &len)) {
> -               csr_write(0x7c7, (ulong)&__thead_pre_start_warm);
> -               csr_write(0x7c6, -1);
> -       }
> -
> -       /* Custom reset method for secondary harts */
> -       val = fdt_getprop(fdt, nodeoff, "entry-reg", &len);
> -       if (len > 0 && val) {
> -          p = (char *)(ulong)fdt64_to_cpu(*val);
> -
> -               val_w = fdt_getprop(fdt, nodeoff, "entry-cnt", &len);
> -               if (len > 0 && val_w) {
> -                       tmp = fdt32_to_cpu(*val_w);
> -
> -                       for (i = 0; i < tmp; i++) {
> -                               t = (ulong)&__thead_pre_start_warm;
> -                               writel(t, p + (8 * i));
> -                               t = (u64)(ulong)&__thead_pre_start_warm >> 32;
> -                               writel(t, p + (8 * i) + 4);
> -                       }
> -               }
> -
> -               val = fdt_getprop(fdt, nodeoff, "control-reg", &len);
> -               if (len > 0 && val) {
> -                       p = (void *)(ulong)fdt64_to_cpu(*val);
> -
> -                       val_w = fdt_getprop(fdt, nodeoff, "control-val", &len);
> -                       if (len > 0 && val_w) {
> -                               tmp = fdt32_to_cpu(*val_w);
> -                               tmp |= readl(p);
> -                               writel(tmp, p);
> -                       }
> -               }
> -       }
> -
> -       sbi_system_reset_add_device(&thead_reset);
> -
> -       return 0;
> -}
> -
> -static const struct fdt_match thead_reset_match[] = {
> -       { .compatible = "thead,reset-sample" },
> -       { },
> -};
> -
> -struct fdt_reset fdt_reset_thead = {
> -       .match_table = thead_reset_match,
> -       .init = thead_reset_init
> -};
> diff --git a/lib/utils/reset/fdt_reset_thead.h b/lib/utils/reset/fdt_reset_thead.h
> deleted file mode 100644
> index d7a686ec9317..000000000000
> --- a/lib/utils/reset/fdt_reset_thead.h
> +++ /dev/null
> @@ -1,23 +0,0 @@
> -/*
> - * SPDX-License-Identifier: BSD-2-Clause
> - */
> -
> -#ifndef __FDT_RESET_THEAD_H__
> -#define __FDT_RESET_THEAD_H__
> -
> -#define MAX_CUSTOM_CSR 32
> -
> -#ifndef __ASSEMBLER__
> -struct custom_csr {
> -       unsigned long index;
> -       unsigned long value;
> -};
> -
> -u64  __fdt_reset_thead_csrr(void);
> -
> -extern struct custom_csr custom_csr[MAX_CUSTOM_CSR];
> -extern u32 __reset_thead_csr_stub[];
> -
> -#endif /* __ASSEMBLER__ */
> -
> -#endif /* __FDT_RESET_THEAD_H__ */
> diff --git a/lib/utils/reset/fdt_reset_thead_asm.S b/lib/utils/reset/fdt_reset_thead_asm.S
> deleted file mode 100644
> index 8237951fd82d..000000000000
> --- a/lib/utils/reset/fdt_reset_thead_asm.S
> +++ /dev/null
> @@ -1,47 +0,0 @@
> -/*
> - * SPDX-License-Identifier: BSD-2-Clause
> - */
> -
> -#include <sbi/riscv_asm.h>
> -#include "fdt_reset_thead.h"
> -
> -/*
> - * csrrs rd, csr, rs1
> - * |31   20|19   15|14   12|11  7|6       0|
> - *    csr     rs1     010     rd   1110011
> - */
> -#define CSR_STUB       addi    x0, x0, 0
> -
> -       .option norvc
> -       .align 3
> -       .global __fdt_reset_thead_csrr
> -__fdt_reset_thead_csrr:
> -       csrrs a0, 0, x0
> -       ret
> -
> -       .align 3
> -       .global __thead_pre_start_warm
> -__thead_pre_start_warm:
> -       /*
> -        * Clear L1 cache & BTB & BHT ...
> -        */
> -       li      t1, 0x70013
> -       csrw    0x7c2, t1
> -       fence rw,rw
> -
> -       lla     t1, custom_csr
> -
> -       .global __reset_thead_csr_stub
> -__reset_thead_csr_stub:
> -.rept  MAX_CUSTOM_CSR
> -       REG_L   t2, 8(t1)
> -       CSR_STUB
> -       addi    t1, t1, 16
> -.endr
> -       /*
> -        * Clear L1 cache & BTB & BHT ...
> -        */
> -       li      t1, 0x70013
> -       csrw    0x7c2, t1
> -       fence rw,rw
> -       j _start_warm
> diff --git a/lib/utils/reset/objects.mk b/lib/utils/reset/objects.mk
> index ea9cedcd8002..b5222347b213 100644
> --- a/lib/utils/reset/objects.mk
> +++ b/lib/utils/reset/objects.mk
> @@ -26,7 +26,3 @@ libsbiutils-objs-$(CONFIG_FDT_RESET_SUNXI_WDT) += reset/fdt_reset_sunxi_wdt.o
>  carray-fdt_reset_drivers-$(CONFIG_FDT_RESET_SYSCON) += fdt_syscon_poweroff
>  carray-fdt_reset_drivers-$(CONFIG_FDT_RESET_SYSCON) += fdt_syscon_reboot
>  libsbiutils-objs-$(CONFIG_FDT_RESET_SYSCON) += reset/fdt_reset_syscon.o
> -
> -carray-fdt_reset_drivers-$(CONFIG_FDT_RESET_THEAD) += fdt_reset_thead
> -libsbiutils-objs-$(CONFIG_FDT_RESET_THEAD) += reset/fdt_reset_thead.o
> -libsbiutils-objs-$(CONFIG_FDT_RESET_THEAD) += reset/fdt_reset_thead_asm.o
> diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
> index c432bc2f8936..95bab61ea5d9 100644
> --- a/platform/generic/configs/defconfig
> +++ b/platform/generic/configs/defconfig
> @@ -27,7 +27,6 @@ CONFIG_FDT_RESET_GPIO=y
>  CONFIG_FDT_RESET_HTIF=y
>  CONFIG_FDT_RESET_SUNXI_WDT=y
>  CONFIG_FDT_RESET_SYSCON=y
> -CONFIG_FDT_RESET_THEAD=y
>  CONFIG_FDT_SERIAL=y
>  CONFIG_FDT_SERIAL_CADENCE=y
>  CONFIG_FDT_SERIAL_GAISLER=y
> --
> 2.36.1
>


From ganboing at gmail.com  Thu Nov 16 03:15:54 2023
From: ganboing at gmail.com (Bo Gan)
Date: Thu, 16 Nov 2023 03:15:54 -0800
Subject: Question about potential missed IPI events
In-Reply-To: <3e26f02e75e820d88ce98098d879975a8a4657a6.camel@126.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
 <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
 <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>
 <af55316c-3647-f720-4000-86639e23a4d3@gmail.com>
 <ec3855f1c0a00c8fe271bf4d170ae33eab55669c.camel@126.com>
 <3113ff3f-f3fa-67e7-3b04-ce775eed5891@gmail.com>
 <3e26f02e75e820d88ce98098d879975a8a4657a6.camel@126.com>
Message-ID: <e67d2532-d6f5-fa7e-4b2b-693d8c2c2217@gmail.com>

On 11/16/23 2:11 AM, Xiang W wrote:
> ? 2023-11-16???? 01:44 -0800?Bo Gan???
>> On 11/16/23 1:19 AM, Xiang W wrote:
>>> ? 2023-11-16???? 00:41 -0800?Bo Gan???
>>>>
>>>> Hi Xiang, Thank you so much for replying. I'd like to revise your chart a little bit:
>>>>
>>>> Case 1, xchg_ulong in `ipi_process` gets reordered before ipi_clear:
>>>>
>>>>  ??? B???????????????? A??????????????? C???????????????? CLINT
>>>>  ??? |???????????????? |??????????????? |????????????????? |
>>>>  ??? | bit_set???????? |??????????????? |????????????????? |
>>>>  ??? | send -----------|----------------|----------------> |
>>>>  ??? |???????????????? | xchg?????????? |????????????????? |
>>>>  ??? |???????????????? |??????????????? | bit_set????????? |
>>>>  ??? |???????????????? |??????????????? | send ----------> |
>>>>  ??? |???????????????? | clear----------|----------------> |
>>>>
>>>> In this case, A would observe the ipi_data->ipi_type *eventually*, but when?
>>>> A won't process C's IPI request until another IPI comes in the future, which
>>>> might cause the IPI request to wait indefinitely. This is not an efficiency
>>>> problem, but a correctness problem. I think the fix should be adding specific
>>>> fence (need to reason about the correct one) between `ipi_dev->ipi_clear` and
>>>> `atomic_raw_xchg_ulong` in `ipi_process`
>>> Your statement is more correct.
>>>
>>
>> For case 1, can we agree there's an issue in the current code, and it needs to
>> be fixed?
> Yes!

Great, and let's fix it!

>>
>>>>
>>>>
>>>> Case 2, xchg_ulong doesn't get reordered, however:
>>>>
>>>>  ??? B???????????????? A??????????????? C???????????????? CLINT
>>>>  ??? |???????????????? |??????????????? |????????????????? |
>>>>  ??? | bit_set???????? |??????????????? |????????????????? |
>>>>  ??? | send -----------|----------------|----------------> |
>>>>  ??? |???????????????? | xchg?????????? |??????????? /---> |
>>>>  ??? |???????????????? | clear----------|-----------/----> |
>>>>  ??? |???????????????? |??????????????? | bit_set? /?????? |
>>>>  ??? |???????????????? |??????????????? | send ----??????? |
>>> Why is there an upward-sloping arrow in your picture? Multiple cores
>>> observing atomic operations at the same address should not be
>>> inconsistent.
>>>
>>
>> Let me clarify case 2 a little bit more. Let me use a downward-sloping
>> arrow. In this case the clear sent from A took so long that it arrived
>> at CLINT after C's send.
>>
>>  ???? B???????????????? A??????????????? C???????????????? CLINT
>>  ???? |???????????????? |??????????????? |????????????????? |
>>  ???? | bit_set ------> |??????????????? |????????????????? |
>>  ???? | send -----------|----------------|----------------> |
>>  ???? |???????????????? | xchg?????????? |????????????????? |
>>  ???? |???????????????? | clear----------|----------------> |
>>  ???? |???????????????? |????????? \???? |????????????????? |
>>  ???? |???????????????? | <---------\----|-bit_set????????? |
>>  ???? |???????????????? |??????????? \?? | send ----------> |
>>  ???? |???????????????? |???????????? \? |????????????????? |
>>  ???? |???????????????? |????????????? --|----------------> |
>>                                   
>> Back to your question. Here we have two memory/IO locations, not one.
>> First is the `ipi_data->ipi_type`, second is the msip MMIO for hart A.
>> Essentially, can the following happen?
>>
>>  ? i.? Assume there's proper fence to ensure xchg orders before clear in A.
>>
>>  ? ii. From A's point of view, the xchg observes bit_set from B,
>>  ????? but not bit_set from C.
>>
>>  ? iii.From CLINT's point of view, it observes send from B, then send from C,
>>  ????? then clear from A.
>>
>> I don't know how to reason about this. The only thing I can think of is the
>> global memory order defined by RVWMO. Can we apply the global memory order
>> to the interaction between hart and clint?
> 
> I understand what you mean. The one I proposed is caused by inconsistent core
> speeds and has nothing to do with memory order. Do you think the memory order
> observed by hart and clint is inconsistent?
> 
> How to solve this? fence should only restrict memory ordering on the same core.
> 
> Regards,
> Xiang W

Yes, my issue is about memory ordering, and yes, I'm questioning the global
memory order observed by hart and clint. What kind of rules do they follow?
Where are the rules defined? In RISCV ISA? Or Sifive manual? If this is not
possible, I'm happy to be lectured on the reasoning.

Bo



From wxjstz at 126.com  Thu Nov 16 03:29:03 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 16 Nov 2023 19:29:03 +0800
Subject: Question about potential missed IPI events
In-Reply-To: <e67d2532-d6f5-fa7e-4b2b-693d8c2c2217@gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
	 <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
	 <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>
	 <af55316c-3647-f720-4000-86639e23a4d3@gmail.com>
	 <ec3855f1c0a00c8fe271bf4d170ae33eab55669c.camel@126.com>
	 <3113ff3f-f3fa-67e7-3b04-ce775eed5891@gmail.com>
	 <3e26f02e75e820d88ce98098d879975a8a4657a6.camel@126.com>
	 <e67d2532-d6f5-fa7e-4b2b-693d8c2c2217@gmail.com>
Message-ID: <262cd13598de718b33683e0da31931006f10cdbb.camel@126.com>

? 2023-11-16???? 03:15 -0800?Bo Gan???
> On 11/16/23 2:11 AM, Xiang W wrote:
> > ? 2023-11-16???? 01:44 -0800?Bo Gan???
> > > On 11/16/23 1:19 AM, Xiang W wrote:
> > > > ? 2023-11-16???? 00:41 -0800?Bo Gan???
> > > > > 
> > > > > Hi Xiang, Thank you so much for replying. I'd like to revise your chart a little bit:
> > > > > 
> > > > > Case 1, xchg_ulong in `ipi_process` gets reordered before ipi_clear:
> > > > > 
> > > > > ???? B???????????????? A??????????????? C???????????????? CLINT
> > > > > ???? |???????????????? |??????????????? |????????????????? |
> > > > > ???? | bit_set???????? |??????????????? |????????????????? |
> > > > > ???? | send -----------|----------------|----------------> |
> > > > > ???? |???????????????? | xchg?????????? |????????????????? |
> > > > > ???? |???????????????? |??????????????? | bit_set????????? |
> > > > > ???? |???????????????? |??????????????? | send ----------> |
> > > > > ???? |???????????????? | clear----------|----------------> |
> > > > > 
> > > > > In this case, A would observe the ipi_data->ipi_type *eventually*, but when?
> > > > > A won't process C's IPI request until another IPI comes in the future, which
> > > > > might cause the IPI request to wait indefinitely. This is not an efficiency
> > > > > problem, but a correctness problem. I think the fix should be adding specific
> > > > > fence (need to reason about the correct one) between `ipi_dev->ipi_clear` and
> > > > > `atomic_raw_xchg_ulong` in `ipi_process`
> > > > Your statement is more correct.
> > > > 
> > > 
> > > For case 1, can we agree there's an issue in the current code, and it needs to
> > > be fixed?
> > Yes!
> 
> Great, and let's fix it!

I do not think fence can be help. We need to use spinlock to enter the critical
section to protect bit_set/send and clear/xchg.

Regards,
Xiang W
> 
> > > 
> > > > > 
> > > > > 
> > > > > Case 2, xchg_ulong doesn't get reordered, however:
> > > > > 
> > > > > ???? B???????????????? A??????????????? C???????????????? CLINT
> > > > > ???? |???????????????? |??????????????? |????????????????? |
> > > > > ???? | bit_set???????? |??????????????? |????????????????? |
> > > > > ???? | send -----------|----------------|----------------> |
> > > > > ???? |???????????????? | xchg?????????? |??????????? /---> |
> > > > > ???? |???????????????? | clear----------|-----------/----> |
> > > > > ???? |???????????????? |??????????????? | bit_set? /?????? |
> > > > > ???? |???????????????? |??????????????? | send ----??????? |
> > > > Why is there an upward-sloping arrow in your picture? Multiple cores
> > > > observing atomic operations at the same address should not be
> > > > inconsistent.
> > > > 
> > > 
> > > Let me clarify case 2 a little bit more. Let me use a downward-sloping
> > > arrow. In this case the clear sent from A took so long that it arrived
> > > at CLINT after C's send.
> > > 
> > > ????? B???????????????? A??????????????? C???????????????? CLINT
> > > ????? |???????????????? |??????????????? |????????????????? |
> > > ????? | bit_set ------> |??????????????? |????????????????? |
> > > ????? | send -----------|----------------|----------------> |
> > > ????? |???????????????? | xchg?????????? |????????????????? |
> > > ????? |???????????????? | clear----------|----------------> |
> > > ????? |???????????????? |????????? \???? |????????????????? |
> > > ????? |???????????????? | <---------\----|-bit_set????????? |
> > > ????? |???????????????? |??????????? \?? | send ----------> |
> > > ????? |???????????????? |???????????? \? |????????????????? |
> > > ????? |???????????????? |????????????? --|----------------> |
> > > ????????????????????????????????? 
> > > Back to your question. Here we have two memory/IO locations, not one.
> > > First is the `ipi_data->ipi_type`, second is the msip MMIO for hart A.
> > > Essentially, can the following happen?
> > > 
> > > ?? i.? Assume there's proper fence to ensure xchg orders before clear in A.
> > > 
> > > ?? ii. From A's point of view, the xchg observes bit_set from B,
> > > ?????? but not bit_set from C.
> > > 
> > > ?? iii.From CLINT's point of view, it observes send from B, then send from C,
> > > ?????? then clear from A.
> > > 
> > > I don't know how to reason about this. The only thing I can think of is the
> > > global memory order defined by RVWMO. Can we apply the global memory order
> > > to the interaction between hart and clint?
> > 
> > I understand what you mean. The one I proposed is caused by inconsistent core
> > speeds and has nothing to do with memory order. Do you think the memory order
> > observed by hart and clint is inconsistent?
> > 
> > How to solve this? fence should only restrict memory ordering on the same core.
> > 
> > Regards,
> > Xiang W
> 
> Yes, my issue is about memory ordering, and yes, I'm questioning the global
> memory order observed by hart and clint. What kind of rules do they follow?
> Where are the rules defined? In RISCV ISA? Or Sifive manual? If this is not
> possible, I'm happy to be lectured on the reasoning.
> 
> Bo
> 
> 


From anup at brainfault.org  Thu Nov 16 03:29:42 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 16:59:42 +0530
Subject: [PATCH v3 0/3] lib: utils: Add T-HEAD C900 ACLINT
In-Reply-To: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB495397CC87AD8A595B45131CBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy0rc4iQ4wV74prW2DOvrKV6kKW+8SGsYWDEYTp2R9y6pw@mail.gmail.com>

On Thu, Nov 16, 2023 at 2:35?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Add T-HEAD C900 ACLINT driver for sg2042
>
> This patch need a new "reg-names" based timer register parsing to get
> worked register. But this patch can be tested by manually setting regs
> in the DT and temporarily setting without_mtimer quirks.
>
> Link: https://lore.kernel.org/linux-riscv/6e48cbe5e60f9ada2fd1fe58e803e127f1a678e5.1696433229.git.unicorn_wang at outlook.com/
> Link: https://lore.kernel.org/linux-riscv/1f6b82a1864477a51db33d3f295889ff985b497b.1696433229.git.unicorn_wang at outlook.com/
>
> Changed from v2:
> 1. use new quirk to identify custom ACLINT
>
> Changed from v1:
> 1. function fix and comment cleanup.
>
> Changed from sg2042 v2:
> 1. Allow ACLINT has its own quirks
> 2. add new quirks for thead,c900-aclint-mtimer
>
> This patch is moved from the patch series of supporting sg2042 as it
> became complex.
> The original patch: platform: generic: Add Sophgo sg2042 support
>
> Inochi Amaoto (3):
>   lib: utils/timer: Allow ACLINT MTIMER driver to setup quirks
>   lib: utils/timer: mtimer: add separate T-Head C9xx CLINT mtimer
>     compatible
>   lib: utils/ipi: mswi: add separate T-Head C9xx CLINT mswi compatible

The "custom_aclint" flag in quirks was looking odd so I renamed it
to "is_clint" flag and did relevant changes at the time of merging this
series.

Applied this series to the riscv/opensbi repo.

Thanks,
Anup

>
>  lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
>  lib/utils/timer/fdt_timer_mtimer.c | 20 +++++++++++++++-----
>  2 files changed, 16 insertions(+), 5 deletions(-)
>
> --
> 2.42.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From wxjstz at 126.com  Thu Nov 16 03:46:44 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 16 Nov 2023 19:46:44 +0800
Subject: Question about potential missed IPI events
In-Reply-To: <e67d2532-d6f5-fa7e-4b2b-693d8c2c2217@gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
	 <d6ff141b-1397-59c2-64cc-b19a36babc40@gmail.com>
	 <60187592b87d94a99cf33bf7099eb628eae3c7c9.camel@126.com>
	 <af55316c-3647-f720-4000-86639e23a4d3@gmail.com>
	 <ec3855f1c0a00c8fe271bf4d170ae33eab55669c.camel@126.com>
	 <3113ff3f-f3fa-67e7-3b04-ce775eed5891@gmail.com>
	 <3e26f02e75e820d88ce98098d879975a8a4657a6.camel@126.com>
	 <e67d2532-d6f5-fa7e-4b2b-693d8c2c2217@gmail.com>
Message-ID: <fcf6a29a1ddbbc7ffeb1eee4c365eba218eb7464.camel@126.com>

? 2023-11-16???? 03:15 -0800?Bo Gan???
> On 11/16/23 2:11 AM, Xiang W wrote:
> > ? 2023-11-16???? 01:44 -0800?Bo Gan???
> > > On 11/16/23 1:19 AM, Xiang W wrote:
> > > > ? 2023-11-16???? 00:41 -0800?Bo Gan???
> > > > > 
> > > > > Hi Xiang, Thank you so much for replying. I'd like to revise your chart a little bit:
> > > > > 
> > > > > Case 1, xchg_ulong in `ipi_process` gets reordered before ipi_clear:
> > > > > 
> > > > > ???? B???????????????? A??????????????? C???????????????? CLINT
> > > > > ???? |???????????????? |??????????????? |????????????????? |
> > > > > ???? | bit_set???????? |??????????????? |????????????????? |
> > > > > ???? | send -----------|----------------|----------------> |
> > > > > ???? |???????????????? | xchg?????????? |????????????????? |
> > > > > ???? |???????????????? |??????????????? | bit_set????????? |
> > > > > ???? |???????????????? |??????????????? | send ----------> |
> > > > > ???? |???????????????? | clear----------|----------------> |
> > > > > 
> > > > > In this case, A would observe the ipi_data->ipi_type *eventually*, but when?
> > > > > A won't process C's IPI request until another IPI comes in the future, which
> > > > > might cause the IPI request to wait indefinitely. This is not an efficiency
> > > > > problem, but a correctness problem. I think the fix should be adding specific
> > > > > fence (need to reason about the correct one) between `ipi_dev->ipi_clear` and
> > > > > `atomic_raw_xchg_ulong` in `ipi_process`
> > > > Your statement is more correct.
> > > > 
> > > 
> > > For case 1, can we agree there's an issue in the current code, and it needs to
> > > be fixed?
> > Yes!
> 
> Great, and let's fix it!
> 
> > > 
> > > > > 
> > > > > 
> > > > > Case 2, xchg_ulong doesn't get reordered, however:
> > > > > 
> > > > > ???? B???????????????? A??????????????? C???????????????? CLINT
> > > > > ???? |???????????????? |??????????????? |????????????????? |
> > > > > ???? | bit_set???????? |??????????????? |????????????????? |
> > > > > ???? | send -----------|----------------|----------------> |
> > > > > ???? |???????????????? | xchg?????????? |??????????? /---> |
> > > > > ???? |???????????????? | clear----------|-----------/----> |
> > > > > ???? |???????????????? |??????????????? | bit_set? /?????? |
> > > > > ???? |???????????????? |??????????????? | send ----??????? |
> > > > Why is there an upward-sloping arrow in your picture? Multiple cores
> > > > observing atomic operations at the same address should not be
> > > > inconsistent.
> > > > 
> > > 
> > > Let me clarify case 2 a little bit more. Let me use a downward-sloping
> > > arrow. In this case the clear sent from A took so long that it arrived
> > > at CLINT after C's send.
> > > 
> > > ????? B???????????????? A??????????????? C???????????????? CLINT
> > > ????? |???????????????? |??????????????? |????????????????? |
> > > ????? | bit_set ------> |??????????????? |????????????????? |
> > > ????? | send -----------|----------------|----------------> |
> > > ????? |???????????????? | xchg?????????? |????????????????? |
> > > ????? |???????????????? | clear----------|----------------> |
> > > ????? |???????????????? |????????? \???? |????????????????? |
> > > ????? |???????????????? | <---------\----|-bit_set????????? |
> > > ????? |???????????????? |??????????? \?? | send ----------> |
> > > ????? |???????????????? |???????????? \? |????????????????? |
> > > ????? |???????????????? |????????????? --|----------------> |
> > > ????????????????????????????????? 
> > > Back to your question. Here we have two memory/IO locations, not one.
> > > First is the `ipi_data->ipi_type`, second is the msip MMIO for hart A.
> > > Essentially, can the following happen?
> > > 
> > > ?? i.? Assume there's proper fence to ensure xchg orders before clear in A.
> > > 
> > > ?? ii. From A's point of view, the xchg observes bit_set from B,
> > > ?????? but not bit_set from C.
> > > 
> > > ?? iii.From CLINT's point of view, it observes send from B, then send from C,
> > > ?????? then clear from A.
> > > 
> > > I don't know how to reason about this. The only thing I can think of is the
> > > global memory order defined by RVWMO. Can we apply the global memory order
> > > to the interaction between hart and clint?
It is not normal for the peripheral to observe a reorder. This is obvious on a
single hart because many operations have timing requirements. This should also
be true on multiple harts, otherwise an error will occur when a device is shared
by multiple harts.

Regards,
Xiang W
> > 
> > I understand what you mean. The one I proposed is caused by inconsistent core
> > speeds and has nothing to do with memory order. Do you think the memory order
> > observed by hart and clint is inconsistent?
> > 
> > How to solve this? fence should only restrict memory ordering on the same core.
> > 
> > Regards,
> > Xiang W
> 
> Yes, my issue is about memory ordering, and yes, I'm questioning the global
> memory order observed by hart and clint. What kind of rules do they follow?
> Where are the rules defined? In RISCV ISA? Or Sifive manual? If this is not
> possible, I'm happy to be lectured on the reasoning.
> 
> Bo
> 
> 


From inochiama at outlook.com  Thu Nov 16 04:35:01 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 16 Nov 2023 20:35:01 +0800
Subject: [PATCH v3 0/3] lib: utils: Add T-HEAD C900 ACLINT
In-Reply-To: <CAAhSdy0rc4iQ4wV74prW2DOvrKV6kKW+8SGsYWDEYTp2R9y6pw@mail.gmail.com>
References: <CAAhSdy0rc4iQ4wV74prW2DOvrKV6kKW+8SGsYWDEYTp2R9y6pw@mail.gmail.com>
Message-ID: <IA1PR20MB49533DAA3BEB25CFF1F6AABCBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>

>
>
>On Thu, Nov 16, 2023 at 2:35?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> Add T-HEAD C900 ACLINT driver for sg2042
>>
>> This patch need a new "reg-names" based timer register parsing to get
>> worked register. But this patch can be tested by manually setting regs
>> in the DT and temporarily setting without_mtimer quirks.
>>
>> Link: https://lore.kernel.org/linux-riscv/6e48cbe5e60f9ada2fd1fe58e803e127f1a678e5.1696433229.git.unicorn_wang at outlook.com/
>> Link: https://lore.kernel.org/linux-riscv/1f6b82a1864477a51db33d3f295889ff985b497b.1696433229.git.unicorn_wang at outlook.com/
>>
>> Changed from v2:
>> 1. use new quirk to identify custom ACLINT
>>
>> Changed from v1:
>> 1. function fix and comment cleanup.
>>
>> Changed from sg2042 v2:
>> 1. Allow ACLINT has its own quirks
>> 2. add new quirks for thead,c900-aclint-mtimer
>>
>> This patch is moved from the patch series of supporting sg2042 as it
>> became complex.
>> The original patch: platform: generic: Add Sophgo sg2042 support
>>
>> Inochi Amaoto (3):
>>   lib: utils/timer: Allow ACLINT MTIMER driver to setup quirks
>>   lib: utils/timer: mtimer: add separate T-Head C9xx CLINT mtimer
>>     compatible
>>   lib: utils/ipi: mswi: add separate T-Head C9xx CLINT mswi compatible
>
>The "custom_aclint" flag in quirks was looking odd so I renamed it
>to "is_clint" flag and did relevant changes at the time of merging this
>series.
>

Thanks.

>Applied this series to the riscv/opensbi repo.
>
>Thanks,
>Anup
>
>>
>>  lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
>>  lib/utils/timer/fdt_timer_mtimer.c | 20 +++++++++++++++-----
>>  2 files changed, 16 insertions(+), 5 deletions(-)
>>
>> --
>> 2.42.1
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi
>


From samuel.holland at sifive.com  Thu Nov 16 04:35:52 2023
From: samuel.holland at sifive.com (Samuel Holland)
Date: Thu, 16 Nov 2023 07:35:52 -0500
Subject: [PATCH] lib: sbi: Correctly limit flushes to a single ASID/VMID
In-Reply-To: <CAAhSdy1tW2YNQzkDqsRWhSw0J7Vfg9DX5hSHarnw246QGL3Zrg@mail.gmail.com>
References: <20231017222027.3668468-1-samuel.holland@sifive.com>
 <CAAhSdy1tW2YNQzkDqsRWhSw0J7Vfg9DX5hSHarnw246QGL3Zrg@mail.gmail.com>
Message-ID: <24cd0349-b63a-46ce-906e-2571c402c87d@sifive.com>

Hi Anup,

On 2023-11-15 10:03 PM, Anup Patel wrote:
> On Wed, Oct 18, 2023 at 3:50?AM Samuel Holland
> <samuel.holland at sifive.com> wrote:
>>
>> Per the SBI specification, the effects of these functions are limited to
>> a specific ASID and/or VMID. This applies even when flushing the entire
>> address space.
>>
>> Signed-off-by: Samuel Holland <samuel.holland at sifive.com>
> 
> This patch looks good to me.
> 
> Reviewed-by: Anup Patel <anup at brainfault.org>
> 
> Applied this patch to the riscv/opensbi repo.
> 
>> ---
>> The SBI spec doesn't say anything about this `start == 0 && size == 0`
>> exception, so I wonder if it should be removed entirely.
> 
> The SBI spec does define behaviour for `start == 0 && size == 0`.
> 
> At the beginning of "Chapter 8. RFENCE Extension" we have the
> following text which describes this behavior:
> 
> "
> The remote fence function acts as a full TLB flush if
> * start_addr and size are both 0
> * size is equal to 2^XLEN-1
> "

Thanks! I didn't see this paragraph in the spec. That explains the previous
implementation (even though I still think it was wrong): "acts as a full TLB
flush" could be interpreted to imply ignoring the ASID.

I would suggest updating the SBI documentation with more precise wording,
something like "applies to the entire address space". We should also copy this
paragraph to ext-legacy.adoc for the legacy remote SFENCE.VMA functions, since
the current paragraph technically does not apply to them.

If you agree, I will add these changes to
https://github.com/riscv-non-isa/riscv-sbi-doc/pull/128
(or should I open a separate pull request?)

Regards,
Samuel

>>
>>  lib/sbi/sbi_tlb.c | 21 +++------------------
>>  1 file changed, 3 insertions(+), 18 deletions(-)
>>
>> diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
>> index ced2854..dad9508 100644
>> --- a/lib/sbi/sbi_tlb.c
>> +++ b/lib/sbi/sbi_tlb.c
>> @@ -111,12 +111,7 @@ void sbi_tlb_local_hfence_vvma_asid(struct sbi_tlb_info *tinfo)
>>         hgatp = csr_swap(CSR_HGATP,
>>                          (vmid << HGATP_VMID_SHIFT) & HGATP_VMID_MASK);
>>
>> -       if (start == 0 && size == 0) {
>> -               __sbi_hfence_vvma_all();
>> -               goto done;
>> -       }
>> -
>> -       if (size == SBI_TLB_FLUSH_ALL) {
>> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
>>                 __sbi_hfence_vvma_asid(asid);
>>                 goto done;
>>         }
>> @@ -138,12 +133,7 @@ void sbi_tlb_local_hfence_gvma_vmid(struct sbi_tlb_info *tinfo)
>>
>>         sbi_pmu_ctr_incr_fw(SBI_PMU_FW_HFENCE_GVMA_VMID_RCVD);
>>
>> -       if (start == 0 && size == 0) {
>> -               __sbi_hfence_gvma_all();
>> -               return;
>> -       }
>> -
>> -       if (size == SBI_TLB_FLUSH_ALL) {
>> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
>>                 __sbi_hfence_gvma_vmid(vmid);
>>                 return;
>>         }
>> @@ -162,13 +152,8 @@ void sbi_tlb_local_sfence_vma_asid(struct sbi_tlb_info *tinfo)
>>
>>         sbi_pmu_ctr_incr_fw(SBI_PMU_FW_SFENCE_VMA_ASID_RCVD);
>>
>> -       if (start == 0 && size == 0) {
>> -               tlb_flush_all();
>> -               return;
>> -       }
>> -
>>         /* Flush entire MM context for a given ASID */
>> -       if (size == SBI_TLB_FLUSH_ALL) {
>> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
>>                 __asm__ __volatile__("sfence.vma x0, %0"
>>                                      :
>>                                      : "r"(asid)
>> --
>> 2.41.0
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi
> 
> Thanks,
> Anup



From anup at brainfault.org  Thu Nov 16 07:18:58 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 20:48:58 +0530
Subject: [PATCH] lib: sbi: Correctly limit flushes to a single ASID/VMID
In-Reply-To: <24cd0349-b63a-46ce-906e-2571c402c87d@sifive.com>
References: <20231017222027.3668468-1-samuel.holland@sifive.com>
 <CAAhSdy1tW2YNQzkDqsRWhSw0J7Vfg9DX5hSHarnw246QGL3Zrg@mail.gmail.com> <24cd0349-b63a-46ce-906e-2571c402c87d@sifive.com>
Message-ID: <CAAhSdy2Z9MtQJzaNPs=fC0GvV02vkGZBFED5vYhEO41d-tsDkg@mail.gmail.com>

On Thu, Nov 16, 2023 at 6:05?PM Samuel Holland
<samuel.holland at sifive.com> wrote:
>
> Hi Anup,
>
> On 2023-11-15 10:03 PM, Anup Patel wrote:
> > On Wed, Oct 18, 2023 at 3:50?AM Samuel Holland
> > <samuel.holland at sifive.com> wrote:
> >>
> >> Per the SBI specification, the effects of these functions are limited to
> >> a specific ASID and/or VMID. This applies even when flushing the entire
> >> address space.
> >>
> >> Signed-off-by: Samuel Holland <samuel.holland at sifive.com>
> >
> > This patch looks good to me.
> >
> > Reviewed-by: Anup Patel <anup at brainfault.org>
> >
> > Applied this patch to the riscv/opensbi repo.
> >
> >> ---
> >> The SBI spec doesn't say anything about this `start == 0 && size == 0`
> >> exception, so I wonder if it should be removed entirely.
> >
> > The SBI spec does define behaviour for `start == 0 && size == 0`.
> >
> > At the beginning of "Chapter 8. RFENCE Extension" we have the
> > following text which describes this behavior:
> >
> > "
> > The remote fence function acts as a full TLB flush if
> > * start_addr and size are both 0
> > * size is equal to 2^XLEN-1
> > "
>
> Thanks! I didn't see this paragraph in the spec. That explains the previous
> implementation (even though I still think it was wrong): "acts as a full TLB
> flush" could be interpreted to imply ignoring the ASID.
>
> I would suggest updating the SBI documentation with more precise wording,
> something like "applies to the entire address space". We should also copy this
> paragraph to ext-legacy.adoc for the legacy remote SFENCE.VMA functions, since
> the current paragraph technically does not apply to them.
>
> If you agree, I will add these changes to
> https://github.com/riscv-non-isa/riscv-sbi-doc/pull/128
> (or should I open a separate pull request?)

I think you should just add a separate commit for clarification
in the same PR.

Regards,
Anup

>
> Regards,
> Samuel
>
> >>
> >>  lib/sbi/sbi_tlb.c | 21 +++------------------
> >>  1 file changed, 3 insertions(+), 18 deletions(-)
> >>
> >> diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
> >> index ced2854..dad9508 100644
> >> --- a/lib/sbi/sbi_tlb.c
> >> +++ b/lib/sbi/sbi_tlb.c
> >> @@ -111,12 +111,7 @@ void sbi_tlb_local_hfence_vvma_asid(struct sbi_tlb_info *tinfo)
> >>         hgatp = csr_swap(CSR_HGATP,
> >>                          (vmid << HGATP_VMID_SHIFT) & HGATP_VMID_MASK);
> >>
> >> -       if (start == 0 && size == 0) {
> >> -               __sbi_hfence_vvma_all();
> >> -               goto done;
> >> -       }
> >> -
> >> -       if (size == SBI_TLB_FLUSH_ALL) {
> >> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
> >>                 __sbi_hfence_vvma_asid(asid);
> >>                 goto done;
> >>         }
> >> @@ -138,12 +133,7 @@ void sbi_tlb_local_hfence_gvma_vmid(struct sbi_tlb_info *tinfo)
> >>
> >>         sbi_pmu_ctr_incr_fw(SBI_PMU_FW_HFENCE_GVMA_VMID_RCVD);
> >>
> >> -       if (start == 0 && size == 0) {
> >> -               __sbi_hfence_gvma_all();
> >> -               return;
> >> -       }
> >> -
> >> -       if (size == SBI_TLB_FLUSH_ALL) {
> >> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
> >>                 __sbi_hfence_gvma_vmid(vmid);
> >>                 return;
> >>         }
> >> @@ -162,13 +152,8 @@ void sbi_tlb_local_sfence_vma_asid(struct sbi_tlb_info *tinfo)
> >>
> >>         sbi_pmu_ctr_incr_fw(SBI_PMU_FW_SFENCE_VMA_ASID_RCVD);
> >>
> >> -       if (start == 0 && size == 0) {
> >> -               tlb_flush_all();
> >> -               return;
> >> -       }
> >> -
> >>         /* Flush entire MM context for a given ASID */
> >> -       if (size == SBI_TLB_FLUSH_ALL) {
> >> +       if ((start == 0 && size == 0) || (size == SBI_TLB_FLUSH_ALL)) {
> >>                 __asm__ __volatile__("sfence.vma x0, %0"
> >>                                      :
> >>                                      : "r"(asid)
> >> --
> >> 2.41.0
> >>
> >>
> >> --
> >> opensbi mailing list
> >> opensbi at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/opensbi
> >
> > Thanks,
> > Anup
>


From anup at brainfault.org  Thu Nov 16 07:43:29 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 21:13:29 +0530
Subject: [PATCH v4 1/2] lib: sbi: separate the swap operation of domain region
In-Reply-To: <IA1PR20MB495392B05FE034A393C7C539BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49535E12C4609D9816C7C824BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB495392B05FE034A393C7C539BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy1wR1gNDfY2BXNzpZRdU9OLfRZUCJjzbp5jUP_kDUnbTA@mail.gmail.com>

On Thu, Nov 16, 2023 at 4:18?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Swapping domain region is a common operation when sorting domain region,
> so separate it as a function to make code clean.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_domain.c | 16 ++++++++++++----
>  1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index 4d248d0..71cb381 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -256,10 +256,20 @@ static const struct sbi_domain_memregion *find_next_subset_region(
>         return ret;
>  }
>
> +static void swap_region(struct sbi_domain_memregion* reg1,
> +                       struct sbi_domain_memregion* reg2)
> +{
> +       struct sbi_domain_memregion treg;
> +
> +       sbi_memcpy(&treg, reg1, sizeof(treg));
> +       sbi_memcpy(reg1, reg2, sizeof(treg));
> +       sbi_memcpy(reg2, &treg, sizeof(treg));
> +}
> +
>  static int sanitize_domain(struct sbi_domain *dom)
>  {
>         u32 i, j, count;
> -       struct sbi_domain_memregion treg, *reg, *reg1;
> +       struct sbi_domain_memregion *reg, *reg1;
>
>         /* Check possible HARTs */
>         if (!dom->possible_harts) {
> @@ -323,9 +333,7 @@ static int sanitize_domain(struct sbi_domain *dom)
>                         if (!is_region_before(reg1, reg))
>                                 continue;
>
> -                       sbi_memcpy(&treg, reg1, sizeof(treg));
> -                       sbi_memcpy(reg1, reg, sizeof(treg));
> -                       sbi_memcpy(reg, &treg, sizeof(treg));
> +                       swap_region(reg, reg1);
>                 }
>         }
>
> --
> 2.42.1
>


From anup at brainfault.org  Thu Nov 16 07:44:07 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 21:14:07 +0530
Subject: [PATCH v4 2/2] lib: sbi: Add regions merging when sanitizing
 domain region
In-Reply-To: <IA1PR20MB4953FEB382550CB12FA24E2FBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49535E12C4609D9816C7C824BBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB4953FEB382550CB12FA24E2FBBB0A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy2rP783YpJGUH5xX64_kotgsaGrSC-xQMmoc34VH0nmzQ@mail.gmail.com>

On Thu, Nov 16, 2023 at 4:19?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> As the domain will reject a new memory region which has a sub-regions
> already in the domain, even the new region is bigger and has the same
> flags. This problem can be solved by relaxing region restriction and
> rechecking when adding and sanitizing domains.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_domain.c | 59 +++++++++++++++++++++++++++-----------------
>  1 file changed, 37 insertions(+), 22 deletions(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index 71cb381..bc60562 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -193,12 +193,11 @@ static bool is_region_subset(const struct sbi_domain_memregion *regA,
>         return false;
>  }
>
> -/** Check if regionA conflicts regionB */
> -static bool is_region_conflict(const struct sbi_domain_memregion *regA,
> -                               const struct sbi_domain_memregion *regB)
> +/** Check if regionA can be replaced by regionB */
> +static bool is_region_compatible(const struct sbi_domain_memregion *regA,
> +                                const struct sbi_domain_memregion *regB)
>  {
> -       if ((is_region_subset(regA, regB) || is_region_subset(regB, regA)) &&
> -           regA->flags == regB->flags)
> +       if (is_region_subset(regA, regB) && regA->flags == regB->flags)
>                 return true;
>
>         return false;
> @@ -266,6 +265,11 @@ static void swap_region(struct sbi_domain_memregion* reg1,
>         sbi_memcpy(reg2, &treg, sizeof(treg));
>  }
>
> +static void clear_region(struct sbi_domain_memregion* reg)
> +{
> +       sbi_memset(reg, 0x0, sizeof(*reg));
> +}
> +
>  static int sanitize_domain(struct sbi_domain *dom)
>  {
>         u32 i, j, count;
> @@ -320,16 +324,6 @@ static int sanitize_domain(struct sbi_domain *dom)
>                 for (j = i + 1; j < count; j++) {
>                         reg1 = &dom->regions[j];
>
> -                       if (is_region_conflict(reg1, reg)) {
> -                               sbi_printf("%s: %s conflict between regions "
> -                                       "(base=0x%lx order=%lu flags=0x%lx) and "
> -                                       "(base=0x%lx order=%lu flags=0x%lx)\n",
> -                                       __func__, dom->name,
> -                                       reg->base, reg->order, reg->flags,
> -                                       reg1->base, reg1->order, reg1->flags);
> -                               return SBI_EINVAL;
> -                       }
> -
>                         if (!is_region_before(reg1, reg))
>                                 continue;
>
> @@ -337,6 +331,31 @@ static int sanitize_domain(struct sbi_domain *dom)
>                 }
>         }
>
> +       /* Remove covered regions */
> +       while(i < (count - 1)) {
> +               bool need_removed = false;
> +               reg = &dom->regions[i];
> +
> +               for (j = i + 1; j < count; j++) {
> +                       reg1 = &dom->regions[j];
> +
> +                       if (is_region_compatible(reg, reg1)) {
> +                               need_removed = true;
> +                               break;
> +                       }
> +               }
> +
> +               /* find a region is superset of reg, remove reg */
> +               if (need_removed) {
> +                       for (j = i; j < (count - 1); j++)
> +                               swap_region(&dom->regions[j],
> +                                           &dom->regions[j + 1]);
> +                       clear_region(&dom->regions[count - 1]);
> +                       count--;
> +               } else
> +                       i++;
> +       }
> +
>         /*
>          * We don't need to check boot HART id of domain because if boot
>          * HART id is not possible/assigned to this domain then it won't
> @@ -576,14 +595,10 @@ int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
>             (ROOT_REGION_MAX <= root_memregs_count))
>                 return SBI_EINVAL;
>
> -       /* Check for conflicts */
> +       /* Check whether compatible region exists for the new one */
>         sbi_domain_for_each_memregion(&root, nreg) {
> -               if (is_region_conflict(reg, nreg)) {
> -                       sbi_printf("%s: is_region_conflict check failed"
> -                       " 0x%lx conflicts existing 0x%lx\n", __func__,
> -                                  reg->base, nreg->base);
> -                       return SBI_EALREADY;
> -               }
> +               if (is_region_compatible(reg, nreg))
> +                       return 0;
>         }
>
>         /* Append the memregion to root memregions */
> --
> 2.42.1
>


From anup at brainfault.org  Thu Nov 16 07:56:06 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 21:26:06 +0530
Subject: [PATCH v4] platform: generic: Add Sophgo sg2042 platform support
In-Reply-To: <IA1PR20MB4953FC1436930F92BA765847BBD4A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953FC1436930F92BA765847BBD4A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy3wdnPbB1WwYpKg3XH6V3vYchPMU3tRf6MXf493eQ1ESw@mail.gmail.com>

On Fri, Oct 20, 2023 at 5:27?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Add Sophgo sg2042 soc support
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> Reviewed-by: Guo Ren <guoren at kernel.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
> This patch depends these patchs:
> 1. platform: generic: add T-HEAD th1520 soc support
> 2. lib: sbi: Add sub-regions check for sanitizing domain
> 3. lib: utils: Add T-HEAD C900 ACLINT
> 4. platform: generic: thead: improve tlb flush errata
>
> Link: https://lists.infradead.org/pipermail/opensbi/2023-October/005676.html
> Link: https://lists.infradead.org/pipermail/opensbi/2023-October/005705.html
> Link: https://lists.infradead.org/pipermail/opensbi/2023-October/005736.html
> Link: https://lists.infradead.org/pipermail/opensbi/2023-October/005712.html
> Link: https://lore.kernel.org/linux-riscv/cover.1696433229.git.unicorn_wang at outlook.com/
>
> Changed from v3:
> 1. remove unused variable to avoid compile failed
>
> Changed from v2:
> 1. replace hart count with number 16 when calculating the timer area.
>    The sg2042 have a fixed number of timer, see the kernel dt in the link.
> 2. fix typo
> 3. move the c900 aclint patch out of this series.
> 4. move the tlb size change patch out of this series.
>
> Changed from v1:
> 1. make number of entries in tlb fifo auto deteted.
>
> ---
>  platform/generic/Kconfig           |  6 +++
>  platform/generic/configs/defconfig |  1 +
>  platform/generic/sophgo/objects.mk |  9 +++++
>  platform/generic/sophgo/sg2042.c   | 62 ++++++++++++++++++++++++++++++
>  4 files changed, 78 insertions(+)
>  create mode 100644 platform/generic/sophgo/objects.mk
>  create mode 100644 platform/generic/sophgo/sg2042.c
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index f9e7ed2..a902cf3 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -49,6 +49,12 @@ config PLATFORM_SIFIVE_FU740
>         depends on FDT_RESET && FDT_I2C
>         default n
>
> +config PLATFORM_SOPHGO_SG2042
> +       bool "Sophgo sg2042 support"
> +       select THEAD_C9XX_ERRATA
> +       select THEAD_C9XX_PMU
> +       default n
> +
>  config PLATFORM_STARFIVE_JH7110
>         bool "StarFive JH7110 support"
>         default n
> diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
> index c432bc2..e1b76c8 100644
> --- a/platform/generic/configs/defconfig
> +++ b/platform/generic/configs/defconfig
> @@ -3,6 +3,7 @@ CONFIG_PLATFORM_ANDES_AE350=y
>  CONFIG_PLATFORM_RENESAS_RZFIVE=y
>  CONFIG_PLATFORM_SIFIVE_FU540=y
>  CONFIG_PLATFORM_SIFIVE_FU740=y
> +CONFIG_PLATFORM_SOPHGO_SG2042=y
>  CONFIG_PLATFORM_STARFIVE_JH7110=y
>  CONFIG_PLATFORM_THEAD=y
>  CONFIG_FDT_GPIO=y
> diff --git a/platform/generic/sophgo/objects.mk b/platform/generic/sophgo/objects.mk
> new file mode 100644
> index 0000000..3d2908a
> --- /dev/null
> +++ b/platform/generic/sophgo/objects.mk
> @@ -0,0 +1,9 @@
> +#
> +# SPDX-License-Identifier: BSD-2-Clause
> +#
> +# Copyright (C) 2023 Inochi Amaoto <inochiama at outlook.com>
> +# Copyright (C) 2023 Alibaba Group Holding Limited.
> +#
> +
> +carray-platform_override_modules-$(CONFIG_PLATFORM_SOPHGO_SG2042) += sophgo_sg2042
> +platform-objs-$(CONFIG_PLATFORM_SOPHGO_SG2042) += sophgo/sg2042.o
> diff --git a/platform/generic/sophgo/sg2042.c b/platform/generic/sophgo/sg2042.c
> new file mode 100644
> index 0000000..9c6acdb
> --- /dev/null
> +++ b/platform/generic/sophgo/sg2042.c
> @@ -0,0 +1,62 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Authors:
> + *   Inochi Amaoto <inochiama at outlook.com>
> + *
> + */
> +
> +#include <platform_override.h>
> +#include <thead/c9xx_errata.h>
> +#include <thead/c9xx_pmu.h>
> +#include <sbi/sbi_const.h>
> +#include <sbi/sbi_domain.h>
> +#include <sbi/sbi_platform.h>
> +#include <sbi/sbi_scratch.h>
> +#include <sbi/sbi_string.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +#include <sbi_utils/timer/aclint_mtimer.h>
> +
> +#define SOPHGO_SG2042_TIMER_BASE       0x70ac000000UL
> +#define SOPHGO_SG2042_TIMER_SIZE       0x10000UL
> +#define SOPHGO_SG2042_TIMER_NUM                16
> +
> +static int sophgo_sg2042_early_init(bool cold_boot,
> +                                   const struct fdt_match *match)
> +{
> +       thead_register_tlb_flush_trap_handler();
> +
> +       /*
> +        * Sophgo sg2042 soc use separate 16 timers while initiating,
> +        * merge them as a single domain to avoid wasting.
> +        */
> +       if (cold_boot)
> +               return sbi_domain_root_add_memrange(SOPHGO_SG2042_TIMER_BASE,
> +                                               SOPHGO_SG2042_TIMER_SIZE *
> +                                               SOPHGO_SG2042_TIMER_NUM,
> +                                               MTIMER_REGION_ALIGN,
> +                                               (SBI_DOMAIN_MEMREGION_MMIO |
> +                                                SBI_DOMAIN_MEMREGION_M_READABLE |
> +                                                SBI_DOMAIN_MEMREGION_M_WRITABLE));
> +
> +
> +       return 0;
> +}
> +
> +static int sophgo_sg2042_extensions_init(const struct fdt_match *match,
> +                                        struct sbi_hart_features *hfeatures)
> +{
> +       thead_c9xx_register_pmu_device();
> +       return 0;
> +}
> +
> +static const struct fdt_match sophgo_sg2042_match[] = {
> +       { .compatible = "sophgo,sg2042" },
> +       { },
> +};
> +
> +const struct platform_override sophgo_sg2042 = {
> +       .match_table            = sophgo_sg2042_match,
> +       .early_init             = sophgo_sg2042_early_init,
> +       .extensions_init        = sophgo_sg2042_extensions_init,
> +};
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Thu Nov 16 08:00:22 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 16 Nov 2023 21:30:22 +0530
Subject: [PATCH v2] lib: utils/serial: Optimize semihosting_putc implementation
In-Reply-To: <4d7d9b69c833a7de7806019f734576a9998b7f76.camel@126.com>
References: <20231018024205.270-1-cp0613@linux.alibaba.com>
 <CAAhSdy0BphFUER+8kiza-Cw1qG3TD7xMZW9KkJeyAjqgxyqk4g@mail.gmail.com> <4d7d9b69c833a7de7806019f734576a9998b7f76.camel@126.com>
Message-ID: <CAAhSdy2Y5ydEKrFx1AGx9NOmk50zWB4adEvsiO0JGKruPGAoag@mail.gmail.com>

On Thu, Nov 16, 2023 at 2:16?PM Xiang W <wxjstz at 126.com> wrote:
>
> ? 2023-11-16???? 11:42 +0530?Anup Patel???
> > On Wed, Oct 18, 2023 at 8:12?AM <cp0613 at linux.alibaba.com> wrote:
> > >
> > > From: Chen Pei <cp0613 at linux.alibaba.com>
> > >
> > > For some debuggers that do not implement SYSWRITEC and SYSREADC
> > > operations, we can use SYSWRITE and SYSREAD instead like the
> > > implementation of semihosting_getc().
> > >
> > > Signed-off-by: Chen Pei <cp0613 at linux.alibaba.com>
> > > Reviewed-by: Xiang W <wxjstz at 126.com>
> >
> > Looks good to me.
> >
> > Reviewed-by: Anup Patel <anup at brainfault.org>
> >
> > Applied this patch to the riscv/opensbi repo.
> I submitted a new workaround.
>
> https://patchwork.ozlabs.org/project/opensbi/patch/20231025043104.748104-1-wxjstz at 126.com/
> https://patchwork.ozlabs.org/project/opensbi/patch/20231025043104.748104-2-wxjstz at 126.com/

Can you rebase and send a new version for review ?

Regards,
Anup

>
> Regards,
> Xiang W
> >
> > Thanks,
> > Anup
> >
> > >
> > > Changed from v1:
> > > 1. simplified by calling semihosting_puts directly.
> > >
> > > ---
> > >  lib/utils/serial/semihosting.c | 10 +++++-----
> > >  1 file changed, 5 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/lib/utils/serial/semihosting.c b/lib/utils/serial/semihosting.c
> > > index ce65887..a27c69e 100644
> > > --- a/lib/utils/serial/semihosting.c
> > > +++ b/lib/utils/serial/semihosting.c
> > > @@ -160,11 +160,6 @@ static long semihosting_write(long fd, const void *memp, size_t len)
> > >
> > >  /* clang-format on */
> > >
> > > -static void semihosting_putc(char ch)
> > > -{
> > > -       semihosting_trap(SYSWRITEC, &ch);
> > > -}
> > > -
> > >  static unsigned long semihosting_puts(const char *str, unsigned long len)
> > >  {
> > >         char ch;
> > > @@ -183,6 +178,11 @@ static unsigned long semihosting_puts(const char *str, unsigned long len)
> > >         return (ret < 0) ? 0 : ret;
> > >  }
> > >
> > > +static void semihosting_putc(char ch)
> > > +{
> > > +       semihosting_puts(&ch, 1);
> > > +}
> > > +
> > >  static int semihosting_getc(void)
> > >  {
> > >         char ch = 0;
> > > --
> > > 2.25.1
> > >
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
>


From wxjstz at 126.com  Thu Nov 16 17:03:58 2023
From: wxjstz at 126.com (Xiang W)
Date: Fri, 17 Nov 2023 09:03:58 +0800
Subject: [PATCH 1/2] lib: sbi: Make console_puts/console_putc interchangeable
Message-ID: <20231117010440.105759-1-wxjstz@126.com>

console_puts/console_putc should replace each other, but the previous
sbi_putc can only use console_putc. This patch improves this problem.

Signed-off-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Guo Ren <guoren at kerenl.org>
---
 lib/sbi/sbi_console.c | 35 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 18 deletions(-)

diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index 2b3b0a3..ab09a5c 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -37,28 +37,22 @@ int sbi_getc(void)
 	return -1;
 }
 
-void sbi_putc(char ch)
-{
-	if (console_dev && console_dev->console_putc) {
-		if (ch == '\n')
-			console_dev->console_putc('\r');
-		console_dev->console_putc(ch);
-	}
-}
-
 static unsigned long nputs(const char *str, unsigned long len)
 {
-	unsigned long i, ret;
+	unsigned long i;
 
-	if (console_dev && console_dev->console_puts) {
-		ret = console_dev->console_puts(str, len);
-	} else {
-		for (i = 0; i < len; i++)
-			sbi_putc(str[i]);
-		ret = len;
+	if (console_dev) {
+		if (console_dev->console_puts)
+			return console_dev->console_puts(str, len);
+		else if (console_dev->console_putc) {
+			for (i = 0; i < len; i++) {
+				if (str[i] == '\n')
+					console_dev->console_putc('\r');
+				console_dev->console_putc(str[i]);
+			}
+		}
 	}
-
-	return ret;
+	return len;
 }
 
 static void nputs_all(const char *str, unsigned long len)
@@ -69,6 +63,11 @@ static void nputs_all(const char *str, unsigned long len)
 		p += nputs(&str[p], len - p);
 }
 
+void sbi_putc(char ch)
+{
+	nputs_all(&ch, 1);
+}
+
 void sbi_puts(const char *str)
 {
 	unsigned long len = sbi_strlen(str);
-- 
2.42.0



From wxjstz at 126.com  Thu Nov 16 17:03:59 2023
From: wxjstz at 126.com (Xiang W)
Date: Fri, 17 Nov 2023 09:03:59 +0800
Subject: [PATCH 2/2] lib: utils/serial: remove semihosting_putc
In-Reply-To: <20231117010440.105759-1-wxjstz@126.com>
References: <20231117010440.105759-1-wxjstz@126.com>
Message-ID: <20231117010440.105759-2-wxjstz@126.com>

For some debuggers that do not implement SYSWRITEC and SYSREADC
operations, we can use SYSWRITE and SYSREAD instead like the
implementation of semihosting_putc(). This makes
semihosting_putc/semihosting_puts similar. Since
console_putc/console_puts are now interchangeable, removing the
semihosting_putc.

Signed-off-by: Chen Pei <cp0613 at linux.alibaba.com>
Signed-off-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Guo Ren <guoren at kerenl.org>
---
 lib/utils/serial/semihosting.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/lib/utils/serial/semihosting.c b/lib/utils/serial/semihosting.c
index a27c69e..19ebaa0 100644
--- a/lib/utils/serial/semihosting.c
+++ b/lib/utils/serial/semihosting.c
@@ -178,11 +178,6 @@ static unsigned long semihosting_puts(const char *str, unsigned long len)
 	return (ret < 0) ? 0 : ret;
 }
 
-static void semihosting_putc(char ch)
-{
-	semihosting_puts(&ch, 1);
-}
-
 static int semihosting_getc(void)
 {
 	char ch = 0;
@@ -199,7 +194,6 @@ static int semihosting_getc(void)
 
 static struct sbi_console_device semihosting_console = {
 	.name = "semihosting",
-	.console_putc = semihosting_putc,
 	.console_puts = semihosting_puts,
 	.console_getc = semihosting_getc
 };
-- 
2.42.0



From wxjstz at 126.com  Thu Nov 16 17:06:50 2023
From: wxjstz at 126.com (Xiang W)
Date: Fri, 17 Nov 2023 09:06:50 +0800
Subject: [PATCH v2] lib: utils/serial: Optimize semihosting_putc
 implementation
In-Reply-To: <CAAhSdy2Y5ydEKrFx1AGx9NOmk50zWB4adEvsiO0JGKruPGAoag@mail.gmail.com>
References: <20231018024205.270-1-cp0613@linux.alibaba.com>
	 <CAAhSdy0BphFUER+8kiza-Cw1qG3TD7xMZW9KkJeyAjqgxyqk4g@mail.gmail.com>
	 <4d7d9b69c833a7de7806019f734576a9998b7f76.camel@126.com>
	 <CAAhSdy2Y5ydEKrFx1AGx9NOmk50zWB4adEvsiO0JGKruPGAoag@mail.gmail.com>
Message-ID: <d514cf38c8425079ca46edbdb68e4e8b117b5f37.camel@126.com>

? 2023-11-16???? 21:30 +0530?Anup Patel???
> On Thu, Nov 16, 2023 at 2:16?PM Xiang W <wxjstz at 126.com> wrote:
> > 
> > ? 2023-11-16???? 11:42 +0530?Anup Patel???
> > > On Wed, Oct 18, 2023 at 8:12?AM <cp0613 at linux.alibaba.com> wrote:
> > > > 
> > > > From: Chen Pei <cp0613 at linux.alibaba.com>
> > > > 
> > > > For some debuggers that do not implement SYSWRITEC and SYSREADC
> > > > operations, we can use SYSWRITE and SYSREAD instead like the
> > > > implementation of semihosting_getc().
> > > > 
> > > > Signed-off-by: Chen Pei <cp0613 at linux.alibaba.com>
> > > > Reviewed-by: Xiang W <wxjstz at 126.com>
> > > 
> > > Looks good to me.
> > > 
> > > Reviewed-by: Anup Patel <anup at brainfault.org>
> > > 
> > > Applied this patch to the riscv/opensbi repo.
> > I submitted a new workaround.
> > 
> > https://patchwork.ozlabs.org/project/opensbi/patch/20231025043104.748104-1-wxjstz at 126.com/
> > https://patchwork.ozlabs.org/project/opensbi/patch/20231025043104.748104-2-wxjstz at 126.com/
> 
> Can you rebase and send a new version for review ?

New version at:
https://patchwork.ozlabs.org/project/opensbi/patch/20231117010440.105759-1-wxjstz at 126.com/
https://patchwork.ozlabs.org/project/opensbi/patch/20231117010440.105759-2-wxjstz at 126.com/

Regards,
Xiang W
> 
> Regards,
> Anup
> 
> > 
> > Regards,
> > Xiang W
> > > 
> > > Thanks,
> > > Anup
> > > 
> > > > 
> > > > Changed from v1:
> > > > 1. simplified by calling semihosting_puts directly.
> > > > 
> > > > ---
> > > > ?lib/utils/serial/semihosting.c | 10 +++++-----
> > > > ?1 file changed, 5 insertions(+), 5 deletions(-)
> > > > 
> > > > diff --git a/lib/utils/serial/semihosting.c b/lib/utils/serial/semihosting.c
> > > > index ce65887..a27c69e 100644
> > > > --- a/lib/utils/serial/semihosting.c
> > > > +++ b/lib/utils/serial/semihosting.c
> > > > @@ -160,11 +160,6 @@ static long semihosting_write(long fd, const void *memp, size_t len)
> > > > 
> > > > ?/* clang-format on */
> > > > 
> > > > -static void semihosting_putc(char ch)
> > > > -{
> > > > -?????? semihosting_trap(SYSWRITEC, &ch);
> > > > -}
> > > > -
> > > > ?static unsigned long semihosting_puts(const char *str, unsigned long len)
> > > > ?{
> > > > ??????? char ch;
> > > > @@ -183,6 +178,11 @@ static unsigned long semihosting_puts(const char *str, unsigned long len)
> > > > ??????? return (ret < 0) ? 0 : ret;
> > > > ?}
> > > > 
> > > > +static void semihosting_putc(char ch)
> > > > +{
> > > > +?????? semihosting_puts(&ch, 1);
> > > > +}
> > > > +
> > > > ?static int semihosting_getc(void)
> > > > ?{
> > > > ??????? char ch = 0;
> > > > --
> > > > 2.25.1
> > > > 
> > > > 
> > > > --
> > > > opensbi mailing list
> > > > opensbi at lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/opensbi
> > 
> 



From inochiama at outlook.com  Thu Nov 16 21:05:24 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Fri, 17 Nov 2023 13:05:24 +0800
Subject: [PATCH] lib: utils/fdt: Allow to use reg-names when parsing ACLINT
Message-ID: <IA1PR20MB4953124B319AEF3573200E09BBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>

Currently, the fdt_parse_aclint_node() follows a fixed order to parse
ACLINT timer. This may cause the undesirable result when the ACLINT
device does not support mtime without adding an empty entry for it in
the DT.

To be robust, make fdt_parse_aclint_node() support "reg-names" property,
so it can parse the DT in an order independent way. For compatibility,
fdt_parse_aclint_node() only use "reg-names" when parsing ACLINT timer,
and will fallback to the old way if "reg-names" property is not found.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
Link: https://lore.kernel.org/all/20231114-skedaddle-precinct-66c8897227bb at squawk/
---
 lib/utils/fdt/fdt_helper.c | 107 +++++++++++++++++++++++++++++++------
 1 file changed, 91 insertions(+), 16 deletions(-)

diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
index 9ae7f09..0f42d68 100644
--- a/lib/utils/fdt/fdt_helper.c
+++ b/lib/utils/fdt/fdt_helper.c
@@ -216,6 +216,32 @@ int fdt_get_node_addr_size(void *fdt, int node, int index,
 	return 0;
 }

+int fdt_get_node_addr_size_by_name(void *fdt, int node, const char *name,
+				   uint64_t *addr, uint64_t *size)
+{
+	int i, j, count;
+	const char *val;
+	const char *regname;
+
+	if (!fdt || node < 0 || !name)
+		return SBI_EINVAL;
+
+	val = fdt_getprop(fdt, node, "reg-names", &count);
+	if (!val)
+		return SBI_ENODEV;
+
+	for (i = 0, j = 0; i < count; i++, j++) {
+		regname = val + i;
+
+		if (strcmp(name, regname) == 0)
+			return fdt_get_node_addr_size(fdt, node, j, addr, size);
+
+		i += strlen(regname);
+	}
+
+	return SBI_ENODEV;
+}
+
 bool fdt_node_is_enabled(void *fdt, int nodeoff)
 {
 	int len;
@@ -873,13 +899,66 @@ int fdt_parse_plic(void *fdt, struct plic_data *plic, const char *compat)
 	return fdt_parse_plic_node(fdt, nodeoffset, plic);
 }

+int fdt_get_aclint_addr_size_by_name(void *fdt, int nodeoffset,
+				     unsigned long *out_addr1,
+				     unsigned long *out_size1,
+				     unsigned long *out_addr2,
+				     unsigned long *out_size2)
+{
+	int rc;
+	uint64_t reg_addr, reg_size;
+
+	rc = fdt_get_node_addr_size_by_name(fdt, nodeoffset, "mtime",
+					    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_size)
+		reg_addr = reg_size = 0;
+	*out_addr1 = reg_addr;
+	*out_size1 = reg_size;
+
+	rc = fdt_get_node_addr_size_by_name(fdt, nodeoffset, "mtimecmp",
+					    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_size)
+		return SBI_ENODEV;
+	*out_addr2 = reg_addr;
+	*out_size2 = reg_size;
+
+	return 0;
+}
+
+int fdt_get_aclint_addr_size(void *fdt, int nodeoffset,
+			     unsigned long *out_addr1,
+			     unsigned long *out_size1,
+			     unsigned long *out_addr2,
+			     unsigned long *out_size2)
+{
+	int rc;
+	uint64_t reg_addr, reg_size;
+
+	rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
+				    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_size)
+		return SBI_ENODEV;
+	*out_addr1 = reg_addr;
+	*out_size1 = reg_size;
+
+	rc = fdt_get_node_addr_size(fdt, nodeoffset, 1,
+				    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_size)
+		reg_addr = reg_size = 0;
+	if (out_addr2)
+		*out_addr2 = reg_addr;
+	if (out_size2)
+		*out_size2 = reg_size;
+
+	return 0;
+}
+
 int fdt_parse_aclint_node(void *fdt, int nodeoffset, bool for_timer,
 			  unsigned long *out_addr1, unsigned long *out_size1,
 			  unsigned long *out_addr2, unsigned long *out_size2,
 			  u32 *out_first_hartid, u32 *out_hart_count)
 {
 	const fdt32_t *val;
-	uint64_t reg_addr, reg_size;
 	int i, rc, count, cpu_offset, cpu_intc_offset;
 	u32 phandle, hwirq, hartid, first_hartid, last_hartid, hart_count;
 	u32 match_hwirq = (for_timer) ? IRQ_M_TIMER : IRQ_M_SOFT;
@@ -889,21 +968,17 @@ int fdt_parse_aclint_node(void *fdt, int nodeoffset, bool for_timer,
 	    !out_first_hartid || !out_hart_count)
 		return SBI_EINVAL;

-	rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
-				    &reg_addr, &reg_size);
-	if (rc < 0 || !reg_size)
-		return SBI_ENODEV;
-	*out_addr1 = reg_addr;
-	*out_size1 = reg_size;
-
-	rc = fdt_get_node_addr_size(fdt, nodeoffset, 1,
-				    &reg_addr, &reg_size);
-	if (rc < 0 || !reg_size)
-		reg_addr = reg_size = 0;
-	if (out_addr2)
-		*out_addr2 = reg_addr;
-	if (out_size2)
-		*out_size2 = reg_size;
+	if (for_timer && out_addr2 && out_size2 &&
+	    fdt_getprop(fdt, nodeoffset, "reg-names", NULL))
+		rc = fdt_get_aclint_addr_size_by_name(fdt, nodeoffset,
+						      out_addr1, out_size1,
+						      out_addr2, out_size2);
+	else
+		rc = fdt_get_aclint_addr_size(fdt, nodeoffset,
+					      out_addr1, out_size1,
+					      out_addr2, out_size2);
+	if (rc)
+		return rc;

 	*out_first_hartid = 0;
 	*out_hart_count = 0;
--
2.42.1



From anup at brainfault.org  Thu Nov 16 23:20:13 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 12:50:13 +0530
Subject: [PATCH 1/3] include: sbi: macros for mseccfg.sseed and .useed
In-Reply-To: <20231028224959.95465-2-heinrich.schuchardt@canonical.com>
References: <20231028224959.95465-1-heinrich.schuchardt@canonical.com> <20231028224959.95465-2-heinrich.schuchardt@canonical.com>
Message-ID: <CAAhSdy31SkYmT+tWTafoSLubEtX2k+awpHAjntWgozvBymx2rA@mail.gmail.com>

On Sun, Oct 29, 2023 at 4:20?AM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> Define macros to access the sseed and the useed bit in the machine
> security configuration register (mseccfg).
>
> Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
> Just resending this patch which should be merged independent of how
> we set mseccfg.sseed in future.
> ---
>  include/sbi/riscv_encoding.h | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
> index a545242..0996d64 100644
> --- a/include/sbi/riscv_encoding.h
> +++ b/include/sbi/riscv_encoding.h
> @@ -678,6 +678,10 @@
>  #define MSECCFG_MMWP                   (_UL(1) << MSECCFG_MMWP_SHIFT)
>  #define MSECCFG_RLB_SHIFT              (2)
>  #define MSECCFG_RLB                    (_UL(1) << MSECCFG_RLB_SHIFT)
> +#define MSECCFG_USEED_SHIFT            (8)
> +#define MSECCFG_USEED                  (_UL(1) << MSECCFG_USEED_SHIFT)
> +#define MSECCFG_SSEED_SHIFT            (9)
> +#define MSECCFG_SSEED                  (_UL(1) << MSECCFG_SSEED_SHIFT)
>
>  /* Counter Overflow CSR */
>  #define CSR_SCOUNTOVF                  0xda0
> --
> 2.40.1
>


From anup at brainfault.org  Thu Nov 16 23:21:27 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 12:51:27 +0530
Subject: [RFC 2/3] lib: sbi: Add Zkr in hart extensions
In-Reply-To: <20231028224959.95465-3-heinrich.schuchardt@canonical.com>
References: <20231028224959.95465-1-heinrich.schuchardt@canonical.com> <20231028224959.95465-3-heinrich.schuchardt@canonical.com>
Message-ID: <CAAhSdy3ASwnDcQW_FE-JF7-2zhQ+-c0_ijBEoYWX+UiJjnZP0g@mail.gmail.com>

On Sun, Oct 29, 2023 at 4:20?AM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> - Add Zkr as extension in sbi_hart_extensions enum
> - Return "zkr" string for Zkr extension from sbi_hart_extension_id2string
>
> Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  include/sbi/sbi_hart.h     | 2 ++
>  lib/sbi/sbi_hart.c         | 3 +++
>  lib/utils/fdt/fdt_helper.c | 1 +
>  3 files changed, 6 insertions(+)
>
> diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
> index e60f415..fe1124a 100644
> --- a/include/sbi/sbi_hart.h
> +++ b/include/sbi/sbi_hart.h
> @@ -40,6 +40,8 @@ enum sbi_hart_extensions {
>         SBI_HART_EXT_ZICNTR,
>         /** HART has Zihpm extension */
>         SBI_HART_EXT_ZIHPM,
> +       /** HART has Zkr extension */
> +       SBI_HART_EXT_ZKR,
>         /** Hart has Smcntrpmf extension */
>         SBI_HART_EXT_SMCNTRPMF,
>
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 29d6481..569596f 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -678,6 +678,9 @@ static inline char *sbi_hart_extension_id2string(int ext)
>         case SBI_HART_EXT_ZIHPM:
>                 estr = "zihpm";
>                 break;
> +       case SBI_HART_EXT_ZKR:
> +               estr = "zkr";
> +               break;
>         case SBI_HART_EXT_SMEPMP:
>                 estr = "smepmp";
>                 break;
> diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
> index 9ae7f09..35be7ba 100644
> --- a/lib/utils/fdt/fdt_helper.c
> +++ b/lib/utils/fdt/fdt_helper.c
> @@ -376,6 +376,7 @@ static int fdt_parse_isa_one_hart(const char *isa, unsigned long *extensions)
>                         }
>
>                 set_multi_letter_ext("smepmp", SBI_HART_EXT_SMEPMP);
> +               set_multi_letter_ext("zkr", SBI_HART_EXT_ZKR);
>  #undef set_multi_letter_ext
>         }
>
> --
> 2.40.1
>


From anup at brainfault.org  Thu Nov 16 23:40:57 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 13:10:57 +0530
Subject: [RFC 3/3] lib: sbi: enable seed access in S-mode
In-Reply-To: <20231028224959.95465-4-heinrich.schuchardt@canonical.com>
References: <20231028224959.95465-1-heinrich.schuchardt@canonical.com> <20231028224959.95465-4-heinrich.schuchardt@canonical.com>
Message-ID: <CAAhSdy2CO3JUUYM-W1kT+QLPbzGb9UAg1RfhrCOuvfEvJh2pvw@mail.gmail.com>

On Sun, Oct 29, 2023 at 4:20?AM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> If ISA extension Zkr is available, set
>
>     mseccfg.sseed=1
>     mseccfg.useed=0
>
> This enables access to the seed CSR in S-mode but not in U-mode.
>
> Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>
> ---
>  lib/sbi/sbi_hart.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 569596f..5a3e8ee 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -530,6 +530,15 @@ int sbi_hart_unmap_saddr(void)
>         return pmp_disable(SBI_SMEPMP_RESV_ENTRY);
>  }
>
> +static void sbi_hart_zkr_configure(struct sbi_scratch *scratch)
> +{
> +       unsigned long mseccfg_val = csr_read(CSR_MSECCFG);
> +
> +       mseccfg_val |= MSECCFG_SSEED;
> +       mseccfg_val &= ~MSECCFG_USEED;
> +       csr_set(CSR_MSECCFG, mseccfg_val);
> +}
> +
>  int sbi_hart_pmp_configure(struct sbi_scratch *scratch)
>  {
>         int rc;
> @@ -551,6 +560,9 @@ int sbi_hart_pmp_configure(struct sbi_scratch *scratch)
>                 rc = sbi_hart_oldpmp_configure(scratch, pmp_count,
>                                                 pmp_gran_log2, pmp_addr_max);
>
> +       if (sbi_hart_has_extension(scratch, SBI_HART_EXT_ZKR))
> +               sbi_hart_zkr_configure(scratch);
> +

This needs to be done from the mstatus_init() function since it is not
related to PMP. I have taken care of this at the time of merging this patch.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

>         /*
>          * As per section 3.7.2 of privileged specification v1.12,
>          * virtual address translations can be speculatively performed
> --
> 2.40.1
>

Regards,
Anup


From anup at brainfault.org  Thu Nov 16 23:59:31 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 13:29:31 +0530
Subject: [PATCH] lib: sbi: change sbi_hart_features.extensions as an array
In-Reply-To: <20231030164046.25108-1-wxjstz@126.com>
References: <20231030164046.25108-1-wxjstz@126.com>
Message-ID: <CAAhSdy1EGbRBCWTUdLihA92SewS1n1ecVbVC1f6jStXa-tJMdA@mail.gmail.com>

On Mon, Oct 30, 2023 at 10:11?PM Xiang W <wxjstz at 126.com> wrote:
>
> In the future there may be a lot of ISA extensions, a 'long' may not
> be able to accommodate, changed to an array for the future.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  include/sbi/sbi_hart.h      |  3 ++-
>  lib/sbi/sbi_hart.c          | 32 ++++++++++++--------------------
>  platform/generic/platform.c |  2 +-
>  3 files changed, 15 insertions(+), 22 deletions(-)
>
> diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
> index e60f415..3b73f63 100644
> --- a/include/sbi/sbi_hart.h
> +++ b/include/sbi/sbi_hart.h
> @@ -11,6 +11,7 @@
>  #define __SBI_HART_H__
>
>  #include <sbi/sbi_types.h>
> +#include <sbi/sbi_bitops.h>
>
>  /** Possible privileged specification versions of a hart */
>  enum sbi_hart_priv_versions {
> @@ -65,7 +66,7 @@ enum sbi_hart_extensions {
>  struct sbi_hart_features {
>         bool detected;
>         int priv_version;
> -       unsigned long extensions;
> +       unsigned long extensions[BITS_TO_LONGS(SBI_HART_EXT_MAX)];
>         unsigned int pmp_count;
>         unsigned int pmp_addr_bits;
>         unsigned long pmp_gran;
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 29d6481..f94eba5 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -614,9 +614,9 @@ static inline void __sbi_hart_update_extension(
>                                         bool enable)
>  {
>         if (enable)
> -               hfeatures->extensions |= BIT(ext);
> +               __set_bit(ext, hfeatures->extensions);
>         else
> -               hfeatures->extensions &= ~BIT(ext);
> +               __clear_bit(ext, hfeatures->extensions);
>  }
>
>  /**
> @@ -649,7 +649,7 @@ bool sbi_hart_has_extension(struct sbi_scratch *scratch,
>         struct sbi_hart_features *hfeatures =
>                         sbi_scratch_offset_ptr(scratch, hart_features_offset);
>
> -       if (hfeatures->extensions & BIT(ext))
> +       if (__test_bit(ext, hfeatures->extensions))
>                 return true;
>         else
>                 return false;
> @@ -712,24 +712,16 @@ void sbi_hart_get_extensions_str(struct sbi_scratch *scratch,
>                 return;
>         sbi_memset(extensions_str, 0, nestr);
>
> -       if (!hfeatures->extensions)
> -               goto done;
> -
> -       do {
> -               if (hfeatures->extensions & BIT(ext)) {
> -                       temp = sbi_hart_extension_id2string(ext);
> -                       if (temp) {
> -                               sbi_snprintf(extensions_str + offset,
> -                                            nestr - offset,
> -                                            "%s,", temp);
> -                               offset = offset + sbi_strlen(temp) + 1;
> -                       }
> +       for_each_set_bit(ext, hfeatures->extensions, SBI_HART_EXT_MAX) {
> +               temp = sbi_hart_extension_id2string(ext);
> +               if (temp) {
> +                       sbi_snprintf(extensions_str + offset,
> +                                    nestr - offset,
> +                                    "%s,", temp);
> +                       offset = offset + sbi_strlen(temp) + 1;
>                 }
> +       }
>
> -               ext++;
> -       } while (ext < SBI_HART_EXT_MAX);
> -
> -done:
>         if (offset)
>                 extensions_str[offset - 1] = '\0';
>         else
> @@ -799,7 +791,7 @@ static int hart_detect_features(struct sbi_scratch *scratch)
>                 return 0;
>
>         /* Clear hart features */
> -       hfeatures->extensions = 0;
> +       sbi_memset(hfeatures->extensions, 0, sizeof(hfeatures->extensions));
>         hfeatures->pmp_count = 0;
>         hfeatures->mhpm_mask = 0;
>
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 66a0b77..9182faa 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -215,7 +215,7 @@ static int generic_extensions_init(struct sbi_hart_features *hfeatures)
>
>         /* Parse the ISA string from FDT and enable the listed extensions */
>         rc = fdt_parse_isa_extensions(fdt_get_address(), current_hartid(),
> -                                     &hfeatures->extensions);
> +                                     hfeatures->extensions);
>
>         if (rc)
>                 return rc;
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From minda.chen at starfivetech.com  Fri Nov 17 01:58:16 2023
From: minda.chen at starfivetech.com (Minda Chen)
Date: Fri, 17 Nov 2023 17:58:16 +0800
Subject: Security and correctness issues around captured FDT pointers
In-Reply-To: <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>
References: <4caf2c9d-36ba-c1e8-8e38-ec733a41e74a@gmail.com>
 <CAK9=C2W3d1t+83kNoWhkA81M59Wen2gGa_EcgOJA1U19WJP4mQ@mail.gmail.com>
 <0a3267fa-3dfe-f630-3202-8f6ed6825609@gmail.com>
Message-ID: <49904cce-9b85-462f-929b-6002ef5fb375@starfivetech.com>



On 2023/11/16 8:25, Bo Gan wrote:
> On 11/13/23 9:29 AM, Anup Patel wrote:
>>
>> Yes, #1 is mostly safe.
>>
>> The #2 needs to be fixed in the generic/starfive/jh7110.c and all other places
>> where it exists.
>>
>> We should certainly stop accessing FDT after cold boot hard is done but I don't
>> think we should zero-out pointers as well.
>>
> Thanks for the reply, Anup. I did a little bit more digging on this and I think
> #1 is only safe when OpenSBI's carrying the FDT itself (through FW_FDT_PATH).
> Otherwise, we have to make sure FDT will never be accessed after cold boot done.
> By zeroing-out the pointers, we can avoid speculative access as well. This helps
> to defense against speculative timing attacks. (Maybe I'm too paranoid?).
> 
> Given OpenSBI starts at very early stage, it's probably uncommon to not have
> FW_FDT_PATH defined. I have a JH7110 on hand, and the OpenSBI (loaded by U-Boot
> SPL) will carry the FDT built from U-Boot. I'd imagine other boards/vendors do
> it similarly. Just in case folks really don't have FW_FDT_PATH, can we emit a
> warning, and suggest the FDT to be protected through other mechanism?
> 
> For #2, can @Minda Chen or @Wei Liang help take a look? I see no reason to have
> the `const char *compatible` field in `struct pmic`. The compatible string is
> always going to be "stf,axp15060-regulator" in the current implementation, so
> what's point of strcmp it again in `pmic_ops`? If it's for future extension,
> can you strcmp it inside `pm_reset_init`, and use an enum to indicate the model?
> 
> Bo
Now just one pmic now. the compatible can be deleted.

But I don't know what are you changed and 1#  2# mean. I can not find your patch in opensbi mail website. 


From anup at brainfault.org  Fri Nov 17 02:29:54 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 15:59:54 +0530
Subject: [PATCH 1/2] lib: sbi: Make console_puts/console_putc interchangeable
In-Reply-To: <20231117010440.105759-1-wxjstz@126.com>
References: <20231117010440.105759-1-wxjstz@126.com>
Message-ID: <CAAhSdy0=Yq-HaZT9oSgp_f1R0cNFjbbRBLD3RJHEO6+r6EVxOQ@mail.gmail.com>

On Fri, Nov 17, 2023 at 6:34?AM Xiang W <wxjstz at 126.com> wrote:
>
> console_puts/console_putc should replace each other, but the previous
> sbi_putc can only use console_putc. This patch improves this problem.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Guo Ren <guoren at kerenl.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_console.c | 35 +++++++++++++++++------------------
>  1 file changed, 17 insertions(+), 18 deletions(-)
>
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index 2b3b0a3..ab09a5c 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -37,28 +37,22 @@ int sbi_getc(void)
>         return -1;
>  }
>
> -void sbi_putc(char ch)
> -{
> -       if (console_dev && console_dev->console_putc) {
> -               if (ch == '\n')
> -                       console_dev->console_putc('\r');
> -               console_dev->console_putc(ch);
> -       }
> -}
> -
>  static unsigned long nputs(const char *str, unsigned long len)
>  {
> -       unsigned long i, ret;
> +       unsigned long i;
>
> -       if (console_dev && console_dev->console_puts) {
> -               ret = console_dev->console_puts(str, len);
> -       } else {
> -               for (i = 0; i < len; i++)
> -                       sbi_putc(str[i]);
> -               ret = len;
> +       if (console_dev) {
> +               if (console_dev->console_puts)
> +                       return console_dev->console_puts(str, len);
> +               else if (console_dev->console_putc) {
> +                       for (i = 0; i < len; i++) {
> +                               if (str[i] == '\n')
> +                                       console_dev->console_putc('\r');
> +                               console_dev->console_putc(str[i]);
> +                       }
> +               }
>         }
> -
> -       return ret;
> +       return len;
>  }
>
>  static void nputs_all(const char *str, unsigned long len)
> @@ -69,6 +63,11 @@ static void nputs_all(const char *str, unsigned long len)
>                 p += nputs(&str[p], len - p);
>  }
>
> +void sbi_putc(char ch)
> +{
> +       nputs_all(&ch, 1);
> +}
> +
>  void sbi_puts(const char *str)
>  {
>         unsigned long len = sbi_strlen(str);
> --
> 2.42.0
>


From anup at brainfault.org  Fri Nov 17 02:30:45 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 16:00:45 +0530
Subject: [PATCH 2/2] lib: utils/serial: remove semihosting_putc
In-Reply-To: <20231117010440.105759-2-wxjstz@126.com>
References: <20231117010440.105759-1-wxjstz@126.com> <20231117010440.105759-2-wxjstz@126.com>
Message-ID: <CAAhSdy1+g2qS1_30=g5TP7sXPY2swf5MhSHP8v6oJ6+BHHx2jQ@mail.gmail.com>

On Fri, Nov 17, 2023 at 6:34?AM Xiang W <wxjstz at 126.com> wrote:
>
> For some debuggers that do not implement SYSWRITEC and SYSREADC
> operations, we can use SYSWRITE and SYSREAD instead like the
> implementation of semihosting_putc(). This makes
> semihosting_putc/semihosting_puts similar. Since
> console_putc/console_puts are now interchangeable, removing the
> semihosting_putc.

This commit description needs to be re-written. I have taken care at
the time of merging this patch.

>
> Signed-off-by: Chen Pei <cp0613 at linux.alibaba.com>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Guo Ren <guoren at kerenl.org>

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/utils/serial/semihosting.c | 6 ------
>  1 file changed, 6 deletions(-)
>
> diff --git a/lib/utils/serial/semihosting.c b/lib/utils/serial/semihosting.c
> index a27c69e..19ebaa0 100644
> --- a/lib/utils/serial/semihosting.c
> +++ b/lib/utils/serial/semihosting.c
> @@ -178,11 +178,6 @@ static unsigned long semihosting_puts(const char *str, unsigned long len)
>         return (ret < 0) ? 0 : ret;
>  }
>
> -static void semihosting_putc(char ch)
> -{
> -       semihosting_puts(&ch, 1);
> -}
> -
>  static int semihosting_getc(void)
>  {
>         char ch = 0;
> @@ -199,7 +194,6 @@ static int semihosting_getc(void)
>
>  static struct sbi_console_device semihosting_console = {
>         .name = "semihosting",
> -       .console_putc = semihosting_putc,
>         .console_puts = semihosting_puts,
>         .console_getc = semihosting_getc
>  };
> --
> 2.42.0
>


From anup at brainfault.org  Fri Nov 17 02:38:30 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 16:08:30 +0530
Subject: [PATCH] lib: utils/fdt: Allow to use reg-names when parsing ACLINT
In-Reply-To: <IA1PR20MB4953124B319AEF3573200E09BBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953124B319AEF3573200E09BBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAAhSdy2x9Gk4NYv9S7Ao4aFhuiFijJmtDDO_-GZxJ6XARyC12w@mail.gmail.com>

On Fri, Nov 17, 2023 at 10:34?AM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Currently, the fdt_parse_aclint_node() follows a fixed order to parse
> ACLINT timer. This may cause the undesirable result when the ACLINT
> device does not support mtime without adding an empty entry for it in
> the DT.
>
> To be robust, make fdt_parse_aclint_node() support "reg-names" property,
> so it can parse the DT in an order independent way. For compatibility,
> fdt_parse_aclint_node() only use "reg-names" when parsing ACLINT timer,
> and will fallback to the old way if "reg-names" property is not found.
>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> Link: https://lore.kernel.org/all/20231114-skedaddle-precinct-66c8897227bb at squawk/
> ---
>  lib/utils/fdt/fdt_helper.c | 107 +++++++++++++++++++++++++++++++------
>  1 file changed, 91 insertions(+), 16 deletions(-)
>
> diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
> index 9ae7f09..0f42d68 100644
> --- a/lib/utils/fdt/fdt_helper.c
> +++ b/lib/utils/fdt/fdt_helper.c
> @@ -216,6 +216,32 @@ int fdt_get_node_addr_size(void *fdt, int node, int index,
>         return 0;
>  }
>
> +int fdt_get_node_addr_size_by_name(void *fdt, int node, const char *name,
> +                                  uint64_t *addr, uint64_t *size)

Declare fdt_get_node_addr_size_by_name() in fdt_helper.h

> +{
> +       int i, j, count;
> +       const char *val;
> +       const char *regname;
> +
> +       if (!fdt || node < 0 || !name)
> +               return SBI_EINVAL;
> +
> +       val = fdt_getprop(fdt, node, "reg-names", &count);
> +       if (!val)
> +               return SBI_ENODEV;
> +
> +       for (i = 0, j = 0; i < count; i++, j++) {
> +               regname = val + i;
> +
> +               if (strcmp(name, regname) == 0)
> +                       return fdt_get_node_addr_size(fdt, node, j, addr, size);
> +
> +               i += strlen(regname);
> +       }
> +
> +       return SBI_ENODEV;
> +}
> +
>  bool fdt_node_is_enabled(void *fdt, int nodeoff)
>  {
>         int len;
> @@ -873,13 +899,66 @@ int fdt_parse_plic(void *fdt, struct plic_data *plic, const char *compat)
>         return fdt_parse_plic_node(fdt, nodeoffset, plic);
>  }
>
> +int fdt_get_aclint_addr_size_by_name(void *fdt, int nodeoffset,
> +                                    unsigned long *out_addr1,
> +                                    unsigned long *out_size1,
> +                                    unsigned long *out_addr2,
> +                                    unsigned long *out_size2)

Make this static.

> +{
> +       int rc;
> +       uint64_t reg_addr, reg_size;
> +
> +       rc = fdt_get_node_addr_size_by_name(fdt, nodeoffset, "mtime",
> +                                           &reg_addr, &reg_size);
> +       if (rc < 0 || !reg_size)
> +               reg_addr = reg_size = 0;
> +       *out_addr1 = reg_addr;
> +       *out_size1 = reg_size;
> +
> +       rc = fdt_get_node_addr_size_by_name(fdt, nodeoffset, "mtimecmp",
> +                                           &reg_addr, &reg_size);
> +       if (rc < 0 || !reg_size)
> +               return SBI_ENODEV;
> +       *out_addr2 = reg_addr;
> +       *out_size2 = reg_size;
> +
> +       return 0;
> +}
> +
> +int fdt_get_aclint_addr_size(void *fdt, int nodeoffset,
> +                            unsigned long *out_addr1,
> +                            unsigned long *out_size1,
> +                            unsigned long *out_addr2,
> +                            unsigned long *out_size2)

Make this static.

> +{
> +       int rc;
> +       uint64_t reg_addr, reg_size;
> +
> +       rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
> +                                   &reg_addr, &reg_size);
> +       if (rc < 0 || !reg_size)
> +               return SBI_ENODEV;
> +       *out_addr1 = reg_addr;
> +       *out_size1 = reg_size;
> +
> +       rc = fdt_get_node_addr_size(fdt, nodeoffset, 1,
> +                                   &reg_addr, &reg_size);
> +       if (rc < 0 || !reg_size)
> +               reg_addr = reg_size = 0;
> +       if (out_addr2)
> +               *out_addr2 = reg_addr;
> +       if (out_size2)
> +               *out_size2 = reg_size;
> +
> +       return 0;
> +}
> +
>  int fdt_parse_aclint_node(void *fdt, int nodeoffset, bool for_timer,
>                           unsigned long *out_addr1, unsigned long *out_size1,
>                           unsigned long *out_addr2, unsigned long *out_size2,
>                           u32 *out_first_hartid, u32 *out_hart_count)
>  {
>         const fdt32_t *val;
> -       uint64_t reg_addr, reg_size;
>         int i, rc, count, cpu_offset, cpu_intc_offset;
>         u32 phandle, hwirq, hartid, first_hartid, last_hartid, hart_count;
>         u32 match_hwirq = (for_timer) ? IRQ_M_TIMER : IRQ_M_SOFT;
> @@ -889,21 +968,17 @@ int fdt_parse_aclint_node(void *fdt, int nodeoffset, bool for_timer,
>             !out_first_hartid || !out_hart_count)
>                 return SBI_EINVAL;
>
> -       rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
> -                                   &reg_addr, &reg_size);
> -       if (rc < 0 || !reg_size)
> -               return SBI_ENODEV;
> -       *out_addr1 = reg_addr;
> -       *out_size1 = reg_size;
> -
> -       rc = fdt_get_node_addr_size(fdt, nodeoffset, 1,
> -                                   &reg_addr, &reg_size);
> -       if (rc < 0 || !reg_size)
> -               reg_addr = reg_size = 0;
> -       if (out_addr2)
> -               *out_addr2 = reg_addr;
> -       if (out_size2)
> -               *out_size2 = reg_size;
> +       if (for_timer && out_addr2 && out_size2 &&
> +           fdt_getprop(fdt, nodeoffset, "reg-names", NULL))
> +               rc = fdt_get_aclint_addr_size_by_name(fdt, nodeoffset,
> +                                                     out_addr1, out_size1,
> +                                                     out_addr2, out_size2);
> +       else
> +               rc = fdt_get_aclint_addr_size(fdt, nodeoffset,
> +                                             out_addr1, out_size1,
> +                                             out_addr2, out_size2);
> +       if (rc)
> +               return rc;
>
>         *out_first_hartid = 0;
>         *out_hart_count = 0;
> --
> 2.42.1
>

Regards,
Anup


From anup at brainfault.org  Fri Nov 17 02:49:56 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 16:19:56 +0530
Subject: [PATCH v2 1/2] lib: sbi: Fix __atomic_op_bit_ord and comments
In-Reply-To: <20231115145929.488427-2-wxjstz@126.com>
References: <20231115145929.488427-1-wxjstz@126.com> <20231115145929.488427-2-wxjstz@126.com>
Message-ID: <CAAhSdy3igwFV7REMOWiZO84xkwuDH53ZUvjRVKTJ8_yPbqGqaw@mail.gmail.com>

On Wed, Nov 15, 2023 at 8:31?PM Xiang W <wxjstz at 126.com> wrote:
>
> The original code returns the value of the word before modification.
> When modifying the upper 32 bits under RV64, the value returned via
> int return will have no meaning. Corrected to return the value of the
> bit. And modify the function description.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  include/sbi/riscv_atomic.h | 8 ++++----
>  lib/sbi/riscv_atomic.c     | 2 +-
>  2 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/include/sbi/riscv_atomic.h b/include/sbi/riscv_atomic.h
> index 3972e0b..c5aa05e 100644
> --- a/include/sbi/riscv_atomic.h
> +++ b/include/sbi/riscv_atomic.h
> @@ -39,14 +39,14 @@ unsigned int atomic_raw_xchg_uint(volatile unsigned int *ptr,
>  unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
>                                     unsigned long newval);
>  /**
> - * Set a bit in an atomic variable and return the new value.
> + * Set a bit in an atomic variable and return the value of bit before modify.
>   * @nr : Bit to set.
>   * @atom: atomic variable to modify
>   */
>  int atomic_set_bit(int nr, atomic_t *atom);
>
>  /**
> - * Clear a bit in an atomic variable and return the new value.
> + * Clear a bit in an atomic variable and return the value of bit before modify.
>   * @nr : Bit to set.
>   * @atom: atomic variable to modify
>   */
> @@ -54,14 +54,14 @@ int atomic_set_bit(int nr, atomic_t *atom);
>  int atomic_clear_bit(int nr, atomic_t *atom);
>
>  /**
> - * Set a bit in any address and return the new value .
> + * Set a bit in any address and return the value of bit before modify.
>   * @nr : Bit to set.
>   * @addr: Address to modify
>   */
>  int atomic_raw_set_bit(int nr, volatile unsigned long *addr);
>
>  /**
> - * Clear a bit in any address and return the new value .
> + * Clear a bit in any address and return the value of bit before modify.
>   * @nr : Bit to set.
>   * @addr: Address to modify
>   */
> diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
> index 528686f..477f709 100644
> --- a/lib/sbi/riscv_atomic.c
> +++ b/lib/sbi/riscv_atomic.c
> @@ -222,7 +222,7 @@ unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
>                                      : "=r"(__res), "+A"(addr[BIT_WORD(nr)]) \
>                                      : "r"(mod(__mask))                      \
>                                      : "memory");                            \
> -               __res;                                                       \
> +               __res & mask ? 1 : 0;                                        \
>         })
>
>  #define __atomic_op_bit(op, mod, nr, addr) \
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Fri Nov 17 02:53:20 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Nov 2023 16:23:20 +0530
Subject: [PATCH v2 2/2] lib: sbi: Replace __atomic_op_bit_ord with
 __atomic intrinsics
In-Reply-To: <20231115145929.488427-3-wxjstz@126.com>
References: <20231115145929.488427-1-wxjstz@126.com> <20231115145929.488427-3-wxjstz@126.com>
Message-ID: <CAAhSdy2qrQDha_dT1184LcFaiuhYe6QS4Uhcn7ot++NKkiHxig@mail.gmail.com>

On Wed, Nov 15, 2023 at 8:30?PM Xiang W <wxjstz at 126.com> wrote:
>
> Simplify atomic-related bit operations through __atomic intrinsics.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/sbi/riscv_atomic.c | 40 ++++++++++++----------------------------
>  1 file changed, 12 insertions(+), 28 deletions(-)
>
> diff --git a/lib/sbi/riscv_atomic.c b/lib/sbi/riscv_atomic.c
> index 477f709..1c852a1 100644
> --- a/lib/sbi/riscv_atomic.c
> +++ b/lib/sbi/riscv_atomic.c
> @@ -206,40 +206,24 @@ unsigned long atomic_raw_xchg_ulong(volatile unsigned long *ptr,
>  #endif
>  }
>
> -#if (__SIZEOF_POINTER__ == 8)
> -#define __AMO(op) "amo" #op ".d"
> -#elif (__SIZEOF_POINTER__ == 4)
> -#define __AMO(op) "amo" #op ".w"
> -#else
> -#error "Unexpected __SIZEOF_POINTER__"
> -#endif
> -
> -#define __atomic_op_bit_ord(op, mod, nr, addr, ord)                          \
> -       ({                                                                   \
> -               unsigned long __res, __mask;                                 \
> -               __mask = BIT_MASK(nr);                                       \
> -               __asm__ __volatile__(__AMO(op) #ord " %0, %2, %1"            \
> -                                    : "=r"(__res), "+A"(addr[BIT_WORD(nr)]) \
> -                                    : "r"(mod(__mask))                      \
> -                                    : "memory");                            \
> -               __res & mask ? 1 : 0;                                        \
> -       })
> -
> -#define __atomic_op_bit(op, mod, nr, addr) \
> -       __atomic_op_bit_ord(op, mod, nr, addr, .aqrl)
> -
> -/* Bitmask modifiers */
> -#define __NOP(x) (x)
> -#define __NOT(x) (~(x))
> -
>  inline int atomic_raw_set_bit(int nr, volatile unsigned long *addr)
>  {
> -       return __atomic_op_bit(or, __NOP, nr, addr);
> +       volatile unsigned long *ptr;
> +       unsigned long res, mask;
> +       mask = BIT_MASK(nr);
> +       ptr = &addr[BIT_WORD(nr)];
> +       res = __atomic_fetch_or(ptr, mask, __ATOMIC_ACQ_REL);
> +       return res & mask ? 1 : 0;
>  }
>
>  inline int atomic_raw_clear_bit(int nr, volatile unsigned long *addr)
>  {
> -       return __atomic_op_bit(and, __NOT, nr, addr);
> +       volatile unsigned long *ptr;
> +       unsigned long res, mask;
> +       mask = BIT_MASK(nr);
> +       ptr = &addr[BIT_WORD(nr)];
> +       res = __atomic_fetch_and(ptr, ~mask, __ATOMIC_ACQ_REL);
> +       return res & mask ? 1 : 0;
>  }
>
>  inline int atomic_set_bit(int nr, atomic_t *atom)
> --
> 2.42.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From inochiama at outlook.com  Fri Nov 17 04:11:00 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Fri, 17 Nov 2023 20:11:00 +0800
Subject: [PATCH] platform: recalculate heap size to support new tlb entry number
Message-ID: <IA1PR20MB4953BC55F07EE072FE9E4288BBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>

Previous patch introduct a change that using hart count as the default
number of tlb entry in the fifo. This makes the default tlb fifo size
grow in square with the number of harts. So the default heap size is
not enough to allocate tlb fifo when the hart count is big.

Use new formula for default heap size:
0x40 * (__num_hart) ^ 2   -> handle the tlb fifo
0x600 * (__num_hart)      -> remove previous static fifo allocation

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
Fixes: 52fd64b ("platform: Uses hart count as the default size of tlb info")
---
 include/sbi/sbi_bitops.h   | 1 +
 include/sbi/sbi_heap.h     | 3 +++
 include/sbi/sbi_platform.h | 8 ++++++--
 lib/sbi/sbi_heap.c         | 2 --
 4 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/include/sbi/sbi_bitops.h b/include/sbi/sbi_bitops.h
index 48a2090..7d90334 100644
--- a/include/sbi/sbi_bitops.h
+++ b/include/sbi/sbi_bitops.h
@@ -26,6 +26,7 @@
 #define BIT_MASK(nr)		(1UL << ((nr) % BITS_PER_LONG))
 #define BIT_WORD(bit)		((bit) / BITS_PER_LONG)
 #define BIT_WORD_OFFSET(bit)	((bit) & (BITS_PER_LONG - 1))
+#define BIT_ALIGN(bit, align)	(((bit) + ((align) - 1)) & ~((align) - 1))
 
 #define GENMASK(h, l) \
 	(((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
diff --git a/include/sbi/sbi_heap.h b/include/sbi/sbi_heap.h
index 88d176e..16755ec 100644
--- a/include/sbi/sbi_heap.h
+++ b/include/sbi/sbi_heap.h
@@ -12,6 +12,9 @@
 
 #include <sbi/sbi_types.h>
 
+/* Alignment of heap base address and size */
+#define HEAP_BASE_ALIGN			1024
+
 struct sbi_scratch;
 
 /** Allocate from heap area */
diff --git a/include/sbi/sbi_platform.h b/include/sbi/sbi_platform.h
index 58b9069..609f608 100644
--- a/include/sbi/sbi_platform.h
+++ b/include/sbi/sbi_platform.h
@@ -44,8 +44,10 @@
 
 #ifndef __ASSEMBLER__
 
+#include <sbi/sbi_bitops.h>
 #include <sbi/sbi_ecall_interface.h>
 #include <sbi/sbi_error.h>
+#include <sbi/sbi_heap.h>
 #include <sbi/sbi_scratch.h>
 #include <sbi/sbi_version.h>
 
@@ -146,8 +148,10 @@ struct sbi_platform_operations {
 #define SBI_PLATFORM_DEFAULT_HART_STACK_SIZE	8192
 
 /** Platform default heap size */
-#define SBI_PLATFORM_DEFAULT_HEAP_SIZE(__num_hart)	\
-					(0x8000 + 0x800 * (__num_hart))
+#define SBI_PLATFORM_DEFAULT_HEAP_SIZE(__num_hart)		\
+	BIT_ALIGN((0x8000 + 0x600 * (__num_hart) +		\
+		   0x40 * (__num_hart) * (__num_hart)),		\
+		  HEAP_BASE_ALIGN)
 
 /** Representation of a platform */
 struct sbi_platform {
diff --git a/lib/sbi/sbi_heap.c b/lib/sbi/sbi_heap.c
index 698c377..bcd404b 100644
--- a/lib/sbi/sbi_heap.c
+++ b/lib/sbi/sbi_heap.c
@@ -14,8 +14,6 @@
 #include <sbi/sbi_scratch.h>
 #include <sbi/sbi_string.h>
 
-/* Alignment of heap base address and size */
-#define HEAP_BASE_ALIGN			1024
 /* Minimum size and alignment of heap allocations */
 #define HEAP_ALLOC_ALIGN		64
 #define HEAP_HOUSEKEEPING_FACTOR	16
-- 
2.42.1



From inochiama at outlook.com  Fri Nov 17 04:13:09 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Fri, 17 Nov 2023 20:13:09 +0800
Subject: [PATCH v2] lib: utils/fdt: Allow to use reg-names when parsing ACLINT
Message-ID: <IA1PR20MB49532E6C3653C0F3EC3EFCDEBBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>

Currently, the fdt_parse_aclint_node() follows a fixed order to parse
ACLINT timer. This may cause the undesirable result when the ACLINT
device does not support mtime without adding an empty entry for it in
the DT.

To be robust, make fdt_parse_aclint_node() support "reg-names" property,
so it can parse the DT in an order independent way. For compatibility,
fdt_parse_aclint_node() only use "reg-names" when parsing ACLINT timer,
and will fallback to the old way if "reg-names" property is not found.

Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
Link: https://lore.kernel.org/all/20231114-skedaddle-precinct-66c8897227bb at squawk/
---
Changed from v1:
1. add fdt_get_node_addr_size_by_name to fdt_helper.h
2. make fdt_get_aclint_addr_size_by_name, fdt_get_aclint_addr_size static
---
 include/sbi_utils/fdt/fdt_helper.h |   3 +
 lib/utils/fdt/fdt_helper.c         | 107 ++++++++++++++++++++++++-----
 2 files changed, 94 insertions(+), 16 deletions(-)

diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
index 5c928ff..430d818 100644
--- a/include/sbi_utils/fdt/fdt_helper.h
+++ b/include/sbi_utils/fdt/fdt_helper.h
@@ -48,6 +48,9 @@ int fdt_parse_phandle_with_args(void *fdt, int nodeoff,
 int fdt_get_node_addr_size(void *fdt, int node, int index,
 			   uint64_t *addr, uint64_t *size);

+int fdt_get_node_addr_size_by_name(void *fdt, int node, const char *name,
+				   uint64_t *addr, uint64_t *size);
+
 bool fdt_node_is_enabled(void *fdt, int nodeoff);

 int fdt_parse_hart_id(void *fdt, int cpu_offset, u32 *hartid);
diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
index 9ae7f09..83299a8 100644
--- a/lib/utils/fdt/fdt_helper.c
+++ b/lib/utils/fdt/fdt_helper.c
@@ -216,6 +216,32 @@ int fdt_get_node_addr_size(void *fdt, int node, int index,
 	return 0;
 }

+int fdt_get_node_addr_size_by_name(void *fdt, int node, const char *name,
+				   uint64_t *addr, uint64_t *size)
+{
+	int i, j, count;
+	const char *val;
+	const char *regname;
+
+	if (!fdt || node < 0 || !name)
+		return SBI_EINVAL;
+
+	val = fdt_getprop(fdt, node, "reg-names", &count);
+	if (!val)
+		return SBI_ENODEV;
+
+	for (i = 0, j = 0; i < count; i++, j++) {
+		regname = val + i;
+
+		if (strcmp(name, regname) == 0)
+			return fdt_get_node_addr_size(fdt, node, j, addr, size);
+
+		i += strlen(regname);
+	}
+
+	return SBI_ENODEV;
+}
+
 bool fdt_node_is_enabled(void *fdt, int nodeoff)
 {
 	int len;
@@ -873,13 +899,66 @@ int fdt_parse_plic(void *fdt, struct plic_data *plic, const char *compat)
 	return fdt_parse_plic_node(fdt, nodeoffset, plic);
 }

+static int fdt_get_aclint_addr_size_by_name(void *fdt, int nodeoffset,
+					    unsigned long *out_addr1,
+					    unsigned long *out_size1,
+					    unsigned long *out_addr2,
+					    unsigned long *out_size2)
+{
+	int rc;
+	uint64_t reg_addr, reg_size;
+
+	rc = fdt_get_node_addr_size_by_name(fdt, nodeoffset, "mtime",
+					    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_size)
+		reg_addr = reg_size = 0;
+	*out_addr1 = reg_addr;
+	*out_size1 = reg_size;
+
+	rc = fdt_get_node_addr_size_by_name(fdt, nodeoffset, "mtimecmp",
+					    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_size)
+		return SBI_ENODEV;
+	*out_addr2 = reg_addr;
+	*out_size2 = reg_size;
+
+	return 0;
+}
+
+static int fdt_get_aclint_addr_size(void *fdt, int nodeoffset,
+				    unsigned long *out_addr1,
+				    unsigned long *out_size1,
+				    unsigned long *out_addr2,
+				    unsigned long *out_size2)
+{
+	int rc;
+	uint64_t reg_addr, reg_size;
+
+	rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
+				    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_size)
+		return SBI_ENODEV;
+	*out_addr1 = reg_addr;
+	*out_size1 = reg_size;
+
+	rc = fdt_get_node_addr_size(fdt, nodeoffset, 1,
+				    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_size)
+		reg_addr = reg_size = 0;
+	if (out_addr2)
+		*out_addr2 = reg_addr;
+	if (out_size2)
+		*out_size2 = reg_size;
+
+	return 0;
+}
+
 int fdt_parse_aclint_node(void *fdt, int nodeoffset, bool for_timer,
 			  unsigned long *out_addr1, unsigned long *out_size1,
 			  unsigned long *out_addr2, unsigned long *out_size2,
 			  u32 *out_first_hartid, u32 *out_hart_count)
 {
 	const fdt32_t *val;
-	uint64_t reg_addr, reg_size;
 	int i, rc, count, cpu_offset, cpu_intc_offset;
 	u32 phandle, hwirq, hartid, first_hartid, last_hartid, hart_count;
 	u32 match_hwirq = (for_timer) ? IRQ_M_TIMER : IRQ_M_SOFT;
@@ -889,21 +968,17 @@ int fdt_parse_aclint_node(void *fdt, int nodeoffset, bool for_timer,
 	    !out_first_hartid || !out_hart_count)
 		return SBI_EINVAL;

-	rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
-				    &reg_addr, &reg_size);
-	if (rc < 0 || !reg_size)
-		return SBI_ENODEV;
-	*out_addr1 = reg_addr;
-	*out_size1 = reg_size;
-
-	rc = fdt_get_node_addr_size(fdt, nodeoffset, 1,
-				    &reg_addr, &reg_size);
-	if (rc < 0 || !reg_size)
-		reg_addr = reg_size = 0;
-	if (out_addr2)
-		*out_addr2 = reg_addr;
-	if (out_size2)
-		*out_size2 = reg_size;
+	if (for_timer && out_addr2 && out_size2 &&
+	    fdt_getprop(fdt, nodeoffset, "reg-names", NULL))
+		rc = fdt_get_aclint_addr_size_by_name(fdt, nodeoffset,
+						      out_addr1, out_size1,
+						      out_addr2, out_size2);
+	else
+		rc = fdt_get_aclint_addr_size(fdt, nodeoffset,
+					      out_addr1, out_size1,
+					      out_addr2, out_size2);
+	if (rc)
+		return rc;

 	*out_first_hartid = 0;
 	*out_hart_count = 0;
--
2.42.1



From heinrich.schuchardt at canonical.com  Fri Nov 17 05:59:31 2023
From: heinrich.schuchardt at canonical.com (Heinrich Schuchardt)
Date: Fri, 17 Nov 2023 14:59:31 +0100
Subject: [PATCH 1/1] lib: sbi_pmu: avoid buffer overflow
Message-ID: <20231117135931.26666-1-heinrich.schuchardt@canonical.com>

total_ctrs is bounded by

    SBI_PMU_FW_CTR_MAX + SBI_PMU_HW_CTR_MAX) == 48

which exceeds BITS_PER_LONG on 32 bit systems.

Iterating over the bits of &cmask results in a buffer overflow when looking
for a bit >= BITS_PER_LONG.

Adjust the iterators in sbi_pmu_ctr_start() and sbi_pmu_ctr_stop()
accordingly.

Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>
---
 lib/sbi/sbi_pmu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index f4c8fc4..185068b 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -445,7 +445,7 @@ int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
 	if (flags & SBI_PMU_START_FLAG_SET_INIT_VALUE)
 		bUpdate = true;
 
-	for_each_set_bit(i, &cmask, total_ctrs) {
+	for_each_set_bit(i, &cmask, BITS_PER_LONG) {
 		cidx = i + cbase;
 		event_idx_type = pmu_ctr_validate(phs, cidx, &event_code);
 		if (event_idx_type < 0)
@@ -540,7 +540,7 @@ int sbi_pmu_ctr_stop(unsigned long cbase, unsigned long cmask,
 	if ((cbase + sbi_fls(cmask)) >= total_ctrs)
 		return SBI_EINVAL;
 
-	for_each_set_bit(i, &cmask, total_ctrs) {
+	for_each_set_bit(i, &cmask, BITS_PER_LONG) {
 		cidx = i + cbase;
 		event_idx_type = pmu_ctr_validate(phs, cidx, &event_code);
 		if (event_idx_type < 0)
-- 
2.40.1



From inochiama at outlook.com  Fri Nov 17 23:28:26 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Sat, 18 Nov 2023 15:28:26 +0800
Subject: [PATCH v2] lib: utils/fdt: Allow to use reg-names when parsing ACLINT
In-Reply-To: <IA1PR20MB49532E6C3653C0F3EC3EFCDEBBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49532E6C3653C0F3EC3EFCDEBBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <IA1PR20MB4953E854596EE7512087A2D6BBB6A@IA1PR20MB4953.namprd20.prod.outlook.com>

Hi Anup,

Now It seems that kernel prefers to use T-HEAD aclint timer binding for
the ACLINT draft, so this patch may be heavily modified. As a result,
I suggest to temporarily hold this patch to make sure there is no need to
change.

This related kernel mail is on [1], feel free to give your advice. I will
inform you once everything is confirmed.

[1] https://lore.kernel.org/all/IA1PR20MB4953C82499C5D81D2C6A020BBBB6A at IA1PR20MB4953.namprd20.prod.outlook.com/


From apatel at ventanamicro.com  Sat Nov 18 04:31:18 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Sat, 18 Nov 2023 18:01:18 +0530
Subject: [PATCH v2] lib: utils/fdt: Allow to use reg-names when parsing ACLINT
In-Reply-To: <IA1PR20MB4953E854596EE7512087A2D6BBB6A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB49532E6C3653C0F3EC3EFCDEBBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>
 <IA1PR20MB4953E854596EE7512087A2D6BBB6A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAK9=C2V54JtE6BARadugA=CzcFkR6B8fpEvFgzh5qwNYc0XsXA@mail.gmail.com>

On Sat, Nov 18, 2023 at 12:58?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Hi Anup,
>
> Now It seems that kernel prefers to use T-HEAD aclint timer binding for
> the ACLINT draft, so this patch may be heavily modified. As a result,
> I suggest to temporarily hold this patch to make sure there is no need to
> change.
>
> This related kernel mail is on [1], feel free to give your advice. I will
> inform you once everything is confirmed.
>
> [1] https://lore.kernel.org/all/IA1PR20MB4953C82499C5D81D2C6A020BBBB6A at IA1PR20MB4953.namprd20.prod.outlook.com/

Yes, I was anyway going to wait a few weeks before merging this.

Regards,
Anup

>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From wxjstz at 126.com  Mon Nov 20 01:52:43 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 20 Nov 2023 17:52:43 +0800
Subject: [PATCH 2/2] lib: sbi: Make ipi send/clear and access ipi_type non-reentrant
In-Reply-To: <20231120095301.105772-1-wxjstz@126.com>
References: <20231120095301.105772-1-wxjstz@126.com>
Message-ID: <20231120095301.105772-2-wxjstz@126.com>

Sending/receiving ipi messages and setting ipi_type should be
non-reentrant. If reentrant, unexpected errors may occur.

Signed-off-by: Xiang W <wxjstz at 126.com>
Reported-by: Bo Gan <ganboing at gmail.com>
---
 lib/sbi/sbi_ipi.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 30cb315..8e05d28 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -11,6 +11,7 @@
 #include <sbi/riscv_asm.h>
 #include <sbi/riscv_atomic.h>
 #include <sbi/riscv_barrier.h>
+#include <sbi/riscv_locks.h>
 #include <sbi/sbi_bitops.h>
 #include <sbi/sbi_domain.h>
 #include <sbi/sbi_error.h>
@@ -25,6 +26,7 @@
 
 struct sbi_ipi_data {
 	unsigned long ipi_type;
+	spinlock_t lock;
 };
 
 _Static_assert(
@@ -66,9 +68,11 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,
 	 * Set IPI type on remote hart's scratch area and
 	 * trigger the interrupt
 	 */
+	spin_lock(&ipi_data->lock);
 	atomic_raw_set_bit(event, &ipi_data->ipi_type);
 	smp_wmb();
 	sbi_ipi_raw_send(remote_hartindex);
+	spin_unlock(&ipi_data->lock);
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
 
@@ -220,8 +224,12 @@ void sbi_ipi_process(void)
 	u32 hartindex = sbi_hartid_to_hartindex(current_hartid());
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_RECVD);
+
+	spin_lock(&ipi_data->lock);
 	sbi_ipi_raw_clear(hartindex);
 	ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
+	spin_unlock(&ipi_data->lock);
+
 	ipi_event = 0;
 	while (ipi_type) {
 		if (ipi_type & 1UL) {
@@ -289,6 +297,7 @@ int sbi_ipi_init(struct sbi_scratch *scratch, bool cold_boot)
 
 	ipi_data = sbi_scratch_offset_ptr(scratch, ipi_data_off);
 	ipi_data->ipi_type = 0x00;
+	SPIN_LOCK_INIT(ipi_data->lock);
 
 	/*
 	 * Initialize platform IPI support. This will also clear any
-- 
2.42.0



From wxjstz at 126.com  Mon Nov 20 01:52:42 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 20 Nov 2023 17:52:42 +0800
Subject: [PATCH 1/2] lib: sbi: simplify sbi_ipi_send/sbi_ipi_process
Message-ID: <20231120095301.105772-1-wxjstz@126.com>

Pass sbi_ipi_raw_send/sbi_ipi_raw_clear instead of direct access
to ipi_dev.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_ipi.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 5571b37..30cb315 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -68,9 +68,7 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,
 	 */
 	atomic_raw_set_bit(event, &ipi_data->ipi_type);
 	smp_wmb();
-
-	if (ipi_dev && ipi_dev->ipi_send)
-		ipi_dev->ipi_send(remote_hartindex);
+	sbi_ipi_raw_send(remote_hartindex);
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
 
@@ -222,9 +220,7 @@ void sbi_ipi_process(void)
 	u32 hartindex = sbi_hartid_to_hartindex(current_hartid());
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_RECVD);
-	if (ipi_dev && ipi_dev->ipi_clear)
-		ipi_dev->ipi_clear(hartindex);
-
+	sbi_ipi_raw_clear(hartindex);
 	ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
 	ipi_event = 0;
 	while (ipi_type) {
-- 
2.42.0



From wxjstz at 126.com  Mon Nov 20 01:57:52 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 20 Nov 2023 17:57:52 +0800
Subject: [PATCH 2/2] lib: sbi: Make ipi send/clear and access ipi_type
 non-reentrant
In-Reply-To: <20231120095301.105772-2-wxjstz@126.com>
References: <20231120095301.105772-1-wxjstz@126.com>
	 <20231120095301.105772-2-wxjstz@126.com>
Message-ID: <3f1751b2c8b0437e72543e2d25f8ef6ae40512fe.camel@126.com>

? 2023-11-20???? 17:52 +0800?Xiang W???
> Sending/receiving ipi messages and setting ipi_type should be
> non-reentrant. If reentrant, unexpected errors may occur.
If reentrant, the following case may occur:

                hart A             hart B
send ipi--------->|                  |
                  | clear ipi        | 
                  |                  | set ipi_type
                  |                  |
                  | xchg ipi_type    |
                  |                  |
                  |<-----------------| send ipi
                  |                  |
                  |                  |

For more discussion, please refer to
https://lists.infradead.org/pipermail/opensbi/2023-November/005817.html

Regards,
Xiang W
> 
> Signed-off-by: Xiang W <wxjstz at 126.com>
> Reported-by: Bo Gan <ganboing at gmail.com>
> ---
> ?lib/sbi/sbi_ipi.c | 9 +++++++++
> ?1 file changed, 9 insertions(+)
> 
> diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
> index 30cb315..8e05d28 100644
> --- a/lib/sbi/sbi_ipi.c
> +++ b/lib/sbi/sbi_ipi.c
> @@ -11,6 +11,7 @@
> ?#include <sbi/riscv_asm.h>
> ?#include <sbi/riscv_atomic.h>
> ?#include <sbi/riscv_barrier.h>
> +#include <sbi/riscv_locks.h>
> ?#include <sbi/sbi_bitops.h>
> ?#include <sbi/sbi_domain.h>
> ?#include <sbi/sbi_error.h>
> @@ -25,6 +26,7 @@
> ?
> ?struct sbi_ipi_data {
> ?	unsigned long ipi_type;
> +	spinlock_t lock;
> ?};
> ?
> ?_Static_assert(
> @@ -66,9 +68,11 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,
> ?	 * Set IPI type on remote hart's scratch area and
> ?	 * trigger the interrupt
> ?	 */
> +	spin_lock(&ipi_data->lock);
> ?	atomic_raw_set_bit(event, &ipi_data->ipi_type);
> ?	smp_wmb();
> ?	sbi_ipi_raw_send(remote_hartindex);
> +	spin_unlock(&ipi_data->lock);
> ?
> ?	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
> ?
> @@ -220,8 +224,12 @@ void sbi_ipi_process(void)
> ?	u32 hartindex = sbi_hartid_to_hartindex(current_hartid());
> ?
> ?	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_RECVD);
> +
> +	spin_lock(&ipi_data->lock);
> ?	sbi_ipi_raw_clear(hartindex);
> ?	ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
> +	spin_unlock(&ipi_data->lock);
> +
> ?	ipi_event = 0;
> ?	while (ipi_type) {
> ?		if (ipi_type & 1UL) {
> @@ -289,6 +297,7 @@ int sbi_ipi_init(struct sbi_scratch *scratch, bool cold_boot)
> ?
> ?	ipi_data = sbi_scratch_offset_ptr(scratch, ipi_data_off);
> ?	ipi_data->ipi_type = 0x00;
> +	SPIN_LOCK_INIT(ipi_data->lock);
> ?
> ?	/*
> ?	 * Initialize platform IPI support. This will also clear any



From apatel at ventanamicro.com  Mon Nov 20 09:47:02 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 20 Nov 2023 23:17:02 +0530
Subject: [PATCH] lib: sbi: Fix PMP granularity handling in sbi_hart_map_saddr()
Message-ID: <20231120174702.812200-1-apatel@ventanamicro.com>

The sbi_hart_map_saddr() must create PMP mapping of size greater
than or equal to PMP granularity otherwise PMP mapping does not
work when size parameter less than sbi_hart_pmp_granularity(scratch).

Fixes: 6e44ef686a9b ("lib: sbi: Add functions to map/unmap shared
memory")
Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 lib/sbi/sbi_hart.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index 77eef49..bf8792a 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -508,7 +508,8 @@ int sbi_hart_map_saddr(unsigned long addr, unsigned long size)
 	if (is_pmp_entry_mapped(SBI_SMEPMP_RESV_ENTRY))
 		return SBI_ENOSPC;
 
-	for (order = log2roundup(size) ; order <= __riscv_xlen; order++) {
+	for (order = log2roundup(MAX(sbi_hart_pmp_granularity(scratch), size));
+	     order <= __riscv_xlen; order++) {
 		if (order < __riscv_xlen) {
 			base = addr & ~((1UL << order) - 1UL);
 			if ((base <= addr) &&
-- 
2.34.1



From apatel at ventanamicro.com  Mon Nov 20 09:49:15 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 20 Nov 2023 23:19:15 +0530
Subject: [PATCH] lib: sbi: Fix PMP granularity handling in sbi_hart_map_saddr()
Message-ID: <20231120174915.812345-1-apatel@ventanamicro.com>

The sbi_hart_map_saddr() must create PMP mapping of size greater
than or equal to PMP granularity otherwise PMP mapping does not
work when size parameter less than sbi_hart_pmp_granularity(scratch).

Fixes: 6e44ef686a9b ("lib: sbi: Add functions to map/unmap shared
memory")
Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 lib/sbi/sbi_hart.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index 77eef49..bf8792a 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -508,7 +508,8 @@ int sbi_hart_map_saddr(unsigned long addr, unsigned long size)
 	if (is_pmp_entry_mapped(SBI_SMEPMP_RESV_ENTRY))
 		return SBI_ENOSPC;
 
-	for (order = log2roundup(size) ; order <= __riscv_xlen; order++) {
+	for (order = log2roundup(MAX(sbi_hart_pmp_granularity(scratch), size));
+	     order <= __riscv_xlen; order++) {
 		if (order < __riscv_xlen) {
 			base = addr & ~((1UL << order) - 1UL);
 			if ((base <= addr) &&
-- 
2.34.1



From apatel at ventanamicro.com  Mon Nov 20 20:34:01 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Tue, 21 Nov 2023 10:04:01 +0530
Subject: Question about potential missed IPI events
In-Reply-To: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
Message-ID: <CAK9=C2W4kTpJUiV99XPMMCtT15E3CTsHwzt7DjMjvpBrKC+TeQ@mail.gmail.com>

On Sat, Nov 11, 2023 at 11:36?AM Bo Gan <ganboing at gmail.com> wrote:
>
> Hi OpenSBI community,
>
> Current IPI send/recv implementation uses ipi_data->ipi_type to indicate
> an IPI event pending. In sbi_ipi_send, I see have the following:
>
> > /*
> >  * Set IPI type on remote hart's scratch area and
> >  * trigger the interrupt
> >  */
> > atomic_raw_set_bit(event, &ipi_data->ipi_type);
> > smp_wmb();
> >
> > if (ipi_dev && ipi_dev->ipi_send)
> >       ipi_dev->ipi_send(remote_hartindex)
>
> And in sbi_ipi_process, the following:
>
> > if (ipi_dev && ipi_dev->ipi_clear)
> >       ipi_dev->ipi_clear(hartindex);
> >
> > ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
> > ipi_event = 0;
> > while (ipi_type) {
>
> I'm think of if there's any possibility of missed IPIs. Assuming we are
> talking about sifive/clint (aclint_mswi.c), I know it means that there're
> both smp_wmb (fence w,w) and __io_bw (fence w,o) between atomic_raw_set_bit
> and the write to remote hart's msip register in clint. Consider the following
>
> Right before Hart A ipi_dev->ipi_clear() for the msip write-1 issued by hart B,

This means hart A is already in sbi_ipi_process() and has not yet read
its own ipi_type using atomic_raw_xchg_ulong().

This also means hart B has issued smp_wmb() which is before ipi_send()
hence the changes in hart A ipi_type done by hart B must be visible to
hart A.

> Hart C did an ipi_dev->ipi_send to hart A. Hart A then cleared the msip bit,

This also means hart C has issued smp_wmb() which is before ipi_send()
hence the changes in hart A ipi_type done by hart C must be visible to
hart A.

> resulting in both IPI (from B and C) coalesced to a single one. In this case,

This is perfectly fine.

We can have N harts (N > 1) do ipi_send() to hart A just before hart A
does ipi_clear(). The fact that hart A is about to do ipi_clear() it means
hart A is already in sbi_ipi_process() and it is going to atomically
read-n-clear its own ipi_type immediately after ipi_clear() so hart A
will not miss IPI from any of the N harts.

> A will observe the ipi_type set by B, due to the smp_wmb + __io_bw, and the IPI
> was indeed generated by B's write to msip. However will the barrier ensure A
> also observe the write to ipi_type from hart C? For the coalescing to happen,

This is an incorrect understanding of ipi_type. The ipi_type is
in-fact a bitmask
of different IPIs muxed over a single physical IPI. Further, we are using atomic
operations to guarantee multiple harts can safely update ipi_type.

> clint will observe C's write-1 before A's write-0, does it imply A'll observe
> C's write to ipi_type from xchg?

Yes, xchg() called by hart A will observe hart C's changes to ipi_type because
we have smp_wmb() just before ipi_send().

>
> I doubt it mainly due to 2 reasons:
>
> 1. ipi_dev->ipi_clear is a writel(msip), which has __io_bw (fence w,o) I don't
>     think it helps preventing the writel to happen before xchg. If that happens,
>     it's possible that A missed C's write to ipi_type.
>
> 2. Suppose the ipi_clear() happened before xchg, from A's standpoint, it's a
>     write, not a read. A wasn't reading from msip, but only writing to it. I'm
>     not sure if A's write to msip after C's write (observed by clint) implies
>     A's observing the same order. A is not reading from msip, so what helps to
>     establish this order? Will the RVWMO global memory order help? Can we think
>     of clint as another hart that follows RVWMO?

The clint HW is supposed to handle writes coming from multiple
harts so we can safely assume that multiple writes reaching to
clint will be completed in the order in which clint observes them.

Things can only break if hart A does xchg() out-of-order before
writel() issued by ipi_clear() completes.

This can be understood by an example which is as follows:
1) hart A orders xchg() before writel() issued by ipi_clear()
2) hart B is about to do set_bit() for hart A ipi_type but
    hart A does xchg() before hart B does set_bit() so
    hart A will not observe anything in ipi_type
3) hart B does set_bit() and ipi_send() before hart A does
    writel() from ipi_clear()

The above result is hart A totally missing IPI from hart B but
this is only possible if hart A orders xchg() before writel() from
ipi_clear(). This is a very very rare scenario not observed on
any RISC-V implementations so far but to prevent this we
need a matching smp_wmb() between ipi_clear() and xchg()
in sbi_ipi_process().

>
> Bo
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From ganboing at gmail.com  Mon Nov 20 21:21:34 2023
From: ganboing at gmail.com (Bo Gan)
Date: Mon, 20 Nov 2023 21:21:34 -0800
Subject: Question about potential missed IPI events
In-Reply-To: <CAK9=C2W4kTpJUiV99XPMMCtT15E3CTsHwzt7DjMjvpBrKC+TeQ@mail.gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
 <CAK9=C2W4kTpJUiV99XPMMCtT15E3CTsHwzt7DjMjvpBrKC+TeQ@mail.gmail.com>
Message-ID: <9e7edfb2-626d-1232-1c6b-f02fb8fa5131@gmail.com>

On 11/20/23 8:34 PM, Anup Patel wrote:
>>
>> I doubt it mainly due to 2 reasons:
>>
>> 1. ipi_dev->ipi_clear is a writel(msip), which has __io_bw (fence w,o) I don't
>>      think it helps preventing the writel to happen before xchg. If that happens,
>>      it's possible that A missed C's write to ipi_type.
>>
>> 2. Suppose the ipi_clear() happened before xchg, from A's standpoint, it's a
>>      write, not a read. A wasn't reading from msip, but only writing to it. I'm
>>      not sure if A's write to msip after C's write (observed by clint) implies
>>      A's observing the same order. A is not reading from msip, so what helps to
>>      establish this order? Will the RVWMO global memory order help? Can we think
>>      of clint as another hart that follows RVWMO?
> 
> The clint HW is supposed to handle writes coming from multiple
> harts so we can safely assume that multiple writes reaching to
> clint will be completed in the order in which clint observes them.
> 
> Things can only break if hart A does xchg() out-of-order before
> writel() issued by ipi_clear() completes.
> 
> This can be understood by an example which is as follows:
> 1) hart A orders xchg() before writel() issued by ipi_clear()
> 2) hart B is about to do set_bit() for hart A ipi_type but
>      hart A does xchg() before hart B does set_bit() so
>      hart A will not observe anything in ipi_type
> 3) hart B does set_bit() and ipi_send() before hart A does
>      writel() from ipi_clear()
> 
> The above result is hart A totally missing IPI from hart B but
> this is only possible if hart A orders xchg() before writel() from
> ipi_clear(). This is a very very rare scenario not observed on
> any RISC-V implementations so far but to prevent this we
> need a matching smp_wmb() between ipi_clear() and xchg()
> in sbi_ipi_process().
> 

Hi Anup,

Thank you so much for the detailed explanation. I totally get your point.
I just have one doubt about smp_wmb. In OpenSBI, smp_wmb is RISCV_FENCE(w,w).
If I understand correctly, it won't enforce MMIO order. Given your analysis,
I assume you meant wmb(), which is RISCV_FENCE(ow,ow). Am I right? Or do we
want more precise fences, such as:

  RISCV_FENCE(w,o) between set_bit(ipi_type) and ipi_dev->ipi_send in Hart B
  RISCV_FENCE(o,w) between ipi_dev->ipi_clear and xchg(ipi_type) in Hart A

Let me know your thoughts, and I can make a patch and send to mailing list.

Bo



From anup at brainfault.org  Mon Nov 20 21:31:57 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Nov 2023 11:01:57 +0530
Subject: Question about potential missed IPI events
In-Reply-To: <9e7edfb2-626d-1232-1c6b-f02fb8fa5131@gmail.com>
References: <35016005-5bf6-d80b-2a5f-6803afea729e@gmail.com>
 <CAK9=C2W4kTpJUiV99XPMMCtT15E3CTsHwzt7DjMjvpBrKC+TeQ@mail.gmail.com> <9e7edfb2-626d-1232-1c6b-f02fb8fa5131@gmail.com>
Message-ID: <CAAhSdy3CdRxqwPMLzZN=Yj5GBHYVX8B=CSuSvn5-EJvKCuwLHw@mail.gmail.com>

On Tue, Nov 21, 2023 at 10:51?AM Bo Gan <ganboing at gmail.com> wrote:
>
> On 11/20/23 8:34 PM, Anup Patel wrote:
> >>
> >> I doubt it mainly due to 2 reasons:
> >>
> >> 1. ipi_dev->ipi_clear is a writel(msip), which has __io_bw (fence w,o) I don't
> >>      think it helps preventing the writel to happen before xchg. If that happens,
> >>      it's possible that A missed C's write to ipi_type.
> >>
> >> 2. Suppose the ipi_clear() happened before xchg, from A's standpoint, it's a
> >>      write, not a read. A wasn't reading from msip, but only writing to it. I'm
> >>      not sure if A's write to msip after C's write (observed by clint) implies
> >>      A's observing the same order. A is not reading from msip, so what helps to
> >>      establish this order? Will the RVWMO global memory order help? Can we think
> >>      of clint as another hart that follows RVWMO?
> >
> > The clint HW is supposed to handle writes coming from multiple
> > harts so we can safely assume that multiple writes reaching to
> > clint will be completed in the order in which clint observes them.
> >
> > Things can only break if hart A does xchg() out-of-order before
> > writel() issued by ipi_clear() completes.
> >
> > This can be understood by an example which is as follows:
> > 1) hart A orders xchg() before writel() issued by ipi_clear()
> > 2) hart B is about to do set_bit() for hart A ipi_type but
> >      hart A does xchg() before hart B does set_bit() so
> >      hart A will not observe anything in ipi_type
> > 3) hart B does set_bit() and ipi_send() before hart A does
> >      writel() from ipi_clear()
> >
> > The above result is hart A totally missing IPI from hart B but
> > this is only possible if hart A orders xchg() before writel() from
> > ipi_clear(). This is a very very rare scenario not observed on
> > any RISC-V implementations so far but to prevent this we
> > need a matching smp_wmb() between ipi_clear() and xchg()
> > in sbi_ipi_process().
> >
>
> Hi Anup,
>
> Thank you so much for the detailed explanation. I totally get your point.
> I just have one doubt about smp_wmb. In OpenSBI, smp_wmb is RISCV_FENCE(w,w).
> If I understand correctly, it won't enforce MMIO order. Given your analysis,
> I assume you meant wmb(), which is RISCV_FENCE(ow,ow). Am I right? Or do we
> want more precise fences, such as:
>
>   RISCV_FENCE(w,o) between set_bit(ipi_type) and ipi_dev->ipi_send in Hart B
>   RISCV_FENCE(o,w) between ipi_dev->ipi_clear and xchg(ipi_type) in Hart A
>
> Let me know your thoughts, and I can make a patch and send to mailing list.

The smp_wmb() is fine between set_bit(ipi_type) and ipi_dev->ipi_send
but we should use wmb() between ipi_clear() and xchg() in sbi_ipi_process().

>
> Bo
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From peterlin at andestech.com  Mon Nov 20 23:44:14 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 21 Nov 2023 15:44:14 +0800
Subject: [PATCH v2 00/11] Add Andes PMU extension support
In-Reply-To: <CAAhSdy3TRicu-kE=JUesndHwGXmHfCBvAMBcH=Sv8bdqXtOV5g@mail.gmail.com>
References: <20231019113713.3508153-1-peterlin@andestech.com>
 <CAAhSdy3TRicu-kE=JUesndHwGXmHfCBvAMBcH=Sv8bdqXtOV5g@mail.gmail.com>
Message-ID: <ZVxfzg5qYfb53qvL@APC323>

Hi Anup,

On Thu, Nov 16, 2023 at 12:31:03PM +0530, Anup Patel wrote:
> On Thu, Oct 19, 2023 at 5:10?PM Yu Chien Peter Lin
> <peterlin at andestech.com> wrote:
> >
> > This patch series enables perf tool to utilize Andes PMU
> > extension via PMU SBI calls, provides PMU device callbacks
> > to achieve event sampling and mode filtering.
> >
> > This version also introduces a platform override fdt_add_pmu_mappings()
> > to create event mappings before PMU setup if a valid PMU node
> > is missing.
> >
> > The last patch provides a PMU node example used on AX45MP cores.
> >
> > The OpenSBI and Linux patches can be found on Andes Technology GitHub
> > - https://github.com/andestech/opensbi/commits/andes-pmu-support-v2
> > - https://github.com/andestech/linux/commits/andes-pmu-support-v2
> >
> > Yu Chien Peter Lin (11):
> >   sbi: sbi_pmu: Improve sbi_pmu_init() error handling
> >   sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
> >   platform: include: andes45: Add PMU related CSR defines
> >   platform: andes: Add Andes custom PMU support
> >   platform: andes: Enable Andes PMU for AE350
> >   platform: rzfive: Enable Andes PMU for RZ/Five
> >   lib: utils: fdt_fixup: Add fdt_add_pmu_mappings() helper function
> >   lib: utils: fdt_fixup: Allow preserving PMU properties
> >   platform: andes: Factor out is_andes() helper
> >   platform: andes: Implement andes_fdt_add_pmu_mappings platform
> >     override
> >   docs: pmu: Add Andes PMU node example
> 
> It would be great to have v3 sooner because next month we
> have OpenSBI v1.4 release.
> 
> Regards,
> Anup

Thanks for the comments on the series, I will update accordingly
and submit the next version soon.

Best regards,
Peter Lin

> >
> >  docs/pmu_support.md                          |  82 ++++
> >  include/sbi/sbi_ecall_interface.h            |   5 +
> >  include/sbi/sbi_pmu.h                        |   6 +
> >  include/sbi/sbi_scratch.h                    |   2 +
> >  include/sbi_utils/fdt/fdt_fixup.h            |  48 +++
> >  lib/sbi/sbi_pmu.c                            |  10 +-
> >  lib/utils/fdt/fdt_fixup.c                    | 101 ++++-
> >  lib/utils/fdt/fdt_pmu.c                      |   2 +-
> >  platform/generic/Kconfig                     |   4 +
> >  platform/generic/andes/Kconfig               |  11 +
> >  platform/generic/andes/ae350.c               |  31 +-
> >  platform/generic/andes/andes_hpm.c           | 381 +++++++++++++++++++
> >  platform/generic/andes/andes_pmu.c           |  81 ++++
> >  platform/generic/andes/objects.mk            |   2 +
> >  platform/generic/include/andes/andes45.h     |  32 ++
> >  platform/generic/include/andes/andes_hpm.h   |  83 ++++
> >  platform/generic/include/andes/andes_pmu.h   |   8 +
> >  platform/generic/include/platform_override.h |   1 +
> >  platform/generic/platform.c                  |  16 +-
> >  platform/generic/renesas/rzfive/rzfive.c     |  18 +-
> >  20 files changed, 909 insertions(+), 15 deletions(-)
> >  create mode 100644 platform/generic/andes/andes_hpm.c
> >  create mode 100644 platform/generic/andes/andes_pmu.c
> >  create mode 100644 platform/generic/include/andes/andes_hpm.h
> >  create mode 100644 platform/generic/include/andes/andes_pmu.h
> >
> > --
> > 2.34.1
> >


From apatel at ventanamicro.com  Tue Nov 21 07:02:59 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Tue, 21 Nov 2023 20:32:59 +0530
Subject: [PATCH] lib: utils/irqchip: Avoid redundant writes to APLIC CLRIE register
Message-ID: <20231121150259.902907-1-apatel@ventanamicro.com>

Each APLIC CLRIE register allows disabling 32 interrupt sources at
a time by writing -1 so no need to write CLRIE register separately
for each interrupt source.

Fixes: 99792653de29 ("lib: utils/irqchip: Add APLIC initialization
library")
Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 lib/utils/irqchip/aplic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/utils/irqchip/aplic.c b/lib/utils/irqchip/aplic.c
index d7fd992..10236e7 100644
--- a/lib/utils/irqchip/aplic.c
+++ b/lib/utils/irqchip/aplic.c
@@ -193,7 +193,7 @@ int aplic_cold_irqchip_init(struct aplic_data *aplic)
 	writel(0, (void *)(aplic->addr + APLIC_DOMAINCFG));
 
 	/* Disable all interrupts */
-	for (i = 0; i <= aplic->num_source; i++)
+	for (i = 0; i <= aplic->num_source; i += 32)
 		writel(-1U, (void *)(aplic->addr + APLIC_CLRIE_BASE +
 				     (i / 32) * sizeof(u32)));
 
-- 
2.34.1



From apatel at ventanamicro.com  Tue Nov 21 08:17:09 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Tue, 21 Nov 2023 21:47:09 +0530
Subject: [PATCH 0/2] Allow relaxed MMIO writes in device IPI callbacks
Message-ID: <20231121161711.904674-1-apatel@ventanamicro.com>

This series improves the barrier usage around device IPI callbacks
such that platform IPI driver can now use relaxed MMIO writes in
device IPI callbacks.

These patches can also be found in the sbi_ipi_barrier_imp_v1 branch
at: https://github.com/avpatel/opensbi.git

Anup Patel (2):
  lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
  lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback

 lib/sbi/sbi_ipi.c           | 31 +++++++++++++++++++++++++------
 lib/utils/ipi/aclint_mswi.c |  4 ++--
 lib/utils/irqchip/imsic.c   |  4 ++--
 3 files changed, 29 insertions(+), 10 deletions(-)

-- 
2.34.1



From apatel at ventanamicro.com  Tue Nov 21 08:17:10 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Tue, 21 Nov 2023 21:47:10 +0530
Subject: [PATCH 1/2] lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
In-Reply-To: <20231121161711.904674-1-apatel@ventanamicro.com>
References: <20231121161711.904674-1-apatel@ventanamicro.com>
Message-ID: <20231121161711.904674-2-apatel@ventanamicro.com>

Currently, we have a smp_wmb() between atomic_raw_set_bit() and
ipi_send() device callback whereas the MMIO writes done by the
device ipi_send() callback will also include a barrier.

We can avoid unnecessary/redundant barriers described above by
allowing relaxed MMIO writes in device ipi_send() callback. To
achieve this, we simply use  wmb() instead of smp_wmb() before
calling device ipi_send().

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 lib/sbi/sbi_ipi.c           | 15 +++++++++++----
 lib/utils/ipi/aclint_mswi.c |  2 +-
 lib/utils/irqchip/imsic.c   |  4 ++--
 3 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 5571b37..1694a23 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -67,14 +67,12 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,
 	 * trigger the interrupt
 	 */
 	atomic_raw_set_bit(event, &ipi_data->ipi_type);
-	smp_wmb();
 
-	if (ipi_dev && ipi_dev->ipi_send)
-		ipi_dev->ipi_send(remote_hartindex);
+	ret = sbi_ipi_raw_send(remote_hartindex);
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
 
-	return 0;
+	return ret;
 }
 
 static int sbi_ipi_sync(struct sbi_scratch *scratch, u32 event)
@@ -243,6 +241,15 @@ int sbi_ipi_raw_send(u32 hartindex)
 	if (!ipi_dev || !ipi_dev->ipi_send)
 		return SBI_EINVAL;
 
+	/*
+	 * Ensure that memory or MMIO writes done before
+	 * this function are not observed after the memory
+	 * or MMIO writes done by the ipi_send() device
+	 * callback. This also allows the ipi_send() device
+	 * callback to use relaxed MMIO writes.
+	 */
+	wmb();
+
 	ipi_dev->ipi_send(hartindex);
 	return 0;
 }
diff --git a/lib/utils/ipi/aclint_mswi.c b/lib/utils/ipi/aclint_mswi.c
index a3bfb4a..bfd6a45 100644
--- a/lib/utils/ipi/aclint_mswi.c
+++ b/lib/utils/ipi/aclint_mswi.c
@@ -41,7 +41,7 @@ static void mswi_ipi_send(u32 hart_index)
 
 	/* Set ACLINT IPI */
 	msip = (void *)mswi->addr;
-	writel(1, &msip[sbi_hartindex_to_hartid(hart_index) -
+	writel_relaxed(1, &msip[sbi_hartindex_to_hartid(hart_index) -
 			mswi->first_hartid]);
 }
 
diff --git a/lib/utils/irqchip/imsic.c b/lib/utils/irqchip/imsic.c
index 78f5895..36ef66c 100644
--- a/lib/utils/irqchip/imsic.c
+++ b/lib/utils/irqchip/imsic.c
@@ -186,8 +186,8 @@ static void imsic_ipi_send(u32 hart_index)
 	}
 
 	if (regs->size && (reloff < regs->size))
-		writel(IMSIC_IPI_ID,
-		       (void *)(regs->addr + reloff + IMSIC_MMIO_PAGE_LE));
+		writel_relaxed(IMSIC_IPI_ID,
+			(void *)(regs->addr + reloff + IMSIC_MMIO_PAGE_LE));
 }
 
 static struct sbi_ipi_device imsic_ipi_device = {
-- 
2.34.1



From apatel at ventanamicro.com  Tue Nov 21 08:17:11 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Tue, 21 Nov 2023 21:47:11 +0530
Subject: [PATCH 2/2] lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback
In-Reply-To: <20231121161711.904674-1-apatel@ventanamicro.com>
References: <20231121161711.904674-1-apatel@ventanamicro.com>
Message-ID: <20231121161711.904674-3-apatel@ventanamicro.com>

Currently, there are no barriers before or after the ipi_clear()
device callback which forces ipi_clear() device callback to always
use non-relaxed MMIO writes.

Instead of above, we use wmb() in after the ipi_clear() device
callback which pairs with the wmb() done before the ipi_send()
device callback. This also allows device ipi_clear() callback
to use relaxed MMIO writes.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 lib/sbi/sbi_ipi.c           | 16 ++++++++++++++--
 lib/utils/ipi/aclint_mswi.c |  2 +-
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 1694a23..0bf446a 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -220,8 +220,7 @@ void sbi_ipi_process(void)
 	u32 hartindex = sbi_hartid_to_hartindex(current_hartid());
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_RECVD);
-	if (ipi_dev && ipi_dev->ipi_clear)
-		ipi_dev->ipi_clear(hartindex);
+	sbi_ipi_raw_clear(hartindex);
 
 	ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
 	ipi_event = 0;
@@ -247,6 +246,8 @@ int sbi_ipi_raw_send(u32 hartindex)
 	 * or MMIO writes done by the ipi_send() device
 	 * callback. This also allows the ipi_send() device
 	 * callback to use relaxed MMIO writes.
+	 *
+	 * This pairs with the wmb() in sbi_ipi_raw_clear().
 	 */
 	wmb();
 
@@ -258,6 +259,17 @@ void sbi_ipi_raw_clear(u32 hartindex)
 {
 	if (ipi_dev && ipi_dev->ipi_clear)
 		ipi_dev->ipi_clear(hartindex);
+
+	/*
+	 * Ensure that memory or MMIO writes after this
+	 * function returns are not observed before the
+	 * memory or MMIO writes done by the ipi_clear()
+	 * device callback. This also allows ipi_clear()
+	 * device callback to use relaxed MMIO writes.
+	 *
+	 * This pairs with the wmb() in sbi_ipi_raw_send().
+	 */
+	wmb();
 }
 
 const struct sbi_ipi_device *sbi_ipi_get_device(void)
diff --git a/lib/utils/ipi/aclint_mswi.c b/lib/utils/ipi/aclint_mswi.c
index bfd6a45..4ae6bb1 100644
--- a/lib/utils/ipi/aclint_mswi.c
+++ b/lib/utils/ipi/aclint_mswi.c
@@ -61,7 +61,7 @@ static void mswi_ipi_clear(u32 hart_index)
 
 	/* Clear ACLINT IPI */
 	msip = (void *)mswi->addr;
-	writel(0, &msip[sbi_hartindex_to_hartid(hart_index) -
+	writel_relaxed(0, &msip[sbi_hartindex_to_hartid(hart_index) -
 			mswi->first_hartid]);
 }
 
-- 
2.34.1



From wxjstz at 126.com  Tue Nov 21 08:46:52 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 22 Nov 2023 00:46:52 +0800
Subject: [PATCH 0/2] Allow relaxed MMIO writes in device IPI callbacks
In-Reply-To: <20231121161711.904674-1-apatel@ventanamicro.com>
References: <20231121161711.904674-1-apatel@ventanamicro.com>
Message-ID: <b36f2dd2b4e99149394735b9d622d96ad062a4ea.camel@126.com>

? 2023-11-21???? 21:47 +0530?Anup Patel???
> This series improves the barrier usage around device IPI callbacks
> such that platform IPI driver can now use relaxed MMIO writes in
> device IPI callbacks.
> 
> These patches can also be found in the sbi_ipi_barrier_imp_v1 branch
> at: https://github.com/avpatel/opensbi.git
This patch is partly similar to this one?
https://lists.infradead.org/pipermail/opensbi/2023-November/005931.html

Then we may also encounter the following case:

                hart A             hart B
send ipi--------->|                  |
                  | clear ipi        | 
                  |                  | set ipi_type
                  |                  |
                  | xchg ipi_type    |
                  |                  |
                  |<-----------------| send ipi
                  |                  |
                  |                  |

To deal with this case, my patch:
https://lists.infradead.org/pipermail/opensbi/2023-November/005932.html

Regards,
Xiang W
> 
> Anup Patel (2):
> ? lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
> ? lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback
> 
> ?lib/sbi/sbi_ipi.c?????????? | 31 +++++++++++++++++++++++++------
> ?lib/utils/ipi/aclint_mswi.c |? 4 ++--
> ?lib/utils/irqchip/imsic.c?? |? 4 ++--
> ?3 files changed, 29 insertions(+), 10 deletions(-)
> 
> -- 
> 2.34.1
> 
> 



From apatel at ventanamicro.com  Tue Nov 21 09:33:48 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Tue, 21 Nov 2023 23:03:48 +0530
Subject: [PATCH 0/2] Allow relaxed MMIO writes in device IPI callbacks
In-Reply-To: <b36f2dd2b4e99149394735b9d622d96ad062a4ea.camel@126.com>
References: <20231121161711.904674-1-apatel@ventanamicro.com> <b36f2dd2b4e99149394735b9d622d96ad062a4ea.camel@126.com>
Message-ID: <CAK9=C2XMVBJq4g7RxEZQ_k4DuT_v85qBm4UafaueoDFxzdRLyw@mail.gmail.com>

On Tue, Nov 21, 2023 at 10:18?PM Xiang W <wxjstz at 126.com> wrote:
>
> ? 2023-11-21???? 21:47 +0530?Anup Patel???
> > This series improves the barrier usage around device IPI callbacks
> > such that platform IPI driver can now use relaxed MMIO writes in
> > device IPI callbacks.
> >
> > These patches can also be found in the sbi_ipi_barrier_imp_v1 branch
> > at: https://github.com/avpatel/opensbi.git
> This patch is partly similar to this one?
> https://lists.infradead.org/pipermail/opensbi/2023-November/005931.html
>
> Then we may also encounter the following case:
>
>                 hart A             hart B
> send ipi--------->|                  |
>                   | clear ipi        |
>                   |                  | set ipi_type
>                   |                  |
>                   | xchg ipi_type    |
>                   |                  |
>                   |<-----------------| send ipi
>                   |                  |
>                   |                  |

This case is handled.

>
> To deal with this case, my patch:
> https://lists.infradead.org/pipermail/opensbi/2023-November/005932.html

Using spinlock for IPI synchronization is very handed
and reduces performance.

Functionally we have an IPI mux built on-top of one HW IPI
which is functionally similar to the Linux generic IPI-mux.

Regards,
Anup

>
> Regards,
> Xiang W
> >
> > Anup Patel (2):
> >   lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
> >   lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback
> >
> >  lib/sbi/sbi_ipi.c           | 31 +++++++++++++++++++++++++------
> >  lib/utils/ipi/aclint_mswi.c |  4 ++--
> >  lib/utils/irqchip/imsic.c   |  4 ++--
> >  3 files changed, 29 insertions(+), 10 deletions(-)
> >
> > --
> > 2.34.1
> >
> >
>


From samuel.holland at sifive.com  Tue Nov 21 16:30:34 2023
From: samuel.holland at sifive.com (Samuel Holland)
Date: Tue, 21 Nov 2023 18:30:34 -0600
Subject: [PATCH] lib: sbi: Fix PMP granularity handling in
 sbi_hart_map_saddr()
In-Reply-To: <20231120174915.812345-1-apatel@ventanamicro.com>
References: <20231120174915.812345-1-apatel@ventanamicro.com>
Message-ID: <67d181fd-e4b7-47d1-b089-f5b1c4da29b1@sifive.com>

Hi Anup,

On 2023-11-20 11:49 AM, Anup Patel wrote:
> The sbi_hart_map_saddr() must create PMP mapping of size greater
> than or equal to PMP granularity otherwise PMP mapping does not
> work when size parameter less than sbi_hart_pmp_granularity(scratch).
> 
> Fixes: 6e44ef686a9b ("lib: sbi: Add functions to map/unmap shared
> memory")

This line should not be wrapped. Otherwise,

Reviewed-by: Samuel Holland <samuel.holland at sifive.com>

> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> ---
>  lib/sbi/sbi_hart.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 77eef49..bf8792a 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -508,7 +508,8 @@ int sbi_hart_map_saddr(unsigned long addr, unsigned long size)
>  	if (is_pmp_entry_mapped(SBI_SMEPMP_RESV_ENTRY))
>  		return SBI_ENOSPC;
>  
> -	for (order = log2roundup(size) ; order <= __riscv_xlen; order++) {
> +	for (order = log2roundup(MAX(sbi_hart_pmp_granularity(scratch), size));

I noticed that sbi_hart_pmp_granularity() is always passed to log2roundup(), and
hfeatures->pmp_gran is originally generated by shifting, so maybe the value
should be stored as the log base 2 to avoid the conversions?

Regards,
Samuel

> +	     order <= __riscv_xlen; order++) {
>  		if (order < __riscv_xlen) {
>  			base = addr & ~((1UL << order) - 1UL);
>  			if ((base <= addr) &&



From ganboing at gmail.com  Tue Nov 21 16:44:59 2023
From: ganboing at gmail.com (Bo Gan)
Date: Tue, 21 Nov 2023 16:44:59 -0800
Subject: [PATCH 0/2] Allow relaxed MMIO writes in device IPI callbacks
In-Reply-To: <20231121161711.904674-1-apatel@ventanamicro.com>
References: <20231121161711.904674-1-apatel@ventanamicro.com>
Message-ID: <ec81a8a5-f6e7-1301-d3ac-5bb3b09e4ab6@gmail.com>

On 11/21/23 8:17 AM, Anup Patel wrote:
> This series improves the barrier usage around device IPI callbacks
> such that platform IPI driver can now use relaxed MMIO writes in
> device IPI callbacks.
> 
> These patches can also be found in the sbi_ipi_barrier_imp_v1 branch
> at: https://github.com/avpatel/opensbi.git
> 
> Anup Patel (2):
>    lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
>    lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback
> 
>   lib/sbi/sbi_ipi.c           | 31 +++++++++++++++++++++++++------
>   lib/utils/ipi/aclint_mswi.c |  4 ++--
>   lib/utils/irqchip/imsic.c   |  4 ++--
>   3 files changed, 29 insertions(+), 10 deletions(-)
> 
Hi Anup,

Thank you so much for fixing this. I hope my previous discussion helped.
Would you mind having a Reported-by: Bo Gan <ganboing at gmail.com> ?
Really appreciate you helping me understand the code.

Bo


From ganboing at gmail.com  Tue Nov 21 16:56:05 2023
From: ganboing at gmail.com (Bo Gan)
Date: Tue, 21 Nov 2023 16:56:05 -0800
Subject: [PATCH 2/2] lib: sbi: Make ipi send/clear and access ipi_type
 non-reentrant
In-Reply-To: <3f1751b2c8b0437e72543e2d25f8ef6ae40512fe.camel@126.com>
References: <20231120095301.105772-1-wxjstz@126.com>
 <20231120095301.105772-2-wxjstz@126.com>
 <3f1751b2c8b0437e72543e2d25f8ef6ae40512fe.camel@126.com>
Message-ID: <06a9c1b4-43d9-4fea-3001-047d55ec3c75@gmail.com>

>> @@ -220,8 +224,12 @@ void sbi_ipi_process(void)
>>  ?	u32 hartindex = sbi_hartid_to_hartindex(current_hartid());
>>   
>>  ?	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_RECVD);
>> +
>> +	spin_lock(&ipi_data->lock);
>>  ?	sbi_ipi_raw_clear(hartindex);
>>  ?	ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
>> +	spin_unlock(&ipi_data->lock);
>> +
>>  ?	ipi_event = 0;
>>  ?	while (ipi_type) {
>>  ?		if (ipi_type & 1UL) {

Hi Xiang,

I think we should drop this, and use Anup's new patch instead. I don't
think the spinlock here is helping to ensure the ordering between

sbi_ipi_raw_clear and xchg(). Furthermore, spinlock in OpenSBI doesn't
order IO, so ipi_raw_clear can leak below spin_unlock. Thus, it won't
address the race condition in previous discussions.

With Anip's new patch that uses wmb() in both sender and receiver, it
fixes the correctness issue, and hopefully has negligible performance
impact: See the discussion here:

http://lists.infradead.org/pipermail/opensbi/2023-November/005937.html

Bo


From peterlin at andestech.com  Tue Nov 21 23:36:02 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:02 +0800
Subject: [PATCH v3 00/15] Add Andes PMU extension support
Message-ID: <20231122073617.379441-1-peterlin@andestech.com>

This patch series enables perf tool to utilize Andes PMU
extension via PMU SBI calls, provides PMU device callbacks
to achieve event sampling and mode filtering.

This version mainly fixes single-core freeze when quitting
'perf top', adds andes_pmu_setup() to provide default event
mappings when fdt_pmu_setup() can't find PMU node. And add
Xandespmu to hart extension list so that the inhibit bits
can be updated in a similar way to how Smcntrpmf does.

The last patch provides a PMU node example used on AX45MP cores.

The OpenSBI and Linux patches can be found on Andes Technology GitHub
- https://github.com/andestech/opensbi/commits/andes-pmu-support-v3
- https://github.com/andestech/linux/commits/andes-pmu-support-v4

Leo Yu-Chi Liang (1):
  lib: ipi: Adjust Andes PLICSW to single-bit-per-hart scheme

Yu Chien Peter Lin (14):
  sbi: sbi_pmu: Improve sbi_pmu_init() error handling
  lib: sbi: Add Xandespmu in hart extensions
  sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
  platform: include: andes45: Add PMU related CSR defines
  platform: generic: Introduce pmu_init() platform override
  platform: andes: Add Andes custom PMU support
  platform: andes: Enable Andes PMU for AE350
  platform: rzfive: Enable Andes PMU for RZ/Five
  lib: utils: fdt_fixup: Allow preserving PMU properties
  platform: andes: Factor out is_andes() helper
  lib: utils: fdt_pmu: Make the fdt_pmu_evt_select table global variable
  lib: utils: fdt_pmu: Do not iterate over the fdt_pmu_evt_select table
  platform: andes: Add Andes default PMU mapping support
  docs: pmu: Add Andes PMU node example

 docs/pmu_support.md                          |  82 ++++
 include/sbi/sbi_ecall_interface.h            |   5 +
 include/sbi/sbi_hart.h                       |   2 +
 include/sbi/sbi_pmu.h                        |   6 +
 include/sbi/sbi_scratch.h                    |   2 +
 include/sbi_utils/fdt/fdt_helper.h           |  17 +
 include/sbi_utils/fdt/fdt_pmu.h              |   6 +
 include/sbi_utils/ipi/andes_plicsw.h         |  23 +-
 lib/sbi/sbi_hart.c                           |   3 +
 lib/sbi/sbi_pmu.c                            |  25 +-
 lib/utils/fdt/fdt_fixup.c                    |   6 +-
 lib/utils/fdt/fdt_pmu.c                      |  17 +-
 lib/utils/ipi/andes_plicsw.c                 | 104 ++---
 platform/generic/Kconfig                     |   4 +
 platform/generic/andes/Kconfig               |  15 +
 platform/generic/andes/ae350.c               |  19 +-
 platform/generic/andes/andes_hpm.c           | 405 +++++++++++++++++++
 platform/generic/andes/andes_pmu.c           | 102 +++++
 platform/generic/andes/objects.mk            |   2 +
 platform/generic/include/andes/andes45.h     |  32 ++
 platform/generic/include/andes/andes_hpm.h   |  81 ++++
 platform/generic/include/andes/andes_pmu.h   |  33 ++
 platform/generic/include/platform_override.h |   1 +
 platform/generic/platform.c                  |  11 +-
 platform/generic/renesas/rzfive/rzfive.c     |   5 +-
 25 files changed, 897 insertions(+), 111 deletions(-)
 create mode 100644 platform/generic/andes/andes_hpm.c
 create mode 100644 platform/generic/andes/andes_pmu.c
 create mode 100644 platform/generic/include/andes/andes_hpm.h
 create mode 100644 platform/generic/include/andes/andes_pmu.h

-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:03 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:03 +0800
Subject: [PATCH v3 01/15] lib: ipi: Adjust Andes PLICSW to single-bit-per-hart scheme
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-2-peterlin@andestech.com>

From: Leo Yu-Chi Liang <ycliang at andestech.com>

The old scheme doesn't allow sending hart0 self-IPI as the
corresponding bit on pending register is hardwired to 0, this
could lead to unhandle IPIs on SMP systems, esp. on single-core.

Furthermore, the limitation of old scheme is 8-core, instead of
reserving source hart information, we assign bit (x + 1) as the
enable and pending bit of hartx, this also expands the bootable
hart number.

The following diagram shows the enable bits of the new scheme
on 32-core Andes platform.

   Pending regs: 0x1000  x---0---0---0---0------0---0
Pending hart ID:             0   1   2   3 ... 30  31
   Interrupt ID:         0   1   2   3   4 ... 31  32
                         |   |   |   |   |      |   |
    Enable regs: 0x2000  x---1---0---0---0-...--0---0---> hart0
                         |   |   |   |   |      |   |
                 0x2080  x---0---1---0---0-...--0---0---> hart1
                         |   |   |   |   |      |   |
                 0x2100  x---0---0---1---0-...--0---0---> hart2
                         |   |   |   |   |      |   |
                 0x2180  x---0---0---0---1-...--0---0---> hart3
                         .   .   .   .   .      .   .
                         .   .   .   .   .      .   .
                         .   .   .   .   .      .   .
                 0x2f00  x---0---0---0---0-...--1---0---> hart30
                         |   |   |   |   |      |   |
                 0x2f80  x---0---0---0---0-...--0---1---> hart31
                         <-------- word 0 -------><--- word 1 --->

To send IPI to hart0, for example, another hart (including hart0
itself) will set bit 1 of first word on the pending register.

We also fix indentation in andes_plicsw.h along with this patch.

Fixes: ce7c490719ed ("lib: utils/ipi: Add Andes fdt ipi driver support")
Signed-off-by: Leo Yu-Chi Liang <ycliang at andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Randolph <randolph at andestech.com>
Reported-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Link: https://lists.infradead.org/pipermail/opensbi/2023-October/005665.html
---
Changes v2 -> v3:
  - New patch
---
 include/sbi_utils/ipi/andes_plicsw.h |  23 +++---
 lib/utils/ipi/andes_plicsw.c         | 104 ++++++++++-----------------
 2 files changed, 46 insertions(+), 81 deletions(-)

diff --git a/include/sbi_utils/ipi/andes_plicsw.h b/include/sbi_utils/ipi/andes_plicsw.h
index e93cda0..0d18444 100644
--- a/include/sbi_utils/ipi/andes_plicsw.h
+++ b/include/sbi_utils/ipi/andes_plicsw.h
@@ -13,30 +13,23 @@
 #ifndef _IPI_ANDES_PLICSW_H_
 #define _IPI_ANDES_PLICSW_H_
 
-#define PLICSW_PRIORITY_BASE 0x4
+#define PLICSW_PRIORITY_BASE	0x4
 
-#define PLICSW_PENDING_BASE 0x1000
-#define PLICSW_PENDING_STRIDE 0x8
+#define PLICSW_PENDING_BASE	0x1000
 
-#define PLICSW_ENABLE_BASE 0x2000
-#define PLICSW_ENABLE_STRIDE 0x80
+#define PLICSW_ENABLE_BASE	0x2000
+#define PLICSW_ENABLE_STRIDE	0x80
 
-#define PLICSW_CONTEXT_BASE 0x200000
-#define PLICSW_CONTEXT_STRIDE 0x1000
-#define PLICSW_CONTEXT_CLAIM 0x4
+#define PLICSW_CONTEXT_BASE	0x200000
+#define PLICSW_CONTEXT_STRIDE	0x1000
+#define PLICSW_CONTEXT_CLAIM	0x4
 
-#define PLICSW_HART_MASK 0x01010101
-
-#define PLICSW_HART_MAX_NR 8
-
-#define PLICSW_REGION_ALIGN 0x1000
+#define PLICSW_REGION_ALIGN	0x1000
 
 struct plicsw_data {
 	unsigned long addr;
 	unsigned long size;
 	uint32_t hart_count;
-	/* hart id to source id table */
-	uint32_t source_id[PLICSW_HART_MAX_NR];
 };
 
 int plicsw_warm_ipi_init(void);
diff --git a/lib/utils/ipi/andes_plicsw.c b/lib/utils/ipi/andes_plicsw.c
index 5693efb..413ac20 100644
--- a/lib/utils/ipi/andes_plicsw.c
+++ b/lib/utils/ipi/andes_plicsw.c
@@ -18,77 +18,45 @@
 
 struct plicsw_data plicsw;
 
-static inline void plicsw_claim(void)
+static void plicsw_ipi_send(u32 hart_index)
 {
-	u32 hartid = current_hartid();
+	ulong pending_reg;
+	u32 interrupt_id, word_index, pending_bit;
+	u32 target_hart = sbi_hartindex_to_hartid(hart_index);
 
-	if (plicsw.hart_count <= hartid)
+	if (plicsw.hart_count <= target_hart)
 		ebreak();
 
-	plicsw.source_id[hartid] =
-		readl((void *)plicsw.addr + PLICSW_CONTEXT_BASE +
-		      PLICSW_CONTEXT_CLAIM + PLICSW_CONTEXT_STRIDE * hartid);
-}
-
-static inline void plicsw_complete(void)
-{
-	u32 hartid = current_hartid();
-	u32 source = plicsw.source_id[hartid];
-
-	writel(source, (void *)plicsw.addr + PLICSW_CONTEXT_BASE +
-			       PLICSW_CONTEXT_CLAIM +
-			       PLICSW_CONTEXT_STRIDE * hartid);
-}
-
-static inline void plic_sw_pending(u32 target_hart)
-{
 	/*
-	 * The pending array registers are w1s type.
-	 * IPI pending array mapping as following:
-	 *
-	 * Pending array start address: base + 0x1000
-	 * ---------------------------------
-	 * | hart3 | hart2 | hart1 | hart0 |
-	 * ---------------------------------
-	 * Each hartX can send IPI to another hart by setting the
-	 * bitY to its own region (see the below).
-	 *
-	 * In each hartX region:
-	 * <---------- PICSW_PENDING_STRIDE -------->
-	 * | bit7 | ... | bit3 | bit2 | bit1 | bit0 |
-	 * ------------------------------------------
-	 * The bitY of hartX region indicates that hartX sends an
-	 * IPI to hartY.
+	 * We assign a single bit for each hart.
+	 * Bit 0 is hardwired to 0, thus unavailable.
+	 * Bit(X+1) indicates that IPI is sent to hartX.
 	 */
-	u32 hartid	    = current_hartid();
-	u32 word_index	    = hartid / 4;
-	u32 per_hart_offset = PLICSW_PENDING_STRIDE * hartid;
-	u32 val		    = 1 << target_hart << per_hart_offset;
+	interrupt_id = target_hart + 1;
+	word_index   = interrupt_id / 32;
+	pending_bit  = interrupt_id % 32;
+	pending_reg  = plicsw.addr + PLICSW_PENDING_BASE + word_index * 4;
 
-	writel(val, (void *)plicsw.addr + PLICSW_PENDING_BASE + word_index * 4);
+	/* Set target hart's mip.MSIP */
+	writel_relaxed(BIT(pending_bit), (void *)pending_reg);
 }
 
-static void plicsw_ipi_send(u32 hart_index)
+static void plicsw_ipi_clear(u32 hart_index)
 {
 	u32 target_hart = sbi_hartindex_to_hartid(hart_index);
+	ulong reg = plicsw.addr + PLICSW_CONTEXT_BASE + PLICSW_CONTEXT_CLAIM +
+		    PLICSW_CONTEXT_STRIDE * target_hart;
 
 	if (plicsw.hart_count <= target_hart)
 		ebreak();
 
-	/* Set PLICSW IPI */
-	plic_sw_pending(target_hart);
-}
+	/* Claim */
+	u32 source = readl((void *)reg);
 
-static void plicsw_ipi_clear(u32 hart_index)
-{
-	u32 target_hart = sbi_hartindex_to_hartid(hart_index);
-
-	if (plicsw.hart_count <= target_hart)
-		ebreak();
+	/* A successful claim will clear mip.MSIP */
 
-	/* Clear PLICSW IPI */
-	plicsw_claim();
-	plicsw_complete();
+	/* Complete */
+	writel(source, (void *)reg);
 }
 
 static struct sbi_ipi_device plicsw_ipi = {
@@ -110,22 +78,26 @@ int plicsw_warm_ipi_init(void)
 int plicsw_cold_ipi_init(struct plicsw_data *plicsw)
 {
 	int rc;
+	u32 interrupt_id, word_index, enable_bit;
+	ulong enable_reg, priority_reg;
 
 	/* Setup source priority */
-	uint32_t *priority = (void *)plicsw->addr + PLICSW_PRIORITY_BASE;
-
-	for (int i = 0; i < plicsw->hart_count; i++)
-		writel(1, &priority[i]);
-
-	/* Setup target enable */
-	uint32_t enable_mask = PLICSW_HART_MASK;
+	for (int i = 0; i < plicsw->hart_count; i++) {
+		priority_reg = plicsw->addr + PLICSW_PRIORITY_BASE + i * 4;
+		writel(1, (void *)priority_reg);
+	}
 
+	/*
+	 * Setup enable for each hart, skip non-existent interrupt ID 0
+	 * which is hardwired to 0.
+	 */
 	for (int i = 0; i < plicsw->hart_count; i++) {
-		uint32_t *enable = (void *)plicsw->addr + PLICSW_ENABLE_BASE +
-				   PLICSW_ENABLE_STRIDE * i;
-		writel(enable_mask, enable);
-		writel(enable_mask, enable + 1);
-		enable_mask <<= 1;
+		interrupt_id = i + 1;
+		word_index   = interrupt_id / 32;
+		enable_bit   = interrupt_id % 32;
+		enable_reg   = plicsw->addr + PLICSW_ENABLE_BASE +
+			       PLICSW_ENABLE_STRIDE * i + 4 * word_index;
+		writel(BIT(enable_bit), (void *)enable_reg);
 	}
 
 	/* Add PLICSW region to the root domain */
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:04 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:04 +0800
Subject: [PATCH v3 02/15] sbi: sbi_pmu: Improve sbi_pmu_init() error handling
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-3-peterlin@andestech.com>

This patch makes the following changes:

- As sbi_platform_pmu_init() returns a negative error code on
  failure, let sbi_pmu_init() to hang by propagating the error
  code.

- In order to distinguish the SBI_EFAIL error returned by
  sbi_pmu_add_*_counter_map(), return SBI_ENOENT to indicate
  that fdt_pmu_setup() failed to locate "riscv,pmu" node, and
  generic_pmu_init() ignores such case.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
Changes v1 -> v2:
  - New patch
Changes v2 -> v3:
  - Include Anup's RB tag
---
 lib/sbi/sbi_pmu.c           | 5 ++++-
 lib/utils/fdt/fdt_pmu.c     | 2 +-
 platform/generic/platform.c | 8 +++++++-
 3 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index f4c8fc4..3cbd4ff 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -957,6 +957,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 	int hpm_count = sbi_fls(sbi_hart_mhpm_mask(scratch));
 	struct sbi_pmu_hart_state *phs;
 	const struct sbi_platform *plat;
+	int rc;
 
 	if (cold_boot) {
 		hw_event_map = sbi_calloc(sizeof(*hw_event_map),
@@ -972,7 +973,9 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 
 		plat = sbi_platform_ptr(scratch);
 		/* Initialize hw pmu events */
-		sbi_platform_pmu_init(plat);
+		rc = sbi_platform_pmu_init(plat);
+		if (rc)
+			return rc;
 
 		/* mcycle & minstret is available always */
 		if (!hpm_count)
diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
index 83301bb..a8d7648 100644
--- a/lib/utils/fdt/fdt_pmu.c
+++ b/lib/utils/fdt/fdt_pmu.c
@@ -74,7 +74,7 @@ int fdt_pmu_setup(void *fdt)
 
 	pmu_offset = fdt_node_offset_by_compatible(fdt, -1, "riscv,pmu");
 	if (pmu_offset < 0)
-		return SBI_EFAIL;
+		return SBI_ENOENT;
 
 	event_ctr_map = fdt_getprop(fdt, pmu_offset,
 				    "riscv,event-to-mhpmcounters", &len);
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index 85acecd..fa400b9 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -265,7 +265,13 @@ static u32 generic_tlb_num_entries(void)
 
 static int generic_pmu_init(void)
 {
-	return fdt_pmu_setup(fdt_get_address());
+	int rc;
+
+	rc = fdt_pmu_setup(fdt_get_address());
+	if (rc && rc != SBI_ENOENT)
+		return rc;
+
+	return 0;
 }
 
 static uint64_t generic_pmu_xlate_to_mhpmevent(uint32_t event_idx,
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:05 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:05 +0800
Subject: [PATCH v3 03/15] lib: sbi: Add Xandespmu in hart extensions
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-4-peterlin@andestech.com>

When SBI PMU updates inhibit bits on mcyclecfg and
minstretcfg, it will check if Smcntrpmf extension
is supported, we need to check Xandespmu similarly
in the pmu_fixed_ctr_update_inhibit_bits() so add
the custom extension to hart extension list.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
Changes v2 -> v3:
  - New patch
---
 include/sbi/sbi_hart.h | 2 ++
 lib/sbi/sbi_hart.c     | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
index 6ee49ff..811e5f5 100644
--- a/include/sbi/sbi_hart.h
+++ b/include/sbi/sbi_hart.h
@@ -45,6 +45,8 @@ enum sbi_hart_extensions {
 	SBI_HART_EXT_ZKR,
 	/** Hart has Smcntrpmf extension */
 	SBI_HART_EXT_SMCNTRPMF,
+	/** Hart has Xandespmu extension */
+	SBI_HART_EXT_XANDESPMU,
 
 	/** Maximum index of Hart extension */
 	SBI_HART_EXT_MAX,
diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index 77eef49..d372af1 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -693,6 +693,9 @@ static inline char *sbi_hart_extension_id2string(int ext)
 	case SBI_HART_EXT_SMCNTRPMF:
 		estr = "smcntrpmf";
 		break;
+	case SBI_HART_EXT_XANDESPMU:
+		estr = "xandespmu";
+		break;
 	default:
 		break;
 	}
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:06 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:06 +0800
Subject: [PATCH v3 04/15] sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-5-peterlin@andestech.com>

Add support for custom PMU extensions to set inhibit bits
on custom CSRs by introducing the PMU device callback
hw_counter_filter_mode(). This allows the perf tool to
restrict event counting under a specified privileged
mode by appending a modifier, e.g. perf record -e event:k
to count events only happening in kernel mode.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
---
Changes v1 -> v2:
  - No change
Changes v2 -> v3:
  - Add pmu_dev->hw_counter_filter_mode() in pmu_fixed_ctr_update_inhibit_bits()
---
 include/sbi/sbi_pmu.h |  6 ++++++
 lib/sbi/sbi_pmu.c     | 20 ++++++++++++++------
 2 files changed, 20 insertions(+), 6 deletions(-)

diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
index 16f6877..d63149c 100644
--- a/include/sbi/sbi_pmu.h
+++ b/include/sbi/sbi_pmu.h
@@ -89,6 +89,12 @@ struct sbi_pmu_device {
 	 * Custom function returning the machine-specific irq-bit.
 	 */
 	int (*hw_counter_irq_bit)(void);
+
+	/**
+	 * Custom function to inhibit counting of events while in
+	 * specified mode.
+	 */
+	void (*hw_counter_filter_mode)(unsigned long flags, int counter_index);
 };
 
 /** Get the PMU platform device */
diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 3cbd4ff..2f255de 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -599,7 +599,10 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,
 		pmu_dev->hw_counter_disable_irq(ctr_idx);
 
 	/* Update the inhibit flags based on inhibit flags received from supervisor */
-	pmu_update_inhibit_flags(flags, &mhpmevent_val);
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
+		pmu_update_inhibit_flags(flags, &mhpmevent_val);
+	if (pmu_dev && pmu_dev->hw_counter_filter_mode)
+		pmu_dev->hw_counter_filter_mode(flags, ctr_idx);
 
 #if __riscv_xlen == 32
 	csr_write_num(CSR_MHPMEVENT3 + ctr_idx - 3, mhpmevent_val & 0xFFFFFFFF);
@@ -620,7 +623,8 @@ static int pmu_fixed_ctr_update_inhibit_bits(int fixed_ctr, unsigned long flags)
 #if __riscv_xlen == 32
 	uint64_t cfgh_csr_no;
 #endif
-	if (!sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF))
+	if (!sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF) &&
+		!sbi_hart_has_extension(scratch, SBI_HART_EXT_XANDESPMU))
 		return fixed_ctr;
 
 	switch (fixed_ctr) {
@@ -641,13 +645,17 @@ static int pmu_fixed_ctr_update_inhibit_bits(int fixed_ctr, unsigned long flags)
 	}
 
 	cfg_val |= MHPMEVENT_MINH;
-	pmu_update_inhibit_flags(flags, &cfg_val);
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF)) {
+		pmu_update_inhibit_flags(flags, &cfg_val);
 #if __riscv_xlen == 32
-	csr_write_num(cfg_csr_no, cfg_val & 0xFFFFFFFF);
-	csr_write_num(cfgh_csr_no, cfg_val >> BITS_PER_LONG);
+		csr_write_num(cfg_csr_no, cfg_val & 0xFFFFFFFF);
+		csr_write_num(cfgh_csr_no, cfg_val >> BITS_PER_LONG);
 #else
-	csr_write_num(cfg_csr_no, cfg_val);
+		csr_write_num(cfg_csr_no, cfg_val);
 #endif
+	}
+	if (pmu_dev && pmu_dev->hw_counter_filter_mode)
+		pmu_dev->hw_counter_filter_mode(flags, fixed_ctr);
 	return fixed_ctr;
 }
 
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:07 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:07 +0800
Subject: [PATCH v3 05/15] platform: include: andes45: Add PMU related CSR defines
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-6-peterlin@andestech.com>

This patch adds CSR for Andes PMU extension.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
Changes v1 -> v2:
  - Rename andes_pmu() -> has_andes_pmu()
Changes v2 -> v3:
  - Include Anup's RB tag
---
 platform/generic/include/andes/andes45.h | 26 ++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
index f570994..ce31617 100644
--- a/platform/generic/include/andes/andes45.h
+++ b/platform/generic/include/andes/andes45.h
@@ -12,6 +12,17 @@
 #define CSR_MDCM_CFG		0xfc1
 #define CSR_MMSC_CFG		0xfc2
 
+/* Machine Trap Related Registers */
+#define CSR_MSLIDELEG		0x7d5
+
+/* Counter Related Registers */
+#define CSR_MCOUNTERWEN		0x7ce
+#define CSR_MCOUNTERINTEN	0x7cf
+#define CSR_MCOUNTERMASK_M	0x7d1
+#define CSR_MCOUNTERMASK_S	0x7d2
+#define CSR_MCOUNTERMASK_U	0x7d3
+#define CSR_MCOUNTEROVF		0x7d4
+
 #define MICM_CFG_ISZ_OFFSET		6
 #define MICM_CFG_ISZ_MASK		(0x7  << MICM_CFG_ISZ_OFFSET)
 
@@ -26,4 +37,19 @@
 #define MCACHE_CTL_CCTL_SUEN_OFFSET	8
 #define MCACHE_CTL_CCTL_SUEN_MASK	(0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
 
+/* Performance monitor */
+#define MMSC_CFG_PMNDS_MASK		(1 << 15)
+#define MIP_PMOVI			(1 << 18)
+
+#ifndef __ASSEMBLER__
+
+#define has_andes_pmu()					\
+({							\
+	(((csr_read(CSR_MMSC_CFG) &			\
+	   MMSC_CFG_PMNDS_MASK)				\
+	  && misa_extension('S')) ? true : false);	\
+})
+
+#endif /* __ASSEMBLER__ */
+
 #endif /* _RISCV_ANDES45_H */
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:08 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:08 +0800
Subject: [PATCH v3 06/15] platform: generic: Introduce pmu_init() platform override
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-7-peterlin@andestech.com>

Add pmu_init() platform override which will be used to register
PMU device and populate event mappings.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
Changes v2 -> v3:
  - New patch
---
 platform/generic/include/platform_override.h | 1 +
 platform/generic/platform.c                  | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
index bf4b112..f2a4327 100644
--- a/platform/generic/include/platform_override.h
+++ b/platform/generic/include/platform_override.h
@@ -27,6 +27,7 @@ struct platform_override {
 	int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
 	int (*extensions_init)(const struct fdt_match *match,
 			       struct sbi_hart_features *hfeatures);
+	int (*pmu_init)(const struct fdt_match *match);
 	void (*fw_init)(void *fdt, const struct fdt_match *match);
 	int (*vendor_ext_provider)(long funcid,
 				   const struct sbi_trap_regs *regs,
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index fa400b9..54a913b 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -267,6 +267,9 @@ static int generic_pmu_init(void)
 {
 	int rc;
 
+	if (generic_plat && generic_plat->pmu_init)
+		return generic_plat->pmu_init(generic_plat_match);
+
 	rc = fdt_pmu_setup(fdt_get_address());
 	if (rc && rc != SBI_ENOENT)
 		return rc;
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:09 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:09 +0800
Subject: [PATCH v3 07/15] platform: andes: Add Andes custom PMU support
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-8-peterlin@andestech.com>

Before the ratification of Sscofpmf, the Andes PMU extension
was designed to support the sampling and filtering with hardware
performance counters (zihpm), it works with the current SBI PMU
extension and Linux SBI PMU driver.

We implement 1) the PMU device callbacks that update the
corresponding bits on custom CSRs, 2) extentions_init() to detect
the hardware support of Andes PMU and initialize the per-hart
PMU related CSR, and 3) pmu_init() to register PMU device and
populate event mappings (only called by coldboot hart).

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
---
Changes v1 -> v2:
  - Fix mode filtering in andes_hw_counter_filter_mode()
  - Return early if pmu is not supported in andes_pmu_init() (suggested by Prabhakar)
  - Don't grant write permissions via CSR_MCOUNTERWEN as not needed
Changes v2 -> v3:
  - Drop Anup's RB tag as we add andes_pmu_extensions_init() to initialize per-hart
    extension on scratch and move andes_pmu_init() to new added platform override pmu_init()
--- 
 platform/generic/andes/Kconfig             |   8 ++
 platform/generic/andes/andes_pmu.c         | 102 +++++++++++++++++++++
 platform/generic/andes/objects.mk          |   1 +
 platform/generic/include/andes/andes_pmu.h |  33 +++++++
 4 files changed, 144 insertions(+)
 create mode 100644 platform/generic/andes/andes_pmu.c
 create mode 100644 platform/generic/include/andes/andes_pmu.h

diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
index a91fb9c..5b2ed91 100644
--- a/platform/generic/andes/Kconfig
+++ b/platform/generic/andes/Kconfig
@@ -7,3 +7,11 @@ config ANDES45_PMA
 config ANDES_SBI
 	bool "Andes SBI support"
 	default n
+
+config ANDES_PMU
+	bool "Andes PMU extension (xandespmu) support"
+	default n
+	help
+	  Andes PMU extension supports the event counter overflow
+	  interrupt and mode filtering, similar to the standard
+	  Sscofpmf and Smcntrpmf.
diff --git a/platform/generic/andes/andes_pmu.c b/platform/generic/andes/andes_pmu.c
new file mode 100644
index 0000000..72003fd
--- /dev/null
+++ b/platform/generic/andes/andes_pmu.c
@@ -0,0 +1,102 @@
+// SPDX-License-Identifier: BSD-2-Clause
+/*
+ * andes_pmu.c - Andes PMU device callbacks and platform overrides
+ *
+ * Copyright (C) 2023 Andes Technology Corporation
+ */
+
+#include <andes/andes45.h>
+#include <andes/andes_hpm.h>
+#include <andes/andes_pmu.h>
+#include <sbi/sbi_bitops.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_pmu.h>
+
+static void andes_hw_counter_enable_irq(uint32_t ctr_idx)
+{
+	unsigned long mip_val;
+
+	if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
+		return;
+
+	mip_val = csr_read(CSR_MIP);
+	if (!(mip_val & MIP_PMOVI))
+		csr_clear(CSR_MCOUNTEROVF, BIT(ctr_idx));
+
+	csr_set(CSR_MCOUNTERINTEN, BIT(ctr_idx));
+}
+
+static void andes_hw_counter_disable_irq(uint32_t ctr_idx)
+{
+	csr_clear(CSR_MCOUNTERINTEN, BIT(ctr_idx));
+}
+
+static void andes_hw_counter_filter_mode(unsigned long flags, int ctr_idx)
+{
+	if (flags & SBI_PMU_CFG_FLAG_SET_UINH)
+		csr_set(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
+	else
+		csr_clear(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
+
+	if (flags & SBI_PMU_CFG_FLAG_SET_SINH)
+		csr_set(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
+	else
+		csr_clear(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
+}
+
+static struct sbi_pmu_device andes_pmu = {
+	.name = "andes_pmu",
+	.hw_counter_enable_irq  = andes_hw_counter_enable_irq,
+	.hw_counter_disable_irq = andes_hw_counter_disable_irq,
+	/*
+	 * We set delegation of supervisor local interrupts via
+	 * 18th bit on mslideleg instead of mideleg, so leave
+	 * hw_counter_irq_bit() callback unimplemented.
+	 */
+	.hw_counter_irq_bit     = NULL,
+	.hw_counter_filter_mode = andes_hw_counter_filter_mode
+};
+
+int andes_pmu_extensions_init(const struct fdt_match *match,
+			      struct sbi_hart_features *hfeatures)
+{
+	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
+
+	if (!has_andes_pmu())
+		return 0;
+
+	/*
+	 * Don't expect both Andes PMU and standard Sscofpmf/Smcntrpmf,
+	 * are supported as they serve the same purpose.
+	 */
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF) ||
+		sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF))
+		return SBI_EINVAL;
+	sbi_hart_update_extension(scratch, SBI_HART_EXT_XANDESPMU, true);
+
+	/* Inhibit all HPM counters in M-mode */
+	csr_write(CSR_MCOUNTERMASK_M, 0xfffffffd);
+	/* Delegate counter overflow interrupt to S-mode */
+	csr_write(CSR_MSLIDELEG, MIP_PMOVI);
+
+	return 0;
+}
+
+int andes_pmu_init(const struct fdt_match *match)
+{
+	int rc;
+	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
+
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_XANDESPMU))
+		sbi_pmu_set_device(&andes_pmu);
+
+	rc = fdt_pmu_setup(fdt_get_address());
+	/*
+	 * Populate default mappings if device-tree doesn't
+	 * provide a valid pmu node.
+	 */
+	if (rc == SBI_ENOENT)
+		return andes_pmu_setup();
+
+	return rc;
+}
diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
index e8f86ea..6a8c66c 100644
--- a/platform/generic/andes/objects.mk
+++ b/platform/generic/andes/objects.mk
@@ -7,3 +7,4 @@ platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
 
 platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
 platform-objs-$(CONFIG_ANDES_SBI) += andes/andes_sbi.o
+platform-objs-$(CONFIG_ANDES_PMU) += andes/andes_pmu.o
diff --git a/platform/generic/include/andes/andes_pmu.h b/platform/generic/include/andes/andes_pmu.h
new file mode 100644
index 0000000..c4281fe
--- /dev/null
+++ b/platform/generic/include/andes/andes_pmu.h
@@ -0,0 +1,33 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) Copyright (c) 2023 Andes Technology Corporation
+ */
+
+#ifndef _RISCV_ANDES_PMU_H
+#define _RISCV_ANDES_PMU_H
+
+#include <sbi/sbi_hart.h>
+#include <sbi_utils/fdt/fdt_pmu.h>
+
+#ifdef CONFIG_ANDES_PMU
+
+int andes_pmu_init(const struct fdt_match *match);
+int andes_pmu_extensions_init(const struct fdt_match *match,
+			      struct sbi_hart_features *hfeatures);
+
+#else
+
+int andes_pmu_init(const struct fdt_match *match)
+{
+	return 0;
+}
+int andes_pmu_extensions_init(const struct fdt_match *match,
+			      struct sbi_hart_features *hfeatures)
+{
+	return 0;
+}
+
+#endif /* CONFIG_ANDES_PMU */
+
+#endif /* _RISCV_ANDES_PMU_H */
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:10 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:10 +0800
Subject: [PATCH v3 08/15] platform: andes: Enable Andes PMU for AE350
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-9-peterlin@andestech.com>

Enable Andes PMU extension support for AE350 platforms.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
---
Changes v1 -> v2:
  - Implement ae350_extensions_init()
Changes v2 -> v3:
  - Use the andes_pmu_extensions_init() and andes_pmu_init() provided by andes_pmu.c
    We have no plan to introduce other extensions in near future so extensions_init()
    bind with andes_pmu_extensions_init() directly.
---
 platform/generic/Kconfig       | 1 +
 platform/generic/andes/ae350.c | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index a902cf3..89e5603 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -32,6 +32,7 @@ config PLATFORM_ALLWINNER_D1
 config PLATFORM_ANDES_AE350
 	bool "Andes AE350 support"
 	select SYS_ATCSMU
+	select ANDES_PMU
 	default n
 
 config PLATFORM_RENESAS_RZFIVE
diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
index 80eca05..dc769b7 100644
--- a/platform/generic/andes/ae350.c
+++ b/platform/generic/andes/ae350.c
@@ -8,6 +8,7 @@
  */
 
 #include <platform_override.h>
+#include <andes/andes_pmu.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 #include <sbi_utils/fdt/fdt_fixup.h>
 #include <sbi_utils/sys/atcsmu.h>
@@ -118,4 +119,6 @@ static const struct fdt_match andes_ae350_match[] = {
 const struct platform_override andes_ae350 = {
 	.match_table = andes_ae350_match,
 	.final_init  = ae350_final_init,
+	.extensions_init = andes_pmu_extensions_init,
+	.pmu_init = andes_pmu_init,
 };
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:11 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:11 +0800
Subject: [PATCH v3 09/15] platform: rzfive: Enable Andes PMU for RZ/Five
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-10-peterlin@andestech.com>

Enable Andes PMU extension support for RZ/Five.
We also staticize renesas_rzfive_early_init() as
it is not used outside of this unit.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
---
Changes v1 -> v2:
  - Implement renesas_rzfive_extensions_init()
  - staticize renesas_rzfive_early_init()
Changes v2 -> v3:
  - Use the andes_pmu_extensions_init() and andes_pmu_init() provided by andes_pmu.c
    We have no plan to introduce other extensions in near future so extensions_init()
    bind with andes_pmu_extensions_init() directly.
---
 platform/generic/Kconfig                 | 1 +
 platform/generic/renesas/rzfive/rzfive.c | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index 89e5603..c9b6011 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -39,6 +39,7 @@ config PLATFORM_RENESAS_RZFIVE
 	bool "Renesas RZ/Five support"
 	select ANDES45_PMA
 	select ANDES_SBI
+	select ANDES_PMU
 	default n
 
 config PLATFORM_SIFIVE_FU540
diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
index a69797b..b5b5e65 100644
--- a/platform/generic/renesas/rzfive/rzfive.c
+++ b/platform/generic/renesas/rzfive/rzfive.c
@@ -5,6 +5,7 @@
  */
 
 #include <andes/andes45_pma.h>
+#include <andes/andes_pmu.h>
 #include <andes/andes_sbi.h>
 #include <platform_override.h>
 #include <sbi/sbi_domain.h>
@@ -29,7 +30,7 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
 					 array_size(renesas_rzfive_pma_regions));
 }
 
-int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
+static int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
 {
 	/*
 	 * Renesas RZ/Five RISC-V SoC has Instruction local memory and
@@ -57,4 +58,6 @@ const struct platform_override renesas_rzfive = {
 	.early_init = renesas_rzfive_early_init,
 	.final_init = renesas_rzfive_final_init,
 	.vendor_ext_provider = andes_sbi_vendor_ext_provider,
+	.extensions_init = andes_pmu_extensions_init,
+	.pmu_init = andes_pmu_init,
 };
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:12 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:12 +0800
Subject: [PATCH v3 10/15] lib: utils: fdt_fixup: Allow preserving PMU properties
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-11-peterlin@andestech.com>

Add a scratch option to control PMU fixup, so the next
stage software can dump the PMU node including event
mapping information for debugging purposes.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
Changes v1 -> v2:
  - New patch
Changes v2 -> v3:
  - Rename to SBI_SCRATCH_PRESERVE_PMU_NODE (suggested by Anup)
  - Include Anup's RB tag
---
 include/sbi/sbi_scratch.h | 2 ++
 lib/utils/fdt/fdt_fixup.c | 6 +++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/include/sbi/sbi_scratch.h b/include/sbi/sbi_scratch.h
index e6a33ba..0f67cde 100644
--- a/include/sbi/sbi_scratch.h
+++ b/include/sbi/sbi_scratch.h
@@ -151,6 +151,8 @@ enum sbi_scratch_options {
 	SBI_SCRATCH_NO_BOOT_PRINTS = (1 << 0),
 	/** Enable runtime debug prints */
 	SBI_SCRATCH_DEBUG_PRINTS = (1 << 1),
+	/** Preserve PMU node properties */
+	SBI_SCRATCH_PRESERVE_PMU_NODE = (1 << 2),
 };
 
 /** Get pointer to sbi_scratch for current HART */
diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index e213ded..cf20edf 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -387,6 +387,8 @@ int fdt_reserved_memory_fixup(void *fdt)
 
 void fdt_fixups(void *fdt)
 {
+	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
+
 	fdt_aplic_fixup(fdt);
 
 	fdt_imsic_fixup(fdt);
@@ -394,5 +396,7 @@ void fdt_fixups(void *fdt)
 	fdt_plic_fixup(fdt);
 
 	fdt_reserved_memory_fixup(fdt);
-	fdt_pmu_fixup(fdt);
+
+	if (!(scratch->options & SBI_SCRATCH_PRESERVE_PMU_NODE))
+		fdt_pmu_fixup(fdt);
 }
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:13 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:13 +0800
Subject: [PATCH v3 11/15] platform: andes: Factor out is_andes() helper
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-12-peterlin@andestech.com>

We will need is_andes(45) in the following patch,
so factor out the code that parses marchid to make
it reusable for checking any Andes CPU variants.

Also improves the comment in ae350_hart_start().

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
Changes v1 -> v2:
  - New patch
Changes v2 -> v3:
  - Include Anup's RB tag
---
 platform/generic/andes/ae350.c           | 16 +++++++---------
 platform/generic/include/andes/andes45.h |  6 ++++++
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
index dc769b7..088ec07 100644
--- a/platform/generic/andes/ae350.c
+++ b/platform/generic/andes/ae350.c
@@ -24,16 +24,14 @@ static struct smu_data smu = { 0 };
 extern void __ae350_enable_coherency_warmboot(void);
 extern void __ae350_disable_coherency(void);
 
-static __always_inline bool is_andes25(void)
-{
-	ulong marchid = csr_read(CSR_MARCHID);
-	return !!(EXTRACT_FIELD(marchid, CSR_MARCHID_MICROID) == 0xa25);
-}
-
 static int ae350_hart_start(u32 hartid, ulong saddr)
 {
-	/* Don't send wakeup command at boot-time */
-	if (!sbi_init_count(hartid) || (is_andes25() && hartid == 0))
+	/*
+	 * Don't send wakeup command when:
+	 * 1) boot-time
+	 * 2) the target hart is non-sleepable 25-series hart0
+	 */
+	if (!sbi_init_count(hartid) || (is_andes(25) && hartid == 0))
 		return sbi_ipi_raw_send(sbi_hartid_to_hartindex(hartid));
 
 	/* Write wakeup command to the sleep hart */
@@ -52,7 +50,7 @@ static int ae350_hart_stop(void)
 	 * L2-cache, instead of turning it off, it should fall
 	 * through and jump to warmboot_addr.
 	 */
-	if (is_andes25() && hartid == 0)
+	if (is_andes(25) && hartid == 0)
 		return SBI_ENOTSUPP;
 
 	if (!smu_support_sleep_mode(&smu, DEEPSLEEP_MODE, hartid))
diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
index ce31617..01f63d4 100644
--- a/platform/generic/include/andes/andes45.h
+++ b/platform/generic/include/andes/andes45.h
@@ -43,6 +43,12 @@
 
 #ifndef __ASSEMBLER__
 
+#define is_andes(series)				\
+({							\
+	char value = csr_read(CSR_MARCHID) & 0xff;	\
+	(series) == (value >> 4) * 10 + (value & 0x0f);	\
+})
+
 #define has_andes_pmu()					\
 ({							\
 	(((csr_read(CSR_MMSC_CFG) &			\
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:14 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:14 +0800
Subject: [PATCH v3 12/15] lib: utils: fdt_pmu: Make the fdt_pmu_evt_select table global variable
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-13-peterlin@andestech.com>

To allow platform override pmu_init() filling the translation table
fdt_pmu_evt_select[] when PMU node doesn't provide such information,
we need to share the table and its entry counter with other .c file.

We also define the structures of PMU property in fdt_helper.h, so we
can initialize the mappings in arrays.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
Changes v2 -> v3:
  - New patch
---
 include/sbi_utils/fdt/fdt_helper.h | 17 +++++++++++++++++
 include/sbi_utils/fdt/fdt_pmu.h    |  6 ++++++
 lib/utils/fdt/fdt_pmu.c            | 13 ++++---------
 3 files changed, 27 insertions(+), 9 deletions(-)

diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
index 5c928ff..f6f505b 100644
--- a/include/sbi_utils/fdt/fdt_helper.h
+++ b/include/sbi_utils/fdt/fdt_helper.h
@@ -34,6 +34,23 @@ struct platform_uart_data {
 	unsigned long reg_offset;
 };
 
+struct fdt_pmu_hw_event_select_map {
+	uint32_t eidx;
+	uint64_t select;
+};
+
+struct fdt_pmu_hw_event_counter_map {
+	uint32_t eidx_start;
+	uint32_t eidx_end;
+	uint32_t ctr_map;
+};
+
+struct fdt_pmu_raw_event_counter_map {
+	uint64_t select;
+	uint64_t select_mask;
+	uint32_t ctr_map;
+};
+
 const struct fdt_match *fdt_match_node(void *fdt, int nodeoff,
 				       const struct fdt_match *match_table);
 
diff --git a/include/sbi_utils/fdt/fdt_pmu.h b/include/sbi_utils/fdt/fdt_pmu.h
index c65cad7..9a13487 100644
--- a/include/sbi_utils/fdt/fdt_pmu.h
+++ b/include/sbi_utils/fdt/fdt_pmu.h
@@ -12,6 +12,7 @@
 #define __FDT_PMU_H__
 
 #include <sbi/sbi_types.h>
+#include <sbi_utils/fdt/fdt_helper.h>
 
 #ifdef CONFIG_FDT_PMU
 
@@ -45,6 +46,11 @@ int fdt_pmu_setup(void *fdt);
  */
 uint64_t fdt_pmu_get_select_value(uint32_t event_idx);
 
+/** The event index to selector value table instance */
+extern struct fdt_pmu_hw_event_select_map fdt_pmu_evt_select[];
+/** The number of valid entries in fdt_pmu_evt_select[] */
+extern uint32_t hw_event_count;
+
 #else
 
 static inline void fdt_pmu_fixup(void *fdt) { }
diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
index a8d7648..cf3220f 100644
--- a/lib/utils/fdt/fdt_pmu.c
+++ b/lib/utils/fdt/fdt_pmu.c
@@ -17,18 +17,13 @@
 
 #define FDT_PMU_HW_EVENT_MAX (SBI_PMU_HW_EVENT_MAX * 2)
 
-struct fdt_pmu_hw_event_select {
-	uint32_t eidx;
-	uint64_t select;
-};
-
-static struct fdt_pmu_hw_event_select fdt_pmu_evt_select[FDT_PMU_HW_EVENT_MAX] = {0};
-static uint32_t hw_event_count;
+struct fdt_pmu_hw_event_select_map fdt_pmu_evt_select[FDT_PMU_HW_EVENT_MAX] = {0};
+uint32_t hw_event_count;
 
 uint64_t fdt_pmu_get_select_value(uint32_t event_idx)
 {
 	int i;
-	struct fdt_pmu_hw_event_select *event;
+	struct fdt_pmu_hw_event_select_map *event;
 
 	for (i = 0; i < SBI_PMU_HW_EVENT_MAX; i++) {
 		event = &fdt_pmu_evt_select[i];
@@ -65,7 +60,7 @@ int fdt_pmu_setup(void *fdt)
 	int i, pmu_offset, len, result;
 	const u32 *event_val;
 	const u32 *event_ctr_map;
-	struct fdt_pmu_hw_event_select *event;
+	struct fdt_pmu_hw_event_select_map *event;
 	uint64_t raw_selector, select_mask;
 	u32 event_idx_start, event_idx_end, ctr_map;
 
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:15 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:15 +0800
Subject: [PATCH v3 13/15] lib: utils: fdt_pmu: Do not iterate over the fdt_pmu_evt_select table
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-14-peterlin@andestech.com>

The valid entry count is tracking by hw_event_count so there
is no need to check the whole table.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
Changes v2 -> v3:
  - New patch
---
 lib/utils/fdt/fdt_pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
index cf3220f..c531b9a 100644
--- a/lib/utils/fdt/fdt_pmu.c
+++ b/lib/utils/fdt/fdt_pmu.c
@@ -25,7 +25,7 @@ uint64_t fdt_pmu_get_select_value(uint32_t event_idx)
 	int i;
 	struct fdt_pmu_hw_event_select_map *event;
 
-	for (i = 0; i < SBI_PMU_HW_EVENT_MAX; i++) {
+	for (i = 0; i < hw_event_count; i++) {
 		event = &fdt_pmu_evt_select[i];
 		if (event->eidx == event_idx)
 			return event->select;
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:16 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:16 +0800
Subject: [PATCH v3 14/15] platform: andes: Add Andes default PMU mapping support
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-15-peterlin@andestech.com>

Provide the andes_pmu_setup() to populate default mappings in
hw_event_map[] and fdt_pmu_evt_select[] when fdt_pmu_setup()
cannot find the PMU node.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
Changes v2 -> v3:
  - New patch
  - Drop fdt_add_pmu_mappings() (suggested by Anup [1])

[1] https://patchwork.ozlabs.org/project/opensbi/patch/20231019113713.3508153-11-peterlin at andestech.com/
---
 include/sbi/sbi_ecall_interface.h          |   5 +
 platform/generic/Kconfig                   |   2 +
 platform/generic/andes/Kconfig             |   7 +
 platform/generic/andes/andes_hpm.c         | 405 +++++++++++++++++++++
 platform/generic/andes/objects.mk          |   1 +
 platform/generic/include/andes/andes_hpm.h |  81 +++++
 6 files changed, 501 insertions(+)
 create mode 100644 platform/generic/andes/andes_hpm.c
 create mode 100644 platform/generic/include/andes/andes_hpm.h

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index 1fe469e..89187e7 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -155,6 +155,11 @@ enum sbi_pmu_hw_cache_op_result_id {
 	SBI_PMU_HW_CACHE_RESULT_MAX,
 };
 
+#define SBI_PMU_HW_CACHE_EVENT_IDX(id, op, res) \
+	(SBI_PMU_EVENT_TYPE_HW_CACHE << SBI_PMU_EVENT_IDX_TYPE_OFFSET | \
+	SBI_PMU_HW_CACHE_##id << 3 | SBI_PMU_HW_CACHE_OP_##op << 1 | \
+	SBI_PMU_HW_CACHE_RESULT_##res)
+
 /**
  * Special "firmware" events provided by the OpenSBI, even if the hardware
  * does not support performance events. These events are encoded as a raw
diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index c9b6011..e652645 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -32,6 +32,7 @@ config PLATFORM_ALLWINNER_D1
 config PLATFORM_ANDES_AE350
 	bool "Andes AE350 support"
 	select SYS_ATCSMU
+	select ANDES_HPM
 	select ANDES_PMU
 	default n
 
@@ -39,6 +40,7 @@ config PLATFORM_RENESAS_RZFIVE
 	bool "Renesas RZ/Five support"
 	select ANDES45_PMA
 	select ANDES_SBI
+	select ANDES_HPM
 	select ANDES_PMU
 	default n
 
diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
index 5b2ed91..de0d752 100644
--- a/platform/generic/andes/Kconfig
+++ b/platform/generic/andes/Kconfig
@@ -8,6 +8,13 @@ config ANDES_SBI
 	bool "Andes SBI support"
 	default n
 
+config ANDES_HPM
+	bool "Andes HPM support"
+	default n
+	help
+	  This provides default HPM event counter mappings
+	  when a pmu node is missing.
+
 config ANDES_PMU
 	bool "Andes PMU extension (xandespmu) support"
 	default n
diff --git a/platform/generic/andes/andes_hpm.c b/platform/generic/andes/andes_hpm.c
new file mode 100644
index 0000000..7fe5ac5
--- /dev/null
+++ b/platform/generic/andes/andes_hpm.c
@@ -0,0 +1,405 @@
+// SPDX-License-Identifier: BSD-2-Clause
+/*
+ * andes_hpm.c - Default event mappings and helper routine
+ *
+ * Copyright (c) 2023 Andes Technology Corporation
+ */
+
+#include <andes/andes45.h>
+#include <andes/andes_hpm.h>
+#include <sbi/sbi_ecall_interface.h>
+#include <sbi/sbi_pmu.h>
+#include <sbi_utils/fdt/fdt_pmu.h>
+
+static const struct fdt_pmu_hw_event_select_map andes45_hw_evt_selects[] = {
+	/* Hardware general events (Type #0) */
+	{
+		/* perf: cycles (eidx: 0x1) */
+		.eidx = SBI_PMU_HW_CPU_CYCLES,
+		.select = ANDES_CYCLES
+	},
+	{
+		/* perf: instructions (eidx: 0x2) */
+		.eidx = SBI_PMU_HW_INSTRUCTIONS,
+		.select = ANDES_INSTRET
+	},
+	{
+		/* perf: cache-references (eidx: 0x3) */
+		.eidx = SBI_PMU_HW_CACHE_REFERENCES,
+		.select = ANDES_DCACHE_ACCESS
+	},
+	{
+		/* perf: cache-misses (eidx: 0x4) */
+		.eidx = SBI_PMU_HW_CACHE_MISSES,
+		.select = ANDES_DCACHE_MISS
+	},
+	{
+		/* perf: branches (eidx: 0x5) */
+		.eidx = SBI_PMU_HW_BRANCH_INSTRUCTIONS,
+		.select = ANDES_CONDITION_BR,
+	},
+	{
+		/* perf: branch-misses (eidx: 0x6) */
+		.eidx = SBI_PMU_HW_BRANCH_MISSES,
+		.select = ANDES_MISPREDICT_CONDITION_BR,
+	},
+	/* Hardware cache events (Type #1) */
+	{
+		/* perf: L1-dcache-loads (eidx: 0x10000) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, ACCESS),
+		.select = ANDES_DCACHE_LOAD_ACCESS
+	},
+	{
+		/* perf: L1-dcache-loads-misses (eidx: 0x10001) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, MISS),
+		.select = ANDES_DCACHE_LOAD_MISS
+	},
+	{
+		/* perf: L1-dcache-stores (eidx: 0x10002) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, ACCESS),
+		.select = ANDES_DCACHE_STORE_ACCESS
+	},
+	{
+		/* perf: L1-dcache-store-misses (eidx: 0x10003) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, MISS),
+		.select = ANDES_DCACHE_STORE_MISS
+	},
+	{
+		/* perf: L1-icache-load (eidx: 0x10008) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, ACCESS),
+		.select = ANDES_ICACHE_ACCESS
+	},
+	{
+		/* perf: L1-icache-load-misses (eidx: 0x10009) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, MISS),
+		.select = ANDES_ICACHE_MISS
+	},
+};
+
+static const struct fdt_pmu_hw_event_counter_map andes45_hw_evt_counters[] = {
+	{
+		/* perf: cycles (eidx: 0x1) */
+		.eidx_start = SBI_PMU_HW_CPU_CYCLES,
+		/* perf: branch-misses (eidx: 0x6) */
+		.eidx_end = SBI_PMU_HW_BRANCH_MISSES,
+		.ctr_map  = ANDES_MHPM_MAP,
+	},
+	{
+		/* perf: L1-dcache-loads (eidx: 0x10000) */
+		.eidx_start = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, ACCESS),
+		/* perf: L1-dcache-store-misses (eidx: 0x10003) */
+		.eidx_end = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, MISS),
+		.ctr_map  = ANDES_MHPM_MAP,
+	},
+	{
+		/* perf: L1-icache-load (eidx: 0x10008) */
+		.eidx_start = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, ACCESS),
+		/* perf: L1-icache-load-misses (eidx: 0x10009) */
+		.eidx_end = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, MISS),
+		.ctr_map  = ANDES_MHPM_MAP,
+	},
+};
+
+static const struct fdt_pmu_raw_event_counter_map andes45_raw_evt_counters[] = {
+	{
+		.select = ANDES_CYCLES,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INSTRET,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_LOAD_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_STORE_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_ATOMIC_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_SYS_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_COMPUTE_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CONDITION_BR,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_TAKEN_CONDITION_BR,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_JAL_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_JALR_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_RET_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CONTROL_TRANS_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_EX9_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_MUL_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_DIV_REMAINDER_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_LOAD_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_STORE_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_ADD_SUB_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_MUL_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_FUSED_MULADD_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_DIV_SQUARE_ROOT_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_OTHER_FLOAT_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_MUL_AND_SUB_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_RETIRED_OP,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_ILM_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DLM_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_ICACHE_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_ICACHE_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_LOAD_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_LOAD_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_STORE_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_STORE_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_WB,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_ICACHE_FILL,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_DCACHE_FILL,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_UNCACHED_IFETCH_FROM_BUS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_UNCACHED_LOAD_FROM_BUS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_UNCACHED_IFETCH,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_UNCACHED_LOAD,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MAIN_ITLB_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MAIN_ITLB_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MAIN_DTLB_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MAIN_DTLB_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_ITLB_FILL,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_PIPE_STALL_CYCLE_DTLB_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_HW_PREFETCH_BUS_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MISPREDICT_CONDITION_BR,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MISPREDICT_TAKE_CONDITION_BR,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MISPREDICT_TARGET_RET_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+};
+
+int andes_pmu_setup(void)
+{
+	int i, rc;
+	struct fdt_pmu_hw_event_select_map *event;
+
+	/*
+	 * At the moment, simply create mapping for any 45-series core
+	 * based on marchid, we may check and differentiate the mapping
+	 * by mimpid.
+	 */
+	if (!is_andes(45))
+		return 0;
+
+	for (i = 0; i < array_size(andes45_hw_evt_counters); i++) {
+		rc = sbi_pmu_add_hw_event_counter_map(
+			andes45_hw_evt_counters[i].eidx_start,
+			andes45_hw_evt_counters[i].eidx_end,
+			andes45_hw_evt_counters[i].ctr_map);
+		if (rc)
+			return rc;
+	}
+
+	for (i = 0; i < array_size(andes45_hw_evt_selects); i++) {
+		event	      = &fdt_pmu_evt_select[hw_event_count];
+		event->eidx   = andes45_hw_evt_selects[i].eidx;
+		event->select = andes45_hw_evt_selects[i].select;
+		hw_event_count++;
+	}
+
+	for (i = 0; i < array_size(andes45_raw_evt_counters); i++) {
+		rc = sbi_pmu_add_raw_event_counter_map(
+			andes45_raw_evt_counters[i].select,
+			andes45_raw_evt_counters[i].select_mask,
+			andes45_raw_evt_counters[i].ctr_map);
+		if (rc)
+			return rc;
+	}
+
+	return 0;
+}
diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
index 6a8c66c..57caaf6 100644
--- a/platform/generic/andes/objects.mk
+++ b/platform/generic/andes/objects.mk
@@ -8,3 +8,4 @@ platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
 platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
 platform-objs-$(CONFIG_ANDES_SBI) += andes/andes_sbi.o
 platform-objs-$(CONFIG_ANDES_PMU) += andes/andes_pmu.o
+platform-objs-$(CONFIG_ANDES_HPM) += andes/andes_hpm.o
diff --git a/platform/generic/include/andes/andes_hpm.h b/platform/generic/include/andes/andes_hpm.h
new file mode 100644
index 0000000..79c75f6
--- /dev/null
+++ b/platform/generic/include/andes/andes_hpm.h
@@ -0,0 +1,81 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Andes Technology Corporation
+ */
+
+#ifndef _ANDES_HPM_H_
+#define _ANDES_HPM_H_
+
+#define ANDES_MHPM_MAP		0x78
+#define ANDES_RAW_EVENT_MASK	~0ULL
+
+/* Event selector for instruction commit events */
+#define ANDES_CYCLES				0x10
+#define ANDES_INSTRET				0x20
+#define ANDES_INT_LOAD_INST			0x30
+#define ANDES_INT_STORE_INST			0x40
+#define ANDES_ATOMIC_INST			0x50
+#define ANDES_SYS_INST				0x60
+#define ANDES_INT_COMPUTE_INST			0x70
+#define ANDES_CONDITION_BR			0x80
+#define ANDES_TAKEN_CONDITION_BR		0x90
+#define ANDES_JAL_INST				0xA0
+#define ANDES_JALR_INST				0xB0
+#define ANDES_RET_INST				0xC0
+#define ANDES_CONTROL_TRANS_INST		0xD0
+#define ANDES_EX9_INST				0xE0
+#define ANDES_INT_MUL_INST			0xF0
+#define ANDES_INT_DIV_REMAINDER_INST		0x100
+#define ANDES_FLOAT_LOAD_INST			0x110
+#define ANDES_FLOAT_STORE_INST			0x120
+#define ANDES_FLOAT_ADD_SUB_INST		0x130
+#define ANDES_FLOAT_MUL_INST			0x140
+#define ANDES_FLOAT_FUSED_MULADD_INST		0x150
+#define ANDES_FLOAT_DIV_SQUARE_ROOT_INST	0x160
+#define ANDES_OTHER_FLOAT_INST			0x170
+#define ANDES_INT_MUL_AND_SUB_INST		0x180
+#define ANDES_RETIRED_OP			0x190
+
+/* Event selector for memory system events */
+#define ANDES_ILM_ACCESS			0x01
+#define ANDES_DLM_ACCESS			0x11
+#define ANDES_ICACHE_ACCESS			0x21
+#define ANDES_ICACHE_MISS			0x31
+#define ANDES_DCACHE_ACCESS			0x41
+#define ANDES_DCACHE_MISS			0x51
+#define ANDES_DCACHE_LOAD_ACCESS		0x61
+#define ANDES_DCACHE_LOAD_MISS			0x71
+#define ANDES_DCACHE_STORE_ACCESS		0x81
+#define ANDES_DCACHE_STORE_MISS			0x91
+#define ANDES_DCACHE_WB				0xA1
+#define ANDES_CYCLE_WAIT_ICACHE_FILL		0xB1
+#define ANDES_CYCLE_WAIT_DCACHE_FILL		0xC1
+#define ANDES_UNCACHED_IFETCH_FROM_BUS		0xD1
+#define ANDES_UNCACHED_LOAD_FROM_BUS		0xE1
+#define ANDES_CYCLE_WAIT_UNCACHED_IFETCH	0xF1
+#define ANDES_CYCLE_WAIT_UNCACHED_LOAD		0x101
+#define ANDES_MAIN_ITLB_ACCESS			0x111
+#define ANDES_MAIN_ITLB_MISS			0x121
+#define ANDES_MAIN_DTLB_ACCESS			0x131
+#define ANDES_MAIN_DTLB_MISS			0x141
+#define ANDES_CYCLE_WAIT_ITLB_FILL		0x151
+#define ANDES_PIPE_STALL_CYCLE_DTLB_MISS	0x161
+#define ANDES_HW_PREFETCH_BUS_ACCESS		0x171
+
+/* Event selector for microarchitecture events */
+#define ANDES_MISPREDICT_CONDITION_BR		0x02
+#define ANDES_MISPREDICT_TAKE_CONDITION_BR	0x12
+#define ANDES_MISPREDICT_TARGET_RET_INST	0x22
+
+#ifdef CONFIG_ANDES_HPM
+
+int andes_pmu_setup(void);
+
+#else
+
+int andes_pmu_setup(void) { return 0; }
+
+#endif /* CONFIG_ANDES_HPM */
+
+#endif /* _ANDES_HPM_H_ */
-- 
2.34.1



From peterlin at andestech.com  Tue Nov 21 23:36:17 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Nov 2023 15:36:17 +0800
Subject: [PATCH v3 15/15] docs: pmu: Add Andes PMU node example
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <20231122073617.379441-16-peterlin@andestech.com>

Add PMU node example for event index to counter index mapping
and selector value translation of Andes' CPUs.

Currently, there are 4 HPM counters that can be used to monitor
all of the events for each hart.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Locus Wei-Han Chen <locus84 at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
Changes v1 -> v2:
  - sync up with datasheet
Changes v2 -> v3:
  - Include Anup's RB tag
---
 docs/pmu_support.md | 82 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/docs/pmu_support.md b/docs/pmu_support.md
index 8cfa08c..9b48f1e 100644
--- a/docs/pmu_support.md
+++ b/docs/pmu_support.md
@@ -125,3 +125,85 @@ pmu {
 					  <0x0 0x2 0xffffffff 0xffffe0ff 0x18>;
 };
 ```
+
+### Example 3
+
+```
+/*
+ * For Andes 45-series platforms. The encodings can be found in the
+ * "Machine Performance Monitoring Event Selector" section
+ * http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
+ */
+pmu {
+	compatible 			= "riscv,pmu";
+	riscv,event-to-mhpmevent =
+					 <0x1 0x0000 0x10>, /* CPU_CYCLES          -> Cycle count */
+					 <0x2 0x0000 0x20>, /* INSTRUCTIONS        -> Retired instruction count */
+					 <0x3 0x0000 0x41>, /* CACHE_REFERENCES    -> D-Cache access */
+					 <0x4 0x0000 0x51>, /* CACHE_MISSES        -> D-Cache miss */
+					 <0x5 0x0000 0x80>, /* BRANCH_INSTRUCTIONS -> Conditional branch instruction count */
+					 <0x6 0x0000 0x02>, /* BRANCH_MISSES       -> Misprediction of conditional branches */
+					 <0x10000 0x0000 0x61>,  /* L1D_READ_ACCESS  -> D-Cache load access */
+					 <0x10001 0x0000 0x71>,  /* L1D_READ_MISS    -> D-Cache load miss */
+					 <0x10002 0x0000 0x81>,  /* L1D_WRITE_ACCESS -> D-Cache store access */
+					 <0x10003 0x0000 0x91>,  /* L1D_WRITE_MISS   -> D-Cache store miss */
+					 <0x10008 0x0000 0x21>,  /* L1I_READ_ACCESS  -> I-Cache access */
+					 <0x10009 0x0000 0x31>;  /* L1I_READ_MISS    -> I-Cache miss */
+	riscv,event-to-mhpmcounters = <0x1 0x6 0x78>,
+							<0x10000 0x10003 0x78>,
+							<0x10008 0x10009 0x78>;
+	riscv,raw-event-to-mhpmcounters =
+						<0x0 0x10 0xffffffff 0xffffffff 0x78>, /* Cycle count */
+						<0x0 0x20 0xffffffff 0xffffffff 0x78>, /* Retired instruction count */
+						<0x0 0x30 0xffffffff 0xffffffff 0x78>, /* Integer load instruction count */
+						<0x0 0x40 0xffffffff 0xffffffff 0x78>, /* Integer store instruction count */
+						<0x0 0x50 0xffffffff 0xffffffff 0x78>, /* Atomic instruction count */
+						<0x0 0x60 0xffffffff 0xffffffff 0x78>, /* System instruction count */
+						<0x0 0x70 0xffffffff 0xffffffff 0x78>, /* Integer computational instruction count */
+						<0x0 0x80 0xffffffff 0xffffffff 0x78>, /* Conditional branch instruction count */
+						<0x0 0x90 0xffffffff 0xffffffff 0x78>, /* Taken conditional branch instruction count */
+						<0x0 0xA0 0xffffffff 0xffffffff 0x78>, /* JAL instruction count */
+						<0x0 0xB0 0xffffffff 0xffffffff 0x78>, /* JALR instruction count */
+						<0x0 0xC0 0xffffffff 0xffffffff 0x78>, /* Return instruction count */
+						<0x0 0xD0 0xffffffff 0xffffffff 0x78>, /* Control transfer instruction count */
+						<0x0 0xE0 0xffffffff 0xffffffff 0x78>, /* EXEC.IT instruction count */
+						<0x0 0xF0 0xffffffff 0xffffffff 0x78>, /* Integer multiplication instruction count */
+						<0x0 0x100 0xffffffff 0xffffffff 0x78>, /* Integer division instruction count */
+						<0x0 0x110 0xffffffff 0xffffffff 0x78>, /* Floating-point load instruction count */
+						<0x0 0x120 0xffffffff 0xffffffff 0x78>, /* Floating-point store instruction count */
+						<0x0 0x130 0xffffffff 0xffffffff 0x78>, /* Floating-point addition/subtraction instruction count */
+						<0x0 0x140 0xffffffff 0xffffffff 0x78>, /* Floating-point multiplication instruction count */
+						<0x0 0x150 0xffffffff 0xffffffff 0x78>, /* Floating-point fused multiply-add instruction count */
+						<0x0 0x160 0xffffffff 0xffffffff 0x78>, /* Floating-point division or square-root instruction count */
+						<0x0 0x170 0xffffffff 0xffffffff 0x78>, /* Other floating-point instruction count */
+						<0x0 0x180 0xffffffff 0xffffffff 0x78>, /* Integer multiplication and add/sub instruction count */
+						<0x0 0x190 0xffffffff 0xffffffff 0x78>, /* Retired operation count */
+						<0x0 0x01 0xffffffff 0xffffffff 0x78>, /* ILM access */
+						<0x0 0x11 0xffffffff 0xffffffff 0x78>, /* DLM access */
+						<0x0 0x21 0xffffffff 0xffffffff 0x78>, /* I-Cache access */
+						<0x0 0x31 0xffffffff 0xffffffff 0x78>, /* I-Cache miss */
+						<0x0 0x41 0xffffffff 0xffffffff 0x78>, /* D-Cache access */
+						<0x0 0x51 0xffffffff 0xffffffff 0x78>, /* D-Cache miss */
+						<0x0 0x61 0xffffffff 0xffffffff 0x78>, /* D-Cache load access */
+						<0x0 0x71 0xffffffff 0xffffffff 0x78>, /* D-Cache load miss */
+						<0x0 0x81 0xffffffff 0xffffffff 0x78>, /* D-Cache store access */
+						<0x0 0x91 0xffffffff 0xffffffff 0x78>, /* D-Cache store miss */
+						<0x0 0xA1 0xffffffff 0xffffffff 0x78>, /* D-Cache writeback */
+						<0x0 0xB1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for I-Cache fill data */
+						<0x0 0xC1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for D-Cache fill data */
+						<0x0 0xD1 0xffffffff 0xffffffff 0x78>, /* Uncached fetch data access from bus */
+						<0x0 0xE1 0xffffffff 0xffffffff 0x78>, /* Uncached load data access from bus */
+						<0x0 0xF1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for uncached fetch data from bus */
+						<0x0 0x101 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for uncached load data from bus */
+						<0x0 0x111 0xffffffff 0xffffffff 0x78>, /* Main ITLB access */
+						<0x0 0x121 0xffffffff 0xffffffff 0x78>, /* Main ITLB miss */
+						<0x0 0x131 0xffffffff 0xffffffff 0x78>, /* Main DTLB access */
+						<0x0 0x141 0xffffffff 0xffffffff 0x78>, /* Main DTLB miss */
+						<0x0 0x151 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for Main ITLB fill data */
+						<0x0 0x161 0xffffffff 0xffffffff 0x78>, /* Pipeline stall cycles caused by Main DTLB miss */
+						<0x0 0x171 0xffffffff 0xffffffff 0x78>, /* Hardware prefetch bus access */
+						<0x0 0x02 0xffffffff 0xffffffff 0x78>, /* Misprediction of conditional branches */
+						<0x0 0x12 0xffffffff 0xffffffff 0x78>, /* Misprediction of taken conditional branches */
+						<0x0 0x22 0xffffffff 0xffffffff 0x78>; /* Misprediction of targets of Return instructions */
+};
+```
-- 
2.34.1



From guoren at kernel.org  Wed Nov 22 00:46:51 2023
From: guoren at kernel.org (Guo Ren)
Date: Wed, 22 Nov 2023 16:46:51 +0800
Subject: [PATCH] platform: recalculate heap size to support new tlb entry number
In-Reply-To: <IA1PR20MB4953BC55F07EE072FE9E4288BBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>
References: <IA1PR20MB4953BC55F07EE072FE9E4288BBB7A@IA1PR20MB4953.namprd20.prod.outlook.com>
Message-ID: <CAJF2gTRuxfGu+VxCqsSEem9KU90vRs+R-QaATt8rKHdwZPj9gQ@mail.gmail.com>

On Fri, Nov 17, 2023 at 8:10?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>
> Previous patch introduct a change that using hart count as the default
> number of tlb entry in the fifo. This makes the default tlb fifo size
> grow in square with the number of harts. So the default heap size is
> not enough to allocate tlb fifo when the hart count is big.
>
> Use new formula for default heap size:
> 0x40 * (__num_hart) ^ 2   -> handle the tlb fifo
> 0x600 * (__num_hart)      -> remove previous static fifo allocation
Yes, ^ 2 is not suitable for sg2042.

>
> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
> Fixes: 52fd64b ("platform: Uses hart count as the default size of tlb info")
> ---
>  include/sbi/sbi_bitops.h   | 1 +
>  include/sbi/sbi_heap.h     | 3 +++
>  include/sbi/sbi_platform.h | 8 ++++++--
>  lib/sbi/sbi_heap.c         | 2 --
>  4 files changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/include/sbi/sbi_bitops.h b/include/sbi/sbi_bitops.h
> index 48a2090..7d90334 100644
> --- a/include/sbi/sbi_bitops.h
> +++ b/include/sbi/sbi_bitops.h
> @@ -26,6 +26,7 @@
>  #define BIT_MASK(nr)           (1UL << ((nr) % BITS_PER_LONG))
>  #define BIT_WORD(bit)          ((bit) / BITS_PER_LONG)
>  #define BIT_WORD_OFFSET(bit)   ((bit) & (BITS_PER_LONG - 1))
> +#define BIT_ALIGN(bit, align)  (((bit) + ((align) - 1)) & ~((align) - 1))
>
>  #define GENMASK(h, l) \
>         (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
> diff --git a/include/sbi/sbi_heap.h b/include/sbi/sbi_heap.h
> index 88d176e..16755ec 100644
> --- a/include/sbi/sbi_heap.h
> +++ b/include/sbi/sbi_heap.h
> @@ -12,6 +12,9 @@
>
>  #include <sbi/sbi_types.h>
>
> +/* Alignment of heap base address and size */
> +#define HEAP_BASE_ALIGN                        1024
> +
>  struct sbi_scratch;
>
>  /** Allocate from heap area */
> diff --git a/include/sbi/sbi_platform.h b/include/sbi/sbi_platform.h
> index 58b9069..609f608 100644
> --- a/include/sbi/sbi_platform.h
> +++ b/include/sbi/sbi_platform.h
> @@ -44,8 +44,10 @@
>
>  #ifndef __ASSEMBLER__
>
> +#include <sbi/sbi_bitops.h>
>  #include <sbi/sbi_ecall_interface.h>
>  #include <sbi/sbi_error.h>
> +#include <sbi/sbi_heap.h>
>  #include <sbi/sbi_scratch.h>
>  #include <sbi/sbi_version.h>
>
> @@ -146,8 +148,10 @@ struct sbi_platform_operations {
>  #define SBI_PLATFORM_DEFAULT_HART_STACK_SIZE   8192
>
>  /** Platform default heap size */
> -#define SBI_PLATFORM_DEFAULT_HEAP_SIZE(__num_hart)     \
> -                                       (0x8000 + 0x800 * (__num_hart))
> +#define SBI_PLATFORM_DEFAULT_HEAP_SIZE(__num_hart)             \
> +       BIT_ALIGN((0x8000 + 0x600 * (__num_hart) +              \
> +                  0x40 * (__num_hart) * (__num_hart)),         \
> +                 HEAP_BASE_ALIGN)
>
>  /** Representation of a platform */
>  struct sbi_platform {
> diff --git a/lib/sbi/sbi_heap.c b/lib/sbi/sbi_heap.c
> index 698c377..bcd404b 100644
> --- a/lib/sbi/sbi_heap.c
> +++ b/lib/sbi/sbi_heap.c
> @@ -14,8 +14,6 @@
>  #include <sbi/sbi_scratch.h>
>  #include <sbi/sbi_string.h>
>
> -/* Alignment of heap base address and size */
> -#define HEAP_BASE_ALIGN                        1024
>  /* Minimum size and alignment of heap allocations */
>  #define HEAP_ALLOC_ALIGN               64
>  #define HEAP_HOUSEKEEPING_FACTOR       16
> --
> 2.42.1
>
Maybe the number of 0x600 needs more discussion. Other:
Reviewed-by: Guo Ren <guoren at kernel.org>

-- 
Best Regards
 Guo Ren


From wxjstz at 126.com  Wed Nov 22 04:54:38 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 22 Nov 2023 20:54:38 +0800
Subject: [PATCH] lib: sbi: Prevent redundant sbi_ipi_process
Message-ID: <20231122125444.333876-1-wxjstz@126.com>

sbi_ipi_process reads ipi_type through xchg. This may occur between
set ipi_type and send_ipi, which will result in redundant interrupt
processing.

The specific case is as follows:

                hart A             hart B
send ipi--------->|                  |
                  | clear ipi        |
                  |                  | set ipi_type
                  |                  |
                  | xchg ipi_type    |
                  |                  |
                  |<-----------------| send ipi
                  |                  |
                  |                  |

This patch fixes this issue.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_ipi.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 0bf446a..6f25a7e 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -39,7 +39,7 @@ static const struct sbi_ipi_event_ops *ipi_ops_array[SBI_IPI_EVENT_MAX];
 static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,
 			u32 event, void *data)
 {
-	int ret;
+	int ret = 0;
 	struct sbi_scratch *remote_scratch = NULL;
 	struct sbi_ipi_data *ipi_data;
 	const struct sbi_ipi_event_ops *ipi_ops;
@@ -66,9 +66,8 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,
 	 * Set IPI type on remote hart's scratch area and
 	 * trigger the interrupt
 	 */
-	atomic_raw_set_bit(event, &ipi_data->ipi_type);
-
-	ret = sbi_ipi_raw_send(remote_hartindex);
+	if (__atomic_fetch_or(&ipi_data->ipi_type, BIT(event), __ATOMIC_ACQ_REL) == 0)
+		ret = sbi_ipi_raw_send(remote_hartindex);
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
 
-- 
2.42.0



From wxjstz at 126.com  Wed Nov 22 04:57:13 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 22 Nov 2023 20:57:13 +0800
Subject: [PATCH] lib: sbi: Prevent redundant sbi_ipi_process
In-Reply-To: <20231122125444.333876-1-wxjstz@126.com>
References: <20231122125444.333876-1-wxjstz@126.com>
Message-ID: <cad90ca66225ff5c548f9563771cadef3cf5e41e.camel@126.com>

? 2023-11-22???? 20:54 +0800?Xiang W???
> sbi_ipi_process reads ipi_type through xchg. This may occur between
> set ipi_type and send_ipi, which will result in redundant interrupt
> processing.
> 
> The specific case is as follows:
> 
> ??????????????? hart A???????????? hart B
> send ipi--------->|????????????????? |
> ????????????????? | clear ipi??????? |
> ????????????????? |????????????????? | set ipi_type
> ????????????????? |????????????????? |
> ????????????????? | xchg ipi_type??? |
> ????????????????? |????????????????? |
> ????????????????? |<-----------------| send ipi
> ????????????????? |????????????????? |
> ????????????????? |????????????????? |
> 
> This patch fixes this issue.

This patch base on
https://github.com/avpatel/opensbi/tree/sbi_ipi_barrier_imp_v1

Regards,
Xiang W
> 
> Signed-off-by: Xiang W <wxjstz at 126.com>
> ---
> ?lib/sbi/sbi_ipi.c | 7 +++----
> ?1 file changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
> index 0bf446a..6f25a7e 100644
> --- a/lib/sbi/sbi_ipi.c
> +++ b/lib/sbi/sbi_ipi.c
> @@ -39,7 +39,7 @@ static const struct sbi_ipi_event_ops *ipi_ops_array[SBI_IPI_EVENT_MAX];
> ?static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,
> ?			u32 event, void *data)
> ?{
> -	int ret;
> +	int ret = 0;
> ?	struct sbi_scratch *remote_scratch = NULL;
> ?	struct sbi_ipi_data *ipi_data;
> ?	const struct sbi_ipi_event_ops *ipi_ops;
> @@ -66,9 +66,8 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartindex,
> ?	 * Set IPI type on remote hart's scratch area and
> ?	 * trigger the interrupt
> ?	 */
> -	atomic_raw_set_bit(event, &ipi_data->ipi_type);
> -
> -	ret = sbi_ipi_raw_send(remote_hartindex);
> +	if (__atomic_fetch_or(&ipi_data->ipi_type, BIT(event), __ATOMIC_ACQ_REL) == 0)
> +		ret = sbi_ipi_raw_send(remote_hartindex);
> ?
> ?	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
> ?


From wxjstz at 126.com  Wed Nov 22 04:59:04 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 22 Nov 2023 20:59:04 +0800
Subject: [PATCH 0/2] Allow relaxed MMIO writes in device IPI callbacks
In-Reply-To: <CAK9=C2XMVBJq4g7RxEZQ_k4DuT_v85qBm4UafaueoDFxzdRLyw@mail.gmail.com>
References: <20231121161711.904674-1-apatel@ventanamicro.com>
	 <b36f2dd2b4e99149394735b9d622d96ad062a4ea.camel@126.com>
	 <CAK9=C2XMVBJq4g7RxEZQ_k4DuT_v85qBm4UafaueoDFxzdRLyw@mail.gmail.com>
Message-ID: <81a431789d6ef8281e563a390c38a2ee68e3e0e3.camel@126.com>

? 2023-11-21???? 23:03 +0530?Anup Patel???
> On Tue, Nov 21, 2023 at 10:18?PM Xiang W <wxjstz at 126.com> wrote:
> > 
> > ? 2023-11-21???? 21:47 +0530?Anup Patel???
> > > This series improves the barrier usage around device IPI callbacks
> > > such that platform IPI driver can now use relaxed MMIO writes in
> > > device IPI callbacks.
> > > 
> > > These patches can also be found in the sbi_ipi_barrier_imp_v1 branch
> > > at: https://github.com/avpatel/opensbi.git
> > This patch is partly similar to this one?
> > https://lists.infradead.org/pipermail/opensbi/2023-November/005931.html
> > 
> > Then we may also encounter the following case:
> > 
> > ??????????????? hart A???????????? hart B
> > send ipi--------->|????????????????? |
> > ????????????????? | clear ipi??????? |
> > ????????????????? |????????????????? | set ipi_type
> > ????????????????? |????????????????? |
> > ????????????????? | xchg ipi_type??? |
> > ????????????????? |????????????????? |
> > ????????????????? |<-----------------| send ipi
> > ????????????????? |????????????????? |
> > ????????????????? |????????????????? |
> 
> This case is handled.
> 
> > 
> > To deal with this case, my patch:
> > https://lists.infradead.org/pipermail/opensbi/2023-November/005932.html
> 
> Using spinlock for IPI synchronization is very handed
> and reduces performance.
> 
> Functionally we have an IPI mux built on-top of one HW IPI
> which is functionally similar to the Linux generic IPI-mux.

New patch at
https://lists.infradead.org/pipermail/opensbi/2023-November/005965.html

Regards,
Xiang W

> 
> Regards,
> Anup
> 
> > 
> > Regards,
> > Xiang W
> > > 
> > > Anup Patel (2):
> > > ? lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
> > > ? lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback
> > > 
> > > ?lib/sbi/sbi_ipi.c?????????? | 31 +++++++++++++++++++++++++------
> > > ?lib/utils/ipi/aclint_mswi.c |? 4 ++--
> > > ?lib/utils/irqchip/imsic.c?? |? 4 ++--
> > > ?3 files changed, 29 insertions(+), 10 deletions(-)
> > > 
> > > --
> > > 2.34.1
> > > 
> > > 
> > 


From wxjstz at 126.com  Wed Nov 22 05:03:22 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 22 Nov 2023 21:03:22 +0800
Subject: [PATCH 2/2] lib: sbi: Make ipi send/clear and access ipi_type
 non-reentrant
In-Reply-To: <06a9c1b4-43d9-4fea-3001-047d55ec3c75@gmail.com>
References: <20231120095301.105772-1-wxjstz@126.com>
	 <20231120095301.105772-2-wxjstz@126.com>
	 <3f1751b2c8b0437e72543e2d25f8ef6ae40512fe.camel@126.com>
	 <06a9c1b4-43d9-4fea-3001-047d55ec3c75@gmail.com>
Message-ID: <132fa69f18f4d3f2cd2ddb8d6cb1e4f6e67db618.camel@126.com>

? 2023-11-21???? 16:56 -0800?Bo Gan???
> > > @@ -220,8 +224,12 @@ void sbi_ipi_process(void)
> > > ??	u32 hartindex = sbi_hartid_to_hartindex(current_hartid());
> > > ? 
> > > ??	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_RECVD);
> > > +
> > > +	spin_lock(&ipi_data->lock);
> > > ??	sbi_ipi_raw_clear(hartindex);
> > > ??	ipi_type = atomic_raw_xchg_ulong(&ipi_data->ipi_type, 0);
> > > +	spin_unlock(&ipi_data->lock);
> > > +
> > > ??	ipi_event = 0;
> > > ??	while (ipi_type) {
> > > ??		if (ipi_type & 1UL) {
> 
> Hi Xiang,
> 
> I think we should drop this, and use Anup's new patch instead. I don't
> think the spinlock here is helping to ensure the ordering between
> 
> sbi_ipi_raw_clear and xchg(). Furthermore, spinlock in OpenSBI doesn't
> order IO, so ipi_raw_clear can leak below spin_unlock. Thus, it won't
> address the race condition in previous discussions.
> 
> With Anip's new patch that uses wmb() in both sender and receiver, it
> fixes the correctness issue, and hopefully has negligible performance
> impact: See the discussion here:
> 
> http://lists.infradead.org/pipermail/opensbi/2023-November/005937.html
> 
> Bo
This bug is not related to reorder. It's one more ipi interrupt is sent.

Regards,
Xiang W



From anup at brainfault.org  Wed Nov 22 07:14:52 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 22 Nov 2023 20:44:52 +0530
Subject: [PATCH] lib: sbi: Fix PMP granularity handling in sbi_hart_map_saddr()
In-Reply-To: <67d181fd-e4b7-47d1-b089-f5b1c4da29b1@sifive.com>
References: <20231120174915.812345-1-apatel@ventanamicro.com> <67d181fd-e4b7-47d1-b089-f5b1c4da29b1@sifive.com>
Message-ID: <CAAhSdy3b+N1JTe7VMEvzpWWg3J+jCkvu80xuJ40C3yPtOJZpCg@mail.gmail.com>

On Wed, Nov 22, 2023 at 6:00?AM Samuel Holland
<samuel.holland at sifive.com> wrote:
>
> Hi Anup,
>
> On 2023-11-20 11:49 AM, Anup Patel wrote:
> > The sbi_hart_map_saddr() must create PMP mapping of size greater
> > than or equal to PMP granularity otherwise PMP mapping does not
> > work when size parameter less than sbi_hart_pmp_granularity(scratch).
> >
> > Fixes: 6e44ef686a9b ("lib: sbi: Add functions to map/unmap shared
> > memory")
>
> This line should not be wrapped. Otherwise,
>
> Reviewed-by: Samuel Holland <samuel.holland at sifive.com>

Thanks, updated the commit description at the time of merging this patch.

Applied this patch to the riscv/opensbi repo.

>
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > ---
> >  lib/sbi/sbi_hart.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> > index 77eef49..bf8792a 100644
> > --- a/lib/sbi/sbi_hart.c
> > +++ b/lib/sbi/sbi_hart.c
> > @@ -508,7 +508,8 @@ int sbi_hart_map_saddr(unsigned long addr, unsigned long size)
> >       if (is_pmp_entry_mapped(SBI_SMEPMP_RESV_ENTRY))
> >               return SBI_ENOSPC;
> >
> > -     for (order = log2roundup(size) ; order <= __riscv_xlen; order++) {
> > +     for (order = log2roundup(MAX(sbi_hart_pmp_granularity(scratch), size));
>
> I noticed that sbi_hart_pmp_granularity() is always passed to log2roundup(), and
> hfeatures->pmp_gran is originally generated by shifting, so maybe the value
> should be stored as the log base 2 to avoid the conversions?

I agree with your suggestion. Please go ahead and submit a patch for this.

>
> Regards,
> Samuel
>
> > +          order <= __riscv_xlen; order++) {
> >               if (order < __riscv_xlen) {
> >                       base = addr & ~((1UL << order) - 1UL);
> >                       if ((base <= addr) &&
>

Regards,
Anup


From anup at brainfault.org  Wed Nov 22 07:29:27 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 22 Nov 2023 20:59:27 +0530
Subject: [PATCH 1/1] lib: sbi_pmu: avoid buffer overflow
In-Reply-To: <20231117135931.26666-1-heinrich.schuchardt@canonical.com>
References: <20231117135931.26666-1-heinrich.schuchardt@canonical.com>
Message-ID: <CAAhSdy33g04XKigsAEHT2HnG27=DKcMZWZ8D9=jwLGp9Uym7Pw@mail.gmail.com>

On Fri, Nov 17, 2023 at 7:29?PM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> total_ctrs is bounded by
>
>     SBI_PMU_FW_CTR_MAX + SBI_PMU_HW_CTR_MAX) == 48
>
> which exceeds BITS_PER_LONG on 32 bit systems.
>
> Iterating over the bits of &cmask results in a buffer overflow when looking
> for a bit >= BITS_PER_LONG.
>
> Adjust the iterators in sbi_pmu_ctr_start() and sbi_pmu_ctr_stop()
> accordingly.
>
> Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_pmu.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index f4c8fc4..185068b 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -445,7 +445,7 @@ int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
>         if (flags & SBI_PMU_START_FLAG_SET_INIT_VALUE)
>                 bUpdate = true;
>
> -       for_each_set_bit(i, &cmask, total_ctrs) {
> +       for_each_set_bit(i, &cmask, BITS_PER_LONG) {
>                 cidx = i + cbase;
>                 event_idx_type = pmu_ctr_validate(phs, cidx, &event_code);
>                 if (event_idx_type < 0)
> @@ -540,7 +540,7 @@ int sbi_pmu_ctr_stop(unsigned long cbase, unsigned long cmask,
>         if ((cbase + sbi_fls(cmask)) >= total_ctrs)
>                 return SBI_EINVAL;
>
> -       for_each_set_bit(i, &cmask, total_ctrs) {
> +       for_each_set_bit(i, &cmask, BITS_PER_LONG) {
>                 cidx = i + cbase;
>                 event_idx_type = pmu_ctr_validate(phs, cidx, &event_code);
>                 if (event_idx_type < 0)
> --
> 2.40.1
>


From samuel.holland at sifive.com  Wed Nov 22 11:44:04 2023
From: samuel.holland at sifive.com (Samuel Holland)
Date: Wed, 22 Nov 2023 11:44:04 -0800
Subject: [RFC PATCH] lib: sbi_hart: Store PMP granularity as log base 2
Message-ID: <20231122194406.229255-1-samuel.holland@sifive.com>

This minimizes the need to call log2roundup() to recover the log value.

Signed-off-by: Samuel Holland <samuel.holland at sifive.com>
---
This actually ends up with a net increase in code size, so it's probably
not worth merging. I'm sending the patch for completeness.

 include/sbi/sbi_hart.h |  4 ++--
 lib/sbi/sbi_hart.c     | 10 +++++-----
 lib/sbi/sbi_init.c     |  2 +-
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
index 6ee49ff..cbcac78 100644
--- a/include/sbi/sbi_hart.h
+++ b/include/sbi/sbi_hart.h
@@ -71,7 +71,7 @@ struct sbi_hart_features {
 	unsigned long extensions[BITS_TO_LONGS(SBI_HART_EXT_MAX)];
 	unsigned int pmp_count;
 	unsigned int pmp_addr_bits;
-	unsigned long pmp_gran;
+	unsigned int pmp_gran_log2;
 	unsigned int mhpm_mask;
 	unsigned int mhpm_bits;
 };
@@ -91,7 +91,7 @@ unsigned int sbi_hart_mhpm_mask(struct sbi_scratch *scratch);
 void sbi_hart_delegation_dump(struct sbi_scratch *scratch,
 			      const char *prefix, const char *suffix);
 unsigned int sbi_hart_pmp_count(struct sbi_scratch *scratch);
-unsigned long sbi_hart_pmp_granularity(struct sbi_scratch *scratch);
+unsigned int sbi_hart_pmp_gran_log2(struct sbi_scratch *scratch);
 unsigned int sbi_hart_pmp_addrbits(struct sbi_scratch *scratch);
 unsigned int sbi_hart_mhpm_bits(struct sbi_scratch *scratch);
 int sbi_hart_pmp_configure(struct sbi_scratch *scratch);
diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index bf8792a..5141160 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -267,12 +267,12 @@ unsigned int sbi_hart_pmp_count(struct sbi_scratch *scratch)
 	return hfeatures->pmp_count;
 }
 
-unsigned long sbi_hart_pmp_granularity(struct sbi_scratch *scratch)
+unsigned int sbi_hart_pmp_gran_log2(struct sbi_scratch *scratch)
 {
 	struct sbi_hart_features *hfeatures =
 			sbi_scratch_offset_ptr(scratch, hart_features_offset);
 
-	return hfeatures->pmp_gran;
+	return hfeatures->pmp_gran_log2;
 }
 
 unsigned int sbi_hart_pmp_addrbits(struct sbi_scratch *scratch)
@@ -508,7 +508,7 @@ int sbi_hart_map_saddr(unsigned long addr, unsigned long size)
 	if (is_pmp_entry_mapped(SBI_SMEPMP_RESV_ENTRY))
 		return SBI_ENOSPC;
 
-	for (order = log2roundup(MAX(sbi_hart_pmp_granularity(scratch), size));
+	for (order = MAX(sbi_hart_pmp_gran_log2(scratch), log2roundup(size));
 	     order <= __riscv_xlen; order++) {
 		if (order < __riscv_xlen) {
 			base = addr & ~((1UL << order) - 1UL);
@@ -547,7 +547,7 @@ int sbi_hart_pmp_configure(struct sbi_scratch *scratch)
 	if (!pmp_count)
 		return 0;
 
-	pmp_gran_log2 = log2roundup(sbi_hart_pmp_granularity(scratch));
+	pmp_gran_log2 = sbi_hart_pmp_gran_log2(scratch);
 	pmp_bits = sbi_hart_pmp_addrbits(scratch) - 1;
 	pmp_addr_max = (1UL << pmp_bits) | ((1UL << pmp_bits) - 1);
 
@@ -871,7 +871,7 @@ static int hart_detect_features(struct sbi_scratch *scratch)
 	 */
 	val = hart_pmp_get_allowed_addr();
 	if (val) {
-		hfeatures->pmp_gran =  1 << (sbi_ffs(val) + 2);
+		hfeatures->pmp_gran_log2 = sbi_ffs(val) + 2;
 		hfeatures->pmp_addr_bits = sbi_fls(val) + 1;
 		/* Detect number of PMP regions. At least PMPADDR0 should be implemented*/
 		__check_csr_64(CSR_PMPADDR0, 0, val, pmp_count, __pmp_skip);
diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
index e723553..bfb8300 100644
--- a/lib/sbi/sbi_init.c
+++ b/lib/sbi/sbi_init.c
@@ -177,7 +177,7 @@ static void sbi_boot_print_hart(struct sbi_scratch *scratch, u32 hartid)
 	sbi_printf("Boot HART PMP Count       : %d\n",
 		   sbi_hart_pmp_count(scratch));
 	sbi_printf("Boot HART PMP Granularity : %lu\n",
-		   sbi_hart_pmp_granularity(scratch));
+		   1UL << sbi_hart_pmp_gran_log2(scratch));
 	sbi_printf("Boot HART PMP Address Bits: %d\n",
 		   sbi_hart_pmp_addrbits(scratch));
 	sbi_printf("Boot HART MHPM Info       : %lu (0x%08x)\n",
-- 
2.42.0



From atishp at atishpatra.org  Wed Nov 22 15:41:28 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Wed, 22 Nov 2023 15:41:28 -0800
Subject: [PATCH v3 10/15] lib: utils: fdt_fixup: Allow preserving PMU properties
In-Reply-To: <20231122073617.379441-11-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-11-peterlin@andestech.com>
Message-ID: <CAOnJCULG1pDE4ukBJqHuePAwzL8gqeWbB3RO6_nYJeCzjP4=Hw@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:40?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add a scratch option to control PMU fixup, so the next
> stage software can dump the PMU node including event
> mapping information for debugging purposes.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes v1 -> v2:
>   - New patch
> Changes v2 -> v3:
>   - Rename to SBI_SCRATCH_PRESERVE_PMU_NODE (suggested by Anup)
>   - Include Anup's RB tag
> ---
>  include/sbi/sbi_scratch.h | 2 ++
>  lib/utils/fdt/fdt_fixup.c | 6 +++++-
>  2 files changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/include/sbi/sbi_scratch.h b/include/sbi/sbi_scratch.h
> index e6a33ba..0f67cde 100644
> --- a/include/sbi/sbi_scratch.h
> +++ b/include/sbi/sbi_scratch.h
> @@ -151,6 +151,8 @@ enum sbi_scratch_options {
>         SBI_SCRATCH_NO_BOOT_PRINTS = (1 << 0),
>         /** Enable runtime debug prints */
>         SBI_SCRATCH_DEBUG_PRINTS = (1 << 1),
> +       /** Preserve PMU node properties */
> +       SBI_SCRATCH_PRESERVE_PMU_NODE = (1 << 2),
>  };
>
>  /** Get pointer to sbi_scratch for current HART */
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index e213ded..cf20edf 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -387,6 +387,8 @@ int fdt_reserved_memory_fixup(void *fdt)
>
>  void fdt_fixups(void *fdt)
>  {
> +       struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> +
>         fdt_aplic_fixup(fdt);
>
>         fdt_imsic_fixup(fdt);
> @@ -394,5 +396,7 @@ void fdt_fixups(void *fdt)
>         fdt_plic_fixup(fdt);
>
>         fdt_reserved_memory_fixup(fdt);
> -       fdt_pmu_fixup(fdt);
> +
> +       if (!(scratch->options & SBI_SCRATCH_PRESERVE_PMU_NODE))
> +               fdt_pmu_fixup(fdt);
>  }
> --
> 2.34.1
>


Reviewed-by: Atish Patra <atishp at rivosinc.com>
-- 
Regards,
Atish


From atishp at atishpatra.org  Wed Nov 22 15:47:43 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Wed, 22 Nov 2023 15:47:43 -0800
Subject: [PATCH v3 02/15] sbi: sbi_pmu: Improve sbi_pmu_init() error handling
In-Reply-To: <20231122073617.379441-3-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-3-peterlin@andestech.com>
Message-ID: <CAOnJCUJ14BFthoaBoroQYFDHZqnq60mLTfQ8A7AB9EXo4_yKBA@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:39?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> This patch makes the following changes:
>
> - As sbi_platform_pmu_init() returns a negative error code on
>   failure, let sbi_pmu_init() to hang by propagating the error
>   code.
>

Why ? PMU is not a critical resource for booting the system. A system
can function without it.
Do we really want to hang the system booting in case sbi_pmu_init fails ?

> - In order to distinguish the SBI_EFAIL error returned by
>   sbi_pmu_add_*_counter_map(), return SBI_ENOENT to indicate
>   that fdt_pmu_setup() failed to locate "riscv,pmu" node, and
>   generic_pmu_init() ignores such case.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes v1 -> v2:
>   - New patch
> Changes v2 -> v3:
>   - Include Anup's RB tag
> ---
>  lib/sbi/sbi_pmu.c           | 5 ++++-
>  lib/utils/fdt/fdt_pmu.c     | 2 +-
>  platform/generic/platform.c | 8 +++++++-
>  3 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index f4c8fc4..3cbd4ff 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -957,6 +957,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
>         int hpm_count = sbi_fls(sbi_hart_mhpm_mask(scratch));
>         struct sbi_pmu_hart_state *phs;
>         const struct sbi_platform *plat;
> +       int rc;
>
>         if (cold_boot) {
>                 hw_event_map = sbi_calloc(sizeof(*hw_event_map),
> @@ -972,7 +973,9 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
>
>                 plat = sbi_platform_ptr(scratch);
>                 /* Initialize hw pmu events */
> -               sbi_platform_pmu_init(plat);
> +               rc = sbi_platform_pmu_init(plat);
> +               if (rc)
> +                       return rc;
>
>                 /* mcycle & minstret is available always */
>                 if (!hpm_count)
> diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
> index 83301bb..a8d7648 100644
> --- a/lib/utils/fdt/fdt_pmu.c
> +++ b/lib/utils/fdt/fdt_pmu.c
> @@ -74,7 +74,7 @@ int fdt_pmu_setup(void *fdt)
>
>         pmu_offset = fdt_node_offset_by_compatible(fdt, -1, "riscv,pmu");
>         if (pmu_offset < 0)
> -               return SBI_EFAIL;
> +               return SBI_ENOENT;
>
>         event_ctr_map = fdt_getprop(fdt, pmu_offset,
>                                     "riscv,event-to-mhpmcounters", &len);
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 85acecd..fa400b9 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -265,7 +265,13 @@ static u32 generic_tlb_num_entries(void)
>
>  static int generic_pmu_init(void)
>  {
> -       return fdt_pmu_setup(fdt_get_address());
> +       int rc;
> +
> +       rc = fdt_pmu_setup(fdt_get_address());
> +       if (rc && rc != SBI_ENOENT)
> +               return rc;
> +
> +       return 0;
>  }
>
>  static uint64_t generic_pmu_xlate_to_mhpmevent(uint32_t event_idx,
> --
> 2.34.1
>


-- 
Regards,
Atish


From atishp at atishpatra.org  Wed Nov 22 16:02:45 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Wed, 22 Nov 2023 16:02:45 -0800
Subject: [PATCH v3 03/15] lib: sbi: Add Xandespmu in hart extensions
In-Reply-To: <20231122073617.379441-4-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-4-peterlin@andestech.com>
Message-ID: <CAOnJCUKTWkMSELNHDmMChU3onCHSF5LdqpciXyikyKtimhtvNw@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:40?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> When SBI PMU updates inhibit bits on mcyclecfg and
> minstretcfg, it will check if Smcntrpmf extension
> is supported, we need to check Xandespmu similarly
> in the pmu_fixed_ctr_update_inhibit_bits() so add
> the custom extension to hart extension list.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> Changes v2 -> v3:
>   - New patch
> ---
>  include/sbi/sbi_hart.h | 2 ++
>  lib/sbi/sbi_hart.c     | 3 +++
>  2 files changed, 5 insertions(+)
>
> diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
> index 6ee49ff..811e5f5 100644
> --- a/include/sbi/sbi_hart.h
> +++ b/include/sbi/sbi_hart.h
> @@ -45,6 +45,8 @@ enum sbi_hart_extensions {
>         SBI_HART_EXT_ZKR,
>         /** Hart has Smcntrpmf extension */
>         SBI_HART_EXT_SMCNTRPMF,
> +       /** Hart has Xandespmu extension */
> +       SBI_HART_EXT_XANDESPMU,
>
>         /** Maximum index of Hart extension */
>         SBI_HART_EXT_MAX,
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 77eef49..d372af1 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -693,6 +693,9 @@ static inline char *sbi_hart_extension_id2string(int ext)
>         case SBI_HART_EXT_SMCNTRPMF:
>                 estr = "smcntrpmf";
>                 break;
> +       case SBI_HART_EXT_XANDESPMU:
> +               estr = "xandespmu";
> +               break;
>         default:
>                 break;
>         }
> --
> 2.34.1
>


Reviewed-by: Atish Patra <atishp at rivosinc.com>

-- 
Regards,
Atish


From atishp at atishpatra.org  Wed Nov 22 16:07:43 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Wed, 22 Nov 2023 16:07:43 -0800
Subject: [PATCH v3 00/15] Add Andes PMU extension support
In-Reply-To: <20231122073617.379441-1-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
Message-ID: <CAOnJCU+qacoXRrs3MDTyskEnJWJ-vUds57m8rKDq7UedkOhqVA@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:39?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> This patch series enables perf tool to utilize Andes PMU
> extension via PMU SBI calls, provides PMU device callbacks
> to achieve event sampling and mode filtering.
>
> This version mainly fixes single-core freeze when quitting
> 'perf top', adds andes_pmu_setup() to provide default event
> mappings when fdt_pmu_setup() can't find PMU node. And add
> Xandespmu to hart extension list so that the inhibit bits
> can be updated in a similar way to how Smcntrpmf does.
>
> The last patch provides a PMU node example used on AX45MP cores.
>

Do you have a publicly available spec that describes what XAndesPMU does ?

> The OpenSBI and Linux patches can be found on Andes Technology GitHub
> - https://github.com/andestech/opensbi/commits/andes-pmu-support-v3
> - https://github.com/andestech/linux/commits/andes-pmu-support-v4
>
> Leo Yu-Chi Liang (1):
>   lib: ipi: Adjust Andes PLICSW to single-bit-per-hart scheme
>
> Yu Chien Peter Lin (14):
>   sbi: sbi_pmu: Improve sbi_pmu_init() error handling
>   lib: sbi: Add Xandespmu in hart extensions
>   sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
>   platform: include: andes45: Add PMU related CSR defines
>   platform: generic: Introduce pmu_init() platform override
>   platform: andes: Add Andes custom PMU support
>   platform: andes: Enable Andes PMU for AE350
>   platform: rzfive: Enable Andes PMU for RZ/Five
>   lib: utils: fdt_fixup: Allow preserving PMU properties
>   platform: andes: Factor out is_andes() helper
>   lib: utils: fdt_pmu: Make the fdt_pmu_evt_select table global variable
>   lib: utils: fdt_pmu: Do not iterate over the fdt_pmu_evt_select table
>   platform: andes: Add Andes default PMU mapping support
>   docs: pmu: Add Andes PMU node example
>
>  docs/pmu_support.md                          |  82 ++++
>  include/sbi/sbi_ecall_interface.h            |   5 +
>  include/sbi/sbi_hart.h                       |   2 +
>  include/sbi/sbi_pmu.h                        |   6 +
>  include/sbi/sbi_scratch.h                    |   2 +
>  include/sbi_utils/fdt/fdt_helper.h           |  17 +
>  include/sbi_utils/fdt/fdt_pmu.h              |   6 +
>  include/sbi_utils/ipi/andes_plicsw.h         |  23 +-
>  lib/sbi/sbi_hart.c                           |   3 +
>  lib/sbi/sbi_pmu.c                            |  25 +-
>  lib/utils/fdt/fdt_fixup.c                    |   6 +-
>  lib/utils/fdt/fdt_pmu.c                      |  17 +-
>  lib/utils/ipi/andes_plicsw.c                 | 104 ++---
>  platform/generic/Kconfig                     |   4 +
>  platform/generic/andes/Kconfig               |  15 +
>  platform/generic/andes/ae350.c               |  19 +-
>  platform/generic/andes/andes_hpm.c           | 405 +++++++++++++++++++
>  platform/generic/andes/andes_pmu.c           | 102 +++++
>  platform/generic/andes/objects.mk            |   2 +
>  platform/generic/include/andes/andes45.h     |  32 ++
>  platform/generic/include/andes/andes_hpm.h   |  81 ++++
>  platform/generic/include/andes/andes_pmu.h   |  33 ++
>  platform/generic/include/platform_override.h |   1 +
>  platform/generic/platform.c                  |  11 +-
>  platform/generic/renesas/rzfive/rzfive.c     |   5 +-
>  25 files changed, 897 insertions(+), 111 deletions(-)
>  create mode 100644 platform/generic/andes/andes_hpm.c
>  create mode 100644 platform/generic/andes/andes_pmu.c
>  create mode 100644 platform/generic/include/andes/andes_hpm.h
>  create mode 100644 platform/generic/include/andes/andes_pmu.h
>
> --
> 2.34.1
>


-- 
Regards,
Atish


From atishp at atishpatra.org  Wed Nov 22 16:24:35 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Wed, 22 Nov 2023 16:24:35 -0800
Subject: [PATCH v3 04/15] sbi: sbi_pmu: Add hw_counter_filter_mode() to
 pmu device
In-Reply-To: <20231122073617.379441-5-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-5-peterlin@andestech.com>
Message-ID: <CAOnJCUK4_H=MeHsUUCDscVD+905k6uV9C3bUUpwuA+kTdsFYUw@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:40?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add support for custom PMU extensions to set inhibit bits
> on custom CSRs by introducing the PMU device callback
> hw_counter_filter_mode(). This allows the perf tool to
> restrict event counting under a specified privileged
> mode by appending a modifier, e.g. perf record -e event:k
> to count events only happening in kernel mode.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - No change
> Changes v2 -> v3:
>   - Add pmu_dev->hw_counter_filter_mode() in pmu_fixed_ctr_update_inhibit_bits()
> ---
>  include/sbi/sbi_pmu.h |  6 ++++++
>  lib/sbi/sbi_pmu.c     | 20 ++++++++++++++------
>  2 files changed, 20 insertions(+), 6 deletions(-)
>
> diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
> index 16f6877..d63149c 100644
> --- a/include/sbi/sbi_pmu.h
> +++ b/include/sbi/sbi_pmu.h
> @@ -89,6 +89,12 @@ struct sbi_pmu_device {
>          * Custom function returning the machine-specific irq-bit.
>          */
>         int (*hw_counter_irq_bit)(void);
> +
> +       /**
> +        * Custom function to inhibit counting of events while in
> +        * specified mode.
> +        */
> +       void (*hw_counter_filter_mode)(unsigned long flags, int counter_index);
>  };
>
>  /** Get the PMU platform device */
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index 3cbd4ff..2f255de 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -599,7 +599,10 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,
>                 pmu_dev->hw_counter_disable_irq(ctr_idx);
>
>         /* Update the inhibit flags based on inhibit flags received from supervisor */
> -       pmu_update_inhibit_flags(flags, &mhpmevent_val);
> +       if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
> +               pmu_update_inhibit_flags(flags, &mhpmevent_val);
> +       if (pmu_dev && pmu_dev->hw_counter_filter_mode)
> +               pmu_dev->hw_counter_filter_mode(flags, ctr_idx);
>
>  #if __riscv_xlen == 32
>         csr_write_num(CSR_MHPMEVENT3 + ctr_idx - 3, mhpmevent_val & 0xFFFFFFFF);
> @@ -620,7 +623,8 @@ static int pmu_fixed_ctr_update_inhibit_bits(int fixed_ctr, unsigned long flags)
>  #if __riscv_xlen == 32
>         uint64_t cfgh_csr_no;
>  #endif
> -       if (!sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF))
> +       if (!sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF) &&
> +               !sbi_hart_has_extension(scratch, SBI_HART_EXT_XANDESPMU))

Instead of adding a check for vendor extension, we can just check for
platform specific hw_counter_filter_mode() availability.
I would prefer to avoid any vendor specific code if possible. Ideally,
all vendor specific details should be abstracted out via pmu-dev.

>                 return fixed_ctr;
>
>         switch (fixed_ctr) {
> @@ -641,13 +645,17 @@ static int pmu_fixed_ctr_update_inhibit_bits(int fixed_ctr, unsigned long flags)
>         }
>
>         cfg_val |= MHPMEVENT_MINH;
> -       pmu_update_inhibit_flags(flags, &cfg_val);
> +       if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF)) {
> +               pmu_update_inhibit_flags(flags, &cfg_val);
>  #if __riscv_xlen == 32
> -       csr_write_num(cfg_csr_no, cfg_val & 0xFFFFFFFF);
> -       csr_write_num(cfgh_csr_no, cfg_val >> BITS_PER_LONG);
> +               csr_write_num(cfg_csr_no, cfg_val & 0xFFFFFFFF);
> +               csr_write_num(cfgh_csr_no, cfg_val >> BITS_PER_LONG);
>  #else
> -       csr_write_num(cfg_csr_no, cfg_val);
> +               csr_write_num(cfg_csr_no, cfg_val);
>  #endif
> +       }
> +       if (pmu_dev && pmu_dev->hw_counter_filter_mode)
> +               pmu_dev->hw_counter_filter_mode(flags, fixed_ctr);
>         return fixed_ctr;
>  }
>
> --
> 2.34.1
>


--
Regards,
Atish


From atishp at atishpatra.org  Wed Nov 22 16:25:29 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Wed, 22 Nov 2023 16:25:29 -0800
Subject: [PATCH v3 06/15] platform: generic: Introduce pmu_init() platform override
In-Reply-To: <20231122073617.379441-7-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-7-peterlin@andestech.com>
Message-ID: <CAOnJCULyvYf3p-hK=4==ww+Y6aLaBA5me8E7AwreFdD9UQgyag@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:40?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add pmu_init() platform override which will be used to register
> PMU device and populate event mappings.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> Changes v2 -> v3:
>   - New patch
> ---
>  platform/generic/include/platform_override.h | 1 +
>  platform/generic/platform.c                  | 3 +++
>  2 files changed, 4 insertions(+)
>
> diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
> index bf4b112..f2a4327 100644
> --- a/platform/generic/include/platform_override.h
> +++ b/platform/generic/include/platform_override.h
> @@ -27,6 +27,7 @@ struct platform_override {
>         int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
>         int (*extensions_init)(const struct fdt_match *match,
>                                struct sbi_hart_features *hfeatures);
> +       int (*pmu_init)(const struct fdt_match *match);
>         void (*fw_init)(void *fdt, const struct fdt_match *match);
>         int (*vendor_ext_provider)(long funcid,
>                                    const struct sbi_trap_regs *regs,
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index fa400b9..54a913b 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -267,6 +267,9 @@ static int generic_pmu_init(void)
>  {
>         int rc;
>
> +       if (generic_plat && generic_plat->pmu_init)
> +               return generic_plat->pmu_init(generic_plat_match);
> +


Reviewed-by: Atish Patra <atishp at rivosinc.com>

>         rc = fdt_pmu_setup(fdt_get_address());
>         if (rc && rc != SBI_ENOENT)
>                 return rc;
> --
> 2.34.1
>


-- 
Regards,
Atish


From atishp at atishpatra.org  Wed Nov 22 16:27:02 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Wed, 22 Nov 2023 16:27:02 -0800
Subject: [PATCH v3 00/15] Add Andes PMU extension support
In-Reply-To: <CAOnJCU+qacoXRrs3MDTyskEnJWJ-vUds57m8rKDq7UedkOhqVA@mail.gmail.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <CAOnJCU+qacoXRrs3MDTyskEnJWJ-vUds57m8rKDq7UedkOhqVA@mail.gmail.com>
Message-ID: <CAOnJCULdLyFugMFuY4JLL2+M2D8J85iU_A91ZFBLxVbhzLGASg@mail.gmail.com>

On Wed, Nov 22, 2023 at 4:07?PM Atish Patra <atishp at atishpatra.org> wrote:
>
> On Tue, Nov 21, 2023 at 11:39?PM Yu Chien Peter Lin
> <peterlin at andestech.com> wrote:
> >
> > This patch series enables perf tool to utilize Andes PMU
> > extension via PMU SBI calls, provides PMU device callbacks
> > to achieve event sampling and mode filtering.
> >
> > This version mainly fixes single-core freeze when quitting
> > 'perf top', adds andes_pmu_setup() to provide default event
> > mappings when fdt_pmu_setup() can't find PMU node. And add
> > Xandespmu to hart extension list so that the inhibit bits
> > can be updated in a similar way to how Smcntrpmf does.
> >
> > The last patch provides a PMU node example used on AX45MP cores.
> >
>
> Do you have a publicly available spec that describes what XAndesPMU does ?

Never mind. I found it in the last patch. It would be good to include
it in the cover letter as well.
http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf

>
> > The OpenSBI and Linux patches can be found on Andes Technology GitHub
> > - https://github.com/andestech/opensbi/commits/andes-pmu-support-v3
> > - https://github.com/andestech/linux/commits/andes-pmu-support-v4
> >
> > Leo Yu-Chi Liang (1):
> >   lib: ipi: Adjust Andes PLICSW to single-bit-per-hart scheme
> >
> > Yu Chien Peter Lin (14):
> >   sbi: sbi_pmu: Improve sbi_pmu_init() error handling
> >   lib: sbi: Add Xandespmu in hart extensions
> >   sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
> >   platform: include: andes45: Add PMU related CSR defines
> >   platform: generic: Introduce pmu_init() platform override
> >   platform: andes: Add Andes custom PMU support
> >   platform: andes: Enable Andes PMU for AE350
> >   platform: rzfive: Enable Andes PMU for RZ/Five
> >   lib: utils: fdt_fixup: Allow preserving PMU properties
> >   platform: andes: Factor out is_andes() helper
> >   lib: utils: fdt_pmu: Make the fdt_pmu_evt_select table global variable
> >   lib: utils: fdt_pmu: Do not iterate over the fdt_pmu_evt_select table
> >   platform: andes: Add Andes default PMU mapping support
> >   docs: pmu: Add Andes PMU node example
> >
> >  docs/pmu_support.md                          |  82 ++++
> >  include/sbi/sbi_ecall_interface.h            |   5 +
> >  include/sbi/sbi_hart.h                       |   2 +
> >  include/sbi/sbi_pmu.h                        |   6 +
> >  include/sbi/sbi_scratch.h                    |   2 +
> >  include/sbi_utils/fdt/fdt_helper.h           |  17 +
> >  include/sbi_utils/fdt/fdt_pmu.h              |   6 +
> >  include/sbi_utils/ipi/andes_plicsw.h         |  23 +-
> >  lib/sbi/sbi_hart.c                           |   3 +
> >  lib/sbi/sbi_pmu.c                            |  25 +-
> >  lib/utils/fdt/fdt_fixup.c                    |   6 +-
> >  lib/utils/fdt/fdt_pmu.c                      |  17 +-
> >  lib/utils/ipi/andes_plicsw.c                 | 104 ++---
> >  platform/generic/Kconfig                     |   4 +
> >  platform/generic/andes/Kconfig               |  15 +
> >  platform/generic/andes/ae350.c               |  19 +-
> >  platform/generic/andes/andes_hpm.c           | 405 +++++++++++++++++++
> >  platform/generic/andes/andes_pmu.c           | 102 +++++
> >  platform/generic/andes/objects.mk            |   2 +
> >  platform/generic/include/andes/andes45.h     |  32 ++
> >  platform/generic/include/andes/andes_hpm.h   |  81 ++++
> >  platform/generic/include/andes/andes_pmu.h   |  33 ++
> >  platform/generic/include/platform_override.h |   1 +
> >  platform/generic/platform.c                  |  11 +-
> >  platform/generic/renesas/rzfive/rzfive.c     |   5 +-
> >  25 files changed, 897 insertions(+), 111 deletions(-)
> >  create mode 100644 platform/generic/andes/andes_hpm.c
> >  create mode 100644 platform/generic/andes/andes_pmu.c
> >  create mode 100644 platform/generic/include/andes/andes_hpm.h
> >  create mode 100644 platform/generic/include/andes/andes_pmu.h
> >
> > --
> > 2.34.1
> >
>
>
> --
> Regards,
> Atish



-- 
Regards,
Atish


From samuel.holland at sifive.com  Wed Nov 22 18:24:49 2023
From: samuel.holland at sifive.com (Samuel Holland)
Date: Wed, 22 Nov 2023 20:24:49 -0600
Subject: [PATCH v3 07/15] platform: andes: Add Andes custom PMU support
In-Reply-To: <20231122073617.379441-8-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
 <20231122073617.379441-8-peterlin@andestech.com>
Message-ID: <70a378e1-484c-4b77-942a-169a99ad00b5@sifive.com>

On 2023-11-22 1:36 AM, Yu Chien Peter Lin wrote:
> Before the ratification of Sscofpmf, the Andes PMU extension
> was designed to support the sampling and filtering with hardware
> performance counters (zihpm), it works with the current SBI PMU
> extension and Linux SBI PMU driver.
> 
> We implement 1) the PMU device callbacks that update the
> corresponding bits on custom CSRs, 2) extentions_init() to detect
> the hardware support of Andes PMU and initialize the per-hart
> PMU related CSR, and 3) pmu_init() to register PMU device and
> populate event mappings (only called by coldboot hart).
> 
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - Fix mode filtering in andes_hw_counter_filter_mode()
>   - Return early if pmu is not supported in andes_pmu_init() (suggested by Prabhakar)
>   - Don't grant write permissions via CSR_MCOUNTERWEN as not needed
> Changes v2 -> v3:
>   - Drop Anup's RB tag as we add andes_pmu_extensions_init() to initialize per-hart
>     extension on scratch and move andes_pmu_init() to new added platform override pmu_init()
> --- 
>  platform/generic/andes/Kconfig             |   8 ++
>  platform/generic/andes/andes_pmu.c         | 102 +++++++++++++++++++++
>  platform/generic/andes/objects.mk          |   1 +
>  platform/generic/include/andes/andes_pmu.h |  33 +++++++
>  4 files changed, 144 insertions(+)
>  create mode 100644 platform/generic/andes/andes_pmu.c
>  create mode 100644 platform/generic/include/andes/andes_pmu.h
> 
> diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
> index a91fb9c..5b2ed91 100644
> --- a/platform/generic/andes/Kconfig
> +++ b/platform/generic/andes/Kconfig
> @@ -7,3 +7,11 @@ config ANDES45_PMA
>  config ANDES_SBI
>  	bool "Andes SBI support"
>  	default n
> +
> +config ANDES_PMU
> +	bool "Andes PMU extension (xandespmu) support"
> +	default n
> +	help
> +	  Andes PMU extension supports the event counter overflow
> +	  interrupt and mode filtering, similar to the standard
> +	  Sscofpmf and Smcntrpmf.
> diff --git a/platform/generic/andes/andes_pmu.c b/platform/generic/andes/andes_pmu.c
> new file mode 100644
> index 0000000..72003fd
> --- /dev/null
> +++ b/platform/generic/andes/andes_pmu.c
> @@ -0,0 +1,102 @@
> +// SPDX-License-Identifier: BSD-2-Clause
> +/*
> + * andes_pmu.c - Andes PMU device callbacks and platform overrides
> + *
> + * Copyright (C) 2023 Andes Technology Corporation
> + */
> +
> +#include <andes/andes45.h>
> +#include <andes/andes_hpm.h>
> +#include <andes/andes_pmu.h>
> +#include <sbi/sbi_bitops.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_pmu.h>
> +
> +static void andes_hw_counter_enable_irq(uint32_t ctr_idx)
> +{
> +	unsigned long mip_val;
> +
> +	if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
> +		return;
> +
> +	mip_val = csr_read(CSR_MIP);
> +	if (!(mip_val & MIP_PMOVI))
> +		csr_clear(CSR_MCOUNTEROVF, BIT(ctr_idx));
> +
> +	csr_set(CSR_MCOUNTERINTEN, BIT(ctr_idx));
> +}
> +
> +static void andes_hw_counter_disable_irq(uint32_t ctr_idx)
> +{
> +	csr_clear(CSR_MCOUNTERINTEN, BIT(ctr_idx));
> +}
> +
> +static void andes_hw_counter_filter_mode(unsigned long flags, int ctr_idx)
> +{
> +	if (flags & SBI_PMU_CFG_FLAG_SET_UINH)
> +		csr_set(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
> +	else
> +		csr_clear(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
> +
> +	if (flags & SBI_PMU_CFG_FLAG_SET_SINH)
> +		csr_set(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
> +	else
> +		csr_clear(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
> +}
> +
> +static struct sbi_pmu_device andes_pmu = {
> +	.name = "andes_pmu",
> +	.hw_counter_enable_irq  = andes_hw_counter_enable_irq,
> +	.hw_counter_disable_irq = andes_hw_counter_disable_irq,
> +	/*
> +	 * We set delegation of supervisor local interrupts via
> +	 * 18th bit on mslideleg instead of mideleg, so leave
> +	 * hw_counter_irq_bit() callback unimplemented.
> +	 */
> +	.hw_counter_irq_bit     = NULL,
> +	.hw_counter_filter_mode = andes_hw_counter_filter_mode
> +};
> +
> +int andes_pmu_extensions_init(const struct fdt_match *match,
> +			      struct sbi_hart_features *hfeatures)
> +{
> +	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> +
> +	if (!has_andes_pmu())
> +		return 0;
> +
> +	/*
> +	 * Don't expect both Andes PMU and standard Sscofpmf/Smcntrpmf,
> +	 * are supported as they serve the same purpose.
> +	 */
> +	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF) ||
> +		sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF))
> +		return SBI_EINVAL;

If Sscofpmf is supported, don't you want to successfully do nothing?

> +	sbi_hart_update_extension(scratch, SBI_HART_EXT_XANDESPMU, true);
> +
> +	/* Inhibit all HPM counters in M-mode */
> +	csr_write(CSR_MCOUNTERMASK_M, 0xfffffffd);
> +	/* Delegate counter overflow interrupt to S-mode */
> +	csr_write(CSR_MSLIDELEG, MIP_PMOVI);
> +
> +	return 0;
> +}
> +
> +int andes_pmu_init(const struct fdt_match *match)
> +{
> +	int rc;
> +	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> +
> +	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_XANDESPMU))
> +		sbi_pmu_set_device(&andes_pmu);
> +
> +	rc = fdt_pmu_setup(fdt_get_address());

generic_pmu_init() already calls fdt_pmu_setup().

> +	/*
> +	 * Populate default mappings if device-tree doesn't
> +	 * provide a valid pmu node.
> +	 */
> +	if (rc == SBI_ENOENT)
> +		return andes_pmu_setup();

This function isn't added until patch 14, so this won't compile. Please make
sure each patch compiles individually.

Regards,
Samuel



From inochiama at outlook.com  Thu Nov 23 03:28:02 2023
From: inochiama at outlook.com (Inochi Amaoto)
Date: Thu, 23 Nov 2023 19:28:02 +0800
Subject: [PATCH] platform: recalculate heap size to support new tlb entry number
In-Reply-To: <CAJF2gTRuxfGu+VxCqsSEem9KU90vRs+R-QaATt8rKHdwZPj9gQ@mail.gmail.com>
References: <CAJF2gTRuxfGu+VxCqsSEem9KU90vRs+R-QaATt8rKHdwZPj9gQ@mail.gmail.com>
Message-ID: <IA1PR20MB4953078FCFE4D0B5735867B5BBB9A@IA1PR20MB4953.namprd20.prod.outlook.com>


>On Fri, Nov 17, 2023 at 8:10?PM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> Previous patch introduct a change that using hart count as the default
>> number of tlb entry in the fifo. This makes the default tlb fifo size
>> grow in square with the number of harts. So the default heap size is
>> not enough to allocate tlb fifo when the hart count is big.
>>
>> Use new formula for default heap size:
>> 0x40 * (__num_hart) ^ 2   -> handle the tlb fifo
>> 0x600 * (__num_hart)      -> remove previous static fifo allocation
>Yes, ^ 2 is not suitable for sg2042.
>

Is there any reason of unsuitability for sg2042. I think this is tight
lower bound: now we have a tlb fifo with __num_hart entries for each hart.
So I think ^ 2 is a must, otherwise a OOM will issue.

>>
>> Signed-off-by: Inochi Amaoto <inochiama at outlook.com>
>> Fixes: 52fd64b ("platform: Uses hart count as the default size of tlb info")
>> ---
>>  include/sbi/sbi_bitops.h   | 1 +
>>  include/sbi/sbi_heap.h     | 3 +++
>>  include/sbi/sbi_platform.h | 8 ++++++--
>>  lib/sbi/sbi_heap.c         | 2 --
>>  4 files changed, 10 insertions(+), 4 deletions(-)
>>
>> diff --git a/include/sbi/sbi_bitops.h b/include/sbi/sbi_bitops.h
>> index 48a2090..7d90334 100644
>> --- a/include/sbi/sbi_bitops.h
>> +++ b/include/sbi/sbi_bitops.h
>> @@ -26,6 +26,7 @@
>>  #define BIT_MASK(nr)           (1UL << ((nr) % BITS_PER_LONG))
>>  #define BIT_WORD(bit)          ((bit) / BITS_PER_LONG)
>>  #define BIT_WORD_OFFSET(bit)   ((bit) & (BITS_PER_LONG - 1))
>> +#define BIT_ALIGN(bit, align)  (((bit) + ((align) - 1)) & ~((align) - 1))
>>
>>  #define GENMASK(h, l) \
>>         (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
>> diff --git a/include/sbi/sbi_heap.h b/include/sbi/sbi_heap.h
>> index 88d176e..16755ec 100644
>> --- a/include/sbi/sbi_heap.h
>> +++ b/include/sbi/sbi_heap.h
>> @@ -12,6 +12,9 @@
>>
>>  #include <sbi/sbi_types.h>
>>
>> +/* Alignment of heap base address and size */
>> +#define HEAP_BASE_ALIGN                        1024
>> +
>>  struct sbi_scratch;
>>
>>  /** Allocate from heap area */
>> diff --git a/include/sbi/sbi_platform.h b/include/sbi/sbi_platform.h
>> index 58b9069..609f608 100644
>> --- a/include/sbi/sbi_platform.h
>> +++ b/include/sbi/sbi_platform.h
>> @@ -44,8 +44,10 @@
>>
>>  #ifndef __ASSEMBLER__
>>
>> +#include <sbi/sbi_bitops.h>
>>  #include <sbi/sbi_ecall_interface.h>
>>  #include <sbi/sbi_error.h>
>> +#include <sbi/sbi_heap.h>
>>  #include <sbi/sbi_scratch.h>
>>  #include <sbi/sbi_version.h>
>>
>> @@ -146,8 +148,10 @@ struct sbi_platform_operations {
>>  #define SBI_PLATFORM_DEFAULT_HART_STACK_SIZE   8192
>>
>>  /** Platform default heap size */
>> -#define SBI_PLATFORM_DEFAULT_HEAP_SIZE(__num_hart)     \
>> -                                       (0x8000 + 0x800 * (__num_hart))
>> +#define SBI_PLATFORM_DEFAULT_HEAP_SIZE(__num_hart)             \
>> +       BIT_ALIGN((0x8000 + 0x600 * (__num_hart) +              \
>> +                  0x40 * (__num_hart) * (__num_hart)),         \
>> +                 HEAP_BASE_ALIGN)
>>
>>  /** Representation of a platform */
>>  struct sbi_platform {
>> diff --git a/lib/sbi/sbi_heap.c b/lib/sbi/sbi_heap.c
>> index 698c377..bcd404b 100644
>> --- a/lib/sbi/sbi_heap.c
>> +++ b/lib/sbi/sbi_heap.c
>> @@ -14,8 +14,6 @@
>>  #include <sbi/sbi_scratch.h>
>>  #include <sbi/sbi_string.h>
>>
>> -/* Alignment of heap base address and size */
>> -#define HEAP_BASE_ALIGN                        1024
>>  /* Minimum size and alignment of heap allocations */
>>  #define HEAP_ALLOC_ALIGN               64
>>  #define HEAP_HOUSEKEEPING_FACTOR       16
>> --
>> 2.42.1
>>
>Maybe the number of 0x600 needs more discussion. Other:

I agree with this. I have no idea about the original value "0x800", so
I only remove the static allocation of previous tlb info for minimized
heap allocation.
Maybe there should be some comment to tell why it need this size.

>Reviewed-by: Guo Ren <guoren at kernel.org>
>
>--
>Best Regards
> Guo Ren
>
>


From atishp at atishpatra.org  Thu Nov 23 08:34:20 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Thu, 23 Nov 2023 08:34:20 -0800
Subject: [PATCH] lib: utils/irqchip: Avoid redundant writes to APLIC CLRIE register
In-Reply-To: <20231121150259.902907-1-apatel@ventanamicro.com>
References: <20231121150259.902907-1-apatel@ventanamicro.com>
Message-ID: <CAOnJCUJC5Rn=SsYAWq7feEw96a8dnncD0D9Mfw6NvFzmJR9Xpw@mail.gmail.com>

On Tue, Nov 21, 2023 at 7:03?AM Anup Patel <apatel at ventanamicro.com> wrote:
>
> Each APLIC CLRIE register allows disabling 32 interrupt sources at
> a time by writing -1 so no need to write CLRIE register separately
> for each interrupt source.
>
> Fixes: 99792653de29 ("lib: utils/irqchip: Add APLIC initialization
> library")
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> ---
>  lib/utils/irqchip/aplic.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/utils/irqchip/aplic.c b/lib/utils/irqchip/aplic.c
> index d7fd992..10236e7 100644
> --- a/lib/utils/irqchip/aplic.c
> +++ b/lib/utils/irqchip/aplic.c
> @@ -193,7 +193,7 @@ int aplic_cold_irqchip_init(struct aplic_data *aplic)
>         writel(0, (void *)(aplic->addr + APLIC_DOMAINCFG));
>
>         /* Disable all interrupts */
> -       for (i = 0; i <= aplic->num_source; i++)
> +       for (i = 0; i <= aplic->num_source; i += 32)
>                 writel(-1U, (void *)(aplic->addr + APLIC_CLRIE_BASE +
>                                      (i / 32) * sizeof(u32)));
>
> --
> 2.34.1
>


Reviewed-by: Atish Patra <atishp at rivosinc.com>
-- 
Regards,
Atish


From atishp at atishpatra.org  Thu Nov 23 08:53:11 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Thu, 23 Nov 2023 08:53:11 -0800
Subject: [PATCH v3 05/15] platform: include: andes45: Add PMU related CSR defines
In-Reply-To: <20231122073617.379441-6-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-6-peterlin@andestech.com>
Message-ID: <CAOnJCUKQ0aDd9rO3rf2PBXmK9+9WqOsHyydji5a5hbutUmZGDg@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:40?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> This patch adds CSR for Andes PMU extension.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes v1 -> v2:
>   - Rename andes_pmu() -> has_andes_pmu()
> Changes v2 -> v3:
>   - Include Anup's RB tag
> ---
>  platform/generic/include/andes/andes45.h | 26 ++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
>
> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index f570994..ce31617 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -12,6 +12,17 @@
>  #define CSR_MDCM_CFG           0xfc1
>  #define CSR_MMSC_CFG           0xfc2
>
> +/* Machine Trap Related Registers */
> +#define CSR_MSLIDELEG          0x7d5
> +
> +/* Counter Related Registers */
> +#define CSR_MCOUNTERWEN                0x7ce
> +#define CSR_MCOUNTERINTEN      0x7cf
> +#define CSR_MCOUNTERMASK_M     0x7d1
> +#define CSR_MCOUNTERMASK_S     0x7d2
> +#define CSR_MCOUNTERMASK_U     0x7d3
> +#define CSR_MCOUNTEROVF                0x7d4
> +
>  #define MICM_CFG_ISZ_OFFSET            6
>  #define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
>
> @@ -26,4 +37,19 @@
>  #define MCACHE_CTL_CCTL_SUEN_OFFSET    8
>  #define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
>
> +/* Performance monitor */
> +#define MMSC_CFG_PMNDS_MASK            (1 << 15)
> +#define MIP_PMOVI                      (1 << 18)
> +
> +#ifndef __ASSEMBLER__
> +
> +#define has_andes_pmu()                                        \
> +({                                                     \
> +       (((csr_read(CSR_MMSC_CFG) &                     \
> +          MMSC_CFG_PMNDS_MASK)                         \
> +         && misa_extension('S')) ? true : false);      \
> +})
> +
> +#endif /* __ASSEMBLER__ */
> +
>  #endif /* _RISCV_ANDES45_H */
> --
> 2.34.1
>


Reviewed-by: Atish Patra <atishp at rivosinc.com>
-- 
Regards,
Atish


From atishp at atishpatra.org  Thu Nov 23 08:55:37 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Thu, 23 Nov 2023 08:55:37 -0800
Subject: [PATCH v3 08/15] platform: andes: Enable Andes PMU for AE350
In-Reply-To: <20231122073617.379441-9-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-9-peterlin@andestech.com>
Message-ID: <CAOnJCUJdS8xENNc2Le3Xf3uqnRbHvA+vPbbQjq8ff+rxe1GY9w@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:40?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Enable Andes PMU extension support for AE350 platforms.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - Implement ae350_extensions_init()
> Changes v2 -> v3:
>   - Use the andes_pmu_extensions_init() and andes_pmu_init() provided by andes_pmu.c
>     We have no plan to introduce other extensions in near future so extensions_init()
>     bind with andes_pmu_extensions_init() directly.
> ---
>  platform/generic/Kconfig       | 1 +
>  platform/generic/andes/ae350.c | 3 +++
>  2 files changed, 4 insertions(+)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index a902cf3..89e5603 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -32,6 +32,7 @@ config PLATFORM_ALLWINNER_D1
>  config PLATFORM_ANDES_AE350
>         bool "Andes AE350 support"
>         select SYS_ATCSMU
> +       select ANDES_PMU
>         default n
>
>  config PLATFORM_RENESAS_RZFIVE
> diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
> index 80eca05..dc769b7 100644
> --- a/platform/generic/andes/ae350.c
> +++ b/platform/generic/andes/ae350.c
> @@ -8,6 +8,7 @@
>   */
>
>  #include <platform_override.h>
> +#include <andes/andes_pmu.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>  #include <sbi_utils/fdt/fdt_fixup.h>
>  #include <sbi_utils/sys/atcsmu.h>
> @@ -118,4 +119,6 @@ static const struct fdt_match andes_ae350_match[] = {
>  const struct platform_override andes_ae350 = {
>         .match_table = andes_ae350_match,
>         .final_init  = ae350_final_init,
> +       .extensions_init = andes_pmu_extensions_init,
> +       .pmu_init = andes_pmu_init,
>  };
> --
> 2.34.1
>


Reviewed-by: Atish Patra <atishp at rivosinc.com>

-- 
Regards,
Atish


From atishp at atishpatra.org  Thu Nov 23 08:56:19 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Thu, 23 Nov 2023 08:56:19 -0800
Subject: [PATCH v3 09/15] platform: rzfive: Enable Andes PMU for RZ/Five
In-Reply-To: <20231122073617.379441-10-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-10-peterlin@andestech.com>
Message-ID: <CAOnJCUJvFbqhvW=ro6SqN8VNf34+B_=yEsse-ONUHXqa4xz4OA@mail.gmail.com>

On Tue, Nov 21, 2023 at 11:40?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Enable Andes PMU extension support for RZ/Five.
> We also staticize renesas_rzfive_early_init() as
> it is not used outside of this unit.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - Implement renesas_rzfive_extensions_init()
>   - staticize renesas_rzfive_early_init()
> Changes v2 -> v3:
>   - Use the andes_pmu_extensions_init() and andes_pmu_init() provided by andes_pmu.c
>     We have no plan to introduce other extensions in near future so extensions_init()
>     bind with andes_pmu_extensions_init() directly.
> ---
>  platform/generic/Kconfig                 | 1 +
>  platform/generic/renesas/rzfive/rzfive.c | 5 ++++-
>  2 files changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index 89e5603..c9b6011 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -39,6 +39,7 @@ config PLATFORM_RENESAS_RZFIVE
>         bool "Renesas RZ/Five support"
>         select ANDES45_PMA
>         select ANDES_SBI
> +       select ANDES_PMU
>         default n
>
>  config PLATFORM_SIFIVE_FU540
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index a69797b..b5b5e65 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -5,6 +5,7 @@
>   */
>
>  #include <andes/andes45_pma.h>
> +#include <andes/andes_pmu.h>
>  #include <andes/andes_sbi.h>
>  #include <platform_override.h>
>  #include <sbi/sbi_domain.h>
> @@ -29,7 +30,7 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
>                                          array_size(renesas_rzfive_pma_regions));
>  }
>
> -int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
> +static int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>  {
>         /*
>          * Renesas RZ/Five RISC-V SoC has Instruction local memory and
> @@ -57,4 +58,6 @@ const struct platform_override renesas_rzfive = {
>         .early_init = renesas_rzfive_early_init,
>         .final_init = renesas_rzfive_final_init,
>         .vendor_ext_provider = andes_sbi_vendor_ext_provider,
> +       .extensions_init = andes_pmu_extensions_init,
> +       .pmu_init = andes_pmu_init,
>  };
> --
> 2.34.1
>


Reviewed-by: Atish Patra <atishp at rivosinc.com>
-- 
Regards,
Atish


From yongxuan.wang at sifive.com  Thu Nov 23 20:55:33 2023
From: yongxuan.wang at sifive.com (Yong-Xuan Wang)
Date: Fri, 24 Nov 2023 12:55:33 +0800
Subject: [PATCH v2 0/5] Add Svadu extension support
In-Reply-To: <CAAhSdy2ALrnjXMUoA0AondR1-pEPyLCafsZX=N=d2gN7WL3nGQ@mail.gmail.com>
References: <20231024101145.32000-1-yongxuan.wang@sifive.com> <CAAhSdy2ALrnjXMUoA0AondR1-pEPyLCafsZX=N=d2gN7WL3nGQ@mail.gmail.com>
Message-ID: <CAMWQL2j42qmbT0EHsd35yaThJB9r_w8v-ZVuiH+YOs97u2-35w@mail.gmail.com>

On Wed, Nov 15, 2023 at 12:46?AM Anup Patel <anup at brainfault.org> wrote:
>
> On Tue, Oct 24, 2023 at 3:42?PM Yong-Xuan Wang <yongxuan.wang at sifive.com> wrote:
> >
> > This series enables Svadu extension support by configuring the menvcfg
> > CSR and, if available, displays the Svadu extension in the boot log.
> >
> > Additionally, we've made some programming improvements in
> > lib/sbi/sbi_hart.c and lib/utils/fdt/fdt_helper.c.
> >
> > ---
> > v2:
> > - Rearrange the patches to do the code refactoring first before adding
> >   new features
> > - Suggested by Anup and Atish, detect extensions from DT instead of
> >   menvcfg CSR
> > - Enable access to some extensions through menvcfg CSR if they are
> >   present in the device tree.
> >
> > Yong-Xuan Wang (5):
> >   lib: sbi: Improve the code of privilege mode and extensions detection
> >   lib: sbi: Refactor the code for enable extensions in menvfg CSR
> >   lib: sbi: Using one array to define the name of extensions
> >   lib: sbi: Detect extensions from the ISA string in DT
> >   lib: sbi: Add support for Svadu extension
>
> For backward compatibility with existing OSes, it is better to have
> supervisor OS explicitly enable Svadu using the upcoming SBI
> FWFT extension instead of enabling it unconditionally whenever
> Svadu extension is available.
>
> Regards,
> Anup
>

Hi Anup,

Thank you! I'll remove the enablement of Svadu extension in the next version.

Regards,
Yong-Xuan

> >
> >  include/sbi/riscv_encoding.h |   6 +-
> >  include/sbi/sbi_hart.h       |  15 +++
> >  lib/sbi/sbi_hart.c           | 226 +++++++++++++----------------------
> >  lib/utils/fdt/fdt_helper.c   |   5 +-
> >  4 files changed, 104 insertions(+), 148 deletions(-)
> >
> > --
> > 2.17.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Thu Nov 23 23:19:56 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 24 Nov 2023 12:49:56 +0530
Subject: [PATCH] lib: utils/irqchip: Avoid redundant writes to APLIC CLRIE register
In-Reply-To: <20231121150259.902907-1-apatel@ventanamicro.com>
References: <20231121150259.902907-1-apatel@ventanamicro.com>
Message-ID: <CAAhSdy2BOKWRuPdjfiy-qS5qJAPpugSWm_gPLFoSKAngoacabg@mail.gmail.com>

On Tue, Nov 21, 2023 at 8:33?PM Anup Patel <apatel at ventanamicro.com> wrote:
>
> Each APLIC CLRIE register allows disabling 32 interrupt sources at
> a time by writing -1 so no need to write CLRIE register separately
> for each interrupt source.
>
> Fixes: 99792653de29 ("lib: utils/irqchip: Add APLIC initialization
> library")
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>

Applied this patch to the riscv/opensbi repo.

Regards,
Anup

> ---
>  lib/utils/irqchip/aplic.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/utils/irqchip/aplic.c b/lib/utils/irqchip/aplic.c
> index d7fd992..10236e7 100644
> --- a/lib/utils/irqchip/aplic.c
> +++ b/lib/utils/irqchip/aplic.c
> @@ -193,7 +193,7 @@ int aplic_cold_irqchip_init(struct aplic_data *aplic)
>         writel(0, (void *)(aplic->addr + APLIC_DOMAINCFG));
>
>         /* Disable all interrupts */
> -       for (i = 0; i <= aplic->num_source; i++)
> +       for (i = 0; i <= aplic->num_source; i += 32)
>                 writel(-1U, (void *)(aplic->addr + APLIC_CLRIE_BASE +
>                                      (i / 32) * sizeof(u32)));
>
> --
> 2.34.1
>


From karthik.bk at incoresemi.com  Fri Nov 24 01:36:07 2023
From: karthik.bk at incoresemi.com (Karthik B K)
Date: Fri, 24 Nov 2023 09:36:07 +0000
Subject: [QUESTION] Usage of negative number notations on unsigned numbers.
Message-ID: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>

Hi. The pmp_get function in lib/sbi/riscv_asm.c uses a negative number notation on an unsigned integer.

pmpaddr = -1UL;

Just curious why it is this way. I believe that the intention here is to represent the largest unsigned long number.
If so, isn't it safer to use (~0UL) ? This could be a very silent bug. Please correct me if I'm wrong.

Best,

karthik.

From schwab at linux-m68k.org  Fri Nov 24 03:08:49 2023
From: schwab at linux-m68k.org (Andreas Schwab)
Date: Fri, 24 Nov 2023 12:08:49 +0100
Subject: [QUESTION] Usage of negative number notations on unsigned numbers.
In-Reply-To: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
	(Karthik B. K.'s message of "Fri, 24 Nov 2023 09:36:07 +0000")
References: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
Message-ID: <878r6n76im.fsf@igel.home>

On Nov 24 2023, Karthik B K wrote:

> If so, isn't it safer to use (~0UL) ? This could be a very silent bug. Please correct me if I'm wrong.

The safest way is to write -1 without any suffix.  This is guaranteed to
evaluate to the maximum value when converted to any unsigned type.

-- 
Andreas Schwab, schwab at linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."


From prabhakar.csengg at gmail.com  Fri Nov 24 06:43:10 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:43:10 +0000
Subject: [PATCH v3 01/15] lib: ipi: Adjust Andes PLICSW to
 single-bit-per-hart scheme
In-Reply-To: <20231122073617.379441-2-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-2-peterlin@andestech.com>
Message-ID: <CA+V-a8tcz8Lkb7iefKPnUPshfmzFFQ-Due2Tid3ORNw8jSNHnQ@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:40?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> From: Leo Yu-Chi Liang <ycliang at andestech.com>
>
> The old scheme doesn't allow sending hart0 self-IPI as the
> corresponding bit on pending register is hardwired to 0, this
> could lead to unhandle IPIs on SMP systems, esp. on single-core.
>
> Furthermore, the limitation of old scheme is 8-core, instead of
> reserving source hart information, we assign bit (x + 1) as the
> enable and pending bit of hartx, this also expands the bootable
> hart number.
>
> The following diagram shows the enable bits of the new scheme
> on 32-core Andes platform.
>
>    Pending regs: 0x1000  x---0---0---0---0------0---0
> Pending hart ID:             0   1   2   3 ... 30  31
>    Interrupt ID:         0   1   2   3   4 ... 31  32
>                          |   |   |   |   |      |   |
>     Enable regs: 0x2000  x---1---0---0---0-...--0---0---> hart0
>                          |   |   |   |   |      |   |
>                  0x2080  x---0---1---0---0-...--0---0---> hart1
>                          |   |   |   |   |      |   |
>                  0x2100  x---0---0---1---0-...--0---0---> hart2
>                          |   |   |   |   |      |   |
>                  0x2180  x---0---0---0---1-...--0---0---> hart3
>                          .   .   .   .   .      .   .
>                          .   .   .   .   .      .   .
>                          .   .   .   .   .      .   .
>                  0x2f00  x---0---0---0---0-...--1---0---> hart30
>                          |   |   |   |   |      |   |
>                  0x2f80  x---0---0---0---0-...--0---1---> hart31
>                          <-------- word 0 -------><--- word 1 --->
>
> To send IPI to hart0, for example, another hart (including hart0
> itself) will set bit 1 of first word on the pending register.
>
> We also fix indentation in andes_plicsw.h along with this patch.
>
> Fixes: ce7c490719ed ("lib: utils/ipi: Add Andes fdt ipi driver support")
> Signed-off-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Randolph <randolph at andestech.com>
> Reported-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> Link: https://lists.infradead.org/pipermail/opensbi/2023-October/005665.html
> ---
> Changes v2 -> v3:
>   - New patch
> ---
>  include/sbi_utils/ipi/andes_plicsw.h |  23 +++---
>  lib/utils/ipi/andes_plicsw.c         | 104 ++++++++++-----------------
>  2 files changed, 46 insertions(+), 81 deletions(-)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/include/sbi_utils/ipi/andes_plicsw.h b/include/sbi_utils/ipi/andes_plicsw.h
> index e93cda0..0d18444 100644
> --- a/include/sbi_utils/ipi/andes_plicsw.h
> +++ b/include/sbi_utils/ipi/andes_plicsw.h
> @@ -13,30 +13,23 @@
>  #ifndef _IPI_ANDES_PLICSW_H_
>  #define _IPI_ANDES_PLICSW_H_
>
> -#define PLICSW_PRIORITY_BASE 0x4
> +#define PLICSW_PRIORITY_BASE   0x4
>
> -#define PLICSW_PENDING_BASE 0x1000
> -#define PLICSW_PENDING_STRIDE 0x8
> +#define PLICSW_PENDING_BASE    0x1000
>
> -#define PLICSW_ENABLE_BASE 0x2000
> -#define PLICSW_ENABLE_STRIDE 0x80
> +#define PLICSW_ENABLE_BASE     0x2000
> +#define PLICSW_ENABLE_STRIDE   0x80
>
> -#define PLICSW_CONTEXT_BASE 0x200000
> -#define PLICSW_CONTEXT_STRIDE 0x1000
> -#define PLICSW_CONTEXT_CLAIM 0x4
> +#define PLICSW_CONTEXT_BASE    0x200000
> +#define PLICSW_CONTEXT_STRIDE  0x1000
> +#define PLICSW_CONTEXT_CLAIM   0x4
>
> -#define PLICSW_HART_MASK 0x01010101
> -
> -#define PLICSW_HART_MAX_NR 8
> -
> -#define PLICSW_REGION_ALIGN 0x1000
> +#define PLICSW_REGION_ALIGN    0x1000
>
>  struct plicsw_data {
>         unsigned long addr;
>         unsigned long size;
>         uint32_t hart_count;
> -       /* hart id to source id table */
> -       uint32_t source_id[PLICSW_HART_MAX_NR];
>  };
>
>  int plicsw_warm_ipi_init(void);
> diff --git a/lib/utils/ipi/andes_plicsw.c b/lib/utils/ipi/andes_plicsw.c
> index 5693efb..413ac20 100644
> --- a/lib/utils/ipi/andes_plicsw.c
> +++ b/lib/utils/ipi/andes_plicsw.c
> @@ -18,77 +18,45 @@
>
>  struct plicsw_data plicsw;
>
> -static inline void plicsw_claim(void)
> +static void plicsw_ipi_send(u32 hart_index)
>  {
> -       u32 hartid = current_hartid();
> +       ulong pending_reg;
> +       u32 interrupt_id, word_index, pending_bit;
> +       u32 target_hart = sbi_hartindex_to_hartid(hart_index);
>
> -       if (plicsw.hart_count <= hartid)
> +       if (plicsw.hart_count <= target_hart)
>                 ebreak();
>
> -       plicsw.source_id[hartid] =
> -               readl((void *)plicsw.addr + PLICSW_CONTEXT_BASE +
> -                     PLICSW_CONTEXT_CLAIM + PLICSW_CONTEXT_STRIDE * hartid);
> -}
> -
> -static inline void plicsw_complete(void)
> -{
> -       u32 hartid = current_hartid();
> -       u32 source = plicsw.source_id[hartid];
> -
> -       writel(source, (void *)plicsw.addr + PLICSW_CONTEXT_BASE +
> -                              PLICSW_CONTEXT_CLAIM +
> -                              PLICSW_CONTEXT_STRIDE * hartid);
> -}
> -
> -static inline void plic_sw_pending(u32 target_hart)
> -{
>         /*
> -        * The pending array registers are w1s type.
> -        * IPI pending array mapping as following:
> -        *
> -        * Pending array start address: base + 0x1000
> -        * ---------------------------------
> -        * | hart3 | hart2 | hart1 | hart0 |
> -        * ---------------------------------
> -        * Each hartX can send IPI to another hart by setting the
> -        * bitY to its own region (see the below).
> -        *
> -        * In each hartX region:
> -        * <---------- PICSW_PENDING_STRIDE -------->
> -        * | bit7 | ... | bit3 | bit2 | bit1 | bit0 |
> -        * ------------------------------------------
> -        * The bitY of hartX region indicates that hartX sends an
> -        * IPI to hartY.
> +        * We assign a single bit for each hart.
> +        * Bit 0 is hardwired to 0, thus unavailable.
> +        * Bit(X+1) indicates that IPI is sent to hartX.
>          */
> -       u32 hartid          = current_hartid();
> -       u32 word_index      = hartid / 4;
> -       u32 per_hart_offset = PLICSW_PENDING_STRIDE * hartid;
> -       u32 val             = 1 << target_hart << per_hart_offset;
> +       interrupt_id = target_hart + 1;
> +       word_index   = interrupt_id / 32;
> +       pending_bit  = interrupt_id % 32;
> +       pending_reg  = plicsw.addr + PLICSW_PENDING_BASE + word_index * 4;
>
> -       writel(val, (void *)plicsw.addr + PLICSW_PENDING_BASE + word_index * 4);
> +       /* Set target hart's mip.MSIP */
> +       writel_relaxed(BIT(pending_bit), (void *)pending_reg);
>  }
>
> -static void plicsw_ipi_send(u32 hart_index)
> +static void plicsw_ipi_clear(u32 hart_index)
>  {
>         u32 target_hart = sbi_hartindex_to_hartid(hart_index);
> +       ulong reg = plicsw.addr + PLICSW_CONTEXT_BASE + PLICSW_CONTEXT_CLAIM +
> +                   PLICSW_CONTEXT_STRIDE * target_hart;
>
>         if (plicsw.hart_count <= target_hart)
>                 ebreak();
>
> -       /* Set PLICSW IPI */
> -       plic_sw_pending(target_hart);
> -}
> +       /* Claim */
> +       u32 source = readl((void *)reg);
>
> -static void plicsw_ipi_clear(u32 hart_index)
> -{
> -       u32 target_hart = sbi_hartindex_to_hartid(hart_index);
> -
> -       if (plicsw.hart_count <= target_hart)
> -               ebreak();
> +       /* A successful claim will clear mip.MSIP */
>
> -       /* Clear PLICSW IPI */
> -       plicsw_claim();
> -       plicsw_complete();
> +       /* Complete */
> +       writel(source, (void *)reg);
>  }
>
>  static struct sbi_ipi_device plicsw_ipi = {
> @@ -110,22 +78,26 @@ int plicsw_warm_ipi_init(void)
>  int plicsw_cold_ipi_init(struct plicsw_data *plicsw)
>  {
>         int rc;
> +       u32 interrupt_id, word_index, enable_bit;
> +       ulong enable_reg, priority_reg;
>
>         /* Setup source priority */
> -       uint32_t *priority = (void *)plicsw->addr + PLICSW_PRIORITY_BASE;
> -
> -       for (int i = 0; i < plicsw->hart_count; i++)
> -               writel(1, &priority[i]);
> -
> -       /* Setup target enable */
> -       uint32_t enable_mask = PLICSW_HART_MASK;
> +       for (int i = 0; i < plicsw->hart_count; i++) {
> +               priority_reg = plicsw->addr + PLICSW_PRIORITY_BASE + i * 4;
> +               writel(1, (void *)priority_reg);
> +       }
>
> +       /*
> +        * Setup enable for each hart, skip non-existent interrupt ID 0
> +        * which is hardwired to 0.
> +        */
>         for (int i = 0; i < plicsw->hart_count; i++) {
> -               uint32_t *enable = (void *)plicsw->addr + PLICSW_ENABLE_BASE +
> -                                  PLICSW_ENABLE_STRIDE * i;
> -               writel(enable_mask, enable);
> -               writel(enable_mask, enable + 1);
> -               enable_mask <<= 1;
> +               interrupt_id = i + 1;
> +               word_index   = interrupt_id / 32;
> +               enable_bit   = interrupt_id % 32;
> +               enable_reg   = plicsw->addr + PLICSW_ENABLE_BASE +
> +                              PLICSW_ENABLE_STRIDE * i + 4 * word_index;
> +               writel(BIT(enable_bit), (void *)enable_reg);
>         }
>
>         /* Add PLICSW region to the root domain */
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:45:18 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:45:18 +0000
Subject: [PATCH v3 02/15] sbi: sbi_pmu: Improve sbi_pmu_init() error handling
In-Reply-To: <20231122073617.379441-3-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-3-peterlin@andestech.com>
Message-ID: <CA+V-a8vOzk6Lk-y+F2uLbObg4e=wp579dfVpNLRuS+FYdzGTJg@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:40?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> This patch makes the following changes:
>
> - As sbi_platform_pmu_init() returns a negative error code on
>   failure, let sbi_pmu_init() to hang by propagating the error
>   code.
>
> - In order to distinguish the SBI_EFAIL error returned by
>   sbi_pmu_add_*_counter_map(), return SBI_ENOENT to indicate
>   that fdt_pmu_setup() failed to locate "riscv,pmu" node, and
>   generic_pmu_init() ignores such case.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes v1 -> v2:
>   - New patch
> Changes v2 -> v3:
>   - Include Anup's RB tag
> ---
>  lib/sbi/sbi_pmu.c           | 5 ++++-
>  lib/utils/fdt/fdt_pmu.c     | 2 +-
>  platform/generic/platform.c | 8 +++++++-
>  3 files changed, 12 insertions(+), 3 deletions(-)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index f4c8fc4..3cbd4ff 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -957,6 +957,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
>         int hpm_count = sbi_fls(sbi_hart_mhpm_mask(scratch));
>         struct sbi_pmu_hart_state *phs;
>         const struct sbi_platform *plat;
> +       int rc;
>
>         if (cold_boot) {
>                 hw_event_map = sbi_calloc(sizeof(*hw_event_map),
> @@ -972,7 +973,9 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
>
>                 plat = sbi_platform_ptr(scratch);
>                 /* Initialize hw pmu events */
> -               sbi_platform_pmu_init(plat);
> +               rc = sbi_platform_pmu_init(plat);
> +               if (rc)
> +                       return rc;
>
>                 /* mcycle & minstret is available always */
>                 if (!hpm_count)
> diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
> index 83301bb..a8d7648 100644
> --- a/lib/utils/fdt/fdt_pmu.c
> +++ b/lib/utils/fdt/fdt_pmu.c
> @@ -74,7 +74,7 @@ int fdt_pmu_setup(void *fdt)
>
>         pmu_offset = fdt_node_offset_by_compatible(fdt, -1, "riscv,pmu");
>         if (pmu_offset < 0)
> -               return SBI_EFAIL;
> +               return SBI_ENOENT;
>
>         event_ctr_map = fdt_getprop(fdt, pmu_offset,
>                                     "riscv,event-to-mhpmcounters", &len);
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 85acecd..fa400b9 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -265,7 +265,13 @@ static u32 generic_tlb_num_entries(void)
>
>  static int generic_pmu_init(void)
>  {
> -       return fdt_pmu_setup(fdt_get_address());
> +       int rc;
> +
> +       rc = fdt_pmu_setup(fdt_get_address());
> +       if (rc && rc != SBI_ENOENT)
> +               return rc;
> +
> +       return 0;
>  }
>
>  static uint64_t generic_pmu_xlate_to_mhpmevent(uint32_t event_idx,
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:45:47 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:45:47 +0000
Subject: [PATCH v3 03/15] lib: sbi: Add Xandespmu in hart extensions
In-Reply-To: <20231122073617.379441-4-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-4-peterlin@andestech.com>
Message-ID: <CA+V-a8sq-ocEJZXm62NtfqQEgXUSGE=gq+kATX1M3obiL9A8+A@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:41?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> When SBI PMU updates inhibit bits on mcyclecfg and
> minstretcfg, it will check if Smcntrpmf extension
> is supported, we need to check Xandespmu similarly
> in the pmu_fixed_ctr_update_inhibit_bits() so add
> the custom extension to hart extension list.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> Changes v2 -> v3:
>   - New patch
> ---
>  include/sbi/sbi_hart.h | 2 ++
>  lib/sbi/sbi_hart.c     | 3 +++
>  2 files changed, 5 insertions(+)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
> index 6ee49ff..811e5f5 100644
> --- a/include/sbi/sbi_hart.h
> +++ b/include/sbi/sbi_hart.h
> @@ -45,6 +45,8 @@ enum sbi_hart_extensions {
>         SBI_HART_EXT_ZKR,
>         /** Hart has Smcntrpmf extension */
>         SBI_HART_EXT_SMCNTRPMF,
> +       /** Hart has Xandespmu extension */
> +       SBI_HART_EXT_XANDESPMU,
>
>         /** Maximum index of Hart extension */
>         SBI_HART_EXT_MAX,
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 77eef49..d372af1 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -693,6 +693,9 @@ static inline char *sbi_hart_extension_id2string(int ext)
>         case SBI_HART_EXT_SMCNTRPMF:
>                 estr = "smcntrpmf";
>                 break;
> +       case SBI_HART_EXT_XANDESPMU:
> +               estr = "xandespmu";
> +               break;
>         default:
>                 break;
>         }
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:47:47 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:47:47 +0000
Subject: [PATCH v3 05/15] platform: include: andes45: Add PMU related CSR defines
In-Reply-To: <20231122073617.379441-6-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-6-peterlin@andestech.com>
Message-ID: <CA+V-a8tEJEy-+O=ksUFGOQV6sdKsSmNe3fDf5Jx8_8W2XCRO2g@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:40?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> This patch adds CSR for Andes PMU extension.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes v1 -> v2:
>   - Rename andes_pmu() -> has_andes_pmu()
> Changes v2 -> v3:
>   - Include Anup's RB tag
> ---
>  platform/generic/include/andes/andes45.h | 26 ++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index f570994..ce31617 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -12,6 +12,17 @@
>  #define CSR_MDCM_CFG           0xfc1
>  #define CSR_MMSC_CFG           0xfc2
>
> +/* Machine Trap Related Registers */
> +#define CSR_MSLIDELEG          0x7d5
> +
> +/* Counter Related Registers */
> +#define CSR_MCOUNTERWEN                0x7ce
> +#define CSR_MCOUNTERINTEN      0x7cf
> +#define CSR_MCOUNTERMASK_M     0x7d1
> +#define CSR_MCOUNTERMASK_S     0x7d2
> +#define CSR_MCOUNTERMASK_U     0x7d3
> +#define CSR_MCOUNTEROVF                0x7d4
> +
>  #define MICM_CFG_ISZ_OFFSET            6
>  #define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
>
> @@ -26,4 +37,19 @@
>  #define MCACHE_CTL_CCTL_SUEN_OFFSET    8
>  #define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
>
> +/* Performance monitor */
> +#define MMSC_CFG_PMNDS_MASK            (1 << 15)
> +#define MIP_PMOVI                      (1 << 18)
> +
> +#ifndef __ASSEMBLER__
> +
> +#define has_andes_pmu()                                        \
> +({                                                     \
> +       (((csr_read(CSR_MMSC_CFG) &                     \
> +          MMSC_CFG_PMNDS_MASK)                         \
> +         && misa_extension('S')) ? true : false);      \
> +})
> +
> +#endif /* __ASSEMBLER__ */
> +
>  #endif /* _RISCV_ANDES45_H */
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:49:11 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:49:11 +0000
Subject: [PATCH v3 06/15] platform: generic: Introduce pmu_init() platform override
In-Reply-To: <20231122073617.379441-7-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-7-peterlin@andestech.com>
Message-ID: <CA+V-a8uFMbPYYQ5jrjQ3VuSMZ0d3i9cacPOYMZXdOew0bBymaQ@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:41?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add pmu_init() platform override which will be used to register
> PMU device and populate event mappings.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> Changes v2 -> v3:
>   - New patch
> ---
>  platform/generic/include/platform_override.h | 1 +
>  platform/generic/platform.c                  | 3 +++
>  2 files changed, 4 insertions(+)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
> index bf4b112..f2a4327 100644
> --- a/platform/generic/include/platform_override.h
> +++ b/platform/generic/include/platform_override.h
> @@ -27,6 +27,7 @@ struct platform_override {
>         int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
>         int (*extensions_init)(const struct fdt_match *match,
>                                struct sbi_hart_features *hfeatures);
> +       int (*pmu_init)(const struct fdt_match *match);
>         void (*fw_init)(void *fdt, const struct fdt_match *match);
>         int (*vendor_ext_provider)(long funcid,
>                                    const struct sbi_trap_regs *regs,
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index fa400b9..54a913b 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -267,6 +267,9 @@ static int generic_pmu_init(void)
>  {
>         int rc;
>
> +       if (generic_plat && generic_plat->pmu_init)
> +               return generic_plat->pmu_init(generic_plat_match);
> +
>         rc = fdt_pmu_setup(fdt_get_address());
>         if (rc && rc != SBI_ENOENT)
>                 return rc;
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:52:38 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:52:38 +0000
Subject: [PATCH v3 08/15] platform: andes: Enable Andes PMU for AE350
In-Reply-To: <20231122073617.379441-9-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-9-peterlin@andestech.com>
Message-ID: <CA+V-a8vJdbmCfGckwM41qkx-8BVun7PKzcirtj83N_XM5CEoRw@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:41?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Enable Andes PMU extension support for AE350 platforms.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - Implement ae350_extensions_init()
> Changes v2 -> v3:
>   - Use the andes_pmu_extensions_init() and andes_pmu_init() provided by andes_pmu.c
>     We have no plan to introduce other extensions in near future so extensions_init()
>     bind with andes_pmu_extensions_init() directly.
> ---
>  platform/generic/Kconfig       | 1 +
>  platform/generic/andes/ae350.c | 3 +++
>  2 files changed, 4 insertions(+)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index a902cf3..89e5603 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -32,6 +32,7 @@ config PLATFORM_ALLWINNER_D1
>  config PLATFORM_ANDES_AE350
>         bool "Andes AE350 support"
>         select SYS_ATCSMU
> +       select ANDES_PMU
>         default n
>
>  config PLATFORM_RENESAS_RZFIVE
> diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
> index 80eca05..dc769b7 100644
> --- a/platform/generic/andes/ae350.c
> +++ b/platform/generic/andes/ae350.c
> @@ -8,6 +8,7 @@
>   */
>
>  #include <platform_override.h>
> +#include <andes/andes_pmu.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>  #include <sbi_utils/fdt/fdt_fixup.h>
>  #include <sbi_utils/sys/atcsmu.h>
> @@ -118,4 +119,6 @@ static const struct fdt_match andes_ae350_match[] = {
>  const struct platform_override andes_ae350 = {
>         .match_table = andes_ae350_match,
>         .final_init  = ae350_final_init,
> +       .extensions_init = andes_pmu_extensions_init,
> +       .pmu_init = andes_pmu_init,
>  };
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:53:23 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:53:23 +0000
Subject: [PATCH v3 09/15] platform: rzfive: Enable Andes PMU for RZ/Five
In-Reply-To: <20231122073617.379441-10-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-10-peterlin@andestech.com>
Message-ID: <CA+V-a8voiqpa9gkqezxVFYkMv+0MeJczqBbNQn5O+1OyJmd5CQ@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:41?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Enable Andes PMU extension support for RZ/Five.
> We also staticize renesas_rzfive_early_init() as
> it is not used outside of this unit.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> ---
> Changes v1 -> v2:
>   - Implement renesas_rzfive_extensions_init()
>   - staticize renesas_rzfive_early_init()
> Changes v2 -> v3:
>   - Use the andes_pmu_extensions_init() and andes_pmu_init() provided by andes_pmu.c
>     We have no plan to introduce other extensions in near future so extensions_init()
>     bind with andes_pmu_extensions_init() directly.
> ---
>  platform/generic/Kconfig                 | 1 +
>  platform/generic/renesas/rzfive/rzfive.c | 5 ++++-
>  2 files changed, 5 insertions(+), 1 deletion(-)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index 89e5603..c9b6011 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -39,6 +39,7 @@ config PLATFORM_RENESAS_RZFIVE
>         bool "Renesas RZ/Five support"
>         select ANDES45_PMA
>         select ANDES_SBI
> +       select ANDES_PMU
>         default n
>
>  config PLATFORM_SIFIVE_FU540
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index a69797b..b5b5e65 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -5,6 +5,7 @@
>   */
>
>  #include <andes/andes45_pma.h>
> +#include <andes/andes_pmu.h>
>  #include <andes/andes_sbi.h>
>  #include <platform_override.h>
>  #include <sbi/sbi_domain.h>
> @@ -29,7 +30,7 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
>                                          array_size(renesas_rzfive_pma_regions));
>  }
>
> -int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
> +static int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>  {
>         /*
>          * Renesas RZ/Five RISC-V SoC has Instruction local memory and
> @@ -57,4 +58,6 @@ const struct platform_override renesas_rzfive = {
>         .early_init = renesas_rzfive_early_init,
>         .final_init = renesas_rzfive_final_init,
>         .vendor_ext_provider = andes_sbi_vendor_ext_provider,
> +       .extensions_init = andes_pmu_extensions_init,
> +       .pmu_init = andes_pmu_init,
>  };
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:54:13 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:54:13 +0000
Subject: [PATCH v3 10/15] lib: utils: fdt_fixup: Allow preserving PMU properties
In-Reply-To: <20231122073617.379441-11-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-11-peterlin@andestech.com>
Message-ID: <CA+V-a8unVQpOFRYdEMVjewWPaa_HU7MK7b-MOLB150CNfkNSBQ@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:41?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add a scratch option to control PMU fixup, so the next
> stage software can dump the PMU node including event
> mapping information for debugging purposes.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes v1 -> v2:
>   - New patch
> Changes v2 -> v3:
>   - Rename to SBI_SCRATCH_PRESERVE_PMU_NODE (suggested by Anup)
>   - Include Anup's RB tag
> ---
>  include/sbi/sbi_scratch.h | 2 ++
>  lib/utils/fdt/fdt_fixup.c | 6 +++++-
>  2 files changed, 7 insertions(+), 1 deletion(-)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/include/sbi/sbi_scratch.h b/include/sbi/sbi_scratch.h
> index e6a33ba..0f67cde 100644
> --- a/include/sbi/sbi_scratch.h
> +++ b/include/sbi/sbi_scratch.h
> @@ -151,6 +151,8 @@ enum sbi_scratch_options {
>         SBI_SCRATCH_NO_BOOT_PRINTS = (1 << 0),
>         /** Enable runtime debug prints */
>         SBI_SCRATCH_DEBUG_PRINTS = (1 << 1),
> +       /** Preserve PMU node properties */
> +       SBI_SCRATCH_PRESERVE_PMU_NODE = (1 << 2),
>  };
>
>  /** Get pointer to sbi_scratch for current HART */
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index e213ded..cf20edf 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -387,6 +387,8 @@ int fdt_reserved_memory_fixup(void *fdt)
>
>  void fdt_fixups(void *fdt)
>  {
> +       struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> +
>         fdt_aplic_fixup(fdt);
>
>         fdt_imsic_fixup(fdt);
> @@ -394,5 +396,7 @@ void fdt_fixups(void *fdt)
>         fdt_plic_fixup(fdt);
>
>         fdt_reserved_memory_fixup(fdt);
> -       fdt_pmu_fixup(fdt);
> +
> +       if (!(scratch->options & SBI_SCRATCH_PRESERVE_PMU_NODE))
> +               fdt_pmu_fixup(fdt);
>  }
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:55:41 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:55:41 +0000
Subject: [PATCH v3 14/15] platform: andes: Add Andes default PMU mapping support
In-Reply-To: <20231122073617.379441-15-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-15-peterlin@andestech.com>
Message-ID: <CA+V-a8tG3a8=xy0gLjWLMApgFoT_FCv5HifCHymVACtYzjyLWw@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:42?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Provide the andes_pmu_setup() to populate default mappings in
> hw_event_map[] and fdt_pmu_evt_select[] when fdt_pmu_setup()
> cannot find the PMU node.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> Changes v2 -> v3:
>   - New patch
>   - Drop fdt_add_pmu_mappings() (suggested by Anup [1])
>
> [1] https://patchwork.ozlabs.org/project/opensbi/patch/20231019113713.3508153-11-peterlin at andestech.com/
> ---
>  include/sbi/sbi_ecall_interface.h          |   5 +
>  platform/generic/Kconfig                   |   2 +
>  platform/generic/andes/Kconfig             |   7 +
>  platform/generic/andes/andes_hpm.c         | 405 +++++++++++++++++++++
>  platform/generic/andes/objects.mk          |   1 +
>  platform/generic/include/andes/andes_hpm.h |  81 +++++
>  6 files changed, 501 insertions(+)
>  create mode 100644 platform/generic/andes/andes_hpm.c
>  create mode 100644 platform/generic/include/andes/andes_hpm.h
>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
> index 1fe469e..89187e7 100644
> --- a/include/sbi/sbi_ecall_interface.h
> +++ b/include/sbi/sbi_ecall_interface.h
> @@ -155,6 +155,11 @@ enum sbi_pmu_hw_cache_op_result_id {
>         SBI_PMU_HW_CACHE_RESULT_MAX,
>  };
>
> +#define SBI_PMU_HW_CACHE_EVENT_IDX(id, op, res) \
> +       (SBI_PMU_EVENT_TYPE_HW_CACHE << SBI_PMU_EVENT_IDX_TYPE_OFFSET | \
> +       SBI_PMU_HW_CACHE_##id << 3 | SBI_PMU_HW_CACHE_OP_##op << 1 | \
> +       SBI_PMU_HW_CACHE_RESULT_##res)
> +
>  /**
>   * Special "firmware" events provided by the OpenSBI, even if the hardware
>   * does not support performance events. These events are encoded as a raw
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index c9b6011..e652645 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -32,6 +32,7 @@ config PLATFORM_ALLWINNER_D1
>  config PLATFORM_ANDES_AE350
>         bool "Andes AE350 support"
>         select SYS_ATCSMU
> +       select ANDES_HPM
>         select ANDES_PMU
>         default n
>
> @@ -39,6 +40,7 @@ config PLATFORM_RENESAS_RZFIVE
>         bool "Renesas RZ/Five support"
>         select ANDES45_PMA
>         select ANDES_SBI
> +       select ANDES_HPM
>         select ANDES_PMU
>         default n
>
> diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
> index 5b2ed91..de0d752 100644
> --- a/platform/generic/andes/Kconfig
> +++ b/platform/generic/andes/Kconfig
> @@ -8,6 +8,13 @@ config ANDES_SBI
>         bool "Andes SBI support"
>         default n
>
> +config ANDES_HPM
> +       bool "Andes HPM support"
> +       default n
> +       help
> +         This provides default HPM event counter mappings
> +         when a pmu node is missing.
> +
>  config ANDES_PMU
>         bool "Andes PMU extension (xandespmu) support"
>         default n
> diff --git a/platform/generic/andes/andes_hpm.c b/platform/generic/andes/andes_hpm.c
> new file mode 100644
> index 0000000..7fe5ac5
> --- /dev/null
> +++ b/platform/generic/andes/andes_hpm.c
> @@ -0,0 +1,405 @@
> +// SPDX-License-Identifier: BSD-2-Clause
> +/*
> + * andes_hpm.c - Default event mappings and helper routine
> + *
> + * Copyright (c) 2023 Andes Technology Corporation
> + */
> +
> +#include <andes/andes45.h>
> +#include <andes/andes_hpm.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi/sbi_pmu.h>
> +#include <sbi_utils/fdt/fdt_pmu.h>
> +
> +static const struct fdt_pmu_hw_event_select_map andes45_hw_evt_selects[] = {
> +       /* Hardware general events (Type #0) */
> +       {
> +               /* perf: cycles (eidx: 0x1) */
> +               .eidx = SBI_PMU_HW_CPU_CYCLES,
> +               .select = ANDES_CYCLES
> +       },
> +       {
> +               /* perf: instructions (eidx: 0x2) */
> +               .eidx = SBI_PMU_HW_INSTRUCTIONS,
> +               .select = ANDES_INSTRET
> +       },
> +       {
> +               /* perf: cache-references (eidx: 0x3) */
> +               .eidx = SBI_PMU_HW_CACHE_REFERENCES,
> +               .select = ANDES_DCACHE_ACCESS
> +       },
> +       {
> +               /* perf: cache-misses (eidx: 0x4) */
> +               .eidx = SBI_PMU_HW_CACHE_MISSES,
> +               .select = ANDES_DCACHE_MISS
> +       },
> +       {
> +               /* perf: branches (eidx: 0x5) */
> +               .eidx = SBI_PMU_HW_BRANCH_INSTRUCTIONS,
> +               .select = ANDES_CONDITION_BR,
> +       },
> +       {
> +               /* perf: branch-misses (eidx: 0x6) */
> +               .eidx = SBI_PMU_HW_BRANCH_MISSES,
> +               .select = ANDES_MISPREDICT_CONDITION_BR,
> +       },
> +       /* Hardware cache events (Type #1) */
> +       {
> +               /* perf: L1-dcache-loads (eidx: 0x10000) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, ACCESS),
> +               .select = ANDES_DCACHE_LOAD_ACCESS
> +       },
> +       {
> +               /* perf: L1-dcache-loads-misses (eidx: 0x10001) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, MISS),
> +               .select = ANDES_DCACHE_LOAD_MISS
> +       },
> +       {
> +               /* perf: L1-dcache-stores (eidx: 0x10002) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, ACCESS),
> +               .select = ANDES_DCACHE_STORE_ACCESS
> +       },
> +       {
> +               /* perf: L1-dcache-store-misses (eidx: 0x10003) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, MISS),
> +               .select = ANDES_DCACHE_STORE_MISS
> +       },
> +       {
> +               /* perf: L1-icache-load (eidx: 0x10008) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, ACCESS),
> +               .select = ANDES_ICACHE_ACCESS
> +       },
> +       {
> +               /* perf: L1-icache-load-misses (eidx: 0x10009) */
> +               .eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, MISS),
> +               .select = ANDES_ICACHE_MISS
> +       },
> +};
> +
> +static const struct fdt_pmu_hw_event_counter_map andes45_hw_evt_counters[] = {
> +       {
> +               /* perf: cycles (eidx: 0x1) */
> +               .eidx_start = SBI_PMU_HW_CPU_CYCLES,
> +               /* perf: branch-misses (eidx: 0x6) */
> +               .eidx_end = SBI_PMU_HW_BRANCH_MISSES,
> +               .ctr_map  = ANDES_MHPM_MAP,
> +       },
> +       {
> +               /* perf: L1-dcache-loads (eidx: 0x10000) */
> +               .eidx_start = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, ACCESS),
> +               /* perf: L1-dcache-store-misses (eidx: 0x10003) */
> +               .eidx_end = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, MISS),
> +               .ctr_map  = ANDES_MHPM_MAP,
> +       },
> +       {
> +               /* perf: L1-icache-load (eidx: 0x10008) */
> +               .eidx_start = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, ACCESS),
> +               /* perf: L1-icache-load-misses (eidx: 0x10009) */
> +               .eidx_end = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, MISS),
> +               .ctr_map  = ANDES_MHPM_MAP,
> +       },
> +};
> +
> +static const struct fdt_pmu_raw_event_counter_map andes45_raw_evt_counters[] = {
> +       {
> +               .select = ANDES_CYCLES,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INSTRET,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_LOAD_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_STORE_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_ATOMIC_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_SYS_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_COMPUTE_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CONDITION_BR,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_TAKEN_CONDITION_BR,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_JAL_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_JALR_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_RET_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CONTROL_TRANS_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_EX9_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_MUL_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_DIV_REMAINDER_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_LOAD_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_STORE_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_ADD_SUB_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_MUL_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_FUSED_MULADD_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_FLOAT_DIV_SQUARE_ROOT_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_OTHER_FLOAT_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_INT_MUL_AND_SUB_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_RETIRED_OP,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_ILM_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DLM_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_ICACHE_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_ICACHE_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_LOAD_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_LOAD_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_STORE_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_STORE_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_DCACHE_WB,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_ICACHE_FILL,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_DCACHE_FILL,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_UNCACHED_IFETCH_FROM_BUS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_UNCACHED_LOAD_FROM_BUS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_UNCACHED_IFETCH,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_UNCACHED_LOAD,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MAIN_ITLB_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MAIN_ITLB_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MAIN_DTLB_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MAIN_DTLB_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_CYCLE_WAIT_ITLB_FILL,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_PIPE_STALL_CYCLE_DTLB_MISS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_HW_PREFETCH_BUS_ACCESS,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MISPREDICT_CONDITION_BR,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MISPREDICT_TAKE_CONDITION_BR,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +       {
> +               .select = ANDES_MISPREDICT_TARGET_RET_INST,
> +               .select_mask = ANDES_RAW_EVENT_MASK,
> +               .ctr_map = ANDES_MHPM_MAP
> +       },
> +};
> +
> +int andes_pmu_setup(void)
> +{
> +       int i, rc;
> +       struct fdt_pmu_hw_event_select_map *event;
> +
> +       /*
> +        * At the moment, simply create mapping for any 45-series core
> +        * based on marchid, we may check and differentiate the mapping
> +        * by mimpid.
> +        */
> +       if (!is_andes(45))
> +               return 0;
> +
> +       for (i = 0; i < array_size(andes45_hw_evt_counters); i++) {
> +               rc = sbi_pmu_add_hw_event_counter_map(
> +                       andes45_hw_evt_counters[i].eidx_start,
> +                       andes45_hw_evt_counters[i].eidx_end,
> +                       andes45_hw_evt_counters[i].ctr_map);
> +               if (rc)
> +                       return rc;
> +       }
> +
> +       for (i = 0; i < array_size(andes45_hw_evt_selects); i++) {
> +               event         = &fdt_pmu_evt_select[hw_event_count];
> +               event->eidx   = andes45_hw_evt_selects[i].eidx;
> +               event->select = andes45_hw_evt_selects[i].select;
> +               hw_event_count++;
> +       }
> +
> +       for (i = 0; i < array_size(andes45_raw_evt_counters); i++) {
> +               rc = sbi_pmu_add_raw_event_counter_map(
> +                       andes45_raw_evt_counters[i].select,
> +                       andes45_raw_evt_counters[i].select_mask,
> +                       andes45_raw_evt_counters[i].ctr_map);
> +               if (rc)
> +                       return rc;
> +       }
> +
> +       return 0;
> +}
> diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
> index 6a8c66c..57caaf6 100644
> --- a/platform/generic/andes/objects.mk
> +++ b/platform/generic/andes/objects.mk
> @@ -8,3 +8,4 @@ platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
>  platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
>  platform-objs-$(CONFIG_ANDES_SBI) += andes/andes_sbi.o
>  platform-objs-$(CONFIG_ANDES_PMU) += andes/andes_pmu.o
> +platform-objs-$(CONFIG_ANDES_HPM) += andes/andes_hpm.o
> diff --git a/platform/generic/include/andes/andes_hpm.h b/platform/generic/include/andes/andes_hpm.h
> new file mode 100644
> index 0000000..79c75f6
> --- /dev/null
> +++ b/platform/generic/include/andes/andes_hpm.h
> @@ -0,0 +1,81 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2023 Andes Technology Corporation
> + */
> +
> +#ifndef _ANDES_HPM_H_
> +#define _ANDES_HPM_H_
> +
> +#define ANDES_MHPM_MAP         0x78
> +#define ANDES_RAW_EVENT_MASK   ~0ULL
> +
> +/* Event selector for instruction commit events */
> +#define ANDES_CYCLES                           0x10
> +#define ANDES_INSTRET                          0x20
> +#define ANDES_INT_LOAD_INST                    0x30
> +#define ANDES_INT_STORE_INST                   0x40
> +#define ANDES_ATOMIC_INST                      0x50
> +#define ANDES_SYS_INST                         0x60
> +#define ANDES_INT_COMPUTE_INST                 0x70
> +#define ANDES_CONDITION_BR                     0x80
> +#define ANDES_TAKEN_CONDITION_BR               0x90
> +#define ANDES_JAL_INST                         0xA0
> +#define ANDES_JALR_INST                                0xB0
> +#define ANDES_RET_INST                         0xC0
> +#define ANDES_CONTROL_TRANS_INST               0xD0
> +#define ANDES_EX9_INST                         0xE0
> +#define ANDES_INT_MUL_INST                     0xF0
> +#define ANDES_INT_DIV_REMAINDER_INST           0x100
> +#define ANDES_FLOAT_LOAD_INST                  0x110
> +#define ANDES_FLOAT_STORE_INST                 0x120
> +#define ANDES_FLOAT_ADD_SUB_INST               0x130
> +#define ANDES_FLOAT_MUL_INST                   0x140
> +#define ANDES_FLOAT_FUSED_MULADD_INST          0x150
> +#define ANDES_FLOAT_DIV_SQUARE_ROOT_INST       0x160
> +#define ANDES_OTHER_FLOAT_INST                 0x170
> +#define ANDES_INT_MUL_AND_SUB_INST             0x180
> +#define ANDES_RETIRED_OP                       0x190
> +
> +/* Event selector for memory system events */
> +#define ANDES_ILM_ACCESS                       0x01
> +#define ANDES_DLM_ACCESS                       0x11
> +#define ANDES_ICACHE_ACCESS                    0x21
> +#define ANDES_ICACHE_MISS                      0x31
> +#define ANDES_DCACHE_ACCESS                    0x41
> +#define ANDES_DCACHE_MISS                      0x51
> +#define ANDES_DCACHE_LOAD_ACCESS               0x61
> +#define ANDES_DCACHE_LOAD_MISS                 0x71
> +#define ANDES_DCACHE_STORE_ACCESS              0x81
> +#define ANDES_DCACHE_STORE_MISS                        0x91
> +#define ANDES_DCACHE_WB                                0xA1
> +#define ANDES_CYCLE_WAIT_ICACHE_FILL           0xB1
> +#define ANDES_CYCLE_WAIT_DCACHE_FILL           0xC1
> +#define ANDES_UNCACHED_IFETCH_FROM_BUS         0xD1
> +#define ANDES_UNCACHED_LOAD_FROM_BUS           0xE1
> +#define ANDES_CYCLE_WAIT_UNCACHED_IFETCH       0xF1
> +#define ANDES_CYCLE_WAIT_UNCACHED_LOAD         0x101
> +#define ANDES_MAIN_ITLB_ACCESS                 0x111
> +#define ANDES_MAIN_ITLB_MISS                   0x121
> +#define ANDES_MAIN_DTLB_ACCESS                 0x131
> +#define ANDES_MAIN_DTLB_MISS                   0x141
> +#define ANDES_CYCLE_WAIT_ITLB_FILL             0x151
> +#define ANDES_PIPE_STALL_CYCLE_DTLB_MISS       0x161
> +#define ANDES_HW_PREFETCH_BUS_ACCESS           0x171
> +
> +/* Event selector for microarchitecture events */
> +#define ANDES_MISPREDICT_CONDITION_BR          0x02
> +#define ANDES_MISPREDICT_TAKE_CONDITION_BR     0x12
> +#define ANDES_MISPREDICT_TARGET_RET_INST       0x22
> +
> +#ifdef CONFIG_ANDES_HPM
> +
> +int andes_pmu_setup(void);
> +
> +#else
> +
> +int andes_pmu_setup(void) { return 0; }
> +
> +#endif /* CONFIG_ANDES_HPM */
> +
> +#endif /* _ANDES_HPM_H_ */
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Fri Nov 24 06:56:10 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Fri, 24 Nov 2023 14:56:10 +0000
Subject: [PATCH v3 15/15] docs: pmu: Add Andes PMU node example
In-Reply-To: <20231122073617.379441-16-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-16-peterlin@andestech.com>
Message-ID: <CA+V-a8swa_Vs82Sn02QDVRV7EtviZWt=x-YTQkDgMuRBnrTzPA@mail.gmail.com>

On Wed, Nov 22, 2023 at 7:42?AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add PMU node example for event index to counter index mapping
> and selector value translation of Andes' CPUs.
>
> Currently, there are 4 HPM counters that can be used to monitor
> all of the events for each hart.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Locus Wei-Han Chen <locus84 at andestech.com>
> Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes v1 -> v2:
>   - sync up with datasheet
> Changes v2 -> v3:
>   - Include Anup's RB tag
> ---
>  docs/pmu_support.md | 82 +++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 82 insertions(+)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/docs/pmu_support.md b/docs/pmu_support.md
> index 8cfa08c..9b48f1e 100644
> --- a/docs/pmu_support.md
> +++ b/docs/pmu_support.md
> @@ -125,3 +125,85 @@ pmu {
>                                           <0x0 0x2 0xffffffff 0xffffe0ff 0x18>;
>  };
>  ```
> +
> +### Example 3
> +
> +```
> +/*
> + * For Andes 45-series platforms. The encodings can be found in the
> + * "Machine Performance Monitoring Event Selector" section
> + * http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
> + */
> +pmu {
> +       compatible                      = "riscv,pmu";
> +       riscv,event-to-mhpmevent =
> +                                        <0x1 0x0000 0x10>, /* CPU_CYCLES          -> Cycle count */
> +                                        <0x2 0x0000 0x20>, /* INSTRUCTIONS        -> Retired instruction count */
> +                                        <0x3 0x0000 0x41>, /* CACHE_REFERENCES    -> D-Cache access */
> +                                        <0x4 0x0000 0x51>, /* CACHE_MISSES        -> D-Cache miss */
> +                                        <0x5 0x0000 0x80>, /* BRANCH_INSTRUCTIONS -> Conditional branch instruction count */
> +                                        <0x6 0x0000 0x02>, /* BRANCH_MISSES       -> Misprediction of conditional branches */
> +                                        <0x10000 0x0000 0x61>,  /* L1D_READ_ACCESS  -> D-Cache load access */
> +                                        <0x10001 0x0000 0x71>,  /* L1D_READ_MISS    -> D-Cache load miss */
> +                                        <0x10002 0x0000 0x81>,  /* L1D_WRITE_ACCESS -> D-Cache store access */
> +                                        <0x10003 0x0000 0x91>,  /* L1D_WRITE_MISS   -> D-Cache store miss */
> +                                        <0x10008 0x0000 0x21>,  /* L1I_READ_ACCESS  -> I-Cache access */
> +                                        <0x10009 0x0000 0x31>;  /* L1I_READ_MISS    -> I-Cache miss */
> +       riscv,event-to-mhpmcounters = <0x1 0x6 0x78>,
> +                                                       <0x10000 0x10003 0x78>,
> +                                                       <0x10008 0x10009 0x78>;
> +       riscv,raw-event-to-mhpmcounters =
> +                                               <0x0 0x10 0xffffffff 0xffffffff 0x78>, /* Cycle count */
> +                                               <0x0 0x20 0xffffffff 0xffffffff 0x78>, /* Retired instruction count */
> +                                               <0x0 0x30 0xffffffff 0xffffffff 0x78>, /* Integer load instruction count */
> +                                               <0x0 0x40 0xffffffff 0xffffffff 0x78>, /* Integer store instruction count */
> +                                               <0x0 0x50 0xffffffff 0xffffffff 0x78>, /* Atomic instruction count */
> +                                               <0x0 0x60 0xffffffff 0xffffffff 0x78>, /* System instruction count */
> +                                               <0x0 0x70 0xffffffff 0xffffffff 0x78>, /* Integer computational instruction count */
> +                                               <0x0 0x80 0xffffffff 0xffffffff 0x78>, /* Conditional branch instruction count */
> +                                               <0x0 0x90 0xffffffff 0xffffffff 0x78>, /* Taken conditional branch instruction count */
> +                                               <0x0 0xA0 0xffffffff 0xffffffff 0x78>, /* JAL instruction count */
> +                                               <0x0 0xB0 0xffffffff 0xffffffff 0x78>, /* JALR instruction count */
> +                                               <0x0 0xC0 0xffffffff 0xffffffff 0x78>, /* Return instruction count */
> +                                               <0x0 0xD0 0xffffffff 0xffffffff 0x78>, /* Control transfer instruction count */
> +                                               <0x0 0xE0 0xffffffff 0xffffffff 0x78>, /* EXEC.IT instruction count */
> +                                               <0x0 0xF0 0xffffffff 0xffffffff 0x78>, /* Integer multiplication instruction count */
> +                                               <0x0 0x100 0xffffffff 0xffffffff 0x78>, /* Integer division instruction count */
> +                                               <0x0 0x110 0xffffffff 0xffffffff 0x78>, /* Floating-point load instruction count */
> +                                               <0x0 0x120 0xffffffff 0xffffffff 0x78>, /* Floating-point store instruction count */
> +                                               <0x0 0x130 0xffffffff 0xffffffff 0x78>, /* Floating-point addition/subtraction instruction count */
> +                                               <0x0 0x140 0xffffffff 0xffffffff 0x78>, /* Floating-point multiplication instruction count */
> +                                               <0x0 0x150 0xffffffff 0xffffffff 0x78>, /* Floating-point fused multiply-add instruction count */
> +                                               <0x0 0x160 0xffffffff 0xffffffff 0x78>, /* Floating-point division or square-root instruction count */
> +                                               <0x0 0x170 0xffffffff 0xffffffff 0x78>, /* Other floating-point instruction count */
> +                                               <0x0 0x180 0xffffffff 0xffffffff 0x78>, /* Integer multiplication and add/sub instruction count */
> +                                               <0x0 0x190 0xffffffff 0xffffffff 0x78>, /* Retired operation count */
> +                                               <0x0 0x01 0xffffffff 0xffffffff 0x78>, /* ILM access */
> +                                               <0x0 0x11 0xffffffff 0xffffffff 0x78>, /* DLM access */
> +                                               <0x0 0x21 0xffffffff 0xffffffff 0x78>, /* I-Cache access */
> +                                               <0x0 0x31 0xffffffff 0xffffffff 0x78>, /* I-Cache miss */
> +                                               <0x0 0x41 0xffffffff 0xffffffff 0x78>, /* D-Cache access */
> +                                               <0x0 0x51 0xffffffff 0xffffffff 0x78>, /* D-Cache miss */
> +                                               <0x0 0x61 0xffffffff 0xffffffff 0x78>, /* D-Cache load access */
> +                                               <0x0 0x71 0xffffffff 0xffffffff 0x78>, /* D-Cache load miss */
> +                                               <0x0 0x81 0xffffffff 0xffffffff 0x78>, /* D-Cache store access */
> +                                               <0x0 0x91 0xffffffff 0xffffffff 0x78>, /* D-Cache store miss */
> +                                               <0x0 0xA1 0xffffffff 0xffffffff 0x78>, /* D-Cache writeback */
> +                                               <0x0 0xB1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for I-Cache fill data */
> +                                               <0x0 0xC1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for D-Cache fill data */
> +                                               <0x0 0xD1 0xffffffff 0xffffffff 0x78>, /* Uncached fetch data access from bus */
> +                                               <0x0 0xE1 0xffffffff 0xffffffff 0x78>, /* Uncached load data access from bus */
> +                                               <0x0 0xF1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for uncached fetch data from bus */
> +                                               <0x0 0x101 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for uncached load data from bus */
> +                                               <0x0 0x111 0xffffffff 0xffffffff 0x78>, /* Main ITLB access */
> +                                               <0x0 0x121 0xffffffff 0xffffffff 0x78>, /* Main ITLB miss */
> +                                               <0x0 0x131 0xffffffff 0xffffffff 0x78>, /* Main DTLB access */
> +                                               <0x0 0x141 0xffffffff 0xffffffff 0x78>, /* Main DTLB miss */
> +                                               <0x0 0x151 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for Main ITLB fill data */
> +                                               <0x0 0x161 0xffffffff 0xffffffff 0x78>, /* Pipeline stall cycles caused by Main DTLB miss */
> +                                               <0x0 0x171 0xffffffff 0xffffffff 0x78>, /* Hardware prefetch bus access */
> +                                               <0x0 0x02 0xffffffff 0xffffffff 0x78>, /* Misprediction of conditional branches */
> +                                               <0x0 0x12 0xffffffff 0xffffffff 0x78>, /* Misprediction of taken conditional branches */
> +                                               <0x0 0x22 0xffffffff 0xffffffff 0x78>; /* Misprediction of targets of Return instructions */
> +};
> +```
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Fri Nov 24 20:42:06 2023
From: anup at brainfault.org (Anup Patel)
Date: Sat, 25 Nov 2023 10:12:06 +0530
Subject: [PATCH v3 10/15] lib: utils: fdt_fixup: Allow preserving PMU properties
In-Reply-To: <20231122073617.379441-11-peterlin@andestech.com>
References: <20231122073617.379441-1-peterlin@andestech.com> <20231122073617.379441-11-peterlin@andestech.com>
Message-ID: <CAAhSdy2iMgXBJcN_OOju3cHTLa6n2hF58YnDK-X5iLAs6NWQKg@mail.gmail.com>

On Wed, Nov 22, 2023 at 1:10?PM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> Add a scratch option to control PMU fixup, so the next
> stage software can dump the PMU node including event
> mapping information for debugging purposes.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes v1 -> v2:
>   - New patch
> Changes v2 -> v3:
>   - Rename to SBI_SCRATCH_PRESERVE_PMU_NODE (suggested by Anup)
>   - Include Anup's RB tag
> ---
>  include/sbi/sbi_scratch.h | 2 ++
>  lib/utils/fdt/fdt_fixup.c | 6 +++++-
>  2 files changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/include/sbi/sbi_scratch.h b/include/sbi/sbi_scratch.h
> index e6a33ba..0f67cde 100644
> --- a/include/sbi/sbi_scratch.h
> +++ b/include/sbi/sbi_scratch.h
> @@ -151,6 +151,8 @@ enum sbi_scratch_options {
>         SBI_SCRATCH_NO_BOOT_PRINTS = (1 << 0),
>         /** Enable runtime debug prints */
>         SBI_SCRATCH_DEBUG_PRINTS = (1 << 1),
> +       /** Preserve PMU node properties */
> +       SBI_SCRATCH_PRESERVE_PMU_NODE = (1 << 2),

I have second thoughts about using the scratch option for this.

Let's not waste a bit in the scratch option just for debugging the PMU node.

I suggest adding the kconfig option FDT_FIXUPS_PRESERVE_PMU_NODE
in <opensbi>lib/utils/fdt/Kconfig and use it in fdt_fixups() function.

Regards,
Anup

>  };
>
>  /** Get pointer to sbi_scratch for current HART */
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index e213ded..cf20edf 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -387,6 +387,8 @@ int fdt_reserved_memory_fixup(void *fdt)
>
>  void fdt_fixups(void *fdt)
>  {
> +       struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> +
>         fdt_aplic_fixup(fdt);
>
>         fdt_imsic_fixup(fdt);
> @@ -394,5 +396,7 @@ void fdt_fixups(void *fdt)
>         fdt_plic_fixup(fdt);
>
>         fdt_reserved_memory_fixup(fdt);
> -       fdt_pmu_fixup(fdt);
> +
> +       if (!(scratch->options & SBI_SCRATCH_PRESERVE_PMU_NODE))
> +               fdt_pmu_fixup(fdt);
>  }
> --
> 2.34.1
>


From anup at brainfault.org  Sun Nov 26 05:18:06 2023
From: anup at brainfault.org (Anup Patel)
Date: Sun, 26 Nov 2023 18:48:06 +0530
Subject: [PATCH 0/2] Allow relaxed MMIO writes in device IPI callbacks
In-Reply-To: <ec81a8a5-f6e7-1301-d3ac-5bb3b09e4ab6@gmail.com>
References: <20231121161711.904674-1-apatel@ventanamicro.com> <ec81a8a5-f6e7-1301-d3ac-5bb3b09e4ab6@gmail.com>
Message-ID: <CAAhSdy0MHBb66gqqpuPoJHNz7L458SGsQW9dTUkQxHHLjeb-dA@mail.gmail.com>

On Wed, Nov 22, 2023 at 6:15?AM Bo Gan <ganboing at gmail.com> wrote:
>
> On 11/21/23 8:17 AM, Anup Patel wrote:
> > This series improves the barrier usage around device IPI callbacks
> > such that platform IPI driver can now use relaxed MMIO writes in
> > device IPI callbacks.
> >
> > These patches can also be found in the sbi_ipi_barrier_imp_v1 branch
> > at: https://github.com/avpatel/opensbi.git
> >
> > Anup Patel (2):
> >    lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
> >    lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback
> >
> >   lib/sbi/sbi_ipi.c           | 31 +++++++++++++++++++++++++------
> >   lib/utils/ipi/aclint_mswi.c |  4 ++--
> >   lib/utils/irqchip/imsic.c   |  4 ++--
> >   3 files changed, 29 insertions(+), 10 deletions(-)
> >
> Hi Anup,
>
> Thank you so much for fixing this. I hope my previous discussion helped.
> Would you mind having a Reported-by: Bo Gan <ganboing at gmail.com> ?
> Really appreciate you helping me understand the code.

Sure, I will add Reported-by

Regards,
Anup


From anup at brainfault.org  Sun Nov 26 05:18:39 2023
From: anup at brainfault.org (Anup Patel)
Date: Sun, 26 Nov 2023 18:48:39 +0530
Subject: [PATCH 0/2] Allow relaxed MMIO writes in device IPI callbacks
In-Reply-To: <20231121161711.904674-1-apatel@ventanamicro.com>
References: <20231121161711.904674-1-apatel@ventanamicro.com>
Message-ID: <CAAhSdy0zbmfYgKw+gNw5CLJOyUWHE5SDEiDiMAt5xKqn6V3B1g@mail.gmail.com>

On Tue, Nov 21, 2023 at 9:48?PM Anup Patel <apatel at ventanamicro.com> wrote:
>
> This series improves the barrier usage around device IPI callbacks
> such that platform IPI driver can now use relaxed MMIO writes in
> device IPI callbacks.
>
> These patches can also be found in the sbi_ipi_barrier_imp_v1 branch
> at: https://github.com/avpatel/opensbi.git
>
> Anup Patel (2):
>   lib: sbi: Allow relaxed MMIO writes in device ipi_send() callback
>   lib: sbi: Allow relaxed MMIO writes in device ipi_clear() callback

Applied this series to the riscv/opensbi repo.

Regards,
Anup

>
>  lib/sbi/sbi_ipi.c           | 31 +++++++++++++++++++++++++------
>  lib/utils/ipi/aclint_mswi.c |  4 ++--
>  lib/utils/irqchip/imsic.c   |  4 ++--
>  3 files changed, 29 insertions(+), 10 deletions(-)
>
> --
> 2.34.1
>


From karthik.bk at incoresemi.com  Mon Nov 27 03:59:56 2023
From: karthik.bk at incoresemi.com (Karthik B K)
Date: Mon, 27 Nov 2023 11:59:56 +0000
Subject: [QUESTION] Usage of negative number notations on unsigned
 numbers.
In-Reply-To: <878r6n76im.fsf@igel.home>
References: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
 <878r6n76im.fsf@igel.home>
Message-ID: <PN2P287MB0754BFB8B175A81159CBBF45FABDA@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>




From:?Andreas Schwab <schwab at linux-m68k.org>
Sent:?Friday, November 24, 2023 4:38 PM
To:?Karthik B K <karthik.bk at incoresemi.com>
Cc:?opensbi at lists.infradead.org <opensbi at lists.infradead.org>
Subject:?Re: [QUESTION] Usage of negative number notations on unsigned numbers.
?
On Nov 24 2023, Karthik B K wrote:

> If so, isn't it safer to use (~0UL) ? This could be a very silent bug. Please correct me if I'm wrong.

The safest way is to write -1 without any suffix.? This is guaranteed to
evaluate to the maximum value when converted to any unsigned type.

This is with the assumption that the encoding for -1 == 2^32-1.
I'm still trying to understand what arithmetic negation mean to an unsigned quantity and its representation in UL.
While 0 has a defined encoding in a UL type, -1 does not!

karthik.

From schwab at linux-m68k.org  Mon Nov 27 04:09:14 2023
From: schwab at linux-m68k.org (Andreas Schwab)
Date: Mon, 27 Nov 2023 13:09:14 +0100
Subject: [QUESTION] Usage of negative number notations on unsigned numbers.
In-Reply-To: <PN2P287MB0754BFB8B175A81159CBBF45FABDA@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
	(Karthik B. K.'s message of "Mon, 27 Nov 2023 11:59:56 +0000")
References: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
	<878r6n76im.fsf@igel.home>
	<PN2P287MB0754BFB8B175A81159CBBF45FABDA@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
Message-ID: <87leajmm8l.fsf@igel.home>

On Nov 27 2023, Karthik B K wrote:

> This is with the assumption that the encoding for -1 == 2^32-1.

No, the representation has nothing to do with that.  It's a value
conversion.

-- 
Andreas Schwab, schwab at linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."


From wxjstz at 126.com  Mon Nov 27 04:39:11 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 27 Nov 2023 20:39:11 +0800
Subject: [PATCH] lib: sbi: Improve sbi_hsm_hart_interruptible_mask
Message-ID: <20231127123933.48851-1-wxjstz@126.com>

Improve bit scanning with for_each_set_bit.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_hsm.c | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index 147f954..bffac27 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -114,22 +114,18 @@ int sbi_hsm_hart_interruptible_mask(const struct sbi_domain *dom,
 				    ulong hbase, ulong *out_hmask)
 {
 	int hstate;
-	ulong i, hmask, dmask;
+	ulong i, dmask;
 
 	*out_hmask = 0;
 	if (!sbi_hartid_valid(hbase))
 		return SBI_EINVAL;
 
 	dmask = sbi_domain_get_assigned_hartmask(dom, hbase);
-	for (i = 0; i < BITS_PER_LONG; i++) {
-		hmask = 1UL << i;
-		if (!(dmask & hmask))
-			continue;
-
+	for_each_set_bit(i, &dmask, BITS_PER_LONG) {
 		hstate = __sbi_hsm_hart_get_state(hbase + i);
 		if (hstate == SBI_HSM_STATE_STARTED ||
 		    hstate == SBI_HSM_STATE_SUSPENDED)
-			*out_hmask |= hmask;
+			*out_hmask |= 1UL << i;
 	}
 
 	return 0;
-- 
2.42.0



From karthik.bk at incoresemi.com  Mon Nov 27 04:55:30 2023
From: karthik.bk at incoresemi.com (Karthik B K)
Date: Mon, 27 Nov 2023 18:25:30 +0530
Subject: [QUESTION] Usage of negative number notations on unsigned
 numbers.
In-Reply-To: <87leajmm8l.fsf@igel.home>
References: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
 <878r6n76im.fsf@igel.home>
 <PN2P287MB0754BFB8B175A81159CBBF45FABDA@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
 <87leajmm8l.fsf@igel.home>
Message-ID: <1b2d1b62-e6d7-4de2-9d27-4892a0f2a9dd@incoresemi.com>


On 27/11/23 17:39, Andreas Schwab wrote:
> On Nov 27 2023, Karthik B K wrote:
>
>> This is with the assumption that the encoding for -1 == 2^32-1.
> No, the representation has nothing to do with that.  It's a value
> conversion.

What's the encoding of (2^32-1)UL and (2^64-1)UL ?


karthik.



From schwab at linux-m68k.org  Mon Nov 27 05:46:24 2023
From: schwab at linux-m68k.org (Andreas Schwab)
Date: Mon, 27 Nov 2023 14:46:24 +0100
Subject: [QUESTION] Usage of negative number notations on unsigned numbers.
In-Reply-To: <1b2d1b62-e6d7-4de2-9d27-4892a0f2a9dd@incoresemi.com> (Karthik
	B. K.'s message of "Mon, 27 Nov 2023 18:25:30 +0530")
References: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
	<878r6n76im.fsf@igel.home>
	<PN2P287MB0754BFB8B175A81159CBBF45FABDA@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
	<87leajmm8l.fsf@igel.home>
	<1b2d1b62-e6d7-4de2-9d27-4892a0f2a9dd@incoresemi.com>
Message-ID: <87h6l7mhqn.fsf@igel.home>

On Nov 27 2023, Karthik B K wrote:

> What's the encoding of (2^32-1)UL and (2^64-1)UL ?

A value does not have an encoding.  Only a memory location does.

-- 
Andreas Schwab, schwab at linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."


From peterlin at andestech.com  Mon Nov 27 21:23:05 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 28 Nov 2023 13:23:05 +0800
Subject: [PATCH v3 00/15] Add Andes PMU extension support
In-Reply-To: <CAOnJCULdLyFugMFuY4JLL2+M2D8J85iU_A91ZFBLxVbhzLGASg@mail.gmail.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
 <CAOnJCU+qacoXRrs3MDTyskEnJWJ-vUds57m8rKDq7UedkOhqVA@mail.gmail.com>
 <CAOnJCULdLyFugMFuY4JLL2+M2D8J85iU_A91ZFBLxVbhzLGASg@mail.gmail.com>
Message-ID: <ZWV5Oe_9T8mzUjtg@APC323>

Hi Atish,

On Wed, Nov 22, 2023 at 04:27:02PM -0800, Atish Patra wrote:
> On Wed, Nov 22, 2023 at 4:07?PM Atish Patra <atishp at atishpatra.org> wrote:
> >
> > On Tue, Nov 21, 2023 at 11:39?PM Yu Chien Peter Lin
> > <peterlin at andestech.com> wrote:
> > >
> > > This patch series enables perf tool to utilize Andes PMU
> > > extension via PMU SBI calls, provides PMU device callbacks
> > > to achieve event sampling and mode filtering.
> > >
> > > This version mainly fixes single-core freeze when quitting
> > > 'perf top', adds andes_pmu_setup() to provide default event
> > > mappings when fdt_pmu_setup() can't find PMU node. And add
> > > Xandespmu to hart extension list so that the inhibit bits
> > > can be updated in a similar way to how Smcntrpmf does.
> > >
> > > The last patch provides a PMU node example used on AX45MP cores.
> > >
> >
> > Do you have a publicly available spec that describes what XAndesPMU does ?
> 
> Never mind. I found it in the last patch. It would be good to include
> it in the cover letter as well.
> http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf

Sure, will add the link in the v4 cover letter.

Thanks,
Peter Lin

> 
> >
> > > The OpenSBI and Linux patches can be found on Andes Technology GitHub
> > > - https://github.com/andestech/opensbi/commits/andes-pmu-support-v3
> > > - https://github.com/andestech/linux/commits/andes-pmu-support-v4
> > >
> > > Leo Yu-Chi Liang (1):
> > >   lib: ipi: Adjust Andes PLICSW to single-bit-per-hart scheme
> > >
> > > Yu Chien Peter Lin (14):
> > >   sbi: sbi_pmu: Improve sbi_pmu_init() error handling
> > >   lib: sbi: Add Xandespmu in hart extensions
> > >   sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
> > >   platform: include: andes45: Add PMU related CSR defines
> > >   platform: generic: Introduce pmu_init() platform override
> > >   platform: andes: Add Andes custom PMU support
> > >   platform: andes: Enable Andes PMU for AE350
> > >   platform: rzfive: Enable Andes PMU for RZ/Five
> > >   lib: utils: fdt_fixup: Allow preserving PMU properties
> > >   platform: andes: Factor out is_andes() helper
> > >   lib: utils: fdt_pmu: Make the fdt_pmu_evt_select table global variable
> > >   lib: utils: fdt_pmu: Do not iterate over the fdt_pmu_evt_select table
> > >   platform: andes: Add Andes default PMU mapping support
> > >   docs: pmu: Add Andes PMU node example
> > >
> > >  docs/pmu_support.md                          |  82 ++++
> > >  include/sbi/sbi_ecall_interface.h            |   5 +
> > >  include/sbi/sbi_hart.h                       |   2 +
> > >  include/sbi/sbi_pmu.h                        |   6 +
> > >  include/sbi/sbi_scratch.h                    |   2 +
> > >  include/sbi_utils/fdt/fdt_helper.h           |  17 +
> > >  include/sbi_utils/fdt/fdt_pmu.h              |   6 +
> > >  include/sbi_utils/ipi/andes_plicsw.h         |  23 +-
> > >  lib/sbi/sbi_hart.c                           |   3 +
> > >  lib/sbi/sbi_pmu.c                            |  25 +-
> > >  lib/utils/fdt/fdt_fixup.c                    |   6 +-
> > >  lib/utils/fdt/fdt_pmu.c                      |  17 +-
> > >  lib/utils/ipi/andes_plicsw.c                 | 104 ++---
> > >  platform/generic/Kconfig                     |   4 +
> > >  platform/generic/andes/Kconfig               |  15 +
> > >  platform/generic/andes/ae350.c               |  19 +-
> > >  platform/generic/andes/andes_hpm.c           | 405 +++++++++++++++++++
> > >  platform/generic/andes/andes_pmu.c           | 102 +++++
> > >  platform/generic/andes/objects.mk            |   2 +
> > >  platform/generic/include/andes/andes45.h     |  32 ++
> > >  platform/generic/include/andes/andes_hpm.h   |  81 ++++
> > >  platform/generic/include/andes/andes_pmu.h   |  33 ++
> > >  platform/generic/include/platform_override.h |   1 +
> > >  platform/generic/platform.c                  |  11 +-
> > >  platform/generic/renesas/rzfive/rzfive.c     |   5 +-
> > >  25 files changed, 897 insertions(+), 111 deletions(-)
> > >  create mode 100644 platform/generic/andes/andes_hpm.c
> > >  create mode 100644 platform/generic/andes/andes_pmu.c
> > >  create mode 100644 platform/generic/include/andes/andes_hpm.h
> > >  create mode 100644 platform/generic/include/andes/andes_pmu.h
> > >
> > > --
> > > 2.34.1
> > >
> >
> >
> > --
> > Regards,
> > Atish
> 
> 
> 
> -- 
> Regards,
> Atish


From karthik.bk at incoresemi.com  Mon Nov 27 21:28:03 2023
From: karthik.bk at incoresemi.com (Karthik B K)
Date: Tue, 28 Nov 2023 10:58:03 +0530
Subject: [QUESTION] Usage of negative number notations on unsigned
 numbers.
In-Reply-To: <87h6l7mhqn.fsf@igel.home>
References: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
 <878r6n76im.fsf@igel.home>
 <PN2P287MB0754BFB8B175A81159CBBF45FABDA@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
 <87leajmm8l.fsf@igel.home>
 <1b2d1b62-e6d7-4de2-9d27-4892a0f2a9dd@incoresemi.com>
 <87h6l7mhqn.fsf@igel.home>
Message-ID: <1c25a9a7-708f-4840-9c7a-a4dedffb2b0a@incoresemi.com>

On 27/11/23 19:16, Andreas Schwab wrote:
> On Nov 27 2023, Karthik B K wrote:
>
>> What's the encoding of (2^32-1)UL and (2^64-1)UL ?
> A value does not have an encoding.  Only a memory location does.
>
So does a register (container of bits), and a type (signed, unsigned, 
float).

The "encoding" is dependent on the "container" (reg or mem) and "type" 
(signed, unsigned, float).

The same bits (11...11) would reflect very different values (-1, 2^32-1, 
quiteNaN).

The container of (2^32-1) is UInt. 32b unsigned integer. -1 cannot be 
represented in a 32b unsigned int.


I'm trying to say, that the unary operator (-) in the expression (-1UL) 
is an integer arithmetic operator.

The unary operator (-) when applied to an unsigned type, results in the 
_negative_ of it's (promoted as per _usual arithmetic conversions_) 
operand, which in this case, is a signed integer.

The result will retain this type. Are we assuming an implicit cast ?

This looks like an illegal set of operations to create the bit pattern 
we desire (11...11) for an unsigned int.


Simply put, a -1 in a UL is an oxymoron. Don't you think ?

Please correct me if I'm wrong.


karthik.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 665 bytes
Desc: OpenPGP digital signature
URL: <http://lists.infradead.org/pipermail/opensbi/attachments/20231128/41bd0218/attachment.sig>

From peterlin at andestech.com  Mon Nov 27 21:33:26 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 28 Nov 2023 13:33:26 +0800
Subject: [PATCH v3 02/15] sbi: sbi_pmu: Improve sbi_pmu_init() error
 handling
In-Reply-To: <CAOnJCUJ14BFthoaBoroQYFDHZqnq60mLTfQ8A7AB9EXo4_yKBA@mail.gmail.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
 <20231122073617.379441-3-peterlin@andestech.com>
 <CAOnJCUJ14BFthoaBoroQYFDHZqnq60mLTfQ8A7AB9EXo4_yKBA@mail.gmail.com>
Message-ID: <ZWV7piCFOP6ynav6@APC323>

Hi Atish,

Thanks for the review.

On Wed, Nov 22, 2023 at 03:47:43PM -0800, Atish Patra wrote:
> On Tue, Nov 21, 2023 at 11:39?PM Yu Chien Peter Lin
> <peterlin at andestech.com> wrote:
> >
> > This patch makes the following changes:
> >
> > - As sbi_platform_pmu_init() returns a negative error code on
> >   failure, let sbi_pmu_init() to hang by propagating the error
> >   code.
> >
> 
> Why ? PMU is not a critical resource for booting the system. A system
> can function without it.
> Do we really want to hang the system booting in case sbi_pmu_init fails ?

Agreed, I will simply print the error code with sbi_dprintf() rather
than returning error.

Best regards,
Peter Lin

> > - In order to distinguish the SBI_EFAIL error returned by
> >   sbi_pmu_add_*_counter_map(), return SBI_ENOENT to indicate
> >   that fdt_pmu_setup() failed to locate "riscv,pmu" node, and
> >   generic_pmu_init() ignores such case.
> >
> > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > Reviewed-by: Anup Patel <anup at brainfault.org>
> > ---
> > Changes v1 -> v2:
> >   - New patch
> > Changes v2 -> v3:
> >   - Include Anup's RB tag
> > ---
> >  lib/sbi/sbi_pmu.c           | 5 ++++-
> >  lib/utils/fdt/fdt_pmu.c     | 2 +-
> >  platform/generic/platform.c | 8 +++++++-
> >  3 files changed, 12 insertions(+), 3 deletions(-)
> >
> > diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> > index f4c8fc4..3cbd4ff 100644
> > --- a/lib/sbi/sbi_pmu.c
> > +++ b/lib/sbi/sbi_pmu.c
> > @@ -957,6 +957,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
> >         int hpm_count = sbi_fls(sbi_hart_mhpm_mask(scratch));
> >         struct sbi_pmu_hart_state *phs;
> >         const struct sbi_platform *plat;
> > +       int rc;
> >
> >         if (cold_boot) {
> >                 hw_event_map = sbi_calloc(sizeof(*hw_event_map),
> > @@ -972,7 +973,9 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
> >
> >                 plat = sbi_platform_ptr(scratch);
> >                 /* Initialize hw pmu events */
> > -               sbi_platform_pmu_init(plat);
> > +               rc = sbi_platform_pmu_init(plat);
> > +               if (rc)
> > +                       return rc;
> >
> >                 /* mcycle & minstret is available always */
> >                 if (!hpm_count)
> > diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
> > index 83301bb..a8d7648 100644
> > --- a/lib/utils/fdt/fdt_pmu.c
> > +++ b/lib/utils/fdt/fdt_pmu.c
> > @@ -74,7 +74,7 @@ int fdt_pmu_setup(void *fdt)
> >
> >         pmu_offset = fdt_node_offset_by_compatible(fdt, -1, "riscv,pmu");
> >         if (pmu_offset < 0)
> > -               return SBI_EFAIL;
> > +               return SBI_ENOENT;
> >
> >         event_ctr_map = fdt_getprop(fdt, pmu_offset,
> >                                     "riscv,event-to-mhpmcounters", &len);
> > diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> > index 85acecd..fa400b9 100644
> > --- a/platform/generic/platform.c
> > +++ b/platform/generic/platform.c
> > @@ -265,7 +265,13 @@ static u32 generic_tlb_num_entries(void)
> >
> >  static int generic_pmu_init(void)
> >  {
> > -       return fdt_pmu_setup(fdt_get_address());
> > +       int rc;
> > +
> > +       rc = fdt_pmu_setup(fdt_get_address());
> > +       if (rc && rc != SBI_ENOENT)
> > +               return rc;
> > +
> > +       return 0;
> >  }
> >
> >  static uint64_t generic_pmu_xlate_to_mhpmevent(uint32_t event_idx,
> > --
> > 2.34.1
> >
> 
> 
> -- 
> Regards,
> Atish


From jrtc27 at jrtc27.com  Mon Nov 27 22:01:33 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Tue, 28 Nov 2023 06:01:33 +0000
Subject: [QUESTION] Usage of negative number notations on unsigned
 numbers.
In-Reply-To: <1c25a9a7-708f-4840-9c7a-a4dedffb2b0a@incoresemi.com>
References: <PN2P287MB0754429FA98AFB8EFD2F4D4BFAB8A@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
 <878r6n76im.fsf@igel.home>
 <PN2P287MB0754BFB8B175A81159CBBF45FABDA@PN2P287MB0754.INDP287.PROD.OUTLOOK.COM>
 <87leajmm8l.fsf@igel.home>
 <1b2d1b62-e6d7-4de2-9d27-4892a0f2a9dd@incoresemi.com>
 <87h6l7mhqn.fsf@igel.home>
 <1c25a9a7-708f-4840-9c7a-a4dedffb2b0a@incoresemi.com>
Message-ID: <84CA5E33-E6C9-4DDD-A66E-423C966624C0@jrtc27.com>

On 28 Nov 2023, at 05:28, Karthik B K <karthik.bk at incoresemi.com> wrote:
> 
> On 27/11/23 19:16, Andreas Schwab wrote:
>> On Nov 27 2023, Karthik B K wrote:
>> 
>>> What's the encoding of (2^32-1)UL and (2^64-1)UL ?
>> A value does not have an encoding.  Only a memory location does.
>> 
> So does a register (container of bits), and a type (signed, unsigned, float).
> 
> The "encoding" is dependent on the "container" (reg or mem) and "type" (signed, unsigned, float).
> 
> The same bits (11...11) would reflect very different values (-1, 2^32-1, quiteNaN).
> 
> The container of (2^32-1) is UInt. 32b unsigned integer. -1 cannot be represented in a 32b unsigned int.
> 
> 
> I'm trying to say, that the unary operator (-) in the expression (-1UL) is an integer arithmetic operator.
> 
> The unary operator (-) when applied to an unsigned type, results in the _negative_ of it's (promoted as per _usual arithmetic conversions_) operand, which in this case, is a signed integer.
> 
> The result will retain this type. Are we assuming an implicit cast ?
> 
> This looks like an illegal set of operations to create the bit pattern we desire (11...11) for an unsigned int.
> 
> 
> Simply put, a -1 in a UL is an oxymoron. Don't you think ?
> 
> Please correct me if I'm wrong.

C99 6.2.5p9:

"The range of nonnegative values of a signed integer type is a subrange
of the corresponding unsigned integer type, and the representation of
the same value in each type is the same.31) A computation involving
unsigned operands can never overflow, because a result that cannot be
represented by the resulting unsigned integer type is reduced modulo
the number that is one greater than the largest value that can be
represented by the resulting type.?

-1UL (parsed as -(1UL); integer constants start with a digit in the
grammar, the - (or leading +) is always a normal unary operator) is
thus well-defined.

If you want to learn C, I?d rather not see it happen on this list.

Jess



From peterlin at andestech.com  Mon Nov 27 22:10:14 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 28 Nov 2023 14:10:14 +0800
Subject: [PATCH v3 04/15] sbi: sbi_pmu: Add hw_counter_filter_mode() to
 pmu device
In-Reply-To: <CAOnJCUK4_H=MeHsUUCDscVD+905k6uV9C3bUUpwuA+kTdsFYUw@mail.gmail.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
 <20231122073617.379441-5-peterlin@andestech.com>
 <CAOnJCUK4_H=MeHsUUCDscVD+905k6uV9C3bUUpwuA+kTdsFYUw@mail.gmail.com>
Message-ID: <ZWWERnFB6qoX6XQg@APC323>

Hi Atish,

On Wed, Nov 22, 2023 at 04:24:35PM -0800, Atish Patra wrote:
> On Tue, Nov 21, 2023 at 11:40?PM Yu Chien Peter Lin
> <peterlin at andestech.com> wrote:
> >
> > Add support for custom PMU extensions to set inhibit bits
> > on custom CSRs by introducing the PMU device callback
> > hw_counter_filter_mode(). This allows the perf tool to
> > restrict event counting under a specified privileged
> > mode by appending a modifier, e.g. perf record -e event:k
> > to count events only happening in kernel mode.
> >
> > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> > ---
> > Changes v1 -> v2:
> >   - No change
> > Changes v2 -> v3:
> >   - Add pmu_dev->hw_counter_filter_mode() in pmu_fixed_ctr_update_inhibit_bits()
> > ---
> >  include/sbi/sbi_pmu.h |  6 ++++++
> >  lib/sbi/sbi_pmu.c     | 20 ++++++++++++++------
> >  2 files changed, 20 insertions(+), 6 deletions(-)
> >
> > diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
> > index 16f6877..d63149c 100644
> > --- a/include/sbi/sbi_pmu.h
> > +++ b/include/sbi/sbi_pmu.h
> > @@ -89,6 +89,12 @@ struct sbi_pmu_device {
> >          * Custom function returning the machine-specific irq-bit.
> >          */
> >         int (*hw_counter_irq_bit)(void);
> > +
> > +       /**
> > +        * Custom function to inhibit counting of events while in
> > +        * specified mode.
> > +        */
> > +       void (*hw_counter_filter_mode)(unsigned long flags, int counter_index);
> >  };
> >
> >  /** Get the PMU platform device */
> > diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> > index 3cbd4ff..2f255de 100644
> > --- a/lib/sbi/sbi_pmu.c
> > +++ b/lib/sbi/sbi_pmu.c
> > @@ -599,7 +599,10 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,
> >                 pmu_dev->hw_counter_disable_irq(ctr_idx);
> >
> >         /* Update the inhibit flags based on inhibit flags received from supervisor */
> > -       pmu_update_inhibit_flags(flags, &mhpmevent_val);
> > +       if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
> > +               pmu_update_inhibit_flags(flags, &mhpmevent_val);
> > +       if (pmu_dev && pmu_dev->hw_counter_filter_mode)
> > +               pmu_dev->hw_counter_filter_mode(flags, ctr_idx);
> >
> >  #if __riscv_xlen == 32
> >         csr_write_num(CSR_MHPMEVENT3 + ctr_idx - 3, mhpmevent_val & 0xFFFFFFFF);
> > @@ -620,7 +623,8 @@ static int pmu_fixed_ctr_update_inhibit_bits(int fixed_ctr, unsigned long flags)
> >  #if __riscv_xlen == 32
> >         uint64_t cfgh_csr_no;
> >  #endif
> > -       if (!sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF))
> > +       if (!sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF) &&
> > +               !sbi_hart_has_extension(scratch, SBI_HART_EXT_XANDESPMU))
> 
> Instead of adding a check for vendor extension, we can just check for
> platform specific hw_counter_filter_mode() availability.

Sure, will do.

Regards,
Peter Lin

> I would prefer to avoid any vendor specific code if possible. Ideally,
> all vendor specific details should be abstracted out via pmu-dev.
> 
> >                 return fixed_ctr;
> >
> >         switch (fixed_ctr) {
> > @@ -641,13 +645,17 @@ static int pmu_fixed_ctr_update_inhibit_bits(int fixed_ctr, unsigned long flags)
> >         }
> >
> >         cfg_val |= MHPMEVENT_MINH;
> > -       pmu_update_inhibit_flags(flags, &cfg_val);
> > +       if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF)) {
> > +               pmu_update_inhibit_flags(flags, &cfg_val);
> >  #if __riscv_xlen == 32
> > -       csr_write_num(cfg_csr_no, cfg_val & 0xFFFFFFFF);
> > -       csr_write_num(cfgh_csr_no, cfg_val >> BITS_PER_LONG);
> > +               csr_write_num(cfg_csr_no, cfg_val & 0xFFFFFFFF);
> > +               csr_write_num(cfgh_csr_no, cfg_val >> BITS_PER_LONG);
> >  #else
> > -       csr_write_num(cfg_csr_no, cfg_val);
> > +               csr_write_num(cfg_csr_no, cfg_val);
> >  #endif
> > +       }
> > +       if (pmu_dev && pmu_dev->hw_counter_filter_mode)
> > +               pmu_dev->hw_counter_filter_mode(flags, fixed_ctr);
> >         return fixed_ctr;
> >  }
> >
> > --
> > 2.34.1
> >
> 
> 
> --
> Regards,
> Atish


From peterlin at andestech.com  Tue Nov 28 01:57:57 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 28 Nov 2023 17:57:57 +0800
Subject: [PATCH v3 10/15] lib: utils: fdt_fixup: Allow preserving PMU
 properties
In-Reply-To: <CAAhSdy2iMgXBJcN_OOju3cHTLa6n2hF58YnDK-X5iLAs6NWQKg@mail.gmail.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
 <20231122073617.379441-11-peterlin@andestech.com>
 <CAAhSdy2iMgXBJcN_OOju3cHTLa6n2hF58YnDK-X5iLAs6NWQKg@mail.gmail.com>
Message-ID: <ZWW5pev4P_JxWWyQ@APC323>

Hi Anup,

On Sat, Nov 25, 2023 at 10:12:06AM +0530, Anup Patel wrote:
> On Wed, Nov 22, 2023 at 1:10?PM Yu Chien Peter Lin
> <peterlin at andestech.com> wrote:
> >
> > Add a scratch option to control PMU fixup, so the next
> > stage software can dump the PMU node including event
> > mapping information for debugging purposes.
> >
> > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > Reviewed-by: Anup Patel <anup at brainfault.org>
> > ---
> > Changes v1 -> v2:
> >   - New patch
> > Changes v2 -> v3:
> >   - Rename to SBI_SCRATCH_PRESERVE_PMU_NODE (suggested by Anup)
> >   - Include Anup's RB tag
> > ---
> >  include/sbi/sbi_scratch.h | 2 ++
> >  lib/utils/fdt/fdt_fixup.c | 6 +++++-
> >  2 files changed, 7 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/sbi/sbi_scratch.h b/include/sbi/sbi_scratch.h
> > index e6a33ba..0f67cde 100644
> > --- a/include/sbi/sbi_scratch.h
> > +++ b/include/sbi/sbi_scratch.h
> > @@ -151,6 +151,8 @@ enum sbi_scratch_options {
> >         SBI_SCRATCH_NO_BOOT_PRINTS = (1 << 0),
> >         /** Enable runtime debug prints */
> >         SBI_SCRATCH_DEBUG_PRINTS = (1 << 1),
> > +       /** Preserve PMU node properties */
> > +       SBI_SCRATCH_PRESERVE_PMU_NODE = (1 << 2),
> 
> I have second thoughts about using the scratch option for this.
> 
> Let's not waste a bit in the scratch option just for debugging the PMU node.
> 
> I suggest adding the kconfig option FDT_FIXUPS_PRESERVE_PMU_NODE
> in <opensbi>lib/utils/fdt/Kconfig and use it in fdt_fixups() function.

OK, will do.

Best regards,
Peter Lin

> Regards,
> Anup
> 
> >  };
> >
> >  /** Get pointer to sbi_scratch for current HART */
> > diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> > index e213ded..cf20edf 100644
> > --- a/lib/utils/fdt/fdt_fixup.c
> > +++ b/lib/utils/fdt/fdt_fixup.c
> > @@ -387,6 +387,8 @@ int fdt_reserved_memory_fixup(void *fdt)
> >
> >  void fdt_fixups(void *fdt)
> >  {
> > +       struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> > +
> >         fdt_aplic_fixup(fdt);
> >
> >         fdt_imsic_fixup(fdt);
> > @@ -394,5 +396,7 @@ void fdt_fixups(void *fdt)
> >         fdt_plic_fixup(fdt);
> >
> >         fdt_reserved_memory_fixup(fdt);
> > -       fdt_pmu_fixup(fdt);
> > +
> > +       if (!(scratch->options & SBI_SCRATCH_PRESERVE_PMU_NODE))
> > +               fdt_pmu_fixup(fdt);
> >  }
> > --
> > 2.34.1
> >


From peterlin at andestech.com  Tue Nov 28 03:02:32 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 28 Nov 2023 19:02:32 +0800
Subject: [PATCH v3 07/15] platform: andes: Add Andes custom PMU support
In-Reply-To: <70a378e1-484c-4b77-942a-169a99ad00b5@sifive.com>
References: <20231122073617.379441-1-peterlin@andestech.com>
 <20231122073617.379441-8-peterlin@andestech.com>
 <70a378e1-484c-4b77-942a-169a99ad00b5@sifive.com>
Message-ID: <ZWXIyJow9xC8adRf@APC323>

Hi Samuel,

Thanks for the review.

On Wed, Nov 22, 2023 at 08:24:49PM -0600, Samuel Holland wrote:
> On 2023-11-22 1:36 AM, Yu Chien Peter Lin wrote:
> > Before the ratification of Sscofpmf, the Andes PMU extension
> > was designed to support the sampling and filtering with hardware
> > performance counters (zihpm), it works with the current SBI PMU
> > extension and Linux SBI PMU driver.
> > 
> > We implement 1) the PMU device callbacks that update the
> > corresponding bits on custom CSRs, 2) extentions_init() to detect
> > the hardware support of Andes PMU and initialize the per-hart
> > PMU related CSR, and 3) pmu_init() to register PMU device and
> > populate event mappings (only called by coldboot hart).
> > 
> > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
> > ---
> > Changes v1 -> v2:
> >   - Fix mode filtering in andes_hw_counter_filter_mode()
> >   - Return early if pmu is not supported in andes_pmu_init() (suggested by Prabhakar)
> >   - Don't grant write permissions via CSR_MCOUNTERWEN as not needed
> > Changes v2 -> v3:
> >   - Drop Anup's RB tag as we add andes_pmu_extensions_init() to initialize per-hart
> >     extension on scratch and move andes_pmu_init() to new added platform override pmu_init()
> > --- 
> >  platform/generic/andes/Kconfig             |   8 ++
> >  platform/generic/andes/andes_pmu.c         | 102 +++++++++++++++++++++
> >  platform/generic/andes/objects.mk          |   1 +
> >  platform/generic/include/andes/andes_pmu.h |  33 +++++++
> >  4 files changed, 144 insertions(+)
> >  create mode 100644 platform/generic/andes/andes_pmu.c
> >  create mode 100644 platform/generic/include/andes/andes_pmu.h
> > 
> > diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
> > index a91fb9c..5b2ed91 100644
> > --- a/platform/generic/andes/Kconfig
> > +++ b/platform/generic/andes/Kconfig
> > @@ -7,3 +7,11 @@ config ANDES45_PMA
> >  config ANDES_SBI
> >  	bool "Andes SBI support"
> >  	default n
> > +
> > +config ANDES_PMU
> > +	bool "Andes PMU extension (xandespmu) support"
> > +	default n
> > +	help
> > +	  Andes PMU extension supports the event counter overflow
> > +	  interrupt and mode filtering, similar to the standard
> > +	  Sscofpmf and Smcntrpmf.
> > diff --git a/platform/generic/andes/andes_pmu.c b/platform/generic/andes/andes_pmu.c
> > new file mode 100644
> > index 0000000..72003fd
> > --- /dev/null
> > +++ b/platform/generic/andes/andes_pmu.c
> > @@ -0,0 +1,102 @@
> > +// SPDX-License-Identifier: BSD-2-Clause
> > +/*
> > + * andes_pmu.c - Andes PMU device callbacks and platform overrides
> > + *
> > + * Copyright (C) 2023 Andes Technology Corporation
> > + */
> > +
> > +#include <andes/andes45.h>
> > +#include <andes/andes_hpm.h>
> > +#include <andes/andes_pmu.h>
> > +#include <sbi/sbi_bitops.h>
> > +#include <sbi/sbi_error.h>
> > +#include <sbi/sbi_pmu.h>
> > +
> > +static void andes_hw_counter_enable_irq(uint32_t ctr_idx)
> > +{
> > +	unsigned long mip_val;
> > +
> > +	if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
> > +		return;
> > +
> > +	mip_val = csr_read(CSR_MIP);
> > +	if (!(mip_val & MIP_PMOVI))
> > +		csr_clear(CSR_MCOUNTEROVF, BIT(ctr_idx));
> > +
> > +	csr_set(CSR_MCOUNTERINTEN, BIT(ctr_idx));
> > +}
> > +
> > +static void andes_hw_counter_disable_irq(uint32_t ctr_idx)
> > +{
> > +	csr_clear(CSR_MCOUNTERINTEN, BIT(ctr_idx));
> > +}
> > +
> > +static void andes_hw_counter_filter_mode(unsigned long flags, int ctr_idx)
> > +{
> > +	if (flags & SBI_PMU_CFG_FLAG_SET_UINH)
> > +		csr_set(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
> > +	else
> > +		csr_clear(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
> > +
> > +	if (flags & SBI_PMU_CFG_FLAG_SET_SINH)
> > +		csr_set(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
> > +	else
> > +		csr_clear(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
> > +}
> > +
> > +static struct sbi_pmu_device andes_pmu = {
> > +	.name = "andes_pmu",
> > +	.hw_counter_enable_irq  = andes_hw_counter_enable_irq,
> > +	.hw_counter_disable_irq = andes_hw_counter_disable_irq,
> > +	/*
> > +	 * We set delegation of supervisor local interrupts via
> > +	 * 18th bit on mslideleg instead of mideleg, so leave
> > +	 * hw_counter_irq_bit() callback unimplemented.
> > +	 */
> > +	.hw_counter_irq_bit     = NULL,
> > +	.hw_counter_filter_mode = andes_hw_counter_filter_mode
> > +};
> > +
> > +int andes_pmu_extensions_init(const struct fdt_match *match,
> > +			      struct sbi_hart_features *hfeatures)
> > +{
> > +	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> > +
> > +	if (!has_andes_pmu())
> > +		return 0;
> > +
> > +	/*
> > +	 * Don't expect both Andes PMU and standard Sscofpmf/Smcntrpmf,
> > +	 * are supported as they serve the same purpose.
> > +	 */
> > +	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF) ||
> > +		sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF))
> > +		return SBI_EINVAL;
> 
> If Sscofpmf is supported, don't you want to successfully do nothing?

If Sscofpmf is supported, it will return early as has_andes_pmu() above
is false.

> > +	sbi_hart_update_extension(scratch, SBI_HART_EXT_XANDESPMU, true);
> > +
> > +	/* Inhibit all HPM counters in M-mode */
> > +	csr_write(CSR_MCOUNTERMASK_M, 0xfffffffd);
> > +	/* Delegate counter overflow interrupt to S-mode */
> > +	csr_write(CSR_MSLIDELEG, MIP_PMOVI);
> > +
> > +	return 0;
> > +}
> > +
> > +int andes_pmu_init(const struct fdt_match *match)
> > +{
> > +	int rc;
> > +	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> > +
> > +	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_XANDESPMU))
> > +		sbi_pmu_set_device(&andes_pmu);
> > +
> > +	rc = fdt_pmu_setup(fdt_get_address());
> 
> generic_pmu_init() already calls fdt_pmu_setup().

OK will remove fdt_pmu_setup() here.

> > +	/*
> > +	 * Populate default mappings if device-tree doesn't
> > +	 * provide a valid pmu node.
> > +	 */
> > +	if (rc == SBI_ENOENT)
> > +		return andes_pmu_setup();
> 
> This function isn't added until patch 14, so this won't compile. Please make
> sure each patch compiles individually.

Sure, thank you for catching this.

Regards,
Peter Lin

> Regards,
> Samuel
> 


From wxjstz at 126.com  Wed Nov 29 05:57:00 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed, 29 Nov 2023 21:57:00 +0800
Subject: [PATCH] lib: sbi: skip wait_for_coldboot when coolboot done
Message-ID: <20231129135717.310197-1-wxjstz@126.com>

When warmboot via HSM, coolboot has been completed and
wait_for_coldboot can be skipped to speed up.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_init.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
index e723553..1f7f522 100644
--- a/lib/sbi/sbi_init.c
+++ b/lib/sbi/sbi_init.c
@@ -492,7 +492,8 @@ static void __noreturn init_warmboot(struct sbi_scratch *scratch, u32 hartid)
 {
 	int hstate;
 
-	wait_for_coldboot(scratch, hartid);
+	if (!__smp_load_acquire(&coldboot_done))
+		wait_for_coldboot(scratch, hartid);
 
 	hstate = sbi_hsm_hart_get_state(sbi_domain_thishart_ptr(), hartid);
 	if (hstate < 0)
-- 
2.42.0



From cleger at rivosinc.com  Thu Nov 30 01:13:17 2023
From: cleger at rivosinc.com (=?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?=)
Date: Thu, 30 Nov 2023 10:13:17 +0100
Subject: [RFC PATCH 0/2] Add support for Supervisor Software Events extension
Message-ID: <20231130091319.1223849-1-cleger@rivosinc.com>

The SBI Supervisor Software Events (SSE) extensions provides a mechanism
to inject software events from an SBI implementation to supervisor
software such that it preempts all other supervisor level traps and
interrupts [1].

Various events are defined and can be send asynchronously to supervisor
software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
as platform specific events. Events can be either local (per-hart) or
global. Events can be nested on top of each other based on priority and
can interrupt the supervisor mode at any time.

This PR adds support for this extension. First commit modifies interrupts
handling to pass the sbi traps regs through interrupt handlers in order to inject
SSE events from this context. Second one add the SSE support itself.

A RFC for Linux which uses this extension is available at [2].

Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
Link: https://lore.kernel.org/linux-riscv/20231026143122.279437-1-cleger at rivosinc.com/ [2]

Cl?ment L?ger (2):
  lib: sbi: provides regs to sbi_ipi_process()
  lib: sbi: add support for Supervisor Software Events extension

 include/sbi/sbi_ecall_interface.h |  36 +-
 include/sbi/sbi_error.h           |   4 +
 include/sbi/sbi_ipi.h             |   6 +-
 include/sbi/sbi_sse.h             | 222 +++++++
 lib/sbi/Kconfig                   |   4 +
 lib/sbi/objects.mk                |   4 +
 lib/sbi/sbi_ecall.c               |   7 +-
 lib/sbi/sbi_ecall_sse.c           |  61 ++
 lib/sbi/sbi_init.c                |  13 +
 lib/sbi/sbi_ipi.c                 |  12 +-
 lib/sbi/sbi_sse.c                 | 963 ++++++++++++++++++++++++++++++
 lib/sbi/sbi_tlb.c                 |   2 +-
 lib/sbi/sbi_trap.c                |   4 +-
 lib/utils/irqchip/imsic.c         |   2 +-
 14 files changed, 1327 insertions(+), 13 deletions(-)
 create mode 100644 include/sbi/sbi_sse.h
 create mode 100644 lib/sbi/sbi_ecall_sse.c
 create mode 100644 lib/sbi/sbi_sse.c

-- 
2.42.0



From cleger at rivosinc.com  Thu Nov 30 01:13:18 2023
From: cleger at rivosinc.com (=?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?=)
Date: Thu, 30 Nov 2023 10:13:18 +0100
Subject: [RFC PATCH 1/2] lib: sbi: provides regs to sbi_ipi_process()
In-Reply-To: <20231130091319.1223849-1-cleger@rivosinc.com>
References: <20231130091319.1223849-1-cleger@rivosinc.com>
Message-ID: <20231130091319.1223849-2-cleger@rivosinc.com>

In order to implement SSE on IPI notifications, provides regs to
sbi_ipi_process(). This will be used in two cases, global event
triggered from one cpu to another and ipi injection to a specific hart.

Signed-off-by: Cl?ment L?ger <cleger at rivosinc.com>
---
 include/sbi/sbi_ipi.h     |  6 ++++--
 lib/sbi/sbi_ipi.c         | 12 +++++++-----
 lib/sbi/sbi_tlb.c         |  2 +-
 lib/sbi/sbi_trap.c        |  4 ++--
 lib/utils/irqchip/imsic.c |  2 +-
 5 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
index d396233..0b32194 100644
--- a/include/sbi/sbi_ipi.h
+++ b/include/sbi/sbi_ipi.h
@@ -11,6 +11,7 @@
 #define __SBI_IPI_H__
 
 #include <sbi/sbi_types.h>
+#include <sbi/sbi_trap.h>
 
 /* clang-format off */
 
@@ -68,7 +69,8 @@ struct sbi_ipi_event_ops {
 	 * Note: This is a mandatory callback and it is called on the
 	 * remote HART after IPI is triggered.
 	 */
-	void (* process)(struct sbi_scratch *scratch);
+	void (* process)(struct sbi_scratch *scratch,
+			 struct sbi_trap_regs *regs);
 };
 
 int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data);
@@ -83,7 +85,7 @@ void sbi_ipi_clear_smode(void);
 
 int sbi_ipi_send_halt(ulong hmask, ulong hbase);
 
-void sbi_ipi_process(void);
+void sbi_ipi_process(struct sbi_trap_regs *regs);
 
 int sbi_ipi_raw_send(u32 hartindex);
 
diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 0bf446a..7fa6113 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -170,7 +170,8 @@ void sbi_ipi_event_destroy(u32 event)
 	ipi_ops_array[event] = NULL;
 }
 
-static void sbi_ipi_process_smode(struct sbi_scratch *scratch)
+static void sbi_ipi_process_smode(struct sbi_scratch *scratch,
+				  struct sbi_trap_regs *regs)
 {
 	csr_set(CSR_MIP, MIP_SSIP);
 }
@@ -192,7 +193,8 @@ void sbi_ipi_clear_smode(void)
 	csr_clear(CSR_MIP, MIP_SSIP);
 }
 
-static void sbi_ipi_process_halt(struct sbi_scratch *scratch)
+static void sbi_ipi_process_halt(struct sbi_scratch *scratch,
+				 struct sbi_trap_regs *regs)
 {
 	sbi_hsm_hart_stop(scratch, true);
 }
@@ -209,7 +211,7 @@ int sbi_ipi_send_halt(ulong hmask, ulong hbase)
 	return sbi_ipi_send_many(hmask, hbase, ipi_halt_event, NULL);
 }
 
-void sbi_ipi_process(void)
+void sbi_ipi_process(struct sbi_trap_regs *regs)
 {
 	unsigned long ipi_type;
 	unsigned int ipi_event;
@@ -228,7 +230,7 @@ void sbi_ipi_process(void)
 		if (ipi_type & 1UL) {
 			ipi_ops = ipi_ops_array[ipi_event];
 			if (ipi_ops && ipi_ops->process)
-				ipi_ops->process(scratch);
+				ipi_ops->process(scratch, regs);
 		}
 		ipi_type = ipi_type >> 1;
 		ipi_event++;
@@ -333,7 +335,7 @@ void sbi_ipi_exit(struct sbi_scratch *scratch)
 	csr_clear(CSR_MIE, MIP_MSIP);
 
 	/* Process pending IPIs */
-	sbi_ipi_process();
+	sbi_ipi_process(NULL);
 
 	/* Platform exit */
 	sbi_platform_ipi_exit(sbi_platform_ptr(scratch));
diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index dad9508..80e7bdf 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -229,7 +229,7 @@ static bool tlb_process_once(struct sbi_scratch *scratch)
 	return false;
 }
 
-static void tlb_process(struct sbi_scratch *scratch)
+static void tlb_process(struct sbi_scratch *scratch, struct sbi_trap_regs *regs)
 {
 	while (tlb_process_once(scratch));
 }
diff --git a/lib/sbi/sbi_trap.c b/lib/sbi/sbi_trap.c
index dbf307c..d574ab1 100644
--- a/lib/sbi/sbi_trap.c
+++ b/lib/sbi/sbi_trap.c
@@ -206,7 +206,7 @@ static int sbi_trap_nonaia_irq(struct sbi_trap_regs *regs, ulong mcause)
 		sbi_timer_process();
 		break;
 	case IRQ_M_SOFT:
-		sbi_ipi_process();
+		sbi_ipi_process(regs);
 		break;
 	case IRQ_M_EXT:
 		return sbi_irqchip_process(regs);
@@ -229,7 +229,7 @@ static int sbi_trap_aia_irq(struct sbi_trap_regs *regs, ulong mcause)
 			sbi_timer_process();
 			break;
 		case IRQ_M_SOFT:
-			sbi_ipi_process();
+			sbi_ipi_process(regs);
 			break;
 		case IRQ_M_EXT:
 			rc = sbi_irqchip_process(regs);
diff --git a/lib/utils/irqchip/imsic.c b/lib/utils/irqchip/imsic.c
index 36ef66c..a207dbc 100644
--- a/lib/utils/irqchip/imsic.c
+++ b/lib/utils/irqchip/imsic.c
@@ -149,7 +149,7 @@ static int imsic_external_irqfn(struct sbi_trap_regs *regs)
 
 		switch (mirq) {
 		case IMSIC_IPI_ID:
-			sbi_ipi_process();
+			sbi_ipi_process(regs);
 			break;
 		default:
 			sbi_printf("%s: unhandled IRQ%d\n",
-- 
2.42.0



From cleger at rivosinc.com  Thu Nov 30 01:13:19 2023
From: cleger at rivosinc.com (=?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?=)
Date: Thu, 30 Nov 2023 10:13:19 +0100
Subject: [RFC PATCH 2/2] lib: sbi: add support for Supervisor Software Events extension
In-Reply-To: <20231130091319.1223849-1-cleger@rivosinc.com>
References: <20231130091319.1223849-1-cleger@rivosinc.com>
Message-ID: <20231130091319.1223849-3-cleger@rivosinc.com>

This extension [1] allows to deliver events from SBI to supervisor via a
software mecanism. This extensions defines events (either local or
global) which are signaled by the SBI on specific signal sources (IRQ,
traps, etc) and are injected to be executed in supervisor mode.

[1] https://lists.riscv.org/g/tech-prs/message/515

Signed-off-by: Cl?ment L?ger <cleger at rivosinc.com>
---
 include/sbi/sbi_ecall_interface.h |  36 +-
 include/sbi/sbi_error.h           |   4 +
 include/sbi/sbi_sse.h             | 222 +++++++
 lib/sbi/Kconfig                   |   4 +
 lib/sbi/objects.mk                |   4 +
 lib/sbi/sbi_ecall.c               |   7 +-
 lib/sbi/sbi_ecall_sse.c           |  61 ++
 lib/sbi/sbi_init.c                |  13 +
 lib/sbi/sbi_sse.c                 | 963 ++++++++++++++++++++++++++++++
 9 files changed, 1312 insertions(+), 2 deletions(-)
 create mode 100644 include/sbi/sbi_sse.h
 create mode 100644 lib/sbi/sbi_ecall_sse.c
 create mode 100644 lib/sbi/sbi_sse.c

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index 1fe469e..f510039 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -32,6 +32,7 @@
 #define SBI_EXT_DBCN				0x4442434E
 #define SBI_EXT_SUSP				0x53555350
 #define SBI_EXT_CPPC				0x43505043
+#define SBI_EXT_SSE				0x535345
 
 /* SBI function IDs for BASE extension*/
 #define SBI_EXT_BASE_GET_SPEC_VERSION		0x0
@@ -290,6 +291,36 @@ enum sbi_cppc_reg_id {
 	SBI_CPPC_NON_ACPI_LAST		= SBI_CPPC_TRANSITION_LATENCY,
 };
 
+/* SBI Function IDs for SSE extension */
+#define SBI_EXT_SSE_GET_ATTR		0x00000000
+#define SBI_EXT_SSE_SET_ATTR		0x00000001
+#define SBI_EXT_SSE_REGISTER		0x00000002
+#define SBI_EXT_SSE_UNREGISTER		0x00000003
+#define SBI_EXT_SSE_ENABLE		0x00000004
+#define SBI_EXT_SSE_DISABLE		0x00000005
+#define SBI_EXT_SSE_COMPLETE		0x00000006
+#define SBI_EXT_SSE_INJECT		0x00000007
+
+/* SBI SSE Event Attributes. */
+#define SBI_SSE_ATTR_STATE		0x00000000
+#define SBI_SSE_ATTR_PRIO		0x00000001
+#define SBI_SSE_ATTR_ALLOW_INJECT	0x00000002
+#define SBI_SSE_ATTR_HART_ID		0x00000003
+#define SBI_SSE_ATTR_PENDING		0x00000004
+
+/* SBI SSE Event IDs. */
+#define SBI_SSE_EVENT_LOCAL_RAS		0x00000000
+#define SBI_SSE_EVENT_GLOBAL_RAS	0x00008000
+#define SBI_SSE_EVENT_LOCAL_ASYNC_PF	0x00010000
+#define SBI_SSE_EVENT_LOCAL_PMU		0x00010001
+#define SBI_SSE_EVENT_LOCAL_DEBUG      	0xffff3fff
+#define SBI_SSE_EVENT_GLOBAL_DEBUG	0xffffbfff
+
+#define SBI_SSE_EVENT_GLOBAL		(1 << 15)
+#define SBI_SSE_EVENT_PLATFORM		(1 << 14)
+
+#define SBI_SSE_COMPLETE_FLAG_EVENT_DISABLE	(1 << 0)
+
 /* SBI base specification related macros */
 #define SBI_SPEC_VERSION_MAJOR_OFFSET		24
 #define SBI_SPEC_VERSION_MAJOR_MASK		0x7f
@@ -309,8 +340,11 @@ enum sbi_cppc_reg_id {
 #define SBI_ERR_ALREADY_AVAILABLE		-6
 #define SBI_ERR_ALREADY_STARTED			-7
 #define SBI_ERR_ALREADY_STOPPED			-8
+#define SBI_ERR_INVALID_STATE			-10
+#define SBI_ERR_BAD_RANGE			-11
+#define SBI_ERR_BUSY				-12
 
-#define SBI_LAST_ERR				SBI_ERR_ALREADY_STOPPED
+#define SBI_LAST_ERR				SBI_ERR_BUSY
 
 /* clang-format on */
 
diff --git a/include/sbi/sbi_error.h b/include/sbi/sbi_error.h
index dd65e14..3cff248 100644
--- a/include/sbi/sbi_error.h
+++ b/include/sbi/sbi_error.h
@@ -23,6 +23,9 @@
 #define SBI_EALREADY		SBI_ERR_ALREADY_AVAILABLE
 #define SBI_EALREADY_STARTED	SBI_ERR_ALREADY_STARTED
 #define SBI_EALREADY_STOPPED	SBI_ERR_ALREADY_STOPPED
+#define SBI_EINVALID_STATE	SBI_ERR_INVALID_STATE
+#define SBI_EBAD_RANGE		SBI_ERR_BAD_RANGE
+#define SBI_EBUSY		SBI_ERR_BUSY
 
 #define SBI_ENODEV		-1000
 #define SBI_ENOSYS		-1001
@@ -34,6 +37,7 @@
 #define SBI_ETRAP		-1007
 #define SBI_EUNKNOWN		-1008
 #define SBI_ENOENT		-1009
+#define SBI_EJUMP		-1010
 
 /* clang-format on */
 
diff --git a/include/sbi/sbi_sse.h b/include/sbi/sbi_sse.h
new file mode 100644
index 0000000..e3a49a5
--- /dev/null
+++ b/include/sbi/sbi_sse.h
@@ -0,0 +1,222 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Rivos Systems.
+ */
+
+#ifndef __SBI_SSE_H__
+#define __SBI_SSE_H__
+
+#include <sbi/sbi_types.h>
+#include <sbi/sbi_list.h>
+#include <sbi/riscv_locks.h>
+
+struct sbi_scratch;
+
+#define EXC_MODE_PP_SHIFT		0
+#define EXC_MODE_PP			BIT(EXC_MODE_PP_SHIFT)
+#define EXC_MODE_PV_SHIFT		1
+#define EXC_MODE_PV			BIT(EXC_MODE_PV_SHIFT)
+#define EXC_MODE_SSTATUS_SPIE_SHIFT	2
+#define EXC_MODE_SSTATUS_SPIE		BIT(EXC_MODE_SSTATUS_SPIE_SHIFT)
+
+struct sse_entry_state {
+	/** Entry program counter */
+	unsigned long pc;
+	/** ra register state */
+	unsigned long ra;
+	/** sp register state */
+	unsigned long sp;
+	/** gp register state */
+	unsigned long gp;
+	/** tp register state */
+	unsigned long tp;
+	/** t0 register state */
+	unsigned long t0;
+	/** t1 register state */
+	unsigned long t1;
+	/** t2 register state */
+	unsigned long t2;
+	/** s0 register state */
+	unsigned long s0;
+	/** s1 register state */
+	unsigned long s1;
+	/** a0 register state */
+	unsigned long a0;
+	/** a1 register state */
+	unsigned long a1;
+	/** a2 register state */
+	unsigned long a2;
+	/** a3 register state */
+	unsigned long a3;
+	/** a4 register state */
+	unsigned long a4;
+	/** a5 register state */
+	unsigned long a5;
+	/** a6 register state */
+	unsigned long a6;
+	/** a7 register state */
+	unsigned long a7;
+	/** s2 register state */
+	unsigned long s2;
+	/** s3 register state */
+	unsigned long s3;
+	/** s4 register state */
+	unsigned long s4;
+	/** s5 register state */
+	unsigned long s5;
+	/** s6 register state */
+	unsigned long s6;
+	/** s7 register state */
+	unsigned long s7;
+	/** s8 register state */
+	unsigned long s8;
+	/** s9 register state */
+	unsigned long s9;
+	/** s10 register state */
+	unsigned long s10;
+	/** s11 register state */
+	unsigned long s11;
+	/** t3 register state */
+	unsigned long t3;
+	/** t4 register state */
+	unsigned long t4;
+	/** t5 register state */
+	unsigned long t5;
+	/** t6 register state */
+	unsigned long t6;
+};
+
+struct sse_interrupted_state {
+	/** Interrupted program counter */
+	unsigned long pc;
+	/** ra register state */
+	unsigned long ra;
+	/** sp register state */
+	unsigned long sp;
+	/** gp register state */
+	unsigned long gp;
+	/** tp register state */
+	unsigned long tp;
+	/** t0 register state */
+	unsigned long t0;
+	/** t1 register state */
+	unsigned long t1;
+	/** t2 register state */
+	unsigned long t2;
+	/** s0 register state */
+	unsigned long s0;
+	/** s1 register state */
+	unsigned long s1;
+	/** a0 register state */
+	unsigned long a0;
+	/** a1 register state */
+	unsigned long a1;
+	/** a2 register state */
+	unsigned long a2;
+	/** a3 register state */
+	unsigned long a3;
+	/** a4 register state */
+	unsigned long a4;
+	/** a5 register state */
+	unsigned long a5;
+	/** a6 register state */
+	unsigned long a6;
+	/** a7 register state */
+	unsigned long a7;
+	/** s2 register state */
+	unsigned long s2;
+	/** s3 register state */
+	unsigned long s3;
+	/** s4 register state */
+	unsigned long s4;
+	/** s5 register state */
+	unsigned long s5;
+	/** s6 register state */
+	unsigned long s6;
+	/** s7 register state */
+	unsigned long s7;
+	/** s8 register state */
+	unsigned long s8;
+	/** s9 register state */
+	unsigned long s9;
+	/** s10 register state */
+	unsigned long s10;
+	/** s11 register state */
+	unsigned long s11;
+	/** t3 register state */
+	unsigned long t3;
+	/** t4 register state */
+	unsigned long t4;
+	/** t5 register state */
+	unsigned long t5;
+	/** t6 register state */
+	unsigned long t6;
+	/** Exception mode */
+	unsigned long exc_mode;
+};
+
+struct sbi_sse_handler_ctx {
+	struct sse_entry_state entry;
+	struct sse_interrupted_state interrupted;
+};
+
+enum sbi_sse_state {
+	SSE_STATE_UNUSED = 0,
+	SSE_STATE_REGISTERED = 1,
+	SSE_STATE_ENABLED = 2,
+	SSE_STATE_RUNNING = 3,
+};
+
+struct sbi_sse_cb_ops {
+	/**
+	 * Called when hart_id is changed on the event.
+	 */
+	void (*set_hartid_cb)(uint32_t event_id, unsigned long hart_id);
+
+	/**
+	 * Called when the SBI_EXT_SSE_COMPLETE is invoked on the event.
+	 */
+	void (*complete_cb)(uint32_t event_id);
+
+	/**
+	 * Called when the SBI_EXT_SSE_REGISTER is invoked on the event.
+	 */
+	void (*register_cb)(uint32_t event_id);
+
+	/**
+	 * Called when the SBI_EXT_SSE_UNREGISTER is invoked on the event.
+	 */
+	void (*unregister_cb)(uint32_t event_id);
+};
+
+/* Set the callback operations for an event
+ * @param event_id Event identifier (SBI_SSE_EVENT_*)
+ * @param cb_ops Callback operations
+ * @return 0 on success, error otherwise
+ */
+int sbi_sse_set_cb_ops(uint32_t event_id, const struct sbi_sse_cb_ops *cb_ops);
+
+/* Inject an event to the current hard
+ * @param event_id Event identifier (SBI_SSE_EVENT_*)
+ * @param regs Registers that were used on SBI entry
+ * @return 0 on success, error otherwise
+ */
+int sbi_sse_inject_event(uint32_t event_id, struct sbi_trap_regs *regs);
+
+int sbi_sse_init(struct sbi_scratch *scratch, bool cold_boot);
+void sbi_sse_exit(struct sbi_scratch *scratch);
+
+/* Interface called from sbi_ecall_sse.c */
+int sbi_sse_get_attr(uint32_t event_id, uint32_t attr_id, unsigned long *out_val);
+int sbi_sse_set_attr(uint32_t event_id, uint32_t attr_id, unsigned long value);
+int sbi_sse_register(uint32_t event_id, unsigned long phys_hi, unsigned long phys_lo);
+int sbi_sse_unregister(uint32_t event_id);
+int sbi_sse_enable(uint32_t event_id);
+int sbi_sse_disable(uint32_t event_id);
+int sbi_sse_complete(uint32_t event_id, uint32_t status, uint32_t flags,
+		     struct sbi_trap_regs *regs);
+int sbi_sse_inject_from_ecall(uint32_t event_id, unsigned long hart_id,
+			      struct sbi_trap_regs *regs);
+
+#endif
diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
index 477775e..1b713e9 100644
--- a/lib/sbi/Kconfig
+++ b/lib/sbi/Kconfig
@@ -46,4 +46,8 @@ config SBI_ECALL_VENDOR
 	bool "Platform-defined vendor extensions"
 	default y
 
+config SBI_ECALL_SSE
+       bool "SSE extension"
+       default y
+
 endmenu
diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
index c699187..011c824 100644
--- a/lib/sbi/objects.mk
+++ b/lib/sbi/objects.mk
@@ -52,6 +52,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_LEGACY) += sbi_ecall_legacy.o
 carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_VENDOR) += ecall_vendor
 libsbi-objs-$(CONFIG_SBI_ECALL_VENDOR) += sbi_ecall_vendor.o
 
+carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_SSE) += ecall_sse
+libsbi-objs-$(CONFIG_SBI_ECALL_SSE) += sbi_ecall_sse.o
+
 libsbi-objs-y += sbi_bitmap.o
 libsbi-objs-y += sbi_bitops.o
 libsbi-objs-y += sbi_console.o
@@ -71,6 +74,7 @@ libsbi-objs-y += sbi_misaligned_ldst.o
 libsbi-objs-y += sbi_platform.o
 libsbi-objs-y += sbi_pmu.o
 libsbi-objs-y += sbi_scratch.o
+libsbi-objs-y += sbi_sse.o
 libsbi-objs-y += sbi_string.o
 libsbi-objs-y += sbi_system.o
 libsbi-objs-y += sbi_timer.o
diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
index 3eb4f0a..e7c1d1b 100644
--- a/lib/sbi/sbi_ecall.c
+++ b/lib/sbi/sbi_ecall.c
@@ -116,7 +116,12 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
 		ret = SBI_ENOTSUPP;
 	}
 
-	if (ret == SBI_ETRAP) {
+	if (ret == SBI_EJUMP) {
+		/* We don't want to modify register content since they could
+		 * have been modified by the handler.
+		 */
+		return 0;
+	} else if (ret == SBI_ETRAP) {
 		trap.epc = regs->mepc;
 		sbi_trap_redirect(regs, &trap);
 	} else {
diff --git a/lib/sbi/sbi_ecall_sse.c b/lib/sbi/sbi_ecall_sse.c
new file mode 100644
index 0000000..4ee35a2
--- /dev/null
+++ b/lib/sbi/sbi_ecall_sse.c
@@ -0,0 +1,61 @@
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_ecall.h>
+#include <sbi/sbi_trap.h>
+#include <sbi/sbi_sse.h>
+
+static int sbi_ecall_sse_handler(unsigned long extid, unsigned long funcid,
+				 const struct sbi_trap_regs *regs,
+				 unsigned long *out_val,
+				 struct sbi_trap_info *out_trap)
+{
+	int ret;
+	unsigned long temp;
+
+	switch (funcid) {
+	case SBI_EXT_SSE_GET_ATTR:
+		ret = sbi_sse_get_attr(regs->a0, regs->a1, &temp);
+		if (ret == 0)
+			*out_val = temp;
+		break;
+	case SBI_EXT_SSE_SET_ATTR:
+		ret = sbi_sse_set_attr(regs->a0, regs->a1, regs->a2);
+		break;
+	case SBI_EXT_SSE_REGISTER:
+		ret = sbi_sse_register(regs->a0, regs->a1, regs->a2);
+		break;
+	case SBI_EXT_SSE_UNREGISTER:
+		ret = sbi_sse_unregister(regs->a0);
+		break;
+	case SBI_EXT_SSE_ENABLE:
+		ret = sbi_sse_enable(regs->a0);
+		break;
+	case SBI_EXT_SSE_DISABLE:
+		ret = sbi_sse_disable(regs->a0);
+		break;
+	case SBI_EXT_SSE_COMPLETE:
+		ret = sbi_sse_complete(regs->a0, regs->a1, regs->a2,
+				       (struct sbi_trap_regs *) regs);
+		break;
+	case SBI_EXT_SSE_INJECT:
+		ret = sbi_sse_inject_from_ecall(regs->a0, regs->a1,
+						(struct sbi_trap_regs *) regs);
+		break;
+	default:
+		ret = SBI_ENOTSUPP;
+	}
+	return ret;
+}
+
+struct sbi_ecall_extension ecall_sse;
+
+static int sbi_ecall_sse_register_extensions(void)
+{
+	return sbi_ecall_register_extension(&ecall_sse);
+}
+
+struct sbi_ecall_extension ecall_sse = {
+	.extid_start		= SBI_EXT_SSE,
+	.extid_end		= SBI_EXT_SSE,
+	.register_extensions	= sbi_ecall_sse_register_extensions,
+	.handle			= sbi_ecall_sse_handler,
+};
diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
index e723553..5aa9992 100644
--- a/lib/sbi/sbi_init.c
+++ b/lib/sbi/sbi_init.c
@@ -23,6 +23,7 @@
 #include <sbi/sbi_irqchip.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_pmu.h>
+#include <sbi/sbi_sse.h>
 #include <sbi/sbi_system.h>
 #include <sbi/sbi_string.h>
 #include <sbi/sbi_timer.h>
@@ -312,6 +313,12 @@ static void __noreturn init_coldboot(struct sbi_scratch *scratch, u32 hartid)
 	if (rc)
 		sbi_hart_hang();
 
+        rc = sbi_sse_init(scratch, true);
+        if (rc) {
+                sbi_printf("%s: sse init failed (error %d)\n", __func__, rc);
+                sbi_hart_hang();
+        }
+
 	rc = sbi_pmu_init(scratch, true);
 	if (rc) {
 		sbi_printf("%s: pmu init failed (error %d)\n",
@@ -432,6 +439,10 @@ static void __noreturn init_warm_startup(struct sbi_scratch *scratch,
 	if (rc)
 		sbi_hart_hang();
 
+	rc = sbi_sse_init(scratch, false);
+	if (rc)
+		sbi_hart_hang();
+
 	rc = sbi_pmu_init(scratch, false);
 	if (rc)
 		sbi_hart_hang();
@@ -636,6 +647,8 @@ void __noreturn sbi_exit(struct sbi_scratch *scratch)
 
 	sbi_platform_early_exit(plat);
 
+	sbi_sse_exit(scratch);
+
 	sbi_pmu_exit(scratch);
 
 	sbi_timer_exit(scratch);
diff --git a/lib/sbi/sbi_sse.c b/lib/sbi/sbi_sse.c
new file mode 100644
index 0000000..7b6be7d
--- /dev/null
+++ b/lib/sbi/sbi_sse.c
@@ -0,0 +1,963 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Rivos Systems Inc.
+ *
+ */
+
+#include <sbi/riscv_asm.h>
+#include <sbi/riscv_barrier.h>
+#include <sbi/riscv_encoding.h>
+#include <sbi/riscv_locks.h>
+#include <sbi/sbi_domain.h>
+#include <sbi/sbi_ecall_interface.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_fifo.h>
+#include <sbi/sbi_heap.h>
+#include <sbi/sbi_hsm.h>
+#include <sbi/sbi_ipi.h>
+#include <sbi/sbi_list.h>
+#include <sbi/sbi_platform.h>
+#include <sbi/sbi_pmu.h>
+#include <sbi/sbi_sse.h>
+#include <sbi/sbi_scratch.h>
+#include <sbi/sbi_string.h>
+#include <sbi/sbi_trap.h>
+
+#include <sbi/sbi_console.h>
+
+#define sse_get_hart_state_ptr(__scratch)	\
+	sbi_scratch_read_type((__scratch), void *, shs_ptr_off)
+
+#define sse_thishart_state_ptr()		\
+	sse_get_hart_state_ptr(sbi_scratch_thishart_ptr())
+
+#define sse_set_hart_state_ptr(__scratch, __sse_state)	\
+	sbi_scratch_write_type((__scratch), void *, shs_ptr_off, (__sse_state))
+
+/*
+ * Rather than using memcpy to copy the context (which does it byte-per-byte),
+ * copy each field which generates ld/lw.
+ */
+#define regs_copy(dst, src) \
+	dst->ra = src->ra; \
+	dst->sp = src->sp; \
+	dst->gp = src->gp; \
+	dst->tp = src->tp; \
+	dst->t0 = src->t0; \
+	dst->t1 = src->t1; \
+	dst->t2 = src->t2; \
+	dst->s0 = src->s0; \
+	dst->s1 = src->s1; \
+	dst->a0 = src->a0; \
+	dst->a1 = src->a1; \
+	dst->a2 = src->a2; \
+	dst->a3 = src->a3; \
+	dst->a4 = src->a4; \
+	dst->a5 = src->a5; \
+	dst->a6 = src->a6; \
+	dst->a7 = src->a7; \
+	dst->s2 = src->s2; \
+	dst->s3 = src->s3; \
+	dst->s4 = src->s4; \
+	dst->s5 = src->s5; \
+	dst->s6 = src->s6; \
+	dst->s7 = src->s7; \
+	dst->s8 = src->s8; \
+	dst->s9 = src->s9; \
+	dst->s10 = src->s10; \
+	dst->s11 = src->s11; \
+	dst->t3 = src->t3; \
+	dst->t4 = src->t4; \
+	dst->t5 = src->t5; \
+	dst->t6 = src->t6
+
+#define EVENT_IS_GLOBAL(__event_id)	((__event_id) & SBI_SSE_EVENT_GLOBAL)
+
+#define EVENT_COUNT	array_size(supported_events)
+
+static const uint32_t supported_events[] =
+{
+	SBI_SSE_EVENT_LOCAL_RAS,
+	SBI_SSE_EVENT_LOCAL_PMU,
+	SBI_SSE_EVENT_LOCAL_ASYNC_PF,
+	SBI_SSE_EVENT_LOCAL_DEBUG,
+	SBI_SSE_EVENT_GLOBAL_RAS,
+	SBI_SSE_EVENT_GLOBAL_DEBUG,
+};
+
+struct sse_ipi_inject_data {
+	uint32_t event_id;
+};
+
+struct sbi_sse_event {
+	enum sbi_sse_state state;
+	bool pending;
+	uint32_t event_id;
+	struct sbi_sse_handler_ctx *ctx;
+	uint32_t prio;
+	unsigned int hartid;
+	const struct sbi_sse_cb_ops *cb_ops;
+	struct sbi_dlist node;
+	spinlock_t lock;
+};
+
+struct sse_hart_state {
+	struct sbi_dlist event_list;
+	spinlock_t list_lock;
+	struct sbi_sse_event *local_events;
+};
+
+static unsigned int local_event_count;
+static unsigned int global_event_count;
+static struct sbi_sse_event *global_events;
+
+static unsigned long sse_inject_fifo_off;
+static unsigned long sse_inject_fifo_mem_off;
+/* Offset of pointer to SSE HART state in scratch space */
+static unsigned long shs_ptr_off;
+
+static u32 sse_ipi_inject_event = SBI_IPI_EVENT_MAX;
+
+static int sse_ipi_inject_send(unsigned long hartid, uint32_t event_id);
+
+static bool sse_event_is_global(struct sbi_sse_event *e)
+{
+	return EVENT_IS_GLOBAL(e->event_id);
+}
+
+static void sse_event_lock(struct sbi_sse_event *e)
+{
+	if (sse_event_is_global(e))
+		spin_lock(&e->lock);
+}
+
+static void sse_event_unlock(struct sbi_sse_event *e)
+{
+	if (sse_event_is_global(e))
+		spin_unlock(&e->lock);
+}
+
+static void sse_event_set_state(struct sbi_sse_event *e,
+				enum sbi_sse_state new_state)
+{
+	enum sbi_sse_state prev_state = e->state;
+
+	e->state = new_state;
+	switch (new_state) {
+		case SSE_STATE_UNUSED:
+			if (prev_state == SSE_STATE_REGISTERED)
+				return;
+		break;
+		case SSE_STATE_REGISTERED:
+			if (prev_state == SSE_STATE_UNUSED ||
+			    prev_state == SSE_STATE_ENABLED) {
+				return;
+			}
+		break;
+		case SSE_STATE_ENABLED:
+			if (prev_state == SSE_STATE_REGISTERED ||
+			    prev_state == SSE_STATE_RUNNING)
+				return;
+		break;
+		case SSE_STATE_RUNNING:
+			if (prev_state == SSE_STATE_ENABLED)
+				return;
+		break;
+	}
+
+	sbi_panic("Invalid SSE state transition: %d -> %d\n", prev_state,
+		  new_state);
+}
+
+static struct sbi_sse_event *sse_event_get(uint32_t event)
+{
+	unsigned int i;
+	struct sbi_sse_event *events, *e;
+	unsigned int count;
+	struct sse_hart_state *shs;
+
+	if (EVENT_IS_GLOBAL(event)) {
+		count = global_event_count;
+		events = global_events;
+	} else {
+		count = local_event_count;
+		shs = sse_thishart_state_ptr();
+		events = shs->local_events;
+	}
+
+	for (i = 0; i < count; i++) {
+		e = &events[i];
+		if (e->event_id == event)
+			return e;
+	}
+
+	return NULL;
+}
+
+static int sse_event_get_attr(struct sbi_sse_event *e, uint32_t attr_id,
+			      unsigned long *out_val)
+{
+	int ret;
+
+	switch (attr_id) {
+	case SBI_SSE_ATTR_STATE:
+		*out_val = e->state;
+		ret = 0;
+		break;
+	case SBI_SSE_ATTR_PRIO:
+		*out_val = e->prio;
+		ret = 0;
+		break;
+	case SBI_SSE_ATTR_ALLOW_INJECT:
+		*out_val = 1;
+		ret = 0;
+		break;
+	case SBI_SSE_ATTR_HART_ID:
+		*out_val = e->hartid;
+		ret = 0;
+		break;
+	case SBI_SSE_ATTR_PENDING:
+		*out_val = e->pending;
+		ret = 0;
+		break;
+	default:
+		ret = SBI_EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+static void sse_event_remove_from_list(struct sbi_sse_event *e)
+{
+	struct sbi_scratch *scratch = sbi_hartid_to_scratch(e->hartid);
+	struct sse_hart_state *state = sse_get_hart_state_ptr(scratch);
+
+	spin_lock(&state->list_lock);
+	sbi_list_del(&e->node);
+	spin_unlock(&state->list_lock);
+}
+
+static void sse_event_add_to_list(struct sbi_sse_event *e)
+{
+	struct sbi_scratch *scratch = sbi_hartid_to_scratch(e->hartid);
+	struct sse_hart_state *state = sse_get_hart_state_ptr(scratch);
+	struct sbi_sse_event *tmp;
+
+	spin_lock(&state->list_lock);
+	sbi_list_for_each_entry(tmp, &state->event_list, node) {
+		if (e->prio < tmp->prio)
+			break;
+		if (e->prio == tmp->prio && e->event_id < tmp->event_id)
+			break;
+	}
+	sbi_list_add_tail(&e->node, &tmp->node);
+
+	spin_unlock(&state->list_lock);
+}
+
+static int sse_event_disable(struct sbi_sse_event *e)
+{
+	if (e->state != SSE_STATE_ENABLED)
+		return SBI_EINVALID_STATE;
+
+	sse_event_remove_from_list(e);
+	sse_event_set_state(e, SSE_STATE_REGISTERED);
+
+	return SBI_OK;
+}
+
+static int sse_event_set_hart_id(struct sbi_sse_event *e, uint32_t event_id,
+				 unsigned long new_hartid)
+{
+	int hstate;
+	unsigned int hartid = (uint32_t) new_hartid;
+	struct sbi_domain * hd = sbi_domain_thishart_ptr();
+
+	if (!sse_event_is_global(e))
+		return SBI_EDENIED;
+
+	if (e->state == SSE_STATE_RUNNING)
+		return SBI_EBUSY;
+
+	if (!sbi_domain_is_assigned_hart(hd, new_hartid))
+		return SBI_EINVAL;
+
+	hstate = sbi_hsm_hart_get_state(hd, hartid);
+	if (hstate != SBI_HSM_STATE_STARTED)
+		return SBI_EINVAL;
+
+	if (new_hartid == e->hartid)
+		return SBI_OK;
+
+	if (e->state >= SSE_STATE_ENABLED)
+		sse_event_remove_from_list(e);
+
+	e->hartid = hartid;
+
+	if (e->cb_ops && e->cb_ops->set_hartid_cb)
+		 e->cb_ops->set_hartid_cb(event_id, e->hartid);
+
+	if (e->state >= SSE_STATE_ENABLED)
+		sse_event_add_to_list(e);
+
+	if (e->pending)
+		sbi_ipi_send_many(BIT(e->hartid), 0, sse_ipi_inject_event, NULL);
+
+	return 0;
+}
+
+static int sse_event_set_attr(struct sbi_sse_event *e, uint32_t event_id,
+			      uint32_t attr_id, unsigned long val)
+{
+	int ret;
+
+	switch (attr_id) {
+	case SBI_SSE_ATTR_PENDING:
+	case SBI_SSE_ATTR_STATE:
+	case SBI_SSE_ATTR_ALLOW_INJECT:
+		/* Read-only */
+		ret = SBI_EDENIED;
+		break;
+	case SBI_SSE_ATTR_PRIO:
+		if (e->state >= SSE_STATE_ENABLED) {
+			ret = SBI_EINVALID_STATE;
+		} else {
+			e->prio = (uint32_t)val;
+			ret = 0;
+		}
+		break;
+	case SBI_SSE_ATTR_HART_ID:
+		ret = sse_event_set_hart_id(e, event_id, val);
+		break;
+	default:
+		ret = SBI_EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+static int sse_event_register(struct sbi_sse_event *e,
+			      struct sbi_sse_handler_ctx *ctx)
+{
+	if (sse_event_is_global(e) && e->hartid != current_hartid())
+		return SBI_EINVAL;
+
+	if (e->state != SSE_STATE_UNUSED)
+		return SBI_EINVALID_STATE;
+
+	e->ctx = ctx;
+	sse_event_set_state(e, SSE_STATE_REGISTERED);
+
+	return 0;
+}
+
+static int sse_event_unregister(struct sbi_sse_event *e)
+{
+	if (e->state != SSE_STATE_REGISTERED)
+		return SBI_EINVALID_STATE;
+
+	sse_event_set_state(e, SSE_STATE_UNUSED);
+	e->ctx = NULL;
+
+	return 0;
+}
+
+static void sse_event_inject(struct sbi_sse_event *e,
+			     struct sbi_sse_event *prev_e,
+			     struct sbi_trap_regs *regs)
+{
+	ulong prev_smode, prev_virt;
+	struct sse_interrupted_state *i_ctx = &e->ctx->interrupted, *prev_i_ctx;
+	struct sse_entry_state *e_ctx = &e->ctx->entry;
+
+	sse_event_set_state(e, SSE_STATE_RUNNING);
+	e->pending = false;
+
+	if (prev_e) {
+		/* We are injected right after another event, copy previous
+		 * event context for correct restoration
+		 */
+		prev_i_ctx = &prev_e->ctx->interrupted;
+
+		regs_copy(i_ctx, prev_i_ctx);
+		i_ctx->exc_mode = prev_i_ctx->exc_mode;
+		i_ctx->pc = prev_i_ctx->pc;
+	} else {
+		regs_copy(i_ctx, regs);
+
+		prev_smode = (regs->mstatus & MSTATUS_MPP) >> MSTATUS_MPP_SHIFT;
+	#if __riscv_xlen == 32
+		prev_virt = (regs->mstatusH & MSTATUSH_MPV) ? 1 : 0;
+	#else
+		prev_virt = (regs->mstatus & MSTATUS_MPV) ? 1 : 0;
+	#endif
+
+		i_ctx->exc_mode = prev_smode << EXC_MODE_PP_SHIFT;
+		i_ctx->exc_mode |= prev_virt << EXC_MODE_PV_SHIFT;
+		if (regs->mstatus & MSTATUS_SPIE)
+			i_ctx->exc_mode |= EXC_MODE_SSTATUS_SPIE;
+		i_ctx->pc = regs->mepc;
+
+		/* We only want to set SPIE for the first event injected after
+		 * entering M-Mode. For the event injected right after another
+		 * event (after calling sse_event_complete(), we will keep the
+		 * saved SPIE).
+		 */
+		regs->mstatus &= ~MSTATUS_SPIE;
+		if (regs->mstatus & MSTATUS_SIE)
+			regs->mstatus |= MSTATUS_SPIE;
+	}
+
+	regs_copy(regs, e_ctx);
+	regs->mepc = e_ctx->pc;
+
+	regs->mstatus &= ~MSTATUS_MPP;
+	regs->mstatus |= (PRV_S << MSTATUS_MPP_SHIFT);
+
+	#if __riscv_xlen == 32
+		regs->mstatusH &= ~MSTATUSH_MPV;
+	#else
+		regs->mstatus &= ~MSTATUS_MPV;
+	#endif
+
+	regs->mstatus &= ~MSTATUS_SIE;
+}
+
+static int sse_event_resume(struct sbi_sse_event *e, struct sbi_trap_regs *regs)
+{
+	struct sse_interrupted_state *i_ctx = &e->ctx->interrupted;
+
+	regs_copy(regs, i_ctx);
+
+	/* Restore previous virtualization state */
+#if __riscv_xlen == 32
+	regs->mstatusH &= ~MSTATUSH_MPV;
+	if (i_ctx->exc_mode & EXC_MODE_PV)
+		regs->mstatusH |= MSTATUSH_MPV;
+#else
+	regs->mstatus &= ~MSTATUS_MPV;
+	if (i_ctx->exc_mode & EXC_MODE_PV)
+		regs->mstatus |= MSTATUS_MPV;
+#endif
+
+	regs->mstatus &= ~MSTATUS_MPP;
+	if (i_ctx->exc_mode & EXC_MODE_PP)
+		regs->mstatus |= (PRV_S << MSTATUS_MPP_SHIFT);
+
+	regs->mstatus &= ~MSTATUS_SIE;
+	if (regs->mstatus & MSTATUS_SPIE)
+		regs->mstatus |= MSTATUS_SIE;
+
+	regs->mstatus &= ~MSTATUS_SPIE;
+	if (i_ctx->exc_mode & EXC_MODE_SSTATUS_SPIE)
+		regs->mstatus |= MSTATUS_SPIE;
+
+	regs->mepc = i_ctx->pc;
+
+	return SBI_EJUMP;
+}
+
+static bool event_is_ready(struct sbi_sse_event *e)
+{
+	if (!e->pending || e->state == SSE_STATE_RUNNING ||
+	    e->hartid != current_hartid()) {
+		return false;
+	}
+
+	return true;
+}
+
+static int sse_process_pending_events(struct sbi_sse_event *prev_e,
+				      struct sbi_trap_regs *regs)
+{
+	int ret = SBI_OK;
+	struct sbi_sse_event *e, *to_run = NULL;
+	struct sse_hart_state *state = sse_thishart_state_ptr();
+
+retry:
+	spin_lock(&state->list_lock);
+	sbi_list_for_each_entry(e, &state->event_list, node) {
+		/*
+		 * List of event is ordered by priority, stop at first running
+		 * event since all other events after this one are of lower
+		 * priority.
+		 */
+		if (e->state == SSE_STATE_RUNNING)
+			break;
+
+		if (event_is_ready(e)) {
+			to_run = e;
+			break;
+		}
+	}
+	spin_unlock(&state->list_lock);
+
+	/*
+	 * We can not lock the event while holding the list lock or we could
+	 * deadlock due to different locking order than other functions.
+	 */
+	if (to_run) {
+		sse_event_lock(to_run);
+		/*
+		 * Reevaluate readiness, for global events, the hart_id might
+		 * have changed, in that case, try to find another ready event
+		 */
+		if (!event_is_ready(e)) {
+			sse_event_unlock(to_run);
+			to_run = NULL;
+			goto retry;
+		}
+
+		sse_event_inject(to_run, prev_e, regs);
+		sse_event_unlock(to_run);
+
+		return SBI_EJUMP;
+	}
+
+	return ret;
+}
+
+static int sse_event_set_pending(struct sbi_sse_event *e)
+{
+	if (e->state != SSE_STATE_RUNNING && e->state != SSE_STATE_ENABLED)
+		return SBI_ERR_INVALID_STATE;
+
+	e->pending = true;
+
+	return SBI_OK;
+}
+
+static void sse_ipi_inject_process(struct sbi_scratch *scratch,
+				   struct sbi_trap_regs *regs)
+{
+	struct sbi_sse_event *e;
+	struct sse_ipi_inject_data evt;
+	struct sbi_fifo *sse_inject_fifo_r =
+			sbi_scratch_offset_ptr(scratch, sse_inject_fifo_off);
+
+	/* This can be the case when sbi_exit() is called */
+	if (!regs)
+		return;
+
+	/* Mark all queued events as pending */
+	while(!sbi_fifo_dequeue(sse_inject_fifo_r, &evt)) {
+		e = sse_event_get(evt.event_id);
+		if (!e)
+			continue;
+
+		sse_event_lock(e);
+		sse_event_set_pending(e);
+		sse_event_unlock(e);
+	}
+
+	sse_process_pending_events(NULL, regs);
+}
+
+static struct sbi_ipi_event_ops sse_ipi_inject_ops = {
+	.name = "IPI_SSE_INJECT",
+	.process = sse_ipi_inject_process,
+};
+
+static int sse_ipi_inject_send(unsigned long hartid, uint32_t event_id)
+{
+	int ret;
+	struct sbi_scratch *remote_scratch = NULL;
+	struct sse_ipi_inject_data evt = {event_id};
+	struct sbi_fifo *sse_inject_fifo_r;
+
+	remote_scratch = sbi_hartid_to_scratch(hartid);
+	if (!remote_scratch)
+		return SBI_EINVAL;
+	sse_inject_fifo_r = sbi_scratch_offset_ptr(remote_scratch,
+						   sse_inject_fifo_off);
+
+	ret = sbi_fifo_enqueue(sse_inject_fifo_r, &evt);
+	if (ret)
+		return SBI_EFAIL;
+
+	ret = sbi_ipi_send_many(BIT(hartid), 0, sse_ipi_inject_event, NULL);
+	if (ret)
+		return SBI_EFAIL;
+
+	return SBI_OK;
+}
+
+static int sse_inject_event(uint32_t event_id, unsigned long hartid,
+			    struct sbi_trap_regs *regs, bool from_ecall)
+{
+	int ret;
+	struct sbi_sse_event *e;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	sse_event_lock(e);
+
+	/* In case of global event, provided hart_id is ignored */
+	if (sse_event_is_global(e))
+		hartid = e->hartid;
+
+	/*
+	 * If coming from an ecall, always use an IPI to send the event, this
+	 * simplifies handling as we don't have to modify epc/a0 for ecall
+	 * return value.
+	 */
+	if (from_ecall || hartid != current_hartid()) {
+		sse_event_unlock(e);
+		return sse_ipi_inject_send(hartid, event_id);
+	}
+
+	/*
+	 * In other cases, directly handle the event on this hart for faster
+	 * handling
+	 */
+	ret = sse_event_set_pending(e);
+	sse_event_unlock(e);
+	if (ret)
+		return ret;
+
+	return sse_process_pending_events(NULL, regs);
+}
+
+static int sse_event_enable(struct sbi_sse_event *e)
+{
+	if (e->state != SSE_STATE_REGISTERED)
+		return SBI_EINVALID_STATE;
+
+	sse_event_set_state(e, SSE_STATE_ENABLED);
+	sse_event_add_to_list(e);
+
+	return SBI_OK;
+}
+
+static int sse_event_complete(struct sbi_sse_event *e, uint32_t status,
+			      uint32_t flags, struct sbi_trap_regs *regs)
+{
+	if (e->state != SSE_STATE_RUNNING)
+		return SBI_EINVALID_STATE;
+
+	if (e->hartid != current_hartid())
+		return SBI_EDENIED;
+
+	if (flags & SBI_SSE_COMPLETE_FLAG_EVENT_DISABLE)
+		sse_event_disable(e);
+	else
+		sse_event_set_state(e, SSE_STATE_ENABLED);
+
+	if (e->cb_ops && e->cb_ops->complete_cb)
+		e->cb_ops->complete_cb(e->event_id);
+
+	if (sse_process_pending_events(e, regs) == SBI_EJUMP)
+		return SBI_EJUMP;
+
+	return sse_event_resume(e, regs);
+}
+
+int sbi_sse_complete(uint32_t event_id, uint32_t status, uint32_t flags,
+		     struct sbi_trap_regs *regs)
+{
+	int ret;
+	struct sbi_sse_event *e;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	sse_event_lock(e);
+	ret = sse_event_complete(e, status, flags, regs);
+	sse_event_unlock(e);
+
+	return ret;
+}
+
+int sbi_sse_enable(uint32_t event_id)
+{
+	int ret;
+	struct sbi_sse_event *e;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	sse_event_lock(e);
+	ret = sse_event_enable(e);
+	sse_event_unlock(e);
+
+	return ret;
+}
+
+int sbi_sse_disable(uint32_t event_id)
+{
+	int ret;
+	struct sbi_sse_event *e;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	sse_event_lock(e);
+	ret = sse_event_disable(e);
+	sse_event_unlock(e);
+
+	return ret;
+}
+
+int sbi_sse_inject_from_ecall(uint32_t event_id, unsigned long hartid,
+			      struct sbi_trap_regs *regs)
+{
+	if (!sbi_domain_is_assigned_hart(sbi_domain_thishart_ptr(), hartid))
+		return SBI_EINVAL;
+
+	return sse_inject_event(event_id, hartid, regs, true);
+}
+
+int sbi_sse_inject_event(uint32_t event_id, struct sbi_trap_regs *regs)
+{
+	return sse_inject_event(event_id, current_hartid(), regs, false);
+}
+
+int sbi_sse_set_cb_ops(uint32_t event_id, const struct sbi_sse_cb_ops *cb_ops)
+{
+	struct sbi_sse_event *e;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	if (cb_ops->set_hartid_cb && !sse_event_is_global(e))
+		return SBI_EINVAL;
+
+	sse_event_lock(e);
+	e->cb_ops = cb_ops;
+	sse_event_unlock(e);
+
+	return SBI_OK;
+}
+
+
+int sbi_sse_get_attr(uint32_t event_id, uint32_t attr_id, unsigned long *out_val)
+{
+	struct sbi_sse_event *e;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	return sse_event_get_attr(e, attr_id, out_val);
+}
+
+int sbi_sse_set_attr(uint32_t event_id, uint32_t attr_id, unsigned long val)
+{
+	int ret;
+	struct sbi_sse_event *e;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	sse_event_lock(e);
+	ret = sse_event_set_attr(e, event_id, attr_id, val);
+	sse_event_unlock(e);
+
+	return ret;
+}
+
+int sbi_sse_register(uint32_t event_id,
+			unsigned long phys_lo,
+			unsigned long phys_hi)
+{
+	int ret;
+	struct sbi_sse_event *e;
+	const unsigned align = __riscv_xlen >> 3;
+	ulong smode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >>
+			MSTATUS_MPP_SHIFT;
+
+	if (phys_lo & (align - 1))
+		return SBI_EINVALID_ADDR;
+
+	/*
+	 * On RV32, the M-mode can only access the first 4GB of
+	 * the physical address space because M-mode does not have
+	 * MMU to access full 34-bit physical address space.
+	 *
+	 * Based on above, we simply fail if the upper 32bits of
+	 * the physical address (i.e. a2 register) is non-zero on
+	 * RV32.
+	 */
+	if (phys_hi)
+		return SBI_EINVALID_ADDR;
+
+	if (!sbi_domain_check_addr_range(sbi_domain_thishart_ptr(), phys_lo,
+					 sizeof(struct sbi_sse_handler_ctx),
+					 smode,
+					 SBI_DOMAIN_READ|SBI_DOMAIN_WRITE))
+		return SBI_EINVALID_ADDR;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	sse_event_lock(e);
+	ret = sse_event_register(e, (struct sbi_sse_handler_ctx *)phys_lo);
+	sse_event_unlock(e);
+
+	if (!ret) {
+		if (e->cb_ops && e->cb_ops->register_cb) {
+			e->cb_ops->register_cb(e->event_id);
+		}
+	}
+
+	return ret;
+}
+
+int sbi_sse_unregister(uint32_t event_id)
+{
+	int ret;
+	struct sbi_sse_event *e;
+
+	e = sse_event_get(event_id);
+	if (!e)
+		return SBI_EINVAL;
+
+	sse_event_lock(e);
+	ret = sse_event_unregister(e);
+	sse_event_unlock(e);
+
+	if (!ret) {
+		if (e->cb_ops && e->cb_ops->unregister_cb) {
+			e->cb_ops->unregister_cb(e->event_id);
+		}
+	}
+
+	return ret;
+}
+
+static void sse_event_init(struct sbi_sse_event *event, uint32_t event_id)
+{
+	event->event_id = event_id;
+	event->hartid = current_hartid();
+	SPIN_LOCK_INIT(event->lock);
+}
+
+static int sse_global_init()
+{
+	unsigned int i, ev = 0;
+
+	for (i = 0; i < EVENT_COUNT; i++) {
+		if (EVENT_IS_GLOBAL(supported_events[i]))
+			global_event_count++;
+		else
+			local_event_count++;
+	}
+
+	global_events = sbi_zalloc(sizeof(*global_events) * global_event_count);
+	if (!global_events)
+		return SBI_ENOMEM;
+
+	for (i = 0; i < EVENT_COUNT; i++) {
+		if (!EVENT_IS_GLOBAL(supported_events[i]))
+			continue;
+
+		sse_event_init(&global_events[ev++], supported_events[i]);
+	}
+
+	return 0;
+}
+
+static void sse_local_init(struct sse_hart_state *shs)
+{
+	unsigned int i, ev = 0;
+
+	SBI_INIT_LIST_HEAD(&shs->event_list);
+	SPIN_LOCK_INIT(shs->list_lock);
+
+	for (i = 0; i < EVENT_COUNT; i++) {
+		if (EVENT_IS_GLOBAL(supported_events[i]))
+			continue;
+
+		sse_event_init(&shs->local_events[ev++], supported_events[i]);
+	}
+}
+
+int sbi_sse_init(struct sbi_scratch *scratch, bool cold_boot)
+{
+	int ret;
+	void *sse_inject_mem;
+	struct sse_hart_state *shs;
+	struct sbi_fifo *sse_inject_q;
+
+	if (cold_boot) {
+		ret = sse_global_init();
+		if (ret)
+			return ret;
+
+		shs_ptr_off = sbi_scratch_alloc_offset(sizeof(void *));
+		if (!shs_ptr_off)
+			return SBI_ENOMEM;
+
+		sse_inject_fifo_off = sbi_scratch_alloc_offset(
+							sizeof(*sse_inject_q));
+		if (!sse_inject_fifo_off) {
+			sbi_scratch_free_offset(shs_ptr_off);
+			return SBI_ENOMEM;
+		}
+
+		sse_inject_fifo_mem_off = sbi_scratch_alloc_offset(
+			EVENT_COUNT * sizeof(struct sse_ipi_inject_data));
+		if (!sse_inject_fifo_mem_off) {
+			sbi_scratch_free_offset(sse_inject_fifo_off);
+			sbi_scratch_free_offset(shs_ptr_off);
+			return SBI_ENOMEM;
+		}
+
+		ret = sbi_ipi_event_create(&sse_ipi_inject_ops);
+		if (ret < 0) {
+			sbi_scratch_free_offset(shs_ptr_off);
+			return ret;
+		}
+		sse_ipi_inject_event = ret;
+	}
+
+	shs = sse_get_hart_state_ptr(scratch);
+	if (!shs) {
+		shs = sbi_zalloc(sizeof(*shs) +
+			sizeof(struct sbi_sse_event) * local_event_count);
+		if (!shs)
+			return SBI_ENOMEM;
+
+		shs->local_events = (struct sbi_sse_event *)(shs + 1);
+
+		sse_set_hart_state_ptr(scratch, shs);
+	}
+
+	sse_local_init(shs);
+
+	sse_inject_q = sbi_scratch_offset_ptr(scratch, sse_inject_fifo_off);
+	sse_inject_mem = sbi_scratch_offset_ptr(scratch,
+						sse_inject_fifo_mem_off);
+
+	sbi_fifo_init(sse_inject_q, sse_inject_mem, EVENT_COUNT,
+		      sizeof(struct sse_ipi_inject_data));
+
+	return 0;
+}
+
+void sbi_sse_exit(struct sbi_scratch *scratch)
+{
+	int i;
+	struct sbi_sse_event *e;
+
+	for (i = 0; i < EVENT_COUNT; i++) {
+		e = sse_event_get(supported_events[i]);
+
+		if (e->hartid != current_hartid())
+			continue;
+
+		if (e->state > SSE_STATE_REGISTERED)
+			sbi_printf("Event %d in invalid state at exit", i);
+	}
+}
-- 
2.42.0



From peterlin at andestech.com  Thu Nov 30 04:41:58 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:41:58 +0800
Subject: [PATCH v4 00/15] Add Andes PMU extension support
Message-ID: <20231130124213.2590640-1-peterlin@andestech.com>

This patch series enables perf tool to utilize Andes PMU
extension via PMU SBI calls, provides PMU device callbacks
to achieve event sampling and mode filtering.

The last patch provides a PMU node example used on AX45MP cores.

The OpenSBI and Linux patches can be found on Andes Technology GitHub
- https://github.com/andestech/opensbi/commits/andes-pmu-support-v4
- https://github.com/andestech/linux/commits/andes-pmu-support-v4

For more details about Andes PMU extension, see Counter Related
Registers in the AX45MP datasheet:
http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf

Leo Yu-Chi Liang (1):
  lib: ipi: Adjust Andes PLICSW to single-bit-per-hart scheme

Yu Chien Peter Lin (14):
  sbi: sbi_pmu: Improve sbi_pmu_init() error handling
  lib: sbi: Add XAndesPMU in hart extensions
  sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
  platform: include: andes45: Add PMU related CSR defines
  platform: generic: Introduce pmu_init() platform override
  platform: andes: Add Andes custom PMU support
  platform: andes: Enable Andes PMU for AE350
  platform: rzfive: Enable Andes PMU for RZ/Five
  lib: utils: fdt_fixup: Allow preserving PMU properties
  platform: andes: Factor out is_andes() helper
  lib: utils: fdt_pmu: Make the fdt_pmu_evt_select table global variable
  lib: utils: fdt_pmu: Do not iterate over the fdt_pmu_evt_select table
  platform: andes: Add Andes default PMU mapping support
  docs: pmu: Add Andes PMU node example

 docs/pmu_support.md                          |  82 ++++
 include/sbi/sbi_ecall_interface.h            |   5 +
 include/sbi/sbi_hart.h                       |   2 +
 include/sbi/sbi_pmu.h                        |   6 +
 include/sbi_utils/fdt/fdt_helper.h           |  17 +
 include/sbi_utils/fdt/fdt_pmu.h              |   6 +
 include/sbi_utils/ipi/andes_plicsw.h         |  23 +-
 lib/sbi/sbi_hart.c                           |   3 +
 lib/sbi/sbi_pmu.c                            |  26 +-
 lib/utils/fdt/Kconfig                        |   7 +
 lib/utils/fdt/fdt_fixup.c                    |   3 +
 lib/utils/fdt/fdt_pmu.c                      |  17 +-
 lib/utils/ipi/andes_plicsw.c                 | 104 ++---
 platform/generic/Kconfig                     |   4 +
 platform/generic/andes/Kconfig               |  15 +
 platform/generic/andes/ae350.c               |  19 +-
 platform/generic/andes/andes_hpm.c           | 405 +++++++++++++++++++
 platform/generic/andes/andes_pmu.c           | 105 +++++
 platform/generic/andes/objects.mk            |   2 +
 platform/generic/include/andes/andes45.h     |  32 ++
 platform/generic/include/andes/andes_hpm.h   |  81 ++++
 platform/generic/include/andes/andes_pmu.h   |  34 ++
 platform/generic/include/platform_override.h |   1 +
 platform/generic/platform.c                  |  14 +-
 platform/generic/renesas/rzfive/rzfive.c     |   5 +-
 25 files changed, 908 insertions(+), 110 deletions(-)
 create mode 100644 platform/generic/andes/andes_hpm.c
 create mode 100644 platform/generic/andes/andes_pmu.c
 create mode 100644 platform/generic/include/andes/andes_hpm.h
 create mode 100644 platform/generic/include/andes/andes_pmu.h

-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:41:59 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:41:59 +0800
Subject: [PATCH v4 01/15] lib: ipi: Adjust Andes PLICSW to single-bit-per-hart scheme
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-2-peterlin@andestech.com>

From: Leo Yu-Chi Liang <ycliang at andestech.com>

The old scheme doesn't allow sending hart0 self-IPI as the
corresponding bit on pending register is hardwired to 0, this
could lead to unhandle IPIs on SMP systems, esp. on single-core.

Furthermore, the limitation of old scheme is 8-core, instead of
reserving source hart information, we assign bit (x + 1) as the
enable and pending bit of hartx, this also expands the bootable
hart number.

The following diagram shows the enable bits of the new scheme
on 32-core Andes platform.

   Pending regs: 0x1000  x---0---0---0---0------0---0
Pending hart ID:             0   1   2   3 ... 30  31
   Interrupt ID:         0   1   2   3   4 ... 31  32
                         |   |   |   |   |      |   |
    Enable regs: 0x2000  x---1---0---0---0-...--0---0---> hart0
                         |   |   |   |   |      |   |
                 0x2080  x---0---1---0---0-...--0---0---> hart1
                         |   |   |   |   |      |   |
                 0x2100  x---0---0---1---0-...--0---0---> hart2
                         |   |   |   |   |      |   |
                 0x2180  x---0---0---0---1-...--0---0---> hart3
                         .   .   .   .   .      .   .
                         .   .   .   .   .      .   .
                         .   .   .   .   .      .   .
                 0x2f00  x---0---0---0---0-...--1---0---> hart30
                         |   |   |   |   |      |   |
                 0x2f80  x---0---0---0---0-...--0---1---> hart31
                         <-------- word 0 -------><--- word 1 --->

To send IPI to hart0, for example, another hart (including hart0
itself) will set bit 1 of first word on the pending register.

We also fix indentation in andes_plicsw.h along with this patch.

Fixes: ce7c490719ed ("lib: utils/ipi: Add Andes fdt ipi driver support")
Signed-off-by: Leo Yu-Chi Liang <ycliang at andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Randolph <randolph at andestech.com>
Reported-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Link: https://lists.infradead.org/pipermail/opensbi/2023-October/005665.html
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v2 -> v3:
  - New patch
Changes v3 -> v4:
  - Include Prabhakar's RB & TB tags
---
 include/sbi_utils/ipi/andes_plicsw.h |  23 +++---
 lib/utils/ipi/andes_plicsw.c         | 104 ++++++++++-----------------
 2 files changed, 46 insertions(+), 81 deletions(-)

diff --git a/include/sbi_utils/ipi/andes_plicsw.h b/include/sbi_utils/ipi/andes_plicsw.h
index e93cda0..0d18444 100644
--- a/include/sbi_utils/ipi/andes_plicsw.h
+++ b/include/sbi_utils/ipi/andes_plicsw.h
@@ -13,30 +13,23 @@
 #ifndef _IPI_ANDES_PLICSW_H_
 #define _IPI_ANDES_PLICSW_H_
 
-#define PLICSW_PRIORITY_BASE 0x4
+#define PLICSW_PRIORITY_BASE	0x4
 
-#define PLICSW_PENDING_BASE 0x1000
-#define PLICSW_PENDING_STRIDE 0x8
+#define PLICSW_PENDING_BASE	0x1000
 
-#define PLICSW_ENABLE_BASE 0x2000
-#define PLICSW_ENABLE_STRIDE 0x80
+#define PLICSW_ENABLE_BASE	0x2000
+#define PLICSW_ENABLE_STRIDE	0x80
 
-#define PLICSW_CONTEXT_BASE 0x200000
-#define PLICSW_CONTEXT_STRIDE 0x1000
-#define PLICSW_CONTEXT_CLAIM 0x4
+#define PLICSW_CONTEXT_BASE	0x200000
+#define PLICSW_CONTEXT_STRIDE	0x1000
+#define PLICSW_CONTEXT_CLAIM	0x4
 
-#define PLICSW_HART_MASK 0x01010101
-
-#define PLICSW_HART_MAX_NR 8
-
-#define PLICSW_REGION_ALIGN 0x1000
+#define PLICSW_REGION_ALIGN	0x1000
 
 struct plicsw_data {
 	unsigned long addr;
 	unsigned long size;
 	uint32_t hart_count;
-	/* hart id to source id table */
-	uint32_t source_id[PLICSW_HART_MAX_NR];
 };
 
 int plicsw_warm_ipi_init(void);
diff --git a/lib/utils/ipi/andes_plicsw.c b/lib/utils/ipi/andes_plicsw.c
index 5693efb..413ac20 100644
--- a/lib/utils/ipi/andes_plicsw.c
+++ b/lib/utils/ipi/andes_plicsw.c
@@ -18,77 +18,45 @@
 
 struct plicsw_data plicsw;
 
-static inline void plicsw_claim(void)
+static void plicsw_ipi_send(u32 hart_index)
 {
-	u32 hartid = current_hartid();
+	ulong pending_reg;
+	u32 interrupt_id, word_index, pending_bit;
+	u32 target_hart = sbi_hartindex_to_hartid(hart_index);
 
-	if (plicsw.hart_count <= hartid)
+	if (plicsw.hart_count <= target_hart)
 		ebreak();
 
-	plicsw.source_id[hartid] =
-		readl((void *)plicsw.addr + PLICSW_CONTEXT_BASE +
-		      PLICSW_CONTEXT_CLAIM + PLICSW_CONTEXT_STRIDE * hartid);
-}
-
-static inline void plicsw_complete(void)
-{
-	u32 hartid = current_hartid();
-	u32 source = plicsw.source_id[hartid];
-
-	writel(source, (void *)plicsw.addr + PLICSW_CONTEXT_BASE +
-			       PLICSW_CONTEXT_CLAIM +
-			       PLICSW_CONTEXT_STRIDE * hartid);
-}
-
-static inline void plic_sw_pending(u32 target_hart)
-{
 	/*
-	 * The pending array registers are w1s type.
-	 * IPI pending array mapping as following:
-	 *
-	 * Pending array start address: base + 0x1000
-	 * ---------------------------------
-	 * | hart3 | hart2 | hart1 | hart0 |
-	 * ---------------------------------
-	 * Each hartX can send IPI to another hart by setting the
-	 * bitY to its own region (see the below).
-	 *
-	 * In each hartX region:
-	 * <---------- PICSW_PENDING_STRIDE -------->
-	 * | bit7 | ... | bit3 | bit2 | bit1 | bit0 |
-	 * ------------------------------------------
-	 * The bitY of hartX region indicates that hartX sends an
-	 * IPI to hartY.
+	 * We assign a single bit for each hart.
+	 * Bit 0 is hardwired to 0, thus unavailable.
+	 * Bit(X+1) indicates that IPI is sent to hartX.
 	 */
-	u32 hartid	    = current_hartid();
-	u32 word_index	    = hartid / 4;
-	u32 per_hart_offset = PLICSW_PENDING_STRIDE * hartid;
-	u32 val		    = 1 << target_hart << per_hart_offset;
+	interrupt_id = target_hart + 1;
+	word_index   = interrupt_id / 32;
+	pending_bit  = interrupt_id % 32;
+	pending_reg  = plicsw.addr + PLICSW_PENDING_BASE + word_index * 4;
 
-	writel(val, (void *)plicsw.addr + PLICSW_PENDING_BASE + word_index * 4);
+	/* Set target hart's mip.MSIP */
+	writel_relaxed(BIT(pending_bit), (void *)pending_reg);
 }
 
-static void plicsw_ipi_send(u32 hart_index)
+static void plicsw_ipi_clear(u32 hart_index)
 {
 	u32 target_hart = sbi_hartindex_to_hartid(hart_index);
+	ulong reg = plicsw.addr + PLICSW_CONTEXT_BASE + PLICSW_CONTEXT_CLAIM +
+		    PLICSW_CONTEXT_STRIDE * target_hart;
 
 	if (plicsw.hart_count <= target_hart)
 		ebreak();
 
-	/* Set PLICSW IPI */
-	plic_sw_pending(target_hart);
-}
+	/* Claim */
+	u32 source = readl((void *)reg);
 
-static void plicsw_ipi_clear(u32 hart_index)
-{
-	u32 target_hart = sbi_hartindex_to_hartid(hart_index);
-
-	if (plicsw.hart_count <= target_hart)
-		ebreak();
+	/* A successful claim will clear mip.MSIP */
 
-	/* Clear PLICSW IPI */
-	plicsw_claim();
-	plicsw_complete();
+	/* Complete */
+	writel(source, (void *)reg);
 }
 
 static struct sbi_ipi_device plicsw_ipi = {
@@ -110,22 +78,26 @@ int plicsw_warm_ipi_init(void)
 int plicsw_cold_ipi_init(struct plicsw_data *plicsw)
 {
 	int rc;
+	u32 interrupt_id, word_index, enable_bit;
+	ulong enable_reg, priority_reg;
 
 	/* Setup source priority */
-	uint32_t *priority = (void *)plicsw->addr + PLICSW_PRIORITY_BASE;
-
-	for (int i = 0; i < plicsw->hart_count; i++)
-		writel(1, &priority[i]);
-
-	/* Setup target enable */
-	uint32_t enable_mask = PLICSW_HART_MASK;
+	for (int i = 0; i < plicsw->hart_count; i++) {
+		priority_reg = plicsw->addr + PLICSW_PRIORITY_BASE + i * 4;
+		writel(1, (void *)priority_reg);
+	}
 
+	/*
+	 * Setup enable for each hart, skip non-existent interrupt ID 0
+	 * which is hardwired to 0.
+	 */
 	for (int i = 0; i < plicsw->hart_count; i++) {
-		uint32_t *enable = (void *)plicsw->addr + PLICSW_ENABLE_BASE +
-				   PLICSW_ENABLE_STRIDE * i;
-		writel(enable_mask, enable);
-		writel(enable_mask, enable + 1);
-		enable_mask <<= 1;
+		interrupt_id = i + 1;
+		word_index   = interrupt_id / 32;
+		enable_bit   = interrupt_id % 32;
+		enable_reg   = plicsw->addr + PLICSW_ENABLE_BASE +
+			       PLICSW_ENABLE_STRIDE * i + 4 * word_index;
+		writel(BIT(enable_bit), (void *)enable_reg);
 	}
 
 	/* Add PLICSW region to the root domain */
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:00 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:00 +0800
Subject: [PATCH v4 02/15] sbi: sbi_pmu: Improve sbi_pmu_init() error handling
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-3-peterlin@andestech.com>

This patch makes the following changes:

- As sbi_platform_pmu_init() returns a negative error code on
  failure, let sbi_pmu_init() print out the error code with
  sbi_dprintf().

- In order to distinguish the SBI_EFAIL error returned by
  sbi_pmu_add_*_counter_map(), return SBI_ENOENT to indicate
  that fdt_pmu_setup() failed to locate "riscv,pmu" node, and
  generic_pmu_init() ignores such case.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v1 -> v2:
  - New patch
Changes v2 -> v3:
  - Include Anup's RB tag
Changes v3 -> v4:
  - Use sbi_dprintf() instead of hang on error
  - Include Prabhakar's RB tag
---
 lib/sbi/sbi_pmu.c           | 6 +++++-
 lib/utils/fdt/fdt_pmu.c     | 2 +-
 platform/generic/platform.c | 8 +++++++-
 3 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 185068b..2ee6e62 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -957,6 +957,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 	int hpm_count = sbi_fls(sbi_hart_mhpm_mask(scratch));
 	struct sbi_pmu_hart_state *phs;
 	const struct sbi_platform *plat;
+	int rc;
 
 	if (cold_boot) {
 		hw_event_map = sbi_calloc(sizeof(*hw_event_map),
@@ -972,7 +973,10 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 
 		plat = sbi_platform_ptr(scratch);
 		/* Initialize hw pmu events */
-		sbi_platform_pmu_init(plat);
+		rc = sbi_platform_pmu_init(plat);
+		if (rc)
+			sbi_dprintf("%s: platform pmu init failed "
+				    "(error %d)\n", __func__, rc);
 
 		/* mcycle & minstret is available always */
 		if (!hpm_count)
diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
index 83301bb..a8d7648 100644
--- a/lib/utils/fdt/fdt_pmu.c
+++ b/lib/utils/fdt/fdt_pmu.c
@@ -74,7 +74,7 @@ int fdt_pmu_setup(void *fdt)
 
 	pmu_offset = fdt_node_offset_by_compatible(fdt, -1, "riscv,pmu");
 	if (pmu_offset < 0)
-		return SBI_EFAIL;
+		return SBI_ENOENT;
 
 	event_ctr_map = fdt_getprop(fdt, pmu_offset,
 				    "riscv,event-to-mhpmcounters", &len);
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index 85acecd..fa400b9 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -265,7 +265,13 @@ static u32 generic_tlb_num_entries(void)
 
 static int generic_pmu_init(void)
 {
-	return fdt_pmu_setup(fdt_get_address());
+	int rc;
+
+	rc = fdt_pmu_setup(fdt_get_address());
+	if (rc && rc != SBI_ENOENT)
+		return rc;
+
+	return 0;
 }
 
 static uint64_t generic_pmu_xlate_to_mhpmevent(uint32_t event_idx,
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:01 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:01 +0800
Subject: [PATCH v4 03/15] lib: sbi: Add XAndesPMU in hart extensions
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-4-peterlin@andestech.com>

Add the custom extension to hart extension list.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v2 -> v3:
  - New patch
Changes v3 -> v4:
  - Include RB tags
---
 include/sbi/sbi_hart.h | 2 ++
 lib/sbi/sbi_hart.c     | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
index 6ee49ff..811e5f5 100644
--- a/include/sbi/sbi_hart.h
+++ b/include/sbi/sbi_hart.h
@@ -45,6 +45,8 @@ enum sbi_hart_extensions {
 	SBI_HART_EXT_ZKR,
 	/** Hart has Smcntrpmf extension */
 	SBI_HART_EXT_SMCNTRPMF,
+	/** Hart has Xandespmu extension */
+	SBI_HART_EXT_XANDESPMU,
 
 	/** Maximum index of Hart extension */
 	SBI_HART_EXT_MAX,
diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index bf8792a..c948373 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -694,6 +694,9 @@ static inline char *sbi_hart_extension_id2string(int ext)
 	case SBI_HART_EXT_SMCNTRPMF:
 		estr = "smcntrpmf";
 		break;
+	case SBI_HART_EXT_XANDESPMU:
+		estr = "xandespmu";
+		break;
 	default:
 		break;
 	}
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:02 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:02 +0800
Subject: [PATCH v4 04/15] sbi: sbi_pmu: Add hw_counter_filter_mode() to pmu device
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-5-peterlin@andestech.com>

Add support for custom PMU extensions to set inhibit bits
on custom CSRs by introducing the PMU device callback
hw_counter_filter_mode(). This allows the perf tool to
restrict event counting under a specified privileged
mode by appending a modifier, e.g. perf record -e event:k
to count events only happening in kernel mode.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
---
Changes v1 -> v2:
  - No change
Changes v2 -> v3:
  - Add pmu_dev->hw_counter_filter_mode() in pmu_fixed_ctr_update_inhibit_bits()
Changes v3 -> v4:
  - Check pmu_dev->hw_counter_filter_mode() instead of custom extension
    in pmu_fixed_ctr_update_inhibit_bits()
---
 include/sbi/sbi_pmu.h |  6 ++++++
 lib/sbi/sbi_pmu.c     | 20 ++++++++++++++------
 2 files changed, 20 insertions(+), 6 deletions(-)

diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
index 16f6877..d63149c 100644
--- a/include/sbi/sbi_pmu.h
+++ b/include/sbi/sbi_pmu.h
@@ -89,6 +89,12 @@ struct sbi_pmu_device {
 	 * Custom function returning the machine-specific irq-bit.
 	 */
 	int (*hw_counter_irq_bit)(void);
+
+	/**
+	 * Custom function to inhibit counting of events while in
+	 * specified mode.
+	 */
+	void (*hw_counter_filter_mode)(unsigned long flags, int counter_index);
 };
 
 /** Get the PMU platform device */
diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 2ee6e62..4b0f5be 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -599,7 +599,10 @@ static int pmu_update_hw_mhpmevent(struct sbi_pmu_hw_event *hw_evt, int ctr_idx,
 		pmu_dev->hw_counter_disable_irq(ctr_idx);
 
 	/* Update the inhibit flags based on inhibit flags received from supervisor */
-	pmu_update_inhibit_flags(flags, &mhpmevent_val);
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF))
+		pmu_update_inhibit_flags(flags, &mhpmevent_val);
+	if (pmu_dev && pmu_dev->hw_counter_filter_mode)
+		pmu_dev->hw_counter_filter_mode(flags, ctr_idx);
 
 #if __riscv_xlen == 32
 	csr_write_num(CSR_MHPMEVENT3 + ctr_idx - 3, mhpmevent_val & 0xFFFFFFFF);
@@ -620,7 +623,8 @@ static int pmu_fixed_ctr_update_inhibit_bits(int fixed_ctr, unsigned long flags)
 #if __riscv_xlen == 32
 	uint64_t cfgh_csr_no;
 #endif
-	if (!sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF))
+	if (!sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF) &&
+		!(pmu_dev && pmu_dev->hw_counter_filter_mode))
 		return fixed_ctr;
 
 	switch (fixed_ctr) {
@@ -641,13 +645,17 @@ static int pmu_fixed_ctr_update_inhibit_bits(int fixed_ctr, unsigned long flags)
 	}
 
 	cfg_val |= MHPMEVENT_MINH;
-	pmu_update_inhibit_flags(flags, &cfg_val);
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF)) {
+		pmu_update_inhibit_flags(flags, &cfg_val);
 #if __riscv_xlen == 32
-	csr_write_num(cfg_csr_no, cfg_val & 0xFFFFFFFF);
-	csr_write_num(cfgh_csr_no, cfg_val >> BITS_PER_LONG);
+		csr_write_num(cfg_csr_no, cfg_val & 0xFFFFFFFF);
+		csr_write_num(cfgh_csr_no, cfg_val >> BITS_PER_LONG);
 #else
-	csr_write_num(cfg_csr_no, cfg_val);
+		csr_write_num(cfg_csr_no, cfg_val);
 #endif
+	}
+	if (pmu_dev && pmu_dev->hw_counter_filter_mode)
+		pmu_dev->hw_counter_filter_mode(flags, fixed_ctr);
 	return fixed_ctr;
 }
 
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:03 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:03 +0800
Subject: [PATCH v4 05/15] platform: include: andes45: Add PMU related CSR defines
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-6-peterlin@andestech.com>

Add CSR definitions for Andes PMU extension.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v1 -> v2:
  - Rename andes_pmu() -> has_andes_pmu()
Changes v2 -> v3:
  - Include Anup's RB tag
Changes v3 -> v4:
  - Include RB tags
---
 platform/generic/include/andes/andes45.h | 26 ++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
index f570994..ce31617 100644
--- a/platform/generic/include/andes/andes45.h
+++ b/platform/generic/include/andes/andes45.h
@@ -12,6 +12,17 @@
 #define CSR_MDCM_CFG		0xfc1
 #define CSR_MMSC_CFG		0xfc2
 
+/* Machine Trap Related Registers */
+#define CSR_MSLIDELEG		0x7d5
+
+/* Counter Related Registers */
+#define CSR_MCOUNTERWEN		0x7ce
+#define CSR_MCOUNTERINTEN	0x7cf
+#define CSR_MCOUNTERMASK_M	0x7d1
+#define CSR_MCOUNTERMASK_S	0x7d2
+#define CSR_MCOUNTERMASK_U	0x7d3
+#define CSR_MCOUNTEROVF		0x7d4
+
 #define MICM_CFG_ISZ_OFFSET		6
 #define MICM_CFG_ISZ_MASK		(0x7  << MICM_CFG_ISZ_OFFSET)
 
@@ -26,4 +37,19 @@
 #define MCACHE_CTL_CCTL_SUEN_OFFSET	8
 #define MCACHE_CTL_CCTL_SUEN_MASK	(0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
 
+/* Performance monitor */
+#define MMSC_CFG_PMNDS_MASK		(1 << 15)
+#define MIP_PMOVI			(1 << 18)
+
+#ifndef __ASSEMBLER__
+
+#define has_andes_pmu()					\
+({							\
+	(((csr_read(CSR_MMSC_CFG) &			\
+	   MMSC_CFG_PMNDS_MASK)				\
+	  && misa_extension('S')) ? true : false);	\
+})
+
+#endif /* __ASSEMBLER__ */
+
 #endif /* _RISCV_ANDES45_H */
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:04 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:04 +0800
Subject: [PATCH v4 06/15] platform: generic: Introduce pmu_init() platform override
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-7-peterlin@andestech.com>

Add pmu_init() platform override, which will be used to register
PMU device and populate event mappings.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v2 -> v3:
  - New patch
Changes v3 -> v4:
  - Only return pmu_init override on error so fdt_pmu_setup() can be executed
  - Include RB tags
---
 platform/generic/include/platform_override.h | 1 +
 platform/generic/platform.c                  | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
index bf4b112..f2a4327 100644
--- a/platform/generic/include/platform_override.h
+++ b/platform/generic/include/platform_override.h
@@ -27,6 +27,7 @@ struct platform_override {
 	int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
 	int (*extensions_init)(const struct fdt_match *match,
 			       struct sbi_hart_features *hfeatures);
+	int (*pmu_init)(const struct fdt_match *match);
 	void (*fw_init)(void *fdt, const struct fdt_match *match);
 	int (*vendor_ext_provider)(long funcid,
 				   const struct sbi_trap_regs *regs,
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index fa400b9..f17e94e 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -267,6 +267,12 @@ static int generic_pmu_init(void)
 {
 	int rc;
 
+	if (generic_plat && generic_plat->pmu_init) {
+		rc = generic_plat->pmu_init(generic_plat_match);
+		if (rc)
+			return rc;
+	}
+
 	rc = fdt_pmu_setup(fdt_get_address());
 	if (rc && rc != SBI_ENOENT)
 		return rc;
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:05 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:05 +0800
Subject: [PATCH v4 07/15] platform: andes: Add Andes custom PMU support
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-8-peterlin@andestech.com>

Before the ratification of Sscofpmf, the Andes PMU extension
was designed to support the sampling and filtering with hardware
performance counters (zihpm), it works with the current SBI PMU
extension and Linux SBI PMU driver.

We implement 1) the PMU device callbacks that update the
corresponding bits on custom CSRs, 2) extentions_init() to detect
the hardware support of Andes PMU and initialize the per-hart
PMU related CSR, and 3) pmu_init() to register PMU device and
populate event mappings.

Also define a andes_pmu_setup() function which is in preparation
for adding default PMU mappings in andes_hpm.h

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
---
Changes v1 -> v2:
  - Fix mode filtering in andes_hw_counter_filter_mode()
  - Return early if pmu is not supported in andes_pmu_init() (suggested by Prabhakar)
  - Don't grant write permissions via CSR_MCOUNTERWEN as not needed
Changes v2 -> v3:
  - Drop Anup's RB tag as we add andes_pmu_extensions_init() to initialize per-hart
    extension on scratch and move andes_pmu_init() to new added platform override pmu_init()
Changes v3 -> v4: (suggested by Samuel)
  - Add andes_hpm.h as an placeholder for andes_pmu_setup() to avoid compile failure
  - Do not call fdt_pmu_setup() in andes_pmu_init() as it will be done in generic_pmu_init()
---
 platform/generic/andes/Kconfig             |   8 ++
 platform/generic/andes/andes_pmu.c         | 105 +++++++++++++++++++++
 platform/generic/andes/objects.mk          |   1 +
 platform/generic/include/andes/andes_hpm.h |  12 +++
 platform/generic/include/andes/andes_pmu.h |  34 +++++++
 5 files changed, 160 insertions(+)
 create mode 100644 platform/generic/andes/andes_pmu.c
 create mode 100644 platform/generic/include/andes/andes_hpm.h
 create mode 100644 platform/generic/include/andes/andes_pmu.h

diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
index a91fb9c..3665b33 100644
--- a/platform/generic/andes/Kconfig
+++ b/platform/generic/andes/Kconfig
@@ -7,3 +7,11 @@ config ANDES45_PMA
 config ANDES_SBI
 	bool "Andes SBI support"
 	default n
+
+config ANDES_PMU
+	bool "Andes PMU extension (XAndesPMU) support"
+	default n
+	help
+	  Andes PMU extension supports the event counter overflow
+	  interrupt and mode filtering, similar to the standard
+	  Sscofpmf and Smcntrpmf.
diff --git a/platform/generic/andes/andes_pmu.c b/platform/generic/andes/andes_pmu.c
new file mode 100644
index 0000000..4b2d45b
--- /dev/null
+++ b/platform/generic/andes/andes_pmu.c
@@ -0,0 +1,105 @@
+// SPDX-License-Identifier: BSD-2-Clause
+/*
+ * andes_pmu.c - Andes PMU device callbacks and platform overrides
+ *
+ * Copyright (C) 2023 Andes Technology Corporation
+ */
+
+#include <andes/andes45.h>
+#include <andes/andes_hpm.h>
+#include <andes/andes_pmu.h>
+#include <sbi/sbi_bitops.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_pmu.h>
+#include <libfdt.h>
+
+static void andes_hw_counter_enable_irq(uint32_t ctr_idx)
+{
+	unsigned long mip_val;
+
+	if (ctr_idx >= SBI_PMU_HW_CTR_MAX)
+		return;
+
+	mip_val = csr_read(CSR_MIP);
+	if (!(mip_val & MIP_PMOVI))
+		csr_clear(CSR_MCOUNTEROVF, BIT(ctr_idx));
+
+	csr_set(CSR_MCOUNTERINTEN, BIT(ctr_idx));
+}
+
+static void andes_hw_counter_disable_irq(uint32_t ctr_idx)
+{
+	csr_clear(CSR_MCOUNTERINTEN, BIT(ctr_idx));
+}
+
+static void andes_hw_counter_filter_mode(unsigned long flags, int ctr_idx)
+{
+	if (flags & SBI_PMU_CFG_FLAG_SET_UINH)
+		csr_set(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
+	else
+		csr_clear(CSR_MCOUNTERMASK_U, BIT(ctr_idx));
+
+	if (flags & SBI_PMU_CFG_FLAG_SET_SINH)
+		csr_set(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
+	else
+		csr_clear(CSR_MCOUNTERMASK_S, BIT(ctr_idx));
+}
+
+static struct sbi_pmu_device andes_pmu = {
+	.name = "andes_pmu",
+	.hw_counter_enable_irq  = andes_hw_counter_enable_irq,
+	.hw_counter_disable_irq = andes_hw_counter_disable_irq,
+	/*
+	 * We set delegation of supervisor local interrupts via
+	 * 18th bit on mslideleg instead of mideleg, so leave
+	 * hw_counter_irq_bit() callback unimplemented.
+	 */
+	.hw_counter_irq_bit     = NULL,
+	.hw_counter_filter_mode = andes_hw_counter_filter_mode
+};
+
+int andes_pmu_extensions_init(const struct fdt_match *match,
+			      struct sbi_hart_features *hfeatures)
+{
+	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
+
+	if (!has_andes_pmu())
+		return 0;
+
+	/*
+	 * Don't expect both Andes PMU and standard Sscofpmf/Smcntrpmf,
+	 * are supported as they serve the same purpose.
+	 */
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCOFPMF) ||
+		sbi_hart_has_extension(scratch, SBI_HART_EXT_SMCNTRPMF))
+		return SBI_EINVAL;
+	sbi_hart_update_extension(scratch, SBI_HART_EXT_XANDESPMU, true);
+
+	/* Inhibit all HPM counters in M-mode */
+	csr_write(CSR_MCOUNTERMASK_M, 0xfffffffd);
+	/* Delegate counter overflow interrupt to S-mode */
+	csr_write(CSR_MSLIDELEG, MIP_PMOVI);
+
+	return 0;
+}
+
+int andes_pmu_init(const struct fdt_match *match)
+{
+	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
+	void *fdt = fdt_get_address();
+	int pmu_offset;
+
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_XANDESPMU))
+		sbi_pmu_set_device(&andes_pmu);
+
+	/*
+	 * Populate default mappings if device-tree doesn't
+	 * provide a valid pmu node.
+	 */
+	pmu_offset = fdt_node_offset_by_compatible(fdt, -1, "riscv,pmu");
+	if (pmu_offset < 0)
+		return (pmu_offset == -FDT_ERR_NOTFOUND) ? andes_pmu_setup()
+							 : SBI_EFAIL;
+
+	return 0;
+}
diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
index e8f86ea..6a8c66c 100644
--- a/platform/generic/andes/objects.mk
+++ b/platform/generic/andes/objects.mk
@@ -7,3 +7,4 @@ platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
 
 platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
 platform-objs-$(CONFIG_ANDES_SBI) += andes/andes_sbi.o
+platform-objs-$(CONFIG_ANDES_PMU) += andes/andes_pmu.o
diff --git a/platform/generic/include/andes/andes_hpm.h b/platform/generic/include/andes/andes_hpm.h
new file mode 100644
index 0000000..b4d71b9
--- /dev/null
+++ b/platform/generic/include/andes/andes_hpm.h
@@ -0,0 +1,12 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Andes Technology Corporation
+ */
+
+#ifndef _ANDES_HPM_H_
+#define _ANDES_HPM_H_
+
+static inline int andes_pmu_setup(void) { return 0; }
+
+#endif /* _ANDES_HPM_H_ */
diff --git a/platform/generic/include/andes/andes_pmu.h b/platform/generic/include/andes/andes_pmu.h
new file mode 100644
index 0000000..f355324
--- /dev/null
+++ b/platform/generic/include/andes/andes_pmu.h
@@ -0,0 +1,34 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) Copyright (c) 2023 Andes Technology Corporation
+ */
+
+#ifndef _RISCV_ANDES_PMU_H
+#define _RISCV_ANDES_PMU_H
+
+#include <sbi/sbi_hart.h>
+#include <sbi_utils/fdt/fdt_helper.h>
+#include <sbi_utils/fdt/fdt_pmu.h>
+
+#ifdef CONFIG_ANDES_PMU
+
+int andes_pmu_init(const struct fdt_match *match);
+int andes_pmu_extensions_init(const struct fdt_match *match,
+			      struct sbi_hart_features *hfeatures);
+
+#else
+
+static inline int andes_pmu_init(const struct fdt_match *match)
+{
+	return 0;
+}
+static inline int andes_pmu_extensions_init(const struct fdt_match *match,
+			      struct sbi_hart_features *hfeatures)
+{
+	return 0;
+}
+
+#endif /* CONFIG_ANDES_PMU */
+
+#endif /* _RISCV_ANDES_PMU_H */
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:06 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:06 +0800
Subject: [PATCH v4 08/15] platform: andes: Enable Andes PMU for AE350
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-9-peterlin@andestech.com>

Enable Andes PMU extension support for AE350 platforms.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v1 -> v2:
  - Implement ae350_extensions_init()
Changes v2 -> v3:
  - Use the andes_pmu_extensions_init() and andes_pmu_init() provided by andes_pmu.c
    We have no plan to introduce other extensions in near future so extensions_init()
    bind with andes_pmu_extensions_init() directly.
Changes v3 -> v4:
  - Include RB tags
---
 platform/generic/Kconfig       | 1 +
 platform/generic/andes/ae350.c | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index a902cf3..89e5603 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -32,6 +32,7 @@ config PLATFORM_ALLWINNER_D1
 config PLATFORM_ANDES_AE350
 	bool "Andes AE350 support"
 	select SYS_ATCSMU
+	select ANDES_PMU
 	default n
 
 config PLATFORM_RENESAS_RZFIVE
diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
index 80eca05..dc769b7 100644
--- a/platform/generic/andes/ae350.c
+++ b/platform/generic/andes/ae350.c
@@ -8,6 +8,7 @@
  */
 
 #include <platform_override.h>
+#include <andes/andes_pmu.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 #include <sbi_utils/fdt/fdt_fixup.h>
 #include <sbi_utils/sys/atcsmu.h>
@@ -118,4 +119,6 @@ static const struct fdt_match andes_ae350_match[] = {
 const struct platform_override andes_ae350 = {
 	.match_table = andes_ae350_match,
 	.final_init  = ae350_final_init,
+	.extensions_init = andes_pmu_extensions_init,
+	.pmu_init = andes_pmu_init,
 };
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:07 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:07 +0800
Subject: [PATCH v4 09/15] platform: rzfive: Enable Andes PMU for RZ/Five
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-10-peterlin@andestech.com>

Enable Andes PMU extension support for RZ/Five.
We also staticize renesas_rzfive_early_init() as
it is not used outside of this unit.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v1 -> v2:
  - Implement renesas_rzfive_extensions_init()
  - staticize renesas_rzfive_early_init()
Changes v2 -> v3:
  - Use the andes_pmu_extensions_init() and andes_pmu_init() provided by andes_pmu.c
    We have no plan to introduce other extensions in near future so extensions_init()
    bind with andes_pmu_extensions_init() directly.
Changes v3 -> v4:
  - Include RB tags
---
 platform/generic/Kconfig                 | 1 +
 platform/generic/renesas/rzfive/rzfive.c | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index 89e5603..c9b6011 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -39,6 +39,7 @@ config PLATFORM_RENESAS_RZFIVE
 	bool "Renesas RZ/Five support"
 	select ANDES45_PMA
 	select ANDES_SBI
+	select ANDES_PMU
 	default n
 
 config PLATFORM_SIFIVE_FU540
diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
index a69797b..b5b5e65 100644
--- a/platform/generic/renesas/rzfive/rzfive.c
+++ b/platform/generic/renesas/rzfive/rzfive.c
@@ -5,6 +5,7 @@
  */
 
 #include <andes/andes45_pma.h>
+#include <andes/andes_pmu.h>
 #include <andes/andes_sbi.h>
 #include <platform_override.h>
 #include <sbi/sbi_domain.h>
@@ -29,7 +30,7 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
 					 array_size(renesas_rzfive_pma_regions));
 }
 
-int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
+static int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
 {
 	/*
 	 * Renesas RZ/Five RISC-V SoC has Instruction local memory and
@@ -57,4 +58,6 @@ const struct platform_override renesas_rzfive = {
 	.early_init = renesas_rzfive_early_init,
 	.final_init = renesas_rzfive_final_init,
 	.vendor_ext_provider = andes_sbi_vendor_ext_provider,
+	.extensions_init = andes_pmu_extensions_init,
+	.pmu_init = andes_pmu_init,
 };
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:08 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:08 +0800
Subject: [PATCH v4 10/15] lib: utils: fdt_fixup: Allow preserving PMU properties
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-11-peterlin@andestech.com>

Add a Kconfig option to control PMU fixup, so the next
stage software can dump the PMU node including event
mapping information for debugging purposes.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
Changes v1 -> v2:
  - New patch
Changes v2 -> v3:
  - Rename to SBI_SCRATCH_PRESERVE_PMU_NODE (suggested by Anup)
  - Include Anup's RB tag
Changes v3 -> v4:
  - Use Kconfig instead (suggested by Anup)
---
 lib/utils/fdt/Kconfig     | 7 +++++++
 lib/utils/fdt/fdt_fixup.c | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/lib/utils/fdt/Kconfig b/lib/utils/fdt/Kconfig
index 23b003b..9a42d15 100644
--- a/lib/utils/fdt/Kconfig
+++ b/lib/utils/fdt/Kconfig
@@ -15,4 +15,11 @@ config FDT_PMU
 	bool "FDT performance monitoring unit (PMU) support"
 	default n
 
+config FDT_FIXUPS_PRESERVE_PMU_NODE
+	bool "Preserve PMU node in device-tree"
+	depends on FDT_PMU
+	default n
+	help
+	  Preserve PMU node properties for debugging purposes.
+
 endif
diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index e213ded..5fc7673 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -394,5 +394,8 @@ void fdt_fixups(void *fdt)
 	fdt_plic_fixup(fdt);
 
 	fdt_reserved_memory_fixup(fdt);
+
+#ifndef CONFIG_FDT_FIXUPS_PRESERVE_PMU_NODE
 	fdt_pmu_fixup(fdt);
+#endif
 }
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:09 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:09 +0800
Subject: [PATCH v4 11/15] platform: andes: Factor out is_andes() helper
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-12-peterlin@andestech.com>

We will need is_andes(45) in the following patch,
so factor out the code that parses marchid to make
it reusable for checking any Andes CPU variants.

Also improves the comment in ae350_hart_start().

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
Changes v1 -> v2:
  - New patch
Changes v2 -> v3:
  - Include Anup's RB tag
Changes v3 -> v4:
  - No change
---
 platform/generic/andes/ae350.c           | 16 +++++++---------
 platform/generic/include/andes/andes45.h |  6 ++++++
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
index dc769b7..088ec07 100644
--- a/platform/generic/andes/ae350.c
+++ b/platform/generic/andes/ae350.c
@@ -24,16 +24,14 @@ static struct smu_data smu = { 0 };
 extern void __ae350_enable_coherency_warmboot(void);
 extern void __ae350_disable_coherency(void);
 
-static __always_inline bool is_andes25(void)
-{
-	ulong marchid = csr_read(CSR_MARCHID);
-	return !!(EXTRACT_FIELD(marchid, CSR_MARCHID_MICROID) == 0xa25);
-}
-
 static int ae350_hart_start(u32 hartid, ulong saddr)
 {
-	/* Don't send wakeup command at boot-time */
-	if (!sbi_init_count(hartid) || (is_andes25() && hartid == 0))
+	/*
+	 * Don't send wakeup command when:
+	 * 1) boot-time
+	 * 2) the target hart is non-sleepable 25-series hart0
+	 */
+	if (!sbi_init_count(hartid) || (is_andes(25) && hartid == 0))
 		return sbi_ipi_raw_send(sbi_hartid_to_hartindex(hartid));
 
 	/* Write wakeup command to the sleep hart */
@@ -52,7 +50,7 @@ static int ae350_hart_stop(void)
 	 * L2-cache, instead of turning it off, it should fall
 	 * through and jump to warmboot_addr.
 	 */
-	if (is_andes25() && hartid == 0)
+	if (is_andes(25) && hartid == 0)
 		return SBI_ENOTSUPP;
 
 	if (!smu_support_sleep_mode(&smu, DEEPSLEEP_MODE, hartid))
diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
index ce31617..01f63d4 100644
--- a/platform/generic/include/andes/andes45.h
+++ b/platform/generic/include/andes/andes45.h
@@ -43,6 +43,12 @@
 
 #ifndef __ASSEMBLER__
 
+#define is_andes(series)				\
+({							\
+	char value = csr_read(CSR_MARCHID) & 0xff;	\
+	(series) == (value >> 4) * 10 + (value & 0x0f);	\
+})
+
 #define has_andes_pmu()					\
 ({							\
 	(((csr_read(CSR_MMSC_CFG) &			\
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:10 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:10 +0800
Subject: [PATCH v4 12/15] lib: utils: fdt_pmu: Make the fdt_pmu_evt_select table global variable
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-13-peterlin@andestech.com>

To allow platform override pmu_init() filling the translation table
fdt_pmu_evt_select[] when PMU node doesn't provide such information,
we need to share the table and its entry counter with other .c file.

We also define the structures of PMU property in fdt_helper.h, so we
can initialize the mappings in arrays.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
Changes v2 -> v3:
  - New patch
Changes v3 -> v4:
  - No change
---
 include/sbi_utils/fdt/fdt_helper.h | 17 +++++++++++++++++
 include/sbi_utils/fdt/fdt_pmu.h    |  6 ++++++
 lib/utils/fdt/fdt_pmu.c            | 13 ++++---------
 3 files changed, 27 insertions(+), 9 deletions(-)

diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
index 5c928ff..f6f505b 100644
--- a/include/sbi_utils/fdt/fdt_helper.h
+++ b/include/sbi_utils/fdt/fdt_helper.h
@@ -34,6 +34,23 @@ struct platform_uart_data {
 	unsigned long reg_offset;
 };
 
+struct fdt_pmu_hw_event_select_map {
+	uint32_t eidx;
+	uint64_t select;
+};
+
+struct fdt_pmu_hw_event_counter_map {
+	uint32_t eidx_start;
+	uint32_t eidx_end;
+	uint32_t ctr_map;
+};
+
+struct fdt_pmu_raw_event_counter_map {
+	uint64_t select;
+	uint64_t select_mask;
+	uint32_t ctr_map;
+};
+
 const struct fdt_match *fdt_match_node(void *fdt, int nodeoff,
 				       const struct fdt_match *match_table);
 
diff --git a/include/sbi_utils/fdt/fdt_pmu.h b/include/sbi_utils/fdt/fdt_pmu.h
index c65cad7..9a13487 100644
--- a/include/sbi_utils/fdt/fdt_pmu.h
+++ b/include/sbi_utils/fdt/fdt_pmu.h
@@ -12,6 +12,7 @@
 #define __FDT_PMU_H__
 
 #include <sbi/sbi_types.h>
+#include <sbi_utils/fdt/fdt_helper.h>
 
 #ifdef CONFIG_FDT_PMU
 
@@ -45,6 +46,11 @@ int fdt_pmu_setup(void *fdt);
  */
 uint64_t fdt_pmu_get_select_value(uint32_t event_idx);
 
+/** The event index to selector value table instance */
+extern struct fdt_pmu_hw_event_select_map fdt_pmu_evt_select[];
+/** The number of valid entries in fdt_pmu_evt_select[] */
+extern uint32_t hw_event_count;
+
 #else
 
 static inline void fdt_pmu_fixup(void *fdt) { }
diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
index a8d7648..cf3220f 100644
--- a/lib/utils/fdt/fdt_pmu.c
+++ b/lib/utils/fdt/fdt_pmu.c
@@ -17,18 +17,13 @@
 
 #define FDT_PMU_HW_EVENT_MAX (SBI_PMU_HW_EVENT_MAX * 2)
 
-struct fdt_pmu_hw_event_select {
-	uint32_t eidx;
-	uint64_t select;
-};
-
-static struct fdt_pmu_hw_event_select fdt_pmu_evt_select[FDT_PMU_HW_EVENT_MAX] = {0};
-static uint32_t hw_event_count;
+struct fdt_pmu_hw_event_select_map fdt_pmu_evt_select[FDT_PMU_HW_EVENT_MAX] = {0};
+uint32_t hw_event_count;
 
 uint64_t fdt_pmu_get_select_value(uint32_t event_idx)
 {
 	int i;
-	struct fdt_pmu_hw_event_select *event;
+	struct fdt_pmu_hw_event_select_map *event;
 
 	for (i = 0; i < SBI_PMU_HW_EVENT_MAX; i++) {
 		event = &fdt_pmu_evt_select[i];
@@ -65,7 +60,7 @@ int fdt_pmu_setup(void *fdt)
 	int i, pmu_offset, len, result;
 	const u32 *event_val;
 	const u32 *event_ctr_map;
-	struct fdt_pmu_hw_event_select *event;
+	struct fdt_pmu_hw_event_select_map *event;
 	uint64_t raw_selector, select_mask;
 	u32 event_idx_start, event_idx_end, ctr_map;
 
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:11 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:11 +0800
Subject: [PATCH v4 13/15] lib: utils: fdt_pmu: Do not iterate over the fdt_pmu_evt_select table
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-14-peterlin@andestech.com>

The valid entry count is tracking by hw_event_count so there
is no need to check the whole table.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
Changes v2 -> v3:
  - New patch
Changes v3 -> v4:
  - No change
---
 lib/utils/fdt/fdt_pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
index cf3220f..c531b9a 100644
--- a/lib/utils/fdt/fdt_pmu.c
+++ b/lib/utils/fdt/fdt_pmu.c
@@ -25,7 +25,7 @@ uint64_t fdt_pmu_get_select_value(uint32_t event_idx)
 	int i;
 	struct fdt_pmu_hw_event_select_map *event;
 
-	for (i = 0; i < SBI_PMU_HW_EVENT_MAX; i++) {
+	for (i = 0; i < hw_event_count; i++) {
 		event = &fdt_pmu_evt_select[i];
 		if (event->eidx == event_idx)
 			return event->select;
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:12 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:12 +0800
Subject: [PATCH v4 14/15] platform: andes: Add Andes default PMU mapping support
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-15-peterlin@andestech.com>

Provide the andes_pmu_setup() to populate default mappings in
hw_event_map[] and fdt_pmu_evt_select[] when fdt_pmu_setup()
cannot find the PMU node.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v2 -> v3:
  - New patch
  - Drop fdt_add_pmu_mappings() (suggested by Anup [1])
Changes v3 -> v4:
  - Include TB tag from Prabhakar
  - Staticize andes_pmu_setup() when CONFIG_PLATFORM_ANDES_AE350 is not set

[1] https://patchwork.ozlabs.org/project/opensbi/patch/20231019113713.3508153-11-peterlin at andestech.com/
---
 include/sbi/sbi_ecall_interface.h          |   5 +
 platform/generic/Kconfig                   |   2 +
 platform/generic/andes/Kconfig             |   7 +
 platform/generic/andes/andes_hpm.c         | 405 +++++++++++++++++++++
 platform/generic/andes/objects.mk          |   1 +
 platform/generic/include/andes/andes_hpm.h |  69 ++++
 6 files changed, 489 insertions(+)
 create mode 100644 platform/generic/andes/andes_hpm.c

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index 1fe469e..89187e7 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -155,6 +155,11 @@ enum sbi_pmu_hw_cache_op_result_id {
 	SBI_PMU_HW_CACHE_RESULT_MAX,
 };
 
+#define SBI_PMU_HW_CACHE_EVENT_IDX(id, op, res) \
+	(SBI_PMU_EVENT_TYPE_HW_CACHE << SBI_PMU_EVENT_IDX_TYPE_OFFSET | \
+	SBI_PMU_HW_CACHE_##id << 3 | SBI_PMU_HW_CACHE_OP_##op << 1 | \
+	SBI_PMU_HW_CACHE_RESULT_##res)
+
 /**
  * Special "firmware" events provided by the OpenSBI, even if the hardware
  * does not support performance events. These events are encoded as a raw
diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index c9b6011..e652645 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -32,6 +32,7 @@ config PLATFORM_ALLWINNER_D1
 config PLATFORM_ANDES_AE350
 	bool "Andes AE350 support"
 	select SYS_ATCSMU
+	select ANDES_HPM
 	select ANDES_PMU
 	default n
 
@@ -39,6 +40,7 @@ config PLATFORM_RENESAS_RZFIVE
 	bool "Renesas RZ/Five support"
 	select ANDES45_PMA
 	select ANDES_SBI
+	select ANDES_HPM
 	select ANDES_PMU
 	default n
 
diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
index 3665b33..eb697a6 100644
--- a/platform/generic/andes/Kconfig
+++ b/platform/generic/andes/Kconfig
@@ -8,6 +8,13 @@ config ANDES_SBI
 	bool "Andes SBI support"
 	default n
 
+config ANDES_HPM
+	bool "Andes HPM support"
+	default n
+	help
+	  This provides default HPM event counter mappings
+	  when a pmu node is missing.
+
 config ANDES_PMU
 	bool "Andes PMU extension (XAndesPMU) support"
 	default n
diff --git a/platform/generic/andes/andes_hpm.c b/platform/generic/andes/andes_hpm.c
new file mode 100644
index 0000000..7fe5ac5
--- /dev/null
+++ b/platform/generic/andes/andes_hpm.c
@@ -0,0 +1,405 @@
+// SPDX-License-Identifier: BSD-2-Clause
+/*
+ * andes_hpm.c - Default event mappings and helper routine
+ *
+ * Copyright (c) 2023 Andes Technology Corporation
+ */
+
+#include <andes/andes45.h>
+#include <andes/andes_hpm.h>
+#include <sbi/sbi_ecall_interface.h>
+#include <sbi/sbi_pmu.h>
+#include <sbi_utils/fdt/fdt_pmu.h>
+
+static const struct fdt_pmu_hw_event_select_map andes45_hw_evt_selects[] = {
+	/* Hardware general events (Type #0) */
+	{
+		/* perf: cycles (eidx: 0x1) */
+		.eidx = SBI_PMU_HW_CPU_CYCLES,
+		.select = ANDES_CYCLES
+	},
+	{
+		/* perf: instructions (eidx: 0x2) */
+		.eidx = SBI_PMU_HW_INSTRUCTIONS,
+		.select = ANDES_INSTRET
+	},
+	{
+		/* perf: cache-references (eidx: 0x3) */
+		.eidx = SBI_PMU_HW_CACHE_REFERENCES,
+		.select = ANDES_DCACHE_ACCESS
+	},
+	{
+		/* perf: cache-misses (eidx: 0x4) */
+		.eidx = SBI_PMU_HW_CACHE_MISSES,
+		.select = ANDES_DCACHE_MISS
+	},
+	{
+		/* perf: branches (eidx: 0x5) */
+		.eidx = SBI_PMU_HW_BRANCH_INSTRUCTIONS,
+		.select = ANDES_CONDITION_BR,
+	},
+	{
+		/* perf: branch-misses (eidx: 0x6) */
+		.eidx = SBI_PMU_HW_BRANCH_MISSES,
+		.select = ANDES_MISPREDICT_CONDITION_BR,
+	},
+	/* Hardware cache events (Type #1) */
+	{
+		/* perf: L1-dcache-loads (eidx: 0x10000) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, ACCESS),
+		.select = ANDES_DCACHE_LOAD_ACCESS
+	},
+	{
+		/* perf: L1-dcache-loads-misses (eidx: 0x10001) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, MISS),
+		.select = ANDES_DCACHE_LOAD_MISS
+	},
+	{
+		/* perf: L1-dcache-stores (eidx: 0x10002) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, ACCESS),
+		.select = ANDES_DCACHE_STORE_ACCESS
+	},
+	{
+		/* perf: L1-dcache-store-misses (eidx: 0x10003) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, MISS),
+		.select = ANDES_DCACHE_STORE_MISS
+	},
+	{
+		/* perf: L1-icache-load (eidx: 0x10008) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, ACCESS),
+		.select = ANDES_ICACHE_ACCESS
+	},
+	{
+		/* perf: L1-icache-load-misses (eidx: 0x10009) */
+		.eidx = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, MISS),
+		.select = ANDES_ICACHE_MISS
+	},
+};
+
+static const struct fdt_pmu_hw_event_counter_map andes45_hw_evt_counters[] = {
+	{
+		/* perf: cycles (eidx: 0x1) */
+		.eidx_start = SBI_PMU_HW_CPU_CYCLES,
+		/* perf: branch-misses (eidx: 0x6) */
+		.eidx_end = SBI_PMU_HW_BRANCH_MISSES,
+		.ctr_map  = ANDES_MHPM_MAP,
+	},
+	{
+		/* perf: L1-dcache-loads (eidx: 0x10000) */
+		.eidx_start = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, READ, ACCESS),
+		/* perf: L1-dcache-store-misses (eidx: 0x10003) */
+		.eidx_end = SBI_PMU_HW_CACHE_EVENT_IDX(L1D, WRITE, MISS),
+		.ctr_map  = ANDES_MHPM_MAP,
+	},
+	{
+		/* perf: L1-icache-load (eidx: 0x10008) */
+		.eidx_start = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, ACCESS),
+		/* perf: L1-icache-load-misses (eidx: 0x10009) */
+		.eidx_end = SBI_PMU_HW_CACHE_EVENT_IDX(L1I, READ, MISS),
+		.ctr_map  = ANDES_MHPM_MAP,
+	},
+};
+
+static const struct fdt_pmu_raw_event_counter_map andes45_raw_evt_counters[] = {
+	{
+		.select = ANDES_CYCLES,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INSTRET,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_LOAD_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_STORE_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_ATOMIC_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_SYS_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_COMPUTE_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CONDITION_BR,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_TAKEN_CONDITION_BR,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_JAL_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_JALR_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_RET_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CONTROL_TRANS_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_EX9_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_MUL_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_DIV_REMAINDER_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_LOAD_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_STORE_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_ADD_SUB_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_MUL_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_FUSED_MULADD_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_FLOAT_DIV_SQUARE_ROOT_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_OTHER_FLOAT_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_INT_MUL_AND_SUB_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_RETIRED_OP,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_ILM_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DLM_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_ICACHE_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_ICACHE_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_LOAD_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_LOAD_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_STORE_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_STORE_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_DCACHE_WB,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_ICACHE_FILL,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_DCACHE_FILL,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_UNCACHED_IFETCH_FROM_BUS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_UNCACHED_LOAD_FROM_BUS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_UNCACHED_IFETCH,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_UNCACHED_LOAD,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MAIN_ITLB_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MAIN_ITLB_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MAIN_DTLB_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MAIN_DTLB_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_CYCLE_WAIT_ITLB_FILL,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_PIPE_STALL_CYCLE_DTLB_MISS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_HW_PREFETCH_BUS_ACCESS,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MISPREDICT_CONDITION_BR,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MISPREDICT_TAKE_CONDITION_BR,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+	{
+		.select = ANDES_MISPREDICT_TARGET_RET_INST,
+		.select_mask = ANDES_RAW_EVENT_MASK,
+		.ctr_map = ANDES_MHPM_MAP
+	},
+};
+
+int andes_pmu_setup(void)
+{
+	int i, rc;
+	struct fdt_pmu_hw_event_select_map *event;
+
+	/*
+	 * At the moment, simply create mapping for any 45-series core
+	 * based on marchid, we may check and differentiate the mapping
+	 * by mimpid.
+	 */
+	if (!is_andes(45))
+		return 0;
+
+	for (i = 0; i < array_size(andes45_hw_evt_counters); i++) {
+		rc = sbi_pmu_add_hw_event_counter_map(
+			andes45_hw_evt_counters[i].eidx_start,
+			andes45_hw_evt_counters[i].eidx_end,
+			andes45_hw_evt_counters[i].ctr_map);
+		if (rc)
+			return rc;
+	}
+
+	for (i = 0; i < array_size(andes45_hw_evt_selects); i++) {
+		event	      = &fdt_pmu_evt_select[hw_event_count];
+		event->eidx   = andes45_hw_evt_selects[i].eidx;
+		event->select = andes45_hw_evt_selects[i].select;
+		hw_event_count++;
+	}
+
+	for (i = 0; i < array_size(andes45_raw_evt_counters); i++) {
+		rc = sbi_pmu_add_raw_event_counter_map(
+			andes45_raw_evt_counters[i].select,
+			andes45_raw_evt_counters[i].select_mask,
+			andes45_raw_evt_counters[i].ctr_map);
+		if (rc)
+			return rc;
+	}
+
+	return 0;
+}
diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
index 6a8c66c..57caaf6 100644
--- a/platform/generic/andes/objects.mk
+++ b/platform/generic/andes/objects.mk
@@ -8,3 +8,4 @@ platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
 platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
 platform-objs-$(CONFIG_ANDES_SBI) += andes/andes_sbi.o
 platform-objs-$(CONFIG_ANDES_PMU) += andes/andes_pmu.o
+platform-objs-$(CONFIG_ANDES_HPM) += andes/andes_hpm.o
diff --git a/platform/generic/include/andes/andes_hpm.h b/platform/generic/include/andes/andes_hpm.h
index b4d71b9..0ae391f 100644
--- a/platform/generic/include/andes/andes_hpm.h
+++ b/platform/generic/include/andes/andes_hpm.h
@@ -7,6 +7,75 @@
 #ifndef _ANDES_HPM_H_
 #define _ANDES_HPM_H_
 
+#define ANDES_MHPM_MAP		0x78
+#define ANDES_RAW_EVENT_MASK	~0ULL
+
+/* Event selector for instruction commit events */
+#define ANDES_CYCLES				0x10
+#define ANDES_INSTRET				0x20
+#define ANDES_INT_LOAD_INST			0x30
+#define ANDES_INT_STORE_INST			0x40
+#define ANDES_ATOMIC_INST			0x50
+#define ANDES_SYS_INST				0x60
+#define ANDES_INT_COMPUTE_INST			0x70
+#define ANDES_CONDITION_BR			0x80
+#define ANDES_TAKEN_CONDITION_BR		0x90
+#define ANDES_JAL_INST				0xA0
+#define ANDES_JALR_INST				0xB0
+#define ANDES_RET_INST				0xC0
+#define ANDES_CONTROL_TRANS_INST		0xD0
+#define ANDES_EX9_INST				0xE0
+#define ANDES_INT_MUL_INST			0xF0
+#define ANDES_INT_DIV_REMAINDER_INST		0x100
+#define ANDES_FLOAT_LOAD_INST			0x110
+#define ANDES_FLOAT_STORE_INST			0x120
+#define ANDES_FLOAT_ADD_SUB_INST		0x130
+#define ANDES_FLOAT_MUL_INST			0x140
+#define ANDES_FLOAT_FUSED_MULADD_INST		0x150
+#define ANDES_FLOAT_DIV_SQUARE_ROOT_INST	0x160
+#define ANDES_OTHER_FLOAT_INST			0x170
+#define ANDES_INT_MUL_AND_SUB_INST		0x180
+#define ANDES_RETIRED_OP			0x190
+
+/* Event selector for memory system events */
+#define ANDES_ILM_ACCESS			0x01
+#define ANDES_DLM_ACCESS			0x11
+#define ANDES_ICACHE_ACCESS			0x21
+#define ANDES_ICACHE_MISS			0x31
+#define ANDES_DCACHE_ACCESS			0x41
+#define ANDES_DCACHE_MISS			0x51
+#define ANDES_DCACHE_LOAD_ACCESS		0x61
+#define ANDES_DCACHE_LOAD_MISS			0x71
+#define ANDES_DCACHE_STORE_ACCESS		0x81
+#define ANDES_DCACHE_STORE_MISS			0x91
+#define ANDES_DCACHE_WB				0xA1
+#define ANDES_CYCLE_WAIT_ICACHE_FILL		0xB1
+#define ANDES_CYCLE_WAIT_DCACHE_FILL		0xC1
+#define ANDES_UNCACHED_IFETCH_FROM_BUS		0xD1
+#define ANDES_UNCACHED_LOAD_FROM_BUS		0xE1
+#define ANDES_CYCLE_WAIT_UNCACHED_IFETCH	0xF1
+#define ANDES_CYCLE_WAIT_UNCACHED_LOAD		0x101
+#define ANDES_MAIN_ITLB_ACCESS			0x111
+#define ANDES_MAIN_ITLB_MISS			0x121
+#define ANDES_MAIN_DTLB_ACCESS			0x131
+#define ANDES_MAIN_DTLB_MISS			0x141
+#define ANDES_CYCLE_WAIT_ITLB_FILL		0x151
+#define ANDES_PIPE_STALL_CYCLE_DTLB_MISS	0x161
+#define ANDES_HW_PREFETCH_BUS_ACCESS		0x171
+
+/* Event selector for microarchitecture events */
+#define ANDES_MISPREDICT_CONDITION_BR		0x02
+#define ANDES_MISPREDICT_TAKE_CONDITION_BR	0x12
+#define ANDES_MISPREDICT_TARGET_RET_INST	0x22
+
+#ifdef CONFIG_ANDES_HPM
+
+int andes_pmu_setup(void);
+
+#else
+
 static inline int andes_pmu_setup(void) { return 0; }
 
+#endif /* CONFIG_ANDES_HPM */
+
 #endif /* _ANDES_HPM_H_ */
-- 
2.34.1



From peterlin at andestech.com  Thu Nov 30 04:42:13 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Thu, 30 Nov 2023 20:42:13 +0800
Subject: [PATCH v4 15/15] docs: pmu: Add Andes PMU node example
In-Reply-To: <20231130124213.2590640-1-peterlin@andestech.com>
References: <20231130124213.2590640-1-peterlin@andestech.com>
Message-ID: <20231130124213.2590640-16-peterlin@andestech.com>

Add PMU node example for event index to counter index mapping
and selector value translation of Andes' CPUs.

Currently, there are 4 HPM counters that can be used to monitor
all of the events for each hart.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Locus Wei-Han Chen <locus84 at andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang at andestech.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
Changes v1 -> v2:
  - sync up with datasheet
Changes v2 -> v3:
  - Include Anup's RB tag
Changes v3 -> v4:
  - Include Prabhakar's RB tag
---
 docs/pmu_support.md | 82 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/docs/pmu_support.md b/docs/pmu_support.md
index 8cfa08c..9b48f1e 100644
--- a/docs/pmu_support.md
+++ b/docs/pmu_support.md
@@ -125,3 +125,85 @@ pmu {
 					  <0x0 0x2 0xffffffff 0xffffe0ff 0x18>;
 };
 ```
+
+### Example 3
+
+```
+/*
+ * For Andes 45-series platforms. The encodings can be found in the
+ * "Machine Performance Monitoring Event Selector" section
+ * http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
+ */
+pmu {
+	compatible 			= "riscv,pmu";
+	riscv,event-to-mhpmevent =
+					 <0x1 0x0000 0x10>, /* CPU_CYCLES          -> Cycle count */
+					 <0x2 0x0000 0x20>, /* INSTRUCTIONS        -> Retired instruction count */
+					 <0x3 0x0000 0x41>, /* CACHE_REFERENCES    -> D-Cache access */
+					 <0x4 0x0000 0x51>, /* CACHE_MISSES        -> D-Cache miss */
+					 <0x5 0x0000 0x80>, /* BRANCH_INSTRUCTIONS -> Conditional branch instruction count */
+					 <0x6 0x0000 0x02>, /* BRANCH_MISSES       -> Misprediction of conditional branches */
+					 <0x10000 0x0000 0x61>,  /* L1D_READ_ACCESS  -> D-Cache load access */
+					 <0x10001 0x0000 0x71>,  /* L1D_READ_MISS    -> D-Cache load miss */
+					 <0x10002 0x0000 0x81>,  /* L1D_WRITE_ACCESS -> D-Cache store access */
+					 <0x10003 0x0000 0x91>,  /* L1D_WRITE_MISS   -> D-Cache store miss */
+					 <0x10008 0x0000 0x21>,  /* L1I_READ_ACCESS  -> I-Cache access */
+					 <0x10009 0x0000 0x31>;  /* L1I_READ_MISS    -> I-Cache miss */
+	riscv,event-to-mhpmcounters = <0x1 0x6 0x78>,
+							<0x10000 0x10003 0x78>,
+							<0x10008 0x10009 0x78>;
+	riscv,raw-event-to-mhpmcounters =
+						<0x0 0x10 0xffffffff 0xffffffff 0x78>, /* Cycle count */
+						<0x0 0x20 0xffffffff 0xffffffff 0x78>, /* Retired instruction count */
+						<0x0 0x30 0xffffffff 0xffffffff 0x78>, /* Integer load instruction count */
+						<0x0 0x40 0xffffffff 0xffffffff 0x78>, /* Integer store instruction count */
+						<0x0 0x50 0xffffffff 0xffffffff 0x78>, /* Atomic instruction count */
+						<0x0 0x60 0xffffffff 0xffffffff 0x78>, /* System instruction count */
+						<0x0 0x70 0xffffffff 0xffffffff 0x78>, /* Integer computational instruction count */
+						<0x0 0x80 0xffffffff 0xffffffff 0x78>, /* Conditional branch instruction count */
+						<0x0 0x90 0xffffffff 0xffffffff 0x78>, /* Taken conditional branch instruction count */
+						<0x0 0xA0 0xffffffff 0xffffffff 0x78>, /* JAL instruction count */
+						<0x0 0xB0 0xffffffff 0xffffffff 0x78>, /* JALR instruction count */
+						<0x0 0xC0 0xffffffff 0xffffffff 0x78>, /* Return instruction count */
+						<0x0 0xD0 0xffffffff 0xffffffff 0x78>, /* Control transfer instruction count */
+						<0x0 0xE0 0xffffffff 0xffffffff 0x78>, /* EXEC.IT instruction count */
+						<0x0 0xF0 0xffffffff 0xffffffff 0x78>, /* Integer multiplication instruction count */
+						<0x0 0x100 0xffffffff 0xffffffff 0x78>, /* Integer division instruction count */
+						<0x0 0x110 0xffffffff 0xffffffff 0x78>, /* Floating-point load instruction count */
+						<0x0 0x120 0xffffffff 0xffffffff 0x78>, /* Floating-point store instruction count */
+						<0x0 0x130 0xffffffff 0xffffffff 0x78>, /* Floating-point addition/subtraction instruction count */
+						<0x0 0x140 0xffffffff 0xffffffff 0x78>, /* Floating-point multiplication instruction count */
+						<0x0 0x150 0xffffffff 0xffffffff 0x78>, /* Floating-point fused multiply-add instruction count */
+						<0x0 0x160 0xffffffff 0xffffffff 0x78>, /* Floating-point division or square-root instruction count */
+						<0x0 0x170 0xffffffff 0xffffffff 0x78>, /* Other floating-point instruction count */
+						<0x0 0x180 0xffffffff 0xffffffff 0x78>, /* Integer multiplication and add/sub instruction count */
+						<0x0 0x190 0xffffffff 0xffffffff 0x78>, /* Retired operation count */
+						<0x0 0x01 0xffffffff 0xffffffff 0x78>, /* ILM access */
+						<0x0 0x11 0xffffffff 0xffffffff 0x78>, /* DLM access */
+						<0x0 0x21 0xffffffff 0xffffffff 0x78>, /* I-Cache access */
+						<0x0 0x31 0xffffffff 0xffffffff 0x78>, /* I-Cache miss */
+						<0x0 0x41 0xffffffff 0xffffffff 0x78>, /* D-Cache access */
+						<0x0 0x51 0xffffffff 0xffffffff 0x78>, /* D-Cache miss */
+						<0x0 0x61 0xffffffff 0xffffffff 0x78>, /* D-Cache load access */
+						<0x0 0x71 0xffffffff 0xffffffff 0x78>, /* D-Cache load miss */
+						<0x0 0x81 0xffffffff 0xffffffff 0x78>, /* D-Cache store access */
+						<0x0 0x91 0xffffffff 0xffffffff 0x78>, /* D-Cache store miss */
+						<0x0 0xA1 0xffffffff 0xffffffff 0x78>, /* D-Cache writeback */
+						<0x0 0xB1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for I-Cache fill data */
+						<0x0 0xC1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for D-Cache fill data */
+						<0x0 0xD1 0xffffffff 0xffffffff 0x78>, /* Uncached fetch data access from bus */
+						<0x0 0xE1 0xffffffff 0xffffffff 0x78>, /* Uncached load data access from bus */
+						<0x0 0xF1 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for uncached fetch data from bus */
+						<0x0 0x101 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for uncached load data from bus */
+						<0x0 0x111 0xffffffff 0xffffffff 0x78>, /* Main ITLB access */
+						<0x0 0x121 0xffffffff 0xffffffff 0x78>, /* Main ITLB miss */
+						<0x0 0x131 0xffffffff 0xffffffff 0x78>, /* Main DTLB access */
+						<0x0 0x141 0xffffffff 0xffffffff 0x78>, /* Main DTLB miss */
+						<0x0 0x151 0xffffffff 0xffffffff 0x78>, /* Cycles waiting for Main ITLB fill data */
+						<0x0 0x161 0xffffffff 0xffffffff 0x78>, /* Pipeline stall cycles caused by Main DTLB miss */
+						<0x0 0x171 0xffffffff 0xffffffff 0x78>, /* Hardware prefetch bus access */
+						<0x0 0x02 0xffffffff 0xffffffff 0x78>, /* Misprediction of conditional branches */
+						<0x0 0x12 0xffffffff 0xffffffff 0x78>, /* Misprediction of taken conditional branches */
+						<0x0 0x22 0xffffffff 0xffffffff 0x78>; /* Misprediction of targets of Return instructions */
+};
+```
-- 
2.34.1



From debug at rivosinc.com  Thu Nov 30 10:13:51 2023
From: debug at rivosinc.com (Deepak Gupta)
Date: Thu, 30 Nov 2023 10:13:51 -0800
Subject: [RFC PATCH 0/2] Add support for Supervisor Software Events
 extension
In-Reply-To: <20231130091319.1223849-1-cleger@rivosinc.com>
References: <20231130091319.1223849-1-cleger@rivosinc.com>
Message-ID: <ZWjQ39BZPzpQfMhz@debug.ba.rivosinc.com>

On Thu, Nov 30, 2023 at 10:13:17AM +0100, Cl?ment L?ger wrote:
>The SBI Supervisor Software Events (SSE) extensions provides a mechanism
>to inject software events from an SBI implementation to supervisor
>software such that it preempts all other supervisor level traps and
>interrupts [1].
>
>Various events are defined and can be send asynchronously to supervisor
>software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
>as platform specific events. Events can be either local (per-hart) or
>global. Events can be nested on top of each other based on priority and
>can interrupt the supervisor mode at any time.
>
>This PR adds support for this extension. First commit modifies interrupts
>handling to pass the sbi traps regs through interrupt handlers in order to inject
>SSE events from this context. Second one add the SSE support itself.

Cross-posting my response from github


I've a general comment on design of SSE which interferes with security.
This is kind of a TOCTOU issue between kernel (less priv) and opensbi (more priv)

If a control flow integrity scheme (software or hardware assisted) is implemented
to ensure control safety of kernel indirect branches, then SSE provides a mechanism
using an attacker can exercise free branches. I believe software base kCFI support
for riscv is already in mainline.

During time of event registration, kernel provides physical address of context. This
context structure acts as context to be used for event inject in S (and along with PC,
SP and other registers) Context to be used where interrupted context of S will be saved
away so that it can be restored later once injected event has finished its course This
entire context buffer is mapped in kernel space and thus is vulnerable to memory corruption.
If any CFI scheme is implemented, its expected that an attacker has "write what anywhere" bug.
Thus this design allows an attacker to corrupt this context and use mechanisms to trigger events
that will make M mode to inject events in kernel and thereby allowing free branches (which goes
against CFI schemes ) in kernel.

Without understanding limitations of opensbi, I would like to propose a solution

During registration, opensbi makes a copy of context structure.
Similarly interrupted context can also live in protected region in M mode memory
Let me know what you think.

>
>A RFC for Linux which uses this extension is available at [2].
>
>Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
>Link: https://lore.kernel.org/linux-riscv/20231026143122.279437-1-cleger at rivosinc.com/ [2]
>
>Cl?ment L?ger (2):
>  lib: sbi: provides regs to sbi_ipi_process()
>  lib: sbi: add support for Supervisor Software Events extension
>
> include/sbi/sbi_ecall_interface.h |  36 +-
> include/sbi/sbi_error.h           |   4 +
> include/sbi/sbi_ipi.h             |   6 +-
> include/sbi/sbi_sse.h             | 222 +++++++
> lib/sbi/Kconfig                   |   4 +
> lib/sbi/objects.mk                |   4 +
> lib/sbi/sbi_ecall.c               |   7 +-
> lib/sbi/sbi_ecall_sse.c           |  61 ++
> lib/sbi/sbi_init.c                |  13 +
> lib/sbi/sbi_ipi.c                 |  12 +-
> lib/sbi/sbi_sse.c                 | 963 ++++++++++++++++++++++++++++++
> lib/sbi/sbi_tlb.c                 |   2 +-
> lib/sbi/sbi_trap.c                |   4 +-
> lib/utils/irqchip/imsic.c         |   2 +-
> 14 files changed, 1327 insertions(+), 13 deletions(-)
> create mode 100644 include/sbi/sbi_sse.h
> create mode 100644 lib/sbi/sbi_ecall_sse.c
> create mode 100644 lib/sbi/sbi_sse.c
>
>-- 
>2.42.0
>


From jrtc27 at jrtc27.com  Thu Nov 30 10:18:05 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Thu, 30 Nov 2023 18:18:05 +0000
Subject: [RFC PATCH 0/2] Add support for Supervisor Software Events
 extension
In-Reply-To: <20231130091319.1223849-1-cleger@rivosinc.com>
References: <20231130091319.1223849-1-cleger@rivosinc.com>
Message-ID: <977FFDD0-F1AD-4E00-88BD-D9070ED8B1E9@jrtc27.com>

On 30 Nov 2023, at 09:13, Cl?ment L?ger <cleger at rivosinc.com> wrote:
> 
> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
> to inject software events from an SBI implementation to supervisor
> software such that it preempts all other supervisor level traps and
> interrupts [1].
> 
> Various events are defined and can be send asynchronously to supervisor
> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
> as platform specific events. Events can be either local (per-hart) or
> global. Events can be nested on top of each other based on priority and
> can interrupt the supervisor mode at any time.
> 
> This PR adds support for this extension. First commit modifies interrupts
> handling to pass the sbi traps regs through interrupt handlers in order to inject
> SSE events from this context. Second one add the SSE support itself.
> 
> A RFC for Linux which uses this extension is available at [2].
> 
> Link: https://lists.riscv.org/g/tech-prs/message/515 [

This is the proposal from many months ago which I objected to. Have
there been any changes to the spec to not tie the SBI?s binary
interface too closely to the ISA and preclude the existence of
extensions that introduce new state SSE would need to save and restore?
I was under the impression from your talk a few weeks ago at LPC, which
even mentioned my comments explicitly, that was being worked on, yet
here we are with a new patch that doesn?t change the spec one bit.

Jess



