From apatel at ventanamicro.com  Wed Feb  1 00:43:57 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Wed, 1 Feb 2023 14:13:57 +0530
Subject: [PATCH 0/2] Not to clear all the bits of mip
In-Reply-To: <CAKddAkDtLOwsT9XTOPY3zpO+h78zGVFx=PtEn-+jafab69ON_Q@mail.gmail.com>
References: <20230117081428.15042-1-nick.hu@sifive.com> <CAKddAkDtLOwsT9XTOPY3zpO+h78zGVFx=PtEn-+jafab69ON_Q@mail.gmail.com>
Message-ID: <CAK9=C2WGnMueTK-aX3LY_emC1_RqPxFAA3szc3S-=gsc3FC9zw@mail.gmail.com>

On Wed, Feb 1, 2023 at 1:52 PM Nick Hu <nick.hu at sifive.com> wrote:
>
> On Tue, Jan 17, 2023 at 4:14 PM Nick Hu <nick.hu at sifive.com> wrote:
>>
>> In generic behavior of QEMU, if we clear the mip.SEIP and the pendings of
>> PLIC still exist, the QEMU may not set the mip.SEIP back immediately. So
>> interrupts won't be handled anymore until the next new device interrupt
>> arrived. We should only clear the SSIP and STIP which set by SW. The rest
>> of the bits should be decided by HW.
>>
>> Nick Hu (2):
>>   firmware: Not to clear all the MIP
>>   lib: sbi_hsm: Use csr_set to restore the MIP
>>
>>  firmware/fw_base.S | 10 ++++++++--
>>  lib/sbi/sbi_hsm.c  |  2 +-
>>  2 files changed, 9 insertions(+), 3 deletions(-)
>>
>> --
>> 2.34.1
>>
> Hi Anup,
>
> Do you have any suggestions on the patches?

This is next on my list. I will review it soon.

Regards,
Anup


From ajones at ventanamicro.com  Wed Feb  1 00:48:09 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 1 Feb 2023 09:48:09 +0100
Subject: [PATCH v5 4/8] lib: sbi: Add sbi_domain_check_addr_range()
 function
In-Reply-To: <20230113114110.1916226-5-apatel@ventanamicro.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-5-apatel@ventanamicro.com>
Message-ID: <20230201084809.42zmyvthwizbggps@orel>

On Fri, Jan 13, 2023 at 05:11:06PM +0530, Anup Patel wrote:
> We add sbi_domain_check_addr_range() helper function to check
> whether a given address range is accessible under a particular
> domain.
> 
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> Reviewed-by: Atish Patra <atishp at rivosinc.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Bin Meng <bmeng at tinylab.org>
> ---
>  include/sbi/sbi_domain.h | 15 +++++++++
>  lib/sbi/sbi_domain.c     | 69 ++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 84 insertions(+)
> 
> diff --git a/include/sbi/sbi_domain.h b/include/sbi/sbi_domain.h
> index bbb3eff..ab1a944 100644
> --- a/include/sbi/sbi_domain.h
> +++ b/include/sbi/sbi_domain.h
> @@ -196,6 +196,21 @@ bool sbi_domain_check_addr(const struct sbi_domain *dom,
>  			   unsigned long addr, unsigned long mode,
>  			   unsigned long access_flags);
>  
> +/**
> + * Check whether we can access specified address range for given mode and
> + * memory region flags under a domain
> + * @param dom pointer to domain
> + * @param addr the start of the address range to be checked
> + * @param size the size of the address range to be checked
> + * @param mode the privilege mode of access
> + * @param access_flags bitmask of domain access types (enum sbi_domain_access)
> + * @return TRUE if access allowed otherwise FALSE
> + */
> +bool sbi_domain_check_addr_range(const struct sbi_domain *dom,
> +				 unsigned long addr, unsigned long size,
> +				 unsigned long mode,
> +				 unsigned long access_flags);
> +
>  /** Dump domain details on the console */
>  void sbi_domain_dump(const struct sbi_domain *dom, const char *suffix);
>  
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index 60fda01..683b6cf 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -208,6 +208,44 @@ static bool is_region_before(const struct sbi_domain_memregion *regA,
>  	return false;
>  }
>  
> +static const struct sbi_domain_memregion *find_region(
> +						const struct sbi_domain *dom,
> +						unsigned long addr)
> +{
> +	unsigned long rstart, rend;
> +	struct sbi_domain_memregion *reg;
> +
> +	sbi_domain_for_each_memregion(dom, reg) {
> +		rstart = reg->base;
> +		rend = (reg->order < __riscv_xlen) ?
> +			rstart + ((1UL << reg->order) - 1) : -1UL;
> +		if (rstart <= addr && addr <= rend)
> +			return reg;
> +	}
> +
> +	return NULL;
> +}
> +
> +static const struct sbi_domain_memregion *find_next_subset_region(
> +				const struct sbi_domain *dom,
> +				const struct sbi_domain_memregion *reg,
> +				unsigned long addr)
> +{
> +	struct sbi_domain_memregion *sreg, *ret = NULL;
> +
> +	sbi_domain_for_each_memregion(dom, sreg) {
> +		if (sreg == reg || (sreg->base <= addr) ||
> +		    !is_region_subset(sreg, reg))
> +			continue;
> +
> +		if (!ret || (sreg->base < ret->base) ||
> +		    ((sreg->base == ret->base) && (sreg->order < ret->order)))
> +			ret = sreg;
> +	}
> +
> +	return ret;
> +}
> +
>  static int sanitize_domain(const struct sbi_platform *plat,
>  			   struct sbi_domain *dom)
>  {
> @@ -316,6 +354,37 @@ static int sanitize_domain(const struct sbi_platform *plat,
>  	return 0;
>  }
>  
> +bool sbi_domain_check_addr_range(const struct sbi_domain *dom,
> +				 unsigned long addr, unsigned long size,
> +				 unsigned long mode,
> +				 unsigned long access_flags)
> +{
> +	unsigned long max = addr + size;
> +	const struct sbi_domain_memregion *reg, *sreg;
> +
> +	if (!dom)
> +		return false;
> +
> +	while (addr < max) {
> +		reg = find_region(dom, addr);
> +		if (!reg)
> +			return false;
> +
> +		if (!sbi_domain_check_addr(dom, addr, mode, access_flags))
> +			return false;
> +
> +		sreg = find_next_subset_region(dom, reg, addr);
> +		if (sreg)
> +			addr = sreg->base;
> +		else if (reg->order < __riscv_xlen)
> +			addr = reg->base + (1UL << reg->order);
> +		else
> +			break;
> +	}
> +
> +	return true;
> +}
> +

It seems like there should be a way to optimize this check / these
searches, but I suppose we can do that later.

Reviewed-by: Andrew Jones <ajones at ventanamicro.com>

Thanks,
drew


From ajones at ventanamicro.com  Wed Feb  1 00:58:15 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 1 Feb 2023 09:58:15 +0100
Subject: [PATCH v5 5/8] lib: sbi: Implement SBI debug console extension
In-Reply-To: <20230113114110.1916226-6-apatel@ventanamicro.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-6-apatel@ventanamicro.com>
Message-ID: <20230201085815.77wcmkw76l2in5qc@orel>

On Fri, Jan 13, 2023 at 05:11:07PM +0530, Anup Patel wrote:
> We implement SBI debug console extension as one of the replacement
> SBI extensions. This extension is only available when OpenSBI platform
> provides a console device to generic library.
> 
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> Reviewed-by: Atish Patra <atishp at rivosinc.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Bin Meng <bmeng at tinylab.org>
> ---
>  lib/sbi/Kconfig          |  4 +++
>  lib/sbi/objects.mk       |  3 ++
>  lib/sbi/sbi_ecall_dbcn.c | 72 ++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 79 insertions(+)
>  create mode 100644 lib/sbi/sbi_ecall_dbcn.c
> 
> diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
> index df74bba..ef6728b 100644
> --- a/lib/sbi/Kconfig
> +++ b/lib/sbi/Kconfig
> @@ -26,6 +26,10 @@ config SBI_ECALL_PMU
>  	bool "Performance Monitoring Unit extension"
>  	default y
>  
> +config SBI_ECALL_DBCN
> +	bool "Debug Console extension"
> +	default y
> +
>  config SBI_ECALL_LEGACY
>  	bool "SBI v0.1 legacy extensions"
>  	default y
> diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
> index c774ebb..319f38d 100644
> --- a/lib/sbi/objects.mk
> +++ b/lib/sbi/objects.mk
> @@ -37,6 +37,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
>  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
>  libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
>  
> +carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_DBCN) += ecall_dbcn
> +libsbi-objs-$(CONFIG_SBI_ECALL_DBCN) += sbi_ecall_dbcn.o
> +
>  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_LEGACY) += ecall_legacy
>  libsbi-objs-$(CONFIG_SBI_ECALL_LEGACY) += sbi_ecall_legacy.o
>  
> diff --git a/lib/sbi/sbi_ecall_dbcn.c b/lib/sbi/sbi_ecall_dbcn.c
> new file mode 100644
> index 0000000..bfa40e9
> --- /dev/null
> +++ b/lib/sbi/sbi_ecall_dbcn.c
> @@ -0,0 +1,72 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Ventana Micro Systems Inc.
> + *
> + * Authors:
> + *   Anup Patel <apatel at ventanamicro.com>
> + */
> +
> +#include <sbi/sbi_console.h>
> +#include <sbi/sbi_domain.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_ecall.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi/sbi_trap.h>
> +#include <sbi/riscv_asm.h>
> +
> +static int sbi_ecall_dbcn_handler(unsigned long extid, unsigned long funcid,
> +				  const struct sbi_trap_regs *regs,
> +				  unsigned long *out_val,
> +				  struct sbi_trap_info *out_trap)
> +{
> +	ulong smode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >>
> +			MSTATUS_MPP_SHIFT;
> +
> +	switch (funcid) {
> +	case SBI_EXT_DBCN_CONSOLE_WRITE:
> +	case SBI_EXT_DBCN_CONSOLE_READ:
> +		/*
> +		 * On RV32, the M-mode can only access the first 4GB of
> +		 * the physical address space because M-mode does not have
> +		 * MMU to access full 34-bit physical address space.
> +		 *
> +		 * Based on above, we simply fail if the upper 32bits of
> +		 * the physical address (i.e. a2 register) is non-zero on
> +		 * RV32.
> +		 */
> +#if __riscv_xlen == 32
> +		if (regs->a2)
> +			return SBI_ERR_FAILED;

The spec drafts says SBI_ERR_FAILED is only for I/O errors. I think
we should extend the spec draft to point this situation out and return
SBI_ERR_INVALID_ADDRESS.

> +#endif
> +		if (!sbi_domain_check_addr_range(sbi_domain_thishart_ptr(),
> +					regs->a1, regs->a0, smode,
> +					SBI_DOMAIN_READ|SBI_DOMAIN_WRITE))
> +			return SBI_EINVALID_ADDR;

The spec draft says this should be SBI_ERR_INVALID_PARAM

> +		if (funcid == SBI_EXT_DBCN_CONSOLE_WRITE)
> +			sbi_nputs((const char *)regs->a1, regs->a0);
> +		else
> +			*out_val = sbi_ngets((char *)regs->a1, regs->a0);
> +		return 0;
> +	case SBI_EXT_DBCN_CONSOLE_WRITE_BYTE:
> +		sbi_putc(regs->a0);
> +		return 0;
> +	default:
> +		break;
> +	}
> +
> +	return SBI_ENOTSUPP;
> +}
> +
> +static int sbi_ecall_dbcn_probe(unsigned long extid, unsigned long *out_val)
> +{
> +	*out_val = (sbi_console_get_device()) ? 1 : 0;

nit: Unnecessary ()

> +	return 0;
> +}
> +
> +struct sbi_ecall_extension ecall_dbcn = {
> +	.extid_start = SBI_EXT_DBCN,
> +	.extid_end = SBI_EXT_DBCN,
> +	.handle = sbi_ecall_dbcn_handler,
> +	.probe = sbi_ecall_dbcn_probe,
> +};
> -- 
> 2.34.1
> 
>

Thanks,
drew


From ajones at ventanamicro.com  Wed Feb  1 01:00:13 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 1 Feb 2023 10:00:13 +0100
Subject: [PATCH v5 6/8] lib: sbi: Add console_puts() callback in the
 console device
In-Reply-To: <20230113114110.1916226-7-apatel@ventanamicro.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-7-apatel@ventanamicro.com>
Message-ID: <20230201090013.ulgfsudxkf3zjhtw@orel>

On Fri, Jan 13, 2023 at 05:11:08PM +0530, Anup Patel wrote:
> We add console_puts() callback in the console device which allows
> console drivers (such as semihosting) to implement a specialized
> way to output character string.
> 
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> Reviewed-by: Atish Patra <atishp at rivosinc.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Bin Meng <bmeng at tinylab.org>
> ---
>  include/sbi/sbi_console.h |  3 +++
>  lib/sbi/sbi_console.c     | 19 ++++++++++++++-----
>  2 files changed, 17 insertions(+), 5 deletions(-)
> 
> diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
> index 660c239..8e95b9d 100644
> --- a/include/sbi/sbi_console.h
> +++ b/include/sbi/sbi_console.h
> @@ -19,6 +19,9 @@ struct sbi_console_device {
>  	/** Write a character to the console output */
>  	void (*console_putc)(char ch);
>  
> +	/** Write a character string to the console output */
> +	void (*console_puts)(const char *str, unsigned long len);
> +
>  	/** Read a character from the console input */
>  	int (*console_getc)(void);
>  };
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index 89d6a49..c1b9f73 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -12,6 +12,7 @@
>  #include <sbi/sbi_hart.h>
>  #include <sbi/sbi_platform.h>
>  #include <sbi/sbi_scratch.h>
> +#include <sbi/sbi_string.h>
>  
>  static const struct sbi_console_device *console_dev = NULL;
>  static spinlock_t console_out_lock	       = SPIN_LOCK_INITIALIZER;
> @@ -44,9 +45,13 @@ void sbi_putc(char ch)
>  void sbi_puts(const char *str)
>  {
>  	spin_lock(&console_out_lock);
> -	while (*str) {
> -		sbi_putc(*str);
> -		str++;
> +	if (console_dev && console_dev->console_puts) {
> +		console_dev->console_puts(str, sbi_strlen(str));
> +	} else {
> +		while (*str) {
> +			sbi_putc(*str);
> +			str++;
> +		}
>  	}
>  	spin_unlock(&console_out_lock);
>  }
> @@ -56,8 +61,12 @@ void sbi_nputs(const char *str, unsigned long len)
>  	unsigned long i;
>  
>  	spin_lock(&console_out_lock);
> -	for (i = 0; i < len; i++)
> -		sbi_putc(str[i]);
> +	if (console_dev && console_dev->console_puts) {
> +		console_dev->console_puts(str, len);
> +	} else {
> +		for (i = 0; i < len; i++)
> +			sbi_putc(str[i]);
> +	}
>  	spin_unlock(&console_out_lock);
>  }
>  
> -- 
> 2.34.1
>

Reviewed-by: Andrew Jones <ajones at ventanamicro.com>


From ajones at ventanamicro.com  Wed Feb  1 01:03:53 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 1 Feb 2023 10:03:53 +0100
Subject: [PATCH v5 7/8] lib: utils/serial: Implement console_puts() for
 semihosting
In-Reply-To: <20230113114110.1916226-8-apatel@ventanamicro.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-8-apatel@ventanamicro.com>
Message-ID: <20230201090353.lg67zpqck4nucl2a@orel>

On Fri, Jan 13, 2023 at 05:11:09PM +0530, Anup Patel wrote:
> We implement console_puts() for semihosting serial driver to speed-up
> semihosting based prints.
> 
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> Reviewed-by: Atish Patra <atishp at rivosinc.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Bin Meng <bmeng at tinylab.org>
> ---
>  lib/utils/serial/semihosting.c | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/lib/utils/serial/semihosting.c b/lib/utils/serial/semihosting.c
> index 86fa296..773b75e 100644
> --- a/lib/utils/serial/semihosting.c
> +++ b/lib/utils/serial/semihosting.c
> @@ -15,6 +15,7 @@
>  
>  #define SYSOPEN     0x01
>  #define SYSWRITEC   0x03
> +#define SYSWRITE    0x05
>  #define SYSREAD     0x06
>  #define SYSREADC    0x07
>  #define SYSERRNO	0x13
> @@ -93,6 +94,7 @@ static int semihosting_errno(void)
>  }
>  
>  static int semihosting_infd = SBI_ENODEV;
> +static int semihosting_outfd = SBI_ENODEV;
>  
>  static long semihosting_open(const char *fname, enum semihosting_open_mode mode)
>  {
> @@ -141,6 +143,21 @@ static long semihosting_read(long fd, void *memp, size_t len)
>  	return len - ret;
>  }
>  
> +static long semihosting_write(long fd, const void *memp, size_t len)
> +{
> +	long ret;
> +	struct semihosting_rdwr_s write;
> +
> +	write.fd = fd;
> +	write.memp = (void *)memp;
> +	write.len = len;
> +
> +	ret = semihosting_trap(SYSWRITE, &write);
> +	if (ret < 0)
> +		return semihosting_errno();
> +	return len - ret;
> +}
> +
>  /* clang-format on */
>  
>  static void semihosting_putc(char ch)
> @@ -148,6 +165,20 @@ static void semihosting_putc(char ch)
>  	semihosting_trap(SYSWRITEC, &ch);
>  }
>  
> +static void semihosting_puts(const char *str, unsigned long len)
> +{
> +	char ch;
> +	unsigned long i;
> +
> +	if (semihosting_outfd < 0) {
> +		for (i = 0; i < len; i++) {
> +			ch = str[i];
> +			semihosting_trap(SYSWRITEC, &ch);
> +		}
> +	} else
> +		semihosting_write(semihosting_outfd, str, len);
> +}
> +
>  static int semihosting_getc(void)
>  {
>  	char ch = 0;
> @@ -165,12 +196,14 @@ static int semihosting_getc(void)
>  static struct sbi_console_device semihosting_console = {
>  	.name = "semihosting",
>  	.console_putc = semihosting_putc,
> +	.console_puts = semihosting_puts,
>  	.console_getc = semihosting_getc
>  };
>  
>  int semihosting_init(void)
>  {
>  	semihosting_infd = semihosting_open(":tt", MODE_READ);
> +	semihosting_outfd = semihosting_open(":tt", MODE_WRITE);
>  
>  	sbi_console_set_device(&semihosting_console);
>  
> -- 
> 2.34.1
>

Reviewed-by: Andrew Jones <ajones at ventanamicro.com>

Thanks,
drew


From ajones at ventanamicro.com  Wed Feb  1 01:22:35 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 1 Feb 2023 10:22:35 +0100
Subject: [PATCH v5 8/8] lib: sbi: Speed-up sbi_printf() and friends using
 nputs()
In-Reply-To: <20230113114110.1916226-9-apatel@ventanamicro.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-9-apatel@ventanamicro.com>
Message-ID: <20230201092235.vessrt2t7jdc5hfl@orel>

On Fri, Jan 13, 2023 at 05:11:10PM +0530, Anup Patel wrote:
> The sbi_printf() is slow for semihosting because it prints one
> character at a time. To speed-up sbi_printf() for semihosting,
> we use a temporary buffer and nputs().
> 
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> ---
>  lib/sbi/sbi_console.c | 52 ++++++++++++++++++++++++++++---------------
>  1 file changed, 34 insertions(+), 18 deletions(-)
> 
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index c1b9f73..41881d7 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -42,31 +42,29 @@ void sbi_putc(char ch)
>  	}
>  }
>  
> -void sbi_puts(const char *str)
> +static void nputs(const char *str, unsigned long len)
>  {
> -	spin_lock(&console_out_lock);
> +	unsigned long i;
> +
>  	if (console_dev && console_dev->console_puts) {
> -		console_dev->console_puts(str, sbi_strlen(str));
> +		console_dev->console_puts(str, len);
>  	} else {
> -		while (*str) {
> -			sbi_putc(*str);
> -			str++;
> -		}
> +		for (i = 0; i < len; i++)
> +			sbi_putc(str[i]);
>  	}
> +}
> +
> +void sbi_puts(const char *str)
> +{
> +	spin_lock(&console_out_lock);
> +	nputs(str, sbi_strlen(str));
>  	spin_unlock(&console_out_lock);
>  }
>  
>  void sbi_nputs(const char *str, unsigned long len)
>  {
> -	unsigned long i;
> -
>  	spin_lock(&console_out_lock);
> -	if (console_dev && console_dev->console_puts) {
> -		console_dev->console_puts(str, len);
> -	} else {
> -		for (i = 0; i < len; i++)
> -			sbi_putc(str[i]);
> -	}
> +	nputs(str, len);
>  	spin_unlock(&console_out_lock);
>  }
>  
> @@ -102,6 +100,7 @@ unsigned long sbi_ngets(char *str, unsigned long len)
>  #define PAD_ZERO 2
>  #define PAD_ALTERNATE 4
>  #define PRINT_BUF_LEN 64
> +#define PRINT_TBUF_MAX 128
>  
>  #define va_start(v, l) __builtin_va_start((v), l)
>  #define va_end __builtin_va_end
> @@ -217,12 +216,26 @@ static int printi(char **out, u32 *out_len, long long i, int b, int sg,
>  
>  static int print(char **out, u32 *out_len, const char *format, va_list args)
>  {
> -	int width, flags;
> -	int pc = 0;
> -	char scr[2];
> +	u32 tbuf_len;
> +	int width, flags, pc = 0;
> +	char scr[2], *tout, tbuf[PRINT_TBUF_MAX];
> +	bool use_tbuf = (!out) ? true : false;

print() is only called with a non-null 'out' parameter or under
console_out_lock, so we can use global variable for the buffer
to avoid putting it on the stack, and it can be larger.

>  	unsigned long long tmp;
>  
> +	if (use_tbuf) {
> +		tbuf_len = PRINT_TBUF_MAX;
> +		tout = tbuf;
> +		out = &tout;
> +		out_len = &tbuf_len;
> +	}
> +
>  	for (; *format != 0; ++format) {
> +		if (use_tbuf && !tbuf_len) {
> +			nputs(tbuf, PRINT_TBUF_MAX);
> +			tbuf_len = PRINT_TBUF_MAX;
> +			tout = tbuf;
> +		}
> +
>  		if (*format == '%') {
>  			++format;
>  			width = flags = 0;
> @@ -348,6 +361,9 @@ literal:
>  		}
>  	}
>  
> +	if (use_tbuf && tbuf_len < PRINT_TBUF_MAX)
> +		nputs(tbuf, PRINT_TBUF_MAX - tbuf_len);
> +
>  	return pc;
>  }
>  
> -- 
> 2.34.1
>

Otherwise,

Reviewed-by: Andrew Jones <ajones at ventanamicro.com>

Thanks,
drew


From ajones at ventanamicro.com  Wed Feb  1 01:30:13 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 1 Feb 2023 10:30:13 +0100
Subject: [PATCH v5 6/8] lib: sbi: Add console_puts() callback in the
 console device
In-Reply-To: <20230113114110.1916226-7-apatel@ventanamicro.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-7-apatel@ventanamicro.com>
Message-ID: <20230201093013.sibkotoeehynl7py@orel>

On Fri, Jan 13, 2023 at 05:11:08PM +0530, Anup Patel wrote:
> We add console_puts() callback in the console device which allows
> console drivers (such as semihosting) to implement a specialized
> way to output character string.
> 
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> Reviewed-by: Atish Patra <atishp at rivosinc.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Bin Meng <bmeng at tinylab.org>
> ---
>  include/sbi/sbi_console.h |  3 +++
>  lib/sbi/sbi_console.c     | 19 ++++++++++++++-----
>  2 files changed, 17 insertions(+), 5 deletions(-)
> 
> diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
> index 660c239..8e95b9d 100644
> --- a/include/sbi/sbi_console.h
> +++ b/include/sbi/sbi_console.h
> @@ -19,6 +19,9 @@ struct sbi_console_device {
>  	/** Write a character to the console output */
>  	void (*console_putc)(char ch);
>  
> +	/** Write a character string to the console output */
> +	void (*console_puts)(const char *str, unsigned long len);

The spec draft says the write command is non-blocking and returns the
number of bytes written. So this interface should at least return the
number of bytes written. And, maybe it should take a flags parameter
where a NON_BLOCKING flag can be set? But, if we can't enforce the
console to be non-blocking at the SBI implementation level, then
maybe the spec should state that the calls may block?

Thanks,
drew

> +
>  	/** Read a character from the console input */
>  	int (*console_getc)(void);
>  };
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index 89d6a49..c1b9f73 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -12,6 +12,7 @@
>  #include <sbi/sbi_hart.h>
>  #include <sbi/sbi_platform.h>
>  #include <sbi/sbi_scratch.h>
> +#include <sbi/sbi_string.h>
>  
>  static const struct sbi_console_device *console_dev = NULL;
>  static spinlock_t console_out_lock	       = SPIN_LOCK_INITIALIZER;
> @@ -44,9 +45,13 @@ void sbi_putc(char ch)
>  void sbi_puts(const char *str)
>  {
>  	spin_lock(&console_out_lock);
> -	while (*str) {
> -		sbi_putc(*str);
> -		str++;
> +	if (console_dev && console_dev->console_puts) {
> +		console_dev->console_puts(str, sbi_strlen(str));
> +	} else {
> +		while (*str) {
> +			sbi_putc(*str);
> +			str++;
> +		}
>  	}
>  	spin_unlock(&console_out_lock);
>  }
> @@ -56,8 +61,12 @@ void sbi_nputs(const char *str, unsigned long len)
>  	unsigned long i;
>  
>  	spin_lock(&console_out_lock);
> -	for (i = 0; i < len; i++)
> -		sbi_putc(str[i]);
> +	if (console_dev && console_dev->console_puts) {
> +		console_dev->console_puts(str, len);
> +	} else {
> +		for (i = 0; i < len; i++)
> +			sbi_putc(str[i]);
> +	}
>  	spin_unlock(&console_out_lock);
>  }
>  
> -- 
> 2.34.1
> 
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From ajones at ventanamicro.com  Wed Feb  1 01:35:58 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 1 Feb 2023 10:35:58 +0100
Subject: [PATCH v5 5/8] lib: sbi: Implement SBI debug console extension
In-Reply-To: <20230113114110.1916226-6-apatel@ventanamicro.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-6-apatel@ventanamicro.com>
Message-ID: <20230201093558.ty5emxqvl5vrdfg3@orel>

On Fri, Jan 13, 2023 at 05:11:07PM +0530, Anup Patel wrote:
> We implement SBI debug console extension as one of the replacement
> SBI extensions. This extension is only available when OpenSBI platform
> provides a console device to generic library.
> 
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> Reviewed-by: Atish Patra <atishp at rivosinc.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Bin Meng <bmeng at tinylab.org>
> ---
>  lib/sbi/Kconfig          |  4 +++
>  lib/sbi/objects.mk       |  3 ++
>  lib/sbi/sbi_ecall_dbcn.c | 72 ++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 79 insertions(+)
>  create mode 100644 lib/sbi/sbi_ecall_dbcn.c
> 
> diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
> index df74bba..ef6728b 100644
> --- a/lib/sbi/Kconfig
> +++ b/lib/sbi/Kconfig
> @@ -26,6 +26,10 @@ config SBI_ECALL_PMU
>  	bool "Performance Monitoring Unit extension"
>  	default y
>  
> +config SBI_ECALL_DBCN
> +	bool "Debug Console extension"
> +	default y
> +
>  config SBI_ECALL_LEGACY
>  	bool "SBI v0.1 legacy extensions"
>  	default y
> diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
> index c774ebb..319f38d 100644
> --- a/lib/sbi/objects.mk
> +++ b/lib/sbi/objects.mk
> @@ -37,6 +37,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
>  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
>  libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
>  
> +carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_DBCN) += ecall_dbcn
> +libsbi-objs-$(CONFIG_SBI_ECALL_DBCN) += sbi_ecall_dbcn.o
> +
>  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_LEGACY) += ecall_legacy
>  libsbi-objs-$(CONFIG_SBI_ECALL_LEGACY) += sbi_ecall_legacy.o
>  
> diff --git a/lib/sbi/sbi_ecall_dbcn.c b/lib/sbi/sbi_ecall_dbcn.c
> new file mode 100644
> index 0000000..bfa40e9
> --- /dev/null
> +++ b/lib/sbi/sbi_ecall_dbcn.c
> @@ -0,0 +1,72 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Ventana Micro Systems Inc.
> + *
> + * Authors:
> + *   Anup Patel <apatel at ventanamicro.com>
> + */
> +
> +#include <sbi/sbi_console.h>
> +#include <sbi/sbi_domain.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_ecall.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi/sbi_trap.h>
> +#include <sbi/riscv_asm.h>
> +
> +static int sbi_ecall_dbcn_handler(unsigned long extid, unsigned long funcid,
> +				  const struct sbi_trap_regs *regs,
> +				  unsigned long *out_val,
> +				  struct sbi_trap_info *out_trap)
> +{
> +	ulong smode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >>
> +			MSTATUS_MPP_SHIFT;
> +
> +	switch (funcid) {
> +	case SBI_EXT_DBCN_CONSOLE_WRITE:
> +	case SBI_EXT_DBCN_CONSOLE_READ:
> +		/*
> +		 * On RV32, the M-mode can only access the first 4GB of
> +		 * the physical address space because M-mode does not have
> +		 * MMU to access full 34-bit physical address space.
> +		 *
> +		 * Based on above, we simply fail if the upper 32bits of
> +		 * the physical address (i.e. a2 register) is non-zero on
> +		 * RV32.
> +		 */
> +#if __riscv_xlen == 32
> +		if (regs->a2)
> +			return SBI_ERR_FAILED;
> +#endif
> +		if (!sbi_domain_check_addr_range(sbi_domain_thishart_ptr(),
> +					regs->a1, regs->a0, smode,
> +					SBI_DOMAIN_READ|SBI_DOMAIN_WRITE))
> +			return SBI_EINVALID_ADDR;
> +		if (funcid == SBI_EXT_DBCN_CONSOLE_WRITE)
> +			sbi_nputs((const char *)regs->a1, regs->a0);

The spec draft says we return the number of bytes written, so we need
sbi_nputs to return that. If we never intend to support partial writes,
then I guess we at least need '*out_val = regs->a0'.

Thanks,
drew

> +		else
> +			*out_val = sbi_ngets((char *)regs->a1, regs->a0);
> +		return 0;
> +	case SBI_EXT_DBCN_CONSOLE_WRITE_BYTE:
> +		sbi_putc(regs->a0);
> +		return 0;
> +	default:
> +		break;
> +	}
> +
> +	return SBI_ENOTSUPP;
> +}
> +
> +static int sbi_ecall_dbcn_probe(unsigned long extid, unsigned long *out_val)
> +{
> +	*out_val = (sbi_console_get_device()) ? 1 : 0;
> +	return 0;
> +}
> +
> +struct sbi_ecall_extension ecall_dbcn = {
> +	.extid_start = SBI_EXT_DBCN,
> +	.extid_end = SBI_EXT_DBCN,
> +	.handle = sbi_ecall_dbcn_handler,
> +	.probe = sbi_ecall_dbcn_probe,
> +};
> -- 
> 2.34.1
> 
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From ajones at ventanamicro.com  Wed Feb  1 01:37:55 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 1 Feb 2023 10:37:55 +0100
Subject: [PATCH v5 6/8] lib: sbi: Add console_puts() callback in the
 console device
In-Reply-To: <20230201093013.sibkotoeehynl7py@orel>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-7-apatel@ventanamicro.com>
 <20230201093013.sibkotoeehynl7py@orel>
Message-ID: <20230201093755.w624jjj5fnbmk6y7@orel>

On Wed, Feb 01, 2023 at 10:30:13AM +0100, Andrew Jones wrote:
> On Fri, Jan 13, 2023 at 05:11:08PM +0530, Anup Patel wrote:
> > We add console_puts() callback in the console device which allows
> > console drivers (such as semihosting) to implement a specialized
> > way to output character string.
> > 
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > Reviewed-by: Atish Patra <atishp at rivosinc.com>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
> > Reviewed-by: Bin Meng <bmeng at tinylab.org>
> > ---
> >  include/sbi/sbi_console.h |  3 +++
> >  lib/sbi/sbi_console.c     | 19 ++++++++++++++-----
> >  2 files changed, 17 insertions(+), 5 deletions(-)
> > 
> > diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
> > index 660c239..8e95b9d 100644
> > --- a/include/sbi/sbi_console.h
> > +++ b/include/sbi/sbi_console.h
> > @@ -19,6 +19,9 @@ struct sbi_console_device {
> >  	/** Write a character to the console output */
> >  	void (*console_putc)(char ch);
> >  
> > +	/** Write a character string to the console output */
> > +	void (*console_puts)(const char *str, unsigned long len);
> 
> The spec draft says the write command is non-blocking and returns the
> number of bytes written. So this interface should at least return the
> number of bytes written. And, maybe it should take a flags parameter
> where a NON_BLOCKING flag can be set? But, if we can't enforce the
> console to be non-blocking at the SBI implementation level, then
> maybe the spec should state that the calls may block?
>

Also, if the console fails it'd be good if it returned an error,
allowing SBI_ERR_FAILED to be returned to the SBI caller as the
spec draft states. It'd be good if console_putc() could also
return an error for the same reason.

Thanks,
drew


From wxjstz at 126.com  Wed Feb  1 07:06:39 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:06:39 +0800
Subject: [PATCH 1/4] lib: sbi: Optimize sbi_tlb
In-Reply-To: <20230201150642.161265-1-wxjstz@126.com>
References: <20230201150642.161265-1-wxjstz@126.com>
Message-ID: <20230201150642.161265-2-wxjstz@126.com>

Originally, the process and sync of sbi_tlb need to wait for each other.
Evasion by atomic addition and subtraction.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_tlb.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index 4c142ea..685b8ea 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -215,7 +215,7 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 {
 	u32 rhartid;
 	struct sbi_scratch *rscratch = NULL;
-	unsigned long *rtlb_sync = NULL;
+	atomic_t *rtlb_sync = NULL;
 
 	tinfo->local_fn(tinfo);
 
@@ -225,7 +225,7 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 			continue;
 
 		rtlb_sync = sbi_scratch_offset_ptr(rscratch, tlb_sync_off);
-		while (atomic_raw_xchg_ulong(rtlb_sync, 1)) ;
+		atomic_sub_return(rtlb_sync, 1);
 	}
 }
 
@@ -257,10 +257,10 @@ static void tlb_process(struct sbi_scratch *scratch)
 
 static void tlb_sync(struct sbi_scratch *scratch)
 {
-	unsigned long *tlb_sync =
+	atomic_t *tlb_sync =
 			sbi_scratch_offset_ptr(scratch, tlb_sync_off);
 
-	while (!atomic_raw_xchg_ulong(tlb_sync, 0)) {
+	while (atomic_read(tlb_sync) > 0) {
 		/*
 		 * While we are waiting for remote hart to set the sync,
 		 * consume fifo requests to avoid deadlock.
@@ -343,6 +343,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 			  u32 remote_hartid, void *data)
 {
 	int ret;
+	atomic_t *tlb_sync;
 	struct sbi_fifo *tlb_fifo_r;
 	struct sbi_tlb_info *tinfo = data;
 	u32 curr_hartid = current_hartid();
@@ -387,6 +388,9 @@ static int tlb_update(struct sbi_scratch *scratch,
 			    curr_hartid, remote_hartid);
 	}
 
+	tlb_sync = sbi_scratch_offset_ptr(scratch, tlb_sync_off);
+	atomic_add_return(tlb_sync, 1);
+
 	return 0;
 }
 
@@ -413,7 +417,7 @@ int sbi_tlb_init(struct sbi_scratch *scratch, bool cold_boot)
 {
 	int ret;
 	void *tlb_mem;
-	unsigned long *tlb_sync;
+	atomic_t *tlb_sync;
 	struct sbi_fifo *tlb_q;
 	const struct sbi_platform *plat = sbi_platform_ptr(scratch);
 
@@ -455,7 +459,7 @@ int sbi_tlb_init(struct sbi_scratch *scratch, bool cold_boot)
 	tlb_q = sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
 	tlb_mem = sbi_scratch_offset_ptr(scratch, tlb_fifo_mem_off);
 
-	*tlb_sync = 0;
+	tlb_sync->counter = 0;
 
 	sbi_fifo_init(tlb_q, tlb_mem,
 		      SBI_TLB_FIFO_NUM_ENTRIES, SBI_TLB_INFO_SIZE);
-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:07:54 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:07:54 +0800
Subject: [PATCH v3 2/4] lib: sbi: Optimize sbi_ipi
In-Reply-To: <20230201150756.161507-1-wxjstz@126.com>
References: <20230201150756.161507-1-wxjstz@126.com>
Message-ID: <20230201150756.161507-3-wxjstz@126.com>

the original sbI_ipi will be processed by hart by hart, after optimization,
send ipi first and finally wait together.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_ipi.c | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)

diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 1bcc2e4..92a33bd 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -31,7 +31,7 @@ static unsigned long ipi_data_off;
 static const struct sbi_ipi_device *ipi_dev = NULL;
 static const struct sbi_ipi_event_ops *ipi_ops_array[SBI_IPI_EVENT_MAX];
 
-static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
+static int sbi_ipi_update(struct sbi_scratch *scratch, u32 remote_hartid,
 			u32 event, void *data)
 {
 	int ret;
@@ -69,6 +69,18 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
 
+	return 0;
+}
+
+static int sbi_ipi_sync(struct sbi_scratch *scratch, u32 event)
+{
+	const struct sbi_ipi_event_ops *ipi_ops;
+
+	if ((SBI_IPI_EVENT_MAX <= event) ||
+	    !ipi_ops_array[event])
+		return SBI_EINVAL;
+	ipi_ops = ipi_ops_array[event];
+
 	if (ipi_ops->sync)
 		ipi_ops->sync(scratch);
 
@@ -83,7 +95,7 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
 int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 {
 	int rc;
-	ulong i, m;
+	ulong i, m, n;
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
@@ -92,19 +104,37 @@ int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 		if (rc)
 			return rc;
 		m &= hmask;
+		n = m;
 
-		/* Send IPIs */
+		/* update IPIs */
 		for (i = hbase; m; i++, m >>= 1) {
 			if (m & 1UL)
-				sbi_ipi_send(scratch, i, event, data);
+				sbi_ipi_update(scratch, i, event, data);
+		}
+
+		/* sync IPIs */
+		m = n;
+		for (i = hbase; m; i++, m >>= 1) {
+			if (m & 1UL)
+				sbi_ipi_sync(scratch, event);
 		}
 	} else {
+		/* update IPIs */
+		hbase = 0;
+		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
+			for (i = hbase; m; i++, m >>= 1) {
+				if (m & 1UL)
+					sbi_ipi_update(scratch, i, event, data);
+			}
+			hbase += BITS_PER_LONG;
+		}
+
+		/* sync IPIs */
 		hbase = 0;
 		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
-			/* Send IPIs */
 			for (i = hbase; m; i++, m >>= 1) {
 				if (m & 1UL)
-					sbi_ipi_send(scratch, i, event, data);
+					sbi_ipi_sync(scratch, event);
 			}
 			hbase += BITS_PER_LONG;
 		}
-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:07:53 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:07:53 +0800
Subject: [PATCH v3 1/4] lib: sbi: Optimize sbi_tlb
In-Reply-To: <20230201150756.161507-1-wxjstz@126.com>
References: <20230201150756.161507-1-wxjstz@126.com>
Message-ID: <20230201150756.161507-2-wxjstz@126.com>

Originally, the process and sync of sbi_tlb need to wait for each other.
Evasion by atomic addition and subtraction.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_tlb.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index 4c142ea..685b8ea 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -215,7 +215,7 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 {
 	u32 rhartid;
 	struct sbi_scratch *rscratch = NULL;
-	unsigned long *rtlb_sync = NULL;
+	atomic_t *rtlb_sync = NULL;
 
 	tinfo->local_fn(tinfo);
 
@@ -225,7 +225,7 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 			continue;
 
 		rtlb_sync = sbi_scratch_offset_ptr(rscratch, tlb_sync_off);
-		while (atomic_raw_xchg_ulong(rtlb_sync, 1)) ;
+		atomic_sub_return(rtlb_sync, 1);
 	}
 }
 
@@ -257,10 +257,10 @@ static void tlb_process(struct sbi_scratch *scratch)
 
 static void tlb_sync(struct sbi_scratch *scratch)
 {
-	unsigned long *tlb_sync =
+	atomic_t *tlb_sync =
 			sbi_scratch_offset_ptr(scratch, tlb_sync_off);
 
-	while (!atomic_raw_xchg_ulong(tlb_sync, 0)) {
+	while (atomic_read(tlb_sync) > 0) {
 		/*
 		 * While we are waiting for remote hart to set the sync,
 		 * consume fifo requests to avoid deadlock.
@@ -343,6 +343,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 			  u32 remote_hartid, void *data)
 {
 	int ret;
+	atomic_t *tlb_sync;
 	struct sbi_fifo *tlb_fifo_r;
 	struct sbi_tlb_info *tinfo = data;
 	u32 curr_hartid = current_hartid();
@@ -387,6 +388,9 @@ static int tlb_update(struct sbi_scratch *scratch,
 			    curr_hartid, remote_hartid);
 	}
 
+	tlb_sync = sbi_scratch_offset_ptr(scratch, tlb_sync_off);
+	atomic_add_return(tlb_sync, 1);
+
 	return 0;
 }
 
@@ -413,7 +417,7 @@ int sbi_tlb_init(struct sbi_scratch *scratch, bool cold_boot)
 {
 	int ret;
 	void *tlb_mem;
-	unsigned long *tlb_sync;
+	atomic_t *tlb_sync;
 	struct sbi_fifo *tlb_q;
 	const struct sbi_platform *plat = sbi_platform_ptr(scratch);
 
@@ -455,7 +459,7 @@ int sbi_tlb_init(struct sbi_scratch *scratch, bool cold_boot)
 	tlb_q = sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
 	tlb_mem = sbi_scratch_offset_ptr(scratch, tlb_fifo_mem_off);
 
-	*tlb_sync = 0;
+	tlb_sync->counter = 0;
 
 	sbi_fifo_init(tlb_q, tlb_mem,
 		      SBI_TLB_FIFO_NUM_ENTRIES, SBI_TLB_INFO_SIZE);
-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:07:52 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:07:52 +0800
Subject: [PATCH v3 0/4] Miscellaneous about sbi_tlb and sbi_ipi
Message-ID: <20230201150756.161507-1-wxjstz@126.com>

Removed the mutual waiting between process and sync in sbi_tlb
Update ipi first then wait together

V3 Changes:
PATCH2: Remove missing sync in update
add PATCH4: Optimize sbi_tlb queue waiting

V2 Changes:
PATCH2: Fix m changes after update IPIs

Xiang W (4):
  lib: sbi: Optimize sbi_tlb
  lib: sbi: Optimize sbi_ipi
  lib: sbi: rename function of sbi_ipi
  lib: sbi: Optimize sbi_tlb queue waiting

 include/sbi/sbi_ipi.h |  3 +++
 lib/sbi/sbi_ipi.c     | 39 ++++++++++++++++++++++++++++++------
 lib/sbi/sbi_tlb.c     | 46 +++++++++++++++++++++----------------------
 3 files changed, 58 insertions(+), 30 deletions(-)

-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:07:56 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:07:56 +0800
Subject: [PATCH v3 4/4] lib: sbi: Optimize sbi_tlb queue waiting
In-Reply-To: <20230201150756.161507-1-wxjstz@126.com>
References: <20230201150756.161507-1-wxjstz@126.com>
Message-ID: <20230201150756.161507-5-wxjstz@126.com>

When tlb_fifo is full, it will wait and affect the ipi update to
other harts. This patch is optimized.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/sbi_ipi.h |  3 +++
 lib/sbi/sbi_ipi.c     | 43 ++++++++++++++++++++-----------------------
 lib/sbi/sbi_tlb.c     |  3 ++-
 3 files changed, 25 insertions(+), 24 deletions(-)

diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
index f6ac807..aad7f77 100644
--- a/include/sbi/sbi_ipi.h
+++ b/include/sbi/sbi_ipi.h
@@ -41,6 +41,9 @@ struct sbi_ipi_event_ops {
 	 * Update callback to save/enqueue data for remote HART
 	 * Note: This is an optional callback and it is called just before
 	 * triggering IPI to remote HART.
+	 * @return 0  success
+	 * @return -1 break IPI, done on local hart
+	 * @return -2 need retry
 	 */
 	int (* update)(struct sbi_scratch *scratch,
 			struct sbi_scratch *remote_scratch,
diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 92a33bd..89c485e 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -94,8 +94,10 @@ static int sbi_ipi_sync(struct sbi_scratch *scratch, u32 event)
  */
 int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 {
-	int rc;
-	ulong i, m, n;
+	int rc, done;
+	ulong i, m;
+	struct sbi_hartmask target_mask = {0};
+	struct sbi_hartmask retry_mask = {0};
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
@@ -104,41 +106,36 @@ int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 		if (rc)
 			return rc;
 		m &= hmask;
-		n = m;
 
-		/* update IPIs */
 		for (i = hbase; m; i++, m >>= 1) {
 			if (m & 1UL)
-				sbi_ipi_update(scratch, i, event, data);
-		}
-
-		/* sync IPIs */
-		m = n;
-		for (i = hbase; m; i++, m >>= 1) {
-			if (m & 1UL)
-				sbi_ipi_sync(scratch, event);
+				sbi_hartmask_set_hart(i, &target_mask);
 		}
 	} else {
-		/* update IPIs */
 		hbase = 0;
 		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
 			for (i = hbase; m; i++, m >>= 1) {
 				if (m & 1UL)
-					sbi_ipi_update(scratch, i, event, data);
+					sbi_hartmask_set_hart(i, &target_mask);
 			}
 			hbase += BITS_PER_LONG;
 		}
+	}
 
-		/* sync IPIs */
-		hbase = 0;
-		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
-			for (i = hbase; m; i++, m >>= 1) {
-				if (m & 1UL)
-					sbi_ipi_sync(scratch, event);
-			}
-			hbase += BITS_PER_LONG;
+	retry_mask = target_mask;
+	do {
+		done = true;
+		sbi_hartmask_for_each_hart(i, &retry_mask) {
+			rc = sbi_ipi_update(scratch, i, event, data);
+			if (rc == -2)
+				done = false;
+			else
+				sbi_hartmask_clear_hart(i, &retry_mask);
 		}
-	}
+	} while (!done);
+
+	/* sync IPIs */
+	sbi_ipi_sync(scratch, event);
 
 	return 0;
 }
diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index c82f53a..673f948 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -367,7 +367,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 		return 1;
 	}
 
-	while (sbi_fifo_enqueue(tlb_fifo_r, data) < 0) {
+	if (sbi_fifo_enqueue(tlb_fifo_r, data) < 0) {
 		/**
 		 * For now, Busy loop until there is space in the fifo.
 		 * There may be case where target hart is also
@@ -379,6 +379,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 		tlb_process_once(scratch);
 		sbi_dprintf("hart%d: hart%d tlb fifo full\n",
 			    curr_hartid, remote_hartid);
+		return -2;
 	}
 
 	tlb_sync = sbi_scratch_offset_ptr(scratch, tlb_sync_off);
-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:07:55 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:07:55 +0800
Subject: [PATCH v3 3/4] lib: sbi: rename function of sbi_ipi
In-Reply-To: <20230201150756.161507-1-wxjstz@126.com>
References: <20230201150756.161507-1-wxjstz@126.com>
Message-ID: <20230201150756.161507-4-wxjstz@126.com>

Renamed some functions and reused some code

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_tlb.c | 27 ++++++++++-----------------
 1 file changed, 10 insertions(+), 17 deletions(-)

diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index 685b8ea..c82f53a 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -211,7 +211,7 @@ static void tlb_pmu_incr_fw_ctr(struct sbi_tlb_info *data)
 		sbi_pmu_ctr_incr_fw(SBI_PMU_FW_HFENCE_VVMA_ASID_SENT);
 }
 
-static void tlb_entry_process(struct sbi_tlb_info *tinfo)
+static void tlb_process_helper(struct sbi_tlb_info *tinfo)
 {
 	u32 rhartid;
 	struct sbi_scratch *rscratch = NULL;
@@ -229,30 +229,23 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 	}
 }
 
-static void tlb_process_count(struct sbi_scratch *scratch, int count)
+static int tlb_process_once(struct sbi_scratch *scratch)
 {
 	struct sbi_tlb_info tinfo;
-	unsigned int deq_count = 0;
 	struct sbi_fifo *tlb_fifo =
 			sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
 
-	while (!sbi_fifo_dequeue(tlb_fifo, &tinfo)) {
-		tlb_entry_process(&tinfo);
-		deq_count++;
-		if (deq_count > count)
-			break;
-
+	if (!sbi_fifo_dequeue(tlb_fifo, &tinfo)) {
+		tlb_process_helper(&tinfo);
+		return 0;
 	}
+
+	return -1;
 }
 
 static void tlb_process(struct sbi_scratch *scratch)
 {
-	struct sbi_tlb_info tinfo;
-	struct sbi_fifo *tlb_fifo =
-			sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
-
-	while (!sbi_fifo_dequeue(tlb_fifo, &tinfo))
-		tlb_entry_process(&tinfo);
+	while (!tlb_process_once(scratch));
 }
 
 static void tlb_sync(struct sbi_scratch *scratch)
@@ -265,7 +258,7 @@ static void tlb_sync(struct sbi_scratch *scratch)
 		 * While we are waiting for remote hart to set the sync,
 		 * consume fifo requests to avoid deadlock.
 		 */
-		tlb_process_count(scratch, 1);
+		tlb_process_once(scratch);
 	}
 
 	return;
@@ -383,7 +376,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 		 * TODO: Introduce a wait/wakeup event mechanism to handle
 		 * this properly.
 		 */
-		tlb_process_count(scratch, 1);
+		tlb_process_once(scratch);
 		sbi_dprintf("hart%d: hart%d tlb fifo full\n",
 			    curr_hartid, remote_hartid);
 	}
-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:06:38 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:06:38 +0800
Subject: [PATCH 0/4] Miscellaneous about sbi_tlb and sbi_ipi
Message-ID: <20230201150642.161265-1-wxjstz@126.com>

Removed the mutual waiting between process and sync in sbi_tlb
Update ipi first then wait together

V3 Changes:
PATCH2: Remove missing sync in update
add PATCH4: Optimize sbi_tlb queue waiting

V2 Changes:
PATCH2: Fix m changes after update IPIs

Xiang W (4):
  lib: sbi: Optimize sbi_tlb
  lib: sbi: Optimize sbi_ipi
  lib: sbi: rename function of sbi_ipi
  lib: sbi: Optimize sbi_tlb queue waiting

 include/sbi/sbi_ipi.h |  3 +++
 lib/sbi/sbi_ipi.c     | 39 ++++++++++++++++++++++++++++++------
 lib/sbi/sbi_tlb.c     | 46 +++++++++++++++++++++----------------------
 3 files changed, 58 insertions(+), 30 deletions(-)

-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:06:42 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:06:42 +0800
Subject: [PATCH 4/4] lib: sbi: Optimize sbi_tlb queue waiting
In-Reply-To: <20230201150642.161265-1-wxjstz@126.com>
References: <20230201150642.161265-1-wxjstz@126.com>
Message-ID: <20230201150642.161265-5-wxjstz@126.com>

When tlb_fifo is full, it will wait and affect the ipi update to
other harts. This patch is optimized.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/sbi_ipi.h |  3 +++
 lib/sbi/sbi_ipi.c     | 43 ++++++++++++++++++++-----------------------
 lib/sbi/sbi_tlb.c     |  3 ++-
 3 files changed, 25 insertions(+), 24 deletions(-)

diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
index f6ac807..aad7f77 100644
--- a/include/sbi/sbi_ipi.h
+++ b/include/sbi/sbi_ipi.h
@@ -41,6 +41,9 @@ struct sbi_ipi_event_ops {
 	 * Update callback to save/enqueue data for remote HART
 	 * Note: This is an optional callback and it is called just before
 	 * triggering IPI to remote HART.
+	 * @return 0  success
+	 * @return -1 break IPI, done on local hart
+	 * @return -2 need retry
 	 */
 	int (* update)(struct sbi_scratch *scratch,
 			struct sbi_scratch *remote_scratch,
diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 92a33bd..89c485e 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -94,8 +94,10 @@ static int sbi_ipi_sync(struct sbi_scratch *scratch, u32 event)
  */
 int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 {
-	int rc;
-	ulong i, m, n;
+	int rc, done;
+	ulong i, m;
+	struct sbi_hartmask target_mask = {0};
+	struct sbi_hartmask retry_mask = {0};
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
@@ -104,41 +106,36 @@ int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 		if (rc)
 			return rc;
 		m &= hmask;
-		n = m;
 
-		/* update IPIs */
 		for (i = hbase; m; i++, m >>= 1) {
 			if (m & 1UL)
-				sbi_ipi_update(scratch, i, event, data);
-		}
-
-		/* sync IPIs */
-		m = n;
-		for (i = hbase; m; i++, m >>= 1) {
-			if (m & 1UL)
-				sbi_ipi_sync(scratch, event);
+				sbi_hartmask_set_hart(i, &target_mask);
 		}
 	} else {
-		/* update IPIs */
 		hbase = 0;
 		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
 			for (i = hbase; m; i++, m >>= 1) {
 				if (m & 1UL)
-					sbi_ipi_update(scratch, i, event, data);
+					sbi_hartmask_set_hart(i, &target_mask);
 			}
 			hbase += BITS_PER_LONG;
 		}
+	}
 
-		/* sync IPIs */
-		hbase = 0;
-		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
-			for (i = hbase; m; i++, m >>= 1) {
-				if (m & 1UL)
-					sbi_ipi_sync(scratch, event);
-			}
-			hbase += BITS_PER_LONG;
+	retry_mask = target_mask;
+	do {
+		done = true;
+		sbi_hartmask_for_each_hart(i, &retry_mask) {
+			rc = sbi_ipi_update(scratch, i, event, data);
+			if (rc == -2)
+				done = false;
+			else
+				sbi_hartmask_clear_hart(i, &retry_mask);
 		}
-	}
+	} while (!done);
+
+	/* sync IPIs */
+	sbi_ipi_sync(scratch, event);
 
 	return 0;
 }
diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index c82f53a..673f948 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -367,7 +367,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 		return 1;
 	}
 
-	while (sbi_fifo_enqueue(tlb_fifo_r, data) < 0) {
+	if (sbi_fifo_enqueue(tlb_fifo_r, data) < 0) {
 		/**
 		 * For now, Busy loop until there is space in the fifo.
 		 * There may be case where target hart is also
@@ -379,6 +379,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 		tlb_process_once(scratch);
 		sbi_dprintf("hart%d: hart%d tlb fifo full\n",
 			    curr_hartid, remote_hartid);
+		return -2;
 	}
 
 	tlb_sync = sbi_scratch_offset_ptr(scratch, tlb_sync_off);
-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:06:41 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:06:41 +0800
Subject: [PATCH 3/4] lib: sbi: rename function of sbi_ipi
In-Reply-To: <20230201150642.161265-1-wxjstz@126.com>
References: <20230201150642.161265-1-wxjstz@126.com>
Message-ID: <20230201150642.161265-4-wxjstz@126.com>

Renamed some functions and reused some code

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_tlb.c | 27 ++++++++++-----------------
 1 file changed, 10 insertions(+), 17 deletions(-)

diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index 685b8ea..c82f53a 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -211,7 +211,7 @@ static void tlb_pmu_incr_fw_ctr(struct sbi_tlb_info *data)
 		sbi_pmu_ctr_incr_fw(SBI_PMU_FW_HFENCE_VVMA_ASID_SENT);
 }
 
-static void tlb_entry_process(struct sbi_tlb_info *tinfo)
+static void tlb_process_helper(struct sbi_tlb_info *tinfo)
 {
 	u32 rhartid;
 	struct sbi_scratch *rscratch = NULL;
@@ -229,30 +229,23 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 	}
 }
 
-static void tlb_process_count(struct sbi_scratch *scratch, int count)
+static int tlb_process_once(struct sbi_scratch *scratch)
 {
 	struct sbi_tlb_info tinfo;
-	unsigned int deq_count = 0;
 	struct sbi_fifo *tlb_fifo =
 			sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
 
-	while (!sbi_fifo_dequeue(tlb_fifo, &tinfo)) {
-		tlb_entry_process(&tinfo);
-		deq_count++;
-		if (deq_count > count)
-			break;
-
+	if (!sbi_fifo_dequeue(tlb_fifo, &tinfo)) {
+		tlb_process_helper(&tinfo);
+		return 0;
 	}
+
+	return -1;
 }
 
 static void tlb_process(struct sbi_scratch *scratch)
 {
-	struct sbi_tlb_info tinfo;
-	struct sbi_fifo *tlb_fifo =
-			sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
-
-	while (!sbi_fifo_dequeue(tlb_fifo, &tinfo))
-		tlb_entry_process(&tinfo);
+	while (!tlb_process_once(scratch));
 }
 
 static void tlb_sync(struct sbi_scratch *scratch)
@@ -265,7 +258,7 @@ static void tlb_sync(struct sbi_scratch *scratch)
 		 * While we are waiting for remote hart to set the sync,
 		 * consume fifo requests to avoid deadlock.
 		 */
-		tlb_process_count(scratch, 1);
+		tlb_process_once(scratch);
 	}
 
 	return;
@@ -383,7 +376,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 		 * TODO: Introduce a wait/wakeup event mechanism to handle
 		 * this properly.
 		 */
-		tlb_process_count(scratch, 1);
+		tlb_process_once(scratch);
 		sbi_dprintf("hart%d: hart%d tlb fifo full\n",
 			    curr_hartid, remote_hartid);
 	}
-- 
2.39.0



From wxjstz at 126.com  Wed Feb  1 07:06:40 2023
From: wxjstz at 126.com (Xiang W)
Date: Wed,  1 Feb 2023 23:06:40 +0800
Subject: [PATCH 2/4] lib: sbi: Optimize sbi_ipi
In-Reply-To: <20230201150642.161265-1-wxjstz@126.com>
References: <20230201150642.161265-1-wxjstz@126.com>
Message-ID: <20230201150642.161265-3-wxjstz@126.com>

the original sbI_ipi will be processed by hart by hart, after optimization,
send ipi first and finally wait together.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_ipi.c | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)

diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 1bcc2e4..92a33bd 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -31,7 +31,7 @@ static unsigned long ipi_data_off;
 static const struct sbi_ipi_device *ipi_dev = NULL;
 static const struct sbi_ipi_event_ops *ipi_ops_array[SBI_IPI_EVENT_MAX];
 
-static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
+static int sbi_ipi_update(struct sbi_scratch *scratch, u32 remote_hartid,
 			u32 event, void *data)
 {
 	int ret;
@@ -69,6 +69,18 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
 
+	return 0;
+}
+
+static int sbi_ipi_sync(struct sbi_scratch *scratch, u32 event)
+{
+	const struct sbi_ipi_event_ops *ipi_ops;
+
+	if ((SBI_IPI_EVENT_MAX <= event) ||
+	    !ipi_ops_array[event])
+		return SBI_EINVAL;
+	ipi_ops = ipi_ops_array[event];
+
 	if (ipi_ops->sync)
 		ipi_ops->sync(scratch);
 
@@ -83,7 +95,7 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
 int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 {
 	int rc;
-	ulong i, m;
+	ulong i, m, n;
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
@@ -92,19 +104,37 @@ int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 		if (rc)
 			return rc;
 		m &= hmask;
+		n = m;
 
-		/* Send IPIs */
+		/* update IPIs */
 		for (i = hbase; m; i++, m >>= 1) {
 			if (m & 1UL)
-				sbi_ipi_send(scratch, i, event, data);
+				sbi_ipi_update(scratch, i, event, data);
+		}
+
+		/* sync IPIs */
+		m = n;
+		for (i = hbase; m; i++, m >>= 1) {
+			if (m & 1UL)
+				sbi_ipi_sync(scratch, event);
 		}
 	} else {
+		/* update IPIs */
+		hbase = 0;
+		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
+			for (i = hbase; m; i++, m >>= 1) {
+				if (m & 1UL)
+					sbi_ipi_update(scratch, i, event, data);
+			}
+			hbase += BITS_PER_LONG;
+		}
+
+		/* sync IPIs */
 		hbase = 0;
 		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
-			/* Send IPIs */
 			for (i = hbase; m; i++, m >>= 1) {
 				if (m & 1UL)
-					sbi_ipi_send(scratch, i, event, data);
+					sbi_ipi_sync(scratch, event);
 			}
 			hbase += BITS_PER_LONG;
 		}
-- 
2.39.0



From jrtc27 at jrtc27.com  Wed Feb  1 08:49:16 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Wed,  1 Feb 2023 16:49:16 +0000
Subject: [PATCH] Makefile: Add missing .dep files for fw_*.elf.ld
Message-ID: <20230201164916.19908-1-jrtc27@jrtc27.com>

Since we don't currently create these, changes to fw_base.ldS do not
cause the preprocessed fw_*.elf.ld files to be rebuilt, and thus
incremental builds can end up failing with missing symbols if crossing
the recent commits that introduced _fw_rw_offset and then replaced it
with _fw_rw_start.

Reported-by: Ben Dooks <ben.dooks at sifive.com>
Signed-off-by: Jessica Clarke <jrtc27 at jrtc27.com>
---
 Makefile | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Makefile b/Makefile
index b742d6d..b20404f 100644
--- a/Makefile
+++ b/Makefile
@@ -254,6 +254,7 @@ deps-y=$(platform-objs-path-y:.o=.dep)
 deps-y+=$(libsbi-objs-path-y:.o=.dep)
 deps-y+=$(libsbiutils-objs-path-y:.o=.dep)
 deps-y+=$(firmware-objs-path-y:.o=.dep)
+deps-y+=$(firmware-elfs-path-y:=.dep)
 
 # Setup platform ABI, ISA and Code Model
 ifndef PLATFORM_RISCV_ABI
@@ -413,6 +414,11 @@ inst_file_list = $(CMD_PREFIX)if [ ! -z "$(4)" ]; then \
 inst_header_dir =  $(CMD_PREFIX)mkdir -p $(1); \
 	     echo " INSTALL   $(subst $(install_root_dir)/,,$(1))"; \
 	     cp -rf $(2) $(1)
+compile_cpp_dep = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \
+	     echo " CPP-DEP   $(subst $(build_dir)/,,$(1))"; \
+	     printf %s `dirname $(1)`/  > $(1) && \
+	     $(CC) $(CPPFLAGS) -x c -MM $(3) \
+	       -MT `basename $(1:.dep=$(2))` >> $(1) || rm -f $(1)
 compile_cpp = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \
 	     echo " CPP       $(subst $(build_dir)/,,$(1))"; \
 	     $(CPP) $(CPPFLAGS) -x c $(2) | grep -v "\#" > $(1)
@@ -543,6 +549,9 @@ $(platform_build_dir)/%.bin: $(platform_build_dir)/%.elf
 $(platform_build_dir)/%.elf: $(platform_build_dir)/%.o $(platform_build_dir)/%.elf.ld $(platform_build_dir)/lib/libplatsbi.a
 	$(call compile_elf,$@,$@.ld,$< $(platform_build_dir)/lib/libplatsbi.a)
 
+$(platform_build_dir)/%.dep: $(src_dir)/%.ldS $(KCONFIG_CONFIG)
+	$(call compile_cpp_dep,$@,.ld,$<)
+
 $(platform_build_dir)/%.ld: $(src_dir)/%.ldS
 	$(call compile_cpp,$@,$<)
 
-- 
2.37.0



From rpathak at ventanamicro.com  Wed Feb  1 20:44:24 2023
From: rpathak at ventanamicro.com (Rahul Pathak)
Date: Thu,  2 Feb 2023 10:14:24 +0530
Subject: [PATCH v2 0/3] Add endianness conversion support
Message-ID: <20230202044427.1253816-1-rpathak@ventanamicro.com>

This series adds a new header which defines the generic
endianness conversion functions.
These new functions replaces the existing byte-swapping
functions defined in libfdt_env.h

New typedefs - leX_t/beX_t and use them in existing
libfdt_env.h

Changelog - v2:
- Added new patch for typedefs related to endianness

Rahul Pathak (3):
  include: Add support for byteorder/endianness conversion
  lib: utils/fdt: Use byteorder conversion functions in libfdt_env.h
  include: types: Add typedefs for endianness

 include/sbi/sbi_byteorder.h   | 57 +++++++++++++++++++++++++++++++++++
 include/sbi/sbi_types.h       |  7 +++++
 lib/utils/libfdt/libfdt_env.h | 35 ++++++++-------------
 3 files changed, 77 insertions(+), 22 deletions(-)
 create mode 100644 include/sbi/sbi_byteorder.h

-- 
2.34.1



From rpathak at ventanamicro.com  Wed Feb  1 20:44:25 2023
From: rpathak at ventanamicro.com (Rahul Pathak)
Date: Thu,  2 Feb 2023 10:14:25 +0530
Subject: [PATCH v2 1/3] include: Add support for byteorder/endianness conversion
In-Reply-To: <20230202044427.1253816-1-rpathak@ventanamicro.com>
References: <20230202044427.1253816-1-rpathak@ventanamicro.com>
Message-ID: <20230202044427.1253816-2-rpathak@ventanamicro.com>

Define macros general byteorder conversion
Define functions for endianness conversion
from general byteorder conversion macros

Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
Reviewed-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Sergey Matyukevich <sergey.matyukevich at syntacore.com>
---
 include/sbi/sbi_byteorder.h | 57 +++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 include/sbi/sbi_byteorder.h

diff --git a/include/sbi/sbi_byteorder.h b/include/sbi/sbi_byteorder.h
new file mode 100644
index 000000000000..680710fe6d91
--- /dev/null
+++ b/include/sbi/sbi_byteorder.h
@@ -0,0 +1,57 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2023 Ventana Micro Systems Inc.
+ */
+
+#ifndef __SBI_BYTEORDER_H__
+#define __SBI_BYTEORDER_H__
+
+#include <sbi/sbi_types.h>
+
+#define EXTRACT_BYTE(x, n)	((unsigned long long)((uint8_t *)&x)[n])
+
+#define BSWAP16(x)	((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
+#define BSWAP32(x)	((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
+			 (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
+#define BSWAP64(x)	((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
+			 (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
+			 (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
+			 (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
+
+
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__  /* CPU(little-endian) */
+#define cpu_to_be16(x)		((uint16_t)BSWAP16(x))
+#define cpu_to_be32(x)		((uint32_t)BSWAP32(x))
+#define cpu_to_be64(x)		((uint64_t)BSWAP64(x))
+
+#define be16_to_cpu(x)		((uint16_t)BSWAP16(x))
+#define be32_to_cpu(x)		((uint32_t)BSWAP32(x))
+#define be64_to_cpu(x)		((uint64_t)BSWAP64(x))
+
+#define cpu_to_le16(x)		((uint16_t)(x))
+#define cpu_to_le32(x)		((uint32_t)(x))
+#define cpu_to_le64(x)		((uint64_t)(x))
+
+#define le16_to_cpu(x)		((uint16_t)(x))
+#define le32_to_cpu(x)		((uint32_t)(x))
+#define le64_to_cpu(x)		((uint64_t)(x))
+#else /* CPU(big-endian) */
+#define cpu_to_be16(x)		((uint16_t)(x))
+#define cpu_to_be32(x)		((uint32_t)(x))
+#define cpu_to_be64(x)		((uint64_t)(x))
+
+#define be16_to_cpu(x)		((uint16_t)(x))
+#define be32_to_cpu(x)		((uint32_t)(x))
+#define be64_to_cpu(x)		((uint64_t)(x))
+
+#define cpu_to_le16(x)		((uint16_t)BSWAP16(x))
+#define cpu_to_le32(x)		((uint32_t)BSWAP32(x))
+#define cpu_to_le64(x)		((uint64_t)BSWAP64(x))
+
+#define le16_to_cpu(x)		((uint16_t)BSWAP16(x))
+#define le32_to_cpu(x)		((uint32_t)BSWAP32(x))
+#define le64_to_cpu(x)		((uint64_t)BSWAP64(x))
+#endif
+
+#endif /* __SBI_BYTEORDER_H__ */
-- 
2.34.1



From rpathak at ventanamicro.com  Wed Feb  1 20:44:26 2023
From: rpathak at ventanamicro.com (Rahul Pathak)
Date: Thu,  2 Feb 2023 10:14:26 +0530
Subject: [PATCH v2 2/3] lib: utils/fdt: Use byteorder conversion functions in libfdt_env.h
In-Reply-To: <20230202044427.1253816-1-rpathak@ventanamicro.com>
References: <20230202044427.1253816-1-rpathak@ventanamicro.com>
Message-ID: <20230202044427.1253816-3-rpathak@ventanamicro.com>

FDT follows big-endian and CPU can be little or big
endian as per the implementation.
libfdt_env.h defines function for conversion between
fdt and cpu byteorder according to the endianness.

Currently, libfdt_env.h defines custom byte swapping
macros and then undefines them. Instead, use the generic
endianness conversion functions

Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
Reviewed-by: Xiang W <wxjstz at 126.com>
---
 lib/utils/libfdt/libfdt_env.h | 29 ++++++++++-------------------
 1 file changed, 10 insertions(+), 19 deletions(-)

diff --git a/lib/utils/libfdt/libfdt_env.h b/lib/utils/libfdt/libfdt_env.h
index f9d9c6783c52..e5ad7698efca 100644
--- a/lib/utils/libfdt/libfdt_env.h
+++ b/lib/utils/libfdt/libfdt_env.h
@@ -9,6 +9,7 @@
 
 #include <sbi/sbi_string.h>
 #include <sbi/sbi_types.h>
+#include <sbi/sbi_byteorder.h>
 
 #define INT_MAX		((int)(~0U >> 1))
 #define UINT_MAX	((unsigned int)~0U)
@@ -41,45 +42,35 @@ typedef uint16_t FDT_BITWISE fdt16_t;
 typedef uint32_t FDT_BITWISE fdt32_t;
 typedef uint64_t FDT_BITWISE fdt64_t;
 
-#define EXTRACT_BYTE(x, n)	((unsigned long long)((uint8_t *)&x)[n])
-#define CPU_TO_FDT16(x) ((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
-#define CPU_TO_FDT32(x) ((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
-			 (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
-#define CPU_TO_FDT64(x) ((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
-			 (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
-			 (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
-			 (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
-
 static inline uint16_t fdt16_to_cpu(fdt16_t x)
 {
-	return (FDT_FORCE uint16_t)CPU_TO_FDT16(x);
+	return (FDT_FORCE uint16_t)be16_to_cpu(x);
 }
+
 static inline fdt16_t cpu_to_fdt16(uint16_t x)
 {
-	return (FDT_FORCE fdt16_t)CPU_TO_FDT16(x);
+	return (FDT_FORCE fdt16_t)cpu_to_be16(x);
 }
 
 static inline uint32_t fdt32_to_cpu(fdt32_t x)
 {
-	return (FDT_FORCE uint32_t)CPU_TO_FDT32(x);
+	return (FDT_FORCE uint32_t)be32_to_cpu(x);
 }
+
 static inline fdt32_t cpu_to_fdt32(uint32_t x)
 {
-	return (FDT_FORCE fdt32_t)CPU_TO_FDT32(x);
+	return (FDT_FORCE fdt32_t)cpu_to_be32(x);
 }
 
 static inline uint64_t fdt64_to_cpu(fdt64_t x)
 {
-	return (FDT_FORCE uint64_t)CPU_TO_FDT64(x);
+	return (FDT_FORCE uint64_t)be64_to_cpu(x);
 }
+
 static inline fdt64_t cpu_to_fdt64(uint64_t x)
 {
-	return (FDT_FORCE fdt64_t)CPU_TO_FDT64(x);
+	return (FDT_FORCE fdt64_t)cpu_to_be64(x);
 }
-#undef CPU_TO_FDT64
-#undef CPU_TO_FDT32
-#undef CPU_TO_FDT16
-#undef EXTRACT_BYTE
 
 #ifdef __APPLE__
 #include <AvailabilityMacros.h>
-- 
2.34.1



From rpathak at ventanamicro.com  Wed Feb  1 20:44:27 2023
From: rpathak at ventanamicro.com (Rahul Pathak)
Date: Thu,  2 Feb 2023 10:14:27 +0530
Subject: [PATCH v2 3/3] include: types: Add typedefs for endianness
In-Reply-To: <20230202044427.1253816-1-rpathak@ventanamicro.com>
References: <20230202044427.1253816-1-rpathak@ventanamicro.com>
Message-ID: <20230202044427.1253816-4-rpathak@ventanamicro.com>

If any variable/memory-location follows certain
endianness then its important to annotate it properly
so that proper conversion can be done before read/write
from that variable/memory.

Also, use these new typedefs in libfdt_env.h for deriving
its own custom fdtX_t types

Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
---
 include/sbi/sbi_types.h       | 7 +++++++
 lib/utils/libfdt/libfdt_env.h | 6 +++---
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/include/sbi/sbi_types.h b/include/sbi/sbi_types.h
index 9c1fef3b8dc6..def88bbad2c3 100644
--- a/include/sbi/sbi_types.h
+++ b/include/sbi/sbi_types.h
@@ -54,6 +54,13 @@ typedef unsigned long		virtual_size_t;
 typedef unsigned long		physical_addr_t;
 typedef unsigned long		physical_size_t;
 
+typedef uint16_t		le16_t;
+typedef uint16_t		be16_t;
+typedef uint32_t		le32_t;
+typedef uint32_t		be32_t;
+typedef uint64_t		le64_t;
+typedef uint64_t		be64_t;
+
 #define true			1
 #define false			0
 
diff --git a/lib/utils/libfdt/libfdt_env.h b/lib/utils/libfdt/libfdt_env.h
index e5ad7698efca..6ebbae01b514 100644
--- a/lib/utils/libfdt/libfdt_env.h
+++ b/lib/utils/libfdt/libfdt_env.h
@@ -38,9 +38,9 @@
 #define strlen		sbi_strlen
 #define strnlen		sbi_strnlen
 
-typedef uint16_t FDT_BITWISE fdt16_t;
-typedef uint32_t FDT_BITWISE fdt32_t;
-typedef uint64_t FDT_BITWISE fdt64_t;
+typedef be16_t FDT_BITWISE fdt16_t;
+typedef be32_t FDT_BITWISE fdt32_t;
+typedef be64_t FDT_BITWISE fdt64_t;
 
 static inline uint16_t fdt16_to_cpu(fdt16_t x)
 {
-- 
2.34.1



From wxjstz at 126.com  Thu Feb  2 00:51:35 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu,  2 Feb 2023 16:51:35 +0800
Subject: [PATCH v4 3/4] lib: sbi: rename function of sbi_ipi
In-Reply-To: <20230202085136.224745-1-wxjstz@126.com>
References: <20230202085136.224745-1-wxjstz@126.com>
Message-ID: <20230202085136.224745-4-wxjstz@126.com>

Renamed some functions and reused some code

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_tlb.c | 27 ++++++++++-----------------
 1 file changed, 10 insertions(+), 17 deletions(-)

diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index 8bab31a..741f4bb 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -211,7 +211,7 @@ static void tlb_pmu_incr_fw_ctr(struct sbi_tlb_info *data)
 		sbi_pmu_ctr_incr_fw(SBI_PMU_FW_HFENCE_VVMA_ASID_SENT);
 }
 
-static void tlb_entry_process(struct sbi_tlb_info *tinfo)
+static void tlb_process_helper(struct sbi_tlb_info *tinfo)
 {
 	u32 rhartid;
 	struct sbi_scratch *rscratch = NULL;
@@ -229,30 +229,23 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 	}
 }
 
-static void tlb_process_count(struct sbi_scratch *scratch, int count)
+static int tlb_process_once(struct sbi_scratch *scratch)
 {
 	struct sbi_tlb_info tinfo;
-	unsigned int deq_count = 0;
 	struct sbi_fifo *tlb_fifo =
 			sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
 
-	while (!sbi_fifo_dequeue(tlb_fifo, &tinfo)) {
-		tlb_entry_process(&tinfo);
-		deq_count++;
-		if (deq_count > count)
-			break;
-
+	if (!sbi_fifo_dequeue(tlb_fifo, &tinfo)) {
+		tlb_process_helper(&tinfo);
+		return 0;
 	}
+
+	return -1;
 }
 
 static void tlb_process(struct sbi_scratch *scratch)
 {
-	struct sbi_tlb_info tinfo;
-	struct sbi_fifo *tlb_fifo =
-			sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
-
-	while (!sbi_fifo_dequeue(tlb_fifo, &tinfo))
-		tlb_entry_process(&tinfo);
+	while (!tlb_process_once(scratch));
 }
 
 static void tlb_sync(struct sbi_scratch *scratch)
@@ -265,7 +258,7 @@ static void tlb_sync(struct sbi_scratch *scratch)
 		 * While we are waiting for remote hart to set the sync,
 		 * consume fifo requests to avoid deadlock.
 		 */
-		tlb_process_count(scratch, 1);
+		tlb_process_once(scratch);
 	}
 
 	return;
@@ -380,7 +373,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 		 * TODO: Introduce a wait/wakeup event mechanism to handle
 		 * this properly.
 		 */
-		tlb_process_count(scratch, 1);
+		tlb_process_once(scratch);
 		sbi_dprintf("hart%d: hart%d tlb fifo full\n",
 			    curr_hartid, remote_hartid);
 	}
-- 
2.39.0



From wxjstz at 126.com  Thu Feb  2 00:51:33 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu,  2 Feb 2023 16:51:33 +0800
Subject: [PATCH v4 1/4] lib: sbi: Optimize sbi_tlb
In-Reply-To: <20230202085136.224745-1-wxjstz@126.com>
References: <20230202085136.224745-1-wxjstz@126.com>
Message-ID: <20230202085136.224745-2-wxjstz@126.com>

Originally, the process and sync of sbi_tlb need to wait for each other.
Evasion by atomic addition and subtraction.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_tlb.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index 4c142ea..8bab31a 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -215,7 +215,7 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 {
 	u32 rhartid;
 	struct sbi_scratch *rscratch = NULL;
-	unsigned long *rtlb_sync = NULL;
+	atomic_t *rtlb_sync = NULL;
 
 	tinfo->local_fn(tinfo);
 
@@ -225,7 +225,7 @@ static void tlb_entry_process(struct sbi_tlb_info *tinfo)
 			continue;
 
 		rtlb_sync = sbi_scratch_offset_ptr(rscratch, tlb_sync_off);
-		while (atomic_raw_xchg_ulong(rtlb_sync, 1)) ;
+		atomic_sub_return(rtlb_sync, 1);
 	}
 }
 
@@ -257,10 +257,10 @@ static void tlb_process(struct sbi_scratch *scratch)
 
 static void tlb_sync(struct sbi_scratch *scratch)
 {
-	unsigned long *tlb_sync =
+	atomic_t *tlb_sync =
 			sbi_scratch_offset_ptr(scratch, tlb_sync_off);
 
-	while (!atomic_raw_xchg_ulong(tlb_sync, 0)) {
+	while (atomic_read(tlb_sync) > 0) {
 		/*
 		 * While we are waiting for remote hart to set the sync,
 		 * consume fifo requests to avoid deadlock.
@@ -343,6 +343,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 			  u32 remote_hartid, void *data)
 {
 	int ret;
+	atomic_t *tlb_sync;
 	struct sbi_fifo *tlb_fifo_r;
 	struct sbi_tlb_info *tinfo = data;
 	u32 curr_hartid = current_hartid();
@@ -369,11 +370,8 @@ static int tlb_update(struct sbi_scratch *scratch,
 	tlb_fifo_r = sbi_scratch_offset_ptr(remote_scratch, tlb_fifo_off);
 
 	ret = sbi_fifo_inplace_update(tlb_fifo_r, data, tlb_update_cb);
-	if (ret != SBI_FIFO_UNCHANGED) {
-		return 1;
-	}
 
-	while (sbi_fifo_enqueue(tlb_fifo_r, data) < 0) {
+	while (ret == SBI_FIFO_UNCHANGED && sbi_fifo_enqueue(tlb_fifo_r, data) < 0) {
 		/**
 		 * For now, Busy loop until there is space in the fifo.
 		 * There may be case where target hart is also
@@ -387,6 +385,9 @@ static int tlb_update(struct sbi_scratch *scratch,
 			    curr_hartid, remote_hartid);
 	}
 
+	tlb_sync = sbi_scratch_offset_ptr(scratch, tlb_sync_off);
+	atomic_add_return(tlb_sync, 1);
+
 	return 0;
 }
 
@@ -413,7 +414,7 @@ int sbi_tlb_init(struct sbi_scratch *scratch, bool cold_boot)
 {
 	int ret;
 	void *tlb_mem;
-	unsigned long *tlb_sync;
+	atomic_t *tlb_sync;
 	struct sbi_fifo *tlb_q;
 	const struct sbi_platform *plat = sbi_platform_ptr(scratch);
 
@@ -455,7 +456,7 @@ int sbi_tlb_init(struct sbi_scratch *scratch, bool cold_boot)
 	tlb_q = sbi_scratch_offset_ptr(scratch, tlb_fifo_off);
 	tlb_mem = sbi_scratch_offset_ptr(scratch, tlb_fifo_mem_off);
 
-	*tlb_sync = 0;
+	tlb_sync->counter = 0;
 
 	sbi_fifo_init(tlb_q, tlb_mem,
 		      SBI_TLB_FIFO_NUM_ENTRIES, SBI_TLB_INFO_SIZE);
-- 
2.39.0



From wxjstz at 126.com  Thu Feb  2 00:51:36 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu,  2 Feb 2023 16:51:36 +0800
Subject: [PATCH v4 4/4] lib: sbi: Optimize sbi_tlb queue waiting
In-Reply-To: <20230202085136.224745-1-wxjstz@126.com>
References: <20230202085136.224745-1-wxjstz@126.com>
Message-ID: <20230202085136.224745-5-wxjstz@126.com>

When tlb_fifo is full, it will wait and affect the ipi update to
other harts. This patch is optimized.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/sbi_ipi.h |  3 +++
 lib/sbi/sbi_ipi.c     | 47 +++++++++++++++++++++----------------------
 lib/sbi/sbi_tlb.c     |  3 ++-
 3 files changed, 28 insertions(+), 25 deletions(-)

diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
index f6ac807..aad7f77 100644
--- a/include/sbi/sbi_ipi.h
+++ b/include/sbi/sbi_ipi.h
@@ -41,6 +41,9 @@ struct sbi_ipi_event_ops {
 	 * Update callback to save/enqueue data for remote HART
 	 * Note: This is an optional callback and it is called just before
 	 * triggering IPI to remote HART.
+	 * @return 0  success
+	 * @return -1 break IPI, done on local hart
+	 * @return -2 need retry
 	 */
 	int (* update)(struct sbi_scratch *scratch,
 			struct sbi_scratch *remote_scratch,
diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 92a33bd..3282aaa 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -53,7 +53,7 @@ static int sbi_ipi_update(struct sbi_scratch *scratch, u32 remote_hartid,
 	if (ipi_ops->update) {
 		ret = ipi_ops->update(scratch, remote_scratch,
 				      remote_hartid, data);
-		if (ret < 0)
+		if (ret != 0)
 			return ret;
 	}
 
@@ -94,51 +94,50 @@ static int sbi_ipi_sync(struct sbi_scratch *scratch, u32 event)
  */
 int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 {
-	int rc;
-	ulong i, m, n;
+	int rc, done;
+	ulong i, m;
+	struct sbi_hartmask target_mask = {0};
+	struct sbi_hartmask retry_mask = {0};
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
+	/* find target harts */
 	if (hbase != -1UL) {
 		rc = sbi_hsm_hart_interruptible_mask(dom, hbase, &m);
 		if (rc)
 			return rc;
 		m &= hmask;
-		n = m;
 
-		/* update IPIs */
 		for (i = hbase; m; i++, m >>= 1) {
 			if (m & 1UL)
-				sbi_ipi_update(scratch, i, event, data);
-		}
-
-		/* sync IPIs */
-		m = n;
-		for (i = hbase; m; i++, m >>= 1) {
-			if (m & 1UL)
-				sbi_ipi_sync(scratch, event);
+				sbi_hartmask_set_hart(i, &target_mask);
 		}
 	} else {
-		/* update IPIs */
 		hbase = 0;
 		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
 			for (i = hbase; m; i++, m >>= 1) {
 				if (m & 1UL)
-					sbi_ipi_update(scratch, i, event, data);
+					sbi_hartmask_set_hart(i, &target_mask);
 			}
 			hbase += BITS_PER_LONG;
 		}
+	}
 
-		/* sync IPIs */
-		hbase = 0;
-		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
-			for (i = hbase; m; i++, m >>= 1) {
-				if (m & 1UL)
-					sbi_ipi_sync(scratch, event);
-			}
-			hbase += BITS_PER_LONG;
+	/* update IPIs */
+	retry_mask = target_mask;
+	do {
+		done = true;
+		sbi_hartmask_for_each_hart(i, &retry_mask) {
+			rc = sbi_ipi_update(scratch, i, event, data);
+			if (rc == -2)
+				done = false;
+			else
+				sbi_hartmask_clear_hart(i, &retry_mask);
 		}
-	}
+	} while (!done);
+
+	/* sync IPIs */
+	sbi_ipi_sync(scratch, event);
 
 	return 0;
 }
diff --git a/lib/sbi/sbi_tlb.c b/lib/sbi/sbi_tlb.c
index 741f4bb..1be2058 100644
--- a/lib/sbi/sbi_tlb.c
+++ b/lib/sbi/sbi_tlb.c
@@ -364,7 +364,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 
 	ret = sbi_fifo_inplace_update(tlb_fifo_r, data, tlb_update_cb);
 
-	while (ret == SBI_FIFO_UNCHANGED && sbi_fifo_enqueue(tlb_fifo_r, data) < 0) {
+	if (ret == SBI_FIFO_UNCHANGED && sbi_fifo_enqueue(tlb_fifo_r, data) < 0) {
 		/**
 		 * For now, Busy loop until there is space in the fifo.
 		 * There may be case where target hart is also
@@ -376,6 +376,7 @@ static int tlb_update(struct sbi_scratch *scratch,
 		tlb_process_once(scratch);
 		sbi_dprintf("hart%d: hart%d tlb fifo full\n",
 			    curr_hartid, remote_hartid);
+		return -2;
 	}
 
 	tlb_sync = sbi_scratch_offset_ptr(scratch, tlb_sync_off);
-- 
2.39.0



From wxjstz at 126.com  Thu Feb  2 00:51:34 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu,  2 Feb 2023 16:51:34 +0800
Subject: [PATCH v4 2/4] lib: sbi: Optimize sbi_ipi
In-Reply-To: <20230202085136.224745-1-wxjstz@126.com>
References: <20230202085136.224745-1-wxjstz@126.com>
Message-ID: <20230202085136.224745-3-wxjstz@126.com>

the original sbI_ipi will be processed by hart by hart, after optimization,
send ipi first and finally wait together.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_ipi.c | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)

diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 1bcc2e4..92a33bd 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -31,7 +31,7 @@ static unsigned long ipi_data_off;
 static const struct sbi_ipi_device *ipi_dev = NULL;
 static const struct sbi_ipi_event_ops *ipi_ops_array[SBI_IPI_EVENT_MAX];
 
-static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
+static int sbi_ipi_update(struct sbi_scratch *scratch, u32 remote_hartid,
 			u32 event, void *data)
 {
 	int ret;
@@ -69,6 +69,18 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
 
 	sbi_pmu_ctr_incr_fw(SBI_PMU_FW_IPI_SENT);
 
+	return 0;
+}
+
+static int sbi_ipi_sync(struct sbi_scratch *scratch, u32 event)
+{
+	const struct sbi_ipi_event_ops *ipi_ops;
+
+	if ((SBI_IPI_EVENT_MAX <= event) ||
+	    !ipi_ops_array[event])
+		return SBI_EINVAL;
+	ipi_ops = ipi_ops_array[event];
+
 	if (ipi_ops->sync)
 		ipi_ops->sync(scratch);
 
@@ -83,7 +95,7 @@ static int sbi_ipi_send(struct sbi_scratch *scratch, u32 remote_hartid,
 int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 {
 	int rc;
-	ulong i, m;
+	ulong i, m, n;
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
@@ -92,19 +104,37 @@ int sbi_ipi_send_many(ulong hmask, ulong hbase, u32 event, void *data)
 		if (rc)
 			return rc;
 		m &= hmask;
+		n = m;
 
-		/* Send IPIs */
+		/* update IPIs */
 		for (i = hbase; m; i++, m >>= 1) {
 			if (m & 1UL)
-				sbi_ipi_send(scratch, i, event, data);
+				sbi_ipi_update(scratch, i, event, data);
+		}
+
+		/* sync IPIs */
+		m = n;
+		for (i = hbase; m; i++, m >>= 1) {
+			if (m & 1UL)
+				sbi_ipi_sync(scratch, event);
 		}
 	} else {
+		/* update IPIs */
+		hbase = 0;
+		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
+			for (i = hbase; m; i++, m >>= 1) {
+				if (m & 1UL)
+					sbi_ipi_update(scratch, i, event, data);
+			}
+			hbase += BITS_PER_LONG;
+		}
+
+		/* sync IPIs */
 		hbase = 0;
 		while (!sbi_hsm_hart_interruptible_mask(dom, hbase, &m)) {
-			/* Send IPIs */
 			for (i = hbase; m; i++, m >>= 1) {
 				if (m & 1UL)
-					sbi_ipi_send(scratch, i, event, data);
+					sbi_ipi_sync(scratch, event);
 			}
 			hbase += BITS_PER_LONG;
 		}
-- 
2.39.0



From wxjstz at 126.com  Thu Feb  2 00:51:32 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu,  2 Feb 2023 16:51:32 +0800
Subject: [PATCH v4 0/4] Miscellaneous about sbi_tlb and sbi_ipi
Message-ID: <20230202085136.224745-1-wxjstz@126.com>

Removed the mutual waiting between process and sync in sbi_tlb
Update ipi first then wait together

V4 Changes:
PATCH1: Fixed not adding one to tlb_sync when SBI_FIFO_UNCHANGED
PATCH4: Add comments

V3 Changes:
PATCH2: Remove missing sync in update
add PATCH4: Optimize sbi_tlb queue waiting

V2 Changes:
PATCH2: Fix m changes after update IPIs

Xiang W (4):
  lib: sbi: Optimize sbi_tlb
  lib: sbi: Optimize sbi_ipi
  lib: sbi: rename function of sbi_ipi
  lib: sbi: Optimize sbi_tlb queue waiting

 include/sbi/sbi_ipi.h |  3 +++
 lib/sbi/sbi_ipi.c     | 43 ++++++++++++++++++++++++++++++-------
 lib/sbi/sbi_tlb.c     | 49 +++++++++++++++++++------------------------
 3 files changed, 61 insertions(+), 34 deletions(-)

-- 
2.39.0



From geomatsi at gmail.com  Thu Feb  2 01:24:04 2023
From: geomatsi at gmail.com (Sergey Matyukevich)
Date: Thu, 2 Feb 2023 12:24:04 +0300
Subject: [PATCH 1/2] include: Add support for byteorder/endianness
 conversion
In-Reply-To: <CA+Oz1=apLYgi29eDggPZF8GKjYteVr3LQDtuVhOSU6wAi84Q9Q@mail.gmail.com>
References: <20230128074617.1917265-1-rpathak@ventanamicro.com>
 <20230128074617.1917265-2-rpathak@ventanamicro.com>
 <Y9jNHM1dHedBPmAf@curiosity>
 <CA+Oz1=apLYgi29eDggPZF8GKjYteVr3LQDtuVhOSU6wAi84Q9Q@mail.gmail.com>
Message-ID: <Y9uBNKl1yPkVRg/D@curiosity>

Hi Rahul,

> Using __builtins_bswapX causes weird issues on gcc,
> due to reasons unknown the functions in libfdt_env.h
> like below emit the library call to __bswapdi2
> which causes errors.
> 
> static inline fdt32_t cpu_to_fdt32(uint32_t x)
> {
> return (FDT_FORCE fdt32_t)cpu_to_be32(x);
> }
> 
> Errors -
> fdt_reset_gpio.c:135: undefined reference to `__bswapsi2'
> /fdt_reset_thead.c:77: undefined reference to `__bswapsi2'
> 
> Clang works perfectly
> 
> Did you face such issues?

Right. I see such issues for gcc revisions v10.2, v11.2, v12.1.
Missing references could be be pulled from libgcc, e.g.: 

diff --git a/Makefile b/Makefile
index 8db6eab..8cae625 100644
--- a/Makefile
+++ b/Makefile
@@ -369,7 +369,7 @@ ASFLAGS		+=	$(firmware-asflags-y)
 ARFLAGS		=	rcs
 
 ELFFLAGS	+=	$(USE_LD_FLAG)
-ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N
+ELFFLAGS	+=	-Wl,--build-id=none -Wl,-N -Wl,-lgcc
 ELFFLAGS	+=	$(platform-ldflags-y)
 ELFFLAGS	+=	$(firmware-ldflags-y)


Regards,
Sergey

> On Tue, Jan 31, 2023 at 1:41 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
> >
> > On Sat, Jan 28, 2023 at 01:16:16PM +0530, Rahul Pathak wrote:
> > > Define macros for general byteorder conversion
> > > Define functions for endianness conversion
> > > using general byteorder conversion macros
> > >
> > > Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
> > > ---
> > >  include/sbi/sbi_byteorder.h | 57 +++++++++++++++++++++++++++++++++++++
> > >  1 file changed, 57 insertions(+)
> > >  create mode 100644 include/sbi/sbi_byteorder.h
> >
> > Reviewed-by: Sergey Matyukevich <sergey.matyukevich at syntacore.com>
> >
> > By the way, is there any advantage of using custom macros instead of
> > compiler builtins ? E.g. see my patch loosely based on FreeBSD code:
> > http://lists.infradead.org/pipermail/opensbi/2022-December/003766.html
> >
> > Regards,
> > Sergey
> >
> >
> > > diff --git a/include/sbi/sbi_byteorder.h b/include/sbi/sbi_byteorder.h
> > > new file mode 100644
> > > index 000000000000..680710fe6d91
> > > --- /dev/null
> > > +++ b/include/sbi/sbi_byteorder.h
> > > @@ -0,0 +1,57 @@
> > > +/*
> > > + * SPDX-License-Identifier: BSD-2-Clause
> > > + *
> > > + * Copyright (c) 2023 Ventana Micro Systems Inc.
> > > + */
> > > +
> > > +#ifndef __SBI_BYTEORDER_H__
> > > +#define __SBI_BYTEORDER_H__
> > > +
> > > +#include <sbi/sbi_types.h>
> > > +
> > > +#define EXTRACT_BYTE(x, n)   ((unsigned long long)((uint8_t *)&x)[n])
> > > +
> > > +#define BSWAP16(x)   ((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
> > > +#define BSWAP32(x)   ((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
> > > +                      (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
> > > +#define BSWAP64(x)   ((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
> > > +                      (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
> > > +                      (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
> > > +                      (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
> > > +
> > > +
> > > +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__  /* CPU(little-endian) */
> > > +#define cpu_to_be16(x)               ((uint16_t)BSWAP16(x))
> > > +#define cpu_to_be32(x)               ((uint32_t)BSWAP32(x))
> > > +#define cpu_to_be64(x)               ((uint64_t)BSWAP64(x))
> > > +
> > > +#define be16_to_cpu(x)               ((uint16_t)BSWAP16(x))
> > > +#define be32_to_cpu(x)               ((uint32_t)BSWAP32(x))
> > > +#define be64_to_cpu(x)               ((uint64_t)BSWAP64(x))
> > > +
> > > +#define cpu_to_le16(x)               ((uint16_t)(x))
> > > +#define cpu_to_le32(x)               ((uint32_t)(x))
> > > +#define cpu_to_le64(x)               ((uint64_t)(x))
> > > +
> > > +#define le16_to_cpu(x)               ((uint16_t)(x))
> > > +#define le32_to_cpu(x)               ((uint32_t)(x))
> > > +#define le64_to_cpu(x)               ((uint64_t)(x))
> > > +#else /* CPU(big-endian) */
> > > +#define cpu_to_be16(x)               ((uint16_t)(x))
> > > +#define cpu_to_be32(x)               ((uint32_t)(x))
> > > +#define cpu_to_be64(x)               ((uint64_t)(x))
> > > +
> > > +#define be16_to_cpu(x)               ((uint16_t)(x))
> > > +#define be32_to_cpu(x)               ((uint32_t)(x))
> > > +#define be64_to_cpu(x)               ((uint64_t)(x))
> > > +
> > > +#define cpu_to_le16(x)               ((uint16_t)BSWAP16(x))
> > > +#define cpu_to_le32(x)               ((uint32_t)BSWAP32(x))
> > > +#define cpu_to_le64(x)               ((uint64_t)BSWAP64(x))
> > > +
> > > +#define le16_to_cpu(x)               ((uint16_t)BSWAP16(x))
> > > +#define le32_to_cpu(x)               ((uint32_t)BSWAP32(x))
> > > +#define le64_to_cpu(x)               ((uint64_t)BSWAP64(x))
> > > +#endif
> > > +
> > > +#endif /* __SBI_BYTEORDER_H__ */
> > > --
> > > 2.34.1
> > >
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
> 
> 
> 
> -- 
> 
> Thanks
> Rahul Pathak


From rpathak at ventanamicro.com  Thu Feb  2 01:32:52 2023
From: rpathak at ventanamicro.com (Rahul Pathak)
Date: Thu, 2 Feb 2023 15:02:52 +0530
Subject: [PATCH 1/2] include: Add support for byteorder/endianness conversion
In-Reply-To: <Y9uBNKl1yPkVRg/D@curiosity>
References: <20230128074617.1917265-1-rpathak@ventanamicro.com>
 <20230128074617.1917265-2-rpathak@ventanamicro.com> <Y9jNHM1dHedBPmAf@curiosity>
 <CA+Oz1=apLYgi29eDggPZF8GKjYteVr3LQDtuVhOSU6wAi84Q9Q@mail.gmail.com> <Y9uBNKl1yPkVRg/D@curiosity>
Message-ID: <CA+Oz1=ayx=Y8E7BCfM1yj6_q5zNb-YN4JXP1X7H7y2eSrwXecg@mail.gmail.com>

But should libgcc be pulled in the opensbi, I doubt that.

Also just for trying i did tried linking with libgcc, it was giving me
build errors
libgcc.a(_bswapdi2.o): can't link double-float modules with soft-float modules

On Thu, Feb 2, 2023 at 2:54 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
>
> Hi Rahul,
>
> > Using __builtins_bswapX causes weird issues on gcc,
> > due to reasons unknown the functions in libfdt_env.h
> > like below emit the library call to __bswapdi2
> > which causes errors.
> >
> > static inline fdt32_t cpu_to_fdt32(uint32_t x)
> > {
> > return (FDT_FORCE fdt32_t)cpu_to_be32(x);
> > }
> >
> > Errors -
> > fdt_reset_gpio.c:135: undefined reference to `__bswapsi2'
> > /fdt_reset_thead.c:77: undefined reference to `__bswapsi2'
> >
> > Clang works perfectly
> >
> > Did you face such issues?
>
> Right. I see such issues for gcc revisions v10.2, v11.2, v12.1.
> Missing references could be be pulled from libgcc, e.g.:
>
> diff --git a/Makefile b/Makefile
> index 8db6eab..8cae625 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -369,7 +369,7 @@ ASFLAGS             +=      $(firmware-asflags-y)
>  ARFLAGS                =       rcs
>
>  ELFFLAGS       +=      $(USE_LD_FLAG)
> -ELFFLAGS       +=      -Wl,--build-id=none -Wl,-N
> +ELFFLAGS       +=      -Wl,--build-id=none -Wl,-N -Wl,-lgcc
>  ELFFLAGS       +=      $(platform-ldflags-y)
>  ELFFLAGS       +=      $(firmware-ldflags-y)
>
>
> Regards,
> Sergey
>
> > On Tue, Jan 31, 2023 at 1:41 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
> > >
> > > On Sat, Jan 28, 2023 at 01:16:16PM +0530, Rahul Pathak wrote:
> > > > Define macros for general byteorder conversion
> > > > Define functions for endianness conversion
> > > > using general byteorder conversion macros
> > > >
> > > > Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
> > > > ---
> > > >  include/sbi/sbi_byteorder.h | 57 +++++++++++++++++++++++++++++++++++++
> > > >  1 file changed, 57 insertions(+)
> > > >  create mode 100644 include/sbi/sbi_byteorder.h
> > >
> > > Reviewed-by: Sergey Matyukevich <sergey.matyukevich at syntacore.com>
> > >
> > > By the way, is there any advantage of using custom macros instead of
> > > compiler builtins ? E.g. see my patch loosely based on FreeBSD code:
> > > http://lists.infradead.org/pipermail/opensbi/2022-December/003766.html
> > >
> > > Regards,
> > > Sergey
> > >
> > >
> > > > diff --git a/include/sbi/sbi_byteorder.h b/include/sbi/sbi_byteorder.h
> > > > new file mode 100644
> > > > index 000000000000..680710fe6d91
> > > > --- /dev/null
> > > > +++ b/include/sbi/sbi_byteorder.h
> > > > @@ -0,0 +1,57 @@
> > > > +/*
> > > > + * SPDX-License-Identifier: BSD-2-Clause
> > > > + *
> > > > + * Copyright (c) 2023 Ventana Micro Systems Inc.
> > > > + */
> > > > +
> > > > +#ifndef __SBI_BYTEORDER_H__
> > > > +#define __SBI_BYTEORDER_H__
> > > > +
> > > > +#include <sbi/sbi_types.h>
> > > > +
> > > > +#define EXTRACT_BYTE(x, n)   ((unsigned long long)((uint8_t *)&x)[n])
> > > > +
> > > > +#define BSWAP16(x)   ((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
> > > > +#define BSWAP32(x)   ((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
> > > > +                      (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
> > > > +#define BSWAP64(x)   ((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
> > > > +                      (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
> > > > +                      (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
> > > > +                      (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
> > > > +
> > > > +
> > > > +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__  /* CPU(little-endian) */
> > > > +#define cpu_to_be16(x)               ((uint16_t)BSWAP16(x))
> > > > +#define cpu_to_be32(x)               ((uint32_t)BSWAP32(x))
> > > > +#define cpu_to_be64(x)               ((uint64_t)BSWAP64(x))
> > > > +
> > > > +#define be16_to_cpu(x)               ((uint16_t)BSWAP16(x))
> > > > +#define be32_to_cpu(x)               ((uint32_t)BSWAP32(x))
> > > > +#define be64_to_cpu(x)               ((uint64_t)BSWAP64(x))
> > > > +
> > > > +#define cpu_to_le16(x)               ((uint16_t)(x))
> > > > +#define cpu_to_le32(x)               ((uint32_t)(x))
> > > > +#define cpu_to_le64(x)               ((uint64_t)(x))
> > > > +
> > > > +#define le16_to_cpu(x)               ((uint16_t)(x))
> > > > +#define le32_to_cpu(x)               ((uint32_t)(x))
> > > > +#define le64_to_cpu(x)               ((uint64_t)(x))
> > > > +#else /* CPU(big-endian) */
> > > > +#define cpu_to_be16(x)               ((uint16_t)(x))
> > > > +#define cpu_to_be32(x)               ((uint32_t)(x))
> > > > +#define cpu_to_be64(x)               ((uint64_t)(x))
> > > > +
> > > > +#define be16_to_cpu(x)               ((uint16_t)(x))
> > > > +#define be32_to_cpu(x)               ((uint32_t)(x))
> > > > +#define be64_to_cpu(x)               ((uint64_t)(x))
> > > > +
> > > > +#define cpu_to_le16(x)               ((uint16_t)BSWAP16(x))
> > > > +#define cpu_to_le32(x)               ((uint32_t)BSWAP32(x))
> > > > +#define cpu_to_le64(x)               ((uint64_t)BSWAP64(x))
> > > > +
> > > > +#define le16_to_cpu(x)               ((uint16_t)BSWAP16(x))
> > > > +#define le32_to_cpu(x)               ((uint32_t)BSWAP32(x))
> > > > +#define le64_to_cpu(x)               ((uint64_t)BSWAP64(x))
> > > > +#endif
> > > > +
> > > > +#endif /* __SBI_BYTEORDER_H__ */
> > > > --
> > > > 2.34.1
> > > >
> > > >
> > > > --
> > > > opensbi mailing list
> > > > opensbi at lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/opensbi
> >
> >
> >
> > --
> >
> > Thanks
> > Rahul Pathak



-- 

Thanks
Rahul Pathak


From wxjstz at 126.com  Sat Feb  4 05:14:45 2023
From: wxjstz at 126.com (Xiang W)
Date: Sat, 04 Feb 2023 21:14:45 +0800
Subject: [PATCH v5 6/8] lib: sbi: Add console_puts() callback in the
 console device
In-Reply-To: <20230201090013.ulgfsudxkf3zjhtw@orel>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
	 <20230113114110.1916226-7-apatel@ventanamicro.com>
	 <20230201090013.ulgfsudxkf3zjhtw@orel>
Message-ID: <eda14e33346db2fc0cca64216ad39323fd6912ae.camel@126.com>

? 2023-02-01???? 10:00 +0100?Andrew Jones???
> On Fri, Jan 13, 2023 at 05:11:08PM +0530, Anup Patel wrote:
> > We add console_puts() callback in the console device which allows
> > console drivers (such as semihosting) to implement a specialized
> > way to output character string.
> > 
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > Reviewed-by: Atish Patra <atishp at rivosinc.com>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
> > Reviewed-by: Bin Meng <bmeng at tinylab.org>
> > ---
> > ?include/sbi/sbi_console.h |? 3 +++
> > ?lib/sbi/sbi_console.c???? | 19 ++++++++++++++-----
> > ?2 files changed, 17 insertions(+), 5 deletions(-)
> > 
> > diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
> > index 660c239..8e95b9d 100644
> > --- a/include/sbi/sbi_console.h
> > +++ b/include/sbi/sbi_console.h
> > @@ -19,6 +19,9 @@ struct sbi_console_device {
> > ????????/** Write a character to the console output */
> > ????????void (*console_putc)(char ch);
> > ?
> > +???????/** Write a character string to the console output */
> > +???????void (*console_puts)(const char *str, unsigned long len);
We should add a comment here that \r\n needs to be output when \n is output.
Otherwise, the output effect of sbi_nputs will be inconsistent.

Regards,
Xiang W
> > +
> > ????????/** Read a character from the console input */
> > ????????int (*console_getc)(void);
> > ?};
> > diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> > index 89d6a49..c1b9f73 100644
> > --- a/lib/sbi/sbi_console.c
> > +++ b/lib/sbi/sbi_console.c
> > @@ -12,6 +12,7 @@
> > ?#include <sbi/sbi_hart.h>
> > ?#include <sbi/sbi_platform.h>
> > ?#include <sbi/sbi_scratch.h>
> > +#include <sbi/sbi_string.h>
> > ?
> > ?static const struct sbi_console_device *console_dev = NULL;
> > ?static spinlock_t console_out_lock??????????? = SPIN_LOCK_INITIALIZER;
> > @@ -44,9 +45,13 @@ void sbi_putc(char ch)
> > ?void sbi_puts(const char *str)
> > ?{
> > ????????spin_lock(&console_out_lock);
> > -???????while (*str) {
> > -???????????????sbi_putc(*str);
> > -???????????????str++;
> > +???????if (console_dev && console_dev->console_puts) {
> > +???????????????console_dev->console_puts(str, sbi_strlen(str));
> > +???????} else {
> > +???????????????while (*str) {
> > +???????????????????????sbi_putc(*str);
> > +???????????????????????str++;
> > +???????????????}
> > ????????}
> > ????????spin_unlock(&console_out_lock);
> > ?}
> > @@ -56,8 +61,12 @@ void sbi_nputs(const char *str, unsigned long len)
> > ????????unsigned long i;
> > ?
> > ????????spin_lock(&console_out_lock);
> > -???????for (i = 0; i < len; i++)
> > -???????????????sbi_putc(str[i]);
> > +???????if (console_dev && console_dev->console_puts) {
> > +???????????????console_dev->console_puts(str, len);
> > +???????} else {
> > +???????????????for (i = 0; i < len; i++)
> > +???????????????????????sbi_putc(str[i]);
> > +???????}
> > ????????spin_unlock(&console_out_lock);
> > ?}
> > ?
> > -- 
> > 2.34.1
> > 
> 
> Reviewed-by: Andrew Jones <ajones at ventanamicro.com>


From prabhakar.mahadev-lad.rj at bp.renesas.com  Sun Feb  5 16:06:21 2023
From: prabhakar.mahadev-lad.rj at bp.renesas.com (Lad Prabhakar)
Date: Mon,  6 Feb 2023 00:06:21 +0000
Subject: [PATCH v4 0/3] Renesas RZ/Five add support to configure PMA and add vendor specific SBI extension
Message-ID: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>

Hi All,

This patch series aims to add support to configure the PMA regions present
on the Andes AX45MP core (andes45-pma.c which can be shared with other
Andes vendors) and configures the required PMA region for RZ/Five SoC.

This patch series also adds the SBI vendor extension
RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to check if IOCP is present on
the core and if un-available it also checks if SW workaround for cache
management needs to be applied.

RFC v3 -> RFC v4
-> Used bit field for flags in struct andes45_pma_region
-> Renamed andes45-pma.h -> andes45_pma.h
-> Included RB tag from Lin san for patch 2/3
-> Moved CSR to platform/generic/include/andes/andes45.h
-> Included sbi/sbi_error.h in andes45-pma.c to fix build issue

RFC v2 -> RFC v3
-> Fixed review comments pointed by Lin-san
   * Used switch case macro
   * Added a check to verify the PMA size is power-of-2
   * Dropped un-needed calculation done in PMA code for NAPOT.
   * Dropped MISA macro
-> Fixed comments pointed by Himanshu
-> Fixed check patch issue

RFC v2 -> RFC v1
-> Fixed review comments pointed by Lin-san
-> Implemented PMA as library so that it can be shared

RFC v1:
https://patchwork.ozlabs.org/project/opensbi/patch/20221212094421.14556-1-prabhakar.mahadev-lad.rj at bp.renesas.com/

Cheers,
Prabhakar

Lad Prabhakar (3):
  platform: generic: renesas: rzfive: Add support to configure the PMA
  platform: generic: renesas: rzfive: Configure the PMA region
  platform: generic: renesas: rzfive: Add SBI EXT to check for enabling
    IOCP errata

 platform/generic/Kconfig                      |   9 +
 platform/generic/include/andes/andes45.h      |  25 +-
 platform/generic/include/andes45_pma.h        |  48 +++
 platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
 platform/generic/renesas/rzfive/objects.mk    |   1 +
 platform/generic/renesas/rzfive/rzfive.c      |  68 ++++
 6 files changed, 499 insertions(+), 2 deletions(-)
 create mode 100644 platform/generic/include/andes45_pma.h
 create mode 100644 platform/generic/renesas/rzfive/andes45-pma.c

-- 
2.17.1



From prabhakar.mahadev-lad.rj at bp.renesas.com  Sun Feb  5 16:06:22 2023
From: prabhakar.mahadev-lad.rj at bp.renesas.com (Lad Prabhakar)
Date: Mon,  6 Feb 2023 00:06:22 +0000
Subject: [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA
In-Reply-To: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <20230206000624.6544-2-prabhakar.mahadev-lad.rj@bp.renesas.com>

I/O Coherence Port (IOCP) provides an AXI interface for connecting
external non-caching masters, such as DMA controllers. The accesses
from IOCP are coherent with D-Caches and L2 Cache.

IOCP is a specification option and is disabled on the Renesas RZ/Five
SoC due to this reason IP blocks using DMA will fail.

The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
block that allows dynamic adjustment of memory attributes in the runtime.
It contains a configurable amount of PMA entries implemented as CSR
registers to control the attributes of memory locations in interest.
Below are the memory attributes supported:
* Device, Non-bufferable
* Device, bufferable
* Memory, Non-cacheable, Non-bufferable
* Memory, Non-cacheable, Bufferable
* Memory, Write-back, No-allocate
* Memory, Write-back, Read-allocate
* Memory, Write-back, Write-allocate
* Memory, Write-back, Read and Write-allocate

More info about PMA (section 10.3):
Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf

As a workaround for SoCs with IOCP disabled CMO needs to be handled by
software. Firstly OpenSBI configures the memory region as
"Memory, Non-cacheable, Bufferable" and passes this region as a global
shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
allocations happen from this region and synchronization callbacks are
implemented to synchronize when doing DMA transactions.

Example PMA region passed as a DT node from OpenSBI:
    reserved-memory {
        #address-cells = <2>;
        #size-cells = <2>;
        ranges;

        pma_resv0 at 58000000 {
            compatible = "shared-dma-pool";
            reg = <0x0 0x58000000 0x0 0x08000000>;
            no-map;
            linux,dma-default;
        };
    };

Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
 platform/generic/Kconfig                      |   8 +
 platform/generic/include/andes45_pma.h        |  48 +++
 platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
 platform/generic/renesas/rzfive/objects.mk    |   1 +
 4 files changed, 407 insertions(+)
 create mode 100644 platform/generic/include/andes45_pma.h
 create mode 100644 platform/generic/renesas/rzfive/andes45-pma.c

diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index c7f198a..9ee9fe3 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -51,3 +51,11 @@ config PLATFORM_STARFIVE_JH7110
 	default n
 
 endif
+
+if PLATFORM_RENESAS_RZFIVE
+
+config ANDES45_PMA
+	bool "Andes PMA support"
+	default n
+
+endif
diff --git a/platform/generic/include/andes45_pma.h b/platform/generic/include/andes45_pma.h
new file mode 100644
index 0000000..37ec77c
--- /dev/null
+++ b/platform/generic/include/andes45_pma.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023 Renesas Electronics Corp.
+ */
+
+#ifndef _ANDES45_PMA_H_
+#define _ANDES45_PMA_H_
+
+#include <sbi/sbi_types.h>
+
+#define ANDES45_MAX_PMA_REGIONS			16
+
+/* Naturally aligned power of 2 region */
+#define ANDES45_PMACFG_ETYP_NAPOT		3
+
+/* Memory, Non-cacheable, Bufferable */
+#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF	(3 << 2)
+
+/**
+ * struct andes45_pma_region - Describes PMA regions
+ *
+ * @pa: Address to be configured in the PMA
+ * @size: Size of the region
+ * @flags: Flags to be set for the PMA region
+ * @dt_populate: Boolean flag indicating if the DT entry should be
+ *               populated for the given PMA region
+ * @shared_dma: Boolean flag if set "shared-dma-pool" property will
+ *              be set in the DT node
+ * @no_map: Boolean flag if set "no-map" property will be set in the
+ *          DT node
+ * @dma_default: Boolean flag if set "linux,dma-default" property will
+ *              be set in the DT node. Note Linux expects single node
+ *              with this property set.
+ */
+struct andes45_pma_region {
+	unsigned long pa;
+	unsigned long size;
+	u8 flags:7;
+	bool dt_populate;
+	bool shared_dma;
+	bool no_map;
+	bool dma_default;
+};
+
+int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
+			      unsigned int pma_regions_count);
+
+#endif /* _ANDES45_PMA_H_ */
diff --git a/platform/generic/renesas/rzfive/andes45-pma.c b/platform/generic/renesas/rzfive/andes45-pma.c
new file mode 100644
index 0000000..881a628
--- /dev/null
+++ b/platform/generic/renesas/rzfive/andes45-pma.c
@@ -0,0 +1,350 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2023 Renesas Electronics Corp.
+ *
+ * Copyright (c) 2020 Andes Technology Corporation
+ *
+ * Authors:
+ *      Nick Hu <nickhu at andestech.com>
+ *      Nylon Chen <nylon7 at andestech.com>
+ *      Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
+ */
+
+#include <andes45_pma.h>
+#include <libfdt.h>
+#include <sbi/riscv_asm.h>
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_console.h>
+#include <sbi/sbi_error.h>
+#include <sbi_utils/fdt/fdt_helper.h>
+
+/* Configuration Registers */
+#define ANDES45_CSR_MMSC_CFG		0xFC2
+#define ANDES45_CSR_MMSC_PPMA_OFFSET	(1 << 30)
+
+#define ANDES45_PMAADDR_0		0xBD0
+
+#define ANDES45_PMACFG_0		0xBC0
+
+static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
+{
+#define switchcase_pma_cfg_read(__pma_cfg_off, __val)		\
+	case __pma_cfg_off:					\
+		__val = csr_read(__pma_cfg_off);		\
+		break;
+#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val)		\
+	switchcase_pma_cfg_read(__pma_cfg_off + 0, __val)	\
+	switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
+
+	unsigned long ret = 0;
+
+	switch (pma_cfg_off) {
+	switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
+
+	default:
+		sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
+		break;
+	}
+
+	return ret;
+
+#undef switchcase_pma_cfg_read_2
+#undef switchcase_pma_cfg_read
+}
+
+static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
+{
+#define switchcase_pma_cfg_write(__pma_cfg_off, __val)		\
+	case __pma_cfg_off:					\
+		csr_write(__pma_cfg_off, __val);		\
+		break;
+#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val)	\
+	switchcase_pma_cfg_write(__pma_cfg_off + 0, __val)	\
+	switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
+
+	switch (pma_cfg_off) {
+	switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
+
+	default:
+		sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
+		break;
+	}
+
+#undef switchcase_pma_cfg_write_2
+#undef switchcase_pma_cfg_write
+}
+
+static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
+{
+#define switchcase_pma_write(__pma_addr_off, __val)		\
+	case __pma_addr_off:					\
+		csr_write(__pma_addr_off, __val);		\
+		break;
+#define switchcase_pma_write_2(__pma_addr_off, __val)		\
+	switchcase_pma_write(__pma_addr_off + 0, __val)		\
+	switchcase_pma_write(__pma_addr_off + 1, __val)
+#define switchcase_pma_write_4(__pma_addr_off, __val)		\
+	switchcase_pma_write_2(__pma_addr_off + 0, __val)	\
+	switchcase_pma_write_2(__pma_addr_off + 2, __val)
+#define switchcase_pma_write_8(__pma_addr_off, __val)		\
+	switchcase_pma_write_4(__pma_addr_off + 0, __val)	\
+	switchcase_pma_write_4(__pma_addr_off + 4, __val)
+#define switchcase_pma_write_16(__pma_addr_off, __val)		\
+	switchcase_pma_write_8(__pma_addr_off + 0, __val)	\
+	switchcase_pma_write_8(__pma_addr_off + 8, __val)
+
+	switch (pma_addr_off) {
+	switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
+
+	default:
+		sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
+		break;
+	}
+
+#undef switchcase_pma_write_16
+#undef switchcase_pma_write_8
+#undef switchcase_pma_write_4
+#undef switchcase_pma_write_2
+#undef switchcase_pma_write
+}
+
+static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
+{
+#define switchcase_pma_read(__pma_addr_off, __val)		\
+	case __pma_addr_off:					\
+		__val = csr_read(__pma_addr_off);		\
+		break;
+#define switchcase_pma_read_2(__pma_addr_off, __val)		\
+	switchcase_pma_read(__pma_addr_off + 0, __val)		\
+	switchcase_pma_read(__pma_addr_off + 1, __val)
+#define switchcase_pma_read_4(__pma_addr_off, __val)		\
+	switchcase_pma_read_2(__pma_addr_off + 0, __val)	\
+	switchcase_pma_read_2(__pma_addr_off + 2, __val)
+#define switchcase_pma_read_8(__pma_addr_off, __val)		\
+	switchcase_pma_read_4(__pma_addr_off + 0, __val)	\
+	switchcase_pma_read_4(__pma_addr_off + 4, __val)
+#define switchcase_pma_read_16(__pma_addr_off, __val)		\
+	switchcase_pma_read_8(__pma_addr_off + 0, __val)	\
+	switchcase_pma_read_8(__pma_addr_off + 8, __val)
+
+	unsigned long ret = 0;
+
+	switch (pma_addr_off) {
+	switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
+
+	default:
+		sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
+		break;
+	}
+
+	return ret;
+
+#undef switchcase_pma_read_16
+#undef switchcase_pma_read_8
+#undef switchcase_pma_read_4
+#undef switchcase_pma_read_2
+#undef switchcase_pma_read
+}
+
+static unsigned long
+andes45_pma_setup(const struct andes45_pma_region *pma_region,
+		  unsigned int entry_id)
+{
+	unsigned long size = pma_region->size;
+	unsigned long addr = pma_region->pa;
+	unsigned int pma_cfg_addr;
+	unsigned long pmacfg_val;
+	unsigned long pmaaddr;
+	char *pmaxcfg;
+
+	/* Check for 4KiB granularity */
+	if (size < (1 << 12))
+		return SBI_EINVAL;
+
+	/* Check size is power of 2 */
+	if (size & (size - 1))
+		return SBI_EINVAL;
+
+	if (entry_id > 15)
+		return SBI_EINVAL;
+
+	if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
+		return SBI_EINVAL;
+
+	if ((addr & (size - 1)) != 0)
+		return SBI_EINVAL;
+
+	pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
+	pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
+	pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
+	*pmaxcfg = 0;
+	*pmaxcfg = pma_region->flags;
+
+	andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
+
+	pmaaddr = (addr >> 2) + (size >> 3) - 1;
+
+	andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
+
+	return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
+			pmaaddr : SBI_EINVAL;
+}
+
+static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
+				unsigned int index, int parent)
+{
+	int na = fdt_address_cells(fdt, 0);
+	int ns = fdt_size_cells(fdt, 0);
+	static bool dma_default = false;
+	fdt32_t addr_high, addr_low;
+	fdt32_t size_high, size_low;
+	int subnode, err;
+	fdt32_t reg[4];
+	fdt32_t *val;
+	char name[32];
+
+	addr_high = (u64)pma->pa >> 32;
+	addr_low = pma->pa;
+	size_high = (u64)pma->size >> 32;
+	size_low = pma->size;
+
+	if (na > 1 && addr_high)
+		sbi_snprintf(name, sizeof(name),
+			     "pma_resv%d@%x,%x", index,
+			     addr_high, addr_low);
+	else
+		sbi_snprintf(name, sizeof(name),
+			     "pma_resv%d@%x", index,
+			     addr_low);
+
+	subnode = fdt_add_subnode(fdt, parent, name);
+	if (subnode < 0)
+		return subnode;
+
+	if (pma->shared_dma) {
+		err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
+		if (err < 0)
+			return err;
+	}
+
+	if (pma->no_map) {
+		err = fdt_setprop_empty(fdt, subnode, "no-map");
+		if (err < 0)
+			return err;
+	}
+
+	/* Linux allows single linux,dma-default region. */
+	if (pma->dma_default) {
+		if (dma_default)
+			return SBI_EINVAL;
+
+		err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
+		if (err < 0)
+			return err;
+		dma_default = true;
+	}
+
+	/* encode the <reg> property value */
+	val = reg;
+	if (na > 1)
+		*val++ = cpu_to_fdt32(addr_high);
+	*val++ = cpu_to_fdt32(addr_low);
+	if (ns > 1)
+		*val++ = cpu_to_fdt32(size_high);
+	*val++ = cpu_to_fdt32(size_low);
+
+	err = fdt_setprop(fdt, subnode, "reg", reg,
+			  (na + ns) * sizeof(fdt32_t));
+	if (err < 0)
+		return err;
+
+	return 0;
+}
+
+static int andes45_fdt_reserved_memory_fixup(void *fdt,
+					     const struct andes45_pma_region *pma,
+					     unsigned int entry)
+{
+	int parent;
+
+	/* try to locate the reserved memory node */
+	parent = fdt_path_offset(fdt, "/reserved-memory");
+	if (parent < 0) {
+		int na = fdt_address_cells(fdt, 0);
+		int ns = fdt_size_cells(fdt, 0);
+		int err;
+
+		/* if such node does not exist, create one */
+		parent = fdt_add_subnode(fdt, 0, "reserved-memory");
+		if (parent < 0)
+			return parent;
+
+		err = fdt_setprop_empty(fdt, parent, "ranges");
+		if (err < 0)
+			return err;
+
+		err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
+		if (err < 0)
+			return err;
+
+		err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
+		if (err < 0)
+			return err;
+	}
+
+	return andes45_fdt_pma_resv(fdt, pma, entry, parent);
+}
+
+int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
+			      unsigned int pma_regions_count)
+{
+	unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
+	unsigned int dt_populate_cnt;
+	unsigned int i, j;
+	unsigned long pa;
+	void *fdt;
+	int ret;
+
+	if (!pma_regions || !pma_regions_count)
+		return 0;
+
+	if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
+		return SBI_EINVAL;
+
+	if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
+		return SBI_ENOTSUPP;
+
+	/* Configure the PMA regions */
+	for (i = 0; i < pma_regions_count; i++) {
+		pa = andes45_pma_setup(&pma_regions[i], i);
+		if (pa == SBI_EINVAL)
+			return SBI_EINVAL;
+	}
+
+	dt_populate_cnt = 0;
+	for (i = 0; i < pma_regions_count; i++) {
+		if (!pma_regions[i].dt_populate)
+			continue;
+		dt_populate_cnt++;
+	}
+
+	if (!dt_populate_cnt)
+		return 0;
+
+	fdt = fdt_get_address();
+
+	ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
+	if (ret < 0)
+		return ret;
+
+	for (i = 0, j = 0; i < pma_regions_count; i++) {
+		if (!pma_regions[i].dt_populate)
+			continue;
+
+		ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
diff --git a/platform/generic/renesas/rzfive/objects.mk b/platform/generic/renesas/rzfive/objects.mk
index 2e7e37f..fd9e7c4 100644
--- a/platform/generic/renesas/rzfive/objects.mk
+++ b/platform/generic/renesas/rzfive/objects.mk
@@ -6,3 +6,4 @@
 
 carray-platform_override_modules-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas_rzfive
 platform-objs-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas/rzfive/rzfive.o
+platform-objs-$(CONFIG_ANDES45_PMA) += renesas/rzfive/andes45-pma.o
-- 
2.17.1



From prabhakar.mahadev-lad.rj at bp.renesas.com  Sun Feb  5 16:06:23 2023
From: prabhakar.mahadev-lad.rj at bp.renesas.com (Lad Prabhakar)
Date: Mon,  6 Feb 2023 00:06:23 +0000
Subject: [PATCH v4 2/3] platform: generic: renesas: rzfive: Configure the PMA region
In-Reply-To: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <20230206000624.6544-3-prabhakar.mahadev-lad.rj@bp.renesas.com>

On the Renesas RZ/Five SoC by default we want to configure 128MiB of memory
ranging from 0x58000000 as a non-cacheable + bufferable region in the PMA
and populate this region as PMA reserve DT node with shared DMA pool and
no-map flags set so that Linux drivers requesting any DMA'able memory go
through this region.

PMA node passed to the above stack:

        reserved-memory {
            #address-cells = <2>;
            #size-cells = <2>;
            ranges;

            pma_resv0 at 58000000 {
                compatible = "shared-dma-pool";
                reg = <0x0 0x58000000 0x0 0x08000000>;
                no-map;
                linux,dma-default;
            };
        };

Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
 platform/generic/Kconfig                 |  1 +
 platform/generic/renesas/rzfive/rzfive.c | 21 +++++++++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index 9ee9fe3..bc5c280 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -35,6 +35,7 @@ config PLATFORM_ANDES_AE350
 
 config PLATFORM_RENESAS_RZFIVE
 	bool "Renesas RZ/Five support"
+	select ANDES45_PMA
 	default n
 
 config PLATFORM_SIFIVE_FU540
diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
index ee9c9c4..eee9c51 100644
--- a/platform/generic/renesas/rzfive/rzfive.c
+++ b/platform/generic/renesas/rzfive/rzfive.c
@@ -4,10 +4,30 @@
  *
  */
 
+#include <andes45_pma.h>
 #include <platform_override.h>
 #include <sbi/sbi_domain.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 
+static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
+	{
+		.pa = 0x58000000,
+		.size = 0x8000000,
+		.flags = ANDES45_PMACFG_ETYP_NAPOT |
+			 ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF,
+		.dt_populate = true,
+		.shared_dma = true,
+		.no_map = true,
+		.dma_default = true,
+	},
+};
+
+static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *match)
+{
+	return andes45_pma_setup_regions(renesas_rzfive_pma_regions,
+					 array_size(renesas_rzfive_pma_regions));
+}
+
 int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
 {
 	/*
@@ -34,4 +54,5 @@ static const struct fdt_match renesas_rzfive_match[] = {
 const struct platform_override renesas_rzfive = {
 	.match_table = renesas_rzfive_match,
 	.early_init = renesas_rzfive_early_init,
+	.final_init = renesas_rzfive_final_init,
 };
-- 
2.17.1



From prabhakar.mahadev-lad.rj at bp.renesas.com  Sun Feb  5 16:06:24 2023
From: prabhakar.mahadev-lad.rj at bp.renesas.com (Lad Prabhakar)
Date: Mon,  6 Feb 2023 00:06:24 +0000
Subject: [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata
In-Reply-To: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <20230206000624.6544-4-prabhakar.mahadev-lad.rj@bp.renesas.com>

I/O Coherence Port (IOCP) provides an AXI interface for connecting
external non-caching masters, such as DMA controllers. The accesses
from IOCP are coherent with D-Caches and L2 Cache.

IOCP is a specification option and is disabled on the Renesas RZ/Five
SoC due to this reason IP blocks using DMA will fail.

As a workaround for SoCs with IOCP disabled CMO needs to be handled by
software. Firstly OpenSBI configures the memory region as
"Memory, Non-cacheable, Bufferable" and passes this region as a global
shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
allocations happen from this region and synchronization callbacks are
implemented to synchronize when doing DMA transactions.

RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
errata should be applied to handle cache management.

Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
---
 platform/generic/include/andes/andes45.h | 25 ++++++++++++-
 platform/generic/renesas/rzfive/rzfive.c | 47 ++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 2 deletions(-)

diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
index 08b3d18..d5d265f 100644
--- a/platform/generic/include/andes/andes45.h
+++ b/platform/generic/include/andes/andes45.h
@@ -4,7 +4,28 @@
 #define CSR_MARCHID_MICROID 0xfff
 
 /* Memory and Miscellaneous Registers */
-#define CSR_MCACHE_CTL 0x7ca
-#define CSR_MCCTLCOMMAND 0x7cc
+#define CSR_MCACHE_CTL		0x7ca
+#define CSR_MCCTLCOMMAND	0x7cc
+#define CSR_MICM_CFG		0xfc0
+#define CSR_MDCM_CFG		0xfc1
+#define CSR_MMSC_CFG		0xfc2
+
+#define MISA_20_OFFSET			20
+#define MISA_20_MASK			(0x1 << MISA_20_OFFSET)
+
+#define MICM_CFG_ISZ_OFFSET		6
+#define MICM_CFG_ISZ_MASK		(0x7  << MICM_CFG_ISZ_OFFSET)
+
+#define MDCM_CFG_DSZ_OFFSET		6
+#define MDCM_CFG_DSZ_MASK		(0x7  << MDCM_CFG_DSZ_OFFSET)
+
+#define MMSC_CFG_CCTLCSR_OFFSET		16
+#define MMSC_CFG_CCTLCSR_MASK		(0x1 << MMSC_CFG_CCTLCSR_OFFSET)
+#define MMSC_IOCP_OFFSET			47
+#define MMSC_IOCP_MASK			(0x1ULL << MMSC_IOCP_OFFSET)
+
+#define MCACHE_CTL_CCTL_SUEN_OFFSET	8
+#define MCACHE_CTL_CCTL_SUEN_MASK	(0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
+
 
 #endif /* _RISCV_ANDES45_H */
diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
index eee9c51..552c747 100644
--- a/platform/generic/renesas/rzfive/rzfive.c
+++ b/platform/generic/renesas/rzfive/rzfive.c
@@ -4,11 +4,17 @@
  *
  */
 
+#include <andes/andes45.h>
 #include <andes45_pma.h>
 #include <platform_override.h>
 #include <sbi/sbi_domain.h>
+#include <sbi/sbi_error.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 
+#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND	0
+
+#define ANDESTECH_SBI_EXT_ANDES			0x900031e
+
 static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
 	{
 		.pa = 0x58000000,
@@ -28,6 +34,46 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
 					 array_size(renesas_rzfive_pma_regions));
 }
 
+static bool renesas_rzfive_cpu_cache_controlable(void)
+{
+	return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
+		 (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
+		(csr_read(CSR_MISA) & MISA_20_MASK) &&
+		(csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
+		(csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
+}
+
+static bool renesas_rzfive_cpu_iocp_disabled(void)
+{
+	return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
+}
+
+static bool renesas_rzfive_apply_iocp_sw_workaround(void)
+{
+	return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
+}
+
+static int renesas_rzfive_vendor_ext_provider(long extid, long funcid,
+					      const struct sbi_trap_regs *regs,
+					      unsigned long *out_value,
+					      struct sbi_trap_info *out_trap,
+					      const struct fdt_match *match)
+{
+	if (extid != ANDESTECH_SBI_EXT_ANDES)
+		return SBI_EINVAL;
+
+	switch (funcid) {
+	case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
+		*out_value = renesas_rzfive_apply_iocp_sw_workaround();
+		break;
+
+	default:
+		break;
+	}
+
+	return 0;
+}
+
 int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
 {
 	/*
@@ -55,4 +101,5 @@ const struct platform_override renesas_rzfive = {
 	.match_table = renesas_rzfive_match,
 	.early_init = renesas_rzfive_early_init,
 	.final_init = renesas_rzfive_final_init,
+	.vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
 };
-- 
2.17.1



From peterlin at andestech.com  Tue Feb  7 07:08:13 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 7 Feb 2023 15:08:13 +0000
Subject: [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support
 to configure the PMA
In-Reply-To: <20230206000624.6544-2-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
 <20230206000624.6544-2-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <Y+JpXYjLUlWp3rPa@APC323>

On Mon, Feb 06, 2023 at 12:06:22AM +0000, Lad Prabhakar wrote:
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
> 
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
> 
> The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
> block that allows dynamic adjustment of memory attributes in the runtime.
> It contains a configurable amount of PMA entries implemented as CSR
> registers to control the attributes of memory locations in interest.
> Below are the memory attributes supported:
> * Device, Non-bufferable
> * Device, bufferable
> * Memory, Non-cacheable, Non-bufferable
> * Memory, Non-cacheable, Bufferable
> * Memory, Write-back, No-allocate
> * Memory, Write-back, Read-allocate
> * Memory, Write-back, Write-allocate
> * Memory, Write-back, Read and Write-allocate
> 
> More info about PMA (section 10.3):
> Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
> 
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
> 
> Example PMA region passed as a DT node from OpenSBI:
>     reserved-memory {
>         #address-cells = <2>;
>         #size-cells = <2>;
>         ranges;
> 
>         pma_resv0 at 58000000 {
>             compatible = "shared-dma-pool";
>             reg = <0x0 0x58000000 0x0 0x08000000>;
>             no-map;
>             linux,dma-default;
>         };
>     };
> 
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>

Regards,
Peter Lin

> ---
>  platform/generic/Kconfig                      |   8 +
>  platform/generic/include/andes45_pma.h        |  48 +++
>  platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
>  platform/generic/renesas/rzfive/objects.mk    |   1 +
>  4 files changed, 407 insertions(+)
>  create mode 100644 platform/generic/include/andes45_pma.h
>  create mode 100644 platform/generic/renesas/rzfive/andes45-pma.c
> 
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index c7f198a..9ee9fe3 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -51,3 +51,11 @@ config PLATFORM_STARFIVE_JH7110
>  	default n
>  
>  endif
> +
> +if PLATFORM_RENESAS_RZFIVE
> +
> +config ANDES45_PMA
> +	bool "Andes PMA support"
> +	default n
> +
> +endif
> diff --git a/platform/generic/include/andes45_pma.h b/platform/generic/include/andes45_pma.h
> new file mode 100644
> index 0000000..37ec77c
> --- /dev/null
> +++ b/platform/generic/include/andes45_pma.h
> @@ -0,0 +1,48 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + */
> +
> +#ifndef _ANDES45_PMA_H_
> +#define _ANDES45_PMA_H_
> +
> +#include <sbi/sbi_types.h>
> +
> +#define ANDES45_MAX_PMA_REGIONS			16
> +
> +/* Naturally aligned power of 2 region */
> +#define ANDES45_PMACFG_ETYP_NAPOT		3
> +
> +/* Memory, Non-cacheable, Bufferable */
> +#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF	(3 << 2)
> +
> +/**
> + * struct andes45_pma_region - Describes PMA regions
> + *
> + * @pa: Address to be configured in the PMA
> + * @size: Size of the region
> + * @flags: Flags to be set for the PMA region
> + * @dt_populate: Boolean flag indicating if the DT entry should be
> + *               populated for the given PMA region
> + * @shared_dma: Boolean flag if set "shared-dma-pool" property will
> + *              be set in the DT node
> + * @no_map: Boolean flag if set "no-map" property will be set in the
> + *          DT node
> + * @dma_default: Boolean flag if set "linux,dma-default" property will
> + *              be set in the DT node. Note Linux expects single node
> + *              with this property set.
> + */
> +struct andes45_pma_region {
> +	unsigned long pa;
> +	unsigned long size;
> +	u8 flags:7;
> +	bool dt_populate;
> +	bool shared_dma;
> +	bool no_map;
> +	bool dma_default;
> +};
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> +			      unsigned int pma_regions_count);
> +
> +#endif /* _ANDES45_PMA_H_ */
> diff --git a/platform/generic/renesas/rzfive/andes45-pma.c b/platform/generic/renesas/rzfive/andes45-pma.c
> new file mode 100644
> index 0000000..881a628
> --- /dev/null
> +++ b/platform/generic/renesas/rzfive/andes45-pma.c
> @@ -0,0 +1,350 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + *
> + * Copyright (c) 2020 Andes Technology Corporation
> + *
> + * Authors:
> + *      Nick Hu <nickhu at andestech.com>
> + *      Nylon Chen <nylon7 at andestech.com>
> + *      Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> + */
> +
> +#include <andes45_pma.h>
> +#include <libfdt.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +
> +/* Configuration Registers */
> +#define ANDES45_CSR_MMSC_CFG		0xFC2
> +#define ANDES45_CSR_MMSC_PPMA_OFFSET	(1 << 30)
> +
> +#define ANDES45_PMAADDR_0		0xBD0
> +
> +#define ANDES45_PMACFG_0		0xBC0
> +
> +static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
> +{
> +#define switchcase_pma_cfg_read(__pma_cfg_off, __val)		\
> +	case __pma_cfg_off:					\
> +		__val = csr_read(__pma_cfg_off);		\
> +		break;
> +#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val)		\
> +	switchcase_pma_cfg_read(__pma_cfg_off + 0, __val)	\
> +	switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
> +
> +	unsigned long ret = 0;
> +
> +	switch (pma_cfg_off) {
> +	switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
> +
> +	default:
> +		sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> +		break;
> +	}
> +
> +	return ret;
> +
> +#undef switchcase_pma_cfg_read_2
> +#undef switchcase_pma_cfg_read
> +}
> +
> +static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
> +{
> +#define switchcase_pma_cfg_write(__pma_cfg_off, __val)		\
> +	case __pma_cfg_off:					\
> +		csr_write(__pma_cfg_off, __val);		\
> +		break;
> +#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val)	\
> +	switchcase_pma_cfg_write(__pma_cfg_off + 0, __val)	\
> +	switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
> +
> +	switch (pma_cfg_off) {
> +	switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
> +
> +	default:
> +		sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> +		break;
> +	}
> +
> +#undef switchcase_pma_cfg_write_2
> +#undef switchcase_pma_cfg_write
> +}
> +
> +static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
> +{
> +#define switchcase_pma_write(__pma_addr_off, __val)		\
> +	case __pma_addr_off:					\
> +		csr_write(__pma_addr_off, __val);		\
> +		break;
> +#define switchcase_pma_write_2(__pma_addr_off, __val)		\
> +	switchcase_pma_write(__pma_addr_off + 0, __val)		\
> +	switchcase_pma_write(__pma_addr_off + 1, __val)
> +#define switchcase_pma_write_4(__pma_addr_off, __val)		\
> +	switchcase_pma_write_2(__pma_addr_off + 0, __val)	\
> +	switchcase_pma_write_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_write_8(__pma_addr_off, __val)		\
> +	switchcase_pma_write_4(__pma_addr_off + 0, __val)	\
> +	switchcase_pma_write_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_write_16(__pma_addr_off, __val)		\
> +	switchcase_pma_write_8(__pma_addr_off + 0, __val)	\
> +	switchcase_pma_write_8(__pma_addr_off + 8, __val)
> +
> +	switch (pma_addr_off) {
> +	switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
> +
> +	default:
> +		sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> +		break;
> +	}
> +
> +#undef switchcase_pma_write_16
> +#undef switchcase_pma_write_8
> +#undef switchcase_pma_write_4
> +#undef switchcase_pma_write_2
> +#undef switchcase_pma_write
> +}
> +
> +static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
> +{
> +#define switchcase_pma_read(__pma_addr_off, __val)		\
> +	case __pma_addr_off:					\
> +		__val = csr_read(__pma_addr_off);		\
> +		break;
> +#define switchcase_pma_read_2(__pma_addr_off, __val)		\
> +	switchcase_pma_read(__pma_addr_off + 0, __val)		\
> +	switchcase_pma_read(__pma_addr_off + 1, __val)
> +#define switchcase_pma_read_4(__pma_addr_off, __val)		\
> +	switchcase_pma_read_2(__pma_addr_off + 0, __val)	\
> +	switchcase_pma_read_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_read_8(__pma_addr_off, __val)		\
> +	switchcase_pma_read_4(__pma_addr_off + 0, __val)	\
> +	switchcase_pma_read_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_read_16(__pma_addr_off, __val)		\
> +	switchcase_pma_read_8(__pma_addr_off + 0, __val)	\
> +	switchcase_pma_read_8(__pma_addr_off + 8, __val)
> +
> +	unsigned long ret = 0;
> +
> +	switch (pma_addr_off) {
> +	switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
> +
> +	default:
> +		sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> +		break;
> +	}
> +
> +	return ret;
> +
> +#undef switchcase_pma_read_16
> +#undef switchcase_pma_read_8
> +#undef switchcase_pma_read_4
> +#undef switchcase_pma_read_2
> +#undef switchcase_pma_read
> +}
> +
> +static unsigned long
> +andes45_pma_setup(const struct andes45_pma_region *pma_region,
> +		  unsigned int entry_id)
> +{
> +	unsigned long size = pma_region->size;
> +	unsigned long addr = pma_region->pa;
> +	unsigned int pma_cfg_addr;
> +	unsigned long pmacfg_val;
> +	unsigned long pmaaddr;
> +	char *pmaxcfg;
> +
> +	/* Check for 4KiB granularity */
> +	if (size < (1 << 12))
> +		return SBI_EINVAL;
> +
> +	/* Check size is power of 2 */
> +	if (size & (size - 1))
> +		return SBI_EINVAL;
> +
> +	if (entry_id > 15)
> +		return SBI_EINVAL;
> +
> +	if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
> +		return SBI_EINVAL;
> +
> +	if ((addr & (size - 1)) != 0)
> +		return SBI_EINVAL;
> +
> +	pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
> +	pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
> +	pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
> +	*pmaxcfg = 0;
> +	*pmaxcfg = pma_region->flags;
> +
> +	andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
> +
> +	pmaaddr = (addr >> 2) + (size >> 3) - 1;
> +
> +	andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
> +
> +	return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
> +			pmaaddr : SBI_EINVAL;
> +}
> +
> +static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
> +				unsigned int index, int parent)
> +{
> +	int na = fdt_address_cells(fdt, 0);
> +	int ns = fdt_size_cells(fdt, 0);
> +	static bool dma_default = false;
> +	fdt32_t addr_high, addr_low;
> +	fdt32_t size_high, size_low;
> +	int subnode, err;
> +	fdt32_t reg[4];
> +	fdt32_t *val;
> +	char name[32];
> +
> +	addr_high = (u64)pma->pa >> 32;
> +	addr_low = pma->pa;
> +	size_high = (u64)pma->size >> 32;
> +	size_low = pma->size;
> +
> +	if (na > 1 && addr_high)
> +		sbi_snprintf(name, sizeof(name),
> +			     "pma_resv%d@%x,%x", index,
> +			     addr_high, addr_low);
> +	else
> +		sbi_snprintf(name, sizeof(name),
> +			     "pma_resv%d@%x", index,
> +			     addr_low);
> +
> +	subnode = fdt_add_subnode(fdt, parent, name);
> +	if (subnode < 0)
> +		return subnode;
> +
> +	if (pma->shared_dma) {
> +		err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
> +		if (err < 0)
> +			return err;
> +	}
> +
> +	if (pma->no_map) {
> +		err = fdt_setprop_empty(fdt, subnode, "no-map");
> +		if (err < 0)
> +			return err;
> +	}
> +
> +	/* Linux allows single linux,dma-default region. */
> +	if (pma->dma_default) {
> +		if (dma_default)
> +			return SBI_EINVAL;
> +
> +		err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
> +		if (err < 0)
> +			return err;
> +		dma_default = true;
> +	}
> +
> +	/* encode the <reg> property value */
> +	val = reg;
> +	if (na > 1)
> +		*val++ = cpu_to_fdt32(addr_high);
> +	*val++ = cpu_to_fdt32(addr_low);
> +	if (ns > 1)
> +		*val++ = cpu_to_fdt32(size_high);
> +	*val++ = cpu_to_fdt32(size_low);
> +
> +	err = fdt_setprop(fdt, subnode, "reg", reg,
> +			  (na + ns) * sizeof(fdt32_t));
> +	if (err < 0)
> +		return err;
> +
> +	return 0;
> +}
> +
> +static int andes45_fdt_reserved_memory_fixup(void *fdt,
> +					     const struct andes45_pma_region *pma,
> +					     unsigned int entry)
> +{
> +	int parent;
> +
> +	/* try to locate the reserved memory node */
> +	parent = fdt_path_offset(fdt, "/reserved-memory");
> +	if (parent < 0) {
> +		int na = fdt_address_cells(fdt, 0);
> +		int ns = fdt_size_cells(fdt, 0);
> +		int err;
> +
> +		/* if such node does not exist, create one */
> +		parent = fdt_add_subnode(fdt, 0, "reserved-memory");
> +		if (parent < 0)
> +			return parent;
> +
> +		err = fdt_setprop_empty(fdt, parent, "ranges");
> +		if (err < 0)
> +			return err;
> +
> +		err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
> +		if (err < 0)
> +			return err;
> +
> +		err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
> +		if (err < 0)
> +			return err;
> +	}
> +
> +	return andes45_fdt_pma_resv(fdt, pma, entry, parent);
> +}
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> +			      unsigned int pma_regions_count)
> +{
> +	unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
> +	unsigned int dt_populate_cnt;
> +	unsigned int i, j;
> +	unsigned long pa;
> +	void *fdt;
> +	int ret;
> +
> +	if (!pma_regions || !pma_regions_count)
> +		return 0;
> +
> +	if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
> +		return SBI_EINVAL;
> +
> +	if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
> +		return SBI_ENOTSUPP;
> +
> +	/* Configure the PMA regions */
> +	for (i = 0; i < pma_regions_count; i++) {
> +		pa = andes45_pma_setup(&pma_regions[i], i);
> +		if (pa == SBI_EINVAL)
> +			return SBI_EINVAL;
> +	}
> +
> +	dt_populate_cnt = 0;
> +	for (i = 0; i < pma_regions_count; i++) {
> +		if (!pma_regions[i].dt_populate)
> +			continue;
> +		dt_populate_cnt++;
> +	}
> +
> +	if (!dt_populate_cnt)
> +		return 0;
> +
> +	fdt = fdt_get_address();
> +
> +	ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
> +	if (ret < 0)
> +		return ret;
> +
> +	for (i = 0, j = 0; i < pma_regions_count; i++) {
> +		if (!pma_regions[i].dt_populate)
> +			continue;
> +
> +		ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	return 0;
> +}
> diff --git a/platform/generic/renesas/rzfive/objects.mk b/platform/generic/renesas/rzfive/objects.mk
> index 2e7e37f..fd9e7c4 100644
> --- a/platform/generic/renesas/rzfive/objects.mk
> +++ b/platform/generic/renesas/rzfive/objects.mk
> @@ -6,3 +6,4 @@
>  
>  carray-platform_override_modules-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas_rzfive
>  platform-objs-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas/rzfive/rzfive.o
> +platform-objs-$(CONFIG_ANDES45_PMA) += renesas/rzfive/andes45-pma.o
> -- 
> 2.17.1
> 


From peterlin at andestech.com  Tue Feb  7 07:21:04 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 7 Feb 2023 15:21:04 +0000
Subject: [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT
 to check for enabling IOCP errata
In-Reply-To: <20230206000624.6544-4-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
 <20230206000624.6544-4-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <Y+JsYFaCFw6mKsym@APC323>

On Mon, Feb 06, 2023 at 12:06:24AM +0000, Lad Prabhakar wrote:
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
> 
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
> 
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
> 
> RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> errata should be applied to handle cache management.
> 
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>

Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>

Regards,
Peter Lin

> ---
>  platform/generic/include/andes/andes45.h | 25 ++++++++++++-
>  platform/generic/renesas/rzfive/rzfive.c | 47 ++++++++++++++++++++++++
>  2 files changed, 70 insertions(+), 2 deletions(-)
> 
> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index 08b3d18..d5d265f 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -4,7 +4,28 @@
>  #define CSR_MARCHID_MICROID 0xfff
>  
>  /* Memory and Miscellaneous Registers */
> -#define CSR_MCACHE_CTL 0x7ca
> -#define CSR_MCCTLCOMMAND 0x7cc
> +#define CSR_MCACHE_CTL		0x7ca
> +#define CSR_MCCTLCOMMAND	0x7cc
> +#define CSR_MICM_CFG		0xfc0
> +#define CSR_MDCM_CFG		0xfc1
> +#define CSR_MMSC_CFG		0xfc2
> +
> +#define MISA_20_OFFSET			20
> +#define MISA_20_MASK			(0x1 << MISA_20_OFFSET)
> +
> +#define MICM_CFG_ISZ_OFFSET		6
> +#define MICM_CFG_ISZ_MASK		(0x7  << MICM_CFG_ISZ_OFFSET)
> +
> +#define MDCM_CFG_DSZ_OFFSET		6
> +#define MDCM_CFG_DSZ_MASK		(0x7  << MDCM_CFG_DSZ_OFFSET)
> +
> +#define MMSC_CFG_CCTLCSR_OFFSET		16
> +#define MMSC_CFG_CCTLCSR_MASK		(0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> +#define MMSC_IOCP_OFFSET			47
> +#define MMSC_IOCP_MASK			(0x1ULL << MMSC_IOCP_OFFSET)
> +
> +#define MCACHE_CTL_CCTL_SUEN_OFFSET	8
> +#define MCACHE_CTL_CCTL_SUEN_MASK	(0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> +
>  
>  #endif /* _RISCV_ANDES45_H */
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index eee9c51..552c747 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -4,11 +4,17 @@
>   *
>   */
>  
> +#include <andes/andes45.h>
>  #include <andes45_pma.h>
>  #include <platform_override.h>
>  #include <sbi/sbi_domain.h>
> +#include <sbi/sbi_error.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>  
> +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND	0
> +
> +#define ANDESTECH_SBI_EXT_ANDES			0x900031e
> +
>  static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
>  	{
>  		.pa = 0x58000000,
> @@ -28,6 +34,46 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
>  					 array_size(renesas_rzfive_pma_regions));
>  }
>  
> +static bool renesas_rzfive_cpu_cache_controlable(void)
> +{
> +	return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
> +		 (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
> +		(csr_read(CSR_MISA) & MISA_20_MASK) &&
> +		(csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
> +		(csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
> +}
> +
> +static bool renesas_rzfive_cpu_iocp_disabled(void)
> +{
> +	return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
> +}
> +
> +static bool renesas_rzfive_apply_iocp_sw_workaround(void)
> +{
> +	return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
> +}
> +
> +static int renesas_rzfive_vendor_ext_provider(long extid, long funcid,
> +					      const struct sbi_trap_regs *regs,
> +					      unsigned long *out_value,
> +					      struct sbi_trap_info *out_trap,
> +					      const struct fdt_match *match)
> +{
> +	if (extid != ANDESTECH_SBI_EXT_ANDES)
> +		return SBI_EINVAL;
> +
> +	switch (funcid) {
> +	case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
> +		*out_value = renesas_rzfive_apply_iocp_sw_workaround();
> +		break;
> +
> +	default:
> +		break;
> +	}
> +
> +	return 0;
> +}
> +
>  int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>  {
>  	/*
> @@ -55,4 +101,5 @@ const struct platform_override renesas_rzfive = {
>  	.match_table = renesas_rzfive_match,
>  	.early_init = renesas_rzfive_early_init,
>  	.final_init = renesas_rzfive_final_init,
> +	.vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
>  };
> -- 
> 2.17.1
> 


From mchitale at ventanamicro.com  Tue Feb  7 08:53:23 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Tue,  7 Feb 2023 22:23:23 +0530
Subject: [PATCH 0/3] SBI PMU firmware counters and events improvement
Message-ID: <20230207165326.3205728-1-mchitale@ventanamicro.com>

This patchset implements the improvements described in the following
proposal from Anup Patel: 

https://lists.riscv.org/g/tech-prs/message/102

Mayuresh Chitale (3):
  lib: sbi_pmu: add callback for counter width
  lib: sbi_pmu: Implement sbi_pmu_counter_fw_read_hi
  lib: sbi_pmu: Reserve space for implementation specific firmware
    events

 include/sbi/sbi_ecall_interface.h | 7 ++++++-
 include/sbi/sbi_pmu.h             | 5 +++++
 lib/sbi/sbi_ecall_pmu.c           | 8 ++++++++
 lib/sbi/sbi_pmu.c                 | 6 ++++++
 4 files changed, 25 insertions(+), 1 deletion(-)

-- 
2.34.1



From mchitale at ventanamicro.com  Tue Feb  7 08:53:24 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Tue,  7 Feb 2023 22:23:24 +0530
Subject: [PATCH 1/3] lib: sbi_pmu: add callback for counter width
In-Reply-To: <20230207165326.3205728-1-mchitale@ventanamicro.com>
References: <20230207165326.3205728-1-mchitale@ventanamicro.com>
Message-ID: <20230207165326.3205728-2-mchitale@ventanamicro.com>

This patch adds a callback to fetch the number of bits implemented for a
custom firmware counter. If the callback fails or is not implemented then
width defaults to 63.

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 include/sbi/sbi_pmu.h | 5 +++++
 lib/sbi/sbi_pmu.c     | 6 ++++++
 2 files changed, 11 insertions(+)

diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
index c365243..b3b75c1 100644
--- a/include/sbi/sbi_pmu.h
+++ b/include/sbi/sbi_pmu.h
@@ -41,6 +41,11 @@ struct sbi_pmu_device {
 	bool (*fw_counter_match_code)(uint32_t counter_index,
 				      uint32_t event_idx_code);
 
+	/**
+	 * Fetch the max width of this counter in number of bits.
+	 */
+	int (*fw_counter_width)(void);
+
 	/**
 	 * Read value of custom firmware counter
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 154dbda..a99c045 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -761,6 +761,7 @@ unsigned long sbi_pmu_num_ctr(void)
 
 int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
 {
+	int width;
 	union sbi_pmu_ctr_info cinfo = {0};
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
@@ -782,6 +783,11 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
 		cinfo.type = SBI_PMU_CTR_TYPE_FW;
 		/* Firmware counters are always 64 bits wide */
 		cinfo.width = 63;
+		if (pmu_dev && pmu_dev->fw_counter_width) {
+			width = pmu_dev->fw_counter_width();
+			if (width)
+				cinfo.width = width - 1;
+		}
 	}
 
 	*ctr_info = cinfo.value;
-- 
2.34.1



From mchitale at ventanamicro.com  Tue Feb  7 08:53:25 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Tue,  7 Feb 2023 22:23:25 +0530
Subject: [PATCH 2/3] lib: sbi_pmu: Implement sbi_pmu_counter_fw_read_hi
In-Reply-To: <20230207165326.3205728-1-mchitale@ventanamicro.com>
References: <20230207165326.3205728-1-mchitale@ventanamicro.com>
Message-ID: <20230207165326.3205728-3-mchitale@ventanamicro.com>

To support 64 bit firmware counters on RV32 systems, we implement
sbi_pmu_counter_fw_read_hi() which returns the upper 32 bits of
the firmware counter value. On RV64 (or higher) systems, this
function will always return zero.

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 include/sbi/sbi_ecall_interface.h | 1 +
 lib/sbi/sbi_ecall_pmu.c           | 8 ++++++++
 2 files changed, 9 insertions(+)

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index a3f2bf4..f5d6667 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -99,6 +99,7 @@
 #define SBI_EXT_PMU_COUNTER_START	0x3
 #define SBI_EXT_PMU_COUNTER_STOP	0x4
 #define SBI_EXT_PMU_COUNTER_FW_READ	0x5
+#define SBI_EXT_PMU_COUNTER_FW_READ_HI	0x6
 
 /** General pmu event codes specified in SBI PMU extension */
 enum sbi_pmu_hw_generic_events_t {
diff --git a/lib/sbi/sbi_ecall_pmu.c b/lib/sbi/sbi_ecall_pmu.c
index 826c8a8..2d1e669 100644
--- a/lib/sbi/sbi_ecall_pmu.c
+++ b/lib/sbi/sbi_ecall_pmu.c
@@ -54,6 +54,14 @@ static int sbi_ecall_pmu_handler(unsigned long extid, unsigned long funcid,
 		ret = sbi_pmu_ctr_fw_read(regs->a0, &temp);
 		*out_val = temp;
 		break;
+	case SBI_EXT_PMU_COUNTER_FW_READ_HI:
+#if __riscv_xlen == 32
+		ret = sbi_pmu_ctr_fw_read(regs->a0, &temp);
+		*out_val = temp >> 32
+#else
+		*out_val = 0;
+#endif
+		break;
 	case SBI_EXT_PMU_COUNTER_START:
 
 #if __riscv_xlen == 32
-- 
2.34.1



From mchitale at ventanamicro.com  Tue Feb  7 08:53:26 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Tue,  7 Feb 2023 22:23:26 +0530
Subject: [PATCH 3/3] lib: sbi_pmu: Reserve space for implementation specific firmware events
In-Reply-To: <20230207165326.3205728-1-mchitale@ventanamicro.com>
References: <20230207165326.3205728-1-mchitale@ventanamicro.com>
Message-ID: <20230207165326.3205728-4-mchitale@ventanamicro.com>

We reserve space for SBI implementation specific custom firmware
events which can be used by M-mode firmwares and HS-mode hypervisors
for their own use. This reserved space is intentionally large to
ensure that SBI implementation has enough space to accommodate
platform specific firmware events as well.

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 include/sbi/sbi_ecall_interface.h | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index f5d6667..32923a8 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -182,7 +182,11 @@ enum sbi_pmu_fw_event_code_id {
 	SBI_PMU_FW_HFENCE_VVMA_RCVD	= 19,
 	SBI_PMU_FW_HFENCE_VVMA_ASID_SENT = 20,
 	SBI_PMU_FW_HFENCE_VVMA_ASID_RCVD = 21,
-	SBI_PMU_FW_MAX,
+	/*
+	 * Event codes from 22 to 127 are reserved for
+	 * SBI implementation specific custom firmware events.
+	 */
+	SBI_PMU_FW_MAX = 128,
 };
 
 /** SBI PMU event idx type */
-- 
2.34.1



From atishp at atishpatra.org  Tue Feb  7 14:37:12 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Tue, 7 Feb 2023 14:37:12 -0800
Subject: [PATCH 1/1] Add RISC-V TEE support
In-Reply-To: <002f01d932f3$36610530$a3230f90$@eswincomputing.com>
References: <002f01d932f3$36610530$a3230f90$@eswincomputing.com>
Message-ID: <CAOnJCUKA4PaRqHcOiMDH=HjrvxnrPCLrr4+5VN3e6Jw==nqbqw@mail.gmail.com>

On Sat, Jan 28, 2023 at 12:33 AM liushiwei <liushiwei at eswincomputing.com> wrote:
>
> Hi, Atish
>         Thank you for your reply.
>         I submitted my ideas to https://github.com/riscv-non-isa/riscv-sbi-doc/pull/106 as requested by other members of the email.
>         You can preview them at https://github.com/liushiwei007/riscv-sbi-doc/blob/master/riscv-sbi.adoc .
>         I'm not sure if I need to send an email to tech-prs at lists.riscv.org or sig-trusted-computing at lists.riscv.org after submitting to riscv-sbi-doc?
>         If so, Is it to send the patch of opensbi I submitted before?
>

The Trusted computing SIG evaluates the overall merit and direction of
any TEE related specifications.
The PRS TG will focus more on the SBI specification related parts.
I would recommend you to present your work to Trusted Computing SIG
first so that everybody agrees on the overall direction
for OP-TEE support in RISC-V.

> Regards,
> Liushiwei
>
>
> -----????-----
> ???: Atish Patra [mailto:atishp at atishpatra.org]
> ????: 2023?1?25? 3:12
> ???: liushiwei <liushiwei at eswincomputing.com>
> ??: Himanshu Chauhan <hchauhan at ventanamicro.com>; opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> ??: Re: ??: [PATCH 1/1] Add RISC-V TEE support
>
> On Wed, Jan 11, 2023 at 11:08 PM liushiwei <liushiwei at eswincomputing.com> wrote:
> >
> > Hi, Himanshu ,  These are my description:
> >
> > In my design, The entire linux space is called REE (Rich Execution
> > Environment), and TEE OS includes its user state called TEE (Trusted execution environment).
> > adding TEE functionality to opensbi requires two configuration items.
> > For example:
> >    add CONFIG_SBI_ECALL_TEE=y in platform/generic/configs/defconfig file.
> >    add CONFIG_TEE_LOAD_ADDR=0x27C000000 platform/generic/objects.mk.
> >    The value of CONFIG_TEE_LOAD_ADDR depends on the actual memory layout,
> >    It's a physical address.
> >
> > When TEE is configured, opensbi adds the following functionality:
> > 1. Boot TEE OS.
> >    If TEE is enabled, tee_os_init() is called before entering
> >    sbi_hart_switch_mode(). tee_os_init saves the current context,
> >    sets a new trap stack address, and runs to CONFIG_TEE_LOAD_ADDR as configured
> >    to complete TEE OS initialization. TEE OS returns via ecall, Go to opensbi
> >    sbi_ecall_tee_handler and use the characteristic value RETURN_ENTRY_DONE
> >    to indicate the return after the TEE OS completes booting. Check whether the
> >    TEE OS boot successfully according to the parameter. If fails,
> >    the system enters wfi and terminates the startup process of opensbi. If successful,
> >    it returns REE to switch into TEE's vector. then trap stack memory is restored,
> >    the context is restored, the tee_os_init call is returned,
> >    and the rest of the process is performed.
> > 2. REE switches to TEE.
> >    When the TEE OS boot successfully, it returns an entry vector for REE into the TEE.
> >    It represents various entry points into TEE OS and is stored in opensbi's global variables.
> >    It is the optee_vectors_t structure, which contains nine entry cases:
> >       1. yield_smc_entry;
> >       2. fast_smc_entry;
> >       3. cpu_on_entry;
> >       4. cpu_off_entry;
> >       5. cpu_resume_entry;
> >       6. cpu_suspend_entry;
> >       7. fiq_entry;
> >       8. system_off_entry;
> >       9. system_reset_entry;
> >    yield_smc_entry means that this function entry TEE will start the thread function
> >    and enter the user state of TEE. It may also switch back to REE with RPC function,
> >    and then return to TEE after REE completes the corresponding function.
> >    For the REE process that sent you this call, it may cause sleep.
> >
> >    fast_smc_entry indicates that this is a quick function that returns after
> >    the TEE OS does something, and that it does not cause the caller to sleep.
> >
> >    yield_smc_entry and fast_smc_entry return opensbi use eigenvalue RETURN_CALL_DONE.
> >    for yield_smc_entry, whether the call returns or the RPC returns is decided by linux.
> >
> >    cpu_on_entry/cpu_off_entry/cpu_resume_entry/cpu_suspend_entry and
> >    fiq_entry/system_off_entry/system_reset_entry they are not implemented currently.
> >
> > 3. TEE switches to REE.
> >    opensbi needs to save the context when REE enters the TEE,
> >    and restore the context when it returns from the TEE.
> >
> > 4. TEE/REE Request a special function.
> >    We have some specific functions, like get hartid from TEE,
> >    it need save and restore the TEE context.
> >
> > Data structure.
> > opensbi adds the sbi_save_context declaration
> >    struct sbi_save_context {
> >       struct sbi_trap_regs regs;
> >       unsigned long sepc;
> >       unsigned long satp;
> >       unsigned long sstatus;
> >       unsigned long sie;
> >       unsigned long stvec;
> >       unsigned long sscratch;
> >       unsigned long scounteren;
> >       unsigned long scause;
> >       unsigned long stval;
> >       unsigned long sip;
> >    };
> >    sbi_save_context include sbi_trap_regs and S mode csr. Used to hold
> > the context of TEE or REE
> >
> >    typedef struct optee_vectors {
> >       optee_vector_isn_t yield_smc_entry;
> >       optee_vector_isn_t fast_smc_entry;
> >       optee_vector_isn_t cpu_on_entry;
> >       optee_vector_isn_t cpu_off_entry;
> >       optee_vector_isn_t cpu_resume_entry;
> >       optee_vector_isn_t cpu_suspend_entry;
> >       optee_vector_isn_t fiq_entry;
> >       optee_vector_isn_t system_off_entry;
> >       optee_vector_isn_t system_reset_entry;
> >    } optee_vectors_t;
> >    optee_vectors represent the various entry points into TEE OS.
> >
> > Variables defined by opensbi
> >    optee_vectors_t *optee_vector_table;
> >    struct sbi_save_context nsec_cpu_context[OPTEED_CORE_COUNT];
> >    struct sbi_save_context sec_cpu_context[OPTEED_CORE_COUNT];
> >    struct sbi_trap_regs cpu_start_context[OPTEED_CORE_COUNT];
> >    typedef ulong tee_tmp_trap_stack[1024];
> >    static tee_tmp_trap_stack tmp_stack[OPTEED_CORE_COUNT];
> >
> >    optee_vector_table value is assigned after the TEE OS boot succeeds.
> >    nsec_cpu_context and sec_cpu_context is to save the context of TEE and REE,
> >    cpu_start_context save the context of opensbi before tee_os_init enter TEE OS,
> >    tee_tmp_stack is the trap stack when TEE OS return tee_os_init.
> >
> > Under the current design, REE does not enable interrupts when entering
> > TEE, and the entire TEE, including opensbi, is the process context of linux.
> > TEE processing must be brief and quick.
> >
>
> Great. Thanks. Your PR should include more details like this with proper FID allocation and description.
> In order to make it a standard RISC-V SBI extension, you need to present this in tech-prs mailing list
> (https://lists.riscv.org/g/tech-prs)
> and get it approved within the group. Here is the policy document[1]
>
> https://docs.google.com/document/d/1bQGHU-wD4uN4mU07oH9adpXbH51NqUbFxcL8qTTIqhA/edit
>
> I think it would be good if you can present your work in trusted computing SIG (https://lists.riscv.org/g/sig-trusted-computing) as well.
>
> >
> > Regards,
> > liushiwei
> > -----????-----
> > ???: Himanshu Chauhan [mailto:hchauhan at ventanamicro.com]
> > ????: 2023?1?11? 23:39
> > ???: liushiwei <liushiwei at eswincomputing.com>
> > ??: opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> > ??: Re: ??: [PATCH 1/1] Add RISC-V TEE support
> >
> > On Wed, Jan 11, 2023 at 08:27:59PM +0800, liushiwei wrote:
> > > Do you mean hardware? Our hardware design referred to arm's
> > > trustzone technology. optee os is a software solution using arm
> > > trustzone hardware, which mainly includes REE(linux), TEE(optee os),
> > > ATF(ARM Trusted firmware), and then our software also developed
> > > these three parts. opensbi is similar to ATF. whether if this is what you want?
> > > The current committed code is not hardware-dependent, but just
> > > continues the idea of this workaround, and we may commit hardware-dependent code later.
> > >
> > No, I meant the software specification.
> >
> > > -----????-----
> > > ???: hchauhan at ventanamicro.com [mailto:hchauhan at ventanamicro.com]
> > > ????: 2023?1?11? 20:03
> > > ???: 'liushiwei' <liushiwei at eswincomputing.com>; opensbi at lists.infradead.
> > > org
> > > ??: chenchaokai at eswincomputing.com
> > > ??: RE: [PATCH 1/1] Add RISC-V TEE support
> > >
> > > -----Original Message-----
> > > > From: opensbi <opensbi-bounces at lists.infradead.org> On Behalf Of
> > > > liushiwei
> > > > Sent: 11 January 2023 07:32
> > > > To: opensbi at lists.infradead.org
> > > > Cc: chenchaokai at eswincomputing.com; liushiwei
> > > <liushiwei at eswincomputing.com>
> > > > Subject: [PATCH 1/1] Add RISC-V TEE support
> > >
> > > >RISC-V Trusted Executable Environment security software includes
> > > >linux,
> > > opensbi, and OP-TEE OS. linux is the non-secure domain, and OP-TEE
> > > OS is the secure domain. At boot time, opensbi boots OP->TEE OS and then starts linux.
> > > At runtime, opensbi acts as a secure monitor, responsible for
> > > context saving and restoring when switching between linux and OP-TEE OS.
> > > >TEE function is off by default, when using configuration is added
> > > >in the
> > > config and objects file, such as platform/generic/configs/defconfig
> > > add CONFIG_SBI_ECALL_TEE = y, In the >platform/generic/objects.mk
> > > add CONFIG_TEE_LOAD_ADDR = 0x27c000000, CONFIG_TEE_LOAD_ADDR is the
> > > starting address of the OP-TEE OS.
> > >
> > > Hi Liushiwei,
> > >
> > > Was there any formal specification or draft for this? Could you
> > > please point me to the draft or specification?
> > >
> > > Regards
> > > Himanshu
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
> > >
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>
>
>
> --
> Regards,
> Atish
>


-- 
Regards,
Atish


From anup at brainfault.org  Tue Feb  7 21:27:02 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 10:57:02 +0530
Subject: [PATCH 1/2] firmware: Not to clear all the MIP
In-Reply-To: <20230117081428.15042-2-nick.hu@sifive.com>
References: <20230117081428.15042-1-nick.hu@sifive.com> <20230117081428.15042-2-nick.hu@sifive.com>
Message-ID: <CAAhSdy061xtCx7zgVYAdj9nVjpoRBoyQ5dj-pPmN43nOfSMkTA@mail.gmail.com>

On Tue, Jan 17, 2023 at 1:44 PM Nick Hu <nick.hu at sifive.com> wrote:
>
> In generic behavior of QEMU, if the pending bits of PLIC are still set and
> we clear the SEIP, the QEMU may not set the SEIP back immediately and the
> interrupt may not be handled anymore until the new interrupts arrived and
> QEMU set the SEIP back which is a generic behavior in QEMU.
>
> Signed-off-by: Nick Hu <nick.hu at sifive.com>
> Signed-off-by: Jim Shu <jim.shu at sifive.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  firmware/fw_base.S | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/firmware/fw_base.S b/firmware/fw_base.S
> index 3f622b3..50f284e 100644
> --- a/firmware/fw_base.S
> +++ b/firmware/fw_base.S
> @@ -422,9 +422,15 @@ _start_warm:
>         li      ra, 0
>         call    _reset_regs
>
> -       /* Disable and clear all interrupts */
> +       /* Disable all interrupts */
>         csrw    CSR_MIE, zero
> -       csrw    CSR_MIP, zero
> +       /*
> +        * Only clear the MIP_SSIP and MIP_STIP. For the platform like QEMU,
> +        * If we clear other interrupts like MIP_SEIP and the pendings of
> +        * PLIC still exist, the QEMU may not set it back immediately.
> +        */
> +       li      t0, (MIP_SSIP | MIP_STIP)
> +       csrc    CSR_MIP, t0
>
>         /* Find HART count and HART stack size */
>         lla     a4, platform
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Feb  7 21:27:24 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 10:57:24 +0530
Subject: [PATCH 2/2] lib: sbi_hsm: Use csr_set to restore the MIP
In-Reply-To: <20230117081428.15042-3-nick.hu@sifive.com>
References: <20230117081428.15042-1-nick.hu@sifive.com> <20230117081428.15042-3-nick.hu@sifive.com>
Message-ID: <CAAhSdy0dYVV2kfJY+Y4+GWXMU=otD8pC=bvHafsiGME1vBfC3Q@mail.gmail.com>

On Tue, Jan 17, 2023 at 1:44 PM Nick Hu <nick.hu at sifive.com> wrote:
>
> If we use the csr_write to restore the MIP, we may clear the SEIP.
> In generic behavior of QEMU, if the pending bits of PLIC are set and we
> clear the SEIP, the QEMU may not set it back immediately. It may cause
> the interrupts won't be handled anymore until the new interrupts arrived
> and QEMU set the bits back.
>
> Signed-off-by: Nick Hu <nick.hu at sifive.com>
> Signed-off-by: Jim Shu <jim.shu at sifive.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/sbi/sbi_hsm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index c0a5505..39cd1ab 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -358,7 +358,7 @@ static void __sbi_hsm_suspend_non_ret_restore(struct sbi_scratch *scratch)
>                                                             hart_data_offset);
>
>         csr_write(CSR_MIE, hdata->saved_mie);
> -       csr_write(CSR_MIP, (hdata->saved_mip & (MIP_SSIP | MIP_STIP)));
> +       csr_set(CSR_MIP, (hdata->saved_mip & (MIP_SSIP | MIP_STIP)));
>  }
>
>  void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch)
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Feb  7 21:39:05 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 11:09:05 +0530
Subject: [PATCH 0/2] Not to clear all the bits of mip
In-Reply-To: <20230117081428.15042-1-nick.hu@sifive.com>
References: <20230117081428.15042-1-nick.hu@sifive.com>
Message-ID: <CAAhSdy3_rq=xdFT_yeaT4G4rrCVF7CKR=gyTDSVzMLrhi0FRrg@mail.gmail.com>

On Tue, Jan 17, 2023 at 1:44 PM Nick Hu <nick.hu at sifive.com> wrote:
>
> In generic behavior of QEMU, if we clear the mip.SEIP and the pendings of
> PLIC still exist, the QEMU may not set the mip.SEIP back immediately. So
> interrupts won't be handled anymore until the next new device interrupt
> arrived. We should only clear the SSIP and STIP which set by SW. The rest
> of the bits should be decided by HW.
>
> Nick Hu (2):
>   firmware: Not to clear all the MIP
>   lib: sbi_hsm: Use csr_set to restore the MIP

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
>  firmware/fw_base.S | 10 ++++++++--
>  lib/sbi/sbi_hsm.c  |  2 +-
>  2 files changed, 9 insertions(+), 3 deletions(-)
>
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Feb  7 21:47:05 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 11:17:05 +0530
Subject: [PATCH] lib: utils: Mark only the largest region as reserved in FDT
In-Reply-To: <20230127041923.112164-1-hchauhan@ventanamicro.com>
References: <20230127041923.112164-1-hchauhan@ventanamicro.com>
Message-ID: <CAAhSdy3ddbpDc7Wix2EqdO-geTiiLTHRnLvvGb2BNr8dcWw1Aw@mail.gmail.com>

On Fri, Jan 27, 2023 at 9:49 AM Himanshu Chauhan
<hchauhan at ventanamicro.com> wrote:
>
> In commit 230278dcf, RX and RW regions were marked separately.
> When the RW region grows (e.g. with more harts) and it isn't a
> power-of-two, sbi_domain_memregion_init will upgrade the region
> to the next power-of-two. This will make RX and RW both start
> at the same base address, like so (with 64 harts):
> Domain0 Region01 : 0x0000000080000000-0x000000008001ffff M: (R,X) S/U: ()
> Domain0 Region02 : 0x0000000080000000-0x00000000800fffff M: (R,W) S/U: ()
>
> This doesn't break the permission enforcement because of static
> priorities in PMP but makes the kernel complain about the regions
> overlapping each other. Like so:
> [    0.000000] OF: reserved mem: OVERLAP DETECTED!
> [    0.000000] mmode_resv0 at 80000000 (0x0000000080000000--0x0000000080020000) \
>         overlaps with mmode_resv1 at 80000000 (0x0000000080000000--0x0000000080100000)
>
> To fix this warning, among the multiple regions having same base
> address but different sizes, add only the largest region as reserved
> region during fdt fixup.
>
> Fixes: 230278dcf (lib: sbi: Add separate entries for firmware RX and RW regions)
>
> Signed-off-by: Himanshu Chauhan <hchauhan at ventanamicro.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/utils/fdt/fdt_fixup.c | 34 ++++++++++++++++++++++++++++++----
>  1 file changed, 30 insertions(+), 4 deletions(-)
>
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 42692cc..4085762 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -1,3 +1,4 @@
> +
>  // SPDX-License-Identifier: BSD-2-Clause
>  /*
>   * fdt_fixup.c - Flat Device Tree parsing helper routines
> @@ -14,6 +15,7 @@
>  #include <sbi/sbi_hart.h>
>  #include <sbi/sbi_scratch.h>
>  #include <sbi/sbi_string.h>
> +#include <sbi/sbi_error.h>
>  #include <sbi_utils/fdt/fdt_fixup.h>
>  #include <sbi_utils/fdt/fdt_pmu.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
> @@ -200,8 +202,10 @@ int fdt_reserved_memory_fixup(void *fdt)
>         struct sbi_domain_memregion *reg;
>         struct sbi_domain *dom = sbi_domain_thishart_ptr();
>         struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> +       unsigned long filtered_base[PMP_COUNT] = { 0 };
> +       unsigned char filtered_order[PMP_COUNT] = { 0 };
>         unsigned long addr, size;
> -       int err, parent, i;
> +       int err, parent, i, j;
>         int na = fdt_address_cells(fdt, 0);
>         int ns = fdt_size_cells(fdt, 0);
>
> @@ -266,11 +270,33 @@ int fdt_reserved_memory_fixup(void *fdt)
>                 if (reg->flags & SBI_DOMAIN_MEMREGION_SU_EXECUTABLE)
>                         continue;
>
> +               if (i > PMP_COUNT) {
> +                       sbi_printf("%s: Too many memory regions to fixup.\n",
> +                                  __func__);
> +                       return SBI_ENOSPC;
> +               }
> +
>                 addr = reg->base;
> -               size = 1UL << reg->order;
> -               fdt_resv_memory_update_node(fdt, addr, size, i, parent,
> -                       (sbi_hart_pmp_count(scratch)) ? false : true);
> +               for (j = 0; j < i; j++) {
> +                       if (addr == filtered_base[j]
> +                           && filtered_order[j] < reg->order) {
> +                               filtered_order[j] = reg->order;
> +                               goto next_entry;
> +                       }
> +               }
> +
> +               filtered_base[i] = reg->base;
> +               filtered_order[i] = reg->order;
>                 i++;
> +       next_entry:
> +       }
> +
> +       for (j = 0; j < i; j++) {
> +               addr = filtered_base[j];
> +               size = 1UL << filtered_order[j];
> +               fdt_resv_memory_update_node(fdt, addr, size, j, parent,
> +                                           (sbi_hart_pmp_count(scratch))
> +                                           ? false : true);
>         }
>
>         return 0;
> --
> 2.39.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Feb  8 03:05:37 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 16:35:37 +0530
Subject: [PATCH] lib: sbi: Ensure domidx_to_domain_table is null-terminated
In-Reply-To: <20230130164224.1364402-1-ajones@ventanamicro.com>
References: <20230130164224.1364402-1-ajones@ventanamicro.com>
Message-ID: <CAAhSdy0=BTLr2vO+D6EzpUhfctRfTrPsQr-nn5ToJhFb57SROQ@mail.gmail.com>

On Mon, Jan 30, 2023 at 10:12 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> sbi_domain_for_each() requires domidx_to_domain_table[] to be
> null-terminated. Allocate one extra element which will always
> be null.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_domain.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index 3ab1fbe0b7de..3813f1b2e97d 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -17,8 +17,12 @@
>  #include <sbi/sbi_scratch.h>
>  #include <sbi/sbi_string.h>
>
> +/*
> + * We allocate an extra element because sbi_domain_for_each() expects
> + * the array to be null-terminated.
> + */
> +struct sbi_domain *domidx_to_domain_table[SBI_DOMAIN_MAX_INDEX + 1] = { 0 };
>  struct sbi_domain *hartid_to_domain_table[SBI_HARTMASK_MAX_BITS] = { 0 };
> -struct sbi_domain *domidx_to_domain_table[SBI_DOMAIN_MAX_INDEX] = { 0 };
>  static u32 domain_count = 0;
>  static bool domain_finalized = false;
>
> --
> 2.39.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Feb  8 03:26:16 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 16:56:16 +0530
Subject: [PATCH] Makefile: Add missing .dep files for fw_*.elf.ld
In-Reply-To: <20230201164916.19908-1-jrtc27@jrtc27.com>
References: <20230201164916.19908-1-jrtc27@jrtc27.com>
Message-ID: <CAAhSdy199zb=8wAhQGigF6xNbpmezv7jaY5CGf8mHV7Okso9Ng@mail.gmail.com>

On Wed, Feb 1, 2023 at 10:19 PM Jessica Clarke <jrtc27 at jrtc27.com> wrote:
>
> Since we don't currently create these, changes to fw_base.ldS do not
> cause the preprocessed fw_*.elf.ld files to be rebuilt, and thus
> incremental builds can end up failing with missing symbols if crossing
> the recent commits that introduced _fw_rw_offset and then replaced it
> with _fw_rw_start.
>
> Reported-by: Ben Dooks <ben.dooks at sifive.com>
> Signed-off-by: Jessica Clarke <jrtc27 at jrtc27.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  Makefile | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index b742d6d..b20404f 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -254,6 +254,7 @@ deps-y=$(platform-objs-path-y:.o=.dep)
>  deps-y+=$(libsbi-objs-path-y:.o=.dep)
>  deps-y+=$(libsbiutils-objs-path-y:.o=.dep)
>  deps-y+=$(firmware-objs-path-y:.o=.dep)
> +deps-y+=$(firmware-elfs-path-y:=.dep)
>
>  # Setup platform ABI, ISA and Code Model
>  ifndef PLATFORM_RISCV_ABI
> @@ -413,6 +414,11 @@ inst_file_list = $(CMD_PREFIX)if [ ! -z "$(4)" ]; then \
>  inst_header_dir =  $(CMD_PREFIX)mkdir -p $(1); \
>              echo " INSTALL   $(subst $(install_root_dir)/,,$(1))"; \
>              cp -rf $(2) $(1)
> +compile_cpp_dep = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \
> +            echo " CPP-DEP   $(subst $(build_dir)/,,$(1))"; \
> +            printf %s `dirname $(1)`/  > $(1) && \
> +            $(CC) $(CPPFLAGS) -x c -MM $(3) \
> +              -MT `basename $(1:.dep=$(2))` >> $(1) || rm -f $(1)
>  compile_cpp = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \
>              echo " CPP       $(subst $(build_dir)/,,$(1))"; \
>              $(CPP) $(CPPFLAGS) -x c $(2) | grep -v "\#" > $(1)
> @@ -543,6 +549,9 @@ $(platform_build_dir)/%.bin: $(platform_build_dir)/%.elf
>  $(platform_build_dir)/%.elf: $(platform_build_dir)/%.o $(platform_build_dir)/%.elf.ld $(platform_build_dir)/lib/libplatsbi.a
>         $(call compile_elf,$@,$@.ld,$< $(platform_build_dir)/lib/libplatsbi.a)
>
> +$(platform_build_dir)/%.dep: $(src_dir)/%.ldS $(KCONFIG_CONFIG)
> +       $(call compile_cpp_dep,$@,.ld,$<)
> +
>  $(platform_build_dir)/%.ld: $(src_dir)/%.ldS
>         $(call compile_cpp,$@,$<)
>
> --
> 2.37.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Feb  8 04:46:58 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 18:16:58 +0530
Subject: [PATCH v2 1/3] include: Add support for byteorder/endianness conversion
In-Reply-To: <20230202044427.1253816-2-rpathak@ventanamicro.com>
References: <20230202044427.1253816-1-rpathak@ventanamicro.com> <20230202044427.1253816-2-rpathak@ventanamicro.com>
Message-ID: <CAAhSdy1AM069BcP6y94yON5iLcYQPZHKKpp0b1O+mWosbq8cQQ@mail.gmail.com>

On Thu, Feb 2, 2023 at 10:15 AM Rahul Pathak <rpathak at ventanamicro.com> wrote:
>
> Define macros general byteorder conversion
> Define functions for endianness conversion
> from general byteorder conversion macros
>
> Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Sergey Matyukevich <sergey.matyukevich at syntacore.com>

For now, this looks good. We can move to built-in compiler macros
as a separate patch.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  include/sbi/sbi_byteorder.h | 57 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 57 insertions(+)
>  create mode 100644 include/sbi/sbi_byteorder.h
>
> diff --git a/include/sbi/sbi_byteorder.h b/include/sbi/sbi_byteorder.h
> new file mode 100644
> index 000000000000..680710fe6d91
> --- /dev/null
> +++ b/include/sbi/sbi_byteorder.h
> @@ -0,0 +1,57 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2023 Ventana Micro Systems Inc.
> + */
> +
> +#ifndef __SBI_BYTEORDER_H__
> +#define __SBI_BYTEORDER_H__
> +
> +#include <sbi/sbi_types.h>
> +
> +#define EXTRACT_BYTE(x, n)     ((unsigned long long)((uint8_t *)&x)[n])
> +
> +#define BSWAP16(x)     ((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
> +#define BSWAP32(x)     ((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
> +                        (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
> +#define BSWAP64(x)     ((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
> +                        (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
> +                        (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
> +                        (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
> +
> +
> +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__  /* CPU(little-endian) */
> +#define cpu_to_be16(x)         ((uint16_t)BSWAP16(x))
> +#define cpu_to_be32(x)         ((uint32_t)BSWAP32(x))
> +#define cpu_to_be64(x)         ((uint64_t)BSWAP64(x))
> +
> +#define be16_to_cpu(x)         ((uint16_t)BSWAP16(x))
> +#define be32_to_cpu(x)         ((uint32_t)BSWAP32(x))
> +#define be64_to_cpu(x)         ((uint64_t)BSWAP64(x))
> +
> +#define cpu_to_le16(x)         ((uint16_t)(x))
> +#define cpu_to_le32(x)         ((uint32_t)(x))
> +#define cpu_to_le64(x)         ((uint64_t)(x))
> +
> +#define le16_to_cpu(x)         ((uint16_t)(x))
> +#define le32_to_cpu(x)         ((uint32_t)(x))
> +#define le64_to_cpu(x)         ((uint64_t)(x))
> +#else /* CPU(big-endian) */
> +#define cpu_to_be16(x)         ((uint16_t)(x))
> +#define cpu_to_be32(x)         ((uint32_t)(x))
> +#define cpu_to_be64(x)         ((uint64_t)(x))
> +
> +#define be16_to_cpu(x)         ((uint16_t)(x))
> +#define be32_to_cpu(x)         ((uint32_t)(x))
> +#define be64_to_cpu(x)         ((uint64_t)(x))
> +
> +#define cpu_to_le16(x)         ((uint16_t)BSWAP16(x))
> +#define cpu_to_le32(x)         ((uint32_t)BSWAP32(x))
> +#define cpu_to_le64(x)         ((uint64_t)BSWAP64(x))
> +
> +#define le16_to_cpu(x)         ((uint16_t)BSWAP16(x))
> +#define le32_to_cpu(x)         ((uint32_t)BSWAP32(x))
> +#define le64_to_cpu(x)         ((uint64_t)BSWAP64(x))
> +#endif
> +
> +#endif /* __SBI_BYTEORDER_H__ */
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Feb  8 04:53:31 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 18:23:31 +0530
Subject: [PATCH v2 2/3] lib: utils/fdt: Use byteorder conversion functions
 in libfdt_env.h
In-Reply-To: <20230202044427.1253816-3-rpathak@ventanamicro.com>
References: <20230202044427.1253816-1-rpathak@ventanamicro.com> <20230202044427.1253816-3-rpathak@ventanamicro.com>
Message-ID: <CAAhSdy3jPCOjWMSb6mctWn37HCCiTR28=Qgvw723yw7LDTeiFg@mail.gmail.com>

On Thu, Feb 2, 2023 at 10:15 AM Rahul Pathak <rpathak at ventanamicro.com> wrote:
>
> FDT follows big-endian and CPU can be little or big
> endian as per the implementation.
> libfdt_env.h defines function for conversion between
> fdt and cpu byteorder according to the endianness.
>
> Currently, libfdt_env.h defines custom byte swapping
> macros and then undefines them. Instead, use the generic
> endianness conversion functions
>
> Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/utils/libfdt/libfdt_env.h | 29 ++++++++++-------------------
>  1 file changed, 10 insertions(+), 19 deletions(-)
>
> diff --git a/lib/utils/libfdt/libfdt_env.h b/lib/utils/libfdt/libfdt_env.h
> index f9d9c6783c52..e5ad7698efca 100644
> --- a/lib/utils/libfdt/libfdt_env.h
> +++ b/lib/utils/libfdt/libfdt_env.h
> @@ -9,6 +9,7 @@
>
>  #include <sbi/sbi_string.h>
>  #include <sbi/sbi_types.h>
> +#include <sbi/sbi_byteorder.h>
>
>  #define INT_MAX                ((int)(~0U >> 1))
>  #define UINT_MAX       ((unsigned int)~0U)
> @@ -41,45 +42,35 @@ typedef uint16_t FDT_BITWISE fdt16_t;
>  typedef uint32_t FDT_BITWISE fdt32_t;
>  typedef uint64_t FDT_BITWISE fdt64_t;
>
> -#define EXTRACT_BYTE(x, n)     ((unsigned long long)((uint8_t *)&x)[n])
> -#define CPU_TO_FDT16(x) ((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
> -#define CPU_TO_FDT32(x) ((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
> -                        (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
> -#define CPU_TO_FDT64(x) ((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
> -                        (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
> -                        (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
> -                        (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
> -
>  static inline uint16_t fdt16_to_cpu(fdt16_t x)
>  {
> -       return (FDT_FORCE uint16_t)CPU_TO_FDT16(x);
> +       return (FDT_FORCE uint16_t)be16_to_cpu(x);
>  }
> +
>  static inline fdt16_t cpu_to_fdt16(uint16_t x)
>  {
> -       return (FDT_FORCE fdt16_t)CPU_TO_FDT16(x);
> +       return (FDT_FORCE fdt16_t)cpu_to_be16(x);
>  }
>
>  static inline uint32_t fdt32_to_cpu(fdt32_t x)
>  {
> -       return (FDT_FORCE uint32_t)CPU_TO_FDT32(x);
> +       return (FDT_FORCE uint32_t)be32_to_cpu(x);
>  }
> +
>  static inline fdt32_t cpu_to_fdt32(uint32_t x)
>  {
> -       return (FDT_FORCE fdt32_t)CPU_TO_FDT32(x);
> +       return (FDT_FORCE fdt32_t)cpu_to_be32(x);
>  }
>
>  static inline uint64_t fdt64_to_cpu(fdt64_t x)
>  {
> -       return (FDT_FORCE uint64_t)CPU_TO_FDT64(x);
> +       return (FDT_FORCE uint64_t)be64_to_cpu(x);
>  }
> +
>  static inline fdt64_t cpu_to_fdt64(uint64_t x)
>  {
> -       return (FDT_FORCE fdt64_t)CPU_TO_FDT64(x);
> +       return (FDT_FORCE fdt64_t)cpu_to_be64(x);
>  }
> -#undef CPU_TO_FDT64
> -#undef CPU_TO_FDT32
> -#undef CPU_TO_FDT16
> -#undef EXTRACT_BYTE
>
>  #ifdef __APPLE__
>  #include <AvailabilityMacros.h>
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Feb  8 04:56:31 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 8 Feb 2023 18:26:31 +0530
Subject: [PATCH v2 3/3] include: types: Add typedefs for endianness
In-Reply-To: <20230202044427.1253816-4-rpathak@ventanamicro.com>
References: <20230202044427.1253816-1-rpathak@ventanamicro.com> <20230202044427.1253816-4-rpathak@ventanamicro.com>
Message-ID: <CAAhSdy01EcPCQAYtF438J6ad4+rh9MeecGG+w6fyDHvCV6jm=g@mail.gmail.com>

On Thu, Feb 2, 2023 at 10:15 AM Rahul Pathak <rpathak at ventanamicro.com> wrote:
>
> If any variable/memory-location follows certain
> endianness then its important to annotate it properly
> so that proper conversion can be done before read/write
> from that variable/memory.
>
> Also, use these new typedefs in libfdt_env.h for deriving
> its own custom fdtX_t types
>
> Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  include/sbi/sbi_types.h       | 7 +++++++
>  lib/utils/libfdt/libfdt_env.h | 6 +++---
>  2 files changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/include/sbi/sbi_types.h b/include/sbi/sbi_types.h
> index 9c1fef3b8dc6..def88bbad2c3 100644
> --- a/include/sbi/sbi_types.h
> +++ b/include/sbi/sbi_types.h
> @@ -54,6 +54,13 @@ typedef unsigned long                virtual_size_t;
>  typedef unsigned long          physical_addr_t;
>  typedef unsigned long          physical_size_t;
>
> +typedef uint16_t               le16_t;
> +typedef uint16_t               be16_t;
> +typedef uint32_t               le32_t;
> +typedef uint32_t               be32_t;
> +typedef uint64_t               le64_t;
> +typedef uint64_t               be64_t;
> +
>  #define true                   1
>  #define false                  0
>
> diff --git a/lib/utils/libfdt/libfdt_env.h b/lib/utils/libfdt/libfdt_env.h
> index e5ad7698efca..6ebbae01b514 100644
> --- a/lib/utils/libfdt/libfdt_env.h
> +++ b/lib/utils/libfdt/libfdt_env.h
> @@ -38,9 +38,9 @@
>  #define strlen         sbi_strlen
>  #define strnlen                sbi_strnlen
>
> -typedef uint16_t FDT_BITWISE fdt16_t;
> -typedef uint32_t FDT_BITWISE fdt32_t;
> -typedef uint64_t FDT_BITWISE fdt64_t;
> +typedef be16_t FDT_BITWISE fdt16_t;
> +typedef be32_t FDT_BITWISE fdt32_t;
> +typedef be64_t FDT_BITWISE fdt64_t;
>
>  static inline uint16_t fdt16_to_cpu(fdt16_t x)
>  {
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From samuel at sholland.org  Wed Feb  8 16:28:10 2023
From: samuel at sholland.org (Samuel Holland)
Date: Wed, 8 Feb 2023 18:28:10 -0600
Subject: [PATCH v2 1/3] include: Add support for byteorder/endianness
 conversion
In-Reply-To: <20230202044427.1253816-2-rpathak@ventanamicro.com>
References: <20230202044427.1253816-1-rpathak@ventanamicro.com>
 <20230202044427.1253816-2-rpathak@ventanamicro.com>
Message-ID: <a191f47b-65e3-3f65-cd61-6fd4eff32f4f@sholland.org>

On 2/1/23 22:44, Rahul Pathak wrote:
> Define macros general byteorder conversion
> Define functions for endianness conversion
> from general byteorder conversion macros
> 
> Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Sergey Matyukevich <sergey.matyukevich at syntacore.com>
> ---
>  include/sbi/sbi_byteorder.h | 57 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 57 insertions(+)
>  create mode 100644 include/sbi/sbi_byteorder.h
> 
> diff --git a/include/sbi/sbi_byteorder.h b/include/sbi/sbi_byteorder.h
> new file mode 100644
> index 000000000000..680710fe6d91
> --- /dev/null
> +++ b/include/sbi/sbi_byteorder.h
> @@ -0,0 +1,57 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2023 Ventana Micro Systems Inc.
> + */
> +
> +#ifndef __SBI_BYTEORDER_H__
> +#define __SBI_BYTEORDER_H__
> +
> +#include <sbi/sbi_types.h>
> +
> +#define EXTRACT_BYTE(x, n)	((unsigned long long)((uint8_t *)&x)[n])
> +
> +#define BSWAP16(x)	((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
> +#define BSWAP32(x)	((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
> +			 (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
> +#define BSWAP64(x)	((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
> +			 (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
> +			 (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
> +			 (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))

This implementation is wrong. It only does any swapping on little-endian
systems. That was the desired behavior for CPU_TO_FDT*, but these macros
need to swap bytes regardless of the native endianness.

(Somehow, none of the other three reviewers noticed this?)

Regards,
Samuel

> +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__  /* CPU(little-endian) */
> +#define cpu_to_be16(x)		((uint16_t)BSWAP16(x))
> +#define cpu_to_be32(x)		((uint32_t)BSWAP32(x))
> +#define cpu_to_be64(x)		((uint64_t)BSWAP64(x))
> +
> +#define be16_to_cpu(x)		((uint16_t)BSWAP16(x))
> +#define be32_to_cpu(x)		((uint32_t)BSWAP32(x))
> +#define be64_to_cpu(x)		((uint64_t)BSWAP64(x))
> +
> +#define cpu_to_le16(x)		((uint16_t)(x))
> +#define cpu_to_le32(x)		((uint32_t)(x))
> +#define cpu_to_le64(x)		((uint64_t)(x))
> +
> +#define le16_to_cpu(x)		((uint16_t)(x))
> +#define le32_to_cpu(x)		((uint32_t)(x))
> +#define le64_to_cpu(x)		((uint64_t)(x))
> +#else /* CPU(big-endian) */
> +#define cpu_to_be16(x)		((uint16_t)(x))
> +#define cpu_to_be32(x)		((uint32_t)(x))
> +#define cpu_to_be64(x)		((uint64_t)(x))
> +
> +#define be16_to_cpu(x)		((uint16_t)(x))
> +#define be32_to_cpu(x)		((uint32_t)(x))
> +#define be64_to_cpu(x)		((uint64_t)(x))
> +
> +#define cpu_to_le16(x)		((uint16_t)BSWAP16(x))
> +#define cpu_to_le32(x)		((uint32_t)BSWAP32(x))
> +#define cpu_to_le64(x)		((uint64_t)BSWAP64(x))
> +
> +#define le16_to_cpu(x)		((uint16_t)BSWAP16(x))
> +#define le32_to_cpu(x)		((uint32_t)BSWAP32(x))
> +#define le64_to_cpu(x)		((uint64_t)BSWAP64(x))
> +#endif
> +
> +#endif /* __SBI_BYTEORDER_H__ */



From anup at brainfault.org  Wed Feb  8 19:25:07 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 9 Feb 2023 08:55:07 +0530
Subject: [PATCH v2 1/3] include: Add support for byteorder/endianness conversion
In-Reply-To: <a191f47b-65e3-3f65-cd61-6fd4eff32f4f@sholland.org>
References: <20230202044427.1253816-1-rpathak@ventanamicro.com>
 <20230202044427.1253816-2-rpathak@ventanamicro.com> <a191f47b-65e3-3f65-cd61-6fd4eff32f4f@sholland.org>
Message-ID: <CAAhSdy3zo1FHNbYq2n=sKg3rC=k2LA9QcDX3RhrXnWC-7ROuyw@mail.gmail.com>

On Thu, Feb 9, 2023 at 5:58 AM Samuel Holland <samuel at sholland.org> wrote:
>
> On 2/1/23 22:44, Rahul Pathak wrote:
> > Define macros general byteorder conversion
> > Define functions for endianness conversion
> > from general byteorder conversion macros
> >
> > Signed-off-by: Rahul Pathak <rpathak at ventanamicro.com>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
> > Reviewed-by: Sergey Matyukevich <sergey.matyukevich at syntacore.com>
> > ---
> >  include/sbi/sbi_byteorder.h | 57 +++++++++++++++++++++++++++++++++++++
> >  1 file changed, 57 insertions(+)
> >  create mode 100644 include/sbi/sbi_byteorder.h
> >
> > diff --git a/include/sbi/sbi_byteorder.h b/include/sbi/sbi_byteorder.h
> > new file mode 100644
> > index 000000000000..680710fe6d91
> > --- /dev/null
> > +++ b/include/sbi/sbi_byteorder.h
> > @@ -0,0 +1,57 @@
> > +/*
> > + * SPDX-License-Identifier: BSD-2-Clause
> > + *
> > + * Copyright (c) 2023 Ventana Micro Systems Inc.
> > + */
> > +
> > +#ifndef __SBI_BYTEORDER_H__
> > +#define __SBI_BYTEORDER_H__
> > +
> > +#include <sbi/sbi_types.h>
> > +
> > +#define EXTRACT_BYTE(x, n)   ((unsigned long long)((uint8_t *)&x)[n])
> > +
> > +#define BSWAP16(x)   ((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
> > +#define BSWAP32(x)   ((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
> > +                      (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
> > +#define BSWAP64(x)   ((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
> > +                      (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
> > +                      (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
> > +                      (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
>
> This implementation is wrong. It only does any swapping on little-endian
> systems. That was the desired behavior for CPU_TO_FDT*, but these macros
> need to swap bytes regardless of the native endianness.
>
> (Somehow, none of the other three reviewers noticed this?)

Argh, this was totally missed. Thanks for catching.

The EXTRACT_BYTE() macro is the culprit and does not do the
right thing for a big-endian host. Also, this macro will fail if "x" is
a constant.

I will send a fix ASAP.

Regards,
Anup

>
> Regards,
> Samuel
>
> > +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__  /* CPU(little-endian) */
> > +#define cpu_to_be16(x)               ((uint16_t)BSWAP16(x))
> > +#define cpu_to_be32(x)               ((uint32_t)BSWAP32(x))
> > +#define cpu_to_be64(x)               ((uint64_t)BSWAP64(x))
> > +
> > +#define be16_to_cpu(x)               ((uint16_t)BSWAP16(x))
> > +#define be32_to_cpu(x)               ((uint32_t)BSWAP32(x))
> > +#define be64_to_cpu(x)               ((uint64_t)BSWAP64(x))
> > +
> > +#define cpu_to_le16(x)               ((uint16_t)(x))
> > +#define cpu_to_le32(x)               ((uint32_t)(x))
> > +#define cpu_to_le64(x)               ((uint64_t)(x))
> > +
> > +#define le16_to_cpu(x)               ((uint16_t)(x))
> > +#define le32_to_cpu(x)               ((uint32_t)(x))
> > +#define le64_to_cpu(x)               ((uint64_t)(x))
> > +#else /* CPU(big-endian) */
> > +#define cpu_to_be16(x)               ((uint16_t)(x))
> > +#define cpu_to_be32(x)               ((uint32_t)(x))
> > +#define cpu_to_be64(x)               ((uint64_t)(x))
> > +
> > +#define be16_to_cpu(x)               ((uint16_t)(x))
> > +#define be32_to_cpu(x)               ((uint32_t)(x))
> > +#define be64_to_cpu(x)               ((uint64_t)(x))
> > +
> > +#define cpu_to_le16(x)               ((uint16_t)BSWAP16(x))
> > +#define cpu_to_le32(x)               ((uint32_t)BSWAP32(x))
> > +#define cpu_to_le64(x)               ((uint64_t)BSWAP64(x))
> > +
> > +#define le16_to_cpu(x)               ((uint16_t)BSWAP16(x))
> > +#define le32_to_cpu(x)               ((uint32_t)BSWAP32(x))
> > +#define le64_to_cpu(x)               ((uint64_t)BSWAP64(x))
> > +#endif
> > +
> > +#endif /* __SBI_BYTEORDER_H__ */
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From apatel at ventanamicro.com  Wed Feb  8 20:02:51 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu,  9 Feb 2023 09:32:51 +0530
Subject: [PATCH] include: sbi: Fix BSWAPx() macros for big-endian host
Message-ID: <20230209040251.451328-1-apatel@ventanamicro.com>

The BSWAPx() macros won't do any swapping for big-endian host
because the EXTRACT_BYTE() macro will pickup bytes in reverse
order. Also, the EXTRACT_BYTE() will generate compile error
for constants.

To fix this, we get remove the EXTRACT_BYTE() macro and re-write
BSWAPx() using simple mask and shift operations.

Fixes: 09b34d8cca51 ("include: Add support for byteorder/endianness
conversion")
Reported-by: Samuel Holland <samuel at sholland.org>
Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 include/sbi/sbi_byteorder.h | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/include/sbi/sbi_byteorder.h b/include/sbi/sbi_byteorder.h
index 680710f..15107e1 100644
--- a/include/sbi/sbi_byteorder.h
+++ b/include/sbi/sbi_byteorder.h
@@ -9,16 +9,20 @@
 
 #include <sbi/sbi_types.h>
 
-#define EXTRACT_BYTE(x, n)	((unsigned long long)((uint8_t *)&x)[n])
-
-#define BSWAP16(x)	((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
-#define BSWAP32(x)	((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
-			 (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
-#define BSWAP64(x)	((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
-			 (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
-			 (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
-			 (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
-
+#define BSWAP16(x)	((((x) & 0x00ff) << 8) | \
+			 (((x) & 0xff00) >> 8))
+#define BSWAP32(x)	((((x) & 0x000000ff) << 24) | \
+			 (((x) & 0x0000ff00) << 8) | \
+			 (((x) & 0x00ff0000) >> 8) | \
+			 (((x) & 0xff000000) >> 24))
+#define BSWAP64(x)	((((x) & 0x00000000000000ffULL) << 56) | \
+			 (((x) & 0x000000000000ff00ULL) << 40) | \
+			 (((x) & 0x0000000000ff0000ULL) << 24) | \
+			 (((x) & 0x00000000ff000000ULL) << 8) | \
+			 (((x) & 0x000000ff00000000ULL) >> 8) | \
+			 (((x) & 0x0000ff0000000000ULL) >> 24) | \
+			 (((x) & 0x00ff000000000000ULL) >> 40) | \
+			 (((x) & 0xff00000000000000ULL) >> 56))
 
 #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__  /* CPU(little-endian) */
 #define cpu_to_be16(x)		((uint16_t)BSWAP16(x))
-- 
2.34.1



From debug at rivosinc.com  Wed Feb  8 20:26:52 2023
From: debug at rivosinc.com (Deepak Gupta)
Date: Wed,  8 Feb 2023 20:26:52 -0800
Subject: [PATCH v3 Zisslpcfi 0/2] Support for zisslpcfi in opensbi
Message-ID: <20230209042654.3568990-1-debug@rivosinc.com>

zisslpcfi [1] extension extends risc-v architecture to mitigate against
control-flow integrity attacks (ROP/JOP/COP).

zisslpcfi uses bits (b23-b26) in (m/s/vs) status CSR for enabling cfi in
U mode and record cfi state. One such state is expected landing pad 
(ELP). If forward cfi is enabled, indirect call/jmp updates hart's ELP
state (1bit) to true. ELP state is cleared only by a landing pad
instruction else trap is delivered with ELP state recorded in sstatus
CSR.

This two patch series adds following changes to opensbi
    - Adds support in opensbi to detect zisslpcfi
    - trap redirection updates ELP state accordingly

Qemu implementation for zisslpcfi can be checked out on github [2]
Strawman linux kernel enabling (still very early) can be checked out on github [3]

[1] - https://github.com/riscv/riscv-cfi
[2] - https://github.com/deepak0414/qemu/tree/gh_Zisslpcfi-0.1
[3] - https://github.com/deepak0414/linux-riscv-cfi/tree/Zisslpcfi-0.1_v6.1-rc2

Deepak Gupta (2):
  include: adding support for Zisslpcfi encodings
  lib: sbi: Zisslpcfi detection and elp cfi state reflect back in status

 include/sbi/riscv_encoding.h | 10 ++++++++++
 include/sbi/sbi_hart.h       |  2 ++
 lib/sbi/sbi_hart.c           | 23 +++++++++++++++++++++++
 lib/sbi/sbi_trap.c           | 16 ++++++++++++++++
 4 files changed, 51 insertions(+)

-- 
2.25.1



From debug at rivosinc.com  Wed Feb  8 20:26:53 2023
From: debug at rivosinc.com (Deepak Gupta)
Date: Wed,  8 Feb 2023 20:26:53 -0800
Subject: [PATCH v3 Zisslpcfi 1/2] include: adding support for Zisslpcfi encodings
In-Reply-To: <20230209042654.3568990-1-debug@rivosinc.com>
References: <20230209042654.3568990-1-debug@rivosinc.com>
Message-ID: <20230209042654.3568990-2-debug@rivosinc.com>

Zisslpcfi extension (see link) introduces b60 (CFI) in menvcfg CSR to
enable shadow stack and landing pad for machine and lesser privileged
modes. Additionally extension introduces new bits in xstatus for cfi
state.

Link: https://github.com/riscv/riscv-cfi

Signed-off-by: Deepak Gupta <debug at rivosinc.com>

---
changelog
v1 --> v3:
 - updated with correct name for extension "zisslpcfi"
 - updated encodings.h with additional csr bit definitions.
---
 include/sbi/riscv_encoding.h | 10 ++++++++++
 include/sbi/sbi_hart.h       |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
index b0f08c8..9c45724 100644
--- a/include/sbi/riscv_encoding.h
+++ b/include/sbi/riscv_encoding.h
@@ -32,6 +32,10 @@
 #define MSTATUS_TVM			_UL(0x00100000)
 #define MSTATUS_TW			_UL(0x00200000)
 #define MSTATUS_TSR			_UL(0x00400000)
+#define MSTATUS_UFCFIEN                 _UL(0x00800000)
+#define MSTATUS_UBCFIEN                 _UL(0x01000000)
+#define MSTATUS_SPELP     		_UL(0x02000000)
+#define MSTATUS_MPELP     		_UL(0x04000000)
 #define MSTATUS32_SD			_UL(0x80000000)
 #if __riscv_xlen == 64
 #define MSTATUS_UXL			_ULL(0x0000000300000000)
@@ -210,9 +214,13 @@
 #if __riscv_xlen > 32
 #define ENVCFG_STCE			(_ULL(1) << 63)
 #define ENVCFG_PBMTE			(_ULL(1) << 62)
+#define ENVCFG_CFI			(_ULL(1) << 60)
+#define ENVCFG_SFCFIEN			(_ULL(1) << 59)
 #else
 #define ENVCFGH_STCE			(_UL(1) << 31)
 #define ENVCFGH_PBMTE			(_UL(1) << 30)
+#define ENVCFGH_CFI			(_UL(1) << 28)
+#define ENVCFGH_SFCFIEN			(_UL(1) << 27)
 #endif
 #define ENVCFG_CBZE			(_UL(1) << 7)
 #define ENVCFG_CBCFE			(_UL(1) << 6)
@@ -229,6 +237,8 @@
 #define CSR_USTATUS			0x000
 #define CSR_UIE				0x004
 #define CSR_UTVEC			0x005
+#define CSR_LPLR			0x006
+#define CSR_SSP				0x020
 
 /* User Trap Handling (N-extension) */
 #define CSR_USCRATCH			0x040
diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
index 95b40e7..d153220 100644
--- a/include/sbi/sbi_hart.h
+++ b/include/sbi/sbi_hart.h
@@ -36,6 +36,8 @@ enum sbi_hart_extensions {
 	SBI_HART_EXT_SMSTATEEN,
 	/** HART has Sstc extension */
 	SBI_HART_EXT_SSTC,
+	/** HART has zisslpcfi extension */
+	SBI_HART_EXT_ZISSLPCFI,
 
 	/** Maximum index of Hart extension */
 	SBI_HART_EXT_MAX,
-- 
2.25.1



From debug at rivosinc.com  Wed Feb  8 20:26:54 2023
From: debug at rivosinc.com (Deepak Gupta)
Date: Wed,  8 Feb 2023 20:26:54 -0800
Subject: [PATCH v3 Zisslpcfi 2/2] lib: sbi: Zisslpcfi detection and elp cfi state reflect back in status
In-Reply-To: <20230209042654.3568990-1-debug@rivosinc.com>
References: <20230209042654.3568990-1-debug@rivosinc.com>
Message-ID: <20230209042654.3568990-3-debug@rivosinc.com>

This patch adds support for zisslpcfi detection in sbi_hart.c
If Zisslpcfi is detected, this turns on menvcfg.CFI

Zisslpcfi records status of cfi state in xstatus csr. Missing landing pad
sets MPELP in mstatus. When SBI is redirecting back to S/VS/HS, SPELP is
set in sstatus/vsstatus.

Signed-off-by: Deepak Gupta <debug at rivosinc.com>

---
changelog
v1 --> v3:
 - instead of using "int" for lplr_exist/ssp_exist use "bool"
 - use smaller case true/false
 - updates to use correct extension name "zisslpcfi"
---
 lib/sbi/sbi_hart.c | 23 +++++++++++++++++++++++
 lib/sbi/sbi_trap.c | 16 ++++++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index 5447c52..f7fb524 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -157,6 +157,16 @@ static void mstatus_init(struct sbi_scratch *scratch)
 #endif
 		}
 
+		if (sbi_hart_has_extension(scratch, SBI_HART_EXT_ZISSLPCFI)) {
+#if __riscv_xlen == 32
+			unsigned long menvcfgh_val;
+			menvcfgh_val = csr_read(CSR_MENVCFGH);
+			menvcfgh_val |= ENVCFGH_CFI;
+			csr_write(CSR_MENVCFGH, menvcfgh_val);
+#else
+			menvcfg_val |= ENVCFG_CFI;
+#endif
+		}
 		csr_write(CSR_MENVCFG, menvcfg_val);
 	}
 
@@ -446,6 +456,8 @@ static inline char *sbi_hart_extension_id2string(int ext)
 	case SBI_HART_EXT_SMSTATEEN:
 		estr = "smstateen";
 		break;
+	case SBI_HART_EXT_ZISSLPCFI:
+		estr = "zisslpcfi";
 	default:
 		break;
 	}
@@ -555,6 +567,7 @@ static int hart_detect_features(struct sbi_scratch *scratch)
 		sbi_scratch_offset_ptr(scratch, hart_features_offset);
 	unsigned long val, oldval;
 	int rc;
+	bool ssp_exist, lplr_exist;
 
 	/* If hart features already detected then do nothing */
 	if (hfeatures->detected)
@@ -693,6 +706,16 @@ __mhpm_skip:
 					SBI_HART_EXT_SMSTATEEN, true);
 	}
 
+	if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
+		val = csr_read_allowed(CSR_SSP, (unsigned long)&trap);
+		ssp_exist = trap.cause == 0;
+		val = csr_read_allowed(CSR_LPLR, (unsigned long)&trap);
+		lplr_exist = trap.cause == 0;
+		if (lplr_exist & ssp_exist)
+			__sbi_hart_update_extension(hfeatures,
+					SBI_HART_EXT_ZISSLPCFI, true);
+	}
+
 	/* Let platform populate extensions */
 	rc = sbi_platform_extensions_init(sbi_platform_thishart_ptr(),
 					  hfeatures);
diff --git a/lib/sbi/sbi_trap.c b/lib/sbi/sbi_trap.c
index c875c90..80d9e65 100644
--- a/lib/sbi/sbi_trap.c
+++ b/lib/sbi/sbi_trap.c
@@ -87,6 +87,7 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
 		      struct sbi_trap_info *trap)
 {
 	ulong hstatus, vsstatus, prev_mode;
+	bool elp = false;
 #if __riscv_xlen == 32
 	bool prev_virt = (regs->mstatusH & MSTATUSH_MPV) ? TRUE : FALSE;
 #else
@@ -100,6 +101,13 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
 	if (prev_mode != PRV_S && prev_mode != PRV_U)
 		return SBI_ENOTSUPP;
 
+	/* If extension has support for CFI, clear MPELP because redirecting to VS or (H)S */
+	if (sbi_hart_has_extension(sbi_scratch_thishart_ptr(), SBI_HART_EXT_ZISSLPCFI)) {
+		elp = (regs->mstatus & MSTATUS_MPELP)? true: false;
+		/* Since redirecting, clear mpelp unconditionally */
+		regs->mstatus &= ~MSTATUS_MPELP;
+	}
+
 	/* If exceptions came from VS/VU-mode, redirect to VS-mode if
 	 * delegated in hedeleg
 	 */
@@ -153,6 +161,10 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
 		/* Get VS-mode SSTATUS CSR */
 		vsstatus = csr_read(CSR_VSSTATUS);
 
+		/*if elp was set, set it back in vsstatus */
+		if (elp)
+			vsstatus |= MSTATUS_SPELP;
+
 		/* Set SPP for VS-mode */
 		vsstatus &= ~SSTATUS_SPP;
 		if (prev_mode == PRV_S)
@@ -193,6 +205,10 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
 
 		/* Clear SIE for S-mode */
 		regs->mstatus &= ~MSTATUS_SIE;
+
+		/* if elp was set, set it back in mstatus */
+		if (elp)
+			regs->mstatus |= MSTATUS_SPELP;
 	}
 
 	return 0;
-- 
2.25.1



From anup at brainfault.org  Thu Feb  9 08:51:44 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 9 Feb 2023 22:21:44 +0530
Subject: [PATCH v5 1/8] include: Add defines for SBI debug console extension
In-Reply-To: <20230131172639.cscl4ixscibnokot@orel>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-2-apatel@ventanamicro.com> <20230131172639.cscl4ixscibnokot@orel>
Message-ID: <CAAhSdy1HM3ErPdoW9e_On7pjewBhXubZ_2GzmYnzv4CfwOODNA@mail.gmail.com>

On Tue, Jan 31, 2023 at 10:56 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> On Fri, Jan 13, 2023 at 05:11:03PM +0530, Anup Patel wrote:
> > We add SBI debug console extension related defines/enum to the
>
> Not adding any enums.

Okay, I will update.

>
> > SBI ecall interface header.
> >
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > Reviewed-by: Atish Patra <atishp at rivosinc.com>
> > Reviewed-by: Bin Meng <bmeng at tinylab.org>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
>
> Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
>
> > ---
> >  include/sbi/sbi_ecall_interface.h | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
> > index a3f2bf4..9d6f474 100644
> > --- a/include/sbi/sbi_ecall_interface.h
> > +++ b/include/sbi/sbi_ecall_interface.h
> > @@ -29,6 +29,7 @@
> >  #define SBI_EXT_HSM                          0x48534D
> >  #define SBI_EXT_SRST                         0x53525354
> >  #define SBI_EXT_PMU                          0x504D55
> > +#define SBI_EXT_DBCN                         0x4442434E
> >
> >  /* SBI function IDs for BASE extension*/
> >  #define SBI_EXT_BASE_GET_SPEC_VERSION                0x0
> > @@ -230,6 +231,11 @@ enum sbi_pmu_ctr_type {
> >  /* Flags defined for counter stop function */
> >  #define SBI_PMU_STOP_FLAG_RESET (1 << 0)
> >
> > +/* SBI function IDs for DBCN extension */
> > +#define SBI_EXT_DBCN_CONSOLE_WRITE           0x0
> > +#define SBI_EXT_DBCN_CONSOLE_READ            0x1
> > +#define SBI_EXT_DBCN_CONSOLE_WRITE_BYTE              0x2
> > +
> >  /* SBI base specification related macros */
> >  #define SBI_SPEC_VERSION_MAJOR_OFFSET                24
> >  #define SBI_SPEC_VERSION_MAJOR_MASK          0x7f
> > --
> > 2.34.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From anup at brainfault.org  Thu Feb  9 08:59:09 2023
From: anup at brainfault.org (Anup Patel)
Date: Thu, 9 Feb 2023 22:29:09 +0530
Subject: [PATCH v5 2/8] lib: sbi: Add sbi_nputs() function
In-Reply-To: <20230131173247.hxh3s4xhlufmnuvm@orel>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-3-apatel@ventanamicro.com> <20230131173247.hxh3s4xhlufmnuvm@orel>
Message-ID: <CAAhSdy0KeJM0uvDyhgDpwDda1tsVxxpaRYkaXym4vTj6a4PJiQ@mail.gmail.com>

On Tue, Jan 31, 2023 at 11:02 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> On Fri, Jan 13, 2023 at 05:11:04PM +0530, Anup Patel wrote:
> > We add new sbi_nputs() which help us print a fixed number of characters
> > from a physical memory location.
> >
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > Reviewed-by: Atish Patra <atishp at rivosinc.com>
> > Reviewed-by: Bin Meng <bmeng at tinylab.org>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
> > ---
> >  include/sbi/sbi_console.h |  2 ++
> >  lib/sbi/sbi_console.c     | 10 ++++++++++
> >  2 files changed, 12 insertions(+)
> >
> > diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
> > index e15b55d..1bdeeb9 100644
> > --- a/include/sbi/sbi_console.h
> > +++ b/include/sbi/sbi_console.h
> > @@ -33,6 +33,8 @@ void sbi_putc(char ch);
> >
> >  void sbi_puts(const char *str);
> >
> > +void sbi_nputs(const char *str, unsigned long len);
> > +
> >  void sbi_gets(char *s, int maxwidth, char endchar);
> >
> >  int __printf(2, 3) sbi_sprintf(char *out, const char *format, ...);
> > diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> > index bb6e1ef..946c61d 100644
> > --- a/lib/sbi/sbi_console.c
> > +++ b/lib/sbi/sbi_console.c
> > @@ -51,6 +51,16 @@ void sbi_puts(const char *str)
> >       spin_unlock(&console_out_lock);
> >  }
> >
> > +void sbi_nputs(const char *str, unsigned long len)
> > +{
> > +     unsigned long i;
> > +
> > +     spin_lock(&console_out_lock);
> > +     for (i = 0; i < len; i++)
> > +             sbi_putc(str[i]);
>
> sbi_putc() adds '\r' after '\n'. So, sbi_nputs("\n\n\n\n", 4)
> actually outputs 8 chars, but I imagine the new debug extension
> will still report it only wrote 4. Is that a problem?

I don't see this as a problem because the additional '\r' is only
for correctly rederring the effect of `\n` on console.

In other words, the nputs() function is still printing all the
characters it got as parameters and additionally it is printing
few more characters.

Regards,
Anup


From jrtc27 at jrtc27.com  Thu Feb  9 09:02:11 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Thu, 9 Feb 2023 17:02:11 +0000
Subject: [PATCH v5 2/8] lib: sbi: Add sbi_nputs() function
In-Reply-To: <CAAhSdy0KeJM0uvDyhgDpwDda1tsVxxpaRYkaXym4vTj6a4PJiQ@mail.gmail.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-3-apatel@ventanamicro.com>
 <20230131173247.hxh3s4xhlufmnuvm@orel>
 <CAAhSdy0KeJM0uvDyhgDpwDda1tsVxxpaRYkaXym4vTj6a4PJiQ@mail.gmail.com>
Message-ID: <1C99EBEE-EC8E-4324-ADAF-9199CE560A15@jrtc27.com>

On 9 Feb 2023, at 16:59, Anup Patel <anup at brainfault.org> wrote:
> 
> On Tue, Jan 31, 2023 at 11:02 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>> 
>> On Fri, Jan 13, 2023 at 05:11:04PM +0530, Anup Patel wrote:
>>> We add new sbi_nputs() which help us print a fixed number of characters
>>> from a physical memory location.
>>> 
>>> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
>>> Reviewed-by: Atish Patra <atishp at rivosinc.com>
>>> Reviewed-by: Bin Meng <bmeng at tinylab.org>
>>> Reviewed-by: Xiang W <wxjstz at 126.com>
>>> ---
>>> include/sbi/sbi_console.h |  2 ++
>>> lib/sbi/sbi_console.c     | 10 ++++++++++
>>> 2 files changed, 12 insertions(+)
>>> 
>>> diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
>>> index e15b55d..1bdeeb9 100644
>>> --- a/include/sbi/sbi_console.h
>>> +++ b/include/sbi/sbi_console.h
>>> @@ -33,6 +33,8 @@ void sbi_putc(char ch);
>>> 
>>> void sbi_puts(const char *str);
>>> 
>>> +void sbi_nputs(const char *str, unsigned long len);
>>> +
>>> void sbi_gets(char *s, int maxwidth, char endchar);
>>> 
>>> int __printf(2, 3) sbi_sprintf(char *out, const char *format, ...);
>>> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
>>> index bb6e1ef..946c61d 100644
>>> --- a/lib/sbi/sbi_console.c
>>> +++ b/lib/sbi/sbi_console.c
>>> @@ -51,6 +51,16 @@ void sbi_puts(const char *str)
>>>      spin_unlock(&console_out_lock);
>>> }
>>> 
>>> +void sbi_nputs(const char *str, unsigned long len)
>>> +{
>>> +     unsigned long i;
>>> +
>>> +     spin_lock(&console_out_lock);
>>> +     for (i = 0; i < len; i++)
>>> +             sbi_putc(str[i]);
>> 
>> sbi_putc() adds '\r' after '\n'. So, sbi_nputs("\n\n\n\n", 4)
>> actually outputs 8 chars, but I imagine the new debug extension
>> will still report it only wrote 4. Is that a problem?
> 
> I don't see this as a problem because the additional '\r' is only
> for correctly rederring the effect of `\n` on console.
> 
> In other words, the nputs() function is still printing all the
> characters it got as parameters and additionally it is printing
> few more characters.

Which is the same as the Unix tty layer when onlcr is set.

Jess



From anup at brainfault.org  Thu Feb  9 20:23:40 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 10 Feb 2023 09:53:40 +0530
Subject: [PATCH v5 5/8] lib: sbi: Implement SBI debug console extension
In-Reply-To: <20230201085815.77wcmkw76l2in5qc@orel>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-6-apatel@ventanamicro.com> <20230201085815.77wcmkw76l2in5qc@orel>
Message-ID: <CAAhSdy0HC23UTSFV6_hep7+QMsp2sHOG-QhQLwGzFBvD1ivgkg@mail.gmail.com>

On Wed, Feb 1, 2023 at 2:28 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> On Fri, Jan 13, 2023 at 05:11:07PM +0530, Anup Patel wrote:
> > We implement SBI debug console extension as one of the replacement
> > SBI extensions. This extension is only available when OpenSBI platform
> > provides a console device to generic library.
> >
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > Reviewed-by: Atish Patra <atishp at rivosinc.com>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
> > Reviewed-by: Bin Meng <bmeng at tinylab.org>
> > ---
> >  lib/sbi/Kconfig          |  4 +++
> >  lib/sbi/objects.mk       |  3 ++
> >  lib/sbi/sbi_ecall_dbcn.c | 72 ++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 79 insertions(+)
> >  create mode 100644 lib/sbi/sbi_ecall_dbcn.c
> >
> > diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
> > index df74bba..ef6728b 100644
> > --- a/lib/sbi/Kconfig
> > +++ b/lib/sbi/Kconfig
> > @@ -26,6 +26,10 @@ config SBI_ECALL_PMU
> >       bool "Performance Monitoring Unit extension"
> >       default y
> >
> > +config SBI_ECALL_DBCN
> > +     bool "Debug Console extension"
> > +     default y
> > +
> >  config SBI_ECALL_LEGACY
> >       bool "SBI v0.1 legacy extensions"
> >       default y
> > diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
> > index c774ebb..319f38d 100644
> > --- a/lib/sbi/objects.mk
> > +++ b/lib/sbi/objects.mk
> > @@ -37,6 +37,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
> >  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
> >  libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
> >
> > +carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_DBCN) += ecall_dbcn
> > +libsbi-objs-$(CONFIG_SBI_ECALL_DBCN) += sbi_ecall_dbcn.o
> > +
> >  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_LEGACY) += ecall_legacy
> >  libsbi-objs-$(CONFIG_SBI_ECALL_LEGACY) += sbi_ecall_legacy.o
> >
> > diff --git a/lib/sbi/sbi_ecall_dbcn.c b/lib/sbi/sbi_ecall_dbcn.c
> > new file mode 100644
> > index 0000000..bfa40e9
> > --- /dev/null
> > +++ b/lib/sbi/sbi_ecall_dbcn.c
> > @@ -0,0 +1,72 @@
> > +/*
> > + * SPDX-License-Identifier: BSD-2-Clause
> > + *
> > + * Copyright (c) 2022 Ventana Micro Systems Inc.
> > + *
> > + * Authors:
> > + *   Anup Patel <apatel at ventanamicro.com>
> > + */
> > +
> > +#include <sbi/sbi_console.h>
> > +#include <sbi/sbi_domain.h>
> > +#include <sbi/sbi_error.h>
> > +#include <sbi/sbi_ecall.h>
> > +#include <sbi/sbi_ecall_interface.h>
> > +#include <sbi/sbi_trap.h>
> > +#include <sbi/riscv_asm.h>
> > +
> > +static int sbi_ecall_dbcn_handler(unsigned long extid, unsigned long funcid,
> > +                               const struct sbi_trap_regs *regs,
> > +                               unsigned long *out_val,
> > +                               struct sbi_trap_info *out_trap)
> > +{
> > +     ulong smode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >>
> > +                     MSTATUS_MPP_SHIFT;
> > +
> > +     switch (funcid) {
> > +     case SBI_EXT_DBCN_CONSOLE_WRITE:
> > +     case SBI_EXT_DBCN_CONSOLE_READ:
> > +             /*
> > +              * On RV32, the M-mode can only access the first 4GB of
> > +              * the physical address space because M-mode does not have
> > +              * MMU to access full 34-bit physical address space.
> > +              *
> > +              * Based on above, we simply fail if the upper 32bits of
> > +              * the physical address (i.e. a2 register) is non-zero on
> > +              * RV32.
> > +              */
> > +#if __riscv_xlen == 32
> > +             if (regs->a2)
> > +                     return SBI_ERR_FAILED;
>
> The spec drafts says SBI_ERR_FAILED is only for I/O errors. I think
> we should extend the spec draft to point this situation out and return
> SBI_ERR_INVALID_ADDRESS.

This is a limitation for OpenSBI RV32 because to support a 34bit
physical address for RV32, we will need special infrastructure
using a temporary page table. Due to this reason, twe reat this
limitation as I/O error on OpenSBI RV32 side.

>
> > +#endif
> > +             if (!sbi_domain_check_addr_range(sbi_domain_thishart_ptr(),
> > +                                     regs->a1, regs->a0, smode,
> > +                                     SBI_DOMAIN_READ|SBI_DOMAIN_WRITE))
> > +                     return SBI_EINVALID_ADDR;
>
> The spec draft says this should be SBI_ERR_INVALID_PARAM

Okay, I will update.

>
> > +             if (funcid == SBI_EXT_DBCN_CONSOLE_WRITE)
> > +                     sbi_nputs((const char *)regs->a1, regs->a0);
> > +             else
> > +                     *out_val = sbi_ngets((char *)regs->a1, regs->a0);
> > +             return 0;
> > +     case SBI_EXT_DBCN_CONSOLE_WRITE_BYTE:
> > +             sbi_putc(regs->a0);
> > +             return 0;
> > +     default:
> > +             break;
> > +     }
> > +
> > +     return SBI_ENOTSUPP;
> > +}
> > +
> > +static int sbi_ecall_dbcn_probe(unsigned long extid, unsigned long *out_val)
> > +{
> > +     *out_val = (sbi_console_get_device()) ? 1 : 0;
>
> nit: Unnecessary ()

Okay, I will update.

>
> > +     return 0;
> > +}
> > +
> > +struct sbi_ecall_extension ecall_dbcn = {
> > +     .extid_start = SBI_EXT_DBCN,
> > +     .extid_end = SBI_EXT_DBCN,
> > +     .handle = sbi_ecall_dbcn_handler,
> > +     .probe = sbi_ecall_dbcn_probe,
> > +};
> > --
> > 2.34.1
> >
> >
>
> Thanks,
> drew

Regards,
Anup


From anup at brainfault.org  Thu Feb  9 20:25:06 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 10 Feb 2023 09:55:06 +0530
Subject: [PATCH v5 5/8] lib: sbi: Implement SBI debug console extension
In-Reply-To: <20230201093558.ty5emxqvl5vrdfg3@orel>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-6-apatel@ventanamicro.com> <20230201093558.ty5emxqvl5vrdfg3@orel>
Message-ID: <CAAhSdy3uL_O+BCqvA-A3W72UVziHVJMBU=xqAj_zNqyxd5diHA@mail.gmail.com>

On Wed, Feb 1, 2023 at 3:06 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> On Fri, Jan 13, 2023 at 05:11:07PM +0530, Anup Patel wrote:
> > We implement SBI debug console extension as one of the replacement
> > SBI extensions. This extension is only available when OpenSBI platform
> > provides a console device to generic library.
> >
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > Reviewed-by: Atish Patra <atishp at rivosinc.com>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
> > Reviewed-by: Bin Meng <bmeng at tinylab.org>
> > ---
> >  lib/sbi/Kconfig          |  4 +++
> >  lib/sbi/objects.mk       |  3 ++
> >  lib/sbi/sbi_ecall_dbcn.c | 72 ++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 79 insertions(+)
> >  create mode 100644 lib/sbi/sbi_ecall_dbcn.c
> >
> > diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
> > index df74bba..ef6728b 100644
> > --- a/lib/sbi/Kconfig
> > +++ b/lib/sbi/Kconfig
> > @@ -26,6 +26,10 @@ config SBI_ECALL_PMU
> >       bool "Performance Monitoring Unit extension"
> >       default y
> >
> > +config SBI_ECALL_DBCN
> > +     bool "Debug Console extension"
> > +     default y
> > +
> >  config SBI_ECALL_LEGACY
> >       bool "SBI v0.1 legacy extensions"
> >       default y
> > diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
> > index c774ebb..319f38d 100644
> > --- a/lib/sbi/objects.mk
> > +++ b/lib/sbi/objects.mk
> > @@ -37,6 +37,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
> >  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
> >  libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
> >
> > +carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_DBCN) += ecall_dbcn
> > +libsbi-objs-$(CONFIG_SBI_ECALL_DBCN) += sbi_ecall_dbcn.o
> > +
> >  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_LEGACY) += ecall_legacy
> >  libsbi-objs-$(CONFIG_SBI_ECALL_LEGACY) += sbi_ecall_legacy.o
> >
> > diff --git a/lib/sbi/sbi_ecall_dbcn.c b/lib/sbi/sbi_ecall_dbcn.c
> > new file mode 100644
> > index 0000000..bfa40e9
> > --- /dev/null
> > +++ b/lib/sbi/sbi_ecall_dbcn.c
> > @@ -0,0 +1,72 @@
> > +/*
> > + * SPDX-License-Identifier: BSD-2-Clause
> > + *
> > + * Copyright (c) 2022 Ventana Micro Systems Inc.
> > + *
> > + * Authors:
> > + *   Anup Patel <apatel at ventanamicro.com>
> > + */
> > +
> > +#include <sbi/sbi_console.h>
> > +#include <sbi/sbi_domain.h>
> > +#include <sbi/sbi_error.h>
> > +#include <sbi/sbi_ecall.h>
> > +#include <sbi/sbi_ecall_interface.h>
> > +#include <sbi/sbi_trap.h>
> > +#include <sbi/riscv_asm.h>
> > +
> > +static int sbi_ecall_dbcn_handler(unsigned long extid, unsigned long funcid,
> > +                               const struct sbi_trap_regs *regs,
> > +                               unsigned long *out_val,
> > +                               struct sbi_trap_info *out_trap)
> > +{
> > +     ulong smode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >>
> > +                     MSTATUS_MPP_SHIFT;
> > +
> > +     switch (funcid) {
> > +     case SBI_EXT_DBCN_CONSOLE_WRITE:
> > +     case SBI_EXT_DBCN_CONSOLE_READ:
> > +             /*
> > +              * On RV32, the M-mode can only access the first 4GB of
> > +              * the physical address space because M-mode does not have
> > +              * MMU to access full 34-bit physical address space.
> > +              *
> > +              * Based on above, we simply fail if the upper 32bits of
> > +              * the physical address (i.e. a2 register) is non-zero on
> > +              * RV32.
> > +              */
> > +#if __riscv_xlen == 32
> > +             if (regs->a2)
> > +                     return SBI_ERR_FAILED;
> > +#endif
> > +             if (!sbi_domain_check_addr_range(sbi_domain_thishart_ptr(),
> > +                                     regs->a1, regs->a0, smode,
> > +                                     SBI_DOMAIN_READ|SBI_DOMAIN_WRITE))
> > +                     return SBI_EINVALID_ADDR;
> > +             if (funcid == SBI_EXT_DBCN_CONSOLE_WRITE)
> > +                     sbi_nputs((const char *)regs->a1, regs->a0);
>
> The spec draft says we return the number of bytes written, so we need
> sbi_nputs to return that. If we never intend to support partial writes,
> then I guess we at least need '*out_val = regs->a0'.

I had already updated the implementation based on the latest draft so
this is already taken care in v6.

Regards,
Anup

>
> Thanks,
> drew
>
> > +             else
> > +                     *out_val = sbi_ngets((char *)regs->a1, regs->a0);
> > +             return 0;
> > +     case SBI_EXT_DBCN_CONSOLE_WRITE_BYTE:
> > +             sbi_putc(regs->a0);
> > +             return 0;
> > +     default:
> > +             break;
> > +     }
> > +
> > +     return SBI_ENOTSUPP;
> > +}
> > +
> > +static int sbi_ecall_dbcn_probe(unsigned long extid, unsigned long *out_val)
> > +{
> > +     *out_val = (sbi_console_get_device()) ? 1 : 0;
> > +     return 0;
> > +}
> > +
> > +struct sbi_ecall_extension ecall_dbcn = {
> > +     .extid_start = SBI_EXT_DBCN,
> > +     .extid_end = SBI_EXT_DBCN,
> > +     .handle = sbi_ecall_dbcn_handler,
> > +     .probe = sbi_ecall_dbcn_probe,
> > +};
> > --
> > 2.34.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From jrtc27 at jrtc27.com  Thu Feb  9 21:19:59 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Fri, 10 Feb 2023 05:19:59 +0000
Subject: [PATCH v5 5/8] lib: sbi: Implement SBI debug console extension
In-Reply-To: <CAAhSdy0HC23UTSFV6_hep7+QMsp2sHOG-QhQLwGzFBvD1ivgkg@mail.gmail.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-6-apatel@ventanamicro.com>
 <20230201085815.77wcmkw76l2in5qc@orel>
 <CAAhSdy0HC23UTSFV6_hep7+QMsp2sHOG-QhQLwGzFBvD1ivgkg@mail.gmail.com>
Message-ID: <205F9423-3BD2-4510-854E-6FDE749BE7AB@jrtc27.com>

On 10 Feb 2023, at 04:23, Anup Patel <anup at brainfault.org> wrote:
> 
> On Wed, Feb 1, 2023 at 2:28 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>> 
>> On Fri, Jan 13, 2023 at 05:11:07PM +0530, Anup Patel wrote:
>>> We implement SBI debug console extension as one of the replacement
>>> SBI extensions. This extension is only available when OpenSBI platform
>>> provides a console device to generic library.
>>> 
>>> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
>>> Reviewed-by: Atish Patra <atishp at rivosinc.com>
>>> Reviewed-by: Xiang W <wxjstz at 126.com>
>>> Reviewed-by: Bin Meng <bmeng at tinylab.org>
>>> ---
>>> lib/sbi/Kconfig          |  4 +++
>>> lib/sbi/objects.mk       |  3 ++
>>> lib/sbi/sbi_ecall_dbcn.c | 72 ++++++++++++++++++++++++++++++++++++++++
>>> 3 files changed, 79 insertions(+)
>>> create mode 100644 lib/sbi/sbi_ecall_dbcn.c
>>> 
>>> diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
>>> index df74bba..ef6728b 100644
>>> --- a/lib/sbi/Kconfig
>>> +++ b/lib/sbi/Kconfig
>>> @@ -26,6 +26,10 @@ config SBI_ECALL_PMU
>>>      bool "Performance Monitoring Unit extension"
>>>      default y
>>> 
>>> +config SBI_ECALL_DBCN
>>> +     bool "Debug Console extension"
>>> +     default y
>>> +
>>> config SBI_ECALL_LEGACY
>>>      bool "SBI v0.1 legacy extensions"
>>>      default y
>>> diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
>>> index c774ebb..319f38d 100644
>>> --- a/lib/sbi/objects.mk
>>> +++ b/lib/sbi/objects.mk
>>> @@ -37,6 +37,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
>>> carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
>>> libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
>>> 
>>> +carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_DBCN) += ecall_dbcn
>>> +libsbi-objs-$(CONFIG_SBI_ECALL_DBCN) += sbi_ecall_dbcn.o
>>> +
>>> carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_LEGACY) += ecall_legacy
>>> libsbi-objs-$(CONFIG_SBI_ECALL_LEGACY) += sbi_ecall_legacy.o
>>> 
>>> diff --git a/lib/sbi/sbi_ecall_dbcn.c b/lib/sbi/sbi_ecall_dbcn.c
>>> new file mode 100644
>>> index 0000000..bfa40e9
>>> --- /dev/null
>>> +++ b/lib/sbi/sbi_ecall_dbcn.c
>>> @@ -0,0 +1,72 @@
>>> +/*
>>> + * SPDX-License-Identifier: BSD-2-Clause
>>> + *
>>> + * Copyright (c) 2022 Ventana Micro Systems Inc.
>>> + *
>>> + * Authors:
>>> + *   Anup Patel <apatel at ventanamicro.com>
>>> + */
>>> +
>>> +#include <sbi/sbi_console.h>
>>> +#include <sbi/sbi_domain.h>
>>> +#include <sbi/sbi_error.h>
>>> +#include <sbi/sbi_ecall.h>
>>> +#include <sbi/sbi_ecall_interface.h>
>>> +#include <sbi/sbi_trap.h>
>>> +#include <sbi/riscv_asm.h>
>>> +
>>> +static int sbi_ecall_dbcn_handler(unsigned long extid, unsigned long funcid,
>>> +                               const struct sbi_trap_regs *regs,
>>> +                               unsigned long *out_val,
>>> +                               struct sbi_trap_info *out_trap)
>>> +{
>>> +     ulong smode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >>
>>> +                     MSTATUS_MPP_SHIFT;
>>> +
>>> +     switch (funcid) {
>>> +     case SBI_EXT_DBCN_CONSOLE_WRITE:
>>> +     case SBI_EXT_DBCN_CONSOLE_READ:
>>> +             /*
>>> +              * On RV32, the M-mode can only access the first 4GB of
>>> +              * the physical address space because M-mode does not have
>>> +              * MMU to access full 34-bit physical address space.
>>> +              *
>>> +              * Based on above, we simply fail if the upper 32bits of
>>> +              * the physical address (i.e. a2 register) is non-zero on
>>> +              * RV32.
>>> +              */
>>> +#if __riscv_xlen == 32
>>> +             if (regs->a2)
>>> +                     return SBI_ERR_FAILED;
>> 
>> The spec drafts says SBI_ERR_FAILED is only for I/O errors. I think
>> we should extend the spec draft to point this situation out and return
>> SBI_ERR_INVALID_ADDRESS.
> 
> This is a limitation for OpenSBI RV32 because to support a 34bit
> physical address for RV32, we will need special infrastructure
> using a temporary page table. Due to this reason, twe reat this
> limitation as I/O error on OpenSBI RV32 side.

Were the interface to take a virtual address (or, really, pointer) this
would not be an issue as the supervisor would already have handled the
mapping for you. Which is what I?ve kept advocating for.

Jess

>> 
>>> +#endif
>>> +             if (!sbi_domain_check_addr_range(sbi_domain_thishart_ptr(),
>>> +                                     regs->a1, regs->a0, smode,
>>> +                                     SBI_DOMAIN_READ|SBI_DOMAIN_WRITE))
>>> +                     return SBI_EINVALID_ADDR;
>> 
>> The spec draft says this should be SBI_ERR_INVALID_PARAM
> 
> Okay, I will update.
> 
>> 
>>> +             if (funcid == SBI_EXT_DBCN_CONSOLE_WRITE)
>>> +                     sbi_nputs((const char *)regs->a1, regs->a0);
>>> +             else
>>> +                     *out_val = sbi_ngets((char *)regs->a1, regs->a0);
>>> +             return 0;
>>> +     case SBI_EXT_DBCN_CONSOLE_WRITE_BYTE:
>>> +             sbi_putc(regs->a0);
>>> +             return 0;
>>> +     default:
>>> +             break;
>>> +     }
>>> +
>>> +     return SBI_ENOTSUPP;
>>> +}
>>> +
>>> +static int sbi_ecall_dbcn_probe(unsigned long extid, unsigned long *out_val)
>>> +{
>>> +     *out_val = (sbi_console_get_device()) ? 1 : 0;
>> 
>> nit: Unnecessary ()
> 
> Okay, I will update.
> 
>> 
>>> +     return 0;
>>> +}
>>> +
>>> +struct sbi_ecall_extension ecall_dbcn = {
>>> +     .extid_start = SBI_EXT_DBCN,
>>> +     .extid_end = SBI_EXT_DBCN,
>>> +     .handle = sbi_ecall_dbcn_handler,
>>> +     .probe = sbi_ecall_dbcn_probe,
>>> +};
>>> --
>>> 2.34.1
>>> 
>>> 
>> 
>> Thanks,
>> drew
> 
> Regards,
> Anup
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



From apatel at ventanamicro.com  Thu Feb  9 21:29:28 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 10:59:28 +0530
Subject: [PATCH v5 5/8] lib: sbi: Implement SBI debug console extension
In-Reply-To: <205F9423-3BD2-4510-854E-6FDE749BE7AB@jrtc27.com>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-6-apatel@ventanamicro.com> <20230201085815.77wcmkw76l2in5qc@orel>
 <CAAhSdy0HC23UTSFV6_hep7+QMsp2sHOG-QhQLwGzFBvD1ivgkg@mail.gmail.com> <205F9423-3BD2-4510-854E-6FDE749BE7AB@jrtc27.com>
Message-ID: <CAK9=C2X+PdZuwarbQ9GD8OrasbAZfq7aRRWQ5U3Jw8M_cymsFQ@mail.gmail.com>

On Fri, Feb 10, 2023 at 10:50 AM Jessica Clarke <jrtc27 at jrtc27.com> wrote:
>
> On 10 Feb 2023, at 04:23, Anup Patel <anup at brainfault.org> wrote:
> >
> > On Wed, Feb 1, 2023 at 2:28 PM Andrew Jones <ajones at ventanamicro.com> wrote:
> >>
> >> On Fri, Jan 13, 2023 at 05:11:07PM +0530, Anup Patel wrote:
> >>> We implement SBI debug console extension as one of the replacement
> >>> SBI extensions. This extension is only available when OpenSBI platform
> >>> provides a console device to generic library.
> >>>
> >>> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> >>> Reviewed-by: Atish Patra <atishp at rivosinc.com>
> >>> Reviewed-by: Xiang W <wxjstz at 126.com>
> >>> Reviewed-by: Bin Meng <bmeng at tinylab.org>
> >>> ---
> >>> lib/sbi/Kconfig          |  4 +++
> >>> lib/sbi/objects.mk       |  3 ++
> >>> lib/sbi/sbi_ecall_dbcn.c | 72 ++++++++++++++++++++++++++++++++++++++++
> >>> 3 files changed, 79 insertions(+)
> >>> create mode 100644 lib/sbi/sbi_ecall_dbcn.c
> >>>
> >>> diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
> >>> index df74bba..ef6728b 100644
> >>> --- a/lib/sbi/Kconfig
> >>> +++ b/lib/sbi/Kconfig
> >>> @@ -26,6 +26,10 @@ config SBI_ECALL_PMU
> >>>      bool "Performance Monitoring Unit extension"
> >>>      default y
> >>>
> >>> +config SBI_ECALL_DBCN
> >>> +     bool "Debug Console extension"
> >>> +     default y
> >>> +
> >>> config SBI_ECALL_LEGACY
> >>>      bool "SBI v0.1 legacy extensions"
> >>>      default y
> >>> diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
> >>> index c774ebb..319f38d 100644
> >>> --- a/lib/sbi/objects.mk
> >>> +++ b/lib/sbi/objects.mk
> >>> @@ -37,6 +37,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
> >>> carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
> >>> libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
> >>>
> >>> +carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_DBCN) += ecall_dbcn
> >>> +libsbi-objs-$(CONFIG_SBI_ECALL_DBCN) += sbi_ecall_dbcn.o
> >>> +
> >>> carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_LEGACY) += ecall_legacy
> >>> libsbi-objs-$(CONFIG_SBI_ECALL_LEGACY) += sbi_ecall_legacy.o
> >>>
> >>> diff --git a/lib/sbi/sbi_ecall_dbcn.c b/lib/sbi/sbi_ecall_dbcn.c
> >>> new file mode 100644
> >>> index 0000000..bfa40e9
> >>> --- /dev/null
> >>> +++ b/lib/sbi/sbi_ecall_dbcn.c
> >>> @@ -0,0 +1,72 @@
> >>> +/*
> >>> + * SPDX-License-Identifier: BSD-2-Clause
> >>> + *
> >>> + * Copyright (c) 2022 Ventana Micro Systems Inc.
> >>> + *
> >>> + * Authors:
> >>> + *   Anup Patel <apatel at ventanamicro.com>
> >>> + */
> >>> +
> >>> +#include <sbi/sbi_console.h>
> >>> +#include <sbi/sbi_domain.h>
> >>> +#include <sbi/sbi_error.h>
> >>> +#include <sbi/sbi_ecall.h>
> >>> +#include <sbi/sbi_ecall_interface.h>
> >>> +#include <sbi/sbi_trap.h>
> >>> +#include <sbi/riscv_asm.h>
> >>> +
> >>> +static int sbi_ecall_dbcn_handler(unsigned long extid, unsigned long funcid,
> >>> +                               const struct sbi_trap_regs *regs,
> >>> +                               unsigned long *out_val,
> >>> +                               struct sbi_trap_info *out_trap)
> >>> +{
> >>> +     ulong smode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >>
> >>> +                     MSTATUS_MPP_SHIFT;
> >>> +
> >>> +     switch (funcid) {
> >>> +     case SBI_EXT_DBCN_CONSOLE_WRITE:
> >>> +     case SBI_EXT_DBCN_CONSOLE_READ:
> >>> +             /*
> >>> +              * On RV32, the M-mode can only access the first 4GB of
> >>> +              * the physical address space because M-mode does not have
> >>> +              * MMU to access full 34-bit physical address space.
> >>> +              *
> >>> +              * Based on above, we simply fail if the upper 32bits of
> >>> +              * the physical address (i.e. a2 register) is non-zero on
> >>> +              * RV32.
> >>> +              */
> >>> +#if __riscv_xlen == 32
> >>> +             if (regs->a2)
> >>> +                     return SBI_ERR_FAILED;
> >>
> >> The spec drafts says SBI_ERR_FAILED is only for I/O errors. I think
> >> we should extend the spec draft to point this situation out and return
> >> SBI_ERR_INVALID_ADDRESS.
> >
> > This is a limitation for OpenSBI RV32 because to support a 34bit
> > physical address for RV32, we will need special infrastructure
> > using a temporary page table. Due to this reason, twe reat this
> > limitation as I/O error on OpenSBI RV32 side.
>
> Were the interface to take a virtual address (or, really, pointer) this
> would not be an issue as the supervisor would already have handled the
> mapping for you. Which is what I?ve kept advocating for.

There are two major issues in passing virtual address:
1) The virtual address could point to anything so SBI implementation
    (Both M-mode firmware and hypervisors) have to translate the
    virtual address into physical address and then check sanity of
    the virtual address. This will involve doing software page table walks
    which are very expensive.
2) SBI implementation will have to use unpriv load/stores for accessing
   data pointed by virtual address which can potentially trap if the page
   table mapping is changed by OS while the SBI implementation was
   accessing it.

Regards,
Anup

>
> Jess
>
> >>
> >>> +#endif
> >>> +             if (!sbi_domain_check_addr_range(sbi_domain_thishart_ptr(),
> >>> +                                     regs->a1, regs->a0, smode,
> >>> +                                     SBI_DOMAIN_READ|SBI_DOMAIN_WRITE))
> >>> +                     return SBI_EINVALID_ADDR;
> >>
> >> The spec draft says this should be SBI_ERR_INVALID_PARAM
> >
> > Okay, I will update.
> >
> >>
> >>> +             if (funcid == SBI_EXT_DBCN_CONSOLE_WRITE)
> >>> +                     sbi_nputs((const char *)regs->a1, regs->a0);
> >>> +             else
> >>> +                     *out_val = sbi_ngets((char *)regs->a1, regs->a0);
> >>> +             return 0;
> >>> +     case SBI_EXT_DBCN_CONSOLE_WRITE_BYTE:
> >>> +             sbi_putc(regs->a0);
> >>> +             return 0;
> >>> +     default:
> >>> +             break;
> >>> +     }
> >>> +
> >>> +     return SBI_ENOTSUPP;
> >>> +}
> >>> +
> >>> +static int sbi_ecall_dbcn_probe(unsigned long extid, unsigned long *out_val)
> >>> +{
> >>> +     *out_val = (sbi_console_get_device()) ? 1 : 0;
> >>
> >> nit: Unnecessary ()
> >
> > Okay, I will update.
> >
> >>
> >>> +     return 0;
> >>> +}
> >>> +
> >>> +struct sbi_ecall_extension ecall_dbcn = {
> >>> +     .extid_start = SBI_EXT_DBCN,
> >>> +     .extid_end = SBI_EXT_DBCN,
> >>> +     .handle = sbi_ecall_dbcn_handler,
> >>> +     .probe = sbi_ecall_dbcn_probe,
> >>> +};
> >>> --
> >>> 2.34.1
> >>>
> >>>
> >>
> >> Thanks,
> >> drew
> >
> > Regards,
> > Anup
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>


From anup at brainfault.org  Thu Feb  9 21:41:19 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 10 Feb 2023 11:11:19 +0530
Subject: [PATCH v5 8/8] lib: sbi: Speed-up sbi_printf() and friends using nputs()
In-Reply-To: <20230201092235.vessrt2t7jdc5hfl@orel>
References: <20230113114110.1916226-1-apatel@ventanamicro.com>
 <20230113114110.1916226-9-apatel@ventanamicro.com> <20230201092235.vessrt2t7jdc5hfl@orel>
Message-ID: <CAAhSdy20PLMEoOvbFeRvas00NyGANQNhk93eU4TVTOieE_fH5w@mail.gmail.com>

On Wed, Feb 1, 2023 at 2:52 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> On Fri, Jan 13, 2023 at 05:11:10PM +0530, Anup Patel wrote:
> > The sbi_printf() is slow for semihosting because it prints one
> > character at a time. To speed-up sbi_printf() for semihosting,
> > we use a temporary buffer and nputs().
> >
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > ---
> >  lib/sbi/sbi_console.c | 52 ++++++++++++++++++++++++++++---------------
> >  1 file changed, 34 insertions(+), 18 deletions(-)
> >
> > diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> > index c1b9f73..41881d7 100644
> > --- a/lib/sbi/sbi_console.c
> > +++ b/lib/sbi/sbi_console.c
> > @@ -42,31 +42,29 @@ void sbi_putc(char ch)
> >       }
> >  }
> >
> > -void sbi_puts(const char *str)
> > +static void nputs(const char *str, unsigned long len)
> >  {
> > -     spin_lock(&console_out_lock);
> > +     unsigned long i;
> > +
> >       if (console_dev && console_dev->console_puts) {
> > -             console_dev->console_puts(str, sbi_strlen(str));
> > +             console_dev->console_puts(str, len);
> >       } else {
> > -             while (*str) {
> > -                     sbi_putc(*str);
> > -                     str++;
> > -             }
> > +             for (i = 0; i < len; i++)
> > +                     sbi_putc(str[i]);
> >       }
> > +}
> > +
> > +void sbi_puts(const char *str)
> > +{
> > +     spin_lock(&console_out_lock);
> > +     nputs(str, sbi_strlen(str));
> >       spin_unlock(&console_out_lock);
> >  }
> >
> >  void sbi_nputs(const char *str, unsigned long len)
> >  {
> > -     unsigned long i;
> > -
> >       spin_lock(&console_out_lock);
> > -     if (console_dev && console_dev->console_puts) {
> > -             console_dev->console_puts(str, len);
> > -     } else {
> > -             for (i = 0; i < len; i++)
> > -                     sbi_putc(str[i]);
> > -     }
> > +     nputs(str, len);
> >       spin_unlock(&console_out_lock);
> >  }
> >
> > @@ -102,6 +100,7 @@ unsigned long sbi_ngets(char *str, unsigned long len)
> >  #define PAD_ZERO 2
> >  #define PAD_ALTERNATE 4
> >  #define PRINT_BUF_LEN 64
> > +#define PRINT_TBUF_MAX 128
> >
> >  #define va_start(v, l) __builtin_va_start((v), l)
> >  #define va_end __builtin_va_end
> > @@ -217,12 +216,26 @@ static int printi(char **out, u32 *out_len, long long i, int b, int sg,
> >
> >  static int print(char **out, u32 *out_len, const char *format, va_list args)
> >  {
> > -     int width, flags;
> > -     int pc = 0;
> > -     char scr[2];
> > +     u32 tbuf_len;
> > +     int width, flags, pc = 0;
> > +     char scr[2], *tout, tbuf[PRINT_TBUF_MAX];
> > +     bool use_tbuf = (!out) ? true : false;
>
> print() is only called with a non-null 'out' parameter or under
> console_out_lock, so we can use global variable for the buffer
> to avoid putting it on the stack, and it can be larger.

Okay, I will update.

>
> >       unsigned long long tmp;
> >
> > +     if (use_tbuf) {
> > +             tbuf_len = PRINT_TBUF_MAX;
> > +             tout = tbuf;
> > +             out = &tout;
> > +             out_len = &tbuf_len;
> > +     }
> > +
> >       for (; *format != 0; ++format) {
> > +             if (use_tbuf && !tbuf_len) {
> > +                     nputs(tbuf, PRINT_TBUF_MAX);
> > +                     tbuf_len = PRINT_TBUF_MAX;
> > +                     tout = tbuf;
> > +             }
> > +
> >               if (*format == '%') {
> >                       ++format;
> >                       width = flags = 0;
> > @@ -348,6 +361,9 @@ literal:
> >               }
> >       }
> >
> > +     if (use_tbuf && tbuf_len < PRINT_TBUF_MAX)
> > +             nputs(tbuf, PRINT_TBUF_MAX - tbuf_len);
> > +
> >       return pc;
> >  }
> >
> > --
> > 2.34.1
> >
>
> Otherwise,
>
> Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
>
> Thanks,
> drew

Regards,
Anup


From apatel at ventanamicro.com  Thu Feb  9 21:46:44 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:44 +0530
Subject: [PATCH v6 0/8] OpenSBI debug console support
Message-ID: <20230210054652.495628-1-apatel@ventanamicro.com>

The latest proposal of SBI debug console extension can be found at:
https://lists.riscv.org/g/tech-prs/message/96

This series implements SBI debug console support in OpenSBI
as-per above draft proposal.

To test these patches corresponding Linux patches can be found in
riscv_sbi_dbcn_v1 branch at: https://github.com/avpatel/linux.git

These patches can also be found in riscv_sbi_dbcn_v6 branch at:
https://github.com/avpatel/opensbi.git

Changes since v5:
 - Rebased on latest OpenSBI sources
 - Updated Reviewed-by tags
 - Addressed Drew's comments in all patches
 - Updated SBI debug console implementation as-per latest proposal
   where console write() is non-blocking.

Changes since v4:
 - Rebased on latest OpenSBI sources
 - Updated Reviewed-by tags
 - Added new PATCH8 to speed-up sbi_printf() using nputs()

Changes since v3:
 - Rebased on OpenSBI v1.2 release
 - Updated Reviewed-by tags
 - Added new PATCH3 for sbi_ngets() function
 - Updated SBI debug console implementation in PATCH5 as-per latest
   proposal having both console write() and read() functions.

Changes since v2:
 - Reworked sbi_domain_check_addr_range() in PATCH3 to make it work
   for overlapping regions.

Changes since v1:
 - New PATCH3 to implement sbi_domain_check_addr_range() function
 - Added checks in PATCH4 for location and size of string to print
 - New PATCH5 to add console_puts() callback in the console device
 - New PATCH6 to implement console_puts() for semihosting

Anup Patel (8):
  include: Add defines for SBI debug console extension
  lib: sbi: Add sbi_nputs() function
  lib: sbi: Add sbi_ngets() function
  lib: sbi: Add sbi_domain_check_addr_range() function
  lib: sbi: Implement SBI debug console extension
  lib: sbi: Add console_puts() callback in the console device
  lib: utils/serial: Implement console_puts() for semihosting
  lib: sbi: Speed-up sbi_printf() and friends using nputs()

 include/sbi/sbi_console.h         |  7 +++
 include/sbi/sbi_domain.h          | 15 ++++++
 include/sbi/sbi_ecall_interface.h |  6 +++
 lib/sbi/Kconfig                   |  4 ++
 lib/sbi/objects.mk                |  3 ++
 lib/sbi/sbi_console.c             | 88 ++++++++++++++++++++++++++++---
 lib/sbi/sbi_domain.c              | 69 ++++++++++++++++++++++++
 lib/sbi/sbi_ecall_dbcn.c          | 72 +++++++++++++++++++++++++
 lib/utils/serial/semihosting.c    | 37 +++++++++++++
 9 files changed, 294 insertions(+), 7 deletions(-)
 create mode 100644 lib/sbi/sbi_ecall_dbcn.c

-- 
2.34.1



From apatel at ventanamicro.com  Thu Feb  9 21:46:45 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:45 +0530
Subject: [PATCH v6 1/8] include: Add defines for SBI debug console extension
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <20230210054652.495628-2-apatel@ventanamicro.com>

We add SBI debug console extension related defines to the
SBI ecall interface header.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Bin Meng <bmeng at tinylab.org>
Reviewed-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/sbi_ecall_interface.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index a3f2bf4..9d6f474 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -29,6 +29,7 @@
 #define SBI_EXT_HSM				0x48534D
 #define SBI_EXT_SRST				0x53525354
 #define SBI_EXT_PMU				0x504D55
+#define SBI_EXT_DBCN				0x4442434E
 
 /* SBI function IDs for BASE extension*/
 #define SBI_EXT_BASE_GET_SPEC_VERSION		0x0
@@ -230,6 +231,11 @@ enum sbi_pmu_ctr_type {
 /* Flags defined for counter stop function */
 #define SBI_PMU_STOP_FLAG_RESET (1 << 0)
 
+/* SBI function IDs for DBCN extension */
+#define SBI_EXT_DBCN_CONSOLE_WRITE		0x0
+#define SBI_EXT_DBCN_CONSOLE_READ		0x1
+#define SBI_EXT_DBCN_CONSOLE_WRITE_BYTE		0x2
+
 /* SBI base specification related macros */
 #define SBI_SPEC_VERSION_MAJOR_OFFSET		24
 #define SBI_SPEC_VERSION_MAJOR_MASK		0x7f
-- 
2.34.1



From apatel at ventanamicro.com  Thu Feb  9 21:46:46 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:46 +0530
Subject: [PATCH v6 2/8] lib: sbi: Add sbi_nputs() function
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <20230210054652.495628-3-apatel@ventanamicro.com>

We add new sbi_nputs() which help us print a fixed number of characters
from a physical memory location.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Bin Meng <bmeng at tinylab.org>
Reviewed-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/sbi_console.h |  2 ++
 lib/sbi/sbi_console.c     | 12 ++++++++++++
 2 files changed, 14 insertions(+)

diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
index e15b55d..dd6a905 100644
--- a/include/sbi/sbi_console.h
+++ b/include/sbi/sbi_console.h
@@ -33,6 +33,8 @@ void sbi_putc(char ch);
 
 void sbi_puts(const char *str);
 
+unsigned long sbi_nputs(const char *str, unsigned long len);
+
 void sbi_gets(char *s, int maxwidth, char endchar);
 
 int __printf(2, 3) sbi_sprintf(char *out, const char *format, ...);
diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index bb6e1ef..2f83a26 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -51,6 +51,18 @@ void sbi_puts(const char *str)
 	spin_unlock(&console_out_lock);
 }
 
+unsigned long sbi_nputs(const char *str, unsigned long len)
+{
+	unsigned long i;
+
+	spin_lock(&console_out_lock);
+	for (i = 0; i < len; i++)
+		sbi_putc(str[i]);
+	spin_unlock(&console_out_lock);
+
+	return i;
+}
+
 void sbi_gets(char *s, int maxwidth, char endchar)
 {
 	int ch;
-- 
2.34.1



From apatel at ventanamicro.com  Thu Feb  9 21:46:47 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:47 +0530
Subject: [PATCH v6 3/8] lib: sbi: Add sbi_ngets() function
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <20230210054652.495628-4-apatel@ventanamicro.com>

We add new sbi_ngets() which help us read characters into a
physical memory location.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
---
 include/sbi/sbi_console.h |  2 ++
 lib/sbi/sbi_console.c     | 15 +++++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
index dd6a905..3c1f5e5 100644
--- a/include/sbi/sbi_console.h
+++ b/include/sbi/sbi_console.h
@@ -37,6 +37,8 @@ unsigned long sbi_nputs(const char *str, unsigned long len);
 
 void sbi_gets(char *s, int maxwidth, char endchar);
 
+unsigned long sbi_ngets(char *str, unsigned long len);
+
 int __printf(2, 3) sbi_sprintf(char *out, const char *format, ...);
 
 int __printf(3, 4) sbi_snprintf(char *out, u32 out_sz, const char *format, ...);
diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index 2f83a26..c734bc0 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -76,6 +76,21 @@ void sbi_gets(char *s, int maxwidth, char endchar)
 	*retval = '\0';
 }
 
+unsigned long sbi_ngets(char *str, unsigned long len)
+{
+	int ch;
+	unsigned long i;
+
+	for (i = 0; i < len; i++) {
+		ch = sbi_getc();
+		if (ch < 0)
+			break;
+		str[i] = ch;
+	}
+
+	return i;
+}
+
 #define PAD_RIGHT 1
 #define PAD_ZERO 2
 #define PAD_ALTERNATE 4
-- 
2.34.1



From apatel at ventanamicro.com  Thu Feb  9 21:46:48 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:48 +0530
Subject: [PATCH v6 4/8] lib: sbi: Add sbi_domain_check_addr_range() function
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <20230210054652.495628-5-apatel@ventanamicro.com>

We add sbi_domain_check_addr_range() helper function to check
whether a given address range is accessible under a particular
domain.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Bin Meng <bmeng at tinylab.org>
Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
---
 include/sbi/sbi_domain.h | 15 +++++++++
 lib/sbi/sbi_domain.c     | 69 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 84 insertions(+)

diff --git a/include/sbi/sbi_domain.h b/include/sbi/sbi_domain.h
index bbb3eff..ab1a944 100644
--- a/include/sbi/sbi_domain.h
+++ b/include/sbi/sbi_domain.h
@@ -196,6 +196,21 @@ bool sbi_domain_check_addr(const struct sbi_domain *dom,
 			   unsigned long addr, unsigned long mode,
 			   unsigned long access_flags);
 
+/**
+ * Check whether we can access specified address range for given mode and
+ * memory region flags under a domain
+ * @param dom pointer to domain
+ * @param addr the start of the address range to be checked
+ * @param size the size of the address range to be checked
+ * @param mode the privilege mode of access
+ * @param access_flags bitmask of domain access types (enum sbi_domain_access)
+ * @return TRUE if access allowed otherwise FALSE
+ */
+bool sbi_domain_check_addr_range(const struct sbi_domain *dom,
+				 unsigned long addr, unsigned long size,
+				 unsigned long mode,
+				 unsigned long access_flags);
+
 /** Dump domain details on the console */
 void sbi_domain_dump(const struct sbi_domain *dom, const char *suffix);
 
diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
index 3813f1b..dc825b0 100644
--- a/lib/sbi/sbi_domain.c
+++ b/lib/sbi/sbi_domain.c
@@ -212,6 +212,44 @@ static bool is_region_before(const struct sbi_domain_memregion *regA,
 	return false;
 }
 
+static const struct sbi_domain_memregion *find_region(
+						const struct sbi_domain *dom,
+						unsigned long addr)
+{
+	unsigned long rstart, rend;
+	struct sbi_domain_memregion *reg;
+
+	sbi_domain_for_each_memregion(dom, reg) {
+		rstart = reg->base;
+		rend = (reg->order < __riscv_xlen) ?
+			rstart + ((1UL << reg->order) - 1) : -1UL;
+		if (rstart <= addr && addr <= rend)
+			return reg;
+	}
+
+	return NULL;
+}
+
+static const struct sbi_domain_memregion *find_next_subset_region(
+				const struct sbi_domain *dom,
+				const struct sbi_domain_memregion *reg,
+				unsigned long addr)
+{
+	struct sbi_domain_memregion *sreg, *ret = NULL;
+
+	sbi_domain_for_each_memregion(dom, sreg) {
+		if (sreg == reg || (sreg->base <= addr) ||
+		    !is_region_subset(sreg, reg))
+			continue;
+
+		if (!ret || (sreg->base < ret->base) ||
+		    ((sreg->base == ret->base) && (sreg->order < ret->order)))
+			ret = sreg;
+	}
+
+	return ret;
+}
+
 static int sanitize_domain(const struct sbi_platform *plat,
 			   struct sbi_domain *dom)
 {
@@ -320,6 +358,37 @@ static int sanitize_domain(const struct sbi_platform *plat,
 	return 0;
 }
 
+bool sbi_domain_check_addr_range(const struct sbi_domain *dom,
+				 unsigned long addr, unsigned long size,
+				 unsigned long mode,
+				 unsigned long access_flags)
+{
+	unsigned long max = addr + size;
+	const struct sbi_domain_memregion *reg, *sreg;
+
+	if (!dom)
+		return false;
+
+	while (addr < max) {
+		reg = find_region(dom, addr);
+		if (!reg)
+			return false;
+
+		if (!sbi_domain_check_addr(dom, addr, mode, access_flags))
+			return false;
+
+		sreg = find_next_subset_region(dom, reg, addr);
+		if (sreg)
+			addr = sreg->base;
+		else if (reg->order < __riscv_xlen)
+			addr = reg->base + (1UL << reg->order);
+		else
+			break;
+	}
+
+	return true;
+}
+
 void sbi_domain_dump(const struct sbi_domain *dom, const char *suffix)
 {
 	u32 i, k;
-- 
2.34.1



From apatel at ventanamicro.com  Thu Feb  9 21:46:49 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:49 +0530
Subject: [PATCH v6 5/8] lib: sbi: Implement SBI debug console extension
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <20230210054652.495628-6-apatel@ventanamicro.com>

We implement SBI debug console extension as one of the replacement
SBI extensions. This extension is only available when OpenSBI platform
provides a console device to generic library.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Bin Meng <bmeng at tinylab.org>
---
 lib/sbi/Kconfig          |  4 +++
 lib/sbi/objects.mk       |  3 ++
 lib/sbi/sbi_ecall_dbcn.c | 72 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 79 insertions(+)
 create mode 100644 lib/sbi/sbi_ecall_dbcn.c

diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
index df74bba..ef6728b 100644
--- a/lib/sbi/Kconfig
+++ b/lib/sbi/Kconfig
@@ -26,6 +26,10 @@ config SBI_ECALL_PMU
 	bool "Performance Monitoring Unit extension"
 	default y
 
+config SBI_ECALL_DBCN
+	bool "Debug Console extension"
+	default y
+
 config SBI_ECALL_LEGACY
 	bool "SBI v0.1 legacy extensions"
 	default y
diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
index c774ebb..319f38d 100644
--- a/lib/sbi/objects.mk
+++ b/lib/sbi/objects.mk
@@ -37,6 +37,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
 carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
 libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
 
+carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_DBCN) += ecall_dbcn
+libsbi-objs-$(CONFIG_SBI_ECALL_DBCN) += sbi_ecall_dbcn.o
+
 carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_LEGACY) += ecall_legacy
 libsbi-objs-$(CONFIG_SBI_ECALL_LEGACY) += sbi_ecall_legacy.o
 
diff --git a/lib/sbi/sbi_ecall_dbcn.c b/lib/sbi/sbi_ecall_dbcn.c
new file mode 100644
index 0000000..fe7e175
--- /dev/null
+++ b/lib/sbi/sbi_ecall_dbcn.c
@@ -0,0 +1,72 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 Ventana Micro Systems Inc.
+ *
+ * Authors:
+ *   Anup Patel <apatel at ventanamicro.com>
+ */
+
+#include <sbi/sbi_console.h>
+#include <sbi/sbi_domain.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_ecall.h>
+#include <sbi/sbi_ecall_interface.h>
+#include <sbi/sbi_trap.h>
+#include <sbi/riscv_asm.h>
+
+static int sbi_ecall_dbcn_handler(unsigned long extid, unsigned long funcid,
+				  const struct sbi_trap_regs *regs,
+				  unsigned long *out_val,
+				  struct sbi_trap_info *out_trap)
+{
+	ulong smode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >>
+			MSTATUS_MPP_SHIFT;
+
+	switch (funcid) {
+	case SBI_EXT_DBCN_CONSOLE_WRITE:
+	case SBI_EXT_DBCN_CONSOLE_READ:
+		/*
+		 * On RV32, the M-mode can only access the first 4GB of
+		 * the physical address space because M-mode does not have
+		 * MMU to access full 34-bit physical address space.
+		 *
+		 * Based on above, we simply fail if the upper 32bits of
+		 * the physical address (i.e. a2 register) is non-zero on
+		 * RV32.
+		 */
+#if __riscv_xlen == 32
+		if (regs->a2)
+			return SBI_ERR_FAILED;
+#endif
+		if (!sbi_domain_check_addr_range(sbi_domain_thishart_ptr(),
+					regs->a1, regs->a0, smode,
+					SBI_DOMAIN_READ|SBI_DOMAIN_WRITE))
+			return SBI_ERR_INVALID_PARAM;
+		if (funcid == SBI_EXT_DBCN_CONSOLE_WRITE)
+			*out_val = sbi_nputs((const char *)regs->a1, regs->a0);
+		else
+			*out_val = sbi_ngets((char *)regs->a1, regs->a0);
+		return 0;
+	case SBI_EXT_DBCN_CONSOLE_WRITE_BYTE:
+		sbi_putc(regs->a0);
+		return 0;
+	default:
+		break;
+	}
+
+	return SBI_ENOTSUPP;
+}
+
+static int sbi_ecall_dbcn_probe(unsigned long extid, unsigned long *out_val)
+{
+	*out_val = sbi_console_get_device() ? 1 : 0;
+	return 0;
+}
+
+struct sbi_ecall_extension ecall_dbcn = {
+	.extid_start = SBI_EXT_DBCN,
+	.extid_end = SBI_EXT_DBCN,
+	.handle = sbi_ecall_dbcn_handler,
+	.probe = sbi_ecall_dbcn_probe,
+};
-- 
2.34.1



From apatel at ventanamicro.com  Thu Feb  9 21:46:50 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:50 +0530
Subject: [PATCH v6 6/8] lib: sbi: Add console_puts() callback in the console device
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <20230210054652.495628-7-apatel@ventanamicro.com>

We add console_puts() callback in the console device which allows
console drivers (such as semihosting) to implement a specialized
way to output character string.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Bin Meng <bmeng at tinylab.org>
Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
---
 include/sbi/sbi_console.h |  3 +++
 lib/sbi/sbi_console.c     | 29 ++++++++++++++++++++++-------
 2 files changed, 25 insertions(+), 7 deletions(-)

diff --git a/include/sbi/sbi_console.h b/include/sbi/sbi_console.h
index 3c1f5e5..0979765 100644
--- a/include/sbi/sbi_console.h
+++ b/include/sbi/sbi_console.h
@@ -19,6 +19,9 @@ struct sbi_console_device {
 	/** Write a character to the console output */
 	void (*console_putc)(char ch);
 
+	/** Write a character string to the console output */
+	unsigned long (*console_puts)(const char *str, unsigned long len);
+
 	/** Read a character from the console input */
 	int (*console_getc)(void);
 };
diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index c734bc0..8a5d9de 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -12,6 +12,7 @@
 #include <sbi/sbi_hart.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_scratch.h>
+#include <sbi/sbi_string.h>
 
 static const struct sbi_console_device *console_dev = NULL;
 static spinlock_t console_out_lock	       = SPIN_LOCK_INITIALIZER;
@@ -43,24 +44,38 @@ void sbi_putc(char ch)
 
 void sbi_puts(const char *str)
 {
+	unsigned long p, len;
+
 	spin_lock(&console_out_lock);
-	while (*str) {
-		sbi_putc(*str);
-		str++;
+	if (console_dev && console_dev->console_puts) {
+		p = 0;
+		len = sbi_strlen(str);
+		while (p < len)
+			p += console_dev->console_puts(&str[p], len - p);
+	} else {
+		while (*str) {
+			sbi_putc(*str);
+			str++;
+		}
 	}
 	spin_unlock(&console_out_lock);
 }
 
 unsigned long sbi_nputs(const char *str, unsigned long len)
 {
-	unsigned long i;
+	unsigned long i, ret;
 
 	spin_lock(&console_out_lock);
-	for (i = 0; i < len; i++)
-		sbi_putc(str[i]);
+	if (console_dev && console_dev->console_puts) {
+		ret = console_dev->console_puts(str, len);
+	} else {
+		for (i = 0; i < len; i++)
+			sbi_putc(str[i]);
+		ret = len;
+	}
 	spin_unlock(&console_out_lock);
 
-	return i;
+	return ret;
 }
 
 void sbi_gets(char *s, int maxwidth, char endchar)
-- 
2.34.1



From apatel at ventanamicro.com  Thu Feb  9 21:46:51 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:51 +0530
Subject: [PATCH v6 7/8] lib: utils/serial: Implement console_puts() for semihosting
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <20230210054652.495628-8-apatel@ventanamicro.com>

We implement console_puts() for semihosting serial driver to speed-up
semihosting based prints.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
Reviewed-by: Atish Patra <atishp at rivosinc.com>
Reviewed-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Bin Meng <bmeng at tinylab.org>
Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
---
 lib/utils/serial/semihosting.c | 37 ++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/lib/utils/serial/semihosting.c b/lib/utils/serial/semihosting.c
index 86fa296..ce65887 100644
--- a/lib/utils/serial/semihosting.c
+++ b/lib/utils/serial/semihosting.c
@@ -15,6 +15,7 @@
 
 #define SYSOPEN     0x01
 #define SYSWRITEC   0x03
+#define SYSWRITE    0x05
 #define SYSREAD     0x06
 #define SYSREADC    0x07
 #define SYSERRNO	0x13
@@ -93,6 +94,7 @@ static int semihosting_errno(void)
 }
 
 static int semihosting_infd = SBI_ENODEV;
+static int semihosting_outfd = SBI_ENODEV;
 
 static long semihosting_open(const char *fname, enum semihosting_open_mode mode)
 {
@@ -141,6 +143,21 @@ static long semihosting_read(long fd, void *memp, size_t len)
 	return len - ret;
 }
 
+static long semihosting_write(long fd, const void *memp, size_t len)
+{
+	long ret;
+	struct semihosting_rdwr_s write;
+
+	write.fd = fd;
+	write.memp = (void *)memp;
+	write.len = len;
+
+	ret = semihosting_trap(SYSWRITE, &write);
+	if (ret < 0)
+		return semihosting_errno();
+	return len - ret;
+}
+
 /* clang-format on */
 
 static void semihosting_putc(char ch)
@@ -148,6 +165,24 @@ static void semihosting_putc(char ch)
 	semihosting_trap(SYSWRITEC, &ch);
 }
 
+static unsigned long semihosting_puts(const char *str, unsigned long len)
+{
+	char ch;
+	long ret;
+	unsigned long i;
+
+	if (semihosting_outfd < 0) {
+		for (i = 0; i < len; i++) {
+			ch = str[i];
+			semihosting_trap(SYSWRITEC, &ch);
+		}
+		ret = len;
+	} else
+		ret = semihosting_write(semihosting_outfd, str, len);
+
+	return (ret < 0) ? 0 : ret;
+}
+
 static int semihosting_getc(void)
 {
 	char ch = 0;
@@ -165,12 +200,14 @@ static int semihosting_getc(void)
 static struct sbi_console_device semihosting_console = {
 	.name = "semihosting",
 	.console_putc = semihosting_putc,
+	.console_puts = semihosting_puts,
 	.console_getc = semihosting_getc
 };
 
 int semihosting_init(void)
 {
 	semihosting_infd = semihosting_open(":tt", MODE_READ);
+	semihosting_outfd = semihosting_open(":tt", MODE_WRITE);
 
 	sbi_console_set_device(&semihosting_console);
 
-- 
2.34.1



From apatel at ventanamicro.com  Thu Feb  9 21:46:52 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 11:16:52 +0530
Subject: [PATCH v6 8/8] lib: sbi: Speed-up sbi_printf() and friends using nputs()
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <20230210054652.495628-9-apatel@ventanamicro.com>

The sbi_printf() is slow for semihosting because it prints one
character at a time. To speed-up sbi_printf() for semihosting,
we use a temporary buffer and nputs().

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
---
 lib/sbi/sbi_console.c | 76 ++++++++++++++++++++++++++++++-------------
 1 file changed, 54 insertions(+), 22 deletions(-)

diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index 8a5d9de..f3ac003 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -14,7 +14,11 @@
 #include <sbi/sbi_scratch.h>
 #include <sbi/sbi_string.h>
 
+#define CONSOLE_TBUF_MAX 256
+
 static const struct sbi_console_device *console_dev = NULL;
+static char console_tbuf[CONSOLE_TBUF_MAX];
+static u32 console_tbuf_len;
 static spinlock_t console_out_lock	       = SPIN_LOCK_INITIALIZER;
 
 bool sbi_isprintable(char c)
@@ -42,37 +46,44 @@ void sbi_putc(char ch)
 	}
 }
 
-void sbi_puts(const char *str)
+static unsigned long nputs(const char *str, unsigned long len)
 {
-	unsigned long p, len;
+	unsigned long i, ret;
 
-	spin_lock(&console_out_lock);
 	if (console_dev && console_dev->console_puts) {
-		p = 0;
-		len = sbi_strlen(str);
-		while (p < len)
-			p += console_dev->console_puts(&str[p], len - p);
+		ret = console_dev->console_puts(str, len);
 	} else {
-		while (*str) {
-			sbi_putc(*str);
-			str++;
-		}
+		for (i = 0; i < len; i++)
+			sbi_putc(str[i]);
+		ret = len;
 	}
+
+	return ret;
+}
+
+static void nputs_all(const char *str, unsigned long len)
+{
+	unsigned long p = 0;
+
+	while (p < len)
+		p += nputs(&str[p], len - p);
+}
+
+void sbi_puts(const char *str)
+{
+	unsigned long len = sbi_strlen(str);
+
+	spin_lock(&console_out_lock);
+	nputs_all(str, len);
 	spin_unlock(&console_out_lock);
 }
 
 unsigned long sbi_nputs(const char *str, unsigned long len)
 {
-	unsigned long i, ret;
+	unsigned long ret;
 
 	spin_lock(&console_out_lock);
-	if (console_dev && console_dev->console_puts) {
-		ret = console_dev->console_puts(str, len);
-	} else {
-		for (i = 0; i < len; i++)
-			sbi_putc(str[i]);
-		ret = len;
-	}
+	ret = nputs(str, len);
 	spin_unlock(&console_out_lock);
 
 	return ret;
@@ -225,12 +236,30 @@ static int printi(char **out, u32 *out_len, long long i, int b, int sg,
 
 static int print(char **out, u32 *out_len, const char *format, va_list args)
 {
-	int width, flags;
-	int pc = 0;
-	char scr[2];
+	int width, flags, pc = 0;
+	char scr[2], *tout;
+	bool use_tbuf = (!out) ? true : false;
 	unsigned long long tmp;
 
+	/*
+	 * The console_tbuf is protected by console_out_lock and
+	 * print() is always called with console_out_lock held
+	 * when out == NULL.
+	 */
+	if (use_tbuf) {
+		console_tbuf_len = CONSOLE_TBUF_MAX;
+		tout = console_tbuf;
+		out = &tout;
+		out_len = &console_tbuf_len;
+	}
+
 	for (; *format != 0; ++format) {
+		if (use_tbuf && !console_tbuf_len) {
+			nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
+			console_tbuf_len = CONSOLE_TBUF_MAX;
+			tout = console_tbuf;
+		}
+
 		if (*format == '%') {
 			++format;
 			width = flags = 0;
@@ -356,6 +385,9 @@ literal:
 		}
 	}
 
+	if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
+		nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
+
 	return pc;
 }
 
-- 
2.34.1



From anup at brainfault.org  Thu Feb  9 23:11:45 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 10 Feb 2023 12:41:45 +0530
Subject: [PATCH] include: sbi: Fix BSWAPx() macros for big-endian host
In-Reply-To: <20230209040251.451328-1-apatel@ventanamicro.com>
References: <20230209040251.451328-1-apatel@ventanamicro.com>
Message-ID: <CAAhSdy0z0qEz9BwapO+GYdj5Q5u4iShaA1eyhkR_gfjVUoB3sA@mail.gmail.com>

On Thu, Feb 9, 2023 at 9:33 AM Anup Patel <apatel at ventanamicro.com> wrote:
>
> The BSWAPx() macros won't do any swapping for big-endian host
> because the EXTRACT_BYTE() macro will pickup bytes in reverse
> order. Also, the EXTRACT_BYTE() will generate compile error
> for constants.
>
> To fix this, we get remove the EXTRACT_BYTE() macro and re-write
> BSWAPx() using simple mask and shift operations.
>
> Fixes: 09b34d8cca51 ("include: Add support for byteorder/endianness
> conversion")
> Reported-by: Samuel Holland <samuel at sholland.org>
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>

Applied this patch to the riscv/opensbi repo

Regards,
Anup

> ---
>  include/sbi/sbi_byteorder.h | 24 ++++++++++++++----------
>  1 file changed, 14 insertions(+), 10 deletions(-)
>
> diff --git a/include/sbi/sbi_byteorder.h b/include/sbi/sbi_byteorder.h
> index 680710f..15107e1 100644
> --- a/include/sbi/sbi_byteorder.h
> +++ b/include/sbi/sbi_byteorder.h
> @@ -9,16 +9,20 @@
>
>  #include <sbi/sbi_types.h>
>
> -#define EXTRACT_BYTE(x, n)     ((unsigned long long)((uint8_t *)&x)[n])
> -
> -#define BSWAP16(x)     ((EXTRACT_BYTE(x, 0) << 8) | EXTRACT_BYTE(x, 1))
> -#define BSWAP32(x)     ((EXTRACT_BYTE(x, 0) << 24) | (EXTRACT_BYTE(x, 1) << 16) | \
> -                        (EXTRACT_BYTE(x, 2) << 8) | EXTRACT_BYTE(x, 3))
> -#define BSWAP64(x)     ((EXTRACT_BYTE(x, 0) << 56) | (EXTRACT_BYTE(x, 1) << 48) | \
> -                        (EXTRACT_BYTE(x, 2) << 40) | (EXTRACT_BYTE(x, 3) << 32) | \
> -                        (EXTRACT_BYTE(x, 4) << 24) | (EXTRACT_BYTE(x, 5) << 16) | \
> -                        (EXTRACT_BYTE(x, 6) << 8) | EXTRACT_BYTE(x, 7))
> -
> +#define BSWAP16(x)     ((((x) & 0x00ff) << 8) | \
> +                        (((x) & 0xff00) >> 8))
> +#define BSWAP32(x)     ((((x) & 0x000000ff) << 24) | \
> +                        (((x) & 0x0000ff00) << 8) | \
> +                        (((x) & 0x00ff0000) >> 8) | \
> +                        (((x) & 0xff000000) >> 24))
> +#define BSWAP64(x)     ((((x) & 0x00000000000000ffULL) << 56) | \
> +                        (((x) & 0x000000000000ff00ULL) << 40) | \
> +                        (((x) & 0x0000000000ff0000ULL) << 24) | \
> +                        (((x) & 0x00000000ff000000ULL) << 8) | \
> +                        (((x) & 0x000000ff00000000ULL) >> 8) | \
> +                        (((x) & 0x0000ff0000000000ULL) >> 24) | \
> +                        (((x) & 0x00ff000000000000ULL) >> 40) | \
> +                        (((x) & 0xff00000000000000ULL) >> 56))
>
>  #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__  /* CPU(little-endian) */
>  #define cpu_to_be16(x)         ((uint16_t)BSWAP16(x))
> --
> 2.34.1
>


From apatel at ventanamicro.com  Thu Feb  9 23:20:49 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 10 Feb 2023 12:50:49 +0530
Subject: [PATCH v6 0/8] OpenSBI debug console support
In-Reply-To: <20230210054652.495628-1-apatel@ventanamicro.com>
References: <20230210054652.495628-1-apatel@ventanamicro.com>
Message-ID: <CAK9=C2WKqySwZJoB9kgNqenQ42AAeLkz=uGuraAa5Pif33u1TA@mail.gmail.com>

On Fri, Feb 10, 2023 at 11:17 AM Anup Patel <apatel at ventanamicro.com> wrote:
>
> The latest proposal of SBI debug console extension can be found at:
> https://lists.riscv.org/g/tech-prs/message/96
>
> This series implements SBI debug console support in OpenSBI
> as-per above draft proposal.
>
> To test these patches corresponding Linux patches can be found in
> riscv_sbi_dbcn_v1 branch at: https://github.com/avpatel/linux.git
>
> These patches can also be found in riscv_sbi_dbcn_v6 branch at:
> https://github.com/avpatel/opensbi.git
>
> Changes since v5:
>  - Rebased on latest OpenSBI sources
>  - Updated Reviewed-by tags
>  - Addressed Drew's comments in all patches
>  - Updated SBI debug console implementation as-per latest proposal
>    where console write() is non-blocking.
>
> Changes since v4:
>  - Rebased on latest OpenSBI sources
>  - Updated Reviewed-by tags
>  - Added new PATCH8 to speed-up sbi_printf() using nputs()
>
> Changes since v3:
>  - Rebased on OpenSBI v1.2 release
>  - Updated Reviewed-by tags
>  - Added new PATCH3 for sbi_ngets() function
>  - Updated SBI debug console implementation in PATCH5 as-per latest
>    proposal having both console write() and read() functions.
>
> Changes since v2:
>  - Reworked sbi_domain_check_addr_range() in PATCH3 to make it work
>    for overlapping regions.
>
> Changes since v1:
>  - New PATCH3 to implement sbi_domain_check_addr_range() function
>  - Added checks in PATCH4 for location and size of string to print
>  - New PATCH5 to add console_puts() callback in the console device
>  - New PATCH6 to implement console_puts() for semihosting
>
> Anup Patel (8):
>   include: Add defines for SBI debug console extension
>   lib: sbi: Add sbi_nputs() function
>   lib: sbi: Add sbi_ngets() function
>   lib: sbi: Add sbi_domain_check_addr_range() function
>   lib: sbi: Implement SBI debug console extension
>   lib: sbi: Add console_puts() callback in the console device
>   lib: utils/serial: Implement console_puts() for semihosting
>   lib: sbi: Speed-up sbi_printf() and friends using nputs()

Applied this series to the riscv/opensbi repo.

Regards,
Anup

>
>  include/sbi/sbi_console.h         |  7 +++
>  include/sbi/sbi_domain.h          | 15 ++++++
>  include/sbi/sbi_ecall_interface.h |  6 +++
>  lib/sbi/Kconfig                   |  4 ++
>  lib/sbi/objects.mk                |  3 ++
>  lib/sbi/sbi_console.c             | 88 ++++++++++++++++++++++++++++---
>  lib/sbi/sbi_domain.c              | 69 ++++++++++++++++++++++++
>  lib/sbi/sbi_ecall_dbcn.c          | 72 +++++++++++++++++++++++++
>  lib/utils/serial/semihosting.c    | 37 +++++++++++++
>  9 files changed, 294 insertions(+), 7 deletions(-)
>  create mode 100644 lib/sbi/sbi_ecall_dbcn.c
>
> --
> 2.34.1
>


From nylon.chen at sifive.com  Fri Feb 10 00:52:38 2023
From: nylon.chen at sifive.com (Nylon Chen)
Date: Fri, 10 Feb 2023 16:52:38 +0800
Subject: [PATCH] lib: sbi_hart: Enable hcontext and scontext
Message-ID: <20230210085238.19723-1-nylon.chen@sifive.com>

According to the description in "riscv-state-enable[0]", to access
h/scontext in S-Mode, we need to enable the 57th bit.

If it is not enabled, an "illegal instruction" error will occur.

Link: https://github.com/riscv/riscv-state-enable/blob/a28bfae443f350d5b4c42874f428367d5b322ffe/content.adoc [0]

Signed-off-by: Nylon Chen <nylon.chen at sifive.com>
Reviewed-by: Zong Li <zong.li at sifive.com>
---
 include/sbi/riscv_encoding.h | 2 ++
 lib/sbi/sbi_hart.c           | 1 +
 2 files changed, 3 insertions(+)

diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
index b0f08c8..4ebed97 100644
--- a/include/sbi/riscv_encoding.h
+++ b/include/sbi/riscv_encoding.h
@@ -736,6 +736,8 @@
 #define SMSTATEEN0_CS			(_ULL(1) << SMSTATEEN0_CS_SHIFT)
 #define SMSTATEEN0_FCSR_SHIFT		1
 #define SMSTATEEN0_FCSR			(_ULL(1) << SMSTATEEN0_FCSR_SHIFT)
+#define SMSTATEEN0_CONTEXT_SHIFT	57
+#define SMSTATEEN0_CONTEXT		(_ULL(1) << SMSTATEEN0_CONTEXT_SHIFT)
 #define SMSTATEEN0_IMSIC_SHIFT		58
 #define SMSTATEEN0_IMSIC		(_ULL(1) << SMSTATEEN0_IMSIC_SHIFT)
 #define SMSTATEEN0_AIA_SHIFT		59
diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index 02ce991..5e06918 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -90,6 +90,7 @@ static void mstatus_init(struct sbi_scratch *scratch)
 		mstateen_val |= ((uint64_t)csr_read(CSR_MSTATEEN0H)) << 32;
 #endif
 		mstateen_val |= SMSTATEEN_STATEN;
+		mstateen_val |= SMSTATEEN0_CONTEXT;
 		mstateen_val |= SMSTATEEN0_HSENVCFG;
 
 		if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA))
-- 
2.36.1



From wxjstz at 126.com  Sat Feb 11 04:52:00 2023
From: wxjstz at 126.com (Xiang W)
Date: Sat, 11 Feb 2023 20:52:00 +0800
Subject: [PATCH] lib: sbi: fix missing '\r' for console
Message-ID: <20230211125200.10381-1-wxjstz@126.com>

print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
will add a \r before the output \n. This patch adds missing \r when
outputting characters via console_dev->puts.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_console.c | 46 +++++++++++++++++++++++++++++--------------
 1 file changed, 31 insertions(+), 15 deletions(-)

diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index f3ac003..7dc053a 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -46,27 +46,43 @@ void sbi_putc(char ch)
 	}
 }
 
+static unsigned long console_puts_all(const char *str, unsigned long len)
+{
+	const char *s = str;
+	const char *e = s + len;
+	while (s < e)
+		s += console_dev->console_puts(s, e - s);
+	return len;
+}
+
+static unsigned long console_puts(const char *str, unsigned long len)
+{
+	const char *s, *p, *e;
+	s = str;
+	e = str + len;
+	while (s < e) {
+		p = sbi_strchr(s, '\n');
+		p = p ? p : e;
+		console_puts_all(s, p - s);
+		if (*p == '\n')
+			console_puts_all("\r\n", 2);
+		s = p + 1;
+	}
+	return len;
+}
+
 static unsigned long nputs(const char *str, unsigned long len)
 {
-	unsigned long i, ret;
+	unsigned long i;
 
 	if (console_dev && console_dev->console_puts) {
-		ret = console_dev->console_puts(str, len);
+		console_puts(str, len);
 	} else {
 		for (i = 0; i < len; i++)
 			sbi_putc(str[i]);
-		ret = len;
 	}
 
-	return ret;
-}
-
-static void nputs_all(const char *str, unsigned long len)
-{
-	unsigned long p = 0;
-
-	while (p < len)
-		p += nputs(&str[p], len - p);
+	return len;
 }
 
 void sbi_puts(const char *str)
@@ -74,7 +90,7 @@ void sbi_puts(const char *str)
 	unsigned long len = sbi_strlen(str);
 
 	spin_lock(&console_out_lock);
-	nputs_all(str, len);
+	nputs(str, len);
 	spin_unlock(&console_out_lock);
 }
 
@@ -255,7 +271,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
 
 	for (; *format != 0; ++format) {
 		if (use_tbuf && !console_tbuf_len) {
-			nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
+			nputs(console_tbuf, CONSOLE_TBUF_MAX);
 			console_tbuf_len = CONSOLE_TBUF_MAX;
 			tout = console_tbuf;
 		}
@@ -386,7 +402,7 @@ literal:
 	}
 
 	if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
-		nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
+		nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
 
 	return pc;
 }
-- 
2.39.1



From samuel at sholland.org  Sun Feb 12 10:49:24 2023
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Feb 2023 12:49:24 -0600
Subject: [PATCH] lib: sbi: fix missing '\r' for console
In-Reply-To: <20230211125200.10381-1-wxjstz@126.com>
References: <20230211125200.10381-1-wxjstz@126.com>
Message-ID: <0ff77dd9-fefd-7239-ef68-1f6a17effa01@sholland.org>

On 2/11/23 06:52, Xiang W wrote:
> print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> will add a \r before the output \n. This patch adds missing \r when
> outputting characters via console_dev->puts.
> 
> Signed-off-by: Xiang W <wxjstz at 126.com>
> ---
>  lib/sbi/sbi_console.c | 46 +++++++++++++++++++++++++++++--------------
>  1 file changed, 31 insertions(+), 15 deletions(-)
> 
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index f3ac003..7dc053a 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -46,27 +46,43 @@ void sbi_putc(char ch)
>  	}
>  }
>  
> +static unsigned long console_puts_all(const char *str, unsigned long len)
> +{
> +	const char *s = str;
> +	const char *e = s + len;
> +	while (s < e)
> +		s += console_dev->console_puts(s, e - s);
> +	return len;
> +}
> +
> +static unsigned long console_puts(const char *str, unsigned long len)
> +{
> +	const char *s, *p, *e;
> +	s = str;
> +	e = str + len;
> +	while (s < e) {
> +		p = sbi_strchr(s, '\n');
> +		p = p ? p : e;
> +		console_puts_all(s, p - s);
> +		if (*p == '\n')

If p == e, it is past the end of the initialized data, so you cannot
dereference it. A better check would be:

	if (p != e)

Regards,
Samuel

> +			console_puts_all("\r\n", 2);
> +		s = p + 1;
> +	}
> +	return len;
> +}
> +
>  static unsigned long nputs(const char *str, unsigned long len)
>  {
> -	unsigned long i, ret;
> +	unsigned long i;
>  
>  	if (console_dev && console_dev->console_puts) {
> -		ret = console_dev->console_puts(str, len);
> +		console_puts(str, len);
>  	} else {
>  		for (i = 0; i < len; i++)
>  			sbi_putc(str[i]);
> -		ret = len;
>  	}
>  
> -	return ret;
> -}
> -
> -static void nputs_all(const char *str, unsigned long len)
> -{
> -	unsigned long p = 0;
> -
> -	while (p < len)
> -		p += nputs(&str[p], len - p);
> +	return len;
>  }
>  
>  void sbi_puts(const char *str)
> @@ -74,7 +90,7 @@ void sbi_puts(const char *str)
>  	unsigned long len = sbi_strlen(str);
>  
>  	spin_lock(&console_out_lock);
> -	nputs_all(str, len);
> +	nputs(str, len);
>  	spin_unlock(&console_out_lock);
>  }
>  
> @@ -255,7 +271,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
>  
>  	for (; *format != 0; ++format) {
>  		if (use_tbuf && !console_tbuf_len) {
> -			nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
> +			nputs(console_tbuf, CONSOLE_TBUF_MAX);
>  			console_tbuf_len = CONSOLE_TBUF_MAX;
>  			tout = console_tbuf;
>  		}
> @@ -386,7 +402,7 @@ literal:
>  	}
>  
>  	if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
> -		nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> +		nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
>  
>  	return pc;
>  }



From wxjstz at 126.com  Sun Feb 12 18:11:09 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 13 Feb 2023 10:11:09 +0800
Subject: [PATCH] lib: sbi: fix missing '\r' for console
In-Reply-To: <0ff77dd9-fefd-7239-ef68-1f6a17effa01@sholland.org>
References: <20230211125200.10381-1-wxjstz@126.com>
	 <0ff77dd9-fefd-7239-ef68-1f6a17effa01@sholland.org>
Message-ID: <386d1955d4a4479f6ac1c0211191b7ad8cbe09b2.camel@126.com>

? 2023-02-12???? 12:49 -0600?Samuel Holland???
> On 2/11/23 06:52, Xiang W wrote:
> > print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> > will add a \r before the output \n. This patch adds missing \r when
> > outputting characters via console_dev->puts.
> > 
> > Signed-off-by: Xiang W <wxjstz at 126.com>
> > ---
> > ?lib/sbi/sbi_console.c | 46 +++++++++++++++++++++++++++++--------------
> > ?1 file changed, 31 insertions(+), 15 deletions(-)
> > 
> > diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> > index f3ac003..7dc053a 100644
> > --- a/lib/sbi/sbi_console.c
> > +++ b/lib/sbi/sbi_console.c
> > @@ -46,27 +46,43 @@ void sbi_putc(char ch)
> > ????????}
> > ?}
> > ?
> > +static unsigned long console_puts_all(const char *str, unsigned long len)
> > +{
> > +???????const char *s = str;
> > +???????const char *e = s + len;
> > +???????while (s < e)
> > +???????????????s += console_dev->console_puts(s, e - s);
> > +???????return len;
> > +}
> > +
> > +static unsigned long console_puts(const char *str, unsigned long len)
> > +{
> > +???????const char *s, *p, *e;
> > +???????s = str;
> > +???????e = str + len;
> > +???????while (s < e) {
> > +???????????????p = sbi_strchr(s, '\n');
> > +???????????????p = p ? p : e;
> > +???????????????console_puts_all(s, p - s);
> > +???????????????if (*p == '\n')
> 
> If p == e, it is past the end of the initialized data, so you cannot
> dereference it. A better check would be:
> 
> ????????if (p != e)
> 
> Regards,
> Samuel
Thank you! Good catch!

Regards,
Xiang W
> 
> > +???????????????????????console_puts_all("\r\n", 2);
> > +???????????????s = p + 1;
> > +???????}
> > +???????return len;
> > +}
> > +
> > ?static unsigned long nputs(const char *str, unsigned long len)
> > ?{
> > -???????unsigned long i, ret;
> > +???????unsigned long i;
> > ?
> > ????????if (console_dev && console_dev->console_puts) {
> > -???????????????ret = console_dev->console_puts(str, len);
> > +???????????????console_puts(str, len);
> > ????????} else {
> > ????????????????for (i = 0; i < len; i++)
> > ????????????????????????sbi_putc(str[i]);
> > -???????????????ret = len;
> > ????????}
> > ?
> > -???????return ret;
> > -}
> > -
> > -static void nputs_all(const char *str, unsigned long len)
> > -{
> > -???????unsigned long p = 0;
> > -
> > -???????while (p < len)
> > -???????????????p += nputs(&str[p], len - p);
> > +???????return len;
> > ?}
> > ?
> > ?void sbi_puts(const char *str)
> > @@ -74,7 +90,7 @@ void sbi_puts(const char *str)
> > ????????unsigned long len = sbi_strlen(str);
> > ?
> > ????????spin_lock(&console_out_lock);
> > -???????nputs_all(str, len);
> > +???????nputs(str, len);
> > ????????spin_unlock(&console_out_lock);
> > ?}
> > ?
> > @@ -255,7 +271,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
> > ?
> > ????????for (; *format != 0; ++format) {
> > ????????????????if (use_tbuf && !console_tbuf_len) {
> > -???????????????????????nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
> > +???????????????????????nputs(console_tbuf, CONSOLE_TBUF_MAX);
> > ????????????????????????console_tbuf_len = CONSOLE_TBUF_MAX;
> > ????????????????????????tout = console_tbuf;
> > ????????????????}
> > @@ -386,7 +402,7 @@ literal:
> > ????????}
> > ?
> > ????????if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
> > -???????????????nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> > +???????????????nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> > ?
> > ????????return pc;
> > ?}


From wxjstz at 126.com  Sun Feb 12 18:08:21 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 13 Feb 2023 10:08:21 +0800
Subject: [PATCH v2] lib: sbi: fix missing '\r' for console
Message-ID: <20230213020821.109087-1-wxjstz@126.com>

print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
will add a \r before the output \n. This patch adds missing \r when
outputting characters via console_dev->puts.

Signed-off-by: Xiang W <wxjstz at 126.com>

Changes since v2:
- Fix the bug reported by Samuel. Prevent p from accessing memory other
  than strings.
---
 lib/sbi/sbi_console.c | 46 +++++++++++++++++++++++++++++--------------
 1 file changed, 31 insertions(+), 15 deletions(-)

diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index f3ac003..f8560c9 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -46,27 +46,43 @@ void sbi_putc(char ch)
 	}
 }
 
+static unsigned long console_puts_all(const char *str, unsigned long len)
+{
+	const char *s = str;
+	const char *e = s + len;
+	while (s < e)
+		s += console_dev->console_puts(s, e - s);
+	return len;
+}
+
+static unsigned long console_puts(const char *str, unsigned long len)
+{
+	const char *s, *p, *e;
+	s = str;
+	e = str + len;
+	while (s < e) {
+		p = sbi_strchr(s, '\n');
+		p = p ? p : e;
+		console_puts_all(s, p - s);
+		if (p < e && *p == '\n')
+			console_puts_all("\r\n", 2);
+		s = p + 1;
+	}
+	return len;
+}
+
 static unsigned long nputs(const char *str, unsigned long len)
 {
-	unsigned long i, ret;
+	unsigned long i;
 
 	if (console_dev && console_dev->console_puts) {
-		ret = console_dev->console_puts(str, len);
+		console_puts(str, len);
 	} else {
 		for (i = 0; i < len; i++)
 			sbi_putc(str[i]);
-		ret = len;
 	}
 
-	return ret;
-}
-
-static void nputs_all(const char *str, unsigned long len)
-{
-	unsigned long p = 0;
-
-	while (p < len)
-		p += nputs(&str[p], len - p);
+	return len;
 }
 
 void sbi_puts(const char *str)
@@ -74,7 +90,7 @@ void sbi_puts(const char *str)
 	unsigned long len = sbi_strlen(str);
 
 	spin_lock(&console_out_lock);
-	nputs_all(str, len);
+	nputs(str, len);
 	spin_unlock(&console_out_lock);
 }
 
@@ -255,7 +271,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
 
 	for (; *format != 0; ++format) {
 		if (use_tbuf && !console_tbuf_len) {
-			nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
+			nputs(console_tbuf, CONSOLE_TBUF_MAX);
 			console_tbuf_len = CONSOLE_TBUF_MAX;
 			tout = console_tbuf;
 		}
@@ -386,7 +402,7 @@ literal:
 	}
 
 	if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
-		nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
+		nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
 
 	return pc;
 }
-- 
2.39.1



From anup at brainfault.org  Sun Feb 12 19:55:54 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 13 Feb 2023 09:25:54 +0530
Subject: [PATCH v2] lib: sbi: fix missing '\r' for console
In-Reply-To: <20230213020821.109087-1-wxjstz@126.com>
References: <20230213020821.109087-1-wxjstz@126.com>
Message-ID: <CAAhSdy3SvbA=XSaoGrSAriN3Qk0LeEkkBafzeAbBHHJnAR3vEA@mail.gmail.com>

On Mon, Feb 13, 2023 at 8:09 AM Xiang W <wxjstz at 126.com> wrote:
>
> print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> will add a \r before the output \n. This patch adds missing \r when
> outputting characters via console_dev->puts.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>
>
> Changes since v2:
> - Fix the bug reported by Samuel. Prevent p from accessing memory other
>   than strings.
> ---
>  lib/sbi/sbi_console.c | 46 +++++++++++++++++++++++++++++--------------
>  1 file changed, 31 insertions(+), 15 deletions(-)
>
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index f3ac003..f8560c9 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -46,27 +46,43 @@ void sbi_putc(char ch)
>         }
>  }
>
> +static unsigned long console_puts_all(const char *str, unsigned long len)
> +{
> +       const char *s = str;
> +       const char *e = s + len;
> +       while (s < e)
> +               s += console_dev->console_puts(s, e - s);
> +       return len;
> +}
> +
> +static unsigned long console_puts(const char *str, unsigned long len)
> +{
> +       const char *s, *p, *e;
> +       s = str;
> +       e = str + len;
> +       while (s < e) {
> +               p = sbi_strchr(s, '\n');
> +               p = p ? p : e;
> +               console_puts_all(s, p - s);
> +               if (p < e && *p == '\n')
> +                       console_puts_all("\r\n", 2);
> +               s = p + 1;
> +       }
> +       return len;
> +}
> +
>  static unsigned long nputs(const char *str, unsigned long len)
>  {
> -       unsigned long i, ret;
> +       unsigned long i;
>
>         if (console_dev && console_dev->console_puts) {
> -               ret = console_dev->console_puts(str, len);
> +               console_puts(str, len);
>         } else {
>                 for (i = 0; i < len; i++)
>                         sbi_putc(str[i]);
> -               ret = len;
>         }
>
> -       return ret;
> -}
> -
> -static void nputs_all(const char *str, unsigned long len)
> -{
> -       unsigned long p = 0;
> -
> -       while (p < len)
> -               p += nputs(&str[p], len - p);
> +       return len;

This patch is already broken because it changes the semantics
of nputs() and nputs_all(). The nputs() is supposed is allowed
to do partial writes whereas nputs_all() only returns after printing
all characters.

Regards,
Anup

>  }
>
>  void sbi_puts(const char *str)
> @@ -74,7 +90,7 @@ void sbi_puts(const char *str)
>         unsigned long len = sbi_strlen(str);
>
>         spin_lock(&console_out_lock);
> -       nputs_all(str, len);
> +       nputs(str, len);
>         spin_unlock(&console_out_lock);
>  }
>
> @@ -255,7 +271,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
>
>         for (; *format != 0; ++format) {
>                 if (use_tbuf && !console_tbuf_len) {
> -                       nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
> +                       nputs(console_tbuf, CONSOLE_TBUF_MAX);
>                         console_tbuf_len = CONSOLE_TBUF_MAX;
>                         tout = console_tbuf;
>                 }
> @@ -386,7 +402,7 @@ literal:
>         }
>
>         if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
> -               nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> +               nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
>
>         return pc;
>  }
> --
> 2.39.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sun Feb 12 20:07:22 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 13 Feb 2023 09:37:22 +0530
Subject: [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to
 configure the PMA
In-Reply-To: <20230206000624.6544-2-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20230206000624.6544-2-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <CAAhSdy1TWkVHpWCz7Ada=7tT62qjPDmqskn1HL-iWLn7eb4=8g@mail.gmail.com>

On Mon, Feb 6, 2023 at 5:36 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
> block that allows dynamic adjustment of memory attributes in the runtime.
> It contains a configurable amount of PMA entries implemented as CSR
> registers to control the attributes of memory locations in interest.
> Below are the memory attributes supported:
> * Device, Non-bufferable
> * Device, bufferable
> * Memory, Non-cacheable, Non-bufferable
> * Memory, Non-cacheable, Bufferable
> * Memory, Write-back, No-allocate
> * Memory, Write-back, Read-allocate
> * Memory, Write-back, Write-allocate
> * Memory, Write-back, Read and Write-allocate
>
> More info about PMA (section 10.3):
> Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> Example PMA region passed as a DT node from OpenSBI:
>     reserved-memory {
>         #address-cells = <2>;
>         #size-cells = <2>;
>         ranges;
>
>         pma_resv0 at 58000000 {
>             compatible = "shared-dma-pool";
>             reg = <0x0 0x58000000 0x0 0x08000000>;
>             no-map;
>             linux,dma-default;
>         };
>     };
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> ---
>  platform/generic/Kconfig                      |   8 +
>  platform/generic/include/andes45_pma.h        |  48 +++
>  platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
>  platform/generic/renesas/rzfive/objects.mk    |   1 +

This file organization does not make sense because after all
these PMA configurations are Andes specific and RZFive
simply using it.

I suggest the following:
1) Rename
    platform/generic/include/andes45_pma.h
    to
    platform/generic/include/andes/andes45_pma.h
2) Rename
    platform/generic/renesas/rzfive/andes45-pma.c
    to
    platform/generic/andes/andes45-pma.c
3) Add platform/generic/andes/Kconfig which defines
    "config ANDES45_PMA" but does not depend on
    PLATFORM_RENESAS_RZFIVE
4) Include platform/generic/andes/Kconfig in
    platform/generic/Kconfig and select ANDES45_PMA
    for PLATFORM_RENESAS_RZFIVE

Regards,
Anup

>  4 files changed, 407 insertions(+)
>  create mode 100644 platform/generic/include/andes45_pma.h
>  create mode 100644 platform/generic/renesas/rzfive/andes45-pma.c
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index c7f198a..9ee9fe3 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -51,3 +51,11 @@ config PLATFORM_STARFIVE_JH7110
>         default n
>
>  endif
> +
> +if PLATFORM_RENESAS_RZFIVE
> +
> +config ANDES45_PMA
> +       bool "Andes PMA support"
> +       default n
> +
> +endif
> diff --git a/platform/generic/include/andes45_pma.h b/platform/generic/include/andes45_pma.h
> new file mode 100644
> index 0000000..37ec77c
> --- /dev/null
> +++ b/platform/generic/include/andes45_pma.h
> @@ -0,0 +1,48 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + */
> +
> +#ifndef _ANDES45_PMA_H_
> +#define _ANDES45_PMA_H_
> +
> +#include <sbi/sbi_types.h>
> +
> +#define ANDES45_MAX_PMA_REGIONS                        16
> +
> +/* Naturally aligned power of 2 region */
> +#define ANDES45_PMACFG_ETYP_NAPOT              3
> +
> +/* Memory, Non-cacheable, Bufferable */
> +#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF  (3 << 2)
> +
> +/**
> + * struct andes45_pma_region - Describes PMA regions
> + *
> + * @pa: Address to be configured in the PMA
> + * @size: Size of the region
> + * @flags: Flags to be set for the PMA region
> + * @dt_populate: Boolean flag indicating if the DT entry should be
> + *               populated for the given PMA region
> + * @shared_dma: Boolean flag if set "shared-dma-pool" property will
> + *              be set in the DT node
> + * @no_map: Boolean flag if set "no-map" property will be set in the
> + *          DT node
> + * @dma_default: Boolean flag if set "linux,dma-default" property will
> + *              be set in the DT node. Note Linux expects single node
> + *              with this property set.
> + */
> +struct andes45_pma_region {
> +       unsigned long pa;
> +       unsigned long size;
> +       u8 flags:7;
> +       bool dt_populate;
> +       bool shared_dma;
> +       bool no_map;
> +       bool dma_default;
> +};
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> +                             unsigned int pma_regions_count);
> +
> +#endif /* _ANDES45_PMA_H_ */
> diff --git a/platform/generic/renesas/rzfive/andes45-pma.c b/platform/generic/renesas/rzfive/andes45-pma.c
> new file mode 100644
> index 0000000..881a628
> --- /dev/null
> +++ b/platform/generic/renesas/rzfive/andes45-pma.c
> @@ -0,0 +1,350 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + *
> + * Copyright (c) 2020 Andes Technology Corporation
> + *
> + * Authors:
> + *      Nick Hu <nickhu at andestech.com>
> + *      Nylon Chen <nylon7 at andestech.com>
> + *      Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> + */
> +
> +#include <andes45_pma.h>
> +#include <libfdt.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +
> +/* Configuration Registers */
> +#define ANDES45_CSR_MMSC_CFG           0xFC2
> +#define ANDES45_CSR_MMSC_PPMA_OFFSET   (1 << 30)
> +
> +#define ANDES45_PMAADDR_0              0xBD0
> +
> +#define ANDES45_PMACFG_0               0xBC0
> +
> +static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
> +{
> +#define switchcase_pma_cfg_read(__pma_cfg_off, __val)          \
> +       case __pma_cfg_off:                                     \
> +               __val = csr_read(__pma_cfg_off);                \
> +               break;
> +#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val)                \
> +       switchcase_pma_cfg_read(__pma_cfg_off + 0, __val)       \
> +       switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
> +
> +       unsigned long ret = 0;
> +
> +       switch (pma_cfg_off) {
> +       switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> +               break;
> +       }
> +
> +       return ret;
> +
> +#undef switchcase_pma_cfg_read_2
> +#undef switchcase_pma_cfg_read
> +}
> +
> +static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
> +{
> +#define switchcase_pma_cfg_write(__pma_cfg_off, __val)         \
> +       case __pma_cfg_off:                                     \
> +               csr_write(__pma_cfg_off, __val);                \
> +               break;
> +#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val)       \
> +       switchcase_pma_cfg_write(__pma_cfg_off + 0, __val)      \
> +       switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
> +
> +       switch (pma_cfg_off) {
> +       switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> +               break;
> +       }
> +
> +#undef switchcase_pma_cfg_write_2
> +#undef switchcase_pma_cfg_write
> +}
> +
> +static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
> +{
> +#define switchcase_pma_write(__pma_addr_off, __val)            \
> +       case __pma_addr_off:                                    \
> +               csr_write(__pma_addr_off, __val);               \
> +               break;
> +#define switchcase_pma_write_2(__pma_addr_off, __val)          \
> +       switchcase_pma_write(__pma_addr_off + 0, __val)         \
> +       switchcase_pma_write(__pma_addr_off + 1, __val)
> +#define switchcase_pma_write_4(__pma_addr_off, __val)          \
> +       switchcase_pma_write_2(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_write_8(__pma_addr_off, __val)          \
> +       switchcase_pma_write_4(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_write_16(__pma_addr_off, __val)         \
> +       switchcase_pma_write_8(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_8(__pma_addr_off + 8, __val)
> +
> +       switch (pma_addr_off) {
> +       switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> +               break;
> +       }
> +
> +#undef switchcase_pma_write_16
> +#undef switchcase_pma_write_8
> +#undef switchcase_pma_write_4
> +#undef switchcase_pma_write_2
> +#undef switchcase_pma_write
> +}
> +
> +static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
> +{
> +#define switchcase_pma_read(__pma_addr_off, __val)             \
> +       case __pma_addr_off:                                    \
> +               __val = csr_read(__pma_addr_off);               \
> +               break;
> +#define switchcase_pma_read_2(__pma_addr_off, __val)           \
> +       switchcase_pma_read(__pma_addr_off + 0, __val)          \
> +       switchcase_pma_read(__pma_addr_off + 1, __val)
> +#define switchcase_pma_read_4(__pma_addr_off, __val)           \
> +       switchcase_pma_read_2(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_read_8(__pma_addr_off, __val)           \
> +       switchcase_pma_read_4(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_read_16(__pma_addr_off, __val)          \
> +       switchcase_pma_read_8(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_8(__pma_addr_off + 8, __val)
> +
> +       unsigned long ret = 0;
> +
> +       switch (pma_addr_off) {
> +       switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> +               break;
> +       }
> +
> +       return ret;
> +
> +#undef switchcase_pma_read_16
> +#undef switchcase_pma_read_8
> +#undef switchcase_pma_read_4
> +#undef switchcase_pma_read_2
> +#undef switchcase_pma_read
> +}
> +
> +static unsigned long
> +andes45_pma_setup(const struct andes45_pma_region *pma_region,
> +                 unsigned int entry_id)
> +{
> +       unsigned long size = pma_region->size;
> +       unsigned long addr = pma_region->pa;
> +       unsigned int pma_cfg_addr;
> +       unsigned long pmacfg_val;
> +       unsigned long pmaaddr;
> +       char *pmaxcfg;
> +
> +       /* Check for 4KiB granularity */
> +       if (size < (1 << 12))
> +               return SBI_EINVAL;
> +
> +       /* Check size is power of 2 */
> +       if (size & (size - 1))
> +               return SBI_EINVAL;
> +
> +       if (entry_id > 15)
> +               return SBI_EINVAL;
> +
> +       if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
> +               return SBI_EINVAL;
> +
> +       if ((addr & (size - 1)) != 0)
> +               return SBI_EINVAL;
> +
> +       pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
> +       pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
> +       pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
> +       *pmaxcfg = 0;
> +       *pmaxcfg = pma_region->flags;
> +
> +       andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
> +
> +       pmaaddr = (addr >> 2) + (size >> 3) - 1;
> +
> +       andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
> +
> +       return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
> +                       pmaaddr : SBI_EINVAL;
> +}
> +
> +static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
> +                               unsigned int index, int parent)
> +{
> +       int na = fdt_address_cells(fdt, 0);
> +       int ns = fdt_size_cells(fdt, 0);
> +       static bool dma_default = false;
> +       fdt32_t addr_high, addr_low;
> +       fdt32_t size_high, size_low;
> +       int subnode, err;
> +       fdt32_t reg[4];
> +       fdt32_t *val;
> +       char name[32];
> +
> +       addr_high = (u64)pma->pa >> 32;
> +       addr_low = pma->pa;
> +       size_high = (u64)pma->size >> 32;
> +       size_low = pma->size;
> +
> +       if (na > 1 && addr_high)
> +               sbi_snprintf(name, sizeof(name),
> +                            "pma_resv%d@%x,%x", index,
> +                            addr_high, addr_low);
> +       else
> +               sbi_snprintf(name, sizeof(name),
> +                            "pma_resv%d@%x", index,
> +                            addr_low);
> +
> +       subnode = fdt_add_subnode(fdt, parent, name);
> +       if (subnode < 0)
> +               return subnode;
> +
> +       if (pma->shared_dma) {
> +               err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       if (pma->no_map) {
> +               err = fdt_setprop_empty(fdt, subnode, "no-map");
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       /* Linux allows single linux,dma-default region. */
> +       if (pma->dma_default) {
> +               if (dma_default)
> +                       return SBI_EINVAL;
> +
> +               err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
> +               if (err < 0)
> +                       return err;
> +               dma_default = true;
> +       }
> +
> +       /* encode the <reg> property value */
> +       val = reg;
> +       if (na > 1)
> +               *val++ = cpu_to_fdt32(addr_high);
> +       *val++ = cpu_to_fdt32(addr_low);
> +       if (ns > 1)
> +               *val++ = cpu_to_fdt32(size_high);
> +       *val++ = cpu_to_fdt32(size_low);
> +
> +       err = fdt_setprop(fdt, subnode, "reg", reg,
> +                         (na + ns) * sizeof(fdt32_t));
> +       if (err < 0)
> +               return err;
> +
> +       return 0;
> +}
> +
> +static int andes45_fdt_reserved_memory_fixup(void *fdt,
> +                                            const struct andes45_pma_region *pma,
> +                                            unsigned int entry)
> +{
> +       int parent;
> +
> +       /* try to locate the reserved memory node */
> +       parent = fdt_path_offset(fdt, "/reserved-memory");
> +       if (parent < 0) {
> +               int na = fdt_address_cells(fdt, 0);
> +               int ns = fdt_size_cells(fdt, 0);
> +               int err;
> +
> +               /* if such node does not exist, create one */
> +               parent = fdt_add_subnode(fdt, 0, "reserved-memory");
> +               if (parent < 0)
> +                       return parent;
> +
> +               err = fdt_setprop_empty(fdt, parent, "ranges");
> +               if (err < 0)
> +                       return err;
> +
> +               err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
> +               if (err < 0)
> +                       return err;
> +
> +               err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       return andes45_fdt_pma_resv(fdt, pma, entry, parent);
> +}
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> +                             unsigned int pma_regions_count)
> +{
> +       unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
> +       unsigned int dt_populate_cnt;
> +       unsigned int i, j;
> +       unsigned long pa;
> +       void *fdt;
> +       int ret;
> +
> +       if (!pma_regions || !pma_regions_count)
> +               return 0;
> +
> +       if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
> +               return SBI_EINVAL;
> +
> +       if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
> +               return SBI_ENOTSUPP;
> +
> +       /* Configure the PMA regions */
> +       for (i = 0; i < pma_regions_count; i++) {
> +               pa = andes45_pma_setup(&pma_regions[i], i);
> +               if (pa == SBI_EINVAL)
> +                       return SBI_EINVAL;
> +       }
> +
> +       dt_populate_cnt = 0;
> +       for (i = 0; i < pma_regions_count; i++) {
> +               if (!pma_regions[i].dt_populate)
> +                       continue;
> +               dt_populate_cnt++;
> +       }
> +
> +       if (!dt_populate_cnt)
> +               return 0;
> +
> +       fdt = fdt_get_address();
> +
> +       ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
> +       if (ret < 0)
> +               return ret;
> +
> +       for (i = 0, j = 0; i < pma_regions_count; i++) {
> +               if (!pma_regions[i].dt_populate)
> +                       continue;
> +
> +               ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
> +               if (ret)
> +                       return ret;
> +       }
> +
> +       return 0;
> +}
> diff --git a/platform/generic/renesas/rzfive/objects.mk b/platform/generic/renesas/rzfive/objects.mk
> index 2e7e37f..fd9e7c4 100644
> --- a/platform/generic/renesas/rzfive/objects.mk
> +++ b/platform/generic/renesas/rzfive/objects.mk
> @@ -6,3 +6,4 @@
>
>  carray-platform_override_modules-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas_rzfive
>  platform-objs-$(CONFIG_PLATFORM_RENESAS_RZFIVE) += renesas/rzfive/rzfive.o
> +platform-objs-$(CONFIG_ANDES45_PMA) += renesas/rzfive/andes45-pma.o
> --
> 2.17.1
>


From anup at brainfault.org  Sun Feb 12 20:15:42 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 13 Feb 2023 09:45:42 +0530
Subject: [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to
 check for enabling IOCP errata
In-Reply-To: <20230206000624.6544-4-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20230206000624.6544-4-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <CAAhSdy2TZQ_C+3o-Qm0Z7H3aehuCOhwvxKXC8MAEhwvbHA2pdg@mail.gmail.com>

On Mon, Feb 6, 2023 at 5:37 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> errata should be applied to handle cache management.
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> ---
>  platform/generic/include/andes/andes45.h | 25 ++++++++++++-
>  platform/generic/renesas/rzfive/rzfive.c | 47 ++++++++++++++++++++++++
>  2 files changed, 70 insertions(+), 2 deletions(-)
>
> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index 08b3d18..d5d265f 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -4,7 +4,28 @@
>  #define CSR_MARCHID_MICROID 0xfff
>
>  /* Memory and Miscellaneous Registers */
> -#define CSR_MCACHE_CTL 0x7ca
> -#define CSR_MCCTLCOMMAND 0x7cc
> +#define CSR_MCACHE_CTL         0x7ca
> +#define CSR_MCCTLCOMMAND       0x7cc
> +#define CSR_MICM_CFG           0xfc0
> +#define CSR_MDCM_CFG           0xfc1
> +#define CSR_MMSC_CFG           0xfc2
> +
> +#define MISA_20_OFFSET                 20
> +#define MISA_20_MASK                   (0x1 << MISA_20_OFFSET)
> +
> +#define MICM_CFG_ISZ_OFFSET            6
> +#define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
> +
> +#define MDCM_CFG_DSZ_OFFSET            6
> +#define MDCM_CFG_DSZ_MASK              (0x7  << MDCM_CFG_DSZ_OFFSET)
> +
> +#define MMSC_CFG_CCTLCSR_OFFSET                16
> +#define MMSC_CFG_CCTLCSR_MASK          (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> +#define MMSC_IOCP_OFFSET                       47
> +#define MMSC_IOCP_MASK                 (0x1ULL << MMSC_IOCP_OFFSET)
> +
> +#define MCACHE_CTL_CCTL_SUEN_OFFSET    8
> +#define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> +
>
>  #endif /* _RISCV_ANDES45_H */
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index eee9c51..552c747 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -4,11 +4,17 @@
>   *
>   */
>
> +#include <andes/andes45.h>
>  #include <andes45_pma.h>
>  #include <platform_override.h>
>  #include <sbi/sbi_domain.h>
> +#include <sbi/sbi_error.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
> +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND      0
> +
> +#define ANDESTECH_SBI_EXT_ANDES                        0x900031e
> +
>  static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
>         {
>                 .pa = 0x58000000,
> @@ -28,6 +34,46 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
>                                          array_size(renesas_rzfive_pma_regions));
>  }
>
> +static bool renesas_rzfive_cpu_cache_controlable(void)
> +{
> +       return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
> +                (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
> +               (csr_read(CSR_MISA) & MISA_20_MASK) &&
> +               (csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
> +               (csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
> +}
> +
> +static bool renesas_rzfive_cpu_iocp_disabled(void)
> +{
> +       return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
> +}
> +
> +static bool renesas_rzfive_apply_iocp_sw_workaround(void)
> +{
> +       return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
> +}
> +
> +static int renesas_rzfive_vendor_ext_provider(long extid, long funcid,
> +                                             const struct sbi_trap_regs *regs,
> +                                             unsigned long *out_value,
> +                                             struct sbi_trap_info *out_trap,
> +                                             const struct fdt_match *match)
> +{
> +       if (extid != ANDESTECH_SBI_EXT_ANDES)
> +               return SBI_EINVAL;

This is also looking strange because you are checking Andes vendor
extension in RZ Five sources.

I will send a small patch which will help you simplify this over here. You
can include that patch in this series in the next revision.

Regards,
Anup

> +
> +       switch (funcid) {
> +       case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
> +               *out_value = renesas_rzfive_apply_iocp_sw_workaround();
> +               break;
> +
> +       default:
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
>  int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>  {
>         /*
> @@ -55,4 +101,5 @@ const struct platform_override renesas_rzfive = {
>         .match_table = renesas_rzfive_match,
>         .early_init = renesas_rzfive_early_init,
>         .final_init = renesas_rzfive_final_init,
> +       .vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
>  };
> --
> 2.17.1
>


From apatel at ventanamicro.com  Sun Feb 12 21:18:04 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 13 Feb 2023 10:48:04 +0530
Subject: [PATCH 1/2] lib: sbi: Align SBI vendor extension id with mvendorid CSR
Message-ID: <20230213051805.603113-1-apatel@ventanamicro.com>

As-per the SBI specification, the lower 24bits of the SBI vendor
extension id is same as lower 24bits of the mvendorid CSR.

We update the SBI vendor extension id checking based on above.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 include/sbi/sbi_platform.h                   | 17 ++++++++---------
 lib/sbi/sbi_ecall_vendor.c                   | 19 +++++++++++++++++--
 platform/generic/include/platform_override.h |  1 -
 platform/generic/platform.c                  | 19 ++++++-------------
 4 files changed, 31 insertions(+), 25 deletions(-)

diff --git a/include/sbi/sbi_platform.h b/include/sbi/sbi_platform.h
index 43d1c03..3a629a6 100644
--- a/include/sbi/sbi_platform.h
+++ b/include/sbi/sbi_platform.h
@@ -126,8 +126,8 @@ struct sbi_platform_operations {
 	/** Exit platform timer for current HART */
 	void (*timer_exit)(void);
 
-	/** platform specific SBI extension implementation probe function */
-	int (*vendor_ext_check)(long extid);
+	/** Check if SBI vendor extension is implemented or not */
+	bool (*vendor_ext_check)(void);
 	/** platform specific SBI extension implementation provider */
 	int (*vendor_ext_provider)(long extid, long funcid,
 				   const struct sbi_trap_regs *regs,
@@ -636,20 +636,19 @@ static inline void sbi_platform_timer_exit(const struct sbi_platform *plat)
 }
 
 /**
- * Check if a vendor extension is implemented or not.
+ * Check if SBI vendor extension is implemented or not.
  *
  * @param plat pointer to struct sbi_platform
- * @param extid	vendor SBI extension id
  *
- * @return 0 if extid is not implemented and 1 if implemented
+ * @return false if not implemented and true if implemented
  */
-static inline int sbi_platform_vendor_ext_check(const struct sbi_platform *plat,
-						long extid)
+static inline bool sbi_platform_vendor_ext_check(
+					const struct sbi_platform *plat)
 {
 	if (plat && sbi_platform_ops(plat)->vendor_ext_check)
-		return sbi_platform_ops(plat)->vendor_ext_check(extid);
+		return sbi_platform_ops(plat)->vendor_ext_check();
 
-	return 0;
+	return false;
 }
 
 /**
diff --git a/lib/sbi/sbi_ecall_vendor.c b/lib/sbi/sbi_ecall_vendor.c
index 9252829..9ea5156 100644
--- a/lib/sbi/sbi_ecall_vendor.c
+++ b/lib/sbi/sbi_ecall_vendor.c
@@ -13,12 +13,23 @@
 #include <sbi/sbi_error.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_trap.h>
+#include <sbi/riscv_asm.h>
+
+static inline unsigned long sbi_ecall_vendor_id(void)
+{
+	return SBI_EXT_VENDOR_START +
+		(csr_read(CSR_MVENDORID) &
+		 (SBI_EXT_VENDOR_END - SBI_EXT_VENDOR_START));
+}
 
 static int sbi_ecall_vendor_probe(unsigned long extid,
 				  unsigned long *out_val)
 {
-	*out_val = sbi_platform_vendor_ext_check(sbi_platform_thishart_ptr(),
-						 extid);
+	if (!sbi_platform_vendor_ext_check(sbi_platform_thishart_ptr()) ||
+	    extid != sbi_ecall_vendor_id())
+		*out_val = 0;
+	else
+		*out_val = 1;
 	return 0;
 }
 
@@ -27,6 +38,10 @@ static int sbi_ecall_vendor_handler(unsigned long extid, unsigned long funcid,
 				    unsigned long *out_val,
 				    struct sbi_trap_info *out_trap)
 {
+	if (!sbi_platform_vendor_ext_check(sbi_platform_thishart_ptr()) ||
+	    extid != sbi_ecall_vendor_id())
+		return SBI_ERR_NOT_SUPPORTED;
+
 	return sbi_platform_vendor_ext_provider(sbi_platform_thishart_ptr(),
 						extid, funcid, regs,
 						out_val, out_trap);
diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
index a59b06a..350c381 100644
--- a/platform/generic/include/platform_override.h
+++ b/platform/generic/include/platform_override.h
@@ -27,7 +27,6 @@ struct platform_override {
 	int (*extensions_init)(const struct fdt_match *match,
 			       struct sbi_hart_features *hfeatures);
 	void (*fw_init)(void *fdt, const struct fdt_match *match);
-	int (*vendor_ext_check)(long extid, const struct fdt_match *match);
 	int (*vendor_ext_provider)(long extid, long funcid,
 				   const struct sbi_trap_regs *regs,
 				   unsigned long *out_value,
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index a34d3b0..88c3ec7 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -180,13 +180,10 @@ static int generic_final_init(bool cold_boot)
 	return 0;
 }
 
-static int generic_vendor_ext_check(long extid)
+static bool generic_vendor_ext_check(void)
 {
-	if (generic_plat && generic_plat->vendor_ext_check)
-		return generic_plat->vendor_ext_check(extid,
-						      generic_plat_match);
-
-	return 0;
+	return (generic_plat && generic_plat->vendor_ext_provider) ?
+		true : false;
 }
 
 static int generic_vendor_ext_provider(long extid, long funcid,
@@ -194,13 +191,9 @@ static int generic_vendor_ext_provider(long extid, long funcid,
 				       unsigned long *out_value,
 				       struct sbi_trap_info *out_trap)
 {
-	if (generic_plat && generic_plat->vendor_ext_provider) {
-		return generic_plat->vendor_ext_provider(extid, funcid, regs,
-							 out_value, out_trap,
-							 generic_plat_match);
-	}
-
-	return SBI_ENOTSUPP;
+	return generic_plat->vendor_ext_provider(extid, funcid, regs,
+						 out_value, out_trap,
+						 generic_plat_match);
 }
 
 static void generic_early_exit(void)
-- 
2.34.1



From apatel at ventanamicro.com  Sun Feb 12 21:18:05 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 13 Feb 2023 10:48:05 +0530
Subject: [PATCH 2/2] include: sbi: Remove extid parameter from vendor_ext_provider() callback
In-Reply-To: <20230213051805.603113-1-apatel@ventanamicro.com>
References: <20230213051805.603113-1-apatel@ventanamicro.com>
Message-ID: <20230213051805.603113-2-apatel@ventanamicro.com>

The extid parameter of vendor_ext_provider() is redundant so let us
remove it.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 include/sbi/sbi_platform.h                   | 9 ++++-----
 lib/sbi/sbi_ecall_vendor.c                   | 2 +-
 platform/generic/include/platform_override.h | 2 +-
 platform/generic/platform.c                  | 4 ++--
 4 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/include/sbi/sbi_platform.h b/include/sbi/sbi_platform.h
index 3a629a6..546c0a6 100644
--- a/include/sbi/sbi_platform.h
+++ b/include/sbi/sbi_platform.h
@@ -129,7 +129,7 @@ struct sbi_platform_operations {
 	/** Check if SBI vendor extension is implemented or not */
 	bool (*vendor_ext_check)(void);
 	/** platform specific SBI extension implementation provider */
-	int (*vendor_ext_provider)(long extid, long funcid,
+	int (*vendor_ext_provider)(long funcid,
 				   const struct sbi_trap_regs *regs,
 				   unsigned long *out_value,
 				   struct sbi_trap_info *out_trap);
@@ -655,7 +655,6 @@ static inline bool sbi_platform_vendor_ext_check(
  * Invoke platform specific vendor SBI extension implementation.
  *
  * @param plat pointer to struct sbi_platform
- * @param extid	vendor SBI extension id
  * @param funcid SBI function id within the extension id
  * @param regs pointer to trap registers passed by the caller
  * @param out_value output value that can be filled by the callee
@@ -665,14 +664,14 @@ static inline bool sbi_platform_vendor_ext_check(
  */
 static inline int sbi_platform_vendor_ext_provider(
 					const struct sbi_platform *plat,
-					long extid, long funcid,
+					long funcid,
 					const struct sbi_trap_regs *regs,
 					unsigned long *out_value,
 					struct sbi_trap_info *out_trap)
 {
 	if (plat && sbi_platform_ops(plat)->vendor_ext_provider) {
-		return sbi_platform_ops(plat)->vendor_ext_provider(extid,
-								funcid, regs,
+		return sbi_platform_ops(plat)->vendor_ext_provider(funcid,
+								regs,
 								out_value,
 								out_trap);
 	}
diff --git a/lib/sbi/sbi_ecall_vendor.c b/lib/sbi/sbi_ecall_vendor.c
index 9ea5156..8b8dab0 100644
--- a/lib/sbi/sbi_ecall_vendor.c
+++ b/lib/sbi/sbi_ecall_vendor.c
@@ -43,7 +43,7 @@ static int sbi_ecall_vendor_handler(unsigned long extid, unsigned long funcid,
 		return SBI_ERR_NOT_SUPPORTED;
 
 	return sbi_platform_vendor_ext_provider(sbi_platform_thishart_ptr(),
-						extid, funcid, regs,
+						funcid, regs,
 						out_val, out_trap);
 }
 
diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
index 350c381..0d9e5ee 100644
--- a/platform/generic/include/platform_override.h
+++ b/platform/generic/include/platform_override.h
@@ -27,7 +27,7 @@ struct platform_override {
 	int (*extensions_init)(const struct fdt_match *match,
 			       struct sbi_hart_features *hfeatures);
 	void (*fw_init)(void *fdt, const struct fdt_match *match);
-	int (*vendor_ext_provider)(long extid, long funcid,
+	int (*vendor_ext_provider)(long funcid,
 				   const struct sbi_trap_regs *regs,
 				   unsigned long *out_value,
 				   struct sbi_trap_info *out_trap,
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index 88c3ec7..0b90fd7 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -186,12 +186,12 @@ static bool generic_vendor_ext_check(void)
 		true : false;
 }
 
-static int generic_vendor_ext_provider(long extid, long funcid,
+static int generic_vendor_ext_provider(long funcid,
 				       const struct sbi_trap_regs *regs,
 				       unsigned long *out_value,
 				       struct sbi_trap_info *out_trap)
 {
-	return generic_plat->vendor_ext_provider(extid, funcid, regs,
+	return generic_plat->vendor_ext_provider(funcid, regs,
 						 out_value, out_trap,
 						 generic_plat_match);
 }
-- 
2.34.1



From wxjstz at 126.com  Sun Feb 12 22:52:16 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 13 Feb 2023 14:52:16 +0800
Subject: [PATCH v2] lib: sbi: fix missing '\r' for console
In-Reply-To: <CAAhSdy3SvbA=XSaoGrSAriN3Qk0LeEkkBafzeAbBHHJnAR3vEA@mail.gmail.com>
References: <20230213020821.109087-1-wxjstz@126.com>
	 <CAAhSdy3SvbA=XSaoGrSAriN3Qk0LeEkkBafzeAbBHHJnAR3vEA@mail.gmail.com>
Message-ID: <cc00eb448d4d27e9a4d8e1b3a0aabbd7935aa941.camel@126.com>

? 2023-02-13???? 09:25 +0530?Anup Patel???
> On Mon, Feb 13, 2023 at 8:09 AM Xiang W <wxjstz at 126.com> wrote:
> > 
> > print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> > will add a \r before the output \n. This patch adds missing \r when
> > outputting characters via console_dev->puts.
> > 
> > Signed-off-by: Xiang W <wxjstz at 126.com>
> > 
> > Changes since v2:
> > - Fix the bug reported by Samuel. Prevent p from accessing memory other
> > ? than strings.
> > ---
> > ?lib/sbi/sbi_console.c | 46 +++++++++++++++++++++++++++++--------------
> > ?1 file changed, 31 insertions(+), 15 deletions(-)
> > 
> > diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> > index f3ac003..f8560c9 100644
> > --- a/lib/sbi/sbi_console.c
> > +++ b/lib/sbi/sbi_console.c
> > @@ -46,27 +46,43 @@ void sbi_putc(char ch)
> > ??????? }
> > ?}
> > 
> > +static unsigned long console_puts_all(const char *str, unsigned long len)
> > +{
> > +?????? const char *s = str;
> > +?????? const char *e = s + len;
> > +?????? while (s < e)
> > +?????????????? s += console_dev->console_puts(s, e - s);
> > +?????? return len;
> > +}
> > +
> > +static unsigned long console_puts(const char *str, unsigned long len)
> > +{
> > +?????? const char *s, *p, *e;
> > +?????? s = str;
> > +?????? e = str + len;
> > +?????? while (s < e) {
> > +?????????????? p = sbi_strchr(s, '\n');
> > +?????????????? p = p ? p : e;
> > +?????????????? console_puts_all(s, p - s);
> > +?????????????? if (p < e && *p == '\n')
> > +?????????????????????? console_puts_all("\r\n", 2);
> > +?????????????? s = p + 1;
> > +?????? }
> > +?????? return len;
> > +}
> > +
> > ?static unsigned long nputs(const char *str, unsigned long len)
> > ?{
> > -?????? unsigned long i, ret;
> > +?????? unsigned long i;
> > 
> > ??????? if (console_dev && console_dev->console_puts) {
> > -?????????????? ret = console_dev->console_puts(str, len);
> > +?????????????? console_puts(str, len);
> > ??????? } else {
> > ??????????????? for (i = 0; i < len; i++)
> > ??????????????????????? sbi_putc(str[i]);
> > -?????????????? ret = len;
> > ??????? }
> > 
> > -?????? return ret;
> > -}
> > -
> > -static void nputs_all(const char *str, unsigned long len)
> > -{
> > -?????? unsigned long p = 0;
> > -
> > -?????? while (p < len)
> > -?????????????? p += nputs(&str[p], len - p);
> > +?????? return len;
> 
> This patch is already broken because it changes the semantics
> of nputs() and nputs_all(). The nputs() is supposed is allowed
> to do partial writes whereas nputs_all() only returns after printing
> all characters.
> 
> Regards,
> Anup
When the platform does not implement console_dev->console_puts, nputs and
nputs_all are effectively the same. Also, I don't quite understand the
circumstances under which partial writes are needed. Will there be a
performance advantage for multiple sbi calls to complete the printing?

Regards,
Xiang W
> 
> > ?}
> > 
> > ?void sbi_puts(const char *str)
> > @@ -74,7 +90,7 @@ void sbi_puts(const char *str)
> > ??????? unsigned long len = sbi_strlen(str);
> > 
> > ??????? spin_lock(&console_out_lock);
> > -?????? nputs_all(str, len);
> > +?????? nputs(str, len);
> > ??????? spin_unlock(&console_out_lock);
> > ?}
> > 
> > @@ -255,7 +271,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
> > 
> > ??????? for (; *format != 0; ++format) {
> > ??????????????? if (use_tbuf && !console_tbuf_len) {
> > -?????????????????????? nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
> > +?????????????????????? nputs(console_tbuf, CONSOLE_TBUF_MAX);
> > ??????????????????????? console_tbuf_len = CONSOLE_TBUF_MAX;
> > ??????????????????????? tout = console_tbuf;
> > ??????????????? }
> > @@ -386,7 +402,7 @@ literal:
> > ??????? }
> > 
> > ??????? if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
> > -?????????????? nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> > +?????????????? nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> > 
> > ??????? return pc;
> > ?}
> > --
> > 2.39.1
> > 
> > 
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From wxjstz at 126.com  Sun Feb 12 22:54:11 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 13 Feb 2023 14:54:11 +0800
Subject: [PATCH v3] lib: sbi: fix missing '\r' for console
Message-ID: <20230213065411.131378-1-wxjstz@126.com>

print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
will add a \r before the output \n. This patch adds missing \r when
outputting characters via console_dev->puts.

Signed-off-by: Xiang W <wxjstz at 126.com>

Changes since v3:
- Prevent string overflow when looking for \n

Changes since v2:
- Fix the bug reported by Samuel. Prevent p from accessing memory other
  than strings.
---
 lib/sbi/sbi_console.c | 47 +++++++++++++++++++++++++++++--------------
 1 file changed, 32 insertions(+), 15 deletions(-)

diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index f3ac003..99436c3 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -46,27 +46,44 @@ void sbi_putc(char ch)
 	}
 }
 
+static unsigned long console_puts_all(const char *str, unsigned long len)
+{
+	const char *s = str;
+	const char *e = s + len;
+	while (s < e)
+		s += console_dev->console_puts(s, e - s);
+	return len;
+}
+
+static unsigned long console_puts(const char *str, unsigned long len)
+{
+	const char *s, *p, *e;
+	s = str;
+	e = str + len;
+	while (s < e) {
+		p = sbi_strchr(s, '\n');
+		if (p == NULL || p >= e)
+			p = e;
+		console_puts_all(s, p - s);
+		if (p < e && *p == '\n')
+			console_puts_all("\r\n", 2);
+		s = p + 1;
+	}
+	return len;
+}
+
 static unsigned long nputs(const char *str, unsigned long len)
 {
-	unsigned long i, ret;
+	unsigned long i;
 
 	if (console_dev && console_dev->console_puts) {
-		ret = console_dev->console_puts(str, len);
+		console_puts(str, len);
 	} else {
 		for (i = 0; i < len; i++)
 			sbi_putc(str[i]);
-		ret = len;
 	}
 
-	return ret;
-}
-
-static void nputs_all(const char *str, unsigned long len)
-{
-	unsigned long p = 0;
-
-	while (p < len)
-		p += nputs(&str[p], len - p);
+	return len;
 }
 
 void sbi_puts(const char *str)
@@ -74,7 +91,7 @@ void sbi_puts(const char *str)
 	unsigned long len = sbi_strlen(str);
 
 	spin_lock(&console_out_lock);
-	nputs_all(str, len);
+	nputs(str, len);
 	spin_unlock(&console_out_lock);
 }
 
@@ -255,7 +272,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
 
 	for (; *format != 0; ++format) {
 		if (use_tbuf && !console_tbuf_len) {
-			nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
+			nputs(console_tbuf, CONSOLE_TBUF_MAX);
 			console_tbuf_len = CONSOLE_TBUF_MAX;
 			tout = console_tbuf;
 		}
@@ -386,7 +403,7 @@ literal:
 	}
 
 	if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
-		nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
+		nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
 
 	return pc;
 }
-- 
2.39.1



From apatel at ventanamicro.com  Sun Feb 12 23:44:24 2023
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 13 Feb 2023 13:14:24 +0530
Subject: [PATCH v2] lib: sbi: fix missing '\r' for console
In-Reply-To: <cc00eb448d4d27e9a4d8e1b3a0aabbd7935aa941.camel@126.com>
References: <20230213020821.109087-1-wxjstz@126.com> <CAAhSdy3SvbA=XSaoGrSAriN3Qk0LeEkkBafzeAbBHHJnAR3vEA@mail.gmail.com>
 <cc00eb448d4d27e9a4d8e1b3a0aabbd7935aa941.camel@126.com>
Message-ID: <CAK9=C2UO-pNACnQvf+L9a1+ioR1gFnx6ZpZMjN4ic3DJEY_YPQ@mail.gmail.com>

On Mon, Feb 13, 2023 at 12:22 PM Xiang W <wxjstz at 126.com> wrote:
>
> ? 2023-02-13???? 09:25 +0530?Anup Patel???
> > On Mon, Feb 13, 2023 at 8:09 AM Xiang W <wxjstz at 126.com> wrote:
> > >
> > > print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> > > will add a \r before the output \n. This patch adds missing \r when
> > > outputting characters via console_dev->puts.
> > >
> > > Signed-off-by: Xiang W <wxjstz at 126.com>
> > >
> > > Changes since v2:
> > > - Fix the bug reported by Samuel. Prevent p from accessing memory other
> > >   than strings.
> > > ---
> > >  lib/sbi/sbi_console.c | 46 +++++++++++++++++++++++++++++--------------
> > >  1 file changed, 31 insertions(+), 15 deletions(-)
> > >
> > > diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> > > index f3ac003..f8560c9 100644
> > > --- a/lib/sbi/sbi_console.c
> > > +++ b/lib/sbi/sbi_console.c
> > > @@ -46,27 +46,43 @@ void sbi_putc(char ch)
> > >         }
> > >  }
> > >
> > > +static unsigned long console_puts_all(const char *str, unsigned long len)
> > > +{
> > > +       const char *s = str;
> > > +       const char *e = s + len;
> > > +       while (s < e)
> > > +               s += console_dev->console_puts(s, e - s);
> > > +       return len;
> > > +}
> > > +
> > > +static unsigned long console_puts(const char *str, unsigned long len)
> > > +{
> > > +       const char *s, *p, *e;
> > > +       s = str;
> > > +       e = str + len;
> > > +       while (s < e) {
> > > +               p = sbi_strchr(s, '\n');
> > > +               p = p ? p : e;
> > > +               console_puts_all(s, p - s);
> > > +               if (p < e && *p == '\n')
> > > +                       console_puts_all("\r\n", 2);
> > > +               s = p + 1;
> > > +       }
> > > +       return len;
> > > +}
> > > +
> > >  static unsigned long nputs(const char *str, unsigned long len)
> > >  {
> > > -       unsigned long i, ret;
> > > +       unsigned long i;
> > >
> > >         if (console_dev && console_dev->console_puts) {
> > > -               ret = console_dev->console_puts(str, len);
> > > +               console_puts(str, len);
> > >         } else {
> > >                 for (i = 0; i < len; i++)
> > >                         sbi_putc(str[i]);
> > > -               ret = len;
> > >         }
> > >
> > > -       return ret;
> > > -}
> > > -
> > > -static void nputs_all(const char *str, unsigned long len)
> > > -{
> > > -       unsigned long p = 0;
> > > -
> > > -       while (p < len)
> > > -               p += nputs(&str[p], len - p);
> > > +       return len;
> >
> > This patch is already broken because it changes the semantics
> > of nputs() and nputs_all(). The nputs() is supposed is allowed
> > to do partial writes whereas nputs_all() only returns after printing
> > all characters.
> >
> > Regards,
> > Anup
> When the platform does not implement console_dev->console_puts, nputs and
> nputs_all are effectively the same. Also, I don't quite understand the
> circumstances under which partial writes are needed. Will there be a
> performance advantage for multiple sbi calls to complete the printing?

The semihosting driver implements console_puts() and the debugger
implementing it can do partial writes using non-blocking file I/O.

Until the original semantics of nputs() and nputs_all() are preserved,
it's a NACK from my side.

Regards,
Anup

>
> Regards,
> Xiang W
> >
> > >  }
> > >
> > >  void sbi_puts(const char *str)
> > > @@ -74,7 +90,7 @@ void sbi_puts(const char *str)
> > >         unsigned long len = sbi_strlen(str);
> > >
> > >         spin_lock(&console_out_lock);
> > > -       nputs_all(str, len);
> > > +       nputs(str, len);
> > >         spin_unlock(&console_out_lock);
> > >  }
> > >
> > > @@ -255,7 +271,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
> > >
> > >         for (; *format != 0; ++format) {
> > >                 if (use_tbuf && !console_tbuf_len) {
> > > -                       nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
> > > +                       nputs(console_tbuf, CONSOLE_TBUF_MAX);
> > >                         console_tbuf_len = CONSOLE_TBUF_MAX;
> > >                         tout = console_tbuf;
> > >                 }
> > > @@ -386,7 +402,7 @@ literal:
> > >         }
> > >
> > >         if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
> > > -               nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> > > +               nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> > >
> > >         return pc;
> > >  }
> > > --
> > > 2.39.1
> > >
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From wxjstz at 126.com  Mon Feb 13 01:06:13 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 13 Feb 2023 17:06:13 +0800
Subject: [PATCH v2] lib: sbi: fix missing '\r' for console
In-Reply-To: <CAK9=C2UO-pNACnQvf+L9a1+ioR1gFnx6ZpZMjN4ic3DJEY_YPQ@mail.gmail.com>
References: <20230213020821.109087-1-wxjstz@126.com>
	 <CAAhSdy3SvbA=XSaoGrSAriN3Qk0LeEkkBafzeAbBHHJnAR3vEA@mail.gmail.com>
	 <cc00eb448d4d27e9a4d8e1b3a0aabbd7935aa941.camel@126.com>
	 <CAK9=C2UO-pNACnQvf+L9a1+ioR1gFnx6ZpZMjN4ic3DJEY_YPQ@mail.gmail.com>
Message-ID: <b6481880503167a858e7db448ef2e7e415ad5bfc.camel@126.com>

? 2023-02-13???? 13:14 +0530?Anup Patel???
> On Mon, Feb 13, 2023 at 12:22 PM Xiang W <wxjstz at 126.com> wrote:
> > 
> > ? 2023-02-13???? 09:25 +0530?Anup Patel???
> > > On Mon, Feb 13, 2023 at 8:09 AM Xiang W <wxjstz at 126.com> wrote:
> > > > 
> > > > print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> > > > will add a \r before the output \n. This patch adds missing \r when
> > > > outputting characters via console_dev->puts.
> > > > 
> > > > Signed-off-by: Xiang W <wxjstz at 126.com>
> > > > 
> > > > Changes since v2:
> > > > - Fix the bug reported by Samuel. Prevent p from accessing memory other
> > > > ? than strings.
> > > > ---
> > > > ?lib/sbi/sbi_console.c | 46 +++++++++++++++++++++++++++++--------------
> > > > ?1 file changed, 31 insertions(+), 15 deletions(-)
> > > > 
> > > > diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> > > > index f3ac003..f8560c9 100644
> > > > --- a/lib/sbi/sbi_console.c
> > > > +++ b/lib/sbi/sbi_console.c
> > > > @@ -46,27 +46,43 @@ void sbi_putc(char ch)
> > > > ??????? }
> > > > ?}
> > > > 
> > > > +static unsigned long console_puts_all(const char *str, unsigned long len)
> > > > +{
> > > > +?????? const char *s = str;
> > > > +?????? const char *e = s + len;
> > > > +?????? while (s < e)
> > > > +?????????????? s += console_dev->console_puts(s, e - s);
> > > > +?????? return len;
> > > > +}
> > > > +
> > > > +static unsigned long console_puts(const char *str, unsigned long len)
> > > > +{
> > > > +?????? const char *s, *p, *e;
> > > > +?????? s = str;
> > > > +?????? e = str + len;
> > > > +?????? while (s < e) {
> > > > +?????????????? p = sbi_strchr(s, '\n');
> > > > +?????????????? p = p ? p : e;
> > > > +?????????????? console_puts_all(s, p - s);
> > > > +?????????????? if (p < e && *p == '\n')
> > > > +?????????????????????? console_puts_all("\r\n", 2);
> > > > +?????????????? s = p + 1;
> > > > +?????? }
> > > > +?????? return len;
> > > > +}
> > > > +
> > > > ?static unsigned long nputs(const char *str, unsigned long len)
> > > > ?{
> > > > -?????? unsigned long i, ret;
> > > > +?????? unsigned long i;
> > > > 
> > > > ??????? if (console_dev && console_dev->console_puts) {
> > > > -?????????????? ret = console_dev->console_puts(str, len);
> > > > +?????????????? console_puts(str, len);
> > > > ??????? } else {
> > > > ??????????????? for (i = 0; i < len; i++)
> > > > ??????????????????????? sbi_putc(str[i]);
> > > > -?????????????? ret = len;
> > > > ??????? }
> > > > 
> > > > -?????? return ret;
> > > > -}
> > > > -
> > > > -static void nputs_all(const char *str, unsigned long len)
> > > > -{
> > > > -?????? unsigned long p = 0;
> > > > -
> > > > -?????? while (p < len)
> > > > -?????????????? p += nputs(&str[p], len - p);
> > > > +?????? return len;
> > > 
> > > This patch is already broken because it changes the semantics
> > > of nputs() and nputs_all(). The nputs() is supposed is allowed
> > > to do partial writes whereas nputs_all() only returns after printing
> > > all characters.
> > > 
> > > Regards,
> > > Anup
> > When the platform does not implement console_dev->console_puts, nputs and
> > nputs_all are effectively the same. Also, I don't quite understand the
> > circumstances under which partial writes are needed. Will there be a
> > performance advantage for multiple sbi calls to complete the printing?
> 
> The semihosting driver implements console_puts() and the debugger
> implementing it can do partial writes using non-blocking file I/O.
> 
> Until the original semantics of nputs() and nputs_all() are preserved,
> it's a NACK from my side.
The same interface should have the same behavior.The following behavior
is weird:
- A platform implements console_puts, the print does not add '\r'
- A platform does not implement console_puts, the print adds '\r'

I will modify the code to implement partial writes as much as possible
please review later

Regards,
Xiang W
> 
> Regards,
> Anup
> 
> > 
> > Regards,
> > Xiang W
> > > 
> > > > ?}
> > > > 
> > > > ?void sbi_puts(const char *str)
> > > > @@ -74,7 +90,7 @@ void sbi_puts(const char *str)
> > > > ??????? unsigned long len = sbi_strlen(str);
> > > > 
> > > > ??????? spin_lock(&console_out_lock);
> > > > -?????? nputs_all(str, len);
> > > > +?????? nputs(str, len);
> > > > ??????? spin_unlock(&console_out_lock);
> > > > ?}
> > > > 
> > > > @@ -255,7 +271,7 @@ static int print(char **out, u32 *out_len, const char *format, va_list args)
> > > > 
> > > > ??????? for (; *format != 0; ++format) {
> > > > ??????????????? if (use_tbuf && !console_tbuf_len) {
> > > > -?????????????????????? nputs_all(console_tbuf, CONSOLE_TBUF_MAX);
> > > > +?????????????????????? nputs(console_tbuf, CONSOLE_TBUF_MAX);
> > > > ??????????????????????? console_tbuf_len = CONSOLE_TBUF_MAX;
> > > > ??????????????????????? tout = console_tbuf;
> > > > ??????????????? }
> > > > @@ -386,7 +402,7 @@ literal:
> > > > ??????? }
> > > > 
> > > > ??????? if (use_tbuf && console_tbuf_len < CONSOLE_TBUF_MAX)
> > > > -?????????????? nputs_all(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> > > > +?????????????? nputs(console_tbuf, CONSOLE_TBUF_MAX - console_tbuf_len);
> > > > 
> > > > ??????? return pc;
> > > > ?}
> > > > --
> > > > 2.39.1
> > > > 
> > > > 
> > > > --
> > > > opensbi mailing list
> > > > opensbi at lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/opensbi
> > 
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From wxjstz at 126.com  Mon Feb 13 01:09:30 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 13 Feb 2023 17:09:30 +0800
Subject: [PATCH v4] lib: sbi: fix missing '\r' for console
Message-ID: <20230213090929.144906-1-wxjstz@126.com>

print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
will add a \r before the output \n. This patch adds missing \r when
outputting characters via console_dev->puts.

Signed-off-by: Xiang W <wxjstz at 126.com>

Changes since v4:
- nputs_all come back
- implement partial writes as much as possible

Changes since v3:
- Prevent string overflow when looking for \n

Changes since v2:
- Fix the bug reported by Samuel. Prevent p from accessing memory other
  than strings.

---
 lib/sbi/sbi_console.c | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index f3ac003..4be5a8f 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -46,12 +46,41 @@ void sbi_putc(char ch)
 	}
 }
 
+static unsigned long console_puts_all(const char *str, unsigned long len)
+{
+	const char *s = str;
+	const char *e = s + len;
+	while (s < e)
+		s += console_dev->console_puts(s, e - s);
+	return len;
+}
+
+static unsigned long console_puts(const char *str, unsigned long len)
+{
+	unsigned long l;
+	const char *s, *p, *e;
+	s = str;
+	e = str + len;
+	while (s < e) {
+		p = sbi_strchr(s, '\n');
+		if (p == NULL || p > e)
+			p = e;
+		l = console_dev->console_puts(s, p - s);
+		if (l < p - s)
+			return s - str + l;
+		if (p < e && *p == '\n')
+			console_puts_all("\r\n", 2);
+		s = p + 1;
+	}
+	return len;
+}
+
 static unsigned long nputs(const char *str, unsigned long len)
 {
 	unsigned long i, ret;
 
 	if (console_dev && console_dev->console_puts) {
-		ret = console_dev->console_puts(str, len);
+		ret = console_puts(str, len);
 	} else {
 		for (i = 0; i < len; i++)
 			sbi_putc(str[i]);
-- 
2.39.1



From prabhakar.csengg at gmail.com  Mon Feb 13 13:46:30 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Mon, 13 Feb 2023 21:46:30 +0000
Subject: [PATCH v4 1/3] platform: generic: renesas: rzfive: Add support to
 configure the PMA
In-Reply-To: <CAAhSdy1TWkVHpWCz7Ada=7tT62qjPDmqskn1HL-iWLn7eb4=8g@mail.gmail.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
 <20230206000624.6544-2-prabhakar.mahadev-lad.rj@bp.renesas.com> <CAAhSdy1TWkVHpWCz7Ada=7tT62qjPDmqskn1HL-iWLn7eb4=8g@mail.gmail.com>
Message-ID: <CA+V-a8vUAFqHK2ydgPDdu1n=cKS67haewR5oTc3SoNFR84OcSQ@mail.gmail.com>

Hi Anup,

Thank you for the review.

On Mon, Feb 13, 2023 at 4:07 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Mon, Feb 6, 2023 at 5:36 AM Lad Prabhakar
> <prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
> >
> > I/O Coherence Port (IOCP) provides an AXI interface for connecting
> > external non-caching masters, such as DMA controllers. The accesses
> > from IOCP are coherent with D-Caches and L2 Cache.
> >
> > IOCP is a specification option and is disabled on the Renesas RZ/Five
> > SoC due to this reason IP blocks using DMA will fail.
> >
> > The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
> > block that allows dynamic adjustment of memory attributes in the runtime.
> > It contains a configurable amount of PMA entries implemented as CSR
> > registers to control the attributes of memory locations in interest.
> > Below are the memory attributes supported:
> > * Device, Non-bufferable
> > * Device, bufferable
> > * Memory, Non-cacheable, Non-bufferable
<snip>
> >  platform/generic/Kconfig                      |   8 +
> >  platform/generic/include/andes45_pma.h        |  48 +++
> >  platform/generic/renesas/rzfive/andes45-pma.c | 350 ++++++++++++++++++
> >  platform/generic/renesas/rzfive/objects.mk    |   1 +
>
> This file organization does not make sense because after all
> these PMA configurations are Andes specific and RZFive
> simply using it.
>
> I suggest the following:
> 1) Rename
>     platform/generic/include/andes45_pma.h
>     to
>     platform/generic/include/andes/andes45_pma.h
> 2) Rename
>     platform/generic/renesas/rzfive/andes45-pma.c
>     to
>     platform/generic/andes/andes45-pma.c
> 3) Add platform/generic/andes/Kconfig which defines
>     "config ANDES45_PMA" but does not depend on
>     PLATFORM_RENESAS_RZFIVE
> 4) Include platform/generic/andes/Kconfig in
>     platform/generic/Kconfig and select ANDES45_PMA
>     for PLATFORM_RENESAS_RZFIVE
>
Ok, I'll do the above in the next version.

Cheers,
Prabhakar


From prabhakar.csengg at gmail.com  Mon Feb 13 13:48:05 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Mon, 13 Feb 2023 21:48:05 +0000
Subject: [PATCH v4 3/3] platform: generic: renesas: rzfive: Add SBI EXT to
 check for enabling IOCP errata
In-Reply-To: <CAAhSdy2TZQ_C+3o-Qm0Z7H3aehuCOhwvxKXC8MAEhwvbHA2pdg@mail.gmail.com>
References: <20230206000624.6544-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
 <20230206000624.6544-4-prabhakar.mahadev-lad.rj@bp.renesas.com> <CAAhSdy2TZQ_C+3o-Qm0Z7H3aehuCOhwvxKXC8MAEhwvbHA2pdg@mail.gmail.com>
Message-ID: <CA+V-a8s96SoLZpL9oj4uqN8M=bZ6mWVxaDGUJsnL9P4sry41Kw@mail.gmail.com>

Hi Anup,

Thank you for the review.

On Mon, Feb 13, 2023 at 4:15 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Mon, Feb 6, 2023 at 5:37 AM Lad Prabhakar
> <prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
> >
> > I/O Coherence Port (IOCP) provides an AXI interface for connecting
> > external non-caching masters, such as DMA controllers. The accesses
> > from IOCP are coherent with D-Caches and L2 Cache.
> >
> > IOCP is a specification option and is disabled on the Renesas RZ/Five
> > SoC due to this reason IP blocks using DMA will fail.
> >
> > As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> > software. Firstly OpenSBI configures the memory region as
> > "Memory, Non-cacheable, Bufferable" and passes this region as a global
> > shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> > allocations happen from this region and synchronization callbacks are
> > implemented to synchronize when doing DMA transactions.
> >
> > RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> > errata should be applied to handle cache management.
> >
> > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> > ---
> >  platform/generic/include/andes/andes45.h | 25 ++++++++++++-
> >  platform/generic/renesas/rzfive/rzfive.c | 47 ++++++++++++++++++++++++
> >  2 files changed, 70 insertions(+), 2 deletions(-)
> >
> > diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> > index 08b3d18..d5d265f 100644
> > --- a/platform/generic/include/andes/andes45.h
> > +++ b/platform/generic/include/andes/andes45.h
> > @@ -4,7 +4,28 @@
> >  #define CSR_MARCHID_MICROID 0xfff
> >
> >  /* Memory and Miscellaneous Registers */
> > -#define CSR_MCACHE_CTL 0x7ca
> > -#define CSR_MCCTLCOMMAND 0x7cc
> > +#define CSR_MCACHE_CTL         0x7ca
> > +#define CSR_MCCTLCOMMAND       0x7cc
> > +#define CSR_MICM_CFG           0xfc0
> > +#define CSR_MDCM_CFG           0xfc1
> > +#define CSR_MMSC_CFG           0xfc2
> > +
> > +#define MISA_20_OFFSET                 20
> > +#define MISA_20_MASK                   (0x1 << MISA_20_OFFSET)
> > +
> > +#define MICM_CFG_ISZ_OFFSET            6
> > +#define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
> > +
> > +#define MDCM_CFG_DSZ_OFFSET            6
> > +#define MDCM_CFG_DSZ_MASK              (0x7  << MDCM_CFG_DSZ_OFFSET)
> > +
> > +#define MMSC_CFG_CCTLCSR_OFFSET                16
> > +#define MMSC_CFG_CCTLCSR_MASK          (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> > +#define MMSC_IOCP_OFFSET                       47
> > +#define MMSC_IOCP_MASK                 (0x1ULL << MMSC_IOCP_OFFSET)
> > +
> > +#define MCACHE_CTL_CCTL_SUEN_OFFSET    8
> > +#define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> > +
> >
> >  #endif /* _RISCV_ANDES45_H */
> > diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> > index eee9c51..552c747 100644
> > --- a/platform/generic/renesas/rzfive/rzfive.c
> > +++ b/platform/generic/renesas/rzfive/rzfive.c
> > @@ -4,11 +4,17 @@
> >   *
> >   */
> >
> > +#include <andes/andes45.h>
> >  #include <andes45_pma.h>
> >  #include <platform_override.h>
> >  #include <sbi/sbi_domain.h>
> > +#include <sbi/sbi_error.h>
> >  #include <sbi_utils/fdt/fdt_helper.h>
> >
> > +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND      0
> > +
> > +#define ANDESTECH_SBI_EXT_ANDES                        0x900031e
> > +
> >  static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
> >         {
> >                 .pa = 0x58000000,
> > @@ -28,6 +34,46 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
> >                                          array_size(renesas_rzfive_pma_regions));
> >  }
> >
> > +static bool renesas_rzfive_cpu_cache_controlable(void)
> > +{
> > +       return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
> > +                (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
> > +               (csr_read(CSR_MISA) & MISA_20_MASK) &&
> > +               (csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
> > +               (csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
> > +}
> > +
> > +static bool renesas_rzfive_cpu_iocp_disabled(void)
> > +{
> > +       return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
> > +}
> > +
> > +static bool renesas_rzfive_apply_iocp_sw_workaround(void)
> > +{
> > +       return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
> > +}
> > +
> > +static int renesas_rzfive_vendor_ext_provider(long extid, long funcid,
> > +                                             const struct sbi_trap_regs *regs,
> > +                                             unsigned long *out_value,
> > +                                             struct sbi_trap_info *out_trap,
> > +                                             const struct fdt_match *match)
> > +{
> > +       if (extid != ANDESTECH_SBI_EXT_ANDES)
> > +               return SBI_EINVAL;
>
> This is also looking strange because you are checking Andes vendor
> extension in RZ Five sources.
>
> I will send a small patch which will help you simplify this over here. You
> can include that patch in this series in the next revision.
>
OK.

Cheers,
Prabhakar


From prabhakar.mahadev-lad.rj at bp.renesas.com  Mon Feb 13 13:51:08 2023
From: prabhakar.mahadev-lad.rj at bp.renesas.com (Lad Prabhakar)
Date: Mon, 13 Feb 2023 21:51:08 +0000
Subject: [PATCH v5 0/3] Renesas RZ/Five add support to configure PMA and add vendor specific SBI extension
Message-ID: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>

Hi All,

This patch series aims to add support to configure the PMA regions present
on the Andes AX45MP core (andes45-pma.c which can be shared with other
Andes vendors) and configures the required PMA region for RZ/Five SoC.

This patch series also adds the SBI vendor extension
RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to check if IOCP is present on
the core and if un-available it also checks if SW workaround for cache
management needs to be applied.

v4 -> v5
-> Included RB tags from Yu Chien Peter Lin
-> Moved PMA configuration code into andes folder
-> Rebased the changes on top of [0]
[0] https://patchwork.ozlabs.org/project/opensbi/list/?series=341535

RFC v3 -> v4
-> Used bit field for flags in struct andes45_pma_region
-> Renamed andes45-pma.h -> andes45_pma.h
-> Included RB tag from Lin san for patch 2/3
-> Moved CSR to platform/generic/include/andes/andes45.h
-> Included sbi/sbi_error.h in andes45-pma.c to fix build issue

RFC v2 -> RFC v3
-> Fixed review comments pointed by Lin-san
   * Used switch case macro
   * Added a check to verify the PMA size is power-of-2
   * Dropped un-needed calculation done in PMA code for NAPOT.
   * Dropped MISA macro
-> Fixed comments pointed by Himanshu
-> Fixed check patch issue

RFC v2 -> RFC v1
-> Fixed review comments pointed by Lin-san
-> Implemented PMA as library so that it can be shared

RFC v1:
https://patchwork.ozlabs.org/project/opensbi/patch/20221212094421.14556-1-prabhakar.mahadev-lad.rj at bp.renesas.com/

Cheers,
Prabhakar

Lad Prabhakar (3):
  platform: generic: renesas: rzfive: Add support to configure the PMA
  platform: generic: renesas: rzfive: Configure the PMA region
  platform: generic: renesas: rzfive: Add SBI EXT to check for enabling
    IOCP errata

 platform/generic/Kconfig                     |   3 +
 platform/generic/andes/Kconfig               |   5 +
 platform/generic/andes/andes45-pma.c         | 350 +++++++++++++++++++
 platform/generic/andes/objects.mk            |   2 +
 platform/generic/include/andes/andes45.h     |  25 +-
 platform/generic/include/andes/andes45_pma.h |  48 +++
 platform/generic/renesas/rzfive/rzfive.c     |  62 ++++
 7 files changed, 493 insertions(+), 2 deletions(-)
 create mode 100644 platform/generic/andes/Kconfig
 create mode 100644 platform/generic/andes/andes45-pma.c
 create mode 100644 platform/generic/include/andes/andes45_pma.h

-- 
2.17.1



From prabhakar.mahadev-lad.rj at bp.renesas.com  Mon Feb 13 13:51:09 2023
From: prabhakar.mahadev-lad.rj at bp.renesas.com (Lad Prabhakar)
Date: Mon, 13 Feb 2023 21:51:09 +0000
Subject: [PATCH v5 1/3] platform: generic: renesas: rzfive: Add support to configure the PMA
In-Reply-To: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <20230213215111.32017-2-prabhakar.mahadev-lad.rj@bp.renesas.com>

I/O Coherence Port (IOCP) provides an AXI interface for connecting
external non-caching masters, such as DMA controllers. The accesses
from IOCP are coherent with D-Caches and L2 Cache.

IOCP is a specification option and is disabled on the Renesas RZ/Five
SoC due to this reason IP blocks using DMA will fail.

The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
block that allows dynamic adjustment of memory attributes in the runtime.
It contains a configurable amount of PMA entries implemented as CSR
registers to control the attributes of memory locations in interest.
Below are the memory attributes supported:
* Device, Non-bufferable
* Device, bufferable
* Memory, Non-cacheable, Non-bufferable
* Memory, Non-cacheable, Bufferable
* Memory, Write-back, No-allocate
* Memory, Write-back, Read-allocate
* Memory, Write-back, Write-allocate
* Memory, Write-back, Read and Write-allocate

More info about PMA (section 10.3):
Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf

As a workaround for SoCs with IOCP disabled CMO needs to be handled by
software. Firstly OpenSBI configures the memory region as
"Memory, Non-cacheable, Bufferable" and passes this region as a global
shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
allocations happen from this region and synchronization callbacks are
implemented to synchronize when doing DMA transactions.

Example PMA region passed as a DT node from OpenSBI:
    reserved-memory {
        #address-cells = <2>;
        #size-cells = <2>;
        ranges;

        pma_resv0 at 58000000 {
            compatible = "shared-dma-pool";
            reg = <0x0 0x58000000 0x0 0x08000000>;
            no-map;
            linux,dma-default;
        };
    };

Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
 platform/generic/Kconfig                     |   2 +
 platform/generic/andes/Kconfig               |   5 +
 platform/generic/andes/andes45-pma.c         | 350 +++++++++++++++++++
 platform/generic/andes/objects.mk            |   2 +
 platform/generic/include/andes/andes45_pma.h |  48 +++
 5 files changed, 407 insertions(+)
 create mode 100644 platform/generic/andes/Kconfig
 create mode 100644 platform/generic/andes/andes45-pma.c
 create mode 100644 platform/generic/include/andes/andes45_pma.h

diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index c7f198a..39fb4e9 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -50,4 +50,6 @@ config PLATFORM_STARFIVE_JH7110
 	bool "StarFive JH7110 support"
 	default n
 
+source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
+
 endif
diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
new file mode 100644
index 0000000..3ad4e4c
--- /dev/null
+++ b/platform/generic/andes/Kconfig
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: BSD-2-Clause
+
+config ANDES45_PMA
+	bool "Andes PMA support"
+	default n
diff --git a/platform/generic/andes/andes45-pma.c b/platform/generic/andes/andes45-pma.c
new file mode 100644
index 0000000..2745bc3
--- /dev/null
+++ b/platform/generic/andes/andes45-pma.c
@@ -0,0 +1,350 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2023 Renesas Electronics Corp.
+ *
+ * Copyright (c) 2020 Andes Technology Corporation
+ *
+ * Authors:
+ *      Nick Hu <nickhu at andestech.com>
+ *      Nylon Chen <nylon7 at andestech.com>
+ *      Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
+ */
+
+#include <andes/andes45_pma.h>
+#include <libfdt.h>
+#include <sbi/riscv_asm.h>
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_console.h>
+#include <sbi/sbi_error.h>
+#include <sbi_utils/fdt/fdt_helper.h>
+
+/* Configuration Registers */
+#define ANDES45_CSR_MMSC_CFG		0xFC2
+#define ANDES45_CSR_MMSC_PPMA_OFFSET	(1 << 30)
+
+#define ANDES45_PMAADDR_0		0xBD0
+
+#define ANDES45_PMACFG_0		0xBC0
+
+static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
+{
+#define switchcase_pma_cfg_read(__pma_cfg_off, __val)		\
+	case __pma_cfg_off:					\
+		__val = csr_read(__pma_cfg_off);		\
+		break;
+#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val)		\
+	switchcase_pma_cfg_read(__pma_cfg_off + 0, __val)	\
+	switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
+
+	unsigned long ret = 0;
+
+	switch (pma_cfg_off) {
+	switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
+
+	default:
+		sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
+		break;
+	}
+
+	return ret;
+
+#undef switchcase_pma_cfg_read_2
+#undef switchcase_pma_cfg_read
+}
+
+static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
+{
+#define switchcase_pma_cfg_write(__pma_cfg_off, __val)		\
+	case __pma_cfg_off:					\
+		csr_write(__pma_cfg_off, __val);		\
+		break;
+#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val)	\
+	switchcase_pma_cfg_write(__pma_cfg_off + 0, __val)	\
+	switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
+
+	switch (pma_cfg_off) {
+	switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
+
+	default:
+		sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
+		break;
+	}
+
+#undef switchcase_pma_cfg_write_2
+#undef switchcase_pma_cfg_write
+}
+
+static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
+{
+#define switchcase_pma_write(__pma_addr_off, __val)		\
+	case __pma_addr_off:					\
+		csr_write(__pma_addr_off, __val);		\
+		break;
+#define switchcase_pma_write_2(__pma_addr_off, __val)		\
+	switchcase_pma_write(__pma_addr_off + 0, __val)		\
+	switchcase_pma_write(__pma_addr_off + 1, __val)
+#define switchcase_pma_write_4(__pma_addr_off, __val)		\
+	switchcase_pma_write_2(__pma_addr_off + 0, __val)	\
+	switchcase_pma_write_2(__pma_addr_off + 2, __val)
+#define switchcase_pma_write_8(__pma_addr_off, __val)		\
+	switchcase_pma_write_4(__pma_addr_off + 0, __val)	\
+	switchcase_pma_write_4(__pma_addr_off + 4, __val)
+#define switchcase_pma_write_16(__pma_addr_off, __val)		\
+	switchcase_pma_write_8(__pma_addr_off + 0, __val)	\
+	switchcase_pma_write_8(__pma_addr_off + 8, __val)
+
+	switch (pma_addr_off) {
+	switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
+
+	default:
+		sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
+		break;
+	}
+
+#undef switchcase_pma_write_16
+#undef switchcase_pma_write_8
+#undef switchcase_pma_write_4
+#undef switchcase_pma_write_2
+#undef switchcase_pma_write
+}
+
+static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
+{
+#define switchcase_pma_read(__pma_addr_off, __val)		\
+	case __pma_addr_off:					\
+		__val = csr_read(__pma_addr_off);		\
+		break;
+#define switchcase_pma_read_2(__pma_addr_off, __val)		\
+	switchcase_pma_read(__pma_addr_off + 0, __val)		\
+	switchcase_pma_read(__pma_addr_off + 1, __val)
+#define switchcase_pma_read_4(__pma_addr_off, __val)		\
+	switchcase_pma_read_2(__pma_addr_off + 0, __val)	\
+	switchcase_pma_read_2(__pma_addr_off + 2, __val)
+#define switchcase_pma_read_8(__pma_addr_off, __val)		\
+	switchcase_pma_read_4(__pma_addr_off + 0, __val)	\
+	switchcase_pma_read_4(__pma_addr_off + 4, __val)
+#define switchcase_pma_read_16(__pma_addr_off, __val)		\
+	switchcase_pma_read_8(__pma_addr_off + 0, __val)	\
+	switchcase_pma_read_8(__pma_addr_off + 8, __val)
+
+	unsigned long ret = 0;
+
+	switch (pma_addr_off) {
+	switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
+
+	default:
+		sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
+		break;
+	}
+
+	return ret;
+
+#undef switchcase_pma_read_16
+#undef switchcase_pma_read_8
+#undef switchcase_pma_read_4
+#undef switchcase_pma_read_2
+#undef switchcase_pma_read
+}
+
+static unsigned long
+andes45_pma_setup(const struct andes45_pma_region *pma_region,
+		  unsigned int entry_id)
+{
+	unsigned long size = pma_region->size;
+	unsigned long addr = pma_region->pa;
+	unsigned int pma_cfg_addr;
+	unsigned long pmacfg_val;
+	unsigned long pmaaddr;
+	char *pmaxcfg;
+
+	/* Check for 4KiB granularity */
+	if (size < (1 << 12))
+		return SBI_EINVAL;
+
+	/* Check size is power of 2 */
+	if (size & (size - 1))
+		return SBI_EINVAL;
+
+	if (entry_id > 15)
+		return SBI_EINVAL;
+
+	if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
+		return SBI_EINVAL;
+
+	if ((addr & (size - 1)) != 0)
+		return SBI_EINVAL;
+
+	pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
+	pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
+	pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
+	*pmaxcfg = 0;
+	*pmaxcfg = pma_region->flags;
+
+	andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
+
+	pmaaddr = (addr >> 2) + (size >> 3) - 1;
+
+	andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
+
+	return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
+			pmaaddr : SBI_EINVAL;
+}
+
+static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
+				unsigned int index, int parent)
+{
+	int na = fdt_address_cells(fdt, 0);
+	int ns = fdt_size_cells(fdt, 0);
+	static bool dma_default = false;
+	fdt32_t addr_high, addr_low;
+	fdt32_t size_high, size_low;
+	int subnode, err;
+	fdt32_t reg[4];
+	fdt32_t *val;
+	char name[32];
+
+	addr_high = (u64)pma->pa >> 32;
+	addr_low = pma->pa;
+	size_high = (u64)pma->size >> 32;
+	size_low = pma->size;
+
+	if (na > 1 && addr_high)
+		sbi_snprintf(name, sizeof(name),
+			     "pma_resv%d@%x,%x", index,
+			     addr_high, addr_low);
+	else
+		sbi_snprintf(name, sizeof(name),
+			     "pma_resv%d@%x", index,
+			     addr_low);
+
+	subnode = fdt_add_subnode(fdt, parent, name);
+	if (subnode < 0)
+		return subnode;
+
+	if (pma->shared_dma) {
+		err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
+		if (err < 0)
+			return err;
+	}
+
+	if (pma->no_map) {
+		err = fdt_setprop_empty(fdt, subnode, "no-map");
+		if (err < 0)
+			return err;
+	}
+
+	/* Linux allows single linux,dma-default region. */
+	if (pma->dma_default) {
+		if (dma_default)
+			return SBI_EINVAL;
+
+		err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
+		if (err < 0)
+			return err;
+		dma_default = true;
+	}
+
+	/* encode the <reg> property value */
+	val = reg;
+	if (na > 1)
+		*val++ = cpu_to_fdt32(addr_high);
+	*val++ = cpu_to_fdt32(addr_low);
+	if (ns > 1)
+		*val++ = cpu_to_fdt32(size_high);
+	*val++ = cpu_to_fdt32(size_low);
+
+	err = fdt_setprop(fdt, subnode, "reg", reg,
+			  (na + ns) * sizeof(fdt32_t));
+	if (err < 0)
+		return err;
+
+	return 0;
+}
+
+static int andes45_fdt_reserved_memory_fixup(void *fdt,
+					     const struct andes45_pma_region *pma,
+					     unsigned int entry)
+{
+	int parent;
+
+	/* try to locate the reserved memory node */
+	parent = fdt_path_offset(fdt, "/reserved-memory");
+	if (parent < 0) {
+		int na = fdt_address_cells(fdt, 0);
+		int ns = fdt_size_cells(fdt, 0);
+		int err;
+
+		/* if such node does not exist, create one */
+		parent = fdt_add_subnode(fdt, 0, "reserved-memory");
+		if (parent < 0)
+			return parent;
+
+		err = fdt_setprop_empty(fdt, parent, "ranges");
+		if (err < 0)
+			return err;
+
+		err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
+		if (err < 0)
+			return err;
+
+		err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
+		if (err < 0)
+			return err;
+	}
+
+	return andes45_fdt_pma_resv(fdt, pma, entry, parent);
+}
+
+int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
+			      unsigned int pma_regions_count)
+{
+	unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
+	unsigned int dt_populate_cnt;
+	unsigned int i, j;
+	unsigned long pa;
+	void *fdt;
+	int ret;
+
+	if (!pma_regions || !pma_regions_count)
+		return 0;
+
+	if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
+		return SBI_EINVAL;
+
+	if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
+		return SBI_ENOTSUPP;
+
+	/* Configure the PMA regions */
+	for (i = 0; i < pma_regions_count; i++) {
+		pa = andes45_pma_setup(&pma_regions[i], i);
+		if (pa == SBI_EINVAL)
+			return SBI_EINVAL;
+	}
+
+	dt_populate_cnt = 0;
+	for (i = 0; i < pma_regions_count; i++) {
+		if (!pma_regions[i].dt_populate)
+			continue;
+		dt_populate_cnt++;
+	}
+
+	if (!dt_populate_cnt)
+		return 0;
+
+	fdt = fdt_get_address();
+
+	ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
+	if (ret < 0)
+		return ret;
+
+	for (i = 0, j = 0; i < pma_regions_count; i++) {
+		if (!pma_regions[i].dt_populate)
+			continue;
+
+		ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
index 28275ef..ea6b561 100644
--- a/platform/generic/andes/objects.mk
+++ b/platform/generic/andes/objects.mk
@@ -4,3 +4,5 @@
 
 carray-platform_override_modules-$(CONFIG_PLATFORM_ANDES_AE350) += andes_ae350
 platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
+
+platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
diff --git a/platform/generic/include/andes/andes45_pma.h b/platform/generic/include/andes/andes45_pma.h
new file mode 100644
index 0000000..37ec77c
--- /dev/null
+++ b/platform/generic/include/andes/andes45_pma.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023 Renesas Electronics Corp.
+ */
+
+#ifndef _ANDES45_PMA_H_
+#define _ANDES45_PMA_H_
+
+#include <sbi/sbi_types.h>
+
+#define ANDES45_MAX_PMA_REGIONS			16
+
+/* Naturally aligned power of 2 region */
+#define ANDES45_PMACFG_ETYP_NAPOT		3
+
+/* Memory, Non-cacheable, Bufferable */
+#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF	(3 << 2)
+
+/**
+ * struct andes45_pma_region - Describes PMA regions
+ *
+ * @pa: Address to be configured in the PMA
+ * @size: Size of the region
+ * @flags: Flags to be set for the PMA region
+ * @dt_populate: Boolean flag indicating if the DT entry should be
+ *               populated for the given PMA region
+ * @shared_dma: Boolean flag if set "shared-dma-pool" property will
+ *              be set in the DT node
+ * @no_map: Boolean flag if set "no-map" property will be set in the
+ *          DT node
+ * @dma_default: Boolean flag if set "linux,dma-default" property will
+ *              be set in the DT node. Note Linux expects single node
+ *              with this property set.
+ */
+struct andes45_pma_region {
+	unsigned long pa;
+	unsigned long size;
+	u8 flags:7;
+	bool dt_populate;
+	bool shared_dma;
+	bool no_map;
+	bool dma_default;
+};
+
+int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
+			      unsigned int pma_regions_count);
+
+#endif /* _ANDES45_PMA_H_ */
-- 
2.17.1



From prabhakar.mahadev-lad.rj at bp.renesas.com  Mon Feb 13 13:51:10 2023
From: prabhakar.mahadev-lad.rj at bp.renesas.com (Lad Prabhakar)
Date: Mon, 13 Feb 2023 21:51:10 +0000
Subject: [PATCH v5 2/3] platform: generic: renesas: rzfive: Configure the PMA region
In-Reply-To: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <20230213215111.32017-3-prabhakar.mahadev-lad.rj@bp.renesas.com>

On the Renesas RZ/Five SoC by default we want to configure 128MiB of memory
ranging from 0x58000000 as a non-cacheable + bufferable region in the PMA
and populate this region as PMA reserve DT node with shared DMA pool and
no-map flags set so that Linux drivers requesting any DMA'able memory go
through this region.

PMA node passed to the above stack:

        reserved-memory {
            #address-cells = <2>;
            #size-cells = <2>;
            ranges;

            pma_resv0 at 58000000 {
                compatible = "shared-dma-pool";
                reg = <0x0 0x58000000 0x0 0x08000000>;
                no-map;
                linux,dma-default;
            };
        };

Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
 platform/generic/Kconfig                 |  1 +
 platform/generic/renesas/rzfive/rzfive.c | 21 +++++++++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
index 39fb4e9..1f4f8e1 100644
--- a/platform/generic/Kconfig
+++ b/platform/generic/Kconfig
@@ -35,6 +35,7 @@ config PLATFORM_ANDES_AE350
 
 config PLATFORM_RENESAS_RZFIVE
 	bool "Renesas RZ/Five support"
+	select ANDES45_PMA
 	default n
 
 config PLATFORM_SIFIVE_FU540
diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
index ee9c9c4..4d71d0d 100644
--- a/platform/generic/renesas/rzfive/rzfive.c
+++ b/platform/generic/renesas/rzfive/rzfive.c
@@ -4,10 +4,30 @@
  *
  */
 
+#include <andes/andes45_pma.h>
 #include <platform_override.h>
 #include <sbi/sbi_domain.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 
+static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
+	{
+		.pa = 0x58000000,
+		.size = 0x8000000,
+		.flags = ANDES45_PMACFG_ETYP_NAPOT |
+			 ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF,
+		.dt_populate = true,
+		.shared_dma = true,
+		.no_map = true,
+		.dma_default = true,
+	},
+};
+
+static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *match)
+{
+	return andes45_pma_setup_regions(renesas_rzfive_pma_regions,
+					 array_size(renesas_rzfive_pma_regions));
+}
+
 int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
 {
 	/*
@@ -34,4 +54,5 @@ static const struct fdt_match renesas_rzfive_match[] = {
 const struct platform_override renesas_rzfive = {
 	.match_table = renesas_rzfive_match,
 	.early_init = renesas_rzfive_early_init,
+	.final_init = renesas_rzfive_final_init,
 };
-- 
2.17.1



From prabhakar.mahadev-lad.rj at bp.renesas.com  Mon Feb 13 13:51:11 2023
From: prabhakar.mahadev-lad.rj at bp.renesas.com (Lad Prabhakar)
Date: Mon, 13 Feb 2023 21:51:11 +0000
Subject: [PATCH v5 3/3] platform: generic: renesas: rzfive: Add SBI EXT to check for enabling IOCP errata
In-Reply-To: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <20230213215111.32017-4-prabhakar.mahadev-lad.rj@bp.renesas.com>

I/O Coherence Port (IOCP) provides an AXI interface for connecting
external non-caching masters, such as DMA controllers. The accesses
from IOCP are coherent with D-Caches and L2 Cache.

IOCP is a specification option and is disabled on the Renesas RZ/Five
SoC due to this reason IP blocks using DMA will fail.

As a workaround for SoCs with IOCP disabled CMO needs to be handled by
software. Firstly OpenSBI configures the memory region as
"Memory, Non-cacheable, Bufferable" and passes this region as a global
shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
allocations happen from this region and synchronization callbacks are
implemented to synchronize when doing DMA transactions.

RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
errata should be applied to handle cache management.

Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
 platform/generic/include/andes/andes45.h | 25 +++++++++++++--
 platform/generic/renesas/rzfive/rzfive.c | 41 ++++++++++++++++++++++++
 2 files changed, 64 insertions(+), 2 deletions(-)

diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
index 08b3d18..d5d265f 100644
--- a/platform/generic/include/andes/andes45.h
+++ b/platform/generic/include/andes/andes45.h
@@ -4,7 +4,28 @@
 #define CSR_MARCHID_MICROID 0xfff
 
 /* Memory and Miscellaneous Registers */
-#define CSR_MCACHE_CTL 0x7ca
-#define CSR_MCCTLCOMMAND 0x7cc
+#define CSR_MCACHE_CTL		0x7ca
+#define CSR_MCCTLCOMMAND	0x7cc
+#define CSR_MICM_CFG		0xfc0
+#define CSR_MDCM_CFG		0xfc1
+#define CSR_MMSC_CFG		0xfc2
+
+#define MISA_20_OFFSET			20
+#define MISA_20_MASK			(0x1 << MISA_20_OFFSET)
+
+#define MICM_CFG_ISZ_OFFSET		6
+#define MICM_CFG_ISZ_MASK		(0x7  << MICM_CFG_ISZ_OFFSET)
+
+#define MDCM_CFG_DSZ_OFFSET		6
+#define MDCM_CFG_DSZ_MASK		(0x7  << MDCM_CFG_DSZ_OFFSET)
+
+#define MMSC_CFG_CCTLCSR_OFFSET		16
+#define MMSC_CFG_CCTLCSR_MASK		(0x1 << MMSC_CFG_CCTLCSR_OFFSET)
+#define MMSC_IOCP_OFFSET			47
+#define MMSC_IOCP_MASK			(0x1ULL << MMSC_IOCP_OFFSET)
+
+#define MCACHE_CTL_CCTL_SUEN_OFFSET	8
+#define MCACHE_CTL_CCTL_SUEN_MASK	(0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
+
 
 #endif /* _RISCV_ANDES45_H */
diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
index 4d71d0d..9530d28 100644
--- a/platform/generic/renesas/rzfive/rzfive.c
+++ b/platform/generic/renesas/rzfive/rzfive.c
@@ -4,11 +4,14 @@
  *
  */
 
+#include <andes/andes45.h>
 #include <andes/andes45_pma.h>
 #include <platform_override.h>
 #include <sbi/sbi_domain.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 
+#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND	0
+
 static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
 	{
 		.pa = 0x58000000,
@@ -28,6 +31,43 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
 					 array_size(renesas_rzfive_pma_regions));
 }
 
+static bool renesas_rzfive_cpu_cache_controlable(void)
+{
+	return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
+		 (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
+		(csr_read(CSR_MISA) & MISA_20_MASK) &&
+		(csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
+		(csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
+}
+
+static bool renesas_rzfive_cpu_iocp_disabled(void)
+{
+	return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
+}
+
+static bool renesas_rzfive_apply_iocp_sw_workaround(void)
+{
+	return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
+}
+
+static int renesas_rzfive_vendor_ext_provider(long funcid,
+					      const struct sbi_trap_regs *regs,
+					      unsigned long *out_value,
+					      struct sbi_trap_info *out_trap,
+					      const struct fdt_match *match)
+{
+	switch (funcid) {
+	case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
+		*out_value = renesas_rzfive_apply_iocp_sw_workaround();
+		break;
+
+	default:
+		break;
+	}
+
+	return 0;
+}
+
 int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
 {
 	/*
@@ -55,4 +95,5 @@ const struct platform_override renesas_rzfive = {
 	.match_table = renesas_rzfive_match,
 	.early_init = renesas_rzfive_early_init,
 	.final_init = renesas_rzfive_final_init,
+	.vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
 };
-- 
2.17.1



From wiagn233 at outlook.com  Wed Feb 15 10:26:28 2023
From: wiagn233 at outlook.com (Shengyu Qu)
Date: Thu, 16 Feb 2023 02:26:28 +0800
Subject: [PATCH] docs: fix typo in fw.md
Message-ID: <TY3P286MB2611F8A2445A0A2ADE1A843498A39@TY3P286MB2611.JPNP286.PROD.OUTLOOK.COM>

In docs/firmware/fw.md, there's a configuration parameter called
FW_TEXT_ADDR, which actually should be FW_TEXT_START, so fix it.

Signed-off-by: Shengyu Qu <wiagn233 at outlook.com>
---
 docs/firmware/fw.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/firmware/fw.md b/docs/firmware/fw.md
index 18faaa9..38351c8 100644
--- a/docs/firmware/fw.md
+++ b/docs/firmware/fw.md
@@ -61,7 +61,7 @@ Firmware Configuration and Compilation
 All firmware types support the following common compile time configuration
 parameters:
 
-* **FW_TEXT_ADDR** - Defines the execution address of the OpenSBI firmware.
+* **FW_TEXT_START** - Defines the execution address of the OpenSBI firmware.
   This configuration parameter is mandatory.
 * **FW_FDT_PATH** - Path to an external flattened device tree binary file to
   be embedded in the *.rodata* section of the final firmware. If this option
-- 
2.25.1



From wxjstz at 126.com  Wed Feb 15 17:57:32 2023
From: wxjstz at 126.com (Xiang W)
Date: Thu, 16 Feb 2023 09:57:32 +0800
Subject: [PATCH] docs: fix typo in fw.md
In-Reply-To: <TY3P286MB2611F8A2445A0A2ADE1A843498A39@TY3P286MB2611.JPNP286.PROD.OUTLOOK.COM>
References: <TY3P286MB2611F8A2445A0A2ADE1A843498A39@TY3P286MB2611.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <8f0529d446e278a0276c78635cbb5888220f90aa.camel@126.com>

? 2023-02-16???? 02:26 +0800?Shengyu Qu???
> In docs/firmware/fw.md, there's a configuration parameter called
> FW_TEXT_ADDR, which actually should be FW_TEXT_START, so fix it.
> 
> Signed-off-by: Shengyu Qu <wiagn233 at outlook.com>
LGTM

Reviewed-by: Xiang W <wxjstz at 126.com>
> ---
> ?docs/firmware/fw.md | 2 +-
> ?1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/docs/firmware/fw.md b/docs/firmware/fw.md
> index 18faaa9..38351c8 100644
> --- a/docs/firmware/fw.md
> +++ b/docs/firmware/fw.md
> @@ -61,7 +61,7 @@ Firmware Configuration and Compilation
> ?All firmware types support the following common compile time configuration
> ?parameters:
> ?
> -* **FW_TEXT_ADDR** - Defines the execution address of the OpenSBI firmware.
> +* **FW_TEXT_START** - Defines the execution address of the OpenSBI firmware.
> ?? This configuration parameter is mandatory.
> ?* **FW_FDT_PATH** - Path to an external flattened device tree binary file to
> ?? be embedded in the *.rodata* section of the final firmware. If this option
> -- 
> 2.25.1
> 
> 



From minda.chen at starfivetech.com  Thu Feb 16 01:21:26 2023
From: minda.chen at starfivetech.com (minda.chen)
Date: Thu, 16 Feb 2023 17:21:26 +0800
Subject: [PATCH] gpio/starfive: add gpio driver and support gpio reset
Message-ID: <20230216092126.99022-1-minda.chen@starfivetech.com>

Add gpio driver and gpio reset function in Starfive
JH7110 SOC platform.

Signed-off-by: minda.chen <minda.chen at starfivetech.com>
---
 lib/utils/gpio/Kconfig             |   3 +
 lib/utils/gpio/fdt_gpio_starfive.c | 117 +++++++++++++++++++++++++++++
 lib/utils/gpio/objects.mk          |   3 +
 platform/generic/configs/defconfig |   1 +
 4 files changed, 124 insertions(+)
 create mode 100644 lib/utils/gpio/fdt_gpio_starfive.c

diff --git a/lib/utils/gpio/Kconfig b/lib/utils/gpio/Kconfig
index 38a9d75..1b338a6 100644
--- a/lib/utils/gpio/Kconfig
+++ b/lib/utils/gpio/Kconfig
@@ -14,6 +14,9 @@ config FDT_GPIO_SIFIVE
 	bool "SiFive GPIO FDT driver"
 	default n
 
+config FDT_GPIO_STARFIVE
+	bool "StarFive GPIO FDT driver"
+	default n
 endif
 
 config GPIO
diff --git a/lib/utils/gpio/fdt_gpio_starfive.c b/lib/utils/gpio/fdt_gpio_starfive.c
new file mode 100644
index 0000000..18cca72
--- /dev/null
+++ b/lib/utils/gpio/fdt_gpio_starfive.c
@@ -0,0 +1,117 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 Starfive
+ *
+ * Authors:
+ *   Minda.chen <Minda.chen at starfivetech.com>
+ */
+
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_console.h>
+#include <sbi_utils/fdt/fdt_helper.h>
+#include <sbi_utils/gpio/fdt_gpio.h>
+
+#define STARFIVE_GPIO_CHIP_MAX		2
+#define STARFIVE_GPIO_PINS_DEF		64
+#define STARFIVE_GPIO_OUTVAL		0x40
+#define STARFIVE_GPIO_MASK		0xff
+#define STARFIVE_GPIO_REG_SHIFT_MASK	0x3
+#define STARFIVE_GPIO_SHIFT_BITS	0x3
+
+struct starfive_gpio_chip {
+	unsigned long addr;
+	struct gpio_chip chip;
+};
+
+static unsigned int starfive_gpio_chip_count;
+static struct starfive_gpio_chip starfive_gpio_chip_array[STARFIVE_GPIO_CHIP_MAX];
+
+static int starfive_gpio_direction_output(struct gpio_pin *gp, int value)
+{
+	u32 val;
+	unsigned long reg_addr;
+	u32 bit_mask, shift_bits;
+	struct starfive_gpio_chip *chip =
+		container_of(gp->chip, struct starfive_gpio_chip, chip);
+
+	/* set out en*/
+	reg_addr = chip->addr + gp->offset;
+	reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
+
+	val = readl((void *)(reg_addr));
+	shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
+		<< STARFIVE_GPIO_SHIFT_BITS;
+	bit_mask = STARFIVE_GPIO_MASK << shift_bits;
+
+	val = readl((void *)reg_addr);
+	val &= ~bit_mask;
+	writel(val, (void *)reg_addr);
+
+	return 0;
+}
+
+static void starfive_gpio_set(struct gpio_pin *gp, int value)
+{
+	u32 val;
+	unsigned long reg_addr;
+	u32 bit_mask, shift_bits;
+	struct starfive_gpio_chip *chip =
+		container_of(gp->chip, struct starfive_gpio_chip, chip);
+
+	reg_addr = chip->addr + gp->offset;
+	reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
+
+	shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
+		<< STARFIVE_GPIO_SHIFT_BITS;
+	bit_mask = STARFIVE_GPIO_MASK << shift_bits;
+	/* set output value */
+	val = readl((void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
+	val &= ~bit_mask;
+	val |= value << shift_bits;
+	writel(val, (void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
+}
+
+extern struct fdt_gpio fdt_gpio_starfive;
+
+static int starfive_gpio_init(void *fdt, int nodeoff, u32 phandle,
+			      const struct fdt_match *match)
+{
+	int rc;
+	struct starfive_gpio_chip *chip;
+	u64 addr;
+
+	if (starfive_gpio_chip_count >= STARFIVE_GPIO_CHIP_MAX)
+		return SBI_ENOSPC;
+	chip = &starfive_gpio_chip_array[starfive_gpio_chip_count];
+
+	rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
+	if (rc)
+		return rc;
+
+	chip->addr = addr;
+	chip->chip.driver = &fdt_gpio_starfive;
+	chip->chip.id = phandle;
+	chip->chip.ngpio = STARFIVE_GPIO_PINS_DEF;
+	chip->chip.direction_output = starfive_gpio_direction_output;
+	chip->chip.set = starfive_gpio_set;
+	rc = gpio_chip_add(&chip->chip);
+	if (rc)
+		return rc;
+
+	starfive_gpio_chip_count++;
+	return 0;
+}
+
+static const struct fdt_match starfive_gpio_match[] = {
+	{ .compatible = "starfive,jh7110-sys-pinctrl" },
+	{ .compatible = "starfive,iomux-pinctrl" },
+	{ },
+};
+
+struct fdt_gpio fdt_gpio_starfive = {
+	.match_table = starfive_gpio_match,
+	.xlate = fdt_gpio_simple_xlate,
+	.init = starfive_gpio_init,
+};
diff --git a/lib/utils/gpio/objects.mk b/lib/utils/gpio/objects.mk
index eedd699..8f4a6a8 100644
--- a/lib/utils/gpio/objects.mk
+++ b/lib/utils/gpio/objects.mk
@@ -13,4 +13,7 @@ libsbiutils-objs-$(CONFIG_FDT_GPIO) += gpio/fdt_gpio_drivers.o
 carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_SIFIVE) += fdt_gpio_sifive
 libsbiutils-objs-$(CONFIG_FDT_GPIO_SIFIVE) += gpio/fdt_gpio_sifive.o
 
+carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_STARFIVE) += fdt_gpio_starfive
+libsbiutils-objs-$(CONFIG_FDT_GPIO_STARFIVE) += gpio/fdt_gpio_starfive.o
+
 libsbiutils-objs-$(CONFIG_GPIO) += gpio/gpio.o
diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
index 4b0842e..2ad953d 100644
--- a/platform/generic/configs/defconfig
+++ b/platform/generic/configs/defconfig
@@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
 CONFIG_PLATFORM_STARFIVE_JH7110=y
 CONFIG_FDT_GPIO=y
 CONFIG_FDT_GPIO_SIFIVE=y
+CONFIG_FDT_GPIO_STARFIVE=y
 CONFIG_FDT_I2C=y
 CONFIG_FDT_I2C_SIFIVE=y
 CONFIG_FDT_IPI=y
-- 
2.17.1



From liushiwei at eswincomputing.com  Thu Feb 16 03:40:39 2023
From: liushiwei at eswincomputing.com (liushiwei)
Date: Thu, 16 Feb 2023 19:40:39 +0800
Subject: [PATCH 1/1] Add RISC-V TEE support
Message-ID: <007e01d941fb$7ef6f1d0$7ce4d570$@eswincomputing.com>

Hi? Atish
	I tried to email tech-prs at lists.riscv.org and sig-trusted-computing at lists.riscv.org, but both returned. My attempts to send a patch directly also returned.
	Looking at this reason, does it seem like I have to join some group before I can send an email?

?Connection to the remote recipient's server was denied for unknown reason.
SMTP through SDN 37, SMTP: (Proxy)Host lists.riscv.org said 510 5.1.1 Your email address, liushiwei at eswincomputing.com, 
is not subscribed to that group.  To subscribe, send an email to sig-trusted-computing+subscribe at lists.riscv.org, 
or visit https://lists.riscv.org/g/sig-trusted-computing?

Regards,
Liushiwei
-----????-----
???: Atish Patra [mailto:atishp at atishpatra.org] 
????: 2023?2?8? 6:37
???: liushiwei <liushiwei at eswincomputing.com>
??: Himanshu Chauhan <hchauhan at ventanamicro.com>; opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
??: Re: Re: [PATCH 1/1] Add RISC-V TEE support

On Sat, Jan 28, 2023 at 12:33 AM liushiwei <liushiwei at eswincomputing.com> wrote:
>
> Hi, Atish
>         Thank you for your reply.
>         I submitted my ideas to https://github.com/riscv-non-isa/riscv-sbi-doc/pull/106 as requested by other members of the email.
>         You can preview them at https://github.com/liushiwei007/riscv-sbi-doc/blob/master/riscv-sbi.adoc .
>         I'm not sure if I need to send an email to tech-prs at lists.riscv.org or sig-trusted-computing at lists.riscv.org after submitting to riscv-sbi-doc?
>         If so, Is it to send the patch of opensbi I submitted before?
>

The Trusted computing SIG evaluates the overall merit and direction of any TEE related specifications.
The PRS TG will focus more on the SBI specification related parts.
I would recommend you to present your work to Trusted Computing SIG first so that everybody agrees on the overall direction for OP-TEE support in RISC-V.

> Regards,
> Liushiwei
>
>
> -----????-----
> ???: Atish Patra [mailto:atishp at atishpatra.org]
> ????: 2023?1?25? 3:12
> ???: liushiwei <liushiwei at eswincomputing.com>
> ??: Himanshu Chauhan <hchauhan at ventanamicro.com>; 
> opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> ??: Re: ??: [PATCH 1/1] Add RISC-V TEE support
>
> On Wed, Jan 11, 2023 at 11:08 PM liushiwei <liushiwei at eswincomputing.com> wrote:
> >
> > Hi, Himanshu ,  These are my description:
> >
> > In my design, The entire linux space is called REE (Rich Execution 
> > Environment), and TEE OS includes its user state called TEE (Trusted execution environment).
> > adding TEE functionality to opensbi requires two configuration items.
> > For example:
> >    add CONFIG_SBI_ECALL_TEE=y in platform/generic/configs/defconfig file.
> >    add CONFIG_TEE_LOAD_ADDR=0x27C000000 platform/generic/objects.mk.
> >    The value of CONFIG_TEE_LOAD_ADDR depends on the actual memory layout,
> >    It's a physical address.
> >
> > When TEE is configured, opensbi adds the following functionality:
> > 1. Boot TEE OS.
> >    If TEE is enabled, tee_os_init() is called before entering
> >    sbi_hart_switch_mode(). tee_os_init saves the current context,
> >    sets a new trap stack address, and runs to CONFIG_TEE_LOAD_ADDR as configured
> >    to complete TEE OS initialization. TEE OS returns via ecall, Go to opensbi
> >    sbi_ecall_tee_handler and use the characteristic value RETURN_ENTRY_DONE
> >    to indicate the return after the TEE OS completes booting. Check whether the
> >    TEE OS boot successfully according to the parameter. If fails,
> >    the system enters wfi and terminates the startup process of opensbi. If successful,
> >    it returns REE to switch into TEE's vector. then trap stack memory is restored,
> >    the context is restored, the tee_os_init call is returned,
> >    and the rest of the process is performed.
> > 2. REE switches to TEE.
> >    When the TEE OS boot successfully, it returns an entry vector for REE into the TEE.
> >    It represents various entry points into TEE OS and is stored in opensbi's global variables.
> >    It is the optee_vectors_t structure, which contains nine entry cases:
> >       1. yield_smc_entry;
> >       2. fast_smc_entry;
> >       3. cpu_on_entry;
> >       4. cpu_off_entry;
> >       5. cpu_resume_entry;
> >       6. cpu_suspend_entry;
> >       7. fiq_entry;
> >       8. system_off_entry;
> >       9. system_reset_entry;
> >    yield_smc_entry means that this function entry TEE will start the thread function
> >    and enter the user state of TEE. It may also switch back to REE with RPC function,
> >    and then return to TEE after REE completes the corresponding function.
> >    For the REE process that sent you this call, it may cause sleep.
> >
> >    fast_smc_entry indicates that this is a quick function that returns after
> >    the TEE OS does something, and that it does not cause the caller to sleep.
> >
> >    yield_smc_entry and fast_smc_entry return opensbi use eigenvalue RETURN_CALL_DONE.
> >    for yield_smc_entry, whether the call returns or the RPC returns is decided by linux.
> >
> >    cpu_on_entry/cpu_off_entry/cpu_resume_entry/cpu_suspend_entry and
> >    fiq_entry/system_off_entry/system_reset_entry they are not implemented currently.
> >
> > 3. TEE switches to REE.
> >    opensbi needs to save the context when REE enters the TEE,
> >    and restore the context when it returns from the TEE.
> >
> > 4. TEE/REE Request a special function.
> >    We have some specific functions, like get hartid from TEE,
> >    it need save and restore the TEE context.
> >
> > Data structure.
> > opensbi adds the sbi_save_context declaration
> >    struct sbi_save_context {
> >       struct sbi_trap_regs regs;
> >       unsigned long sepc;
> >       unsigned long satp;
> >       unsigned long sstatus;
> >       unsigned long sie;
> >       unsigned long stvec;
> >       unsigned long sscratch;
> >       unsigned long scounteren;
> >       unsigned long scause;
> >       unsigned long stval;
> >       unsigned long sip;
> >    };
> >    sbi_save_context include sbi_trap_regs and S mode csr. Used to 
> > hold the context of TEE or REE
> >
> >    typedef struct optee_vectors {
> >       optee_vector_isn_t yield_smc_entry;
> >       optee_vector_isn_t fast_smc_entry;
> >       optee_vector_isn_t cpu_on_entry;
> >       optee_vector_isn_t cpu_off_entry;
> >       optee_vector_isn_t cpu_resume_entry;
> >       optee_vector_isn_t cpu_suspend_entry;
> >       optee_vector_isn_t fiq_entry;
> >       optee_vector_isn_t system_off_entry;
> >       optee_vector_isn_t system_reset_entry;
> >    } optee_vectors_t;
> >    optee_vectors represent the various entry points into TEE OS.
> >
> > Variables defined by opensbi
> >    optee_vectors_t *optee_vector_table;
> >    struct sbi_save_context nsec_cpu_context[OPTEED_CORE_COUNT];
> >    struct sbi_save_context sec_cpu_context[OPTEED_CORE_COUNT];
> >    struct sbi_trap_regs cpu_start_context[OPTEED_CORE_COUNT];
> >    typedef ulong tee_tmp_trap_stack[1024];
> >    static tee_tmp_trap_stack tmp_stack[OPTEED_CORE_COUNT];
> >
> >    optee_vector_table value is assigned after the TEE OS boot succeeds.
> >    nsec_cpu_context and sec_cpu_context is to save the context of TEE and REE,
> >    cpu_start_context save the context of opensbi before tee_os_init enter TEE OS,
> >    tee_tmp_stack is the trap stack when TEE OS return tee_os_init.
> >
> > Under the current design, REE does not enable interrupts when 
> > entering TEE, and the entire TEE, including opensbi, is the process context of linux.
> > TEE processing must be brief and quick.
> >
>
> Great. Thanks. Your PR should include more details like this with proper FID allocation and description.
> In order to make it a standard RISC-V SBI extension, you need to 
> present this in tech-prs mailing list
> (https://lists.riscv.org/g/tech-prs)
> and get it approved within the group. Here is the policy document[1]
>
> https://docs.google.com/document/d/1bQGHU-wD4uN4mU07oH9adpXbH51NqUbFxc
> L8qTTIqhA/edit
>
> I think it would be good if you can present your work in trusted computing SIG (https://lists.riscv.org/g/sig-trusted-computing) as well.
>
> >
> > Regards,
> > liushiwei
> > -----????-----
> > ???: Himanshu Chauhan [mailto:hchauhan at ventanamicro.com]
> > ????: 2023?1?11? 23:39
> > ???: liushiwei <liushiwei at eswincomputing.com>
> > ??: opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> > ??: Re: ??: [PATCH 1/1] Add RISC-V TEE support
> >
> > On Wed, Jan 11, 2023 at 08:27:59PM +0800, liushiwei wrote:
> > > Do you mean hardware? Our hardware design referred to arm's 
> > > trustzone technology. optee os is a software solution using arm 
> > > trustzone hardware, which mainly includes REE(linux), TEE(optee 
> > > os), ATF(ARM Trusted firmware), and then our software also 
> > > developed these three parts. opensbi is similar to ATF. whether if this is what you want?
> > > The current committed code is not hardware-dependent, but just 
> > > continues the idea of this workaround, and we may commit hardware-dependent code later.
> > >
> > No, I meant the software specification.
> >
> > > -----????-----
> > > ???: hchauhan at ventanamicro.com [mailto:hchauhan at ventanamicro.com]
> > > ????: 2023?1?11? 20:03
> > > ???: 'liushiwei' <liushiwei at eswincomputing.com>; opensbi at lists.infradead.
> > > org
> > > ??: chenchaokai at eswincomputing.com
> > > ??: RE: [PATCH 1/1] Add RISC-V TEE support
> > >
> > > -----Original Message-----
> > > > From: opensbi <opensbi-bounces at lists.infradead.org> On Behalf Of 
> > > > liushiwei
> > > > Sent: 11 January 2023 07:32
> > > > To: opensbi at lists.infradead.org
> > > > Cc: chenchaokai at eswincomputing.com; liushiwei
> > > <liushiwei at eswincomputing.com>
> > > > Subject: [PATCH 1/1] Add RISC-V TEE support
> > >
> > > >RISC-V Trusted Executable Environment security software includes 
> > > >linux,
> > > opensbi, and OP-TEE OS. linux is the non-secure domain, and OP-TEE 
> > > OS is the secure domain. At boot time, opensbi boots OP->TEE OS and then starts linux.
> > > At runtime, opensbi acts as a secure monitor, responsible for 
> > > context saving and restoring when switching between linux and OP-TEE OS.
> > > >TEE function is off by default, when using configuration is added 
> > > >in the
> > > config and objects file, such as 
> > > platform/generic/configs/defconfig
> > > add CONFIG_SBI_ECALL_TEE = y, In the >platform/generic/objects.mk 
> > > add CONFIG_TEE_LOAD_ADDR = 0x27c000000, CONFIG_TEE_LOAD_ADDR is 
> > > the starting address of the OP-TEE OS.
> > >
> > > Hi Liushiwei,
> > >
> > > Was there any formal specification or draft for this? Could you 
> > > please point me to the draft or specification?
> > >
> > > Regards
> > > Himanshu
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
> > >
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>
>
>
> --
> Regards,
> Atish
>


--
Regards,
Atish



From wangliu at iscas.ac.cn  Thu Feb 16 23:55:51 2023
From: wangliu at iscas.ac.cn (wangliu)
Date: Fri, 17 Feb 2023 15:55:51 +0800
Subject: [PATCH] lib: utils: Mark only the largest region as reserved in
 FDT
In-Reply-To: <CAAhSdy3ddbpDc7Wix2EqdO-geTiiLTHRnLvvGb2BNr8dcWw1Aw@mail.gmail.com>
References: <CAAhSdy3ddbpDc7Wix2EqdO-geTiiLTHRnLvvGb2BNr8dcWw1Aw@mail.gmail.com>
Message-ID: <1ac79baf-bef3-9d9a-a151-9cbaf88e61ff@iscas.ac.cn>

Hi,
I am trying to generate a fw_payload.bin, but the mainline opensbi 
failed to compile.
I have retried the operation in the newly installed system environment, 
please do let me know if more info are to be provided.

This is my server information:
    $ uname -a
    Linux debian 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 (2022-10-21) 
x86_64 GNU/Linux
    $ cat /etc/debian_version
    11.5

This is my operation:
    $ make     CROSS_COMPILE=riscv64-linux-gnu-     PLATFORM=generic 
FW_PAYLOAD_PATH=../u-boot/u-boot.bin

And this is my log:

Loaded configuration 
'/home/snail/opensbi/platform/generic/configs/defconfig'
Configuration saved to 
'/home/snail/opensbi/build/platform/generic/kconfig/.config'
  CPP-DEP   platform/generic/firmware/fw_payload.elf.dep
  CPP-DEP   platform/generic/firmware/fw_jump.elf.dep
  CPP-DEP   platform/generic/firmware/fw_dynamic.elf.dep
  CPP-DEP   platform/generic/firmware/payloads/test.elf.dep
  AS-DEP    platform/generic/firmware/fw_payload.dep
  AS-DEP    platform/generic/firmware/fw_jump.dep
  AS-DEP    platform/generic/firmware/fw_dynamic.dep
  AS-DEP    platform/generic/firmware/payloads/test_head.dep
  CC-DEP    platform/generic/firmware/payloads/test_main.dep
  MERGE-DEP platform/generic/firmware/payloads/test.dep
  AS-DEP    lib/sbi/sbi_expected_trap.dep
  CC-DEP    lib/sbi/sbi_unpriv.dep
  CC-DEP    lib/sbi/sbi_trap.dep
  CC-DEP    lib/sbi/sbi_tlb.dep
  ...

  ...

  ...

  CC        platform/generic/lib/utils/ipi/fdt_ipi_drivers.o
  CC        platform/generic/lib/utils/ipi/fdt_ipi_mswi.o
  CC        platform/generic/lib/utils/ipi/fdt_ipi_plicsw.o
  CC        platform/generic/lib/utils/i2c/i2c.o
  CC        platform/generic/lib/utils/i2c/fdt_i2c.o
  CARRAY    platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.c
  CC        platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.o
  CC        platform/generic/lib/utils/i2c/fdt_i2c_sifive.o
  CC        platform/generic/lib/utils/gpio/fdt_gpio.o
  CARRAY    platform/generic/lib/utils/gpio/fdt_gpio_drivers.c
  CC        platform/generic/lib/utils/gpio/fdt_gpio_drivers.o
  CC        platform/generic/lib/utils/gpio/fdt_gpio_sifive.o
  CC        platform/generic/lib/utils/gpio/gpio.o
  CC        platform/generic/lib/utils/fdt/fdt_domain.o
  CC        platform/generic/lib/utils/fdt/fdt_pmu.o
  CC        platform/generic/lib/utils/fdt/fdt_helper.o
  CC        platform/generic/lib/utils/fdt/fdt_fixup.o
/home/snail/opensbi/lib/utils/fdt/fdt_fixup.c: In function 
?fdt_reserved_memory_fixup?:
/home/snail/opensbi/lib/utils/fdt/fdt_fixup.c:376:2: error: label at end 
of compound statement
   376 |  next_entry:
       |  ^~~~~~~~~~
make: *** [Makefile:569: 
/home/snail/opensbi/build/platform/generic/lib/utils/fdt/fdt_fixup.o] 
Error 1

The opensbi v1.2 version I have success to build.
I try to revert this commit code 
(https://github.com/riscv-software-src/opensbi/commit/199189bd1c172aab5a9733c0ffaaa14bbebb3323), 
then the code build successful.



From rpathakmailbox at gmail.com  Fri Feb 17 01:15:35 2023
From: rpathakmailbox at gmail.com (Rahul Pathak)
Date: Fri, 17 Feb 2023 14:45:35 +0530
Subject: [PATCH] lib: utils: Mark only the largest region as reserved in FDT
In-Reply-To: <1ac79baf-bef3-9d9a-a151-9cbaf88e61ff@iscas.ac.cn>
References: <CAAhSdy3ddbpDc7Wix2EqdO-geTiiLTHRnLvvGb2BNr8dcWw1Aw@mail.gmail.com>
 <1ac79baf-bef3-9d9a-a151-9cbaf88e61ff@iscas.ac.cn>
Message-ID: <CAMSwOgpcL7-0hT1tmWssoN0_HUvtqQD2oe_7PLhZ6ZtA1Y-Lkg@mail.gmail.com>

For me gcc does not beak, but clang does

Can you try with this -

diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index 619e4f5cbaf7..f00f3d6c94b6 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -374,6 +374,7 @@ int fdt_reserved_memory_fixup(void *fdt)
  filtered_order[i] = reg->order;
  i++;
  next_entry:
+ {}
  }

  for (j = 0; j < i; j++) {

On Fri, Feb 17, 2023 at 1:27 PM wangliu <wangliu at iscas.ac.cn> wrote:
>
> Hi,
> I am trying to generate a fw_payload.bin, but the mainline opensbi
> failed to compile.
> I have retried the operation in the newly installed system environment,
> please do let me know if more info are to be provided.
>
> This is my server information:
>     $ uname -a
>     Linux debian 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 (2022-10-21)
> x86_64 GNU/Linux
>     $ cat /etc/debian_version
>     11.5
>
> This is my operation:
>     $ make     CROSS_COMPILE=riscv64-linux-gnu-     PLATFORM=generic
> FW_PAYLOAD_PATH=../u-boot/u-boot.bin
>
> And this is my log:
>
> Loaded configuration
> '/home/snail/opensbi/platform/generic/configs/defconfig'
> Configuration saved to
> '/home/snail/opensbi/build/platform/generic/kconfig/.config'
>   CPP-DEP   platform/generic/firmware/fw_payload.elf.dep
>   CPP-DEP   platform/generic/firmware/fw_jump.elf.dep
>   CPP-DEP   platform/generic/firmware/fw_dynamic.elf.dep
>   CPP-DEP   platform/generic/firmware/payloads/test.elf.dep
>   AS-DEP    platform/generic/firmware/fw_payload.dep
>   AS-DEP    platform/generic/firmware/fw_jump.dep
>   AS-DEP    platform/generic/firmware/fw_dynamic.dep
>   AS-DEP    platform/generic/firmware/payloads/test_head.dep
>   CC-DEP    platform/generic/firmware/payloads/test_main.dep
>   MERGE-DEP platform/generic/firmware/payloads/test.dep
>   AS-DEP    lib/sbi/sbi_expected_trap.dep
>   CC-DEP    lib/sbi/sbi_unpriv.dep
>   CC-DEP    lib/sbi/sbi_trap.dep
>   CC-DEP    lib/sbi/sbi_tlb.dep
>   ...
>
>   ...
>
>   ...
>
>   CC        platform/generic/lib/utils/ipi/fdt_ipi_drivers.o
>   CC        platform/generic/lib/utils/ipi/fdt_ipi_mswi.o
>   CC        platform/generic/lib/utils/ipi/fdt_ipi_plicsw.o
>   CC        platform/generic/lib/utils/i2c/i2c.o
>   CC        platform/generic/lib/utils/i2c/fdt_i2c.o
>   CARRAY    platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.c
>   CC        platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.o
>   CC        platform/generic/lib/utils/i2c/fdt_i2c_sifive.o
>   CC        platform/generic/lib/utils/gpio/fdt_gpio.o
>   CARRAY    platform/generic/lib/utils/gpio/fdt_gpio_drivers.c
>   CC        platform/generic/lib/utils/gpio/fdt_gpio_drivers.o
>   CC        platform/generic/lib/utils/gpio/fdt_gpio_sifive.o
>   CC        platform/generic/lib/utils/gpio/gpio.o
>   CC        platform/generic/lib/utils/fdt/fdt_domain.o
>   CC        platform/generic/lib/utils/fdt/fdt_pmu.o
>   CC        platform/generic/lib/utils/fdt/fdt_helper.o
>   CC        platform/generic/lib/utils/fdt/fdt_fixup.o
> /home/snail/opensbi/lib/utils/fdt/fdt_fixup.c: In function
> ?fdt_reserved_memory_fixup?:
> /home/snail/opensbi/lib/utils/fdt/fdt_fixup.c:376:2: error: label at end
> of compound statement
>    376 |  next_entry:
>        |  ^~~~~~~~~~
> make: *** [Makefile:569:
> /home/snail/opensbi/build/platform/generic/lib/utils/fdt/fdt_fixup.o]
> Error 1
>
> The opensbi v1.2 version I have success to build.
> I try to revert this commit code
> (https://github.com/riscv-software-src/opensbi/commit/199189bd1c172aab5a9733c0ffaaa14bbebb3323),
> then the code build successful.
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From wangliu at iscas.ac.cn  Fri Feb 17 01:29:47 2023
From: wangliu at iscas.ac.cn (wangliu)
Date: Fri, 17 Feb 2023 17:29:47 +0800
Subject: [PATCH] lib: utils: Mark only the largest region as reserved in
 FDT
In-Reply-To: <CAMSwOgpcL7-0hT1tmWssoN0_HUvtqQD2oe_7PLhZ6ZtA1Y-Lkg@mail.gmail.com>
References: <CAAhSdy3ddbpDc7Wix2EqdO-geTiiLTHRnLvvGb2BNr8dcWw1Aw@mail.gmail.com>
 <1ac79baf-bef3-9d9a-a151-9cbaf88e61ff@iscas.ac.cn>
 <CAMSwOgpcL7-0hT1tmWssoN0_HUvtqQD2oe_7PLhZ6ZtA1Y-Lkg@mail.gmail.com>
Message-ID: <901927b1-7ce9-724e-90b8-4a99a9178973@iscas.ac.cn>

It works, the opensbi can build successful.
This is the diff:
$ git diff
diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index 619e4f5..1e84361 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -374,6 +374,7 @@ int fdt_reserved_memory_fixup(void *fdt)
                 filtered_order[i] = reg->order;
                 i++;
         next_entry:
+       {}
         }

         for (j = 0; j < i; j++) {



? 2/17/23 17:15, Rahul Pathak ??:
> For me gcc does not beak, but clang does
> 
> Can you try with this -
> 
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 619e4f5cbaf7..f00f3d6c94b6 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -374,6 +374,7 @@ int fdt_reserved_memory_fixup(void *fdt)
>    filtered_order[i] = reg->order;
>    i++;
>    next_entry:
> + {}
>    }
> 
>    for (j = 0; j < i; j++) {
> 
> On Fri, Feb 17, 2023 at 1:27 PM wangliu <wangliu at iscas.ac.cn> wrote:
>>
>> Hi,
>> I am trying to generate a fw_payload.bin, but the mainline opensbi
>> failed to compile.
>> I have retried the operation in the newly installed system environment,
>> please do let me know if more info are to be provided.
>>
>> This is my server information:
>>      $ uname -a
>>      Linux debian 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 (2022-10-21)
>> x86_64 GNU/Linux
>>      $ cat /etc/debian_version
>>      11.5
>>
>> This is my operation:
>>      $ make     CROSS_COMPILE=riscv64-linux-gnu-     PLATFORM=generic
>> FW_PAYLOAD_PATH=../u-boot/u-boot.bin
>>
>> And this is my log:
>>
>> Loaded configuration
>> '/home/snail/opensbi/platform/generic/configs/defconfig'
>> Configuration saved to
>> '/home/snail/opensbi/build/platform/generic/kconfig/.config'
>>    CPP-DEP   platform/generic/firmware/fw_payload.elf.dep
>>    CPP-DEP   platform/generic/firmware/fw_jump.elf.dep
>>    CPP-DEP   platform/generic/firmware/fw_dynamic.elf.dep
>>    CPP-DEP   platform/generic/firmware/payloads/test.elf.dep
>>    AS-DEP    platform/generic/firmware/fw_payload.dep
>>    AS-DEP    platform/generic/firmware/fw_jump.dep
>>    AS-DEP    platform/generic/firmware/fw_dynamic.dep
>>    AS-DEP    platform/generic/firmware/payloads/test_head.dep
>>    CC-DEP    platform/generic/firmware/payloads/test_main.dep
>>    MERGE-DEP platform/generic/firmware/payloads/test.dep
>>    AS-DEP    lib/sbi/sbi_expected_trap.dep
>>    CC-DEP    lib/sbi/sbi_unpriv.dep
>>    CC-DEP    lib/sbi/sbi_trap.dep
>>    CC-DEP    lib/sbi/sbi_tlb.dep
>>    ...
>>
>>    ...
>>
>>    ...
>>
>>    CC        platform/generic/lib/utils/ipi/fdt_ipi_drivers.o
>>    CC        platform/generic/lib/utils/ipi/fdt_ipi_mswi.o
>>    CC        platform/generic/lib/utils/ipi/fdt_ipi_plicsw.o
>>    CC        platform/generic/lib/utils/i2c/i2c.o
>>    CC        platform/generic/lib/utils/i2c/fdt_i2c.o
>>    CARRAY    platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.c
>>    CC        platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.o
>>    CC        platform/generic/lib/utils/i2c/fdt_i2c_sifive.o
>>    CC        platform/generic/lib/utils/gpio/fdt_gpio.o
>>    CARRAY    platform/generic/lib/utils/gpio/fdt_gpio_drivers.c
>>    CC        platform/generic/lib/utils/gpio/fdt_gpio_drivers.o
>>    CC        platform/generic/lib/utils/gpio/fdt_gpio_sifive.o
>>    CC        platform/generic/lib/utils/gpio/gpio.o
>>    CC        platform/generic/lib/utils/fdt/fdt_domain.o
>>    CC        platform/generic/lib/utils/fdt/fdt_pmu.o
>>    CC        platform/generic/lib/utils/fdt/fdt_helper.o
>>    CC        platform/generic/lib/utils/fdt/fdt_fixup.o
>> /home/snail/opensbi/lib/utils/fdt/fdt_fixup.c: In function
>> ?fdt_reserved_memory_fixup?:
>> /home/snail/opensbi/lib/utils/fdt/fdt_fixup.c:376:2: error: label at end
>> of compound statement
>>     376 |  next_entry:
>>         |  ^~~~~~~~~~
>> make: *** [Makefile:569:
>> /home/snail/opensbi/build/platform/generic/lib/utils/fdt/fdt_fixup.o]
>> Error 1
>>
>> The opensbi v1.2 version I have success to build.
>> I try to revert this commit code
>> (https://github.com/riscv-software-src/opensbi/commit/199189bd1c172aab5a9733c0ffaaa14bbebb3323),
>> then the code build successful.
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi



From rpathakmailbox at gmail.com  Fri Feb 17 01:34:07 2023
From: rpathakmailbox at gmail.com (Rahul Pathak)
Date: Fri, 17 Feb 2023 15:04:07 +0530
Subject: [PATCH] lib: utils: Mark only the largest region as reserved in FDT
In-Reply-To: <901927b1-7ce9-724e-90b8-4a99a9178973@iscas.ac.cn>
References: <CAAhSdy3ddbpDc7Wix2EqdO-geTiiLTHRnLvvGb2BNr8dcWw1Aw@mail.gmail.com>
 <1ac79baf-bef3-9d9a-a151-9cbaf88e61ff@iscas.ac.cn> <CAMSwOgpcL7-0hT1tmWssoN0_HUvtqQD2oe_7PLhZ6ZtA1Y-Lkg@mail.gmail.com>
 <901927b1-7ce9-724e-90b8-4a99a9178973@iscas.ac.cn>
Message-ID: <CAMSwOgqPJPJpRdXsvj-cdStwQoSFBQFMmen_RU3_EPGRro+_PQ@mail.gmail.com>

I am though surprised that for me it's not happening with gcc.
What is the gcc version for you?

I have 12.2.1

On Fri, Feb 17, 2023 at 2:59 PM wangliu <wangliu at iscas.ac.cn> wrote:
>
> It works, the opensbi can build successful.
> This is the diff:
> $ git diff
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 619e4f5..1e84361 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -374,6 +374,7 @@ int fdt_reserved_memory_fixup(void *fdt)
>                  filtered_order[i] = reg->order;
>                  i++;
>          next_entry:
> +       {}
>          }
>
>          for (j = 0; j < i; j++) {
>
>
>
> ? 2/17/23 17:15, Rahul Pathak ??:
> > For me gcc does not beak, but clang does
> >
> > Can you try with this -
> >
> > diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> > index 619e4f5cbaf7..f00f3d6c94b6 100644
> > --- a/lib/utils/fdt/fdt_fixup.c
> > +++ b/lib/utils/fdt/fdt_fixup.c
> > @@ -374,6 +374,7 @@ int fdt_reserved_memory_fixup(void *fdt)
> >    filtered_order[i] = reg->order;
> >    i++;
> >    next_entry:
> > + {}
> >    }
> >
> >    for (j = 0; j < i; j++) {
> >
> > On Fri, Feb 17, 2023 at 1:27 PM wangliu <wangliu at iscas.ac.cn> wrote:
> >>
> >> Hi,
> >> I am trying to generate a fw_payload.bin, but the mainline opensbi
> >> failed to compile.
> >> I have retried the operation in the newly installed system environment,
> >> please do let me know if more info are to be provided.
> >>
> >> This is my server information:
> >>      $ uname -a
> >>      Linux debian 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 (2022-10-21)
> >> x86_64 GNU/Linux
> >>      $ cat /etc/debian_version
> >>      11.5
> >>
> >> This is my operation:
> >>      $ make     CROSS_COMPILE=riscv64-linux-gnu-     PLATFORM=generic
> >> FW_PAYLOAD_PATH=../u-boot/u-boot.bin
> >>
> >> And this is my log:
> >>
> >> Loaded configuration
> >> '/home/snail/opensbi/platform/generic/configs/defconfig'
> >> Configuration saved to
> >> '/home/snail/opensbi/build/platform/generic/kconfig/.config'
> >>    CPP-DEP   platform/generic/firmware/fw_payload.elf.dep
> >>    CPP-DEP   platform/generic/firmware/fw_jump.elf.dep
> >>    CPP-DEP   platform/generic/firmware/fw_dynamic.elf.dep
> >>    CPP-DEP   platform/generic/firmware/payloads/test.elf.dep
> >>    AS-DEP    platform/generic/firmware/fw_payload.dep
> >>    AS-DEP    platform/generic/firmware/fw_jump.dep
> >>    AS-DEP    platform/generic/firmware/fw_dynamic.dep
> >>    AS-DEP    platform/generic/firmware/payloads/test_head.dep
> >>    CC-DEP    platform/generic/firmware/payloads/test_main.dep
> >>    MERGE-DEP platform/generic/firmware/payloads/test.dep
> >>    AS-DEP    lib/sbi/sbi_expected_trap.dep
> >>    CC-DEP    lib/sbi/sbi_unpriv.dep
> >>    CC-DEP    lib/sbi/sbi_trap.dep
> >>    CC-DEP    lib/sbi/sbi_tlb.dep
> >>    ...
> >>
> >>    ...
> >>
> >>    ...
> >>
> >>    CC        platform/generic/lib/utils/ipi/fdt_ipi_drivers.o
> >>    CC        platform/generic/lib/utils/ipi/fdt_ipi_mswi.o
> >>    CC        platform/generic/lib/utils/ipi/fdt_ipi_plicsw.o
> >>    CC        platform/generic/lib/utils/i2c/i2c.o
> >>    CC        platform/generic/lib/utils/i2c/fdt_i2c.o
> >>    CARRAY    platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.c
> >>    CC        platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.o
> >>    CC        platform/generic/lib/utils/i2c/fdt_i2c_sifive.o
> >>    CC        platform/generic/lib/utils/gpio/fdt_gpio.o
> >>    CARRAY    platform/generic/lib/utils/gpio/fdt_gpio_drivers.c
> >>    CC        platform/generic/lib/utils/gpio/fdt_gpio_drivers.o
> >>    CC        platform/generic/lib/utils/gpio/fdt_gpio_sifive.o
> >>    CC        platform/generic/lib/utils/gpio/gpio.o
> >>    CC        platform/generic/lib/utils/fdt/fdt_domain.o
> >>    CC        platform/generic/lib/utils/fdt/fdt_pmu.o
> >>    CC        platform/generic/lib/utils/fdt/fdt_helper.o
> >>    CC        platform/generic/lib/utils/fdt/fdt_fixup.o
> >> /home/snail/opensbi/lib/utils/fdt/fdt_fixup.c: In function
> >> ?fdt_reserved_memory_fixup?:
> >> /home/snail/opensbi/lib/utils/fdt/fdt_fixup.c:376:2: error: label at end
> >> of compound statement
> >>     376 |  next_entry:
> >>         |  ^~~~~~~~~~
> >> make: *** [Makefile:569:
> >> /home/snail/opensbi/build/platform/generic/lib/utils/fdt/fdt_fixup.o]
> >> Error 1
> >>
> >> The opensbi v1.2 version I have success to build.
> >> I try to revert this commit code
> >> (https://github.com/riscv-software-src/opensbi/commit/199189bd1c172aab5a9733c0ffaaa14bbebb3323),
> >> then the code build successful.
> >>
> >>
> >> --
> >> opensbi mailing list
> >> opensbi at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/opensbi
>


From wangliu at iscas.ac.cn  Fri Feb 17 01:38:49 2023
From: wangliu at iscas.ac.cn (wangliu)
Date: Fri, 17 Feb 2023 17:38:49 +0800
Subject: [PATCH] lib: utils: Mark only the largest region as reserved in
 FDT
In-Reply-To: <CAMSwOgqPJPJpRdXsvj-cdStwQoSFBQFMmen_RU3_EPGRro+_PQ@mail.gmail.com>
References: <CAAhSdy3ddbpDc7Wix2EqdO-geTiiLTHRnLvvGb2BNr8dcWw1Aw@mail.gmail.com>
 <1ac79baf-bef3-9d9a-a151-9cbaf88e61ff@iscas.ac.cn>
 <CAMSwOgpcL7-0hT1tmWssoN0_HUvtqQD2oe_7PLhZ6ZtA1Y-Lkg@mail.gmail.com>
 <901927b1-7ce9-724e-90b8-4a99a9178973@iscas.ac.cn>
 <CAMSwOgqPJPJpRdXsvj-cdStwQoSFBQFMmen_RU3_EPGRro+_PQ@mail.gmail.com>
Message-ID: <412775aa-47d0-455e-5af2-b1023255cd45@iscas.ac.cn>

My server's gcc version is 10.2.1, it seems like too old.

$ gcc --version
gcc (Debian 10.2.1-6) 10.2.1 20210110
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.



? 2/17/23 17:34, Rahul Pathak ??:
> I am though surprised that for me it's not happening with gcc.
> What is the gcc version for you?
> 
> I have 12.2.1
> 
> On Fri, Feb 17, 2023 at 2:59 PM wangliu <wangliu at iscas.ac.cn> wrote:
>>
>> It works, the opensbi can build successful.
>> This is the diff:
>> $ git diff
>> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
>> index 619e4f5..1e84361 100644
>> --- a/lib/utils/fdt/fdt_fixup.c
>> +++ b/lib/utils/fdt/fdt_fixup.c
>> @@ -374,6 +374,7 @@ int fdt_reserved_memory_fixup(void *fdt)
>>                   filtered_order[i] = reg->order;
>>                   i++;
>>           next_entry:
>> +       {}
>>           }
>>
>>           for (j = 0; j < i; j++) {
>>
>>
>>
>> ? 2/17/23 17:15, Rahul Pathak ??:
>>> For me gcc does not beak, but clang does
>>>
>>> Can you try with this -
>>>
>>> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
>>> index 619e4f5cbaf7..f00f3d6c94b6 100644
>>> --- a/lib/utils/fdt/fdt_fixup.c
>>> +++ b/lib/utils/fdt/fdt_fixup.c
>>> @@ -374,6 +374,7 @@ int fdt_reserved_memory_fixup(void *fdt)
>>>     filtered_order[i] = reg->order;
>>>     i++;
>>>     next_entry:
>>> + {}
>>>     }
>>>
>>>     for (j = 0; j < i; j++) {
>>>
>>> On Fri, Feb 17, 2023 at 1:27 PM wangliu <wangliu at iscas.ac.cn> wrote:
>>>>
>>>> Hi,
>>>> I am trying to generate a fw_payload.bin, but the mainline opensbi
>>>> failed to compile.
>>>> I have retried the operation in the newly installed system environment,
>>>> please do let me know if more info are to be provided.
>>>>
>>>> This is my server information:
>>>>       $ uname -a
>>>>       Linux debian 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 (2022-10-21)
>>>> x86_64 GNU/Linux
>>>>       $ cat /etc/debian_version
>>>>       11.5
>>>>
>>>> This is my operation:
>>>>       $ make     CROSS_COMPILE=riscv64-linux-gnu-     PLATFORM=generic
>>>> FW_PAYLOAD_PATH=../u-boot/u-boot.bin
>>>>
>>>> And this is my log:
>>>>
>>>> Loaded configuration
>>>> '/home/snail/opensbi/platform/generic/configs/defconfig'
>>>> Configuration saved to
>>>> '/home/snail/opensbi/build/platform/generic/kconfig/.config'
>>>>     CPP-DEP   platform/generic/firmware/fw_payload.elf.dep
>>>>     CPP-DEP   platform/generic/firmware/fw_jump.elf.dep
>>>>     CPP-DEP   platform/generic/firmware/fw_dynamic.elf.dep
>>>>     CPP-DEP   platform/generic/firmware/payloads/test.elf.dep
>>>>     AS-DEP    platform/generic/firmware/fw_payload.dep
>>>>     AS-DEP    platform/generic/firmware/fw_jump.dep
>>>>     AS-DEP    platform/generic/firmware/fw_dynamic.dep
>>>>     AS-DEP    platform/generic/firmware/payloads/test_head.dep
>>>>     CC-DEP    platform/generic/firmware/payloads/test_main.dep
>>>>     MERGE-DEP platform/generic/firmware/payloads/test.dep
>>>>     AS-DEP    lib/sbi/sbi_expected_trap.dep
>>>>     CC-DEP    lib/sbi/sbi_unpriv.dep
>>>>     CC-DEP    lib/sbi/sbi_trap.dep
>>>>     CC-DEP    lib/sbi/sbi_tlb.dep
>>>>     ...
>>>>
>>>>     ...
>>>>
>>>>     ...
>>>>
>>>>     CC        platform/generic/lib/utils/ipi/fdt_ipi_drivers.o
>>>>     CC        platform/generic/lib/utils/ipi/fdt_ipi_mswi.o
>>>>     CC        platform/generic/lib/utils/ipi/fdt_ipi_plicsw.o
>>>>     CC        platform/generic/lib/utils/i2c/i2c.o
>>>>     CC        platform/generic/lib/utils/i2c/fdt_i2c.o
>>>>     CARRAY    platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.c
>>>>     CC        platform/generic/lib/utils/i2c/fdt_i2c_adapter_drivers.o
>>>>     CC        platform/generic/lib/utils/i2c/fdt_i2c_sifive.o
>>>>     CC        platform/generic/lib/utils/gpio/fdt_gpio.o
>>>>     CARRAY    platform/generic/lib/utils/gpio/fdt_gpio_drivers.c
>>>>     CC        platform/generic/lib/utils/gpio/fdt_gpio_drivers.o
>>>>     CC        platform/generic/lib/utils/gpio/fdt_gpio_sifive.o
>>>>     CC        platform/generic/lib/utils/gpio/gpio.o
>>>>     CC        platform/generic/lib/utils/fdt/fdt_domain.o
>>>>     CC        platform/generic/lib/utils/fdt/fdt_pmu.o
>>>>     CC        platform/generic/lib/utils/fdt/fdt_helper.o
>>>>     CC        platform/generic/lib/utils/fdt/fdt_fixup.o
>>>> /home/snail/opensbi/lib/utils/fdt/fdt_fixup.c: In function
>>>> ?fdt_reserved_memory_fixup?:
>>>> /home/snail/opensbi/lib/utils/fdt/fdt_fixup.c:376:2: error: label at end
>>>> of compound statement
>>>>      376 |  next_entry:
>>>>          |  ^~~~~~~~~~
>>>> make: *** [Makefile:569:
>>>> /home/snail/opensbi/build/platform/generic/lib/utils/fdt/fdt_fixup.o]
>>>> Error 1
>>>>
>>>> The opensbi v1.2 version I have success to build.
>>>> I try to revert this commit code
>>>> (https://github.com/riscv-software-src/opensbi/commit/199189bd1c172aab5a9733c0ffaaa14bbebb3323),
>>>> then the code build successful.
>>>>
>>>>
>>>> --
>>>> opensbi mailing list
>>>> opensbi at lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/opensbi
>>



From mchitale at ventanamicro.com  Fri Feb 17 06:44:08 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Fri, 17 Feb 2023 20:14:08 +0530
Subject: [PATCH v2 0/5] SBI PMU firmware counters and events improvement
Message-ID: <20230217144413.3720307-1-mchitale@ventanamicro.com>

This patchset implements the improvements described in the following
proposal from Anup Patel: 

https://lists.riscv.org/g/tech-prs/message/102

Changes in v2:
- Update Risc V platform specific firmware event handling as per
  the SBI spec update: https://lists.riscv.org/g/tech-prs/message/211

Mayuresh Chitale (5):
  lib: sbi_pmu: add callback for counter width
  lib: sbi_pmu: Implement sbi_pmu_counter_fw_read_hi
  lib: sbi_pmu: Reserve space for implementation specific firmware
    events
  lib: sbi_pmu: Update custom PMU device ops
  lib: sbi_pmu: Add hartid parameter PMU device ops

 include/sbi/sbi_ecall_interface.h |  12 +++
 include/sbi/sbi_pmu.h             |  33 +++++---
 lib/sbi/sbi_ecall_pmu.c           |   8 ++
 lib/sbi/sbi_pmu.c                 | 120 +++++++++++++++++++-----------
 4 files changed, 121 insertions(+), 52 deletions(-)

-- 
2.34.1



From mchitale at ventanamicro.com  Fri Feb 17 06:44:09 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Fri, 17 Feb 2023 20:14:09 +0530
Subject: [PATCH v2 1/5] lib: sbi_pmu: add callback for counter width
In-Reply-To: <20230217144413.3720307-1-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com>
Message-ID: <20230217144413.3720307-2-mchitale@ventanamicro.com>

This patch adds a callback to fetch the number of bits implemented for a
custom firmware counter. If the callback fails or is not implemented then
width defaults to 63.

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 include/sbi/sbi_pmu.h | 5 +++++
 lib/sbi/sbi_pmu.c     | 6 ++++++
 2 files changed, 11 insertions(+)

diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
index c365243..b3b75c1 100644
--- a/include/sbi/sbi_pmu.h
+++ b/include/sbi/sbi_pmu.h
@@ -41,6 +41,11 @@ struct sbi_pmu_device {
 	bool (*fw_counter_match_code)(uint32_t counter_index,
 				      uint32_t event_idx_code);
 
+	/**
+	 * Fetch the max width of this counter in number of bits.
+	 */
+	int (*fw_counter_width)(void);
+
 	/**
 	 * Read value of custom firmware counter
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 154dbda..a99c045 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -761,6 +761,7 @@ unsigned long sbi_pmu_num_ctr(void)
 
 int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
 {
+	int width;
 	union sbi_pmu_ctr_info cinfo = {0};
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
@@ -782,6 +783,11 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
 		cinfo.type = SBI_PMU_CTR_TYPE_FW;
 		/* Firmware counters are always 64 bits wide */
 		cinfo.width = 63;
+		if (pmu_dev && pmu_dev->fw_counter_width) {
+			width = pmu_dev->fw_counter_width();
+			if (width)
+				cinfo.width = width - 1;
+		}
 	}
 
 	*ctr_info = cinfo.value;
-- 
2.34.1



From mchitale at ventanamicro.com  Fri Feb 17 06:44:10 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Fri, 17 Feb 2023 20:14:10 +0530
Subject: [PATCH v2 2/5] lib: sbi_pmu: Implement sbi_pmu_counter_fw_read_hi
In-Reply-To: <20230217144413.3720307-1-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com>
Message-ID: <20230217144413.3720307-3-mchitale@ventanamicro.com>

To support 64 bit firmware counters on RV32 systems, we implement
sbi_pmu_counter_fw_read_hi() which returns the upper 32 bits of
the firmware counter value. On RV64 (or higher) systems, this
function will always return zero.

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 include/sbi/sbi_ecall_interface.h | 1 +
 lib/sbi/sbi_ecall_pmu.c           | 8 ++++++++
 2 files changed, 9 insertions(+)

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index a3f2bf4..f5d6667 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -99,6 +99,7 @@
 #define SBI_EXT_PMU_COUNTER_START	0x3
 #define SBI_EXT_PMU_COUNTER_STOP	0x4
 #define SBI_EXT_PMU_COUNTER_FW_READ	0x5
+#define SBI_EXT_PMU_COUNTER_FW_READ_HI	0x6
 
 /** General pmu event codes specified in SBI PMU extension */
 enum sbi_pmu_hw_generic_events_t {
diff --git a/lib/sbi/sbi_ecall_pmu.c b/lib/sbi/sbi_ecall_pmu.c
index 826c8a8..2d1e669 100644
--- a/lib/sbi/sbi_ecall_pmu.c
+++ b/lib/sbi/sbi_ecall_pmu.c
@@ -54,6 +54,14 @@ static int sbi_ecall_pmu_handler(unsigned long extid, unsigned long funcid,
 		ret = sbi_pmu_ctr_fw_read(regs->a0, &temp);
 		*out_val = temp;
 		break;
+	case SBI_EXT_PMU_COUNTER_FW_READ_HI:
+#if __riscv_xlen == 32
+		ret = sbi_pmu_ctr_fw_read(regs->a0, &temp);
+		*out_val = temp >> 32
+#else
+		*out_val = 0;
+#endif
+		break;
 	case SBI_EXT_PMU_COUNTER_START:
 
 #if __riscv_xlen == 32
-- 
2.34.1



From mchitale at ventanamicro.com  Fri Feb 17 06:44:11 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Fri, 17 Feb 2023 20:14:11 +0530
Subject: [PATCH v2 3/5] lib: sbi_pmu: Reserve space for implementation specific firmware events
In-Reply-To: <20230217144413.3720307-1-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com>
Message-ID: <20230217144413.3720307-4-mchitale@ventanamicro.com>

We reserve space for SBI implementation specific custom firmware
events which can be used by M-mode firmwares and HS-mode hypervisors
for their own use. This reserved space is intentionally large to
ensure that SBI implementation has enough space to accommodate
platform specific firmware events as well.

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 include/sbi/sbi_ecall_interface.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index f5d6667..5732419 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -183,6 +183,17 @@ enum sbi_pmu_fw_event_code_id {
 	SBI_PMU_FW_HFENCE_VVMA_ASID_SENT = 20,
 	SBI_PMU_FW_HFENCE_VVMA_ASID_RCVD = 21,
 	SBI_PMU_FW_MAX,
+	/*
+	 * Event codes 22 to 255 are reserved for future use.
+	 * Event codes 256 to 65534 are reserved for SBI implementation
+	 * specific custom firmware events.
+	 */
+	SBI_PMU_FW_RESERVED_MAX = 0xFFFE,
+	/*
+	* Event code 0xFFFF is used for platform specific firmware
+	* events where the event data contains any event specific information.
+	*/
+	SBI_PMU_FW_PLATFORM = 0xFFFF,
 };
 
 /** SBI PMU event idx type */
-- 
2.34.1



From mchitale at ventanamicro.com  Fri Feb 17 06:44:12 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Fri, 17 Feb 2023 20:14:12 +0530
Subject: [PATCH v2 4/5] lib: sbi_pmu: Update custom PMU device ops
In-Reply-To: <20230217144413.3720307-1-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com>
Message-ID: <20230217144413.3720307-5-mchitale@ventanamicro.com>

Update the implmentation of some of the PMU device operations to make them
inline with the SBI spec updates for the platform specific firmware events.

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 include/sbi/sbi_pmu.h |  16 ++++---
 lib/sbi/sbi_pmu.c     | 102 ++++++++++++++++++++++++++----------------
 2 files changed, 73 insertions(+), 45 deletions(-)

diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
index b3b75c1..467ec03 100644
--- a/include/sbi/sbi_pmu.h
+++ b/include/sbi/sbi_pmu.h
@@ -30,16 +30,15 @@ struct sbi_pmu_device {
 
 	/**
 	 * Validate event code of custom firmware event
-	 * Note: SBI_PMU_FW_MAX <= event_idx_code
 	 */
-	int (*fw_event_validate_code)(uint32_t event_idx_code);
+	int (*fw_event_validate_code)(uint64_t event_idx_code);
 
 	/**
 	 * Match custom firmware counter with custom firmware event
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
 	 */
 	bool (*fw_counter_match_code)(uint32_t counter_index,
-				      uint32_t event_idx_code);
+				      uint64_t event_idx_code);
 
 	/**
 	 * Fetch the max width of this counter in number of bits.
@@ -52,14 +51,19 @@ struct sbi_pmu_device {
 	 */
 	uint64_t (*fw_counter_read_value)(uint32_t counter_index);
 
+	/**
+	 * Write value to custom firmware counter
+	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
+	 */
+	void (*fw_counter_write_value)(uint32_t counter_index,
+				       uint64_t value);
+
 	/**
 	 * Start custom firmware counter
-	 * Note: SBI_PMU_FW_MAX <= event_idx_code
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
 	 */
 	int (*fw_counter_start)(uint32_t counter_index,
-				uint32_t event_idx_code,
-				uint64_t init_val, bool init_val_update);
+				uint64_t event_idx_code);
 
 	/**
 	 * Stop custom firmware counter
diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index a99c045..05e2571 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -65,8 +65,12 @@ static uint32_t active_events[SBI_HARTMASK_MAX_BITS][SBI_PMU_HW_CTR_MAX + SBI_PM
 #endif
 static unsigned long fw_counters_started[SBI_HARTMASK_MAX_BITS];
 
-/* Values of firmwares counters on each HART */
-static uint64_t fw_counters_value[SBI_HARTMASK_MAX_BITS][SBI_PMU_FW_CTR_MAX] = {0};
+/*
+ * Counter values for SBI firmware events and event codes for platform
+ * firmware events. Both are mutually exclusive and hence can optimally share
+ * the same memory.
+ */
+static uint64_t fw_counters_data[SBI_HARTMASK_MAX_BITS][SBI_PMU_FW_CTR_MAX] = {0};
 
 /* Maximum number of hardware events available */
 static uint32_t num_hw_events;
@@ -106,7 +110,7 @@ static bool pmu_event_select_overlap(struct sbi_pmu_hw_event *evt,
 	return false;
 }
 
-static int pmu_event_validate(unsigned long event_idx)
+static int pmu_event_validate(unsigned long event_idx, uint64_t edata)
 {
 	uint32_t event_idx_type = get_cidx_type(event_idx);
 	uint32_t event_idx_code = get_cidx_code(event_idx);
@@ -118,9 +122,9 @@ static int pmu_event_validate(unsigned long event_idx)
 		event_idx_code_max = SBI_PMU_HW_GENERAL_MAX;
 		break;
 	case SBI_PMU_EVENT_TYPE_FW:
-		if (SBI_PMU_FW_MAX <= event_idx_code &&
+		if (SBI_PMU_FW_PLATFORM == event_idx_code &&
 		    pmu_dev && pmu_dev->fw_event_validate_code)
-			return pmu_dev->fw_event_validate_code(event_idx_code);
+			return pmu_dev->fw_event_validate_code(edata);
 		else
 			event_idx_code_max = SBI_PMU_FW_MAX;
 		break;
@@ -183,12 +187,14 @@ int sbi_pmu_ctr_fw_read(uint32_t cidx, uint64_t *cval)
 	if (event_idx_type != SBI_PMU_EVENT_TYPE_FW)
 		return SBI_EINVAL;
 
-	if (SBI_PMU_FW_MAX <= event_code &&
-	    pmu_dev && pmu_dev->fw_counter_read_value)
-		fw_counters_value[hartid][cidx - num_hw_ctrs] =
-			pmu_dev->fw_counter_read_value(cidx - num_hw_ctrs);
-
-	*cval = fw_counters_value[hartid][cidx - num_hw_ctrs];
+	if (SBI_PMU_FW_PLATFORM == event_code) {
+		if (pmu_dev && pmu_dev->fw_counter_read_value)
+			*cval = pmu_dev->fw_counter_read_value(cidx -
+							       num_hw_ctrs);
+		else
+			*cval = 0;
+	} else
+		*cval = fw_counters_data[hartid][cidx - num_hw_ctrs];
 
 	return 0;
 }
@@ -357,22 +363,30 @@ int sbi_pmu_irq_bit(void)
 }
 
 static int pmu_ctr_start_fw(uint32_t cidx, uint32_t event_code,
-			    uint64_t ival, bool ival_update)
+			    uint64_t event_data, uint64_t ival,
+			    bool ival_update)
 {
-	int ret;
 	u32 hartid = current_hartid();
 
-	if (SBI_PMU_FW_MAX <= event_code &&
-	    pmu_dev && pmu_dev->fw_counter_start) {
-		ret = pmu_dev->fw_counter_start(cidx - num_hw_ctrs,
-						event_code,
-						ival, ival_update);
-		if (ret)
-			return ret;
+	if (SBI_PMU_FW_PLATFORM == event_code) {
+		if (!pmu_dev ||
+		    !pmu_dev->fw_counter_write_value ||
+		    !pmu_dev->fw_counter_start) {
+			return SBI_EINVAL;
+		    }
+
+		if (ival_update)
+			pmu_dev->fw_counter_write_value(cidx - num_hw_ctrs,
+							ival);
+
+		if (pmu_dev->fw_counter_start(hartid, cidx - num_hw_ctrs,
+					      event_data))
+			return SBI_EINVAL;
+	} else {
+		if (ival_update)
+			fw_counters_data[hartid][cidx - num_hw_ctrs] = ival;
 	}
 
-	if (ival_update)
-		fw_counters_value[hartid][cidx - num_hw_ctrs] = ival;
 	fw_counters_started[hartid] |= BIT(cidx - num_hw_ctrs);
 
 	return 0;
@@ -381,8 +395,10 @@ static int pmu_ctr_start_fw(uint32_t cidx, uint32_t event_code,
 int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
 		      unsigned long flags, uint64_t ival)
 {
+	u32 hartid = current_hartid();
 	int event_idx_type;
 	uint32_t event_code;
+	uint64_t edata;
 	int ret = SBI_EINVAL;
 	bool bUpdate = false;
 	int i, cidx;
@@ -399,8 +415,13 @@ int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
 		if (event_idx_type < 0)
 			/* Continue the start operation for other counters */
 			continue;
-		else if (event_idx_type == SBI_PMU_EVENT_TYPE_FW)
-			ret = pmu_ctr_start_fw(cidx, event_code, ival, bUpdate);
+		else if (event_idx_type == SBI_PMU_EVENT_TYPE_FW) {
+			edata = (event_code == SBI_PMU_FW_PLATFORM) ?
+				 fw_counters_data[hartid][cidx - num_hw_ctrs]
+				 : 0x0;
+			ret = pmu_ctr_start_fw(cidx, event_code, edata, ival,
+					       bUpdate);
+		}
 		else
 			ret = pmu_ctr_start_hw(cidx, ival, bUpdate);
 	}
@@ -434,7 +455,7 @@ static int pmu_ctr_stop_fw(uint32_t cidx, uint32_t event_code)
 {
 	int ret;
 
-	if (SBI_PMU_FW_MAX <= event_code &&
+	if (SBI_PMU_FW_PLATFORM == event_code &&
 	    pmu_dev && pmu_dev->fw_counter_stop) {
 		ret = pmu_dev->fw_counter_stop(cidx - num_hw_ctrs);
 		if (ret)
@@ -640,7 +661,7 @@ static int pmu_ctr_find_hw(unsigned long cbase, unsigned long cmask, unsigned lo
  * check.
  */
 static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask,
-			   uint32_t event_code, u32 hartid)
+			   uint32_t event_code, u32 hartid, uint32_t edata)
 {
 	int i, cidx;
 
@@ -650,10 +671,11 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask,
 			continue;
 		if (active_events[hartid][i] != SBI_PMU_EVENT_IDX_INVALID)
 			continue;
-		if (SBI_PMU_FW_MAX <= event_code &&
+		if (SBI_PMU_FW_PLATFORM == event_code &&
 		    pmu_dev && pmu_dev->fw_counter_match_code) {
-			if (!pmu_dev->fw_counter_match_code(cidx - num_hw_ctrs,
-							    event_code))
+			if (!pmu_dev->fw_counter_match_code(cidx -
+							    num_hw_ctrs,
+							    edata))
 				continue;
 		}
 
@@ -675,7 +697,7 @@ int sbi_pmu_ctr_cfg_match(unsigned long cidx_base, unsigned long cidx_mask,
 	if ((cidx_base + sbi_fls(cidx_mask)) >= total_ctrs)
 		return SBI_EINVAL;
 
-	event_type = pmu_event_validate(event_idx);
+	event_type = pmu_event_validate(event_idx, event_data);
 	if (event_type < 0)
 		return SBI_EINVAL;
 	event_code = get_cidx_code(event_idx);
@@ -693,7 +715,8 @@ int sbi_pmu_ctr_cfg_match(unsigned long cidx_base, unsigned long cidx_mask,
 
 	if (event_type == SBI_PMU_EVENT_TYPE_FW) {
 		/* Any firmware counter can be used track any firmware event */
-		ctr_idx = pmu_ctr_find_fw(cidx_base, cidx_mask, event_code, hartid);
+		ctr_idx = pmu_ctr_find_fw(cidx_base, cidx_mask, event_code,
+					  hartid, event_data);
 	} else {
 		ctr_idx = pmu_ctr_find_hw(cidx_base, cidx_mask, flags, event_idx,
 					  event_data);
@@ -710,15 +733,16 @@ skip_match:
 		if (flags & SBI_PMU_CFG_FLAG_AUTO_START)
 			pmu_ctr_start_hw(ctr_idx, 0, false);
 	} else if (event_type == SBI_PMU_EVENT_TYPE_FW) {
+		if (event_code == SBI_PMU_FW_PLATFORM)
+			fw_counters_data[hartid][ctr_idx - num_hw_ctrs] =
+								event_data;
 		if (flags & SBI_PMU_CFG_FLAG_CLEAR_VALUE)
-			fw_counters_value[hartid][ctr_idx - num_hw_ctrs] = 0;
+			fw_counters_data[hartid][ctr_idx - num_hw_ctrs] = 0;
 		if (flags & SBI_PMU_CFG_FLAG_AUTO_START) {
-			if (SBI_PMU_FW_MAX <= event_code &&
+			if (SBI_PMU_FW_PLATFORM == event_code &&
 			    pmu_dev && pmu_dev->fw_counter_start) {
-				ret = pmu_dev->fw_counter_start(
-					ctr_idx - num_hw_ctrs, event_code,
-					fw_counters_value[hartid][ctr_idx - num_hw_ctrs],
-					true);
+				ret = pmu_dev->fw_counter_start(ctr_idx - num_hw_ctrs,
+							event_data);
 				if (ret)
 					return ret;
 			}
@@ -743,7 +767,7 @@ int sbi_pmu_ctr_incr_fw(enum sbi_pmu_fw_event_code_id fw_id)
 	for (cidx = num_hw_ctrs; cidx < total_ctrs; cidx++) {
 		if (get_cidx_code(active_events[hartid][cidx]) == fw_id &&
 		    (fw_counters_started[hartid] & BIT(cidx - num_hw_ctrs))) {
-			fcounter = &fw_counters_value[hartid][cidx - num_hw_ctrs];
+			fcounter = &fw_counters_data[hartid][cidx - num_hw_ctrs];
 			break;
 		}
 	}
@@ -803,7 +827,7 @@ static void pmu_reset_event_map(u32 hartid)
 	for (j = 3; j < total_ctrs; j++)
 		active_events[hartid][j] = SBI_PMU_EVENT_IDX_INVALID;
 	for (j = 0; j < SBI_PMU_FW_CTR_MAX; j++)
-		fw_counters_value[hartid][j] = 0;
+		fw_counters_data[hartid][j] = 0;
 	fw_counters_started[hartid] = 0;
 }
 
-- 
2.34.1



From mchitale at ventanamicro.com  Fri Feb 17 06:44:13 2023
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Fri, 17 Feb 2023 20:14:13 +0530
Subject: [PATCH v2 5/5] lib: sbi_pmu: Add hartid parameter PMU device ops
In-Reply-To: <20230217144413.3720307-1-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com>
Message-ID: <20230217144413.3720307-6-mchitale@ventanamicro.com>

Pass the hartid parameter to custom PMU device ops. It is required by
platform event handler to be able to program correct registers for a
given counter.

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 include/sbi/sbi_pmu.h | 18 +++++++++++-------
 lib/sbi/sbi_pmu.c     | 28 +++++++++++++++++-----------
 2 files changed, 28 insertions(+), 18 deletions(-)

diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
index 467ec03..90a564f 100644
--- a/include/sbi/sbi_pmu.h
+++ b/include/sbi/sbi_pmu.h
@@ -31,14 +31,15 @@ struct sbi_pmu_device {
 	/**
 	 * Validate event code of custom firmware event
 	 */
-	int (*fw_event_validate_code)(uint64_t event_idx_code);
+	int (*fw_event_validate_encoding)(uint64_t event_idx_code);
 
 	/**
 	 * Match custom firmware counter with custom firmware event
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
 	 */
-	bool (*fw_counter_match_code)(uint32_t counter_index,
-				      uint64_t event_idx_code);
+	bool (*fw_counter_match_encoding)(uint32_t hartid,
+					  uint32_t counter_index,
+					  uint64_t event_idx_code);
 
 	/**
 	 * Fetch the max width of this counter in number of bits.
@@ -49,27 +50,30 @@ struct sbi_pmu_device {
 	 * Read value of custom firmware counter
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
 	 */
-	uint64_t (*fw_counter_read_value)(uint32_t counter_index);
+	uint64_t (*fw_counter_read_value)(uint32_t hartid,
+					  uint32_t counter_index);
 
 	/**
 	 * Write value to custom firmware counter
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
 	 */
-	void (*fw_counter_write_value)(uint32_t counter_index,
+	void (*fw_counter_write_value)(uint32_t hartid,
+				       uint32_t counter_index,
 				       uint64_t value);
 
 	/**
 	 * Start custom firmware counter
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
 	 */
-	int (*fw_counter_start)(uint32_t counter_index,
+	int (*fw_counter_start)(uint32_t hartid,
+				uint32_t counter_index,
 				uint64_t event_idx_code);
 
 	/**
 	 * Stop custom firmware counter
 	 * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
 	 */
-	int (*fw_counter_stop)(uint32_t counter_index);
+	int (*fw_counter_stop)(uint32_t hartid, uint32_t counter_index);
 
 	/**
 	 * Custom enable irq for hardware counter
diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 05e2571..4479aa4 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -123,8 +123,8 @@ static int pmu_event_validate(unsigned long event_idx, uint64_t edata)
 		break;
 	case SBI_PMU_EVENT_TYPE_FW:
 		if (SBI_PMU_FW_PLATFORM == event_idx_code &&
-		    pmu_dev && pmu_dev->fw_event_validate_code)
-			return pmu_dev->fw_event_validate_code(edata);
+		    pmu_dev && pmu_dev->fw_event_validate_encoding)
+			return pmu_dev->fw_event_validate_encoding(edata);
 		else
 			event_idx_code_max = SBI_PMU_FW_MAX;
 		break;
@@ -189,7 +189,8 @@ int sbi_pmu_ctr_fw_read(uint32_t cidx, uint64_t *cval)
 
 	if (SBI_PMU_FW_PLATFORM == event_code) {
 		if (pmu_dev && pmu_dev->fw_counter_read_value)
-			*cval = pmu_dev->fw_counter_read_value(cidx -
+			*cval = pmu_dev->fw_counter_read_value(hartid,
+							       cidx -
 							       num_hw_ctrs);
 		else
 			*cval = 0;
@@ -376,7 +377,8 @@ static int pmu_ctr_start_fw(uint32_t cidx, uint32_t event_code,
 		    }
 
 		if (ival_update)
-			pmu_dev->fw_counter_write_value(cidx - num_hw_ctrs,
+			pmu_dev->fw_counter_write_value(hartid,
+							cidx - num_hw_ctrs,
 							ival);
 
 		if (pmu_dev->fw_counter_start(hartid, cidx - num_hw_ctrs,
@@ -453,11 +455,12 @@ static int pmu_ctr_stop_hw(uint32_t cidx)
 
 static int pmu_ctr_stop_fw(uint32_t cidx, uint32_t event_code)
 {
+	u32 hartid = current_hartid();
 	int ret;
 
 	if (SBI_PMU_FW_PLATFORM == event_code &&
 	    pmu_dev && pmu_dev->fw_counter_stop) {
-		ret = pmu_dev->fw_counter_stop(cidx - num_hw_ctrs);
+		ret = pmu_dev->fw_counter_stop(hartid, cidx - num_hw_ctrs);
 		if (ret)
 			return ret;
 	}
@@ -672,10 +675,11 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask,
 		if (active_events[hartid][i] != SBI_PMU_EVENT_IDX_INVALID)
 			continue;
 		if (SBI_PMU_FW_PLATFORM == event_code &&
-		    pmu_dev && pmu_dev->fw_counter_match_code) {
-			if (!pmu_dev->fw_counter_match_code(cidx -
-							    num_hw_ctrs,
-							    edata))
+		    pmu_dev && pmu_dev->fw_counter_match_encoding) {
+			if (!pmu_dev->fw_counter_match_encoding(hartid,
+							        cidx -
+								num_hw_ctrs,
+								edata))
 				continue;
 		}
 
@@ -741,8 +745,10 @@ skip_match:
 		if (flags & SBI_PMU_CFG_FLAG_AUTO_START) {
 			if (SBI_PMU_FW_PLATFORM == event_code &&
 			    pmu_dev && pmu_dev->fw_counter_start) {
-				ret = pmu_dev->fw_counter_start(ctr_idx - num_hw_ctrs,
-							event_data);
+				ret = pmu_dev->fw_counter_start(hartid,
+								ctr_idx -
+								num_hw_ctrs,
+								event_data);
 				if (ret)
 					return ret;
 			}
-- 
2.34.1



From anup at brainfault.org  Fri Feb 17 08:24:29 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Feb 2023 21:54:29 +0530
Subject: [PATCH] lib: sbi_hart: Enable hcontext and scontext
In-Reply-To: <20230210085238.19723-1-nylon.chen@sifive.com>
References: <20230210085238.19723-1-nylon.chen@sifive.com>
Message-ID: <CAAhSdy08RiPB2D95YMpXH+vM4e1n9OMwvJ8t-_bfB9VMJo1DNw@mail.gmail.com>

On Fri, Feb 10, 2023 at 2:22 PM Nylon Chen <nylon.chen at sifive.com> wrote:
>
> According to the description in "riscv-state-enable[0]", to access
> h/scontext in S-Mode, we need to enable the 57th bit.
>
> If it is not enabled, an "illegal instruction" error will occur.
>
> Link: https://github.com/riscv/riscv-state-enable/blob/a28bfae443f350d5b4c42874f428367d5b322ffe/content.adoc [0]
>
> Signed-off-by: Nylon Chen <nylon.chen at sifive.com>
> Reviewed-by: Zong Li <zong.li at sifive.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  include/sbi/riscv_encoding.h | 2 ++
>  lib/sbi/sbi_hart.c           | 1 +
>  2 files changed, 3 insertions(+)
>
> diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
> index b0f08c8..4ebed97 100644
> --- a/include/sbi/riscv_encoding.h
> +++ b/include/sbi/riscv_encoding.h
> @@ -736,6 +736,8 @@
>  #define SMSTATEEN0_CS                  (_ULL(1) << SMSTATEEN0_CS_SHIFT)
>  #define SMSTATEEN0_FCSR_SHIFT          1
>  #define SMSTATEEN0_FCSR                        (_ULL(1) << SMSTATEEN0_FCSR_SHIFT)
> +#define SMSTATEEN0_CONTEXT_SHIFT       57
> +#define SMSTATEEN0_CONTEXT             (_ULL(1) << SMSTATEEN0_CONTEXT_SHIFT)
>  #define SMSTATEEN0_IMSIC_SHIFT         58
>  #define SMSTATEEN0_IMSIC               (_ULL(1) << SMSTATEEN0_IMSIC_SHIFT)
>  #define SMSTATEEN0_AIA_SHIFT           59
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 02ce991..5e06918 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -90,6 +90,7 @@ static void mstatus_init(struct sbi_scratch *scratch)
>                 mstateen_val |= ((uint64_t)csr_read(CSR_MSTATEEN0H)) << 32;
>  #endif
>                 mstateen_val |= SMSTATEEN_STATEN;
> +               mstateen_val |= SMSTATEEN0_CONTEXT;
>                 mstateen_val |= SMSTATEEN0_HSENVCFG;
>
>                 if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA))
> --
> 2.36.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From peterlin at andestech.com  Sat Feb 18 22:33:35 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Sun, 19 Feb 2023 14:33:35 +0800
Subject: [PATCH] lib: utils: fdt_fixup: Fix compile error
Message-ID: <20230219063335.30952-1-peterlin@andestech.com>

When building with GCC-10 or older versions, it throws the following
error:

 CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
 CC        platform/generic/lib/utils/fdt/fdt_fixup.o
lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
  376 |  next_entry:
      |  ^~~~~~~~~~

Thus move the label "next_entry" to the beginning of the for loop.
Fixed issue#288 [1]

[1]: https://github.com/riscv-software-src/opensbi/issues/288

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
 lib/utils/fdt/fdt_fixup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index 619e4f5..9dac98a 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -345,6 +345,7 @@ int fdt_reserved_memory_fixup(void *fdt)
 
 	i = 0;
 	sbi_domain_for_each_memregion(dom, reg) {
+next_entry:
 		/* Ignore MMIO or READABLE or WRITABLE or EXECUTABLE regions */
 		if (reg->flags & SBI_DOMAIN_MEMREGION_MMIO)
 			continue;
@@ -373,7 +374,6 @@ int fdt_reserved_memory_fixup(void *fdt)
 		filtered_base[i] = reg->base;
 		filtered_order[i] = reg->order;
 		i++;
-	next_entry:
 	}
 
 	for (j = 0; j < i; j++) {
-- 
2.34.1



From schwab at linux-m68k.org  Sun Feb 19 01:11:09 2023
From: schwab at linux-m68k.org (Andreas Schwab)
Date: Sun, 19 Feb 2023 10:11:09 +0100
Subject: [PATCH] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <20230219063335.30952-1-peterlin@andestech.com> (Yu Chien Peter
	Lin's message of "Sun, 19 Feb 2023 14:33:35 +0800")
References: <20230219063335.30952-1-peterlin@andestech.com>
Message-ID: <874jrij9zm.fsf@linux-m68k.org>

On Feb 19 2023, Yu Chien Peter Lin wrote:

> Thus move the label "next_entry" to the beginning of the for loop.

This changes the semantics, because jumping to the top of the loop skips
the update expression.

-- 
Andreas Schwab, schwab at linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."


From wxjstz at 126.com  Sun Feb 19 01:28:31 2023
From: wxjstz at 126.com (Xiang W)
Date: Sun, 19 Feb 2023 17:28:31 +0800
Subject: [PATCH] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <20230219063335.30952-1-peterlin@andestech.com>
References: <20230219063335.30952-1-peterlin@andestech.com>
Message-ID: <ce307b4bb7b3f46ba601ead6bd74915d0e51c961.camel@126.com>

? 2023-02-19???? 14:33 +0800?Yu Chien Peter Lin???
> When building with GCC-10 or older versions, it throws the following
> error:
> 
> ?CC-DEP??? platform/generic/lib/utils/fdt/fdt_fixup.dep
> ?CC??????? platform/generic/lib/utils/fdt/fdt_fixup.o
> lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
> lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
> ? 376 |? next_entry:
> ????? |? ^~~~~~~~~~
> 
> Thus move the label "next_entry" to the beginning of the for loop.
> Fixed issue#288 [1]
> 
> [1]: https://github.com/riscv-software-src/opensbi/issues/288
> 
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> ?lib/utils/fdt/fdt_fixup.c | 2 +-
> ?1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 619e4f5..9dac98a 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -345,6 +345,7 @@ int fdt_reserved_memory_fixup(void *fdt)
> ?
> ????????i = 0;
> ????????sbi_domain_for_each_memregion(dom, reg) {
> +next_entry:
> ????????????????/* Ignore MMIO or READABLE or WRITABLE or EXECUTABLE regions */
> ????????????????if (reg->flags & SBI_DOMAIN_MEMREGION_MMIO)
> ????????????????????????continue;
> @@ -373,7 +374,6 @@ int fdt_reserved_memory_fixup(void *fdt)
> ????????????????filtered_base[i] = reg->base;
> ????????????????filtered_order[i] = reg->order;
> ????????????????i++;
> -???????next_entry:
> ????????}
I suggest to remove goto. 

diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index 619e4f5..e23517d 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
                        return SBI_ENOSPC;
                }
 
+               int overlap = 0;
                addr = reg->base;
                for (j = 0; j < i; j++) {
                        if (addr == filtered_base[j]
                            && filtered_order[j] < reg->order) {
+                               overlap = 1;
                                filtered_order[j] = reg->order;
-                               goto next_entry;
+                               break;
                        }
                }
 
-               filtered_base[i] = reg->base;
-               filtered_order[i] = reg->order;
-               i++;
-       next_entry:
+               if (!overlap) {
+                       filtered_base[i] = reg->base;
+                       filtered_order[i] = reg->order;
+                       i++;
+               }
        }
 
        for (j = 0; j < i; j++) {



Regards,
Xiang
> ?
> ????????for (j = 0; j < i; j++) {
> -- 
> 2.34.1
> 
> 



From peterlin at andestech.com  Sun Feb 19 09:44:19 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Sun, 19 Feb 2023 17:44:19 +0000
Subject: [PATCH] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <874jrij9zm.fsf@linux-m68k.org>
References: <20230219063335.30952-1-peterlin@andestech.com>
 <874jrij9zm.fsf@linux-m68k.org>
Message-ID: <Y/Jf2w3TaEfRiMlz@APC323>

On Sun, Feb 19, 2023 at 10:11:09AM +0100, Andreas Schwab wrote:
> On Feb 19 2023, Yu Chien Peter Lin wrote:
> 
> > Thus move the label "next_entry" to the beginning of the for loop.
> 
> This changes the semantics, because jumping to the top of the loop skips
> the update expression.
> 
> -- 
> Andreas Schwab, schwab at linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
> "And now for something completely different."

Hi Andreas,

Thanks for pointing this out!

Hi Xiang,

I've set the patch state to superseded,
Could you send your fix?

Thanks
Peter Lin


From wxjstz at 126.com  Sun Feb 19 02:15:59 2023
From: wxjstz at 126.com (Xiang W)
Date: Sun, 19 Feb 2023 18:15:59 +0800
Subject: [PATCH] lib: utils: fdt_fixup: Fix compile error
Message-ID: <20230219101559.708444-1-wxjstz@126.com>

When building with GCC-10 or older versions, it throws the following
error:

 CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
 CC        platform/generic/lib/utils/fdt/fdt_fixup.o
lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
  376 |  next_entry:
      |  ^~~~~~~~~~

Thus move the label "next_entry" to the beginning of the for loop.
Fixed issue#288 [1]

[1]: https://github.com/riscv-software-src/opensbi/issues/288

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index 619e4f5..e23517d 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
 			return SBI_ENOSPC;
 		}
 
+		int overlap = 0;
 		addr = reg->base;
 		for (j = 0; j < i; j++) {
 			if (addr == filtered_base[j]
 			    && filtered_order[j] < reg->order) {
+				overlap = 1;
 				filtered_order[j] = reg->order;
-				goto next_entry;
+				break;
 			}
 		}
 
-		filtered_base[i] = reg->base;
-		filtered_order[i] = reg->order;
-		i++;
-	next_entry:
+		if (!overlap) {
+			filtered_base[i] = reg->base;
+			filtered_order[i] = reg->order;
+			i++;
+		}
 	}
 
 	for (j = 0; j < i; j++) {
-- 
2.39.1



From schwab at linux-m68k.org  Sun Feb 19 02:20:43 2023
From: schwab at linux-m68k.org (Andreas Schwab)
Date: Sun, 19 Feb 2023 11:20:43 +0100
Subject: [PATCH] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <20230219101559.708444-1-wxjstz@126.com> (Xiang W.'s message of
	"Sun, 19 Feb 2023 18:15:59 +0800")
References: <20230219101559.708444-1-wxjstz@126.com>
Message-ID: <87zg9ahs78.fsf@linux-m68k.org>

On Feb 19 2023, Xiang W wrote:

> Thus move the label "next_entry" to the beginning of the for loop.

This is not what the patch does.

-- 
Andreas Schwab, schwab at linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."


From wxjstz at 126.com  Sun Feb 19 02:31:16 2023
From: wxjstz at 126.com (Xiang W)
Date: Sun, 19 Feb 2023 18:31:16 +0800
Subject: [PATCH v2] lib: utils: fdt_fixup: Fix compile error
Message-ID: <20230219103116.710854-1-wxjstz@126.com>

When building with GCC-10 or older versions, it throws the following
error:

 CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
 CC        platform/generic/lib/utils/fdt/fdt_fixup.o
lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
  376 |  next_entry:
      |  ^~~~~~~~~~

Remove the goto statement. Fixed issue#288 [1]

[1]: https://github.com/riscv-software-src/opensbi/issues/288

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Signed-off-by: Xiang W <wxjstz at 126.com>
---
Changes in v2:
- Fix a comment issue raised by Andreas Schwab

 lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index 619e4f5..e23517d 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
 			return SBI_ENOSPC;
 		}
 
+		int overlap = 0;
 		addr = reg->base;
 		for (j = 0; j < i; j++) {
 			if (addr == filtered_base[j]
 			    && filtered_order[j] < reg->order) {
+				overlap = 1;
 				filtered_order[j] = reg->order;
-				goto next_entry;
+				break;
 			}
 		}
 
-		filtered_base[i] = reg->base;
-		filtered_order[i] = reg->order;
-		i++;
-	next_entry:
+		if (!overlap) {
+			filtered_base[i] = reg->base;
+			filtered_order[i] = reg->order;
+			i++;
+		}
 	}
 
 	for (j = 0; j < i; j++) {
-- 
2.39.1



From wxjstz at 126.com  Sun Feb 19 04:41:12 2023
From: wxjstz at 126.com (Xiang W)
Date: Sun, 19 Feb 2023 20:41:12 +0800
Subject: [PATCH] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <87zg9ahs78.fsf@linux-m68k.org>
References: <20230219101559.708444-1-wxjstz@126.com>
	 <87zg9ahs78.fsf@linux-m68k.org>
Message-ID: <dfc354c6fe7d172aa5ee81a79a0d2393a3ad61be.camel@126.com>

? 2023-02-19???? 11:20 +0100?Andreas Schwab???
> On Feb 19 2023, Xiang W wrote:
> 
> > Thus move the label "next_entry" to the beginning of the for loop.
> 
> This is not what the patch does.
new patch at https://lists.infradead.org/pipermail/opensbi/2023-February/004494.html

Regards,
Xiang
> 



From e.shatokhin at yadro.com  Sun Feb 19 14:20:29 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Mon, 20 Feb 2023 01:20:29 +0300
Subject: [PATCH 0/2] lib: sbi: Prevent system hang when bringing up secondary harts
Message-ID: <20230219222031.3053255-1-e.shatokhin@yadro.com>

The boot hart can race with the secondary harts when it tries to bring them
up at startup time. This results in occasional system hangs at boot, which
can be reproduced in QEMU with a Linux guest, even with '-machine virt'.

When the Linux kernel needs to bring up the non-boot harts, the boot hart
enters opensbi via a ecall and eventually gets to sbi_hsm_hart_start().
Other (secondary) harts are spinning in the while loop in
sbi_hsm_hart_wait() at that moment:

	/* Wait for state transition requested by sbi_hsm_hart_start() */
	while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
		wfi();
	};

If I understand it correctly, the idea is that sbi_hsm_hart_start() should
first set the secondary hart's state to SBI_HSM_STATE_START_PENDING
atomically and check the previous value of the state. This makes sure no
other hart would try to modify the secondary hart's scratch area in the
meantime.

The boot hart then reads the init count of the secondary hart and sets the
address it should eventually jump to, as well as other data:

	init_count = sbi_init_count(hartid);
	rscratch->next_arg1 = arg1;
	rscratch->next_addr = saddr;
	rscratch->next_mode = smode;

Then, the boot hart sends IPI to the secondary hart, so that it would stop
spinning in the loop above and would continue.

However, as we observed during the test runs for our systems, the secondary
harts could leave the loop in sbi_hsm_hart_wait() as soon as their state is
set to SBI_HSM_STATE_START_PENDING. As a result, the following situation is
possible:


Boot hart (#0)                        Secondary hart (#1)
runs Linux startup code               waits in sbi_hsm_hart_wait()
sbi_ecall(SBI_EXT_HSM,
          SBI_EXT_HSM_HART_START,
          ...)
enters sbi_hsm_hart_start()
sets state to START_PENDING
                                      leaves sbi_hsm_hart_wait()
                                      runs to the end of init_warmboot()
                                      returns to scratch->next_addr
                                      (next_addr can be garbage here)

sets next_addr, etc. for hart #1
(no good: hart #1 has already left)

sends IPI to hart #1
(no good either)
                   (system might hang or crash)

This can be reproduced easier, if one inserts a delay into
sbi_hsm_hart_start() right before "init_count = sbi_init_count(hartid);".
In our case, "sbi_timer_mdelay(10)" was enough for the issue to manifest
each time.

The patch series fixes 2 issues that could make the secondary harts exit
the while loop in sbi_hsm_hart_wait() prematurely, i.e. before the IPI:

1. "wfi" instruction is just a hint for the CPU, it might or might not
actually wait for an interrupt to happen in the hart. RISC-V Privileged
Architectures spec. v20211203 even states that "a legal implementation is to
simply implement WFI as a NOP".

2. Since commit 50d4fde1c5a4 ("lib: Remove redundant sbi_platform_ipi_clear()
calls"), the IPI sent from the boot hart in wake_coldboot_harts() is never
cleared in the secondary harts. So, they enter sbi_hsm_hart_wait() with
already pending IPIs and do not wait for new ones.

In both cases, the secondary hart might leave the loop as soon as its state
is set to SBI_HSM_STATE_START_PENDING, leading to the race described above.

Regards,
Evgenii





From e.shatokhin at yadro.com  Sun Feb 19 14:20:30 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Mon, 20 Feb 2023 01:20:30 +0300
Subject: [PATCH 1/2] lib: sbi: Actually wait for an IPI in sbi_hsm_hart_wait()
In-Reply-To: <20230219222031.3053255-1-e.shatokhin@yadro.com>
References: <20230219222031.3053255-1-e.shatokhin@yadro.com>
Message-ID: <20230219222031.3053255-2-e.shatokhin@yadro.com>

"wfi" instruction is not guaranteed to wait for an interrupt to be received
by the hart, it is just a hint for the CPU. According to RISC-V Privileged
Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
legal.

As a result, a secondary (non-boot) hart waiting in the while loop in
sbi_hsm_hart_wait() might leave the loop and continue even if it got no
IPI. This could lead to the following race condition when booting Linux,
for example:

  Boot hart (#0)                        Secondary hart (#1)
  runs Linux startup code               waits in sbi_hsm_hart_wait()

  sbi_ecall(SBI_EXT_HSM,
            SBI_EXT_HSM_HART_START,
            ...)
  enters sbi_hsm_hart_start()
  sets state of hart #1 to START_PENDING
                                        leaves sbi_hsm_hart_wait()
                                        runs to the end of init_warmboot()
                                        returns to scratch->next_addr
                                        (next_addr can be garbage here)

  sets next_addr, etc. for hart #1
  (no good: hart #1 has already left)

  sends IPI to hart #1
  (no good either)

If this happens, the secondary hart jumps to a wrong next_addr at the end
of init_warmboot(), which leads to a system hang or crash.

To reproduce the issue more reliably, one could add a delay in
sbi_hsm_hart_start(): after setting the hart's state but before sending
IPI to that hart:

    hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
                            SBI_HSM_STATE_START_PENDING);
    ...
  + sbi_timer_mdelay(10);
    init_count = sbi_init_count(hartid);
    rscratch->next_arg1 = arg1;
    rscratch->next_addr = saddr;

The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
and 2 or more CPUs, with Linux as the guest OS.

Let us actually check in sbi_hsm_hart_wait(), if the waiting hart has
received an IPI.

Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
---
 lib/sbi/sbi_hsm.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index 8499bb1..0e5ff3d 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -108,6 +108,8 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
 static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
 {
 	unsigned long saved_mie;
+	unsigned long ipi_received;
+	unsigned int hstate;
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
 	/* Save MIE CSR */
@@ -117,9 +119,11 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
 	csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
 
 	/* Wait for state transition requested by sbi_hsm_hart_start() */
-	while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
+	do {
 		wfi();
-	};
+		ipi_received = csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP);
+		hstate = atomic_read(&hdata->state);
+	} while (hstate != SBI_HSM_STATE_START_PENDING || !ipi_received);
 
 	/* Restore MIE CSR */
 	csr_write(CSR_MIE, saved_mie);
-- 
2.34.1




From e.shatokhin at yadro.com  Sun Feb 19 14:20:31 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Mon, 20 Feb 2023 01:20:31 +0300
Subject: [PATCH 2/2] lib: sbi: Clear IPIs before init_warm_startup in non-boot harts
In-Reply-To: <20230219222031.3053255-1-e.shatokhin@yadro.com>
References: <20230219222031.3053255-1-e.shatokhin@yadro.com>
Message-ID: <20230219222031.3053255-3-e.shatokhin@yadro.com>

Since commit 50d4fde1c5a4 ("lib: Remove redundant sbi_platform_ipi_clear()
calls"), the IPI sent from the boot hart in wake_coldboot_harts() is not
cleared in the secondary harts until they reach sbi_ipi_init(). However,
sbi_hsm_init() and sbi_hsm_hart_wait() are called earlier, so a secondary
hart might enter sbi_hsm_hart_wait() with an already pending IPI.

Because of this, the secondary hart might exit the "wfi" loop in
sbi_hsm_hart_wait() right after the boot hart changes state of that hart to
SBI_HSM_STATE_START_PENDING. Then, depending on the hardware, it might be
able to reach sbi_hart_switch_mode() at the end of init_warmboot() before
the boot hart has set next_addr, etc., for it in sbi_hsm_hart_start(). In
this case, the secondary hart will jump to a wrong address.

This behaviour can be observed, for example, in a QEMU VM (QEMU 7.2.0) with
"-machine virt" running a Linux guest. Inserting delays after setting the
hart state to SBI_HSM_STATE_START_PENDING in sbi_hsm_hart_start() allows
reproducing the issue more reliably.

The comment in wait_for_coldboot() suggests that the initial IPI is needed
in the warm resume path, so let us clear it before init_warm_startup()
only.

To do this, sbi_ipi_raw_clear() was created similar to sbi_ipi_raw_send().

Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
---
 include/sbi/sbi_ipi.h | 2 ++
 lib/sbi/sbi_init.c    | 6 ++++--
 lib/sbi/sbi_ipi.c     | 6 ++++++
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
index f6ac807..f384e74 100644
--- a/include/sbi/sbi_ipi.h
+++ b/include/sbi/sbi_ipi.h
@@ -77,6 +77,8 @@ void sbi_ipi_process(void);
 
 int sbi_ipi_raw_send(u32 target_hart);
 
+void sbi_ipi_raw_clear(u32 target_hart);
+
 const struct sbi_ipi_device *sbi_ipi_get_device(void);
 
 void sbi_ipi_set_device(const struct sbi_ipi_device *dev);
diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
index e353c33..a9a4084 100644
--- a/lib/sbi/sbi_init.c
+++ b/lib/sbi/sbi_init.c
@@ -438,10 +438,12 @@ static void __noreturn init_warmboot(struct sbi_scratch *scratch, u32 hartid)
 	if (hstate < 0)
 		sbi_hart_hang();
 
-	if (hstate == SBI_HSM_STATE_SUSPENDED)
+	if (hstate == SBI_HSM_STATE_SUSPENDED) {
 		init_warm_resume(scratch);
-	else
+	} else {
+		sbi_ipi_raw_clear(hartid);
 		init_warm_startup(scratch, hartid);
+	}
 
 	sbi_hart_switch_mode(hartid, scratch->next_arg1,
 			     scratch->next_addr,
diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 1bcc2e4..b9f6205 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -217,6 +217,12 @@ int sbi_ipi_raw_send(u32 target_hart)
 	return 0;
 }
 
+void sbi_ipi_raw_clear(u32 target_hart)
+{
+	if (ipi_dev && ipi_dev->ipi_clear)
+		ipi_dev->ipi_clear(target_hart);
+}
+
 const struct sbi_ipi_device *sbi_ipi_get_device(void)
 {
 	return ipi_dev;
-- 
2.34.1




From jrtc27 at jrtc27.com  Sun Feb 19 14:38:12 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Sun, 19 Feb 2023 22:38:12 +0000
Subject: [PATCH 1/2] lib: sbi: Actually wait for an IPI in
 sbi_hsm_hart_wait()
In-Reply-To: <20230219222031.3053255-2-e.shatokhin@yadro.com>
References: <20230219222031.3053255-1-e.shatokhin@yadro.com>
 <20230219222031.3053255-2-e.shatokhin@yadro.com>
Message-ID: <19015380-F8C5-4DF3-AA0A-E4C2EDAC350F@jrtc27.com>

On 19 Feb 2023, at 22:20, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
> 
> "wfi" instruction is not guaranteed to wait for an interrupt to be received
> by the hart, it is just a hint for the CPU. According to RISC-V Privileged
> Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
> legal.
> 
> As a result, a secondary (non-boot) hart waiting in the while loop in
> sbi_hsm_hart_wait() might leave the loop and continue even if it got no
> IPI. This could lead to the following race condition when booting Linux,
> for example:
> 
>  Boot hart (#0)                        Secondary hart (#1)
>  runs Linux startup code               waits in sbi_hsm_hart_wait()
> 
>  sbi_ecall(SBI_EXT_HSM,
>            SBI_EXT_HSM_HART_START,
>            ...)
>  enters sbi_hsm_hart_start()
>  sets state of hart #1 to START_PENDING
>                                        leaves sbi_hsm_hart_wait()
>                                        runs to the end of init_warmboot()
>                                        returns to scratch->next_addr
>                                        (next_addr can be garbage here)
> 
>  sets next_addr, etc. for hart #1
>  (no good: hart #1 has already left)
> 
>  sends IPI to hart #1
>  (no good either)
> 
> If this happens, the secondary hart jumps to a wrong next_addr at the end
> of init_warmboot(), which leads to a system hang or crash.
> 
> To reproduce the issue more reliably, one could add a delay in
> sbi_hsm_hart_start(): after setting the hart's state but before sending
> IPI to that hart:
> 
>    hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
>                            SBI_HSM_STATE_START_PENDING);
>    ...
>  + sbi_timer_mdelay(10);
>    init_count = sbi_init_count(hartid);
>    rscratch->next_arg1 = arg1;
>    rscratch->next_addr = saddr;
> 
> The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
> and 2 or more CPUs, with Linux as the guest OS.
> 
> Let us actually check in sbi_hsm_hart_wait(), if the waiting hart has
> received an IPI.

Using the IPI itself as the synchronisation barrier seems awfully
brittle. Better to split SBI_HSM_STATE_START_PENDING into two states so
there?s an additional atomic_write that the atomic_read here can
synchronise with.

Jess

> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
> ---
> lib/sbi/sbi_hsm.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index 8499bb1..0e5ff3d 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -108,6 +108,8 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
> static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
> {
> 	unsigned long saved_mie;
> +	unsigned long ipi_received;
> +	unsigned int hstate;
> 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
> 							    hart_data_offset);
> 	/* Save MIE CSR */
> @@ -117,9 +119,11 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
> 	csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
> 
> 	/* Wait for state transition requested by sbi_hsm_hart_start() */
> -	while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
> +	do {
> 		wfi();
> -	};
> +		ipi_received = csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP);
> +		hstate = atomic_read(&hdata->state);
> +	} while (hstate != SBI_HSM_STATE_START_PENDING || !ipi_received);
> 
> 	/* Restore MIE CSR */
> 	csr_write(CSR_MIE, saved_mie);
> -- 
> 2.34.1
> 
> 
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



From jrtc27 at jrtc27.com  Sun Feb 19 14:41:32 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Sun, 19 Feb 2023 22:41:32 +0000
Subject: [PATCH 1/2] lib: sbi: Actually wait for an IPI in
 sbi_hsm_hart_wait()
In-Reply-To: <19015380-F8C5-4DF3-AA0A-E4C2EDAC350F@jrtc27.com>
References: <20230219222031.3053255-1-e.shatokhin@yadro.com>
 <20230219222031.3053255-2-e.shatokhin@yadro.com>
 <19015380-F8C5-4DF3-AA0A-E4C2EDAC350F@jrtc27.com>
Message-ID: <FEDB1517-A300-4E33-B8F5-D68D2CCA5A20@jrtc27.com>

On 19 Feb 2023, at 22:38, Jessica Clarke <jrtc27 at jrtc27.com> wrote:
> 
> On 19 Feb 2023, at 22:20, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>> 
>> "wfi" instruction is not guaranteed to wait for an interrupt to be received
>> by the hart, it is just a hint for the CPU. According to RISC-V Privileged
>> Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
>> legal.
>> 
>> As a result, a secondary (non-boot) hart waiting in the while loop in
>> sbi_hsm_hart_wait() might leave the loop and continue even if it got no
>> IPI. This could lead to the following race condition when booting Linux,
>> for example:
>> 
>> Boot hart (#0)                        Secondary hart (#1)
>> runs Linux startup code               waits in sbi_hsm_hart_wait()
>> 
>> sbi_ecall(SBI_EXT_HSM,
>>           SBI_EXT_HSM_HART_START,
>>           ...)
>> enters sbi_hsm_hart_start()
>> sets state of hart #1 to START_PENDING
>>                                       leaves sbi_hsm_hart_wait()
>>                                       runs to the end of init_warmboot()
>>                                       returns to scratch->next_addr
>>                                       (next_addr can be garbage here)
>> 
>> sets next_addr, etc. for hart #1
>> (no good: hart #1 has already left)
>> 
>> sends IPI to hart #1
>> (no good either)
>> 
>> If this happens, the secondary hart jumps to a wrong next_addr at the end
>> of init_warmboot(), which leads to a system hang or crash.
>> 
>> To reproduce the issue more reliably, one could add a delay in
>> sbi_hsm_hart_start(): after setting the hart's state but before sending
>> IPI to that hart:
>> 
>>   hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
>>                           SBI_HSM_STATE_START_PENDING);
>>   ...
>> + sbi_timer_mdelay(10);
>>   init_count = sbi_init_count(hartid);
>>   rscratch->next_arg1 = arg1;
>>   rscratch->next_addr = saddr;
>> 
>> The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
>> and 2 or more CPUs, with Linux as the guest OS.
>> 
>> Let us actually check in sbi_hsm_hart_wait(), if the waiting hart has
>> received an IPI.
> 
> Using the IPI itself as the synchronisation barrier seems awfully
> brittle. Better to split SBI_HSM_STATE_START_PENDING into two states so
> there?s an additional atomic_write that the atomic_read here can
> synchronise with.

Of course, care is needed to not expose this outside OpenSBI as these
states are part of the SBI. That is, from the outside world, both
states should appear as START_PENDING.

Jess

>> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
>> ---
>> lib/sbi/sbi_hsm.c | 8 ++++++--
>> 1 file changed, 6 insertions(+), 2 deletions(-)
>> 
>> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
>> index 8499bb1..0e5ff3d 100644
>> --- a/lib/sbi/sbi_hsm.c
>> +++ b/lib/sbi/sbi_hsm.c
>> @@ -108,6 +108,8 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>> static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>> {
>> 	unsigned long saved_mie;
>> +	unsigned long ipi_received;
>> +	unsigned int hstate;
>> 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>> 							    hart_data_offset);
>> 	/* Save MIE CSR */
>> @@ -117,9 +119,11 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>> 	csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
>> 
>> 	/* Wait for state transition requested by sbi_hsm_hart_start() */
>> -	while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
>> +	do {
>> 		wfi();
>> -	};
>> +		ipi_received = csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP);
>> +		hstate = atomic_read(&hdata->state);
>> +	} while (hstate != SBI_HSM_STATE_START_PENDING || !ipi_received);
>> 
>> 	/* Restore MIE CSR */
>> 	csr_write(CSR_MIE, saved_mie);
>> -- 
>> 2.34.1
>> 
>> 
>> 
>> -- 
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi



From e.shatokhin at yadro.com  Mon Feb 20 00:28:49 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Mon, 20 Feb 2023 11:28:49 +0300
Subject: [PATCH 1/2] lib: sbi: Actually wait for an IPI in
 sbi_hsm_hart_wait()
In-Reply-To: <19015380-F8C5-4DF3-AA0A-E4C2EDAC350F@jrtc27.com>
References: <20230219222031.3053255-1-e.shatokhin@yadro.com>
 <20230219222031.3053255-2-e.shatokhin@yadro.com>
 <19015380-F8C5-4DF3-AA0A-E4C2EDAC350F@jrtc27.com>
Message-ID: <af82fecf-776c-d1e1-544a-360783a8c002@yadro.com>

Hi,

On 20.02.2023 01:38, Jessica Clarke wrote:
> On 19 Feb 2023, at 22:20, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>>
>> "wfi" instruction is not guaranteed to wait for an interrupt to be received
>> by the hart, it is just a hint for the CPU. According to RISC-V Privileged
>> Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
>> legal.
>>
>> As a result, a secondary (non-boot) hart waiting in the while loop in
>> sbi_hsm_hart_wait() might leave the loop and continue even if it got no
>> IPI. This could lead to the following race condition when booting Linux,
>> for example:
>>
>>   Boot hart (#0)                        Secondary hart (#1)
>>   runs Linux startup code               waits in sbi_hsm_hart_wait()
>>
>>   sbi_ecall(SBI_EXT_HSM,
>>             SBI_EXT_HSM_HART_START,
>>             ...)
>>   enters sbi_hsm_hart_start()
>>   sets state of hart #1 to START_PENDING
>>                                         leaves sbi_hsm_hart_wait()
>>                                         runs to the end of init_warmboot()
>>                                         returns to scratch->next_addr
>>                                         (next_addr can be garbage here)
>>
>>   sets next_addr, etc. for hart #1
>>   (no good: hart #1 has already left)
>>
>>   sends IPI to hart #1
>>   (no good either)
>>
>> If this happens, the secondary hart jumps to a wrong next_addr at the end
>> of init_warmboot(), which leads to a system hang or crash.
>>
>> To reproduce the issue more reliably, one could add a delay in
>> sbi_hsm_hart_start(): after setting the hart's state but before sending
>> IPI to that hart:
>>
>>     hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
>>                             SBI_HSM_STATE_START_PENDING);
>>     ...
>>   + sbi_timer_mdelay(10);
>>     init_count = sbi_init_count(hartid);
>>     rscratch->next_arg1 = arg1;
>>     rscratch->next_addr = saddr;
>>
>> The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
>> and 2 or more CPUs, with Linux as the guest OS.
>>
>> Let us actually check in sbi_hsm_hart_wait(), if the waiting hart has
>> received an IPI.
> 
> Using the IPI itself as the synchronisation barrier seems awfully
> brittle. Better to split SBI_HSM_STATE_START_PENDING into two states so
> there?s an additional atomic_write that the atomic_read here can
> synchronise with.

Yes, using an additional state was the first idea I tried to implement 
to fix the issue. It turned out to be too error-prone, however: 
SBI_HSM_STATE_START_PENDING is used in a number of places in opensbi, 
and it is far from obvious how to use the new state correctly there, 
esp. without violation of the spec.

On the other hand, wait_for_coldboot() and wake_coldboot_harts() already 
use IPI and a condition variable 'coldboot_done' for synchronization, 
and it seems to work OK.

So, I suppose, relying on IPI in sbi_hsm_hart_wait() and 
sbi_hsm_hart_start() too might be a way to go. Looks like, it was the 
intended way:

* Set the state to START_PENDING and get exclusive access to the target 
hart's scratch area.
* Write next_addr, next_arg1, etc., for the target hart.
* Send IPI to the target hart to confirm that it might continue.

WFI might or might not wait for an interrupt in the target hart, so the 
boot hart must send an IPI to the target anyway to wake it reliably.

Or, perhaps, I am missing some particular issue with this approach?

> 
> Jess
> 
>> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
>> ---
>> lib/sbi/sbi_hsm.c | 8 ++++++--
>> 1 file changed, 6 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
>> index 8499bb1..0e5ff3d 100644
>> --- a/lib/sbi/sbi_hsm.c
>> +++ b/lib/sbi/sbi_hsm.c
>> @@ -108,6 +108,8 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>> static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>> {
>>        unsigned long saved_mie;
>> +     unsigned long ipi_received;
>> +     unsigned int hstate;
>>        struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>>                                                            hart_data_offset);
>>        /* Save MIE CSR */
>> @@ -117,9 +119,11 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>        csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
>>
>>        /* Wait for state transition requested by sbi_hsm_hart_start() */
>> -     while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
>> +     do {
>>                wfi();
>> -     };
>> +             ipi_received = csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP);
>> +             hstate = atomic_read(&hdata->state);
>> +     } while (hstate != SBI_HSM_STATE_START_PENDING || !ipi_received);
>>
>>        /* Restore MIE CSR */
>>        csr_write(CSR_MIE, saved_mie);
>> --
>> 2.34.1

Regards,
Evgenii






From jrtc27 at jrtc27.com  Mon Feb 20 00:37:23 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Mon, 20 Feb 2023 08:37:23 +0000
Subject: [PATCH 1/2] lib: sbi: Actually wait for an IPI in
 sbi_hsm_hart_wait()
In-Reply-To: <af82fecf-776c-d1e1-544a-360783a8c002@yadro.com>
References: <20230219222031.3053255-1-e.shatokhin@yadro.com>
 <20230219222031.3053255-2-e.shatokhin@yadro.com>
 <19015380-F8C5-4DF3-AA0A-E4C2EDAC350F@jrtc27.com>
 <af82fecf-776c-d1e1-544a-360783a8c002@yadro.com>
Message-ID: <508ADC60-D1F1-4F05-8321-05FDA14F224E@jrtc27.com>

On 20 Feb 2023, at 08:28, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
> 
> Hi,
> 
> On 20.02.2023 01:38, Jessica Clarke wrote:
>> On 19 Feb 2023, at 22:20, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>>> 
>>> "wfi" instruction is not guaranteed to wait for an interrupt to be received
>>> by the hart, it is just a hint for the CPU. According to RISC-V Privileged
>>> Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
>>> legal.
>>> 
>>> As a result, a secondary (non-boot) hart waiting in the while loop in
>>> sbi_hsm_hart_wait() might leave the loop and continue even if it got no
>>> IPI. This could lead to the following race condition when booting Linux,
>>> for example:
>>> 
>>>  Boot hart (#0)                        Secondary hart (#1)
>>>  runs Linux startup code               waits in sbi_hsm_hart_wait()
>>> 
>>>  sbi_ecall(SBI_EXT_HSM,
>>>            SBI_EXT_HSM_HART_START,
>>>            ...)
>>>  enters sbi_hsm_hart_start()
>>>  sets state of hart #1 to START_PENDING
>>>                                        leaves sbi_hsm_hart_wait()
>>>                                        runs to the end of init_warmboot()
>>>                                        returns to scratch->next_addr
>>>                                        (next_addr can be garbage here)
>>> 
>>>  sets next_addr, etc. for hart #1
>>>  (no good: hart #1 has already left)
>>> 
>>>  sends IPI to hart #1
>>>  (no good either)
>>> 
>>> If this happens, the secondary hart jumps to a wrong next_addr at the end
>>> of init_warmboot(), which leads to a system hang or crash.
>>> 
>>> To reproduce the issue more reliably, one could add a delay in
>>> sbi_hsm_hart_start(): after setting the hart's state but before sending
>>> IPI to that hart:
>>> 
>>>    hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
>>>                            SBI_HSM_STATE_START_PENDING);
>>>    ...
>>>  + sbi_timer_mdelay(10);
>>>    init_count = sbi_init_count(hartid);
>>>    rscratch->next_arg1 = arg1;
>>>    rscratch->next_addr = saddr;
>>> 
>>> The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
>>> and 2 or more CPUs, with Linux as the guest OS.
>>> 
>>> Let us actually check in sbi_hsm_hart_wait(), if the waiting hart has
>>> received an IPI.
>> Using the IPI itself as the synchronisation barrier seems awfully
>> brittle. Better to split SBI_HSM_STATE_START_PENDING into two states so
>> there?s an additional atomic_write that the atomic_read here can
>> synchronise with.
> 
> Yes, using an additional state was the first idea I tried to implement to fix the issue. It turned out to be too error-prone, however: SBI_HSM_STATE_START_PENDING is used in a number of places in opensbi, and it is far from obvious how to use the new state correctly there, esp. without violation of the spec.

I only see 5, and they don?t look particularly hard...

> On the other hand, wait_for_coldboot() and wake_coldboot_harts() already use IPI and a condition variable 'coldboot_done' for synchronization, and it seems to work OK.

But coldboot_done is what those synchronise on; the IPI is merely the
mechanism by which the waiter gets forcefully woken and made to check
coldboot_done.

> So, I suppose, relying on IPI in sbi_hsm_hart_wait() and sbi_hsm_hart_start() too might be a way to go. Looks like, it was the intended way:
> 
> * Set the state to START_PENDING and get exclusive access to the target hart's scratch area.
> * Write next_addr, next_arg1, etc., for the target hart.
> * Send IPI to the target hart to confirm that it might continue.
> 
> WFI might or might not wait for an interrupt in the target hart, so the boot hart must send an IPI to the target anyway to wake it reliably.

But you can never truly know that the IPI you got is for the reason you
think you got it. You *must* have additional state to indicate that the
scratch area has been initialised; the fact you received an IPI somehow
is not sufficient.

Jess

> Or, perhaps, I am missing some particular issue with this approach?
> 
>> Jess
>>> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
>>> ---
>>> lib/sbi/sbi_hsm.c | 8 ++++++--
>>> 1 file changed, 6 insertions(+), 2 deletions(-)
>>> 
>>> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
>>> index 8499bb1..0e5ff3d 100644
>>> --- a/lib/sbi/sbi_hsm.c
>>> +++ b/lib/sbi/sbi_hsm.c
>>> @@ -108,6 +108,8 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>>> static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>> {
>>>       unsigned long saved_mie;
>>> +     unsigned long ipi_received;
>>> +     unsigned int hstate;
>>>       struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>>>                                                           hart_data_offset);
>>>       /* Save MIE CSR */
>>> @@ -117,9 +119,11 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>>       csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
>>> 
>>>       /* Wait for state transition requested by sbi_hsm_hart_start() */
>>> -     while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
>>> +     do {
>>>               wfi();
>>> -     };
>>> +             ipi_received = csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP);
>>> +             hstate = atomic_read(&hdata->state);
>>> +     } while (hstate != SBI_HSM_STATE_START_PENDING || !ipi_received);
>>> 
>>>       /* Restore MIE CSR */
>>>       csr_write(CSR_MIE, saved_mie);
>>> --
>>> 2.34.1
> 
> Regards,
> Evgenii



From e.shatokhin at yadro.com  Mon Feb 20 00:56:31 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Mon, 20 Feb 2023 11:56:31 +0300
Subject: [PATCH 1/2] lib: sbi: Actually wait for an IPI in
 sbi_hsm_hart_wait()
In-Reply-To: <508ADC60-D1F1-4F05-8321-05FDA14F224E@jrtc27.com>
References: <20230219222031.3053255-1-e.shatokhin@yadro.com>
 <20230219222031.3053255-2-e.shatokhin@yadro.com>
 <19015380-F8C5-4DF3-AA0A-E4C2EDAC350F@jrtc27.com>
 <af82fecf-776c-d1e1-544a-360783a8c002@yadro.com>
 <508ADC60-D1F1-4F05-8321-05FDA14F224E@jrtc27.com>
Message-ID: <bf12cae3-d9c9-b7e1-99f9-2d4e351fd3b1@yadro.com>

On 20.02.2023 11:37, Jessica Clarke wrote:
> ?????????! ?????? ?????? ?? ???????? ????????!?
> 
> On 20 Feb 2023, at 08:28, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>>
>> Hi,
>>
>> On 20.02.2023 01:38, Jessica Clarke wrote:
>>> On 19 Feb 2023, at 22:20, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>>>>
>>>> "wfi" instruction is not guaranteed to wait for an interrupt to be received
>>>> by the hart, it is just a hint for the CPU. According to RISC-V Privileged
>>>> Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
>>>> legal.
>>>>
>>>> As a result, a secondary (non-boot) hart waiting in the while loop in
>>>> sbi_hsm_hart_wait() might leave the loop and continue even if it got no
>>>> IPI. This could lead to the following race condition when booting Linux,
>>>> for example:
>>>>
>>>>   Boot hart (#0)                        Secondary hart (#1)
>>>>   runs Linux startup code               waits in sbi_hsm_hart_wait()
>>>>
>>>>   sbi_ecall(SBI_EXT_HSM,
>>>>             SBI_EXT_HSM_HART_START,
>>>>             ...)
>>>>   enters sbi_hsm_hart_start()
>>>>   sets state of hart #1 to START_PENDING
>>>>                                         leaves sbi_hsm_hart_wait()
>>>>                                         runs to the end of init_warmboot()
>>>>                                         returns to scratch->next_addr
>>>>                                         (next_addr can be garbage here)
>>>>
>>>>   sets next_addr, etc. for hart #1
>>>>   (no good: hart #1 has already left)
>>>>
>>>>   sends IPI to hart #1
>>>>   (no good either)
>>>>
>>>> If this happens, the secondary hart jumps to a wrong next_addr at the end
>>>> of init_warmboot(), which leads to a system hang or crash.
>>>>
>>>> To reproduce the issue more reliably, one could add a delay in
>>>> sbi_hsm_hart_start(): after setting the hart's state but before sending
>>>> IPI to that hart:
>>>>
>>>>     hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
>>>>                             SBI_HSM_STATE_START_PENDING);
>>>>     ...
>>>>   + sbi_timer_mdelay(10);
>>>>     init_count = sbi_init_count(hartid);
>>>>     rscratch->next_arg1 = arg1;
>>>>     rscratch->next_addr = saddr;
>>>>
>>>> The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
>>>> and 2 or more CPUs, with Linux as the guest OS.
>>>>
>>>> Let us actually check in sbi_hsm_hart_wait(), if the waiting hart has
>>>> received an IPI.
>>> Using the IPI itself as the synchronisation barrier seems awfully
>>> brittle. Better to split SBI_HSM_STATE_START_PENDING into two states so
>>> there?s an additional atomic_write that the atomic_read here can
>>> synchronise with.
>>
>> Yes, using an additional state was the first idea I tried to implement to fix the issue. It turned out to be too error-prone, however: SBI_HSM_STATE_START_PENDING is used in a number of places in opensbi, and it is far from obvious how to use the new state correctly there, esp. without violation of the spec.
> 
> I only see 5, and they don?t look particularly hard...
> 
>> On the other hand, wait_for_coldboot() and wake_coldboot_harts() already use IPI and a condition variable 'coldboot_done' for synchronization, and it seems to work OK.
> 
> But coldboot_done is what those synchronise on; the IPI is merely the
> mechanism by which the waiter gets forcefully woken and made to check
> coldboot_done.
> 
>> So, I suppose, relying on IPI in sbi_hsm_hart_wait() and sbi_hsm_hart_start() too might be a way to go. Looks like, it was the intended way:
>>
>> * Set the state to START_PENDING and get exclusive access to the target hart's scratch area.
>> * Write next_addr, next_arg1, etc., for the target hart.
>> * Send IPI to the target hart to confirm that it might continue.
>>
>> WFI might or might not wait for an interrupt in the target hart, so the boot hart must send an IPI to the target anyway to wake it reliably.
> 
> But you can never truly know that the IPI you got is for the reason you
> think you got it. 

You are right, and my second patch is just about such problem.

> You *must* have additional state to indicate that the
> scratch area has been initialised; the fact you received an IPI somehow
> is not sufficient.

How about using a condition variable here too and keeping it in struct 
sbi_hsm_data? I would rather not touch the hdata->state itself as I 
explained earlier.

IPI will then be used only to wake the target hart, not to confirm that 
next_addr and other data are ready.

> 
> Jess
> 
>> Or, perhaps, I am missing some particular issue with this approach?
>>
>>> Jess
>>>> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
>>>> ---
>>>> lib/sbi/sbi_hsm.c | 8 ++++++--
>>>> 1 file changed, 6 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
>>>> index 8499bb1..0e5ff3d 100644
>>>> --- a/lib/sbi/sbi_hsm.c
>>>> +++ b/lib/sbi/sbi_hsm.c
>>>> @@ -108,6 +108,8 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>>>> static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>>> {
>>>>        unsigned long saved_mie;
>>>> +     unsigned long ipi_received;
>>>> +     unsigned int hstate;
>>>>        struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>>>>                                                            hart_data_offset);
>>>>        /* Save MIE CSR */
>>>> @@ -117,9 +119,11 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>>>        csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
>>>>
>>>>        /* Wait for state transition requested by sbi_hsm_hart_start() */
>>>> -     while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
>>>> +     do {
>>>>                wfi();
>>>> -     };
>>>> +             ipi_received = csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP);
>>>> +             hstate = atomic_read(&hdata->state);
>>>> +     } while (hstate != SBI_HSM_STATE_START_PENDING || !ipi_received);
>>>>
>>>>        /* Restore MIE CSR */
>>>>        csr_write(CSR_MIE, saved_mie);
>>>> --
>>>> 2.34.1
>>
>> Regards,
>> Evgenii
> 
> 




From jrtc27 at jrtc27.com  Mon Feb 20 01:00:59 2023
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Mon, 20 Feb 2023 09:00:59 +0000
Subject: [PATCH 1/2] lib: sbi: Actually wait for an IPI in
 sbi_hsm_hart_wait()
In-Reply-To: <bf12cae3-d9c9-b7e1-99f9-2d4e351fd3b1@yadro.com>
References: <20230219222031.3053255-1-e.shatokhin@yadro.com>
 <20230219222031.3053255-2-e.shatokhin@yadro.com>
 <19015380-F8C5-4DF3-AA0A-E4C2EDAC350F@jrtc27.com>
 <af82fecf-776c-d1e1-544a-360783a8c002@yadro.com>
 <508ADC60-D1F1-4F05-8321-05FDA14F224E@jrtc27.com>
 <bf12cae3-d9c9-b7e1-99f9-2d4e351fd3b1@yadro.com>
Message-ID: <B5EC308D-E768-49BB-A638-D3174526A372@jrtc27.com>

On 20 Feb 2023, at 08:56, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
> 
> On 20.02.2023 11:37, Jessica Clarke wrote:
>> ?????????! ?????? ?????? ?? ???????? ????????!?
>> On 20 Feb 2023, at 08:28, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>>> 
>>> Hi,
>>> 
>>> On 20.02.2023 01:38, Jessica Clarke wrote:
>>>> On 19 Feb 2023, at 22:20, Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>>>>> 
>>>>> "wfi" instruction is not guaranteed to wait for an interrupt to be received
>>>>> by the hart, it is just a hint for the CPU. According to RISC-V Privileged
>>>>> Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
>>>>> legal.
>>>>> 
>>>>> As a result, a secondary (non-boot) hart waiting in the while loop in
>>>>> sbi_hsm_hart_wait() might leave the loop and continue even if it got no
>>>>> IPI. This could lead to the following race condition when booting Linux,
>>>>> for example:
>>>>> 
>>>>>  Boot hart (#0)                        Secondary hart (#1)
>>>>>  runs Linux startup code               waits in sbi_hsm_hart_wait()
>>>>> 
>>>>>  sbi_ecall(SBI_EXT_HSM,
>>>>>            SBI_EXT_HSM_HART_START,
>>>>>            ...)
>>>>>  enters sbi_hsm_hart_start()
>>>>>  sets state of hart #1 to START_PENDING
>>>>>                                        leaves sbi_hsm_hart_wait()
>>>>>                                        runs to the end of init_warmboot()
>>>>>                                        returns to scratch->next_addr
>>>>>                                        (next_addr can be garbage here)
>>>>> 
>>>>>  sets next_addr, etc. for hart #1
>>>>>  (no good: hart #1 has already left)
>>>>> 
>>>>>  sends IPI to hart #1
>>>>>  (no good either)
>>>>> 
>>>>> If this happens, the secondary hart jumps to a wrong next_addr at the end
>>>>> of init_warmboot(), which leads to a system hang or crash.
>>>>> 
>>>>> To reproduce the issue more reliably, one could add a delay in
>>>>> sbi_hsm_hart_start(): after setting the hart's state but before sending
>>>>> IPI to that hart:
>>>>> 
>>>>>    hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
>>>>>                            SBI_HSM_STATE_START_PENDING);
>>>>>    ...
>>>>>  + sbi_timer_mdelay(10);
>>>>>    init_count = sbi_init_count(hartid);
>>>>>    rscratch->next_arg1 = arg1;
>>>>>    rscratch->next_addr = saddr;
>>>>> 
>>>>> The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
>>>>> and 2 or more CPUs, with Linux as the guest OS.
>>>>> 
>>>>> Let us actually check in sbi_hsm_hart_wait(), if the waiting hart has
>>>>> received an IPI.
>>>> Using the IPI itself as the synchronisation barrier seems awfully
>>>> brittle. Better to split SBI_HSM_STATE_START_PENDING into two states so
>>>> there?s an additional atomic_write that the atomic_read here can
>>>> synchronise with.
>>> 
>>> Yes, using an additional state was the first idea I tried to implement to fix the issue. It turned out to be too error-prone, however: SBI_HSM_STATE_START_PENDING is used in a number of places in opensbi, and it is far from obvious how to use the new state correctly there, esp. without violation of the spec.
>> I only see 5, and they don?t look particularly hard...
>>> On the other hand, wait_for_coldboot() and wake_coldboot_harts() already use IPI and a condition variable 'coldboot_done' for synchronization, and it seems to work OK.
>> But coldboot_done is what those synchronise on; the IPI is merely the
>> mechanism by which the waiter gets forcefully woken and made to check
>> coldboot_done.
>>> So, I suppose, relying on IPI in sbi_hsm_hart_wait() and sbi_hsm_hart_start() too might be a way to go. Looks like, it was the intended way:
>>> 
>>> * Set the state to START_PENDING and get exclusive access to the target hart's scratch area.
>>> * Write next_addr, next_arg1, etc., for the target hart.
>>> * Send IPI to the target hart to confirm that it might continue.
>>> 
>>> WFI might or might not wait for an interrupt in the target hart, so the boot hart must send an IPI to the target anyway to wake it reliably.
>> But you can never truly know that the IPI you got is for the reason you
>> think you got it. 
> 
> You are right, and my second patch is just about such problem.
> 
>> You *must* have additional state to indicate that the
>> scratch area has been initialised; the fact you received an IPI somehow
>> is not sufficient.
> 
> How about using a condition variable here too and keeping it in struct sbi_hsm_data? I would rather not touch the hdata->state itself as I explained earlier.
> 
> IPI will then be used only to wake the target hart, not to confirm that next_addr and other data are ready.

That works, but you need to be very careful about how and when you
initialise it, since it?s not as simple as putting it in the existing
state word.

Jess

>> Jess
>>> Or, perhaps, I am missing some particular issue with this approach?
>>> 
>>>> Jess
>>>>> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
>>>>> ---
>>>>> lib/sbi/sbi_hsm.c | 8 ++++++--
>>>>> 1 file changed, 6 insertions(+), 2 deletions(-)
>>>>> 
>>>>> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
>>>>> index 8499bb1..0e5ff3d 100644
>>>>> --- a/lib/sbi/sbi_hsm.c
>>>>> +++ b/lib/sbi/sbi_hsm.c
>>>>> @@ -108,6 +108,8 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>>>>> static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>>>> {
>>>>>       unsigned long saved_mie;
>>>>> +     unsigned long ipi_received;
>>>>> +     unsigned int hstate;
>>>>>       struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>>>>>                                                           hart_data_offset);
>>>>>       /* Save MIE CSR */
>>>>> @@ -117,9 +119,11 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>>>>       csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
>>>>> 
>>>>>       /* Wait for state transition requested by sbi_hsm_hart_start() */
>>>>> -     while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
>>>>> +     do {
>>>>>               wfi();
>>>>> -     };
>>>>> +             ipi_received = csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP);
>>>>> +             hstate = atomic_read(&hdata->state);
>>>>> +     } while (hstate != SBI_HSM_STATE_START_PENDING || !ipi_received);
>>>>> 
>>>>>       /* Restore MIE CSR */
>>>>>       csr_write(CSR_MIE, saved_mie);
>>>>> --
>>>>> 2.34.1
>>> 
>>> Regards,
>>> Evgenii



From anup at brainfault.org  Mon Feb 20 07:20:26 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 20 Feb 2023 20:50:26 +0530
Subject: [PATCH v5 1/3] platform: generic: renesas: rzfive: Add support to
 configure the PMA
In-Reply-To: <20230213215111.32017-2-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20230213215111.32017-2-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <CAAhSdy1jPp0-AYKN4JRQg1=GTMi4CfE-SJ_nDiARymcCtzATSw@mail.gmail.com>

On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
> block that allows dynamic adjustment of memory attributes in the runtime.
> It contains a configurable amount of PMA entries implemented as CSR
> registers to control the attributes of memory locations in interest.
> Below are the memory attributes supported:
> * Device, Non-bufferable
> * Device, bufferable
> * Memory, Non-cacheable, Non-bufferable
> * Memory, Non-cacheable, Bufferable
> * Memory, Write-back, No-allocate
> * Memory, Write-back, Read-allocate
> * Memory, Write-back, Write-allocate
> * Memory, Write-back, Read and Write-allocate
>
> More info about PMA (section 10.3):
> Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> Example PMA region passed as a DT node from OpenSBI:
>     reserved-memory {
>         #address-cells = <2>;
>         #size-cells = <2>;
>         ranges;
>
>         pma_resv0 at 58000000 {
>             compatible = "shared-dma-pool";
>             reg = <0x0 0x58000000 0x0 0x08000000>;
>             no-map;
>             linux,dma-default;
>         };
>     };
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  platform/generic/Kconfig                     |   2 +
>  platform/generic/andes/Kconfig               |   5 +
>  platform/generic/andes/andes45-pma.c         | 350 +++++++++++++++++++
>  platform/generic/andes/objects.mk            |   2 +
>  platform/generic/include/andes/andes45_pma.h |  48 +++
>  5 files changed, 407 insertions(+)
>  create mode 100644 platform/generic/andes/Kconfig
>  create mode 100644 platform/generic/andes/andes45-pma.c
>  create mode 100644 platform/generic/include/andes/andes45_pma.h
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index c7f198a..39fb4e9 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -50,4 +50,6 @@ config PLATFORM_STARFIVE_JH7110
>         bool "StarFive JH7110 support"
>         default n
>
> +source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
> +
>  endif
> diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
> new file mode 100644
> index 0000000..3ad4e4c
> --- /dev/null
> +++ b/platform/generic/andes/Kconfig
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: BSD-2-Clause
> +
> +config ANDES45_PMA
> +       bool "Andes PMA support"
> +       default n
> diff --git a/platform/generic/andes/andes45-pma.c b/platform/generic/andes/andes45-pma.c
> new file mode 100644
> index 0000000..2745bc3
> --- /dev/null
> +++ b/platform/generic/andes/andes45-pma.c
> @@ -0,0 +1,350 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + *
> + * Copyright (c) 2020 Andes Technology Corporation
> + *
> + * Authors:
> + *      Nick Hu <nickhu at andestech.com>
> + *      Nylon Chen <nylon7 at andestech.com>
> + *      Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> + */
> +
> +#include <andes/andes45_pma.h>
> +#include <libfdt.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +
> +/* Configuration Registers */
> +#define ANDES45_CSR_MMSC_CFG           0xFC2
> +#define ANDES45_CSR_MMSC_PPMA_OFFSET   (1 << 30)
> +
> +#define ANDES45_PMAADDR_0              0xBD0
> +
> +#define ANDES45_PMACFG_0               0xBC0
> +
> +static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
> +{
> +#define switchcase_pma_cfg_read(__pma_cfg_off, __val)          \
> +       case __pma_cfg_off:                                     \
> +               __val = csr_read(__pma_cfg_off);                \
> +               break;
> +#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val)                \
> +       switchcase_pma_cfg_read(__pma_cfg_off + 0, __val)       \
> +       switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
> +
> +       unsigned long ret = 0;
> +
> +       switch (pma_cfg_off) {
> +       switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> +               break;
> +       }
> +
> +       return ret;
> +
> +#undef switchcase_pma_cfg_read_2
> +#undef switchcase_pma_cfg_read
> +}
> +
> +static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
> +{
> +#define switchcase_pma_cfg_write(__pma_cfg_off, __val)         \
> +       case __pma_cfg_off:                                     \
> +               csr_write(__pma_cfg_off, __val);                \
> +               break;
> +#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val)       \
> +       switchcase_pma_cfg_write(__pma_cfg_off + 0, __val)      \
> +       switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
> +
> +       switch (pma_cfg_off) {
> +       switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> +               break;
> +       }
> +
> +#undef switchcase_pma_cfg_write_2
> +#undef switchcase_pma_cfg_write
> +}
> +
> +static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
> +{
> +#define switchcase_pma_write(__pma_addr_off, __val)            \
> +       case __pma_addr_off:                                    \
> +               csr_write(__pma_addr_off, __val);               \
> +               break;
> +#define switchcase_pma_write_2(__pma_addr_off, __val)          \
> +       switchcase_pma_write(__pma_addr_off + 0, __val)         \
> +       switchcase_pma_write(__pma_addr_off + 1, __val)
> +#define switchcase_pma_write_4(__pma_addr_off, __val)          \
> +       switchcase_pma_write_2(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_write_8(__pma_addr_off, __val)          \
> +       switchcase_pma_write_4(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_write_16(__pma_addr_off, __val)         \
> +       switchcase_pma_write_8(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_8(__pma_addr_off + 8, __val)
> +
> +       switch (pma_addr_off) {
> +       switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> +               break;
> +       }
> +
> +#undef switchcase_pma_write_16
> +#undef switchcase_pma_write_8
> +#undef switchcase_pma_write_4
> +#undef switchcase_pma_write_2
> +#undef switchcase_pma_write
> +}
> +
> +static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
> +{
> +#define switchcase_pma_read(__pma_addr_off, __val)             \
> +       case __pma_addr_off:                                    \
> +               __val = csr_read(__pma_addr_off);               \
> +               break;
> +#define switchcase_pma_read_2(__pma_addr_off, __val)           \
> +       switchcase_pma_read(__pma_addr_off + 0, __val)          \
> +       switchcase_pma_read(__pma_addr_off + 1, __val)
> +#define switchcase_pma_read_4(__pma_addr_off, __val)           \
> +       switchcase_pma_read_2(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_read_8(__pma_addr_off, __val)           \
> +       switchcase_pma_read_4(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_read_16(__pma_addr_off, __val)          \
> +       switchcase_pma_read_8(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_8(__pma_addr_off + 8, __val)
> +
> +       unsigned long ret = 0;
> +
> +       switch (pma_addr_off) {
> +       switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> +               break;
> +       }
> +
> +       return ret;
> +
> +#undef switchcase_pma_read_16
> +#undef switchcase_pma_read_8
> +#undef switchcase_pma_read_4
> +#undef switchcase_pma_read_2
> +#undef switchcase_pma_read
> +}
> +
> +static unsigned long
> +andes45_pma_setup(const struct andes45_pma_region *pma_region,
> +                 unsigned int entry_id)
> +{
> +       unsigned long size = pma_region->size;
> +       unsigned long addr = pma_region->pa;
> +       unsigned int pma_cfg_addr;
> +       unsigned long pmacfg_val;
> +       unsigned long pmaaddr;
> +       char *pmaxcfg;
> +
> +       /* Check for 4KiB granularity */
> +       if (size < (1 << 12))
> +               return SBI_EINVAL;
> +
> +       /* Check size is power of 2 */
> +       if (size & (size - 1))
> +               return SBI_EINVAL;
> +
> +       if (entry_id > 15)
> +               return SBI_EINVAL;
> +
> +       if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
> +               return SBI_EINVAL;
> +
> +       if ((addr & (size - 1)) != 0)
> +               return SBI_EINVAL;
> +
> +       pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
> +       pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
> +       pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
> +       *pmaxcfg = 0;
> +       *pmaxcfg = pma_region->flags;
> +
> +       andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
> +
> +       pmaaddr = (addr >> 2) + (size >> 3) - 1;
> +
> +       andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
> +
> +       return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
> +                       pmaaddr : SBI_EINVAL;
> +}
> +
> +static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
> +                               unsigned int index, int parent)
> +{
> +       int na = fdt_address_cells(fdt, 0);
> +       int ns = fdt_size_cells(fdt, 0);
> +       static bool dma_default = false;
> +       fdt32_t addr_high, addr_low;
> +       fdt32_t size_high, size_low;
> +       int subnode, err;
> +       fdt32_t reg[4];
> +       fdt32_t *val;
> +       char name[32];
> +
> +       addr_high = (u64)pma->pa >> 32;
> +       addr_low = pma->pa;
> +       size_high = (u64)pma->size >> 32;
> +       size_low = pma->size;
> +
> +       if (na > 1 && addr_high)
> +               sbi_snprintf(name, sizeof(name),
> +                            "pma_resv%d@%x,%x", index,
> +                            addr_high, addr_low);
> +       else
> +               sbi_snprintf(name, sizeof(name),
> +                            "pma_resv%d@%x", index,
> +                            addr_low);
> +
> +       subnode = fdt_add_subnode(fdt, parent, name);
> +       if (subnode < 0)
> +               return subnode;
> +
> +       if (pma->shared_dma) {
> +               err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       if (pma->no_map) {
> +               err = fdt_setprop_empty(fdt, subnode, "no-map");
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       /* Linux allows single linux,dma-default region. */
> +       if (pma->dma_default) {
> +               if (dma_default)
> +                       return SBI_EINVAL;
> +
> +               err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
> +               if (err < 0)
> +                       return err;
> +               dma_default = true;
> +       }
> +
> +       /* encode the <reg> property value */
> +       val = reg;
> +       if (na > 1)
> +               *val++ = cpu_to_fdt32(addr_high);
> +       *val++ = cpu_to_fdt32(addr_low);
> +       if (ns > 1)
> +               *val++ = cpu_to_fdt32(size_high);
> +       *val++ = cpu_to_fdt32(size_low);
> +
> +       err = fdt_setprop(fdt, subnode, "reg", reg,
> +                         (na + ns) * sizeof(fdt32_t));
> +       if (err < 0)
> +               return err;
> +
> +       return 0;
> +}
> +
> +static int andes45_fdt_reserved_memory_fixup(void *fdt,
> +                                            const struct andes45_pma_region *pma,
> +                                            unsigned int entry)
> +{
> +       int parent;
> +
> +       /* try to locate the reserved memory node */
> +       parent = fdt_path_offset(fdt, "/reserved-memory");
> +       if (parent < 0) {
> +               int na = fdt_address_cells(fdt, 0);
> +               int ns = fdt_size_cells(fdt, 0);
> +               int err;
> +
> +               /* if such node does not exist, create one */
> +               parent = fdt_add_subnode(fdt, 0, "reserved-memory");
> +               if (parent < 0)
> +                       return parent;
> +
> +               err = fdt_setprop_empty(fdt, parent, "ranges");
> +               if (err < 0)
> +                       return err;
> +
> +               err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
> +               if (err < 0)
> +                       return err;
> +
> +               err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       return andes45_fdt_pma_resv(fdt, pma, entry, parent);
> +}
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> +                             unsigned int pma_regions_count)
> +{
> +       unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
> +       unsigned int dt_populate_cnt;
> +       unsigned int i, j;
> +       unsigned long pa;
> +       void *fdt;
> +       int ret;
> +
> +       if (!pma_regions || !pma_regions_count)
> +               return 0;
> +
> +       if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
> +               return SBI_EINVAL;
> +
> +       if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
> +               return SBI_ENOTSUPP;
> +
> +       /* Configure the PMA regions */
> +       for (i = 0; i < pma_regions_count; i++) {
> +               pa = andes45_pma_setup(&pma_regions[i], i);
> +               if (pa == SBI_EINVAL)
> +                       return SBI_EINVAL;
> +       }
> +
> +       dt_populate_cnt = 0;
> +       for (i = 0; i < pma_regions_count; i++) {
> +               if (!pma_regions[i].dt_populate)
> +                       continue;
> +               dt_populate_cnt++;
> +       }
> +
> +       if (!dt_populate_cnt)
> +               return 0;
> +
> +       fdt = fdt_get_address();
> +
> +       ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
> +       if (ret < 0)
> +               return ret;
> +
> +       for (i = 0, j = 0; i < pma_regions_count; i++) {
> +               if (!pma_regions[i].dt_populate)
> +                       continue;
> +
> +               ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
> +               if (ret)
> +                       return ret;
> +       }
> +
> +       return 0;
> +}
> diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
> index 28275ef..ea6b561 100644
> --- a/platform/generic/andes/objects.mk
> +++ b/platform/generic/andes/objects.mk
> @@ -4,3 +4,5 @@
>
>  carray-platform_override_modules-$(CONFIG_PLATFORM_ANDES_AE350) += andes_ae350
>  platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
> +
> +platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
> diff --git a/platform/generic/include/andes/andes45_pma.h b/platform/generic/include/andes/andes45_pma.h
> new file mode 100644
> index 0000000..37ec77c
> --- /dev/null
> +++ b/platform/generic/include/andes/andes45_pma.h
> @@ -0,0 +1,48 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + */
> +
> +#ifndef _ANDES45_PMA_H_
> +#define _ANDES45_PMA_H_
> +
> +#include <sbi/sbi_types.h>
> +
> +#define ANDES45_MAX_PMA_REGIONS                        16
> +
> +/* Naturally aligned power of 2 region */
> +#define ANDES45_PMACFG_ETYP_NAPOT              3
> +
> +/* Memory, Non-cacheable, Bufferable */
> +#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF  (3 << 2)
> +
> +/**
> + * struct andes45_pma_region - Describes PMA regions
> + *
> + * @pa: Address to be configured in the PMA
> + * @size: Size of the region
> + * @flags: Flags to be set for the PMA region
> + * @dt_populate: Boolean flag indicating if the DT entry should be
> + *               populated for the given PMA region
> + * @shared_dma: Boolean flag if set "shared-dma-pool" property will
> + *              be set in the DT node
> + * @no_map: Boolean flag if set "no-map" property will be set in the
> + *          DT node
> + * @dma_default: Boolean flag if set "linux,dma-default" property will
> + *              be set in the DT node. Note Linux expects single node
> + *              with this property set.
> + */
> +struct andes45_pma_region {
> +       unsigned long pa;
> +       unsigned long size;
> +       u8 flags:7;
> +       bool dt_populate;
> +       bool shared_dma;
> +       bool no_map;
> +       bool dma_default;
> +};
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> +                             unsigned int pma_regions_count);
> +
> +#endif /* _ANDES45_PMA_H_ */
> --
> 2.17.1
>


From anup at brainfault.org  Mon Feb 20 07:21:32 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 20 Feb 2023 20:51:32 +0530
Subject: [PATCH v5 2/3] platform: generic: renesas: rzfive: Configure the
 PMA region
In-Reply-To: <20230213215111.32017-3-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20230213215111.32017-3-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <CAAhSdy0--LsA2qMa6EGo1puABQh-xv-OmWxTgYoh0AqAjDipMg@mail.gmail.com>

On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>
> On the Renesas RZ/Five SoC by default we want to configure 128MiB of memory
> ranging from 0x58000000 as a non-cacheable + bufferable region in the PMA
> and populate this region as PMA reserve DT node with shared DMA pool and
> no-map flags set so that Linux drivers requesting any DMA'able memory go
> through this region.
>
> PMA node passed to the above stack:
>
>         reserved-memory {
>             #address-cells = <2>;
>             #size-cells = <2>;
>             ranges;
>
>             pma_resv0 at 58000000 {
>                 compatible = "shared-dma-pool";
>                 reg = <0x0 0x58000000 0x0 0x08000000>;
>                 no-map;
>                 linux,dma-default;
>             };
>         };
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  platform/generic/Kconfig                 |  1 +
>  platform/generic/renesas/rzfive/rzfive.c | 21 +++++++++++++++++++++
>  2 files changed, 22 insertions(+)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index 39fb4e9..1f4f8e1 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -35,6 +35,7 @@ config PLATFORM_ANDES_AE350
>
>  config PLATFORM_RENESAS_RZFIVE
>         bool "Renesas RZ/Five support"
> +       select ANDES45_PMA
>         default n
>
>  config PLATFORM_SIFIVE_FU540
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index ee9c9c4..4d71d0d 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -4,10 +4,30 @@
>   *
>   */
>
> +#include <andes/andes45_pma.h>
>  #include <platform_override.h>
>  #include <sbi/sbi_domain.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
> +static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
> +       {
> +               .pa = 0x58000000,
> +               .size = 0x8000000,
> +               .flags = ANDES45_PMACFG_ETYP_NAPOT |
> +                        ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF,
> +               .dt_populate = true,
> +               .shared_dma = true,
> +               .no_map = true,
> +               .dma_default = true,
> +       },
> +};
> +
> +static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *match)
> +{
> +       return andes45_pma_setup_regions(renesas_rzfive_pma_regions,
> +                                        array_size(renesas_rzfive_pma_regions));
> +}
> +
>  int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>  {
>         /*
> @@ -34,4 +54,5 @@ static const struct fdt_match renesas_rzfive_match[] = {
>  const struct platform_override renesas_rzfive = {
>         .match_table = renesas_rzfive_match,
>         .early_init = renesas_rzfive_early_init,
> +       .final_init = renesas_rzfive_final_init,
>  };
> --
> 2.17.1
>


From anup at brainfault.org  Mon Feb 20 07:31:32 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 20 Feb 2023 21:01:32 +0530
Subject: [PATCH v5 3/3] platform: generic: renesas: rzfive: Add SBI EXT to
 check for enabling IOCP errata
In-Reply-To: <20230213215111.32017-4-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20230213215111.32017-4-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <CAAhSdy046QNs-ae+mKs1J0E0bPv_Ew0JP4sNQa2RVzPmW6u8EA@mail.gmail.com>

On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> errata should be applied to handle cache management.
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
>  platform/generic/include/andes/andes45.h | 25 +++++++++++++--
>  platform/generic/renesas/rzfive/rzfive.c | 41 ++++++++++++++++++++++++
>  2 files changed, 64 insertions(+), 2 deletions(-)
>
> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> index 08b3d18..d5d265f 100644
> --- a/platform/generic/include/andes/andes45.h
> +++ b/platform/generic/include/andes/andes45.h
> @@ -4,7 +4,28 @@
>  #define CSR_MARCHID_MICROID 0xfff
>
>  /* Memory and Miscellaneous Registers */
> -#define CSR_MCACHE_CTL 0x7ca
> -#define CSR_MCCTLCOMMAND 0x7cc
> +#define CSR_MCACHE_CTL         0x7ca
> +#define CSR_MCCTLCOMMAND       0x7cc
> +#define CSR_MICM_CFG           0xfc0
> +#define CSR_MDCM_CFG           0xfc1
> +#define CSR_MMSC_CFG           0xfc2
> +
> +#define MISA_20_OFFSET                 20
> +#define MISA_20_MASK                   (0x1 << MISA_20_OFFSET)
> +
> +#define MICM_CFG_ISZ_OFFSET            6
> +#define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
> +
> +#define MDCM_CFG_DSZ_OFFSET            6
> +#define MDCM_CFG_DSZ_MASK              (0x7  << MDCM_CFG_DSZ_OFFSET)
> +
> +#define MMSC_CFG_CCTLCSR_OFFSET                16
> +#define MMSC_CFG_CCTLCSR_MASK          (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> +#define MMSC_IOCP_OFFSET                       47
> +#define MMSC_IOCP_MASK                 (0x1ULL << MMSC_IOCP_OFFSET)
> +
> +#define MCACHE_CTL_CCTL_SUEN_OFFSET    8
> +#define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> +
>
>  #endif /* _RISCV_ANDES45_H */
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index 4d71d0d..9530d28 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -4,11 +4,14 @@
>   *
>   */
>
> +#include <andes/andes45.h>
>  #include <andes/andes45_pma.h>
>  #include <platform_override.h>
>  #include <sbi/sbi_domain.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
> +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND      0

The SBI vendor extension_id will be the same for all SoCs based on Andes
CPUs so the function_id space will be shared across all such SoCs.

How does Andes plan to manage the function_id space of SBI vendor extension ?

I suggest the following:
1) The function_id == 0 can be used to return an SoC identifier. For example,
     Renesas RzFive can have an SoC identifier "0".
2) Move RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to a
    separate header at platform/generic/include/andes/andes_sbi.h and
    change it's value to "1"

The above suggestion will help multiple SoC with Andes CPU share same
SBI function_id space for SBI vendor extension.

Regards,
Anup


> +
>  static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
>         {
>                 .pa = 0x58000000,
> @@ -28,6 +31,43 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
>                                          array_size(renesas_rzfive_pma_regions));
>  }
>
> +static bool renesas_rzfive_cpu_cache_controlable(void)
> +{
> +       return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
> +                (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
> +               (csr_read(CSR_MISA) & MISA_20_MASK) &&
> +               (csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
> +               (csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
> +}
> +
> +static bool renesas_rzfive_cpu_iocp_disabled(void)
> +{
> +       return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
> +}
> +
> +static bool renesas_rzfive_apply_iocp_sw_workaround(void)
> +{
> +       return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
> +}
> +
> +static int renesas_rzfive_vendor_ext_provider(long funcid,
> +                                             const struct sbi_trap_regs *regs,
> +                                             unsigned long *out_value,
> +                                             struct sbi_trap_info *out_trap,
> +                                             const struct fdt_match *match)
> +{
> +       switch (funcid) {
> +       case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
> +               *out_value = renesas_rzfive_apply_iocp_sw_workaround();
> +               break;
> +
> +       default:
> +               break;
> +       }
> +
> +       return 0;
> +}
> +
>  int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>  {
>         /*
> @@ -55,4 +95,5 @@ const struct platform_override renesas_rzfive = {
>         .match_table = renesas_rzfive_match,
>         .early_init = renesas_rzfive_early_init,
>         .final_init = renesas_rzfive_final_init,
> +       .vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
>  };
> --
> 2.17.1
>


From anup at brainfault.org  Mon Feb 20 07:38:18 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 20 Feb 2023 21:08:18 +0530
Subject: [PATCH v2] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <20230219103116.710854-1-wxjstz@126.com>
References: <20230219103116.710854-1-wxjstz@126.com>
Message-ID: <CAAhSdy0UonkLe-+wezkh16xeyw=kxD4hkNb_uVKDsBGhojbvug@mail.gmail.com>

On Sun, Feb 19, 2023 at 4:02 PM Xiang W <wxjstz at 126.com> wrote:
>
> When building with GCC-10 or older versions, it throws the following
> error:
>
>  CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
>  CC        platform/generic/lib/utils/fdt/fdt_fixup.o
> lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
> lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
>   376 |  next_entry:
>       |  ^~~~~~~~~~
>
> Remove the goto statement. Fixed issue#288 [1]
>
> [1]: https://github.com/riscv-software-src/opensbi/issues/288
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Signed-off-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
> Changes in v2:
> - Fix a comment issue raised by Andreas Schwab
>
>  lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 619e4f5..e23517d 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
>                         return SBI_ENOSPC;
>                 }
>
> +               int overlap = 0;
>                 addr = reg->base;
>                 for (j = 0; j < i; j++) {
>                         if (addr == filtered_base[j]
>                             && filtered_order[j] < reg->order) {
> +                               overlap = 1;
>                                 filtered_order[j] = reg->order;
> -                               goto next_entry;
> +                               break;
>                         }
>                 }
>
> -               filtered_base[i] = reg->base;
> -               filtered_order[i] = reg->order;
> -               i++;
> -       next_entry:
> +               if (!overlap) {
> +                       filtered_base[i] = reg->base;
> +                       filtered_order[i] = reg->order;
> +                       i++;
> +               }
>         }
>
>         for (j = 0; j < i; j++) {
> --
> 2.39.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From bmeng.cn at gmail.com  Mon Feb 20 17:25:16 2023
From: bmeng.cn at gmail.com (Bin Meng)
Date: Tue, 21 Feb 2023 09:25:16 +0800
Subject: [PATCH v2] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <20230219103116.710854-1-wxjstz@126.com>
References: <20230219103116.710854-1-wxjstz@126.com>
Message-ID: <CAEUhbmVV4AZDZB3vMwpYJxESvnQPJEFJhaH_f2zqcNUzmeyC4w@mail.gmail.com>

On Sun, Feb 19, 2023 at 6:33 PM Xiang W <wxjstz at 126.com> wrote:
>
> When building with GCC-10 or older versions, it throws the following
> error:
>
>  CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
>  CC        platform/generic/lib/utils/fdt/fdt_fixup.o
> lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
> lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
>   376 |  next_entry:
>       |  ^~~~~~~~~~
>
> Remove the goto statement. Fixed issue#288 [1]

nits: Use "Resolves" tag

>
> [1]: https://github.com/riscv-software-src/opensbi/issues/288

So this becomes:

Resolves: https://github.com/riscv-software-src/opensbi/issues/288

>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> ---
> Changes in v2:
> - Fix a comment issue raised by Andreas Schwab
>
>  lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 619e4f5..e23517d 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
>                         return SBI_ENOSPC;
>                 }
>
> +               int overlap = 0;

bool overlap = false;

>                 addr = reg->base;
>                 for (j = 0; j < i; j++) {
>                         if (addr == filtered_base[j]
>                             && filtered_order[j] < reg->order) {
> +                               overlap = 1;
>                                 filtered_order[j] = reg->order;
> -                               goto next_entry;
> +                               break;
>                         }
>                 }
>
> -               filtered_base[i] = reg->base;
> -               filtered_order[i] = reg->order;
> -               i++;
> -       next_entry:
> +               if (!overlap) {
> +                       filtered_base[i] = reg->base;
> +                       filtered_order[i] = reg->order;
> +                       i++;
> +               }
>         }
>
>         for (j = 0; j < i; j++) {
> --
> 2.39.1
>

Regards,
Bin


From bmeng.cn at gmail.com  Mon Feb 20 20:16:36 2023
From: bmeng.cn at gmail.com (Bin Meng)
Date: Tue, 21 Feb 2023 12:16:36 +0800
Subject: [PATCH v3] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <20230221041203.819559-1-wxjstz@126.com>
References: <20230221041203.819559-1-wxjstz@126.com>
Message-ID: <CAEUhbmVOkR-nGhofwBokxs8=MpjdCWg1h2jaf=FfHqmigZ585w@mail.gmail.com>

On Tue, Feb 21, 2023 at 12:13 PM Xiang W <wxjstz at 126.com> wrote:
>
> When building with GCC-10 or older versions, it throws the following
> error:
>
>  CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
>  CC        platform/generic/lib/utils/fdt/fdt_fixup.o
> lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
> lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
>   376 |  next_entry:
>       |  ^~~~~~~~~~
>
> Remove the goto statement. Resolves issue#288 [1]

This is still not correct

Should be:

Remove the goto statement.

Resolves: https://github.com/riscv-software-src/opensbi/issues/288
SoB
SoB
RB
etc...

>
> [1]: https://github.com/riscv-software-src/opensbi/issues/288
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes in v3:
> - Fix a comment issue raised by Bin Meng
> - Changed the type of "overlap" from int to bool as suggested by Bin Meng
>
> Changes in v2:
> - Fix a comment issue raised by Andreas Schwab
>  lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 619e4f5..c10179b 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
>                         return SBI_ENOSPC;
>                 }
>
> +               bool overlap = false;
>                 addr = reg->base;
>                 for (j = 0; j < i; j++) {
>                         if (addr == filtered_base[j]
>                             && filtered_order[j] < reg->order) {
> +                               overlap = true;
>                                 filtered_order[j] = reg->order;
> -                               goto next_entry;
> +                               break;
>                         }
>                 }
>
> -               filtered_base[i] = reg->base;
> -               filtered_order[i] = reg->order;
> -               i++;
> -       next_entry:
> +               if (!overlap) {
> +                       filtered_base[i] = reg->base;
> +                       filtered_order[i] = reg->order;
> +                       i++;
> +               }
>         }
>
>         for (j = 0; j < i; j++) {
> --

Regards,
Bin


From wxjstz at 126.com  Mon Feb 20 20:12:03 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 21 Feb 2023 12:12:03 +0800
Subject: [PATCH v3] lib: utils: fdt_fixup: Fix compile error
Message-ID: <20230221041203.819559-1-wxjstz@126.com>

When building with GCC-10 or older versions, it throws the following
error:

 CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
 CC        platform/generic/lib/utils/fdt/fdt_fixup.o
lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
  376 |  next_entry:
      |  ^~~~~~~~~~

Remove the goto statement. Resolves issue#288 [1]

[1]: https://github.com/riscv-software-src/opensbi/issues/288

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Signed-off-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
Changes in v3:
- Fix a comment issue raised by Bin Meng
- Changed the type of "overlap" from int to bool as suggested by Bin Meng

Changes in v2:
- Fix a comment issue raised by Andreas Schwab
 lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index 619e4f5..c10179b 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
 			return SBI_ENOSPC;
 		}
 
+		bool overlap = false;
 		addr = reg->base;
 		for (j = 0; j < i; j++) {
 			if (addr == filtered_base[j]
 			    && filtered_order[j] < reg->order) {
+				overlap = true;
 				filtered_order[j] = reg->order;
-				goto next_entry;
+				break;
 			}
 		}
 
-		filtered_base[i] = reg->base;
-		filtered_order[i] = reg->order;
-		i++;
-	next_entry:
+		if (!overlap) {
+			filtered_base[i] = reg->base;
+			filtered_order[i] = reg->order;
+			i++;
+		}
 	}
 
 	for (j = 0; j < i; j++) {
-- 
2.39.1



From wxjstz at 126.com  Mon Feb 20 21:07:07 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 21 Feb 2023 13:07:07 +0800
Subject: [PATCH v4] lib: utils: fdt_fixup: Fix compile error
Message-ID: <20230221050707.825994-1-wxjstz@126.com>

When building with GCC-10 or older versions, it throws the following
error:

 CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
 CC        platform/generic/lib/utils/fdt/fdt_fixup.o
lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
  376 |  next_entry:
      |  ^~~~~~~~~~

Remove the goto statement.

Resolves: https://github.com/riscv-software-src/opensbi/issues/288

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Signed-off-by: Xiang W <wxjstz at 126.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
Changes in v4:
 - Continue to fix the comment in Changes v3

Changes in v3:
- Fix a comment issue raised by Bin Meng
- Changed the type of "overlap" from int to bool as suggested by Bin Meng

Changes in v2:
- Fix a comment issue raised by Andreas Schwab
 lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

 lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
index 619e4f5..c10179b 100644
--- a/lib/utils/fdt/fdt_fixup.c
+++ b/lib/utils/fdt/fdt_fixup.c
@@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
 			return SBI_ENOSPC;
 		}
 
+		bool overlap = false;
 		addr = reg->base;
 		for (j = 0; j < i; j++) {
 			if (addr == filtered_base[j]
 			    && filtered_order[j] < reg->order) {
+				overlap = true;
 				filtered_order[j] = reg->order;
-				goto next_entry;
+				break;
 			}
 		}
 
-		filtered_base[i] = reg->base;
-		filtered_order[i] = reg->order;
-		i++;
-	next_entry:
+		if (!overlap) {
+			filtered_base[i] = reg->base;
+			filtered_order[i] = reg->order;
+			i++;
+		}
 	}
 
 	for (j = 0; j < i; j++) {
-- 
2.39.1



From bmeng.cn at gmail.com  Mon Feb 20 21:44:19 2023
From: bmeng.cn at gmail.com (Bin Meng)
Date: Tue, 21 Feb 2023 13:44:19 +0800
Subject: [PATCH v4] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <20230221050707.825994-1-wxjstz@126.com>
References: <20230221050707.825994-1-wxjstz@126.com>
Message-ID: <CAEUhbmWK0-4bDcwRnmjbweUWkAGeXrNNqBTK67ZJeP7k27wmag@mail.gmail.com>

On Tue, Feb 21, 2023 at 1:08 PM Xiang W <wxjstz at 126.com> wrote:
>
> When building with GCC-10 or older versions, it throws the following
> error:
>
>  CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
>  CC        platform/generic/lib/utils/fdt/fdt_fixup.o
> lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
> lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
>   376 |  next_entry:
>       |  ^~~~~~~~~~
>
> Remove the goto statement.
>
> Resolves: https://github.com/riscv-software-src/opensbi/issues/288
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
> Changes in v4:
>  - Continue to fix the comment in Changes v3
>
> Changes in v3:
> - Fix a comment issue raised by Bin Meng
> - Changed the type of "overlap" from int to bool as suggested by Bin Meng
>
> Changes in v2:
> - Fix a comment issue raised by Andreas Schwab
>  lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
>  lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>

Reviewed-by: Bin Meng <bmeng at tinylab.org>


From peterlin at andestech.com  Tue Feb 21 09:06:54 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Tue, 21 Feb 2023 17:06:54 +0000
Subject: [PATCH v5 3/3] platform: generic: renesas: rzfive: Add SBI EXT
 to check for enabling IOCP errata
In-Reply-To: <CAAhSdy046QNs-ae+mKs1J0E0bPv_Ew0JP4sNQa2RVzPmW6u8EA@mail.gmail.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
 <20230213215111.32017-4-prabhakar.mahadev-lad.rj@bp.renesas.com>
 <CAAhSdy046QNs-ae+mKs1J0E0bPv_Ew0JP4sNQa2RVzPmW6u8EA@mail.gmail.com>
Message-ID: <Y/T6Lh2ASrllYBTw@APC323>

On Mon, Feb 20, 2023 at 09:01:32PM +0530, Anup Patel wrote:
> On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
> <prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
> >
> > I/O Coherence Port (IOCP) provides an AXI interface for connecting
> > external non-caching masters, such as DMA controllers. The accesses
> > from IOCP are coherent with D-Caches and L2 Cache.
> >
> > IOCP is a specification option and is disabled on the Renesas RZ/Five
> > SoC due to this reason IP blocks using DMA will fail.
> >
> > As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> > software. Firstly OpenSBI configures the memory region as
> > "Memory, Non-cacheable, Bufferable" and passes this region as a global
> > shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> > allocations happen from this region and synchronization callbacks are
> > implemented to synchronize when doing DMA transactions.
> >
> > RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> > errata should be applied to handle cache management.
> >
> > Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> > Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > ---
> >  platform/generic/include/andes/andes45.h | 25 +++++++++++++--
> >  platform/generic/renesas/rzfive/rzfive.c | 41 ++++++++++++++++++++++++
> >  2 files changed, 64 insertions(+), 2 deletions(-)
> >
> > diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> > index 08b3d18..d5d265f 100644
> > --- a/platform/generic/include/andes/andes45.h
> > +++ b/platform/generic/include/andes/andes45.h
> > @@ -4,7 +4,28 @@
> >  #define CSR_MARCHID_MICROID 0xfff
> >
> >  /* Memory and Miscellaneous Registers */
> > -#define CSR_MCACHE_CTL 0x7ca
> > -#define CSR_MCCTLCOMMAND 0x7cc
> > +#define CSR_MCACHE_CTL         0x7ca
> > +#define CSR_MCCTLCOMMAND       0x7cc
> > +#define CSR_MICM_CFG           0xfc0
> > +#define CSR_MDCM_CFG           0xfc1
> > +#define CSR_MMSC_CFG           0xfc2
> > +
> > +#define MISA_20_OFFSET                 20
> > +#define MISA_20_MASK                   (0x1 << MISA_20_OFFSET)
> > +
> > +#define MICM_CFG_ISZ_OFFSET            6
> > +#define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
> > +
> > +#define MDCM_CFG_DSZ_OFFSET            6
> > +#define MDCM_CFG_DSZ_MASK              (0x7  << MDCM_CFG_DSZ_OFFSET)
> > +
> > +#define MMSC_CFG_CCTLCSR_OFFSET                16
> > +#define MMSC_CFG_CCTLCSR_MASK          (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> > +#define MMSC_IOCP_OFFSET                       47
> > +#define MMSC_IOCP_MASK                 (0x1ULL << MMSC_IOCP_OFFSET)
> > +
> > +#define MCACHE_CTL_CCTL_SUEN_OFFSET    8
> > +#define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> > +
> >
> >  #endif /* _RISCV_ANDES45_H */
> > diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> > index 4d71d0d..9530d28 100644
> > --- a/platform/generic/renesas/rzfive/rzfive.c
> > +++ b/platform/generic/renesas/rzfive/rzfive.c
> > @@ -4,11 +4,14 @@
> >   *
> >   */
> >
> > +#include <andes/andes45.h>
> >  #include <andes/andes45_pma.h>
> >  #include <platform_override.h>
> >  #include <sbi/sbi_domain.h>
> >  #include <sbi_utils/fdt/fdt_helper.h>
> >
> > +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND      0
> 
> The SBI vendor extension_id will be the same for all SoCs based on Andes
> CPUs so the function_id space will be shared across all such SoCs.
> 
> How does Andes plan to manage the function_id space of SBI vendor extension ?
> 
> I suggest the following:
> 1) The function_id == 0 can be used to return an SoC identifier. For example,
>      Renesas RzFive can have an SoC identifier "0".
> 2) Move RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to a
>     separate header at platform/generic/include/andes/andes_sbi.h and
>     change it's value to "1"
> 
> The above suggestion will help multiple SoC with Andes CPU share same
> SBI function_id space for SBI vendor extension.
> 
> Regards,
> Anup

Hi Anup,

Does this mean that all the SoC using Andes CPUs must have a unique
fid for each vendor SBI call function? If so, how do we make the new
vendors aware of this rule? As this won't be defined in the SBI spec.

The fid#0 - fid#24 have been used in our production code, as long as
the renesas_rzfive_vendor_ext_provider hook only gets called when the
root node compatible string is "renesas,r9a07g043f01", the functions
will be invoked correctly even though the fid conflicts with other
Andes SoCs?

Best regards,
Peter Lin


From conor at kernel.org  Tue Feb 21 02:06:31 2023
From: conor at kernel.org (Conor Dooley)
Date: Tue, 21 Feb 2023 10:06:31 +0000
Subject: =?US-ASCII?Q?Re=3A_=5BPATCH_v5_3/3=5D_platfo?= =?US-ASCII?Q?rm=3A_generic=3A_renesas=3A_rzf?= =?US-ASCII?Q?ive=3A_Add_SBI_EXT_to_check_for_enabling_IOCP_errata?=
In-Reply-To: <CAAhSdy046QNs-ae+mKs1J0E0bPv_Ew0JP4sNQa2RVzPmW6u8EA@mail.gmail.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20230213215111.32017-4-prabhakar.mahadev-lad.rj@bp.renesas.com> <CAAhSdy046QNs-ae+mKs1J0E0bPv_Ew0JP4sNQa2RVzPmW6u8EA@mail.gmail.com>
Message-ID: <4FB80591-F4AE-4735-82CA-4B69B129DE2B@kernel.org>



On 20 February 2023 15:31:32 GMT, Anup Patel <anup at brainfault.org> wrote:
>On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
><prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>>
>> I/O Coherence Port (IOCP) provides an AXI interface for connecting
>> external non-caching masters, such as DMA controllers. The accesses
>> from IOCP are coherent with D-Caches and L2 Cache.
>>
>> IOCP is a specification option and is disabled on the Renesas RZ/Five
>> SoC due to this reason IP blocks using DMA will fail.
>>
>> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
>> software. Firstly OpenSBI configures the memory region as
>> "Memory, Non-cacheable, Bufferable" and passes this region as a global
>> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
>> allocations happen from this region and synchronization callbacks are
>> implemented to synchronize when doing DMA transactions.
>>
>> RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
>> errata should be applied to handle cache management.
>>
>> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
>> Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
>> ---
>>  platform/generic/include/andes/andes45.h | 25 +++++++++++++--
>>  platform/generic/renesas/rzfive/rzfive.c | 41 ++++++++++++++++++++++++
>>  2 files changed, 64 insertions(+), 2 deletions(-)
>>
>> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
>> index 08b3d18..d5d265f 100644
>> --- a/platform/generic/include/andes/andes45.h
>> +++ b/platform/generic/include/andes/andes45.h
>> @@ -4,7 +4,28 @@
>>  #define CSR_MARCHID_MICROID 0xfff
>>
>>  /* Memory and Miscellaneous Registers */
>> -#define CSR_MCACHE_CTL 0x7ca
>> -#define CSR_MCCTLCOMMAND 0x7cc
>> +#define CSR_MCACHE_CTL         0x7ca
>> +#define CSR_MCCTLCOMMAND       0x7cc
>> +#define CSR_MICM_CFG           0xfc0
>> +#define CSR_MDCM_CFG           0xfc1
>> +#define CSR_MMSC_CFG           0xfc2
>> +
>> +#define MISA_20_OFFSET                 20
>> +#define MISA_20_MASK                   (0x1 << MISA_20_OFFSET)
>> +
>> +#define MICM_CFG_ISZ_OFFSET            6
>> +#define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
>> +
>> +#define MDCM_CFG_DSZ_OFFSET            6
>> +#define MDCM_CFG_DSZ_MASK              (0x7  << MDCM_CFG_DSZ_OFFSET)
>> +
>> +#define MMSC_CFG_CCTLCSR_OFFSET                16
>> +#define MMSC_CFG_CCTLCSR_MASK          (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
>> +#define MMSC_IOCP_OFFSET                       47
>> +#define MMSC_IOCP_MASK                 (0x1ULL << MMSC_IOCP_OFFSET)
>> +
>> +#define MCACHE_CTL_CCTL_SUEN_OFFSET    8
>> +#define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
>> +
>>
>>  #endif /* _RISCV_ANDES45_H */
>> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
>> index 4d71d0d..9530d28 100644
>> --- a/platform/generic/renesas/rzfive/rzfive.c
>> +++ b/platform/generic/renesas/rzfive/rzfive.c
>> @@ -4,11 +4,14 @@
>>   *
>>   */
>>
>> +#include <andes/andes45.h>
>>  #include <andes/andes45_pma.h>
>>  #include <platform_override.h>
>>  #include <sbi/sbi_domain.h>
>>  #include <sbi_utils/fdt/fdt_helper.h>
>>
>> +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND      0
>
>The SBI vendor extension_id will be the same for all SoCs based on Andes
>CPUs so the function_id space will be shared across all such SoCs.

Apologies for sticking my nose in here, but I'm a bit confused as to why is being added as an rz/five specific thing in the first place?
My uninformed view of the world sees other purchasers of the IP needing to apply this workaround too, no?

>How does Andes plan to manage the function_id space of SBI vendor extension ?
>
>I suggest the following:
>1) The function_id == 0 can be used to return an SoC identifier. For example,
>     Renesas RzFive can have an SoC identifier "0".
>2) Move RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to a
>    separate header at platform/generic/include/andes/andes_sbi.h and
>    change it's value to "1"
>
>The above suggestion will help multiple SoC with Andes CPU share same
>SBI function_id space for SBI vendor extension.

Sharing here makes complete sense to me, so that you don't end up duplicating this the next time someone buys the IP without the iocp.

Cheers,
Conor.

>
>
>> +
>>  static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
>>         {
>>                 .pa = 0x58000000,
>> @@ -28,6 +31,43 @@ static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *mat
>>                                          array_size(renesas_rzfive_pma_regions));
>>  }
>>
>> +static bool renesas_rzfive_cpu_cache_controlable(void)
>> +{
>> +       return (((csr_read(CSR_MICM_CFG) & MICM_CFG_ISZ_MASK) ||
>> +                (csr_read(CSR_MDCM_CFG) & MDCM_CFG_DSZ_MASK)) &&
>> +               (csr_read(CSR_MISA) & MISA_20_MASK) &&
>> +               (csr_read(CSR_MMSC_CFG) & MMSC_CFG_CCTLCSR_MASK) &&
>> +               (csr_read(CSR_MCACHE_CTL) & MCACHE_CTL_CCTL_SUEN_MASK));
>> +}
>> +
>> +static bool renesas_rzfive_cpu_iocp_disabled(void)
>> +{
>> +       return (csr_read(CSR_MMSC_CFG) & MMSC_IOCP_MASK) ? false : true;
>> +}
>> +
>> +static bool renesas_rzfive_apply_iocp_sw_workaround(void)
>> +{
>> +       return renesas_rzfive_cpu_cache_controlable() & renesas_rzfive_cpu_iocp_disabled();
>> +}
>> +
>> +static int renesas_rzfive_vendor_ext_provider(long funcid,
>> +                                             const struct sbi_trap_regs *regs,
>> +                                             unsigned long *out_value,
>> +                                             struct sbi_trap_info *out_trap,
>> +                                             const struct fdt_match *match)
>> +{
>> +       switch (funcid) {
>> +       case RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND:
>> +               *out_value = renesas_rzfive_apply_iocp_sw_workaround();
>> +               break;
>> +
>> +       default:
>> +               break;
>> +       }
>> +
>> +       return 0;
>> +}
>> +
>>  int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>>  {
>>         /*
>> @@ -55,4 +95,5 @@ const struct platform_override renesas_rzfive = {
>>         .match_table = renesas_rzfive_match,
>>         .early_init = renesas_rzfive_early_init,
>>         .final_init = renesas_rzfive_final_init,
>> +       .vendor_ext_provider = renesas_rzfive_vendor_ext_provider,
>>  };
>> --
>> 2.17.1
>>
>


From peterlin at andestech.com  Tue Feb 21 02:47:56 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Tue, 21 Feb 2023 18:47:56 +0800
Subject: [PATCH] lib: sbi: sbi_ecall: Improve return SBI error
Message-ID: <20230221104756.28573-1-peterlin@andestech.com>

We should also check if the return error code is greater than 0
(SBI_SUCCESS), as this is an invalid error.

Also rename the variable returned via a0 to 'out_err' and use
SBI_ERR_NOT_SUPPORTED when the extension does not have a handler
for consistency with the standard SBI errors defined in Table 1
of the SBI specification.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
---
 lib/sbi/sbi_ecall.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
index 27ce5d49..8accf675 100644
--- a/lib/sbi/sbi_ecall.c
+++ b/lib/sbi/sbi_ecall.c
@@ -97,34 +97,34 @@ void sbi_ecall_unregister_extension(struct sbi_ecall_extension *ext)
 
 int sbi_ecall_handler(struct sbi_trap_regs *regs)
 {
-	int ret = 0;
+	int out_err = 0;
+	unsigned long out_val = 0;
 	struct sbi_ecall_extension *ext;
 	unsigned long extension_id = regs->a7;
 	unsigned long func_id = regs->a6;
 	struct sbi_trap_info trap = {0};
-	unsigned long out_val = 0;
 	bool is_0_1_spec = 0;
 
 	ext = sbi_ecall_find_extension(extension_id);
 	if (ext && ext->handle) {
-		ret = ext->handle(extension_id, func_id,
+		out_err = ext->handle(extension_id, func_id,
 				  regs, &out_val, &trap);
 		if (extension_id >= SBI_EXT_0_1_SET_TIMER &&
 		    extension_id <= SBI_EXT_0_1_SHUTDOWN)
 			is_0_1_spec = 1;
 	} else {
-		ret = SBI_ENOTSUPP;
+		out_err = SBI_ERR_NOT_SUPPORTED;
 	}
 
-	if (ret == SBI_ETRAP) {
+	if (out_err == SBI_ETRAP) {
 		trap.epc = regs->mepc;
 		sbi_trap_redirect(regs, &trap);
 	} else {
-		if (ret < SBI_LAST_ERR) {
+		if (out_err < SBI_LAST_ERR || SBI_SUCCESS < out_err) {
 			sbi_printf("%s: Invalid error %d for ext=0x%lx "
-				   "func=0x%lx\n", __func__, ret,
+				   "func=0x%lx\n", __func__, out_err,
 				   extension_id, func_id);
-			ret = SBI_ERR_FAILED;
+			out_err = SBI_ERR_FAILED;
 		}
 
 		/*
@@ -136,7 +136,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
 		 * case should be handled differently.
 		 */
 		regs->mepc += 4;
-		regs->a0 = ret;
+		regs->a0 = out_err;
 		if (!is_0_1_spec)
 			regs->a1 = out_val;
 	}
-- 
2.34.1



From wxjstz at 126.com  Tue Feb 21 03:34:46 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 21 Feb 2023 19:34:46 +0800
Subject: [PATCH] lib: sbi: sbi_ecall: Improve return SBI error
In-Reply-To: <20230221104756.28573-1-peterlin@andestech.com>
References: <20230221104756.28573-1-peterlin@andestech.com>
Message-ID: <2c3c52436f6dadf09fe733a257a46943ff57728a.camel@126.com>

? 2023-02-21???? 18:47 +0800?Yu Chien Peter Lin???
> We should also check if the return error code is greater than 0
> (SBI_SUCCESS), as this is an invalid error.
> 
> Also rename the variable returned via a0 to 'out_err' and use
> SBI_ERR_NOT_SUPPORTED when the extension does not have a handler
> for consistency with the standard SBI errors defined in Table 1
> of the SBI specification.
> 
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
> ?lib/sbi/sbi_ecall.c | 18 +++++++++---------
> ?1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> index 27ce5d49..8accf675 100644
> --- a/lib/sbi/sbi_ecall.c
> +++ b/lib/sbi/sbi_ecall.c
> @@ -97,34 +97,34 @@ void sbi_ecall_unregister_extension(struct sbi_ecall_extension *ext)
> ?
> ?int sbi_ecall_handler(struct sbi_trap_regs *regs)
> ?{
> -???????int ret = 0;
> +???????int out_err = 0;
There is no need to rename ret to out_err because a0 is not always an
error code in legacy extensions

Other than the above, it looks good to me.

Reviewed-by: Xiang W <wxjstz at 126.com>
> +???????unsigned long out_val = 0;
> ????????struct sbi_ecall_extension *ext;
> ????????unsigned long extension_id = regs->a7;
> ????????unsigned long func_id = regs->a6;
> ????????struct sbi_trap_info trap = {0};
> -???????unsigned long out_val = 0;
> ????????bool is_0_1_spec = 0;
> ?
> ????????ext = sbi_ecall_find_extension(extension_id);
> ????????if (ext && ext->handle) {
> -???????????????ret = ext->handle(extension_id, func_id,
> +???????????????out_err = ext->handle(extension_id, func_id,
> ????????????????????????????????? regs, &out_val, &trap);
> ????????????????if (extension_id >= SBI_EXT_0_1_SET_TIMER &&
> ??????????????????? extension_id <= SBI_EXT_0_1_SHUTDOWN)
> ????????????????????????is_0_1_spec = 1;
> ????????} else {
> -???????????????ret = SBI_ENOTSUPP;
> +???????????????out_err = SBI_ERR_NOT_SUPPORTED;
> ????????}
> ?
> -???????if (ret == SBI_ETRAP) {
> +???????if (out_err == SBI_ETRAP) {
> ????????????????trap.epc = regs->mepc;
> ????????????????sbi_trap_redirect(regs, &trap);
> ????????} else {
> -???????????????if (ret < SBI_LAST_ERR) {
> +???????????????if (out_err < SBI_LAST_ERR || SBI_SUCCESS < out_err) {
> ????????????????????????sbi_printf("%s: Invalid error %d for ext=0x%lx "
> -????????????????????????????????? "func=0x%lx\n", __func__, ret,
> +????????????????????????????????? "func=0x%lx\n", __func__, out_err,
> ?????????????????????????????????? extension_id, func_id);
> -???????????????????????ret = SBI_ERR_FAILED;
> +???????????????????????out_err = SBI_ERR_FAILED;
> ????????????????}
> ?
> ????????????????/*
> @@ -136,7 +136,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
> ???????????????? * case should be handled differently.
> ???????????????? */
> ????????????????regs->mepc += 4;
> -???????????????regs->a0 = ret;
> +???????????????regs->a0 = out_err;
> ????????????????if (!is_0_1_spec)
> ????????????????????????regs->a1 = out_val;
> ????????}
> -- 
> 2.34.1
> 
> 


From ajones at ventanamicro.com  Tue Feb 21 09:21:24 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Tue, 21 Feb 2023 18:21:24 +0100
Subject: [PATCH] lib: sbi: sbi_ecall: Improve return SBI error
In-Reply-To: <20230221104756.28573-1-peterlin@andestech.com>
References: <20230221104756.28573-1-peterlin@andestech.com>
Message-ID: <20230221172124.34ccwi3cg2c2ilff@orel>

On Tue, Feb 21, 2023 at 06:47:56PM +0800, Yu Chien Peter Lin wrote:
> We should also check if the return error code is greater than 0
> (SBI_SUCCESS), as this is an invalid error.
> 
> Also rename the variable returned via a0 to 'out_err' and use
> SBI_ERR_NOT_SUPPORTED when the extension does not have a handler
> for consistency with the standard SBI errors defined in Table 1
> of the SBI specification.

It's best to do renames separate from functional changes, and, in
this case, the ret -> out_err rename feels like unnecessary churn.

Thanks,
drew

> 
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> ---
>  lib/sbi/sbi_ecall.c | 18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> index 27ce5d49..8accf675 100644
> --- a/lib/sbi/sbi_ecall.c
> +++ b/lib/sbi/sbi_ecall.c
> @@ -97,34 +97,34 @@ void sbi_ecall_unregister_extension(struct sbi_ecall_extension *ext)
>  
>  int sbi_ecall_handler(struct sbi_trap_regs *regs)
>  {
> -	int ret = 0;
> +	int out_err = 0;
> +	unsigned long out_val = 0;
>  	struct sbi_ecall_extension *ext;
>  	unsigned long extension_id = regs->a7;
>  	unsigned long func_id = regs->a6;
>  	struct sbi_trap_info trap = {0};
> -	unsigned long out_val = 0;
>  	bool is_0_1_spec = 0;
>  
>  	ext = sbi_ecall_find_extension(extension_id);
>  	if (ext && ext->handle) {
> -		ret = ext->handle(extension_id, func_id,
> +		out_err = ext->handle(extension_id, func_id,
>  				  regs, &out_val, &trap);
>  		if (extension_id >= SBI_EXT_0_1_SET_TIMER &&
>  		    extension_id <= SBI_EXT_0_1_SHUTDOWN)
>  			is_0_1_spec = 1;
>  	} else {
> -		ret = SBI_ENOTSUPP;
> +		out_err = SBI_ERR_NOT_SUPPORTED;
>  	}
>  
> -	if (ret == SBI_ETRAP) {
> +	if (out_err == SBI_ETRAP) {
>  		trap.epc = regs->mepc;
>  		sbi_trap_redirect(regs, &trap);
>  	} else {
> -		if (ret < SBI_LAST_ERR) {
> +		if (out_err < SBI_LAST_ERR || SBI_SUCCESS < out_err) {
>  			sbi_printf("%s: Invalid error %d for ext=0x%lx "
> -				   "func=0x%lx\n", __func__, ret,
> +				   "func=0x%lx\n", __func__, out_err,
>  				   extension_id, func_id);
> -			ret = SBI_ERR_FAILED;
> +			out_err = SBI_ERR_FAILED;
>  		}
>  
>  		/*
> @@ -136,7 +136,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
>  		 * case should be handled differently.
>  		 */
>  		regs->mepc += 4;
> -		regs->a0 = ret;
> +		regs->a0 = out_err;
>  		if (!is_0_1_spec)
>  			regs->a1 = out_val;
>  	}
> -- 
> 2.34.1
> 
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Feb 21 09:23:31 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Feb 2023 22:53:31 +0530
Subject: [PATCH] lib: sbi: sbi_ecall: Improve return SBI error
In-Reply-To: <20230221172124.34ccwi3cg2c2ilff@orel>
References: <20230221104756.28573-1-peterlin@andestech.com> <20230221172124.34ccwi3cg2c2ilff@orel>
Message-ID: <CAAhSdy1TEiRkw796rg5yNqLuOog33s4xUJhSdcxGhrG07kxgsg@mail.gmail.com>

On Tue, Feb 21, 2023 at 10:51 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> On Tue, Feb 21, 2023 at 06:47:56PM +0800, Yu Chien Peter Lin wrote:
> > We should also check if the return error code is greater than 0
> > (SBI_SUCCESS), as this is an invalid error.
> >
> > Also rename the variable returned via a0 to 'out_err' and use
> > SBI_ERR_NOT_SUPPORTED when the extension does not have a handler
> > for consistency with the standard SBI errors defined in Table 1
> > of the SBI specification.
>
> It's best to do renames separate from functional changes, and, in
> this case, the ret -> out_err rename feels like unnecessary churn.

Maybe we can do something similar to KVM RISC-V ?

Regards,
Anup

>
> Thanks,
> drew
>
> >
> > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > ---
> >  lib/sbi/sbi_ecall.c | 18 +++++++++---------
> >  1 file changed, 9 insertions(+), 9 deletions(-)
> >
> > diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> > index 27ce5d49..8accf675 100644
> > --- a/lib/sbi/sbi_ecall.c
> > +++ b/lib/sbi/sbi_ecall.c
> > @@ -97,34 +97,34 @@ void sbi_ecall_unregister_extension(struct sbi_ecall_extension *ext)
> >
> >  int sbi_ecall_handler(struct sbi_trap_regs *regs)
> >  {
> > -     int ret = 0;
> > +     int out_err = 0;
> > +     unsigned long out_val = 0;
> >       struct sbi_ecall_extension *ext;
> >       unsigned long extension_id = regs->a7;
> >       unsigned long func_id = regs->a6;
> >       struct sbi_trap_info trap = {0};
> > -     unsigned long out_val = 0;
> >       bool is_0_1_spec = 0;
> >
> >       ext = sbi_ecall_find_extension(extension_id);
> >       if (ext && ext->handle) {
> > -             ret = ext->handle(extension_id, func_id,
> > +             out_err = ext->handle(extension_id, func_id,
> >                                 regs, &out_val, &trap);
> >               if (extension_id >= SBI_EXT_0_1_SET_TIMER &&
> >                   extension_id <= SBI_EXT_0_1_SHUTDOWN)
> >                       is_0_1_spec = 1;
> >       } else {
> > -             ret = SBI_ENOTSUPP;
> > +             out_err = SBI_ERR_NOT_SUPPORTED;
> >       }
> >
> > -     if (ret == SBI_ETRAP) {
> > +     if (out_err == SBI_ETRAP) {
> >               trap.epc = regs->mepc;
> >               sbi_trap_redirect(regs, &trap);
> >       } else {
> > -             if (ret < SBI_LAST_ERR) {
> > +             if (out_err < SBI_LAST_ERR || SBI_SUCCESS < out_err) {
> >                       sbi_printf("%s: Invalid error %d for ext=0x%lx "
> > -                                "func=0x%lx\n", __func__, ret,
> > +                                "func=0x%lx\n", __func__, out_err,
> >                                  extension_id, func_id);
> > -                     ret = SBI_ERR_FAILED;
> > +                     out_err = SBI_ERR_FAILED;
> >               }
> >
> >               /*
> > @@ -136,7 +136,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
> >                * case should be handled differently.
> >                */
> >               regs->mepc += 4;
> > -             regs->a0 = ret;
> > +             regs->a0 = out_err;
> >               if (!is_0_1_spec)
> >                       regs->a1 = out_val;
> >       }
> > --
> > 2.34.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From ajones at ventanamicro.com  Tue Feb 21 10:07:24 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Tue, 21 Feb 2023 19:07:24 +0100
Subject: [PATCH] lib: sbi: sbi_ecall: Improve return SBI error
In-Reply-To: <CAAhSdy1TEiRkw796rg5yNqLuOog33s4xUJhSdcxGhrG07kxgsg@mail.gmail.com>
References: <20230221104756.28573-1-peterlin@andestech.com>
 <20230221172124.34ccwi3cg2c2ilff@orel>
 <CAAhSdy1TEiRkw796rg5yNqLuOog33s4xUJhSdcxGhrG07kxgsg@mail.gmail.com>
Message-ID: <20230221180724.vtmg4dgc4msb7io6@orel>

On Tue, Feb 21, 2023 at 10:53:31PM +0530, Anup Patel wrote:
> On Tue, Feb 21, 2023 at 10:51 PM Andrew Jones <ajones at ventanamicro.com> wrote:
> >
> > On Tue, Feb 21, 2023 at 06:47:56PM +0800, Yu Chien Peter Lin wrote:
> > > We should also check if the return error code is greater than 0
> > > (SBI_SUCCESS), as this is an invalid error.
> > >
> > > Also rename the variable returned via a0 to 'out_err' and use
> > > SBI_ERR_NOT_SUPPORTED when the extension does not have a handler
> > > for consistency with the standard SBI errors defined in Table 1
> > > of the SBI specification.
> >
> > It's best to do renames separate from functional changes, and, in
> > this case, the ret -> out_err rename feels like unnecessary churn.
> 
> Maybe we can do something similar to KVM RISC-V ?

We could if we have three types of return values to manage like we do
with KVM. For anyone who isn't familiar with how KVM will manage things,
once the PMU patches are merged, it'll work like this:

 1. Linux error while emulating an SBI call (e.g. ENOMEM)
 2. 'sbiret.error' generated during emulation
 3. 'sbiret.value' generated during emulation

(1) is the return value of an SBI emulation function,

    int sbi_func(struct kvm_vcpu_sbi_return *retdata)

(2) is kvm_vcpu_sbi_return.out_val
(3) is kvm_vcpu_sbi_return.err_val


But don't we only have (2) and (3) here?

Thanks,
drew

> 
> Regards,
> Anup
> 
> >
> > Thanks,
> > drew
> >
> > >
> > > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > > ---
> > >  lib/sbi/sbi_ecall.c | 18 +++++++++---------
> > >  1 file changed, 9 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> > > index 27ce5d49..8accf675 100644
> > > --- a/lib/sbi/sbi_ecall.c
> > > +++ b/lib/sbi/sbi_ecall.c
> > > @@ -97,34 +97,34 @@ void sbi_ecall_unregister_extension(struct sbi_ecall_extension *ext)
> > >
> > >  int sbi_ecall_handler(struct sbi_trap_regs *regs)
> > >  {
> > > -     int ret = 0;
> > > +     int out_err = 0;
> > > +     unsigned long out_val = 0;
> > >       struct sbi_ecall_extension *ext;
> > >       unsigned long extension_id = regs->a7;
> > >       unsigned long func_id = regs->a6;
> > >       struct sbi_trap_info trap = {0};
> > > -     unsigned long out_val = 0;
> > >       bool is_0_1_spec = 0;
> > >
> > >       ext = sbi_ecall_find_extension(extension_id);
> > >       if (ext && ext->handle) {
> > > -             ret = ext->handle(extension_id, func_id,
> > > +             out_err = ext->handle(extension_id, func_id,
> > >                                 regs, &out_val, &trap);
> > >               if (extension_id >= SBI_EXT_0_1_SET_TIMER &&
> > >                   extension_id <= SBI_EXT_0_1_SHUTDOWN)
> > >                       is_0_1_spec = 1;
> > >       } else {
> > > -             ret = SBI_ENOTSUPP;
> > > +             out_err = SBI_ERR_NOT_SUPPORTED;
> > >       }
> > >
> > > -     if (ret == SBI_ETRAP) {
> > > +     if (out_err == SBI_ETRAP) {
> > >               trap.epc = regs->mepc;
> > >               sbi_trap_redirect(regs, &trap);
> > >       } else {
> > > -             if (ret < SBI_LAST_ERR) {
> > > +             if (out_err < SBI_LAST_ERR || SBI_SUCCESS < out_err) {
> > >                       sbi_printf("%s: Invalid error %d for ext=0x%lx "
> > > -                                "func=0x%lx\n", __func__, ret,
> > > +                                "func=0x%lx\n", __func__, out_err,
> > >                                  extension_id, func_id);
> > > -                     ret = SBI_ERR_FAILED;
> > > +                     out_err = SBI_ERR_FAILED;
> > >               }
> > >
> > >               /*
> > > @@ -136,7 +136,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
> > >                * case should be handled differently.
> > >                */
> > >               regs->mepc += 4;
> > > -             regs->a0 = ret;
> > > +             regs->a0 = out_err;
> > >               if (!is_0_1_spec)
> > >                       regs->a1 = out_val;
> > >       }
> > > --
> > > 2.34.1
> > >
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From atishp at atishpatra.org  Tue Feb 21 12:22:53 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Tue, 21 Feb 2023 12:22:53 -0800
Subject: [PATCH 1/1] Add RISC-V TEE support
In-Reply-To: <007e01d941fb$7ef6f1d0$7ce4d570$@eswincomputing.com>
References: <007e01d941fb$7ef6f1d0$7ce4d570$@eswincomputing.com>
Message-ID: <CAOnJCULgCiQY4+VvmL1zsxdg6utcKt_piE33S7FdXYorVWkkrg@mail.gmail.com>

On Thu, Feb 16, 2023 at 3:40 AM liushiwei <liushiwei at eswincomputing.com> wrote:
>
> Hi? Atish
>         I tried to email tech-prs at lists.riscv.org and sig-trusted-computing at lists.riscv.org, but both returned. My attempts to send a patch directly also returned.
>         Looking at this reason, does it seem like I have to join some group before I can send an email?
>
> ?Connection to the remote recipient's server was denied for unknown reason.
> SMTP through SDN 37, SMTP: (Proxy)Host lists.riscv.org said 510 5.1.1 Your email address, liushiwei at eswincomputing.com,
> is not subscribed to that group.  To subscribe, send an email to sig-trusted-computing+subscribe at lists.riscv.org,
> or visit https://lists.riscv.org/g/sig-trusted-computing?
>

Yes. You have to subscribe to each group. Either you can join as an
individual member or your organization if that's already a RVI member.

> Regards,
> Liushiwei
> -----????-----
> ???: Atish Patra [mailto:atishp at atishpatra.org]
> ????: 2023?2?8? 6:37
> ???: liushiwei <liushiwei at eswincomputing.com>
> ??: Himanshu Chauhan <hchauhan at ventanamicro.com>; opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> ??: Re: Re: [PATCH 1/1] Add RISC-V TEE support
>
> On Sat, Jan 28, 2023 at 12:33 AM liushiwei <liushiwei at eswincomputing.com> wrote:
> >
> > Hi, Atish
> >         Thank you for your reply.
> >         I submitted my ideas to https://github.com/riscv-non-isa/riscv-sbi-doc/pull/106 as requested by other members of the email.
> >         You can preview them at https://github.com/liushiwei007/riscv-sbi-doc/blob/master/riscv-sbi.adoc .
> >         I'm not sure if I need to send an email to tech-prs at lists.riscv.org or sig-trusted-computing at lists.riscv.org after submitting to riscv-sbi-doc?
> >         If so, Is it to send the patch of opensbi I submitted before?
> >
>
> The Trusted computing SIG evaluates the overall merit and direction of any TEE related specifications.
> The PRS TG will focus more on the SBI specification related parts.
> I would recommend you to present your work to Trusted Computing SIG first so that everybody agrees on the overall direction for OP-TEE support in RISC-V.
>
> > Regards,
> > Liushiwei
> >
> >
> > -----????-----
> > ???: Atish Patra [mailto:atishp at atishpatra.org]
> > ????: 2023?1?25? 3:12
> > ???: liushiwei <liushiwei at eswincomputing.com>
> > ??: Himanshu Chauhan <hchauhan at ventanamicro.com>;
> > opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> > ??: Re: ??: [PATCH 1/1] Add RISC-V TEE support
> >
> > On Wed, Jan 11, 2023 at 11:08 PM liushiwei <liushiwei at eswincomputing.com> wrote:
> > >
> > > Hi, Himanshu ,  These are my description:
> > >
> > > In my design, The entire linux space is called REE (Rich Execution
> > > Environment), and TEE OS includes its user state called TEE (Trusted execution environment).
> > > adding TEE functionality to opensbi requires two configuration items.
> > > For example:
> > >    add CONFIG_SBI_ECALL_TEE=y in platform/generic/configs/defconfig file.
> > >    add CONFIG_TEE_LOAD_ADDR=0x27C000000 platform/generic/objects.mk.
> > >    The value of CONFIG_TEE_LOAD_ADDR depends on the actual memory layout,
> > >    It's a physical address.
> > >
> > > When TEE is configured, opensbi adds the following functionality:
> > > 1. Boot TEE OS.
> > >    If TEE is enabled, tee_os_init() is called before entering
> > >    sbi_hart_switch_mode(). tee_os_init saves the current context,
> > >    sets a new trap stack address, and runs to CONFIG_TEE_LOAD_ADDR as configured
> > >    to complete TEE OS initialization. TEE OS returns via ecall, Go to opensbi
> > >    sbi_ecall_tee_handler and use the characteristic value RETURN_ENTRY_DONE
> > >    to indicate the return after the TEE OS completes booting. Check whether the
> > >    TEE OS boot successfully according to the parameter. If fails,
> > >    the system enters wfi and terminates the startup process of opensbi. If successful,
> > >    it returns REE to switch into TEE's vector. then trap stack memory is restored,
> > >    the context is restored, the tee_os_init call is returned,
> > >    and the rest of the process is performed.
> > > 2. REE switches to TEE.
> > >    When the TEE OS boot successfully, it returns an entry vector for REE into the TEE.
> > >    It represents various entry points into TEE OS and is stored in opensbi's global variables.
> > >    It is the optee_vectors_t structure, which contains nine entry cases:
> > >       1. yield_smc_entry;
> > >       2. fast_smc_entry;
> > >       3. cpu_on_entry;
> > >       4. cpu_off_entry;
> > >       5. cpu_resume_entry;
> > >       6. cpu_suspend_entry;
> > >       7. fiq_entry;
> > >       8. system_off_entry;
> > >       9. system_reset_entry;
> > >    yield_smc_entry means that this function entry TEE will start the thread function
> > >    and enter the user state of TEE. It may also switch back to REE with RPC function,
> > >    and then return to TEE after REE completes the corresponding function.
> > >    For the REE process that sent you this call, it may cause sleep.
> > >
> > >    fast_smc_entry indicates that this is a quick function that returns after
> > >    the TEE OS does something, and that it does not cause the caller to sleep.
> > >
> > >    yield_smc_entry and fast_smc_entry return opensbi use eigenvalue RETURN_CALL_DONE.
> > >    for yield_smc_entry, whether the call returns or the RPC returns is decided by linux.
> > >
> > >    cpu_on_entry/cpu_off_entry/cpu_resume_entry/cpu_suspend_entry and
> > >    fiq_entry/system_off_entry/system_reset_entry they are not implemented currently.
> > >
> > > 3. TEE switches to REE.
> > >    opensbi needs to save the context when REE enters the TEE,
> > >    and restore the context when it returns from the TEE.
> > >
> > > 4. TEE/REE Request a special function.
> > >    We have some specific functions, like get hartid from TEE,
> > >    it need save and restore the TEE context.
> > >
> > > Data structure.
> > > opensbi adds the sbi_save_context declaration
> > >    struct sbi_save_context {
> > >       struct sbi_trap_regs regs;
> > >       unsigned long sepc;
> > >       unsigned long satp;
> > >       unsigned long sstatus;
> > >       unsigned long sie;
> > >       unsigned long stvec;
> > >       unsigned long sscratch;
> > >       unsigned long scounteren;
> > >       unsigned long scause;
> > >       unsigned long stval;
> > >       unsigned long sip;
> > >    };
> > >    sbi_save_context include sbi_trap_regs and S mode csr. Used to
> > > hold the context of TEE or REE
> > >
> > >    typedef struct optee_vectors {
> > >       optee_vector_isn_t yield_smc_entry;
> > >       optee_vector_isn_t fast_smc_entry;
> > >       optee_vector_isn_t cpu_on_entry;
> > >       optee_vector_isn_t cpu_off_entry;
> > >       optee_vector_isn_t cpu_resume_entry;
> > >       optee_vector_isn_t cpu_suspend_entry;
> > >       optee_vector_isn_t fiq_entry;
> > >       optee_vector_isn_t system_off_entry;
> > >       optee_vector_isn_t system_reset_entry;
> > >    } optee_vectors_t;
> > >    optee_vectors represent the various entry points into TEE OS.
> > >
> > > Variables defined by opensbi
> > >    optee_vectors_t *optee_vector_table;
> > >    struct sbi_save_context nsec_cpu_context[OPTEED_CORE_COUNT];
> > >    struct sbi_save_context sec_cpu_context[OPTEED_CORE_COUNT];
> > >    struct sbi_trap_regs cpu_start_context[OPTEED_CORE_COUNT];
> > >    typedef ulong tee_tmp_trap_stack[1024];
> > >    static tee_tmp_trap_stack tmp_stack[OPTEED_CORE_COUNT];
> > >
> > >    optee_vector_table value is assigned after the TEE OS boot succeeds.
> > >    nsec_cpu_context and sec_cpu_context is to save the context of TEE and REE,
> > >    cpu_start_context save the context of opensbi before tee_os_init enter TEE OS,
> > >    tee_tmp_stack is the trap stack when TEE OS return tee_os_init.
> > >
> > > Under the current design, REE does not enable interrupts when
> > > entering TEE, and the entire TEE, including opensbi, is the process context of linux.
> > > TEE processing must be brief and quick.
> > >
> >
> > Great. Thanks. Your PR should include more details like this with proper FID allocation and description.
> > In order to make it a standard RISC-V SBI extension, you need to
> > present this in tech-prs mailing list
> > (https://lists.riscv.org/g/tech-prs)
> > and get it approved within the group. Here is the policy document[1]
> >
> > https://docs.google.com/document/d/1bQGHU-wD4uN4mU07oH9adpXbH51NqUbFxc
> > L8qTTIqhA/edit
> >
> > I think it would be good if you can present your work in trusted computing SIG (https://lists.riscv.org/g/sig-trusted-computing) as well.
> >
> > >
> > > Regards,
> > > liushiwei
> > > -----????-----
> > > ???: Himanshu Chauhan [mailto:hchauhan at ventanamicro.com]
> > > ????: 2023?1?11? 23:39
> > > ???: liushiwei <liushiwei at eswincomputing.com>
> > > ??: opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> > > ??: Re: ??: [PATCH 1/1] Add RISC-V TEE support
> > >
> > > On Wed, Jan 11, 2023 at 08:27:59PM +0800, liushiwei wrote:
> > > > Do you mean hardware? Our hardware design referred to arm's
> > > > trustzone technology. optee os is a software solution using arm
> > > > trustzone hardware, which mainly includes REE(linux), TEE(optee
> > > > os), ATF(ARM Trusted firmware), and then our software also
> > > > developed these three parts. opensbi is similar to ATF. whether if this is what you want?
> > > > The current committed code is not hardware-dependent, but just
> > > > continues the idea of this workaround, and we may commit hardware-dependent code later.
> > > >
> > > No, I meant the software specification.
> > >
> > > > -----????-----
> > > > ???: hchauhan at ventanamicro.com [mailto:hchauhan at ventanamicro.com]
> > > > ????: 2023?1?11? 20:03
> > > > ???: 'liushiwei' <liushiwei at eswincomputing.com>; opensbi at lists.infradead.
> > > > org
> > > > ??: chenchaokai at eswincomputing.com
> > > > ??: RE: [PATCH 1/1] Add RISC-V TEE support
> > > >
> > > > -----Original Message-----
> > > > > From: opensbi <opensbi-bounces at lists.infradead.org> On Behalf Of
> > > > > liushiwei
> > > > > Sent: 11 January 2023 07:32
> > > > > To: opensbi at lists.infradead.org
> > > > > Cc: chenchaokai at eswincomputing.com; liushiwei
> > > > <liushiwei at eswincomputing.com>
> > > > > Subject: [PATCH 1/1] Add RISC-V TEE support
> > > >
> > > > >RISC-V Trusted Executable Environment security software includes
> > > > >linux,
> > > > opensbi, and OP-TEE OS. linux is the non-secure domain, and OP-TEE
> > > > OS is the secure domain. At boot time, opensbi boots OP->TEE OS and then starts linux.
> > > > At runtime, opensbi acts as a secure monitor, responsible for
> > > > context saving and restoring when switching between linux and OP-TEE OS.
> > > > >TEE function is off by default, when using configuration is added
> > > > >in the
> > > > config and objects file, such as
> > > > platform/generic/configs/defconfig
> > > > add CONFIG_SBI_ECALL_TEE = y, In the >platform/generic/objects.mk
> > > > add CONFIG_TEE_LOAD_ADDR = 0x27c000000, CONFIG_TEE_LOAD_ADDR is
> > > > the starting address of the OP-TEE OS.
> > > >
> > > > Hi Liushiwei,
> > > >
> > > > Was there any formal specification or draft for this? Could you
> > > > please point me to the draft or specification?
> > > >
> > > > Regards
> > > > Himanshu
> > > >
> > > > --
> > > > opensbi mailing list
> > > > opensbi at lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/opensbi
> > > >
> > >
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
> >
> >
> >
> > --
> > Regards,
> > Atish
> >
>
>
> --
> Regards,
> Atish
>


-- 
Regards,
Atish


From e.shatokhin at yadro.com  Tue Feb 21 13:08:49 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Wed, 22 Feb 2023 00:08:49 +0300
Subject: [PATCH v2 0/2] lib: sbi: Prevent system hang when bringing up secondary harts
Message-ID: <20230221210851.3145840-1-e.shatokhin@yadro.com>

The boot hart can race with the secondary harts when it tries to bring them
up at startup time. This results in occasional system hangs at boot, which
can be reproduced in QEMU with a Linux guest, even with '-machine virt'.

When the Linux kernel needs to bring up the non-boot harts, the boot hart
enters opensbi via a ecall and eventually gets to sbi_hsm_hart_start().
Other (secondary) harts are spinning in the while loop in
sbi_hsm_hart_wait() at that moment:

	/* Wait for state transition requested by sbi_hsm_hart_start() */
	while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
		wfi();
	};

If I understand it correctly, the idea is that sbi_hsm_hart_start() should
first set the secondary hart's state to SBI_HSM_STATE_START_PENDING
atomically and check the previous value of the state. This makes sure no
other hart would try to modify the secondary hart's scratch area in the
meantime.

The boot hart then reads the init count of the secondary hart and sets the
address it should eventually jump to, as well as other data:

	init_count = sbi_init_count(hartid);
	rscratch->next_arg1 = arg1;
	rscratch->next_addr = saddr;
	rscratch->next_mode = smode;

Then, the boot hart sends IPI to the secondary hart, so that it would stop
spinning in the loop above and would continue.

However, as we observed during the test runs for our systems, the secondary
harts could leave the loop in sbi_hsm_hart_wait() as soon as their state is
set to SBI_HSM_STATE_START_PENDING. As a result, the following situation is
possible:


Boot hart (#0)                        Secondary hart (#1)
runs Linux startup code               waits in sbi_hsm_hart_wait()
sbi_ecall(SBI_EXT_HSM,
          SBI_EXT_HSM_HART_START,
          ...)
enters sbi_hsm_hart_start()
sets state to START_PENDING
                                      leaves sbi_hsm_hart_wait()
                                      runs to the end of init_warmboot()
                                      returns to scratch->next_addr
                                      (next_addr can be garbage here)

sets next_addr, etc. for hart #1
(no good: hart #1 has already left)

sends IPI to hart #1
(no good either)
                   (system might hang or crash)

This can be reproduced easier, if one inserts a delay into
sbi_hsm_hart_start() right before "init_count = sbi_init_count(hartid);".
In our case, "sbi_timer_mdelay(10)" was enough for the issue to manifest
each time.

The patch series fixes 2 issues that could make the secondary harts exit
the while loop in sbi_hsm_hart_wait() prematurely, i.e. before the IPI:

1. "wfi" instruction is just a hint for the CPU, it might or might not
actually wait for an interrupt to happen in the hart. RISC-V Privileged
Architectures spec. v20211203 even states that "a legal implementation is to
simply implement WFI as a NOP".

2. Since commit 50d4fde1c5a4 ("lib: Remove redundant sbi_platform_ipi_clear()
calls"), the IPI sent from the boot hart in wake_coldboot_harts() is never
cleared in the secondary harts. So, they enter sbi_hsm_hart_wait() with
already pending IPIs and do not wait for new ones.

In both cases, the secondary hart might leave the loop as soon as its state
is set to SBI_HSM_STATE_START_PENDING, leading to the race described above.

Changes in v2:

* Added a per-hart variable to properly synchronize the boot hart and the
  secondary hart. This should be safer than just waiting for any IPI in
  sbi_hsm_hart_wait(), because the hart might, in theory, get an IPI
  unrelated to sbi_hsm_hart_start().

* Updated the description of the second patch: clearing of the pending IPIs
  is no longer needed to prevent the race. Now, it rather allows avoiding
  a potentially inefficient busy-wait.

Regards,
Evgenii






From e.shatokhin at yadro.com  Tue Feb 21 13:08:50 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Wed, 22 Feb 2023 00:08:50 +0300
Subject: [PATCH v2 1/2] lib: sbi: Wait in sbi_hsm_hart_wait() till the hart is prepared to start
In-Reply-To: <20230221210851.3145840-1-e.shatokhin@yadro.com>
References: <20230221210851.3145840-1-e.shatokhin@yadro.com>
Message-ID: <20230221210851.3145840-2-e.shatokhin@yadro.com>

"wfi" instruction is not guaranteed to wait for an interrupt to be received
by the hart, it is just a hint for the CPU. According to RISC-V Privileged
Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
legal.

As a result, a secondary (non-boot) hart waiting in the while loop in
sbi_hsm_hart_wait() might leave the loop and continue even if it got no
IPI or it got an IPI unrelated to sbi_hsm_hart_start(). This could lead to
the following race condition when booting Linux, for example:

  Boot hart (#0)                        Secondary hart (#1)
  runs Linux startup code               waits in sbi_hsm_hart_wait()

  sbi_ecall(SBI_EXT_HSM,
            SBI_EXT_HSM_HART_START,
            ...)
  enters sbi_hsm_hart_start()
  sets state of hart #1 to START_PENDING
                                        leaves sbi_hsm_hart_wait()
                                        runs to the end of init_warmboot()
                                        returns to scratch->next_addr
                                        (next_addr can be garbage here)

  sets next_addr, etc. for hart #1
  (no good: hart #1 has already left)

  sends IPI to hart #1
  (no good either)

If this happens, the secondary hart jumps to a wrong next_addr at the end
of init_warmboot(), which leads to a system hang or crash.

To reproduce the issue more reliably, one could add a delay in
sbi_hsm_hart_start() after setting the hart's state but before sending
IPI to that hart:

    hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
                            SBI_HSM_STATE_START_PENDING);
    ...
  + sbi_timer_mdelay(10);
    init_count = sbi_init_count(hartid);
    rscratch->next_arg1 = arg1;
    rscratch->next_addr = saddr;

The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
and 2 or more CPUs, with Linux as the guest OS.

This patch adds an auxiliary per-hart state variable, aux_state, as well as
its accessor functions, to synchronize execution of the boot hart and the
starting secondary hart.

Initially, aux_state is 0: sbi_scratch_alloc_offset() zeroes the allocated
memory. The boot hart sets it to 1 after it has set next_addr, etc., for the
secondary hart, then sends the IPI to the latter.

The secondary hart waits in sbi_hsm_hart_wait() both for aux_state to become
non-zero and for an IPI to happen. This should be safe even if the secondary
hart somehow gets an IPI unrelated to sbi_hsm_hart_start().

Memory barriers are used to make sure the writes to next_addr, etc., in the
boot hart are visible to the secondary hart if it reads non-zero aux_state.

Only one bit of aux_state is actually used here, so, this per-hart variable
can be used to hold other data in the future, if needed.

Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
---
 lib/sbi/sbi_hsm.c | 35 +++++++++++++++++++++++++++++++++--
 1 file changed, 33 insertions(+), 2 deletions(-)

diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index 8499bb1..24061c5 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -35,6 +35,7 @@ struct sbi_hsm_data {
 	unsigned long suspend_type;
 	unsigned long saved_mie;
 	unsigned long saved_mip;
+	unsigned long aux_state;
 };
 
 static inline int __sbi_hsm_hart_get_state(u32 hartid)
@@ -93,6 +94,23 @@ int sbi_hsm_hart_interruptible_mask(const struct sbi_domain *dom,
 	return 0;
 }
 
+static inline bool sbi_hsm_is_runnable(struct sbi_hsm_data *hdata)
+{
+	unsigned long aux_state = __smp_load_acquire(&hdata->aux_state);
+
+	return !!aux_state;
+}
+
+static inline void sbi_hsm_set_runnable(struct sbi_hsm_data *hdata)
+{
+	__smp_store_release(&hdata->aux_state, 1);
+}
+
+static inline void sbi_hsm_reset_runnable(struct sbi_hsm_data *hdata)
+{
+	__smp_store_release(&hdata->aux_state, 0);
+}
+
 void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
 {
 	u32 oldstate;
@@ -108,6 +126,7 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
 static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
 {
 	unsigned long saved_mie;
+	unsigned long cmip;
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
 	/* Save MIE CSR */
@@ -117,13 +136,22 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
 	csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
 
 	/* Wait for state transition requested by sbi_hsm_hart_start() */
-	while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
-		wfi();
+	while (!sbi_hsm_is_runnable(hdata)) {
+		do {
+			wfi();
+			cmip = csr_read(CSR_MIP);
+		} while (!(cmip & (MIP_MSIP | MIP_MEIP)));
 	};
 
 	/* Restore MIE CSR */
 	csr_write(CSR_MIE, saved_mie);
 
+	/*
+	 * If the hart ever needs to wait for such state transition again,
+	 * make sure the hart's data allows that.
+	 */
+	sbi_hsm_reset_runnable(hdata);
+
 	/*
 	 * No need to clear IPI here because the sbi_ipi_init() will
 	 * clear it for current HART via sbi_platform_ipi_init().
@@ -284,6 +312,9 @@ int sbi_hsm_hart_start(struct sbi_scratch *scratch,
 	rscratch->next_addr = saddr;
 	rscratch->next_mode = smode;
 
+	/* Inform the hart that it may continue as soon as it gets IPI. */
+	sbi_hsm_set_runnable(hdata);
+
 	if (hsm_device_has_hart_hotplug() ||
 	   (hsm_device_has_hart_secondary_boot() && !init_count)) {
 		return hsm_device_hart_start(hartid, scratch->warmboot_addr);
-- 
2.34.1




From e.shatokhin at yadro.com  Tue Feb 21 13:08:51 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Wed, 22 Feb 2023 00:08:51 +0300
Subject: [PATCH v2 2/2] lib: sbi: Clear IPIs before init_warm_startup in non-boot harts
In-Reply-To: <20230221210851.3145840-1-e.shatokhin@yadro.com>
References: <20230221210851.3145840-1-e.shatokhin@yadro.com>
Message-ID: <20230221210851.3145840-3-e.shatokhin@yadro.com>

Since commit 50d4fde1c5a4 ("lib: Remove redundant sbi_platform_ipi_clear()
calls"), the IPI sent from the boot hart in wake_coldboot_harts() is not
cleared in the secondary harts until they reach sbi_ipi_init(). However,
sbi_hsm_init() and sbi_hsm_hart_wait() are called earlier, so a secondary
hart might enter sbi_hsm_hart_wait() with an already pending IPI.

sbi_hsm_hart_wait() makes sure the hart leaves the loop only when it is
actually ready, so a pending unrelated IPI should not cause safety issues.
However, it might be inefficient on certain hardware, because it prevents
"wfi" from stalling the hart even if the hardware supports this, making the
hart needlessly spin in a "busy-wait" loop.

This behaviour can be observed, for example, in a QEMU VM (QEMU 7.2.0) with
"-machine virt" running a Linux guest. Inserting delays in
sbi_hsm_hart_start() allows reproducing the issue more reliably.

The comment in wait_for_coldboot() suggests that the initial IPI is needed
in the warm resume path, so let us clear it before init_warm_startup()
only.

To do this, sbi_ipi_raw_clear() was created similar to sbi_ipi_raw_send().

Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
---
 include/sbi/sbi_ipi.h | 2 ++
 lib/sbi/sbi_init.c    | 6 ++++--
 lib/sbi/sbi_ipi.c     | 6 ++++++
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
index f6ac807..f384e74 100644
--- a/include/sbi/sbi_ipi.h
+++ b/include/sbi/sbi_ipi.h
@@ -77,6 +77,8 @@ void sbi_ipi_process(void);
 
 int sbi_ipi_raw_send(u32 target_hart);
 
+void sbi_ipi_raw_clear(u32 target_hart);
+
 const struct sbi_ipi_device *sbi_ipi_get_device(void);
 
 void sbi_ipi_set_device(const struct sbi_ipi_device *dev);
diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
index e353c33..a9a4084 100644
--- a/lib/sbi/sbi_init.c
+++ b/lib/sbi/sbi_init.c
@@ -438,10 +438,12 @@ static void __noreturn init_warmboot(struct sbi_scratch *scratch, u32 hartid)
 	if (hstate < 0)
 		sbi_hart_hang();
 
-	if (hstate == SBI_HSM_STATE_SUSPENDED)
+	if (hstate == SBI_HSM_STATE_SUSPENDED) {
 		init_warm_resume(scratch);
-	else
+	} else {
+		sbi_ipi_raw_clear(hartid);
 		init_warm_startup(scratch, hartid);
+	}
 
 	sbi_hart_switch_mode(hartid, scratch->next_arg1,
 			     scratch->next_addr,
diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
index 1bcc2e4..b9f6205 100644
--- a/lib/sbi/sbi_ipi.c
+++ b/lib/sbi/sbi_ipi.c
@@ -217,6 +217,12 @@ int sbi_ipi_raw_send(u32 target_hart)
 	return 0;
 }
 
+void sbi_ipi_raw_clear(u32 target_hart)
+{
+	if (ipi_dev && ipi_dev->ipi_clear)
+		ipi_dev->ipi_clear(target_hart);
+}
+
 const struct sbi_ipi_device *sbi_ipi_get_device(void)
 {
 	return ipi_dev;
-- 
2.34.1




From peterlin at andestech.com  Wed Feb 22 00:42:48 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Wed, 22 Feb 2023 08:42:48 +0000
Subject: [PATCH] lib: sbi: sbi_ecall: Improve return SBI error
In-Reply-To: <2c3c52436f6dadf09fe733a257a46943ff57728a.camel@126.com>
References: <20230221104756.28573-1-peterlin@andestech.com>
 <2c3c52436f6dadf09fe733a257a46943ff57728a.camel@126.com>
Message-ID: <Y/XViK1LimqbfwSx@APC323>

On Tue, Feb 21, 2023 at 07:34:46PM +0800, Xiang W wrote:
> ? 2023-02-21???? 18:47 +0800?Yu Chien Peter Lin???
> > We should also check if the return error code is greater than 0
> > (SBI_SUCCESS), as this is an invalid error.
> > 
> > Also rename the variable returned via a0 to 'out_err' and use
> > SBI_ERR_NOT_SUPPORTED when the extension does not have a handler
> > for consistency with the standard SBI errors defined in Table 1
> > of the SBI specification.
> > 
> > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > ---
> > ?lib/sbi/sbi_ecall.c | 18 +++++++++---------
> > ?1 file changed, 9 insertions(+), 9 deletions(-)
> > 
> > diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> > index 27ce5d49..8accf675 100644
> > --- a/lib/sbi/sbi_ecall.c
> > +++ b/lib/sbi/sbi_ecall.c
> > @@ -97,34 +97,34 @@ void sbi_ecall_unregister_extension(struct sbi_ecall_extension *ext)
> > ?
> > ?int sbi_ecall_handler(struct sbi_trap_regs *regs)
> > ?{
> > -???????int ret = 0;
> > +???????int out_err = 0;
> There is no need to rename ret to out_err because a0 is not always an
> error code in legacy extensions
> Other than the above, it looks good to me.
> 
> Reviewed-by: Xiang W <wxjstz at 126.com>

Hi Xiang, Andrew,

Thanks for the review,
Sure I'll use 'ret' as it is in next patch.

Best regards,
Peter Lin


From atishp at atishpatra.org  Tue Feb 21 16:50:22 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Tue, 21 Feb 2023 16:50:22 -0800
Subject: [PATCH v2 1/5] lib: sbi_pmu: add callback for counter width
In-Reply-To: <20230217144413.3720307-2-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com> <20230217144413.3720307-2-mchitale@ventanamicro.com>
Message-ID: <CAOnJCUJ6d2f1yakMRDgGzi81NC-e2gkCXEGk4uFFGd0by4M=Mw@mail.gmail.com>

On Fri, Feb 17, 2023 at 6:44 AM Mayuresh Chitale
<mchitale at ventanamicro.com> wrote:
>
> This patch adds a callback to fetch the number of bits implemented for a
> custom firmware counter. If the callback fails or is not implemented then
> width defaults to 63.
>
> Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
> ---
>  include/sbi/sbi_pmu.h | 5 +++++
>  lib/sbi/sbi_pmu.c     | 6 ++++++
>  2 files changed, 11 insertions(+)
>
> diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
> index c365243..b3b75c1 100644
> --- a/include/sbi/sbi_pmu.h
> +++ b/include/sbi/sbi_pmu.h
> @@ -41,6 +41,11 @@ struct sbi_pmu_device {
>         bool (*fw_counter_match_code)(uint32_t counter_index,
>                                       uint32_t event_idx_code);
>
> +       /**
> +        * Fetch the max width of this counter in number of bits.
> +        */
> +       int (*fw_counter_width)(void);
> +
>         /**
>          * Read value of custom firmware counter
>          * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index 154dbda..a99c045 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -761,6 +761,7 @@ unsigned long sbi_pmu_num_ctr(void)
>
>  int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
>  {
> +       int width;
>         union sbi_pmu_ctr_info cinfo = {0};
>         struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
>
> @@ -782,6 +783,11 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
>                 cinfo.type = SBI_PMU_CTR_TYPE_FW;
>                 /* Firmware counters are always 64 bits wide */
>                 cinfo.width = 63;
> +               if (pmu_dev && pmu_dev->fw_counter_width) {
> +                       width = pmu_dev->fw_counter_width();
> +                       if (width)
> +                               cinfo.width = width - 1;
> +               }
>         }
>
>         *ctr_info = cinfo.value;
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


Reviewed-by: Atish Patra <atishp at rivosinc.com>

-- 
Regards,
Atish


From atishp at atishpatra.org  Tue Feb 21 16:50:35 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Tue, 21 Feb 2023 16:50:35 -0800
Subject: [PATCH v2 2/5] lib: sbi_pmu: Implement sbi_pmu_counter_fw_read_hi
In-Reply-To: <20230217144413.3720307-3-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com> <20230217144413.3720307-3-mchitale@ventanamicro.com>
Message-ID: <CAOnJCUJR6Ss834QTJsWF72U3jQyOS+7jqNiLPuCtRAAFCC7bFA@mail.gmail.com>

On Fri, Feb 17, 2023 at 6:44 AM Mayuresh Chitale
<mchitale at ventanamicro.com> wrote:
>
> To support 64 bit firmware counters on RV32 systems, we implement
> sbi_pmu_counter_fw_read_hi() which returns the upper 32 bits of
> the firmware counter value. On RV64 (or higher) systems, this
> function will always return zero.
>
> Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
> ---
>  include/sbi/sbi_ecall_interface.h | 1 +
>  lib/sbi/sbi_ecall_pmu.c           | 8 ++++++++
>  2 files changed, 9 insertions(+)
>
> diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
> index a3f2bf4..f5d6667 100644
> --- a/include/sbi/sbi_ecall_interface.h
> +++ b/include/sbi/sbi_ecall_interface.h
> @@ -99,6 +99,7 @@
>  #define SBI_EXT_PMU_COUNTER_START      0x3
>  #define SBI_EXT_PMU_COUNTER_STOP       0x4
>  #define SBI_EXT_PMU_COUNTER_FW_READ    0x5
> +#define SBI_EXT_PMU_COUNTER_FW_READ_HI 0x6
>
>  /** General pmu event codes specified in SBI PMU extension */
>  enum sbi_pmu_hw_generic_events_t {
> diff --git a/lib/sbi/sbi_ecall_pmu.c b/lib/sbi/sbi_ecall_pmu.c
> index 826c8a8..2d1e669 100644
> --- a/lib/sbi/sbi_ecall_pmu.c
> +++ b/lib/sbi/sbi_ecall_pmu.c
> @@ -54,6 +54,14 @@ static int sbi_ecall_pmu_handler(unsigned long extid, unsigned long funcid,
>                 ret = sbi_pmu_ctr_fw_read(regs->a0, &temp);
>                 *out_val = temp;
>                 break;
> +       case SBI_EXT_PMU_COUNTER_FW_READ_HI:
> +#if __riscv_xlen == 32
> +               ret = sbi_pmu_ctr_fw_read(regs->a0, &temp);
> +               *out_val = temp >> 32
> +#else
> +               *out_val = 0;
> +#endif
> +               break;
>         case SBI_EXT_PMU_COUNTER_START:
>
>  #if __riscv_xlen == 32
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


Reviewed-by: Atish Patra <atishp at rivosinc.com>

-- 
Regards,
Atish


From atishp at atishpatra.org  Tue Feb 21 16:51:02 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Tue, 21 Feb 2023 16:51:02 -0800
Subject: [PATCH v2 3/5] lib: sbi_pmu: Reserve space for implementation
 specific firmware events
In-Reply-To: <20230217144413.3720307-4-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com> <20230217144413.3720307-4-mchitale@ventanamicro.com>
Message-ID: <CAOnJCUKFLW5P4_tM9cazuQe3ksY+_ArWZxjJ9VcykAn1a08NNw@mail.gmail.com>

On Fri, Feb 17, 2023 at 6:44 AM Mayuresh Chitale
<mchitale at ventanamicro.com> wrote:
>
> We reserve space for SBI implementation specific custom firmware
> events which can be used by M-mode firmwares and HS-mode hypervisors
> for their own use. This reserved space is intentionally large to
> ensure that SBI implementation has enough space to accommodate
> platform specific firmware events as well.
>
> Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
> ---
>  include/sbi/sbi_ecall_interface.h | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
> index f5d6667..5732419 100644
> --- a/include/sbi/sbi_ecall_interface.h
> +++ b/include/sbi/sbi_ecall_interface.h
> @@ -183,6 +183,17 @@ enum sbi_pmu_fw_event_code_id {
>         SBI_PMU_FW_HFENCE_VVMA_ASID_SENT = 20,
>         SBI_PMU_FW_HFENCE_VVMA_ASID_RCVD = 21,
>         SBI_PMU_FW_MAX,
> +       /*
> +        * Event codes 22 to 255 are reserved for future use.
> +        * Event codes 256 to 65534 are reserved for SBI implementation
> +        * specific custom firmware events.
> +        */
> +       SBI_PMU_FW_RESERVED_MAX = 0xFFFE,
> +       /*
> +       * Event code 0xFFFF is used for platform specific firmware
> +       * events where the event data contains any event specific information.
> +       */
> +       SBI_PMU_FW_PLATFORM = 0xFFFF,
>  };
>
>  /** SBI PMU event idx type */
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



Reviewed-by: Atish Patra <atishp at rivosinc.com>
-- 
Regards,
Atish


From atishp at atishpatra.org  Tue Feb 21 17:03:51 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Tue, 21 Feb 2023 17:03:51 -0800
Subject: [PATCH v2 0/5] SBI PMU firmware counters and events improvement
In-Reply-To: <20230217144413.3720307-1-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com>
Message-ID: <CAOnJCUK36xPkTpz=AKwbArpdC+_pk5cL+Q5KuFmOeh6KfVsFBA@mail.gmail.com>

On Fri, Feb 17, 2023 at 6:44 AM Mayuresh Chitale
<mchitale at ventanamicro.com> wrote:
>
> This patchset implements the improvements described in the following
> proposal from Anup Patel:
>
> https://lists.riscv.org/g/tech-prs/message/102
>

For the larger audience, there is a v2 available now.
https://lists.riscv.org/g/tech-prs/message/211

> Changes in v2:
> - Update Risc V platform specific firmware event handling as per
>   the SBI spec update: https://lists.riscv.org/g/tech-prs/message/211
>
> Mayuresh Chitale (5):
>   lib: sbi_pmu: add callback for counter width
>   lib: sbi_pmu: Implement sbi_pmu_counter_fw_read_hi
>   lib: sbi_pmu: Reserve space for implementation specific firmware
>     events
>   lib: sbi_pmu: Update custom PMU device ops
>   lib: sbi_pmu: Add hartid parameter PMU device ops
>
>  include/sbi/sbi_ecall_interface.h |  12 +++
>  include/sbi/sbi_pmu.h             |  33 +++++---
>  lib/sbi/sbi_ecall_pmu.c           |   8 ++
>  lib/sbi/sbi_pmu.c                 | 120 +++++++++++++++++++-----------
>  4 files changed, 121 insertions(+), 52 deletions(-)
>
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



-- 
Regards,
Atish


From peterlin at andestech.com  Wed Feb 22 02:05:03 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Wed, 22 Feb 2023 10:05:03 +0000
Subject: [PATCH v5 3/3] platform: generic: renesas: rzfive: Add SBI EXT
 to check for enabling IOCP errata
In-Reply-To: <4FB80591-F4AE-4735-82CA-4B69B129DE2B@kernel.org>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
 <20230213215111.32017-4-prabhakar.mahadev-lad.rj@bp.renesas.com>
 <CAAhSdy046QNs-ae+mKs1J0E0bPv_Ew0JP4sNQa2RVzPmW6u8EA@mail.gmail.com>
 <4FB80591-F4AE-4735-82CA-4B69B129DE2B@kernel.org>
Message-ID: <Y/Xozwqbm/wuxm2g@APC323>

On Tue, Feb 21, 2023 at 10:06:31AM +0000, Conor Dooley wrote:
> 
> 
> On 20 February 2023 15:31:32 GMT, Anup Patel <anup at brainfault.org> wrote:
> >On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
> ><prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
> >>
> >> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> >> external non-caching masters, such as DMA controllers. The accesses
> >> from IOCP are coherent with D-Caches and L2 Cache.
> >>
> >> IOCP is a specification option and is disabled on the Renesas RZ/Five
> >> SoC due to this reason IP blocks using DMA will fail.
> >>
> >> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> >> software. Firstly OpenSBI configures the memory region as
> >> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> >> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> >> allocations happen from this region and synchronization callbacks are
> >> implemented to synchronize when doing DMA transactions.
> >>
> >> RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND SBI EXT checks if the IOCP
> >> errata should be applied to handle cache management.
> >>
> >> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> >> Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>
> >> ---
> >>  platform/generic/include/andes/andes45.h | 25 +++++++++++++--
> >>  platform/generic/renesas/rzfive/rzfive.c | 41 ++++++++++++++++++++++++
> >>  2 files changed, 64 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/platform/generic/include/andes/andes45.h b/platform/generic/include/andes/andes45.h
> >> index 08b3d18..d5d265f 100644
> >> --- a/platform/generic/include/andes/andes45.h
> >> +++ b/platform/generic/include/andes/andes45.h
> >> @@ -4,7 +4,28 @@
> >>  #define CSR_MARCHID_MICROID 0xfff
> >>
> >>  /* Memory and Miscellaneous Registers */
> >> -#define CSR_MCACHE_CTL 0x7ca
> >> -#define CSR_MCCTLCOMMAND 0x7cc
> >> +#define CSR_MCACHE_CTL         0x7ca
> >> +#define CSR_MCCTLCOMMAND       0x7cc
> >> +#define CSR_MICM_CFG           0xfc0
> >> +#define CSR_MDCM_CFG           0xfc1
> >> +#define CSR_MMSC_CFG           0xfc2
> >> +
> >> +#define MISA_20_OFFSET                 20
> >> +#define MISA_20_MASK                   (0x1 << MISA_20_OFFSET)
> >> +
> >> +#define MICM_CFG_ISZ_OFFSET            6
> >> +#define MICM_CFG_ISZ_MASK              (0x7  << MICM_CFG_ISZ_OFFSET)
> >> +
> >> +#define MDCM_CFG_DSZ_OFFSET            6
> >> +#define MDCM_CFG_DSZ_MASK              (0x7  << MDCM_CFG_DSZ_OFFSET)
> >> +
> >> +#define MMSC_CFG_CCTLCSR_OFFSET                16
> >> +#define MMSC_CFG_CCTLCSR_MASK          (0x1 << MMSC_CFG_CCTLCSR_OFFSET)
> >> +#define MMSC_IOCP_OFFSET                       47
> >> +#define MMSC_IOCP_MASK                 (0x1ULL << MMSC_IOCP_OFFSET)
> >> +
> >> +#define MCACHE_CTL_CCTL_SUEN_OFFSET    8
> >> +#define MCACHE_CTL_CCTL_SUEN_MASK      (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET)
> >> +
> >>
> >>  #endif /* _RISCV_ANDES45_H */
> >> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> >> index 4d71d0d..9530d28 100644
> >> --- a/platform/generic/renesas/rzfive/rzfive.c
> >> +++ b/platform/generic/renesas/rzfive/rzfive.c
> >> @@ -4,11 +4,14 @@
> >>   *
> >>   */
> >>
> >> +#include <andes/andes45.h>
> >>  #include <andes/andes45_pma.h>
> >>  #include <platform_override.h>
> >>  #include <sbi/sbi_domain.h>
> >>  #include <sbi_utils/fdt/fdt_helper.h>
> >>
> >> +#define RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND      0
> >
> >The SBI vendor extension_id will be the same for all SoCs based on Andes
> >CPUs so the function_id space will be shared across all such SoCs.
> 
> Apologies for sticking my nose in here, but I'm a bit confused as to why is being added as an rz/five specific thing in the first place?
> My uninformed view of the world sees other purchasers of the IP needing to apply this workaround too, no?
> 
> >How does Andes plan to manage the function_id space of SBI vendor extension ?
> >
> >I suggest the following:
> >1) The function_id == 0 can be used to return an SoC identifier. For example,
> >     Renesas RzFive can have an SoC identifier "0".
> >2) Move RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to a
> >    separate header at platform/generic/include/andes/andes_sbi.h and
> >    change it's value to "1"
> >
> >The above suggestion will help multiple SoC with Andes CPU share same
> >SBI function_id space for SBI vendor extension.
> 
> Sharing here makes complete sense to me, so that you don't end up duplicating this the next time someone buys the IP without the iocp.
> 
> Cheers,
> Conor.

Hi Anup, Conor,

Apologies for any confusion caused, please disregard my previous email.
Sure we can go ahead with your suggestion of sharing the FID space with
all SoC based on Andes CPU.

Thanks,
Peter Lin




From atishp at atishpatra.org  Tue Feb 21 18:42:09 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Tue, 21 Feb 2023 18:42:09 -0800
Subject: [PATCH v2 4/5] lib: sbi_pmu: Update custom PMU device ops
In-Reply-To: <20230217144413.3720307-5-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com> <20230217144413.3720307-5-mchitale@ventanamicro.com>
Message-ID: <CAOnJCU+geopBQy2XL8MctTcX8o60qY_c5ZMQyxpQ49rW=0oG0Q@mail.gmail.com>

On Fri, Feb 17, 2023 at 6:44 AM Mayuresh Chitale
<mchitale at ventanamicro.com> wrote:
>
> Update the implmentation of some of the PMU device operations to make them

/s/implmentation/implementation

> inline with the SBI spec updates for the platform specific firmware events.
>

This patch can be further split into smaller patches as it adds
multiple things in a single patch.
The commit text needs to improvement as well to indicate the purpose
more closely.

1. Allows event data for firmware events.
2. Changed the sbi_pmu_device functions to use uint64_t instead of uint32_t
3. Restrict the platform pmu device ops for SBI_PMU_FW_PLATFORM which
is a behavioral change from existing
implemenation. Even though there are no actual uses for this feature
upstream, this should be done in a separate patch
with appropriate commit message.

> Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
> ---
>  include/sbi/sbi_pmu.h |  16 ++++---
>  lib/sbi/sbi_pmu.c     | 102 ++++++++++++++++++++++++++----------------
>  2 files changed, 73 insertions(+), 45 deletions(-)
>
> diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
> index b3b75c1..467ec03 100644
> --- a/include/sbi/sbi_pmu.h
> +++ b/include/sbi/sbi_pmu.h
> @@ -30,16 +30,15 @@ struct sbi_pmu_device {
>
>         /**
>          * Validate event code of custom firmware event
> -        * Note: SBI_PMU_FW_MAX <= event_idx_code
>          */
> -       int (*fw_event_validate_code)(uint32_t event_idx_code);
> +       int (*fw_event_validate_code)(uint64_t event_idx_code);
>
>         /**
>          * Match custom firmware counter with custom firmware event
>          * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
>          */
>         bool (*fw_counter_match_code)(uint32_t counter_index,
> -                                     uint32_t event_idx_code);
> +                                     uint64_t event_idx_code);
>
>         /**
>          * Fetch the max width of this counter in number of bits.
> @@ -52,14 +51,19 @@ struct sbi_pmu_device {
>          */
>         uint64_t (*fw_counter_read_value)(uint32_t counter_index);
>
> +       /**
> +        * Write value to custom firmware counter
> +        * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
> +        */
> +       void (*fw_counter_write_value)(uint32_t counter_index,
> +                                      uint64_t value);
> +
>         /**
>          * Start custom firmware counter
> -        * Note: SBI_PMU_FW_MAX <= event_idx_code
>          * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
>          */
>         int (*fw_counter_start)(uint32_t counter_index,
> -                               uint32_t event_idx_code,
> -                               uint64_t init_val, bool init_val_update);
> +                               uint64_t event_idx_code);
>
>         /**
>          * Stop custom firmware counter
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index a99c045..05e2571 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -65,8 +65,12 @@ static uint32_t active_events[SBI_HARTMASK_MAX_BITS][SBI_PMU_HW_CTR_MAX + SBI_PM
>  #endif
>  static unsigned long fw_counters_started[SBI_HARTMASK_MAX_BITS];
>
> -/* Values of firmwares counters on each HART */
> -static uint64_t fw_counters_value[SBI_HARTMASK_MAX_BITS][SBI_PMU_FW_CTR_MAX] = {0};
> +/*
> + * Counter values for SBI firmware events and event codes for platform
> + * firmware events. Both are mutually exclusive and hence can optimally share
> + * the same memory.
> + */
> +static uint64_t fw_counters_data[SBI_HARTMASK_MAX_BITS][SBI_PMU_FW_CTR_MAX] = {0};
>
>  /* Maximum number of hardware events available */
>  static uint32_t num_hw_events;
> @@ -106,7 +110,7 @@ static bool pmu_event_select_overlap(struct sbi_pmu_hw_event *evt,
>         return false;
>  }
>
> -static int pmu_event_validate(unsigned long event_idx)
> +static int pmu_event_validate(unsigned long event_idx, uint64_t edata)
>  {
>         uint32_t event_idx_type = get_cidx_type(event_idx);
>         uint32_t event_idx_code = get_cidx_code(event_idx);
> @@ -118,9 +122,9 @@ static int pmu_event_validate(unsigned long event_idx)
>                 event_idx_code_max = SBI_PMU_HW_GENERAL_MAX;
>                 break;
>         case SBI_PMU_EVENT_TYPE_FW:
> -               if (SBI_PMU_FW_MAX <= event_idx_code &&
> +               if (SBI_PMU_FW_PLATFORM == event_idx_code &&
>                     pmu_dev && pmu_dev->fw_event_validate_code)
> -                       return pmu_dev->fw_event_validate_code(event_idx_code);
> +                       return pmu_dev->fw_event_validate_code(edata);
>                 else
>                         event_idx_code_max = SBI_PMU_FW_MAX;
>                 break;
> @@ -183,12 +187,14 @@ int sbi_pmu_ctr_fw_read(uint32_t cidx, uint64_t *cval)
>         if (event_idx_type != SBI_PMU_EVENT_TYPE_FW)
>                 return SBI_EINVAL;
>
> -       if (SBI_PMU_FW_MAX <= event_code &&
> -           pmu_dev && pmu_dev->fw_counter_read_value)
> -               fw_counters_value[hartid][cidx - num_hw_ctrs] =
> -                       pmu_dev->fw_counter_read_value(cidx - num_hw_ctrs);
> -
> -       *cval = fw_counters_value[hartid][cidx - num_hw_ctrs];
> +       if (SBI_PMU_FW_PLATFORM == event_code) {
> +               if (pmu_dev && pmu_dev->fw_counter_read_value)
> +                       *cval = pmu_dev->fw_counter_read_value(cidx -
> +                                                              num_hw_ctrs);
> +               else
> +                       *cval = 0;
> +       } else
> +               *cval = fw_counters_data[hartid][cidx - num_hw_ctrs];
>
>         return 0;
>  }
> @@ -357,22 +363,30 @@ int sbi_pmu_irq_bit(void)
>  }
>
>  static int pmu_ctr_start_fw(uint32_t cidx, uint32_t event_code,
> -                           uint64_t ival, bool ival_update)
> +                           uint64_t event_data, uint64_t ival,
> +                           bool ival_update)
>  {
> -       int ret;
>         u32 hartid = current_hartid();
>
> -       if (SBI_PMU_FW_MAX <= event_code &&
> -           pmu_dev && pmu_dev->fw_counter_start) {
> -               ret = pmu_dev->fw_counter_start(cidx - num_hw_ctrs,
> -                                               event_code,
> -                                               ival, ival_update);
> -               if (ret)
> -                       return ret;
> +       if (SBI_PMU_FW_PLATFORM == event_code) {
> +               if (!pmu_dev ||
> +                   !pmu_dev->fw_counter_write_value ||
> +                   !pmu_dev->fw_counter_start) {
> +                       return SBI_EINVAL;
> +                   }
> +
> +               if (ival_update)
> +                       pmu_dev->fw_counter_write_value(cidx - num_hw_ctrs,
> +                                                       ival);
> +
> +               if (pmu_dev->fw_counter_start(hartid, cidx - num_hw_ctrs,
> +                                             event_data))
> +                       return SBI_EINVAL;
> +       } else {
> +               if (ival_update)
> +                       fw_counters_data[hartid][cidx - num_hw_ctrs] = ival;
>         }
>
> -       if (ival_update)
> -               fw_counters_value[hartid][cidx - num_hw_ctrs] = ival;
>         fw_counters_started[hartid] |= BIT(cidx - num_hw_ctrs);
>
>         return 0;
> @@ -381,8 +395,10 @@ static int pmu_ctr_start_fw(uint32_t cidx, uint32_t event_code,
>  int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
>                       unsigned long flags, uint64_t ival)
>  {
> +       u32 hartid = current_hartid();
>         int event_idx_type;
>         uint32_t event_code;
> +       uint64_t edata;
>         int ret = SBI_EINVAL;
>         bool bUpdate = false;
>         int i, cidx;
> @@ -399,8 +415,13 @@ int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
>                 if (event_idx_type < 0)
>                         /* Continue the start operation for other counters */
>                         continue;
> -               else if (event_idx_type == SBI_PMU_EVENT_TYPE_FW)
> -                       ret = pmu_ctr_start_fw(cidx, event_code, ival, bUpdate);
> +               else if (event_idx_type == SBI_PMU_EVENT_TYPE_FW) {
> +                       edata = (event_code == SBI_PMU_FW_PLATFORM) ?
> +                                fw_counters_data[hartid][cidx - num_hw_ctrs]
> +                                : 0x0;
> +                       ret = pmu_ctr_start_fw(cidx, event_code, edata, ival,
> +                                              bUpdate);
> +               }
>                 else
>                         ret = pmu_ctr_start_hw(cidx, ival, bUpdate);
>         }
> @@ -434,7 +455,7 @@ static int pmu_ctr_stop_fw(uint32_t cidx, uint32_t event_code)
>  {
>         int ret;
>
> -       if (SBI_PMU_FW_MAX <= event_code &&
> +       if (SBI_PMU_FW_PLATFORM == event_code &&
>             pmu_dev && pmu_dev->fw_counter_stop) {
>                 ret = pmu_dev->fw_counter_stop(cidx - num_hw_ctrs);
>                 if (ret)
> @@ -640,7 +661,7 @@ static int pmu_ctr_find_hw(unsigned long cbase, unsigned long cmask, unsigned lo
>   * check.
>   */
>  static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask,
> -                          uint32_t event_code, u32 hartid)
> +                          uint32_t event_code, u32 hartid, uint32_t edata)
>  {
>         int i, cidx;
>
> @@ -650,10 +671,11 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask,
>                         continue;
>                 if (active_events[hartid][i] != SBI_PMU_EVENT_IDX_INVALID)
>                         continue;
> -               if (SBI_PMU_FW_MAX <= event_code &&
> +               if (SBI_PMU_FW_PLATFORM == event_code &&
>                     pmu_dev && pmu_dev->fw_counter_match_code) {
> -                       if (!pmu_dev->fw_counter_match_code(cidx - num_hw_ctrs,
> -                                                           event_code))
> +                       if (!pmu_dev->fw_counter_match_code(cidx -
> +                                                           num_hw_ctrs,
> +                                                           edata))
>                                 continue;
>                 }
>
> @@ -675,7 +697,7 @@ int sbi_pmu_ctr_cfg_match(unsigned long cidx_base, unsigned long cidx_mask,
>         if ((cidx_base + sbi_fls(cidx_mask)) >= total_ctrs)
>                 return SBI_EINVAL;
>
> -       event_type = pmu_event_validate(event_idx);
> +       event_type = pmu_event_validate(event_idx, event_data);
>         if (event_type < 0)
>                 return SBI_EINVAL;
>         event_code = get_cidx_code(event_idx);
> @@ -693,7 +715,8 @@ int sbi_pmu_ctr_cfg_match(unsigned long cidx_base, unsigned long cidx_mask,
>
>         if (event_type == SBI_PMU_EVENT_TYPE_FW) {
>                 /* Any firmware counter can be used track any firmware event */
> -               ctr_idx = pmu_ctr_find_fw(cidx_base, cidx_mask, event_code, hartid);
> +               ctr_idx = pmu_ctr_find_fw(cidx_base, cidx_mask, event_code,
> +                                         hartid, event_data);
>         } else {
>                 ctr_idx = pmu_ctr_find_hw(cidx_base, cidx_mask, flags, event_idx,
>                                           event_data);
> @@ -710,15 +733,16 @@ skip_match:
>                 if (flags & SBI_PMU_CFG_FLAG_AUTO_START)
>                         pmu_ctr_start_hw(ctr_idx, 0, false);
>         } else if (event_type == SBI_PMU_EVENT_TYPE_FW) {
> +               if (event_code == SBI_PMU_FW_PLATFORM)
> +                       fw_counters_data[hartid][ctr_idx - num_hw_ctrs] =
> +                                                               event_data;
>                 if (flags & SBI_PMU_CFG_FLAG_CLEAR_VALUE)
> -                       fw_counters_value[hartid][ctr_idx - num_hw_ctrs] = 0;
> +                       fw_counters_data[hartid][ctr_idx - num_hw_ctrs] = 0;
>                 if (flags & SBI_PMU_CFG_FLAG_AUTO_START) {
> -                       if (SBI_PMU_FW_MAX <= event_code &&
> +                       if (SBI_PMU_FW_PLATFORM == event_code &&
>                             pmu_dev && pmu_dev->fw_counter_start) {
> -                               ret = pmu_dev->fw_counter_start(
> -                                       ctr_idx - num_hw_ctrs, event_code,
> -                                       fw_counters_value[hartid][ctr_idx - num_hw_ctrs],
> -                                       true);
> +                               ret = pmu_dev->fw_counter_start(ctr_idx - num_hw_ctrs,
> +                                                       event_data);
>                                 if (ret)
>                                         return ret;
>                         }
> @@ -743,7 +767,7 @@ int sbi_pmu_ctr_incr_fw(enum sbi_pmu_fw_event_code_id fw_id)
>         for (cidx = num_hw_ctrs; cidx < total_ctrs; cidx++) {
>                 if (get_cidx_code(active_events[hartid][cidx]) == fw_id &&
>                     (fw_counters_started[hartid] & BIT(cidx - num_hw_ctrs))) {
> -                       fcounter = &fw_counters_value[hartid][cidx - num_hw_ctrs];
> +                       fcounter = &fw_counters_data[hartid][cidx - num_hw_ctrs];
>                         break;
>                 }
>         }
> @@ -803,7 +827,7 @@ static void pmu_reset_event_map(u32 hartid)
>         for (j = 3; j < total_ctrs; j++)
>                 active_events[hartid][j] = SBI_PMU_EVENT_IDX_INVALID;
>         for (j = 0; j < SBI_PMU_FW_CTR_MAX; j++)
> -               fw_counters_value[hartid][j] = 0;
> +               fw_counters_data[hartid][j] = 0;
>         fw_counters_started[hartid] = 0;
>  }
>
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



-- 
Regards,
Atish


From peterlin at andestech.com  Tue Feb 21 18:48:54 2023
From: peterlin at andestech.com (Yu Chien Peter Lin)
Date: Wed, 22 Feb 2023 10:48:54 +0800
Subject: [PATCH v2] lib: sbi: sbi_ecall: Check the range of SBI error
Message-ID: <20230222024854.24876-1-peterlin@andestech.com>

We should also check if the return error code is greater than 0
(SBI_SUCCESS), as this is an invalid error.

Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
Reviewed-by: Xiang W <wxjstz at 126.com>
---
Changes v1 -> v2:
- Include Xiang's Reviewed-by
- Drop the 'ret -> out_err' rename
---
 lib/sbi/sbi_ecall.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
index 27ce5d49..d0f01665 100644
--- a/lib/sbi/sbi_ecall.c
+++ b/lib/sbi/sbi_ecall.c
@@ -120,7 +120,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
 		trap.epc = regs->mepc;
 		sbi_trap_redirect(regs, &trap);
 	} else {
-		if (ret < SBI_LAST_ERR) {
+		if (ret < SBI_LAST_ERR || SBI_SUCCESS < ret) {
 			sbi_printf("%s: Invalid error %d for ext=0x%lx "
 				   "func=0x%lx\n", __func__, ret,
 				   extension_id, func_id);
-- 
2.34.1



From atishp at atishpatra.org  Tue Feb 21 18:49:49 2023
From: atishp at atishpatra.org (Atish Patra)
Date: Tue, 21 Feb 2023 18:49:49 -0800
Subject: [PATCH v2 5/5] lib: sbi_pmu: Add hartid parameter PMU device ops
In-Reply-To: <20230217144413.3720307-6-mchitale@ventanamicro.com>
References: <20230217144413.3720307-1-mchitale@ventanamicro.com> <20230217144413.3720307-6-mchitale@ventanamicro.com>
Message-ID: <CAOnJCUK99ApA+XPX6Oxe-ojDhW9m5s_7+vRTOpkJA9YJe2L4fQ@mail.gmail.com>

On Fri, Feb 17, 2023 at 6:45 AM Mayuresh Chitale
<mchitale at ventanamicro.com> wrote:
>
> Pass the hartid parameter to custom PMU device ops. It is required by
> platform event handler to be able to program correct registers for a
> given counter.
>

This can be improved..something along the lines

"Platform event handler may leverage the hartid to program  per hart
specific registers
for a given counter."

> Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
> ---
>  include/sbi/sbi_pmu.h | 18 +++++++++++-------
>  lib/sbi/sbi_pmu.c     | 28 +++++++++++++++++-----------
>  2 files changed, 28 insertions(+), 18 deletions(-)
>
> diff --git a/include/sbi/sbi_pmu.h b/include/sbi/sbi_pmu.h
> index 467ec03..90a564f 100644
> --- a/include/sbi/sbi_pmu.h
> +++ b/include/sbi/sbi_pmu.h
> @@ -31,14 +31,15 @@ struct sbi_pmu_device {
>         /**
>          * Validate event code of custom firmware event
>          */
> -       int (*fw_event_validate_code)(uint64_t event_idx_code);
> +       int (*fw_event_validate_encoding)(uint64_t event_idx_code);
>
>         /**
>          * Match custom firmware counter with custom firmware event
>          * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
>          */
> -       bool (*fw_counter_match_code)(uint32_t counter_index,
> -                                     uint64_t event_idx_code);
> +       bool (*fw_counter_match_encoding)(uint32_t hartid,
> +                                         uint32_t counter_index,
> +                                         uint64_t event_idx_code);
>
>         /**
>          * Fetch the max width of this counter in number of bits.
> @@ -49,27 +50,30 @@ struct sbi_pmu_device {
>          * Read value of custom firmware counter
>          * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
>          */
> -       uint64_t (*fw_counter_read_value)(uint32_t counter_index);
> +       uint64_t (*fw_counter_read_value)(uint32_t hartid,
> +                                         uint32_t counter_index);
>
>         /**
>          * Write value to custom firmware counter
>          * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
>          */
> -       void (*fw_counter_write_value)(uint32_t counter_index,
> +       void (*fw_counter_write_value)(uint32_t hartid,
> +                                      uint32_t counter_index,
>                                        uint64_t value);
>
>         /**
>          * Start custom firmware counter
>          * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
>          */
> -       int (*fw_counter_start)(uint32_t counter_index,
> +       int (*fw_counter_start)(uint32_t hartid,
> +                               uint32_t counter_index,
>                                 uint64_t event_idx_code);
>
>         /**
>          * Stop custom firmware counter
>          * Note: 0 <= counter_index < SBI_PMU_FW_CTR_MAX
>          */
> -       int (*fw_counter_stop)(uint32_t counter_index);
> +       int (*fw_counter_stop)(uint32_t hartid, uint32_t counter_index);
>
>         /**
>          * Custom enable irq for hardware counter
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index 05e2571..4479aa4 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -123,8 +123,8 @@ static int pmu_event_validate(unsigned long event_idx, uint64_t edata)
>                 break;
>         case SBI_PMU_EVENT_TYPE_FW:
>                 if (SBI_PMU_FW_PLATFORM == event_idx_code &&
> -                   pmu_dev && pmu_dev->fw_event_validate_code)
> -                       return pmu_dev->fw_event_validate_code(edata);
> +                   pmu_dev && pmu_dev->fw_event_validate_encoding)
> +                       return pmu_dev->fw_event_validate_encoding(edata);
>                 else
>                         event_idx_code_max = SBI_PMU_FW_MAX;
>                 break;
> @@ -189,7 +189,8 @@ int sbi_pmu_ctr_fw_read(uint32_t cidx, uint64_t *cval)
>
>         if (SBI_PMU_FW_PLATFORM == event_code) {
>                 if (pmu_dev && pmu_dev->fw_counter_read_value)
> -                       *cval = pmu_dev->fw_counter_read_value(cidx -
> +                       *cval = pmu_dev->fw_counter_read_value(hartid,
> +                                                              cidx -
>                                                                num_hw_ctrs);
>                 else
>                         *cval = 0;
> @@ -376,7 +377,8 @@ static int pmu_ctr_start_fw(uint32_t cidx, uint32_t event_code,
>                     }
>
>                 if (ival_update)
> -                       pmu_dev->fw_counter_write_value(cidx - num_hw_ctrs,
> +                       pmu_dev->fw_counter_write_value(hartid,
> +                                                       cidx - num_hw_ctrs,
>                                                         ival);
>
>                 if (pmu_dev->fw_counter_start(hartid, cidx - num_hw_ctrs,
> @@ -453,11 +455,12 @@ static int pmu_ctr_stop_hw(uint32_t cidx)
>
>  static int pmu_ctr_stop_fw(uint32_t cidx, uint32_t event_code)
>  {
> +       u32 hartid = current_hartid();
>         int ret;
>
>         if (SBI_PMU_FW_PLATFORM == event_code &&
>             pmu_dev && pmu_dev->fw_counter_stop) {
> -               ret = pmu_dev->fw_counter_stop(cidx - num_hw_ctrs);
> +               ret = pmu_dev->fw_counter_stop(hartid, cidx - num_hw_ctrs);
>                 if (ret)
>                         return ret;
>         }
> @@ -672,10 +675,11 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask,
>                 if (active_events[hartid][i] != SBI_PMU_EVENT_IDX_INVALID)
>                         continue;
>                 if (SBI_PMU_FW_PLATFORM == event_code &&
> -                   pmu_dev && pmu_dev->fw_counter_match_code) {
> -                       if (!pmu_dev->fw_counter_match_code(cidx -
> -                                                           num_hw_ctrs,
> -                                                           edata))
> +                   pmu_dev && pmu_dev->fw_counter_match_encoding) {
> +                       if (!pmu_dev->fw_counter_match_encoding(hartid,
> +                                                               cidx -
> +                                                               num_hw_ctrs,
> +                                                               edata))
>                                 continue;
>                 }
>
> @@ -741,8 +745,10 @@ skip_match:
>                 if (flags & SBI_PMU_CFG_FLAG_AUTO_START) {
>                         if (SBI_PMU_FW_PLATFORM == event_code &&
>                             pmu_dev && pmu_dev->fw_counter_start) {
> -                               ret = pmu_dev->fw_counter_start(ctr_idx - num_hw_ctrs,
> -                                                       event_data);
> +                               ret = pmu_dev->fw_counter_start(hartid,
> +                                                               ctr_idx -
> +                                                               num_hw_ctrs,
> +                                                               event_data);
>                                 if (ret)
>                                         return ret;
>                         }
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

IMO, this patch can be applied before PATCH4.

Either Way,

Reviewed-by: Atish Patra <atishp at rivosinc.com>
-- 
Regards,
Atish


From anup at brainfault.org  Tue Feb 21 19:29:52 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 22 Feb 2023 08:59:52 +0530
Subject: [PATCH] lib: sbi: sbi_ecall: Improve return SBI error
In-Reply-To: <20230221180724.vtmg4dgc4msb7io6@orel>
References: <20230221104756.28573-1-peterlin@andestech.com>
 <20230221172124.34ccwi3cg2c2ilff@orel> <CAAhSdy1TEiRkw796rg5yNqLuOog33s4xUJhSdcxGhrG07kxgsg@mail.gmail.com>
 <20230221180724.vtmg4dgc4msb7io6@orel>
Message-ID: <CAAhSdy1NH+05cc5_LukHLahVx6efc6=YvN-2FK1Qpb_0DzkVbg@mail.gmail.com>

On Tue, Feb 21, 2023 at 11:37 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> On Tue, Feb 21, 2023 at 10:53:31PM +0530, Anup Patel wrote:
> > On Tue, Feb 21, 2023 at 10:51 PM Andrew Jones <ajones at ventanamicro.com> wrote:
> > >
> > > On Tue, Feb 21, 2023 at 06:47:56PM +0800, Yu Chien Peter Lin wrote:
> > > > We should also check if the return error code is greater than 0
> > > > (SBI_SUCCESS), as this is an invalid error.
> > > >
> > > > Also rename the variable returned via a0 to 'out_err' and use
> > > > SBI_ERR_NOT_SUPPORTED when the extension does not have a handler
> > > > for consistency with the standard SBI errors defined in Table 1
> > > > of the SBI specification.
> > >
> > > It's best to do renames separate from functional changes, and, in
> > > this case, the ret -> out_err rename feels like unnecessary churn.
> >
> > Maybe we can do something similar to KVM RISC-V ?
>
> We could if we have three types of return values to manage like we do
> with KVM. For anyone who isn't familiar with how KVM will manage things,
> once the PMU patches are merged, it'll work like this:
>
>  1. Linux error while emulating an SBI call (e.g. ENOMEM)
>  2. 'sbiret.error' generated during emulation
>  3. 'sbiret.value' generated during emulation
>
> (1) is the return value of an SBI emulation function,
>
>     int sbi_func(struct kvm_vcpu_sbi_return *retdata)
>
> (2) is kvm_vcpu_sbi_return.out_val
> (3) is kvm_vcpu_sbi_return.err_val
>
>
> But don't we only have (2) and (3) here?

We do have (1) as well because SBI_Exxx are supposed
to be errors internal to OpenSBI whereas SBI_ERR_xxx
are standard SBI errors.

Maybe as a separate series we can improve error returned
by SBI calls in OpenSBI?

Regards,
Anup

>
> Thanks,
> drew
>
> >
> > Regards,
> > Anup
> >
> > >
> > > Thanks,
> > > drew
> > >
> > > >
> > > > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > > > ---
> > > >  lib/sbi/sbi_ecall.c | 18 +++++++++---------
> > > >  1 file changed, 9 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> > > > index 27ce5d49..8accf675 100644
> > > > --- a/lib/sbi/sbi_ecall.c
> > > > +++ b/lib/sbi/sbi_ecall.c
> > > > @@ -97,34 +97,34 @@ void sbi_ecall_unregister_extension(struct sbi_ecall_extension *ext)
> > > >
> > > >  int sbi_ecall_handler(struct sbi_trap_regs *regs)
> > > >  {
> > > > -     int ret = 0;
> > > > +     int out_err = 0;
> > > > +     unsigned long out_val = 0;
> > > >       struct sbi_ecall_extension *ext;
> > > >       unsigned long extension_id = regs->a7;
> > > >       unsigned long func_id = regs->a6;
> > > >       struct sbi_trap_info trap = {0};
> > > > -     unsigned long out_val = 0;
> > > >       bool is_0_1_spec = 0;
> > > >
> > > >       ext = sbi_ecall_find_extension(extension_id);
> > > >       if (ext && ext->handle) {
> > > > -             ret = ext->handle(extension_id, func_id,
> > > > +             out_err = ext->handle(extension_id, func_id,
> > > >                                 regs, &out_val, &trap);
> > > >               if (extension_id >= SBI_EXT_0_1_SET_TIMER &&
> > > >                   extension_id <= SBI_EXT_0_1_SHUTDOWN)
> > > >                       is_0_1_spec = 1;
> > > >       } else {
> > > > -             ret = SBI_ENOTSUPP;
> > > > +             out_err = SBI_ERR_NOT_SUPPORTED;
> > > >       }
> > > >
> > > > -     if (ret == SBI_ETRAP) {
> > > > +     if (out_err == SBI_ETRAP) {
> > > >               trap.epc = regs->mepc;
> > > >               sbi_trap_redirect(regs, &trap);
> > > >       } else {
> > > > -             if (ret < SBI_LAST_ERR) {
> > > > +             if (out_err < SBI_LAST_ERR || SBI_SUCCESS < out_err) {
> > > >                       sbi_printf("%s: Invalid error %d for ext=0x%lx "
> > > > -                                "func=0x%lx\n", __func__, ret,
> > > > +                                "func=0x%lx\n", __func__, out_err,
> > > >                                  extension_id, func_id);
> > > > -                     ret = SBI_ERR_FAILED;
> > > > +                     out_err = SBI_ERR_FAILED;
> > > >               }
> > > >
> > > >               /*
> > > > @@ -136,7 +136,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
> > > >                * case should be handled differently.
> > > >                */
> > > >               regs->mepc += 4;
> > > > -             regs->a0 = ret;
> > > > +             regs->a0 = out_err;
> > > >               if (!is_0_1_spec)
> > > >                       regs->a1 = out_val;
> > > >       }
> > > > --
> > > > 2.34.1
> > > >
> > > >
> > > > --
> > > > opensbi mailing list
> > > > opensbi at lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/opensbi
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi


From ajones at ventanamicro.com  Wed Feb 22 04:24:05 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Wed, 22 Feb 2023 13:24:05 +0100
Subject: [PATCH v2] lib: sbi: sbi_ecall: Check the range of SBI error
In-Reply-To: <20230222024854.24876-1-peterlin@andestech.com>
References: <20230222024854.24876-1-peterlin@andestech.com>
Message-ID: <20230222122405.uoi2kavjdngy5q5a@orel>

On Wed, Feb 22, 2023 at 10:48:54AM +0800, Yu Chien Peter Lin wrote:
> We should also check if the return error code is greater than 0
> (SBI_SUCCESS), as this is an invalid error.
> 
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> ---
> Changes v1 -> v2:
> - Include Xiang's Reviewed-by
> - Drop the 'ret -> out_err' rename

The SBI_ENOTSUPP -> SBI_ERR_NOT_SUPPORTED rename was also dropped, but as
Anup suggests, we should probably do a full rework of that in another
series later.

Reviewed-by: Andrew Jones <ajones at ventanamicro.com>

Thanks,
drew

> ---
>  lib/sbi/sbi_ecall.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> index 27ce5d49..d0f01665 100644
> --- a/lib/sbi/sbi_ecall.c
> +++ b/lib/sbi/sbi_ecall.c
> @@ -120,7 +120,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
>  		trap.epc = regs->mepc;
>  		sbi_trap_redirect(regs, &trap);
>  	} else {
> -		if (ret < SBI_LAST_ERR) {
> +		if (ret < SBI_LAST_ERR || SBI_SUCCESS < ret) {
>  			sbi_printf("%s: Invalid error %d for ext=0x%lx "
>  				   "func=0x%lx\n", __func__, ret,
>  				   extension_id, func_id);
> -- 
> 2.34.1
> 


From peterlin at andestech.com  Thu Feb 23 02:50:40 2023
From: peterlin at andestech.com (Yu-Chien Peter Lin)
Date: Thu, 23 Feb 2023 10:50:40 +0000
Subject: [PATCH v2] lib: sbi: sbi_ecall: Check the range of SBI error
In-Reply-To: <20230222122405.uoi2kavjdngy5q5a@orel>
References: <20230222024854.24876-1-peterlin@andestech.com>
 <20230222122405.uoi2kavjdngy5q5a@orel>
Message-ID: <Y/dFAAKyBXCCegcb@APC323>

Hi Andrew,

On Wed, Feb 22, 2023 at 01:24:05PM +0100, Andrew Jones wrote:
> On Wed, Feb 22, 2023 at 10:48:54AM +0800, Yu Chien Peter Lin wrote:
> > We should also check if the return error code is greater than 0
> > (SBI_SUCCESS), as this is an invalid error.
> > 
> > Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> > Reviewed-by: Xiang W <wxjstz at 126.com>
> > ---
> > Changes v1 -> v2:
> > - Include Xiang's Reviewed-by
> > - Drop the 'ret -> out_err' rename
> 
> The SBI_ENOTSUPP -> SBI_ERR_NOT_SUPPORTED rename was also dropped, but as
> Anup suggests, we should probably do a full rework of that in another
> series later.
> 
> Reviewed-by: Andrew Jones <ajones at ventanamicro.com>
> 
> Thanks,
> drew

Agreed, thanks for the review.

Best regards,
Peter Lin

> 
> > ---
> >  lib/sbi/sbi_ecall.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> > index 27ce5d49..d0f01665 100644
> > --- a/lib/sbi/sbi_ecall.c
> > +++ b/lib/sbi/sbi_ecall.c
> > @@ -120,7 +120,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
> >  		trap.epc = regs->mepc;
> >  		sbi_trap_redirect(regs, &trap);
> >  	} else {
> > -		if (ret < SBI_LAST_ERR) {
> > +		if (ret < SBI_LAST_ERR || SBI_SUCCESS < ret) {
> >  			sbi_printf("%s: Invalid error %d for ext=0x%lx "
> >  				   "func=0x%lx\n", __func__, ret,
> >  				   extension_id, func_id);
> > -- 
> > 2.34.1
> > 


From bmeng at tinylab.org  Thu Feb 23 01:53:48 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Thu, 23 Feb 2023 17:53:48 +0800
Subject: [PATCH v2] lib: sbi_domain: Avoid overwriting coldboot hart's scratch->arg1
Message-ID: <20230223095348.1723380-1-bmeng@tinylab.org>

In sbi_domain_finalize(), when locating the coldboot hart's domain,
the coldboot hart's scratch->arg1 will be overwritten by the domain
configuration. scratch->arg1 holds the FDT address of the coldboot
hart, and in later boot process OpenSBI codes still read the FDT
here and there, which leads to a crash.

To fix this, we change fdt_get_address() to return root domain's
next_arg1 as the FDT pointer.

Resolves: https://github.com/riscv-software-src/opensbi/issues/281
Fixes: b1678af210dc ("lib: sbi: Add initial domain support")
Reported-by: Marouene Boubakri <marouene.boubakri at nxp.com>
Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

Changes in v2:
- Change fdt_get_address() to return root.next_arg1

 include/sbi_utils/fdt/fdt_helper.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
index 09f3095..e17b906 100644
--- a/include/sbi_utils/fdt/fdt_helper.h
+++ b/include/sbi_utils/fdt/fdt_helper.h
@@ -12,6 +12,7 @@
 
 #include <sbi/sbi_types.h>
 #include <sbi/sbi_scratch.h>
+#include <sbi/sbi_domain.h>
 
 struct fdt_match {
 	const char *compatible;
@@ -109,7 +110,7 @@ int fdt_parse_compat_addr(void *fdt, uint64_t *addr,
 
 static inline void *fdt_get_address(void)
 {
-	return sbi_scratch_thishart_arg1_ptr();
+	return (void *)root.next_arg1;
 }
 
 #endif /* __FDT_HELPER_H__ */
-- 
2.25.1



From bmeng at tinylab.org  Thu Feb 23 02:40:06 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Thu, 23 Feb 2023 18:40:06 +0800
Subject: [PATCH v3 1/4] lib: utils/fdt: Fix fdt_pmu.c header dependency
Message-ID: <20230223104009.1727401-1-bmeng@tinylab.org>

The code calls sbi_scratch_thishart_ptr() from sbi_scratch.h which
is not directly included. Fix such dependency.

Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

 lib/utils/fdt/fdt_pmu.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
index 4829f21..83301bb 100644
--- a/lib/utils/fdt/fdt_pmu.c
+++ b/lib/utils/fdt/fdt_pmu.c
@@ -12,6 +12,7 @@
 #include <sbi/sbi_hart.h>
 #include <sbi/sbi_error.h>
 #include <sbi/sbi_pmu.h>
+#include <sbi/sbi_scratch.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 
 #define FDT_PMU_HW_EVENT_MAX (SBI_PMU_HW_EVENT_MAX * 2)
-- 
2.25.1



From bmeng at tinylab.org  Thu Feb 23 02:40:07 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Thu, 23 Feb 2023 18:40:07 +0800
Subject: [PATCH v3 2/4] platform: generic/allwinner: Fix sun20i-d1.c header dependency
In-Reply-To: <20230223104009.1727401-1-bmeng@tinylab.org>
References: <20230223104009.1727401-1-bmeng@tinylab.org>
Message-ID: <20230223104009.1727401-2-bmeng@tinylab.org>

The code calls various macros from riscv_asm.h and sbi_scratch.h
which are not directly included. Fix such dependency.

Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

 platform/generic/allwinner/sun20i-d1.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
index 295c159..0f0a9f3 100644
--- a/platform/generic/allwinner/sun20i-d1.c
+++ b/platform/generic/allwinner/sun20i-d1.c
@@ -6,12 +6,14 @@
 
 #include <platform_override.h>
 #include <thead_c9xx.h>
+#include <sbi/riscv_asm.h>
 #include <sbi/riscv_io.h>
 #include <sbi/sbi_bitops.h>
 #include <sbi/sbi_ecall_interface.h>
 #include <sbi/sbi_error.h>
 #include <sbi/sbi_hsm.h>
 #include <sbi/sbi_pmu.h>
+#include <sbi/sbi_scratch.h>
 #include <sbi_utils/fdt/fdt_fixup.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 #include <sbi_utils/irqchip/fdt_irqchip_plic.h>
-- 
2.25.1



From bmeng at tinylab.org  Thu Feb 23 02:40:08 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Thu, 23 Feb 2023 18:40:08 +0800
Subject: [PATCH v3 3/4] platform: generic/andes: Fix ae350.c header dependency
In-Reply-To: <20230223104009.1727401-1-bmeng@tinylab.org>
References: <20230223104009.1727401-1-bmeng@tinylab.org>
Message-ID: <20230223104009.1727401-3-bmeng@tinylab.org>

The code calls various macros from riscv_asm.h which is not directly
included. Fix such dependency.

Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

 platform/generic/andes/ae350.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
index 89f5b74..01bd02d 100644
--- a/platform/generic/andes/ae350.c
+++ b/platform/generic/andes/ae350.c
@@ -11,6 +11,7 @@
 #include <sbi_utils/fdt/fdt_helper.h>
 #include <sbi_utils/fdt/fdt_fixup.h>
 #include <sbi_utils/sys/atcsmu.h>
+#include <sbi/riscv_asm.h>
 #include <sbi/sbi_bitops.h>
 #include <sbi/sbi_error.h>
 #include <sbi/sbi_hsm.h>
-- 
2.25.1



From bmeng at tinylab.org  Thu Feb 23 02:40:09 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Thu, 23 Feb 2023 18:40:09 +0800
Subject: [PATCH v3 4/4] include: fdt/fdt_helper: Change fdt_get_address() to return root.next_arg1
In-Reply-To: <20230223104009.1727401-1-bmeng@tinylab.org>
References: <20230223104009.1727401-1-bmeng@tinylab.org>
Message-ID: <20230223104009.1727401-4-bmeng@tinylab.org>

In sbi_domain_finalize(), when locating the coldboot hart's domain,
the coldboot hart's scratch->arg1 will be overwritten by the domain
configuration. However scratch->arg1 holds the FDT address of the
coldboot hart, and is still being accessed by fdt_get_address() in
later boot process. scratch->arg1 could then contain completely
garbage and lead to a crash.

To fix this, we change fdt_get_address() to return root domain's
next_arg1 as the FDT pointer.

Resolves: https://github.com/riscv-software-src/opensbi/issues/281
Fixes: b1678af210dc ("lib: sbi: Add initial domain support")
Reported-by: Marouene Boubakri <marouene.boubakri at nxp.com>
Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

Changes in v3:
- Drop <sbi/sbi_scratch.h> as it is no longer needed

Changes in v2:
- Change fdt_get_address() to return root.next_arg1

 include/sbi_utils/fdt/fdt_helper.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
index 09f3095..39d7f3a 100644
--- a/include/sbi_utils/fdt/fdt_helper.h
+++ b/include/sbi_utils/fdt/fdt_helper.h
@@ -11,7 +11,7 @@
 #define __FDT_HELPER_H__
 
 #include <sbi/sbi_types.h>
-#include <sbi/sbi_scratch.h>
+#include <sbi/sbi_domain.h>
 
 struct fdt_match {
 	const char *compatible;
@@ -109,7 +109,7 @@ int fdt_parse_compat_addr(void *fdt, uint64_t *addr,
 
 static inline void *fdt_get_address(void)
 {
-	return sbi_scratch_thishart_arg1_ptr();
+	return (void *)root.next_arg1;
 }
 
 #endif /* __FDT_HELPER_H__ */
-- 
2.25.1



From liushiwei at eswincomputing.com  Thu Feb 23 03:23:28 2023
From: liushiwei at eswincomputing.com (liushiwei)
Date: Thu, 23 Feb 2023 19:23:28 +0800
Subject: [PATCH 1/1] Add RISC-V TEE support
Message-ID: <003201d94779$4147a6d0$c3d6f470$@eswincomputing.com>

	Ok, thanks Atish, I'll try to join the group first.

Regards,
Liushiwei

-----????-----
???: Atish Patra [mailto:atishp at atishpatra.org] 
????: 2023?2?22? 4:23
???: liushiwei <liushiwei at eswincomputing.com>
??: Himanshu Chauhan <hchauhan at ventanamicro.com>; opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
??: Re: Re: [PATCH 1/1] Add RISC-V TEE support

On Thu, Feb 16, 2023 at 3:40 AM liushiwei <liushiwei at eswincomputing.com> wrote:
>
> Hi? Atish
>         I tried to email tech-prs at lists.riscv.org and sig-trusted-computing at lists.riscv.org, but both returned. My attempts to send a patch directly also returned.
>         Looking at this reason, does it seem like I have to join some 
> group before I can send an email?
>
> ?Connection to the remote recipient's server was denied for unknown reason.
> SMTP through SDN 37, SMTP: (Proxy)Host lists.riscv.org said 510 5.1.1 
> Your email address, liushiwei at eswincomputing.com, is not subscribed to 
> that group.  To subscribe, send an email to 
> sig-trusted-computing+subscribe at lists.riscv.org,
> or visit https://lists.riscv.org/g/sig-trusted-computing?
>

Yes. You have to subscribe to each group. Either you can join as an individual member or your organization if that's already a RVI member.

> Regards,
> Liushiwei
> -----????-----
> ???: Atish Patra [mailto:atishp at atishpatra.org]
> ????: 2023?2?8? 6:37
> ???: liushiwei <liushiwei at eswincomputing.com>
> ??: Himanshu Chauhan <hchauhan at ventanamicro.com>; 
> opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> ??: Re: Re: [PATCH 1/1] Add RISC-V TEE support
>
> On Sat, Jan 28, 2023 at 12:33 AM liushiwei <liushiwei at eswincomputing.com> wrote:
> >
> > Hi, Atish
> >         Thank you for your reply.
> >         I submitted my ideas to https://github.com/riscv-non-isa/riscv-sbi-doc/pull/106 as requested by other members of the email.
> >         You can preview them at https://github.com/liushiwei007/riscv-sbi-doc/blob/master/riscv-sbi.adoc .
> >         I'm not sure if I need to send an email to tech-prs at lists.riscv.org or sig-trusted-computing at lists.riscv.org after submitting to riscv-sbi-doc?
> >         If so, Is it to send the patch of opensbi I submitted before?
> >
>
> The Trusted computing SIG evaluates the overall merit and direction of any TEE related specifications.
> The PRS TG will focus more on the SBI specification related parts.
> I would recommend you to present your work to Trusted Computing SIG first so that everybody agrees on the overall direction for OP-TEE support in RISC-V.
>
> > Regards,
> > Liushiwei
> >
> >
> > -----????-----
> > ???: Atish Patra [mailto:atishp at atishpatra.org]
> > ????: 2023?1?25? 3:12
> > ???: liushiwei <liushiwei at eswincomputing.com>
> > ??: Himanshu Chauhan <hchauhan at ventanamicro.com>; 
> > opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> > ??: Re: ??: [PATCH 1/1] Add RISC-V TEE support
> >
> > On Wed, Jan 11, 2023 at 11:08 PM liushiwei <liushiwei at eswincomputing.com> wrote:
> > >
> > > Hi, Himanshu ,  These are my description:
> > >
> > > In my design, The entire linux space is called REE (Rich Execution 
> > > Environment), and TEE OS includes its user state called TEE (Trusted execution environment).
> > > adding TEE functionality to opensbi requires two configuration items.
> > > For example:
> > >    add CONFIG_SBI_ECALL_TEE=y in platform/generic/configs/defconfig file.
> > >    add CONFIG_TEE_LOAD_ADDR=0x27C000000 platform/generic/objects.mk.
> > >    The value of CONFIG_TEE_LOAD_ADDR depends on the actual memory layout,
> > >    It's a physical address.
> > >
> > > When TEE is configured, opensbi adds the following functionality:
> > > 1. Boot TEE OS.
> > >    If TEE is enabled, tee_os_init() is called before entering
> > >    sbi_hart_switch_mode(). tee_os_init saves the current context,
> > >    sets a new trap stack address, and runs to CONFIG_TEE_LOAD_ADDR as configured
> > >    to complete TEE OS initialization. TEE OS returns via ecall, Go to opensbi
> > >    sbi_ecall_tee_handler and use the characteristic value RETURN_ENTRY_DONE
> > >    to indicate the return after the TEE OS completes booting. Check whether the
> > >    TEE OS boot successfully according to the parameter. If fails,
> > >    the system enters wfi and terminates the startup process of opensbi. If successful,
> > >    it returns REE to switch into TEE's vector. then trap stack memory is restored,
> > >    the context is restored, the tee_os_init call is returned,
> > >    and the rest of the process is performed.
> > > 2. REE switches to TEE.
> > >    When the TEE OS boot successfully, it returns an entry vector for REE into the TEE.
> > >    It represents various entry points into TEE OS and is stored in opensbi's global variables.
> > >    It is the optee_vectors_t structure, which contains nine entry cases:
> > >       1. yield_smc_entry;
> > >       2. fast_smc_entry;
> > >       3. cpu_on_entry;
> > >       4. cpu_off_entry;
> > >       5. cpu_resume_entry;
> > >       6. cpu_suspend_entry;
> > >       7. fiq_entry;
> > >       8. system_off_entry;
> > >       9. system_reset_entry;
> > >    yield_smc_entry means that this function entry TEE will start the thread function
> > >    and enter the user state of TEE. It may also switch back to REE with RPC function,
> > >    and then return to TEE after REE completes the corresponding function.
> > >    For the REE process that sent you this call, it may cause sleep.
> > >
> > >    fast_smc_entry indicates that this is a quick function that returns after
> > >    the TEE OS does something, and that it does not cause the caller to sleep.
> > >
> > >    yield_smc_entry and fast_smc_entry return opensbi use eigenvalue RETURN_CALL_DONE.
> > >    for yield_smc_entry, whether the call returns or the RPC returns is decided by linux.
> > >
> > >    cpu_on_entry/cpu_off_entry/cpu_resume_entry/cpu_suspend_entry and
> > >    fiq_entry/system_off_entry/system_reset_entry they are not implemented currently.
> > >
> > > 3. TEE switches to REE.
> > >    opensbi needs to save the context when REE enters the TEE,
> > >    and restore the context when it returns from the TEE.
> > >
> > > 4. TEE/REE Request a special function.
> > >    We have some specific functions, like get hartid from TEE,
> > >    it need save and restore the TEE context.
> > >
> > > Data structure.
> > > opensbi adds the sbi_save_context declaration
> > >    struct sbi_save_context {
> > >       struct sbi_trap_regs regs;
> > >       unsigned long sepc;
> > >       unsigned long satp;
> > >       unsigned long sstatus;
> > >       unsigned long sie;
> > >       unsigned long stvec;
> > >       unsigned long sscratch;
> > >       unsigned long scounteren;
> > >       unsigned long scause;
> > >       unsigned long stval;
> > >       unsigned long sip;
> > >    };
> > >    sbi_save_context include sbi_trap_regs and S mode csr. Used to 
> > > hold the context of TEE or REE
> > >
> > >    typedef struct optee_vectors {
> > >       optee_vector_isn_t yield_smc_entry;
> > >       optee_vector_isn_t fast_smc_entry;
> > >       optee_vector_isn_t cpu_on_entry;
> > >       optee_vector_isn_t cpu_off_entry;
> > >       optee_vector_isn_t cpu_resume_entry;
> > >       optee_vector_isn_t cpu_suspend_entry;
> > >       optee_vector_isn_t fiq_entry;
> > >       optee_vector_isn_t system_off_entry;
> > >       optee_vector_isn_t system_reset_entry;
> > >    } optee_vectors_t;
> > >    optee_vectors represent the various entry points into TEE OS.
> > >
> > > Variables defined by opensbi
> > >    optee_vectors_t *optee_vector_table;
> > >    struct sbi_save_context nsec_cpu_context[OPTEED_CORE_COUNT];
> > >    struct sbi_save_context sec_cpu_context[OPTEED_CORE_COUNT];
> > >    struct sbi_trap_regs cpu_start_context[OPTEED_CORE_COUNT];
> > >    typedef ulong tee_tmp_trap_stack[1024];
> > >    static tee_tmp_trap_stack tmp_stack[OPTEED_CORE_COUNT];
> > >
> > >    optee_vector_table value is assigned after the TEE OS boot succeeds.
> > >    nsec_cpu_context and sec_cpu_context is to save the context of TEE and REE,
> > >    cpu_start_context save the context of opensbi before tee_os_init enter TEE OS,
> > >    tee_tmp_stack is the trap stack when TEE OS return tee_os_init.
> > >
> > > Under the current design, REE does not enable interrupts when 
> > > entering TEE, and the entire TEE, including opensbi, is the process context of linux.
> > > TEE processing must be brief and quick.
> > >
> >
> > Great. Thanks. Your PR should include more details like this with proper FID allocation and description.
> > In order to make it a standard RISC-V SBI extension, you need to 
> > present this in tech-prs mailing list
> > (https://lists.riscv.org/g/tech-prs)
> > and get it approved within the group. Here is the policy document[1]
> >
> > https://docs.google.com/document/d/1bQGHU-wD4uN4mU07oH9adpXbH51NqUbF
> > xc
> > L8qTTIqhA/edit
> >
> > I think it would be good if you can present your work in trusted computing SIG (https://lists.riscv.org/g/sig-trusted-computing) as well.
> >
> > >
> > > Regards,
> > > liushiwei
> > > -----????-----
> > > ???: Himanshu Chauhan [mailto:hchauhan at ventanamicro.com]
> > > ????: 2023?1?11? 23:39
> > > ???: liushiwei <liushiwei at eswincomputing.com>
> > > ??: opensbi at lists.infradead.org; chenchaokai at eswincomputing.com
> > > ??: Re: ??: [PATCH 1/1] Add RISC-V TEE support
> > >
> > > On Wed, Jan 11, 2023 at 08:27:59PM +0800, liushiwei wrote:
> > > > Do you mean hardware? Our hardware design referred to arm's 
> > > > trustzone technology. optee os is a software solution using arm 
> > > > trustzone hardware, which mainly includes REE(linux), TEE(optee 
> > > > os), ATF(ARM Trusted firmware), and then our software also 
> > > > developed these three parts. opensbi is similar to ATF. whether if this is what you want?
> > > > The current committed code is not hardware-dependent, but just 
> > > > continues the idea of this workaround, and we may commit hardware-dependent code later.
> > > >
> > > No, I meant the software specification.
> > >
> > > > -----????-----
> > > > ???: hchauhan at ventanamicro.com 
> > > > [mailto:hchauhan at ventanamicro.com]
> > > > ????: 2023?1?11? 20:03
> > > > ???: 'liushiwei' <liushiwei at eswincomputing.com>; opensbi at lists.infradead.
> > > > org
> > > > ??: chenchaokai at eswincomputing.com
> > > > ??: RE: [PATCH 1/1] Add RISC-V TEE support
> > > >
> > > > -----Original Message-----
> > > > > From: opensbi <opensbi-bounces at lists.infradead.org> On Behalf 
> > > > > Of liushiwei
> > > > > Sent: 11 January 2023 07:32
> > > > > To: opensbi at lists.infradead.org
> > > > > Cc: chenchaokai at eswincomputing.com; liushiwei
> > > > <liushiwei at eswincomputing.com>
> > > > > Subject: [PATCH 1/1] Add RISC-V TEE support
> > > >
> > > > >RISC-V Trusted Executable Environment security software 
> > > > >includes linux,
> > > > opensbi, and OP-TEE OS. linux is the non-secure domain, and 
> > > > OP-TEE OS is the secure domain. At boot time, opensbi boots OP->TEE OS and then starts linux.
> > > > At runtime, opensbi acts as a secure monitor, responsible for 
> > > > context saving and restoring when switching between linux and OP-TEE OS.
> > > > >TEE function is off by default, when using configuration is 
> > > > >added in the
> > > > config and objects file, such as 
> > > > platform/generic/configs/defconfig
> > > > add CONFIG_SBI_ECALL_TEE = y, In the 
> > > > >platform/generic/objects.mk add CONFIG_TEE_LOAD_ADDR = 
> > > > 0x27c000000, CONFIG_TEE_LOAD_ADDR is the starting address of the OP-TEE OS.
> > > >
> > > > Hi Liushiwei,
> > > >
> > > > Was there any formal specification or draft for this? Could you 
> > > > please point me to the draft or specification?
> > > >
> > > > Regards
> > > > Himanshu
> > > >
> > > > --
> > > > opensbi mailing list
> > > > opensbi at lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/opensbi
> > > >
> > >
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
> >
> >
> >
> > --
> > Regards,
> > Atish
> >
>
>
> --
> Regards,
> Atish
>


--
Regards,
Atish



From bmeng at tinylab.org  Thu Feb 23 06:52:38 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Thu, 23 Feb 2023 22:52:38 +0800
Subject: [RFC PATCH] Kconfig: Add an option for debugging OpenSBI
Message-ID: <20230223145238.1750383-1-bmeng@tinylab.org>

Add a new Kconfig option CONFIG_OPENSBI_DEBUG to prevent compiler
optimizations using -O2.

Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

 Kconfig  | 7 +++++++
 Makefile | 7 ++++++-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/Kconfig b/Kconfig
index acfc138..123ea66 100644
--- a/Kconfig
+++ b/Kconfig
@@ -23,3 +23,10 @@ source "$(OPENSBI_SRC_DIR)/lib/sbi/Kconfig"
 source "$(OPENSBI_SRC_DIR)/lib/utils/Kconfig"
 
 source "$(OPENSBI_SRC_DIR)/firmware/Kconfig"
+
+menu "General Setup"
+
+config OPENSBI_DEBUG
+	bool "Disable compiler optimizations for debugging"
+
+endmenu
diff --git a/Makefile b/Makefile
index b20404f..b4ed548 100644
--- a/Makefile
+++ b/Makefile
@@ -331,7 +331,12 @@ GENFLAGS	+=	$(libsbiutils-genflags-y)
 GENFLAGS	+=	$(platform-genflags-y)
 GENFLAGS	+=	$(firmware-genflags-y)
 
-CFLAGS		=	-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2
+CFLAGS		=	-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing
+ifeq ($(CONFIG_OPENSBI_DEBUG),y)
+CFLAGS		+=	-O0
+else
+CFLAGS		+=	-O2
+endif
 CFLAGS		+=	-fno-omit-frame-pointer -fno-optimize-sibling-calls -mstrict-align
 # enable -m(no-)save-restore option by CC_SUPPORT_SAVE_RESTORE
 ifeq ($(CC_SUPPORT_SAVE_RESTORE),y)
-- 
2.25.1



From bmeng at tinylab.org  Thu Feb 23 20:28:25 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Fri, 24 Feb 2023 12:28:25 +0800
Subject: [PATCH 1/3] lib: sbi/sbi_domain: cosmetic style fixes
Message-ID: <20230224042827.1768931-1-bmeng@tinylab.org>

Minor updates to the comments for language and style fixes.

Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

 lib/sbi/sbi_domain.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
index dc825b0..d2f58a2 100644
--- a/lib/sbi/sbi_domain.c
+++ b/lib/sbi/sbi_domain.c
@@ -337,7 +337,7 @@ static int sanitize_domain(const struct sbi_platform *plat,
 	/*
 	 * Check next mode
 	 *
-	 * We only allow next mode to be S-mode or U-mode.so that we can
+	 * We only allow next mode to be S-mode or U-mode, so that we can
 	 * protect M-mode context and enforce checks on memory accesses.
 	 */
 	if (dom->next_mode != PRV_S &&
@@ -347,7 +347,7 @@ static int sanitize_domain(const struct sbi_platform *plat,
 		return SBI_EINVAL;
 	}
 
-	/* Check next address and next mode*/
+	/* Check next address and next mode */
 	if (!sbi_domain_check_addr(dom, dom->next_addr, dom->next_mode,
 				   SBI_DOMAIN_EXECUTE)) {
 		sbi_printf("%s: %s next booting stage address 0x%lx can't "
-- 
2.25.1



From bmeng at tinylab.org  Thu Feb 23 20:28:26 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Fri, 24 Feb 2023 12:28:26 +0800
Subject: [PATCH 2/3] lib: utils/fdt/fdt_domain: Simplify region access permission check
In-Reply-To: <20230224042827.1768931-1-bmeng@tinylab.org>
References: <20230224042827.1768931-1-bmeng@tinylab.org>
Message-ID: <20230224042827.1768931-2-bmeng@tinylab.org>

The region access permission check in __fdt_parse_region() can be
simplified as masking SBI_DOMAIN_MEMREGION_{M,SU}_ACCESS_MASK is
enough.

While we are here, update the confusing comments to match the codes.

Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

 lib/utils/fdt/fdt_domain.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/lib/utils/fdt/fdt_domain.c b/lib/utils/fdt/fdt_domain.c
index 2b51a8e..bc30010 100644
--- a/lib/utils/fdt/fdt_domain.c
+++ b/lib/utils/fdt/fdt_domain.c
@@ -244,13 +244,11 @@ static int __fdt_parse_region(void *fdt, int domain_offset,
 	 * access permissions. M-mode regions can only be part of
 	 * root domain.
 	 *
-	 * SU permission bits can't be all zeroes and M-mode permission
-	 * bits must be all set.
+	 * SU permission bits can't be all zeroes when M-mode permission
+	 * bits have at least one bit set.
 	 */
-	if (!((region_access & SBI_DOMAIN_MEMREGION_SU_ACCESS_MASK)
-	     & SBI_DOMAIN_MEMREGION_SU_RWX)
-	    && ((region_access & SBI_DOMAIN_MEMREGION_M_ACCESS_MASK)
-		& SBI_DOMAIN_MEMREGION_M_RWX))
+	if (!(region_access & SBI_DOMAIN_MEMREGION_SU_ACCESS_MASK)
+	    && (region_access & SBI_DOMAIN_MEMREGION_M_ACCESS_MASK))
 		return SBI_EINVAL;
 
 	/* Find next region of the domain */
-- 
2.25.1



From bmeng at tinylab.org  Thu Feb 23 20:28:27 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Fri, 24 Feb 2023 12:28:27 +0800
Subject: [PATCH 3/3] docs: domain_support: Update the DT example
In-Reply-To: <20230224042827.1768931-1-bmeng@tinylab.org>
References: <20230224042827.1768931-1-bmeng@tinylab.org>
Message-ID: <20230224042827.1768931-3-bmeng@tinylab.org>

commit 3e2f573e707e ("lib: utils: Disallow non-root domains from adding M-mode regions")
added access permission check in __fdt_parse_region(). With the
existing DT example in the doc OpenSBI won't boot anymore.

Let's update the DT example so that it can work out of the box.

Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

 docs/domain_support.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/domain_support.md b/docs/domain_support.md
index 4ccbcff..2a9ca3d 100644
--- a/docs/domain_support.md
+++ b/docs/domain_support.md
@@ -254,7 +254,7 @@ be done:
             tdomain: trusted-domain {
                 compatible = "opensbi,domain,instance";
                 possible-harts = <&cpu0>;
-                regions = <&tmem 0x7>, <&tuart 0x7>;
+                regions = <&tmem 0x3f>, <&tuart 0x3f>;
                 boot-hart = <&cpu0>;
                 next-arg1 = <0x0 0x0>;
                 next-addr = <0x0 0x80100000>;
@@ -265,7 +265,7 @@ be done:
             udomain: untrusted-domain {
                 compatible = "opensbi,domain,instance";
                 possible-harts = <&cpu1 &cpu2 &cpu3 &cpu4>;
-                regions = <&tmem 0x0>, <&tuart 0x0>, <&allmem 0x7>;
+                regions = <&tmem 0x0>, <&tuart 0x0>, <&allmem 0x3f>;
             };
         };
     };
-- 
2.25.1



From anup at brainfault.org  Thu Feb 23 20:48:29 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 24 Feb 2023 10:18:29 +0530
Subject: [PATCH v2] lib: sbi_domain: Avoid overwriting coldboot hart's scratch->arg1
In-Reply-To: <20230223095348.1723380-1-bmeng@tinylab.org>
References: <20230223095348.1723380-1-bmeng@tinylab.org>
Message-ID: <CAAhSdy3JXUPXUO4dcwstXWSpXS3YLKe=6FjvRA6J3wCPj4BW1w@mail.gmail.com>

On Thu, Feb 23, 2023 at 3:24 PM Bin Meng <bmeng at tinylab.org> wrote:
>
> In sbi_domain_finalize(), when locating the coldboot hart's domain,
> the coldboot hart's scratch->arg1 will be overwritten by the domain
> configuration. scratch->arg1 holds the FDT address of the coldboot
> hart, and in later boot process OpenSBI codes still read the FDT
> here and there, which leads to a crash.
>
> To fix this, we change fdt_get_address() to return root domain's
> next_arg1 as the FDT pointer.
>
> Resolves: https://github.com/riscv-software-src/opensbi/issues/281
> Fixes: b1678af210dc ("lib: sbi: Add initial domain support")
> Reported-by: Marouene Boubakri <marouene.boubakri at nxp.com>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>
> Changes in v2:
> - Change fdt_get_address() to return root.next_arg1
>
>  include/sbi_utils/fdt/fdt_helper.h | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
> index 09f3095..e17b906 100644
> --- a/include/sbi_utils/fdt/fdt_helper.h
> +++ b/include/sbi_utils/fdt/fdt_helper.h
> @@ -12,6 +12,7 @@
>
>  #include <sbi/sbi_types.h>
>  #include <sbi/sbi_scratch.h>
> +#include <sbi/sbi_domain.h>
>
>  struct fdt_match {
>         const char *compatible;
> @@ -109,7 +110,7 @@ int fdt_parse_compat_addr(void *fdt, uint64_t *addr,
>
>  static inline void *fdt_get_address(void)
>  {
> -       return sbi_scratch_thishart_arg1_ptr();
> +       return (void *)root.next_arg1;
>  }
>
>  #endif /* __FDT_HELPER_H__ */
> --
> 2.25.1
>


From anup at brainfault.org  Thu Feb 23 20:49:38 2023
From: anup at brainfault.org (Anup Patel)
Date: Fri, 24 Feb 2023 10:19:38 +0530
Subject: [RFC PATCH] Kconfig: Add an option for debugging OpenSBI
In-Reply-To: <20230223145238.1750383-1-bmeng@tinylab.org>
References: <20230223145238.1750383-1-bmeng@tinylab.org>
Message-ID: <CAAhSdy2pDpYtYgydyaWyxeQDQD+VgmDM1nfwD23xAKOBBWvEKw@mail.gmail.com>

On Thu, Feb 23, 2023 at 8:23 PM Bin Meng <bmeng at tinylab.org> wrote:
>
> Add a new Kconfig option CONFIG_OPENSBI_DEBUG to prevent compiler
> optimizations using -O2.
>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>
> ---
>
>  Kconfig  | 7 +++++++
>  Makefile | 7 ++++++-
>  2 files changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/Kconfig b/Kconfig
> index acfc138..123ea66 100644
> --- a/Kconfig
> +++ b/Kconfig
> @@ -23,3 +23,10 @@ source "$(OPENSBI_SRC_DIR)/lib/sbi/Kconfig"
>  source "$(OPENSBI_SRC_DIR)/lib/utils/Kconfig"
>
>  source "$(OPENSBI_SRC_DIR)/firmware/Kconfig"
> +
> +menu "General Setup"
> +
> +config OPENSBI_DEBUG
> +       bool "Disable compiler optimizations for debugging"
> +
> +endmenu\

Rather than adding a Kconfig option, I would suggest having a
simple make command-line option. For example, "make DEBUG=1"

Regards,
Anup

> diff --git a/Makefile b/Makefile
> index b20404f..b4ed548 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -331,7 +331,12 @@ GENFLAGS   +=      $(libsbiutils-genflags-y)
>  GENFLAGS       +=      $(platform-genflags-y)
>  GENFLAGS       +=      $(firmware-genflags-y)
>
> -CFLAGS         =       -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2
> +CFLAGS         =       -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing
> +ifeq ($(CONFIG_OPENSBI_DEBUG),y)
> +CFLAGS         +=      -O0
> +else
> +CFLAGS         +=      -O2
> +endif
>  CFLAGS         +=      -fno-omit-frame-pointer -fno-optimize-sibling-calls -mstrict-align
>  # enable -m(no-)save-restore option by CC_SUPPORT_SAVE_RESTORE
>  ifeq ($(CC_SUPPORT_SAVE_RESTORE),y)
> --
> 2.25.1
>


From minda.chen at starfivetech.com  Fri Feb 24 01:57:46 2023
From: minda.chen at starfivetech.com (Minda Chen)
Date: Fri, 24 Feb 2023 17:57:46 +0800
Subject: [PATCH 2/2] config: starfive: add PMIC power ops in JH7110 visionfive2 board
In-Reply-To: <20230224095746.40466-1-minda.chen@starfivetech.com>
References: <20230224095746.40466-1-minda.chen@starfivetech.com>
Message-ID: <20230224095746.40466-3-minda.chen@starfivetech.com>

add reboot and poweroff support. The whole reboot and shutdown
pm op includes shutdown jh7110 pmu device power domain
and access on board pmic register through I2C.

Signed-off-by: Minda Chen <minda.chen at starfivetech.com>
---
 platform/generic/configs/defconfig |   1 +
 platform/generic/starfive/jh7110.c | 272 +++++++++++++++++++++++++++++
 2 files changed, 273 insertions(+)

diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
index 2ad953d..1fece49 100644
--- a/platform/generic/configs/defconfig
+++ b/platform/generic/configs/defconfig
@@ -9,6 +9,7 @@ CONFIG_FDT_GPIO_SIFIVE=y
 CONFIG_FDT_GPIO_STARFIVE=y
 CONFIG_FDT_I2C=y
 CONFIG_FDT_I2C_SIFIVE=y
+CONFIG_FDT_I2C_STARFIVE=y
 CONFIG_FDT_IPI=y
 CONFIG_FDT_IPI_MSWI=y
 CONFIG_FDT_IPI_PLICSW=y
diff --git a/platform/generic/starfive/jh7110.c b/platform/generic/starfive/jh7110.c
index c665658..31c6bd3 100644
--- a/platform/generic/starfive/jh7110.c
+++ b/platform/generic/starfive/jh7110.c
@@ -5,14 +5,285 @@
  *
  * Authors:
  *   Wei Liang Lim <weiliang.lim at starfivetech.com>
+ *   Minda Chen <minda.chen at starfivetech.com>
  */
 
 #include <libfdt.h>
 #include <platform_override.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_bitops.h>
+#include <sbi/sbi_system.h>
+#include <sbi/sbi_console.h>
+#include <sbi/sbi_timer.h>
+#include <sbi/riscv_io.h>
 #include <sbi_utils/fdt/fdt_helper.h>
+#include <sbi_utils/reset/fdt_reset.h>
+#include <sbi_utils/i2c/fdt_i2c.h>
 
+struct pmic {
+	struct i2c_adapter *adapter;
+	u32 dev_addr;
+	const char *compatible;
+};
+
+struct jh7110 {
+	u64 pmu_reg_base;
+	u64 clk_reg_base;
+	u32 i2c_index;
+};
+
+static struct pmic pmic_inst;
+static struct jh7110 jh7110_inst;
 static u32 selected_hartid = -1;
 
+/* PMU register define */
+#define HW_EVENT_TURN_ON_MASK		0x04
+#define HW_EVENT_TURN_OFF_MASK		0x08
+#define SW_TURN_ON_POWER_MODE		0x0C
+#define SW_TURN_OFF_POWER_MODE		0x10
+#define SW_ENCOURAGE			0x44
+#define PMU_INT_MASK			0x48
+#define PCH_BYPASS			0x4C
+#define PCH_PSTATE			0x50
+#define PCH_TIMEOUT			0x54
+#define LP_TIMEOUT			0x58
+#define HW_TURN_ON_MODE			0x5C
+#define CURR_POWER_MODE			0x80
+#define PMU_EVENT_STATUS		0x88
+#define PMU_INT_STATUS			0x8C
+
+/* sw encourage cfg */
+#define SW_MODE_ENCOURAGE_EN_LO		0x05
+#define SW_MODE_ENCOURAGE_EN_HI		0x50
+#define SW_MODE_ENCOURAGE_DIS_LO	0x0A
+#define SW_MODE_ENCOURAGE_DIS_HI	0xA0
+#define SW_MODE_ENCOURAGE_ON		0xFF
+
+#define DEVICE_PD_MASK			0xfc
+#define SYSTOP_CPU_PD_MASK		0x3
+
+#define TIMEOUT_COUNT			100000
+#define AXP15060_POWER_REG		0x32
+#define AXP15060_POWER_OFF_BIT		BIT(7)
+#define AXP15060_RESET_BIT		BIT(6)
+
+#define I2C_APB_CLK_OFFSET		0x228
+#define I2C_APB_CLK_ENABLE_BIT		BIT(31)
+
+static int pm_system_reset_check(u32 type, u32 reason)
+{
+	switch (type) {
+	case SBI_SRST_RESET_TYPE_SHUTDOWN:
+		return 1;
+	case SBI_SRST_RESET_TYPE_COLD_REBOOT:
+		return 255;
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+static int wait_pmu_pd_state(u32 mask)
+{
+	int count = 0;
+	unsigned long addr = jh7110_inst.pmu_reg_base;
+	u32 val;
+
+	do {
+		val = readl((void *)(addr + CURR_POWER_MODE));
+		if (val == mask)
+			return 0;
+
+		sbi_timer_udelay(2);
+		count += 1;
+		if (count == TIMEOUT_COUNT)
+			return SBI_ETIMEDOUT;
+	} while (1);
+}
+
+static int shutdown_device_power_domain(void)
+{
+	unsigned long addr = jh7110_inst.pmu_reg_base;
+	u32 curr_mode;
+	int ret = 0;
+
+	curr_mode = readl((void *)(addr + CURR_POWER_MODE));
+	curr_mode &= DEVICE_PD_MASK;
+
+	if (curr_mode) {
+		writel(curr_mode, (void *)(addr + SW_TURN_OFF_POWER_MODE));
+		writel(SW_MODE_ENCOURAGE_ON, (void *)(addr + SW_ENCOURAGE));
+		writel(SW_MODE_ENCOURAGE_DIS_LO, (void *)(addr + SW_ENCOURAGE));
+		writel(SW_MODE_ENCOURAGE_DIS_HI, (void *)(addr + SW_ENCOURAGE));
+		ret = wait_pmu_pd_state(SYSTOP_CPU_PD_MASK);
+		if (ret)
+			sbi_printf("%s shutdown device power %x error\n",
+				   __func__, curr_mode);
+	}
+	return ret;
+}
+
+static void pmic_ops(struct pmic *pmic, int type)
+{
+	int ret = 0;
+	u8 val;
+
+	ret = shutdown_device_power_domain();
+
+	if (ret)
+		return;
+
+	if (!sbi_strcmp("stf,axp15060-regulator", pmic->compatible)) {
+		ret = i2c_adapter_reg_read(pmic->adapter, pmic->dev_addr,
+					   AXP15060_POWER_REG, &val);
+
+		if (ret) {
+			sbi_printf("%s: cannot read pmic power register\n",
+				   __func__);
+			return;
+		}
+
+		val |= AXP15060_POWER_OFF_BIT;
+		if (type == SBI_SRST_RESET_TYPE_SHUTDOWN)
+			val |= AXP15060_POWER_OFF_BIT;
+		else
+			val |= AXP15060_RESET_BIT;
+
+		ret = i2c_adapter_reg_write(pmic->adapter, pmic->dev_addr,
+					    AXP15060_POWER_REG, val);
+		if (ret)
+			sbi_printf("%s: cannot write pmic power register\n",
+				   __func__);
+	}
+}
+
+static void pmic_i2c_clk_enable(void)
+{
+	u64 clock_base;
+	unsigned int val;
+
+	clock_base = jh7110_inst.clk_reg_base +
+		I2C_APB_CLK_OFFSET +
+		(jh7110_inst.i2c_index << 2);
+
+	val = readl((void *)clock_base);
+
+	if (!val)
+		writel(I2C_APB_CLK_ENABLE_BIT, (void *)(clock_base));
+}
+
+static void pm_system_reset(u32 type, u32 reason)
+{
+	if (pmic_inst.adapter) {
+		switch (type) {
+		case SBI_SRST_RESET_TYPE_SHUTDOWN:
+		case SBI_SRST_RESET_TYPE_COLD_REBOOT:
+			/* i2c clk may be disabled by kernel driver */
+			pmic_i2c_clk_enable();
+			pmic_ops(&pmic_inst, type);
+			break;
+		default:
+			break;
+		}
+	}
+
+	sbi_hart_hang();
+}
+
+static struct sbi_system_reset_device pm_reset = {
+	.name = "pm-reset",
+	.system_reset_check = pm_system_reset_check,
+	.system_reset = pm_system_reset
+};
+
+static int pm_reset_init(void *fdt, int nodeoff,
+			 const struct fdt_match *match)
+{
+	int rc;
+	int i2c_bus;
+	struct i2c_adapter *adapter;
+	u64 addr;
+
+	rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
+	if (rc)
+		return rc;
+
+	pmic_inst.dev_addr = addr;
+	pmic_inst.compatible = match->compatible;
+
+	i2c_bus = fdt_parent_offset(fdt, nodeoff);
+	if (i2c_bus < 0)
+		return i2c_bus;
+
+	/* i2c adapter get */
+	rc = fdt_i2c_adapter_get(fdt, i2c_bus, &adapter);
+	if (rc)
+		return rc;
+
+	pmic_inst.adapter = adapter;
+
+	sbi_system_reset_add_device(&pm_reset);
+
+	return 0;
+}
+
+static const struct fdt_match pm_reset_match[] = {
+	{ .compatible = "stf,axp15060-regulator", .data = (void *)true },
+	{ },
+};
+
+static struct fdt_reset fdt_reset_pmic = {
+	.match_table = pm_reset_match,
+	.init = pm_reset_init,
+};
+
+static int starfive_jh7110_inst_init(void *fdt)
+{
+	int noff, rc = 0;
+	const char *name;
+	u64 addr;
+
+	noff = fdt_node_offset_by_compatible(fdt, -1, "starfive,jh7110-pmu");
+	if (-1 < noff) {
+		rc = fdt_get_node_addr_size(fdt, noff, 0, &addr, NULL);
+		if (rc)
+			goto err;
+		jh7110_inst.pmu_reg_base = addr;
+	}
+
+	noff = fdt_node_offset_by_compatible(fdt, -1, "starfive,jh7110-clkgen");
+	if (-1 < noff) {
+		rc = fdt_get_node_addr_size(fdt, noff, 0, &addr, NULL);
+		if (rc)
+			goto err;
+		jh7110_inst.clk_reg_base = addr;
+	}
+
+	if (pmic_inst.adapter) {
+		name = fdt_get_name(fdt, pmic_inst.adapter->id, NULL);
+		if (!sbi_strncmp(name, "i2c", 3))
+			jh7110_inst.i2c_index = name[3] - '0';
+		else
+			rc = SBI_EINVAL;
+	}
+err:
+	return rc;
+}
+
+static int starfive_jh7110_final_init(bool cold_boot,
+				      const struct fdt_match *match)
+{
+	void *fdt = fdt_get_address();
+
+	if (cold_boot) {
+		fdt_reset_driver_init(fdt, &fdt_reset_pmic);
+		return starfive_jh7110_inst_init(fdt);
+	}
+
+	return 0;
+}
+
 static bool starfive_jh7110_cold_boot_allowed(u32 hartid,
 				   const struct fdt_match *match)
 {
@@ -44,4 +315,5 @@ const struct platform_override starfive_jh7110 = {
 	.match_table = starfive_jh7110_match,
 	.cold_boot_allowed = starfive_jh7110_cold_boot_allowed,
 	.fw_init = starfive_jh7110_fw_init,
+	.final_init = starfive_jh7110_final_init,
 };
-- 
2.17.1



From minda.chen at starfivetech.com  Fri Feb 24 01:57:44 2023
From: minda.chen at starfivetech.com (Minda Chen)
Date: Fri, 24 Feb 2023 17:57:44 +0800
Subject: [PATCH 0/2] Add StarFive JH7110 SoC shutdown and reboot ops
Message-ID: <20230224095746.40466-1-minda.chen@starfivetech.com>

Patch2 needs to apply after patch in [1].

[1] https://lists.infradead.org/pipermail/opensbi/2023-February/004474.html

Test board is JH7110 visionfive v2 board. shutdown and reboot function
require to access axp15060 PMIC through i2c.

patch1 is i2c driver. i2c ip is designware. dw_i2c.c includes designware
i2c minimal driver which can be used by other platform.
fdt_i2c_starfive.c contains StarFive JH7110 platform and fdt information.

patch2 is JH7110 visionfive2 board reboot and shutdown ops.
The i2c clk will be disabled by kernel i2c driver in power off progress
(suspend ops).

Before access PMIC control bits to reboot or shutdown. Require
to set off device power domain in JH7110 SoC power management unit.

For more JH7110 and visionFive v2 information and support,
you can visit RVspace wiki[2].

[2] https://wiki.rvspace.org/

Minda Chen (2):
  lib: utils/i2c: Add minimal StarFive jh7110 I2C driver
  config: starfive: add PMIC power ops in JH7110 visionfive2 board

 include/sbi_utils/i2c/dw_i2c.h     |  21 +++
 lib/utils/i2c/Kconfig              |   8 +
 lib/utils/i2c/dw_i2c.c             | 190 ++++++++++++++++++++
 lib/utils/i2c/fdt_i2c_starfive.c   |  63 +++++++
 lib/utils/i2c/objects.mk           |   5 +
 platform/generic/configs/defconfig |   1 +
 platform/generic/starfive/jh7110.c | 272 +++++++++++++++++++++++++++++
 7 files changed, 560 insertions(+)
 create mode 100644 include/sbi_utils/i2c/dw_i2c.h
 create mode 100644 lib/utils/i2c/dw_i2c.c
 create mode 100644 lib/utils/i2c/fdt_i2c_starfive.c

-- 
2.17.1



From minda.chen at starfivetech.com  Fri Feb 24 01:57:45 2023
From: minda.chen at starfivetech.com (Minda Chen)
Date: Fri, 24 Feb 2023 17:57:45 +0800
Subject: [PATCH 1/2] lib: utils/i2c: Add minimal StarFive jh7110 I2C driver
In-Reply-To: <20230224095746.40466-1-minda.chen@starfivetech.com>
References: <20230224095746.40466-1-minda.chen@starfivetech.com>
Message-ID: <20230224095746.40466-2-minda.chen@starfivetech.com>

Starfive JH7110 I2C IP is synopsys designware.
Minimum StarFIve I2C driver to read/send bytes over I2C bus.

This allows querying information and perform operation of onboard PMIC,
as well as power-off and reset.

Signed-off-by: Minda Chen <minda.chen at starfivetech.com>
---
 include/sbi_utils/i2c/dw_i2c.h   |  21 ++++
 lib/utils/i2c/Kconfig            |   8 ++
 lib/utils/i2c/dw_i2c.c           | 190 +++++++++++++++++++++++++++++++
 lib/utils/i2c/fdt_i2c_starfive.c |  63 ++++++++++
 lib/utils/i2c/objects.mk         |   5 +
 5 files changed, 287 insertions(+)
 create mode 100644 include/sbi_utils/i2c/dw_i2c.h
 create mode 100644 lib/utils/i2c/dw_i2c.c
 create mode 100644 lib/utils/i2c/fdt_i2c_starfive.c

diff --git a/include/sbi_utils/i2c/dw_i2c.h b/include/sbi_utils/i2c/dw_i2c.h
new file mode 100644
index 0000000..88703e0
--- /dev/null
+++ b/include/sbi_utils/i2c/dw_i2c.h
@@ -0,0 +1,21 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 StarFive Technology Co., Ltd.
+ *
+ * Author: Minda Chen <minda.chen at starfivetech.com>
+ */
+
+#ifndef __DW_I2C_H__
+#define __DW_I2C_H__
+
+#include <sbi_utils/i2c/i2c.h>
+
+int dw_i2c_init(struct i2c_adapter *, int nodeoff);
+
+struct dw_i2c_adapter {
+	unsigned long addr;
+	struct i2c_adapter adapter;
+};
+
+#endif
diff --git a/lib/utils/i2c/Kconfig b/lib/utils/i2c/Kconfig
index 46a3454..0bc6a58 100644
--- a/lib/utils/i2c/Kconfig
+++ b/lib/utils/i2c/Kconfig
@@ -14,8 +14,16 @@ config FDT_I2C_SIFIVE
 	bool "SiFive I2C FDT driver"
 	default n
 
+config FDT_I2C_STARFIVE
+	bool "STARFIVE I2C FDT driver"
+	select I2C_DW
+	default n
 endif
 
+config I2C_DW
+	bool "Synopsys Designware I2C support"
+	default n
+
 config I2C
 	bool "I2C support"
 	default n
diff --git a/lib/utils/i2c/dw_i2c.c b/lib/utils/i2c/dw_i2c.c
new file mode 100644
index 0000000..e2ffc71
--- /dev/null
+++ b/lib/utils/i2c/dw_i2c.c
@@ -0,0 +1,190 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 starfivetech.com
+ *
+ * Authors:
+ *   Minda Chen <minda.chen at starfivetech.com>
+ */
+
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_timer.h>
+#include <sbi/sbi_console.h>
+#include <sbi/sbi_string.h>
+#include <sbi/sbi_bitops.h>
+#include <sbi_utils/i2c/dw_i2c.h>
+
+#define DW_IC_CON		0x00
+#define DW_IC_TAR		0x04
+#define DW_IC_SAR		0x08
+#define DW_IC_DATA_CMD		0x10
+#define DW_IC_SS_SCL_HCNT	0x14
+#define DW_IC_SS_SCL_LCNT	0x18
+#define DW_IC_FS_SCL_HCNT	0x1c
+#define DW_IC_FS_SCL_LCNT	0x20
+#define DW_IC_HS_SCL_HCNT	0x24
+#define DW_IC_HS_SCL_LCNT	0x28
+#define DW_IC_INTR_STAT		0x2c
+#define DW_IC_INTR_MASK		0x30
+#define DW_IC_RAW_INTR_STAT	0x34
+#define DW_IC_RX_TL		0x38
+#define DW_IC_TX_TL		0x3c
+#define DW_IC_CLR_INTR		0x40
+#define DW_IC_CLR_RX_UNDER	0x44
+#define DW_IC_CLR_RX_OVER	0x48
+#define DW_IC_CLR_TX_OVER	0x4c
+#define DW_IC_CLR_RD_REQ	0x50
+#define DW_IC_CLR_TX_ABRT	0x54
+#define DW_IC_CLR_RX_DONE	0x58
+#define DW_IC_CLR_ACTIVITY	0x5c
+#define DW_IC_CLR_STOP_DET	0x60
+#define DW_IC_CLR_START_DET	0x64
+#define DW_IC_CLR_GEN_CALL	0x68
+#define DW_IC_ENABLE		0x6c
+#define DW_IC_STATUS		0x70
+#define DW_IC_TXFLR		0x74
+#define DW_IC_RXFLR		0x78
+#define DW_IC_SDA_HOLD		0x7c
+#define DW_IC_TX_ABRT_SOURCE	0x80
+#define DW_IC_ENABLE_STATUS	0x9c
+#define DW_IC_CLR_RESTART_DET	0xa8
+#define DW_IC_COMP_PARAM_1	0xf4
+#define DW_IC_COMP_VERSION	0xf8
+
+#define DW_I2C_STATUS_TXFIFO_EMPTY	BIT(2)
+#define DW_I2C_STATUS_RXFIFO_NOT_EMPTY	BIT(3)
+
+#define IC_DATA_CMD_READ	BIT(8)
+#define IC_DATA_CMD_STOP	BIT(9)
+#define IC_DATA_CMD_RESTART	BIT(10)
+#define IC_INT_STATUS_STOPDET	BIT(9)
+
+static inline void dw_i2c_setreg(struct dw_i2c_adapter *adap,
+				 u8 reg, u32 value)
+{
+	writel(value, (void *)adap->addr + reg);
+}
+
+static inline u32 dw_i2c_getreg(struct dw_i2c_adapter *adap,
+				u32 reg)
+{
+	return readl((void *)adap->addr + reg);
+}
+
+static int dw_i2c_adapter_poll(struct dw_i2c_adapter *adap,
+			       u32 mask, u32 addr,
+			       bool inverted)
+{
+	unsigned int timeout = 10; /* msec */
+	int count = 0;
+	u32 val;
+
+	do {
+		val = dw_i2c_getreg(adap, addr);
+		if (inverted) {
+			if (!(val & mask))
+				return 0;
+		} else {
+			if (val & mask)
+				return 0;
+		}
+		sbi_timer_udelay(2);
+		count += 1;
+		if (count == (timeout * 1000))
+			return SBI_ETIMEDOUT;
+	} while (1);
+}
+
+#define dw_i2c_adapter_poll_rxrdy(adap)	\
+	dw_i2c_adapter_poll(adap, DW_I2C_STATUS_RXFIFO_NOT_EMPTY, DW_IC_STATUS, 0)
+#define dw_i2c_adapter_poll_txfifo_ready(adap)	\
+	dw_i2c_adapter_poll(adap, DW_I2C_STATUS_TXFIFO_EMPTY, DW_IC_STATUS, 0)
+
+static int dw_i2c_write_addr(struct dw_i2c_adapter *adap, u8 addr)
+{
+	dw_i2c_setreg(adap, DW_IC_ENABLE, 0);
+	dw_i2c_setreg(adap, DW_IC_TAR, addr);
+	dw_i2c_setreg(adap, DW_IC_ENABLE, 1);
+
+	return 0;
+}
+
+static int dw_i2c_adapter_read(struct i2c_adapter *ia, u8 addr,
+			       u8 reg, u8 *buffer, int len)
+{
+	struct dw_i2c_adapter *adap =
+		container_of(ia, struct dw_i2c_adapter, adapter);
+	int rc;
+
+	dw_i2c_write_addr(adap, addr);
+
+	rc = dw_i2c_adapter_poll_txfifo_ready(adap);
+	if (rc)
+		return rc;
+
+	/* set register address */
+	dw_i2c_setreg(adap, DW_IC_DATA_CMD, reg);
+
+	/* set value */
+	while (len) {
+		if (len == 1)
+			dw_i2c_setreg(adap, DW_IC_DATA_CMD,
+				      IC_DATA_CMD_READ | IC_DATA_CMD_STOP);
+		else
+			dw_i2c_setreg(adap, DW_IC_DATA_CMD, IC_DATA_CMD_READ);
+
+		rc = dw_i2c_adapter_poll_rxrdy(adap);
+		if (rc)
+			return rc;
+
+		*buffer = dw_i2c_getreg(adap, DW_IC_DATA_CMD) & 0xff;
+		buffer++;
+		len--;
+	}
+
+	return 0;
+}
+
+static int dw_i2c_adapter_write(struct i2c_adapter *ia, u8 addr,
+				u8 reg, u8 *buffer, int len)
+{
+	struct dw_i2c_adapter *adap =
+		container_of(ia, struct dw_i2c_adapter, adapter);
+	int rc;
+
+	dw_i2c_write_addr(adap, addr);
+
+	rc = dw_i2c_adapter_poll_txfifo_ready(adap);
+	if (rc)
+		return rc;
+
+	/* set register address */
+	dw_i2c_setreg(adap, DW_IC_DATA_CMD, reg);
+
+	while (len) {
+		rc = dw_i2c_adapter_poll_txfifo_ready(adap);
+		if (rc)
+			return rc;
+
+		if (len == 1)
+			dw_i2c_setreg(adap, DW_IC_DATA_CMD, *buffer | IC_DATA_CMD_STOP);
+		else
+			dw_i2c_setreg(adap, DW_IC_DATA_CMD, *buffer);
+
+		buffer++;
+		len--;
+	}
+	rc = dw_i2c_adapter_poll_txfifo_ready(adap);
+
+	return rc;
+}
+
+int dw_i2c_init(struct i2c_adapter *adapter, int nodeoff)
+{
+	adapter->id = nodeoff;
+	adapter->write = dw_i2c_adapter_write;
+	adapter->read = dw_i2c_adapter_read;
+
+	return i2c_adapter_add(adapter);
+}
diff --git a/lib/utils/i2c/fdt_i2c_starfive.c b/lib/utils/i2c/fdt_i2c_starfive.c
new file mode 100644
index 0000000..7b7f3cc
--- /dev/null
+++ b/lib/utils/i2c/fdt_i2c_starfive.c
@@ -0,0 +1,63 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 starfivetech.com
+ *
+ * Authors:
+ *   Minda Chen <minda.chen at starfivetech.com>
+ */
+
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_string.h>
+#include <sbi_utils/fdt/fdt_helper.h>
+#include <sbi_utils/i2c/dw_i2c.h>
+#include <sbi_utils/i2c/fdt_i2c.h>
+
+#define STARFIVE_I2C_ADAPTER_MAX	7
+
+static unsigned int starfive_i2c_adapter_count;
+static struct dw_i2c_adapter
+	starfive_i2c_adapter_array[STARFIVE_I2C_ADAPTER_MAX];
+
+extern struct fdt_i2c_adapter fdt_i2c_adapter_starfive;
+
+static int starfive_i2c_init(void *fdt, int nodeoff,
+			     const struct fdt_match *match)
+{
+	int rc;
+	struct dw_i2c_adapter *adapter;
+	u64 addr;
+
+	if (starfive_i2c_adapter_count >= STARFIVE_I2C_ADAPTER_MAX)
+		return SBI_ENOSPC;
+
+	adapter = &starfive_i2c_adapter_array[starfive_i2c_adapter_count];
+
+	rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
+	if (rc)
+		return rc;
+
+	adapter->addr = addr;
+	adapter->adapter.driver = &fdt_i2c_adapter_starfive;
+
+	rc = dw_i2c_init(&adapter->adapter, nodeoff);
+	if (rc)
+		return rc;
+
+	starfive_i2c_adapter_count++;
+
+	return 0;
+}
+
+static const struct fdt_match starfive_i2c_match[] = {
+	{ .compatible = "snps,designware-i2c" },
+	{ .compatible = "starfive,jh7110-i2c" },
+	{ },
+};
+
+struct fdt_i2c_adapter fdt_i2c_adapter_starfive = {
+	.match_table = starfive_i2c_match,
+	.init = starfive_i2c_init,
+};
+
diff --git a/lib/utils/i2c/objects.mk b/lib/utils/i2c/objects.mk
index a0fbbb5..6ceffe8 100644
--- a/lib/utils/i2c/objects.mk
+++ b/lib/utils/i2c/objects.mk
@@ -14,3 +14,8 @@ libsbiutils-objs-$(CONFIG_FDT_I2C) += i2c/fdt_i2c_adapter_drivers.o
 
 carray-fdt_i2c_adapter_drivers-$(CONFIG_FDT_I2C_SIFIVE) += fdt_i2c_adapter_sifive
 libsbiutils-objs-$(CONFIG_FDT_I2C_SIFIVE) += i2c/fdt_i2c_sifive.o
+
+carray-fdt_i2c_adapter_drivers-$(CONFIG_FDT_I2C_STARFIVE) += fdt_i2c_adapter_starfive
+libsbiutils-objs-$(CONFIG_FDT_I2C_STARFIVE) += i2c/fdt_i2c_starfive.o
+
+libsbiutils-objs-$(CONFIG_I2C_DW) += i2c/dw_i2c.o
-- 
2.17.1



From wxjstz at 126.com  Fri Feb 24 06:30:29 2023
From: wxjstz at 126.com (Xiang W)
Date: Fri, 24 Feb 2023 22:30:29 +0800
Subject: [RFC PATCH] Kconfig: Add an option for debugging OpenSBI
In-Reply-To: <CAAhSdy2pDpYtYgydyaWyxeQDQD+VgmDM1nfwD23xAKOBBWvEKw@mail.gmail.com>
References: <20230223145238.1750383-1-bmeng@tinylab.org>
	 <CAAhSdy2pDpYtYgydyaWyxeQDQD+VgmDM1nfwD23xAKOBBWvEKw@mail.gmail.com>
Message-ID: <2a95ccab0877534191ef8b3db23e2a8161e4bc3c.camel@126.com>

? 2023-02-24???? 10:19 +0530?Anup Patel???
> On Thu, Feb 23, 2023 at 8:23 PM Bin Meng <bmeng at tinylab.org> wrote:
> > 
> > Add a new Kconfig option CONFIG_OPENSBI_DEBUG to prevent compiler
> > optimizations using -O2.
> > 
> > Signed-off-by: Bin Meng <bmeng at tinylab.org>
> > ---
> > 
> > ?Kconfig? | 7 +++++++
> > ?Makefile | 7 ++++++-
> > ?2 files changed, 13 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Kconfig b/Kconfig
> > index acfc138..123ea66 100644
> > --- a/Kconfig
> > +++ b/Kconfig
> > @@ -23,3 +23,10 @@ source "$(OPENSBI_SRC_DIR)/lib/sbi/Kconfig"
> > ?source "$(OPENSBI_SRC_DIR)/lib/utils/Kconfig"
> > 
> > ?source "$(OPENSBI_SRC_DIR)/firmware/Kconfig"
> > +
> > +menu "General Setup"
> > +
> > +config OPENSBI_DEBUG
> > +?????? bool "Disable compiler optimizations for debugging"
> > +
> > +endmenu\
> 
> Rather than adding a Kconfig option, I would suggest having a
> simple make command-line option. For example, "make DEBUG=1"
> 
Agree to add a command line option so that don't need to re-run
make menuconfig when finally generate the release firmware

Regards,
Xiang W
> Regards,
> Anup
> 
> > diff --git a/Makefile b/Makefile
> > index b20404f..b4ed548 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -331,7 +331,12 @@ GENFLAGS?? +=????? $(libsbiutils-genflags-y)
> > ?GENFLAGS?????? +=????? $(platform-genflags-y)
> > ?GENFLAGS?????? +=????? $(firmware-genflags-y)
> > 
> > -CFLAGS???????? =?????? -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2
> > +CFLAGS???????? =?????? -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing
> > +ifeq ($(CONFIG_OPENSBI_DEBUG),y)
> > +CFLAGS???????? +=????? -O0
> > +else
> > +CFLAGS???????? +=????? -O2
> > +endif
> > ?CFLAGS???????? +=????? -fno-omit-frame-pointer -fno-optimize-sibling-calls -mstrict-align
> > ?# enable -m(no-)save-restore option by CC_SUPPORT_SAVE_RESTORE
> > ?ifeq ($(CC_SUPPORT_SAVE_RESTORE),y)
> > --
> > 2.25.1
> > 
> 



From bmeng.cn at gmail.com  Sun Feb 26 18:09:44 2023
From: bmeng.cn at gmail.com (Bin Meng)
Date: Mon, 27 Feb 2023 10:09:44 +0800
Subject: [PATCH v2] lib: sbi_domain: Avoid overwriting coldboot hart's scratch->arg1
In-Reply-To: <CAAhSdy3JXUPXUO4dcwstXWSpXS3YLKe=6FjvRA6J3wCPj4BW1w@mail.gmail.com>
References: <20230223095348.1723380-1-bmeng@tinylab.org> <CAAhSdy3JXUPXUO4dcwstXWSpXS3YLKe=6FjvRA6J3wCPj4BW1w@mail.gmail.com>
Message-ID: <CAEUhbmU4Wf=7WtKhQe5Y5PWC3T3p8coL6NnUOmFP94EzcyVGYQ@mail.gmail.com>

Hi Anup,

On Fri, Feb 24, 2023 at 12:50?PM Anup Patel <anup at brainfault.org> wrote:
>
> On Thu, Feb 23, 2023 at 3:24 PM Bin Meng <bmeng at tinylab.org> wrote:
> >
> > In sbi_domain_finalize(), when locating the coldboot hart's domain,
> > the coldboot hart's scratch->arg1 will be overwritten by the domain
> > configuration. scratch->arg1 holds the FDT address of the coldboot
> > hart, and in later boot process OpenSBI codes still read the FDT
> > here and there, which leads to a crash.
> >
> > To fix this, we change fdt_get_address() to return root domain's
> > next_arg1 as the FDT pointer.
> >
> > Resolves: https://github.com/riscv-software-src/opensbi/issues/281
> > Fixes: b1678af210dc ("lib: sbi: Add initial domain support")
> > Reported-by: Marouene Boubakri <marouene.boubakri at nxp.com>
> > Signed-off-by: Bin Meng <bmeng at tinylab.org>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Thanks for the review!

This v2 was superseded by v3 @
https://patchwork.ozlabs.org/project/opensbi/list/?series=343310

Regards,
Bin


From bmeng at tinylab.org  Sun Feb 26 18:35:06 2023
From: bmeng at tinylab.org (Bin Meng)
Date: Mon, 27 Feb 2023 10:35:06 +0800
Subject: [PATCH] make: Add a command line option for debugging OpenSBI
Message-ID: <20230227023506.1818620-1-bmeng@tinylab.org>

Add a new make command line option "make DEBUG=1" to prevent compiler
optimizations using -O2.

Signed-off-by: Bin Meng <bmeng at tinylab.org>
---

 Makefile  |  7 ++++++-
 README.md | 13 +++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index b20404f..a26a39b 100644
--- a/Makefile
+++ b/Makefile
@@ -331,7 +331,12 @@ GENFLAGS	+=	$(libsbiutils-genflags-y)
 GENFLAGS	+=	$(platform-genflags-y)
 GENFLAGS	+=	$(firmware-genflags-y)
 
-CFLAGS		=	-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2
+CFLAGS		=	-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing
+ifneq ($(DEBUG),)
+CFLAGS		+=	-O0
+else
+CFLAGS		+=	-O2
+endif
 CFLAGS		+=	-fno-omit-frame-pointer -fno-optimize-sibling-calls -mstrict-align
 # enable -m(no-)save-restore option by CC_SUPPORT_SAVE_RESTORE
 ifeq ($(CC_SUPPORT_SAVE_RESTORE),y)
diff --git a/README.md b/README.md
index 895bbf2..a6bcb0c 100644
--- a/README.md
+++ b/README.md
@@ -298,6 +298,19 @@ NOTE: Using `BUILD_INFO=y` without specifying SOURCE_DATE_EPOCH will violate
 purpose, and should NOT be used in a product which follows "reproducible
 builds".
 
+Building with optimization off for debugging
+--------------------------------------------
+
+When debugging OpenSBI, we may want to turn off the compiler optimization and
+make debugging produce the expected results for a better debugging experience.
+To build with optimization off we can just simply add `DEBUG=1`, like:
+```
+make DEBUG=1
+```
+
+This definition is ONLY for development and debug purpose, and should NOT be
+used in a product build.
+
 Contributing to OpenSBI
 -----------------------
 
-- 
2.25.1



From wxjstz at 126.com  Sun Feb 26 19:11:05 2023
From: wxjstz at 126.com (Xiang W)
Date: Mon, 27 Feb 2023 11:11:05 +0800
Subject: [PATCH] make: Add a command line option for debugging OpenSBI
In-Reply-To: <20230227023506.1818620-1-bmeng@tinylab.org>
References: <20230227023506.1818620-1-bmeng@tinylab.org>
Message-ID: <43cfe473b7b96c91a0aa05b92e3fb02b9f81effe.camel@126.com>

? 2023-02-27???? 10:35 +0800?Bin Meng???
> Add a new make command line option "make DEBUG=1" to prevent compiler
> optimizations using -O2.
> 
> Signed-off-by: Bin Meng <bmeng at tinylab.org>
LGTM

Reviewed-by: Xiang W <wxjstz at 126.com>
> ---
> 
> ?Makefile? |? 7 ++++++-
> ?README.md | 13 +++++++++++++
> ?2 files changed, 19 insertions(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index b20404f..a26a39b 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -331,7 +331,12 @@ GENFLAGS???+=??????$(libsbiutils-genflags-y)
> ?GENFLAGS???????+=??????$(platform-genflags-y)
> ?GENFLAGS???????+=??????$(firmware-genflags-y)
> ?
> -CFLAGS?????????=???????-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2
> +CFLAGS?????????=???????-g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing
> +ifneq ($(DEBUG),)
> +CFLAGS?????????+=??????-O0
> +else
> +CFLAGS?????????+=??????-O2
> +endif
> ?CFLAGS?????????+=??????-fno-omit-frame-pointer -fno-optimize-sibling-calls -mstrict-align
> ?# enable -m(no-)save-restore option by CC_SUPPORT_SAVE_RESTORE
> ?ifeq ($(CC_SUPPORT_SAVE_RESTORE),y)
> diff --git a/README.md b/README.md
> index 895bbf2..a6bcb0c 100644
> --- a/README.md
> +++ b/README.md
> @@ -298,6 +298,19 @@ NOTE: Using `BUILD_INFO=y` without specifying SOURCE_DATE_EPOCH will violate
> ?purpose, and should NOT be used in a product which follows "reproducible
> ?builds".
> ?
> +Building with optimization off for debugging
> +--------------------------------------------
> +
> +When debugging OpenSBI, we may want to turn off the compiler optimization and
> +make debugging produce the expected results for a better debugging experience.
> +To build with optimization off we can just simply add `DEBUG=1`, like:
> +```
> +make DEBUG=1
> +```
> +
> +This definition is ONLY for development and debug purpose, and should NOT be
> +used in a product build.
> +
> ?Contributing to OpenSBI
> ?-----------------------
> ?
> -- 
> 2.25.1
> 
> 



From anup at brainfault.org  Sun Feb 26 20:27:29 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 09:57:29 +0530
Subject: [PATCH v2] lib: sbi_domain: Avoid overwriting coldboot hart's scratch->arg1
In-Reply-To: <CAEUhbmU4Wf=7WtKhQe5Y5PWC3T3p8coL6NnUOmFP94EzcyVGYQ@mail.gmail.com>
References: <20230223095348.1723380-1-bmeng@tinylab.org> <CAAhSdy3JXUPXUO4dcwstXWSpXS3YLKe=6FjvRA6J3wCPj4BW1w@mail.gmail.com>
 <CAEUhbmU4Wf=7WtKhQe5Y5PWC3T3p8coL6NnUOmFP94EzcyVGYQ@mail.gmail.com>
Message-ID: <CAAhSdy3=UcSAXgyp-7SpDTkHoyhOJBL6JZ0WzSWaPARQS3ePFg@mail.gmail.com>

On Mon, Feb 27, 2023 at 7:39 AM Bin Meng <bmeng.cn at gmail.com> wrote:
>
> Hi Anup,
>
> On Fri, Feb 24, 2023 at 12:50?PM Anup Patel <anup at brainfault.org> wrote:
> >
> > On Thu, Feb 23, 2023 at 3:24 PM Bin Meng <bmeng at tinylab.org> wrote:
> > >
> > > In sbi_domain_finalize(), when locating the coldboot hart's domain,
> > > the coldboot hart's scratch->arg1 will be overwritten by the domain
> > > configuration. scratch->arg1 holds the FDT address of the coldboot
> > > hart, and in later boot process OpenSBI codes still read the FDT
> > > here and there, which leads to a crash.
> > >
> > > To fix this, we change fdt_get_address() to return root domain's
> > > next_arg1 as the FDT pointer.
> > >
> > > Resolves: https://github.com/riscv-software-src/opensbi/issues/281
> > > Fixes: b1678af210dc ("lib: sbi: Add initial domain support")
> > > Reported-by: Marouene Boubakri <marouene.boubakri at nxp.com>
> > > Signed-off-by: Bin Meng <bmeng at tinylab.org>
> >
> > Looks good to me.
> >
> > Reviewed-by: Anup Patel <anup at brainfault.org>
>
> Thanks for the review!
>
> This v2 was superseded by v3 @
> https://patchwork.ozlabs.org/project/opensbi/list/?series=343310

Ahh okay, thanks for pointing.

Regards,
Anup


From anup at brainfault.org  Sun Feb 26 20:45:17 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 10:15:17 +0530
Subject: [PATCH v3 1/4] lib: utils/fdt: Fix fdt_pmu.c header dependency
In-Reply-To: <20230223104009.1727401-1-bmeng@tinylab.org>
References: <20230223104009.1727401-1-bmeng@tinylab.org>
Message-ID: <CAAhSdy2fPr1TWCy2OAjsLOGrx=ZBRpD2TCJXAwdBGDc-CpjymQ@mail.gmail.com>

On Thu, Feb 23, 2023 at 4:10 PM Bin Meng <bmeng at tinylab.org> wrote:
>
> The code calls sbi_scratch_thishart_ptr() from sbi_scratch.h which
> is not directly included. Fix such dependency.
>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
>  lib/utils/fdt/fdt_pmu.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/lib/utils/fdt/fdt_pmu.c b/lib/utils/fdt/fdt_pmu.c
> index 4829f21..83301bb 100644
> --- a/lib/utils/fdt/fdt_pmu.c
> +++ b/lib/utils/fdt/fdt_pmu.c
> @@ -12,6 +12,7 @@
>  #include <sbi/sbi_hart.h>
>  #include <sbi/sbi_error.h>
>  #include <sbi/sbi_pmu.h>
> +#include <sbi/sbi_scratch.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
>  #define FDT_PMU_HW_EVENT_MAX (SBI_PMU_HW_EVENT_MAX * 2)
> --
> 2.25.1
>


From anup at brainfault.org  Sun Feb 26 20:45:35 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 10:15:35 +0530
Subject: [PATCH v3 2/4] platform: generic/allwinner: Fix sun20i-d1.c
 header dependency
In-Reply-To: <20230223104009.1727401-2-bmeng@tinylab.org>
References: <20230223104009.1727401-1-bmeng@tinylab.org> <20230223104009.1727401-2-bmeng@tinylab.org>
Message-ID: <CAAhSdy0Yb3Sn9X=byDF-sdMyzEM9NDTtsc3K40EBUB4bBtH45w@mail.gmail.com>

On Thu, Feb 23, 2023 at 4:11 PM Bin Meng <bmeng at tinylab.org> wrote:
>
> The code calls various macros from riscv_asm.h and sbi_scratch.h
> which are not directly included. Fix such dependency.
>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
>  platform/generic/allwinner/sun20i-d1.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
> index 295c159..0f0a9f3 100644
> --- a/platform/generic/allwinner/sun20i-d1.c
> +++ b/platform/generic/allwinner/sun20i-d1.c
> @@ -6,12 +6,14 @@
>
>  #include <platform_override.h>
>  #include <thead_c9xx.h>
> +#include <sbi/riscv_asm.h>
>  #include <sbi/riscv_io.h>
>  #include <sbi/sbi_bitops.h>
>  #include <sbi/sbi_ecall_interface.h>
>  #include <sbi/sbi_error.h>
>  #include <sbi/sbi_hsm.h>
>  #include <sbi/sbi_pmu.h>
> +#include <sbi/sbi_scratch.h>
>  #include <sbi_utils/fdt/fdt_fixup.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>  #include <sbi_utils/irqchip/fdt_irqchip_plic.h>
> --
> 2.25.1
>


From anup at brainfault.org  Sun Feb 26 20:45:52 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 10:15:52 +0530
Subject: [PATCH v3 3/4] platform: generic/andes: Fix ae350.c header dependency
In-Reply-To: <20230223104009.1727401-3-bmeng@tinylab.org>
References: <20230223104009.1727401-1-bmeng@tinylab.org> <20230223104009.1727401-3-bmeng@tinylab.org>
Message-ID: <CAAhSdy1NtEKv=X7ad_vZQFkDvcchmpvMWTbnoroOVJeT+0_a6g@mail.gmail.com>

On Thu, Feb 23, 2023 at 4:12 PM Bin Meng <bmeng at tinylab.org> wrote:
>
> The code calls various macros from riscv_asm.h which is not directly
> included. Fix such dependency.
>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
>  platform/generic/andes/ae350.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/platform/generic/andes/ae350.c b/platform/generic/andes/ae350.c
> index 89f5b74..01bd02d 100644
> --- a/platform/generic/andes/ae350.c
> +++ b/platform/generic/andes/ae350.c
> @@ -11,6 +11,7 @@
>  #include <sbi_utils/fdt/fdt_helper.h>
>  #include <sbi_utils/fdt/fdt_fixup.h>
>  #include <sbi_utils/sys/atcsmu.h>
> +#include <sbi/riscv_asm.h>
>  #include <sbi/sbi_bitops.h>
>  #include <sbi/sbi_error.h>
>  #include <sbi/sbi_hsm.h>
> --
> 2.25.1
>


From anup at brainfault.org  Sun Feb 26 20:46:06 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 10:16:06 +0530
Subject: [PATCH v3 4/4] include: fdt/fdt_helper: Change fdt_get_address()
 to return root.next_arg1
In-Reply-To: <20230223104009.1727401-4-bmeng@tinylab.org>
References: <20230223104009.1727401-1-bmeng@tinylab.org> <20230223104009.1727401-4-bmeng@tinylab.org>
Message-ID: <CAAhSdy0Ov1kqD0cqi87Dzh7b67Rwmvq8E-3-V8xXoP4zSUm_bg@mail.gmail.com>

On Thu, Feb 23, 2023 at 4:13 PM Bin Meng <bmeng at tinylab.org> wrote:
>
> In sbi_domain_finalize(), when locating the coldboot hart's domain,
> the coldboot hart's scratch->arg1 will be overwritten by the domain
> configuration. However scratch->arg1 holds the FDT address of the
> coldboot hart, and is still being accessed by fdt_get_address() in
> later boot process. scratch->arg1 could then contain completely
> garbage and lead to a crash.
>
> To fix this, we change fdt_get_address() to return root domain's
> next_arg1 as the FDT pointer.
>
> Resolves: https://github.com/riscv-software-src/opensbi/issues/281
> Fixes: b1678af210dc ("lib: sbi: Add initial domain support")
> Reported-by: Marouene Boubakri <marouene.boubakri at nxp.com>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
> Changes in v3:
> - Drop <sbi/sbi_scratch.h> as it is no longer needed
>
> Changes in v2:
> - Change fdt_get_address() to return root.next_arg1
>
>  include/sbi_utils/fdt/fdt_helper.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
> index 09f3095..39d7f3a 100644
> --- a/include/sbi_utils/fdt/fdt_helper.h
> +++ b/include/sbi_utils/fdt/fdt_helper.h
> @@ -11,7 +11,7 @@
>  #define __FDT_HELPER_H__
>
>  #include <sbi/sbi_types.h>
> -#include <sbi/sbi_scratch.h>
> +#include <sbi/sbi_domain.h>
>
>  struct fdt_match {
>         const char *compatible;
> @@ -109,7 +109,7 @@ int fdt_parse_compat_addr(void *fdt, uint64_t *addr,
>
>  static inline void *fdt_get_address(void)
>  {
> -       return sbi_scratch_thishart_arg1_ptr();
> +       return (void *)root.next_arg1;
>  }
>
>  #endif /* __FDT_HELPER_H__ */
> --
> 2.25.1
>


From anup at brainfault.org  Sun Feb 26 21:22:14 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 10:52:14 +0530
Subject: [PATCH v4] lib: utils: fdt_fixup: Fix compile error
In-Reply-To: <20230221050707.825994-1-wxjstz@126.com>
References: <20230221050707.825994-1-wxjstz@126.com>
Message-ID: <CAAhSdy3Juch4TmNn+6GQkbK1AhBCRA0UnQYbGmRmnrOLzAc9cA@mail.gmail.com>

On Tue, Feb 21, 2023 at 10:37 AM Xiang W <wxjstz at 126.com> wrote:
>
> When building with GCC-10 or older versions, it throws the following
> error:
>
>  CC-DEP    platform/generic/lib/utils/fdt/fdt_fixup.dep
>  CC        platform/generic/lib/utils/fdt/fdt_fixup.o
> lib/utils/fdt/fdt_fixup.c: In function 'fdt_reserved_memory_fixup':
> lib/utils/fdt/fdt_fixup.c:376:2: error: label at end of compound statement
>   376 |  next_entry:
>       |  ^~~~~~~~~~
>
> Remove the goto statement.
>
> Resolves: https://github.com/riscv-software-src/opensbi/issues/288
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Signed-off-by: Xiang W <wxjstz at 126.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
> Changes in v4:
>  - Continue to fix the comment in Changes v3
>
> Changes in v3:
> - Fix a comment issue raised by Bin Meng
> - Changed the type of "overlap" from int to bool as suggested by Bin Meng
>
> Changes in v2:
> - Fix a comment issue raised by Andreas Schwab
>  lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
>  lib/utils/fdt/fdt_fixup.c | 13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/lib/utils/fdt/fdt_fixup.c b/lib/utils/fdt/fdt_fixup.c
> index 619e4f5..c10179b 100644
> --- a/lib/utils/fdt/fdt_fixup.c
> +++ b/lib/utils/fdt/fdt_fixup.c
> @@ -361,19 +361,22 @@ int fdt_reserved_memory_fixup(void *fdt)
>                         return SBI_ENOSPC;
>                 }
>
> +               bool overlap = false;
>                 addr = reg->base;
>                 for (j = 0; j < i; j++) {
>                         if (addr == filtered_base[j]
>                             && filtered_order[j] < reg->order) {
> +                               overlap = true;
>                                 filtered_order[j] = reg->order;
> -                               goto next_entry;
> +                               break;
>                         }
>                 }
>
> -               filtered_base[i] = reg->base;
> -               filtered_order[i] = reg->order;
> -               i++;
> -       next_entry:
> +               if (!overlap) {
> +                       filtered_base[i] = reg->base;
> +                       filtered_order[i] = reg->order;
> +                       i++;
> +               }
>         }
>
>         for (j = 0; j < i; j++) {
> --
> 2.39.1
>


From anup at brainfault.org  Sun Feb 26 21:26:04 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 10:56:04 +0530
Subject: [PATCH] docs: fix typo in fw.md
In-Reply-To: <TY3P286MB2611F8A2445A0A2ADE1A843498A39@TY3P286MB2611.JPNP286.PROD.OUTLOOK.COM>
References: <TY3P286MB2611F8A2445A0A2ADE1A843498A39@TY3P286MB2611.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAAhSdy14BFhQNd-cLHiZW59U=M90wqRMys5LmxnE46w9p_nr0A@mail.gmail.com>

On Wed, Feb 15, 2023 at 11:57 PM Shengyu Qu <wiagn233 at outlook.com> wrote:
>
> In docs/firmware/fw.md, there's a configuration parameter called
> FW_TEXT_ADDR, which actually should be FW_TEXT_START, so fix it.
>
> Signed-off-by: Shengyu Qu <wiagn233 at outlook.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  docs/firmware/fw.md | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/docs/firmware/fw.md b/docs/firmware/fw.md
> index 18faaa9..38351c8 100644
> --- a/docs/firmware/fw.md
> +++ b/docs/firmware/fw.md
> @@ -61,7 +61,7 @@ Firmware Configuration and Compilation
>  All firmware types support the following common compile time configuration
>  parameters:
>
> -* **FW_TEXT_ADDR** - Defines the execution address of the OpenSBI firmware.
> +* **FW_TEXT_START** - Defines the execution address of the OpenSBI firmware.
>    This configuration parameter is mandatory.
>  * **FW_FDT_PATH** - Path to an external flattened device tree binary file to
>    be embedded in the *.rodata* section of the final firmware. If this option
> --
> 2.25.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sun Feb 26 21:54:22 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 11:24:22 +0530
Subject: [PATCH] lib: sbi_hart: Enable hcontext and scontext
In-Reply-To: <20230210085238.19723-1-nylon.chen@sifive.com>
References: <20230210085238.19723-1-nylon.chen@sifive.com>
Message-ID: <CAAhSdy0+Ywcrf5Vu8EiYQRyyN-U-neuV4BVAXjRX9gGTs_rdMQ@mail.gmail.com>

On Fri, Feb 10, 2023 at 2:22 PM Nylon Chen <nylon.chen at sifive.com> wrote:
>
> According to the description in "riscv-state-enable[0]", to access
> h/scontext in S-Mode, we need to enable the 57th bit.
>
> If it is not enabled, an "illegal instruction" error will occur.
>
> Link: https://github.com/riscv/riscv-state-enable/blob/a28bfae443f350d5b4c42874f428367d5b322ffe/content.adoc [0]
>
> Signed-off-by: Nylon Chen <nylon.chen at sifive.com>
> Reviewed-by: Zong Li <zong.li at sifive.com>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  include/sbi/riscv_encoding.h | 2 ++
>  lib/sbi/sbi_hart.c           | 1 +
>  2 files changed, 3 insertions(+)
>
> diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
> index b0f08c8..4ebed97 100644
> --- a/include/sbi/riscv_encoding.h
> +++ b/include/sbi/riscv_encoding.h
> @@ -736,6 +736,8 @@
>  #define SMSTATEEN0_CS                  (_ULL(1) << SMSTATEEN0_CS_SHIFT)
>  #define SMSTATEEN0_FCSR_SHIFT          1
>  #define SMSTATEEN0_FCSR                        (_ULL(1) << SMSTATEEN0_FCSR_SHIFT)
> +#define SMSTATEEN0_CONTEXT_SHIFT       57
> +#define SMSTATEEN0_CONTEXT             (_ULL(1) << SMSTATEEN0_CONTEXT_SHIFT)
>  #define SMSTATEEN0_IMSIC_SHIFT         58
>  #define SMSTATEEN0_IMSIC               (_ULL(1) << SMSTATEEN0_IMSIC_SHIFT)
>  #define SMSTATEEN0_AIA_SHIFT           59
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index 02ce991..5e06918 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -90,6 +90,7 @@ static void mstatus_init(struct sbi_scratch *scratch)
>                 mstateen_val |= ((uint64_t)csr_read(CSR_MSTATEEN0H)) << 32;
>  #endif
>                 mstateen_val |= SMSTATEEN_STATEN;
> +               mstateen_val |= SMSTATEEN0_CONTEXT;
>                 mstateen_val |= SMSTATEEN0_HSENVCFG;
>
>                 if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA))
> --
> 2.36.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sun Feb 26 22:09:23 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 11:39:23 +0530
Subject: [PATCH 1/2] lib: sbi: Align SBI vendor extension id with
 mvendorid CSR
In-Reply-To: <20230213051805.603113-1-apatel@ventanamicro.com>
References: <20230213051805.603113-1-apatel@ventanamicro.com>
Message-ID: <CAAhSdy0OJtz-nj_KF=b3d6wLkoefcn1NR++xO=e5BMoExg3g+g@mail.gmail.com>

On Mon, Feb 13, 2023 at 10:48 AM Anup Patel <apatel at ventanamicro.com> wrote:
>
> As-per the SBI specification, the lower 24bits of the SBI vendor
> extension id is same as lower 24bits of the mvendorid CSR.
>
> We update the SBI vendor extension id checking based on above.
>
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>

Applied this patch to the riscv/opensbi repo.

Regards,
Anup

> ---
>  include/sbi/sbi_platform.h                   | 17 ++++++++---------
>  lib/sbi/sbi_ecall_vendor.c                   | 19 +++++++++++++++++--
>  platform/generic/include/platform_override.h |  1 -
>  platform/generic/platform.c                  | 19 ++++++-------------
>  4 files changed, 31 insertions(+), 25 deletions(-)
>
> diff --git a/include/sbi/sbi_platform.h b/include/sbi/sbi_platform.h
> index 43d1c03..3a629a6 100644
> --- a/include/sbi/sbi_platform.h
> +++ b/include/sbi/sbi_platform.h
> @@ -126,8 +126,8 @@ struct sbi_platform_operations {
>         /** Exit platform timer for current HART */
>         void (*timer_exit)(void);
>
> -       /** platform specific SBI extension implementation probe function */
> -       int (*vendor_ext_check)(long extid);
> +       /** Check if SBI vendor extension is implemented or not */
> +       bool (*vendor_ext_check)(void);
>         /** platform specific SBI extension implementation provider */
>         int (*vendor_ext_provider)(long extid, long funcid,
>                                    const struct sbi_trap_regs *regs,
> @@ -636,20 +636,19 @@ static inline void sbi_platform_timer_exit(const struct sbi_platform *plat)
>  }
>
>  /**
> - * Check if a vendor extension is implemented or not.
> + * Check if SBI vendor extension is implemented or not.
>   *
>   * @param plat pointer to struct sbi_platform
> - * @param extid        vendor SBI extension id
>   *
> - * @return 0 if extid is not implemented and 1 if implemented
> + * @return false if not implemented and true if implemented
>   */
> -static inline int sbi_platform_vendor_ext_check(const struct sbi_platform *plat,
> -                                               long extid)
> +static inline bool sbi_platform_vendor_ext_check(
> +                                       const struct sbi_platform *plat)
>  {
>         if (plat && sbi_platform_ops(plat)->vendor_ext_check)
> -               return sbi_platform_ops(plat)->vendor_ext_check(extid);
> +               return sbi_platform_ops(plat)->vendor_ext_check();
>
> -       return 0;
> +       return false;
>  }
>
>  /**
> diff --git a/lib/sbi/sbi_ecall_vendor.c b/lib/sbi/sbi_ecall_vendor.c
> index 9252829..9ea5156 100644
> --- a/lib/sbi/sbi_ecall_vendor.c
> +++ b/lib/sbi/sbi_ecall_vendor.c
> @@ -13,12 +13,23 @@
>  #include <sbi/sbi_error.h>
>  #include <sbi/sbi_platform.h>
>  #include <sbi/sbi_trap.h>
> +#include <sbi/riscv_asm.h>
> +
> +static inline unsigned long sbi_ecall_vendor_id(void)
> +{
> +       return SBI_EXT_VENDOR_START +
> +               (csr_read(CSR_MVENDORID) &
> +                (SBI_EXT_VENDOR_END - SBI_EXT_VENDOR_START));
> +}
>
>  static int sbi_ecall_vendor_probe(unsigned long extid,
>                                   unsigned long *out_val)
>  {
> -       *out_val = sbi_platform_vendor_ext_check(sbi_platform_thishart_ptr(),
> -                                                extid);
> +       if (!sbi_platform_vendor_ext_check(sbi_platform_thishart_ptr()) ||
> +           extid != sbi_ecall_vendor_id())
> +               *out_val = 0;
> +       else
> +               *out_val = 1;
>         return 0;
>  }
>
> @@ -27,6 +38,10 @@ static int sbi_ecall_vendor_handler(unsigned long extid, unsigned long funcid,
>                                     unsigned long *out_val,
>                                     struct sbi_trap_info *out_trap)
>  {
> +       if (!sbi_platform_vendor_ext_check(sbi_platform_thishart_ptr()) ||
> +           extid != sbi_ecall_vendor_id())
> +               return SBI_ERR_NOT_SUPPORTED;
> +
>         return sbi_platform_vendor_ext_provider(sbi_platform_thishart_ptr(),
>                                                 extid, funcid, regs,
>                                                 out_val, out_trap);
> diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
> index a59b06a..350c381 100644
> --- a/platform/generic/include/platform_override.h
> +++ b/platform/generic/include/platform_override.h
> @@ -27,7 +27,6 @@ struct platform_override {
>         int (*extensions_init)(const struct fdt_match *match,
>                                struct sbi_hart_features *hfeatures);
>         void (*fw_init)(void *fdt, const struct fdt_match *match);
> -       int (*vendor_ext_check)(long extid, const struct fdt_match *match);
>         int (*vendor_ext_provider)(long extid, long funcid,
>                                    const struct sbi_trap_regs *regs,
>                                    unsigned long *out_value,
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index a34d3b0..88c3ec7 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -180,13 +180,10 @@ static int generic_final_init(bool cold_boot)
>         return 0;
>  }
>
> -static int generic_vendor_ext_check(long extid)
> +static bool generic_vendor_ext_check(void)
>  {
> -       if (generic_plat && generic_plat->vendor_ext_check)
> -               return generic_plat->vendor_ext_check(extid,
> -                                                     generic_plat_match);
> -
> -       return 0;
> +       return (generic_plat && generic_plat->vendor_ext_provider) ?
> +               true : false;
>  }
>
>  static int generic_vendor_ext_provider(long extid, long funcid,
> @@ -194,13 +191,9 @@ static int generic_vendor_ext_provider(long extid, long funcid,
>                                        unsigned long *out_value,
>                                        struct sbi_trap_info *out_trap)
>  {
> -       if (generic_plat && generic_plat->vendor_ext_provider) {
> -               return generic_plat->vendor_ext_provider(extid, funcid, regs,
> -                                                        out_value, out_trap,
> -                                                        generic_plat_match);
> -       }
> -
> -       return SBI_ENOTSUPP;
> +       return generic_plat->vendor_ext_provider(extid, funcid, regs,
> +                                                out_value, out_trap,
> +                                                generic_plat_match);
>  }
>
>  static void generic_early_exit(void)
> --
> 2.34.1
>


From anup at brainfault.org  Sun Feb 26 22:09:44 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 11:39:44 +0530
Subject: [PATCH 2/2] include: sbi: Remove extid parameter from
 vendor_ext_provider() callback
In-Reply-To: <20230213051805.603113-2-apatel@ventanamicro.com>
References: <20230213051805.603113-1-apatel@ventanamicro.com> <20230213051805.603113-2-apatel@ventanamicro.com>
Message-ID: <CAAhSdy0NYbFfriCp3dAHBw=7GO33qeXQhs4S3bnkQsO8GG2pTQ@mail.gmail.com>

On Mon, Feb 13, 2023 at 10:48 AM Anup Patel <apatel at ventanamicro.com> wrote:
>
> The extid parameter of vendor_ext_provider() is redundant so let us
> remove it.
>
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>

Applied this patch to the riscv/opensbi repo.

Regards,
Anup

> ---
>  include/sbi/sbi_platform.h                   | 9 ++++-----
>  lib/sbi/sbi_ecall_vendor.c                   | 2 +-
>  platform/generic/include/platform_override.h | 2 +-
>  platform/generic/platform.c                  | 4 ++--
>  4 files changed, 8 insertions(+), 9 deletions(-)
>
> diff --git a/include/sbi/sbi_platform.h b/include/sbi/sbi_platform.h
> index 3a629a6..546c0a6 100644
> --- a/include/sbi/sbi_platform.h
> +++ b/include/sbi/sbi_platform.h
> @@ -129,7 +129,7 @@ struct sbi_platform_operations {
>         /** Check if SBI vendor extension is implemented or not */
>         bool (*vendor_ext_check)(void);
>         /** platform specific SBI extension implementation provider */
> -       int (*vendor_ext_provider)(long extid, long funcid,
> +       int (*vendor_ext_provider)(long funcid,
>                                    const struct sbi_trap_regs *regs,
>                                    unsigned long *out_value,
>                                    struct sbi_trap_info *out_trap);
> @@ -655,7 +655,6 @@ static inline bool sbi_platform_vendor_ext_check(
>   * Invoke platform specific vendor SBI extension implementation.
>   *
>   * @param plat pointer to struct sbi_platform
> - * @param extid        vendor SBI extension id
>   * @param funcid SBI function id within the extension id
>   * @param regs pointer to trap registers passed by the caller
>   * @param out_value output value that can be filled by the callee
> @@ -665,14 +664,14 @@ static inline bool sbi_platform_vendor_ext_check(
>   */
>  static inline int sbi_platform_vendor_ext_provider(
>                                         const struct sbi_platform *plat,
> -                                       long extid, long funcid,
> +                                       long funcid,
>                                         const struct sbi_trap_regs *regs,
>                                         unsigned long *out_value,
>                                         struct sbi_trap_info *out_trap)
>  {
>         if (plat && sbi_platform_ops(plat)->vendor_ext_provider) {
> -               return sbi_platform_ops(plat)->vendor_ext_provider(extid,
> -                                                               funcid, regs,
> +               return sbi_platform_ops(plat)->vendor_ext_provider(funcid,
> +                                                               regs,
>                                                                 out_value,
>                                                                 out_trap);
>         }
> diff --git a/lib/sbi/sbi_ecall_vendor.c b/lib/sbi/sbi_ecall_vendor.c
> index 9ea5156..8b8dab0 100644
> --- a/lib/sbi/sbi_ecall_vendor.c
> +++ b/lib/sbi/sbi_ecall_vendor.c
> @@ -43,7 +43,7 @@ static int sbi_ecall_vendor_handler(unsigned long extid, unsigned long funcid,
>                 return SBI_ERR_NOT_SUPPORTED;
>
>         return sbi_platform_vendor_ext_provider(sbi_platform_thishart_ptr(),
> -                                               extid, funcid, regs,
> +                                               funcid, regs,
>                                                 out_val, out_trap);
>  }
>
> diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
> index 350c381..0d9e5ee 100644
> --- a/platform/generic/include/platform_override.h
> +++ b/platform/generic/include/platform_override.h
> @@ -27,7 +27,7 @@ struct platform_override {
>         int (*extensions_init)(const struct fdt_match *match,
>                                struct sbi_hart_features *hfeatures);
>         void (*fw_init)(void *fdt, const struct fdt_match *match);
> -       int (*vendor_ext_provider)(long extid, long funcid,
> +       int (*vendor_ext_provider)(long funcid,
>                                    const struct sbi_trap_regs *regs,
>                                    unsigned long *out_value,
>                                    struct sbi_trap_info *out_trap,
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 88c3ec7..0b90fd7 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -186,12 +186,12 @@ static bool generic_vendor_ext_check(void)
>                 true : false;
>  }
>
> -static int generic_vendor_ext_provider(long extid, long funcid,
> +static int generic_vendor_ext_provider(long funcid,
>                                        const struct sbi_trap_regs *regs,
>                                        unsigned long *out_value,
>                                        struct sbi_trap_info *out_trap)
>  {
> -       return generic_plat->vendor_ext_provider(extid, funcid, regs,
> +       return generic_plat->vendor_ext_provider(funcid, regs,
>                                                  out_value, out_trap,
>                                                  generic_plat_match);
>  }
> --
> 2.34.1
>


From anup at brainfault.org  Sun Feb 26 22:10:23 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 11:40:23 +0530
Subject: [PATCH v5 1/3] platform: generic: renesas: rzfive: Add support to
 configure the PMA
In-Reply-To: <20230213215111.32017-2-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20230213215111.32017-2-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <CAAhSdy2X8Z_JvM-+yuQnBmyfEvupHn6e-1y2+Vv8wUGVmdt23g@mail.gmail.com>

On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>
> I/O Coherence Port (IOCP) provides an AXI interface for connecting
> external non-caching masters, such as DMA controllers. The accesses
> from IOCP are coherent with D-Caches and L2 Cache.
>
> IOCP is a specification option and is disabled on the Renesas RZ/Five
> SoC due to this reason IP blocks using DMA will fail.
>
> The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA)
> block that allows dynamic adjustment of memory attributes in the runtime.
> It contains a configurable amount of PMA entries implemented as CSR
> registers to control the attributes of memory locations in interest.
> Below are the memory attributes supported:
> * Device, Non-bufferable
> * Device, bufferable
> * Memory, Non-cacheable, Non-bufferable
> * Memory, Non-cacheable, Bufferable
> * Memory, Write-back, No-allocate
> * Memory, Write-back, Read-allocate
> * Memory, Write-back, Write-allocate
> * Memory, Write-back, Read and Write-allocate
>
> More info about PMA (section 10.3):
> Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
>
> As a workaround for SoCs with IOCP disabled CMO needs to be handled by
> software. Firstly OpenSBI configures the memory region as
> "Memory, Non-cacheable, Bufferable" and passes this region as a global
> shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA
> allocations happen from this region and synchronization callbacks are
> implemented to synchronize when doing DMA transactions.
>
> Example PMA region passed as a DT node from OpenSBI:
>     reserved-memory {
>         #address-cells = <2>;
>         #size-cells = <2>;
>         ranges;
>
>         pma_resv0 at 58000000 {
>             compatible = "shared-dma-pool";
>             reg = <0x0 0x58000000 0x0 0x08000000>;
>             no-map;
>             linux,dma-default;
>         };
>     };
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/Kconfig                     |   2 +
>  platform/generic/andes/Kconfig               |   5 +
>  platform/generic/andes/andes45-pma.c         | 350 +++++++++++++++++++
>  platform/generic/andes/objects.mk            |   2 +
>  platform/generic/include/andes/andes45_pma.h |  48 +++
>  5 files changed, 407 insertions(+)
>  create mode 100644 platform/generic/andes/Kconfig
>  create mode 100644 platform/generic/andes/andes45-pma.c
>  create mode 100644 platform/generic/include/andes/andes45_pma.h
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index c7f198a..39fb4e9 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -50,4 +50,6 @@ config PLATFORM_STARFIVE_JH7110
>         bool "StarFive JH7110 support"
>         default n
>
> +source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
> +
>  endif
> diff --git a/platform/generic/andes/Kconfig b/platform/generic/andes/Kconfig
> new file mode 100644
> index 0000000..3ad4e4c
> --- /dev/null
> +++ b/platform/generic/andes/Kconfig
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: BSD-2-Clause
> +
> +config ANDES45_PMA
> +       bool "Andes PMA support"
> +       default n
> diff --git a/platform/generic/andes/andes45-pma.c b/platform/generic/andes/andes45-pma.c
> new file mode 100644
> index 0000000..2745bc3
> --- /dev/null
> +++ b/platform/generic/andes/andes45-pma.c
> @@ -0,0 +1,350 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + *
> + * Copyright (c) 2020 Andes Technology Corporation
> + *
> + * Authors:
> + *      Nick Hu <nickhu at andestech.com>
> + *      Nylon Chen <nylon7 at andestech.com>
> + *      Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> + */
> +
> +#include <andes/andes45_pma.h>
> +#include <libfdt.h>
> +#include <sbi/riscv_asm.h>
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +
> +/* Configuration Registers */
> +#define ANDES45_CSR_MMSC_CFG           0xFC2
> +#define ANDES45_CSR_MMSC_PPMA_OFFSET   (1 << 30)
> +
> +#define ANDES45_PMAADDR_0              0xBD0
> +
> +#define ANDES45_PMACFG_0               0xBC0
> +
> +static inline unsigned long andes45_pma_read_cfg(unsigned int pma_cfg_off)
> +{
> +#define switchcase_pma_cfg_read(__pma_cfg_off, __val)          \
> +       case __pma_cfg_off:                                     \
> +               __val = csr_read(__pma_cfg_off);                \
> +               break;
> +#define switchcase_pma_cfg_read_2(__pma_cfg_off, __val)                \
> +       switchcase_pma_cfg_read(__pma_cfg_off + 0, __val)       \
> +       switchcase_pma_cfg_read(__pma_cfg_off + 2, __val)
> +
> +       unsigned long ret = 0;
> +
> +       switch (pma_cfg_off) {
> +       switchcase_pma_cfg_read_2(ANDES45_PMACFG_0, ret)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> +               break;
> +       }
> +
> +       return ret;
> +
> +#undef switchcase_pma_cfg_read_2
> +#undef switchcase_pma_cfg_read
> +}
> +
> +static inline void andes45_pma_write_cfg(unsigned int pma_cfg_off, unsigned long val)
> +{
> +#define switchcase_pma_cfg_write(__pma_cfg_off, __val)         \
> +       case __pma_cfg_off:                                     \
> +               csr_write(__pma_cfg_off, __val);                \
> +               break;
> +#define switchcase_pma_cfg_write_2(__pma_cfg_off, __val)       \
> +       switchcase_pma_cfg_write(__pma_cfg_off + 0, __val)      \
> +       switchcase_pma_cfg_write(__pma_cfg_off + 2, __val)
> +
> +       switch (pma_cfg_off) {
> +       switchcase_pma_cfg_write_2(ANDES45_PMACFG_0, val)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA CFG offset %#x", __func__, pma_cfg_off);
> +               break;
> +       }
> +
> +#undef switchcase_pma_cfg_write_2
> +#undef switchcase_pma_cfg_write
> +}
> +
> +static inline void andes45_pma_write_addr(unsigned int pma_addr_off, unsigned long val)
> +{
> +#define switchcase_pma_write(__pma_addr_off, __val)            \
> +       case __pma_addr_off:                                    \
> +               csr_write(__pma_addr_off, __val);               \
> +               break;
> +#define switchcase_pma_write_2(__pma_addr_off, __val)          \
> +       switchcase_pma_write(__pma_addr_off + 0, __val)         \
> +       switchcase_pma_write(__pma_addr_off + 1, __val)
> +#define switchcase_pma_write_4(__pma_addr_off, __val)          \
> +       switchcase_pma_write_2(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_write_8(__pma_addr_off, __val)          \
> +       switchcase_pma_write_4(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_write_16(__pma_addr_off, __val)         \
> +       switchcase_pma_write_8(__pma_addr_off + 0, __val)       \
> +       switchcase_pma_write_8(__pma_addr_off + 8, __val)
> +
> +       switch (pma_addr_off) {
> +       switchcase_pma_write_16(ANDES45_PMAADDR_0, val)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> +               break;
> +       }
> +
> +#undef switchcase_pma_write_16
> +#undef switchcase_pma_write_8
> +#undef switchcase_pma_write_4
> +#undef switchcase_pma_write_2
> +#undef switchcase_pma_write
> +}
> +
> +static inline unsigned long andes45_pma_read_addr(unsigned int pma_addr_off)
> +{
> +#define switchcase_pma_read(__pma_addr_off, __val)             \
> +       case __pma_addr_off:                                    \
> +               __val = csr_read(__pma_addr_off);               \
> +               break;
> +#define switchcase_pma_read_2(__pma_addr_off, __val)           \
> +       switchcase_pma_read(__pma_addr_off + 0, __val)          \
> +       switchcase_pma_read(__pma_addr_off + 1, __val)
> +#define switchcase_pma_read_4(__pma_addr_off, __val)           \
> +       switchcase_pma_read_2(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_2(__pma_addr_off + 2, __val)
> +#define switchcase_pma_read_8(__pma_addr_off, __val)           \
> +       switchcase_pma_read_4(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_4(__pma_addr_off + 4, __val)
> +#define switchcase_pma_read_16(__pma_addr_off, __val)          \
> +       switchcase_pma_read_8(__pma_addr_off + 0, __val)        \
> +       switchcase_pma_read_8(__pma_addr_off + 8, __val)
> +
> +       unsigned long ret = 0;
> +
> +       switch (pma_addr_off) {
> +       switchcase_pma_read_16(ANDES45_PMAADDR_0, ret)
> +
> +       default:
> +               sbi_panic("%s: Unknown PMA ADDR offset %#x", __func__, pma_addr_off);
> +               break;
> +       }
> +
> +       return ret;
> +
> +#undef switchcase_pma_read_16
> +#undef switchcase_pma_read_8
> +#undef switchcase_pma_read_4
> +#undef switchcase_pma_read_2
> +#undef switchcase_pma_read
> +}
> +
> +static unsigned long
> +andes45_pma_setup(const struct andes45_pma_region *pma_region,
> +                 unsigned int entry_id)
> +{
> +       unsigned long size = pma_region->size;
> +       unsigned long addr = pma_region->pa;
> +       unsigned int pma_cfg_addr;
> +       unsigned long pmacfg_val;
> +       unsigned long pmaaddr;
> +       char *pmaxcfg;
> +
> +       /* Check for 4KiB granularity */
> +       if (size < (1 << 12))
> +               return SBI_EINVAL;
> +
> +       /* Check size is power of 2 */
> +       if (size & (size - 1))
> +               return SBI_EINVAL;
> +
> +       if (entry_id > 15)
> +               return SBI_EINVAL;
> +
> +       if (!(pma_region->flags & ANDES45_PMACFG_ETYP_NAPOT))
> +               return SBI_EINVAL;
> +
> +       if ((addr & (size - 1)) != 0)
> +               return SBI_EINVAL;
> +
> +       pma_cfg_addr = entry_id / 8 ? ANDES45_PMACFG_0 + 2 : ANDES45_PMACFG_0;
> +       pmacfg_val = andes45_pma_read_cfg(pma_cfg_addr);
> +       pmaxcfg = (char *)&pmacfg_val + (entry_id % 8);
> +       *pmaxcfg = 0;
> +       *pmaxcfg = pma_region->flags;
> +
> +       andes45_pma_write_cfg(pma_cfg_addr, pmacfg_val);
> +
> +       pmaaddr = (addr >> 2) + (size >> 3) - 1;
> +
> +       andes45_pma_write_addr(ANDES45_PMAADDR_0 + entry_id, pmaaddr);
> +
> +       return andes45_pma_read_addr(ANDES45_PMAADDR_0 + entry_id) == pmaaddr ?
> +                       pmaaddr : SBI_EINVAL;
> +}
> +
> +static int andes45_fdt_pma_resv(void *fdt, const struct andes45_pma_region *pma,
> +                               unsigned int index, int parent)
> +{
> +       int na = fdt_address_cells(fdt, 0);
> +       int ns = fdt_size_cells(fdt, 0);
> +       static bool dma_default = false;
> +       fdt32_t addr_high, addr_low;
> +       fdt32_t size_high, size_low;
> +       int subnode, err;
> +       fdt32_t reg[4];
> +       fdt32_t *val;
> +       char name[32];
> +
> +       addr_high = (u64)pma->pa >> 32;
> +       addr_low = pma->pa;
> +       size_high = (u64)pma->size >> 32;
> +       size_low = pma->size;
> +
> +       if (na > 1 && addr_high)
> +               sbi_snprintf(name, sizeof(name),
> +                            "pma_resv%d@%x,%x", index,
> +                            addr_high, addr_low);
> +       else
> +               sbi_snprintf(name, sizeof(name),
> +                            "pma_resv%d@%x", index,
> +                            addr_low);
> +
> +       subnode = fdt_add_subnode(fdt, parent, name);
> +       if (subnode < 0)
> +               return subnode;
> +
> +       if (pma->shared_dma) {
> +               err = fdt_setprop_string(fdt, subnode, "compatible", "shared-dma-pool");
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       if (pma->no_map) {
> +               err = fdt_setprop_empty(fdt, subnode, "no-map");
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       /* Linux allows single linux,dma-default region. */
> +       if (pma->dma_default) {
> +               if (dma_default)
> +                       return SBI_EINVAL;
> +
> +               err = fdt_setprop_empty(fdt, subnode, "linux,dma-default");
> +               if (err < 0)
> +                       return err;
> +               dma_default = true;
> +       }
> +
> +       /* encode the <reg> property value */
> +       val = reg;
> +       if (na > 1)
> +               *val++ = cpu_to_fdt32(addr_high);
> +       *val++ = cpu_to_fdt32(addr_low);
> +       if (ns > 1)
> +               *val++ = cpu_to_fdt32(size_high);
> +       *val++ = cpu_to_fdt32(size_low);
> +
> +       err = fdt_setprop(fdt, subnode, "reg", reg,
> +                         (na + ns) * sizeof(fdt32_t));
> +       if (err < 0)
> +               return err;
> +
> +       return 0;
> +}
> +
> +static int andes45_fdt_reserved_memory_fixup(void *fdt,
> +                                            const struct andes45_pma_region *pma,
> +                                            unsigned int entry)
> +{
> +       int parent;
> +
> +       /* try to locate the reserved memory node */
> +       parent = fdt_path_offset(fdt, "/reserved-memory");
> +       if (parent < 0) {
> +               int na = fdt_address_cells(fdt, 0);
> +               int ns = fdt_size_cells(fdt, 0);
> +               int err;
> +
> +               /* if such node does not exist, create one */
> +               parent = fdt_add_subnode(fdt, 0, "reserved-memory");
> +               if (parent < 0)
> +                       return parent;
> +
> +               err = fdt_setprop_empty(fdt, parent, "ranges");
> +               if (err < 0)
> +                       return err;
> +
> +               err = fdt_setprop_u32(fdt, parent, "#size-cells", ns);
> +               if (err < 0)
> +                       return err;
> +
> +               err = fdt_setprop_u32(fdt, parent, "#address-cells", na);
> +               if (err < 0)
> +                       return err;
> +       }
> +
> +       return andes45_fdt_pma_resv(fdt, pma, entry, parent);
> +}
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> +                             unsigned int pma_regions_count)
> +{
> +       unsigned long mmsc = csr_read(ANDES45_CSR_MMSC_CFG);
> +       unsigned int dt_populate_cnt;
> +       unsigned int i, j;
> +       unsigned long pa;
> +       void *fdt;
> +       int ret;
> +
> +       if (!pma_regions || !pma_regions_count)
> +               return 0;
> +
> +       if (pma_regions_count > ANDES45_MAX_PMA_REGIONS)
> +               return SBI_EINVAL;
> +
> +       if ((mmsc & ANDES45_CSR_MMSC_PPMA_OFFSET) == 0)
> +               return SBI_ENOTSUPP;
> +
> +       /* Configure the PMA regions */
> +       for (i = 0; i < pma_regions_count; i++) {
> +               pa = andes45_pma_setup(&pma_regions[i], i);
> +               if (pa == SBI_EINVAL)
> +                       return SBI_EINVAL;
> +       }
> +
> +       dt_populate_cnt = 0;
> +       for (i = 0; i < pma_regions_count; i++) {
> +               if (!pma_regions[i].dt_populate)
> +                       continue;
> +               dt_populate_cnt++;
> +       }
> +
> +       if (!dt_populate_cnt)
> +               return 0;
> +
> +       fdt = fdt_get_address();
> +
> +       ret = fdt_open_into(fdt, fdt, fdt_totalsize(fdt) + (64 * dt_populate_cnt));
> +       if (ret < 0)
> +               return ret;
> +
> +       for (i = 0, j = 0; i < pma_regions_count; i++) {
> +               if (!pma_regions[i].dt_populate)
> +                       continue;
> +
> +               ret = andes45_fdt_reserved_memory_fixup(fdt, &pma_regions[i], j++);
> +               if (ret)
> +                       return ret;
> +       }
> +
> +       return 0;
> +}
> diff --git a/platform/generic/andes/objects.mk b/platform/generic/andes/objects.mk
> index 28275ef..ea6b561 100644
> --- a/platform/generic/andes/objects.mk
> +++ b/platform/generic/andes/objects.mk
> @@ -4,3 +4,5 @@
>
>  carray-platform_override_modules-$(CONFIG_PLATFORM_ANDES_AE350) += andes_ae350
>  platform-objs-$(CONFIG_PLATFORM_ANDES_AE350) += andes/ae350.o andes/sleep.o
> +
> +platform-objs-$(CONFIG_ANDES45_PMA) += andes/andes45-pma.o
> diff --git a/platform/generic/include/andes/andes45_pma.h b/platform/generic/include/andes/andes45_pma.h
> new file mode 100644
> index 0000000..37ec77c
> --- /dev/null
> +++ b/platform/generic/include/andes/andes45_pma.h
> @@ -0,0 +1,48 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2023 Renesas Electronics Corp.
> + */
> +
> +#ifndef _ANDES45_PMA_H_
> +#define _ANDES45_PMA_H_
> +
> +#include <sbi/sbi_types.h>
> +
> +#define ANDES45_MAX_PMA_REGIONS                        16
> +
> +/* Naturally aligned power of 2 region */
> +#define ANDES45_PMACFG_ETYP_NAPOT              3
> +
> +/* Memory, Non-cacheable, Bufferable */
> +#define ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF  (3 << 2)
> +
> +/**
> + * struct andes45_pma_region - Describes PMA regions
> + *
> + * @pa: Address to be configured in the PMA
> + * @size: Size of the region
> + * @flags: Flags to be set for the PMA region
> + * @dt_populate: Boolean flag indicating if the DT entry should be
> + *               populated for the given PMA region
> + * @shared_dma: Boolean flag if set "shared-dma-pool" property will
> + *              be set in the DT node
> + * @no_map: Boolean flag if set "no-map" property will be set in the
> + *          DT node
> + * @dma_default: Boolean flag if set "linux,dma-default" property will
> + *              be set in the DT node. Note Linux expects single node
> + *              with this property set.
> + */
> +struct andes45_pma_region {
> +       unsigned long pa;
> +       unsigned long size;
> +       u8 flags:7;
> +       bool dt_populate;
> +       bool shared_dma;
> +       bool no_map;
> +       bool dma_default;
> +};
> +
> +int andes45_pma_setup_regions(const struct andes45_pma_region *pma_regions,
> +                             unsigned int pma_regions_count);
> +
> +#endif /* _ANDES45_PMA_H_ */
> --
> 2.17.1
>


From anup at brainfault.org  Sun Feb 26 22:10:55 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 11:40:55 +0530
Subject: [PATCH v5 2/3] platform: generic: renesas: rzfive: Configure the
 PMA region
In-Reply-To: <20230213215111.32017-3-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20230213215111.32017-3-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <CAAhSdy0iKCqAq=HNNTmk21qRYFV0VOQsDfuzrndcw_Tg5botEg@mail.gmail.com>

On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>
> On the Renesas RZ/Five SoC by default we want to configure 128MiB of memory
> ranging from 0x58000000 as a non-cacheable + bufferable region in the PMA
> and populate this region as PMA reserve DT node with shared DMA pool and
> no-map flags set so that Linux drivers requesting any DMA'able memory go
> through this region.
>
> PMA node passed to the above stack:
>
>         reserved-memory {
>             #address-cells = <2>;
>             #size-cells = <2>;
>             ranges;
>
>             pma_resv0 at 58000000 {
>                 compatible = "shared-dma-pool";
>                 reg = <0x0 0x58000000 0x0 0x08000000>;
>                 no-map;
>                 linux,dma-default;
>             };
>         };
>
> Signed-off-by: Lad Prabhakar <prabhakar.mahadev-lad.rj at bp.renesas.com>
> Reviewed-by: Yu Chien Peter Lin <peterlin at andestech.com>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  platform/generic/Kconfig                 |  1 +
>  platform/generic/renesas/rzfive/rzfive.c | 21 +++++++++++++++++++++
>  2 files changed, 22 insertions(+)
>
> diff --git a/platform/generic/Kconfig b/platform/generic/Kconfig
> index 39fb4e9..1f4f8e1 100644
> --- a/platform/generic/Kconfig
> +++ b/platform/generic/Kconfig
> @@ -35,6 +35,7 @@ config PLATFORM_ANDES_AE350
>
>  config PLATFORM_RENESAS_RZFIVE
>         bool "Renesas RZ/Five support"
> +       select ANDES45_PMA
>         default n
>
>  config PLATFORM_SIFIVE_FU540
> diff --git a/platform/generic/renesas/rzfive/rzfive.c b/platform/generic/renesas/rzfive/rzfive.c
> index ee9c9c4..4d71d0d 100644
> --- a/platform/generic/renesas/rzfive/rzfive.c
> +++ b/platform/generic/renesas/rzfive/rzfive.c
> @@ -4,10 +4,30 @@
>   *
>   */
>
> +#include <andes/andes45_pma.h>
>  #include <platform_override.h>
>  #include <sbi/sbi_domain.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
>
> +static const struct andes45_pma_region renesas_rzfive_pma_regions[] = {
> +       {
> +               .pa = 0x58000000,
> +               .size = 0x8000000,
> +               .flags = ANDES45_PMACFG_ETYP_NAPOT |
> +                        ANDES45_PMACFG_MTYP_MEM_NON_CACHE_BUF,
> +               .dt_populate = true,
> +               .shared_dma = true,
> +               .no_map = true,
> +               .dma_default = true,
> +       },
> +};
> +
> +static int renesas_rzfive_final_init(bool cold_boot, const struct fdt_match *match)
> +{
> +       return andes45_pma_setup_regions(renesas_rzfive_pma_regions,
> +                                        array_size(renesas_rzfive_pma_regions));
> +}
> +
>  int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *match)
>  {
>         /*
> @@ -34,4 +54,5 @@ static const struct fdt_match renesas_rzfive_match[] = {
>  const struct platform_override renesas_rzfive = {
>         .match_table = renesas_rzfive_match,
>         .early_init = renesas_rzfive_early_init,
> +       .final_init = renesas_rzfive_final_init,
>  };
> --
> 2.17.1
>


From anup at brainfault.org  Sun Feb 26 22:12:50 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 11:42:50 +0530
Subject: [PATCH v5 0/3] Renesas RZ/Five add support to configure PMA and
 add vendor specific SBI extension
In-Reply-To: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com>
Message-ID: <CAAhSdy3PepnH+xoqWhnurK1Rn+q51vp579FeG3VnZfFV+n-5ww@mail.gmail.com>

On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
<prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
>
> Hi All,
>
> This patch series aims to add support to configure the PMA regions present
> on the Andes AX45MP core (andes45-pma.c which can be shared with other
> Andes vendors) and configures the required PMA region for RZ/Five SoC.
>
> This patch series also adds the SBI vendor extension
> RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to check if IOCP is present on
> the core and if un-available it also checks if SW workaround for cache
> management needs to be applied.
>
> v4 -> v5
> -> Included RB tags from Yu Chien Peter Lin
> -> Moved PMA configuration code into andes folder
> -> Rebased the changes on top of [0]
> [0] https://patchwork.ozlabs.org/project/opensbi/list/?series=341535
>
> RFC v3 -> v4
> -> Used bit field for flags in struct andes45_pma_region
> -> Renamed andes45-pma.h -> andes45_pma.h
> -> Included RB tag from Lin san for patch 2/3
> -> Moved CSR to platform/generic/include/andes/andes45.h
> -> Included sbi/sbi_error.h in andes45-pma.c to fix build issue
>
> RFC v2 -> RFC v3
> -> Fixed review comments pointed by Lin-san
>    * Used switch case macro
>    * Added a check to verify the PMA size is power-of-2
>    * Dropped un-needed calculation done in PMA code for NAPOT.
>    * Dropped MISA macro
> -> Fixed comments pointed by Himanshu
> -> Fixed check patch issue
>
> RFC v2 -> RFC v1
> -> Fixed review comments pointed by Lin-san
> -> Implemented PMA as library so that it can be shared
>
> RFC v1:
> https://patchwork.ozlabs.org/project/opensbi/patch/20221212094421.14556-1-prabhakar.mahadev-lad.rj at bp.renesas.com/
>
> Cheers,
> Prabhakar
>
> Lad Prabhakar (3):
>   platform: generic: renesas: rzfive: Add support to configure the PMA
>   platform: generic: renesas: rzfive: Configure the PMA region
>   platform: generic: renesas: rzfive: Add SBI EXT to check for enabling
>     IOCP errata

Applied first two patches of this series. Please work with Andes folks
and send v6 of PATCH3.

Thanks,
Anup

>
>  platform/generic/Kconfig                     |   3 +
>  platform/generic/andes/Kconfig               |   5 +
>  platform/generic/andes/andes45-pma.c         | 350 +++++++++++++++++++
>  platform/generic/andes/objects.mk            |   2 +
>  platform/generic/include/andes/andes45.h     |  25 +-
>  platform/generic/include/andes/andes45_pma.h |  48 +++
>  platform/generic/renesas/rzfive/rzfive.c     |  62 ++++
>  7 files changed, 493 insertions(+), 2 deletions(-)
>  create mode 100644 platform/generic/andes/Kconfig
>  create mode 100644 platform/generic/andes/andes45-pma.c
>  create mode 100644 platform/generic/include/andes/andes45_pma.h
>
> --
> 2.17.1
>


From anup at brainfault.org  Sun Feb 26 22:23:42 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 11:53:42 +0530
Subject: [PATCH v2] lib: sbi: sbi_ecall: Check the range of SBI error
In-Reply-To: <20230222024854.24876-1-peterlin@andestech.com>
References: <20230222024854.24876-1-peterlin@andestech.com>
Message-ID: <CAAhSdy2LXU77Y0o2BP=0TPTik-6svmwQtVr6m0px7WMRjGcxLw@mail.gmail.com>

On Wed, Feb 22, 2023 at 8:19 AM Yu Chien Peter Lin
<peterlin at andestech.com> wrote:
>
> We should also check if the return error code is greater than 0
> (SBI_SUCCESS), as this is an invalid error.
>
> Signed-off-by: Yu Chien Peter Lin <peterlin at andestech.com>
> Reviewed-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
> Changes v1 -> v2:
> - Include Xiang's Reviewed-by
> - Drop the 'ret -> out_err' rename
> ---
>  lib/sbi/sbi_ecall.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_ecall.c b/lib/sbi/sbi_ecall.c
> index 27ce5d49..d0f01665 100644
> --- a/lib/sbi/sbi_ecall.c
> +++ b/lib/sbi/sbi_ecall.c
> @@ -120,7 +120,7 @@ int sbi_ecall_handler(struct sbi_trap_regs *regs)
>                 trap.epc = regs->mepc;
>                 sbi_trap_redirect(regs, &trap);
>         } else {
> -               if (ret < SBI_LAST_ERR) {
> +               if (ret < SBI_LAST_ERR || SBI_SUCCESS < ret) {
>                         sbi_printf("%s: Invalid error %d for ext=0x%lx "
>                                    "func=0x%lx\n", __func__, ret,
>                                    extension_id, func_id);
> --
> 2.34.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sun Feb 26 22:44:03 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 12:14:03 +0530
Subject: [PATCH] gpio/starfive: add gpio driver and support gpio reset
In-Reply-To: <20230216092126.99022-1-minda.chen@starfivetech.com>
References: <20230216092126.99022-1-minda.chen@starfivetech.com>
Message-ID: <CAAhSdy3gUbX3+54-i6PSo99UMGVE60VSYALRLA5QVazSqLCQmQ@mail.gmail.com>

On Thu, Feb 16, 2023 at 2:52 PM minda.chen <minda.chen at starfivetech.com> wrote:
>
> Add gpio driver and gpio reset function in Starfive
> JH7110 SOC platform.
>
> Signed-off-by: minda.chen <minda.chen at starfivetech.com>

I don't see a PATCH enabling it in defconfig.

How is this driver used in OpenSBI ?

Regards,
Anup

> ---
>  lib/utils/gpio/Kconfig             |   3 +
>  lib/utils/gpio/fdt_gpio_starfive.c | 117 +++++++++++++++++++++++++++++
>  lib/utils/gpio/objects.mk          |   3 +
>  platform/generic/configs/defconfig |   1 +
>  4 files changed, 124 insertions(+)
>  create mode 100644 lib/utils/gpio/fdt_gpio_starfive.c
>
> diff --git a/lib/utils/gpio/Kconfig b/lib/utils/gpio/Kconfig
> index 38a9d75..1b338a6 100644
> --- a/lib/utils/gpio/Kconfig
> +++ b/lib/utils/gpio/Kconfig
> @@ -14,6 +14,9 @@ config FDT_GPIO_SIFIVE
>         bool "SiFive GPIO FDT driver"
>         default n
>
> +config FDT_GPIO_STARFIVE
> +       bool "StarFive GPIO FDT driver"
> +       default n
>  endif
>
>  config GPIO
> diff --git a/lib/utils/gpio/fdt_gpio_starfive.c b/lib/utils/gpio/fdt_gpio_starfive.c
> new file mode 100644
> index 0000000..18cca72
> --- /dev/null
> +++ b/lib/utils/gpio/fdt_gpio_starfive.c
> @@ -0,0 +1,117 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Starfive
> + *
> + * Authors:
> + *   Minda.chen <Minda.chen at starfivetech.com>
> + */
> +
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +#include <sbi_utils/gpio/fdt_gpio.h>
> +
> +#define STARFIVE_GPIO_CHIP_MAX         2
> +#define STARFIVE_GPIO_PINS_DEF         64
> +#define STARFIVE_GPIO_OUTVAL           0x40
> +#define STARFIVE_GPIO_MASK             0xff
> +#define STARFIVE_GPIO_REG_SHIFT_MASK   0x3
> +#define STARFIVE_GPIO_SHIFT_BITS       0x3
> +
> +struct starfive_gpio_chip {
> +       unsigned long addr;
> +       struct gpio_chip chip;
> +};
> +
> +static unsigned int starfive_gpio_chip_count;
> +static struct starfive_gpio_chip starfive_gpio_chip_array[STARFIVE_GPIO_CHIP_MAX];
> +
> +static int starfive_gpio_direction_output(struct gpio_pin *gp, int value)
> +{
> +       u32 val;
> +       unsigned long reg_addr;
> +       u32 bit_mask, shift_bits;
> +       struct starfive_gpio_chip *chip =
> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
> +
> +       /* set out en*/
> +       reg_addr = chip->addr + gp->offset;
> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
> +
> +       val = readl((void *)(reg_addr));
> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
> +               << STARFIVE_GPIO_SHIFT_BITS;
> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
> +
> +       val = readl((void *)reg_addr);
> +       val &= ~bit_mask;
> +       writel(val, (void *)reg_addr);
> +
> +       return 0;
> +}
> +
> +static void starfive_gpio_set(struct gpio_pin *gp, int value)
> +{
> +       u32 val;
> +       unsigned long reg_addr;
> +       u32 bit_mask, shift_bits;
> +       struct starfive_gpio_chip *chip =
> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
> +
> +       reg_addr = chip->addr + gp->offset;
> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
> +
> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
> +               << STARFIVE_GPIO_SHIFT_BITS;
> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
> +       /* set output value */
> +       val = readl((void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
> +       val &= ~bit_mask;
> +       val |= value << shift_bits;
> +       writel(val, (void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
> +}
> +
> +extern struct fdt_gpio fdt_gpio_starfive;
> +
> +static int starfive_gpio_init(void *fdt, int nodeoff, u32 phandle,
> +                             const struct fdt_match *match)
> +{
> +       int rc;
> +       struct starfive_gpio_chip *chip;
> +       u64 addr;
> +
> +       if (starfive_gpio_chip_count >= STARFIVE_GPIO_CHIP_MAX)
> +               return SBI_ENOSPC;
> +       chip = &starfive_gpio_chip_array[starfive_gpio_chip_count];
> +
> +       rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
> +       if (rc)
> +               return rc;
> +
> +       chip->addr = addr;
> +       chip->chip.driver = &fdt_gpio_starfive;
> +       chip->chip.id = phandle;
> +       chip->chip.ngpio = STARFIVE_GPIO_PINS_DEF;
> +       chip->chip.direction_output = starfive_gpio_direction_output;
> +       chip->chip.set = starfive_gpio_set;
> +       rc = gpio_chip_add(&chip->chip);
> +       if (rc)
> +               return rc;
> +
> +       starfive_gpio_chip_count++;
> +       return 0;
> +}
> +
> +static const struct fdt_match starfive_gpio_match[] = {
> +       { .compatible = "starfive,jh7110-sys-pinctrl" },
> +       { .compatible = "starfive,iomux-pinctrl" },
> +       { },
> +};
> +
> +struct fdt_gpio fdt_gpio_starfive = {
> +       .match_table = starfive_gpio_match,
> +       .xlate = fdt_gpio_simple_xlate,
> +       .init = starfive_gpio_init,
> +};
> diff --git a/lib/utils/gpio/objects.mk b/lib/utils/gpio/objects.mk
> index eedd699..8f4a6a8 100644
> --- a/lib/utils/gpio/objects.mk
> +++ b/lib/utils/gpio/objects.mk
> @@ -13,4 +13,7 @@ libsbiutils-objs-$(CONFIG_FDT_GPIO) += gpio/fdt_gpio_drivers.o
>  carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_SIFIVE) += fdt_gpio_sifive
>  libsbiutils-objs-$(CONFIG_FDT_GPIO_SIFIVE) += gpio/fdt_gpio_sifive.o
>
> +carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_STARFIVE) += fdt_gpio_starfive
> +libsbiutils-objs-$(CONFIG_FDT_GPIO_STARFIVE) += gpio/fdt_gpio_starfive.o
> +
>  libsbiutils-objs-$(CONFIG_GPIO) += gpio/gpio.o
> diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
> index 4b0842e..2ad953d 100644
> --- a/platform/generic/configs/defconfig
> +++ b/platform/generic/configs/defconfig
> @@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
>  CONFIG_PLATFORM_STARFIVE_JH7110=y
>  CONFIG_FDT_GPIO=y
>  CONFIG_FDT_GPIO_SIFIVE=y
> +CONFIG_FDT_GPIO_STARFIVE=y
>  CONFIG_FDT_I2C=y
>  CONFIG_FDT_I2C_SIFIVE=y
>  CONFIG_FDT_IPI=y
> --
> 2.17.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From prabhakar.csengg at gmail.com  Mon Feb 27 01:03:43 2023
From: prabhakar.csengg at gmail.com (Lad, Prabhakar)
Date: Mon, 27 Feb 2023 09:03:43 +0000
Subject: [PATCH v5 0/3] Renesas RZ/Five add support to configure PMA and
 add vendor specific SBI extension
In-Reply-To: <CAAhSdy3PepnH+xoqWhnurK1Rn+q51vp579FeG3VnZfFV+n-5ww@mail.gmail.com>
References: <20230213215111.32017-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <CAAhSdy3PepnH+xoqWhnurK1Rn+q51vp579FeG3VnZfFV+n-5ww@mail.gmail.com>
Message-ID: <CA+V-a8v7AGOTTm2J5-rX8Xyhyvt-u97WNShTGm0qfHb7NUE6sA@mail.gmail.com>

Hi Anup,

On Mon, Feb 27, 2023 at 6:13?AM Anup Patel <anup at brainfault.org> wrote:
>
> On Tue, Feb 14, 2023 at 3:21 AM Lad Prabhakar
> <prabhakar.mahadev-lad.rj at bp.renesas.com> wrote:
> >
> > Hi All,
> >
> > This patch series aims to add support to configure the PMA regions present
> > on the Andes AX45MP core (andes45-pma.c which can be shared with other
> > Andes vendors) and configures the required PMA region for RZ/Five SoC.
> >
> > This patch series also adds the SBI vendor extension
> > RENESAS_RZFIVE_SBI_EXT_IOCP_SW_WORKAROUND to check if IOCP is present on
> > the core and if un-available it also checks if SW workaround for cache
> > management needs to be applied.
> >
> > v4 -> v5
> > -> Included RB tags from Yu Chien Peter Lin
> > -> Moved PMA configuration code into andes folder
> > -> Rebased the changes on top of [0]
> > [0] https://patchwork.ozlabs.org/project/opensbi/list/?series=341535
> >
> > RFC v3 -> v4
> > -> Used bit field for flags in struct andes45_pma_region
> > -> Renamed andes45-pma.h -> andes45_pma.h
> > -> Included RB tag from Lin san for patch 2/3
> > -> Moved CSR to platform/generic/include/andes/andes45.h
> > -> Included sbi/sbi_error.h in andes45-pma.c to fix build issue
> >
> > RFC v2 -> RFC v3
> > -> Fixed review comments pointed by Lin-san
> >    * Used switch case macro
> >    * Added a check to verify the PMA size is power-of-2
> >    * Dropped un-needed calculation done in PMA code for NAPOT.
> >    * Dropped MISA macro
> > -> Fixed comments pointed by Himanshu
> > -> Fixed check patch issue
> >
> > RFC v2 -> RFC v1
> > -> Fixed review comments pointed by Lin-san
> > -> Implemented PMA as library so that it can be shared
> >
> > RFC v1:
> > https://patchwork.ozlabs.org/project/opensbi/patch/20221212094421.14556-1-prabhakar.mahadev-lad.rj at bp.renesas.com/
> >
> > Cheers,
> > Prabhakar
> >
> > Lad Prabhakar (3):
> >   platform: generic: renesas: rzfive: Add support to configure the PMA
> >   platform: generic: renesas: rzfive: Configure the PMA region
> >   platform: generic: renesas: rzfive: Add SBI EXT to check for enabling
> >     IOCP errata
>
> Applied first two patches of this series. Please work with Andes folks
> and send v6 of PATCH3.
>
Sure will do, thanks.

Cheers,
Prabhakar


From ajones at ventanamicro.com  Mon Feb 27 02:30:56 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:30:56 +0100
Subject: [PATCH v2 01/11] lib: sbi_hsm: Factor out invalid state detection
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-2-ajones@ventanamicro.com>

Remove some redundant code by creating an invalid state detection
macro.

No functional change intended.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_hsm.c | 65 +++++++++++++++++------------------------------
 1 file changed, 24 insertions(+), 41 deletions(-)

diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index 8499bb1ddeae..b71ee6ba4e80 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -26,6 +26,15 @@
 #include <sbi/sbi_timer.h>
 #include <sbi/sbi_console.h>
 
+#define __sbi_hsm_hart_change_state(hdata, oldstate, newstate)		\
+({									\
+	long state = atomic_cmpxchg(&(hdata)->state, oldstate, newstate); \
+	if (state != (oldstate))					\
+		sbi_printf("%s: ERR: The hart is in invalid state [%lu]\n", \
+			   __func__, state);				\
+	state == (oldstate);						\
+})
+
 static const struct sbi_hsm_device *hsm_dev = NULL;
 static unsigned long hart_data_offset;
 
@@ -95,13 +104,11 @@ int sbi_hsm_hart_interruptible_mask(const struct sbi_domain *dom,
 
 void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
 {
-	u32 oldstate;
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
 
-	oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_START_PENDING,
-				  SBI_HSM_STATE_STARTED);
-	if (oldstate != SBI_HSM_STATE_START_PENDING)
+	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_START_PENDING,
+					 SBI_HSM_STATE_STARTED))
 		sbi_hart_hang();
 }
 
@@ -217,14 +224,12 @@ int sbi_hsm_init(struct sbi_scratch *scratch, u32 hartid, bool cold_boot)
 
 void __noreturn sbi_hsm_exit(struct sbi_scratch *scratch)
 {
-	u32 hstate;
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
 	void (*jump_warmboot)(void) = (void (*)(void))scratch->warmboot_addr;
 
-	hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOP_PENDING,
-				SBI_HSM_STATE_STOPPED);
-	if (hstate != SBI_HSM_STATE_STOP_PENDING)
+	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STOP_PENDING,
+					 SBI_HSM_STATE_STOPPED))
 		goto fail_exit;
 
 	if (hsm_device_has_hart_hotplug()) {
@@ -298,7 +303,6 @@ int sbi_hsm_hart_start(struct sbi_scratch *scratch,
 
 int sbi_hsm_hart_stop(struct sbi_scratch *scratch, bool exitnow)
 {
-	int oldstate;
 	const struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
@@ -306,13 +310,9 @@ int sbi_hsm_hart_stop(struct sbi_scratch *scratch, bool exitnow)
 	if (!dom)
 		return SBI_EFAIL;
 
-	oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STARTED,
-				  SBI_HSM_STATE_STOP_PENDING);
-	if (oldstate != SBI_HSM_STATE_STARTED) {
-		sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
-			   __func__, oldstate);
+	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STARTED,
+					 SBI_HSM_STATE_STOP_PENDING))
 		return SBI_EFAIL;
-	}
 
 	if (exitnow)
 		sbi_exit(scratch);
@@ -362,36 +362,26 @@ static void __sbi_hsm_suspend_non_ret_restore(struct sbi_scratch *scratch)
 
 void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch)
 {
-	int oldstate;
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
 
 	/* If current HART was SUSPENDED then set RESUME_PENDING state */
-	oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_SUSPENDED,
-			SBI_HSM_STATE_RESUME_PENDING);
-	if (oldstate != SBI_HSM_STATE_SUSPENDED) {
-		sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
-			   __func__, oldstate);
+	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_SUSPENDED,
+					 SBI_HSM_STATE_RESUME_PENDING))
 		sbi_hart_hang();
-	}
 
 	hsm_device_hart_resume();
 }
 
 void sbi_hsm_hart_resume_finish(struct sbi_scratch *scratch)
 {
-	u32 oldstate;
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
 
 	/* If current HART was RESUME_PENDING then set STARTED state */
-	oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_RESUME_PENDING,
-				  SBI_HSM_STATE_STARTED);
-	if (oldstate != SBI_HSM_STATE_RESUME_PENDING) {
-		sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
-			   __func__, oldstate);
+	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_RESUME_PENDING,
+					 SBI_HSM_STATE_STARTED))
 		sbi_hart_hang();
-	}
 
 	/*
 	 * Restore some of the M-mode CSRs which we are re-configured by
@@ -403,7 +393,7 @@ void sbi_hsm_hart_resume_finish(struct sbi_scratch *scratch)
 int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 			 ulong raddr, ulong rmode, ulong arg1)
 {
-	int oldstate, ret;
+	int ret;
 	const struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
@@ -437,11 +427,8 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 	scratch->next_mode = rmode;
 
 	/* Directly move from STARTED to SUSPENDED state */
-	oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STARTED,
-				  SBI_HSM_STATE_SUSPENDED);
-	if (oldstate != SBI_HSM_STATE_STARTED) {
-		sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
-			   __func__, oldstate);
+	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STARTED,
+					 SBI_HSM_STATE_SUSPENDED)) {
 		ret = SBI_EDENIED;
 		goto fail_restore_state;
 	}
@@ -484,13 +471,9 @@ fail_restore_state:
 	 * We might have successfully resumed from retentive suspend
 	 * or suspend failed. In both cases, we restore state of hart.
 	 */
-	oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_SUSPENDED,
-				  SBI_HSM_STATE_STARTED);
-	if (oldstate != SBI_HSM_STATE_SUSPENDED) {
-		sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
-			   __func__, oldstate);
+	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_SUSPENDED,
+					 SBI_HSM_STATE_STARTED))
 		sbi_hart_hang();
-	}
 
 	return ret;
 }
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:30:55 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:30:55 +0100
Subject: [PATCH v2 00/11] SBI system suspend (SUSP) extension
Message-ID: <20230227103106.137995-1-ajones@ventanamicro.com>

This series implements the SBI system suspend support in OpenSBI
as-per the draft proposal for SBI system suspend which can be found at
https://github.com/jones-drew/riscv-sbi-doc/commit/39fbea947468e85b0ba92626477747ac7a27b91f

The first half of the series are a few cleanups and fixes for HSM,
as system suspend makes use of some of its code and patterns. The
second half implements support for the extension and includes a patch
which allows testing to be enabled, even when low-level firmware does
not provide support. Regarding the test mode, it's currently enabled
by adding a 'system-suspend-test' DT property, but Anup has suggested
that FW_OPTION could also be used. I'm all ears for what people think
the best way to control test functionality in OpenSBI is and am happy
to switch it. To test with Linux, the kernel must include the patch at
https://github.com/jones-drew/linux/commits/riscv/sbi-susp-rfc and
be built with CONFIG_SUSPEND enabled.

These patches may also be found at
https://github.com/jones-drew/opensbi/commits/susp-v2

v2:
 - Rebased on latest master
 - Added warmboot_addr parameter to platform suspend device's
   system_suspend call. This was suggested by Anup, as not all
   platforms can easily return from this call. These platforms
   may now opt to jump to the warmboot address directly. I kept
   Anup's r-b's since the change is minor and he suggested it.
   Note, the parameter is for convenience and to make the code
   more self-documenting, as a platform could fish the warmboot
   address out of scratch itself.

v1 (changes since RFC):
 - Rebased on latest master
 - Added a print line to sbi_boot_print_general() [Anup]
 - Picked up Anup's r-b's

Andrew Jones (11):
  lib: sbi_hsm: Factor out invalid state detection
  lib: sbi_hsm: Don't try to restore state on failed change
  lib: sbi_hsm: Ensure errors are consistent with spec
  lib: sbi_hsm: Move misplaced comment
  lib: sbi_hsm: Remove unnecessary include
  lib: sbi_hsm: Export some functions
  lib: sbi: Add system suspend skeleton
  lib: sbi: Add system_suspend_allowed domain property
  lib: sbi: Implement system suspend
  docs: Correct opensbi-domain property name
  platform: generic: Add system suspend test

 docs/domain_support.md            |  15 +++-
 include/sbi/sbi_domain.h          |   2 +
 include/sbi/sbi_ecall_interface.h |   8 +++
 include/sbi/sbi_hsm.h             |   4 ++
 include/sbi/sbi_system.h          |  27 +++++++
 lib/sbi/Kconfig                   |   4 ++
 lib/sbi/objects.mk                |   3 +
 lib/sbi/sbi_domain.c              |   4 ++
 lib/sbi/sbi_ecall_hsm.c           |   4 +-
 lib/sbi/sbi_ecall_susp.c          |  48 +++++++++++++
 lib/sbi/sbi_hsm.c                 |  92 +++++++++++-------------
 lib/sbi/sbi_init.c                |   4 ++
 lib/sbi/sbi_system.c              | 112 ++++++++++++++++++++++++++++++
 lib/utils/fdt/fdt_domain.c        |   7 ++
 platform/generic/platform.c       |  20 +++++-
 15 files changed, 297 insertions(+), 57 deletions(-)
 create mode 100644 lib/sbi/sbi_ecall_susp.c

-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:30:57 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:30:57 +0100
Subject: [PATCH v2 02/11] lib: sbi_hsm: Don't try to restore state on failed change
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-3-ajones@ventanamicro.com>

When a state change fails there's no need to restore the original
state as it remains the same.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_hsm.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index b71ee6ba4e80..7ee4159256be 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -428,10 +428,8 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 
 	/* Directly move from STARTED to SUSPENDED state */
 	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STARTED,
-					 SBI_HSM_STATE_SUSPENDED)) {
-		ret = SBI_EDENIED;
-		goto fail_restore_state;
-	}
+					 SBI_HSM_STATE_SUSPENDED))
+		return SBI_EDENIED;
 
 	/* Save the suspend type */
 	hdata->suspend_type = suspend_type;
@@ -466,7 +464,6 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 		jump_warmboot();
 	}
 
-fail_restore_state:
 	/*
 	 * We might have successfully resumed from retentive suspend
 	 * or suspend failed. In both cases, we restore state of hart.
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:30:59 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:30:59 +0100
Subject: [PATCH v2 04/11] lib: sbi_hsm: Move misplaced comment
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-5-ajones@ventanamicro.com>

While non-retentive suspend is not allowed for M-mode, the comment
at the top of sbi_hsm_hart_suspend() implied suspend wasn't allowed
for M-mode at all. Move the comment above the mode check which is
inside a suspend type is non-retentive check.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_hsm.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index 3455520b8a94..294156a9a7bd 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -398,8 +398,6 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
 
-	/* For now, we only allow suspend from S-mode or U-mode. */
-
 	/* Sanity check on domain assigned to current HART */
 	if (!dom)
 		return SBI_EFAIL;
@@ -414,6 +412,10 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 
 	/* Additional sanity check for non-retentive suspend */
 	if (suspend_type & SBI_HSM_SUSP_NON_RET_BIT) {
+		/*
+		 * For now, we only allow non-retentive suspend from
+		 * S-mode or U-mode.
+		 */
 		if (rmode != PRV_S && rmode != PRV_U)
 			return SBI_EFAIL;
 		if (dom && !sbi_domain_check_addr(dom, raddr, rmode,
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:31:00 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:31:00 +0100
Subject: [PATCH v2 05/11] lib: sbi_hsm: Remove unnecessary include
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-6-ajones@ventanamicro.com>

Also remove a superfluous semicolon and add a blank line.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_ecall_hsm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/sbi/sbi_ecall_hsm.c b/lib/sbi/sbi_ecall_hsm.c
index 703a824493b4..f1b41d073b22 100644
--- a/lib/sbi/sbi_ecall_hsm.c
+++ b/lib/sbi/sbi_ecall_hsm.c
@@ -12,7 +12,6 @@
 #include <sbi/sbi_ecall_interface.h>
 #include <sbi/sbi_error.h>
 #include <sbi/sbi_trap.h>
-#include <sbi/sbi_version.h>
 #include <sbi/sbi_hsm.h>
 #include <sbi/sbi_scratch.h>
 #include <sbi/riscv_asm.h>
@@ -45,7 +44,8 @@ static int sbi_ecall_hsm_handler(unsigned long extid, unsigned long funcid,
 		break;
 	default:
 		ret = SBI_ENOTSUPP;
-	};
+	}
+
 	if (ret >= 0) {
 		*out_val = ret;
 		ret = 0;
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:30:58 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:30:58 +0100
Subject: [PATCH v2 03/11] lib: sbi_hsm: Ensure errors are consistent with spec
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-4-ajones@ventanamicro.com>

HSM functions define when SBI_ERR_INVALID_PARAM should be returned.
Ensure it's not used for reasons that don't meet the definitions by
using the catch-all code, SBI_ERR_FAILED, for those reasons instead.
Also, in one case sbi_hart_suspend() may have returned SBI_ERR_DENIED,
which isn't defined for that function at all. Use SBI_ERR_FAILED for
that case too.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_hsm.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index 7ee4159256be..3455520b8a94 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -402,7 +402,7 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 
 	/* Sanity check on domain assigned to current HART */
 	if (!dom)
-		return SBI_EINVAL;
+		return SBI_EFAIL;
 
 	/* Sanity check on suspend type */
 	if (SBI_HSM_SUSPEND_RET_DEFAULT < suspend_type &&
@@ -415,7 +415,7 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 	/* Additional sanity check for non-retentive suspend */
 	if (suspend_type & SBI_HSM_SUSP_NON_RET_BIT) {
 		if (rmode != PRV_S && rmode != PRV_U)
-			return SBI_EINVAL;
+			return SBI_EFAIL;
 		if (dom && !sbi_domain_check_addr(dom, raddr, rmode,
 						  SBI_DOMAIN_EXECUTE))
 			return SBI_EINVALID_ADDR;
@@ -429,7 +429,7 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 	/* Directly move from STARTED to SUSPENDED state */
 	if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STARTED,
 					 SBI_HSM_STATE_SUSPENDED))
-		return SBI_EDENIED;
+		return SBI_EFAIL;
 
 	/* Save the suspend type */
 	hdata->suspend_type = suspend_type;
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:31:01 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:31:01 +0100
Subject: [PATCH v2 06/11] lib: sbi_hsm: Export some functions
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-7-ajones@ventanamicro.com>

A coming patch can make use of a few internal hsm functions if
we export them.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 include/sbi/sbi_hsm.h |  4 ++++
 lib/sbi/sbi_hsm.c     | 12 ++++++++++--
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/include/sbi/sbi_hsm.h b/include/sbi/sbi_hsm.h
index 1170127baa87..c0b483030de8 100644
--- a/include/sbi/sbi_hsm.h
+++ b/include/sbi/sbi_hsm.h
@@ -69,9 +69,13 @@ void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch);
 void sbi_hsm_hart_resume_finish(struct sbi_scratch *scratch);
 int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 			 ulong raddr, ulong rmode, ulong arg1);
+bool sbi_hsm_hart_change_state(struct sbi_scratch *scratch, long oldstate,
+			       long newstate);
+int __sbi_hsm_hart_get_state(u32 hartid);
 int sbi_hsm_hart_get_state(const struct sbi_domain *dom, u32 hartid);
 int sbi_hsm_hart_interruptible_mask(const struct sbi_domain *dom,
 				    ulong hbase, ulong *out_hmask);
+void __sbi_hsm_suspend_non_ret_save(struct sbi_scratch *scratch);
 void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid);
 
 #endif
diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index 294156a9a7bd..3eeeb586cbb8 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -46,7 +46,15 @@ struct sbi_hsm_data {
 	unsigned long saved_mip;
 };
 
-static inline int __sbi_hsm_hart_get_state(u32 hartid)
+bool sbi_hsm_hart_change_state(struct sbi_scratch *scratch, long oldstate,
+			       long newstate)
+{
+	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
+							    hart_data_offset);
+	return __sbi_hsm_hart_change_state(hdata, oldstate, newstate);
+}
+
+int __sbi_hsm_hart_get_state(u32 hartid)
 {
 	struct sbi_hsm_data *hdata;
 	struct sbi_scratch *scratch;
@@ -328,7 +336,7 @@ static int __sbi_hsm_suspend_default(struct sbi_scratch *scratch)
 	return 0;
 }
 
-static void __sbi_hsm_suspend_non_ret_save(struct sbi_scratch *scratch)
+void __sbi_hsm_suspend_non_ret_save(struct sbi_scratch *scratch)
 {
 	struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
 							    hart_data_offset);
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:31:05 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:31:05 +0100
Subject: [PATCH v2 10/11] docs: Correct opensbi-domain property name
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-11-ajones@ventanamicro.com>

Replace the commas with dashes to correct the name.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 docs/domain_support.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/docs/domain_support.md b/docs/domain_support.md
index e2ff896bd550..265d8d85786e 100644
--- a/docs/domain_support.md
+++ b/docs/domain_support.md
@@ -207,9 +207,9 @@ platform support can provide the HART to domain instance assignment using
 platform specific callback.
 
 The HART to domain instance assignment can be parsed from the device tree
-using optional DT property **opensbi,domain** in each CPU DT node. The
-value of DT property **opensbi,domain** is the DT phandle of the domain
-instance DT node. If **opensbi,domain** DT property is not specified then
+using optional DT property **opensbi-domain** in each CPU DT node. The
+value of DT property **opensbi-domain** is the DT phandle of the domain
+instance DT node. If **opensbi-domain** DT property is not specified then
 corresponding HART is assigned to **the ROOT domain**.
 
 ### Domain Configuration Only Accessible to OpenSBI
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:31:02 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:31:02 +0100
Subject: [PATCH v2 07/11] lib: sbi: Add system suspend skeleton
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-8-ajones@ventanamicro.com>

Add the SUSP extension probe and ecall support, but for now the
system suspend function is just a stub.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 include/sbi/sbi_ecall_interface.h |  8 ++++++
 include/sbi/sbi_system.h          | 26 +++++++++++++++++
 lib/sbi/Kconfig                   |  4 +++
 lib/sbi/objects.mk                |  3 ++
 lib/sbi/sbi_ecall_susp.c          | 48 +++++++++++++++++++++++++++++++
 lib/sbi/sbi_init.c                |  4 +++
 lib/sbi/sbi_system.c              | 26 +++++++++++++++++
 7 files changed, 119 insertions(+)
 create mode 100644 lib/sbi/sbi_ecall_susp.c

diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
index 9d6f4743651d..4c378c3e7c30 100644
--- a/include/sbi/sbi_ecall_interface.h
+++ b/include/sbi/sbi_ecall_interface.h
@@ -30,6 +30,7 @@
 #define SBI_EXT_SRST				0x53525354
 #define SBI_EXT_PMU				0x504D55
 #define SBI_EXT_DBCN				0x4442434E
+#define SBI_EXT_SUSP				0x53555350
 
 /* SBI function IDs for BASE extension*/
 #define SBI_EXT_BASE_GET_SPEC_VERSION		0x0
@@ -236,6 +237,13 @@ enum sbi_pmu_ctr_type {
 #define SBI_EXT_DBCN_CONSOLE_READ		0x1
 #define SBI_EXT_DBCN_CONSOLE_WRITE_BYTE		0x2
 
+/* SBI function IDs for SUSP extension */
+#define SBI_EXT_SUSP_SUSPEND			0x0
+
+#define SBI_SUSP_SLEEP_TYPE_SUSPEND		0x0
+#define SBI_SUSP_SLEEP_TYPE_LAST		SBI_SUSP_SLEEP_TYPE_SUSPEND
+#define SBI_SUSP_PLATFORM_SLEEP_START		0x80000000
+
 /* SBI base specification related macros */
 #define SBI_SPEC_VERSION_MAJOR_OFFSET		24
 #define SBI_SPEC_VERSION_MAJOR_MASK		0x7f
diff --git a/include/sbi/sbi_system.h b/include/sbi/sbi_system.h
index 84c281383d8c..d70ef8bb8874 100644
--- a/include/sbi/sbi_system.h
+++ b/include/sbi/sbi_system.h
@@ -43,4 +43,30 @@ bool sbi_system_reset_supported(u32 reset_type, u32 reset_reason);
 
 void __noreturn sbi_system_reset(u32 reset_type, u32 reset_reason);
 
+/** System suspend device */
+struct sbi_system_suspend_device {
+	/** Name of the system suspend device */
+	char name[32];
+
+	/* Check whether sleep type is supported by the device */
+	int (*system_suspend_check)(u32 sleep_type);
+
+	/**
+	 * Suspend the system
+	 *
+	 * @sleep_type: The sleep type identifier passed to the SBI call.
+	 * @warmboot_addr:
+	 *     This is the same as sbi_scratch.warmboot_addr. Some platforms
+	 *     may not be able to return from system_suspend(), so they will
+	 *     jump directly to this address instead. Platforms which can
+	 *     return from system_suspend() may ignore this parameter.
+	 */
+	int (*system_suspend)(u32 sleep_type, unsigned long warmboot_addr);
+};
+
+const struct sbi_system_suspend_device *sbi_system_suspend_get_device(void);
+void sbi_system_suspend_set_device(struct sbi_system_suspend_device *dev);
+bool sbi_system_suspend_supported(u32 sleep_type);
+int sbi_system_suspend(u32 sleep_type, ulong resume_addr, ulong opaque);
+
 #endif
diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
index ef6728bab3ea..7eb3273d546e 100644
--- a/lib/sbi/Kconfig
+++ b/lib/sbi/Kconfig
@@ -22,6 +22,10 @@ config SBI_ECALL_SRST
 	bool "System Reset extension"
 	default y
 
+config SBI_ECALL_SUSP
+	bool "System Suspend extension"
+	default y
+
 config SBI_ECALL_PMU
 	bool "Performance Monitoring Unit extension"
 	default y
diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
index 319f38dce702..770238b25fa6 100644
--- a/lib/sbi/objects.mk
+++ b/lib/sbi/objects.mk
@@ -34,6 +34,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_HSM) += sbi_ecall_hsm.o
 carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_SRST) += ecall_srst
 libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
 
+carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_SUSP) += ecall_susp
+libsbi-objs-$(CONFIG_SBI_ECALL_SUSP) += sbi_ecall_susp.o
+
 carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
 libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
 
diff --git a/lib/sbi/sbi_ecall_susp.c b/lib/sbi/sbi_ecall_susp.c
new file mode 100644
index 000000000000..f20126c49a60
--- /dev/null
+++ b/lib/sbi/sbi_ecall_susp.c
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: BSD-2-Clause
+#include <sbi/sbi_ecall.h>
+#include <sbi/sbi_ecall_interface.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_trap.h>
+#include <sbi/sbi_system.h>
+
+static int sbi_ecall_susp_handler(unsigned long extid, unsigned long funcid,
+				  const struct sbi_trap_regs *regs,
+				  unsigned long *out_val,
+				  struct sbi_trap_info *out_trap)
+{
+	int ret = SBI_ENOTSUPP;
+
+	if (funcid == SBI_EXT_SUSP_SUSPEND)
+		ret = sbi_system_suspend(regs->a0, regs->a1, regs->a2);
+
+	if (ret >= 0) {
+		*out_val = ret;
+		ret = 0;
+	}
+
+	return ret;
+}
+
+static int sbi_ecall_susp_probe(unsigned long extid, unsigned long *out_val)
+{
+	u32 type, count = 0;
+
+	/*
+	 * At least one suspend type should be supported by the
+	 * platform for the SBI SUSP extension to be usable.
+	 */
+	for (type = 0; type <= SBI_SUSP_SLEEP_TYPE_LAST; type++) {
+		if (sbi_system_suspend_supported(type))
+			count++;
+	}
+
+	*out_val = count ? 1 : 0;
+	return 0;
+}
+
+struct sbi_ecall_extension ecall_susp = {
+	.extid_start = SBI_EXT_SUSP,
+	.extid_end = SBI_EXT_SUSP,
+	.handle = sbi_ecall_susp_handler,
+	.probe = sbi_ecall_susp_probe,
+};
diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
index e353c3348303..bc60a4279354 100644
--- a/lib/sbi/sbi_init.c
+++ b/lib/sbi/sbi_init.c
@@ -69,6 +69,7 @@ static void sbi_boot_print_general(struct sbi_scratch *scratch)
 	const struct sbi_timer_device *tdev;
 	const struct sbi_console_device *cdev;
 	const struct sbi_system_reset_device *srdev;
+	const struct sbi_system_suspend_device *susp_dev;
 	const struct sbi_platform *plat = sbi_platform_ptr(scratch);
 
 	if (scratch->options & SBI_SCRATCH_NO_BOOT_PRINTS)
@@ -103,6 +104,9 @@ static void sbi_boot_print_general(struct sbi_scratch *scratch)
 	srdev = sbi_system_reset_get_device(SBI_SRST_RESET_TYPE_SHUTDOWN, 0);
 	sbi_printf("Platform Shutdown Device  : %s\n",
 		   (srdev) ? srdev->name : "---");
+	susp_dev = sbi_system_suspend_get_device();
+	sbi_printf("Platform Suspend Device   : %s\n",
+		   (susp_dev) ? susp_dev->name : "---");
 
 	/* Firmware details */
 	sbi_printf("Firmware Base             : 0x%lx\n", scratch->fw_start);
diff --git a/lib/sbi/sbi_system.c b/lib/sbi/sbi_system.c
index f37c811d0bf3..5c123a6c9d8d 100644
--- a/lib/sbi/sbi_system.c
+++ b/lib/sbi/sbi_system.c
@@ -92,3 +92,29 @@ void __noreturn sbi_system_reset(u32 reset_type, u32 reset_reason)
 	/* If platform specific reset did not work then do sbi_exit() */
 	sbi_exit(scratch);
 }
+
+static const struct sbi_system_suspend_device *suspend_dev = NULL;
+
+const struct sbi_system_suspend_device *sbi_system_suspend_get_device(void)
+{
+	return suspend_dev;
+}
+
+void sbi_system_suspend_set_device(struct sbi_system_suspend_device *dev)
+{
+	if (!dev || suspend_dev)
+		return;
+
+	suspend_dev = dev;
+}
+
+bool sbi_system_suspend_supported(u32 sleep_type)
+{
+	return suspend_dev && suspend_dev->system_suspend_check &&
+	       suspend_dev->system_suspend_check(sleep_type);
+}
+
+int sbi_system_suspend(u32 sleep_type, ulong resume_addr, ulong opaque)
+{
+	return 0;
+}
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:31:04 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:31:04 +0100
Subject: [PATCH v2 09/11] lib: sbi: Implement system suspend
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-10-ajones@ventanamicro.com>

Fill the implementation of the system suspend ecall. A platform
implementation of the suspend callbacks is still required for this
to do anything.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 lib/sbi/sbi_system.c | 57 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_system.c b/lib/sbi/sbi_system.c
index 5c123a6c9d8d..7fa5ea82b7a6 100644
--- a/lib/sbi/sbi_system.c
+++ b/lib/sbi/sbi_system.c
@@ -116,5 +116,60 @@ bool sbi_system_suspend_supported(u32 sleep_type)
 
 int sbi_system_suspend(u32 sleep_type, ulong resume_addr, ulong opaque)
 {
-	return 0;
+	int ret = SBI_ENOTSUPP;
+	const struct sbi_domain *dom = sbi_domain_thishart_ptr();
+	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
+	void (*jump_warmboot)(void) = (void (*)(void))scratch->warmboot_addr;
+	unsigned int hartid = current_hartid();
+	unsigned long prev_mode;
+	unsigned long i;
+
+	if (!dom || !dom->system_suspend_allowed)
+		return SBI_EFAIL;
+
+	if (!suspend_dev || !suspend_dev->system_suspend)
+		return SBI_EFAIL;
+
+	if (!sbi_system_suspend_supported(sleep_type))
+		return SBI_ENOTSUPP;
+
+	prev_mode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >> MSTATUS_MPP_SHIFT;
+	if (prev_mode != PRV_S && prev_mode != PRV_U)
+		return SBI_EFAIL;
+
+	sbi_hartmask_for_each_hart(i, &dom->assigned_harts) {
+		if (i == hartid)
+			continue;
+		if (__sbi_hsm_hart_get_state(i) != SBI_HSM_STATE_STOPPED)
+			return SBI_EFAIL;
+	}
+
+	if (!sbi_domain_check_addr(dom, resume_addr, prev_mode,
+				   SBI_DOMAIN_EXECUTE))
+		return SBI_EINVALID_ADDR;
+
+	if (!sbi_hsm_hart_change_state(scratch, SBI_HSM_STATE_STARTED,
+				       SBI_HSM_STATE_SUSPENDED))
+		return SBI_EFAIL;
+
+	/* Prepare for resume */
+	scratch->next_mode = prev_mode;
+	scratch->next_addr = resume_addr;
+	scratch->next_arg1 = opaque;
+
+	__sbi_hsm_suspend_non_ret_save(scratch);
+
+	/* Suspend */
+	ret = suspend_dev->system_suspend(sleep_type, scratch->warmboot_addr);
+	if (ret != SBI_OK) {
+		if (!sbi_hsm_hart_change_state(scratch, SBI_HSM_STATE_SUSPENDED,
+					       SBI_HSM_STATE_STARTED))
+			sbi_hart_hang();
+		return ret;
+	}
+
+	/* Resume */
+	jump_warmboot();
+
+	__builtin_unreachable();
 }
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:31:03 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:31:03 +0100
Subject: [PATCH v2 08/11] lib: sbi: Add system_suspend_allowed domain property
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-9-ajones@ventanamicro.com>

Only privileged domains should be allowed to suspend the entire
system. Give the root domain this property by default and allow
other domains to be given the property by specifying it in the
DT.

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 docs/domain_support.md     | 5 +++++
 include/sbi/sbi_domain.h   | 2 ++
 lib/sbi/sbi_domain.c       | 4 ++++
 lib/utils/fdt/fdt_domain.c | 7 +++++++
 4 files changed, 18 insertions(+)

diff --git a/docs/domain_support.md b/docs/domain_support.md
index 4ccbcff87195..e2ff896bd550 100644
--- a/docs/domain_support.md
+++ b/docs/domain_support.md
@@ -52,6 +52,7 @@ has following details:
 * **next_mode** - Privilege mode of the next booting stage for this
   domain. This can be either S-mode or U-mode.
 * **system_reset_allowed** - Is domain allowed to reset the system?
+* **system_suspend_allowed** - Is domain allowed to suspend the system?
 
 The memory regions represented by **regions** in **struct sbi_domain** have
 following additional constraints to align with RISC-V PMP requirements:
@@ -91,6 +92,7 @@ following manner:
 * **next_mode** - Next booting stage mode in coldboot HART scratch space
   is the next mode for the ROOT domain
 * **system_reset_allowed** - The ROOT domain is allowed to reset the system
+* **system_suspend_allowed** - The ROOT domain is allowed to suspend the system
 
 Domain Effects
 --------------
@@ -195,6 +197,8 @@ The DT properties of a domain instance DT node are as follows:
   stage mode of coldboot HART** is used as default value.
 * **system-reset-allowed** (Optional) - A boolean flag representing
   whether the domain instance is allowed to do system reset.
+* **system-suspend-allowed** (Optional) - A boolean flag representing
+  whether the domain instance is allowed to do system suspend.
 
 ### Assigning HART To Domain Instance
 
@@ -260,6 +264,7 @@ be done:
                 next-addr = <0x0 0x80100000>;
                 next-mode = <0x0>;
                 system-reset-allowed;
+                system-suspend-allowed;
             };
 
             udomain: untrusted-domain {
diff --git a/include/sbi/sbi_domain.h b/include/sbi/sbi_domain.h
index ab1a944b474b..eaca7f093271 100644
--- a/include/sbi/sbi_domain.h
+++ b/include/sbi/sbi_domain.h
@@ -120,6 +120,8 @@ struct sbi_domain {
 	unsigned long next_mode;
 	/** Is domain allowed to reset the system */
 	bool system_reset_allowed;
+	/** Is domain allowed to suspend the system */
+	bool system_suspend_allowed;
 };
 
 /** The root domain instance */
diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
index dc825b0d95f6..b1cd837e4b70 100644
--- a/lib/sbi/sbi_domain.c
+++ b/lib/sbi/sbi_domain.c
@@ -38,6 +38,7 @@ struct sbi_domain root = {
 	.possible_harts = &root_hmask,
 	.regions = root_memregs,
 	.system_reset_allowed = true,
+	.system_suspend_allowed = true,
 };
 
 bool sbi_domain_is_assigned_hart(const struct sbi_domain *dom, u32 hartid)
@@ -467,6 +468,9 @@ void sbi_domain_dump(const struct sbi_domain *dom, const char *suffix)
 
 	sbi_printf("Domain%d SysReset    %s: %s\n",
 		   dom->index, suffix, (dom->system_reset_allowed) ? "yes" : "no");
+
+	sbi_printf("Domain%d SysSuspend  %s: %s\n",
+		   dom->index, suffix, (dom->system_suspend_allowed) ? "yes" : "no");
 }
 
 void sbi_domain_dump_all(const char *suffix)
diff --git a/lib/utils/fdt/fdt_domain.c b/lib/utils/fdt/fdt_domain.c
index 2b51a8ebe132..33653e2f555f 100644
--- a/lib/utils/fdt/fdt_domain.c
+++ b/lib/utils/fdt/fdt_domain.c
@@ -419,6 +419,13 @@ static int __fdt_parse_domain(void *fdt, int domain_offset, void *opaque)
 	else
 		dom->system_reset_allowed = false;
 
+	/* Read "system-suspend-allowed" DT property */
+	if (fdt_get_property(fdt, domain_offset,
+			     "system-suspend-allowed", NULL))
+		dom->system_suspend_allowed = true;
+	else
+		dom->system_suspend_allowed = false;
+
 	/* Find /cpus DT node */
 	cpus_offset = fdt_path_offset(fdt, "/cpus");
 	if (cpus_offset < 0)
-- 
2.39.1



From ajones at ventanamicro.com  Mon Feb 27 02:31:06 2023
From: ajones at ventanamicro.com (Andrew Jones)
Date: Mon, 27 Feb 2023 11:31:06 +0100
Subject: [PATCH v2 11/11] platform: generic: Add system suspend test
In-Reply-To: <20230227103106.137995-1-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com>
Message-ID: <20230227103106.137995-12-ajones@ventanamicro.com>

When the system-suspend-test property is present in the domain config
node as shown below, implement system suspend with a simple 5 second
delay followed by a WFI. This allows testing system suspend when the
low-level firmware doesn't support it.

  / {
    chosen {
      opensbi-domains {
          compatible = "opensbi,domain,config";
          system-suspend-test;
      };

Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
Reviewed-by: Anup Patel <anup at brainfault.org>
---
 docs/domain_support.md      |  4 ++++
 include/sbi/sbi_system.h    |  1 +
 lib/sbi/sbi_system.c        | 31 +++++++++++++++++++++++++++++++
 platform/generic/platform.c | 20 +++++++++++++++++++-
 4 files changed, 55 insertions(+), 1 deletion(-)

diff --git a/docs/domain_support.md b/docs/domain_support.md
index 265d8d85786e..c13014a5f7cf 100644
--- a/docs/domain_support.md
+++ b/docs/domain_support.md
@@ -126,6 +126,9 @@ The DT properties of a domain configuration DT node are as follows:
 * **compatible** (Mandatory) - The compatible string of the domain
   configuration. This DT property should have value *"opensbi,domain,config"*
 
+* **system-suspend-test** (Optional) - When present, enable a system
+  suspend test implementation which simply waits five seconds and issues a WFI.
+
 ### Domain Memory Region Node
 
 The domain memory region DT node describes details of a memory region and
@@ -234,6 +237,7 @@ be done:
     chosen {
         opensbi-domains {
             compatible = "opensbi,domain,config";
+            system-suspend-test;
 
             tmem: tmem {
                 compatible = "opensbi,domain,memregion";
diff --git a/include/sbi/sbi_system.h b/include/sbi/sbi_system.h
index d70ef8bb8874..b4e86aef8949 100644
--- a/include/sbi/sbi_system.h
+++ b/include/sbi/sbi_system.h
@@ -66,6 +66,7 @@ struct sbi_system_suspend_device {
 
 const struct sbi_system_suspend_device *sbi_system_suspend_get_device(void);
 void sbi_system_suspend_set_device(struct sbi_system_suspend_device *dev);
+void sbi_system_suspend_test_enable(void);
 bool sbi_system_suspend_supported(u32 sleep_type);
 int sbi_system_suspend(u32 sleep_type, ulong resume_addr, ulong opaque);
 
diff --git a/lib/sbi/sbi_system.c b/lib/sbi/sbi_system.c
index 7fa5ea82b7a6..6eaf6c9508e0 100644
--- a/lib/sbi/sbi_system.c
+++ b/lib/sbi/sbi_system.c
@@ -17,6 +17,7 @@
 #include <sbi/sbi_system.h>
 #include <sbi/sbi_ipi.h>
 #include <sbi/sbi_init.h>
+#include <sbi/sbi_timer.h>
 
 static SBI_LIST_HEAD(reset_devices_list);
 
@@ -108,6 +109,36 @@ void sbi_system_suspend_set_device(struct sbi_system_suspend_device *dev)
 	suspend_dev = dev;
 }
 
+static int sbi_system_suspend_test_check(u32 sleep_type)
+{
+	return sleep_type == SBI_SUSP_SLEEP_TYPE_SUSPEND;
+}
+
+static int sbi_system_suspend_test_suspend(u32 sleep_type,
+					   unsigned long warmboot_addr)
+{
+	if (sleep_type != SBI_SUSP_SLEEP_TYPE_SUSPEND)
+		return SBI_EINVAL;
+
+	sbi_timer_mdelay(5000);
+
+	/* Wait for interrupt */
+	wfi();
+
+	return SBI_OK;
+}
+
+static struct sbi_system_suspend_device sbi_system_suspend_test = {
+	.name = "system-suspend-test",
+	.system_suspend_check = sbi_system_suspend_test_check,
+	.system_suspend = sbi_system_suspend_test_suspend,
+};
+
+void sbi_system_suspend_test_enable(void)
+{
+	sbi_system_suspend_set_device(&sbi_system_suspend_test);
+}
+
 bool sbi_system_suspend_supported(u32 sleep_type)
 {
 	return suspend_dev && suspend_dev->system_suspend_check &&
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index 0b90fd7c1446..eeefef4c9533 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -13,6 +13,7 @@
 #include <sbi/sbi_hartmask.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_string.h>
+#include <sbi/sbi_system.h>
 #include <sbi_utils/fdt/fdt_domain.h>
 #include <sbi_utils/fdt/fdt_fixup.h>
 #include <sbi_utils/fdt/fdt_helper.h>
@@ -219,7 +220,24 @@ static int generic_extensions_init(struct sbi_hart_features *hfeatures)
 
 static int generic_domains_init(void)
 {
-	return fdt_domains_populate(fdt_get_address());
+	void *fdt = fdt_get_address();
+	int offset, ret;
+
+	ret = fdt_domains_populate(fdt);
+	if (ret < 0)
+		return ret;
+
+	offset = fdt_path_offset(fdt, "/chosen");
+
+	if (offset >= 0) {
+		offset = fdt_node_offset_by_compatible(fdt, offset,
+						       "opensbi,domain,config");
+		if (offset >= 0 &&
+		    fdt_get_property(fdt, offset, "system-suspend-test", NULL))
+			sbi_system_suspend_test_enable();
+	}
+
+	return 0;
 }
 
 static u64 generic_tlbr_flush_limit(void)
-- 
2.39.1



From minda.chen at starfivetech.com  Mon Feb 27 03:23:40 2023
From: minda.chen at starfivetech.com (Minda Chen)
Date: Mon, 27 Feb 2023 19:23:40 +0800
Subject: [PATCH] gpio/starfive: add gpio driver and support gpio reset
In-Reply-To: <CAAhSdy3gUbX3+54-i6PSo99UMGVE60VSYALRLA5QVazSqLCQmQ@mail.gmail.com>
References: <20230216092126.99022-1-minda.chen@starfivetech.com>
 <CAAhSdy3gUbX3+54-i6PSo99UMGVE60VSYALRLA5QVazSqLCQmQ@mail.gmail.com>
Message-ID: <d0ca6d1c-6761-6b35-6ca8-4244937b1392@starfivetech.com>



On 2023/2/27 14:44, Anup Patel wrote:
> On Thu, Feb 16, 2023 at 2:52 PM minda.chen <minda.chen at starfivetech.com> wrote:
>>
>> Add gpio driver and gpio reset function in Starfive
>> JH7110 SOC platform.
>>
>> Signed-off-by: minda.chen <minda.chen at starfivetech.com>
> 
> I don't see a PATCH enabling it in defconfig.
> 
> How is this driver used in OpenSBI ?
> 
> Regards,
> Anup
> 
I have enabled CONFIG_FDT_GPIO_STARFIVE in defconfig in this patch.

 --- a/platform/generic/configs/defconfig
 +++ b/platform/generic/configs/defconfig
 @@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
  CONFIG_PLATFORM_STARFIVE_JH7110=y
  CONFIG_FDT_GPIO=y
  CONFIG_FDT_GPIO_SIFIVE=y
 +CONFIG_FDT_GPIO_STARFIVE=y

May I set this change to second commit ?

This patch is a gpio reset implementation for sbi ecall SBI_EXT_SRST_RESET.

>> ---
>>  lib/utils/gpio/Kconfig             |   3 +
>>  lib/utils/gpio/fdt_gpio_starfive.c | 117 +++++++++++++++++++++++++++++
>>  lib/utils/gpio/objects.mk          |   3 +
>>  platform/generic/configs/defconfig |   1 +
defconfig have been changed.

>>  4 files changed, 124 insertions(+)
>>  create mode 100644 lib/utils/gpio/fdt_gpio_starfive.c
>>
>> diff --git a/lib/utils/gpio/Kconfig b/lib/utils/gpio/Kconfig
>> index 38a9d75..1b338a6 100644
>> --- a/lib/utils/gpio/Kconfig
>> +++ b/lib/utils/gpio/Kconfig
>> @@ -14,6 +14,9 @@ config FDT_GPIO_SIFIVE
>>         bool "SiFive GPIO FDT driver"
>>         default n
>>
>> +config FDT_GPIO_STARFIVE
>> +       bool "StarFive GPIO FDT driver"
>> +       default n
>>  endif
>>
>>  config GPIO
>> diff --git a/lib/utils/gpio/fdt_gpio_starfive.c b/lib/utils/gpio/fdt_gpio_starfive.c
>> new file mode 100644
>> index 0000000..18cca72
>> --- /dev/null
>> +++ b/lib/utils/gpio/fdt_gpio_starfive.c
>> @@ -0,0 +1,117 @@
>> +/*
>> + * SPDX-License-Identifier: BSD-2-Clause
>> + *
>> + * Copyright (c) 2022 Starfive
>> + *
>> + * Authors:
>> + *   Minda.chen <Minda.chen at starfivetech.com>
>> + */
>> +
>> +#include <sbi/riscv_io.h>
>> +#include <sbi/sbi_error.h>
>> +#include <sbi/sbi_console.h>
>> +#include <sbi_utils/fdt/fdt_helper.h>
>> +#include <sbi_utils/gpio/fdt_gpio.h>
>> +
>> +#define STARFIVE_GPIO_CHIP_MAX         2
>> +#define STARFIVE_GPIO_PINS_DEF         64
>> +#define STARFIVE_GPIO_OUTVAL           0x40
>> +#define STARFIVE_GPIO_MASK             0xff
>> +#define STARFIVE_GPIO_REG_SHIFT_MASK   0x3
>> +#define STARFIVE_GPIO_SHIFT_BITS       0x3
>> +
>> +struct starfive_gpio_chip {
>> +       unsigned long addr;
>> +       struct gpio_chip chip;
>> +};
>> +
>> +static unsigned int starfive_gpio_chip_count;
>> +static struct starfive_gpio_chip starfive_gpio_chip_array[STARFIVE_GPIO_CHIP_MAX];
>> +
>> +static int starfive_gpio_direction_output(struct gpio_pin *gp, int value)
>> +{
>> +       u32 val;
>> +       unsigned long reg_addr;
>> +       u32 bit_mask, shift_bits;
>> +       struct starfive_gpio_chip *chip =
>> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
>> +
>> +       /* set out en*/
>> +       reg_addr = chip->addr + gp->offset;
>> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
>> +
>> +       val = readl((void *)(reg_addr));
>> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
>> +               << STARFIVE_GPIO_SHIFT_BITS;
>> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
>> +
>> +       val = readl((void *)reg_addr);
>> +       val &= ~bit_mask;
>> +       writel(val, (void *)reg_addr);
>> +
>> +       return 0;
>> +}
>> +
>> +static void starfive_gpio_set(struct gpio_pin *gp, int value)
>> +{
>> +       u32 val;
>> +       unsigned long reg_addr;
>> +       u32 bit_mask, shift_bits;
>> +       struct starfive_gpio_chip *chip =
>> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
>> +
>> +       reg_addr = chip->addr + gp->offset;
>> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
>> +
>> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
>> +               << STARFIVE_GPIO_SHIFT_BITS;
>> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
>> +       /* set output value */
>> +       val = readl((void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
>> +       val &= ~bit_mask;
>> +       val |= value << shift_bits;
>> +       writel(val, (void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
>> +}
>> +
>> +extern struct fdt_gpio fdt_gpio_starfive;
>> +
>> +static int starfive_gpio_init(void *fdt, int nodeoff, u32 phandle,
>> +                             const struct fdt_match *match)
>> +{
>> +       int rc;
>> +       struct starfive_gpio_chip *chip;
>> +       u64 addr;
>> +
>> +       if (starfive_gpio_chip_count >= STARFIVE_GPIO_CHIP_MAX)
>> +               return SBI_ENOSPC;
>> +       chip = &starfive_gpio_chip_array[starfive_gpio_chip_count];
>> +
>> +       rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
>> +       if (rc)
>> +               return rc;
>> +
>> +       chip->addr = addr;
>> +       chip->chip.driver = &fdt_gpio_starfive;
>> +       chip->chip.id = phandle;
>> +       chip->chip.ngpio = STARFIVE_GPIO_PINS_DEF;
>> +       chip->chip.direction_output = starfive_gpio_direction_output;
>> +       chip->chip.set = starfive_gpio_set;
>> +       rc = gpio_chip_add(&chip->chip);
>> +       if (rc)
>> +               return rc;
>> +
>> +       starfive_gpio_chip_count++;
>> +       return 0;
>> +}
>> +
>> +static const struct fdt_match starfive_gpio_match[] = {
>> +       { .compatible = "starfive,jh7110-sys-pinctrl" },
>> +       { .compatible = "starfive,iomux-pinctrl" },
>> +       { },
>> +};
>> +
>> +struct fdt_gpio fdt_gpio_starfive = {
>> +       .match_table = starfive_gpio_match,
>> +       .xlate = fdt_gpio_simple_xlate,
>> +       .init = starfive_gpio_init,
>> +};
>> diff --git a/lib/utils/gpio/objects.mk b/lib/utils/gpio/objects.mk
>> index eedd699..8f4a6a8 100644
>> --- a/lib/utils/gpio/objects.mk
>> +++ b/lib/utils/gpio/objects.mk
>> @@ -13,4 +13,7 @@ libsbiutils-objs-$(CONFIG_FDT_GPIO) += gpio/fdt_gpio_drivers.o
>>  carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_SIFIVE) += fdt_gpio_sifive
>>  libsbiutils-objs-$(CONFIG_FDT_GPIO_SIFIVE) += gpio/fdt_gpio_sifive.o
>>
>> +carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_STARFIVE) += fdt_gpio_starfive
>> +libsbiutils-objs-$(CONFIG_FDT_GPIO_STARFIVE) += gpio/fdt_gpio_starfive.o
>> +
>>  libsbiutils-objs-$(CONFIG_GPIO) += gpio/gpio.o
>> diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
>> index 4b0842e..2ad953d 100644
>> --- a/platform/generic/configs/defconfig
>> +++ b/platform/generic/configs/defconfig
>> @@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
>>  CONFIG_PLATFORM_STARFIVE_JH7110=y
>>  CONFIG_FDT_GPIO=y
>>  CONFIG_FDT_GPIO_SIFIVE=y
>> +CONFIG_FDT_GPIO_STARFIVE=y
>>  CONFIG_FDT_I2C=y
>>  CONFIG_FDT_I2C_SIFIVE=y
>>  CONFIG_FDT_IPI=y
>> --
>> 2.17.1
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Mon Feb 27 05:54:13 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:24:13 +0530
Subject: [PATCH 1/3] lib: sbi/sbi_domain: cosmetic style fixes
In-Reply-To: <20230224042827.1768931-1-bmeng@tinylab.org>
References: <20230224042827.1768931-1-bmeng@tinylab.org>
Message-ID: <CAAhSdy1oWPSLCvO-UYTXUc-szijk6qqxZbF7-s0Zp41GWn-qWQ@mail.gmail.com>

On Fri, Feb 24, 2023 at 9:58 AM Bin Meng <bmeng at tinylab.org> wrote:
>
> Minor updates to the comments for language and style fixes.
>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
>  lib/sbi/sbi_domain.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index dc825b0..d2f58a2 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -337,7 +337,7 @@ static int sanitize_domain(const struct sbi_platform *plat,
>         /*
>          * Check next mode
>          *
> -        * We only allow next mode to be S-mode or U-mode.so that we can
> +        * We only allow next mode to be S-mode or U-mode, so that we can
>          * protect M-mode context and enforce checks on memory accesses.
>          */
>         if (dom->next_mode != PRV_S &&
> @@ -347,7 +347,7 @@ static int sanitize_domain(const struct sbi_platform *plat,
>                 return SBI_EINVAL;
>         }
>
> -       /* Check next address and next mode*/
> +       /* Check next address and next mode */
>         if (!sbi_domain_check_addr(dom, dom->next_addr, dom->next_mode,
>                                    SBI_DOMAIN_EXECUTE)) {
>                 sbi_printf("%s: %s next booting stage address 0x%lx can't "
> --
> 2.25.1
>


From anup at brainfault.org  Mon Feb 27 05:54:34 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:24:34 +0530
Subject: [PATCH 2/3] lib: utils/fdt/fdt_domain: Simplify region access
 permission check
In-Reply-To: <20230224042827.1768931-2-bmeng@tinylab.org>
References: <20230224042827.1768931-1-bmeng@tinylab.org> <20230224042827.1768931-2-bmeng@tinylab.org>
Message-ID: <CAAhSdy0zYcBhdONeqzKU-Xr4gDuPnOZwQM++W8neYhC5sh8fKQ@mail.gmail.com>

On Fri, Feb 24, 2023 at 10:01 AM Bin Meng <bmeng at tinylab.org> wrote:
>
> The region access permission check in __fdt_parse_region() can be
> simplified as masking SBI_DOMAIN_MEMREGION_{M,SU}_ACCESS_MASK is
> enough.
>
> While we are here, update the confusing comments to match the codes.
>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
>  lib/utils/fdt/fdt_domain.c | 10 ++++------
>  1 file changed, 4 insertions(+), 6 deletions(-)
>
> diff --git a/lib/utils/fdt/fdt_domain.c b/lib/utils/fdt/fdt_domain.c
> index 2b51a8e..bc30010 100644
> --- a/lib/utils/fdt/fdt_domain.c
> +++ b/lib/utils/fdt/fdt_domain.c
> @@ -244,13 +244,11 @@ static int __fdt_parse_region(void *fdt, int domain_offset,
>          * access permissions. M-mode regions can only be part of
>          * root domain.
>          *
> -        * SU permission bits can't be all zeroes and M-mode permission
> -        * bits must be all set.
> +        * SU permission bits can't be all zeroes when M-mode permission
> +        * bits have at least one bit set.
>          */
> -       if (!((region_access & SBI_DOMAIN_MEMREGION_SU_ACCESS_MASK)
> -            & SBI_DOMAIN_MEMREGION_SU_RWX)
> -           && ((region_access & SBI_DOMAIN_MEMREGION_M_ACCESS_MASK)
> -               & SBI_DOMAIN_MEMREGION_M_RWX))
> +       if (!(region_access & SBI_DOMAIN_MEMREGION_SU_ACCESS_MASK)
> +           && (region_access & SBI_DOMAIN_MEMREGION_M_ACCESS_MASK))
>                 return SBI_EINVAL;
>
>         /* Find next region of the domain */
> --
> 2.25.1
>


From anup at brainfault.org  Mon Feb 27 05:55:15 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:25:15 +0530
Subject: [PATCH 3/3] docs: domain_support: Update the DT example
In-Reply-To: <20230224042827.1768931-3-bmeng@tinylab.org>
References: <20230224042827.1768931-1-bmeng@tinylab.org> <20230224042827.1768931-3-bmeng@tinylab.org>
Message-ID: <CAAhSdy3uT1aEak8VA32H_Skm8BGnqXmNcfz2N2HyDJ9vSSWKBw@mail.gmail.com>

On Fri, Feb 24, 2023 at 10:02 AM Bin Meng <bmeng at tinylab.org> wrote:
>
> commit 3e2f573e707e ("lib: utils: Disallow non-root domains from adding M-mode regions")
> added access permission check in __fdt_parse_region(). With the
> existing DT example in the doc OpenSBI won't boot anymore.
>
> Let's update the DT example so that it can work out of the box.
>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
>  docs/domain_support.md | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/docs/domain_support.md b/docs/domain_support.md
> index 4ccbcff..2a9ca3d 100644
> --- a/docs/domain_support.md
> +++ b/docs/domain_support.md
> @@ -254,7 +254,7 @@ be done:
>              tdomain: trusted-domain {
>                  compatible = "opensbi,domain,instance";
>                  possible-harts = <&cpu0>;
> -                regions = <&tmem 0x7>, <&tuart 0x7>;
> +                regions = <&tmem 0x3f>, <&tuart 0x3f>;
>                  boot-hart = <&cpu0>;
>                  next-arg1 = <0x0 0x0>;
>                  next-addr = <0x0 0x80100000>;
> @@ -265,7 +265,7 @@ be done:
>              udomain: untrusted-domain {
>                  compatible = "opensbi,domain,instance";
>                  possible-harts = <&cpu1 &cpu2 &cpu3 &cpu4>;
> -                regions = <&tmem 0x0>, <&tuart 0x0>, <&allmem 0x7>;
> +                regions = <&tmem 0x0>, <&tuart 0x0>, <&allmem 0x3f>;
>              };
>          };
>      };
> --
> 2.25.1
>


From anup at brainfault.org  Mon Feb 27 06:00:34 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:30:34 +0530
Subject: [PATCH] gpio/starfive: add gpio driver and support gpio reset
In-Reply-To: <d0ca6d1c-6761-6b35-6ca8-4244937b1392@starfivetech.com>
References: <20230216092126.99022-1-minda.chen@starfivetech.com>
 <CAAhSdy3gUbX3+54-i6PSo99UMGVE60VSYALRLA5QVazSqLCQmQ@mail.gmail.com> <d0ca6d1c-6761-6b35-6ca8-4244937b1392@starfivetech.com>
Message-ID: <CAAhSdy1fMYFVddf2UqkY8P0QBexeUt0FZLCg768LMkezUTzXAw@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:53 PM Minda Chen <minda.chen at starfivetech.com> wrote:
>
>
>
> On 2023/2/27 14:44, Anup Patel wrote:
> > On Thu, Feb 16, 2023 at 2:52 PM minda.chen <minda.chen at starfivetech.com> wrote:
> >>
> >> Add gpio driver and gpio reset function in Starfive
> >> JH7110 SOC platform.
> >>
> >> Signed-off-by: minda.chen <minda.chen at starfivetech.com>
> >
> > I don't see a PATCH enabling it in defconfig.
> >
> > How is this driver used in OpenSBI ?
> >
> > Regards,
> > Anup
> >
> I have enabled CONFIG_FDT_GPIO_STARFIVE in defconfig in this patch.

Okay.

>
>  --- a/platform/generic/configs/defconfig
>  +++ b/platform/generic/configs/defconfig
>  @@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
>   CONFIG_PLATFORM_STARFIVE_JH7110=y
>   CONFIG_FDT_GPIO=y
>   CONFIG_FDT_GPIO_SIFIVE=y
>  +CONFIG_FDT_GPIO_STARFIVE=y
>
> May I set this change to second commit ?

No need for a separate patch.

>
> This patch is a gpio reset implementation for sbi ecall SBI_EXT_SRST_RESET.

Fare enough but you also send another series
"Add StarFive JH7110 SoC shutdown and reboot ops"
which provides system reboot based on I2C PMIC.

Basically, we have two separate system reboot ops
for JH7110, is that right ?

Regards,
Anup


>
> >> ---
> >>  lib/utils/gpio/Kconfig             |   3 +
> >>  lib/utils/gpio/fdt_gpio_starfive.c | 117 +++++++++++++++++++++++++++++
> >>  lib/utils/gpio/objects.mk          |   3 +
> >>  platform/generic/configs/defconfig |   1 +
> defconfig have been changed.
>
> >>  4 files changed, 124 insertions(+)
> >>  create mode 100644 lib/utils/gpio/fdt_gpio_starfive.c
> >>
> >> diff --git a/lib/utils/gpio/Kconfig b/lib/utils/gpio/Kconfig
> >> index 38a9d75..1b338a6 100644
> >> --- a/lib/utils/gpio/Kconfig
> >> +++ b/lib/utils/gpio/Kconfig
> >> @@ -14,6 +14,9 @@ config FDT_GPIO_SIFIVE
> >>         bool "SiFive GPIO FDT driver"
> >>         default n
> >>
> >> +config FDT_GPIO_STARFIVE
> >> +       bool "StarFive GPIO FDT driver"
> >> +       default n
> >>  endif
> >>
> >>  config GPIO
> >> diff --git a/lib/utils/gpio/fdt_gpio_starfive.c b/lib/utils/gpio/fdt_gpio_starfive.c
> >> new file mode 100644
> >> index 0000000..18cca72
> >> --- /dev/null
> >> +++ b/lib/utils/gpio/fdt_gpio_starfive.c
> >> @@ -0,0 +1,117 @@
> >> +/*
> >> + * SPDX-License-Identifier: BSD-2-Clause
> >> + *
> >> + * Copyright (c) 2022 Starfive
> >> + *
> >> + * Authors:
> >> + *   Minda.chen <Minda.chen at starfivetech.com>
> >> + */
> >> +
> >> +#include <sbi/riscv_io.h>
> >> +#include <sbi/sbi_error.h>
> >> +#include <sbi/sbi_console.h>
> >> +#include <sbi_utils/fdt/fdt_helper.h>
> >> +#include <sbi_utils/gpio/fdt_gpio.h>
> >> +
> >> +#define STARFIVE_GPIO_CHIP_MAX         2
> >> +#define STARFIVE_GPIO_PINS_DEF         64
> >> +#define STARFIVE_GPIO_OUTVAL           0x40
> >> +#define STARFIVE_GPIO_MASK             0xff
> >> +#define STARFIVE_GPIO_REG_SHIFT_MASK   0x3
> >> +#define STARFIVE_GPIO_SHIFT_BITS       0x3
> >> +
> >> +struct starfive_gpio_chip {
> >> +       unsigned long addr;
> >> +       struct gpio_chip chip;
> >> +};
> >> +
> >> +static unsigned int starfive_gpio_chip_count;
> >> +static struct starfive_gpio_chip starfive_gpio_chip_array[STARFIVE_GPIO_CHIP_MAX];
> >> +
> >> +static int starfive_gpio_direction_output(struct gpio_pin *gp, int value)
> >> +{
> >> +       u32 val;
> >> +       unsigned long reg_addr;
> >> +       u32 bit_mask, shift_bits;
> >> +       struct starfive_gpio_chip *chip =
> >> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
> >> +
> >> +       /* set out en*/
> >> +       reg_addr = chip->addr + gp->offset;
> >> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
> >> +
> >> +       val = readl((void *)(reg_addr));
> >> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
> >> +               << STARFIVE_GPIO_SHIFT_BITS;
> >> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
> >> +
> >> +       val = readl((void *)reg_addr);
> >> +       val &= ~bit_mask;
> >> +       writel(val, (void *)reg_addr);
> >> +
> >> +       return 0;
> >> +}
> >> +
> >> +static void starfive_gpio_set(struct gpio_pin *gp, int value)
> >> +{
> >> +       u32 val;
> >> +       unsigned long reg_addr;
> >> +       u32 bit_mask, shift_bits;
> >> +       struct starfive_gpio_chip *chip =
> >> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
> >> +
> >> +       reg_addr = chip->addr + gp->offset;
> >> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
> >> +
> >> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
> >> +               << STARFIVE_GPIO_SHIFT_BITS;
> >> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
> >> +       /* set output value */
> >> +       val = readl((void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
> >> +       val &= ~bit_mask;
> >> +       val |= value << shift_bits;
> >> +       writel(val, (void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
> >> +}
> >> +
> >> +extern struct fdt_gpio fdt_gpio_starfive;
> >> +
> >> +static int starfive_gpio_init(void *fdt, int nodeoff, u32 phandle,
> >> +                             const struct fdt_match *match)
> >> +{
> >> +       int rc;
> >> +       struct starfive_gpio_chip *chip;
> >> +       u64 addr;
> >> +
> >> +       if (starfive_gpio_chip_count >= STARFIVE_GPIO_CHIP_MAX)
> >> +               return SBI_ENOSPC;
> >> +       chip = &starfive_gpio_chip_array[starfive_gpio_chip_count];
> >> +
> >> +       rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
> >> +       if (rc)
> >> +               return rc;
> >> +
> >> +       chip->addr = addr;
> >> +       chip->chip.driver = &fdt_gpio_starfive;
> >> +       chip->chip.id = phandle;
> >> +       chip->chip.ngpio = STARFIVE_GPIO_PINS_DEF;
> >> +       chip->chip.direction_output = starfive_gpio_direction_output;
> >> +       chip->chip.set = starfive_gpio_set;
> >> +       rc = gpio_chip_add(&chip->chip);
> >> +       if (rc)
> >> +               return rc;
> >> +
> >> +       starfive_gpio_chip_count++;
> >> +       return 0;
> >> +}
> >> +
> >> +static const struct fdt_match starfive_gpio_match[] = {
> >> +       { .compatible = "starfive,jh7110-sys-pinctrl" },
> >> +       { .compatible = "starfive,iomux-pinctrl" },
> >> +       { },
> >> +};
> >> +
> >> +struct fdt_gpio fdt_gpio_starfive = {
> >> +       .match_table = starfive_gpio_match,
> >> +       .xlate = fdt_gpio_simple_xlate,
> >> +       .init = starfive_gpio_init,
> >> +};
> >> diff --git a/lib/utils/gpio/objects.mk b/lib/utils/gpio/objects.mk
> >> index eedd699..8f4a6a8 100644
> >> --- a/lib/utils/gpio/objects.mk
> >> +++ b/lib/utils/gpio/objects.mk
> >> @@ -13,4 +13,7 @@ libsbiutils-objs-$(CONFIG_FDT_GPIO) += gpio/fdt_gpio_drivers.o
> >>  carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_SIFIVE) += fdt_gpio_sifive
> >>  libsbiutils-objs-$(CONFIG_FDT_GPIO_SIFIVE) += gpio/fdt_gpio_sifive.o
> >>
> >> +carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_STARFIVE) += fdt_gpio_starfive
> >> +libsbiutils-objs-$(CONFIG_FDT_GPIO_STARFIVE) += gpio/fdt_gpio_starfive.o
> >> +
> >>  libsbiutils-objs-$(CONFIG_GPIO) += gpio/gpio.o
> >> diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
> >> index 4b0842e..2ad953d 100644
> >> --- a/platform/generic/configs/defconfig
> >> +++ b/platform/generic/configs/defconfig
> >> @@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
> >>  CONFIG_PLATFORM_STARFIVE_JH7110=y
> >>  CONFIG_FDT_GPIO=y
> >>  CONFIG_FDT_GPIO_SIFIVE=y
> >> +CONFIG_FDT_GPIO_STARFIVE=y
> >>  CONFIG_FDT_I2C=y
> >>  CONFIG_FDT_I2C_SIFIVE=y
> >>  CONFIG_FDT_IPI=y
> >> --
> >> 2.17.1
> >>
> >>
> >> --
> >> opensbi mailing list
> >> opensbi at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Mon Feb 27 06:21:48 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:51:48 +0530
Subject: [PATCH v2 01/11] lib: sbi_hsm: Factor out invalid state detection
In-Reply-To: <20230227103106.137995-2-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-2-ajones@ventanamicro.com>
Message-ID: <CAAhSdy0y2w=w-XQTVoE_rhqnybmbcgQt7uE5T9BZptbnrRpafA@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> Remove some redundant code by creating an invalid state detection
> macro.
>
> No functional change intended.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_hsm.c | 65 +++++++++++++++++------------------------------
>  1 file changed, 24 insertions(+), 41 deletions(-)
>
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index 8499bb1ddeae..b71ee6ba4e80 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -26,6 +26,15 @@
>  #include <sbi/sbi_timer.h>
>  #include <sbi/sbi_console.h>
>
> +#define __sbi_hsm_hart_change_state(hdata, oldstate, newstate)         \
> +({                                                                     \
> +       long state = atomic_cmpxchg(&(hdata)->state, oldstate, newstate); \
> +       if (state != (oldstate))                                        \
> +               sbi_printf("%s: ERR: The hart is in invalid state [%lu]\n", \
> +                          __func__, state);                            \
> +       state == (oldstate);                                            \
> +})
> +
>  static const struct sbi_hsm_device *hsm_dev = NULL;
>  static unsigned long hart_data_offset;
>
> @@ -95,13 +104,11 @@ int sbi_hsm_hart_interruptible_mask(const struct sbi_domain *dom,
>
>  void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>  {
> -       u32 oldstate;
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
>
> -       oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_START_PENDING,
> -                                 SBI_HSM_STATE_STARTED);
> -       if (oldstate != SBI_HSM_STATE_START_PENDING)
> +       if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_START_PENDING,
> +                                        SBI_HSM_STATE_STARTED))
>                 sbi_hart_hang();
>  }
>
> @@ -217,14 +224,12 @@ int sbi_hsm_init(struct sbi_scratch *scratch, u32 hartid, bool cold_boot)
>
>  void __noreturn sbi_hsm_exit(struct sbi_scratch *scratch)
>  {
> -       u32 hstate;
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
>         void (*jump_warmboot)(void) = (void (*)(void))scratch->warmboot_addr;
>
> -       hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOP_PENDING,
> -                               SBI_HSM_STATE_STOPPED);
> -       if (hstate != SBI_HSM_STATE_STOP_PENDING)
> +       if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STOP_PENDING,
> +                                        SBI_HSM_STATE_STOPPED))
>                 goto fail_exit;
>
>         if (hsm_device_has_hart_hotplug()) {
> @@ -298,7 +303,6 @@ int sbi_hsm_hart_start(struct sbi_scratch *scratch,
>
>  int sbi_hsm_hart_stop(struct sbi_scratch *scratch, bool exitnow)
>  {
> -       int oldstate;
>         const struct sbi_domain *dom = sbi_domain_thishart_ptr();
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
> @@ -306,13 +310,9 @@ int sbi_hsm_hart_stop(struct sbi_scratch *scratch, bool exitnow)
>         if (!dom)
>                 return SBI_EFAIL;
>
> -       oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STARTED,
> -                                 SBI_HSM_STATE_STOP_PENDING);
> -       if (oldstate != SBI_HSM_STATE_STARTED) {
> -               sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
> -                          __func__, oldstate);
> +       if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STARTED,
> +                                        SBI_HSM_STATE_STOP_PENDING))
>                 return SBI_EFAIL;
> -       }
>
>         if (exitnow)
>                 sbi_exit(scratch);
> @@ -362,36 +362,26 @@ static void __sbi_hsm_suspend_non_ret_restore(struct sbi_scratch *scratch)
>
>  void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch)
>  {
> -       int oldstate;
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
>
>         /* If current HART was SUSPENDED then set RESUME_PENDING state */
> -       oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_SUSPENDED,
> -                       SBI_HSM_STATE_RESUME_PENDING);
> -       if (oldstate != SBI_HSM_STATE_SUSPENDED) {
> -               sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
> -                          __func__, oldstate);
> +       if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_SUSPENDED,
> +                                        SBI_HSM_STATE_RESUME_PENDING))
>                 sbi_hart_hang();
> -       }
>
>         hsm_device_hart_resume();
>  }
>
>  void sbi_hsm_hart_resume_finish(struct sbi_scratch *scratch)
>  {
> -       u32 oldstate;
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
>
>         /* If current HART was RESUME_PENDING then set STARTED state */
> -       oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_RESUME_PENDING,
> -                                 SBI_HSM_STATE_STARTED);
> -       if (oldstate != SBI_HSM_STATE_RESUME_PENDING) {
> -               sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
> -                          __func__, oldstate);
> +       if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_RESUME_PENDING,
> +                                        SBI_HSM_STATE_STARTED))
>                 sbi_hart_hang();
> -       }
>
>         /*
>          * Restore some of the M-mode CSRs which we are re-configured by
> @@ -403,7 +393,7 @@ void sbi_hsm_hart_resume_finish(struct sbi_scratch *scratch)
>  int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>                          ulong raddr, ulong rmode, ulong arg1)
>  {
> -       int oldstate, ret;
> +       int ret;
>         const struct sbi_domain *dom = sbi_domain_thishart_ptr();
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
> @@ -437,11 +427,8 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>         scratch->next_mode = rmode;
>
>         /* Directly move from STARTED to SUSPENDED state */
> -       oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STARTED,
> -                                 SBI_HSM_STATE_SUSPENDED);
> -       if (oldstate != SBI_HSM_STATE_STARTED) {
> -               sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
> -                          __func__, oldstate);
> +       if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STARTED,
> +                                        SBI_HSM_STATE_SUSPENDED)) {
>                 ret = SBI_EDENIED;
>                 goto fail_restore_state;
>         }
> @@ -484,13 +471,9 @@ fail_restore_state:
>          * We might have successfully resumed from retentive suspend
>          * or suspend failed. In both cases, we restore state of hart.
>          */
> -       oldstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_SUSPENDED,
> -                                 SBI_HSM_STATE_STARTED);
> -       if (oldstate != SBI_HSM_STATE_SUSPENDED) {
> -               sbi_printf("%s: ERR: The hart is in invalid state [%u]\n",
> -                          __func__, oldstate);
> +       if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_SUSPENDED,
> +                                        SBI_HSM_STATE_STARTED))
>                 sbi_hart_hang();
> -       }
>
>         return ret;
>  }
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:22:07 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:52:07 +0530
Subject: [PATCH v2 02/11] lib: sbi_hsm: Don't try to restore state on
 failed change
In-Reply-To: <20230227103106.137995-3-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-3-ajones@ventanamicro.com>
Message-ID: <CAAhSdy3SAeOT8rY6b0cOc+COAdPapXGEjtH8H_gMdLMLkHK+-g@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> When a state change fails there's no need to restore the original
> state as it remains the same.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_hsm.c | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index b71ee6ba4e80..7ee4159256be 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -428,10 +428,8 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>
>         /* Directly move from STARTED to SUSPENDED state */
>         if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STARTED,
> -                                        SBI_HSM_STATE_SUSPENDED)) {
> -               ret = SBI_EDENIED;
> -               goto fail_restore_state;
> -       }
> +                                        SBI_HSM_STATE_SUSPENDED))
> +               return SBI_EDENIED;
>
>         /* Save the suspend type */
>         hdata->suspend_type = suspend_type;
> @@ -466,7 +464,6 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>                 jump_warmboot();
>         }
>
> -fail_restore_state:
>         /*
>          * We might have successfully resumed from retentive suspend
>          * or suspend failed. In both cases, we restore state of hart.
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:22:23 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:52:23 +0530
Subject: [PATCH v2 03/11] lib: sbi_hsm: Ensure errors are consistent with spec
In-Reply-To: <20230227103106.137995-4-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-4-ajones@ventanamicro.com>
Message-ID: <CAAhSdy2iOzJi+fwe7_yA8J+d_vw0u42PwRYEf3mYWmbx0zZTtA@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> HSM functions define when SBI_ERR_INVALID_PARAM should be returned.
> Ensure it's not used for reasons that don't meet the definitions by
> using the catch-all code, SBI_ERR_FAILED, for those reasons instead.
> Also, in one case sbi_hart_suspend() may have returned SBI_ERR_DENIED,
> which isn't defined for that function at all. Use SBI_ERR_FAILED for
> that case too.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_hsm.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index 7ee4159256be..3455520b8a94 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -402,7 +402,7 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>
>         /* Sanity check on domain assigned to current HART */
>         if (!dom)
> -               return SBI_EINVAL;
> +               return SBI_EFAIL;
>
>         /* Sanity check on suspend type */
>         if (SBI_HSM_SUSPEND_RET_DEFAULT < suspend_type &&
> @@ -415,7 +415,7 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>         /* Additional sanity check for non-retentive suspend */
>         if (suspend_type & SBI_HSM_SUSP_NON_RET_BIT) {
>                 if (rmode != PRV_S && rmode != PRV_U)
> -                       return SBI_EINVAL;
> +                       return SBI_EFAIL;
>                 if (dom && !sbi_domain_check_addr(dom, raddr, rmode,
>                                                   SBI_DOMAIN_EXECUTE))
>                         return SBI_EINVALID_ADDR;
> @@ -429,7 +429,7 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>         /* Directly move from STARTED to SUSPENDED state */
>         if (!__sbi_hsm_hart_change_state(hdata, SBI_HSM_STATE_STARTED,
>                                          SBI_HSM_STATE_SUSPENDED))
> -               return SBI_EDENIED;
> +               return SBI_EFAIL;
>
>         /* Save the suspend type */
>         hdata->suspend_type = suspend_type;
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:22:42 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:52:42 +0530
Subject: [PATCH v2 04/11] lib: sbi_hsm: Move misplaced comment
In-Reply-To: <20230227103106.137995-5-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-5-ajones@ventanamicro.com>
Message-ID: <CAAhSdy2NAYSqJGorCXMdEoT68q-oFrx6SvGBaaaObgWG0vwHMA@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> While non-retentive suspend is not allowed for M-mode, the comment
> at the top of sbi_hsm_hart_suspend() implied suspend wasn't allowed
> for M-mode at all. Move the comment above the mode check which is
> inside a suspend type is non-retentive check.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_hsm.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index 3455520b8a94..294156a9a7bd 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -398,8 +398,6 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
>
> -       /* For now, we only allow suspend from S-mode or U-mode. */
> -
>         /* Sanity check on domain assigned to current HART */
>         if (!dom)
>                 return SBI_EFAIL;
> @@ -414,6 +412,10 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>
>         /* Additional sanity check for non-retentive suspend */
>         if (suspend_type & SBI_HSM_SUSP_NON_RET_BIT) {
> +               /*
> +                * For now, we only allow non-retentive suspend from
> +                * S-mode or U-mode.
> +                */
>                 if (rmode != PRV_S && rmode != PRV_U)
>                         return SBI_EFAIL;
>                 if (dom && !sbi_domain_check_addr(dom, raddr, rmode,
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:22:57 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:52:57 +0530
Subject: [PATCH v2 05/11] lib: sbi_hsm: Remove unnecessary include
In-Reply-To: <20230227103106.137995-6-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-6-ajones@ventanamicro.com>
Message-ID: <CAAhSdy1EfP1jhFZjLF6YcJHps0PT4xpZY=LHiAjX2RMA05tQTg@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> Also remove a superfluous semicolon and add a blank line.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_ecall_hsm.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/lib/sbi/sbi_ecall_hsm.c b/lib/sbi/sbi_ecall_hsm.c
> index 703a824493b4..f1b41d073b22 100644
> --- a/lib/sbi/sbi_ecall_hsm.c
> +++ b/lib/sbi/sbi_ecall_hsm.c
> @@ -12,7 +12,6 @@
>  #include <sbi/sbi_ecall_interface.h>
>  #include <sbi/sbi_error.h>
>  #include <sbi/sbi_trap.h>
> -#include <sbi/sbi_version.h>
>  #include <sbi/sbi_hsm.h>
>  #include <sbi/sbi_scratch.h>
>  #include <sbi/riscv_asm.h>
> @@ -45,7 +44,8 @@ static int sbi_ecall_hsm_handler(unsigned long extid, unsigned long funcid,
>                 break;
>         default:
>                 ret = SBI_ENOTSUPP;
> -       };
> +       }
> +
>         if (ret >= 0) {
>                 *out_val = ret;
>                 ret = 0;
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:23:27 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:53:27 +0530
Subject: [PATCH v2 06/11] lib: sbi_hsm: Export some functions
In-Reply-To: <20230227103106.137995-7-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-7-ajones@ventanamicro.com>
Message-ID: <CAAhSdy3=ACBCPK64Mn5qb-MUaf_zAoU0RWzmkXtcy24+i3Xwgg@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> A coming patch can make use of a few internal hsm functions if
> we export them.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  include/sbi/sbi_hsm.h |  4 ++++
>  lib/sbi/sbi_hsm.c     | 12 ++++++++++--
>  2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/include/sbi/sbi_hsm.h b/include/sbi/sbi_hsm.h
> index 1170127baa87..c0b483030de8 100644
> --- a/include/sbi/sbi_hsm.h
> +++ b/include/sbi/sbi_hsm.h
> @@ -69,9 +69,13 @@ void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch);
>  void sbi_hsm_hart_resume_finish(struct sbi_scratch *scratch);
>  int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>                          ulong raddr, ulong rmode, ulong arg1);
> +bool sbi_hsm_hart_change_state(struct sbi_scratch *scratch, long oldstate,
> +                              long newstate);
> +int __sbi_hsm_hart_get_state(u32 hartid);
>  int sbi_hsm_hart_get_state(const struct sbi_domain *dom, u32 hartid);
>  int sbi_hsm_hart_interruptible_mask(const struct sbi_domain *dom,
>                                     ulong hbase, ulong *out_hmask);
> +void __sbi_hsm_suspend_non_ret_save(struct sbi_scratch *scratch);
>  void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid);
>
>  #endif
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index 294156a9a7bd..3eeeb586cbb8 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -46,7 +46,15 @@ struct sbi_hsm_data {
>         unsigned long saved_mip;
>  };
>
> -static inline int __sbi_hsm_hart_get_state(u32 hartid)
> +bool sbi_hsm_hart_change_state(struct sbi_scratch *scratch, long oldstate,
> +                              long newstate)
> +{
> +       struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
> +                                                           hart_data_offset);
> +       return __sbi_hsm_hart_change_state(hdata, oldstate, newstate);
> +}
> +
> +int __sbi_hsm_hart_get_state(u32 hartid)
>  {
>         struct sbi_hsm_data *hdata;
>         struct sbi_scratch *scratch;
> @@ -328,7 +336,7 @@ static int __sbi_hsm_suspend_default(struct sbi_scratch *scratch)
>         return 0;
>  }
>
> -static void __sbi_hsm_suspend_non_ret_save(struct sbi_scratch *scratch)
> +void __sbi_hsm_suspend_non_ret_save(struct sbi_scratch *scratch)
>  {
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:24:16 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:54:16 +0530
Subject: [PATCH v2 07/11] lib: sbi: Add system suspend skeleton
In-Reply-To: <20230227103106.137995-8-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-8-ajones@ventanamicro.com>
Message-ID: <CAAhSdy1r9gcwypnA-RG5P3HX-03WEZ1YTGdCv160qTev3e=MxQ@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> Add the SUSP extension probe and ecall support, but for now the
> system suspend function is just a stub.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
>  include/sbi/sbi_ecall_interface.h |  8 ++++++
>  include/sbi/sbi_system.h          | 26 +++++++++++++++++
>  lib/sbi/Kconfig                   |  4 +++
>  lib/sbi/objects.mk                |  3 ++
>  lib/sbi/sbi_ecall_susp.c          | 48 +++++++++++++++++++++++++++++++
>  lib/sbi/sbi_init.c                |  4 +++
>  lib/sbi/sbi_system.c              | 26 +++++++++++++++++
>  7 files changed, 119 insertions(+)
>  create mode 100644 lib/sbi/sbi_ecall_susp.c
>
> diff --git a/include/sbi/sbi_ecall_interface.h b/include/sbi/sbi_ecall_interface.h
> index 9d6f4743651d..4c378c3e7c30 100644
> --- a/include/sbi/sbi_ecall_interface.h
> +++ b/include/sbi/sbi_ecall_interface.h
> @@ -30,6 +30,7 @@
>  #define SBI_EXT_SRST                           0x53525354
>  #define SBI_EXT_PMU                            0x504D55
>  #define SBI_EXT_DBCN                           0x4442434E
> +#define SBI_EXT_SUSP                           0x53555350
>
>  /* SBI function IDs for BASE extension*/
>  #define SBI_EXT_BASE_GET_SPEC_VERSION          0x0
> @@ -236,6 +237,13 @@ enum sbi_pmu_ctr_type {
>  #define SBI_EXT_DBCN_CONSOLE_READ              0x1
>  #define SBI_EXT_DBCN_CONSOLE_WRITE_BYTE                0x2
>
> +/* SBI function IDs for SUSP extension */
> +#define SBI_EXT_SUSP_SUSPEND                   0x0
> +
> +#define SBI_SUSP_SLEEP_TYPE_SUSPEND            0x0
> +#define SBI_SUSP_SLEEP_TYPE_LAST               SBI_SUSP_SLEEP_TYPE_SUSPEND
> +#define SBI_SUSP_PLATFORM_SLEEP_START          0x80000000
> +
>  /* SBI base specification related macros */
>  #define SBI_SPEC_VERSION_MAJOR_OFFSET          24
>  #define SBI_SPEC_VERSION_MAJOR_MASK            0x7f
> diff --git a/include/sbi/sbi_system.h b/include/sbi/sbi_system.h
> index 84c281383d8c..d70ef8bb8874 100644
> --- a/include/sbi/sbi_system.h
> +++ b/include/sbi/sbi_system.h
> @@ -43,4 +43,30 @@ bool sbi_system_reset_supported(u32 reset_type, u32 reset_reason);
>
>  void __noreturn sbi_system_reset(u32 reset_type, u32 reset_reason);
>
> +/** System suspend device */
> +struct sbi_system_suspend_device {
> +       /** Name of the system suspend device */
> +       char name[32];
> +
> +       /* Check whether sleep type is supported by the device */
> +       int (*system_suspend_check)(u32 sleep_type);
> +
> +       /**
> +        * Suspend the system
> +        *
> +        * @sleep_type: The sleep type identifier passed to the SBI call.
> +        * @warmboot_addr:

I have renamed this parameter to mmode_resume_addr which is more
appropriate.

> +        *     This is the same as sbi_scratch.warmboot_addr. Some platforms
> +        *     may not be able to return from system_suspend(), so they will
> +        *     jump directly to this address instead. Platforms which can
> +        *     return from system_suspend() may ignore this parameter.
> +        */
> +       int (*system_suspend)(u32 sleep_type, unsigned long warmboot_addr);
> +};
> +
> +const struct sbi_system_suspend_device *sbi_system_suspend_get_device(void);
> +void sbi_system_suspend_set_device(struct sbi_system_suspend_device *dev);
> +bool sbi_system_suspend_supported(u32 sleep_type);
> +int sbi_system_suspend(u32 sleep_type, ulong resume_addr, ulong opaque);
> +
>  #endif
> diff --git a/lib/sbi/Kconfig b/lib/sbi/Kconfig
> index ef6728bab3ea..7eb3273d546e 100644
> --- a/lib/sbi/Kconfig
> +++ b/lib/sbi/Kconfig
> @@ -22,6 +22,10 @@ config SBI_ECALL_SRST
>         bool "System Reset extension"
>         default y
>
> +config SBI_ECALL_SUSP
> +       bool "System Suspend extension"
> +       default y
> +
>  config SBI_ECALL_PMU
>         bool "Performance Monitoring Unit extension"
>         default y
> diff --git a/lib/sbi/objects.mk b/lib/sbi/objects.mk
> index 319f38dce702..770238b25fa6 100644
> --- a/lib/sbi/objects.mk
> +++ b/lib/sbi/objects.mk
> @@ -34,6 +34,9 @@ libsbi-objs-$(CONFIG_SBI_ECALL_HSM) += sbi_ecall_hsm.o
>  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_SRST) += ecall_srst
>  libsbi-objs-$(CONFIG_SBI_ECALL_SRST) += sbi_ecall_srst.o
>
> +carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_SUSP) += ecall_susp
> +libsbi-objs-$(CONFIG_SBI_ECALL_SUSP) += sbi_ecall_susp.o
> +
>  carray-sbi_ecall_exts-$(CONFIG_SBI_ECALL_PMU) += ecall_pmu
>  libsbi-objs-$(CONFIG_SBI_ECALL_PMU) += sbi_ecall_pmu.o
>
> diff --git a/lib/sbi/sbi_ecall_susp.c b/lib/sbi/sbi_ecall_susp.c
> new file mode 100644
> index 000000000000..f20126c49a60
> --- /dev/null
> +++ b/lib/sbi/sbi_ecall_susp.c
> @@ -0,0 +1,48 @@
> +// SPDX-License-Identifier: BSD-2-Clause
> +#include <sbi/sbi_ecall.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_trap.h>
> +#include <sbi/sbi_system.h>
> +
> +static int sbi_ecall_susp_handler(unsigned long extid, unsigned long funcid,
> +                                 const struct sbi_trap_regs *regs,
> +                                 unsigned long *out_val,
> +                                 struct sbi_trap_info *out_trap)
> +{
> +       int ret = SBI_ENOTSUPP;
> +
> +       if (funcid == SBI_EXT_SUSP_SUSPEND)
> +               ret = sbi_system_suspend(regs->a0, regs->a1, regs->a2);
> +
> +       if (ret >= 0) {
> +               *out_val = ret;
> +               ret = 0;
> +       }
> +
> +       return ret;
> +}
> +
> +static int sbi_ecall_susp_probe(unsigned long extid, unsigned long *out_val)
> +{
> +       u32 type, count = 0;
> +
> +       /*
> +        * At least one suspend type should be supported by the
> +        * platform for the SBI SUSP extension to be usable.
> +        */
> +       for (type = 0; type <= SBI_SUSP_SLEEP_TYPE_LAST; type++) {
> +               if (sbi_system_suspend_supported(type))
> +                       count++;
> +       }
> +
> +       *out_val = count ? 1 : 0;
> +       return 0;
> +}
> +
> +struct sbi_ecall_extension ecall_susp = {
> +       .extid_start = SBI_EXT_SUSP,
> +       .extid_end = SBI_EXT_SUSP,
> +       .handle = sbi_ecall_susp_handler,
> +       .probe = sbi_ecall_susp_probe,
> +};
> diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
> index e353c3348303..bc60a4279354 100644
> --- a/lib/sbi/sbi_init.c
> +++ b/lib/sbi/sbi_init.c
> @@ -69,6 +69,7 @@ static void sbi_boot_print_general(struct sbi_scratch *scratch)
>         const struct sbi_timer_device *tdev;
>         const struct sbi_console_device *cdev;
>         const struct sbi_system_reset_device *srdev;
> +       const struct sbi_system_suspend_device *susp_dev;
>         const struct sbi_platform *plat = sbi_platform_ptr(scratch);
>
>         if (scratch->options & SBI_SCRATCH_NO_BOOT_PRINTS)
> @@ -103,6 +104,9 @@ static void sbi_boot_print_general(struct sbi_scratch *scratch)
>         srdev = sbi_system_reset_get_device(SBI_SRST_RESET_TYPE_SHUTDOWN, 0);
>         sbi_printf("Platform Shutdown Device  : %s\n",
>                    (srdev) ? srdev->name : "---");
> +       susp_dev = sbi_system_suspend_get_device();
> +       sbi_printf("Platform Suspend Device   : %s\n",
> +                  (susp_dev) ? susp_dev->name : "---");
>
>         /* Firmware details */
>         sbi_printf("Firmware Base             : 0x%lx\n", scratch->fw_start);
> diff --git a/lib/sbi/sbi_system.c b/lib/sbi/sbi_system.c
> index f37c811d0bf3..5c123a6c9d8d 100644
> --- a/lib/sbi/sbi_system.c
> +++ b/lib/sbi/sbi_system.c
> @@ -92,3 +92,29 @@ void __noreturn sbi_system_reset(u32 reset_type, u32 reset_reason)
>         /* If platform specific reset did not work then do sbi_exit() */
>         sbi_exit(scratch);
>  }
> +
> +static const struct sbi_system_suspend_device *suspend_dev = NULL;
> +
> +const struct sbi_system_suspend_device *sbi_system_suspend_get_device(void)
> +{
> +       return suspend_dev;
> +}
> +
> +void sbi_system_suspend_set_device(struct sbi_system_suspend_device *dev)
> +{
> +       if (!dev || suspend_dev)
> +               return;
> +
> +       suspend_dev = dev;
> +}
> +
> +bool sbi_system_suspend_supported(u32 sleep_type)
> +{
> +       return suspend_dev && suspend_dev->system_suspend_check &&
> +              suspend_dev->system_suspend_check(sleep_type);
> +}
> +
> +int sbi_system_suspend(u32 sleep_type, ulong resume_addr, ulong opaque)
> +{
> +       return 0;
> +}
> --
> 2.39.1
>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup


From anup at brainfault.org  Mon Feb 27 06:24:31 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:54:31 +0530
Subject: [PATCH v2 08/11] lib: sbi: Add system_suspend_allowed domain property
In-Reply-To: <20230227103106.137995-9-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-9-ajones@ventanamicro.com>
Message-ID: <CAAhSdy2pHqKETGkQ-1OnAMMzvOpjZ+Xwmyna6syMdYw3mJiqnw@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> Only privileged domains should be allowed to suspend the entire
> system. Give the root domain this property by default and allow
> other domains to be given the property by specifying it in the
> DT.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  docs/domain_support.md     | 5 +++++
>  include/sbi/sbi_domain.h   | 2 ++
>  lib/sbi/sbi_domain.c       | 4 ++++
>  lib/utils/fdt/fdt_domain.c | 7 +++++++
>  4 files changed, 18 insertions(+)
>
> diff --git a/docs/domain_support.md b/docs/domain_support.md
> index 4ccbcff87195..e2ff896bd550 100644
> --- a/docs/domain_support.md
> +++ b/docs/domain_support.md
> @@ -52,6 +52,7 @@ has following details:
>  * **next_mode** - Privilege mode of the next booting stage for this
>    domain. This can be either S-mode or U-mode.
>  * **system_reset_allowed** - Is domain allowed to reset the system?
> +* **system_suspend_allowed** - Is domain allowed to suspend the system?
>
>  The memory regions represented by **regions** in **struct sbi_domain** have
>  following additional constraints to align with RISC-V PMP requirements:
> @@ -91,6 +92,7 @@ following manner:
>  * **next_mode** - Next booting stage mode in coldboot HART scratch space
>    is the next mode for the ROOT domain
>  * **system_reset_allowed** - The ROOT domain is allowed to reset the system
> +* **system_suspend_allowed** - The ROOT domain is allowed to suspend the system
>
>  Domain Effects
>  --------------
> @@ -195,6 +197,8 @@ The DT properties of a domain instance DT node are as follows:
>    stage mode of coldboot HART** is used as default value.
>  * **system-reset-allowed** (Optional) - A boolean flag representing
>    whether the domain instance is allowed to do system reset.
> +* **system-suspend-allowed** (Optional) - A boolean flag representing
> +  whether the domain instance is allowed to do system suspend.
>
>  ### Assigning HART To Domain Instance
>
> @@ -260,6 +264,7 @@ be done:
>                  next-addr = <0x0 0x80100000>;
>                  next-mode = <0x0>;
>                  system-reset-allowed;
> +                system-suspend-allowed;
>              };
>
>              udomain: untrusted-domain {
> diff --git a/include/sbi/sbi_domain.h b/include/sbi/sbi_domain.h
> index ab1a944b474b..eaca7f093271 100644
> --- a/include/sbi/sbi_domain.h
> +++ b/include/sbi/sbi_domain.h
> @@ -120,6 +120,8 @@ struct sbi_domain {
>         unsigned long next_mode;
>         /** Is domain allowed to reset the system */
>         bool system_reset_allowed;
> +       /** Is domain allowed to suspend the system */
> +       bool system_suspend_allowed;
>  };
>
>  /** The root domain instance */
> diff --git a/lib/sbi/sbi_domain.c b/lib/sbi/sbi_domain.c
> index dc825b0d95f6..b1cd837e4b70 100644
> --- a/lib/sbi/sbi_domain.c
> +++ b/lib/sbi/sbi_domain.c
> @@ -38,6 +38,7 @@ struct sbi_domain root = {
>         .possible_harts = &root_hmask,
>         .regions = root_memregs,
>         .system_reset_allowed = true,
> +       .system_suspend_allowed = true,
>  };
>
>  bool sbi_domain_is_assigned_hart(const struct sbi_domain *dom, u32 hartid)
> @@ -467,6 +468,9 @@ void sbi_domain_dump(const struct sbi_domain *dom, const char *suffix)
>
>         sbi_printf("Domain%d SysReset    %s: %s\n",
>                    dom->index, suffix, (dom->system_reset_allowed) ? "yes" : "no");
> +
> +       sbi_printf("Domain%d SysSuspend  %s: %s\n",
> +                  dom->index, suffix, (dom->system_suspend_allowed) ? "yes" : "no");
>  }
>
>  void sbi_domain_dump_all(const char *suffix)
> diff --git a/lib/utils/fdt/fdt_domain.c b/lib/utils/fdt/fdt_domain.c
> index 2b51a8ebe132..33653e2f555f 100644
> --- a/lib/utils/fdt/fdt_domain.c
> +++ b/lib/utils/fdt/fdt_domain.c
> @@ -419,6 +419,13 @@ static int __fdt_parse_domain(void *fdt, int domain_offset, void *opaque)
>         else
>                 dom->system_reset_allowed = false;
>
> +       /* Read "system-suspend-allowed" DT property */
> +       if (fdt_get_property(fdt, domain_offset,
> +                            "system-suspend-allowed", NULL))
> +               dom->system_suspend_allowed = true;
> +       else
> +               dom->system_suspend_allowed = false;
> +
>         /* Find /cpus DT node */
>         cpus_offset = fdt_path_offset(fdt, "/cpus");
>         if (cpus_offset < 0)
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:24:46 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:54:46 +0530
Subject: [PATCH v2 09/11] lib: sbi: Implement system suspend
In-Reply-To: <20230227103106.137995-10-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-10-ajones@ventanamicro.com>
Message-ID: <CAAhSdy07PSoxmiRi3-+vT70iBQAjkLq+1tK75biSLMwm8H_qKw@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> Fill the implementation of the system suspend ecall. A platform
> implementation of the suspend callbacks is still required for this
> to do anything.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_system.c | 57 +++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 56 insertions(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_system.c b/lib/sbi/sbi_system.c
> index 5c123a6c9d8d..7fa5ea82b7a6 100644
> --- a/lib/sbi/sbi_system.c
> +++ b/lib/sbi/sbi_system.c
> @@ -116,5 +116,60 @@ bool sbi_system_suspend_supported(u32 sleep_type)
>
>  int sbi_system_suspend(u32 sleep_type, ulong resume_addr, ulong opaque)
>  {
> -       return 0;
> +       int ret = SBI_ENOTSUPP;
> +       const struct sbi_domain *dom = sbi_domain_thishart_ptr();
> +       struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> +       void (*jump_warmboot)(void) = (void (*)(void))scratch->warmboot_addr;
> +       unsigned int hartid = current_hartid();
> +       unsigned long prev_mode;
> +       unsigned long i;
> +
> +       if (!dom || !dom->system_suspend_allowed)
> +               return SBI_EFAIL;
> +
> +       if (!suspend_dev || !suspend_dev->system_suspend)
> +               return SBI_EFAIL;
> +
> +       if (!sbi_system_suspend_supported(sleep_type))
> +               return SBI_ENOTSUPP;
> +
> +       prev_mode = (csr_read(CSR_MSTATUS) & MSTATUS_MPP) >> MSTATUS_MPP_SHIFT;
> +       if (prev_mode != PRV_S && prev_mode != PRV_U)
> +               return SBI_EFAIL;
> +
> +       sbi_hartmask_for_each_hart(i, &dom->assigned_harts) {
> +               if (i == hartid)
> +                       continue;
> +               if (__sbi_hsm_hart_get_state(i) != SBI_HSM_STATE_STOPPED)
> +                       return SBI_EFAIL;
> +       }
> +
> +       if (!sbi_domain_check_addr(dom, resume_addr, prev_mode,
> +                                  SBI_DOMAIN_EXECUTE))
> +               return SBI_EINVALID_ADDR;
> +
> +       if (!sbi_hsm_hart_change_state(scratch, SBI_HSM_STATE_STARTED,
> +                                      SBI_HSM_STATE_SUSPENDED))
> +               return SBI_EFAIL;
> +
> +       /* Prepare for resume */
> +       scratch->next_mode = prev_mode;
> +       scratch->next_addr = resume_addr;
> +       scratch->next_arg1 = opaque;
> +
> +       __sbi_hsm_suspend_non_ret_save(scratch);
> +
> +       /* Suspend */
> +       ret = suspend_dev->system_suspend(sleep_type, scratch->warmboot_addr);
> +       if (ret != SBI_OK) {
> +               if (!sbi_hsm_hart_change_state(scratch, SBI_HSM_STATE_SUSPENDED,
> +                                              SBI_HSM_STATE_STARTED))
> +                       sbi_hart_hang();
> +               return ret;
> +       }
> +
> +       /* Resume */
> +       jump_warmboot();
> +
> +       __builtin_unreachable();
>  }
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:25:02 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:55:02 +0530
Subject: [PATCH v2 10/11] docs: Correct opensbi-domain property name
In-Reply-To: <20230227103106.137995-11-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-11-ajones@ventanamicro.com>
Message-ID: <CAAhSdy1Z-qTB7_et327LXtc8os1QEc3nB8FjzJnHjdBScp59cA@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> Replace the commas with dashes to correct the name.
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  docs/domain_support.md | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/docs/domain_support.md b/docs/domain_support.md
> index e2ff896bd550..265d8d85786e 100644
> --- a/docs/domain_support.md
> +++ b/docs/domain_support.md
> @@ -207,9 +207,9 @@ platform support can provide the HART to domain instance assignment using
>  platform specific callback.
>
>  The HART to domain instance assignment can be parsed from the device tree
> -using optional DT property **opensbi,domain** in each CPU DT node. The
> -value of DT property **opensbi,domain** is the DT phandle of the domain
> -instance DT node. If **opensbi,domain** DT property is not specified then
> +using optional DT property **opensbi-domain** in each CPU DT node. The
> +value of DT property **opensbi-domain** is the DT phandle of the domain
> +instance DT node. If **opensbi-domain** DT property is not specified then
>  corresponding HART is assigned to **the ROOT domain**.
>
>  ### Domain Configuration Only Accessible to OpenSBI
> --
> 2.39.1
>


From anup at brainfault.org  Mon Feb 27 06:25:58 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 19:55:58 +0530
Subject: [PATCH v2 11/11] platform: generic: Add system suspend test
In-Reply-To: <20230227103106.137995-12-ajones@ventanamicro.com>
References: <20230227103106.137995-1-ajones@ventanamicro.com> <20230227103106.137995-12-ajones@ventanamicro.com>
Message-ID: <CAAhSdy2o5PEZZ0XjrncRxoALr_9we9Xe0zFC3Q7hq1Kzz=ePRw@mail.gmail.com>

On Mon, Feb 27, 2023 at 4:01 PM Andrew Jones <ajones at ventanamicro.com> wrote:
>
> When the system-suspend-test property is present in the domain config
> node as shown below, implement system suspend with a simple 5 second
> delay followed by a WFI. This allows testing system suspend when the
> low-level firmware doesn't support it.
>
>   / {
>     chosen {
>       opensbi-domains {
>           compatible = "opensbi,domain,config";
>           system-suspend-test;
>       };
>
> Signed-off-by: Andrew Jones <ajones at ventanamicro.com>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> ---
>  docs/domain_support.md      |  4 ++++
>  include/sbi/sbi_system.h    |  1 +
>  lib/sbi/sbi_system.c        | 31 +++++++++++++++++++++++++++++++
>  platform/generic/platform.c | 20 +++++++++++++++++++-
>  4 files changed, 55 insertions(+), 1 deletion(-)
>
> diff --git a/docs/domain_support.md b/docs/domain_support.md
> index 265d8d85786e..c13014a5f7cf 100644
> --- a/docs/domain_support.md
> +++ b/docs/domain_support.md
> @@ -126,6 +126,9 @@ The DT properties of a domain configuration DT node are as follows:
>  * **compatible** (Mandatory) - The compatible string of the domain
>    configuration. This DT property should have value *"opensbi,domain,config"*
>
> +* **system-suspend-test** (Optional) - When present, enable a system
> +  suspend test implementation which simply waits five seconds and issues a WFI.
> +
>  ### Domain Memory Region Node
>
>  The domain memory region DT node describes details of a memory region and
> @@ -234,6 +237,7 @@ be done:
>      chosen {
>          opensbi-domains {
>              compatible = "opensbi,domain,config";
> +            system-suspend-test;
>
>              tmem: tmem {
>                  compatible = "opensbi,domain,memregion";
> diff --git a/include/sbi/sbi_system.h b/include/sbi/sbi_system.h
> index d70ef8bb8874..b4e86aef8949 100644
> --- a/include/sbi/sbi_system.h
> +++ b/include/sbi/sbi_system.h
> @@ -66,6 +66,7 @@ struct sbi_system_suspend_device {
>
>  const struct sbi_system_suspend_device *sbi_system_suspend_get_device(void);
>  void sbi_system_suspend_set_device(struct sbi_system_suspend_device *dev);
> +void sbi_system_suspend_test_enable(void);
>  bool sbi_system_suspend_supported(u32 sleep_type);
>  int sbi_system_suspend(u32 sleep_type, ulong resume_addr, ulong opaque);
>
> diff --git a/lib/sbi/sbi_system.c b/lib/sbi/sbi_system.c
> index 7fa5ea82b7a6..6eaf6c9508e0 100644
> --- a/lib/sbi/sbi_system.c
> +++ b/lib/sbi/sbi_system.c
> @@ -17,6 +17,7 @@
>  #include <sbi/sbi_system.h>
>  #include <sbi/sbi_ipi.h>
>  #include <sbi/sbi_init.h>
> +#include <sbi/sbi_timer.h>
>
>  static SBI_LIST_HEAD(reset_devices_list);
>
> @@ -108,6 +109,36 @@ void sbi_system_suspend_set_device(struct sbi_system_suspend_device *dev)
>         suspend_dev = dev;
>  }
>
> +static int sbi_system_suspend_test_check(u32 sleep_type)
> +{
> +       return sleep_type == SBI_SUSP_SLEEP_TYPE_SUSPEND;
> +}
> +
> +static int sbi_system_suspend_test_suspend(u32 sleep_type,
> +                                          unsigned long warmboot_addr)

Renamed this parameter to mmode_resume_addr at the time of
merging this patch.

> +{
> +       if (sleep_type != SBI_SUSP_SLEEP_TYPE_SUSPEND)
> +               return SBI_EINVAL;
> +
> +       sbi_timer_mdelay(5000);
> +
> +       /* Wait for interrupt */
> +       wfi();
> +
> +       return SBI_OK;
> +}
> +
> +static struct sbi_system_suspend_device sbi_system_suspend_test = {
> +       .name = "system-suspend-test",
> +       .system_suspend_check = sbi_system_suspend_test_check,
> +       .system_suspend = sbi_system_suspend_test_suspend,
> +};
> +
> +void sbi_system_suspend_test_enable(void)
> +{
> +       sbi_system_suspend_set_device(&sbi_system_suspend_test);
> +}
> +
>  bool sbi_system_suspend_supported(u32 sleep_type)
>  {
>         return suspend_dev && suspend_dev->system_suspend_check &&
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 0b90fd7c1446..eeefef4c9533 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -13,6 +13,7 @@
>  #include <sbi/sbi_hartmask.h>
>  #include <sbi/sbi_platform.h>
>  #include <sbi/sbi_string.h>
> +#include <sbi/sbi_system.h>
>  #include <sbi_utils/fdt/fdt_domain.h>
>  #include <sbi_utils/fdt/fdt_fixup.h>
>  #include <sbi_utils/fdt/fdt_helper.h>
> @@ -219,7 +220,24 @@ static int generic_extensions_init(struct sbi_hart_features *hfeatures)
>
>  static int generic_domains_init(void)
>  {
> -       return fdt_domains_populate(fdt_get_address());
> +       void *fdt = fdt_get_address();
> +       int offset, ret;
> +
> +       ret = fdt_domains_populate(fdt);
> +       if (ret < 0)
> +               return ret;
> +
> +       offset = fdt_path_offset(fdt, "/chosen");
> +
> +       if (offset >= 0) {
> +               offset = fdt_node_offset_by_compatible(fdt, offset,
> +                                                      "opensbi,domain,config");
> +               if (offset >= 0 &&
> +                   fdt_get_property(fdt, offset, "system-suspend-test", NULL))
> +                       sbi_system_suspend_test_enable();
> +       }
> +
> +       return 0;
>  }
>
>  static u64 generic_tlbr_flush_limit(void)
> --
> 2.39.1
>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup


From anup at brainfault.org  Mon Feb 27 07:24:48 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 20:54:48 +0530
Subject: [PATCH v4] lib: sbi: fix missing '\r' for console
In-Reply-To: <20230213090929.144906-1-wxjstz@126.com>
References: <20230213090929.144906-1-wxjstz@126.com>
Message-ID: <CAAhSdy2425+iLY1z4QjQ1awpUzhx-MuXjPsj58ZztSx1u16qaw@mail.gmail.com>

On Mon, Feb 13, 2023 at 2:40 PM Xiang W <wxjstz at 126.com> wrote:
>
> print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> will add a \r before the output \n. This patch adds missing \r when
> outputting characters via console_dev->puts.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>
>
> Changes since v4:
> - nputs_all come back
> - implement partial writes as much as possible
>
> Changes since v3:
> - Prevent string overflow when looking for \n
>
> Changes since v2:
> - Fix the bug reported by Samuel. Prevent p from accessing memory other
>   than strings.
>
> ---
>  lib/sbi/sbi_console.c | 31 ++++++++++++++++++++++++++++++-
>  1 file changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index f3ac003..4be5a8f 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -46,12 +46,41 @@ void sbi_putc(char ch)
>         }
>  }
>
> +static unsigned long console_puts_all(const char *str, unsigned long len)
> +{
> +       const char *s = str;
> +       const char *e = s + len;
> +       while (s < e)
> +               s += console_dev->console_puts(s, e - s);
> +       return len;
> +}
> +
> +static unsigned long console_puts(const char *str, unsigned long len)
> +{
> +       unsigned long l;
> +       const char *s, *p, *e;
> +       s = str;
> +       e = str + len;
> +       while (s < e) {
> +               p = sbi_strchr(s, '\n');

Introduce "char *sbi_strnchr(const char *s, int c, size_t count)" and
use it here. This will simplify the below condition in "if ()" as well.

> +               if (p == NULL || p > e)
> +                       p = e;
> +               l = console_dev->console_puts(s, p - s);
> +               if (l < p - s)
> +                       return s - str + l;
> +               if (p < e && *p == '\n')
> +                       console_puts_all("\r\n", 2);
> +               s = p + 1;
> +       }
> +       return len;
> +}
> +
>  static unsigned long nputs(const char *str, unsigned long len)
>  {
>         unsigned long i, ret;
>
>         if (console_dev && console_dev->console_puts) {
> -               ret = console_dev->console_puts(str, len);
> +               ret = console_puts(str, len);
>         } else {
>                 for (i = 0; i < len; i++)
>                         sbi_putc(str[i]);
> --
> 2.39.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From wxjstz at 126.com  Mon Feb 27 08:19:04 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 28 Feb 2023 00:19:04 +0800
Subject: [PATCH v5 0/2] fix missing '\r' for console
Message-ID: <20230227161906.634775-1-wxjstz@126.com>

Changes since v5:
- Add sbi_strnchr as suggested by Anup

Changes since v4:
- nputs_all come back
- implement partial writes as much as possible

Changes since v3:
- Prevent string overflow when looking for \n

Changes since v2:
- Fix the bug reported by Samuel. Prevent p from accessing memory other
  than strings.

Xiang W (2):
  lib: sbi: Add sbi_strnchr implementation
  lib: sbi: fix missing '\r' for console

 include/sbi/sbi_string.h |  2 ++
 lib/sbi/sbi_console.c    | 31 ++++++++++++++++++++++++++++++-
 lib/sbi/sbi_string.c     | 10 ++++++++++
 3 files changed, 42 insertions(+), 1 deletion(-)

-- 
2.39.2



From wxjstz at 126.com  Mon Feb 27 08:19:05 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 28 Feb 2023 00:19:05 +0800
Subject: [PATCH v5 1/2] lib: sbi: Add sbi_strnchr implementation
In-Reply-To: <20230227161906.634775-1-wxjstz@126.com>
References: <20230227161906.634775-1-wxjstz@126.com>
Message-ID: <20230227161906.634775-2-wxjstz@126.com>

This commit add an implementation of sbi_strnchr

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/sbi_string.h |  2 ++
 lib/sbi/sbi_string.c     | 10 ++++++++++
 2 files changed, 12 insertions(+)

diff --git a/include/sbi/sbi_string.h b/include/sbi/sbi_string.h
index b7c2bc2..6941382 100644
--- a/include/sbi/sbi_string.h
+++ b/include/sbi/sbi_string.h
@@ -31,6 +31,8 @@ char *sbi_strncpy(char *dest, const char *src, size_t count);
 
 char *sbi_strchr(const char *s, int c);
 
+char *sbi_strnchr(const char *s, int c, size_t count);
+
 char *sbi_strrchr(const char *s, int c);
 
 void *sbi_memset(void *s, int c, size_t count);
diff --git a/lib/sbi/sbi_string.c b/lib/sbi/sbi_string.c
index 9ebea69..82b8c7e 100644
--- a/lib/sbi/sbi_string.c
+++ b/lib/sbi/sbi_string.c
@@ -97,6 +97,16 @@ char *sbi_strchr(const char *s, int c)
 		return (char *)s;
 }
 
+char *sbi_strnchr(const char *s, int c, size_t count)
+{
+	while (count-- > 0) {
+		if (*s == (char)c)
+			return (char *)s;
+		s++;
+	}
+	return NULL;
+}
+
 char *sbi_strrchr(const char *s, int c)
 {
 	const char *last = s + sbi_strlen(s);
-- 
2.39.2



From wxjstz at 126.com  Mon Feb 27 08:19:06 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 28 Feb 2023 00:19:06 +0800
Subject: [PATCH v5 2/2] lib: sbi: fix missing '\r' for console
In-Reply-To: <20230227161906.634775-1-wxjstz@126.com>
References: <20230227161906.634775-1-wxjstz@126.com>
Message-ID: <20230227161906.634775-3-wxjstz@126.com>

print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
will add a \r before the output \n. This patch adds missing \r when
outputting characters via console_dev->puts.

Signed-off-by: Xiang W <wxjstz at 126.com>
---
 lib/sbi/sbi_console.c | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
index f3ac003..adeadcb 100644
--- a/lib/sbi/sbi_console.c
+++ b/lib/sbi/sbi_console.c
@@ -46,12 +46,41 @@ void sbi_putc(char ch)
 	}
 }
 
+static unsigned long console_puts_all(const char *str, unsigned long len)
+{
+	const char *s = str;
+	const char *e = s + len;
+	while (s < e)
+		s += console_dev->console_puts(s, e - s);
+	return len;
+}
+
+static unsigned long console_puts(const char *str, unsigned long len)
+{
+	unsigned long l;
+	const char *s, *p, *e;
+	s = str;
+	e = str + len;
+	while (s < e) {
+		p = sbi_strnchr(s, '\n', e - s);
+		if (p == NULL)
+			p = e;
+		l = console_dev->console_puts(s, p - s);
+		if (l < p - s)
+			return s - str + l;
+		if (p < e && *p == '\n')
+			console_puts_all("\r\n", 2);
+		s = p + 1;
+	}
+	return len;
+}
+
 static unsigned long nputs(const char *str, unsigned long len)
 {
 	unsigned long i, ret;
 
 	if (console_dev && console_dev->console_puts) {
-		ret = console_dev->console_puts(str, len);
+		ret = console_puts(str, len);
 	} else {
 		for (i = 0; i < len; i++)
 			sbi_putc(str[i]);
-- 
2.39.2



From anup at brainfault.org  Mon Feb 27 08:25:11 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 21:55:11 +0530
Subject: [PATCH v5 1/2] lib: sbi: Add sbi_strnchr implementation
In-Reply-To: <20230227161906.634775-2-wxjstz@126.com>
References: <20230227161906.634775-1-wxjstz@126.com> <20230227161906.634775-2-wxjstz@126.com>
Message-ID: <CAAhSdy24RocBc19ihJYg6fW1eqWrjWO6BNZwrXJUW3KiFvNSzA@mail.gmail.com>

On Mon, Feb 27, 2023 at 9:49?PM Xiang W <wxjstz at 126.com> wrote:
>
> This commit add an implementation of sbi_strnchr
>
> Signed-off-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  include/sbi/sbi_string.h |  2 ++
>  lib/sbi/sbi_string.c     | 10 ++++++++++
>  2 files changed, 12 insertions(+)
>
> diff --git a/include/sbi/sbi_string.h b/include/sbi/sbi_string.h
> index b7c2bc2..6941382 100644
> --- a/include/sbi/sbi_string.h
> +++ b/include/sbi/sbi_string.h
> @@ -31,6 +31,8 @@ char *sbi_strncpy(char *dest, const char *src, size_t count);
>
>  char *sbi_strchr(const char *s, int c);
>
> +char *sbi_strnchr(const char *s, int c, size_t count);
> +
>  char *sbi_strrchr(const char *s, int c);
>
>  void *sbi_memset(void *s, int c, size_t count);
> diff --git a/lib/sbi/sbi_string.c b/lib/sbi/sbi_string.c
> index 9ebea69..82b8c7e 100644
> --- a/lib/sbi/sbi_string.c
> +++ b/lib/sbi/sbi_string.c
> @@ -97,6 +97,16 @@ char *sbi_strchr(const char *s, int c)
>                 return (char *)s;
>  }
>
> +char *sbi_strnchr(const char *s, int c, size_t count)
> +{
> +       while (count-- > 0) {
> +               if (*s == (char)c)
> +                       return (char *)s;
> +               s++;
> +       }
> +       return NULL;
> +}
> +
>  char *sbi_strrchr(const char *s, int c)
>  {
>         const char *last = s + sbi_strlen(s);
> --
> 2.39.2
>


From anup at brainfault.org  Mon Feb 27 08:25:28 2023
From: anup at brainfault.org (Anup Patel)
Date: Mon, 27 Feb 2023 21:55:28 +0530
Subject: [PATCH v5 2/2] lib: sbi: fix missing '\r' for console
In-Reply-To: <20230227161906.634775-3-wxjstz@126.com>
References: <20230227161906.634775-1-wxjstz@126.com> <20230227161906.634775-3-wxjstz@126.com>
Message-ID: <CAAhSdy2vCAyff+_CMBQ-z+PtcqFQPT58uifA0w5SyvR7bvtcPw@mail.gmail.com>

On Mon, Feb 27, 2023 at 9:50?PM Xiang W <wxjstz at 126.com> wrote:
>
> print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> will add a \r before the output \n. This patch adds missing \r when
> outputting characters via console_dev->puts.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/sbi/sbi_console.c | 31 ++++++++++++++++++++++++++++++-
>  1 file changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index f3ac003..adeadcb 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -46,12 +46,41 @@ void sbi_putc(char ch)
>         }
>  }
>
> +static unsigned long console_puts_all(const char *str, unsigned long len)
> +{
> +       const char *s = str;
> +       const char *e = s + len;
> +       while (s < e)
> +               s += console_dev->console_puts(s, e - s);
> +       return len;
> +}
> +
> +static unsigned long console_puts(const char *str, unsigned long len)
> +{
> +       unsigned long l;
> +       const char *s, *p, *e;
> +       s = str;
> +       e = str + len;
> +       while (s < e) {
> +               p = sbi_strnchr(s, '\n', e - s);
> +               if (p == NULL)
> +                       p = e;
> +               l = console_dev->console_puts(s, p - s);
> +               if (l < p - s)
> +                       return s - str + l;
> +               if (p < e && *p == '\n')
> +                       console_puts_all("\r\n", 2);
> +               s = p + 1;
> +       }
> +       return len;
> +}
> +
>  static unsigned long nputs(const char *str, unsigned long len)
>  {
>         unsigned long i, ret;
>
>         if (console_dev && console_dev->console_puts) {
> -               ret = console_dev->console_puts(str, len);
> +               ret = console_puts(str, len);
>         } else {
>                 for (i = 0; i < len; i++)
>                         sbi_putc(str[i]);
> --
> 2.39.2
>


From bmeng.cn at gmail.com  Mon Feb 27 17:22:48 2023
From: bmeng.cn at gmail.com (Bin Meng)
Date: Tue, 28 Feb 2023 09:22:48 +0800
Subject: [PATCH v4] lib: sbi: fix missing '\r' for console
In-Reply-To: <20230213090929.144906-1-wxjstz@126.com>
References: <20230213090929.144906-1-wxjstz@126.com>
Message-ID: <CAEUhbmWJ3C16NYOvQFyPu8MPy7zJ0kJP=sDwyVRDAXhrNGMQUQ@mail.gmail.com>

On Mon, Feb 13, 2023 at 5:11?PM Xiang W <wxjstz at 126.com> wrote:
>
> print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> will add a \r before the output \n. This patch adds missing \r when
> outputting characters via console_dev->puts.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>
>
> Changes since v4:
> - nputs_all come back
> - implement partial writes as much as possible

These change logs should go below ---

>
> Changes since v3:
> - Prevent string overflow when looking for \n
>
> Changes since v2:
> - Fix the bug reported by Samuel. Prevent p from accessing memory other
>   than strings.
>
> ---
>  lib/sbi/sbi_console.c | 31 ++++++++++++++++++++++++++++++-
>  1 file changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index f3ac003..4be5a8f 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -46,12 +46,41 @@ void sbi_putc(char ch)
>         }
>  }
>
> +static unsigned long console_puts_all(const char *str, unsigned long len)
> +{
> +       const char *s = str;
> +       const char *e = s + len;
> +       while (s < e)
> +               s += console_dev->console_puts(s, e - s);
> +       return len;
> +}
> +
> +static unsigned long console_puts(const char *str, unsigned long len)
> +{
> +       unsigned long l;
> +       const char *s, *p, *e;
> +       s = str;
> +       e = str + len;
> +       while (s < e) {
> +               p = sbi_strchr(s, '\n');
> +               if (p == NULL || p > e)
> +                       p = e;
> +               l = console_dev->console_puts(s, p - s);
> +               if (l < p - s)
> +                       return s - str + l;
> +               if (p < e && *p == '\n')
> +                       console_puts_all("\r\n", 2);
> +               s = p + 1;
> +       }
> +       return len;
> +}
> +
>  static unsigned long nputs(const char *str, unsigned long len)
>  {
>         unsigned long i, ret;
>
>         if (console_dev && console_dev->console_puts) {
> -               ret = console_dev->console_puts(str, len);
> +               ret = console_puts(str, len);
>         } else {
>                 for (i = 0; i < len; i++)
>                         sbi_putc(str[i]);
> --

Regards,
Bin


From wxjstz at 126.com  Mon Feb 27 17:34:33 2023
From: wxjstz at 126.com (Xiang W)
Date: Tue, 28 Feb 2023 09:34:33 +0800
Subject: [PATCH v4] lib: sbi: fix missing '\r' for console
In-Reply-To: <CAEUhbmWJ3C16NYOvQFyPu8MPy7zJ0kJP=sDwyVRDAXhrNGMQUQ@mail.gmail.com>
References: <20230213090929.144906-1-wxjstz@126.com>
	 <CAEUhbmWJ3C16NYOvQFyPu8MPy7zJ0kJP=sDwyVRDAXhrNGMQUQ@mail.gmail.com>
Message-ID: <5f3c5da100cafae5d34a778ae6b632ec205da743.camel@126.com>

? 2023-02-28???? 09:22 +0800?Bin Meng???
> On Mon, Feb 13, 2023 at 5:11?PM Xiang W <wxjstz at 126.com> wrote:
> > 
> > print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> > will add a \r before the output \n. This patch adds missing \r when
> > outputting characters via console_dev->puts.
> > 
> > Signed-off-by: Xiang W <wxjstz at 126.com>
> > 
> > Changes since v4:
> > - nputs_all come back
> > - implement partial writes as much as possible
> 
> These change logs should go below ---
Thanks for pointing that out. New patch at
https://lists.infradead.org/pipermail/opensbi/2023-February/004601.html

Regards,
Xiang W
> 
> > 
> > Changes since v3:
> > - Prevent string overflow when looking for \n
> > 
> > Changes since v2:
> > - Fix the bug reported by Samuel. Prevent p from accessing memory other
> > ? than strings.
> > 
> > ---
> > ?lib/sbi/sbi_console.c | 31 ++++++++++++++++++++++++++++++-
> > ?1 file changed, 30 insertions(+), 1 deletion(-)
> > 
> > diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> > index f3ac003..4be5a8f 100644
> > --- a/lib/sbi/sbi_console.c
> > +++ b/lib/sbi/sbi_console.c
> > @@ -46,12 +46,41 @@ void sbi_putc(char ch)
> > ??????? }
> > ?}
> > 
> > +static unsigned long console_puts_all(const char *str, unsigned long len)
> > +{
> > +?????? const char *s = str;
> > +?????? const char *e = s + len;
> > +?????? while (s < e)
> > +?????????????? s += console_dev->console_puts(s, e - s);
> > +?????? return len;
> > +}
> > +
> > +static unsigned long console_puts(const char *str, unsigned long len)
> > +{
> > +?????? unsigned long l;
> > +?????? const char *s, *p, *e;
> > +?????? s = str;
> > +?????? e = str + len;
> > +?????? while (s < e) {
> > +?????????????? p = sbi_strchr(s, '\n');
> > +?????????????? if (p == NULL || p > e)
> > +?????????????????????? p = e;
> > +?????????????? l = console_dev->console_puts(s, p - s);
> > +?????????????? if (l < p - s)
> > +?????????????????????? return s - str + l;
> > +?????????????? if (p < e && *p == '\n')
> > +?????????????????????? console_puts_all("\r\n", 2);
> > +?????????????? s = p + 1;
> > +?????? }
> > +?????? return len;
> > +}
> > +
> > ?static unsigned long nputs(const char *str, unsigned long len)
> > ?{
> > ??????? unsigned long i, ret;
> > 
> > ??????? if (console_dev && console_dev->console_puts) {
> > -?????????????? ret = console_dev->console_puts(str, len);
> > +?????????????? ret = console_puts(str, len);
> > ??????? } else {
> > ??????????????? for (i = 0; i < len; i++)
> > ??????????????????????? sbi_putc(str[i]);
> > --
> 
> Regards,
> Bin
> 


From anup at brainfault.org  Mon Feb 27 20:25:41 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 28 Feb 2023 09:55:41 +0530
Subject: [PATCH v5 1/2] lib: sbi: Add sbi_strnchr implementation
In-Reply-To: <20230227161906.634775-2-wxjstz@126.com>
References: <20230227161906.634775-1-wxjstz@126.com> <20230227161906.634775-2-wxjstz@126.com>
Message-ID: <CAAhSdy0on4VS8cjwRD8u5cCuyZ1J3JVT4OPO6OSsJCmznR9QZQ@mail.gmail.com>

On Mon, Feb 27, 2023 at 9:49 PM Xiang W <wxjstz at 126.com> wrote:
>
> This commit add an implementation of sbi_strnchr
>
> Signed-off-by: Xiang W <wxjstz at 126.com>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  include/sbi/sbi_string.h |  2 ++
>  lib/sbi/sbi_string.c     | 10 ++++++++++
>  2 files changed, 12 insertions(+)
>
> diff --git a/include/sbi/sbi_string.h b/include/sbi/sbi_string.h
> index b7c2bc2..6941382 100644
> --- a/include/sbi/sbi_string.h
> +++ b/include/sbi/sbi_string.h
> @@ -31,6 +31,8 @@ char *sbi_strncpy(char *dest, const char *src, size_t count);
>
>  char *sbi_strchr(const char *s, int c);
>
> +char *sbi_strnchr(const char *s, int c, size_t count);
> +
>  char *sbi_strrchr(const char *s, int c);
>
>  void *sbi_memset(void *s, int c, size_t count);
> diff --git a/lib/sbi/sbi_string.c b/lib/sbi/sbi_string.c
> index 9ebea69..82b8c7e 100644
> --- a/lib/sbi/sbi_string.c
> +++ b/lib/sbi/sbi_string.c
> @@ -97,6 +97,16 @@ char *sbi_strchr(const char *s, int c)
>                 return (char *)s;
>  }
>
> +char *sbi_strnchr(const char *s, int c, size_t count)
> +{
> +       while (count-- > 0) {
> +               if (*s == (char)c)
> +                       return (char *)s;
> +               s++;
> +       }
> +       return NULL;
> +}
> +
>  char *sbi_strrchr(const char *s, int c)
>  {
>         const char *last = s + sbi_strlen(s);
> --
> 2.39.2
>


From anup at brainfault.org  Mon Feb 27 20:25:57 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 28 Feb 2023 09:55:57 +0530
Subject: [PATCH v5 2/2] lib: sbi: fix missing '\r' for console
In-Reply-To: <20230227161906.634775-3-wxjstz@126.com>
References: <20230227161906.634775-1-wxjstz@126.com> <20230227161906.634775-3-wxjstz@126.com>
Message-ID: <CAAhSdy3X2JY4siWuJKurbBN4z5RiJd=XDAhWYCFRioToBVDVhg@mail.gmail.com>

On Mon, Feb 27, 2023 at 9:50 PM Xiang W <wxjstz at 126.com> wrote:
>
> print is finally implemented by sbi_putc or console_dev->puts. sbi_putc
> will add a \r before the output \n. This patch adds missing \r when
> outputting characters via console_dev->puts.
>
> Signed-off-by: Xiang W <wxjstz at 126.com>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/sbi/sbi_console.c | 31 ++++++++++++++++++++++++++++++-
>  1 file changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_console.c b/lib/sbi/sbi_console.c
> index f3ac003..adeadcb 100644
> --- a/lib/sbi/sbi_console.c
> +++ b/lib/sbi/sbi_console.c
> @@ -46,12 +46,41 @@ void sbi_putc(char ch)
>         }
>  }
>
> +static unsigned long console_puts_all(const char *str, unsigned long len)
> +{
> +       const char *s = str;
> +       const char *e = s + len;
> +       while (s < e)
> +               s += console_dev->console_puts(s, e - s);
> +       return len;
> +}
> +
> +static unsigned long console_puts(const char *str, unsigned long len)
> +{
> +       unsigned long l;
> +       const char *s, *p, *e;
> +       s = str;
> +       e = str + len;
> +       while (s < e) {
> +               p = sbi_strnchr(s, '\n', e - s);
> +               if (p == NULL)
> +                       p = e;
> +               l = console_dev->console_puts(s, p - s);
> +               if (l < p - s)
> +                       return s - str + l;
> +               if (p < e && *p == '\n')
> +                       console_puts_all("\r\n", 2);
> +               s = p + 1;
> +       }
> +       return len;
> +}
> +
>  static unsigned long nputs(const char *str, unsigned long len)
>  {
>         unsigned long i, ret;
>
>         if (console_dev && console_dev->console_puts) {
> -               ret = console_dev->console_puts(str, len);
> +               ret = console_puts(str, len);
>         } else {
>                 for (i = 0; i < len; i++)
>                         sbi_putc(str[i]);
> --
> 2.39.2
>


From anup at brainfault.org  Mon Feb 27 20:41:55 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 28 Feb 2023 10:11:55 +0530
Subject: [PATCH v2 2/2] lib: sbi: Clear IPIs before init_warm_startup in
 non-boot harts
In-Reply-To: <20230221210851.3145840-3-e.shatokhin@yadro.com>
References: <20230221210851.3145840-1-e.shatokhin@yadro.com> <20230221210851.3145840-3-e.shatokhin@yadro.com>
Message-ID: <CAAhSdy1KWF_j=uVdijefit8ZD1nqDtSNh=3gZy96E_JxUEForw@mail.gmail.com>

On Wed, Feb 22, 2023 at 2:39 AM Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>
> Since commit 50d4fde1c5a4 ("lib: Remove redundant sbi_platform_ipi_clear()
> calls"), the IPI sent from the boot hart in wake_coldboot_harts() is not
> cleared in the secondary harts until they reach sbi_ipi_init(). However,
> sbi_hsm_init() and sbi_hsm_hart_wait() are called earlier, so a secondary
> hart might enter sbi_hsm_hart_wait() with an already pending IPI.
>
> sbi_hsm_hart_wait() makes sure the hart leaves the loop only when it is
> actually ready, so a pending unrelated IPI should not cause safety issues.
> However, it might be inefficient on certain hardware, because it prevents
> "wfi" from stalling the hart even if the hardware supports this, making the
> hart needlessly spin in a "busy-wait" loop.
>
> This behaviour can be observed, for example, in a QEMU VM (QEMU 7.2.0) with
> "-machine virt" running a Linux guest. Inserting delays in
> sbi_hsm_hart_start() allows reproducing the issue more reliably.
>
> The comment in wait_for_coldboot() suggests that the initial IPI is needed
> in the warm resume path, so let us clear it before init_warm_startup()
> only.
>
> To do this, sbi_ipi_raw_clear() was created similar to sbi_ipi_raw_send().
>
> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
> ---
>  include/sbi/sbi_ipi.h | 2 ++
>  lib/sbi/sbi_init.c    | 6 ++++--
>  lib/sbi/sbi_ipi.c     | 6 ++++++
>  3 files changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
> index f6ac807..f384e74 100644
> --- a/include/sbi/sbi_ipi.h
> +++ b/include/sbi/sbi_ipi.h
> @@ -77,6 +77,8 @@ void sbi_ipi_process(void);
>
>  int sbi_ipi_raw_send(u32 target_hart);
>
> +void sbi_ipi_raw_clear(u32 target_hart);
> +
>  const struct sbi_ipi_device *sbi_ipi_get_device(void);
>
>  void sbi_ipi_set_device(const struct sbi_ipi_device *dev);
> diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
> index e353c33..a9a4084 100644
> --- a/lib/sbi/sbi_init.c
> +++ b/lib/sbi/sbi_init.c
> @@ -438,10 +438,12 @@ static void __noreturn init_warmboot(struct sbi_scratch *scratch, u32 hartid)
>         if (hstate < 0)
>                 sbi_hart_hang();
>
> -       if (hstate == SBI_HSM_STATE_SUSPENDED)
> +       if (hstate == SBI_HSM_STATE_SUSPENDED) {
>                 init_warm_resume(scratch);
> -       else
> +       } else {
> +               sbi_ipi_raw_clear(hartid);

This effectively does nothing in many cases because for a lot of
platforms all HARTs enter OpenSBI at the same time and the HARTs
entering warm boot path will call sbi_ipi_raw_clear() even before
ipi_dev is provided by platform code.

>                 init_warm_startup(scratch, hartid);
> +       }
>
>         sbi_hart_switch_mode(hartid, scratch->next_arg1,
>                              scratch->next_addr,
> diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
> index 1bcc2e4..b9f6205 100644
> --- a/lib/sbi/sbi_ipi.c
> +++ b/lib/sbi/sbi_ipi.c
> @@ -217,6 +217,12 @@ int sbi_ipi_raw_send(u32 target_hart)
>         return 0;
>  }
>
> +void sbi_ipi_raw_clear(u32 target_hart)
> +{
> +       if (ipi_dev && ipi_dev->ipi_clear)
> +               ipi_dev->ipi_clear(target_hart);
> +}
> +
>  const struct sbi_ipi_device *sbi_ipi_get_device(void)
>  {
>         return ipi_dev;
> --
> 2.34.1
>
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From anup at brainfault.org  Mon Feb 27 21:06:18 2023
From: anup at brainfault.org (Anup Patel)
Date: Tue, 28 Feb 2023 10:36:18 +0530
Subject: [PATCH v2 1/2] lib: sbi: Wait in sbi_hsm_hart_wait() till the
 hart is prepared to start
In-Reply-To: <20230221210851.3145840-2-e.shatokhin@yadro.com>
References: <20230221210851.3145840-1-e.shatokhin@yadro.com> <20230221210851.3145840-2-e.shatokhin@yadro.com>
Message-ID: <CAAhSdy2EwxbwNV2fvshUDr49h6Ja6K3j8Ah4w1_TbYEbZ4UaOA@mail.gmail.com>

On Wed, Feb 22, 2023 at 2:39 AM Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>
> "wfi" instruction is not guaranteed to wait for an interrupt to be received
> by the hart, it is just a hint for the CPU. According to RISC-V Privileged
> Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
> legal.
>
> As a result, a secondary (non-boot) hart waiting in the while loop in
> sbi_hsm_hart_wait() might leave the loop and continue even if it got no
> IPI or it got an IPI unrelated to sbi_hsm_hart_start(). This could lead to
> the following race condition when booting Linux, for example:
>
>   Boot hart (#0)                        Secondary hart (#1)
>   runs Linux startup code               waits in sbi_hsm_hart_wait()
>
>   sbi_ecall(SBI_EXT_HSM,
>             SBI_EXT_HSM_HART_START,
>             ...)
>   enters sbi_hsm_hart_start()
>   sets state of hart #1 to START_PENDING
>                                         leaves sbi_hsm_hart_wait()
>                                         runs to the end of init_warmboot()
>                                         returns to scratch->next_addr
>                                         (next_addr can be garbage here)
>
>   sets next_addr, etc. for hart #1
>   (no good: hart #1 has already left)
>
>   sends IPI to hart #1
>   (no good either)
>
> If this happens, the secondary hart jumps to a wrong next_addr at the end
> of init_warmboot(), which leads to a system hang or crash.
>
> To reproduce the issue more reliably, one could add a delay in
> sbi_hsm_hart_start() after setting the hart's state but before sending
> IPI to that hart:
>
>     hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
>                             SBI_HSM_STATE_START_PENDING);
>     ...
>   + sbi_timer_mdelay(10);
>     init_count = sbi_init_count(hartid);
>     rscratch->next_arg1 = arg1;
>     rscratch->next_addr = saddr;
>
> The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
> and 2 or more CPUs, with Linux as the guest OS.

Good catch. This is a valid issue which needs to be fixed.

>
> This patch adds an auxiliary per-hart state variable, aux_state, as well as
> its accessor functions, to synchronize execution of the boot hart and the
> starting secondary hart.
>
> Initially, aux_state is 0: sbi_scratch_alloc_offset() zeroes the allocated
> memory. The boot hart sets it to 1 after it has set next_addr, etc., for the
> secondary hart, then sends the IPI to the latter.
>
> The secondary hart waits in sbi_hsm_hart_wait() both for aux_state to become
> non-zero and for an IPI to happen. This should be safe even if the secondary
> hart somehow gets an IPI unrelated to sbi_hsm_hart_start().
>
> Memory barriers are used to make sure the writes to next_addr, etc., in the
> boot hart are visible to the secondary hart if it reads non-zero aux_state.
>
> Only one bit of aux_state is actually used here, so, this per-hart variable
> can be used to hold other data in the future, if needed.

Adding aux_state is an overly complicated solution in my opinion.

Instead, you can simply do the following:

    rscratch->next_arg1 = arg1;
    rscratch->next_addr = saddr;
    rscratch->next_mode = smode;

    /*
     * Barrier to ensure that the above writes have happened before
     * updating the HART state.
     */
    smp_wmb();

Before "hdata = sbi_scratch_offset_ptr(rscratch, hart_data_offset);" in
sbi_hsm_hart_start().

>
> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>

> ---
>  lib/sbi/sbi_hsm.c | 35 +++++++++++++++++++++++++++++++++--
>  1 file changed, 33 insertions(+), 2 deletions(-)
>
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index 8499bb1..24061c5 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -35,6 +35,7 @@ struct sbi_hsm_data {
>         unsigned long suspend_type;
>         unsigned long saved_mie;
>         unsigned long saved_mip;
> +       unsigned long aux_state;
>  };
>
>  static inline int __sbi_hsm_hart_get_state(u32 hartid)
> @@ -93,6 +94,23 @@ int sbi_hsm_hart_interruptible_mask(const struct sbi_domain *dom,
>         return 0;
>  }
>
> +static inline bool sbi_hsm_is_runnable(struct sbi_hsm_data *hdata)
> +{
> +       unsigned long aux_state = __smp_load_acquire(&hdata->aux_state);
> +
> +       return !!aux_state;
> +}
> +
> +static inline void sbi_hsm_set_runnable(struct sbi_hsm_data *hdata)
> +{
> +       __smp_store_release(&hdata->aux_state, 1);
> +}
> +
> +static inline void sbi_hsm_reset_runnable(struct sbi_hsm_data *hdata)
> +{
> +       __smp_store_release(&hdata->aux_state, 0);
> +}
> +
>  void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>  {
>         u32 oldstate;
> @@ -108,6 +126,7 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>  static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>  {
>         unsigned long saved_mie;
> +       unsigned long cmip;
>         struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>                                                             hart_data_offset);
>         /* Save MIE CSR */
> @@ -117,13 +136,22 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>         csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
>
>         /* Wait for state transition requested by sbi_hsm_hart_start() */
> -       while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
> -               wfi();
> +       while (!sbi_hsm_is_runnable(hdata)) {
> +               do {
> +                       wfi();
> +                       cmip = csr_read(CSR_MIP);
> +               } while (!(cmip & (MIP_MSIP | MIP_MEIP)));
>         };
>
>         /* Restore MIE CSR */
>         csr_write(CSR_MIE, saved_mie);
>
> +       /*
> +        * If the hart ever needs to wait for such state transition again,
> +        * make sure the hart's data allows that.
> +        */
> +       sbi_hsm_reset_runnable(hdata);
> +
>         /*
>          * No need to clear IPI here because the sbi_ipi_init() will
>          * clear it for current HART via sbi_platform_ipi_init().
> @@ -284,6 +312,9 @@ int sbi_hsm_hart_start(struct sbi_scratch *scratch,
>         rscratch->next_addr = saddr;
>         rscratch->next_mode = smode;
>
> +       /* Inform the hart that it may continue as soon as it gets IPI. */
> +       sbi_hsm_set_runnable(hdata);
> +
>         if (hsm_device_has_hart_hotplug() ||
>            (hsm_device_has_hart_secondary_boot() && !init_count)) {
>                 return hsm_device_hart_start(hartid, scratch->warmboot_addr);
> --
> 2.34.1
>
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From minda.chen at starfivetech.com  Tue Feb 28 02:30:14 2023
From: minda.chen at starfivetech.com (Minda Chen)
Date: Tue, 28 Feb 2023 18:30:14 +0800
Subject: [PATCH] gpio/starfive: add gpio driver and support gpio reset
In-Reply-To: <CAAhSdy1fMYFVddf2UqkY8P0QBexeUt0FZLCg768LMkezUTzXAw@mail.gmail.com>
References: <20230216092126.99022-1-minda.chen@starfivetech.com>
 <CAAhSdy3gUbX3+54-i6PSo99UMGVE60VSYALRLA5QVazSqLCQmQ@mail.gmail.com>
 <d0ca6d1c-6761-6b35-6ca8-4244937b1392@starfivetech.com>
 <CAAhSdy1fMYFVddf2UqkY8P0QBexeUt0FZLCg768LMkezUTzXAw@mail.gmail.com>
Message-ID: <2968fc4e-96ad-ccf6-d313-2ec370f87af4@starfivetech.com>



On 2023/2/27 22:00, Anup Patel wrote:
> On Mon, Feb 27, 2023 at 4:53 PM Minda Chen <minda.chen at starfivetech.com> wrote:
>>
>>
>>
>> On 2023/2/27 14:44, Anup Patel wrote:
>> > On Thu, Feb 16, 2023 at 2:52 PM minda.chen <minda.chen at starfivetech.com> wrote:
>> >>
>> >> Add gpio driver and gpio reset function in Starfive
>> >> JH7110 SOC platform.
>> >>
>> >> Signed-off-by: minda.chen <minda.chen at starfivetech.com>
>> >
>> > I don't see a PATCH enabling it in defconfig.
>> >
>> > How is this driver used in OpenSBI ?
>> >
>> > Regards,
>> > Anup
>> >
>> I have enabled CONFIG_FDT_GPIO_STARFIVE in defconfig in this patch.
> 
> Okay.
> 
>>
>>  --- a/platform/generic/configs/defconfig
>>  +++ b/platform/generic/configs/defconfig
>>  @@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
>>   CONFIG_PLATFORM_STARFIVE_JH7110=y
>>   CONFIG_FDT_GPIO=y
>>   CONFIG_FDT_GPIO_SIFIVE=y
>>  +CONFIG_FDT_GPIO_STARFIVE=y
>>
>> May I set this change to second commit ?
> 
> No need for a separate patch.
> 
>>
>> This patch is a gpio reset implementation for sbi ecall SBI_EXT_SRST_RESET.
> 
> Fare enough but you also send another series
> "Add StarFive JH7110 SoC shutdown and reboot ops"
> which provides system reboot based on I2C PMIC.
> 
> Basically, we have two separate system reboot ops
> for JH7110, is that right ?
> 
> Regards,
> Anup
> 
Yes, GPIO reset is for uboot warm reset, PMIC reset is for uboot cold reset and linux reboot.
  
> 
>>
>> >> ---
>> >>  lib/utils/gpio/Kconfig             |   3 +
>> >>  lib/utils/gpio/fdt_gpio_starfive.c | 117 +++++++++++++++++++++++++++++
>> >>  lib/utils/gpio/objects.mk          |   3 +
>> >>  platform/generic/configs/defconfig |   1 +
>> defconfig have been changed.
>>
>> >>  4 files changed, 124 insertions(+)
>> >>  create mode 100644 lib/utils/gpio/fdt_gpio_starfive.c
>> >>
>> >> diff --git a/lib/utils/gpio/Kconfig b/lib/utils/gpio/Kconfig
>> >> index 38a9d75..1b338a6 100644
>> >> --- a/lib/utils/gpio/Kconfig
>> >> +++ b/lib/utils/gpio/Kconfig
>> >> @@ -14,6 +14,9 @@ config FDT_GPIO_SIFIVE
>> >>         bool "SiFive GPIO FDT driver"
>> >>         default n
>> >>
>> >> +config FDT_GPIO_STARFIVE
>> >> +       bool "StarFive GPIO FDT driver"
>> >> +       default n
>> >>  endif
>> >>
>> >>  config GPIO
>> >> diff --git a/lib/utils/gpio/fdt_gpio_starfive.c b/lib/utils/gpio/fdt_gpio_starfive.c
>> >> new file mode 100644
>> >> index 0000000..18cca72
>> >> --- /dev/null
>> >> +++ b/lib/utils/gpio/fdt_gpio_starfive.c
>> >> @@ -0,0 +1,117 @@
>> >> +/*
>> >> + * SPDX-License-Identifier: BSD-2-Clause
>> >> + *
>> >> + * Copyright (c) 2022 Starfive
>> >> + *
>> >> + * Authors:
>> >> + *   Minda.chen <Minda.chen at starfivetech.com>
>> >> + */
>> >> +
>> >> +#include <sbi/riscv_io.h>
>> >> +#include <sbi/sbi_error.h>
>> >> +#include <sbi/sbi_console.h>
>> >> +#include <sbi_utils/fdt/fdt_helper.h>
>> >> +#include <sbi_utils/gpio/fdt_gpio.h>
>> >> +
>> >> +#define STARFIVE_GPIO_CHIP_MAX         2
>> >> +#define STARFIVE_GPIO_PINS_DEF         64
>> >> +#define STARFIVE_GPIO_OUTVAL           0x40
>> >> +#define STARFIVE_GPIO_MASK             0xff
>> >> +#define STARFIVE_GPIO_REG_SHIFT_MASK   0x3
>> >> +#define STARFIVE_GPIO_SHIFT_BITS       0x3
>> >> +
>> >> +struct starfive_gpio_chip {
>> >> +       unsigned long addr;
>> >> +       struct gpio_chip chip;
>> >> +};
>> >> +
>> >> +static unsigned int starfive_gpio_chip_count;
>> >> +static struct starfive_gpio_chip starfive_gpio_chip_array[STARFIVE_GPIO_CHIP_MAX];
>> >> +
>> >> +static int starfive_gpio_direction_output(struct gpio_pin *gp, int value)
>> >> +{
>> >> +       u32 val;
>> >> +       unsigned long reg_addr;
>> >> +       u32 bit_mask, shift_bits;
>> >> +       struct starfive_gpio_chip *chip =
>> >> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
>> >> +
>> >> +       /* set out en*/
>> >> +       reg_addr = chip->addr + gp->offset;
>> >> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
>> >> +
>> >> +       val = readl((void *)(reg_addr));
>> >> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
>> >> +               << STARFIVE_GPIO_SHIFT_BITS;
>> >> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
>> >> +
>> >> +       val = readl((void *)reg_addr);
>> >> +       val &= ~bit_mask;
>> >> +       writel(val, (void *)reg_addr);
>> >> +
>> >> +       return 0;
>> >> +}
>> >> +
>> >> +static void starfive_gpio_set(struct gpio_pin *gp, int value)
>> >> +{
>> >> +       u32 val;
>> >> +       unsigned long reg_addr;
>> >> +       u32 bit_mask, shift_bits;
>> >> +       struct starfive_gpio_chip *chip =
>> >> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
>> >> +
>> >> +       reg_addr = chip->addr + gp->offset;
>> >> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
>> >> +
>> >> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
>> >> +               << STARFIVE_GPIO_SHIFT_BITS;
>> >> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
>> >> +       /* set output value */
>> >> +       val = readl((void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
>> >> +       val &= ~bit_mask;
>> >> +       val |= value << shift_bits;
>> >> +       writel(val, (void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
>> >> +}
>> >> +
>> >> +extern struct fdt_gpio fdt_gpio_starfive;
>> >> +
>> >> +static int starfive_gpio_init(void *fdt, int nodeoff, u32 phandle,
>> >> +                             const struct fdt_match *match)
>> >> +{
>> >> +       int rc;
>> >> +       struct starfive_gpio_chip *chip;
>> >> +       u64 addr;
>> >> +
>> >> +       if (starfive_gpio_chip_count >= STARFIVE_GPIO_CHIP_MAX)
>> >> +               return SBI_ENOSPC;
>> >> +       chip = &starfive_gpio_chip_array[starfive_gpio_chip_count];
>> >> +
>> >> +       rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
>> >> +       if (rc)
>> >> +               return rc;
>> >> +
>> >> +       chip->addr = addr;
>> >> +       chip->chip.driver = &fdt_gpio_starfive;
>> >> +       chip->chip.id = phandle;
>> >> +       chip->chip.ngpio = STARFIVE_GPIO_PINS_DEF;
>> >> +       chip->chip.direction_output = starfive_gpio_direction_output;
>> >> +       chip->chip.set = starfive_gpio_set;
>> >> +       rc = gpio_chip_add(&chip->chip);
>> >> +       if (rc)
>> >> +               return rc;
>> >> +
>> >> +       starfive_gpio_chip_count++;
>> >> +       return 0;
>> >> +}
>> >> +
>> >> +static const struct fdt_match starfive_gpio_match[] = {
>> >> +       { .compatible = "starfive,jh7110-sys-pinctrl" },
>> >> +       { .compatible = "starfive,iomux-pinctrl" },
>> >> +       { },
>> >> +};
>> >> +
>> >> +struct fdt_gpio fdt_gpio_starfive = {
>> >> +       .match_table = starfive_gpio_match,
>> >> +       .xlate = fdt_gpio_simple_xlate,
>> >> +       .init = starfive_gpio_init,
>> >> +};
>> >> diff --git a/lib/utils/gpio/objects.mk b/lib/utils/gpio/objects.mk
>> >> index eedd699..8f4a6a8 100644
>> >> --- a/lib/utils/gpio/objects.mk
>> >> +++ b/lib/utils/gpio/objects.mk
>> >> @@ -13,4 +13,7 @@ libsbiutils-objs-$(CONFIG_FDT_GPIO) += gpio/fdt_gpio_drivers.o
>> >>  carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_SIFIVE) += fdt_gpio_sifive
>> >>  libsbiutils-objs-$(CONFIG_FDT_GPIO_SIFIVE) += gpio/fdt_gpio_sifive.o
>> >>
>> >> +carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_STARFIVE) += fdt_gpio_starfive
>> >> +libsbiutils-objs-$(CONFIG_FDT_GPIO_STARFIVE) += gpio/fdt_gpio_starfive.o
>> >> +
>> >>  libsbiutils-objs-$(CONFIG_GPIO) += gpio/gpio.o
>> >> diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
>> >> index 4b0842e..2ad953d 100644
>> >> --- a/platform/generic/configs/defconfig
>> >> +++ b/platform/generic/configs/defconfig
>> >> @@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
>> >>  CONFIG_PLATFORM_STARFIVE_JH7110=y
>> >>  CONFIG_FDT_GPIO=y
>> >>  CONFIG_FDT_GPIO_SIFIVE=y
>> >> +CONFIG_FDT_GPIO_STARFIVE=y
>> >>  CONFIG_FDT_I2C=y
>> >>  CONFIG_FDT_I2C_SIFIVE=y
>> >>  CONFIG_FDT_IPI=y
>> >> --
>> >> 2.17.1
>> >>
>> >>
>> >> --
>> >> opensbi mailing list
>> >> opensbi at lists.infradead.org
>> >> http://lists.infradead.org/mailman/listinfo/opensbi


From e.shatokhin at yadro.com  Tue Feb 28 07:56:09 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Tue, 28 Feb 2023 18:56:09 +0300
Subject: [PATCH v2 1/2] lib: sbi: Wait in sbi_hsm_hart_wait() till the
 hart is prepared to start
In-Reply-To: <CAAhSdy2EwxbwNV2fvshUDr49h6Ja6K3j8Ah4w1_TbYEbZ4UaOA@mail.gmail.com>
References: <20230221210851.3145840-1-e.shatokhin@yadro.com>
 <20230221210851.3145840-2-e.shatokhin@yadro.com>
 <CAAhSdy2EwxbwNV2fvshUDr49h6Ja6K3j8Ah4w1_TbYEbZ4UaOA@mail.gmail.com>
Message-ID: <6eb6ddad-6ddf-1b6a-aad8-7a4df8657641@yadro.com>

On 28.02.2023 08:06, Anup Patel wrote:
> On Wed, Feb 22, 2023 at 2:39 AM Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>>
>> "wfi" instruction is not guaranteed to wait for an interrupt to be received
>> by the hart, it is just a hint for the CPU. According to RISC-V Privileged
>> Architectures spec. v20211203, even an implementation of "wfi" as "nop" is
>> legal.
>>
>> As a result, a secondary (non-boot) hart waiting in the while loop in
>> sbi_hsm_hart_wait() might leave the loop and continue even if it got no
>> IPI or it got an IPI unrelated to sbi_hsm_hart_start(). This could lead to
>> the following race condition when booting Linux, for example:
>>
>>    Boot hart (#0)                        Secondary hart (#1)
>>    runs Linux startup code               waits in sbi_hsm_hart_wait()
>>
>>    sbi_ecall(SBI_EXT_HSM,
>>              SBI_EXT_HSM_HART_START,
>>              ...)
>>    enters sbi_hsm_hart_start()
>>    sets state of hart #1 to START_PENDING
>>                                          leaves sbi_hsm_hart_wait()
>>                                          runs to the end of init_warmboot()
>>                                          returns to scratch->next_addr
>>                                          (next_addr can be garbage here)
>>
>>    sets next_addr, etc. for hart #1
>>    (no good: hart #1 has already left)
>>
>>    sends IPI to hart #1
>>    (no good either)
>>
>> If this happens, the secondary hart jumps to a wrong next_addr at the end
>> of init_warmboot(), which leads to a system hang or crash.
>>
>> To reproduce the issue more reliably, one could add a delay in
>> sbi_hsm_hart_start() after setting the hart's state but before sending
>> IPI to that hart:
>>
>>      hstate = atomic_cmpxchg(&hdata->state, SBI_HSM_STATE_STOPPED,
>>                              SBI_HSM_STATE_START_PENDING);
>>      ...
>>    + sbi_timer_mdelay(10);
>>      init_count = sbi_init_count(hartid);
>>      rscratch->next_arg1 = arg1;
>>      rscratch->next_addr = saddr;
>>
>> The issue can be reproduced, for example, in a QEMU VM with '-machine virt'
>> and 2 or more CPUs, with Linux as the guest OS.
> 
> Good catch. This is a valid issue which needs to be fixed.
> 
>>
>> This patch adds an auxiliary per-hart state variable, aux_state, as well as
>> its accessor functions, to synchronize execution of the boot hart and the
>> starting secondary hart.
>>
>> Initially, aux_state is 0: sbi_scratch_alloc_offset() zeroes the allocated
>> memory. The boot hart sets it to 1 after it has set next_addr, etc., for the
>> secondary hart, then sends the IPI to the latter.
>>
>> The secondary hart waits in sbi_hsm_hart_wait() both for aux_state to become
>> non-zero and for an IPI to happen. This should be safe even if the secondary
>> hart somehow gets an IPI unrelated to sbi_hsm_hart_start().
>>
>> Memory barriers are used to make sure the writes to next_addr, etc., in the
>> boot hart are visible to the secondary hart if it reads non-zero aux_state.
>>
>> Only one bit of aux_state is actually used here, so, this per-hart variable
>> can be used to hold other data in the future, if needed.
> 
> Adding aux_state is an overly complicated solution in my opinion.
> 
> Instead, you can simply do the following:
> 
>      rscratch->next_arg1 = arg1;
>      rscratch->next_addr = saddr;
>      rscratch->next_mode = smode;
> 
>      /*
>       * Barrier to ensure that the above writes have happened before
>       * updating the HART state.
>       */
>      smp_wmb();
> 
> Before "hdata = sbi_scratch_offset_ptr(rscratch, hart_data_offset);" in
> sbi_hsm_hart_start().

True, the proposed fix it is more complicated than I would like it to 
be, but there is a reason.

Should we worry about the scenario when two or more harts enter 
sbi_hsm_hart_start() with the same target hartid? It must not happen 
during normal operation of the OS, but, in theory, errors in the 
higher-level software could lead to this.

Setting hstate to SBI_HSM_STATE_START_PENDING atomically and checking 
its previous values before changing *rscratch in sbi_hsm_hart_start() 
guarantees that the current hart has exclusive access to *rscratch after 
that. If we move rscratch->next_arg1 = arg1, etc., before hdata = 
sbi_scratch_offset_ptr(...), it would be possible for two harts to race 
and write different values to *rscratch for the same target hart. It is 
very unlikely to happen, but still.

If it is somehow guaranteed that the current hart has exclusive access 
to *rscratch even at the beginning of sbi_hsm_hart_start(), then yes, we 
can drop aux_state, etc., and use a simpler fix.

smp_wmb() will then be paired with rmb() in atomic_read(&hdata->state) 
from sbi_hsm_hart_wait() and it would make sure the writes are seen in 
correct order.

So, should we worry about misbehaving OSes which could allow 2 or more 
harts to enter sbi_hsm_hart_start() for the same target hart in parallel?

What do you think?

> 
>>
>> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
> 
>> ---
>>   lib/sbi/sbi_hsm.c | 35 +++++++++++++++++++++++++++++++++--
>>   1 file changed, 33 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
>> index 8499bb1..24061c5 100644
>> --- a/lib/sbi/sbi_hsm.c
>> +++ b/lib/sbi/sbi_hsm.c
>> @@ -35,6 +35,7 @@ struct sbi_hsm_data {
>>          unsigned long suspend_type;
>>          unsigned long saved_mie;
>>          unsigned long saved_mip;
>> +       unsigned long aux_state;
>>   };
>>
>>   static inline int __sbi_hsm_hart_get_state(u32 hartid)
>> @@ -93,6 +94,23 @@ int sbi_hsm_hart_interruptible_mask(const struct sbi_domain *dom,
>>          return 0;
>>   }
>>
>> +static inline bool sbi_hsm_is_runnable(struct sbi_hsm_data *hdata)
>> +{
>> +       unsigned long aux_state = __smp_load_acquire(&hdata->aux_state);
>> +
>> +       return !!aux_state;
>> +}
>> +
>> +static inline void sbi_hsm_set_runnable(struct sbi_hsm_data *hdata)
>> +{
>> +       __smp_store_release(&hdata->aux_state, 1);
>> +}
>> +
>> +static inline void sbi_hsm_reset_runnable(struct sbi_hsm_data *hdata)
>> +{
>> +       __smp_store_release(&hdata->aux_state, 0);
>> +}
>> +
>>   void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>>   {
>>          u32 oldstate;
>> @@ -108,6 +126,7 @@ void sbi_hsm_prepare_next_jump(struct sbi_scratch *scratch, u32 hartid)
>>   static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>   {
>>          unsigned long saved_mie;
>> +       unsigned long cmip;
>>          struct sbi_hsm_data *hdata = sbi_scratch_offset_ptr(scratch,
>>                                                              hart_data_offset);
>>          /* Save MIE CSR */
>> @@ -117,13 +136,22 @@ static void sbi_hsm_hart_wait(struct sbi_scratch *scratch, u32 hartid)
>>          csr_set(CSR_MIE, MIP_MSIP | MIP_MEIP);
>>
>>          /* Wait for state transition requested by sbi_hsm_hart_start() */
>> -       while (atomic_read(&hdata->state) != SBI_HSM_STATE_START_PENDING) {
>> -               wfi();
>> +       while (!sbi_hsm_is_runnable(hdata)) {
>> +               do {
>> +                       wfi();
>> +                       cmip = csr_read(CSR_MIP);
>> +               } while (!(cmip & (MIP_MSIP | MIP_MEIP)));
>>          };
>>
>>          /* Restore MIE CSR */
>>          csr_write(CSR_MIE, saved_mie);
>>
>> +       /*
>> +        * If the hart ever needs to wait for such state transition again,
>> +        * make sure the hart's data allows that.
>> +        */
>> +       sbi_hsm_reset_runnable(hdata);
>> +
>>          /*
>>           * No need to clear IPI here because the sbi_ipi_init() will
>>           * clear it for current HART via sbi_platform_ipi_init().
>> @@ -284,6 +312,9 @@ int sbi_hsm_hart_start(struct sbi_scratch *scratch,
>>          rscratch->next_addr = saddr;
>>          rscratch->next_mode = smode;
>>
>> +       /* Inform the hart that it may continue as soon as it gets IPI. */
>> +       sbi_hsm_set_runnable(hdata);
>> +
>>          if (hsm_device_has_hart_hotplug() ||
>>             (hsm_device_has_hart_secondary_boot() && !init_count)) {
>>                  return hsm_device_hart_start(hartid, scratch->warmboot_addr);
>> --
>> 2.34.1
>>
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi
> 
> Regards,
> Anup
> 

Regards,
Evgenii



From e.shatokhin at yadro.com  Tue Feb 28 08:19:27 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Tue, 28 Feb 2023 19:19:27 +0300
Subject: [PATCH v2 2/2] lib: sbi: Clear IPIs before init_warm_startup in
 non-boot harts
In-Reply-To: <CAAhSdy1KWF_j=uVdijefit8ZD1nqDtSNh=3gZy96E_JxUEForw@mail.gmail.com>
References: <20230221210851.3145840-1-e.shatokhin@yadro.com>
 <20230221210851.3145840-3-e.shatokhin@yadro.com>
 <CAAhSdy1KWF_j=uVdijefit8ZD1nqDtSNh=3gZy96E_JxUEForw@mail.gmail.com>
Message-ID: <9293d8b4-e0b7-7ce6-2df9-afaf1bb5dec9@yadro.com>

On 28.02.2023 07:41, Anup Patel wrote:
> On Wed, Feb 22, 2023 at 2:39 AM Evgenii Shatokhin <e.shatokhin at yadro.com> wrote:
>>
>> Since commit 50d4fde1c5a4 ("lib: Remove redundant sbi_platform_ipi_clear()
>> calls"), the IPI sent from the boot hart in wake_coldboot_harts() is not
>> cleared in the secondary harts until they reach sbi_ipi_init(). However,
>> sbi_hsm_init() and sbi_hsm_hart_wait() are called earlier, so a secondary
>> hart might enter sbi_hsm_hart_wait() with an already pending IPI.
>>
>> sbi_hsm_hart_wait() makes sure the hart leaves the loop only when it is
>> actually ready, so a pending unrelated IPI should not cause safety issues.
>> However, it might be inefficient on certain hardware, because it prevents
>> "wfi" from stalling the hart even if the hardware supports this, making the
>> hart needlessly spin in a "busy-wait" loop.
>>
>> This behaviour can be observed, for example, in a QEMU VM (QEMU 7.2.0) with
>> "-machine virt" running a Linux guest. Inserting delays in
>> sbi_hsm_hart_start() allows reproducing the issue more reliably.
>>
>> The comment in wait_for_coldboot() suggests that the initial IPI is needed
>> in the warm resume path, so let us clear it before init_warm_startup()
>> only.
>>
>> To do this, sbi_ipi_raw_clear() was created similar to sbi_ipi_raw_send().
>>
>> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
>> ---
>>   include/sbi/sbi_ipi.h | 2 ++
>>   lib/sbi/sbi_init.c    | 6 ++++--
>>   lib/sbi/sbi_ipi.c     | 6 ++++++
>>   3 files changed, 12 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
>> index f6ac807..f384e74 100644
>> --- a/include/sbi/sbi_ipi.h
>> +++ b/include/sbi/sbi_ipi.h
>> @@ -77,6 +77,8 @@ void sbi_ipi_process(void);
>>
>>   int sbi_ipi_raw_send(u32 target_hart);
>>
>> +void sbi_ipi_raw_clear(u32 target_hart);
>> +
>>   const struct sbi_ipi_device *sbi_ipi_get_device(void);
>>
>>   void sbi_ipi_set_device(const struct sbi_ipi_device *dev);
>> diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
>> index e353c33..a9a4084 100644
>> --- a/lib/sbi/sbi_init.c
>> +++ b/lib/sbi/sbi_init.c
>> @@ -438,10 +438,12 @@ static void __noreturn init_warmboot(struct sbi_scratch *scratch, u32 hartid)
>>          if (hstate < 0)
>>                  sbi_hart_hang();
>>
>> -       if (hstate == SBI_HSM_STATE_SUSPENDED)
>> +       if (hstate == SBI_HSM_STATE_SUSPENDED) {
>>                  init_warm_resume(scratch);
>> -       else
>> +       } else {
>> +               sbi_ipi_raw_clear(hartid);
> 
> This effectively does nothing in many cases because for a lot of
> platforms all HARTs enter OpenSBI at the same time and the HARTs
> entering warm boot path will call sbi_ipi_raw_clear() even before
> ipi_dev is provided by platform code.

Right, it would be a no-op then.

However, this is not the case when Linux 6.x boots, e.g. in QEMU with 
"-machine virt" or with other platforms which provide "cold" IPI init 
callbacks.

Here, the boot hart initializes ipi_dev early. For example, in QEMU with 
"-machine virt", it does the following:
init_coldboot() => sbi_ipi_init() => ... => aclint_mswi_cold_init().

Other harts are waiting in sbi_hsm_init() at the moment and will only 
call sbi_ipi_init() after they leave sbi_hsm_hart_wait(), etc.

While they are waiting, their previous IPI remains pending. It can be 
easily demonstrated if one checks the relevant bits of CSR_MIP in 
sbi_hsm_hart_wait() right before the wfi loop. A delay in 
sbi_hsm_hart_start(), same as for the first issue, also helps make it 
visible.

csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP) == 0x8 in my case at that 
point. That is, MIP_MSIP bit is set even before the boot hart has send 
an IPI to the target hart in sbi_hsm_hart_start(). Not good.

So, sbi_ipi_raw_clear(hartid) really helps in such cases, even if it is 
a no-op in other situations.

> 
>>                  init_warm_startup(scratch, hartid);
>> +       }
>>
>>          sbi_hart_switch_mode(hartid, scratch->next_arg1,
>>                               scratch->next_addr,
>> diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
>> index 1bcc2e4..b9f6205 100644
>> --- a/lib/sbi/sbi_ipi.c
>> +++ b/lib/sbi/sbi_ipi.c
>> @@ -217,6 +217,12 @@ int sbi_ipi_raw_send(u32 target_hart)
>>          return 0;
>>   }
>>
>> +void sbi_ipi_raw_clear(u32 target_hart)
>> +{
>> +       if (ipi_dev && ipi_dev->ipi_clear)
>> +               ipi_dev->ipi_clear(target_hart);
>> +}
>> +
>>   const struct sbi_ipi_device *sbi_ipi_get_device(void)
>>   {
>>          return ipi_dev;
>> --
>> 2.34.1
>>
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi
> 
> Regards,
> Anup
> 

Regards,
Evgenii




From e.shatokhin at yadro.com  Tue Feb 28 08:54:54 2023
From: e.shatokhin at yadro.com (Evgenii Shatokhin)
Date: Tue, 28 Feb 2023 19:54:54 +0300
Subject: [PATCH v2 2/2] lib: sbi: Clear IPIs before init_warm_startup in
 non-boot harts
In-Reply-To: <9293d8b4-e0b7-7ce6-2df9-afaf1bb5dec9@yadro.com>
References: <20230221210851.3145840-1-e.shatokhin@yadro.com>
 <20230221210851.3145840-3-e.shatokhin@yadro.com>
 <CAAhSdy1KWF_j=uVdijefit8ZD1nqDtSNh=3gZy96E_JxUEForw@mail.gmail.com>
 <9293d8b4-e0b7-7ce6-2df9-afaf1bb5dec9@yadro.com>
Message-ID: <ff7778a5-a48a-6248-c78a-510afc0340f0@yadro.com>

On 28.02.2023 19:19, Evgenii Shatokhin wrote:
> On 28.02.2023 07:41, Anup Patel wrote:
>> On Wed, Feb 22, 2023 at 2:39 AM Evgenii Shatokhin 
>> <e.shatokhin at yadro.com> wrote:
>>>
>>> Since commit 50d4fde1c5a4 ("lib: Remove redundant 
>>> sbi_platform_ipi_clear()
>>> calls"), the IPI sent from the boot hart in wake_coldboot_harts() is not
>>> cleared in the secondary harts until they reach sbi_ipi_init(). However,
>>> sbi_hsm_init() and sbi_hsm_hart_wait() are called earlier, so a 
>>> secondary
>>> hart might enter sbi_hsm_hart_wait() with an already pending IPI.
>>>
>>> sbi_hsm_hart_wait() makes sure the hart leaves the loop only when it is
>>> actually ready, so a pending unrelated IPI should not cause safety 
>>> issues.
>>> However, it might be inefficient on certain hardware, because it 
>>> prevents
>>> "wfi" from stalling the hart even if the hardware supports this, 
>>> making the
>>> hart needlessly spin in a "busy-wait" loop.
>>>
>>> This behaviour can be observed, for example, in a QEMU VM (QEMU 
>>> 7.2.0) with
>>> "-machine virt" running a Linux guest. Inserting delays in
>>> sbi_hsm_hart_start() allows reproducing the issue more reliably.
>>>
>>> The comment in wait_for_coldboot() suggests that the initial IPI is 
>>> needed
>>> in the warm resume path, so let us clear it before init_warm_startup()
>>> only.
>>>
>>> To do this, sbi_ipi_raw_clear() was created similar to 
>>> sbi_ipi_raw_send().
>>>
>>> Signed-off-by: Evgenii Shatokhin <e.shatokhin at yadro.com>
>>> ---
>>> ? include/sbi/sbi_ipi.h | 2 ++
>>> ? lib/sbi/sbi_init.c??? | 6 ++++--
>>> ? lib/sbi/sbi_ipi.c???? | 6 ++++++
>>> ? 3 files changed, 12 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/include/sbi/sbi_ipi.h b/include/sbi/sbi_ipi.h
>>> index f6ac807..f384e74 100644
>>> --- a/include/sbi/sbi_ipi.h
>>> +++ b/include/sbi/sbi_ipi.h
>>> @@ -77,6 +77,8 @@ void sbi_ipi_process(void);
>>>
>>> ? int sbi_ipi_raw_send(u32 target_hart);
>>>
>>> +void sbi_ipi_raw_clear(u32 target_hart);
>>> +
>>> ? const struct sbi_ipi_device *sbi_ipi_get_device(void);
>>>
>>> ? void sbi_ipi_set_device(const struct sbi_ipi_device *dev);
>>> diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c
>>> index e353c33..a9a4084 100644
>>> --- a/lib/sbi/sbi_init.c
>>> +++ b/lib/sbi/sbi_init.c
>>> @@ -438,10 +438,12 @@ static void __noreturn init_warmboot(struct 
>>> sbi_scratch *scratch, u32 hartid)
>>> ???????? if (hstate < 0)
>>> ???????????????? sbi_hart_hang();
>>>
>>> -?????? if (hstate == SBI_HSM_STATE_SUSPENDED)
>>> +?????? if (hstate == SBI_HSM_STATE_SUSPENDED) {
>>> ???????????????? init_warm_resume(scratch);
>>> -?????? else
>>> +?????? } else {
>>> +?????????????? sbi_ipi_raw_clear(hartid);
>>
>> This effectively does nothing in many cases because for a lot of
>> platforms all HARTs enter OpenSBI at the same time and the HARTs
>> entering warm boot path will call sbi_ipi_raw_clear() even before
>> ipi_dev is provided by platform code.
> 
> Right, it would be a no-op then.
> 
> However, this is not the case when Linux 6.x boots, e.g. in QEMU with 
> "-machine virt" or with other platforms which provide "cold" IPI init 
> callbacks.
> 
> Here, the boot hart initializes ipi_dev early. For example, in QEMU with 
> "-machine virt", it does the following:
> init_coldboot() => sbi_ipi_init() => ... => aclint_mswi_cold_init().
> 
> Other harts are waiting in sbi_hsm_init() at the moment and will only 
> call sbi_ipi_init() after they leave sbi_hsm_hart_wait(), etc.

Correction: the secondary harts are waiting in init_warmboot() at that 
moment rather than in sbi_hsm_init(). Sorry, my bad.

Still, the boot hart only wakes them up with wake_coldboot_harts() after 
it has called sbi_ipi_init(), etc. So, by the time the secondary harts 
get to init_warm_startup(), ipi_dev should already be initialized and 
sbi_ipi_raw_clear() should work.

> 
> While they are waiting, their previous IPI remains pending. It can be 
> easily demonstrated if one checks the relevant bits of CSR_MIP in 
> sbi_hsm_hart_wait() right before the wfi loop. A delay in 
> sbi_hsm_hart_start(), same as for the first issue, also helps make it 
> visible.
> 
> csr_read(CSR_MIP) & (MIP_MSIP | MIP_MEIP) == 0x8 in my case at that 
> point. That is, MIP_MSIP bit is set even before the boot hart has send 
> an IPI to the target hart in sbi_hsm_hart_start(). Not good.
> 
> So, sbi_ipi_raw_clear(hartid) really helps in such cases, even if it is 
> a no-op in other situations.


> 
>>
>>> ???????????????? init_warm_startup(scratch, hartid);
>>> +?????? }
>>>
>>> ???????? sbi_hart_switch_mode(hartid, scratch->next_arg1,
>>> ????????????????????????????? scratch->next_addr,
>>> diff --git a/lib/sbi/sbi_ipi.c b/lib/sbi/sbi_ipi.c
>>> index 1bcc2e4..b9f6205 100644
>>> --- a/lib/sbi/sbi_ipi.c
>>> +++ b/lib/sbi/sbi_ipi.c
>>> @@ -217,6 +217,12 @@ int sbi_ipi_raw_send(u32 target_hart)
>>> ???????? return 0;
>>> ? }
>>>
>>> +void sbi_ipi_raw_clear(u32 target_hart)
>>> +{
>>> +?????? if (ipi_dev && ipi_dev->ipi_clear)
>>> +?????????????? ipi_dev->ipi_clear(target_hart);
>>> +}
>>> +
>>> ? const struct sbi_ipi_device *sbi_ipi_get_device(void)
>>> ? {
>>> ???????? return ipi_dev;
>>> -- 
>>> 2.34.1
>>>
>>>
>>>
>>> -- 
>>> opensbi mailing list
>>> opensbi at lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/opensbi
>>
>> Regards,
>> Anup
>>

Regards,
Evgenii



From anup at brainfault.org  Tue Feb 28 19:47:57 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Mar 2023 09:17:57 +0530
Subject: [PATCH] gpio/starfive: add gpio driver and support gpio reset
In-Reply-To: <20230216092126.99022-1-minda.chen@starfivetech.com>
References: <20230216092126.99022-1-minda.chen@starfivetech.com>
Message-ID: <CAAhSdy3tpF8rUfO=6mJqwReRZB-NrMdY=vYs2ZqFdxxNtYe27w@mail.gmail.com>

On Thu, Feb 16, 2023 at 2:52 PM minda.chen <minda.chen at starfivetech.com> wrote:
>
> Add gpio driver and gpio reset function in Starfive
> JH7110 SOC platform.
>
> Signed-off-by: minda.chen <minda.chen at starfivetech.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  lib/utils/gpio/Kconfig             |   3 +
>  lib/utils/gpio/fdt_gpio_starfive.c | 117 +++++++++++++++++++++++++++++
>  lib/utils/gpio/objects.mk          |   3 +
>  platform/generic/configs/defconfig |   1 +
>  4 files changed, 124 insertions(+)
>  create mode 100644 lib/utils/gpio/fdt_gpio_starfive.c
>
> diff --git a/lib/utils/gpio/Kconfig b/lib/utils/gpio/Kconfig
> index 38a9d75..1b338a6 100644
> --- a/lib/utils/gpio/Kconfig
> +++ b/lib/utils/gpio/Kconfig
> @@ -14,6 +14,9 @@ config FDT_GPIO_SIFIVE
>         bool "SiFive GPIO FDT driver"
>         default n
>
> +config FDT_GPIO_STARFIVE
> +       bool "StarFive GPIO FDT driver"
> +       default n
>  endif
>
>  config GPIO
> diff --git a/lib/utils/gpio/fdt_gpio_starfive.c b/lib/utils/gpio/fdt_gpio_starfive.c
> new file mode 100644
> index 0000000..18cca72
> --- /dev/null
> +++ b/lib/utils/gpio/fdt_gpio_starfive.c
> @@ -0,0 +1,117 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Starfive
> + *
> + * Authors:
> + *   Minda.chen <Minda.chen at starfivetech.com>
> + */
> +
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +#include <sbi_utils/gpio/fdt_gpio.h>
> +
> +#define STARFIVE_GPIO_CHIP_MAX         2
> +#define STARFIVE_GPIO_PINS_DEF         64
> +#define STARFIVE_GPIO_OUTVAL           0x40
> +#define STARFIVE_GPIO_MASK             0xff
> +#define STARFIVE_GPIO_REG_SHIFT_MASK   0x3
> +#define STARFIVE_GPIO_SHIFT_BITS       0x3
> +
> +struct starfive_gpio_chip {
> +       unsigned long addr;
> +       struct gpio_chip chip;
> +};
> +
> +static unsigned int starfive_gpio_chip_count;
> +static struct starfive_gpio_chip starfive_gpio_chip_array[STARFIVE_GPIO_CHIP_MAX];
> +
> +static int starfive_gpio_direction_output(struct gpio_pin *gp, int value)
> +{
> +       u32 val;
> +       unsigned long reg_addr;
> +       u32 bit_mask, shift_bits;
> +       struct starfive_gpio_chip *chip =
> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
> +
> +       /* set out en*/
> +       reg_addr = chip->addr + gp->offset;
> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
> +
> +       val = readl((void *)(reg_addr));
> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
> +               << STARFIVE_GPIO_SHIFT_BITS;
> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
> +
> +       val = readl((void *)reg_addr);
> +       val &= ~bit_mask;
> +       writel(val, (void *)reg_addr);
> +
> +       return 0;
> +}
> +
> +static void starfive_gpio_set(struct gpio_pin *gp, int value)
> +{
> +       u32 val;
> +       unsigned long reg_addr;
> +       u32 bit_mask, shift_bits;
> +       struct starfive_gpio_chip *chip =
> +               container_of(gp->chip, struct starfive_gpio_chip, chip);
> +
> +       reg_addr = chip->addr + gp->offset;
> +       reg_addr &= ~(STARFIVE_GPIO_REG_SHIFT_MASK);
> +
> +       shift_bits = (gp->offset & STARFIVE_GPIO_REG_SHIFT_MASK)
> +               << STARFIVE_GPIO_SHIFT_BITS;
> +       bit_mask = STARFIVE_GPIO_MASK << shift_bits;
> +       /* set output value */
> +       val = readl((void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
> +       val &= ~bit_mask;
> +       val |= value << shift_bits;
> +       writel(val, (void *)(reg_addr + STARFIVE_GPIO_OUTVAL));
> +}
> +
> +extern struct fdt_gpio fdt_gpio_starfive;
> +
> +static int starfive_gpio_init(void *fdt, int nodeoff, u32 phandle,
> +                             const struct fdt_match *match)
> +{
> +       int rc;
> +       struct starfive_gpio_chip *chip;
> +       u64 addr;
> +
> +       if (starfive_gpio_chip_count >= STARFIVE_GPIO_CHIP_MAX)
> +               return SBI_ENOSPC;
> +       chip = &starfive_gpio_chip_array[starfive_gpio_chip_count];
> +
> +       rc = fdt_get_node_addr_size(fdt, nodeoff, 0, &addr, NULL);
> +       if (rc)
> +               return rc;
> +
> +       chip->addr = addr;
> +       chip->chip.driver = &fdt_gpio_starfive;
> +       chip->chip.id = phandle;
> +       chip->chip.ngpio = STARFIVE_GPIO_PINS_DEF;
> +       chip->chip.direction_output = starfive_gpio_direction_output;
> +       chip->chip.set = starfive_gpio_set;
> +       rc = gpio_chip_add(&chip->chip);
> +       if (rc)
> +               return rc;
> +
> +       starfive_gpio_chip_count++;
> +       return 0;
> +}
> +
> +static const struct fdt_match starfive_gpio_match[] = {
> +       { .compatible = "starfive,jh7110-sys-pinctrl" },
> +       { .compatible = "starfive,iomux-pinctrl" },
> +       { },
> +};
> +
> +struct fdt_gpio fdt_gpio_starfive = {
> +       .match_table = starfive_gpio_match,
> +       .xlate = fdt_gpio_simple_xlate,
> +       .init = starfive_gpio_init,
> +};
> diff --git a/lib/utils/gpio/objects.mk b/lib/utils/gpio/objects.mk
> index eedd699..8f4a6a8 100644
> --- a/lib/utils/gpio/objects.mk
> +++ b/lib/utils/gpio/objects.mk
> @@ -13,4 +13,7 @@ libsbiutils-objs-$(CONFIG_FDT_GPIO) += gpio/fdt_gpio_drivers.o
>  carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_SIFIVE) += fdt_gpio_sifive
>  libsbiutils-objs-$(CONFIG_FDT_GPIO_SIFIVE) += gpio/fdt_gpio_sifive.o
>
> +carray-fdt_gpio_drivers-$(CONFIG_FDT_GPIO_STARFIVE) += fdt_gpio_starfive
> +libsbiutils-objs-$(CONFIG_FDT_GPIO_STARFIVE) += gpio/fdt_gpio_starfive.o
> +
>  libsbiutils-objs-$(CONFIG_GPIO) += gpio/gpio.o
> diff --git a/platform/generic/configs/defconfig b/platform/generic/configs/defconfig
> index 4b0842e..2ad953d 100644
> --- a/platform/generic/configs/defconfig
> +++ b/platform/generic/configs/defconfig
> @@ -6,6 +6,7 @@ CONFIG_PLATFORM_SIFIVE_FU740=y
>  CONFIG_PLATFORM_STARFIVE_JH7110=y
>  CONFIG_FDT_GPIO=y
>  CONFIG_FDT_GPIO_SIFIVE=y
> +CONFIG_FDT_GPIO_STARFIVE=y
>  CONFIG_FDT_I2C=y
>  CONFIG_FDT_I2C_SIFIVE=y
>  CONFIG_FDT_IPI=y
> --
> 2.17.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Feb 28 20:00:02 2023
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Mar 2023 09:30:02 +0530
Subject: [PATCH] make: Add a command line option for debugging OpenSBI
In-Reply-To: <20230227023506.1818620-1-bmeng@tinylab.org>
References: <20230227023506.1818620-1-bmeng@tinylab.org>
Message-ID: <CAAhSdy2BhBHz6zA=Ni5YScQ21noxyn-HzygCd+om4OpsxjSUJQ@mail.gmail.com>

On Mon, Feb 27, 2023 at 8:05 AM Bin Meng <bmeng at tinylab.org> wrote:
>
> Add a new make command line option "make DEBUG=1" to prevent compiler
> optimizations using -O2.
>
> Signed-off-by: Bin Meng <bmeng at tinylab.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
>  Makefile  |  7 ++++++-
>  README.md | 13 +++++++++++++
>  2 files changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index b20404f..a26a39b 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -331,7 +331,12 @@ GENFLAGS   +=      $(libsbiutils-genflags-y)
>  GENFLAGS       +=      $(platform-genflags-y)
>  GENFLAGS       +=      $(firmware-genflags-y)
>
> -CFLAGS         =       -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2
> +CFLAGS         =       -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing
> +ifneq ($(DEBUG),)
> +CFLAGS         +=      -O0
> +else
> +CFLAGS         +=      -O2
> +endif
>  CFLAGS         +=      -fno-omit-frame-pointer -fno-optimize-sibling-calls -mstrict-align
>  # enable -m(no-)save-restore option by CC_SUPPORT_SAVE_RESTORE
>  ifeq ($(CC_SUPPORT_SAVE_RESTORE),y)
> diff --git a/README.md b/README.md
> index 895bbf2..a6bcb0c 100644
> --- a/README.md
> +++ b/README.md
> @@ -298,6 +298,19 @@ NOTE: Using `BUILD_INFO=y` without specifying SOURCE_DATE_EPOCH will violate
>  purpose, and should NOT be used in a product which follows "reproducible
>  builds".
>
> +Building with optimization off for debugging
> +--------------------------------------------
> +
> +When debugging OpenSBI, we may want to turn off the compiler optimization and
> +make debugging produce the expected results for a better debugging experience.
> +To build with optimization off we can just simply add `DEBUG=1`, like:
> +```
> +make DEBUG=1
> +```
> +
> +This definition is ONLY for development and debug purpose, and should NOT be
> +used in a product build.
> +
>  Contributing to OpenSBI
>  -----------------------
>
> --
> 2.25.1
>


