From anup at brainfault.org  Wed Jun  1 03:51:01 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 16:21:01 +0530
Subject: [PATCH] lib: sbi_timer: Drop unnecessary get_platform_ticks wrapper
In-Reply-To: <CAAhSdy1=LbZCtVOiQG_E4MAHOTixMu2hsrG_jP=AcUL3NzgFNg@mail.gmail.com>
References: <20220529170048.60646-1-samuel@sholland.org> <CAAhSdy1=LbZCtVOiQG_E4MAHOTixMu2hsrG_jP=AcUL3NzgFNg@mail.gmail.com>
Message-ID: <CAAhSdy1+2jXiyASqqqnnncP2AjQoskXmrM==n-GAuX+ddW2pdw@mail.gmail.com>

On Mon, May 30, 2022 at 10:48 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Sun, May 29, 2022 at 10:31 PM Samuel Holland <samuel at sholland.org> wrote:
> >
> > The device's timer_value callback is already the right prototype to use
> > for the get_time_val function pointer.
> >
> > Signed-off-by: Samuel Holland <samuel at sholland.org>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo

Thanks,
Anup

>
> Regards,
> Anup
>
> > ---
> >
> >  lib/sbi/sbi_timer.c | 7 +------
> >  1 file changed, 1 insertion(+), 6 deletions(-)
> >
> > diff --git a/lib/sbi/sbi_timer.c b/lib/sbi/sbi_timer.c
> > index 353886f..b0a6e63 100644
> > --- a/lib/sbi/sbi_timer.c
> > +++ b/lib/sbi/sbi_timer.c
> > @@ -44,11 +44,6 @@ static u64 get_ticks(void)
> >  }
> >  #endif
> >
> > -static u64 get_platform_ticks(void)
> > -{
> > -       return timer_dev->timer_value();
> > -}
> > -
> >  static void nop_delay_fn(void *opaque)
> >  {
> >         cpu_relax();
> > @@ -167,7 +162,7 @@ void sbi_timer_set_device(const struct sbi_timer_device *dev)
> >
> >         timer_dev = dev;
> >         if (!get_time_val && timer_dev->timer_value)
> > -               get_time_val = get_platform_ticks;
> > +               get_time_val = timer_dev->timer_value;
> >  }
> >
> >  int sbi_timer_init(struct sbi_scratch *scratch, bool cold_boot)
> > --
> > 2.35.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Jun  1 03:51:28 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 16:21:28 +0530
Subject: [PATCH 1/2] lib: sbi_illegal_insn: Constify illegal_insn_table
In-Reply-To: <CAAhSdy39KPsBzvmgbO2Vo-ADC5ocjn-nXRYmpb4d5+7qoZDVBQ@mail.gmail.com>
References: <20220529182531.30377-1-samuel@sholland.org> <CAAhSdy39KPsBzvmgbO2Vo-ADC5ocjn-nXRYmpb4d5+7qoZDVBQ@mail.gmail.com>
Message-ID: <CAAhSdy1M5mFkesG9sinGMQKRreWtOZBJaYtr1sCkkv4v-+6Chg@mail.gmail.com>

On Mon, May 30, 2022 at 11:30 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Sun, May 29, 2022 at 11:55 PM Samuel Holland <samuel at sholland.org> wrote:
> >
> > This table does not need to be modified at runtime.
> >
> > Signed-off-by: Samuel Holland <samuel at sholland.org>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo

Thanks,
Anup

>
> Regards,
> Anup
>
> > ---
> >
> >  lib/sbi/sbi_illegal_insn.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
> > index bfe7d61..39ef990 100644
> > --- a/lib/sbi/sbi_illegal_insn.c
> > +++ b/lib/sbi/sbi_illegal_insn.c
> > @@ -80,7 +80,7 @@ static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
> >         return 0;
> >  }
> >
> > -static illegal_insn_func illegal_insn_table[32] = {
> > +static const illegal_insn_func illegal_insn_table[32] = {
> >         truly_illegal_insn, /* 0 */
> >         truly_illegal_insn, /* 1 */
> >         truly_illegal_insn, /* 2 */
> > --
> > 2.35.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Jun  1 04:54:01 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 17:24:01 +0530
Subject: [PATCH 2/2] lib: sbi_illegal_insn: Add emulation for fence.tso
In-Reply-To: <CAAhSdy2qJNafqQnQfYvR1fUop2siv7eDNYQU6WMz61ui_7FuNQ@mail.gmail.com>
References: <20220529182531.30377-1-samuel@sholland.org> <20220529182531.30377-2-samuel@sholland.org>
 <CAAhSdy2qJNafqQnQfYvR1fUop2siv7eDNYQU6WMz61ui_7FuNQ@mail.gmail.com>
Message-ID: <CAAhSdy0VLbMt=MXBq4xWys9wkfX+TOBZW9mFxRjDT1hj5ty=SA@mail.gmail.com>

On Mon, May 30, 2022 at 11:38 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Sun, May 29, 2022 at 11:55 PM Samuel Holland <samuel at sholland.org> wrote:
> >
> > While OpenC906 appears to properly decode `fence.tso` as a fence
> > instruction[1], the version of the C906 taped out in the Allwinner D1
> > does not, and raises illegal instruction.
> >
> > Handle this errata by emulating `fence.tso` as `fence rw, rw`.
> >
> > [1]: https://github.com/T-head-Semi/openc906/blob/30827e7f/C906_RTL_FACTORY/gen_rtl/idu/rtl/aq_idu_id_decd.v#L2097
> >
> > Signed-off-by: Samuel Holland <samuel at sholland.org>
> > ---
> >
> >  lib/sbi/sbi_illegal_insn.c | 14 +++++++++++++-
> >  1 file changed, 13 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/sbi/sbi_illegal_insn.c b/lib/sbi/sbi_illegal_insn.c
> > index 39ef990..44c9385 100644
> > --- a/lib/sbi/sbi_illegal_insn.c
> > +++ b/lib/sbi/sbi_illegal_insn.c
> > @@ -8,6 +8,7 @@
> >   */
> >
> >  #include <sbi/riscv_asm.h>
> > +#include <sbi/riscv_barrier.h>
> >  #include <sbi/riscv_encoding.h>
> >  #include <sbi/sbi_bitops.h>
> >  #include <sbi/sbi_emulate_csr.h>
> > @@ -32,6 +33,17 @@ static int truly_illegal_insn(ulong insn, struct sbi_trap_regs *regs)
> >         return sbi_trap_redirect(regs, &trap);
> >  }
> >
> > +static int misc_mem_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
> > +{
> > +       /* Errata workaround: emulate `fence.tso` as `fence rw, rw`. */
> > +       if (insn == 0x8330000fUL) {
>
> Instead of hard-coding "fence.tso" encoding, I suggest to add match
> and mask defines in riscv_encoding.h.
>
> Something like this:
> #define INSN_MATCH_FENCE_TSO 0x8330000f
> #define INSN_MASK_FENCE_TSO 0xffffffff
>
> > +               smp_mb();
> > +               return 0;
> > +       }
> > +
> > +       return truly_illegal_insn(insn, regs);
> > +}
> > +
> >  static int system_opcode_insn(ulong insn, struct sbi_trap_regs *regs)
> >  {
> >         int do_write, rs1_num = (insn >> 15) & 0x1f;
> > @@ -84,7 +96,7 @@ static const illegal_insn_func illegal_insn_table[32] = {
> >         truly_illegal_insn, /* 0 */
> >         truly_illegal_insn, /* 1 */
> >         truly_illegal_insn, /* 2 */
> > -       truly_illegal_insn, /* 3 */
> > +       misc_mem_opcode_insn, /* 3 */
> >         truly_illegal_insn, /* 4 */
> >         truly_illegal_insn, /* 5 */
> >         truly_illegal_insn, /* 6 */
> > --
> > 2.35.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>
> Apart from a minor comment above, this looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

I have taken care of the minor comment at the time of merging
this patch.

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
> Regards,
> Anup


From anup at brainfault.org  Wed Jun  1 05:11:37 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 17:41:37 +0530
Subject: [RFC PATCH v2 2/9] lib: sbi_hsm: Assume a consistent resume address
In-Reply-To: <20220530033738.27127-3-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-3-samuel@sholland.org>
Message-ID: <CAAhSdy1fE4j2q9EBWmwcrT2dH8ExTfCnpVuAyNhJAYEP3rabwA@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> The suspend code needs to know the resume address for two reasons:
>   1) Programing some hardware register or management firmware. Here we
>      assume the hardware/firmware maintains its state between suspends,
>      so it only needs to be programmed once at startup.

s/Programing/Programming/

>   2) When a non-retentive suspend request ends up being retentive, due
>      to lack of hardware support, pending interrupt, or for some other
>      reason. However, the behavior here is not platform-dependent, and
>      this can be handled in the generic hart suspend function.
>
> Since neither situation requires the platform-level suspend function to
> know the resume address, stop passing it to that function. Instead,
> handle the non-retentive to retentive situation generically.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi/sbi_hsm.h |  4 ++--
>  lib/sbi/sbi_hsm.c     | 46 +++++++++++++++++++------------------------
>  2 files changed, 22 insertions(+), 28 deletions(-)
>
> diff --git a/include/sbi/sbi_hsm.h b/include/sbi/sbi_hsm.h
> index 6da8cee..d6cc468 100644
> --- a/include/sbi/sbi_hsm.h
> +++ b/include/sbi/sbi_hsm.h
> @@ -34,9 +34,9 @@ struct sbi_hsm_device {
>          * the hart resumes normal execution.
>          *
>          * For successful non-retentive suspend, the hart will resume from
> -        * specified resume address
> +        * the warm boot entry point.
>          */
> -       int (*hart_suspend)(u32 suspend_type, ulong raddr);
> +       int (*hart_suspend)(u32 suspend_type);
>
>         /**
>          * Perform platform-specific actions to resume from a suspended state.
> diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
> index a7b6a80..1165acc 100644
> --- a/lib/sbi/sbi_hsm.c
> +++ b/lib/sbi/sbi_hsm.c
> @@ -171,10 +171,10 @@ static int hsm_device_hart_stop(void)
>         return SBI_ENOTSUPP;
>  }
>
> -static int hsm_device_hart_suspend(u32 suspend_type, ulong raddr)
> +static int hsm_device_hart_suspend(u32 suspend_type)
>  {
>         if (hsm_dev && hsm_dev->hart_suspend)
> -               return hsm_dev->hart_suspend(suspend_type, raddr);
> +               return hsm_dev->hart_suspend(suspend_type);
>         return SBI_ENOTSUPP;
>  }
>
> @@ -319,7 +319,7 @@ int sbi_hsm_hart_stop(struct sbi_scratch *scratch, bool exitnow)
>         return 0;
>  }
>
> -static int __sbi_hsm_suspend_ret_default(struct sbi_scratch *scratch)
> +static int __sbi_hsm_suspend_default(struct sbi_scratch *scratch)
>  {
>         /* Wait for interrupt */
>         wfi();
> @@ -359,23 +359,6 @@ static void __sbi_hsm_suspend_non_ret_restore(struct sbi_scratch *scratch)
>         csr_write(CSR_MIP, (hdata->saved_mip & (MIP_SSIP | MIP_STIP)));
>  }
>
> -static int __sbi_hsm_suspend_non_ret_default(struct sbi_scratch *scratch,
> -                                            ulong raddr)
> -{
> -       void (*jump_warmboot)(void) = (void (*)(void))scratch->warmboot_addr;
> -
> -       /* Wait for interrupt */
> -       wfi();
> -
> -       /*
> -        * Directly jump to warm reboot to simulate resume from a
> -        * non-retentive suspend.
> -        */
> -       jump_warmboot();
> -
> -       return 0;
> -}
> -
>  void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch)
>  {
>         int oldstate;
> @@ -473,17 +456,28 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
>                 __sbi_hsm_suspend_non_ret_save(scratch);
>
>         /* Try platform specific suspend */
> -       ret = hsm_device_hart_suspend(suspend_type, scratch->warmboot_addr);
> +       ret = hsm_device_hart_suspend(suspend_type);
>         if (ret == SBI_ENOTSUPP) {
>                 /* Try generic implementation of default suspend types */
> -               if (suspend_type == SBI_HSM_SUSPEND_RET_DEFAULT) {
> -                       ret = __sbi_hsm_suspend_ret_default(scratch);
> -               } else if (suspend_type == SBI_HSM_SUSPEND_NON_RET_DEFAULT) {
> -                       ret = __sbi_hsm_suspend_non_ret_default(scratch,
> -                                               scratch->warmboot_addr);
> +               if (suspend_type == SBI_HSM_SUSPEND_RET_DEFAULT ||
> +                   suspend_type == SBI_HSM_SUSPEND_NON_RET_DEFAULT) {
> +                       ret = __sbi_hsm_suspend_default(scratch);
>                 }
>         }
>
> +       /*
> +        * The platform may have coordinated a retentive suspend, or it may
> +        * have exited early from a non-retentive suspend. Either way, the
> +        * caller is not expecting a successful return, so jump to the warm
> +        * boot entry point to simulate resume from a non-retentive suspend.
> +        */
> +       if (ret == 0 && (suspend_type & SBI_HSM_SUSP_NON_RET_BIT)) {
> +               void (*jump_warmboot)(void) =
> +                       (void (*)(void))scratch->warmboot_addr;
> +
> +               jump_warmboot();
> +       }
> +
>  fail_restore_state:
>         /*
>          * We might have successfully resumed from retentive suspend
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Jun  1 05:17:32 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 17:47:32 +0530
Subject: [RFC PATCH v2 3/9] lib: irqchip/plic: Constify plic_data pointers
In-Reply-To: <20220530033738.27127-4-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-4-samuel@sholland.org>
Message-ID: <CAAhSdy2AiQAKBUBQUrNwTobJ+jTfL00i+Qbx=AOZkQK+SMa2Lw@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> None of the functions modify the passed-in plic_data, so mark it const.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi_utils/irqchip/plic.h |  9 +++++----
>  lib/utils/irqchip/plic.c         | 11 ++++++-----
>  2 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/include/sbi_utils/irqchip/plic.h b/include/sbi_utils/irqchip/plic.h
> index 0e56d80..50be5df 100644
> --- a/include/sbi_utils/irqchip/plic.h
> +++ b/include/sbi_utils/irqchip/plic.h
> @@ -17,13 +17,14 @@ struct plic_data {
>         unsigned long num_src;
>  };
>
> -int plic_warm_irqchip_init(struct plic_data *plic,
> +int plic_warm_irqchip_init(const struct plic_data *plic,
>                            int m_cntx_id, int s_cntx_id);
>
> -int plic_cold_irqchip_init(struct plic_data *plic);
> +int plic_cold_irqchip_init(const struct plic_data *plic);
>
> -void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val);
> +void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val);
>
> -void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val);
> +void plic_set_ie(const struct plic_data *plic, u32 cntxid,
> +                u32 word_index, u32 val);
>
>  #endif
> diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
> index 0f58f3e..4c334ec 100644
> --- a/lib/utils/irqchip/plic.c
> +++ b/lib/utils/irqchip/plic.c
> @@ -21,14 +21,14 @@
>  #define PLIC_CONTEXT_BASE 0x200000
>  #define PLIC_CONTEXT_STRIDE 0x1000
>
> -static void plic_set_priority(struct plic_data *plic, u32 source, u32 val)
> +static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
>  {
>         volatile void *plic_priority = (char *)plic->addr +
>                         PLIC_PRIORITY_BASE + 4 * source;
>         writel(val, plic_priority);
>  }
>
> -void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val)
> +void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
>  {
>         volatile void *plic_thresh;
>
> @@ -40,7 +40,8 @@ void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val)
>         writel(val, plic_thresh);
>  }
>
> -void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val)
> +void plic_set_ie(const struct plic_data *plic, u32 cntxid,
> +                u32 word_index, u32 val)
>  {
>         volatile char *plic_ie;
>
> @@ -52,7 +53,7 @@ void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val)
>         writel(val, plic_ie + word_index * 4);
>  }
>
> -int plic_warm_irqchip_init(struct plic_data *plic,
> +int plic_warm_irqchip_init(const struct plic_data *plic,
>                            int m_cntx_id, int s_cntx_id)
>  {
>         size_t i, ie_words;
> @@ -85,7 +86,7 @@ int plic_warm_irqchip_init(struct plic_data *plic,
>         return 0;
>  }
>
> -int plic_cold_irqchip_init(struct plic_data *plic)
> +int plic_cold_irqchip_init(const struct plic_data *plic)
>  {
>         int i;
>
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Jun  1 05:52:16 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 18:22:16 +0530
Subject: [RFC PATCH v2 4/9] lib: irqchip/plic: Factor out a context init function
In-Reply-To: <20220530033738.27127-5-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-5-samuel@sholland.org>
Message-ID: <CAAhSdy1C48-rsR0-J49fEwx76idpJGZj4u_eEOEU3Ys+Pa9=8Q@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> This simplifies both the callers and the callees by removing duplicated
> code and consolidating the error handling. It also fixes two bugs in the
> process:
>   1) ie_words was one too large when plic->num_src was a multiple of 32.
>   2) plic_set_ie takes a 32-bit mask, not a Boolean value, so the FPGA
>      platforms previously only enabled one out of every 32 interrupts.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi_utils/irqchip/plic.h   |  8 ++---
>  lib/utils/irqchip/plic.c           | 55 ++++++++++++++++--------------
>  platform/fpga/ariane/platform.c    | 19 +++++------
>  platform/fpga/openpiton/platform.c | 19 +++++------
>  4 files changed, 48 insertions(+), 53 deletions(-)
>
> diff --git a/include/sbi_utils/irqchip/plic.h b/include/sbi_utils/irqchip/plic.h
> index 50be5df..8f21af6 100644
> --- a/include/sbi_utils/irqchip/plic.h
> +++ b/include/sbi_utils/irqchip/plic.h
> @@ -17,14 +17,12 @@ struct plic_data {
>         unsigned long num_src;
>  };
>
> +int plic_context_init(const struct plic_data *plic, int context_id,
> +                     bool enable, u32 threshold);
> +
>  int plic_warm_irqchip_init(const struct plic_data *plic,
>                            int m_cntx_id, int s_cntx_id);
>
>  int plic_cold_irqchip_init(const struct plic_data *plic);
>
> -void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val);
> -
> -void plic_set_ie(const struct plic_data *plic, u32 cntxid,
> -                u32 word_index, u32 val);
> -
>  #endif
> diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
> index 4c334ec..9bd3bf1 100644
> --- a/lib/utils/irqchip/plic.c
> +++ b/lib/utils/irqchip/plic.c
> @@ -5,6 +5,7 @@
>   *
>   * Authors:
>   *   Anup Patel <anup.patel at wdc.com>
> + *   Samuel Holland <samuel at sholland.org>
>   */
>
>  #include <sbi/riscv_io.h>
> @@ -28,61 +29,63 @@ static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
>         writel(val, plic_priority);
>  }
>
> -void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
> +static void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
>  {
>         volatile void *plic_thresh;
>
> -       if (!plic)
> -               return;
> -
>         plic_thresh = (char *)plic->addr +
>                       PLIC_CONTEXT_BASE + PLIC_CONTEXT_STRIDE * cntxid;
>         writel(val, plic_thresh);
>  }
>
> -void plic_set_ie(const struct plic_data *plic, u32 cntxid,
> -                u32 word_index, u32 val)
> +static void plic_set_ie(const struct plic_data *plic, u32 cntxid,
> +                       u32 word_index, u32 val)
>  {
>         volatile char *plic_ie;
>
> -       if (!plic)
> -               return;
> -
>         plic_ie = (char *)plic->addr +
>                    PLIC_ENABLE_BASE + PLIC_ENABLE_STRIDE * cntxid;
>         writel(val, plic_ie + word_index * 4);
>  }
>
> -int plic_warm_irqchip_init(const struct plic_data *plic,
> -                          int m_cntx_id, int s_cntx_id)
> +int plic_context_init(const struct plic_data *plic, int context_id,
> +                     bool enable, u32 threshold)
>  {
> -       size_t i, ie_words;
> +       u32 ie_words, ie_value;
>
> -       if (!plic)
> +       if (!plic || context_id < 0)
>                 return SBI_EINVAL;
>
> -       ie_words = plic->num_src / 32 + 1;
> +       ie_words = (plic->num_src + 31) / 32;
> +       ie_value = enable ? 0xffffffffU : 0U;
> +
> +       for (u32 i = 0; i < ie_words; i++)
> +               plic_set_ie(plic, context_id, i, ie_value);
> +
> +       plic_set_thresh(plic, context_id, threshold);
> +
> +       return 0;
> +}
> +
> +int plic_warm_irqchip_init(const struct plic_data *plic,
> +                          int m_cntx_id, int s_cntx_id)
> +{
> +       int ret;
>
>         /* By default, disable all IRQs for M-mode of target HART */
>         if (m_cntx_id > -1) {
> -               for (i = 0; i < ie_words; i++)
> -                       plic_set_ie(plic, m_cntx_id, i, 0);
> +               ret = plic_context_init(plic, m_cntx_id, false, 0x7);
> +               if (ret)
> +                       return ret;
>         }
>
>         /* By default, disable all IRQs for S-mode of target HART */
>         if (s_cntx_id > -1) {
> -               for (i = 0; i < ie_words; i++)
> -                       plic_set_ie(plic, s_cntx_id, i, 0);
> +               ret = plic_context_init(plic, m_cntx_id, false, 0x7);
> +               if (ret)
> +                       return ret;
>         }
>
> -       /* By default, disable M-mode threshold */
> -       if (m_cntx_id > -1)
> -               plic_set_thresh(plic, m_cntx_id, 0x7);
> -
> -       /* By default, disable S-mode threshold */
> -       if (s_cntx_id > -1)
> -               plic_set_thresh(plic, s_cntx_id, 0x7);
> -
>         return 0;
>  }
>
> diff --git a/platform/fpga/ariane/platform.c b/platform/fpga/ariane/platform.c
> index 3fdc03b..56a666b 100644
> --- a/platform/fpga/ariane/platform.c
> +++ b/platform/fpga/ariane/platform.c
> @@ -99,24 +99,21 @@ static int ariane_console_init(void)
>
>  static int plic_ariane_warm_irqchip_init(int m_cntx_id, int s_cntx_id)
>  {
> -       size_t i, ie_words = ARIANE_PLIC_NUM_SOURCES / 32 + 1;
> +       int ret;
>
>         /* By default, enable all IRQs for M-mode of target HART */
>         if (m_cntx_id > -1) {
> -               for (i = 0; i < ie_words; i++)
> -                       plic_set_ie(&plic, m_cntx_id, i, 1);
> +               ret = plic_context_init(&plic, m_cntx_id, true, 0x1);
> +               if (ret)
> +                       return ret;
>         }
> +
>         /* Enable all IRQs for S-mode of target HART */
>         if (s_cntx_id > -1) {
> -               for (i = 0; i < ie_words; i++)
> -                       plic_set_ie(&plic, s_cntx_id, i, 1);
> +               ret = plic_context_init(&plic, s_cntx_id, true, 0x0);
> +               if (ret)
> +                       return ret;
>         }
> -       /* By default, enable M-mode threshold */
> -       if (m_cntx_id > -1)
> -               plic_set_thresh(&plic, m_cntx_id, 1);
> -       /* By default, disable S-mode threshold */
> -       if (s_cntx_id > -1)
> -               plic_set_thresh(&plic, s_cntx_id, 0);
>
>         return 0;
>  }
> diff --git a/platform/fpga/openpiton/platform.c b/platform/fpga/openpiton/platform.c
> index 15e289a..7ca2123 100644
> --- a/platform/fpga/openpiton/platform.c
> +++ b/platform/fpga/openpiton/platform.c
> @@ -134,24 +134,21 @@ static int openpiton_console_init(void)
>
>  static int plic_openpiton_warm_irqchip_init(int m_cntx_id, int s_cntx_id)
>  {
> -       size_t i, ie_words = plic.num_src / 32 + 1;
> +       int ret;
>
>         /* By default, enable all IRQs for M-mode of target HART */
>         if (m_cntx_id > -1) {
> -               for (i = 0; i < ie_words; i++)
> -                       plic_set_ie(&plic, m_cntx_id, i, 1);
> +               ret = plic_context_init(&plic, m_cntx_id, true, 0x1);
> +               if (ret)
> +                       return ret;
>         }
> +
>         /* Enable all IRQs for S-mode of target HART */
>         if (s_cntx_id > -1) {
> -               for (i = 0; i < ie_words; i++)
> -                       plic_set_ie(&plic, s_cntx_id, i, 1);
> +               ret = plic_context_init(&plic, s_cntx_id, true, 0x0);
> +               if (ret)
> +                       return ret;
>         }
> -       /* By default, enable M-mode threshold */
> -       if (m_cntx_id > -1)
> -               plic_set_thresh(&plic, m_cntx_id, 1);
> -       /* By default, disable S-mode threshold */
> -       if (s_cntx_id > -1)
> -               plic_set_thresh(&plic, s_cntx_id, 0);
>
>         return 0;
>  }
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Jun  1 05:56:57 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 18:26:57 +0530
Subject: [RFC PATCH v2 5/9] lib: irqchip/plic: Add context save/restore helpers
In-Reply-To: <20220530033738.27127-6-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-6-samuel@sholland.org>
Message-ID: <CAAhSdy2qRdw-BXESa-3ZpvX=Ax2LbB6L4e1b8KnmF1J2r0-OKw@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> These can be used by platform code to save the PLIC context state, if
> it would otherwise be lost during non-retentive suspend. The platform
> is responsible for allocating all necessary storage.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi_utils/irqchip/plic.h |  6 ++++
>  lib/utils/irqchip/plic.c         | 51 ++++++++++++++++++++++++++++++--
>  2 files changed, 54 insertions(+), 3 deletions(-)
>
> diff --git a/include/sbi_utils/irqchip/plic.h b/include/sbi_utils/irqchip/plic.h
> index 8f21af6..21b2266 100644
> --- a/include/sbi_utils/irqchip/plic.h
> +++ b/include/sbi_utils/irqchip/plic.h
> @@ -17,6 +17,12 @@ struct plic_data {
>         unsigned long num_src;
>  };
>
> +void plic_context_save(const struct plic_data *plic, int context_id,
> +                      u32 *enable, u32 *threshold);
> +
> +void plic_context_restore(const struct plic_data *plic, int context_id,
> +                         const u32 *enable, u32 threshold);
> +
>  int plic_context_init(const struct plic_data *plic, int context_id,
>                       bool enable, u32 threshold);
>
> diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
> index 9bd3bf1..0c64078 100644
> --- a/lib/utils/irqchip/plic.c
> +++ b/lib/utils/irqchip/plic.c
> @@ -29,6 +29,16 @@ static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
>         writel(val, plic_priority);
>  }
>
> +static u32 plic_get_thresh(const struct plic_data *plic, u32 cntxid)
> +{
> +       volatile void *plic_thresh;
> +
> +       plic_thresh = (char *)plic->addr +
> +                     PLIC_CONTEXT_BASE + PLIC_CONTEXT_STRIDE * cntxid;
> +
> +       return readl(plic_thresh);
> +}
> +
>  static void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
>  {
>         volatile void *plic_thresh;
> @@ -38,14 +48,49 @@ static void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
>         writel(val, plic_thresh);
>  }
>
> +static u32 plic_get_ie(const struct plic_data *plic, u32 cntxid,
> +                      u32 word_index)
> +{
> +       volatile void *plic_ie;
> +
> +       plic_ie = (char *)plic->addr +
> +                  PLIC_ENABLE_BASE + PLIC_ENABLE_STRIDE * cntxid +
> +                  4 * word_index;
> +
> +       return readl(plic_ie);
> +}
> +
>  static void plic_set_ie(const struct plic_data *plic, u32 cntxid,
>                         u32 word_index, u32 val)
>  {
> -       volatile char *plic_ie;
> +       volatile void *plic_ie;
>
>         plic_ie = (char *)plic->addr +
> -                  PLIC_ENABLE_BASE + PLIC_ENABLE_STRIDE * cntxid;
> -       writel(val, plic_ie + word_index * 4);
> +                  PLIC_ENABLE_BASE + PLIC_ENABLE_STRIDE * cntxid +
> +                  4 * word_index;
> +       writel(val, plic_ie);
> +}
> +
> +void plic_context_save(const struct plic_data *plic, int context_id,
> +                      u32 *enable, u32 *threshold)
> +{
> +       u32 ie_words = (plic->num_src + 31) / 32;
> +
> +       for (u32 i = 0; i < ie_words; i++)
> +               enable[i] = plic_get_ie(plic, context_id, i);
> +
> +       *threshold = plic_get_thresh(plic, context_id);
> +}
> +
> +void plic_context_restore(const struct plic_data *plic, int context_id,
> +                         const u32 *enable, u32 threshold)
> +{
> +       u32 ie_words = (plic->num_src + 31) / 32;
> +
> +       for (u32 i = 0; i < ie_words; i++)
> +               plic_set_ie(plic, context_id, i, enable[i]);
> +
> +       plic_set_thresh(plic, context_id, threshold);
>  }
>
>  int plic_context_init(const struct plic_data *plic, int context_id,
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Jun  1 06:00:08 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 18:30:08 +0530
Subject: [RFC PATCH v2 6/9] lib: irqchip/plic: Add priority save/restore helpers
In-Reply-To: <20220530033738.27127-7-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-7-samuel@sholland.org>
Message-ID: <CAAhSdy12bfaULJPpO6d-QtoUZuEv0Rq6u2KDnk0m-5w3__r8-w@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> These can be used by platform code to save the PLIC priority state, if
> it would otherwise be lost during non-retentive suspend. The platform
> is responsible for allocating all necessary storage.
>
> As a space optimization, store the saved priority values as 8-bit
> integers, since that is large enough to hold any priority value on the
> relevant platforms.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi_utils/irqchip/plic.h |  4 ++++
>  lib/utils/irqchip/plic.c         | 20 ++++++++++++++++++++
>  2 files changed, 24 insertions(+)
>
> diff --git a/include/sbi_utils/irqchip/plic.h b/include/sbi_utils/irqchip/plic.h
> index 21b2266..566fd7a 100644
> --- a/include/sbi_utils/irqchip/plic.h
> +++ b/include/sbi_utils/irqchip/plic.h
> @@ -17,6 +17,10 @@ struct plic_data {
>         unsigned long num_src;
>  };
>
> +void plic_priority_save(const struct plic_data *plic, u8 *priority);
> +
> +void plic_priority_restore(const struct plic_data *plic, const u8 *priority);
> +
>  void plic_context_save(const struct plic_data *plic, int context_id,
>                        u32 *enable, u32 *threshold);
>
> diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
> index 0c64078..b0329b8 100644
> --- a/lib/utils/irqchip/plic.c
> +++ b/lib/utils/irqchip/plic.c
> @@ -22,6 +22,13 @@
>  #define PLIC_CONTEXT_BASE 0x200000
>  #define PLIC_CONTEXT_STRIDE 0x1000
>
> +static u32 plic_get_priority(const struct plic_data *plic, u32 source)
> +{
> +       volatile void *plic_priority = (char *)plic->addr +
> +                       PLIC_PRIORITY_BASE + 4 * source;
> +       return readl(plic_priority);
> +}
> +
>  static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
>  {
>         volatile void *plic_priority = (char *)plic->addr +
> @@ -29,6 +36,19 @@ static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
>         writel(val, plic_priority);
>  }
>
> +/* So far, priorities on all consumers of these functions fit in 8 bits. */
> +void plic_priority_save(const struct plic_data *plic, u8 *priority)
> +{
> +       for (u32 i = 0; i < plic->num_src; i++)
> +               priority[i] = plic_get_priority(plic, i);
> +}
> +
> +void plic_priority_restore(const struct plic_data *plic, const u8 *priority)
> +{
> +       for (u32 i = 0; i < plic->num_src; i++)
> +               plic_set_priority(plic, i, priority[i]);
> +}
> +
>  static u32 plic_get_thresh(const struct plic_data *plic, u32 cntxid)
>  {
>         volatile void *plic_thresh;
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Wed Jun  1 06:11:58 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 18:41:58 +0530
Subject: [RFC PATCH v2 7/9] lib: utils/irqchip: Add FDT wrappers for PLIC
 save/restore functions
In-Reply-To: <20220530033738.27127-8-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-8-samuel@sholland.org>
Message-ID: <CAAhSdy0sFLLz50NWY0nsj6TrsP1REQXOAeRggFKGcQAz+1Tv0Q@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> These functions save/restore the state of the PLIC associated with the
> current hart. Only the S-mode context is saved/restored, under the
> assumption that none of the interrupts are enabled in M-mode anyway.

Instead of saving/restoring only S-mode context, I suggest we can either:
1) Save both M-mode and S-mode context
2) Add "bool mmode" parameter to decide which context to save/restore

I am inclined towards option#2 above.

>
> Signed-off-by: Samuel Holland <samuel at sholland.org>
> ---
>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi_utils/irqchip/fdt_irqchip_plic.h | 20 ++++++++++++
>  lib/utils/irqchip/fdt_irqchip_plic.c         | 32 ++++++++++++++++++++
>  2 files changed, 52 insertions(+)
>  create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
>
> diff --git a/include/sbi_utils/irqchip/fdt_irqchip_plic.h b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> new file mode 100644
> index 0000000..66650a2
> --- /dev/null
> +++ b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> @@ -0,0 +1,20 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Samuel Holland <samuel at sholland.org>
> + */
> +
> +#ifndef __IRQCHIP_FDT_IRQCHIP_PLIC_H__
> +#define __IRQCHIP_FDT_IRQCHIP_PLIC_H__
> +
> +#include <sbi/sbi_types.h>
> +
> +void irqchip_plic_priority_save(u8 *priority);
> +
> +void irqchip_plic_priority_restore(const u8 *priority);
> +
> +void irqchip_plic_context_save(u32 *enable, u32 *threshold);
> +
> +void irqchip_plic_context_restore(const u32 *enable, u32 threshold);
> +
> +#endif
> diff --git a/lib/utils/irqchip/fdt_irqchip_plic.c b/lib/utils/irqchip/fdt_irqchip_plic.c
> index b2d2a1b..37090d3 100644
> --- a/lib/utils/irqchip/fdt_irqchip_plic.c
> +++ b/lib/utils/irqchip/fdt_irqchip_plic.c
> @@ -24,6 +24,38 @@ static struct plic_data plic[PLIC_MAX_NR];
>  static struct plic_data *plic_hartid2data[SBI_HARTMASK_MAX_BITS];
>  static int plic_hartid2context[SBI_HARTMASK_MAX_BITS][2];
>
> +void irqchip_plic_priority_save(u8 *priority)
> +{
> +       struct plic_data *plic = plic_hartid2data[current_hartid()];
> +
> +       plic_priority_save(plic, priority);
> +}
> +
> +void irqchip_plic_priority_restore(const u8 *priority)
> +{
> +       struct plic_data *plic = plic_hartid2data[current_hartid()];
> +
> +       plic_priority_restore(plic, priority);
> +}
> +
> +void irqchip_plic_context_save(u32 *enable, u32 *threshold)
> +{
> +       u32 hartid = current_hartid();
> +
> +       plic_context_save(plic_hartid2data[hartid],
> +                         plic_hartid2context[hartid][1],

Don't assume that plic_hartid2context[hartid][1] != NULL because some
of the platforms such as SiFive FU540 and FU740 have core0 as a
embedded microcontroller with only M-mode.

Better to do nothing if plic_hartid2context[hartid][1] == NULL

> +                         enable, threshold);
> +}
> +
> +void irqchip_plic_context_restore(const u32 *enable, u32 threshold)
> +{
> +       u32 hartid = current_hartid();
> +
> +       plic_context_restore(plic_hartid2data[hartid],
> +                            plic_hartid2context[hartid][1],

Same comment as above.

> +                            enable, threshold);
> +}
> +
>  static int irqchip_plic_warm_init(void)
>  {
>         u32 hartid = current_hartid();
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From anup at brainfault.org  Wed Jun  1 06:15:30 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 1 Jun 2022 18:45:30 +0530
Subject: [RFC PATCH v2 7/9] lib: utils/irqchip: Add FDT wrappers for PLIC
 save/restore functions
In-Reply-To: <20220530033738.27127-8-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-8-samuel@sholland.org>
Message-ID: <CAAhSdy240crXp4VPQhG2-en4uv+v5oJy-HDhT6nBr7NYZ9jv-w@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> These functions save/restore the state of the PLIC associated with the
> current hart. Only the S-mode context is saved/restored, under the
> assumption that none of the interrupts are enabled in M-mode anyway.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>
> ---
>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi_utils/irqchip/fdt_irqchip_plic.h | 20 ++++++++++++
>  lib/utils/irqchip/fdt_irqchip_plic.c         | 32 ++++++++++++++++++++
>  2 files changed, 52 insertions(+)
>  create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
>
> diff --git a/include/sbi_utils/irqchip/fdt_irqchip_plic.h b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> new file mode 100644
> index 0000000..66650a2
> --- /dev/null
> +++ b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> @@ -0,0 +1,20 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Samuel Holland <samuel at sholland.org>
> + */
> +
> +#ifndef __IRQCHIP_FDT_IRQCHIP_PLIC_H__
> +#define __IRQCHIP_FDT_IRQCHIP_PLIC_H__
> +
> +#include <sbi/sbi_types.h>
> +
> +void irqchip_plic_priority_save(u8 *priority);
> +
> +void irqchip_plic_priority_restore(const u8 *priority);
> +
> +void irqchip_plic_context_save(u32 *enable, u32 *threshold);
> +
> +void irqchip_plic_context_restore(const u32 *enable, u32 threshold);

Since these functions are part of the FDT based PLIC driver, I suggest
using "fdt_plic_" instead of "irqchip_plic_" prefix for function names.

Regards,
Anup

> +
> +#endif
> diff --git a/lib/utils/irqchip/fdt_irqchip_plic.c b/lib/utils/irqchip/fdt_irqchip_plic.c
> index b2d2a1b..37090d3 100644
> --- a/lib/utils/irqchip/fdt_irqchip_plic.c
> +++ b/lib/utils/irqchip/fdt_irqchip_plic.c
> @@ -24,6 +24,38 @@ static struct plic_data plic[PLIC_MAX_NR];
>  static struct plic_data *plic_hartid2data[SBI_HARTMASK_MAX_BITS];
>  static int plic_hartid2context[SBI_HARTMASK_MAX_BITS][2];
>
> +void irqchip_plic_priority_save(u8 *priority)
> +{
> +       struct plic_data *plic = plic_hartid2data[current_hartid()];
> +
> +       plic_priority_save(plic, priority);
> +}
> +
> +void irqchip_plic_priority_restore(const u8 *priority)
> +{
> +       struct plic_data *plic = plic_hartid2data[current_hartid()];
> +
> +       plic_priority_restore(plic, priority);
> +}
> +
> +void irqchip_plic_context_save(u32 *enable, u32 *threshold)
> +{
> +       u32 hartid = current_hartid();
> +
> +       plic_context_save(plic_hartid2data[hartid],
> +                         plic_hartid2context[hartid][1],
> +                         enable, threshold);
> +}
> +
> +void irqchip_plic_context_restore(const u32 *enable, u32 threshold)
> +{
> +       u32 hartid = current_hartid();
> +
> +       plic_context_restore(plic_hartid2data[hartid],
> +                            plic_hartid2context[hartid][1],
> +                            enable, threshold);
> +}
> +
>  static int irqchip_plic_warm_init(void)
>  {
>         u32 hartid = current_hartid();
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From heiko at sntech.de  Wed Jun  1 07:30:35 2022
From: heiko at sntech.de (Heiko =?ISO-8859-1?Q?St=FCbner?=)
Date: Wed, 01 Jun 2022 16:30:35 +0200
Subject: console putchar/getchar ecalls and sbi v02
Message-ID: <2594568.q0ZmV6gNhb@diego>

Hi,

I just stumbled across the issue that the console_putchar and _getchar
ecalls are only part of the v0.1 specification and the spec states that
they "are expected to be deprecated; they have no replacement."

As I was using that for their earlycon functionality for a bit and haven't
found the reasoning behind the deprecation so far, I guess I'm curious
why it is going away :-) .

The commit adding the spec-note [0] also gives no explanation for it
and Google also wasn't helpful.

Any hints?


Thanks
Heiko



[0] https://github.com/riscv-non-isa/riscv-sbi-doc/commit/705e9556d25dc070926b99f8ccbcdc5073407447




From apatel at ventanamicro.com  Wed Jun  1 07:46:17 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Wed, 1 Jun 2022 20:16:17 +0530
Subject: console putchar/getchar ecalls and sbi v02
In-Reply-To: <2594568.q0ZmV6gNhb@diego>
References: <2594568.q0ZmV6gNhb@diego>
Message-ID: <CAK9=C2XRBgtb--K8o7geFxWjvza6ztNX-oMObwVnsUYz28+0+Q@mail.gmail.com>

Hi Heiko,

On Wed, Jun 1, 2022 at 8:00 PM Heiko St?bner <heiko at sntech.de> wrote:
>
> Hi,
>
> I just stumbled across the issue that the console_putchar and _getchar
> ecalls are only part of the v0.1 specification and the spec states that
> they "are expected to be deprecated; they have no replacement."
>
> As I was using that for their earlycon functionality for a bit and haven't
> found the reasoning behind the deprecation so far, I guess I'm curious
> why it is going away :-) .
>
> The commit adding the spec-note [0] also gives no explanation for it
> and Google also wasn't helpful.
>
> Any hints?

Almost all SBI v0.1 calls have been replaced by SBI v0.2 extensions,
except console_putchar() and _getchar() calls.

The console_putchar() was certainly quite useful for early prints but
since it is going to be deprecated, we need something equivalent or
better in the SBI spec.

I do have a draft "SBI debug console" proposal which is based on
shared memory and replaces the SBI v0.1 console_putchar(). I will
send-out this proposal soon.

Regards,
Anup

>
>
> Thanks
> Heiko
>
>
>
> [0] https://github.com/riscv-non-isa/riscv-sbi-doc/commit/705e9556d25dc070926b99f8ccbcdc5073407447
>
>


From heiko at sntech.de  Wed Jun  1 07:55:33 2022
From: heiko at sntech.de (Heiko =?ISO-8859-1?Q?St=FCbner?=)
Date: Wed, 01 Jun 2022 16:55:33 +0200
Subject: console putchar/getchar ecalls and sbi v02
In-Reply-To: <CAK9=C2XRBgtb--K8o7geFxWjvza6ztNX-oMObwVnsUYz28+0+Q@mail.gmail.com>
References: <2594568.q0ZmV6gNhb@diego> <CAK9=C2XRBgtb--K8o7geFxWjvza6ztNX-oMObwVnsUYz28+0+Q@mail.gmail.com>
Message-ID: <1848033.eGJsNajkDb@diego>

Hi Anup,

Am Mittwoch, 1. Juni 2022, 16:46:17 CEST schrieb Anup Patel:
> Hi Heiko,
> 
> On Wed, Jun 1, 2022 at 8:00 PM Heiko St?bner <heiko at sntech.de> wrote:
> >
> > Hi,
> >
> > I just stumbled across the issue that the console_putchar and _getchar
> > ecalls are only part of the v0.1 specification and the spec states that
> > they "are expected to be deprecated; they have no replacement."
> >
> > As I was using that for their earlycon functionality for a bit and haven't
> > found the reasoning behind the deprecation so far, I guess I'm curious
> > why it is going away :-) .
> >
> > The commit adding the spec-note [0] also gives no explanation for it
> > and Google also wasn't helpful.
> >
> > Any hints?
> 
> Almost all SBI v0.1 calls have been replaced by SBI v0.2 extensions,
> except console_putchar() and _getchar() calls.
> 
> The console_putchar() was certainly quite useful for early prints but
> since it is going to be deprecated, we need something equivalent or
> better in the SBI spec.
> 
> I do have a draft "SBI debug console" proposal which is based on
> shared memory and replaces the SBI v0.1 console_putchar(). I will
> send-out this proposal soon.

Hmm, that sounds great for the earlycon-portion, but could you maybe
also think about a really simple putchar option like the current one?

Background is that the current putchar works in all conditions, so even
in the earliest kernel startup code I could manually put in a
sbi_console_putchar() call to track down where a hang was.

That even worked in the early startup assembly when doing the ecall
manually. 

This is extremely helpful when you are on "consumer-grade"
hardware were other debug options are somewhat limited.


Thanks
Heiko



> >
> >
> > Thanks
> > Heiko
> >
> >
> >
> > [0] https://github.com/riscv-non-isa/riscv-sbi-doc/commit/705e9556d25dc070926b99f8ccbcdc5073407447
> >
> >
> 






From apatel at ventanamicro.com  Wed Jun  1 08:21:34 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Wed, 1 Jun 2022 20:51:34 +0530
Subject: console putchar/getchar ecalls and sbi v02
In-Reply-To: <1848033.eGJsNajkDb@diego>
References: <2594568.q0ZmV6gNhb@diego> <CAK9=C2XRBgtb--K8o7geFxWjvza6ztNX-oMObwVnsUYz28+0+Q@mail.gmail.com>
 <1848033.eGJsNajkDb@diego>
Message-ID: <CAK9=C2XaYvXqkSeD8ieWYswTdgygVgXZF6PgdiUEqvG2eLAi_Q@mail.gmail.com>

On Wed, Jun 1, 2022 at 8:25 PM Heiko St?bner <heiko at sntech.de> wrote:
>
> Hi Anup,
>
> Am Mittwoch, 1. Juni 2022, 16:46:17 CEST schrieb Anup Patel:
> > Hi Heiko,
> >
> > On Wed, Jun 1, 2022 at 8:00 PM Heiko St?bner <heiko at sntech.de> wrote:
> > >
> > > Hi,
> > >
> > > I just stumbled across the issue that the console_putchar and _getchar
> > > ecalls are only part of the v0.1 specification and the spec states that
> > > they "are expected to be deprecated; they have no replacement."
> > >
> > > As I was using that for their earlycon functionality for a bit and haven't
> > > found the reasoning behind the deprecation so far, I guess I'm curious
> > > why it is going away :-) .
> > >
> > > The commit adding the spec-note [0] also gives no explanation for it
> > > and Google also wasn't helpful.
> > >
> > > Any hints?
> >
> > Almost all SBI v0.1 calls have been replaced by SBI v0.2 extensions,
> > except console_putchar() and _getchar() calls.
> >
> > The console_putchar() was certainly quite useful for early prints but
> > since it is going to be deprecated, we need something equivalent or
> > better in the SBI spec.
> >
> > I do have a draft "SBI debug console" proposal which is based on
> > shared memory and replaces the SBI v0.1 console_putchar(). I will
> > send-out this proposal soon.
>
> Hmm, that sounds great for the earlycon-portion, but could you maybe
> also think about a really simple putchar option like the current one?
>
> Background is that the current putchar works in all conditions, so even
> in the earliest kernel startup code I could manually put in a
> sbi_console_putchar() call to track down where a hang was.
>
> That even worked in the early startup assembly when doing the ecall
> manually.
>
> This is extremely helpful when you are on "consumer-grade"
> hardware were other debug options are somewhat limited.

Setting up shared memory is really simple. In fact, it is just one
additional call for assembly level debugging.

Once the shared memory is set up, users can print any number
of characters in one SBI call from assembly sources.

Regards,
Anup

>
>
> Thanks
> Heiko
>
>
>
> > >
> > >
> > > Thanks
> > > Heiko
> > >
> > >
> > >
> > > [0] https://github.com/riscv-non-isa/riscv-sbi-doc/commit/705e9556d25dc070926b99f8ccbcdc5073407447
> > >
> > >
> >
>
>
>
>


From jan.remes at codasip.com  Wed Jun  1 08:45:45 2022
From: jan.remes at codasip.com (=?UTF-8?Q?Jan_Reme=C5=A1?=)
Date: Wed, 1 Jun 2022 17:45:45 +0200
Subject: console putchar/getchar ecalls and sbi v02
In-Reply-To: <CAK9=C2XaYvXqkSeD8ieWYswTdgygVgXZF6PgdiUEqvG2eLAi_Q@mail.gmail.com>
References: <2594568.q0ZmV6gNhb@diego> <CAK9=C2XRBgtb--K8o7geFxWjvza6ztNX-oMObwVnsUYz28+0+Q@mail.gmail.com>
 <1848033.eGJsNajkDb@diego> <CAK9=C2XaYvXqkSeD8ieWYswTdgygVgXZF6PgdiUEqvG2eLAi_Q@mail.gmail.com>
Message-ID: <CALXH3=+c4hprea3M8mQGV_TzZ+yqeEbVa32DDSR58tnB-dgyCw@mail.gmail.com>

On Wed, Jun 1, 2022 at 5:22 PM Anup Patel <apatel at ventanamicro.com> wrote:
>
> On Wed, Jun 1, 2022 at 8:25 PM Heiko St?bner <heiko at sntech.de> wrote:
> >
> > Hi Anup,
> >
> > Am Mittwoch, 1. Juni 2022, 16:46:17 CEST schrieb Anup Patel:
> > > Hi Heiko,
> > >
> > > On Wed, Jun 1, 2022 at 8:00 PM Heiko St?bner <heiko at sntech.de> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I just stumbled across the issue that the console_putchar and _getchar
> > > > ecalls are only part of the v0.1 specification and the spec states that
> > > > they "are expected to be deprecated; they have no replacement."
> > > >
> > > > As I was using that for their earlycon functionality for a bit and haven't
> > > > found the reasoning behind the deprecation so far, I guess I'm curious
> > > > why it is going away :-) .
> > > >
> > > > The commit adding the spec-note [0] also gives no explanation for it
> > > > and Google also wasn't helpful.
> > > >
> > > > Any hints?
> > >
> > > Almost all SBI v0.1 calls have been replaced by SBI v0.2 extensions,
> > > except console_putchar() and _getchar() calls.
> > >
> > > The console_putchar() was certainly quite useful for early prints but
> > > since it is going to be deprecated, we need something equivalent or
> > > better in the SBI spec.
> > >
> > > I do have a draft "SBI debug console" proposal which is based on
> > > shared memory and replaces the SBI v0.1 console_putchar(). I will
> > > send-out this proposal soon.
> >
> > Hmm, that sounds great for the earlycon-portion, but could you maybe
> > also think about a really simple putchar option like the current one?
> >
> > Background is that the current putchar works in all conditions, so even
> > in the earliest kernel startup code I could manually put in a
> > sbi_console_putchar() call to track down where a hang was.
> >
> > That even worked in the early startup assembly when doing the ecall
> > manually.
> >
> > This is extremely helpful when you are on "consumer-grade"
> > hardware were other debug options are somewhat limited.
>
> Setting up shared memory is really simple. In fact, it is just one
> additional call for assembly level debugging.
>
> Once the shared memory is set up, users can print any number
> of characters in one SBI call from assembly sources.
>
> Regards,
> Anup
>
There is value in putchar() that Just Works. But I guess it could be
implemented as a convenience macro/function on top of the
shared-memory ecall implementation?

Something like:

void *SBI_DEBUG_CONSOLE_MEM; /* Assumed pointer used for setting up
the shared memory */

void sbi_utils_putchar(char c)
{
    char buffer[2];
    void *oldbuf;

    oldbuf = SBI_DEBUG_CONSOLE_MEM;
    SBI_DEBUG_CONSOLE_MEM = buffer;
    buffer[0] = c;
    buffer[1] = '\0';
    <call new ecall>

    SBI_DEBUG_CONSOLE_MEM = oldbuf;
}

Regards,
  Jan

> >
> >
> > Thanks
> > Heiko
> >
> >
> >
> > > >
> > > >
> > > > Thanks
> > > > Heiko
> > > >
> > > >
> > > >
> > > > [0] https://github.com/riscv-non-isa/riscv-sbi-doc/commit/705e9556d25dc070926b99f8ccbcdc5073407447
> > > >
> > > >
> > >
> >
> >
> >
> >
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From apatel at ventanamicro.com  Wed Jun  1 09:10:56 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Wed, 1 Jun 2022 21:40:56 +0530
Subject: console putchar/getchar ecalls and sbi v02
In-Reply-To: <CALXH3=+c4hprea3M8mQGV_TzZ+yqeEbVa32DDSR58tnB-dgyCw@mail.gmail.com>
References: <2594568.q0ZmV6gNhb@diego> <CAK9=C2XRBgtb--K8o7geFxWjvza6ztNX-oMObwVnsUYz28+0+Q@mail.gmail.com>
 <1848033.eGJsNajkDb@diego> <CAK9=C2XaYvXqkSeD8ieWYswTdgygVgXZF6PgdiUEqvG2eLAi_Q@mail.gmail.com>
 <CALXH3=+c4hprea3M8mQGV_TzZ+yqeEbVa32DDSR58tnB-dgyCw@mail.gmail.com>
Message-ID: <CAK9=C2XBmJCpoghpgfVSaF7sm1sGRt=Op1wFsM585+YaR8V=UA@mail.gmail.com>

On Wed, Jun 1, 2022 at 9:16 PM Jan Reme? <jan.remes at codasip.com> wrote:
>
> On Wed, Jun 1, 2022 at 5:22 PM Anup Patel <apatel at ventanamicro.com> wrote:
> >
> > On Wed, Jun 1, 2022 at 8:25 PM Heiko St?bner <heiko at sntech.de> wrote:
> > >
> > > Hi Anup,
> > >
> > > Am Mittwoch, 1. Juni 2022, 16:46:17 CEST schrieb Anup Patel:
> > > > Hi Heiko,
> > > >
> > > > On Wed, Jun 1, 2022 at 8:00 PM Heiko St?bner <heiko at sntech.de> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I just stumbled across the issue that the console_putchar and _getchar
> > > > > ecalls are only part of the v0.1 specification and the spec states that
> > > > > they "are expected to be deprecated; they have no replacement."
> > > > >
> > > > > As I was using that for their earlycon functionality for a bit and haven't
> > > > > found the reasoning behind the deprecation so far, I guess I'm curious
> > > > > why it is going away :-) .
> > > > >
> > > > > The commit adding the spec-note [0] also gives no explanation for it
> > > > > and Google also wasn't helpful.
> > > > >
> > > > > Any hints?
> > > >
> > > > Almost all SBI v0.1 calls have been replaced by SBI v0.2 extensions,
> > > > except console_putchar() and _getchar() calls.
> > > >
> > > > The console_putchar() was certainly quite useful for early prints but
> > > > since it is going to be deprecated, we need something equivalent or
> > > > better in the SBI spec.
> > > >
> > > > I do have a draft "SBI debug console" proposal which is based on
> > > > shared memory and replaces the SBI v0.1 console_putchar(). I will
> > > > send-out this proposal soon.
> > >
> > > Hmm, that sounds great for the earlycon-portion, but could you maybe
> > > also think about a really simple putchar option like the current one?
> > >
> > > Background is that the current putchar works in all conditions, so even
> > > in the earliest kernel startup code I could manually put in a
> > > sbi_console_putchar() call to track down where a hang was.
> > >
> > > That even worked in the early startup assembly when doing the ecall
> > > manually.
> > >
> > > This is extremely helpful when you are on "consumer-grade"
> > > hardware were other debug options are somewhat limited.
> >
> > Setting up shared memory is really simple. In fact, it is just one
> > additional call for assembly level debugging.
> >
> > Once the shared memory is set up, users can print any number
> > of characters in one SBI call from assembly sources.
> >
> > Regards,
> > Anup
> >
> There is value in putchar() that Just Works. But I guess it could be
> implemented as a convenience macro/function on top of the
> shared-memory ecall implementation?
>
> Something like:
>
> void *SBI_DEBUG_CONSOLE_MEM; /* Assumed pointer used for setting up
> the shared memory */
>
> void sbi_utils_putchar(char c)
> {
>     char buffer[2];
>     void *oldbuf;
>
>     oldbuf = SBI_DEBUG_CONSOLE_MEM;
>     SBI_DEBUG_CONSOLE_MEM = buffer;
>     buffer[0] = c;
>     buffer[1] = '\0';
>     <call new ecall>
>
>     SBI_DEBUG_CONSOLE_MEM = oldbuf;
> }

Seems like this is an interesting topic for many folks so, let
me post my draft proposal in a few minutes.

Regards,
Anup

>
> Regards,
>   Jan
>
> > >
> > >
> > > Thanks
> > > Heiko
> > >
> > >
> > >
> > > > >
> > > > >
> > > > > Thanks
> > > > > Heiko
> > > > >
> > > > >
> > > > >
> > > > > [0] https://github.com/riscv-non-isa/riscv-sbi-doc/commit/705e9556d25dc070926b99f8ccbcdc5073407447
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From apatel at ventanamicro.com  Wed Jun  1 09:17:32 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Wed, 1 Jun 2022 21:47:32 +0530
Subject: SBI Debug Console Extension Proposal (Draft v1)
Message-ID: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>

Hi All,

Below is the draft proposal for SBI Debug Console Extension.

Please review it and provide feedback.

Thanks,
Anup

Debug Console Extension (EID #0x4442434E "DBCN")
================================================

The debug console extension defines a generic mechanism for boot-time
early prints from supervisor-mode software which allows users to catch
boot-time issues in supervisor-mode software.

This extension replaces legacy console putchar (EID #0x01) extension
and it is better in following ways:
1) It follows the new calling convention defined for SBI v1.0
   (or higher).
2) It is based on a shared memory area between SBI implementation
   and supervisor-mode software so multiple characters can be
   printed using a single SBI call.

The supervisor-mode software must set the shared memory area before
printing characters on the debug console. Also, all HARTs share the
same shared memory area so only one HART needs to set it at boot-time.

Function: Set Console Area (FID #0)
-----------------------------------

struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
                                         unsigned long size)

Set the shared memory area specified by `addr_div_by_2` and `size`
parameters. The `addr_div_by_4` parameter is base address of the
shared memory area right shifted by 2 whereas `size` parameter is
the size of shared memory area in bytes.

The shared memory area should be normal cacheable memory for the
supervisor-mode software. Also, the shared memory area is global
across all HARTs so SBI implementation must ensure atomicity in
setting the shared memory area.

Errors:
SBI_SUCCESS                - Shared memory area set successfully.
SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
                          `addr_div_by_2` and `size` parameters
                          is not normal cacheable memory or not
                          accessible to supervisor-mode software.

Function: Console Puts (FID #1)
-------------------------------

struct sbiret sbi_debug_console_puts(unsigned long area_offset,
                                     unsigned long num_chars)

Print the string specified by `area_offset` and `num_chars` on
the debug console. The `area_offset` parameter is the start of
string in the shard memory area whereas `num_chars` parameter
is the number of characters (or bytes) in the string.

This is a blocking SBI call and will only return after printing
all characters of the string.

Errors:
SBI_SUCCESS                - Characters printed successfully.
SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
                          `area_offset`) or end of the string
                          (i.e. `area_offset + num_chars`) is
                          outside shared memory area.


From dylan at rivosinc.com  Wed Jun  1 11:21:26 2022
From: dylan at rivosinc.com (Dylan Reid)
Date: Wed, 1 Jun 2022 18:21:26 +0000
Subject: [sig-hypervisors] SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
Message-ID: <20220601182126.xnv5w2y4yhy2s6tk@rivosinc.com>

On Wed, Jun 01, 2022 at 09:47:32PM +0530, Anup Patel wrote:
> Hi All,
> 
> Below is the draft proposal for SBI Debug Console Extension.
> 
> Please review it and provide feedback.
> 
> Thanks,
> Anup
> 
> Debug Console Extension (EID #0x4442434E "DBCN")
> ================================================
> 
> The debug console extension defines a generic mechanism for boot-time
> early prints from supervisor-mode software which allows users to catch
> boot-time issues in supervisor-mode software.
> 
> This extension replaces legacy console putchar (EID #0x01) extension
> and it is better in following ways:

Thanks, it will be nice to drop putchar.

> 1) It follows the new calling convention defined for SBI v1.0
>    (or higher).
> 2) It is based on a shared memory area between SBI implementation
>    and supervisor-mode software so multiple characters can be
>    printed using a single SBI call.
> 
> The supervisor-mode software must set the shared memory area before
> printing characters on the debug console. Also, all HARTs share the
> same shared memory area so only one HART needs to set it at boot-time.
> 
> Function: Set Console Area (FID #0)
> -----------------------------------
> 
> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
>                                          unsigned long size)
> 
> Set the shared memory area specified by `addr_div_by_2` and `size`
> parameters. The `addr_div_by_4` parameter is base address of the
> shared memory area right shifted by 2 whereas `size` parameter is
> the size of shared memory area in bytes.
> 
> The shared memory area should be normal cacheable memory for the
> supervisor-mode software. Also, the shared memory area is global
> across all HARTs so SBI implementation must ensure atomicity in
> setting the shared memory area.
> 
> Errors:
> SBI_SUCCESS                - Shared memory area set successfully.
> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
>                           `addr_div_by_2` and `size` parameters
>                           is not normal cacheable memory or not
>                           accessible to supervisor-mode software.
> 
> Function: Console Puts (FID #1)
> -------------------------------
> 
> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
>                                      unsigned long num_chars)

What is the motivation for `area_offset`? Will the supervisor use
different offsets for different harts?

What are the advantages and disadvantages of the offset vs. using a ring
buffer for example?

> 
> Print the string specified by `area_offset` and `num_chars` on
> the debug console. The `area_offset` parameter is the start of
> string in the shard memory area whereas `num_chars` parameter
> is the number of characters (or bytes) in the string.
> 
> This is a blocking SBI call and will only return after printing
> all characters of the string.
> 
> Errors:
> SBI_SUCCESS                - Characters printed successfully.
> SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
>                           `area_offset`) or end of the string
>                           (i.e. `area_offset + num_chars`) is
>                           outside shared memory area.
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#101): https://lists.riscv.org/g/sig-hypervisors/message/101
> Mute This Topic: https://lists.riscv.org/mt/91480123/6320247
> Group Owner: sig-hypervisors+owner at lists.riscv.org
> Unsubscribe: https://lists.riscv.org/g/sig-hypervisors/leave/11085586/6320247/1694467148/xyzzy [dylan at rivosinc.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 
> 


From heinrich.schuchardt at canonical.com  Wed Jun  1 11:29:47 2022
From: heinrich.schuchardt at canonical.com (Heinrich Schuchardt)
Date: Wed, 1 Jun 2022 20:29:47 +0200
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
Message-ID: <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com>

On 6/1/22 18:17, Anup Patel wrote:
> Hi All,
> 
> Below is the draft proposal for SBI Debug Console Extension.
> 
> Please review it and provide feedback.
> 
> Thanks,
> Anup
> 
> Debug Console Extension (EID #0x4442434E "DBCN")
> ================================================
> 
> The debug console extension defines a generic mechanism for boot-time
> early prints from supervisor-mode software which allows users to catch
> boot-time issues in supervisor-mode software.
> 
> This extension replaces legacy console putchar (EID #0x01) extension
> and it is better in following ways:

Thanks for starting to close this gap.

> 1) It follows the new calling convention defined for SBI v1.0
>     (or higher).
> 2) It is based on a shared memory area between SBI implementation
>     and supervisor-mode software so multiple characters can be
>     printed using a single SBI call.

I miss a discussion of the conflicts that can arise if the configuration 
of the serial console is changed by the caller.

Do we need an ecall that closes the SBI console to further access?

> 
> The supervisor-mode software must set the shared memory area before
> printing characters on the debug console. Also, all HARTs share the
> same shared memory area so only one HART needs to set it at boot-time.

Isn't it M-mode software that has to program the MMU to allow all harts 
in M-mode and S-mode access to the memory area? What is the S-mode 
software to do about the memory area prior to calling 
sbi_debug_console_set_area()?

> 
> Function: Set Console Area (FID #0)
> -----------------------------------
> 
> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
>                                           unsigned long size)

The console output is needed on the very start of the S-mode software, 
before setting up anything.

Can we avoid this extra function?

Can we simply assume that the caller of sbi_debug_console_puts() 
provides a physical address pointer to a memory area that is suitable?

> 
> Set the shared memory area specified by `addr_div_by_2` and `size`

%s/addr_div_by_2/addr_div_by_4/

> parameters. The `addr_div_by_4` parameter is base address of the

%s/is base/is the base/

> shared memory area right shifted by 2 whereas `size` parameter is
> the size of shared memory area in bytes.

Why shifting the address? I would prefer to keep it simple for the 
caller. If the alignment is not suitable, return an error.

But why is an alignment needed here at all? And why 4 aligned?

> 
> The shared memory area should be normal cacheable memory for the
> supervisor-mode software. Also, the shared memory area is global
> across all HARTs so SBI implementation must ensure atomicity in
> setting the shared memory area.
> 
> Errors:
> SBI_SUCCESS                - Shared memory area set successfully.
> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
>                            `addr_div_by_2` and `size` parameters
>                            is not normal cacheable memory or not
>                            accessible to supervisor-mode software.
> 
> Function: Console Puts (FID #1)
> -------------------------------
> 
> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
>                                       unsigned long num_chars)

I would prefer to simply pass a physical address pointer here with no 
requirements on alignment. And no prior SBI call.

Do we need num_chars? Are we expecting to provide binary output? Using 
0x00 as terminator would be adequate in most cases.

What is the requirement on the console? Does it have to support 8bit 
output to allow for UTF-8?

Do we make any assumptions about encoding?

How would we handle a console set up to 7bit + parity if a character > 
0x7f is sent?

> 
> Print the string specified by `area_offset` and `num_chars` on
> the debug console. The `area_offset` parameter is the start of
> string in the shard memory area whereas `num_chars` parameter

%s/shard/shared/

> is the number of characters (or bytes) in the string.
> 
> This is a blocking SBI call and will only return after printing
> all characters of the string.
> 
> Errors:
> SBI_SUCCESS                - Characters printed successfully.
> SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
>                            `area_offset`) or end of the string
>                            (i.e. `area_offset + num_chars`) is
>                            outside shared memory area.

There could be other reasons of failures:

* set up of the UART failed in OpenSBI
* no UART defined in the device-tree
* ...

So let us add SBI_ERR_FAILED to the list.

Best regards

Heinrich


From heiko at sntech.de  Wed Jun  1 11:32:51 2022
From: heiko at sntech.de (Heiko =?ISO-8859-1?Q?St=FCbner?=)
Date: Wed, 01 Jun 2022 20:32:51 +0200
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
Message-ID: <5286636.29KlJPOoH8@diego>

Hi Anup,

Am Mittwoch, 1. Juni 2022, 18:17:32 CEST schrieb Anup Patel:
> Hi All,
> 
> Below is the draft proposal for SBI Debug Console Extension.
> 
> Please review it and provide feedback.
> 
> Thanks,
> Anup
> 
> Debug Console Extension (EID #0x4442434E "DBCN")
> ================================================
> 
> The debug console extension defines a generic mechanism for boot-time
> early prints from supervisor-mode software which allows users to catch
> boot-time issues in supervisor-mode software.
> 
> This extension replaces legacy console putchar (EID #0x01) extension
> and it is better in following ways:
> 1) It follows the new calling convention defined for SBI v1.0
>    (or higher).
> 2) It is based on a shared memory area between SBI implementation
>    and supervisor-mode software so multiple characters can be
>    printed using a single SBI call.
> 
> The supervisor-mode software must set the shared memory area before
> printing characters on the debug console. Also, all HARTs share the
> same shared memory area so only one HART needs to set it at boot-time.
> 
> Function: Set Console Area (FID #0)
> -----------------------------------
> 
> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
>                                          unsigned long size)
> 
> Set the shared memory area specified by `addr_div_by_2` and `size`

typo in the "div_by_2" (not 4 like below and in the function itself) ?


> parameters. The `addr_div_by_4` parameter is base address of the
> shared memory area right shifted by 2 whereas `size` parameter is
> the size of shared memory area in bytes.
> 
> The shared memory area should be normal cacheable memory for the
> supervisor-mode software. Also, the shared memory area is global
> across all HARTs so SBI implementation must ensure atomicity in
> setting the shared memory area.
> 
> Errors:
> SBI_SUCCESS                - Shared memory area set successfully.
> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
>                           `addr_div_by_2` and `size` parameters
>                           is not normal cacheable memory or not
>                           accessible to supervisor-mode software.
> 
> Function: Console Puts (FID #1)
> -------------------------------
> 
> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
>                                      unsigned long num_chars)
> 
> Print the string specified by `area_offset` and `num_chars` on
> the debug console. The `area_offset` parameter is the start of
> string in the shard memory area whereas `num_chars` parameter
> is the number of characters (or bytes) in the string.
> 
> This is a blocking SBI call and will only return after printing
> all characters of the string.
> 
> Errors:
> SBI_SUCCESS                - Characters printed successfully.
> SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
>                           `area_offset`) or end of the string
>                           (i.e. `area_offset + num_chars`) is
>                           outside shared memory area.

This will vastly reduce the number of needed ecalls when outputting
characters, so this will probably improve performance quite a bit :-)


I guess I still would like to have an _additional_ single-character
putc call. As mentioned in the other thread [0], especially on consumer
hardware [where there is no elaborate debug infrastructure] this can
be a very handy debugging tool even in the earliest stages of a
booting kernel (both before relocation and even inside the startup
assembly).

I.e. just doing a
	li a7, 1
	li a6, 0
	li a0, 36
	ecall

in any kernel assembly will just output a "$" character right now, without
needing any preparation at all - same with using the current 
sbi_console_putchar() directly in c-code.

This _can_ be very helpful in some cases, so I guess it would be nice
to keep such a functionality around also in the new spec.


Thanks
Heiko


[0] http://lists.infradead.org/pipermail/opensbi/2022-June/002796.html





From wxjstz at 126.com  Wed Jun  1 20:43:43 2022
From: wxjstz at 126.com (Xiang W)
Date: Thu, 02 Jun 2022 11:43:43 +0800
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
Message-ID: <d40d2aafa18951be1021f7385d9e17614848d5a6.camel@126.com>

? 2022-06-01???? 21:47 +0530?Anup Patel???
> Hi All,
> 
> Below is the draft proposal for SBI Debug Console Extension.
> 
> Please review it and provide feedback.
> 
> Thanks,
> Anup
The use of these two APIs is too complicated, and the supervisor-mode
software needs to create a data structure to manage the Console Area.
I recommend using the simpler interface:

struct sbiret sbi_debug_console_puts(unsigned long addr,
                                     unsigned long num_chars)

Regards,
Xiang W
> 
> Debug Console Extension (EID #0x4442434E "DBCN")
> ================================================
> 
> The debug console extension defines a generic mechanism for boot-time
> early prints from supervisor-mode software which allows users to catch
> boot-time issues in supervisor-mode software.
> 
> This extension replaces legacy console putchar (EID #0x01) extension
> and it is better in following ways:
> 1) It follows the new calling convention defined for SBI v1.0
> ?? (or higher).
> 2) It is based on a shared memory area between SBI implementation
> ?? and supervisor-mode software so multiple characters can be
> ?? printed using a single SBI call.
> 
> The supervisor-mode software must set the shared memory area before
> printing characters on the debug console. Also, all HARTs share the
> same shared memory area so only one HART needs to set it at boot-time.
> 
> Function: Set Console Area (FID #0)
> -----------------------------------
> 
> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> ???????????????????????????????????????? unsigned long size)
> 
> Set the shared memory area specified by `addr_div_by_2` and `size`
> parameters. The `addr_div_by_4` parameter is base address of the
> shared memory area right shifted by 2 whereas `size` parameter is
> the size of shared memory area in bytes.
> 
> The shared memory area should be normal cacheable memory for the
> supervisor-mode software. Also, the shared memory area is global
> across all HARTs so SBI implementation must ensure atomicity in
> setting the shared memory area.
> 
> Errors:
> SBI_SUCCESS??????????????? - Shared memory area set successfully.
> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> ????????????????????????? `addr_div_by_2` and `size` parameters
> ????????????????????????? is not normal cacheable memory or not
> ????????????????????????? accessible to supervisor-mode software.
> 
> Function: Console Puts (FID #1)
> -------------------------------
> 
> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> ???????????????????????????????????? unsigned long num_chars)
> 
> Print the string specified by `area_offset` and `num_chars` on
> the debug console. The `area_offset` parameter is the start of
> string in the shard memory area whereas `num_chars` parameter
> is the number of characters (or bytes) in the string.
> 
> This is a blocking SBI call and will only return after printing
> all characters of the string.
> 
> Errors:
> SBI_SUCCESS??????????????? - Characters printed successfully.
> SBI_ERR_INVALID_ADDRESS??? - The start of the string (i.e.
> ????????????????????????? `area_offset`) or end of the string
> ????????????????????????? (i.e. `area_offset + num_chars`) is
> ????????????????????????? outside shared memory area.
> 




From wxjstz at 126.com  Wed Jun  1 21:37:46 2022
From: wxjstz at 126.com (Xiang W)
Date: Thu, 02 Jun 2022 12:37:46 +0800
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
Message-ID: <7a73635aff9396daa96f38fa48466bc4adbc9f1c.camel@126.com>

? 2022-06-01???? 21:47 +0530?Anup Patel???
> Hi All,
> 
> Below is the draft proposal for SBI Debug Console Extension.
> 
> Please review it and provide feedback.
> 
> Thanks,
> Anup
> 
> Debug Console Extension (EID #0x4442434E "DBCN")
> ================================================
> 
> The debug console extension defines a generic mechanism for boot-time
> early prints from supervisor-mode software which allows users to catch
> boot-time issues in supervisor-mode software.
> 
> This extension replaces legacy console putchar (EID #0x01) extension
> and it is better in following ways:
> 1) It follows the new calling convention defined for SBI v1.0
> ?? (or higher).
> 2) It is based on a shared memory area between SBI implementation
> ?? and supervisor-mode software so multiple characters can be
> ?? printed using a single SBI call.
> 
> The supervisor-mode software must set the shared memory area before
> printing characters on the debug console. Also, all HARTs share the
> same shared memory area so only one HART needs to set it at boot-time.
> 
> Function: Set Console Area (FID #0)
> -----------------------------------
> 
> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> ???????????????????????????????????????? unsigned long size)
> 
> Set the shared memory area specified by `addr_div_by_2` and `size`
> parameters. The `addr_div_by_4` parameter is base address of the
> shared memory area right shifted by 2 whereas `size` parameter is
> the size of shared memory area in bytes.
> 
> The shared memory area should be normal cacheable memory for the
> supervisor-mode software. Also, the shared memory area is global
> across all HARTs so SBI implementation must ensure atomicity in
> setting the shared memory area.
> 
> Errors:
> SBI_SUCCESS??????????????? - Shared memory area set successfully.
> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> ????????????????????????? `addr_div_by_2` and `size` parameters
> ????????????????????????? is not normal cacheable memory or not
> ????????????????????????? accessible to supervisor-mode software.
> 

Shared memory can be a single extension. The three interfaces are as follows

/* The supervisor hands a piece of physical memory to sbi for shared memory */
struct sbiret sbi_shared_memory_extrend(unsigned long addr, unsigned long size);


/* The supervisor applies for a piece of physical memory from sbi */
struct sbiret sbi_shared_memory_alloc(unsigned long size);


/* The supervisor notifies sbi to release the requested memory */
struct sbiret sbi_shared_memory_free(unsigned long addr);


> Function: Console Puts (FID #1)
> -------------------------------
> 
> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> ???????????????????????????????????? unsigned long num_chars)
> 
> Print the string specified by `area_offset` and `num_chars` on
> the debug console. The `area_offset` parameter is the start of
> string in the shard memory area whereas `num_chars` parameter
> is the number of characters (or bytes) in the string.
> 
> This is a blocking SBI call and will only return after printing
> all characters of the string.
> 
> Errors:
> SBI_SUCCESS??????????????? - Characters printed successfully.
> SBI_ERR_INVALID_ADDRESS??? - The start of the string (i.e.
> ????????????????????????? `area_offset`) or end of the string
> ????????????????????????? (i.e. `area_offset + num_chars`) is
> ????????????????????????? outside shared memory area.
> 




From apatel at ventanamicro.com  Thu Jun  2 01:08:28 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 13:38:28 +0530
Subject: [RISC-V] [tech-unixplatformspec] [sig-hypervisors] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <20220601182126.xnv5w2y4yhy2s6tk@rivosinc.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <20220601182126.xnv5w2y4yhy2s6tk@rivosinc.com>
Message-ID: <CAK9=C2UpSB3GtRf9B-KU5m0Y=2=o7HvK6-q0kncF4X4=JRAdTw@mail.gmail.com>

On Wed, Jun 1, 2022 at 11:51 PM Dylan Reid <dylan at rivosinc.com> wrote:
>
> On Wed, Jun 01, 2022 at 09:47:32PM +0530, Anup Patel wrote:
> > Hi All,
> >
> > Below is the draft proposal for SBI Debug Console Extension.
> >
> > Please review it and provide feedback.
> >
> > Thanks,
> > Anup
> >
> > Debug Console Extension (EID #0x4442434E "DBCN")
> > ================================================
> >
> > The debug console extension defines a generic mechanism for boot-time
> > early prints from supervisor-mode software which allows users to catch
> > boot-time issues in supervisor-mode software.
> >
> > This extension replaces legacy console putchar (EID #0x01) extension
> > and it is better in following ways:
>
> Thanks, it will be nice to drop putchar.
>
> > 1) It follows the new calling convention defined for SBI v1.0
> >    (or higher).
> > 2) It is based on a shared memory area between SBI implementation
> >    and supervisor-mode software so multiple characters can be
> >    printed using a single SBI call.
> >
> > The supervisor-mode software must set the shared memory area before
> > printing characters on the debug console. Also, all HARTs share the
> > same shared memory area so only one HART needs to set it at boot-time.
> >
> > Function: Set Console Area (FID #0)
> > -----------------------------------
> >
> > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> >                                          unsigned long size)
> >
> > Set the shared memory area specified by `addr_div_by_2` and `size`
> > parameters. The `addr_div_by_4` parameter is base address of the
> > shared memory area right shifted by 2 whereas `size` parameter is
> > the size of shared memory area in bytes.
> >
> > The shared memory area should be normal cacheable memory for the
> > supervisor-mode software. Also, the shared memory area is global
> > across all HARTs so SBI implementation must ensure atomicity in
> > setting the shared memory area.
> >
> > Errors:
> > SBI_SUCCESS                - Shared memory area set successfully.
> > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> >                           `addr_div_by_2` and `size` parameters
> >                           is not normal cacheable memory or not
> >                           accessible to supervisor-mode software.
> >
> > Function: Console Puts (FID #1)
> > -------------------------------
> >
> > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> >                                      unsigned long num_chars)
>
> What is the motivation for `area_offset`? Will the supervisor use
> different offsets for different harts?

There are variety of ways in which supervisor software can use
tarea_offset:
1) Use lock to serialize access to shared memory and always
use fixed offset (maybe zero) from all HARTs
2) No lock to protect shared memory and instead each HART
will use separate offsets to print

In addition to above, bare-metal test code (or assembly sources)
can have pre-populated strings (i.e. "PASS", "FAIL", "ERROR", etc)
in shared memory and simply use different offsets to print different
strings.

>
> What are the advantages and disadvantages of the offset vs. using a ring
> buffer for example?

Mandating a ring on shared memory will make things complicated
for bare metal test code (or assembly sources). Also, there is no
scheduler in M-mode firmware to have worker thread for consuming
bytes from a ring.

The "area_offset" is relatively more flexible in this case because it
allows sophisticated supervisor software to create ring on shared
memory without the SBI implementation knowing about it where:
1) The head & tail will be maintained by supervisor software
2) supervisor software will have a consumer thread to consume
bytes from ring and print using puts()

>
> >
> > Print the string specified by `area_offset` and `num_chars` on
> > the debug console. The `area_offset` parameter is the start of
> > string in the shard memory area whereas `num_chars` parameter
> > is the number of characters (or bytes) in the string.
> >
> > This is a blocking SBI call and will only return after printing
> > all characters of the string.
> >
> > Errors:
> > SBI_SUCCESS                - Characters printed successfully.
> > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> >                           `area_offset`) or end of the string
> >                           (i.e. `area_offset + num_chars`) is
> >                           outside shared memory area.
> >
> >
> >
> >
> >
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#1709): https://lists.riscv.org/g/tech-unixplatformspec/message/1709
> Mute This Topic: https://lists.riscv.org/mt/91482999/1774265
> Group Owner: tech-unixplatformspec+owner at lists.riscv.org
> Unsubscribe: https://lists.riscv.org/g/tech-unixplatformspec/unsub [apatel at ventanamicro.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

Regards,
Anup


From apatel at ventanamicro.com  Thu Jun  2 01:44:36 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 14:14:36 +0530
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com>
Message-ID: <CAK9=C2WLZqkYsgZL5WCJGjWGK=YGV3f51d8yvD3ecx+JPURgpw@mail.gmail.com>

On Wed, Jun 1, 2022 at 11:59 PM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> On 6/1/22 18:17, Anup Patel wrote:
> > Hi All,
> >
> > Below is the draft proposal for SBI Debug Console Extension.
> >
> > Please review it and provide feedback.
> >
> > Thanks,
> > Anup
> >
> > Debug Console Extension (EID #0x4442434E "DBCN")
> > ================================================
> >
> > The debug console extension defines a generic mechanism for boot-time
> > early prints from supervisor-mode software which allows users to catch
> > boot-time issues in supervisor-mode software.
> >
> > This extension replaces legacy console putchar (EID #0x01) extension
> > and it is better in following ways:
>
> Thanks for starting to close this gap.
>
> > 1) It follows the new calling convention defined for SBI v1.0
> >     (or higher).
> > 2) It is based on a shared memory area between SBI implementation
> >     and supervisor-mode software so multiple characters can be
> >     printed using a single SBI call.
>
> I miss a discussion of the conflicts that can arise if the configuration
> of the serial console is changed by the caller.
>
> Do we need an ecall that closes the SBI console to further access?

Usually, the serial port related code in M-mode firmware only uses
status and data registers so for most serial ports support the M-mode
firmware will adapt to serial port configuration changes.

In fact, this is why we never had a special SBI call for serial port
reconfiguration in legacy SBI v0.1 as well.

In case of virtualization, the serial port (or console) is emulated so
the special SBI call is not useful for virtualization.

>
> >
> > The supervisor-mode software must set the shared memory area before
> > printing characters on the debug console. Also, all HARTs share the
> > same shared memory area so only one HART needs to set it at boot-time.
>
> Isn't it M-mode software that has to program the MMU to allow all harts
> in M-mode and S-mode access to the memory area? What is the S-mode
> software to do about the memory area prior to calling
> sbi_debug_console_set_area()?

Actually, it's the S-mode software which is voluntarily giving a portion of
its memory to be used as shared memory. The proposal only mandates
that whatever memory is selected by S-mode software should be a
regular cacheable memory (not IO memory). Also, if Svpbmt is available
then S-mode software should only use memory type 0 in the PTEs.

>
> >
> > Function: Set Console Area (FID #0)
> > -----------------------------------
> >
> > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> >                                           unsigned long size)
>
> The console output is needed on the very start of the S-mode software,
> before setting up anything.
>
> Can we avoid this extra function?
>
> Can we simply assume that the caller of sbi_debug_console_puts()
> provides a physical address pointer to a memory area that is suitable?

Theoretically, we can avoid the extra function to set shared memory area
but it will complicate things in future when we have supervisor software
encrypting it's own memory (using special ISA support) because in this
case supervisor software will have to unprotect memory every time the
sbi_debug_console_puts() is called and protect it again after the call.

>
> >
> > Set the shared memory area specified by `addr_div_by_2` and `size`
>
> %s/addr_div_by_2/addr_div_by_4/

Okay, I will update.

>
> > parameters. The `addr_div_by_4` parameter is base address of the
>
> %s/is base/is the base/

Okay, I will update.

>
> > shared memory area right shifted by 2 whereas `size` parameter is
> > the size of shared memory area in bytes.
>
> Why shifting the address? I would prefer to keep it simple for the
> caller. If the alignment is not suitable, return an error.
>
> But why is an alignment needed here at all? And why 4 aligned?

For RV32 S-mode, the physical address space is 34bits wide but
"unsigned long" is 32bits wide. This is because Sv32 PTEs allow
34bits of PPN. In fact, even instructions such as HFENCE.GVMA
have this "address right shift by 2" requirement based on this rationale.

>
> >
> > The shared memory area should be normal cacheable memory for the
> > supervisor-mode software. Also, the shared memory area is global
> > across all HARTs so SBI implementation must ensure atomicity in
> > setting the shared memory area.
> >
> > Errors:
> > SBI_SUCCESS                - Shared memory area set successfully.
> > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> >                            `addr_div_by_2` and `size` parameters
> >                            is not normal cacheable memory or not
> >                            accessible to supervisor-mode software.
> >
> > Function: Console Puts (FID #1)
> > -------------------------------
> >
> > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> >                                       unsigned long num_chars)
>
> I would prefer to simply pass a physical address pointer here with no
> requirements on alignment. And no prior SBI call.
>
> Do we need num_chars? Are we expecting to provide binary output? Using
> 0x00 as terminator would be adequate in most cases.

Bare-metal tests (or assembly sources) can print sub-strings from
a large per-populated string in shared memory. Assuming that string
is always terminated by 0x00 in sbi_debug_console_puts() will break
this flexibility.

>
> What is the requirement on the console? Does it have to support 8bit
> output to allow for UTF-8?

We need to clarify this. Suggestions ?

>
> Do we make any assumptions about encoding?

Same as above, this needs more clarification. Suggestions ?

I am of the opinion to keep such encoding related assumptions to be
minimal.

>
> How would we handle a console set up to 7bit + parity if a character >
> 0x7f is sent?

I would consider this to be part of the clarification we add for encoding.

>
> >
> > Print the string specified by `area_offset` and `num_chars` on
> > the debug console. The `area_offset` parameter is the start of
> > string in the shard memory area whereas `num_chars` parameter
>
> %s/shard/shared/

Okay, I will update.

>
> > is the number of characters (or bytes) in the string.
> >
> > This is a blocking SBI call and will only return after printing
> > all characters of the string.
> >
> > Errors:
> > SBI_SUCCESS                - Characters printed successfully.
> > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> >                            `area_offset`) or end of the string
> >                            (i.e. `area_offset + num_chars`) is
> >                            outside shared memory area.
>
> There could be other reasons of failures:
>
> * set up of the UART failed in OpenSBI
> * no UART defined in the device-tree
> * ...
>
> So let us add SBI_ERR_FAILED to the list.

Okay, I will add.

>
> Best regards
>
> Heinrich

Regards,
Anup


From apatel at ventanamicro.com  Thu Jun  2 01:47:58 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 14:17:58 +0530
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <5286636.29KlJPOoH8@diego>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <5286636.29KlJPOoH8@diego>
Message-ID: <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>

On Thu, Jun 2, 2022 at 12:02 AM Heiko St?bner <heiko at sntech.de> wrote:
>
> Hi Anup,
>
> Am Mittwoch, 1. Juni 2022, 18:17:32 CEST schrieb Anup Patel:
> > Hi All,
> >
> > Below is the draft proposal for SBI Debug Console Extension.
> >
> > Please review it and provide feedback.
> >
> > Thanks,
> > Anup
> >
> > Debug Console Extension (EID #0x4442434E "DBCN")
> > ================================================
> >
> > The debug console extension defines a generic mechanism for boot-time
> > early prints from supervisor-mode software which allows users to catch
> > boot-time issues in supervisor-mode software.
> >
> > This extension replaces legacy console putchar (EID #0x01) extension
> > and it is better in following ways:
> > 1) It follows the new calling convention defined for SBI v1.0
> >    (or higher).
> > 2) It is based on a shared memory area between SBI implementation
> >    and supervisor-mode software so multiple characters can be
> >    printed using a single SBI call.
> >
> > The supervisor-mode software must set the shared memory area before
> > printing characters on the debug console. Also, all HARTs share the
> > same shared memory area so only one HART needs to set it at boot-time.
> >
> > Function: Set Console Area (FID #0)
> > -----------------------------------
> >
> > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> >                                          unsigned long size)
> >
> > Set the shared memory area specified by `addr_div_by_2` and `size`
>
> typo in the "div_by_2" (not 4 like below and in the function itself) ?
>
>
> > parameters. The `addr_div_by_4` parameter is base address of the
> > shared memory area right shifted by 2 whereas `size` parameter is
> > the size of shared memory area in bytes.
> >
> > The shared memory area should be normal cacheable memory for the
> > supervisor-mode software. Also, the shared memory area is global
> > across all HARTs so SBI implementation must ensure atomicity in
> > setting the shared memory area.
> >
> > Errors:
> > SBI_SUCCESS                - Shared memory area set successfully.
> > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> >                           `addr_div_by_2` and `size` parameters
> >                           is not normal cacheable memory or not
> >                           accessible to supervisor-mode software.
> >
> > Function: Console Puts (FID #1)
> > -------------------------------
> >
> > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> >                                      unsigned long num_chars)
> >
> > Print the string specified by `area_offset` and `num_chars` on
> > the debug console. The `area_offset` parameter is the start of
> > string in the shard memory area whereas `num_chars` parameter
> > is the number of characters (or bytes) in the string.
> >
> > This is a blocking SBI call and will only return after printing
> > all characters of the string.
> >
> > Errors:
> > SBI_SUCCESS                - Characters printed successfully.
> > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> >                           `area_offset`) or end of the string
> >                           (i.e. `area_offset + num_chars`) is
> >                           outside shared memory area.
>
> This will vastly reduce the number of needed ecalls when outputting
> characters, so this will probably improve performance quite a bit :-)
>
>
> I guess I still would like to have an _additional_ single-character
> putc call. As mentioned in the other thread [0], especially on consumer
> hardware [where there is no elaborate debug infrastructure] this can
> be a very handy debugging tool even in the earliest stages of a
> booting kernel (both before relocation and even inside the startup
> assembly).
>
> I.e. just doing a
>         li a7, 1
>         li a6, 0
>         li a0, 36
>         ecall
>
> in any kernel assembly will just output a "$" character right now, without
> needing any preparation at all - same with using the current
> sbi_console_putchar() directly in c-code.
>
> This _can_ be very helpful in some cases, so I guess it would be nice
> to keep such a functionality around also in the new spec.

You can easily create multiple pre-populated strings using ".asciiz" in
assembly sources. Just set the base address of pre-populated strings
once on boot hart and print from anywhere using usual 4-5 instruction
(similar to what posted above).

>
>
> Thanks
> Heiko
>
>
> [0] http://lists.infradead.org/pipermail/opensbi/2022-June/002796.html
>
>
>

Regards,
Anup


From apatel at ventanamicro.com  Thu Jun  2 01:49:14 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 14:19:14 +0530
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <d40d2aafa18951be1021f7385d9e17614848d5a6.camel@126.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <d40d2aafa18951be1021f7385d9e17614848d5a6.camel@126.com>
Message-ID: <CAK9=C2WHgmt7JYo7ZmRJwwFW4DJSZ-+FUUSVFz6UnHt1YoWQEA@mail.gmail.com>

On Thu, Jun 2, 2022 at 9:14 AM Xiang W <wxjstz at 126.com> wrote:
>
> ? 2022-06-01???? 21:47 +0530?Anup Patel???
> > Hi All,
> >
> > Below is the draft proposal for SBI Debug Console Extension.
> >
> > Please review it and provide feedback.
> >
> > Thanks,
> > Anup
> The use of these two APIs is too complicated, and the supervisor-mode
> software needs to create a data structure to manage the Console Area.
> I recommend using the simpler interface:
>
> struct sbiret sbi_debug_console_puts(unsigned long addr,
>                                      unsigned long num_chars)

Heinrich has a similar suggestion so please see my response to
his comment.

Regards,
Anup

>
> Regards,
> Xiang W
> >
> > Debug Console Extension (EID #0x4442434E "DBCN")
> > ================================================
> >
> > The debug console extension defines a generic mechanism for boot-time
> > early prints from supervisor-mode software which allows users to catch
> > boot-time issues in supervisor-mode software.
> >
> > This extension replaces legacy console putchar (EID #0x01) extension
> > and it is better in following ways:
> > 1) It follows the new calling convention defined for SBI v1.0
> >    (or higher).
> > 2) It is based on a shared memory area between SBI implementation
> >    and supervisor-mode software so multiple characters can be
> >    printed using a single SBI call.
> >
> > The supervisor-mode software must set the shared memory area before
> > printing characters on the debug console. Also, all HARTs share the
> > same shared memory area so only one HART needs to set it at boot-time.
> >
> > Function: Set Console Area (FID #0)
> > -----------------------------------
> >
> > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> >                                          unsigned long size)
> >
> > Set the shared memory area specified by `addr_div_by_2` and `size`
> > parameters. The `addr_div_by_4` parameter is base address of the
> > shared memory area right shifted by 2 whereas `size` parameter is
> > the size of shared memory area in bytes.
> >
> > The shared memory area should be normal cacheable memory for the
> > supervisor-mode software. Also, the shared memory area is global
> > across all HARTs so SBI implementation must ensure atomicity in
> > setting the shared memory area.
> >
> > Errors:
> > SBI_SUCCESS                - Shared memory area set successfully.
> > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> >                           `addr_div_by_2` and `size` parameters
> >                           is not normal cacheable memory or not
> >                           accessible to supervisor-mode software.
> >
> > Function: Console Puts (FID #1)
> > -------------------------------
> >
> > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> >                                      unsigned long num_chars)
> >
> > Print the string specified by `area_offset` and `num_chars` on
> > the debug console. The `area_offset` parameter is the start of
> > string in the shard memory area whereas `num_chars` parameter
> > is the number of characters (or bytes) in the string.
> >
> > This is a blocking SBI call and will only return after printing
> > all characters of the string.
> >
> > Errors:
> > SBI_SUCCESS                - Characters printed successfully.
> > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> >                           `area_offset`) or end of the string
> >                           (i.e. `area_offset + num_chars`) is
> >                           outside shared memory area.
> >
>
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From heiko at sntech.de  Thu Jun  2 01:50:56 2022
From: heiko at sntech.de (Heiko =?ISO-8859-1?Q?St=FCbner?=)
Date: Thu, 02 Jun 2022 10:50:56 +0200
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com> <5286636.29KlJPOoH8@diego> <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
Message-ID: <5091667.6fTUFtlzNn@diego>

Am Donnerstag, 2. Juni 2022, 10:47:58 CEST schrieb Anup Patel:
> On Thu, Jun 2, 2022 at 12:02 AM Heiko St?bner <heiko at sntech.de> wrote:
> >
> > Hi Anup,
> >
> > Am Mittwoch, 1. Juni 2022, 18:17:32 CEST schrieb Anup Patel:
> > > Hi All,
> > >
> > > Below is the draft proposal for SBI Debug Console Extension.
> > >
> > > Please review it and provide feedback.
> > >
> > > Thanks,
> > > Anup
> > >
> > > Debug Console Extension (EID #0x4442434E "DBCN")
> > > ================================================
> > >
> > > The debug console extension defines a generic mechanism for boot-time
> > > early prints from supervisor-mode software which allows users to catch
> > > boot-time issues in supervisor-mode software.
> > >
> > > This extension replaces legacy console putchar (EID #0x01) extension
> > > and it is better in following ways:
> > > 1) It follows the new calling convention defined for SBI v1.0
> > >    (or higher).
> > > 2) It is based on a shared memory area between SBI implementation
> > >    and supervisor-mode software so multiple characters can be
> > >    printed using a single SBI call.
> > >
> > > The supervisor-mode software must set the shared memory area before
> > > printing characters on the debug console. Also, all HARTs share the
> > > same shared memory area so only one HART needs to set it at boot-time.
> > >
> > > Function: Set Console Area (FID #0)
> > > -----------------------------------
> > >
> > > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> > >                                          unsigned long size)
> > >
> > > Set the shared memory area specified by `addr_div_by_2` and `size`
> >
> > typo in the "div_by_2" (not 4 like below and in the function itself) ?
> >
> >
> > > parameters. The `addr_div_by_4` parameter is base address of the
> > > shared memory area right shifted by 2 whereas `size` parameter is
> > > the size of shared memory area in bytes.
> > >
> > > The shared memory area should be normal cacheable memory for the
> > > supervisor-mode software. Also, the shared memory area is global
> > > across all HARTs so SBI implementation must ensure atomicity in
> > > setting the shared memory area.
> > >
> > > Errors:
> > > SBI_SUCCESS                - Shared memory area set successfully.
> > > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> > >                           `addr_div_by_2` and `size` parameters
> > >                           is not normal cacheable memory or not
> > >                           accessible to supervisor-mode software.
> > >
> > > Function: Console Puts (FID #1)
> > > -------------------------------
> > >
> > > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> > >                                      unsigned long num_chars)
> > >
> > > Print the string specified by `area_offset` and `num_chars` on
> > > the debug console. The `area_offset` parameter is the start of
> > > string in the shard memory area whereas `num_chars` parameter
> > > is the number of characters (or bytes) in the string.
> > >
> > > This is a blocking SBI call and will only return after printing
> > > all characters of the string.
> > >
> > > Errors:
> > > SBI_SUCCESS                - Characters printed successfully.
> > > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> > >                           `area_offset`) or end of the string
> > >                           (i.e. `area_offset + num_chars`) is
> > >                           outside shared memory area.
> >
> > This will vastly reduce the number of needed ecalls when outputting
> > characters, so this will probably improve performance quite a bit :-)
> >
> >
> > I guess I still would like to have an _additional_ single-character
> > putc call. As mentioned in the other thread [0], especially on consumer
> > hardware [where there is no elaborate debug infrastructure] this can
> > be a very handy debugging tool even in the earliest stages of a
> > booting kernel (both before relocation and even inside the startup
> > assembly).
> >
> > I.e. just doing a
> >         li a7, 1
> >         li a6, 0
> >         li a0, 36
> >         ecall
> >
> > in any kernel assembly will just output a "$" character right now, without
> > needing any preparation at all - same with using the current
> > sbi_console_putchar() directly in c-code.
> >
> > This _can_ be very helpful in some cases, so I guess it would be nice
> > to keep such a functionality around also in the new spec.
> 
> You can easily create multiple pre-populated strings using ".asciiz" in
> assembly sources. Just set the base address of pre-populated strings
> once on boot hart and print from anywhere using usual 4-5 instruction
> (similar to what posted above).

ok, sounds like a plan as well :-)


> > [0] http://lists.infradead.org/pipermail/opensbi/2022-June/002796.html
> >
> >
> >
> 
> Regards,
> Anup
> 






From heinrich.schuchardt at canonical.com  Thu Jun  2 02:13:23 2022
From: heinrich.schuchardt at canonical.com (Heinrich Schuchardt)
Date: Thu, 2 Jun 2022 11:13:23 +0200
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <CAK9=C2WLZqkYsgZL5WCJGjWGK=YGV3f51d8yvD3ecx+JPURgpw@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com>
 <CAK9=C2WLZqkYsgZL5WCJGjWGK=YGV3f51d8yvD3ecx+JPURgpw@mail.gmail.com>
Message-ID: <307d7e4e-62f8-8dec-8b09-ea69ca2c29d9@canonical.com>

On 6/2/22 10:44, Anup Patel wrote:
> On Wed, Jun 1, 2022 at 11:59 PM Heinrich Schuchardt
> <heinrich.schuchardt at canonical.com> wrote:
>>
>> On 6/1/22 18:17, Anup Patel wrote:
>>> Hi All,
>>>
>>> Below is the draft proposal for SBI Debug Console Extension.
>>>
>>> Please review it and provide feedback.
>>>
>>> Thanks,
>>> Anup
>>>
>>> Debug Console Extension (EID #0x4442434E "DBCN")
>>> ================================================
>>>
>>> The debug console extension defines a generic mechanism for boot-time
>>> early prints from supervisor-mode software which allows users to catch
>>> boot-time issues in supervisor-mode software.
>>>
>>> This extension replaces legacy console putchar (EID #0x01) extension
>>> and it is better in following ways:
>>
>> Thanks for starting to close this gap.
>>
>>> 1) It follows the new calling convention defined for SBI v1.0
>>>      (or higher).
>>> 2) It is based on a shared memory area between SBI implementation
>>>      and supervisor-mode software so multiple characters can be
>>>      printed using a single SBI call.
>>
>> I miss a discussion of the conflicts that can arise if the configuration
>> of the serial console is changed by the caller.
>>
>> Do we need an ecall that closes the SBI console to further access?
> 
> Usually, the serial port related code in M-mode firmware only uses
> status and data registers so for most serial ports support the M-mode
> firmware will adapt to serial port configuration changes.
> 
> In fact, this is why we never had a special SBI call for serial port
> reconfiguration in legacy SBI v0.1 as well.
> 
> In case of virtualization, the serial port (or console) is emulated so
> the special SBI call is not useful for virtualization.
> 
>>
>>>
>>> The supervisor-mode software must set the shared memory area before
>>> printing characters on the debug console. Also, all HARTs share the
>>> same shared memory area so only one HART needs to set it at boot-time.
>>
>> Isn't it M-mode software that has to program the MMU to allow all harts
>> in M-mode and S-mode access to the memory area? What is the S-mode
>> software to do about the memory area prior to calling
>> sbi_debug_console_set_area()?
> 
> Actually, it's the S-mode software which is voluntarily giving a portion of
> its memory to be used as shared memory. The proposal only mandates
> that whatever memory is selected by S-mode software should be a
> regular cacheable memory (not IO memory). Also, if Svpbmt is available
> then S-mode software should only use memory type 0 in the PTEs.
> 
>>
>>>
>>> Function: Set Console Area (FID #0)
>>> -----------------------------------
>>>
>>> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
>>>                                            unsigned long size)
>>
>> The console output is needed on the very start of the S-mode software,
>> before setting up anything.
>>
>> Can we avoid this extra function?
>>
>> Can we simply assume that the caller of sbi_debug_console_puts()
>> provides a physical address pointer to a memory area that is suitable?
> 
> Theoretically, we can avoid the extra function to set shared memory area
> but it will complicate things in future when we have supervisor software
> encrypting it's own memory (using special ISA support) because in this
> case supervisor software will have to unprotect memory every time the
> sbi_debug_console_puts() is called and protect it again after the call.

Currently this function is just a nop(). It is not needed in this 
revision of the extension.

The function might be called repeatedly by different threads with 
different values. How do you want to keep track of all of these 
different areas?

Memory shared between different security realms will arise in many 
different scenarios. As this is not console specific it should be in a 
separate extension. That extension should be defined once we have 
clarity about how security realms are managed.

> 
>>
>>>
>>> Set the shared memory area specified by `addr_div_by_2` and `size`
>>
>> %s/addr_div_by_2/addr_div_by_4/
> 
> Okay, I will update.
> 
>>
>>> parameters. The `addr_div_by_4` parameter is base address of the
>>
>> %s/is base/is the base/
> 
> Okay, I will update.
> 
>>
>>> shared memory area right shifted by 2 whereas `size` parameter is
>>> the size of shared memory area in bytes.
>>
>> Why shifting the address? I would prefer to keep it simple for the
>> caller. If the alignment is not suitable, return an error.
>>
>> But why is an alignment needed here at all? And why 4 aligned?
> 
> For RV32 S-mode, the physical address space is 34bits wide but
> "unsigned long" is 32bits wide. This is because Sv32 PTEs allow
> 34bits of PPN. In fact, even instructions such as HFENCE.GVMA
> have this "address right shift by 2" requirement based on this rationale.
> 
>>
>>>
>>> The shared memory area should be normal cacheable memory for the
>>> supervisor-mode software. Also, the shared memory area is global
>>> across all HARTs so SBI implementation must ensure atomicity in
>>> setting the shared memory area.
>>>
>>> Errors:
>>> SBI_SUCCESS                - Shared memory area set successfully.
>>> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
>>>                             `addr_div_by_2` and `size` parameters
>>>                             is not normal cacheable memory or not
>>>                             accessible to supervisor-mode software.
>>>
>>> Function: Console Puts (FID #1)
>>> -------------------------------
>>>
>>> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
>>>                                        unsigned long num_chars)
>>
>> I would prefer to simply pass a physical address pointer here with no
>> requirements on alignment. And no prior SBI call.

sbi_debug_console_set_area() might be called with different values by 
different threads. An offset is ambiguous as it does not define to which 
of the different shared areas it relates. Please, use a pointer.

>>
>> Do we need num_chars? Are we expecting to provide binary output? Using
>> 0x00 as terminator would be adequate in most cases.
> 
> Bare-metal tests (or assembly sources) can print sub-strings from
> a large per-populated string in shared memory. Assuming that string
> is always terminated by 0x00 in sbi_debug_console_puts() will break
> this flexibility.

OK

> 
>>
>> What is the requirement on the console? Does it have to support 8bit
>> output to allow for UTF-8?
> 
> We need to clarify this. Suggestions ?

The platform specification would be the right place to require 8-bit 
support for the console.

> 
>>
>> Do we make any assumptions about encoding?
> 
> Same as above, this needs more clarification. Suggestions ?

We should add to the platform specification that UTF-8 output is assumed 
on the serial console.

> 
> I am of the opinion to keep such encoding related assumptions to be
> minimal.
> 
>>
>> How would we handle a console set up to 7bit + parity if a character >
>> 0x7f is sent?
> 
> I would consider this to be part of the clarification we add for encoding.

We should state if extra bits are ignored or the bytes are not send.
The easiest thing is to just ignore the extra bits. So let't state this 
here.

Best regards

Heinrich

> 
>>
>>>
>>> Print the string specified by `area_offset` and `num_chars` on
>>> the debug console. The `area_offset` parameter is the start of
>>> string in the shard memory area whereas `num_chars` parameter
>>
>> %s/shard/shared/
> 
> Okay, I will update.
> 
>>
>>> is the number of characters (or bytes) in the string.
>>>
>>> This is a blocking SBI call and will only return after printing
>>> all characters of the string.
>>>
>>> Errors:
>>> SBI_SUCCESS                - Characters printed successfully.
>>> SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
>>>                             `area_offset`) or end of the string
>>>                             (i.e. `area_offset + num_chars`) is
>>>                             outside shared memory area.
>>
>> There could be other reasons of failures:
>>
>> * set up of the UART failed in OpenSBI
>> * no UART defined in the device-tree
>> * ...
>>
>> So let us add SBI_ERR_FAILED to the list.
> 
> Okay, I will add.
> 
>>
>> Best regards
>>
>> Heinrich
> 
> Regards,
> Anup



From heiko at sntech.de  Thu Jun  2 02:28:28 2022
From: heiko at sntech.de (Heiko =?ISO-8859-1?Q?St=FCbner?=)
Date: Thu, 02 Jun 2022 11:28:28 +0200
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <5091667.6fTUFtlzNn@diego>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com> <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com> <5091667.6fTUFtlzNn@diego>
Message-ID: <3070891.fEcJ0Lxnt5@diego>

Am Donnerstag, 2. Juni 2022, 10:50:56 CEST schrieb Heiko St?bner:
> Am Donnerstag, 2. Juni 2022, 10:47:58 CEST schrieb Anup Patel:
> > On Thu, Jun 2, 2022 at 12:02 AM Heiko St?bner <heiko at sntech.de> wrote:
> > >
> > > Hi Anup,
> > >
> > > Am Mittwoch, 1. Juni 2022, 18:17:32 CEST schrieb Anup Patel:
> > > > Hi All,
> > > >
> > > > Below is the draft proposal for SBI Debug Console Extension.
> > > >
> > > > Please review it and provide feedback.
> > > >
> > > > Thanks,
> > > > Anup
> > > >
> > > > Debug Console Extension (EID #0x4442434E "DBCN")
> > > > ================================================
> > > >
> > > > The debug console extension defines a generic mechanism for boot-time
> > > > early prints from supervisor-mode software which allows users to catch
> > > > boot-time issues in supervisor-mode software.
> > > >
> > > > This extension replaces legacy console putchar (EID #0x01) extension
> > > > and it is better in following ways:
> > > > 1) It follows the new calling convention defined for SBI v1.0
> > > >    (or higher).
> > > > 2) It is based on a shared memory area between SBI implementation
> > > >    and supervisor-mode software so multiple characters can be
> > > >    printed using a single SBI call.
> > > >
> > > > The supervisor-mode software must set the shared memory area before
> > > > printing characters on the debug console. Also, all HARTs share the
> > > > same shared memory area so only one HART needs to set it at boot-time.
> > > >
> > > > Function: Set Console Area (FID #0)
> > > > -----------------------------------
> > > >
> > > > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> > > >                                          unsigned long size)
> > > >
> > > > Set the shared memory area specified by `addr_div_by_2` and `size`
> > >
> > > typo in the "div_by_2" (not 4 like below and in the function itself) ?
> > >
> > >
> > > > parameters. The `addr_div_by_4` parameter is base address of the
> > > > shared memory area right shifted by 2 whereas `size` parameter is
> > > > the size of shared memory area in bytes.
> > > >
> > > > The shared memory area should be normal cacheable memory for the
> > > > supervisor-mode software. Also, the shared memory area is global
> > > > across all HARTs so SBI implementation must ensure atomicity in
> > > > setting the shared memory area.
> > > >
> > > > Errors:
> > > > SBI_SUCCESS                - Shared memory area set successfully.
> > > > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> > > >                           `addr_div_by_2` and `size` parameters
> > > >                           is not normal cacheable memory or not
> > > >                           accessible to supervisor-mode software.
> > > >
> > > > Function: Console Puts (FID #1)
> > > > -------------------------------
> > > >
> > > > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> > > >                                      unsigned long num_chars)
> > > >
> > > > Print the string specified by `area_offset` and `num_chars` on
> > > > the debug console. The `area_offset` parameter is the start of
> > > > string in the shard memory area whereas `num_chars` parameter
> > > > is the number of characters (or bytes) in the string.
> > > >
> > > > This is a blocking SBI call and will only return after printing
> > > > all characters of the string.
> > > >
> > > > Errors:
> > > > SBI_SUCCESS                - Characters printed successfully.
> > > > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> > > >                           `area_offset`) or end of the string
> > > >                           (i.e. `area_offset + num_chars`) is
> > > >                           outside shared memory area.
> > >
> > > This will vastly reduce the number of needed ecalls when outputting
> > > characters, so this will probably improve performance quite a bit :-)
> > >
> > >
> > > I guess I still would like to have an _additional_ single-character
> > > putc call. As mentioned in the other thread [0], especially on consumer
> > > hardware [where there is no elaborate debug infrastructure] this can
> > > be a very handy debugging tool even in the earliest stages of a
> > > booting kernel (both before relocation and even inside the startup
> > > assembly).
> > >
> > > I.e. just doing a
> > >         li a7, 1
> > >         li a6, 0
> > >         li a0, 36
> > >         ecall
> > >
> > > in any kernel assembly will just output a "$" character right now, without
> > > needing any preparation at all - same with using the current
> > > sbi_console_putchar() directly in c-code.
> > >
> > > This _can_ be very helpful in some cases, so I guess it would be nice
> > > to keep such a functionality around also in the new spec.
> > 
> > You can easily create multiple pre-populated strings using ".asciiz" in
> > assembly sources. Just set the base address of pre-populated strings
> > once on boot hart and print from anywhere using usual 4-5 instruction
> > (similar to what posted above).
> 
> ok, sounds like a plan as well :-)

though, how does that relate to the time before MMU setup?

I.e. in response to Heinrich's mail you talk about svpbmt, so I guess you
expect virtual memory there, so what is the expected value-type before
the mmu is setup in S-mode ?


> > > [0] http://lists.infradead.org/pipermail/opensbi/2022-June/002796.html
> > >
> > >
> > >
> > 
> > Regards,
> > Anup
> > 
> 
> 






From konrad.schwarz at siemens.com  Thu Jun  2 05:00:37 2022
From: konrad.schwarz at siemens.com (Schwarz, Konrad)
Date: Thu, 2 Jun 2022 12:00:37 +0000
Subject: [sig-hypervisors] SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
Message-ID: <dc89b97c48364769b1bf7bbc51dd440b@siemens.com>

Hi Anup,

> From: sig-hypervisors at lists.riscv.org <sig-hypervisors at lists.riscv.org> On Behalf Of Anup Patel via
> lists.riscv.org
> Subject: [sig-hypervisors] SBI Debug Console Extension Proposal (Draft v1)
> 
> Below is the draft proposal for SBI Debug Console Extension.

Here are my thoughts:

* Guest memory access: I think this would be the first SBI extension to require access to
  guest memory. This needs to be considered carefully, but I think the higher bandwidth afforded by
  the interface is useful enough to allow this.

* API:
	* Currently, only a write interface is provided. It would be much better to have a
	  read/write interface.

        Benefits of this would be to allow a hypervisor to control an OS, e.g., for testing purposes
        or to automate installation tasks.  Inter-guest communication could also be realized
        via such an interface.

	* As the relationship between SBI and OS is the same as OS and user process, an interface
	  in the style of e.g., Unix IO is possible.

        * Global shared memory buffer design and alternatives:

        The API should have per read() or write() parameters for buffer address and length.
	  This makes it easy for different parts of the OS kernel to output strings directly,
	  without requiring thread synchronization of the shared buffer used by the present proposal.
	  A single buffer (i.e., shared memory) will in most SW architectures require an
	  extra copying of the formatted output string into the shared memory region, which
	  would be avoided using per read()/write() call parameters.
        The exact same argument applies to the SBI implementation: a multi-hart machine
	  utilizing a single shared memory block to communicate with S-mode software
        will require m-mode thread synchronization when accessing the block.
        Having only a single shared memory block will lead to scaling problems
        on high hart count machines.

        I see no advantages of the proposed design to dedicate a block of memory to
	  I/O in advance (and no modern API does this).
        The SBI implementation will still need to be prepared to handle
        access faults on each read and write call, there is no amortization of one-time costs
	  regarding address translation or permission checking that I see.  In the H-extension,
        the guest memory access instructions are as effective as possible, for M-mode code,
        I'm sure that efficient access to S-mode virtual addresses is possible as well.
        (Or the convention can be made that addresses are specified as guest-physical,
        which should be avoided if efficient alternatives are available in RISC-V.
        Note that the draft proposal does not speak of this at all).

	  * Multiple device support: a parameter should be allocated to allow the OS to
	  select which of possibly several output devices to utilize, like the "file
	  descriptor" parameter of the POSIX read()/write() interface.

	  This raises the question of binding these file descriptors to physical devices,
        but a start could be made in analogy to Unix (again), where devices 0, 1, 2 are
        assigned to standard in, standard out, and standard error, and in many cases
        will be attached to the same physical device, and that are pre-opened when the
	  OS starts. As in Unix, the semantics of these would be roughly defined,
        but an OS could write out its boot logging strings to 2, and drive an interactive
        console process (such as a shell) from 0 and 1.

	  The question of binding string-valued names to file descriptor (open()) and
	  closing them, etc., could be deferred to a later date and possibly
        be implementation defined.

        * Asynchronicity and flow control: Unix solves this (poorly) via select()
        or the SIGIO signal.  If SBI introduces an interface for this, this will need to
        co-exist with the OS scheduling that is largely driven by the interrupt controller,
	  which is a hard thing to do.

        I think that the SBI interface should
	  be best effort, copying all data from the client immediately (as Unix does):
        when buffer space in the SBI implementation is exhausted,
        it should return short counts; if the SBI client is prepared to wait, a blocking
	  flush call could be added, which returns when some measure space is
        available in the SBI implementation's output buffers (or when input is available).
        
        The existing Unix interfaces for this can again be used as a guide; they
        should be mature enough to cover all relevant cases.

        For highest performance, an S-mode interrupt could be synthesized by the
	  SBI implementation when buffer space is available.

With best regards,
Konrad Schwarz

Siemens AG
T CED SES-DE
Otto-Hahn-Ring 6
81739 Munich, Germany
Phone: +49 (89) 7805-22579
Fax.: +49 (89) 636-33045
Mobile: +49 (1522) 8864636
mailto:konrad.schwarz at siemens.com
www.siemens.com

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim Hagemann Snabe; Managing Board: Roland Busch, Chairman, President and Chief Executive Officer; Cedrik Neike, Matthias Rebellius, Ralf P. Thomas, Judith Wiese; Registered offices: Berlin and Munich, Germany; Commercial registries: Berlin-Charlottenburg, HRB 12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322


From apatel at ventanamicro.com  Thu Jun  2 05:32:54 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 18:02:54 +0530
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <7a73635aff9396daa96f38fa48466bc4adbc9f1c.camel@126.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <7a73635aff9396daa96f38fa48466bc4adbc9f1c.camel@126.com>
Message-ID: <CAK9=C2XtKOyrjRFm7kmhamK=0Vv+NDQwr09kEUuzXs==zEaGMQ@mail.gmail.com>

On Thu, Jun 2, 2022 at 10:08 AM Xiang W <wxjstz at 126.com> wrote:
>
> ? 2022-06-01???? 21:47 +0530?Anup Patel???
> > Hi All,
> >
> > Below is the draft proposal for SBI Debug Console Extension.
> >
> > Please review it and provide feedback.
> >
> > Thanks,
> > Anup
> >
> > Debug Console Extension (EID #0x4442434E "DBCN")
> > ================================================
> >
> > The debug console extension defines a generic mechanism for boot-time
> > early prints from supervisor-mode software which allows users to catch
> > boot-time issues in supervisor-mode software.
> >
> > This extension replaces legacy console putchar (EID #0x01) extension
> > and it is better in following ways:
> > 1) It follows the new calling convention defined for SBI v1.0
> >    (or higher).
> > 2) It is based on a shared memory area between SBI implementation
> >    and supervisor-mode software so multiple characters can be
> >    printed using a single SBI call.
> >
> > The supervisor-mode software must set the shared memory area before
> > printing characters on the debug console. Also, all HARTs share the
> > same shared memory area so only one HART needs to set it at boot-time.
> >
> > Function: Set Console Area (FID #0)
> > -----------------------------------
> >
> > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> >                                          unsigned long size)
> >
> > Set the shared memory area specified by `addr_div_by_2` and `size`
> > parameters. The `addr_div_by_4` parameter is base address of the
> > shared memory area right shifted by 2 whereas `size` parameter is
> > the size of shared memory area in bytes.
> >
> > The shared memory area should be normal cacheable memory for the
> > supervisor-mode software. Also, the shared memory area is global
> > across all HARTs so SBI implementation must ensure atomicity in
> > setting the shared memory area.
> >
> > Errors:
> > SBI_SUCCESS                - Shared memory area set successfully.
> > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> >                           `addr_div_by_2` and `size` parameters
> >                           is not normal cacheable memory or not
> >                           accessible to supervisor-mode software.
> >
>
> Shared memory can be a single extension. The three interfaces are as follows
>
> /* The supervisor hands a piece of physical memory to sbi for shared memory */
> struct sbiret sbi_shared_memory_extrend(unsigned long addr, unsigned long size);
>
>
> /* The supervisor applies for a piece of physical memory from sbi */
> struct sbiret sbi_shared_memory_alloc(unsigned long size);
>
>
> /* The supervisor notifies sbi to release the requested memory */
> struct sbiret sbi_shared_memory_free(unsigned long addr);

Clearly, if we have a separate SBI extension to manage shared memory
then we will end-up with such memory management calls. Memory allocators
are generally hard to get it right and this also adds lot of bookkeeping and
state management in SBI implementations (e.g. OpenSBI, KVM RISC-V,
and various hypervisors).

Further, some of the SBI extensions (such as Steal Time Accounting) will
have separate shared memory for each HART whereas some (such as
Debug Console) will have global shared memory for all HARTs.

For clean and modular SBI implementations, I would recommend that
each SBI extension define its own shared memory setup API.

Regards,
Anup

>
>
> > Function: Console Puts (FID #1)
> > -------------------------------
> >
> > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> >                                      unsigned long num_chars)
> >
> > Print the string specified by `area_offset` and `num_chars` on
> > the debug console. The `area_offset` parameter is the start of
> > string in the shard memory area whereas `num_chars` parameter
> > is the number of characters (or bytes) in the string.
> >
> > This is a blocking SBI call and will only return after printing
> > all characters of the string.
> >
> > Errors:
> > SBI_SUCCESS                - Characters printed successfully.
> > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> >                           `area_offset`) or end of the string
> >                           (i.e. `area_offset + num_chars`) is
> >                           outside shared memory area.
> >
>
>


From apatel at ventanamicro.com  Thu Jun  2 05:43:17 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 18:13:17 +0530
Subject: SBI Debug Console Extension Proposal (Draft v1)
In-Reply-To: <3070891.fEcJ0Lxnt5@diego>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
 <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego>
Message-ID: <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>

On Thu, Jun 2, 2022 at 2:58 PM Heiko St?bner <heiko at sntech.de> wrote:
>
> Am Donnerstag, 2. Juni 2022, 10:50:56 CEST schrieb Heiko St?bner:
> > Am Donnerstag, 2. Juni 2022, 10:47:58 CEST schrieb Anup Patel:
> > > On Thu, Jun 2, 2022 at 12:02 AM Heiko St?bner <heiko at sntech.de> wrote:
> > > >
> > > > Hi Anup,
> > > >
> > > > Am Mittwoch, 1. Juni 2022, 18:17:32 CEST schrieb Anup Patel:
> > > > > Hi All,
> > > > >
> > > > > Below is the draft proposal for SBI Debug Console Extension.
> > > > >
> > > > > Please review it and provide feedback.
> > > > >
> > > > > Thanks,
> > > > > Anup
> > > > >
> > > > > Debug Console Extension (EID #0x4442434E "DBCN")
> > > > > ================================================
> > > > >
> > > > > The debug console extension defines a generic mechanism for boot-time
> > > > > early prints from supervisor-mode software which allows users to catch
> > > > > boot-time issues in supervisor-mode software.
> > > > >
> > > > > This extension replaces legacy console putchar (EID #0x01) extension
> > > > > and it is better in following ways:
> > > > > 1) It follows the new calling convention defined for SBI v1.0
> > > > >    (or higher).
> > > > > 2) It is based on a shared memory area between SBI implementation
> > > > >    and supervisor-mode software so multiple characters can be
> > > > >    printed using a single SBI call.
> > > > >
> > > > > The supervisor-mode software must set the shared memory area before
> > > > > printing characters on the debug console. Also, all HARTs share the
> > > > > same shared memory area so only one HART needs to set it at boot-time.
> > > > >
> > > > > Function: Set Console Area (FID #0)
> > > > > -----------------------------------
> > > > >
> > > > > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> > > > >                                          unsigned long size)
> > > > >
> > > > > Set the shared memory area specified by `addr_div_by_2` and `size`
> > > >
> > > > typo in the "div_by_2" (not 4 like below and in the function itself) ?
> > > >
> > > >
> > > > > parameters. The `addr_div_by_4` parameter is base address of the
> > > > > shared memory area right shifted by 2 whereas `size` parameter is
> > > > > the size of shared memory area in bytes.
> > > > >
> > > > > The shared memory area should be normal cacheable memory for the
> > > > > supervisor-mode software. Also, the shared memory area is global
> > > > > across all HARTs so SBI implementation must ensure atomicity in
> > > > > setting the shared memory area.
> > > > >
> > > > > Errors:
> > > > > SBI_SUCCESS                - Shared memory area set successfully.
> > > > > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> > > > >                           `addr_div_by_2` and `size` parameters
> > > > >                           is not normal cacheable memory or not
> > > > >                           accessible to supervisor-mode software.
> > > > >
> > > > > Function: Console Puts (FID #1)
> > > > > -------------------------------
> > > > >
> > > > > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> > > > >                                      unsigned long num_chars)
> > > > >
> > > > > Print the string specified by `area_offset` and `num_chars` on
> > > > > the debug console. The `area_offset` parameter is the start of
> > > > > string in the shard memory area whereas `num_chars` parameter
> > > > > is the number of characters (or bytes) in the string.
> > > > >
> > > > > This is a blocking SBI call and will only return after printing
> > > > > all characters of the string.
> > > > >
> > > > > Errors:
> > > > > SBI_SUCCESS                - Characters printed successfully.
> > > > > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> > > > >                           `area_offset`) or end of the string
> > > > >                           (i.e. `area_offset + num_chars`) is
> > > > >                           outside shared memory area.
> > > >
> > > > This will vastly reduce the number of needed ecalls when outputting
> > > > characters, so this will probably improve performance quite a bit :-)
> > > >
> > > >
> > > > I guess I still would like to have an _additional_ single-character
> > > > putc call. As mentioned in the other thread [0], especially on consumer
> > > > hardware [where there is no elaborate debug infrastructure] this can
> > > > be a very handy debugging tool even in the earliest stages of a
> > > > booting kernel (both before relocation and even inside the startup
> > > > assembly).
> > > >
> > > > I.e. just doing a
> > > >         li a7, 1
> > > >         li a6, 0
> > > >         li a0, 36
> > > >         ecall
> > > >
> > > > in any kernel assembly will just output a "$" character right now, without
> > > > needing any preparation at all - same with using the current
> > > > sbi_console_putchar() directly in c-code.
> > > >
> > > > This _can_ be very helpful in some cases, so I guess it would be nice
> > > > to keep such a functionality around also in the new spec.
> > >
> > > You can easily create multiple pre-populated strings using ".asciiz" in
> > > assembly sources. Just set the base address of pre-populated strings
> > > once on boot hart and print from anywhere using usual 4-5 instruction
> > > (similar to what posted above).
> >
> > ok, sounds like a plan as well :-)
>
> though, how does that relate to the time before MMU setup?
>
> I.e. in response to Heinrich's mail you talk about svpbmt, so I guess you
> expect virtual memory there, so what is the expected value-type before
> the mmu is setup in S-mode ?

The memory type should be 0 (i.e. PMA) for the shared memory between
SBI implementation and supervisor software. Before MMU setup, the
memory type is by default 0 (i.e. PMA) so we don't have to mandate
any memory type for MMU disabled case.

We only have issue on systems with Svpmbt where supervisor software
can potentially map the shared memory as non-cacheable or IO (memory
type != 0) using PTE memory type bits.

In addition to above, a SBI implementation must ensure that the shared
memory address provided by supervisor software is a regular memory
(not MMIO device). This can be easily achieved in OpenSBI, KVM RISC-V,
and various hypervisors.

Regards,
Anup

>
>
> > > > [0] http://lists.infradead.org/pipermail/opensbi/2022-June/002796.html
> > > >
> > > >
> > > >
> > >
> > > Regards,
> > > Anup
> > >
> >
> >
>
>
>
>


From apatel at ventanamicro.com  Thu Jun  2 05:59:44 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 18:29:44 +0530
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <307d7e4e-62f8-8dec-8b09-ea69ca2c29d9@canonical.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com> <CAK9=C2WLZqkYsgZL5WCJGjWGK=YGV3f51d8yvD3ecx+JPURgpw@mail.gmail.com>
 <307d7e4e-62f8-8dec-8b09-ea69ca2c29d9@canonical.com>
Message-ID: <CAK9=C2VAbPhscKAg+nP25-YrX0UadD6VBkNhqeic-LPQkvma1g@mail.gmail.com>

On Thu, Jun 2, 2022 at 2:43 PM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> On 6/2/22 10:44, Anup Patel wrote:
> > On Wed, Jun 1, 2022 at 11:59 PM Heinrich Schuchardt
> > <heinrich.schuchardt at canonical.com> wrote:
> >>
> >> On 6/1/22 18:17, Anup Patel wrote:
> >>> Hi All,
> >>>
> >>> Below is the draft proposal for SBI Debug Console Extension.
> >>>
> >>> Please review it and provide feedback.
> >>>
> >>> Thanks,
> >>> Anup
> >>>
> >>> Debug Console Extension (EID #0x4442434E "DBCN")
> >>> ================================================
> >>>
> >>> The debug console extension defines a generic mechanism for boot-time
> >>> early prints from supervisor-mode software which allows users to catch
> >>> boot-time issues in supervisor-mode software.
> >>>
> >>> This extension replaces legacy console putchar (EID #0x01) extension
> >>> and it is better in following ways:
> >>
> >> Thanks for starting to close this gap.
> >>
> >>> 1) It follows the new calling convention defined for SBI v1.0
> >>>      (or higher).
> >>> 2) It is based on a shared memory area between SBI implementation
> >>>      and supervisor-mode software so multiple characters can be
> >>>      printed using a single SBI call.
> >>
> >> I miss a discussion of the conflicts that can arise if the configuration
> >> of the serial console is changed by the caller.
> >>
> >> Do we need an ecall that closes the SBI console to further access?
> >
> > Usually, the serial port related code in M-mode firmware only uses
> > status and data registers so for most serial ports support the M-mode
> > firmware will adapt to serial port configuration changes.
> >
> > In fact, this is why we never had a special SBI call for serial port
> > reconfiguration in legacy SBI v0.1 as well.
> >
> > In case of virtualization, the serial port (or console) is emulated so
> > the special SBI call is not useful for virtualization.
> >
> >>
> >>>
> >>> The supervisor-mode software must set the shared memory area before
> >>> printing characters on the debug console. Also, all HARTs share the
> >>> same shared memory area so only one HART needs to set it at boot-time.
> >>
> >> Isn't it M-mode software that has to program the MMU to allow all harts
> >> in M-mode and S-mode access to the memory area? What is the S-mode
> >> software to do about the memory area prior to calling
> >> sbi_debug_console_set_area()?
> >
> > Actually, it's the S-mode software which is voluntarily giving a portion of
> > its memory to be used as shared memory. The proposal only mandates
> > that whatever memory is selected by S-mode software should be a
> > regular cacheable memory (not IO memory). Also, if Svpbmt is available
> > then S-mode software should only use memory type 0 in the PTEs.
> >
> >>
> >>>
> >>> Function: Set Console Area (FID #0)
> >>> -----------------------------------
> >>>
> >>> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> >>>                                            unsigned long size)
> >>
> >> The console output is needed on the very start of the S-mode software,
> >> before setting up anything.
> >>
> >> Can we avoid this extra function?
> >>
> >> Can we simply assume that the caller of sbi_debug_console_puts()
> >> provides a physical address pointer to a memory area that is suitable?
> >
> > Theoretically, we can avoid the extra function to set shared memory area
> > but it will complicate things in future when we have supervisor software
> > encrypting it's own memory (using special ISA support) because in this
> > case supervisor software will have to unprotect memory every time the
> > sbi_debug_console_puts() is called and protect it again after the call.
>
> Currently this function is just a nop(). It is not needed in this
> revision of the extension.
>
> The function might be called repeatedly by different threads with
> different values. How do you want to keep track of all of these
> different areas?

The shared memory area in case of this SBI extension will be shared
across all HARTs so the SBI implementation will ensure atomicity
in setting/reading shared memory coordinates. This way multiple
HARTs can call the sbi_debug_console_set_area() but only the
last call will be in effect.

Following text in the proposal explains above:
"The supervisor-mode software must set the shared memory area before
printing characters on the debug console. Also, all HARTs share the
same shared memory area so only one HART needs to set it at boot-time."

Maybe this text is not giving a clear picture ?

>
> Memory shared between different security realms will arise in many
> different scenarios. As this is not console specific it should be in a
> separate extension. That extension should be defined once we have
> clarity about how security realms are managed.

In the case of this extension, the shared memory is global across all
HARTs and it mostly contains bytes to be printed. In case of steal
time accounting, the shared memory is separate for each HART and
it is a well-defined data structure.

Clearly, the usage of shared memory is extension specific.

The VirtIO devices are a good (and time tested) example of shared
memory based approaches. Over there as well, the shared memory
(i.e. various VirtIO rings) are setup by Guest and there is no central
pool of shared memory (i.e no dedicated VirtIO device managing
shared memory).

Similar to VirtIO world, we should let SBI extension define its own
shared memory usage and API.

Regards,
Anup

>
> >
> >>
> >>>
> >>> Set the shared memory area specified by `addr_div_by_2` and `size`
> >>
> >> %s/addr_div_by_2/addr_div_by_4/
> >
> > Okay, I will update.
> >
> >>
> >>> parameters. The `addr_div_by_4` parameter is base address of the
> >>
> >> %s/is base/is the base/
> >
> > Okay, I will update.
> >
> >>
> >>> shared memory area right shifted by 2 whereas `size` parameter is
> >>> the size of shared memory area in bytes.
> >>
> >> Why shifting the address? I would prefer to keep it simple for the
> >> caller. If the alignment is not suitable, return an error.
> >>
> >> But why is an alignment needed here at all? And why 4 aligned?
> >
> > For RV32 S-mode, the physical address space is 34bits wide but
> > "unsigned long" is 32bits wide. This is because Sv32 PTEs allow
> > 34bits of PPN. In fact, even instructions such as HFENCE.GVMA
> > have this "address right shift by 2" requirement based on this rationale.
> >
> >>
> >>>
> >>> The shared memory area should be normal cacheable memory for the
> >>> supervisor-mode software. Also, the shared memory area is global
> >>> across all HARTs so SBI implementation must ensure atomicity in
> >>> setting the shared memory area.
> >>>
> >>> Errors:
> >>> SBI_SUCCESS                - Shared memory area set successfully.
> >>> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> >>>                             `addr_div_by_2` and `size` parameters
> >>>                             is not normal cacheable memory or not
> >>>                             accessible to supervisor-mode software.
> >>>
> >>> Function: Console Puts (FID #1)
> >>> -------------------------------
> >>>
> >>> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> >>>                                        unsigned long num_chars)
> >>
> >> I would prefer to simply pass a physical address pointer here with no
> >> requirements on alignment. And no prior SBI call.
>
> sbi_debug_console_set_area() might be called with different values by
> different threads. An offset is ambiguous as it does not define to which
> of the different shared areas it relates. Please, use a pointer.
>
> >>
> >> Do we need num_chars? Are we expecting to provide binary output? Using
> >> 0x00 as terminator would be adequate in most cases.
> >
> > Bare-metal tests (or assembly sources) can print sub-strings from
> > a large per-populated string in shared memory. Assuming that string
> > is always terminated by 0x00 in sbi_debug_console_puts() will break
> > this flexibility.
>
> OK
>
> >
> >>
> >> What is the requirement on the console? Does it have to support 8bit
> >> output to allow for UTF-8?
> >
> > We need to clarify this. Suggestions ?
>
> The platform specification would be the right place to require 8-bit
> support for the console.
>
> >
> >>
> >> Do we make any assumptions about encoding?
> >
> > Same as above, this needs more clarification. Suggestions ?
>
> We should add to the platform specification that UTF-8 output is assumed
> on the serial console.
>
> >
> > I am of the opinion to keep such encoding related assumptions to be
> > minimal.
> >
> >>
> >> How would we handle a console set up to 7bit + parity if a character >
> >> 0x7f is sent?
> >
> > I would consider this to be part of the clarification we add for encoding.
>
> We should state if extra bits are ignored or the bytes are not send.
> The easiest thing is to just ignore the extra bits. So let't state this
> here.
>
> Best regards
>
> Heinrich
>
> >
> >>
> >>>
> >>> Print the string specified by `area_offset` and `num_chars` on
> >>> the debug console. The `area_offset` parameter is the start of
> >>> string in the shard memory area whereas `num_chars` parameter
> >>
> >> %s/shard/shared/
> >
> > Okay, I will update.
> >
> >>
> >>> is the number of characters (or bytes) in the string.
> >>>
> >>> This is a blocking SBI call and will only return after printing
> >>> all characters of the string.
> >>>
> >>> Errors:
> >>> SBI_SUCCESS                - Characters printed successfully.
> >>> SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> >>>                             `area_offset`) or end of the string
> >>>                             (i.e. `area_offset + num_chars`) is
> >>>                             outside shared memory area.
> >>
> >> There could be other reasons of failures:
> >>
> >> * set up of the UART failed in OpenSBI
> >> * no UART defined in the device-tree
> >> * ...
> >>
> >> So let us add SBI_ERR_FAILED to the list.
> >
> > Okay, I will add.
> >
> >>
> >> Best regards
> >>
> >> Heinrich
> >
> > Regards,
> > Anup
>


From apatel at ventanamicro.com  Thu Jun  2 07:22:55 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 19:52:55 +0530
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
 <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego> <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
Message-ID: <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>

On Thu, Jun 2, 2022 at 6:29 PM Ved Shanbhogue <ved at rivosinc.com> wrote:
>
> Should we keep this simple in the SBI - only have register based inputs - to send and receive 1 byte in each call?
> Keeping it a simple out_byte or in_byte - a serial port like interface seems the simplest and most secure.

The legacy SBI v0.1 putchar() and getchar() are byte-level send/receive calls.

This is very slow for virtualized world particularly KVM RISC-V because each
SBI v0.1 putchar() or getchar() will trap to KVM RISC-V and KVM RISC-V will
forward it to user-space QEMU or KVMTOOL. This means each early print
character using SBI v0.1 putchar() will go all the way to host user-space and
come back. This is horribly slow for KVM Guest. This becomes further slower
for nested virtualization.

The MMIO based early prints are further worse because we have at least two
MMIO traps for every character where each MMIO exits to host user-space.

The shared memory based SBI puts() drastically reduces the number of SBI
traps hence reducing boot time with early prints enabled.

> I worry about bugs/security issues that can be caused by M-mode firmware accessing strings in untrusted memory.

The VirtIO based para-virt devices rely heavily on shared memory so I think
it is possible to address security concerns related to shared memory.

> The API as defined does not say whether the address is a virtual address or a physical address.

It is a physical address. I will clarify this in Draft v2.

> If it is a virtual address then the SBI call will need to use a MPRV load/store to gather this data and will also need to deal with page fault, access faults, etc. that may occur on such accesses.

Yes, MPRV (or HLV/HSV) based load/store have performance issues
particularly due to page faults. These are prevalent in some of the
legacy SBI v0.1 calls. With SBI v0.2 (or higher), we have tried to
ensure that we don't use virtual addresses as parameter in newer SBI
calls.

> Based on discussion it did not seem like it needs to be much fancier than this as this is for early OS/VMM code till it has enough functionality to directly interact with a uart.

The goal of the shared memory based SBI call for early prints is to
minimize the number of traps which in-turn helps virtualization to
drastically reduce boot-time.

Regards,
Anup

>
> regards
> ved
>
>
>
> On Thu, Jun 2, 2022 at 7:43 AM Anup Patel <apatel at ventanamicro.com> wrote:
>>
>> On Thu, Jun 2, 2022 at 2:58 PM Heiko St?bner <heiko at sntech.de> wrote:
>> >
>> > Am Donnerstag, 2. Juni 2022, 10:50:56 CEST schrieb Heiko St?bner:
>> > > Am Donnerstag, 2. Juni 2022, 10:47:58 CEST schrieb Anup Patel:
>> > > > On Thu, Jun 2, 2022 at 12:02 AM Heiko St?bner <heiko at sntech.de> wrote:
>> > > > >
>> > > > > Hi Anup,
>> > > > >
>> > > > > Am Mittwoch, 1. Juni 2022, 18:17:32 CEST schrieb Anup Patel:
>> > > > > > Hi All,
>> > > > > >
>> > > > > > Below is the draft proposal for SBI Debug Console Extension.
>> > > > > >
>> > > > > > Please review it and provide feedback.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Anup
>> > > > > >
>> > > > > > Debug Console Extension (EID #0x4442434E "DBCN")
>> > > > > > ================================================
>> > > > > >
>> > > > > > The debug console extension defines a generic mechanism for boot-time
>> > > > > > early prints from supervisor-mode software which allows users to catch
>> > > > > > boot-time issues in supervisor-mode software.
>> > > > > >
>> > > > > > This extension replaces legacy console putchar (EID #0x01) extension
>> > > > > > and it is better in following ways:
>> > > > > > 1) It follows the new calling convention defined for SBI v1.0
>> > > > > >    (or higher).
>> > > > > > 2) It is based on a shared memory area between SBI implementation
>> > > > > >    and supervisor-mode software so multiple characters can be
>> > > > > >    printed using a single SBI call.
>> > > > > >
>> > > > > > The supervisor-mode software must set the shared memory area before
>> > > > > > printing characters on the debug console. Also, all HARTs share the
>> > > > > > same shared memory area so only one HART needs to set it at boot-time.
>> > > > > >
>> > > > > > Function: Set Console Area (FID #0)
>> > > > > > -----------------------------------
>> > > > > >
>> > > > > > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
>> > > > > >                                          unsigned long size)
>> > > > > >
>> > > > > > Set the shared memory area specified by `addr_div_by_2` and `size`
>> > > > >
>> > > > > typo in the "div_by_2" (not 4 like below and in the function itself) ?
>> > > > >
>> > > > >
>> > > > > > parameters. The `addr_div_by_4` parameter is base address of the
>> > > > > > shared memory area right shifted by 2 whereas `size` parameter is
>> > > > > > the size of shared memory area in bytes.
>> > > > > >
>> > > > > > The shared memory area should be normal cacheable memory for the
>> > > > > > supervisor-mode software. Also, the shared memory area is global
>> > > > > > across all HARTs so SBI implementation must ensure atomicity in
>> > > > > > setting the shared memory area.
>> > > > > >
>> > > > > > Errors:
>> > > > > > SBI_SUCCESS                - Shared memory area set successfully.
>> > > > > > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
>> > > > > >                           `addr_div_by_2` and `size` parameters
>> > > > > >                           is not normal cacheable memory or not
>> > > > > >                           accessible to supervisor-mode software.
>> > > > > >
>> > > > > > Function: Console Puts (FID #1)
>> > > > > > -------------------------------
>> > > > > >
>> > > > > > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
>> > > > > >                                      unsigned long num_chars)
>> > > > > >
>> > > > > > Print the string specified by `area_offset` and `num_chars` on
>> > > > > > the debug console. The `area_offset` parameter is the start of
>> > > > > > string in the shard memory area whereas `num_chars` parameter
>> > > > > > is the number of characters (or bytes) in the string.
>> > > > > >
>> > > > > > This is a blocking SBI call and will only return after printing
>> > > > > > all characters of the string.
>> > > > > >
>> > > > > > Errors:
>> > > > > > SBI_SUCCESS                - Characters printed successfully.
>> > > > > > SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
>> > > > > >                           `area_offset`) or end of the string
>> > > > > >                           (i.e. `area_offset + num_chars`) is
>> > > > > >                           outside shared memory area.
>> > > > >
>> > > > > This will vastly reduce the number of needed ecalls when outputting
>> > > > > characters, so this will probably improve performance quite a bit :-)
>> > > > >
>> > > > >
>> > > > > I guess I still would like to have an _additional_ single-character
>> > > > > putc call. As mentioned in the other thread [0], especially on consumer
>> > > > > hardware [where there is no elaborate debug infrastructure] this can
>> > > > > be a very handy debugging tool even in the earliest stages of a
>> > > > > booting kernel (both before relocation and even inside the startup
>> > > > > assembly).
>> > > > >
>> > > > > I.e. just doing a
>> > > > >         li a7, 1
>> > > > >         li a6, 0
>> > > > >         li a0, 36
>> > > > >         ecall
>> > > > >
>> > > > > in any kernel assembly will just output a "$" character right now, without
>> > > > > needing any preparation at all - same with using the current
>> > > > > sbi_console_putchar() directly in c-code.
>> > > > >
>> > > > > This _can_ be very helpful in some cases, so I guess it would be nice
>> > > > > to keep such a functionality around also in the new spec.
>> > > >
>> > > > You can easily create multiple pre-populated strings using ".asciiz" in
>> > > > assembly sources. Just set the base address of pre-populated strings
>> > > > once on boot hart and print from anywhere using usual 4-5 instruction
>> > > > (similar to what posted above).
>> > >
>> > > ok, sounds like a plan as well :-)
>> >
>> > though, how does that relate to the time before MMU setup?
>> >
>> > I.e. in response to Heinrich's mail you talk about svpbmt, so I guess you
>> > expect virtual memory there, so what is the expected value-type before
>> > the mmu is setup in S-mode ?
>>
>> The memory type should be 0 (i.e. PMA) for the shared memory between
>> SBI implementation and supervisor software. Before MMU setup, the
>> memory type is by default 0 (i.e. PMA) so we don't have to mandate
>> any memory type for MMU disabled case.
>>
>> We only have issue on systems with Svpmbt where supervisor software
>> can potentially map the shared memory as non-cacheable or IO (memory
>> type != 0) using PTE memory type bits.
>>
>> In addition to above, a SBI implementation must ensure that the shared
>> memory address provided by supervisor software is a regular memory
>> (not MMIO device). This can be easily achieved in OpenSBI, KVM RISC-V,
>> and various hypervisors.
>>
>> Regards,
>> Anup
>>
>> >
>> >
>> > > > > [0] http://lists.infradead.org/pipermail/opensbi/2022-June/002796.html
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > > > Regards,
>> > > > Anup
>> > > >
>> > >
>> > >
>> >
>> >
>> >
>> >
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#1721): https://lists.riscv.org/g/tech-unixplatformspec/message/1721
>> Mute This Topic: https://lists.riscv.org/mt/91480122/6317211
>> Group Owner: tech-unixplatformspec+owner at lists.riscv.org
>> Unsubscribe: https://lists.riscv.org/g/tech-unixplatformspec/unsub [ved at rivosinc.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>


From ved at rivosinc.com  Thu Jun  2 08:00:32 2022
From: ved at rivosinc.com (Ved Shanbhogue)
Date: Thu, 2 Jun 2022 10:00:32 -0500
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
 <5091667.6fTUFtlzNn@diego>
 <3070891.fEcJ0Lxnt5@diego>
 <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
Message-ID: <YpjQkMhoKuoCj/Do@ved-XPS-8940>

On Thu, Jun 02, 2022 at 07:52:55PM +0530, Anup Patel wrote:
>On Thu, Jun 2, 2022 at 6:29 PM Ved Shanbhogue <ved at rivosinc.com> wrote:
>This is very slow for virtualized world particularly KVM RISC-V because each
>SBI v0.1 putchar() or getchar() will trap to KVM RISC-V and KVM RISC-V will
>forward it to user-space QEMU or KVMTOOL. This means each early print
>character using SBI v0.1 putchar() will go all the way to host user-space and
>come back. This is horribly slow for KVM Guest. This becomes further slower
>for nested virtualization.
>
Since this is for debug and really early phase debug till enough of the guest
boots up to use a VFIO based char driver provided by the VMM, I am not sure
that the slowness matters.
Even this SBI call I expected the VMM to intercept and if the VMM has all 
emulation in the user space VMM - e.g. the console tty is open by the user
space VMM, I am not sure this would avoid that trip to user space. 
If the motivation is primarily VM debug then perhaps a standardized set of
hypercalls implemented by KVM makes more sense than SBI calls that would
need to be built into the M-mode firmware? 

>
>> I worry about bugs/security issues that can be caused by M-mode firmware accessing strings in untrusted memory.
>
>The VirtIO based para-virt devices rely heavily on shared memory so I think
>it is possible to address security concerns related to shared memory.

Yes, and now that I understand the motivation better why dont we define this 
as a hypercall/pv-ops interface to a VMM than a SBI call to the M-mode 
firmware and needing to build a virt-io like framework in firmware.

>
>> The API as defined does not say whether the address is a virtual address or a physical address.
>
>It is a physical address. I will clarify this in Draft v2.
>
Thanks. I was not sure since we had all the discussion about Svpbmt.

>
>> Based on discussion it did not seem like it needs to be much fancier than this as this is for early OS/VMM code till it has enough functionality to directly interact with a uart.
>
>The goal of the shared memory based SBI call for early prints is to
>minimize the number of traps which in-turn helps virtualization to
>drastically reduce boot-time.
>

Understand that better now. But if that is the main motivation then
I am not understanding why we would want to push all of this into
M-mode firmware vs. defining a set of standardized pv-ops to be used
by guest OSes.

regards
ved


From apatel at ventanamicro.com  Thu Jun  2 08:17:51 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 2 Jun 2022 20:47:51 +0530
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <YpjQkMhoKuoCj/Do@ved-XPS-8940>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
 <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego> <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com> <YpjQkMhoKuoCj/Do@ved-XPS-8940>
Message-ID: <CAK9=C2UV4WT-wK0TQh2c1EUYMmSHm0rAgFFz7mtN75ZPQkjb-A@mail.gmail.com>

On Thu, Jun 2, 2022 at 8:30 PM Ved Shanbhogue <ved at rivosinc.com> wrote:
>
> On Thu, Jun 02, 2022 at 07:52:55PM +0530, Anup Patel wrote:
> >On Thu, Jun 2, 2022 at 6:29 PM Ved Shanbhogue <ved at rivosinc.com> wrote:
> >This is very slow for virtualized world particularly KVM RISC-V because each
> >SBI v0.1 putchar() or getchar() will trap to KVM RISC-V and KVM RISC-V will
> >forward it to user-space QEMU or KVMTOOL. This means each early print
> >character using SBI v0.1 putchar() will go all the way to host user-space and
> >come back. This is horribly slow for KVM Guest. This becomes further slower
> >for nested virtualization.
> >
> Since this is for debug and really early phase debug till enough of the guest
> boots up to use a VFIO based char driver provided by the VMM, I am not sure
> that the slowness matters.
> Even this SBI call I expected the VMM to intercept and if the VMM has all
> emulation in the user space VMM - e.g. the console tty is open by the user
> space VMM, I am not sure this would avoid that trip to user space.
> If the motivation is primarily VM debug then perhaps a standardized set of
> hypercalls implemented by KVM makes more sense than SBI calls that would
> need to be built into the M-mode firmware?

We have a requirement in the OS-A platform spec to mandate a particular type
of UART for early prints but there were objections on selecting one type of UART
over another.

This SBI debug console extension is also a way to avoid standardizing a
particular type of UART in OS-A platform spec.

>
> >
> >> I worry about bugs/security issues that can be caused by M-mode firmware accessing strings in untrusted memory.
> >
> >The VirtIO based para-virt devices rely heavily on shared memory so I think
> >it is possible to address security concerns related to shared memory.
>
> Yes, and now that I understand the motivation better why dont we define this
> as a hypercall/pv-ops interface to a VMM than a SBI call to the M-mode
> firmware and needing to build a virt-io like framework in firmware.

The VirtIO framework has rings in shared memory but over here it is just
shared memory without any formatted data structure.

>
> >
> >> The API as defined does not say whether the address is a virtual address or a physical address.
> >
> >It is a physical address. I will clarify this in Draft v2.
> >
> Thanks. I was not sure since we had all the discussion about Svpbmt.
>
> >
> >> Based on discussion it did not seem like it needs to be much fancier than this as this is for early OS/VMM code till it has enough functionality to directly interact with a uart.
> >
> >The goal of the shared memory based SBI call for early prints is to
> >minimize the number of traps which in-turn helps virtualization to
> >drastically reduce boot-time.
> >
>
> Understand that better now. But if that is the main motivation then
> I am not understanding why we would want to push all of this into
> M-mode firmware vs. defining a set of standardized pv-ops to be used
> by guest OSes.

The SBI debug console also provides a standard way of early prints for
supervisor software running natively (directly under M-mode) irrespective
of the type of UART/Console available on the console so it's not just
for a virtualized world. Although, the virtualized world has an added
performance advantage due to reduced traps.

Regards,
Anup


From wxjstz at 126.com  Thu Jun  2 09:00:24 2022
From: wxjstz at 126.com (Xiang W)
Date: Fri, 03 Jun 2022 00:00:24 +0800
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <CAK9=C2VAbPhscKAg+nP25-YrX0UadD6VBkNhqeic-LPQkvma1g@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
	 <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com>
	 <CAK9=C2WLZqkYsgZL5WCJGjWGK=YGV3f51d8yvD3ecx+JPURgpw@mail.gmail.com>
	 <307d7e4e-62f8-8dec-8b09-ea69ca2c29d9@canonical.com>
	 <CAK9=C2VAbPhscKAg+nP25-YrX0UadD6VBkNhqeic-LPQkvma1g@mail.gmail.com>
Message-ID: <b5ed26a9242bf56c3addbb926cdd1ec9cba7b0d5.camel@126.com>

? 2022-06-02???? 18:29 +0530?Anup Patel???
> On Thu, Jun 2, 2022 at 2:43 PM Heinrich Schuchardt
> <heinrich.schuchardt at canonical.com> wrote:
> > 
> > On 6/2/22 10:44, Anup Patel wrote:
> > > On Wed, Jun 1, 2022 at 11:59 PM Heinrich Schuchardt
> > > <heinrich.schuchardt at canonical.com> wrote:
> > > > 
> > > > On 6/1/22 18:17, Anup Patel wrote:
> > > > > Hi All,
> > > > > 
> > > > > Below is the draft proposal for SBI Debug Console Extension.
> > > > > 
> > > > > Please review it and provide feedback.
> > > > > 
> > > > > Thanks,
> > > > > Anup
> > > > > 
> > > > > Debug Console Extension (EID #0x4442434E "DBCN")
> > > > > ================================================
> > > > > 
> > > > > The debug console extension defines a generic mechanism for boot-time
> > > > > early prints from supervisor-mode software which allows users to catch
> > > > > boot-time issues in supervisor-mode software.
> > > > > 
> > > > > This extension replaces legacy console putchar (EID #0x01) extension
> > > > > and it is better in following ways:
> > > > 
> > > > Thanks for starting to close this gap.
> > > > 
> > > > > 1) It follows the new calling convention defined for SBI v1.0
> > > > > ???? (or higher).
> > > > > 2) It is based on a shared memory area between SBI implementation
> > > > > ???? and supervisor-mode software so multiple characters can be
> > > > > ???? printed using a single SBI call.
> > > > 
> > > > I miss a discussion of the conflicts that can arise if the configuration
> > > > of the serial console is changed by the caller.
> > > > 
> > > > Do we need an ecall that closes the SBI console to further access?
> > > 
> > > Usually, the serial port related code in M-mode firmware only uses
> > > status and data registers so for most serial ports support the M-mode
> > > firmware will adapt to serial port configuration changes.
> > > 
> > > In fact, this is why we never had a special SBI call for serial port
> > > reconfiguration in legacy SBI v0.1 as well.
> > > 
> > > In case of virtualization, the serial port (or console) is emulated so
> > > the special SBI call is not useful for virtualization.
> > > 
> > > > 
> > > > > 
> > > > > The supervisor-mode software must set the shared memory area before
> > > > > printing characters on the debug console. Also, all HARTs share the
> > > > > same shared memory area so only one HART needs to set it at boot-time.
> > > > 
> > > > Isn't it M-mode software that has to program the MMU to allow all harts
> > > > in M-mode and S-mode access to the memory area? What is the S-mode
> > > > software to do about the memory area prior to calling
> > > > sbi_debug_console_set_area()?
> > > 
> > > Actually, it's the S-mode software which is voluntarily giving a portion of
> > > its memory to be used as shared memory. The proposal only mandates
> > > that whatever memory is selected by S-mode software should be a
> > > regular cacheable memory (not IO memory). Also, if Svpbmt is available
> > > then S-mode software should only use memory type 0 in the PTEs.
> > > 
> > > > 
> > > > > 
> > > > > Function: Set Console Area (FID #0)
> > > > > -----------------------------------
> > > > > 
> > > > > struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> > > > > ?????????????????????????????????????????? unsigned long size)
> > > > 
> > > > The console output is needed on the very start of the S-mode software,
> > > > before setting up anything.
> > > > 
> > > > Can we avoid this extra function?
> > > > 
> > > > Can we simply assume that the caller of sbi_debug_console_puts()
> > > > provides a physical address pointer to a memory area that is suitable?
> > > 
> > > Theoretically, we can avoid the extra function to set shared memory area
> > > but it will complicate things in future when we have supervisor software
> > > encrypting it's own memory (using special ISA support) because in this
> > > case supervisor software will have to unprotect memory every time the
> > > sbi_debug_console_puts() is called and protect it again after the call.
> > 
> > Currently this function is just a nop(). It is not needed in this
> > revision of the extension.
> > 
> > The function might be called repeatedly by different threads with
> > different values. How do you want to keep track of all of these
> > different areas?
> 
> The shared memory area in case of this SBI extension will be shared
> across all HARTs so the SBI implementation will ensure atomicity
> in setting/reading shared memory coordinates. This way multiple
> HARTs can call the sbi_debug_console_set_area() but only the
> last call will be in effect.
> 
> Following text in the proposal explains above:
> "The supervisor-mode software must set the shared memory area before
> printing characters on the debug console. Also, all HARTs share the
> same shared memory area so only one HART needs to set it at boot-time."
> 
> Maybe this text is not giving a clear picture ?

There is a strict calling order between the two APIs, which?
adds to the complexity. If we pass the physical address directly
to sbi_debug_console_puts will circumvent these problems.

I would like to know the necessity of setting up shared memory first.

Regards,
Xiang W
> 
> > 
> > Memory shared between different security realms will arise in many
> > different scenarios. As this is not console specific it should be in a
> > separate extension. That extension should be defined once we have
> > clarity about how security realms are managed.
> 
> In the case of this extension, the shared memory is global across all
> HARTs and it mostly contains bytes to be printed. In case of steal
> time accounting, the shared memory is separate for each HART and
> it is a well-defined data structure.
> 
> Clearly, the usage of shared memory is extension specific.
> 
> The VirtIO devices are a good (and time tested) example of shared
> memory based approaches. Over there as well, the shared memory
> (i.e. various VirtIO rings) are setup by Guest and there is no central
> pool of shared memory (i.e no dedicated VirtIO device managing
> shared memory).
> 
> Similar to VirtIO world, we should let SBI extension define its own
> shared memory usage and API.
> 
> Regards,
> Anup
> 
> > 
> > > 
> > > > 
> > > > > 
> > > > > Set the shared memory area specified by `addr_div_by_2` and `size`
> > > > 
> > > > %s/addr_div_by_2/addr_div_by_4/
> > > 
> > > Okay, I will update.
> > > 
> > > > 
> > > > > parameters. The `addr_div_by_4` parameter is base address of the
> > > > 
> > > > %s/is base/is the base/
> > > 
> > > Okay, I will update.
> > > 
> > > > 
> > > > > shared memory area right shifted by 2 whereas `size` parameter is
> > > > > the size of shared memory area in bytes.
> > > > 
> > > > Why shifting the address? I would prefer to keep it simple for the
> > > > caller. If the alignment is not suitable, return an error.
> > > > 
> > > > But why is an alignment needed here at all? And why 4 aligned?
> > > 
> > > For RV32 S-mode, the physical address space is 34bits wide but
> > > "unsigned long" is 32bits wide. This is because Sv32 PTEs allow
> > > 34bits of PPN. In fact, even instructions such as HFENCE.GVMA
> > > have this "address right shift by 2" requirement based on this rationale.
> > > 
> > > > 
> > > > > 
> > > > > The shared memory area should be normal cacheable memory for the
> > > > > supervisor-mode software. Also, the shared memory area is global
> > > > > across all HARTs so SBI implementation must ensure atomicity in
> > > > > setting the shared memory area.
> > > > > 
> > > > > Errors:
> > > > > SBI_SUCCESS??????????????? - Shared memory area set successfully.
> > > > > SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> > > > > ??????????????????????????? `addr_div_by_2` and `size` parameters
> > > > > ??????????????????????????? is not normal cacheable memory or not
> > > > > ??????????????????????????? accessible to supervisor-mode software.
> > > > > 
> > > > > Function: Console Puts (FID #1)
> > > > > -------------------------------
> > > > > 
> > > > > struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> > > > > ?????????????????????????????????????? unsigned long num_chars)
> > > > 
> > > > I would prefer to simply pass a physical address pointer here with no
> > > > requirements on alignment. And no prior SBI call.
> > 
> > sbi_debug_console_set_area() might be called with different values by
> > different threads. An offset is ambiguous as it does not define to which
> > of the different shared areas it relates. Please, use a pointer.
> > 
> > > > 
> > > > Do we need num_chars? Are we expecting to provide binary output? Using
> > > > 0x00 as terminator would be adequate in most cases.
> > > 
> > > Bare-metal tests (or assembly sources) can print sub-strings from
> > > a large per-populated string in shared memory. Assuming that string
> > > is always terminated by 0x00 in sbi_debug_console_puts() will break
> > > this flexibility.
> > 
> > OK
> > 
> > > 
> > > > 
> > > > What is the requirement on the console? Does it have to support 8bit
> > > > output to allow for UTF-8?
> > > 
> > > We need to clarify this. Suggestions ?
> > 
> > The platform specification would be the right place to require 8-bit
> > support for the console.
> > 
> > > 
> > > > 
> > > > Do we make any assumptions about encoding?
> > > 
> > > Same as above, this needs more clarification. Suggestions ?
> > 
> > We should add to the platform specification that UTF-8 output is assumed
> > on the serial console.
> > 
> > > 
> > > I am of the opinion to keep such encoding related assumptions to be
> > > minimal.
> > > 
> > > > 
> > > > How would we handle a console set up to 7bit + parity if a character >
> > > > 0x7f is sent?
> > > 
> > > I would consider this to be part of the clarification we add for encoding.
> > 
> > We should state if extra bits are ignored or the bytes are not send.
> > The easiest thing is to just ignore the extra bits. So let't state this
> > here.
> > 
> > Best regards
> > 
> > Heinrich
> > 
> > > 
> > > > 
> > > > > 
> > > > > Print the string specified by `area_offset` and `num_chars` on
> > > > > the debug console. The `area_offset` parameter is the start of
> > > > > string in the shard memory area whereas `num_chars` parameter
> > > > 
> > > > %s/shard/shared/
> > > 
> > > Okay, I will update.
> > > 
> > > > 
> > > > > is the number of characters (or bytes) in the string.
> > > > > 
> > > > > This is a blocking SBI call and will only return after printing
> > > > > all characters of the string.
> > > > > 
> > > > > Errors:
> > > > > SBI_SUCCESS??????????????? - Characters printed successfully.
> > > > > SBI_ERR_INVALID_ADDRESS??? - The start of the string (i.e.
> > > > > ??????????????????????????? `area_offset`) or end of the string
> > > > > ??????????????????????????? (i.e. `area_offset + num_chars`) is
> > > > > ??????????????????????????? outside shared memory area.
> > > > 
> > > > There could be other reasons of failures:
> > > > 
> > > > * set up of the UART failed in OpenSBI
> > > > * no UART defined in the device-tree
> > > > * ...
> > > > 
> > > > So let us add SBI_ERR_FAILED to the list.
> > > 
> > > Okay, I will add.
> > > 
> > > > 
> > > > Best regards
> > > > 
> > > > Heinrich
> > > 
> > > Regards,
> > > Anup
> > 
> 




From atishp at rivosinc.com  Thu Jun  2 11:55:12 2022
From: atishp at rivosinc.com (Atish Kumar Patra)
Date: Thu, 2 Jun 2022 11:55:12 -0700
Subject: [RISC-V][tech-os-a-see] [RISC-V] [tech-unixplatformspec] SBI
 Debug Console Extension Proposal (Draft v1)
In-Reply-To: <CAK9=C2UV4WT-wK0TQh2c1EUYMmSHm0rAgFFz7mtN75ZPQkjb-A@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
 <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego> <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
 <YpjQkMhoKuoCj/Do@ved-XPS-8940> <CAK9=C2UV4WT-wK0TQh2c1EUYMmSHm0rAgFFz7mtN75ZPQkjb-A@mail.gmail.com>
Message-ID: <CAHBxVyEQOGOLM=cCDWvVfidS+4SoecVM2T6gbF48VuRCiirExQ@mail.gmail.com>

On Thu, Jun 2, 2022 at 8:18 AM Anup Patel <apatel at ventanamicro.com> wrote:
>
> On Thu, Jun 2, 2022 at 8:30 PM Ved Shanbhogue <ved at rivosinc.com> wrote:
> >
> > On Thu, Jun 02, 2022 at 07:52:55PM +0530, Anup Patel wrote:
> > >On Thu, Jun 2, 2022 at 6:29 PM Ved Shanbhogue <ved at rivosinc.com> wrote:
> > >This is very slow for virtualized world particularly KVM RISC-V because each
> > >SBI v0.1 putchar() or getchar() will trap to KVM RISC-V and KVM RISC-V will
> > >forward it to user-space QEMU or KVMTOOL. This means each early print
> > >character using SBI v0.1 putchar() will go all the way to host user-space and
> > >come back. This is horribly slow for KVM Guest. This becomes further slower
> > >for nested virtualization.
> > >
> > Since this is for debug and really early phase debug till enough of the guest
> > boots up to use a VFIO based char driver provided by the VMM, I am not sure
> > that the slowness matters.
> > Even this SBI call I expected the VMM to intercept and if the VMM has all
> > emulation in the user space VMM - e.g. the console tty is open by the user
> > space VMM, I am not sure this would avoid that trip to user space.
> > If the motivation is primarily VM debug then perhaps a standardized set of
> > hypercalls implemented by KVM makes more sense than SBI calls that would
> > need to be built into the M-mode firmware?
>
> We have a requirement in the OS-A platform spec to mandate a particular type
> of UART for early prints but there were objections on selecting one type of UART
> over another.
>
> This SBI debug console extension is also a way to avoid standardizing a
> particular type of UART in OS-A platform spec.
>
> >
> > >
> > >> I worry about bugs/security issues that can be caused by M-mode firmware accessing strings in untrusted memory.
> > >
> > >The VirtIO based para-virt devices rely heavily on shared memory so I think
> > >it is possible to address security concerns related to shared memory.
> >
> > Yes, and now that I understand the motivation better why dont we define this
> > as a hypercall/pv-ops interface to a VMM than a SBI call to the M-mode
> > firmware and needing to build a virt-io like framework in firmware.
>
> The VirtIO framework has rings in shared memory but over here it is just
> shared memory without any formatted data structure.
>
> >
> > >
> > >> The API as defined does not say whether the address is a virtual address or a physical address.
> > >
> > >It is a physical address. I will clarify this in Draft v2.
> > >
> > Thanks. I was not sure since we had all the discussion about Svpbmt.
> >
> > >
> > >> Based on discussion it did not seem like it needs to be much fancier than this as this is for early OS/VMM code till it has enough functionality to directly interact with a uart.
> > >
> > >The goal of the shared memory based SBI call for early prints is to
> > >minimize the number of traps which in-turn helps virtualization to
> > >drastically reduce boot-time.
> > >
> >
> > Understand that better now. But if that is the main motivation then
> > I am not understanding why we would want to push all of this into
> > M-mode firmware vs. defining a set of standardized pv-ops to be used
> > by guest OSes.
>
> The SBI debug console also provides a standard way of early prints for
> supervisor software running natively (directly under M-mode) irrespective
> of the type of UART/Console available on the console so it's not just
> for a virtualized world. Although, the virtualized world has an added
> performance advantage due to reduced traps.
>

It's also helpful in early board bringup and early debugging as
pointed out by Heiko as well.
To address some of the concerns raised above, how about putting some
restrictions on sbi_debug_console_set_area ?

1. Instead of any size, restrict the shared memory region to a fixed
size (256 byte should be enough for early prints) ?
It is still a common area across all harts. Thus, atomicity needs to
be ensured by the M-mode or HS mode.
We may explore setting up a per-heart area to avoid synchronization
issues but platforms with large hart counts may have to
share multiple pages depending on the page size.

2. The shared memory region should be set up only once during a boot
cycle. Any further invocation of sbi_debug_console_set_area
should return appropriate errors.

If the per heart shared memory region approach is preferred, the above
restriction applies once per hart.

> Regards,
> Anup
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#31): https://lists.riscv.org/g/tech-os-a-see/message/31
> Mute This Topic: https://lists.riscv.org/mt/91498450/1774178
> Group Owner: tech-os-a-see+owner at lists.riscv.org
> Unsubscribe: https://lists.riscv.org/g/tech-os-a-see/leave/11230180/1774178/2121601819/xyzzy [atishp at rivosinc.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>


From stefano.stabellini at xilinx.com  Thu Jun  2 12:39:50 2022
From: stefano.stabellini at xilinx.com (Stefano Stabellini)
Date: Thu, 2 Jun 2022 12:39:50 -0700
Subject: [sig-hypervisors] SBI Debug Console Extension Proposal (Draft
 v1)
In-Reply-To: <dc89b97c48364769b1bf7bbc51dd440b@siemens.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com> <dc89b97c48364769b1bf7bbc51dd440b@siemens.com>
Message-ID: <alpine.DEB.2.22.394.2206021231060.2783803@ubuntu-linux-20-04-desktop>

On Thu, 2 Jun 2022, Schwarz, Konrad via lists.riscv.org wrote:
> Hi Anup,
> 
> > From: sig-hypervisors at lists.riscv.org <sig-hypervisors at lists.riscv.org> On Behalf Of Anup Patel via
> > lists.riscv.org
> > Subject: [sig-hypervisors] SBI Debug Console Extension Proposal (Draft v1)
> > 
> > Below is the draft proposal for SBI Debug Console Extension.
> 
> Here are my thoughts:
> 
> * Guest memory access: I think this would be the first SBI extension to require access to
>   guest memory. This needs to be considered carefully, but I think the higher bandwidth afforded by
>   the interface is useful enough to allow this.

> * API:
> 	* Currently, only a write interface is provided. It would be much better to have a
> 	  read/write interface.
> 
>         Benefits of this would be to allow a hypervisor to control an OS, e.g., for testing purposes
>         or to automate installation tasks.  Inter-guest communication could also be realized
>         via such an interface.
> 

This could be done. As an example Xen provides the hypercall
HYPERVISOR_console_io. The first parameter is the operation:
CONSOLEIO_write or CONSOLEIO_read.

The interface in RISC-V spec language would be something along these
lines:

struct sbiret sbi_debug_console_io(unsigned long operation, /* read or write */
                                   unsigned long address,
                                   unsigned long num_chars)

There is no memory area pre-registration required, however appropriate
checks on the validity of the address provided should always be done in
the implementation.

The good thing about getting rid of the pre-registration is that
multiple threads could make concurrent sbi_debug_console_io requests on
different CPUs.

I think it might be a good idea in the guest-side implementation (e.g.
Linux or Zephyr) to choose a specific memory area for this. However, it
doesn't have to be part of the interface. I think it should be an
implementation detail.

The firmware/hypervisor-side implementation of sbi_debug_console_io
can deal with any addresses provided as long as they are valid.


From stefano.stabellini at xilinx.com  Thu Jun  2 12:43:26 2022
From: stefano.stabellini at xilinx.com (Stefano Stabellini)
Date: Thu, 2 Jun 2022 12:43:26 -0700
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <YpjQkMhoKuoCj/Do@ved-XPS-8940>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com> <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com> <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego> <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com> <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com> <YpjQkMhoKuoCj/Do@ved-XPS-8940>
Message-ID: <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>

On Thu, 2 Jun 2022, Vedvyas Shanbhogue via lists.riscv.org wrote:
> > > Based on discussion it did not seem like it needs to be much fancier than
> > > this as this is for early OS/VMM code till it has enough functionality to
> > > directly interact with a uart.
> > 
> > The goal of the shared memory based SBI call for early prints is to
> > minimize the number of traps which in-turn helps virtualization to
> > drastically reduce boot-time.
> > 
> 
> Understand that better now. But if that is the main motivation then
> I am not understanding why we would want to push all of this into
> M-mode firmware vs. defining a set of standardized pv-ops to be used
> by guest OSes.

That is because it is useful to have debug console output when porting a
hypervisor or baremetal code to a new board.

Of course, if a hypervisor is already available for the board, then it
would be just as easy to use a paravirtualized interface, e.g. Xen's
HYPERVISOR_console_io hypercall. But somebody has to port the
hypervisor first :-)


From ved at rivosinc.com  Thu Jun  2 13:01:33 2022
From: ved at rivosinc.com (Ved Shanbhogue)
Date: Thu, 2 Jun 2022 15:01:33 -0500
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
 <5091667.6fTUFtlzNn@diego>
 <3070891.fEcJ0Lxnt5@diego>
 <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
 <YpjQkMhoKuoCj/Do@ved-XPS-8940>
 <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
Message-ID: <YpkXHYwhz8efa6ON@ved-XPS-8940>

>Of course, if a hypervisor is already available for the board, then it
>would be just as easy to use a paravirtualized interface, e.g. Xen's
>HYPERVISOR_console_io hypercall. But somebody has to port the
>hypervisor first :-)
>
Of course a console is needed. But I was questioning the need for something much
more elaborate than a putchar/getchar interface. I understand its needed to port
the hypervisor but I undersatnd it would be needed for very early parts of the 
hypervisor where it does not even have the ability to master a uart on its own.

regards
ved


From stefano.stabellini at xilinx.com  Thu Jun  2 13:29:28 2022
From: stefano.stabellini at xilinx.com (Stefano Stabellini)
Date: Thu, 2 Jun 2022 13:29:28 -0700
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <YpkXHYwhz8efa6ON@ved-XPS-8940>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com> <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com> <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego> <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com> <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com> <YpjQkMhoKuoCj/Do@ved-XPS-8940> <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
 <YpkXHYwhz8efa6ON@ved-XPS-8940>
Message-ID: <alpine.DEB.2.22.394.2206021327250.2783803@ubuntu-linux-20-04-desktop>

On Thu, 2 Jun 2022, Vedvyas Shanbhogue via lists.riscv.org wrote:
> > Of course, if a hypervisor is already available for the board, then it
> > would be just as easy to use a paravirtualized interface, e.g. Xen's
> > HYPERVISOR_console_io hypercall. But somebody has to port the
> > hypervisor first :-)
> > 
> Of course a console is needed. But I was questioning the need for something
> much
> more elaborate than a putchar/getchar interface. I understand its needed to
> port
> the hypervisor but I undersatnd it would be needed for very early parts of the
> hypervisor where it does not even have the ability to master a uart on its
> own.

Yeah, I think you are right that it is not a must-have feature but a
nice-to-have feature.


From ved at rivosinc.com  Thu Jun  2 14:05:06 2022
From: ved at rivosinc.com (Ved Shanbhogue)
Date: Thu, 2 Jun 2022 16:05:06 -0500
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <CA+Qh7TnDUN4kBKJqkeSQoypTm71Sz8QhzqMSE9xVDrre_gGt8w@mail.gmail.com>
References: <5091667.6fTUFtlzNn@diego>
 <3070891.fEcJ0Lxnt5@diego>
 <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
 <YpjQkMhoKuoCj/Do@ved-XPS-8940>
 <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
 <YpkXHYwhz8efa6ON@ved-XPS-8940>
 <alpine.DEB.2.22.394.2206021327250.2783803@ubuntu-linux-20-04-desktop>
 <CA+Qh7TnDUN4kBKJqkeSQoypTm71Sz8QhzqMSE9xVDrre_gGt8w@mail.gmail.com>
Message-ID: <YpkmAkGcS2qzSX2K@ved-XPS-8940>

On Thu, Jun 02, 2022 at 01:50:01PM -0700, Greg Favor wrote:
>On Thu, Jun 2, 2022 at 1:29 PM Stefano Stabellini <
>stefano.stabellini at xilinx.com> wrote:
>
>> On Thu, 2 Jun 2022, Vedvyas Shanbhogue via lists.riscv.org wrote:
>> > Of course a console is needed. But I was questioning the need for
>> something much
>> > more elaborate than a putchar/getchar interface. I understand its needed
>> to port
>> > the hypervisor but I undersatnd it would be needed for very early parts
>> of the
>> > hypervisor where it does not even have the ability to master a uart on
>> its own.
>>
>> Yeah, I think you are right that it is not a must-have feature but a
>> nice-to-have feature.
>
>
>Even though we all (me included) tend to think that slowing down something
>like this will be inconsequential to the time to boot and/or who cares how
>fast or slow is boot time, in the past I have found that to be a risky
>presumption in certain production environments where boot time matters
>(especially with frequent standing up of VMs from scratch).  So a
>worthwhile sanity check question would be 1) what percentage of early boot
>time is spent writing to the console with per-string SBI calls, and 2) by
>what order of magnitude does that percentage inflate due to SBI calls for
>every individual character.
>
I think the extension was proposed to be a Debug console extension.

In a production environment if 10's of VMs were booting having them emit
to the physical console - where all of those outputs will get interspersed
seems to be not so useful. In a production environment would we expect the
VMM to provide a console service and not need SBI calls into M-mode firmare
to emit VM output to a physical console?

I was thinking this was about early boot debug and not so much about needing
a console at early boot. In most production deployments there may not be 
someone sitting at a terminal watching prints and so it may make more sense
for VM console outputs to be logged into a persistent file somewhere for example.
Or be held in the VM itself - e.g. a kernel ring buffer that someone may want
to display using a tool like dmesg when needed but otherwise does not get 
emitted chattily to a physical console.

Hope I am understanding this right.

regards
ved


From apatel at ventanamicro.com  Thu Jun  2 20:00:45 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 3 Jun 2022 08:30:45 +0530
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <alpine.DEB.2.22.394.2206021354290.2783803@ubuntu-linux-20-04-desktop>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
 <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego> <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
 <YpjQkMhoKuoCj/Do@ved-XPS-8940> <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
 <YpkXHYwhz8efa6ON@ved-XPS-8940> <alpine.DEB.2.22.394.2206021327250.2783803@ubuntu-linux-20-04-desktop>
 <CA+Qh7TnDUN4kBKJqkeSQoypTm71Sz8QhzqMSE9xVDrre_gGt8w@mail.gmail.com> <alpine.DEB.2.22.394.2206021354290.2783803@ubuntu-linux-20-04-desktop>
Message-ID: <CAK9=C2XbTQysZghPa393A7R=y8OhD2Hyg_4iqxNsgJBAGj4O7Q@mail.gmail.com>

Hi Stefano,

On Fri, Jun 3, 2022 at 2:35 AM Stefano Stabellini
<stefano.stabellini at xilinx.com> wrote:
>
> On Thu, 2 Jun 2022, Greg Favor wrote:
> > On Thu, Jun 2, 2022 at 1:29 PM Stefano Stabellini <stefano.stabellini at xilinx.com> wrote:
> >       On Thu, 2 Jun 2022, Vedvyas Shanbhogue via lists.riscv.org wrote:
> >       > Of course a console is needed. But I was questioning the need for something much
> >       > more elaborate than a putchar/getchar interface. I understand its needed to port
> >       > the hypervisor but I undersatnd it would be needed for very early parts of the
> >       > hypervisor where it does not even have the ability to master a uart on its own.
> >
> >       Yeah, I think you are right that it is not a must-have feature but a
> >       nice-to-have feature.
> >
> >
> > Even though we all (me included) tend to think that slowing down something like this will be inconsequential to the time to boot and/or who
> > cares how fast or slow is boot time, in the past I have found that to be a risky presumption in certain production environments where boot
> > time matters (especially with frequent standing up of VMs from scratch).  So a worthwhile sanity check question would be 1) what percentage
> > of early boot time is spent writing to the console with per-string SBI calls, and 2) by what order of magnitude does that percentage
> > inflate due to SBI calls for every individual character.
>
> The numbers you asked will help but I just wanted to add that in my
> opinion a per-character putchar interface is not suitable for production
> due to the reasons you mentioned. I would compile it out in non-DEBUG
> builds.
>
> On the other hand the per-string print interface might be usable in
> production, even not just early boot. However, of course, a proper UART
> driver would be even better.
>
> This is why I think it is nice to have: it expands the potential uses of
> the SBI console interface, and it is simple and easy to use. At the same
> time it is not a must-have because you could get away with just putchat
> in early boot for DEBUG builds, and nothing + UART driver in non-DEBUG
> builds.
>
> That is why I am in favor of this addition, although I recognize it is
> not critical.

We have legacy SBI v0.1 calls where most of them are replaced by
newer SBI v0.2 (or higher) calls. Only the SBI v0.1 putchar() does not
have a replacement. This putchar() was being used in various cases
for early prints from OSes and bootloaders.

The SBI debug console extension is an attempt to replace the legacy
SBI v0.1 putchar(). The use of shared memory in this proposal is
motivated by the need to allow users to print multiple characters in
one SBI call.

The legacy SBI v0.1 also had a getchar() call which is deprecated and
does not need any newer SBI call replacing it because it is expected
all platforms (including virtual platforms emulated by hypervisors) will
have a proper interrupt driver console for supervisor software. The
proposed SBI debug console extension only tries to provide a solution
for early prints.

Regards,
Anup


From atishp at rivosinc.com  Thu Jun  2 23:56:56 2022
From: atishp at rivosinc.com (Atish Kumar Patra)
Date: Thu, 2 Jun 2022 23:56:56 -0700
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <307d7e4e-62f8-8dec-8b09-ea69ca2c29d9@canonical.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com> <CAK9=C2WLZqkYsgZL5WCJGjWGK=YGV3f51d8yvD3ecx+JPURgpw@mail.gmail.com>
 <307d7e4e-62f8-8dec-8b09-ea69ca2c29d9@canonical.com>
Message-ID: <CAHBxVyEDR-16-nEd0rRCvegfAxPf2xSMB6HZpe3269GjmVUzQw@mail.gmail.com>

On Thu, Jun 2, 2022 at 2:13 AM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> On 6/2/22 10:44, Anup Patel wrote:
> > On Wed, Jun 1, 2022 at 11:59 PM Heinrich Schuchardt
> > <heinrich.schuchardt at canonical.com> wrote:
> >>
> >> On 6/1/22 18:17, Anup Patel wrote:
> >>> Hi All,
> >>>
> >>> Below is the draft proposal for SBI Debug Console Extension.
> >>>
> >>> Please review it and provide feedback.
> >>>
> >>> Thanks,
> >>> Anup
> >>>
> >>> Debug Console Extension (EID #0x4442434E "DBCN")
> >>> ================================================
> >>>
> >>> The debug console extension defines a generic mechanism for boot-time
> >>> early prints from supervisor-mode software which allows users to catch
> >>> boot-time issues in supervisor-mode software.
> >>>
> >>> This extension replaces legacy console putchar (EID #0x01) extension
> >>> and it is better in following ways:
> >>
> >> Thanks for starting to close this gap.
> >>
> >>> 1) It follows the new calling convention defined for SBI v1.0
> >>>      (or higher).
> >>> 2) It is based on a shared memory area between SBI implementation
> >>>      and supervisor-mode software so multiple characters can be
> >>>      printed using a single SBI call.
> >>
> >> I miss a discussion of the conflicts that can arise if the configuration
> >> of the serial console is changed by the caller.
> >>
> >> Do we need an ecall that closes the SBI console to further access?
> >
> > Usually, the serial port related code in M-mode firmware only uses
> > status and data registers so for most serial ports support the M-mode
> > firmware will adapt to serial port configuration changes.
> >
> > In fact, this is why we never had a special SBI call for serial port
> > reconfiguration in legacy SBI v0.1 as well.
> >
> > In case of virtualization, the serial port (or console) is emulated so
> > the special SBI call is not useful for virtualization.
> >
> >>
> >>>
> >>> The supervisor-mode software must set the shared memory area before
> >>> printing characters on the debug console. Also, all HARTs share the
> >>> same shared memory area so only one HART needs to set it at boot-time.
> >>
> >> Isn't it M-mode software that has to program the MMU to allow all harts
> >> in M-mode and S-mode access to the memory area? What is the S-mode
> >> software to do about the memory area prior to calling
> >> sbi_debug_console_set_area()?
> >
> > Actually, it's the S-mode software which is voluntarily giving a portion of
> > its memory to be used as shared memory. The proposal only mandates
> > that whatever memory is selected by S-mode software should be a
> > regular cacheable memory (not IO memory). Also, if Svpbmt is available
> > then S-mode software should only use memory type 0 in the PTEs.
> >
> >>
> >>>
> >>> Function: Set Console Area (FID #0)
> >>> -----------------------------------
> >>>
> >>> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> >>>                                            unsigned long size)
> >>
> >> The console output is needed on the very start of the S-mode software,
> >> before setting up anything.
> >>
> >> Can we avoid this extra function?
> >>
> >> Can we simply assume that the caller of sbi_debug_console_puts()
> >> provides a physical address pointer to a memory area that is suitable?
> >
> > Theoretically, we can avoid the extra function to set shared memory area
> > but it will complicate things in future when we have supervisor software
> > encrypting it's own memory (using special ISA support) because in this
> > case supervisor software will have to unprotect memory every time the
> > sbi_debug_console_puts() is called and protect it again after the call.
>
> Currently this function is just a nop(). It is not needed in this
> revision of the extension.
>
> The function might be called repeatedly by different threads with
> different values. How do you want to keep track of all of these
> different areas?
>
> Memory shared between different security realms will arise in many
> different scenarios. As this is not console specific it should be in a
> separate extension. That extension should be defined once we have
> clarity about how security realms are managed.
>

I would like to understand more about the security concerns you raised.
Can you please elaborate on this ?

As you pointed out, there are other possible use cases(e.g STA, PMU)
for shared memory between
S & M mode or VS & HS mode. It would be good to address these concerns right now
instead of discussing those in each individual extension context.

> >
> >>
> >>>
> >>> Set the shared memory area specified by `addr_div_by_2` and `size`
> >>
> >> %s/addr_div_by_2/addr_div_by_4/
> >
> > Okay, I will update.
> >
> >>
> >>> parameters. The `addr_div_by_4` parameter is base address of the
> >>
> >> %s/is base/is the base/
> >
> > Okay, I will update.
> >
> >>
> >>> shared memory area right shifted by 2 whereas `size` parameter is
> >>> the size of shared memory area in bytes.
> >>
> >> Why shifting the address? I would prefer to keep it simple for the
> >> caller. If the alignment is not suitable, return an error.
> >>
> >> But why is an alignment needed here at all? And why 4 aligned?
> >
> > For RV32 S-mode, the physical address space is 34bits wide but
> > "unsigned long" is 32bits wide. This is because Sv32 PTEs allow
> > 34bits of PPN. In fact, even instructions such as HFENCE.GVMA
> > have this "address right shift by 2" requirement based on this rationale.
> >
> >>
> >>>
> >>> The shared memory area should be normal cacheable memory for the
> >>> supervisor-mode software. Also, the shared memory area is global
> >>> across all HARTs so SBI implementation must ensure atomicity in
> >>> setting the shared memory area.
> >>>
> >>> Errors:
> >>> SBI_SUCCESS                - Shared memory area set successfully.
> >>> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> >>>                             `addr_div_by_2` and `size` parameters
> >>>                             is not normal cacheable memory or not
> >>>                             accessible to supervisor-mode software.
> >>>
> >>> Function: Console Puts (FID #1)
> >>> -------------------------------
> >>>
> >>> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> >>>                                        unsigned long num_chars)
> >>
> >> I would prefer to simply pass a physical address pointer here with no
> >> requirements on alignment. And no prior SBI call.
>
> sbi_debug_console_set_area() might be called with different values by
> different threads. An offset is ambiguous as it does not define to which
> of the different shared areas it relates. Please, use a pointer.
>
> >>
> >> Do we need num_chars? Are we expecting to provide binary output? Using
> >> 0x00 as terminator would be adequate in most cases.
> >
> > Bare-metal tests (or assembly sources) can print sub-strings from
> > a large per-populated string in shared memory. Assuming that string
> > is always terminated by 0x00 in sbi_debug_console_puts() will break
> > this flexibility.
>
> OK
>
> >
> >>
> >> What is the requirement on the console? Does it have to support 8bit
> >> output to allow for UTF-8?
> >
> > We need to clarify this. Suggestions ?
>
> The platform specification would be the right place to require 8-bit
> support for the console.
>
> >
> >>
> >> Do we make any assumptions about encoding?
> >
> > Same as above, this needs more clarification. Suggestions ?
>
> We should add to the platform specification that UTF-8 output is assumed
> on the serial console.
>
> >
> > I am of the opinion to keep such encoding related assumptions to be
> > minimal.
> >
> >>
> >> How would we handle a console set up to 7bit + parity if a character >
> >> 0x7f is sent?
> >
> > I would consider this to be part of the clarification we add for encoding.
>
> We should state if extra bits are ignored or the bytes are not send.
> The easiest thing is to just ignore the extra bits. So let't state this
> here.
>
> Best regards
>
> Heinrich
>
> >
> >>
> >>>
> >>> Print the string specified by `area_offset` and `num_chars` on
> >>> the debug console. The `area_offset` parameter is the start of
> >>> string in the shard memory area whereas `num_chars` parameter
> >>
> >> %s/shard/shared/
> >
> > Okay, I will update.
> >
> >>
> >>> is the number of characters (or bytes) in the string.
> >>>
> >>> This is a blocking SBI call and will only return after printing
> >>> all characters of the string.
> >>>
> >>> Errors:
> >>> SBI_SUCCESS                - Characters printed successfully.
> >>> SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> >>>                             `area_offset`) or end of the string
> >>>                             (i.e. `area_offset + num_chars`) is
> >>>                             outside shared memory area.
> >>
> >> There could be other reasons of failures:
> >>
> >> * set up of the UART failed in OpenSBI
> >> * no UART defined in the device-tree
> >> * ...
> >>
> >> So let us add SBI_ERR_FAILED to the list.
> >
> > Okay, I will add.
> >
> >>
> >> Best regards
> >>
> >> Heinrich
> >
> > Regards,
> > Anup
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#1717): https://lists.riscv.org/g/tech-unixplatformspec/message/1717
> Mute This Topic: https://lists.riscv.org/mt/91480122/1774178
> Group Owner: tech-unixplatformspec+owner at lists.riscv.org
> Unsubscribe: https://lists.riscv.org/g/tech-unixplatformspec/unsub [atishp at rivosinc.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>


From heinrich.schuchardt at canonical.com  Fri Jun  3 00:23:16 2022
From: heinrich.schuchardt at canonical.com (Heinrich Schuchardt)
Date: Fri, 3 Jun 2022 09:23:16 +0200
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <CAHBxVyEDR-16-nEd0rRCvegfAxPf2xSMB6HZpe3269GjmVUzQw@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com>
 <CAK9=C2WLZqkYsgZL5WCJGjWGK=YGV3f51d8yvD3ecx+JPURgpw@mail.gmail.com>
 <307d7e4e-62f8-8dec-8b09-ea69ca2c29d9@canonical.com>
 <CAHBxVyEDR-16-nEd0rRCvegfAxPf2xSMB6HZpe3269GjmVUzQw@mail.gmail.com>
Message-ID: <16ce4a1f-9a88-de85-d1d1-05cc5a542366@canonical.com>

On 6/3/22 08:56, Atish Kumar Patra wrote:
> On Thu, Jun 2, 2022 at 2:13 AM Heinrich Schuchardt
> <heinrich.schuchardt at canonical.com> wrote:
>>
>> On 6/2/22 10:44, Anup Patel wrote:
>>> On Wed, Jun 1, 2022 at 11:59 PM Heinrich Schuchardt
>>> <heinrich.schuchardt at canonical.com> wrote:
>>>>
>>>> On 6/1/22 18:17, Anup Patel wrote:
>>>>> Hi All,
>>>>>
>>>>> Below is the draft proposal for SBI Debug Console Extension.
>>>>>
>>>>> Please review it and provide feedback.
>>>>>
>>>>> Thanks,
>>>>> Anup
>>>>>
>>>>> Debug Console Extension (EID #0x4442434E "DBCN")
>>>>> ================================================
>>>>>
>>>>> The debug console extension defines a generic mechanism for boot-time
>>>>> early prints from supervisor-mode software which allows users to catch
>>>>> boot-time issues in supervisor-mode software.
>>>>>
>>>>> This extension replaces legacy console putchar (EID #0x01) extension
>>>>> and it is better in following ways:
>>>>
>>>> Thanks for starting to close this gap.
>>>>
>>>>> 1) It follows the new calling convention defined for SBI v1.0
>>>>>       (or higher).
>>>>> 2) It is based on a shared memory area between SBI implementation
>>>>>       and supervisor-mode software so multiple characters can be
>>>>>       printed using a single SBI call.
>>>>
>>>> I miss a discussion of the conflicts that can arise if the configuration
>>>> of the serial console is changed by the caller.
>>>>
>>>> Do we need an ecall that closes the SBI console to further access?
>>>
>>> Usually, the serial port related code in M-mode firmware only uses
>>> status and data registers so for most serial ports support the M-mode
>>> firmware will adapt to serial port configuration changes.
>>>
>>> In fact, this is why we never had a special SBI call for serial port
>>> reconfiguration in legacy SBI v0.1 as well.
>>>
>>> In case of virtualization, the serial port (or console) is emulated so
>>> the special SBI call is not useful for virtualization.
>>>
>>>>
>>>>>
>>>>> The supervisor-mode software must set the shared memory area before
>>>>> printing characters on the debug console. Also, all HARTs share the
>>>>> same shared memory area so only one HART needs to set it at boot-time.
>>>>
>>>> Isn't it M-mode software that has to program the MMU to allow all harts
>>>> in M-mode and S-mode access to the memory area? What is the S-mode
>>>> software to do about the memory area prior to calling
>>>> sbi_debug_console_set_area()?
>>>
>>> Actually, it's the S-mode software which is voluntarily giving a portion of
>>> its memory to be used as shared memory. The proposal only mandates
>>> that whatever memory is selected by S-mode software should be a
>>> regular cacheable memory (not IO memory). Also, if Svpbmt is available
>>> then S-mode software should only use memory type 0 in the PTEs.
>>>
>>>>
>>>>>
>>>>> Function: Set Console Area (FID #0)
>>>>> -----------------------------------
>>>>>
>>>>> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
>>>>>                                             unsigned long size)
>>>>
>>>> The console output is needed on the very start of the S-mode software,
>>>> before setting up anything.
>>>>
>>>> Can we avoid this extra function?
>>>>
>>>> Can we simply assume that the caller of sbi_debug_console_puts()
>>>> provides a physical address pointer to a memory area that is suitable?
>>>
>>> Theoretically, we can avoid the extra function to set shared memory area
>>> but it will complicate things in future when we have supervisor software
>>> encrypting it's own memory (using special ISA support) because in this
>>> case supervisor software will have to unprotect memory every time the
>>> sbi_debug_console_puts() is called and protect it again after the call.
>>
>> Currently this function is just a nop(). It is not needed in this
>> revision of the extension.
>>
>> The function might be called repeatedly by different threads with
>> different values. How do you want to keep track of all of these
>> different areas?
>>
>> Memory shared between different security realms will arise in many
>> different scenarios. As this is not console specific it should be in a
>> separate extension. That extension should be defined once we have
>> clarity about how security realms are managed.
>>
> 
> I would like to understand more about the security concerns you raised.
> Can you please elaborate on this ?

I have no security concerns. My thought was that it might be better to 
manage shared memory in a separate extension for all use cases.

Best regards

Heinrich

> 
> As you pointed out, there are other possible use cases(e.g STA, PMU)
> for shared memory between
> S & M mode or VS & HS mode. It would be good to address these concerns right now
> instead of discussing those in each individual extension context.
> 
>>>
>>>>
>>>>>
>>>>> Set the shared memory area specified by `addr_div_by_2` and `size`
>>>>
>>>> %s/addr_div_by_2/addr_div_by_4/
>>>
>>> Okay, I will update.
>>>
>>>>
>>>>> parameters. The `addr_div_by_4` parameter is base address of the
>>>>
>>>> %s/is base/is the base/
>>>
>>> Okay, I will update.
>>>
>>>>
>>>>> shared memory area right shifted by 2 whereas `size` parameter is
>>>>> the size of shared memory area in bytes.
>>>>
>>>> Why shifting the address? I would prefer to keep it simple for the
>>>> caller. If the alignment is not suitable, return an error.
>>>>
>>>> But why is an alignment needed here at all? And why 4 aligned?
>>>
>>> For RV32 S-mode, the physical address space is 34bits wide but
>>> "unsigned long" is 32bits wide. This is because Sv32 PTEs allow
>>> 34bits of PPN. In fact, even instructions such as HFENCE.GVMA
>>> have this "address right shift by 2" requirement based on this rationale.
>>>
>>>>
>>>>>
>>>>> The shared memory area should be normal cacheable memory for the
>>>>> supervisor-mode software. Also, the shared memory area is global
>>>>> across all HARTs so SBI implementation must ensure atomicity in
>>>>> setting the shared memory area.
>>>>>
>>>>> Errors:
>>>>> SBI_SUCCESS                - Shared memory area set successfully.
>>>>> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
>>>>>                              `addr_div_by_2` and `size` parameters
>>>>>                              is not normal cacheable memory or not
>>>>>                              accessible to supervisor-mode software.
>>>>>
>>>>> Function: Console Puts (FID #1)
>>>>> -------------------------------
>>>>>
>>>>> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
>>>>>                                         unsigned long num_chars)
>>>>
>>>> I would prefer to simply pass a physical address pointer here with no
>>>> requirements on alignment. And no prior SBI call.
>>
>> sbi_debug_console_set_area() might be called with different values by
>> different threads. An offset is ambiguous as it does not define to which
>> of the different shared areas it relates. Please, use a pointer.
>>
>>>>
>>>> Do we need num_chars? Are we expecting to provide binary output? Using
>>>> 0x00 as terminator would be adequate in most cases.
>>>
>>> Bare-metal tests (or assembly sources) can print sub-strings from
>>> a large per-populated string in shared memory. Assuming that string
>>> is always terminated by 0x00 in sbi_debug_console_puts() will break
>>> this flexibility.
>>
>> OK
>>
>>>
>>>>
>>>> What is the requirement on the console? Does it have to support 8bit
>>>> output to allow for UTF-8?
>>>
>>> We need to clarify this. Suggestions ?
>>
>> The platform specification would be the right place to require 8-bit
>> support for the console.
>>
>>>
>>>>
>>>> Do we make any assumptions about encoding?
>>>
>>> Same as above, this needs more clarification. Suggestions ?
>>
>> We should add to the platform specification that UTF-8 output is assumed
>> on the serial console.
>>
>>>
>>> I am of the opinion to keep such encoding related assumptions to be
>>> minimal.
>>>
>>>>
>>>> How would we handle a console set up to 7bit + parity if a character >
>>>> 0x7f is sent?
>>>
>>> I would consider this to be part of the clarification we add for encoding.
>>
>> We should state if extra bits are ignored or the bytes are not send.
>> The easiest thing is to just ignore the extra bits. So let't state this
>> here.
>>
>> Best regards
>>
>> Heinrich
>>
>>>
>>>>
>>>>>
>>>>> Print the string specified by `area_offset` and `num_chars` on
>>>>> the debug console. The `area_offset` parameter is the start of
>>>>> string in the shard memory area whereas `num_chars` parameter
>>>>
>>>> %s/shard/shared/
>>>
>>> Okay, I will update.
>>>
>>>>
>>>>> is the number of characters (or bytes) in the string.
>>>>>
>>>>> This is a blocking SBI call and will only return after printing
>>>>> all characters of the string.
>>>>>
>>>>> Errors:
>>>>> SBI_SUCCESS                - Characters printed successfully.
>>>>> SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
>>>>>                              `area_offset`) or end of the string
>>>>>                              (i.e. `area_offset + num_chars`) is
>>>>>                              outside shared memory area.
>>>>
>>>> There could be other reasons of failures:
>>>>
>>>> * set up of the UART failed in OpenSBI
>>>> * no UART defined in the device-tree
>>>> * ...
>>>>
>>>> So let us add SBI_ERR_FAILED to the list.
>>>
>>> Okay, I will add.
>>>
>>>>
>>>> Best regards
>>>>
>>>> Heinrich
>>>
>>> Regards,
>>> Anup
>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#1717): https://lists.riscv.org/g/tech-unixplatformspec/message/1717
>> Mute This Topic: https://lists.riscv.org/mt/91480122/1774178
>> Group Owner: tech-unixplatformspec+owner at lists.riscv.org
>> Unsubscribe: https://lists.riscv.org/g/tech-unixplatformspec/unsub [atishp at rivosinc.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>



From atishp at rivosinc.com  Fri Jun  3 00:33:27 2022
From: atishp at rivosinc.com (Atish Kumar Patra)
Date: Fri, 3 Jun 2022 00:33:27 -0700
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <16ce4a1f-9a88-de85-d1d1-05cc5a542366@canonical.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <72af98ee-6bb4-542f-021d-66e9dd946226@canonical.com> <CAK9=C2WLZqkYsgZL5WCJGjWGK=YGV3f51d8yvD3ecx+JPURgpw@mail.gmail.com>
 <307d7e4e-62f8-8dec-8b09-ea69ca2c29d9@canonical.com> <CAHBxVyEDR-16-nEd0rRCvegfAxPf2xSMB6HZpe3269GjmVUzQw@mail.gmail.com>
 <16ce4a1f-9a88-de85-d1d1-05cc5a542366@canonical.com>
Message-ID: <CAHBxVyFoDOFC5z1i2+C7LDUJXhRD8+dQcdYyw8xYcnX=2P6TOA@mail.gmail.com>

On Fri, Jun 3, 2022 at 12:23 AM Heinrich Schuchardt
<heinrich.schuchardt at canonical.com> wrote:
>
> On 6/3/22 08:56, Atish Kumar Patra wrote:
> > On Thu, Jun 2, 2022 at 2:13 AM Heinrich Schuchardt
> > <heinrich.schuchardt at canonical.com> wrote:
> >>
> >> On 6/2/22 10:44, Anup Patel wrote:
> >>> On Wed, Jun 1, 2022 at 11:59 PM Heinrich Schuchardt
> >>> <heinrich.schuchardt at canonical.com> wrote:
> >>>>
> >>>> On 6/1/22 18:17, Anup Patel wrote:
> >>>>> Hi All,
> >>>>>
> >>>>> Below is the draft proposal for SBI Debug Console Extension.
> >>>>>
> >>>>> Please review it and provide feedback.
> >>>>>
> >>>>> Thanks,
> >>>>> Anup
> >>>>>
> >>>>> Debug Console Extension (EID #0x4442434E "DBCN")
> >>>>> ================================================
> >>>>>
> >>>>> The debug console extension defines a generic mechanism for boot-time
> >>>>> early prints from supervisor-mode software which allows users to catch
> >>>>> boot-time issues in supervisor-mode software.
> >>>>>
> >>>>> This extension replaces legacy console putchar (EID #0x01) extension
> >>>>> and it is better in following ways:
> >>>>
> >>>> Thanks for starting to close this gap.
> >>>>
> >>>>> 1) It follows the new calling convention defined for SBI v1.0
> >>>>>       (or higher).
> >>>>> 2) It is based on a shared memory area between SBI implementation
> >>>>>       and supervisor-mode software so multiple characters can be
> >>>>>       printed using a single SBI call.
> >>>>
> >>>> I miss a discussion of the conflicts that can arise if the configuration
> >>>> of the serial console is changed by the caller.
> >>>>
> >>>> Do we need an ecall that closes the SBI console to further access?
> >>>
> >>> Usually, the serial port related code in M-mode firmware only uses
> >>> status and data registers so for most serial ports support the M-mode
> >>> firmware will adapt to serial port configuration changes.
> >>>
> >>> In fact, this is why we never had a special SBI call for serial port
> >>> reconfiguration in legacy SBI v0.1 as well.
> >>>
> >>> In case of virtualization, the serial port (or console) is emulated so
> >>> the special SBI call is not useful for virtualization.
> >>>
> >>>>
> >>>>>
> >>>>> The supervisor-mode software must set the shared memory area before
> >>>>> printing characters on the debug console. Also, all HARTs share the
> >>>>> same shared memory area so only one HART needs to set it at boot-time.
> >>>>
> >>>> Isn't it M-mode software that has to program the MMU to allow all harts
> >>>> in M-mode and S-mode access to the memory area? What is the S-mode
> >>>> software to do about the memory area prior to calling
> >>>> sbi_debug_console_set_area()?
> >>>
> >>> Actually, it's the S-mode software which is voluntarily giving a portion of
> >>> its memory to be used as shared memory. The proposal only mandates
> >>> that whatever memory is selected by S-mode software should be a
> >>> regular cacheable memory (not IO memory). Also, if Svpbmt is available
> >>> then S-mode software should only use memory type 0 in the PTEs.
> >>>
> >>>>
> >>>>>
> >>>>> Function: Set Console Area (FID #0)
> >>>>> -----------------------------------
> >>>>>
> >>>>> struct sbiret sbi_debug_console_set_area(unsigned long addr_div_by_4,
> >>>>>                                             unsigned long size)
> >>>>
> >>>> The console output is needed on the very start of the S-mode software,
> >>>> before setting up anything.
> >>>>
> >>>> Can we avoid this extra function?
> >>>>
> >>>> Can we simply assume that the caller of sbi_debug_console_puts()
> >>>> provides a physical address pointer to a memory area that is suitable?
> >>>
> >>> Theoretically, we can avoid the extra function to set shared memory area
> >>> but it will complicate things in future when we have supervisor software
> >>> encrypting it's own memory (using special ISA support) because in this
> >>> case supervisor software will have to unprotect memory every time the
> >>> sbi_debug_console_puts() is called and protect it again after the call.
> >>
> >> Currently this function is just a nop(). It is not needed in this
> >> revision of the extension.
> >>
> >> The function might be called repeatedly by different threads with
> >> different values. How do you want to keep track of all of these
> >> different areas?
> >>
> >> Memory shared between different security realms will arise in many
> >> different scenarios. As this is not console specific it should be in a
> >> separate extension. That extension should be defined once we have
> >> clarity about how security realms are managed.
> >>
> >
> > I would like to understand more about the security concerns you raised.
> > Can you please elaborate on this ?
>
> I have no security concerns. My thought was that it might be better to
> manage shared memory in a separate extension for all use cases.
>

Ahh I see. Sorry I misunderstood your comment.
The designated shared memory for debug console is shared across
the platforms while STA or PMU use case will have per cpu shared memory.

I agree that all the concerns related to shared memory can be discussed
together and all the common constraints can be described at one place.

But defining a separate extension for the shared memory may be problematic
as we need to manage the extension dependencies.

> Best regards
>
> Heinrich
>
> >
> > As you pointed out, there are other possible use cases(e.g STA, PMU)
> > for shared memory between
> > S & M mode or VS & HS mode. It would be good to address these concerns right now
> > instead of discussing those in each individual extension context.
> >
> >>>
> >>>>
> >>>>>
> >>>>> Set the shared memory area specified by `addr_div_by_2` and `size`
> >>>>
> >>>> %s/addr_div_by_2/addr_div_by_4/
> >>>
> >>> Okay, I will update.
> >>>
> >>>>
> >>>>> parameters. The `addr_div_by_4` parameter is base address of the
> >>>>
> >>>> %s/is base/is the base/
> >>>
> >>> Okay, I will update.
> >>>
> >>>>
> >>>>> shared memory area right shifted by 2 whereas `size` parameter is
> >>>>> the size of shared memory area in bytes.
> >>>>
> >>>> Why shifting the address? I would prefer to keep it simple for the
> >>>> caller. If the alignment is not suitable, return an error.
> >>>>
> >>>> But why is an alignment needed here at all? And why 4 aligned?
> >>>
> >>> For RV32 S-mode, the physical address space is 34bits wide but
> >>> "unsigned long" is 32bits wide. This is because Sv32 PTEs allow
> >>> 34bits of PPN. In fact, even instructions such as HFENCE.GVMA
> >>> have this "address right shift by 2" requirement based on this rationale.
> >>>
> >>>>
> >>>>>
> >>>>> The shared memory area should be normal cacheable memory for the
> >>>>> supervisor-mode software. Also, the shared memory area is global
> >>>>> across all HARTs so SBI implementation must ensure atomicity in
> >>>>> setting the shared memory area.
> >>>>>
> >>>>> Errors:
> >>>>> SBI_SUCCESS                - Shared memory area set successfully.
> >>>>> SBI_ERR_INVALID_ADDRESS - The shared memory area pointed by
> >>>>>                              `addr_div_by_2` and `size` parameters
> >>>>>                              is not normal cacheable memory or not
> >>>>>                              accessible to supervisor-mode software.
> >>>>>
> >>>>> Function: Console Puts (FID #1)
> >>>>> -------------------------------
> >>>>>
> >>>>> struct sbiret sbi_debug_console_puts(unsigned long area_offset,
> >>>>>                                         unsigned long num_chars)
> >>>>
> >>>> I would prefer to simply pass a physical address pointer here with no
> >>>> requirements on alignment. And no prior SBI call.
> >>
> >> sbi_debug_console_set_area() might be called with different values by
> >> different threads. An offset is ambiguous as it does not define to which
> >> of the different shared areas it relates. Please, use a pointer.
> >>
> >>>>
> >>>> Do we need num_chars? Are we expecting to provide binary output? Using
> >>>> 0x00 as terminator would be adequate in most cases.
> >>>
> >>> Bare-metal tests (or assembly sources) can print sub-strings from
> >>> a large per-populated string in shared memory. Assuming that string
> >>> is always terminated by 0x00 in sbi_debug_console_puts() will break
> >>> this flexibility.
> >>
> >> OK
> >>
> >>>
> >>>>
> >>>> What is the requirement on the console? Does it have to support 8bit
> >>>> output to allow for UTF-8?
> >>>
> >>> We need to clarify this. Suggestions ?
> >>
> >> The platform specification would be the right place to require 8-bit
> >> support for the console.
> >>
> >>>
> >>>>
> >>>> Do we make any assumptions about encoding?
> >>>
> >>> Same as above, this needs more clarification. Suggestions ?
> >>
> >> We should add to the platform specification that UTF-8 output is assumed
> >> on the serial console.
> >>
> >>>
> >>> I am of the opinion to keep such encoding related assumptions to be
> >>> minimal.
> >>>
> >>>>
> >>>> How would we handle a console set up to 7bit + parity if a character >
> >>>> 0x7f is sent?
> >>>
> >>> I would consider this to be part of the clarification we add for encoding.
> >>
> >> We should state if extra bits are ignored or the bytes are not send.
> >> The easiest thing is to just ignore the extra bits. So let't state this
> >> here.
> >>
> >> Best regards
> >>
> >> Heinrich
> >>
> >>>
> >>>>
> >>>>>
> >>>>> Print the string specified by `area_offset` and `num_chars` on
> >>>>> the debug console. The `area_offset` parameter is the start of
> >>>>> string in the shard memory area whereas `num_chars` parameter
> >>>>
> >>>> %s/shard/shared/
> >>>
> >>> Okay, I will update.
> >>>
> >>>>
> >>>>> is the number of characters (or bytes) in the string.
> >>>>>
> >>>>> This is a blocking SBI call and will only return after printing
> >>>>> all characters of the string.
> >>>>>
> >>>>> Errors:
> >>>>> SBI_SUCCESS                - Characters printed successfully.
> >>>>> SBI_ERR_INVALID_ADDRESS    - The start of the string (i.e.
> >>>>>                              `area_offset`) or end of the string
> >>>>>                              (i.e. `area_offset + num_chars`) is
> >>>>>                              outside shared memory area.
> >>>>
> >>>> There could be other reasons of failures:
> >>>>
> >>>> * set up of the UART failed in OpenSBI
> >>>> * no UART defined in the device-tree
> >>>> * ...
> >>>>
> >>>> So let us add SBI_ERR_FAILED to the list.
> >>>
> >>> Okay, I will add.
> >>>
> >>>>
> >>>> Best regards
> >>>>
> >>>> Heinrich
> >>>
> >>> Regards,
> >>> Anup
> >>
> >>
> >>
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >> Links: You receive all messages sent to this group.
> >> View/Reply Online (#1717): https://lists.riscv.org/g/tech-unixplatformspec/message/1717
> >> Mute This Topic: https://lists.riscv.org/mt/91480122/1774178
> >> Group Owner: tech-unixplatformspec+owner at lists.riscv.org
> >> Unsubscribe: https://lists.riscv.org/g/tech-unixplatformspec/unsub [atishp at rivosinc.com]
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >>
> >>
>


From konrad.schwarz at siemens.com  Fri Jun  3 00:49:04 2022
From: konrad.schwarz at siemens.com (Schwarz, Konrad)
Date: Fri, 3 Jun 2022 07:49:04 +0000
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <CAK9=C2XbTQysZghPa393A7R=y8OhD2Hyg_4iqxNsgJBAGj4O7Q@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
 <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego>
 <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
 <YpjQkMhoKuoCj/Do@ved-XPS-8940>
 <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
 <YpkXHYwhz8efa6ON@ved-XPS-8940>
 <alpine.DEB.2.22.394.2206021327250.2783803@ubuntu-linux-20-04-desktop>
 <CA+Qh7TnDUN4kBKJqkeSQoypTm71Sz8QhzqMSE9xVDrre_gGt8w@mail.gmail.com>
 <alpine.DEB.2.22.394.2206021354290.2783803@ubuntu-linux-20-04-desktop>
 <CAK9=C2XbTQysZghPa393A7R=y8OhD2Hyg_4iqxNsgJBAGj4O7Q@mail.gmail.com>
Message-ID: <2f93033e0b07411e92384c00722eb742@siemens.com>


> From: sig-hypervisors at lists.riscv.org <sig-hypervisors at lists.riscv.org> On Behalf Of Anup Patel via
> lists.riscv.org
> Sent: Friday, June 3, 2022 5:01

> We have legacy SBI v0.1 calls where most of them are replaced by
> newer SBI v0.2 (or higher) calls. Only the SBI v0.1 putchar() does not
> have a replacement. This putchar() was being used in various cases
> for early prints from OSes and bootloaders.
> 
> The SBI debug console extension is an attempt to replace the legacy
> SBI v0.1 putchar(). The use of shared memory in this proposal is
> motivated by the need to allow users to print multiple characters in
> one SBI call.

As I pointed out in an earlier message, the design using a shared memory block
is a hindrance.  Instead, each call should provide a pointer to the (virtual)
address of the character buffer.

> The legacy SBI v0.1 also had a getchar() call which is deprecated and
> does not need any newer SBI call replacing it because it is expected
> all platforms (including virtual platforms emulated by hypervisors) will
> have a proper interrupt driver console for supervisor software. The
> proposed SBI debug console extension only tries to provide a solution
> for early prints.

Consider the case where we have more VMs than physical UARTs -- this
will actually be the norm in most deployments.

(To counter the argument that the hypervisor can provide virtual UARTs
for its guests, note that a dedicated para-virtualized interface is much more
efficient than trapping individual memory accesses to simulated HW registers).

By making the interface a bit richer, as discussed in my earlier proposal,
a hypervisor can cover a much larger set of use cases. 


-- 
Konrad

From Alistair.Francis at wdc.com  Fri Jun  3 01:47:02 2022
From: Alistair.Francis at wdc.com (Alistair Francis)
Date: Fri, 3 Jun 2022 08:47:02 +0000
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <2f93033e0b07411e92384c00722eb742@siemens.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
	 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
	 <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego>
	 <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
	 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
	 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
	 <YpjQkMhoKuoCj/Do@ved-XPS-8940>
	 <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
	 <YpkXHYwhz8efa6ON@ved-XPS-8940>
	 <alpine.DEB.2.22.394.2206021327250.2783803@ubuntu-linux-20-04-desktop>
	 <CA+Qh7TnDUN4kBKJqkeSQoypTm71Sz8QhzqMSE9xVDrre_gGt8w@mail.gmail.com>
	 <alpine.DEB.2.22.394.2206021354290.2783803@ubuntu-linux-20-04-desktop>
	 <CAK9=C2XbTQysZghPa393A7R=y8OhD2Hyg_4iqxNsgJBAGj4O7Q@mail.gmail.com>
	 <2f93033e0b07411e92384c00722eb742@siemens.com>
Message-ID: <bd57d6943d48cf9c7615ec1c043a323474e479ff.camel@wdc.com>

On Fri, 2022-06-03 at 07:49 +0000, Schwarz, Konrad wrote:
> 
> > From:
> > sig-hypervisors at lists.riscv.org?<sig-hypervisors at lists.riscv.org>
> > On Behalf Of Anup Patel via
> > lists.riscv.org
> > Sent: Friday, June 3, 2022 5:01
> 
> > We have legacy SBI v0.1 calls where most of them are replaced by
> > newer SBI v0.2 (or higher) calls. Only the SBI v0.1 putchar() does
> > not
> > have a replacement. This putchar() was being used in various cases
> > for early prints from OSes and bootloaders.
> > 
> > The SBI debug console extension is an attempt to replace the legacy
> > SBI v0.1 putchar(). The use of shared memory in this proposal is
> > motivated by the need to allow users to print multiple characters
> > in
> > one SBI call.
> 
> As I pointed out in an earlier message, the design using a shared
> memory block
> is a hindrance.? Instead, each call should provide a pointer to the
> (virtual)
> address of the character buffer.

I agree

With the current design there is no way for the supervisor software to
reclaim the memory once it has shared it with OpenSBI. We can add a
close() call, but it seems simpler and safer to instead just pass an
address and size to print.

> 
> > The legacy SBI v0.1 also had a getchar() call which is deprecated
> > and
> > does not need any newer SBI call replacing it because it is
> > expected
> > all platforms (including virtual platforms emulated by hypervisors)
> > will
> > have a proper interrupt driver console for supervisor software. The
> > proposed SBI debug console extension only tries to provide a
> > solution
> > for early prints.
> 
> Consider the case where we have more VMs than physical UARTs -- this
> will actually be the norm in most deployments.

In this case you should use virtIO or some other more powerful
interface

> 
> (To counter the argument that the hypervisor can provide virtual
> UARTs
> for its guests, note that a dedicated para-virtualized interface is
> much more
> efficient than trapping individual memory accesses to simulated HW
> registers).

Agreed, but tha seems up to the Hypervisor to implement an effient
virtual UART implementation. For example the Xen HYPERVISOR_console_io
that Stefano mentioned earlier. Plus then we can leverage existing
implementations insted of inveting a another virtual serial device.

> 
> By making the interface a bit richer, as discussed in my earlier
> proposal,
> a hypervisor can cover a much larger set of use cases. 

Which we also then need to support in M-mode firmware. The more complex
the firmware becomes the more exposed it is.

Alistair

> 
> 
> -- 
> Konrad
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#1747):
> https://lists.riscv.org/g/tech-unixplatformspec/message/1747
> Mute This Topic: https://lists.riscv.org/mt/91507495/1782774
> Group Owner: tech-unixplatformspec+owner at lists.riscv.org
> Unsubscribe:
> https://lists.riscv.org/g/tech-unixplatformspec/unsub?[alistair.francis at wdc.com
> ]
> -=-=-=-=-=-=-=-=-=-=-=-
> 
> 


From konrad.schwarz at siemens.com  Sun Jun  5 07:56:17 2022
From: konrad.schwarz at siemens.com (Schwarz, Konrad)
Date: Sun, 5 Jun 2022 14:56:17 +0000
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <bd57d6943d48cf9c7615ec1c043a323474e479ff.camel@wdc.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
	 <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com>
	 <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego>
	 <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
	 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com>
	 <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com>
	 <YpjQkMhoKuoCj/Do@ved-XPS-8940>
	 <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
	 <YpkXHYwhz8efa6ON@ved-XPS-8940>
	 <alpine.DEB.2.22.394.2206021327250.2783803@ubuntu-linux-20-04-desktop>
	 <CA+Qh7TnDUN4kBKJqkeSQoypTm71Sz8QhzqMSE9xVDrre_gGt8w@mail.gmail.com>
	 <alpine.DEB.2.22.394.2206021354290.2783803@ubuntu-linux-20-04-desktop>
	 <CAK9=C2XbTQysZghPa393A7R=y8OhD2Hyg_4iqxNsgJBAGj4O7Q@mail.gmail.com>
	 <2f93033e0b07411e92384c00722eb742@siemens.com>
 <bd57d6943d48cf9c7615ec1c043a323474e479ff.camel@wdc.com>
Message-ID: <d38afd171c164548a4f66e85fa1b9e50@siemens.com>




> -----Original Message-----
> From: Alistair Francis <Alistair.Francis at wdc.com>
> Sent: Friday, June 3, 2022 10:47
> > Consider the case where we have more VMs than physical UARTs -- this
> > will actually be the norm in most deployments.
> 
> In this case you should use virtIO or some other more powerful
> interface
> 
> >
> > (To counter the argument that the hypervisor can provide virtual
> > UARTs
> > for its guests, note that a dedicated para-virtualized interface is
> > much more
> > efficient than trapping individual memory accesses to simulated HW
> > registers).
> 
> Agreed, but tha seems up to the Hypervisor to implement an effient
> virtual UART implementation. For example the Xen HYPERVISOR_console_io
> that Stefano mentioned earlier. Plus then we can leverage existing
> implementations insted of inveting a another virtual serial device.

But this makes it incumbent on each hypervisor to provide such an
interface, and requires each OS to provide drivers for each
hypervisor it knows about.  You then have an N:M situation,
which is bad for the RISC-V platform.

> > By making the interface a bit richer, as discussed in my earlier
> > proposal,
> > a hypervisor can cover a much larger set of use cases.
> 
> Which we also then need to support in M-mode firmware. The more complex
> the firmware becomes the more exposed it is.

The same argument applies to hypervisors. What fundamental difference
between M-mode and HS-mode software is there that makes the risk
so much different?

An implementation wishing to minimize its responsibilities can
always return run-time errors for functionality it does not
wish to support.

-- 
Konrad

From darius at bluespec.com  Mon Jun  6 03:50:02 2022
From: darius at bluespec.com (Darius Rad)
Date: Mon, 6 Jun 2022 06:50:02 -0400
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v1)
In-Reply-To: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com>
Message-ID: <Yp3b2pjG/CgLV2Za@bruce.bluespec.com>

On Wed, Jun 01, 2022 at 09:47:32PM +0530, Anup Patel wrote:
> 
> Debug Console Extension (EID #0x4442434E "DBCN")
> ================================================
> 
> The debug console extension defines a generic mechanism for boot-time
> early prints from supervisor-mode software which allows users to catch
> boot-time issues in supervisor-mode software.
> 
> This extension replaces legacy console putchar (EID #0x01) extension
> and it is better in following ways:
> 1) It follows the new calling convention defined for SBI v1.0
>    (or higher).
> 2) It is based on a shared memory area between SBI implementation
>    and supervisor-mode software so multiple characters can be
>    printed using a single SBI call.
> 
> The supervisor-mode software must set the shared memory area before
> printing characters on the debug console. Also, all HARTs share the
> same shared memory area so only one HART needs to set it at boot-time.
> 

It seems to me that the rationale and summary of the extension is
insufficient, and non-normative commentary is lacking.  The ensuing mailing
list discussion suggests that there are additional requirements and
assumptions that influenced the design of the interface, but are not
mentioned here.  It is important to document such information, in order to
have a constructive discussion now, and also to preserve such information
for readers in the future who may not have the benefit of reading the
mailing list discussions.

// darius



From stefano.stabellini at xilinx.com  Thu Jun  2 14:05:47 2022
From: stefano.stabellini at xilinx.com (Stefano Stabellini)
Date: Thu, 2 Jun 2022 14:05:47 -0700
Subject: [sig-hypervisors] [RISC-V] [tech-unixplatformspec] SBI Debug
 Console Extension Proposal (Draft v1)
In-Reply-To: <CA+Qh7TnDUN4kBKJqkeSQoypTm71Sz8QhzqMSE9xVDrre_gGt8w@mail.gmail.com>
References: <CAK9=C2UpWA1x6DT8jo8hxvsnPJVLPAUMDTzhY5bYayAENVHdDg@mail.gmail.com> <CAK9=C2XXu9JpdsTLFiamQkgj_P6titwAohPdgwWUCm-xN8A2kw@mail.gmail.com> <5091667.6fTUFtlzNn@diego> <3070891.fEcJ0Lxnt5@diego> <CAK9=C2WETO-XJxTg1_jCLWaxkMJ8ubKkpeg3xKNnUvtJicrjyA@mail.gmail.com>
 <CAG5apL1PfuCoYOVChaXO2wp93RnUifuxQZq1xebpNvH=BnXxrA@mail.gmail.com> <CAK9=C2XB3wBu0agiCfKZzN57waK6-0sq1emjwb2DG5m9V634xw@mail.gmail.com> <YpjQkMhoKuoCj/Do@ved-XPS-8940> <alpine.DEB.2.22.394.2206021240390.2783803@ubuntu-linux-20-04-desktop>
 <YpkXHYwhz8efa6ON@ved-XPS-8940> <alpine.DEB.2.22.394.2206021327250.2783803@ubuntu-linux-20-04-desktop> <CA+Qh7TnDUN4kBKJqkeSQoypTm71Sz8QhzqMSE9xVDrre_gGt8w@mail.gmail.com>
Message-ID: <alpine.DEB.2.22.394.2206021354290.2783803@ubuntu-linux-20-04-desktop>

On Thu, 2 Jun 2022, Greg Favor wrote:
> On Thu, Jun 2, 2022 at 1:29 PM Stefano Stabellini <stefano.stabellini at xilinx.com> wrote:
>       On Thu, 2 Jun 2022, Vedvyas Shanbhogue via lists.riscv.org wrote:
>       > Of course a console is needed. But I was questioning the need for something much
>       > more elaborate than a putchar/getchar interface. I understand its needed to port
>       > the hypervisor but I undersatnd it would be needed for very early parts of the
>       > hypervisor where it does not even have the ability to master a uart on its own.
> 
>       Yeah, I think you are right that it is not a must-have feature but a
>       nice-to-have feature.
> 
> 
> Even though we all (me included) tend to think that slowing down something like this will be inconsequential to the time to boot and/or who
> cares how fast or slow is boot?time, in the past I have found that to be a risky presumption in certain production environments where boot
> time matters (especially with frequent standing up of VMs from scratch).? So a worthwhile sanity check question would be 1) what percentage
> of early boot time is spent writing to the console with per-string SBI calls, and 2) by what order of magnitude does that percentage
> inflate due to SBI calls for every individual character.

The numbers you asked will help but I just wanted to add that in my
opinion a per-character putchar interface is not suitable for production
due to the reasons you mentioned. I would compile it out in non-DEBUG
builds.

On the other hand the per-string print interface might be usable in
production, even not just early boot. However, of course, a proper UART
driver would be even better.

This is why I think it is nice to have: it expands the potential uses of
the SBI console interface, and it is simple and easy to use. At the same
time it is not a must-have because you could get away with just putchat
in early boot for DEBUG builds, and nothing + UART driver in non-DEBUG
builds.

That is why I am in favor of this addition, although I recognize it is
not critical.

From dramforever at live.com  Wed Jun  8 02:30:52 2022
From: dramforever at live.com (dramforever)
Date: Wed,  8 Jun 2022 17:30:52 +0800
Subject: [PATCH] lib: sbi_trap: Redirect exception based on hedeleg
Message-ID: <TYYP286MB14397827A57ED2BA6AA100E3C6A49@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>

HS-mode software can choose what exceptions to delegate to VS-mode using
the hedeleg CSR. Synthetic VS/VU-mode exceptions should also honor
hedeleg. They should be redirected to VS-mode if and only if delegated
by HS-mode.

Signed-off-by: dramforever <dramforever at live.com>
---
 lib/sbi/sbi_trap.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/lib/sbi/sbi_trap.c b/lib/sbi/sbi_trap.c
index 2c509e5..effd456 100644
--- a/lib/sbi/sbi_trap.c
+++ b/lib/sbi/sbi_trap.c
@@ -99,17 +99,14 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
 	if (prev_mode != PRV_S && prev_mode != PRV_U)
 		return SBI_ENOTSUPP;
 
-	/* For certain exceptions from VS/VU-mode we redirect to VS-mode */
+	/* If exceptions came from VS/VU-mode, redirect to VS-mode if
+	 * delegated in hedeleg
+	 */
 	if (misa_extension('H') && prev_virt) {
-		switch (trap->cause) {
-		case CAUSE_FETCH_PAGE_FAULT:
-		case CAUSE_LOAD_PAGE_FAULT:
-		case CAUSE_STORE_PAGE_FAULT:
+		if (trap->cause < __riscv_xlen &&
+		    csr_read(CSR_HEDELEG) & (1 << trap->cause)) {
 			next_virt = TRUE;
-			break;
-		default:
-			break;
-		};
+		}
 	}
 
 	/* Update MSTATUS MPV bits */
-- 
2.36.0



From wxjstz at 126.com  Wed Jun  8 21:05:55 2022
From: wxjstz at 126.com (Xiang W)
Date: Thu, 09 Jun 2022 12:05:55 +0800
Subject: [PATCH] lib: sbi_trap: Redirect exception based on hedeleg
In-Reply-To: <TYYP286MB14397827A57ED2BA6AA100E3C6A49@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB14397827A57ED2BA6AA100E3C6A49@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <0332b680c44c1f02e0c7d1bca5ab9c55d1c917af.camel@126.com>

? 2022-06-08???? 17:30 +0800?dramforever???
> HS-mode software can choose what exceptions to delegate to VS-mode using
> the hedeleg CSR. Synthetic VS/VU-mode exceptions should also honor
> hedeleg. They should be redirected to VS-mode if and only if delegated
> by HS-mode.
> 
> Signed-off-by: dramforever <dramforever at live.com>
look good to me.

Reviewed-by: Xiang W <wxjstz at 126.com>
> ---
> ?lib/sbi/sbi_trap.c | 15 ++++++---------
> ?1 file changed, 6 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/sbi/sbi_trap.c b/lib/sbi/sbi_trap.c
> index 2c509e5..effd456 100644
> --- a/lib/sbi/sbi_trap.c
> +++ b/lib/sbi/sbi_trap.c
> @@ -99,17 +99,14 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
> ????????if (prev_mode != PRV_S && prev_mode != PRV_U)
> ????????????????return SBI_ENOTSUPP;
> ?
> -???????/* For certain exceptions from VS/VU-mode we redirect to VS-mode */
> +???????/* If exceptions came from VS/VU-mode, redirect to VS-mode if
> +??????? * delegated in hedeleg
> +??????? */
> ????????if (misa_extension('H') && prev_virt) {
> -???????????????switch (trap->cause) {
> -???????????????case CAUSE_FETCH_PAGE_FAULT:
> -???????????????case CAUSE_LOAD_PAGE_FAULT:
> -???????????????case CAUSE_STORE_PAGE_FAULT:
> +???????????????if (trap->cause < __riscv_xlen &&
> +?????????????????? csr_read(CSR_HEDELEG) & (1 << trap->cause)) {
> ????????????????????????next_virt = TRUE;
> -???????????????????????break;
> -???????????????default:
> -???????????????????????break;
> -???????????????};
> +???????????????}
> ????????}
> ?
> ????????/* Update MSTATUS MPV bits */
> -- 
> 2.36.0
> 
> 




From dramforever at live.com  Thu Jun  9 00:06:29 2022
From: dramforever at live.com (dramforever)
Date: Thu,  9 Jun 2022 15:06:29 +0800
Subject: [PATCH 0/3] Fixes for tval/tinst when using sbi_trap_redirect()
Message-ID: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>

I went through usages of sbi_trap_redirect() and saw some cases where
after redirecting, stval/htinst could be set to incorrect values. I
tried fixing what I noticed.

Note that these patches are *UNTESTED*.

- For the htinst fixes, no current implementation (QEMU or Spike) of the
  hypervisor extension writes mtinst/htinst to non-zero values, so the
  code paths are unused.
- The sbi_get_insn() fix requires a rare occasion where the instruction
  is fetched normally then unmapped from memory before OpenSBI can load
  it again, and I have not been reproduce.

Please bear in mind when reviewing.

dramforever (3):
  include: sbi: Add mtinst/htinst psuedoinstructions
  lib: sbi: Fixup tinst for exceptions in sbi_misaligned_*()
  lib: sbi: Fix tval and tinst for sbi_get_insn()

 include/sbi/riscv_encoding.h  | 20 ++++++++++++++++++++
 lib/sbi/sbi_misaligned_ldst.c | 16 ++++++++++++++++
 lib/sbi/sbi_unpriv.c          |  8 +++++---
 3 files changed, 41 insertions(+), 3 deletions(-)

-- 
2.36.0



From dramforever at live.com  Thu Jun  9 00:07:29 2022
From: dramforever at live.com (dramforever)
Date: Thu,  9 Jun 2022 15:07:29 +0800
Subject: [PATCH 1/3] include: sbi: Add mtinst/htinst psuedoinstructions
In-Reply-To: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <TYYP286MB14399472C06DD67BB8C55351C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>

Add psuedoinstruction encodings written to mtinst/htinst for faults
caused by implicit memory access for VS-stage address translation

Signed-off-by: dramforever <dramforever at live.com>
---
 include/sbi/riscv_encoding.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
index a164768..2292858 100644
--- a/include/sbi/riscv_encoding.h
+++ b/include/sbi/riscv_encoding.h
@@ -841,6 +841,26 @@
 #define INSN_MASK_FENCE_TSO		0xffffffff
 #define INSN_MATCH_FENCE_TSO		0x8330000f
 
+#if __riscv_xlen == 64
+
+/* 64-bit read for VS-stage address translation (RV64) */
+#define INSN_PSEUDO_VS_LOAD		0x00003000
+
+/* 64-bit write for VS-stage address translation (RV64) */
+#define INSN_PSEUDO_VS_STORE	0x00003020
+
+#elif __riscv_xlen == 32
+
+/* 32-bit read for VS-stage address translation (RV32) */
+#define INSN_PSEUDO_VS_LOAD		0x00002000
+
+/* 32-bit write for VS-stage address translation (RV32) */
+#define INSN_PSEUDO_VS_STORE	0x00002020
+
+#else
+#error "Unexpected __riscv_xlen"
+#endif
+
 #define INSN_16BIT_MASK			0x3
 #define INSN_32BIT_MASK			0x1c
 
-- 
2.36.0



From dramforever at live.com  Thu Jun  9 00:07:30 2022
From: dramforever at live.com (dramforever)
Date: Thu,  9 Jun 2022 15:07:30 +0800
Subject: [PATCH 2/3] lib: sbi: Fixup tinst for exceptions in sbi_misaligned_*()
In-Reply-To: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <TYYP286MB14399AD2F6CFFB0CECD334E9C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>

If there is an exception while emulating a misaligned load/store, fixup
uptrap.tinst before redirecting. Otherwise, HS-mode software may receive
an htinst describing the lbu/sb instruction that faulted during
emulation[1].

[1]: https://github.com/riscv-software-src/opensbi/issues/258

Signed-off-by: dramforever <dramforever at live.com>
---
 lib/sbi/sbi_misaligned_ldst.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/lib/sbi/sbi_misaligned_ldst.c b/lib/sbi/sbi_misaligned_ldst.c
index c879ce7..fd11798 100644
--- a/lib/sbi/sbi_misaligned_ldst.c
+++ b/lib/sbi/sbi_misaligned_ldst.c
@@ -22,6 +22,18 @@ union reg_data {
 	u64 data_u64;
 };
 
+static ulong sbi_misaligned_tinst_fixup(ulong orig_tinst, ulong new_tinst,
+					ulong addr_offset)
+{
+	if (new_tinst == INSN_PSEUDO_VS_LOAD ||
+	    new_tinst == INSN_PSEUDO_VS_STORE)
+		return new_tinst;
+	else if (orig_tinst == 0)
+		return 0UL;
+	else
+		return orig_tinst | (addr_offset << SH_RS1);
+}
+
 int sbi_misaligned_load_handler(ulong addr, ulong tval2, ulong tinst,
 				struct sbi_trap_regs *regs)
 {
@@ -126,6 +138,8 @@ int sbi_misaligned_load_handler(ulong addr, ulong tval2, ulong tinst,
 						&uptrap);
 		if (uptrap.cause) {
 			uptrap.epc = regs->mepc;
+			uptrap.tinst = sbi_misaligned_tinst_fixup(
+				tinst, uptrap.tinst, i);
 			return sbi_trap_redirect(regs, &uptrap);
 		}
 	}
@@ -238,6 +252,8 @@ int sbi_misaligned_store_handler(ulong addr, ulong tval2, ulong tinst,
 			     &uptrap);
 		if (uptrap.cause) {
 			uptrap.epc = regs->mepc;
+			uptrap.tinst = sbi_misaligned_tinst_fixup(
+				tinst, uptrap.tinst, i);
 			return sbi_trap_redirect(regs, &uptrap);
 		}
 	}
-- 
2.36.0



From dramforever at live.com  Thu Jun  9 00:07:31 2022
From: dramforever at live.com (dramforever)
Date: Thu,  9 Jun 2022 15:07:31 +0800
Subject: [PATCH 3/3] lib: sbi: Fix tval and tinst for sbi_get_insn()
In-Reply-To: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <TYYP286MB14393C382F94A378E9F529CDC6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>

We should not change trap->tval to mepc because mtval already points to
the faulting portion of the emulated instruction fetch, which is also
what stval is expected to be.

In addition, htinst is only allowed to be zero for instruction access
faults or page faults, and is only allowed to be zero or a
psuedoinstruction for instruction guest-page faults. Fix trap->tinst for
these cases.

Signed-off-by: dramforever <dramforever at live.com>
---
 lib/sbi/sbi_unpriv.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/lib/sbi/sbi_unpriv.c b/lib/sbi/sbi_unpriv.c
index 73b530c..9a34a71 100644
--- a/lib/sbi/sbi_unpriv.c
+++ b/lib/sbi/sbi_unpriv.c
@@ -149,15 +149,17 @@ ulong sbi_get_insn(ulong mepc, struct sbi_trap_info *trap)
 	switch (trap->cause) {
 	case CAUSE_LOAD_ACCESS:
 		trap->cause = CAUSE_FETCH_ACCESS;
-		trap->tval = mepc;
+		trap->tinst = 0UL;
 		break;
 	case CAUSE_LOAD_PAGE_FAULT:
 		trap->cause = CAUSE_FETCH_PAGE_FAULT;
-		trap->tval = mepc;
+		trap->tinst = 0UL;
 		break;
 	case CAUSE_LOAD_GUEST_PAGE_FAULT:
 		trap->cause = CAUSE_FETCH_GUEST_PAGE_FAULT;
-		trap->tval = mepc;
+		if (trap->tinst != INSN_PSEUDO_VS_LOAD &&
+		    trap->tinst != INSN_PSEUDO_VS_STORE)
+			trap->tinst = 0UL;
 		break;
 	default:
 		break;
-- 
2.36.0



From apatel at ventanamicro.com  Thu Jun  9 01:51:59 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 9 Jun 2022 14:21:59 +0530
Subject: [PATCH 0/3] Fixes for tval/tinst when using sbi_trap_redirect()
In-Reply-To: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAK9=C2Uk=e4Hv=_VwWL9k0S=aNj450R=JNQUx-wbwFDwHS3heg@mail.gmail.com>

On Thu, Jun 9, 2022 at 12:37 PM dramforever <dramforever at live.com> wrote:
>
> I went through usages of sbi_trap_redirect() and saw some cases where
> after redirecting, stval/htinst could be set to incorrect values. I
> tried fixing what I noticed.

Overall, most fixes are in the right direction. I will review it soon.

>
> Note that these patches are *UNTESTED*.
>
> - For the htinst fixes, no current implementation (QEMU or Spike) of the
>   hypervisor extension writes mtinst/htinst to non-zero values, so the
>   code paths are unused.

Can you test this series with QEMU nested virtualization fixes which
include mtinst/htinst emulation ?
https://lore.kernel.org/all/20220609033011.752714-1-apatel at ventanamicro.com/

> - The sbi_get_insn() fix requires a rare occasion where the instruction
>   is fetched normally then unmapped from memory before OpenSBI can load
>   it again, and I have not been reproduce.

Yes, this case can happen but it is very rare and difficult to reproduce.

Regards,
Anup

>
> Please bear in mind when reviewing.
>
> dramforever (3):
>   include: sbi: Add mtinst/htinst psuedoinstructions
>   lib: sbi: Fixup tinst for exceptions in sbi_misaligned_*()
>   lib: sbi: Fix tval and tinst for sbi_get_insn()
>
>  include/sbi/riscv_encoding.h  | 20 ++++++++++++++++++++
>  lib/sbi/sbi_misaligned_ldst.c | 16 ++++++++++++++++
>  lib/sbi/sbi_unpriv.c          |  8 +++++---
>  3 files changed, 41 insertions(+), 3 deletions(-)
>
> --
> 2.36.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From apatel at ventanamicro.com  Thu Jun  9 05:13:33 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 9 Jun 2022 17:43:33 +0530
Subject: [PATCH 0/3] Fixes for tval/tinst when using sbi_trap_redirect()
In-Reply-To: <TYYP286MB1439D89DE7AF6803C451904FC6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <CAK9=C2Uk=e4Hv=_VwWL9k0S=aNj450R=JNQUx-wbwFDwHS3heg@mail.gmail.com> <TYYP286MB1439D89DE7AF6803C451904FC6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAK9=C2XWsE8Rzb3YHUE3a5Js2448v0S5qUS0m3Y--E798qe=Gg@mail.gmail.com>

On Thu, Jun 9, 2022 at 5:36 PM dram <dramforever at live.com> wrote:
>
>
> >> Note that these patches are *UNTESTED*.
> >>
> >> - For the htinst fixes, no current implementation (QEMU or Spike) of the
> >>   hypervisor extension writes mtinst/htinst to non-zero values, so the
> >>   code paths are unused.
> > Can you test this series with QEMU nested virtualization fixes which
> > include mtinst/htinst emulation ?
> > https://lore.kernel.org/all/20220609033011.752714-1-apatel at ventanamicro.com/
> I just realized that QEMU does not raise misaligned exceptions, so even with
> these patches we still couldn't test the sbi_misaligned_*() patch. Do you have
> suggestions on how we can test misaligned load/store emulation?

Spike does raise misaligned load/store exceptions so we need to add
htinst/mtinst support in Spike to test this.

Regards,
Anup

>
>


From dramforever at live.com  Thu Jun  9 05:06:13 2022
From: dramforever at live.com (dram)
Date: Thu, 9 Jun 2022 20:06:13 +0800
Subject: [PATCH 0/3] Fixes for tval/tinst when using sbi_trap_redirect()
In-Reply-To: <CAK9=C2Uk=e4Hv=_VwWL9k0S=aNj450R=JNQUx-wbwFDwHS3heg@mail.gmail.com>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <CAK9=C2Uk=e4Hv=_VwWL9k0S=aNj450R=JNQUx-wbwFDwHS3heg@mail.gmail.com>
Message-ID: <TYYP286MB1439D89DE7AF6803C451904FC6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>


>> Note that these patches are *UNTESTED*.
>>
>> - For the htinst fixes, no current implementation (QEMU or Spike) of the
>>   hypervisor extension writes mtinst/htinst to non-zero values, so the
>>   code paths are unused.
> Can you test this series with QEMU nested virtualization fixes which
> include mtinst/htinst emulation ?
> https://lore.kernel.org/all/20220609033011.752714-1-apatel at ventanamicro.com/
I just realized that QEMU does not raise misaligned exceptions, so even with
these patches we still couldn't test the sbi_misaligned_*() patch. Do you have
suggestions on how we can test misaligned load/store emulation?




From mchitale at ventanamicro.com  Thu Jun  9 21:49:20 2022
From: mchitale at ventanamicro.com (Mayuresh Chitale)
Date: Fri, 10 Jun 2022 10:19:20 +0530
Subject: [PATCH] platform: generic: add overrides for vendor extensions
Message-ID: <20220610044920.547047-1-mchitale@ventanamicro.com>

Allow the vendor_ext_check and vendor_ext_provider APIs of the
generic platform to be overridden by other platforms

Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
---
 platform/generic/include/platform_override.h |  7 ++++++
 platform/generic/platform.c                  | 25 ++++++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
index 4af8754..e55da25 100644
--- a/platform/generic/include/platform_override.h
+++ b/platform/generic/include/platform_override.h
@@ -11,6 +11,7 @@
 #define __PLATFORM_OVERRIDE_H__
 
 #include <sbi/sbi_types.h>
+#include <sbi/sbi_trap.h>
 
 struct platform_override {
 	const struct fdt_match *match_table;
@@ -21,6 +22,12 @@ struct platform_override {
 	void (*early_exit)(const struct fdt_match *match);
 	void (*final_exit)(const struct fdt_match *match);
 	int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
+	int (*vendor_ext_check)(long extid, const struct fdt_match *match);
+	int (*vendor_ext_provider)(long extid, long funcid,
+				   const struct sbi_trap_regs *regs,
+				   unsigned long *out_value,
+				   struct sbi_trap_info *out_trap,
+				   const struct fdt_match *match);
 };
 
 #endif
diff --git a/platform/generic/platform.c b/platform/generic/platform.c
index 7651273..bf93af6 100644
--- a/platform/generic/platform.c
+++ b/platform/generic/platform.c
@@ -168,6 +168,29 @@ static int generic_final_init(bool cold_boot)
 	return 0;
 }
 
+static int generic_vendor_ext_check(long extid)
+{
+	if (generic_plat && generic_plat->vendor_ext_check)
+		return generic_plat->vendor_ext_check(extid,
+						      generic_plat_match);
+
+	return 0;
+}
+
+static int generic_vendor_ext_provider(long extid, long funcid,
+				       const struct sbi_trap_regs *regs,
+				       unsigned long *out_value,
+				       struct sbi_trap_info *out_trap)
+{
+	if (generic_plat && generic_plat->vendor_ext_provider) {
+		return generic_plat->vendor_ext_provider(extid, funcid, regs,
+							 out_value, out_trap,
+							 generic_plat_match);
+	}
+
+	return SBI_ENOTSUPP;
+}
+
 static void generic_early_exit(void)
 {
 	if (generic_plat && generic_plat->early_exit)
@@ -236,6 +259,8 @@ const struct sbi_platform_operations platform_ops = {
 	.get_tlbr_flush_limit	= generic_tlbr_flush_limit,
 	.timer_init		= fdt_timer_init,
 	.timer_exit		= fdt_timer_exit,
+	.vendor_ext_check	= generic_vendor_ext_check,
+	.vendor_ext_provider	= generic_vendor_ext_provider,
 };
 
 struct sbi_platform platform = {
-- 
2.25.1



From anup at brainfault.org  Sat Jun 11 21:23:29 2022
From: anup at brainfault.org (Anup Patel)
Date: Sun, 12 Jun 2022 09:53:29 +0530
Subject: [RFC PATCH v2 8/9] lib: utils/irqchip: Add wrapper for T-HEAD
 PLIC delegation
In-Reply-To: <20220530033738.27127-9-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-9-samuel@sholland.org>
Message-ID: <CAAhSdy0RDhK6G8AsSVaU5kQULVdc1Zcf0HJ9WmRii_iCSydAHQ@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> The delegation bit is lost along with the rest of the PLIC state when
> the CPU power domain in the Allwinner D1 is powered down, so the PLIC
> needs to be re-delegated to S-mode during the hart resume path.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>
> ---

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi_utils/irqchip/fdt_irqchip_plic.h | 2 ++
>  lib/utils/irqchip/fdt_irqchip_plic.c         | 7 +++++++
>  2 files changed, 9 insertions(+)
>
> diff --git a/include/sbi_utils/irqchip/fdt_irqchip_plic.h b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> index 66650a2..d1ab5bd 100644
> --- a/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> +++ b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> @@ -17,4 +17,6 @@ void irqchip_plic_context_save(u32 *enable, u32 *threshold);
>
>  void irqchip_plic_context_restore(const u32 *enable, u32 threshold);
>
> +void thead_plic_restore(void);
> +
>  #endif
> diff --git a/lib/utils/irqchip/fdt_irqchip_plic.c b/lib/utils/irqchip/fdt_irqchip_plic.c
> index 37090d3..5589863 100644
> --- a/lib/utils/irqchip/fdt_irqchip_plic.c
> +++ b/lib/utils/irqchip/fdt_irqchip_plic.c
> @@ -151,6 +151,13 @@ static void thead_plic_plat_init(struct plic_data *pd)
>         writel_relaxed(BIT(0), (char *)pd->addr + THEAD_PLIC_CTRL_REG);
>  }
>
> +void thead_plic_restore(void)
> +{
> +       struct plic_data *plic = plic_hartid2data[current_hartid()];
> +
> +       thead_plic_plat_init(plic);
> +}
> +
>  static const struct fdt_match irqchip_plic_match[] = {
>         { .compatible = "riscv,plic0" },
>         { .compatible = "sifive,plic-1.0.0" },
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sat Jun 11 21:24:42 2022
From: anup at brainfault.org (Anup Patel)
Date: Sun, 12 Jun 2022 09:54:42 +0530
Subject: [RFC PATCH v2 9/9] platform: Add HSM implementation for Allwinner D1
In-Reply-To: <20220530033738.27127-10-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org> <20220530033738.27127-10-samuel@sholland.org>
Message-ID: <CAAhSdy30uVmY0nYA=BhLxQB8FAZBfBxHf_xxnmG=Utchc64jSQ@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> Allwinner D1 contains a "PPU" power domain controller which can
> automatically power down/up the CPU power domain. This power domain
> includes the C906 core along with its CLINT and PLIC.
>
> This HSM implementation supports non-retentive hart suspend by:
>   1) Saving/restoring state that is lost during hart suspend,
>   2) Performing cache maintenance before/after hart suspend,
>   3) Configuring wakeup sources before hart suspend, and
>   4) Asking the PPU to power down the hart when it enters WFI.
>
> Since this HSM implementation is for a single-core SoC, it does not need
> to worry about concurrency or saving multiple instances of state.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>
> Changes in v2:
>  - Rebase on top of the current master branch
>  - Use array_size() from sbi_types.h
>  - Use a separate variable for each CSR instead of an array
>  - Use the PLIC helpers added in earlier patches
>  - Use the saved SIE bits to program the wakeup interrupt mask
>  - Set the entry address only once during cold boot
>  - Use the generic implementation for retentive suspend
>  - Clean up/add comments throughout the HSM implementation
>
>  platform/generic/allwinner/objects.mk  |   6 +
>  platform/generic/allwinner/sun20i-d1.c | 210 +++++++++++++++++++++++++
>  2 files changed, 216 insertions(+)
>  create mode 100644 platform/generic/allwinner/objects.mk
>  create mode 100644 platform/generic/allwinner/sun20i-d1.c
>
> diff --git a/platform/generic/allwinner/objects.mk b/platform/generic/allwinner/objects.mk
> new file mode 100644
> index 0000000..9e36ab6
> --- /dev/null
> +++ b/platform/generic/allwinner/objects.mk
> @@ -0,0 +1,6 @@
> +#
> +# SPDX-License-Identifier: BSD-2-Clause
> +#
> +
> +carray-platform_override_modules-y += sun20i_d1
> +platform-objs-y += allwinner/sun20i-d1.o
> diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
> new file mode 100644
> index 0000000..5b88c2d
> --- /dev/null
> +++ b/platform/generic/allwinner/sun20i-d1.c
> @@ -0,0 +1,210 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Samuel Holland <samuel at sholland.org>
> + */
> +
> +#include <platform_override.h>
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_bitops.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_hsm.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +#include <sbi_utils/irqchip/fdt_irqchip_plic.h>
> +
> +#define SUN20I_D1_CCU_BASE             ((void *)0x02001000)
> +#define SUN20I_D1_RISCV_CFG_BASE       ((void *)0x06010000)
> +#define SUN20I_D1_PPU_BASE             ((void *)0x07001000)
> +#define SUN20I_D1_PRCM_BASE            ((void *)0x07010000)
> +
> +/*
> + * CCU
> + */
> +
> +#define CCU_BGR_ENABLE                 (BIT(16) | BIT(0))
> +
> +#define RISCV_CFG_BGR_REG              0xd0c
> +#define PPU_BGR_REG                    0x1ac
> +
> +/*
> + * CSRs
> + */
> +
> +#define CSR_MXSTATUS                   0x7c0
> +#define CSR_MHCR                       0x7c1
> +#define CSR_MCOR                       0x7c2
> +#define CSR_MHINT                      0x7c5
> +
> +static unsigned long csr_mxstatus;
> +static unsigned long csr_mhcr;
> +static unsigned long csr_mhint;
> +
> +static void sun20i_d1_csr_save(void)
> +{
> +       /* Save custom CSRs. */
> +       csr_mxstatus    = csr_read(CSR_MXSTATUS);
> +       csr_mhcr        = csr_read(CSR_MHCR);
> +       csr_mhint       = csr_read(CSR_MHINT);
> +
> +       /* Flush and disable caches. */
> +       csr_write(CSR_MCOR, 0x22);
> +       csr_write(CSR_MHCR, 0x0);
> +}
> +
> +static void sun20i_d1_csr_restore(void)
> +{
> +       /* Invalidate caches and the branch predictor. */
> +       csr_write(CSR_MCOR, 0x70013);
> +
> +       /* Restore custom CSRs, including the cache state. */
> +       csr_write(CSR_MXSTATUS, csr_mxstatus);
> +       csr_write(CSR_MHCR,     csr_mhcr);
> +       csr_write(CSR_MHINT,    csr_mhint);
> +}
> +
> +/*
> + * PLIC
> + */
> +
> +#define PLIC_SOURCES                   176
> +#define PLIC_IE_WORDS                  ((PLIC_SOURCES + 31) / 32)
> +
> +static u8 plic_priority[PLIC_SOURCES];
> +static u32 plic_sie[PLIC_IE_WORDS];
> +static u32 plic_threshold;
> +
> +static void sun20i_d1_plic_save(void)
> +{
> +       irqchip_plic_context_save(plic_sie, &plic_threshold);
> +       irqchip_plic_priority_save(plic_priority);
> +}
> +
> +static void sun20i_d1_plic_restore(void)
> +{
> +       thead_plic_restore();
> +       irqchip_plic_priority_restore(plic_priority);
> +       irqchip_plic_context_restore(plic_sie, plic_threshold);
> +}
> +
> +/*
> + * PPU
> + */
> +
> +#define PPU_PD_ACTIVE_CTRL             0x2c
> +
> +static void sun20i_d1_ppu_save(void)
> +{
> +       /* Enable MMIO access. Do not assume S-mode leaves the clock enabled. */
> +       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_PRCM_BASE + PPU_BGR_REG);
> +
> +       /* Activate automatic power-down during the next WFI. */
> +       writel_relaxed(1, SUN20I_D1_PPU_BASE + PPU_PD_ACTIVE_CTRL);
> +}
> +
> +static void sun20i_d1_ppu_restore(void)
> +{
> +       /* Disable automatic power-down. */
> +       writel_relaxed(0, SUN20I_D1_PPU_BASE + PPU_PD_ACTIVE_CTRL);
> +}
> +
> +/*
> + * RISCV_CFG
> + */
> +
> +#define RESET_ENTRY_LO_REG             0x0004
> +#define RESET_ENTRY_HI_REG             0x0008
> +#define WAKEUP_EN_REG                  0x0020
> +#define WAKEUP_MASK_REG(i)             (0x0024 + 4 * (i))
> +
> +static void sun20i_d1_riscv_cfg_save(void)
> +{
> +       /* Enable MMIO access. Do not assume S-mode leaves the clock enabled. */
> +       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_CCU_BASE + RISCV_CFG_BGR_REG);
> +
> +       /*
> +        * Copy the SIE bits to the wakeup registers. D1 has 160 "real"
> +        * interrupt sources, numbered 16-175. These are the ones that map to
> +        * the wakeup mask registers (the offset is for GIC compatibility). So
> +        * copying SIE to the wakeup mask needs some bit manipulation.
> +        */
> +       for (int i = 0; i < PLIC_IE_WORDS - 1; i++)
> +               writel_relaxed(plic_sie[i] >> 16 | plic_sie[i + 1] << 16,
> +                              SUN20I_D1_RISCV_CFG_BASE + WAKEUP_MASK_REG(i));
> +
> +       /* Enable PPU wakeup for interrupts. */
> +       writel_relaxed(1, SUN20I_D1_RISCV_CFG_BASE + WAKEUP_EN_REG);
> +}
> +
> +static void sun20i_d1_riscv_cfg_restore(void)
> +{
> +       /* Disable PPU wakeup for interrupts. */
> +       writel_relaxed(0, SUN20I_D1_RISCV_CFG_BASE + WAKEUP_EN_REG);
> +}
> +
> +static void sun20i_d1_riscv_cfg_init(void)
> +{
> +       unsigned long entry = sbi_hartid_to_scratch(0)->warmboot_addr;
> +
> +       /* Enable MMIO access. */
> +       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_CCU_BASE + RISCV_CFG_BGR_REG);
> +
> +       /* Program the reset entry address. */
> +       writel_relaxed((u32)entry, SUN20I_D1_RISCV_CFG_BASE + RESET_ENTRY_LO_REG);
> +       writel_relaxed(entry >> 32, SUN20I_D1_RISCV_CFG_BASE + RESET_ENTRY_HI_REG);
> +}
> +
> +static int sun20i_d1_hart_suspend(u32 suspend_type)
> +{
> +       /* Use the generic code for retentive suspend. */
> +       if (!(suspend_type & SBI_HSM_SUSP_NON_RET_BIT))
> +               return SBI_ENOTSUPP;
> +
> +       sun20i_d1_plic_save();
> +       sun20i_d1_ppu_save();
> +       sun20i_d1_riscv_cfg_save();
> +       sun20i_d1_csr_save();
> +
> +       /*
> +        * If no interrupt is pending, this will power down the CPU power
> +        * domain. Otherwise, this will fall through, and the generic HSM
> +        * code will jump to the resume address.
> +        */
> +       wfi();
> +
> +       return 0;
> +}
> +
> +static void sun20i_d1_hart_resume(void)
> +{
> +       sun20i_d1_csr_restore();
> +       sun20i_d1_riscv_cfg_restore();
> +       sun20i_d1_ppu_restore();
> +       sun20i_d1_plic_restore();
> +}
> +
> +static const struct sbi_hsm_device sun20i_d1_ppu = {
> +       .name           = "sun20i-d1-ppu",
> +       .hart_suspend   = sun20i_d1_hart_suspend,
> +       .hart_resume    = sun20i_d1_hart_resume,
> +};
> +
> +static int sun20i_d1_final_init(bool cold_boot, const struct fdt_match *match)
> +{
> +       if (cold_boot) {
> +               sun20i_d1_riscv_cfg_init();
> +               sbi_hsm_set_device(&sun20i_d1_ppu);
> +       }
> +
> +       return 0;
> +}
> +
> +static const struct fdt_match sun20i_d1_match[] = {
> +       { .compatible = "allwinner,sun20i-d1" },
> +       { },
> +};
> +
> +const struct platform_override sun20i_d1 = {
> +       .match_table    = sun20i_d1_match,
> +       .final_init     = sun20i_d1_final_init,
> +};
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sat Jun 11 21:26:02 2022
From: anup at brainfault.org (Anup Patel)
Date: Sun, 12 Jun 2022 09:56:02 +0530
Subject: [RFC PATCH v2 0/9] HSM implementation for Allwinner D1
In-Reply-To: <20220530033738.27127-1-samuel@sholland.org>
References: <20220530033738.27127-1-samuel@sholland.org>
Message-ID: <CAAhSdy2U0gnkgGpoLFf=6KWf6bbMrMeDjo09QZggh=A_uG_Eiw@mail.gmail.com>

On Mon, May 30, 2022 at 9:07 AM Samuel Holland <samuel at sholland.org> wrote:
>
> This series adds a minimal working HSM implementation for the Allwinner
> D1 SoC. It works with the Linux sbi_cpuidle driver. Even with canned
> values for entry/exit latency, I am seeing 30-45 mW reduction in power
> usage at idle.
>
> Since this is the first HSM driver submission, there is no example to
> follow. I have updated the series based on the comments to the first
> RFC. Still, comments welcome!
>
> Regards,
> Samuel
>
> Changes in v2:
>  - Rebase on top of the current master branch
>  - Use array_size() from sbi_types.h
>  - Use a separate variable for each CSR instead of an array
>  - Use the PLIC helpers added in earlier patches
>  - Use the saved SIE bits to program the wakeup interrupt mask
>  - Set the entry address only once during cold boot
>  - Use the generic implementation for retentive suspend
>  - Clean up/add comments throughout the HSM implementation
>
> Samuel Holland (9):
>   lib: sbi_hsm: Call a device hook during hart resume
>   lib: sbi_hsm: Assume a consistent resume address
>   lib: irqchip/plic: Constify plic_data pointers
>   lib: irqchip/plic: Factor out a context init function
>   lib: irqchip/plic: Add context save/restore helpers
>   lib: irqchip/plic: Add priority save/restore helpers
>   lib: utils/irqchip: Add FDT wrappers for PLIC save/restore functions
>   lib: utils/irqchip: Add wrapper for T-HEAD PLIC delegation
>   platform: Add HSM implementation for Allwinner D1

Overall this series is in good shape except a minor comment in PATCH7.

Please send v3.

Thanks,
Anup

>
>  include/sbi/sbi_hsm.h                        |  12 +-
>  include/sbi_utils/irqchip/fdt_irqchip_plic.h |  22 ++
>  include/sbi_utils/irqchip/plic.h             |  19 +-
>  lib/sbi/sbi_hsm.c                            |  54 ++---
>  lib/utils/irqchip/fdt_irqchip_plic.c         |  39 ++++
>  lib/utils/irqchip/plic.c                     | 125 ++++++++---
>  platform/fpga/ariane/platform.c              |  19 +-
>  platform/fpga/openpiton/platform.c           |  19 +-
>  platform/generic/allwinner/objects.mk        |   6 +
>  platform/generic/allwinner/sun20i-d1.c       | 210 +++++++++++++++++++
>  10 files changed, 442 insertions(+), 83 deletions(-)
>  create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
>  create mode 100644 platform/generic/allwinner/objects.mk
>  create mode 100644 platform/generic/allwinner/sun20i-d1.c
>
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sat Jun 11 21:30:11 2022
From: anup at brainfault.org (Anup Patel)
Date: Sun, 12 Jun 2022 10:00:11 +0530
Subject: [PATCH] lib: sbi_trap: Redirect exception based on hedeleg
In-Reply-To: <TYYP286MB14397827A57ED2BA6AA100E3C6A49@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB14397827A57ED2BA6AA100E3C6A49@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAAhSdy1ip6n5B7yjnEQWAKv7m1AYA36--S4U79FBf3UU_NoDUQ@mail.gmail.com>

On Wed, Jun 8, 2022 at 3:01 PM dramforever <dramforever at live.com> wrote:
>
> HS-mode software can choose what exceptions to delegate to VS-mode using
> the hedeleg CSR. Synthetic VS/VU-mode exceptions should also honor
> hedeleg. They should be redirected to VS-mode if and only if delegated
> by HS-mode.
>
> Signed-off-by: dramforever <dramforever at live.com>
> ---
>  lib/sbi/sbi_trap.c | 15 ++++++---------
>  1 file changed, 6 insertions(+), 9 deletions(-)
>
> diff --git a/lib/sbi/sbi_trap.c b/lib/sbi/sbi_trap.c
> index 2c509e5..effd456 100644
> --- a/lib/sbi/sbi_trap.c
> +++ b/lib/sbi/sbi_trap.c
> @@ -99,17 +99,14 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
>         if (prev_mode != PRV_S && prev_mode != PRV_U)
>                 return SBI_ENOTSUPP;
>
> -       /* For certain exceptions from VS/VU-mode we redirect to VS-mode */
> +       /* If exceptions came from VS/VU-mode, redirect to VS-mode if
> +        * delegated in hedeleg
> +        */
>         if (misa_extension('H') && prev_virt) {
> -               switch (trap->cause) {
> -               case CAUSE_FETCH_PAGE_FAULT:
> -               case CAUSE_LOAD_PAGE_FAULT:
> -               case CAUSE_STORE_PAGE_FAULT:
> +               if (trap->cause < __riscv_xlen &&
> +                   csr_read(CSR_HEDELEG) & (1 << trap->cause)) {

Use BIT(trap->cause) here.

>                         next_virt = TRUE;
> -                       break;
> -               default:
> -                       break;
> -               };
> +               }
>         }
>
>         /* Update MSTATUS MPV bits */
> --
> 2.36.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Apart from a minor comment above, this looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup


From anup at brainfault.org  Sat Jun 11 21:30:51 2022
From: anup at brainfault.org (Anup Patel)
Date: Sun, 12 Jun 2022 10:00:51 +0530
Subject: [PATCH] platform: generic: add overrides for vendor extensions
In-Reply-To: <20220610044920.547047-1-mchitale@ventanamicro.com>
References: <20220610044920.547047-1-mchitale@ventanamicro.com>
Message-ID: <CAAhSdy3hRji-urK15oYTduEijVGUBPYi8XR4X3AY+uzzTLkXnQ@mail.gmail.com>

On Fri, Jun 10, 2022 at 10:20 AM Mayuresh Chitale
<mchitale at ventanamicro.com> wrote:
>
> Allow the vendor_ext_check and vendor_ext_provider APIs of the
> generic platform to be overridden by other platforms
>
> Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  platform/generic/include/platform_override.h |  7 ++++++
>  platform/generic/platform.c                  | 25 ++++++++++++++++++++
>  2 files changed, 32 insertions(+)
>
> diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
> index 4af8754..e55da25 100644
> --- a/platform/generic/include/platform_override.h
> +++ b/platform/generic/include/platform_override.h
> @@ -11,6 +11,7 @@
>  #define __PLATFORM_OVERRIDE_H__
>
>  #include <sbi/sbi_types.h>
> +#include <sbi/sbi_trap.h>
>
>  struct platform_override {
>         const struct fdt_match *match_table;
> @@ -21,6 +22,12 @@ struct platform_override {
>         void (*early_exit)(const struct fdt_match *match);
>         void (*final_exit)(const struct fdt_match *match);
>         int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
> +       int (*vendor_ext_check)(long extid, const struct fdt_match *match);
> +       int (*vendor_ext_provider)(long extid, long funcid,
> +                                  const struct sbi_trap_regs *regs,
> +                                  unsigned long *out_value,
> +                                  struct sbi_trap_info *out_trap,
> +                                  const struct fdt_match *match);
>  };
>
>  #endif
> diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> index 7651273..bf93af6 100644
> --- a/platform/generic/platform.c
> +++ b/platform/generic/platform.c
> @@ -168,6 +168,29 @@ static int generic_final_init(bool cold_boot)
>         return 0;
>  }
>
> +static int generic_vendor_ext_check(long extid)
> +{
> +       if (generic_plat && generic_plat->vendor_ext_check)
> +               return generic_plat->vendor_ext_check(extid,
> +                                                     generic_plat_match);
> +
> +       return 0;
> +}
> +
> +static int generic_vendor_ext_provider(long extid, long funcid,
> +                                      const struct sbi_trap_regs *regs,
> +                                      unsigned long *out_value,
> +                                      struct sbi_trap_info *out_trap)
> +{
> +       if (generic_plat && generic_plat->vendor_ext_provider) {
> +               return generic_plat->vendor_ext_provider(extid, funcid, regs,
> +                                                        out_value, out_trap,
> +                                                        generic_plat_match);
> +       }
> +
> +       return SBI_ENOTSUPP;
> +}
> +
>  static void generic_early_exit(void)
>  {
>         if (generic_plat && generic_plat->early_exit)
> @@ -236,6 +259,8 @@ const struct sbi_platform_operations platform_ops = {
>         .get_tlbr_flush_limit   = generic_tlbr_flush_limit,
>         .timer_init             = fdt_timer_init,
>         .timer_exit             = fdt_timer_exit,
> +       .vendor_ext_check       = generic_vendor_ext_check,
> +       .vendor_ext_provider    = generic_vendor_ext_provider,
>  };
>
>  struct sbi_platform platform = {
> --
> 2.25.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From dramforever at live.com  Sat Jun 11 22:57:37 2022
From: dramforever at live.com (dramforever)
Date: Sun, 12 Jun 2022 13:57:37 +0800
Subject: [PATCH v2] lib: sbi_trap: Redirect exception based on hedeleg
In-Reply-To: <TYYP286MB14397827A57ED2BA6AA100E3C6A49@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <TYYP286MB1439777CF535CC8DF8388B0CC6A89@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>

HS-mode software can choose what exceptions to delegate to VS-mode using
the hedeleg CSR. Synthetic VS/VU-mode exceptions should also honor
hedeleg. They should be redirected to VS-mode if and only if delegated
by HS-mode.

Signed-off-by: dramforever <dramforever at live.com>
---

Changes since v1:
 - Use BIT() macro instead of (1 << trap->cause)

 lib/sbi/sbi_trap.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/lib/sbi/sbi_trap.c b/lib/sbi/sbi_trap.c
index 2c509e5..a54a131 100644
--- a/lib/sbi/sbi_trap.c
+++ b/lib/sbi/sbi_trap.c
@@ -99,17 +99,14 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
 	if (prev_mode != PRV_S && prev_mode != PRV_U)
 		return SBI_ENOTSUPP;
 
-	/* For certain exceptions from VS/VU-mode we redirect to VS-mode */
+	/* If exceptions came from VS/VU-mode, redirect to VS-mode if
+	 * delegated in hedeleg
+	 */
 	if (misa_extension('H') && prev_virt) {
-		switch (trap->cause) {
-		case CAUSE_FETCH_PAGE_FAULT:
-		case CAUSE_LOAD_PAGE_FAULT:
-		case CAUSE_STORE_PAGE_FAULT:
+		if (trap->cause < __riscv_xlen &&
+		    csr_read(CSR_HEDELEG) & BIT(trap->cause)) {
 			next_virt = TRUE;
-			break;
-		default:
-			break;
-		};
+		}
 	}
 
 	/* Update MSTATUS MPV bits */
-- 
2.36.1



From misterjdrg at gmail.com  Sun Jun 12 10:18:21 2022
From: misterjdrg at gmail.com (Bohdan Fedorov)
Date: Sun, 12 Jun 2022 20:18:21 +0300
Subject: [PATCH 1/2] lib: utils/serial: add virtio mmio driver
Message-ID: <20220612171822.148906-1-misterjdrg@gmail.com>

Simple virtio-console driver.
Useful for running opensbi in virtualized environments.

dt-bindings:
<linux>/Documentation/devicetree/bindings/virtio/mmio.yaml

Signed-off-by: Bohdan Fedorov <misterjdrg at gmail.com>
---
 include/sbi_utils/fdt/fdt_helper.h     |  3 +
 include/sbi_utils/serial/virtio-uart.h | 19 +++++++
 lib/utils/fdt/fdt_helper.c             | 18 ++++++
 lib/utils/serial/fdt_serial_virtio.c   | 37 +++++++++++++
 lib/utils/serial/objects.mk            |  4 ++
 lib/utils/serial/virtio-uart.c         | 76 ++++++++++++++++++++++++++
 6 files changed, 157 insertions(+)
 create mode 100644 include/sbi_utils/serial/virtio-uart.h
 create mode 100644 lib/utils/serial/fdt_serial_virtio.c
 create mode 100644 lib/utils/serial/virtio-uart.c

diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
index c60af35..80307bb 100644
--- a/include/sbi_utils/fdt/fdt_helper.h
+++ b/include/sbi_utils/fdt/fdt_helper.h
@@ -65,6 +65,9 @@ int fdt_parse_shakti_uart_node(void *fdt, int nodeoffset,
 int fdt_parse_sifive_uart_node(void *fdt, int nodeoffset,
 			       struct platform_uart_data *uart);
 
+int fdt_parse_virtio_uart_node(void *fdt, int nodeoffset,
+			       struct platform_uart_data *uart);
+
 int fdt_parse_uart8250_node(void *fdt, int nodeoffset,
 			    struct platform_uart_data *uart);
 
diff --git a/include/sbi_utils/serial/virtio-uart.h b/include/sbi_utils/serial/virtio-uart.h
new file mode 100644
index 0000000..4fdfebc
--- /dev/null
+++ b/include/sbi_utils/serial/virtio-uart.h
@@ -0,0 +1,19 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2019 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ *   Bohdan Fedorov <misterjdrg at gmail.com>
+ *
+ * Based on:
+ *   sifive-uart
+ */
+
+#ifndef __SERIAL_SIFIVE_UART_H__
+#define __SERIAL_SIFIVE_UART_H__
+
+
+int virtio_uart_init(unsigned long base);
+
+#endif
diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
index 2bdb1ef..9bfd5e7 100644
--- a/lib/utils/fdt/fdt_helper.c
+++ b/lib/utils/fdt/fdt_helper.c
@@ -388,6 +388,24 @@ int fdt_parse_shakti_uart_node(void *fdt, int nodeoffset,
 	return 0;
 }
 
+int fdt_parse_virtio_uart_node(void *fdt, int nodeoffset,
+			       struct platform_uart_data *uart)
+{
+	int rc;
+	uint64_t reg_addr, reg_size;
+
+	if (nodeoffset < 0 || !uart || !fdt)
+		return SBI_ENODEV;
+
+	rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
+				    &reg_addr, &reg_size);
+	if (rc < 0 || !reg_addr || !reg_size)
+		return SBI_ENODEV;
+	uart->addr = reg_addr;
+
+	return 0;
+}
+
 int fdt_parse_sifive_uart_node(void *fdt, int nodeoffset,
 			       struct platform_uart_data *uart)
 {
diff --git a/lib/utils/serial/fdt_serial_virtio.c b/lib/utils/serial/fdt_serial_virtio.c
new file mode 100644
index 0000000..115c5ad
--- /dev/null
+++ b/lib/utils/serial/fdt_serial_virtio.c
@@ -0,0 +1,37 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2020 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ *   Bohdan Fedorov <misterjdrg at gmail.com>
+ *
+ * Based on virtio-uart
+ */
+
+#include <sbi_utils/fdt/fdt_helper.h>
+#include <sbi_utils/serial/fdt_serial.h>
+#include <sbi_utils/serial/virtio-uart.h>
+
+static int serial_virtio_init(void *fdt, int nodeoff,
+				const struct fdt_match *match)
+{
+	int rc;
+	struct platform_uart_data uart;
+
+	rc = fdt_parse_virtio_uart_node(fdt, nodeoff, &uart);
+	if (rc)
+		return rc;
+
+	return virtio_uart_init(uart.addr);
+}
+
+static const struct fdt_match serial_virtio_match[] = {
+	{ .compatible = "virtio,mmio" },
+	{ },
+};
+
+struct fdt_serial fdt_serial_virtio = {
+	.match_table = serial_virtio_match,
+	.init = serial_virtio_init
+};
diff --git a/lib/utils/serial/objects.mk b/lib/utils/serial/objects.mk
index d26a74e..60e829b 100644
--- a/lib/utils/serial/objects.mk
+++ b/lib/utils/serial/objects.mk
@@ -22,6 +22,9 @@ libsbiutils-objs-y += serial/fdt_serial_shakti.o
 carray-fdt_serial_drivers-y += fdt_serial_sifive
 libsbiutils-objs-y += serial/fdt_serial_sifive.o
 
+carray-fdt_serial_drivers-y += fdt_serial_virtio
+libsbiutils-objs-y += serial/fdt_serial_virtio.o
+
 carray-fdt_serial_drivers-y += fdt_serial_litex
 libsbiutils-objs-y += serial/fdt_serial_litex.o
 
@@ -34,6 +37,7 @@ libsbiutils-objs-y += serial/fdt_serial_xlnx_uartlite.o
 libsbiutils-objs-y += serial/gaisler-uart.o
 libsbiutils-objs-y += serial/shakti-uart.o
 libsbiutils-objs-y += serial/sifive-uart.o
+libsbiutils-objs-y += serial/virtio-uart.o
 libsbiutils-objs-y += serial/litex-uart.o
 libsbiutils-objs-y += serial/uart8250.o
 libsbiutils-objs-y += serial/xlnx-uartlite.o
diff --git a/lib/utils/serial/virtio-uart.c b/lib/utils/serial/virtio-uart.c
new file mode 100644
index 0000000..1d4266f
--- /dev/null
+++ b/lib/utils/serial/virtio-uart.c
@@ -0,0 +1,76 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2019 Western Digital Corporation or its affiliates.
+ *
+ * Authors:
+ *   Bohdan Fedorov <misterjdrg at gmial.com>
+ *
+ * Based on:
+ *   sifive-uart
+ */
+
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_console.h>
+#include <sbi_utils/serial/virtio-uart.h>
+
+/* clang-format off */
+
+#define VIRTIO_MAGIC                  0x0
+#define VIRTIO_MAGIC_VALUE            0x74726976
+#define VIRTIO_DEVICE_ID	      0x8
+#define VIRTIO_DEVICE_ID_CONSOLE      0x3
+#define VIRTIO_CONSOLE_F_EMERG_WRITE  (1 << 2)
+#define VIRTIO_DEVICE_FEATURES 	      0x10
+#define VIRTIO_DEVICE_FEATURES_SEL    0x14
+#define VIRTIO_CONSOLE_EMERG_WR       0x108
+
+/* clang-format on */
+
+static volatile void *uart_base;
+
+static u32 get_reg(u32 off)
+{
+	return readl(uart_base + off);
+}
+
+static void set_reg(u32 off, u32 val)
+{
+	writel(val, uart_base + off);
+}
+
+static void virtio_uart_putc(char ch)
+{
+	set_reg(VIRTIO_CONSOLE_EMERG_WR, ch);
+}
+
+static struct sbi_console_device virtio_console = {
+	.name = "virtio_uart",
+	.console_putc = virtio_uart_putc,
+	.console_getc = NULL,
+};
+
+int virtio_uart_init(unsigned long base)
+{
+	uart_base = (volatile void *)base;
+	
+	// Chech magic
+	if(get_reg(VIRTIO_MAGIC) != VIRTIO_MAGIC_VALUE) {
+		return SBI_ENODEV;
+	}
+
+	// Virtio device is console
+	if(get_reg(VIRTIO_DEVICE_ID) != VIRTIO_DEVICE_ID_CONSOLE) {
+		return SBI_ENODEV;
+	}
+
+	// Virtio console has emergency write feature
+	set_reg(VIRTIO_DEVICE_FEATURES_SEL, 0);
+	if((get_reg(VIRTIO_DEVICE_FEATURES) & VIRTIO_CONSOLE_F_EMERG_WRITE) == 0) {
+		return SBI_ENODEV;
+	}
+
+	sbi_console_set_device(&virtio_console);
+	return 0;
+}
-- 
2.36.1



From misterjdrg at gmail.com  Sun Jun 12 10:18:22 2022
From: misterjdrg at gmail.com (Bohdan Fedorov)
Date: Sun, 12 Jun 2022 20:18:22 +0300
Subject: [PATCH 2/2] lib: utils/serial: Check all compatible devices for driver
In-Reply-To: <20220612171822.148906-1-misterjdrg@gmail.com>
References: <20220612171822.148906-1-misterjdrg@gmail.com>
Message-ID: <20220612171822.148906-2-misterjdrg@gmail.com>

System can have multiple virtio,mmio compatible devices.

We need check all of them to choose serial with emergency
write feature.

Signed-off-by: Bohdan Fedorov <misterjdrg at gmail.com>
---
 lib/utils/serial/fdt_serial.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)

diff --git a/lib/utils/serial/fdt_serial.c b/lib/utils/serial/fdt_serial.c
index 1a4bf9e..761d567 100644
--- a/lib/utils/serial/fdt_serial.c
+++ b/lib/utils/serial/fdt_serial.c
@@ -76,19 +76,24 @@ int fdt_serial_init(void)
 	for (pos = 0; pos < fdt_serial_drivers_size; pos++) {
 		drv = fdt_serial_drivers[pos];
 
-		noff = fdt_find_match(fdt, -1, drv->match_table, &match);
-		if (noff < 0)
-			continue;
+		noff = -1;
+		do {
+			noff = fdt_find_match(fdt, noff, drv->match_table, &match);
 
-		if (drv->init) {
-			rc = drv->init(fdt, noff, match);
-			if (rc == SBI_ENODEV)
-				continue;
-			if (rc)
-				return rc;
-		}
-		current_driver = drv;
-		break;
+			if (noff < 0)
+				break;
+
+			if (drv->init) {
+				rc = drv->init(fdt, noff, match);
+				if (rc == SBI_ENODEV)
+					continue;
+				if (rc)
+					return rc;
+			}
+
+			current_driver = drv;
+			goto done;
+		} while (noff >= 0);
 	}
 
 done:
-- 
2.36.1



From samuel at sholland.org  Sun Jun 12 17:38:38 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 19:38:38 -0500
Subject: [PATCH] lib: sbi: Fix version dependency for Sscofpmf
Message-ID: <20220613003838.13028-1-samuel@sholland.org>

mcounteren was added in the privileged spec v1.10 and mcountinhibit was
added in v1.11. Sscofpmf does not depend on anything in v1.12, so the
minimum privileged spec version should have been v1.11.

Fixes: d4b563c881d6 ("lib: sbi: Remove MCOUNTEREN and SCOUNTEREN hart features")
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

 lib/sbi/sbi_hart.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index de86fee..5bc6d52 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -645,7 +645,7 @@ __mhpm_skip:
 		hfeatures->priv_version = SBI_HART_PRIV_VER_1_12;
 
 	/* Counter overflow/filtering is not useful without mcounter/inhibit */
-	if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
+	if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_11) {
 		/* Detect if hart supports sscofpmf */
 		csr_read_allowed(CSR_SCOUNTOVF, (unsigned long)&trap);
 		if (!trap.cause)
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:46 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:46 -0500
Subject: [PATCH v3 0/9] HSM implementation for Allwinner D1
Message-ID: <20220613010355.28014-1-samuel@sholland.org>

This series adds a minimal working HSM implementation for the Allwinner
D1 SoC. It works with the Linux sbi_cpuidle driver. Even with canned
values for entry/exit latency, I am seeing 30-45 mW reduction in power
usage at idle.

Since this is the first HSM driver submission, there is no example to
follow. I have updated the series based on the comments to the RFCs.
Still, comments welcome!

Regards,
Samuel

Changes in v3:
 - Fix typo in commit message
 - Move comment about priority range to function declaration
 - Rename irqchip_plic_* functions to fdt_plic_*
 - Add a "bool smode" parameter instead of hard coding the context ID

Changes in v2:
 - Rebase on top of the current master branch
 - Use array_size() from sbi_types.h
 - Use a separate variable for each CSR instead of an array
 - Use the PLIC helpers added in earlier patches
 - Use the saved SIE bits to program the wakeup interrupt mask
 - Set the entry address only once during cold boot
 - Use the generic implementation for retentive suspend
 - Clean up/add comments throughout the HSM implementation

Samuel Holland (9):
  lib: sbi_hsm: Call a device hook during hart resume
  lib: sbi_hsm: Assume a consistent resume address
  lib: irqchip/plic: Constify plic_data pointers
  lib: irqchip/plic: Factor out a context init function
  lib: irqchip/plic: Add context save/restore helpers
  lib: irqchip/plic: Add priority save/restore helpers
  lib: utils/irqchip: Add FDT wrappers for PLIC save/restore functions
  lib: utils/irqchip: Add wrapper for T-HEAD PLIC delegation
  platform: Add HSM implementation for Allwinner D1

 include/sbi/sbi_hsm.h                        |  12 +-
 include/sbi_utils/irqchip/fdt_irqchip_plic.h |  22 ++
 include/sbi_utils/irqchip/plic.h             |  20 +-
 lib/sbi/sbi_hsm.c                            |  54 ++---
 lib/utils/irqchip/fdt_irqchip_plic.c         |  39 ++++
 lib/utils/irqchip/plic.c                     | 124 ++++++++---
 platform/fpga/ariane/platform.c              |  19 +-
 platform/fpga/openpiton/platform.c           |  19 +-
 platform/generic/allwinner/objects.mk        |   6 +
 platform/generic/allwinner/sun20i-d1.c       | 210 +++++++++++++++++++
 10 files changed, 442 insertions(+), 83 deletions(-)
 create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
 create mode 100644 platform/generic/allwinner/objects.mk
 create mode 100644 platform/generic/allwinner/sun20i-d1.c

-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:47 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:47 -0500
Subject: [PATCH v3 1/9] lib: sbi_hsm: Call a device hook during hart resume
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-2-samuel@sholland.org>

Non-retentive suspend states may require platform-specific actions
during resume. For example, firmware may need to save and restore the
values of custom CSRs. Add a hook to support this.

Reviewed-by: Anup Patel <anup at brainfault.org>
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

(no changes since v1)

 include/sbi/sbi_hsm.h | 8 ++++++++
 lib/sbi/sbi_hsm.c     | 8 ++++++++
 2 files changed, 16 insertions(+)

diff --git a/include/sbi/sbi_hsm.h b/include/sbi/sbi_hsm.h
index c16e871..6da8cee 100644
--- a/include/sbi/sbi_hsm.h
+++ b/include/sbi/sbi_hsm.h
@@ -37,6 +37,14 @@ struct sbi_hsm_device {
 	 * specified resume address
 	 */
 	int (*hart_suspend)(u32 suspend_type, ulong raddr);
+
+	/**
+	 * Perform platform-specific actions to resume from a suspended state.
+	 *
+	 * This includes restoring any platform state that was lost during
+	 * non-retentive suspend.
+	 */
+	void (*hart_resume)(void);
 };
 
 struct sbi_domain;
diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index c4d2c6d..a7b6a80 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -178,6 +178,12 @@ static int hsm_device_hart_suspend(u32 suspend_type, ulong raddr)
 	return SBI_ENOTSUPP;
 }
 
+static void hsm_device_hart_resume(void)
+{
+	if (hsm_dev && hsm_dev->hart_resume)
+		hsm_dev->hart_resume();
+}
+
 int sbi_hsm_init(struct sbi_scratch *scratch, u32 hartid, bool cold_boot)
 {
 	u32 i;
@@ -384,6 +390,8 @@ void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch)
 			   __func__, oldstate);
 		sbi_hart_hang();
 	}
+
+	hsm_device_hart_resume();
 }
 
 void sbi_hsm_hart_resume_finish(struct sbi_scratch *scratch)
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:48 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:48 -0500
Subject: [PATCH v3 2/9] lib: sbi_hsm: Assume a consistent resume address
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-3-samuel@sholland.org>

The suspend code needs to know the resume address for two reasons:
  1) Programming some hardware register or management firmware. Here we
     assume the hardware/firmware maintains its state between suspends,
     so it only needs to be programmed once at startup.
  2) When a non-retentive suspend request ends up being retentive, due
     to lack of hardware support, pending interrupt, or for some other
     reason. However, the behavior here is not platform-dependent, and
     this can be handled in the generic hart suspend function.

Since neither situation requires the platform-level suspend function to
know the resume address, stop passing it to that function. Instead,
handle the non-retentive to retentive situation generically.

Reviewed-by: Anup Patel <anup at brainfault.org>
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

Changes in v3:
 - Fix typo in commit message

Changes in v2:
 - New patch for v2

 include/sbi/sbi_hsm.h |  4 ++--
 lib/sbi/sbi_hsm.c     | 46 +++++++++++++++++++------------------------
 2 files changed, 22 insertions(+), 28 deletions(-)

diff --git a/include/sbi/sbi_hsm.h b/include/sbi/sbi_hsm.h
index 6da8cee..d6cc468 100644
--- a/include/sbi/sbi_hsm.h
+++ b/include/sbi/sbi_hsm.h
@@ -34,9 +34,9 @@ struct sbi_hsm_device {
 	 * the hart resumes normal execution.
 	 *
 	 * For successful non-retentive suspend, the hart will resume from
-	 * specified resume address
+	 * the warm boot entry point.
 	 */
-	int (*hart_suspend)(u32 suspend_type, ulong raddr);
+	int (*hart_suspend)(u32 suspend_type);
 
 	/**
 	 * Perform platform-specific actions to resume from a suspended state.
diff --git a/lib/sbi/sbi_hsm.c b/lib/sbi/sbi_hsm.c
index a7b6a80..1165acc 100644
--- a/lib/sbi/sbi_hsm.c
+++ b/lib/sbi/sbi_hsm.c
@@ -171,10 +171,10 @@ static int hsm_device_hart_stop(void)
 	return SBI_ENOTSUPP;
 }
 
-static int hsm_device_hart_suspend(u32 suspend_type, ulong raddr)
+static int hsm_device_hart_suspend(u32 suspend_type)
 {
 	if (hsm_dev && hsm_dev->hart_suspend)
-		return hsm_dev->hart_suspend(suspend_type, raddr);
+		return hsm_dev->hart_suspend(suspend_type);
 	return SBI_ENOTSUPP;
 }
 
@@ -319,7 +319,7 @@ int sbi_hsm_hart_stop(struct sbi_scratch *scratch, bool exitnow)
 	return 0;
 }
 
-static int __sbi_hsm_suspend_ret_default(struct sbi_scratch *scratch)
+static int __sbi_hsm_suspend_default(struct sbi_scratch *scratch)
 {
 	/* Wait for interrupt */
 	wfi();
@@ -359,23 +359,6 @@ static void __sbi_hsm_suspend_non_ret_restore(struct sbi_scratch *scratch)
 	csr_write(CSR_MIP, (hdata->saved_mip & (MIP_SSIP | MIP_STIP)));
 }
 
-static int __sbi_hsm_suspend_non_ret_default(struct sbi_scratch *scratch,
-					     ulong raddr)
-{
-	void (*jump_warmboot)(void) = (void (*)(void))scratch->warmboot_addr;
-
-	/* Wait for interrupt */
-	wfi();
-
-	/*
-	 * Directly jump to warm reboot to simulate resume from a
-	 * non-retentive suspend.
-	 */
-	jump_warmboot();
-
-	return 0;
-}
-
 void sbi_hsm_hart_resume_start(struct sbi_scratch *scratch)
 {
 	int oldstate;
@@ -473,17 +456,28 @@ int sbi_hsm_hart_suspend(struct sbi_scratch *scratch, u32 suspend_type,
 		__sbi_hsm_suspend_non_ret_save(scratch);
 
 	/* Try platform specific suspend */
-	ret = hsm_device_hart_suspend(suspend_type, scratch->warmboot_addr);
+	ret = hsm_device_hart_suspend(suspend_type);
 	if (ret == SBI_ENOTSUPP) {
 		/* Try generic implementation of default suspend types */
-		if (suspend_type == SBI_HSM_SUSPEND_RET_DEFAULT) {
-			ret = __sbi_hsm_suspend_ret_default(scratch);
-		} else if (suspend_type == SBI_HSM_SUSPEND_NON_RET_DEFAULT) {
-			ret = __sbi_hsm_suspend_non_ret_default(scratch,
-						scratch->warmboot_addr);
+		if (suspend_type == SBI_HSM_SUSPEND_RET_DEFAULT ||
+		    suspend_type == SBI_HSM_SUSPEND_NON_RET_DEFAULT) {
+			ret = __sbi_hsm_suspend_default(scratch);
 		}
 	}
 
+	/*
+	 * The platform may have coordinated a retentive suspend, or it may
+	 * have exited early from a non-retentive suspend. Either way, the
+	 * caller is not expecting a successful return, so jump to the warm
+	 * boot entry point to simulate resume from a non-retentive suspend.
+	 */
+	if (ret == 0 && (suspend_type & SBI_HSM_SUSP_NON_RET_BIT)) {
+		void (*jump_warmboot)(void) =
+			(void (*)(void))scratch->warmboot_addr;
+
+		jump_warmboot();
+	}
+
 fail_restore_state:
 	/*
 	 * We might have successfully resumed from retentive suspend
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:49 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:49 -0500
Subject: [PATCH v3 3/9] lib: irqchip/plic: Constify plic_data pointers
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-4-samuel@sholland.org>

None of the functions modify the passed-in plic_data, so mark it const.

Reviewed-by: Anup Patel <anup at brainfault.org>
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

(no changes since v2)

Changes in v2:
 - New patch for v2

 include/sbi_utils/irqchip/plic.h |  9 +++++----
 lib/utils/irqchip/plic.c         | 11 ++++++-----
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/include/sbi_utils/irqchip/plic.h b/include/sbi_utils/irqchip/plic.h
index 0e56d80..50be5df 100644
--- a/include/sbi_utils/irqchip/plic.h
+++ b/include/sbi_utils/irqchip/plic.h
@@ -17,13 +17,14 @@ struct plic_data {
 	unsigned long num_src;
 };
 
-int plic_warm_irqchip_init(struct plic_data *plic,
+int plic_warm_irqchip_init(const struct plic_data *plic,
 			   int m_cntx_id, int s_cntx_id);
 
-int plic_cold_irqchip_init(struct plic_data *plic);
+int plic_cold_irqchip_init(const struct plic_data *plic);
 
-void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val);
+void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val);
 
-void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val);
+void plic_set_ie(const struct plic_data *plic, u32 cntxid,
+		 u32 word_index, u32 val);
 
 #endif
diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
index 0f58f3e..4c334ec 100644
--- a/lib/utils/irqchip/plic.c
+++ b/lib/utils/irqchip/plic.c
@@ -21,14 +21,14 @@
 #define PLIC_CONTEXT_BASE 0x200000
 #define PLIC_CONTEXT_STRIDE 0x1000
 
-static void plic_set_priority(struct plic_data *plic, u32 source, u32 val)
+static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
 {
 	volatile void *plic_priority = (char *)plic->addr +
 			PLIC_PRIORITY_BASE + 4 * source;
 	writel(val, plic_priority);
 }
 
-void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val)
+void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
 {
 	volatile void *plic_thresh;
 
@@ -40,7 +40,8 @@ void plic_set_thresh(struct plic_data *plic, u32 cntxid, u32 val)
 	writel(val, plic_thresh);
 }
 
-void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val)
+void plic_set_ie(const struct plic_data *plic, u32 cntxid,
+		 u32 word_index, u32 val)
 {
 	volatile char *plic_ie;
 
@@ -52,7 +53,7 @@ void plic_set_ie(struct plic_data *plic, u32 cntxid, u32 word_index, u32 val)
 	writel(val, plic_ie + word_index * 4);
 }
 
-int plic_warm_irqchip_init(struct plic_data *plic,
+int plic_warm_irqchip_init(const struct plic_data *plic,
 			   int m_cntx_id, int s_cntx_id)
 {
 	size_t i, ie_words;
@@ -85,7 +86,7 @@ int plic_warm_irqchip_init(struct plic_data *plic,
 	return 0;
 }
 
-int plic_cold_irqchip_init(struct plic_data *plic)
+int plic_cold_irqchip_init(const struct plic_data *plic)
 {
 	int i;
 
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:50 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:50 -0500
Subject: [PATCH v3 4/9] lib: irqchip/plic: Factor out a context init function
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-5-samuel@sholland.org>

This simplifies both the callers and the callees by removing duplicated
code and consolidating the error handling. It also fixes two bugs in the
process:
  1) ie_words was one too large when plic->num_src was a multiple of 32.
  2) plic_set_ie takes a 32-bit mask, not a Boolean value, so the FPGA
     platforms previously only enabled one out of every 32 interrupts.

Reviewed-by: Anup Patel <anup at brainfault.org>
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

(no changes since v2)

Changes in v2:
 - New patch for v2

 include/sbi_utils/irqchip/plic.h   |  8 ++---
 lib/utils/irqchip/plic.c           | 55 ++++++++++++++++--------------
 platform/fpga/ariane/platform.c    | 19 +++++------
 platform/fpga/openpiton/platform.c | 19 +++++------
 4 files changed, 48 insertions(+), 53 deletions(-)

diff --git a/include/sbi_utils/irqchip/plic.h b/include/sbi_utils/irqchip/plic.h
index 50be5df..8f21af6 100644
--- a/include/sbi_utils/irqchip/plic.h
+++ b/include/sbi_utils/irqchip/plic.h
@@ -17,14 +17,12 @@ struct plic_data {
 	unsigned long num_src;
 };
 
+int plic_context_init(const struct plic_data *plic, int context_id,
+		      bool enable, u32 threshold);
+
 int plic_warm_irqchip_init(const struct plic_data *plic,
 			   int m_cntx_id, int s_cntx_id);
 
 int plic_cold_irqchip_init(const struct plic_data *plic);
 
-void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val);
-
-void plic_set_ie(const struct plic_data *plic, u32 cntxid,
-		 u32 word_index, u32 val);
-
 #endif
diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
index 4c334ec..9bd3bf1 100644
--- a/lib/utils/irqchip/plic.c
+++ b/lib/utils/irqchip/plic.c
@@ -5,6 +5,7 @@
  *
  * Authors:
  *   Anup Patel <anup.patel at wdc.com>
+ *   Samuel Holland <samuel at sholland.org>
  */
 
 #include <sbi/riscv_io.h>
@@ -28,61 +29,63 @@ static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
 	writel(val, plic_priority);
 }
 
-void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
+static void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
 {
 	volatile void *plic_thresh;
 
-	if (!plic)
-		return;
-
 	plic_thresh = (char *)plic->addr +
 		      PLIC_CONTEXT_BASE + PLIC_CONTEXT_STRIDE * cntxid;
 	writel(val, plic_thresh);
 }
 
-void plic_set_ie(const struct plic_data *plic, u32 cntxid,
-		 u32 word_index, u32 val)
+static void plic_set_ie(const struct plic_data *plic, u32 cntxid,
+			u32 word_index, u32 val)
 {
 	volatile char *plic_ie;
 
-	if (!plic)
-		return;
-
 	plic_ie = (char *)plic->addr +
 		   PLIC_ENABLE_BASE + PLIC_ENABLE_STRIDE * cntxid;
 	writel(val, plic_ie + word_index * 4);
 }
 
-int plic_warm_irqchip_init(const struct plic_data *plic,
-			   int m_cntx_id, int s_cntx_id)
+int plic_context_init(const struct plic_data *plic, int context_id,
+		      bool enable, u32 threshold)
 {
-	size_t i, ie_words;
+	u32 ie_words, ie_value;
 
-	if (!plic)
+	if (!plic || context_id < 0)
 		return SBI_EINVAL;
 
-	ie_words = plic->num_src / 32 + 1;
+	ie_words = (plic->num_src + 31) / 32;
+	ie_value = enable ? 0xffffffffU : 0U;
+
+	for (u32 i = 0; i < ie_words; i++)
+		plic_set_ie(plic, context_id, i, ie_value);
+
+	plic_set_thresh(plic, context_id, threshold);
+
+	return 0;
+}
+
+int plic_warm_irqchip_init(const struct plic_data *plic,
+			   int m_cntx_id, int s_cntx_id)
+{
+	int ret;
 
 	/* By default, disable all IRQs for M-mode of target HART */
 	if (m_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(plic, m_cntx_id, i, 0);
+		ret = plic_context_init(plic, m_cntx_id, false, 0x7);
+		if (ret)
+			return ret;
 	}
 
 	/* By default, disable all IRQs for S-mode of target HART */
 	if (s_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(plic, s_cntx_id, i, 0);
+		ret = plic_context_init(plic, m_cntx_id, false, 0x7);
+		if (ret)
+			return ret;
 	}
 
-	/* By default, disable M-mode threshold */
-	if (m_cntx_id > -1)
-		plic_set_thresh(plic, m_cntx_id, 0x7);
-
-	/* By default, disable S-mode threshold */
-	if (s_cntx_id > -1)
-		plic_set_thresh(plic, s_cntx_id, 0x7);
-
 	return 0;
 }
 
diff --git a/platform/fpga/ariane/platform.c b/platform/fpga/ariane/platform.c
index 3fdc03b..56a666b 100644
--- a/platform/fpga/ariane/platform.c
+++ b/platform/fpga/ariane/platform.c
@@ -99,24 +99,21 @@ static int ariane_console_init(void)
 
 static int plic_ariane_warm_irqchip_init(int m_cntx_id, int s_cntx_id)
 {
-	size_t i, ie_words = ARIANE_PLIC_NUM_SOURCES / 32 + 1;
+	int ret;
 
 	/* By default, enable all IRQs for M-mode of target HART */
 	if (m_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(&plic, m_cntx_id, i, 1);
+		ret = plic_context_init(&plic, m_cntx_id, true, 0x1);
+		if (ret)
+			return ret;
 	}
+
 	/* Enable all IRQs for S-mode of target HART */
 	if (s_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(&plic, s_cntx_id, i, 1);
+		ret = plic_context_init(&plic, s_cntx_id, true, 0x0);
+		if (ret)
+			return ret;
 	}
-	/* By default, enable M-mode threshold */
-	if (m_cntx_id > -1)
-		plic_set_thresh(&plic, m_cntx_id, 1);
-	/* By default, disable S-mode threshold */
-	if (s_cntx_id > -1)
-		plic_set_thresh(&plic, s_cntx_id, 0);
 
 	return 0;
 }
diff --git a/platform/fpga/openpiton/platform.c b/platform/fpga/openpiton/platform.c
index 15e289a..7ca2123 100644
--- a/platform/fpga/openpiton/platform.c
+++ b/platform/fpga/openpiton/platform.c
@@ -134,24 +134,21 @@ static int openpiton_console_init(void)
 
 static int plic_openpiton_warm_irqchip_init(int m_cntx_id, int s_cntx_id)
 {
-	size_t i, ie_words = plic.num_src / 32 + 1;
+	int ret;
 
 	/* By default, enable all IRQs for M-mode of target HART */
 	if (m_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(&plic, m_cntx_id, i, 1);
+		ret = plic_context_init(&plic, m_cntx_id, true, 0x1);
+		if (ret)
+			return ret;
 	}
+
 	/* Enable all IRQs for S-mode of target HART */
 	if (s_cntx_id > -1) {
-		for (i = 0; i < ie_words; i++)
-			plic_set_ie(&plic, s_cntx_id, i, 1);
+		ret = plic_context_init(&plic, s_cntx_id, true, 0x0);
+		if (ret)
+			return ret;
 	}
-	/* By default, enable M-mode threshold */
-	if (m_cntx_id > -1)
-		plic_set_thresh(&plic, m_cntx_id, 1);
-	/* By default, disable S-mode threshold */
-	if (s_cntx_id > -1)
-		plic_set_thresh(&plic, s_cntx_id, 0);
 
 	return 0;
 }
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:51 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:51 -0500
Subject: [PATCH v3 5/9] lib: irqchip/plic: Add context save/restore helpers
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-6-samuel@sholland.org>

These can be used by platform code to save the PLIC context state, if
it would otherwise be lost during non-retentive suspend. The platform
is responsible for allocating all necessary storage.

Reviewed-by: Anup Patel <anup at brainfault.org>
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

(no changes since v2)

Changes in v2:
 - New patch for v2

 include/sbi_utils/irqchip/plic.h |  6 ++++
 lib/utils/irqchip/plic.c         | 51 ++++++++++++++++++++++++++++++--
 2 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/include/sbi_utils/irqchip/plic.h b/include/sbi_utils/irqchip/plic.h
index 8f21af6..21b2266 100644
--- a/include/sbi_utils/irqchip/plic.h
+++ b/include/sbi_utils/irqchip/plic.h
@@ -17,6 +17,12 @@ struct plic_data {
 	unsigned long num_src;
 };
 
+void plic_context_save(const struct plic_data *plic, int context_id,
+		       u32 *enable, u32 *threshold);
+
+void plic_context_restore(const struct plic_data *plic, int context_id,
+			  const u32 *enable, u32 threshold);
+
 int plic_context_init(const struct plic_data *plic, int context_id,
 		      bool enable, u32 threshold);
 
diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
index 9bd3bf1..0c64078 100644
--- a/lib/utils/irqchip/plic.c
+++ b/lib/utils/irqchip/plic.c
@@ -29,6 +29,16 @@ static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
 	writel(val, plic_priority);
 }
 
+static u32 plic_get_thresh(const struct plic_data *plic, u32 cntxid)
+{
+	volatile void *plic_thresh;
+
+	plic_thresh = (char *)plic->addr +
+		      PLIC_CONTEXT_BASE + PLIC_CONTEXT_STRIDE * cntxid;
+
+	return readl(plic_thresh);
+}
+
 static void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
 {
 	volatile void *plic_thresh;
@@ -38,14 +48,49 @@ static void plic_set_thresh(const struct plic_data *plic, u32 cntxid, u32 val)
 	writel(val, plic_thresh);
 }
 
+static u32 plic_get_ie(const struct plic_data *plic, u32 cntxid,
+		       u32 word_index)
+{
+	volatile void *plic_ie;
+
+	plic_ie = (char *)plic->addr +
+		   PLIC_ENABLE_BASE + PLIC_ENABLE_STRIDE * cntxid +
+		   4 * word_index;
+
+	return readl(plic_ie);
+}
+
 static void plic_set_ie(const struct plic_data *plic, u32 cntxid,
 			u32 word_index, u32 val)
 {
-	volatile char *plic_ie;
+	volatile void *plic_ie;
 
 	plic_ie = (char *)plic->addr +
-		   PLIC_ENABLE_BASE + PLIC_ENABLE_STRIDE * cntxid;
-	writel(val, plic_ie + word_index * 4);
+		   PLIC_ENABLE_BASE + PLIC_ENABLE_STRIDE * cntxid +
+		   4 * word_index;
+	writel(val, plic_ie);
+}
+
+void plic_context_save(const struct plic_data *plic, int context_id,
+		       u32 *enable, u32 *threshold)
+{
+	u32 ie_words = (plic->num_src + 31) / 32;
+
+	for (u32 i = 0; i < ie_words; i++)
+		enable[i] = plic_get_ie(plic, context_id, i);
+
+	*threshold = plic_get_thresh(plic, context_id);
+}
+
+void plic_context_restore(const struct plic_data *plic, int context_id,
+			  const u32 *enable, u32 threshold)
+{
+	u32 ie_words = (plic->num_src + 31) / 32;
+
+	for (u32 i = 0; i < ie_words; i++)
+		plic_set_ie(plic, context_id, i, enable[i]);
+
+	plic_set_thresh(plic, context_id, threshold);
 }
 
 int plic_context_init(const struct plic_data *plic, int context_id,
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:52 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:52 -0500
Subject: [PATCH v3 6/9] lib: irqchip/plic: Add priority save/restore helpers
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-7-samuel@sholland.org>

These can be used by platform code to save the PLIC priority state, if
it would otherwise be lost during non-retentive suspend. The platform
is responsible for allocating all necessary storage.

As a space optimization, store the saved priority values as 8-bit
integers, since that is large enough to hold any priority value on the
relevant platforms.

Reviewed-by: Anup Patel <anup at brainfault.org>
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

Changes in v3:
 - Move comment about priority range to function declaration

Changes in v2:
 - New patch for v2

 include/sbi_utils/irqchip/plic.h |  5 +++++
 lib/utils/irqchip/plic.c         | 19 +++++++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/include/sbi_utils/irqchip/plic.h b/include/sbi_utils/irqchip/plic.h
index 21b2266..48c24f0 100644
--- a/include/sbi_utils/irqchip/plic.h
+++ b/include/sbi_utils/irqchip/plic.h
@@ -17,6 +17,11 @@ struct plic_data {
 	unsigned long num_src;
 };
 
+/* So far, priorities on all consumers of these functions fit in 8 bits. */
+void plic_priority_save(const struct plic_data *plic, u8 *priority);
+
+void plic_priority_restore(const struct plic_data *plic, const u8 *priority);
+
 void plic_context_save(const struct plic_data *plic, int context_id,
 		       u32 *enable, u32 *threshold);
 
diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
index 0c64078..8c25b9d 100644
--- a/lib/utils/irqchip/plic.c
+++ b/lib/utils/irqchip/plic.c
@@ -22,6 +22,13 @@
 #define PLIC_CONTEXT_BASE 0x200000
 #define PLIC_CONTEXT_STRIDE 0x1000
 
+static u32 plic_get_priority(const struct plic_data *plic, u32 source)
+{
+	volatile void *plic_priority = (char *)plic->addr +
+			PLIC_PRIORITY_BASE + 4 * source;
+	return readl(plic_priority);
+}
+
 static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
 {
 	volatile void *plic_priority = (char *)plic->addr +
@@ -29,6 +36,18 @@ static void plic_set_priority(const struct plic_data *plic, u32 source, u32 val)
 	writel(val, plic_priority);
 }
 
+void plic_priority_save(const struct plic_data *plic, u8 *priority)
+{
+	for (u32 i = 0; i < plic->num_src; i++)
+		priority[i] = plic_get_priority(plic, i);
+}
+
+void plic_priority_restore(const struct plic_data *plic, const u8 *priority)
+{
+	for (u32 i = 0; i < plic->num_src; i++)
+		plic_set_priority(plic, i, priority[i]);
+}
+
 static u32 plic_get_thresh(const struct plic_data *plic, u32 cntxid)
 {
 	volatile void *plic_thresh;
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:53 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:53 -0500
Subject: [PATCH v3 7/9] lib: utils/irqchip: Add FDT wrappers for PLIC save/restore functions
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-8-samuel@sholland.org>

These functions save/restore the state of the PLIC associated with the
current hart. The context save/restore functions only manipulate a
single context, since most likely the M-mode context is unused and does
not need to be saved.

Signed-off-by: Samuel Holland <samuel at sholland.org>
---

Changes in v3:
 - Rename irqchip_plic_* functions to fdt_plic_*
 - Add a "bool smode" parameter instead of hard coding the context ID

Changes in v2:
 - New patch for v2

 include/sbi_utils/irqchip/fdt_irqchip_plic.h | 20 ++++++++++++
 lib/utils/irqchip/fdt_irqchip_plic.c         | 32 ++++++++++++++++++++
 2 files changed, 52 insertions(+)
 create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h

diff --git a/include/sbi_utils/irqchip/fdt_irqchip_plic.h b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
new file mode 100644
index 0000000..f158b02
--- /dev/null
+++ b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
@@ -0,0 +1,20 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 Samuel Holland <samuel at sholland.org>
+ */
+
+#ifndef __IRQCHIP_FDT_IRQCHIP_PLIC_H__
+#define __IRQCHIP_FDT_IRQCHIP_PLIC_H__
+
+#include <sbi/sbi_types.h>
+
+void fdt_plic_priority_save(u8 *priority);
+
+void fdt_plic_priority_restore(const u8 *priority);
+
+void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold);
+
+void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold);
+
+#endif
diff --git a/lib/utils/irqchip/fdt_irqchip_plic.c b/lib/utils/irqchip/fdt_irqchip_plic.c
index b2d2a1b..89b12d4 100644
--- a/lib/utils/irqchip/fdt_irqchip_plic.c
+++ b/lib/utils/irqchip/fdt_irqchip_plic.c
@@ -24,6 +24,38 @@ static struct plic_data plic[PLIC_MAX_NR];
 static struct plic_data *plic_hartid2data[SBI_HARTMASK_MAX_BITS];
 static int plic_hartid2context[SBI_HARTMASK_MAX_BITS][2];
 
+void fdt_plic_priority_save(u8 *priority)
+{
+	struct plic_data *plic = plic_hartid2data[current_hartid()];
+
+	plic_priority_save(plic, priority);
+}
+
+void fdt_plic_priority_restore(const u8 *priority)
+{
+	struct plic_data *plic = plic_hartid2data[current_hartid()];
+
+	plic_priority_restore(plic, priority);
+}
+
+void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold)
+{
+	u32 hartid = current_hartid();
+
+	plic_context_save(plic_hartid2data[hartid],
+			  plic_hartid2context[hartid][smode],
+			  enable, threshold);
+}
+
+void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold)
+{
+	u32 hartid = current_hartid();
+
+	plic_context_restore(plic_hartid2data[hartid],
+			     plic_hartid2context[hartid][smode],
+			     enable, threshold);
+}
+
 static int irqchip_plic_warm_init(void)
 {
 	u32 hartid = current_hartid();
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:54 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:54 -0500
Subject: [PATCH v3 8/9] lib: utils/irqchip: Add wrapper for T-HEAD PLIC delegation
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-9-samuel@sholland.org>

The delegation bit is lost along with the rest of the PLIC state when
the CPU power domain in the Allwinner D1 is powered down, so the PLIC
needs to be re-delegated to S-mode during the hart resume path.

Reviewed-by: Anup Patel <anup at brainfault.org>
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

(no changes since v2)

Changes in v2:
 - New patch for v2

 include/sbi_utils/irqchip/fdt_irqchip_plic.h | 2 ++
 lib/utils/irqchip/fdt_irqchip_plic.c         | 7 +++++++
 2 files changed, 9 insertions(+)

diff --git a/include/sbi_utils/irqchip/fdt_irqchip_plic.h b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
index f158b02..98d4de5 100644
--- a/include/sbi_utils/irqchip/fdt_irqchip_plic.h
+++ b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
@@ -17,4 +17,6 @@ void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold);
 
 void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold);
 
+void thead_plic_restore(void);
+
 #endif
diff --git a/lib/utils/irqchip/fdt_irqchip_plic.c b/lib/utils/irqchip/fdt_irqchip_plic.c
index 89b12d4..a6e185c 100644
--- a/lib/utils/irqchip/fdt_irqchip_plic.c
+++ b/lib/utils/irqchip/fdt_irqchip_plic.c
@@ -151,6 +151,13 @@ static void thead_plic_plat_init(struct plic_data *pd)
 	writel_relaxed(BIT(0), (char *)pd->addr + THEAD_PLIC_CTRL_REG);
 }
 
+void thead_plic_restore(void)
+{
+	struct plic_data *plic = plic_hartid2data[current_hartid()];
+
+	thead_plic_plat_init(plic);
+}
+
 static const struct fdt_match irqchip_plic_match[] = {
 	{ .compatible = "riscv,plic0" },
 	{ .compatible = "sifive,plic-1.0.0" },
-- 
2.35.1



From samuel at sholland.org  Sun Jun 12 18:03:55 2022
From: samuel at sholland.org (Samuel Holland)
Date: Sun, 12 Jun 2022 20:03:55 -0500
Subject: [PATCH v3 9/9] platform: Add HSM implementation for Allwinner D1
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <20220613010355.28014-10-samuel@sholland.org>

Allwinner D1 contains a "PPU" power domain controller which can
automatically power down/up the CPU power domain. This power domain
includes the C906 core along with its CLINT and PLIC.

This HSM implementation supports non-retentive hart suspend by:
  1) Saving/restoring state that is lost during hart suspend,
  2) Performing cache maintenance before/after hart suspend,
  3) Configuring wakeup sources before hart suspend, and
  4) Asking the PPU to power down the hart when it enters WFI.

Since this HSM implementation is for a single-core SoC, it does not need
to worry about concurrency or saving multiple instances of state.

Reviewed-by: Anup Patel <anup at brainfault.org>
Signed-off-by: Samuel Holland <samuel at sholland.org>
---

(no changes since v2)

Changes in v2:
 - Rebase on top of the current master branch
 - Use array_size() from sbi_types.h
 - Use a separate variable for each CSR instead of an array
 - Use the PLIC helpers added in earlier patches
 - Use the saved SIE bits to program the wakeup interrupt mask
 - Set the entry address only once during cold boot
 - Use the generic implementation for retentive suspend
 - Clean up/add comments throughout the HSM implementation

 platform/generic/allwinner/objects.mk  |   6 +
 platform/generic/allwinner/sun20i-d1.c | 210 +++++++++++++++++++++++++
 2 files changed, 216 insertions(+)
 create mode 100644 platform/generic/allwinner/objects.mk
 create mode 100644 platform/generic/allwinner/sun20i-d1.c

diff --git a/platform/generic/allwinner/objects.mk b/platform/generic/allwinner/objects.mk
new file mode 100644
index 0000000..9e36ab6
--- /dev/null
+++ b/platform/generic/allwinner/objects.mk
@@ -0,0 +1,6 @@
+#
+# SPDX-License-Identifier: BSD-2-Clause
+#
+
+carray-platform_override_modules-y += sun20i_d1
+platform-objs-y += allwinner/sun20i-d1.o
diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
new file mode 100644
index 0000000..2de12e4
--- /dev/null
+++ b/platform/generic/allwinner/sun20i-d1.c
@@ -0,0 +1,210 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright (c) 2022 Samuel Holland <samuel at sholland.org>
+ */
+
+#include <platform_override.h>
+#include <sbi/riscv_io.h>
+#include <sbi/sbi_bitops.h>
+#include <sbi/sbi_ecall_interface.h>
+#include <sbi/sbi_error.h>
+#include <sbi/sbi_hsm.h>
+#include <sbi_utils/fdt/fdt_helper.h>
+#include <sbi_utils/irqchip/fdt_irqchip_plic.h>
+
+#define SUN20I_D1_CCU_BASE		((void *)0x02001000)
+#define SUN20I_D1_RISCV_CFG_BASE	((void *)0x06010000)
+#define SUN20I_D1_PPU_BASE		((void *)0x07001000)
+#define SUN20I_D1_PRCM_BASE		((void *)0x07010000)
+
+/*
+ * CCU
+ */
+
+#define CCU_BGR_ENABLE			(BIT(16) | BIT(0))
+
+#define RISCV_CFG_BGR_REG		0xd0c
+#define PPU_BGR_REG			0x1ac
+
+/*
+ * CSRs
+ */
+
+#define CSR_MXSTATUS			0x7c0
+#define CSR_MHCR			0x7c1
+#define CSR_MCOR			0x7c2
+#define CSR_MHINT			0x7c5
+
+static unsigned long csr_mxstatus;
+static unsigned long csr_mhcr;
+static unsigned long csr_mhint;
+
+static void sun20i_d1_csr_save(void)
+{
+	/* Save custom CSRs. */
+	csr_mxstatus	= csr_read(CSR_MXSTATUS);
+	csr_mhcr	= csr_read(CSR_MHCR);
+	csr_mhint	= csr_read(CSR_MHINT);
+
+	/* Flush and disable caches. */
+	csr_write(CSR_MCOR, 0x22);
+	csr_write(CSR_MHCR, 0x0);
+}
+
+static void sun20i_d1_csr_restore(void)
+{
+	/* Invalidate caches and the branch predictor. */
+	csr_write(CSR_MCOR, 0x70013);
+
+	/* Restore custom CSRs, including the cache state. */
+	csr_write(CSR_MXSTATUS,	csr_mxstatus);
+	csr_write(CSR_MHCR,	csr_mhcr);
+	csr_write(CSR_MHINT,	csr_mhint);
+}
+
+/*
+ * PLIC
+ */
+
+#define PLIC_SOURCES			176
+#define PLIC_IE_WORDS			((PLIC_SOURCES + 31) / 32)
+
+static u8 plic_priority[PLIC_SOURCES];
+static u32 plic_sie[PLIC_IE_WORDS];
+static u32 plic_threshold;
+
+static void sun20i_d1_plic_save(void)
+{
+	fdt_plic_context_save(true, plic_sie, &plic_threshold);
+	fdt_plic_priority_save(plic_priority);
+}
+
+static void sun20i_d1_plic_restore(void)
+{
+	thead_plic_restore();
+	fdt_plic_priority_restore(plic_priority);
+	fdt_plic_context_restore(true, plic_sie, plic_threshold);
+}
+
+/*
+ * PPU
+ */
+
+#define PPU_PD_ACTIVE_CTRL		0x2c
+
+static void sun20i_d1_ppu_save(void)
+{
+	/* Enable MMIO access. Do not assume S-mode leaves the clock enabled. */
+	writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_PRCM_BASE + PPU_BGR_REG);
+
+	/* Activate automatic power-down during the next WFI. */
+	writel_relaxed(1, SUN20I_D1_PPU_BASE + PPU_PD_ACTIVE_CTRL);
+}
+
+static void sun20i_d1_ppu_restore(void)
+{
+	/* Disable automatic power-down. */
+	writel_relaxed(0, SUN20I_D1_PPU_BASE + PPU_PD_ACTIVE_CTRL);
+}
+
+/*
+ * RISCV_CFG
+ */
+
+#define RESET_ENTRY_LO_REG		0x0004
+#define RESET_ENTRY_HI_REG		0x0008
+#define WAKEUP_EN_REG			0x0020
+#define WAKEUP_MASK_REG(i)		(0x0024 + 4 * (i))
+
+static void sun20i_d1_riscv_cfg_save(void)
+{
+	/* Enable MMIO access. Do not assume S-mode leaves the clock enabled. */
+	writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_CCU_BASE + RISCV_CFG_BGR_REG);
+
+	/*
+	 * Copy the SIE bits to the wakeup registers. D1 has 160 "real"
+	 * interrupt sources, numbered 16-175. These are the ones that map to
+	 * the wakeup mask registers (the offset is for GIC compatibility). So
+	 * copying SIE to the wakeup mask needs some bit manipulation.
+	 */
+	for (int i = 0; i < PLIC_IE_WORDS - 1; i++)
+		writel_relaxed(plic_sie[i] >> 16 | plic_sie[i + 1] << 16,
+			       SUN20I_D1_RISCV_CFG_BASE + WAKEUP_MASK_REG(i));
+
+	/* Enable PPU wakeup for interrupts. */
+	writel_relaxed(1, SUN20I_D1_RISCV_CFG_BASE + WAKEUP_EN_REG);
+}
+
+static void sun20i_d1_riscv_cfg_restore(void)
+{
+	/* Disable PPU wakeup for interrupts. */
+	writel_relaxed(0, SUN20I_D1_RISCV_CFG_BASE + WAKEUP_EN_REG);
+}
+
+static void sun20i_d1_riscv_cfg_init(void)
+{
+	unsigned long entry = sbi_hartid_to_scratch(0)->warmboot_addr;
+
+	/* Enable MMIO access. */
+	writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_CCU_BASE + RISCV_CFG_BGR_REG);
+
+	/* Program the reset entry address. */
+	writel_relaxed((u32)entry, SUN20I_D1_RISCV_CFG_BASE + RESET_ENTRY_LO_REG);
+	writel_relaxed(entry >> 32, SUN20I_D1_RISCV_CFG_BASE + RESET_ENTRY_HI_REG);
+}
+
+static int sun20i_d1_hart_suspend(u32 suspend_type)
+{
+	/* Use the generic code for retentive suspend. */
+	if (!(suspend_type & SBI_HSM_SUSP_NON_RET_BIT))
+		return SBI_ENOTSUPP;
+
+	sun20i_d1_plic_save();
+	sun20i_d1_ppu_save();
+	sun20i_d1_riscv_cfg_save();
+	sun20i_d1_csr_save();
+
+	/*
+	 * If no interrupt is pending, this will power down the CPU power
+	 * domain. Otherwise, this will fall through, and the generic HSM
+	 * code will jump to the resume address.
+	 */
+	wfi();
+
+	return 0;
+}
+
+static void sun20i_d1_hart_resume(void)
+{
+	sun20i_d1_csr_restore();
+	sun20i_d1_riscv_cfg_restore();
+	sun20i_d1_ppu_restore();
+	sun20i_d1_plic_restore();
+}
+
+static const struct sbi_hsm_device sun20i_d1_ppu = {
+	.name		= "sun20i-d1-ppu",
+	.hart_suspend	= sun20i_d1_hart_suspend,
+	.hart_resume	= sun20i_d1_hart_resume,
+};
+
+static int sun20i_d1_final_init(bool cold_boot, const struct fdt_match *match)
+{
+	if (cold_boot) {
+		sun20i_d1_riscv_cfg_init();
+		sbi_hsm_set_device(&sun20i_d1_ppu);
+	}
+
+	return 0;
+}
+
+static const struct fdt_match sun20i_d1_match[] = {
+	{ .compatible = "allwinner,sun20i-d1" },
+	{ },
+};
+
+const struct platform_override sun20i_d1 = {
+	.match_table	= sun20i_d1_match,
+	.final_init	= sun20i_d1_final_init,
+};
-- 
2.35.1



From anup at brainfault.org  Sun Jun 12 22:46:47 2022
From: anup at brainfault.org (Anup Patel)
Date: Mon, 13 Jun 2022 11:16:47 +0530
Subject: [PATCH v2] lib: sbi_trap: Redirect exception based on hedeleg
In-Reply-To: <TYYP286MB1439777CF535CC8DF8388B0CC6A89@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB14397827A57ED2BA6AA100E3C6A49@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <TYYP286MB1439777CF535CC8DF8388B0CC6A89@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAAhSdy03B36sg3yUoYRpVA6RyaiKBR1uD0JzBxBcE0fuqr_fhg@mail.gmail.com>

On Sun, Jun 12, 2022 at 11:31 AM dramforever <dramforever at live.com> wrote:
>
> HS-mode software can choose what exceptions to delegate to VS-mode using
> the hedeleg CSR. Synthetic VS/VU-mode exceptions should also honor
> hedeleg. They should be redirected to VS-mode if and only if delegated
> by HS-mode.
>
> Signed-off-by: dramforever <dramforever at live.com>

When posting new patch revisions, please add here "Reviewed-by" tags
already obtained in previous revisions. This helps reviewers to know
which patches are already reviewed.

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>
> Changes since v1:
>  - Use BIT() macro instead of (1 << trap->cause)
>
>  lib/sbi/sbi_trap.c | 15 ++++++---------
>  1 file changed, 6 insertions(+), 9 deletions(-)
>
> diff --git a/lib/sbi/sbi_trap.c b/lib/sbi/sbi_trap.c
> index 2c509e5..a54a131 100644
> --- a/lib/sbi/sbi_trap.c
> +++ b/lib/sbi/sbi_trap.c
> @@ -99,17 +99,14 @@ int sbi_trap_redirect(struct sbi_trap_regs *regs,
>         if (prev_mode != PRV_S && prev_mode != PRV_U)
>                 return SBI_ENOTSUPP;
>
> -       /* For certain exceptions from VS/VU-mode we redirect to VS-mode */
> +       /* If exceptions came from VS/VU-mode, redirect to VS-mode if
> +        * delegated in hedeleg
> +        */
>         if (misa_extension('H') && prev_virt) {
> -               switch (trap->cause) {
> -               case CAUSE_FETCH_PAGE_FAULT:
> -               case CAUSE_LOAD_PAGE_FAULT:
> -               case CAUSE_STORE_PAGE_FAULT:
> +               if (trap->cause < __riscv_xlen &&
> +                   csr_read(CSR_HEDELEG) & BIT(trap->cause)) {
>                         next_virt = TRUE;
> -                       break;
> -               default:
> -                       break;
> -               };
> +               }
>         }
>
>         /* Update MSTATUS MPV bits */
> --
> 2.36.1
>


From anup at brainfault.org  Sun Jun 12 23:11:50 2022
From: anup at brainfault.org (Anup Patel)
Date: Mon, 13 Jun 2022 11:41:50 +0530
Subject: [PATCH] platform: generic: add overrides for vendor extensions
In-Reply-To: <CAAhSdy3hRji-urK15oYTduEijVGUBPYi8XR4X3AY+uzzTLkXnQ@mail.gmail.com>
References: <20220610044920.547047-1-mchitale@ventanamicro.com> <CAAhSdy3hRji-urK15oYTduEijVGUBPYi8XR4X3AY+uzzTLkXnQ@mail.gmail.com>
Message-ID: <CAAhSdy0PmmG7V9upm9WCurkFmuY3Y0dhn-gZJWECwEoq5U0Whw@mail.gmail.com>

On Sun, Jun 12, 2022 at 10:00 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Fri, Jun 10, 2022 at 10:20 AM Mayuresh Chitale
> <mchitale at ventanamicro.com> wrote:
> >
> > Allow the vendor_ext_check and vendor_ext_provider APIs of the
> > generic platform to be overridden by other platforms
> >
> > Signed-off-by: Mayuresh Chitale <mchitale at ventanamicro.com>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
> Regards,
> Anup
>
> > ---
> >  platform/generic/include/platform_override.h |  7 ++++++
> >  platform/generic/platform.c                  | 25 ++++++++++++++++++++
> >  2 files changed, 32 insertions(+)
> >
> > diff --git a/platform/generic/include/platform_override.h b/platform/generic/include/platform_override.h
> > index 4af8754..e55da25 100644
> > --- a/platform/generic/include/platform_override.h
> > +++ b/platform/generic/include/platform_override.h
> > @@ -11,6 +11,7 @@
> >  #define __PLATFORM_OVERRIDE_H__
> >
> >  #include <sbi/sbi_types.h>
> > +#include <sbi/sbi_trap.h>
> >
> >  struct platform_override {
> >         const struct fdt_match *match_table;
> > @@ -21,6 +22,12 @@ struct platform_override {
> >         void (*early_exit)(const struct fdt_match *match);
> >         void (*final_exit)(const struct fdt_match *match);
> >         int (*fdt_fixup)(void *fdt, const struct fdt_match *match);
> > +       int (*vendor_ext_check)(long extid, const struct fdt_match *match);
> > +       int (*vendor_ext_provider)(long extid, long funcid,
> > +                                  const struct sbi_trap_regs *regs,
> > +                                  unsigned long *out_value,
> > +                                  struct sbi_trap_info *out_trap,
> > +                                  const struct fdt_match *match);
> >  };
> >
> >  #endif
> > diff --git a/platform/generic/platform.c b/platform/generic/platform.c
> > index 7651273..bf93af6 100644
> > --- a/platform/generic/platform.c
> > +++ b/platform/generic/platform.c
> > @@ -168,6 +168,29 @@ static int generic_final_init(bool cold_boot)
> >         return 0;
> >  }
> >
> > +static int generic_vendor_ext_check(long extid)
> > +{
> > +       if (generic_plat && generic_plat->vendor_ext_check)
> > +               return generic_plat->vendor_ext_check(extid,
> > +                                                     generic_plat_match);
> > +
> > +       return 0;
> > +}
> > +
> > +static int generic_vendor_ext_provider(long extid, long funcid,
> > +                                      const struct sbi_trap_regs *regs,
> > +                                      unsigned long *out_value,
> > +                                      struct sbi_trap_info *out_trap)
> > +{
> > +       if (generic_plat && generic_plat->vendor_ext_provider) {
> > +               return generic_plat->vendor_ext_provider(extid, funcid, regs,
> > +                                                        out_value, out_trap,
> > +                                                        generic_plat_match);
> > +       }
> > +
> > +       return SBI_ENOTSUPP;
> > +}
> > +
> >  static void generic_early_exit(void)
> >  {
> >         if (generic_plat && generic_plat->early_exit)
> > @@ -236,6 +259,8 @@ const struct sbi_platform_operations platform_ops = {
> >         .get_tlbr_flush_limit   = generic_tlbr_flush_limit,
> >         .timer_init             = fdt_timer_init,
> >         .timer_exit             = fdt_timer_exit,
> > +       .vendor_ext_check       = generic_vendor_ext_check,
> > +       .vendor_ext_provider    = generic_vendor_ext_provider,
> >  };
> >
> >  struct sbi_platform platform = {
> > --
> > 2.25.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sun Jun 12 23:20:57 2022
From: anup at brainfault.org (Anup Patel)
Date: Mon, 13 Jun 2022 11:50:57 +0530
Subject: [PATCH v3 7/9] lib: utils/irqchip: Add FDT wrappers for PLIC
 save/restore functions
In-Reply-To: <20220613010355.28014-8-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org> <20220613010355.28014-8-samuel@sholland.org>
Message-ID: <CAAhSdy2TNtn1Tw4CM0oMOMMybVr3_p0euY49cAkoYLym9E1Szg@mail.gmail.com>

On Mon, Jun 13, 2022 at 6:34 AM Samuel Holland <samuel at sholland.org> wrote:
>
> These functions save/restore the state of the PLIC associated with the
> current hart. The context save/restore functions only manipulate a
> single context, since most likely the M-mode context is unused and does
> not need to be saved.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>
> Changes in v3:
>  - Rename irqchip_plic_* functions to fdt_plic_*
>  - Add a "bool smode" parameter instead of hard coding the context ID
>
> Changes in v2:
>  - New patch for v2
>
>  include/sbi_utils/irqchip/fdt_irqchip_plic.h | 20 ++++++++++++
>  lib/utils/irqchip/fdt_irqchip_plic.c         | 32 ++++++++++++++++++++
>  2 files changed, 52 insertions(+)
>  create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
>
> diff --git a/include/sbi_utils/irqchip/fdt_irqchip_plic.h b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> new file mode 100644
> index 0000000..f158b02
> --- /dev/null
> +++ b/include/sbi_utils/irqchip/fdt_irqchip_plic.h
> @@ -0,0 +1,20 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Samuel Holland <samuel at sholland.org>
> + */
> +
> +#ifndef __IRQCHIP_FDT_IRQCHIP_PLIC_H__
> +#define __IRQCHIP_FDT_IRQCHIP_PLIC_H__
> +
> +#include <sbi/sbi_types.h>
> +
> +void fdt_plic_priority_save(u8 *priority);
> +
> +void fdt_plic_priority_restore(const u8 *priority);
> +
> +void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold);
> +
> +void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold);
> +
> +#endif
> diff --git a/lib/utils/irqchip/fdt_irqchip_plic.c b/lib/utils/irqchip/fdt_irqchip_plic.c
> index b2d2a1b..89b12d4 100644
> --- a/lib/utils/irqchip/fdt_irqchip_plic.c
> +++ b/lib/utils/irqchip/fdt_irqchip_plic.c
> @@ -24,6 +24,38 @@ static struct plic_data plic[PLIC_MAX_NR];
>  static struct plic_data *plic_hartid2data[SBI_HARTMASK_MAX_BITS];
>  static int plic_hartid2context[SBI_HARTMASK_MAX_BITS][2];
>
> +void fdt_plic_priority_save(u8 *priority)
> +{
> +       struct plic_data *plic = plic_hartid2data[current_hartid()];
> +
> +       plic_priority_save(plic, priority);
> +}
> +
> +void fdt_plic_priority_restore(const u8 *priority)
> +{
> +       struct plic_data *plic = plic_hartid2data[current_hartid()];
> +
> +       plic_priority_restore(plic, priority);
> +}
> +
> +void fdt_plic_context_save(bool smode, u32 *enable, u32 *threshold)
> +{
> +       u32 hartid = current_hartid();
> +
> +       plic_context_save(plic_hartid2data[hartid],
> +                         plic_hartid2context[hartid][smode],
> +                         enable, threshold);
> +}
> +
> +void fdt_plic_context_restore(bool smode, const u32 *enable, u32 threshold)
> +{
> +       u32 hartid = current_hartid();
> +
> +       plic_context_restore(plic_hartid2data[hartid],
> +                            plic_hartid2context[hartid][smode],
> +                            enable, threshold);
> +}
> +
>  static int irqchip_plic_warm_init(void)
>  {
>         u32 hartid = current_hartid();
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From apatel at ventanamicro.com  Sun Jun 12 23:42:11 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 13 Jun 2022 12:12:11 +0530
Subject: [PATCH v3 9/9] platform: Add HSM implementation for Allwinner D1
In-Reply-To: <20220613010355.28014-10-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org> <20220613010355.28014-10-samuel@sholland.org>
Message-ID: <CAK9=C2VhtLSz-2pFmHCr+g9SC0A=Y1FS1gag6q=UOBqHhTaKvg@mail.gmail.com>

On Mon, Jun 13, 2022 at 6:34 AM Samuel Holland <samuel at sholland.org> wrote:
>
> Allwinner D1 contains a "PPU" power domain controller which can
> automatically power down/up the CPU power domain. This power domain
> includes the C906 core along with its CLINT and PLIC.
>
> This HSM implementation supports non-retentive hart suspend by:
>   1) Saving/restoring state that is lost during hart suspend,
>   2) Performing cache maintenance before/after hart suspend,
>   3) Configuring wakeup sources before hart suspend, and
>   4) Asking the PPU to power down the hart when it enters WFI.
>
> Since this HSM implementation is for a single-core SoC, it does not need
> to worry about concurrency or saving multiple instances of state.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>
> Signed-off-by: Samuel Holland <samuel at sholland.org>
> ---
>
> (no changes since v2)
>
> Changes in v2:
>  - Rebase on top of the current master branch
>  - Use array_size() from sbi_types.h
>  - Use a separate variable for each CSR instead of an array
>  - Use the PLIC helpers added in earlier patches
>  - Use the saved SIE bits to program the wakeup interrupt mask
>  - Set the entry address only once during cold boot
>  - Use the generic implementation for retentive suspend
>  - Clean up/add comments throughout the HSM implementation
>
>  platform/generic/allwinner/objects.mk  |   6 +
>  platform/generic/allwinner/sun20i-d1.c | 210 +++++++++++++++++++++++++
>  2 files changed, 216 insertions(+)
>  create mode 100644 platform/generic/allwinner/objects.mk
>  create mode 100644 platform/generic/allwinner/sun20i-d1.c
>
> diff --git a/platform/generic/allwinner/objects.mk b/platform/generic/allwinner/objects.mk
> new file mode 100644
> index 0000000..9e36ab6
> --- /dev/null
> +++ b/platform/generic/allwinner/objects.mk
> @@ -0,0 +1,6 @@
> +#
> +# SPDX-License-Identifier: BSD-2-Clause
> +#
> +
> +carray-platform_override_modules-y += sun20i_d1
> +platform-objs-y += allwinner/sun20i-d1.o
> diff --git a/platform/generic/allwinner/sun20i-d1.c b/platform/generic/allwinner/sun20i-d1.c
> new file mode 100644
> index 0000000..2de12e4
> --- /dev/null
> +++ b/platform/generic/allwinner/sun20i-d1.c
> @@ -0,0 +1,210 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2022 Samuel Holland <samuel at sholland.org>
> + */
> +
> +#include <platform_override.h>
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_bitops.h>
> +#include <sbi/sbi_ecall_interface.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_hsm.h>
> +#include <sbi_utils/fdt/fdt_helper.h>
> +#include <sbi_utils/irqchip/fdt_irqchip_plic.h>
> +
> +#define SUN20I_D1_CCU_BASE             ((void *)0x02001000)
> +#define SUN20I_D1_RISCV_CFG_BASE       ((void *)0x06010000)
> +#define SUN20I_D1_PPU_BASE             ((void *)0x07001000)
> +#define SUN20I_D1_PRCM_BASE            ((void *)0x07010000)
> +
> +/*
> + * CCU
> + */
> +
> +#define CCU_BGR_ENABLE                 (BIT(16) | BIT(0))
> +
> +#define RISCV_CFG_BGR_REG              0xd0c
> +#define PPU_BGR_REG                    0x1ac
> +
> +/*
> + * CSRs
> + */
> +
> +#define CSR_MXSTATUS                   0x7c0
> +#define CSR_MHCR                       0x7c1
> +#define CSR_MCOR                       0x7c2
> +#define CSR_MHINT                      0x7c5
> +
> +static unsigned long csr_mxstatus;
> +static unsigned long csr_mhcr;
> +static unsigned long csr_mhint;
> +
> +static void sun20i_d1_csr_save(void)
> +{
> +       /* Save custom CSRs. */
> +       csr_mxstatus    = csr_read(CSR_MXSTATUS);
> +       csr_mhcr        = csr_read(CSR_MHCR);
> +       csr_mhint       = csr_read(CSR_MHINT);
> +
> +       /* Flush and disable caches. */
> +       csr_write(CSR_MCOR, 0x22);
> +       csr_write(CSR_MHCR, 0x0);
> +}
> +
> +static void sun20i_d1_csr_restore(void)
> +{
> +       /* Invalidate caches and the branch predictor. */
> +       csr_write(CSR_MCOR, 0x70013);
> +
> +       /* Restore custom CSRs, including the cache state. */
> +       csr_write(CSR_MXSTATUS, csr_mxstatus);
> +       csr_write(CSR_MHCR,     csr_mhcr);
> +       csr_write(CSR_MHINT,    csr_mhint);
> +}
> +
> +/*
> + * PLIC
> + */
> +
> +#define PLIC_SOURCES                   176
> +#define PLIC_IE_WORDS                  ((PLIC_SOURCES + 31) / 32)
> +
> +static u8 plic_priority[PLIC_SOURCES];
> +static u32 plic_sie[PLIC_IE_WORDS];
> +static u32 plic_threshold;
> +
> +static void sun20i_d1_plic_save(void)
> +{
> +       fdt_plic_context_save(true, plic_sie, &plic_threshold);
> +       fdt_plic_priority_save(plic_priority);
> +}
> +
> +static void sun20i_d1_plic_restore(void)
> +{
> +       thead_plic_restore();
> +       fdt_plic_priority_restore(plic_priority);
> +       fdt_plic_context_restore(true, plic_sie, plic_threshold);
> +}
> +
> +/*
> + * PPU
> + */
> +
> +#define PPU_PD_ACTIVE_CTRL             0x2c
> +
> +static void sun20i_d1_ppu_save(void)
> +{
> +       /* Enable MMIO access. Do not assume S-mode leaves the clock enabled. */
> +       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_PRCM_BASE + PPU_BGR_REG);
> +
> +       /* Activate automatic power-down during the next WFI. */
> +       writel_relaxed(1, SUN20I_D1_PPU_BASE + PPU_PD_ACTIVE_CTRL);
> +}
> +
> +static void sun20i_d1_ppu_restore(void)
> +{
> +       /* Disable automatic power-down. */
> +       writel_relaxed(0, SUN20I_D1_PPU_BASE + PPU_PD_ACTIVE_CTRL);
> +}
> +
> +/*
> + * RISCV_CFG
> + */
> +
> +#define RESET_ENTRY_LO_REG             0x0004
> +#define RESET_ENTRY_HI_REG             0x0008
> +#define WAKEUP_EN_REG                  0x0020
> +#define WAKEUP_MASK_REG(i)             (0x0024 + 4 * (i))
> +
> +static void sun20i_d1_riscv_cfg_save(void)
> +{
> +       /* Enable MMIO access. Do not assume S-mode leaves the clock enabled. */
> +       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_CCU_BASE + RISCV_CFG_BGR_REG);
> +
> +       /*
> +        * Copy the SIE bits to the wakeup registers. D1 has 160 "real"
> +        * interrupt sources, numbered 16-175. These are the ones that map to
> +        * the wakeup mask registers (the offset is for GIC compatibility). So
> +        * copying SIE to the wakeup mask needs some bit manipulation.
> +        */
> +       for (int i = 0; i < PLIC_IE_WORDS - 1; i++)
> +               writel_relaxed(plic_sie[i] >> 16 | plic_sie[i + 1] << 16,
> +                              SUN20I_D1_RISCV_CFG_BASE + WAKEUP_MASK_REG(i));
> +
> +       /* Enable PPU wakeup for interrupts. */
> +       writel_relaxed(1, SUN20I_D1_RISCV_CFG_BASE + WAKEUP_EN_REG);
> +}
> +
> +static void sun20i_d1_riscv_cfg_restore(void)
> +{
> +       /* Disable PPU wakeup for interrupts. */
> +       writel_relaxed(0, SUN20I_D1_RISCV_CFG_BASE + WAKEUP_EN_REG);
> +}
> +
> +static void sun20i_d1_riscv_cfg_init(void)
> +{
> +       unsigned long entry = sbi_hartid_to_scratch(0)->warmboot_addr;
> +
> +       /* Enable MMIO access. */
> +       writel_relaxed(CCU_BGR_ENABLE, SUN20I_D1_CCU_BASE + RISCV_CFG_BGR_REG);
> +
> +       /* Program the reset entry address. */
> +       writel_relaxed((u32)entry, SUN20I_D1_RISCV_CFG_BASE + RESET_ENTRY_LO_REG);
> +       writel_relaxed(entry >> 32, SUN20I_D1_RISCV_CFG_BASE + RESET_ENTRY_HI_REG);

This line gives a compile error for RV32. I have updated it at the time
of merging this patch.

> +}
> +
> +static int sun20i_d1_hart_suspend(u32 suspend_type)
> +{
> +       /* Use the generic code for retentive suspend. */
> +       if (!(suspend_type & SBI_HSM_SUSP_NON_RET_BIT))
> +               return SBI_ENOTSUPP;
> +
> +       sun20i_d1_plic_save();
> +       sun20i_d1_ppu_save();
> +       sun20i_d1_riscv_cfg_save();
> +       sun20i_d1_csr_save();
> +
> +       /*
> +        * If no interrupt is pending, this will power down the CPU power
> +        * domain. Otherwise, this will fall through, and the generic HSM
> +        * code will jump to the resume address.
> +        */
> +       wfi();
> +
> +       return 0;
> +}
> +
> +static void sun20i_d1_hart_resume(void)
> +{
> +       sun20i_d1_csr_restore();
> +       sun20i_d1_riscv_cfg_restore();
> +       sun20i_d1_ppu_restore();
> +       sun20i_d1_plic_restore();
> +}
> +
> +static const struct sbi_hsm_device sun20i_d1_ppu = {
> +       .name           = "sun20i-d1-ppu",
> +       .hart_suspend   = sun20i_d1_hart_suspend,
> +       .hart_resume    = sun20i_d1_hart_resume,
> +};
> +
> +static int sun20i_d1_final_init(bool cold_boot, const struct fdt_match *match)
> +{
> +       if (cold_boot) {
> +               sun20i_d1_riscv_cfg_init();
> +               sbi_hsm_set_device(&sun20i_d1_ppu);
> +       }
> +
> +       return 0;
> +}
> +
> +static const struct fdt_match sun20i_d1_match[] = {
> +       { .compatible = "allwinner,sun20i-d1" },
> +       { },
> +};
> +
> +const struct platform_override sun20i_d1 = {
> +       .match_table    = sun20i_d1_match,
> +       .final_init     = sun20i_d1_final_init,
> +};
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From anup at brainfault.org  Sun Jun 12 23:46:08 2022
From: anup at brainfault.org (Anup Patel)
Date: Mon, 13 Jun 2022 12:16:08 +0530
Subject: [PATCH v3 0/9] HSM implementation for Allwinner D1
In-Reply-To: <20220613010355.28014-1-samuel@sholland.org>
References: <20220613010355.28014-1-samuel@sholland.org>
Message-ID: <CAAhSdy0feuX5ZCo0RqdXvaQzVGCDcWUj2B9p2Qv-O1G+g+f=ZA@mail.gmail.com>

On Mon, Jun 13, 2022 at 6:34 AM Samuel Holland <samuel at sholland.org> wrote:
>
> This series adds a minimal working HSM implementation for the Allwinner
> D1 SoC. It works with the Linux sbi_cpuidle driver. Even with canned
> values for entry/exit latency, I am seeing 30-45 mW reduction in power
> usage at idle.
>
> Since this is the first HSM driver submission, there is no example to
> follow. I have updated the series based on the comments to the RFCs.
> Still, comments welcome!

Thanks, this help us harden the OpenSBI HSM implementation.

>
> Regards,
> Samuel
>
> Changes in v3:
>  - Fix typo in commit message
>  - Move comment about priority range to function declaration
>  - Rename irqchip_plic_* functions to fdt_plic_*
>  - Add a "bool smode" parameter instead of hard coding the context ID
>
> Changes in v2:
>  - Rebase on top of the current master branch
>  - Use array_size() from sbi_types.h
>  - Use a separate variable for each CSR instead of an array
>  - Use the PLIC helpers added in earlier patches
>  - Use the saved SIE bits to program the wakeup interrupt mask
>  - Set the entry address only once during cold boot
>  - Use the generic implementation for retentive suspend
>  - Clean up/add comments throughout the HSM implementation
>
> Samuel Holland (9):
>   lib: sbi_hsm: Call a device hook during hart resume
>   lib: sbi_hsm: Assume a consistent resume address
>   lib: irqchip/plic: Constify plic_data pointers
>   lib: irqchip/plic: Factor out a context init function
>   lib: irqchip/plic: Add context save/restore helpers
>   lib: irqchip/plic: Add priority save/restore helpers
>   lib: utils/irqchip: Add FDT wrappers for PLIC save/restore functions
>   lib: utils/irqchip: Add wrapper for T-HEAD PLIC delegation
>   platform: Add HSM implementation for Allwinner D1

I have tested this series on QEMU virt machine using a custom DTB.

Applied this series on the riscv/opensbi repo

Thanks,
Anup

>
>  include/sbi/sbi_hsm.h                        |  12 +-
>  include/sbi_utils/irqchip/fdt_irqchip_plic.h |  22 ++
>  include/sbi_utils/irqchip/plic.h             |  20 +-
>  lib/sbi/sbi_hsm.c                            |  54 ++---
>  lib/utils/irqchip/fdt_irqchip_plic.c         |  39 ++++
>  lib/utils/irqchip/plic.c                     | 124 ++++++++---
>  platform/fpga/ariane/platform.c              |  19 +-
>  platform/fpga/openpiton/platform.c           |  19 +-
>  platform/generic/allwinner/objects.mk        |   6 +
>  platform/generic/allwinner/sun20i-d1.c       | 210 +++++++++++++++++++
>  10 files changed, 442 insertions(+), 83 deletions(-)
>  create mode 100644 include/sbi_utils/irqchip/fdt_irqchip_plic.h
>  create mode 100644 platform/generic/allwinner/objects.mk
>  create mode 100644 platform/generic/allwinner/sun20i-d1.c
>
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From apatel at ventanamicro.com  Thu Jun 16 04:58:56 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 16 Jun 2022 17:28:56 +0530
Subject: [PATCH] lib: utils: Remove CSRs that set/clear an IMSIC interrupt file bits
Message-ID: <20220616115856.1073885-1-apatel@ventanamicro.com>

Based on architecture review committee feedback, the [m|s|vs]seteienum,
[m|s|vs]clreienum, [m|s|vs]seteipnum, and [m|s|vs]clreipnum CSRs are
removed in the latest AIA draft v0.3.0 specification.
(Refer, https://github.com/riscv/riscv-aia/releases/tag/0.3.0-draft.31)

These CSRs were mostly for software convenience and software can always
use [m|s|vs]iselect and [m|s|vs]ireg CSRs to update the IMSIC interrupt
file bits.

We update the IMSIC programming as-per above to match the latest AIA
draft specification.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 include/sbi/riscv_encoding.h | 24 +++--------------
 lib/utils/irqchip/imsic.c    | 51 +++++++++++++++++++++++++++++++-----
 2 files changed, 48 insertions(+), 27 deletions(-)

diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
index a164768..68b4a7f 100644
--- a/include/sbi/riscv_encoding.h
+++ b/include/sbi/riscv_encoding.h
@@ -336,14 +336,8 @@
 #define CSR_SIREG			0x151
 
 /* Supervisor-Level Interrupts (AIA) */
-#define CSR_STOPI			0xdb0
-
-/* Supervisor-Level IMSIC Interface (AIA) */
-#define CSR_SSETEIPNUM			0x158
-#define CSR_SCLREIPNUM			0x159
-#define CSR_SSETEIENUM			0x15a
-#define CSR_SCLREIENUM			0x15b
 #define CSR_STOPEI			0x15c
+#define CSR_STOPI			0xdb0
 
 /* Supervisor-Level High-Half CSRs (AIA) */
 #define CSR_SIEH			0x114
@@ -405,14 +399,8 @@
 #define CSR_VSIREG			0x251
 
 /* VS-Level Interrupts (H-extension with AIA) */
-#define CSR_VSTOPI			0xeb0
-
-/* VS-Level IMSIC Interface (H-extension with AIA) */
-#define CSR_VSSETEIPNUM		0x258
-#define CSR_VSCLREIPNUM		0x259
-#define CSR_VSSETEIENUM		0x25a
-#define CSR_VSCLREIENUM		0x25b
 #define CSR_VSTOPEI			0x25c
+#define CSR_VSTOPI			0xeb0
 
 /* Hypervisor and VS-Level High-Half CSRs (H-extension with AIA) */
 #define CSR_HIDELEGH			0x613
@@ -693,14 +681,8 @@
 #define CSR_MIREG			0x351
 
 /* Machine-Level Interrupts (AIA) */
-#define CSR_MTOPI			0xfb0
-
-/* Machine-Level IMSIC Interface (AIA) */
-#define CSR_MSETEIPNUM			0x358
-#define CSR_MCLREIPNUM			0x359
-#define CSR_MSETEIENUM			0x35a
-#define CSR_MCLREIENUM			0x35b
 #define CSR_MTOPEI			0x35c
+#define CSR_MTOPI			0xfb0
 
 /* Virtual Interrupts for Supervisor Level (AIA) */
 #define CSR_MVIEN			0x308
diff --git a/lib/utils/irqchip/imsic.c b/lib/utils/irqchip/imsic.c
index b29a1e9..98f2cb6 100644
--- a/lib/utils/irqchip/imsic.c
+++ b/lib/utils/irqchip/imsic.c
@@ -38,10 +38,14 @@
 
 #define IMSIC_EIP63			0xbf
 
+#define IMSIC_EIPx_BITS			32
+
 #define IMSIC_EIE0			0xc0
 
 #define IMSIC_EIE63			0xff
 
+#define IMSIC_EIEx_BITS			32
+
 #define IMSIC_DISABLE_EIDELIVERY	0
 #define IMSIC_ENABLE_EIDELIVERY		1
 #define IMSIC_DISABLE_EITHRESHOLD	1
@@ -63,6 +67,18 @@ do { \
 	__v; \
 })
 
+#define imsic_csr_set(__c, __v)		\
+do { \
+	csr_write(CSR_MISELECT, __c); \
+	csr_set(CSR_MIREG, __v); \
+} while (0)
+
+#define imsic_csr_clear(__c, __v)	\
+do { \
+	csr_write(CSR_MISELECT, __c); \
+	csr_clear(CSR_MIREG, __v); \
+} while (0)
+
 static struct imsic_data *imsic_hartid2data[SBI_HARTMASK_MAX_BITS];
 static int imsic_hartid2file[SBI_HARTMASK_MAX_BITS];
 
@@ -140,6 +156,31 @@ static struct sbi_ipi_device imsic_ipi_device = {
 	.ipi_send	= imsic_ipi_send
 };
 
+static void imsic_local_eix_update(unsigned long base_id,
+				   unsigned long num_id, bool pend, bool val)
+{
+	unsigned long i, isel, ireg;
+	unsigned long id = base_id, last_id = base_id + num_id;
+
+	while (id < last_id) {
+		isel = id / __riscv_xlen;
+		isel *= __riscv_xlen / IMSIC_EIPx_BITS;
+		isel += (pend) ? IMSIC_EIP0 : IMSIC_EIE0;
+
+		ireg = 0;
+		for (i = id & (__riscv_xlen - 1);
+		     (id < last_id) && (i < __riscv_xlen); i++) {
+			ireg |= BIT(i);
+			id++;
+		}
+
+		if (val)
+			imsic_csr_set(isel, ireg);
+		else
+			imsic_csr_clear(isel, ireg);
+	}
+}
+
 void imsic_local_irqchip_init(void)
 {
 	/*
@@ -158,12 +199,11 @@ void imsic_local_irqchip_init(void)
 	imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_ENABLE_EIDELIVERY);
 
 	/* Enable IPI */
-	csr_write(CSR_MSETEIENUM, IMSIC_IPI_ID);
+	imsic_local_eix_update(IMSIC_IPI_ID, 1, false, true);
 }
 
 int imsic_warm_irqchip_init(void)
 {
-	unsigned long i;
 	struct imsic_data *imsic = imsic_hartid2data[current_hartid()];
 
 	/* Sanity checks */
@@ -171,11 +211,10 @@ int imsic_warm_irqchip_init(void)
 		return SBI_EINVAL;
 
 	/* Disable all interrupts */
-	for (i = 1; i <= imsic->num_ids; i++)
-		csr_write(CSR_MCLREIENUM, i);
+	imsic_local_eix_update(1, imsic->num_ids, false, false);
 
-	/* Clear IPI */
-	csr_write(CSR_MCLREIPNUM, IMSIC_IPI_ID);
+	/* Clear IPI pending */
+	imsic_local_eix_update(IMSIC_IPI_ID, 1, true, false);
 
 	/* Local IMSIC initialization */
 	imsic_local_irqchip_init();
-- 
2.34.1



From anup at brainfault.org  Thu Jun 16 21:12:08 2022
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Jun 2022 09:42:08 +0530
Subject: [PATCH] lib: sbi: Fix version dependency for Sscofpmf
In-Reply-To: <20220613003838.13028-1-samuel@sholland.org>
References: <20220613003838.13028-1-samuel@sholland.org>
Message-ID: <CAAhSdy0Y49egjSeOfTC0OkSEmTf03EK1cHECi-ARjZj3oNtWaA@mail.gmail.com>

On Mon, Jun 13, 2022 at 6:08 AM Samuel Holland <samuel at sholland.org> wrote:
>
> mcounteren was added in the privileged spec v1.10 and mcountinhibit was
> added in v1.11. Sscofpmf does not depend on anything in v1.12, so the
> minimum privileged spec version should have been v1.11.
>
> Fixes: d4b563c881d6 ("lib: sbi: Remove MCOUNTEREN and SCOUNTEREN hart features")
> Signed-off-by: Samuel Holland <samuel at sholland.org>

I think there is some confusion here.

The mcounteren and mcountinhibit are not part of the Sscofpmf extension and
OpenSBI PMU implementation does support platforms without Sscofpmf.

Regards,
Anup

> ---
>
>  lib/sbi/sbi_hart.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> index de86fee..5bc6d52 100644
> --- a/lib/sbi/sbi_hart.c
> +++ b/lib/sbi/sbi_hart.c
> @@ -645,7 +645,7 @@ __mhpm_skip:
>                 hfeatures->priv_version = SBI_HART_PRIV_VER_1_12;
>
>         /* Counter overflow/filtering is not useful without mcounter/inhibit */
> -       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
> +       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_11) {
>                 /* Detect if hart supports sscofpmf */
>                 csr_read_allowed(CSR_SCOUNTOVF, (unsigned long)&trap);
>                 if (!trap.cause)
> --
> 2.35.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From samuel at sholland.org  Thu Jun 16 21:28:41 2022
From: samuel at sholland.org (Samuel Holland)
Date: Thu, 16 Jun 2022 23:28:41 -0500
Subject: [PATCH] lib: sbi: Fix version dependency for Sscofpmf
In-Reply-To: <CAAhSdy0Y49egjSeOfTC0OkSEmTf03EK1cHECi-ARjZj3oNtWaA@mail.gmail.com>
References: <20220613003838.13028-1-samuel@sholland.org>
 <CAAhSdy0Y49egjSeOfTC0OkSEmTf03EK1cHECi-ARjZj3oNtWaA@mail.gmail.com>
Message-ID: <97c665fe-7e52-cd56-c198-84e796fa8263@sholland.org>

Hi Anup,

On 6/16/22 11:12 PM, Anup Patel wrote:
> On Mon, Jun 13, 2022 at 6:08 AM Samuel Holland <samuel at sholland.org> wrote:
>>
>> mcounteren was added in the privileged spec v1.10 and mcountinhibit was
>> added in v1.11. Sscofpmf does not depend on anything in v1.12, so the
>> minimum privileged spec version should have been v1.11.
>>
>> Fixes: d4b563c881d6 ("lib: sbi: Remove MCOUNTEREN and SCOUNTEREN hart features")
>> Signed-off-by: Samuel Holland <samuel at sholland.org>
> 
> I think there is some confusion here.
> 
> The mcounteren and mcountinhibit are not part of the Sscofpmf extension and
> OpenSBI PMU implementation does support platforms without Sscofpmf.

I am aware of that. I am referencing the comment on the previous line, and the
code that was changed by the referenced commit. The code, as originally written,
attempted to optimize out the Sscofpmf check based on the presence of those two
CSRs.

When the hart features were refactored, the CSR feature was replaced by the
wrong version, i.e. the behavior was accidentally changed by the refactoring.

This patch only attempts to restore the behavior from before the refactoring. If
that behavior was wrong to start with, that is a different matter.

Regards,
Samuel

> Regards,
> Anup
> 
>> ---
>>
>>  lib/sbi/sbi_hart.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
>> index de86fee..5bc6d52 100644
>> --- a/lib/sbi/sbi_hart.c
>> +++ b/lib/sbi/sbi_hart.c
>> @@ -645,7 +645,7 @@ __mhpm_skip:
>>                 hfeatures->priv_version = SBI_HART_PRIV_VER_1_12;
>>
>>         /* Counter overflow/filtering is not useful without mcounter/inhibit */
>> -       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
>> +       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_11) {
>>                 /* Detect if hart supports sscofpmf */
>>                 csr_read_allowed(CSR_SCOUNTOVF, (unsigned long)&trap);
>>                 if (!trap.cause)
>> --
>> 2.35.1
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi



From anup at brainfault.org  Thu Jun 16 21:56:57 2022
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Jun 2022 10:26:57 +0530
Subject: [PATCH] lib: sbi: Fix version dependency for Sscofpmf
In-Reply-To: <97c665fe-7e52-cd56-c198-84e796fa8263@sholland.org>
References: <20220613003838.13028-1-samuel@sholland.org> <CAAhSdy0Y49egjSeOfTC0OkSEmTf03EK1cHECi-ARjZj3oNtWaA@mail.gmail.com>
 <97c665fe-7e52-cd56-c198-84e796fa8263@sholland.org>
Message-ID: <CAAhSdy24ukPCh0LFOZVKtDyzHdxTWw348nhueQ-mUC5JyiLYAA@mail.gmail.com>

On Fri, Jun 17, 2022 at 9:58 AM Samuel Holland <samuel at sholland.org> wrote:
>
> Hi Anup,
>
> On 6/16/22 11:12 PM, Anup Patel wrote:
> > On Mon, Jun 13, 2022 at 6:08 AM Samuel Holland <samuel at sholland.org> wrote:
> >>
> >> mcounteren was added in the privileged spec v1.10 and mcountinhibit was
> >> added in v1.11. Sscofpmf does not depend on anything in v1.12, so the
> >> minimum privileged spec version should have been v1.11.
> >>
> >> Fixes: d4b563c881d6 ("lib: sbi: Remove MCOUNTEREN and SCOUNTEREN hart features")
> >> Signed-off-by: Samuel Holland <samuel at sholland.org>
> >
> > I think there is some confusion here.
> >
> > The mcounteren and mcountinhibit are not part of the Sscofpmf extension and
> > OpenSBI PMU implementation does support platforms without Sscofpmf.
>
> I am aware of that. I am referencing the comment on the previous line, and the
> code that was changed by the referenced commit. The code, as originally written,
> attempted to optimize out the Sscofpmf check based on the presence of those two
> CSRs.
>
> When the hart features were refactored, the CSR feature was replaced by the
> wrong version, i.e. the behavior was accidentally changed by the refactoring.
>
> This patch only attempts to restore the behavior from before the refactoring. If
> that behavior was wrong to start with, that is a different matter.

I agree there is nothing in Sscofpmf which prevents a Priv v1.11 HART from
implementing it so this now becomes a strange choice on the OpenSBI side.

IMO, the previous behaviour of detecting and using Sscofpmf irrespective
of the Priv spec version was wrong because Sscofpmf is defined assuming
Priv v1.12 spec. In fact, that's why Sscofpmf defines bits for filtering events
in VS/VU modes.

Is there a case where we would need Sscofpmf for a Priv v1.11 HART ?

Regards,
Anup

> > Regards,
> > Anup
> >
> >> ---
> >>
> >>  lib/sbi/sbi_hart.c | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> >> index de86fee..5bc6d52 100644
> >> --- a/lib/sbi/sbi_hart.c
> >> +++ b/lib/sbi/sbi_hart.c
> >> @@ -645,7 +645,7 @@ __mhpm_skip:
> >>                 hfeatures->priv_version = SBI_HART_PRIV_VER_1_12;
> >>
> >>         /* Counter overflow/filtering is not useful without mcounter/inhibit */
> >> -       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
> >> +       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_11) {
> >>                 /* Detect if hart supports sscofpmf */
> >>                 csr_read_allowed(CSR_SCOUNTOVF, (unsigned long)&trap);
> >>                 if (!trap.cause)
> >> --
> >> 2.35.1
> >>
> >>
> >> --
> >> opensbi mailing list
> >> opensbi at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/opensbi
>


From samuel at sholland.org  Thu Jun 16 22:19:37 2022
From: samuel at sholland.org (Samuel Holland)
Date: Fri, 17 Jun 2022 00:19:37 -0500
Subject: [PATCH] lib: sbi: Fix version dependency for Sscofpmf
In-Reply-To: <CAAhSdy24ukPCh0LFOZVKtDyzHdxTWw348nhueQ-mUC5JyiLYAA@mail.gmail.com>
References: <20220613003838.13028-1-samuel@sholland.org>
 <CAAhSdy0Y49egjSeOfTC0OkSEmTf03EK1cHECi-ARjZj3oNtWaA@mail.gmail.com>
 <97c665fe-7e52-cd56-c198-84e796fa8263@sholland.org>
 <CAAhSdy24ukPCh0LFOZVKtDyzHdxTWw348nhueQ-mUC5JyiLYAA@mail.gmail.com>
Message-ID: <ba9f1314-0988-6c90-3460-34d2f29157ab@sholland.org>

On 6/16/22 11:56 PM, Anup Patel wrote:
> On Fri, Jun 17, 2022 at 9:58 AM Samuel Holland <samuel at sholland.org> wrote:
>>
>> Hi Anup,
>>
>> On 6/16/22 11:12 PM, Anup Patel wrote:
>>> On Mon, Jun 13, 2022 at 6:08 AM Samuel Holland <samuel at sholland.org> wrote:
>>>>
>>>> mcounteren was added in the privileged spec v1.10 and mcountinhibit was
>>>> added in v1.11. Sscofpmf does not depend on anything in v1.12, so the
>>>> minimum privileged spec version should have been v1.11.
>>>>
>>>> Fixes: d4b563c881d6 ("lib: sbi: Remove MCOUNTEREN and SCOUNTEREN hart features")
>>>> Signed-off-by: Samuel Holland <samuel at sholland.org>
>>>
>>> I think there is some confusion here.
>>>
>>> The mcounteren and mcountinhibit are not part of the Sscofpmf extension and
>>> OpenSBI PMU implementation does support platforms without Sscofpmf.
>>
>> I am aware of that. I am referencing the comment on the previous line, and the
>> code that was changed by the referenced commit. The code, as originally written,
>> attempted to optimize out the Sscofpmf check based on the presence of those two
>> CSRs.
>>
>> When the hart features were refactored, the CSR feature was replaced by the
>> wrong version, i.e. the behavior was accidentally changed by the refactoring.
>>
>> This patch only attempts to restore the behavior from before the refactoring. If
>> that behavior was wrong to start with, that is a different matter.
> 
> I agree there is nothing in Sscofpmf which prevents a Priv v1.11 HART from
> implementing it so this now becomes a strange choice on the OpenSBI side.
> 
> IMO, the previous behaviour of detecting and using Sscofpmf irrespective
> of the Priv spec version was wrong because Sscofpmf is defined assuming
> Priv v1.12 spec. In fact, that's why Sscofpmf defines bits for filtering events
> in VS/VU modes.
> 
> Is there a case where we would need Sscofpmf for a Priv v1.11 HART ?

I was attempting to add C906 PMU support as an errata on top of the standard
extension, which is when I noticed this check. (C906 supports v1.11). However, I
realized C906's custom overflow handling is different enough from Sscofpmf that
I should implement it separately.

So I do not know if a v1.11+Sscofpmf case does or should exist. Maybe the check
is correct now, in which case the comment is a bit misleading.

Regards,
Samuel

>>>> ---
>>>>
>>>>  lib/sbi/sbi_hart.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
>>>> index de86fee..5bc6d52 100644
>>>> --- a/lib/sbi/sbi_hart.c
>>>> +++ b/lib/sbi/sbi_hart.c
>>>> @@ -645,7 +645,7 @@ __mhpm_skip:
>>>>                 hfeatures->priv_version = SBI_HART_PRIV_VER_1_12;
>>>>
>>>>         /* Counter overflow/filtering is not useful without mcounter/inhibit */
>>>> -       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
>>>> +       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_11) {
>>>>                 /* Detect if hart supports sscofpmf */
>>>>                 csr_read_allowed(CSR_SCOUNTOVF, (unsigned long)&trap);
>>>>                 if (!trap.cause)
>>>> --
>>>> 2.35.1
>>>>
>>>>
>>>> --
>>>> opensbi mailing list
>>>> opensbi at lists.infradead.org
>>>> http://lists.infradead.org/mailman/listinfo/opensbi
>>



From anup at brainfault.org  Thu Jun 16 23:43:28 2022
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Jun 2022 12:13:28 +0530
Subject: [PATCH] lib: sbi: Fix version dependency for Sscofpmf
In-Reply-To: <ba9f1314-0988-6c90-3460-34d2f29157ab@sholland.org>
References: <20220613003838.13028-1-samuel@sholland.org> <CAAhSdy0Y49egjSeOfTC0OkSEmTf03EK1cHECi-ARjZj3oNtWaA@mail.gmail.com>
 <97c665fe-7e52-cd56-c198-84e796fa8263@sholland.org> <CAAhSdy24ukPCh0LFOZVKtDyzHdxTWw348nhueQ-mUC5JyiLYAA@mail.gmail.com>
 <ba9f1314-0988-6c90-3460-34d2f29157ab@sholland.org>
Message-ID: <CAAhSdy32h8kvY5OLVQtUS7=m1Nx7Mpvi-u5-Gz1GTh2y4vNemg@mail.gmail.com>

On Fri, Jun 17, 2022 at 10:49 AM Samuel Holland <samuel at sholland.org> wrote:
>
> On 6/16/22 11:56 PM, Anup Patel wrote:
> > On Fri, Jun 17, 2022 at 9:58 AM Samuel Holland <samuel at sholland.org> wrote:
> >>
> >> Hi Anup,
> >>
> >> On 6/16/22 11:12 PM, Anup Patel wrote:
> >>> On Mon, Jun 13, 2022 at 6:08 AM Samuel Holland <samuel at sholland.org> wrote:
> >>>>
> >>>> mcounteren was added in the privileged spec v1.10 and mcountinhibit was
> >>>> added in v1.11. Sscofpmf does not depend on anything in v1.12, so the
> >>>> minimum privileged spec version should have been v1.11.
> >>>>
> >>>> Fixes: d4b563c881d6 ("lib: sbi: Remove MCOUNTEREN and SCOUNTEREN hart features")
> >>>> Signed-off-by: Samuel Holland <samuel at sholland.org>
> >>>
> >>> I think there is some confusion here.
> >>>
> >>> The mcounteren and mcountinhibit are not part of the Sscofpmf extension and
> >>> OpenSBI PMU implementation does support platforms without Sscofpmf.
> >>
> >> I am aware of that. I am referencing the comment on the previous line, and the
> >> code that was changed by the referenced commit. The code, as originally written,
> >> attempted to optimize out the Sscofpmf check based on the presence of those two
> >> CSRs.
> >>
> >> When the hart features were refactored, the CSR feature was replaced by the
> >> wrong version, i.e. the behavior was accidentally changed by the refactoring.
> >>
> >> This patch only attempts to restore the behavior from before the refactoring. If
> >> that behavior was wrong to start with, that is a different matter.
> >
> > I agree there is nothing in Sscofpmf which prevents a Priv v1.11 HART from
> > implementing it so this now becomes a strange choice on the OpenSBI side.
> >
> > IMO, the previous behaviour of detecting and using Sscofpmf irrespective
> > of the Priv spec version was wrong because Sscofpmf is defined assuming
> > Priv v1.12 spec. In fact, that's why Sscofpmf defines bits for filtering events
> > in VS/VU modes.
> >
> > Is there a case where we would need Sscofpmf for a Priv v1.11 HART ?
>
> I was attempting to add C906 PMU support as an errata on top of the standard
> extension, which is when I noticed this check. (C906 supports v1.11). However, I
> realized C906's custom overflow handling is different enough from Sscofpmf that
> I should implement it separately.
>
> So I do not know if a v1.11+Sscofpmf case does or should exist. Maybe the check
> is correct now, in which case the comment is a bit misleading.

Instead of setting SSCOFPMF extension flag for C906, I suggest we extend
sbi_pmu.c using "struct sbi_pmu_overflow_ops" abstraction. The sbi_pmu.c
will have two implementations of "struct sbi_pmu_overflow_ops": 1) "dummy"
implementation which will be used when there is no HW counter overflow
support 2) "sscofpmf" implementation which will be used when HW support
Sscofpmf extension.

The platform override for Allwinner D1 platform can provide its own
implementation of "struct sbi_pmu_overflow_ops".

Regards,
Anup

>
> Regards,
> Samuel
>
> >>>> ---
> >>>>
> >>>>  lib/sbi/sbi_hart.c | 2 +-
> >>>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> >>>> index de86fee..5bc6d52 100644
> >>>> --- a/lib/sbi/sbi_hart.c
> >>>> +++ b/lib/sbi/sbi_hart.c
> >>>> @@ -645,7 +645,7 @@ __mhpm_skip:
> >>>>                 hfeatures->priv_version = SBI_HART_PRIV_VER_1_12;
> >>>>
> >>>>         /* Counter overflow/filtering is not useful without mcounter/inhibit */
> >>>> -       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
> >>>> +       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_11) {
> >>>>                 /* Detect if hart supports sscofpmf */
> >>>>                 csr_read_allowed(CSR_SCOUNTOVF, (unsigned long)&trap);
> >>>>                 if (!trap.cause)
> >>>> --
> >>>> 2.35.1
> >>>>
> >>>>
> >>>> --
> >>>> opensbi mailing list
> >>>> opensbi at lists.infradead.org
> >>>> http://lists.infradead.org/mailman/listinfo/opensbi
> >>
>


From anup at brainfault.org  Thu Jun 16 23:44:32 2022
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Jun 2022 12:14:32 +0530
Subject: [PATCH 1/3] include: sbi: Add mtinst/htinst psuedoinstructions
In-Reply-To: <TYYP286MB14399472C06DD67BB8C55351C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <TYYP286MB14399472C06DD67BB8C55351C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAAhSdy1xmJi5AB59xxOFuK1dyuXKwE77r_KpkBoB4AX2=s3m2w@mail.gmail.com>

On Thu, Jun 9, 2022 at 12:38 PM dramforever <dramforever at live.com> wrote:
>
> Add psuedoinstruction encodings written to mtinst/htinst for faults
> caused by implicit memory access for VS-stage address translation
>
> Signed-off-by: dramforever <dramforever at live.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  include/sbi/riscv_encoding.h | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
>
> diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
> index a164768..2292858 100644
> --- a/include/sbi/riscv_encoding.h
> +++ b/include/sbi/riscv_encoding.h
> @@ -841,6 +841,26 @@
>  #define INSN_MASK_FENCE_TSO            0xffffffff
>  #define INSN_MATCH_FENCE_TSO           0x8330000f
>
> +#if __riscv_xlen == 64
> +
> +/* 64-bit read for VS-stage address translation (RV64) */
> +#define INSN_PSEUDO_VS_LOAD            0x00003000
> +
> +/* 64-bit write for VS-stage address translation (RV64) */
> +#define INSN_PSEUDO_VS_STORE   0x00003020
> +
> +#elif __riscv_xlen == 32
> +
> +/* 32-bit read for VS-stage address translation (RV32) */
> +#define INSN_PSEUDO_VS_LOAD            0x00002000
> +
> +/* 32-bit write for VS-stage address translation (RV32) */
> +#define INSN_PSEUDO_VS_STORE   0x00002020
> +
> +#else
> +#error "Unexpected __riscv_xlen"
> +#endif
> +
>  #define INSN_16BIT_MASK                        0x3
>  #define INSN_32BIT_MASK                        0x1c
>
> --
> 2.36.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Thu Jun 16 23:45:44 2022
From: anup at brainfault.org (Anup Patel)
Date: Fri, 17 Jun 2022 12:15:44 +0530
Subject: [PATCH 2/3] lib: sbi: Fixup tinst for exceptions in sbi_misaligned_*()
In-Reply-To: <TYYP286MB14399AD2F6CFFB0CECD334E9C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <TYYP286MB14399AD2F6CFFB0CECD334E9C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAAhSdy2T=tVgsxCPS+A36x39idwnSQSg+TbA_Bt11QthYSbg9w@mail.gmail.com>

On Thu, Jun 9, 2022 at 12:38 PM dramforever <dramforever at live.com> wrote:
>
> If there is an exception while emulating a misaligned load/store, fixup
> uptrap.tinst before redirecting. Otherwise, HS-mode software may receive
> an htinst describing the lbu/sb instruction that faulted during
> emulation[1].
>
> [1]: https://github.com/riscv-software-src/opensbi/issues/258
>
> Signed-off-by: dramforever <dramforever at live.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/sbi/sbi_misaligned_ldst.c | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
>
> diff --git a/lib/sbi/sbi_misaligned_ldst.c b/lib/sbi/sbi_misaligned_ldst.c
> index c879ce7..fd11798 100644
> --- a/lib/sbi/sbi_misaligned_ldst.c
> +++ b/lib/sbi/sbi_misaligned_ldst.c
> @@ -22,6 +22,18 @@ union reg_data {
>         u64 data_u64;
>  };
>
> +static ulong sbi_misaligned_tinst_fixup(ulong orig_tinst, ulong new_tinst,
> +                                       ulong addr_offset)
> +{
> +       if (new_tinst == INSN_PSEUDO_VS_LOAD ||
> +           new_tinst == INSN_PSEUDO_VS_STORE)
> +               return new_tinst;
> +       else if (orig_tinst == 0)
> +               return 0UL;
> +       else
> +               return orig_tinst | (addr_offset << SH_RS1);
> +}
> +
>  int sbi_misaligned_load_handler(ulong addr, ulong tval2, ulong tinst,
>                                 struct sbi_trap_regs *regs)
>  {
> @@ -126,6 +138,8 @@ int sbi_misaligned_load_handler(ulong addr, ulong tval2, ulong tinst,
>                                                 &uptrap);
>                 if (uptrap.cause) {
>                         uptrap.epc = regs->mepc;
> +                       uptrap.tinst = sbi_misaligned_tinst_fixup(
> +                               tinst, uptrap.tinst, i);
>                         return sbi_trap_redirect(regs, &uptrap);
>                 }
>         }
> @@ -238,6 +252,8 @@ int sbi_misaligned_store_handler(ulong addr, ulong tval2, ulong tinst,
>                              &uptrap);
>                 if (uptrap.cause) {
>                         uptrap.epc = regs->mepc;
> +                       uptrap.tinst = sbi_misaligned_tinst_fixup(
> +                               tinst, uptrap.tinst, i);
>                         return sbi_trap_redirect(regs, &uptrap);
>                 }
>         }
> --
> 2.36.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Fri Jun 17 22:58:44 2022
From: anup at brainfault.org (Anup Patel)
Date: Sat, 18 Jun 2022 11:28:44 +0530
Subject: [PATCH 3/3] lib: sbi: Fix tval and tinst for sbi_get_insn()
In-Reply-To: <TYYP286MB14393C382F94A378E9F529CDC6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <TYYP286MB14393C382F94A378E9F529CDC6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAAhSdy3oT8JNB2X8TG_xWE9qVuX-k4BRo-JHP3s_uiKnRKP11Q@mail.gmail.com>

On Thu, Jun 9, 2022 at 12:38 PM dramforever <dramforever at live.com> wrote:
>
> We should not change trap->tval to mepc because mtval already points to
> the faulting portion of the emulated instruction fetch, which is also
> what stval is expected to be.
>
> In addition, htinst is only allowed to be zero for instruction access
> faults or page faults, and is only allowed to be zero or a
> psuedoinstruction for instruction guest-page faults. Fix trap->tinst for
> these cases.
>
> Signed-off-by: dramforever <dramforever at live.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/sbi/sbi_unpriv.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/lib/sbi/sbi_unpriv.c b/lib/sbi/sbi_unpriv.c
> index 73b530c..9a34a71 100644
> --- a/lib/sbi/sbi_unpriv.c
> +++ b/lib/sbi/sbi_unpriv.c
> @@ -149,15 +149,17 @@ ulong sbi_get_insn(ulong mepc, struct sbi_trap_info *trap)
>         switch (trap->cause) {
>         case CAUSE_LOAD_ACCESS:
>                 trap->cause = CAUSE_FETCH_ACCESS;
> -               trap->tval = mepc;
> +               trap->tinst = 0UL;
>                 break;
>         case CAUSE_LOAD_PAGE_FAULT:
>                 trap->cause = CAUSE_FETCH_PAGE_FAULT;
> -               trap->tval = mepc;
> +               trap->tinst = 0UL;
>                 break;
>         case CAUSE_LOAD_GUEST_PAGE_FAULT:
>                 trap->cause = CAUSE_FETCH_GUEST_PAGE_FAULT;
> -               trap->tval = mepc;
> +               if (trap->tinst != INSN_PSEUDO_VS_LOAD &&
> +                   trap->tinst != INSN_PSEUDO_VS_STORE)
> +                       trap->tinst = 0UL;
>                 break;
>         default:
>                 break;
> --
> 2.36.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Sat Jun 18 03:22:03 2022
From: anup at brainfault.org (Anup Patel)
Date: Sat, 18 Jun 2022 15:52:03 +0530
Subject: [PATCH 1/2] lib: utils/serial: add virtio mmio driver
In-Reply-To: <20220612171822.148906-1-misterjdrg@gmail.com>
References: <20220612171822.148906-1-misterjdrg@gmail.com>
Message-ID: <CAAhSdy1rjztYmzNVjO+RGBy7xNSgBtQ60WMFAP6MeAvf7cH6VQ@mail.gmail.com>

On Sun, Jun 12, 2022 at 10:49 PM Bohdan Fedorov <misterjdrg at gmail.com> wrote:
>
> Simple virtio-console driver.
> Useful for running opensbi in virtualized environments.

The term "virtualized environments" can also mean Guest/VM
(VS-mode) under a hypervisor. Most hypervisors (including KVM)
don't emulate/virtualize M-mode for Guest/VM and instead the
hypervisor will itself handle SBI calls coming from Guest/VM.

I assume the intention is to use OpenSBI under RISC-V system
emulators, simulators, or ISS which uses VirtIO devices for I/O ??

>
> dt-bindings:
> <linux>/Documentation/devicetree/bindings/virtio/mmio.yaml
>
> Signed-off-by: Bohdan Fedorov <misterjdrg at gmail.com>
> ---
>  include/sbi_utils/fdt/fdt_helper.h     |  3 +
>  include/sbi_utils/serial/virtio-uart.h | 19 +++++++
>  lib/utils/fdt/fdt_helper.c             | 18 ++++++
>  lib/utils/serial/fdt_serial_virtio.c   | 37 +++++++++++++
>  lib/utils/serial/objects.mk            |  4 ++
>  lib/utils/serial/virtio-uart.c         | 76 ++++++++++++++++++++++++++
>  6 files changed, 157 insertions(+)
>  create mode 100644 include/sbi_utils/serial/virtio-uart.h
>  create mode 100644 lib/utils/serial/fdt_serial_virtio.c
>  create mode 100644 lib/utils/serial/virtio-uart.c

Please don't use the "virtio-uart" term instead use "virtio-console".

The VirtIO spec allows different types of transports for accessing device
config registers. The VirtIO MMIO is one of the possible transports and
we might have PCIe or platform specific transports as well.

I suggest the following refactoring of this patch:

1) Add a include/sbi_utils/virtio/virtio.h header which defines an abstract
    VirtIO device. This abstract VirtIO device will be instantiated by some
    VirtIO transport code. The include/sbi_utils/virtio/virtio.h will have
    something like below:

    struct virtio_transport {
        char name[32];
        int (*write_config)(struct virtio_device *dev,
                                    u32 offset, void *dst, u32 dst_len);
        int (*read_config)(struct virtio_device *dev,
                                     u32 offset, void *src, u32 src_len);
    };

    struct virtio_device {
        char name[32];
        struct virtio_transport *trans;
    };

    static inline int virtio_device_read_config(struct virtio_device *dev,

u32 offset, void *dst, u32 dst_len)
    {
         ....
    }

    static inline int virtio_device_write_config(struct virtio_device *dev,

u32 offset, void *dst, u32 dst_len)
    {
         ....
    }

2) Add include/sbi_utils/virtio/virtio-mmio.h and lib/utils/virtio/virtio-mmio.c
    which help platforms to instantiate VirtIO device using MMIO transport.
    The virtio-mmio.h and virtio-mmio.c can have just one function.

    int virtio_mmio_setup(struct virtio_device *dev, const char *name,
                                       unsigned long addr);

3) The include/sbi_utils/serial/virtio-console.h and
lib/utils/serial/virtio-console.c
     should only use include/sbi_utils/virtio/virtio.h

There can be separate patches for the above points as well.

>
> diff --git a/include/sbi_utils/fdt/fdt_helper.h b/include/sbi_utils/fdt/fdt_helper.h
> index c60af35..80307bb 100644
> --- a/include/sbi_utils/fdt/fdt_helper.h
> +++ b/include/sbi_utils/fdt/fdt_helper.h
> @@ -65,6 +65,9 @@ int fdt_parse_shakti_uart_node(void *fdt, int nodeoffset,
>  int fdt_parse_sifive_uart_node(void *fdt, int nodeoffset,
>                                struct platform_uart_data *uart);
>
> +int fdt_parse_virtio_uart_node(void *fdt, int nodeoffset,
> +                              struct platform_uart_data *uart);
> +
>  int fdt_parse_uart8250_node(void *fdt, int nodeoffset,
>                             struct platform_uart_data *uart);
>
> diff --git a/include/sbi_utils/serial/virtio-uart.h b/include/sbi_utils/serial/virtio-uart.h
> new file mode 100644
> index 0000000..4fdfebc
> --- /dev/null
> +++ b/include/sbi_utils/serial/virtio-uart.h
> @@ -0,0 +1,19 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2019 Western Digital Corporation or its affiliates.
> + *
> + * Authors:
> + *   Bohdan Fedorov <misterjdrg at gmail.com>
> + *
> + * Based on:
> + *   sifive-uart
> + */
> +
> +#ifndef __SERIAL_SIFIVE_UART_H__
> +#define __SERIAL_SIFIVE_UART_H__
> +
> +
> +int virtio_uart_init(unsigned long base);
> +
> +#endif
> diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
> index 2bdb1ef..9bfd5e7 100644
> --- a/lib/utils/fdt/fdt_helper.c
> +++ b/lib/utils/fdt/fdt_helper.c
> @@ -388,6 +388,24 @@ int fdt_parse_shakti_uart_node(void *fdt, int nodeoffset,
>         return 0;
>  }
>
> +int fdt_parse_virtio_uart_node(void *fdt, int nodeoffset,
> +                              struct platform_uart_data *uart)
> +{
> +       int rc;
> +       uint64_t reg_addr, reg_size;
> +
> +       if (nodeoffset < 0 || !uart || !fdt)
> +               return SBI_ENODEV;
> +
> +       rc = fdt_get_node_addr_size(fdt, nodeoffset, 0,
> +                                   &reg_addr, &reg_size);
> +       if (rc < 0 || !reg_addr || !reg_size)
> +               return SBI_ENODEV;
> +       uart->addr = reg_addr;
> +
> +       return 0;
> +}
> +

Please drop fdt_parse_virtio_uart_node() and instead directly
use fdt_get_node_addr_size().

>  int fdt_parse_sifive_uart_node(void *fdt, int nodeoffset,
>                                struct platform_uart_data *uart)
>  {
> diff --git a/lib/utils/serial/fdt_serial_virtio.c b/lib/utils/serial/fdt_serial_virtio.c
> new file mode 100644
> index 0000000..115c5ad
> --- /dev/null
> +++ b/lib/utils/serial/fdt_serial_virtio.c
> @@ -0,0 +1,37 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2020 Western Digital Corporation or its affiliates.
> + *
> + * Authors:
> + *   Bohdan Fedorov <misterjdrg at gmail.com>
> + *
> + * Based on virtio-uart
> + */
> +
> +#include <sbi_utils/fdt/fdt_helper.h>
> +#include <sbi_utils/serial/fdt_serial.h>
> +#include <sbi_utils/serial/virtio-uart.h>
> +
> +static int serial_virtio_init(void *fdt, int nodeoff,
> +                               const struct fdt_match *match)
> +{
> +       int rc;
> +       struct platform_uart_data uart;
> +
> +       rc = fdt_parse_virtio_uart_node(fdt, nodeoff, &uart);
> +       if (rc)
> +               return rc;
> +
> +       return virtio_uart_init(uart.addr);
> +}
> +
> +static const struct fdt_match serial_virtio_match[] = {
> +       { .compatible = "virtio,mmio" },
> +       { },
> +};
> +
> +struct fdt_serial fdt_serial_virtio = {
> +       .match_table = serial_virtio_match,
> +       .init = serial_virtio_init
> +};
> diff --git a/lib/utils/serial/objects.mk b/lib/utils/serial/objects.mk
> index d26a74e..60e829b 100644
> --- a/lib/utils/serial/objects.mk
> +++ b/lib/utils/serial/objects.mk
> @@ -22,6 +22,9 @@ libsbiutils-objs-y += serial/fdt_serial_shakti.o
>  carray-fdt_serial_drivers-y += fdt_serial_sifive
>  libsbiutils-objs-y += serial/fdt_serial_sifive.o
>
> +carray-fdt_serial_drivers-y += fdt_serial_virtio
> +libsbiutils-objs-y += serial/fdt_serial_virtio.o
> +
>  carray-fdt_serial_drivers-y += fdt_serial_litex
>  libsbiutils-objs-y += serial/fdt_serial_litex.o
>
> @@ -34,6 +37,7 @@ libsbiutils-objs-y += serial/fdt_serial_xlnx_uartlite.o
>  libsbiutils-objs-y += serial/gaisler-uart.o
>  libsbiutils-objs-y += serial/shakti-uart.o
>  libsbiutils-objs-y += serial/sifive-uart.o
> +libsbiutils-objs-y += serial/virtio-uart.o
>  libsbiutils-objs-y += serial/litex-uart.o
>  libsbiutils-objs-y += serial/uart8250.o
>  libsbiutils-objs-y += serial/xlnx-uartlite.o
> diff --git a/lib/utils/serial/virtio-uart.c b/lib/utils/serial/virtio-uart.c
> new file mode 100644
> index 0000000..1d4266f
> --- /dev/null
> +++ b/lib/utils/serial/virtio-uart.c
> @@ -0,0 +1,76 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright (c) 2019 Western Digital Corporation or its affiliates.
> + *
> + * Authors:
> + *   Bohdan Fedorov <misterjdrg at gmial.com>
> + *
> + * Based on:
> + *   sifive-uart
> + */
> +
> +#include <sbi/riscv_io.h>
> +#include <sbi/sbi_error.h>
> +#include <sbi/sbi_console.h>
> +#include <sbi_utils/serial/virtio-uart.h>
> +
> +/* clang-format off */
> +
> +#define VIRTIO_MAGIC                  0x0
> +#define VIRTIO_MAGIC_VALUE            0x74726976
> +#define VIRTIO_DEVICE_ID             0x8
> +#define VIRTIO_DEVICE_ID_CONSOLE      0x3
> +#define VIRTIO_CONSOLE_F_EMERG_WRITE  (1 << 2)
> +#define VIRTIO_DEVICE_FEATURES               0x10
> +#define VIRTIO_DEVICE_FEATURES_SEL    0x14
> +#define VIRTIO_CONSOLE_EMERG_WR       0x108
> +
> +/* clang-format on */
> +
> +static volatile void *uart_base;
> +
> +static u32 get_reg(u32 off)
> +{
> +       return readl(uart_base + off);
> +}
> +
> +static void set_reg(u32 off, u32 val)
> +{
> +       writel(val, uart_base + off);
> +}
> +
> +static void virtio_uart_putc(char ch)
> +{
> +       set_reg(VIRTIO_CONSOLE_EMERG_WR, ch);
> +}
> +
> +static struct sbi_console_device virtio_console = {
> +       .name = "virtio_uart",
> +       .console_putc = virtio_uart_putc,
> +       .console_getc = NULL,
> +};
> +
> +int virtio_uart_init(unsigned long base)
> +{
> +       uart_base = (volatile void *)base;
> +
> +       // Chech magic

Please use "/* */" comments instead of "//".

> +       if(get_reg(VIRTIO_MAGIC) != VIRTIO_MAGIC_VALUE) {
> +               return SBI_ENODEV;
> +       }
> +
> +       // Virtio device is console
> +       if(get_reg(VIRTIO_DEVICE_ID) != VIRTIO_DEVICE_ID_CONSOLE) {
> +               return SBI_ENODEV;
> +       }
> +
> +       // Virtio console has emergency write feature
> +       set_reg(VIRTIO_DEVICE_FEATURES_SEL, 0);
> +       if((get_reg(VIRTIO_DEVICE_FEATURES) & VIRTIO_CONSOLE_F_EMERG_WRITE) == 0) {
> +               return SBI_ENODEV;
> +       }
> +
> +       sbi_console_set_device(&virtio_console);
> +       return 0;
> +}
> --
> 2.36.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From atishp at atishpatra.org  Mon Jun 20 12:57:32 2022
From: atishp at atishpatra.org (Atish Patra)
Date: Mon, 20 Jun 2022 12:57:32 -0700
Subject: [PATCH] lib: sbi: Fix version dependency for Sscofpmf
In-Reply-To: <CAAhSdy32h8kvY5OLVQtUS7=m1Nx7Mpvi-u5-Gz1GTh2y4vNemg@mail.gmail.com>
References: <20220613003838.13028-1-samuel@sholland.org> <CAAhSdy0Y49egjSeOfTC0OkSEmTf03EK1cHECi-ARjZj3oNtWaA@mail.gmail.com>
 <97c665fe-7e52-cd56-c198-84e796fa8263@sholland.org> <CAAhSdy24ukPCh0LFOZVKtDyzHdxTWw348nhueQ-mUC5JyiLYAA@mail.gmail.com>
 <ba9f1314-0988-6c90-3460-34d2f29157ab@sholland.org> <CAAhSdy32h8kvY5OLVQtUS7=m1Nx7Mpvi-u5-Gz1GTh2y4vNemg@mail.gmail.com>
Message-ID: <CAOnJCUJN0sQ_BeRL+JqaJ6VZDResr1ooX+uf9PUNscAUh96dnQ@mail.gmail.com>

On Thu, Jun 16, 2022 at 11:44 PM Anup Patel <anup at brainfault.org> wrote:
>
> On Fri, Jun 17, 2022 at 10:49 AM Samuel Holland <samuel at sholland.org> wrote:
> >
> > On 6/16/22 11:56 PM, Anup Patel wrote:
> > > On Fri, Jun 17, 2022 at 9:58 AM Samuel Holland <samuel at sholland.org> wrote:
> > >>
> > >> Hi Anup,
> > >>
> > >> On 6/16/22 11:12 PM, Anup Patel wrote:
> > >>> On Mon, Jun 13, 2022 at 6:08 AM Samuel Holland <samuel at sholland.org> wrote:
> > >>>>
> > >>>> mcounteren was added in the privileged spec v1.10 and mcountinhibit was
> > >>>> added in v1.11. Sscofpmf does not depend on anything in v1.12, so the
> > >>>> minimum privileged spec version should have been v1.11.
> > >>>>
> > >>>> Fixes: d4b563c881d6 ("lib: sbi: Remove MCOUNTEREN and SCOUNTEREN hart features")
> > >>>> Signed-off-by: Samuel Holland <samuel at sholland.org>
> > >>>
> > >>> I think there is some confusion here.
> > >>>
> > >>> The mcounteren and mcountinhibit are not part of the Sscofpmf extension and
> > >>> OpenSBI PMU implementation does support platforms without Sscofpmf.
> > >>
> > >> I am aware of that. I am referencing the comment on the previous line, and the
> > >> code that was changed by the referenced commit. The code, as originally written,
> > >> attempted to optimize out the Sscofpmf check based on the presence of those two
> > >> CSRs.
> > >>
> > >> When the hart features were refactored, the CSR feature was replaced by the
> > >> wrong version, i.e. the behavior was accidentally changed by the refactoring.
> > >>
> > >> This patch only attempts to restore the behavior from before the refactoring. If
> > >> that behavior was wrong to start with, that is a different matter.
> > >
> > > I agree there is nothing in Sscofpmf which prevents a Priv v1.11 HART from
> > > implementing it so this now becomes a strange choice on the OpenSBI side.
> > >
> > > IMO, the previous behaviour of detecting and using Sscofpmf irrespective
> > > of the Priv spec version was wrong because Sscofpmf is defined assuming
> > > Priv v1.12 spec. In fact, that's why Sscofpmf defines bits for filtering events
> > > in VS/VU modes.
> > >
> > > Is there a case where we would need Sscofpmf for a Priv v1.11 HART ?
> >
> > I was attempting to add C906 PMU support as an errata on top of the standard
> > extension, which is when I noticed this check. (C906 supports v1.11). However, I
> > realized C906's custom overflow handling is different enough from Sscofpmf that
> > I should implement it separately.
> >

Yes. It is completely different from Sscofpmf with many more CSRs.

> > So I do not know if a v1.11+Sscofpmf case does or should exist. Maybe the check
> > is correct now, in which case the comment is a bit misleading.
>
> Instead of setting SSCOFPMF extension flag for C906, I suggest we extend
> sbi_pmu.c using "struct sbi_pmu_overflow_ops" abstraction. The sbi_pmu.c
> will have two implementations of "struct sbi_pmu_overflow_ops": 1) "dummy"
> implementation which will be used when there is no HW counter overflow
> support 2) "sscofpmf" implementation which will be used when HW support
> Sscofpmf extension.
>
> The platform override for Allwinner D1 platform can provide its own
> implementation of "struct sbi_pmu_overflow_ops".
>

IIRC, c906 PMU uses a different interrupt (not local) ID for overflow interrupt.
Even though we make it configurable in OpenSBI with "struct
sbi_pmu_overflow_ops"
upstream drive won't support it in the current version.

> Regards,
> Anup
>
> >
> > Regards,
> > Samuel
> >
> > >>>> ---
> > >>>>
> > >>>>  lib/sbi/sbi_hart.c | 2 +-
> > >>>>  1 file changed, 1 insertion(+), 1 deletion(-)
> > >>>>
> > >>>> diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
> > >>>> index de86fee..5bc6d52 100644
> > >>>> --- a/lib/sbi/sbi_hart.c
> > >>>> +++ b/lib/sbi/sbi_hart.c
> > >>>> @@ -645,7 +645,7 @@ __mhpm_skip:
> > >>>>                 hfeatures->priv_version = SBI_HART_PRIV_VER_1_12;
> > >>>>
> > >>>>         /* Counter overflow/filtering is not useful without mcounter/inhibit */
> > >>>> -       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_12) {
> > >>>> +       if (hfeatures->priv_version >= SBI_HART_PRIV_VER_1_11) {
> > >>>>                 /* Detect if hart supports sscofpmf */
> > >>>>                 csr_read_allowed(CSR_SCOUNTOVF, (unsigned long)&trap);
> > >>>>                 if (!trap.cause)
> > >>>> --
> > >>>> 2.35.1
> > >>>>
> > >>>>
> > >>>> --
> > >>>> opensbi mailing list
> > >>>> opensbi at lists.infradead.org
> > >>>> http://lists.infradead.org/mailman/listinfo/opensbi
> > >>
> >
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



-- 
Regards,
Atish


From atishp at atishpatra.org  Mon Jun 20 12:59:54 2022
From: atishp at atishpatra.org (Atish Patra)
Date: Mon, 20 Jun 2022 12:59:54 -0700
Subject: [PATCH] lib: utils: Remove CSRs that set/clear an IMSIC interrupt
 file bits
In-Reply-To: <20220616115856.1073885-1-apatel@ventanamicro.com>
References: <20220616115856.1073885-1-apatel@ventanamicro.com>
Message-ID: <CAOnJCU+kdBHE0d0hZBo0K5yzL-VB5pQNtQt7y1XL0mwj8XQBxQ@mail.gmail.com>

On Thu, Jun 16, 2022 at 4:59 AM Anup Patel <apatel at ventanamicro.com> wrote:
>
> Based on architecture review committee feedback, the [m|s|vs]seteienum,
> [m|s|vs]clreienum, [m|s|vs]seteipnum, and [m|s|vs]clreipnum CSRs are
> removed in the latest AIA draft v0.3.0 specification.
> (Refer, https://github.com/riscv/riscv-aia/releases/tag/0.3.0-draft.31)
>
> These CSRs were mostly for software convenience and software can always
> use [m|s|vs]iselect and [m|s|vs]ireg CSRs to update the IMSIC interrupt
> file bits.
>
> We update the IMSIC programming as-per above to match the latest AIA
> draft specification.
>
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> ---
>  include/sbi/riscv_encoding.h | 24 +++--------------
>  lib/utils/irqchip/imsic.c    | 51 +++++++++++++++++++++++++++++++-----
>  2 files changed, 48 insertions(+), 27 deletions(-)
>
> diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
> index a164768..68b4a7f 100644
> --- a/include/sbi/riscv_encoding.h
> +++ b/include/sbi/riscv_encoding.h
> @@ -336,14 +336,8 @@
>  #define CSR_SIREG                      0x151
>
>  /* Supervisor-Level Interrupts (AIA) */
> -#define CSR_STOPI                      0xdb0
> -
> -/* Supervisor-Level IMSIC Interface (AIA) */
> -#define CSR_SSETEIPNUM                 0x158
> -#define CSR_SCLREIPNUM                 0x159
> -#define CSR_SSETEIENUM                 0x15a
> -#define CSR_SCLREIENUM                 0x15b
>  #define CSR_STOPEI                     0x15c
> +#define CSR_STOPI                      0xdb0
>
>  /* Supervisor-Level High-Half CSRs (AIA) */
>  #define CSR_SIEH                       0x114
> @@ -405,14 +399,8 @@
>  #define CSR_VSIREG                     0x251
>
>  /* VS-Level Interrupts (H-extension with AIA) */
> -#define CSR_VSTOPI                     0xeb0
> -
> -/* VS-Level IMSIC Interface (H-extension with AIA) */
> -#define CSR_VSSETEIPNUM                0x258
> -#define CSR_VSCLREIPNUM                0x259
> -#define CSR_VSSETEIENUM                0x25a
> -#define CSR_VSCLREIENUM                0x25b
>  #define CSR_VSTOPEI                    0x25c
> +#define CSR_VSTOPI                     0xeb0
>
>  /* Hypervisor and VS-Level High-Half CSRs (H-extension with AIA) */
>  #define CSR_HIDELEGH                   0x613
> @@ -693,14 +681,8 @@
>  #define CSR_MIREG                      0x351
>
>  /* Machine-Level Interrupts (AIA) */
> -#define CSR_MTOPI                      0xfb0
> -
> -/* Machine-Level IMSIC Interface (AIA) */
> -#define CSR_MSETEIPNUM                 0x358
> -#define CSR_MCLREIPNUM                 0x359
> -#define CSR_MSETEIENUM                 0x35a
> -#define CSR_MCLREIENUM                 0x35b
>  #define CSR_MTOPEI                     0x35c
> +#define CSR_MTOPI                      0xfb0
>
>  /* Virtual Interrupts for Supervisor Level (AIA) */
>  #define CSR_MVIEN                      0x308
> diff --git a/lib/utils/irqchip/imsic.c b/lib/utils/irqchip/imsic.c
> index b29a1e9..98f2cb6 100644
> --- a/lib/utils/irqchip/imsic.c
> +++ b/lib/utils/irqchip/imsic.c
> @@ -38,10 +38,14 @@
>
>  #define IMSIC_EIP63                    0xbf
>
> +#define IMSIC_EIPx_BITS                        32
> +
>  #define IMSIC_EIE0                     0xc0
>
>  #define IMSIC_EIE63                    0xff
>
> +#define IMSIC_EIEx_BITS                        32
> +
>  #define IMSIC_DISABLE_EIDELIVERY       0
>  #define IMSIC_ENABLE_EIDELIVERY                1
>  #define IMSIC_DISABLE_EITHRESHOLD      1
> @@ -63,6 +67,18 @@ do { \
>         __v; \
>  })
>
> +#define imsic_csr_set(__c, __v)                \
> +do { \
> +       csr_write(CSR_MISELECT, __c); \
> +       csr_set(CSR_MIREG, __v); \
> +} while (0)
> +
> +#define imsic_csr_clear(__c, __v)      \
> +do { \
> +       csr_write(CSR_MISELECT, __c); \
> +       csr_clear(CSR_MIREG, __v); \
> +} while (0)
> +
>  static struct imsic_data *imsic_hartid2data[SBI_HARTMASK_MAX_BITS];
>  static int imsic_hartid2file[SBI_HARTMASK_MAX_BITS];
>
> @@ -140,6 +156,31 @@ static struct sbi_ipi_device imsic_ipi_device = {
>         .ipi_send       = imsic_ipi_send
>  };
>
> +static void imsic_local_eix_update(unsigned long base_id,
> +                                  unsigned long num_id, bool pend, bool val)
> +{
> +       unsigned long i, isel, ireg;
> +       unsigned long id = base_id, last_id = base_id + num_id;
> +
> +       while (id < last_id) {
> +               isel = id / __riscv_xlen;
> +               isel *= __riscv_xlen / IMSIC_EIPx_BITS;
> +               isel += (pend) ? IMSIC_EIP0 : IMSIC_EIE0;
> +
> +               ireg = 0;
> +               for (i = id & (__riscv_xlen - 1);
> +                    (id < last_id) && (i < __riscv_xlen); i++) {
> +                       ireg |= BIT(i);
> +                       id++;
> +               }
> +
> +               if (val)
> +                       imsic_csr_set(isel, ireg);
> +               else
> +                       imsic_csr_clear(isel, ireg);
> +       }
> +}
> +
>  void imsic_local_irqchip_init(void)
>  {
>         /*
> @@ -158,12 +199,11 @@ void imsic_local_irqchip_init(void)
>         imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_ENABLE_EIDELIVERY);
>
>         /* Enable IPI */
> -       csr_write(CSR_MSETEIENUM, IMSIC_IPI_ID);
> +       imsic_local_eix_update(IMSIC_IPI_ID, 1, false, true);
>  }
>
>  int imsic_warm_irqchip_init(void)
>  {
> -       unsigned long i;
>         struct imsic_data *imsic = imsic_hartid2data[current_hartid()];
>
>         /* Sanity checks */
> @@ -171,11 +211,10 @@ int imsic_warm_irqchip_init(void)
>                 return SBI_EINVAL;
>
>         /* Disable all interrupts */
> -       for (i = 1; i <= imsic->num_ids; i++)
> -               csr_write(CSR_MCLREIENUM, i);
> +       imsic_local_eix_update(1, imsic->num_ids, false, false);
>
> -       /* Clear IPI */
> -       csr_write(CSR_MCLREIPNUM, IMSIC_IPI_ID);
> +       /* Clear IPI pending */
> +       imsic_local_eix_update(IMSIC_IPI_ID, 1, true, false);
>
>         /* Local IMSIC initialization */
>         imsic_local_irqchip_init();
> --
> 2.34.1
>

Looks good to me.

Reviewed-by: Atish Patra <atishp at rivosinc.com>


-- 
Regards,
Atish


From anup at brainfault.org  Mon Jun 20 21:20:42 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 09:50:42 +0530
Subject: [RFC PATCH 2/2] lib: utils/timer: Add a separate compatible for
 the D1 CLINT
In-Reply-To: <20211020015838.45399-3-samuel@sholland.org>
References: <20211020015838.45399-1-samuel@sholland.org> <20211020015838.45399-3-samuel@sholland.org>
Message-ID: <CAAhSdy3r20Q4mBkTR58Befo+SYhsLx3ueq=MOb4UQXk3P=8F-w@mail.gmail.com>

On Wed, Oct 20, 2021 at 7:28 AM Samuel Holland <samuel at sholland.org> wrote:
>
> The CLINT in the Allwinner D1 SoC apparently does not support 64-bit
> MMIO access. A property was added to support this quirk (and that
> property was copied to the ACLINT MTIMER code). However, since this
> difference in behavior makes the D1 CLINT incompatible with the SiFive
> CLINT's programming interface, a better solution is to use a separate
> compatible string.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Due to some unknown reason, this patch slipped through cracks and was
never applied.

Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

> ---
>  docs/platform/thead-c9xx.md        |  4 +--
>  lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
>  lib/utils/timer/fdt_timer_mtimer.c | 39 +++++++++++++++++++-----------
>  3 files changed, 27 insertions(+), 17 deletions(-)
>
> diff --git a/docs/platform/thead-c9xx.md b/docs/platform/thead-c9xx.md
> index 3490ed5..ced784d 100644
> --- a/docs/platform/thead-c9xx.md
> +++ b/docs/platform/thead-c9xx.md
> @@ -52,12 +52,11 @@ DTS Example1: (Single core, eg: Allwinner D1 - c906)
>                 ranges;
>
>                 clint0: clint at 14000000 {
> -                       compatible = "riscv,clint0";
> +                       compatible = "allwinner,sun20i-d1-clint";
>                         interrupts-extended = <
>                                 &cpu0_intc  3 &cpu0_intc  7
>                                 >;
>                         reg = <0x0 0x14000000 0x0 0x04000000>;
> -                       clint,has-no-64bit-mmio;
>                 };
>
>                 intc: interrupt-controller at 10000000 {
> @@ -163,7 +162,6 @@ DTS Example2: (Multi cores with soc reset-regs)
>                                 &cpu4_intc  3 &cpu4_intc  7
>                                 >;
>                         reg = <0xff 0xdc000000 0x0 0x04000000>;
> -                       clint,has-no-64bit-mmio;
>                 };
>
>                 intc: interrupt-controller at ffd8000000 {
> diff --git a/lib/utils/ipi/fdt_ipi_mswi.c b/lib/utils/ipi/fdt_ipi_mswi.c
> index 0176941..af69e16 100644
> --- a/lib/utils/ipi/fdt_ipi_mswi.c
> +++ b/lib/utils/ipi/fdt_ipi_mswi.c
> @@ -54,6 +54,7 @@ static int ipi_mswi_cold_init(void *fdt, int nodeoff,
>  static const unsigned long clint_offset = CLINT_MSWI_OFFSET;
>
>  static const struct fdt_match ipi_mswi_match[] = {
> +       { .compatible = "allwinner,sun20i-d1-clint", .data = &clint_offset },
>         { .compatible = "riscv,clint0", .data = &clint_offset },
>         { .compatible = "sifive,clint0", .data = &clint_offset },
>         { .compatible = "riscv,aclint-mswi" },
> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> index 1ad8508..e140567 100644
> --- a/lib/utils/timer/fdt_timer_mtimer.c
> +++ b/lib/utils/timer/fdt_timer_mtimer.c
> @@ -15,6 +15,11 @@
>
>  #define MTIMER_MAX_NR                  16
>
> +struct timer_mtimer_quirks {
> +       unsigned int    mtime_offset;
> +       bool            has_64bit_mmio;
> +};
> +
>  static unsigned long mtimer_count = 0;
>  static struct aclint_mtimer_data mtimer[MTIMER_MAX_NR];
>  static struct aclint_mtimer_data *mt_reference = NULL;
> @@ -23,7 +28,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                                   const struct fdt_match *match)
>  {
>         int i, rc;
> -       unsigned long offset, addr[2], size[2];
> +       unsigned long addr[2], size[2];
>         struct aclint_mtimer_data *mt;
>
>         if (MTIMER_MAX_NR <= mtimer_count)
> @@ -43,28 +48,25 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                 return rc;
>
>         if (match->data) { /* SiFive CLINT */
> +               const struct timer_mtimer_quirks *quirks = match->data;
> +
>                 /* Set CLINT addresses */
>                 mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
>                 mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
>                 mt->mtime_addr = addr[0] + ACLINT_DEFAULT_MTIME_OFFSET;
>                 mt->mtime_size = size[0] - mt->mtimecmp_size;
>                 /* Adjust MTIMER address and size for CLINT device */
> -               offset = *((unsigned long *)match->data);
> -               mt->mtime_addr += offset;
> -               mt->mtimecmp_addr += offset;
> -               mt->mtime_size -= offset;
> -               /* Parse additional CLINT properties */
> -               if (fdt_getprop(fdt, nodeoff, "clint,has-no-64bit-mmio", &rc))
> -                       mt->has_64bit_mmio = false;
> +               mt->mtime_addr += quirks->mtime_offset;
> +               mt->mtimecmp_addr += quirks->mtime_offset;
> +               mt->mtime_size -= quirks->mtime_offset;
> +               /* Apply additional CLINT quirks */
> +               mt->has_64bit_mmio = quirks->has_64bit_mmio;
>         } else { /* RISC-V ACLINT MTIMER */
>                 /* Set ACLINT MTIMER addresses */
>                 mt->mtime_addr = addr[0];
>                 mt->mtime_size = size[0];
>                 mt->mtimecmp_addr = addr[1];
>                 mt->mtimecmp_size = size[1];
> -               /* Parse additional ACLINT MTIMER properties */
> -               if (fdt_getprop(fdt, nodeoff, "mtimer,no-64bit-mmio", &rc))
> -                       mt->has_64bit_mmio = false;
>         }
>
>         /* Check if MTIMER device has shared MTIME address */
> @@ -107,11 +109,20 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>         return 0;
>  }
>
> -static unsigned long clint_offset = CLINT_MTIMER_OFFSET;
> +static const struct timer_mtimer_quirks d1_clint_quirks = {
> +       .mtime_offset   = CLINT_MTIMER_OFFSET,
> +       .has_64bit_mmio = false,
> +};
> +
> +static const struct timer_mtimer_quirks sifive_clint_quirks = {
> +       .mtime_offset   = CLINT_MTIMER_OFFSET,
> +       .has_64bit_mmio = true,
> +};
>
>  static const struct fdt_match timer_mtimer_match[] = {
> -       { .compatible = "riscv,clint0", .data = &clint_offset },
> -       { .compatible = "sifive,clint0", .data = &clint_offset },
> +       { .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
> +       { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
> +       { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
>         { .compatible = "riscv,aclint-mtimer" },
>         { },
>  };
> --
> 2.32.0
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Mon Jun 20 21:21:32 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 09:51:32 +0530
Subject: [PATCH 1/3] include: sbi: Add mtinst/htinst psuedoinstructions
In-Reply-To: <CAAhSdy1xmJi5AB59xxOFuK1dyuXKwE77r_KpkBoB4AX2=s3m2w@mail.gmail.com>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <TYYP286MB14399472C06DD67BB8C55351C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <CAAhSdy1xmJi5AB59xxOFuK1dyuXKwE77r_KpkBoB4AX2=s3m2w@mail.gmail.com>
Message-ID: <CAAhSdy0DuE-k6xFM9mx9N=sK=kpGDNH=2TtDTYJKxw-YsNXsYA@mail.gmail.com>

On Fri, Jun 17, 2022 at 12:14 PM Anup Patel <anup at brainfault.org> wrote:
>
> On Thu, Jun 9, 2022 at 12:38 PM dramforever <dramforever at live.com> wrote:
> >
> > Add psuedoinstruction encodings written to mtinst/htinst for faults
> > caused by implicit memory access for VS-stage address translation
> >
> > Signed-off-by: dramforever <dramforever at live.com>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
> Regards,
> Anup
>
> > ---
> >  include/sbi/riscv_encoding.h | 20 ++++++++++++++++++++
> >  1 file changed, 20 insertions(+)
> >
> > diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
> > index a164768..2292858 100644
> > --- a/include/sbi/riscv_encoding.h
> > +++ b/include/sbi/riscv_encoding.h
> > @@ -841,6 +841,26 @@
> >  #define INSN_MASK_FENCE_TSO            0xffffffff
> >  #define INSN_MATCH_FENCE_TSO           0x8330000f
> >
> > +#if __riscv_xlen == 64
> > +
> > +/* 64-bit read for VS-stage address translation (RV64) */
> > +#define INSN_PSEUDO_VS_LOAD            0x00003000
> > +
> > +/* 64-bit write for VS-stage address translation (RV64) */
> > +#define INSN_PSEUDO_VS_STORE   0x00003020
> > +
> > +#elif __riscv_xlen == 32
> > +
> > +/* 32-bit read for VS-stage address translation (RV32) */
> > +#define INSN_PSEUDO_VS_LOAD            0x00002000
> > +
> > +/* 32-bit write for VS-stage address translation (RV32) */
> > +#define INSN_PSEUDO_VS_STORE   0x00002020
> > +
> > +#else
> > +#error "Unexpected __riscv_xlen"
> > +#endif
> > +
> >  #define INSN_16BIT_MASK                        0x3
> >  #define INSN_32BIT_MASK                        0x1c
> >
> > --
> > 2.36.0
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Mon Jun 20 21:21:45 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 09:51:45 +0530
Subject: [PATCH 2/3] lib: sbi: Fixup tinst for exceptions in sbi_misaligned_*()
In-Reply-To: <CAAhSdy2T=tVgsxCPS+A36x39idwnSQSg+TbA_Bt11QthYSbg9w@mail.gmail.com>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <TYYP286MB14399AD2F6CFFB0CECD334E9C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <CAAhSdy2T=tVgsxCPS+A36x39idwnSQSg+TbA_Bt11QthYSbg9w@mail.gmail.com>
Message-ID: <CAAhSdy0_wCu+q7LtCGM2CVGRWjd4HHwguwwTgU=kuUGpKDmPSA@mail.gmail.com>

On Fri, Jun 17, 2022 at 12:15 PM Anup Patel <anup at brainfault.org> wrote:
>
> On Thu, Jun 9, 2022 at 12:38 PM dramforever <dramforever at live.com> wrote:
> >
> > If there is an exception while emulating a misaligned load/store, fixup
> > uptrap.tinst before redirecting. Otherwise, HS-mode software may receive
> > an htinst describing the lbu/sb instruction that faulted during
> > emulation[1].
> >
> > [1]: https://github.com/riscv-software-src/opensbi/issues/258
> >
> > Signed-off-by: dramforever <dramforever at live.com>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
> Regards,
> Anup
>
> > ---
> >  lib/sbi/sbi_misaligned_ldst.c | 16 ++++++++++++++++
> >  1 file changed, 16 insertions(+)
> >
> > diff --git a/lib/sbi/sbi_misaligned_ldst.c b/lib/sbi/sbi_misaligned_ldst.c
> > index c879ce7..fd11798 100644
> > --- a/lib/sbi/sbi_misaligned_ldst.c
> > +++ b/lib/sbi/sbi_misaligned_ldst.c
> > @@ -22,6 +22,18 @@ union reg_data {
> >         u64 data_u64;
> >  };
> >
> > +static ulong sbi_misaligned_tinst_fixup(ulong orig_tinst, ulong new_tinst,
> > +                                       ulong addr_offset)
> > +{
> > +       if (new_tinst == INSN_PSEUDO_VS_LOAD ||
> > +           new_tinst == INSN_PSEUDO_VS_STORE)
> > +               return new_tinst;
> > +       else if (orig_tinst == 0)
> > +               return 0UL;
> > +       else
> > +               return orig_tinst | (addr_offset << SH_RS1);
> > +}
> > +
> >  int sbi_misaligned_load_handler(ulong addr, ulong tval2, ulong tinst,
> >                                 struct sbi_trap_regs *regs)
> >  {
> > @@ -126,6 +138,8 @@ int sbi_misaligned_load_handler(ulong addr, ulong tval2, ulong tinst,
> >                                                 &uptrap);
> >                 if (uptrap.cause) {
> >                         uptrap.epc = regs->mepc;
> > +                       uptrap.tinst = sbi_misaligned_tinst_fixup(
> > +                               tinst, uptrap.tinst, i);
> >                         return sbi_trap_redirect(regs, &uptrap);
> >                 }
> >         }
> > @@ -238,6 +252,8 @@ int sbi_misaligned_store_handler(ulong addr, ulong tval2, ulong tinst,
> >                              &uptrap);
> >                 if (uptrap.cause) {
> >                         uptrap.epc = regs->mepc;
> > +                       uptrap.tinst = sbi_misaligned_tinst_fixup(
> > +                               tinst, uptrap.tinst, i);
> >                         return sbi_trap_redirect(regs, &uptrap);
> >                 }
> >         }
> > --
> > 2.36.0
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Mon Jun 20 21:22:03 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 09:52:03 +0530
Subject: [PATCH 3/3] lib: sbi: Fix tval and tinst for sbi_get_insn()
In-Reply-To: <CAAhSdy3oT8JNB2X8TG_xWE9qVuX-k4BRo-JHP3s_uiKnRKP11Q@mail.gmail.com>
References: <TYYP286MB1439F38753D033042F83C8E0C6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <TYYP286MB14393C382F94A378E9F529CDC6A79@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
 <CAAhSdy3oT8JNB2X8TG_xWE9qVuX-k4BRo-JHP3s_uiKnRKP11Q@mail.gmail.com>
Message-ID: <CAAhSdy20kgPonjvjVwRB853q7+_cOmQdA3t2Zeeh1NWbixjQ3g@mail.gmail.com>

On Sat, Jun 18, 2022 at 11:28 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Thu, Jun 9, 2022 at 12:38 PM dramforever <dramforever at live.com> wrote:
> >
> > We should not change trap->tval to mepc because mtval already points to
> > the faulting portion of the emulated instruction fetch, which is also
> > what stval is expected to be.
> >
> > In addition, htinst is only allowed to be zero for instruction access
> > faults or page faults, and is only allowed to be zero or a
> > psuedoinstruction for instruction guest-page faults. Fix trap->tinst for
> > these cases.
> >
> > Signed-off-by: dramforever <dramforever at live.com>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
> Regards,
> Anup
>
> > ---
> >  lib/sbi/sbi_unpriv.c | 8 +++++---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/lib/sbi/sbi_unpriv.c b/lib/sbi/sbi_unpriv.c
> > index 73b530c..9a34a71 100644
> > --- a/lib/sbi/sbi_unpriv.c
> > +++ b/lib/sbi/sbi_unpriv.c
> > @@ -149,15 +149,17 @@ ulong sbi_get_insn(ulong mepc, struct sbi_trap_info *trap)
> >         switch (trap->cause) {
> >         case CAUSE_LOAD_ACCESS:
> >                 trap->cause = CAUSE_FETCH_ACCESS;
> > -               trap->tval = mepc;
> > +               trap->tinst = 0UL;
> >                 break;
> >         case CAUSE_LOAD_PAGE_FAULT:
> >                 trap->cause = CAUSE_FETCH_PAGE_FAULT;
> > -               trap->tval = mepc;
> > +               trap->tinst = 0UL;
> >                 break;
> >         case CAUSE_LOAD_GUEST_PAGE_FAULT:
> >                 trap->cause = CAUSE_FETCH_GUEST_PAGE_FAULT;
> > -               trap->tval = mepc;
> > +               if (trap->tinst != INSN_PSEUDO_VS_LOAD &&
> > +                   trap->tinst != INSN_PSEUDO_VS_STORE)
> > +                       trap->tinst = 0UL;
> >                 break;
> >         default:
> >                 break;
> > --
> > 2.36.0
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Mon Jun 20 21:22:31 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 09:52:31 +0530
Subject: [PATCH] lib: utils: Remove CSRs that set/clear an IMSIC interrupt
 file bits
In-Reply-To: <CAOnJCU+kdBHE0d0hZBo0K5yzL-VB5pQNtQt7y1XL0mwj8XQBxQ@mail.gmail.com>
References: <20220616115856.1073885-1-apatel@ventanamicro.com> <CAOnJCU+kdBHE0d0hZBo0K5yzL-VB5pQNtQt7y1XL0mwj8XQBxQ@mail.gmail.com>
Message-ID: <CAAhSdy1yhA07CLLZMeWRna7mK=+noW72efCqZED3gRP4F22vMg@mail.gmail.com>

On Tue, Jun 21, 2022 at 1:30 AM Atish Patra <atishp at atishpatra.org> wrote:
>
> On Thu, Jun 16, 2022 at 4:59 AM Anup Patel <apatel at ventanamicro.com> wrote:
> >
> > Based on architecture review committee feedback, the [m|s|vs]seteienum,
> > [m|s|vs]clreienum, [m|s|vs]seteipnum, and [m|s|vs]clreipnum CSRs are
> > removed in the latest AIA draft v0.3.0 specification.
> > (Refer, https://github.com/riscv/riscv-aia/releases/tag/0.3.0-draft.31)
> >
> > These CSRs were mostly for software convenience and software can always
> > use [m|s|vs]iselect and [m|s|vs]ireg CSRs to update the IMSIC interrupt
> > file bits.
> >
> > We update the IMSIC programming as-per above to match the latest AIA
> > draft specification.
> >
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > ---
> >  include/sbi/riscv_encoding.h | 24 +++--------------
> >  lib/utils/irqchip/imsic.c    | 51 +++++++++++++++++++++++++++++++-----
> >  2 files changed, 48 insertions(+), 27 deletions(-)
> >
> > diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
> > index a164768..68b4a7f 100644
> > --- a/include/sbi/riscv_encoding.h
> > +++ b/include/sbi/riscv_encoding.h
> > @@ -336,14 +336,8 @@
> >  #define CSR_SIREG                      0x151
> >
> >  /* Supervisor-Level Interrupts (AIA) */
> > -#define CSR_STOPI                      0xdb0
> > -
> > -/* Supervisor-Level IMSIC Interface (AIA) */
> > -#define CSR_SSETEIPNUM                 0x158
> > -#define CSR_SCLREIPNUM                 0x159
> > -#define CSR_SSETEIENUM                 0x15a
> > -#define CSR_SCLREIENUM                 0x15b
> >  #define CSR_STOPEI                     0x15c
> > +#define CSR_STOPI                      0xdb0
> >
> >  /* Supervisor-Level High-Half CSRs (AIA) */
> >  #define CSR_SIEH                       0x114
> > @@ -405,14 +399,8 @@
> >  #define CSR_VSIREG                     0x251
> >
> >  /* VS-Level Interrupts (H-extension with AIA) */
> > -#define CSR_VSTOPI                     0xeb0
> > -
> > -/* VS-Level IMSIC Interface (H-extension with AIA) */
> > -#define CSR_VSSETEIPNUM                0x258
> > -#define CSR_VSCLREIPNUM                0x259
> > -#define CSR_VSSETEIENUM                0x25a
> > -#define CSR_VSCLREIENUM                0x25b
> >  #define CSR_VSTOPEI                    0x25c
> > +#define CSR_VSTOPI                     0xeb0
> >
> >  /* Hypervisor and VS-Level High-Half CSRs (H-extension with AIA) */
> >  #define CSR_HIDELEGH                   0x613
> > @@ -693,14 +681,8 @@
> >  #define CSR_MIREG                      0x351
> >
> >  /* Machine-Level Interrupts (AIA) */
> > -#define CSR_MTOPI                      0xfb0
> > -
> > -/* Machine-Level IMSIC Interface (AIA) */
> > -#define CSR_MSETEIPNUM                 0x358
> > -#define CSR_MCLREIPNUM                 0x359
> > -#define CSR_MSETEIENUM                 0x35a
> > -#define CSR_MCLREIENUM                 0x35b
> >  #define CSR_MTOPEI                     0x35c
> > +#define CSR_MTOPI                      0xfb0
> >
> >  /* Virtual Interrupts for Supervisor Level (AIA) */
> >  #define CSR_MVIEN                      0x308
> > diff --git a/lib/utils/irqchip/imsic.c b/lib/utils/irqchip/imsic.c
> > index b29a1e9..98f2cb6 100644
> > --- a/lib/utils/irqchip/imsic.c
> > +++ b/lib/utils/irqchip/imsic.c
> > @@ -38,10 +38,14 @@
> >
> >  #define IMSIC_EIP63                    0xbf
> >
> > +#define IMSIC_EIPx_BITS                        32
> > +
> >  #define IMSIC_EIE0                     0xc0
> >
> >  #define IMSIC_EIE63                    0xff
> >
> > +#define IMSIC_EIEx_BITS                        32
> > +
> >  #define IMSIC_DISABLE_EIDELIVERY       0
> >  #define IMSIC_ENABLE_EIDELIVERY                1
> >  #define IMSIC_DISABLE_EITHRESHOLD      1
> > @@ -63,6 +67,18 @@ do { \
> >         __v; \
> >  })
> >
> > +#define imsic_csr_set(__c, __v)                \
> > +do { \
> > +       csr_write(CSR_MISELECT, __c); \
> > +       csr_set(CSR_MIREG, __v); \
> > +} while (0)
> > +
> > +#define imsic_csr_clear(__c, __v)      \
> > +do { \
> > +       csr_write(CSR_MISELECT, __c); \
> > +       csr_clear(CSR_MIREG, __v); \
> > +} while (0)
> > +
> >  static struct imsic_data *imsic_hartid2data[SBI_HARTMASK_MAX_BITS];
> >  static int imsic_hartid2file[SBI_HARTMASK_MAX_BITS];
> >
> > @@ -140,6 +156,31 @@ static struct sbi_ipi_device imsic_ipi_device = {
> >         .ipi_send       = imsic_ipi_send
> >  };
> >
> > +static void imsic_local_eix_update(unsigned long base_id,
> > +                                  unsigned long num_id, bool pend, bool val)
> > +{
> > +       unsigned long i, isel, ireg;
> > +       unsigned long id = base_id, last_id = base_id + num_id;
> > +
> > +       while (id < last_id) {
> > +               isel = id / __riscv_xlen;
> > +               isel *= __riscv_xlen / IMSIC_EIPx_BITS;
> > +               isel += (pend) ? IMSIC_EIP0 : IMSIC_EIE0;
> > +
> > +               ireg = 0;
> > +               for (i = id & (__riscv_xlen - 1);
> > +                    (id < last_id) && (i < __riscv_xlen); i++) {
> > +                       ireg |= BIT(i);
> > +                       id++;
> > +               }
> > +
> > +               if (val)
> > +                       imsic_csr_set(isel, ireg);
> > +               else
> > +                       imsic_csr_clear(isel, ireg);
> > +       }
> > +}
> > +
> >  void imsic_local_irqchip_init(void)
> >  {
> >         /*
> > @@ -158,12 +199,11 @@ void imsic_local_irqchip_init(void)
> >         imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_ENABLE_EIDELIVERY);
> >
> >         /* Enable IPI */
> > -       csr_write(CSR_MSETEIENUM, IMSIC_IPI_ID);
> > +       imsic_local_eix_update(IMSIC_IPI_ID, 1, false, true);
> >  }
> >
> >  int imsic_warm_irqchip_init(void)
> >  {
> > -       unsigned long i;
> >         struct imsic_data *imsic = imsic_hartid2data[current_hartid()];
> >
> >         /* Sanity checks */
> > @@ -171,11 +211,10 @@ int imsic_warm_irqchip_init(void)
> >                 return SBI_EINVAL;
> >
> >         /* Disable all interrupts */
> > -       for (i = 1; i <= imsic->num_ids; i++)
> > -               csr_write(CSR_MCLREIENUM, i);
> > +       imsic_local_eix_update(1, imsic->num_ids, false, false);
> >
> > -       /* Clear IPI */
> > -       csr_write(CSR_MCLREIPNUM, IMSIC_IPI_ID);
> > +       /* Clear IPI pending */
> > +       imsic_local_eix_update(IMSIC_IPI_ID, 1, true, false);
> >
> >         /* Local IMSIC initialization */
> >         imsic_local_irqchip_init();
> > --
> > 2.34.1
> >
>
> Looks good to me.
>
> Reviewed-by: Atish Patra <atishp at rivosinc.com>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
>
> --
> Regards,
> Atish


From samuel at sholland.org  Mon Jun 20 21:43:54 2022
From: samuel at sholland.org (Samuel Holland)
Date: Mon, 20 Jun 2022 23:43:54 -0500
Subject: [RFC PATCH 2/2] lib: utils/timer: Add a separate compatible for
 the D1 CLINT
In-Reply-To: <CAAhSdy3r20Q4mBkTR58Befo+SYhsLx3ueq=MOb4UQXk3P=8F-w@mail.gmail.com>
References: <20211020015838.45399-1-samuel@sholland.org>
 <20211020015838.45399-3-samuel@sholland.org>
 <CAAhSdy3r20Q4mBkTR58Befo+SYhsLx3ueq=MOb4UQXk3P=8F-w@mail.gmail.com>
Message-ID: <d57d4cb7-bf64-c1c8-a0bf-174e36207d3c@sholland.org>

On 6/20/22 11:20 PM, Anup Patel wrote:
> On Wed, Oct 20, 2021 at 7:28 AM Samuel Holland <samuel at sholland.org> wrote:
>>
>> The CLINT in the Allwinner D1 SoC apparently does not support 64-bit
>> MMIO access. A property was added to support this quirk (and that
>> property was copied to the ACLINT MTIMER code). However, since this
>> difference in behavior makes the D1 CLINT incompatible with the SiFive
>> CLINT's programming interface, a better solution is to use a separate
>> compatible string.
>>
>> Signed-off-by: Samuel Holland <samuel at sholland.org>
> 
> Due to some unknown reason, this patch slipped through cracks and was
> never applied.

Oh, I intentionally did not send a new version of the series because I later
found out that C9xx CLINT has no mtime register at all[1][2]. The issue has
nothing to do with 64-bit vs 32-bit access. So this change is wrong; I will need
to send a follow-up removing the allwinner,sun20i-d1-clint compatibles.

Regards,
Samuel

[1]:
https://github.com/T-head-Semi/openc906/blob/main/C906_RTL_FACTORY/gen_rtl/clint/rtl/clint_func.v#L217
[2]:
https://github.com/T-head-Semi/openc910/blob/main/C910_RTL_FACTORY/gen_rtl/clint/rtl/ct_clint_func.v#L247

> Reviewed-by: Anup Patel <anup at brainfault.org>
> 
> Applied this patch to the riscv/opensbi repo.
> 
> Thanks,
> Anup
> 
>> ---
>>  docs/platform/thead-c9xx.md        |  4 +--
>>  lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
>>  lib/utils/timer/fdt_timer_mtimer.c | 39 +++++++++++++++++++-----------
>>  3 files changed, 27 insertions(+), 17 deletions(-)
>>
>> diff --git a/docs/platform/thead-c9xx.md b/docs/platform/thead-c9xx.md
>> index 3490ed5..ced784d 100644
>> --- a/docs/platform/thead-c9xx.md
>> +++ b/docs/platform/thead-c9xx.md
>> @@ -52,12 +52,11 @@ DTS Example1: (Single core, eg: Allwinner D1 - c906)
>>                 ranges;
>>
>>                 clint0: clint at 14000000 {
>> -                       compatible = "riscv,clint0";
>> +                       compatible = "allwinner,sun20i-d1-clint";
>>                         interrupts-extended = <
>>                                 &cpu0_intc  3 &cpu0_intc  7
>>                                 >;
>>                         reg = <0x0 0x14000000 0x0 0x04000000>;
>> -                       clint,has-no-64bit-mmio;
>>                 };
>>
>>                 intc: interrupt-controller at 10000000 {
>> @@ -163,7 +162,6 @@ DTS Example2: (Multi cores with soc reset-regs)
>>                                 &cpu4_intc  3 &cpu4_intc  7
>>                                 >;
>>                         reg = <0xff 0xdc000000 0x0 0x04000000>;
>> -                       clint,has-no-64bit-mmio;
>>                 };
>>
>>                 intc: interrupt-controller at ffd8000000 {
>> diff --git a/lib/utils/ipi/fdt_ipi_mswi.c b/lib/utils/ipi/fdt_ipi_mswi.c
>> index 0176941..af69e16 100644
>> --- a/lib/utils/ipi/fdt_ipi_mswi.c
>> +++ b/lib/utils/ipi/fdt_ipi_mswi.c
>> @@ -54,6 +54,7 @@ static int ipi_mswi_cold_init(void *fdt, int nodeoff,
>>  static const unsigned long clint_offset = CLINT_MSWI_OFFSET;
>>
>>  static const struct fdt_match ipi_mswi_match[] = {
>> +       { .compatible = "allwinner,sun20i-d1-clint", .data = &clint_offset },
>>         { .compatible = "riscv,clint0", .data = &clint_offset },
>>         { .compatible = "sifive,clint0", .data = &clint_offset },
>>         { .compatible = "riscv,aclint-mswi" },
>> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
>> index 1ad8508..e140567 100644
>> --- a/lib/utils/timer/fdt_timer_mtimer.c
>> +++ b/lib/utils/timer/fdt_timer_mtimer.c
>> @@ -15,6 +15,11 @@
>>
>>  #define MTIMER_MAX_NR                  16
>>
>> +struct timer_mtimer_quirks {
>> +       unsigned int    mtime_offset;
>> +       bool            has_64bit_mmio;
>> +};
>> +
>>  static unsigned long mtimer_count = 0;
>>  static struct aclint_mtimer_data mtimer[MTIMER_MAX_NR];
>>  static struct aclint_mtimer_data *mt_reference = NULL;
>> @@ -23,7 +28,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>>                                   const struct fdt_match *match)
>>  {
>>         int i, rc;
>> -       unsigned long offset, addr[2], size[2];
>> +       unsigned long addr[2], size[2];
>>         struct aclint_mtimer_data *mt;
>>
>>         if (MTIMER_MAX_NR <= mtimer_count)
>> @@ -43,28 +48,25 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>>                 return rc;
>>
>>         if (match->data) { /* SiFive CLINT */
>> +               const struct timer_mtimer_quirks *quirks = match->data;
>> +
>>                 /* Set CLINT addresses */
>>                 mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
>>                 mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
>>                 mt->mtime_addr = addr[0] + ACLINT_DEFAULT_MTIME_OFFSET;
>>                 mt->mtime_size = size[0] - mt->mtimecmp_size;
>>                 /* Adjust MTIMER address and size for CLINT device */
>> -               offset = *((unsigned long *)match->data);
>> -               mt->mtime_addr += offset;
>> -               mt->mtimecmp_addr += offset;
>> -               mt->mtime_size -= offset;
>> -               /* Parse additional CLINT properties */
>> -               if (fdt_getprop(fdt, nodeoff, "clint,has-no-64bit-mmio", &rc))
>> -                       mt->has_64bit_mmio = false;
>> +               mt->mtime_addr += quirks->mtime_offset;
>> +               mt->mtimecmp_addr += quirks->mtime_offset;
>> +               mt->mtime_size -= quirks->mtime_offset;
>> +               /* Apply additional CLINT quirks */
>> +               mt->has_64bit_mmio = quirks->has_64bit_mmio;
>>         } else { /* RISC-V ACLINT MTIMER */
>>                 /* Set ACLINT MTIMER addresses */
>>                 mt->mtime_addr = addr[0];
>>                 mt->mtime_size = size[0];
>>                 mt->mtimecmp_addr = addr[1];
>>                 mt->mtimecmp_size = size[1];
>> -               /* Parse additional ACLINT MTIMER properties */
>> -               if (fdt_getprop(fdt, nodeoff, "mtimer,no-64bit-mmio", &rc))
>> -                       mt->has_64bit_mmio = false;
>>         }
>>
>>         /* Check if MTIMER device has shared MTIME address */
>> @@ -107,11 +109,20 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>>         return 0;
>>  }
>>
>> -static unsigned long clint_offset = CLINT_MTIMER_OFFSET;
>> +static const struct timer_mtimer_quirks d1_clint_quirks = {
>> +       .mtime_offset   = CLINT_MTIMER_OFFSET,
>> +       .has_64bit_mmio = false,
>> +};
>> +
>> +static const struct timer_mtimer_quirks sifive_clint_quirks = {
>> +       .mtime_offset   = CLINT_MTIMER_OFFSET,
>> +       .has_64bit_mmio = true,
>> +};
>>
>>  static const struct fdt_match timer_mtimer_match[] = {
>> -       { .compatible = "riscv,clint0", .data = &clint_offset },
>> -       { .compatible = "sifive,clint0", .data = &clint_offset },
>> +       { .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
>> +       { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
>> +       { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
>>         { .compatible = "riscv,aclint-mtimer" },
>>         { },
>>  };
>> --
>> 2.32.0
>>
>>
>> --
>> opensbi mailing list
>> opensbi at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/opensbi



From anup at brainfault.org  Mon Jun 20 21:54:06 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 10:24:06 +0530
Subject: [RFC PATCH 2/2] lib: utils/timer: Add a separate compatible for
 the D1 CLINT
In-Reply-To: <d57d4cb7-bf64-c1c8-a0bf-174e36207d3c@sholland.org>
References: <20211020015838.45399-1-samuel@sholland.org> <20211020015838.45399-3-samuel@sholland.org>
 <CAAhSdy3r20Q4mBkTR58Befo+SYhsLx3ueq=MOb4UQXk3P=8F-w@mail.gmail.com> <d57d4cb7-bf64-c1c8-a0bf-174e36207d3c@sholland.org>
Message-ID: <CAAhSdy0+2GY1U==iGHn5udFwSWUTEEKbtG=fSxu2vC4As1E2gg@mail.gmail.com>

On Tue, Jun 21, 2022 at 10:13 AM Samuel Holland <samuel at sholland.org> wrote:
>
> On 6/20/22 11:20 PM, Anup Patel wrote:
> > On Wed, Oct 20, 2021 at 7:28 AM Samuel Holland <samuel at sholland.org> wrote:
> >>
> >> The CLINT in the Allwinner D1 SoC apparently does not support 64-bit
> >> MMIO access. A property was added to support this quirk (and that
> >> property was copied to the ACLINT MTIMER code). However, since this
> >> difference in behavior makes the D1 CLINT incompatible with the SiFive
> >> CLINT's programming interface, a better solution is to use a separate
> >> compatible string.
> >>
> >> Signed-off-by: Samuel Holland <samuel at sholland.org>
> >
> > Due to some unknown reason, this patch slipped through cracks and was
> > never applied.
>
> Oh, I intentionally did not send a new version of the series because I later
> found out that C9xx CLINT has no mtime register at all[1][2]. The issue has
> nothing to do with 64-bit vs 32-bit access. So this change is wrong; I will need
> to send a follow-up removing the allwinner,sun20i-d1-clint compatibles.

I had tested on QEMU virt and sifive_u machine so at least this does not
break anything on QEMU and SiFive boards.

Just a heads-up, we are planning to do OpenSBI v1.1 release sometime
this week.

Do you have a list of things that are still missing to support upstream
OpenSBI on D1 board ?

Regards,
Anup

>
> Regards,
> Samuel
>
> [1]:
> https://github.com/T-head-Semi/openc906/blob/main/C906_RTL_FACTORY/gen_rtl/clint/rtl/clint_func.v#L217
> [2]:
> https://github.com/T-head-Semi/openc910/blob/main/C910_RTL_FACTORY/gen_rtl/clint/rtl/ct_clint_func.v#L247
>
> > Reviewed-by: Anup Patel <anup at brainfault.org>
> >
> > Applied this patch to the riscv/opensbi repo.
> >
> > Thanks,
> > Anup
> >
> >> ---
> >>  docs/platform/thead-c9xx.md        |  4 +--
> >>  lib/utils/ipi/fdt_ipi_mswi.c       |  1 +
> >>  lib/utils/timer/fdt_timer_mtimer.c | 39 +++++++++++++++++++-----------
> >>  3 files changed, 27 insertions(+), 17 deletions(-)
> >>
> >> diff --git a/docs/platform/thead-c9xx.md b/docs/platform/thead-c9xx.md
> >> index 3490ed5..ced784d 100644
> >> --- a/docs/platform/thead-c9xx.md
> >> +++ b/docs/platform/thead-c9xx.md
> >> @@ -52,12 +52,11 @@ DTS Example1: (Single core, eg: Allwinner D1 - c906)
> >>                 ranges;
> >>
> >>                 clint0: clint at 14000000 {
> >> -                       compatible = "riscv,clint0";
> >> +                       compatible = "allwinner,sun20i-d1-clint";
> >>                         interrupts-extended = <
> >>                                 &cpu0_intc  3 &cpu0_intc  7
> >>                                 >;
> >>                         reg = <0x0 0x14000000 0x0 0x04000000>;
> >> -                       clint,has-no-64bit-mmio;
> >>                 };
> >>
> >>                 intc: interrupt-controller at 10000000 {
> >> @@ -163,7 +162,6 @@ DTS Example2: (Multi cores with soc reset-regs)
> >>                                 &cpu4_intc  3 &cpu4_intc  7
> >>                                 >;
> >>                         reg = <0xff 0xdc000000 0x0 0x04000000>;
> >> -                       clint,has-no-64bit-mmio;
> >>                 };
> >>
> >>                 intc: interrupt-controller at ffd8000000 {
> >> diff --git a/lib/utils/ipi/fdt_ipi_mswi.c b/lib/utils/ipi/fdt_ipi_mswi.c
> >> index 0176941..af69e16 100644
> >> --- a/lib/utils/ipi/fdt_ipi_mswi.c
> >> +++ b/lib/utils/ipi/fdt_ipi_mswi.c
> >> @@ -54,6 +54,7 @@ static int ipi_mswi_cold_init(void *fdt, int nodeoff,
> >>  static const unsigned long clint_offset = CLINT_MSWI_OFFSET;
> >>
> >>  static const struct fdt_match ipi_mswi_match[] = {
> >> +       { .compatible = "allwinner,sun20i-d1-clint", .data = &clint_offset },
> >>         { .compatible = "riscv,clint0", .data = &clint_offset },
> >>         { .compatible = "sifive,clint0", .data = &clint_offset },
> >>         { .compatible = "riscv,aclint-mswi" },
> >> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> >> index 1ad8508..e140567 100644
> >> --- a/lib/utils/timer/fdt_timer_mtimer.c
> >> +++ b/lib/utils/timer/fdt_timer_mtimer.c
> >> @@ -15,6 +15,11 @@
> >>
> >>  #define MTIMER_MAX_NR                  16
> >>
> >> +struct timer_mtimer_quirks {
> >> +       unsigned int    mtime_offset;
> >> +       bool            has_64bit_mmio;
> >> +};
> >> +
> >>  static unsigned long mtimer_count = 0;
> >>  static struct aclint_mtimer_data mtimer[MTIMER_MAX_NR];
> >>  static struct aclint_mtimer_data *mt_reference = NULL;
> >> @@ -23,7 +28,7 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> >>                                   const struct fdt_match *match)
> >>  {
> >>         int i, rc;
> >> -       unsigned long offset, addr[2], size[2];
> >> +       unsigned long addr[2], size[2];
> >>         struct aclint_mtimer_data *mt;
> >>
> >>         if (MTIMER_MAX_NR <= mtimer_count)
> >> @@ -43,28 +48,25 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> >>                 return rc;
> >>
> >>         if (match->data) { /* SiFive CLINT */
> >> +               const struct timer_mtimer_quirks *quirks = match->data;
> >> +
> >>                 /* Set CLINT addresses */
> >>                 mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
> >>                 mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
> >>                 mt->mtime_addr = addr[0] + ACLINT_DEFAULT_MTIME_OFFSET;
> >>                 mt->mtime_size = size[0] - mt->mtimecmp_size;
> >>                 /* Adjust MTIMER address and size for CLINT device */
> >> -               offset = *((unsigned long *)match->data);
> >> -               mt->mtime_addr += offset;
> >> -               mt->mtimecmp_addr += offset;
> >> -               mt->mtime_size -= offset;
> >> -               /* Parse additional CLINT properties */
> >> -               if (fdt_getprop(fdt, nodeoff, "clint,has-no-64bit-mmio", &rc))
> >> -                       mt->has_64bit_mmio = false;
> >> +               mt->mtime_addr += quirks->mtime_offset;
> >> +               mt->mtimecmp_addr += quirks->mtime_offset;
> >> +               mt->mtime_size -= quirks->mtime_offset;
> >> +               /* Apply additional CLINT quirks */
> >> +               mt->has_64bit_mmio = quirks->has_64bit_mmio;
> >>         } else { /* RISC-V ACLINT MTIMER */
> >>                 /* Set ACLINT MTIMER addresses */
> >>                 mt->mtime_addr = addr[0];
> >>                 mt->mtime_size = size[0];
> >>                 mt->mtimecmp_addr = addr[1];
> >>                 mt->mtimecmp_size = size[1];
> >> -               /* Parse additional ACLINT MTIMER properties */
> >> -               if (fdt_getprop(fdt, nodeoff, "mtimer,no-64bit-mmio", &rc))
> >> -                       mt->has_64bit_mmio = false;
> >>         }
> >>
> >>         /* Check if MTIMER device has shared MTIME address */
> >> @@ -107,11 +109,20 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> >>         return 0;
> >>  }
> >>
> >> -static unsigned long clint_offset = CLINT_MTIMER_OFFSET;
> >> +static const struct timer_mtimer_quirks d1_clint_quirks = {
> >> +       .mtime_offset   = CLINT_MTIMER_OFFSET,
> >> +       .has_64bit_mmio = false,
> >> +};
> >> +
> >> +static const struct timer_mtimer_quirks sifive_clint_quirks = {
> >> +       .mtime_offset   = CLINT_MTIMER_OFFSET,
> >> +       .has_64bit_mmio = true,
> >> +};
> >>
> >>  static const struct fdt_match timer_mtimer_match[] = {
> >> -       { .compatible = "riscv,clint0", .data = &clint_offset },
> >> -       { .compatible = "sifive,clint0", .data = &clint_offset },
> >> +       { .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
> >> +       { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
> >> +       { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
> >>         { .compatible = "riscv,aclint-mtimer" },
> >>         { },
> >>  };
> >> --
> >> 2.32.0
> >>
> >>
> >> --
> >> opensbi mailing list
> >> opensbi at lists.infradead.org
> >> http://lists.infradead.org/mailman/listinfo/opensbi
>


From samuel at sholland.org  Mon Jun 20 22:20:50 2022
From: samuel at sholland.org (Samuel Holland)
Date: Tue, 21 Jun 2022 00:20:50 -0500
Subject: [RFC PATCH 2/2] lib: utils/timer: Add a separate compatible for
 the D1 CLINT
In-Reply-To: <CAAhSdy0+2GY1U==iGHn5udFwSWUTEEKbtG=fSxu2vC4As1E2gg@mail.gmail.com>
References: <20211020015838.45399-1-samuel@sholland.org>
 <20211020015838.45399-3-samuel@sholland.org>
 <CAAhSdy3r20Q4mBkTR58Befo+SYhsLx3ueq=MOb4UQXk3P=8F-w@mail.gmail.com>
 <d57d4cb7-bf64-c1c8-a0bf-174e36207d3c@sholland.org>
 <CAAhSdy0+2GY1U==iGHn5udFwSWUTEEKbtG=fSxu2vC4As1E2gg@mail.gmail.com>
Message-ID: <d9249dd3-ae0d-d2ff-f785-d2a1df2e6d45@sholland.org>

On 6/20/22 11:54 PM, Anup Patel wrote:
> On Tue, Jun 21, 2022 at 10:13 AM Samuel Holland <samuel at sholland.org> wrote:
>>
>> On 6/20/22 11:20 PM, Anup Patel wrote:
>>> On Wed, Oct 20, 2021 at 7:28 AM Samuel Holland <samuel at sholland.org> wrote:
>>>>
>>>> The CLINT in the Allwinner D1 SoC apparently does not support 64-bit
>>>> MMIO access. A property was added to support this quirk (and that
>>>> property was copied to the ACLINT MTIMER code). However, since this
>>>> difference in behavior makes the D1 CLINT incompatible with the SiFive
>>>> CLINT's programming interface, a better solution is to use a separate
>>>> compatible string.
>>>>
>>>> Signed-off-by: Samuel Holland <samuel at sholland.org>
>>>
>>> Due to some unknown reason, this patch slipped through cracks and was
>>> never applied.
>>
>> Oh, I intentionally did not send a new version of the series because I later
>> found out that C9xx CLINT has no mtime register at all[1][2]. The issue has
>> nothing to do with 64-bit vs 32-bit access. So this change is wrong; I will need
>> to send a follow-up removing the allwinner,sun20i-d1-clint compatibles.
> 
> I had tested on QEMU virt and sifive_u machine so at least this does not
> break anything on QEMU and SiFive boards.

OK, I will send a follow-up patch tomorrow.

> Just a heads-up, we are planning to do OpenSBI v1.1 release sometime
> this week.
> 
> Do you have a list of things that are still missing to support upstream
> OpenSBI on D1 board ?

D1 already has enough support to boot/run with v1.0.

The only remaining functionality I know of is:
 - CLINT, which needs a DT binding plus further driver changes, and
 - PMU, which will be nontrivial to implement.

Lack of CLINT driver is no real problem, because D1 has several other MMIO
timers available, so the SBI timer extension is not even used right now.

And the generic PMU code gracefully handles the lack of overflow interrupt
support, so that can also be added later.

Regards,
Samuel


From anup at brainfault.org  Mon Jun 20 22:43:00 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 11:13:00 +0530
Subject: Upcoming OpenSBI v1.1 release
Message-ID: <CAAhSdy29-mCr6g+x3V_HMrsqwwP-9-5m1TW-4VGDd20aWh1NFQ@mail.gmail.com>

Hi All,

We are planning to release OpenSBI v1.1 this week. Most likely Friday
(24th June 2022) or Saturday (25th June 2022).

It would be very helpful if you can try the latest OpenSBI on your
RISC-V board and see if there are no obvious issues.

Thanks and Regards,
Anup


From guoren at kernel.org  Tue Jun 21 01:49:16 2022
From: guoren at kernel.org (Guo Ren)
Date: Tue, 21 Jun 2022 16:49:16 +0800
Subject: [RFC PATCH 2/2] lib: utils/timer: Add a separate compatible for
 the D1 CLINT
In-Reply-To: <d9249dd3-ae0d-d2ff-f785-d2a1df2e6d45@sholland.org>
References: <20211020015838.45399-1-samuel@sholland.org> <20211020015838.45399-3-samuel@sholland.org>
 <CAAhSdy3r20Q4mBkTR58Befo+SYhsLx3ueq=MOb4UQXk3P=8F-w@mail.gmail.com>
 <d57d4cb7-bf64-c1c8-a0bf-174e36207d3c@sholland.org> <CAAhSdy0+2GY1U==iGHn5udFwSWUTEEKbtG=fSxu2vC4As1E2gg@mail.gmail.com>
 <d9249dd3-ae0d-d2ff-f785-d2a1df2e6d45@sholland.org>
Message-ID: <CAJF2gTQQ=eZqqyHGjmrZd3JwAiU9JR6xZTid4mtDN-6pHS4wrQ@mail.gmail.com>

On Tue, Jun 21, 2022 at 1:20 PM Samuel Holland <samuel at sholland.org> wrote:
>
> On 6/20/22 11:54 PM, Anup Patel wrote:
> > On Tue, Jun 21, 2022 at 10:13 AM Samuel Holland <samuel at sholland.org> wrote:
> >>
> >> On 6/20/22 11:20 PM, Anup Patel wrote:
> >>> On Wed, Oct 20, 2021 at 7:28 AM Samuel Holland <samuel at sholland.org> wrote:
> >>>>
> >>>> The CLINT in the Allwinner D1 SoC apparently does not support 64-bit
> >>>> MMIO access. A property was added to support this quirk (and that
> >>>> property was copied to the ACLINT MTIMER code). However, since this
> >>>> difference in behavior makes the D1 CLINT incompatible with the SiFive
> >>>> CLINT's programming interface, a better solution is to use a separate
> >>>> compatible string.
> >>>>
> >>>> Signed-off-by: Samuel Holland <samuel at sholland.org>
> >>>
> >>> Due to some unknown reason, this patch slipped through cracks and was
> >>> never applied.
> >>
> >> Oh, I intentionally did not send a new version of the series because I later
> >> found out that C9xx CLINT has no mtime register at all[1][2]. The issue has
> >> nothing to do with 64-bit vs 32-bit access. So this change is wrong; I will need
> >> to send a follow-up removing the allwinner,sun20i-d1-clint compatibles.
> >
> > I had tested on QEMU virt and sifive_u machine so at least this does not
> > break anything on QEMU and SiFive boards.
>
> OK, I will send a follow-up patch tomorrow.
>
> > Just a heads-up, we are planning to do OpenSBI v1.1 release sometime
> > this week.
> >
> > Do you have a list of things that are still missing to support upstream
> > OpenSBI on D1 board ?
>
> D1 already has enough support to boot/run with v1.0.
>
> The only remaining functionality I know of is:
>  - CLINT, which needs a DT binding plus further driver changes, and
>  - PMU, which will be nontrivial to implement.
>
> Lack of CLINT driver is no real problem, because D1 has several other MMIO
> timers available, so the SBI timer extension is not even used right now.
Another solution is modifying [1]. Using get_cycles64() in clint_get_cycles64.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/clocksource/timer-clint.c

C906 has s-mode time cmp register:
parameter STIMECMP0 = 16'hD000;
parameter STIMECMPH0 = 16'hD004;

>
> And the generic PMU code gracefully handles the lack of overflow interrupt
> support, so that can also be added later.
>
> Regards,
> Samuel



-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/


From jan.remes at codasip.com  Tue Jun 21 05:07:39 2022
From: jan.remes at codasip.com (Jan Remes)
Date: Tue, 21 Jun 2022 14:07:39 +0200
Subject: [PATCH] lib: irqchip/plic: fix typo in plic_warm_irqchip_init
Message-ID: <20220621120739.422990-1-jan.remes@codasip.com>

The second invocation of plic_context_init() incorrectly calls the
function with m_cntx_id instead of s_cntx_id. This breaks systems which
only have 1 external interrupt per hart.

Fixes: 8c362e7 ("lib: irqchip/plic: Factor out a context init function")
Signed-off-by: Jan Remes <jan.remes at codasip.com>
---
 lib/utils/irqchip/plic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
index 8c25b9d..73d7788 100644
--- a/lib/utils/irqchip/plic.c
+++ b/lib/utils/irqchip/plic.c
@@ -145,7 +145,7 @@ int plic_warm_irqchip_init(const struct plic_data *plic,
 
 	/* By default, disable all IRQs for S-mode of target HART */
 	if (s_cntx_id > -1) {
-		ret = plic_context_init(plic, m_cntx_id, false, 0x7);
+		ret = plic_context_init(plic, s_cntx_id, false, 0x7);
 		if (ret)
 			return ret;
 	}
-- 
2.36.1



From apatel at ventanamicro.com  Tue Jun 21 06:11:32 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Tue, 21 Jun 2022 18:41:32 +0530
Subject: [PATCH] lib: utils/mtimer: Add has_broken_mtime quirk
Message-ID: <20220621131132.110311-1-apatel@ventanamicro.com>

The Allwinner D1 SoC has custom CLINT implementation where the
MTIME register of the timer is not accessible as MMIO register
instead the time CSR is to be used as MTIME.

We introduce "has_broken_mtime" quirk in MTIMER library and FDT
based MTIMER driver to address the Allwiner D1 situation. And
while we are here, let us extend the quirk mechanism in FDT based
MTIMER driver for both CLINT and ACLINT.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 include/sbi_utils/timer/aclint_mtimer.h |  1 +
 lib/utils/timer/aclint_mtimer.c         | 30 +++++++++++++++++++------
 lib/utils/timer/fdt_timer_mtimer.c      | 29 ++++++++++++++++++------
 3 files changed, 46 insertions(+), 14 deletions(-)

diff --git a/include/sbi_utils/timer/aclint_mtimer.h b/include/sbi_utils/timer/aclint_mtimer.h
index f02cc62..454a92f 100644
--- a/include/sbi_utils/timer/aclint_mtimer.h
+++ b/include/sbi_utils/timer/aclint_mtimer.h
@@ -32,6 +32,7 @@ struct aclint_mtimer_data {
 	u32 first_hartid;
 	u32 hart_count;
 	bool has_64bit_mmio;
+	bool has_broken_mtime;
 	bool has_shared_mtime;
 	/* Private details (initialized and used by ACLINT MTIMER library) */
 	struct aclint_mtimer_data *time_delta_reference;
diff --git a/lib/utils/timer/aclint_mtimer.c b/lib/utils/timer/aclint_mtimer.c
index a957b1c..5db3120 100644
--- a/lib/utils/timer/aclint_mtimer.c
+++ b/lib/utils/timer/aclint_mtimer.c
@@ -60,6 +60,12 @@ static u64 mtimer_value(void)
 	return mt->time_rd(time_val);
 }
 
+static u64 mtimer_dummy_value(void)
+{
+	/* Dummy function for MTIMER devices with broken MTIME */
+	return 0;
+}
+
 static void mtimer_event_stop(void)
 {
 	u32 target_hart = current_hartid();
@@ -95,7 +101,9 @@ void aclint_mtimer_sync(struct aclint_mtimer_data *mt)
 	struct aclint_mtimer_data *reference;
 
 	/* Sync-up non-shared MTIME if reference is available */
-	if (mt->has_shared_mtime || !mt->time_delta_reference)
+	if (mt->has_broken_mtime ||
+	    mt->has_shared_mtime ||
+	    !mt->time_delta_reference)
 		return;
 
 	reference = mt->time_delta_reference;
@@ -189,6 +197,10 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
 	if (reference && mt->mtime_freq != reference->mtime_freq)
 		return SBI_EINVAL;
 
+	/* Override timer_value() callback for broken MTIME */
+	if (mt->has_broken_mtime)
+		mtimer.timer_value = mtimer_dummy_value;
+
 	/* Initialize private data */
 	aclint_mtimer_set_reference(mt, reference);
 	mt->time_rd = mtimer_time_rd32;
@@ -207,21 +219,25 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
 		mtimer_hartid2data[mt->first_hartid + i] = mt;
 
 	/* Add MTIMER regions to the root domain */
-	if (mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
+	if (!mt->has_broken_mtime &&
+	    mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
 		rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
 					mt->mtime_size + mt->mtimecmp_size);
 		if (rc)
 			return rc;
-	} else if (mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
+	} else if (!mt->has_broken_mtime &&
+		   mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
 		rc = aclint_mtimer_add_regions(mt->mtime_addr,
 					mt->mtime_size + mt->mtimecmp_size);
 		if (rc)
 			return rc;
 	} else {
-		rc = aclint_mtimer_add_regions(mt->mtime_addr,
-						mt->mtime_size);
-		if (rc)
-			return rc;
+		if (!mt->has_broken_mtime) {
+			rc = aclint_mtimer_add_regions(mt->mtime_addr,
+							mt->mtime_size);
+			if (rc)
+				return rc;
+		}
 
 		rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
 						mt->mtimecmp_size);
diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
index e140567..bc547ac 100644
--- a/lib/utils/timer/fdt_timer_mtimer.c
+++ b/lib/utils/timer/fdt_timer_mtimer.c
@@ -16,8 +16,10 @@
 #define MTIMER_MAX_NR			16
 
 struct timer_mtimer_quirks {
+	bool		is_clint;
 	unsigned int	mtime_offset;
 	bool		has_64bit_mmio;
+	bool		has_broken_mtime;
 };
 
 static unsigned long mtimer_count = 0;
@@ -30,6 +32,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 	int i, rc;
 	unsigned long addr[2], size[2];
 	struct aclint_mtimer_data *mt;
+	const struct timer_mtimer_quirks *quirks;
+
+	if (!match->data)
+		return SBI_ENOSYS;
+	quirks = match->data;
 
 	if (MTIMER_MAX_NR <= mtimer_count)
 		return SBI_ENOSPC;
@@ -40,16 +47,15 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 				   &mt->first_hartid, &mt->hart_count);
 	if (rc)
 		return rc;
-	mt->has_64bit_mmio = true;
+	mt->has_64bit_mmio = quirks->has_64bit_mmio;
 	mt->has_shared_mtime = false;
+	mt->has_broken_mtime = quirks->has_broken_mtime;
 
 	rc = fdt_parse_timebase_frequency(fdt, &mt->mtime_freq);
 	if (rc)
 		return rc;
 
-	if (match->data) { /* SiFive CLINT */
-		const struct timer_mtimer_quirks *quirks = match->data;
-
+	if (quirks->is_clint) { /* SiFive CLINT */
 		/* Set CLINT addresses */
 		mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
 		mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
@@ -59,8 +65,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 		mt->mtime_addr += quirks->mtime_offset;
 		mt->mtimecmp_addr += quirks->mtime_offset;
 		mt->mtime_size -= quirks->mtime_offset;
-		/* Apply additional CLINT quirks */
-		mt->has_64bit_mmio = quirks->has_64bit_mmio;
 	} else { /* RISC-V ACLINT MTIMER */
 		/* Set ACLINT MTIMER addresses */
 		mt->mtime_addr = addr[0];
@@ -110,20 +114,31 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 }
 
 static const struct timer_mtimer_quirks d1_clint_quirks = {
+	.is_clint	= true,
 	.mtime_offset	= CLINT_MTIMER_OFFSET,
 	.has_64bit_mmio	= false,
+	.has_broken_mtime = true,
 };
 
 static const struct timer_mtimer_quirks sifive_clint_quirks = {
+	.is_clint	= true,
 	.mtime_offset	= CLINT_MTIMER_OFFSET,
 	.has_64bit_mmio	= true,
+	.has_broken_mtime = false,
+};
+
+static const struct timer_mtimer_quirks aclint_quirks = {
+	.is_clint	= false,
+	.mtime_offset	= 0,
+	.has_64bit_mmio	= true,
+	.has_broken_mtime = false,
 };
 
 static const struct fdt_match timer_mtimer_match[] = {
 	{ .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
 	{ .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
 	{ .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
-	{ .compatible = "riscv,aclint-mtimer" },
+	{ .compatible = "riscv,aclint-mtimer", .data = &aclint_quirks },
 	{ },
 };
 
-- 
2.34.1



From anup at brainfault.org  Tue Jun 21 06:29:53 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 18:59:53 +0530
Subject: [RFC PATCH 2/2] lib: utils/timer: Add a separate compatible for
 the D1 CLINT
In-Reply-To: <d9249dd3-ae0d-d2ff-f785-d2a1df2e6d45@sholland.org>
References: <20211020015838.45399-1-samuel@sholland.org> <20211020015838.45399-3-samuel@sholland.org>
 <CAAhSdy3r20Q4mBkTR58Befo+SYhsLx3ueq=MOb4UQXk3P=8F-w@mail.gmail.com>
 <d57d4cb7-bf64-c1c8-a0bf-174e36207d3c@sholland.org> <CAAhSdy0+2GY1U==iGHn5udFwSWUTEEKbtG=fSxu2vC4As1E2gg@mail.gmail.com>
 <d9249dd3-ae0d-d2ff-f785-d2a1df2e6d45@sholland.org>
Message-ID: <CAAhSdy3PJuAj-jY5ifJ_pRZXsH7zx07JMuBVCHBjoUUu=Tf=ZA@mail.gmail.com>

On Tue, Jun 21, 2022 at 10:50 AM Samuel Holland <samuel at sholland.org> wrote:
>
> On 6/20/22 11:54 PM, Anup Patel wrote:
> > On Tue, Jun 21, 2022 at 10:13 AM Samuel Holland <samuel at sholland.org> wrote:
> >>
> >> On 6/20/22 11:20 PM, Anup Patel wrote:
> >>> On Wed, Oct 20, 2021 at 7:28 AM Samuel Holland <samuel at sholland.org> wrote:
> >>>>
> >>>> The CLINT in the Allwinner D1 SoC apparently does not support 64-bit
> >>>> MMIO access. A property was added to support this quirk (and that
> >>>> property was copied to the ACLINT MTIMER code). However, since this
> >>>> difference in behavior makes the D1 CLINT incompatible with the SiFive
> >>>> CLINT's programming interface, a better solution is to use a separate
> >>>> compatible string.
> >>>>
> >>>> Signed-off-by: Samuel Holland <samuel at sholland.org>
> >>>
> >>> Due to some unknown reason, this patch slipped through cracks and was
> >>> never applied.
> >>
> >> Oh, I intentionally did not send a new version of the series because I later
> >> found out that C9xx CLINT has no mtime register at all[1][2]. The issue has
> >> nothing to do with 64-bit vs 32-bit access. So this change is wrong; I will need
> >> to send a follow-up removing the allwinner,sun20i-d1-clint compatibles.
> >
> > I had tested on QEMU virt and sifive_u machine so at least this does not
> > break anything on QEMU and SiFive boards.
>
> OK, I will send a follow-up patch tomorrow.

Thanks. I was thinking of having a quirk in the mtimer driver for the D1 clint
situation. I sent a quirk patch but I have tested only on QEMU RV32 and RV64.

>
> > Just a heads-up, we are planning to do OpenSBI v1.1 release sometime
> > this week.
> >
> > Do you have a list of things that are still missing to support upstream
> > OpenSBI on D1 board ?
>
> D1 already has enough support to boot/run with v1.0.
>
> The only remaining functionality I know of is:
>  - CLINT, which needs a DT binding plus further driver changes, and
>  - PMU, which will be nontrivial to implement.
>
> Lack of CLINT driver is no real problem, because D1 has several other MMIO
> timers available, so the SBI timer extension is not even used right now.
>
> And the generic PMU code gracefully handles the lack of overflow interrupt
> support, so that can also be added later.
>
> Regards,
> Samuel

Regards,
Anup


From guoren at kernel.org  Tue Jun 21 07:03:43 2022
From: guoren at kernel.org (Guo Ren)
Date: Tue, 21 Jun 2022 22:03:43 +0800
Subject: [PATCH] lib: utils/mtimer: Add has_broken_mtime quirk
In-Reply-To: <20220621131132.110311-1-apatel@ventanamicro.com>
References: <20220621131132.110311-1-apatel@ventanamicro.com>
Message-ID: <CAJF2gTTtjON3yUZ0btg1OuVVOZorhgwKrL4GrwViP6htOYxNgQ@mail.gmail.com>

On Tue, Jun 21, 2022 at 9:11 PM Anup Patel <apatel at ventanamicro.com> wrote:
>
> The Allwinner D1 SoC has custom CLINT implementation where the
> MTIME register of the timer is not accessible as MMIO register
> instead the time CSR is to be used as MTIME.
>
> We introduce "has_broken_mtime" quirk in MTIMER library and FDT
> based MTIMER driver to address the Allwiner D1 situation. And
> while we are here, let us extend the quirk mechanism in FDT based
> MTIMER driver for both CLINT and ACLINT.
>
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> ---
>  include/sbi_utils/timer/aclint_mtimer.h |  1 +
>  lib/utils/timer/aclint_mtimer.c         | 30 +++++++++++++++++++------
>  lib/utils/timer/fdt_timer_mtimer.c      | 29 ++++++++++++++++++------
>  3 files changed, 46 insertions(+), 14 deletions(-)
>
> diff --git a/include/sbi_utils/timer/aclint_mtimer.h b/include/sbi_utils/timer/aclint_mtimer.h
> index f02cc62..454a92f 100644
> --- a/include/sbi_utils/timer/aclint_mtimer.h
> +++ b/include/sbi_utils/timer/aclint_mtimer.h
> @@ -32,6 +32,7 @@ struct aclint_mtimer_data {
>         u32 first_hartid;
>         u32 hart_count;
>         bool has_64bit_mmio;
> +       bool has_broken_mtime;
>         bool has_shared_mtime;
>         /* Private details (initialized and used by ACLINT MTIMER library) */
>         struct aclint_mtimer_data *time_delta_reference;
> diff --git a/lib/utils/timer/aclint_mtimer.c b/lib/utils/timer/aclint_mtimer.c
> index a957b1c..5db3120 100644
> --- a/lib/utils/timer/aclint_mtimer.c
> +++ b/lib/utils/timer/aclint_mtimer.c
> @@ -60,6 +60,12 @@ static u64 mtimer_value(void)
>         return mt->time_rd(time_val);
>  }
>
> +static u64 mtimer_dummy_value(void)
> +{
> +       /* Dummy function for MTIMER devices with broken MTIME */
Could we get value from CSR_TIME? (define CSR_TIME           0xc01)

> +       return 0;
> +}
> +
>  static void mtimer_event_stop(void)
>  {
>         u32 target_hart = current_hartid();
> @@ -95,7 +101,9 @@ void aclint_mtimer_sync(struct aclint_mtimer_data *mt)
>         struct aclint_mtimer_data *reference;
>
>         /* Sync-up non-shared MTIME if reference is available */
> -       if (mt->has_shared_mtime || !mt->time_delta_reference)
> +       if (mt->has_broken_mtime ||
> +           mt->has_shared_mtime ||
> +           !mt->time_delta_reference)
>                 return;
>
>         reference = mt->time_delta_reference;
> @@ -189,6 +197,10 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
>         if (reference && mt->mtime_freq != reference->mtime_freq)
>                 return SBI_EINVAL;
>
> +       /* Override timer_value() callback for broken MTIME */
> +       if (mt->has_broken_mtime)
> +               mtimer.timer_value = mtimer_dummy_value;
> +
>         /* Initialize private data */
>         aclint_mtimer_set_reference(mt, reference);
>         mt->time_rd = mtimer_time_rd32;
> @@ -207,21 +219,25 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
>                 mtimer_hartid2data[mt->first_hartid + i] = mt;
>
>         /* Add MTIMER regions to the root domain */
> -       if (mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
> +       if (!mt->has_broken_mtime &&
> +           mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
>                 rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
>                                         mt->mtime_size + mt->mtimecmp_size);
>                 if (rc)
>                         return rc;
> -       } else if (mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
> +       } else if (!mt->has_broken_mtime &&
> +                  mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
>                 rc = aclint_mtimer_add_regions(mt->mtime_addr,
>                                         mt->mtime_size + mt->mtimecmp_size);
>                 if (rc)
>                         return rc;
>         } else {
> -               rc = aclint_mtimer_add_regions(mt->mtime_addr,
> -                                               mt->mtime_size);
> -               if (rc)
> -                       return rc;
> +               if (!mt->has_broken_mtime) {
> +                       rc = aclint_mtimer_add_regions(mt->mtime_addr,
> +                                                       mt->mtime_size);
> +                       if (rc)
> +                               return rc;
> +               }
>
>                 rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
>                                                 mt->mtimecmp_size);
> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> index e140567..bc547ac 100644
> --- a/lib/utils/timer/fdt_timer_mtimer.c
> +++ b/lib/utils/timer/fdt_timer_mtimer.c
> @@ -16,8 +16,10 @@
>  #define MTIMER_MAX_NR                  16
>
>  struct timer_mtimer_quirks {
> +       bool            is_clint;
>         unsigned int    mtime_offset;
>         bool            has_64bit_mmio;
> +       bool            has_broken_mtime;
>  };
>
>  static unsigned long mtimer_count = 0;
> @@ -30,6 +32,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>         int i, rc;
>         unsigned long addr[2], size[2];
>         struct aclint_mtimer_data *mt;
> +       const struct timer_mtimer_quirks *quirks;
> +
> +       if (!match->data)
> +               return SBI_ENOSYS;
> +       quirks = match->data;
>
>         if (MTIMER_MAX_NR <= mtimer_count)
>                 return SBI_ENOSPC;
> @@ -40,16 +47,15 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                                    &mt->first_hartid, &mt->hart_count);
>         if (rc)
>                 return rc;
> -       mt->has_64bit_mmio = true;
> +       mt->has_64bit_mmio = quirks->has_64bit_mmio;
>         mt->has_shared_mtime = false;
> +       mt->has_broken_mtime = quirks->has_broken_mtime;
>
>         rc = fdt_parse_timebase_frequency(fdt, &mt->mtime_freq);
>         if (rc)
>                 return rc;
>
> -       if (match->data) { /* SiFive CLINT */
> -               const struct timer_mtimer_quirks *quirks = match->data;
> -
> +       if (quirks->is_clint) { /* SiFive CLINT */
>                 /* Set CLINT addresses */
>                 mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
>                 mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
> @@ -59,8 +65,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>                 mt->mtime_addr += quirks->mtime_offset;
>                 mt->mtimecmp_addr += quirks->mtime_offset;
>                 mt->mtime_size -= quirks->mtime_offset;
> -               /* Apply additional CLINT quirks */
> -               mt->has_64bit_mmio = quirks->has_64bit_mmio;
>         } else { /* RISC-V ACLINT MTIMER */
>                 /* Set ACLINT MTIMER addresses */
>                 mt->mtime_addr = addr[0];
> @@ -110,20 +114,31 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>  }
>
>  static const struct timer_mtimer_quirks d1_clint_quirks = {
> +       .is_clint       = true,
>         .mtime_offset   = CLINT_MTIMER_OFFSET,
>         .has_64bit_mmio = false,
> +       .has_broken_mtime = true,
>  };
>
>  static const struct timer_mtimer_quirks sifive_clint_quirks = {
> +       .is_clint       = true,
>         .mtime_offset   = CLINT_MTIMER_OFFSET,
>         .has_64bit_mmio = true,
> +       .has_broken_mtime = false,
> +};
> +
> +static const struct timer_mtimer_quirks aclint_quirks = {
> +       .is_clint       = false,
> +       .mtime_offset   = 0,
> +       .has_64bit_mmio = true,
> +       .has_broken_mtime = false,
>  };
>
>  static const struct fdt_match timer_mtimer_match[] = {
>         { .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
>         { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
>         { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
> -       { .compatible = "riscv,aclint-mtimer" },
> +       { .compatible = "riscv,aclint-mtimer", .data = &aclint_quirks },
>         { },
>  };
>
> --
> 2.34.1
>


--
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/


From anup at brainfault.org  Tue Jun 21 08:04:13 2022
From: anup at brainfault.org (Anup Patel)
Date: Tue, 21 Jun 2022 20:34:13 +0530
Subject: [PATCH] lib: utils/mtimer: Add has_broken_mtime quirk
In-Reply-To: <CAJF2gTTtjON3yUZ0btg1OuVVOZorhgwKrL4GrwViP6htOYxNgQ@mail.gmail.com>
References: <20220621131132.110311-1-apatel@ventanamicro.com> <CAJF2gTTtjON3yUZ0btg1OuVVOZorhgwKrL4GrwViP6htOYxNgQ@mail.gmail.com>
Message-ID: <CAAhSdy2ugpwy8Dc++68CTEXsVMHLyxfrUE817cUyX5q7UP_wCQ@mail.gmail.com>

On Tue, Jun 21, 2022 at 7:33 PM Guo Ren <guoren at kernel.org> wrote:
>
> On Tue, Jun 21, 2022 at 9:11 PM Anup Patel <apatel at ventanamicro.com> wrote:
> >
> > The Allwinner D1 SoC has custom CLINT implementation where the
> > MTIME register of the timer is not accessible as MMIO register
> > instead the time CSR is to be used as MTIME.
> >
> > We introduce "has_broken_mtime" quirk in MTIMER library and FDT
> > based MTIMER driver to address the Allwiner D1 situation. And
> > while we are here, let us extend the quirk mechanism in FDT based
> > MTIMER driver for both CLINT and ACLINT.
> >
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > ---
> >  include/sbi_utils/timer/aclint_mtimer.h |  1 +
> >  lib/utils/timer/aclint_mtimer.c         | 30 +++++++++++++++++++------
> >  lib/utils/timer/fdt_timer_mtimer.c      | 29 ++++++++++++++++++------
> >  3 files changed, 46 insertions(+), 14 deletions(-)
> >
> > diff --git a/include/sbi_utils/timer/aclint_mtimer.h b/include/sbi_utils/timer/aclint_mtimer.h
> > index f02cc62..454a92f 100644
> > --- a/include/sbi_utils/timer/aclint_mtimer.h
> > +++ b/include/sbi_utils/timer/aclint_mtimer.h
> > @@ -32,6 +32,7 @@ struct aclint_mtimer_data {
> >         u32 first_hartid;
> >         u32 hart_count;
> >         bool has_64bit_mmio;
> > +       bool has_broken_mtime;
> >         bool has_shared_mtime;
> >         /* Private details (initialized and used by ACLINT MTIMER library) */
> >         struct aclint_mtimer_data *time_delta_reference;
> > diff --git a/lib/utils/timer/aclint_mtimer.c b/lib/utils/timer/aclint_mtimer.c
> > index a957b1c..5db3120 100644
> > --- a/lib/utils/timer/aclint_mtimer.c
> > +++ b/lib/utils/timer/aclint_mtimer.c
> > @@ -60,6 +60,12 @@ static u64 mtimer_value(void)
> >         return mt->time_rd(time_val);
> >  }
> >
> > +static u64 mtimer_dummy_value(void)
> > +{
> > +       /* Dummy function for MTIMER devices with broken MTIME */
> Could we get value from CSR_TIME? (define CSR_TIME           0xc01)

The lib/sbi/sbi_timer.c will use CSR_TIME whenever it is available so
we don't need to access it here.

Maybe we can drop mtimer_dummy_value() and set
"mtimer.timer_value = NULL" when "has_broken_mtime = TRUE" ?

Can you try this patch on D1 ?

Regards,
Anup

>
> > +       return 0;
> > +}
> > +
> >  static void mtimer_event_stop(void)
> >  {
> >         u32 target_hart = current_hartid();
> > @@ -95,7 +101,9 @@ void aclint_mtimer_sync(struct aclint_mtimer_data *mt)
> >         struct aclint_mtimer_data *reference;
> >
> >         /* Sync-up non-shared MTIME if reference is available */
> > -       if (mt->has_shared_mtime || !mt->time_delta_reference)
> > +       if (mt->has_broken_mtime ||
> > +           mt->has_shared_mtime ||
> > +           !mt->time_delta_reference)
> >                 return;
> >
> >         reference = mt->time_delta_reference;
> > @@ -189,6 +197,10 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
> >         if (reference && mt->mtime_freq != reference->mtime_freq)
> >                 return SBI_EINVAL;
> >
> > +       /* Override timer_value() callback for broken MTIME */
> > +       if (mt->has_broken_mtime)
> > +               mtimer.timer_value = mtimer_dummy_value;
> > +
> >         /* Initialize private data */
> >         aclint_mtimer_set_reference(mt, reference);
> >         mt->time_rd = mtimer_time_rd32;
> > @@ -207,21 +219,25 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
> >                 mtimer_hartid2data[mt->first_hartid + i] = mt;
> >
> >         /* Add MTIMER regions to the root domain */
> > -       if (mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
> > +       if (!mt->has_broken_mtime &&
> > +           mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
> >                 rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
> >                                         mt->mtime_size + mt->mtimecmp_size);
> >                 if (rc)
> >                         return rc;
> > -       } else if (mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
> > +       } else if (!mt->has_broken_mtime &&
> > +                  mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
> >                 rc = aclint_mtimer_add_regions(mt->mtime_addr,
> >                                         mt->mtime_size + mt->mtimecmp_size);
> >                 if (rc)
> >                         return rc;
> >         } else {
> > -               rc = aclint_mtimer_add_regions(mt->mtime_addr,
> > -                                               mt->mtime_size);
> > -               if (rc)
> > -                       return rc;
> > +               if (!mt->has_broken_mtime) {
> > +                       rc = aclint_mtimer_add_regions(mt->mtime_addr,
> > +                                                       mt->mtime_size);
> > +                       if (rc)
> > +                               return rc;
> > +               }
> >
> >                 rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
> >                                                 mt->mtimecmp_size);
> > diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> > index e140567..bc547ac 100644
> > --- a/lib/utils/timer/fdt_timer_mtimer.c
> > +++ b/lib/utils/timer/fdt_timer_mtimer.c
> > @@ -16,8 +16,10 @@
> >  #define MTIMER_MAX_NR                  16
> >
> >  struct timer_mtimer_quirks {
> > +       bool            is_clint;
> >         unsigned int    mtime_offset;
> >         bool            has_64bit_mmio;
> > +       bool            has_broken_mtime;
> >  };
> >
> >  static unsigned long mtimer_count = 0;
> > @@ -30,6 +32,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> >         int i, rc;
> >         unsigned long addr[2], size[2];
> >         struct aclint_mtimer_data *mt;
> > +       const struct timer_mtimer_quirks *quirks;
> > +
> > +       if (!match->data)
> > +               return SBI_ENOSYS;
> > +       quirks = match->data;
> >
> >         if (MTIMER_MAX_NR <= mtimer_count)
> >                 return SBI_ENOSPC;
> > @@ -40,16 +47,15 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> >                                    &mt->first_hartid, &mt->hart_count);
> >         if (rc)
> >                 return rc;
> > -       mt->has_64bit_mmio = true;
> > +       mt->has_64bit_mmio = quirks->has_64bit_mmio;
> >         mt->has_shared_mtime = false;
> > +       mt->has_broken_mtime = quirks->has_broken_mtime;
> >
> >         rc = fdt_parse_timebase_frequency(fdt, &mt->mtime_freq);
> >         if (rc)
> >                 return rc;
> >
> > -       if (match->data) { /* SiFive CLINT */
> > -               const struct timer_mtimer_quirks *quirks = match->data;
> > -
> > +       if (quirks->is_clint) { /* SiFive CLINT */
> >                 /* Set CLINT addresses */
> >                 mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
> >                 mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
> > @@ -59,8 +65,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> >                 mt->mtime_addr += quirks->mtime_offset;
> >                 mt->mtimecmp_addr += quirks->mtime_offset;
> >                 mt->mtime_size -= quirks->mtime_offset;
> > -               /* Apply additional CLINT quirks */
> > -               mt->has_64bit_mmio = quirks->has_64bit_mmio;
> >         } else { /* RISC-V ACLINT MTIMER */
> >                 /* Set ACLINT MTIMER addresses */
> >                 mt->mtime_addr = addr[0];
> > @@ -110,20 +114,31 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> >  }
> >
> >  static const struct timer_mtimer_quirks d1_clint_quirks = {
> > +       .is_clint       = true,
> >         .mtime_offset   = CLINT_MTIMER_OFFSET,
> >         .has_64bit_mmio = false,
> > +       .has_broken_mtime = true,
> >  };
> >
> >  static const struct timer_mtimer_quirks sifive_clint_quirks = {
> > +       .is_clint       = true,
> >         .mtime_offset   = CLINT_MTIMER_OFFSET,
> >         .has_64bit_mmio = true,
> > +       .has_broken_mtime = false,
> > +};
> > +
> > +static const struct timer_mtimer_quirks aclint_quirks = {
> > +       .is_clint       = false,
> > +       .mtime_offset   = 0,
> > +       .has_64bit_mmio = true,
> > +       .has_broken_mtime = false,
> >  };
> >
> >  static const struct fdt_match timer_mtimer_match[] = {
> >         { .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
> >         { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
> >         { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
> > -       { .compatible = "riscv,aclint-mtimer" },
> > +       { .compatible = "riscv,aclint-mtimer", .data = &aclint_quirks },
> >         { },
> >  };
> >
> > --
> > 2.34.1
> >
>
>
> --
> Best Regards
>  Guo Ren
>
> ML: https://lore.kernel.org/linux-csky/


From wxjstz at 126.com  Tue Jun 21 08:53:16 2022
From: wxjstz at 126.com (Xiang W)
Date: Tue, 21 Jun 2022 23:53:16 +0800
Subject: [PATCH v4] lib: sbi: Improve csr read and write
In-Reply-To: <81DE08C6-50D4-4CC2-AE63-266D727F083F@jrtc27.com>
References: <20220526144214.308921-1-wxjstz@126.com>
	 <CAK9=C2U9KyVTJhXN9jOqqR7JMYz0Mn9ar=seXbPg1+UdHhhbpg@mail.gmail.com>
	 <1e615125574b838bc99dd58239874fa2e1685bdb.camel@126.com>
	 <81DE08C6-50D4-4CC2-AE63-266D727F083F@jrtc27.com>
Message-ID: <eb75c146d4657e71ca67dce5517438e80f4af962.camel@126.com>

? 2022-05-26???? 16:28 +0100?Jessica Clarke???
> On 26 May 2022, at 16:19, Xiang W <wxjstz at 126.com> wrote:
> > 
> > ? 2022-05-26???? 20:35 +0530?Anup Patel???
> > > On Thu, May 26, 2022 at 8:12 PM Xiang W <wxjstz at 126.com> wrote:
> > > > 
> > > > Reduce the overhead of csr read and write code by producing a small
> > > > piece of code to perform csr read and write.
> > > > 
> > > > Signed-off-by: Xiang W <wxjstz at 126.com>
> > > > ---
> > > > Changes in v4:
> > > > - add fence.i sync instruction memory
> > > > 
> > > > Changes in v3:
> > > > - Prevent unnecessary optimizations by the compiler
> > > > 
> > > > Changes in v2:
> > > > - Fix thread safety related bugs as suggested by Anup
> > > > 
> > > > lib/sbi/riscv_asm.c | 145 ++++----------------------------------------
> > > > 1 file changed, 13 insertions(+), 132 deletions(-)
> > > > 
> > > > diff --git a/lib/sbi/riscv_asm.c b/lib/sbi/riscv_asm.c
> > > > index a09cf78..452cb19 100644
> > > > --- a/lib/sbi/riscv_asm.c
> > > > +++ b/lib/sbi/riscv_asm.c
> > > > @@ -9,6 +9,7 @@
> > > > 
> > > > #include <sbi/riscv_asm.h>
> > > > #include <sbi/riscv_encoding.h>
> > > > +#include <sbi/riscv_barrier.h>
> > > > #include <sbi/sbi_error.h>
> > > > #include <sbi/sbi_platform.h>
> > > > #include <sbi/sbi_console.h>
> > > > @@ -93,142 +94,22 @@ void misa_string(int xlen, char *out, unsigned int out_sz)
> > > > 
> > > > unsigned long csr_read_num(int csr_num)
> > > > {
> > > > -#define switchcase_csr_read(__csr_num, __val)? \
> > > > -? case __csr_num:? \
> > > > -? __val = csr_read(__csr_num);? \
> > > > -? break;
> > > > -#define switchcase_csr_read_2(__csr_num, __val)? \
> > > > -? switchcase_csr_read(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_read(__csr_num + 1, __val)
> > > > -#define switchcase_csr_read_4(__csr_num, __val)? \
> > > > -? switchcase_csr_read_2(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_read_2(__csr_num + 2, __val)
> > > > -#define switchcase_csr_read_8(__csr_num, __val)? \
> > > > -? switchcase_csr_read_4(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_read_4(__csr_num + 4, __val)
> > > > -#define switchcase_csr_read_16(__csr_num, __val)? \
> > > > -? switchcase_csr_read_8(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_read_8(__csr_num + 8, __val)
> > > > -#define switchcase_csr_read_32(__csr_num, __val)? \
> > > > -? switchcase_csr_read_16(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_read_16(__csr_num + 16, __val)
> > > > -#define switchcase_csr_read_64(__csr_num, __val)? \
> > > > -? switchcase_csr_read_32(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_read_32(__csr_num + 32, __val)
> > > > -
> > > > -? unsigned long ret = 0;
> > > > -
> > > > -? switch (csr_num) {
> > > > -? switchcase_csr_read_16(CSR_PMPCFG0, ret)
> > > > -? switchcase_csr_read_64(CSR_PMPADDR0, ret)
> > > > -? switchcase_csr_read(CSR_MCYCLE, ret)
> > > > -? switchcase_csr_read(CSR_MINSTRET, ret)
> > > > -? switchcase_csr_read(CSR_MHPMCOUNTER3, ret)
> > > > -? switchcase_csr_read_4(CSR_MHPMCOUNTER4, ret)
> > > > -? switchcase_csr_read_8(CSR_MHPMCOUNTER8, ret)
> > > > -? switchcase_csr_read_16(CSR_MHPMCOUNTER16, ret)
> > > > -? switchcase_csr_read(CSR_MCOUNTINHIBIT, ret)
> > > > -? switchcase_csr_read(CSR_MHPMEVENT3, ret)
> > > > -? switchcase_csr_read_4(CSR_MHPMEVENT4, ret)
> > > > -? switchcase_csr_read_8(CSR_MHPMEVENT8, ret)
> > > > -? switchcase_csr_read_16(CSR_MHPMEVENT16, ret)
> > > > -#if __riscv_xlen == 32
> > > > -? switchcase_csr_read(CSR_MCYCLEH, ret)
> > > > -? switchcase_csr_read(CSR_MINSTRETH, ret)
> > > > -? switchcase_csr_read(CSR_MHPMCOUNTER3H, ret)
> > > > -? switchcase_csr_read_4(CSR_MHPMCOUNTER4H, ret)
> > > > -? switchcase_csr_read_8(CSR_MHPMCOUNTER8H, ret)
> > > > -? switchcase_csr_read_16(CSR_MHPMCOUNTER16H, ret)
> > > > -? /**
> > > > -? * The CSR range MHPMEVENT[3-16]H are available only if sscofpmf
> > > > -? * extension is present. The caller must ensure that.
> > > > -? */
> > > > -? switchcase_csr_read(CSR_MHPMEVENT3H, ret)
> > > > -? switchcase_csr_read_4(CSR_MHPMEVENT4H, ret)
> > > > -? switchcase_csr_read_8(CSR_MHPMEVENT8H, ret)
> > > > -? switchcase_csr_read_16(CSR_MHPMEVENT16H, ret)
> > > > -#endif
> > > > -
> > > > -? default:
> > > > -? sbi_panic("%s: Unknown CSR %#x", __func__, csr_num);
> > > > -? break;
> > > > -? };
> > > > -
> > > > -? return ret;
> > > > -
> > > > -#undef switchcase_csr_read_64
> > > > -#undef switchcase_csr_read_32
> > > > -#undef switchcase_csr_read_16
> > > > -#undef switchcase_csr_read_8
> > > > -#undef switchcase_csr_read_4
> > > > -#undef switchcase_csr_read_2
> > > > -#undef switchcase_csr_read
> > > > +? volatile uint32_t opcode_buff[2];
> > > > +? typedef unsigned long (*read_f)(void);
> > > > +? opcode_buff[0] = (csr_num << 20) | 0x00002573; /* csrr a0, csr */
> > > > +? opcode_buff[1] = 0x00008067; /* ret */
> > > > +? RISCV_FENCE_I;
> > > > +? return ((read_f)opcode_buff)();
> > > > }
> > > > 
> > > > void csr_write_num(int csr_num, unsigned long val)
> > > > {
> > > > -#define switchcase_csr_write(__csr_num, __val)? \
> > > > -? case __csr_num:? \
> > > > -? csr_write(__csr_num, __val);? \
> > > > -? break;
> > > > -#define switchcase_csr_write_2(__csr_num, __val)? \
> > > > -? switchcase_csr_write(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_write(__csr_num + 1, __val)
> > > > -#define switchcase_csr_write_4(__csr_num, __val)? \
> > > > -? switchcase_csr_write_2(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_write_2(__csr_num + 2, __val)
> > > > -#define switchcase_csr_write_8(__csr_num, __val)? \
> > > > -? switchcase_csr_write_4(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_write_4(__csr_num + 4, __val)
> > > > -#define switchcase_csr_write_16(__csr_num, __val)? \
> > > > -? switchcase_csr_write_8(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_write_8(__csr_num + 8, __val)
> > > > -#define switchcase_csr_write_32(__csr_num, __val)? \
> > > > -? switchcase_csr_write_16(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_write_16(__csr_num + 16, __val)
> > > > -#define switchcase_csr_write_64(__csr_num, __val)? \
> > > > -? switchcase_csr_write_32(__csr_num + 0, __val)? \
> > > > -? switchcase_csr_write_32(__csr_num + 32, __val)
> > > > -
> > > > -? switch (csr_num) {
> > > > -? switchcase_csr_write_16(CSR_PMPCFG0, val)
> > > > -? switchcase_csr_write_64(CSR_PMPADDR0, val)
> > > > -? switchcase_csr_write(CSR_MCYCLE, val)
> > > > -? switchcase_csr_write(CSR_MINSTRET, val)
> > > > -? switchcase_csr_write(CSR_MHPMCOUNTER3, val)
> > > > -? switchcase_csr_write_4(CSR_MHPMCOUNTER4, val)
> > > > -? switchcase_csr_write_8(CSR_MHPMCOUNTER8, val)
> > > > -? switchcase_csr_write_16(CSR_MHPMCOUNTER16, val)
> > > > -#if __riscv_xlen == 32
> > > > -? switchcase_csr_write(CSR_MCYCLEH, val)
> > > > -? switchcase_csr_write(CSR_MINSTRETH, val)
> > > > -? switchcase_csr_write(CSR_MHPMCOUNTER3H, val)
> > > > -? switchcase_csr_write_4(CSR_MHPMCOUNTER4H, val)
> > > > -? switchcase_csr_write_8(CSR_MHPMCOUNTER8H, val)
> > > > -? switchcase_csr_write_16(CSR_MHPMCOUNTER16H, val)
> > > > -? switchcase_csr_write(CSR_MHPMEVENT3H, val)
> > > > -? switchcase_csr_write_4(CSR_MHPMEVENT4H, val)
> > > > -? switchcase_csr_write_8(CSR_MHPMEVENT8H, val)
> > > > -? switchcase_csr_write_16(CSR_MHPMEVENT16H, val)
> > > > -#endif
> > > > -? switchcase_csr_write(CSR_MCOUNTINHIBIT, val)
> > > > -? switchcase_csr_write(CSR_MHPMEVENT3, val)
> > > > -? switchcase_csr_write_4(CSR_MHPMEVENT4, val)
> > > > -? switchcase_csr_write_8(CSR_MHPMEVENT8, val)
> > > > -? switchcase_csr_write_16(CSR_MHPMEVENT16, val)
> > > > -
> > > > -? default:
> > > > -? sbi_panic("%s: Unknown CSR %#x", __func__, csr_num);
> > > > -? break;
> > > > -? };
> > > > -
> > > > -#undef switchcase_csr_write_64
> > > > -#undef switchcase_csr_write_32
> > > > -#undef switchcase_csr_write_16
> > > > -#undef switchcase_csr_write_8
> > > > -#undef switchcase_csr_write_4
> > > > -#undef switchcase_csr_write_2
> > > > -#undef switchcase_csr_write
> > > > +? volatile uint32_t opcode_buff[2];
> > > > +? typedef void (*write_f)(unsigned long val);
> > > > +? opcode_buff[0] = (csr_num << 20) | 0x00051073; /* csrw csr, a0 */
> > > > +? opcode_buff[1] = 0x00008067; /* ret */
> > > > +? RISCV_FENCE_I;
> > > 
> > > I agree we need FENCE.I in the path but this is
> > > like a big hammer in the execution path.
> > > 
> > > This is now much slower compared to the existing
> > > approach because every CSR access using this
> > > function will execute FENCE.I
> > I passed the qemu test, and it can be run normally with
> > or without fence.i. But I'm not sure if the exception
> > happens on other platform or we have a more lightweight
> > way.
> 
> There is not. Optimising this kind of bad practice code is not a goal
> of modern architectures. After all, it totally precludes having W^X
> protection in OpenSBI (e.g. via the PMP, or via CHERI), especially
> executable stacks form a key part of *the* classic buffer overflow
> shell code attack.
> 
> Jess

Currently opensbi does not have strict RWX permission control. We only?
need W and X permissions not to exist at the same time to avoid security
risks. If opensbi has RWX permission control, we only need to change the
permission of opcode_buff before and after running. The stack default is
RW permission, we can change the permission of opcode_buff to RX before
the call, and change the permission to RW after the call is completed.

I think these jobs can be put after opensbi has rwx permission control.

Regards,
Xiang W
> 
> 




From guoren at kernel.org  Tue Jun 21 18:55:03 2022
From: guoren at kernel.org (Guo Ren)
Date: Wed, 22 Jun 2022 09:55:03 +0800
Subject: [PATCH] lib: utils/mtimer: Add has_broken_mtime quirk
In-Reply-To: <CAAhSdy2ugpwy8Dc++68CTEXsVMHLyxfrUE817cUyX5q7UP_wCQ@mail.gmail.com>
References: <20220621131132.110311-1-apatel@ventanamicro.com>
 <CAJF2gTTtjON3yUZ0btg1OuVVOZorhgwKrL4GrwViP6htOYxNgQ@mail.gmail.com> <CAAhSdy2ugpwy8Dc++68CTEXsVMHLyxfrUE817cUyX5q7UP_wCQ@mail.gmail.com>
Message-ID: <CAJF2gTTbkK7ucQCMa3cR_arEP=Oq=OJepOYL=NH1-Ff4m3Mk_Q@mail.gmail.com>

On Tue, Jun 21, 2022 at 11:04 PM Anup Patel <anup at brainfault.org> wrote:
>
> On Tue, Jun 21, 2022 at 7:33 PM Guo Ren <guoren at kernel.org> wrote:
> >
> > On Tue, Jun 21, 2022 at 9:11 PM Anup Patel <apatel at ventanamicro.com> wrote:
> > >
> > > The Allwinner D1 SoC has custom CLINT implementation where the
> > > MTIME register of the timer is not accessible as MMIO register
> > > instead the time CSR is to be used as MTIME.
> > >
> > > We introduce "has_broken_mtime" quirk in MTIMER library and FDT
> > > based MTIMER driver to address the Allwiner D1 situation. And
> > > while we are here, let us extend the quirk mechanism in FDT based
> > > MTIMER driver for both CLINT and ACLINT.
> > >
> > > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > > ---
> > >  include/sbi_utils/timer/aclint_mtimer.h |  1 +
> > >  lib/utils/timer/aclint_mtimer.c         | 30 +++++++++++++++++++------
> > >  lib/utils/timer/fdt_timer_mtimer.c      | 29 ++++++++++++++++++------
> > >  3 files changed, 46 insertions(+), 14 deletions(-)
> > >
> > > diff --git a/include/sbi_utils/timer/aclint_mtimer.h b/include/sbi_utils/timer/aclint_mtimer.h
> > > index f02cc62..454a92f 100644
> > > --- a/include/sbi_utils/timer/aclint_mtimer.h
> > > +++ b/include/sbi_utils/timer/aclint_mtimer.h
> > > @@ -32,6 +32,7 @@ struct aclint_mtimer_data {
> > >         u32 first_hartid;
> > >         u32 hart_count;
> > >         bool has_64bit_mmio;
> > > +       bool has_broken_mtime;
> > >         bool has_shared_mtime;
> > >         /* Private details (initialized and used by ACLINT MTIMER library) */
> > >         struct aclint_mtimer_data *time_delta_reference;
> > > diff --git a/lib/utils/timer/aclint_mtimer.c b/lib/utils/timer/aclint_mtimer.c
> > > index a957b1c..5db3120 100644
> > > --- a/lib/utils/timer/aclint_mtimer.c
> > > +++ b/lib/utils/timer/aclint_mtimer.c
> > > @@ -60,6 +60,12 @@ static u64 mtimer_value(void)
> > >         return mt->time_rd(time_val);
> > >  }
> > >
> > > +static u64 mtimer_dummy_value(void)
> > > +{
> > > +       /* Dummy function for MTIMER devices with broken MTIME */
> > Could we get value from CSR_TIME? (define CSR_TIME           0xc01)
>
> The lib/sbi/sbi_timer.c will use CSR_TIME whenever it is available so
> we don't need to access it here.
>
> Maybe we can drop mtimer_dummy_value() and set
> "mtimer.timer_value = NULL" when "has_broken_mtime = TRUE" ?
Agree, that make code clean.

>
> Can you try this patch on D1 ?
I would, but I need to spend some time preparing the environment.

>
> Regards,
> Anup
>
> >
> > > +       return 0;
> > > +}
> > > +
> > >  static void mtimer_event_stop(void)
> > >  {
> > >         u32 target_hart = current_hartid();
> > > @@ -95,7 +101,9 @@ void aclint_mtimer_sync(struct aclint_mtimer_data *mt)
> > >         struct aclint_mtimer_data *reference;
> > >
> > >         /* Sync-up non-shared MTIME if reference is available */
> > > -       if (mt->has_shared_mtime || !mt->time_delta_reference)
> > > +       if (mt->has_broken_mtime ||
> > > +           mt->has_shared_mtime ||
> > > +           !mt->time_delta_reference)
> > >                 return;
> > >
> > >         reference = mt->time_delta_reference;
> > > @@ -189,6 +197,10 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
> > >         if (reference && mt->mtime_freq != reference->mtime_freq)
> > >                 return SBI_EINVAL;
> > >
> > > +       /* Override timer_value() callback for broken MTIME */
> > > +       if (mt->has_broken_mtime)
> > > +               mtimer.timer_value = mtimer_dummy_value;
> > > +
> > >         /* Initialize private data */
> > >         aclint_mtimer_set_reference(mt, reference);
> > >         mt->time_rd = mtimer_time_rd32;
> > > @@ -207,21 +219,25 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
> > >                 mtimer_hartid2data[mt->first_hartid + i] = mt;
> > >
> > >         /* Add MTIMER regions to the root domain */
> > > -       if (mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
> > > +       if (!mt->has_broken_mtime &&
> > > +           mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
> > >                 rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
> > >                                         mt->mtime_size + mt->mtimecmp_size);
> > >                 if (rc)
> > >                         return rc;
> > > -       } else if (mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
> > > +       } else if (!mt->has_broken_mtime &&
> > > +                  mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
> > >                 rc = aclint_mtimer_add_regions(mt->mtime_addr,
> > >                                         mt->mtime_size + mt->mtimecmp_size);
> > >                 if (rc)
> > >                         return rc;
> > >         } else {
> > > -               rc = aclint_mtimer_add_regions(mt->mtime_addr,
> > > -                                               mt->mtime_size);
> > > -               if (rc)
> > > -                       return rc;
> > > +               if (!mt->has_broken_mtime) {
> > > +                       rc = aclint_mtimer_add_regions(mt->mtime_addr,
> > > +                                                       mt->mtime_size);
> > > +                       if (rc)
> > > +                               return rc;
> > > +               }
> > >
> > >                 rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
> > >                                                 mt->mtimecmp_size);
> > > diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> > > index e140567..bc547ac 100644
> > > --- a/lib/utils/timer/fdt_timer_mtimer.c
> > > +++ b/lib/utils/timer/fdt_timer_mtimer.c
> > > @@ -16,8 +16,10 @@
> > >  #define MTIMER_MAX_NR                  16
> > >
> > >  struct timer_mtimer_quirks {
> > > +       bool            is_clint;
> > >         unsigned int    mtime_offset;
> > >         bool            has_64bit_mmio;
> > > +       bool            has_broken_mtime;
> > >  };
> > >
> > >  static unsigned long mtimer_count = 0;
> > > @@ -30,6 +32,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> > >         int i, rc;
> > >         unsigned long addr[2], size[2];
> > >         struct aclint_mtimer_data *mt;
> > > +       const struct timer_mtimer_quirks *quirks;
> > > +
> > > +       if (!match->data)
> > > +               return SBI_ENOSYS;
> > > +       quirks = match->data;
> > >
> > >         if (MTIMER_MAX_NR <= mtimer_count)
> > >                 return SBI_ENOSPC;
> > > @@ -40,16 +47,15 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> > >                                    &mt->first_hartid, &mt->hart_count);
> > >         if (rc)
> > >                 return rc;
> > > -       mt->has_64bit_mmio = true;
> > > +       mt->has_64bit_mmio = quirks->has_64bit_mmio;
> > >         mt->has_shared_mtime = false;
> > > +       mt->has_broken_mtime = quirks->has_broken_mtime;
> > >
> > >         rc = fdt_parse_timebase_frequency(fdt, &mt->mtime_freq);
> > >         if (rc)
> > >                 return rc;
> > >
> > > -       if (match->data) { /* SiFive CLINT */
> > > -               const struct timer_mtimer_quirks *quirks = match->data;
> > > -
> > > +       if (quirks->is_clint) { /* SiFive CLINT */
> > >                 /* Set CLINT addresses */
> > >                 mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
> > >                 mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
> > > @@ -59,8 +65,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> > >                 mt->mtime_addr += quirks->mtime_offset;
> > >                 mt->mtimecmp_addr += quirks->mtime_offset;
> > >                 mt->mtime_size -= quirks->mtime_offset;
> > > -               /* Apply additional CLINT quirks */
> > > -               mt->has_64bit_mmio = quirks->has_64bit_mmio;
> > >         } else { /* RISC-V ACLINT MTIMER */
> > >                 /* Set ACLINT MTIMER addresses */
> > >                 mt->mtime_addr = addr[0];
> > > @@ -110,20 +114,31 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> > >  }
> > >
> > >  static const struct timer_mtimer_quirks d1_clint_quirks = {
> > > +       .is_clint       = true,
> > >         .mtime_offset   = CLINT_MTIMER_OFFSET,
> > >         .has_64bit_mmio = false,
> > > +       .has_broken_mtime = true,
> > >  };
> > >
> > >  static const struct timer_mtimer_quirks sifive_clint_quirks = {
> > > +       .is_clint       = true,
> > >         .mtime_offset   = CLINT_MTIMER_OFFSET,
> > >         .has_64bit_mmio = true,
> > > +       .has_broken_mtime = false,
> > > +};
> > > +
> > > +static const struct timer_mtimer_quirks aclint_quirks = {
> > > +       .is_clint       = false,
> > > +       .mtime_offset   = 0,
> > > +       .has_64bit_mmio = true,
> > > +       .has_broken_mtime = false,
> > >  };
> > >
> > >  static const struct fdt_match timer_mtimer_match[] = {
> > >         { .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
> > >         { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
> > >         { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
> > > -       { .compatible = "riscv,aclint-mtimer" },
> > > +       { .compatible = "riscv,aclint-mtimer", .data = &aclint_quirks },
> > >         { },
> > >  };
> > >
> > > --
> > > 2.34.1
> > >
> >
> >
> > --
> > Best Regards
> >  Guo Ren
> >
> > ML: https://lore.kernel.org/linux-csky/



-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/


From anup at brainfault.org  Tue Jun 21 21:57:07 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 22 Jun 2022 10:27:07 +0530
Subject: [PATCH] lib: irqchip/plic: fix typo in plic_warm_irqchip_init
In-Reply-To: <20220621120739.422990-1-jan.remes@codasip.com>
References: <20220621120739.422990-1-jan.remes@codasip.com>
Message-ID: <CAAhSdy1=SS1bX_pujJZfJUqFaSkeSZCvEWs8zYXP=QKuW5YC-g@mail.gmail.com>

On Tue, Jun 21, 2022 at 5:38 PM Jan Remes <jan.remes at codasip.com> wrote:
>
> The second invocation of plic_context_init() incorrectly calls the
> function with m_cntx_id instead of s_cntx_id. This breaks systems which
> only have 1 external interrupt per hart.
>
> Fixes: 8c362e7 ("lib: irqchip/plic: Factor out a context init function")
> Signed-off-by: Jan Remes <jan.remes at codasip.com>

Looks good to me.

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup

> ---
>  lib/utils/irqchip/plic.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
> index 8c25b9d..73d7788 100644
> --- a/lib/utils/irqchip/plic.c
> +++ b/lib/utils/irqchip/plic.c
> @@ -145,7 +145,7 @@ int plic_warm_irqchip_init(const struct plic_data *plic,
>
>         /* By default, disable all IRQs for S-mode of target HART */
>         if (s_cntx_id > -1) {
> -               ret = plic_context_init(plic, m_cntx_id, false, 0x7);
> +               ret = plic_context_init(plic, s_cntx_id, false, 0x7);
>                 if (ret)
>                         return ret;
>         }
> --
> 2.36.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From samuel at sholland.org  Tue Jun 21 21:57:42 2022
From: samuel at sholland.org (Samuel Holland)
Date: Tue, 21 Jun 2022 23:57:42 -0500
Subject: [PATCH] lib: utils/timer: Remove Allwinner D1 CLINT compatibles
Message-ID: <20220622045742.18200-1-samuel@sholland.org>

The allwinner,sun20i-d1-clint compatible string is not documented in any
official binding, so it should not be used by drivers.

The MSWI in the D1 CLINT is compatible with the ACLINT specification, so
it can take advantage of generic driver support. However, that is only
possible if the MSWI and MTIMER are split into separate DT nodes. This
means the final binding for this device is likely to be incompatible
with what is implemented here.

Remove this compatible string from the driver to prevent it from
appearing in a stable version and causing future issues.

Signed-off-by: Samuel Holland <samuel at sholland.org>
---

 lib/utils/ipi/fdt_ipi_mswi.c       | 1 -
 lib/utils/timer/fdt_timer_mtimer.c | 6 ------
 2 files changed, 7 deletions(-)

diff --git a/lib/utils/ipi/fdt_ipi_mswi.c b/lib/utils/ipi/fdt_ipi_mswi.c
index af69e16..0176941 100644
--- a/lib/utils/ipi/fdt_ipi_mswi.c
+++ b/lib/utils/ipi/fdt_ipi_mswi.c
@@ -54,7 +54,6 @@ static int ipi_mswi_cold_init(void *fdt, int nodeoff,
 static const unsigned long clint_offset = CLINT_MSWI_OFFSET;
 
 static const struct fdt_match ipi_mswi_match[] = {
-	{ .compatible = "allwinner,sun20i-d1-clint", .data = &clint_offset },
 	{ .compatible = "riscv,clint0", .data = &clint_offset },
 	{ .compatible = "sifive,clint0", .data = &clint_offset },
 	{ .compatible = "riscv,aclint-mswi" },
diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
index e140567..7b8546b 100644
--- a/lib/utils/timer/fdt_timer_mtimer.c
+++ b/lib/utils/timer/fdt_timer_mtimer.c
@@ -109,18 +109,12 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 	return 0;
 }
 
-static const struct timer_mtimer_quirks d1_clint_quirks = {
-	.mtime_offset	= CLINT_MTIMER_OFFSET,
-	.has_64bit_mmio	= false,
-};
-
 static const struct timer_mtimer_quirks sifive_clint_quirks = {
 	.mtime_offset	= CLINT_MTIMER_OFFSET,
 	.has_64bit_mmio	= true,
 };
 
 static const struct fdt_match timer_mtimer_match[] = {
-	{ .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
 	{ .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
 	{ .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
 	{ .compatible = "riscv,aclint-mtimer" },
-- 
2.35.1



From samuel at sholland.org  Tue Jun 21 21:59:23 2022
From: samuel at sholland.org (Samuel Holland)
Date: Tue, 21 Jun 2022 23:59:23 -0500
Subject: [PATCH] lib: irqchip/plic: fix typo in plic_warm_irqchip_init
In-Reply-To: <20220621120739.422990-1-jan.remes@codasip.com>
References: <20220621120739.422990-1-jan.remes@codasip.com>
Message-ID: <7472a47d-be9e-b7c1-a3c1-1e2fde2e835d@sholland.org>

On 6/21/22 7:07 AM, Jan Remes wrote:
> The second invocation of plic_context_init() incorrectly calls the
> function with m_cntx_id instead of s_cntx_id. This breaks systems which
> only have 1 external interrupt per hart.
> 
> Fixes: 8c362e7 ("lib: irqchip/plic: Factor out a context init function")
> Signed-off-by: Jan Remes <jan.remes at codasip.com>

Reviewed-by: Samuel Holland <samuel at sholland.org>


From anup at brainfault.org  Tue Jun 21 22:17:22 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 22 Jun 2022 10:47:22 +0530
Subject: [PATCH] lib: utils/timer: Remove Allwinner D1 CLINT compatibles
In-Reply-To: <20220622045742.18200-1-samuel@sholland.org>
References: <20220622045742.18200-1-samuel@sholland.org>
Message-ID: <CAAhSdy2qL6pUF+2KG-pZbByW3RKWk64gCm8Ugx15ShusTuHCDg@mail.gmail.com>

On Wed, Jun 22, 2022 at 10:27 AM Samuel Holland <samuel at sholland.org> wrote:
>
> The allwinner,sun20i-d1-clint compatible string is not documented in any
> official binding, so it should not be used by drivers.
>
> The MSWI in the D1 CLINT is compatible with the ACLINT specification, so
> it can take advantage of generic driver support. However, that is only
> possible if the MSWI and MTIMER are split into separate DT nodes. This
> means the final binding for this device is likely to be incompatible
> with what is implemented here.

I don't think splitting D1 CLINT into two separate DT nodes is
wrong because traditionally CLINT has been a composite device
(consisting of a timer and IPI device).

If you want resemblance of DT with HW then you can do something
like below:

clint {
    mswi at abc {
        ...
    };

    mtimer at xyz {
        ...
    };
};

>
> Remove this compatible string from the driver to prevent it from
> appearing in a stable version and causing future issues.
>
> Signed-off-by: Samuel Holland <samuel at sholland.org>

Reviewed-by: Anup Patel <anup at brainfault.org>

Regards,
Anup


> ---
>
>  lib/utils/ipi/fdt_ipi_mswi.c       | 1 -
>  lib/utils/timer/fdt_timer_mtimer.c | 6 ------
>  2 files changed, 7 deletions(-)
>
> diff --git a/lib/utils/ipi/fdt_ipi_mswi.c b/lib/utils/ipi/fdt_ipi_mswi.c
> index af69e16..0176941 100644
> --- a/lib/utils/ipi/fdt_ipi_mswi.c
> +++ b/lib/utils/ipi/fdt_ipi_mswi.c
> @@ -54,7 +54,6 @@ static int ipi_mswi_cold_init(void *fdt, int nodeoff,
>  static const unsigned long clint_offset = CLINT_MSWI_OFFSET;
>
>  static const struct fdt_match ipi_mswi_match[] = {
> -       { .compatible = "allwinner,sun20i-d1-clint", .data = &clint_offset },
>         { .compatible = "riscv,clint0", .data = &clint_offset },
>         { .compatible = "sifive,clint0", .data = &clint_offset },
>         { .compatible = "riscv,aclint-mswi" },
> diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> index e140567..7b8546b 100644
> --- a/lib/utils/timer/fdt_timer_mtimer.c
> +++ b/lib/utils/timer/fdt_timer_mtimer.c
> @@ -109,18 +109,12 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
>         return 0;
>  }
>
> -static const struct timer_mtimer_quirks d1_clint_quirks = {
> -       .mtime_offset   = CLINT_MTIMER_OFFSET,
> -       .has_64bit_mmio = false,
> -};
> -
>  static const struct timer_mtimer_quirks sifive_clint_quirks = {
>         .mtime_offset   = CLINT_MTIMER_OFFSET,
>         .has_64bit_mmio = true,
>  };
>
>  static const struct fdt_match timer_mtimer_match[] = {
> -       { .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
>         { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
>         { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
>         { .compatible = "riscv,aclint-mtimer" },
> --
> 2.35.1
>


From anup at brainfault.org  Tue Jun 21 22:30:33 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 22 Jun 2022 11:00:33 +0530
Subject: [PATCH] lib: irqchip/plic: fix typo in plic_warm_irqchip_init
In-Reply-To: <CAAhSdy1=SS1bX_pujJZfJUqFaSkeSZCvEWs8zYXP=QKuW5YC-g@mail.gmail.com>
References: <20220621120739.422990-1-jan.remes@codasip.com> <CAAhSdy1=SS1bX_pujJZfJUqFaSkeSZCvEWs8zYXP=QKuW5YC-g@mail.gmail.com>
Message-ID: <CAAhSdy3NDG0uMK8+p+8b1BnyNod8PesxFoZPPf4A=P+G-Yznbg@mail.gmail.com>

On Wed, Jun 22, 2022 at 10:27 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Tue, Jun 21, 2022 at 5:38 PM Jan Remes <jan.remes at codasip.com> wrote:
> >
> > The second invocation of plic_context_init() incorrectly calls the
> > function with m_cntx_id instead of s_cntx_id. This breaks systems which
> > only have 1 external interrupt per hart.
> >
> > Fixes: 8c362e7 ("lib: irqchip/plic: Factor out a context init function")
> > Signed-off-by: Jan Remes <jan.remes at codasip.com>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
> Regards,
> Anup
>
> > ---
> >  lib/utils/irqchip/plic.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/lib/utils/irqchip/plic.c b/lib/utils/irqchip/plic.c
> > index 8c25b9d..73d7788 100644
> > --- a/lib/utils/irqchip/plic.c
> > +++ b/lib/utils/irqchip/plic.c
> > @@ -145,7 +145,7 @@ int plic_warm_irqchip_init(const struct plic_data *plic,
> >
> >         /* By default, disable all IRQs for S-mode of target HART */
> >         if (s_cntx_id > -1) {
> > -               ret = plic_context_init(plic, m_cntx_id, false, 0x7);
> > +               ret = plic_context_init(plic, s_cntx_id, false, 0x7);
> >                 if (ret)
> >                         return ret;
> >         }
> > --
> > 2.36.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi


From anup at brainfault.org  Tue Jun 21 22:30:52 2022
From: anup at brainfault.org (Anup Patel)
Date: Wed, 22 Jun 2022 11:00:52 +0530
Subject: [PATCH] lib: utils/timer: Remove Allwinner D1 CLINT compatibles
In-Reply-To: <CAAhSdy2qL6pUF+2KG-pZbByW3RKWk64gCm8Ugx15ShusTuHCDg@mail.gmail.com>
References: <20220622045742.18200-1-samuel@sholland.org> <CAAhSdy2qL6pUF+2KG-pZbByW3RKWk64gCm8Ugx15ShusTuHCDg@mail.gmail.com>
Message-ID: <CAAhSdy0yzHt8A_z0nkwh+sQx5Bz-7-16rpBpWjzcethVmjWajg@mail.gmail.com>

On Wed, Jun 22, 2022 at 10:47 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Wed, Jun 22, 2022 at 10:27 AM Samuel Holland <samuel at sholland.org> wrote:
> >
> > The allwinner,sun20i-d1-clint compatible string is not documented in any
> > official binding, so it should not be used by drivers.
> >
> > The MSWI in the D1 CLINT is compatible with the ACLINT specification, so
> > it can take advantage of generic driver support. However, that is only
> > possible if the MSWI and MTIMER are split into separate DT nodes. This
> > means the final binding for this device is likely to be incompatible
> > with what is implemented here.
>
> I don't think splitting D1 CLINT into two separate DT nodes is
> wrong because traditionally CLINT has been a composite device
> (consisting of a timer and IPI device).
>
> If you want resemblance of DT with HW then you can do something
> like below:
>
> clint {
>     mswi at abc {
>         ...
>     };
>
>     mtimer at xyz {
>         ...
>     };
> };
>
> >
> > Remove this compatible string from the driver to prevent it from
> > appearing in a stable version and causing future issues.
> >
> > Signed-off-by: Samuel Holland <samuel at sholland.org>
>
> Reviewed-by: Anup Patel <anup at brainfault.org>

Applied this patch to the riscv/opensbi repo.

Thanks,
Anup

>
> Regards,
> Anup
>
>
> > ---
> >
> >  lib/utils/ipi/fdt_ipi_mswi.c       | 1 -
> >  lib/utils/timer/fdt_timer_mtimer.c | 6 ------
> >  2 files changed, 7 deletions(-)
> >
> > diff --git a/lib/utils/ipi/fdt_ipi_mswi.c b/lib/utils/ipi/fdt_ipi_mswi.c
> > index af69e16..0176941 100644
> > --- a/lib/utils/ipi/fdt_ipi_mswi.c
> > +++ b/lib/utils/ipi/fdt_ipi_mswi.c
> > @@ -54,7 +54,6 @@ static int ipi_mswi_cold_init(void *fdt, int nodeoff,
> >  static const unsigned long clint_offset = CLINT_MSWI_OFFSET;
> >
> >  static const struct fdt_match ipi_mswi_match[] = {
> > -       { .compatible = "allwinner,sun20i-d1-clint", .data = &clint_offset },
> >         { .compatible = "riscv,clint0", .data = &clint_offset },
> >         { .compatible = "sifive,clint0", .data = &clint_offset },
> >         { .compatible = "riscv,aclint-mswi" },
> > diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
> > index e140567..7b8546b 100644
> > --- a/lib/utils/timer/fdt_timer_mtimer.c
> > +++ b/lib/utils/timer/fdt_timer_mtimer.c
> > @@ -109,18 +109,12 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
> >         return 0;
> >  }
> >
> > -static const struct timer_mtimer_quirks d1_clint_quirks = {
> > -       .mtime_offset   = CLINT_MTIMER_OFFSET,
> > -       .has_64bit_mmio = false,
> > -};
> > -
> >  static const struct timer_mtimer_quirks sifive_clint_quirks = {
> >         .mtime_offset   = CLINT_MTIMER_OFFSET,
> >         .has_64bit_mmio = true,
> >  };
> >
> >  static const struct fdt_match timer_mtimer_match[] = {
> > -       { .compatible = "allwinner,sun20i-d1-clint", .data = &d1_clint_quirks },
> >         { .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
> >         { .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
> >         { .compatible = "riscv,aclint-mtimer" },
> > --
> > 2.35.1
> >


From samuel at sholland.org  Tue Jun 21 22:31:41 2022
From: samuel at sholland.org (Samuel Holland)
Date: Wed, 22 Jun 2022 00:31:41 -0500
Subject: [RFC PATCH 2/2] lib: utils/timer: Add a separate compatible for
 the D1 CLINT
In-Reply-To: <CAAhSdy3PJuAj-jY5ifJ_pRZXsH7zx07JMuBVCHBjoUUu=Tf=ZA@mail.gmail.com>
References: <20211020015838.45399-1-samuel@sholland.org>
 <20211020015838.45399-3-samuel@sholland.org>
 <CAAhSdy3r20Q4mBkTR58Befo+SYhsLx3ueq=MOb4UQXk3P=8F-w@mail.gmail.com>
 <d57d4cb7-bf64-c1c8-a0bf-174e36207d3c@sholland.org>
 <CAAhSdy0+2GY1U==iGHn5udFwSWUTEEKbtG=fSxu2vC4As1E2gg@mail.gmail.com>
 <d9249dd3-ae0d-d2ff-f785-d2a1df2e6d45@sholland.org>
 <CAAhSdy3PJuAj-jY5ifJ_pRZXsH7zx07JMuBVCHBjoUUu=Tf=ZA@mail.gmail.com>
Message-ID: <d38ecf54-2851-dfe6-6f22-d96cd62227cf@sholland.org>

On 6/21/22 8:29 AM, Anup Patel wrote:
> On Tue, Jun 21, 2022 at 10:50 AM Samuel Holland <samuel at sholland.org> wrote:
>>
>> On 6/20/22 11:54 PM, Anup Patel wrote:
>>> On Tue, Jun 21, 2022 at 10:13 AM Samuel Holland <samuel at sholland.org> wrote:
>>>>
>>>> On 6/20/22 11:20 PM, Anup Patel wrote:
>>>>> On Wed, Oct 20, 2021 at 7:28 AM Samuel Holland <samuel at sholland.org> wrote:
>>>>>>
>>>>>> The CLINT in the Allwinner D1 SoC apparently does not support 64-bit
>>>>>> MMIO access. A property was added to support this quirk (and that
>>>>>> property was copied to the ACLINT MTIMER code). However, since this
>>>>>> difference in behavior makes the D1 CLINT incompatible with the SiFive
>>>>>> CLINT's programming interface, a better solution is to use a separate
>>>>>> compatible string.
>>>>>>
>>>>>> Signed-off-by: Samuel Holland <samuel at sholland.org>
>>>>>
>>>>> Due to some unknown reason, this patch slipped through cracks and was
>>>>> never applied.
>>>>
>>>> Oh, I intentionally did not send a new version of the series because I later
>>>> found out that C9xx CLINT has no mtime register at all[1][2]. The issue has
>>>> nothing to do with 64-bit vs 32-bit access. So this change is wrong; I will need
>>>> to send a follow-up removing the allwinner,sun20i-d1-clint compatibles.
>>>
>>> I had tested on QEMU virt and sifive_u machine so at least this does not
>>> break anything on QEMU and SiFive boards.
>>
>> OK, I will send a follow-up patch tomorrow.
> 
> Thanks. I was thinking of having a quirk in the mtimer driver for the D1 clint
> situation. I sent a quirk patch but I have tested only on QEMU RV32 and RV64.
I think your patch is what we want to do eventually, but I want to get the
binding settled first.

Regards,
Samuel


From apatel at ventanamicro.com  Tue Jun 21 22:44:23 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Wed, 22 Jun 2022 11:14:23 +0530
Subject: [PATCH v2] lib: utils/mtimer: Add has_broken_mtime quirk
Message-ID: <20220622054423.182460-1-apatel@ventanamicro.com>

The Allwinner D1 SoC has custom CLINT implementation where the
MTIME register of the timer is not accessible as MMIO register
instead the time CSR is to be used as MTIME.

We introduce "has_broken_mtime" quirk in MTIMER library and FDT
based MTIMER driver to address the Allwiner D1 situation. And
while we are here, let us extend the quirk mechanism in FDT based
MTIMER driver for both CLINT and ACLINT.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
Changes since v1:
 - Rebased on latest OpenSBI
 - Removed mtimer_dummy_value() from aclint_mtimer.c
---
 include/sbi_utils/timer/aclint_mtimer.h |  1 +
 lib/utils/timer/aclint_mtimer.c         | 24 +++++++++++++++-------
 lib/utils/timer/fdt_timer_mtimer.c      | 27 ++++++++++++++++++-------
 3 files changed, 38 insertions(+), 14 deletions(-)

diff --git a/include/sbi_utils/timer/aclint_mtimer.h b/include/sbi_utils/timer/aclint_mtimer.h
index f02cc62..454a92f 100644
--- a/include/sbi_utils/timer/aclint_mtimer.h
+++ b/include/sbi_utils/timer/aclint_mtimer.h
@@ -32,6 +32,7 @@ struct aclint_mtimer_data {
 	u32 first_hartid;
 	u32 hart_count;
 	bool has_64bit_mmio;
+	bool has_broken_mtime;
 	bool has_shared_mtime;
 	/* Private details (initialized and used by ACLINT MTIMER library) */
 	struct aclint_mtimer_data *time_delta_reference;
diff --git a/lib/utils/timer/aclint_mtimer.c b/lib/utils/timer/aclint_mtimer.c
index a957b1c..cf492d1 100644
--- a/lib/utils/timer/aclint_mtimer.c
+++ b/lib/utils/timer/aclint_mtimer.c
@@ -95,7 +95,9 @@ void aclint_mtimer_sync(struct aclint_mtimer_data *mt)
 	struct aclint_mtimer_data *reference;
 
 	/* Sync-up non-shared MTIME if reference is available */
-	if (mt->has_shared_mtime || !mt->time_delta_reference)
+	if (mt->has_broken_mtime ||
+	    mt->has_shared_mtime ||
+	    !mt->time_delta_reference)
 		return;
 
 	reference = mt->time_delta_reference;
@@ -189,6 +191,10 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
 	if (reference && mt->mtime_freq != reference->mtime_freq)
 		return SBI_EINVAL;
 
+	/* Set timer_value() callback to NULL for broken MTIME */
+	if (mt->has_broken_mtime)
+		mtimer.timer_value = NULL;
+
 	/* Initialize private data */
 	aclint_mtimer_set_reference(mt, reference);
 	mt->time_rd = mtimer_time_rd32;
@@ -207,21 +213,25 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
 		mtimer_hartid2data[mt->first_hartid + i] = mt;
 
 	/* Add MTIMER regions to the root domain */
-	if (mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
+	if (!mt->has_broken_mtime &&
+	    mt->mtime_addr == (mt->mtimecmp_addr + mt->mtimecmp_size)) {
 		rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
 					mt->mtime_size + mt->mtimecmp_size);
 		if (rc)
 			return rc;
-	} else if (mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
+	} else if (!mt->has_broken_mtime &&
+		   mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
 		rc = aclint_mtimer_add_regions(mt->mtime_addr,
 					mt->mtime_size + mt->mtimecmp_size);
 		if (rc)
 			return rc;
 	} else {
-		rc = aclint_mtimer_add_regions(mt->mtime_addr,
-						mt->mtime_size);
-		if (rc)
-			return rc;
+		if (!mt->has_broken_mtime) {
+			rc = aclint_mtimer_add_regions(mt->mtime_addr,
+							mt->mtime_size);
+			if (rc)
+				return rc;
+		}
 
 		rc = aclint_mtimer_add_regions(mt->mtimecmp_addr,
 						mt->mtimecmp_size);
diff --git a/lib/utils/timer/fdt_timer_mtimer.c b/lib/utils/timer/fdt_timer_mtimer.c
index 7b8546b..41fe0a0 100644
--- a/lib/utils/timer/fdt_timer_mtimer.c
+++ b/lib/utils/timer/fdt_timer_mtimer.c
@@ -16,8 +16,10 @@
 #define MTIMER_MAX_NR			16
 
 struct timer_mtimer_quirks {
+	bool		is_clint;
 	unsigned int	mtime_offset;
 	bool		has_64bit_mmio;
+	bool		has_broken_mtime;
 };
 
 static unsigned long mtimer_count = 0;
@@ -30,6 +32,11 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 	int i, rc;
 	unsigned long addr[2], size[2];
 	struct aclint_mtimer_data *mt;
+	const struct timer_mtimer_quirks *quirks;
+
+	if (!match->data)
+		return SBI_ENOSYS;
+	quirks = match->data;
 
 	if (MTIMER_MAX_NR <= mtimer_count)
 		return SBI_ENOSPC;
@@ -40,16 +47,15 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 				   &mt->first_hartid, &mt->hart_count);
 	if (rc)
 		return rc;
-	mt->has_64bit_mmio = true;
+	mt->has_64bit_mmio = quirks->has_64bit_mmio;
 	mt->has_shared_mtime = false;
+	mt->has_broken_mtime = quirks->has_broken_mtime;
 
 	rc = fdt_parse_timebase_frequency(fdt, &mt->mtime_freq);
 	if (rc)
 		return rc;
 
-	if (match->data) { /* SiFive CLINT */
-		const struct timer_mtimer_quirks *quirks = match->data;
-
+	if (quirks->is_clint) { /* SiFive CLINT */
 		/* Set CLINT addresses */
 		mt->mtimecmp_addr = addr[0] + ACLINT_DEFAULT_MTIMECMP_OFFSET;
 		mt->mtimecmp_size = ACLINT_DEFAULT_MTIMECMP_SIZE;
@@ -59,8 +65,6 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 		mt->mtime_addr += quirks->mtime_offset;
 		mt->mtimecmp_addr += quirks->mtime_offset;
 		mt->mtime_size -= quirks->mtime_offset;
-		/* Apply additional CLINT quirks */
-		mt->has_64bit_mmio = quirks->has_64bit_mmio;
 	} else { /* RISC-V ACLINT MTIMER */
 		/* Set ACLINT MTIMER addresses */
 		mt->mtime_addr = addr[0];
@@ -110,14 +114,23 @@ static int timer_mtimer_cold_init(void *fdt, int nodeoff,
 }
 
 static const struct timer_mtimer_quirks sifive_clint_quirks = {
+	.is_clint	= true,
 	.mtime_offset	= CLINT_MTIMER_OFFSET,
 	.has_64bit_mmio	= true,
+	.has_broken_mtime = false,
+};
+
+static const struct timer_mtimer_quirks aclint_quirks = {
+	.is_clint	= false,
+	.mtime_offset	= 0,
+	.has_64bit_mmio	= true,
+	.has_broken_mtime = false,
 };
 
 static const struct fdt_match timer_mtimer_match[] = {
 	{ .compatible = "riscv,clint0", .data = &sifive_clint_quirks },
 	{ .compatible = "sifive,clint0", .data = &sifive_clint_quirks },
-	{ .compatible = "riscv,aclint-mtimer" },
+	{ .compatible = "riscv,aclint-mtimer", .data = &aclint_quirks },
 	{ },
 };
 
-- 
2.34.1



From dramforever at live.com  Wed Jun 22 00:12:45 2022
From: dramforever at live.com (dramforever)
Date: Wed, 22 Jun 2022 15:12:45 +0800
Subject: [PATCH v4] lib: sbi: Improve csr read and write
In-Reply-To: <eb75c146d4657e71ca67dce5517438e80f4af962.camel@126.com>
References: <20220526144214.308921-1-wxjstz@126.com>
 <CAK9=C2U9KyVTJhXN9jOqqR7JMYz0Mn9ar=seXbPg1+UdHhhbpg@mail.gmail.com>
 <1e615125574b838bc99dd58239874fa2e1685bdb.camel@126.com>
 <81DE08C6-50D4-4CC2-AE63-266D727F083F@jrtc27.com>
 <eb75c146d4657e71ca67dce5517438e80f4af962.camel@126.com>
Message-ID: <TYYP286MB1439D9E3CB6446A5AA91353AC6B29@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>


On 6/21/22 23:53, Xiang W wrote:
> ? 2022-05-26???? 16:28 +0100?Jessica Clarke???
>> On 26 May 2022, at 16:19, Xiang W <wxjstz at 126.com> wrote:
>>> ? 2022-05-26???? 20:35 +0530?Anup Patel???
>>>> On Thu, May 26, 2022 at 8:12 PM Xiang W <wxjstz at 126.com> wrote:
>>>>> <patch contents snipped>
>>>> I agree we need FENCE.I in the path but this is
>>>> like a big hammer in the execution path.
>>>>
>>>> This is now much slower compared to the existing
>>>> approach because every CSR access using this
>>>> function will execute FENCE.I
>>> I passed the qemu test, and it can be run normally with
>>> or without fence.i. But I'm not sure if the exception
>>> happens on other platform or we have a more lightweight
>>> way.
>> There is not. Optimising this kind of bad practice code is not a goal
>> of modern architectures. After all, it totally precludes having W^X
>> protection in OpenSBI (e.g. via the PMP, or via CHERI), especially
>> executable stacks form a key part of *the* classic buffer overflow
>> shell code attack.
>>
>> Jess
> Currently opensbi does not have strict RWX permission control. We only?
> need W and X permissions not to exist at the same time to avoid security
> risks. If opensbi has RWX permission control, we only need to change the
> permission of opcode_buff before and after running. The stack default is
> RW permission, we can change the permission of opcode_buff to RX before
> the call, and change the permission to RW after the call is completed.
>
> I think these jobs can be put after opensbi has rwx permission control.
>
> Regards,
> Xiang W

But in that case, wouldn't this codegen approach be even slower? Is
there any particular reason to believe that this will be faster than the
current switch-case? Some benchmark on existing silicon would be nice.

The original implementation compiles down a jump table, with several
branches and bound checks at the beginning. It really is just a hunch,
but I find it difficult to be convinced how generating code could be faster.

Optimizing the current switch-case approach into a better jump table
seems to be the way to go here. Currently the compiler does not seem to
take advantage of the fact that every csr* instruction is exactly 4
bytes. There also seems to be a few extra instructions for dealing with
int instead of long, though I am not sure. Furthermore, having separate
functions for hpmcounter, pmpcfg, etc. might help with avoiding some
branches, and possibly also make code clearer at call site.

Regards,
dram


From adnan.chowdhury at sifive.com  Wed Jun 22 02:36:48 2022
From: adnan.chowdhury at sifive.com (Adnan Chowdhury)
Date: Wed, 22 Jun 2022 10:36:48 +0100
Subject: FDT based DesignWare I3C driver for the generic platform
Message-ID: <CAAsT5XoYO1=mvDydx_XB8SS6-SAUi2=RqC4veV57TwN0LGdohw@mail.gmail.com>

Hello Team,

This is my first email to the list so please accept my apology if I'm
not clear about anything.

I've been working for an announced SoC. This piece of hardware comes
with a DesignWare I3C block. We'll be using OpenSBI in our software
stack for the SoC. The plan is to use OpenSBI's generic platform for
the project. I've written an FDT based I3C driver so that the
DesignWare I3C can be used. I've implemented it in two parts:

- fdt_i3c.c/h: A generic FDT I3C module that parses the FDT for the
presence of I3C controllers.
- dwc_i3c.c/h: This module has all the necessary functions that can be
used in our project (namely I2C private read/write), and is expandable
for adding more functionalities in the future. It uses fdt_i3c.c/h.

I need your feedback on whether it's a good idea to add these modules
to OpenSBI. If you think it's appropriate then I will send out the
patches. If you think it should be implemented differently, please
suggest that too.

Thank you for your time and any valuable feedback on this.

Best Regards,
Adnan Rahman Chowdhury


From apatel at ventanamicro.com  Wed Jun 22 04:51:52 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Wed, 22 Jun 2022 17:21:52 +0530
Subject: FDT based DesignWare I3C driver for the generic platform
In-Reply-To: <CAAsT5XoYO1=mvDydx_XB8SS6-SAUi2=RqC4veV57TwN0LGdohw@mail.gmail.com>
References: <CAAsT5XoYO1=mvDydx_XB8SS6-SAUi2=RqC4veV57TwN0LGdohw@mail.gmail.com>
Message-ID: <CAK9=C2X-7Rt3bKSNRxPGzOU2azK0A2xSio2wAi_Cv2sn+QNHiA@mail.gmail.com>

On Wed, Jun 22, 2022 at 3:07 PM Adnan Chowdhury
<adnan.chowdhury at sifive.com> wrote:
>
> Hello Team,
>
> This is my first email to the list so please accept my apology if I'm
> not clear about anything.

No need to be apologetic for asking questions.

>
> I've been working for an announced SoC. This piece of hardware comes
> with a DesignWare I3C block. We'll be using OpenSBI in our software
> stack for the SoC. The plan is to use OpenSBI's generic platform for
> the project. I've written an FDT based I3C driver so that the

That's really nice.

I assume you need i3c for system reset or something else ?

> DesignWare I3C can be used. I've implemented it in two parts:
>
> - fdt_i3c.c/h: A generic FDT I3C module that parses the FDT for the
> presence of I3C controllers.
> - dwc_i3c.c/h: This module has all the necessary functions that can be
> used in our project (namely I2C private read/write), and is expandable
> for adding more functionalities in the future. It uses fdt_i3c.c/h.

I think the i3c support can be similar to the OpenSBI i2c support:

1) Simple i3c bus library (manage and find i3c buses)
lib/utils/i3c/i3c.c
include/sbi_utils/i3c/i3c.h

2) FDT based i3c bus (find and initialize i3c buses from FDT)
lib/utils/i3c/fdt_i3c.c
include/sbi_utils/i3c/fdt_i3c.h

The i3c bus instance is analogous to the i2c adapter instance but the
major difference between i2c and i3c will be the APIs or operations
supported for devices.

Also, for every i3c bus instance the i3c.c will also register a corresponding
i2c adapter instance so that various i2c devices can be accessed via i3c
bus instance.

The i3c.c/h and fdt_i3c.c/h should only add APIs or functions which are
needed to keep the i3c support simple and minimalistic.

>
> I need your feedback on whether it's a good idea to add these modules
> to OpenSBI. If you think it's appropriate then I will send out the
> patches. If you think it should be implemented differently, please
> suggest that too.

Sure, please send your patches to this list (using git send-email).

>
> Thank you for your time and any valuable feedback on this.
>
> Best Regards,
> Adnan Rahman Chowdhury
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup


From wxjstz at 126.com  Wed Jun 22 07:57:47 2022
From: wxjstz at 126.com (Xiang W)
Date: Wed, 22 Jun 2022 22:57:47 +0800
Subject: [PATCH v4] lib: sbi: Improve csr read and write
In-Reply-To: <TYYP286MB1439D9E3CB6446A5AA91353AC6B29@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
References: <20220526144214.308921-1-wxjstz@126.com>
	 <CAK9=C2U9KyVTJhXN9jOqqR7JMYz0Mn9ar=seXbPg1+UdHhhbpg@mail.gmail.com>
	 <1e615125574b838bc99dd58239874fa2e1685bdb.camel@126.com>
	 <81DE08C6-50D4-4CC2-AE63-266D727F083F@jrtc27.com>
	 <eb75c146d4657e71ca67dce5517438e80f4af962.camel@126.com>
	 <TYYP286MB1439D9E3CB6446A5AA91353AC6B29@TYYP286MB1439.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <9fd598b84978511bcaeb9c4384d0b0c958d55d47.camel@126.com>

? 2022-06-22???? 15:12 +0800?dramforever???
> 
> On 6/21/22 23:53, Xiang W wrote:
> > ? 2022-05-26???? 16:28 +0100?Jessica Clarke???
> > > On 26 May 2022, at 16:19, Xiang W <wxjstz at 126.com> wrote:
> > > > ? 2022-05-26???? 20:35 +0530?Anup Patel???
> > > > > On Thu, May 26, 2022 at 8:12 PM Xiang W <wxjstz at 126.com> wrote:
> > > > > > <patch contents snipped>
> > > > > I agree we need FENCE.I in the path but this is
> > > > > like a big hammer in the execution path.
> > > > > 
> > > > > This is now much slower compared to the existing
> > > > > approach because every CSR access using this
> > > > > function will execute FENCE.I
> > > > I passed the qemu test, and it can be run normally with
> > > > or without fence.i. But I'm not sure if the exception
> > > > happens on other platform or we have a more lightweight
> > > > way.
> > > There is not. Optimising this kind of bad practice code is not a goal
> > > of modern architectures. After all, it totally precludes having W^X
> > > protection in OpenSBI (e.g. via the PMP, or via CHERI), especially
> > > executable stacks form a key part of *the* classic buffer overflow
> > > shell code attack.
> > > 
> > > Jess
> > Currently opensbi does not have strict RWX permission control. We only?
> > need W and X permissions not to exist at the same time to avoid security
> > risks. If opensbi has RWX permission control, we only need to change the
> > permission of opcode_buff before and after running. The stack default is
> > RW permission, we can change the permission of opcode_buff to RX before
> > the call, and change the permission to RW after the call is completed.
> > 
> > I think these jobs can be put after opensbi has rwx permission control.
> > 
> > Regards,
> > Xiang W
> 
> But in that case, wouldn't this codegen approach be even slower? Is
> there any particular reason to believe that this will be faster than the
> current switch-case? Some benchmark on existing silicon would be nice.

Yes, it will definitely be slower. This will only reduce the maintenance
cost of csr_read_num/csr_write_num, without adding code to read a new csr.

Regards,
Xiang W
> 
> The original implementation compiles down a jump table, with several
> branches and bound checks at the beginning. It really is just a hunch,
> but I find it difficult to be convinced how generating code could be faster.
> 
> Optimizing the current switch-case approach into a better jump table
> seems to be the way to go here. Currently the compiler does not seem to
> take advantage of the fact that every csr* instruction is exactly 4
> bytes. There also seems to be a few extra instructions for dealing with
> int instead of long, though I am not sure. Furthermore, having separate
> functions for hpmcounter, pmpcfg, etc. might help with avoiding some
> branches, and possibly also make code clearer at call site.
> 
> Regards,
> dram
> 




From adnan.chowdhury at sifive.com  Wed Jun 22 08:06:42 2022
From: adnan.chowdhury at sifive.com (Adnan Chowdhury)
Date: Wed, 22 Jun 2022 16:06:42 +0100
Subject: FDT based DesignWare I3C driver for the generic platform
In-Reply-To: <CAK9=C2X-7Rt3bKSNRxPGzOU2azK0A2xSio2wAi_Cv2sn+QNHiA@mail.gmail.com>
References: <CAAsT5XoYO1=mvDydx_XB8SS6-SAUi2=RqC4veV57TwN0LGdohw@mail.gmail.com>
 <CAK9=C2X-7Rt3bKSNRxPGzOU2azK0A2xSio2wAi_Cv2sn+QNHiA@mail.gmail.com>
Message-ID: <CAAsT5Xo8fGYG9VR6+aikq3DmOPb42uw0yJEGhDtAEBmZ6Ab2Hg@mail.gmail.com>

On Wed, Jun 22, 2022 at 12:52 PM Anup Patel <apatel at ventanamicro.com> wrote:
>
> On Wed, Jun 22, 2022 at 3:07 PM Adnan Chowdhury
> <adnan.chowdhury at sifive.com> wrote:
> >
> > Hello Team,
> >
> > This is my first email to the list so please accept my apology if I'm
> > not clear about anything.
>
> No need to be apologetic for asking questions.
>
> >
> > I've been working for an announced SoC. This piece of hardware comes
> > with a DesignWare I3C block. We'll be using OpenSBI in our software
> > stack for the SoC. The plan is to use OpenSBI's generic platform for
> > the project. I've written an FDT based I3C driver so that the
>
> That's really nice.
>
> I assume you need i3c for system reset or something else ?

The OpenSBI will access a power management IC which uses I2C protocol.
The SoC only has DesignWare I3C block so the driver needs to support that.
However, there are other I3C devices connected to the SoC, but OpenSBI won't
be using them, at least no plan as of current project requirement.

>
> > DesignWare I3C can be used. I've implemented it in two parts:
> >
> > - fdt_i3c.c/h: A generic FDT I3C module that parses the FDT for the
> > presence of I3C controllers.
> > - dwc_i3c.c/h: This module has all the necessary functions that can be
> > used in our project (namely I2C private read/write), and is expandable
> > for adding more functionalities in the future. It uses fdt_i3c.c/h.
>
> I think the i3c support can be similar to the OpenSBI i2c support:
>
> 1) Simple i3c bus library (manage and find i3c buses)
> lib/utils/i3c/i3c.c
> include/sbi_utils/i3c/i3c.h

At this moment, we only have a development board available, not the final
SoC. On the development board no I2C or I3C devices are present. There is a
Keterex I2C adapter plugged into the DW I3C port. This is how I've developed
and tested the driver.

Because on the final SoC OpenSBI won't be accessing any I3C devices, I
thought of not writing i3c.c/h module. At this moment I don't have any way
to test the API specific to I3C.

>
> 2) FDT based i3c bus (find and initialize i3c buses from FDT)
> lib/utils/i3c/fdt_i3c.c
> include/sbi_utils/i3c/fdt_i3c.h
>
> The i3c bus instance is analogous to the i2c adapter instance but the
> major difference between i2c and i3c will be the APIs or operations
> supported for devices.
>
> Also, for every i3c bus instance the i3c.c will also register a corresponding
> i2c adapter instance so that various i2c devices can be accessed via i3c
> bus instance.

I knew that DW I3C will only be used for I2C communication. However, I wasn't
sure how to use the existing fdt_i2c.c/h module and i2c_adapter. They don't
seem appropriate as I'm dealing with I3C controllers. So I went ahead and
written my own fdt_i3c.c/h. Now it looks very different than fdt_i2c.c/h
because I've implemented only necessary functions to be used only in my project
that I can test.

Also I wanted a more generic fdt_i3c module that can be used to initialize
I3C controllers from FDT. Unlike fdt_i2c module which initializes the
i2c_adapter_drivers, fdt_i3c has no knowledge about the drivers. Given a
fdt_match, fdt_i3c only parses the FDT to look for relevant controllers and
make them ready to be used by the drivers (dw_i3c in my case).

Is this approach appropriate?

>
> The i3c.c/h and fdt_i3c.c/h should only add APIs or functions which are
> needed to keep the i3c support simple and minimalistic.

Agreed. I tried to do that. However, there is no i3c.c/h in my changes.

>
> >
> > I need your feedback on whether it's a good idea to add these modules
> > to OpenSBI. If you think it's appropriate then I will send out the
> > patches. If you think it should be implemented differently, please
> > suggest that too.
>
> Sure, please send your patches to this list (using git send-email).

Will do as soon as I complete my final testing and documentation.

>
> >
> > Thank you for your time and any valuable feedback on this.
> >
> > Best Regards,
> > Adnan Rahman Chowdhury
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>
> Regards,
> Anup


From geomatsi at gmail.com  Thu Jun 23 07:13:35 2022
From: geomatsi at gmail.com (Sergey Matyukevich)
Date: Thu, 23 Jun 2022 17:13:35 +0300
Subject: [PATCH 1/1] lib: pmu: allow to use the highest available counter
Message-ID: <20220623141335.360952-1-geomatsi@gmail.com>

Both OpenSBI and OS explicitly assume that there is no hardware counter
with index 1: hardware uses that bit for TM control. So OpenSBI filters
out that index in sanity checks. However the range sanity checks do not
treat that index in a special way. As a result, OpenSBI does not allow
to use the firmware counter with the highest index. This change modifies
range checks to allow access to the highest index firmware counter.

The simple test is to make sure that there is no counter multiplexing
in the following command:

$ perf stat -e \
	r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
	r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
	r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
	r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
	ls

Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>

---

Note that perf RISC-V SBI Linux driver is adapted for the current OpenSBI
behavior: it passes counters mask with exluded highest valid index. So the
accompanying fix is also required for Linux, see the patches posted to
the risc-v mailing list:

https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c

 lib/sbi/sbi_pmu.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 3ecf536..b386d33 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -115,7 +115,7 @@ static int pmu_ctr_validate(uint32_t cidx, uint32_t *event_idx_code)
 
 	event_idx_val = active_events[hartid][cidx];
 
-	if (cidx >= total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
+	if (cidx > total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
 		return SBI_EINVAL;
 
 	event_idx_type = get_cidx_type(event_idx_val);
@@ -347,13 +347,13 @@ int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
 	int ret = SBI_EINVAL;
 	bool bUpdate = FALSE;
 
-	if (sbi_fls(ctr_mask) >= total_ctrs)
+	if (sbi_fls(ctr_mask) > total_ctrs)
 		return ret;
 
 	if (flags & SBI_PMU_START_FLAG_SET_INIT_VALUE)
 		bUpdate = TRUE;
 
-	for_each_set_bit_from(cbase, &ctr_mask, total_ctrs) {
+	for_each_set_bit_from(cbase, &ctr_mask, total_ctrs + 1) {
 		event_idx_type = pmu_ctr_validate(cbase, &event_code);
 		if (event_idx_type < 0)
 			/* Continue the start operation for other counters */
@@ -423,10 +423,10 @@ int sbi_pmu_ctr_stop(unsigned long cbase, unsigned long cmask,
 	uint32_t event_code;
 	unsigned long ctr_mask = cmask << cbase;
 
-	if (sbi_fls(ctr_mask) >= total_ctrs)
+	if (sbi_fls(ctr_mask) > total_ctrs)
 		return SBI_EINVAL;
 
-	for_each_set_bit_from(cbase, &ctr_mask, total_ctrs) {
+	for_each_set_bit_from(cbase, &ctr_mask, total_ctrs + 1) {
 		event_idx_type = pmu_ctr_validate(cbase, &event_code);
 		if (event_idx_type < 0)
 			/* Continue the stop operation for other counters */
@@ -598,7 +598,7 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask, u32 hartid)
 	else
 		fw_base = cbase;
 
-	for (i = fw_base; i < total_ctrs; i++)
+	for (i = fw_base; i <= total_ctrs; i++)
 		if ((active_events[hartid][i] == SBI_PMU_EVENT_IDX_INVALID) &&
 		    ((1UL << i) & ctr_mask))
 			return i;
@@ -618,7 +618,7 @@ int sbi_pmu_ctr_cfg_match(unsigned long cidx_base, unsigned long cidx_mask,
 	unsigned long tmp = cidx_mask << cidx_base;
 
 	/* Do a basic sanity check of counter base & mask */
-	if (sbi_fls(tmp) >= total_ctrs || event_type >= SBI_PMU_EVENT_TYPE_MAX)
+	if (sbi_fls(tmp) > total_ctrs || event_type >= SBI_PMU_EVENT_TYPE_MAX)
 		return SBI_EINVAL;
 
 	if (flags & SBI_PMU_CFG_FLAG_SKIP_MATCH) {
@@ -690,7 +690,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
 	/* Sanity check. Counter1 is not mapped at all */
-	if (cidx >= total_ctrs || cidx == 1)
+	if (cidx > total_ctrs || cidx == 1)
 		return SBI_EINVAL;
 
 	/* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
@@ -719,7 +719,7 @@ static void pmu_reset_event_map(u32 hartid)
 	int j;
 
 	/* Initialize the counter to event mapping table */
-	for (j = 3; j < total_ctrs; j++)
+	for (j = 3; j <= total_ctrs; j++)
 		active_events[hartid][j] = SBI_PMU_EVENT_IDX_INVALID;
 	for (j = 0; j < SBI_PMU_FW_CTR_MAX; j++)
 		sbi_memset(&fw_event_map[hartid][j], 0,
-- 
2.36.1



From anup at brainfault.org  Thu Jun 23 08:30:30 2022
From: anup at brainfault.org (Anup Patel)
Date: Thu, 23 Jun 2022 21:00:30 +0530
Subject: [PATCH 1/1] lib: pmu: allow to use the highest available counter
In-Reply-To: <20220623141335.360952-1-geomatsi@gmail.com>
References: <20220623141335.360952-1-geomatsi@gmail.com>
Message-ID: <CAAhSdy35CK1J+ZdMMU6ZPDA4hvcQbLe==YcO6en6-+0o-gcaYA@mail.gmail.com>

On Thu, Jun 23, 2022 at 7:43 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
>
> Both OpenSBI and OS explicitly assume that there is no hardware counter
> with index 1: hardware uses that bit for TM control. So OpenSBI filters
> out that index in sanity checks. However the range sanity checks do not
> treat that index in a special way. As a result, OpenSBI does not allow
> to use the firmware counter with the highest index. This change modifies
> range checks to allow access to the highest index firmware counter.
>
> The simple test is to make sure that there is no counter multiplexing
> in the following command:
>
> $ perf stat -e \
>         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
>         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
>         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
>         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
>         ls
>
> Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
>
> ---
>
> Note that perf RISC-V SBI Linux driver is adapted for the current OpenSBI
> behavior: it passes counters mask with exluded highest valid index. So the
> accompanying fix is also required for Linux, see the patches posted to
> the risc-v mailing list:
>
> https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
>
>  lib/sbi/sbi_pmu.c | 18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)
>
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index 3ecf536..b386d33 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -115,7 +115,7 @@ static int pmu_ctr_validate(uint32_t cidx, uint32_t *event_idx_code)
>
>         event_idx_val = active_events[hartid][cidx];
>
> -       if (cidx >= total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> +       if (cidx > total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))

Instead of changing all comparisons of total_ctrs, why not set
"num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;" in sbi_pmu_init() ?

Regards,
Anup

>                 return SBI_EINVAL;
>
>         event_idx_type = get_cidx_type(event_idx_val);
> @@ -347,13 +347,13 @@ int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
>         int ret = SBI_EINVAL;
>         bool bUpdate = FALSE;
>
> -       if (sbi_fls(ctr_mask) >= total_ctrs)
> +       if (sbi_fls(ctr_mask) > total_ctrs)
>                 return ret;
>
>         if (flags & SBI_PMU_START_FLAG_SET_INIT_VALUE)
>                 bUpdate = TRUE;
>
> -       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs) {
> +       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs + 1) {
>                 event_idx_type = pmu_ctr_validate(cbase, &event_code);
>                 if (event_idx_type < 0)
>                         /* Continue the start operation for other counters */
> @@ -423,10 +423,10 @@ int sbi_pmu_ctr_stop(unsigned long cbase, unsigned long cmask,
>         uint32_t event_code;
>         unsigned long ctr_mask = cmask << cbase;
>
> -       if (sbi_fls(ctr_mask) >= total_ctrs)
> +       if (sbi_fls(ctr_mask) > total_ctrs)
>                 return SBI_EINVAL;
>
> -       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs) {
> +       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs + 1) {
>                 event_idx_type = pmu_ctr_validate(cbase, &event_code);
>                 if (event_idx_type < 0)
>                         /* Continue the stop operation for other counters */
> @@ -598,7 +598,7 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask, u32 hartid)
>         else
>                 fw_base = cbase;
>
> -       for (i = fw_base; i < total_ctrs; i++)
> +       for (i = fw_base; i <= total_ctrs; i++)
>                 if ((active_events[hartid][i] == SBI_PMU_EVENT_IDX_INVALID) &&
>                     ((1UL << i) & ctr_mask))
>                         return i;
> @@ -618,7 +618,7 @@ int sbi_pmu_ctr_cfg_match(unsigned long cidx_base, unsigned long cidx_mask,
>         unsigned long tmp = cidx_mask << cidx_base;
>
>         /* Do a basic sanity check of counter base & mask */
> -       if (sbi_fls(tmp) >= total_ctrs || event_type >= SBI_PMU_EVENT_TYPE_MAX)
> +       if (sbi_fls(tmp) > total_ctrs || event_type >= SBI_PMU_EVENT_TYPE_MAX)
>                 return SBI_EINVAL;
>
>         if (flags & SBI_PMU_CFG_FLAG_SKIP_MATCH) {
> @@ -690,7 +690,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
>         struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
>
>         /* Sanity check. Counter1 is not mapped at all */
> -       if (cidx >= total_ctrs || cidx == 1)
> +       if (cidx > total_ctrs || cidx == 1)
>                 return SBI_EINVAL;
>
>         /* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
> @@ -719,7 +719,7 @@ static void pmu_reset_event_map(u32 hartid)
>         int j;
>
>         /* Initialize the counter to event mapping table */
> -       for (j = 3; j < total_ctrs; j++)
> +       for (j = 3; j <= total_ctrs; j++)
>                 active_events[hartid][j] = SBI_PMU_EVENT_IDX_INVALID;
>         for (j = 0; j < SBI_PMU_FW_CTR_MAX; j++)
>                 sbi_memset(&fw_event_map[hartid][j], 0,
> --
> 2.36.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From apatel at ventanamicro.com  Thu Jun 23 08:31:35 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Thu, 23 Jun 2022 21:01:35 +0530
Subject: [PATCH] include: sbi: Change spec version to 1.0
Message-ID: <20220623153135.209272-1-apatel@ventanamicro.com>

Now that SBI v1.0 specification is ratified, we change spec verion
implemented by OpenSBI to v1.0.

Signed-off-by: Anup Patel <apatel at ventanamicro.com>
---
 include/sbi/sbi_ecall.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/sbi/sbi_ecall.h b/include/sbi/sbi_ecall.h
index 9500d04..046a56a 100644
--- a/include/sbi/sbi_ecall.h
+++ b/include/sbi/sbi_ecall.h
@@ -13,8 +13,8 @@
 #include <sbi/sbi_types.h>
 #include <sbi/sbi_list.h>
 
-#define SBI_ECALL_VERSION_MAJOR		0
-#define SBI_ECALL_VERSION_MINOR		3
+#define SBI_ECALL_VERSION_MAJOR		1
+#define SBI_ECALL_VERSION_MINOR		0
 #define SBI_OPENSBI_IMPID		1
 
 struct sbi_trap_regs;
-- 
2.34.1



From atishp at atishpatra.org  Thu Jun 23 09:38:35 2022
From: atishp at atishpatra.org (Atish Patra)
Date: Thu, 23 Jun 2022 09:38:35 -0700
Subject: [PATCH] include: sbi: Change spec version to 1.0
In-Reply-To: <20220623153135.209272-1-apatel@ventanamicro.com>
References: <20220623153135.209272-1-apatel@ventanamicro.com>
Message-ID: <CAOnJCU+gV5OQCb4TzeeV5S4Q=uc_tApTVrv+Js7=osJv0CAQDA@mail.gmail.com>

On Thu, Jun 23, 2022 at 8:31 AM Anup Patel <apatel at ventanamicro.com> wrote:
>
> Now that SBI v1.0 specification is ratified, we change spec verion
> implemented by OpenSBI to v1.0.
>
> Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> ---
>  include/sbi/sbi_ecall.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/sbi/sbi_ecall.h b/include/sbi/sbi_ecall.h
> index 9500d04..046a56a 100644
> --- a/include/sbi/sbi_ecall.h
> +++ b/include/sbi/sbi_ecall.h
> @@ -13,8 +13,8 @@
>  #include <sbi/sbi_types.h>
>  #include <sbi/sbi_list.h>
>
> -#define SBI_ECALL_VERSION_MAJOR                0
> -#define SBI_ECALL_VERSION_MINOR                3
> +#define SBI_ECALL_VERSION_MAJOR                1
> +#define SBI_ECALL_VERSION_MINOR                0
>  #define SBI_OPENSBI_IMPID              1
>
>  struct sbi_trap_regs;
> --
> 2.34.1
>


Reviewed-by: Atish Patra <atishp at rivosinc.com>
-- 
Regards,
Atish


From atishp at atishpatra.org  Thu Jun 23 10:59:30 2022
From: atishp at atishpatra.org (Atish Patra)
Date: Thu, 23 Jun 2022 10:59:30 -0700
Subject: [PATCH 1/1] lib: pmu: allow to use the highest available counter
In-Reply-To: <CAAhSdy35CK1J+ZdMMU6ZPDA4hvcQbLe==YcO6en6-+0o-gcaYA@mail.gmail.com>
References: <20220623141335.360952-1-geomatsi@gmail.com> <CAAhSdy35CK1J+ZdMMU6ZPDA4hvcQbLe==YcO6en6-+0o-gcaYA@mail.gmail.com>
Message-ID: <CAOnJCUKMfPAJEGgkhT5c-EKC7XXO9tDwNY7Me5vcZm-Sbt-fcw@mail.gmail.com>

On Thu, Jun 23, 2022 at 8:31 AM Anup Patel <anup at brainfault.org> wrote:
>
> On Thu, Jun 23, 2022 at 7:43 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
> >
> > Both OpenSBI and OS explicitly assume that there is no hardware counter
> > with index 1: hardware uses that bit for TM control. So OpenSBI filters
> > out that index in sanity checks. However the range sanity checks do not
> > treat that index in a special way. As a result, OpenSBI does not allow
> > to use the firmware counter with the highest index. This change modifies
> > range checks to allow access to the highest index firmware counter.
> >
> > The simple test is to make sure that there is no counter multiplexing
> > in the following command:
> >
> > $ perf stat -e \
> >         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
> >         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
> >         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
> >         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
> >         ls
> >
> > Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> >
> > ---
> >
> > Note that perf RISC-V SBI Linux driver is adapted for the current OpenSBI
> > behavior: it passes counters mask with exluded highest valid index. So the
> > accompanying fix is also required for Linux, see the patches posted to
> > the risc-v mailing list:
> >
> > https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
> >
> >  lib/sbi/sbi_pmu.c | 18 +++++++++---------
> >  1 file changed, 9 insertions(+), 9 deletions(-)
> >
> > diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> > index 3ecf536..b386d33 100644
> > --- a/lib/sbi/sbi_pmu.c
> > +++ b/lib/sbi/sbi_pmu.c
> > @@ -115,7 +115,7 @@ static int pmu_ctr_validate(uint32_t cidx, uint32_t *event_idx_code)
> >
> >         event_idx_val = active_events[hartid][cidx];
> >
> > -       if (cidx >= total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > +       if (cidx > total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
>
> Instead of changing all comparisons of total_ctrs, why not set
> "num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;" in sbi_pmu_init() ?
>

Yes. That is simpler. We need a few other fixes along with that as
well. Here is the diff

diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 3ecf536002a4..a532746c74be 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -594,7 +594,7 @@ static int pmu_ctr_find_fw(unsigned long cbase,
unsigned long cmask, u32 hartid)
        unsigned long ctr_mask = cmask << cbase;

        if (cbase <= num_hw_ctrs)
-               fw_base = num_hw_ctrs + 1;
+               fw_base = num_hw_ctrs;
        else
                fw_base = cbase;

@@ -694,7 +694,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned
long *ctr_info)
                return SBI_EINVAL;

        /* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
-       if (cidx <= num_hw_ctrs) {
+       if (cidx < num_hw_ctrs) {
                cinfo.type = SBI_PMU_CTR_TYPE_HW;
                cinfo.csr = CSR_CYCLE + cidx;
                /* mcycle & minstret are always 64 bit */
@@ -749,7 +749,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool
cold_boot)
                sbi_platform_pmu_init(plat);

                /* mcycle & minstret is available always */
-               num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 2;
+               num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;
                total_ctrs = num_hw_ctrs + SBI_PMU_FW_CTR_MAX;
        }

As the counters are zero indexed, the firmware counter should start
from num_hw_ctrs
E.g. If a hardware has 8 programmable counters, num_hw_ctrs will be set to 11.
with counter value (0-10). Firmware counters should range from 11-26.

> Regards,
> Anup
>
> >                 return SBI_EINVAL;
> >
> >         event_idx_type = get_cidx_type(event_idx_val);
> > @@ -347,13 +347,13 @@ int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
> >         int ret = SBI_EINVAL;
> >         bool bUpdate = FALSE;
> >
> > -       if (sbi_fls(ctr_mask) >= total_ctrs)
> > +       if (sbi_fls(ctr_mask) > total_ctrs)
> >                 return ret;
> >
> >         if (flags & SBI_PMU_START_FLAG_SET_INIT_VALUE)
> >                 bUpdate = TRUE;
> >
> > -       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs) {
> > +       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs + 1) {
> >                 event_idx_type = pmu_ctr_validate(cbase, &event_code);
> >                 if (event_idx_type < 0)
> >                         /* Continue the start operation for other counters */
> > @@ -423,10 +423,10 @@ int sbi_pmu_ctr_stop(unsigned long cbase, unsigned long cmask,
> >         uint32_t event_code;
> >         unsigned long ctr_mask = cmask << cbase;
> >
> > -       if (sbi_fls(ctr_mask) >= total_ctrs)
> > +       if (sbi_fls(ctr_mask) > total_ctrs)
> >                 return SBI_EINVAL;
> >
> > -       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs) {
> > +       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs + 1) {
> >                 event_idx_type = pmu_ctr_validate(cbase, &event_code);
> >                 if (event_idx_type < 0)
> >                         /* Continue the stop operation for other counters */
> > @@ -598,7 +598,7 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask, u32 hartid)
> >         else
> >                 fw_base = cbase;
> >
> > -       for (i = fw_base; i < total_ctrs; i++)
> > +       for (i = fw_base; i <= total_ctrs; i++)
> >                 if ((active_events[hartid][i] == SBI_PMU_EVENT_IDX_INVALID) &&
> >                     ((1UL << i) & ctr_mask))
> >                         return i;
> > @@ -618,7 +618,7 @@ int sbi_pmu_ctr_cfg_match(unsigned long cidx_base, unsigned long cidx_mask,
> >         unsigned long tmp = cidx_mask << cidx_base;
> >
> >         /* Do a basic sanity check of counter base & mask */
> > -       if (sbi_fls(tmp) >= total_ctrs || event_type >= SBI_PMU_EVENT_TYPE_MAX)
> > +       if (sbi_fls(tmp) > total_ctrs || event_type >= SBI_PMU_EVENT_TYPE_MAX)
> >                 return SBI_EINVAL;
> >
> >         if (flags & SBI_PMU_CFG_FLAG_SKIP_MATCH) {
> > @@ -690,7 +690,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
> >         struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> >
> >         /* Sanity check. Counter1 is not mapped at all */
> > -       if (cidx >= total_ctrs || cidx == 1)
> > +       if (cidx > total_ctrs || cidx == 1)
> >                 return SBI_EINVAL;
> >
> >         /* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
> > @@ -719,7 +719,7 @@ static void pmu_reset_event_map(u32 hartid)
> >         int j;
> >
> >         /* Initialize the counter to event mapping table */
> > -       for (j = 3; j < total_ctrs; j++)
> > +       for (j = 3; j <= total_ctrs; j++)
> >                 active_events[hartid][j] = SBI_PMU_EVENT_IDX_INVALID;
> >         for (j = 0; j < SBI_PMU_FW_CTR_MAX; j++)
> >                 sbi_memset(&fw_event_map[hartid][j], 0,
> > --
> > 2.36.1
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



-- 
Regards,
Atish


From anup at brainfault.org  Thu Jun 23 20:38:01 2022
From: anup at brainfault.org (Anup Patel)
Date: Fri, 24 Jun 2022 09:08:01 +0530
Subject: [PATCH] include: sbi: Change spec version to 1.0
In-Reply-To: <CAOnJCU+gV5OQCb4TzeeV5S4Q=uc_tApTVrv+Js7=osJv0CAQDA@mail.gmail.com>
References: <20220623153135.209272-1-apatel@ventanamicro.com> <CAOnJCU+gV5OQCb4TzeeV5S4Q=uc_tApTVrv+Js7=osJv0CAQDA@mail.gmail.com>
Message-ID: <CAAhSdy2479rs05j8GKPvqCmnZavL0bB=ysKBnvZuO+4jvei7sA@mail.gmail.com>

On Thu, Jun 23, 2022 at 10:08 PM Atish Patra <atishp at atishpatra.org> wrote:
>
> On Thu, Jun 23, 2022 at 8:31 AM Anup Patel <apatel at ventanamicro.com> wrote:
> >
> > Now that SBI v1.0 specification is ratified, we change spec verion
> > implemented by OpenSBI to v1.0.
> >
> > Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> > ---
> >  include/sbi/sbi_ecall.h | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/include/sbi/sbi_ecall.h b/include/sbi/sbi_ecall.h
> > index 9500d04..046a56a 100644
> > --- a/include/sbi/sbi_ecall.h
> > +++ b/include/sbi/sbi_ecall.h
> > @@ -13,8 +13,8 @@
> >  #include <sbi/sbi_types.h>
> >  #include <sbi/sbi_list.h>
> >
> > -#define SBI_ECALL_VERSION_MAJOR                0
> > -#define SBI_ECALL_VERSION_MINOR                3
> > +#define SBI_ECALL_VERSION_MAJOR                1
> > +#define SBI_ECALL_VERSION_MINOR                0
> >  #define SBI_OPENSBI_IMPID              1
> >
> >  struct sbi_trap_regs;
> > --
> > 2.34.1
> >
>
>
> Reviewed-by: Atish Patra <atishp at rivosinc.com>

Applied this patch on the riscv/opensbi repo.

Thanks,
Anup


From anup at brainfault.org  Thu Jun 23 20:40:13 2022
From: anup at brainfault.org (Anup Patel)
Date: Fri, 24 Jun 2022 09:10:13 +0530
Subject: [PATCH 1/1] lib: pmu: allow to use the highest available counter
In-Reply-To: <CAOnJCUKMfPAJEGgkhT5c-EKC7XXO9tDwNY7Me5vcZm-Sbt-fcw@mail.gmail.com>
References: <20220623141335.360952-1-geomatsi@gmail.com> <CAAhSdy35CK1J+ZdMMU6ZPDA4hvcQbLe==YcO6en6-+0o-gcaYA@mail.gmail.com>
 <CAOnJCUKMfPAJEGgkhT5c-EKC7XXO9tDwNY7Me5vcZm-Sbt-fcw@mail.gmail.com>
Message-ID: <CAAhSdy2C5YMGZ6QFQvmr_e_=5_onTD0tZu+5voEcbPjKn_XY=g@mail.gmail.com>

Hi Atish and Sergey,

On Thu, Jun 23, 2022 at 11:29 PM Atish Patra <atishp at atishpatra.org> wrote:
>
> On Thu, Jun 23, 2022 at 8:31 AM Anup Patel <anup at brainfault.org> wrote:
> >
> > On Thu, Jun 23, 2022 at 7:43 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
> > >
> > > Both OpenSBI and OS explicitly assume that there is no hardware counter
> > > with index 1: hardware uses that bit for TM control. So OpenSBI filters
> > > out that index in sanity checks. However the range sanity checks do not
> > > treat that index in a special way. As a result, OpenSBI does not allow
> > > to use the firmware counter with the highest index. This change modifies
> > > range checks to allow access to the highest index firmware counter.
> > >
> > > The simple test is to make sure that there is no counter multiplexing
> > > in the following command:
> > >
> > > $ perf stat -e \
> > >         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
> > >         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
> > >         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
> > >         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
> > >         ls
> > >
> > > Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> > >
> > > ---
> > >
> > > Note that perf RISC-V SBI Linux driver is adapted for the current OpenSBI
> > > behavior: it passes counters mask with exluded highest valid index. So the
> > > accompanying fix is also required for Linux, see the patches posted to
> > > the risc-v mailing list:
> > >
> > > https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
> > >
> > >  lib/sbi/sbi_pmu.c | 18 +++++++++---------
> > >  1 file changed, 9 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> > > index 3ecf536..b386d33 100644
> > > --- a/lib/sbi/sbi_pmu.c
> > > +++ b/lib/sbi/sbi_pmu.c
> > > @@ -115,7 +115,7 @@ static int pmu_ctr_validate(uint32_t cidx, uint32_t *event_idx_code)
> > >
> > >         event_idx_val = active_events[hartid][cidx];
> > >
> > > -       if (cidx >= total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > > +       if (cidx > total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> >
> > Instead of changing all comparisons of total_ctrs, why not set
> > "num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;" in sbi_pmu_init() ?
> >
>
> Yes. That is simpler. We need a few other fixes along with that as
> well. Here is the diff
>
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index 3ecf536002a4..a532746c74be 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -594,7 +594,7 @@ static int pmu_ctr_find_fw(unsigned long cbase,
> unsigned long cmask, u32 hartid)
>         unsigned long ctr_mask = cmask << cbase;
>
>         if (cbase <= num_hw_ctrs)
> -               fw_base = num_hw_ctrs + 1;
> +               fw_base = num_hw_ctrs;
>         else
>                 fw_base = cbase;
>
> @@ -694,7 +694,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned
> long *ctr_info)
>                 return SBI_EINVAL;
>
>         /* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
> -       if (cidx <= num_hw_ctrs) {
> +       if (cidx < num_hw_ctrs) {
>                 cinfo.type = SBI_PMU_CTR_TYPE_HW;
>                 cinfo.csr = CSR_CYCLE + cidx;
>                 /* mcycle & minstret are always 64 bit */
> @@ -749,7 +749,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool
> cold_boot)
>                 sbi_platform_pmu_init(plat);
>
>                 /* mcycle & minstret is available always */
> -               num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 2;
> +               num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;
>                 total_ctrs = num_hw_ctrs + SBI_PMU_FW_CTR_MAX;
>         }
>
> As the counters are zero indexed, the firmware counter should start
> from num_hw_ctrs
> E.g. If a hardware has 8 programmable counters, num_hw_ctrs will be set to 11.
> with counter value (0-10). Firmware counters should range from 11-26.

Can one of you quickly send a v2 (preferably today) so that I can include it for
OpenSBI v1.1 release ?

Regards,
Anup

>
> > Regards,
> > Anup
> >
> > >                 return SBI_EINVAL;
> > >
> > >         event_idx_type = get_cidx_type(event_idx_val);
> > > @@ -347,13 +347,13 @@ int sbi_pmu_ctr_start(unsigned long cbase, unsigned long cmask,
> > >         int ret = SBI_EINVAL;
> > >         bool bUpdate = FALSE;
> > >
> > > -       if (sbi_fls(ctr_mask) >= total_ctrs)
> > > +       if (sbi_fls(ctr_mask) > total_ctrs)
> > >                 return ret;
> > >
> > >         if (flags & SBI_PMU_START_FLAG_SET_INIT_VALUE)
> > >                 bUpdate = TRUE;
> > >
> > > -       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs) {
> > > +       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs + 1) {
> > >                 event_idx_type = pmu_ctr_validate(cbase, &event_code);
> > >                 if (event_idx_type < 0)
> > >                         /* Continue the start operation for other counters */
> > > @@ -423,10 +423,10 @@ int sbi_pmu_ctr_stop(unsigned long cbase, unsigned long cmask,
> > >         uint32_t event_code;
> > >         unsigned long ctr_mask = cmask << cbase;
> > >
> > > -       if (sbi_fls(ctr_mask) >= total_ctrs)
> > > +       if (sbi_fls(ctr_mask) > total_ctrs)
> > >                 return SBI_EINVAL;
> > >
> > > -       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs) {
> > > +       for_each_set_bit_from(cbase, &ctr_mask, total_ctrs + 1) {
> > >                 event_idx_type = pmu_ctr_validate(cbase, &event_code);
> > >                 if (event_idx_type < 0)
> > >                         /* Continue the stop operation for other counters */
> > > @@ -598,7 +598,7 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask, u32 hartid)
> > >         else
> > >                 fw_base = cbase;
> > >
> > > -       for (i = fw_base; i < total_ctrs; i++)
> > > +       for (i = fw_base; i <= total_ctrs; i++)
> > >                 if ((active_events[hartid][i] == SBI_PMU_EVENT_IDX_INVALID) &&
> > >                     ((1UL << i) & ctr_mask))
> > >                         return i;
> > > @@ -618,7 +618,7 @@ int sbi_pmu_ctr_cfg_match(unsigned long cidx_base, unsigned long cidx_mask,
> > >         unsigned long tmp = cidx_mask << cidx_base;
> > >
> > >         /* Do a basic sanity check of counter base & mask */
> > > -       if (sbi_fls(tmp) >= total_ctrs || event_type >= SBI_PMU_EVENT_TYPE_MAX)
> > > +       if (sbi_fls(tmp) > total_ctrs || event_type >= SBI_PMU_EVENT_TYPE_MAX)
> > >                 return SBI_EINVAL;
> > >
> > >         if (flags & SBI_PMU_CFG_FLAG_SKIP_MATCH) {
> > > @@ -690,7 +690,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
> > >         struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
> > >
> > >         /* Sanity check. Counter1 is not mapped at all */
> > > -       if (cidx >= total_ctrs || cidx == 1)
> > > +       if (cidx > total_ctrs || cidx == 1)
> > >                 return SBI_EINVAL;
> > >
> > >         /* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
> > > @@ -719,7 +719,7 @@ static void pmu_reset_event_map(u32 hartid)
> > >         int j;
> > >
> > >         /* Initialize the counter to event mapping table */
> > > -       for (j = 3; j < total_ctrs; j++)
> > > +       for (j = 3; j <= total_ctrs; j++)
> > >                 active_events[hartid][j] = SBI_PMU_EVENT_IDX_INVALID;
> > >         for (j = 0; j < SBI_PMU_FW_CTR_MAX; j++)
> > >                 sbi_memset(&fw_event_map[hartid][j], 0,
> > > --
> > > 2.36.1
> > >
> > >
> > > --
> > > opensbi mailing list
> > > opensbi at lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/opensbi
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>
>
>
> --
> Regards,
> Atish


From atishp at rivosinc.com  Thu Jun 23 23:50:27 2022
From: atishp at rivosinc.com (Atish Patra)
Date: Thu, 23 Jun 2022 23:50:27 -0700
Subject: [PATCH] lib: pmu: allow to use the highest available counter
Message-ID: <20220624065028.2895176-1-atishp@rivosinc.com>

From: Sergey Matyukevich <geomatsi at gmail.com>

Both OpenSBI and OS explicitly assume that there is no hardware counter
with index 1: hardware uses that bit for TM control. So OpenSBI filters
out that index in sanity checks.
However, OpenSBI doesn't consider TM bit while returning the number of
counters. As a result, the supervisor software is aware of less number
of firmware counter.

The simple test is to make sure that there is no counter multiplexing
in the following command:

$ perf stat -e \
        r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
        r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
        r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
        r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
        ls
(16 firmware events with 16 counters won't require multiplexing)

Return accurate number of counters and update the firmware counter
starting index.

Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
Signed-off-by: Atish Patra <atishp at rivosinc.com>
---
 lib/sbi/sbi_pmu.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 3ecf536002a4..a532746c74be 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -594,7 +594,7 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask, u32 hartid)
 	unsigned long ctr_mask = cmask << cbase;
 
 	if (cbase <= num_hw_ctrs)
-		fw_base = num_hw_ctrs + 1;
+		fw_base = num_hw_ctrs;
 	else
 		fw_base = cbase;
 
@@ -694,7 +694,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
 		return SBI_EINVAL;
 
 	/* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
-	if (cidx <= num_hw_ctrs) {
+	if (cidx < num_hw_ctrs) {
 		cinfo.type = SBI_PMU_CTR_TYPE_HW;
 		cinfo.csr = CSR_CYCLE + cidx;
 		/* mcycle & minstret are always 64 bit */
@@ -749,7 +749,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 		sbi_platform_pmu_init(plat);
 
 		/* mcycle & minstret is available always */
-		num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 2;
+		num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;
 		total_ctrs = num_hw_ctrs + SBI_PMU_FW_CTR_MAX;
 	}
 
-- 
2.25.1



From geomatsi at gmail.com  Fri Jun 24 00:03:03 2022
From: geomatsi at gmail.com (Sergey Matyukevich)
Date: Fri, 24 Jun 2022 10:03:03 +0300
Subject: [PATCH] lib: pmu: allow to use the highest available counter
In-Reply-To: <20220624065028.2895176-1-atishp@rivosinc.com>
References: <20220624065028.2895176-1-atishp@rivosinc.com>
Message-ID: <YrVhp3jFrUTR/yhF@curiosity>

Hi,

> Both OpenSBI and OS explicitly assume that there is no hardware counter
> with index 1: hardware uses that bit for TM control. So OpenSBI filters
> out that index in sanity checks.
> However, OpenSBI doesn't consider TM bit while returning the number of
> counters. As a result, the supervisor software is aware of less number
> of firmware counter.
> 
> The simple test is to make sure that there is no counter multiplexing
> in the following command:
> 
> $ perf stat -e \
>         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
>         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
>         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
>         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
>         ls
> (16 firmware events with 16 counters won't require multiplexing)
> 
> Return accurate number of counters and update the firmware counter

This is not correct.

> starting index.
> 
> Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> Signed-off-by: Atish Patra <atishp at rivosinc.com>

snip

> -		num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 2;
> +		num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;
>  		total_ctrs = num_hw_ctrs + SBI_PMU_FW_CTR_MAX;
>  	}

I would suggest _not_ to merge this change in its current form. Current
approach reports incorrect number of hardware counters just to make
kernel driver provide accurate counter mask in all the subsequent
SBI requests. That is going to impact all the other SBI implementations
as well: all of them will have to pass incorrect hw counter number in
order to get correct counters mask from the kernel.

Regards,
Sergey


From geomatsi at gmail.com  Fri Jun 24 00:47:48 2022
From: geomatsi at gmail.com (Sergey Matyukevich)
Date: Fri, 24 Jun 2022 10:47:48 +0300
Subject: [PATCH 1/1] lib: pmu: allow to use the highest available counter
In-Reply-To: <CAOnJCUKMfPAJEGgkhT5c-EKC7XXO9tDwNY7Me5vcZm-Sbt-fcw@mail.gmail.com>
References: <20220623141335.360952-1-geomatsi@gmail.com>
 <CAAhSdy35CK1J+ZdMMU6ZPDA4hvcQbLe==YcO6en6-+0o-gcaYA@mail.gmail.com>
 <CAOnJCUKMfPAJEGgkhT5c-EKC7XXO9tDwNY7Me5vcZm-Sbt-fcw@mail.gmail.com>
Message-ID: <YrVsJKq4vsf8giuH@curiosity>

Hello,

> > > Both OpenSBI and OS explicitly assume that there is no hardware counter
> > > with index 1: hardware uses that bit for TM control. So OpenSBI filters
> > > out that index in sanity checks. However the range sanity checks do not
> > > treat that index in a special way. As a result, OpenSBI does not allow
> > > to use the firmware counter with the highest index. This change modifies
> > > range checks to allow access to the highest index firmware counter.
> > >
> > > The simple test is to make sure that there is no counter multiplexing
> > > in the following command:
> > >
> > > $ perf stat -e \
> > >         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
> > >         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
> > >         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
> > >         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
> > >         ls
> > >
> > > Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> > >
> > > ---
> > >
> > > Note that perf RISC-V SBI Linux driver is adapted for the current OpenSBI
> > > behavior: it passes counters mask with exluded highest valid index. So the
> > > accompanying fix is also required for Linux, see the patches posted to
> > > the risc-v mailing list:
> > >
> > > https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
> > >
> > >  lib/sbi/sbi_pmu.c | 18 +++++++++---------
> > >  1 file changed, 9 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> > > index 3ecf536..b386d33 100644
> > > --- a/lib/sbi/sbi_pmu.c
> > > +++ b/lib/sbi/sbi_pmu.c
> > > @@ -115,7 +115,7 @@ static int pmu_ctr_validate(uint32_t cidx, uint32_t *event_idx_code)
> > >
> > >         event_idx_val = active_events[hartid][cidx];
> > >
> > > -       if (cidx >= total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > > +       if (cidx > total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> >
> > Instead of changing all comparisons of total_ctrs, why not set
> > "num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;" in sbi_pmu_init() ?
> >
> 
> Yes. That is simpler. We need a few other fixes along with that as
> well. Here is the diff

Indeed, this is simpler. But as I mentioned in my reply to v2, this way
you report incorrect number of hw counters to kernel just to get the
correct mask of counters in response. And that behavior will have to
be followed by all the other SBI implementations. 

Current OpenSBI implementation implies the only gap in counters: index 1.
So my approach in v1 version of this patch was as follows:
- send the accurate number of counters to the kernel (hw + fw)
- fix range checks to take into account gap for index 1
This way OpenSBI just sends correct information to the kernel and
expects (and verifies) correct data in response. Accompanying kernel
fixes are the first two patches in the kernel series, see:
https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c

The 3rd kernel patch switches from array to IDR. With that change kernel
driver will support any gaps in counters, not only index 1. So OpenSBI
or any other SBI implementation will be able to exclude some hw or fw
counters if needed without changing kernel driver. So far I can see
two possible examples where it could be useful:
- platform specific quirks in OpenSBI to exclude non-functional counters
- keep some hw counters for internal use in OpenSBI or any other SBI implementation

Regards,
Sergey


From apatel at ventanamicro.com  Fri Jun 24 01:44:07 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 24 Jun 2022 14:14:07 +0530
Subject: [PATCH 1/1] lib: pmu: allow to use the highest available counter
In-Reply-To: <YrVsJKq4vsf8giuH@curiosity>
References: <20220623141335.360952-1-geomatsi@gmail.com> <CAAhSdy35CK1J+ZdMMU6ZPDA4hvcQbLe==YcO6en6-+0o-gcaYA@mail.gmail.com>
 <CAOnJCUKMfPAJEGgkhT5c-EKC7XXO9tDwNY7Me5vcZm-Sbt-fcw@mail.gmail.com> <YrVsJKq4vsf8giuH@curiosity>
Message-ID: <CAK9=C2UMN3g9Kwtn-L7nu0vmj7pcvDCH3MMbmN8DfcivLJQ3Ew@mail.gmail.com>

On Fri, Jun 24, 2022 at 1:18 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
>
> Hello,
>
> > > > Both OpenSBI and OS explicitly assume that there is no hardware counter
> > > > with index 1: hardware uses that bit for TM control. So OpenSBI filters
> > > > out that index in sanity checks. However the range sanity checks do not
> > > > treat that index in a special way. As a result, OpenSBI does not allow
> > > > to use the firmware counter with the highest index. This change modifies
> > > > range checks to allow access to the highest index firmware counter.
> > > >
> > > > The simple test is to make sure that there is no counter multiplexing
> > > > in the following command:
> > > >
> > > > $ perf stat -e \
> > > >         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
> > > >         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
> > > >         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
> > > >         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
> > > >         ls
> > > >
> > > > Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> > > >
> > > > ---
> > > >
> > > > Note that perf RISC-V SBI Linux driver is adapted for the current OpenSBI
> > > > behavior: it passes counters mask with exluded highest valid index. So the
> > > > accompanying fix is also required for Linux, see the patches posted to
> > > > the risc-v mailing list:
> > > >
> > > > https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
> > > >
> > > >  lib/sbi/sbi_pmu.c | 18 +++++++++---------
> > > >  1 file changed, 9 insertions(+), 9 deletions(-)
> > > >
> > > > diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> > > > index 3ecf536..b386d33 100644
> > > > --- a/lib/sbi/sbi_pmu.c
> > > > +++ b/lib/sbi/sbi_pmu.c
> > > > @@ -115,7 +115,7 @@ static int pmu_ctr_validate(uint32_t cidx, uint32_t *event_idx_code)
> > > >
> > > >         event_idx_val = active_events[hartid][cidx];
> > > >
> > > > -       if (cidx >= total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > > > +       if (cidx > total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > >
> > > Instead of changing all comparisons of total_ctrs, why not set
> > > "num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;" in sbi_pmu_init() ?
> > >
> >
> > Yes. That is simpler. We need a few other fixes along with that as
> > well. Here is the diff
>
> Indeed, this is simpler. But as I mentioned in my reply to v2, this way
> you report incorrect number of hw counters to kernel just to get the
> correct mask of counters in response. And that behavior will have to
> be followed by all the other SBI implementations.

I don't think this patch reports an incorrect number of counters because
the time CSR is indeed a counter which counts timer ticks.

Just like cycle and instret CSRs, the time CSR is also a fixed counter
which cannot count any other HW event.

The only special thing about time CSR is that as-per RISC-V privilege
specification the time CSR is always counting and there is no way to
halt/resume it because it is an alias of platform-level free-running
mtime counter.

It's totally up to the SBI implementation whether it wants to expose
fixed HW counters (cycle, time, and insret) as SBI PMU counters
or not. In fact, the SBI PMU specification defines counter_idx as
logical counter with no relation to HW counter numbering.

A perfectly valid SBI implementation is allowed to do various things:
1) Intermix HW counters and FW counters (e.g. counter_idx = 0 is
    FW counter but counter_idx = 1 is HW counter)
2) Make a particular HW counter unusable through SBI PMU due
    to some errata (e.g. If mhpmcounter3 is broken then platform
    will not set bit[3] in any of the counter-mask provided in the
   "riscv,pmu" DT node which is parsed by OpenSBI). The
   sbi_pmu_config_matching() will never select a counter_idx
   which is not part of any counter-mask provided by the platform.
3) Create on-demand mapping of counter_idx to HW counters
   because SBI implementation is doing trap-n-emulate of
   counters. This is particularly true for hypervisors like KVM
   when counters are shared between host and guest so the
   hypervisor will on-demand allocate a HW counter and map
   it to a logical counter_idx seen by the guest. (Note: same
   approach is taken by KVM ARM)

>
> Current OpenSBI implementation implies the only gap in counters: index 1.
> So my approach in v1 version of this patch was as follows:
> - send the accurate number of counters to the kernel (hw + fw)
> - fix range checks to take into account gap for index 1
> This way OpenSBI just sends correct information to the kernel and
> expects (and verifies) correct data in response. Accompanying kernel
> fixes are the first two patches in the kernel series, see:
> https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
>
> The 3rd kernel patch switches from array to IDR. With that change kernel
> driver will support any gaps in counters, not only index 1. So OpenSBI
> or any other SBI implementation will be able to exclude some hw or fw
> counters if needed without changing kernel driver. So far I can see
> two possible examples where it could be useful:
> - platform specific quirks in OpenSBI to exclude non-functional counters
> - keep some hw counters for internal use in OpenSBI or any other SBI implementation

I think there is some confusion with regards to counter_idx.

As-per SBI spec, the counter_idx is a logical number so if there are N
counters then 0 <= counter_idx < N. There are no gaps in counter_idx
numbering. Of course, an SBI implementation might intentionally never
configure a particular counter_idx because it is special or broken.

Regards,
Anup

>
> Regards,
> Sergey
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From apatel at ventanamicro.com  Fri Jun 24 01:48:42 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Fri, 24 Jun 2022 14:18:42 +0530
Subject: [PATCH] lib: pmu: allow to use the highest available counter
In-Reply-To: <YrVhp3jFrUTR/yhF@curiosity>
References: <20220624065028.2895176-1-atishp@rivosinc.com> <YrVhp3jFrUTR/yhF@curiosity>
Message-ID: <CAK9=C2XD=ViCiBNi3uWb-=ifvLWyQ8H2ZyG2m7sKGi0Wenav_Q@mail.gmail.com>

On Fri, Jun 24, 2022 at 12:33 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
>
> Hi,
>
> > Both OpenSBI and OS explicitly assume that there is no hardware counter
> > with index 1: hardware uses that bit for TM control. So OpenSBI filters
> > out that index in sanity checks.
> > However, OpenSBI doesn't consider TM bit while returning the number of
> > counters. As a result, the supervisor software is aware of less number
> > of firmware counter.
> >
> > The simple test is to make sure that there is no counter multiplexing
> > in the following command:
> >
> > $ perf stat -e \
> >         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
> >         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
> >         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
> >         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
> >         ls
> > (16 firmware events with 16 counters won't require multiplexing)
> >
> > Return accurate number of counters and update the firmware counter
>
> This is not correct.
>
> > starting index.
> >
> > Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> > Signed-off-by: Atish Patra <atishp at rivosinc.com>
>
> snip
>
> > -             num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 2;
> > +             num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;
> >               total_ctrs = num_hw_ctrs + SBI_PMU_FW_CTR_MAX;
> >       }
>
> I would suggest _not_ to merge this change in its current form. Current
> approach reports incorrect number of hardware counters just to make
> kernel driver provide accurate counter mask in all the subsequent
> SBI requests. That is going to impact all the other SBI implementations
> as well: all of them will have to pass incorrect hw counter number in
> order to get correct counters mask from the kernel.

Please my comments on the previous patch. I fail to see how this
patch impacts other SBI implementations. Can you elaborate more ?

Regards,
Anup

>
> Regards,
> Sergey
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi


From geomatsi at gmail.com  Fri Jun 24 03:01:53 2022
From: geomatsi at gmail.com (Sergey Matyukevich)
Date: Fri, 24 Jun 2022 13:01:53 +0300
Subject: [PATCH 1/1] lib: pmu: allow to use the highest available counter
In-Reply-To: <CAK9=C2UMN3g9Kwtn-L7nu0vmj7pcvDCH3MMbmN8DfcivLJQ3Ew@mail.gmail.com>
References: <20220623141335.360952-1-geomatsi@gmail.com>
 <CAAhSdy35CK1J+ZdMMU6ZPDA4hvcQbLe==YcO6en6-+0o-gcaYA@mail.gmail.com>
 <CAOnJCUKMfPAJEGgkhT5c-EKC7XXO9tDwNY7Me5vcZm-Sbt-fcw@mail.gmail.com>
 <YrVsJKq4vsf8giuH@curiosity>
 <CAK9=C2UMN3g9Kwtn-L7nu0vmj7pcvDCH3MMbmN8DfcivLJQ3Ew@mail.gmail.com>
Message-ID: <YrWLkUidwKEJZZac@curiosity>

Hi Anup,

> > > > > Both OpenSBI and OS explicitly assume that there is no hardware counter
> > > > > with index 1: hardware uses that bit for TM control. So OpenSBI filters
> > > > > out that index in sanity checks. However the range sanity checks do not
> > > > > treat that index in a special way. As a result, OpenSBI does not allow
> > > > > to use the firmware counter with the highest index. This change modifies
> > > > > range checks to allow access to the highest index firmware counter.
> > > > >
> > > > > The simple test is to make sure that there is no counter multiplexing
> > > > > in the following command:
> > > > >
> > > > > $ perf stat -e \
> > > > >         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
> > > > >         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
> > > > >         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
> > > > >         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
> > > > >         ls
> > > > >
> > > > > Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> > > > >
> > > > > ---
> > > > >
> > > > > Note that perf RISC-V SBI Linux driver is adapted for the current OpenSBI
> > > > > behavior: it passes counters mask with exluded highest valid index. So the
> > > > > accompanying fix is also required for Linux, see the patches posted to
> > > > > the risc-v mailing list:
> > > > >
> > > > > https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
> > > > >
> > > > >  lib/sbi/sbi_pmu.c | 18 +++++++++---------
> > > > >  1 file changed, 9 insertions(+), 9 deletions(-)
> > > > >
> > > > > diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> > > > > index 3ecf536..b386d33 100644
> > > > > --- a/lib/sbi/sbi_pmu.c
> > > > > +++ b/lib/sbi/sbi_pmu.c
> > > > > @@ -115,7 +115,7 @@ static int pmu_ctr_validate(uint32_t cidx, uint32_t *event_idx_code)
> > > > >
> > > > >         event_idx_val = active_events[hartid][cidx];
> > > > >
> > > > > -       if (cidx >= total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > > > > +       if (cidx > total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > > >
> > > > Instead of changing all comparisons of total_ctrs, why not set
> > > > "num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;" in sbi_pmu_init() ?
> > > >
> > >
> > > Yes. That is simpler. We need a few other fixes along with that as
> > > well. Here is the diff
> >
> > Indeed, this is simpler. But as I mentioned in my reply to v2, this way
> > you report incorrect number of hw counters to kernel just to get the
> > correct mask of counters in response. And that behavior will have to
> > be followed by all the other SBI implementations.
> 
> I don't think this patch reports an incorrect number of counters because
> the time CSR is indeed a counter which counts timer ticks.
> 
> Just like cycle and instret CSRs, the time CSR is also a fixed counter
> which cannot count any other HW event.
> 
> The only special thing about time CSR is that as-per RISC-V privilege
> specification the time CSR is always counting and there is no way to
> halt/resume it because it is an alias of platform-level free-running
> mtime counter.
> 
> It's totally up to the SBI implementation whether it wants to expose
> fixed HW counters (cycle, time, and insret) as SBI PMU counters
> or not. In fact, the SBI PMU specification defines counter_idx as
> logical counter with no relation to HW counter numbering.
> 
> A perfectly valid SBI implementation is allowed to do various things:
> 1) Intermix HW counters and FW counters (e.g. counter_idx = 0 is
>     FW counter but counter_idx = 1 is HW counter)
> 2) Make a particular HW counter unusable through SBI PMU due
>     to some errata (e.g. If mhpmcounter3 is broken then platform
>     will not set bit[3] in any of the counter-mask provided in the
>    "riscv,pmu" DT node which is parsed by OpenSBI). The
>    sbi_pmu_config_matching() will never select a counter_idx
>    which is not part of any counter-mask provided by the platform.
> 3) Create on-demand mapping of counter_idx to HW counters
>    because SBI implementation is doing trap-n-emulate of
>    counters. This is particularly true for hypervisors like KVM
>    when counters are shared between host and guest so the
>    hypervisor will on-demand allocate a HW counter and map
>    it to a logical counter_idx seen by the guest. (Note: same
>    approach is taken by KVM ARM)
> 
> >
> > Current OpenSBI implementation implies the only gap in counters: index 1.
> > So my approach in v1 version of this patch was as follows:
> > - send the accurate number of counters to the kernel (hw + fw)
> > - fix range checks to take into account gap for index 1
> > This way OpenSBI just sends correct information to the kernel and
> > expects (and verifies) correct data in response. Accompanying kernel
> > fixes are the first two patches in the kernel series, see:
> > https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
> >
> > The 3rd kernel patch switches from array to IDR. With that change kernel
> > driver will support any gaps in counters, not only index 1. So OpenSBI
> > or any other SBI implementation will be able to exclude some hw or fw
> > counters if needed without changing kernel driver. So far I can see
> > two possible examples where it could be useful:
> > - platform specific quirks in OpenSBI to exclude non-functional counters
> > - keep some hw counters for internal use in OpenSBI or any other SBI implementation
> 
> I think there is some confusion with regards to counter_idx.
> 
> As-per SBI spec, the counter_idx is a logical number so if there are N
> counters then 0 <= counter_idx < N. There are no gaps in counter_idx
> numbering. Of course, an SBI implementation might intentionally never
> configure a particular counter_idx because it is special or broken.

Thanks for clarifications! So, in brief, it is OK for valid SBI
implementation to do the following:

- to report _total_ number of counters (hw + fw), including special
  ones (like timer) or broken ones or reserved ones
- sbi_pmu_ctr_cfg_match should never send special/reserved/broken
  counters to OS

That makes sense to me. So v2 changes suggested by Atish look reasonable.
But I guess commit message needs to be clarified. 

I will shortly send you v3 version of the OpenSBI patch with updated
commit message. I will also send v2 of the kernel patches. 

Regards,
Sergey


From geomatsi at gmail.com  Fri Jun 24 04:03:30 2022
From: geomatsi at gmail.com (Sergey Matyukevich)
Date: Fri, 24 Jun 2022 14:03:30 +0300
Subject: [PATCH v3 1/1] lib: pmu: allow to use the highest available counter
Message-ID: <20220624110330.452640-1-geomatsi@gmail.com>

OpenSBI explicitly assumes that there is no pmu hardware counter with
index 1: hardware uses that bit for TM control. So OpenSBI filters
out that index in sanity checks. However OpenSBI also excludes that
counter when reports total amount of hardware counters to Linux. As
a result, Linux uses incomplete counters mask excluding the highest
available counter.

Return accurate number of counters, update the firmware counter
starting index, fix range checks that include num_hw_ctrs.

The simple test is to make sure that there is no counter multiplexing
in the following command:

$ perf stat -e \
	r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
	r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
	r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
	r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
	ls

Note that 16 firmware events with 16 counters won't require multiplexing.

Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
Signed-off-by: Atish Patra <atishp at rivosinc.com>
---

Hi all,

Tested on QEMU with and without OpenSBI firmware counters enabled.

v2 -> v3:
- Fix range checks that include num_hw_ctrs
- Update commit message

v1 -> v2:
- Rework: report accurate number of hardware counters to OS instead of
  handling timer counter in a special way. Special handling should be
  done in sbi_pmu_ctr_cfg_match

Regards,
Sergey

 lib/sbi/sbi_pmu.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
index 3ecf536..3f5fd10 100644
--- a/lib/sbi/sbi_pmu.c
+++ b/lib/sbi/sbi_pmu.c
@@ -297,7 +297,7 @@ static int pmu_ctr_start_hw(uint32_t cidx, uint64_t ival, bool ival_update)
 	unsigned long mctr_inhbt;
 
 	/* Make sure the counter index lies within the range and is not TM bit */
-	if (cidx > num_hw_ctrs || cidx == 1)
+	if (cidx >= num_hw_ctrs || cidx == 1)
 		return SBI_EINVAL;
 
 	if (sbi_hart_priv_version(scratch) < SBI_HART_PRIV_VER_1_11)
@@ -378,7 +378,7 @@ static int pmu_ctr_stop_hw(uint32_t cidx)
 	mctr_inhbt = csr_read(CSR_MCOUNTINHIBIT);
 
 	/* Make sure the counter index lies within the range and is not TM bit */
-	if (cidx > num_hw_ctrs || cidx == 1)
+	if (cidx >= num_hw_ctrs || cidx == 1)
 		return SBI_EINVAL;
 
 	if (!__test_bit(cidx, &mctr_inhbt)) {
@@ -516,7 +516,7 @@ static int pmu_ctr_find_hw(unsigned long cbase, unsigned long cmask, unsigned lo
 	u32 hartid = current_hartid();
 	struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
 
-	if (cbase > num_hw_ctrs)
+	if (cbase >= num_hw_ctrs)
 		return SBI_EINVAL;
 
 	/**
@@ -593,8 +593,8 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask, u32 hartid)
 	int fw_base;
 	unsigned long ctr_mask = cmask << cbase;
 
-	if (cbase <= num_hw_ctrs)
-		fw_base = num_hw_ctrs + 1;
+	if (cbase < num_hw_ctrs)
+		fw_base = num_hw_ctrs;
 	else
 		fw_base = cbase;
 
@@ -694,7 +694,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
 		return SBI_EINVAL;
 
 	/* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
-	if (cidx <= num_hw_ctrs) {
+	if (cidx < num_hw_ctrs) {
 		cinfo.type = SBI_PMU_CTR_TYPE_HW;
 		cinfo.csr = CSR_CYCLE + cidx;
 		/* mcycle & minstret are always 64 bit */
@@ -749,7 +749,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
 		sbi_platform_pmu_init(plat);
 
 		/* mcycle & minstret is available always */
-		num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 2;
+		num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;
 		total_ctrs = num_hw_ctrs + SBI_PMU_FW_CTR_MAX;
 	}
 
-- 
2.36.1



From anup at brainfault.org  Fri Jun 24 04:30:19 2022
From: anup at brainfault.org (Anup Patel)
Date: Fri, 24 Jun 2022 17:00:19 +0530
Subject: [PATCH 1/1] lib: pmu: allow to use the highest available counter
In-Reply-To: <YrWLkUidwKEJZZac@curiosity>
References: <20220623141335.360952-1-geomatsi@gmail.com> <CAAhSdy35CK1J+ZdMMU6ZPDA4hvcQbLe==YcO6en6-+0o-gcaYA@mail.gmail.com>
 <CAOnJCUKMfPAJEGgkhT5c-EKC7XXO9tDwNY7Me5vcZm-Sbt-fcw@mail.gmail.com>
 <YrVsJKq4vsf8giuH@curiosity> <CAK9=C2UMN3g9Kwtn-L7nu0vmj7pcvDCH3MMbmN8DfcivLJQ3Ew@mail.gmail.com>
 <YrWLkUidwKEJZZac@curiosity>
Message-ID: <CAAhSdy2p5v_vs4p7SkNDZ4UT5NrcURemQ9r2a39aLLOGXTNz7Q@mail.gmail.com>

On Fri, Jun 24, 2022 at 3:31 PM Sergey Matyukevich <geomatsi at gmail.com> wrote:
>
> Hi Anup,
>
> > > > > > Both OpenSBI and OS explicitly assume that there is no hardware counter
> > > > > > with index 1: hardware uses that bit for TM control. So OpenSBI filters
> > > > > > out that index in sanity checks. However the range sanity checks do not
> > > > > > treat that index in a special way. As a result, OpenSBI does not allow
> > > > > > to use the firmware counter with the highest index. This change modifies
> > > > > > range checks to allow access to the highest index firmware counter.
> > > > > >
> > > > > > The simple test is to make sure that there is no counter multiplexing
> > > > > > in the following command:
> > > > > >
> > > > > > $ perf stat -e \
> > > > > >         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
> > > > > >         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
> > > > > >         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
> > > > > >         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
> > > > > >         ls
> > > > > >
> > > > > > Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> > > > > >
> > > > > > ---
> > > > > >
> > > > > > Note that perf RISC-V SBI Linux driver is adapted for the current OpenSBI
> > > > > > behavior: it passes counters mask with exluded highest valid index. So the
> > > > > > accompanying fix is also required for Linux, see the patches posted to
> > > > > > the risc-v mailing list:
> > > > > >
> > > > > > https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
> > > > > >
> > > > > >  lib/sbi/sbi_pmu.c | 18 +++++++++---------
> > > > > >  1 file changed, 9 insertions(+), 9 deletions(-)
> > > > > >
> > > > > > diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> > > > > > index 3ecf536..b386d33 100644
> > > > > > --- a/lib/sbi/sbi_pmu.c
> > > > > > +++ b/lib/sbi/sbi_pmu.c
> > > > > > @@ -115,7 +115,7 @@ static int pmu_ctr_validate(uint32_t cidx, uint32_t *event_idx_code)
> > > > > >
> > > > > >         event_idx_val = active_events[hartid][cidx];
> > > > > >
> > > > > > -       if (cidx >= total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > > > > > +       if (cidx > total_ctrs || (event_idx_val == SBI_PMU_EVENT_IDX_INVALID))
> > > > >
> > > > > Instead of changing all comparisons of total_ctrs, why not set
> > > > > "num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;" in sbi_pmu_init() ?
> > > > >
> > > >
> > > > Yes. That is simpler. We need a few other fixes along with that as
> > > > well. Here is the diff
> > >
> > > Indeed, this is simpler. But as I mentioned in my reply to v2, this way
> > > you report incorrect number of hw counters to kernel just to get the
> > > correct mask of counters in response. And that behavior will have to
> > > be followed by all the other SBI implementations.
> >
> > I don't think this patch reports an incorrect number of counters because
> > the time CSR is indeed a counter which counts timer ticks.
> >
> > Just like cycle and instret CSRs, the time CSR is also a fixed counter
> > which cannot count any other HW event.
> >
> > The only special thing about time CSR is that as-per RISC-V privilege
> > specification the time CSR is always counting and there is no way to
> > halt/resume it because it is an alias of platform-level free-running
> > mtime counter.
> >
> > It's totally up to the SBI implementation whether it wants to expose
> > fixed HW counters (cycle, time, and insret) as SBI PMU counters
> > or not. In fact, the SBI PMU specification defines counter_idx as
> > logical counter with no relation to HW counter numbering.
> >
> > A perfectly valid SBI implementation is allowed to do various things:
> > 1) Intermix HW counters and FW counters (e.g. counter_idx = 0 is
> >     FW counter but counter_idx = 1 is HW counter)
> > 2) Make a particular HW counter unusable through SBI PMU due
> >     to some errata (e.g. If mhpmcounter3 is broken then platform
> >     will not set bit[3] in any of the counter-mask provided in the
> >    "riscv,pmu" DT node which is parsed by OpenSBI). The
> >    sbi_pmu_config_matching() will never select a counter_idx
> >    which is not part of any counter-mask provided by the platform.
> > 3) Create on-demand mapping of counter_idx to HW counters
> >    because SBI implementation is doing trap-n-emulate of
> >    counters. This is particularly true for hypervisors like KVM
> >    when counters are shared between host and guest so the
> >    hypervisor will on-demand allocate a HW counter and map
> >    it to a logical counter_idx seen by the guest. (Note: same
> >    approach is taken by KVM ARM)
> >
> > >
> > > Current OpenSBI implementation implies the only gap in counters: index 1.
> > > So my approach in v1 version of this patch was as follows:
> > > - send the accurate number of counters to the kernel (hw + fw)
> > > - fix range checks to take into account gap for index 1
> > > This way OpenSBI just sends correct information to the kernel and
> > > expects (and verifies) correct data in response. Accompanying kernel
> > > fixes are the first two patches in the kernel series, see:
> > > https://lore.kernel.org/linux-riscv/20220623112735.357093-1-geomatsi at gmail.com/T/#m4811c83cf767cdaaeb31e2530d4a51034fcf6c3c
> > >
> > > The 3rd kernel patch switches from array to IDR. With that change kernel
> > > driver will support any gaps in counters, not only index 1. So OpenSBI
> > > or any other SBI implementation will be able to exclude some hw or fw
> > > counters if needed without changing kernel driver. So far I can see
> > > two possible examples where it could be useful:
> > > - platform specific quirks in OpenSBI to exclude non-functional counters
> > > - keep some hw counters for internal use in OpenSBI or any other SBI implementation
> >
> > I think there is some confusion with regards to counter_idx.
> >
> > As-per SBI spec, the counter_idx is a logical number so if there are N
> > counters then 0 <= counter_idx < N. There are no gaps in counter_idx
> > numbering. Of course, an SBI implementation might intentionally never
> > configure a particular counter_idx because it is special or broken.
>
> Thanks for clarifications! So, in brief, it is OK for valid SBI
> implementation to do the following:
>
> - to report _total_ number of counters (hw + fw), including special
>   ones (like timer) or broken ones or reserved ones

Yes, as long as SBI implementation controls what counters can be
configured successfully.

> - sbi_pmu_ctr_cfg_match should never send special/reserved/broken
>   counters to OS

Yes, sbi_pmu_counter_config_matching() should:
1) Never select broken or reserved or special counters
2) Select fixed counters (such as cycle or insret) only for specific PMU event

>
> That makes sense to me. So v2 changes suggested by Atish look reasonable.
> But I guess commit message needs to be clarified.
>
> I will shortly send you v3 version of the OpenSBI patch with updated
> commit message. I will also send v2 of the kernel patches.

Sure, thanks.

Also, thanks for catching this issue.

>
> Regards,
> Sergey

Regards,
Anup


From atishp at atishpatra.org  Fri Jun 24 09:45:12 2022
From: atishp at atishpatra.org (Atish Patra)
Date: Fri, 24 Jun 2022 09:45:12 -0700
Subject: [PATCH v3 1/1] lib: pmu: allow to use the highest available counter
In-Reply-To: <20220624110330.452640-1-geomatsi@gmail.com>
References: <20220624110330.452640-1-geomatsi@gmail.com>
Message-ID: <CAOnJCUKYZc1T8AOd-KzSU92bqGE3+RDHzrk6K8d3-bedUrJojw@mail.gmail.com>

On Fri, Jun 24, 2022 at 4:04 AM Sergey Matyukevich <geomatsi at gmail.com> wrote:
>
> OpenSBI explicitly assumes that there is no pmu hardware counter with
> index 1: hardware uses that bit for TM control. So OpenSBI filters
> out that index in sanity checks. However OpenSBI also excludes that
> counter when reports total amount of hardware counters to Linux. As
> a result, Linux uses incomplete counters mask excluding the highest
> available counter.
>
> Return accurate number of counters, update the firmware counter
> starting index, fix range checks that include num_hw_ctrs.
>
> The simple test is to make sure that there is no counter multiplexing
> in the following command:
>
> $ perf stat -e \
>         r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
>         r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
>         r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
>         r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f  \
>         ls
>
> Note that 16 firmware events with 16 counters won't require multiplexing.
>
> Signed-off-by: Sergey Matyukevich <geomatsi at gmail.com>
> Signed-off-by: Atish Patra <atishp at rivosinc.com>
> ---
>
> Hi all,
>
> Tested on QEMU with and without OpenSBI firmware counters enabled.
>
> v2 -> v3:
> - Fix range checks that include num_hw_ctrs
> - Update commit message
>
> v1 -> v2:
> - Rework: report accurate number of hardware counters to OS instead of
>   handling timer counter in a special way. Special handling should be
>   done in sbi_pmu_ctr_cfg_match
>
> Regards,
> Sergey
>
>  lib/sbi/sbi_pmu.c | 14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/lib/sbi/sbi_pmu.c b/lib/sbi/sbi_pmu.c
> index 3ecf536..3f5fd10 100644
> --- a/lib/sbi/sbi_pmu.c
> +++ b/lib/sbi/sbi_pmu.c
> @@ -297,7 +297,7 @@ static int pmu_ctr_start_hw(uint32_t cidx, uint64_t ival, bool ival_update)
>         unsigned long mctr_inhbt;
>
>         /* Make sure the counter index lies within the range and is not TM bit */
> -       if (cidx > num_hw_ctrs || cidx == 1)
> +       if (cidx >= num_hw_ctrs || cidx == 1)
>                 return SBI_EINVAL;
>
>         if (sbi_hart_priv_version(scratch) < SBI_HART_PRIV_VER_1_11)
> @@ -378,7 +378,7 @@ static int pmu_ctr_stop_hw(uint32_t cidx)
>         mctr_inhbt = csr_read(CSR_MCOUNTINHIBIT);
>
>         /* Make sure the counter index lies within the range and is not TM bit */
> -       if (cidx > num_hw_ctrs || cidx == 1)
> +       if (cidx >= num_hw_ctrs || cidx == 1)
>                 return SBI_EINVAL;
>
>         if (!__test_bit(cidx, &mctr_inhbt)) {
> @@ -516,7 +516,7 @@ static int pmu_ctr_find_hw(unsigned long cbase, unsigned long cmask, unsigned lo
>         u32 hartid = current_hartid();
>         struct sbi_scratch *scratch = sbi_scratch_thishart_ptr();
>
> -       if (cbase > num_hw_ctrs)
> +       if (cbase >= num_hw_ctrs)
>                 return SBI_EINVAL;
>
>         /**
> @@ -593,8 +593,8 @@ static int pmu_ctr_find_fw(unsigned long cbase, unsigned long cmask, u32 hartid)
>         int fw_base;
>         unsigned long ctr_mask = cmask << cbase;
>
> -       if (cbase <= num_hw_ctrs)
> -               fw_base = num_hw_ctrs + 1;
> +       if (cbase < num_hw_ctrs)
> +               fw_base = num_hw_ctrs;
>         else
>                 fw_base = cbase;
>
> @@ -694,7 +694,7 @@ int sbi_pmu_ctr_get_info(uint32_t cidx, unsigned long *ctr_info)
>                 return SBI_EINVAL;
>
>         /* We have 31 HW counters with 31 being the last index(MHPMCOUNTER31) */
> -       if (cidx <= num_hw_ctrs) {
> +       if (cidx < num_hw_ctrs) {
>                 cinfo.type = SBI_PMU_CTR_TYPE_HW;
>                 cinfo.csr = CSR_CYCLE + cidx;
>                 /* mcycle & minstret are always 64 bit */
> @@ -749,7 +749,7 @@ int sbi_pmu_init(struct sbi_scratch *scratch, bool cold_boot)
>                 sbi_platform_pmu_init(plat);
>
>                 /* mcycle & minstret is available always */
> -               num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 2;
> +               num_hw_ctrs = sbi_hart_mhpm_count(scratch) + 3;
>                 total_ctrs = num_hw_ctrs + SBI_PMU_FW_CTR_MAX;
>         }
>
> --
> 2.36.1
>
>
> --
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi

Thanks for fixing the issue. LGTM.

Reviewed-by: Atish Patra <atishp at rivosinc.com>

-- 
Regards,
Atish


From anup at brainfault.org  Fri Jun 24 21:55:09 2022
From: anup at brainfault.org (Anup Patel)
Date: Sat, 25 Jun 2022 10:25:09 +0530
Subject: OpenSBI v1.1 Released
Message-ID: <CAAhSdy0axPaavha0U3VLYsDCqi3DGq=+fjDXC71nfnm_TeFYmw@mail.gmail.com>

Hi All,

The OpenSBI v1.1 is now available on GitHub at:
https://github.com/riscv-software-src/opensbi/releases

This release has:
* SBI PMU improvements
* RISC-V AIA v0.3.0 draft support
* Simple external interrupt handling framework
* Xilinx UART-Lite driver
* RISC-V privilege specification v1.12 support
* RISC-V Svpbmt extension support
* RISC-V Smstateen extension support
* RISC-V Sstc extension support
* RISC-V privilege specification version detection
* Platform callback to populate HART extensions
* Compile time C arrays support
* Probing FDT based drivers using compile time C arrays
* SBI HSM improvements
* Allwinner D1 platform support
* Trap redirection improvements related to [m|h]tinst CSR
* SBI v1.0 specification support

Overall, this release mainly adds support for various RISC-V ISA
extensions ratified in December 2021 along with other improvements.

Many thanks to everyone who has contributed to this release.

Best Regards,
Anup Patel


From apatel at ventanamicro.com  Mon Jun 27 01:46:49 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 27 Jun 2022 14:16:49 +0530
Subject: SBI Debug Console Extension Proposal (Draft v2)
Message-ID: <CAK9=C2V2WNMcV9Yb7ZbtcWokzDj1uKh3gf7dNZA091QHb2fB6A@mail.gmail.com>

Hi All,

Based on feedback, below is the updated draft proposal of the
SBI Debug Console Extension ...

The motivations behind this proposal is as follows:
1) There is no new SBI extension replacing the legacy SBI v0.1
putchar() and getchar() calls. These legacy SBI v0.1 calls are
now disabled by default in Linux and will be eventually deprecated.
We need a replacement at least for the SBI v0.1 putchar() which
is widely used for early prints in various OSes and Bootloaders.
2) The OS-A Platforms (or SEE) specification need to mandate
a particular type of UART (8250, ns16550, or SiFive) as a standard
way for boot-time early prints in supervisor-mode software. Using
SBI debug console extension, the OS-A Platforms (or SEE)
specifications don't need to mandate any type of UART.
3) The legacy SBI v0.1 putchar() only supports printing one
character at a time. For some of the hypervisors (particularly KVM),
SBI based early prints (i.e. earlycon=sbi) is slow because each
SBI v0.1 putchar() trap will be forwarded to the KVM user-space
(KVMTOOL or QEMU).

In short, the SBI debug console extension defines a standard way
for boot-time early prints in supervisor-mode software.

Please review it and provide your feedback.

Regards,
Anup

SBI Debug Console Extension (EID #0x4442434E "DBCN")
====================================================

The debug console extension defines a generic mechanism for boot-time
early prints from supervisor-mode software which allows users to catch
boot-time issues in supervisor-mode software.

This extension replaces legacy console putchar (EID #0x01) extension
and it is better in following ways:
1) It follows the new calling convention defined for SBI v1.0
   (or higher).
2) It allows supervisor software to print multiple characters
   in a single SBI call.

If the underlying physical console has extra bits for error checking
(or correction) then these extra bits should be handled by the
SBI implementation.

Function: Console Puts (FID #0)
-------------------------------

struct sbiret sbi_debug_console_puts(unsigned long addr_div_by_4,
                                     unsigned long num_chars)

Print the string specified by the `addr_div_by_4` and `num_chars`
parameters on the debug console. The `addr_div_by_4` parameter is
the physical base address of the string right shifted by 2 whereas
the `num_chars` parameter is the number of characters (or bytes) in
the string.

The memory pointed the `addr_div_by_4` and `num_chars` parameters
must be:
1) Accessible to supervisor-mode
2) Cacheable and writable memory

This is a blocking SBI call and will only return after printing
all characters of the string.

Errors:
SBI_SUCCESS                - Characters printed successfully.
SBI_ERR_INVALID_ADDRESS    - The string pointed by `addr_div_by_4`
                          and `num_chars` parameters is either not
                          accessible to supervisor-mode or does not
                          point to cacheable and writable memory.
SBI_ERR_FAILED          - Failed to print characters due to
                          I/O errors.


From xypron.glpk at gmx.de  Mon Jun 27 04:15:01 2022
From: xypron.glpk at gmx.de (Heinrich Schuchardt)
Date: Mon, 27 Jun 2022 13:15:01 +0200
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v2)
In-Reply-To: <CAK9=C2V2WNMcV9Yb7ZbtcWokzDj1uKh3gf7dNZA091QHb2fB6A@mail.gmail.com>
References: <CAK9=C2V2WNMcV9Yb7ZbtcWokzDj1uKh3gf7dNZA091QHb2fB6A@mail.gmail.com>
Message-ID: <aadb863a-f578-7740-6fff-6c962ab38ae0@gmx.de>

On 6/27/22 10:46, Anup Patel wrote:
> Hi All,
>
> Based on feedback, below is the updated draft proposal of the
> SBI Debug Console Extension ...
>
> The motivations behind this proposal is as follows:
> 1) There is no new SBI extension replacing the legacy SBI v0.1
> putchar() and getchar() calls. These legacy SBI v0.1 calls are
> now disabled by default in Linux and will be eventually deprecated.
> We need a replacement at least for the SBI v0.1 putchar() which
> is widely used for early prints in various OSes and Bootloaders.
> 2) The OS-A Platforms (or SEE) specification need to mandate
> a particular type of UART (8250, ns16550, or SiFive) as a standard
> way for boot-time early prints in supervisor-mode software. Using
> SBI debug console extension, the OS-A Platforms (or SEE)
> specifications don't need to mandate any type of UART.
> 3) The legacy SBI v0.1 putchar() only supports printing one
> character at a time. For some of the hypervisors (particularly KVM),
> SBI based early prints (i.e. earlycon=sbi) is slow because each
> SBI v0.1 putchar() trap will be forwarded to the KVM user-space
> (KVMTOOL or QEMU).
>
> In short, the SBI debug console extension defines a standard way
> for boot-time early prints in supervisor-mode software.
>
> Please review it and provide your feedback.
>
> Regards,
> Anup
>
> SBI Debug Console Extension (EID #0x4442434E "DBCN")
> ====================================================
>
> The debug console extension defines a generic mechanism for boot-time
> early prints from supervisor-mode software which allows users to catch

Why should only S-mode and not HS-mode be allowed? I think the modes
that are allowed to access this extension should be enumerated more clearly.

How are systems treated that only have M-mode and U-mode?

> boot-time issues in supervisor-mode software.
>
> This extension replaces legacy console putchar (EID #0x01) extension
> and it is better in following ways:
> 1) It follows the new calling convention defined for SBI v1.0
>     (or higher).
> 2) It allows supervisor software to print multiple characters
>     in a single SBI call.
>
> If the underlying physical console has extra bits for error checking
> (or correction) then these extra bits should be handled by the
> SBI implementation.
>
> Function: Console Puts (FID #0)
> -------------------------------
>
> struct sbiret sbi_debug_console_puts(unsigned long addr_div_by_4,
>                                       unsigned long num_chars)
>
> Print the string specified by the `addr_div_by_4` and `num_chars`
> parameters on the debug console. The `addr_div_by_4` parameter is
> the physical base address of the string right shifted by 2 whereas
> the `num_chars` parameter is the number of characters (or bytes) in
> the string.
>
> The memory pointed the `addr_div_by_4` and `num_chars` parameters
> must be:
> 1) Accessible to supervisor-mode
> 2) Cacheable and writable memory

Accessibility by supervisor-mode is required due to security
considerations: You should not be able to print out M-mode's secrets via
S-mode software. This should be clearly stated.

MUST an SBI implementation check this feature? Probably yes. How shall
it be checked?

The SBI is not meant to change the string and therefore needs no write
access. There is no reason to require that the memory should be
writable. The string could reside in NOR flash.

Caching may speed up the SBI code. But why should we require this
specific memory region being cached?

One of the replies to your v1 patch requested to have the same
capabilities as putchar(): just pass a single character in a register.
Should a second FID be added?

Best regards

Heinrich

>
> This is a blocking SBI call and will only return after printing
> all characters of the string.
>
> Errors:
> SBI_SUCCESS                - Characters printed successfully.
> SBI_ERR_INVALID_ADDRESS    - The string pointed by `addr_div_by_4`
>                            and `num_chars` parameters is either not
>                            accessible to supervisor-mode or does not
>                            point to cacheable and writable memory.
> SBI_ERR_FAILED          - Failed to print characters due to
>                            I/O errors.
>



From apatel at ventanamicro.com  Mon Jun 27 05:08:43 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 27 Jun 2022 17:38:43 +0530
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v2)
In-Reply-To: <aadb863a-f578-7740-6fff-6c962ab38ae0@gmx.de>
References: <CAK9=C2V2WNMcV9Yb7ZbtcWokzDj1uKh3gf7dNZA091QHb2fB6A@mail.gmail.com>
 <aadb863a-f578-7740-6fff-6c962ab38ae0@gmx.de>
Message-ID: <CAK9=C2VBnFQUD9v1XmxX3Oz7QE7KXgCbg453gZ-BLYRodespHg@mail.gmail.com>

On Mon, Jun 27, 2022 at 4:45 PM Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>
> On 6/27/22 10:46, Anup Patel wrote:
> > Hi All,
> >
> > Based on feedback, below is the updated draft proposal of the
> > SBI Debug Console Extension ...
> >
> > The motivations behind this proposal is as follows:
> > 1) There is no new SBI extension replacing the legacy SBI v0.1
> > putchar() and getchar() calls. These legacy SBI v0.1 calls are
> > now disabled by default in Linux and will be eventually deprecated.
> > We need a replacement at least for the SBI v0.1 putchar() which
> > is widely used for early prints in various OSes and Bootloaders.
> > 2) The OS-A Platforms (or SEE) specification need to mandate
> > a particular type of UART (8250, ns16550, or SiFive) as a standard
> > way for boot-time early prints in supervisor-mode software. Using
> > SBI debug console extension, the OS-A Platforms (or SEE)
> > specifications don't need to mandate any type of UART.
> > 3) The legacy SBI v0.1 putchar() only supports printing one
> > character at a time. For some of the hypervisors (particularly KVM),
> > SBI based early prints (i.e. earlycon=sbi) is slow because each
> > SBI v0.1 putchar() trap will be forwarded to the KVM user-space
> > (KVMTOOL or QEMU).
> >
> > In short, the SBI debug console extension defines a standard way
> > for boot-time early prints in supervisor-mode software.
> >
> > Please review it and provide your feedback.
> >
> > Regards,
> > Anup
> >
> > SBI Debug Console Extension (EID #0x4442434E "DBCN")
> > ====================================================
> >
> > The debug console extension defines a generic mechanism for boot-time
> > early prints from supervisor-mode software which allows users to catch
>
> Why should only S-mode and not HS-mode be allowed? I think the modes
> that are allowed to access this extension should be enumerated more clearly.

The term "supervisor-mode" over here means:
* HS-mode or VS-mode on systems with H-extension
* S-mode on systems without H-extension
(Please see the introduction chapter of SBI specification)

In fact, both HS-mode and VS-mode are actually supervisor-mode with
different capabilities.

>
> How are systems treated that only have M-mode and U-mode?

For systems with only M-mode and U-mode, there is no supervisor-mode
hence there is no SBI on such systems. These are embedded-class
systems which usually run some kind of RTOS.

>
> > boot-time issues in supervisor-mode software.
> >
> > This extension replaces legacy console putchar (EID #0x01) extension
> > and it is better in following ways:
> > 1) It follows the new calling convention defined for SBI v1.0
> >     (or higher).
> > 2) It allows supervisor software to print multiple characters
> >     in a single SBI call.
> >
> > If the underlying physical console has extra bits for error checking
> > (or correction) then these extra bits should be handled by the
> > SBI implementation.
> >
> > Function: Console Puts (FID #0)
> > -------------------------------
> >
> > struct sbiret sbi_debug_console_puts(unsigned long addr_div_by_4,
> >                                       unsigned long num_chars)
> >
> > Print the string specified by the `addr_div_by_4` and `num_chars`
> > parameters on the debug console. The `addr_div_by_4` parameter is
> > the physical base address of the string right shifted by 2 whereas
> > the `num_chars` parameter is the number of characters (or bytes) in
> > the string.
> >
> > The memory pointed the `addr_div_by_4` and `num_chars` parameters
> > must be:
> > 1) Accessible to supervisor-mode
> > 2) Cacheable and writable memory
>
> Accessibility by supervisor-mode is required due to security
> considerations: You should not be able to print out M-mode's secrets via
> S-mode software. This should be clearly stated.

The point#1 tries to say this.

Also, a system might be partitioned into domains so a supervisor software
running in a particular domain can only pass addresses accessible in that
domain.

>
> MUST an SBI implementation check this feature? Probably yes. How shall
> it be checked?

It's up to the SBI implementation how to check this feature.

For OpenSBI, we have domain regions so OpenSBI will check the address
against regions of the domain assigned to calling HART.

For KVM, the user-space tool (QEMU/KVMTOO) knows the guest physical
address range of Guest RAM which can be used to validate the address.

>
> The SBI is not meant to change the string and therefore needs no write
> access. There is no reason to require that the memory should be
> writable. The string could reside in NOR flash.
>
> Caching may speed up the SBI code. But why should we require this
> specific memory region being cached?

This is more a requirement on the supervisor-mode software because
we might have a system with Svpbmt so on such system supervisor-mode
should not pass an address which is mapped as non-cacheable in the
page table.

We can certainly re-word this requirement to:
"Cacheable and readable memory"

>
> One of the replies to your v1 patch requested to have the same
> capabilities as putchar(): just pass a single character in a register.
> Should a second FID be added?

It's pretty easy to print a single character using the proposed puts()
call.

For assembly source perspective, this will be just one extra instruction.

Regards,
Anup

>
> Best regards
>
> Heinrich
>
> >
> > This is a blocking SBI call and will only return after printing
> > all characters of the string.
> >
> > Errors:
> > SBI_SUCCESS                - Characters printed successfully.
> > SBI_ERR_INVALID_ADDRESS    - The string pointed by `addr_div_by_4`
> >                            and `num_chars` parameters is either not
> >                            accessible to supervisor-mode or does not
> >                            point to cacheable and writable memory.
> > SBI_ERR_FAILED          - Failed to print characters due to
> >                            I/O errors.
> >
>


From heinrich.schuchardt at canonical.com  Mon Jun 27 05:40:19 2022
From: heinrich.schuchardt at canonical.com (Heinrich Schuchardt)
Date: Mon, 27 Jun 2022 14:40:19 +0200
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v2)
In-Reply-To: <CAK9=C2VBnFQUD9v1XmxX3Oz7QE7KXgCbg453gZ-BLYRodespHg@mail.gmail.com>
References: <CAK9=C2V2WNMcV9Yb7ZbtcWokzDj1uKh3gf7dNZA091QHb2fB6A@mail.gmail.com>
 <aadb863a-f578-7740-6fff-6c962ab38ae0@gmx.de>
 <CAK9=C2VBnFQUD9v1XmxX3Oz7QE7KXgCbg453gZ-BLYRodespHg@mail.gmail.com>
Message-ID: <43c88449-13d5-7737-814e-bcbce5a8e87f@canonical.com>

On 6/27/22 14:08, Anup Patel wrote:
> On Mon, Jun 27, 2022 at 4:45 PM Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>>
>> On 6/27/22 10:46, Anup Patel wrote:
>>> Hi All,
>>>
>>> Based on feedback, below is the updated draft proposal of the
>>> SBI Debug Console Extension ...
>>>
>>> The motivations behind this proposal is as follows:
>>> 1) There is no new SBI extension replacing the legacy SBI v0.1
>>> putchar() and getchar() calls. These legacy SBI v0.1 calls are
>>> now disabled by default in Linux and will be eventually deprecated.
>>> We need a replacement at least for the SBI v0.1 putchar() which
>>> is widely used for early prints in various OSes and Bootloaders.
>>> 2) The OS-A Platforms (or SEE) specification need to mandate
>>> a particular type of UART (8250, ns16550, or SiFive) as a standard
>>> way for boot-time early prints in supervisor-mode software. Using
>>> SBI debug console extension, the OS-A Platforms (or SEE)
>>> specifications don't need to mandate any type of UART.
>>> 3) The legacy SBI v0.1 putchar() only supports printing one
>>> character at a time. For some of the hypervisors (particularly KVM),
>>> SBI based early prints (i.e. earlycon=sbi) is slow because each
>>> SBI v0.1 putchar() trap will be forwarded to the KVM user-space
>>> (KVMTOOL or QEMU).
>>>
>>> In short, the SBI debug console extension defines a standard way
>>> for boot-time early prints in supervisor-mode software.
>>>
>>> Please review it and provide your feedback.
>>>
>>> Regards,
>>> Anup
>>>
>>> SBI Debug Console Extension (EID #0x4442434E "DBCN")
>>> ====================================================
>>>
>>> The debug console extension defines a generic mechanism for boot-time
>>> early prints from supervisor-mode software which allows users to catch
>>
>> Why should only S-mode and not HS-mode be allowed? I think the modes
>> that are allowed to access this extension should be enumerated more clearly.
> 
> The term "supervisor-mode" over here means:
> * HS-mode or VS-mode on systems with H-extension
> * S-mode on systems without H-extension
> (Please see the introduction chapter of SBI specification)
> 
> In fact, both HS-mode and VS-mode are actually supervisor-mode with
> different capabilities.
> 
>>
>> How are systems treated that only have M-mode and U-mode?
> 
> For systems with only M-mode and U-mode, there is no supervisor-mode
> hence there is no SBI on such systems. These are embedded-class
> systems which usually run some kind of RTOS.
> 
>>
>>> boot-time issues in supervisor-mode software.
>>>
>>> This extension replaces legacy console putchar (EID #0x01) extension
>>> and it is better in following ways:
>>> 1) It follows the new calling convention defined for SBI v1.0
>>>      (or higher).
>>> 2) It allows supervisor software to print multiple characters
>>>      in a single SBI call.
>>>
>>> If the underlying physical console has extra bits for error checking
>>> (or correction) then these extra bits should be handled by the
>>> SBI implementation.
>>>
>>> Function: Console Puts (FID #0)
>>> -------------------------------
>>>
>>> struct sbiret sbi_debug_console_puts(unsigned long addr_div_by_4,
>>>                                        unsigned long num_chars)
>>>
>>> Print the string specified by the `addr_div_by_4` and `num_chars`
>>> parameters on the debug console. The `addr_div_by_4` parameter is
>>> the physical base address of the string right shifted by 2 whereas
>>> the `num_chars` parameter is the number of characters (or bytes) in
>>> the string.
>>>
>>> The memory pointed the `addr_div_by_4` and `num_chars` parameters
>>> must be:
>>> 1) Accessible to supervisor-mode
>>> 2) Cacheable and writable memory
>>
>> Accessibility by supervisor-mode is required due to security
>> considerations: You should not be able to print out M-mode's secrets via
>> S-mode software. This should be clearly stated.
> 
> The point#1 tries to say this.
> 
> Also, a system might be partitioned into domains so a supervisor software
> running in a particular domain can only pass addresses accessible in that
> domain.
> 
>>
>> MUST an SBI implementation check this feature? Probably yes. How shall
>> it be checked?
> 
> It's up to the SBI implementation how to check this feature.

As this is a security feature we should require:

The SBI MUST check that the full memory range is read-accessible by the 
caller.

> 
> For OpenSBI, we have domain regions so OpenSBI will check the address
> against regions of the domain assigned to calling HART.
> 
> For KVM, the user-space tool (QEMU/KVMTOO) knows the guest physical
> address range of Guest RAM which can be used to validate the address.
> 
>>
>> The SBI is not meant to change the string and therefore needs no write
>> access. There is no reason to require that the memory should be
>> writable. The string could reside in NOR flash.
>>
>> Caching may speed up the SBI code. But why should we require this
>> specific memory region being cached?
> 
> This is more a requirement on the supervisor-mode software because
> we might have a system with Svpbmt so on such system supervisor-mode
> should not pass an address which is mapped as non-cacheable in the
> page table.
> 
> We can certainly re-word this requirement to:
> "Cacheable and readable memory"

Here too we should make it clear if the SBI MUST or MAY check the 
property. I guess MAY is enough.

Best regards

Heinrich

> 
>>
>> One of the replies to your v1 patch requested to have the same
>> capabilities as putchar(): just pass a single character in a register.
>> Should a second FID be added?
> 
> It's pretty easy to print a single character using the proposed puts()
> call.
> 
> For assembly source perspective, this will be just one extra instruction.
> 
> Regards,
> Anup
> 
>>
>> Best regards
>>
>> Heinrich
>>
>>>
>>> This is a blocking SBI call and will only return after printing
>>> all characters of the string.
>>>
>>> Errors:
>>> SBI_SUCCESS                - Characters printed successfully.
>>> SBI_ERR_INVALID_ADDRESS    - The string pointed by `addr_div_by_4`
>>>                             and `num_chars` parameters is either not
>>>                             accessible to supervisor-mode or does not
>>>                             point to cacheable and writable memory.
>>> SBI_ERR_FAILED          - Failed to print characters due to
>>>                             I/O errors.
>>>
>>
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#1761): https://lists.riscv.org/g/tech-unixplatformspec/message/1761
> Mute This Topic: https://lists.riscv.org/mt/92016509/6300605
> Group Owner: tech-unixplatformspec+owner at lists.riscv.org
> Unsubscribe: https://lists.riscv.org/g/tech-unixplatformspec/unsub [heinrich.schuchardt at canonical.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 
> 



From ved at rivosinc.com  Mon Jun 27 05:46:12 2022
From: ved at rivosinc.com (Ved Shanbhogue)
Date: Mon, 27 Jun 2022 07:46:12 -0500
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v2)
In-Reply-To: <CAK9=C2VBnFQUD9v1XmxX3Oz7QE7KXgCbg453gZ-BLYRodespHg@mail.gmail.com>
References: <CAK9=C2V2WNMcV9Yb7ZbtcWokzDj1uKh3gf7dNZA091QHb2fB6A@mail.gmail.com>
 <aadb863a-f578-7740-6fff-6c962ab38ae0@gmx.de>
 <CAK9=C2VBnFQUD9v1XmxX3Oz7QE7KXgCbg453gZ-BLYRodespHg@mail.gmail.com>
Message-ID: <YrmmlAK4rQmEbkq/@ved-XPS-8940>

On Mon, Jun 27, 2022 at 05:38:43PM +0530, Anup Patel wrote:
>Also, a system might be partitioned into domains so a supervisor software
>running in a particular domain can only pass addresses accessible in that
>domain.
>
So domain is a new term which is not presently defined in the privileged 
specification or in the SBI specification. 

I think I get what you may be stating here i.e., M-mode may configure/
reconfigure PMPs to restrict the physical addresses that are accessible to 
the supervisor-mode. Perhaps a future Smpu extension may add to this mix.

To avoid inventing a new term, we could reword #1 to state that the physical 
address must be accessible, as determined by the PMP and/or PMA rules, to the
supervisor mode software invoking this SBI function. 

To further restrict this perhaps say "accessible to read"? As we may not want 
execute-only to be allowed by this extension.

>
>For OpenSBI, we have domain regions so OpenSBI will check the address
>against regions of the domain assigned to calling HART.

Since "domain" is a concept that is not defined by either the privilege
specification or the SBI specification we may want to avoid using that
term (or add a definition).

>This is more a requirement on the supervisor-mode software because
>we might have a system with Svpbmt so on such system supervisor-mode
>should not pass an address which is mapped as non-cacheable in the
>page table.
>
This is confusing since the description states that the parameter is a
physical address but the later comment states its a virtual address
and the memory type override using page-based memory type provided by
the virtual memory system is allowed. Could we clarify if its a physical
address - in which case only the coherence and cachability PMA should 
provide the memory type - or a virtual address in which case page-based 
memory type override is possible.

Further, a brief explanation about why cachability matters for this 
extension would be useful. Is this trying to imply some kind of early
boot execution where caches may be placed in a special mode that allows
operation when main memory is not available.

regards
ved


From apatel at ventanamicro.com  Mon Jun 27 06:28:15 2022
From: apatel at ventanamicro.com (Anup Patel)
Date: Mon, 27 Jun 2022 18:58:15 +0530
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v2)
In-Reply-To: <YrmmlAK4rQmEbkq/@ved-XPS-8940>
References: <CAK9=C2V2WNMcV9Yb7ZbtcWokzDj1uKh3gf7dNZA091QHb2fB6A@mail.gmail.com>
 <aadb863a-f578-7740-6fff-6c962ab38ae0@gmx.de> <CAK9=C2VBnFQUD9v1XmxX3Oz7QE7KXgCbg453gZ-BLYRodespHg@mail.gmail.com>
 <YrmmlAK4rQmEbkq/@ved-XPS-8940>
Message-ID: <CAK9=C2WEYHEo-5i6dNkxT=8owD98UHx-wJtb_oFwu-95=Pee5Q@mail.gmail.com>

On Mon, Jun 27, 2022 at 6:16 PM Ved Shanbhogue <ved at rivosinc.com> wrote:
>
> On Mon, Jun 27, 2022 at 05:38:43PM +0530, Anup Patel wrote:
> >Also, a system might be partitioned into domains so a supervisor software
> >running in a particular domain can only pass addresses accessible in that
> >domain.
> >
> So domain is a new term which is not presently defined in the privileged
> specification or in the SBI specification.

The term "domain" is not used in the proposal since it is OpenSBI specific.
My comment was mainly for Heinrich since he is already aware of OpenSBI
domains.

In general, other M-mode firmwares might also implement some kind of
system level partitioning.

>
> I think I get what you may be stating here i.e., M-mode may configure/
> reconfigure PMPs to restrict the physical addresses that are accessible to
> the supervisor-mode. Perhaps a future Smpu extension may add to this mix.
>
> To avoid inventing a new term, we could reword #1 to state that the physical
> address must be accessible, as determined by the PMP and/or PMA rules, to the
> supervisor mode software invoking this SBI function.
>
> To further restrict this perhaps say "accessible to read"? As we may not want
> execute-only to be allowed by this extension.

I mostly agree with your wording. Considering hypervisors and future memory
protection ISA extensions, let's avoid using terms PMP.

For #1, we can say:
The SBI implementation MUST check that the supervisor-mode software
is allowed to read the specified physical address range on the calling HART.

>
> >
> >For OpenSBI, we have domain regions so OpenSBI will check the address
> >against regions of the domain assigned to calling HART.
>
> Since "domain" is a concept that is not defined by either the privilege
> specification or the SBI specification we may want to avoid using that
> term (or add a definition).

The term "domain" was only in the context of OpenSBI. We should not use
this term in the SBI specification.

>
> >This is more a requirement on the supervisor-mode software because
> >we might have a system with Svpbmt so on such system supervisor-mode
> >should not pass an address which is mapped as non-cacheable in the
> >page table.
> >
> This is confusing since the description states that the parameter is a
> physical address but the later comment states its a virtual address
> and the memory type override using page-based memory type provided by
> the virtual memory system is allowed. Could we clarify if its a physical
> address - in which case only the coherence and cachability PMA should
> provide the memory type - or a virtual address in which case page-based
> memory type override is possible.
>
> Further, a brief explanation about why cachability matters for this
> extension would be useful. Is this trying to imply some kind of early
> boot execution where caches may be placed in a special mode that allows
> operation when main memory is not available.

I agree the #2 requirement is not clear enough.

For #2, we can say:
The supervisor-mode software MUST not override the memory type of
specified physical address range using page-based memory types.

The rationale is that for M-mode the memory type is always defined by
PMA(s) so if supervisor-mode software overrides memory type using
Svpbmt then M-mode firmware will not see same contents as the
supervisor-mode software.

Regards,
Anup


From ved at rivosinc.com  Mon Jun 27 06:52:42 2022
From: ved at rivosinc.com (Ved Shanbhogue)
Date: Mon, 27 Jun 2022 08:52:42 -0500
Subject: [RISC-V] [tech-unixplatformspec] SBI Debug Console Extension
 Proposal (Draft v2)
In-Reply-To: <CAK9=C2WEYHEo-5i6dNkxT=8owD98UHx-wJtb_oFwu-95=Pee5Q@mail.gmail.com>
References: <CAK9=C2V2WNMcV9Yb7ZbtcWokzDj1uKh3gf7dNZA091QHb2fB6A@mail.gmail.com>
 <aadb863a-f578-7740-6fff-6c962ab38ae0@gmx.de>
 <CAK9=C2VBnFQUD9v1XmxX3Oz7QE7KXgCbg453gZ-BLYRodespHg@mail.gmail.com>
 <YrmmlAK4rQmEbkq/@ved-XPS-8940>
 <CAK9=C2WEYHEo-5i6dNkxT=8owD98UHx-wJtb_oFwu-95=Pee5Q@mail.gmail.com>
Message-ID: <Yrm2KomP8IdT5Co3@ved-XPS-8940>

On Mon, Jun 27, 2022 at 06:58:15PM +0530, Anup Patel wrote:
>
>The rationale is that for M-mode the memory type is always defined by
>PMA(s) so if supervisor-mode software overrides memory type using
>Svpbmt then M-mode firmware will not see same contents as the
>supervisor-mode software.

I get the intent now. But we may not want to prohibit that.

We may want to document that the SBI will access this memory using
the PMA attribute. 

If the supervisor has accessed this same location using different 
cachability attribute than the PMA then a loss of coherence or 
unexpected memory ordering may occur and the invoking software 
should follow the rules and sequences defined in the Svpbmt 
specification to prevent the loss of coherence and memory ordering.

This does not place a restriction but warns against the issue and
points to the right sequence if there is a legitimate reason to 
do it.

This should not be specific to this function but applicable to any
function that the SBI defines with a memory operand and so could be
stated more generally in the SBI specification.

regards
ved


From adnan.chowdhury at sifive.com  Wed Jun 29 06:42:56 2022
From: adnan.chowdhury at sifive.com (Adnan Rahman Chowdhury)
Date: Wed, 29 Jun 2022 14:42:56 +0100
Subject: [RFC PATCH 0/1] Added a blocking wait function that waits until a certain condition is true or timeout occurs
Message-ID: <20220629134257.374341-1-adnan.chowdhury@sifive.com>

Motivation: Suppose a peripheral needs to be configured to transmit data.
There is an SFR bit which indicates that the peripheral is ready to transmit.
The firmware should check the bit and will only transmit data when the
peripheral is ready. When the firmware starts polling the SFR, the peripheral
could be busy transmitting/receiving other data so the firmware must wait
till that completes. Assuming that there is no other way, the
firmware shouldn't wait indefinitely.

This patch series addresses this problem. There is a new function that
constantly checks whether a certain condition is met and times out after a
certain amount of time.

Adnan Rahman Chowdhury (1):
  lib: sbi_timer: Added a blocking wait function that waits until a
    certain condition is true or timeout occurs

 include/sbi/sbi_timer.h |  3 +++
 lib/sbi/sbi_timer.c     | 18 ++++++++++++++++++
 2 files changed, 21 insertions(+)

-- 
2.30.2



From adnan.chowdhury at sifive.com  Wed Jun 29 06:42:57 2022
From: adnan.chowdhury at sifive.com (Adnan Rahman Chowdhury)
Date: Wed, 29 Jun 2022 14:42:57 +0100
Subject: [RFC PATCH 1/1] lib: sbi_timer: Added a blocking wait function that waits until a certain condition is true or timeout occurs
In-Reply-To: <20220629134257.374341-1-adnan.chowdhury@sifive.com>
References: <20220629134257.374341-1-adnan.chowdhury@sifive.com>
Message-ID: <20220629134257.374341-2-adnan.chowdhury@sifive.com>

Motivation: Suppose a peripheral needs to be configured to transmit data.
There is an SFR bit which indicates that the peripheral is ready to transmit.
The firmware should check the bit and will only transmit data when the
peripheral is ready. When the firmware starts polling the SFR, the peripheral
could be busy transmitting/receiving other data so the firmware must wait
till that completes. Assuming that there is no other way, the
firmware shouldn't wait indefinitely.

The function sbi_timer_waitms_until() will constantly check whether a certain
condition is met, or timeout occurs. It can be used for the cases when a
timeout is required.

Signed-off-by: Adnan Rahman Chowdhury <adnan.chowdhury at sifive.com>
---
 include/sbi/sbi_timer.h |  3 +++
 lib/sbi/sbi_timer.c     | 18 ++++++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/include/sbi/sbi_timer.h b/include/sbi/sbi_timer.h
index 63ef1af..ea3dbdd 100644
--- a/include/sbi/sbi_timer.h
+++ b/include/sbi/sbi_timer.h
@@ -48,6 +48,9 @@ static inline void sbi_timer_udelay(ulong usecs)
 	sbi_timer_delay_loop(usecs, 1000000, NULL, NULL);
 }
 
+bool sbi_timer_waitms_until(bool (*predicate)(void *), void *arg,
+			    uint64_t timeout_ms);
+
 /** Get timer value for current HART */
 u64 sbi_timer_value(void);
 
diff --git a/lib/sbi/sbi_timer.c b/lib/sbi/sbi_timer.c
index acdba92..d93036d 100644
--- a/lib/sbi/sbi_timer.c
+++ b/lib/sbi/sbi_timer.c
@@ -81,6 +81,24 @@ void sbi_timer_delay_loop(ulong units, u64 unit_freq,
 		delay_fn(opaque);
 }
 
+bool sbi_timer_waitms_until(bool(*predicate)(void*), void* arg,
+			    uint64_t timeout_ms)
+{
+	bool ret = true;
+	uint64_t start_time = sbi_timer_value();
+	uint64_t ticks =
+		(sbi_timer_get_device()->timer_freq / 1000) *
+		(timeout_ms);
+	do {
+		if (predicate(arg))
+			break;
+		if (sbi_timer_value() - start_time  >= ticks)
+			ret = false;
+	} while(ret);
+
+	return ret;
+}
+
 u64 sbi_timer_value(void)
 {
 	if (get_time_val)
-- 
2.30.2



From wxjstz at 126.com  Wed Jun 29 07:57:11 2022
From: wxjstz at 126.com (Xiang W)
Date: Wed, 29 Jun 2022 22:57:11 +0800
Subject: [RFC PATCH 1/1] lib: sbi_timer: Added a blocking wait function
 that waits until a certain condition is true or timeout occurs
In-Reply-To: <20220629134257.374341-2-adnan.chowdhury@sifive.com>
References: <20220629134257.374341-1-adnan.chowdhury@sifive.com>
	 <20220629134257.374341-2-adnan.chowdhury@sifive.com>
Message-ID: <c5251783fd9b0ff9039f1b1da56d9164e579590e.camel@126.com>

? 2022-06-29???? 14:42 +0100?Adnan Rahman Chowdhury???
> Motivation: Suppose a peripheral needs to be configured to transmit data.
> There is an SFR bit which indicates that the peripheral is ready to transmit.
> The firmware should check the bit and will only transmit data when the
> peripheral is ready. When the firmware starts polling the SFR, the peripheral
> could be busy transmitting/receiving other data so the firmware must wait
> till that completes. Assuming that there is no other way, the
> firmware shouldn't wait indefinitely.
> 
> The function sbi_timer_waitms_until() will constantly check whether a certain
> condition is met, or timeout occurs. It can be used for the cases when a
> timeout is required.
> 
> Signed-off-by: Adnan Rahman Chowdhury <adnan.chowdhury at sifive.com>
LGTM

Reviewed-by: Xiang W <wxjstz at 126.com>
> ---
> ?include/sbi/sbi_timer.h |? 3 +++
> ?lib/sbi/sbi_timer.c???? | 18 ++++++++++++++++++
> ?2 files changed, 21 insertions(+)
> 
> diff --git a/include/sbi/sbi_timer.h b/include/sbi/sbi_timer.h
> index 63ef1af..ea3dbdd 100644
> --- a/include/sbi/sbi_timer.h
> +++ b/include/sbi/sbi_timer.h
> @@ -48,6 +48,9 @@ static inline void sbi_timer_udelay(ulong usecs)
> ????????sbi_timer_delay_loop(usecs, 1000000, NULL, NULL);
> ?}
> ?
> +bool sbi_timer_waitms_until(bool (*predicate)(void *), void *arg,
> +?????????????????????????? uint64_t timeout_ms);
> +
> ?/** Get timer value for current HART */
> ?u64 sbi_timer_value(void);
> ?
> diff --git a/lib/sbi/sbi_timer.c b/lib/sbi/sbi_timer.c
> index acdba92..d93036d 100644
> --- a/lib/sbi/sbi_timer.c
> +++ b/lib/sbi/sbi_timer.c
> @@ -81,6 +81,24 @@ void sbi_timer_delay_loop(ulong units, u64 unit_freq,
> ????????????????delay_fn(opaque);
> ?}
> ?
> +bool sbi_timer_waitms_until(bool(*predicate)(void*), void* arg,
> +?????????????????????????? uint64_t timeout_ms)
> +{
> +???????bool ret = true;
> +???????uint64_t start_time = sbi_timer_value();
> +???????uint64_t ticks =
> +???????????????(sbi_timer_get_device()->timer_freq / 1000) *
> +???????????????(timeout_ms);
> +???????do {
> +???????????????if (predicate(arg))
> +???????????????????????break;
> +???????????????if (sbi_timer_value() - start_time? >= ticks)
> +???????????????????????ret = false;
> +???????} while(ret);
> +
> +???????return ret;
> +}
> +
> ?u64 sbi_timer_value(void)
> ?{
> ????????if (get_time_val)
> -- 
> 2.30.2
> 
> 




From jrtc27 at jrtc27.com  Wed Jun 29 08:04:18 2022
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Wed, 29 Jun 2022 16:04:18 +0100
Subject: [RFC PATCH 1/1] lib: sbi_timer: Added a blocking wait function
 that waits until a certain condition is true or timeout occurs
In-Reply-To: <20220629134257.374341-2-adnan.chowdhury@sifive.com>
References: <20220629134257.374341-1-adnan.chowdhury@sifive.com>
 <20220629134257.374341-2-adnan.chowdhury@sifive.com>
Message-ID: <E084D9FD-D992-487F-B699-BFE8E2C73E16@jrtc27.com>

On 29 Jun 2022, at 14:42, Adnan Rahman Chowdhury <adnan.chowdhury at sifive.com> wrote:
> 
> Motivation: Suppose a peripheral needs to be configured to transmit data.
> There is an SFR bit which indicates that the peripheral is ready to transmit.
> The firmware should check the bit and will only transmit data when the
> peripheral is ready. When the firmware starts polling the SFR, the peripheral
> could be busy transmitting/receiving other data so the firmware must wait
> till that completes. Assuming that there is no other way, the
> firmware shouldn't wait indefinitely.
> 
> The function sbi_timer_waitms_until() will constantly check whether a certain
> condition is met, or timeout occurs. It can be used for the cases when a
> timeout is required.
> 
> Signed-off-by: Adnan Rahman Chowdhury <adnan.chowdhury at sifive.com>
> ---
> include/sbi/sbi_timer.h |  3 +++
> lib/sbi/sbi_timer.c     | 18 ++++++++++++++++++
> 2 files changed, 21 insertions(+)
> 
> diff --git a/include/sbi/sbi_timer.h b/include/sbi/sbi_timer.h
> index 63ef1af..ea3dbdd 100644
> --- a/include/sbi/sbi_timer.h
> +++ b/include/sbi/sbi_timer.h
> @@ -48,6 +48,9 @@ static inline void sbi_timer_udelay(ulong usecs)
> 	sbi_timer_delay_loop(usecs, 1000000, NULL, NULL);
> }
> 
> +bool sbi_timer_waitms_until(bool (*predicate)(void *), void *arg,
> +			    uint64_t timeout_ms);
> +
> /** Get timer value for current HART */
> u64 sbi_timer_value(void);
> 
> diff --git a/lib/sbi/sbi_timer.c b/lib/sbi/sbi_timer.c
> index acdba92..d93036d 100644
> --- a/lib/sbi/sbi_timer.c
> +++ b/lib/sbi/sbi_timer.c
> @@ -81,6 +81,24 @@ void sbi_timer_delay_loop(ulong units, u64 unit_freq,
> 		delay_fn(opaque);
> }
> 
> +bool sbi_timer_waitms_until(bool(*predicate)(void*), void* arg,
> +			    uint64_t timeout_ms)

bool (*predicate)(void *) and void *arg

> +{
> +	bool ret = true;
> +	uint64_t start_time = sbi_timer_value();
> +	uint64_t ticks =
> +		(sbi_timer_get_device()->timer_freq / 1000) *
> +		(timeout_ms);

Parens around a variable?

> +	do {
> +		if (predicate(arg))
> +			break;
> +		if (sbi_timer_value() - start_time  >= ticks)

if (sbi_timer_value() - start_time >= ticks)

> +			ret = false;
> +	} while(ret);

while (ret)

Though this would be much more natural as:

while (!prediate(arg)) {
	if (sbi_timer_value() - start_time >= ticks)
		return false;
}

return true;

Or if you really want to avoid early return:

ret = true;
while (!prediate(arg)) {
	if (sbi_timer_value() - start_time >= ticks) {
		ret = false;
		break;
	}
}

return ret;

But I think early return is clearer here.

Jess

> +
> +	return ret;
> +}
> +
> u64 sbi_timer_value(void)
> {
> 	if (get_time_val)
> -- 
> 2.30.2
> 
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



From adnan.chowdhury at sifive.com  Wed Jun 29 08:30:23 2022
From: adnan.chowdhury at sifive.com (Adnan Chowdhury)
Date: Wed, 29 Jun 2022 16:30:23 +0100
Subject: [RFC PATCH 1/1] lib: sbi_timer: Added a blocking wait function
 that waits until a certain condition is true or timeout occurs
In-Reply-To: <c5251783fd9b0ff9039f1b1da56d9164e579590e.camel@126.com>
References: <20220629134257.374341-1-adnan.chowdhury@sifive.com>
 <20220629134257.374341-2-adnan.chowdhury@sifive.com> <c5251783fd9b0ff9039f1b1da56d9164e579590e.camel@126.com>
Message-ID: <CAAsT5Xr3WUY7Lf-ZsyNe4b1xRZ0krLNxNRAF9rypKGk2c97dbQ@mail.gmail.com>

Thank you Xiang.

On Wed, Jun 29, 2022 at 3:58 PM Xiang W <wxjstz at 126.com> wrote:
>
> ? 2022-06-29???? 14:42 +0100?Adnan Rahman Chowdhury???
> > Motivation: Suppose a peripheral needs to be configured to transmit data.
> > There is an SFR bit which indicates that the peripheral is ready to transmit.
> > The firmware should check the bit and will only transmit data when the
> > peripheral is ready. When the firmware starts polling the SFR, the peripheral
> > could be busy transmitting/receiving other data so the firmware must wait
> > till that completes. Assuming that there is no other way, the
> > firmware shouldn't wait indefinitely.
> >
> > The function sbi_timer_waitms_until() will constantly check whether a certain
> > condition is met, or timeout occurs. It can be used for the cases when a
> > timeout is required.
> >
> > Signed-off-by: Adnan Rahman Chowdhury <adnan.chowdhury at sifive.com>
> LGTM
>
> Reviewed-by: Xiang W <wxjstz at 126.com>
> > ---
> >  include/sbi/sbi_timer.h |  3 +++
> >  lib/sbi/sbi_timer.c     | 18 ++++++++++++++++++
> >  2 files changed, 21 insertions(+)
> >
> > diff --git a/include/sbi/sbi_timer.h b/include/sbi/sbi_timer.h
> > index 63ef1af..ea3dbdd 100644
> > --- a/include/sbi/sbi_timer.h
> > +++ b/include/sbi/sbi_timer.h
> > @@ -48,6 +48,9 @@ static inline void sbi_timer_udelay(ulong usecs)
> >         sbi_timer_delay_loop(usecs, 1000000, NULL, NULL);
> >  }
> >
> > +bool sbi_timer_waitms_until(bool (*predicate)(void *), void *arg,
> > +                           uint64_t timeout_ms);
> > +
> >  /** Get timer value for current HART */
> >  u64 sbi_timer_value(void);
> >
> > diff --git a/lib/sbi/sbi_timer.c b/lib/sbi/sbi_timer.c
> > index acdba92..d93036d 100644
> > --- a/lib/sbi/sbi_timer.c
> > +++ b/lib/sbi/sbi_timer.c
> > @@ -81,6 +81,24 @@ void sbi_timer_delay_loop(ulong units, u64 unit_freq,
> >                 delay_fn(opaque);
> >  }
> >
> > +bool sbi_timer_waitms_until(bool(*predicate)(void*), void* arg,
> > +                           uint64_t timeout_ms)
> > +{
> > +       bool ret = true;
> > +       uint64_t start_time = sbi_timer_value();
> > +       uint64_t ticks =
> > +               (sbi_timer_get_device()->timer_freq / 1000) *
> > +               (timeout_ms);
> > +       do {
> > +               if (predicate(arg))
> > +                       break;
> > +               if (sbi_timer_value() - start_time  >= ticks)
> > +                       ret = false;
> > +       } while(ret);
> > +
> > +       return ret;
> > +}
> > +
> >  u64 sbi_timer_value(void)
> >  {
> >         if (get_time_val)
> > --
> > 2.30.2
> >
> >
>
>


-- 
Best Regards,
Adnan Rahman Chowdhury


From adnan.chowdhury at sifive.com  Wed Jun 29 08:33:44 2022
From: adnan.chowdhury at sifive.com (Adnan Chowdhury)
Date: Wed, 29 Jun 2022 16:33:44 +0100
Subject: [RFC PATCH 1/1] lib: sbi_timer: Added a blocking wait function
 that waits until a certain condition is true or timeout occurs
In-Reply-To: <E084D9FD-D992-487F-B699-BFE8E2C73E16@jrtc27.com>
References: <20220629134257.374341-1-adnan.chowdhury@sifive.com>
 <20220629134257.374341-2-adnan.chowdhury@sifive.com> <E084D9FD-D992-487F-B699-BFE8E2C73E16@jrtc27.com>
Message-ID: <CAAsT5XodOzAWWJCJDzeq2RURguWg0UgNX_bo8urFvCEA6fzzRg@mail.gmail.com>

On Wed, Jun 29, 2022 at 4:04 PM Jessica Clarke <jrtc27 at jrtc27.com> wrote:
>
> On 29 Jun 2022, at 14:42, Adnan Rahman Chowdhury <adnan.chowdhury at sifive.com> wrote:
> >
> > Motivation: Suppose a peripheral needs to be configured to transmit data.
> > There is an SFR bit which indicates that the peripheral is ready to transmit.
> > The firmware should check the bit and will only transmit data when the
> > peripheral is ready. When the firmware starts polling the SFR, the peripheral
> > could be busy transmitting/receiving other data so the firmware must wait
> > till that completes. Assuming that there is no other way, the
> > firmware shouldn't wait indefinitely.
> >
> > The function sbi_timer_waitms_until() will constantly check whether a certain
> > condition is met, or timeout occurs. It can be used for the cases when a
> > timeout is required.
> >
> > Signed-off-by: Adnan Rahman Chowdhury <adnan.chowdhury at sifive.com>
> > ---
> > include/sbi/sbi_timer.h |  3 +++
> > lib/sbi/sbi_timer.c     | 18 ++++++++++++++++++
> > 2 files changed, 21 insertions(+)
> >
> > diff --git a/include/sbi/sbi_timer.h b/include/sbi/sbi_timer.h
> > index 63ef1af..ea3dbdd 100644
> > --- a/include/sbi/sbi_timer.h
> > +++ b/include/sbi/sbi_timer.h
> > @@ -48,6 +48,9 @@ static inline void sbi_timer_udelay(ulong usecs)
> >       sbi_timer_delay_loop(usecs, 1000000, NULL, NULL);
> > }
> >
> > +bool sbi_timer_waitms_until(bool (*predicate)(void *), void *arg,
> > +                         uint64_t timeout_ms);
> > +
> > /** Get timer value for current HART */
> > u64 sbi_timer_value(void);
> >
> > diff --git a/lib/sbi/sbi_timer.c b/lib/sbi/sbi_timer.c
> > index acdba92..d93036d 100644
> > --- a/lib/sbi/sbi_timer.c
> > +++ b/lib/sbi/sbi_timer.c
> > @@ -81,6 +81,24 @@ void sbi_timer_delay_loop(ulong units, u64 unit_freq,
> >               delay_fn(opaque);
> > }
> >
> > +bool sbi_timer_waitms_until(bool(*predicate)(void*), void* arg,
> > +                         uint64_t timeout_ms)
>
> bool (*predicate)(void *) and void *arg

Thanks for pointing this out.

>
> > +{
> > +     bool ret = true;
> > +     uint64_t start_time = sbi_timer_value();
> > +     uint64_t ticks =
> > +             (sbi_timer_get_device()->timer_freq / 1000) *
> > +             (timeout_ms);
>
> Parens around a variable?

This was actually a macro in the project I'm working on. Later I moved
it to be a function, looks suitable for sbi_timer. The parentheses came
from the macro. Will remove it.

>
> > +     do {
> > +             if (predicate(arg))
> > +                     break;
> > +             if (sbi_timer_value() - start_time  >= ticks)
>
> if (sbi_timer_value() - start_time >= ticks)
>
> > +                     ret = false;
> > +     } while(ret);
>
> while (ret)
>
> Though this would be much more natural as:
>
> while (!prediate(arg)) {
>         if (sbi_timer_value() - start_time >= ticks)
>                 return false;
> }
>
> return true;

This version is cleaner. Will update the patch. Thank you.

>
> Or if you really want to avoid early return:
>
> ret = true;
> while (!prediate(arg)) {
>         if (sbi_timer_value() - start_time >= ticks) {
>                 ret = false;
>                 break;
>         }
> }
>
> return ret;
>
> But I think early return is clearer here.
>
> Jess
>
> > +
> > +     return ret;
> > +}
> > +
> > u64 sbi_timer_value(void)
> > {
> >       if (get_time_val)
> > --
> > 2.30.2
> >
> >
> > --
> > opensbi mailing list
> > opensbi at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/opensbi
>

--
Best Regards,
Adnan Rahman Chowdhury


From adnan.chowdhury at sifive.com  Thu Jun 30 02:46:34 2022
From: adnan.chowdhury at sifive.com (Adnan Rahman Chowdhury)
Date: Thu, 30 Jun 2022 10:46:34 +0100
Subject: [PATCH v2 0/1] Added a blocking wait function that waits until a certain condition is satisfied or timeout occurs
Message-ID: <20220630094635.389645-1-adnan.chowdhury@sifive.com>

Motivation: Suppose a peripheral needs to be configured to transmit data.
There is an SFR bit which indicates that the peripheral is ready to transmit.
The firmware should check the bit and will only transmit data when the
peripheral is ready. When the firmware starts polling the SFR, the peripheral
could be busy transmitting/receiving other data so the firmware must wait
till that completes. Assuming that there is no other way, the firmware
shouldn't wait indefinitely.

This patch series addresses this problem. There is a new function that
constantly checks whether a certain condition is met and times out after a
certain amount of time.

Changes in v2:
  - Code refactored to make the implementation cleaner.
  - Added documentation for the API.

Adnan Rahman Chowdhury (1):
  lib: sbi_timer: Added a blocking wait function that waits until a
    certain condition is satisfied or timeout occurs

 include/sbi/sbi_timer.h | 18 ++++++++++++++++++
 lib/sbi/sbi_timer.c     | 13 +++++++++++++
 2 files changed, 31 insertions(+)

-- 
2.30.2



From adnan.chowdhury at sifive.com  Thu Jun 30 02:46:35 2022
From: adnan.chowdhury at sifive.com (Adnan Rahman Chowdhury)
Date: Thu, 30 Jun 2022 10:46:35 +0100
Subject: [PATCH v2 1/1] lib: sbi_timer: Added a blocking wait function that waits until a certain condition is satisfied or timeout occurs
In-Reply-To: <20220630094635.389645-1-adnan.chowdhury@sifive.com>
References: <20220630094635.389645-1-adnan.chowdhury@sifive.com>
Message-ID: <20220630094635.389645-2-adnan.chowdhury@sifive.com>

Motivation: Suppose a peripheral needs to be configured to transmit
data. There is an SFR bit which indicates that the peripheral is ready
to transmit. The firmware should check the bit and will only transmit
data when the peripheral is ready. When the firmware starts polling the
SFR, the peripheral could be busy transmitting/receiving other data so
the firmware must wait till that completes. Assuming that there is no
other way, the firmware shouldn't wait indefinitely.

The function sbi_timer_waitms_until() will constantly check whether a
certain condition is satisfied, or timeout occurs. It can be used for
the cases when a timeout is required.

Signed-off-by: Adnan Rahman Chowdhury <adnan.chowdhury at sifive.com>
Reviewed-by: Xiang W <wxjstz at 126.com>
---
 include/sbi/sbi_timer.h | 18 ++++++++++++++++++
 lib/sbi/sbi_timer.c     | 13 +++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/include/sbi/sbi_timer.h b/include/sbi/sbi_timer.h
index 63ef1af..ac48e2b 100644
--- a/include/sbi/sbi_timer.h
+++ b/include/sbi/sbi_timer.h
@@ -48,6 +48,24 @@ static inline void sbi_timer_udelay(ulong usecs)
 	sbi_timer_delay_loop(usecs, 1000000, NULL, NULL);
 }
 
+/**
+ * A blocking function that will wait until @p predicate returns true or
+ * @p timeout_ms milliseconds elapsed. @p arg will be passed as argument to
+ * @p predicate function.
+ *
+ * @param predicate Pointer to a function that returns true if certain
+ * condition is met. It shouldn't block the code execution.
+ * @param arg Argument to pass to @p predicate.
+ * @param timeout_ms Timeout value in milliseconds. The function will return
+ * false if @p timeout_ms time period elapsed but still @p predicate doesn't
+ * return true.
+ *
+ * @return true if @p predicate returns true within @p timeout_ms, false
+ * otherwise.
+ */
+bool sbi_timer_waitms_until(bool (*predicate)(void *), void *arg,
+			    uint64_t timeout_ms);
+
 /** Get timer value for current HART */
 u64 sbi_timer_value(void);
 
diff --git a/lib/sbi/sbi_timer.c b/lib/sbi/sbi_timer.c
index acdba92..649a4e6 100644
--- a/lib/sbi/sbi_timer.c
+++ b/lib/sbi/sbi_timer.c
@@ -81,6 +81,19 @@ void sbi_timer_delay_loop(ulong units, u64 unit_freq,
 		delay_fn(opaque);
 }
 
+bool sbi_timer_waitms_until(bool (*predicate)(void *), void *arg,
+			    uint64_t timeout_ms)
+{
+	uint64_t start_time = sbi_timer_value();
+	uint64_t ticks =
+		(sbi_timer_get_device()->timer_freq / 1000) *
+		timeout_ms;
+	while(!predicate(arg))
+		if (sbi_timer_value() - start_time  >= ticks)
+			return false;
+	return true;
+}
+
 u64 sbi_timer_value(void)
 {
 	if (get_time_val)
-- 
2.30.2



From ptprasanna at gmail.com  Thu Jun 30 22:04:04 2022
From: ptprasanna at gmail.com (Prasanna T)
Date: Fri,  1 Jul 2022 10:34:04 +0530
Subject: [PATCH] Changes specific to Shakti uart
Message-ID: <20220701050404.12688-1-ptprasanna@gmail.com>

---
 include/sbi_utils/serial/shakti-uart.h | 4 ++++
 lib/utils/fdt/fdt_helper.c             | 2 +-
 lib/utils/serial/shakti-uart.c         | 7 ++++---
 3 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/include/sbi_utils/serial/shakti-uart.h b/include/sbi_utils/serial/shakti-uart.h
index bcb019e..08043be 100644
--- a/include/sbi_utils/serial/shakti-uart.h
+++ b/include/sbi_utils/serial/shakti-uart.h
@@ -9,6 +9,10 @@
 
 #include <sbi/sbi_types.h>
 
+void shakti_uart_putc(char ch);
+
+int shakti_uart_getc(void);
+
 int shakti_uart_init(unsigned long base, u32 in_freq, u32 baudrate);
 
 #endif
diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
index 2bdb1ef..40597e7 100644
--- a/lib/utils/fdt/fdt_helper.c
+++ b/lib/utils/fdt/fdt_helper.c
@@ -31,7 +31,7 @@
 #define DEFAULT_GAISLER_UART_REG_IO_WIDTH	4
 
 #define DEFAULT_SHAKTI_UART_FREQ		50000000
-#define DEFAULT_SHAKTI_UART_BAUD		115200
+#define DEFAULT_SHAKTI_UART_BAUD		19200
 
 const struct fdt_match *fdt_match_node(void *fdt, int nodeoff,
 				       const struct fdt_match *match_table)
diff --git a/lib/utils/serial/shakti-uart.c b/lib/utils/serial/shakti-uart.c
index 5f2fe75..d3fd5f1 100644
--- a/lib/utils/serial/shakti-uart.c
+++ b/lib/utils/serial/shakti-uart.c
@@ -20,20 +20,21 @@
 
 #define UART_TX_FULL  0x2
 #define UART_RX_FULL  0x8
+#define UART_RX_NOT_EMPTY 0x4
 
 static volatile char *uart_base;
 
 static void shakti_uart_putc(char ch)
 {
-	while((readw(uart_base + REG_STATUS) & UART_TX_FULL))
+	while((readb(uart_base + REG_STATUS) & UART_TX_FULL))
 		;
 	writeb(ch, uart_base + REG_TX);
 }
 
 static int shakti_uart_getc(void)
 {
-	u16 status = readw(uart_base + REG_STATUS);
-	if (status & UART_RX_FULL)
+	u16 status = readb(uart_base + REG_STATUS);
+	if (status & UART_RX_NOT_EMPTY) // MASKING 3rd bit for UARTv2. If UARTv1 use MASK UART_RX_FULL.
 		return readb(uart_base + REG_RX);
 	return -1;
 }
-- 
2.17.1



From jrtc27 at jrtc27.com  Thu Jun 30 22:42:00 2022
From: jrtc27 at jrtc27.com (Jessica Clarke)
Date: Fri, 1 Jul 2022 06:42:00 +0100
Subject: [PATCH] Changes specific to Shakti uart
In-Reply-To: <20220701050404.12688-1-ptprasanna@gmail.com>
References: <20220701050404.12688-1-ptprasanna@gmail.com>
Message-ID: <5AA321C6-BB01-4E2E-824C-9B640095E6AC@jrtc27.com>

On 1 Jul 2022, at 06:04, Prasanna T <ptprasanna at gmail.com> wrote:
> 

No commit message and a vague subject.

> ---
> include/sbi_utils/serial/shakti-uart.h | 4 ++++
> lib/utils/fdt/fdt_helper.c             | 2 +-
> lib/utils/serial/shakti-uart.c         | 7 ++++---
> 3 files changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/include/sbi_utils/serial/shakti-uart.h b/include/sbi_utils/serial/shakti-uart.h
> index bcb019e..08043be 100644
> --- a/include/sbi_utils/serial/shakti-uart.h
> +++ b/include/sbi_utils/serial/shakti-uart.h
> @@ -9,6 +9,10 @@
> 
> #include <sbi/sbi_types.h>
> 
> +void shakti_uart_putc(char ch);
> +
> +int shakti_uart_getc(void);
> +

These are wrong to add.

> int shakti_uart_init(unsigned long base, u32 in_freq, u32 baudrate);
> 
> #endif
> diff --git a/lib/utils/fdt/fdt_helper.c b/lib/utils/fdt/fdt_helper.c
> index 2bdb1ef..40597e7 100644
> --- a/lib/utils/fdt/fdt_helper.c
> +++ b/lib/utils/fdt/fdt_helper.c
> @@ -31,7 +31,7 @@
> #define DEFAULT_GAISLER_UART_REG_IO_WIDTH	4
> 
> #define DEFAULT_SHAKTI_UART_FREQ		50000000
> -#define DEFAULT_SHAKTI_UART_BAUD		115200
> +#define DEFAULT_SHAKTI_UART_BAUD		19200

Why change this?

> const struct fdt_match *fdt_match_node(void *fdt, int nodeoff,
> 				       const struct fdt_match *match_table)
> diff --git a/lib/utils/serial/shakti-uart.c b/lib/utils/serial/shakti-uart.c
> index 5f2fe75..d3fd5f1 100644
> --- a/lib/utils/serial/shakti-uart.c
> +++ b/lib/utils/serial/shakti-uart.c
> @@ -20,20 +20,21 @@
> 
> #define UART_TX_FULL  0x2
> #define UART_RX_FULL  0x8
> +#define UART_RX_NOT_EMPTY 0x4

Keep these sorted by their value.

> static volatile char *uart_base;
> 
> static void shakti_uart_putc(char ch)
> {
> -	while((readw(uart_base + REG_STATUS) & UART_TX_FULL))
> +	while((readb(uart_base + REG_STATUS) & UART_TX_FULL))

Why does this need changing? Was it always broken or did something
change?

> 		;
> 	writeb(ch, uart_base + REG_TX);
> }
> 
> static int shakti_uart_getc(void)
> {
> -	u16 status = readw(uart_base + REG_STATUS);
> -	if (status & UART_RX_FULL)
> +	u16 status = readb(uart_base + REG_STATUS);

As with putc.

> +	if (status & UART_RX_NOT_EMPTY) // MASKING 3rd bit for UARTv2. If UARTv1 use MASK UART_RX_FULL.

So this seems to explain the changes, UARTv2 needs a different driver,
so don?t break UARTv1, extend this driver to support both
simultaneously?

Jess

> 		return readb(uart_base + REG_RX);
> 	return -1;
> }
> -- 
> 2.17.1
> 
> 
> -- 
> opensbi mailing list
> opensbi at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/opensbi



