[PATCH v11 12/14] irqchip/riscv-aplic: Add support for MSI-mode

Anup Patel anup at brainfault.org
Tue Nov 14 01:21:46 PST 2023


On Wed, Nov 8, 2023 at 9:02 PM Ben <figure1802 at 126.com> wrote:
>
>
> At 2023-11-08 22:56:59, "Anup Patel" <apatel at ventanamicro.com> wrote:
> >On Wed, Nov 8, 2023 at 8:23 PM Ben <figure1802 at 126.com> wrote:
> >>
> >> At 2023-11-08 22:43:25, "Anup Patel" <apatel at ventanamicro.com> wrote:
> >> >On Sat, Nov 4, 2023 at 6:30 AM Ben <figure1802 at 126.com> wrote:
> >> >>
> >> >> At 2023-10-24 01:27:58, "Anup Patel" <apatel at ventanamicro.com> wrote:
> >> >> >The RISC-V advanced platform-level interrupt controller (APLIC) has
> >> >> >two modes of operation: 1) Direct mode and 2) MSI mode.
> >> >> >(For more details, refer https://github.com/riscv/riscv-aia)
> >> >> >
> >> >> >In APLIC MSI-mode, wired interrupts are forwared as message signaled
> >> >> >interrupts (MSIs) to CPUs via IMSIC.
> >> >> >
> >> >> >We extend the existing APLIC irqchip driver to support MSI-mode for
> >> >> >RISC-V platforms having both wired interrupts and MSIs.
> >> >> >
> >> >> >Signed-off-by: Anup Patel <apatel at ventanamicro.com>
> >> >> >---
> >> >> > drivers/irqchip/Kconfig                |   6 +
> >> >> > drivers/irqchip/Makefile               |   1 +
> >> >> > drivers/irqchip/irq-riscv-aplic-main.c |   2 +-
> >> >> > drivers/irqchip/irq-riscv-aplic-main.h |   8 +
> >> >> > drivers/irqchip/irq-riscv-aplic-msi.c  | 285 +++++++++++++++++++++++++
> >> >> > 5 files changed, 301 insertions(+), 1 deletion(-)
> >> >> > create mode 100644 drivers/irqchip/irq-riscv-aplic-msi.c
> >> >> >
> >> >> >diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> >> >> >index 1996cc6f666a..7adc4dbe07ff 100644
> >> >> >--- a/drivers/irqchip/Kconfig
> >> >> >+++ b/drivers/irqchip/Kconfig
> >> >> >@@ -551,6 +551,12 @@ config RISCV_APLIC
> >> >> >       depends on RISCV
> >> >> >       select IRQ_DOMAIN_HIERARCHY
> >> >> >
> >> >> >+config RISCV_APLIC_MSI
> >> >> >+      bool
> >> >> >+      depends on RISCV_APLIC
> >> >> >+      select GENERIC_MSI_IRQ
> >> >> >+      default RISCV_APLIC
> >> >> >+
> >> >> > config RISCV_IMSIC
> >> >> >       bool
> >> >> >       depends on RISCV
> >> >> >diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> >> >> >index 7f8289790ed8..47995fdb2c60 100644
> >> >> >--- a/drivers/irqchip/Makefile
> >> >> >+++ b/drivers/irqchip/Makefile
> >> >> >@@ -96,6 +96,7 @@ obj-$(CONFIG_CSKY_MPINTC)            += irq-csky-mpintc.o
> >> >> > obj-$(CONFIG_CSKY_APB_INTC)           += irq-csky-apb-intc.o
> >> >> > obj-$(CONFIG_RISCV_INTC)              += irq-riscv-intc.o
> >> >> > obj-$(CONFIG_RISCV_APLIC)             += irq-riscv-aplic-main.o irq-riscv-aplic-direct.o
> >> >> >+obj-$(CONFIG_RISCV_APLIC_MSI)         += irq-riscv-aplic-msi.o
> >> >> > obj-$(CONFIG_RISCV_IMSIC)             += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o
> >> >> > obj-$(CONFIG_SIFIVE_PLIC)             += irq-sifive-plic.o
> >> >> > obj-$(CONFIG_IMX_IRQSTEER)            += irq-imx-irqsteer.o
> >> >> >diff --git a/drivers/irqchip/irq-riscv-aplic-main.c b/drivers/irqchip/irq-riscv-aplic-main.c
> >> >> >index 87450708a733..d1b342b66551 100644
> >> >> >--- a/drivers/irqchip/irq-riscv-aplic-main.c
> >> >> >+++ b/drivers/irqchip/irq-riscv-aplic-main.c
> >> >> >@@ -205,7 +205,7 @@ static int aplic_probe(struct platform_device *pdev)
> >> >> >               msi_mode = of_property_present(to_of_node(dev->fwnode),
> >> >> >                                               "msi-parent");
> >> >> >       if (msi_mode)
> >> >> >-              rc = -ENODEV;
> >> >> >+              rc = aplic_msi_setup(dev, regs);
> >> >> >       else
> >> >> >               rc = aplic_direct_setup(dev, regs);
> >> >> >       if (rc) {
> >> >> >diff --git a/drivers/irqchip/irq-riscv-aplic-main.h b/drivers/irqchip/irq-riscv-aplic-main.h
> >> >> >index 474a04229334..78267ec58098 100644
> >> >> >--- a/drivers/irqchip/irq-riscv-aplic-main.h
> >> >> >+++ b/drivers/irqchip/irq-riscv-aplic-main.h
> >> >> >@@ -41,5 +41,13 @@ void aplic_init_hw_global(struct aplic_priv *priv, bool msi_mode);
> >> >> > int aplic_setup_priv(struct aplic_priv *priv, struct device *dev,
> >> >> >                    void __iomem *regs);
> >> >> > int aplic_direct_setup(struct device *dev, void __iomem *regs);
> >> >> >+#ifdef CONFIG_RISCV_APLIC_MSI
> >> >> >+int aplic_msi_setup(struct device *dev, void __iomem *regs);
> >> >> >+#else
> >> >> >+static inline int aplic_msi_setup(struct device *dev, void __iomem *regs)
> >> >> >+{
> >> >> >+      return -ENODEV;
> >> >> >+}
> >> >> >+#endif
> >> >> >
> >> >> > #endif
> >> >> >diff --git a/drivers/irqchip/irq-riscv-aplic-msi.c b/drivers/irqchip/irq-riscv-aplic-msi.c
> >> >> >new file mode 100644
> >> >> >index 000000000000..086d00e0429e
> >> >> >--- /dev/null
> >> >> >+++ b/drivers/irqchip/irq-riscv-aplic-msi.c
> >> >> >@@ -0,0 +1,285 @@
> >> >> >+// SPDX-License-Identifier: GPL-2.0
> >> >> >+/*
> >> >> >+ * Copyright (C) 2021 Western Digital Corporation or its affiliates.
> >> >> >+ * Copyright (C) 2022 Ventana Micro Systems Inc.
> >> >> >+ */
> >> >> >+
> >> >> >+#include <linux/bitops.h>
> >> >> >+#include <linux/cpu.h>
> >> >> >+#include <linux/interrupt.h>
> >> >> >+#include <linux/irqchip.h>
> >> >> >+#include <linux/irqchip/riscv-aplic.h>
> >> >> >+#include <linux/irqchip/riscv-imsic.h>
> >> >> >+#include <linux/module.h>
> >> >> >+#include <linux/msi.h>
> >> >> >+#include <linux/of_irq.h>
> >> >> >+#include <linux/platform_device.h>
> >> >> >+#include <linux/printk.h>
> >> >> >+#include <linux/smp.h>
> >> >> >+
> >> >> >+#include "irq-riscv-aplic-main.h"
> >> >> >+
> >> >> >+static void aplic_msi_irq_unmask(struct irq_data *d)
> >> >> >+{
> >> >> >+      aplic_irq_unmask(d);
> >> >> >+      irq_chip_unmask_parent(d);
> >> >> >+}
> >> >> >+
> >> >> >+static void aplic_msi_irq_mask(struct irq_data *d)
> >> >> >+{
> >> >> >+      aplic_irq_mask(d);
> >> >> >+      irq_chip_mask_parent(d);
> >> >> >+}
> >> >> >+
> >> >> >+static void aplic_msi_irq_eoi(struct irq_data *d)
> >> >> >+{
> >> >> >+      struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
> >> >> >+      u32 reg_off, reg_mask;
> >> >> >+
> >> >> >+      /*
> >> >> >+       * EOI handling only required only for level-triggered
> >> >> >+       * interrupts in APLIC MSI mode.
> >> >> >+       */
> >> >> >+
> >> >> >+      reg_off = APLIC_CLRIP_BASE + ((d->hwirq / APLIC_IRQBITS_PER_REG) * 4);
> >> >> >+      reg_mask = BIT(d->hwirq % APLIC_IRQBITS_PER_REG);
> >> >> >+      switch (irqd_get_trigger_type(d)) {
> >> >> >+      case IRQ_TYPE_LEVEL_LOW:
> >> >> >+              if (!(readl(priv->regs + reg_off) & reg_mask))
> >> >> >+                      writel(d->hwirq, priv->regs + APLIC_SETIPNUM_LE);
> >> >> >+              break;
> >> >> >+      case IRQ_TYPE_LEVEL_HIGH:
> >> >> >+              if (readl(priv->regs + reg_off) & reg_mask)
> >> >> >+                      writel(d->hwirq, priv->regs + APLIC_SETIPNUM_LE);
> >> >> >+              break;
> >> >> >+      }
> >> >> >+}
> >> >> >+
> >> >> >+static struct irq_chip aplic_msi_chip = {
> >> >> >+      .name           = "APLIC-MSI",
> >> >> >+      .irq_mask       = aplic_msi_irq_mask,
> >> >> >+      .irq_unmask     = aplic_msi_irq_unmask,
> >> >> >+      .irq_set_type   = aplic_irq_set_type,
> >> >> >+      .irq_eoi        = aplic_msi_irq_eoi,
> >> >> >+#ifdef CONFIG_SMP
> >> >> >+      .irq_set_affinity = irq_chip_set_affinity_parent,
> >> >> >+#endif
> >> >> >+      .flags          = IRQCHIP_SET_TYPE_MASKED |
> >> >> >+                        IRQCHIP_SKIP_SET_WAKE |
> >> >> >+                        IRQCHIP_MASK_ON_SUSPEND,
> >> >> >+};
> >> >> >+
> >> >> >+static int aplic_msi_irqdomain_translate(struct irq_domain *d,
> >> >> >+                                       struct irq_fwspec *fwspec,
> >> >> >+                                       unsigned long *hwirq,
> >> >> >+                                       unsigned int *type)
> >> >> >+{
> >> >> >+      struct aplic_priv *priv = platform_msi_get_host_data(d);
> >> >> >+
> >> >> >+      return aplic_irqdomain_translate(fwspec, priv->gsi_base, hwirq, type);
> >> >> >+}
> >> >> >+
> >> >> >+static int aplic_msi_irqdomain_alloc(struct irq_domain *domain,
> >> >> >+                                   unsigned int virq, unsigned int nr_irqs,
> >> >> >+                                   void *arg)
> >> >> >+{
> >> >> >+      int i, ret;
> >> >> >+      unsigned int type;
> >> >> >+      irq_hw_number_t hwirq;
> >> >> >+      struct irq_fwspec *fwspec = arg;
> >> >> >+      struct aplic_priv *priv = platform_msi_get_host_data(domain);
> >> >> >+
> >> >> >+      ret = aplic_irqdomain_translate(fwspec, priv->gsi_base, &hwirq, &type);
> >> >> >+      if (ret)
> >> >> >+              return ret;
> >> >>
> >> >> In your patchset, the wired IRQ and IRQ of platform device will go into APLIC-MSI domain firstly.
> >> >
> >> >Yes, that is correct. In general, this applies to AIA specification
> >> >and nothing to do with this patchset.
> >> >
> >> >> Let me assume here is a MSI IRQ not wired IRQ on a device, and it is a platform device in system.
> >> >> so in aplic_irqdomain_translate() function, it will parse the APLIC physical IRQ number by fwspec->param[0],
> >> >> but this is not a wried IRQ, it is a MSI IRQ, it should not has a APLIC physical IRQ number, the hwirq number should be allocated by MSI bitmap,
> >> >> what value will be parse by DTS? zero or negative?
> >> >
> >> >For platform devices with MSI support, the MSIs will directly target
> >> >the per-HART
> >> >IMSICs and the DT node of such devices will never point to APLIC as the parent
> >> >MSI controller.
> >> >
> >>
> >> >The IMSIC driver implements the IMSIC-PLAT domain for platform MSIs.
> >>
> >> Have you test this case on QEMU? would you like share the test steps?
> >
> >The APLIC in MSI-mode acts like a platform device with MSIs so yes this
> >is tested on QEMU.
>
> yet, I know the wired interrupt with MSI-mode of APLIC has tested on QEMU by virtio devices, but i want to know how about the MSI interrupt with non-PCI device?
> have you tested this case?

I don't see any difference in the way APLIC MSI-mode platform
device use MSI versus any other platform using MSI.

We also have the RISC-V IOMMU platform device which uses
MSIs provided by the IMSIC-PLAT domain.

Regards,
Anup



More information about the linux-riscv mailing list