[PATCH v11 0/6] PCI EP driver support MSI doorbell from host
Frank Li
Frank.Li at nxp.com
Mon Sep 19 08:40:32 PDT 2022
┌───────┐ ┌──────────┐
│ │ │ │
┌─────────────┐ │ │ │ PCI Host │
│ MSI │◄┐ │ │ │ │
│ Controller │ │ │ │ │ │
└─────────────┘ └─┼───────┼──────────┼─Bar0 │
│ PCI │ │ Bar1 │
│ Func │ │ Bar2 │
│ │ │ Bar3 │
│ │ │ Bar4 │
│ ├─────────►│ │
└───────┘ └──────────┘
Many PCI controllers provided Endpoint functions.
Generally PCI endpoint is hardware, which is not running a rich OS,
like linux.
But Linux also supports endpoint functions. PCI Host write BAR<n> space
like write to memory. The EP side can't know memory changed by the Host
driver.
PCI Spec has not defined a standard method to do that. Only define
MSI(x) to let EP notified RC status change.
The basic idea is to trigger an IRQ when PCI RC writes to a memory
address. That's what MSI controller provided. EP drivers just need to
request a platform MSI interrupt, struct MSI_msg *msg will pass down a
memory address and data. EP driver will map such memory address to
one of PCI BAR<n>. Host just writes such an address to trigger EP side
IRQ.
If system have gic-its, only need update PCI EP side driver. But i.MX
have not chip support gic-its yet. So we have to use MU to simulate a
MSI controller. Although only 4 MSI IRQs are simulated, it matched
vntb(pci-epf-vntb) network requirement.
After enable MSI, ping delay reduce < 1ms from ~8ms
IRQchip: imx mu worked as MSI controller:
let imx mu worked as MSI controllers. Although IP is not design
as MSI controller, we still can use it if limited IRQ number to 4.
pcie: endpoint: pci-epf-vntb: add endpoint MSI support
Based on ntb-next branch. https://github.com/jonmason/ntb/commits/ntb-next
Using MSI as door bell registers
mu-msi patches (1-4) and vntb patch(5-6) is totally independently.
These can be applied by irqchip and pci's maintainer seperatedly.
i.MX EP function driver is upstreaming by Richard Zhu.
Some dts change missed at this patches. below is reference dts change
--- a/arch/arm64/boot/dts/freescale/imx8-ss-hsio.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8-ss-hsio.dtsi
@@ -160,5 +160,6 @@ pcieb_ep: pcie_ep at 5f010000 {
num-ib-windows = <6>;
num-ob-windows = <6>;
status = "disabled";
+ MSI-parent = <&lsio_mu12>;
};
--- a/arch/arm64/boot/dts/freescale/imx8-ss-lsio.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8-ss-lsio.dtsi
@@ -172,6 +172,19 @@ lsio_mu6: mailbox at 5d210000 {
status = "disabled";
};
+ lsio_mu12: mailbox at 5d270000 {
+ compatible = "fsl,imx6sx-mu-MSI";
+ msi-controller;
+ interrupt-controller;
+ reg = <0x5d270000 0x10000>, /* A side */
+ <0x5d300000 0x10000>; /* B side */
+ reg-names = "a", "b";
+ interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_HIGH>;
+ power-domains = <&pd IMX_SC_R_MU_12A>,
+ <&pd IMX_SC_R_MU_12B>;
+ power-domain-names = "a", "b";
+ };
+
Change Log
- Change from v10 to v11
irqchips: no change
pcie:
clean up build warning foundby kernel test robot
clean up kernel-doc warning.
clean up vhost VHost.
- Change from v9 to v10
fixed build error reported by kernel test robot <lkp at intel.com>
irqchips:
fixed accoring to Marc Zyngier's comments
Added new patch platform-msi: export symbol
platform_msi_create_irq_domain()
Using one lock for both reg and alloc msi irq
Using predefined macro to init cfg data
pcie: endpoint:
fixed according to Manivannan Sadhasivam's feedback
Added makeup patch before enable msi irq
PCI: endpoint: makeup pci-epf-vntb.c
- Change from v8 to v9
fix dt_bind_check error
- Change from v7 to v8
irqchip: using name process-a-side as resource bind name
pcie: endpoint:
- fix build error reported by kernel test robot <lkp at intel.com>
- rename epf_db_phy to epf_db_phys
- rework error message
- rework commit message
- change ntb to vtb at apply irq.
- kept name msi_virqbase because it is msi irq base number,
not base address.
- Change from v6 to v7
pcie: endpoint: add endpoint MSI support
Fine tuning commit message
Fixed issues, reviewed by Bjorn Helgaas
- Change from v5 to v6
Fixed build error found by kernel test robot
- Change from v4 to v5
Fixed dt-binding document
add msi-cell
add interrupt max number
update naming reg-names and power-domain-names.
Fixed irqchip-Add-IMX-MU-MSI-controller-driver.patch
rework commit message
remove some field in struct imx_mu_dcfg
error handle when link power domain failure.
add irq_domain_update_bus_token
- Change from v3 to v4
Fixed dt-binding document according to Krzysztof Kozlowski's feedback
Fixed irqchip-imx-mu-worked-as-msi-controller according to Marc Zyngier's
comments.
There are still two important points, which I am not sure.
1. clean irq_set_affinity after platform_msi_create_irq_domain.
Some function, like platform_msi_write_msg() is static.
so I have to set MSI_FLAG_USE_DEF_CHIP_OPS flags, which will
set irq_set_affinity to default one.
2. about comments
> + msi_data->msi_domain = platform_msi_create_irq_domain(
> + of_node_to_fwnode(msi_data->pdev->dev.of_node),
> + &imx_mu_msi_domain_info,
> + msi_data->parent);
"And you don't get an error due to the fact that you use the same
fwnode for both domains without overriding the domain bus token?"
I did not understand yet.
Fixed static check warning, reported by Dan Carpenter
pcie: endpoint: pci-epf-vntb: add endpoint MSI support
- Change from v2 to v3
Fixed dt-binding docment check failure
Fixed typo a cover letter.
Change according Bjorn's comments at patch
pcie: endpoint: pci-epf-vntb: add endpoint MSI support
- from V1 to V2
Fixed fsl,mu-msi.yaml's problem
Fixed irq-imx-mu-msi.c problem according Marc Zyngier's feeback
Added a new patch to allow pass down .pm by IRQCHIP_PLATFORM_DRIVER_END
Frank Li (6):
platform-msi: export symbol platform_msi_create_irq_domain()
irqchip: allow pass down .pm field at IRQCHIP_PLATFORM_DRIVER_END
irqchip: Add IMX MU MSI controller driver
dt-bindings: irqchip: imx mu work as msi controller
PCI: endpoint: cleanup pci-epf-vntb.c
PCI: endpoint: Add vNTB MSI support
.../interrupt-controller/fsl,mu-msi.yaml | 99 ++++
drivers/base/platform-msi.c | 1 +
drivers/irqchip/Kconfig | 14 +
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-imx-mu-msi.c | 455 ++++++++++++++++++
drivers/pci/endpoint/functions/pci-epf-vntb.c | 296 ++++++++----
include/linux/irqchip.h | 4 +-
7 files changed, 786 insertions(+), 84 deletions(-)
create mode 100644 Documentation/devicetree/bindings/interrupt-controller/fsl,mu-msi.yaml
create mode 100644 drivers/irqchip/irq-imx-mu-msi.c
--
2.35.1
More information about the linux-arm-kernel
mailing list