[PATCH] ARM: mvebu: fix overlap of Crypto SRAM with PCIe memory window

Boris Brezillon boris.brezillon at free-electrons.com
Fri Mar 11 01:36:33 PST 2016


On Tue,  8 Mar 2016 16:59:57 +0100
Thomas Petazzoni <thomas.petazzoni at free-electrons.com> wrote:

> When the Crypto SRAM mappings were added to the Device Tree files
> describing the Armada XP boards in commit c466d997bb16 ("ARM: mvebu:
> define crypto SRAM ranges for all armada-xp boards"), the fact that
> those mappings were overlaping with the PCIe memory aperture was
> overlooked. Due to this, we currently have for all Armada XP platforms
> a situation that looks like this:
> 
> Memory mapping on Armada XP boards with internal registers at
> 0xf1000000:
> 
>  - 0x00000000 -> 0xf0000000	3.75G 	RAM
>  - 0xf0000000 -> 0xf1000000	16M	NOR flashes (AXP GP / AXP DB)
>  - 0xf1000000 -> 0xf1100000	1M	internal registers
>  - 0xf8000000 -> 0xffe0000	126M	PCIe memory aperture
>  - 0xf8100000 -> 0xf8110000	64KB	Crypto SRAM #0	=> OVERLAPS WITH PCIE !
>  - 0xf8110000 -> 0xf8120000	64KB	Crypto SRAM #1	=> OVERLAPS WITH PCIE !
>  - 0xffe00000 -> 0xfff00000	1M	PCIe I/O aperture
>  - 0xfff0000  -> 0xffffffff	1M	BootROM
> 
> The overlap means that when PCIe devices are added, depending on their
> memory window needs, they might or might not be mapped into the
> physical address space. Indeed, they will not be mapped if the area
> allocated in the PCIe memory aperture by the PCI core overlaps with
> one of the Crypto SRAM. Typically, a Intel IGB PCIe NIC that needs 8MB
> of PCIe memory will see its PCIe memory window allocated from
> 0xf80000000 for 8MB, which overlaps with the Crypto SRAM windows. Due
> to this, the PCIe window is not created, and any attempt to access the
> PCIe window makes the kernel explode:
> 
> [    3.302213] igb: Copyright (c) 2007-2014 Intel Corporation.
> [    3.307841] pci 0000:00:09.0: enabling device (0140 -> 0143)
> [    3.313539] mvebu_mbus: cannot add window '4:f8', conflicts with another window
> [    3.320870] mvebu-pcie soc:pcie-controller: Could not create MBus window at [mem 0xf8000000-0xf87fffff]: -22
> [    3.330811] Unhandled fault: external abort on non-linefetch (0x1008) at 0xf08c0018
> 
> This problem does not occur on Armada 370 boards, because we use the
> following memory mapping (for boards that have internal registers at
> 0xf1000000):
> 
>  - 0x00000000 -> 0xf0000000	3.75G 	RAM
>  - 0xf0000000 -> 0xf1000000	16M	NOR flashes (AXP GP / AXP DB)
>  - 0xf1000000 -> 0xf1100000	1M	internal registers
>  - 0xf1100000 -> 0xf1110000	64KB	Crypto SRAM #0 => OK !
>  - 0xf8000000 -> 0xffe0000	126M	PCIe memory
>  - 0xffe00000 -> 0xfff00000	1M	PCIe I/O
>  - 0xfff0000  -> 0xffffffff	1M	BootROM
> 
> Obviously, the solution is to align the location of the Crypto SRAM
> mappings of Armada XP to be similar with the ones on Armada 370, i.e
> have them between the "internal registers" area and the beginning of
> the PCIe aperture.
> 
> However, we have a special case with the OpenBlocks AX3-4 platform,
> which has a 128 MB NOR flash. Currently, this NOR flash is mapped from
> 0xf0000000 to 0xf8000000. This is possible because on OpenBlocks
> AX3-4, the internal registers are not at 0xf1000000. And this explains
> why the Crypto SRAM mappings were not configured at the same place on
> Armada XP.
> 
> Hence, the solution is two-fold:
> 
>  (1) Move the NOR flash mapping on Armada XP OpenBlocks AX3-4 from
>      0xe8000000 to 0xf0000000. This frees the 0xf0000000 ->
>      0xf80000000 space.
> 
>  (2) Move the Crypto SRAM mappings on Armada XP to be similar to
>      Armada 370 (except of course that Armada XP has two Crypto SRAM
>      and not one).
> 
> After this patch, the memory mapping on Armada XP boards with
> registers at 0xf1 is:
> 
>  - 0x00000000 -> 0xf0000000	3.75G 	RAM
>  - 0xf0000000 -> 0xf1000000	16M	NOR flashes (AXP GP / AXP DB)
>  - 0xf1000000 -> 0xf1100000	1M	internal registers
>  - 0xf1100000 -> 0xf1110000	64KB	Crypto SRAM #0
>  - 0xf1110000 -> 0xf1120000	64KB	Crypto SRAM #1
>  - 0xf8000000 -> 0xffe0000	126M	PCIe memory
>  - 0xffe00000 -> 0xfff00000	1M	PCIe I/O
>  - 0xfff0000  -> 0xffffffff	1M	BootROM
> 
> And the memory mapping for the special case of the OpenBlocks AX3-4
> (internal registers at 0xd0000000, NOR of 128 MB):
> 
>  - 0x00000000 -> 0xc0000000	3G 	RAM
>  - 0xd0000000 -> 0xd1000000	1M	internal registers
>  - 0xe800000  -> 0xf0000000	128M	NOR flash
>  - 0xf1100000 -> 0xf1110000	64KB	Crypto SRAM #0
>  - 0xf1110000 -> 0xf1120000	64KB	Crypto SRAM #1
>  - 0xf8000000 -> 0xffe0000	126M	PCIe memory
>  - 0xffe00000 -> 0xfff00000	1M	PCIe I/O
>  - 0xfff0000  -> 0xffffffff	1M	BootROM
> 
> Fixes: c466d997bb16 ("ARM: mvebu: define crypto SRAM ranges for all armada-xp boards")
> Reported-by: Phil Sutter <phil at nwl.cc>
> Cc: Phil Sutter <phil at nwl.cc>
> Cc: <stable at kernel.org>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni at free-electrons.com>

Reviewed-by: Boris Brezillon <boris.brezillon at free-electrons.com>

> ---
>  arch/arm/boot/dts/armada-xp-axpwifiap.dts        | 4 ++--
>  arch/arm/boot/dts/armada-xp-db.dts               | 4 ++--
>  arch/arm/boot/dts/armada-xp-gp.dts               | 4 ++--
>  arch/arm/boot/dts/armada-xp-lenovo-ix4-300d.dts  | 4 ++--
>  arch/arm/boot/dts/armada-xp-linksys-mamba.dts    | 4 ++--
>  arch/arm/boot/dts/armada-xp-matrix.dts           | 4 ++--
>  arch/arm/boot/dts/armada-xp-netgear-rn2120.dts   | 4 ++--
>  arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts | 6 +++---
>  arch/arm/boot/dts/armada-xp-synology-ds414.dts   | 4 ++--
>  9 files changed, 19 insertions(+), 19 deletions(-)
> 
> diff --git a/arch/arm/boot/dts/armada-xp-axpwifiap.dts b/arch/arm/boot/dts/armada-xp-axpwifiap.dts
> index 23fc670..5c21b23 100644
> --- a/arch/arm/boot/dts/armada-xp-axpwifiap.dts
> +++ b/arch/arm/boot/dts/armada-xp-axpwifiap.dts
> @@ -70,8 +70,8 @@
>  	soc {
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xf1000000 0x100000
>  			  MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
> -			  MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			  MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			  MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			  MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		pcie-controller {
>  			status = "okay";
> diff --git a/arch/arm/boot/dts/armada-xp-db.dts b/arch/arm/boot/dts/armada-xp-db.dts
> index f774101..ebe1d26 100644
> --- a/arch/arm/boot/dts/armada-xp-db.dts
> +++ b/arch/arm/boot/dts/armada-xp-db.dts
> @@ -76,8 +76,8 @@
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xf1000000 0x100000
>  			  MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
>  			  MBUS_ID(0x01, 0x2f) 0 0 0xf0000000 0x1000000
> -			  MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			  MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			  MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			  MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		devbus-bootcs {
>  			status = "okay";
> diff --git a/arch/arm/boot/dts/armada-xp-gp.dts b/arch/arm/boot/dts/armada-xp-gp.dts
> index 4878d73..5730b87 100644
> --- a/arch/arm/boot/dts/armada-xp-gp.dts
> +++ b/arch/arm/boot/dts/armada-xp-gp.dts
> @@ -95,8 +95,8 @@
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xf1000000 0x100000
>  			  MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
>  			  MBUS_ID(0x01, 0x2f) 0 0 0xf0000000 0x1000000
> -			  MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			  MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			  MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			  MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		devbus-bootcs {
>  			status = "okay";
> diff --git a/arch/arm/boot/dts/armada-xp-lenovo-ix4-300d.dts b/arch/arm/boot/dts/armada-xp-lenovo-ix4-300d.dts
> index fb9e1bb..8af463f 100644
> --- a/arch/arm/boot/dts/armada-xp-lenovo-ix4-300d.dts
> +++ b/arch/arm/boot/dts/armada-xp-lenovo-ix4-300d.dts
> @@ -65,8 +65,8 @@
>  	soc {
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xd0000000 0x100000
>  			MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
> -			MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		pcie-controller {
>  			status = "okay";
> diff --git a/arch/arm/boot/dts/armada-xp-linksys-mamba.dts b/arch/arm/boot/dts/armada-xp-linksys-mamba.dts
> index 6e9820e..b89e6cf 100644
> --- a/arch/arm/boot/dts/armada-xp-linksys-mamba.dts
> +++ b/arch/arm/boot/dts/armada-xp-linksys-mamba.dts
> @@ -70,8 +70,8 @@
>  	soc {
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xf1000000 0x100000
>  			  MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
> -			  MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			  MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			  MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			  MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		pcie-controller {
>  			status = "okay";
> diff --git a/arch/arm/boot/dts/armada-xp-matrix.dts b/arch/arm/boot/dts/armada-xp-matrix.dts
> index 6ab3383..6522b04 100644
> --- a/arch/arm/boot/dts/armada-xp-matrix.dts
> +++ b/arch/arm/boot/dts/armada-xp-matrix.dts
> @@ -68,8 +68,8 @@
>  	soc {
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xf1000000 0x100000
>  			  MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
> -			  MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			  MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			  MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			  MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		internal-regs {
>  			serial at 12000 {
> diff --git a/arch/arm/boot/dts/armada-xp-netgear-rn2120.dts b/arch/arm/boot/dts/armada-xp-netgear-rn2120.dts
> index 62175a8..d19f44c 100644
> --- a/arch/arm/boot/dts/armada-xp-netgear-rn2120.dts
> +++ b/arch/arm/boot/dts/armada-xp-netgear-rn2120.dts
> @@ -64,8 +64,8 @@
>  	soc {
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xd0000000 0x100000
>  			  MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
> -			  MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			  MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			  MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			  MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		pcie-controller {
>  			status = "okay";
> diff --git a/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts b/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
> index a5db177..853bd39 100644
> --- a/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
> +++ b/arch/arm/boot/dts/armada-xp-openblocks-ax3-4.dts
> @@ -65,9 +65,9 @@
>  	soc {
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xd0000000 0x100000
>  			  MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
> -			  MBUS_ID(0x01, 0x2f) 0 0 0xf0000000 0x8000000
> -			  MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			  MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			  MBUS_ID(0x01, 0x2f) 0 0 0xe8000000 0x8000000
> +			  MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			  MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		devbus-bootcs {
>  			status = "okay";
> diff --git a/arch/arm/boot/dts/armada-xp-synology-ds414.dts b/arch/arm/boot/dts/armada-xp-synology-ds414.dts
> index 2391b11..d17dab0 100644
> --- a/arch/arm/boot/dts/armada-xp-synology-ds414.dts
> +++ b/arch/arm/boot/dts/armada-xp-synology-ds414.dts
> @@ -78,8 +78,8 @@
>  	soc {
>  		ranges = <MBUS_ID(0xf0, 0x01) 0 0 0xf1000000 0x100000
>  			  MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
> -			  MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
> -			  MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
> +			  MBUS_ID(0x09, 0x09) 0 0 0xf1100000 0x10000
> +			  MBUS_ID(0x09, 0x05) 0 0 0xf1110000 0x10000>;
>  
>  		pcie-controller {
>  			status = "okay";



-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com



More information about the linux-arm-kernel mailing list