[PATCH] ARM: OMAP2+: Fix oops with LPAE and more than 2GB of memory

[Tony Lindgren]

* Lokesh Vutla <a0131933 at ti.com> [151015 01:02]:
> Hi Tony,
> 
> On Wednesday 14 October 2015 09:32 PM, Tony Lindgren wrote:
> > * Lokesh Vutla <a0131933 at ti.com> [151013 20:53]:
> >> Hi Tony,
> >>
> >> On Wednesday 14 October 2015 04:43 AM, Tony Lindgren wrote:
> >>> On boards with more than 2GB of RAM booting goes wrong with things not working
> >>> and we're getting lots of l3 warnings:
> >>>
> >>> WARNING: CPU: 0 PID: 1 at drivers/bus/omap_l3_noc.c:147 l3_interrupt_handler+0x260/0x384()
> >>> 44000000.ocp:L3 Custom Error: MASTER MMC6 TARGET DMM1 (Idle): Data Access in User mode during Functional access
> >>> ...
> >>> [<c044e158>] (scsi_add_host_with_dma) from [<c04705c8>] (ata_scsi_add_hosts+0x5c/0x18c)
> >>> [<c04705c8>] (ata_scsi_add_hosts) from [<c046b13c>] (ata_host_register+0x150/0x2cc)
> >>> [<c046b13c>] (ata_host_register) from [<c046b38c>] (ata_host_activate+0xd4/0x124)
> >>> [<c046b38c>] (ata_host_activate) from [<c047f42c>] (ahci_host_activate+0x5c/0x194)
> >>> [<c047f42c>] (ahci_host_activate) from [<c0480854>] (ahci_platform_init_host+0x1f0/0x3f0)
> >>> [<c0480854>] (ahci_platform_init_host) from [<c047c9dc>] (ahci_probe+0x70/0x98)
> >>> [<c047c9dc>] (ahci_probe) from [<c04220cc>] (platform_drv_probe+0x54/0xb4)
> >>>
> >>> Let's fix the issue by enabling ZONE_DMA for LPAE.
> >>
> >> May I know on which platform you have reproduced this?
> > 
> > This is on the 4GB version of isee igepv5 GEP0050-RB10
> > 
> > https://isee.biz/products/igep-processor-boards/igepv5-omap5432
> > 
> >> Just wondering what other changes you made for booting a OMAP5+ based
> >> board with more than 2GB.
> > 
> > Just the minimal dts changes I posted yesterday to use shared dtsi file
> > for omap5-uevm variants. Then boot with something like this in the kernel
> > cmdline:
> > 
> > mem=2032M at 0x80000000 mem=2048M at 0x300000000
> Is it the upper 2GB memory starts from 0x3_0000_0000? Looking at the
> Table 4-4. "AXI Access memory map" in TRM, it tells it starts from
> 0x2_0000_0000.(or am I missing something?)
> 
> Any chance you tried doing memtest for all this memory?

Seems to work just fine, it's been up since I posted $subject patch., I tried
a kernel compile on it already earlier and just started memtester 3900M which
seems to only capable of testing 2924MB though.

> IIUC, It is not allowed to use the mix of L3 address mapping and MA
> physical address mapping. TRM specifies that
> "It is expected that the OS uses either the lower 2-GiB space and the
> lower aliased address of emif(a) and (b), or the continuous 8-GiB space
> and the upper aliasing of the emif(a) and (b) for all EMIF accesses."

I think that depends on the chip selects that can be used. The above
is probably for the case where all 8GB was to be used.

> I tried a similar 4GB setup and observed few stuff:
> - Need to have ARM_PV_FIXUP enabled something similar that is done in
> Keystone inorder to shift to higher address space.[1] [2]

Seems that's optional for 4GB.. But probably a must for 8GB :)

> - All the peripherals other than MPU, can access only 2GB. So we need to
> restrict the DMA and need an addresses translation. Along with this
> patch [3] [4] are needed.

Yes there's no memory accessible beyond 2GB for the DMA hardware. I'll
post an updated patch with the 2GB limit for dma_zone_size after some
more tests.

For the dma-ranges, looks like I have not hit that one yet. Which drivers
did you run into problems without that?

Regards,

Tony

> [1]
> https://git.ti.com/~lokesh/ti-linux-kernel/lokesh-ti-linux-kernel/commit/be7d2d7e18f473e6078cd7c0b25ec3c58f40736b
> [2]
> https://git.ti.com/~lokesh/ti-linux-kernel/lokesh-ti-linux-kernel/commit/dd0a564ee565405bec5e9f58f0c97fb6714110ac
> [3]
> https://git.ti.com/~lokesh/ti-linux-kernel/lokesh-ti-linux-kernel/commit/985ed401e04926cb88b23646271dc96900abc50d
> [4]
> https://git.ti.com/~lokesh/ti-linux-kernel/lokesh-ti-linux-kernel/commit/c1f7568c4cc6f3146b7137783bd403967aabd0d8