[PATCH] ARM: OMAP2+: Fix oops with LPAE and more than 2GB of memory

[Lokesh Vutla]

+Richard

On Thursday 15 October 2015 07:35 PM, Tony Lindgren wrote:
> * Lokesh Vutla <a0131933 at ti.com> [151015 01:02]:
>> Hi Tony,
>>
>> On Wednesday 14 October 2015 09:32 PM, Tony Lindgren wrote:
>>> * Lokesh Vutla <a0131933 at ti.com> [151013 20:53]:
>>>> Hi Tony,
>>>>
>>>> On Wednesday 14 October 2015 04:43 AM, Tony Lindgren wrote:
>>>>> On boards with more than 2GB of RAM booting goes wrong with things not working
>>>>> and we're getting lots of l3 warnings:
>>>>>
>>>>> WARNING: CPU: 0 PID: 1 at drivers/bus/omap_l3_noc.c:147 l3_interrupt_handler+0x260/0x384()
>>>>> 44000000.ocp:L3 Custom Error: MASTER MMC6 TARGET DMM1 (Idle): Data Access in User mode during Functional access
>>>>> ...
>>>>> [<c044e158>] (scsi_add_host_with_dma) from [<c04705c8>] (ata_scsi_add_hosts+0x5c/0x18c)
>>>>> [<c04705c8>] (ata_scsi_add_hosts) from [<c046b13c>] (ata_host_register+0x150/0x2cc)
>>>>> [<c046b13c>] (ata_host_register) from [<c046b38c>] (ata_host_activate+0xd4/0x124)
>>>>> [<c046b38c>] (ata_host_activate) from [<c047f42c>] (ahci_host_activate+0x5c/0x194)
>>>>> [<c047f42c>] (ahci_host_activate) from [<c0480854>] (ahci_platform_init_host+0x1f0/0x3f0)
>>>>> [<c0480854>] (ahci_platform_init_host) from [<c047c9dc>] (ahci_probe+0x70/0x98)
>>>>> [<c047c9dc>] (ahci_probe) from [<c04220cc>] (platform_drv_probe+0x54/0xb4)
>>>>>
>>>>> Let's fix the issue by enabling ZONE_DMA for LPAE.
>>>>
>>>> May I know on which platform you have reproduced this?
>>>
>>> This is on the 4GB version of isee igepv5 GEP0050-RB10
>>>
>>> https://isee.biz/products/igep-processor-boards/igepv5-omap5432
>>>
>>>> Just wondering what other changes you made for booting a OMAP5+ based
>>>> board with more than 2GB.
>>>
>>> Just the minimal dts changes I posted yesterday to use shared dtsi file
>>> for omap5-uevm variants. Then boot with something like this in the kernel
>>> cmdline:
>>>
>>> mem=2032M at 0x80000000 mem=2048M at 0x300000000
>> Is it the upper 2GB memory starts from 0x3_0000_0000? Looking at the
>> Table 4-4. "AXI Access memory map" in TRM, it tells it starts from
>> 0x2_0000_0000.(or am I missing something?)
>>
>> Any chance you tried doing memtest for all this memory?
> 
> Seems to work just fine, it's been up since I posted $subject patch., I tried
> a kernel compile on it already earlier and just started memtester 3900M which
> seems to only capable of testing 2924MB though.
> 
>> IIUC, It is not allowed to use the mix of L3 address mapping and MA
>> physical address mapping. TRM specifies that
>> "It is expected that the OS uses either the lower 2-GiB space and the
>> lower aliased address of emif(a) and (b), or the continuous 8-GiB space
>> and the upper aliasing of the emif(a) and (b) for all EMIF accesses."
> 
> I think that depends on the chip selects that can be used. The above
> is probably for the case where all 8GB was to be used.

Richard, Do you agree with this?

> 
>> I tried a similar 4GB setup and observed few stuff:
>> - Need to have ARM_PV_FIXUP enabled something similar that is done in
>> Keystone inorder to shift to higher address space.[1] [2]
> 
> Seems that's optional for 4GB.. But probably a must for 8GB :)
> 
>> - All the peripherals other than MPU, can access only 2GB. So we need to
>> restrict the DMA and need an addresses translation. Along with this
>> patch [3] [4] are needed.
> 
> Yes there's no memory accessible beyond 2GB for the DMA hardware. I'll
> post an updated patch with the 2GB limit for dma_zone_size after some
> more tests.
> 
> For the dma-ranges, looks like I have not hit that one yet. Which drivers
> did you run into problems without that?

This is needed only when you completely shift to higher address space.

> 
> Regards,
> 
> Tony
> 
>> [1]
>> https://git.ti.com/~lokesh/ti-linux-kernel/lokesh-ti-linux-kernel/commit/be7d2d7e18f473e6078cd7c0b25ec3c58f40736b
>> [2]
>> https://git.ti.com/~lokesh/ti-linux-kernel/lokesh-ti-linux-kernel/commit/dd0a564ee565405bec5e9f58f0c97fb6714110ac
>> [3]
>> https://git.ti.com/~lokesh/ti-linux-kernel/lokesh-ti-linux-kernel/commit/985ed401e04926cb88b23646271dc96900abc50d
>> [4]
>> https://git.ti.com/~lokesh/ti-linux-kernel/lokesh-ti-linux-kernel/commit/c1f7568c4cc6f3146b7137783bd403967aabd0d8
>