ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4

[Stefan Wahren]

Hi Russel,

Am 17.05.23 um 16:45 schrieb Russell King (Oracle):
> On Wed, May 17, 2023 at 04:35:35PM +0200, Stefan Wahren wrote:
>> Am 17.05.23 um 12:54 schrieb Arnd Bergmann:
>>> On Wed, May 17, 2023, at 12:34, Arnd Bergmann wrote:
>>>> On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote:
>>>>> Am 17.05.23 um 12:06 schrieb Robin Murphy:
>>>>>> On 2023-05-17 07:23, Stefan Wahren wrote:
>>>>
>>>> I think LPAE doesn't affect it, the configurations that Stefan
>>>> mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with
>>>> VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume
>>>> that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken.
>>>>
>>>>> This one?
>>>>> [    0.000000] software IO TLB: area num 4.
>>>>> [    0.000000] software IO TLB: mapped [mem
>>>>> 0x0000000069e7f000-0x000000006de7f000] (64MB)
>>>>
>>>> Indeed, that is clearly above the .dma_zone_size value from
>>>> the machine descriptor.
>>>
>>> It looks like Arm does not set the ARCH_LOW_ADDRESS_LIMIT
>>> constant for swiotlb, so it gets the default 4GB maximum.
>>>
>>> I think this would work on arm, though it's probably not the
>>> correct fix in general, as I think MAX_DMA_ADDRESS is meant
>>> to refer to the ISA_DMA_API limit, not the ZONE_DMA limit.:
>>>
>>> --- a/include/linux/memblock.h
>>> +++ b/include/linux/memblock.h
>>> @@ -394,7 +394,7 @@ static inline int memblock_get_region_node(const struct memblock_region *r)
>>>    #define MEMBLOCK_LOW_LIMIT 0
>>>    #ifndef ARCH_LOW_ADDRESS_LIMIT
>>> -#define ARCH_LOW_ADDRESS_LIMIT  0xffffffffUL
>>> +#define ARCH_LOW_ADDRESS_LIMIT  MAX_DMA_ADDRESS
>>>    #endif
>>>    phys_addr_t memblock_phys_alloc_range(phys_addr_t size, phys_addr_t align,
>>>
>>> Can you give this a try?
>>
>> FWIW: Unfortunately this change didn't had any effect.
> 
> As expected, because MAX_DMA_ADDRESS is an integer virtual address, and
> ARCH_LOW_ADDRESS_LIMIT wants a physical address.
> 
> I think we need to define ARCH_LOW_ADDRESS_LIMIT in
> arch/arm/incude/asm/dma.h as:
> 
> #ifndef CONFIG_ZONE_DMA
> ...
> #else
> ...
> extern phys_addr_t arm_dma_limit;
> #define ARCH_LOW_ADDRESS_LIMIT arm_dma_limit
> #endif
> 

Yes, the follow patch worked:

diff --git a/arch/arm/include/asm/dma.h b/arch/arm/include/asm/dma.h
index c6aded1b069c..e2a1916013e7 100644
--- a/arch/arm/include/asm/dma.h
+++ b/arch/arm/include/asm/dma.h
@@ -12,6 +12,9 @@
  	extern phys_addr_t arm_dma_zone_size; \
  	arm_dma_zone_size && arm_dma_zone_size < (0x100000000ULL - 
PAGE_OFFSET) ? \
  		(PAGE_OFFSET + arm_dma_zone_size) : 0xffffffffUL; })
+
+extern phys_addr_t arm_dma_limit;
+#define ARCH_LOW_ADDRESS_LIMIT arm_dma_limit
  #endif

  #ifdef CONFIG_ISA_DMA_API

I successfully tested the following combinations:

Raspberry Pi 4 (8 GB RAM)
VMSPLIT_3G + LPAE
VMSPLIT_2G + LPAE
VMSPLIT_1G + LPAE

Raspberry Pi 3 (1 GB RAM)
VMSPLIT_3G
VMSPLIT_2G

Thanks to all for the fast feedback. Unfortunately i don't feel 
comfortable to prepare a patch with the proper wording.