[PATCH] riscv: make ZONE_DMA32 optional

Mon Sep 23 02:46:11 PDT 2024

On 20/09/2024 14:18, Drew Fustini wrote:
> On Fri, Sep 20, 2024 at 01:58:23AM -0700, Palmer Dabbelt wrote:
>> On Tue, 27 Aug 2024 16:10:20 PDT (-0700), dfustini at tenstorrent.com wrote:
>>> On Tue, Aug 27, 2024 at 02:36:11PM +0300, Vladimir Kondratiev wrote:
>>>> It is not necessary any RISCV platform has ZONE_DMA32.
>>>>
>>>> Example - if platform has no DRAM in [0..4G] region,
>>>> it will report failure like below each boot.
>>>>
>>>> [    0.088709] swapper/0: page allocation failure: order:7, mode:0xcc4(GFP_KERNEL|GFP_DMA32), nodemask=(null),cpuset=/
>>>> [    0.088832] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.11.0-rc5 #30
>>>> [    0.088864] Call Trace:
>>>> [    0.088869] [<ffffffff800059f2>] dump_backtrace+0x1c/0x24
>>>> [    0.088910] [<ffffffff805f328c>] show_stack+0x2c/0x38
>>>> [    0.088957] [<ffffffff805fd800>] dump_stack_lvl+0x52/0x74
>>>> [    0.088987] [<ffffffff805fd836>] dump_stack+0x14/0x1c
>>>> [    0.089010] [<ffffffff801a23a8>] warn_alloc+0xf4/0x176
>>>> [    0.089041] [<ffffffff801a3052>] __alloc_pages_noprof+0xc28/0xcb4
>>>> [    0.089067] [<ffffffff80086eda>] atomic_pool_expand+0x62/0x1f8
>>>> [    0.089090] [<ffffffff8080d674>] __dma_atomic_pool_init+0x46/0x9e
>>>> [    0.089115] [<ffffffff8080d762>] dma_atomic_pool_init+0x96/0x11c
>>>> [    0.089139] [<ffffffff80002146>] do_one_initcall+0x5c/0x1b2
>>>> [    0.089158] [<ffffffff8080127c>] kernel_init_freeable+0x214/0x274
>>>> [    0.089190] [<ffffffff805fefd8>] kernel_init+0x1e/0x10a
>>>> [    0.089209] [<ffffffff8060748a>] ret_from_fork+0xe/0x1c
>>>>
>>>> Signed-off-by: Vladimir Kondratiev <vladimir.kondratiev at mobileye.com>
>>>> ---
>>>>   arch/riscv/Kconfig | 2 +-
>>>>   mm/Kconfig         | 2 +-
>>>>   2 files changed, 2 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>>> index 0f3cd7c3a436..94a573112625 100644
>>>> --- a/arch/riscv/Kconfig
>>>> +++ b/arch/riscv/Kconfig
>>>> @@ -50,6 +50,7 @@ config RISCV
>>>>   	select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
>>>>   	select ARCH_HAS_UBSAN
>>>>   	select ARCH_HAS_VDSO_DATA
>>>> +	select ARCH_HAS_ZONE_DMA_SET if 64BIT
>>>>   	select ARCH_KEEP_MEMBLOCK if ACPI
>>>>   	select ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE	if 64BIT && MMU
>>>>   	select ARCH_OPTIONAL_KERNEL_RWX if ARCH_HAS_STRICT_KERNEL_RWX
>>>> @@ -200,7 +201,6 @@ config RISCV
>>>>   	select THREAD_INFO_IN_TASK
>>>>   	select TRACE_IRQFLAGS_SUPPORT
>>>>   	select UACCESS_MEMCPY if !MMU
>>>> -	select ZONE_DMA32 if 64BIT
>>>>
>>>>   config CLANG_SUPPORTS_DYNAMIC_FTRACE
>>>>   	def_bool CC_IS_CLANG
>>>> diff --git a/mm/Kconfig b/mm/Kconfig
>>>> index b72e7d040f78..97c85da98e89 100644
>>>> --- a/mm/Kconfig
>>>> +++ b/mm/Kconfig
>>>> @@ -1032,7 +1032,7 @@ config ZONE_DMA
>>>>   config ZONE_DMA32
>>>>   	bool "Support DMA32 zone" if ARCH_HAS_ZONE_DMA_SET
>>>>   	depends on !X86_32
>>>> -	default y if ARM64
>>>> +	default y if ARM64 || (RISCV && 64BIT)
>>>>
>>>>   config ZONE_DEVICE
>>>>   	bool "Device memory (pmem, HMM, etc...) hotplug support"
>>>> --
>>>> 2.37.3
>>>>
>>>
>>> Reviewed-by: Drew Fustini <dfustini at tenstorrent.com>
>>>
>>> Thanks for sending this patch as I've also encountered that annoying
>>> error on systems with DRAM above 4GB.
>>>
>>> I tested this patch by changing the qemu virt machine to have DRAM
>>> starting at 2^32:
>>>
>>> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
>>> index cef41c150aaf..3033a2560edb 100644
>>> --- a/hw/riscv/virt.c
>>> +++ b/hw/riscv/virt.c
>>> @@ -87,7 +87,7 @@ static const MemMapEntry virt_memmap[] = {
>>>       [VIRT_IMSIC_S] =      { 0x28000000, VIRT_IMSIC_MAX_SIZE },
>>>       [VIRT_PCIE_ECAM] =    { 0x30000000,    0x10000000 },
>>>       [VIRT_PCIE_MMIO] =    { 0x40000000,    0x40000000 },
>>> -    [VIRT_DRAM] =         { 0x80000000,           0x0 },
>>> +    [VIRT_DRAM] =        { 0x100000000,           0x0 },
>>>   };
>>>
>>>   /* PCIe high mmio is fixed for RV32 */
>>
>> IIRC the ZONE_DMA32 stuff existed for some of the early SiFive systems,
>> where the expansion daughterboard's PCIe controller (via a Xilinx FPGA)
>> could only handle 32-bit DMA addreses.  I think there's a similar quirk in
>> the Microsemi PCIe controller on the PolarFire boards, but Conor would know
>> for sure.

The Fu740 SoC has a mirror of part of DRAM in 32bit space specifically
I think for PCIe where something may be attached via a PCIe-PCI bridge.

> I don't think this patch would affect those systems that need ZONE_DMA2.
> I believe it just makes it possible to disable it in the kernel config.
> The platform I'm working on has no memory below 4GB and all the PCIe
> devices that I care about are not 32-bit constrained. Therefore I just
> want to be able to turn it off in my .config.
> 
> Thanks,
> Drew

I tried this a while ago and IIRC you run into issues with DMA
allocations from certain drivers that expect this to exist.

With new platforms that don't have any memory in DMA32 space
this may become more common. I've already had to deal with this
for an internal dev project, and it ended up having a couple of
hacks into the allocation code to silence warnings/errors.

This of course may have been fixed, given this was last year.

-- 
Ben Dooks				http://www.codethink.co.uk/
Senior Engineer				Codethink - Providing Genius

https://www.codethink.co.uk/privacy.html