Neophyte questions about PCIe

[Mason]

On 10/03/2017 17:00, Robin Murphy wrote:

> On 10/03/17 15:35, David Laight wrote:
>
>> Robin Murphy wrote:
>> 
>>> The appropriate DT property would be "dma-ranges", i.e.
>>>
>>> pci at ... {
>>> 	...
>>> 	dma-ranges = <(PCI bus address) (CPU phys address) (size)>;
>>> }
>>
>> Isn't that just saying which physical addresses the cpu can assign
>> for buffers for those devices?
>> There is also an offset between the 'cpu physical address' and the
>> 'dma address'.
> 
> That offset is inherent in what "dma-ranges" describes. Say (for ease of
> calculation) that BAR0 has been put at a mem space address of 0x20000000
> and maps the first 1GB of physical DRAM. That would give us:
> 
> 	dma-ranges = <0x20000000 0x80000000 0x40000000>;
> 
> Then a "virt = dma_alloc_coherent(..., &handle, ...)", borrowing the
> numbers from earlier in the thread, would automatically end up with:
> 
> 	virt == 0xd0855000;
> 	handle == 0x2e07e000;
> 
> (with the physical address of 0x8e07e000 in between being irrelevant to
> the consuming driver)
> 
> It is true that the device's DMA mask assignment is also part and parcel
> of this, whereby we will limit what physical addresses the kernel
> considers valid for DMA involving devices behind this range to the lower
> 3GB (i.e. 0x80000000 + 0x40000000 - 1). With a bit of luck,
> CONFIG_DMABOUNCE should do the rest of the job of working around that
> where necessary.

AFAICT, the parser for the "dma-ranges" property is implemented
in of_dma_get_range() in drivers/of/address.c

http://lxr.free-electrons.com/source/drivers/of/address.c#L808

/**
 * of_dma_get_range - Get DMA range info
 * @np:		device node to get DMA range info
 * @dma_addr:	pointer to store initial DMA address of DMA range
 * @paddr:	pointer to store initial CPU address of DMA range
 * @size:	pointer to store size of DMA range
 *
 * Look in bottom up direction for the first "dma-ranges" property
 * and parse it.
 *  dma-ranges format:
 *	DMA addr (dma_addr)	: naddr cells
 *	CPU addr (phys_addr_t)	: pna cells
 *	size			: nsize cells
 *
 * It returns -ENODEV if "dma-ranges" property was not found
 * for this device in DT.
 */

I didn't find anything relevant in Documentation/devicetree/bindings
except Documentation/devicetree/bindings/iommu/iommu.txt but I'm not
sure this applies to my SoC.

It's not clear to me how ranges and dma-ranges interact...
Is it perhaps: ranges for cpu-to-bus, dma-ranges for bus-to-cpu?


ePAPR (Version 1.1 -- 08 April) provides a formal definition

2.3.9 dma-ranges

Property: dma-ranges

Value type: <empty> or <prop-encoded-array> encoded as arbitrary number
of triplets of (child-bus-address, parent-bus-address, length).

Description:

The dma-ranges property is used to describe the direct memory access (DMA) structure of a
memory-mapped bus whose device tree parent can be accessed from DMA operations
originating from the bus. It provides a means of defining a mapping or translation between the
physical address space of the bus and the physical address space of the parent of the bus.
The format of the value of the dma-ranges property is an arbitrary number of triplets of
(child-bus-address, parent-bus-address, length). Each triplet specified describes a contiguous
DMA address range.

o The child-bus-address is a physical address within the child bus' address space. The
number of cells to represent the address depends on the bus and can be determined
from the #address-cells of this node (the node in which the dma-ranges property
appears).

o The parent-bus-address is a physical address within the parent bus' address space.
The number of cells to represent the parent address is bus dependent and can be
determined from the #address-cells property of the node that defines the parent's
address space.

o The length specifies the size of the range in the child's address space. The number of
cells to represent the size can be determined from the #size-cells of this node (the
node in which the dma-ranges property appears).


I'm still digging :-)

Regards.