[PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Stuart Yoder]

> -----Original Message-----
> From: Will Deacon [mailto:will.deacon at arm.com]
> Sent: Monday, June 16, 2014 12:04 PM
> To: Yoder Stuart-B08248
> Cc: Sethi Varun-B16395; Thierry Reding; Mark Rutland;
> devicetree at vger.kernel.org; linux-samsung-soc at vger.kernel.org; Pawel
> Moll; Arnd Bergmann; Ian Campbell; Grant Grundler; Stephen Warren; linux-
> kernel at vger.kernel.org; Marc Zyngier; Linux IOMMU; Rob Herring; Kumar
> Gala; linux-tegra at vger.kernel.org; Cho KyongHo; Dave P Martin; linux-arm-
> kernel at lists.infradead.org
> Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
> bindings
> 
> Hi Stuart,
> 
> On Mon, Jun 16, 2014 at 05:56:32PM +0100, Stuart Yoder wrote:
> > > Do you have use-cases where you really need to change these mappings
> > > dynamically?
> >
> > Yes.  In the case of a PCI bus-- you may not know in advance how many
> > PCI devices there are until you probe the bus.   We have another FSL
> > proprietary bus we call the "fsl-mc" bus that is similar.
> 
> For that case, though, you could still describe an algorithmic
> transformation from RequesterID to StreamID which corresponds to a fixed
> mapping.
> 
> > Another thing to consider-- starting with SMMUv2, as you know, there
> > is a new distributed architecture with multiple TBUs and a centralized
> > TCU that walks the SMMU page tables.  So instead of sprinkling multiple
> > SMMUs all over an SoC you now have the option a 1 central TCU and
> sprinkling
> > multiple TBUs around.   However, this means that the stream ID
> namespace
> > is now global and can be pretty limited.  In the SMMU implementation we
> > have there are only 64 stream ID total for our Soc.  But we have many
> more
> > masters than that.
> >
> > So we look at stream IDs as really corresponding to an 'isolation
> context'
> > and not to a bus master.  An isolation context is the domain you are
> > trying to isolate with the SMMU.  Devices that all belong to the same
> > 'isolation context' can share the same stream ID, since they share
> > the same domain and page tables.
> 
> Ok, this is more compelling.
> 
> > So, perhaps by default some/most SMMU masters may have a default stream
> ID
> > of 0x0 that is used by the host...and that could be represented
> > statically in the device tree.
> >
> > But, we absolutely will need to dynamically set new stream IDs
> > into masters when a new IOMMU 'domain' is created and devices
> > are added to it.   All the devices in a domain will share
> > the same stream ID.
> >
> > So whatever we do, let's please have an architecture flexible enough
> > to allow for this.
> 
> What is the software interface to the logic that assigns the StreamIDs?
> Is
> it part of the SMMU, or a separate device (or set of devices)?

For us at the hardware level there are a few different ways that the streamIDs
can be set.  It is not part of the SMMU.  In the cases where there is simply
1 device to 1 streamID (e.g. USB controller) there is an SoC register where
you just program the stream ID.   In the case of PCI, our PCI controller
has a RequesterID-to-streamID table that you set via some PCI controller
registers.

The way we generally thought it would work was something like
this:
   -u-boot/bootloader makes any static streamID allocation if needed,
    sets a default streamID  (e.g. 0x0) in device and expresses
    that in the device tree
   -device tree would express relationship between devices
    (including bus controllers) and the SMMU through mmu-masters
    property
   -u-boot would express the range of unused (or used) streamIDs via a new
    device tree property so the kernel SMMU driver knows what streamIDs are
    free
   -in the SMMU driver a different vendor specific 'add_device' callback
    could be used to handle our special cases where we need to set/change
    the stream ID for devices added to a domain

Thanks,
Stuart