[PATCH v7 00/22] Generic DT bindings for PCI IOMMUs and ARM SMMU

[Robin Murphy]

Hi Eric,

On 15/09/16 17:46, Auger Eric wrote:
[...]
> Hum OK; thanks for the explanation. With that implementation however,
> don't we face back the issue we encountered in early stage of default
> domain implementation:
> 
> With this sample config (AMD overdrive + I350-T2 + 2VFs per PF) I fill
> the 8 context banks. Whereas practically we didn't need them before?
> 
> 00:00.0 0600: 1022:1a00
> 	Subsystem: 1022:1a00
> 00:02.0 0600: 1022:1a01
> 00:02.2 0604: 1022:1a02
> 	Kernel driver in use: pcieport
> 01:00.0 0200: 8086:1521 (rev 01)
> 	Subsystem: 8086:0002
> 	Kernel driver in use: igb
> 01:00.1 0200: 8086:1521 (rev 01)
> 	Subsystem: 8086:0002
> 	Kernel driver in use: igb
> 01:10.0 0200: 8086:1520 (rev 01) -> context 5
> 	Subsystem: 8086:0002
> 	Kernel driver in use: vfio-pci
> 01:10.1 0200: 8086:1520 (rev 01) -> context 7
> 	Subsystem: 8086:0002
> 	Kernel driver in use: igbvf
> 01:10.4 0200: 8086:1520 (rev 01) -> context 6
> 	Subsystem: 8086:0002
> 	Kernel driver in use: igbvf
> 01:10.5 0200: 8086:1520 (rev 01) -> shortage
> 	Subsystem: 8086:0002
> 	Kernel driver in use: igbvf
> 
> So I can't even do passthrough anymore with that config. Is there
> anything wrong in my setup/understanding?

It's kind of hard to avoid, really - people want DMA ops support (try
plugging a card which can only do 32-bit DMA into that Seattle, for
instance); DMA ops need default domains; default domains are allocated
per group; each domain requires a context bank to back it; thus if you
have 9 groups and 8 context banks you're in a corner. It would relieve
the pressure to have a single default domain per SMMU, but we've no way
to do that due to the way the iommu_domain_alloc() API is intended to work.

Ultimately, it's a hardware limitation of that platform - plug in a card
with 16 VFs with ACS, and either way you're stuck. There are a number of
bodges I can think of that would make your specific situation work, but
none of them are really sufficiently general to consider upstreaming.
The most logical thing to do right now, if you were happy without DMA
ops using the old binding before, is to keep using the old binding (just
fixing your DT with #stream-id-cells=0 on the host controller so as not
to create the fake aliasing problem). Hopefully future platforms will be
in a position to couple their PCI host controllers to an IOMMU which is
actually designed to support a PCI host controller.

What I probably will do, though, since we have the functionality in
place for the sake of the old binding, and I think there are other folks
who want PCI iommu-map support but would prefer not to bother with DMA
ops on the host, is add a command-line option to disable DMA domains
even for the generic bindings.

Robin.