[PATCH v6 0/5] PCI: Unify ECAM constants in native PCI Express drivers
Michael Walle
michael at walle.cc
Wed Dec 9 08:08:00 EST 2020
[+ Vladimir and Alex]
Am 2020-12-09 13:36, schrieb Bjorn Helgaas:
> On Tue, Dec 08, 2020 at 04:41:50PM +0100, Michael Walle wrote:
>> >On Sun, 29 Nov 2020 23:07:38 +0000, Krzysztof Wilczyński wrote:
>> >> Unify ECAM-related constants into a single set of standard constants
>> >> defining memory address shift values for the byte-level address that can
>> >> be used when accessing the PCI Express Configuration Space, and then
>> >> move native PCI Express controller drivers to use newly introduced
>> >> definitions retiring any driver-specific ones.
>> >>
>> >> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
>> >> PCI Express specification (see PCI Express Base Specification, Revision
>> >> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
>> >> implement it the same way.
>> >>
>> >> [...]
>> >
>> >Applied to pci/ecam, thanks!
>> >
>> >[1/5] PCI: Unify ECAM constants in native PCI Express drivers
>> > https://git.kernel.org/lpieralisi/pci/c/f3c07cf692
>
>> Patch 1/5 breaks LS1028A boards:
>>
>> [..]
>> [ 1.144426] pci-host-generic 1f0000000.pcie: host bridge
>> /soc/pcie at 1f0000000 ranges:
>> [ 1.152276] pci-host-generic 1f0000000.pcie: MEM
>> 0x01f8000000..0x01f815ffff -> 0x0000000000
>> [ 1.161161] pci-host-generic 1f0000000.pcie: MEM
>> 0x01f8160000..0x01f81cffff -> 0x0000000000
>> [ 1.170043] pci-host-generic 1f0000000.pcie: MEM
>> 0x01f81d0000..0x01f81effff -> 0x0000000000
>> [ 1.178924] pci-host-generic 1f0000000.pcie: MEM
>> 0x01f81f0000..0x01f820ffff -> 0x0000000000
>> [ 1.187805] pci-host-generic 1f0000000.pcie: MEM
>> 0x01f8210000..0x01f822ffff -> 0x0000000000
>> [ 1.196686] pci-host-generic 1f0000000.pcie: MEM
>> 0x01f8230000..0x01f824ffff -> 0x0000000000
>> [ 1.205562] pci-host-generic 1f0000000.pcie: MEM
>> 0x01fc000000..0x01fc3fffff -> 0x0000000000
>
> Can you attach your DT?
That would be the following:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/freescale/fsl-ls1028a-kontron-sl28-var3-ads2.dts
You'll find the PCI devices/bridge in:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi
> The fact that all these windows map to PCI
> bus address 0 looks broken. Prior to patch 1/5, do the devices below
> this bridge actually work?
Yes, these should be the onboard network controller and ethernet
switch.
> Looks like you're using the pci-host-generic driver; which of the
> .compatible strings (pci-host-cam-generic, pci-host-ecam-generic,
> marvell,armada8k-pcie-ecam, etc) are you using? (I think that's in
> the DT as well.)
compatible = "pci-host-ecam-generic";
>> [ 1.214465] pci-host-generic 1f0000000.pcie: ECAM at [mem
>> 0x1f0000000-0x1f00fffff] for [bus 00]
>
>
>> [ 1.223318] pci-host-generic 1f0000000.pcie: PCI host bridge to bus
>> 0000:00
>> [ 1.230350] pci_bus 0000:00: root bus resource [bus 00]
>> [ 1.235625] pci_bus 0000:00: root bus resource [mem
>> 0x1f8000000-0x1f815ffff] (bus address [0x00000000-0x0015ffff])
>> [ 1.246077] pci_bus 0000:00: root bus resource [mem
>> 0x1f8160000-0x1f81cffff pref] (bus address [0x00000000-0x0006ffff])
>> [ 1.256969] pci_bus 0000:00: root bus resource [mem
>> 0x1f81d0000-0x1f81effff] (bus address [0x00000000-0x0001ffff])
>> [ 1.267427] pci_bus 0000:00: root bus resource [mem
>> 0x1f81f0000-0x1f820ffff pref] (bus address [0x00000000-0x0001ffff])
>> [ 1.278326] pci_bus 0000:00: root bus resource [mem
>> 0x1f8210000-0x1f822ffff] (bus address [0x00000000-0x0001ffff])
>> [ 1.288779] pci_bus 0000:00: root bus resource [mem
>> 0x1f8230000-0x1f824ffff pref] (bus address [0x00000000-0x0001ffff])
>> [ 1.299669] pci_bus 0000:00: root bus resource [mem
>> 0x1fc000000-0x1fc3fffff] (bus address [0x00000000-0x003fffff])
>> [ 1.310138] pci 0000:00:00.0: [1957:e100] type 00 class 0x020001
>> [ 1.316234] pci 0000:00:00.0: BAR 0: [mem 0x1f8000000-0x1f803ffff
>> 64bit] (from Enhanced Allocation, properties 0x0)
>> [ 1.326776] pci 0000:00:00.0: BAR 2: [mem 0x1f8160000-0x1f816ffff
>> 64bit pref] (from Enhanced Allocation, properties 0x1)
>> [ 1.337759] pci 0000:00:00.0: VF BAR 0: [mem
>> 0x1f81d0000-0x1f81dffff 64bit] (from Enhanced Allocation, properties
>> 0x4)
>> [ 1.348563] pci 0000:00:00.0: VF BAR 2: [mem
>> 0x1f81f0000-0x1f81fffff 64bit pref] (from Enhanced Allocation,
>> properties 0x3)
>> [ 1.359821] pci 0000:00:00.0: PME# supported from D0 D3hot
>> [ 1.365368] pci 0000:00:00.0: VF(n) BAR0 space: [mem
>> 0x1f81d0000-0x1f81effff 64bit] (contains BAR0 for 2 VFs)
>> [ 1.375381] pci 0000:00:00.0: VF(n) BAR2 space: [mem
>> 0x1f81f0000-0x1f820ffff 64bit pref] (contains BAR2 for 2 VFs)
>> [ 1.385983] Unable to handle kernel paging request at virtual
>> address ffff800012132000
>
> If ffff800012132000 were an actual ECAM address, we would expect the
> low 20 bits to contain the device number, function number, and
> config register offset, i.e.,
>
> dev (0xffff800012132000 >> 15) & 0x01f = 0x6
> fn (0xffff800012132000 >> 12) & 0x007 = 0x2
> reg (0xffff800012132000) & 0xfff = 0
>
> but that's non-sensical since we probe for devices in order. So maybe
> this is a bad pointer somewhere else. I looked at pci_ecam_map_bus()
> but didn't see an obvious problem. Maybe we could brute-force debug
> this by adding some printks there.
I'm not that deep into PCI, but if you could tell me what to add where,
I'm happy to help ;)
>> [ 1.393972] Mem abort info:
>> [ 1.396783] ESR = 0x96000007
>> [ 1.399859] EC = 0x25: DABT (current EL), IL = 32 bits
>> [ 1.405215] SET = 0, FnV = 0
>> [ 1.408290] EA = 0, S1PTW = 0
>> [ 1.411453] Data abort info:
>> [ 1.414352] ISV = 0, ISS = 0x00000007
>> [ 1.418216] CM = 0, WnR = 0
>> [ 1.421205] swapper pgtable: 4k pages, 48-bit VAs,
>> pgdp=000000008369c000
>> [ 1.427966] [ffff800012132000] pgd=00000020fffff003,
>> p4d=00000020fffff003, pud=00000020ffffe003, pmd=00000020ffffa003,
>> pte=0000000000000000
>> [ 1.440618] Internal error: Oops: 96000007 [#1] PREEMPT SMP
>> [ 1.446239] Modules linked in:
>> [ 1.449320] CPU: 1 PID: 1 Comm: swapper/0 Not tainted
>> 5.10.0-rc3-00101-g2f378db5c89 #191
>> [ 1.457484] Hardware name: Kontron SMARC-sAL28 (Single PHY) on
>> SMARC Eval 2.0 carrier (DT)
>> [ 1.465827] pstate: 20000085 (nzCv daIf -PAN -UAO -TCO BTYPE=--)
>> [ 1.471892] pc : pci_generic_config_read+0x38/0xe0
>> [ 1.476723] lr : pci_generic_config_read+0x24/0xe0
>> [ 1.481553] sp : ffff80001211b920
>> [ 1.484891] x29: ffff80001211b920 x28: 0000000000000000
>> [ 1.490252] x27: ffff8000116a04bc x26: 0000000000000000
>> [ 1.495612] x25: 0000000000000001 x24: ffff80001211ba54
>> [ 1.500972] x23: ffff0020009c3800 x22: 0000000000000000
>> [ 1.506332] x21: 0000000000000087 x20: ffff80001211b994
>> [ 1.511692] x19: 0000000000000004 x18: 0000000000000000
>> [ 1.517052] x17: 0000000000000000 x16: 00000000d5edfbc1
>> [ 1.522412] x15: ffffffffffffffff x14: ffff800011cf9948
>> [ 1.527772] x13: ffff002000305a1c x12: 0000000000000030
>> [ 1.533132] x11: 0101010101010101 x10: 7f7f7f7f7f7f7f7f
>> [ 1.538491] x9 : 2c6b7173626d686f x8 : 000000000000ea60
>> [ 1.543851] x7 : ffff80001211ba54 x6 : 0000000000000000
>> [ 1.549211] x5 : 0000000000000000 x4 : ffff800012131000
>> [ 1.554570] x3 : 0000000000000000 x2 : 0000000000000000
>> [ 1.559930] x1 : 0000000000001000 x0 : ffff800012132000
>> [ 1.565290] Call trace:
>> [ 1.567752] pci_generic_config_read+0x38/0xe0
>> [ 1.572233] pci_bus_read_config_dword+0x84/0xd8
>> [ 1.576890] pci_bus_generic_read_dev_vendor_id+0x34/0x1b0
>> [ 1.582423] pci_bus_read_dev_vendor_id+0x4c/0x70
>> [ 1.587167] pci_scan_single_device+0x84/0xe0
>> [ 1.591559] pci_scan_slot+0x6c/0x120
>> [ 1.595250] pci_scan_child_bus_extend+0x54/0x298
>> [ 1.599994] pci_scan_root_bus_bridge+0xd4/0xf0
>> [ 1.604562] pci_host_probe+0x18/0xb0
>> [ 1.608254] pci_host_common_probe+0x13c/0x1a0
>> [ 1.612735] platform_drv_probe+0x54/0xa8
>> [ 1.616777] really_probe+0xe4/0x3b8
>> [ 1.620380] driver_probe_device+0x58/0xb8
>> [ 1.624509] device_driver_attach+0x74/0x80
>> [ 1.628725] __driver_attach+0x58/0xe0
>> [ 1.632503] bus_for_each_dev+0x74/0xc8
>> [ 1.636369] driver_attach+0x24/0x30
>> [ 1.639972] bus_add_driver+0x18c/0x1f0
>> [ 1.643838] driver_register+0x64/0x120
>> [ 1.647704] __platform_driver_register+0x48/0x58
>> [ 1.652449] gen_pci_driver_init+0x1c/0x28
>> [ 1.656580] do_one_initcall+0x4c/0x2c0
>> [ 1.660447] kernel_init_freeable+0x1e4/0x250
>> [ 1.664840] kernel_init+0x14/0x118
>> [ 1.668355] ret_from_fork+0x10/0x34
>> [ 1.671961] Code: 7100067f 540001c0 71000a7f 54000300 (b9400001)
>> [ 1.678114] ---[ end trace 0aca1b048661e8b3 ]---
>> [ 1.682770] note: swapper/0[1] exited with preempt_count 1
>> [ 1.688305] Kernel panic - not syncing: Attempted to kill init!
>> exitcode=0x0000000b
>> [ 1.696031] SMP: stopping secondary CPUs
>> [ 1.699989] Kernel Offset: disabled
>> [ 1.703503] CPU features: 0x0240022,61006008
>> [ 1.707806] Memory Limit: none
>> [ 1.710884] ---[ end Kernel panic - not syncing: Attempted to kill
>> init! exitcode=0x0000000b ]---
>>
>> There is a LS1028A eval board in kernelci here:
>> https://lavalab.nxp.com/scheduler/job/170566
>>
>> I actually have this board which also have a LS1028A SoC:
>> https://lavalab.kontron.com/scheduler/job/1771
>>
>> But in the latter you won't see much because earlycon isn't active.
>> [I'm
>> about to fix that.]
>>
>> By reverting patch 1/5, the board will work again.
>>
>> -michael
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
-michael
More information about the linux-arm-kernel
mailing list