Capture image from imx219 sensor on i.MX8MM
Tim Harvey
tharvey at gateworks.com
Fri Jul 7 14:57:39 PDT 2023
On Tue, Feb 14, 2023 at 5:57 AM Frieder Schrempf
<frieder.schrempf at kontron.de> wrote:
>
> On 14.02.23 14:19, Frieder Schrempf wrote:
> > On 14.02.23 11:52, Laurent Pinchart wrote:
> >> Hi Frieder,
> >>
> >> On Tue, Feb 14, 2023 at 11:18:33AM +0100, Frieder Schrempf wrote:
> >>> On 09.02.23 23:19, Laurent Pinchart wrote:
> >>>> On Tue, Feb 07, 2023 at 04:24:45PM +0100, Frieder Schrempf wrote:
> >>>>> On 06.02.23 16:50, Laurent Pinchart wrote:
> >>>>>> On Mon, Feb 06, 2023 at 04:38:27PM +0100, Frieder Schrempf wrote:
> >>>>>>> On 06.02.23 16:31, Laurent Pinchart wrote:
> >>>>>>>> On Mon, Feb 06, 2023 at 03:24:41PM +0100, Frieder Schrempf wrote:
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> I'm trying to get a camera stream on our i.MX8MM board up and running
> >>>>>>>>> with Linux 6.1. I'm using the RPi v2.1 camera module (imx219) as sensor.
> >>>>>>>>>
> >>>>>>>>> I basically copied the devicetree setup from Tim's overlay at
> >>>>>>>>> arch/arm64/boot/dts/freescale/imx8mm-venice-gw72xx-0x-imx219.dts.
> >>>>>>>>>
> >>>>>>>>> With all the drivers enabled the devices (imx219, imx7-media-csi,
> >>>>>>>>> imx-mipi-csis) seem to probe just fine and I see /dev/video0 and
> >>>>>>>>> /dev/media0.
> >>>>>>>>>
> >>>>>>>>> For the v4l subdevices I would expect to see one for the sensor and one
> >>>>>>>>> for the CSI bridge. But only the latter is there (see below).
> >>>>>>>>>
> >>>>>>>>> Is this correct? Am I missing something? How can I setup/enable the
> >>>>>>>>> pipeline/stream?
> >>>>>>>>
> >>>>>>>> Your expectations are correct, but the result isn't. You should see the
> >>>>>>>> camera sensor in the media graph. Has the imx219 been probed by the
> >>>>>>>> driver ? Did probe succeed ?
> >>>>>>>
> >>>>>>> Yes, the imx219_probe() works just fine and returns with code 0. I also
> >>>>>>> see v4l2_async_register_subdev_sensor() and the underlying code being
> >>>>>>> called.
> >>>>>>>
> >>>>>>> So I really don't understand why I'm missing the sensor subdev. Any
> >>>>>>> ideas how to continue debugging?
> >>>>>>
> >>>>>> The issue seems to be with v4l2-async then. There's a debugfs file that
> >>>>>> lists the missing v4l2-async subdevs (I don't recall the path by heard),
> >>>>>> that can give useful information. Enabling the debug messages from
> >>>>>> v4l2-async.c may helpt too.
> >>>>>
> >>>>> +Cc: Dave, Sakari, Javier (thanks for your help and debugging hints via
> >>>>> mail/IRC)
> >>>>>
> >>>>> /sys/kernel/debug/v4l2-async/pending_async_subdevices gives me:
> >>>>>
> >>>>> csis-32e30000.mipi-csi:
> >>>>> [fwnode] dev=1-0010, node=/soc at 0/bus at 30800000/i2c at 30a30000/sensor at 10/port/endpoint
> >>>>> imx219 1-0010:
> >>>>> imx-media:
> >>>>>
> >>>>> So it looks like the async subdev for the imx219 sensor is still pending.
> >>>>
> >>>> Indeed, that seems to be the problem.
> >>>>
> >>>>> I have spent quite a few hours trying to understand how the code is
> >>>>> supposed to work, but I'm afraid it's too much for my brain at the moment ;)
> >>>>>
> >>>>> I have attached a debug patch [1] adding printks in v4l2-async.c and the
> >>>>> boot log [2] with all the output below. Hopefully someone can have a
> >>>>> look and make something out of that. Feel free to let me know via email
> >>>>> or IRC if I can provide any further information.
> >>>>
> >>>> To be honest your debug messages are quite cryptic. They may be readable
> >>>> when tracing the code on a live system, but offline, that's a different
> >>>> story.
> >>>
> >>> I know. Sorry for that. I just posted what I hacked together for
> >>> understanding the code flow. Sakari's debug patch is a better start, but
> >>> it doesn't really cover the problem I see, as the matching code is never
> >>> called. The issue is elsewhere and causes the fwnode matching to not
> >>> take place at all.
> >>>
> >>>> More comments below.
> >>>>
> >>>>> [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
> >>>>> [ 0.000000] Linux version 6.2.0-rc7-ktn (oe-user at oe-host) (aarch64-ktn-linux-gcc (GCC) 11.3.0, GNU ld (GNU Binutils) 2.38.20220708) #1 SMP PREEMPT Tue Feb 7 08:25:13 UTC 2023
> >>>>> [ 0.000000] Machine model: Kontron BL i.MX8MM (N801X S)
> >>>>> [ 0.000000] efi: UEFI not found.
> >>>>> [ 0.000000] NUMA: No NUMA configuration found
> >>>>> [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000013fffffff]
> >>>>> [ 0.000000] NUMA: NODE_DATA [mem 0x13f7c2b00-0x13f7c4fff]
> >>>>> [ 0.000000] Zone ranges:
> >>>>> [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]
> >>>>> [ 0.000000] DMA32 empty
> >>>>> [ 0.000000] Normal [mem 0x0000000100000000-0x000000013fffffff]
> >>>>> [ 0.000000] Movable zone start for each node
> >>>>> [ 0.000000] Early memory node ranges
> >>>>> [ 0.000000] node 0: [mem 0x0000000040000000-0x000000013fffffff]
> >>>>> [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000013fffffff]
> >>>>> [ 0.000000] cma: Reserved 1024 MiB at 0x00000000bbe00000
> >>>>> [ 0.000000] psci: probing for conduit method from DT.
> >>>>> [ 0.000000] psci: PSCIv1.1 detected in firmware.
> >>>>> [ 0.000000] psci: Using standard PSCI v0.2 function IDs
> >>>>> [ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
> >>>>> [ 0.000000] psci: SMC Calling Convention v1.2
> >>>>> [ 0.000000] percpu: Embedded 20 pages/cpu s42024 r8192 d31704 u81920
> >>>>> [ 0.000000] Detected VIPT I-cache on CPU0
> >>>>> [ 0.000000] CPU features: detected: GIC system register CPU interface
> >>>>> [ 0.000000] CPU features: detected: ARM erratum 845719
> >>>>> [ 0.000000] alternatives: applying boot alternatives
> >>>>> [ 0.000000] Fallback order for Node 0: 0
> >>>>> [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1032192
> >>>>> [ 0.000000] Policy zone: Normal
> >>>>> [ 0.000000] Kernel command line: root=/dev/nfs ip=192.168.1.11:192.168.1.10:::kontron-mx8mm:eth0:any nfsroot=192.168.1.10:/nfsroot,v3,tcp cma=1G rootwait
> >>>>> [ 0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
> >>>>> [ 0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
> >>>>> [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
> >>>>> [ 0.000000] software IO TLB: area num 4.
> >>>>> [ 0.000000] software IO TLB: mapped [mem 0x00000000fbfff000-0x00000000fffff000] (64MB)
> >>>>> [ 0.000000] Memory: 2976216K/4194304K available (13568K kernel code, 1222K rwdata, 4840K rodata, 2368K init, 545K bss, 169512K reserved, 1048576K cma-reserved)
> >>>>> [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
> >>>>> [ 0.000000] rcu: Preemptible hierarchical RCU implementation.
> >>>>> [ 0.000000] rcu: RCU event tracing is enabled.
> >>>>> [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
> >>>>> [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
> >>>>> [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
> >>>>> [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
> >>>>> [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
> >>>>> [ 0.000000] GICv3: 128 SPIs implemented
> >>>>> [ 0.000000] GICv3: 0 Extended SPIs implemented
> >>>>> [ 0.000000] Root IRQ handler: gic_handle_irq
> >>>>> [ 0.000000] GICv3: GICv3 features: 16 PPIs
> >>>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000038880000
> >>>>> [ 0.000000] ITS: No ITS available, not enabling LPIs
> >>>>> [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
> >>>>> [ 0.000000] arch_timer: cp15 timer(s) running at 8.00MHz (phys).
> >>>>> [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 440795202120 ns
> >>>>> [ 0.000000] sched_clock: 56 bits at 8MHz, resolution 125ns, wraps every 2199023255500ns
> >>>>> [ 0.000353] Console: colour dummy device 80x25
> >>>>> [ 0.000363] printk: console [tty0] enabled
> >>>>> [ 0.000858] Calibrating delay loop (skipped), value calculated using timer frequency.. 16.00 BogoMIPS (lpj=32000)
> >>>>> [ 0.000881] pid_max: default: 32768 minimum: 301
> >>>>> [ 0.000940] LSM: initializing lsm=capability,integrity
> >>>>> [ 0.001043] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
> >>>>> [ 0.001072] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
> >>>>> [ 0.002504] rcu: Hierarchical SRCU implementation.
> >>>>> [ 0.002526] rcu: Max phase no-delay instances is 1000.
> >>>>> [ 0.003357] EFI services will not be available.
> >>>>> [ 0.003583] smp: Bringing up secondary CPUs ...
> >>>>> [ 0.004061] Detected VIPT I-cache on CPU1
> >>>>> [ 0.004136] GICv3: CPU1: found redistributor 1 region 0:0x00000000388a0000
> >>>>> [ 0.004178] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
> >>>>> [ 0.004649] Detected VIPT I-cache on CPU2
> >>>>> [ 0.004712] GICv3: CPU2: found redistributor 2 region 0:0x00000000388c0000
> >>>>> [ 0.004736] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
> >>>>> [ 0.005177] Detected VIPT I-cache on CPU3
> >>>>> [ 0.005238] GICv3: CPU3: found redistributor 3 region 0:0x00000000388e0000
> >>>>> [ 0.005259] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
> >>>>> [ 0.005316] smp: Brought up 1 node, 4 CPUs
> >>>>> [ 0.005403] SMP: Total of 4 processors activated.
> >>>>> [ 0.005414] CPU features: detected: 32-bit EL0 Support
> >>>>> [ 0.005424] CPU features: detected: 32-bit EL1 Support
> >>>>> [ 0.005440] CPU features: detected: CRC32 instructions
> >>>>> [ 0.005506] CPU: All CPU(s) started at EL2
> >>>>> [ 0.005524] alternatives: applying system-wide alternatives
> >>>>> [ 0.007432] devtmpfs: initialized
> >>>>> [ 0.013542] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
> >>>>> [ 0.013594] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
> >>>>> [ 0.043563] pinctrl core: initialized pinctrl subsystem
> >>>>> [ 0.045549] DMI not present or invalid.
> >>>>> [ 0.046159] NET: Registered PF_NETLINK/PF_ROUTE protocol family
> >>>>> [ 0.047073] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
> >>>>> [ 0.047276] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
> >>>>> [ 0.047441] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
> >>>>> [ 0.047501] audit: initializing netlink subsys (disabled)
> >>>>> [ 0.047623] audit: type=2000 audit(0.044:1): state=initialized audit_enabled=0 res=1
> >>>>> [ 0.048000] thermal_sys: Registered thermal governor 'step_wise'
> >>>>> [ 0.048033] cpuidle: using governor menu
> >>>>> [ 0.048199] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
> >>>>> [ 0.048287] ASID allocator initialised with 65536 entries
> >>>>> [ 0.048955] Serial: AMBA PL011 UART driver
> >>>>> [ 0.055254] imx8mm-pinctrl 30330000.pinctrl: initialized IMX pinctrl driver
> >>>>> [ 0.066886] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
> >>>>> [ 0.066917] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
> >>>>> [ 0.066932] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages
> >>>>> [ 0.066944] HugeTLB: 508 KiB vmemmap can be freed for a 32.0 MiB page
> >>>>> [ 0.066958] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
> >>>>> [ 0.066972] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
> >>>>> [ 0.066987] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages
> >>>>> [ 0.067000] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page
> >>>>> [ 0.068588] ACPI: Interpreter disabled.
> >>>>> [ 0.069385] iommu: Default domain type: Translated
> >>>>> [ 0.069405] iommu: DMA domain TLB invalidation policy: strict mode
> >>>>> [ 0.069649] SCSI subsystem initialized
> >>>>> [ 0.069932] usbcore: registered new interface driver usbfs
> >>>>> [ 0.069969] usbcore: registered new interface driver hub
> >>>>> [ 0.070007] usbcore: registered new device driver usb
> >>>>> [ 0.070643] mc: Linux media interface: v0.10
> >>>>> [ 0.070690] videodev: Linux video capture interface: v2.00
> >>>>> [ 0.070754] pps_core: LinuxPPS API ver. 1 registered
> >>>>> [ 0.070765] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti at linux.it>
> >>>>> [ 0.070791] PTP clock support registered
> >>>>> [ 0.070920] EDAC MC: Ver: 3.0.0
> >>>>> [ 0.071496] FPGA manager framework
> >>>>> [ 0.071573] Advanced Linux Sound Architecture Driver Initialized.
> >>>>> [ 0.072353] vgaarb: loaded
> >>>>> [ 0.072652] clocksource: Switched to clocksource arch_sys_counter
> >>>>> [ 0.072831] VFS: Disk quotas dquot_6.6.0
> >>>>> [ 0.072867] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> >>>>> [ 0.073038] pnp: PnP ACPI: disabled
> >>>>> [ 0.079203] NET: Registered PF_INET protocol family
> >>>>> [ 0.079432] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
> >>>>> [ 0.082247] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
> >>>>> [ 0.082335] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
> >>>>> [ 0.082361] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
> >>>>> [ 0.082592] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
> >>>>> [ 0.083553] TCP: Hash tables configured (established 32768 bind 32768)
> >>>>> [ 0.083705] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
> >>>>> [ 0.083806] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
> >>>>> [ 0.084021] NET: Registered PF_UNIX/PF_LOCAL protocol family
> >>>>> [ 0.084433] RPC: Registered named UNIX socket transport module.
> >>>>> [ 0.084451] RPC: Registered udp transport module.
> >>>>> [ 0.084462] RPC: Registered tcp transport module.
> >>>>> [ 0.084474] RPC: Registered tcp NFSv4.1 backchannel transport module.
> >>>>> [ 0.084493] PCI: CLS 0 bytes, default 64
> >>>>> [ 0.085132] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available
> >>>>> [ 0.085846] kvm [1]: IPA Size Limit: 40 bits
> >>>>> [ 0.087132] kvm [1]: GICv3: no GICV resource entry
> >>>>> [ 0.087146] kvm [1]: disabling GICv2 emulation
> >>>>> [ 0.087164] kvm [1]: GIC system register CPU interface enabled
> >>>>> [ 0.087258] kvm [1]: vgic interrupt IRQ9
> >>>>> [ 0.087360] kvm [1]: Hyp mode initialized successfully
> >>>>> [ 0.088486] Initialise system trusted keyrings
> >>>>> [ 0.088687] workingset: timestamp_bits=42 max_order=20 bucket_order=0
> >>>>> [ 0.088988] squashfs: version 4.0 (2009/01/31) Phillip Lougher
> >>>>> [ 0.089207] NFS: Registering the id_resolver key type
> >>>>> [ 0.089262] Key type id_resolver registered
> >>>>> [ 0.089274] Key type id_legacy registered
> >>>>> [ 0.089303] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
> >>>>> [ 0.089317] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
> >>>>> [ 0.089466] 9p: Installing v9fs 9p2000 file system support
> >>>>> [ 0.124669] Key type asymmetric registered
> >>>>> [ 0.124683] Asymmetric key parser 'x509' registered
> >>>>> [ 0.124735] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 244)
> >>>>> [ 0.124753] io scheduler mq-deadline registered
> >>>>> [ 0.124764] io scheduler kyber registered
> >>>>> [ 0.128006] EINJ: ACPI disabled.
> >>>>> [ 0.136799] SoC: i.MX8MM revision 1.0
> >>>>> [ 0.141870] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> >>>>> [ 0.143967] 30860000.serial: ttymxc0 at MMIO 0x30860000 (irq = 16, base_baud = 1500000) is a IMX
> >>>>> [ 0.144488] 30880000.serial: ttymxc2 at MMIO 0x30880000 (irq = 17, base_baud = 1500000) is a IMX
> >>>>> [ 0.144522] imx-uart 30880000.serial: Console IMX rounded baud rate from 114286 to 114300
> >>>>> [ 0.144571] printk: console [ttymxc2] enabled
> >>>>> [ 1.126108] 30890000.serial: ttymxc1 at MMIO 0x30890000 (irq = 18, base_baud = 1500000) is a IMX
> >>>>> [ 1.144256] loop: module loaded
> >>>>> [ 1.150889] tun: Universal TUN/TAP device driver, 1.6
> >>>>> [ 1.156153] CAN device driver interface
> >>>>> [ 1.160606] thunder_xcv, ver 1.0
> >>>>> [ 1.163879] thunder_bgx, ver 1.0
> >>>>> [ 1.167145] nicpf, ver 1.0
> >>>>> [ 1.170627] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
> >>>>> [ 1.177871] hns3: Copyright (c) 2017 Huawei Corporation.
> >>>>> [ 1.183231] hclge is initializing
> >>>>> [ 1.186586] e1000e: Intel(R) PRO/1000 Network Driver
> >>>>> [ 1.191565] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
> >>>>> [ 1.197519] igb: Intel(R) Gigabit Ethernet Network Driver
> >>>>> [ 1.202932] igb: Copyright (c) 2007-2014 Intel Corporation.
> >>>>> [ 1.208538] igbvf: Intel(R) Gigabit Virtual Function Network Driver
> >>>>> [ 1.214821] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
> >>>>> [ 1.220881] sky2: driver version 1.30
> >>>>> [ 1.225044] VFIO - User Level meta-driver version: 0.3
> >>>>> [ 1.231934] usbcore: registered new interface driver usb-storage
> >>>>> [ 1.240846] snvs_rtc 30370000.snvs:snvs-rtc-lp: registered as rtc0
> >>>>> [ 1.247080] snvs_rtc 30370000.snvs:snvs-rtc-lp: setting system clock to 1970-01-01T00:00:00 UTC (0)
> >>>>> [ 1.256271] i2c_dev: i2c /dev entries driver
> >>>>> [ 1.261317] imx7-csi 32e20000.csi: Registered csi capture as /dev/video0
> >>>>> [ 1.268186] __v4l2_async_nf_register:610
> >>>>> [ 1.272130] imx-media:
> >>>>> [ 1.274676] v4l2_async_nf_try_all_subdevs:447
> >>>>> [ 1.279048] imx-media:
> >>>>> [ 1.281593] v4l2_async_nf_try_all_subdevs:454
> >>>>> [ 1.285965] imx7_csi_notify_complete:2178
> >>>>
> >>>> This doesn't seem right. The complete function should only be called
> >>>> once all the subdevices required by the notifier have been found, and at
> >>>> this point, the csis hasn't even probed. You may want to check the
> >>>> device tree.
> >>>
> >>> But it looks like the notifier for the csi doesn't have any subdev
> >>> dependencies. The "imx-media: " shows that a notifier for the csi is in
> >>> the notifier_list, but the notif->waiting is empty.
> >>>
> >>> Is that correct? How is the dependency graph supposed to look like? Is
> >>> it csi -> csis -> imx219 or the other way round? In the latter case the
> >>> imx219 should wait for all the other subdevs, right?
> >>
> >> The imx7-media-csi driver registers a notifier in
> >> imx7_csi_async_register(), which should have one entry corresponding to
> >> the imx-mipi-csis.
> >>
> >> The code seems to silently ignore some errors, which I don't think is
> >> right. It originates from the shared code base with the i.MX6 IPUv3
> >> drivers, and should be fixed now. I'd start by checking what happens
> >> there.
> >
> > Ok, I had a look at imx7_csi_async_register() and the call to
> > fwnode_graph_get_endpoint_by_id() returns NULL. This probably should
> > already cause the probe to fail, but is currently ignored.
> >
> > It looks like the remote endpoint lookup cannot be resolved as the csi
> > is probed first and the remote device (mipi_csis) is not yet available.
> >
> > How is this supposed to work in the first place? Do we need to defer the
> > probe of the csi until the mipi_csis is available?
>
> As discussed on IRC, in the attachment there is the DTB from the device
> for further debugging.
Frieder,
Where did you end up with this? I have imx219 capture working on
imx8mm on a 5.15 kernel with lots of backports but have not managed to
repeat it on a 6.4 kernel. I don't run into the issue you are running
into (I have the imx-media device showing) but the media-ctl entities
have changed since my 5.15 kernel and I can't figure out how to link
them up and configure them with media-ctl.
On my imx8mm-venice-gw72xx with imx8mm-venice-gw72xx-0x-imx219.dtso I see:
root at jammy-venice:~# cat /sys/bus/media/devices/media*/model
imx-media
hantro-vpu
hantro-vpu
root at jammy-venice:~# cat /sys/bus/media/devices/media0/model
imx-media
root at jammy-venice:~# media-ctl --device /dev/media0 -p
Media controller API version 6.4.0
Media device information
------------------------
driver imx7-csi
model imx-media
serial
bus info platform:32e20000.csi
hw revision 0x0
driver version 6.4.0
Device topology
- entity 1: csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:UYVY8_2X8/640x480 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
<- "csis-32e30000.mipi-csi":1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:UYVY8_2X8/640x480 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
-> "csi capture":0 [ENABLED,IMMUTABLE]
- entity 4: csi capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "csi":1 [ENABLED,IMMUTABLE]
- entity 10: csis-32e30000.mipi-csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:UYVY8_1X16/640x480 field:none
colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
<- "imx219 2-0010":0 []
pad1: Source
[fmt:UYVY8_1X16/640x480 field:none
colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
-> "csi":0 [ENABLED,IMMUTABLE]
- entity 15: imx219 2-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:SRGGB10_1X10/3280x2464 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range
crop.bounds:(8,8)/3280x2464
crop:(8,8)/3280x2464]
-> "csis-32e30000.mipi-csi":0 []
When I try to link and configure the pipeline I run into issues I
don't know how to resolve:
root at jammy-venice:~# media-ctl --device /dev/media0 --reset
root at jammy-venice:~# media-ctl --device /dev/media0 --links "'imx219
2-0010':0->'csis-32e30000.mipi-csi.0':0[1]"
Unable to parse link: Invalid argument (22)
^^^ not sure why I can't link imx219 source pad with mipi-csi sink pad
What changed for me from my 5.15 kernel is the entities and pads
above. My 5.15 kernel looks like this:
root at jammy-venice:~# cat /sys/bus/media/devices/media2/model
imx-media
root at jammy-venice:~# media-ctl --device /dev/media2 -p
Media controller API version 5.15.15
Media device information
------------------------
driver imx7-csi
model imx-media
serial
bus info platform:32e20000.csi
hw revision 0x0
driver version 5.15.15
Device topology
- entity 1: csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:UYVY8_2X8/800x600 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
<- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:UYVY8_2X8/800x600 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:lim-range]
-> "csi capture":0 [ENABLED,IMMUTABLE]
- entity 4: csi capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video2
pad0: Sink
<- "csi":1 [ENABLED,IMMUTABLE]
- entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:UYVY8_1X16/640x480 field:none
colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
<- "imx219 2-0010":0 []
pad1: Source
[fmt:UYVY8_1X16/640x480 field:none
colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
-> "csi":0 [ENABLED,IMMUTABLE]
- entity 15: imx219 2-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:SRGGB10_1X10/3280x2464 field:none colorspace:srgb
xfer:srgb ycbcr:601 quantization:full-range
crop.bounds:(8,8)/3280x2464
crop:(8,8)/3280x2464]
-> "imx7-mipi-csis.0":0 []
# configure media controller links
media-ctl --device /dev/media2 --reset
# link the imx219 source pad (0) to the imx7-mipi-csis sink pad (0)
media-ctl --device /dev/media2 --links "'imx219
2-0010':0->'imx7-mipi-csis.0':0[1]"
# configure the imx219 source pad (0) for srgb 640x480
media-ctl --device /dev/media2 -v -V "'imx219 2-0010':0
[fmt:SRGGB8/640x480 field:none]"
# configure the csi sink pad (0) for srgb 640x480
media-ctl --device /dev/media2 -v -V "'csi':0 [fmt:SRGGB8/640x480 field:none]"
# stream 640x480 at 30fps
gst-launch-1.0 v4l2src device=/dev/video0 !
video/x-bayer,format=rggb,width=640,height=480,framerate=10/1 !
bayer2rgb ! fbdevsink # works
Maybe you have gotten further than this by now?
If you are still having issues getting imx-media to register we can
look at kernel configs. I believe the following are necessary:
CONFIG_VIDEO_IMX219=y
CONFIG_VIDEO_IMX_MIPI_CSIS=y
CONFIG_VIDEO_IMX7_CSI=y
Best regards,
Tim
More information about the linux-arm-kernel
mailing list