Capture image from imx219 sensor on i.MX8MM
Frieder Schrempf
frieder.schrempf at kontron.de
Tue Jul 11 23:27:12 PDT 2023
Hi Tim,
On 11.07.23 22:02, Tim Harvey wrote:
> On Mon, Jul 10, 2023 at 12:58 AM Frieder Schrempf
> <frieder.schrempf at kontron.de> wrote:
>>
>> Hi Tim,
>>
>> On 07.07.23 23:57, Tim Harvey wrote:
>> [...]
>>>
>>> Frieder,
>>>
>>> Where did you end up with this? I have imx219 capture working on
>>> imx8mm on a 5.15 kernel with lots of backports but have not managed to
>>> repeat it on a 6.4 kernel. I don't run into the issue you are running
>>> into (I have the imx-media device showing) but the media-ctl entities
>>> have changed since my 5.15 kernel and I can't figure out how to link
>>> them up and configure them with media-ctl.
>>>
>>> On my imx8mm-venice-gw72xx with imx8mm-venice-gw72xx-0x-imx219.dtso I see:
>>> root at jammy-venice:~# cat /sys/bus/media/devices/media*/model
>>> imx-media
>>> hantro-vpu
>>> hantro-vpu
>>> root at jammy-venice:~# cat /sys/bus/media/devices/media0/model
>>> imx-media
>>> root at jammy-venice:~# media-ctl --device /dev/media0 -p
>>> Media controller API version 6.4.0
>>>
>>> Media device information
>>> ------------------------
>>> driver imx7-csi
>>> model imx-media
>>> serial
>>> bus info platform:32e20000.csi
>>> hw revision 0x0
>>> driver version 6.4.0
>>>
>>> Device topology
>>> - entity 1: csi (2 pads, 2 links)
>>> type V4L2 subdev subtype Unknown flags 0
>>> device node name /dev/v4l-subdev0
>>> pad0: Sink
>>> [fmt:UYVY8_2X8/640x480 field:none colorspace:srgb
>>> xfer:srgb ycbcr:601 quantization:lim-range]
>>> <- "csis-32e30000.mipi-csi":1 [ENABLED,IMMUTABLE]
>>> pad1: Source
>>> [fmt:UYVY8_2X8/640x480 field:none colorspace:srgb
>>> xfer:srgb ycbcr:601 quantization:lim-range]
>>> -> "csi capture":0 [ENABLED,IMMUTABLE]
>>>
>>> - entity 4: csi capture (1 pad, 1 link)
>>> type Node subtype V4L flags 0
>>> device node name /dev/video0
>>> pad0: Sink
>>> <- "csi":1 [ENABLED,IMMUTABLE]
>>>
>>> - entity 10: csis-32e30000.mipi-csi (2 pads, 2 links)
>>> type V4L2 subdev subtype Unknown flags 0
>>> device node name /dev/v4l-subdev1
>>> pad0: Sink
>>> [fmt:UYVY8_1X16/640x480 field:none
>>> colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
>>> <- "imx219 2-0010":0 []
>>> pad1: Source
>>> [fmt:UYVY8_1X16/640x480 field:none
>>> colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
>>> -> "csi":0 [ENABLED,IMMUTABLE]
>>>
>>> - entity 15: imx219 2-0010 (1 pad, 1 link)
>>> type V4L2 subdev subtype Sensor flags 0
>>> device node name /dev/v4l-subdev2
>>> pad0: Source
>>> [fmt:SRGGB10_1X10/3280x2464 field:none colorspace:srgb
>>> xfer:srgb ycbcr:601 quantization:full-range
>>> crop.bounds:(8,8)/3280x2464
>>> crop:(8,8)/3280x2464]
>>> -> "csis-32e30000.mipi-csi":0 []
>>>
>>> When I try to link and configure the pipeline I run into issues I
>>> don't know how to resolve:
>>> root at jammy-venice:~# media-ctl --device /dev/media0 --reset
>>> root at jammy-venice:~# media-ctl --device /dev/media0 --links "'imx219
>>> 2-0010':0->'csis-32e30000.mipi-csi.0':0[1]"
>>> Unable to parse link: Invalid argument (22)
>>> ^^^ not sure why I can't link imx219 source pad with mipi-csi sink pad
>>>
>>> What changed for me from my 5.15 kernel is the entities and pads
>>> above. My 5.15 kernel looks like this:
>>> root at jammy-venice:~# cat /sys/bus/media/devices/media2/model
>>> imx-media
>>> root at jammy-venice:~# media-ctl --device /dev/media2 -p
>>> Media controller API version 5.15.15
>>>
>>> Media device information
>>> ------------------------
>>> driver imx7-csi
>>> model imx-media
>>> serial
>>> bus info platform:32e20000.csi
>>> hw revision 0x0
>>> driver version 5.15.15
>>>
>>> Device topology
>>> - entity 1: csi (2 pads, 2 links)
>>> type V4L2 subdev subtype Unknown flags 0
>>> device node name /dev/v4l-subdev0
>>> pad0: Sink
>>> [fmt:UYVY8_2X8/800x600 field:none colorspace:srgb
>>> xfer:srgb ycbcr:601 quantization:lim-range]
>>> <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
>>> pad1: Source
>>> [fmt:UYVY8_2X8/800x600 field:none colorspace:srgb
>>> xfer:srgb ycbcr:601 quantization:lim-range]
>>> -> "csi capture":0 [ENABLED,IMMUTABLE]
>>>
>>> - entity 4: csi capture (1 pad, 1 link)
>>> type Node subtype V4L flags 0
>>> device node name /dev/video2
>>> pad0: Sink
>>> <- "csi":1 [ENABLED,IMMUTABLE]
>>>
>>> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
>>> type V4L2 subdev subtype Unknown flags 0
>>> device node name /dev/v4l-subdev1
>>> pad0: Sink
>>> [fmt:UYVY8_1X16/640x480 field:none
>>> colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
>>> <- "imx219 2-0010":0 []
>>> pad1: Source
>>> [fmt:UYVY8_1X16/640x480 field:none
>>> colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
>>> -> "csi":0 [ENABLED,IMMUTABLE]
>>>
>>> - entity 15: imx219 2-0010 (1 pad, 1 link)
>>> type V4L2 subdev subtype Sensor flags 0
>>> device node name /dev/v4l-subdev2
>>> pad0: Source
>>> [fmt:SRGGB10_1X10/3280x2464 field:none colorspace:srgb
>>> xfer:srgb ycbcr:601 quantization:full-range
>>> crop.bounds:(8,8)/3280x2464
>>> crop:(8,8)/3280x2464]
>>> -> "imx7-mipi-csis.0":0 []
>>> # configure media controller links
>>> media-ctl --device /dev/media2 --reset
>>> # link the imx219 source pad (0) to the imx7-mipi-csis sink pad (0)
>>> media-ctl --device /dev/media2 --links "'imx219
>>> 2-0010':0->'imx7-mipi-csis.0':0[1]"
>>> # configure the imx219 source pad (0) for srgb 640x480
>>> media-ctl --device /dev/media2 -v -V "'imx219 2-0010':0
>>> [fmt:SRGGB8/640x480 field:none]"
>>> # configure the csi sink pad (0) for srgb 640x480
>>> media-ctl --device /dev/media2 -v -V "'csi':0 [fmt:SRGGB8/640x480 field:none]"
>>> # stream 640x480 at 30fps
>>> gst-launch-1.0 v4l2src device=/dev/video0 !
>>> video/x-bayer,format=rggb,width=640,height=480,framerate=10/1 !
>>> bayer2rgb ! fbdevsink # works
>>>
>>> Maybe you have gotten further than this by now?
>>>
>>> If you are still having issues getting imx-media to register we can
>>> look at kernel configs. I believe the following are necessary:
>>> CONFIG_VIDEO_IMX219=y
>>> CONFIG_VIDEO_IMX_MIPI_CSIS=y
>>> CONFIG_VIDEO_IMX7_CSI=y
>>
>> I got this working back in February with Linux 6.2-rc8 and a few
>> additional patches (see [1]), but I haven't tried this anymore since then.
>>
>> In my notes I have the following commands to test the camera:
>>
>> media-ctl -d /dev/media0 -l "'imx219 1-0010':0 ->
>> 'csis-32e30000.mipi-csi':0[1]"
>
> Hi Frieder,
>
> Thank you - this helped me identify a typo in my link configuration.
>
> My link configuration had an invalid argument in it which is why I was
> getting the invalid argument:
> # media-ctl -l "'imx219 2-0010':0->'csis-32e30000.mipi-csi.0':0[1]"
> Unable to parse link: Invalid argument (22)
> ^^^ the entity name does not include the .0 pad
>
> I believe this is the correct setup for imx219 on 6.5-rc1 with imx219
> for 640x480 raw8 bayer:
> media-ctl -l "'imx219 2-0010':0 -> 'csis-32e30000.mipi-csi':0 [1]"
> media-ctl -v -V "'imx219 2-0010':0 [fmt:SRGGB8/640x480 field:none]"
> media-ctl -v -V "'csis-32e30000.mipi-csi':0 [fmt:SRGGB8/640x480 field:none]"
> media-ctl -v -V "'csi':0 [fmt:SRGGB8/640x480 field:none]"
>
> I still can't get this to stream via gstreamer however:
> # gst-launch-1.0 v4l2src device=/dev/video0 !
> video/x-bayer,format=rggb,width=640,height=480,framerate=10/1 !
> bayer2rgb ! fbdevsink
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> [ 411.185357] imx7-csi 32e20000.csi: capture format not valid
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed
> to allocate required memory.
> Additional debug info:
> ../sys/v4l2/gstv4l2src.c(759): gst_v4l2src_decide_allocation ():
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> Buffer pool activation failed
> Execution ended after 0:00:00.010357497
> Setting pipeline to NULL ...
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> Internal data stream error.
> Additional debug info:
> ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop ():
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> streaming stopped, reason not-negotiated (-4)
> Freeing pipeline ...
>
> I'm not sure if your notes show you how you captured/streamed data
> after the linux media pipeline configuration?
I have the following gstreamer commands in my notes I used for streaming
the raw data to a host machine via network and viewing it remotely:
Host:
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=X-GST
! queue ! rtpgstdepay ! bayer2rgb ! videoconvert ! autovideosink
Client:
gst-launch-1.0 v4l2src device=/dev/video0 !
'video/x-bayer,format=rggb,width=1920,height=1080' ! rtpgstpay ! queue !
udpsink host=192.168.1.10 port=5000
But as you are getting "imx7-csi 32e20000.csi: capture format not
valid", I would assume that there is something wrong with the format and
your gstreamer pipeline might be correct in general.
In contrast to my setup you are using fmt:SRGGB8 instead of
fmt:SRGGB8_1X8. No idea if this makes any difference...
Does streaming with v4l2-ctl work for you without errors? I used the
following for testing:
v4l2-ctl -d /dev/video0
--set-fmt-video=width=1920,height=1080,pixelformat=RG10 --stream-mmap
Best regards
Frieder
More information about the linux-arm-kernel
mailing list