[PATCH v4 3/5] media: platform: mediatek: isp_30: add mediatek ISP3.0 sensor interface

[Julien Stephan]

Le sam. 15 juin 2024 à 02:47, Laurent Pinchart
<laurent.pinchart at ideasonboard.com> a écrit :
>
> On Fri, Jun 14, 2024 at 04:54:47PM +0200, Julien Stephan wrote:
> > Le ven. 14 juin 2024 à 16:43, Laurent Pinchart a écrit :
> > > On Fri, Jun 14, 2024 at 04:14:52PM +0200, Julien Stephan wrote:
> > > > Le ven. 14 juin 2024 à 14:34, Laurent Pinchart a écrit :
> > > > > On Fri, Jun 14, 2024 at 12:38:15PM +0200, Julien Stephan wrote:
> > > > > > Le mer. 12 juin 2024 à 10:06, AngeloGioacchino Del Regno a écrit :
> > > > > > >
> > > > > > > Il 10/06/24 16:39, Julien Stephan ha scritto:
> > > > > > [...]
> > > > > > > >>
> > > > > > > >>> +     writel(0x10001, input->base + SENINF_TG1_SEN_CK);
> > > > > > > >>
> > > > > > > >> Unroll this one... this is the TG1 sensor clock divider.
> > > > > > > >>
> > > > > > > >> CLKFL GENMASK(5, 0)
> > > > > > > >> CLKRS GENMASK(13, 8)
> > > > > > > >> CLKCNT GENMASK(21,16)
> > > > > > > >>
> > > > > > > >> Like this, I don't get what you're trying to set, because you're using a fixed
> > > > > > > >> sensor clock rate, meaning that only a handful of camera sensors will be usable.
> > > > > > > >>
> > > > > > > >> Is this 8Mhz? 16? 24? what? :-)
> > > > > > > >>
> > > > > > > >> Two hints:
> > > > > > > >>    - sensor_clk = clk_get_rate(isp_clk) / (tg1_sen_ck_clkcnt + 1);
> > > > > > > >>    - int mtk_seninf_set_sensor_clk(u8 rate_mhz);
> > > > > > > >>
> > > > > > > >> Please :-)
> > > > > > > >
> > > > > > > > Hi Angelo,
> > > > > > > >
> > > > > > > > I think I get your point about not hardcoding the sensor rate, but I
> > > > > > > > am not sure how to use
> > > > > > > > a mtk_seninf_set_sensor_clk(u8 rate_mhz); function.
> > > > > > > >
> > > > > > > > Where would it be called? How is it exposed to the user?
> > > > > > > >
> > > > > > >
> > > > > > > As for where: setup, streaming start, resolution change (which may be covered
> > > > > > > by streaming start anyway, as a change should be calling stop->start anyway).
> > > > > > >
> > > > > > > And for the how is it exposed to the user - well, depends what you mean for user,
> > > > > > > but it's all standard V4L2 API :-)
> > > > > > >
> > > > > > > Last but not least, I can give you another hint....
> > > > > > >
> > > > > > > struct media_entity *entity = (something_here);
> > > > > > > struct media_pad *mpad;
> > > > > > > struct v4l2_subdev *cam_subdev;
> > > > > > > struct v4l2_ctrl *ctl;
> > > > > > > s64 link_frequency, pixel_clock;
> > > > > > >
> > > > > > > if (entity->pads[0].flags & MEDIA_PAD_FL_SINK)
> > > > > > >     return -E_NOT_A_CAMERA_SENSOR_WE_IGNORE_THIS_ONE;
> > > > > > >
> > > > > > > pad = media_pad_remote_pad_first(&entity->pads[0]);
> > > > > > > if (!pad)
> > > > > > >    return -ENOENT;
> > > > > > >
> > > > > > > if (!is_media_entity_v4l2_subdev(pad->entity))
> > > > > > >    return -ENOENT;
> > > > > > >
> > > > > > > if (pad->entity->function != MEDIA_ENT_F_CAM_SENSOR)
> > > > > > >    return -ENOENT;
> > > > > > >
> > > > > >
> > > > > > Hi Angelo,
> > > > > >
> > > > > > Thank you for the detailed explanation :)
> > > > > > However, I can't make it work because in my case, seninf is connected
> > > > > > to an external ISP
> > > > > > so pad->entity->function == MEDIA_ENT_F_PROC_VIDEO_ISP.
> > > > > >
> > > > > > How can I get the pad corresponding to the sensor?
> > > > >
> > > > > You don't have to. You can drop that check, and get the link frequency
> > > > > of the source subdev with v4l2_get_link_freq(), whatever it is.
> > > > >
> > > > > > > cam_subdev = media_entity_to_v4l2_subdev(pad->entity);
> > > > > > > ctl = v4l2_ctrl_find(subdev->ctrl_handler, V4L2_CID_PIXEL_RATE);
> > > > > >
> > > > > > Is this mandatory to implement V4L2_CID_PIXEL_RATE ?
> > > > > > Should I return an error if not found?
> > > > >
> > > > > Does SENINF need to know both the pixel rate and link frequency ?
> > > > > V4L2_CID_PIXEL_RATE is very ill-defined, at the moment it only makes
> > > > > sense as a value relative to the sensor pixel array, and doesn't really
> > > > > apply further down in the pipeline. What information do you need to
> > > > > program the SENINF ?
> > > >
> > > > Hi Laurent,
> > > >
> > > > I need to know the clock divider for the sensor
> > >
> > > Could you provide some details on how the SENINF uses that divisor ?
> > > What does it control, and what are the constraints ?
> >
> > According to the datasheet,  SENINF_TG1_SEN_CK[21:16] :CLKCNT : Sensor
> > master clock will be ISP_clock/(CLKCNT+1) where CLKCNT >= 1
>
> I'll need more information. My guess so far is that there's a FIFO
> somewhere in the SENINF, with the pixel bus clocked by the CSI-2 clock
> before the FIFO, and by the "Sensor master clock" after the FIFO. Is
> that right ? If so, the simplest approach would be to use the link
> frequency to compute the pixel clock before the FIFO, and make sure that
> the sensor master clock will be larger than or equal to that.
>
> A better approach from a power consumption point of view would be to
> consider horizontal blanking. The FIFO can fill faster than it gets
> emptied during the active portion of the line and then drain during
> blanking. This allows for a slower clock on the output side. You will
> need to pick an output clock frequency that
>
> - on average is larger than the number of active pixels per line divided
>   by the line duration ; and
>
> - ensures the FIFO never overflows during the active portion of the line,
>   for cases where the line length is larger than the FIFO size.
>

Hi Laurent and Angelo,

Thank you for your help on this. I just sent a v5 with such a
function. But it seems like changing
this register has no effect. I asked some feedback from mediatek's app
engineer but they are OoO
until 16/7. I am waiting for them.

Cheers
Julien
> > > > > > > /* multiplier is usually bits per pixel, divider is usually num of lanes */
> > > > > > > link_frequency = v4l2_get_link_freq(cam_subdev->ctrl_handler, multiplier, divider);
> > > > > > > pixel_clock = v4l2_ctrl_g_ctrl_int64(ctl);
> > > > > >
> > > > > > How to know the sensor clock given link_frequency and pixel_clock?
> > > > > > Can you point me to drivers doing something similar?
> > > > > >
> > > > > > >
> > > > > > > ....now you know what the sensor wants, set the seninf sensor clock accordingly.
> > > > > > >
> > > > > > > Cheers
> > > > > > > Angelo
> > > > > > >
> > > > > > [...]
>
> --
> Regards,
>
> Laurent Pinchart