Seeking guidance for Adding ISP Driver for RK3568

[Yogesh Hegde]

Hi Michael, 
On Tue, Jul 11, 2023 at 11:29:34AM +0200, Michael Riesch wrote:
> Hi Yogesh,
> 
> Cc: Laurent as I had a chat with him about 1.
> 
> On 7/11/23 09:11, Yogesh Hegde wrote:
> > Hi,
> > 
> > I am seeking your expert guidance and assistance in adding the ISP driver for 
> > RK3568 to the Linux Kernel.
> > 
> > Before I start, I have a few questions regarding the process of 
> > adding the ISP driver for RK3568 and upstreaming it to the Linux Kernel. 
> > I greatly appreciate any insights you can provide on the following points:
> > 
> > 1. Integration with existing RK3399 ISP driver: Since the ISP driver for RK3399
> >    has already been upstreamed, I would like to understand if the ISP driver 
> >    for RK3568 will be added to the existing RK3399 ISP driver or if it will be 
> >    a completely new driver ?
> 
> That's a very good question, but unfortunately there is no clear answer
> to it. We were able to hack the existing mainline rkisp1 driver so that
> it works with the RK3568 ISP (rkisp v2.1), but the result is far from
> being mainline material.
> 
> I would say that there is enough overlap that the RK3568 ISP code should
> enter drivers/media/platform/rockchip/rkisp1/ but on the other hand the
> differences are so significant that there should be separate
> rkisp2-{csi,params,...}.c files.
> 
> Example for overlap: With some refactoring applied, rkisp1-capture.c
> could support the RK3568 ISP as well.
> 
> Example for significant changes: the CSI subdevice is fundamentally
> different, so most likely there will be a rkisp2-csi.c.
> 
> Example for a mixture of both: a new stats/params interface to user
> space will be required for the RK3568 ISP, but if this is designed to be
> general enough, it might be reused for the older ISP versions at some point.
> 
> > 2. Upstreaming process and staging: I would like to know if the upstreaming 
> >    process for the RK3568 ISP driver will involve starting with the downstream 
> >    driver and adding it to the staging area. From there, would it be merged 
> >    into the media subsystem or follow a different process altogether?
> 
> I think the tendency is towards skipping staging, but I don't have a say
> on that one.
> 
> > 3. Permission requirements from Rockchip: To ensure a smooth upstreaming 
> >    process, I would like to know if I need any permissions or approvals from 
> >    Rockchip, the manufacturer of the RK3568 chipset. If permissions are 
> >    required, could you please guide me on how to obtain them?
> 
> Downstream code is GPL'ed, so it should be fine.
> 
> > I would also like to mention that while this is my first time upstreaming a 
> > full driver, I do have some experience working with the media subsystem.
> 
> Cool! Just be aware that you picked a complex beast for your first full
> driver.
> 
Thank you for taking the time to respond and provide clarity regarding the 
process.

> What camera sensor hardware do you have?

I have Raspi Cam v1.3 (OV5647) and Raspberry Pi Camera module 2 (IMX219).
>From the Rock 3A schematics, the MIPI CSI port is Raspberry Pi compatible. 


Thanks and Regards
Yogesh