[PATCH 1/6] media: uapi: Add MEDIA_BUS_FMT_SGRGB_IGIG_GBGR_IGIG media bus formats

[Marco Felsch]

Hi Laurent,

On 21-04-29 04:51, Laurent Pinchart wrote:
> Hi Marco,
> 
> Thank you for the patch.
> 
> On Tue, Apr 27, 2021 at 02:06:56PM +0200, Marco Felsch wrote:
> > Add special 8/12bit bayer media bus format for the OnSemi AR0237IR
> > camera sensor [1]. OnSemi calls this format RGB-IR, the pixel array
> > with the interleaved IR pixels looks like:
> > 
> >         |  G |  R |  G |  B | ...
> >         +----+----+----+----+---
> >         | IR |  G | IR |  G | ...
> >         +----+----+----+----+---
> >         |  G |  B |  G |  R | ...
> >         +----+----+----+----+---
> >         | IR |  G | IR |  G | ...
> >         +----+----+----+----+---
> >         | .. | .. | .. | .. | ..
> > 
> > [1] https://www.framos.com/media/pdf/96/ac/8f/AR0237CS-D-PDF-framos.pdf
> 
> I think we're reaching a limit of the media bus codes model here, due to
> a historical mistake. The four possible Bayer patterns, times the
> different number of bits per pixel, creates a lot of media bus codes,
> and drivers for CSI-2 receivers and IP cores further down the pipeline
> have to support them all.

That's correct but it is not bayer related. Currently it is what it is,
if a new code is added it must be propagated through all the involved
subdevs. On the other hand I wouldn't say that it is better to support
new codes per default for all drivers. Since this would add a lot of
untested code paths.

> This is already painful, and if we had a
> non-Bayer pattern such as this one,

That's not exactly true since it is a bayer pattern but instead of using
4x4 it uses 8x8 and it as some special pixels.

> we'll open the door to an explosion
> of the number of media bus codes (imagine all the different possible
> arrangements of this pattern, for instance if you enable horizontal
> and/or vertical flipping on the sensor). All drivers would need to be
> updated to support these new bus codes, and this really kills the
> current model.

Yep, I know what you mean but as I said above I think that adding it
explicite is the better abbroach since it involves somone who add _and_
test the new code on the particular platform.

> The historical mistake was to tie the Bayer pattern with the media bus
> code. We should really have specified raw 8/10/12/14/16 media bus codes,
> and conveyed the pattern separately. Most IP cores in the pipeline don't
> need to care about the pattern at all, and those who do (that's mostly
> ISPs) could be programmed explicitly by userspace as long as we have an
> API to retrieve the pattern from the sensor. I believe it's time to bite
> the bullet and go in that direction. I'm sorry for this case of yak
> shaving, but it really wouldn't be manageable anymore :-(

I got all your points and would agree but this is not a bayer only
related problem. You will have this problem with all new other formats
as well. I'm with you, most IP cores don't care but I wouldn't
guarantee that.

Regards,
  Marco