[PATCH 1/6] media: uapi: Add MEDIA_BUS_FMT_SGRGB_IGIG_GBGR_IGIG media bus formats

[Laurent Pinchart]

Hi Marco,

On Thu, Apr 29, 2021 at 09:49:03AM +0200, Marco Felsch wrote:
> On 21-04-29 04:51, Laurent Pinchart wrote:
> > On Tue, Apr 27, 2021 at 02:06:56PM +0200, Marco Felsch wrote:
> > > Add special 8/12bit bayer media bus format for the OnSemi AR0237IR
> > > camera sensor [1]. OnSemi calls this format RGB-IR, the pixel array
> > > with the interleaved IR pixels looks like:
> > > 
> > >         |  G |  R |  G |  B | ...
> > >         +----+----+----+----+---
> > >         | IR |  G | IR |  G | ...
> > >         +----+----+----+----+---
> > >         |  G |  B |  G |  R | ...
> > >         +----+----+----+----+---
> > >         | IR |  G | IR |  G | ...
> > >         +----+----+----+----+---
> > >         | .. | .. | .. | .. | ..
> > > 
> > > [1] https://www.framos.com/media/pdf/96/ac/8f/AR0237CS-D-PDF-framos.pdf
> > 
> > I think we're reaching a limit of the media bus codes model here, due to
> > a historical mistake. The four possible Bayer patterns, times the
> > different number of bits per pixel, creates a lot of media bus codes,
> > and drivers for CSI-2 receivers and IP cores further down the pipeline
> > have to support them all.
> 
> That's correct but it is not bayer related. Currently it is what it is,
> if a new code is added it must be propagated through all the involved
> subdevs. On the other hand I wouldn't say that it is better to support
> new codes per default for all drivers. Since this would add a lot of
> untested code paths.

It's not an issue limited to Bayer patterns, but they make the issue
worse given the number of possible combinations (think about adding DPCM
and ALAW compression on top of that...).

> > This is already painful, and if we had a
> > non-Bayer pattern such as this one,
> 
> That's not exactly true since it is a bayer pattern but instead of using
> 4x4 it uses 8x8 and it as some special pixels.
> 
> > we'll open the door to an explosion
> > of the number of media bus codes (imagine all the different possible
> > arrangements of this pattern, for instance if you enable horizontal
> > and/or vertical flipping on the sensor). All drivers would need to be
> > updated to support these new bus codes, and this really kills the
> > current model.
> 
> Yep, I know what you mean but as I said above I think that adding it
> explicite is the better abbroach since it involves somone who add _and_
> test the new code on the particular platform.
> 
> > The historical mistake was to tie the Bayer pattern with the media bus
> > code. We should really have specified raw 8/10/12/14/16 media bus codes,
> > and conveyed the pattern separately. Most IP cores in the pipeline don't
> > need to care about the pattern at all, and those who do (that's mostly
> > ISPs) could be programmed explicitly by userspace as long as we have an
> > API to retrieve the pattern from the sensor. I believe it's time to bite
> > the bullet and go in that direction. I'm sorry for this case of yak
> > shaving, but it really wouldn't be manageable anymore :-(
> 
> I got all your points and would agree but this is not a bayer only
> related problem. You will have this problem with all new other formats
> as well. I'm with you, most IP cores don't care but I wouldn't
> guarantee that.

Sorry, but adding new media bus formats like this one will just not
scale. We have two options, either fixing the issue, or considering that
V4L2 is a barely alive API with no future, and merging this without
caring anymore.

-- 
Regards,

Laurent Pinchart