[PATCH v2 2/3] dt-bindings: arm: mediatek: mmsys: Add OF graph support for board path
CK Hu (胡俊光)
ck.hu at mediatek.com
Sun May 12 23:15:01 PDT 2024
On Fri, 2024-05-10 at 12:14 +0200, AngeloGioacchino Del Regno wrote:
> Il 10/05/24 11:34, CK Hu (胡俊光) ha scritto:
> > On Thu, 2024-05-09 at 11:27 +0200, AngeloGioacchino Del Regno
> > wrote:
> > > Il 09/05/24 07:42, CK Hu (胡俊光) ha scritto:
> > > > On Wed, 2024-05-08 at 15:03 +0200, AngeloGioacchino Del Regno
> > > > wrote:
> > > > > Il 08/05/24 09:19, CK Hu (胡俊光) ha scritto:
> > > > > > On Tue, 2024-05-07 at 16:07 +0200, AngeloGioacchino Del
> > > > > > Regno
> > > > > > wrote:
> > > > > > > Il 07/05/24 08:59, CK Hu (胡俊光) ha scritto:
> > > > > > > > On Thu, 2024-05-02 at 10:50 +0200, AngeloGioacchino Del
> > > > > > > > Regno
> > > > > > > > wrote:
> > > > > > > > > Il 25/04/24 04:23, CK Hu (胡俊光) ha scritto:
> > > > > > > > > > Hi, Angelo:
> > > > > > > > > >
> > > > > > > > > > On Tue, 2024-04-09 at 14:02 +0200, AngeloGioacchino
> > > > > > > > > > Del
> > > > > > > > > > Regno
> > > > > > > > > > wrote:
> > > > > > > > > > > Document OF graph on MMSYS/VDOSYS: this supports
> > > > > > > > > > > up
> > > > > > > > > > > to
> > > > > > > > > > > three
> > > > > > > > > > > DDP
> > > > > > > > > > > paths
> > > > > > > > > > > per HW instance (so potentially up to six
> > > > > > > > > > > displays
> > > > > > > > > > > for
> > > > > > > > > > > multi-
> > > > > > > > > > > vdo
> > > > > > > > > > > SoCs).
> > > > > > > > > > >
> > > > > > > > > > > The MMSYS or VDOSYS is always the first component
> > > > > > > > > > > in
> > > > > > > > > > > the
> > > > > > > > > > > DDP
> > > > > > > > > > > pipeline,
> > > > > > > > > > > so it only supports an output port with multiple
> > > > > > > > > > > endpoints -
> > > > > > > > > > > where
> > > > > > > > > > > each
> > > > > > > > > > > endpoint defines the starting point for one of
> > > > > > > > > > > the
> > > > > > > > > > > (currently
> > > > > > > > > > > three)
> > > > > > > > > > > possible hardware paths.
> > > > > > > > > > >
> > > > > > > > > > > Signed-off-by: AngeloGioacchino Del Regno <
> > > > > > > > > > > angelogioacchino.delregno at collabora.com>
> > > > > > > > > > > ---
> > > > > > > > > > > .../bindings/arm/mediatek/mediatek,mmsys.ya
> > > > > > > > > > > ml |
> > > > > > > > > > > 23
> > > > > > > > > > > +++++++++++++++++++
> > > > > > > > > > > 1 file changed, 23 insertions(+)
> > > > > > > > > > >
> > > > > > > > > > > diff --git
> > > > > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/
> > > > > > > > > > > medi
> > > > > > > > > > > atek
> > > > > > > > > > > ,mms
> > > > > > > > > > > ys.y
> > > > > > > > > > > aml
> > > > > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/
> > > > > > > > > > > medi
> > > > > > > > > > > atek
> > > > > > > > > > > ,mms
> > > > > > > > > > > ys.y
> > > > > > > > > > > aml
> > > > > > > > > > > index b3c6888c1457..4e9acd966aa5 100644
> > > > > > > > > > > ---
> > > > > > > > > > > a/Documentation/devicetree/bindings/arm/mediatek/
> > > > > > > > > > > medi
> > > > > > > > > > > atek
> > > > > > > > > > > ,mms
> > > > > > > > > > > ys.y
> > > > > > > > > > > aml
> > > > > > > > > > > +++
> > > > > > > > > > > b/Documentation/devicetree/bindings/arm/mediatek/
> > > > > > > > > > > medi
> > > > > > > > > > > atek
> > > > > > > > > > > ,mms
> > > > > > > > > > > ys.y
> > > > > > > > > > > aml
> > > > > > > > > > > @@ -93,6 +93,29 @@ properties:
> > > > > > > > > > > '#reset-cells':
> > > > > > > > > > > const: 1
> > > > > > > > > > >
> > > > > > > > > > > + port:
> > > > > > > > > > > + $ref: /schemas/graph.yaml#/properties/port
> > > > > > > > > > > + description:
> > > > > > > > > > > + Output port node. This port connects the
> > > > > > > > > > > MMSYS/VDOSYS
> > > > > > > > > > > output
> > > > > > > > > > > to
> > > > > > > > > > > + the first component of one display
> > > > > > > > > > > pipeline,
> > > > > > > > > > > for
> > > > > > > > > > > example
> > > > > > > > > > > one
> > > > > > > > > > > of
> > > > > > > > > > > + the available OVL or RDMA blocks.
> > > > > > > > > > > + Some MediaTek SoCs support up to three
> > > > > > > > > > > display
> > > > > > > > > > > outputs
> > > > > > > > > > > per
> > > > > > > > > > > MMSYS.
> > > > > > > > > > > + properties:
> > > > > > > > > > > + endpoint at 0:
> > > > > > > > > > > + $ref:
> > > > > > > > > > > /schemas/graph.yaml#/properties/endpoint
> > > > > > > > > > > + description: Output to the primary
> > > > > > > > > > > display
> > > > > > > > > > > pipeline
> > > > > > > > > > > +
> > > > > > > > > > > + endpoint at 1:
> > > > > > > > > > > + $ref:
> > > > > > > > > > > /schemas/graph.yaml#/properties/endpoint
> > > > > > > > > > > + description: Output to the secondary
> > > > > > > > > > > display
> > > > > > > > > > > pipeline
> > > > > > > > > > > +
> > > > > > > > > > > + endpoint at 2:
> > > > > > > > > > > + $ref:
> > > > > > > > > > > /schemas/graph.yaml#/properties/endpoint
> > > > > > > > > > > + description: Output to the tertiary
> > > > > > > > > > > display
> > > > > > > > > > > pipeline
> > > > > > > > > > > +
> > > > > > > > > > > + required:
> > > > > > > > > > > + - endpoint at 0
> > > > > > > > > > > +
> > > > > > > > > >
> > > > > > > > > > mmsys/vdosys does not output data to the first
> > > > > > > > > > component of
> > > > > > > > > > display
> > > > > > > > > > pipeline, so this connection looks 'virtual'. Shall
> > > > > > > > > > we
> > > > > > > > > > add
> > > > > > > > > > something
> > > > > > > > > > virtual in device tree? You add this in order to
> > > > > > > > > > decide
> > > > > > > > > > which
> > > > > > > > > > pipeline
> > > > > > > > > > is 1st, 2nd, 3rd, but for device it don't care
> > > > > > > > > > which
> > > > > > > > > > one is
> > > > > > > > > > first.
> > > > > > > > > > In
> > > > > > > > > > computer, software could change which display is
> > > > > > > > > > the
> > > > > > > > > > primary
> > > > > > > > > > display.
> > > > > > > > > > I'm not sure it's good to decide display order in
> > > > > > > > > > device
> > > > > > > > > > tree?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Devicetree describes hardware, so nothing virtual can
> > > > > > > > > be
> > > > > > > > > present
> > > > > > > > > -
> > > > > > > > > and in any case,
> > > > > > > > > the primary/secondary/tertiary pipeline is in
> > > > > > > > > relation to
> > > > > > > > > MM/VDO
> > > > > > > > > SYS,
> > > > > > > > > not referred
> > > > > > > > > to software.
> > > > > > > > >
> > > > > > > > > Better explaining, the primary pipeline is not
> > > > > > > > > necessarily
> > > > > > > > > the
> > > > > > > > > primary display in
> > > > > > > > > DRM terms: that's a concept that is completely
> > > > > > > > > detached
> > > > > > > > > from
> > > > > > > > > the
> > > > > > > > > scope of this
> > > > > > > > > series and this graph - and it's something that shall
> > > > > > > > > be
> > > > > > > > > managed
> > > > > > > > > solely by the
> > > > > > > > > driver (mediatek-drm in this case).
> > > > > > > > >
> > > > > > > > > Coming back to the connection looking, but *not*
> > > > > > > > > being
> > > > > > > > > virtual:
> > > > > > > > > the
> > > > > > > > > sense here is
> > > > > > > > > that the MM/VDOSYS blocks are used in the display
> > > > > > > > > pipeline to
> > > > > > > > > "stitch" together
> > > > > > > > > the various display pipeline hardware blocks, or,
> > > > > > > > > said
> > > > > > > > > differently,
> > > > > > > > > setting up the
> > > > > > > > > routing between all of those (P.S.:
> > > > > > > > > mmsys_mtxxxx_routing_table!)
> > > > > > > > > through the VDO
> > > > > > > > > Input Selection (VDOx_SEL_IN) or Output Selection
> > > > > > > > > (VDOx_SEL_OUT)
> > > > > > > > > and
> > > > > > > > > with the
> > > > > > > > > assistance of the VDO Multiple Output Mask
> > > > > > > > > (VDOx_MOUT)
> > > > > > > > > for
> > > > > > > > > the
> > > > > > > > > multiple outputs
> > > > > > > > > usecase, both of which, are described by this graph.
> > > > > > > >
> > > > > > > > I agree this part, but this is related to display
> > > > > > > > device OF
> > > > > > > > graph.
> > > > > > > > These display device would output video data from one
> > > > > > > > device
> > > > > > > > and
> > > > > > > > input
> > > > > > > > to another video device. These video device would not
> > > > > > > > input
> > > > > > > > or
> > > > > > > > output
> > > > > > > > video data to mmsys/vdosys.
> > > > > > > >
> > > > > > > > >
> > > > > > > > > This means that the VDOSYS is really the "master" of
> > > > > > > > > the
> > > > > > > > > display
> > > > > > > > > pipeline since
> > > > > > > > > everything gets enabled, mixed and matched from there
> > > > > > > > > -
> > > > > > > > > and
> > > > > > > > > that's in
> > > > > > > > > the sense
> > > > > > > > > of hardware operation, so we are *really* (and not
> > > > > > > > > virtually!)
> > > > > > > > > flipping switches.
> > > > > > > >
> > > > > > > > I agree mmsys/vdosys is master of video pipeline, so
> > > > > > > > let's
> > > > > > > > define
> > > > > > > > what
> > > > > > > > the port in mmsys/vdosys is. If the port means the
> > > > > > > > master
> > > > > > > > relationship,
> > > > > > > > mmsys/vdosys should output port to every display
> > > > > > > > device. Or
> > > > > > > > use
> > > > > > > > a
> > > > > > > > simply way to show the master relation ship
> > > > > > > >
> > > > > > > > mmsys-subdev = <&ovl0, &rdma0, &color0, ...>, <&ovl1,
> > > > > > > > &rdma1,
> > > > > > > > &color1,
> > > > > > > > ...>;
> > > > > > > >
> > > > > > >
> > > > > > > There's no need to list all of the VDO0/VDO1/mmsys
> > > > > > > devices in
> > > > > > > one
> > > > > > > big
> > > > > > > array
> > > > > > > property, because the actual possible devices can be
> > > > > > > defined:
> > > > > > > 1. In the bindings; and
> > > > > > > 2. In the actual OF graph that we write for each
> > > > > > > SoC+board
> > > > > > > combination.
> > > > > > >
> > > > > > > A graph cannot contain a connection to a device that
> > > > > > > cannot
> > > > > > > be
> > > > > > > connected to
> > > > > > > the previous, so, your "mmsys-subdev" list can be
> > > > > > > retrieved
> > > > > > > by
> > > > > > > looking at the
> > > > > > > graph:
> > > > > > > - Start from VDO0/1 or MMSYS
> > > > > > > - Walk through (visually, even) OUTPUT ports
> > > > > > > - VDO0 (read output ep) -> ovl0 (read output ep)
> > > > > > > ->
> > > > > > > rdma0
> > > > > > > (read
> > > > > > > output ep) ->
> > > > > > > color0 (...) -> etc
> > > > > > > - Nothing more - it's all defined there.
> > > > > > >
> > > > > > > >
> > > > > > > > Another problem is how to group display device? If two
> > > > > > > > pipeline
> > > > > > > > could
> > > > > > > > be route to the same display interface, such as
> > > > > > > >
> > > > > > > > rdma0 -> dsi
> > > > > > > > rdma1 -> dsi
> > > > > > > >
> > > > > > > > Would this be single group?
> > > > > > >
> > > > > > > There are multiple ways of doing this, but one that comes
> > > > > > > to
> > > > > > > my
> > > > > > > mind
> > > > > > > right now and
> > > > > > > that looks clean as well is the following:
> > > > > > >
> > > > > > > ovl0 at ef01 {
> > > > > > > .....
> > > > > > > ports {
> > > > > > > port at 0 {
> > > > > > > reg = <0>;
> > > > > > > ovl0_in: endpoint {
> > > > > > > remote-endpoint = <&vdosys0_out>;
> > > > > > > };
> > > > > > > };
> > > > > >
> > > > > > I'm not sure how do you define this port from OVL to
> > > > > > vdosys. If
> > > > > > this
> > > > > > port means 'master relationship', others could add port in
> > > > > > COLOR to
> > > > > > point to vdosys because COLOR and vdosys has the 'master
> > > > > > relationship'
> > > > > > and I could not reject this. So we need more specific
> > > > > > definition of
> > > > > > this port.
> > > > >
> > > > >
> > > > > > Only the 'first' device in pipeline could have this port?
> > > > >
> > > > > Correct. Only the first device in a pipeline - and this is
> > > > > actually a
> > > > > restriction
> > > > > that the generic binding definition of port already gives, in
> > > > > a
> > > > > way.
> > > > >
> > > > >
> > > > > > In mt8173, one pipeline is
> > > > > >
> > > > > > ovl -> color -> aal -> od -> rdma -> ufo -> dsi
> > > > > >
> > > > > > But rdma has an option to read data from od or directly
> > > > > > from
> > > > > > DRAM.
> > > > > > If
> > > > > > from DRAM, the pipeline would be changed to
> > > > > >
> > > > > > rdma -> ufo -> dsi
> > > > > >
> > > > > >
> > > > > > So it's confused which one is 'first'.
> > > > >
> > > > > That's why the pipeline is *board-specific* and not soc-
> > > > > generic!
> > > > >
> > > > > And what you described is *exactly* the reason why I'm adding
> > > > > support
> > > > > for the
> > > > > OF graphs in mediatek-drm: specifying the correct pipeline
> > > > > for
> > > > > each
> > > > > board as per
> > > > > what each board wants to use (said differently: for each
> > > > > board's
> > > > > *capabilities*).
> > > > >
> > > > > So, if on a certain board you want to skip OD, you can hook
> > > > > RDMA
> > > > > up
> > > > > directly to
> > > > > MMSYS/VDOSYS.
> > > > >
> > > > > In MT8173, one pipeline for one board uses endpoints IN/OUT
> > > > > like
> > > > > this:
> > > > >
> > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI
> > > > >
> > > > > and for another board, endpoints will be like
> > > > >
> > > > > MMSYS -> RDMA -> UFO -> DSI
> > > > >
> > > > > ...which is the exact same as you described, and I think that
> > > > > your
> > > > > confusion comes
> > > > > from the fact that you didn't put MMSYS at the beginning of
> > > > > the
> > > > > pipeline :-)
> > > >
> > > > In one board, both OVL and RDMA could switch dynamically.
> > > > Because
> > > > each
> > > > one could be the first in one board, mmsys point to both ovl
> > > > and
> > > > rdma?
> > > >
> > >
> > > No.
> > >
> > > MMSYS would still point ONLY to OVL, because OVL is the "earliest
> > > point"
> > > of the pipeline that this one board does support.
> > >
> > > In that case, RDMA being present at a later point in the pipeline
> > > does not
> > > matter and does not prevent us from *temporarily* skipping OVL-
> > > COLOR-
> > > AAL-OD
> > > and going MMSYS->RDMA *directly*.
> > >
> > > Switching dynamically is a driver duty and will be 100% possible
> > > (as
> > > much
> > > as it is right now) to dynamically switch OVL and RDMA as long as
> > > both are
> > > present in the pipeline.
> > >
> > > With this exact pipeline:
> > >
> > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI
> > >
> > > the driver _can switch dynamically_ between MMSYS->OVL->...->RDMA
> > > and
> > > MMSYS->RDMA as the driver itself *is allowed to* temporarily
> > > ignore
> > > part
> > > of the pipeline.
> > >
> > > Please note that, in case it is needed (trust me on this: it's
> > > not
> > > needed)
> > > a custom property in the endpoint node can always be introduced
> > > later, so
> > > that you can declare a node like
> > >
> > > endpoint at 0 {
> > > remote-endpoint = <&ovl0_in>;
> > > mediatek,short-path = <&rdma0_in>;
> > > };
> > >
> > > ...but again, that's never going to be needed, as the driver
> > > already
> > > does
> > > have knowledge of the fact that RDMA is in the pipeline, so it is
> > > possible
> > > to simply do a temporary override in the driver.
> > >
> > > What the OF Graph support does is to build the same arrays, that
> > > we
> > > currently
> > > have hardcoded in mediatek-drm, by reading from device tree.
> > >
> > > Nothing else and nothing more - for now.
> > >
> > > Having the OF Graph support makes us able to also add new dual-
> > > path
> > > support
> > > with more complicated connections than the current ones, without
> > > any
> > > problem
> > > and, in many cases, without even modifying the bindings from what
> > > I
> > > provided
> > > in this series.
> >
> > OK, please add the information we discussed into binding document
> > to
> > prevent anyone misusing this binding.
> >
>
> Sorry CK, but the binding *does* already have this information inside
> - not
> in terms of "phrases", but in terms of restrictions of the binding...
>
> If you want, though, I can add this information in the description of
> the
> commit that is adding this binding, is that okay for you?
I think what we discuss could be translated to restrictions. This is a
restriction description:
If one component has option to be first component or middle component
of one pipeline, it's treated as middle component not first component.
In mt8195, there are 8 vdo1_rdma. So each one is the first component of
display pipeline. So the maximum display pipe number is not 3, maybe 8
or more.
Regards,
CK
>
> Thanks,
> Angelo
>
> > Regards,
> > CK
> >
> > >
> > > Cheers,
> > > Angelo
> > >
> > > > Regards,
> > > > CK
> > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > In case you need any *temporary override* on any board that
> > > > > defines a
> > > > > pipeline like
> > > > >
> > > > > MMSYS -> OVL -> COLOR -> AAL -> OD -> RDMA -> UFO -> DSI
> > > > >
> > > > > so that the pipeline *temporarily* becomes (for power
> > > > > management,
> > > > > or
> > > > > for any other
> > > > > reason) RDMA -> UFO -> DSI .... that's not a concern: the
> > > > > graph
> > > > > is
> > > > > present, and it
> > > > > is used to tell to the driver what is the regular pipeline to
> > > > > use.
> > > > > Eventual temporary overrides can be managed transparently
> > > > > inside
> > > > > of
> > > > > the driver with
> > > > > C code and no changes to the devicetree are required.
> > > > >
> > > > >
> > > > > > I don't know how to decide which device could point to
> > > > > > mmsys/vdosys. So
> > > > > > please give a specific definition.
> > > > > >
> > > > >
> > > > > Nothing points TO mmsys/vdosys. It is mmsys/vdosys pointing
> > > > > to a
> > > > > device.
> > > > >
> > > > > So, mmsys/vdosys must point to the *first device in the
> > > > > pipeline*.
> > > > >
> > > > > Any other doubt?
> > > > >
> > > > > Cheers,
> > > > > Angelo
> > > > >
> > > > > > Regards,
> > > > > > CK
> > > > > >
> > > > > > >
> > > > > > > port at 1 {
> > > > > > > reg = <1>;
> > > > > > > ovl0_out0: endpoint at 0 {
> > > > > > > remote-endpoint = <&rdma0_in>;
> > > > > > > };
> > > > > > > ovl0_out1: endpoint at 1 {
> > > > > > > remote-endpoint = <&rdma1_in>;
> > > > > > > };
> > > > > > > };
> > > > > > > };
> > > > > > > };
> > > > > > >
> > > > > > > rdma0 at 1234 {
> > > > > > > .....
> > > > > > > ports {
> > > > > > > port at 0 {
> > > > > > > reg = <0>;
> > > > > > > rdma0_in: endpoint {
> > > > > > > remote-endpoint = <&ovl0_out0>; /* assuming
> > > > > > > ovl0
> > > > > > > outputs to
> > > > > > > rdma0...*/
> > > > > > > };
> > > > > > > };
> > > > > > > port at 1 {
> > > > > > > reg = <1>;
> > > > > > > rdma0_out: endpoint at 1 {
> > > > > > > remote-endpoint = <&dsi_dual_intf0_in>;
> > > > > > > };
> > > > > > > };
> > > > > > > };
> > > > > > > };
> > > > > > >
> > > > > > >
> > > > > > > rdma1 at 5678 {
> > > > > > > .....
> > > > > > > ports {
> > > > > > > port at 0 {
> > > > > > > reg = <0>;
> > > > > > > rdma1_in: endpoint {
> > > > > > > /* assuming ovl0 outputs to rdma1 as well...
> > > > > > > can
> > > > > > > be
> > > > > > > something else. */
> > > > > > > remote-endpoint = <&ovl0_out1>;
> > > > > > > };
> > > > > > > };
> > > > > > > port at 1 {
> > > > > > > reg = <1>;
> > > > > > > rdma1_out: endpoint {
> > > > > > > remote-endpoint = <&dsi_dual_intf1_in>;
> > > > > > > };
> > > > > > > };
> > > > > > > };
> > > > > > > };
> > > > > > >
> > > > > > >
> > > > > > > dsi at 9abcd {
> > > > > > > .....
> > > > > > > ports {
> > > > > > > port at 0 {
> > > > > > > reg = <0>;
> > > > > > > /* Where endpoint at 0 could be always DSI LEFT
> > > > > > > CTRL */
> > > > > > > dsi_dual_intf0_in: endpoint at 0 {
> > > > > > > remote-endpoint = <&rdma0_out>;
> > > > > > > };
> > > > > > > /* ...and @1 could be always DSI RIGHT CTRL */
> > > > > > > dsi_dual_intf1_in: endpoint at 1 {
> > > > > > > remote-endpoint = <&rdma1_out>;
> > > > > > > };
> > > > > > > };
> > > > > > >
> > > > > > > port at 1 {
> > > > > > > reg = <1>;
> > > > > > > dsi0_out: endpoint {
> > > > > > > remote-endpoint = <&dsi_panel_in>;
> > > > > > > };
> > > > > > > };
> > > > > > > };
> > > > > > > };
> > > > > > >
> > > > > > > ...for a dual-dsi panel, it'd be a similar graph.
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Angelo
> > > > > > >
> > > > > > > >
> > > > > > > > mmsys-subdev = <&rdma0, &rdma1, &dsi>;
> > > > > > > >
> > > > > > > > Or two group?
> > > > > > > >
> > > > > > > > mmsys-subdev = <&rdma0, &dsi>, <&rdma1, &dsi>;
> > > > > > > >
> > > > > > > > I think we should clearly define this.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > CK
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Angelo
> > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > CK
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > required:
> > > > > > > > > > > - compatible
> > > > > > > > > > > - reg
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > >
> > >
> > >
>
>
More information about the linux-arm-kernel
mailing list