[PATCH 09/11] ice: implement dpll interface to control cgu

[Kubalewski, Arkadiusz]

>From: Jiri Pirko <jiri at resnulli.us>
>Sent: Friday, July 21, 2023 9:33 AM
>
>Thu, Jul 20, 2023 at 07:31:14PM CEST, arkadiusz.kubalewski at intel.com wrote:
>>>From: Jiri Pirko <jiri at resnulli.us>
>>>Sent: Thursday, July 20, 2023 4:09 PM
>>>
>>>Thu, Jul 20, 2023 at 11:19:01AM CEST, vadim.fedorenko at linux.dev wrote:
>>>>From: Arkadiusz Kubalewski <arkadiusz.kubalewski at intel.com>
>>>
>>>[...]
>>>
>>>
>>>>+/**
>>>>+ * ice_dpll_pin_enable - enable a pin on dplls
>>>>+ * @hw: board private hw structure
>>>>+ * @pin: pointer to a pin
>>>>+ * @pin_type: type of pin being enabled
>>>>+ * @extack: error reporting
>>>>+ *
>>>>+ * Enable a pin on both dplls. Store current state in pin->flags.
>>>>+ *
>>>>+ * Context: Called under pf->dplls.lock
>>>>+ * Return:
>>>>+ * * 0 - OK
>>>>+ * * negative - error
>>>>+ */
>>>>+static int
>>>>+ice_dpll_pin_enable(struct ice_hw *hw, struct ice_dpll_pin *pin,
>>>>+		    enum ice_dpll_pin_type pin_type,
>>>>+		    struct netlink_ext_ack *extack)
>>>>+{
>>>>+	u8 flags = 0;
>>>>+	int ret;
>>>>+
>>>
>>>
>>>
>>>I don't follow. Howcome you don't check if the mode is freerun here or
>>>not? Is it valid to enable a pin when freerun mode? What happens?
>>>
>>
>>Because you are probably still thinking the modes are somehow connected
>>to the state of the pin, but it is the other way around.
>>The dpll device mode is a state of DPLL before pins are even considered.
>>If the dpll is in mode FREERUN, it shall not try to synchronize or monitor
>>any of the pins.
>>
>>>Also, I am probably slow, but I still don't see anywhere in this
>>>patchset any description about why we need the freerun mode. What is
>>>diffrerent between:
>>>1) freerun mode
>>>2) automatic mode & all pins disabled?
>>
>>The difference:
>>Case I:
>>1. set dpll to FREERUN and configure the source as if it would be in
>>AUTOMATIC
>>2. switch to AUTOMATIC
>>3. connecting to the valid source takes ~50 seconds
>>
>>Case II:
>>1. set dpll to AUTOMATIC, set all the source to disconnected
>>2. switch one valid source to SELECTABLE
>>3. connecting to the valid source takes ~10 seconds
>>
>>Basically in AUTOMATIC mode the sources are still monitored even when they
>>are not in SELECTABLE state, while in FREERUN there is no such monitoring,
>>so in the end process of synchronizing with the source takes much longer as
>>dpll need to start the process from scratch.
>
>I believe this is implementation detail of your HW. How you do it is up
>to you. User does not have any visibility to this behaviour, therefore
>makes no sense to expose UAPI that is considering it. Please drop it at
>least for the initial patchset version. If you really need it later on
>(which I honestly doubt), you can send it as a follow-up patchset.
>

And we will have the same discussion later.. But implementation is already
there.
As said in our previous discussion, without mode_set there is no point to have
command DEVICE_SET at all, and there you said that you are ok with having the
command as a placeholder, which doesn't make sense, since it is not used. 

Also this is not HW implementation detail but a synchronizer chip feature,
once dpll is in FREERUN mode, the measurements like phase offset between the
input and dpll's output won't be available.

For the user there is a difference..
Enabling the FREERUN mode is a reset button on the dpll's state machine,
where disconnecting sources is not, as they are still used, monitored and
measured.
So probably most important fact that you are missing here: assuming the user
disconnects the pin that dpll was locked with, our dpll doesn't go into UNLOCKED
state but into HOLDOVER.

>
>
>>
>>>
>>>Isn't the behaviour of 1) and 2) exactly the same? If no, why? This
>>>needs to be documented, please.
>>>
>>
>>Sure will add the description of FREERUN to the docs.
>
>No, please drop it from this patchset. I have no clue why you readded
>it in the first place in the last patchset version.
>

mode_set was there from the very beginning.. now implemented in ice driver
as it should.

>
>>
>>>
>>>
>>>Another question, I asked the last time as well, but was not heard:
>>>Consider example where you have 2 netdevices, eth0 and eth1, each
>>>connected with a single DPLL pin:
>>>eth0 - DPLL pin 10 (DPLL device id 2)
>>>eth1 - DPLL pin 11 (DPLL device id 2)
>>>
>>>You have a SyncE daemon running on top eth0 and eth1.
>>>
>>>Could you please describe following 2 flows?
>>>
>>>1) SyncE daemon selects eth0 as a source of clock
>>>2) SyncE daemon selects eth1 as a source of clock
>>>
>>>
>>>For mlx5 it goes like:
>>>
>>>DPLL device mode is MANUAL.
>>>1)
>>> SynceE daemon uses RTNetlink to obtain DPLL pin number of eth0
>>>    -> pin_id: 10
>>> SenceE daemon will use PIN_GET with pin_id 10 to get DPLL device id
>>>    -> device_id: 2
>>
>>Not sure if it needs to obtain the dpll id in this step, but it doesn't
>>relate to the dpll interface..
>
>Sure it has to. The PIN_SET accepts pin_id and device_id attrs as input.
>You need to set the state on a pin on a certain DPLL device.
>

The thing is pin can be connected to multiple dplls and SyncE daemon shall
know already something about the dpll it is managing.
Not saying it is not needed, I am saying this is not a moment the SyncE daemon
learns it.
But let's park it, as this is not really relevant.

>
>>
>>> SynceE daemon does PIN_SET cmd on pin_id 10, device_id 2 -> state =
>>>CONNECTED
>>>
>>>2)
>>> SynceE daemon uses RTNetlink to obtain DPLL pin number of eth1
>>>    -> pin_id: 11
>>> SenceE daemon will use PIN_GET with pin_id 11 to get DPLL device id
>>>    -> device_id: 2
>>> SynceE daemon does PIN_SET cmd on pin_id 10, device_id 2 -> state =
>>>CONNECTED
>>> (that will in HW disconnect previously connected pin 10, there will be
>>>  notification of pin_id 10, device_id -> state DISCONNECT)
>>>
>>
>>This flow is similar for ice, but there are some differences, although
>>they come from the fact, the ice is using AUTOMATIC mode and recovered
>>clock pins which are not directly connected to a dpll (connected through
>>the MUX pin).
>>
>>1)
>>a) SyncE daemon uses RTNetlink to obtain DPLL pin number of eth0 ->
>>pin_id: 13
>>b) SyncE daemon uses PIN_GET to find a parent MUX type pin -> pin_id: 2
>>   (in case of dpll_id is needed, would be find in this response also)
>>c) SyncE daemon uses PIN_SET to set parent MUX type pin (pin_id: 2) to
>>   pin-state: SELECTABLE and highest priority (i.e. pin-prio:0, while all the
>>   other pins shall be lower prio i.e. pin-prio:1)
>
>Yeah, for this you need pin_id 2 and device_id. Because you are setting
>state on DPLL device.
>
>
>>d) SyncE daemon uses PIN_SET to set state of pin_id:13 to CONNECTED with
>>   parent pin (pin-id:2)
>
>For this you need pin_id and pin_parent_id because you set the state on
>a parent pin.
>
>
>Yeah, this is exactly why I initially was in favour of hiding all the
>muxes and magic around it hidden from the user. Now every userspace app
>working with this has to implement a logic of tracking pin and the mux
>parents (possibly multiple levels) and configure everything. But it just
>need a simple thing: "select this pin as a source" :/
>
>
>Jakub, isn't this sort of unnecessary HW-details complexicity exposure
>in UAPI you were against in the past? Am I missing something?
>

Multiple level of muxes possibly could be hidden in the driver, but the fact
they exist is not possible to be hidden from the user if the DPLL is in
AUTOMATIC mode.
For MANUAL mode dpll the muxes could be also hidden.
Yeah, we have in ice most complicated scenario of AUTOMATIC mode + MUXED type
pin.

Thank you!
Arkadiusz

>
>
>>
>>2) (basically the same, only eth1 would get different pin_id.)
>>a) SyncE daemon uses RTNetlink to obtain DPLL pin number of eth0 ->
>>pin_id: 14
>>b) SyncE daemon uses PIN_GET to find parent MUX type pin -> pin_id: 2
>>c) SyncE daemon uses PIN_SET to set parent MUX type pin (pin_id: 2) to
>>   pin-state: SELECTABLE and highest priority (i.e. pin-prio:0, while all the
>>   other pins shall be lower prio i.e. pin-prio:1)
>>d) SyncE daemon uses PIN_SET to set state of pin_id:14 to CONNECTED with
>>   parent pin (pin-id:2)
>>
>>Where step c) is required due to AUTOMATIC mode, and step d) required due to
>>phy recovery clock pin being connected through the MUX type pin.
>>
>>Thank you!
>>Arkadiusz
>>
>>>
>>>Thanks!
>>>
>>>
>>>[...]