[PATCH 1/3] pmdomain: ti_sci: add per-device latency constraint management

[Kevin Hilman]

Markus Schneider-Pargmann <msp at baylibre.com> writes:

> On Tue, Aug 06, 2024 at 03:53:49PM GMT, Kevin Hilman wrote:
>> Markus Schneider-Pargmann <msp at baylibre.com> writes:
>> 
>> > Hi Kevin,
>> >
>> > On Mon, Aug 05, 2024 at 04:38:39PM GMT, Kevin Hilman wrote:
>> >> For each device in a TI SCI PM domain, check whether the device has
>> >> any resume latency constraints set via per-device PM QoS.  If
>> >> constraints are set, send them to DM via the new SCI constraints API.
>> >> 
>> >> Checking for constraints happen:
>> >> 
>> >> 1) before SCI PM domain power off (->power_off() hook)
>> >> 2) before system-wide suspend (via ->suspend() hook)
>> >> 
>> >> For TI SCI devices that are runtime PM enabled, check (1) will be the
>> >> primary method, and will happen when the TI SCI PM domain is powered
>> >> off (e.g. when the runtime PM usecount of the last device in that
>> >> domain goes to zero.)
>> >> 
>> >> For devices that are either not runtime PM enabled, or are not yet
>> >> runtime suspended (e.g. due to being used during the suspend path),
>> >> the constraints check will happen by check(2).
>> >> 
>> >> Since constraints can be sent by either (1) or (2), driver keeps track
>> >> of whether a valid constraint has been sent already.
>> >> 
>> >> An important detail here is that the PM domain driver inserts itself
>> >> into the path of both the ->suspend() and ->resume() hook path
>> >> of *all* devices in the PM domain.  This allows generic PM domain code
>> >> to handle the constraint management and communication with TI SCI.
>> >> 
>> >> Further, this allows device drivers to use existing PM QoS APIs to
>> >> add/update constraints.
>> >> 
>> >> DM firmware clears constraints during its resume, so Linux has
>> >> to check/update/send constraints each time system suspends.
>> >> 
>> >> Co-developed-by: Vibhore Vardhan <vibhore at ti.com>
>> >> Signed-off-by: Vibhore Vardhan <vibhore at ti.com>
>> >> Signed-off-by: Kevin Hilman <khilman at baylibre.com>
>> >> Signed-off-by: Dhruva Gole <d-gole at ti.com>
>> >
>> > Reviewed-by: Markus Schneider-Pargmann <msp at baylibre.com>
>> >
>> > In general this looks good, two small things below.
>> >
>> >> ---
>> >>  drivers/pmdomain/ti/ti_sci_pm_domains.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>> >>  1 file changed, 91 insertions(+), 1 deletion(-)
>> >> 
>> >> diff --git a/drivers/pmdomain/ti/ti_sci_pm_domains.c b/drivers/pmdomain/ti/ti_sci_pm_domains.c
>> >> index 1510d5ddae3d..4dc48a97f9b8 100644
>> >> --- a/drivers/pmdomain/ti/ti_sci_pm_domains.c
>> >> +++ b/drivers/pmdomain/ti/ti_sci_pm_domains.c
>> >> @@ -13,6 +13,8 @@
>> >>  #include <linux/platform_device.h>
>> >>  #include <linux/pm_domain.h>
>> >>  #include <linux/slab.h>
>> >> +#include <linux/pm_qos.h>
>> >> +#include <linux/pm_runtime.h>
>> >>  #include <linux/soc/ti/ti_sci_protocol.h>
>> >>  #include <dt-bindings/soc/ti,sci_pm_domain.h>
>> >>  
>> >> @@ -47,10 +49,46 @@ struct ti_sci_pm_domain {
>> >>  	struct generic_pm_domain pd;
>> >>  	struct list_head node;
>> >>  	struct ti_sci_genpd_provider *parent;
>> >> +	s32 lat_constraint;
>> >> +	bool constraint_sent;
>> >>  };
>> >>  
>> >>  #define genpd_to_ti_sci_pd(gpd) container_of(gpd, struct ti_sci_pm_domain, pd)
>> >>  
>> >> +static inline bool ti_sci_pd_is_valid_constraint(s32 val)
>> >> +{
>> >> +	return val != PM_QOS_RESUME_LATENCY_NO_CONSTRAINT;
>> >> +}
>> >> +
>> >> +static int ti_sci_pd_send_constraint(struct device *dev, s32 val)
>> >> +{
>> >> +	struct generic_pm_domain *genpd = pd_to_genpd(dev->pm_domain);
>> >> +	struct ti_sci_pm_domain *pd = genpd_to_ti_sci_pd(genpd);
>> >> +	const struct ti_sci_handle *ti_sci = pd->parent->ti_sci;
>> >> +	int ret;
>> >> +
>> >> +	ret = ti_sci->ops.pm_ops.set_latency_constraint(ti_sci, val, TISCI_MSG_CONSTRAINT_SET);
>> >> +	if (!ret) {
>> >> +		pd->constraint_sent = true;
>> >> +		dev_dbg(dev, "ti_sci_pd: ID:%d set latency constraint %d\n",
>> >> +			pd->idx, val);
>> >> +	} else {
>> >> +		dev_err(dev, "ti_sci_pd: set latency constraint failed: ret=%d\n",
>> >> +			ret);
>> >> +	}
>> >> +
>> >> +	return ret;
>> >> +}
>> >> +
>> >> +static inline void ti_sci_pd_clear_constraints(struct device *dev)
>> >> +{
>> >> +	struct generic_pm_domain *genpd = pd_to_genpd(dev->pm_domain);
>> >> +	struct ti_sci_pm_domain *pd = genpd_to_ti_sci_pd(genpd);
>> >> +
>> >> +	pd->lat_constraint = PM_QOS_RESUME_LATENCY_NO_CONSTRAINT;
>> >> +	pd->constraint_sent = false;
>> >> +}
>> >> +
>> >>  /*
>> >>   * ti_sci_pd_power_off(): genpd power down hook
>> >>   * @domain: pointer to the powerdomain to power off
>> >> @@ -59,6 +97,18 @@ static int ti_sci_pd_power_off(struct generic_pm_domain *domain)
>> >>  {
>> >>  	struct ti_sci_pm_domain *pd = genpd_to_ti_sci_pd(domain);
>> >>  	const struct ti_sci_handle *ti_sci = pd->parent->ti_sci;
>> >> +	struct pm_domain_data *pdd;
>> >> +
>> >> +	list_for_each_entry(pdd, &domain->dev_list, list_node) {
>> >> +		struct device *dev = pdd->dev;
>> >> +		s32 val;
>> >> +
>> >> +		/* If device has any resume latency constraints, send 'em */
>> >> +		val = dev_pm_qos_read_value(dev, DEV_PM_QOS_RESUME_LATENCY);
>> >> +		if (ti_sci_pd_is_valid_constraint(val) && !pd->constraint_sent)
>> >> +			ti_sci_pd_send_constraint(dev, val);
>> >> +		pd->lat_constraint = val;
>> >> +	}
>> >>  
>> >>  	return ti_sci->ops.dev_ops.put_device(ti_sci, pd->idx);
>> >>  }
>> >> @@ -79,6 +129,38 @@ static int ti_sci_pd_power_on(struct generic_pm_domain *domain)
>> >>  		return ti_sci->ops.dev_ops.get_device(ti_sci, pd->idx);
>> >>  }
>> >>  
>> >> +static int ti_sci_pd_resume(struct device *dev)
>> >> +{
>> >> +	ti_sci_pd_clear_constraints(dev);
>> >> +	return pm_generic_resume(dev);
>> >> +}
>> >> +
>> >> +static int ti_sci_pd_suspend(struct device *dev)
>> >> +{
>> >> +	struct generic_pm_domain *genpd = pd_to_genpd(dev->pm_domain);
>> >> +	struct ti_sci_pm_domain *pd = genpd_to_ti_sci_pd(genpd);
>> >> +	s32 val;
>> >> +	int ret;
>> >> +
>> >> +	ret = pm_generic_suspend(dev);
>> >> +	if (ret)
>> >> +		return ret;
>> >> +
>> >> +	/* Check if device has any resume latency constraints */
>> >> +	val = dev_pm_qos_read_value(dev, DEV_PM_QOS_RESUME_LATENCY);
>> >> +	if (ti_sci_pd_is_valid_constraint(val) && !pd->constraint_sent) {
>> >> +		if (genpd && genpd->status == GENPD_STATE_OFF)
>> >> +			dev_warn(dev, "%s: %s: already off.\n", genpd->name, __func__);
>> >> +		else if (pm_runtime_suspended(dev))
>> >> +			dev_warn(dev, "%s: %s: already RPM suspended.\n", genpd->name, __func__);
>> >> +		else
>> >> +			ti_sci_pd_send_constraint(dev, val);
>> >> +	}
>> >> +	pd->lat_constraint = val;
>> >
>> > Could you get rid of pd->constraint_sent? I don't really see a situation
>> > where it would be necessary.
>> 
>> It is necessary for runtime PM enabled devices that also use
>> constraints.  For a device like that, if a constraint is present it will
>> be sent when the PM domain is powered off (right after the runtime PM
>> usecount goes to zero.)  Later, during system-wide suspend, the suspend
>> hook for that same device might try to send the constraint again if we
>> don't keep track of whether the constraint was already sent.
>
> But wouldn't in that case pd->lat_constraint also be valid as you set it
> only after sending the constraint? So if you check for that instead of
> pd->constraint_sent, could you remove constraint_sent? On resume it is
> always reset as well.
>
> It just looked to me like if it is valid then it would have also been
> sent, which is the same semantic as the bool.

hmm, yeah.  I think you're right that constraint valid and constraint
sent are the same thing.

In earlier versions of the firmware, there was slightly different
behavior about how constraints were cleared, but with how it works now,
I think it can be removed.

I'll test with that for the next version.

Thanks for the review!

Kevin