[RFC] accel/rocket: DVFS on RK3588 - a hardware constraint, and some numbers

[Igor Paunovic]

Hi Jiaxing,

> Your rows are one inference thread with a bit-exact oracle, so they
> cannot see this class at all. Before the OPP table is settled, run
> the oracle with all three cores loaded at 900 and 1000 MHz on 850 mV.

You were right, and it stopped me sending. The series was packed and
checked and I was an hour from the send command. Every number I have
posted, including the ones in this thread, came from a single inference
thread. Not one of them could have seen what you found.

So I built the test I did not have. Three concurrent clients, each
checking its own output bit-exact, on the two top rates of the table:

  900 MHz, rail 800 mV    3 x 199 inf/s, 598 total, all bit-exact
  1000 MHz, rail 850 mV   3 x 204 inf/s, 611 total, all bit-exact

Single-client control on the same rates: 232 and 240 inf/s. So the
aggregate is 2.57x and 2.55x of one client, which is the part that
makes the bit-exact result worth anything - the three cores really were
computing at the same time, not queueing behind one. Kernel log clean
through all four passes. Voltages read back from vdd_npu_s0 during the
run, put there by the OPP core, not by me.

One honest note on how I proved the overlap, because I got it wrong
first. I had the test sample runtime_status of the three cores and
report how many were active. It said three of three, one hundred per
cent - and it said that with a single client too, because this series
holds every core resumed while the clock is raised. It measures power
state, not work. The aggregate throughput is the real evidence; the
sampler was telling me what I wanted to hear.

So: on RK3588 the answer to your question is that it holds, at the
voltages the table names. What I cannot tell you is where the edge is.
I have not run 1000 MHz at 800 mV to find out how much margin 850 is
buying, and I would rather not guess in a commit message.

> whatever the table ends up as, it needs the voltage column with the
> rate: a rate without its rail is what mainline has today, and it is
> what corrupts.

The table carries it: 200 to 700 MHz at 700 mV, 800 at 750, 900 at 800,
1000 at 850, which is Rockchip's own table for this part. Your four
days are now a paragraph in my cover letter with your name on it, since
that measurement is the reason the test exists.

> On RK3576 an assigned-clock-rates on the SCMI clock in the NPU node
> hangs the board before the console comes up

That is worth more than the question I was going to ask about it. I
have kept assigned-clock-rates on all three RK3588 nodes and listed it
as an open question, on the grounds that of_clk_set_defaults() runs on
every probe over the shared clock and could quietly lower a raised
rate - which I have not managed to make happen. Your board says the
property can do considerably worse than that. I will say so in the
cover and let the maintainers decide whether it goes.

Two other things while I have you.

I am sending a fix ahead of the series: rocket_remove() decrements
num_cores while find_core_for_dev() searches only that far, so the last
core is never found on unbind, num_cores never reaches zero, and a
rebind writes rdev->cores[3] on a three-element array. Silent in
mainline today; UBSAN caught it once devfreq started walking the array
from a worker. It has a Fixes: tag and Cc: stable. You may want it on
RK3576 too, where the same code runs with two cores.

And the small one I have been sitting on. My series carries the
clocks-by-name patch as 1/7 so it applies on its own, and that copy
still has your Signed-off-by from v11. You are not in the delivery path
of my series, so by submitting-patches.rst it does not belong on that
copy and I would drop it there - your Reviewed-by is the credit that
matters and it stays, on both copies. On the v11 copy your sign-off is
correct and I would not touch it. It is your name, so I would rather
ask than decide.

Igor