[PATCH v7 08/10] accel/rocket: add RK3576 NPU (RKNN) support

[Igor Paunovic]

Hi Jiaxing,

Your conv2d-cal result made me curious whether the clamp is a property of
the RKNN output stage in general, so I built the same shape here: 1x1
conv, 64 input channels, 8x8 surface, uint8, out_zp 128, no fused
activation - and compared against both references. On RK3588 the answer
is the opposite of yours.

  output channels    vs raw CPU              vs max(cpu, out_zp)
   56                56 of 56 (47 exact)      0 of 56
  128               128 of 128 (113 exact)    6 of 128

Maxdiff against the raw reference is 1 in both cases; against the clamped
reference it is 128. Where the raw reference falls below the zero point,
the hardware output spans 0..127 rather than sitting at 128, and only
0.6-0.8% of hardware values land exactly on the zero point, which is
about what the distribution gives you by chance.

So RK3588 does not clamp at the output zero point, and the clamp you are
seeing is not something the DPU output stage does everywhere. That is one
more thing it is not, and I think it is the expensive one to rule out by
inspection.

Three controls, since a "hardware equals CPU" result is exactly what a
silent delegate fallback would also produce:

  - the per-core NPU interrupt counters advance by one per run, so the
    job did reach the hardware
  - 15 of the 128 channels differ from the CPU by one, which a CPU
    fallback could not produce - it would be bit-identical
  - 46% (oc=128) and 59% (oc=56) of the raw reference lies below the zero
    point, so there was something for a clamp to remove

One thing I cannot tell from here: whether the difference is the silicon
or the userspace path. My runs are upstream Mesa main with no RK3576
patches, and yours are your working tree, so the two differ in more than
the SoC. If you still have the RK3576 board on a stock upstream Mesa, the
same shape there would separate those two explanations - and if it turns
out to be userspace rather than silicon, that is a much easier bug to
find than a hardware one.

Yes to the 88 and 120 sweep on RK3576, and thank you for offering it. If
the toggle follows the same rule at those counts I can re-run the RK3588
side with the forced constant for symmetry, so we would have both forms
on both SoCs at four channel counts.

Best regards,
Igor


On Sun, Aug 16, 2026 at 9:58 PM Jiaxing Hu <gahing at gahingwoo.com> wrote:
>
> Hi Igor,
>
> You are right that it is fitted, and decoding it against registers.xml was
> worth more than the fit was. I had been treating those two words as opaque
> constants and comparing predicates, when the thing that varies is one bit.
>
> Your reading also named an experiment I had never run. Every measurement this
> board has produced was of the value my predicate happens to emit, so 16 and 41
> output channels had only ever run with SIZE_E_2 0, and 56, 64 and 128 only with
> 1. The cell nobody had filled in was SIZE_E_2 1 at the small counts. Forcing
> the constant fills it.
>
>   output channels    predicate     forced SIZE_E_2 1
>    16                16 of 16      NPU job timed out, 0 of 16
>    41                41 of 41      0 of 41
>    56                56 of 56      56 of 56
>   128              128 of 128    128 of 128
>
> So each form fails on the shapes the other handles, which makes it a two sided
> measurement rather than the one sided fit it started as. Taken with your RK3588
> result, RK3588 does not need the toggle and RK3576 does, and what the toggle
> selects is SIZE_E_2 rather than anything about the channel count as such. I
> will write it that way, as the field with a reason, rather than as a modulo or
> a parity of something.
>
> The other bits you flagged I cannot defend yet. RGP_CNTER 8, OW_SRC 1 and the
> two in the reserved range came from vendor captures and have never been varied
> one at a time. That is a sweep this board can run and I will do it before the
> Mesa series goes out, since a value nobody can explain is a value nobody should
> be asked to review.
>
> On the unclamped reference, thank you for using it. It changed what I thought I
> had here as well. The clamp is only free where the quantised output range
> starts at zero, and on a middle zero point layer it rewrites about half the
> surface. conv2d-cal is out_zp 128 with no fused activation, and against the
> unclamped output it is 0 of 128 channels rather than 128 of 128, with the
> hardware equal to max(cpu, out_zp) to within one everywhere. Where that clamp
> comes from is still open. It is not the output offset, not the BS block, not
> any register in the stream, and not the coefficient records, all four measured
> rather than argued.
>
> If you want another shape, an output channel count of 88 or 120 on RK3576 would
> tell us whether the toggle follows the same rule up there, and those are exactly
> the counts you already have on the other SoC.
>
> Jiaxing