[BUG] rkvdec-vdpu383-h264: wrong pixels at horizontal de-blocking edges y=4 and y=12

[Simon Wright]

Hi Detlev,

Following up on the BL31 lead from my last mail -- we dug further and
found the root cause.

It's an un-primed hardware state after power-up, not BL31 and not a
per-frame register. The Rockchip BSP runs a one-shot priming decode at
every decoder power-on -- rk3576_workaround_run() in
drivers/video/rockchip/mpp/hack/mpp_hack_rk3576.c builds a tiny
self-contained H.264 task and runs it through the decoder at probe and
on every pm_runtime resume. Mainline rkvdec has no equivalent, so the
first decode(s) after each power-up run with indeterminate internal
deblock state.

That makes it a non-deterministic power-up-state race, which is
consistent with everything in the thread: your inability to reproduce
it, the "10% of the time vs every time" split between the two reports,
and the BL31 correlation I sent last time. The odds depend on what the
boot stack leaves in the IP, which would explain why it tracked BL31
version -- but the warmup removes that dependence. (It's also why the
vendor driver is always correct: it does this priming; that's what it
has that mainline doesn't, rather than a hardware difference.)

The fix is to run that warmup at pm_runtime resume -- the minimal
sufficient trigger; running it only at start_streaming left a ~10%
residual. With it on we get 64/64 decodes bit-exact against a
reproduced baseline race, on the same v1.20 Armbian stack that
previously showed the bug; with it off the race returns. So it's
independent of BL31.

A reference implementation -- the DMA priming buffer plus the
link-register kick sequence, ported from the BSP -- is in the repo I
shared (the warmup code itself is in the sibling rkvdec-vdpu383-vp9 /
-av1 repos, src/rkvdec-link.c):

  https://github.com/SympleNZ/rkvdec-vdpu383-h264-bug

I'm not sure of the right mainline form for a vendor-disassembly-
derived priming sequence, so I've left it as a report + reference
rather than a patch -- you're much better placed to decide how, or
whether, it belongs in rkvdec. Happy to test any version on the RK3576
boards here.

(This is a correctness fix only; mainline VDPU383 throughput is a
separate matter and isn't affected by it.)

Regards,
Simon