[PATCH] media: rkvdec: hevc: cap EXT SPS RPS control counts before descriptor assembly

[Detlev Casanova]

Hi Michael,

On 5/13/26 14:19, Michael Bommarito wrote:
> V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS and
> V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS are registered as dynamic-size
> controls with a per-control element cap of 65. The V4L2 control core
> enforces only the payload-element cap. It does not bound the spec-derived
> count fields that the rkvdec HEVC helper later uses to walk fixed hardware
> descriptor tables and temporary helper arrays:
>
>    - struct rkvdec_rps::refs[32]
>    - struct rkvdec_rps::short_term_ref_sets[64]
>    - struct calculated_rps_st_set::delta_poc_s0[16] / delta_poc_s1[16]
Still, did you try just changing the cap to 64 (.cfg.dims = { 64 },) ?
You'd need a test that sets the control from userspace though.

It should refuse setting the control if there are more than 64 elements, 
therefore the hevc decoder will not run any function using the count 
values from the SPS (See  rkvdec-vdpu381-hevc.c:601)
> A userspace V4L2 client that can open the Rockchip RKVDEC m2m decoder node
> may submit SPS/RPS controls whose counts exceed those capacities or whose
> prediction reference index underflows. rkvdec_hevc_assemble_hw_rps() then
> walks past the descriptor table or temporary-array bounds.
>
> KASAN under a small KUnit harness wrapping the real helper reports
> slab-out-of-bounds in all of:
>
>    - num_short_term_ref_pic_sets = 65 (write past short_term_ref_sets[64])
>    - num_long_term_ref_pics_sps = 33  (write past refs[32], intra-struct)
>    - ext_sps_st_rps[i].num_negative_pics or num_positive_pics > 16
>      (write past delta_poc_s0[16] inside calculated_rps_st_set)
>    - INTER_REF_PIC_SET_PRED with delta_idx_minus1 + 1 > i
>      (u8 ref_rps_idx underflow then OOB read on calculated_rps_st_sets)
>
> Validate the SPS/RPS counts before calling the assembly helpers. The cap
> values match both the HEVC spec ranges (num_short_term_ref_pic_sets <= 64,
> num_long_term_ref_pics_sps <= 32) and the fixed driver descriptor and
> helper-array capacities. Reject controls whose counts exceed those, and
> reject prediction entries whose reference index would underflow.
>
> Fixes: c9a59dc2acc7 ("media: rkvdec: Add HEVC support for the VDPU381 variant")
> Signed-off-by: Michael Bommarito <michael.bommarito at gmail.com>
> Assisted-by: Claude:claude-opus-4-7
> ---
>
> I don't have a RK3588 / RK3576 board to confirm this through a real
> /dev/videoN request path yet, but was convinced enough by:
>
>    1. Static reach: registration of EXT_SPS_ST_RPS / EXT_SPS_LT_RPS with
>       .dims = { 65 } at drivers/media/platform/rockchip/rkvdec/rkvdec.c
>       :239-287, the SPS-count-driven loops in rkvdec-hevc-common.c
>       :213-225 and :228-251, and v4l2-ctrls-core.c :1213-1277, which
>       validates only EXT RPS flags and not the spec-derived count fields.
>
>    2. A KUnit harness (separate, not in this patch) that allocates one
>       struct rkvdec_rps + a single calculated_rps_st_sets element via
>       kunit_kzalloc / kzalloc and calls the real
>       rkvdec_hevc_assemble_hw_rps() helper. Under UML + KASAN_GENERIC with
>       the kasan_multi_shot boot param, on a stock tree it produces these
>       reports:
>
>         BUG: KASAN: slab-out-of-bounds in
>           rkvdec_hevc_assemble_hw_rps+0xb0c/0x1080
>           (num_short_term_ref_pic_sets = 65, write of size 36 0 bytes past
>           the rps allocation)
>
>         BUG: KASAN: slab-out-of-bounds in
>           rkvdec_hevc_assemble_hw_rps (num_negative_pics = 64, write past
>           the single-element kzalloc'd calculated_rps_st_sets buffer)
>
>         BUG: KASAN: slab-use-after-free / slab-out-of-bounds reads via
>           u8 ref_rps_idx underflow at calculated_rps_st_sets[255]
>           (INTER_REF_PIC_SET_PRED with delta_idx_minus1 = 0, idx = 0)
>
>       The num_long_term_ref_pics_sps = 33 case is invisible to KASAN
>       (the OOB write lands inside struct rkvdec_rps) but corrupts
>       short_term_ref_sets[0]; the harness asserts that case explicitly.
>
>    3. Same harness on the patched tree: all five cases (four adversarial
>       plus a legitimate-limit regression with ST=64, LT=32, num_neg=1)
>       pass clean, no KASAN reports.
>
> If hardware-side validation actually does reject these counts before
> rkvdec_hevc_assemble_hw_rps() runs and this patch is unnecessary, please
> say so and I will withdraw it. If it is reachable, I will follow up with
> a runtime hardware splat once the Orange Pi board I bought arrives.
>
> Let me know if you want a patch set with the KUnit harnesses too.
>
> checkpatch.pl: 0 errors / 0 warnings.
>
>   .../rockchip/rkvdec/rkvdec-hevc-common.c      | 49 +++++++++++++++++++
>   1 file changed, 49 insertions(+)
>
> diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
> index 3119f3bc9f98..895fb16bc572 100644
> --- a/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
> +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec-hevc-common.c
> @@ -408,9 +408,58 @@ static void rkvdec_hevc_prepare_hw_st_rps(struct rkvdec_hevc_run *run, struct rk
>   	memcpy(cache, run->ext_sps_st_rps, sizeof(struct v4l2_ctrl_hevc_ext_sps_st_rps));
>   }
>   
> +/*
> + * V4L2 caps the EXT_SPS RPS payload length but not the SPS-derived counts
> + * that the helpers walk. Caps match the HEVC spec ranges.
> + */
> +#define RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS	64
> +#define RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS	32
> +#define RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS	16
> +
> +static int rkvdec_hevc_validate_rps_ctrls(struct rkvdec_hevc_run *run)
> +{
> +	const struct v4l2_ctrl_hevc_sps *sps = run->sps;
> +
> +	if (run->ext_sps_lt_rps &&
> +	    sps->num_long_term_ref_pics_sps >
> +	    RKVDEC_HEVC_MAX_LONG_TERM_REF_PICS_SPS)
> +		return -EINVAL;
> +
> +	if (run->ext_sps_st_rps) {
> +		unsigned int i;
> +
> +		if (sps->num_short_term_ref_pic_sets >
> +		    RKVDEC_HEVC_MAX_SHORT_TERM_REF_PIC_SETS)
> +			return -EINVAL;
> +
> +		for (i = 0; i < sps->num_short_term_ref_pic_sets; i++) {
> +			const struct v4l2_ctrl_hevc_ext_sps_st_rps *r =
> +				&run->ext_sps_st_rps[i];
> +
> +			if (r->num_negative_pics >
> +			    RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS ||
> +			    r->num_positive_pics >
> +			    RKVDEC_HEVC_MAX_RPS_NEG_POS_PICS)
> +				return -EINVAL;
> +
> +			if ((r->flags &
> +			     V4L2_HEVC_EXT_SPS_ST_RPS_FLAG_INTER_REF_PIC_SET_PRED) &&
> +			    (unsigned int)r->delta_idx_minus1 + 1 > i)
> +				return -EINVAL;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
>   void rkvdec_hevc_assemble_hw_rps(struct rkvdec_hevc_run *run, struct rkvdec_rps *rps,
>   				 struct v4l2_ctrl_hevc_ext_sps_st_rps *st_cache)
>   {
> +	if (rkvdec_hevc_validate_rps_ctrls(run)) {
> +		pr_err_ratelimited("rkvdec: rejecting HEVC SPS/RPS controls with out-of-range counts\n");
> +		return;
> +	}
> +
>   	rkvdec_hevc_prepare_hw_st_rps(run, rps, st_cache);
>   	rkvdec_hevc_assemble_hw_lt_rps(run, rps);
>   }