[PATCH] media: meson: vdec: size capture planes from the aligned canvas

[sashiko-bot at kernel.org]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] media: meson: vdec: integer overflow in get_output_size() causes OOB DMA write
--

commit 28bcd6a9db6c6247ecace49156692588db867097
Author: Michael Freidkin <freidkin at gmail.com>

media: meson: vdec: size capture planes from the aligned canvas

This commit aligns the width and height to 32 before calculating the output
size in get_output_size(). This fixes an issue where the allocated capture
planes were too small for the advertised stride when the width is not a
multiple of 32.

> diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
> index c18bf352b1996..be29d5a11c2e6 100644
> --- a/drivers/staging/media/meson/vdec/vdec.c
> +++ b/drivers/staging/media/meson/vdec/vdec.c
> @@ -34,7 +34,7 @@ struct dummy_buf {
>  
>  static u32 get_output_size(u32 width, u32 height)
>  {
> -	return ALIGN(width * height, SZ_64K);
> +	return ALIGN(ALIGN(width, 32) * ALIGN(height, 32), SZ_64K);

[Severity: Critical]
Does this new alignment calculation introduce an integer overflow that
results in a 0-byte DMA buffer?

A crafted VP9 bitstream can provide untrusted dimensions up to 65535x65535.
The firmware parses these into param->p.width and param->p.height in
drivers/staging/media/meson/vdec/codec_vp9.c:codec_vp9_process_rpm(), which
passes them directly to amvdec_src_change(). The dimensions are assigned
to sess->width and sess->height without clamping.

If a user handles the resulting source change event by calling
VIDIOC_REQBUFS directly, skipping VIDIOC_S_FMT which would normally clamp
the dimensions, get_output_size() is called with 65535x65535.

When these dimensions are aligned to 32 here, ALIGN(65535, 32) *
ALIGN(65535, 32) evaluates to 65536 * 65536, or 4,294,967,296. Because
get_output_size() uses u32, this multiplication overflows and wraps
exactly to 0.

vdec_queue_setup() then fetches this overflowed output size (0) to
initialize the DMA plane sizes, bypassing size validations and requesting a
0-byte DMA buffer. The hardware would then attempt to decode the frame into
this tiny buffer, writing gigabytes of data out of bounds.

Prior to this patch, width * height for 65535x65535 was 4,294,836,225,
which did not overflow u32 and correctly caused a safe allocation failure.

>  }
>  
>  u32 amvdec_get_output_size(struct amvdec_session *sess)

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260928165045.642354-1-freidkin@gmail.com?part=1