[PATCH v2] media: vcodec: Fix potential array out-of-bounds in encoder queue_setup

[Chen-Yu Tsai]

On Thu, Aug 10, 2023 at 4:23 PM Wei Chen <harperchen1110 at gmail.com> wrote:
>
> variable *nplanes is provided by user via system call argument. The
> possible value of q_data->fmt->num_planes is 1-3, while the value
> of *nplanes can be 1-8. The array access by index i can cause array
> out-of-bounds.
>
> Fix this bug by checking *nplanes against the array size.
>
> Fixes: 4e855a6efa54 ("[media] vcodec: mediatek: Add Mediatek V4L2 Video Encoder Driver")
> Signed-off-by: Wei Chen <harperchen1110 at gmail.com>
> Cc: stable at vger.kernel.org

Reviewed-by: Chen-Yu Tsai <wenst at chromium.org>