[PATCH] media: mediatek: vcodec: fix the error sizeimage for 10bit bitstream

[Sebastian Fricke]

Hey Yunfei,

On 03.04.2024 17:30, Yunfei Dong wrote:
>The sizeimage of each plane are calculated the same way for 8bit and

s/The sizeimage of each plane are/The sizeimage for each plane is/

>10bit bitstream. Need to enlarge the sizeimage with simeimage*5/4 for
>10bit bitstream when try and set fmt.

s/bitstream/bistreams/
s/Need to enlarge the sizeimage with simeimage*5/4 for 10bit bitstream when try and set fmt./
   Scale up the sizeimage by 25% for 10-bit bitstreams in try_fmt./

>
>Fixes: 9d86be9bda6c ("media: mediatek: vcodec: Add driver to support 10bit")
>Signed-off-by: Yunfei Dong <yunfei.dong at mediatek.com>
>---
> .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  | 47 ++++++++++++++-----
> 1 file changed, 34 insertions(+), 13 deletions(-)
>
>diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
>index 9107707de6c4..45209894f1fe 100644
>--- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
>+++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
>@@ -259,6 +259,7 @@ static int vidioc_try_fmt(struct mtk_vcodec_dec_ctx *ctx, struct v4l2_format *f,
> 		pix_fmt_mp->num_planes = 1;
> 		pix_fmt_mp->plane_fmt[0].bytesperline = 0;
> 	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
>+		unsigned int dram_y, dram_c, dram_y_10bit, dram_c_10bit;
> 		int tmp_w, tmp_h;
>
> 		/*
>@@ -280,22 +281,42 @@ static int vidioc_try_fmt(struct mtk_vcodec_dec_ctx *ctx, struct v4l2_format *f,
> 		    (pix_fmt_mp->height + 64) <= frmsize->max_height)
> 			pix_fmt_mp->height += 64;
>
>-		mtk_v4l2_vdec_dbg(0, ctx,
>-				  "before resize wxh=%dx%d, after resize wxh=%dx%d, sizeimage=%d",
>-				  tmp_w, tmp_h, pix_fmt_mp->width, pix_fmt_mp->height,
>-				  pix_fmt_mp->width * pix_fmt_mp->height);
>+		dram_y = pix_fmt_mp->width * pix_fmt_mp->height;
>+		dram_c = dram_y / 2;
>+
>+		dram_y_10bit = dram_y * 5 / 4;
>+		dram_c_10bit = dram_y_10bit / 2;

I'd skip the two 10 bit variables (dram_y_10bit & dram_c_10bit) and
instead do it like this:

```
   dram_stride = pix_fmt_mp->width;
   if (ctx->is_10bit_bitstream)
     dram_stride = dram_stride * 5 / 4;

   dram_y = dram_stride * pix_fmt_mp->height;
   dram_c = dram_y / 2;

   if (pix_fmt_mp->num_planes == 1) {
     pix_fmt_mp->plane_fmt[0].bytesperline = dram_stride;
     pix_fmt_mp->plane_fmt[0].sizeimage = dram_y + dram_c;
   } else {
     pix_fmt_mp->plane_fmt[0].bytesperline = dram_stride;
     pix_fmt_mp->plane_fmt[1].bytesperline = dram_stride;
     pix_fmt_mp->plane_fmt[0].sizeimage = dram_y;
     pix_fmt_mp->plane_fmt[1].sizeimage = dram_c;
     ...
   }
```

Also, why do you call all the variables dram?

Please this isn't tested, but shows the general direction to repeat a
lot less code.

Greetings,
Sebastian

>
> 		pix_fmt_mp->num_planes = fmt->num_planes;
>-		pix_fmt_mp->plane_fmt[0].sizeimage =
>-				pix_fmt_mp->width * pix_fmt_mp->height;
>-		pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
>-
>-		if (pix_fmt_mp->num_planes == 2) {
>-			pix_fmt_mp->plane_fmt[1].sizeimage =
>-				(pix_fmt_mp->width * pix_fmt_mp->height) / 2;
>-			pix_fmt_mp->plane_fmt[1].bytesperline =
>-				pix_fmt_mp->width;
>+		if (pix_fmt_mp->num_planes == 1) {
>+			if (ctx->is_10bit_bitstream) {
>+				pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width * 5 / 4;
>+				pix_fmt_mp->plane_fmt[0].sizeimage = dram_y_10bit + dram_c_10bit;
>+			} else {
>+				pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
>+				pix_fmt_mp->plane_fmt[0].sizeimage = dram_y + dram_c;
>+			}
>+		} else {
>+			if (ctx->is_10bit_bitstream) {
>+				pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width * 5 / 4;
>+				pix_fmt_mp->plane_fmt[1].bytesperline = pix_fmt_mp->width * 5 / 4;
>+
>+				pix_fmt_mp->plane_fmt[0].sizeimage = dram_y_10bit;
>+				pix_fmt_mp->plane_fmt[1].sizeimage = dram_c_10bit;
>+			} else {
>+				pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
>+				pix_fmt_mp->plane_fmt[1].bytesperline = pix_fmt_mp->width;
>+
>+				pix_fmt_mp->plane_fmt[0].sizeimage = dram_y;
>+				pix_fmt_mp->plane_fmt[1].sizeimage = dram_c;
>+			}
> 		}
>+
>+		mtk_v4l2_vdec_dbg(0, ctx,
>+				  "before resize:%dx%d, after resize:%dx%d, sizeimage=0x%x_0x%x",
>+				  tmp_w, tmp_h, pix_fmt_mp->width, pix_fmt_mp->height,
>+				  pix_fmt_mp->plane_fmt[0].sizeimage,
>+				  pix_fmt_mp->plane_fmt[1].sizeimage);
> 	}
>
> 	pix_fmt_mp->flags = 0;
>-- 
>2.25.1
>