[PATCH v3 11/27] media: rockchip: rga: avoid odd frame sizes for YUV formats

[Sven Püschel]

Hi Nicolas,

On 3/20/26 6:47 PM, Nicolas Dufresne wrote:
> Le mardi 27 janvier 2026 à 15:39 +0100, Sven Püschel a écrit :
>> Avoid odd frame sizes for YUV formats, as they may cause undefined
>> behavior. This is done in preparation for the RGA3, which hangs when the
>> output format is set to 129x129 pixel YUV420 SP (NV12).
>>
>> This requirement is documented explicitly for the RGA3 in  section 5.6.3
>> of the RK3588 TRM Part 2. For the RGA2 the RK3588 TRM Part 2
>> (section 6.1.2) and RK3568 TRM Part 2 (section 14.2) only mentions the
>> x/y offsets and stride aligning requirements. But the vendor driver for
>> the RGA2 also contains checks for the width and height to be aligned to
>> 2 bytes.
>>
>> Signed-off-by: Sven Püschel <s.pueschel at pengutronix.de>
>> ---
>>   drivers/media/platform/rockchip/rga/rga.c | 19 ++++++++++++++-----
>>   1 file changed, 14 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/media/platform/rockchip/rga/rga.c b/drivers/media/platform/rockchip/rga/rga.c
>> index 21a3c6cd38dbc..4fa6adb10b7ee 100644
>> --- a/drivers/media/platform/rockchip/rga/rga.c
>> +++ b/drivers/media/platform/rockchip/rga/rga.c
>> @@ -337,6 +337,19 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f)
>>   	struct rga_ctx *ctx = file_to_rga_ctx(file);
>>   	const struct rga_hw *hw = ctx->rga->hw;
>>   	struct rga_fmt *fmt;
>> +	struct v4l2_frmsize_stepwise frmsize = {
>> +		.min_width = hw->min_width,
>> +		.max_width = hw->max_width,
>> +		.min_height = hw->min_height,
>> +		.max_height = hw->max_height,
>> +		.step_width = 1,
>> +		.step_height = 1,
>> +	};
>> +
>> +	if (v4l2_is_format_yuv(v4l2_format_info(pix_fmt->pixelformat))) {
>> +		frmsize.step_width = 2;
>> +		frmsize.step_height = 2;
> For V4L2_PIX_FMT_YUV422P and NV16 this is 2/1. I believe you can generalize this
> with the format info, and skip this conditions:
>
> 		.step_with = info->vdiv,
> 		.step_height = info->hdiv
>
> Though, I'm saying that from a pixel format perspective, if the HW needs 2/2 for
> all YUV formats, let me know, I'll give my Rb.

I've introduced this limitation after running into the 129x129 hang and 
noticing that the librga just disallowed it. I've did a quick test with 
128x129 RK_FORMAT_YCbCr_422_SP (NV16) and the librga also doesn't allow it:

rga_api version 1.10.1_[4]
[  303.177480] rga: 1807   1807  : [tgid:1807] Destroy handle[1] when 
the user exits
101, check error! Invalid parameters: src, Error yuv not align to 2, 
rect[x,y,w,h] = [0, 0, 128, 129], wstride = 128, hstride = 129, format = 
0x800(cbcr422sp)

The librga docs [1] also indicate that for all YUV formats on all RGA 
versions the "height stride must be 2-aligned" (based on the librga 
error output "height stride" probably means "height in pixels"). 
Therefore I've copied this requirement globally into the driver to avoid 
potential problems (e.g. hangs or using memory outside of the actual image).

Sincerely
     Sven

[1] 
https://codeberg.org/airockchip/librga/src/branch/main/docs/Rockchip_Developer_Guide_RGA_EN.md#image-format-alignment-instructions

> Nicolas
>
>> +	}
>>   
>>   	if (V4L2_TYPE_IS_CAPTURE(f->type)) {
>>   		const struct rga_frame *frm;
>> @@ -358,11 +371,7 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f)
>>   	if (!fmt)
>>   		fmt = &hw->formats[0];
>>   
>> -	pix_fmt->width = clamp(pix_fmt->width,
>> -			       hw->min_width, hw->max_width);
>> -	pix_fmt->height = clamp(pix_fmt->height,
>> -				hw->min_height, hw->max_height);
>> -
>> +	v4l2_apply_frmsize_constraints(&pix_fmt->width, &pix_fmt->height, &frmsize);
>>   	v4l2_fill_pixfmt_mp(pix_fmt, fmt->fourcc, pix_fmt->width, pix_fmt->height);
>>   	pix_fmt->field = V4L2_FIELD_NONE;
>>