[PATCH v2 00/13] media: rockchip: rga: add support for multi-planar formats

[Michael Tretter]

Hi Hans,

On Fri, 24 Nov 2023 14:39:37 +0100, Hans Verkuil wrote:
> On 13/10/2023 13:00, Michael Tretter wrote:
> > This is v2 of the series that adds support for the V4L2 multi-planar API
> > to the Rockchip RGA driver. Once the RGA driver supports the
> > multi-planar API, it is easier to share buffers with other V4L2 drivers
> > that also support the multi-planar API and may not expose planar formats
> > with contiguous planes.
> > 
> > v2 fixes the smatch warnings and compile errors of v1. Furthermore, now
> > the DMA mask is set to 32 bits for coherent, too, and the gfp_flags are
> > configured to ensure that buffers are allocated in the lower 4GB area.
> > 
> > With non-contiguous planes, the U and V planes may not start at the same
> > offset as with the continuous planes. Therefore, the RGA driver cannot
> > rely on its calculation of the plane offsets based on the format and
> > frame size anymore, but must remember the offsets when it created the
> > mapping. Therefore, I also reworked how the DMA mapping is handled.
> > 
> > As a bonus, the RGA driver should now work correctly on devices with
> > more than 4 GB of memory. Video buffers should now be allocated within
> > the 4 GB boundary and an import of buffers that have higher addresses
> > into the driver should fail.
> > 
> > Patch 1 fixes the swizzling of RGA formats. While testing all formats of
> > the driver on rk3568, I discovered that the color channels of the RGB
> > formats are wrong when converting to NV12. I didn't test this on other
> > SoCs with an RGA and I am not sure, if they behave differently regarding
> > the color channels. Please report, if this breaks the color conversion
> > on other SoCs, and I will make this SoC-specific.
> > 
> > Patches 2 to 6 are the rework the DMA descriptor handling for the RGA
> > MMU. The patches clean up, how the driver uses the DMA API, and make the
> > creation of the descriptor list more explicit. Furthermore, the driver
> > is changed to keep the mapping per video buffer instead of using a
> > single mapping that is updated with every buffer.
> > 
> > Patches 7 to 11 prepare the driver for the multi-planar API including a
> > cleanup of the format handling in the buffer, and finally switch to the
> > multi-planar API.
> 
> Switching over to the multi-planar API is always a bit scary. How did you
> test this? Using gstreamer? Other apps?

I tested with GStreamer. My use case is using the RGA to convert RGB Buffers
to NV12 to feed them to a Hantro JPEG encoder. I also tried to separate the
preparatory work, the switch to the multi-planar API, and the addition of
actually multi-planar formats into separate patches.

Is there anything that I should take special care of or explicitly test, when
switching the driver to the multi-planar API?

Michael

> 
> Regards,
> 
> 	Hans
> 
> > 
> > Patch 12 updates the code that creates the DMA-descriptor mapping to
> > correctly handle buffers with multiple planes. The driver has to iterate
> > all planes and make them the continuous for the RGA.
> > 
> > Patch 13 enables the NV12M format, which is the multi-planar variant of
> > the NV12 format.
> > 
> > Michael
> > 
> > Signed-off-by: Michael Tretter <m.tretter at pengutronix.de>
> > ---
> > Changes in v2:
> > - Fix smatch warnings
> > - Fix cast to dst_mmu_pages/src_mmu_pages to fix compile error in Patch 2
> > - Remove check for upper_32_bits when filling the DMA descriptor table
> > - Remove useless dma_sync_single_for_device()
> > - Set DMA mask for DMA coherent
> > - Set gfp_flags to __GFP_DMA32
> > - Link to v1: https://lore.kernel.org/r/20230914-rockchip-rga-multiplanar-v1-0-abfd77260ae3@pengutronix.de
> > 
> > ---
> > Michael Tretter (13):
> >       media: rockchip: rga: fix swizzling for RGB formats
> >       media: rockchip: rga: extract helper to fill descriptors
> >       media: rockchip: rga: allocate DMA descriptors per buffer
> >       media: rockchip: rga: split src and dst buffer setup
> >       media: rockchip: rga: pre-calculate plane offsets
> >       media: rockchip: rga: set dma mask to 32 bits
> >       media: rockchip: rga: use clamp() to clamp size to limits
> >       media: rockchip: rga: use pixelformat to find format
> >       media: rockchip: rga: add local variable for pix_format
> >       media: rockchip: rga: use macros for testing buffer type
> >       media: rockchip: rga: switch to multi-planar API
> >       media: rockchip: rga: rework buffer handling for multi-planar formats
> >       media: rockchip: rga: add NV12M support
> > 
> >  drivers/media/platform/rockchip/rga/rga-buf.c | 162 ++++++++++++++++------
> >  drivers/media/platform/rockchip/rga/rga-hw.c  | 146 ++++++++++++--------
> >  drivers/media/platform/rockchip/rga/rga.c     | 189 ++++++++++++--------------
> >  drivers/media/platform/rockchip/rga/rga.h     |  35 ++++-
> >  4 files changed, 328 insertions(+), 204 deletions(-)
> > ---
> > base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235
> > change-id: 20230914-rockchip-rga-multiplanar-0d7c79b1ba93
> > 
> > Best regards,
> 
>