[PATCH v2 2/4] media: rockchip: Add JPEG decoder driver
Sascha Hauer
s.hauer at pengutronix.de
Tue Aug 25 06:44:07 PDT 2026
Add a driver for the JPEG hardware decoder Rockchip integrates into a
number of its SoCs. Downstream it is known as the VDPU720, which is the
name the register definitions carry here.
Exposes one V4L2 M2M device implementing the stateful decoder interface:
JPEG input -> NV12 output. The frame header is parsed on the CPU when a
coded buffer is queued, which is where the resolution for
V4L2_EVENT_SOURCE_CHANGE comes from; userspace cannot know the frame size
without parsing the bitstream itself, so it learns it from the event and
sizes its capture buffers accordingly. V4L2_DEC_CMD_STOP drains and
raises V4L2_EVENT_EOS.
Hardware requirements:
- a DMA side buffer with Q-tables (zigzag->raster), Huffman mincode and
value tables, rebuilt from the frame header on every run
- a two-phase IRQ clear sequence, which some revisions require. It is
done unconditionally, see rkjpegd-vdpu720-regs.h
- 16-byte stream alignment with a start-byte offset
- MCU-aligned PIC_H (e.g. 1080p YUV420 needs 1088, not 1080)
- FILL_DOWN_E on every NV12 conversion, the output chroma is vertically
subsampled so the hardware completes the bottom of the picture
- DRI (restart interval) support when present
- a neutral chroma plane written by the driver for a grayscale frame.
The output format converter has no YUV400 path, so the hardware writes
the luma plane and leaves the chroma alone, which comes out green
Every address register is a single 32 bit one with no high order
companion anywhere in the map, so the DMA masks are capped accordingly.
That is no constraint in practice: the block sits behind the Rockchip
IOMMU, whose address space is 32 bit wide by construction, and memory
above 4GB is reached through it.
A frame that carries no EOI marker is rejected: that is what a coded
buffer too small for the frame looks like, and the hardware would
otherwise hand out a half decoded picture indistinguishable from a good
one. A frame larger than the negotiated capture format is rejected too,
since its dimensions would be programmed against strides taken from that
format.
Error recovery tries the in-block soft reset first and falls back to
pulsing the reset lines. The interrupt handler decodes the error status
registers into a readable diagnostic.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Sascha Hauer <s.hauer at pengutronix.de>
---
MAINTAINERS | 1 +
drivers/media/platform/rockchip/Kconfig | 1 +
drivers/media/platform/rockchip/Makefile | 1 +
drivers/media/platform/rockchip/rkjpegd/Kconfig | 14 +
drivers/media/platform/rockchip/rkjpegd/Makefile | 3 +
.../rockchip/rkjpegd/rkjpegd-vdpu720-regs.h | 230 ++
drivers/media/platform/rockchip/rkjpegd/rkjpegd.c | 2328 ++++++++++++++++++++
7 files changed, 2578 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index baea7e42663a7..9bc3dda5a4b09 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -23431,6 +23431,7 @@ L: linux-media at vger.kernel.org
L: linux-rockchip at lists.infradead.org
S: Maintained
F: Documentation/devicetree/bindings/media/rockchip,jpeg-decoder.yaml
+F: drivers/media/platform/rockchip/rkjpegd/
ROCKCHIP RK3568 RANDOM NUMBER GENERATOR SUPPORT
M: Daniel Golle <daniel at makrotopia.org>
diff --git a/drivers/media/platform/rockchip/Kconfig b/drivers/media/platform/rockchip/Kconfig
index ba401d32f01ba..53d28eb89e472 100644
--- a/drivers/media/platform/rockchip/Kconfig
+++ b/drivers/media/platform/rockchip/Kconfig
@@ -5,4 +5,5 @@ comment "Rockchip media platform drivers"
source "drivers/media/platform/rockchip/rga/Kconfig"
source "drivers/media/platform/rockchip/rkcif/Kconfig"
source "drivers/media/platform/rockchip/rkisp1/Kconfig"
+source "drivers/media/platform/rockchip/rkjpegd/Kconfig"
source "drivers/media/platform/rockchip/rkvdec/Kconfig"
diff --git a/drivers/media/platform/rockchip/Makefile b/drivers/media/platform/rockchip/Makefile
index 0e0b2cbbd4bd2..c256c51ced269 100644
--- a/drivers/media/platform/rockchip/Makefile
+++ b/drivers/media/platform/rockchip/Makefile
@@ -2,4 +2,5 @@
obj-y += rga/
obj-y += rkcif/
obj-y += rkisp1/
+obj-y += rkjpegd/
obj-y += rkvdec/
diff --git a/drivers/media/platform/rockchip/rkjpegd/Kconfig b/drivers/media/platform/rockchip/rkjpegd/Kconfig
new file mode 100644
index 0000000000000..4e2b40ef60d83
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkjpegd/Kconfig
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0
+config VIDEO_ROCKCHIP_JPEGD
+ tristate "Rockchip JPEG decoder driver"
+ depends on ARCH_ROCKCHIP || COMPILE_TEST
+ depends on VIDEO_DEV
+ select MEDIA_CONTROLLER
+ select V4L2_JPEG_HELPER
+ select V4L2_MEM2MEM_DEV
+ select VIDEOBUF2_DMA_CONTIG
+ help
+ Support for the JPEG decoder Rockchip integrates into a number of
+ its SoCs, decoding JPEG and MJPEG frames to NV12.
+ To compile this driver as a module, choose M here: the module
+ will be called rockchip-jpegd.
diff --git a/drivers/media/platform/rockchip/rkjpegd/Makefile b/drivers/media/platform/rockchip/rkjpegd/Makefile
new file mode 100644
index 0000000000000..2aaf6bfcfdb74
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkjpegd/Makefile
@@ -0,0 +1,3 @@
+obj-$(CONFIG_VIDEO_ROCKCHIP_JPEGD) += rockchip-jpegd.o
+
+rockchip-jpegd-y += rkjpegd.o
diff --git a/drivers/media/platform/rockchip/rkjpegd/rkjpegd-vdpu720-regs.h b/drivers/media/platform/rockchip/rkjpegd/rkjpegd-vdpu720-regs.h
new file mode 100644
index 0000000000000..5e2cb017cca51
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkjpegd/rkjpegd-vdpu720-regs.h
@@ -0,0 +1,230 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Rockchip VPU720 JPEG decoder register definitions
+ *
+ * Derived from downstream Rockchip MPP HAL (hal_jpegd_rkv_reg.h).
+ * Copyright (C) 2020 Rockchip Electronics Co., Ltd.
+ * Copyright (C) 2026 WolfVision GmbH
+ */
+#ifndef RKJPEGD_VDPU720_REGS_H_
+#define RKJPEGD_VDPU720_REGS_H_
+
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+#include <linux/align.h>
+#include <linux/types.h>
+
+#define VDPU720_REG_VERSION 0x000
+#define VDPU720_PROD_NUM GENMASK(31, 16)
+#define VDPU720_BIT_DEPTH BIT(8)
+
+#define VDPU720_REG_INT 0x004
+#define VDPU720_DEC_E BIT(0)
+#define VDPU720_IRQ_DIS BIT(1)
+#define VDPU720_TIMEOUT_E BIT(2)
+#define VDPU720_BUF_EMPTY_E BIT(3)
+#define VDPU720_BUF_EMPTY_RELOAD BIT(4)
+#define VDPU720_SOFT_RST_EN BIT(5)
+#define VDPU720_IRQ_RAW BIT(6)
+#define VDPU720_WAIT_RESET_E BIT(7)
+#define VDPU720_IRQ BIT(8)
+#define VDPU720_DEC_RDY BIT(9)
+#define VDPU720_BUS_ERR BIT(10)
+#define VDPU720_DEC_ERR BIT(11)
+#define VDPU720_TIMEOUT BIT(12)
+#define VDPU720_BUF_EMPTY BIT(13)
+#define VDPU720_SOFT_RST_RDY BIT(14)
+
+/* Error status bits used to decide whether a hardware reset is needed */
+#define VDPU720_ERR_MASK (VDPU720_BUS_ERR | VDPU720_DEC_ERR | \
+ VDPU720_TIMEOUT | VDPU720_BUF_EMPTY)
+
+/*
+ * VPU720 IRQ clear mask.
+ *
+ * Some revisions require a two-step IRQ acknowledgment before checking
+ * VDPU720_IRQ_RAW. Compute the masked value and write it back, then
+ * write 0 to fully clear. The downstream BSP uses:
+ *
+ * clr = (~(0x00fe7f40 & status)) & (0xff0180bf & status)
+ *
+ * Preserve "status" bits from 0xff0180bf while clearing only those
+ * bits NOT already set in 0x00fe7f40.
+ *
+ * It is applied unconditionally here. The BSP gates it on the revision in
+ * VDPU720_REG_VERSION matching 0xdb1f0006 exactly, and the RK3588 reports
+ * 0xdb1f0005, so it is not needed on that part. The extra write is
+ * harmless there and saves a revision check that would need silicon we do
+ * not have to validate.
+ */
+#define VDPU720_IRQ_CLR_KEEP 0xff0180bf
+#define VDPU720_IRQ_CLR_COND 0x00fe7f40
+
+#define VDPU720_REG_SYS 0x008
+#define VDPU720_FORCE_SOFTRST BIT(17) /* set when dec_e=0 before soft-reset */
+#define VDPU720_FILL_DOWN_E BIT(24) /* fill bottom padding rows */
+#define VDPU720_FILL_RIGHT_E BIT(25)
+#define VDPU720_OUT_SEQ BIT(26) /* 0=raster, 1=tile */
+#define VDPU720_YUV_OUT_FMT GENMASK(29, 27)
+#define VDPU720_YUV_OUT_FMT_NATIVE 0 /* no format conversion */
+#define VDPU720_YUV_OUT_FMT_NV12 3 /* output as NV12 */
+
+#define VDPU720_REG_PIC_SIZE 0x00c
+#define VDPU720_PIC_W_M1 GENMASK(15, 0)
+#define VDPU720_PIC_H_M1 GENMASK(31, 16)
+
+#define VDPU720_REG_PIC_FMT 0x010
+#define VDPU720_JPEG_MODE GENMASK(2, 0)
+#define VDPU720_JPEG_MODE_YUV400 0
+#define VDPU720_JPEG_MODE_YUV411 1
+#define VDPU720_JPEG_MODE_YUV420 2
+#define VDPU720_JPEG_MODE_YUV422 3
+#define VDPU720_JPEG_MODE_YUV440 4
+#define VDPU720_JPEG_MODE_YUV444 5
+#define VDPU720_PIX_DEPTH GENMASK(6, 4)
+#define VDPU720_PIX_DEPTH_8 0
+#define VDPU720_PIX_DEPTH_12 1
+/* qtables_sel: number of Q-table sets (0..3) */
+#define VDPU720_QTBL_SEL GENMASK(9, 8)
+/* htables_sel: number of H-table sets (0..3) */
+#define VDPU720_HTBL_SEL GENMASK(13, 12)
+/* dri_e: restart interval enable */
+#define VDPU720_DRI_E BIT(15)
+/* dri_mcu_num_m1: restart interval MCU count minus 1 */
+#define VDPU720_DRI_MCU_M1 GENMASK(31, 16)
+
+#define VDPU720_REG_HOR_STRIDE 0x014
+#define VDPU720_Y_HOR_STRIDE GENMASK(15, 0)
+#define VDPU720_UV_HOR_STRIDE GENMASK(31, 16)
+
+#define VDPU720_REG_Y_VSTRIDE 0x018
+#define VDPU720_Y_VSTRIDE GENMASK(31, 4)
+
+#define VDPU720_REG_TBL_LEN 0x01c
+#define VDPU720_QTBL_LEN GENMASK(4, 0)
+#define VDPU720_HTBL_MINCODE_LEN GENMASK(12, 8)
+#define VDPU720_HTBL_VALUE_LEN GENMASK(21, 16)
+/* bit 16 of the Y horizontal stride, low 16 bits live in REG5 */
+#define VDPU720_Y_HOR_STRIDE_H BIT(24)
+
+#define VDPU720_REG_STRM_LEN 0x020
+#define VDPU720_STRM_START_BYTE GENMASK(3, 0)
+#define VDPU720_STRM_LEN GENMASK(31, 4)
+
+#define VDPU720_REG_QTBL_BASE 0x024 /* Q-table side buffer, 64-byte aligned */
+#define VDPU720_REG_HTBL_MINCODE 0x028 /* H-mincode table, 64-byte aligned */
+#define VDPU720_REG_HTBL_VALUE 0x02c /* H-value table, 64-byte aligned */
+#define VDPU720_REG_STRM_BASE 0x030 /* JPEG entropy stream, 16-byte aligned */
+#define VDPU720_REG_OUT_BASE 0x034 /* NV12 output buffer, 64-byte aligned */
+
+#define VDPU720_REG_STRM_ERR 0x038
+#define VDPU720_ERROR_PRC_MODE BIT(0)
+#define VDPU720_STRM_FFFF_ERR_MODE GENMASK(6, 5)
+#define VDPU720_STRM_OTHER_MODE GENMASK(8, 7)
+/* Recommended default: accept errors, skip 0xFFFF, skip unknown markers */
+#define VDPU720_STRM_ERR_DFLT (VDPU720_ERROR_PRC_MODE | \
+ FIELD_PREP_CONST(VDPU720_STRM_FFFF_ERR_MODE, 2) | \
+ FIELD_PREP_CONST(VDPU720_STRM_OTHER_MODE, 2))
+
+#define VDPU720_REG_CLK_GATE 0x040
+#define VDPU720_CLK_GATE_ALL 0xff
+
+#define VDPU720_REG_PERF_CTRL 0x078
+#define VDPU720_PERF_WORK_E BIT(0)
+#define VDPU720_PERF_CLR_E BIT(1)
+#define VDPU720_PERF_CNT_TYPE BIT(3)
+#define VDPU720_PERF_RD_LAT_ID GENMASK(7, 4)
+
+#define VDPU720_REG_AXI_CFG 0x07c
+#define VDPU720_ADDR_ALIGN_TYPE GENMASK(1, 0)
+#define VDPU720_AR_CNT_ID_TYPE BIT(2) /* 1 = count sw_ar_count_id only */
+#define VDPU720_AW_CNT_ID_TYPE BIT(3) /* 1 = count sw_aw_count_id only */
+#define VDPU720_AR_COUNT_ID GENMASK(7, 4)
+#define VDPU720_AW_COUNT_ID GENMASK(11, 8)
+#define VDPU720_RD_TOTAL_BYTES_MODE BIT(12) /* 1 = count sw_ar_count_id bytes only */
+
+#define VDPU720_REG_DBG_MCU_POS 0x080
+#define VDPU720_DBG_MCU_POS_X GENMASK(15, 0) /* column in MCU units */
+#define VDPU720_DBG_MCU_POS_Y GENMASK(31, 16) /* row in MCU units */
+
+#define VDPU720_REG_DBG_ERROR 0x084
+#define VDPU720_DERR_DRI_SEQ BIT(0) /* DRI not at expected sequence */
+#define VDPU720_DERR_STREAM_R0 BIT(1) /* special marker 0 detected */
+#define VDPU720_DERR_STREAM_R1 BIT(2) /* special marker 1 detected */
+#define VDPU720_DERR_STREAM_FFFF BIT(3) /* 0xFFFF sequence in stream */
+#define VDPU720_DERR_OTHER_MARK BIT(4) /* unknown JPEG marker */
+#define VDPU720_DERR_MCU_CNT_L BIT(8) /* restart mark arrived too early */
+#define VDPU720_DERR_MCU_CNT_M BIT(9) /* restart mark arrived too late */
+#define VDPU720_DERR_EOI_NO_END BIT(10) /* EOI before frame complete */
+#define VDPU720_DERR_END_NO_EOI BIT(11) /* frame complete without EOI */
+#define VDPU720_DERR_OVERFLOW BIT(12) /* Huffman coefficient overflow */
+#define VDPU720_DERR_HUFF_EMPTY BIT(13) /* bitstream empty before EOI */
+#define VDPU720_DERR_FLAGS GENMASK(13, 0) /* all of the above */
+#define VDPU720_DERR_FIRST_IDX GENMASK(19, 16) /* index of first error */
+
+#define VDPU720_REG_PERF_RD_MAX_LAT 0x088 /* peak read latency (clock cycles) */
+#define VDPU720_REG_PERF_RD_LAT_SAMP 0x08c /* read transactions above lat threshold */
+#define VDPU720_REG_PERF_RD_LAT_ACC 0x090 /* accumulated read latency sum */
+#define VDPU720_REG_PERF_RD_BYTES 0x094 /* total AXI read bytes this frame */
+#define VDPU720_REG_PERF_WR_BYTES 0x098 /* total AXI write bytes this frame */
+
+#define VDPU720_REG_PERF_CYCLES 0x09c
+
+/*
+ * Side-buffer layout for Q-tables and Huffman tables
+ *
+ * The VPU720 JPEG decoder reads quantisation tables and Huffman
+ * tables from a contiguous DMA buffer with the following layout:
+ *
+ * [0, QTBL_SIZE): Q-table data (u16, raster-scan order)
+ * [HMINCODE_OFF, +HMIN_SZ): Huffman mincode table
+ * [HVALUE_OFF, +HVAL_SZ): Huffman value table
+ *
+ * The Q-tables are per component, one entry each
+ */
+#define VDPU720_NB_COMPONENTS 3
+#define VDPU720_QTBL_ENTRIES 64 /* 64 coefficients per table */
+#define VDPU720_QTBL_COMP_SIZE (VDPU720_QTBL_ENTRIES * sizeof(u16))
+#define VDPU720_QTBL_SIZE (VDPU720_QTBL_COMP_SIZE * VDPU720_NB_COMPONENTS)
+
+/*
+ * The Huffman tables are not per component. The hardware holds two sets and
+ * has no per component selector register, so the mapping is fixed: the first
+ * set is used for the luma component and the second one for both chroma
+ * components. A grayscale frame only needs the first.
+ */
+#define VDPU720_NB_HTBL_SETS 2
+
+/*
+ * Per-set mincode layout: 16 DC mincodes + 8 DC accaddr pairs +
+ * 16 AC mincodes + 8 AC accaddr pairs = 48 u16 = 96 bytes
+ */
+#define VDPU720_HMINCODE_SET_SIZE (48 * sizeof(u16))
+#define VDPU720_HMINCODE_SIZE (VDPU720_HMINCODE_SET_SIZE * VDPU720_NB_HTBL_SETS)
+#define VDPU720_HMINCODE_OFF VDPU720_QTBL_SIZE
+
+/* Per-set value layout: 16 DC values + 176 AC values = 192 bytes */
+#define VDPU720_HVALUE_SET_SIZE 192
+#define VDPU720_HVALUE_SIZE (VDPU720_HVALUE_SET_SIZE * VDPU720_NB_HTBL_SETS)
+#define VDPU720_HVALUE_OFF (VDPU720_HMINCODE_OFF + \
+ ALIGN(VDPU720_HMINCODE_SIZE, 64))
+
+#define VDPU720_TABLE_BUF_SIZE (VDPU720_HVALUE_OFF + VDPU720_HVALUE_SIZE)
+
+/*
+ * The three table length registers count 16 byte units, minus one. Derive
+ * them from the sizes above so that what is programmed always matches what
+ * the driver writes into the side buffer.
+ */
+#define VDPU720_TBL_LEN_UNIT 16
+#define VDPU720_TBL_LEN(bytes) ((bytes) / VDPU720_TBL_LEN_UNIT - 1)
+
+/*
+ * Huffman value sub-layout per set (192 bytes total):
+ * bytes [0..15]: DC code values (up to 12 valid entries)
+ * bytes [16..191]: AC code values (up to 162 valid entries)
+ */
+#define VDPU720_DC_VALUES_MAX 16
+#define VDPU720_AC_VALUES_MAX 176 /* 12*16 - 16 */
+
+#endif /* RKJPEGD_VDPU720_REGS_H_ */
diff --git a/drivers/media/platform/rockchip/rkjpegd/rkjpegd.c b/drivers/media/platform/rockchip/rkjpegd/rkjpegd.c
new file mode 100644
index 0000000000000..377964eb74a64
--- /dev/null
+++ b/drivers/media/platform/rockchip/rkjpegd/rkjpegd.c
@@ -0,0 +1,2328 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Rockchip JPEG decoder driver
+ *
+ * The register programming is ported from the Rockchip MPP HAL
+ * (hal_jpegd_rkv.c / hal_jpegd_vpu7xx_com.c) and the downstream
+ * mpp_jpgdec.c kernel driver.
+ *
+ * Copyright (C) 2020 Rockchip Electronics Co., Ltd.
+ * Copyright (C) 2026 WolfVision GmbH
+ * Author: Lucas Sinn <lucas.sinn at wolfvision.net>
+ * Copyright (C) 2026 Pengutronix, Sascha Hauer <s.hauer at pengutronix.de>
+ */
+
+#include <linux/align.h>
+#include <linux/bitfield.h>
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/log2.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/reset.h>
+#include <linux/slab.h>
+#include <linux/videodev2.h>
+#include <linux/workqueue.h>
+
+#include <media/media-device.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-fh.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-jpeg.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-core.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "rkjpegd-vdpu720-regs.h"
+
+#define RKJPEGD_NAME "rockchip-jpegd"
+
+/*
+ * The reference manual gives 48x48 to 65536x65536, but a 32 bit sizeimage
+ * wraps well before the top of that range. Cap at four times 4K in each
+ * direction and hold both the coded format and the bitstream to it.
+ */
+#define RKJPEGD_MIN_WIDTH 48
+#define RKJPEGD_MIN_HEIGHT 48
+#define RKJPEGD_MAX_SIZE 16384
+
+/* The coded queue steps in MCU-sized units, the raw one in macroblocks. */
+#define RKJPEGD_CODED_STEP 8
+#define RKJPEGD_RAW_STEP 16
+
+/* Bytes per pixel used to size a coded buffer for a given resolution. */
+#define RKJPEGD_CODED_MAX_DEPTH 2
+
+/* Milliseconds a frame may take before the watchdog resets the block. */
+#define RKJPEGD_TIMEOUT_MS 2000
+
+static const char * const rkjpegd_clk_names[] = {
+ "axi", "ahb",
+};
+
+#define RKJPEGD_NUM_CLOCKS ARRAY_SIZE(rkjpegd_clk_names)
+
+/**
+ * struct rkjpegd_aux_buf - auxiliary DMA buffer for hardware tables
+ *
+ * @cpu: CPU pointer to the buffer.
+ * @dma: DMA address of the buffer.
+ * @size: Size of the buffer in bytes.
+ */
+struct rkjpegd_aux_buf {
+ void *cpu;
+ dma_addr_t dma;
+ size_t size;
+};
+
+/**
+ * struct rkjpegd_src_buf - a coded buffer and the header parsed out of it
+ *
+ * @base: videobuf2 mem2mem buffer, must be first.
+ * @header: header as returned by v4l2_jpeg_parse_header().
+ * @scan: storage @header.scan points at.
+ * @quantization_tables: storage @header.quantization_tables points at.
+ * @huffman_tables: storage @header.huffman_tables points at.
+ * @parsed: @header describes a frame this hardware can decode.
+ *
+ * The references in @header point into the buffer payload, which stays
+ * mapped for as long as the buffer is queued, so the header parsed when the
+ * buffer arrives is still usable when the job runs.
+ */
+struct rkjpegd_src_buf {
+ struct v4l2_m2m_buffer base;
+ struct v4l2_jpeg_header header;
+ struct v4l2_jpeg_scan_header scan;
+ struct v4l2_jpeg_reference quantization_tables[4];
+ struct v4l2_jpeg_reference huffman_tables[4];
+ bool parsed;
+};
+
+static inline struct rkjpegd_src_buf *
+vb2_to_rkjpegd_src_buf(struct vb2_buffer *vb)
+{
+ return container_of(to_vb2_v4l2_buffer(vb), struct rkjpegd_src_buf,
+ base.vb);
+}
+
+/**
+ * struct rkjpegd_dev - the decoder device
+ *
+ * @v4l2_dev: V4L2 device.
+ * @mdev: media device.
+ * @vdev: video device.
+ * @m2m_dev: mem2mem device.
+ * @dev: driver model device.
+ * @clocks: clocks named by @rkjpegd_clk_names.
+ * @resets: the block's reset lines, as one array control.
+ * @regs: register window.
+ * @irq: the block's interrupt, masked while the watchdog has
+ * the hardware to itself.
+ * @vdev_lock: serialises ioctls and the videobuf2 queues.
+ * @watchdog_work: fires when a job does not complete in time.
+ */
+struct rkjpegd_dev {
+ struct v4l2_device v4l2_dev;
+ struct media_device mdev;
+ struct video_device vdev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct device *dev;
+ struct clk_bulk_data clocks[RKJPEGD_NUM_CLOCKS];
+ struct reset_control *resets;
+ void __iomem *regs;
+ int irq;
+ struct mutex vdev_lock; /* serialises ioctls */
+ struct delayed_work watchdog_work;
+};
+
+/**
+ * struct rkjpegd_ctx - one open file handle
+ *
+ * @fh: V4L2 file handle.
+ * @dev: device this context belongs to.
+ * @src_fmt: coded format on the output queue.
+ * @dst_fmt: raw format on the capture queue.
+ * @crop: visible part of a capture buffer. The decoder
+ * writes whole MCUs, so a frame whose width or
+ * height is not a multiple of one is padded,
+ * see vdpu720_capture_width().
+ * @sequence_cap: capture buffer sequence counter.
+ * @sequence_out: output buffer sequence counter.
+ * @source_change: a resolution change was reported and the
+ * capture queue has not been set up again yet.
+ * @initial_source_change: the first parsed header must report a change
+ * even when it matches the negotiated format.
+ * @table_base: quantisation and Huffman table side buffer,
+ * rebuilt from the frame header on every run.
+ *
+ * @source_change and @initial_source_change, and the @dst_fmt and @crop they
+ * renegotiate, are written from ioctl and videobuf2 callbacks, which hold
+ * @rkjpegd_dev.vdev_lock, and from rkjpegd_device_run(), which does not: the
+ * mem2mem core runs it from its job workqueue as well as from a queue
+ * callback. Nothing serialises the two, but @dst_fmt and @crop are complete
+ * before the source change event is queued, and an application is told to
+ * query them only once it has dequeued that event, so one that follows the
+ * decoder interface cannot observe them half updated.
+ */
+struct rkjpegd_ctx {
+ struct v4l2_fh fh;
+ struct rkjpegd_dev *dev;
+ struct v4l2_pix_format_mplane src_fmt;
+ struct v4l2_pix_format_mplane dst_fmt;
+ struct v4l2_rect crop;
+ u32 sequence_cap;
+ u32 sequence_out;
+ bool source_change;
+ bool initial_source_change;
+ struct rkjpegd_aux_buf table_base;
+};
+
+static inline struct rkjpegd_ctx *file_to_rkjpegd_ctx(struct file *filp)
+{
+ return container_of(file_to_v4l2_fh(filp), struct rkjpegd_ctx, fh);
+}
+
+static inline void rkjpegd_write_relaxed(struct rkjpegd_dev *jpegd,
+ u32 val, u32 reg)
+{
+ writel_relaxed(val, jpegd->regs + reg);
+}
+
+static inline void rkjpegd_write(struct rkjpegd_dev *jpegd, u32 val, u32 reg)
+{
+ writel(val, jpegd->regs + reg);
+}
+
+static inline u32 rkjpegd_read(struct rkjpegd_dev *jpegd, u32 reg)
+{
+ return readl(jpegd->regs + reg);
+}
+
+static inline void rkjpegd_write_addr(struct rkjpegd_dev *jpegd, u32 reg,
+ dma_addr_t addr)
+{
+ rkjpegd_write(jpegd, lower_32_bits(addr), reg);
+}
+
+static const struct v4l2_event rkjpegd_eos_event = {
+ .type = V4L2_EVENT_EOS,
+};
+
+static const struct v4l2_event rkjpegd_src_change_event = {
+ .type = V4L2_EVENT_SOURCE_CHANGE,
+ .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
+};
+
+/*
+ * Colorimetry is a property of the picture, and a JPEG frame header carries
+ * nothing that describes it. Seed both queues with what JFIF implies and
+ * from then on propagate whatever userspace sets on one queue to the other,
+ * rather than overriding it: an application that knows the picture is BT.709
+ * has no other way to say so, and one that has its format overridden takes it
+ * as the decoder refusing the stream.
+ */
+static void rkjpegd_set_default_colorimetry(struct v4l2_pix_format_mplane *pix_mp)
+{
+ pix_mp->colorspace = V4L2_COLORSPACE_JPEG;
+ pix_mp->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ pix_mp->quantization = V4L2_QUANTIZATION_DEFAULT;
+ pix_mp->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static void rkjpegd_propagate_colorimetry(struct v4l2_pix_format_mplane *to,
+ const struct v4l2_pix_format_mplane *from)
+{
+ to->colorspace = from->colorspace;
+ to->ycbcr_enc = from->ycbcr_enc;
+ to->quantization = from->quantization;
+ to->xfer_func = from->xfer_func;
+}
+
+static void rkjpegd_fill_raw_fmt(struct v4l2_pix_format_mplane *pix_mp,
+ u32 width, u32 height)
+{
+ pix_mp->pixelformat = V4L2_PIX_FMT_NV12;
+ pix_mp->width = width;
+ pix_mp->height = height;
+ pix_mp->field = V4L2_FIELD_NONE;
+ pix_mp->num_planes = 1;
+ pix_mp->plane_fmt[0].bytesperline = width;
+ pix_mp->plane_fmt[0].sizeimage = width * height * 3 / 2;
+ memset(pix_mp->plane_fmt[0].reserved, 0,
+ sizeof(pix_mp->plane_fmt[0].reserved));
+ memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
+}
+
+static void rkjpegd_fill_coded_fmt(struct v4l2_pix_format_mplane *pix_mp,
+ u32 width, u32 height, u32 sizeimage)
+{
+ pix_mp->pixelformat = V4L2_PIX_FMT_JPEG;
+ pix_mp->width = width;
+ pix_mp->height = height;
+ pix_mp->field = V4L2_FIELD_NONE;
+ pix_mp->num_planes = 1;
+ pix_mp->plane_fmt[0].bytesperline = 0;
+
+ /*
+ * A compressed frame has no size the driver could derive, so let
+ * userspace ask for one and only fall back to an upper bound when it
+ * does not. A frame that then does not fit is caught when its header
+ * is parsed, where the missing end of image marker gives it away.
+ */
+ if (!sizeimage)
+ sizeimage = width * height * RKJPEGD_CODED_MAX_DEPTH;
+ pix_mp->plane_fmt[0].sizeimage = sizeimage;
+
+ memset(pix_mp->plane_fmt[0].reserved, 0,
+ sizeof(pix_mp->plane_fmt[0].reserved));
+ memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
+}
+
+static void rkjpegd_reset_fmts(struct rkjpegd_ctx *ctx)
+{
+ u32 width = ALIGN(RKJPEGD_MIN_WIDTH, RKJPEGD_RAW_STEP);
+ u32 height = ALIGN(RKJPEGD_MIN_HEIGHT, RKJPEGD_RAW_STEP);
+
+ rkjpegd_fill_coded_fmt(&ctx->src_fmt, width, height, 0);
+ rkjpegd_fill_raw_fmt(&ctx->dst_fmt, width, height);
+ rkjpegd_set_default_colorimetry(&ctx->src_fmt);
+ rkjpegd_set_default_colorimetry(&ctx->dst_fmt);
+ ctx->crop.left = 0;
+ ctx->crop.top = 0;
+ ctx->crop.width = width;
+ ctx->crop.height = height;
+}
+
+static int rkjpegd_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, RKJPEGD_NAME, sizeof(cap->driver));
+ strscpy(cap->card, RKJPEGD_NAME, sizeof(cap->card));
+
+ return 0;
+}
+
+static int rkjpegd_enum_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index)
+ return -EINVAL;
+
+ f->pixelformat = V4L2_PIX_FMT_NV12;
+
+ return 0;
+}
+
+static int rkjpegd_enum_fmt_vid_out(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ if (f->index)
+ return -EINVAL;
+
+ f->pixelformat = V4L2_PIX_FMT_JPEG;
+
+ /*
+ * The frame dimensions come out of the bitstream rather than out of
+ * the coded format, which is what V4L2_EVENT_SOURCE_CHANGE reports.
+ * Userspace has no other way to find out that the event is worth
+ * waiting for, and one that does not wait sets the capture queue up
+ * from whatever the coded format happened to say.
+ */
+ f->flags |= V4L2_FMT_FLAG_DYN_RESOLUTION;
+
+ return 0;
+}
+
+static int rkjpegd_enum_framesizes(struct file *file, void *priv,
+ struct v4l2_frmsizeenum *fsize)
+{
+ if (fsize->index)
+ return -EINVAL;
+
+ /*
+ * Only the coded format has a size userspace gets to pick. The
+ * decoded one follows the bitstream, so there is nothing to enumerate
+ * for it, and saying otherwise is actively harmful: a step here
+ * describes the sizes that may be asked for, and userspace turns it
+ * into a set the visible rectangle then has to be a member of. The
+ * capture buffer is padded to a macroblock, the picture in it is not,
+ * so a 1080 line frame would fall outside a set built on a step of 16
+ * and could not be negotiated at all.
+ */
+ if (fsize->pixel_format != V4L2_PIX_FMT_JPEG)
+ return -ENOTTY;
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise.min_width = RKJPEGD_MIN_WIDTH;
+ fsize->stepwise.max_width = RKJPEGD_MAX_SIZE;
+ fsize->stepwise.step_width = RKJPEGD_CODED_STEP;
+ fsize->stepwise.min_height = RKJPEGD_MIN_HEIGHT;
+ fsize->stepwise.max_height = RKJPEGD_MAX_SIZE;
+ fsize->stepwise.step_height = RKJPEGD_CODED_STEP;
+
+ return 0;
+}
+
+static int rkjpegd_g_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ f->fmt.pix_mp = file_to_rkjpegd_ctx(file)->dst_fmt;
+
+ return 0;
+}
+
+static int rkjpegd_g_fmt_vid_out(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ f->fmt.pix_mp = file_to_rkjpegd_ctx(file)->src_fmt;
+
+ return 0;
+}
+
+/*
+ * The capture resolution follows the bitstream, not userspace: it is set from
+ * the frame header when a source change is reported and only read back here.
+ */
+static int rkjpegd_try_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkjpegd_ctx *ctx = file_to_rkjpegd_ctx(file);
+
+ rkjpegd_fill_raw_fmt(&f->fmt.pix_mp, ctx->dst_fmt.width,
+ ctx->dst_fmt.height);
+
+ return 0;
+}
+
+static int rkjpegd_try_fmt_vid_out(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
+ u32 sizeimage = pix_mp->num_planes == 1 ?
+ pix_mp->plane_fmt[0].sizeimage : 0;
+
+ v4l_bound_align_image(&pix_mp->width,
+ RKJPEGD_MIN_WIDTH, RKJPEGD_MAX_SIZE,
+ ilog2(RKJPEGD_CODED_STEP),
+ &pix_mp->height,
+ RKJPEGD_MIN_HEIGHT, RKJPEGD_MAX_SIZE,
+ ilog2(RKJPEGD_CODED_STEP), 0);
+
+ rkjpegd_fill_coded_fmt(pix_mp, pix_mp->width, pix_mp->height, sizeimage);
+
+ return 0;
+}
+
+static int rkjpegd_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkjpegd_ctx *ctx = file_to_rkjpegd_ctx(file);
+ struct vb2_queue *vq = v4l2_m2m_get_dst_vq(ctx->fh.m2m_ctx);
+
+ if (vb2_is_busy(vq))
+ return -EBUSY;
+
+ rkjpegd_try_fmt_vid_cap(file, priv, f);
+ ctx->dst_fmt = f->fmt.pix_mp;
+ rkjpegd_propagate_colorimetry(&ctx->src_fmt, &ctx->dst_fmt);
+
+ return 0;
+}
+
+static int rkjpegd_s_fmt_vid_out(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct rkjpegd_ctx *ctx = file_to_rkjpegd_ctx(file);
+ struct vb2_queue *vq = v4l2_m2m_get_src_vq(ctx->fh.m2m_ctx);
+ int ret;
+
+ if (vb2_is_busy(vq))
+ return -EBUSY;
+
+ ret = rkjpegd_try_fmt_vid_out(file, priv, f);
+ if (ret)
+ return ret;
+
+ ctx->src_fmt = f->fmt.pix_mp;
+
+ /*
+ * Picking a coded format invalidates whatever was negotiated on the
+ * capture queue. Seed it from the coded resolution so a userspace
+ * that already knows the frame size can allocate without waiting for
+ * the first source change.
+ */
+ rkjpegd_fill_raw_fmt(&ctx->dst_fmt,
+ ALIGN(ctx->src_fmt.width, RKJPEGD_RAW_STEP),
+ ALIGN(ctx->src_fmt.height, RKJPEGD_RAW_STEP));
+ rkjpegd_propagate_colorimetry(&ctx->dst_fmt, &ctx->src_fmt);
+ ctx->crop.left = 0;
+ ctx->crop.top = 0;
+ ctx->crop.width = ctx->src_fmt.width;
+ ctx->crop.height = ctx->src_fmt.height;
+
+ return 0;
+}
+
+static int rkjpegd_g_selection(struct file *file, void *priv,
+ struct v4l2_selection *s)
+{
+ struct rkjpegd_ctx *ctx = file_to_rkjpegd_ctx(file);
+
+ if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE &&
+ s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ return -EINVAL;
+
+ switch (s->target) {
+ case V4L2_SEL_TGT_COMPOSE:
+ case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+ s->r = ctx->crop;
+ break;
+ case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+ case V4L2_SEL_TGT_COMPOSE_PADDED:
+ s->r.left = 0;
+ s->r.top = 0;
+ s->r.width = ctx->dst_fmt.width;
+ s->r.height = ctx->dst_fmt.height;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int rkjpegd_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub)
+{
+ switch (sub->type) {
+ case V4L2_EVENT_EOS:
+ return v4l2_event_subscribe(fh, sub, 0, NULL);
+ case V4L2_EVENT_SOURCE_CHANGE:
+ return v4l2_src_change_event_subscribe(fh, sub);
+ default:
+ /* The decoder takes no controls, so there is nothing else. */
+ return -EINVAL;
+ }
+}
+
+static void rkjpegd_set_last_buffer(struct rkjpegd_ctx *ctx)
+{
+ struct vb2_v4l2_buffer *next_dst_buf;
+
+ next_dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+ if (!next_dst_buf) {
+ ctx->fh.m2m_ctx->is_draining = true;
+ ctx->fh.m2m_ctx->next_buf_last = true;
+ return;
+ }
+
+ v4l2_m2m_last_buffer_done(ctx->fh.m2m_ctx, next_dst_buf);
+}
+
+static int rkjpegd_decoder_cmd(struct file *file, void *priv,
+ struct v4l2_decoder_cmd *cmd)
+{
+ struct rkjpegd_ctx *ctx = file_to_rkjpegd_ctx(file);
+ int ret;
+
+ ret = v4l2_m2m_ioctl_try_decoder_cmd(file, priv, cmd);
+ if (ret < 0)
+ return ret;
+
+ if (!vb2_is_streaming(v4l2_m2m_get_src_vq(ctx->fh.m2m_ctx)))
+ return 0;
+
+ ret = v4l2_m2m_ioctl_decoder_cmd(file, priv, cmd);
+ if (ret < 0)
+ return ret;
+
+ if (cmd->cmd == V4L2_DEC_CMD_STOP &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
+ v4l2_event_queue_fh(&ctx->fh, &rkjpegd_eos_event);
+
+ if (cmd->cmd == V4L2_DEC_CMD_START &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
+ vb2_clear_last_buffer_dequeued(&ctx->fh.m2m_ctx->cap_q_ctx.q);
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops rkjpegd_ioctl_ops = {
+ .vidioc_querycap = rkjpegd_querycap,
+ .vidioc_enum_framesizes = rkjpegd_enum_framesizes,
+
+ .vidioc_enum_fmt_vid_cap = rkjpegd_enum_fmt_vid_cap,
+ .vidioc_g_fmt_vid_cap_mplane = rkjpegd_g_fmt_vid_cap,
+ .vidioc_try_fmt_vid_cap_mplane = rkjpegd_try_fmt_vid_cap,
+ .vidioc_s_fmt_vid_cap_mplane = rkjpegd_s_fmt_vid_cap,
+
+ .vidioc_enum_fmt_vid_out = rkjpegd_enum_fmt_vid_out,
+ .vidioc_g_fmt_vid_out_mplane = rkjpegd_g_fmt_vid_out,
+ .vidioc_try_fmt_vid_out_mplane = rkjpegd_try_fmt_vid_out,
+ .vidioc_s_fmt_vid_out_mplane = rkjpegd_s_fmt_vid_out,
+
+ .vidioc_g_selection = rkjpegd_g_selection,
+
+ .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+ .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+ .vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+ .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+ .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+ .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+ .vidioc_remove_bufs = v4l2_m2m_ioctl_remove_bufs,
+
+ .vidioc_streamon = v4l2_m2m_ioctl_streamon,
+ .vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
+
+ .vidioc_try_decoder_cmd = v4l2_m2m_ioctl_try_decoder_cmd,
+ .vidioc_decoder_cmd = rkjpegd_decoder_cmd,
+
+ .vidioc_subscribe_event = rkjpegd_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static void rkjpegd_arm_watchdog(struct rkjpegd_dev *jpegd)
+{
+ schedule_delayed_work(&jpegd->watchdog_work,
+ msecs_to_jiffies(RKJPEGD_TIMEOUT_MS));
+}
+
+static void rkjpegd_job_finish_no_pm(struct rkjpegd_ctx *ctx,
+ enum vb2_buffer_state state)
+{
+ struct vb2_v4l2_buffer *src, *dst;
+
+ src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+ dst = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+ if (WARN_ON(!src) || WARN_ON(!dst))
+ return;
+
+ src->sequence = ctx->sequence_out++;
+ dst->sequence = ctx->sequence_cap++;
+
+ vb2_set_plane_payload(&dst->vb2_buf, 0,
+ state == VB2_BUF_STATE_DONE ?
+ ctx->dst_fmt.plane_fmt[0].sizeimage : 0);
+
+ if (v4l2_m2m_is_last_draining_src_buf(ctx->fh.m2m_ctx, src)) {
+ dst->flags |= V4L2_BUF_FLAG_LAST;
+ v4l2_event_queue_fh(&ctx->fh, &rkjpegd_eos_event);
+ v4l2_m2m_mark_stopped(ctx->fh.m2m_ctx);
+ }
+
+ v4l2_m2m_buf_done_and_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx,
+ state);
+}
+
+static void rkjpegd_job_finish(struct rkjpegd_ctx *ctx,
+ enum vb2_buffer_state state)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+
+ clk_bulk_disable(RKJPEGD_NUM_CLOCKS, jpegd->clocks);
+ pm_runtime_put_autosuspend(jpegd->dev);
+
+ rkjpegd_job_finish_no_pm(ctx, state);
+}
+
+static void rkjpegd_irq_done(struct rkjpegd_dev *jpegd,
+ enum vb2_buffer_state state)
+{
+ struct rkjpegd_ctx *ctx = v4l2_m2m_get_curr_priv(jpegd->m2m_dev);
+
+ if (!ctx)
+ return;
+
+ if (cancel_delayed_work(&jpegd->watchdog_work))
+ rkjpegd_job_finish(ctx, state);
+}
+
+/**
+ * vdpu720_jpeg_mode() - map JPEG sampling factors to the hardware mode
+ * @frame: parsed JPEG frame header
+ *
+ * The sampling factor tuple is determined by the luma channel's factors
+ * relative to the maximum in the frame. Standard JFIF layouts only,
+ * anything else returns -EINVAL: the mode also picks the MCU height and
+ * therefore PIC_H, so guessing one would decode into a wrong image.
+ *
+ * Return: a VDPU720_JPEG_MODE_* value, or -EINVAL for a sampling layout
+ * the hardware cannot be told about.
+ */
+static int vdpu720_jpeg_mode(const struct v4l2_jpeg_frame_header *frame)
+{
+ u8 h0, v0;
+
+ if (frame->num_components == 1)
+ return VDPU720_JPEG_MODE_YUV400;
+
+ /* Component 0 always carries luma in JFIF */
+ h0 = frame->component[0].horizontal_sampling_factor;
+ v0 = frame->component[0].vertical_sampling_factor;
+
+ if (h0 == 1 && v0 == 1)
+ return VDPU720_JPEG_MODE_YUV444;
+ if (h0 == 2 && v0 == 1)
+ return VDPU720_JPEG_MODE_YUV422;
+ if (h0 == 2 && v0 == 2)
+ return VDPU720_JPEG_MODE_YUV420;
+ if (h0 == 4 && v0 == 1)
+ return VDPU720_JPEG_MODE_YUV411;
+ if (h0 == 1 && v0 == 2)
+ return VDPU720_JPEG_MODE_YUV440;
+
+ return -EINVAL;
+}
+
+/**
+ * vdpu720_mcu_width() - horizontal size of a mode's minimum coded unit
+ * @jpeg_mode: a VDPU720_JPEG_MODE_* value
+ *
+ * MCU_W follows the luma horizontal sampling factor, h0 * 8. YUV420 and
+ * YUV422 subsample the luma horizontally by 2 and YUV411 by 4, giving 16 and
+ * 32 pixels. The rest is 8.
+ *
+ * Return: the MCU width in pixels.
+ */
+static u32 vdpu720_mcu_width(int jpeg_mode)
+{
+ if (jpeg_mode == VDPU720_JPEG_MODE_YUV411)
+ return 32;
+ if (jpeg_mode == VDPU720_JPEG_MODE_YUV420 ||
+ jpeg_mode == VDPU720_JPEG_MODE_YUV422)
+ return 16;
+
+ return 8;
+}
+
+/**
+ * vdpu720_mcu_height() - vertical size of a mode's minimum coded unit
+ * @jpeg_mode: a VDPU720_JPEG_MODE_* value
+ *
+ * MCU_H follows the luma vertical sampling factor, v0 * 8. YUV420 and YUV440
+ * subsample the luma vertically by 2, giving 16 pixels. The rest is 8.
+ *
+ * Return: the MCU height in pixels.
+ */
+static u32 vdpu720_mcu_height(int jpeg_mode)
+{
+ if (jpeg_mode == VDPU720_JPEG_MODE_YUV420 ||
+ jpeg_mode == VDPU720_JPEG_MODE_YUV440)
+ return 16;
+
+ return 8;
+}
+
+/**
+ * vdpu720_capture_width() - capture buffer width for a picture width
+ * @hdr: parsed JPEG header, or NULL for a frame that was not parsed
+ * @width: picture width in pixels
+ *
+ * A capture buffer is padded to a macroblock, but the decoder writes whole
+ * MCUs and a YUV411 MCU is 32 pixels wide, past the macroblock every other
+ * mode stops at. Pad to whichever of the two is wider so that the last MCU
+ * column always has a row to land in. 4:1:1 at 720 pixels is the ordinary
+ * case rather than a corner one: 720 is a multiple of 16 but not of 32, so a
+ * macroblock aligned buffer would be a full MCU column short of what the
+ * decoder writes.
+ *
+ * The height needs no equivalent. MCU_H tops out at the 16 pixels a
+ * macroblock is already aligned to.
+ *
+ * A frame the parser refused has no sampling factors to ask for, and no job
+ * runs for it either, so the macroblock is all it needs.
+ *
+ * Return: the buffer width in pixels.
+ */
+static u32 vdpu720_capture_width(const struct v4l2_jpeg_header *hdr, u32 width)
+{
+ int jpeg_mode;
+
+ if (!hdr)
+ return ALIGN(width, RKJPEGD_RAW_STEP);
+
+ jpeg_mode = vdpu720_jpeg_mode(&hdr->frame);
+ if (jpeg_mode < 0)
+ return ALIGN(width, RKJPEGD_RAW_STEP);
+
+ return ALIGN(width, max_t(u32, vdpu720_mcu_width(jpeg_mode),
+ RKJPEGD_RAW_STEP));
+}
+
+/**
+ * vdpu720_nb_htbl_sets() - number of Huffman table sets the hardware reads
+ * @num_components: number of components in the scan
+ *
+ * One for a grayscale frame, two for a colour one. vdpu720_write_htbl()
+ * fills that many sets and vdpu720_fill_regs() sizes HTBL_SEL and the length
+ * registers from the same number, so the two cannot drift apart.
+ *
+ * Return: the number of table sets to write.
+ */
+static unsigned int vdpu720_nb_htbl_sets(unsigned int num_components)
+{
+ return num_components == 1 ? 1 : VDPU720_NB_HTBL_SETS;
+}
+
+/**
+ * vdpu720_write_qtbl() - write all Q-tables into the DMA side buffer
+ * @ctx: context whose side buffer receives the tables
+ * @hdr: parsed JPEG header the tables are taken from
+ *
+ * Tables are stored sequentially, one per component in component order.
+ * Each entry is widened to u16 and reordered from JPEG zig-zag scan to
+ * natural raster-scan order, matching the hardware expectation.
+ *
+ * Return: 0 on success, -EINVAL if the frame refers to a quantization
+ * table it does not carry.
+ */
+static int vdpu720_write_qtbl(struct rkjpegd_ctx *ctx,
+ const struct v4l2_jpeg_header *hdr)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ u16 *base = ctx->table_base.cpu;
+ unsigned int k, i;
+
+ for (k = 0; k < hdr->frame.num_components; k++) {
+ u8 tq_id = hdr->frame.component[k].quantization_table_selector;
+ u8 qtbl[VDPU720_QTBL_ENTRIES];
+ u16 *dst;
+
+ /*
+ * v4l2_jpeg_parse_header() filled quantization_tables[] by
+ * destination selector (packed DQT segments handled); .start
+ * points at the 64 Qk values, already past the Pq|Tq byte.
+ */
+ if (tq_id > 3 || !hdr->quantization_tables[tq_id].start) {
+ dev_err_ratelimited(jpegd->dev,
+ "Q-table %u not found for component %u\n",
+ tq_id, k);
+ return -EINVAL;
+ }
+
+ /*
+ * Bulk-copy the Q-table out of the uncached source buffer once;
+ * the per-element zigzag reads below would otherwise each be a
+ * separate uncached bus transaction.
+ */
+ memcpy(qtbl, hdr->quantization_tables[tq_id].start, sizeof(qtbl));
+ dst = base + k * VDPU720_QTBL_ENTRIES;
+
+ /*
+ * Reorder zigzag -> raster scan.
+ * v4l2_jpeg_zigzag_scan_index[z] = raster position of zigzag
+ * element z. JPEG Q-tables are stored in zigzag order; the
+ * hardware expects them in natural raster (row-major) order.
+ */
+ for (i = 0; i < VDPU720_QTBL_ENTRIES; i++)
+ dst[v4l2_jpeg_zigzag_scan_index[i]] = (u16)qtbl[i];
+ }
+
+ return 0;
+}
+
+/**
+ * vdpu720_compute_mincode() - build the minimum Huffman code arrays
+ * @bits: BITS[16], the number of codes of each length 1..16
+ * @min_code: output, minimum code value per length (16 entries)
+ * @acc_addr: output, accumulated symbol-table address per length (16 entries)
+ *
+ * Derives the two arrays the hardware needs for one Huffman table, DC or AC,
+ * from that table's BITS array. Algorithm ported verbatim from
+ * jpegd_vpu7xx_write_htbl().
+ */
+static void vdpu720_compute_mincode(const u8 *bits, u16 *min_code, u16 *acc_addr)
+{
+ u16 code = 0, addr = 0;
+ unsigned int j;
+
+ for (j = 0; j < 16; j++) {
+ u16 len = bits[j];
+
+ if (len == 0 && j > 0) {
+ if (code > ((u16)(min_code[j - 1]) << 1))
+ min_code[j] = code;
+ else
+ min_code[j] = (u16)(min_code[j - 1]) << 1;
+ } else {
+ min_code[j] = code;
+ }
+
+ code += len;
+ addr += len;
+ acc_addr[j] = addr;
+ code <<= 1;
+ }
+
+ /* Sentinel: set min_code[0] to the last valid code + count */
+ if (bits[15])
+ min_code[0] = min_code[15] + bits[15] - 1;
+ else
+ min_code[0] = min_code[15];
+}
+
+/**
+ * vdpu720_write_htbl() - fill the Huffman mincode and value sub-buffers
+ * @ctx: context whose side buffer receives the tables
+ * @hdr: parsed JPEG header the tables are taken from
+ *
+ * One set is written per vdpu720_nb_htbl_sets(), fed by the scan component
+ * that uses it: the first by the luma component and the second by the first
+ * chroma one. The hardware has no room for a third set and no per component
+ * selector register, so a frame whose two chroma components disagree on their
+ * tables cannot be described to it and is refused rather than decoded with
+ * the wrong table for the last component.
+ *
+ * Per-set layout in the mincode buffer:
+ * 16 x u16 DC min-codes
+ * 8 x u16 DC accumulated addresses (packed pairs)
+ * 16 x u16 AC min-codes
+ * 8 x u16 AC accumulated addresses (packed pairs)
+ *
+ * Per-set layout in the value buffer (192 bytes):
+ * 16 bytes DC code values
+ * 176 bytes AC code values
+ *
+ * Return: 0 on success, -EINVAL if the scan refers to a Huffman table it
+ * does not carry or needs more table sets than the hardware has.
+ */
+static int vdpu720_write_htbl(struct rkjpegd_ctx *ctx,
+ const struct v4l2_jpeg_header *hdr)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ const struct v4l2_jpeg_scan_header *scan = hdr->scan;
+ u8 *tbl_base = ctx->table_base.cpu;
+ u16 *p_mincode = (u16 *)(tbl_base + VDPU720_HMINCODE_OFF);
+ u8 *p_value = tbl_base + VDPU720_HVALUE_OFF;
+ unsigned int nb_sets = vdpu720_nb_htbl_sets(scan->num_components);
+ unsigned int k, i;
+
+ /* The last set is shared by every remaining component */
+ for (k = nb_sets; k < scan->num_components; k++) {
+ if (scan->component[k].dc_entropy_coding_table_selector !=
+ scan->component[nb_sets - 1].dc_entropy_coding_table_selector ||
+ scan->component[k].ac_entropy_coding_table_selector !=
+ scan->component[nb_sets - 1].ac_entropy_coding_table_selector) {
+ dev_err_ratelimited(jpegd->dev,
+ "JPEG component %u uses other Huffman tables than component %u\n",
+ k, nb_sets - 1);
+ return -EINVAL;
+ }
+ }
+
+ for (k = 0; k < nb_sets; k++) {
+ u8 dc_sel = scan->component[k].dc_entropy_coding_table_selector;
+ u8 ac_sel = scan->component[k].ac_entropy_coding_table_selector;
+ u8 dc_bits[16], ac_bits[16];
+ const u8 *dc_src, *ac_src, *dc_vals, *ac_vals;
+ unsigned int dc_huffval_len, ac_huffval_len;
+ u16 min_dc[16], acc_dc[16];
+ u16 min_ac[16], acc_ac[16];
+
+ /*
+ * v4l2_jpeg_parse_header() filled huffman_tables[] indexed by
+ * (Tc << 1) | Th - Tc=0/1 (DC/AC) class, Th=0/1 (luma/chroma)
+ * id (packed DHT segments handled). .start points at BITS[16],
+ * already past the Tc|Th byte.
+ */
+ if (dc_sel > 1 || ac_sel > 1 ||
+ !hdr->huffman_tables[dc_sel].start ||
+ !hdr->huffman_tables[2 | ac_sel].start) {
+ dev_err_ratelimited(jpegd->dev,
+ "H-table not found for component %u (dc=%u ac=%u)\n",
+ k, dc_sel, ac_sel);
+ return -EINVAL;
+ }
+
+ /*
+ * Layout at .start: [BITS 16B] [HUFFVAL sum(BITS)B]. Derive the
+ * HUFFVAL length from BITS (a packed segment's length would
+ * over-count when several tables share it).
+ */
+ dc_src = hdr->huffman_tables[dc_sel].start;
+ ac_src = hdr->huffman_tables[2 | ac_sel].start;
+
+ /*
+ * Bulk-copy the two BITS arrays out of the uncached source; they
+ * are otherwise walked byte-by-byte twice (the length sum here and
+ * again in vdpu720_compute_mincode()). The HUFFVAL blocks stay in
+ * the source and are copied out in one memcpy() further down.
+ */
+ memcpy(dc_bits, dc_src, sizeof(dc_bits));
+ memcpy(ac_bits, ac_src, sizeof(ac_bits));
+ dc_vals = dc_src + 16;
+ ac_vals = ac_src + 16;
+
+ dc_huffval_len = 0;
+ for (i = 0; i < 16; i++)
+ dc_huffval_len += dc_bits[i];
+ ac_huffval_len = 0;
+ for (i = 0; i < 16; i++)
+ ac_huffval_len += ac_bits[i];
+
+ /*
+ * The value block holds VDPU720_DC_VALUES_MAX DC and
+ * VDPU720_AC_VALUES_MAX AC symbols, and the accumulated
+ * addresses below are packed two per u16, so they have to stay
+ * within a byte as well. A table the hardware cannot hold
+ * would otherwise be truncated into it silently and decode to
+ * a wrong image. Both limits are covered by this check, the
+ * value block is the tighter of the two.
+ */
+ if (dc_huffval_len > VDPU720_DC_VALUES_MAX ||
+ ac_huffval_len > VDPU720_AC_VALUES_MAX) {
+ dev_err_ratelimited(jpegd->dev,
+ "JPEG Huffman table too large for component %u (dc=%u ac=%u)\n",
+ k, dc_huffval_len, ac_huffval_len);
+ return -EINVAL;
+ }
+
+ vdpu720_compute_mincode(dc_bits, min_dc, acc_dc);
+ vdpu720_compute_mincode(ac_bits, min_ac, acc_ac);
+
+ for (i = 0; i < 16; i++)
+ *p_mincode++ = min_dc[i];
+ for (i = 0; i < 8; i++)
+ *p_mincode++ = (u16)acc_dc[2 * i] |
+ ((u16)acc_dc[2 * i + 1] << 8);
+ for (i = 0; i < 16; i++)
+ *p_mincode++ = min_ac[i];
+ for (i = 0; i < 8; i++)
+ *p_mincode++ = (u16)acc_ac[2 * i] |
+ ((u16)acc_ac[2 * i + 1] << 8);
+
+ /* Zero-pad the value block, then fill DC then AC values. */
+ memset(p_value, 0, VDPU720_HVALUE_SET_SIZE);
+ memcpy(p_value, dc_vals, dc_huffval_len);
+ memcpy(p_value + VDPU720_DC_VALUES_MAX, ac_vals, ac_huffval_len);
+ p_value += VDPU720_HVALUE_SET_SIZE;
+ }
+
+ return 0;
+}
+
+/**
+ * vdpu720_fill_regs() - program the decoder registers for one frame
+ * @ctx: context the job belongs to
+ * @hdr: parsed JPEG header
+ * @tbl_dma: DMA address of the Q/Huffman table side buffer
+ * @strm_dma: DMA address of the entropy stream, 16-byte aligned
+ * @strm_start_byte: offset of the first stream byte within that word
+ * @strm_len_blks: stream length in 16-byte blocks, minus one
+ * @out_dma: DMA address of the destination NV12 buffer
+ *
+ * Return: 0 on success, negative errno for a frame the hardware cannot be
+ * programmed for.
+ */
+static int vdpu720_fill_regs(struct rkjpegd_ctx *ctx,
+ const struct v4l2_jpeg_header *hdr,
+ dma_addr_t tbl_dma,
+ dma_addr_t strm_dma, u32 strm_start_byte,
+ u32 strm_len_blks,
+ dma_addr_t out_dma)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ u32 jpeg_width = hdr->frame.width;
+ u32 jpeg_height = hdr->frame.height;
+ u32 buf_width = ctx->dst_fmt.width;
+ u32 buf_height = ctx->dst_fmt.height;
+ u32 w_align = ALIGN(buf_width, 16);
+ u32 y_stride = w_align >> 4; /* units of 16 pixels */
+ u32 y_vstride = y_stride * buf_height; /* sets the UV plane offset */
+ u32 nb_comp = hdr->frame.num_components;
+ /*
+ * qtbl_sel = number of Q-table entries written to the side buffer,
+ * one per component. Using the DQT segment count is wrong when a
+ * camera packs all Q-tables into a single segment while there are
+ * three components: the hardware would only read one Q-table, leaving
+ * Cb/Cr with garbage.
+ */
+ u32 qtbl_sel = nb_comp;
+ /*
+ * H-table sets: one for grayscale (luma only), two for colour (luma +
+ * chroma). The three table lengths below are computed from these two
+ * counts and the side buffer layout, so what is programmed is what
+ * vdpu720_write_qtbl() and vdpu720_write_htbl() actually wrote.
+ */
+ u32 htbl_sel = vdpu720_nb_htbl_sets(nb_comp);
+ u32 mcu_width, mcu_height, jpeg_width_aligned, jpeg_height_aligned;
+ u32 qtbl_len, hmin_len, hval_len;
+ int jpeg_mode;
+ u32 reg;
+
+ jpeg_mode = vdpu720_jpeg_mode(&hdr->frame);
+ if (jpeg_mode < 0) {
+ dev_err_ratelimited(jpegd->dev,
+ "unsupported JPEG sampling factors %ux%u\n",
+ hdr->frame.component[0].horizontal_sampling_factor,
+ hdr->frame.component[0].vertical_sampling_factor);
+ return jpeg_mode;
+ }
+
+ mcu_width = vdpu720_mcu_width(jpeg_mode);
+ mcu_height = vdpu720_mcu_height(jpeg_mode);
+ jpeg_width_aligned = ALIGN(jpeg_width, mcu_width);
+ jpeg_height_aligned = ALIGN(jpeg_height, mcu_height);
+
+ /*
+ * The picture dimensions below are taken from the bitstream while the
+ * strides are taken from the negotiated capture format. A frame that
+ * is larger than what was negotiated would make the decoder write
+ * beyond the capture buffer, so refuse it rather than program the
+ * hardware with the two sets of numbers mixed.
+ *
+ * Both are compared MCU aligned, which is the extent the decoder
+ * actually writes: it emits whole MCUs, and an encoder pads the right
+ * and bottom edges out to the same boundary. Comparing the raw width
+ * would let a YUV411 frame through that was negotiated against a
+ * macroblock rather than against its 32 pixel MCU, one MCU column
+ * short - vdpu720_capture_width() is what keeps that from happening
+ * for a capture queue this driver sized itself.
+ */
+ if (jpeg_width_aligned > buf_width || jpeg_height_aligned > buf_height) {
+ dev_err_ratelimited(jpegd->dev,
+ "JPEG %ux%u does not fit the negotiated %ux%u\n",
+ jpeg_width_aligned, jpeg_height_aligned,
+ buf_width, buf_height);
+ return -EINVAL;
+ }
+
+ /*
+ * REG2: system config - always output NV12.
+ *
+ * FILL_DOWN_E completes the bottom of the picture for the vertically
+ * subsampled output chroma. The reference driver sets it for every
+ * NV12 conversion, whatever the picture and buffer heights are.
+ *
+ * FILL_RIGHT_E does the same for the right hand edge, but only an
+ * 8 pixel MCU width can stop short of the buffer: a wider MCU reaches
+ * the edge of a buffer padded to it by vdpu720_capture_width().
+ */
+ reg = FIELD_PREP(VDPU720_YUV_OUT_FMT, VDPU720_YUV_OUT_FMT_NV12) |
+ VDPU720_FILL_DOWN_E;
+ if (ALIGN(jpeg_width, mcu_width) < ALIGN(jpeg_width, 16))
+ reg |= VDPU720_FILL_RIGHT_E;
+ rkjpegd_write_relaxed(jpegd, reg, VDPU720_REG_SYS);
+
+ /*
+ * REG3: picture dimensions
+ *
+ * PIC_W stays at the raw header width while PIC_H is rounded up to
+ * the MCU boundary. The rounding is there because the vertical MCU
+ * count is derived from PIC_H, see above; the reference driver
+ * programs the raw width here and rounding it up has not been needed.
+ * Either way the buffer is padded out to the MCU width, so the last
+ * MCU column has a row to land in.
+ */
+ rkjpegd_write_relaxed(jpegd,
+ FIELD_PREP(VDPU720_PIC_W_M1, jpeg_width - 1) |
+ FIELD_PREP(VDPU720_PIC_H_M1, jpeg_height_aligned - 1),
+ VDPU720_REG_PIC_SIZE);
+
+ /* REG4: JPEG format, Q/H table counts, restart interval */
+ qtbl_len = VDPU720_TBL_LEN(qtbl_sel * VDPU720_QTBL_COMP_SIZE);
+ hmin_len = VDPU720_TBL_LEN(htbl_sel * VDPU720_HMINCODE_SET_SIZE);
+ hval_len = VDPU720_TBL_LEN(htbl_sel * VDPU720_HVALUE_SET_SIZE);
+
+ reg = FIELD_PREP(VDPU720_JPEG_MODE, jpeg_mode) |
+ FIELD_PREP(VDPU720_PIX_DEPTH, VDPU720_PIX_DEPTH_8) |
+ FIELD_PREP(VDPU720_QTBL_SEL, qtbl_sel) |
+ FIELD_PREP(VDPU720_HTBL_SEL, htbl_sel);
+ if (hdr->restart_interval) {
+ reg |= VDPU720_DRI_E;
+ reg |= FIELD_PREP(VDPU720_DRI_MCU_M1, hdr->restart_interval - 1);
+ }
+ rkjpegd_write_relaxed(jpegd, reg, VDPU720_REG_PIC_FMT);
+
+ /* REG5: horizontal virtual strides */
+ rkjpegd_write_relaxed(jpegd,
+ FIELD_PREP(VDPU720_Y_HOR_STRIDE, y_stride) |
+ FIELD_PREP(VDPU720_UV_HOR_STRIDE, y_stride),
+ VDPU720_REG_HOR_STRIDE);
+
+ /* REG6: total Y-plane size (stride-units * height) */
+ rkjpegd_write_relaxed(jpegd, FIELD_PREP(VDPU720_Y_VSTRIDE, y_vstride),
+ VDPU720_REG_Y_VSTRIDE);
+
+ /* REG7: table lengths + high stride bit */
+ reg = FIELD_PREP(VDPU720_QTBL_LEN, qtbl_len) |
+ FIELD_PREP(VDPU720_HTBL_MINCODE_LEN, hmin_len) |
+ FIELD_PREP(VDPU720_HTBL_VALUE_LEN, hval_len) |
+ FIELD_PREP(VDPU720_Y_HOR_STRIDE_H, y_stride >> 16);
+ rkjpegd_write_relaxed(jpegd, reg, VDPU720_REG_TBL_LEN);
+
+ /* REG8: stream length and start byte */
+ rkjpegd_write_relaxed(jpegd,
+ FIELD_PREP(VDPU720_STRM_START_BYTE, strm_start_byte) |
+ FIELD_PREP(VDPU720_STRM_LEN, strm_len_blks),
+ VDPU720_REG_STRM_LEN);
+
+ /* REG9-REG11: Q/H table DMA addresses (side buffer) */
+ rkjpegd_write_addr(jpegd, VDPU720_REG_QTBL_BASE, tbl_dma);
+ rkjpegd_write_addr(jpegd, VDPU720_REG_HTBL_MINCODE,
+ tbl_dma + VDPU720_HMINCODE_OFF);
+ rkjpegd_write_addr(jpegd, VDPU720_REG_HTBL_VALUE,
+ tbl_dma + VDPU720_HVALUE_OFF);
+
+ /* REG12: stream base (16-byte aligned) */
+ rkjpegd_write_addr(jpegd, VDPU720_REG_STRM_BASE, strm_dma);
+
+ /* REG13: NV12 output buffer */
+ rkjpegd_write_addr(jpegd, VDPU720_REG_OUT_BASE, out_dma);
+
+ /* REG14: stream error handling defaults */
+ rkjpegd_write_relaxed(jpegd, VDPU720_STRM_ERR_DFLT, VDPU720_REG_STRM_ERR);
+
+ /* REG16: enable all internal clock gates */
+ rkjpegd_write_relaxed(jpegd, VDPU720_CLK_GATE_ALL, VDPU720_REG_CLK_GATE);
+
+ /* REG30: AXI performance counter */
+ rkjpegd_write_relaxed(jpegd,
+ VDPU720_PERF_WORK_E | VDPU720_PERF_CLR_E |
+ VDPU720_PERF_CNT_TYPE |
+ FIELD_PREP(VDPU720_PERF_RD_LAT_ID, 0xa),
+ VDPU720_REG_PERF_CTRL);
+
+ return 0;
+}
+
+/**
+ * vdpu720_fill_chroma() - write neutral chroma for a grayscale frame
+ * @ctx: context the job belongs to
+ * @dst_buf: capture buffer whose chroma plane is filled
+ *
+ * The output format converter has no YUV400 path: VDPU720_YUV_OUT_FMT_NV12
+ * only covers the subsampled colour modes, and for a single component frame
+ * the hardware writes the luma plane and leaves the chroma plane untouched.
+ *
+ * Fill the plane here, before the hardware is started: once the decode is
+ * running the interrupt can complete the job and hand the buffer to
+ * userspace at any time.
+ *
+ * Return: 0 on success, -EINVAL if the capture buffer has no kernel mapping.
+ */
+static int vdpu720_fill_chroma(struct rkjpegd_ctx *ctx,
+ struct vb2_v4l2_buffer *dst_buf)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ u32 y_size = ctx->dst_fmt.plane_fmt[0].bytesperline * ctx->dst_fmt.height;
+ u32 size = ctx->dst_fmt.plane_fmt[0].sizeimage;
+ void *dst_cpu;
+
+ dst_cpu = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
+ if (!dst_cpu) {
+ dev_err_ratelimited(jpegd->dev,
+ "JPEG capture buffer has no kernel mapping\n");
+ return -EINVAL;
+ }
+
+ memset(dst_cpu + y_size, 0x80, size - y_size);
+
+ return 0;
+}
+
+/**
+ * rkjpegd_vdpu720_init() - allocate the table side buffer
+ * @ctx: context to allocate the Q/Huffman table buffer for
+ *
+ * Return: 0 on success, -ENOMEM if the allocation failed.
+ */
+static int rkjpegd_vdpu720_init(struct rkjpegd_ctx *ctx)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+
+ ctx->table_base.size = VDPU720_TABLE_BUF_SIZE;
+ ctx->table_base.cpu = dma_alloc_noncoherent(jpegd->dev,
+ ctx->table_base.size,
+ &ctx->table_base.dma,
+ DMA_TO_DEVICE, GFP_KERNEL);
+ if (!ctx->table_base.cpu)
+ return -ENOMEM;
+
+ return 0;
+}
+
+/**
+ * rkjpegd_vdpu720_exit() - free the table side buffer
+ * @ctx: context the buffer belongs to
+ */
+static void rkjpegd_vdpu720_exit(struct rkjpegd_ctx *ctx)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+
+ dma_free_noncoherent(jpegd->dev, ctx->table_base.size,
+ ctx->table_base.cpu, ctx->table_base.dma,
+ DMA_TO_DEVICE);
+}
+
+/**
+ * rkjpegd_vdpu720_run() - fill the side buffer, program the registers and
+ * start the hardware
+ * @ctx: context holding the queues and the side buffer
+ *
+ * The header was parsed when the coded buffer was queued and its references
+ * still point into that buffer, which stays mapped until the job completes.
+ *
+ * Return: 0 with the hardware started and the watchdog armed, or a negative
+ * errno for a frame that cannot be decoded.
+ */
+static int rkjpegd_vdpu720_run(struct rkjpegd_ctx *ctx)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ struct vb2_v4l2_buffer *src_buf, *dst_buf;
+ struct rkjpegd_src_buf *src;
+ const struct v4l2_jpeg_header *hdr;
+ dma_addr_t src_dma, dst_dma;
+ u32 data_offset, payload;
+ u32 hw_strm_off, strm_off, strm_start_byte, strm_len_blks;
+ int ret;
+
+ src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+ dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+ src = vb2_to_rkjpegd_src_buf(&src_buf->vb2_buf);
+
+ if (!src->parsed)
+ return -EINVAL;
+
+ hdr = &src->header;
+ src_dma = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
+ dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
+ data_offset = src_buf->vb2_buf.planes[0].data_offset;
+ payload = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+
+ /*
+ * Rebuild the Q/H-table side buffer every frame. table_base is a
+ * cached (dma_alloc_noncoherent) buffer and write_qtbl()/write_htbl()
+ * stage the source tables through cached stack buffers, so the build
+ * stays on cached memory; dma_sync_single_for_device() then flushes it
+ * to DRAM before the hardware reads it.
+ */
+ memset(ctx->table_base.cpu, 0, ctx->table_base.size);
+
+ ret = vdpu720_write_qtbl(ctx, hdr);
+ if (ret)
+ return ret;
+
+ ret = vdpu720_write_htbl(ctx, hdr);
+ if (ret)
+ return ret;
+
+ dma_sync_single_for_device(jpegd->dev, ctx->table_base.dma,
+ ctx->table_base.size, DMA_TO_DEVICE);
+
+ /*
+ * The stream register must be 16-byte aligned. Round down to the
+ * nearest 16-byte boundary and record the sub-block start byte.
+ *
+ * Both are taken from the start of the plane rather than from the
+ * payload. data_offset is set by userspace in VIDIOC_QBUF and
+ * videobuf2 only rejects it when it is not smaller than bytesused,
+ * so it carries arbitrary low bits. Splitting an address that
+ * already includes it would leave STRM_BASE unaligned by those bits
+ * with no way to encode them, and the hardware would start reading
+ * from the wrong offset.
+ */
+ strm_off = data_offset + hdr->ecs_offset;
+ hw_strm_off = strm_off & ~0xfU;
+ strm_start_byte = strm_off & 0xfU;
+ strm_len_blks = (ALIGN(payload - hw_strm_off, 16) - 1) >> 4;
+
+ ret = vdpu720_fill_regs(ctx, hdr, ctx->table_base.dma,
+ src_dma + hw_strm_off, strm_start_byte,
+ strm_len_blks, dst_dma);
+ if (ret)
+ return ret;
+
+ if (hdr->frame.num_components == 1) {
+ ret = vdpu720_fill_chroma(ctx, dst_buf);
+ if (ret)
+ return ret;
+ }
+
+ rkjpegd_arm_watchdog(jpegd);
+
+ /*
+ * Enable the buffer empty condition along with the timeout. A frame
+ * whose entropy data ends before the last macroblock row runs the
+ * decoder off the end of the stream, and with the condition masked it
+ * has nothing to report and waits. VDPU720_ERR_MASK already treats
+ * the resulting status as an error, so the handler is prepared for a
+ * status the hardware was never told to raise.
+ */
+ rkjpegd_write(jpegd,
+ VDPU720_DEC_E | VDPU720_TIMEOUT_E | VDPU720_BUF_EMPTY_E,
+ VDPU720_REG_INT);
+
+ return 0;
+}
+
+/**
+ * vdpu720_soft_reset() - ask the block to reset itself
+ * @jpegd: device to reset
+ *
+ * Trigger the in-block soft reset and wait for it to report ready. Sleeps
+ * while polling, so it must not be called from atomic context.
+ *
+ * Return: 0 once the block reports the reset complete, -ETIMEDOUT if it does
+ * not do so within 10 ms.
+ */
+static int vdpu720_soft_reset(struct rkjpegd_dev *jpegd)
+{
+ u32 status;
+ int ret;
+
+ /*
+ * If the decoder is idle (DEC_E=0), set FORCE_SOFTRESET_VALID
+ * before triggering the soft reset, per downstream BSP behaviour.
+ */
+ status = rkjpegd_read(jpegd, VDPU720_REG_INT);
+ if (!(status & VDPU720_DEC_E))
+ rkjpegd_write(jpegd, VDPU720_FORCE_SOFTRST, VDPU720_REG_SYS);
+
+ rkjpegd_write(jpegd, status | VDPU720_SOFT_RST_EN, VDPU720_REG_INT);
+
+ ret = readl_relaxed_poll_timeout(jpegd->regs + VDPU720_REG_INT, status,
+ status & VDPU720_SOFT_RST_RDY,
+ 5, 10000);
+ if (ret)
+ dev_warn(jpegd->dev, "soft reset timed out\n");
+
+ return ret;
+}
+
+/**
+ * vdpu720_hard_reset() - pulse the block's reset lines
+ * @jpegd: device to reset
+ *
+ * reset_control_reset() is not usable here. The lines come from a Rockchip
+ * CRU, and rockchip_softrst_ops in drivers/clk/rockchip/softrst.c implements
+ * only .assert and .deassert, so reset_control_reset() returns -ENOTSUPP
+ * without touching the hardware. Drive the pulse by hand instead.
+ *
+ * Only reached from the watchdog, which runs from a workqueue, so sleeping
+ * between the two halves is fine.
+ *
+ * Return: 0 on success, negative errno if a reset control could not be
+ * asserted or deasserted.
+ */
+static int vdpu720_hard_reset(struct rkjpegd_dev *jpegd)
+{
+ int ret;
+
+ ret = reset_control_assert(jpegd->resets);
+ if (ret)
+ return ret;
+
+ usleep_range(10, 20);
+
+ return reset_control_deassert(jpegd->resets);
+}
+
+/**
+ * rkjpegd_vdpu720_reset() - error recovery reset called by the watchdog
+ * @ctx: context whose job timed out
+ *
+ * Try a soft reset first; fall back to a full hardware reset if the soft
+ * reset does not complete.
+ */
+static void rkjpegd_vdpu720_reset(struct rkjpegd_ctx *ctx)
+{
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ int ret;
+
+ ret = vdpu720_soft_reset(jpegd);
+ if (ret) {
+ dev_warn(jpegd->dev, "falling back to hard reset\n");
+
+ ret = vdpu720_hard_reset(jpegd);
+ if (ret)
+ dev_err(jpegd->dev, "hard reset failed: %d\n", ret);
+ }
+
+ rkjpegd_write(jpegd, 0, VDPU720_REG_INT);
+}
+
+static irqreturn_t rkjpegd_vdpu720_irq(int irq, void *dev_id)
+{
+ struct rkjpegd_dev *jpegd = dev_id;
+ enum vb2_buffer_state state;
+ u32 status, clr_mask;
+
+ status = rkjpegd_read(jpegd, VDPU720_REG_INT);
+
+ /*
+ * Two-phase IRQ clear. Write back a masked subset of status bits
+ * before checking IRQ_RAW, which some revisions require.
+ */
+ clr_mask = (~(VDPU720_IRQ_CLR_COND & status)) &
+ (VDPU720_IRQ_CLR_KEEP & status);
+ rkjpegd_write(jpegd, clr_mask, VDPU720_REG_INT);
+
+ if (!(status & VDPU720_IRQ_RAW))
+ return IRQ_NONE;
+
+ rkjpegd_write(jpegd, 0, VDPU720_REG_INT);
+
+ state = (status & VDPU720_ERR_MASK) ?
+ VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE;
+
+ if (status & VDPU720_DEC_ERR) {
+ u32 mcu_pos = rkjpegd_read(jpegd, VDPU720_REG_DBG_MCU_POS);
+ u32 err_info = rkjpegd_read(jpegd, VDPU720_REG_DBG_ERROR);
+
+ dev_warn_ratelimited(jpegd->dev,
+ "decode error: MCU pos=(%u,%u) flags=0x%04x [%s%s%s%s%s%s%s%s%s%s] first_idx=%u\n",
+ (u32)FIELD_GET(VDPU720_DBG_MCU_POS_X, mcu_pos),
+ (u32)FIELD_GET(VDPU720_DBG_MCU_POS_Y, mcu_pos),
+ (u32)FIELD_GET(VDPU720_DERR_FLAGS, err_info),
+ (err_info & VDPU720_DERR_DRI_SEQ) ? "dri_seq " : "",
+ (err_info & VDPU720_DERR_STREAM_FFFF) ? "ffff " : "",
+ (err_info & VDPU720_DERR_OTHER_MARK) ? "bad_mark " : "",
+ (err_info & VDPU720_DERR_MCU_CNT_L) ? "dri_early " : "",
+ (err_info & VDPU720_DERR_MCU_CNT_M) ? "dri_late " : "",
+ (err_info & VDPU720_DERR_EOI_NO_END) ? "eoi_early " : "",
+ (err_info & VDPU720_DERR_END_NO_EOI) ? "no_eoi " : "",
+ (err_info & VDPU720_DERR_OVERFLOW) ? "overflow " : "",
+ (err_info & VDPU720_DERR_HUFF_EMPTY) ? "huff_empty " : "",
+ (err_info & (VDPU720_DERR_STREAM_R0 |
+ VDPU720_DERR_STREAM_R1)) ? "stream_mark " : "",
+ (u32)FIELD_GET(VDPU720_DERR_FIRST_IDX, err_info));
+
+ rkjpegd_write(jpegd, err_info, VDPU720_REG_DBG_ERROR);
+ }
+
+ rkjpegd_irq_done(jpegd, state);
+
+ return IRQ_HANDLED;
+}
+
+static void rkjpegd_watchdog(struct work_struct *work)
+{
+ struct rkjpegd_dev *jpegd = container_of(to_delayed_work(work),
+ struct rkjpegd_dev,
+ watchdog_work);
+ struct rkjpegd_ctx *ctx = v4l2_m2m_get_curr_priv(jpegd->m2m_dev);
+
+ if (!ctx)
+ return;
+
+ disable_irq(jpegd->irq);
+
+ dev_err(jpegd->dev, "frame processing timed out\n");
+ rkjpegd_vdpu720_reset(ctx);
+
+ /*
+ * The frame is handed back as an error even if it did complete: the
+ * block has been reset underneath it and the interrupt telling us so
+ * was cleared with it. After RKJPEGD_TIMEOUT_MS that is what it is.
+ */
+ rkjpegd_job_finish(ctx, VB2_BUF_STATE_ERROR);
+
+ enable_irq(jpegd->irq);
+}
+
+/**
+ * rkjpegd_source_change() - report a resolution change to userspace
+ * @ctx: context the coded buffer belongs to
+ * @src_buf: coded buffer whose header the new resolution is taken from
+ *
+ * Renegotiates the capture format from @src_buf and reports it, unless it
+ * already describes what was negotiated. Sets @rkjpegd_ctx.source_change,
+ * which stops rkjpegd_job_ready() from letting any further job run until the
+ * capture queue has been set up again.
+ *
+ * Called both when a coded buffer is queued and when one reaches the head of
+ * the queue, see the comments at those two call sites.
+ */
+static void rkjpegd_source_change(struct rkjpegd_ctx *ctx,
+ struct rkjpegd_src_buf *src_buf)
+{
+ const struct v4l2_jpeg_header *hdr = NULL;
+ u32 width, height, buf_width, buf_height;
+
+ if (src_buf->parsed) {
+ hdr = &src_buf->header;
+ width = hdr->frame.width;
+ height = hdr->frame.height;
+ } else {
+ /*
+ * A frame the parser refused carries no dimensions of its own,
+ * and the job will hand it back with an error. Report the
+ * change anyway, from the coded format userspace configured.
+ * V4L2_FMT_FLAG_DYN_RESOLUTION tells an application to wait
+ * for this event before it sets the capture queue up, so one
+ * that never arrives leaves it waiting for a frame that cannot
+ * come instead of seeing the error and giving up.
+ */
+ width = ctx->src_fmt.width;
+ height = ctx->src_fmt.height;
+ }
+ buf_width = vdpu720_capture_width(hdr, width);
+ buf_height = ALIGN(height, RKJPEGD_RAW_STEP);
+
+ if (!ctx->initial_source_change &&
+ ctx->dst_fmt.width == buf_width &&
+ ctx->dst_fmt.height == buf_height &&
+ ctx->crop.width == width && ctx->crop.height == height)
+ return;
+
+ dev_dbg(ctx->dev->dev, "source change to %ux%u\n", width, height);
+
+ rkjpegd_fill_raw_fmt(&ctx->dst_fmt, buf_width, buf_height);
+ ctx->crop.left = 0;
+ ctx->crop.top = 0;
+ ctx->crop.width = width;
+ ctx->crop.height = height;
+
+ v4l2_event_queue_fh(&ctx->fh, &rkjpegd_src_change_event);
+ ctx->source_change = true;
+ ctx->initial_source_change = false;
+
+ if (vb2_is_streaming(v4l2_m2m_get_dst_vq(ctx->fh.m2m_ctx)))
+ rkjpegd_set_last_buffer(ctx);
+}
+
+static void rkjpegd_device_run(void *priv)
+{
+ struct rkjpegd_ctx *ctx = priv;
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ struct vb2_v4l2_buffer *src, *dst;
+ int ret;
+
+ src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+ dst = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+ if (WARN_ON(!src) || WARN_ON(!dst))
+ return;
+
+ /*
+ * rkjpegd_buf_queue() only looks for a resolution change while the
+ * coded queue is empty, so a change carried by a buffer queued behind
+ * others is not seen there. Look again now that this buffer has
+ * reached the head: everything queued before it has been decoded and
+ * handed back, which is the point the specification wants the change
+ * reported at.
+ *
+ * Doing it here rather than earlier is also what keeps the buffers
+ * queued behind this one correct. They are still coded frames of the
+ * new resolution, and they stay queued while the capture queue is set
+ * up again, so they are decoded into buffers that fit them.
+ */
+ rkjpegd_source_change(ctx, vb2_to_rkjpegd_src_buf(&src->vb2_buf));
+ if (ctx->source_change) {
+ /*
+ * rkjpegd_set_last_buffer() has given dst back to userspace
+ * with V4L2_BUF_FLAG_LAST set, so it must not be touched here.
+ * Finish the job without consuming src either: it is decoded
+ * once the capture queue has been set up for it.
+ */
+ v4l2_m2m_job_finish(jpegd->m2m_dev, ctx->fh.m2m_ctx);
+ return;
+ }
+
+ ret = pm_runtime_resume_and_get(jpegd->dev);
+ if (ret < 0)
+ goto err_finish;
+
+ ret = clk_bulk_enable(RKJPEGD_NUM_CLOCKS, jpegd->clocks);
+ if (ret)
+ goto err_pm_put;
+
+ v4l2_m2m_buf_copy_metadata(src, dst);
+
+ ret = rkjpegd_vdpu720_run(ctx);
+ if (ret)
+ goto err_clk_disable;
+
+ return;
+
+err_clk_disable:
+ clk_bulk_disable(RKJPEGD_NUM_CLOCKS, jpegd->clocks);
+err_pm_put:
+ pm_runtime_put_autosuspend(jpegd->dev);
+err_finish:
+ rkjpegd_job_finish_no_pm(ctx, VB2_BUF_STATE_ERROR);
+}
+
+static int rkjpegd_job_ready(void *priv)
+{
+ struct rkjpegd_ctx *ctx = priv;
+
+ return ctx->source_change ? 0 : 1;
+}
+
+static const struct v4l2_m2m_ops rkjpegd_m2m_ops = {
+ .device_run = rkjpegd_device_run,
+ .job_ready = rkjpegd_job_ready,
+};
+
+/*
+ * Bitstream inspection
+ *
+ * The header is parsed when the buffer is queued rather than when the job
+ * runs: the resolution it carries is what a source change reports, and the
+ * references it hands out point into the payload, which stays mapped until
+ * the buffer is given back.
+ */
+
+static bool rkjpegd_has_eoi(const void *data, u32 len)
+{
+ u8 tail[64];
+ u32 tail_len, i;
+
+ /*
+ * v4l2_jpeg_parse_header() stops at the start of scan marker and never
+ * looks at the entropy coded data behind it, so a frame truncated
+ * because it did not fit the buffer parses without an error. The
+ * hardware would decode as many macroblocks as it finds and hand out a
+ * half filled frame indistinguishable from a good one.
+ *
+ * An end of image marker cannot appear inside the entropy coded data,
+ * where 0xff bytes are stuffed, so finding one means the frame is
+ * complete. Only the last few bytes are searched, which covers a
+ * payload padded up to a 64 byte boundary and any short trailer.
+ * Looking further back is not worth it: the buffer is uncached, so a
+ * walk over the whole payload costs one bus transaction per byte, and
+ * a payload ending far behind its marker is one whose bytesused was
+ * never set, in which case videobuf2 substitutes the full plane length
+ * and a recycled buffer holds the previous frame's bytes back there
+ * anyway. The tail is copied out in one memcpy() for the same reason.
+ */
+ tail_len = min_t(u32, len, sizeof(tail));
+ memcpy(tail, data + len - tail_len, tail_len);
+
+ for (i = 0; i + 1 < tail_len; i++)
+ if (tail[i] == 0xff && tail[i + 1] == 0xd9)
+ return true;
+
+ return false;
+}
+
+static bool rkjpegd_header_supported(struct rkjpegd_dev *jpegd,
+ const struct v4l2_jpeg_header *header,
+ u32 len)
+{
+ if (header->frame.width < RKJPEGD_MIN_WIDTH ||
+ header->frame.height < RKJPEGD_MIN_HEIGHT ||
+ header->frame.width > RKJPEGD_MAX_SIZE ||
+ header->frame.height > RKJPEGD_MAX_SIZE) {
+ dev_err_ratelimited(jpegd->dev,
+ "unsupported JPEG picture size %ux%u, not within %ux%u..%ux%u\n",
+ header->frame.width, header->frame.height,
+ RKJPEGD_MIN_WIDTH, RKJPEGD_MIN_HEIGHT,
+ RKJPEGD_MAX_SIZE, RKJPEGD_MAX_SIZE);
+ return false;
+ }
+
+ /*
+ * v4l2_jpeg_parse_header() accepts twelve bit samples for SOF1, but
+ * the register programming always asks for eight bit ones.
+ */
+ if (header->frame.precision != 8) {
+ dev_err_ratelimited(jpegd->dev,
+ "unsupported JPEG sample precision %u\n",
+ header->frame.precision);
+ return false;
+ }
+
+ if (header->frame.num_components != 1 &&
+ header->frame.num_components != 3) {
+ dev_err_ratelimited(jpegd->dev,
+ "unsupported JPEG component count %u\n",
+ header->frame.num_components);
+ return false;
+ }
+
+ if (header->scan->num_components != header->frame.num_components) {
+ dev_err_ratelimited(jpegd->dev,
+ "JPEG scan covers %u of %u components, non interleaved scans are not supported\n",
+ header->scan->num_components,
+ header->frame.num_components);
+ return false;
+ }
+
+ if (header->ecs_offset >= len) {
+ dev_err_ratelimited(jpegd->dev,
+ "JPEG entropy coded data starts beyond the payload\n");
+ return false;
+ }
+
+ return true;
+}
+
+static void rkjpegd_parse_src_buf(struct rkjpegd_ctx *ctx,
+ struct vb2_buffer *vb)
+{
+ struct rkjpegd_src_buf *src_buf = vb2_to_rkjpegd_src_buf(vb);
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ u32 data_offset = vb->planes[0].data_offset;
+ u32 len = vb2_get_plane_payload(vb, 0);
+ void *data = vb2_plane_vaddr(vb, 0);
+ int ret;
+
+ memset(&src_buf->header, 0, sizeof(src_buf->header));
+ memset(&src_buf->scan, 0, sizeof(src_buf->scan));
+ memset(src_buf->quantization_tables, 0,
+ sizeof(src_buf->quantization_tables));
+ memset(src_buf->huffman_tables, 0, sizeof(src_buf->huffman_tables));
+ src_buf->header.scan = &src_buf->scan;
+ src_buf->header.quantization_tables = src_buf->quantization_tables;
+ src_buf->header.huffman_tables = src_buf->huffman_tables;
+ src_buf->parsed = false;
+
+ if (!data) {
+ dev_err_ratelimited(jpegd->dev,
+ "JPEG buffer has no kernel mapping\n");
+ return;
+ }
+
+ if (len <= data_offset || len - data_offset < 4) {
+ dev_err_ratelimited(jpegd->dev,
+ "JPEG payload of %u bytes is too short\n",
+ len);
+ return;
+ }
+
+ data += data_offset;
+ len -= data_offset;
+
+ ret = v4l2_jpeg_parse_header(data, len, &src_buf->header);
+ if (ret < 0) {
+ dev_warn_ratelimited(jpegd->dev,
+ "failed to parse JPEG header: %d (len=%u first_bytes=%*ph)\n",
+ ret, len, min_t(int, len, 8), data);
+ return;
+ }
+
+ if (!rkjpegd_header_supported(jpegd, &src_buf->header, len))
+ return;
+
+ if (!rkjpegd_has_eoi(data, len)) {
+ dev_err_ratelimited(jpegd->dev,
+ "truncated JPEG, no end of image marker at the end of the %u byte payload (buffer too small?)\n",
+ len);
+ return;
+ }
+
+ src_buf->parsed = true;
+}
+
+static int rkjpegd_queue_setup(struct vb2_queue *vq, unsigned int *num_buffers,
+ unsigned int *num_planes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct rkjpegd_ctx *ctx = vb2_get_drv_priv(vq);
+ struct v4l2_pix_format_mplane *pix_mp;
+
+ pix_mp = V4L2_TYPE_IS_OUTPUT(vq->type) ? &ctx->src_fmt : &ctx->dst_fmt;
+
+ if (*num_planes) {
+ if (*num_planes != 1)
+ return -EINVAL;
+ if (sizes[0] < pix_mp->plane_fmt[0].sizeimage)
+ return -EINVAL;
+ return 0;
+ }
+
+ *num_planes = 1;
+ sizes[0] = pix_mp->plane_fmt[0].sizeimage;
+
+ /*
+ * The first frame to arrive after the coded queue was set up has to
+ * report its resolution even when it happens to match what userspace
+ * guessed, otherwise an application waiting for the event before it
+ * allocates capture buffers never gets one.
+ */
+ if (V4L2_TYPE_IS_OUTPUT(vq->type))
+ ctx->initial_source_change = true;
+
+ return 0;
+}
+
+static int rkjpegd_buf_out_validate(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+ vbuf->field = V4L2_FIELD_NONE;
+
+ return 0;
+}
+
+static int rkjpegd_buf_prepare(struct vb2_buffer *vb)
+{
+ struct vb2_queue *vq = vb->vb2_queue;
+ struct rkjpegd_ctx *ctx = vb2_get_drv_priv(vq);
+ struct v4l2_pix_format_mplane *pix_mp;
+
+ if (V4L2_TYPE_IS_OUTPUT(vq->type)) {
+ pix_mp = &ctx->src_fmt;
+
+ if (vb2_plane_size(vb, 0) < pix_mp->plane_fmt[0].sizeimage)
+ return -EINVAL;
+
+ return 0;
+ }
+
+ pix_mp = &ctx->dst_fmt;
+
+ /*
+ * A capture buffer queued into a queue that is still streaming while
+ * a source change is pending has the size of the previous format and
+ * is about to be reallocated, and no job runs until it has been. Do
+ * not reject it here, and do not measure it against the new format
+ * either, which is larger than the buffer is whenever the resolution
+ * grew. It carries no picture and rkjpegd_stop_streaming() hands it
+ * back before the queue is built again.
+ *
+ * A buffer queued while the queue is not streaming is the other case.
+ * It belongs to the setup for the format that was just reported,
+ * whether that is the first of a stream or the one answering a source
+ * change, and it is what the next job decodes into, so it is measured
+ * like any other.
+ */
+ if (ctx->source_change && vb2_is_streaming(vq))
+ return 0;
+
+ if (vb2_plane_size(vb, 0) < pix_mp->plane_fmt[0].sizeimage)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void rkjpegd_buf_queue(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct rkjpegd_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+ if (V4L2_TYPE_IS_CAPTURE(vb->vb2_queue->type)) {
+ if (vb2_is_streaming(vb->vb2_queue) &&
+ v4l2_m2m_dst_buf_is_last(ctx->fh.m2m_ctx)) {
+ vbuf->field = V4L2_FIELD_NONE;
+ vbuf->sequence = ctx->sequence_cap++;
+ v4l2_m2m_last_buffer_done(ctx->fh.m2m_ctx, vbuf);
+ v4l2_event_queue_fh(&ctx->fh, &rkjpegd_eos_event);
+ return;
+ }
+
+ v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
+ return;
+ }
+
+ rkjpegd_parse_src_buf(ctx, vb);
+
+ /*
+ * Only look at a resolution change while nothing else is pending. A
+ * buffer queued behind frames that have not been decoded yet would
+ * otherwise renegotiate the capture queue out from under them;
+ * rkjpegd_device_run() picks that case up once the buffer reaches the
+ * head of the queue.
+ *
+ * This early report is still needed for the first buffer of a stream.
+ * The mem2mem core runs no job until both queues are streaming, so
+ * rkjpegd_device_run() cannot report the resolution an application has
+ * to know before it can set the capture queue up and stream it on.
+ */
+ if (!v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx))
+ rkjpegd_source_change(ctx, vb2_to_rkjpegd_src_buf(vb));
+
+ v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
+}
+
+static int rkjpegd_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+ struct rkjpegd_ctx *ctx = vb2_get_drv_priv(vq);
+
+ v4l2_m2m_update_start_streaming_state(ctx->fh.m2m_ctx, vq);
+
+ if (V4L2_TYPE_IS_OUTPUT(vq->type)) {
+ ctx->sequence_out = 0;
+ } else {
+ ctx->sequence_cap = 0;
+ ctx->source_change = false;
+ ctx->initial_source_change = false;
+ }
+
+ return 0;
+}
+
+static void rkjpegd_stop_streaming(struct vb2_queue *vq)
+{
+ struct rkjpegd_ctx *ctx = vb2_get_drv_priv(vq);
+ struct vb2_v4l2_buffer *vbuf;
+
+ for (;;) {
+ if (V4L2_TYPE_IS_OUTPUT(vq->type))
+ vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+ else
+ vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+ if (!vbuf)
+ break;
+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
+ }
+
+ v4l2_m2m_update_stop_streaming_state(ctx->fh.m2m_ctx, vq);
+
+ /*
+ * A drain that was requested before the source change was reported
+ * has not run yet; keep it pending across the capture queue restart.
+ */
+ if (V4L2_TYPE_IS_CAPTURE(vq->type) && ctx->source_change &&
+ ctx->fh.m2m_ctx->last_src_buf)
+ ctx->fh.m2m_ctx->is_draining = true;
+
+ if (V4L2_TYPE_IS_OUTPUT(vq->type) &&
+ v4l2_m2m_has_stopped(ctx->fh.m2m_ctx))
+ v4l2_event_queue_fh(&ctx->fh, &rkjpegd_eos_event);
+}
+
+static const struct vb2_ops rkjpegd_queue_ops = {
+ .queue_setup = rkjpegd_queue_setup,
+ .buf_out_validate = rkjpegd_buf_out_validate,
+ .buf_prepare = rkjpegd_buf_prepare,
+ .buf_queue = rkjpegd_buf_queue,
+ .start_streaming = rkjpegd_start_streaming,
+ .stop_streaming = rkjpegd_stop_streaming,
+};
+
+static int rkjpegd_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq)
+{
+ struct rkjpegd_ctx *ctx = priv;
+ struct rkjpegd_dev *jpegd = ctx->dev;
+ int ret;
+
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ src_vq->drv_priv = ctx;
+ src_vq->ops = &rkjpegd_queue_ops;
+ src_vq->mem_ops = &vb2_dma_contig_memops;
+ src_vq->buf_struct_size = sizeof(struct rkjpegd_src_buf);
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->lock = &jpegd->vdev_lock;
+ src_vq->dev = jpegd->v4l2_dev.dev;
+
+ /*
+ * Driver does mostly sequential access, so sacrifice TLB efficiency
+ * for faster allocation. Both queues keep their kernel mapping: the
+ * quantisation and Huffman tables are read out of the coded buffer,
+ * and a grayscale frame leaves its chroma plane to the driver.
+ */
+ src_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
+
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ dst_vq->drv_priv = ctx;
+ dst_vq->ops = &rkjpegd_queue_ops;
+ dst_vq->mem_ops = &vb2_dma_contig_memops;
+ dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->lock = &jpegd->vdev_lock;
+ dst_vq->dev = jpegd->v4l2_dev.dev;
+ dst_vq->dma_attrs = DMA_ATTR_ALLOC_SINGLE_PAGES;
+
+ return vb2_queue_init(dst_vq);
+}
+
+static int rkjpegd_open(struct file *filp)
+{
+ struct rkjpegd_dev *jpegd = video_drvdata(filp);
+ struct rkjpegd_ctx *ctx;
+ int ret;
+
+ ctx = kzalloc_obj(*ctx);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->dev = jpegd;
+ rkjpegd_reset_fmts(ctx);
+ v4l2_fh_init(&ctx->fh, video_devdata(filp));
+
+ ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(jpegd->m2m_dev, ctx,
+ rkjpegd_queue_init);
+ if (IS_ERR(ctx->fh.m2m_ctx)) {
+ ret = PTR_ERR(ctx->fh.m2m_ctx);
+ goto err_free_ctx;
+ }
+
+ ret = rkjpegd_vdpu720_init(ctx);
+ if (ret)
+ goto err_cleanup_m2m_ctx;
+
+ v4l2_fh_add(&ctx->fh, filp);
+
+ return 0;
+
+err_cleanup_m2m_ctx:
+ v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+err_free_ctx:
+ v4l2_fh_exit(&ctx->fh);
+ kfree(ctx);
+
+ return ret;
+}
+
+static int rkjpegd_release(struct file *filp)
+{
+ struct rkjpegd_ctx *ctx = file_to_rkjpegd_ctx(filp);
+
+ v4l2_fh_del(&ctx->fh, filp);
+ v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+ rkjpegd_vdpu720_exit(ctx);
+ v4l2_fh_exit(&ctx->fh);
+ kfree(ctx);
+
+ return 0;
+}
+
+static const struct v4l2_file_operations rkjpegd_fops = {
+ .owner = THIS_MODULE,
+ .open = rkjpegd_open,
+ .release = rkjpegd_release,
+ .poll = v4l2_m2m_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = v4l2_m2m_fop_mmap,
+};
+
+static int rkjpegd_v4l2_init(struct rkjpegd_dev *jpegd)
+{
+ int ret;
+
+ ret = v4l2_device_register(jpegd->dev, &jpegd->v4l2_dev);
+ if (ret) {
+ dev_err(jpegd->dev, "failed to register V4L2 device\n");
+ return ret;
+ }
+
+ jpegd->m2m_dev = v4l2_m2m_init(&rkjpegd_m2m_ops);
+ if (IS_ERR(jpegd->m2m_dev)) {
+ v4l2_err(&jpegd->v4l2_dev, "failed to init mem2mem device\n");
+ ret = PTR_ERR(jpegd->m2m_dev);
+ goto err_unregister_v4l2;
+ }
+
+ jpegd->mdev.dev = jpegd->dev;
+ strscpy(jpegd->mdev.model, RKJPEGD_NAME, sizeof(jpegd->mdev.model));
+ media_device_init(&jpegd->mdev);
+ jpegd->v4l2_dev.mdev = &jpegd->mdev;
+
+ jpegd->vdev.lock = &jpegd->vdev_lock;
+ jpegd->vdev.v4l2_dev = &jpegd->v4l2_dev;
+ jpegd->vdev.fops = &rkjpegd_fops;
+ jpegd->vdev.release = video_device_release_empty;
+ jpegd->vdev.vfl_dir = VFL_DIR_M2M;
+ jpegd->vdev.device_caps = V4L2_CAP_STREAMING |
+ V4L2_CAP_VIDEO_M2M_MPLANE;
+ jpegd->vdev.ioctl_ops = &rkjpegd_ioctl_ops;
+ video_set_drvdata(&jpegd->vdev, jpegd);
+ strscpy(jpegd->vdev.name, RKJPEGD_NAME, sizeof(jpegd->vdev.name));
+
+ ret = video_register_device(&jpegd->vdev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ v4l2_err(&jpegd->v4l2_dev, "failed to register video device\n");
+ goto err_cleanup_mc;
+ }
+
+ ret = v4l2_m2m_register_media_controller(jpegd->m2m_dev, &jpegd->vdev,
+ MEDIA_ENT_F_PROC_VIDEO_DECODER);
+ if (ret) {
+ v4l2_err(&jpegd->v4l2_dev,
+ "failed to init V4L2 M2M media controller\n");
+ goto err_unregister_vdev;
+ }
+
+ ret = media_device_register(&jpegd->mdev);
+ if (ret) {
+ v4l2_err(&jpegd->v4l2_dev, "failed to register media device\n");
+ goto err_unregister_mc;
+ }
+
+ return 0;
+
+err_unregister_mc:
+ v4l2_m2m_unregister_media_controller(jpegd->m2m_dev);
+err_unregister_vdev:
+ video_unregister_device(&jpegd->vdev);
+err_cleanup_mc:
+ media_device_cleanup(&jpegd->mdev);
+ v4l2_m2m_release(jpegd->m2m_dev);
+err_unregister_v4l2:
+ v4l2_device_unregister(&jpegd->v4l2_dev);
+
+ return ret;
+}
+
+static void rkjpegd_v4l2_cleanup(struct rkjpegd_dev *jpegd)
+{
+ media_device_unregister(&jpegd->mdev);
+ v4l2_m2m_unregister_media_controller(jpegd->m2m_dev);
+ video_unregister_device(&jpegd->vdev);
+ media_device_cleanup(&jpegd->mdev);
+ v4l2_m2m_release(jpegd->m2m_dev);
+ v4l2_device_unregister(&jpegd->v4l2_dev);
+}
+
+static int rkjpegd_probe(struct platform_device *pdev)
+{
+ struct rkjpegd_dev *jpegd;
+ unsigned int i;
+ int ret;
+
+ jpegd = devm_kzalloc(&pdev->dev, sizeof(*jpegd), GFP_KERNEL);
+ if (!jpegd)
+ return -ENOMEM;
+
+ jpegd->dev = &pdev->dev;
+ platform_set_drvdata(pdev, jpegd);
+ mutex_init(&jpegd->vdev_lock);
+ INIT_DELAYED_WORK(&jpegd->watchdog_work, rkjpegd_watchdog);
+
+ for (i = 0; i < RKJPEGD_NUM_CLOCKS; i++)
+ jpegd->clocks[i].id = rkjpegd_clk_names[i];
+
+ ret = devm_clk_bulk_get(&pdev->dev, RKJPEGD_NUM_CLOCKS, jpegd->clocks);
+ if (ret)
+ return ret;
+
+ jpegd->resets = devm_reset_control_array_get_exclusive(&pdev->dev);
+ if (IS_ERR(jpegd->resets))
+ return dev_err_probe(&pdev->dev, PTR_ERR(jpegd->resets),
+ "failed to get resets\n");
+
+ jpegd->regs = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(jpegd->regs))
+ return PTR_ERR(jpegd->regs);
+
+ ret = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (ret)
+ return dev_err_probe(&pdev->dev, ret, "failed to set DMA mask\n");
+
+ jpegd->irq = platform_get_irq(pdev, 0);
+ if (jpegd->irq < 0)
+ return jpegd->irq;
+
+ ret = devm_request_irq(&pdev->dev, jpegd->irq, rkjpegd_vdpu720_irq, 0,
+ dev_name(&pdev->dev), jpegd);
+ if (ret)
+ return dev_err_probe(&pdev->dev, ret, "failed to request irq\n");
+
+ ret = clk_bulk_prepare(RKJPEGD_NUM_CLOCKS, jpegd->clocks);
+ if (ret)
+ return dev_err_probe(&pdev->dev, ret,
+ "failed to prepare clocks\n");
+
+ pm_runtime_set_autosuspend_delay(&pdev->dev, 100);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_enable(&pdev->dev);
+
+ ret = reset_control_deassert(jpegd->resets);
+ if (ret) {
+ dev_err_probe(&pdev->dev, ret, "failed to deassert resets\n");
+ goto err_disable_pm;
+ }
+
+ ret = rkjpegd_v4l2_init(jpegd);
+ if (ret)
+ goto err_assert_resets;
+
+ return 0;
+
+err_assert_resets:
+ reset_control_assert(jpegd->resets);
+err_disable_pm:
+ pm_runtime_dont_use_autosuspend(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
+ clk_bulk_unprepare(RKJPEGD_NUM_CLOCKS, jpegd->clocks);
+
+ return ret;
+}
+
+static void rkjpegd_remove(struct platform_device *pdev)
+{
+ struct rkjpegd_dev *jpegd = platform_get_drvdata(pdev);
+
+ cancel_delayed_work_sync(&jpegd->watchdog_work);
+ rkjpegd_v4l2_cleanup(jpegd);
+ reset_control_assert(jpegd->resets);
+ pm_runtime_dont_use_autosuspend(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
+ clk_bulk_unprepare(RKJPEGD_NUM_CLOCKS, jpegd->clocks);
+}
+
+/*
+ * The clocks are prepared once and only enabled and disabled around a job,
+ * which is what lets rkjpegd_job_finish() run from the interrupt handler.
+ * There is nothing else to do around a runtime transition: the power domain
+ * is handled by genpd on the device's behalf.
+ *
+ * A system transition is not the same. pm_runtime_force_suspend() calls the
+ * runtime suspend callback whatever the usage count says, so genpd drops the
+ * domain under a job that is still running: userspace is frozen by then, but
+ * a decode started just before the freeze is not, and it has until the
+ * watchdog expires to finish. Park the mem2mem queue first and wait the
+ * running job out. That wait is bounded by the same watchdog, which runs on
+ * the unfreezable system workqueue and hands the job back either way.
+ *
+ * The queue has to come back either way as well. A device whose suspend
+ * returned an error is never handed to the resume callback.
+ */
+static int rkjpegd_suspend(struct device *dev)
+{
+ struct rkjpegd_dev *jpegd = dev_get_drvdata(dev);
+ int ret;
+
+ v4l2_m2m_suspend(jpegd->m2m_dev);
+
+ ret = pm_runtime_force_suspend(dev);
+ if (ret)
+ v4l2_m2m_resume(jpegd->m2m_dev);
+
+ return ret;
+}
+
+static int rkjpegd_resume(struct device *dev)
+{
+ struct rkjpegd_dev *jpegd = dev_get_drvdata(dev);
+ int ret;
+
+ ret = pm_runtime_force_resume(dev);
+
+ v4l2_m2m_resume(jpegd->m2m_dev);
+
+ return ret;
+}
+
+static const struct dev_pm_ops rkjpegd_pm_ops = {
+ SYSTEM_SLEEP_PM_OPS(rkjpegd_suspend, rkjpegd_resume)
+};
+
+static const struct of_device_id of_rkjpegd_match[] = {
+ { .compatible = "rockchip,rk3568-jpegd" },
+ { .compatible = "rockchip,rk3588-jpegd" },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, of_rkjpegd_match);
+
+static struct platform_driver rkjpegd_driver = {
+ .probe = rkjpegd_probe,
+ .remove = rkjpegd_remove,
+ .driver = {
+ .name = RKJPEGD_NAME,
+ .of_match_table = of_rkjpegd_match,
+ .pm = pm_sleep_ptr(&rkjpegd_pm_ops),
+ },
+};
+module_platform_driver(rkjpegd_driver);
+
+MODULE_DESCRIPTION("Rockchip JPEG decoder driver");
+MODULE_AUTHOR("Lucas Sinn <lucas.sinn at wolfvision.net>");
+MODULE_LICENSE("GPL");
--
2.47.3
More information about the linux-arm-kernel
mailing list