[PATCH v6 11/17] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS control

Mon May 30 02:18:36 PDT 2022

On 29/05/2022 08:40, Jernej Škrabec wrote:
> Hi!
> 
> This series looks very good and I plan to test it shortly on Cedrus, but I 
> have one major concern below.
> 
> Dne petek, 27. maj 2022 ob 16:31:28 CEST je Benjamin Gaignard napisal(a):
>> The number of 'entry point offset' can be very variable.
>> Instead of using a large static array define a v4l2 dynamic array
>> of U32 (V4L2_CTRL_TYPE_U32).
>> The number of entry point offsets is reported by the elems field
>> and in struct v4l2_ctrl_hevc_slice_params.num_entry_point_offsets
>> field.
> 
> Slice control by itself is variable length array, so you would actually need 
> 2D variable array for entry points which is not supported. However, easy 
> workaround for that is to flatten 2D array to 1D and either have another slice 
> control field which would tell first entry point index for convenience or let 
> driver calculate it by adding up all num_entry_point_offsets of previous 
> slices.
> 
> Hans, what do you think?

If I would support 2D variable array sizes, wouldn't that be more elegant?

The current implementation doesn't support that, but as the commit log for
patch 1/17 says:

"Currently dynamically sized arrays are limited to one dimensional arrays,
but that might change in the future if there is a need for it."

Let me know if you agree, and I'll try to implement this. It's been a while
since I last looked at this, so I'm not sure how much work it is, but it is
probably worth a shot.

Regards,

	Hans

> 
> Note, it seems that H265 decoding on Cedrus still works without entry points, 
> so this problem can be solved later. I'm not sure what we lose with that but 
> it was suggested that this could influence speed or error resilience or both. 
> However, I think we're close to solve it, so I'd like to do that now.
> 
> Best regards,
> Jernej
> 
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard at collabora.com>
>> ---
>>  .../userspace-api/media/v4l/ext-ctrls-codec.rst       | 11 +++++++++++
>>  drivers/media/v4l2-core/v4l2-ctrls-defs.c             |  5 +++++
>>  include/media/hevc-ctrls.h                            |  5 ++++-
>>  3 files changed, 20 insertions(+), 1 deletion(-)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
> Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index 0796b1563daa..05228e280f66 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -3010,6 +3010,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>      * - __u32
>>        - ``data_bit_offset``
>>        - Offset (in bits) to the video data in the current slice data.
>> +    * - __u32
>> +      - ``num_entry_point_offsets``
>> +      - Specifies the number of entry point offset syntax elements in the 
> slice header.
>>      * - __u8
>>        - ``nal_unit_type``
>>        - Specifies the coding type of the slice (B, P or I).
>> @@ -3150,6 +3153,14 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>  
>>      \normalsize
>>  
>> +``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
>> +    Specifies entry point offsets in bytes.
>> +    This control is a dynamically sized array. The number of entry point
>> +    offsets is reported by the ``elems`` field.
>> +    This bitstream parameter is defined according to :ref:`hevc`.
>> +    They are described in section 7.4.7.1 "General slice segment header
>> +    semantics" of the specification.
>> +
>>  ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
>>      Specifies the HEVC scaling matrix parameters used for the scaling 
> process
>>      for transform coefficients.
>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-
> core/v4l2-ctrls-defs.c
>> index d594efbcbb93..e22921e7ea61 100644
>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>> @@ -1188,6 +1188,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>>  	case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS:		return 
> "HEVC Decode Parameters";
>>  	case V4L2_CID_STATELESS_HEVC_DECODE_MODE:		return 
> "HEVC Decode Mode";
>>  	case V4L2_CID_STATELESS_HEVC_START_CODE:		return 
> "HEVC Start Code";
>> +	case V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS:	return 
> "HEVC Entry Point Offsets";
>>  
>>  	/* Colorimetry controls */
>>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! 
> */
>> @@ -1518,6 +1519,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum 
> v4l2_ctrl_type *type,
>>  	case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS:
>>  		*type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
>>  		break;
>> +	case V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS:
>> +		*type = V4L2_CTRL_TYPE_U32;
>> +		*flags |= V4L2_CTRL_FLAG_DYNAMIC_ARRAY;
>> +		break;
>>  	case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR:
>>  		*type = V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR;
>>  		break;
>> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
>> index a3c829ef531a..1319cb99ae3f 100644
>> --- a/include/media/hevc-ctrls.h
>> +++ b/include/media/hevc-ctrls.h
>> @@ -20,6 +20,7 @@
>>  #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS	(V4L2_CID_CODEC_BASE 
> + 1012)
>>  #define V4L2_CID_STATELESS_HEVC_DECODE_MODE	(V4L2_CID_CODEC_BASE + 1015)
>>  #define V4L2_CID_STATELESS_HEVC_START_CODE	(V4L2_CID_CODEC_BASE + 1016)
>> +#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (V4L2_CID_CODEC_BASE + 
> 1017)
>>  
>>  /* enum v4l2_ctrl_type type values */
>>  #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
>> @@ -318,6 +319,8 @@ struct v4l2_hevc_pred_weight_table {
>>   *
>>   * @bit_size: size (in bits) of the current slice data
>>   * @data_bit_offset: offset (in bits) to the video data in the current slice 
> data
>> + * @num_entry_point_offsets: specifies the number of entry point offset syntax
>> + *			     elements in the slice header.
>>   * @nal_unit_type: specifies the coding type of the slice (B, P or I)
>>   * @nuh_temporal_id_plus1: minus 1 specifies a temporal identifier for the 
> NAL unit
>>   * @slice_type: see V4L2_HEVC_SLICE_TYPE_{}
>> @@ -360,7 +363,7 @@ struct v4l2_hevc_pred_weight_table {
>>  struct v4l2_ctrl_hevc_slice_params {
>>  	__u32	bit_size;
>>  	__u32	data_bit_offset;
>> -
>> +	__u32	num_entry_point_offsets;
>>  	/* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>>  	__u8	nal_unit_type;
>>  	__u8	nuh_temporal_id_plus1;
>> -- 
>> 2.32.0
>>
>>
> 
> 