Merge pull request #6271 from jernejsk/vp9-10bit

Allwinner: VP9 10-bit support
This commit is contained in:
CvH 2022-03-04 11:56:31 +01:00 committed by GitHub
commit 60d175010d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 1033 additions and 16 deletions

View File

@ -1,7 +1,7 @@
From 904af26693095364851bbc6c6557fca9b3437b69 Mon Sep 17 00:00:00 2001 From 904af26693095364851bbc6c6557fca9b3437b69 Mon Sep 17 00:00:00 2001
From: Jonas Karlman <jonas@kwiboo.se> From: Jonas Karlman <jonas@kwiboo.se>
Date: Mon, 3 Dec 2018 23:48:04 +0100 Date: Mon, 3 Dec 2018 23:48:04 +0100
Subject: [PATCH 01/16] avutil: add av_buffer_pool_flush() Subject: [PATCH 01/17] avutil: add av_buffer_pool_flush()
Used by V4L2 request API hwaccel Used by V4L2 request API hwaccel
@ -55,7 +55,7 @@ index 241a80ed6709..f41363faf1dc 100644
From ec84dc22e99f544e4de7c43e7f8ef9ab7ee8e19b Mon Sep 17 00:00:00 2001 From ec84dc22e99f544e4de7c43e7f8ef9ab7ee8e19b Mon Sep 17 00:00:00 2001
From: Jonas Karlman <jonas@kwiboo.se> From: Jonas Karlman <jonas@kwiboo.se>
Date: Sat, 15 Dec 2018 22:32:16 +0100 Date: Sat, 15 Dec 2018 22:32:16 +0100
Subject: [PATCH 02/16] Add common V4L2 request API code Subject: [PATCH 02/17] Add common V4L2 request API code
Signed-off-by: Jonas Karlman <jonas@kwiboo.se> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
--- ---
@ -1242,7 +1242,7 @@ index 000000000000..58d2aa70af80
From e432d3151f4c5507e40fb1fe8b3c3fb7c7a0a08c Mon Sep 17 00:00:00 2001 From e432d3151f4c5507e40fb1fe8b3c3fb7c7a0a08c Mon Sep 17 00:00:00 2001
From: Ezequiel Garcia <ezequiel@collabora.com> From: Ezequiel Garcia <ezequiel@collabora.com>
Date: Wed, 20 Feb 2019 11:18:00 -0300 Date: Wed, 20 Feb 2019 11:18:00 -0300
Subject: [PATCH 03/16] h264dec: add idr_pic_id to slice context Subject: [PATCH 03/17] h264dec: add idr_pic_id to slice context
Used by V4L2 request API h264 hwaccel Used by V4L2 request API h264 hwaccel
@ -1282,7 +1282,7 @@ index b7b19ba4f162..0698ab95ba51 100644
From 84564d13ec0ec40f408622ff6b0d900723bbab5b Mon Sep 17 00:00:00 2001 From 84564d13ec0ec40f408622ff6b0d900723bbab5b Mon Sep 17 00:00:00 2001
From: Boris Brezillon <boris.brezillon@collabora.com> From: Boris Brezillon <boris.brezillon@collabora.com>
Date: Wed, 22 May 2019 14:44:22 +0200 Date: Wed, 22 May 2019 14:44:22 +0200
Subject: [PATCH 04/16] h264dec: add ref_pic_marking and pic_order_cnt bit_size Subject: [PATCH 04/17] h264dec: add ref_pic_marking and pic_order_cnt bit_size
to slice context to slice context
Used by V4L2 request API h264 hwaccel Used by V4L2 request API h264 hwaccel
@ -1359,7 +1359,7 @@ index 0698ab95ba51..2b39e82c3b03 100644
From 5a8628cf6368fe18457d02bf551d5935609efab5 Mon Sep 17 00:00:00 2001 From 5a8628cf6368fe18457d02bf551d5935609efab5 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@siol.net> From: Jernej Skrabec <jernej.skrabec@siol.net>
Date: Sat, 15 Dec 2018 22:32:16 +0100 Date: Sat, 15 Dec 2018 22:32:16 +0100
Subject: [PATCH 05/16] Add V4L2 request API h264 hwaccel Subject: [PATCH 05/17] Add V4L2 request API h264 hwaccel
Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net> Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net>
Signed-off-by: Jonas Karlman <jonas@kwiboo.se> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
@ -1920,7 +1920,7 @@ index 000000000000..88da8f0a2db0
From 02b8fb17c2a019463dcab4baa1cb0bec63353183 Mon Sep 17 00:00:00 2001 From 02b8fb17c2a019463dcab4baa1cb0bec63353183 Mon Sep 17 00:00:00 2001
From: Jonas Karlman <jonas@kwiboo.se> From: Jonas Karlman <jonas@kwiboo.se>
Date: Sat, 15 Dec 2018 22:32:16 +0100 Date: Sat, 15 Dec 2018 22:32:16 +0100
Subject: [PATCH 06/16] Add V4L2 request API mpeg2 hwaccel Subject: [PATCH 06/17] Add V4L2 request API mpeg2 hwaccel
Signed-off-by: Jonas Karlman <jonas@kwiboo.se> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
--- ---
@ -2170,7 +2170,7 @@ index 000000000000..84d53209c79d
From 1cd61e5730acc12c39c964bcf13c73a54203a390 Mon Sep 17 00:00:00 2001 From 1cd61e5730acc12c39c964bcf13c73a54203a390 Mon Sep 17 00:00:00 2001
From: Boris Brezillon <boris.brezillon@collabora.com> From: Boris Brezillon <boris.brezillon@collabora.com>
Date: Wed, 22 May 2019 14:46:58 +0200 Date: Wed, 22 May 2019 14:46:58 +0200
Subject: [PATCH 07/16] Add V4L2 request API vp8 hwaccel Subject: [PATCH 07/17] Add V4L2 request API vp8 hwaccel
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com> Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
@ -2443,7 +2443,7 @@ index d16e7b6aa34e..8ee768d875ad 100644
From 3c09465f9dee3355b0ef68140b86e9e66797b5fc Mon Sep 17 00:00:00 2001 From 3c09465f9dee3355b0ef68140b86e9e66797b5fc Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@siol.net> From: Jernej Skrabec <jernej.skrabec@siol.net>
Date: Sat, 15 Dec 2018 22:32:16 +0100 Date: Sat, 15 Dec 2018 22:32:16 +0100
Subject: [PATCH 08/16] Add V4L2 request API hevc hwaccel Subject: [PATCH 08/17] Add V4L2 request API hevc hwaccel
Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net> Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net>
Signed-off-by: Jonas Karlman <jonas@kwiboo.se> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
@ -3157,7 +3157,7 @@ index 000000000000..3a9d564ce20b
From 140874637c7d3d99b69e6b30f0ad32b07d4e43e8 Mon Sep 17 00:00:00 2001 From 140874637c7d3d99b69e6b30f0ad32b07d4e43e8 Mon Sep 17 00:00:00 2001
From: Boris Brezillon <boris.brezillon@collabora.com> From: Boris Brezillon <boris.brezillon@collabora.com>
Date: Thu, 12 Dec 2019 16:13:55 +0100 Date: Thu, 12 Dec 2019 16:13:55 +0100
Subject: [PATCH 09/16] Add V4L2 request API VP9 hwaccel Subject: [PATCH 09/17] Add V4L2 request API VP9 hwaccel
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
@ -3885,7 +3885,7 @@ index 54726df742f9..fee3568736f7 100644
From ae118fbbef1e287c5101d7ceaa164727d825c4c2 Mon Sep 17 00:00:00 2001 From ae118fbbef1e287c5101d7ceaa164727d825c4c2 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@siol.net> From: Jernej Skrabec <jernej.skrabec@siol.net>
Date: Thu, 14 Feb 2019 23:20:05 +0100 Date: Thu, 14 Feb 2019 23:20:05 +0100
Subject: [PATCH 10/16] Add and use private linux v5.16 headers for V4L2 Subject: [PATCH 10/17] Add and use private linux v5.16 headers for V4L2
request API ctrls request API ctrls
Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net> Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net>
@ -4184,7 +4184,7 @@ index 3a9d564ce20b..f645c538c25c 100644
From afe6727d8c300c77a72549b4e094910f09242a42 Mon Sep 17 00:00:00 2001 From afe6727d8c300c77a72549b4e094910f09242a42 Mon Sep 17 00:00:00 2001
From: Jonas Karlman <jonas@kwiboo.se> From: Jonas Karlman <jonas@kwiboo.se>
Date: Mon, 29 Apr 2019 22:08:59 +0000 Date: Mon, 29 Apr 2019 22:08:59 +0000
Subject: [PATCH 11/16] HACK: hwcontext_drm: do not require drm device Subject: [PATCH 11/17] HACK: hwcontext_drm: do not require drm device
Signed-off-by: Jonas Karlman <jonas@kwiboo.se> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
--- ---
@ -4211,7 +4211,7 @@ index 7a9fdbd263d4..6297d1f9b613 100644
From bcad5ac25923b3d2395edceca821129c00b57b9e Mon Sep 17 00:00:00 2001 From bcad5ac25923b3d2395edceca821129c00b57b9e Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@siol.net> From: Jernej Skrabec <jernej.skrabec@siol.net>
Date: Sat, 15 Dec 2018 22:32:16 +0100 Date: Sat, 15 Dec 2018 22:32:16 +0100
Subject: [PATCH 12/16] WIP: hevc entry point offsets Subject: [PATCH 12/17] WIP: hevc entry point offsets
Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net> Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net>
--- ---
@ -4258,7 +4258,7 @@ index f645c538c25c..601202a77d7a 100644
From 694b79d528295ae4d073c5972d35173116ff2b69 Mon Sep 17 00:00:00 2001 From 694b79d528295ae4d073c5972d35173116ff2b69 Mon Sep 17 00:00:00 2001
From: Jonas Karlman <jonas@kwiboo.se> From: Jonas Karlman <jonas@kwiboo.se>
Date: Fri, 15 May 2020 16:54:05 +0000 Date: Fri, 15 May 2020 16:54:05 +0000
Subject: [PATCH 13/16] WIP: add NV15 and NV20 support Subject: [PATCH 13/17] WIP: add NV15 and NV20 support
Signed-off-by: Jonas Karlman <jonas@kwiboo.se> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
--- ---
@ -4345,7 +4345,7 @@ index 5234b5049b0d..0b294feff2eb 100644
From dd29c8e4b504f12be27db8e188ba5bbaade7559f Mon Sep 17 00:00:00 2001 From dd29c8e4b504f12be27db8e188ba5bbaade7559f Mon Sep 17 00:00:00 2001
From: Jonas Karlman <jonas@kwiboo.se> From: Jonas Karlman <jonas@kwiboo.se>
Date: Mon, 27 Jul 2020 23:15:45 +0000 Date: Mon, 27 Jul 2020 23:15:45 +0000
Subject: [PATCH 14/16] HACK: define drm NV15 and NV20 format Subject: [PATCH 14/17] HACK: define drm NV15 and NV20 format
--- ---
libavcodec/v4l2_request.c | 8 ++++++++ libavcodec/v4l2_request.c | 8 ++++++++
@ -4374,7 +4374,7 @@ index 0b294feff2eb..a8f0ee79eeef 100644
From 0d56f3e7b04a5d574d38dd885000a6eb19eff903 Mon Sep 17 00:00:00 2001 From 0d56f3e7b04a5d574d38dd885000a6eb19eff903 Mon Sep 17 00:00:00 2001
From: Jonas Karlman <jonas@kwiboo.se> From: Jonas Karlman <jonas@kwiboo.se>
Date: Wed, 13 May 2020 22:51:21 +0000 Date: Wed, 13 May 2020 22:51:21 +0000
Subject: [PATCH 15/16] WIP: hevc rkvdec fields Subject: [PATCH 15/17] WIP: hevc rkvdec fields
Signed-off-by: Jonas Karlman <jonas@kwiboo.se> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
--- ---
@ -4490,7 +4490,7 @@ index 601202a77d7a..be7838244447 100644
From eb9a3213d56d9f6a177f16e907f57e0063b23244 Mon Sep 17 00:00:00 2001 From eb9a3213d56d9f6a177f16e907f57e0063b23244 Mon Sep 17 00:00:00 2001
From: Alex Bee <knaerzche@gmail.com> From: Alex Bee <knaerzche@gmail.com>
Date: Sun, 19 Sep 2021 13:10:55 +0200 Date: Sun, 19 Sep 2021 13:10:55 +0200
Subject: [PATCH 16/16] v4l2_request: validate supported framesizes Subject: [PATCH 16/17] v4l2_request: validate supported framesizes
Signed-off-by: Alex Bee <knaerzche@gmail.com> Signed-off-by: Alex Bee <knaerzche@gmail.com>
--- ---
@ -4553,3 +4553,181 @@ index a8f0ee79eeef..2fbe16634160 100644
fmtdesc.index++; fmtdesc.index++;
} }
From 5607e918e68f9eb3280914b04e93ebff766e9e02 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@gmail.com>
Date: Sun, 27 Feb 2022 18:54:21 +0100
Subject: [PATCH 17/17] Improve VP9 decoding
---
libavcodec/v4l2_request_vp9.c | 110 +++++++++++++++++++---------------
1 file changed, 62 insertions(+), 48 deletions(-)
diff --git a/libavcodec/v4l2_request_vp9.c b/libavcodec/v4l2_request_vp9.c
index 9b95c76cdb82..ec0300f66dbe 100644
--- a/libavcodec/v4l2_request_vp9.c
+++ b/libavcodec/v4l2_request_vp9.c
@@ -22,78 +22,62 @@
typedef struct V4L2RequestControlsVP9 {
struct v4l2_ctrl_vp9_frame decode_params;
+ struct v4l2_ctrl_vp9_compressed_hdr chp;
} V4L2RequestControlsVP9;
-static int v4l2_request_vp9_set_frame_ctx(AVCodecContext *avctx)
+static void v4l2_request_vp9_set_frame_ctx(AVCodecContext *avctx)
{
VP9Context *s = avctx->priv_data;
- struct v4l2_ctrl_vp9_compressed_hdr chp;
- struct v4l2_ext_control control[] = {
- {
- .id = V4L2_CID_STATELESS_VP9_COMPRESSED_HDR,
- .ptr = &chp,
- .size = sizeof(chp),
- },
- };
+ const VP9Frame *f = &s->s.frames[CUR_FRAME];
+ V4L2RequestControlsVP9 *controls = f->hwaccel_picture_private;
+ struct v4l2_ctrl_vp9_compressed_hdr *chp = &controls->chp;
- memset(&chp, 0, sizeof(chp));
+ memset(chp, 0, sizeof(&chp));
- chp.tx_mode = s->s.h.txfmmode;
- memcpy(chp.tx8, s->prob_raw.p.tx8p, sizeof(s->prob_raw.p.tx8p));
- memcpy(chp.tx16, s->prob_raw.p.tx16p, sizeof(s->prob_raw.p.tx16p));
- memcpy(chp.tx32, s->prob_raw.p.tx32p, sizeof(s->prob_raw.p.tx32p));
+ chp->tx_mode = s->s.h.txfmmode;
+ memcpy(chp->tx8, s->prob_raw.p.tx8p, sizeof(s->prob_raw.p.tx8p));
+ memcpy(chp->tx16, s->prob_raw.p.tx16p, sizeof(s->prob_raw.p.tx16p));
+ memcpy(chp->tx32, s->prob_raw.p.tx32p, sizeof(s->prob_raw.p.tx32p));
for (unsigned i = 0; i < 4; i++) {
for (unsigned j = 0; j < 2; j++) {
for (unsigned k = 0; k < 2; k++) {
for (unsigned l = 0; l < 6; l++) {
for (unsigned m = 0; m < 6; m++) {
- memcpy(chp.coef[i][j][k][l][m], s->prob_raw.coef[i][j][k][l][m], sizeof(chp.coef[0][0][0][0][0]));
+ memcpy(chp->coef[i][j][k][l][m], s->prob_raw.coef[i][j][k][l][m], sizeof(chp->coef[0][0][0][0][0]));
}
}
}
}
}
- memcpy(chp.skip, s->prob_raw.p.skip, sizeof(s->prob_raw.p.skip));
- memcpy(chp.inter_mode, s->prob_raw.p.mv_mode, sizeof(s->prob_raw.p.mv_mode));
- memcpy(chp.interp_filter, s->prob_raw.p.filter, sizeof(s->prob_raw.p.filter));
- memcpy(chp.is_inter, s->prob_raw.p.intra, sizeof(s->prob_raw.p.intra));
- memcpy(chp.comp_mode, s->prob_raw.p.comp, sizeof(s->prob_raw.p.comp));
- memcpy(chp.single_ref, s->prob_raw.p.single_ref, sizeof(s->prob_raw.p.single_ref));
- memcpy(chp.comp_ref, s->prob_raw.p.comp_ref, sizeof(s->prob_raw.p.comp_ref));
- memcpy(chp.y_mode, s->prob_raw.p.y_mode, sizeof(s->prob_raw.p.y_mode));
+ memcpy(chp->skip, s->prob_raw.p.skip, sizeof(s->prob_raw.p.skip));
+ memcpy(chp->inter_mode, s->prob_raw.p.mv_mode, sizeof(s->prob_raw.p.mv_mode));
+ memcpy(chp->interp_filter, s->prob_raw.p.filter, sizeof(s->prob_raw.p.filter));
+ memcpy(chp->is_inter, s->prob_raw.p.intra, sizeof(s->prob_raw.p.intra));
+ memcpy(chp->comp_mode, s->prob_raw.p.comp, sizeof(s->prob_raw.p.comp));
+ memcpy(chp->single_ref, s->prob_raw.p.single_ref, sizeof(s->prob_raw.p.single_ref));
+ memcpy(chp->comp_ref, s->prob_raw.p.comp_ref, sizeof(s->prob_raw.p.comp_ref));
+ memcpy(chp->y_mode, s->prob_raw.p.y_mode, sizeof(s->prob_raw.p.y_mode));
for (unsigned i = 0; i < 10; i++)
- memcpy(chp.uv_mode[i], s->prob.p.uv_mode[i], sizeof(s->prob.p.uv_mode[0]));
+ memcpy(chp->uv_mode[i], s->prob.p.uv_mode[i], sizeof(s->prob.p.uv_mode[0]));
for (unsigned i = 0; i < 4; i++)
- memcpy(chp.partition[i * 4], s->prob_raw.p.partition[i], sizeof(s->prob_raw.p.partition[0]));
- memcpy(chp.mv.joint, s->prob_raw.p.mv_joint, sizeof(s->prob_raw.p.mv_joint));
+ memcpy(chp->partition[i * 4], s->prob_raw.p.partition[i], sizeof(s->prob_raw.p.partition[0]));
+ memcpy(chp->mv.joint, s->prob_raw.p.mv_joint, sizeof(s->prob_raw.p.mv_joint));
for (unsigned i = 0; i < 2; i++) {
- chp.mv.sign[i] = s->prob_raw.p.mv_comp[i].sign;
- memcpy(chp.mv.classes[i], s->prob_raw.p.mv_comp[i].classes, sizeof(s->prob_raw.p.mv_comp[0].classes));
- chp.mv.class0_bit[i] = s->prob_raw.p.mv_comp[i].class0;
- memcpy(chp.mv.bits[i], s->prob_raw.p.mv_comp[i].bits, sizeof(s->prob_raw.p.mv_comp[0].bits));
- memcpy(chp.mv.class0_fr[i], s->prob_raw.p.mv_comp[i].class0_fp, sizeof(s->prob_raw.p.mv_comp[0].class0_fp));
- memcpy(chp.mv.fr[i], s->prob_raw.p.mv_comp[i].fp, sizeof(s->prob_raw.p.mv_comp[0].fp));
- chp.mv.class0_hp[i] = s->prob_raw.p.mv_comp[i].class0_hp;
- chp.mv.hp[i] = s->prob_raw.p.mv_comp[i].hp;
+ chp->mv.sign[i] = s->prob_raw.p.mv_comp[i].sign;
+ memcpy(chp->mv.classes[i], s->prob_raw.p.mv_comp[i].classes, sizeof(s->prob_raw.p.mv_comp[0].classes));
+ chp->mv.class0_bit[i] = s->prob_raw.p.mv_comp[i].class0;
+ memcpy(chp->mv.bits[i], s->prob_raw.p.mv_comp[i].bits, sizeof(s->prob_raw.p.mv_comp[0].bits));
+ memcpy(chp->mv.class0_fr[i], s->prob_raw.p.mv_comp[i].class0_fp, sizeof(s->prob_raw.p.mv_comp[0].class0_fp));
+ memcpy(chp->mv.fr[i], s->prob_raw.p.mv_comp[i].fp, sizeof(s->prob_raw.p.mv_comp[0].fp));
+ chp->mv.class0_hp[i] = s->prob_raw.p.mv_comp[i].class0_hp;
+ chp->mv.hp[i] = s->prob_raw.p.mv_comp[i].hp;
}
-
- return ff_v4l2_request_set_controls(avctx, control, FF_ARRAY_ELEMS(control));
}
-static int v4l2_request_vp9_start_frame(AVCodecContext *avctx,
- av_unused const uint8_t *buffer,
- av_unused uint32_t size)
+static void fill_frame(struct v4l2_ctrl_vp9_frame *dec_params, AVCodecContext *avctx)
{
const VP9Context *s = avctx->priv_data;
- const VP9Frame *f = &s->s.frames[CUR_FRAME];
- V4L2RequestControlsVP9 *controls = f->hwaccel_picture_private;
- struct v4l2_ctrl_vp9_frame *dec_params = &controls->decode_params;
const ThreadFrame *ref;
- int ret;
-
- ret = v4l2_request_vp9_set_frame_ctx(avctx);
- if (ret)
- return ret;
memset(dec_params, 0, sizeof(*dec_params));
@@ -208,6 +192,19 @@ static int v4l2_request_vp9_start_frame(AVCodecContext *avctx,
if (s->s.h.segmentation.feat[i].skip_enabled)
dec_params->seg.feature_enabled[i] |= 1 << V4L2_VP9_SEG_LVL_SKIP;
}
+}
+
+static int v4l2_request_vp9_start_frame(AVCodecContext *avctx,
+ av_unused const uint8_t *buffer,
+ av_unused uint32_t size)
+{
+ const VP9Context *s = avctx->priv_data;
+ const VP9Frame *f = &s->s.frames[CUR_FRAME];
+ V4L2RequestControlsVP9 *controls = f->hwaccel_picture_private;
+
+ v4l2_request_vp9_set_frame_ctx(avctx);
+
+ fill_frame(&controls->decode_params, avctx);
return ff_v4l2_request_reset_frame(avctx, f->tf.f);
}
@@ -233,6 +230,11 @@ static int v4l2_request_vp9_end_frame(AVCodecContext *avctx)
.ptr = &controls->decode_params,
.size = sizeof(controls->decode_params),
},
+ {
+ .id = V4L2_CID_STATELESS_VP9_COMPRESSED_HDR,
+ .ptr = &controls->chp,
+ .size = sizeof(controls->chp),
+ },
};
ret = ff_v4l2_request_decode_frame(avctx, f->tf.f, control, FF_ARRAY_ELEMS(control));
@@ -247,8 +249,20 @@ static int v4l2_request_vp9_end_frame(AVCodecContext *avctx)
static int v4l2_request_vp9_init(AVCodecContext *avctx)
{
+ struct v4l2_ctrl_vp9_frame frame;
+
+ struct v4l2_ext_control control[] = {
+ {
+ .id = V4L2_CID_STATELESS_VP9_FRAME,
+ .ptr = &frame,
+ .size = sizeof(frame),
+ },
+ };
+
+ fill_frame(&frame, avctx);
+
// TODO: check V4L2_CID_MPEG_VIDEO_VP9_PROFILE
- return ff_v4l2_request_init(avctx, V4L2_PIX_FMT_VP9_FRAME, 3 * 1024 * 1024, NULL, 0);
+ return ff_v4l2_request_init(avctx, V4L2_PIX_FMT_VP9_FRAME, 3 * 1024 * 1024, control, FF_ARRAY_ELEMS(control));
}
const AVHWAccel ff_vp9_v4l2request_hwaccel = {

View File

@ -0,0 +1,301 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Date: Wed, 8 Dec 2021 17:44:18 +0100
Subject: [PATCH] media: hantro: Fix G2/HEVC negotiated pixelformat
G2/HEVC is broken because driver capture queue pixelformat ioctl G_FMT
returns VT12 while G2/HEVC always generate NV12 frames:
video1: VIDIOC_S_FMT: type=vid-out-mplane, width=2560, height=1600, format=S265 little-endian (0x35363253), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0
plane 0: bytesperline=0 sizeimage=6144000
video1: VIDIOC_S_EXT_CTRLS: which=0x0, count=1, error_idx=0, request_fd=0, name=HEVC Sequence Parameter Set, id/size=0x990cf0/32
video1: VIDIOC_G_FMT: type=vid-cap-mplane, width=2560, height=1600, format=VT12 little-endian (0x32315456), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0
plane 0: bytesperline=2560 sizeimage=6144000
video1: VIDIOC_ENUM_FMT: index=0, type=vid-cap-mplane, flags=0x0, pixelformat=NV12 little-endian (0x3231564e), mbus_code=0x0000, description='Y/CbCr 4:2:0'
video1: VIDIOC_ENUM_FMT: error -22: index=1, type=vid-cap-mplane, flags=0x0, pixelformat=.... little-endian (0x00000000), mbus_code=0x0000, description=''
video1: VIDIOC_G_FMT: type=vid-cap-mplane, width=2560, height=1600, format=VT12 little-endian (0x32315456), field=none, colorspace=0, num_planes=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0
Use the postprocessor functions introduced by Hantro G2/VP9 codec series
to fix the issue and remove duplicated buffer management.
This allow Hantro G2/HEVC to produce NV12_4L4 and NV12.
Fluster scores are 77/147 for HEVC and 129/303 for VP9 (no regression).
Beauty, Jockey and ShakeNDry bitstreams from UVG (http://ultravideo.fi/)
set have also been tested.
Fixes: 53a3e71095c5 ("media: hantro: Simplify postprocessor")
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
.../staging/media/hantro/hantro_g2_hevc_dec.c | 25 +++---
drivers/staging/media/hantro/hantro_hevc.c | 79 +++----------------
drivers/staging/media/hantro/hantro_hw.h | 11 +++
.../staging/media/hantro/hantro_postproc.c | 3 +
drivers/staging/media/hantro/hantro_v4l2.c | 19 ++---
5 files changed, 41 insertions(+), 96 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
index f62608b0b408..99d8ea7543da 100644
--- a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -354,6 +354,8 @@ static int set_ref(struct hantro_ctx *ctx)
const struct v4l2_hevc_dpb_entry *dpb = decode_params->dpb;
dma_addr_t luma_addr, chroma_addr, mv_addr = 0;
struct hantro_dev *vpu = ctx->dev;
+ struct vb2_v4l2_buffer *vb2_dst;
+ struct hantro_decoded_buffer *dst;
size_t cr_offset = hantro_hevc_chroma_offset(sps);
size_t mv_offset = hantro_hevc_motion_vectors_offset(sps);
u32 max_ref_frames;
@@ -439,10 +441,15 @@ static int set_ref(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_REF_MV_ADDR(i), mv_addr);
}
- luma_addr = hantro_hevc_get_ref_buf(ctx, decode_params->pic_order_cnt_val);
+ vb2_dst = hantro_get_dst_buf(ctx);
+ dst = vb2_to_hantro_decoded_buf(&vb2_dst->vb2_buf);
+ luma_addr = hantro_get_dec_buf_addr(ctx, &dst->base.vb.vb2_buf);
if (!luma_addr)
return -ENOMEM;
+ if (hantro_hevc_add_ref_buf(ctx, decode_params->pic_order_cnt_val, luma_addr))
+ return -EINVAL;
+
chroma_addr = luma_addr + cr_offset;
mv_addr = luma_addr + mv_offset;
@@ -469,16 +476,12 @@ static int set_ref(struct hantro_ctx *ctx)
static void set_buffers(struct hantro_ctx *ctx)
{
- struct vb2_v4l2_buffer *src_buf, *dst_buf;
+ struct vb2_v4l2_buffer *src_buf;
struct hantro_dev *vpu = ctx->dev;
- const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
- const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
- size_t cr_offset = hantro_hevc_chroma_offset(sps);
- dma_addr_t src_dma, dst_dma;
+ dma_addr_t src_dma;
u32 src_len, src_buf_len;
src_buf = hantro_get_src_buf(ctx);
- dst_buf = hantro_get_dst_buf(ctx);
/* Source (stream) buffer. */
src_dma = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
@@ -491,11 +494,6 @@ static void set_buffers(struct hantro_ctx *ctx)
hantro_reg_write(vpu, &g2_strm_start_offset, 0);
hantro_reg_write(vpu, &g2_write_mvs_e, 1);
- /* Destination (decoded frame) buffer. */
- dst_dma = hantro_get_dec_buf_addr(ctx, &dst_buf->vb2_buf);
-
- hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
- hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma + cr_offset);
hantro_write_addr(vpu, G2_TILE_SIZES_ADDR, ctx->hevc_dec.tile_sizes.dma);
hantro_write_addr(vpu, G2_TILE_FILTER_ADDR, ctx->hevc_dec.tile_filter.dma);
hantro_write_addr(vpu, G2_TILE_SAO_ADDR, ctx->hevc_dec.tile_sao.dma);
@@ -588,9 +586,6 @@ int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
/* Don't compress buffers */
hantro_reg_write(vpu, &g2_ref_compress_bypass, 1);
- /* use NV12 as output format */
- hantro_reg_write(vpu, &g2_out_rs_e, 1);
-
/* Bus width and max burst */
hantro_reg_write(vpu, &g2_buswidth, BUS_WIDTH_128);
hantro_reg_write(vpu, &g2_max_burst, 16);
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index ee03123e7704..b49a41d7ae91 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -44,47 +44,6 @@ size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps)
return ALIGN((cr_offset * 3) / 2, G2_ALIGN);
}
-static size_t hantro_hevc_mv_size(const struct v4l2_ctrl_hevc_sps *sps)
-{
- u32 min_cb_log2_size_y = sps->log2_min_luma_coding_block_size_minus3 + 3;
- u32 ctb_log2_size_y = min_cb_log2_size_y + sps->log2_diff_max_min_luma_coding_block_size;
- u32 pic_width_in_ctbs_y = (sps->pic_width_in_luma_samples + (1 << ctb_log2_size_y) - 1)
- >> ctb_log2_size_y;
- u32 pic_height_in_ctbs_y = (sps->pic_height_in_luma_samples + (1 << ctb_log2_size_y) - 1)
- >> ctb_log2_size_y;
- size_t mv_size;
-
- mv_size = pic_width_in_ctbs_y * pic_height_in_ctbs_y *
- (1 << (2 * (ctb_log2_size_y - 4))) * 16;
-
- vpu_debug(4, "%dx%d (CTBs) %zu MV bytes\n",
- pic_width_in_ctbs_y, pic_height_in_ctbs_y, mv_size);
-
- return mv_size;
-}
-
-static size_t hantro_hevc_ref_size(struct hantro_ctx *ctx)
-{
- const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
- const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
-
- return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
-}
-
-static void hantro_hevc_ref_free(struct hantro_ctx *ctx)
-{
- struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
- struct hantro_dev *vpu = ctx->dev;
- int i;
-
- for (i = 0; i < NUM_REF_PICTURES; i++) {
- if (hevc_dec->ref_bufs[i].cpu)
- dma_free_coherent(vpu->dev, hevc_dec->ref_bufs[i].size,
- hevc_dec->ref_bufs[i].cpu,
- hevc_dec->ref_bufs[i].dma);
- }
-}
-
static void hantro_hevc_ref_init(struct hantro_ctx *ctx)
{
struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
@@ -108,37 +67,25 @@ dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx,
}
}
- /* Allocate a new reference buffer */
+ return 0;
+}
+
+int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx, int poc, dma_addr_t addr)
+{
+ struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+ int i;
+
+ /* Add a new reference buffer */
for (i = 0; i < NUM_REF_PICTURES; i++) {
if (hevc_dec->ref_bufs_poc[i] == UNUSED_REF) {
- if (!hevc_dec->ref_bufs[i].cpu) {
- struct hantro_dev *vpu = ctx->dev;
-
- /*
- * Allocate the space needed for the raw data +
- * motion vector data. Optimizations could be to
- * allocate raw data in non coherent memory and only
- * clear the motion vector data.
- */
- hevc_dec->ref_bufs[i].cpu =
- dma_alloc_coherent(vpu->dev,
- hantro_hevc_ref_size(ctx),
- &hevc_dec->ref_bufs[i].dma,
- GFP_KERNEL);
- if (!hevc_dec->ref_bufs[i].cpu)
- return 0;
-
- hevc_dec->ref_bufs[i].size = hantro_hevc_ref_size(ctx);
- }
hevc_dec->ref_bufs_used |= 1 << i;
- memset(hevc_dec->ref_bufs[i].cpu, 0, hantro_hevc_ref_size(ctx));
hevc_dec->ref_bufs_poc[i] = poc;
-
- return hevc_dec->ref_bufs[i].dma;
+ hevc_dec->ref_bufs[i].dma = addr;
+ return 0;
}
}
- return 0;
+ return -EINVAL;
}
void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx)
@@ -314,8 +261,6 @@ void hantro_hevc_dec_exit(struct hantro_ctx *ctx)
hevc_dec->tile_bsd.cpu,
hevc_dec->tile_bsd.dma);
hevc_dec->tile_bsd.cpu = NULL;
-
- hantro_hevc_ref_free(ctx);
}
int hantro_hevc_dec_init(struct hantro_ctx *ctx)
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index cff817ca8d22..a018748fc4bf 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -346,6 +346,7 @@ void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
int hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
+int hantro_hevc_add_ref_buf(struct hantro_ctx *ctx, int poc, dma_addr_t addr);
void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps);
size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps);
@@ -395,6 +396,16 @@ hantro_h264_mv_size(unsigned int width, unsigned int height)
return 64 * MB_WIDTH(width) * MB_WIDTH(height) + 32;
}
+static inline size_t
+hantro_hevc_mv_size(unsigned int width, unsigned int height)
+{
+ /*
+ * A CTB can be 64x64, 32x32 or 16x16.
+ * Allocated memory for the "worse" case: 16x16
+ */
+ return width * height / 16;
+}
+
int hantro_g1_mpeg2_dec_run(struct hantro_ctx *ctx);
int rockchip_vpu2_mpeg2_dec_run(struct hantro_ctx *ctx);
void hantro_mpeg2_dec_copy_qtable(u8 *qtable,
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index a7774ad4c445..248abe5423f0 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -146,6 +146,9 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
ctx->dst_fmt.height);
+ else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE)
+ buf_size += hantro_hevc_mv_size(ctx->dst_fmt.width,
+ ctx->dst_fmt.height);
for (i = 0; i < num_buffers; ++i) {
struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c
index c319f0e5fe60..e595905b3bd7 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.c
+++ b/drivers/staging/media/hantro/hantro_v4l2.c
@@ -148,20 +148,6 @@ static int vidioc_enum_fmt(struct file *file, void *priv,
unsigned int num_fmts, i, j = 0;
bool skip_mode_none;
- /*
- * The HEVC decoder on the G2 core needs a little quirk to offer NV12
- * only on the capture side. Once the post-processor logic is used,
- * we will be able to expose NV12_4L4 and NV12 as the other cases,
- * and therefore remove this quirk.
- */
- if (capture && ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE) {
- if (f->index == 0) {
- f->pixelformat = V4L2_PIX_FMT_NV12;
- return 0;
- }
- return -EINVAL;
- }
-
/*
* When dealing with an encoder:
* - on the capture side we want to filter out all MODE_NONE formats.
@@ -302,6 +288,11 @@ static int hantro_try_fmt(const struct hantro_ctx *ctx,
pix_mp->plane_fmt[0].sizeimage +=
hantro_vp9_mv_size(pix_mp->width,
pix_mp->height);
+ else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE &&
+ !hantro_needs_postproc(ctx, fmt))
+ pix_mp->plane_fmt[0].sizeimage +=
+ hantro_hevc_mv_size(pix_mp->width,
+ pix_mp->height);
} else if (!pix_mp->plane_fmt[0].sizeimage) {
/*
* For coded formats the application can specify

View File

@ -0,0 +1,52 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Date: Sun, 27 Feb 2022 08:43:18 +0100
Subject: [PATCH] media: Add P010 tiled format
Add P010 tiled format
Signed-off-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
[rebased and updated pixel format name]
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
drivers/media/v4l2-core/v4l2-common.c | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 3 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index df34b2a283bc..1db0020e08c0 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -277,6 +277,7 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
/* Tiled YUV formats */
{ .format = V4L2_PIX_FMT_NV12_4L4, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 2 },
+ { .format = V4L2_PIX_FMT_P010_4L4, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 4, 0, 0 }, .hdiv = 2, .vdiv = 2 },
/* YUV planar formats, non contiguous variant */
{ .format = V4L2_PIX_FMT_YUV420M, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 3, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 2, .vdiv = 2 },
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 9ac557b8e146..048f326c57b9 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1302,6 +1302,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_NV12_4L4: descr = "Y/CbCr 4:2:0 (4x4 Linear)"; break;
case V4L2_PIX_FMT_NV12_16L16: descr = "Y/CbCr 4:2:0 (16x16 Linear)"; break;
case V4L2_PIX_FMT_NV12_32L32: descr = "Y/CbCr 4:2:0 (32x32 Linear)"; break;
+ case V4L2_PIX_FMT_P010_4L4: descr = "P010 tiled"; break;
case V4L2_PIX_FMT_NV12M: descr = "Y/CbCr 4:2:0 (N-C)"; break;
case V4L2_PIX_FMT_NV21M: descr = "Y/CrCb 4:2:0 (N-C)"; break;
case V4L2_PIX_FMT_NV16M: descr = "Y/CbCr 4:2:2 (N-C)"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index df8b9c486ba1..772dbadd1a24 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -628,6 +628,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_NV12_4L4 v4l2_fourcc('V', 'T', '1', '2') /* 12 Y/CbCr 4:2:0 4x4 tiles */
#define V4L2_PIX_FMT_NV12_16L16 v4l2_fourcc('H', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 16x16 tiles */
#define V4L2_PIX_FMT_NV12_32L32 v4l2_fourcc('S', 'T', '1', '2') /* 12 Y/CbCr 4:2:0 32x32 tiles */
+#define V4L2_PIX_FMT_P010_4L4 v4l2_fourcc('T', '0', '1', '0') /* 12 Y/CbCr 4:2:0 10-bit 4x4 macroblocks */
/* Tiled YUV formats, non contiguous planes */
#define V4L2_PIX_FMT_NV12MT v4l2_fourcc('T', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 64x32 tiles */

View File

@ -0,0 +1,51 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@gmail.com>
Date: Sun, 27 Feb 2022 09:01:00 +0100
Subject: [PATCH] media: Add P010 format
Add P010 format, which is commonly used for 10-bit videos.
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
drivers/media/v4l2-core/v4l2-common.c | 2 ++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 4 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 1db0020e08c0..4ede36546e9c 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -275,6 +275,8 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
{ .format = V4L2_PIX_FMT_YUV422P, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 3, .bpp = { 1, 1, 1, 0 }, .hdiv = 2, .vdiv = 1 },
{ .format = V4L2_PIX_FMT_GREY, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+ { .format = V4L2_PIX_FMT_P010, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 4, 0, 0 }, .hdiv = 2, .vdiv = 2 },
+
/* Tiled YUV formats */
{ .format = V4L2_PIX_FMT_NV12_4L4, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 2 },
{ .format = V4L2_PIX_FMT_P010_4L4, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 4, 0, 0 }, .hdiv = 2, .vdiv = 2 },
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 048f326c57b9..a8d999e23e5b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1295,6 +1295,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_M420: descr = "YUV 4:2:0 (M420)"; break;
case V4L2_PIX_FMT_NV12: descr = "Y/CbCr 4:2:0"; break;
case V4L2_PIX_FMT_NV21: descr = "Y/CrCb 4:2:0"; break;
+ case V4L2_PIX_FMT_P010: descr = "10-bit Y/CbCr 4:2:0"; break;
case V4L2_PIX_FMT_NV16: descr = "Y/CbCr 4:2:2"; break;
case V4L2_PIX_FMT_NV61: descr = "Y/CrCb 4:2:2"; break;
case V4L2_PIX_FMT_NV24: descr = "Y/CbCr 4:4:4"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 772dbadd1a24..211bc11a48cb 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -597,6 +597,7 @@ struct v4l2_pix_format {
/* two planes -- one Y, one Cr + Cb interleaved */
#define V4L2_PIX_FMT_NV12 v4l2_fourcc('N', 'V', '1', '2') /* 12 Y/CbCr 4:2:0 */
#define V4L2_PIX_FMT_NV21 v4l2_fourcc('N', 'V', '2', '1') /* 12 Y/CrCb 4:2:0 */
+#define V4L2_PIX_FMT_P010 v4l2_fourcc('P', '0', '1', '0') /* 24 Y/CbCr 4:2:0 10-bit */
#define V4L2_PIX_FMT_NV16 v4l2_fourcc('N', 'V', '1', '6') /* 16 Y/CbCr 4:2:2 */
#define V4L2_PIX_FMT_NV61 v4l2_fourcc('N', 'V', '6', '1') /* 16 Y/CrCb 4:2:2 */
#define V4L2_PIX_FMT_NV24 v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr 4:4:4 */

View File

@ -0,0 +1,151 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@gmail.com>
Date: Sun, 27 Feb 2022 14:59:04 +0100
Subject: [PATCH] media: hantro: Support format filtering by depth
In preparation for supporting 10-bit formats, add mechanism which will
filter formats based on pixel depth.
Hantro G2 supports only one decoding format natively and that is based
on bit depth of current video frame. Additionally, it makes no sense to
upconvert bitness, so filter those out too.
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
drivers/staging/media/hantro/hantro.h | 4 ++
drivers/staging/media/hantro/hantro_v4l2.c | 48 ++++++++++++++++++++--
drivers/staging/media/hantro/hantro_v4l2.h | 1 +
3 files changed, 50 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index 06d0f3597694..c6525ee8d89a 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -227,6 +227,7 @@ struct hantro_dev {
*
* @ctrl_handler: Control handler used to register controls.
* @jpeg_quality: User-specified JPEG compression quality.
+ * @bit_depth: Bit depth of current frame
*
* @codec_ops: Set of operations related to codec mode.
* @postproc: Post-processing context.
@@ -252,6 +253,7 @@ struct hantro_ctx {
struct v4l2_ctrl_handler ctrl_handler;
int jpeg_quality;
+ int bit_depth;
const struct hantro_codec_ops *codec_ops;
struct hantro_postproc_ctx postproc;
@@ -278,6 +280,7 @@ struct hantro_ctx {
* @enc_fmt: Format identifier for encoder registers.
* @frmsize: Supported range of frame sizes (only for bitstream formats).
* @postprocessed: Indicates if this format needs the post-processor.
+ * @match_depth: Indicates if format bit depth must match video bit depth
*/
struct hantro_fmt {
char *name;
@@ -288,6 +291,7 @@ struct hantro_fmt {
enum hantro_enc_fmt enc_fmt;
struct v4l2_frmsize_stepwise frmsize;
bool postprocessed;
+ bool match_depth;
};
struct hantro_reg {
diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c
index e595905b3bd7..1214fa2f64ae 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.c
+++ b/drivers/staging/media/hantro/hantro_v4l2.c
@@ -64,6 +64,42 @@ hantro_get_postproc_formats(const struct hantro_ctx *ctx,
return ctx->dev->variant->postproc_fmts;
}
+int hantro_get_formath_depth(u32 fourcc)
+{
+ switch (fourcc) {
+ case V4L2_PIX_FMT_P010:
+ case V4L2_PIX_FMT_P010_4L4:
+ return 10;
+ default:
+ return 8;
+ }
+}
+
+static bool
+hantro_check_depth_match(const struct hantro_ctx *ctx,
+ const struct hantro_fmt *fmt)
+{
+ int fmt_depth, ctx_depth = 8;
+
+ if (!fmt->match_depth && !fmt->postprocessed)
+ return true;
+
+ /* 0 means default depth, which is 8 */
+ if (ctx->bit_depth)
+ ctx_depth = ctx->bit_depth;
+
+ fmt_depth = hantro_get_formath_depth(fmt->fourcc);
+
+ /*
+ * Allow only downconversion for postproc formats for now.
+ * It may be possible to relax that on some HW.
+ */
+ if (!fmt->match_depth)
+ return fmt_depth <= ctx_depth;
+
+ return fmt_depth == ctx_depth;
+}
+
static const struct hantro_fmt *
hantro_find_format(const struct hantro_ctx *ctx, u32 fourcc)
{
@@ -91,7 +127,8 @@ hantro_get_default_fmt(const struct hantro_ctx *ctx, bool bitstream)
formats = hantro_get_formats(ctx, &num_fmts);
for (i = 0; i < num_fmts; i++) {
if (bitstream == (formats[i].codec_mode !=
- HANTRO_MODE_NONE))
+ HANTRO_MODE_NONE) &&
+ hantro_check_depth_match(ctx, &formats[i]))
return &formats[i];
}
return NULL;
@@ -163,11 +200,13 @@ static int vidioc_enum_fmt(struct file *file, void *priv,
formats = hantro_get_formats(ctx, &num_fmts);
for (i = 0; i < num_fmts; i++) {
bool mode_none = formats[i].codec_mode == HANTRO_MODE_NONE;
+ fmt = &formats[i];
if (skip_mode_none == mode_none)
continue;
+ if (!hantro_check_depth_match(ctx, fmt))
+ continue;
if (j == f->index) {
- fmt = &formats[i];
f->pixelformat = fmt->fourcc;
return 0;
}
@@ -183,8 +222,11 @@ static int vidioc_enum_fmt(struct file *file, void *priv,
return -EINVAL;
formats = hantro_get_postproc_formats(ctx, &num_fmts);
for (i = 0; i < num_fmts; i++) {
+ fmt = &formats[i];
+
+ if (!hantro_check_depth_match(ctx, fmt))
+ continue;
if (j == f->index) {
- fmt = &formats[i];
f->pixelformat = fmt->fourcc;
return 0;
}
diff --git a/drivers/staging/media/hantro/hantro_v4l2.h b/drivers/staging/media/hantro/hantro_v4l2.h
index 18bc682c8556..f4a5905ed518 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.h
+++ b/drivers/staging/media/hantro/hantro_v4l2.h
@@ -22,5 +22,6 @@ extern const struct v4l2_ioctl_ops hantro_ioctl_ops;
extern const struct vb2_ops hantro_queue_ops;
void hantro_reset_fmts(struct hantro_ctx *ctx);
+int hantro_get_formath_depth(u32 fourcc);
#endif /* HANTRO_V4L2_H_ */

View File

@ -0,0 +1,91 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@gmail.com>
Date: Sun, 27 Feb 2022 15:08:34 +0100
Subject: [PATCH] media: hantro: postproc: Fix buffer size calculation
When allocating aux buffers for postprocessing, it's assumed that base
buffer size is the same as that of output. Coincidentally, that's true
most of the time, but not always. 10-bit source also needs aux buffer
size which is appropriate for 10-bit native format, even if the output
format is 8-bit. Similarly, mv sizes and other extra buffer size also
depends on source width/height, not destination.
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
.../staging/media/hantro/hantro_postproc.c | 24 +++++++++++++------
drivers/staging/media/hantro/hantro_v4l2.c | 2 +-
drivers/staging/media/hantro/hantro_v4l2.h | 2 ++
3 files changed, 20 insertions(+), 8 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index 248abe5423f0..1a76628d5754 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -12,6 +12,7 @@
#include "hantro_hw.h"
#include "hantro_g1_regs.h"
#include "hantro_g2_regs.h"
+#include "hantro_v4l2.h"
#define HANTRO_PP_REG_WRITE(vpu, reg_name, val) \
{ \
@@ -137,18 +138,27 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
struct vb2_queue *cap_queue = &m2m_ctx->cap_q_ctx.q;
unsigned int num_buffers = cap_queue->num_buffers;
+ struct v4l2_pix_format_mplane pix_mp;
+ const struct hantro_fmt *fmt;
unsigned int i, buf_size;
- buf_size = ctx->dst_fmt.plane_fmt[0].sizeimage;
+ /* this should always pick native format */
+ fmt = hantro_get_default_fmt(ctx, false);
+ if (!fmt)
+ return -EINVAL;
+ v4l2_fill_pixfmt_mp(&pix_mp, fmt->fourcc, ctx->src_fmt.width,
+ ctx->src_fmt.height);
+
+ buf_size = pix_mp.plane_fmt[0].sizeimage;
if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_H264_SLICE)
- buf_size += hantro_h264_mv_size(ctx->dst_fmt.width,
- ctx->dst_fmt.height);
+ buf_size += hantro_h264_mv_size(pix_mp.width,
+ pix_mp.height);
else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_VP9_FRAME)
- buf_size += hantro_vp9_mv_size(ctx->dst_fmt.width,
- ctx->dst_fmt.height);
+ buf_size += hantro_vp9_mv_size(pix_mp.width,
+ pix_mp.height);
else if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_HEVC_SLICE)
- buf_size += hantro_hevc_mv_size(ctx->dst_fmt.width,
- ctx->dst_fmt.height);
+ buf_size += hantro_hevc_mv_size(pix_mp.width,
+ pix_mp.height);
for (i = 0; i < num_buffers; ++i) {
struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c
index 1214fa2f64ae..69d2a108e1e6 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.c
+++ b/drivers/staging/media/hantro/hantro_v4l2.c
@@ -118,7 +118,7 @@ hantro_find_format(const struct hantro_ctx *ctx, u32 fourcc)
return NULL;
}
-static const struct hantro_fmt *
+const struct hantro_fmt *
hantro_get_default_fmt(const struct hantro_ctx *ctx, bool bitstream)
{
const struct hantro_fmt *formats;
diff --git a/drivers/staging/media/hantro/hantro_v4l2.h b/drivers/staging/media/hantro/hantro_v4l2.h
index f4a5905ed518..cc9a645be886 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.h
+++ b/drivers/staging/media/hantro/hantro_v4l2.h
@@ -23,5 +23,7 @@ extern const struct vb2_ops hantro_queue_ops;
void hantro_reset_fmts(struct hantro_ctx *ctx);
int hantro_get_formath_depth(u32 fourcc);
+const struct hantro_fmt *
+hantro_get_default_fmt(const struct hantro_ctx *ctx, bool bitstream);
#endif /* HANTRO_V4L2_H_ */

View File

@ -0,0 +1,56 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@gmail.com>
Date: Sun, 27 Feb 2022 15:17:03 +0100
Subject: [PATCH] media: hantro: postproc: Fix legacy regs configuration
Some postproc legacy registers were set in VP9 code. Move them to
postproc and fix their value.
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
drivers/staging/media/hantro/hantro_g2_vp9_dec.c | 8 --------
drivers/staging/media/hantro/hantro_postproc.c | 10 ++++++++++
2 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
index 91c21b634fab..c9cb11fd95af 100644
--- a/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
+++ b/drivers/staging/media/hantro/hantro_g2_vp9_dec.c
@@ -515,16 +515,8 @@ static void
config_bit_depth(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_frame *dec_params)
{
if (ctx->dev->variant->legacy_regs) {
- u8 pp_shift = 0;
-
hantro_reg_write(ctx->dev, &g2_bit_depth_y, dec_params->bit_depth);
hantro_reg_write(ctx->dev, &g2_bit_depth_c, dec_params->bit_depth);
- hantro_reg_write(ctx->dev, &g2_rs_out_bit_depth, dec_params->bit_depth);
-
- if (dec_params->bit_depth > 8)
- pp_shift = 16 - dec_params->bit_depth;
-
- hantro_reg_write(ctx->dev, &g2_pp_pix_shift, pp_shift);
hantro_reg_write(ctx->dev, &g2_pix_shift, 0);
} else {
hantro_reg_write(ctx->dev, &g2_bit_depth_y_minus8, dec_params->bit_depth - 8);
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index 1a76628d5754..11ae663f11b7 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -113,6 +113,16 @@ static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma + chroma_offset);
+ if (ctx->dev->variant->legacy_regs) {
+ int out_depth = hantro_get_formath_depth(ctx->dst_fmt.pixelformat);
+ u8 pp_shift = 0;
+
+ if (out_depth > 8)
+ pp_shift = 16 - out_depth;
+
+ hantro_reg_write(ctx->dev, &g2_rs_out_bit_depth, out_depth);
+ hantro_reg_write(ctx->dev, &g2_pp_pix_shift, pp_shift);
+ }
hantro_reg_write(vpu, &g2_out_rs_e, 1);
}

View File

@ -0,0 +1,61 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@gmail.com>
Date: Sun, 27 Feb 2022 15:19:34 +0100
Subject: [PATCH] media: hantro: Store VP9 bit depth in context
Now that we have proper infrastructure for postprocessing 10-bit
formats, store VP9 bit depth in context.
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
drivers/staging/media/hantro/hantro_drv.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index 6a51f39dde56..305090365e74 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -320,6 +320,24 @@ static int hantro_hevc_s_ctrl(struct v4l2_ctrl *ctrl)
return 0;
}
+static int hantro_vp9_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct hantro_ctx *ctx;
+
+ ctx = container_of(ctrl->handler,
+ struct hantro_ctx, ctrl_handler);
+
+ switch (ctrl->id) {
+ case V4L2_CID_STATELESS_VP9_FRAME:
+ ctx->bit_depth = ctrl->p_new.p_vp9_frame->bit_depth;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static const struct v4l2_ctrl_ops hantro_ctrl_ops = {
.try_ctrl = hantro_try_ctrl,
};
@@ -332,6 +350,10 @@ static const struct v4l2_ctrl_ops hantro_hevc_ctrl_ops = {
.s_ctrl = hantro_hevc_s_ctrl,
};
+static const struct v4l2_ctrl_ops hantro_vp9_ctrl_ops = {
+ .s_ctrl = hantro_vp9_s_ctrl,
+};
+
static const struct hantro_ctrl controls[] = {
{
.codec = HANTRO_JPEG_ENCODER,
@@ -478,6 +500,7 @@ static const struct hantro_ctrl controls[] = {
.codec = HANTRO_VP9_DECODER,
.cfg = {
.id = V4L2_CID_STATELESS_VP9_FRAME,
+ .ops = &hantro_vp9_ctrl_ops,
},
}, {
.codec = HANTRO_VP9_DECODER,

View File

@ -0,0 +1,41 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@gmail.com>
Date: Sun, 27 Feb 2022 15:21:05 +0100
Subject: [PATCH] media: hantro: sunxi: Enable 10-bit decoding
Now that infrastructure for 10-bit decoding exists, enable it for
Allwinner H6.
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
drivers/staging/media/hantro/sunxi_vpu_hw.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/staging/media/hantro/sunxi_vpu_hw.c b/drivers/staging/media/hantro/sunxi_vpu_hw.c
index c0edd5856a0c..8d8a8937d6aa 100644
--- a/drivers/staging/media/hantro/sunxi_vpu_hw.c
+++ b/drivers/staging/media/hantro/sunxi_vpu_hw.c
@@ -15,12 +15,23 @@ static const struct hantro_fmt sunxi_vpu_postproc_fmts[] = {
.codec_mode = HANTRO_MODE_NONE,
.postprocessed = true,
},
+ {
+ .fourcc = V4L2_PIX_FMT_P010,
+ .codec_mode = HANTRO_MODE_NONE,
+ .postprocessed = true,
+ },
};
static const struct hantro_fmt sunxi_vpu_dec_fmts[] = {
{
.fourcc = V4L2_PIX_FMT_NV12_4L4,
.codec_mode = HANTRO_MODE_NONE,
+ .match_depth = true,
+ },
+ {
+ .fourcc = V4L2_PIX_FMT_P010_4L4,
+ .codec_mode = HANTRO_MODE_NONE,
+ .match_depth = true,
},
{
.fourcc = V4L2_PIX_FMT_VP9_FRAME,

View File

@ -0,0 +1,35 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Jernej Skrabec <jernej.skrabec@gmail.com>
Date: Sun, 27 Feb 2022 17:59:18 +0100
Subject: [PATCH] media: hantro: postproc: Properly calculate chroma offset
Currently chroma offset calculation assumes only 1 byte per luma, with
no consideration for stride.
Take necessary information from destination pixel format which makes
calculation completely universal.
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
drivers/staging/media/hantro/hantro_postproc.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index 11ae663f11b7..d8358d3289dc 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -105,11 +105,13 @@ static void hantro_postproc_g2_enable(struct hantro_ctx *ctx)
{
struct hantro_dev *vpu = ctx->dev;
struct vb2_v4l2_buffer *dst_buf;
- size_t chroma_offset = ctx->dst_fmt.width * ctx->dst_fmt.height;
+ size_t chroma_offset;
dma_addr_t dst_dma;
dst_buf = hantro_get_dst_buf(ctx);
dst_dma = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
+ chroma_offset = ctx->dst_fmt.plane_fmt[0].bytesperline *
+ ctx->dst_fmt.height;
hantro_write_addr(vpu, G2_RS_OUT_LUMA_ADDR, dst_dma);
hantro_write_addr(vpu, G2_RS_OUT_CHROMA_ADDR, dst_dma + chroma_offset);