[PATCH v4 16/20] arm64: enable Permission Indirection Extension (PIE)

[Neil Armstrong]

Hi Joey,

On 06/06/2023 16:58, Joey Gouly wrote:
> Now that the necessary changes have been made, set the Permission Indirection
> registers and enable the Permission Indirection Extension.
> 
> Signed-off-by: Joey Gouly <joey.gouly at arm.com>
> Cc: Catalin Marinas <catalin.marinas at arm.com>
> Cc: Will Deacon <will at kernel.org>
> Reviewed-by: Catalin Marinas <catalin.marinas at arm.com>

This patch on linux-next causes a great amount of:

X     xxx.xxxxxx Emulated RAZ for ID register: ISS 0x36002f

messages printed by the system firmware on the Qualcomm SM8550 SoC,
and the platform is barely usable.

Here is the SoC cpuinfo for reference:
# cat /proc/cpuinfo
processor	: 0
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bti
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd46
CPU revision	: 1

processor	: 1
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bti
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd46
CPU revision	: 1

processor	: 2
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bti
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd46
CPU revision	: 1

processor	: 3
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bti
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd4d
CPU revision	: 0

processor	: 4
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bti
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd4d
CPU revision	: 0

processor	: 5
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bti
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x2
CPU part	: 0xd47
CPU revision	: 0

processor	: 6
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bti
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x2
CPU part	: 0xd47
CPU revision	: 0

processor	: 7
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bti
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd4e
CPU revision	: 0

The bisect log for reference:
# bad: [925294c9aa184801cc0a451b69a18dd0fe7d847d] Add linux-next specific files for 20230615
# good: [858fd168a95c5b9669aac8db6c14a9aeab446375] Linux 6.4-rc6
git bisect start 'FETCH_HEAD' 'v6.4-rc6'
# bad: [c20f7e5e521ee3f50b064cdb441f65075ca6eb17] Merge branch 'nand/next' of git://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux.git
git bisect bad c20f7e5e521ee3f50b064cdb441f65075ca6eb17
# bad: [9b3c3144b2d3b5022370883e2834887fc7f3d5d3] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux.git
git bisect bad 9b3c3144b2d3b5022370883e2834887fc7f3d5d3
# bad: [8a2e6adeea094195f860f1f5dd799c9f0015dd92] Merge branch 'at91-next' of git://git.kernel.org/pub/scm/linux/kernel/git/at91/linux.git
git bisect bad 8a2e6adeea094195f860f1f5dd799c9f0015dd92
# good: [4f826d17f9de4d708f2c07bb40a104426a22b384] Merge branch 'mm-everything' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
git bisect good 4f826d17f9de4d708f2c07bb40a104426a22b384
# good: [62f97a067a7e77fd2016ce7d413fceaaf5882385] Merge branch 'for-next' of git://git.infradead.org/users/hch/dma-mapping.git
git bisect good 62f97a067a7e77fd2016ce7d413fceaaf5882385
# good: [e5047345bb6c8ee1e5d319c989dc9e3442f891c7] soc: document merges
git bisect good e5047345bb6c8ee1e5d319c989dc9e3442f891c7
# good: [cca5c2136d9ca679a9b0cdb51ce64c79cd7c092c] Merge branches 'for-next/kpti', 'for-next/missing-proto-warn', 'for-next/iss2-decode', 'for-next/kselftest', 'for-next/misc', 'for-next/feat_mops', 'for-next/module-alloc', 'for-next/sysreg', 'for-next/cpucap', 'for-next/acpi', 'for-next/kdump' and 'for-next/acpi-doc' into for-next/core
git bisect good cca5c2136d9ca679a9b0cdb51ce64c79cd7c092c
# bad: [b6719d317df3ab47dc86776c499fc1e2009e1b37] Merge branch 'for-next/core' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
git bisect bad b6719d317df3ab47dc86776c499fc1e2009e1b37
# bad: [ee053e03b08e1b287d9a43152e4623a04cb24fe6] KVM: selftests: get-reg-list: support ID register features
git bisect bad ee053e03b08e1b287d9a43152e4623a04cb24fe6
# good: [86f9de9db1783b32e8812fe21c2c8cf02cf911ff] KVM: arm64: Save/restore PIE registers
git bisect good 86f9de9db1783b32e8812fe21c2c8cf02cf911ff
# good: [7df7170965a28c61f80a57b655b0cc10adb88ab9] arm64: disable EL2 traps for PIE
git bisect good 7df7170965a28c61f80a57b655b0cc10adb88ab9
# bad: [9e9bb6ede00a84275b65bb8d00812c1e24b5fa7e] arm64: enable Permission Indirection Extension (PIE)
git bisect bad 9e9bb6ede00a84275b65bb8d00812c1e24b5fa7e
# good: [eeda243dfeb996fe236c624796630c16237a18d6] arm64: add encodings of PIRx_ELx registers
git bisect good eeda243dfeb996fe236c624796630c16237a18d6
# first bad commit: [9e9bb6ede00a84275b65bb8d00812c1e24b5fa7e] arm64: enable Permission Indirection Extension (PIE)

Thanks,
Neil

> ---
>   arch/arm64/mm/proc.S | 15 +++++++++++++++
>   1 file changed, 15 insertions(+)
> 
> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> index 9513a8d2ce0e..2baeec419f62 100644
> --- a/arch/arm64/mm/proc.S
> +++ b/arch/arm64/mm/proc.S
> @@ -454,6 +454,21 @@ SYM_FUNC_START(__cpu_setup)
>   #endif	/* CONFIG_ARM64_HW_AFDBM */
>   	msr	mair_el1, mair
>   	msr	tcr_el1, tcr
> +
> +	mrs_s	x1, SYS_ID_AA64MMFR3_EL1
> +	ubfx	x1, x1, #ID_AA64MMFR3_EL1_S1PIE_SHIFT, #4
> +	cbz	x1, .Lskip_indirection
> +
> +	mov_q	x0, PIE_E0
> +	msr	REG_PIRE0_EL1, x0
> +	mov_q	x0, PIE_E1
> +	msr	REG_PIR_EL1, x0
> +
> +	mov	x0, TCR2_EL1x_PIE
> +	msr	REG_TCR2_EL1, x0
> +
> +.Lskip_indirection:
> +
>   	/*
>   	 * Prepare SCTLR
>   	 */