[PATCH 2/2] arm64: errata: Work around AmpereOne's erratum AC04_CPU_23

[D Scott Phillips]

Oliver Upton <oliver.upton at linux.dev> writes:

> Hi,
>
> On Tue, Apr 15, 2025 at 08:47:11AM -0700, D Scott Phillips wrote:
>> Updates to HCR_EL2 can rarely corrupt simultaneous translations from
>> either earlier translations (back to the previous dsb) or later
>> translations (up to the next isb). Put a dsb before and isb after writes
>> to HCR_EL2.
>> 
>> Signed-off-by: D Scott Phillips <scott at os.amperecomputing.com>
>> ---
>>  arch/arm64/Kconfig              | 13 +++++++++++++
>>  arch/arm64/include/asm/sysreg.h |  7 +++++++
>>  arch/arm64/kernel/cpu_errata.c  | 14 ++++++++++++++
>>  arch/arm64/tools/cpucaps        |  1 +
>>  4 files changed, 35 insertions(+)
>> 
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index e5fd87446a3b8..2a2e1c8de6a16 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -481,6 +481,19 @@ config AMPERE_ERRATUM_AC03_CPU_38
>>  
>>  	  If unsure, say Y.
>>  
>> +config AMPERE_ERRATUM_AC04_CPU_23
>> +        bool "AmpereOne: AC04_CPU_23:  Failure to synchronize writes to HCR_EL2 may corrupt address translations."
>> +	default y
>> +	help
>> +	  This option adds an alternative code sequence to work around Ampere
>> +	  errata AC04_CPU_23 on AmpereOne.
>> +
>> +	  Updates to HCR_EL2 can rarely corrupt simultaneous translations from
>> +	  either earlier translations (back to the previous dsb) or later
>> +	  translations (up to the next isb).
>> +
>> +	  If unsure, say Y.
>> +
>>  config ARM64_WORKAROUND_CLEAN_CACHE
>>  	bool
>>  
>> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
>> index e7781f7e7f7a7..253de5bc68834 100644
>> --- a/arch/arm64/include/asm/sysreg.h
>> +++ b/arch/arm64/include/asm/sysreg.h
>> @@ -1142,6 +1142,10 @@
>>  	(IS_ENABLED(CONFIG_AMPERE_ERRATUM_AC03_CPU_36) &&	\
>>  	 __sysreg_is_hcr_el2(r) &&				\
>>  	 alternative_has_cap_unlikely(ARM64_WORKAROUND_AMPERE_AC03_CPU_36))
>> +#define __hcr_el2_ac04_cpu_23(r)				\
>> +	(IS_ENABLED(CONFIG_AMPERE_ERRATUM_AC04_CPU_23) &&	\
>> +	 __sysreg_is_hcr_el2(r) &&				\
>> +	 alternative_has_cap_unlikely(ARM64_WORKAROUND_AMPERE_AC04_CPU_23))
>>  
>>  /*
>>   * The "Z" constraint normally means a zero immediate, but when combined with
>> @@ -1154,6 +1158,9 @@
>>  		asm volatile("mrs %0, daif; msr daifset, #0xf;"	\
>>  			     "msr hcr_el2, %x1; msr daif, %0"	\
>>  		: "=&r"(__daif) : "rZ" (__val));		\
>> +	} else if (__hcr_el2_ac04_cpu_23(r)) {			\
>> +		asm volatile("dsb nsh; msr hcr_el2, %x0; isb"	\
>> +			     : : "rZ" (__val));			\
>
> At least from your erratum description it isn't clear to me that this
> eliminates the problem and only narrows the window of opportunity.
> Couldn't the implementation speculatively fetch translations with an
> unsynchronized HCR up to the ISB? Do we know what translation regimes
> are affected by the erratum?

Hi Oliver, I got some clarification from the core folks. The issue
affects the data side of the core only, not the instruction side.  I'll
update my description to include that.

Speculation after the `msr hcr_el2` couldn't launch a problem
translation when the `msr` is followed by an `isb` like this.


Marc Zyngier <maz at kernel.org> writes:

> On Tue, 15 Apr 2025 16:47:11 +0100,
> If the write to HCR_EL2 can corrupt translations, what guarantees that
> such write placed on a page boundary (therefore requiring another TLB
> lookup to continue in sequence) will be able to get to the ISB?

Hi Marc, I understand now from the core team that an ISB on another page
will be ok as the problem is on the data side only.