[kvm-unit-tests PATCH 08/33] arm: realm: Make uart available before MMU is enabled

[Alexandru Elisei]

Hi,

On Mon, Apr 22, 2024 at 01:09:13PM +0100, Suzuki K Poulose wrote:
> Hi Alexandru
> 
> On 22/04/2024 12:58, Alexandru Elisei wrote:
> > Hi,
> > 
> > On Fri, Apr 12, 2024 at 11:33:43AM +0100, Suzuki K Poulose wrote:
> > > From: Joey Gouly <joey.gouly at arm.com>
> > > 
> > > A Realm must access any emulated I/O mappings with the PTE_NS_SHARED bit set.
> > > This is modelled as a PTE attribute, but is actually part of the address.
> > > 
> > > So, when MMU is disabled, the "physical address" must reflect this bit set. We
> > > access the UART early before the MMU is enabled. So, make sure the UART is
> > > accessed always with the bit set.
> > > 
> > > Signed-off-by: Joey Gouly <joey.gouly at arm.com>
> > > Signed-off-by: Suzuki K Poulose <suzuki.poulose at arm.com>
> > > ---
> > >   lib/arm/asm/pgtable.h   |  5 +++++
> > >   lib/arm/io.c            | 24 +++++++++++++++++++++++-
> > >   lib/arm64/asm/pgtable.h |  5 +++++
> > >   3 files changed, 33 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/lib/arm/asm/pgtable.h b/lib/arm/asm/pgtable.h
> > > index 350039ff..7e85e7c6 100644
> > > --- a/lib/arm/asm/pgtable.h
> > > +++ b/lib/arm/asm/pgtable.h
> > > @@ -112,4 +112,9 @@ static inline pte_t *pte_alloc(pmd_t *pmd, unsigned long addr)
> > >   	return pte_offset(pmd, addr);
> > >   }
> > > +static inline unsigned long arm_shared_phys_alias(void *x)
> > > +{
> > > +	return ((unsigned long)(x) | PTE_NS_SHARED);
> > > +}
> > 
> > Is it allowed for a realm to run in aarch32 mode?
> 
> No. Realm EL1 must be Aarch64.

Ok, then can you make the above function return the original address?

> 
> > 
> > > +
> > >   #endif /* _ASMARM_PGTABLE_H_ */
> > > diff --git a/lib/arm/io.c b/lib/arm/io.c
> > > index 836fa854..127727e4 100644
> > > --- a/lib/arm/io.c
> > > +++ b/lib/arm/io.c
> > > @@ -15,6 +15,8 @@
> > >   #include <asm/psci.h>
> > >   #include <asm/spinlock.h>
> > >   #include <asm/io.h>
> > > +#include <asm/mmu-api.h>
> > > +#include <asm/pgtable.h>
> > >   #include "io.h"
> > > @@ -30,6 +32,24 @@ static struct spinlock uart_lock;
> > >   static volatile u8 *uart0_base = UART_EARLY_BASE;
> > >   bool is_pl011_uart;
> > > +static inline volatile u8 *get_uart_base(void)
> > > +{
> > > +	/*
> > > +	 * The address of the UART base may be different
> > > +	 * based on whether we are running with/without
> > > +	 * MMU enabled.
> > > +	 *
> > > +	 * For realms, we must force to use the shared physical
> > > +	 * alias with MMU disabled, to make sure the I/O can
> > > +	 * be emulated.
> > > +	 * When the MMU is turned ON, the mappings are created
> > > +	 * appropriately.
> > > +	 */
> > > +	if (mmu_enabled())
> > > +		return uart0_base;
> > > +	return (u8 *)arm_shared_phys_alias((void *)uart0_base);
> > > +}
> > > +
> > >   static void uart0_init_fdt(void)
> > >   {
> > >   	/*
> > > @@ -109,9 +129,11 @@ void io_init(void)
> > >   void puts(const char *s)
> > >   {
> > > +	volatile u8 *uart_base = get_uart_base();

Is it just my email client or is the indentation missing here?

> > > +
> > >   	spin_lock(&uart_lock);
> > >   	while (*s)
> > > -		writeb(*s++, uart0_base);
> > > +		writeb(*s++, uart_base);
> > >   	spin_unlock(&uart_lock);
> > >   }
> > > diff --git a/lib/arm64/asm/pgtable.h b/lib/arm64/asm/pgtable.h
> > > index 5b9f40b0..871c03e9 100644
> > > --- a/lib/arm64/asm/pgtable.h
> > > +++ b/lib/arm64/asm/pgtable.h
> > > @@ -28,6 +28,11 @@ extern unsigned long prot_ns_shared;
> > >   */
> > >   #define PTE_NS_SHARED		(prot_ns_shared)
> > > +static inline unsigned long arm_shared_phys_alias(void *addr)
> > > +{
> > > +	return ((unsigned long)addr | PTE_NS_SHARED);
> > > +}
> > 
> > Have you considered specifying the correct UART address at compile time using
> > ./configure --earlycon?
> 
> Do you mean
> 
> ./configure --earlycon=<NS-Alias-normal-UART-Address> for Realms ?
> 
> If so, there are multiple issues with that :
> 
> 1. A payload could be run in a normal VM or a Realm VM. Having the above
> restricts using the same payload in different worlds. (e.g., comparison).
> 
> 2. The IPA width of the Realm and thus the PTE_NS_SHARED is dynamic
> and really depends on what the VMM decides to choose the IPA size.
> (Could be based on user input).

Does it depend on the chosen IPA size, or on the VA size, or on both?
Because uart0_base is used with both the MMU on and off.

Thanks,
Alex

> 
> If any of the above fails, and a wrong earlycon address could result
> in "Synchronouse External Abort" into the Realm (if it is in protected
> IPA).
> 
> Suzuki
> > 
> > Thanks,
> > Alex
> > > +
> > >   /*
> > >    * Highest possible physical address supported.
> > >    */
> > > -- 
> > > 2.34.1
> > > 
>