Why does the firmware memory region have no permissions?

[Chang, Abner (HPS SW/FW Technologist)]

> -----Original Message-----
> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> Sent: Friday, June 4, 2021 3:45 PM
> To: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>;
> Heinrich Schuchardt <xypron.glpk at gmx.de>; Daniel Schaefer
> <daniel at danielschaefer.me>
> Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>
> Subject: RE: Why does the firmware memory region have no permissions?
> 
> 
> 
> > -----Original Message-----
> > From: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>
> > Sent: 04 June 2021 13:11
> > To: Heinrich Schuchardt <xypron.glpk at gmx.de>; Anup Patel
> > <Anup.Patel at wdc.com>; Daniel Schaefer <daniel at danielschaefer.me>
> > Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>
> > Subject: RE: Why does the firmware memory region have no permissions?
> >
> >
> >
> > > -----Original Message-----
> > > From: Heinrich Schuchardt [mailto:xypron.glpk at gmx.de]
> > > Sent: Friday, June 4, 2021 3:08 PM
> > > To: Anup Patel <Anup.Patel at wdc.com>; Chang, Abner (HPS SW/FW
> > > Technologist) <abner.chang at hpe.com>; Daniel Schaefer
> > > <daniel at danielschaefer.me>
> > > Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>
> > > Subject: Re: Why does the firmware memory region have no permissions?
> > >
> > > On 6/4/21 8:57 AM, Anup Patel wrote:
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Chang, Abner (HPS SW/FW Technologist)
> > <abner.chang at hpe.com>
> > > >> Sent: 04 June 2021 12:00
> > > >> To: Heinrich Schuchardt <xypron.glpk at gmx.de>; Anup Patel
> > > >> <Anup.Patel at wdc.com>; Daniel Schaefer
> <daniel at danielschaefer.me>
> > > >> Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>
> > > >> Subject: RE: Why does the firmware memory region have no
> > permissions?
> > > >>
> > > >>
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Heinrich Schuchardt [mailto:xypron.glpk at gmx.de]
> > > >>> Sent: Friday, June 4, 2021 1:15 AM
> > > >>> To: Anup Patel <Anup.Patel at wdc.com>; Chang, Abner (HPS SW/FW
> > > >>> Technologist) <abner.chang at hpe.com>; Daniel Schaefer
> > > >>> <daniel at danielschaefer.me>
> > > >>> Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>
> > > >>> Subject: Re: Why does the firmware memory region have no
> > > permissions?
> > > >>>
> > > >>> On 6/3/21 6:51 PM, Heinrich Schuchardt wrote:
> > > >>>> On 6/3/21 5:34 PM, Anup Patel wrote:
> > > >>>>> +Heinrich
> > > >>>>>
> > > >>>>>> -----Original Message-----
> > > >>>>>> From: Chang, Abner (HPS SW/FW Technologist)
> > > >>> <abner.chang at hpe.com>
> > > >>>>>> Sent: 03 June 2021 20:05
> > > >>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> > > >>>>>> <daniel at danielschaefer.me>
> > > >>>>>> Cc: opensbi at lists.infradead.org
> > > >>>>>> Subject: RE: Why does the firmware memory region have no
> > > >>> permissions?
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>> -----Original Message-----
> > > >>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> > > >>>>>>> Sent: Wednesday, June 2, 2021 11:15 PM
> > > >>>>>>> To: Chang, Abner (HPS SW/FW Technologist)
> > > >> <abner.chang at hpe.com>;
> > > >>>>>>> Daniel Schaefer <daniel at danielschaefer.me>
> > > >>>>>>> Cc: opensbi at lists.infradead.org
> > > >>>>>>> Subject: RE: Why does the firmware memory region have no
> > > >>> permissions?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>> -----Original Message-----
> > > >>>>>>>> From: Chang, Abner (HPS SW/FW Technologist)
> > > >>> <abner.chang at hpe.com>
> > > >>>>>>>> Sent: 02 June 2021 20:19
> > > >>>>>>>> To: Chang, Abner (HPS SW/FW Technologist)
> > > >>> <abner.chang at hpe.com>;
> > > >>>>>>> Anup
> > > >>>>>>>> Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> > > >>>>>>> <daniel at danielschaefer.me>
> > > >>>>>>>> Cc: opensbi at lists.infradead.org
> > > >>>>>>>> Subject: RE: Why does the firmware memory region have no
> > > >>> permissions?
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> -----Original Message-----
> > > >>>>>>>>> From: opensbi [mailto:opensbi-bounces at lists.infradead.org]
> > > >>>>>>>>> On Behalf Of Chang, Abner (HPS SW/FW Technologist)
> > > >>>>>>>>> Sent: Saturday, May 15, 2021 11:30 PM
> > > >>>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> > > >>>>>>>>> <daniel at danielschaefer.me>
> > > >>>>>>>>> Cc: opensbi at lists.infradead.org
> > > >>>>>>>>> Subject: RE: Why does the firmware memory region have no
> > > >>>>>> permissions?
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>> -----Original Message-----
> > > >>>>>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> > > >>>>>>>>>> Sent: Saturday, May 15, 2021 4:09 PM
> > > >>>>>>>>>> To: Chang, Abner (HPS SW/FW Technologist)
> > > >>>>>> <abner.chang at hpe.com>;
> > > >>>>>>>>> Daniel
> > > >>>>>>>>>> Schaefer <daniel at danielschaefer.me>
> > > >>>>>>>>>> Cc: opensbi at lists.infradead.org
> > > >>>>>>>>>> Subject: RE: Why does the firmware memory region have
> no
> > > >>>>>>> permissions?
> > > >>>>>>>>>>
> > > >>>>>>>>>> Hi Abner,
> > > >>>>>>>>>>
> > > >>>>>>>>>>> -----Original Message-----
> > > >>>>>>>>>>> From: Chang, Abner (HPS SW/FW Technologist)
> > > >>>>>>>>> <abner.chang at hpe.com>
> > > >>>>>>>>>>> Sent: 15 May 2021 11:47
> > > >>>>>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> > > >>>>>>>>>>> <daniel at danielschaefer.me>
> > > >>>>>>>>>>> Cc: opensbi at lists.infradead.org
> > > >>>>>>>>>>> Subject: RE: Why does the firmware memory region have
> no
> > > >>>>>>>> permissions?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> -----Original Message-----
> > > >>>>>>>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> > > >>>>>>>>>>>> Sent: Friday, May 14, 2021 7:58 PM
> > > >>>>>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
> > > >>>>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS
> SW/FW
> > > >>>>>>>>> Technologist)
> > > >>>>>>>>>>>> <abner.chang at hpe.com>
> > > >>>>>>>>>>>> Subject: RE: Why does the firmware memory region have
> no
> > > >>>>>>>>> permissions?
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> -----Original Message-----
> > > >>>>>>>>>>>>> From: Anup Patel
> > > >>>>>>>>>>>>> Sent: 14 May 2021 17:22
> > > >>>>>>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
> > > >>>>>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS
> > > SW/FW
> > > >>>>>>>>>>>>> Technologist) <abner.chang at hpe.com>
> > > >>>>>>>>>>>>> Subject: RE: Why does the firmware memory region
> have
> > > no
> > > >>>>>>>>>>> permissions?
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> -----Original Message-----
> > > >>>>>>>>>>>>>> From: opensbi <opensbi-
> bounces at lists.infradead.org>
> > > On
> > > >>>>>>>>>>>>>> Behalf
> > > >>>>>>>>> Of
> > > >>>>>>>>>>>>>> Daniel Schaefer
> > > >>>>>>>>>>>>>> Sent: 13 May 2021 10:26
> > > >>>>>>>>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>
> > > >>>>>>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS
> > > SW/FW
> > > >>>>>>>>>>>>>> Technologist) <abner.chang at hpe.com>
> > > >>>>>>>>>>>>>> Subject: Why does the firmware memory region have
> no
> > > >>>>>>>>> permissions?
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Whoops, put CC as subject...
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> On 5/12/21 6:20 PM, Daniel Schaefer wrote:
> > > >>>>>>>>>>>>>>> Hi Anup,
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> I'm in the process of upgrading EDKII to OpenSBI 0.9
> > > >>>>>>>>>>>>>>> and using the Generic
> > > >>>>>>>>>>>>>> Platform.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Previously we were doing sbi_init with M-Mode,
> adding
> > > our
> > > >>>>>>>>>>>>>>> SBI extension and then calling sbi_switch_mode to
> > > switch
> > > >>>>>>>>>>>>>>> to S-
> > > >>>>>>>> Mode.
> > > >>>>>>>>>>>>>>> Now sbi_init disallows initializing to M-Mode, so I'm
> > > >>>>>>>>>>>>>>> directly switching to S-Mode. It seems that even
> from
> > > >>>>>>>>>>>>>>> S-Mode I can register our SBI
> > > >>>>>>>>>>>>>> extension with sbi_ecall_register_extension.
> > > >>>>>>>>>>>>>>> Is that correct?
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> The OpenSBI sources are meant to run only from M-
> mode
> > > so
> > > >> we
> > > >>>>>>>>> cannot
> > > >>>>>>>>>>>>> register SBI extension using
> sbi_ecall_register_extension().
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> The  sbi_switch_mode() is not stricter due to OpenSBI
> > > >>>>>>>>>>>>> domain
> > > >>>>>>>>> support.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> The sbi_hart_switch_mode() is fine. It's the
> > > >>>>>>>>>>>> sbi_domain_init() which is enforcing next booting stage
> > > >>>>>>>>>>>> to be at lower privilege for
> > > >>>>>>> root
> > > >>>>>>>> domain.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> For time being, you can try removing checks on "dom-
> > > >>>>>>> next_mode"
> > > >>>>>>>>>>>> in sanitize_domain()
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Hi Anup, I think we currently skip that check for moving
> > > >>>>>>>>>>> on the
> > > >>>>>>>>>>> edk2 boot process. So do you have plan to remove this
> > check?
> > > >>>>>>>>>>> Or any
> > > >>>>>>>> alternative?
> > > >>>>>>>>>>> I think it is unnecessary having this check on the next
> > > >>>>>>>>>>> privilege
> > > >>>>>> mode.
> > > >>>>>>>>> That
> > > >>>>>>>>>>> should be at OEM discretion of which privilege mode to
> run
> > > >>>>>>>>>>> their next firmware stage based on the platform design?
> > > >>>>>>>>>>
> > > >>>>>>>>>> This is an important check required by OpenSBI domain
> > > >>>>>>>>>> support so that next booting stage cannot tamper with PMP
> > > >>>>>>>>>> configuration (and other security configuration) done by
> > > OpenSBI.
> > > >>>>>>>>>
> > > >>>>>>>>> I understand the importance of not giving any chance to
> > > >>>>>>>>> tamper
> > > >>> PMP
> > > >>>>>>>>> setting, however this could be the responsibility of the
> > > >>>>>>>>> next boot phase
> > > >>>>>>>> before OS.
> > > >>>>>>>>> OpenSBI as the early phase boot firmware should be
> generally
> > > >>>>>>>>> provide SBIs to platform variants, and have the flexibility
> > > >>>>>>>>> to hand off to either M-mode or S-mode firmware (Actually I
> > > >>>>>>>>> don't think OpenSBI
> > > >>>>>>> should
> > > >>>>>>>> handle this).
> > > >>>>>>>>> Platform code is provided by OEM/vendor, OpenSBI should
> > > allow
> > > >>>>>>>>> it if platform code says I would like to run my next phase
> > > >>>>>>>>> firmware in M-
> > > >>>>>>> mode.
> > > >>>>>>>>> We restrict the privilege phase for the next phase in
> > > >>>>>>>>> OpenSBI also not compliant with the UEFI spec which says
> > > >>>>>>>>> UEFI RISC-V firmware could be executed in either M-mode
> or
> > > >>>>>>>>> S-mode. Some
> > > EFI
> > > >>>>>>>>> driver
> > > >>> may
> > > >>>>>>>>> be loaded in
> > > >>>>>>>>> S- mode but provide the M-mode code such as management
> > > mode,
> > > >>> the
> > > >>>>>>>>> Platform Runtime Mechanism or some other use cases. The
> > next
> > > >>>>>>> firmware
> > > >>>
> > > >>> Compare this to EDK II with TF-A. No EDK II code runs in EL3. TF-A
> > > >>> is the only EL3 code.
> > > >>>
> > > >>> Same for RISC-V: OpenSBI is the M-mode firmware. No other part of
> > > >>> the firmware shall run in M-mode. I cannot see that the UEFI
> > > >>> specification requires a different segregation of duties.
> > > >> That paragraph stated in UEFI spec is based on RISC-V privilege
> > > >> spec,
> > > which
> > > >> says all platforms must implement M-mode.
> > > >> The simplest platform can only has M-mode. That makes sense to say
> > > UEFI
> > > >> firmware can run on either M-mode or S-mode.
> > > >
> > > > UEFI spec is based on RISC-V privilege spec does not mean UEFI has
> > > > to run in M-mode only.
> > > >
> > > > The simplest platform having M-mode will not support regular S-mode
> > > > Linux which is the client software for UEFI so if there is not UEFI
> > > > client on a simple M-mode only platform then why should EDK2 target
> > > > M-mode only platform ??
> > > >
> > > > On similar lines, OpenSBI only supports platforms where S-mode is
> > > > available because there is no point in providing SBI services for
> > > > platform not having S-mode.
> > > >
> > > >> I also curious about why opensbi restricts to switch the next mode
> > > >> to M- mode. How does opensbi runs on the platform only provides M-
> > mode?
> > > >
> > > > That's because OpenSBI acts as secure monitor. The TEE will run as
> > > > secure OpenSBI domain whereas Linux (non-secure) software will run
> > > > as non-secure OpenSBI domain.
> > > >
> > > >>
> > > >>>
> > > >>> What RISC-V is lacking up to now is run modes for trusted
> > > >>> execution environments like OP-TEE. This is where you would place
> > > >>> management mode code and not in M-mode.
> > > >> Yes, that is the plan to have management mode runs in TEE, do you
> > > >> have suggestion how do we load the MM driver to TEE from S-mode,
> in
> > > >> which phase we can we initial the MM environment and load the MM
> > > >> driver to
> > > TEE
> > > >> through DXE dispatcher?  and all under S-mode.
> > > >> Is any of S-mode entities can load itself to TEE?
> > >
> > >
> > > https://static.linaro.org/connect/lvc20/presentations/LVC20-302-0.pdf
> > > gives an overview of MM running in OP-TEE.
> > >
> > > INVALID URI REMOVED
> > >
> >
> ecture/trusted_applications.html__;!!NpxR!wPhqffxytOLb6OGvj0Hizxh8G5H
> > > lQIk344bJPo8V66GY4Yh-9WKpAPCkrliqFZY$
> > > describes that trusted applications can either be appended to OP-TEE
> > > or can be loaded via remote procedure calls. For U-Boot on ARM we are
> > > currently using an StMM appended to OP-TEE.
> > Yes, I know that ARM loads the standalone MM to SPM for the standalone
> > MM driver, unfortunately , most of MM implementation on servers are
> based
> > on the standard MM which is dispatched by DXE. That would be a key point
> > for us how mush efforts and risks to turn those implementations into TEE
> > with standalone MM. Suppose we can just simply migrate our existing MM
> > implantations to RISC-V TEE if the abstract driver of standard MM
> > environment for RISC-V is provided to edk2. That would be not good to us
> if
> > the upcoming TEE/ opensbi Secure monitor extension has a big gap to the
> > existing implantation.
> 
> That's why we need to define a standard SBI TEE calls which everyone
> agrees. We have unixplatform mailing where SBI extensions are discussed
> and standardized.
Good.
> 
> It really seems like that you are happy the EDK2 specific SBI FW calls and
> don't want to migrate to standard SBI TEE calls.
Did I say that? no.. you get me wrong.
I said in another email that we don't have to rely on EDK2 specific SBI FW if opensbi TEE has provides the corresponding extensions. :)

In this process, you are
> reluctant to support EDK2 inside Guest/VM.
> 
> Regards,
> Anup
> 
> >
> > Regards,
> > Abner
> >
> > >
> > > Best regards
> > >
> > > Heinrich
> > >
> > > >
> > > > We need to define SBI TEE calls where OpenSBI will forward these TEE
> > > > calls from Linux (non-secure) domain to OP-TEE (secure) domain.
> > > > These SBI TEE calls will have mechanism to load MM driver to secure
> > > > domain (or similar things).
> > > >
> > > > Once we define the SBI TEE calls and possible ways to load TEE, the
> > > > RISC-V TEE story will become complete.
> > > >
> > > > The OpenSBI domain support and SBI TEE calls will make RISC-V TEE
> > > > competitive with ARM TF-A.
> > > >
> > > > I am totally convinced now that EDK2 should entirely run in S-mode
> > > > so that we can re-use EDK2 for Guest/VM. The SBI provider for EDK2
> > > > could be OpenSBI or hypervisors. This will be exactly same as how
> > > > EDK2 runs in ARM world.
> > > >
> > > > Regards,
> > > > Anup
> > > >
> > > >>
> > > >> Regards,
> > > >> Abner
> > > >>
> > > >>>
> > > >>> Best regards
> > > >>>
> > > >>> Heinrich
> > > >>>
> > > >>>>>>>>> has the responsibility to switch to S-mode when handoff to
> > > >>>>>>>>> OS or software if the platform design requires that (I
> > > >>>>>>>>> remember we have the simi  lar sentence in
> > > >>>>>>>>> riscv-platform-spec). EDK2 code can't just remove the check
> > > >>>>>>>>> "dom->next_mode", we use
> > > OpenSBI
> > > >>> without any
> > > >>>>>>>> changes.
> > > >>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> I am still worried about the custom SBI extension required
> > > >>>>>>>>>> by
> > > >> EDK2.
> > > >>>>>>>>>> This will not work when running EDK2 inside Guest/VM
> > > because
> > > >>>>>>>>>> Guest/VM boots in VS-mode and the SBI calls are provided
> by
> > > >>>>>>>>>> hypervisors (KVM, Xvisor, etc). I think you should revisit
> > > >>>>>>>>>> EDK2 boot-flow to make it compatible with virtualization
> > > >>>>>>>>>> and OpenSBI
> > > >>>>>>> domains.
> > > >>>>>>>>>
> > > >>>>>>>>> Ok, I will revisit this. Thanks for the reminder.
> > > >>>>>>>>
> > > >>>>>>>> Hi Anup,
> > > >>>>>>>> I have few questions regard to HSM support in opensbi,
> > > >>>>>>>>
> > > >>>>>>>> - Is the purpose of invoking platform_domain_init at the end
> > > >>>>>>>> of
> > > >>>>>>>> sbi_init() to let platform code to register the domains
> > > >>>>>>>> through
> > > >>>>>> sbi_domain_register()?
> > > >>>>>>>
> > > >>>>>>> Yes, domains need to be populated as late as possible so that
> > > >>>>>>> domains are switched only after all initialization is completed.
> > > >>>>>>>
> > > >>>>>>>> - What is the reason that each domain is requested to have
> > > >>>>>>>> the memory region of ROOT_FW_REGION?
> > > >>>>>>>
> > > >>>>>>> The ROOT_FW_REGION protects the firmware itself. The
> > > fw_region is
> > > >>>>>>> based on fw_start and fw_size members of the "struct
> > sbi_scratch".
> > > >>>>>>>
> > > >>>>>>>> - I think opensbi will have the implement of switching the
> > > >>>>>>>> next mode to
> > > >>>>>>> HSM
> > > >>>>>>>> later?
> > > >>>>>>>
> > > >>>>>>> Can you elaborate why you need this ?
> > > >>>>>> Ah no I was just wonder how opensbi switch the next mode to
> > HSM.
> > > >>>>>> You
> > > >>> had
> > > >>>>>> the answer in below.
> > > >>>>>>>
> > > >>>>>>>> - How does opensbi loads the hypervisor in HSM?
> > > >>>>>>>
> > > >>>>>>> When H-extension is available the next mode is automatically
> > > >>>>>>> HS-mode (i.e. S-mode with virt=off).
> > > >>>>>>>
> > > >>>>>>> You still did not share how you will make EDK2 boot flow work
> > > >>>>>>> for VS-modes because hypervisor will start Guest/VM directly
> > > >>>>>>> in VS-mode and M-mode components of EDK2 can't run inside
> > > >> Guest/VM
> > > >>>>>>
> > > >>>>>> I don't have the full picture of EDK2 boot flow for HSM yet. I
> > > >>>>>> am stillthinking how the Management Mode work on HSM
> > especially
> > > >>>>>> to the multiple
> > > >>> type1
> > > >>>>>
> > > >>>>> HS-mode and VS-mode are both S-mode with differing capabilities.
> > > >>>>> A software written for S-mode (without using H-extension CSRs)
> > > >>>>> should run
> > > >>> unmodified
> > > >>>>> in both HS-mode and VS-mode.
> > > >>>>>
> > > >>>>>> hypervisors run on the each domain. Also, the platform errors
> > > >>>>>> or RAS errors on server platform has to deliver the error event
> > > >>>>>> to the corresponding
> > > >>>>>
> > > >>>>> This headache of hypervisor so hypervisor will use appropriate
> > > >>>>> HW features to virtualize RAS events/errors.
> > > >>>>>
> > > >>>>>> domains. The idea as I mentioned earlier, EDK2 FW is not
> > > >>>>>> necessarily to be
> > > >>>>>
> > > >>>>> I totally disagree. In ARM64 world, people use EDK2 inside
> > > >>>>> Guest/VM so why can't we do the same in RISC-V world.
> > > >>>>>
> > > >>>>> Same distros which run natively without hypervisor will expect
> > > >>>>> to run in the same way inside Guest/VM. Are you suggesting that
> > > >>>>> EDK2 will not be available to distros inside Guest/VM ??
> > > >>>>>
> > > >>>>>> executed as a VS entity. EDK2 FW is executed when processor
> > > power
> > > >>>>>> on and it uses opensbi as a library to initial the basic
> > > >>>>>> platform
> > > >>>>>> (vendor) and opensbi initialization, then edk2 FW run through
> > > >>>>>> PEI/DXE for the OEM platform and proprietary features
> > > >>>>>> initialization. EDK2 still can jump to opensbi to run the
> > > >>>>>> domain initialization at the last boot stage, says BDS,  and
> > > >>>>>> then switch to HSM. We have to consider those server features
> > > >>>>>> such as critical platform error handling, event logging,
> > > >>>>>> FW<->BMC communication, Remote FW configuration through
> > Redfish
> > > >>>>>> beyond the HSM, or
> > > some of
> > > >>>>>> above drivers can run in HSM before hypervisor is launched, I
> > > >>>>>> am not sure yet.
> > > >>>>>
> > > >>>>> If other architectures are able to use EDK2 inside Guest/VM then
> > > >>>>> I don't see why we can't do the same for RISC-V.
> > > >>>>>
> > > >>>>>> SBI FW extension shouldn't be the problem because HSM
> > > >>>>>> hypervisor can still bypass the sbi invocations from VS entity
> > > >>>>>> to M-mode if the sbi function number is in the firmware range,
> > > >>>>>> right? e.g. SBI FW extension can stillprovide the management
> > > >>>>>> mode interface to (V)S mode entity.
> > > >>>>>
> > > >>>>> Bypassing SBI FW calls from VS-mode to M-mode will have it's own
> > > >>>>> issues and I am reluctant to go this direction without fully
> > > >>>>> understanding why
> > > >>>>> EDK2 needs SBI FW calls.
> > > >>>>
> > > >>>> EDK II (and U-Boot) need the SBI system reset extension to
> > > >>>> implement the
> > > >>>> ResetSystem() runtime service which operating systems use to
> > > >>>> reboot or poweroff.
> > > >>>>
> > > >>>> The hypervisor must provide an SBI implementation running in HS
> > > mode.
> > > >>>> This is why we have trap delegation registers:
> > > >>>>
> > > >>>> "When a trap occurs in HS-mode or U-mode, it goes to M-mode,
> > > >>>> unless delegated by medeleg or mideleg, in which case it goes to
> HS-
> > mode.
> > > >>>> When a trap occurs in VS-mode or VU-mode, it goes to M-mode,
> > > unless
> > > >>> delegated
> > > >>>> by medeleg or mideleg, in which case it goes to HS-mode, unless
> > > >>>> further delegated by hedeleg or hideleg, in which case it goes to
> > > >>>> VS-
> > > >> mode."
> > > >>>>
> > > >>>> EDK II will run in VS mode. EDK II's ecalls will be handled by
> > > >>>> the hypervisor's SBI implementation.
> > > >>>>
> > > >>>> It is not important if EDK II is compiled with or without the
> > > >>>> OpenSBI library. In both cases the hypervisor must invoke EDK
> > > >>>> II's PEI entry point and not enter EDK2's SEC phase (cf.
> > > >>>> INVALID URI REMOVED
> > > >>>
> > >
> specification/2_design_discussion/23_boot_sequence__;!!NpxR!11uLLL9PJI
> > > >>> r gauJtSLgUAqsEYMnbRf43C_kxGqaYi7ZLodCE40XfAtUXp4R9brg$ ).
> > > >>>>
> > > >>>>
> > > >>>> On ARM EDK II is a BL33 payload of TF-A. TF-A is not in the code
> > > >>>> base of EDK II.
> > > >>>>
> > > >>>> We should do the same on RISC-V: Remove OpenSBI from the EDK
> II
> > > code
> > > >>>> base and compile upstream OpenSBI with EDK II as FW_PAYLOAD.
> > > >>>>
> > > >>>> This way we don't have to worry about differences between EDK II
> > > >>>> running in S-mode and VS-mode.
> > > >>>>
> > > >>>> @Rick, Bin
> > > >>>> The same will have to be done for U-Boot's SBI support. We should
> > > >>>> not require SPL starting OpenSBI to have SBI support on
> QEMU/KVM.
> > > >>>>
> > > >>>> Best regards
> > > >>>>
> > > >>>> Heinrich
> > > >>>>
> > > >>>>>
> > > >>>>> The fact that you need to depend on SBI FW extension seems to
> be
> > > >>> becoming
> > > >>>>> a road block for EDK2 inside Guest/VM.
> > > >>>>>
> > > >>>>> Regards,
> > > >>>>> Anup
> > > >>>>>
> > > >>>>>>
> > > >>>>>> Abner
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>>
> > > >>>>>>> Regards,
> > > >>>>>>> Anup
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Regards,
> > > >>>>>>>> Abner
> > > >>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> Regards,
> > > >>>>>>>>> Abner
> > > >>>>>>>>>>
> > > >>>>>>>>>> Regards,
> > > >>>>>>>>>> Anup
> > > >>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Thanks and regards,
> > > >>>>>>>>>>> Abner
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Next booting stage has to run from lower privilege
> mode
> > > >>>>>>>>>>>>> (S-mode or
> > > >>>>>>>>>>>>> U-
> > > >>>>>>>>>>>>> mode) otherwise OpenSBI cannot protect itself from
> next
> > > >>>>>>>>>>>>> booting stage if it starts in M-mode.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> However, sbi_init when directly initializing to S-Mode
> > > >>>>>>>>>>>>>>> checks that the
> > > >>>>>>>>>>>>>> start_address is executable.
> > > >>>>>>>>>>>>>>> So I'm wondering why the FW region isn't set as
> > > >>>>>>>>>>>>>>> executable in
> > > >>>>>>>>>>> OpenSBI?
> > > >>>>>>>>>>>>>>> How do other FWs like U-Boot get around this?
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>
> > > >>
> > >
> > https://github.com/riscv/opensbi/commit/b1678af210dc4b4e6d586d6d966
> > > >>>>>>>> 1
> > > >>>>>>>>>>>>> 7
> > > >>>>>>>>>>>>>> e
> > > >>>>>>>>>>>>>>> 9641618994#diff-
> > > >>>>>>>>>>>>>>
> > > >>>>>> 6e8e352a8a90ba5a7adbb58a806ed9b6404c2c67db416332f9c05a
> > > >>>>>>>>>>>>>>> 6b322eecd6R346
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> If I try to set my own regions by adding
> > > >>>>>>>>>>>>>>> .domains_root_regions I get another error because
> > > OpenSBI
> > > >>>>>>>>>>>>>>> checks that I have a region that is the same as the FW
> > > >>>>>>>>>>>>>>> region added by OpenSBI. If I duplicate the FW
> region
> > > and
> > > >>>>>>>>>>>>>>> mark the first one as executable I can pass the
> > > >>>>>>>>>>>>>>> executable check and also the check that there's an
> > > >>>>>>>>> FW
> > > >>>>>>>>>>> region.
> > > >>>>>>>>>>>>>>> Additionally we have to manually call pmp_set in our
> > > >>>>>>>>>>>>>>> custom platform to make the FW region RWX.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> That seems like a workaround, however. Do you
> have
> > > any
> > > >>>>>>>>>>>>>>> suggestion to
> > > >>>>>>>>>>>>>> properly fix it?
> > > >>>>>>>>>>>>>>> I'm sure we're misunderstand something.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I suggest two things:
> > > >>>>>>>>>>>>> 1) Register your custom SBI extension from M-mode
> only
> > > >>>>>>>>>>>>> before switching to S-mode
> > > >>>>>>>>>>>>> 2) Make sure that fw_start and fw_size set in the
> > > >>>>>>>>>>>>> sbi_scratch for each HART only point to the M-mode
> code
> > > and
> > > >>>>>> data.
> > > >>>>>>>>>>>>> Preferably have S-mode code and data not linked in the
> > > same
> > > >>>>>>>>>>>>> binary as M-mode code
> > > >>>>>>>>> and
> > > >>>>>>>>>>> data.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> For context: We're writing the start addr and size of
> > > >>>>>>>>>>>>>>> our FW image into the scratch space before OpenSBI
> is
> > > >>>>>> initialized.
> > > >>>>>>>>>>>>>>> Therefore we're expecting it to set the PMP settings
> > > >>>>>> correctly.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Please check out my workaround commit:
> > > >>>>>>>>>>>>>>> https://github.com/riscv/riscv-edk2-
> > > >>>>>>>>> platforms/commit/a5ac63096ca
> > > >>>>>>>>>>>>>>> 5da7
> > > >>>>>>>>>>>>>>> 95
> > > >>>>>>>>>>>>>>> 042baf650170643fe219cab
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Thanks,
> > > >>>>>>>>>>>>>>> Daniel
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> Regards
> > > >>>>>>>>>>>>> Anup
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Regards,
> > > >>>>>>>>>>>> Anup
> > > >>>>>>>>>
> > > >>>>>>>>> --
> > > >>>>>>>>> opensbi mailing list
> > > >>>>>>>>> opensbi at lists.infradead.org
> > > >>>>>>>>> http://lists.infradead.org/mailman/listinfo/opensbi
> > > >>>>
> > > >