Why does the firmware memory region have no permissions?

[Anup Patel]

> -----Original Message-----
> From: Heinrich Schuchardt <xypron.glpk at gmx.de>
> Sent: 03 June 2021 22:21
> To: Anup Patel <Anup.Patel at wdc.com>; Chang, Abner (HPS SW/FW
> Technologist) <abner.chang at hpe.com>; Daniel Schaefer
> <daniel at danielschaefer.me>
> Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>; Rick
> Chen <rick at andestech.com>; Bin Meng <bmeng.cn at gmail.com>
> Subject: Re: Why does the firmware memory region have no permissions?
> 
> On 6/3/21 5:34 PM, Anup Patel wrote:
> > +Heinrich
> >
> >> -----Original Message-----
> >> From: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>
> >> Sent: 03 June 2021 20:05
> >> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >> <daniel at danielschaefer.me>
> >> Cc: opensbi at lists.infradead.org
> >> Subject: RE: Why does the firmware memory region have no permissions?
> >>
> >>
> >>
> >>> -----Original Message-----
> >>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>> Sent: Wednesday, June 2, 2021 11:15 PM
> >>> To: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>;
> >>> Daniel Schaefer <daniel at danielschaefer.me>
> >>> Cc: opensbi at lists.infradead.org
> >>> Subject: RE: Why does the firmware memory region have no
> permissions?
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Chang, Abner (HPS SW/FW Technologist)
> <abner.chang at hpe.com>
> >>>> Sent: 02 June 2021 20:19
> >>>> To: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>;
> >>> Anup
> >>>> Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>> <daniel at danielschaefer.me>
> >>>> Cc: opensbi at lists.infradead.org
> >>>> Subject: RE: Why does the firmware memory region have no
> permissions?
> >>>>
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: opensbi [mailto:opensbi-bounces at lists.infradead.org] On
> >>>>> Behalf Of Chang, Abner (HPS SW/FW Technologist)
> >>>>> Sent: Saturday, May 15, 2021 11:30 PM
> >>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>>>> <daniel at danielschaefer.me>
> >>>>> Cc: opensbi at lists.infradead.org
> >>>>> Subject: RE: Why does the firmware memory region have no
> >> permissions?
> >>>>>
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>>>>> Sent: Saturday, May 15, 2021 4:09 PM
> >>>>>> To: Chang, Abner (HPS SW/FW Technologist)
> >> <abner.chang at hpe.com>;
> >>>>> Daniel
> >>>>>> Schaefer <daniel at danielschaefer.me>
> >>>>>> Cc: opensbi at lists.infradead.org
> >>>>>> Subject: RE: Why does the firmware memory region have no
> >>> permissions?
> >>>>>>
> >>>>>> Hi Abner,
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Chang, Abner (HPS SW/FW Technologist)
> >>>>> <abner.chang at hpe.com>
> >>>>>>> Sent: 15 May 2021 11:47
> >>>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>>>>>> <daniel at danielschaefer.me>
> >>>>>>> Cc: opensbi at lists.infradead.org
> >>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>> permissions?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>>>>>>> Sent: Friday, May 14, 2021 7:58 PM
> >>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
> >>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>> Technologist)
> >>>>>>>> <abner.chang at hpe.com>
> >>>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>>> permissions?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Anup Patel
> >>>>>>>>> Sent: 14 May 2021 17:22
> >>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
> >>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>>>>>> Technologist) <abner.chang at hpe.com>
> >>>>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>>>>> permissions?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: opensbi <opensbi-bounces at lists.infradead.org> On
> Behalf
> >>>>> Of
> >>>>>>>>>> Daniel Schaefer
> >>>>>>>>>> Sent: 13 May 2021 10:26
> >>>>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>
> >>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>>>>>>> Technologist) <abner.chang at hpe.com>
> >>>>>>>>>> Subject: Why does the firmware memory region have no
> >>>>> permissions?
> >>>>>>>>>>
> >>>>>>>>>> Whoops, put CC as subject...
> >>>>>>>>>>
> >>>>>>>>>> On 5/12/21 6:20 PM, Daniel Schaefer wrote:
> >>>>>>>>>>> Hi Anup,
> >>>>>>>>>>>
> >>>>>>>>>>> I'm in the process of upgrading EDKII to OpenSBI 0.9 and
> >>>>>>>>>>> using the Generic
> >>>>>>>>>> Platform.
> >>>>>>>>>>>
> >>>>>>>>>>> Previously we were doing sbi_init with M-Mode, adding our
> >>>>>>>>>>> SBI extension and then calling sbi_switch_mode to switch to
> >>>>>>>>>>> S-
> >>>> Mode.
> >>>>>>>>>>> Now sbi_init disallows initializing to M-Mode, so I'm
> >>>>>>>>>>> directly switching to S-Mode. It seems that even from S-Mode
> >>>>>>>>>>> I can register our SBI
> >>>>>>>>>> extension with sbi_ecall_register_extension.
> >>>>>>>>>>> Is that correct?
> >>>>>>>>>
> >>>>>>>>> The OpenSBI sources are meant to run only from M-mode so we
> >>>>> cannot
> >>>>>>>>> register SBI extension using sbi_ecall_register_extension().
> >>>>>>>>>
> >>>>>>>>> The  sbi_switch_mode() is not stricter due to OpenSBI domain
> >>>>> support.
> >>>>>>>>
> >>>>>>>> The sbi_hart_switch_mode() is fine. It's the
> >>>>>>>> sbi_domain_init() which is enforcing next booting stage to be
> >>>>>>>> at lower privilege for
> >>> root
> >>>> domain.
> >>>>>>>>
> >>>>>>>> For time being, you can try removing checks on "dom-
> >>> next_mode"
> >>>>>>>> in sanitize_domain()
> >>>>>>>
> >>>>>>> Hi Anup, I think we currently skip that check for moving on the
> >>>>>>> edk2 boot process. So do you have plan to remove this check?
> >>>>>>> Or any
> >>>> alternative?
> >>>>>>> I think it is unnecessary having this check on the next
> >>>>>>> privilege
> >> mode.
> >>>>> That
> >>>>>>> should be at OEM discretion of which privilege mode to run their
> >>>>>>> next firmware stage based on the platform design?
> >>>>>>
> >>>>>> This is an important check required by OpenSBI domain support so
> >>>>>> that next booting stage cannot tamper with PMP configuration (and
> >>>>>> other security configuration) done by OpenSBI.
> >>>>>
> >>>>> I understand the importance of not giving any chance to tamper PMP
> >>>>> setting, however this could be the responsibility of the next boot
> >>>>> phase
> >>>> before OS.
> >>>>> OpenSBI as the early phase boot firmware should be generally
> >>>>> provide SBIs to platform variants, and have the flexibility to
> >>>>> hand off to either M-mode or S-mode firmware (Actually I don't
> >>>>> think OpenSBI
> >>> should
> >>>> handle this).
> >>>>> Platform code is provided by OEM/vendor, OpenSBI should allow it
> >>>>> if platform code says I would like to run my next phase firmware
> >>>>> in M-
> >>> mode.
> >>>>> We restrict the privilege phase for the next phase in OpenSBI also
> >>>>> not compliant with the UEFI spec which says UEFI RISC-V firmware
> >>>>> could be executed in either M-mode or S-mode. Some EFI driver may
> >>>>> be loaded in
> >>>>> S- mode but provide the M-mode code such as management mode,
> the
> >>>>> Platform Runtime Mechanism or some other use cases. The next
> >>> firmware
> >>>>> has the responsibility to switch to S-mode when handoff to OS or
> >>>>> software if the platform design requires that (I remember we have
> >>>>> the simi  lar sentence in riscv-platform-spec). EDK2 code can't
> >>>>> just remove the check "dom->next_mode", we use OpenSBI without
> any
> >>>> changes.
> >>>>>
> >>>>>>
> >>>>>> I am still worried about the custom SBI extension required by EDK2.
> >>>>>> This will not work when running EDK2 inside Guest/VM because
> >>>>>> Guest/VM boots in VS-mode and the SBI calls are provided by
> >>>>>> hypervisors (KVM, Xvisor, etc). I think you should revisit EDK2
> >>>>>> boot-flow to make it compatible with virtualization and OpenSBI
> >>> domains.
> >>>>>
> >>>>> Ok, I will revisit this. Thanks for the reminder.
> >>>>
> >>>> Hi Anup,
> >>>> I have few questions regard to HSM support in opensbi,
> >>>>
> >>>> - Is the purpose of invoking platform_domain_init at the end of
> >>>> sbi_init() to let platform code to register the domains through
> >> sbi_domain_register()?
> >>>
> >>> Yes, domains need to be populated as late as possible so that
> >>> domains are switched only after all initialization is completed.
> >>>
> >>>> - What is the reason that each domain is requested to have the
> >>>> memory region of ROOT_FW_REGION?
> >>>
> >>> The ROOT_FW_REGION protects the firmware itself. The fw_region is
> >>> based on fw_start and fw_size members of the "struct sbi_scratch".
> >>>
> >>>> - I think opensbi will have the implement of switching the next
> >>>> mode to
> >>> HSM
> >>>> later?
> >>>
> >>> Can you elaborate why you need this ?
> >> Ah no I was just wonder how opensbi switch the next mode to HSM. You
> >> had the answer in below.
> >>>
> >>>> - How does opensbi loads the hypervisor in HSM?
> >>>
> >>> When H-extension is available the next mode is automatically HS-mode
> >>> (i.e. S-mode with virt=off).
> >>>
> >>> You still did not share how you will make EDK2 boot flow work for
> >>> VS-modes because hypervisor will start Guest/VM directly in VS-mode
> >>> and M-mode components of EDK2 can't run inside Guest/VM
> >>
> >> I don't have the full picture of EDK2 boot flow for HSM yet. I am
> >> still thinking how the Management Mode work on HSM especially to the
> >> multiple type1
> >
> > HS-mode and VS-mode are both S-mode with differing capabilities. A
> > software written for S-mode (without using H-extension CSRs) should
> > run unmodified in both HS-mode and VS-mode.
> >
> >> hypervisors run on the each domain. Also, the platform errors or RAS
> >> errors on server platform has to deliver the error event to the
> >> corresponding
> >
> > This headache of hypervisor so hypervisor will use appropriate HW
> > features to virtualize RAS events/errors.
> >
> >> domains. The idea as I mentioned earlier, EDK2 FW is not necessarily
> >> to be
> >
> > I totally disagree. In ARM64 world, people use EDK2 inside Guest/VM so
> > why can't we do the same in RISC-V world.
> >
> > Same distros which run natively without hypervisor will expect to run
> > in the same way inside Guest/VM. Are you suggesting that EDK2 will not
> > be available to distros inside Guest/VM ??
> >
> >> executed as a VS entity. EDK2 FW is executed when processor power on
> >> and it uses opensbi as a library to initial the basic platform
> >> (vendor) and opensbi initialization, then edk2 FW run through PEI/DXE
> >> for the OEM platform and proprietary features initialization. EDK2
> >> still can jump to opensbi to run the domain initialization at the
> >> last boot stage, says BDS,  and then switch to HSM. We have to
> >> consider those server features such as critical platform error
> >> handling, event logging, FW<->BMC communication, Remote FW
> >> configuration through Redfish beyond the HSM, or some of above drivers
> can run in HSM before hypervisor is launched, I am not sure yet.
> >
> > If other architectures are able to use EDK2 inside Guest/VM then I
> > don't see why we can't do the same for RISC-V.
> >
> >> SBI FW extension shouldn't be the problem because HSM hypervisor can
> >> still bypass the sbi invocations from VS entity to M-mode if the sbi
> >> function number is in the firmware range, right? e.g. SBI FW
> >> extension can still provide the management mode interface to (V)S mode
> entity.
> >
> > Bypassing SBI FW calls from VS-mode to M-mode will have it's own
> > issues and I am reluctant to go this direction without fully
> > understanding why
> > EDK2 needs SBI FW calls.
> 
> EDK II (and U-Boot) need the SBI system reset extension to implement the
> ResetSystem() runtime service which operating systems use to reboot or
> poweroff.
> 
> The hypervisor must provide an SBI implementation running in HS mode.
> This is why we have trap delegation registers:
> 
> "When a trap occurs in HS-mode or U-mode, it goes to M-mode, unless
> delegated by medeleg or mideleg, in which case it goes to HS-mode. When a
> trap occurs in VS-mode or VU-mode, it goes to M-mode, unless delegated by
> medeleg or mideleg, in which case it goes to HS-mode, unless further
> delegated by hedeleg or hideleg, in which case it goes to VS-mode."
> 
> EDK II will run in VS mode. EDK II's ecalls will be handled by the hypervisor's
> SBI implementation.
> 
> It is not important if EDK II is compiled with or without the OpenSBI library. In
> both cases the hypervisor must invoke EDK II's PEI entry point and not enter
> EDK2's SEC phase (cf.
> https://edk2-docs.gitbook.io/edk-ii-build-
> specification/2_design_discussion/23_boot_sequence).
> 
> On ARM EDK II is a BL33 payload of TF-A. TF-A is not in the code base of EDK II.
> 
> We should do the same on RISC-V: Remove OpenSBI from the EDK II code base
> and compile upstream OpenSBI with EDK II as FW_PAYLOAD.
> 
> This way we don't have to worry about differences between EDK II running in
> S-mode and VS-mode.
> 
> @Rick, Bin
> The same will have to be done for U-Boot's SBI support. We should not
> require SPL starting OpenSBI to have SBI support on QEMU/KVM.

The U-Boot S-mode (or U-Boot proper) is totally independent of
who starts OpenSBI because on SiFive Unleashed we have two
boot-flows:
SiFive FSBL => OpenSBI => U-Boot S-mode
U-Boot SPL => OpenSBI => U-Boot S-mode

Although, I have never tested U-Boot S-mode inside KVM RISC-V
or Xvisor RISC-V Guest but I won't be surprised if it just works.

Regards,
Anup

> 
> Best regards
> 
> Heinrich
> 
> >
> > The fact that you need to depend on SBI FW extension seems to be
> > becoming a road block for EDK2 inside Guest/VM.
> >
> > Regards,
> > Anup
> >
> >>
> >> Abner
> >>
> >>
> >>>
> >>> Regards,
> >>> Anup
> >>>
> >>>>
> >>>> Regards,
> >>>> Abner
> >>>>
> >>>>>
> >>>>> Regards,
> >>>>> Abner
> >>>>>>
> >>>>>> Regards,
> >>>>>> Anup
> >>>>>>
> >>>>>>>
> >>>>>>> Thanks and regards,
> >>>>>>> Abner
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Next booting stage has to run from lower privilege mode
> >>>>>>>>> (S-mode or
> >>>>>>>>> U-
> >>>>>>>>> mode) otherwise OpenSBI cannot protect itself from next
> >>>>>>>>> booting stage if it starts in M-mode.
> >>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> However, sbi_init when directly initializing to S-Mode
> >>>>>>>>>>> checks that the
> >>>>>>>>>> start_address is executable.
> >>>>>>>>>>> So I'm wondering why the FW region isn't set as executable
> >>>>>>>>>>> in
> >>>>>>> OpenSBI?
> >>>>>>>>>>> How do other FWs like U-Boot get around this?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>
> https://github.com/riscv/opensbi/commit/b1678af210dc4b4e6d586d6d966
> >>>> 1
> >>>>>>>>> 7
> >>>>>>>>>> e
> >>>>>>>>>>> 9641618994#diff-
> >>>>>>>>>>
> >> 6e8e352a8a90ba5a7adbb58a806ed9b6404c2c67db416332f9c05a
> >>>>>>>>>>> 6b322eecd6R346
> >>>>>>>>>>>
> >>>>>>>>>>> If I try to set my own regions by adding
> >>>>>>>>>>> .domains_root_regions I get another error because OpenSBI
> >>>>>>>>>>> checks that I have a region that is the same as the FW
> >>>>>>>>>>> region added by OpenSBI. If I duplicate the FW region and
> >>>>>>>>>>> mark the first one as executable I can pass the executable
> >>>>>>>>>>> check and also the check that there's an
> >>>>> FW
> >>>>>>> region.
> >>>>>>>>>>> Additionally we have to manually call pmp_set in our custom
> >>>>>>>>>>> platform to make the FW region RWX.
> >>>>>>>>>>>
> >>>>>>>>>>> That seems like a workaround, however. Do you have any
> >>>>>>>>>>> suggestion to
> >>>>>>>>>> properly fix it?
> >>>>>>>>>>> I'm sure we're misunderstand something.
> >>>>>>>>>
> >>>>>>>>> I suggest two things:
> >>>>>>>>> 1) Register your custom SBI extension from M-mode only before
> >>>>>>>>> switching to S-mode
> >>>>>>>>> 2) Make sure that fw_start and fw_size set in the sbi_scratch
> >>>>>>>>> for each HART only point to the M-mode code and
> >> data.
> >>>>>>>>> Preferably have S-mode code and data not linked in the same
> >>>>>>>>> binary as M-mode code
> >>>>> and
> >>>>>>> data.
> >>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> For context: We're writing the start addr and size of our FW
> >>>>>>>>>>> image into the scratch space before OpenSBI is
> >> initialized.
> >>>>>>>>>>> Therefore we're expecting it to set the PMP settings
> >> correctly.
> >>>>>>>>>>>
> >>>>>>>>>>> Please check out my workaround commit:
> >>>>>>>>>>> https://github.com/riscv/riscv-edk2-
> >>>>> platforms/commit/a5ac63096ca
> >>>>>>>>>>> 5da7
> >>>>>>>>>>> 95
> >>>>>>>>>>> 042baf650170643fe219cab
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Daniel
> >>>>>>>>>
> >>>>>>>>> Regards
> >>>>>>>>> Anup
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Anup
> >>>>>
> >>>>> --
> >>>>> opensbi mailing list
> >>>>> opensbi at lists.infradead.org
> >>>>> http://lists.infradead.org/mailman/listinfo/opensbi