Why does the firmware memory region have no permissions?

Chang, Abner (HPS SW/FW Technologist) abner.chang at hpe.com
Thu Jun 3 23:30:20 PDT 2021 


> -----Original Message-----
> From: Heinrich Schuchardt [mailto:xypron.glpk at gmx.de]
> Sent: Friday, June 4, 2021 1:15 AM
> To: Anup Patel <Anup.Patel at wdc.com>; Chang, Abner (HPS SW/FW
> Technologist) <abner.chang at hpe.com>; Daniel Schaefer
> <daniel at danielschaefer.me>
> Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>
> Subject: Re: Why does the firmware memory region have no permissions?
> 
> On 6/3/21 6:51 PM, Heinrich Schuchardt wrote:
> > On 6/3/21 5:34 PM, Anup Patel wrote:
> >> +Heinrich
> >>
> >>> -----Original Message-----
> >>> From: Chang, Abner (HPS SW/FW Technologist)
> <abner.chang at hpe.com>
> >>> Sent: 03 June 2021 20:05
> >>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>> <daniel at danielschaefer.me>
> >>> Cc: opensbi at lists.infradead.org
> >>> Subject: RE: Why does the firmware memory region have no
> permissions?
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>>> Sent: Wednesday, June 2, 2021 11:15 PM
> >>>> To: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>;
> >>>> Daniel Schaefer <daniel at danielschaefer.me>
> >>>> Cc: opensbi at lists.infradead.org
> >>>> Subject: RE: Why does the firmware memory region have no
> permissions?
> >>>>
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Chang, Abner (HPS SW/FW Technologist)
> <abner.chang at hpe.com>
> >>>>> Sent: 02 June 2021 20:19
> >>>>> To: Chang, Abner (HPS SW/FW Technologist)
> <abner.chang at hpe.com>;
> >>>> Anup
> >>>>> Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>>> <daniel at danielschaefer.me>
> >>>>> Cc: opensbi at lists.infradead.org
> >>>>> Subject: RE: Why does the firmware memory region have no
> permissions?
> >>>>>
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: opensbi [mailto:opensbi-bounces at lists.infradead.org] On
> >>>>>> Behalf Of Chang, Abner (HPS SW/FW Technologist)
> >>>>>> Sent: Saturday, May 15, 2021 11:30 PM
> >>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>>>>> <daniel at danielschaefer.me>
> >>>>>> Cc: opensbi at lists.infradead.org
> >>>>>> Subject: RE: Why does the firmware memory region have no
> >>> permissions?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>>>>>> Sent: Saturday, May 15, 2021 4:09 PM
> >>>>>>> To: Chang, Abner (HPS SW/FW Technologist)
> >>> <abner.chang at hpe.com>;
> >>>>>> Daniel
> >>>>>>> Schaefer <daniel at danielschaefer.me>
> >>>>>>> Cc: opensbi at lists.infradead.org
> >>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>> permissions?
> >>>>>>>
> >>>>>>> Hi Abner,
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Chang, Abner (HPS SW/FW Technologist)
> >>>>>> <abner.chang at hpe.com>
> >>>>>>>> Sent: 15 May 2021 11:47
> >>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>>>>>>> <daniel at danielschaefer.me>
> >>>>>>>> Cc: opensbi at lists.infradead.org
> >>>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>>> permissions?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>>>>>>>> Sent: Friday, May 14, 2021 7:58 PM
> >>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
> >>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>>> Technologist)
> >>>>>>>>> <abner.chang at hpe.com>
> >>>>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>>>> permissions?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: Anup Patel
> >>>>>>>>>> Sent: 14 May 2021 17:22
> >>>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
> >>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>>>>>>> Technologist) <abner.chang at hpe.com>
> >>>>>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>>>>>> permissions?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: opensbi <opensbi-bounces at lists.infradead.org> On
> >>>>>>>>>>> Behalf
> >>>>>> Of
> >>>>>>>>>>> Daniel Schaefer
> >>>>>>>>>>> Sent: 13 May 2021 10:26
> >>>>>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>
> >>>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>>>>>>>> Technologist) <abner.chang at hpe.com>
> >>>>>>>>>>> Subject: Why does the firmware memory region have no
> >>>>>> permissions?
> >>>>>>>>>>>
> >>>>>>>>>>> Whoops, put CC as subject...
> >>>>>>>>>>>
> >>>>>>>>>>> On 5/12/21 6:20 PM, Daniel Schaefer wrote:
> >>>>>>>>>>>> Hi Anup,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm in the process of upgrading EDKII to OpenSBI 0.9
> >>>>>>>>>>>> and using the Generic
> >>>>>>>>>>> Platform.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Previously we were doing sbi_init with M-Mode, adding
> >>>>>>>>>>>> our SBI extension and then calling sbi_switch_mode to
> >>>>>>>>>>>> switch to S-
> >>>>> Mode.
> >>>>>>>>>>>> Now sbi_init disallows initializing to M-Mode, so I'm
> >>>>>>>>>>>> directly switching to S-Mode. It seems that even from
> >>>>>>>>>>>> S-Mode I can register our SBI
> >>>>>>>>>>> extension with sbi_ecall_register_extension.
> >>>>>>>>>>>> Is that correct?
> >>>>>>>>>>
> >>>>>>>>>> The OpenSBI sources are meant to run only from M-mode so
> >>>>>>>>>> we
> >>>>>> cannot
> >>>>>>>>>> register SBI extension using sbi_ecall_register_extension().
> >>>>>>>>>>
> >>>>>>>>>> The  sbi_switch_mode() is not stricter due to OpenSBI
> >>>>>>>>>> domain
> >>>>>> support.
> >>>>>>>>>
> >>>>>>>>> The sbi_hart_switch_mode() is fine. It's the
> >>>>>>>>> sbi_domain_init() which is enforcing next booting stage to
> >>>>>>>>> be at lower privilege for
> >>>> root
> >>>>> domain.
> >>>>>>>>>
> >>>>>>>>> For time being, you can try removing checks on "dom-
> >>>> next_mode"
> >>>>>>>>> in sanitize_domain()
> >>>>>>>>
> >>>>>>>> Hi Anup, I think we currently skip that check for moving on
> >>>>>>>> the
> >>>>>>>> edk2 boot process. So do you have plan to remove this check?
> >>>>>>>> Or any
> >>>>> alternative?
> >>>>>>>> I think it is unnecessary having this check on the next privilege
> >>> mode.
> >>>>>> That
> >>>>>>>> should be at OEM discretion of which privilege mode to run
> >>>>>>>> their next firmware stage based on the platform design?
> >>>>>>>
> >>>>>>> This is an important check required by OpenSBI domain support so
> >>>>>>> that next booting stage cannot tamper with PMP configuration
> >>>>>>> (and other security configuration) done by OpenSBI.
> >>>>>>
> >>>>>> I understand the importance of not giving any chance to tamper
> PMP
> >>>>>> setting, however this could be the responsibility of the next boot
> >>>>>> phase
> >>>>> before OS.
> >>>>>> OpenSBI as the early phase boot firmware should be generally
> >>>>>> provide SBIs to platform variants, and have the flexibility to
> >>>>>> hand off to either M-mode or S-mode firmware (Actually I don't
> >>>>>> think OpenSBI
> >>>> should
> >>>>> handle this).
> >>>>>> Platform code is provided by OEM/vendor, OpenSBI should allow it
> >>>>>> if platform code says I would like to run my next phase firmware
> >>>>>> in M-
> >>>> mode.
> >>>>>> We restrict the privilege phase for the next phase in OpenSBI also
> >>>>>> not compliant with the UEFI spec which says UEFI RISC-V firmware
> >>>>>> could be executed in either M-mode or S-mode. Some EFI driver
> may
> >>>>>> be loaded in
> >>>>>> S- mode but provide the M-mode code such as management mode,
> the
> >>>>>> Platform Runtime Mechanism or some other use cases. The next
> >>>> firmware
> 
> Compare this to EDK II with TF-A. No EDK II code runs in EL3. TF-A is
> the only EL3 code.
> 
> Same for RISC-V: OpenSBI is the M-mode firmware. No other part of the
> firmware shall run in M-mode. I cannot see that the UEFI specification
> requires a different segregation of duties.
That paragraph stated in UEFI spec is based on RISC-V privilege spec, which says all platforms must implement M-mode.
The simplest platform can only has M-mode. That makes sense to say UEFI firmware can run on either M-mode or S-mode.
I also curious about why opensbi restricts to switch the next mode to M-mode. How does opensbi runs on the platform only
provides M-mode?

> 
> What RISC-V is lacking up to now is run modes for trusted execution
> environments like OP-TEE. This is where you would place management mode
> code and not in M-mode.
Yes, that is the plan to have management mode runs in TEE, do you have suggestion how do we load the MM driver to TEE from S-mode, in which phase we can we initial the MM environment and load the MM driver to TEE through DXE dispatcher?  and all under S-mode.
Is any of S-mode entities can load itself to TEE?

Regards,
Abner

> 
> Best regards
> 
> Heinrich
> 
> >>>>>> has the responsibility to switch to S-mode when handoff to OS or
> >>>>>> software if the platform design requires that (I remember we have
> >>>>>> the simi  lar sentence in riscv-platform-spec). EDK2 code can't
> >>>>>> just remove the check "dom->next_mode", we use OpenSBI
> without any
> >>>>> changes.
> >>>>>>
> >>>>>>>
> >>>>>>> I am still worried about the custom SBI extension required by EDK2.
> >>>>>>> This will not work when running EDK2 inside Guest/VM because
> >>>>>>> Guest/VM boots in VS-mode and the SBI calls are provided by
> >>>>>>> hypervisors (KVM, Xvisor, etc). I think you should revisit EDK2
> >>>>>>> boot-flow to make it compatible with virtualization and OpenSBI
> >>>> domains.
> >>>>>>
> >>>>>> Ok, I will revisit this. Thanks for the reminder.
> >>>>>
> >>>>> Hi Anup,
> >>>>> I have few questions regard to HSM support in opensbi,
> >>>>>
> >>>>> - Is the purpose of invoking platform_domain_init at the end of
> >>>>> sbi_init() to let platform code to register the domains through
> >>> sbi_domain_register()?
> >>>>
> >>>> Yes, domains need to be populated as late as possible so that domains
> >>>> are switched only after all initialization is completed.
> >>>>
> >>>>> - What is the reason that each domain is requested to have the
> >>>>> memory region of ROOT_FW_REGION?
> >>>>
> >>>> The ROOT_FW_REGION protects the firmware itself. The fw_region is
> >>>> based on fw_start and fw_size members of the "struct sbi_scratch".
> >>>>
> >>>>> - I think opensbi will have the implement of switching the next mode
> >>>>> to
> >>>> HSM
> >>>>> later?
> >>>>
> >>>> Can you elaborate why you need this ?
> >>> Ah no I was just wonder how opensbi switch the next mode to HSM. You
> had
> >>> the answer in below.
> >>>>
> >>>>> - How does opensbi loads the hypervisor in HSM?
> >>>>
> >>>> When H-extension is available the next mode is automatically HS-mode
> >>>> (i.e. S-mode with virt=off).
> >>>>
> >>>> You still did not share how you will make EDK2 boot flow work for
> >>>> VS-modes because hypervisor will start Guest/VM directly in VS-mode
> >>>> and M-mode components of EDK2 can't run inside Guest/VM
> >>>
> >>> I don't have the full picture of EDK2 boot flow for HSM yet. I am
> >>> stillthinking
> >>> how the Management Mode work on HSM especially to the multiple
> type1
> >>
> >> HS-mode and VS-mode are both S-mode with differing capabilities. A
> >> software
> >> written for S-mode (without using H-extension CSRs) should run
> unmodified
> >> in both HS-mode and VS-mode.
> >>
> >>> hypervisors run on the each domain. Also, the platform errors or RAS
> >>> errors
> >>> on server platform has to deliver the error event to the corresponding
> >>
> >> This headache of hypervisor so hypervisor will use appropriate HW
> >> features
> >> to virtualize RAS events/errors.
> >>
> >>> domains. The idea as I mentioned earlier, EDK2 FW is not necessarily
> >>> to be
> >>
> >> I totally disagree. In ARM64 world, people use EDK2 inside Guest/VM so
> >> why can't we do the same in RISC-V world.
> >>
> >> Same distros which run natively without hypervisor will expect to run in
> >> the same way inside Guest/VM. Are you suggesting that EDK2 will not
> >> be available to distros inside Guest/VM ??
> >>
> >>> executed as a VS entity. EDK2 FW is executed when processor power on
> >>> and it
> >>> uses opensbi as a library to initial the basic platform (vendor) and
> >>> opensbi
> >>> initialization, then edk2 FW run through PEI/DXE for the OEM platform
> >>> and
> >>> proprietary features initialization. EDK2 still can jump to opensbi
> >>> to run the
> >>> domain initialization at the last boot stage, says BDS,  and then
> >>> switch to
> >>> HSM. We have to consider those server features such as critical
> >>> platform error
> >>> handling, event logging, FW<->BMC communication, Remote FW
> >>> configuration through Redfish beyond the HSM, or some of above
> >>> drivers can
> >>> run in HSM before hypervisor is launched, I am not sure yet.
> >>
> >> If other architectures are able to use EDK2 inside Guest/VM then I don't
> >> see why we can't do the same for RISC-V.
> >>
> >>> SBI FW extension shouldn't be the problem because HSM hypervisor can
> >>> still
> >>> bypass the sbi invocations from VS entity to M-mode if the sbi function
> >>> number is in the firmware range, right? e.g. SBI FW extension can
> >>> stillprovide
> >>> the management mode interface to (V)S mode entity.
> >>
> >> Bypassing SBI FW calls from VS-mode to M-mode will have it's own issues
> >> and I am reluctant to go this direction without fully understanding why
> >> EDK2 needs SBI FW calls.
> >
> > EDK II (and U-Boot) need the SBI system reset extension to implement the
> > ResetSystem() runtime service which operating systems use to reboot or
> > poweroff.
> >
> > The hypervisor must provide an SBI implementation running in HS mode.
> > This is why we have trap delegation registers:
> >
> > "When a trap occurs in HS-mode or U-mode, it goes to M-mode, unless
> > delegated by medeleg or mideleg, in which case it goes to HS-mode. When
> > a trap occurs in VS-mode or VU-mode, it goes to M-mode, unless
> delegated
> > by medeleg or mideleg, in which case it goes to HS-mode, unless further
> > delegated by hedeleg or hideleg, in which case it goes to VS-mode."
> >
> > EDK II will run in VS mode. EDK II's ecalls will be handled by the
> > hypervisor's SBI implementation.
> >
> > It is not important if EDK II is compiled with or without the OpenSBI
> > library. In both cases the hypervisor must invoke EDK II's PEI entry
> > point and not enter EDK2's SEC phase (cf.
> > INVALID URI REMOVED
> specification/2_design_discussion/23_boot_sequence__;!!NpxR!11uLLL9PJIr
> gauJtSLgUAqsEYMnbRf43C_kxGqaYi7ZLodCE40XfAtUXp4R9brg$ ).
> >
> >
> > On ARM EDK II is a BL33 payload of TF-A. TF-A is not in the code base of
> > EDK II.
> >
> > We should do the same on RISC-V: Remove OpenSBI from the EDK II code
> > base and compile upstream OpenSBI with EDK II as FW_PAYLOAD.
> >
> > This way we don't have to worry about differences between EDK II running
> > in S-mode and VS-mode.
> >
> > @Rick, Bin
> > The same will have to be done for U-Boot's SBI support. We should not
> > require SPL starting OpenSBI to have SBI support on QEMU/KVM.
> >
> > Best regards
> >
> > Heinrich
> >
> >>
> >> The fact that you need to depend on SBI FW extension seems to be
> becoming
> >> a road block for EDK2 inside Guest/VM.
> >>
> >> Regards,
> >> Anup
> >>
> >>>
> >>> Abner
> >>>
> >>>
> >>>>
> >>>> Regards,
> >>>> Anup
> >>>>
> >>>>>
> >>>>> Regards,
> >>>>> Abner
> >>>>>
> >>>>>>
> >>>>>> Regards,
> >>>>>> Abner
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Anup
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Thanks and regards,
> >>>>>>>> Abner
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Next booting stage has to run from lower privilege mode
> >>>>>>>>>> (S-mode or
> >>>>>>>>>> U-
> >>>>>>>>>> mode) otherwise OpenSBI cannot protect itself from next
> >>>>>>>>>> booting stage if it starts in M-mode.
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> However, sbi_init when directly initializing to S-Mode
> >>>>>>>>>>>> checks that the
> >>>>>>>>>>> start_address is executable.
> >>>>>>>>>>>> So I'm wondering why the FW region isn't set as
> >>>>>>>>>>>> executable in
> >>>>>>>> OpenSBI?
> >>>>>>>>>>>> How do other FWs like U-Boot get around this?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>
> https://github.com/riscv/opensbi/commit/b1678af210dc4b4e6d586d6d966
> >>>>> 1
> >>>>>>>>>> 7
> >>>>>>>>>>> e
> >>>>>>>>>>>> 9641618994#diff-
> >>>>>>>>>>>
> >>> 6e8e352a8a90ba5a7adbb58a806ed9b6404c2c67db416332f9c05a
> >>>>>>>>>>>> 6b322eecd6R346
> >>>>>>>>>>>>
> >>>>>>>>>>>> If I try to set my own regions by adding
> >>>>>>>>>>>> .domains_root_regions I get another error because
> >>>>>>>>>>>> OpenSBI checks that I have a region that is the same
> >>>>>>>>>>>> as the FW region added by OpenSBI. If I duplicate the
> >>>>>>>>>>>> FW region and mark the first one as executable I can
> >>>>>>>>>>>> pass the executable check and also the check that
> >>>>>>>>>>>> there's an
> >>>>>> FW
> >>>>>>>> region.
> >>>>>>>>>>>> Additionally we have to manually call pmp_set in our
> >>>>>>>>>>>> custom platform to make the FW region RWX.
> >>>>>>>>>>>>
> >>>>>>>>>>>> That seems like a workaround, however. Do you have any
> >>>>>>>>>>>> suggestion to
> >>>>>>>>>>> properly fix it?
> >>>>>>>>>>>> I'm sure we're misunderstand something.
> >>>>>>>>>>
> >>>>>>>>>> I suggest two things:
> >>>>>>>>>> 1) Register your custom SBI extension from M-mode only
> >>>>>>>>>> before switching to S-mode
> >>>>>>>>>> 2) Make sure that fw_start and fw_size set in the
> >>>>>>>>>> sbi_scratch for each HART only point to the M-mode code and
> >>> data.
> >>>>>>>>>> Preferably have S-mode code and data not linked in the
> >>>>>>>>>> same binary as M-mode code
> >>>>>> and
> >>>>>>>> data.
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> For context: We're writing the start addr and size of
> >>>>>>>>>>>> our FW image into the scratch space before OpenSBI is
> >>> initialized.
> >>>>>>>>>>>> Therefore we're expecting it to set the PMP settings
> >>> correctly.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please check out my workaround commit:
> >>>>>>>>>>>> https://github.com/riscv/riscv-edk2-
> >>>>>> platforms/commit/a5ac63096ca
> >>>>>>>>>>>> 5da7
> >>>>>>>>>>>> 95
> >>>>>>>>>>>> 042baf650170643fe219cab
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Daniel
> >>>>>>>>>>
> >>>>>>>>>> Regards
> >>>>>>>>>> Anup
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Anup
> >>>>>>
> >>>>>> --
> >>>>>> opensbi mailing list
> >>>>>> opensbi at lists.infradead.org
> >>>>>> http://lists.infradead.org/mailman/listinfo/opensbi
> >

More information about the opensbi mailing list