Why does the firmware memory region have no permissions?

[Anup Patel]

> -----Original Message-----
> From: Heinrich Schuchardt <xypron.glpk at gmx.de>
> Sent: 03 June 2021 22:45
> To: Anup Patel <Anup.Patel at wdc.com>; Chang, Abner (HPS SW/FW
> Technologist) <abner.chang at hpe.com>; Daniel Schaefer
> <daniel at danielschaefer.me>
> Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>
> Subject: Re: Why does the firmware memory region have no permissions?
> 
> On 6/3/21 6:51 PM, Heinrich Schuchardt wrote:
> > On 6/3/21 5:34 PM, Anup Patel wrote:
> >> +Heinrich
> >>
> >>> -----Original Message-----
> >>> From: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>
> >>> Sent: 03 June 2021 20:05
> >>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>> <daniel at danielschaefer.me>
> >>> Cc: opensbi at lists.infradead.org
> >>> Subject: RE: Why does the firmware memory region have no
> permissions?
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>>> Sent: Wednesday, June 2, 2021 11:15 PM
> >>>> To: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>;
> >>>> Daniel Schaefer <daniel at danielschaefer.me>
> >>>> Cc: opensbi at lists.infradead.org
> >>>> Subject: RE: Why does the firmware memory region have no
> permissions?
> >>>>
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Chang, Abner (HPS SW/FW Technologist)
> <abner.chang at hpe.com>
> >>>>> Sent: 02 June 2021 20:19
> >>>>> To: Chang, Abner (HPS SW/FW Technologist)
> <abner.chang at hpe.com>;
> >>>> Anup
> >>>>> Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>>> <daniel at danielschaefer.me>
> >>>>> Cc: opensbi at lists.infradead.org
> >>>>> Subject: RE: Why does the firmware memory region have no
> permissions?
> >>>>>
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: opensbi [mailto:opensbi-bounces at lists.infradead.org] On
> >>>>>> Behalf Of Chang, Abner (HPS SW/FW Technologist)
> >>>>>> Sent: Saturday, May 15, 2021 11:30 PM
> >>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>>>>> <daniel at danielschaefer.me>
> >>>>>> Cc: opensbi at lists.infradead.org
> >>>>>> Subject: RE: Why does the firmware memory region have no
> >>> permissions?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>>>>>> Sent: Saturday, May 15, 2021 4:09 PM
> >>>>>>> To: Chang, Abner (HPS SW/FW Technologist)
> >>> <abner.chang at hpe.com>;
> >>>>>> Daniel
> >>>>>>> Schaefer <daniel at danielschaefer.me>
> >>>>>>> Cc: opensbi at lists.infradead.org
> >>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>> permissions?
> >>>>>>>
> >>>>>>> Hi Abner,
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Chang, Abner (HPS SW/FW Technologist)
> >>>>>> <abner.chang at hpe.com>
> >>>>>>>> Sent: 15 May 2021 11:47
> >>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
> >>>>>>>> <daniel at danielschaefer.me>
> >>>>>>>> Cc: opensbi at lists.infradead.org
> >>>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>>> permissions?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
> >>>>>>>>> Sent: Friday, May 14, 2021 7:58 PM
> >>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
> >>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>>> Technologist)
> >>>>>>>>> <abner.chang at hpe.com>
> >>>>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>>>> permissions?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: Anup Patel
> >>>>>>>>>> Sent: 14 May 2021 17:22
> >>>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
> >>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>>>>>>> Technologist) <abner.chang at hpe.com>
> >>>>>>>>>> Subject: RE: Why does the firmware memory region have no
> >>>>>>>> permissions?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: opensbi <opensbi-bounces at lists.infradead.org> On
> >>>>>>>>>>> Behalf
> >>>>>> Of
> >>>>>>>>>>> Daniel Schaefer
> >>>>>>>>>>> Sent: 13 May 2021 10:26
> >>>>>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>
> >>>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
> >>>>>>>>>>> Technologist) <abner.chang at hpe.com>
> >>>>>>>>>>> Subject: Why does the firmware memory region have no
> >>>>>> permissions?
> >>>>>>>>>>>
> >>>>>>>>>>> Whoops, put CC as subject...
> >>>>>>>>>>>
> >>>>>>>>>>> On 5/12/21 6:20 PM, Daniel Schaefer wrote:
> >>>>>>>>>>>> Hi Anup,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I'm in the process of upgrading EDKII to OpenSBI 0.9 and
> >>>>>>>>>>>> using the Generic
> >>>>>>>>>>> Platform.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Previously we were doing sbi_init with M-Mode, adding our
> >>>>>>>>>>>> SBI extension and then calling sbi_switch_mode to switch to
> >>>>>>>>>>>> S-
> >>>>> Mode.
> >>>>>>>>>>>> Now sbi_init disallows initializing to M-Mode, so I'm
> >>>>>>>>>>>> directly switching to S-Mode. It seems that even from
> >>>>>>>>>>>> S-Mode I can register our SBI
> >>>>>>>>>>> extension with sbi_ecall_register_extension.
> >>>>>>>>>>>> Is that correct?
> >>>>>>>>>>
> >>>>>>>>>> The OpenSBI sources are meant to run only from M-mode so we
> >>>>>> cannot
> >>>>>>>>>> register SBI extension using sbi_ecall_register_extension().
> >>>>>>>>>>
> >>>>>>>>>> The  sbi_switch_mode() is not stricter due to OpenSBI domain
> >>>>>> support.
> >>>>>>>>>
> >>>>>>>>> The sbi_hart_switch_mode() is fine. It's the
> >>>>>>>>> sbi_domain_init() which is enforcing next booting stage to be
> >>>>>>>>> at lower privilege for
> >>>> root
> >>>>> domain.
> >>>>>>>>>
> >>>>>>>>> For time being, you can try removing checks on "dom-
> >>>> next_mode"
> >>>>>>>>> in sanitize_domain()
> >>>>>>>>
> >>>>>>>> Hi Anup, I think we currently skip that check for moving on the
> >>>>>>>> edk2 boot process. So do you have plan to remove this check?
> >>>>>>>> Or any
> >>>>> alternative?
> >>>>>>>> I think it is unnecessary having this check on the next
> >>>>>>>> privilege
> >>> mode.
> >>>>>> That
> >>>>>>>> should be at OEM discretion of which privilege mode to run
> >>>>>>>> their next firmware stage based on the platform design?
> >>>>>>>
> >>>>>>> This is an important check required by OpenSBI domain support so
> >>>>>>> that next booting stage cannot tamper with PMP configuration
> >>>>>>> (and other security configuration) done by OpenSBI.
> >>>>>>
> >>>>>> I understand the importance of not giving any chance to tamper
> >>>>>> PMP setting, however this could be the responsibility of the next
> >>>>>> boot phase
> >>>>> before OS.
> >>>>>> OpenSBI as the early phase boot firmware should be generally
> >>>>>> provide SBIs to platform variants, and have the flexibility to
> >>>>>> hand off to either M-mode or S-mode firmware (Actually I don't
> >>>>>> think OpenSBI
> >>>> should
> >>>>> handle this).
> >>>>>> Platform code is provided by OEM/vendor, OpenSBI should allow it
> >>>>>> if platform code says I would like to run my next phase firmware
> >>>>>> in M-
> >>>> mode.
> >>>>>> We restrict the privilege phase for the next phase in OpenSBI
> >>>>>> also not compliant with the UEFI spec which says UEFI RISC-V
> >>>>>> firmware could be executed in either M-mode or S-mode. Some EFI
> >>>>>> driver may be loaded in
> >>>>>> S- mode but provide the M-mode code such as management mode,
> the
> >>>>>> Platform Runtime Mechanism or some other use cases. The next
> >>>> firmware
> 
> Compare this to EDK II with TF-A. No EDK II code runs in EL3. TF-A is the only
> EL3 code.
> 
> Same for RISC-V: OpenSBI is the M-mode firmware. No other part of the
> firmware shall run in M-mode. I cannot see that the UEFI specification
> requires a different segregation of duties.
> 
> What RISC-V is lacking up to now is run modes for trusted execution
> environments like OP-TEE. This is where you would place management mode
> code and not in M-mode.

There is already a OP-TEE port for RISC-V which is based on OpenSBI domains
where secured software runs in it's own domain and non-secured software
runs in separate domain.

Here's the video:
https://www.youtube.com/watch?v=uR1YwN47yY8

Regards,
Anup

> 
> Best regards
> 
> Heinrich
> 
> >>>>>> has the responsibility to switch to S-mode when handoff to OS or
> >>>>>> software if the platform design requires that (I remember we have
> >>>>>> the simi  lar sentence in riscv-platform-spec). EDK2 code can't
> >>>>>> just remove the check "dom->next_mode", we use OpenSBI without
> >>>>>> any
> >>>>> changes.
> >>>>>>
> >>>>>>>
> >>>>>>> I am still worried about the custom SBI extension required by EDK2.
> >>>>>>> This will not work when running EDK2 inside Guest/VM because
> >>>>>>> Guest/VM boots in VS-mode and the SBI calls are provided by
> >>>>>>> hypervisors (KVM, Xvisor, etc). I think you should revisit EDK2
> >>>>>>> boot-flow to make it compatible with virtualization and OpenSBI
> >>>> domains.
> >>>>>>
> >>>>>> Ok, I will revisit this. Thanks for the reminder.
> >>>>>
> >>>>> Hi Anup,
> >>>>> I have few questions regard to HSM support in opensbi,
> >>>>>
> >>>>> - Is the purpose of invoking platform_domain_init at the end of
> >>>>> sbi_init() to let platform code to register the domains through
> >>> sbi_domain_register()?
> >>>>
> >>>> Yes, domains need to be populated as late as possible so that
> >>>> domains are switched only after all initialization is completed.
> >>>>
> >>>>> - What is the reason that each domain is requested to have the
> >>>>> memory region of ROOT_FW_REGION?
> >>>>
> >>>> The ROOT_FW_REGION protects the firmware itself. The fw_region is
> >>>> based on fw_start and fw_size members of the "struct sbi_scratch".
> >>>>
> >>>>> - I think opensbi will have the implement of switching the next
> >>>>> mode to
> >>>> HSM
> >>>>> later?
> >>>>
> >>>> Can you elaborate why you need this ?
> >>> Ah no I was just wonder how opensbi switch the next mode to HSM. You
> >>> had the answer in below.
> >>>>
> >>>>> - How does opensbi loads the hypervisor in HSM?
> >>>>
> >>>> When H-extension is available the next mode is automatically
> >>>> HS-mode (i.e. S-mode with virt=off).
> >>>>
> >>>> You still did not share how you will make EDK2 boot flow work for
> >>>> VS-modes because hypervisor will start Guest/VM directly in VS-mode
> >>>> and M-mode components of EDK2 can't run inside Guest/VM
> >>>
> >>> I don't have the full picture of EDK2 boot flow for HSM yet. I am
> >>> stillthinking how the Management Mode work on HSM especially to the
> >>> multiple type1
> >>
> >> HS-mode and VS-mode are both S-mode with differing capabilities. A
> >> software written for S-mode (without using H-extension CSRs) should
> >> run unmodified in both HS-mode and VS-mode.
> >>
> >>> hypervisors run on the each domain. Also, the platform errors or RAS
> >>> errors on server platform has to deliver the error event to the
> >>> corresponding
> >>
> >> This headache of hypervisor so hypervisor will use appropriate HW
> >> features to virtualize RAS events/errors.
> >>
> >>> domains. The idea as I mentioned earlier, EDK2 FW is not necessarily
> >>> to be
> >>
> >> I totally disagree. In ARM64 world, people use EDK2 inside Guest/VM
> >> so why can't we do the same in RISC-V world.
> >>
> >> Same distros which run natively without hypervisor will expect to run
> >> in the same way inside Guest/VM. Are you suggesting that EDK2 will
> >> not be available to distros inside Guest/VM ??
> >>
> >>> executed as a VS entity. EDK2 FW is executed when processor power on
> >>> and it uses opensbi as a library to initial the basic platform
> >>> (vendor) and opensbi initialization, then edk2 FW run through
> >>> PEI/DXE for the OEM platform and proprietary features
> >>> initialization. EDK2 still can jump to opensbi to run the domain
> >>> initialization at the last boot stage, says BDS,  and then switch to
> >>> HSM. We have to consider those server features such as critical
> >>> platform error handling, event logging, FW<->BMC communication,
> >>> Remote FW configuration through Redfish beyond the HSM, or some of
> >>> above drivers can run in HSM before hypervisor is launched, I am not
> >>> sure yet.
> >>
> >> If other architectures are able to use EDK2 inside Guest/VM then I
> >> don't see why we can't do the same for RISC-V.
> >>
> >>> SBI FW extension shouldn't be the problem because HSM hypervisor can
> >>> still bypass the sbi invocations from VS entity to M-mode if the sbi
> >>> function number is in the firmware range, right? e.g. SBI FW
> >>> extension can stillprovide the management mode interface to (V)S
> >>> mode entity.
> >>
> >> Bypassing SBI FW calls from VS-mode to M-mode will have it's own
> >> issues and I am reluctant to go this direction without fully
> >> understanding why
> >> EDK2 needs SBI FW calls.
> >
> > EDK II (and U-Boot) need the SBI system reset extension to implement
> > the
> > ResetSystem() runtime service which operating systems use to reboot or
> > poweroff.
> >
> > The hypervisor must provide an SBI implementation running in HS mode.
> > This is why we have trap delegation registers:
> >
> > "When a trap occurs in HS-mode or U-mode, it goes to M-mode, unless
> > delegated by medeleg or mideleg, in which case it goes to HS-mode.
> > When a trap occurs in VS-mode or VU-mode, it goes to M-mode, unless
> > delegated by medeleg or mideleg, in which case it goes to HS-mode,
> > unless further delegated by hedeleg or hideleg, in which case it goes to VS-
> mode."
> >
> > EDK II will run in VS mode. EDK II's ecalls will be handled by the
> > hypervisor's SBI implementation.
> >
> > It is not important if EDK II is compiled with or without the OpenSBI
> > library. In both cases the hypervisor must invoke EDK II's PEI entry
> > point and not enter EDK2's SEC phase (cf.
> > https://edk2-docs.gitbook.io/edk-ii-build-
> specification/2_design_discussion/23_boot_sequence).
> >
> >
> > On ARM EDK II is a BL33 payload of TF-A. TF-A is not in the code base
> > of EDK II.
> >
> > We should do the same on RISC-V: Remove OpenSBI from the EDK II code
> > base and compile upstream OpenSBI with EDK II as FW_PAYLOAD.
> >
> > This way we don't have to worry about differences between EDK II
> > running in S-mode and VS-mode.
> >
> > @Rick, Bin
> > The same will have to be done for U-Boot's SBI support. We should not
> > require SPL starting OpenSBI to have SBI support on QEMU/KVM.
> >
> > Best regards
> >
> > Heinrich
> >
> >>
> >> The fact that you need to depend on SBI FW extension seems to be
> >> becoming a road block for EDK2 inside Guest/VM.
> >>
> >> Regards,
> >> Anup
> >>
> >>>
> >>> Abner
> >>>
> >>>
> >>>>
> >>>> Regards,
> >>>> Anup
> >>>>
> >>>>>
> >>>>> Regards,
> >>>>> Abner
> >>>>>
> >>>>>>
> >>>>>> Regards,
> >>>>>> Abner
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Anup
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Thanks and regards,
> >>>>>>>> Abner
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Next booting stage has to run from lower privilege mode
> >>>>>>>>>> (S-mode or
> >>>>>>>>>> U-
> >>>>>>>>>> mode) otherwise OpenSBI cannot protect itself from next
> >>>>>>>>>> booting stage if it starts in M-mode.
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> However, sbi_init when directly initializing to S-Mode
> >>>>>>>>>>>> checks that the
> >>>>>>>>>>> start_address is executable.
> >>>>>>>>>>>> So I'm wondering why the FW region isn't set as executable
> >>>>>>>>>>>> in
> >>>>>>>> OpenSBI?
> >>>>>>>>>>>> How do other FWs like U-Boot get around this?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>
> https://github.com/riscv/opensbi/commit/b1678af210dc4b4e6d586d6d966
> >>>>> 1
> >>>>>>>>>> 7
> >>>>>>>>>>> e
> >>>>>>>>>>>> 9641618994#diff-
> >>>>>>>>>>>
> >>> 6e8e352a8a90ba5a7adbb58a806ed9b6404c2c67db416332f9c05a
> >>>>>>>>>>>> 6b322eecd6R346
> >>>>>>>>>>>>
> >>>>>>>>>>>> If I try to set my own regions by adding
> >>>>>>>>>>>> .domains_root_regions I get another error because OpenSBI
> >>>>>>>>>>>> checks that I have a region that is the same as the FW
> >>>>>>>>>>>> region added by OpenSBI. If I duplicate the FW region and
> >>>>>>>>>>>> mark the first one as executable I can pass the executable
> >>>>>>>>>>>> check and also the check that there's an
> >>>>>> FW
> >>>>>>>> region.
> >>>>>>>>>>>> Additionally we have to manually call pmp_set in our custom
> >>>>>>>>>>>> platform to make the FW region RWX.
> >>>>>>>>>>>>
> >>>>>>>>>>>> That seems like a workaround, however. Do you have any
> >>>>>>>>>>>> suggestion to
> >>>>>>>>>>> properly fix it?
> >>>>>>>>>>>> I'm sure we're misunderstand something.
> >>>>>>>>>>
> >>>>>>>>>> I suggest two things:
> >>>>>>>>>> 1) Register your custom SBI extension from M-mode only before
> >>>>>>>>>> switching to S-mode
> >>>>>>>>>> 2) Make sure that fw_start and fw_size set in the sbi_scratch
> >>>>>>>>>> for each HART only point to the M-mode code and
> >>> data.
> >>>>>>>>>> Preferably have S-mode code and data not linked in the same
> >>>>>>>>>> binary as M-mode code
> >>>>>> and
> >>>>>>>> data.
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> For context: We're writing the start addr and size of our
> >>>>>>>>>>>> FW image into the scratch space before OpenSBI is
> >>> initialized.
> >>>>>>>>>>>> Therefore we're expecting it to set the PMP settings
> >>> correctly.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please check out my workaround commit:
> >>>>>>>>>>>> https://github.com/riscv/riscv-edk2-
> >>>>>> platforms/commit/a5ac63096ca
> >>>>>>>>>>>> 5da7
> >>>>>>>>>>>> 95
> >>>>>>>>>>>> 042baf650170643fe219cab
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Daniel
> >>>>>>>>>>
> >>>>>>>>>> Regards
> >>>>>>>>>> Anup
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Anup
> >>>>>>
> >>>>>> --
> >>>>>> opensbi mailing list
> >>>>>> opensbi at lists.infradead.org
> >>>>>> http://lists.infradead.org/mailman/listinfo/opensbi
> >