Why does the firmware memory region have no permissions?

Jessica Clarke jrtc27 at jrtc27.com
Thu Jun 3 10:33:16 PDT 2021 

On 3 Jun 2021, at 18:27, Anup Patel <Anup.Patel at wdc.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Heinrich Schuchardt <xypron.glpk at gmx.de>
>> Sent: 03 June 2021 22:21
>> To: Anup Patel <Anup.Patel at wdc.com>; Chang, Abner (HPS SW/FW
>> Technologist) <abner.chang at hpe.com>; Daniel Schaefer
>> <daniel at danielschaefer.me>
>> Cc: opensbi at lists.infradead.org; Atish Patra <Atish.Patra at wdc.com>; Rick
>> Chen <rick at andestech.com>; Bin Meng <bmeng.cn at gmail.com>
>> Subject: Re: Why does the firmware memory region have no permissions?
>> 
>> On 6/3/21 5:34 PM, Anup Patel wrote:
>>> +Heinrich
>>> 
>>>> -----Original Message-----
>>>> From: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>
>>>> Sent: 03 June 2021 20:05
>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
>>>> <daniel at danielschaefer.me>
>>>> Cc: opensbi at lists.infradead.org
>>>> Subject: RE: Why does the firmware memory region have no permissions?
>>>> 
>>>> 
>>>> 
>>>>> -----Original Message-----
>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
>>>>> Sent: Wednesday, June 2, 2021 11:15 PM
>>>>> To: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>;
>>>>> Daniel Schaefer <daniel at danielschaefer.me>
>>>>> Cc: opensbi at lists.infradead.org
>>>>> Subject: RE: Why does the firmware memory region have no
>> permissions?
>>>>> 
>>>>> 
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Chang, Abner (HPS SW/FW Technologist)
>> <abner.chang at hpe.com>
>>>>>> Sent: 02 June 2021 20:19
>>>>>> To: Chang, Abner (HPS SW/FW Technologist) <abner.chang at hpe.com>;
>>>>> Anup
>>>>>> Patel <Anup.Patel at wdc.com>; Daniel Schaefer
>>>>> <daniel at danielschaefer.me>
>>>>>> Cc: opensbi at lists.infradead.org
>>>>>> Subject: RE: Why does the firmware memory region have no
>> permissions?
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: opensbi [mailto:opensbi-bounces at lists.infradead.org] On
>>>>>>> Behalf Of Chang, Abner (HPS SW/FW Technologist)
>>>>>>> Sent: Saturday, May 15, 2021 11:30 PM
>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
>>>>>>> <daniel at danielschaefer.me>
>>>>>>> Cc: opensbi at lists.infradead.org
>>>>>>> Subject: RE: Why does the firmware memory region have no
>>>> permissions?
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
>>>>>>>> Sent: Saturday, May 15, 2021 4:09 PM
>>>>>>>> To: Chang, Abner (HPS SW/FW Technologist)
>>>> <abner.chang at hpe.com>;
>>>>>>> Daniel
>>>>>>>> Schaefer <daniel at danielschaefer.me>
>>>>>>>> Cc: opensbi at lists.infradead.org
>>>>>>>> Subject: RE: Why does the firmware memory region have no
>>>>> permissions?
>>>>>>>> 
>>>>>>>> Hi Abner,
>>>>>>>> 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Chang, Abner (HPS SW/FW Technologist)
>>>>>>> <abner.chang at hpe.com>
>>>>>>>>> Sent: 15 May 2021 11:47
>>>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>; Daniel Schaefer
>>>>>>>>> <daniel at danielschaefer.me>
>>>>>>>>> Cc: opensbi at lists.infradead.org
>>>>>>>>> Subject: RE: Why does the firmware memory region have no
>>>>>> permissions?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Anup Patel [mailto:Anup.Patel at wdc.com]
>>>>>>>>>> Sent: Friday, May 14, 2021 7:58 PM
>>>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
>>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
>>>>>>> Technologist)
>>>>>>>>>> <abner.chang at hpe.com>
>>>>>>>>>> Subject: RE: Why does the firmware memory region have no
>>>>>>> permissions?
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Anup Patel
>>>>>>>>>>> Sent: 14 May 2021 17:22
>>>>>>>>>>> To: Daniel Schaefer <daniel at danielschaefer.me>
>>>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
>>>>>>>>>>> Technologist) <abner.chang at hpe.com>
>>>>>>>>>>> Subject: RE: Why does the firmware memory region have no
>>>>>>>>> permissions?
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: opensbi <opensbi-bounces at lists.infradead.org> On
>> Behalf
>>>>>>> Of
>>>>>>>>>>>> Daniel Schaefer
>>>>>>>>>>>> Sent: 13 May 2021 10:26
>>>>>>>>>>>> To: Anup Patel <Anup.Patel at wdc.com>
>>>>>>>>>>>> Cc: opensbi at lists.infradead.org; Chang, Abner (HPS SW/FW
>>>>>>>>>>>> Technologist) <abner.chang at hpe.com>
>>>>>>>>>>>> Subject: Why does the firmware memory region have no
>>>>>>> permissions?
>>>>>>>>>>>> 
>>>>>>>>>>>> Whoops, put CC as subject...
>>>>>>>>>>>> 
>>>>>>>>>>>> On 5/12/21 6:20 PM, Daniel Schaefer wrote:
>>>>>>>>>>>>> Hi Anup,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I'm in the process of upgrading EDKII to OpenSBI 0.9 and
>>>>>>>>>>>>> using the Generic
>>>>>>>>>>>> Platform.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Previously we were doing sbi_init with M-Mode, adding our
>>>>>>>>>>>>> SBI extension and then calling sbi_switch_mode to switch to
>>>>>>>>>>>>> S-
>>>>>> Mode.
>>>>>>>>>>>>> Now sbi_init disallows initializing to M-Mode, so I'm
>>>>>>>>>>>>> directly switching to S-Mode. It seems that even from S-Mode
>>>>>>>>>>>>> I can register our SBI
>>>>>>>>>>>> extension with sbi_ecall_register_extension.
>>>>>>>>>>>>> Is that correct?
>>>>>>>>>>> 
>>>>>>>>>>> The OpenSBI sources are meant to run only from M-mode so we
>>>>>>> cannot
>>>>>>>>>>> register SBI extension using sbi_ecall_register_extension().
>>>>>>>>>>> 
>>>>>>>>>>> The  sbi_switch_mode() is not stricter due to OpenSBI domain
>>>>>>> support.
>>>>>>>>>> 
>>>>>>>>>> The sbi_hart_switch_mode() is fine. It's the
>>>>>>>>>> sbi_domain_init() which is enforcing next booting stage to be
>>>>>>>>>> at lower privilege for
>>>>> root
>>>>>> domain.
>>>>>>>>>> 
>>>>>>>>>> For time being, you can try removing checks on "dom-
>>>>> next_mode"
>>>>>>>>>> in sanitize_domain()
>>>>>>>>> 
>>>>>>>>> Hi Anup, I think we currently skip that check for moving on the
>>>>>>>>> edk2 boot process. So do you have plan to remove this check?
>>>>>>>>> Or any
>>>>>> alternative?
>>>>>>>>> I think it is unnecessary having this check on the next
>>>>>>>>> privilege
>>>> mode.
>>>>>>> That
>>>>>>>>> should be at OEM discretion of which privilege mode to run their
>>>>>>>>> next firmware stage based on the platform design?
>>>>>>>> 
>>>>>>>> This is an important check required by OpenSBI domain support so
>>>>>>>> that next booting stage cannot tamper with PMP configuration (and
>>>>>>>> other security configuration) done by OpenSBI.
>>>>>>> 
>>>>>>> I understand the importance of not giving any chance to tamper PMP
>>>>>>> setting, however this could be the responsibility of the next boot
>>>>>>> phase
>>>>>> before OS.
>>>>>>> OpenSBI as the early phase boot firmware should be generally
>>>>>>> provide SBIs to platform variants, and have the flexibility to
>>>>>>> hand off to either M-mode or S-mode firmware (Actually I don't
>>>>>>> think OpenSBI
>>>>> should
>>>>>> handle this).
>>>>>>> Platform code is provided by OEM/vendor, OpenSBI should allow it
>>>>>>> if platform code says I would like to run my next phase firmware
>>>>>>> in M-
>>>>> mode.
>>>>>>> We restrict the privilege phase for the next phase in OpenSBI also
>>>>>>> not compliant with the UEFI spec which says UEFI RISC-V firmware
>>>>>>> could be executed in either M-mode or S-mode. Some EFI driver may
>>>>>>> be loaded in
>>>>>>> S- mode but provide the M-mode code such as management mode,
>> the
>>>>>>> Platform Runtime Mechanism or some other use cases. The next
>>>>> firmware
>>>>>>> has the responsibility to switch to S-mode when handoff to OS or
>>>>>>> software if the platform design requires that (I remember we have
>>>>>>> the simi  lar sentence in riscv-platform-spec). EDK2 code can't
>>>>>>> just remove the check "dom->next_mode", we use OpenSBI without
>> any
>>>>>> changes.
>>>>>>> 
>>>>>>>> 
>>>>>>>> I am still worried about the custom SBI extension required by EDK2.
>>>>>>>> This will not work when running EDK2 inside Guest/VM because
>>>>>>>> Guest/VM boots in VS-mode and the SBI calls are provided by
>>>>>>>> hypervisors (KVM, Xvisor, etc). I think you should revisit EDK2
>>>>>>>> boot-flow to make it compatible with virtualization and OpenSBI
>>>>> domains.
>>>>>>> 
>>>>>>> Ok, I will revisit this. Thanks for the reminder.
>>>>>> 
>>>>>> Hi Anup,
>>>>>> I have few questions regard to HSM support in opensbi,
>>>>>> 
>>>>>> - Is the purpose of invoking platform_domain_init at the end of
>>>>>> sbi_init() to let platform code to register the domains through
>>>> sbi_domain_register()?
>>>>> 
>>>>> Yes, domains need to be populated as late as possible so that
>>>>> domains are switched only after all initialization is completed.
>>>>> 
>>>>>> - What is the reason that each domain is requested to have the
>>>>>> memory region of ROOT_FW_REGION?
>>>>> 
>>>>> The ROOT_FW_REGION protects the firmware itself. The fw_region is
>>>>> based on fw_start and fw_size members of the "struct sbi_scratch".
>>>>> 
>>>>>> - I think opensbi will have the implement of switching the next
>>>>>> mode to
>>>>> HSM
>>>>>> later?
>>>>> 
>>>>> Can you elaborate why you need this ?
>>>> Ah no I was just wonder how opensbi switch the next mode to HSM. You
>>>> had the answer in below.
>>>>> 
>>>>>> - How does opensbi loads the hypervisor in HSM?
>>>>> 
>>>>> When H-extension is available the next mode is automatically HS-mode
>>>>> (i.e. S-mode with virt=off).
>>>>> 
>>>>> You still did not share how you will make EDK2 boot flow work for
>>>>> VS-modes because hypervisor will start Guest/VM directly in VS-mode
>>>>> and M-mode components of EDK2 can't run inside Guest/VM
>>>> 
>>>> I don't have the full picture of EDK2 boot flow for HSM yet. I am
>>>> still thinking how the Management Mode work on HSM especially to the
>>>> multiple type1
>>> 
>>> HS-mode and VS-mode are both S-mode with differing capabilities. A
>>> software written for S-mode (without using H-extension CSRs) should
>>> run unmodified in both HS-mode and VS-mode.
>>> 
>>>> hypervisors run on the each domain. Also, the platform errors or RAS
>>>> errors on server platform has to deliver the error event to the
>>>> corresponding
>>> 
>>> This headache of hypervisor so hypervisor will use appropriate HW
>>> features to virtualize RAS events/errors.
>>> 
>>>> domains. The idea as I mentioned earlier, EDK2 FW is not necessarily
>>>> to be
>>> 
>>> I totally disagree. In ARM64 world, people use EDK2 inside Guest/VM so
>>> why can't we do the same in RISC-V world.
>>> 
>>> Same distros which run natively without hypervisor will expect to run
>>> in the same way inside Guest/VM. Are you suggesting that EDK2 will not
>>> be available to distros inside Guest/VM ??
>>> 
>>>> executed as a VS entity. EDK2 FW is executed when processor power on
>>>> and it uses opensbi as a library to initial the basic platform
>>>> (vendor) and opensbi initialization, then edk2 FW run through PEI/DXE
>>>> for the OEM platform and proprietary features initialization. EDK2
>>>> still can jump to opensbi to run the domain initialization at the
>>>> last boot stage, says BDS,  and then switch to HSM. We have to
>>>> consider those server features such as critical platform error
>>>> handling, event logging, FW<->BMC communication, Remote FW
>>>> configuration through Redfish beyond the HSM, or some of above drivers
>> can run in HSM before hypervisor is launched, I am not sure yet.
>>> 
>>> If other architectures are able to use EDK2 inside Guest/VM then I
>>> don't see why we can't do the same for RISC-V.
>>> 
>>>> SBI FW extension shouldn't be the problem because HSM hypervisor can
>>>> still bypass the sbi invocations from VS entity to M-mode if the sbi
>>>> function number is in the firmware range, right? e.g. SBI FW
>>>> extension can still provide the management mode interface to (V)S mode
>> entity.
>>> 
>>> Bypassing SBI FW calls from VS-mode to M-mode will have it's own
>>> issues and I am reluctant to go this direction without fully
>>> understanding why
>>> EDK2 needs SBI FW calls.
>> 
>> EDK II (and U-Boot) need the SBI system reset extension to implement the
>> ResetSystem() runtime service which operating systems use to reboot or
>> poweroff.
>> 
>> The hypervisor must provide an SBI implementation running in HS mode.
>> This is why we have trap delegation registers:
>> 
>> "When a trap occurs in HS-mode or U-mode, it goes to M-mode, unless
>> delegated by medeleg or mideleg, in which case it goes to HS-mode. When a
>> trap occurs in VS-mode or VU-mode, it goes to M-mode, unless delegated by
>> medeleg or mideleg, in which case it goes to HS-mode, unless further
>> delegated by hedeleg or hideleg, in which case it goes to VS-mode."
>> 
>> EDK II will run in VS mode. EDK II's ecalls will be handled by the hypervisor's
>> SBI implementation.
>> 
>> It is not important if EDK II is compiled with or without the OpenSBI library. In
>> both cases the hypervisor must invoke EDK II's PEI entry point and not enter
>> EDK2's SEC phase (cf.
>> https://edk2-docs.gitbook.io/edk-ii-build-
>> specification/2_design_discussion/23_boot_sequence).
>> 
>> On ARM EDK II is a BL33 payload of TF-A. TF-A is not in the code base of EDK II.
>> 
>> We should do the same on RISC-V: Remove OpenSBI from the EDK II code base
>> and compile upstream OpenSBI with EDK II as FW_PAYLOAD.
>> 
>> This way we don't have to worry about differences between EDK II running in
>> S-mode and VS-mode.
>> 
>> @Rick, Bin
>> The same will have to be done for U-Boot's SBI support. We should not
>> require SPL starting OpenSBI to have SBI support on QEMU/KVM.
> 
> The U-Boot S-mode (or U-Boot proper) is totally independent of
> who starts OpenSBI because on SiFive Unleashed we have two
> boot-flows:
> SiFive FSBL => OpenSBI => U-Boot S-mode
> U-Boot SPL => OpenSBI => U-Boot S-mode
> 
> Although, I have never tested U-Boot S-mode inside KVM RISC-V
> or Xvisor RISC-V Guest but I won't be surprised if it just works.

FWIW I’ve run it on top of *BBL* (because we still have yet to migrate
our CHERI-RISC-V work to OpenSBI) on QEMU, and a VirtIO-based FPGA
environment, and U-Boot worked just fine in S-mode.

Jess

More information about the opensbi mailing list