[Question] How to testing SDEI client driver

[Gavin Shan]

Hi James,

On 7/1/20 9:57 PM, James Morse wrote:
> On 30/06/2020 06:17, Gavin Shan wrote:
>> I'm currently looking into SDEI client driver and reworking on it so that
>> it can provide capability/services to arm64/kvm to get it virtualized.
> 
> What do you mean by virtualised? The expectation is the VMM would implement the 'firmware'
> side of this. 'events' are most likely to come from the VMM, and having to handshake with
> the kernel to work out if the event you want to inject is registered and enabled is
> over-complicated. Supporting it in the VMM means you can notify a different vCPU if that
> is appropriate, or take a different action if the event isn't registered.
> 
> This was all blocked on finding a future-proof way for tools like Qemu to consume
> reference code from ATF.
> 

Sorry that I didn't mention the story a bit last time. We plan to use SDEI to
deliver the notification (signal) from host to guest, needed by the asynchronous
page fault feature. The RFCv2 patchset was post a while ago [1]. For the SDEI
events needed by the async page fault, it's originated from KVM (host). In order
to achieve the goal, KVM needs some code so that SDEI event can be injected and
delivered. Also, the SDEI related hypercalls needs to be handled either.

Since we're here, I plan to expand the scope so that the firmware owned SDEI
events (private/shared) can be passed through to multiple VMs. Lets say they're
passthrou event. For these passthrou events, they can be shared by multiple VMs
either.

I would call this kind of feature as SDEI virtualization, but there might be
better name for it :)

[1] https://lore.kernel.org/kvmarm/924ee966-7412-f9ff-c2b0-598e4abbb05c@redhat.com/

> 
>> The
>> primary reason is we want to use SDEI to deliver the asynchronous page fault
>> notification from host to guest.
> 
> As an NMI?! Yuck!
> The SDEI handler reads memory, you'd need to stop it being re-entrant. It exits through
> the IRQ vector, (which is necessary for forward-progress given a synchronous RAS event,
> and for KVM to trigger guest-exit before the 'real' work that is offloaded to an irq
> handler can run), its going to be 'fun' to have any guarantee of forward-progress if this
> is involved with stage2.
> 

Yeah, It's something similar to NMI. The notification (signal) has to be
delivered in synchronous mode. Yes, The SDEI specification already mentioned
this: the client handler should have all required resources in place before
the handler is going to run. However, I don't see it's a problem so far.
Lets wait and see if it's a real issue until I post the RFC patchset :)

> 
>> The code of rework on SDEI client driver, including some cleanup,
> 
> Cleanup is always welcome.
> 

Thanks, James.

> 
>> is almost
>> done. Currently, I have issues on how to test/verify the client driver.
> 
> ...
> 
>> Any suggestions regarding this are appreciated.
> 
>> It seems that TRF (Trusted Firmware) is the only firmware with SDEI service
>> implemented and supported.
> 
> This project calls itself TF-A. ATF is the other widely used name. (I've never seen TRF
> before)
> 

Yeah, I must have provided wrong name. Here is the git repo I was
looking into:

    https://github.com/ARM-software/arm-trusted-firmware

> 
>> If so, does it mean I need to install TRF on my bare metal machine?
>> I'm wandering how it can be installed and not sure if
>> there is any document about this.
> 
> Firmware should come with the platform. You'd need to know intricate details about power
> management and initialising parts of the SoC to port it.
> 
> ATF has a port for the fast-model/foundation model. I test this with ATF in the fast-model.
> 

I have no idea of fast-model and foundation model, and I got nothing
from below commands in ATF git repo:

[gwshan at localhost atf]$ git grep -i fast | grep -i model
[gwshan at localhost atf]$ git grep -i fundation | grep -i model

> 
>> Besides, GHES seems the only user of SDEI in the linux kernel. If so, is
>> there a way to inject the relevant errors and how?
> 
> It is, and unfortunately last time I checked, upstream ATF doesn't have the firmware-first
> stuff for this. Its too SoC specific.
> 
> I test this by binding the fast-model's SP804 one-shot interrupt controller as an event,
> then plumbing that into GHES. Its more of case-study in why the bindable-irq stuff is
> nasty than usable error injection method.
> I can push the most recently rebased version of this, but you'd also need to hack-up a
> HEST table with GHES entries to actually get it running.
> > 
> But, unless you are working on EL3 firmare, or a VMM, I don't think SDEI is what you want.
> What problem are you trying to solve?
> 

Thanks for the information. It seems I also need to emulate SDEI event by
myself in order to test it. The best way for me is to inject SDEI event
from KVM. By the way, the code you had is part of the firmware used by
bare-metal machine or VM?

The issue we want to resolve is to deliver async page fault notification
as mentioned above. Please let me know if there are more concerns :)

Thanks,
Gavin