[PATCH v11 00/43] KVM: arm64: Nested Virtualization support (FEAT_NV2 only)

[Ganapatrao Kulkarni]

On 27-11-2023 02:52 pm, Marc Zyngier wrote:
> On Mon, 27 Nov 2023 07:26:58 +0000,
> Ganapatrao Kulkarni <gankulkarni at os.amperecomputing.com> wrote:
>>
>>
>>
>> On 24-11-2023 08:02 pm, Marc Zyngier wrote:
>>> On Fri, 24 Nov 2023 13:22:22 +0000,
>>> Ganapatrao Kulkarni <gankulkarni at os.amperecomputing.com> wrote:
>>>>
>>>>> How is this value possible if the write to HCR_EL2 has taken place?
>>>>> When do you sample this?
>>>>
>>>> I am not sure how and where it got set. I think, whatever it is set,
>>>> it is due to false return of vcpu_el2_e2h_is_set(). Need to
>>>> understand/debug.
>>>> The vhcr_el2 value I have shared is traced along with hcr in function
>>>> __activate_traps/__compute_hcr.
>>>
>>> Here's my hunch:
>>>
>>> The guest boots with E2H=0, because we don't advertise anything else
>>> on your HW. So we run with NV1=1 until we try to *upgrade* to VHE. NV2
>>> means that HCR_EL2 is writable (to memory) without a trap. But we're
>>> still running with NV1=1.
>>>
>>> Subsequently, we access a sysreg that should never trap for a VHE
>>> guest, but we're with the wrong config. Bad things happen.
>>>
>>> Unfortunately, NV2 is pretty much incompatible with E2H being updated,
>>> because it cannot perform the changes that this would result into at
>>> the point where they should happen. We can try and do a best effort
>>> handling, but you can always trick it.
>>>
>>> Anyway, can you see if the hack below helps? I'm not keen on it at
>>> all, but this would be a good data point.
>>
>> Thanks Marc, this diff fixes the issue.
>> Just wondering what is changed w.r.t to L1 handling from V10 to V11
>> that it requires this trick?
> 
> Not completely sure. Before v11, anything that would trap would be
> silently handled by the FEAT_NV code. Now, a trap for something that
> is supposed to be redirected to VNCR results in an UNDEF exception.
> 
> I suspect that the exception is handled again as a call to
> __finalise_el2(), probably because the write to VBAR_EL1 didn't do
> what it was supposed to do.
> 
>> Also why this was not seen on your platform, is it E2H0 enabled?
> 
> It doesn't have FEAT_E2H0, and that's the whole point. No E2H0, no
> problems, as the guest cannot trick the host into losing track of the
> state (which I'm pretty sure can happen even with this ugly hack).
> 
> I will probably completely disable NV1 support in the next drop, and
> make NV support only VHE guests. Which is the only mode that makes any
> sense anyway.
> 

Thanks, absolutely makes sense to have *VHE-only* L1, looking forward to 
a next drop.

Thanks,
Ganapat