[PATCH v11 00/43] KVM: arm64: Nested Virtualization support (FEAT_NV2 only)

[Ganapatrao Kulkarni]

On 27-11-2023 05:15 pm, Marc Zyngier wrote:
> On Mon, 27 Nov 2023 10:59:36 +0000,
> Ganapatrao Kulkarni <gankulkarni at os.amperecomputing.com> wrote:
>>
>>
>>
>> On 27-11-2023 02:52 pm, Marc Zyngier wrote:
>>> On Mon, 27 Nov 2023 07:26:58 +0000,
>>> Ganapatrao Kulkarni <gankulkarni at os.amperecomputing.com> wrote:
>>>>
>>>>
>>>>
>>>> On 24-11-2023 08:02 pm, Marc Zyngier wrote:
>>>>> On Fri, 24 Nov 2023 13:22:22 +0000,
>>>>> Ganapatrao Kulkarni <gankulkarni at os.amperecomputing.com> wrote:
>>>>>>
>>>>>>> How is this value possible if the write to HCR_EL2 has taken place?
>>>>>>> When do you sample this?
>>>>>>
>>>>>> I am not sure how and where it got set. I think, whatever it is set,
>>>>>> it is due to false return of vcpu_el2_e2h_is_set(). Need to
>>>>>> understand/debug.
>>>>>> The vhcr_el2 value I have shared is traced along with hcr in function
>>>>>> __activate_traps/__compute_hcr.
>>>>>
>>>>> Here's my hunch:
>>>>>
>>>>> The guest boots with E2H=0, because we don't advertise anything else
>>>>> on your HW. So we run with NV1=1 until we try to *upgrade* to VHE. NV2
>>>>> means that HCR_EL2 is writable (to memory) without a trap. But we're
>>>>> still running with NV1=1.
>>>>>
>>>>> Subsequently, we access a sysreg that should never trap for a VHE
>>>>> guest, but we're with the wrong config. Bad things happen.
>>>>>
>>>>> Unfortunately, NV2 is pretty much incompatible with E2H being updated,
>>>>> because it cannot perform the changes that this would result into at
>>>>> the point where they should happen. We can try and do a best effort
>>>>> handling, but you can always trick it.
>>>>>
>>>>> Anyway, can you see if the hack below helps? I'm not keen on it at
>>>>> all, but this would be a good data point.
>>>>
>>>> Thanks Marc, this diff fixes the issue.
>>>> Just wondering what is changed w.r.t to L1 handling from V10 to V11
>>>> that it requires this trick?
>>>
>>> Not completely sure. Before v11, anything that would trap would be
>>> silently handled by the FEAT_NV code. Now, a trap for something that
>>> is supposed to be redirected to VNCR results in an UNDEF exception.
>>>
>>> I suspect that the exception is handled again as a call to
>>> __finalise_el2(), probably because the write to VBAR_EL1 didn't do
>>> what it was supposed to do.
>>>
>>>> Also why this was not seen on your platform, is it E2H0 enabled?
>>>
>>> It doesn't have FEAT_E2H0, and that's the whole point. No E2H0, no
>>> problems, as the guest cannot trick the host into losing track of the
>>> state (which I'm pretty sure can happen even with this ugly hack).
>>>
>>> I will probably completely disable NV1 support in the next drop, and
>>> make NV support only VHE guests. Which is the only mode that makes any
>>> sense anyway.
>>>
>>
>> Thanks, absolutely makes sense to have *VHE-only* L1, looking forward
>> to a next drop.
> 
> Note that this won't be restricted to L1, but will affect *everything.
> 
Ok.

> No non-VHE guest will be supported at any level whatsoever, and NV
> will always expose ID_AA64MMFR4_EL1.E2H0=0b1110, indicating that
> HCR_EL2.NV1 is RES0, on top of ID_AA64MMFR4_EL1.NV_frac=1 (NV2 only).

OK, Even I was thinking the same instead of the work-around/trick, then 
I felt it is still needed since the L1 may be any distro kernel and it 
may not have code to interpret/decode ID_AA64MMFR4_EL1.E2H0.


Thanks,
Ganapat
> 
> 	M.
>