[PATCH v10 00/59] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support

[Marc Zyngier]

On Wed, 17 May 2023 09:59:45 +0100,
Eric Auger <eauger at redhat.com> wrote:
> 
> Hi Marc,
> 
> On 5/16/23 22:28, Marc Zyngier wrote:
> > On Tue, 16 May 2023 17:53:14 +0100,
> > Eric Auger <eauger at redhat.com> wrote:
> >>
> >> Hi Marc,
> >>
> >> On 5/15/23 19:30, Marc Zyngier wrote:
> >>> This is the 4th drop of NV support on arm64 for this year.
> >>>
> >>> For the previous episodes, see [1].
> >>>
> >>> What's changed:
> >>>
> >>> - New framework to track system register traps that are reinjected in
> >>>   guest EL2. It is expected to replace the discrete handling we have
> >>>   enjoyed so far, which didn't scale at all. This has already fixed a
> >>>   number of bugs that were hidden (a bunch of traps were never
> >>>   forwarded...). Still a work in progress, but this is going in the
> >>>   right direction.
> >>>
> >>> - Allow the L1 hypervisor to have a S2 that has an input larger than
> >>>   the L0 IPA space. This fixes a number of subtle issues, depending on
> >>>   how the initial guest was created.
> >>>
> >>> - Consequently, the patch series has gone longer again. Boo. But
> >>>   hopefully some of it is easier to review...
> >>>
> >>> [1] https://lore.kernel.org/r/20230405154008.3552854-1-maz@kernel.org
> >>
> >> I have started testing this and when booting my fedora guest I get
> >>
> >> [  151.796544] kvm [7617]: Unsupported guest sys_reg access at:
> >> 23f425fd0 [80000209]
> >> [  151.796544]  { Op0( 3), Op1( 3), CRn(14), CRm( 3), Op2( 1), func_write },
> >>
> >> as soon as the host has kvm-arm.mode=nested
> >>
> >> This seems to be triggered very early by EDK2
> >> (ArmPkg/Drivers/TimerDxe/TimerDxe.c).
> >>
> >> If I am not wrong this CNTV_CTL_EL0. Do you have any idea?
> > 
> > So here's my current analysis:
> > 
> > I assume you are running EDK2 as the L1 guest in a nested
> > configuration. I also assume that you are not running on an Apple
> > CPU. If these assumptions are correct, then EDK2 runs at vEL2, and is
> > in nVHE mode.
> > 
> > Finally, I'm going to assume that your implementation has FEAT_ECV and
> > FEAT_NV2, because I can't see how it could fail otherwise.
> all the above is correct.
> > 
> > In these precise conditions, KVM sets the CNTHCTL_EL2.EL1TVT bit so
> > that we can trap the EL0 virtual timer and faithfully emulate it (it
> > is otherwise written to memory, which isn't very helpful).
> 
> indeed
> > 
> > As it turns out, we don't handle these traps. I didn't spot it because
> > my test machines are all Apple boxes that don't have a nVHE mode, so
> > nothing on the nVHE path is getting *ANY* coverage. Hint: having
> > access to such a machine would help (shipping address on request!).
> > Otherwise, I'll eventually kill the nVHE support altogether.
> > 
> > I have written the following patch, which compiles, but that I cannot
> > test with my current setup. Could you please give it a go?
> 
> with the patch below, my guest boots nicely. You did it great on the 1st
> shot!!! So this fixes my issue. I will continue testing the v10.

Thanks a lot for reporting the issue and testing my hacks. I'll
eventually fold it into the rest of the series.

By the way, what are you using as your VMM? I'd really like to
reproduce your setup.

Cheers,

	M.

-- 
Without deviation from the norm, progress is not possible.