[PATCH v10 00/59] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support

[Marc Zyngier]

Hey Eric,

On Tue, 16 May 2023 17:53:14 +0100,
Eric Auger <eauger at redhat.com> wrote:
> 
> Hi Marc,
> 
> On 5/15/23 19:30, Marc Zyngier wrote:
> > This is the 4th drop of NV support on arm64 for this year.
> > 
> > For the previous episodes, see [1].
> > 
> > What's changed:
> > 
> > - New framework to track system register traps that are reinjected in
> >   guest EL2. It is expected to replace the discrete handling we have
> >   enjoyed so far, which didn't scale at all. This has already fixed a
> >   number of bugs that were hidden (a bunch of traps were never
> >   forwarded...). Still a work in progress, but this is going in the
> >   right direction.
> > 
> > - Allow the L1 hypervisor to have a S2 that has an input larger than
> >   the L0 IPA space. This fixes a number of subtle issues, depending on
> >   how the initial guest was created.
> > 
> > - Consequently, the patch series has gone longer again. Boo. But
> >   hopefully some of it is easier to review...
> > 
> > [1] https://lore.kernel.org/r/20230405154008.3552854-1-maz@kernel.org
> 
> I have started testing this and when booting my fedora guest I get
> 
> [  151.796544] kvm [7617]: Unsupported guest sys_reg access at:
> 23f425fd0 [80000209]
> [  151.796544]  { Op0( 3), Op1( 3), CRn(14), CRm( 3), Op2( 1), func_write },
> 
> as soon as the host has kvm-arm.mode=nested

Very odd. A write to CNTV_CTL_EL0 that fails, meaning that we have ECV
traps for the virtual timer set, and so this is probably done from
(virtual) EL2. Which of course we're not properly handling. Duh.

> This seems to be triggered very early by EDK2
> (ArmPkg/Drivers/TimerDxe/TimerDxe.c).
> 
> If I am not wrong this CNTV_CTL_EL0. Do you have any idea?

I have a good idea, but I could use some more information:

- Is this a nested guest? Or a non nested guest?

- Can you stash your EDK2 build somewhere so that I can try to
  reproduce it?

- What do you use for VMM on the host?

I'm running EDK2 on both L1 and L2 guests (some Debian builds), and I
don't see any issue. But when running EDK2 on L1, I run it non-nested.

> By the way I got this already with your v9
> (git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
> kvm-arm64/nv-6.4-WIP) and with your v10 (I cherry-picked your
> maz/kvm-arm64/nv-6.5-WIP branch).

That's an interesting data point. At least, it hasn't regressed
further... I'll try and cook a test tomorrow and run it on my own
rig. Which may or may not behave like yours, you never know...

Thanks a lot for the report!

	M.

-- 
Without deviation from the norm, progress is not possible.