[PATCH v8 01/69] arm64: Add ARM64_HAS_NESTED_VIRT cpufeature

[Marc Zyngier]

On Tue, 31 Jan 2023 20:04:15 +0000,
Oliver Upton <oliver.upton at linux.dev> wrote:
> 
> On Tue, Jan 31, 2023 at 05:34:39PM +0000, Suzuki K Poulose wrote:
> > On 31/01/2023 14:00, Marc Zyngier wrote:
> 
> [...]
> 
> > > What is exactly the objection here? NV is more or less a VHE++ mode,
> > > but is also completely experimental and incomplete.
> > 
> > I am all in for making this an "optional", only enabled it when "I know
> > what I want".
> > 
> > kvm-arm.mode=nv kind of seems that the KVM driver is conditioned
> > mainly for running NV (comparing with the other existing options
> > for kvm-arm.mode).
> > 
> > In reality, as you confirmed, NV is an *additional* capability
> > of a VHE hypervisor. So it would be good to "opt" in for "nv" capability
> > support.
> > 
> > e.g,
> > 
> >    kvm-arm.nv=on
> > 
> > Thinking more about it, either is fine.
> 
> Marc, I'm curious, how do you plan to glue hVHE + NV together (if at
> all)? We may need two separate options for this so the user could
> separately configure NV for their hVHE KVM instance.

I really don't plan to support them together. But if we wanted to
support something, I'd rather express it as a composition of the
various options, as I suggested to Suzuki earlier. Something along the
lines of:

	kvm-arm.mode=hvhe,nested

But the extra complexity is mind-boggling, frankly. And the result
will suck terribly. NV is already exit-heavy, compared to single-level
virtualisation. But now you double the cost of the exit by moving all
the load/put work into the entry-exit phase.

To give you an idea, an L2 guest under a hVHE L1 is ~30% slower than
the same guest running under a VHE L1 with an exit-heavy workload
(virtio-9p + hackbench). Making the L0 host hVHE would be even worse.
We may be able to improve it to some extent, but it will always be the
sucky option.

Also, given where we are at the support stage (we basically offer an
ARMv8.1 L1 environment), the impetus to support this sort of
contraption is..  hrmm... low. I'd rather spend my energy on
architecture correctness and feature-parity with single level.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.