[RFC PATCH 0/2] Add support for a fake, para-virtualised machine

[Stefano Stabellini]

sorry if I missed anybody from CC, I don't have the original list

On Tue, 4 Dec 2012, Will Deacon wrote:
> On Mon, Dec 03, 2012 at 09:54:09PM +0000, Rob Herring wrote:
> > On 12/03/2012 11:52 AM, Will Deacon wrote:
> > > When running Linux on a para-virtualised platform (that is, one where
> > > the guest is aware that it is dealing with virtual devices sitting on
> > > things like virtio or xenbus) we require very little in the way of
> > > platform code and piggy-backing on top of an existing platform can
> > > require a lot of device emulation for very little gain.
> > > 
> > > These two patches introduce mach-virt: a very simple, DT-based machine
> > > which can be used with kvmtool in conjunction with virtio-based devices.
> > > It's not hard to imagine the same machine being targetted by Xen, which
> > > currently emulates a minimal variant of the vexpress platform.
> > > 
> > > Note that this patch series depends on the timer rework from Mark
> > > Rutland, posted on Friday:
> > > 
> > >   http://lists.infradead.org/pipermail/linux-arm-kernel/2012-November/135651.html
> > > 
> > > All feedback welcome. We suspect that most controversy will be around
> > > the name of the thing :)
> > 
> > We've discussed this before at conferences. I don't know that we
> > concluded this wasn't needed, but it certainly leaned that direction.
> 
> I too leaned that direction before I started looking at kvm in detail and,
> since then, I've changed my mind when it comes to para-virtualisation.
> 
> The reason for this is that there is absolutely no reason to emulate some
> components of a real platform and then bolt virtual devices onto it once
> you've got enough to get it going. It leads to a right royal mess in
> userspace, where you have to write a load of non-reusable emulation code and
> it leads to churn in the kernel because you're constantly at odds with
> people trying to develop the platform code based on the actual hardware
> they have.
> 
> With a virtualisation-capable ARMv7 system, all you *need* to boot SMP
> Debian is:
> 
> 	- A v7 CPU with virt extensions
> 	- vGIC
> 	- architected timers
> 
> *everything* else can be described using virtio devices in the device-tree,
> essentially allowing you to generate platforms based on the above and boot
> the same kernel on them.

That's right, however for Xen there would be a single hypervisor node
and no virtio devices: everything else is going to be on xenbus.


> > So what has changed? You're not going to save code space because we're
> > building multiple platforms together. You'll save some boot time, but a
> > stripped down dtb with only the minimal peripherals would probably save
> > nearly as much time. 
> 
> It's really got nothing to do with code space or boot speed. What it *is*
> about is avoiding the tight coupling with a real platform and suffering as a
> result. Yes, you can strip down the DT for a real platform but you'll likely
> still have to emulate things like the SP804 in order to boot. That's not to
> mention any platform-specific system register interfaces which are required
> early on.
> 
> We can't even re-use the socfpga code (which is incredibly minimal) without
> emulating the dw_apb_timer.
> 
> > However, I do have concerns with using VExpress as
> > the guest. For example, you can't support a non-PAE guest with 4GB of
> > RAM on VExpress (maybe if the vexpress code gets all memory map info
> > from DT).
> 
> Yes, vexpress is even less suitable for this.

My understanding was that vexpress was supposed to become a machine
fully discoverable via DT, and that of course includes the memory map.
Actually I think that we are pretty close to that already.

However if you prefer to introduce a new machine for that purpose, I am
OK with that too. I am sure we can base xenvm on that.


> > Is this really complete? Will we need reset, poweroff, hotplug, and
> > suspend/resume support for example? Unlike most initial platform
> > submissions which are minimal, I think seeing full support would be
> > useful here. Then we can better gauge how much we are really saving.
> 
> The code is complete in the sense that you can boot an SMP guest running
> Debian with console, network, block etc. etc. but you're right to point out
> the absence of power-management support.
> 
> However, power-management in KVM guests is a *much* larger problem and not
> one that has been solved adequately as of yet. There are suggestions that it
> should be handled entirely in firmware, with the guest making smc calls to
> request power-management operations but this is yet to materialise and, as
> such, we can't yet use it here.
> 
> We could look at building a virtio-based power controller but that's going
> to come up too late for SMP booting (although will give us hotplug, reset
> etc).

In Xen x86 we implement hotplug via xenbus and power management via
hypercalls. So we should be able to get away with it wihout doing any
emulation or virtio.