[PATCH v4 04/14] KVM: ARM: Initial skeleton to compile KVM support
Will Deacon
will.deacon at arm.com
Mon Nov 19 09:41:10 EST 2012
On Sat, Nov 10, 2012 at 03:42:38PM +0000, Christoffer Dall wrote:
> Targets KVM support for Cortex A-15 processors.
>
> Contains all the framework components, make files, header files, some
> tracing functionality, and basic user space API.
>
> Only supported core is Cortex-A15 for now.
>
> Most functionality is in arch/arm/kvm/* or arch/arm/include/asm/kvm_*.h.
>
> Reviewed-by: Marcelo Tosatti <mtosatti at redhat.com>
> Signed-off-by: Rusty Russell <rusty.russell at linaro.org>
> Signed-off-by: Marc Zyngier <marc.zyngier at arm.com>
> Signed-off-by: Christoffer Dall <c.dall at virtualopensystems.com>
[...]
> 4.69 KVM_GET_ONE_REG
>
> Capability: KVM_CAP_ONE_REG
> @@ -1791,6 +1800,7 @@ The list of registers accessible using this interface is identical to the
> list in 4.68.
>
>
> +
Whitespace.
> 4.70 KVM_KVMCLOCK_CTRL
>
> Capability: KVM_CAP_KVMCLOCK_CTRL
> @@ -2072,6 +2082,46 @@ KVM_S390_INT_EXTERNAL_CALL (vcpu) - sigp external call; source cpu in parm
> Note that the vcpu ioctl is asynchronous to vcpu execution.
>
>
> +4.77 KVM_ARM_VCPU_INIT
> +
> +Capability: basic
> +Architectures: arm
> +Type: vcpu ioctl
> +Parameters: struct struct kvm_vcpu_init (in)
> +Returns: 0 on success; -1 on error
> +Errors:
> + EINVAL: the target is unknown, or the combination of features is invalid.
> + ENOENT: a features bit specified is unknown.
> +
> +This tells KVM what type of CPU to present to the guest, and what
> +optional features it should have. This will cause a reset of the cpu
> +registers to their initial values. If this is not called, KVM_RUN will
> +return ENOEXEC for that vcpu.
> +
> +Note that because some registers reflect machine topology, all vcpus
> +should be created before this ioctl is invoked.
> +
> +
> +4.78 KVM_GET_REG_LIST
> +
> +Capability: basic
> +Architectures: arm
> +Type: vcpu ioctl
> +Parameters: struct kvm_reg_list (in/out)
> +Returns: 0 on success; -1 on error
> +Errors:
> + E2BIG: the reg index list is too big to fit in the array specified by
> + the user (the number required will be written into n).
> +
> +struct kvm_reg_list {
> + __u64 n; /* number of registers in reg[] */
> + __u64 reg[0];
> +};
> +
> +This ioctl returns the guest registers that are supported for the
> +KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
> +
> +
> 5. The kvm_run structure
> ------------------------
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index ade7e92..43a997a 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -2311,3 +2311,5 @@ source "security/Kconfig"
> source "crypto/Kconfig"
>
> source "lib/Kconfig"
> +
> +source "arch/arm/kvm/Kconfig"
> diff --git a/arch/arm/Makefile b/arch/arm/Makefile
> index 5f914fc..433becd 100644
> --- a/arch/arm/Makefile
> +++ b/arch/arm/Makefile
> @@ -257,6 +257,7 @@ core-$(CONFIG_XEN) += arch/arm/xen/
> core-y += arch/arm/kernel/ arch/arm/mm/ arch/arm/common/
> core-y += arch/arm/net/
> core-y += arch/arm/crypto/
> +core-y += arch/arm/kvm/
Predicate this on CONFIG_KVM_ARM_HOST, then add some obj-y to the
kvm/Makefile.
> core-y += $(machdirs) $(platdirs)
>
> drivers-$(CONFIG_OPROFILE) += arch/arm/oprofile/
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> new file mode 100644
> index 0000000..67826b2
> --- /dev/null
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -0,0 +1,114 @@
> +/*
> + * Copyright (C) 2012 - Virtual Open Systems and Columbia University
> + * Author: Christoffer Dall <c.dall at virtualopensystems.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, write to the Free Software
> + * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
> + */
> +
> +#ifndef __ARM_KVM_HOST_H__
> +#define __ARM_KVM_HOST_H__
> +
> +#include <asm/kvm.h>
> +#include <asm/kvm_asm.h>
> +
> +#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
One thing I noticed when hacking kvmtool is that we don't support the
KVM_CAP_{MAX,NR}_VCPUS capabilities. Why is this? Since we have a
maximum of 8 due to limitations of the GIC, it might be nice to put that
somewhere visible to userspace.
> +#define KVM_MEMORY_SLOTS 32
> +#define KVM_PRIVATE_MEM_SLOTS 4
> +#define KVM_COALESCED_MMIO_PAGE_OFFSET 1
> +
> +#define KVM_VCPU_MAX_FEATURES 0
> +
> +/* We don't currently support large pages. */
> +#define KVM_HPAGE_GFN_SHIFT(x) 0
> +#define KVM_NR_PAGE_SIZES 1
> +#define KVM_PAGES_PER_HPAGE(x) (1UL<<31)
> +
> +struct kvm_vcpu;
> +u32 *kvm_vcpu_reg(struct kvm_vcpu *vcpu, u8 reg_num, u32 mode);
> +int kvm_target_cpu(void);
> +int kvm_reset_vcpu(struct kvm_vcpu *vcpu);
> +void kvm_reset_coprocs(struct kvm_vcpu *vcpu);
> +
> +struct kvm_arch {
> + /* VTTBR value associated with below pgd and vmid */
> + u64 vttbr;
> +
> + /*
> + * Anything that is not used directly from assembly code goes
> + * here.
> + */
> +
> + /* The VMID generation used for the virt. memory system */
> + u64 vmid_gen;
> + u32 vmid;
> +
> + /* Stage-2 page table */
> + pgd_t *pgd;
> +};
> +
> +#define KVM_NR_MEM_OBJS 40
> +
> +/*
> + * We don't want allocation failures within the mmu code, so we preallocate
> + * enough memory for a single page fault in a cache.
> + */
> +struct kvm_mmu_memory_cache {
> + int nobjs;
> + void *objects[KVM_NR_MEM_OBJS];
> +};
> +
> +struct kvm_vcpu_arch {
> + struct kvm_regs regs;
> +
> + int target; /* Processor target */
> + DECLARE_BITMAP(features, KVM_VCPU_MAX_FEATURES);
> +
> + /* System control coprocessor (cp15) */
> + u32 cp15[NR_CP15_REGS];
> +
> + /* The CPU type we expose to the VM */
> + u32 midr;
> +
> + /* Exception Information */
> + u32 hsr; /* Hyp Syndrom Register */
pedantic: syndrome
> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
> new file mode 100644
> index 0000000..00d61a6
> --- /dev/null
> +++ b/arch/arm/include/uapi/asm/kvm.h
> @@ -0,0 +1,82 @@
> +/*
> + * Copyright (C) 2012 - Virtual Open Systems and Columbia University
> + * Author: Christoffer Dall <c.dall at virtualopensystems.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, write to the Free Software
> + * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
> + */
> +
> +#ifndef __ARM_KVM_H__
> +#define __ARM_KVM_H__
> +
> +#include <asm/types.h>
> +#include <asm/ptrace.h>
> +
> +#define __KVM_HAVE_GUEST_DEBUG
> +
> +#define KVM_REG_SIZE(id) \
> + (1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
> +
> +struct kvm_regs {
> + struct pt_regs usr_regs;/* R0_usr - R14_usr, PC, CPSR */
> + __u32 svc_regs[3]; /* SP_svc, LR_svc, SPSR_svc */
> + __u32 abt_regs[3]; /* SP_abt, LR_abt, SPSR_abt */
> + __u32 und_regs[3]; /* SP_und, LR_und, SPSR_und */
> + __u32 irq_regs[3]; /* SP_irq, LR_irq, SPSR_irq */
> + __u32 fiq_regs[8]; /* R8_fiq - R14_fiq, SPSR_fiq */
> +};
> +
> +/* Supported Processor Types */
> +#define KVM_ARM_TARGET_CORTEX_A15 0
> +#define KVM_ARM_NUM_TARGETS 1
In future, it would be nice to have a way of probing the supported
targets, rather than trying a bunch of them and seeing which ioctl
finally succeeds.
> diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
> new file mode 100644
> index 0000000..c7eb178
> --- /dev/null
> +++ b/arch/arm/kvm/Kconfig
> @@ -0,0 +1,55 @@
> +#
> +# KVM configuration
> +#
> +
> +source "virt/kvm/Kconfig"
> +
> +menuconfig VIRTUALIZATION
> + bool "Virtualization"
> + ---help---
> + Say Y here to get to see options for using your Linux host to run
> + other operating systems inside virtual machines (guests).
> + This option alone does not add any kernel code.
> +
> + If you say N, all options in this submenu will be skipped and
> + disabled.
> +
> +if VIRTUALIZATION
> +
> +config KVM
> + bool "Kernel-based Virtual Machine (KVM) support"
> + select PREEMPT_NOTIFIERS
> + select ANON_INODES
> + select KVM_MMIO
> + depends on ARM_VIRT_EXT && ARM_LPAE
> + ---help---
> + Support hosting virtualized guest machines. You will also
> + need to select one or more of the processor modules below.
> +
> + This module provides access to the hardware capabilities through
> + a character device node named /dev/kvm.
> +
> + If unsure, say N.
> +
> +config KVM_ARM_HOST
> + bool "KVM host support for ARM cpus."
> + depends on KVM
> + depends on MMU
> + depends on CPU_V7 && ARM_VIRT_EXT
ARM_VIRT_EXT already implies CPU_V7 but, as I suggested earlier, it
should probably imply ARM_LPAE which would make much of this simpler.
> + ---help---
> + Provides host support for ARM processors.
> +
> +config KVM_ARM_MAX_VCPUS
> + int "Number maximum supported virtual CPUs per VM"
> + depends on KVM_ARM_HOST
> + default 4
> + help
> + Static number of max supported virtual CPUs per VM.
> +
> + If you choose a high number, the vcpu structures will be quite
> + large, so only choose a reasonable number that you expect to
> + actually use.
> +
> +source drivers/virtio/Kconfig
> +
> +endif # VIRTUALIZATION
I guess the XEN config options should also be moved under here?
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> new file mode 100644
> index 0000000..ead5a4e
> --- /dev/null
> +++ b/arch/arm/kvm/arm.c
[...]
> +long kvm_arch_vcpu_ioctl(struct file *filp,
> + unsigned int ioctl, unsigned long arg)
> +{
> + struct kvm_vcpu *vcpu = filp->private_data;
> + void __user *argp = (void __user *)arg;
> +
> + switch (ioctl) {
> + case KVM_ARM_VCPU_INIT: {
> + struct kvm_vcpu_init init;
> +
> + if (copy_from_user(&init, argp, sizeof init))
> + return -EFAULT;
> +
> + return kvm_vcpu_set_target(vcpu, &init);
> +
> + }
> + case KVM_SET_ONE_REG:
> + case KVM_GET_ONE_REG: {
> + struct kvm_one_reg reg;
> + if (copy_from_user(®, argp, sizeof(reg)))
> + return -EFAULT;
> + if (ioctl == KVM_SET_ONE_REG)
> + return kvm_arm_set_reg(vcpu, ®);
> + else
> + return kvm_arm_get_reg(vcpu, ®);
> + }
> + case KVM_GET_REG_LIST: {
> + struct kvm_reg_list __user *user_list = argp;
> + struct kvm_reg_list reg_list;
> + unsigned n;
> +
> + if (copy_from_user(®_list, user_list, sizeof reg_list))
> + return -EFAULT;
> + n = reg_list.n;
> + reg_list.n = kvm_arm_num_regs(vcpu);
> + if (copy_to_user(user_list, ®_list, sizeof reg_list))
> + return -EFAULT;
> + if (n < reg_list.n)
> + return -E2BIG;
> + return kvm_arm_copy_reg_indices(vcpu, user_list->reg);
> + }
> + default:
> + return -EINVAL;
> + }
> +}
Your use of brackets with sizeof is inconsistent -- just always make it
look like a function call.
> diff --git a/arch/arm/kvm/emulate.c b/arch/arm/kvm/emulate.c
> new file mode 100644
> index 0000000..32e1172
> --- /dev/null
> +++ b/arch/arm/kvm/emulate.c
> @@ -0,0 +1,149 @@
> +/*
> + * Copyright (C) 2012 - Virtual Open Systems and Columbia University
> + * Author: Christoffer Dall <c.dall at virtualopensystems.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License, version 2, as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, write to the Free Software
> + * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
> + */
> +
> +#include <asm/kvm_emulate.h>
> +
> +#define VCPU_NR_MODES 6
> +#define REG_OFFSET(_reg) \
> + (offsetof(struct kvm_regs, _reg) / sizeof(u32))
> +
> +#define USR_REG_OFFSET(_num) REG_OFFSET(usr_regs.uregs[_num])
> +
> +static const unsigned long vcpu_reg_offsets[VCPU_NR_MODES][15] = {
> + /* USR/SYS Registers */
> + {
> + USR_REG_OFFSET(0), USR_REG_OFFSET(1), USR_REG_OFFSET(2),
> + USR_REG_OFFSET(3), USR_REG_OFFSET(4), USR_REG_OFFSET(5),
> + USR_REG_OFFSET(6), USR_REG_OFFSET(7), USR_REG_OFFSET(8),
> + USR_REG_OFFSET(9), USR_REG_OFFSET(10), USR_REG_OFFSET(11),
> + USR_REG_OFFSET(12), USR_REG_OFFSET(13), USR_REG_OFFSET(14),
> + },
> +
> + /* FIQ Registers */
> + {
> + USR_REG_OFFSET(0), USR_REG_OFFSET(1), USR_REG_OFFSET(2),
> + USR_REG_OFFSET(3), USR_REG_OFFSET(4), USR_REG_OFFSET(5),
> + USR_REG_OFFSET(6), USR_REG_OFFSET(7),
> + REG_OFFSET(fiq_regs[0]), /* r8 */
> + REG_OFFSET(fiq_regs[1]), /* r9 */
> + REG_OFFSET(fiq_regs[2]), /* r10 */
> + REG_OFFSET(fiq_regs[3]), /* r11 */
> + REG_OFFSET(fiq_regs[4]), /* r12 */
> + REG_OFFSET(fiq_regs[5]), /* r13 */
> + REG_OFFSET(fiq_regs[6]), /* r14 */
> + },
> +
> + /* IRQ Registers */
> + {
> + USR_REG_OFFSET(0), USR_REG_OFFSET(1), USR_REG_OFFSET(2),
> + USR_REG_OFFSET(3), USR_REG_OFFSET(4), USR_REG_OFFSET(5),
> + USR_REG_OFFSET(6), USR_REG_OFFSET(7), USR_REG_OFFSET(8),
> + USR_REG_OFFSET(9), USR_REG_OFFSET(10), USR_REG_OFFSET(11),
> + USR_REG_OFFSET(12),
> + REG_OFFSET(irq_regs[0]), /* r13 */
> + REG_OFFSET(irq_regs[1]), /* r14 */
> + },
> +
> + /* SVC Registers */
> + {
> + USR_REG_OFFSET(0), USR_REG_OFFSET(1), USR_REG_OFFSET(2),
> + USR_REG_OFFSET(3), USR_REG_OFFSET(4), USR_REG_OFFSET(5),
> + USR_REG_OFFSET(6), USR_REG_OFFSET(7), USR_REG_OFFSET(8),
> + USR_REG_OFFSET(9), USR_REG_OFFSET(10), USR_REG_OFFSET(11),
> + USR_REG_OFFSET(12),
> + REG_OFFSET(svc_regs[0]), /* r13 */
> + REG_OFFSET(svc_regs[1]), /* r14 */
> + },
> +
> + /* ABT Registers */
> + {
> + USR_REG_OFFSET(0), USR_REG_OFFSET(1), USR_REG_OFFSET(2),
> + USR_REG_OFFSET(3), USR_REG_OFFSET(4), USR_REG_OFFSET(5),
> + USR_REG_OFFSET(6), USR_REG_OFFSET(7), USR_REG_OFFSET(8),
> + USR_REG_OFFSET(9), USR_REG_OFFSET(10), USR_REG_OFFSET(11),
> + USR_REG_OFFSET(12),
> + REG_OFFSET(abt_regs[0]), /* r13 */
> + REG_OFFSET(abt_regs[1]), /* r14 */
> + },
> +
> + /* UND Registers */
> + {
> + USR_REG_OFFSET(0), USR_REG_OFFSET(1), USR_REG_OFFSET(2),
> + USR_REG_OFFSET(3), USR_REG_OFFSET(4), USR_REG_OFFSET(5),
> + USR_REG_OFFSET(6), USR_REG_OFFSET(7), USR_REG_OFFSET(8),
> + USR_REG_OFFSET(9), USR_REG_OFFSET(10), USR_REG_OFFSET(11),
> + USR_REG_OFFSET(12),
> + REG_OFFSET(und_regs[0]), /* r13 */
> + REG_OFFSET(und_regs[1]), /* r14 */
> + },
> +};
> +
> +/*
> + * Return a pointer to the register number valid in the current mode of
> + * the virtual CPU.
> + */
> +u32 *vcpu_reg(struct kvm_vcpu *vcpu, u8 reg_num)
> +{
> + u32 *reg_array = (u32 *)&vcpu->arch.regs;
> + u32 mode = *vcpu_cpsr(vcpu) & MODE_MASK;
> +
> + switch (mode) {
> + case USR_MODE...SVC_MODE:
> + mode &= ~MODE32_BIT; /* 0 ... 3 */
> + break;
> +
> + case ABT_MODE:
> + mode = 4;
> + break;
> +
> + case UND_MODE:
> + mode = 5;
> + break;
> +
> + case SYSTEM_MODE:
> + mode = 0;
> + break;
> +
> + default:
> + BUG();
> + }
> +
> + return reg_array + vcpu_reg_offsets[mode][reg_num];
> +}
]
Have some file-local #defines for those magic mode numbers.
> +
> +/*
> + * Return the SPSR for the current mode of the virtual CPU.
> + */
> +u32 *vcpu_spsr(struct kvm_vcpu *vcpu)
> +{
> + u32 mode = *vcpu_cpsr(vcpu) & MODE_MASK;
> + switch (mode) {
> + case SVC_MODE:
> + return &vcpu->arch.regs.svc_regs[2];
> + case ABT_MODE:
> + return &vcpu->arch.regs.abt_regs[2];
> + case UND_MODE:
> + return &vcpu->arch.regs.und_regs[2];
> + case IRQ_MODE:
> + return &vcpu->arch.regs.irq_regs[2];
> + case FIQ_MODE:
> + return &vcpu->arch.regs.fiq_regs[7];
> + default:
> + BUG();
> + }
> +}
Same here: have some #defines for the reg indices. They would also be
useful to userspace when manipulating the vcpu state (currently there is
just a comment in the header).
Will
More information about the linux-arm-kernel
mailing list