[RFC PATCH 00/45] KVM: arm/arm64: Rework virtual GIC emulation

[Christoffer Dall]

On Wed, Mar 30, 2016 at 03:13:20PM +0100, Vladimir Murzin wrote:
> On 30/03/16 14:56, Christoffer Dall wrote:
> > Hi Vladimir,
> > 
> > On Wed, Mar 30, 2016 at 12:52:27PM +0100, Vladimir Murzin wrote:
> >> On 30/03/16 12:42, Vladimir Murzin wrote:
> >>> On 29/03/16 14:12, Vladimir Murzin wrote:
> >>>> Hi Andre,
> >>>>
> >>>> On 25/03/16 02:04, Andre Przywara wrote:
> >>>>> Please have a look at the series, review it and give the code some
> >>>>> serious testing (and possibly debugging). All feedback is appreciated.
> >>>>
> >>>> I see that with the new vgic implementation kvmtool starts throwing:
> >>>>
> >>>> kvm [1273]: Unable to register VGICv3 redist MMIO regions
> >>>>
> >>>
> >>> It comes from kvm_io_bus_register_dev()
> >>>
> >>> if (bus->dev_count - bus->ioeventfd_count > NR_IOBUS_DEVS - 1)
> >>> 	return -ENOSPC;
> >>>
> >>> with bus->dev_count being 1000
> >>>
> >>> [<ffffffc00009f93c>] kvm_io_bus_register_dev+0x130/0x148
> >>> [<ffffffc0000ad274>] kvm_vgic_register_mmio_region+0xac/0xdc
> >>> [<ffffffc0000ad4d4>] vgic_register_redist_regions+0xb8/0x158
> >>> [<ffffffc0000abb90>] vgic_v3_map_resources+0x5c/0xf0
> >>> [<ffffffc0000aae04>] kvm_vgic_map_resources+0x40/0x84
> >>> [<ffffffc0000a23d0>] kvm_arch_vcpu_ioctl_run+0x3f0/0x400
> >>> [<ffffffc00009d484>] kvm_vcpu_ioctl+0x2d4/0x6ec
> >>> [<ffffffc0001c4eec>] do_vfs_ioctl+0xb4/0x760
> >>> [<ffffffc0001c561c>] SyS_ioctl+0x84/0x98
> >>> [<ffffffc000085d30>] el0_svc_naked+0x24/0x28
> >>>
> >>
> >> and one more thing I've forgotten to mention... something odd happens on
> >> destroy path
> >>
> >> INFO: task kvm-vcpu-0:1123 blocked for more than 120 seconds.
> >>       Tainted: G        W       4.5.0-rc6+ #432
> >> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> >> kvm-vcpu-0      D ffffffc000086bcc     0  1123   1114 0x00000008
> >> Call trace:
> >> [<ffffffc000086bcc>] __switch_to+0x90/0xa4
> >> [<ffffffc0006c4390>] __schedule+0x188/0x59c
> >> [<ffffffc0006c47e0>] schedule+0x3c/0xa0
> >> [<ffffffc0006c4bf4>] schedule_preempt_disabled+0x20/0x38
> >> [<ffffffc0006c618c>] __mutex_lock_slowpath+0xc4/0x148
> >> [<ffffffc0006c6254>] mutex_lock+0x44/0x5c
> >> [<ffffffc0000aacb8>] kvm_vgic_destroy+0x20/0xb0
> >> [<ffffffc0000abbdc>] vgic_v3_map_resources+0xa8/0xf0
> >> [<ffffffc0000aae04>] kvm_vgic_map_resources+0x40/0x84
> >> [<ffffffc0000a23d0>] kvm_arch_vcpu_ioctl_run+0x3f0/0x400
> >> [<ffffffc00009d484>] kvm_vcpu_ioctl+0x2d4/0x6ec
> >> [<ffffffc0001c4eec>] do_vfs_ioctl+0xb4/0x760
> >> [<ffffffc0001c561c>] SyS_ioctl+0x84/0x98
> >> [<ffffffc000085d30>] el0_svc_naked+0x24/0x28
> >>
> > 
> > Is this also with many many VCPUs or in general?
> > 
> 
> It's seen with many many VCPUs. Basically, I see
> 
> kvm [1273]: Unable to register VGICv3 redist MMIO regions
> 
> and then (if I'm patient enough) I see that backtrace. So the flow looks
> like:
> 
> el0_svc_naked
> ...
> vgic_v3_map_resources
>    vgic_register_redist_regions
>        kvm_vgic_register_mmio_region
>            kvm_io_bus_register_dev	// return -ENOSPC
>    kvm_vgic_destroy
>        mutex_lock
> 

ok, thanks for the clarification!

-Christoffer