[RFC PATCH 00/45] KVM: arm/arm64: Rework virtual GIC emulation

[Vladimir Murzin]

On 30/03/16 14:56, Christoffer Dall wrote:
> Hi Vladimir,
> 
> On Wed, Mar 30, 2016 at 12:52:27PM +0100, Vladimir Murzin wrote:
>> On 30/03/16 12:42, Vladimir Murzin wrote:
>>> On 29/03/16 14:12, Vladimir Murzin wrote:
>>>> Hi Andre,
>>>>
>>>> On 25/03/16 02:04, Andre Przywara wrote:
>>>>> Please have a look at the series, review it and give the code some
>>>>> serious testing (and possibly debugging). All feedback is appreciated.
>>>>
>>>> I see that with the new vgic implementation kvmtool starts throwing:
>>>>
>>>> kvm [1273]: Unable to register VGICv3 redist MMIO regions
>>>>
>>>
>>> It comes from kvm_io_bus_register_dev()
>>>
>>> if (bus->dev_count - bus->ioeventfd_count > NR_IOBUS_DEVS - 1)
>>> 	return -ENOSPC;
>>>
>>> with bus->dev_count being 1000
>>>
>>> [<ffffffc00009f93c>] kvm_io_bus_register_dev+0x130/0x148
>>> [<ffffffc0000ad274>] kvm_vgic_register_mmio_region+0xac/0xdc
>>> [<ffffffc0000ad4d4>] vgic_register_redist_regions+0xb8/0x158
>>> [<ffffffc0000abb90>] vgic_v3_map_resources+0x5c/0xf0
>>> [<ffffffc0000aae04>] kvm_vgic_map_resources+0x40/0x84
>>> [<ffffffc0000a23d0>] kvm_arch_vcpu_ioctl_run+0x3f0/0x400
>>> [<ffffffc00009d484>] kvm_vcpu_ioctl+0x2d4/0x6ec
>>> [<ffffffc0001c4eec>] do_vfs_ioctl+0xb4/0x760
>>> [<ffffffc0001c561c>] SyS_ioctl+0x84/0x98
>>> [<ffffffc000085d30>] el0_svc_naked+0x24/0x28
>>>
>>
>> and one more thing I've forgotten to mention... something odd happens on
>> destroy path
>>
>> INFO: task kvm-vcpu-0:1123 blocked for more than 120 seconds.
>>       Tainted: G        W       4.5.0-rc6+ #432
>> "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> kvm-vcpu-0      D ffffffc000086bcc     0  1123   1114 0x00000008
>> Call trace:
>> [<ffffffc000086bcc>] __switch_to+0x90/0xa4
>> [<ffffffc0006c4390>] __schedule+0x188/0x59c
>> [<ffffffc0006c47e0>] schedule+0x3c/0xa0
>> [<ffffffc0006c4bf4>] schedule_preempt_disabled+0x20/0x38
>> [<ffffffc0006c618c>] __mutex_lock_slowpath+0xc4/0x148
>> [<ffffffc0006c6254>] mutex_lock+0x44/0x5c
>> [<ffffffc0000aacb8>] kvm_vgic_destroy+0x20/0xb0
>> [<ffffffc0000abbdc>] vgic_v3_map_resources+0xa8/0xf0
>> [<ffffffc0000aae04>] kvm_vgic_map_resources+0x40/0x84
>> [<ffffffc0000a23d0>] kvm_arch_vcpu_ioctl_run+0x3f0/0x400
>> [<ffffffc00009d484>] kvm_vcpu_ioctl+0x2d4/0x6ec
>> [<ffffffc0001c4eec>] do_vfs_ioctl+0xb4/0x760
>> [<ffffffc0001c561c>] SyS_ioctl+0x84/0x98
>> [<ffffffc000085d30>] el0_svc_naked+0x24/0x28
>>
> 
> Is this also with many many VCPUs or in general?
> 

It's seen with many many VCPUs. Basically, I see

kvm [1273]: Unable to register VGICv3 redist MMIO regions

and then (if I'm patient enough) I see that backtrace. So the flow looks
like:

el0_svc_naked
...
vgic_v3_map_resources
   vgic_register_redist_regions
       kvm_vgic_register_mmio_region
           kvm_io_bus_register_dev	// return -ENOSPC
   kvm_vgic_destroy
       mutex_lock

Cheers
Vladimir

> Thanks,
> -Christoffer
> 
>