[PATCH v7 07/16] arm64: kvm: allows kvm cpu hotplug

[Marc Zyngier]

On 25/04/16 10:16, James Morse wrote:
> Hi,
> 
> On 25/04/16 09:41, AKASHI Takahiro wrote:
>> On Wed, Apr 20, 2016 at 12:19:45PM +0100, James Morse wrote:
>>> On 20/04/16 11:37, Marc Zyngier wrote:
>>>> On 19/04/16 18:37, James Morse wrote:
>>>>> It looks like x86 uses the extable to work around this, their vmx_vcpu_run() has:
>>>>>> 		__ex(ASM_VMX_VMLAUNCH) "\n\t"
>>>>> Where __ex ends up calling ____kvm_handle_fault_on_reboot(), with a nearby comment:
>>>>>> * Hardware virtualization extension instructions may fault if a
>>>>>> * reboot turns off virtualization while processes are running.
>>>>>> * Trap the fault and ignore the instruction if that happens.
>>>>
>>>> I very much like that approach, to be honest. Tearing down a CPU is
>>>> something exceptional, so let's make it an actual exception.
>>>>
>>>> It is now pretty easy to discriminate between KVM functions and stub
>>>> functions thanks to your earlier patch, so if we end up calling the
>>>> hyp-stub because we've torn down KVM's EL2, let's just return an
>>>> appropriate error code (ARM_EXCEPTION_HYP_GONE), and handle it at EL1.
>>>
>>> Okay. kexec uses kvm_call_hyp() against the hyp-stub to do the kernel-copy and
>>> hand over to purgatory, but we could change that to a new 'special' builtin
>>> call, something like HVC_KEXEC_CALL_HYP. It never calls it with kvm loaded, so
>>> there is no reason the calls have to be same.
>>>
>>> Given hibernate doesn't hit this issue, I will drop this hunk from this version
>>> of the patch, and repost hibernate incorporating the feedback so far. I will
>>> provide a patch for kexec to do the above.
>>
>> Thanks, but you don' have to.
> 
> I was wrong with the 'hibernate doesn't hit this issue', with this patch we
> re-install the hyp-stub during system reboot, and race with the scheduler.
> ('reboot -f' while running a guest).
> 
> 
>> If the fix below is acceptable, we will merge it to our next kexec/kdump
>> patch series.
> 
> I'm testing something that looks very similar at the moment.
> 
> 
>> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
>> index eba89e4..31b5224 100644
>> --- a/arch/arm64/kvm/handle_exit.c
>> +++ b/arch/arm64/kvm/handle_exit.c
>> @@ -186,6 +186,10 @@ int handle_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>  		exit_handler = kvm_get_exit_handler(vcpu);
>>  
>>  		return exit_handler(vcpu, run);
>> +	case ARM_EXCEPTION_HYP_GONE:
>> +		/* due to kexec reboot */
>> +		run->exit_reason = KVM_EXIT_SHUTDOWN;
>> +		return 0;
> 
> Is it fair to throw this back out to user space? While the hypervisor doesn't
> have long to live, it may not be expecting this exit_reason. I couldn't see a
> value for 'suprise cpu removal', and it looks like the x86 code causes the vcpu
> to spin round enter guest...

Yeah, KVM_EXIT_SHUTDOWN is a vcpu reset (which on x86 is caused by a
triple fault). KVM_EXIT_FAIL_ENTRY seems slightly better.

As for getting back to userspace, I don't see that as a problem (though
the documentation on that part of the API is... sparse).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...