[RFC PATCH 0/6] KVM: arm64: Errata management for VM Live migration

[Cornelia Huck]

On Fri, Oct 11 2024, Shameerali Kolothum Thodi <shameerali.kolothum.thodi at huawei.com> wrote:

>> -----Original Message-----
>> From: Cornelia Huck <cohuck at redhat.com>
>> Sent: Friday, October 11, 2024 2:18 PM
>> To: Shameerali Kolothum Thodi
>> <shameerali.kolothum.thodi at huawei.com>; kvmarm at lists.linux.dev;
>> maz at kernel.org; oliver.upton at linux.dev
>> Cc: catalin.marinas at arm.com; will at kernel.org; mark.rutland at arm.com;
>> eric.auger at redhat.com; yuzenghui <yuzenghui at huawei.com>; Wangzhou
>> (B) <wangzhou1 at hisilicon.com>; jiangkunkun
>> <jiangkunkun at huawei.com>; Jonathan Cameron
>> <jonathan.cameron at huawei.com>; Anthony Jebson
>> <anthony.jebson at huawei.com>; linux-arm-kernel at lists.infradead.org;
>> Linuxarm <linuxarm at huawei.com>
>> Subject: Re: [RFC PATCH 0/6] KVM: arm64: Errata management for VM Live
>> migration
>> 
>> On Fri, Oct 11 2024, Shameer Kolothum
>> <shameerali.kolothum.thodi at huawei.com> wrote:
>> 
>> > Hi,
>> >
>> > On ARM64 platforms most of the errata workarounds are based on CPU
>> > MIDR/REVIDR values and a number of these workarounds need to be
>> > implemented by the Guest kernel as well. This creates a problem when
>> > Guest needs to be migrated to a platform that differs in these
>> > MIDR/REVIDR values even if the VMM can come up with a common
>> minimum
>> > feature list for the Guest using the recently introduced "Writable
>> > ID registers" support.
>> >
>> > (This is roughly based on a discussion I had with Marc and Oliver
>> > at KVM forum. Marc outlined his idea for a solution and this is an
>> > attempt to implement it. Thanks to both and I take all the blame
>> > if this is nowhere near what is intended/required)
>> >
>> > This RFC proposes a solution to handle the above issue by introducing
>> > the following,
>> >
>> > 1. A new VM IOCTL,
>> >    KVM_ARM_SET_MIGRN_TARGET_CPUS  _IOW(KVMIO,  0xb7, struct
>> kvm_arm_migrn_cpus)
>> >    This can be used by the userspace(VMM) to set the target CPUs the
>> >    Guest will run in its lifetime. See patch #2
>> > 2. Add hypercall support for Guest kernel to retrieve any migration
>> >    errata bitmap(ARM_SMCCC_VENDOR_HYP_KVM_MIGRN_ERRATA)
>> >    The above will return the bitmaps in R0-R3 registers. See patch #4
>> > 3. The "capability" field in struct arm64_cpu_capabilities is a generated
>> >    one at present and may get renumbered or reordered. Hence, we can't
>> use
>> >    this directly for migration errata bitmaps. Instead, introduced
>> >    "migartion_safe_cap", which has to be set statically for any
>> >    erratum that needs to be enabled and is safe for migration
>> >    purposes. See patches 3 & 6.
>> > 4. Rest of the patches includes the plumbing required to populate the
>> >    errata bitmap based on the target CPUs set by the VMM and update the
>> >    system_cap based on it.
>> >
>> > ToDos:-
>> >   -We still need a way to  handle the error in setting the invariant
>> >    registers(MIDR/REVIDR/AIDR) during Guest migration. Perhaps we can
>> >    handle it in userspace?
>> > -  Possibly we could do better to avoid the additional
>> "migartion_safe_cap" use.
>> >    Suggestions welcome.
>> >   -There are errata that require more than MIDR/REVIDR, eg: CTR_EL0.
>> >    How to handle those?
>> >   -Check for locking requirements if any.
>> >
>> > This is lightly tested on a HiSilicon ARM64 platform.
>> >
>> > Please take a look and let me know your thoughts.
>> 
>> So, I've only taken a very quick look at it, but IIUC, the idea is for
>> the VMM to do the following:
>> - figure out where we want to possibly run
>
>
> Yes.
>
>> - figure out the least common denominator for writable features
>
> Yes
>
>> - tell KVM about the possible target cpu resp. the errata it wants
>
> This might change as per Marc's comments. VMM has to handle hypercall
> directly and provide the target CPU list to Guest.
>
>> - build a frankencpu via the writable id reg infrastructure, fiddling
>>   with invariant handling as needed?
>
> That is my idea of handling the invariant registers. If the VMM has provided a list of 
> target CPUs for the Guest,  make sure the incoming CPU is part of the list 
> during migration and ignore invariant(MIDR/REVIDR) reg SET errors.

Thanks!

I hope I'll be able to spend some time on this next week.