[RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Eric Auger]

Hi

On 12/6/21 10:52 AM, Alexandru Elisei wrote:
> Hi,
> 
> On Sat, Dec 04, 2021 at 09:39:59AM -0800, Reiji Watanabe wrote:
>> Hi Eric,
>>
>> On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger at redhat.com> wrote:
>>>
>>> Hi Reiji,
>>>
>>> On 12/4/21 2:04 AM, Reiji Watanabe wrote:
>>>> Hi Eric,
>>>>
>>>> On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger at redhat.com> wrote:
>>>>>
>>>>> Hi Reiji,
>>>>>
>>>>> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
>>>>>> Hi Eric,
>>>>>>
>>>>>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger at redhat.com> wrote:
>>>>>>>
>>>>>>> Hi Reiji,
>>>>>>>
>>>>>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
>>>>>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
>>>>>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
>>>>>>>> expose the value for the guest as it is.  Since KVM doesn't support
>>>>>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
>>>>>>>> exopse 0x0 (PMU is not implemented) instead.
>>>>>>> s/exopse/expose
>>>>>>>>
>>>>>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
>>>>>>>> to 0x0 when it is 0xf.
>>>>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
>>>>>>> guest should not use it as a PMUv3?
>>>>>>
>>>>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
>>>>>>> guest should not use it as a PMUv3?
>>>>>>
>>>>>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
>>>>>> Arm ARM says:
>>>>>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
>>>>>>    PMUv3 not supported."
>>>>>>
>>>>>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
>>>>>> be exposed to guests (And this patch series doesn't allow userspace
>>>>>> to set the fields to 0xf for guests).
>>>>> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
>>>>> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
>>>>> init request if the host pmu is implementation defined?
>>>>
>>>> KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
>>>> kvm_reset_vcpu() if the host PMU is implementation defined.
>>>
>>> OK. This was not obvsious to me.
>>>
>>>                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
>>>                         ret = -EINVAL;
>>>                         goto out;
>>>                 }
>>>
>>> kvm_perf_init
>>> +       if (perf_num_counters() > 0)
>>> +               static_branch_enable(&kvm_arm_pmu_available);
>>>
>>> But I believe you ;-), sorry for the noise
>>
>> Thank you for the review !
>>
>> I didn't find the code above in v5.16-rc3, which is the base code of
>> this series.  So, I'm not sure where the code came from (any kvmarm
>> repository branch ??).
>>
>> What I see in v5.16-rc3 is:
>> ----
>> int kvm_perf_init(void)
>> {
>>         return perf_register_guest_info_callbacks(&kvm_guest_cbs);
>> }
>>
>> void kvm_host_pmu_init(struct arm_pmu *pmu)
>> {
>>         if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
>>             !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
>>                 static_branch_enable(&kvm_arm_pmu_available);
>> }
>> ----
>>
>> And I don't find any other code that enables kvm_arm_pmu_available.
> 
> The code was recently changed (in v5.15 I think), I think Eric is looking
> at an older version.

Yes I was "googling" kvm_arm_pmu_available enablement and I missed the
kvm_pmu_probe_pmuver() != ID_AA64DFR0_PMUVER_IMP_DEF check addition. So
except the heterogenous case reported by Alexandru below, we should be
fine. Sorry for the noise.

Thanks

Eric
> 
>>
>> Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
>> if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
>> ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
>> I think we should fix that to not allow that.
> 
> I recently started looking into that too. If there's only one PMU, then the
> guest won't see the value IMP DEF for PMUVer (userspace cannot set the PMU
> feature because !kvm_arm_support_pmu_v3()).
> 
> On heterogeneous systems with multiple PMUs, it gets complicated. I don't
> have any such hardware, but what I think will happen is that KVM will
> enable the static branch if there is at least one PMU with
> PMUVer != IMP_DEF, even if there are other PMUs with PMUVer = IMP_DEF. But
> read_sanitised_ftr_reg() will always return 0 for the
> PMUVer field because the field is defined as FTR_EXACT with a safe value of
> 0 in cpufeature.c. So the guest ends up seeing PMUVer = 0.
> 
> I'm not sure if this is the case because I'm not familiar with the cpu
> features code, but I planning to investigate further.
> 
> Thanks,
> Alex
> 
>> (I'm not sure how KVM's PMUV3 support code is implemented in the
>> code that you are looking at though)
>>
>> Thanks,
>> Reiji
>