[RFC PATCH v3 09/29] KVM: arm64: Hide IMPLEMENTATION DEFINED PMU support for the guest

[Reiji Watanabe]

Hi Eric,

On Mon, Dec 6, 2021 at 2:25 AM Eric Auger <eauger at redhat.com> wrote:
>
> Hi
>
> On 12/6/21 10:52 AM, Alexandru Elisei wrote:
> > Hi,
> >
> > On Sat, Dec 04, 2021 at 09:39:59AM -0800, Reiji Watanabe wrote:
> >> Hi Eric,
> >>
> >> On Sat, Dec 4, 2021 at 6:14 AM Eric Auger <eauger at redhat.com> wrote:
> >>>
> >>> Hi Reiji,
> >>>
> >>> On 12/4/21 2:04 AM, Reiji Watanabe wrote:
> >>>> Hi Eric,
> >>>>
> >>>> On Thu, Dec 2, 2021 at 2:57 AM Eric Auger <eauger at redhat.com> wrote:
> >>>>>
> >>>>> Hi Reiji,
> >>>>>
> >>>>> On 11/30/21 6:32 AM, Reiji Watanabe wrote:
> >>>>>> Hi Eric,
> >>>>>>
> >>>>>> On Thu, Nov 25, 2021 at 12:30 PM Eric Auger <eauger at redhat.com> wrote:
> >>>>>>>
> >>>>>>> Hi Reiji,
> >>>>>>>
> >>>>>>> On 11/17/21 7:43 AM, Reiji Watanabe wrote:
> >>>>>>>> When ID_AA64DFR0_EL1.PMUVER or ID_DFR0_EL1.PERFMON is 0xf, which
> >>>>>>>> means IMPLEMENTATION DEFINED PMU supported, KVM unconditionally
> >>>>>>>> expose the value for the guest as it is.  Since KVM doesn't support
> >>>>>>>> IMPLEMENTATION DEFINED PMU for the guest, in that case KVM should
> >>>>>>>> exopse 0x0 (PMU is not implemented) instead.
> >>>>>>> s/exopse/expose
> >>>>>>>>
> >>>>>>>> Change cpuid_feature_cap_perfmon_field() to update the field value
> >>>>>>>> to 0x0 when it is 0xf.
> >>>>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> >>>>>>> guest should not use it as a PMUv3?
> >>>>>>
> >>>>>>> is it wrong to expose the guest with a Perfmon value of 0xF? Then the
> >>>>>>> guest should not use it as a PMUv3?
> >>>>>>
> >>>>>> For the value 0xf in ID_AA64DFR0_EL1.PMUVER and ID_DFR0_EL1.PERFMON,
> >>>>>> Arm ARM says:
> >>>>>>   "IMPLEMENTATION DEFINED form of performance monitors supported,
> >>>>>>    PMUv3 not supported."
> >>>>>>
> >>>>>> Since the PMU that KVM supports for guests is PMUv3, 0xf shouldn't
> >>>>>> be exposed to guests (And this patch series doesn't allow userspace
> >>>>>> to set the fields to 0xf for guests).
> >>>>> What I don't get is why this isn't detected before (in kvm_reset_vcpu).
> >>>>> if the VCPU was initialized with KVM_ARM_VCPU_PMU_V3 can we honor this
> >>>>> init request if the host pmu is implementation defined?
> >>>>
> >>>> KVM_ARM_VCPU_INIT with KVM_ARM_VCPU_PMU_V3 will fail in
> >>>> kvm_reset_vcpu() if the host PMU is implementation defined.
> >>>
> >>> OK. This was not obvsious to me.
> >>>
> >>>                 if (kvm_vcpu_has_pmu(vcpu) && !kvm_arm_support_pmu_v3()) {
> >>>                         ret = -EINVAL;
> >>>                         goto out;
> >>>                 }
> >>>
> >>> kvm_perf_init
> >>> +       if (perf_num_counters() > 0)
> >>> +               static_branch_enable(&kvm_arm_pmu_available);
> >>>
> >>> But I believe you ;-), sorry for the noise
> >>
> >> Thank you for the review !
> >>
> >> I didn't find the code above in v5.16-rc3, which is the base code of
> >> this series.  So, I'm not sure where the code came from (any kvmarm
> >> repository branch ??).
> >>
> >> What I see in v5.16-rc3 is:
> >> ----
> >> int kvm_perf_init(void)
> >> {
> >>         return perf_register_guest_info_callbacks(&kvm_guest_cbs);
> >> }
> >>
> >> void kvm_host_pmu_init(struct arm_pmu *pmu)
> >> {
> >>         if (pmu->pmuver != 0 && pmu->pmuver != ID_AA64DFR0_PMUVER_IMP_DEF &&
> >>             !kvm_arm_support_pmu_v3() && !is_protected_kvm_enabled())
> >>                 static_branch_enable(&kvm_arm_pmu_available);
> >> }
> >> ----
> >>
> >> And I don't find any other code that enables kvm_arm_pmu_available.
> >
> > The code was recently changed (in v5.15 I think), I think Eric is looking
> > at an older version.
>
> Yes I was "googling" kvm_arm_pmu_available enablement and I missed the
> kvm_pmu_probe_pmuver() != ID_AA64DFR0_PMUVER_IMP_DEF check addition. So
> except the heterogenous case reported by Alexandru below, we should be
> fine. Sorry for the noise.

Understood. Thank you for the confirmation.

Regards,
Reiji


> >>
> >> Looking at the KVM's PMUV3 support code for guests in v5.16-rc3,
> >> if KVM allows userspace to configure KVM_ARM_VCPU_PMU_V3 even with
> >> ID_AA64DFR0_PMUVER_IMP_DEF on the host (, which I don't think it does),
> >> I think we should fix that to not allow that.
> >
> > I recently started looking into that too. If there's only one PMU, then the
> > guest won't see the value IMP DEF for PMUVer (userspace cannot set the PMU
> > feature because !kvm_arm_support_pmu_v3()).
> >
> > On heterogeneous systems with multiple PMUs, it gets complicated. I don't
> > have any such hardware, but what I think will happen is that KVM will
> > enable the static branch if there is at least one PMU with
> > PMUVer != IMP_DEF, even if there are other PMUs with PMUVer = IMP_DEF. But
> > read_sanitised_ftr_reg() will always return 0 for the
> > PMUVer field because the field is defined as FTR_EXACT with a safe value of
> > 0 in cpufeature.c. So the guest ends up seeing PMUVer = 0.
> >
> > I'm not sure if this is the case because I'm not familiar with the cpu
> > features code, but I planning to investigate further.
> >
> > Thanks,
> > Alex
> >
> >> (I'm not sure how KVM's PMUV3 support code is implemented in the
> >> code that you are looking at though)
> >>
> >> Thanks,
> >> Reiji
> >
>