[RFC PATCH v2 0/2] lib: sbi_pmu: Add PMU snapshot extension

[Inochi Amaoto]

>On Wed, Jan 10, 2024 at 4:34 PM Inochi Amaoto <inochiama at outlook.com> wrote:
>>
>> Although PMU snapshot extension is not very useful for normal use,
>> it can provide a recoverable snapshot for the PMU system. It is useful
>> for KVM VMs, especially for VM snapshot.
>>
>
>Can you please further clarify how adding the PMU snapshot support in
>OpenSBI helps the KVM VM snapshot?
>During a VM snapshot, the host would have saved the values in the vcpu
>structure already.
>The PMU snapshot save/restore happens only in SBI calls. Thus, the
>host has to invoke the SBI call (in context switch path)
>to get an updated copy in the shared memory. PMU snapshot helps in
>Virtualization as the host virtualizes the counters which
>results in traps for each CSR read access within the guest. However,
>the host can directly read the CSR as well.
>

Yes, I have already notice the vcpu have saved these values. IIRC, only
`SBI_PMU_START_FLAG_INIT_FROM_SNAPSHOT` can restore the interrupt state.
This is not useful for the normal run, but may be required when needing
a persistent snapshot, (migration and VM snapshot). As a result, the host
should only call this when saving/restoring the VM to persistent storage,
and should not call this in the normal run for performance issue.

>Adding PMU snapshot in OpenSBI may affect negatively as well depending
>on the state of the L1 cache and the access latency.
>CSR access should be quite fast in most of the implementations. If it
>results in a miss, the latency would be even higher as you have to
>fetch the data from L2 or DRAM.
>

I have already notice this latency problem. As I mention before, this
should be called by the hypervisor when vcpu init/exit. So this problem
may be acceptable. In fact, I found that we can directly implement the
vcpu PMU sampling by pass a page for guset to host SBI. But I struggle on
improving it because fetching snapshot is too costly as you said.