[RFC PATCH v6 00/35] KVM: arm64: Add Statistical Profiling Extension (SPE) support

[Leo Yan]

Hi Alexandru,

Just couples general questions for myself to easier understand the series
(sorry if I asked duplicated questions).

> I wanted the focus to be on pinning memory at stage 2 (that's patches #29, 'KVM:
> arm64: Pin the SPE buffer in the host and map it at stage 2', to #3, 'KVM:
> arm64: Add hugetlb support for SPE') and I would very much like to start a
> discussion around that.

I am confused for "pinning memory at stage 2" and then I read "Pin the
SPE buffer in the host".  I read Chapter 2 Specification, ARM DEN 0154,
my conclusion is:

1) You set PMBLIMITR_EL1.nVM == 0 (virtual address mode) so that the
   driver uses the same mode whether it is running in a host or in a
   guest.

2) The KVM hypervisor needs to parse the VA -> IPA -> PA with:

   Guest stage-1 table (managed in guest OS);
   Guest stage-2 table (managed in KVM hypervisor);

3) In the end, the KVM hypervisor pins physical pages on the host
   stage-1 page table for:

   The physical pages are pinned for Guest stage-1 table;
   The physical pages are pinned for Guest stage-2 table;
   The physical pages are pinned for used for TRBE buffer in guest.

Due the host might migrate or swap pages, so all the pin operations
happen on the host's page table.  The pin operations never to be set up
in guest's stage-2 table, right?

> The problem
> ===========
> 
> When the Statistical Profiling Unit (SPU from now on) encounter a fault when
> it attempts to write a record to memory, two things happen: profiling is
> stopped, and the fault is reported to the CPU via an interrupt, not an
> exception. This creates a blackout window during which the CPU executes
> instructions which aren't profiled. The SPE driver avoid this by keeping the
> buffer mapped while ProfilingBufferEnabled() = true. But when running as a
> guest under KVM, the SPU will trigger stage 2 faults, with the associated
> blackout windows.

My understanding is that there are two prominent challenges for SPE
virtualization:

1) Allocation: we need to allocate trace buffer with mapping both
   guest's stage-1 and stage-2 before enabling SPU.  (For me, the free
   buffer is never an issue as we always disable the SPU before
   releasing the resource).

2) Pin: the physical pages used by trace buffer and the relevant stage-1
   and stage-2 tables must be pinned during the session.

I will read (and learn) the patches in next days.

Thanks,
Leo