[PATCH v3 00/18] KVM: arm64: Rework timer offsetting for fun and profit

[Veith, Simon]

Hello Marc,

thanks again for your proposal.

On Fri, 2023-03-24 at 14:46 +0000, Marc Zyngier wrote:
> This series aims at satisfying multiple goals:
> 
> - allow a VMM to atomically restore a timer offset for a whole VM
>   instead of updating the offset each time a vcpu get its counter
>   written
> 
> - allow a VMM to save/restore the physical timer context, something
>   that we cannot do at the moment due to the lack of offsetting
> 
> - provide a framework that is suitable for NV support, where we get
>   both global and per timer, per vcpu offsetting, and manage
>   interrupts in a less braindead way.
> 
> We fix a couple of issues along the way, both from a stylistic and
> correctness perspective. This results in a new per VM KVM API that
> allows a global offset to be set at any point in time, overriding
> both
> of the timer counter writebacks.
> 
> We also take this opportunity to rework the way IRQs are associated
> with timers, something that was always a bit dodgy. This relies on a
> new lock, which should disappear once Oliver's lock ordering series
> is
> merged (we can reuse the config_lock for this).
> 
> This has been tested with nVHE, VHE and NV. I do not have access to
> CNTPOFF-aware HW, but Colton managed to give it a go. Note that the
> NV patches in this series are here to give a perspective on how this
> gets used.
> 
> I've updated the arch_timer selftest to allow an offset to be
> provided
> from the command line, and fixed a couple of glaring issues along the
> way.
> 
> Note that this is at best 6.4 material. I have a branch stashed at
> [0]
> and based on 6.3-rc3, as well as a minimal example of the use of the
> API at [3] based on kvmtool.
> 
> Simon: I'd appreciate some feedback as whether this change fits your
> requirements, given that you brought this up the first place.

The interface looks good to me. I have yet to deliver on my promise to
test your patch series with our userspace; I am on leave this week, but
I'll give your latest iteration a go next week.

Thanks
Simon

> Thanks,
> 
>         M.
> 
> * From v2 [2]:
> 
>   - Fixed 32bit handling of the physical counter when the offset is
>     non-zero
> 
>   - Dropped unused -O option from the selftest
> 
>   - Added lockdep_assert_held() to (un)lock_all_vcpus()
> 
>   - Reordered the last two patches
> 
>   - Added Colton's RBs, with thanks
> 
>   - Dropped the initial patch which has been merged
> 
>   - Rebased on 6.3-rc3
> 
> * From v1 [1]:
> 
>   - Switched from a dual offset to a single one which gets applied to
>     both virtual and physical counters. Which means that NV doesn't
>     behave oddly anymore by ignoring the virtual offset.
> 
>   - Some cosmetic repainting of the UAPI symbols
> 
>   - Added patches to rework the IRQ mapping to timers
> 
>   - Patch #1 on its way to Paolo
> 
>   - Rebased on 6.3-rc1
> 
> [0]
> https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/timer-vm-offsets
> [1] https://lore.kernel.org/r/20230216142123.2638675-1-maz@kernel.org
> [2] https://lore.kernel.org/r/20230313124837.2264882-5-maz@kernel.org
> [3]
> https://git.kernel.org/pub/scm/linux/kernel/git/maz/kvmtool.git/commit/?h=zero-offset&id=3b1253073ee57c0d92baf7b214362829b487b8d5
> 
> Marc Zyngier (18):
>   KVM: arm64: timers: Use a per-vcpu, per-timer accumulator for
>     fractional ns
>   arm64: Add CNTPOFF_EL2 register definition
>   arm64: Add HAS_ECV_CNTPOFF capability
>   KVM: arm64: timers: Use CNTPOFF_EL2 to offset the physical timer
>   KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2
>   KVM: arm64: Expose {un,}lock_all_vcpus() to the rest of KVM
>   KVM: arm64: timers: Allow userspace to set the global counter
> offset
>   KVM: arm64: timers: Allow save/restoring of the physical timer
>   KVM: arm64: timers: Rationalise per-vcpu timer init
>   KVM: arm64: timers: Abstract per-timer IRQ access
>   KVM: arm64: timers: Move the timer IRQs into arch_timer_vm_data
>   KVM: arm64: Abstract the number of valid timers per vcpu
>   KVM: arm64: Document KVM_ARM_SET_CNT_OFFSETS and co
>   KVM: arm64: nv: timers: Add a per-timer, per-vcpu offset
>   KVM: arm64: nv: timers: Support hyp timer emulation
>   KVM: arm64: selftests: Add physical timer registers to the sysreg
> list
>   KVM: arm64: selftests: Deal with spurious timer interrupts
>   KVM: arm64: selftests: Augment existing timer test to handle
> variable
>     offset
> 
>  Documentation/virt/kvm/api.rst                |  38 ++
>  arch/arm64/include/asm/kvm_host.h             |  13 +
>  arch/arm64/include/asm/sysreg.h               |   2 +
>  arch/arm64/include/uapi/asm/kvm.h             |  11 +
>  arch/arm64/kernel/cpufeature.c                |  11 +
>  arch/arm64/kvm/arch_timer.c                   | 539 ++++++++++++++--
> --
>  arch/arm64/kvm/arm.c                          |  53 ++
>  arch/arm64/kvm/guest.c                        |  29 +-
>  arch/arm64/kvm/hyp/nvhe/timer-sr.c            |  18 +-
>  arch/arm64/kvm/hypercalls.c                   |   2 +-
>  arch/arm64/kvm/sys_regs.c                     |   9 +
>  arch/arm64/kvm/trace_arm.h                    |   6 +-
>  arch/arm64/kvm/vgic/vgic-kvm-device.c         |  38 --
>  arch/arm64/kvm/vgic/vgic.c                    |  15 +
>  arch/arm64/kvm/vgic/vgic.h                    |   3 -
>  arch/arm64/tools/cpucaps                      |   1 +
>  arch/arm64/tools/sysreg                       |   4 +
>  include/clocksource/arm_arch_timer.h          |   1 +
>  include/kvm/arm_arch_timer.h                  |  36 +-
>  include/kvm/arm_vgic.h                        |   1 +
>  include/uapi/linux/kvm.h                      |   3 +
>  .../selftests/kvm/aarch64/arch_timer.c        |  56 +-
>  .../selftests/kvm/aarch64/get-reg-list.c      |   5 +-
>  23 files changed, 690 insertions(+), 204 deletions(-)
> 
> --
> 2.34.1
> 




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879