[PATCH v2] KVM: arm64: Remove size-order align in the nVHE hyp private VA range

[Marek Szyprowski]

On 11.08.2023 13:20, Vincent Donnefort wrote:
> commit f922c13e778d ("KVM: arm64: Introduce
> pkvm_alloc_private_va_range()") and commit 92abe0f81e13 ("KVM: arm64:
> Introduce hyp_alloc_private_va_range()") added an alignment for the
> start address of any allocation into the nVHE hypervisor private VA
> range.
>
> This alignment (order of the size of the allocation) intends to enable
> efficient stack verification (if the PAGE_SHIFT bit is zero, the stack
> pointer is on the guard page and a stack overflow occurred).
>
> But this is only necessary for stack allocation and can waste a lot of
> VA space. So instead make stack-specific functions, handling the guard
> page requirements, while other users (e.g.  fixmap) will only get page
> alignment.
>
> Signed-off-by: Vincent Donnefort <vdonnefort at google.com>

This change, merged to linux-next as f156a7d13fc3 ("KVM: arm64: Remove 
size-order align in the nVHE hyp private VA range"), introduced the 
following regression on Raspberry Pi4b board. Here is a log observed on 
linux next-20230828 release:

--->8---

kvm [1]: IPA Size Limit: 44 bits
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at arch/arm64/kvm/hyp/pgtable.c:453 
hyp_map_walker+0xb0/0x120
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.5.0-next-20230828 #13889
Hardware name: Raspberry Pi 4 Model B (DT)
pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : hyp_map_walker+0xb0/0x120
lr : hyp_map_walker+0x30/0x120
...
Call trace:
  hyp_map_walker+0xb0/0x120
  kvm_pgtable_visitor_cb.isra.0+0x38/0x5c
  __kvm_pgtable_walk+0x1dc/0x32c
  __kvm_pgtable_walk+0xd0/0x32c
  __kvm_pgtable_walk+0xd0/0x32c
  __kvm_pgtable_walk+0xd0/0x32c
  kvm_pgtable_walk+0xd0/0x224
  kvm_pgtable_hyp_map+0x8c/0xf0
  __create_hyp_mappings+0x98/0xc0
  __create_hyp_private_mapping+0xf0/0x14c
  create_hyp_io_mappings+0x8c/0x118
  vgic_v2_probe+0xbc/0x1ec
  kvm_vgic_hyp_init+0xcc/0x210
  kvm_arm_init+0x6a4/0x1040
  do_one_initcall+0x74/0x2f0
  kernel_init_freeable+0x28c/0x4dc
  kernel_init+0x24/0x1dc
  ret_from_fork+0x10/0x20
irq event stamp: 72194
hardirqs last  enabled at (72193): [<ffff8000811f2c48>] 
_raw_spin_unlock_irqrestore+0x74/0x78
hardirqs last disabled at (72194): [<ffff8000811df128>] el1_dbg+0x24/0x8c
softirqs last  enabled at (72182): [<ffff800080010a10>] 
__do_softirq+0x438/0x4ec
softirqs last disabled at (72173): [<ffff800080016980>] 
____do_softirq+0x10/0x1c
---[ end trace 0000000000000000 ]---
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at arch/arm64/kvm/hyp/pgtable.c:470 
hyp_map_walker+0x10c/0x120
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Tainted: G        W 6.5.0-next-20230828 #13889
Hardware name: Raspberry Pi 4 Model B (DT)
pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : hyp_map_walker+0x10c/0x120
lr : hyp_map_walker+0x30/0x120
...
Call trace:
  hyp_map_walker+0x10c/0x120
  kvm_pgtable_visitor_cb.isra.0+0x38/0x5c
  __kvm_pgtable_walk+0x1dc/0x32c
  __kvm_pgtable_walk+0xd0/0x32c
  __kvm_pgtable_walk+0xd0/0x32c
  __kvm_pgtable_walk+0xd0/0x32c
  kvm_pgtable_walk+0xd0/0x224
  kvm_pgtable_hyp_map+0x8c/0xf0
  __create_hyp_mappings+0x98/0xc0
  __create_hyp_private_mapping+0xf0/0x14c
  create_hyp_io_mappings+0x8c/0x118
  vgic_v2_probe+0xbc/0x1ec
  kvm_vgic_hyp_init+0xcc/0x210
  kvm_arm_init+0x6a4/0x1040
  do_one_initcall+0x74/0x2f0
  kernel_init_freeable+0x28c/0x4dc
  kernel_init+0x24/0x1dc
  ret_from_fork+0x10/0x20
irq event stamp: 72214
hardirqs last  enabled at (72213): [<ffff8000811df8f8>] 
exit_to_kernel_mode+0x34/0x124
hardirqs last disabled at (72214): [<ffff8000811df128>] el1_dbg+0x24/0x8c
softirqs last  enabled at (72212): [<ffff800080010a10>] 
__do_softirq+0x438/0x4ec
softirqs last disabled at (72197): [<ffff800080016980>] 
____do_softirq+0x10/0x1c
---[ end trace 0000000000000000 ]---
kvm [1]: Cannot map VCTRL into hyp

-------


With $subject patch reverted on top of next-20230828, kvm initializes 
properly on Raspberry Pi4b:

--->8---
kvm [1]: IPA Size Limit: 44 bits
kvm [1]: vgic interrupt IRQ9
kvm [1]: Hyp mode initialized successfully

-------

Let me know if I can help somehow debugging this issue.


Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland