[BUG] ARM64 KVM: Data abort executing post-indexed LDR on MMIO address

[Marc Zyngier]

On Fri, 04 Oct 2024 20:50:18 +0100,
Ahmad Fatoum <a.fatoum at pengutronix.de> wrote:
> 
> Hi,
> 
> On 04.10.24 14:10, Peter Maydell wrote:
> > On Fri, 4 Oct 2024 at 12:51, Ahmad Fatoum <a.fatoum at pengutronix.de> wrote:
> >> On 04.10.24 12:40, Peter Maydell wrote:
> >>> Don't do this -- KVM doesn't support it. For access to MMIO,
> >>> stick to instructions which will set the ISV bit in ESR_EL1.
> >>>
> >>> That is:
> >>>
> >>>  * AArch64 loads and stores of a single general-purpose register
> >>>    (including the register specified with 0b11111, including those
> >>>    with Acquire/Release semantics, but excluding Load Exclusive
> >>>    or Store Exclusive and excluding those with writeback).
> >>>  * AArch32 instructions where the instruction:
> >>>     - Is an LDR, LDA, LDRT, LDRSH, LDRSHT, LDRH, LDAH, LDRHT,
> >>>       LDRSB, LDRSBT, LDRB, LDAB, LDRBT, STR, STL, STRT, STRH,
> >>>       STLH, STRHT, STRB, STLB, or STRBT instruction.
> >>>     - Is not performing register writeback.
> >>>     - Is not using R15 as a source or destination register.
> >>>
> >>> Your instruction is doing writeback. Do the address update
> >>> as a separate instruction.
> 
> With readl/writel implemented in assembly, I get beyond that point, but
> now I get a data abort running an DC IVAC instruction on address 0x1000,
> where the cfi-flash is located. This instruction is part of a routine
> to remap the cfi-flash to start a page later, so the zero page can be
> mapped faulting.
> 
> Simple reproducer:
> 
> start:
>         ldr     x0, =0x1000
>         ldr     x1, =0x1040
>         bl      v8_inv_dcache_range
> 
>         mov     w10, '!'
>         bl      putch
> 
>         ret
> 
> v8_inv_dcache_range:
>         mrs     x3, ctr_el0
>         lsr     x3, x3, #16
>         and     x3, x3, #0xf
>         mov     x2, #0x4
>         lsl     x2, x2, x3
>         sub     x3, x2, #0x1
>         bic     x0, x0, x3
> 1:
>         dc      ivac, x0
>         add     x0, x0, x2
>         cmp     x0, x1
>         b.cc    1b
>         dsb     sy
>         ret
> 
> This prints ! without KVM, but triggers a data abort before that with -enable-kvm:
> 
>   DABT (current EL) exception (ESR 0x96000010) at 0x0000000000001000
>   elr: 000000007fbe0550 lr : 000000007fbe01ac
>   [snip]
> 
>   Call trace:
>   [<7fbe0550>] (v8_inv_dcache_range+0x1c/0x34) from [<7fbe0218>] (arch_remap_range+0x64/0x70)
>   [<7fbe0218>] (arch_remap_range+0x64/0x70) from [<7fb8795c>] (of_platform_device_create+0x1e8/0x22c)
>   [<7fb8795c>] (of_platform_device_create+0x1e8/0x22c) from [<7fb87a04>] (of_platform_bus_create+0x64/0xbc)
>   [snip]
> 
> Any idea what this is about?

IIRC, the QEMU flash is implemented as a read-only memslot. A data
cache invalidation is a *write*, as it can be (and is) upgraded to a
clean+invalidate by the HW.

KVM cannot satisfy the write, for obvious reasons, and tells the guest
to bugger off (__gfn_to_pfn_memslot() returns KVM_PFN_ERR_RO_FAULT,
which satisfies is_error_noslot_pfn() -- a slight oddity, but hey, why
not).

In the end, you get an exception. We could relax this by
special-casing CMOs to RO memslots, but this doesn't look great.

The real question is: what are you trying to achieve with this?

	M.

-- 
Without deviation from the norm, progress is not possible.