cacheflush completely broken, suspecting PAN+LPAE

[Linus Walleij]

Hi Michal,

On Mon, Nov 11, 2024 at 11:38 PM Michał Pecio <michal.pecio at gmail.com> wrote:

> I installed v6.11.5 on Tegra K1 (Cortex-A15) with tegra_defconfing +
> CONFIG_ARM_LPAE + a few drivers + minor patches for driver issues.
>
> gdb segfaults on startup, strace shows this:
>
> openat(AT_FDCWD, "/usr/lib/guile/3.0/ccache/ice-9/psyntax-pp.go", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 11
> _llseek(11, 0, [477309], SEEK_END)      = 0
> mmap2(NULL, 477309, PROT_READ, MAP_PRIVATE, 11, 0) = 0xb378a000
> close(11)                               = 0
> mprotect(0xb37da000, 43512, PROT_READ|PROT_WRITE) = 0
> mmap2(NULL, 262144, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb374a000
> cacheflush(0xb374a000, 0xb374b000, 0)   = -1 EFAULT (Bad address)
> cacheflush(0xb374a000, 0xb374b000, 0)   = -1 EFAULT (Bad address)
> cacheflush(0xb374a000, 0xb374b000, 0)   = -1 EFAULT (Bad address)
> futex(0xb6f0bb14, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> cacheflush(0xb374a000, 0xb374b000, 0)   = -1 EFAULT (Bad address)
> cacheflush(0xb374a000, 0xb374b000, 0)   = -1 EFAULT (Bad address)
> --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x4} ---
>
> guile is apparently a scripting language with JIT compiler, disabling
> JIT resolves the crash, so cacheflush is a big suspect at this point.
>
> No apparent valid reason for those failures and no recent changes to
> cacheflush handling that I see, so I searched for commits touching LPAE
> (perhaps the most uncommon part of my system) and I quickly found the
> ARM_PAN feature, which claims to monkey with page tables. Hmm...
>
> Disabled ARM_PAN, cacheflush returns 0 now and gdb crashes no more.
>
> So I guess it looks like there is a problem with this feature, perhaps
> a missing "permit user accesss" somewhere?

We are trying to locate the issue, which I think is the same as this
but not sure:
https://bugzilla.kernel.org/show_bug.cgi?id=219247

I have been trying to replicate it on a Chromebook but didn't get so far
yet because the installation is pretty idiomatic :/ also there is only appears
in a single Qt program and not as predictable as here.

But. It appears the code is issuing cacheflush() which I guess ends
up in arm_syscall() here:

        case NR(cacheflush):
                return do_cache_op(regs->ARM_r0, regs->ARM_r1, regs->ARM_r2);

To here:

static inline int
do_cache_op(unsigned long start, unsigned long end, int flags)
{
        if (end < start || flags)
                return -EINVAL;

        if (!access_ok((void __user *)start, end - start))
                return -EFAULT;

        return __do_cache_op(start, end);
}

Here userspace access should be fine because we have entered a
syscall from userspace. I tried to emulate the situation with this
program:

#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>

#define NR_cacheflush 0xf0002

/* libgcc */
extern void __clear_cache(void *, void *);

int main (int argc, char **argv) {
  void *addr;
  int ret;

  printf("Test()\n");
  addr = mmap(NULL, 0x1000, PROT_READ|PROT_WRITE|PROT_EXEC,
              MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
  if (addr == MAP_FAILED) {
    printf("mmap() failed\n");
    exit(1);
  }

The libgcc version is what guile uses.
But it all just works fine for me.

I added prints in the cacheflush trap:

diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c
index 480e307501bb..400650519bd1 100644
--- a/arch/arm/kernel/traps.c
+++ b/arch/arm/kernel/traps.c
@@ -592,11 +592,14 @@ __do_cache_op(unsigned long start, unsigned long end)
 static inline int
 do_cache_op(unsigned long start, unsigned long end, int flags)
 {
+       pr_info("%s(%08lx-%08lx)\n", __func__, start, end);
        if (end < start || flags)
                return -EINVAL;

-       if (!access_ok((void __user *)start, end - start))
+       if (!access_ok((void __user *)start, end - start)) {
+               pr_err("ACCESS NOT OK\n");
                return -EFAULT;
+       }

        return __do_cache_op(start, end);
 }

And they all work fine with the test program.
This is on v6.12-rc1.

Does this test program work also on your set-up?

Yours,
Linus Walleij