[PATCH] arm64: Implement prctl(PR_{G,S}ET_TSC)

[Peter Collingbourne]

On Mon, Apr 29, 2024 at 1:36 AM Keno Fischer <keno at juliahub.com> wrote:
>
> First a note that this has been previously discussed on this list for
> the same motivation and with the same questions asked, so let
> me link that first:
>
> Link: https://lore.kernel.org/linux-arm-kernel/CAP045ApiMSvP--f2E0=VdMbjE8oibvy921m8JASf4kaCCuU2RA@mail.gmail.com/T/

Ah, I wasn't aware that this was previously discussed. I agree with
your points from that thread regarding prctl vs ptrace.

> > It seems to me that this sort of "trap and inspect" behaviour is in
> > the realm of ptrace(), and not that of prctl(), because I can't
> > imagine the debugged program calling that by itself.
>
> In the rr use case, this is indeed used to force a ptrace signal stop
> for trap/inspect (or emulate in replay mode). Of course the history
> of PR_SET_TSC is complicated and was not originally intended for
> this use case. Rather, it was intended as a hardening mechanism
> against cache-timing/speculative execution attacks (this was
> pre-Spectre, so all the discussion at the time is a bit theoretical).
> Of course, the Spectre experience has shown that you don't really
> actually need architectural timers for any of this, so perhaps it wasn't
> all that useful in the first place.

That was what I originally thought as well, but then I went back to
look at the history:
https://lore.kernel.org/all/Pine.GSO.4.56.0804131126080.28138@keg.few.vu.nl/

It looks like the original author was also using this for
deterministic replay. So rr is using this prctl for its original
purpose.

It's also worth noting that the name of the prctl was chosen with the
intent that it could be implemented on other architectures.
https://lore.kernel.org/all/4813B1EF.5090806@zytor.com/

> I think this could reasonably be made a PTRACE_EVENT, but that would
> not be useable for hardening if somebody wanted to do that in the future
> (I am not personally aware of any such request, but it doesn't seem
> unreasonable). Of course, my concerns about adding yet another kind
> of ptrace stop mentioned last time still exist.

Yes, in general an in-process control seems more useful than an
out-of-process control, since anything that you would be able to do
with ptrace could also be done with prctl (tracer can inject a call to
the prctl and handle signal-delivery-stops), and it avoids needing an
additional process (which will complicate debugging of the ptraced
process since it cannot have more than one tracer, and will be
incompatible with ptrace_scope=3) in cases where that is not otherwise
necessary.

Peter

> Perhaps a third option would be to put this into seccomp instead. One
> could imagine a flag addition to SECCOMP_SET_MODE_FILTER that
> would enable filtering for all otherwise emulated instructions (not just
> a CNTVCT_EL0 read) and an appropriate bpf program to decide
> allow/disallow/trap/signal/whatever. Of course, there are folks already
> unhappy with using seccomp for non-security purposes (as rr already
> does), so there may be some objection there.
>
> Similar concerns about making ptrace stops more complicated apply
> (basically the issue is that what you can do to a tracee at a
> ptrace stop very much depends on where exactly in the kernel the
> tracee is stopped, which ptrace doesn't always tell you,
> so to be fully correct, ptracers need to do some very elaborate
> state machine tracking).
>
> > How does rr work on non-x86 architectures? How does this interoperate
> > with AArch32?
>
> rr has significant microarchitecture dependence. It works on most modern x86
> chips (AMD worse than Intel with the exception of some Intel Atom uarchs)
> and more recent high-end AArch64 chips as long as the programs only use
> lse atomics and not llsc. AArch32 is not supported. See here for a list:
>
> https://github.com/rr-debugger/rr/blob/822863c92d8f52778700f0375ac1b705193a2152/src/PerfCounters.cc#L198-L237
>
> Thanks,
> Keno