[musl] Re: [PATCH v8 00/38] arm64/gcs: Provide support for GCS in userspace

[Edgecombe, Rick P]

On Wed, 2024-02-21 at 12:57 -0500, dalias at libc.org wrote:
> > This feels like it's getting complicated and I fear it may be an
> > uphill
> > struggle to get such code merged, at least for arm64.  My instinct
> > is
> > that it's going to be much more robust and generally tractable to
> > let
> > things run to some suitable synchronisation point and then disable
> > there, but if we're going to do that then userspace can hopefully
> > arrange to do the disabling itself through the standard disable
> > interface anyway.  Presumably it'll want to notice things being
> > disabled
> > at some point anyway?  TBH that's been how all the prior proposals
> > for
> > process wide disable I've seen were done.
> 
> If it's possible to disable per-thread rather than per-process, some
> things are easier. Disabling on account of using alt stacks only
> needs
> to be done on the threads using those stacks. However, for dlopen
> purposes you need a way to disable shadow stack for the whole
> process.
> Initially this is only needed for the thread that called dlopen, but
> it needs to have propagated to any thread that synchronizes with
> completion of the call to dlopen by the time that synchronization
> occurs, and since that synchronization can happen in lots of
> different
> ways that are purely userspace (thanks to futexes being userspace in
> the uncontended case), I don't see any way to make it work without
> extremely invasive, high-cost checks.

For glibc's use, we talked about a couple of options.
1. A mode to start suppressing the #UD's from the shadow stack
instructions
2. A mode to start suppressing #CPs (the exception that happens when
the shadow stack doesn't match). So the shadow stack instructions
continue to operate normally, but if the shadow stack gets mismatched
due to lack of support, the ret is emulated. It probably is safer (but
still not perfect), but the performance penalty of emulating every RET
after things get screwed up would be a significant down side. There
also needs to be clean handling of shadow stack #PFs.
3. Per-thread locking that is used around all shadow stack operations
that could be sensitive to disabling. This could be maybe exposed to
apps in case they want to use shadow stack instructions manually. Then
during dlopen() it waits until it can cleanly disable shadow stack for
each thread. In each critical sections there are checks for whether
shadow stack is still enabled.

3 is the cleanest and safest I think, and it was thought it might not
need kernel help, due to a scheme Florian had to direct signals to
specific threads. It's my preference at this point.

1 and 2 are POCed here, if you are interested:
https://github.com/rpedgeco/linux/commits/shstk_suppress_rfc/

> 
> If folks on the kernel side are not going to be amenable to doing the
> things that are easy for the kernel to make it work without breaking
> compatibility with existing interfaces, but that are impossible or
> near-impossible for userspace to do, this seems like a dead-end. And
> I
> suspect an operation to "disable shadow stack, but without making
> threads still in SS-critical sections crash" is going to be
> necessary..

I think we have to work through all the alternative before we can
accuse the kernel of not being amenable. Is there something that you
would like to see out of this conversation that is not happening?