[PATCH v5] arm64: mte: allow async MTE to be upgraded to sync on a per-CPU basis

[Will Deacon]

On Mon, Jun 21, 2021 at 04:18:59PM +0100, Catalin Marinas wrote:
> On Mon, Jun 21, 2021 at 01:39:37PM +0100, Will Deacon wrote:
> > On Fri, Jun 18, 2021 at 04:09:55PM +0100, Catalin Marinas wrote:
> > > On Thu, Jun 17, 2021 at 10:58:30PM +0100, Will Deacon wrote:
> > > > On Tue, Jun 15, 2021 at 01:38:07PM -0700, Peter Collingbourne wrote:
> > > > > +Upgrading to stricter tag checking modes
> > > > > +----------------------------------------
> > > > > +
> > > > > +On some CPUs the performance of MTE in stricter tag checking modes
> > > > > +is similar to that of less strict tag checking modes. This makes it
> > > > > +worthwhile to enable stricter checks on those CPUs when a less strict
> > > > > +checking mode is requested, in order to gain the error detection
> > > > > +benefits of the stricter checks without the performance downsides. To
> > > > > +opt into upgrading to a stricter checking mode on those CPUs, the user
> > > > > +can set the ``PR_MTE_DYNAMIC_TCF`` flag bit in the ``flags`` argument
> > > > > +to the ``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call.
> > > > > +
> > > > > +This feature is currently only supported for upgrading from
> > > > > +asynchronous mode. To configure a CPU to upgrade from asynchronous mode
> > > > > +to synchronous mode, a privileged user may write the value ``1`` to
> > > > > +``/sys/devices/system/cpu/cpu<N>/mte_upgrade_async``, and to disable
> > > > > +upgrading they may write the value ``0``. By default the feature is
> > > > > +disabled on all CPUs.
> > > > > +
> > > > >  Initial process state
> > > > >  ---------------------
> > > > >  
> > > > > @@ -128,6 +147,7 @@ On ``execve()``, the new process has the following configuration:
> > > > >  - ``PR_TAGGED_ADDR_ENABLE`` set to 0 (disabled)
> > > > >  - Tag checking mode set to ``PR_MTE_TCF_NONE``
> > > > >  - ``PR_MTE_TAG_MASK`` set to 0 (all tags excluded)
> > > > > +- ``PR_MTE_DYNAMIC_TCF`` set to 0 (disabled)
> > > > >  - ``PSTATE.TCO`` set to 0
> > > > >  - ``PROT_MTE`` not set on any of the initial memory maps
> > > > 
> > > > Something about this doesn't sit right with me, as we're mixing a per-task
> > > > interface with a per-cpu interface for selecting async/sync MTE and the
> > > > priorities are somewhat confusing.
> > > > 
> > > > I think a better interface would be for the sysfs entry for each CPU to
> > > > allow selection between:
> > > > 
> > > > 	task  : Honour the prctl() (current behaviour)
> > > > 	async : Force async for tasks using MTE
> > > > 	sync  : Force sync for tasks using MTE
> > > > 	none  : MTE disabled
> > > > 
> > > > i.e. the per-cpu setting is an override.
> > > 
> > > As Peter mentioned, forcing it is a potential ABI break, so such feature
> > > would need backporting to 5.10. There's also a minor use-case that came
> > > up in the early discussions - an app may want to use async mode only for
> > > reporting but forcing it to sync would break such application (since
> > > sync mode prevents the faulty access from taking place).
> > 
> > Why is it an ABI break? The default will be "task" which behaves exactly as
> > things do today. If the policy is explicitly changed by userspace, then you
> > get new behaviour. I don't really see why this is different to e.g. writing
> > /proc/sys/vm/mmap_min_addr and having some applications fail because they
> > rely on the default setting.
> 
> That's slightly different from mmap_min_addr, it depends on the user
> expectations. Most applications have no interest in a MAP_FIXED of
> address 0, so they wouldn't notice a non-zero setting.

Not sure; if I set mmap_min_addr to 1GB then applications start to break
(32-bit applications SEGV instantly). So I think it's pretty similar.

> The semantics of MTE TCF none/sync/async are different and an
> application may have different expectations. For example, it may not
> want any tag checking, just being able to read/write tags. Or it may
> want just some lightweight monitoring with a simple restart after an
> async signal (sync requires either emulating the access or setting the
> PSTATE.TCO bit).

I think this is an argument against doing this at all. Realistically,
if PR_MTE_DYNAMIC_TCF is the "go fast" mode, then people will set it
blindly and applications will just be expected to deal with a combination
of sync and async faults; I really don't think the flag changes anything in
that regards. It's not a buy-in from the application, given that the visible
behaviour is unknown and dynamic.

> Forcing the TCF via sysfs may be seen as a user problem but that's
> pretty much rendering the application choice of the tag check mode
> useless since an admin could override it.

I think the application choice _is_ useless if we decide to offer a
mechanism where it is set on a per-cpu basis instead.

> > > So I'd rather leave it up to the user task to decide whether its choice
> > > can be changed. Peter introduced a new PR_MTE_DYNAMIC_TCF for this
> > > purpose (or a different name if you have a better suggestion).
> > 
> > I don't see how PR_MTE_DYNAMIC_TCF is useful to userspace, really. It's an
> > extra bit of logic to go and set it, but then what? It doesn't know which
> > behaviour it's getting, so it just feels like an extra hoop to jump through
> > without actually adding anything useful.
> 
> Well, without this additional bit, an application can't rely on the mode
> it requested. With an additional forced tagged address enable, we might
> as well remove the prctl() altogether (well, that wouldn't be a bad
> thing).

I think the prctl() is still useful to say whether an application wants MTE
or not, but I'm inclined to agree that sync vs async shouldn't be part of
the prctl() call.

> Given that there are no real users of MTE yet, we have some choice of
> tweaking the ABI, backporting to 5.10. The question: is the expectation
> that the sysfs forcing of TCF is limited to deployments where the user
> space is tightly controlled (e.g. Android with most apps starting from
> zygote) or we allow it to become more of a general hint of what's the
> fastest check on a CPU? If the former, I'm fine with forcing without any
> additional bit, though I'd still prefer the opt-in. For the latter, I'd
> like some wider discussion with non-Android folk on what they expect
> from the TCF setting. Otherwise simply using PROT_MTE would may lead to
> tag check faults.

I don't think there's anything Android-specific here. The problem being
solved concerns big/little SoCs with MTE, and I think it's up to the
distribution how the sysfs stuff is used.

Will