[kvmtool PATCH 2/2] aarch64: Add support for MTE

[Catalin Marinas]

On Wed, Mar 23, 2022 at 12:03:33PM +0000, Alexandru Elisei wrote:
> On Wed, Mar 23, 2022 at 10:31:15AM +0000, Vladimir Murzin wrote:
> > On 3/21/22 5:08 PM, Alexandru Elisei wrote:
> > > On Mon, Mar 21, 2022 at 03:40:18PM +0000, Vladimir Murzin wrote:
> > > > Can we enable it unconditionally if KVM_CAP_ARM_MTE is supported like we do for
> > > > PAC and SVE?
> > > 
> > > I thought about that, the reason I chose to enable it based a kvmtool
> > > command line option, instead of always being enabled if available, is
> > > because of the overhead of sanitising the MTE tags on each stage 2 data
> > > abort. Steven, am I overreacting and that overhead is negligible?
> > > 
> > > Also, as far as I know, PAC and SVE incur basically no overhead in KVM
> > > until the guest starts to use those features.
> > > 
> > > Do you have a specific reason for wanting MTE to always be enabled if
> > > available? I'm happy to be convinced to make MTE enabled by default, I
> > > don't have preference either way.
> > 
> > Well, automatically enabling if available would align with what we do for
> > other features in kvmtool and Linux itself - we tend to default y for new
> > features, even MTE, thus improving chances to get reports back early if
> > something (even performance) goes wrong. Just my 2p.
> 
> According to Steven, for each 4k page the kernel uses an 128 byte buffer to
> store the tags, and then some extra memory is used to keep track of the
> buffers. Let's take the case of a VM with 1GB of memory, and be
> conservative and only account for the tag buffer. In this case, the tag
> buffers alone will be 32MB.

The 128 byte buffer is only allocated *if* the page is swapped out. So
if the host swaps out the entire 1GB guest memory, it will incur the
32MB cost. But a fully swapped out VM is pretty useless, so this doesn't
happen very often.

We do have the cost of zeroing the tags when the page is mapped into
guests. I don't remember the KVM implementation but maybe we can
optimise this to zero the tags at the same time with zeroing the data
as we do for anonymous pages in the host (unless it does this already).

We could use the fine-grained traps to detect when MAIR_EL1 was
configured by the guest for Tagged memory but, at least with Linux, we
do this at boot irrespective of whether any application will use MTE or
not. Future architecture versions may address this problem.

> The kernel documentation for MTE suggests that in order to take advantage
> of it, software must be modified and recompiled. That means that users that
> don't want to use MTE won't get to exercise MTE because they won't be using
> MTE enabled software, but they will pay the overhead regardless. This of
> course assumes that going forward software won't be using MTE by default.

Not all software needs to be recompiled. It's sufficient most of the
time to replace the glibc with an MTE-aware one.

I have a slight preference for MTE default in kvmtool, maybe with an
option to disable it if one doesn't care about the feature or we find
the overhead to be significant (I don't think it would be).

-- 
Catalin