kCFI && patchable-function-entry=M,N

[Peter Zijlstra]

On Fri, Oct 21, 2022 at 04:56:20PM +0100, Mark Rutland wrote:
> Hi,
> 
> For arm64, I'd like to use -fatchable-function-entry=M,N (where N > 0), for our
> ftrace implementation, which instruments *some* but not all functions.
> Unfortuntately, this doesn't play nicely with -fsanitize=kcfi, as instrumented
> and non-instrumented functions don't agree on where the type hash should live
> relative to the function entry point, making them incompatible with one another.
> AFAICT, there's no mechanism today to get them to agree.
> 
> Today we use -fatchable-function-entry=2, which happens to avoid this.

> ... but I understand that for x86, folk want the pre-function NOPs to
> fall-through into the body of the function.

Yep.

> Is there any mechanism today that we could use to solve this, or could we
> extend clang to have some options to control this behaviour?

So the main pain-point for you is differentiating between function with
notrace and those without it, right?

That is; suppose you (like x86) globally do:
-fpatchable-function-entry=4,2 to get a consistent function signature,
you're up a creek because you use the __patchable_function_entries
section to drive ftrace and now every function will have it.

So perhaps something like:

 -fpatchable-function-entry=N,M,sectionname

would help, then you can have notrace be the same layout, except a
different section. Eg. something like:

 #define notrace __attribute__((patchable_function_entry(4,2,__notrace_function_entries)))

It does make the whole: CFLAGS_REMOVE_file.o = $(CC_FLAGS_FTRACE)
a bit of a pain, but I've long favoured removing all that and having
explitic notrace attributes on all relevant functions.

Then again; perhaps it could be made to work by ensuring CFLAGS starts
with:

 -fpatchable-function-entry=4,2,__notrace_function_entries

and have CC_FLAGS_FTRACE include (and hence override with)

 -fpatchable-function-entry=4,2,__ftrace_function_entries

assuming that with duplicate argument the last is effective.