linux interprets an fcntl int arg as long

['Szabolcs Nagy']

The 11/01/2022 10:02, David Laight wrote:
> From: Szabolcs Nagy
> > Sent: 01 November 2022 09:11
> > 
> > The 10/31/2022 21:46, Theodore Ts'o wrote:
> > > On Mon, Oct 31, 2022 at 12:44:59PM +0000, Szabolcs Nagy wrote:
> > > > and such fcntl call can happen with c code that just passes
> > > > F_SEAL_WRITE since it is an int and e.g. with aarch64 pcs rules
> > > > it is passed in a register where top bits can be non-zero
> > > > (unlikely in practice but valid).
> > >
> > > In Linux's aarch64 ABI, an int is a 4-byte value.  It is *not* an
> > > 8-byte value.  So passing in "F_SEAL_WRITE | 0xF00000000" as an int
> > > (as in your example) is simply not valid thing for the userspace
> > > program to do.
> > >
> > > Now, if there is a C program which has "int c = F_SEAL_WRITE", if the
> > > PCS allows the compiler to pass a function paramter c --- for example
> > > f(a, b, c) --- where the 4-byte paramter 'c' is placed in a 64-bit
> > > register where the high bits of the 64-bit register contains non-zero
> > > garbage values, I would argue that this is a bug in the PCS and/or the
> > > compiler.
> > 
> > the callee uses va_arg(ap, type) to get the argument,
> > and if the type is wider than what was actually passed
> > then anything can happen. in practice what happens is
> > that the top bits can be non-zero.
> > 
> > many pcs are affected (aarch64 is the one i know well,
> > but at least x86_64, arm are affected too). and even if
> > it was aarch64 pcs only, it is incompetent to say that
> > the pcs is wrong: that's a constraint we are working with.
> > 
> > the kernel must not read a wider type than what it
> > documents as argument to variadic functions in the c api.
> > (it does not make much sense to expect anything there
> > anyway, but it can break userspace)
> 
> The Linux kernel just assumes that the varargs call looks like
> a non-varags call with the same parameters.
> (It doesn't use va_arg())
> All syscall arguments are passed in registers (unlike BSDs
> where they can also be on the user stack).
> On 64bit systems the same registers are expected to be used
> for 64bit and 32bit integers and for pointers.
> 32bit values usually get masked because they get passed to
> a function with an 'int' argument.
> 
> If any fcntl() calls require a 64bit value and the C ABI
> might leave non-zero high bits in an register containing
> a 32bit value (esp. to a varargs function) then the calling
> code will need to cast such arguments to 64 bits.

the entire point of my mail is that it is not possible
to tell in the libc if the vararg is pointer or int.

so in case a user passed an int, the libc cannot fix
that up, like it usually does for other cases where
linux syscall abi is incompatible with the c api.

let me go through step by step what is going on:

user code:
----------
int fcntl(int, int, ...);
void foobar(int fd, int foo, long arg)
{
	/* top bits of arg are used by the user (non-zero),
	   they are not for the syscall. */

	/* bottom bits of arg are passed to fcntl:
	   the compiler passes arg unmodified in a register
	   where top bits are set, but that's fine: the callee
	   only supposed to use the bottom 32bits.  */
	fcntl(fd, F_ADD_SEALS, (int)arg);
}

libc code:
----------
long internal_syscall(int, long, long, long, long, long, long);
int fcntl(int fd, int cmd, ...)
{
	va_list ap;
	va_start(ap, cmd);
	/* this is non-conforming C: wrong type in va_arg,
	   but that's not relevant since libc can implement
	   this as target specific asm, the important bit is
	   that the correct type is not known: libc cannot
	   replicate the kernel side dispatch logic because
	   new cmd can be introduced in the future with
	   arbitrary type arg.

	   top 32bits of arg are non-zero, libc cannot
	   zero them here as arg may be long or pointer.  */
	long arg = va_arg(ap, long);
	va_end(ap);
	return internal_syscall(SYS_fcntl, fd, cmd, arg, 0, 0, 0);
}

kernel code:
------------
SYSCALL_DEFINE3(fcntl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
{
	...
	switch (cmd) {
	...
	/* this part is in memfd_fcntl.  */
	case F_ADD_SEALS:
		/* uses arg as long instead of the documented
		   int type and fails because top bits are set.  */
	}
}



> 
> OTOH I suspect the argument is either absent, int or pointer.
> So it should mask the value to 32 bits.

how does that work on 64bit targets?

and how does it work on 32bit targets if a new cmd
is introduced with short int argument (posix does
not guarantee that it won't)?

> 
> Note that there are ABI where 'int' and 'pointer' get passed
> in different registers.
> Fortunately none will support Linux!

what i explained above is not theoretical abi or libc:
this is how things work now in practice on x86_64 and
aarch64 and it is clearly broken.

normally when linux does something silly like this the
libc fixes it up (since we want to support old kernels),
but in this case that would mean replicating the entire
ioctl, prctl, fcntl, etc dispatch logic in userspace
which is impractical and cannot be future proof.

so going forward i'd like linux devs to consider the
3 points i raised in my original mail. and fix
memfd_fcntl accordingly.

thanks.


complete example (tested on aarch64 and x86_64):

$ cat a.c
#define _GNU_SOURCE 1
#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>

int g = 0;
__attribute__((noinline)) /* pretend to be in another TU. */
int foobar(int fd, int c, long arg)
{
        if (arg & 0x100000000) /* use top bits for something */
                g+=c;
        return fcntl(fd, F_ADD_SEALS, (int)arg);
}

int main (int argc, char *argv[])
{
  int fd = memfd_create ("tst", MFD_CLOEXEC|MFD_ALLOW_SEALING);
  if (fd < 0 ) return -1;
  int r = foobar(fd, argc, F_SEAL_WRITE | 0xF00000000);
  printf("%d %d %m\n", g, r);
  return 0;
}
$ gcc -O2 a.c
$ ./a.out
1 -1 Invalid argument
$ strace ./a.out 2>&1 |grep fcntl
fcntl(3, F_ADD_SEALS, F_SEAL_WRITE|0xf00000000) = -1 EINVAL (Invalid argument)