linux interprets an fcntl int arg as long

[Mark Rutland]

On Tue, Nov 01, 2022 at 12:19:51PM +0000, David Laight wrote:
> From: 'Szabolcs Nagy' <szabolcs.nagy at arm.com>
> > Sent: 01 November 2022 11:45
> > 
> > The 11/01/2022 10:02, David Laight wrote:
> > > From: Szabolcs Nagy

> > kernel code:
> > ------------
> > SYSCALL_DEFINE3(fcntl, unsigned int, fd, unsigned int, cmd, unsigned long, arg)
> > {
> 
> That is just a wrapper and calls do_fcntl().
> which needs changing to be add:
> 	arg &= ~0U;
> before the switch(cmd) {

Just to check, do you mean the switch in do_fcntl(), or the switch within memfd_fcntl() ?

The former handles other APIs which do expect arg to be a long (e.g.
F_SET_RW_HINT and F_GET_RW_HINT expect it to hold a full 64-bit pointer), so
that'd break things.

The latter would work (as would casting arg to int when calling
memfd_add_seals()).

Mark.