[PATCH] ARC: Improve cmpxchng syscall implementation

[Alexey Brodkin]

Hi Vineet,

On Mon, 2018-03-19 at 11:29 -0700, Vineet Gupta wrote:
> On 03/19/2018 04:00 AM, Alexey Brodkin wrote:
> > arc_usr_cmpxchg syscall is supposed to be used on platforms
> > that lack support of Load-Locked/Store-Conditional instructions
> > in hardware. And in that case we mimic missing hardware features
> > with help of kernel's sycall that "atomically" checks current
> > value in memory and then if it matches caller expectation new
> > value is written to that same location.
> > 
> 
> ...
> ...
> 
> > 
> > 2. What's worse if we're dealing with data from not yet allocated
> >     page (think of pre-copy-on-write state) we'll successfully
> >     read data but on write we'll silently return to user-space
> >     with correct result 
> 
> This is technically incorrect, even for reading, you need a page, which could be 
> common zero page in certain cases.

Ok I'll reword it like.

> 
> (which we really read just before). That leads
> >     to very strange problems in user-space app further down the line
> >     because new value was never written to the destination.
> > 
> > 3. Regardless of what went wrong we'll return from syscall
> >     and user-space application will continue to execute.
> >     Even if user's pointer was completely bogus.
> 
> Again we are exaggerating (from technical correctness POV) - if user pointer was 
> bogs, the read would not have worked in first place etc. So lets tone down the 
> rhetoric.

Ok here I may rephrase it like that:
------------------------------->8-----------------------------
3. Regardless of what went wrong we'll return from syscall
   and user-space application will continue to execute.
------------------------------->8-----------------------------

> 
> >     In case of hardware LL/SC that app would have been killed
> >     by the kernel.
> > 
> > With that change we attempt to imrove on all 3 items above:
> > 
> > 1. We still disable preemption around read-and-write of
> >     user's data but if we happen to fail with either of them
> >     we're enabling preemption and try to force page fault so
> >     that we have a correct mapping in the TLB. Then re-try
> >     again in "atomic" context.
> > 
> > 2. If real page fault fails or even access_ok() returns false
> >     we send SIGSEGV to the user-space process so if something goes
> >     seriously wrong we'll know about it much earlier.
> > 
> 
> 
> >   
> >   	/*
> >   	 * This is only for old cores lacking LLOCK/SCOND, which by defintion
> > @@ -60,23 +62,48 @@ SYSCALL_DEFINE3(arc_usr_cmpxchg, int *, uaddr, int, expected, int, new)
> >   	/* Z indicates to userspace if operation succeded */
> >   	regs->status32 &= ~STATUS_Z_MASK;
> >   
> > -	if (!access_ok(VERIFY_WRITE, uaddr, sizeof(int)))
> > -		return -EFAULT;
> > +	ret = access_ok(VERIFY_WRITE, uaddr, sizeof(*uaddr));
> > +	if (!ret)
> > +		goto fail;
> >   
> > +again:
> >   	preempt_disable();
> >   
> > -	if (__get_user(uval, uaddr))
> > -		goto done;
> > -
> > -	if (uval == expected) {
> > -		if (!__put_user(new, uaddr))
> > +	ret = __get_user(val, uaddr);
> > +	if (ret == -EFAULT) {
> 
> 
> Lets see if this warrants adding complexity ! This implies that TLB entry with 
> Read permissions didn't exist for reading the var and page fault handler could not 
> wire up even a zero page due to preempt_disable, meaning it was something not 
> touched by userspace already - sort of uninitialized variable in user code.

Ok I completely missed the fact that fast path TLB miss handler is being
executed even if we have preemption disabled. So given the mapping exist
we do not need to retry with enabled preemption.

Still maybe I'm a bit paranoid here but IMHO it's good to be ready for a corner-case
when the pointer is completely bogus and there's no mapping for him.
I understand that today we only expect this syscall to be used from libc's
internals but as long as syscall exists nobody stops anybody from using it
directly without libc. So maybe instead of doing get_user_pages_fast() just
send a SIGSEGV to the process? At least user will realize there's some problem
at earlier stage.

> Otherwise it is extremely unlikely to start with a TLB entry with Read 
> permissions, followed by syscall Trap only to find the entry missing, unless a 
> global TLB flush came from other cores, right in the middle. But this syscall is 
> not guaranteed to work with SMP anyways, so lets ignore any SMP misdoings here.

Well but that's exactly the situation I was debugging: we start from data from read-only
page and on attempt to write back modified value COW machinery gets involved.

That was on UP platform.

> Now in case it was *an* uninitialized var, do we have to guarantee any well 
> defined semantics for the kernel emulation of cmpxchg ? IMO it should be fine to 
> return 0 or -EFAULT etc. Infact -EFAULT is better as it will force a retry loop on 
> user side, given the typical cmpxchg usage pattern.

The problem is libc only expects to get a value read from memory.
And in theory expected value might be -14 which is basically -EFAULT.
I'm not talking about 0 at all because in some cases that's exactly what
user-space expects.

So if we read unexpected value then we'll just return it without even attempting
to write.

If we read expected data but fail to write then we'll send a SIGSEGV and
return whatever... let it be -EFAULT - anyways the app will be killed on exit from
this syscall.

-Alexey