[PATCH v3 05/18] arm64: Do not enable uaccess for flush_icache_range

[Mark Rutland]

On Thu, May 20, 2021 at 03:02:16PM +0100, Mark Rutland wrote:
> On Thu, May 20, 2021 at 01:43:53PM +0100, Fuad Tabba wrote:
> > __flush_icache_range works on the kernel linear map, and doesn't
> > need uaccess. The existing code is a side-effect of its current
> > implementation with __flush_cache_user_range fallthrough.
> > 
> > Instead of fallthrough to share the code, use a common macro for
> > the two where the caller specifies an optional fixup label if
> > user access is needed. If provided, this label would be used to
> > generate an extable entry.
> > 
> > No functional change intended.
> > Possible performance impact due to the reduced number of
> > instructions.
> > 
> > Reported-by: Catalin Marinas <catalin.marinas at arm.com>
> > Reported-by: Will Deacon <will at kernel.org>
> > Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/
> > Signed-off-by: Fuad Tabba <tabba at google.com>
> 
> I have one comment below, but either way this looks good to me, so:
> 
> Acked-by: Mark Rutland <mark.rutland at arm.com>
> 
> > ---
> >  arch/arm64/mm/cache.S | 64 +++++++++++++++++++++++++++----------------
> >  1 file changed, 41 insertions(+), 23 deletions(-)
> > 
> > diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S
> > index 5ff8dfa86975..c6bc3b8138e1 100644
> > --- a/arch/arm64/mm/cache.S
> > +++ b/arch/arm64/mm/cache.S
> > @@ -14,6 +14,41 @@
> >  #include <asm/alternative.h>
> >  #include <asm/asm-uaccess.h>
> >  
> > +/*
> > + *	__flush_cache_range(start,end) [fixup]
> > + *
> > + *	Ensure that the I and D caches are coherent within specified region.
> > + *	This is typically used when code has been written to a memory region,
> > + *	and will be executed.
> > + *
> > + *	- start   - virtual start address of region
> > + *	- end     - virtual end address of region
> > + *	- fixup   - optional label to branch to on user fault
> > + */
> > +.macro	__flush_cache_range, fixup
> > +alternative_if ARM64_HAS_CACHE_IDC
> > +	dsb	ishst
> > +	b	.Ldc_skip_\@
> > +alternative_else_nop_endif
> > +	dcache_line_size x2, x3
> > +	sub	x3, x2, #1
> > +	bic	x4, x0, x3
> > +.Ldc_loop_\@:
> > +user_alt "dc cvau, x4",  "dc civac, x4",  ARM64_WORKAROUND_CLEAN_CACHE, \fixup
> > +	add	x4, x4, x2
> > +	cmp	x4, x1
> > +	b.lo	.Ldc_loop_\@
> > +	dsb	ish
> 
> As on the prior patch, I reckon it'd be nicer overall to align with the
> *by_line macros and have an explicit _cond_extable here, e.g.
> 
> | .Ldc_op\@:
> | 	alternative_insn "dc cvau, x4",  "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE
> | 	add	x4, x4, x2
> | 	cmp     x4, x1
> | 	b.lo	.Ldc_op\@
> | 	dsb	ish
> | ...
> | 	// just before the .endm
> | 	_cond_extable .Ldc_op\@, \fixup
> 
> ... and with some rework it might be possible to use dcache_by_line_op
> directly here (it currently clobbers the base and end, so can't be used
> as-is).

Having thought about this a bit more, it's simple enough to do that now:

| alternative_if ARM64_HAS_CACHE_IDC
| 	dsb	ishst
| 	b	.Ldc_skip_\@
| alternative_else_nop_endif
| 	mov	x0, x2
| 	add	x3, x0, x1
| 	dcache_by_line_op cvau, ishst, x2, x3, x4, x5, \fixup
| .Ldc_skip_\@

... and when we just need to change the ADD to a MOV when we change the
macro to take the end in x1.

Note that dcache_by_line_op will automatically upgrade 'cvau' to 'civac'
when ARM64_WORKAROUND_CLEAN_CACHE is present, so the resulting logic is
the same.

Thanks,
Mark.