[External] Re: [PATCH] RISC-V: Enable Zicbom in usermode

[Andrew Jones]

On Thu, Oct 31, 2024 at 04:29:49PM +0800, yunhui cui wrote:
> Hi Jessica,
> 
> On Sat, Oct 26, 2024 at 12:32 AM Jessica Clarke <jrtc27 at jrtc27.com> wrote:
> >
> > On 25 Oct 2024, at 11:16, Conor Dooley <conor at kernel.org> wrote:
> > > On Fri, Oct 25, 2024 at 05:15:27PM +0800, Yunhui Cui wrote:
> > >> Like Zicboz, by enabling the corresponding bits of senvcfg,
> > >> the instructions cbo.clean, cbo.flush, and cbo.inval can be
> > >> executed normally in user mode.
> > >>
> > >> Signed-off-by: Yunhui Cui <cuiyunhui at bytedance.com>
> > >> ---
> > >> arch/riscv/kernel/cpufeature.c | 2 +-
> > >> 1 file changed, 1 insertion(+), 1 deletion(-)
> > >>
> > >> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> > >> index 1992ea64786e..bc850518ab41 100644
> > >> --- a/arch/riscv/kernel/cpufeature.c
> > >> +++ b/arch/riscv/kernel/cpufeature.c
> > >> @@ -924,7 +924,7 @@ unsigned long riscv_get_elf_hwcap(void)
> > >> void __init riscv_user_isa_enable(void)
> > >> {
> > >> if (riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICBOZ))
> > >> - current->thread.envcfg |= ENVCFG_CBZE;
> > >> + current->thread.envcfg |= ENVCFG_CBIE | ENVCFG_CBCFE | ENVCFG_CBZE;
> > >
> > > I believe we previously decided that userspace should not be allowed to
> > > use zicbom, but that not withstanding - this is wrong. It should be
> > > checking for Zicbom, not Zicboz.
> >
> > Allowing clean/flush is safe but has the same problems as fence.i with
> > regards to migrating between harts. Allowing invalidate, unless mapped
> > to flush, is not safe in general unless the kernel does a lot of
> > flushing to avoid userspace accessing data it shouldn’t be able to see.
> >
> > Also, ENVCFG_CBIE is a mask for a multi-bit field, which happens to
> > have the same value as ENVCFG_CBIE_INV (i.e. really is making cbo.inval
> > be an invalidate). I note that the KVM code, which this likely copied
> > from(?), makes the same mistake, but there that is the intended
> > behaviour, if misleading about what the field really is.
> >
> > So, with suitable caveats, allowing clean/flush could be a reasonable
> > thing to do (maybe useful for userspace drivers so long as they pin
> > themselves to a specific hart?), but invalidate should only ever be
> > allowed if mapped to flush.
> >
> > Jess
> >
> Yes. The original intention is to enable clean/flush/invalid. So
> ENVCFG_CBIE | ENVCFG_CBCFE is added. When one core initiates an
> invalidation, other cores will also invalidate the corresponding cache
> line. So do we not need to worry about this problem? Moreover,
> invalidation is not found in the logic of disabling preemption in the
> kernel. Or perhaps binding cores belongs to the user-space's own
> logic. Can this patch be fixed as RISCV_ISA_EXT_ZICBOM and then a v2
> be sent?

Jessica points out that CBIE should only be set if it's 0b01 but it's
currently 0b11, so just changing RISCV_ISA_EXT_ZICBOM is insufficient
for v2.

Is there a use case for usermode to have an invalidate without a clean?
If so, then is that use case not present on Arm platforms or is there a
workaround for Arm platforms? Because Arm's 'DC IVAC' is only allowed for
EL1 and higher. To be consistent with Arm we can add cbo.flush and
cbo.clean (CBCFE) since Arm does allow exposing 'DC CIVAC' and 'DC CVAC'
to EL0. Since it looks like CBIE=0b01 would just make cbo.inval an alias
of cbo.flush, then I'm not sure it makes sense to set it, in fact it may
just confuse things.

Thanks,
drew