[PATCH v11 10/14] riscv: hwprobe: Add thead vendor extension probing

Wed Nov 13 23:23:24 PST 2024

On Thu, Nov 14, 2024 at 02:54:17PM +0800, Yangyu Chen wrote:
> 
> 
> On 11/14/24 12:46, Charlie Jenkins wrote:
> > On Thu, Nov 14, 2024 at 11:26:47AM +0800, Yangyu Chen wrote:
> > > 
> > > 
> > > On 11/14/24 11:02, Charlie Jenkins wrote:
> > > > On Thu, Nov 14, 2024 at 10:44:37AM +0800, Yangyu Chen wrote:
> > > > > 
> > > > > 
> > > > > On 11/14/24 10:21, Charlie Jenkins wrote:
> > > > > > Add a new hwprobe key "RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0" which
> > > > > > allows userspace to probe for the new RISCV_ISA_VENDOR_EXT_XTHEADVECTOR
> > > > > > vendor extension.
> > > > > > 
> > > > > 
> > > > > Hi Charlie,
> > > > > 
> > > > > How about changing the name of the key from
> > > > > "RISCV_ISA_VENDOR_EXT_XTHEADVECTOR" to "RISCV_HWPROBE_KEY_VENDOR_EXT_0" and
> > > > > use marchid to identify what the vendor is, each vendor will have its own
> > > > > bit definition in this value. So we can avoid adding so many hwprobe keys
> > > > > for each vendor in the future.
> > > > > 
> > > > > I proposed a commit here: https://github.com/cyyself/linux/commit/36390645d85d1ac75dd71172f167719df4297f59
> > > > 
> > > > I actually originally had this in one of my first versions of this
> > > > series but was convinced by Conor to change it. The problem with it was
> > > > that tying vendor extensions to mvendorid means that it is enforced by
> > > > the kernel that vendors cannot share vendor extensions. It is possible
> > > > for vendor A to purchase IP that contains a vendor extension from vendor
> > > > B. This vendor extension should work on platforms created by vendor A
> > > > and vendor B. However, vendor A and vendor B have different mvendorids,
> > > > so the kernel can't support this if it is tied to mvendorid.  It could
> > > > be solved by duplicating every extension that vendors have, but then
> > > > userspace software would have to keep in mind the mvendorid they are
> > > > running on and check the different extensions for the different vendors
> > > > even though the implementation of the extension is the same.
> > > > 
> > > > The original conversation where Conor and I agreed that it was better to
> > > > have vendor extensions not rely on mvendorid:
> > > > 
> > > > https://lore.kernel.org/linux-riscv/20240416-husband-flavored-96c1dad58b6e@wendy/
> > > > 
> > > 
> > > Thanks for your explanation. I will strongly agree with Conor's opinion if
> > > the feature bitmask does not exist in RISC-V C-ABI.
> > > 
> > > However, as the feature mask defined in RISC-V C-ABI[1] uses the design
> > > depending on marchid currently, should we reconsider this key for its use
> > > case? The current target_clones and taget_version implemented in GCC[2] and
> > > LLVM[3] also use the bitmask defined in C-ABI. I think if we use this key
> > > depending on marchid, to make a key shared with all vendors will make this
> > > cleaner.
> > 
> > Changing this will break linux userspace API. It is a non-workable
> > solution for the kernel to associate extensions with marchid/mvendorid
> > for the reasons provided. I fail to see why this ABI would require the
> > kernel to behave in this manner. The ABI provides the marchid to be used
> > by function multi-versioning and applications are free to use the
> > marchid to change which function they want to compile. However, if they
> > want to know if an extension is supported, then they need to use
> > hwprobe. If they want to check if xtheadvector is supported, then they> call hwprobe with the xtheadvector key. This is true no matter what the
> > mvendorid of the system is.
> 
> A userspace software can use either c-api defined feature masks or directly
> use hwprobe syscall. If they use c-api defined feature masks as GCC or LLVM
> did for compiler generated IFUNC resolver, the bitmask is guarded by
> mvendorid. So my point at that time was that if the C-API defined way became
> mainstream, why should we keep this key only for T-Head to increase the
> maintenance overhead?

Yes that makes sense. I was thinking that the Andes PMU extension had a
hwprobe key, but I realized that it does not. This patch has been on the
lists for so long I lost track! I was trying to design this to be
forward-thinking, I believe that it makes more sense this way, but I am
interested in the opinion of the c-api maintainers.

> 
> This has been discussed here before in RISC-V C-API: https://github.com/riscv-non-isa/riscv-c-api-doc/pull/74#issuecomment-2128844747
> 
> But now (from the last email), you convinced me. So, I would like to make
> the c-api change: https://github.com/riscv-non-isa/riscv-c-api-doc/issues/96
> 

Thank you for opening that!

> > This does not add any complexity, "clean"
> > code can equally be written following this scheme or following a scheme
> > that relies on mvendorid. Ditching the reliance on mvendorid in the
> > kernel allows the kernel to be as generic as possible, and allow
> > whatever ABIs or hardware that exist to have a resiliant way of
> > communicating with the kernel.
> > 
> 
> OK. I'm just concerned about when these vendors will add the hwprobe key for
> their own extension, which may introduce a potential merge conflict in the
> kernel tree. It can also be a disaster if the hardware vendor ships their
> kernel with these under-review patches for their products with hwprobe key
> conflict with mainline kernel.
> 
> But we can avoid this now by adding each key for each vendor to avoid
> potential conflict in the future. This can be a separate patch for future
> work, so there is nothing to change here.

Yes that is unfortunately the downside of hwprobe that it is a
centralized source of these keys, and that can be exacerbated by this
scheme were vendor keys are not completely isolated from each other.
That would be very unfortunate if a vendor ships a kernel and binaries
that has different keys than the mainline kernel. Hopefully vendors
don't do that, but it should be manageable for vendors to submit their own
keys.

Thank you for bringing this up, it is an important issue! A main goal of
this series was to get vendor extensions in a state that would be able
to grow into a future when there are lots of vendors.

> 
> Thanks,
> Yangyu Chen
> 
> > - CHarlie
> > 
> > > 
> > > [1] https://github.com/riscv-non-isa/riscv-c-api-doc/blob/main/src/c-api.adoc#function-multi-version
> > > [2] https://github.com/gcc-mirror/gcc/blob/8564d0948c72df0a66d7eb47e15c6ab43e9b25ce/gcc/config/riscv/riscv.cc#L13016
> > > [3] https://github.com/llvm/llvm-project/blob/f407dff50cdcbcfee9dd92397d3792627c3ac708/clang/lib/CodeGen/CGBuiltin.cpp#L14627
> > > 
> > > > > 
> > > > > > This new key will allow userspace code to probe for which thead vendor
> > > > > > extensions are supported. This API is modeled to be consistent with
> > > > > > RISCV_HWPROBE_KEY_IMA_EXT_0. The bitmask returned will have each bit
> > > > > > corresponding to a supported thead vendor extension of the cpumask set.
> > > > > > Just like RISCV_HWPROBE_KEY_IMA_EXT_0, this allows a userspace program
> > > > > > to determine all of the supported thead vendor extensions in one call.
> > > > > > 
> > > > > > Signed-off-by: Charlie Jenkins <charlie at rivosinc.com>
> > > > > > Reviewed-by: Evan Green <evan at rivosinc.com>
> > > > > > ---
> > > > > >     arch/riscv/include/asm/hwprobe.h                   |  3 +-
> > > > > >     .../include/asm/vendor_extensions/thead_hwprobe.h  | 19 +++++++++++
> > > > > >     .../include/asm/vendor_extensions/vendor_hwprobe.h | 37 ++++++++++++++++++++++
> > > > > >     arch/riscv/include/uapi/asm/hwprobe.h              |  3 +-
> > > > > >     arch/riscv/include/uapi/asm/vendor/thead.h         |  3 ++
> > > > > >     arch/riscv/kernel/sys_hwprobe.c                    |  5 +++
> > > > > >     arch/riscv/kernel/vendor_extensions/Makefile       |  1 +
> > > > > >     .../riscv/kernel/vendor_extensions/thead_hwprobe.c | 19 +++++++++++
> > > > > >     8 files changed, 88 insertions(+), 2 deletions(-)
> > > > > > 
> > > > > 
> > > 
> > > 
> 
> 