[PATCH V11 00/17] riscv: Add Native/Paravirt qspinlock support

Leonardo Bras Soares Passos leobras at redhat.com
Mon Nov 13 02:19:01 PST 2023


On Sun, Nov 12, 2023 at 1:24 AM Guo Ren <guoren at kernel.org> wrote:
>
> On Mon, Nov 6, 2023 at 3:42 PM Leonardo Bras <leobras at redhat.com> wrote:
> >
> > On Sun, Sep 10, 2023 at 04:28:54AM -0400, guoren at kernel.org wrote:
> > > From: Guo Ren <guoren at linux.alibaba.com>
> > >
> > > patch[1 - 10]: Native   qspinlock
> > > patch[11 -17]: Paravirt qspinlock
> > >
> > > patch[4]: Add prefetchw in qspinlock's xchg_tail when cpus >= 16k
> > >
> > > This series based on:
> > >  - [RFC PATCH v5 0/5] Rework & improve riscv cmpxchg.h and atomic.h
> > >    https://lore.kernel.org/linux-riscv/20230810040349.92279-2-leobras@redhat.com/
> > >  - [PATCH V3] asm-generic: ticket-lock: Optimize arch_spin_value_unlocked
> > >    https://lore.kernel.org/linux-riscv/20230908154339.3250567-1-guoren@kernel.org/
> > >
> > > I merge them into sg2042-master branch, then you could directly try it on
> > > sg2042 hardware platform:
> > >
> > > https://github.com/guoren83/linux/tree/sg2042-master-qspinlock-64ilp32_v5
> > >
> > > Use sophgo_mango_ubuntu_defconfig for sg2042 64/128 cores hardware
> > > platform.
> > >
> > > Native qspinlock
> > > ================
> > >
> > > This time we've proved the qspinlock on th1520 [1] & sg2042 [2], which
> > > gives stability and performance improvement. All T-HEAD processors have
> > > a strong LR/SC forward progress guarantee than the requirements of the
> > > ISA, which could satisfy the xchg_tail of native_qspinlock. Now,
> > > qspinlock has been run with us for more than 1 year, and we have enough
> > > confidence to enable it for all the T-HEAD processors. Of causes, we
> > > found a livelock problem with the qspinlock lock torture test from the
> > > CPU store merge buffer delay mechanism, which caused the queued spinlock
> > > becomes a dead ring and RCU warning to come out. We introduce a custom
> > > WRITE_ONCE to solve this. Do we need explicit ISA instruction to signal
> > > it? Or let hardware handle this.
> > >
> > > We've tested the patch on SOPHGO sg2042 & th1520 and passed the stress
> > > test on Fedora & Ubuntu & OpenEuler ... Here is the performance
> > > comparison between qspinlock and ticket_lock on sg2042 (64 cores):
> > >
> > > sysbench test=threads threads=32 yields=100 lock=8 (+13.8%):
> > >   queued_spinlock 0.5109/0.00
> > >   ticket_spinlock 0.5814/0.00
> > >
> > > perf futex/hash (+6.7%):
> > >   queued_spinlock 1444393 operations/sec (+- 0.09%)
> > >   ticket_spinlock 1353215 operations/sec (+- 0.15%)
> > >
> > > perf futex/wake-parallel (+8.6%):
> > >   queued_spinlock (waking 1/64 threads) in 0.0253 ms (+-2.90%)
> > >   ticket_spinlock (waking 1/64 threads) in 0.0275 ms (+-3.12%)
> > >
> > > perf futex/requeue (+4.2%):
> > >   queued_spinlock Requeued 64 of 64 threads in 0.0785 ms (+-0.55%)
> > >   ticket_spinlock Requeued 64 of 64 threads in 0.0818 ms (+-4.12%)
> > >
> > > System Benchmarks (+6.4%)
> > >   queued_spinlock:
> > >     System Benchmarks Index Values               BASELINE       RESULT    INDEX
> > >     Dhrystone 2 using register variables         116700.0  628613745.4  53865.8
> > >     Double-Precision Whetstone                       55.0     182422.8  33167.8
> > >     Execl Throughput                                 43.0      13116.6   3050.4
> > >     File Copy 1024 bufsize 2000 maxblocks          3960.0    7762306.2  19601.8
> > >     File Copy 256 bufsize 500 maxblocks            1655.0    3417556.8  20649.9
> > >     File Copy 4096 bufsize 8000 maxblocks          5800.0    7427995.7  12806.9
> > >     Pipe Throughput                               12440.0   23058600.5  18535.9
> > >     Pipe-based Context Switching                   4000.0    2835617.7   7089.0
> > >     Process Creation                                126.0      12537.3    995.0
> > >     Shell Scripts (1 concurrent)                     42.4      57057.4  13456.9
> > >     Shell Scripts (8 concurrent)                      6.0       7367.1  12278.5
> > >     System Call Overhead                          15000.0   33308301.3  22205.5
> > >                                                                        ========
> > >     System Benchmarks Index Score                                       12426.1
> > >
> > >   ticket_spinlock:
> > >     System Benchmarks Index Values               BASELINE       RESULT    INDEX
> > >     Dhrystone 2 using register variables         116700.0  626541701.9  53688.2
> > >     Double-Precision Whetstone                       55.0     181921.0  33076.5
> > >     Execl Throughput                                 43.0      12625.1   2936.1
> > >     File Copy 1024 bufsize 2000 maxblocks          3960.0    6553792.9  16550.0
> > >     File Copy 256 bufsize 500 maxblocks            1655.0    3189231.6  19270.3
> > >     File Copy 4096 bufsize 8000 maxblocks          5800.0    7221277.0  12450.5
> > >     Pipe Throughput                               12440.0   20594018.7  16554.7
> > >     Pipe-based Context Switching                   4000.0    2571117.7   6427.8
> > >     Process Creation                                126.0      10798.4    857.0
> > >     Shell Scripts (1 concurrent)                     42.4      57227.5  13497.1
> > >     Shell Scripts (8 concurrent)                      6.0       7329.2  12215.3
> > >     System Call Overhead                          15000.0   30766778.4  20511.2
> > >                                                                        ========
> > >     System Benchmarks Index Score                                       11670.7
> > >
> > > The qspinlock has a significant improvement on SOPHGO SG2042 64
> > > cores platform than the ticket_lock.
> > >
> > > Paravirt qspinlock
> > > ==================
> > >
> > > We implemented kvm_kick_cpu/kvm_wait_cpu and add tracepoints to observe the
> > > behaviors. Also, introduce a new SBI extension SBI_EXT_PVLOCK (0xAB0401). If the
> > > name and number are approved, I will send a formal proposal to the SBI spec.
> > >
> >
> > Hello Guo Ren,
> >
> > Any update on this series?
> Found a nested virtualization problem, and I'm solving that. After
> that, I'll update v12.

Oh, nice to hear :)
I am very excited about this series, please let me know of any update.

Thanks!
Leo

>
> >
> > Thanks!
> > Leo
> >
> >
> > > Changlog:
> > > V11:
> > >  - Based on Leonardo Bras's cmpxchg_small patches v5.
> > >  - Based on Guo Ren's Optimize arch_spin_value_unlocked patch v3.
> > >  - Remove abusing alternative framework and use jump_label instead.
> > >  - Introduce prefetch.w to improve T-HEAD processors' LR/SC forward progress
> > >    guarantee.
> > >  - Optimize qspinlock xchg_tail when NR_CPUS >= 16K.
> > >
> > > V10:
> > > https://lore.kernel.org/linux-riscv/20230802164701.192791-1-guoren@kernel.org/
> > >  - Using an alternative framework instead of static_key_branch in the
> > >    asm/spinlock.h.
> > >  - Fixup store merge buffer problem, which causes qspinlock lock
> > >    torture test livelock.
> > >  - Add paravirt qspinlock support, include KVM backend
> > >  - Add Compact NUMA-awared qspinlock support
> > >
> > > V9:
> > > https://lore.kernel.org/linux-riscv/20220808071318.3335746-1-guoren@kernel.org/
> > >  - Cleanup generic ticket-lock code, (Using smp_mb__after_spinlock as
> > >    RCsc)
> > >  - Add qspinlock and combo-lock for riscv
> > >  - Add qspinlock to openrisc
> > >  - Use generic header in csky
> > >  - Optimize cmpxchg & atomic code
> > >
> > > V8:
> > > https://lore.kernel.org/linux-riscv/20220724122517.1019187-1-guoren@kernel.org/
> > >  - Coding convention ticket fixup
> > >  - Move combo spinlock into riscv and simply asm-generic/spinlock.h
> > >  - Fixup xchg16 with wrong return value
> > >  - Add csky qspinlock
> > >  - Add combo & qspinlock & ticket-lock comparison
> > >  - Clean up unnecessary riscv acquire and release definitions
> > >  - Enable ARCH_INLINE_READ*/WRITE*/SPIN* for riscv & csky
> > >
> > > V7:
> > > https://lore.kernel.org/linux-riscv/20220628081946.1999419-1-guoren@kernel.org/
> > >  - Add combo spinlock (ticket & queued) support
> > >  - Rename ticket_spinlock.h
> > >  - Remove unnecessary atomic_read in ticket_spin_value_unlocked
> > >
> > > V6:
> > > https://lore.kernel.org/linux-riscv/20220621144920.2945595-1-guoren@kernel.org/
> > >  - Fixup Clang compile problem Reported-by: kernel test robot
> > >  - Cleanup asm-generic/spinlock.h
> > >  - Remove changelog in patch main comment part, suggested by
> > >    Conor.Dooley
> > >  - Remove "default y if NUMA" in Kconfig
> > >
> > > V5:
> > > https://lore.kernel.org/linux-riscv/20220620155404.1968739-1-guoren@kernel.org/
> > >  - Update comment with RISC-V forward guarantee feature.
> > >  - Back to V3 direction and optimize asm code.
> > >
> > > V4:
> > > https://lore.kernel.org/linux-riscv/1616868399-82848-4-git-send-email-guoren@kernel.org/
> > >  - Remove custom sub-word xchg implementation
> > >  - Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 in locking/qspinlock
> > >
> > > V3:
> > > https://lore.kernel.org/linux-riscv/1616658937-82063-1-git-send-email-guoren@kernel.org/
> > >  - Coding convention by Peter Zijlstra's advices
> > >
> > > V2:
> > > https://lore.kernel.org/linux-riscv/1606225437-22948-2-git-send-email-guoren@kernel.org/
> > >  - Coding convention in cmpxchg.h
> > >  - Re-implement short xchg
> > >  - Remove char & cmpxchg implementations
> > >
> > > V1:
> > > https://lore.kernel.org/linux-riscv/20190211043829.30096-1-michaeljclark@mac.com/
> > >  - Using cmpxchg loop to implement sub-word atomic
> > >
> > >
> > > Guo Ren (17):
> > >   asm-generic: ticket-lock: Reuse arch_spinlock_t of qspinlock
> > >   asm-generic: ticket-lock: Move into ticket_spinlock.h
> > >   riscv: Use Zicbop in arch_xchg when available
> > >   locking/qspinlock: Improve xchg_tail for number of cpus >= 16k
> > >   riscv: qspinlock: Add basic queued_spinlock support
> > >   riscv: qspinlock: Introduce combo spinlock
> > >   riscv: qspinlock: Introduce qspinlock param for command line
> > >   riscv: qspinlock: Add virt_spin_lock() support for KVM guest
> > >   riscv: qspinlock: errata: Add ERRATA_THEAD_WRITE_ONCE fixup
> > >   riscv: qspinlock: errata: Enable qspinlock for T-HEAD processors
> > >   RISC-V: paravirt: pvqspinlock: Add paravirt qspinlock skeleton
> > >   RISC-V: paravirt: pvqspinlock: Add nopvspin kernel parameter
> > >   RISC-V: paravirt: pvqspinlock: Add SBI implementation
> > >   RISC-V: paravirt: pvqspinlock: Add kconfig entry
> > >   RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait
> > >   RISC-V: paravirt: pvqspinlock: KVM: Add paravirt qspinlock skeleton
> > >   RISC-V: paravirt: pvqspinlock: KVM: Implement
> > >     kvm_sbi_ext_pvlock_kick_cpu()
> > >
> > >  .../admin-guide/kernel-parameters.txt         |   8 +-
> > >  arch/riscv/Kconfig                            |  50 ++++++++
> > >  arch/riscv/Kconfig.errata                     |  19 +++
> > >  arch/riscv/errata/thead/errata.c              |  29 +++++
> > >  arch/riscv/include/asm/Kbuild                 |   2 +-
> > >  arch/riscv/include/asm/cmpxchg.h              |   4 +-
> > >  arch/riscv/include/asm/errata_list.h          |  13 --
> > >  arch/riscv/include/asm/hwcap.h                |   1 +
> > >  arch/riscv/include/asm/insn-def.h             |   5 +
> > >  arch/riscv/include/asm/kvm_vcpu_sbi.h         |   1 +
> > >  arch/riscv/include/asm/processor.h            |  13 ++
> > >  arch/riscv/include/asm/qspinlock.h            |  35 ++++++
> > >  arch/riscv/include/asm/qspinlock_paravirt.h   |  29 +++++
> > >  arch/riscv/include/asm/rwonce.h               |  24 ++++
> > >  arch/riscv/include/asm/sbi.h                  |  14 +++
> > >  arch/riscv/include/asm/spinlock.h             | 113 ++++++++++++++++++
> > >  arch/riscv/include/asm/vendorid_list.h        |  14 +++
> > >  arch/riscv/include/uapi/asm/kvm.h             |   1 +
> > >  arch/riscv/kernel/Makefile                    |   1 +
> > >  arch/riscv/kernel/cpufeature.c                |   1 +
> > >  arch/riscv/kernel/qspinlock_paravirt.c        |  83 +++++++++++++
> > >  arch/riscv/kernel/sbi.c                       |   2 +-
> > >  arch/riscv/kernel/setup.c                     |  60 ++++++++++
> > >  .../kernel/trace_events_filter_paravirt.h     |  60 ++++++++++
> > >  arch/riscv/kvm/Makefile                       |   1 +
> > >  arch/riscv/kvm/vcpu_sbi.c                     |   4 +
> > >  arch/riscv/kvm/vcpu_sbi_pvlock.c              |  57 +++++++++
> > >  include/asm-generic/rwonce.h                  |   2 +
> > >  include/asm-generic/spinlock.h                |  87 +-------------
> > >  include/asm-generic/spinlock_types.h          |  12 +-
> > >  include/asm-generic/ticket_spinlock.h         | 103 ++++++++++++++++
> > >  kernel/locking/qspinlock.c                    |   5 +-
> > >  32 files changed, 739 insertions(+), 114 deletions(-)
> > >  create mode 100644 arch/riscv/include/asm/qspinlock.h
> > >  create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h
> > >  create mode 100644 arch/riscv/include/asm/rwonce.h
> > >  create mode 100644 arch/riscv/include/asm/spinlock.h
> > >  create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c
> > >  create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h
> > >  create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c
> > >  create mode 100644 include/asm-generic/ticket_spinlock.h
> > >
> > > --
> > > 2.36.1
> > >
> >
>
>
> --
> Best Regards
>  Guo Ren
>




More information about the linux-riscv mailing list