[PATCH bpf-next v2 0/3] bpf, riscv: use BPF prog pack allocator in BPF JIT

[Puranjay Mohan]

On Fri, Aug 25, 2023 at 10:06 AM Pu Lehui <pulehui at huawei.com> wrote:
>
>
>
> On 2023/8/24 21:31, Puranjay Mohan wrote:
> > Changes in v1 -> v2:
> > 1. Implement a new function patch_text_set_nosync() to be used in bpf_arch_text_invalidate().
> >     The implementation in v1 called patch_text_nosync() in a loop and it was bad as it would
> >     call flush_icache_range() for every word making it really slow. This was found by running
> >     the test_tag selftest which would take forever to complete.
> >
> > Here is some data to prove the V2 fixes the problem:
> >
> > Without this series:
> > root at rv-selftester:~/src/kselftest/bpf# time ./test_tag
> > test_tag: OK (40945 tests)
> >
> > real    7m47.562s
> > user    0m24.145s
> > sys     6m37.064s
> >
> > With this series applied:
> > root at rv-selftester:~/src/selftest/bpf# time ./test_tag
> > test_tag: OK (40945 tests)
> >
> > real    7m29.472s
> > user    0m25.865s
> > sys     6m18.401s
> >
> > BPF programs currently consume a page each on RISCV. For systems with many BPF
> > programs, this adds significant pressure to instruction TLB. High iTLB pressure
> > usually causes slow down for the whole system.
> >
> > Song Liu introduced the BPF prog pack allocator[1] to mitigate the above issue.
> > It packs multiple BPF programs into a single huge page. It is currently only
> > enabled for the x86_64 BPF JIT.
> >
> > I enabled this allocator on the ARM64 BPF JIT[2]. It is being reviewed now.
> >
> > This patch series enables the BPF prog pack allocator for the RISCV BPF JIT.
> > This series needs a patch[3] from the ARM64 series to work.
>
> Is there a new version for arm64 currently? Maybe we could submit this
> patch first as a separate patch to avoid dependencies.

Okay, I will send that patch as a separate patch because it is needed for all
architectures.

>
> >
> > ======================================================
> > Performance Analysis of prog pack allocator on RISCV64
> > ======================================================
> >
> > Test setup:
> > ===========
> >
> > Host machine: Debian GNU/Linux 11 (bullseye)
> > Qemu Version: QEMU emulator version 8.0.3 (Debian 1:8.0.3+dfsg-1)
> > u-boot-qemu Version: 2023.07+dfsg-1
> > opensbi Version: 1.3-1
> >
> > To test the performance of the BPF prog pack allocator on RV, a stresser
> > tool[4] linked below was built. This tool loads 8 BPF programs on the system and
> > triggers 5 of them in an infinite loop by doing system calls.
> >
> > The runner script starts 20 instances of the above which loads 8*20=160 BPF
> > programs on the system, 5*20=100 of which are being constantly triggered.
> > The script is passed a command which would be run in the above environment.
> >
> > The script was run with following perf command:
> > ./run.sh "perf stat -a \
> >          -e iTLB-load-misses \
> >          -e dTLB-load-misses  \
> >          -e dTLB-store-misses \
> >          -e instructions \
> >          --timeout 60000"
> >
> > The output of the above command is discussed below before and after enabling the
> > BPF prog pack allocator.
> >
> > The tests were run on qemu-system-riscv64 with 8 cpus, 16G memory. The rootfs
> > was created using Bjorn's riscv-cross-builder[5] docker container linked below.
> >
> > Results
> > =======
> >
> > Before enabling prog pack allocator:
> > ------------------------------------
> >
> > Performance counter stats for 'system wide':
> >
> >             4939048      iTLB-load-misses
> >             5468689      dTLB-load-misses
> >              465234      dTLB-store-misses
> >       1441082097998      instructions
> >
> >        60.045791200 seconds time elapsed
> >
> > After enabling prog pack allocator:
> > -----------------------------------
> >
> > Performance counter stats for 'system wide':
> >
> >             3430035      iTLB-load-misses
> >             5008745      dTLB-load-misses
> >              409944      dTLB-store-misses
> >       1441535637988      instructions
> >
> >        60.046296600 seconds time elapsed
> >
> > Improvements in metrics
> > =======================
> >
> > It was expected that the iTLB-load-misses would decrease as now a single huge
> > page is used to keep all the BPF programs compared to a single page for each
> > program earlier.
> >
> > --------------------------------------------
> > The improvement in iTLB-load-misses: -30.5 %
> > --------------------------------------------
> >
> > I repeated this expriment more than 100 times in different setups and the
> > improvement was always greater than 30%.
> >
> > This patch series is boot tested on the Starfive VisionFive 2 board[6].
> > The performance analysis was not done on the board because it doesn't
> > expose iTLB-load-misses, etc. The stresser program was run on the board to test
> > the loading and unloading of BPF programs
> >
> > [1] https://lore.kernel.org/bpf/20220204185742.271030-1-song@kernel.org/
> > [2] https://lore.kernel.org/all/20230626085811.3192402-1-puranjay12@gmail.com/
> > [3] https://lore.kernel.org/all/20230626085811.3192402-2-puranjay12@gmail.com/
> > [4] https://github.com/puranjaymohan/BPF-Allocator-Bench
> > [5] https://github.com/bjoto/riscv-cross-builder
> > [6] https://www.starfivetech.com/en/site/boards
> >
> > Puranjay Mohan (3):
> >    riscv: extend patch_text_nosync() for multiple pages
> >    riscv: implement a memset like function for text
> >    bpf, riscv: use prog pack allocator in the BPF JIT
> >
> >   arch/riscv/include/asm/patch.h  |   1 +
> >   arch/riscv/kernel/patch.c       | 113 ++++++++++++++++++++++++++++++--
> >   arch/riscv/net/bpf_jit.h        |   3 +
> >   arch/riscv/net/bpf_jit_comp64.c |  56 +++++++++++++---
> >   arch/riscv/net/bpf_jit_core.c   | 113 +++++++++++++++++++++++++++-----
> >   5 files changed, 255 insertions(+), 31 deletions(-)
> >



-- 
Thanks and Regards

Yours Truly,

Puranjay Mohan