[RESEND PATCH v3] arm64: enable THP_SWAP for arm64
Huang, Ying
ying.huang at intel.com
Mon Jul 18 22:46:24 PDT 2022
Barry Song <21cnbao at gmail.com> writes:
> On Tue, Jul 19, 2022 at 3:59 PM Barry Song <21cnbao at gmail.com> wrote:
>>
>> On Tue, Jul 19, 2022 at 3:35 PM Anshuman Khandual
>> <anshuman.khandual at arm.com> wrote:
>> >
>> >
>> >
>> > On 7/19/22 08:58, Huang, Ying wrote:
>> > > Anshuman Khandual <anshuman.khandual at arm.com> writes:
>> > >
>> > >> On 7/19/22 06:53, Barry Song wrote:
>> > >>> On Tue, Jul 19, 2022 at 12:44 PM Huang, Ying <ying.huang at intel.com> wrote:
>> > >>>>
>> > >>>> Barry Song <21cnbao at gmail.com> writes:
>> > >>>>
>> > >>>>> From: Barry Song <v-songbaohua at oppo.com>
>> > >>>>>
>> > >>>>> THP_SWAP has been proven to improve the swap throughput significantly
>> > >>>>> on x86_64 according to commit bd4c82c22c367e ("mm, THP, swap: delay
>> > >>>>> splitting THP after swapped out").
>> > >>>>> As long as arm64 uses 4K page size, it is quite similar with x86_64
>> > >>>>> by having 2MB PMD THP. THP_SWAP is architecture-independent, thus,
>> > >>>>> enabling it on arm64 will benefit arm64 as well.
>> > >>>>> A corner case is that MTE has an assumption that only base pages
>> > >>>>> can be swapped. We won't enable THP_SWAP for ARM64 hardware with
>> > >>>>> MTE support until MTE is reworked to coexist with THP_SWAP.
>> > >>>>>
>> > >>>>> A micro-benchmark is written to measure thp swapout throughput as
>> > >>>>> below,
>> > >>>>>
>> > >>>>> unsigned long long tv_to_ms(struct timeval tv)
>> > >>>>> {
>> > >>>>> return tv.tv_sec * 1000 + tv.tv_usec / 1000;
>> > >>>>> }
>> > >>>>>
>> > >>>>> main()
>> > >>>>> {
>> > >>>>> struct timeval tv_b, tv_e;;
>> > >>>>> #define SIZE 400*1024*1024
>> > >>>>> volatile void *p = mmap(NULL, SIZE, PROT_READ | PROT_WRITE,
>> > >>>>> MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>> > >>>>> if (!p) {
>> > >>>>> perror("fail to get memory");
>> > >>>>> exit(-1);
>> > >>>>> }
>> > >>>>>
>> > >>>>> madvise(p, SIZE, MADV_HUGEPAGE);
>> > >>>>> memset(p, 0x11, SIZE); /* write to get mem */
>> > >>>>>
>> > >>>>> gettimeofday(&tv_b, NULL);
>> > >>>>> madvise(p, SIZE, MADV_PAGEOUT);
>> > >>>>> gettimeofday(&tv_e, NULL);
>> > >>>>>
>> > >>>>> printf("swp out bandwidth: %ld bytes/ms\n",
>> > >>>>> SIZE/(tv_to_ms(tv_e) - tv_to_ms(tv_b)));
>> > >>>>> }
>> > >>>>>
>> > >>>>> Testing is done on rk3568 64bit quad core processor Quad Core
>> > >>>>> Cortex-A55 platform - ROCK 3A.
>> > >>>>> thp swp throughput w/o patch: 2734bytes/ms (mean of 10 tests)
>> > >>>>> thp swp throughput w/ patch: 3331bytes/ms (mean of 10 tests)
>> > >>>>>
>> > >>>>> Cc: "Huang, Ying" <ying.huang at intel.com>
>> > >>>>> Cc: Minchan Kim <minchan at kernel.org>
>> > >>>>> Cc: Johannes Weiner <hannes at cmpxchg.org>
>> > >>>>> Cc: Hugh Dickins <hughd at google.com>
>> > >>>>> Cc: Andrea Arcangeli <aarcange at redhat.com>
>> > >>>>> Cc: Anshuman Khandual <anshuman.khandual at arm.com>
>> > >>>>> Cc: Steven Price <steven.price at arm.com>
>> > >>>>> Cc: Yang Shi <shy828301 at gmail.com>
>> > >>>>> Signed-off-by: Barry Song <v-songbaohua at oppo.com>
>> > >>>>> ---
>> > >>>>> -v3:
>> > >>>>> * refine the commit log;
>> > >>>>> * add a benchmark result;
>> > >>>>> * refine the macro of arch_thp_swp_supported
>> > >>>>> Thanks to the comments of Anshuman, Andrew, Steven
>> > >>>>>
>> > >>>>> arch/arm64/Kconfig | 1 +
>> > >>>>> arch/arm64/include/asm/pgtable.h | 6 ++++++
>> > >>>>> include/linux/huge_mm.h | 12 ++++++++++++
>> > >>>>> mm/swap_slots.c | 2 +-
>> > >>>>> 4 files changed, 20 insertions(+), 1 deletion(-)
>> > >>>>>
>> > >>>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> > >>>>> index 1652a9800ebe..e1c540e80eec 100644
>> > >>>>> --- a/arch/arm64/Kconfig
>> > >>>>> +++ b/arch/arm64/Kconfig
>> > >>>>> @@ -101,6 +101,7 @@ config ARM64
>> > >>>>> select ARCH_WANT_HUGETLB_PAGE_OPTIMIZE_VMEMMAP
>> > >>>>> select ARCH_WANT_LD_ORPHAN_WARN
>> > >>>>> select ARCH_WANTS_NO_INSTR
>> > >>>>> + select ARCH_WANTS_THP_SWAP if ARM64_4K_PAGES
>> > >>>>> select ARCH_HAS_UBSAN_SANITIZE_ALL
>> > >>>>> select ARM_AMBA
>> > >>>>> select ARM_ARCH_TIMER
>> > >>>>> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
>> > >>>>> index 0b6632f18364..78d6f6014bfb 100644
>> > >>>>> --- a/arch/arm64/include/asm/pgtable.h
>> > >>>>> +++ b/arch/arm64/include/asm/pgtable.h
>> > >>>>> @@ -45,6 +45,12 @@
>> > >>>>> __flush_tlb_range(vma, addr, end, PUD_SIZE, false, 1)
>> > >>>>> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>> > >>>>>
>> > >>>>> +static inline bool arch_thp_swp_supported(void)
>> > >>>>> +{
>> > >>>>> + return !system_supports_mte();
>> > >>>>> +}
>> > >>>>> +#define arch_thp_swp_supported arch_thp_swp_supported
>> > >>>>> +
>> > >>>>> /*
>> > >>>>> * Outside of a few very special situations (e.g. hibernation), we always
>> > >>>>> * use broadcast TLB invalidation instructions, therefore a spurious page
>> > >>>>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>> > >>>>> index de29821231c9..4ddaf6ad73ef 100644
>> > >>>>> --- a/include/linux/huge_mm.h
>> > >>>>> +++ b/include/linux/huge_mm.h
>> > >>>>> @@ -461,4 +461,16 @@ static inline int split_folio_to_list(struct folio *folio,
>> > >>>>> return split_huge_page_to_list(&folio->page, list);
>> > >>>>> }
>> > >>>>>
>> > >>>>> +/*
>> > >>>>> + * archs that select ARCH_WANTS_THP_SWAP but don't support THP_SWP due to
>> > >>>>> + * limitations in the implementation like arm64 MTE can override this to
>> > >>>>> + * false
>> > >>>>> + */
>> > >>>>> +#ifndef arch_thp_swp_supported
>> > >>>>> +static inline bool arch_thp_swp_supported(void)
>> > >>>>> +{
>> > >>>>> + return true;
>> > >>>>> +}
>> > >>>>
>> > >>>> How about the following?
>> > >>>>
>> > >>>> static inline bool arch_wants_thp_swap(void)
>> > >>>> {
>> > >>>> return IS_ENABLED(ARCH_WANTS_THP_SWAP);
>> > >>>> }
>> > >>>
>> > >>> This looks good. then i'll need to change arm64 to
>> > >>>
>> > >>> +static inline bool arch_thp_swp_supported(void)
>> > >>> +{
>> > >>> + return IS_ENABLED(ARCH_WANTS_THP_SWAP) && !system_supports_mte();
>> > >>> +}
>> > >>
>> > >> Why ? CONFIG_THP_SWAP depends on ARCH_WANTS_THP_SWAP. In folio_alloc_swap(),
>> > >> IS_ENABLED(CONFIG_THP_SWAP) enabled, will also imply ARCH_WANTS_THP_SWAP too
>> > >> is enabled. Hence checking for ARCH_WANTS_THP_SWAP again does not make sense
>> > >> either in the generic fallback stub, or in arm64 platform override. Because
>> > >> without ARCH_WANTS_THP_SWAP enabled, arch_thp_swp_supported() should never
>> > >> be called in the first place.
>> > >
>> > > For the only caller now, the checking looks redundant. But the original
>> > > proposed implementation as follows,
>> > >
>> > > static inline bool arch_thp_swp_supported(void)
>> > > {
>> > > return true;
>> > > }
>> > >
>> > > will return true even on architectures that don't support/want THP swap.
>> >
>> > But the function will never be called on for those platforms.
>> >
>> > > That will confuse people too.
>> >
>> > I dont see how.
>> >
>> > >
>> > > And the "redundant" checking has no run time overhead, because compiler
>> > > will do the trick.
>> > I understand that, but dont think this indirection is necessary.
>>
>> Hi Anshuman, Hi Ying,
>> Thanks for the comments of both of you. Does the below look ok?
>>
>> generic,
>>
>> static inline bool arch_wants_thp_swap(void)
>> {
>> return IS_ENABLED(CONFIG_THP_SWAP);
>> }
>>
>
> sorry, i actually meant arch_thp_swp_supported() but not
> arch_wants_thp_swap() in generic code,
>
> static inline bool arch_thp_swp_supported(void)
> {
> return IS_ENABLED(CONFIG_THP_SWAP);
> }
IS_ENABLED(CONFIG_THP_SWAP) doesn't match the name too. It's an option
selected by users. arch_thp_swp_supported() is to report the
capability.
Best Regards,
Huang, Ying
>> arm64,
>>
>> static inline bool arch_thp_swp_supported(void)
>> {
>> return IS_ENABLED(CONFIG_THP_SWAP) && !system_supports_mte();
>> }
>>
>> caller,
>>
>> folio_alloc_swap(struct folio *folio)
>> {
>>
>> if (folio_test_large(folio)) {
>> - if (IS_ENABLED(CONFIG_THP_SWAP))
>> + if (arch_thp_swp_supported())
>> get_swap_pages(1, &entry, folio_nr_pages(folio));
>> goto out;
>> }
>>
>> Thanks
>> Barry
More information about the linux-arm-kernel
mailing list