[PATCH v5 3/5] mm: LARGE_ANON_FOLIO for improved performance

[Matthew Wilcox]

On Thu, Aug 31, 2023 at 10:15:09AM -0700, Yang Shi wrote:
> On Thu, Aug 31, 2023 at 12:57 AM David Hildenbrand <david at redhat.com> wrote:
> > Let's talk about that in a bi-weekly MM session. (I proposed it as a
> > topic for next week).
> >
> > As raised in another mail, we can then discuss
> > * how we want to call this feature (transparent large pages? there is
> >    the concern that "THP" might confuse users. Maybe we can consider
> >    "large" the more generic version and "huge" only PMD-size, TBD)
> 
> I tend to agree. "Huge" means PMD-mappable (transparent or HugeTLB),
> "Large" means any order but less than PMD-mappable order, "Gigantic"
> means PUD mappable. This should incur the least confusion IMHO.

"Large" means any order > 0.  The limitation to <= PMD_ORDER is simply
because I don't want to go through the whole VM and fix all the places
that assume that pmd_page() returns a head page.  The benefit to doing so
is quite small, and the work to achieve it is quite large.  The amount of
work needed should decrease over time as we convert more code to folios,
so deferring it is the right decision today.

But nobody should have the impression that large folios are smaller
than PMD size, nor even less than or equal.  Just like they shouldn't
think that large folios depend on CONFIG_TRANSPARENT_HUGEPAGE.  They do
today, but that's purely an implementation detail that will be removed
eventually.

> > I think there *really* has to be a way to disable it for a running
> > system, otherwise no distro will dare pulling it in, even after we
> > figured out the other stuff.
> 
> TBH I really don't like to tie large folio to THP toggles. THP
> (PMD-mappable) is just a special case of LAF. The large folio should
> be tried whenever it is possible ideally. But I do agree we may not be
> able to achieve the ideal case at the time being, and also understand
> the concern about regression in early adoption, so a knob that can
> disable large folio may be needed for now. But it should be just a
> simple binary knob (on/off), and should not be a part of kernel ABI
> (temporary and debugging only) IMHO.

Best of luck trying to remove it after you've shipped it ... we've
never been able to remove any of the THP toggles, only make them more
complicated.

> One more thing we may discuss is whether huge page madvise APIs should
> take effect for large folio or not.

They already do for file large folios; we listen to MADV_HUGEPAGE and
attempt to allocate PMD_ORDER folios for faults.