[PATCH makedumpfile 0/9] Improvements to makedumpfile extensions, plus userspace stack tracing extension

[Stephen Brennan]

Hello all,

Building on Tao Liu's excellent work for makedumpfile extensions, I'd like to
share some incremental improvements to the extension system, as well as my own
two extensions which enable userspace stack tracing. There are three topics
related to the core makedumpfile extension system which are addressed here:

1. Tail pages handling. As patch 1 explains, extensions currently get called for
tail pages, but their decisions are not respected. I believe the best approach
is to only call extensions for the head page, but allow extensions to return a
decision related to just a subset of the pages. To this end, patch 7 adds
PG_INCLUDE_HEAD to include just the head page (leaving the rest up to the
default policy), and patch 9 shows the elfheader extension which uses that
functionality.

2. Counting pages included by extensions. This is done by patch 4. It alters the
logic because it's really helpful to know whether a page would have been
included by makedumpfile's dump-level filtering anyway, so we only count the
additional pages.

3. Sharing makedumpfile's logic and knowledge about page metadata. Currently,
extensions just get a "pcache" pointer, which means they must redo whatever
logic makedumpfile has already done, in order to get information like
compound_order. Patches 3, 5, and 6 factor out the page metadata into a new
struct pginfo, which gets passed to extensions, and it also makes more of the
helpers like "isSlab()" usable with this pointer.

With those improvements, the final two patches (8 & 9) implement a pair of
extensions that are useful for including a minimal amount of information which
can be used to do stack traces of the userspace processes in a vmcore. The
"userstack" extension includes the stack memory itself for each thread, and the
"elfheader" extension includes the first page of mapped ELF files, so that ELF
build IDs can be identified from the vmcore.

My testing so far:
- Live testing (/proc/kcore) x86_64 on Oracle UEK 8 (6.12.x based)
- Refiltering on vmcores generated from UEK8, 7 (5.15.x based), 6 (5.4.x based)
- Using the resulting filtered vmcores with drgn's contrib/pstack.py to generate
  userspace stack traces.

I still have significantly more testing to do and then share out:
- Testing in kexec environment
- Testing with upstream kernels, also verifying in a Fedora userspace
- Sharing runtime and size comparisons with different workloads

Despite the limited testing, I wanted to share these patches as they are,
because I think the improvements for the core extensions framework are useful
already, and don't require as much verification.

For the extensions themselves, I would like to consider merging them as well, if
there is interest. I think it would be useful for makedumpfile to contain shared
extensions, including those for the GPU buffers which Tao Liu was working on, as
well as the userspace stack tracing ones which I'm sharing.

Thanks,
Stephen

Stephen Brennan (9):
  Do not call extensions for tail pages
  Honor CFLAGS in extension/Makefile
  Share page information with extension callbacks
  Introduce a stat for pages retained by extension
  Move page checks into makedumpfile.h
  Simplify arguments for page checks
  Add PG_INCLUDE_HEAD extension return status
  Add userstack extension
  Add elfheader extension

 extension.c             |  10 +-
 extension.h             |  10 +-
 extensions/Makefile     |   8 +-
 extensions/elfheader.c  |  93 ++++++++++++
 extensions/userstack.c  | 325 ++++++++++++++++++++++++++++++++++++++++
 extensions/vma_mtree.c  | 140 +++++++++++++++++
 extensions/vma_mtree.h  |   7 +
 extensions/vma_rbtree.c |  56 +++++++
 extensions/vma_rbtree.h |  12 ++
 makedumpfile.c          | 188 +++++++++--------------
 makedumpfile.h          |  78 +++++++++-
 11 files changed, 798 insertions(+), 129 deletions(-)
 create mode 100644 extensions/elfheader.c
 create mode 100644 extensions/userstack.c
 create mode 100644 extensions/vma_mtree.c
 create mode 100644 extensions/vma_mtree.h
 create mode 100644 extensions/vma_rbtree.c
 create mode 100644 extensions/vma_rbtree.h

-- 
2.47.3