[PATCH RFC 09/10] kfence, Documentation: add KFENCE documentation
Marco Elver
elver at google.com
Fri Sep 11 03:46:01 EDT 2020
On Fri, 11 Sep 2020 at 09:14, Dmitry Vyukov <dvyukov at google.com> wrote:
>
> On Mon, Sep 7, 2020 at 3:41 PM Marco Elver <elver at google.com> wrote:
> >
> > Add KFENCE documentation in dev-tools/kfence.rst, and add to index.
> >
> > Co-developed-by: Alexander Potapenko <glider at google.com>
> > Signed-off-by: Alexander Potapenko <glider at google.com>
> > Signed-off-by: Marco Elver <elver at google.com>
> > ---
> > Documentation/dev-tools/index.rst | 1 +
> > Documentation/dev-tools/kfence.rst | 285 +++++++++++++++++++++++++++++
> > 2 files changed, 286 insertions(+)
> > create mode 100644 Documentation/dev-tools/kfence.rst
> >
> > diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst
> > index f7809c7b1ba9..1b1cf4f5c9d9 100644
> > --- a/Documentation/dev-tools/index.rst
> > +++ b/Documentation/dev-tools/index.rst
> > @@ -22,6 +22,7 @@ whole; patches welcome!
> > ubsan
> > kmemleak
> > kcsan
> > + kfence
> > gdb-kernel-debugging
> > kgdb
> > kselftest
> > diff --git a/Documentation/dev-tools/kfence.rst b/Documentation/dev-tools/kfence.rst
> > new file mode 100644
> > index 000000000000..254f4f089104
> > --- /dev/null
> > +++ b/Documentation/dev-tools/kfence.rst
> > @@ -0,0 +1,285 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +Kernel Electric-Fence (KFENCE)
> > +==============================
> > +
> > +Kernel Electric-Fence (KFENCE) is a low-overhead sampling-based memory safety
> > +error detector. KFENCE detects heap out-of-bounds access, use-after-free, and
> > +invalid-free errors.
> > +
> > +KFENCE is designed to be enabled in production kernels, and has near zero
> > +performance overhead. Compared to KASAN, KFENCE trades performance for
> > +precision. The main motivation behind KFENCE's design, is that with enough
> > +total uptime KFENCE will detect bugs in code paths not typically exercised by
> > +non-production test workloads. One way to quickly achieve a large enough total
> > +uptime is when the tool is deployed across a large fleet of machines.
> > +
> > +Usage
> > +-----
> > +
> > +To enable KFENCE, configure the kernel with::
> > +
> > + CONFIG_KFENCE=y
> > +
> > +KFENCE provides several other configuration options to customize behaviour (see
> > +the respective help text in ``lib/Kconfig.kfence`` for more info).
> > +
> > +Tuning performance
> > +~~~~~~~~~~~~~~~~~~
> > +
> > +The most important parameter is KFENCE's sample interval, which can be set via
> > +the kernel boot parameter ``kfence.sample_interval`` in milliseconds. The
> > +sample interval determines the frequency with which heap allocations will be
> > +guarded by KFENCE. The default is configurable via the Kconfig option
> > +``CONFIG_KFENCE_SAMPLE_INTERVAL``. Setting ``kfence.sample_interval=0``
> > +disables KFENCE.
> > +
> > +With the Kconfig option ``CONFIG_KFENCE_NUM_OBJECTS`` (default 255), the number
> > +of available guarded objects can be controlled. Each object requires 2 pages,
> > +one for the object itself and the other one used as a guard page; object pages
> > +are interleaved with guard pages, and every object page is therefore surrounded
> > +by two guard pages.
> > +
> > +The total memory dedicated to the KFENCE memory pool can be computed as::
> > +
> > + ( #objects + 1 ) * 2 * PAGE_SIZE
> > +
> > +Using the default config, and assuming a page size of 4 KiB, results in
> > +dedicating 2 MiB to the KFENCE memory pool.
> > +
> > +Error reports
> > +~~~~~~~~~~~~~
> > +
> > +A typical out-of-bounds access looks like this::
> > +
> > + ==================================================================
> > + BUG: KFENCE: out-of-bounds in test_out_of_bounds_read+0xa3/0x22b
> > +
> > + Out-of-bounds access at 0xffffffffb672efff (left of kfence-#17):
> > + test_out_of_bounds_read+0xa3/0x22b
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + kfence-#17 [0xffffffffb672f000-0xffffffffb672f01f, size=32, cache=kmalloc-32] allocated in:
> > + __kfence_alloc+0x42d/0x4c0
> > + __kmalloc+0x133/0x200
> > + test_alloc+0xf3/0x25b
> > + test_out_of_bounds_read+0x98/0x22b
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + CPU: 4 PID: 107 Comm: kunit_try_catch Not tainted 5.8.0-rc6+ #7
> > + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014
> > + ==================================================================
> > +
> > +The header of the report provides a short summary of the function involved in
> > +the access. It is followed by more detailed information about the access and
> > +its origin.
> > +
> > +Use-after-free accesses are reported as::
> > +
> > + ==================================================================
> > + BUG: KFENCE: use-after-free in test_use_after_free_read+0xb3/0x143
> > +
> > + Use-after-free access at 0xffffffffb673dfe0:
> > + test_use_after_free_read+0xb3/0x143
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + kfence-#24 [0xffffffffb673dfe0-0xffffffffb673dfff, size=32, cache=kmalloc-32] allocated in:
> > + __kfence_alloc+0x277/0x4c0
> > + __kmalloc+0x133/0x200
> > + test_alloc+0xf3/0x25b
> > + test_use_after_free_read+0x76/0x143
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
>
> Empty line between stacks for consistency and readability.
Done for v2.
> > + freed in:
> > + kfence_guarded_free+0x158/0x380
> > + __kfence_free+0x38/0xc0
> > + test_use_after_free_read+0xa8/0x143
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + CPU: 4 PID: 109 Comm: kunit_try_catch Tainted: G W 5.8.0-rc6+ #7
> > + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014
> > + ==================================================================
> > +
> > +KFENCE also reports on invalid frees, such as double-frees::
> > +
> > + ==================================================================
> > + BUG: KFENCE: invalid free in test_double_free+0xdc/0x171
> > +
> > + Invalid free of 0xffffffffb6741000:
> > + test_double_free+0xdc/0x171
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + kfence-#26 [0xffffffffb6741000-0xffffffffb674101f, size=32, cache=kmalloc-32] allocated in:
> > + __kfence_alloc+0x42d/0x4c0
> > + __kmalloc+0x133/0x200
> > + test_alloc+0xf3/0x25b
> > + test_double_free+0x76/0x171
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > + freed in:
> > + kfence_guarded_free+0x158/0x380
> > + __kfence_free+0x38/0xc0
> > + test_double_free+0xa8/0x171
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + CPU: 4 PID: 111 Comm: kunit_try_catch Tainted: G W 5.8.0-rc6+ #7
> > + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014
> > + ==================================================================
> > +
> > +KFENCE also uses pattern-based redzones on the other side of an object's guard
> > +page, to detect out-of-bounds writes on the unprotected side of the object.
> > +These are reported on frees::
> > +
> > + ==================================================================
> > + BUG: KFENCE: memory corruption in test_kmalloc_aligned_oob_write+0xef/0x184
> > +
> > + Detected corrupted memory at 0xffffffffb6797ff9 [ 0xac . . . . . . ]:
> > + test_kmalloc_aligned_oob_write+0xef/0x184
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + kfence-#69 [0xffffffffb6797fb0-0xffffffffb6797ff8, size=73, cache=kmalloc-96] allocated in:
> > + __kfence_alloc+0x277/0x4c0
> > + __kmalloc+0x133/0x200
> > + test_alloc+0xf3/0x25b
> > + test_kmalloc_aligned_oob_write+0x57/0x184
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + CPU: 4 PID: 120 Comm: kunit_try_catch Tainted: G W 5.8.0-rc6+ #7
> > + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014
> > + ==================================================================
> > +
> > +For such errors, the address where the corruption as well as the corrupt bytes
> > +are shown.
> > +
> > +And finally, KFENCE may also report on invalid accesses to any protected page
> > +where it was not possible to determine an associated object, e.g. if adjacent
> > +object pages had not yet been allocated::
> > +
> > + ==================================================================
> > + BUG: KFENCE: invalid access in test_invalid_access+0x26/0xe0
> > +
> > + Invalid access at 0xffffffffb670b00a:
> > + test_invalid_access+0x26/0xe0
> > + kunit_try_run_case+0x51/0x85
> > + kunit_generic_run_threadfn_adapter+0x16/0x30
> > + kthread+0x137/0x160
> > + ret_from_fork+0x22/0x30
> > +
> > + CPU: 4 PID: 124 Comm: kunit_try_catch Tainted: G W 5.8.0-rc6+ #7
> > + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014
> > + ==================================================================
> > +
> > +DebugFS interface
> > +~~~~~~~~~~~~~~~~~
> > +
> > +Some debugging information is exposed via debugfs:
> > +
> > +* The file ``/sys/kernel/debug/kfence/stats`` provides runtime statistics.
> > +
> > +* The file ``/sys/kernel/debug/kfence/objects`` provides a list of objects
> > + allocated via KFENCE, including those already freed but protected.
> > +
> > +Implementation Details
> > +----------------------
> > +
> > +Guarded allocations are set up based on the sample interval. After expiration
> > +of the sample interval, a guarded allocation from the KFENCE object pool is
> > +returned to the main allocator (SLAB or SLUB). At this point, the timer is
> > +reset, and the next allocation is set up after the expiration of the interval.
> > +To "gate" a KFENCE allocation through the main allocator's fast-path without
> > +overhead, KFENCE relies on static branches via the static keys infrastructure.
> > +The static branch is toggled to redirect the allocation to KFENCE.
> > +
> > +KFENCE objects each reside on a dedicated page, at either the left or right
>
> Do we mention anywhere explicitly that KFENCE currently only supports
> allocations <=page_size?
> May be worth mentioning. It kinda follows from implementation but
> quite implicitly. One may also be confused assuming KFENCE handles
> larger allocations, but then not being able to figure out.
We can add a note that "allocation sizes up to PAGE_SIZE are supported".
Thanks,
-- Marco
More information about the linux-arm-kernel
mailing list