Measure Kernel Memory Allocation with perf kmem

perf kmem records kernel memory allocation events so you can inspect slab or page allocator statistics for a real workload. The examples use the installed perf-kmem(1) manual from linux-tools-common version 6.8.0-142.142. Allow 15 to 30 minutes, plus the time needed to reproduce the workload you want to measure.

This guide measures a workload. It does not tune the kernel, change allocator settings or delete a data file. Recording kernel tracepoints can require elevated privileges, so start with the checks below as an ordinary user and use sudo only when the recorder reports that access is insufficient.

1. Check the installed perf tool

First establish which executable your shell will run and which package version is installed:

$ command -v perf
/usr/bin/perf
$ dpkg-query -W -f='${Package} ${Version}\n' linux-tools-common
linux-tools-common 6.8.0-142.142

The package version is not necessarily the running kernel version. This machine runs kernel 6.8.0-139, while linux-tools-common is 6.8.0-142.142. On this host the wrapper reports that matching kernel-specific perf tools are missing, so a real recording cannot be verified here. Install the normal matching tools package through your operating system's package process before relying on the later examples.

2. Record one bounded workload

Choose a short, representative command. The recorder writes the perf data file while the command runs, so a five-second sleep is a harmless smoke test but not a useful workload by itself:

$ sudo perf kmem record -o /tmp/perf-kmem.data -- /usr/bin/sleep 5

• record is the first variant of perf kmem: it captures kmem events from an arbitrary command. The options after record are passed to perf record, which is why -o selects the output file. The explicit path keeps this test separate from any existing default perf.data in the current directory.
• May need root. Kernel tracepoints and perf events are restricted by the host's security policy. If the unprivileged command works, keep it unprivileged. If it fails, read the error before adding sudo: a missing tracepoint or mismatched tool is not fixed by more privilege.

$ test -s /tmp/perf-kmem.data && echo 'recording file is non-empty'
recording file is non-empty
$ ls -lh /tmp/perf-kmem.data
-rw------- 1 root root  ... /tmp/perf-kmem.data

The exact size and ownership vary. Because sudo created the file, you may need elevated privilege to inspect or remove it. Do not overwrite an existing capture accidentally: choose a new output path, or move a valuable capture aside before recording.

3. Read the default allocation statistics

Use stat to analyse the capture. Tell it which file to read with -i:

$ sudo perf kmem -i /tmp/perf-kmem.data stat

The default input for stat is perf.data, unless standard input is a FIFO. Making the input explicit prevents an easy mistake: reading an older capture in the working directory and attributing its numbers to the latest test.

The default sort keys differ by allocator. Slab output uses frag,hit,bytes; page output uses bytes,hit. The report is a summary of the events in the file, not a live view of current memory use. A successful command should print a report rather than alter the capture.

$ sudo test -r /tmp/perf-kmem.data && echo readable
readable

4. Separate slab and page analysis

Select the allocator before stat. For slab allocations, show the busiest callsites first:

$ sudo perf kmem --slab --sort=bytes,hit stat -i /tmp/perf-kmem.data

For page allocator events, sort by allocated bytes and then hit count:

$ sudo perf kmem --page --sort=bytes,hit stat -i /tmp/perf-kmem.data

• Slab sort keys: ptr, callsite, bytes, hit, pingpong, frag.
• Page sort keys: page, callsite, bytes, hit, order, migtype, gfp.
• Order matters: put a mode option such as --slab or --page before --sort, because the sort vocabulary depends on that choice.

To focus on allocation callsites, request the per-callsite view:

$ sudo perf kmem --caller stat -i /tmp/perf-kmem.data

Use --alloc for per-allocation statistics. Limit a large report with --line=20, and add --verbose when symbol addresses or other extra detail are needed. These are analysis choices; they do not change the recorded data.

5. Measure live pages or a time window

Page analysis normally reports total allocation statistics. Add --live when you need pages still allocated at the end of the captured activity:

$ sudo perf kmem --page --live stat -i /tmp/perf-kmem.data

--live only works with page analysis. It is not a general switch for making slab output current.

You can restrict analysis to a time interval using seconds and microseconds:

$ sudo perf kmem --page --time=12.000000,18.500000 stat -i /tmp/perf-kmem.data

Use an empty start or stop when the interval should run to the beginning or end of the file. For example, --time=,18.500000 starts at the beginning, while --time=12.000000, runs to the end. These times refer to the capture's timestamps, not wall-clock dates.

Common traps

• Empty or surprisingly small report. Can mean the workload did not exercise the selected allocator, the capture window was too short, or the kernel and perf build do not expose the expected events. Repeat the workload with a longer bounded interval and inspect both --slab and --page rather than changing kernel settings blindly.
• Recording fails with permission errors. Check the host's perf restrictions and security policy with the administrator.
• Recording fails because the event is unavailable. Install the perf package matching the running kernel. The local wrapper's warning names the expected packages, such as linux-tools-6.8.0-139-generic and linux-cloud-tools-6.8.0-139-generic; package availability depends on the distribution repositories. Do not claim a report is valid merely because a file was created.

$ sudo rm -- /tmp/perf-kmem.data

This removal is irreversible. There is no undo command; rerun the recording step to create a new capture.