Profile Arm Memory Latency with perf and SPE

perf can drive Arm's Statistical Profiling Extension (SPE) to record which instructions caused slow loads, stores and branches, addresses and all. By the end you will have a perf.data recording, a report of the decoded samples, and a way to inspect memory access details.

Allow 10 to 20 minutes for a first run. You need an Arm CPU with SPE support, a kernel with the ARM_SPE_PMU option enabled, and a workload you can run repeatedly. The examples use the perf interface supplied by the linux-tools-common package. The machine used for this guide is not suitable for a live capture: it is x86_64, and its installed perf reports that matching kernel tools are missing, a useful reminder to check the target host first.

Checkpoint 1: confirm the target

Run these ordinary, read-only checks on the Arm host:

uname -m
perf --version
ls -ld /sys/bus/event_source/devices/arm_spe

You want an Arm machine, a working perf binary, and an arm_spe PMU directory. The local manpage is dated 1 September 2026. The installed wrapper here identifies itself as perf but warns that perf is not installed for kernel 6.8.0-139, so a version-looking command alone does not prove recording will work.

Checkpoint 2: record a small baseline

Start with the simplest capture. Replace ./mybench with a real executable and its arguments:

perf record -e arm_spe// -- ./mybench

This writes raw SPE data to perf.data in the current directory. Recording normally needs access to the performance monitoring interface; if the system policy rejects the command, ask an administrator about the permitted perf_event_paranoid setting, or run with the required privilege. Use sudo only when your host policy requires it, and remember a root-created output file may not be writable by your normal account.

For a first capture, raise the sample period rather than collecting as aggressively as possible:

perf record -c 100000 -e arm_spe// -- ./mybench

The manpage says the period is programmed as the SPE interval, and recommends a higher value because the hardware-derived minimum applies when no interval is specified. The useful value depends on the CPU and workload: a large output file is a practical sign you need a higher interval. SPE is statistical, so a short run or a sparse workload can legitimately produce little data.

Checkpoint 3: narrow the records

SPE configuration parameters go between the two slashes, separated by commas. Capture loads with at least 10 units of latency like this:

perf record -c 100000 \
  -e arm_spe/load_filter=1,min_latency=10/ -- ./mybench

min_latency is the total latency measured from the point sampling started on the instruction, not merely the time spent executing the load. Other useful filters: store_filter=1 for stores, branch_filter=1 for branches. These filters change what is retained; they do not turn SPE into a complete trace.

To retain only selected events, use the event mask the manpage documents. Bit 1 is instruction retired, bit 3 an L1 data-cache refill, bit 5 a TLB refill, bit 7 a mispredict, and bit 11 a misaligned access. For retired instructions:

perf record -c 100000 \
  -e arm_spe/event_filter=2/ -- ./mybench

For mispredicted branches, the corresponding example is:

perf record -c 100000 \
  -e arm_spe/event_filter=0x80/ -- ./mybench

Leave jitter=1 enabled unless you have a reason to remove the pseudo-random interval perturbation. It helps avoid resonance between a regular workload and a regular sampling interval.

Checkpoint 4: decode and inspect

Recording does not decode SPE packets. Decoding happens when you open the file with perf report or perf script. First request one instruction sample per decoded instruction, with no further downsampling:

perf report --itrace=i1i

The report may show groups such as arm_spe//, dummy:u, l1d-miss, tlb-access and memory. The first two are implementation details and expected to be empty. The other groups are not necessarily unique samples: one instruction can carry several associated events, so do not add the group counts as though they were independent observations.

To inspect load and store information attached to samples, use:

perf report --mem-mode

For a script-friendly view, use perf script on the same file. If you need to preserve the raw output while experimenting with reports, copy perf.data to a separate working directory first. The report commands are read-only; deleting the file is the only action in this guide that would remove captured evidence, so keep it until the result is recorded elsewhere.

Understand the limits before acting on a result

SPE samples include a program counter, timing, PMU events, and for loads and stores, data addresses, cache information and data origin. That gives precise attribution without tracing every operation, but no call-graph information. Results stay statistical and should be compared across repeatable runs.

Some implementations sample micro-operations rather than architectural instructions, so an instruction expanded into two micro-operations is twice as likely to enter the sample population. The manpage points to the sample_pop and inst_retired PMU events for estimating this coarse effect. Data-source meanings are also implementation defined, because cache layouts differ between processors.

Common traps

• Cannot find PMU 'arm_spe'. Check the kernel driver, module loading, KPTI requirement, CPU architecture and whether a virtual machine hides SPE.
• Arm SPE CONTEXT packets not found. Root privilege is needed for context packets, which improve PID assignment for kernel samples. For a user-space-only profile, the manpage says this warning can be ignored.
• Too few samples. Run the workload longer or use a lower period, while watching output size and collision counts.
• Too many collisions. A new sample drops when the previous sampled operation has not finished. The sample_collision PMU event is a rough guide, but it counts collisions before filtering, not an exact filtered-sample loss count.
• Very large perf.data. Increase the sampling interval and repeat the capture. A larger file does not mean a more accurate profile.