Read CPU Usage with mpstat Without Misreading Its Averages

You will finish with a short, repeatable way to inspect CPU utilisation, compare logical processors, collect a fixed number of samples, and hand machine-readable output to another tool. The examples use mpstat from sysstat 12.6.1, installed on a Linux 6.8 host with eight online CPUs.

Allow about ten minutes. You need a shell and the sysstat package. These commands only read kernel statistics, so they do not need sudo and they do not change services or system configuration.

1. Confirm the command and its data source

Check the binary, package version, and the kernel statistics filesystem before interpreting a report:

$ command -v mpstat
/usr/bin/mpstat
$ mpstat -V
sysstat version 12.6.1
(C) Sebastien Godard (sysstat <at> orange.fr)
$ test -r /proc/stat && echo /proc/stat-readable
/proc/stat-readable

mpstat reads statistics exposed through /proc. Its manual says the /proc filesystem must be mounted. A missing or unreadable /proc/stat is therefore a host problem, not a reason to add elevated privileges.

Checkpoint: if command -v finds nothing, install sysstat through your normal distribution process. Do not copy a binary from an unrelated host just to make the example run.

2. Take a baseline report

With no interval, mpstat reports CPU activity since boot. With no activity option, the default is the CPU utilisation report:

$ mpstat
Linux 6.8.0-139-generic (server.example)  09/25/26  _x86_64_  (8 CPU)

04:03:00     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:03:00     all    6.10    0.14    1.78    2.84    0.00    0.07    0.00    0.00    0.00   89.07

The all row is an average across the processors, not a separate CPU. The percentages describe how CPU time was classified: for example, %usr is application time, %sys is kernel time, %iowait is idle time while disk I/O was outstanding, and %idle is idle time without that outstanding request. A single since-boot average can hide a short spike that happened five minutes ago.

3. Measure a live interval

Pass seconds as the first positional argument. Add a count when you want the command to stop:

$ mpstat 2 5
Linux 6.8.0-139-generic (server.example)  09/25/26  _x86_64_  (8 CPU)

04:04:00     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:04:02     all    3.20    0.00    1.10    0.40    0.00    0.03    0.00    0.00    0.00   95.27
Average:     all    3.20    0.00    1.10    0.40    0.00    0.03    0.00    0.00    0.00   95.27

This requests five reports two seconds apart. The displayed values will differ because they describe your host. The final Average: row summarises the interval reports. If you omit the count, for example mpstat 2, reports continue until you press Ctrl+C. Stopping that read-only command has no recovery action because it changes nothing.

Do not use mpstat 0 5 expecting five instantaneous reports. The manual defines zero, or no interval, as the since-boot report, and a count is only paired with a non-zero interval.

4. Compare individual processors

Use -P ALL when one busy logical processor could be hidden by the global average:

$ mpstat -P ALL 1 1
04:05:01     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:05:02     all   12.77    0.00    2.78    1.90    0.00    0.00    0.00    0.00    0.00   82.55
04:05:02       0    1.02    0.00    1.02    0.00    0.00    0.00    0.00    0.00    0.00   97.96
04:05:02       7   78.79    0.00   10.10    0.00    0.00    0.00    0.00    0.00   11.11

The example is abbreviated: a real eight-CPU report includes every online processor. Compare the per-CPU rows with all before deciding that the whole machine is overloaded. A high %steal suggests a virtual machine is waiting while its hypervisor runs another virtual CPU; a high %iowait points towards waiting on I/O, but it is not proof that storage is the only bottleneck.

To inspect selected processors, use a comma-separated list or ranges, such as:

$ mpstat -P 0,2-3 1 3

Processor numbering starts at zero. Offline processors are not displayed. If the selected processor does not exist or is offline, correct the list rather than treating an empty result as a performance measurement.

5. Add topology or NUMA context

On a multi-socket or NUMA host, add topology columns to the CPU report:

$ mpstat -T -P ALL 1 1
04:06:01     CPU    CORE   SOCK   NODE    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle

The exact rows depend on the machine. -T adds logical core, socket and NUMA node information. Use -n for a node summary, and -N ALL to request all NUMA nodes. These options are useful when a workload behaves differently after moving between sockets, but they do not explain memory bandwidth or process placement by themselves.

6. Use JSON for collection

For a script or an incident record, request JSON instead of scraping aligned columns:

$ mpstat -o JSON 1 1
{"sysstat": {
        "hosts": [{
                "nodename": "server.example",
                "sysname": "Linux",
                "release": "6.8.0-139-generic",
                "machine": "x86_64",
                "number-of-cpus": 8,
                "statistics": [{"timestamp": "04:07:02", "cpu-load": [{"cpu": "all", "usr": 4.20, "idle": 94.10}]}]
        }]
}}

The real object contains more fields and varies with the selected report. The manpage says JSON field order is undefined and new fields may be added. Parse by field name, tolerate additions, and do not compare the raw text as if it had a stable layout. Validate the captured document with your JSON parser before storing it.

7. Reduce noise and select other reports

For logs, disable terminal colour explicitly:

$ S_COLORS=never mpstat 1 3

Colour is intended for terminal display and does not indicate that a value is an error. The --dec=0, --dec=1 and --dec=2 options select zero, one or two decimal places. The default is two:

$ mpstat --dec=0 1 1

For interrupt rates, use a specific keyword rather than assuming the default CPU report includes them:

$ mpstat -I SUM 1 1
04:08:01     CPU    intr/s
04:08:02     all  1234.00

-I CPU reports individual hardware interrupts, -I SCPU reports software interrupts, and -I ALL requests all three interrupt views. The output can be much larger than the summary, so start with SUM when you only need a per-processor total.