Read CPU Usage with mpstat Without Misreading It

mpstat can quietly average away the one busy core that is actually causing your problem, unless you know to ask it per processor. This covers taking a baseline, watching a live interval, comparing individual CPUs, and exporting JSON, using mpstat from sysstat 12.6.1 on a Linux 6.8 host with eight online CPUs. Allow about ten minutes.

You need a shell and the sysstat package. These commands only read kernel statistics, so no sudo is needed and nothing gets changed.

1. Confirm the command and its data source

$ command -v mpstat
/usr/bin/mpstat
$ mpstat -V
sysstat version 12.6.1
(C) Sebastien Godard (sysstat <at> orange.fr)
$ test -r /proc/stat && echo /proc/stat-readable
/proc/stat-readable

mpstat reads statistics exposed through /proc, and its manual says that filesystem must be mounted. A missing or unreadable /proc/stat is a host problem, not a reason to reach for elevated privileges.

2. Take a baseline report

With no interval, mpstat reports CPU activity since boot, using the default CPU utilisation report:

$ mpstat
Linux 6.8.0-139-generic (server.example)  09/25/26  _x86_64_  (8 CPU)

04:03:00     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:03:00     all    6.10    0.14    1.78    2.84    0.00    0.07    0.00    0.00    0.00   89.07

• all is an average, not a CPU. It summarises every processor, not one of them.
• %usr is application time, %sys is kernel time.
• %iowait is idle time with disk I/O outstanding; %idle is idle time without it.

A single since-boot average can hide a spike that happened five minutes ago and has since faded into the noise.

3. Measure a live interval

Pass seconds as the first positional argument, and add a count when you want the command to stop:

$ mpstat 2 5
Linux 6.8.0-139-generic (server.example)  09/25/26  _x86_64_  (8 CPU)

04:04:00     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:04:02     all    3.20    0.00    1.10    0.40    0.00    0.03    0.00    0.00    0.00   95.27
Average:     all    3.20    0.00    1.10    0.40    0.00    0.03    0.00    0.00    0.00   95.27

That requests five reports, two seconds apart; your own numbers will differ. The final Average: row summarises the interval reports. Leave off the count, as in mpstat 2, and it keeps reporting until you press Ctrl+C, which has no recovery step to worry about since it is read-only.

4. Compare individual processors

Use -P ALL when one busy logical processor could be hiding inside the global average:

$ mpstat -P ALL 1 1
04:05:01     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
04:05:02     all   12.77    0.00    2.78    1.90    0.00    0.00    0.00    0.00    0.00   82.55
04:05:02       0    1.02    0.00    1.02    0.00    0.00    0.00    0.00    0.00    0.00   97.96
04:05:02       7   78.79    0.00   10.10    0.00    0.00    0.00    0.00    0.00   11.11

That is abbreviated: a real eight-CPU report lists every online processor. Compare the per-CPU rows against all before deciding the whole machine is overloaded.

• High %steal. Suggests a virtual machine waiting while its hypervisor runs another virtual CPU.
• High %iowait. Points towards I/O waiting, but is not proof storage is the only bottleneck.

To inspect selected processors, use a comma-separated list or a range:

$ mpstat -P 0,2-3 1 3

Processor numbering starts at zero, and offline processors are not shown. If a selected processor does not exist or is offline, fix the list rather than reading an empty result as a performance measurement.

5. Add topology or NUMA context

On a multi-socket or NUMA host, add topology columns:

$ mpstat -T -P ALL 1 1
04:06:01     CPU    CORE   SOCK   NODE    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle

The exact rows depend on the machine. -T adds logical core, socket and NUMA node columns; -n gives a node summary; -N ALL requests every NUMA node. Useful when a workload behaves differently after moving between sockets, but none of it explains memory bandwidth or process placement on its own.

6. Use JSON for collection

For a script or an incident record, ask for JSON instead of scraping aligned columns:

$ mpstat -o JSON 1 1
{"sysstat": {
        "hosts": [{
                "nodename": "server.example",
                "sysname": "Linux",
                "release": "6.8.0-139-generic",
                "machine": "x86_64",
                "number-of-cpus": 8,
                "statistics": [{"timestamp": "04:07:02", "cpu-load": [{"cpu": "all", "usr": 4.20, "idle": 94.10}]}]
        }]
}}

The real object has more fields and varies with the report you selected. The manpage says JSON field order is undefined and new fields may appear over time. Parse by field name, tolerate additions, and validate the captured document with your JSON parser before storing it: do not compare the raw text as if the layout were fixed.

7. Reduce noise and select other reports

For logs, turn off terminal colour explicitly:

$ S_COLORS=never mpstat 1 3

Colour is for terminal display; it does not mean a value is an error. --dec=0, --dec=1 and --dec=2 select zero, one or two decimal places, the default being two:

$ mpstat --dec=0 1 1

For interrupt rates, ask for a specific keyword rather than assuming the default CPU report includes them:

$ mpstat -I SUM 1 1
04:08:01     CPU    intr/s
04:08:02     all  1234.00

-I CPU reports individual hardware interrupts, -I SCPU reports software interrupts, and -I ALL asks for all three views. Output can be much larger than the summary, so start with SUM when a per-processor total is all you need.