Read and Verify a trace-cmd Version 6 Trace File

trace-cmd.dat.v6 is the on-disk layout trace-cmd wrote before format version 7 came along. This guide has you capture a short trace.dat, check its binary header for the version 6 marker, and read its metadata without ever opening it in an editor.

Allow about 15 minutes. You need the installed trace-cmd command, a shell, and permission to access the kernel tracing interface. The examples use trace-cmd package version 3.2-1ubuntu2, whose executable reports 3.2.0 on this machine. The format is a storage layout, not a config file you hand-edit.

1. Check the installed command

Confirm which executable will run and which subcommands it provides:

$ command -v trace-cmd
/usr/bin/trace-cmd
$ trace-cmd --version
trace-cmd version 3.2.0 (not-a-git-repo)

The version string describes the program, not necessarily the format version of every file it can read. The file itself carries its own version marker, so check both when you are investigating an old capture.

Checkpoint: if command -v finds nothing, stop here and install the package through your normal system-management process. Do not copy a random executable into a system directory.

2. Record a deliberately short trace

Enable the sched_switch event while a one-second sleep runs, and write the capture to /tmp/trace-v6.dat:

$ sudo trace-cmd record -e sched_switch -o /tmp/trace-v6.dat -- sleep 1

This normally needs sudo because kernel tracing access is restricted on most systems. The capture itself can contain process names, command lines, timing information and kernel symbol data, so treat it as sensitive before you share it. It changes tracing state only while it runs; that is not a service restart, but it does add a little tracing overhead. If the event is unavailable or tracing is blocked by policy, the command fails rather than producing a useless file.

Check the result as an ordinary user:

$ test -s /tmp/trace-v6.dat && printf '%s\n' 'capture exists and is non-empty'
capture exists and is non-empty
$ file /tmp/trace-v6.dat
/tmp/trace-v6.dat: data

Your file output may say less. What matters is that the path exists and has a non-zero size. Nothing here touches your other input files. Once the investigation is over, remove this temporary capture with rm -- /tmp/trace-v6.dat if you no longer need it.

3. Verify the version 6 magic and header

The first bytes defined by trace-cmd.dat.v6(5) are a fixed magic value, the ASCII word tracing, and a null-terminated version string. Display the first 20 bytes in hex and printable form:

$ od -An -tx1z -N20 /tmp/trace-v6.dat
 17 08 44 74 72 61 63 69 6e 67 00 36 00 ...

The exact spacing and the bytes after the version depend on the header, but the sequence you are looking for is:

Do not infer the version from the filename. A file can be called anything other than trace.dat, and the name carries no authority over what is actually inside it.

4. Read the machine-dependent fields

Right after the version string, the header records the file byte order and the number of bytes used for a userspace long on the target machine. A flag of 0 means little endian and 1 means big endian; the long-size byte is 4 for 32-bit values or 8 for 64-bit ones.

That long size belongs to the target userspace, not the kernel, and the distinction matters when a trace moves between machines. Every number after this point in the file, including the 32-bit target page size that follows, uses the declared byte order.

Do not decode later integers by assuming the reader machine's native layout. Use trace-cmd to read the file, or use the header's endian flag if you are writing a specialised parser.

5. Inspect metadata with trace-cmd

Use the installed reader for normal investigation rather than a hex dump:

$ trace-cmd dump -i /tmp/trace-v6.dat --summary
$ trace-cmd dump -i /tmp/trace-v6.dat --systems
$ trace-cmd dump -i /tmp/trace-v6.dat --flyrecord

dump can report the stored header page and event header, recorded systems and events, kallsyms data, saved command lines, options, and the per-CPU flyrecord offsets. All of that is safer and less error-prone than slicing the binary with guessed offsets. For a straight validation check, ask trace-cmd to validate the file:

$ trace-cmd dump -i /tmp/trace-v6.dat -v

Checkpoint: a successful run returns to the prompt without an error. If it reports a malformed file, preserve the original and investigate a copy; do not overwrite the only copy while you experiment with a parser.

6. Read the recorded events

The usual human-facing reader is trace-cmd report:

$ trace-cmd report -i /tmp/trace-v6.dat | sed -n '1,20p'

The output depends on the events, kernel and available plugins, and it should contain trace records if the one-second workload generated any. Use trace-cmd report -i /tmp/trace-v6.dat --cpus to list CPUs with recorded content, or --version to show the trace-cmd version saved in the record, when that metadata is present.

Tip: the header carries event format text copied from the target's tracing directories, kallsyms data from /proc/kallsyms when recorded, printk format strings when present, and saved PID-to-command mappings. That embedded context is why a capture can stay useful long after the original machine changes, and also why it may disclose more than you expect.

7. Know what the final header says

After the metadata, the format stores the number of CPUs with matching trace data, plus one of three markers: options, latency or flyrecord. A latency file ends with ASCII text taken from the target tracing output. A flyrecord file instead stores, for each recorded CPU, a 64-bit file offset and a 64-bit data size.

CPU data is page-aligned to the target page size and copied straight from the Ftrace ring buffer. Readers may map it page by page, falling back to ordinary reads when mapping is not possible. Treat that as an implementation detail to respect when parsing, never a reason to edit padding or move the data around.

Done means