Compare LLVM Optimisation Remarks with llvm-remarkutil-20

llvm-remarkutil-20 turns a compiler's optimisation remarks into CSV instruction counts, remark tallies, and a size comparison between two builds. The examples use LLVM 20.1.8 on Debian or Ubuntu, and they do not modify a project, install a package or need root.

Allow about 20 minutes. You need the llvm-20 tools and a compiler capable of writing optimisation remarks. The command reads remark files produced by a build; it does not generate those files itself. Keep the two input files tied to the same source and target when measuring a code change.

1. Check the installed tool

Confirm the binary and package version before relying on an example. This is an ordinary read-only check:

$ command -v llvm-remarkutil-20
/usr/bin/llvm-remarkutil-20
$ llvm-remarkutil-20 --version
Ubuntu LLVM version 20.1.8
  Optimized build.
$ dpkg-query -W -f='${Package} ${Version}\n' llvm-20
llvm-20 1:20.1.8~++20250804090239+87f0227cb601-1~exp1~20250804210352.139

2. Produce a remark file during compilation

For a C or C++ build, ask Clang to save optimisation remarks. The output name is explicit, so it is easier to find than a compiler-generated default:

$ clang-20 -O2 -g -fsave-optimization-record \
    -foptimization-record-file=build/sample.opt.yaml \
    -c src/sample.c -o build/sample.o

For instruction counts, the remark file must contain remarks from the assembly printer. A practical test build can request them with -Rpass-missed=asm-printer:

$ clang-20 -O2 -g -Rpass-missed=asm-printer \
    -fsave-optimization-record \
    -foptimization-record-file=build/sample.opt.yaml \
    -c src/sample.c -o build/sample.o
$ test -s build/sample.opt.yaml && echo "remark file is present"
remark file is present

The compiler options affect what gets recorded. If instruction-count later reports no functions, inspect the YAML and rebuild with assembly-printer remarks enabled. Do not treat an empty report as proof the program has no instructions.

3. Convert between YAML and bitstream

Use yaml2bitstream when a consumer needs LLVM's compact bitstream representation:

$ llvm-remarkutil-20 yaml2bitstream \
    build/sample.opt.yaml -o build/sample.opt.bitstream
$ test -s build/sample.opt.bitstream && echo "bitstream is present"
bitstream is present

Convert it back when a human needs a text inspection or a tool expects YAML:

$ llvm-remarkutil-20 bitstream2yaml \
    build/sample.opt.bitstream -o build/sample.roundtrip.yaml
$ head -n 4 build/sample.roundtrip.yaml
--- !Passed
Pass:            inline
Name:            Inlined

4. Extract instruction counts

Read a YAML remark file and write a CSV table. The parser is required and must match the input format:

$ llvm-remarkutil-20 instruction-count \
    --parser=yaml build/sample.opt.yaml -o build/instruction-count.csv
$ cat build/instruction-count.csv
Function,InstructionCount
add,2
main,2

The exact functions and numbers depend on the source, target and optimisation settings. Add --use-debug-loc when the source location is useful:

$ llvm-remarkutil-20 instruction-count \
    --parser=yaml --use-debug-loc build/sample.opt.yaml \
    -o build/instruction-count-with-locations.csv

For a bitstream file, change only the parser value and input path to --parser=bitstream. A common trap is assuming the command counts every machine instruction in an object file. It counts the instruction-count remarks present in the remark stream, which need assembly-printer remarks to exist at all.

5. Count remarks by group or property

The count subcommand can count all remarks, group them by function or source, or count numeric remark arguments. Start with a total count:

$ llvm-remarkutil-20 count \
    --parser=yaml --count-by=remark-name --group-by=total \
    build/sample.opt.yaml
Total,Count
Total,7

To see counts per function, change the group:

$ llvm-remarkutil-20 count \
    --parser=yaml --count-by=remark-name --group-by=function \
    build/sample.opt.yaml
Function,Count

The rows depend on the remarks in your file. --group-by=source needs debug locations. --count-by=arg needs --args or --rargs, and the selected argument values must be numeric. Use --remark-name, --pass-name or their regular-expression forms to narrow the result.

6. Compare two builds

Keep two remark files from the same source compiled with different optimisation settings or compiler versions. Put the older file first, then the newer file:

$ llvm-remarkutil-20 size-diff \
    --parser=yaml build/old.opt.yaml build/new.opt.yaml
add, > add, 1 instrs, 0 stack B
instruction count: 1 (50%)
stack byte usage: 0 (0%)

The human report shows functions added, removed or present in both files, followed by aggregate instruction and stack-byte changes. The exact lines vary with the inputs. In the comparison above, the second file has one more instruction for add.

For automation, request JSON and optionally pretty-print it:

$ llvm-remarkutil-20 size-diff \
    --parser=yaml --report_style=json --pretty \
    build/old.opt.yaml build/new.opt.yaml > build/size-diff.json
$ test -s build/size-diff.json && echo "JSON report is present"
JSON report is present

In JSON, InBoth, OnlyInA and OnlyInB identify function membership. The InstCount and StackSize arrays hold values for file A and file B; calculate B minus A in your reporting code. A size increase is a measurement to investigate, not automatically a regression: target, debug settings and compiler changes can all affect it.

7. Diagnose the usual failures

• Unknown parser or unreadable input: check the file with file and match --parser=yaml or --parser=bitstream to its format.
• No instruction rows: inspect the remark stream for Pass: asm-printer and Name: InstructionCount. Rebuild with assembly-printer remarks if they are absent.
• Unexpectedly large differences: verify both files came from the same source, target architecture and comparable compiler flags. Comparing unrelated builds produces numbers without a useful baseline.
• A command fails after an output path was used: check whether the output is partial before consuming it. Rerun to a fresh temporary path, then replace a final report only after it has passed validation.

No elevated privilege is needed for any command in this workflow. Do not use sudo to read or convert a project-local remark file. If the inputs contain sensitive source paths or optimisation details, protect the files using the normal permissions for your build directory before sharing reports.