Reduce an LLVM 20 Failure with bugpoint-20

Your LLVM 20 build crashes or miscompiles, and the reproducer is thousands of lines long. bugpoint-20 chops it down until only the trigger is left. You will finish with a smaller LLVM IR test case that still shows an optimiser crash, a miscompilation, or a bad native-code result. This guide uses the bugpoint-20 shipped by llvm-20, reported here as LLVM 20.1.8.

Allow 15 minutes for a small reduction and considerably longer for a large or intermittent failure. Run it as an ordinary user in a disposable directory. No step here needs sudo.

Before you start

bugpoint is a reducer and test orchestrator, not a general debugger. It repeatedly runs your LLVM input and the selected passes or code generator, removing pieces while checking whether the same failure remains. It can investigate optimiser crashes, optimiser miscompilations and incorrect native code generation.

Write down three things before you run it:

A crash is usually easy to recognise. A miscompilation needs a trustworthy reference output, and a custom compile check needs a script with a deliberate exit status.

Tip: if the failure is intermittent, make the reproducer deterministic first.

$ command -v bugpoint-20
/usr/bin/bugpoint-20
$ bugpoint-20 --version
Ubuntu LLVM version 20.1.8
  Optimized build.

1. Work on a copy of the reproducer

Make a private workspace and copy the input into it. The reducer creates temporary files and may leave reduced artefacts behind. Keeping those files away from your source tree makes it clear what changed and gives you an easy recovery path: remove the workspace when you have saved the useful result.

$ mkdir -p "$HOME/bugpoint-work"
$ cp /path/to/reproducer.ll "$HOME/bugpoint-work/case.ll"
$ cd "$HOME/bugpoint-work"
$ cp case.ll case.ll.before-bugpoint
$ sha256sum case.ll case.ll.before-bugpoint

Tip: the second copy is your undo point. Do not overwrite the original reproducer while experimenting, particularly if the test executes generated code or loads a plugin.

2. Start with the smallest useful command

Give bugpoint-20 the input, then name any LLVM passes involved in the failure. The execution mode tells it how to run the reduced program:

For an interpreter or JIT failure, a starting shape is:

$ bugpoint-20 -run-jit case.ll opt
Read input file      : 'case.ll'
*** All input ok

Replace opt with the pass or tool that actually participates in your failure. The command is expected to take time and can run the test repeatedly. A successful exit status means bugpoint found a problem, not that the original program was correct; a non-zero status means it hit an error or could not establish the failure.

Tip: do not treat the first output line as proof of a reduction. Wait for the final diagnostic and inspect the files bugpoint leaves in the workspace. If it cannot choose a safe execution tool, select the run mode explicitly and provide a reference output with -output where appropriate.

3. Pass program arguments without losing them

Arguments after -args are passed to the test program whenever it runs. The separator immediately after -args matters when a program argument begins with a hyphen, because it stops bugpoint reading that argument as its own option.

$ bugpoint-20 -run-jit case.ll -args -- --mode reduced --input sample.dat

There are separate forwarding options for the LLVM tool under test and the safe tool. Use the same separator rule for -tool-args, and keep tool options distinct from test-program options:

$ bugpoint-20 -run-llc case.ll -tool-args -- -mtriple=x86_64-pc-linux-gnu

Tip: a common slip is putting -mtriple after -args, which changes the program's input instead of the compiler invocation. Confirm which process should receive every option before launching a long reduction.

4. Make output comparison explicit

For a miscompilation, use -output to name a known-good reference output. Without it, bugpoint tries to create a reference using a safe backend and running the program. That automatic choice can fail, or be unsuitable when the failure concerns the reference backend itself.

$ ./known-good-runner > expected.txt
$ bugpoint-20 -run-llc --safe-run-llc -output expected.txt case.ll opt

The safe backend is the comparison path, not the suspected path. This LLVM 20.1.8 binary names the safe choices --safe-run-llc and --safe-run-custom; it does not permit the interpreter or JIT as a safe backend. If exit status is part of the failure, add -append-exit-code=true so different exit codes count as different output.

$ bugpoint-20 -run-llc --safe-run-llc \
    -output expected.txt -append-exit-code=true case.ll opt

5. Use a custom compile check for compile-only failures

If the failure is in generated assembly or diagnostics and the output must not be linked or executed, use -compile-command. The command must exit non-zero when the input is interesting and zero when it is not.

Warning: that polarity is easy to reverse. Test the script on both a known failing and a known passing input before handing it to the reducer.

#!/bin/sh
set -eu
llc "$@" -o reduced.s
if FileCheck checks.ll < reduced.s; then
    exit 1
fi
exit 0

Save this as check-compile.sh, make it executable, and run it through bugpoint:

$ chmod u+x check-compile.sh
$ ./check-compile.sh case.ll; echo "check status: $?"
$ bugpoint-20 -compile-command ./check-compile.sh case.ll

The example marks an input interesting when FileCheck succeeds, so it deliberately returns failure in that case. Adapt the check to your real bug.

Security warning: do not use a script that deletes source files, changes system configuration or launches untrusted output with elevated privileges.

6. Inspect, verify and recover

When bugpoint reports success, identify the reduced .ll or .bc file it names, then rerun the original failing command against that file. Repeat the test several times if the issue is intermittent. Also check that the reduced input still has the expected output, exit status or compiler diagnostic, rather than merely crashing for an unrelated reason.

$ find . -maxdepth 1 -type f -printf '%f\n' | sort
$ opt-20 -S reduced.bc -o /dev/null
$ sha256sum case.ll.before-bugpoint reduced.bc

Use -verbose-errors=true when a generic <crash> is not enough to prove the same failure remains. Keep -disable-dce or -disable-simplifycfg for the narrow case where the pass being investigated is itself one of bugpoint's default reduction passes. Those switches can make the result larger and the run slower.

Recovery: stop the run with your terminal interrupt if it is consuming too much time, then restore the untouched reproducer from case.ll.before-bugpoint. Removing the disposable workspace is the clean undo action:

$ cd ..
$ rm -rf "$HOME/bugpoint-work"

Destructive action: only run that removal after checking the path. It cannot recover an unsaved reduced case.

Done means