Reduce a Failing LLVM IR Test Case with llvm-reduce-18

llvm-reduce-18 shrinks a crashing LLVM IR file down to the smallest version that still triggers the bug. It works by repeatedly stripping parts out and keeping each cut only when your own test script says the remaining file is still interesting. This guide uses the llvm-reduce-18 supplied by Ubuntu's llvm-18 package, version 18.1.3.

Allow roughly 10 minutes for a small example and considerably longer for a large or expensive reproducer. You need a reproducible IR or MIR input, a command that exposes the failure, and a POSIX shell script that returns success when the failure is present. No root access is needed. Work in a disposable directory and keep the original input unchanged.

1. Confirm the tool and preserve the input

Check that the versioned executable is the one you expect:

llvm-reduce-18 --version
dpkg-query -W -f='${Package} ${Version}\n' llvm-18

On the system used for this guide, the first command reports Ubuntu LLVM version 18.1.3 and the package is llvm-18 1:18.1.3-1ubuntu1.

• Copy before you experiment. Make a working copy of the failing input before touching anything.
• Skip --in-place on the first run. The manpage warns that it replaces the input file.

mkdir -p /tmp/llvm-reduce-work
cp /path/to/failing-test.ll /tmp/llvm-reduce-work/input.ll
cd /tmp/llvm-reduce-work

Safety boundary: The reducer executes the interestingness test many times. Treat the input and every command used by the script as untrusted test data. Use a dedicated temporary directory, avoid scripts that delete files or change services, and never put secrets in test arguments or the input.

2. Make the interestingness test exact

The test receives the candidate file as its first argument. It must exit with status zero when the candidate still demonstrates the bug, and non-zero otherwise. For a compiler crash, run the same compiler command you already use and match a stable error marker:

#!/bin/sh
set -eu

/usr/bin/opt-18 -O2 -disable-output "$1" 2>&1 \
  | grep -F 'Assertion failed at line 1234 of WhateverFile.cpp'

Save this as interesting.sh and make it executable:

chmod 755 interesting.sh
./interesting.sh input.ll
echo "test exit status: $?"

The expected status is zero for the original reproducer. If the test prints an error from opt-18 but still returns zero because of a pipeline mistake, fix the script before reducing anything. grep earns its place at the end of the pipeline here because its zero status is the signal you actually want. Match a distinctive, stable symptom rather than a line number that changes between builds.

3. Run a first reduction

Give the test script and input to llvm-reduce-18, and write the result to a new file:

llvm-reduce-18 \
  --test=./interesting.sh \
  input.ll \
  -o reduced.ll

The command prints progress for reduction passes and finishes with a line like:

Done reducing! Reduced testcase: reduced.ll

A zero exit status means normal operation, not proof the output is actually useful. Confirm both the test result and the output file:

./interesting.sh reduced.ll
llvm-as-18 reduced.ll -o /tmp/llvm-reduce-work/reduced.bc
wc -l input.ll reduced.ll

Checkpoint: the first command above should still return zero. The llvm-as-18 check catches a malformed textual IR file, and wc gives you a quick, visible size comparison. Reduction is not guaranteed to produce the smallest possible file, and a flaky test can cause unrelated or excessive removals.

4. Control speed and scope

By default, LLVM runs all available delta passes and may repeat the full set up to five times. Start with the default when correctness matters more than speed. For a quick, repeatable diagnostic, restrict the work:

llvm-reduce-18 \
  --test=./interesting.sh \
  --delta-passes=function-bodies \
  --max-pass-iterations=1 \
  -j 1 \
  input.ll \
  -o reduced-pass.ll

• -j 1 disables parallelism. That makes test output easier to follow and cuts load, but it does not make an unstable test stable.
• --print-delta-passes lists pass names before you pick one.
• --skip-delta-passes=pass1,pass2 drops a pass that does not suit this input.

These narrower runs are useful for diagnosis, but a final reduction normally deserves a broader pass set. For machine-code IR, select the input language explicitly with -x=mir; for ordinary textual LLVM IR, use -x=ir. The tool can also read standard input when the input argument is -, but a named file is easier to audit and rerun.

Common traps and recovery

• The original test is not interesting: run the script directly and repair the reproducer first. LLVM reports an error rather than producing a meaningful reduction.
• The reduced file no longer fails: rerun the exact script, then check whether the matched diagnostic was actually produced by the intended tool. Tighten the marker or remove flaky timing and environment checks.
• The run is too slow: use -j, fewer pass iterations, or selected delta passes for an exploratory run, then restore a broader run before treating the result as final.
• The original file was replaced: this only happens when --in-place was used. Restore it from version control or your untouched copy. With the commands above, discard a bad result with rm reduced.ll; the source remains input.ll.

Warning: do not use --in-place unless you have a verified backup and genuinely want the replacement. The option changes the input file, while an ordinary output path keeps the before-and-after files around for review.

Done means

• Version confirmed: llvm-reduce-18 --version identifies the expected LLVM 18 installation.
• Original preserved: the input still exists and the interestingness script returns zero for it.
• Reduction completed: the reducer exits normally and records a reduced output file.
• Bug still present: the script returns zero for the reduced file too.
• Output still parses: llvm-as-18 accepts the reduced IR, when the assembler is available.
• Method documented: you can explain which pass settings and diagnostic marker produced the result.