Generate Reproducible LLVM IR Test Cases with llvm-stress-20

You will create a small, repeatable LLVM IR test case with llvm-stress-20, save it as a .ll file, and check that LLVM can assemble it. Allow about ten minutes if LLVM 20 is already installed. The commands are unprivileged and write only to the directory you choose.

This guide targets the installed Ubuntu package llvm-20, version 20.1.8. The command generates random-looking IR for testing LLVM components. It is a test-input generator, not a compiler for source code and not a performance benchmark.

1. Check the installed version and options

Start by confirming which executable is being used:

$ command -v llvm-stress-20
/usr/bin/llvm-stress-20
$ llvm-stress-20 --version
Ubuntu LLVM version 20.1.8
  Optimized build.

The current binary describes --size as an estimated number of instructions in the generated function. The installed manual page uses older wording, calling it the size of the generated file. For this package, use the binary's help text and treat the value as an approximate instruction count. The output length will vary with the generated instructions.

$ llvm-stress-20 --help
USAGE: llvm-stress-20 [options]
  -o <filename>             - Override output filename
  --seed=<uint>             - Seed used for randomness
  --size=<uint>             - Estimated size of the generated function

Checkpoint: if command -v finds a different executable, stop and check that its version and option syntax match the examples before continuing.

2. Generate a small test case to standard output

Use a fixed seed while developing or debugging a test. The short option spellings documented by the local manual work with this installed binary:

$ llvm-stress-20 -size=8 -seed=123 > sample.ll
$ printf 'status=%s, bytes=' "$?"
status=0, bytes=514

Without -o, the generated textual IR goes to standard output, so shell redirection is the natural way to save it. The exact byte count and instructions depend on the LLVM build and requested size. Inspect the beginning of the file without editing it:

$ head -n 8 sample.ll
; ModuleID = '/tmp/autogen.bc'
source_filename = "/tmp/autogen.bc"

define void @autogen_SD123(ptr %0, ptr %1, ptr %2, i32 %3, i64 %4, i8 %5) {
BB:
  %A4 = alloca i32, align 4

The module identifier is generated metadata and is not a promise about where your file is stored. Do not use the sample function name as a stable interface.

3. Save output with the program's output option

For a script or test fixture, -o keeps the generated file separate from diagnostic output:

$ llvm-stress-20 -size=16 -seed=123 -o generated.ll
$ test -s generated.ll && head -n 3 generated.ll
; ModuleID = '/tmp/autogen.bc'
source_filename = "/tmp/autogen.bc"

Choose a new path before running the command. Both shell redirection and -o can replace an existing file. If the fixture matters, preserve it first:

$ cp --preserve=all generated.ll generated.ll.bak
$ llvm-stress-20 -size=16 -seed=124 -o generated.new.ll
$ mv generated.new.ll generated.ll

The final mv changes the fixture only after generation succeeds. If generation fails, leave the original in place and inspect the error. To undo this example, restore the backup with mv generated.ll.bak generated.ll after checking that it is the intended version.

4. Make the test case repeatable

A seed controls the random instruction choices. Reusing the same seed, size and LLVM build gives a useful reproduction starting point:

$ llvm-stress-20 -size=16 -seed=123 -o run-a.ll
$ llvm-stress-20 -size=16 -seed=123 -o run-b.ll
$ sha256sum run-a.ll run-b.ll
3e6bc64614db2d75d3e2ed732f74646f8442cd1b57b76f734d3de7f4637da51d  run-a.ll
3e6bc64614db2d75d3e2ed732f74646f8442cd1b57b76f734d3de7f4637da51d  run-b.ll

Record the command, seed, size and LLVM version with a bug report. A different seed should normally produce different IR. Reproducibility is not a cross-version guarantee: upgrading LLVM can change the generator even when the seed is unchanged.

5. Validate the generated IR

A zero exit status from llvm-stress-20 means generation completed. It does not replace a parser check. If the matching assembler is installed, assemble the file into bitcode:

$ llvm-as-20 generated.ll -o generated.bc
$ printf 'status=%s, bytes=' "$?"
status=0, bytes=1636

The assembler reads the text and writes a separate bitcode file. A successful result is a practical check that the generated file is valid LLVM IR for this toolchain. Keep generated.ll when investigating a failure because it is the readable reproduction. If llvm-as-20 is not installed, use the LLVM assembler from the same toolchain rather than assuming another version has identical parsing behaviour.

6. Diagnose the common failures

If the output file is missing, check the parent directory and write permission without using elevated privileges:

$ ls -ld .
$ test -w . && echo writable
$ llvm-stress-20 -size=8 -o ./generated.ll

In the installed build, an output path whose parent does not exist produces status 1 and the message No such file or directory. Create the intended directory first, or correct the path. Do not run the generator with sudo to hide a path or ownership mistake.

A size of zero is accepted by this version and still returns status 0, so do not use success alone to decide whether a useful fixture was produced. Check that the file is non-empty and assemble it. Conversely, a non-zero assembler status means the next debugging step is to inspect the generated .ll and the exact tool versions, not to overwrite it with a fresh random case.

The optional --enable-scalable-vectors switch asks this LLVM 20 binary to include scalable vector types. Use it only when the component under test needs that IR feature, and validate the result with the matching assembler. It is not needed for an ordinary scalar stress case.