Build and Run a Small lit Test Suite with LLVM 20

Reading about lit is one thing; running one from nothing teaches you far more in fifteen minutes than another manual page. This guide builds a minimal LLVM lit suite, runs it, selects one test, and turns a failure into something readable enough to actually fix. Allow about 15 minutes if Python 3 and the LLVM 20 tools are already installed. Everything happens in a temporary directory, and none of it needs root.

There is one packaging wrinkle worth knowing about upfront: the lit-20(1) manual ships with the llvm-20 package, but no lit-20 command sits on the normal PATH. The runnable entry point is /usr/lib/llvm-20/build/utils/lit/lit.py. Set a shell variable once and use it for the rest of this guide.

1. Check the installed entry point

Confirm the package version and the executable before building any examples on top of them. The package version and lit's own reported version are two different pieces of information, so note both when you file a test result:

$ dpkg-query -W -f='${Package} ${Version}\n' llvm-20 llvm-20-tools
llvm-20 1:20.1.8~++20250804090239+87f0227cb601-1~exp1~20250804210352.139
llvm-20-tools 1:20.1.8~++20250804090239+87f0227cb601-1~exp1~20250804210352.139
$ LIT20=/usr/lib/llvm-20/build/utils/lit/lit.py
$ test -f "$LIT20" && python3 "$LIT20" --version
lit 20.1.0dev

Your package revision or version string may read differently. If the file check fails, stop and locate the lit entry point your own distribution supplies rather than guessing a path or bolting on another copy.

2. Create a test suite marker

lit won't treat just any directory as a test suite. It searches upwards from each input looking for lit.cfg or lit.site.cfg, which names the suite, picks a test format, and tells lit where the source and execution roots live.

Build a disposable suite under /tmp using the shell-test format, where commands live in comments starting with RUN::

$ mkdir -p /tmp/lit20-guide/suite
$ cd /tmp/lit20-guide/suite
$ touch lit.cfg pass.sh
$ chmod +x pass.sh

Edit lit.cfg so it contains:

import lit.formats

config.name = "lit20-guide"
config.test_format = lit.formats.ShTest(not lit_config.useValgrind)
config.suffixes = [".sh"]
config.test_source_root = "/tmp/lit20-guide/suite"
config.test_exec_root = "/tmp/lit20-guide/suite"

Put this single line in pass.sh:

# RUN: true

The absolute paths keep this smoke test predictable. In a real project, derive them from the build layout rather than copying these temporary ones.

3. Discover and run the test

Ask lit what it discovered before you run anything. This catches a missing config file, a wrong suffix, or a path outside the suite before it wastes your time:

$ python3 "$LIT20" --show-suites /tmp/lit20-guide/suite
-- Test Suites --
  lit20-guide - 1 tests
    Source Root: /tmp/lit20-guide/suite
    Exec Root  : /tmp/lit20-guide/suite
$ python3 "$LIT20" --show-tests /tmp/lit20-guide/suite
-- Available Tests --
  lit20-guide :: pass.sh

Then run the suite in succinct mode:

$ python3 "$LIT20" -q /tmp/lit20-guide/suite

Total Discovered Tests: 1
$ printf 'exit status: %s\n' "$?"
exit status: 0

The default worker count comes from the available CPUs, so even a one-test run pulls in the normal concurrency machinery. Use -j 1 if you want a single worker while chasing order-dependent behaviour.

4. Select a subset without editing the suite

For a larger suite, pass a test file directly or use a regular-expression filter, which matches against test names including the suite name and relative path. This runs only the test named:

$ python3 "$LIT20" --filter='pass\.sh' /tmp/lit20-guide/suite

-- Testing: 1 tests, 1 workers --
PASS: lit20-guide :: pass.sh (1 of 1)

Testing Time: 0.22s

Total Discovered Tests: 1
  Passed: 1 (100.00%)

Quote the regular expression in a script: an unquoted one is fair game for shell expansion, and a broad expression can select far more than you intended. --filter-out=REGEXP removes matching tests instead of selecting them.

If filtering leaves nothing, lit treats that as an error by default. Add --allow-empty-runs only when an empty selection is genuinely expected in your workflow: it hides a useful signal the rest of the time.

5. Make a failure worth looking at

Change the test command to a deliberate failure:

# RUN: false

Then run it verbosely:

$ python3 "$LIT20" -v /tmp/lit20-guide/suite
FAIL: lit20-guide :: pass.sh (1 of 1)
******************** TEST 'lit20-guide :: pass.sh' FAILED ********************
Test 'lit20-guide :: pass.sh' failed as a result of exit code 1.
********************
Total Discovered Tests: 1

Timing and progress text vary run to run; the facts that matter are the test name, the failing command, and the non-zero exit status. With -v, lit prints each command before it runs and adds a RUN: at line N marker connecting the failure to its source line. A run containing FAIL or XPASS exits 1.

Restore # RUN: true before you carry on. If the test wrote files or touched a shared build directory, remove only those known temporary outputs after checking the path. This is not a guide for reaching for a broad recursive delete in a real build tree.

6. Control time, order and reports

--timeout=N caps each individual test and defaults to 0, meaning no per-test limit. --max-time=N caps the approximate duration of the whole run, and --max-tests=N caps its test count. Pick the one matching the failure you're actually containing.

The default smart order runs previously failed tests first, then favours longer ones. Use --order=lexical for a repeatable path order, or --order=random to flush out order dependencies. Timing data lives in .lit_test_times.txt under the execution root: treat it as generated state, not a result.

For CI or another tool, write a machine-readable report to an explicit destination:

$ python3 "$LIT20" --xunit-xml-output=/tmp/lit20-guide/results.xml /tmp/lit20-guide/suite
$ test -s /tmp/lit20-guide/results.xml && echo report-written
report-written

Don't overwrite a report another job is still reading. Give concurrent runs separate output paths.

7. Clean up the temporary example

This guide has only ever touched files below /tmp/lit20-guide. Once you've seen the output, remove that exact directory if you don't need it any more. Warning: this removal is irreversible, so verify the path first:

$ test "/tmp/lit20-guide" = /tmp/lit20-guide && test -d /tmp/lit20-guide
$ rm -rf -- /tmp/lit20-guide

Do not adapt that final command to a source or build directory. If you'd rather keep the suite, skip the removal and swap the absolute paths for ones that fit the real project.