Generate and Inspect LLVM Records with llvm-tblgen-18

TableGen files do nothing on their own until llvm-tblgen-18 parses them into records a backend can use. This covers parsing an LLVM TableGen file, inspecting the records it produces, and writing the machine-readable result to JSON. The examples use llvm-tblgen-18 from Ubuntu's llvm-18 package, version 18.1.3.

Allow about fifteen minutes. You need a shell, the installed llvm-18 executable, and a .td file with any files it includes. The examples below use the installed LLVM 18 headers, so they do not change a project checkout. No command needs sudo.

1. Confirm the installed tool

Start with read-only checks. The package installs the versioned executable as llvm-tblgen-18; the manpage describes the wider *-tblgen family, including llvm-tblgen, clang-tblgen, lldb-tblgen and mlir-tblgen.

$ command -v llvm-tblgen-18
/usr/bin/llvm-tblgen-18
$ llvm-tblgen-18 -version
Ubuntu LLVM version 18.1.3
  Optimized build.
$ dpkg-query -W -f='${Package} ${Version}\n' llvm-18
llvm-18 1:18.1.3-1ubuntu1

The exact package revision is distribution-specific. Record it alongside your build logs when generated files must be reproducible.

2. Choose the input and include path

TableGen reads a target description file, conventionally ending in .td. Pass the input path as the final positional argument, and use -I for directories containing files referenced by include statements.

$ TD=/usr/include/llvm-18/llvm/IR/Intrinsics.td
$ INCLUDE_DIR=/usr/include/llvm-18
$ test -r "$TD" && echo "input is readable"
input is readable
$ llvm-tblgen-18 -I "$INCLUDE_DIR" "$TD" -null-backend
$ printf 'exit status: %s\n' "$?"
exit status: 0

-null-backend parses the source and builds the records, then stops before a backend emits output. It is a handy first checkpoint when you are debugging an include path or timing the frontend on its own. A successful parse does not mean a particular generated C++ backend is correct.

3. Inspect the default records

With no backend selection, the default backend prints records to standard output, and that output can be huge, so send it to a temporary file or pipe it into a pager. The following command only reads the installed input:

$ llvm-tblgen-18 -I "$INCLUDE_DIR" "$TD" -o /tmp/intrinsics.records
$ head -n 12 /tmp/intrinsics.records
// Generated by llvm-tblgen
class Intrinsic<string name, list<LLVMType> ret_types, list<LLVMType> param_types> {
  string LLVMName = name;
  ...

The precise first records and their formatting depend on the installed TableGen source. Check the exit status and file size before treating the output as complete:

$ test -s /tmp/intrinsics.records && echo "records written"
records written
$ wc -l -c /tmp/intrinsics.records
<line-count> <byte-count> /tmp/intrinsics.records

Replace the angle-bracket placeholders in that last output with the numbers your machine actually prints. Do not paste generated records back into a source file as if they were the original declarations: they are expanded backend input, not a stable source format.

4. Emit JSON for automation

The general -dump-json backend emits every record as JSON. Give the output an explicit name rather than mixing it with diagnostics on standard output.

$ llvm-tblgen-18 -I "$INCLUDE_DIR" "$TD" \
    -dump-json -o /tmp/intrinsics.json
$ test -s /tmp/intrinsics.json && echo "JSON written"
JSON written
$ python3 -m json.tool /tmp/intrinsics.json > /dev/null
$ printf 'JSON syntax: valid\n'
JSON syntax: valid

The JSON is meant for further automated processing. Its schema reflects the records and the TableGen implementation, so pin the LLVM major version in anything that consumes it and test upgrades deliberately. Do not assume key ordering or a complete field list will stay identical across major releases.

5. Select a backend that answers a narrow question

Backends are selected by options. -print-enums prints enumeration values for a class, for example, and -class picks which class to inspect. This is most useful against a target-specific input that actually defines the relevant class:

$ llvm-tblgen-18 TARGET.td \
    -print-enums -class=CLASS_NAME \
    -I /path/to/project/include
<enumeration values printed by the installed input>

TARGET.td, CLASS_NAME and the include directory are placeholders, not files the package ships. Use a real class from the target's TableGen sources. Ask the installed tool for the exact backend list when working on a checkout from a different LLVM release:

$ llvm-tblgen-18 -help | less
$ llvm-tblgen-18 -help-list | less

-help-list gives a plain list, which is easier to search or capture in a script. Do not copy an option from documentation for a newer LLVM release without checking this output first.

6. Make generated files safe to update

TableGen happily overwrites the file named by -o. Shell redirection such as > generated.inc truncates its destination before TableGen even starts, so a parse error can leave an incomplete file behind. Use a temporary destination and swap it in only after validation.

$ out=/path/to/build/generated.inc
$ tmp="${out}.new"
$ llvm-tblgen-18 -I /path/to/project/include \
    /path/to/project/Target.td -gen-instr-info -o "$tmp"
$ test -s "$tmp"
$ mv -- "$tmp" "$out"
$ printf 'installed: %s\n' "$out"
installed: /path/to/build/generated.inc

This example changes a project build output, so check the path before running it. If TableGen fails, remove the incomplete .new file after inspecting the diagnostic and leave the previous generated file in place. -write-if-changed can cut down on needless rewrites, but it is not a substitute for this temporary-file pattern.

7. Diagnose the common failures

A missing input or include usually means the path is wrong, not that you need elevated privileges:

$ test -r /path/to/input.td || echo "input is not readable"
$ find /path/to/project -name '*.td' -type f | head
$ llvm-tblgen-18 -I /path/to/project/include /path/to/input.td -null-backend
$ printf 'exit status: %s\n' "$?"
exit status: <non-zero on error>

Use -debug for debug output, -time-phases for parser and backend timing, and -stats for collected backend statistics. These options explain a run; they do not fix a malformed declaration or a missing include.

Keep source and generated files from different LLVM major versions apart. A command can parse cleanly while producing output that no longer matches the headers or consumer code in a different checkout, so compare the executable's -version, the include tree and the build's expected LLVM major version before you go chasing generated C++.