Run CMake Tests Reliably with ctest

You will run the tests registered in a CMake build directory, narrow a failure to one test, and capture enough output to diagnose it. The examples use the ctest 3.28.3 installed with CMake on this machine. Allow about 10 minutes for a first pass, plus the time needed by the test suite itself.

Before you start

You need a configured CMake build tree containing CTestTestfile.cmake, and permission to execute its tests. You do not normally need root. Do not run ctest from the source directory unless that directory is also the intended build directory: if you omit --test-dir, ctest looks in the current directory.

Change the placeholder below to the absolute or relative path of your build tree. The command only runs tests; it does not compile the project.

BUILD_DIR=/path/to/project/build
ctest --test-dir "$BUILD_DIR"

1. Check what will run

Start with a dry listing. -N, also written --show-only, disables execution and lists the tests selected by the other options.

ctest --test-dir "$BUILD_DIR" --show-only

Use the list to catch the most common error: a valid build directory that belongs to a different checkout or configuration. To print the labels without running tests, use:

ctest --test-dir "$BUILD_DIR" --print-labels

Checkpoint: Continue only when the listed names and labels belong to the build you meant to test. An empty list usually means the tree is not configured for testing, or that you pointed ctest at the wrong directory.

2. Run the complete suite

Run the ordinary suite and read its final summary.

ctest --test-dir "$BUILD_DIR"

CTest normally suppresses output from passing tests. A passing test suite ends with a summary showing the number of tests passed. A failing test makes the command return a non-zero status, which makes this suitable for shell scripts and CI.

If your project has more than one configuration in the same build tree, select it explicitly. Debug and Release are common names, but use the configuration your generator created.

ctest --test-dir "$BUILD_DIR" --build-config Debug

3. Make failures useful

When a test fails, repeat it with its own output. --output-on-failure keeps successful tests quiet and prints output from a failing test.

ctest --test-dir "$BUILD_DIR" --output-on-failure

For a deeper investigation, -V shows all test output and -VV asks for even more. Use these when the test itself does not explain the failure.

ctest --test-dir "$BUILD_DIR" -V

Run one test by matching its name with a regular expression. The match is not a literal string, so characters such as ., + and [ have regular-expression meanings.

ctest --test-dir "$BUILD_DIR" --tests-regex '^parser-basic$' --output-on-failure

First test a selector with --show-only if the expression is unfamiliar. An expression that matches nothing can look like a successful run unless you notice that no tests were selected.

4. Select tests by labels

Labels are useful when names are inconsistent. -L includes tests whose labels match a regular expression; -LE excludes matching labels.

ctest --test-dir "$BUILD_DIR" --label-regex '^unit$' --output-on-failure
ctest --test-dir "$BUILD_DIR" --label-exclude 'slow|network'

Multiple -L options are an AND: each expression must match at least one label on the test. To express alternatives, use one expression such as unit|integration. A test with no labels is never selected by -L, and is never excluded by -LE.

Name and index filters can be combined. By default, ctest runs the intersection. Add -U when you explicitly want the union of an -R selection and an -I selection.

5. Use parallel execution carefully

Once the serial run is trustworthy, add a modest job count.

ctest --test-dir "$BUILD_DIR" --parallel 4 --output-on-failure

--parallel, or -j, limits how many tests ctest runs concurrently. It does not make tests safe to run together: tests that share a port, directory, database or other external state can still interfere. Start with a low value and raise it only when the project declares its dependencies correctly.

If the machine becomes overloaded, --test-load 6.0 asks ctest not to start tests when doing so may push the CPU load above that threshold. This is a scheduling hint, not a strict performance guarantee.

For a suspected ordering dependency, --schedule-random randomises scheduling. Reproduce any failure with a serial run and verbose output before changing the test suite.

6. Repeat flaky tests and save evidence

To look for an intermittent failure, require a test to pass repeatedly:

ctest --test-dir "$BUILD_DIR" --tests-regex '^network-login$' --repeat until-fail:20 --output-on-failure

until-pass:10 instead retries failures up to ten times, while after-timeout:10 retries only timeouts. These modes can hide a real reliability problem if you use them as a CI success rule, so keep the attempt output and investigate a flaky test rather than treating a retry as proof of correctness.

Write the complete ctest output to a file with --output-log. The file is overwritten if it already exists, so choose a disposable or uniquely named path.

ctest --test-dir "$BUILD_DIR" --output-on-failure --output-log /tmp/ctest-run.log
sed -n '1,200p' /tmp/ctest-run.log

JUnit output is also available with --output-junit. It overwrites the destination if it exists:

ctest --test-dir "$BUILD_DIR" --output-junit /tmp/ctest-results.xml

Warning: Do not put secrets in command arguments or test output. CTest logs the output you ask it to capture, and test programs may print environment values or tokens.

7. Recover from a partial run

If a run was interrupted, -F enables ctest failover so it can resume the previously interrupted test-set execution. If the earlier run completed, -F has no effect.

ctest --test-dir "$BUILD_DIR" -F --output-on-failure

For the simpler case where the run completed and some tests failed, use --rerun-failed. It deliberately ignores other test-selection options such as -L, -R and -I.

ctest --test-dir "$BUILD_DIR" --rerun-failed --output-on-failure

If no tests are found, CTest 3.28 can be told whether that is an error or an intentional empty suite:

ctest --test-dir "$BUILD_DIR" --no-tests=error

Use this in automation when silently passing an unconfigured or incomplete build would be dangerous.

Useful checks without running tests

Confirm the installed program and ask it for its built-in option list:

ctest --version
ctest --help

For CMake terminology, ctest --help <keyword> prints help for a property, variable, command, policy, generator or module. The installed 3.28 manpage records that this broader keyword form changed in CMake 3.28; older installations may accept command names only.

Done means