Run Fail2Ban's Unit Tests with fail2ban-testcases

You will finish with a repeatable way to run the Fail2Ban test battery installed on your Linux host, reduce environmental noise, select tests when needed, and interpret a non-zero result without mistaking it for a service failure. The examples use fail2ban-testcases 1.0.2 from package version 1.0.2-3ubuntu0.1.

Allow about ten minutes for a normal run and a little longer if you need to investigate failures. You need a shell and the installed fail2ban package. The examples run the test command as your normal user. Do not begin with sudo: elevated privileges can hide permission and environment problems, and the test runner is not a replacement for administering the Fail2Ban service.

1. Confirm the installed runner

Check the executable and version before relying on option names. These are read-only commands and do not need elevated privileges:

$ command -v fail2ban-testcases
/usr/bin/fail2ban-testcases
$ fail2ban-testcases --version
fail2ban-testcases 1.0.2
$ dpkg-query -W -f='${Package} ${Version}\n' fail2ban
fail2ban 1.0.2-3ubuntu0.1

The package version and the program version agree on this machine. If your output differs, keep the installed manual page and help output beside you while comparing results. Test names, timing and failure details can change between releases.

2. Read the option contract

Ask the installed command for its help rather than guessing at switches:

$ fail2ban-testcases --help

The runner accepts optional regular-expression filters after its options. The useful controls for a first run are --no-network or -n, --no-gamin or -g, --memory-db or -m, and --fast or -f. Verbosity can be increased with repeated -v or set with --verbosity=0 through --verbosity=4.

Do not confuse -i with an ordinary filter. The manual describes it as negating the regular-expression filter, so it ignores tests matched by the expressions that follow. Use it only when you have a reason to exclude a known group.

3. Run a conservative full check

For a host-level smoke check, disable tests that need network access or gamin, use an in-memory database, and reduce wait intervals:

$ fail2ban-testcases --no-network --no-gamin --memory-db --fast
Fail2ban 1.0.2 test suite. Python 3.12.3 ... Please wait...
... test progress ...
Ran 516 tests in 5.703s
FAILED (failures=6, errors=8, skipped=23)

The final lines above are an example from this machine, not a promised result. A clean run ends with a successful test summary and status 0. The runner may print progress characters, logging output or a different count. Capture the output when the result matters:

$ fail2ban-testcases --no-network --no-gamin --memory-db --fast > fail2ban-testcases.log 2>&1
$ status=$?
$ printf 'test-run status: %s\n' "$status"
test-run status: 0

The shell variable must be saved immediately after the test command. Running printf first would replace the status you are trying to record. The log file is ordinary working data; remove it after review if it contains nothing you need to retain.

4. Read a failing run before changing anything

A non-zero status means that one or more tests failed or errored. It does not automatically mean the Fail2Ban daemon is broken, and it does not authorise changing jails, filters, firewall rules or the persistent database.

Inspect the end of the captured log first:

$ tail -n 60 fail2ban-testcases.log
$ rg -n '^(FAIL|ERROR)|^FAILED|^Ran ' fail2ban-testcases.log

Separate a test failure from an infrastructure symptom. On this installed host, the conservative run returned status 1 because several tests encountered host-specific conditions, including a missing test log file, configuration interpolation errors and a database-repair expectation that did not match the observed repair result. The suite still reported how many tests ran. Record the exact exception, skipped count and environment before deciding whether a package or test setup needs work.

Some tests create temporary files under /tmp. Do not delete a path mentioned by a failing test while the runner is still active. After the process has exited, remove only a temporary file you have identified as belonging to this run. Never remove /var/lib/fail2ban, a jail database or a log file as a shortcut for making tests green.

5. Narrow the run with a filter

When the full suite gives you a large report, pass a regular expression for the test names or groups you want to investigate. Start with one expression and keep the output:

$ fail2ban-testcases --no-network --no-gamin --memory-db --fast 'Database'
$ printf 'filtered status: %s\n' "$?"

The exact names available are release- and package-specific. If the expression matches nothing, the result may be a quick run with no useful coverage rather than proof that a component is healthy. Use the full run's report to choose a distinctive name, then compare the filtered result with the original failure.

To exclude a known matching group, use the documented negating form:

$ fail2ban-testcases --no-network --no-gamin --memory-db --fast -i 'Network'

Keep exclusions visible in review notes. A passing run with tests intentionally ignored is not equivalent to a complete passing suite.

6. Increase detail only when it helps

Use a numerical verbosity level for a controlled amount of additional output:

$ fail2ban-testcases --verbosity=2 --no-network --no-gamin --memory-db --fast

--log-level=LEVEL changes the logger level, while --log-direct prevents lazy logging inside tests. --log-traceback adds compressed traceback information and --full-traceback requests uncompressed tracebacks. These switches change diagnostics, not the test selection.

Do not turn on every diagnostic option in a first attempt. More output makes it harder to find the first actionable error. Add one setting, rerun the smallest useful filter, and compare the resulting traceback with the original log.

7. Decide what the result proves

A successful exit status tells you that the selected tests completed successfully in the environment and mode you chose. It does not prove that every network-dependent, gamin-dependent or file-database test ran. It also does not prove that your configured jails will block an address correctly.

Conversely, a failure in a test that needs a real log path, particular configuration interpolation or a specific database tool may describe the test environment rather than a production outage. Reproduce the failure without --fast when timing could matter, and without a filter when you need the complete context. Make any repair to package files or test fixtures through a reviewed, reversible change, then rerun the same command and compare the logs.

No undo command is needed for the normal examples: they run tests and write only the log you explicitly redirect. If you created fail2ban-testcases.log in the current directory, preserve it for review or delete that single file after the investigation. Do not stop or restart the Fail2Ban service solely because a unit test failed.