Search Compressed Logs Safely with xzgrep

xzgrep lets you grep straight into an xz-compressed log without decompressing it first. This guide uses the installed xzgrep from XZ Utils 5.8.4, with the xz-utils package reported locally as version 5.6.1+really5.4.5-1ubuntu0.3. The command found first in PATH is the 5.8.4 script, so check your own path before relying on version-specific details.

Allow about fifteen minutes. You need a shell, xzgrep, and readable files whose contents you want to inspect. The examples only read files and create temporary test data: no sudo, no service restart, no decompressing an archive in place.

1. Check which xzgrep you are running

Start with the command path and version. These are ordinary, read-only checks:

$ command -v xzgrep
/home/linuxbrew/.linuxbrew/bin/xzgrep
$ xzgrep --version
xzgrep (XZ Utils) 5.8.4
$ dpkg-query -W -f='${Package} ${Version}\n' xz-utils
xz-utils 5.6.1+really5.4.5-1ubuntu0.3

The package version and the executable version can differ when another installation appears earlier in PATH. The manpage installed here describes the xz-utils script family and says that options are passed straight through to grep, so the useful search options are grep options, not a separate xzgrep set.

2. Search an xz-compressed file

The file suffix selects how the contents are decompressed. A normal basic regular expression is the first argument after the options. Add -n to show line numbers:

$ xzgrep -n 'timeout\|refused' /var/log/example.log.xz
18:connect: connection refused
42:request timeout after 30 seconds

Keep the pattern in single quotes when the shell should pass characters such as |, $ or * to grep unchanged. With one input file, grep prints matching lines without a filename prefix; give it more than one file and the filename gets added automatically:

$ xzgrep -n 'refused' /var/log/example.log.xz /var/log/example.log.1.xz
/var/log/example.log.xz:18:connect: connection refused
/var/log/example.log.1.xz:7:connection refused

An ordinary uncompressed file is also accepted, which is what makes one command work across a directory of rotated and current logs, provided each file carries the suffix the installed script expects.

3. Choose extended or literal matching

Use the alias xzegrep for extended regular expressions, where alternation uses | and grouping drops the backslashes:

$ xzegrep -n 'timeout|refused' /var/log/example.log.xz
18:connect: connection refused
42:request timeout after 30 seconds

Use xzfgrep when the pattern is a fixed string rather than a regex, so punctuation such as the brackets in [WARN] is matched literally:

$ xzfgrep -n '[WARN]' /var/log/example.log.xz
63:[WARN] retrying connection

lzgrep, lzegrep and lzfgrep are compatibility aliases held over from the old LZMA Utils names, and they behave the same way. Prefer the xz-named command in new scripts so its purpose is obvious to the next reader.

4. Search a compressed stream

With no file argument, xzgrep reads standard input, so an xz stream can be piped straight in:

$ xz -c /var/log/example.log | xzgrep -n 'refused'
18:connect: connection refused

Do not assume every format accepted as a file argument also works on standard input. The local manpage is explicit that gzip, bzip2, lzop and zstd compressed input is not supported this way. For those, either pass the compressed filename to xzgrep directly, or decompress with the matching tool and pipe plain text to grep.

5. Handle the three useful exit statuses

Do not rely on the presence of output alone in a script. xzgrep returns three distinct results:

• 0 means at least one match was found and nothing went wrong.
• 1 means no match, and also nothing went wrong.
• Anything above 1 means an error occurred, and whether a match also happened is left unknown.

if xzgrep -q 'refused' /var/log/example.log.xz; then
    printf '%s\n' 'match found'
else
    status=$?
    case "$status" in
        1) printf '%s\n' 'no match' ;;
        *) printf 'search failed with status %s\n' "$status" >&2; exit "$status" ;;
    esac
fi

Here -q suppresses matching output while the exit status still carries the result. Capture $? immediately: running another command first replaces the status you meant to inspect.

$ xzgrep -q 'refused' /path/to/missing.log.xz
xzgrep: /path/to/missing.log.xz: No such file or directory
$ printf 'status: %s\n' "$?"
status: 2

6. Avoid environment and filename traps

If GREP is set, xzgrep uses it in place of the normal grep, grep -E or grep -F call, which can silently change a script's meaning. Check it before a repeatable search:

$ printf 'GREP=%s\n' "${GREP-}"
$ unset GREP

The suffix matters too: the local documentation lists xz, lzma, gzip, bzip2, lzop and zstd. A file with an unusual name may get treated as plain text even though its bytes are compressed. Rename it only when that is safe, or decompress it properly and search the stream. Do not overwrite a valuable log while experimenting with redirection.