Search Compressed Logs Safely with xzgrep

You will search compressed and uncompressed log files without unpacking them first. This guide uses the installed xzgrep from XZ Utils 5.8.4, with the xz-utils package reported locally as version 5.6.1+really5.4.5-1ubuntu0.3. The command found first in PATH is the 5.8.4 script, so check your own path before relying on version-specific details.

Allow about fifteen minutes. You need a shell, xzgrep, and readable files whose contents you want to inspect. The examples only read files and create temporary test data. They do not need sudo, and they do not change a service or decompress an archive in place.

1. Check which xzgrep you are running

Start with the command path and version. These are ordinary, read-only checks:

$ command -v xzgrep
/home/linuxbrew/.linuxbrew/bin/xzgrep
$ xzgrep --version
xzgrep (XZ Utils) 5.8.4
$ dpkg-query -W -f='${Package} ${Version}\n' xz-utils
xz-utils 5.6.1+really5.4.5-1ubuntu0.3

The package version and the executable version can differ when another installation appears earlier in PATH. The manpage installed here describes the xz-utils script family and says that options are passed to grep. That means the useful search options are grep options, not a separate xzgrep option set.

2. Search an xz-compressed file

The file suffix selects how the contents are decompressed. A normal basic regular expression is the first argument after the options. Add -n to show line numbers:

$ xzgrep -n 'timeout\|refused' /var/log/example.log.xz
18:connect: connection refused
42:request timeout after 30 seconds

Keep the pattern in single quotes when the shell should pass characters such as |, $ or * to grep unchanged. The exact output depends on the log. With one input file, grep normally prints matching lines without a filename prefix. With multiple files, the filename is included:

$ xzgrep -n 'refused' /var/log/example.log.xz /var/log/example.log.1.xz
/var/log/example.log.xz:18:connect: connection refused
/var/log/example.log.1.xz:7:connection refused

An ordinary uncompressed file is also accepted. This lets one command cover a directory containing both rotated and current logs, provided each file has the suffix expected by the installed script.

3. Choose extended or literal matching

Use the alias xzegrep when you want extended regular expressions. In that mode, alternation uses | and grouping uses parentheses without backslashes:

$ xzegrep -n 'timeout|refused' /var/log/example.log.xz
18:connect: connection refused
42:request timeout after 30 seconds

Use xzfgrep for a fixed string. It searches literally, so punctuation in a value such as [WARN] is not treated as a regular-expression character:

$ xzfgrep -n '[WARN]' /var/log/example.log.xz
63:[WARN] retrying connection

The names lzgrep, lzegrep and lzfgrep are compatibility aliases for the older LZMA Utils names. They select the same basic, extended or fixed-string behaviour. Prefer the xz-named command in new scripts so its purpose is clear.

4. Search a compressed stream

With no file argument, xzgrep reads standard input. An xz stream can therefore be piped directly:

$ xz -c /var/log/example.log | xzgrep -n 'refused'
18:connect: connection refused

Do not assume every format accepted as a file argument also works on standard input. The local manpage explicitly says that gzip, bzip2, lzop and zstd compressed input is not supported in this mode. For those formats, pass the compressed filename to xzgrep, or use the format's decompressor explicitly and pipe the resulting plain text to grep.

5. Handle the three useful exit statuses

Do not use the presence of output as the only test in a script. The command returns 0 when at least one match was found and no error occurred, 1 when there was no match and no error occurred, and a value greater than 1 when an error occurred. A greater-than-one result leaves it unknown whether a match was also found.

if xzgrep -q 'refused' /var/log/example.log.xz; then
    printf '%s\n' 'match found'
else
    status=$?
    case "$status" in
        1) printf '%s\n' 'no match' ;;
        *) printf 'search failed with status %s\n' "$status" >&2; exit "$status" ;;
    esac
fi

Here -q suppresses matching output, while the exit status still carries the result. Capture $? immediately. Running another command first replaces the status you meant to inspect.

To distinguish a harmless no-match from a bad path, try a missing file only as a diagnostic:

$ xzgrep -q 'refused' /path/to/missing.log.xz
xzgrep: /path/to/missing.log.xz: No such file or directory
$ printf 'status: %s\n' "$?"
status: 2

6. Avoid environment and filename traps

If GREP is set, xzgrep uses it instead of the normal grep, grep -E or grep -F command. That can silently change a script's meaning. Check it before a repeatable search:

$ printf 'GREP=%s\n' "${GREP-}"
$ unset GREP

Only set GREP deliberately, and never let an untrusted value provide shell options or a command. In a script, an explicit command such as xzgrep -F or xzegrep is easier to review.

The suffix matters too. The local documentation lists xz, lzma, gzip, bzip2, lzop and zstd files. A file with an unusual name may be treated as uncompressed even if its bytes are compressed. Rename it only when that is safe, or decompress it with the correct tool and search the resulting stream. Do not overwrite a valuable log while experimenting with redirection.