Search plain and compressed logs with zgrep

You will search a regular expression across gzip-compressed logs, ordinary text files and standard input without unpacking files into your working directory. The examples use gzip 1.12, installed with the Debian gzip package. Allow about ten minutes if the files and pattern are known.

Before starting, you need a shell, the zgrep command and read access to the files. This guide only reads data. It does not alter, decompress in place, rotate or delete a log, and the commands do not need elevated privileges.

1. Confirm the installed command

Check which executable your shell will run and record its version:

$ command -v zgrep
/usr/bin/zgrep
$ zgrep --version | head -2
zgrep (gzip) 1.12
Copyright (C) 2010-2018 Free Software Foundation, Inc.

The installed manual describes zgrep as a wrapper that invokes grep on compressed or gzipped files. It passes options to grep, while arranging for each named file to be uncompressed before searching. It also accepts input from standard input when no filename is supplied.

2. Search one compressed file

Give the pattern first and the compressed filename afterwards. The -n option is a normal grep option: it adds the matching line number to each result.

$ zgrep -n 'ERROR' /path/to/application.log.gz
17:ERROR: database connection refused
42:ERROR: retry limit reached

The output is the matching text, not a new decompressed file. The original .gz file is read and left unchanged. Quote patterns that contain shell metacharacters or spaces, so the shell does not interpret them before zgrep receives them.

To search for a literal phrase, quote it as one argument:

$ zgrep -n 'connection refused' /path/to/application.log.gz

For a regular expression, use the expression syntax supported by the installed grep. For example, this matches either warning or error at the start of a line:

$ zgrep -n '^(WARNING|ERROR):' /path/to/application.log.gz

That expression uses the extended regular expression form, so add -E when your pattern needs alternation in the usual GNU grep syntax:

$ zgrep -En '^(WARNING|ERROR):' /path/to/application.log.gz

3. Search plain and compressed files together

zgrep can take several filenames. It handles a plain file as well as a compressed one, which is useful while a log is being rotated:

$ zgrep -n 'ERROR' /var/log/application.log /var/log/application.log.1.gz
/var/log/application.log:17:ERROR: database connection refused
/var/log/application.log.1.gz:903:ERROR: retry limit reached

With more than one input file, the filename is included in the output. That label is useful when collecting results, so do not add -h unless you deliberately want to remove it.

Do not use a broad wildcard without checking what it expands to. A command such as zgrep 'ERROR' /var/log/*.gz may include unrelated services and may also fail when the shell pattern matches nothing, depending on the shell settings. Start with printf '%s\n' /var/log/application.log* to inspect the exact file list.

4. Search data from standard input

When there is no filename, the manual says that standard input is decompressed if necessary and fed to grep. This lets you compose a pipeline:

$ gzip -cd /path/to/application.log.gz | zgrep -n 'ERROR'
17:ERROR: database connection refused

Here gzip -cd performs the decompression and zgrep searches the resulting plain stream. In this particular pipeline, the decompression is explicit, so use it when the upstream command is already producing text. If you are passing a compressed stream directly to zgrep, test that exact pipeline on a small input first, because the useful behaviour depends on what the producer writes.

A safer way to inspect a command's result is to save its status immediately:

$ zgrep -n 'ERROR' /path/to/application.log.gz
$ status=$?
$ printf 'zgrep status: %s\n' "$status"
zgrep status: 0

5. Treat the exit status as a result

The installed manual defines three statuses:

• 0 means at least one line matched.
• 1 means no line matched.
• 2 means there was trouble, such as an unreadable or missing input.

A no-match result is not the same as a failed search. In a script, branch on the status rather than searching the output for a string:

if zgrep -q 'ERROR' /path/to/application.log.gz; then
    printf '%s\n' 'An error line exists'
else
    status=$?
    case "$status" in
        1) printf '%s\n' 'No error line found' ;;
        *) printf 'Search failed with status %s\n' "$status" >&2; exit "$status" ;;
    esac
fi

The -q option suppresses matching output, leaving the status as the signal. Keep the distinction between status 1 and status 2 in monitoring: one can be an expected clean result, while the other needs investigation.

6. Check the common failure modes

If a file is missing or cannot be read, rerun a read-only check:

$ ls -l /path/to/application.log.gz
$ test -r /path/to/application.log.gz && printf '%s\n' readable

Do not immediately add sudo. First confirm the path and permissions. Use elevated privileges only when your system policy grants you access to that log and the search genuinely requires it. If the file contains secrets or personal data, remember that matching lines are written to your terminal and may be captured in a terminal log or redirected file.

The manual lists several unsupported recursive or filtering options, including -r, -R, --include, --exclude, -d, -Z and -z. Do not assume that an option accepted by your standalone grep is supported by this installed zgrep. Check man zgrep before putting a new option into a script.

If you set the GREP environment variable, zgrep uses it as the grep program. Treat that variable as part of the command's configuration: inspect it with printf '%s\n' "$GREP" when results are surprising, and avoid inheriting an unreviewed value into automation.