Compress and Verify Files with zstd on Linux

An untested zstd archive is a hope, not a backup. Learn to compress, test the archive, restore identical bytes and stream it through a pipe. Allow about 10 minutes.

• You need: a shell and write access to a test directory.
• No root: none of the normal commands need sudo.
• Tested on: zstd 1.5.7, installed on the machine used for this guide.

1. Check the installed version

Check the executable before relying on a distribution-specific feature:

$ zstd -q -V
1.5.7

Oddly, the installed manpage describes zstd 1.5.5 while the executable reports 1.5.7. Every example below uses options present in both.

• -q makes the version output script-friendly.
• zstd --help gives the human-readable version.

2. Compress a file and keep the original

Change to the directory holding the file and pass its name. Compression is the default operation and the default level is 3. zstd appends .zst to make a new file and keeps the source.

$ zstd report.txt
report.txt          :  4.10%   ( 1.41 KiB =>     59 B, report.txt.zst)
$ ls -l report.txt report.txt.zst

The progress line and sizes depend on your file. What matters is that both names exist. For quiet operation, especially in scripts, add -q:

$ zstd -q report.txt

3. Inspect and test the archive

-l lists what the compressed file records: compressed and uncompressed sizes, ratio, frame count and checksum.

$ zstd -l report.txt.zst
Frames  Skips  Compressed  Uncompressed  Ratio  Check  Filename
     1      0       59 B       1.41 KiB  24.407  XXH64  report.txt.zst

-t checks integrity without creating or removing anything. zstd decompresses to nowhere and checks the result.

$ zstd -t report.txt.zst
report.txt.zst: 1440 bytes

In a script, rely on the exit status:

$ if zstd -q -t report.txt.zst; then
>     printf '%s\n' 'archive passed its integrity test'
> else
>     printf '%s\n' 'archive failed its integrity test' >&2
> fi
archive passed its integrity test

4. Restore the original bytes

Decompression drops the .zst suffix to pick the output name, and keeps the compressed file by default.

$ zstd -q -d report.txt.zst
$ cmp report.txt <(zstd -q -d -c report.txt.zst)

The first command will normally refuse to replace an existing report.txt. That is why the check uses process substitution: -c writes decompressed data to standard output, so no file changes.

On shells without process substitution, decompress to a new name explicitly:

$ zstd -q -d report.txt.zst -o restored-report.txt
$ cmp -- report.txt restored-report.txt

5. Stream data between commands

-c leaves input files alone and writes to standard output. That is the normal shape for a pipeline:

$ tar -cf - ./project-directory | zstd -q -c > project.tar.zst
$ zstd -q -d -c project.tar.zst | tar -tf -

zstd has two built-in guards here:

• It will not write compressed data to a terminal by accident. Redirecting to a file still makes the destination explicit.
• It refuses to read compressed data from an interactive terminal. Feed it a pipe or a redirected file.

A quick round trip that prints the original text:

$ printf '%s\n' 'stream example' | zstd -q -c | zstd -q -d -c
stream example

6. Choose speed, size and cleanup deliberately

Levels 1 to 19 trade compression time for output size, with 3 as the default. Higher is not automatically better: it can take much longer to save a modest amount of space.

$ zstd -q -3 report.txt -o report-level3.txt.zst
$ zstd -q -9 report.txt -o report-level9.txt.zst

-T sets the number of compression workers, and -T0 asks zstd to detect the number of physical CPU cores. That can eat serious CPU and memory, so save it for workloads that benefit from parallel compression rather than making it a universal default:

$ zstd -q -T0 -3 large-file.bin -o large-file.bin.zst

--rm removes source files after successful compression or decompression, and is silently ignored when the output is standard output.

$ zstd -q --rm report.txt

That leaves report.txt.zst and deletes report.txt.

Common traps

• Wrong output name. zstd derives names from the input. Use -o when you need another directory or name, and check for collisions before using -f.
• Unexpected console text. Single-file compression shows progress and a summary. Add -q for a quiet script, but do not suppress errors blindly with repeated -q.
• No filename in the archive. zstd stores the contents, not the input filename or file attributes. Preserve naming and metadata separately if your workflow needs them.
• Large-memory options. --ultra enables levels 20 to 22 and raises memory requirements, decompression included. --long also enlarges the compressor and decompressor window. Use these only when the receiving side can support the settings.
• Environment surprises. ZSTD_CLEVEL and ZSTD_NBTHREADS can change the compression defaults; command-line -# and -T# override them. Check the environment when a script behaves differently under a service account.