Use zstdmt for Fast, Reversible Multi-threaded Compression

By the end of this guide you will have compressed a file with zstdmt, checked the result, and restored the original data without deleting the source. You need the zstd package, a shell, and enough free space for the compressed output. Allow about five minutes for the first run; choosing sensible thread and memory limits takes longer only when the files or host are large.

1. Check which zstdmt you are using

zstdmt is the multi-threaded entry point for Zstandard. The installed manpage describes it as equivalent to zstd -T0: use the detected number of physical CPU cores for compression. Check both the executable and its version before relying on behaviour from a different installation.

$ command -v zstdmt
/path/to/zstdmt
$ zstdmt --version
*** Zstandard CLI (64-bit) v1.5.5, by Yann Collet ***

The exact version and path on your machine may differ. The local manpage used for this guide is from zstd 1.5.5. If a shell alias, container image or alternate package supplies the command, read that installation's manual as well.

2. Compress one file and keep the source

Run zstdmt with the input file. Compression is the default operation, level 3 is the documented default, and the result is written beside the source with .zst appended. The source is kept unless you explicitly request removal.

$ zstdmt /path/to/report.log
/path/to/report.log   : 18.42%   ( 1024 KiB -> 188 KiB)
$ ls -lh /path/to/report.log /path/to/report.log.zst

The progress and ratio vary with the data. Text often compresses well; already-compressed media may not. If the destination already exists, zstdmt normally refuses to overwrite it. That refusal is useful: stop and decide whether the old output is disposable before using -f.

To choose a different normal compression level, use a number from 1 to 19. Higher levels generally trade time and memory for a smaller result, but the useful point depends on your data and workload.

$ zstdmt -6 /path/to/report.log
/path/to/report.log   : 17.91%   ( 1024 KiB -> 183 KiB)

3. Set the worker count deliberately

zstdmt already selects a thread count through -T0. Make it explicit when sharing a host or running it from a job scheduler. -T4 asks for four compression worker threads. The command still needs I/O and memory, so four workers are not a promise of four times the throughput.

$ zstdmt -T4 -6 -o /path/to/report.log.zst /path/to/report.log
/path/to/report.log   : 17.91%   ( 1024 KiB -> 183 KiB)

-o gives the output an exact name. It does not make overwriting safe. If you need a repeatable script, select a new temporary destination, check its result, then move it into place as a separate, reviewed operation. Keep -f out of unattended commands unless replacement is genuinely intended.

For a one-off setting, use -T rather than changing the environment. The manpage also documents ZSTD_NBTHREADS for the compression thread count and ZSTD_CLEVEL for the compression level. Command-line options override those variables. Treat inherited environment variables as part of the job's input when diagnosing an unexpected result.

4. Verify the compressed file before relying on it

Use test mode to decompress and checksum the data while discarding the decompressed bytes. It creates and removes no files, so it is suitable as a checkpoint before copying or deleting anything.

$ zstdmt --test /path/to/report.log.zst
/path/to/report.log.zst : 1024 KiB...     OK

For a quiet script, add -q and inspect the exit status. A zero status means the integrity check succeeded; a non-zero status means the file should not be treated as a valid replacement. A successful test cannot prove that you copied the file to the intended host or directory, so check the path too.

$ zstdmt -q --test /path/to/report.log.zst
$ printf 'test status: %s\n' "$?"
test status: 0

5. Restore the original into a chosen destination

Decompression with -d removes the .zst suffix when it derives a filename. Use -o when the destination matters, such as restoring into a staging directory. The source archive is kept by default.

$ zstdmt -d -o /tmp/report-restored.log /path/to/report.log.zst
/path/to/report.log.zst: 1024 KiB...     report-restored.log
$ cmp -- /path/to/report.log /tmp/report-restored.log
$ echo "restored bytes match"
restored bytes match

cmp produces no output when the files match. If it reports a difference, preserve both files and investigate; do not retry with -f over the top of evidence.

6. Use streams when a file is not the right interface

When input is redirected or piped, zstdmt reads standard input and writes standard output. This is useful for a pipeline, but output to a terminal is refused unless forced. Redirect to a named file so binary data does not land in your terminal.

$ tar -cf - /path/to/project | zstdmt -T4 -o project.tar.zst
$ zstdmt -dc project.tar.zst | tar -tf - | head
path/to/project/
path/to/project/README.txt

The -c option writes processed data to standard output and keeps the input unchanged. In pipelines, check each command's status rather than assuming that a later command proves compression succeeded. A truncated stream can otherwise look like an ordinary pipeline failure.

7. Avoid the destructive and resource-heavy traps

Do not start with --rm. It removes source files after successful compression or decompression. With -o, zstdmt can prompt for confirmation; with -f, that confirmation can be suppressed. First run the non-destructive command, test the result, and confirm the destination. If you used --rm and the source is gone, restore it from a verified copy or backup; there is no zstdmt undo command.

Watch memory as well as CPU. -T0 can use many physical cores. More workers can increase throughput, but also increase resource pressure and compete with other services. Long-distance mode, enabled with --long, increases the window and memory use for compression and decompression. Use it only when long matches justify that cost.

Do not confuse testing with extraction. --test validates checksums and discards decompressed data. It does not create a restored file. For that, use -d and choose a destination.

Done means

• zstdmt --version identified the executable you intended to run.
• The compressed file exists beside the source or at the explicit -o path.
• zstdmt -q --test FILE.zst returned status 0.
• A decompression into a separate destination matched the original with cmp.
• You did not use --rm or -f until the output and recovery plan were clear.