Compress and Check Files Safely with xz

xz compresses a file well, but the plain form deletes your source the moment it succeeds. This guide builds a small habit around it: compress with --keep, test the archive, then restore it, all without losing the original.

The examples use the xz command from the installed xz-utils package. Allow about ten minutes. You need a shell, a file you can safely copy, and enough free space for a second copy.

There is a version detail worth checking first. On this machine, xz --version reports XZ Utils 5.8.4, while the package database reports xz-utils 5.6.1+really5.4.5-1ubuntu0.3. The command's own output is the useful description of the executable that will run. Check your machine rather than assuming these numbers match.

Safety boundary: Plain xz FILE removes the input after a successful compression. The guide uses --keep while you learn the workflow. Do not use --force casually: it can replace an existing target file and bypass several input safety checks.

1. Check the installed command

Run these ordinary, read-only checks. They need no elevated privileges:

$ command -v xz
$ xz --version
$ dpkg-query -W -f='${Package} ${Version}\n' xz-utils

Expected output includes an executable path, a line beginning xz (XZ Utils), and the installed package record. If command -v finds nothing, install or repair xz-utils through your normal system package process. Do not work around a missing command by downloading an unrelated binary into a system directory.

Checkpoint: Record the executable path and version before comparing output with another machine. Compression output can vary between XZ Utils versions even when the options are identical, although the file format remains the same.

2. Compress while keeping the source

Change /path/to/report.txt to an existing regular file. The -k option means keep the input. With no output option, xz creates a new file by appending .xz:

$ xz --keep /path/to/report.txt
$ ls -lh /path/to/report.txt /path/to/report.txt.xz

The expected result is two files. The original remains readable and the compressed file has the new suffix. xz copies common ownership, permissions and timestamps to the target, but it does not copy every kind of metadata, such as access control lists or extended attributes.

Without --keep, successful compression removes the source only after the compressed target has been closed. If compression fails, xz does not remove it. If you need the source for a backup, audit trail or repeatable build, keep it explicitly rather than relying on memory.

For a quick size check:

$ xz --list /path/to/report.txt.xz

This prints compressed and uncompressed information and changes no files. It needs a seekable compressed file, so it is not a substitute for testing a stream from standard input.

3. Test integrity before unpacking

Use --test for a read-only integrity check. It decompresses internally and discards the output, so it does not create or remove a file:

$ xz --test /path/to/report.txt.xz
$ printf 'xz status: %s\n' "$?"

A status of 0 means the test succeeded. Status 1 means an error occurred; status 2 means a warning occurred without an actual error. Save the status immediately, before running another command. A successful integrity test proves that xz can decode the stream, not that its contents are the document you intended to receive.

For a script, make the failure stop the workflow instead of parsing human-readable output:

if ! xz --test -- /path/to/report.txt.xz; then
    printf '%s\n' 'Compressed file failed its integrity test' >&2
    exit 1
fi

The -- marker separates options from the filename. It is a useful habit when a path comes from a variable or an external list.

4. Restore the file without deleting the archive

Because the source file is still present in this workflow, restore into a separate directory through standard output:

$ mkdir -p /tmp/xz-restore
$ xz --decompress --stdout /path/to/report.txt.xz > /tmp/xz-restore/report.txt
$ cmp -- /tmp/xz-restore/report.txt /path/to/report.txt

The final command is a useful check when the original source is still available. Otherwise verify the restored file with the checksum or application-specific test that belongs to your workflow. If you instead use --decompress --keep without --stdout, xz derives the target name by removing .xz and refuses to overwrite that target if it already exists.

--stdout implies --keep. It is also the normal shape for a pipeline such as xz -dc archive.tar.xz | tar -tf -. xz refuses to write compressed data to a terminal, which helps prevent a screen full of binary bytes.

5. Choose a preset with the recipient in mind

The default compression preset is -6. It is normally a sensible distribution default, but higher numbers are not automatically better. Presets -7 through -9 require more memory for both compression and decompression, and the manpage specifically warns against blindly using -9.

Use a lower preset when speed or memory matters, and measure the result on representative data:

$ xz --keep -4 /path/to/report.txt
$ xz --list /path/to/report.txt.xz

Do not run this example if report.txt.xz already exists, because xz skips an existing target. Remove or rename only the target you created, after checking its path, or choose a different input copy. Preset -e selects a slower variant of the chosen level. It may save a little more space, but can also produce a worse result, and it does not reduce decompression memory.

6. Set a decompression memory boundary

Compressed data can be small while requiring substantial memory to decode. For an untrusted or resource-constrained input, set a limit and treat failure as a normal result:

$ xz --memlimit-decompress=128MiB --test -- /path/to/report.txt.xz

If the file needs more than the limit, xz reports an error and does not produce decoded output. The limit can also be placed in XZ_DEFAULTS for user-wide defaults, or XZ_OPT for a script or tool that cannot add command-line options. Environment variables are easy to forget, so check them when an apparently ordinary command behaves differently:

$ printf 'XZ_DEFAULTS=%s\nXZ_OPT=%s\n' "${XZ_DEFAULTS-}" "${XZ_OPT-}"

For scripts, prefer an explicit command-line limit when you can. Do not set a hidden global option in a service without documenting it, because it can make valid archives fail later.

Aliases and common traps

unxz is the decompression-oriented alias. xzcat writes decompressed data to standard output. The legacy names lzma, unlzma and lzcat operate on the older .lzma format as well as the command's compatible modes. For new files, prefer native .xz unless a receiving system requires something else.

Do not confuse --list with --test: list reports container details, while test checks that decoding succeeds. Do not use --single-stream to hide unexplained trailing data. Normally trailing data makes xz report corruption; that option deliberately tells it to decode only the first .xz stream, so use it only when you understand the format of the input.

No step here needs sudo. Elevated privileges do not repair a corrupt archive, and running xz as root can create files owned by root or expose data to a wider set of users. If a destination is not writable, fix ownership or permissions through the system's normal administration process after checking the exact path.

Done means