Compress and restore sorted word lists with prezip-bin

You will turn a sorted text word list into a compact prezip file, restore it to text, and check that the result is usable. This guide covers the prezip-bin installed with Aspell on this machine. Allow about ten minutes for a small list, or longer if you need to inspect a large dictionary before replacing anything.

The examples read and write files in your current directory. They do not need sudo. Keep the original list until the decompressed copy has passed your checks.

1. Check the installed command

Confirm which executable your shell will run and record both package and program versions. The package version and the compressor's own version are different pieces of information:

$ command -v prezip-bin
/usr/bin/prezip-bin
$ dpkg-query -W -f='${Package} ${Version}\n' aspell
aspell 0.60.8.1-1build1
$ prezip-bin -V
prezip, a prefix delta compressor. Version 0.1.1, 2004-11-06

The local manual describes -z for compression, -d for decompression and -V for the version. It also says that the program uses standard input and standard output, so shell redirection is part of the normal interface.

2. Prepare a sorted word list

prezip-bin is intended for sorted word lists, not arbitrary text. Sort a copy or generate the list as a pipeline. The locale matters: using LC_COLLATE=C gives a repeatable byte-oriented order instead of depending on the host's regional settings.

$ LC_COLLATE=C sort -u < words.txt > words.sorted.txt
$ sed -n '1,5p' words.sorted.txt
alpha
alpine
beta
delta
gamma

Use a text editor or a small inspection command to remove leading and trailing blank spaces used only for formatting. The manual warns that those spaces are retained in the compressed output. Do not strip meaningful spaces from a format where they are part of the word data.

This is also the point to check the input rather than after compression:

$ test -s words.sorted.txt && echo 'non-empty input'
non-empty input
$ LC_COLLATE=C sort -c words.sorted.txt
$ printf 'input lines: '
$ wc -l < words.sorted.txt

sort -c exits successfully when the file is already ordered according to the selected locale. An empty word list may be valid for your workflow, but test -s makes an accidental empty input visible before it becomes a small, plausible-looking binary file.

3. Compress to a new binary file

Run prezip-bin -z with the list on standard input and a new path on standard output:

$ prezip-bin -z < words.sorted.txt > words.pz 2> words.pz.err
$ status=$?
$ printf 'prezip-bin exit: %s\n' "$status"
prezip-bin exit: 0
$ test ! -s words.pz.err && echo 'no diagnostics'
no diagnostics
$ wc -c words.sorted.txt words.pz
18 words.sorted.txt
19 words.pz
37 total

The byte counts are only an example. A prezip file is not guaranteed to be smaller: the manual explicitly warns that an unsuited input can grow. Treat a zero exit status as a successful encoding, then verify the contents by decompressing it.

4. Restore the text list and compare it

Use -d to read the binary file and write the restored list to a separate path:

$ prezip-bin -d < words.pz > words.restored.txt
$ cmp --silent words.sorted.txt words.restored.txt
$ printf 'comparison exit: %s\n' "$?"
comparison exit: 0
$ wc -l words.sorted.txt words.restored.txt
 3 words.sorted.txt
 3 words.restored.txt
 6 total

cmp returns zero only when the files are byte-for-byte identical. That is the useful check for a round trip. If the original list used a particular line ending or final newline, this comparison will detect that too.

There is no persistent configuration to undo in this workflow. To discard the generated files after checking them, remove only the explicit temporary paths you created:

$ rm -- words.pz words.pz.err words.restored.txt

This deletion is irreversible. Keep the files if they are part of a dictionary build or if you have not yet checked the consumer that will read the compressed list.

5. Feed a restored list into Aspell

The documented use case is to pipe a decompressed word list into Aspell's master-dictionary builder. On this installation, give Aspell an explicit language; without one, it tries to open an <unknown> language data file. Choose the language that matches your list. Write the resulting dictionary under a deliberate path, and test the command without replacing an existing dictionary:

$ prezip-bin -d < words.pz | aspell --lang=en create master ./words.rws
$ printf 'create exit: %s\n' "${PIPESTATUS[1]}"
create exit: 0
$ file ./words.rws
./words.rws: data

PIPESTATUS is a Bash array: element zero is prezip-bin and element one is aspell. Check both when a pipeline matters, because a shell can otherwise report only the last command's status:

$ prezip-bin -d < words.pz | aspell --lang=en create master ./words.rws
$ printf 'decompress=%s create=%s\n' "${PIPESTATUS[0]}" "${PIPESTATUS[1]}"
decompress=0 create=0

If the destination dictionary already exists, stop before using that command or select a new filename. Replacing a dictionary can affect spelling checks for programs that use it. Restore the previous file from your backup or package-managed copy rather than guessing at its contents.

6. Diagnose bad input

An unknown option or missing command produces a non-zero status and a usage diagnostic. Decompression has more specific failures. The installed command reports unknown format when the input is not a recognised compressed word-list format:

$ prezip-bin -d < not-a-word-list.pz > restored.txt
unknown format
$ printf 'exit: %s, output bytes: ' "$?"
exit: 2, output bytes: 
$ wc -c < restored.txt
0

A file that begins plausibly but ends early can produce corrupt input or unexpected EOF. The manual warns that corruption may leave some valid data in the output, so treat any non-zero status as a failed restore. Write to a disposable or temporary destination and do not replace a known-good list until the status and comparison both pass.

For a script, capture the status directly and preserve the old output:

$ tmp='words.restored.txt.new'
$ if prezip-bin -d < words.pz > "$tmp"; then
>   cmp --silent words.sorted.txt "$tmp" && mv -- "$tmp" words.restored.txt
> else
>   printf '%s\n' 'decompression failed; old output was not changed' >&2
>   rm -- "$tmp"
> fi

The temporary file is removed on failure. The original restored file, if present, is untouched.