Batch-Convert Manual Pages to UTF-8 with man-recode

A pile of manual pages arrived in mismatched legacy encodings, and man-recode converts the lot to one encoding in a single pass. It can write suffixed output files, or replace each input after stripping its compression extension, all while leaving the originals alone. Allow about ten minutes.

You need the man-db package, readable manual-page files and a writable destination directory. These commands run as your own user; do not use sudo just because the source pages live in a system directory. Copy them to a working directory first, or choose a writable output location instead.

1. Check the installed command

This is a read-only check. Confirm the version and option shape before scripting against it:

$ man-recode --version
man-recode 2.12.0
$ man-recode --help
Usage: man-recode [OPTION...]
            -t CODE {--suffix SUFFIX | --in-place} FILENAME...

The important rule: -t names the output encoding, and you must choose exactly one output mode, --suffix or --in-place. Check the package version separately if you need it:

$ dpkg-query -W -f='${Package} ${Version}\n' man-db
man-db 2.12.0-4build2

2. Inspect the inputs before converting

Work on a copy. The input can be compressed, such as .gz, but man-recode's output is always uncompressed:

$ mkdir -p "$HOME/man-recode-work"
$ cp /usr/share/man/man1/man-recode.1.gz "$HOME/man-recode-work/"
$ file "$HOME/man-recode-work/man-recode.1.gz"
/home/you/man-recode-work/man-recode.1.gz: gzip compressed data, ...

Replace the source path with your own file, or a carefully chosen list of files. Before a batch run, check both readability and the exact names the shell will pass through:

$ find "$HOME/man-recode-work" -maxdepth 1 -type f -name '*.gz' -print
$ test -r "$HOME/man-recode-work/man-recode.1.gz" && echo readable
readable

3. Create new files with a suffix first

--suffix is the safest starting move: it leaves the compressed input alone and appends your suffix after stripping the compression extension:

$ man-recode --to-code=UTF-8 --suffix=.utf8 "$HOME/man-recode-work/man-recode.1.gz"
$ find "$HOME/man-recode-work" -maxdepth 1 -type f -printf '%f\n' | sort
man-recode.1.gz
man-recode.1.utf8
$ file "$HOME/man-recode-work/man-recode.1.utf8"
/home/you/man-recode-work/man-recode.1.utf8: troff or preprocessor input, ASCII text

Notice the output is man-recode.1.utf8, not man-recode.1.gz.utf8: the suffix is a naming choice, not a request to compress. Pick a suffix that cannot be confused with the original format, such as .utf8 or .converted.

For several known files, list them explicitly or use a narrow, reviewed glob:

$ man-recode -t UTF-8 --suffix=.utf8 \
    "$HOME/man-recode-work/man-recode.1.gz" \
    "$HOME/man-recode-work/other-page.1.gz"

4. Know how the input encoding gets chosen

man-recode does not need a separate input encoding for every page. It first looks for a declaration on the page's first line, one that can name an encoding such as UTF-8 or ISO-8859-1, sometimes alongside a preprocessor name. With no declaration, it guesses from the filename.

That default suits a mixed collection well, and it is also the main source of surprises. A misleading filename, or a declaration that is not actually on the first line, can send it the wrong way. Check both when output looks damaged or full of replacement characters:

$ zcat "$HOME/man-recode-work/man-recode.1.gz" | sed -n '1p'
."\ -*- coding: UTF-8 -*-
$ man-recode -d -t UTF-8 --suffix=.debug.utf8 \
    "$HOME/man-recode-work/man-recode.1.gz"
guessed input encoding UTF-8 for /home/you/man-recode-work/man-recode.1.gz

Use --debug to see the encoding decision while diagnosing a page, not as a substitute for actually checking the resulting file.

5. Replace inputs only after a backup

--in-place writes the converted content back under the input's base name, again stripping the compression extension. This changes state and can destroy your only copy, so back up first or stick with the suffix workflow:

$ cp --preserve=all \
    "$HOME/man-recode-work/man-recode.1.gz" \
    "$HOME/man-recode-work/man-recode.1.gz.bak"
$ man-recode -t UTF-8 --in-place \
    "$HOME/man-recode-work/man-recode.1.gz"
$ find "$HOME/man-recode-work" -maxdepth 1 -type f -printf '%f\n' | sort
man-recode.1.gz.bak
man-recode.1
man-recode.1.utf8

The converted file is now man-recode.1; the compressed name is gone entirely. Check the result before deleting the backup:

$ file "$HOME/man-recode-work/man-recode.1"
$ man-recode -d -t UTF-8 --suffix=.verification \
    "$HOME/man-recode-work/man-recode.1"
guessed input encoding UTF-8 for /home/you/man-recode-work/man-recode.1

6. Handle failures without hiding them

A missing or unreadable input produces a non-zero exit status, and the default mode reports it:

$ man-recode -t UTF-8 --suffix=.utf8 /path/to/missing-page.1
man-recode: can't open /path/to/missing-page.1
$ printf 'exit status: %s\n' "$?"
exit status: 1

--quiet suppresses conversion error messages, but it does not turn a failed conversion into a success. Keep it out of interactive troubleshooting, and use it only where the surrounding script actually records and checks the exit status:

$ man-recode -q -t UTF-8 --suffix=.utf8 /path/to/missing-page.1
$ printf 'exit status: %s\n' "$?"
exit status: 1

If a page fails, check its path, permissions and first-line declaration. Do not respond by overwriting the source repeatedly or running as root: a privilege change cannot fix an incorrect encoding declaration.