Build a Searchable Permuted Index with GNU ptx

GNU ptx turns a plain text file into a permuted index, one output line per keyword occurrence with the surrounding words kept as context. It is a strange-looking tool the first time you meet it, built for compact text indexes and roff or TeX input, and this guide gets you from a first run to a script you can trust. The examples use GNU ptx from coreutils 9.4, matching the installed manual page.

Allow about fifteen minutes. You need a shell and a readable text file. The normal examples only read their input and write to standard output, so they do not need sudo. Keep the original input until you have checked the generated index.

1. Check the installed command

Confirm which executable you will run and record its version:

$ command -v ptx
/usr/bin/ptx
$ ptx --version
ptx (GNU coreutils) 9.4

Checkpoint: if command -v ptx prints nothing, stop and install the coreutils package through your normal distribution process. Do not copy a binary into a shared system directory just to make this example work.

2. Generate a basic index

Create or choose a text file with ordinary prose. For a quick test, pipe text directly into ptx:

$ printf '%s\n' 'The quick brown fox jumps over the lazy dog.' 'A quick fox tests ptx output.' | ptx -w 60
   .                             A quick fox tests ptx output
   over the lazy dog.            The quick brown fox jumps
   lazy dog.         The quick   brown fox jumps over the
       fox jumps over the lazy   dog.                    /brown
               The quick brown   fox jumps over the lazy dog.
                       A quick   fox tests ptx output.

Each line is centred on a keyword, with the surrounding words as context. The slash is the default truncation marker where context has been cut at an input line boundary. The output is deliberately rearranged: it is not a summary and should not be read as one.

With a file, put the input name after the options:

$ ptx -w 80 /path/to/notes.txt > notes.index
$ test -s notes.index && printf '%s\n' 'Index created'

Redirection creates or truncates notes.index before ptx runs. If that name already holds a useful index, write to a new destination such as notes.index.new instead, check it, then swap it in deliberately with mv notes.index.new notes.index. If ptx fails, leave the old index alone and only remove the incomplete new file once you have confirmed it is disposable.

3. Control the amount of context

The -w option sets the output width in columns, excluding a reference when references are enabled. A larger value keeps more context but produces wider lines:

$ ptx --width=40 < notes.txt
$ ptx --width=100 < notes.txt

Use the narrow width for a small terminal or a compact report, and the wider one when the index will be read as a file. Leave out -w and GNU ptx falls back to its built-in default, which is not a promise about your terminal's width, so set it explicitly in any script you plan to rerun.

The -g option controls the gap in columns between output fields. Keep it modest when you need to fit a fixed-width report:

$ ptx --width=60 --gap-size=2 notes.txt > notes.index

Checkpoint: inspect the first few lines without opening the file in an editor that might wrap them:

$ sed -n '1,5p' notes.index

4. Remove noise or limit the keywords

Long prose produces plenty of unhelpful entries for common words. Put words to ignore, separated according to the file's word rules, in an ignore file and pass it with -i:

$ printf '%s\n' 'the' 'and' 'with' > common-words.txt
$ ptx --ignore-file=common-words.txt --width=80 notes.txt > notes.index

The ignore file changes which words can become keywords; it does not delete those words from the surrounding context. Treat it as part of your indexing configuration and keep it beside the command or build recipe that uses it.

For a deliberately small index, use an only file instead:

$ printf '%s\n' 'error' 'warning' 'timeout' > review-words.txt
$ ptx --only-file=review-words.txt --width=80 notes.txt > review.index

Do not combine -i and -o casually. If a word is both excluded and selected, the resulting keyword set can surprise whoever maintains these files next. Check the output and document the precedence you intend rather than guessing from a partial result.

5. Add references when the input has line labels

Use -r when the first field on each input line is a reference: ptx then treats that field as a reference rather than ordinary context.

$ printf '%s\n' 'intro first line' 'appendix second line' | ptx --references --width=50
intro                        first line
intro                first   line
appendix                second   line
appendix                         second line

By default, references affect the available line width. -R moves references to the right and excludes them from that width calculation:

$ ptx --references --right-side-refs --width=50 notes-with-labels.txt > notes.index

6. Choose an output format

Plain output is the default. If another typesetting tool will consume the index, request roff or TeX directives explicitly:

$ ptx --format=roff --width=80 notes.txt > notes.roff
$ ptx --format=tex --width=80 notes.txt > notes.tex

GNU ptx's roff output starts with directives such as .xx, rather than the aligned plain-text display you saw earlier. Verify the format before handing the file to a formatter:

$ sed -n '1,3p' notes.roff
.xx "" "" "..." ""

Do not use -t expecting a typeset result: the installed manual marks typeset mode as not implemented. -O and -T are the implemented format selectors documented for this version.

7. Handle failures without losing the source

If ptx reports an input error, check the path and permission first:

$ test -r /path/to/notes.txt && printf '%s\n' 'Input is readable'
$ ls -l /path/to/notes.txt

Reading a file normally needs no elevated privilege. If the file is protected, ask its owner or use an approved read-only access path. Do not make a private document world-readable just to generate an index.

If output comes back empty, check whether an only file excluded every candidate word, or an ignore file excluded all the keywords: compare a run without -i or -o against the configured one. If lines are unexpectedly short, increase --width and check whether the input has short lines or sentence boundaries limiting context.

To sort lower-case and upper-case forms together, use -f. This changes sorting behaviour, not the text stored in the input:

$ ptx --ignore-case --width=80 notes.txt > notes.index

Recovery: there is no undo command because ptx does not edit the input. Recovery just means keeping the source safe and swapping in a generated destination only after it has passed your checks.

Done means