Compare Sorted Files Safely with comm

Use comm to see what two sorted lists share and where they part company, and beware output that looks fine but is quietly wrong. The usual culprit is input that is not sorted the way the active locale expects. This guide gets you a repeatable comparison and a way to catch that problem. The examples use GNU comm from GNU coreutils 9.4, and it takes about ten minutes.

• You need: a shell and two newline-delimited text files.
• What it touches: nothing. Files are read, never edited or deleted.
• Privileges: ordinary user throughout. sudo is neither required nor helpful for comparing files you can already read.

1. Make both inputs comparable

comm expects both files to be sorted. That is part of its input contract, not an optional tidy-up. Sort with the same locale you will use for the comparison. The explicit LC_ALL=C below makes byte-oriented ordering predictable across hosts:

$ LC_ALL=C sort -u input-a.txt > sorted-a.txt
$ LC_ALL=C sort -u input-b.txt > sorted-b.txt
$ comm --check-order sorted-a.txt sorted-b.txt > /dev/null
$ printf 'comm accepted both files\n'
comm accepted both files

The output files are new files. If you already know the inputs were produced in the same locale and are sorted, you can compare them directly. Do not sort one file in a different locale and assume the results are interchangeable: comm honours LC_COLLATE, so the ordering rule matters as much as the visible characters.

$ comm --version | head -n 1
comm (GNU coreutils) 9.4

2. Read the default three-column result

Run the comparison with no options:

$ LC_ALL=C comm sorted-a.txt sorted-b.txt
apple
		banana
		cherry
	date

• Column one: lines unique to the first file.
• Column two: lines unique to the second file.
• Column three: lines present in both.

The default separator is a tab, so the leading whitespace is meaningful. A line at the left edge belongs to column one, while a common line starts after two column separators.

3. Select the result you need

The options -1, -2 and -3 suppress their respective columns. To print the intersection, suppress the two unique columns:

$ LC_ALL=C comm -12 sorted-a.txt sorted-b.txt
banana
cherry

To print the symmetric difference, suppress the common column:

$ LC_ALL=C comm -3 sorted-a.txt sorted-b.txt
apple
	date

That second output still has a tab before the lines unique to the second file. A suppressed column is removed from the result, but the remaining columns keep their position. If a downstream program needs one unambiguous separator, use an output delimiter:

$ LC_ALL=C comm --output-delimiter=' | ' sorted-a.txt sorted-b.txt
apple
 |  | banana
 |  | cherry
 | date

Every column boundary gets the delimiter, so with a delimiter containing spaces, common lines show two of them. For machine processing, choose a separator that cannot occur in your data, or use separate invocations when you need a plain one-column list.

4. Check failures instead of trusting plausible output

Without --check-order, an incorrectly sorted file can give output that looks reasonable. Use the option in validation and deployment scripts:

$ LC_ALL=C comm --check-order sorted-a.txt sorted-b.txt > /dev/null
$ printf 'exit status: %s\n' "$?"
exit status: 0

Status 0 means the comparison completed and both inputs passed the order check. Now see the failure mode, using an intentionally unsorted first file:

$ printf '%s\n' cherry apple > unsorted-a.txt
$ LC_ALL=C comm --check-order unsorted-a.txt sorted-b.txt
	banana
	cherry
comm: file 1 is not in sorted order
$ printf 'exit status: %s\n' "$?"
exit status: 1

The exact diagnostic, and how much output comes before it, can vary with the input. The useful result is the non-zero status and the file named in the error. Fix the producer, or regenerate the file in the same locale.

5. Add counts when a report needs totals

--total appends counts for the three columns. It is handy for a quick reconciliation, but it does not replace checking the actual lines:

$ LC_ALL=C comm --total sorted-a.txt sorted-b.txt
apple
		banana
		cherry
	date
1	1	2	total

The final row reports one line unique to the first file, one unique to the second, and two common lines. If you suppress columns, the total row follows the selected output shape.

6. Handle streams and unusual delimiters

One input may be standard input, written as a single -. Both inputs cannot be standard input, because comm needs two independent streams:

$ LC_ALL=C comm -12 - sorted-b.txt < sorted-a.txt
banana
cherry

The -z or --zero-terminated option changes the record delimiter from newline to NUL. Use it only when both inputs are NUL-delimited and the receiving command understands that format:

$ LC_ALL=C comm -z -12 sorted-a.z sorted-b.z | tr '\0' '\n'
banana
cherry