Articles / Linux

697 words

  • linux
  • xargs
  • find
  • shell
  • sysadmin
  • debugging

Why xargs Mangles Filenames with Spaces, and the -0 Fix

Pipe find into xargs and it works for years, right up until someone saves a file called Q3 report.pdf. Then your cleanup script either fails noisily or, worse, quietly acts on the wrong files. The cause is simple: xargs never sees filenames, only a stream of text it has to guess how to split.

Reproducing it in ten seconds

Make a scratch directory with one awkward name and one ordinary one:

mkdir demo && cd demo
touch "my notes.txt" plain.txt
find . -name '*.txt' | xargs ls -l

You get something like this (GNU findutils):

ls: cannot access './my': No such file or directory
ls: cannot access 'notes.txt': No such file or directory
-rw-r--r-- 1 andy andy 0 Oct  8 10:00 ./plain.txt

xargs read ./my notes.txt, decided the space was a separator, and handed ls two arguments that do not exist. The same thing happens with rm, which is where it stops being funny.

The default rules are odder than "split on spaces"

By default xargs splits on blanks and newlines. It also interprets single quotes, double quotes and backslashes, a bit like a very small shell. That makes the failures more varied than you might expect.

touch "it's here.txt"
find . -name '*.txt' | xargs ls -l

This one fails outright:

xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option

That error message is the manual telling you the answer. Spaces give wrong arguments; stray quotes abort the run; a backslash in a name gets eaten. Anything the shell would treat specially in a filename, default xargs treats specially too.

The fix: NUL-separated input

Have find end each name with a NUL byte, and tell xargs to split on only that:

find . -name '*.txt' -print0 | xargs -0 ls -l

With -0, quotes and backslashes lose their special meaning too, so all three problems above vanish at once. Each name arrives as exactly one argument, byte for byte.

Quick detour: why NUL?

A Unix filename can contain any byte except two: / and NUL. Slash is the path separator, and NUL ends C strings, so the kernel cannot store it in a name. A path as a whole therefore can never contain NUL, which makes it the only truly safe delimiter. Newline looks safe and is not, because touch $'a\nb.txt' is perfectly legal.

Why not just split on newlines?

GNU xargs has -d '\n' (and -I{} behaves similarly). It fixes spaces and quotes, and plenty of people stop there.

find . -name '*.txt' | xargs -d '\n' ls -l

It is fine for your own tidy directories. It still breaks on a name containing a newline, and -d is not in POSIX, so it will not work on a stock BSD or macOS xargs. -0 works on all of them.

Other tools that speak NUL

The pipeline only stays safe if every stage uses the same delimiter. Several common tools have a switch for it:

  • grep -lZ pattern ... (or --null) prints matching filenames NUL-terminated.
  • git ls-files -z lists tracked files the same way.
  • sort -z sorts NUL-separated records.
  • printf '%s\0' "$@" builds a NUL list from shell arguments.

For example, compressing every log that mentions a given error:

grep -rlZ 'connection reset' /var/log/myapp | xargs -0 -r gzip --

Three small pitfalls that survive -0

  1. Empty input still runs the command on GNU xargs. find matches nothing, xargs rm runs with no arguments, and you get an error (or worse, a command with a harmless-looking default). Add -r (--no-run-if-empty). BSD does this by default, which is exactly why scripts behave differently between machines.
  2. Names starting with a dash look like options. find . prefixes results with ./, which saves you, but lists from other sources may not. End the options with -- as in the gzip -- above.
  3. Large lists get split across several runs because of the argument size limit. That is normal, but it means a command like tar czf out.tgz will overwrite its archive on each batch. Use tar --null -T - instead, which reads the list itself.

Or skip xargs entirely

If all you need is "run this command on every match", find can batch arguments itself, with no text parsing in between:

find . -name '*.txt' -exec ls -l {} +

The + form packs many names into each invocation, just as xargs does. Reach for xargs -0 when you need something -exec cannot do, such as -P 4 to run four jobs in parallel, or when the list comes from a tool other than find.

The habit worth keeping is small: whenever a filename list crosses a pipe, make both sides agree on NUL, and add -r while you are there.

← All articles