Why ls Reports a Directory as 4096 Bytes and Never Shrinks It
Run ls -ld on any directory on ext4 and you will see a size of 4096, whether it holds one file or none. Fill it with a million files, delete them all, and it is still huge. That number is not the size of the contents, and it is not a bug.
A directory is a file full of names
On Unix a directory is a special file whose data is a list of (name, inode number) records. The size field in its inode is the length of that list in bytes, as the filesystem stores it. ls just prints what stat returns.
ext4 allocates that list in whole blocks, and the usual block size is 4096 bytes. A brand new directory holds . and .., which needs one block. So the minimum is 4096, and the size is always a multiple of the block size.
$ mkdir demo
$ stat -c '%s bytes, %b blocks of %B' demo
4096 bytes, 8 blocks of 512
The %b count is in 512-byte units, so 8 of them is one 4096-byte block.
It grows when the entries stop fitting
Each entry costs a small fixed header plus the name, rounded up. Longer names mean fewer entries per block. Once a block is full, ext4 adds another, and the size steps from 4096 to 8192 and so on.
$ cd demo
$ for i in $(seq 1 100000); do : > "file-number-$i"; done
$ stat -c '%s' .
Expect a few megabytes, depending on name length. The exact figure varies, so measure it on your own machine rather than trusting mine.
Deleting files does not give the blocks back
Now remove everything and look again.
$ rm -- file-number-*
$ ls -A | wc -l
0
$ stat -c '%s' .
The size is unchanged. ext4 never truncates a directory when entries are removed. Unlinking a name just folds its record into the previous one (the free space is absorbed into the neighbour's record length) or marks it unused. The block stays attached to the directory.
The reasoning is plausible enough: shrinking would mean moving records around while other processes may be reading the directory, and directories that grow large tend to grow large again. But the result surprises people.
Quick detour: other filesystems answer differently
Hang on, this is interesting. The "size" of a directory is whatever the filesystem decides to report, and POSIX leaves it unspecified.
- ext4 reports blocks of directory data, so multiples of the block size.
- tmpfs reports a number derived from how many entries it holds, so it does shrink as you delete.
- btrfs reports a value based on the lengths of the names inside, with no block rounding.
- XFS reports small figures for tiny directories, because a few entries can live inside the inode itself.
So never use a directory's st_size to mean "how much is in here". Use du for disk usage and ls | wc -l style counting for entries. Even du will count the retained directory blocks.
Does it matter?
Mostly no. A few megabytes of leftover directory is cheap. It matters in two cases:
- A spool or cache directory that once held millions of files. Reading it (
ls,readdirin your program) has to walk every leftover block, so listing an empty directory can be oddly slow. - Backups or copy tools that preserve sparse or large directory metadata, where the oddity shows up in size reports.
The slow listing on a "empty" directory is the classic symptom: the first sign is usually a cron job that scans a queue directory and takes seconds to find nothing.
How to reclaim it
The simple route is to make a fresh directory and swap it in. Directories are cheap to create, and the new one starts at 4096 bytes.
$ mkdir queue.new
$ chmod --reference=queue queue.new
$ chown --reference=queue queue.new
$ mv queue queue.old && mv queue.new queue
$ rmdir queue.old
If anything holds the old directory open, or is in the middle of writing into it, do this while it is stopped; two mv calls are not one atomic step. If other files remain in the old directory, move them across before the rmdir.
The other option is offline: run e2fsck -fD on the unmounted filesystem. The -D flag optimises directories, which compacts and reindexes them. It is a heavier tool and needs the filesystem unmounted, so for a single directory the swap above is nearly always easier.
Checking your own filesystem
If you are unsure what a given mount does, try it. Make a directory on that mount, add a few thousand files, delete them, and compare stat -c %s at each step. It takes under a minute and tells you more than the documentation, since this behaviour is a property of the filesystem rather than of ls.