Remove Duplicate Dovecot Mail with doveadm deduplicate

Your Dovecot mailbox has the same message twice, and doveadm deduplicate will delete one copy for good. This guide shows you how to inspect first, back up, then expunge one mailbox only, in 15 to 30 minutes.

It covers Dovecot 2.3.21 from package dovecot-core version 1:2.3.21+dfsg1-2ubuntu6.5. You need shell access to the Dovecot host and permission to read the mailbox. Budget extra time for a backup or snapshot if your storage offers one.

• Which copy goes: Dovecot expunges the newer copy and keeps the older one.
• No undo: the command has no undo option. Recovery means restoring from a backup.

1. Confirm the installed command

Check which binary will run and read the local synopsis. These checks are ordinary and do not alter mail:

$ command -v doveadm
/usr/bin/doveadm
$ dpkg-query -W -f='${Package} ${Version}\n' dovecot-core
dovecot-core 1:2.3.21+dfsg1-2ubuntu6.5
$ doveadm help deduplicate | sed -n '1,24p'
DOVEADM-DEDUPLICATE(1)              Dovecot             DOVEADM-DEDUPLICATE(1)

NAME
       doveadm-deduplicate - expunge duplicate messages

Do not trust an example copied from another host without checking its version. This guide follows the installed 2.3 command. Newer Dovecot documentation can add options or change which user-selection forms are required.

2. Choose one mailbox and one user

Start with a single mailbox. Deduplication across multiple mailboxes is not supported, so a query that matches several folders does not become a cross-folder job. Put the real values in shell variables so the command is easy to review:

USER='[email protected]'
MAILBOX='Archive.2025'

• Use -u for the first run. The installed 2.3 manpage accepts it with a user name or mask.
• Leave -A and -F out. -A means every user and -F means one username per line. Both depend on correct user database iteration, and -A can include system users when the passwd user database is used.
• Use your usual account. On a managed server that is commonly a Dovecot administrator account or root. Do not add sudo until a permission error requires it.
• Keep -S for special cases. It connects through a local or TCP doveadm socket. Use it only when your deployment explicitly requires a remote administrative path.

3. Inspect candidates without changing mail

Before expunging anything, fetch the message GUID and UID for the chosen mailbox and sort the result. This is the read-only inspection pattern from the installed manpage:

$ doveadm -f table fetch -u "$USER" 'guid uid' mailbox "$MAILBOX" | sort
guid                             uid
8aad0f0a30169f4bea620000ca356bad 18751
8aad0f0a30169f4bea620000ca356bad 18756
923e301ab9219b4b4f440000ca356bad 18748
923e301ab9219b4b4f440000ca356bad 18753
...

Repeated GUIDs are the signal for the default mode. In the example, each repeated GUID marks a pair of messages with the same stored message identity. The UID is mailbox-local and only helps you spot which records are involved. It is not what default matching uses.

A real mailbox will show different GUIDs and UIDs. If the command fails, check these:

• Names. The exact user and mailbox name.
• Authentication. Access to the doveadm service.
• Permissions. Read access to the mailbox.

An empty result does not prove the mailbox is clean. It may mean the query or mailbox selection was wrong.

4. Decide what counts as a duplicate

Without extra options, the installed command matches on message GUID. That is the safer default when accidental duplication copied the same stored message. It deletes the newest duplicate and keeps the oldest.

The -m option switches the comparison to the Message-Id header:

$ doveadm deduplicate -m -u "$USER" mailbox "$MAILBOX"

To investigate Message-ID values before choosing -m, use a read-only fetch that includes the header and test it on a small, explicitly selected mailbox. The deduplicate manpage guarantees the matching mode, but it does not define a backup or restore mechanism.

5. Make a recovery point

A copy of the mailbox is not a backup unless it is independent and restorable. A second view of the same underlying storage will not save you from an accidental expunge. If you have no tested recovery point, stop after inspection and arrange one.

There is no rollback command for doveadm deduplicate. Recovery means restoring the affected mailbox from your backup or snapshot, which may overwrite newer legitimate changes. Record the snapshot identifier and the exact command you intend to run.

6. Run a narrowly scoped deduplication

Review the user, mailbox and matching mode one last time. Then run the default GUID-based operation for that one mailbox:

$ doveadm deduplicate -u "$USER" mailbox "$MAILBOX"
$ printf 'exit status: %s\n' "$?"
exit status: 0

The command may print little or nothing. Status 0 means the operation completed, not that it removed any particular number of messages. Add the global -v option when you want the documented verbosity and progress counter:

$ doveadm -v deduplicate -u "$USER" mailbox "$MAILBOX"

7. Verify the result and investigate failures

Run the same read-only fetch again:

$ doveadm -f table fetch -u "$USER" 'guid uid' mailbox "$MAILBOX" | sort
guid                             uid
8aad0f0a30169f4bea620000ca356bad 18751
923e301ab9219b4b4f440000ca356bad 18748
a7999e1530739c4bd26d0000ca356bad 18749
...

In GUID mode, the repeated GUIDs from before should no longer have multiple UIDs in this mailbox. Check the mailbox count and a sample of recent mail through your normal IMAP or monitoring tools. If an expected message is missing, stop making changes and use the recorded recovery point.

A non-zero status needs diagnosis before you retry. Recheck the user selector, mailbox spelling, service availability and permissions. If you are running against another Dovecot instance, verify the -S socket or host and port. Retrying blindly makes an incident harder to reconstruct.