Rewrite History Safely with git filter-branch

git filter-branch can strip one file from every commit in your history, but it rewrites commit IDs as it goes. Somebody committed a secrets file and now it is sitting in every commit downstream, which is exactly when this tool earns its keep. You will finish with a tested command that removes the path, a way to check the result, and a recovery route if the rewrite goes wrong. The examples use Git 2.43.0 from Ubuntu package git-man 1:2.43.0-1ubuntu7.3.

• Time: 15 to 30 minutes for a small repository, plus time to review the result.
• You need: a clean working tree, a full local clone and enough free disk space for temporary objects.
• Warning: this changes commit IDs and is not routine cleanup. Do not run it against a published repository until every consumer has agreed to the rewritten history.

1. Decide whether this is even the right tool

The installed manual opens with its own warning: git filter-branch is slow and has pitfalls that cannot be fixed without breaking compatibility. For a new, large-scale rewrite, look at git filter-repo instead, the replacement the Git documentation itself names. This guide covers filter-branch because it is installed here and still earns its keep for a controlled, existing workflow.

Do not use history rewriting to fix one recent commit. A plain correction commit, git revert, or a carefully scoped rebase is easier to review and distribute. Carry on only when the old objects genuinely must stop appearing in the rewritten refs.

2. Make a disposable working copy

Start from a fresh clone or a filesystem copy, and leave the original untouched until the result has passed review. Then check the repository's state and note the refs you intend to rewrite:

$ git status --short
$ git branch --show-current
$ git tag --list
$ git show-ref

git filter-branch rewrites only the positive refs its revision arguments select. A command ending in HEAD affects the current branch; one ending in -- --all passes --all to revision selection and pulls in every ref the walk can reach. The -- separator matters here: everything before it belongs to filter-branch, everything after it belongs to git rev-list.

3. Remove a path with the faster filter

For a file that should vanish from every selected commit, use an index filter. It edits the index without checking out every tree, so it is normally much faster than a tree filter:

$ FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch -f \
    --index-filter 'git rm --cached --ignore-unmatch -- secrets.txt' \
    --prune-empty --tag-name-filter cat -- --all

• Replace secrets.txt with the repository-relative path you have confirmed.
• --ignore-unmatch lets the filter succeed in commits where the path never existed.
• The -- after git rm stops a path beginning with a hyphen from being read as an option.
• --prune-empty removes commits left empty, as long as they have no more than one non-pruned parent.
• --tag-name-filter cat updates tag names pointing into rewritten history, but it does not preserve signed tag signatures: rewritten tag objects cannot keep their originals. Review tags separately if signatures or release automation matter to you.

The command is destructive to the working copy's refs and can take a long time. It does not need sudo; use elevated privileges only to fix a genuine filesystem permission problem in a repository you control, never to make a rewrite feel safer than it is. Expect progress lines and a final line like this:

Ref 'refs/heads/main' was rewritten

4. Verify the new history

Check the current tree, search every rewritten ref for the path, and inspect the commit graph:

$ git ls-tree -r --name-only HEAD | grep -F -- 'secrets.txt'
$ git log --all --name-only --format= -- secrets.txt
$ git log --graph --decorate --oneline --all
$ git fsck --no-reflogs --unreachable

The first two commands should produce no path output. The graph should still show the commits and merges you expect, allowing for commits emptied by the removal. git fsck may report unreachable objects: filter-branch keeps the old tips under refs/original/, and reflogs can retain old objects too. Unreachable objects here are not proof the rewrite failed.

$ git for-each-ref --format='%(refname) %(objectname:short)' refs/original/
$ git diff --stat refs/original/refs/heads/main main

The backup ref name follows the original, so substitute your branch name if it is not main. Compare representative commits, file lists and tags, not just the tip hash.

5. Recover if the rewrite is wrong

Do this before deleting backups or expiring reflogs. If the current branch is wrong, restore it from its original ref:

$ git update-ref refs/heads/main refs/original/refs/heads/main
$ git checkout main
$ git log --oneline -5

Do not run the cleanup below until the rewritten repository has been reviewed and a separate backup exists. Removing original refs, expiring reflogs and pruning objects is irreversible in ordinary Git recovery terms:

$ git for-each-ref --format='%(refname)' refs/original/ \
    | while IFS= read -r ref; do git update-ref -d "$ref"; done
$ git reflog expire --expire=now --all
$ git gc --prune=now

A safer route to a smaller repository is to clone the reviewed result with a file:// URL, which avoids carrying over the old unreachable objects that a plain local-path clone would keep. Hold on to the original clone until the new one has been checked.

Common traps

• A file was renamed at some point. Search historical names before filtering, or an old copy of the content will remain under its previous name.
• A positive ref slips past a revision limiter. Read the selected range carefully, especially when mixing a branch name with path arguments.
• Shell filters run once per commit. Quote paths and test the filter on a disposable clone first: a non-zero filter status aborts the whole operation.
• Tree filters check out every commit and ignore ignore rules. Prefer an index filter whenever the change is a path removal or rename that does not need file contents inspected.
• Force-pushing rewritten history casually breaks collaborators. They must rebase, reclone or explicitly replace their old refs, and signed tags need a separate release decision.