Remove sensitive files and their commits from Git history

Question

I would like to put a Git project on GitHub but it contains certain files with sensitive data (usernames and passwords, like /config/deploy.rb for capistrano).

I kno

Answer 1

Changing your passwords is a good idea, but for the process of removing password's from your repo's history, I recommend the BFG Repo-Cleaner, a faster, simpler alternative to git-filter-branch explicitly designed for removing private data from Git repos.

Create a private.txt file listing the passwords, etc, that you want to remove (one entry per line) and then run this command:

$ java -jar bfg.jar  --replace-text private.txt  my-repo.git

All files under a threshold size (1MB by default) in your repo's history will be scanned, and any matching string (that isn't in your latest commit) will be replaced with the string "***REMOVED***". You can then use git gc to clean away the dead data:

$ git gc --prune=now --aggressive

The BFG is typically 10-50x faster than running git-filter-branch and the options are simplified and tailored around these two common use-cases:

• Removing Crazy Big Files
• Removing Passwords, Credentials & other Private data

Full disclosure: I'm the author of the BFG Repo-Cleaner.

Answer 2

To be clear: The accepted answer is correct. Try it first. However, it may be unnecessarily complex for some use cases, particularly if you encounter obnoxious errors such as 'fatal: bad revision --prune-empty', or really don't care about the history of your repo.

An alternative would be:

1. cd to project's base branch
2. Remove the sensitive code / file
3. rm -rf .git/ # Remove all git info from your code
4. Go to github and delete your repository
5. Follow this guide to push your code to a new repository as you normally would - https://help.github.com/articles/adding-an-existing-project-to-github-using-the-command-line/

This will of course remove all commit history branches, and issues from both your github repo, and your local git repo. If this is unacceptable you will have to use an alternate approach.

Call this the nuclear option.

Answer 3

I recommend this script by David Underhill, worked like a charm for me.

It adds these commands in addition natacado's filter-branch to clean up the mess it leaves behind:

rm -rf .git/refs/original/
git reflog expire --all
git gc --aggressive --prune

Full script (all credit to David Underhill)

#!/bin/bash
set -o errexit

# Author: David Underhill
# Script to permanently delete files/folders from your git repository.  To use 
# it, cd to your repository's root and then run the script with a list of paths
# you want to delete, e.g., git-delete-history path1 path2

if [ $# -eq 0 ]; then
    exit 0
fi

# make sure we're at the root of git repo
if [ ! -d .git ]; then
    echo "Error: must run this script from the root of a git repository"
    exit 1
fi

# remove all paths passed as arguments from the history of the repo
files=$@
git filter-branch --index-filter \
"git rm -rf --cached --ignore-unmatch $files" HEAD

# remove the temporary history git-filter-branch
# otherwise leaves behind for a long time
rm -rf .git/refs/original/ && \
git reflog expire --all && \
git gc --aggressive --prune

The last two commands may work better if changed to the following:

git reflog expire --expire=now --all && \
git gc --aggressive --prune=now

Answer 4

I've had to do this a few times to-date. Note that this only works on 1 file at a time.

1. Get a list of all commits that modified a file. The one at the bottom will the the first commit:

   git log --pretty=oneline --branches -- pathToFile

2. To remove the file from history use the first commit sha1 and the path to file from the previous command, and fill them into this command:

   git filter-branch --index-filter 'git rm --cached --ignore-unmatch <path-to-file>' -- <sha1-where-the-file-was-first-added>..

Answer 5

Here is my solution in windows

git filter-branch --tree-filter "rm -f 'filedir/filename'" HEAD

git push --force

make sure that the path is correct otherwise it won't work

I hope it helps