An addition to the accepted answer, if your mistakenly-added file was huge, you'll probably notice that, even after removing it from the index with 'git reset', it still seems to occupy space in the .git directory.
This is nothing to be worried about; the file is indeed still in the repository, but only as a "loose object". It will not be copied to other repositories (via clone, push), and the space will be eventually reclaimed - though perhaps not very soon. If you are anxious, you can run:
git gc --prune=now
Update (what follows is my attempt to clear some confusion that can arise from the most upvoted answers):
So, which is the real undo of git add?
git reset HEAD <file> ?
or
git rm --cached <file>?
Strictly speaking, and if I'm not mistaken: none.
git add cannot be undone - safely, in general.
Let's recall first what git add <file> actually does:
If <file> was not previously tracked, git add adds it to the cache, with its current content.
If <file> was already tracked, git add saves the current content (snapshot, version) to the cache. In Git, this action is still called add, (not mere update it), because two different versions (snapshots) of a file are regarded as two different items: hence, we are indeed adding a new item to the cache, to be eventually committed later.
In light of this, the question is slightly ambiguous:
I mistakenly added files using the command...
The OP's scenario seems to be the first one (untracked file), we want the "undo" to remove the file (not just the current contents) from the tracked items. If this is the case, then it's ok to run git rm --cached <file>.
And we could also run git reset HEAD <file>. This is in general preferable, because it works in both scenarios: it also does the undo when we wrongly added a version of an already tracked item.
But there are two caveats.
First: There is (as pointed out in the answer) only one scenario in which git reset HEAD doesn't work, but git rm --cached does: a new repository (no commits). But, really, this a practically irrelevant case.
Second: Be aware that git reset HEAD can't magically recover the previously cached file contents, it just resynchronises it from the HEAD. If our misguided git add overwrote a previous staged uncommitted version, we can't recover it. That's why, strictly speaking, we cannot undo [*].
Example:
$ git init
$ echo "version 1" > file.txt
$ git add file.txt # First add of file.txt
$ git commit -m 'first commit'
$ echo "version 2" > file.txt
$ git add file.txt # Stage (don't commit) "version 2" of file.txt
$ git diff --cached file.txt
-version 1
+version 2
$ echo "version 3" > file.txt
$ git diff file.txt
-version 2
+version 3
$ git add file.txt # Oops we didn't mean this
$ git reset HEAD file.txt # Undo?
$ git diff --cached file.txt # No dif, of course. stage == HEAD
$ git diff file.txt # We have irrevocably lost "version 2"
-version 1
+version 3
Of course, this is not very critical if we just follow the usual lazy workflow of doing 'git add' only for adding new files (case 1), and we update new contents via the commit, git commit -a command.
* (Edit: the above is practically correct, but still there can be some slightly hackish/convoluted ways for recovering changes that were staged, but not committed and then overwritten - see the comments by Johannes Matokic and iolsmit)
