If you type:
git status
Git will tell you what is staged, etc., including instructions on how to unstage:
use "git reset HEAD <file>..." to unstage
I find Git does a pretty good job of nudging me to do the right thing in situations like this.
Note: Recent Git versions (1.8.4.x) have changed this message:
(use "git rm --cached <file>..." to unstage)