I would like to run Git 1.6.x for Windows from a USB memory stick.
There is a Windows version of Git available at: http://code.google.com/p/msysgit/
Installing a
Your e-mail address, name and many other settings are stored in the global configuration file .gitconfig in your home dir.
git-config
Maybe the -f option helps you for specifing the different location of the config file.
I'm not sure, if it's the same application as mentioned, but because Google Code doesn't exist anymore, you may try the Git portable on Sourceforge instead.
When I went to the msysgit download page today, there was a download link for "PortableGit" that was new as of 22MAR09. Using this version might provide you with more of the control you need. I have not tried it, but the name suggests it's geared more toward USB-stick installations.
I think those are stored in the user part of the local copy of the repository. If they aren't, then you can always override them in the user settings within the .git directory.
You could use the portable version from http://code.google.com/p/msysgit/