When I went to the msysgit download page today, there was a download link for "PortableGit" that was new as of 22MAR09. Using this version might provide you with more of the control you need. I have not tried it, but the name suggests it's geared more toward USB-stick installations.