If you have a ~/.ssh/config
file, you need to use the scp-like syntax for your ssh url:
git clone debian:gitolite-admin.git
With a config like:
host debian
user gitolite
hostname debian
identityfile ~/.ssh/gitolite
port 2222
And you are not supposed to specify the full path of the repo.