How to know local repo is different from remote repo, without fetch?

Question

I got tens of repos, my script should update them if any difference happened, new commits, new tag, new branch. Fetch is kind of slow for tens of repos in my case, I'd like to know if there is any quick command could meet my requirement.

Answer 1

You can use the git ls-remote plumbing command to obtain the state of the remotes without fetch.

Here, let’s use git itself as a lightweight database, to keep track of the state of the remote.

Put the following in a script; you can enable it later as a git alias shell function for convenience. Run inside your repo.

REMOTE_SUM=$(git ls-remote --tags --heads 2>/dev/null | git hash-object --stdin)
if git cat-file -e $REMOTE_SUM
then
    echo Remote check-summed up-to-date.
else
    echo Remote changed, fetching...
    git ls-remote --tags --heads 2>/dev/null | \
        git hash-object -w --stdin &>/dev/null
    git fetch
fi

_{Some of the necessary error checking was omitted, and code was duplicated for sake of clarity.}

Explanation

Listing all the remote tips with git ls-remote --tags --heads generates output such as:

From /home/user/tmp/repo2
777201715768a4d82f374f7224e68164a916ac1f        refs/heads/bar
78981922613b2afb6025042ff6bd878ac1994e85        refs/heads/master
...

In turn we hash the above picture of the remote repo as a single hash via git hash-object --stdin and check if we've previously seen it by querying for the hash in git with git cat-file -e. If we haven't seen it, the remote picture must have changed, and we record it first in git with git hash-object -w, to accommodate races between pulling and committing on the remote, and then proceed to fetch the remote.

One can integrate this with a git pre-fetch functionality: pre-fetch hook functionality in git, but that's out of the scope of this answer.

Addendum

Note that the above will generate loose objects in git that occasionally will need to be garbage collected with git gc, and possibly --prune explicitly.

Further, the above should work as long as commits are not rearranged on purpose in such a way that branch tips remain the same. This would be /quite uncommon/ and goes against git guidelines of changing pushed state, but hey, the worst thing that can happen is that you skip a fetch.

Also note that ls-remote works on a single remote. To work with multiple remotes, you'll have to extend the script by generating a list of remotes with git remote show and work with each one in turn.

Answer 2

You don't have access to the origin server

You can't using only git.

EDIT

As per another answer, git ls-remote may be of use to you.

However, as you would have to ls-remote all the repositories, if your problem is a network lag, it won't be solved with ls-remote.

---

You have access to the origin server

1. Setup a hook when there is a write on the repo on the server. The hook would mark the repo as being modified. You can for instance create a repo_name__MODIFIED file somewhere on the server).
2. Before updating the repo, check if the repo is modified. For the given example, check if the file repo_name__MODIFIED exist on the server.
3. If the repo was modified, before updating the repo, mark it as unmodified (just before the fetch). In the case of our example, simply delete the repo_name__MODIFIED file on the server.

---

Note

Why is the fetch so long? git will only fetch the new commits, if there are no modifications to the origin, it should be very fast!

Answer 3

If you have set up your local repositories using git clone or git remote add $REMOTE_NAME $REMOTE_URL, all the information that you need to compare your local branches to their remote counterparts (at the time of your last git fetch) is already there.

If you have set up remote tracking branches, git status tells you if about differences to the remote branch you are tracking, like this.

$ git status
On branch master
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   perl/Makefile.am

no changes added to commit (use "git add" and/or "git commit -a")

There's also a parseable form:

$ git status --porcelain -b
## master...origin/master [behind 1]
 M perl/Makefile.am