Skip processing of Git revisions in post-receive hook that have already been previously processed

Question

I have a git post-receive hook that extracts all the revisions that were added during a "git push" and does some processing on each one (such as sending notification emails). This works great except when merging; e.g.:

1. I make some commits on branch1 and then push branch1. The post-receive hook processes the commits correctly.
2. I merge branch1 into branch2 and then push branch2. The post-receive hook processes all the merged commits a second time.

How can I avoid this? Below is the beginning of my post-receive hook where I extract the commits that should be processed (at the end $COMMITS holds the list of commits to process).

#!/bin/sh

REPO_PATH=`pwd`
COMMITS=''

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# for each ref that was updated during the push
while read OLD_REV NEW_REV REF_NAME; do
  OLD_REV="`git rev-parse $OLD_REV`"
  NEW_REV="`git rev-parse $NEW_REV`"
  if expr "$OLD_REV" : '0*$' >/dev/null; then
    # if the branch was created, add all revisions in the new branch; skip tags
    if ! expr "$REF_NAME" : 'refs/tags/' >/dev/null; then
      REF_REV="`git rev-parse $REF_NAME`"
      REF_NAME="`git name-rev --name-only $REF_REV`"
      COMMITS="$COMMITS `git rev-list $REF_NAME | git name-rev --stdin | grep -G \($REF_NAME.*\) | awk '{ print $1 }' | tr '\n' ' '`"
    fi

  elif expr "$NEW_REV" : '0*$' >/dev/null; then
    # don't think branch deletes ever hit a post-receive hook, so we should never get here
    printf ''
  else
    # add any commits in this push
    COMMITS="$COMMITS `git rev-parse --not --all | grep -v $(git rev-parse $REF_NAME) | git rev-list --reverse --stdin $(git merge-base $OLD_REV $NEW_REV)..$NEW_REV | tr '\n' ' '`"
  fi
done

Answer 1

Look at $(prefix)/share/git-core/contrib/hooks/post-receive-email, which does just what (I think) you want. Basically it uses git for-each-ref to find the names of all branches, and then exclude every commit that's reachable from some branch other than the one being updated:

if [ "$change_type" = create ]
then
    # Show all revisions exclusive to this (new) branch.
    revspec=$newrev
else
    # Branch update; show revisions not part of $oldrev.
    revspec=$oldrev..$newrev
fi

other_branches=$(git for-each-ref --format='%(refname)' refs/heads/ |
     grep -F -v $refname)
git rev-parse --not $other_branches | git rev-list --pretty --stdin $revspec

(I've simplified it here, and hopefully not damaged anything in my cut-and-paste job. The inputs here are: $change_type is create if $oldrev is all-zeros, otherwise it's update; $oldrev is the old rev SHA1 from the line recently-read from stdin; $newrev is the new rev SHA1; and $refname is the full name, e.g., refs/heads/topic.)

Answer 2

What we do is to keep the hash of the previously processed commits in a text file. Every time the hook runs, it looks in that file to check if a given commit has already been processed or not. If it did not process that commit yet, process it and then log that commit to the file.

This is not very scalable, as the text files would only grow as more commits are added to the repository and the time to check for a given commit would also grow.

Answer 3

We did this by having the post-receive hook stop processing when it encountered a merge commit (a commit with two or more parents). This requires a bit of discipline when pushing merges to ensure that other "real" commits aren't thrown out. The discipline is to always push before merging and then push the merge separately.

Answer 4

I implemented this completely in a post-receive hook. It notifies trac of only new commits since the last fetch without duplicating, regardless of whether the new commits were pushed to a single branch or multiple branches at the same time. This method keeps a file called TRAC_HEAD in your git directory for tracking which commits have already been processed.

It is recommended that you run cat refs/heads/* > TRAC HEAD in your .git directory before enabling the hook.

#!/bin/sh
#
# Reads and notifies trac of only new commits that have not yet been dealt with.
#
# The "post-receive" script is run after receive-pack has accepted a pack
# and the repository has been updated.  It is passed arguments in through
# stdin in the form
#  <oldrev> <newrev> <refname>
# For example:
#  aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refs/heads/master
#

TRAC_PATH="/path/to/trac/env"

# Read the standard input
while read oldrev newrev refname ; do

        echo "Processing branch: $refname"

        # Read the last revisions for each branch from TRAC_HEAD
        exclusions=$(cat TRAC_HEAD | uniq |  sed -e 's/^/^/' -e 's/ / ^/g' | xargs echo)

        echo "Exclusion list: $exclusions"

        git rev-list --reverse $newrev $exclusions | while read rev ; do
                trac-admin $TRAC_PATH changeset added '(default)' $rev
                echo "Processed: $rev"
        done

        # Add to the exclusions file the latest revision from this branch
        echo $newrev >> TRAC_HEAD

done

# Update the TRAC_HEAD file
cat refs/heads/* > TRAC_HEAD