Tracking 3rd party code with Git

Question

I can\'t seem to grok the different solutions I\'ve found and studied for tracking external code. Let alone understand how to apply them to my use case...

Would you

Answer 1

There are two separate problems here:

1. How do you maintain local forks of remote projects, and
2. How do you keep a copy of remote projects in your own tree?

Problem 1 is pretty easy by itself. Just do something like:

git clone git://example.com/foo.git
cd foo
git remote add upstream git://example.com/foo.git
git remote rm origin
git remote add origin ssh://.../my-forked-foo.git
git push origin

You can then work on your forked repository normally. When you want to merge in upstream changes, run:

git pull upstream master

As for problem 2, one option is to use submodules. For this, cd into your main project, and run:

git submodule add ssh://.../my-forked-foo.git local/path/for/foo

If I use git submodules, what do I need to know?

You may find git submodules to be a little bit tricky at times. Here are some things to keep in mind:

1. Always commit the submodule before committing the parent.
2. Always push the submodule before pushing the parent.
3. Make sure that the submodule's HEAD points to a branch before committing to it. (If you're a bash user, I recommend using git-completion to put the current branch name in your prompt.)
4. Always run 'git submodule update' after switching branches or pulling changes.

You can work around (4) to a certain extent by using an alias created by one of my coworkers:

git config --global alias.pull-recursive '!git pull && git submodule update --init'

...and then running:

git pull-recursive

If git submodules are so tricky, what are the advantages?

1. You can check out the main project without checking out the submodules. This is useful when the submodules are huge, and you don't need them on certain platforms.
2. If you have experienced git users, it's possible to have multiple forks of your submodule, and link them with different forks of your main project.
3. Someday, somebody might actually fix git submodules to work more gracefully. The deepest parts of the submodule implementation are actually quite good; it's just the upper-level tools that are broken.

git submodules aren't for me. What next?

If you don't want to use git submodules, you might want to look into git merge's subtree strategy. This keeps everything in one repository.

What if the upstream repository uses Subversion?

This is pretty easy if you know how to use git svn:

git svn clone -s https://example.com/foo
cd foo
git remote add origin ssh://.../my-forked-foo.git
git push origin

Then set up a local tracking branch in git.

git push origin master:local-fork
git checkout -b local-fork origin/local-fork

Then, to merge from upstream, run:

git svn fetch
git merge trunk

(I haven't tested this code, but it's more-or-less how we maintain one submodule with an upstream SVN repository.)

Don't use git svn rebase, because it will make it very difficult to use git submodule in the parent project without losing data. Just treat the Subversion branches as read-only mirrors of upstream, and merge from them explicitly.

If you need to access the upstream Subversion repository on another machine, try:

git svn init -s https://example.com/foo
git svn fetch

You should then be able to merge changes from upstream as before.

Answer 2

I use git submodules to track reusable apps in my Django projects, but it is kind of messy in the long run.

It is messy for deployment because you can't get a clean archive of the whole tree (with submodules) using git archive. There are some tricks, but nothing perfect. Besides, the submodule update mecanism is not that good for working with submodules branches.

You might have to take a look at virtualenv and pip, because they had some recent improvements in order to work with external repositories.

pip : http://pip.openplans.org/ and working with pip/virtualenv : http://www.b-list.org/weblog/2008/dec/15/pip/

Answer 3

I think my answer to to another questions gives exactly a nice solution for the problem described here, without going in to the hell of submodules (which I have tried, but does not even get close to the svn:externals I was used to)

Still, have a look at this answer: Do you version control the invidual apps or the whole project or both?

Before deleting my answer again, I was not aware I couldn't copy my own answer to another question, even if I Am convinced it is usefull as an answer. Sorry, but give this answer a try, it really is a nice solution. So I hope I Am allowed to refer to my own anser to another question.

Answer 4

I've looked around a bit more and stumbled upon Braid. It's a tool that automates vendor branches in Git. It can use Git or SVN repos.

By crawling through the source I found out that it uses the subtree strategy. And seems to make it really simple! Plus, it seems to fulfill all my requirements!

Before I jump in and use it: does anyone here have any experience with Braid? I would like to find out about possible cons if there are any. Also, if you haven't used Braid, but have some expertise in Git, what do you think about it, at first sight?