git submodule svn external

Question

Let\'s say I have 3 git repositories, each with a lib and tests folder in the root. All 3 repositories are part of what I want to be a single packa

Answer 1

You can have a forth 'lib' main Git repo, with:

• lib/vendor/package content
• 3 submodules (see "true nature of submodules")

But this reference is totally independent from any 'svn:external' property you may have setup in a mirror SVN repo.

So if you have 3 SVN repos already, you can git-svn them, publish them on GitHub, and then create a fourth repo on GitHub in which you will add those 3 Git repos as submodules, following the submodules tutorial (supposing here you have all 4 repos already on GitHub)

$ mkdir -p lib/vendor/package
$ cd lib/vendor/package
$ for package in a b c d; do
$ git submodule add git://github.com/path/to/$package.git $package
$ done
$ cd ..
$ git commit -m "lib with submodules"
$ git push

Answer 2

You are right, Git submodules can not directly do exactly what you want. It works in SVN because the root of a repository, branches, and any subdirectory thereof are the same kind of object. In Git, a repository, a branch, and a directory are all distinct kinds of objects (you can not use a directory as a full repository or as a branch).

There are a couple of indirect ways to accomplish what you want though.

Using Submodules and Symlinks

The core of a Git submodule is a clone of another repository in the work tree of the “superproject”^*. Git only clones full repositories. It is not possible to clone just a single subdirectory out of an existing repository^†.

^* Normal submodules also require a special reference in the superproject's commits/index and (normally) an entry in the superproject's .gitmodules file. It is possible to have non-tracked clones of other repositories in an unrelated working tree, but such usage does not create a submodule.
^† Git 1.7.0 and later has a “sparse checkout” feature, but it would not help to relocate the lib directory the top level of each submodule clone.

You might, however be able to use Git's support for symbolic links to do something that is fairly close:

#
# Make the lib directory of each submodule appear in the superproject as
# lib/vendor/packages/$submod_name
#
# With this structure in each of the submodules (a, b, c):
#
#    lib/
#    tests/
#
# We end up with this structure in the superproject:
#
#    lib/
#        vendor/
#            packages/
#                a     (a symlink to ../../../_submodules/a/lib)
#                b     (a symlink to ../../../_submodules/b/lib)
#                c     (a symlink to ../../../_submodules/c/lib)
#    _submodules
#        a/            (a Git submodule)
#            lib/
#            tests/
#        b/            (a Git submodule)
#            lib/
#            tests/
#        c/            (a Git submodule)
#            lib/
#            tests/
#
add_one() {
    dir=lib/vendor/package
    dest="$dir/$1"
    # use fewer ".."s to put the _submodules closer to the symlinks
    s=../../../_submodules/"$1"
    git submodule add "$2" "$dir/$s"
    ln -s "$s"/lib "$dest"
    git add "$dest"
}

cd "$main_repo_toplevel"
mkdir -p lib/vendor/package
add_one a git@githost.example.com:user/package-a.git
add_one b git://public.example.com/work/package-b-dev.git
add_one c ssh://special.example.com/foo.git

Using git subtree

apenwarr's git subtree can split off and merge parts of repositories (i.e. individual subdirectories; it is a wrapper around “subtree merging” with other nice features). The first step would be to extract the history of lib in each of your sub-projects. Then, either directly use the extracted history as a submodule, or use git subtree to do a subtree merge into your main repository. Either way, this would introduce an extra step (re-extracting the lib history) before you could integrate changes from a sub-project into your main repository.