Convert Git submodule to subtree

Question

How can I convert a git submodule (with a folder in the local file system as a remote) to a git subtree, preferably preserving the commit history of the submodule?

Answer 1

Alexander Mikhailian's script modified by @GaspardP did not work for me.

I modified it, and made an improvement. Now, the new subtrees will point to the same commit as the old submodules did. Previously, the script would simply download the latest commit from the target repositories, potentially causing compatibility issues.

https://gist.github.com/Nikita240/0c98cea8f53a15e69699cd8bc40657c4

#!/bin/bash -x
# This script will convert all your git submodules into git subtrees.
# This script ensures that your new subtrees point to the same commits as the
# old submodules did, unlike most other scripts that do this.
# THIS SCRIPT MUST BE PLACED OUTSIDE OF YOUR REPOSITORY!!!!!!!!!!
# Otherwise, the script will interfere with the git commits.
# Save the script in your home directory as `~/subtrees.sh`
# `cd` into your repository
# Run `~/subtrees.sh`
# Enjoy!

# extract the list of submodules from .gitmodule
cat .gitmodules |while read i
do
if [[ $i == \[submodule* ]]; then
    echo converting $i

    read i

    # extract the module's prefix
    mpath=$(echo $i | grep -E "(\S+)$" -o)

    echo path: $mpath

    read i

    # extract the url of the submodule
    murl=$(echo $i|cut -d\= -f2|xargs)

    echo url: $murl

    # extract the module name
    mname=$(basename $mpath)

    echo name: $mname

    # extract the referenced commit
    mcommit=$(git submodule status $mpath | grep -E "\S+" -o | head -1)

    echo commit: $mcommit

    # deinit the module
    git submodule deinit $mpath

    # remove the module from git
    git rm -r --cached $mpath

    # remove the module from the filesystem
    rm -rf $mpath

    # commit the change
    git commit -m "Removed $mpath submodule at commit $mcommit"

    # add the remote
    git remote add -f $mname $murl

    # add the subtree
    git subtree add --prefix $mpath $mcommit --squash

    # commit any left over uncommited changes
    git commit -a -m "$mname cleaned up"

    # fetch the files
    git fetch $murl master

    echo
fi
done
git rm .gitmodules
git commit -a -m "Removed .gitmodules"

Answer 2

The following bash script is based on Alexander Mikhailian's post (http://mikhailian.mova.org/node/233). I modified it slightly to call subtree add instead of read-tree. It will fetch the list of submodules from the .gitmodule and extract the module's prefix, name and url. It then removes each submodule and adds them back as a subtree at the same location. It also adds the remote of each submodule as a remote so you can update the subtree by providing its name instead of its url later on (i.e. git subtree pull -P Foo Foo master --squash instead of git subtree pull -P Foo https://example.com/foo.git master --squash)

You may remove the --squash argument if you want to import the full history of the subtree to your repository. With --squash, will only import the head of the subtree into your repository. This should probably be what most people want.

For more information, you may want to read this post by atlassian: http://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/

#!/bin/bash -x
# extract the list of submodules from .gitmodule
cat .gitmodules |while read i
do
if [[ $i == \[submodule* ]]; then
    echo converting $i

    # extract the module's prefix
    mpath=$(echo $i | cut -d\" -f2)

    # skip two lines
    read i; read i;

    # extract the url of the submodule
    murl=$(echo $i|cut -d\= -f2|xargs)

    # extract the module name
    mname=$(basename $mpath)

    # deinit the module
    git submodule deinit $mpath

    # remove the module from git
    git rm -r --cached $mpath

    # remove the module from the filesystem
    rm -rf $mpath

    # commit the change
    git commit -m "Removed $mpath submodule"

    # add the remote
    git remote add -f $mname $murl

    # add the subtree
    git subtree add --prefix $mpath $mname master --squash

    # fetch the files
    git fetch $murl master
fi
done
git rm .gitmodules