Easy way to pull latest of all git submodules

Question

We\'re using git submodules to manage a couple of large projects that have dependencies on many other libraries we\'ve developed. Each library is a separate repo brought int

Answer 1

Henrik is on the right track. The 'foreach' command can execute any arbitrary shell script. Two options to pull the very latest might be,

git submodule foreach git pull origin master

and,

git submodule foreach /path/to/some/cool/script.sh

That will iterate through all initialized submodules and run the given commands.

Answer 2

On init running the following command:

git submodule update --init --recursive

from within the git repo directory, works best for me.

This will pull all latest including submodules.

Explained

git - the base command to perform any git command
    submodule - Inspects, updates and manages submodules.
        update - Update the registered submodules to match what the superproject
        expects by cloning missing submodules and updating the working tree of the
        submodules. The "updating" can be done in several ways depending on command
        line options and the value of submodule.<name>.update configuration variable.
            --init without the explicit init step if you do not intend to customize
            any submodule locations.
            --recursive is specified, this command will recurse into the registered
            submodules, and update any nested submodules within.

---

After this you can just run:

git submodule update --recursive

from within the git repo directory, works best for me.

This will pull all latest including submodules.

Answer 3

As it may happens that the default branch of your submodules is not master, this is how I automate the full Git submodules upgrades:

git submodule init
git submodule update
git submodule foreach 'git fetch origin; git checkout $(git rev-parse --abbrev-ref HEAD); git reset --hard origin/$(git rev-parse --abbrev-ref HEAD); git submodule update --recursive; git clean -dfx'

Answer 4

I don't know since which version of git this is working, but that's what you're searching for:

git submodule update --recursive

I use it with git pull to update the root repository, too:

git pull && git submodule update --recursive

Answer 5

Remark: not too easy way, but workable and it has its own unique pros.

If one want to clone only HEAD revision of a repository and only HEADs of all the its submodules (i.e. to checkout "trunk"), then one can use following Lua script. Sometimes simple command git submodule update --init --recursive --remote --no-fetch --depth=1 can result in an unrecoverable git error. In this case one need to clean up subdirectory of .git/modules directory and clone submodule manually using git clone --separate-git-dir command. The only complexity is to find out URL, path of .git directory of submodule and path of submodule in superproject tree.

Remark: the script is only tested against https://github.com/boostorg/boost.git repository. Its peculiarities: all the submodules hosted on the same host and .gitmodules contains only relative URLs.

-- mkdir boost ; cd boost ; lua ../git-submodules-clone-HEAD.lua https://github.com/boostorg/boost.git .
local module_url = arg[1] or 'https://github.com/boostorg/boost.git'
local module = arg[2] or module_url:match('.+/([_%d%a]+)%.git')
local branch = arg[3] or 'master'
function execute(command)
    print('# ' .. command)
    return os.execute(command)
end
-- execute('rm -rf ' .. module)
if not execute('git clone --single-branch --branch master --depth=1 ' .. module_url .. ' ' .. module) then
    io.stderr:write('can\'t clone repository from ' .. module_url .. ' to ' .. module .. '\n')
    return 1
end
-- cd $module ; git submodule update --init --recursive --remote --no-fetch --depth=1
execute('mkdir -p ' .. module .. '/.git/modules')
assert(io.input(module .. '/.gitmodules'))
local lines = {}
for line in io.lines() do
    table.insert(lines, line)
end
local submodule
local path
local submodule_url
for _, line in ipairs(lines) do
    local submodule_ = line:match('^%[submodule %"([_%d%a]-)%"%]$')
    if submodule_ then
        submodule = submodule_
        path = nil
        submodule_url = nil
    else
        local path_ = line:match('^%s*path = (.+)$')
        if path_ then
            path = path_
        else
            submodule_url = line:match('^%s*url = (.+)$')
        end
        if submodule and path and submodule_url then
            -- execute('rm -rf ' .. path)
            local git_dir = module .. '/.git/modules/' .. path:match('^.-/(.+)$')
            -- execute('rm -rf ' .. git_dir)
            execute('mkdir -p $(dirname "' .. git_dir .. '")')
            if not execute('git clone --depth=1 --single-branch --branch=' .. branch .. ' --separate-git-dir ' .. git_dir .. ' ' .. module_url .. '/' .. submodule_url .. ' ' .. module .. '/' .. path) then
                io.stderr:write('can\'t clone submodule ' .. submodule .. '\n')
                return 1
            end
            path = nil
            submodule_url = nil
        end
    end
end

Answer 6

git pull --recurse-submodules --jobs=10

a feature git first learned in 1.8.5.

Until the bug is fixed, for the first time you do need to run

git submodule update --init --recursive