Git pull fatal: Out of memory, malloc failed

Question

I\'ve a repo on https://bitbucket.org/

Few days ago by a mistake big number of image files were pushed in the repo. then files were deleted via another push. after that

Answer 1

In my case it was something as simple as trying to pull a big repo in a 1GB RAM box without swap.

I followed this tutorial https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-14-04 to create some swap space on the server and worked.

Their "faster" way:

sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

You can make these changes permanent by adding to /etc/fstab:

/swapfile   none    swap    sw    0   0

They recommend adding to /etc/sysctl.conf:

vm.swappiness=10
vm.vfs_cache_pressure = 50

Answer 2

I recently encountered this issue with one of my repositories. Similar error, hinting at a large file hidden in the repo somewhere.

Cloning into 'test_framework'...
remote: Counting objects: 11889, done.
remote: Compressing objects: 100% (5991/5991), done.
Receiving objects:  66% (7847/11889), 3.22 MiB | 43 KiB/sremote: fatal: Out of memory, malloc failed     (tried to allocate 893191377 bytes)
remote: aborting due to possible repository corruption on the remote side.
fatal: early EOFs:  66% (7933/11889), 3.24 MiB
fatal: index-pack failed

To get around this I temporarily created a large swap drive (in excess of the 893191377 bytes the server was asking for) following Method 2 from here: http://www.thegeekstuff.com/2010/08/how-to-add-swap-space/

This allowed me to successfully clone and then remove the culprit (someone had checked in an sql dumpfile). You can use:

git filter-branch --tree-filter 'rm -rf dumpfile.sql' HEAD

to remove the file from the git repo.

Answer 3

Even if the big image files have been deleted after having being pushed, they do stay in the git history.

I would suggest to forcibly remove them from the git history (I think that is possible, but it involves a delicate procedure that I don't know).

Alternatively, pull the repository before the mistakenly added files, patch the repository to make the relevant small patches, clone that, and use that (perhaps with a dump/restore) as your master git.

^{I don't know well the details, but I did read it could be possible}

Answer 4

If you are the only one using this repo, you can follow the git filter-branch option described in "How to purge a huge file from commits history in Git?"

The simpler option is cloning the repo to an old commit, and force push it, as described in "git-filter-branch to delete large file".

Either one would force any collaborator to reset his/her own local repo to the new state you are publishing. Again, if you are the only collaborator, it isn't an issue.