Broken pipe when pushing to git repository

问题

I'm trying to push for the first time a code to my git repository but i get the following error:

Counting objects: 222026, done. 
Compressing objects: 100% (208850/208850), done. 
Write failed: Broken pipe222026) 
error: pack-objects died of signal 13 
fatal: The remote end hung up unexpectedly error: failed to push some refs to 'ssh://git@bitbucket.org/<...>'

I tried to increase the http buffer size (git config http.postBuffer 524288000), I tried to git repack, but it did not work.

I was able to push a very similar size code to another repository (it was not working like this one, but after the git repack it did work). I'm trying to push it to bitbucket.

Any ideas?

回答1:

Simple solution is to increase the HTTP post buffer size to allow for larger chunks to be pushed up to the remote repo. To do that, simply type:

git config http.postBuffer 52428800

The number is in bytes, so in this case I have set it to 50MB. The default is 1MB.

回答2:

I had that issue when working with an arch distro on VMWare.

Adding

IPQoS=throughput

to my ssh config (~/.ssh/config) did the trick for me.

回答3:

I had the same problem, and this worked for me:

git gc --aggressive --prune

It took a while, but after it was done all git operations started working faster.
The push operation that previously failed then succeeded, probably because it became fast enough to avoid some timeout related issue.

回答4:

I met the same problem when uploading my gigabytes of data to github repository. Increasing the HTTP buffer size did not work for this size of data. I am not sure if it is a problem of git itself or github server. Anyway I made a shell script to handle this problem, which uploades files in the current directory step by step, in each step less than 100 MB of data. It is working fine for me. It takes time but I can just detach screen and wait overnight.

Here is the shell script: https://gist.github.com/sekika/570495bd0627acff6c836de18e78f6fd

回答5:

Note that a push can still freeze (even with postBuffer increased) when its pack files are corrupted (ie pack-objects fails)

That will be fixed in git 2.9 (June 2016). And better managed with Git 2.25 (Q1 2020)

See commit c4b2751, commit df85757, commit 3e8b06d, commit c792d7b, commit 739cf49 (19 Apr 2016) by Jeff King (peff).
^{(Merged by Junio C Hamano -- gitster -- in commit d689301, 29 Apr 2016)}

"git push" from a corrupt repository that attempts to push a large number of refs deadlocked; the thread to relay rejection notices for these ref updates blocked on writing them to the main thread, after the main thread at the receiving end notices that the push failed and decides not to read these notices and return a failure.

Commit 739cf49 has all the details.

send-pack: close demux pipe before finishing async process

This fixes a deadlock on the client side when pushing a large number of refs from a corrupted repo.

With Git 2.25 (Q1 2020), Error handling after "git push" finishes sending the packdata and waits for the response to the remote side has been improved.

See commit ad7a403 (13 Nov 2019) by Jeff King (peff).
^{(Merged by Junio C Hamano -- gitster -- in commit 3ae8def, 01 Dec 2019)}

send-pack: check remote ref status on pack-objects failure

^{Helped-by: SZEDER Gábor}
^{Signed-off-by: Jeff King}

When we're pushing a pack and our local pack-objects fails, we enter an error code path that does a few things:

Set the status of every ref to REF_STATUS_NONE

Call receive_unpack_status() to try to get an error report from the other side

Return an error to the caller

If pack-objects failed because the connection to the server dropped, there's not much more we can do than report the hangup. And indeed, step 2 will try to read a packet from the other side, which will die() in the packet-reading code with "the remote end hung up unexpectedly".

But if the connection didn't die, then the most common issue is that the remote index-pack or unpack-objects complained about our pack (we could also have a local pack-objects error, but this ends up being the same; we'd send an incomplete pack and the remote side would complain).

In that case we do report the error from the other side (because of step 2), but we fail to say anything further about the refs.

The issue is two-fold:

in step 1, the "NONE" status is not "we saw an error, so we have no status".
It generally means "this ref did not match our refspecs, so we didn't try to push it". So when we print out the push status table, we won't mention any refs at all!
But even if we had a status enum for "pack-objects error", we wouldn't want to blindly set it for every ref. For example, in a non-atomic push we might have rejected some refs already on the client side (e.g., REF_STATUS_REJECT_NODELETE) and we'd want to report that.

in step 2, we read just the unpack status.
But receive-pack will also tell us about each ref (usually that it rejected them because of the unpacker error).

So a much better strategy is to leave the ref status fields as they are (usually EXPECTING_REPORT) and then actually receive (and print) the full per-ref status.

This case is actually covered in the test suite, as t5504.8, which writes a pack that will be rejected by the remote unpack-objects.
But it's racy. Because our pack is small, most of the time pack-objects manages to write the whole thing before the remote rejects it, and so it returns success and we print out the errors from the remote.
But very occasionally (or when run under --stress), it goes slow enough to see a failure in writing, and git push reports nothing for the refs.

With this patch, the test should behave consistently.

There shouldn't be any downside to this approach.

If we really did see the connection drop, we'd already die in receive_unpack_status(), and we'll continue to do so.

If the connection drops after we get the unpack status but before we see any ref status, we'll still print the remote unpacker error, but will now say "remote end hung up" instead of returning the error up the call-stack.
But as discussed, we weren't showing anything more useful than that with the current code. And anyway, that case is quite unlikely (the connection dropping at that point would have to be unrelated to the pack-objects error, because of the ordering of events).

In the future we might want to handle packet-read errors ourself instead of dying, which would print a full ref status table even for hangups.
But in the meantime, this patch should be a strict improvement.