Distributed version control for HUGE projects - is it feasible?

Question

We\'re pretty happy with SVN right now, but Joel\'s tutorial intrigued me. So I was wondering - would it be feasible in our situation too?

The thing is - our SVN reposit

Answer 1

I am using git on a fairly large c#/.net project (68 projects in 1 solution) and the TFS footprint of a fresh fetch of the full tree is ~500Mb. The git repo, storing a fair amount of commits locally weighs in at ~800Mb. The compaction and the way that storage works internally in git is excellent. It is amazing to see so many changes packed in to such a small amount of space.

Answer 2

Do you need all history? If you only need the last year or two, you could consider leaving the current repository in a read-only state for historical reference. Then create a new repository with only recent history by performing svnadmin dump with the lower bound revision, which forms the basis for your new distributed repository.

I do agree with the other answer that 100MB working copy and 68K revisions isn't that big. Give it a shot.

Answer 3

Don't worry about repository space requirements. My anecdote: when I converted our codebase from SVN to git (full history - I think), I found that the clone used less space than just the WVN working directory. SVN keeps a pristine copy of all your checked-out files: look at $PWD/.svn/text-base/ in any SVN checkout. With git the entire history takes less space.

What really surprised me was how network-efficient git is. I did a git clone of a project at a well-connected place, then took it home on a flash disk, where I keep it up to date with git fetch / git pull, with just my puny little GPRS connection. I wouldn't dare to do the same in an SVN-controlled project.

You really owe it to yourself to at least try it. I think you'll be amazed at just how wrong your centralised-VCS-centric assumptions were.

Answer 4

You'd split your one huge repository into lots of smaller repositories, each for each module in your old repo. That way people would simply hold as repositories whatever SVN projects they would have had before. Not much more space required than before.

Answer 5

Git can obviously work with a project as big as yours since, as you pointed, Linux kernel alone is bigger.

The challenge (don't know if you manage big files) with Mercurial and Git is that they can't manage big files (so far).

I've experience moving a project your size (and around for 15 years too) from CVS/SVN (a mix of the two actually) into Plastic SCM for distributed and centralized (the two workflows happening inside the same organization at the same time) development.

The move will never be seamless since it's not only a tech problem but involves a lot of people (a project as big as yours probably involves several hundreds of developers, doesn't it?), but there are importers to automate migration and training can be done very fast too.

Answer 6

100MB of source code is less than the Linux kernel. Changelog between Linux kernel 2.6.33 and 2.6.34-rc1 has 6604 commits. Your repository scale doesn't sound intimidating to me.

• Linux kernel 2.6.34-rc1 uncompressed from .tar.bz2 archive: 445MB
• Linux kernel 2.6 head checked out from main Linus tree: 827MB

Twice as much, but still peanuts with the big hard drives we all have.