350GB SVN repo creates atleast 1MB revision for even a simplest task like branch/tag

末鹿安然 提交于 2019-11-30 21:19:52

Posted same question to users@subversion.sapache.org and got this answer from B Smith-Mannschott - which explains everything. I do have a directory in the path that contains 16000 folders - for every commit. Thank you B Smith-Mannschott for the detailed response. Posting reply here for others' benefit.


Does your repository contain a directory with very many entries? Are the changes that produce the large commits being made in or below such a directory?

Let's assume to commit a single change to a single file to your repository. Let's further assume the file is located here, in your repository:

/project/trunk/some-really-large-directory/notes/blah.txt

When you commit the change to blah.txt, the new revision will rewrite the directory nodes between 'blah.txt' and the root of the repository: /project/trunk/some-really-large-directory/notes, /project/trunk/some-really-large-directory, /project/trunk, /project, /. When rewriting a directory node, FSFS always stores the new version in its entirety. (This is different from the way changes to files are stored, which are generally as differences to some previous version of the same file.)

If /project/trunk/some-really-large-directory/ contains, say 10000 files, then each commit to blah.txt will store a full copy of this directory (with its 10'000 names) in your repository.

I noticed this when I started keeping a personal wiki under version control a few years ago. It was a flat directory of over 10'000 text files. I quickly noticed that commits were pretty big. (I've since switched to git for that task, for this and other reasons.)

see also http://svn.apache.org/repos/asf/subversion/trunk/notes/subversion-design.html#server.fs.struct.bubble-up

There is a very simple solution. Assuming that your repository contains a huge quantity of historical tags, you can move them to /tags-archive and make this directory Read Only. When you will create new tags under /tags there problem won't occur anymore.

Note that you need to use the URL to URL move. E.g.

svn move https://svn.example.com/MyRepo/tags https://svn.example.com/MyRepo/tags-archive -m "Your Log Message"

This solution helped to solve the problem for a repository that contained about 350,000 tags in a single directory.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!