How can I fix the SVN import line endings error?

Question

I have to import an huge SVN repository that I have to transfer from one server to another. So I exported it from the old server:

svnadmin dump . > archive.sv         


        

Answer 1

Use svndumptool:

svndumptool.py eolfix-prop svn:ignore svn.dump svn.dump.repaired

@tangens solution works too for me, but it's not perfect as I'm getting an extra space character as the replacement of the carriage return characters. I did however test that the svn:ignore still works with that extra space, but I didn't test the other SVN properties.

The downside of using the svndumptool is it can only work on one svn property at one time, which is time consuming if your dump files are big.

---

Some findings

You might be curious on why @tangens didn't replace the ^M with empty character. If you try to replace it with empty character, you will be getting this error:

svnadmin: E140001: Dumpstream data appears to be malformed

The dump file stores Prop-content-length property which will be matched against the content of the actual content of the property. Replacing ^M to an empty character will reduce the property content length, hence the error.

svndumptool will change the Prop-content-length and Content-length respectively.

Answer 2

I ran into this error when upgrading a 1.6 repo to 1.8. I found a number of "Fixes" on the net.

The --bypass-prop-validation did not appeal to me because it postpones the problem, the next time you need to restore the repo you will hit the same promblem.

I found a bash script to loop through the revisions getting the comments and setting them again but this did not work.

An improvement on ventura10's solution did the job. Because of the size of the dump I ended up using the single command to remove the unwanted characters while restoring the dump

    sed -e '/^svn:log$/,/^K / s/^M/ /' -e '/^svn:ignore$/,/^PROPS-END$/ s/^M/\n/' /path/to/svn.dump | svnadmin load /path/to/repo

NOTE:

Where ^M is a control character, that means 0D in hex. To get it use ^V^M (control V control M) instead ^M (circumflex M or control M)

Answer 3

You have 2 options, repair the source or disable prop validation.

Repairing the source (svn:log and svn:ignore):

sed -e '/^svn:log$/,/^K / s/^M/ /' -e '/^svn:ignore$/,/^PROPS-END$/ s/^M/\n/' archive.svn > repaired-archive.svn

svnadmin load . < repaired-archive.svn

Where ^M is a control character, that means 0D in hex. To get it use ^V^M (control V control M) instead ^M (circumflex M or control M)

Disabling prop validation:

svnadmin load --bypass-prop-validation . < archive.svn

Answer 4

I used svnadmin load --normalize-props {myrepo} < {mydumpfile} and it worked perfectly.

The --normalize-props switch was added to subversion 1.10. For details, refer to https://subversion.apache.org/docs/release-notes/1.10.html#normalize-props

Reproducing for convenience -

New --normalize-props option for svnadmin load

A new --normalize-props option has been added to the svnadmin load command. This option can be used to automatically repair non-LF line endings in properties such as svn:log or svn:ignore. Such invalid line endings were accepted by older servers and can be found in repository dumps of older repositories, but are rejected by Subversion 1.6 and later.

Calling svnadmin load with --normalize-props will automatically repair all invalid property line endings found in the dumpstream, thus ensuring that the appropriate values are loaded into the repository.

Answer 5

Have you changed the server version? This is a known issue in 1.6, and causes problems when going from 1.4 or 1.5.

Subversion 1.6 no longer accepts carriage returns (^M) in property files. You'll need to fix the line breaks in your svn:ignore file(s), or recreate if that's easier.

Alternatively, you could go for Subversion 1.7, or use uberSVN.

Answer 6

I just repaired a svn dump file successfully. It also had a CRLF in the properties, which caused an Exception in SVN Edge dump import (they have a really bad import routine). Then I "installed" svnserve for testing, which was much easier than expected (instructions for Windows OS):

1. download and install TortoiseSVN or another software that includes svn command line tools. I already had it installed
2. start cmd.exe, run svnserve -d. This cmd window is busy now.
3. start another cmd.execreate a repo: svnadmin create d:\svn_repos\test12
4. load the dump into the repo: svnadmin load d:\svn_repos\test12 < d:\temp\svn\backup_test.dump

svnserve will give you detailed info what failed.

And here's what you have to repair (original on left, repaired on right side):