How to merge >1000 xml files into one using Java

Question

I am trying to merge many xml files into one. I have successfully done that in DOM, but this solution is limited to a few files. When I run it on multiple files >1000 I am getti

Answer 1

I think what you're doing is valid. The only way to make it scale to really huge numbers of files is to use a text based approach with streaming, so you never keep the whole thing in memory. But, hey! Good news. Memory is cheap these days, and 64bit JVMs are all the rage, so maybe all you need is to increase the the heap size. Try re-running your program with a -Xms1g JVM option (allocates 1Gb initial heap size).

I also tend to use XOM for all my DOM requirements. Give it a go. Much more efficient. Don't know for sure on the memory requirements, but its orders of magnitude faster in my experience.