How to write (big) XML to a file in C#?

Question

Folks,

Please, what\'s a good way of writing really big XML documents (upto say 500 MB) in C# .NET 3.5? I\'ve had a bit of search around, and can\'t seem to find anythin

Answer 1

For writing large xml, XmlWriter (directly) is your friend - but it is harder to use. The other option would be to use DOM/object-model approaches and combine them, which is probably doable if you seize control of theXmlWriterSettings and disable the xml marker, and get rid of the namespace declarations...

using System;
using System.Collections.Generic;
using System.Xml;
using System.Xml.Serialization;    
public class Foo {
    [XmlAttribute]
    public int Id { get; set; }
    public string Bar { get; set; }
}
static class Program {
    [STAThread]
    static void Main() {
        using (XmlWriter xw = XmlWriter.Create("out.xml")) {
            xw.WriteStartElement("xml");
            XmlSerializer ser = new XmlSerializer(typeof(Foo));
            XmlSerializerNamespaces ns = new XmlSerializerNamespaces();
            ns.Add("","");
            foreach (Foo foo in FooGenerator()) {
                ser.Serialize(xw, foo, ns);
            }
            xw.WriteEndElement();
        }
    }    
    // streaming approach; only have the smallest amount of program
    // data in memory at once - in this case, only a single `Foo` is
    // ever in use at a time
    static IEnumerable<Foo> FooGenerator() {
        for (int i = 0; i < 40; i++) {
            yield return new Foo { Id = i, Bar = "Foo " + i };
        }
    }
}

Answer 2

Use a XmlWriter:

[...] a writer that provides a fast, non-cached, forward-only means of generating streams or files containing XML data.

Answer 3

Did you consider compressing it before writing it to disk? With XML you can reach more than 10 times compressing and even more. it will probably take you less time to compress the file and write the compressed version than to read the whole 500Mb version.