StringBuilder.ToString() throws OutOfMemoryException

问题

I have a created a StringBuilder of length "132370292", when I try to get the string using the ToString() method it throws OutOfMemoryException.

StringBuilder SB = new StringBuilder();

for(int i =0; i<=5000; i++)
{
    SB.Append("Some Junk Data for testing. My Actual Data is created from different sources by Appending to the String Builder.");
}

try
{
    string str = SB.ToString(); // Throws OOM mostly
    Console.WriteLine("String Created Successfully");
}
catch(OutOfMemoryException ex)
{
    StreamWriter sw = new StreamWriter(@"c:\memo.txt", true);
    sw.Write(SB.ToString()); //Always writes to the file without any error
    Console.WriteLine("Written to File Successfully");
}

What is the reason for the OOM while creating a new string and why it doesn't throw OOM while writing to a file?

Machine Details: 64-bit, Windows-7, 2GB RAM, .NET version 2.0

回答1:

What is the reason for the OOM while creating a new string

Because you're running out of memory - or at least, the CLR can't allocate an object with the size you've requested. It's really that simple. If you want to avoid the errors, don't try to create strings that don't fit into memory. Note that even if you have a lot of memory, and even if you're running a 64-bit CLR, there are limits to the size of objects that can be created.

and why it doesn't throw OOM while writing to a file ?

Because you have more disk space than memory.

I'm pretty sure the code isn't exactly as you're describing though. This line would fail to compile:

sw.write(SB.ToString());

... because the method is Write rather than write. And if you're actually calling SB.ToString(), then that's just as likely to fail as str = SB.ToString().

It seems more likely that you're actually writing to the file in a streaming fashion, e.g.

using (var writer = File.CreateText(...))
{
    for (int i = 0; i < 5000; i++)
    {
        writer.Write(mytext);
    }
}

That way you never need to have huge amounts of text in memory - it just writes it to disk as it goes, possibly with some buffering, but not enough to cause memory issues.

回答2:

Workaround: Suppose you would want to write a big string stored in StringBuilder to a StreamWriter, I would do a write this way to avoid SB.ToString's OOM exception. But if your OOM exception is due to StringBuilder's content add itself, you should work on that.

public const int CHUNK_STRING_LENGTH = 30000;
while (SB.Length > CHUNK_STRING_LENGTH )
{
    sw.Write(SB.ToString(0, CHUNK_STRING_LENGTH ));
    SB.Remove(0, CHUNK_STRING_LENGTH );
}
sw.Write(SB);