Friday, November 21, 2008

C# OutOfMemoryException and StringBuilder

We had an interesting OutOfMemoryException this week. When creating really long strings and adding them to a StringBuilder, we get the OutOfMemoryException way before we really run out of memory. Come to find out, StringBuilder is searching for contiguous memory and may not find that long before you run out of real memory.

To help find your upper limit of memory use, .Net provides the friendly MemoryFailPoint as shown below:

using System;
using System.Runtime;
using System.Text;
namespace PlayingAround {
class Memory {
private static void Main() {
MemoryFailPoint memFailPoint;
StringBuilder sb = new StringBuilder("1234567890");

for (int i = 0; i < 100; i++) {
int aboutToGetMemoryInBytes = sb.Length*5;
Console.Out.WriteLine("aboutToGetMemoryInBytes = " + aboutToGetMemoryInBytes.ToString("#,##0"));
int aboutToGetMemoryInMegaBytes = (int)(1 + (aboutToGetMemoryInBytes >> 20)); //round up to the next megabyte
Console.Out.WriteLine("aboutToGetMemoryInMegaBytes = " + aboutToGetMemoryInMegaBytes.ToString("#,##0"));
using (memFailPoint = new MemoryFailPoint(aboutToGetMemoryInMegaBytes)) {
Console.Out.WriteLine("GC.GetTotalMemory(true) = " + GC.GetTotalMemory(true).ToString("#,##0"));
Console.Out.WriteLine("sb.Length = " + sb.Length.ToString("#,##0"));
catch (InsufficientMemoryException) {
var maxMem = GC.GetTotalMemory(true);
Console.Out.WriteLine("maxMem = " + maxMem);
} catch (OutOfMemoryException) {
Console.Out.Write("press return to exit.");


Anonymous said...

And the solution is ???

Mitch Fincher said...

Well, not a lot of good options.
You could use something besides StringBuilder that doesn't require contiguous memory, like an array of strings.
Or if you are up for it, write your own version of StringBuilder that behind the scenes uses a collection of regular strings, or a collection of smaller StringBuilders to store your strings.
Or a really smart StringBuilder substitute that uses one StringBuilder to back it, until it gets the out-of-memory error, and then it splits the string into two StringBuilders for you.
Or you can probably come up with something better.
Let me know how you decide to fix this.