large-object-heap

I have to transfer large files between computers on via unreliable connections using WCF. Because I want to be able to resume the file and I don't want to be limited in my filesize by WCF, I am chunking the files into 1MB pieces. These "chunk" are transported as stream. Which works quite nice, so far. My steps are: open filestream read chunk from file into byte[] and create memorystream transfer chunk back to 2. until the whole file is sent My problem is in step 2. I assume that when I create a memory stream from a byte array, it will end up on the LOH and ultimately cause an outofmemory

I have an other active question HERE regarding some hopeless memory issues that possibly involve LOH Fragmentation among possibly other unknowns. What my question now is, what is the accepted way of doing things? If my app needs to be done in Visual C#, and needs to deal with large arrays to the tune of int[4000000], how can I not be doomed by the garbage collector's refusal to deal with the LOH? It would seem that I am forced to make any large arrays global, and never use the word "new" around any of them. So, I'm left with ungraceful global arrays with "maxindex" variables instead of neatly

I am wondering why I'm not able to allocate more that 1,000 MB of memory in my 32-bit .NET process. The following mini application throws an OutOfMemoryException after having allocated 1,000 MB. Why 1,000 MB, and not say 1.8 GB? Is there a process-wide setting I could change? static void Main(string[] args) { ArrayList list = new ArrayList(); int i = 0; while (true) { list.Add(new byte[1024 * 1024 * 10]); // 10 MB i += 10; Console.WriteLine(i); } } PS: Garbage collecting does not help. Edit, to clarify what I want: I have written a server application which deals with very large amounts of data

I have read about Generations and Large object heap. But I still fail to understand what is the significance (or benefit) of having Large object heap? What could have went wrong (in terms of performance or memory) if CLR would have just relied on Generation 2 (Considering that threshold for Gen0 and Gen1 is small to handle Large objects) for storing large objects? A garbage collection doesn't just get rid of unreferenced objects, it also compacts the heap. That's a very important optimization. It doesn't just make memory usage more efficient (no unused holes), it makes the CPU cache much more