MemoryStream usage leads to out of memory exception

Question

I\'am facing issues when using MemoryStream multiple times.

Example:

For Each XImage As XImage In pdfDocument.Pages(pageCount).Resources.Images
   D         


        

Answer 1

The memory from the stream is freed. I promise you. Really, it is.

What is not freed is the address space in your application formerly occupied by that memory. There's plenty of ram available to your computer, but your specific application crashes because it can't find a place within it's address table to allocate any more.

The reason you hit the limit is that the MemoryStream recycles its buffer as it grows. It uses a byte[] internally to hold its data, and the array is initialized to a certain size by default. As you write to the stream, if you exceed the size of your array the stream uses a doubling algorithm to allocate new arrays. Information is then copied from the old array to new. After this, the old array can and will be collected, but it will not be compacted (think: defragged). The result is in holes in your address space that will no longer be used. One MemoryStream might use several arrays, resulting in several memory holes worth a total address space potentially much larger than the source data.

AFAIK, there is no way at this time to force the garbage collector to compact your memory address space. The solution therefore is to allocate a big block that will handle your largest image, and then reuse that same block over and over, so that you don't end up with memory addresses that can't be reached.

For this code, that means creating the memorystream outside of the loop, and passing an integer to the constructor so that it is initialized to a reasonable number of bytes. You'll find this also gives you a nice performance boost, as your application suddenly no longer spends time frequently copying data from one byte array to another, meaning this is the better option even if you could compact your address table:

Using imageStream As New MemoryStream(307200) 'start at 300K... gives you some breathing room for larger images
    For Each XImage As XImage In pdfDocument.Pages(pageCount).Resources.Images

       'reset the stream, but keep using the same memory
       imageStream.Seek(0, SeekOrigin.Begin)
       imageStream.SetLength(0)

       XImage.Save(imageStream, System.Drawing.Imaging.ImageFormat.Jpeg)

       ' some further processing

    Next
End Using