Can you use List> to get around the 2gb object limit?

Question

I\'m running up against the 2gb object limit in c# (this applies even in 64 bit for some annoying reason) with a large collection of structs (est. size of 4.2 gig in total).

Answer 1

Now obviously using List is going to give me a list of size 4.2gb give or take, but would using a list made up of smaller lists, which in turn contain a portion of the structs, allow me to jump this limit?

Yes - though, if you're trying to work around this limit, I'd consider using arrays yourself instead of letting the List class manage the array.

The 2gb single object limit in the CLR is exactly that, a single object instance. When you make an array of a struct (which List uses internally), the entire array is "one object instance" in the CLR. However, by using a List> or a jagged array, each internal list/array is a separate object, which allows you to effectively have any size object you wish.

The CLR team actually blogged about this, and provided a sample BigArray implementation that acts like a single List, but does the "block" management internally for you. This is another option for getting >2gb lists.

Note that .NET 4.5 will have the option to provide larger than 2gb objects on x64, but it will be something you have to explicitly opt in to having.