RegEx, StringBuilder and Large Object Heap Fragmentation

Question

How can I run lots of RegExes (to find matches) in big strings without causing LOH fragmentation?

It\'s .NET Framework 4.0 so I\'m using StringBuilder so it

Answer 1

One alternative would be to find some way of performing reg-ex matches on a non-array based data structure. Unfortunately, a quick Google didn't bring up much in terms of stream based reg-ex libraries. I would guess that the reg-ex algorithm would need to do a lot of back tracking, which isn't supported by streams.

Do you absolutely require the full power of regular expressions? Could you perhaps implement your own simpler search functions that could work on linked lists of strings all under 85kb?

Also, LOH fragmentation only really causes issues if you hold on to the large object references for long periods. If you're constantly creating and destroying them, the LOH shouldn't grow.

FWIW, I've fount the RedGate ANTS memory profiler very good at tracking down objects in the LOH and levels of fragmentation.