I have a web application that I wrote using a lot of different 3rd party components, a CMS and of course my code. For some reason I get out of memory exception.
Since it appears you can reproduce your issue, sometimes the simplest approach is to remove things you think might be the root cause, and test again (unless you need a lot of time to see if it grows a lot or not?)
At some point the bug will stop happening, and you'll know what bit of code was responsible.
However, depending on your code base, it's not always easy to remove code and still have something that you can test (i.e. without crashing the application).