This problem is almost always related to the GDI Object count, User Object count or Handle count and usually not because of an out-of-memory condition on your machine.
When I am tracking one of these bugs, I open ProcessExplorer and watch these columns: Handles, Threads, GDI Objects, USER Objects, Private Bytes, Virtual Size and Working Set.
(In my experience, the problem is usually an object leak due to an event handler holding the object and preventing it from being disposed.)