I have a fairly serious bug in my program - occasional calls to new() throw a bad_alloc.
From the documentation I can find on bad_alloc, it seems to be thrown for these
bad_alloc can also be thrown when you have a bug that is overwriting the pointers that the heap uses to manage the pool of memory that it uses to allocate from.
The most common cause of that is that you are writing past the end of an allocated block of memory, (or before the start, but that's less common). Almost as common is writing to a memory block after it has been freed. This is called heap corruption.
Also, I should note, a 32 bit process in Windows has at most 2GB of address space (3GB for large-address-aware programs). This is regardless of how much RAM you have installed, the memory is virtual, and allocations don't fail until you run out of address space, even if you only have 1GB of RAM.
Here is a good discussion of memory corruption in C++ http://www.eventhelix.com/RealtimeMantra/Basics/debugging_software_crashes_2.htm
bad_alloc can be thrown by other code as well.
I've seen it used by a limiting memory pool designed for use with STL containers. When the size limit was hit, it threw bad_alloc and the software just had to handle it.
A strategy for debugging this would be to set a catchpoint in the debugger. In gdb one would say “catch throw” to ask the debugger you stop whenever the program throws an exception. You can refine that to catch only std::bad_alloc if the program throws and catches lots of exceptions in its normal course of business. The debugger will stop inside whichever allocation function generates std::bad_alloc, allowing you to poke around. In this case, you would quickly learn that number of bytes requested is not what you expect.
I've actually had this problem before and it was fixed by cleaning and rebuilding the project. Always worth a try when you have weird behaviour (unless it is a huge project that takes hours to compile).
Few clarifications:
Every process in windows gets 4GB virtual memory, out of which 2GB is for user space and remaining for kernel space. The 4GB of RAM won't contribute to the virtual memory but it is for physical memory.
In the 2GB memory, all EXE, DLL gets loaded and hardly 1.6 - 1.7GB available for memory allocation. In this memory if there is no contiguous memory for allocation then the memory allocation fails.
Another possible problem is that, while you mention that the program is using less than 5MB, you don't mention how much space it's trying to allocate. You could have some race condition that's corrupting the value that you use to determine the allocation size, and it could be trying to allocate 37TB or somesuch nonsense.
Not particularly likely, I suppose, but worth checking.