I am programming something in C that creates a lot of Pthreads in Linux on a 256Mb system. I usually have +200Mb free.
When I run the program with a low amount of thread
The system threads library is just not designed to support large numbers of threads on systems with very limited memory. You need to either use a threading library designed for that purpose or, better, use fewer threads.