Pthreads - High memory usage

Question

I am programming something in C that creates a lot of Pthreads in Linux on a 256Mb system. I usually have +200Mb free.

When I run the program with a low amount of threads it works, but once I make it create around 100 threads it gives errors because the system runs out of memory. I did several tests and each threads use almost 2Mb. The stack size of the threads is set to 16Kb.

The code I use to create each thread:

pthread_attr_t attr;
pthread_attr_init(&attr);
size_t stacksize;
stacksize = (double) 16*1024;
int res = pthread_attr_setstacksize (&attr, stacksize);
int res2 = pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
if (res != 0 || res2 != 0) {
    logs << "pthread_attr_XX: error "+int2string(res);
    exit(-1);
}
pthread_t id;
pthread_create(&id, &attr, &Class::thread_callback, &some_var);

Is it normal or am I missing something? Thanks.

Answer 1

Not sure it will help, but try calling setrlimit with RLIMIT_STACK to limit the stack size to 16k before creating your first thread.

Answer 2

The system threads library is just not designed to support large numbers of threads on systems with very limited memory. You need to either use a threading library designed for that purpose or, better, use fewer threads.

Answer 3

Yeah, it's fairly common for pretty much any operating system to choke and die on that many threads. It's not a problem that OS makers (Linux or otherwise) have given much attention to since very few systems have that many CPUs, so your code wouldn't likely executing much faster with 100 threads than it would with 8 (or however many CPUs you have).

I think what you need to do is some sort of thread pooling. I also suspect the ulimit answer on this page would be useful if you really need that many threads, but like I said, I don't think that many threads gets you much in most circumstances and if you fix it in your program instead of in the system, it'll make it more portable.

Answer 4

Why stacksize = (double) 16*1024; ? It is integer.

Try to set it to 32 or 64 kilobytes, because 16*1024 may be not allowed; there can be also TLS in the stack (even you doesn't use TLS; libraries do, including libc).

So, change the line to

stacksize = 64*1024; 

and check how much memory is consumed.

Answer 5

If you want lower overheads consider user-space threading technologies such as fibers, ala co-operative task management.

http://en.wikipedia.org/wiki/Fiber_(computer_science)

http://www.evanjones.ca/software/threading.html

GNU Portable threads:

http://www.gnu.org/software/pth/

Boost C++ co-routines:

http://www.boost.org/doc/libs/1_60_0/libs/coroutine/doc/html/index.html

Windows-only fibers as an FYI:

http://msdn.microsoft.com/en-us/library/ms682661(v=vs.85).aspx

See the Wikipedia link for more example implementations.

Answer 6

May be this is the reason:

"(Recognizing leaks

If you create a joinable thread but forget to join it, its resources or private memory are always kept in the process space and never reclaimed. Always join the joinable threads; by not joining them, you risk serious memory leaks.)"

Read it here: http://www.ibm.com/developerworks/library/l-memory-leaks/