NPTL caps maximum threads at 65528?

Question

The following code is supposed to make 100,000 threads:

/* compile with:   gcc -lpthread -o thread-limit thread-limit.c */
/* originally from: http://www.vol         


        

Answer 1

One possible issue is the local variable thread in the main program. I think that pthread_t would be 8 bytes on your 64-bit machine (assuming 64-bit build). That would be 800,000 bytes on the stack. Your stack limit of 512K would be a problem I think. 512K / 8 = 65536, which is suspiciously near the number of threads you are creating. You might try dynamically allocating that array instead of putting it on the stack.

Answer 2

Are you trying to search for a formula to calculate max threads possible per process?

Linux implements max number of threads per process indirectly!!

number of threads = total virtual memory / (stack size*1024*1024)

Thus, the number of threads per process can be increased by increasing total virtual memory or by decreasing stack size. But, decreasing stack size too much can lead to code failure due to stack overflow while max virtual memory is equals to the swap memory.

Check you machine:

Total Virtual Memory: ulimit -v (default is unlimited, thus you need to increase swap memory to increase this)

Total Stack Size: ulimit -s (default is 8Mb)

Command to increase these values:

ulimit -s newvalue

ulimit -v newvalue

*Replace new value with the value you want to put as limit.

References:

http://dustycodes.wordpress.com/2012/02/09/increasing-number-of-threads-per-process/

Answer 3

This might help set the stack size in the program to the smallest it can go (if that's not enough you pick):

/* compile with:   gcc -lpthread -o thread-limit thread-limit.c */
/* originally from: http://www.volano.com/linuxnotes.html */

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
#include <string.h>

#define MAX_THREADS 100000
int i;

void run(void) {
  sleep(60 * 60);
}

int main(int argc, char *argv[]) {
  int rc = 0;
  pthread_t thread[MAX_THREADS];
  pthread_attr_t thread_attr;

  pthread_attr_init(&thread_attr);
  pthread_attr_setstacksize(&thread_attr, PTHREAD_STACK_MIN);

  printf("Creating threads ...\n");
  for (i = 0; i < MAX_THREADS && rc == 0; i++) {
    rc = pthread_create(&(thread[i]), &thread_attr, (void *) &run, NULL);
    if (rc == 0) {
      pthread_detach(thread[i]);
      if ((i + 1) % 100 == 0)
    printf("%i threads so far ...\n", i + 1);
    }
    else
    {
      printf("Failed with return code %i creating thread %i (%s).\n",
         rc, i + 1, strerror(rc));

      // can we allocate memory?
      char *block = NULL;
      block = malloc(65545);
      if(block == NULL)
        printf("Malloc failed too :( \n");
      else
        printf("Malloc worked, hmmm\n");
    }
  }
sleep(60*60); // ctrl+c to exit; makes it easier to see mem use
  exit(0);
}

additionally you could add a call like this: pthread_attr_setguardsize(&thread_attr, 0); just after the call to pthread_attr_setstacksize() but then you'd loose stack overrun detection entirely, and it'd only save you 4k of address space and zero actual memory.

Answer 4

pilcrow's mention of /proc/sys/vm/max_map_count is right on track; raising this value allows more threads to be opened; not sure of the exact formula involved, but a 1mil+ value allows for some 300k+ threads.

(For anyone else experimenting with 100k+ threads, do look at pthread_create's mmap issues... making new threads gets really slow really fast when lower memory is used up.)