Handling more than 1024 file descriptors, in C on Linux

Question

I am working on a threaded network server using epoll (edge triggered) and threads and I\'m using httperf to benchmark my server.

So far, it\'s performing really wel

Answer 1

Thanks for all your answers but I think I've found the culprit. After redefining __FD_SETSIZE in my program everything started to move a lot faster. Of course ulimit also needs to be raised, but without __FD_SETSIZE my program never takes advantage of it.

Answer 2

Please see the C10K problem page. It contains an in-depth discussion on how to achieve the '10000 simultaneous connections' goal, while maintaining high-performance and managing to serve each client.

It also contains information on how to increase the performance of your kernel when handling a large number of connections at once.

Answer 3

Just don't.

Yes, I mean that.

If you need to increase the file descriptors, there's a hidden bug in your code. Hunt it down instead of treating its symptoms. Remember to close file descriptors when you're done.