kernel-based (Linux) data relay between two TCP sockets

Question

I wrote TCP relay server which works like peer-to-peer router (supernode).

The simplest case are two opened sockets and data relay between them:

Answer 1

OpenBSD implements SO_SPLICE:

• relayd asiabsdcon2013 slides / paper
• http://www.manualpages.de/OpenBSD/OpenBSD-5.0/man2/setsockopt.2.html
• http://metacpan.org/pod/BSD::Socket::Splice .

Does Linux support something similar or only own kernel-module is the solution?

• TCPSP
• SP-MOD described here
• TCP-Splicer described here
• L4/L7 switch
• HAProxy

Answer 2

Even for loads as tiny as 2000 concurrent connections, I'd never go with threads. They have the highest stack and switching overhead, simply because it's always more expensive to ensure that you can be interrupted anywhere than when you can only be interrupted at specific places. Just use epoll() and splice (if your sockets are TCP, which seems to be the case) and you'll be fine. You can even make epoll work in event triggered mode, where you only register your fds once.

If you absolutely want to use threads, use one thread per CPU core to spread the load, but if you need to do this, it means you're playing at speeds where affinity, RAM location on each CPU socket etc... plays a significant role, which doesn't seem to be the case in your question. So I'm assuming that a single thread is more than enough in your case.