Python sockets/port forwarding

Question

I\'ve written server and client programs with python

Server.py

 import socket

sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)

host =         


        

Answer 1

You have to change the socket.gethostname() in the server script to the empty string (or just directly call socket.bind(('', port))).

Your problem is not in the python but in the usage of sockets generally. When you create socket you just prepare your process to receive/send some data from/to another process.

Server

The first step for creating a socket you have to specify what kind of protocol will be used for communication between those processes. In your case it is the socket.AF_INET which is constant for use of IP protocol and the socket.SOCK_STREAM is specify reliable stream-oriented service. The reliable stream-oriented service means that you want to be sure that every single sent byte will be delivered to the other side and nothing can be lost during the communication (the underlying OS will use TCP protocol for that). From this point we are using IPv4 protocol (because we set the socket.AF_INET)

The second step is bind it to address. The bind process assign address where you expected that client will join (with your socket's settings it's a IP address and the TCP port). Your PC has multiple IP address (well at least two). It's always has 127.0.0.1 which is called callback and it works only when your applications communicate on the same PC (that is you Linux - Linux scenario in the question) and then you have IP which is used for communication with others computers (let's pretend it is 10.0.0.1).

When you call socket.bind(('127.0.0.1', 5555)) you're setting the socket to listen only for communication from the same PC. If you call socket.bind(('10.0.0.1', 5555)) then the socket setting is ready to receive data targeted to the 10.0.0.1 address.

But what if you have 10 IPs or more and you want to receive everything (with right TCP port). For those scenarios you can leave the IP address in bind() empty and it does exactly what you want.

With Python's version of bind() you can enter also "computer name" instead of the concrete IP. The socket.gethostname() call return your computer's name. The problem is in the translation of "computer name" to the IP which Python makes behind your backs. The translation has some rules but generally your "computer name" can be translated into any IP address which you have set on your computer. In your case the your computer's name is converted into 127.0.0.1 and that's why communication works only between processes on the same computer.

After socket.bind() you have the socket ready to use but it is still "inactive". The socket.listen() activate the socket and wait while someone want to connect. When socket receives new connection request it will put into queue and wait for processing.

That's what socket.accept() do. It pulls the connection request from queue, accept it and establish the stream (remember the socket.SOCK_STREAM while you set up the socket) between the server and the client. The new stream is actually new socket but ready to communicate with other side.

What did happen with the old socket? Well it's still alive and you can call socket.listen() again to get another stream (connection).

How is possible to have multiple sockets on the same port

Every connection within computer's network is defined by flow which is 5-tuple of:

• L4 protocol (usually TCP or UDP)
• Source IP address
• Source L4 port
• Destination IP address
• Destination L4 port

When you create new connection from client the flow can look like this (TCP, 192.168.0.1, 12345, 10.0.0.1, 55555). Just for clarification the server's response flow is (TCP, 10.0.0.1, 55555, 192.168.0.1, 12345) but it isn't important for us. If you create another connection from client that it will differ at source TCP port (if you do it from another computer that it will differ also at the source IP). Only from this information you can distinguish every connection created to your computer.

When you create a server socket in your code and call socket.listen() it listen for any flow with this pattern (TCP, *, *, *, 55555) (the * means match everything). So when you get connection with (TCP, 192.168.0.1, 12345, 10.0.0.1, 55555) then socket.accept() create another socket which works only with this one concrete flow while the old socket still accepting new connections which wasn't established.

When operating system receives a packet it looks in the packet and check the flow. From this point it can happen a several scenarios:

• The packet's flow match all 5 items exactly (without usage of *). Then the packet's content is delivered to the queue associated with that socket (you're reading the queue when you call socket.recv()).
• The packet's flow matched socket with associated flow contains * then it is considered as new connection and you can call scoket.accept().
• The operating system doesn't contain open socket which would match the flow. In that case the OS refuse connection (or just ignore the packet it depends on firewall settings).

Probably some example can clarify those scenarios. The operating system has something like table where it map flows to sockets. When you call socket.bind() it will assign flow to the socket. After the call the table can look like this:

+=====================================+========+
|                Flow                 | Socket |
+=====================================+========+
| (TCP, *, *, *, 55555)               |      1 |
+-------------------------------------+--------+

When it receive packet with flow (TCP, 1.1.1.1, 10, 10.0.0.1, 10) then it won't match any flow (last port won't match). So the connection is refused. If it receives a packet with flow (TCP, 1.1.1.1, 10, 10.0.0.1, 55555) then the packet is delivered to the socket 1 (because there is a match). The socket.accept() call creates a new socket and record in the table.

+=====================================+========+
|                Flow                 | Socket |
+=====================================+========+
| (TCP, 1.1.1.1, 10, 10.0.0.1, 55555) |      2 |
+-------------------------------------+--------+
| (TCP, *, *, *, 55555)               |      1 |
+-------------------------------------+--------+

Now you got 2 sockets for 1 port. Every received packet which match the flow associated with the socket 2 also match flow associated with socket 1 (on the contrary it does not apply). It's not a problem because the socket 2 has preciser match (is doesn't use the *) so any data with that flow will be delivered to socket 2.

How to server multiple connections

When you want to do a "real" server than you're application should be able process multiple connection (without restarting). There are 2 basic approaches:

1. sequential processing

   try:
       l = prepare_socket()
       while True:
           l.listen()
           s, a = socket.accept()
           process_connection(s) # before return you should call s.close()
   except KeyboardInterrupt:
       l.close()
   

   In this case you can process only one client while others clients have to wait for accept. If the process_connection() takes too long then others clients will timeout.

2. parallel processing

   import threading
   threads = []
   
   try:
       l = prepare_socket()
       while True:
           l.listen()
           s, a = socket.accept()
           t = threading.Thread(target=process_connection, s)
           threads.append(t)
           t.start()
   except KeyboardInterrupt:
       for t in threads:
           t.join()
       l.close()
   

   Now when you receive new connection it will create new thread so every connection is processed in parallel. The main disadvantage of this solution is that you have to solve common troubles with threading (like access to shared memory, deadlocks etc.).

Beware those are only example codes and they are not complete! For example it doesn't contain code for graceful exit on unexpected exceptions.

Servers in the Python

The Python also contains module called socketserver which contains shortcuts to create servers. Here you can find example how to use it.

Client

With the client it's much more simpler than with the server. You just have to create socket with some settings (same as server side) and then tell it where is the server is (what is its IP and TCP port). This is accomplished through socket.connect() call. As bonus it also establish the stream between your client and server so from this point you can communicate.

---

You can find more information about socktes at the Beej's Guide to Network Programming. It's written for usage with C but the concepts are the same.