Python: SocketServer closes TCP connection unexpectedly

Question

I would like to implement a TCP/IP network client application that sends requests to a Python SocketServer and expects responses in return. I have started out with the offic

Answer 1

The problem here is that in a TCPHandler, a "request" is actually a complete connection, from beginning to end.* Your handler gets called on accept, and when you return from it, the socket gets closed.

If you want to build a request-response-protocol handler on top of that, which processes multiple protocol-level requests on a single socket-level request, you have to do that yourself (or use a higher-level framework). (Subclasses like BaseHTTPServer demonstrate how to do this.)

For example, you can just use a loop within your handle function. Of course you probably want to add an exception handler here and/or deal with EOF from rfile (note self.rfile.readline() will return '' for EOF, but '\n' for a blank line, so you have to check it before calling strip unless you want a blank line to mean "quit" in your protocol). For example:

def handle(self):
    try:
        while True:
            request  = self.rfile.readline()
            if not request:
                break
            request = request.rstrip()
            print "RX [%s]: %s" % (self.client_address[0], request)

            response = self.processRequest(request)

            print "TX [%s]: %s" % (self.client_address[0], response)
            self.wfile.write(response + '\n')
    except Exception as e:
        print "ER [%s]: %r" % (self.client_address[0], e)
    print "DC [%s]: disconnected" % (self.client_address[0])

---

This will often work with your existing client, at least over localhost on an unloaded machine, but it's not actually correct, and "often works" is rarely good enough. See TCP sockets are byte streams, not message streams for a longer discussion, but briefly, you also need to do the stuff mentioned in David Schwarz's answer: append newlines to what you write from the server (which I already did above), and have the client read line by line instead of just trying to read 1024 bytes at a time (which you can do by writing your own buffer-and-split-lines code, or just by using the makefile method, so it can use rfile.readline() just like the server side does.)

Not fixing the client won't cause the problems that answer claims, but it will cause problems like this:

Sent:     request type 01
Received: resp
Sent:     request type 02
Received: onse type 01
response typ

And you can see that in a real program that actually tried to process the responses programmatically, a response like resp or onse type 01\nresponse typ isn't going to be very useful…

---

_{* Note that SocketServer is an ancient design that nobody really loves. There's a reason Python 3 added asyncio, and that people in Python 2 usually use third-party frameworks like Twisted and gevent. They're both simpler for simple tasks, and more flexible/powerful (and a lot more efficient) for complex tasks.}

Answer 2

The client is broken. It calls recv just once and then closes the connection without ensuring it has received everything the server had to send.

The correct way to fix this depends on the protocol you are using, which you haven't explained. It's not apparent from the server code what the protocol for server to client "messages" is. What is the client supposed to expect to get from the server? The server, for example, expects a line from the client, a message marked with a newline to indicate the end of the line. So the server knows when it has received a message by checking for a newline. How is the client supposed to do it? Where is that code?

I'll assume for the sake of argument that your protocol always uses messages and defines a message as terminated by a newline.

    request  = self.rfile.readline().strip()

This reads a message because it calls readline.

sent = sock.sendall(data + '\n')

This sends a message because it sends a series of bytes terminated by a newline.

received = sock.recv(1024)

Oops, this just receives some bytes, not a message. There needs to be code to check if a newline was received and, if not, call recv again, otherwise, calling close will force an abnormal close of the socket (because the message was not, and can never be, received), which is precisely what you are seeing.