Using gevent with python xmlrpclib

Question

Is it possible to use python's standard libs xmlrpclib with gevent? Currently i'm tried to use monkey.patch_all(), but without success.

from gevent import monkey
monkey.patch_all()

import gevent

import time

import xmlrpclib
from SimpleXMLRPCServer import SimpleXMLRPCServer

import urllib2

def fetch(url):
        g = gevent.spawn(urllib2.urlopen, url)
        return g.get().read()
def is_even(n):
    return n%2 == 0

def req(url):
        return fetch(url)

server = SimpleXMLRPCServer(("localhost", 8000))
print "Listening on port 8000..."
server.register_function(is_even, "is_even")
server.register_function(req, "req")
server.serve_forever()

urllib2.urlopen is blocking server. It looks to me, that monkey.patch_all did not patched socket, that's why it blocks.

Answer 1

The socket is patched fine, but there are other problems with your code.

First, this

def fetch(url):
    g = gevent.spawn(urllib2.urlopen, url)
    return g.get().read()

is the same as

def fetch(url):
    return urllib2.urlopen(url).read()

You're spawning a new greenlet here but then blocking the current one until that new one is done. It does not make things concurrent. It's exactly the same as just running urlopen and waiting for it to finish.

Second, in order to take advantage of gevent there got to be more than one lightweight thread (greenlet) running at the same time.

SimpleXMLRPCServer, however, is defined as

class SimpleXMLRPCServer(SocketServer.TCPServer,
                         SimpleXMLRPCDispatcher):

which means it serves one connection at a time.

If you make your own SimpleXMLRPCServer class, but use ThreadingTCPServer instead of TCPServer, you should be able to benefit from using gevent here.

monkey.patch_all() patches threading to become greenlet-based, so such server will spawn a new greenlet for each new connection.