Communicating with a running python daemon

Question

I wrote a small Python application that runs as a daemon. It utilizes threading and queues.

I\'m looking for general approaches to altering this application so that

Answer 1

Use werkzeug and make your daemon include an HTTP-based WSGI server.

Your daemon has a collection of small WSGI apps to respond with status information.

Your client simply uses urllib2 to make POST or GET requests to localhost:somePort. Your client and server must agree on the port number (and the URL's).

This is very simple to implement and very scalable. Adding new commands is a trivial exercise.

Note that your daemon does not have to respond in HTML (that's often simple, though). Our daemons respond to the WSGI-requests with JSON-encoded status objects.

Answer 2

I would use twisted with a named pipe or just open up a socket. Take a look at the echo server and client examples. You would need to modify the echo server to check for some string passed by the client and then respond with whatever requested info.

Because of Python's threading issues you are going to have trouble responding to information requests while simultaneously continuing to do whatever the daemon is meant to do anyways. Asynchronous techniques or forking another processes are your only real option.

Answer 3

Assuming you're under *nix, you can send signals to a running program with kill from a shell (and analogs in many other environments). To handle them from within python check out the signal module.

Answer 4

# your server

from twisted.web import xmlrpc, server
from twisted.internet import reactor

class MyServer(xmlrpc.XMLRPC):

    def xmlrpc_monitor(self, params):        
        return server_related_info

if __name__ == '__main__':
    r = MyServer()
    reactor.listenTCP(8080, Server.Site(r))
    reactor.run()

client can be written using xmlrpclib, check example code here.

Answer 5

What about having it run an http server?

It seems crazy but running a simple web server for administrating your server requires just a few lines using web.py

You can also consider creating a unix pipe.

Answer 6

You can do this using multiprocessing managers (https://docs.python.org/3/library/multiprocessing.html#managers):

Managers provide a way to create data which can be shared between different processes, including sharing over a network between processes running on different machines. A manager object controls a server process which manages shared objects. Other processes can access the shared objects by using proxies.

Example server:

from multiprocessing.managers import BaseManager

class RemoteOperations:
    def add(self, a, b):
        print('adding in server process!')
        return a + b

    def multiply(self, a, b):
        print('multiplying in server process!')
        return a * b

class RemoteManager(BaseManager):
    pass

RemoteManager.register('RemoteOperations', RemoteOperations)

manager = RemoteManager(address=('', 12345), authkey=b'secret')
manager.get_server().serve_forever()

Example client:

from multiprocessing.managers import BaseManager

class RemoteManager(BaseManager):
    pass

RemoteManager.register('RemoteOperations')
manager = RemoteManager(address=('localhost', 12345), authkey=b'secret')
manager.connect()

remoteops = manager.RemoteOperations()
print(remoteops.add(2, 3))
print(remoteops.multiply(2, 3))