can a python script know that another instance of the same script is running… and then talk to it?

Question

I\'d like to prevent multiple instances of the same long-running python command-line script from running at the same time, and I\'d like the new instance to be able to send data

Answer 1

The Alex Martelli approach of setting up a communications channel is the appropriate one. I would use a multiprocessing.connection.Listener to create a listener, in your choice. Documentation at: http://docs.python.org/library/multiprocessing.html#multiprocessing-listeners-clients

Rather than using AF_INET (sockets) you may elect to use AF_UNIX for Linux and AF_PIPE for Windows. Hopefully a small "if" wouldn't hurt.

Edit: I guess an example wouldn't hurt. It is a basic one, though.

#!/usr/bin/env python

from multiprocessing.connection import Listener, Client
import socket
from array import array
from sys import argv

def myloop(address):
    try:
        listener = Listener(*address)
        conn = listener.accept()
        serve(conn)
    except socket.error, e:
        conn = Client(*address)
        conn.send('this is a client')
        conn.send('close')

def serve(conn):
    while True:
        msg = conn.recv()
        if msg.upper() == 'CLOSE':
            break
        print msg
    conn.close()

if __name__ == '__main__':
    address = ('/tmp/testipc', 'AF_UNIX')
    myloop(address)

This works on OS X, so it needs testing with both Linux and (after substituting the right address) Windows. A lot of caveats exists from a security point, the main one being that conn.recv unpickles its data, so you are almost always better of with recv_bytes.

Answer 2

Sounds like your best bet is sticking with a pid file but have it not only contain the process Id - have it also include the port number that the prior instance is listening on. So when starting up check for the pid file and if present see if a process with that Id is running - if so send your data to it and quit otherwise overwrite the pid file with the current process's info.

Answer 3

Perhaps try using sockets for communication?

Answer 4

The general approach is to have the script, on startup, set up a communication channel in a way that's guaranteed to be exclusive (other attempts to set up the same channel fail in a predictable way) so that further instances of the script can detect the first one's running and talk to it.

Your requirements for cross-platform functionality strongly point towards using a socket as the communication channel in question: you can designate a "well known port" that's reserved for your script, say 12345, and open a socket on that port listening to localhost only (127.0.0.1). If the attempt to open that socket fails, because the port in question is "taken", then you can connect to that port number instead, and that will let you communicate with the existing script.

If you're not familiar with socket programming, there's a good HOWTO doc here. You can also look at the relevant chapter in Python in a Nutshell (I'm biased about that one, of course;-).