Daemonizing python's BaseHTTPServer

Question

I am working on a daemon where I need to embed a HTTP server. I am attempting to do it with BaseHTTPServer, which when I run it in the foreground, it works fine, but when I tr

Answer 1

Just use daemontools or some other similar script instead of rolling your own daemonizing process. It is much better to keep this off your script.

Also, your best option: Don't use BaseHTTPServer. It is really bad. There are many good HTTP servers for python, i.e. cherrypy or paste. Both includes ready-to-use daemonizing scripts.

Answer 2

Here's how to do this with the python-daemon library:

from BaseHTTPServer import (HTTPServer, BaseHTTPRequestHandler)
import contextlib

import daemon

from my_app_config import config

# Make the HTTP Server instance.
server = HTTPServer(
    (config['HTTPServer']['listen'], config['HTTPServer']['port']),
    BaseHTTPRequestHandler)

# Make the context manager for becoming a daemon process.
daemon_context = daemon.DaemonContext()
daemon_context.files_preserve = [server.fileno()]

# Become a daemon process.
with daemon_context:
    server.serve_forever()

As usual for a daemon, you need to decide how you will interact with the program after it becomes a daemon. For example, you might register a systemd service, or write a PID file, etc. That's all outside the scope of the question though.

In particular, it's outside the scope of the question to ask: once it's become a daemon process (necessarily detached from any controlling terminal), how do I stop the daemon process? That's up to you to decide, as part of defining the program's behaviour.

Answer 3

After a bit of googling I finally stumbled over this BaseHTTPServer documentation and after that I ended up with:

from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from SocketServer import ThreadingMixIn

class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
  """Handle requests in a separate thread."""

server = ThreadedHTTPServer((config['HTTPServer']['listen'],config['HTTPServer']['port']), HTTPHandler)
server.serve_forever()

Which for the most part comes after I fork and ended up resolving my problem.

Answer 4

Since this has solicited answers since I originally posted, I thought that I'd share a little info.

The issue with the output has to do with the fact that the default handler for the logging module uses the StreamHandler. The best way to handle this is to create your own handlers. In the case where you want to use the default logging module, you can do something like this:

# Get the default logger
default_logger = logging.getLogger('')

# Add the handler
default_logger.addHandler(myotherhandler)

# Remove the default stream handler
for handler in default_logger.handlers:
    if isinstance(handler, logging.StreamHandler):
        default_logger.removeHandler(handler)

Also at this point I have moved to using the very nice Tornado project for my embedded http servers.

Answer 5

A simple solution that worked for me was to override the BaseHTTPRequestHandler method log_message(), so we prevent any kind of writing in stdout and avoid problems when demonizing.

class CustomRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):

    def log_message(self, format, *args):
            pass

...
rest of custom class code
...

Answer 6

You start by instantiating a HTTPServer. But you don't actually tell it to start serving in any of the supplied code. In your child process try calling server.serve_forever().

See this for reference