Cleanup of resources with the SMTP module of Twisted Python

Question

This is related to the previously answered question here: Logging SMTP connections with Twisted. I have a database resource that I create in each instance of ConsoleMessageDelivery that I need to ensure is cleaned up when the socket is closed. I have a WrappingFactory called DenyFactory and the DenyFactory.unregisterProtocol method is called when the socket is closed, but I have no way (that I can figure out) how to access the resource created in the ConsoleMessageDelivery instance that's being destroyed. I tried a del() method in ConsoleMessageDelivery but that's never called. What's the best way to clean up a resource in this scenario?

class ConsoleMessageDelivery:
    implements(smtp.IMessageDelivery)

    def receivedHeader(self, helo, origin, recipients):
        myHostname, clientIP = helo
        headerValue = "by %s from %s with ESMTP ; %s" % (myHostname, clientIP, smtp.rfc822date())
        # email.Header.Header used for automatic wrapping of long lines
        return "Received: %s" % Header(headerValue)

    def validateFrom(self, helo, origin):
        # All addresses are accepted
        return origin

    def validateTo(self, user):
        if user.dest.local == "console":
            return lambda: ConsoleMessage()
        raise smtp.SMTPBadRcpt(user)

class ConsoleMessage:
    implements(smtp.IMessage)

    def __init__(self):
        self.lines = []

    def lineReceived(self, line):
        self.lines.append(line)

    def eomReceived(self):
        return defer.succeed(None)

    def connectionLost(self):
        # There was an error, throw away the stored lines
        self.lines = None

class ConsoleSMTPFactory(smtp.SMTPFactory):
    protocol = smtp.ESMTP

    def __init__(self, *a, **kw):
        smtp.SMTPFactory.__init__(self, *a, **kw)
        self.delivery = ConsoleMessageDelivery()

    def buildProtocol(self, addr):
        p = smtp.SMTPFactory.buildProtocol(self, addr)
        p.delivery = self.delivery
        return p

class DenyFactory(WrappingFactory):

    def buildProtocol(self, clientAddress):
        if clientAddress.host == '1.3.3.7':
            # Reject it
            return None
        # Accept everything else
        return WrappingFactory.buildProtocol(self, clientAddress)

    def unregisterProtocol(self, p):
        print "Unregister called"

Answer 1

First, don't ever use __del__, particularly if you have some resources you want to clean up. __del__ prevents the garbage collection of objects in reference cycles. (Alternatively, switch to PyPy which can collect such objects by imposing an arbitrary ordering on the collection of objects in the cycle.)

Next, consider opening your database connection (or start a connection pool) in the message delivery factory and sharing it between all of the message delivery objects. This way you don't need to clean the connections of, because you'll re-use them for future messages, and you aren't allocating a new one for each message, so there's no leak.

Finally, if you really need any per-transaction objects, you can clean them up in your eomReceived or connectionLost implementations on the IMessage object. One of these methods will be called once the DATA portion of the SMTP transaction is complete (either because all data was received or because the connection was lost). Note that because SMTP supports delivery of a message to multiple recipients in a single transaction, there may be more than one IMessage object participating, even though there is only a single IMessageDelivery object. So you may want to keep a counter - match up the number of calls to successful validateTo calls on the message delivery object with the number of eomReceived/connectionLost calls on the message objects. When the same number of calls of each have happened, the transaction is done.