We start several crawlers. If 1 hits an error then an email is sent. The problem is that the log is sometimes too large (they are flat files).
question: how could we