How to redirect stderr in Python?

Question

I would like to log all the output of a Python script. I tried:

import sys

log = []

class writer(object):
    def write(self, data):
        log.append(dat         


        

Answer 1

I can't think of an easy way. The python process's standard error is living on a lower level than a python file object (C vs. python).

You could wrap the python script in a second python script and use subprocess.Popen. It's also possible you could pull some magic like this in a single script:

import os
import subprocess
import sys

cat = subprocess.Popen("/bin/cat", stdin=subprocess.PIPE, stdout=subprocess.PIPE)
os.close(sys.stderr.fileno())
os.dup2(cat.stdin.fileno(), sys.stderr.fileno())

And then use select.poll() to check cat.stdout regularly to find output.

Yes, that seems to work.

The problem I foresee is that most of the time, something printed to stderr by python indicates it's about to exit. The more usual way to handle this would be via exceptions.

---------Edit

Somehow I missed the os.pipe() function.

import os, sys
r, w = os.pipe()
os.close(sys.stderr.fileno())
os.dup2(w, sys.stderr.fileno())

Then read from r

Answer 2

To add to Ned's answer, it is difficult to capture the errors on the fly during the compilation.

You can write several print statements in your script and you can stdout to a file, it will stop writing to the file when the error occurs. To debug the code you could check the last logged output and check your script after that point.

Something like this:

# Add to the beginning of the script execution(eg: if __name__ == "__main__":).
from datetime import datetime
dt = datetime.now()
script_dir = os.path.dirname(os.path.abspath(__file__))      # gets the path of the script
stdout_file = script_dir+r'\logs\log'+('').join(str(dt.date()).split("-"))+r'.log'
sys.stdout = open(stdout_file, 'w')

This will create a log file and stream the print statements to the file.

Note: Watch out for escape characters in your filepath while concatenating with script_dir in the second line from the last in the code. You might want something similar to raw string. You can check this thread for this.

Answer 3

You can't do anything in Python code that can capture errors during the compilation of that same code. How could it? If the compiler can't finish compiling the code, it won't run the code, so your redirection hasn't even taken effect yet.

That's where your (undesired) subprocess comes in. You can write Python code that redirects the stdout, then invokes the Python interpreter to compile some other piece of code.

Answer 4

import sys
import tkinter

# ********************************************

def mklistenconsswitch(*printf: callable) -> callable:
    def wrapper(*fcs: callable) -> callable:
        def newf(data):
            [prf(data) for prf in fcs]
        return newf
    stdoutw, stderrw = sys.stdout.write, sys.stderr.write
    funcs = [(wrapper(sys.stdout.write, *printf), wrapper(sys.stderr.write, *printf)), (stdoutw, stderrw)]
    def switch():
        sys.stdout.write, sys.stderr.write = dummy = funcs[0]
        funcs[0] = funcs[1]
        funcs[1] = dummy
    return switch

# ********************************************

def datasupplier():
    i = 5.5
    while i > 0:
        yield i
        i -= .5

def testloop():
    print(supplier.__next__())
    svvitch()
    root.after(500, testloop)

root = tkinter.Tk()
cons = tkinter.Text(root)
cons.pack(fill='both', expand=True)
supplier = datasupplier()
svvitch = mklistenconsswitch(lambda text: cons.insert('end', text))
testloop()
root.mainloop()

Answer 5

Actually, if you're using linux/mac os, you can just use file redirect to do that. For example, if you're going to run "a.py" and record all the messages it will generate into file "a.out", it would just be

python a.py 2>&1 > a.out

The first part redirects stderr to stdout, and the second redirects that to a file called a.out.

For a longer list of redirection operators in Linux/Unix, see https://askubuntu.com/questions/420981/how-do-i-save-terminal-output-to-a-file

Answer 6

Since python 3.5 you can use contextlib.redirect_stderr

with open('help.txt', 'w') as f:
    with redirect_stdout(f):
        help(pow)