Suppressing printout of “Exception … ignored” message in Python 3

Question

There is a known issue in Python, where \"close failed in file object destructor\" when \"Broken pipe\" happens on stdout - Python tracker Issue 11380; also seen in python - Why

Answer 1

Just some more notes about this - problem still not solved... first:

Issue 6294: Improve shutdown exception ignored message - Python tracker

This error message is generated in PyErr_WriteUnraisable, which is called from many contexts, including __del__ methods. A __del__ method called during shutdown is most likely what is generating the error you are speaking of, but as far as I know the __del__ method has no way to know that it is being called during shutdown in particular. So the proposed fix to the message won't work. [....]
However, because this is a message you can't even trap it should be completely safe to change it.

Well, thanks for this message that you cannot trap, very convenient. I believe this is somehow related to Ignore exceptions printed to stderr in del() - Stack Overflow, although that post (apparently) talks about custom __del__ methods.

Using a bit of the following resources:

• Use Python’s sys.settrace() for fun and for profit | Reminiscential: of or pertaining to remembrance
• Python Global Exception Handling - Stack Overflow

... I modified the script, so I overload all possible handlers I can, to see if there isn't a space somewhere where I can "handle" this exception so it isn't "ignored":

import sys
import atexit
import signal
import inspect, pprint

def signalPIPE_handler(signal, frame):
    sys.stderr.write('signalPIPE_handler!'+str(sys.exc_info())+'\n')
    return #sys.exit(0) # just return doesn't exit!
signal.signal(signal.SIGPIPE, signalPIPE_handler)

_old_excepthook = sys.excepthook
def myexcepthook(exctype, value, intraceback):
  import sys
  import traceback
  sys.stderr.write("myexcepthook\n")
  if exctype == IOError:
    sys.stderr.write(" IOError intraceback:\n")
    traceback.print_tb(intraceback)
  else:
    _old_excepthook(exctype, value, intraceback)
sys.excepthook = myexcepthook

def _trace(frame, event, arg):
  if event == 'exception':
    while frame is not None:
      filename, lineno = frame.f_code.co_filename, frame.f_lineno
      sys.stderr.write("_trace exc frame: " + filename \
        + " " + str(lineno) + " " + str(frame.f_trace) + str(arg) + "\n")
      if arg[0] == IOError:
        myexcepthook(arg[0], arg[1], arg[2])
      frame = frame.f_back
  return _trace
sys.settrace(_trace)

def exiter():
  import sys
  sys.stderr.write("Exiting\n")
atexit.register(exiter)

def main():
  teststr = "Hello " * 5
  try:
    sys.stdout.write(teststr + "\n")
    sys.stdout.flush()
  except IOError:
    sys.stderr.write("Exc: " + str(sys.exc_info()[0]) + "\n")
    #sys.exit(0)


if __name__ == "__main__":
  main()

Note the difference in how this script runs:

$ python2.7 testprint.py | echo

signalPIPE_handler!(None, None, None)
_trace exc frame: testprint.py 44 (, (32, 'Broken pipe'), )
myexcepthook
 IOError intraceback:
  File "testprint.py", line 44, in main
    sys.stdout.flush()
_trace exc frame: testprint.py 51 None(, (32, 'Broken pipe'), )
myexcepthook
 IOError intraceback:
  File "testprint.py", line 44, in main
    sys.stdout.flush()
Exc: 
Exiting

$ python3.2 testprint.py | echo

signalPIPE_handler!(None, None, None)
_trace exc frame: testprint.py 44 (, (32, 'Broken pipe'), )
myexcepthook
 IOError intraceback:
  File "testprint.py", line 44, in main
    sys.stdout.flush()
_trace exc frame: testprint.py 51 None(, (32, 'Broken pipe'), )
myexcepthook
 IOError intraceback:
  File "testprint.py", line 44, in main
    sys.stdout.flush()
Exc: 
signalPIPE_handler!(None, None, None)
Exiting
signalPIPE_handler!(None, None, None)
Exception IOError: (32, 'Broken pipe') in <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'> ignored

Note that signalPIPE_handler runs two times more in Python 3! I think, if there was some sort of "exception queue" in Python, I could "peek" in it, and remove remaining events in the signalPIPE_handler, so as to suppress the Exception ... ignored message... but I don't know of any such thing.

Finally, these resources are nice when trying to debug with gdb:

• c - gdb - debugging with pipe - Stack Overflow
• linux - Using gdb to single-step assembly code outside specified executable causes error "cannot find bounds of current function" - Stack Overflow

... since I don't have python3-dbg, all this reduces to stepping through machine instructions (layout asm in gdb, then Ctrl-X + A), which doesn't really tell me much. But here is how to trigger the problem in gdb:

In one terminal:

$ mkfifo foo 
$ gdb python3.2
...
Reading symbols from /usr/bin/python3.2...(no debugging symbols found)...done.
(gdb) run testprint.py > foo
Starting program: /usr/bin/python3.2 testprint.py > foo

Here it will block; in another terminal in the same diretory do:

$ echo 

... then return to first terminal - you should see:

...
Starting program: /usr/bin/python3.2 testprint.py > foo
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".

Program received signal SIGPIPE, Broken pipe.
0x0012e416 in __kernel_vsyscall ()
(gdb) bt
#0  0x0012e416 in __kernel_vsyscall ()
#1  0x0013c483 in __write_nocancel () from /lib/i386-linux-gnu/libpthread.so.0
#2  0x0815b549 in ?? ()
#3  0x08170507 in ?? ()
#4  0x08175e43 in PyObject_CallMethodObjArgs ()
#5  0x0815df21 in ?? ()
#6  0x0815f94e in ?? ()
#7  0x0815fb05 in ?? ()
#8  0x08170507 in ?? ()
#9  0x08175cb1 in _PyObject_CallMethod_SizeT ()
#10 0x08164851 in ?? ()
#11 0x080a3a36 in PyEval_EvalFrameEx ()
#12 0x080a3a53 in PyEval_EvalFrameEx ()
#13 0x080a43c8 in PyEval_EvalCodeEx ()
#14 0x080a466f in PyEval_EvalCode ()
#15 0x080c6e9d in PyRun_FileExFlags ()
#16 0x080c70c0 in PyRun_SimpleFileExFlags ()
#17 0x080db537 in Py_Main ()
#18 0x0805deee in main ()
(gdb) finish
Run till exit from #0  0x0012e416 in __kernel_vsyscall ()
0x0013c483 in __write_nocancel () from /lib/i386-linux-gnu/libpthread.so.0
...

Unfortunately, I don't have the possibility to build Python3 from source and debug it now; so I'll hope for an answer from someone who knows :)

Cheers!