I\'m developing an app using a Python library urllib
and it is sometimes rising exceptions due to not being able to access an URL.
However, the exception is
urllib can raise a lot of exceptions.
You need to put a try block around the call into urllib and figure how to handle the exceptions for example:
try:
resp = urllib.request.urlopen(req)
except URLError as e:
# analyse e to figure out the detail
...
Certainly under python2's urllib lots of other exceptions are thrown. I'm not sure about python3's urllib.
pdb
has only incremental frame positioning (moving up or down the list of frames).
To get the feature you want, you can try trepan
(github repository). It has an IPython extension here. You then use the command frame -1
once the exception shows up:
Frame (absolute frame positioning)
frame [thread-Name*|*thread-number] [frame-number]
Change the current frame to frame frame-number if specified, or the current frame, 0, if no frame number specified.
If a thread name or thread number is given, change the current frame to a frame in that thread. Dot (.) can be used to indicate the name of the current frame the debugger is stopped in.
A negative number indicates the position from the other or least-recently-entered end. So frame -1 moves to the oldest frame, and frame 0 moves to the newest frame. Any variable or expression that evaluates to a number can be used as a position, however due to parsing limitations, the position expression has to be seen as a single blank-delimited parameter. That is, the expression (5*3)-1 is okay while (5 * 3) - 1) isn’t.
Once you are in the desired frame, you can use edit
to modify your code.
You may find the command backtrace
useful too as it gives a stack trace with the less recent call at the bottom.
trepan
depends on uncompyle6
available here.
pydb
provides a similar feature but was unfortunately not ported to Python3.
Otherwise, you may decide to be patient and wait for improvements. In IPython/core/debugger.py:
"""
Pdb debugger class.
Modified from the standard pdb.Pdb class to avoid including readline, so that
the command line completion of other programs which include this isn't damaged.
In the future, this class will be expanded with improvements over the standard pdb.
[...]
"""
It can be done with some hacking. These docs show how you can turn on post-mortem debugging with the following code in the entry point:
import sys
from IPython.core import ultratb
sys.excepthook = ultratb.FormattedTB(mode='Verbose',
color_scheme='Linux', call_pdb=1)
Stepping through this hook after an exception is raised shows that we need to tinker with the debugger
method. Unfortunately I can see no better way to do this other than to copy the entire method and modify it where needed (I tried modifying self.tb
but traceback objects are read only and can't be used with copy.deepcopy
). Here's a demo:
import json
import sys
from IPython.core import debugger, ultratb
from IPython.core.display_trap import DisplayTrap
class CustomTB(ultratb.FormattedTB):
def debugger(self, force=False):
if force or self.call_pdb:
if self.pdb is None:
self.pdb = debugger.Pdb(
self.color_scheme_table.active_scheme_name)
# the system displayhook may have changed, restore the original
# for pdb
display_trap = DisplayTrap(hook=sys.__displayhook__)
with display_trap:
self.pdb.reset()
# Find the right frame so we don't pop up inside ipython itself
if hasattr(self, 'tb') and self.tb is not None:
etb = self.tb
else:
etb = self.tb = sys.last_traceback
# only modification is here -----+
# |
# V
while self.tb is not None and '/lib/python3' not in self.tb.tb_next.tb_frame.f_code.co_filename:
self.tb = self.tb.tb_next
if etb and etb.tb_next:
etb = etb.tb_next
self.pdb.botframe = etb.tb_frame
self.pdb.interaction(self.tb.tb_frame, self.tb)
if hasattr(self, 'tb'):
del self.tb
sys.excepthook = CustomTB(mode='Verbose',
color_scheme='Linux', call_pdb=1)
def foo():
bar()
def bar():
json.dumps(json)
foo()
As you can see it stops searching through the traceback when it's about to reach library code. Here's the result:
TypeErrorTraceback (most recent call last)
/Users/alexhall/Dropbox/python/sandbox3/sandbox.py in <module>()
40 json.dumps(json)
41
---> 42 foo()
global foo = <function foo at 0x1031358c8>
/Users/alexhall/Dropbox/python/sandbox3/sandbox.py in foo()
35
36 def foo():
---> 37 bar()
global bar = <function bar at 0x103135950>
38
39 def bar():
/Users/alexhall/Dropbox/python/sandbox3/sandbox.py in bar()
38
39 def bar():
---> 40 json.dumps(json)
global json.dumps = <function dumps at 0x10168b268>
global json = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
41
42 foo()
/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py in dumps(obj=<module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False, **kw={})
228 cls is None and indent is None and separators is None and
229 default is None and not sort_keys and not kw):
--> 230 return _default_encoder.encode(obj)
global _default_encoder.encode = <bound method JSONEncoder.encode of <json.encoder.JSONEncoder object at 0x10166e8d0>>
obj = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
231 if cls is None:
232 cls = JSONEncoder
/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/encoder.py in encode(self=<json.encoder.JSONEncoder object>, o=<module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>)
197 # exceptions aren't as detailed. The list call should be roughly
198 # equivalent to the PySequence_Fast that ''.join() would do.
--> 199 chunks = self.iterencode(o, _one_shot=True)
chunks = undefined
self.iterencode = <bound method JSONEncoder.iterencode of <json.encoder.JSONEncoder object at 0x10166e8d0>>
o = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
global _one_shot = undefined
200 if not isinstance(chunks, (list, tuple)):
201 chunks = list(chunks)
/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/encoder.py in iterencode(self=<json.encoder.JSONEncoder object>, o=<module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>, _one_shot=True)
255 self.key_separator, self.item_separator, self.sort_keys,
256 self.skipkeys, _one_shot)
--> 257 return _iterencode(o, 0)
_iterencode = <_json.Encoder object at 0x1031296d8>
o = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
258
259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/encoder.py in default(self=<json.encoder.JSONEncoder object>, o=<module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>)
178
179 ""
--> 180 raise TypeError(repr(o) + " is not JSON serializable")
global TypeError = undefined
global repr = undefined
o = <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'>
181
182 def encode(self, o):
TypeError: <module 'json' from '/Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py'> is not JSON serializable
> /Users/alexhall/Dropbox/python/sandbox3/sandbox.py(40)bar()
38
39 def bar():
---> 40 json.dumps(json)
41
42 foo()
ipdb> down
> /Users/alexhall/.pyenv/versions/3.5.0/lib/python3.5/json/__init__.py(230)dumps()
228 cls is None and indent is None and separators is None and
229 default is None and not sort_keys and not kw):
--> 230 return _default_encoder.encode(obj)
231 if cls is None:
232 cls = JSONEncoder
ipdb>
Basically the full traceback is still printed out but ipdb
starts at your own code. If you enter the down
command you find yourself in a library frame.
I think the answer is no.
pdb stops at the exception and shows you the stack.
Why would it be useful to hide the real source of the exception?
If it worked as you seem to be requesting and hides the 6 layers of stack how would you work out what to fix?
If this is still not on topic please add to your question.
I would go with modifying the code:
try:
resp = urllib.request.urlopen(req)
except Exception as e:
raise RuntimeError(e)
That way:
You may also monkeypatch urllib.request.urlopen()
function:
class MonkeyPatchUrllib(object):
def __enter__(self):
self.__urlopen = urllib.request.urlopen
urllib.request.urlopen = self
def __exit__(self, exception_type, exception_value, traceback):
urllib.request.urlopen = self.__urlopen
def __call__(self, *args, **kwargs):
try:
return self.__urlopen(*args, **kwargs)
except Exception as e:
raise RuntimeError(e)
Any time you have an exception raised in urlibopen()
call within the context manager scope:
with MonkeyPatchUrllib():
#your code here
%pdb will move you only 1 level away from your code.
[EDIT]
With sys.exc_info()
it is possible to preserve a more verbose context of the original exception (like its traceback).