Step-by-step debugging with IPython

Question

From what I have read, there are two ways to debug code in Python:

• With a traditional debugger such as pdb or ipdb. This supports c

Answer 1

(Update on May 28, 2016) Using RealGUD in Emacs

For anyone in Emacs, this thread shows how to accomplish everything described in the OP (and more) using

1. a new important debugger in Emacs called RealGUD which can operate with any debugger (including ipdb).
2. The Emacs package isend-mode.

The combination of these two packages is extremely powerful and allows one to recreate exactly the behavior described in the OP and do even more.

More info on the wiki article of RealGUD for ipdb.

---

Original answer:

After having tried many different methods for debugging Python, including everything mentioned in this thread, one of my preferred ways of debugging Python with IPython is with embedded shells.

Defining a custom embedded IPython shell:

Add the following on a script to your PYTHONPATH, so that the method ipsh() becomes available.

import inspect

# First import the embed function
from IPython.terminal.embed import InteractiveShellEmbed
from IPython.config.loader import Config

# Configure the prompt so that I know I am in a nested (embedded) shell
cfg = Config()
prompt_config = cfg.PromptManager
prompt_config.in_template = 'N.In <\\#>: '
prompt_config.in2_template = '   .\\D.: '
prompt_config.out_template = 'N.Out<\\#>: '

# Messages displayed when I drop into and exit the shell.
banner_msg = ("\n**Nested Interpreter:\n"
"Hit Ctrl-D to exit interpreter and continue program.\n"
"Note that if you use %kill_embedded, you can fully deactivate\n"
"This embedded instance so it will never turn on again")   
exit_msg = '**Leaving Nested interpreter'

# Wrap it in a function that gives me more context:
def ipsh():
    ipshell = InteractiveShellEmbed(config=cfg, banner1=banner_msg, exit_msg=exit_msg)

    frame = inspect.currentframe().f_back
    msg   = 'Stopped at {0.f_code.co_filename} at line {0.f_lineno}'.format(frame)

    # Go back one level! 
    # This is needed because the call to ipshell is inside the function ipsh()
    ipshell(msg,stack_depth=2)

Then, whenever I want to debug something in my code, I place ipsh() right at the location where I need to do object inspection, etc. For example, say I want to debug my_function below

Using it:

def my_function(b):
  a = b
  ipsh() # <- This will embed a full-fledged IPython interpreter
  a = 4

and then I invoke my_function(2) in one of the following ways:

1. Either by running a Python program that invokes this function from a Unix shell
2. Or by invoking it directly from IPython

Regardless of how I invoke it, the interpreter stops at the line that says ipsh(). Once you are done, you can do Ctrl-D and Python will resume execution (with any variable updates that you made). Note that, if you run the code from a regular IPython the IPython shell (case 2 above), the new IPython shell will be nested inside the one from which you invoked it, which is perfectly fine, but it's good to be aware of. Eitherway, once the interpreter stops on the location of ipsh, I can inspect the value of a (which be 2), see what functions and objects are defined, etc.

The problem:

The solution above can be used to have Python stop anywhere you want in your code, and then drop you into a fully-fledged IPython interpreter. Unfortunately it does not let you add or remove breakpoints once you invoke the script, which is highly frustrating. In my opinion, this is the only thing that is preventing IPython from becoming a great debugging tool for Python.

The best you can do for now:

A workaround is to place ipsh() a priori at the different locations where you want the Python interpreter to launch an IPython shell (i.e. a breakpoint). You can then "jump" between different pre-defined, hard-coded "breakpoints" with Ctrl-D, which would exit the current embedded IPython shell and stop again whenever the interpreter hits the next call to ipsh().

If you go this route, one way to exit "debugging mode" and ignore all subsequent breakpoints, is to use ipshell.dummy_mode = True which will make Python ignore any subsequent instantiations of the ipshell object that we created above.

Answer 2

One option is to use an IDE like Spyder which should allow you to interact with your code while debugging (using an IPython console, in fact). In fact, Spyder is very MATLAB-like, which I presume was intentional. That includes variable inspectors, variable editing, built-in access to documentation, etc.

Answer 3

Looks like the approach in @gaborous's answer is deprecated.

The new approach seems to be:

from IPython.core import debugger
debug = debugger.Pdb().set_trace

def buggy_method():
    debug()

Answer 4

You can start IPython from within ipdb!

Induce the ipdb debugger¹:

import idpb; ipdb.set_trace()

Enter IPython from within in the ipdb> console²:

from IPython import embed; embed()

Return to the ipdb> console from within IPython:

exit

---

If you're lucky enough to be using Emacs, things can be made even more convenient!

This requires using M-x shell. Using yasnippet and bm, define the following snippet. This will replace the text ipdb in the editor with the set-trace line. After inserting the snippet, the line will be highlighted so that it is easily noticeable and navigable. Use M-x bm-next to navigate.

# -*- mode: snippet -*-
# name: ipdb
# key: ipdb
# expand-env: ((yas-after-exit-snippet-hook #'bm-toggle))
# --
import ipdb; ipdb.set_trace()

---

¹ All on one line for easy deletion. Since imports only happen once, this form ensures ipdb will be imported when you need it with no extra overhead.

² You can save yourself some typing by importing IPython within your .pdbrc file:

try:
    from IPython import embed
except:
    pass

This allows you to simply call embed() from within ipdb (of course, only when IPython is installed).

Answer 5

Running from inside Emacs' IPython-shell and breakpoint set via pdb.set_trace() should work.

Checked with python-mode.el, M-x ipython RET etc.

Answer 6

You can start IPython session from pudb and go back to the debugging session as you like.

BTW, ipdb is using IPython behind the scenes and you can actually use IPython functionality such as TAB completion and magic commands (the one starts with %). If you are OK with ipdb you can start it from IPython using commands such as %run and %debug. ipdb session is actually better than plain IPython one in the sense you can go up and down in the stack trace etc. What is missing in ipdb for "object inspection"?

Also, python.el bundled with Emacs >= 24.3 has nice ipdb support.