Build a Call graph in python including modules and functions?

Question

I have a bunch of scripts to perform a task. And I really need to know the call graph of the project because it is very confusing. I am not able to execute the code because it needs extra HW and SW to do so. However, I need to understand the logic behind it. So, I need to know if there is a tool (which do not require any python file execution) that can build a call graph using the modules instead of the trace or python parser. I have such tools for C but not for python.
Thank you.

Answer 1

The best tool I've found is called pyan, and was originally written by Edmund Horner, improved by him, and then given colorization and other features by Juha Jeronen. That version has useful commandline options:

Usage: pyan.py FILENAME... [--dot|--tgf]

Analyse one or more Python source files and generate an approximate call graph
of the modules, classes and functions within them.

Options:
  -h, --help           show this help message and exit
  --dot                output in GraphViz dot format
  --tgf                output in Trivial Graph Format
  -v, --verbose        verbose output
  -d, --defines        add edges for 'defines' relationships [default]
  -n, --no-defines     do not add edges for 'defines' relationships
  -u, --uses           add edges for 'uses' relationships [default]
  -N, --no-uses        do not add edges for 'uses' relationships
  -c, --colored        color nodes according to namespace [dot only]
  -g, --grouped        group nodes (create subgraphs) according to namespace
                       [dot only]
  -e, --nested-groups  create nested groups (subgraphs) for nested namespaces
                       (implies -g) [dot only]

Here's the result of running pyan.py --dot -c -e pyan.py | fdp -Tpng:

Edmund Horner's original code is now best found in his github repository, and somebody has also made a repository with both versions, from where you can download Juha Jeronen's version. I've made a clean version combining their contributions into my own repository just for pyan, since both repositories have lots of other software.

Answer 2

You might want to check out pycallgraph:

pycallgraph

Also in this link a more manual approach is described:

generating-call-graphs-for-understanding-and-refactoring-python-code

Answer 3

In short, no such tool exists. Python is far too dynamic of a language to be able to generate a call graph without executing the code.

Here's some code which clearly demonstrates some of the very dynamic features of python:

class my_obj(object):
    def __init__(self, item):
        self.item = item
    def item_to_power(self, power):
        return self.item ** power

def strange_power_call(obj):
    to_call = "item_to_power"
    return getattr(obj, to_call)(4)

a = eval("my" + "_obj" + "(12)")
b = strange_power_call(a)

Note that we're using eval to create an instance of my_obj and also using getattr to call one of its methods. These are both methods that would make it extremely difficult to create a static call graph for python. Additionally, there are all sorts of difficult to analyze ways of importing modules.

I think your best bet is going to be to sit down with the code base and a pad of paper, and start taking notes by hand. This will have the dual benefit of making you more familiar with the code base, and will not be easily tricked by difficult to parse scenarios.

Answer 4

No such tool exists for any program in any language. Creating such a tool would be equivalent to solving the halting problem, which is undecidable. Simply put, given an arbitrary program and its input there is no algorithm to determine whether the program will halt or run forever. Similarly, there's no algorithm that can determine whether function x will call function y, or whether a certain line of code will execute, etc. Clearly for certain programs one can determine these behaviors. E.g, a 1 line program with a print statement will trivially execute the one line and exit. But arbitrary programs can be arbitrarily complex, so it can be proven that no algorithm exists to determine these behaviors for an arbitrary program. Unfortunately, you need to be able to run the program to solve this problem.