Circular (or cyclic) imports in Python

Question

What will happen if two modules import each other?

To generalize the problem, what about the cyclic imports in Python?

Answer 1

Cyclic imports terminate, but you need to be careful not to use the cyclically-imported modules during module initialization.

Consider the following files:

a.py:

print "a in"
import sys
print "b imported: %s" % ("b" in sys.modules, )
import b
print "a out"

b.py:

print "b in"
import a
print "b out"
x = 3

If you execute a.py, you'll get the following:

$ python a.py
a in
b imported: False
b in
a in
b imported: True
a out
b out
a out

On the second import of b.py (in the second a in), the Python interpreter does not import b again, because it already exists in the module dict.

If you try to access b.x from a during module initialization, you will get an AttributeError.

Append the following line to a.py:

print b.x

Then, the output is:

$ python a.py
a in                    
b imported: False
b in
a in
b imported: True
a out
Traceback (most recent call last):
  File "a.py", line 4, in <module>
    import b
  File "/home/shlomme/tmp/x/b.py", line 2, in <module>
    import a
 File "/home/shlomme/tmp/x/a.py", line 7, in <module>
    print b.x
AttributeError: 'module' object has no attribute 'x'

This is because modules are executed on import and at the time b.x is accessed, the line x = 3 has not be executed yet, which will only happen after b out.

Answer 2

I completely agree with pythoneer's answer here. But I have stumbled on some code that was flawed with circular imports and caused issues when trying to add unit tests. So to quickly patch it without changing everything you can resolve the issue by doing a dynamic import.

# Hack to import something without circular import issue
def load_module(name):
    """Load module using imp.find_module"""
    names = name.split(".")
    path = None
    for name in names:
        f, path, info = imp.find_module(name, path)
        path = [path]
    return imp.load_module(name, f, path[0], info)
constants = load_module("app.constants")

Again, this isn't a permanent fix but may help someone that wants to fix an import error without changing too much of the code.

Cheers!

Answer 3

As other answers describe this pattern is acceptable in python:

def dostuff(self):
     from foo import bar
     ...

Which will avoid the execution of the import statement when the file is imported by other modules. Only if there is a logical circular dependency, this will fail.

Most Circular Imports are not actually logical circular imports but rather raise ImportError errors, because of the way import() evaluates top level statements of the entire file when called.

These ImportErrors can almost always be avoided if you positively want your imports on top:

Consider this circular import:

App A

# profiles/serializers.py

from images.serializers import SimplifiedImageSerializer

class SimplifiedProfileSerializer(serializers.Serializer):
    name = serializers.CharField()

class ProfileSerializer(SimplifiedProfileSerializer):
    recent_images = SimplifiedImageSerializer(many=True)

App B

# images/serializers.py

from profiles.serializers import SimplifiedProfileSerializer

class SimplifiedImageSerializer(serializers.Serializer):
    title = serializers.CharField()

class ImageSerializer(SimplifiedImageSerializer):
    profile = SimplifiedProfileSerializer()

From David Beazleys excellent talk Modules and Packages: Live and Let Die! - PyCon 2015, 1:54:00, here is a way to deal with circular imports in python:

try:
    from images.serializers import SimplifiedImageSerializer
except ImportError:
    import sys
    SimplifiedImageSerializer = sys.modules[__package__ + '.SimplifiedImageSerializer']

This tries to import SimplifiedImageSerializer and if ImportError is raised, because it already is imported, it will pull it from the importcache.

PS: You have to read this entire post in David Beazley's voice.

Answer 4

There are a lot of great answers here. While there are usually quick solutions to the problem, some of which feel more pythonic than others, if you have the luxury of doing some refactoring, another approach is to analyze the organization of your code, and try to remove the circular dependency. You may find, for example, that you have:

File a.py

from b import B

class A:
    @staticmethod
    def save_result(result):
        print('save the result')

    @staticmethod
    def do_something_a_ish(param):
        A.save_result(A.use_param_like_a_would(param))

    @staticmethod
    def do_something_related_to_b(param):
        B.do_something_b_ish(param)

File b.py

from a import A

class B:
    @staticmethod
    def do_something_b_ish(param):
        A.save_result(B.use_param_like_b_would(param))

In this case, just moving one static method to a separate file, say c.py:

File c.py

def save_result(result):
    print('save the result')

will allow removing the save_result method from A, and thus allow removing the import of A from a in b:

Refactored File a.py

from b import B
from c import save_result

class A:
    @staticmethod
    def do_something_a_ish(param):
        A.save_result(A.use_param_like_a_would(param))

    @staticmethod
    def do_something_related_to_b(param):
        B.do_something_b_ish(param)

Refactored File b.py

from c import save_result

class B:
    @staticmethod
    def do_something_b_ish(param):
        save_result(B.use_param_like_b_would(param))

In summary, if you have a tool (e.g. pylint or PyCharm) that reports on methods that can be static, just throwing a staticmethod decorator on them might not be the best way to silence the warning. Even though the method seems related to the class, it might be better to separate it out, especially if you have several closely related modules that might need the same functionality and you intend to practice DRY principles.

Answer 5

Ok, I think I have a pretty cool solution. Let's say you have file a and file b. You have a def or a class in file b that you want to use in module a, but you have something else, either a def, class, or variable from file a that you need in your definition or class in file b. What you can do is, at the bottom of file a, after calling the function or class in file a that is needed in file b, but before calling the function or class from file b that you need for file a, say import b Then, and here is the key part, in all of the definitions or classes in file b that need the def or class from file a (let's call it CLASS), you say from a import CLASS

This works because you can import file b without Python executing any of the import statements in file b, and thus you elude any circular imports.

For example:

File a:

class A(object):

     def __init__(self, name):

         self.name = name

CLASS = A("me")

import b

go = B(6)

go.dostuff

File b:

class B(object):

     def __init__(self, number):

         self.number = number

     def dostuff(self):

         from a import CLASS

         print "Hello " + CLASS.name + ", " + str(number) + " is an interesting number."

Voila.

Answer 6

Circular imports can be confusing because import does two things:

1. it executes imported module code
2. adds imported module to importing module global symbol table

The former is done only once, while the latter at each import statement. Circular import creates situation when importing module uses imported one with partially executed code. In consequence it will not see objects created after import statement. Below code sample demonstrates it.

Circular imports are not the ultimate evil to be avoided at all cost. In some frameworks like Flask they are quite natural and tweaking your code to eliminate them does not make the code better.

main.py

print 'import b'
import b
print 'a in globals() {}'.format('a' in globals())
print 'import a'
import a
print 'a in globals() {}'.format('a' in globals())
if __name__ == '__main__':
    print 'imports done'
    print 'b has y {}, a is b.a {}'.format(hasattr(b, 'y'), a is b.a)

b.by

print "b in, __name__ = {}".format(__name__)
x = 3
print 'b imports a'
import a
y = 5
print "b out"

a.py

print 'a in, __name__ = {}'.format(__name__)
print 'a imports b'
import b
print 'b has x {}'.format(hasattr(b, 'x'))
print 'b has y {}'.format(hasattr(b, 'y'))
print "a out"

python main.py output with comments

import b
b in, __name__ = b    # b code execution started
b imports a
a in, __name__ = a    # a code execution started
a imports b           # b code execution is already in progress
b has x True
b has y False         # b defines y after a import,
a out
b out
a in globals() False  # import only adds a to main global symbol table 
import a
a in globals() True
imports done
b has y True, a is b.a True # all b objects are available