Circular import dependency in Python

Question

Let\'s say I have the following directory structure:

a\\
    __init__.py
    b\\
        __init__.py
        c\\
            __init__.py
            c_file.p         


        

Answer 1

If a depends on c and c depends on a, aren't they actually the same unit then?

You should really examine why you have split a and c into two packages, because either you have some code you should split off into another package (to make them both depend on that new package, but not each other), or you should merge them into one package.

Answer 2

Another solution is to use a proxy for the d_file.

For example, let's say that you want to share the blah class with the c_file. The d_file thus contains:

class blah:
    def __init__(self):
        print("blah")

Here is what you enter in c_file.py:

# do not import the d_file ! 
# instead, use a place holder for the proxy of d_file
# it will be set by a's __init__.py after imports are done
d_file = None 

def c_blah(): # a function that calls d_file's blah
    d_file.blah()

And in a's init.py:

from b.c import c_file
from b.d import d_file

class Proxy(object): # module proxy
    pass
d_file_proxy = Proxy()
# now you need to explicitly list the class(es) exposed by d_file
d_file_proxy.blah = d_file.blah 
# finally, share the proxy with c_file
c_file.d_file = d_file_proxy

# c_file is now able to call d_file.blah
c_file.c_blah() 

Answer 3

The problem is that when running from a directory, by default only the packages that are sub directories are visible as candidate imports, so you cannot import a.b.d. You can however import b.d. since b is a sub package of a.

If you really want to import a.b.d in c/__init__.py you can accomplish this by changing the system path to be one directory above a and change the import in a/__init__.py to be import a.b.c.

Your a/__init__.py should look like this:

import sys
import os
# set sytem path to be directory above so that a can be a 
# package namespace
DIRECTORY_SCRIPT = os.path.dirname(os.path.realpath(__file__)) 
sys.path.insert(0,DIRECTORY_SCRIPT+"/..")
import a.b.c

An additional difficulty arises when you want to run modules in c as scripts. Here the packages a and b do not exist. You can hack the __int__.py in the c directory to point the sys.path to the top-level directory and then import __init__ in any modules inside c to be able to use the full path to import a.b.d. I doubt that it is good practice to import __init__.py but it has worked for my use cases.

Answer 4

Circular Dependencies due to Type Hints

With type hints, there are more opportunities for creating circular imports. Fortunately, there is a solution using the special constant: typing.TYPE_CHECKING.

The following example defines a Vertex class and an Edge class. An edge is defined by two vertices and a vertex maintains a list of the adjacent edges to which it belongs.

 

Without Type Hints, No Error

File: vertex.py

class Vertex:
    def __init__(self, label):
        self.label = label
        self.adjacency_list = []

File: edge.py

class Edge:
    def __init__(self, v1, v2):
        self.v1 = v1
        self.v2 = v2

 

Type Hints Cause ImportError

ImportError: cannot import name 'Edge' from partially initialized module 'edge' (most likely due to a circular import)

File: vertex.py

from typing import List
from edge import Edge


class Vertex:
    def __init__(self, label: str):
        self.label = label
        self.adjacency_list: List[Edge] = []

File: edge.py

from vertex import Vertex


class Edge:
    def __init__(self, v1: Vertex, v2: Vertex):
        self.v1 = v1
        self.v2 = v2

 

Solution using TYPE_CHECKING

File: vertex.py

from typing import List, TYPE_CHECKING

if TYPE_CHECKING:
    from edge import Edge


class Vertex:
    def __init__(self, label: str):
        self.label = label
        self.adjacency_list: List['Edge'] = []

File: edge.py

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from vertex import Vertex


class Edge:
    def __init__(self, v1: 'Vertex', v2: 'Vertex'):
        self.v1 = v1
        self.v2 = v2

 

Quoted vs. Unquoted Type Hints

In versions of Python prior to 3.10, conditionally imported types must be enclosed in quotes, making them “forward references”, which hides them from the interpreter runtime.

In Python 3.7, 3.8, and 3.9, a workaround is to use the following special import.

from __future__ import annotations

This enables using unquoted type hints combined with conditional imports.

Python 3.10 (See PEP 563 -- Postponed Evaluation of Annotations)

In Python 3.10, function and variable annotations will no longer be evaluated at definition time. Instead, a string form will be preserved in the respective annotations dictionary. Static type checkers will see no difference in behavior, whereas tools using annotations at runtime will have to perform postponed evaluation.

The string form is obtained from the AST during the compilation step, which means that the string form might not preserve the exact formatting of the source. Note: if an annotation was a string literal already, it will still be wrapped in a string.

Answer 5

I've wondered this a couple times (usually while dealing with models that need to know about each other). The simple solution is just to import the whole module, then reference the thing that you need.

So instead of doing

from models import Student

in one, and

from models import Classroom

in the other, just do

import models

in one of them, then call models.Classroom when you need it.

Answer 6

You may defer the import, for example in a/__init__.py:

def my_function():
    from a.b.c import Blah
    return Blah()

that is, defer the import until it is really needed. However, I would also have a close look at my package definitions/uses, as a cyclic dependency like the one pointed out might indicate a design problem.