Tool for pinpointing circular imports in Python/Django?

问题

I have a Django app and somewhere in it is a recursive import that is causing problems. Because of the size of the app I'm having a problem pinpointing the cause of the circular import.

I know that the answer is "just don't write circular imports" but the problem is I'm having a hard time figuring out where the circular import is coming from, so ideally a tool that traced the import back to its origin would be ideal.

Does such a tool exist? Barring that, I feel like I am doing everything I can to avoid circular import problems -- moving imports to the bottom of the page if possible, moving them inside of functions rather than having them at the top, etc. but still running into problems. I'm wondering if there are any tips or tricks for avoiding them altogether.

To elaborate a bit...

In Django specifically when it encounters a circular import, sometimes it throws an error but sometimes it passes through silently but results in a situation where certain models or fields just aren't there. Frustratingly, this often happens in one context (say, the WSGI server) and not in another (the shell). So testing in the shell something like this will work:

Foo.objects.filter(bar__name='Test')

but in the web throws the error:

FieldError: Cannot resolve keyword 'bar__name' into field. Choices are: ...

With several fields conspicuously missing.

So it can't be a straightforward problem with the code since it does work in the shell but not via the website.

Some tool that figured out just what was going on would be great. ImportError is maybe the least helpful exception message ever.

回答1:

The cause of the import error is easily found, in the backtrace of the ImportError exception.

When you look in the backtrace, you'll see that the module has been imported before. One of it's imports imported something else, executed main code, and now imports that first module. Since the first module was not fully initialized (it was still stuck at it's import code), you now get errors of symbols not found. Which makes sense, because the main code of the module didn't reach that point yet.

Common causes in Django are:

1. Importing a subpackage from a totally different module,

   e.g. from mymodule.admin.utils import ...

   This will load admin/__init__.py first, which likely imports a while load of other packages (e.g. models, admin views). The admin view gets initialized with admin.site.register(..) so the constructor could start importing more stuff. At some point that might hit your module issuing the first statement.

   I had such statement in my middleware, you can guess where that ended me up with. ;)

2. Mixing form fields, widgets and models.

   Because the model can provide a "formfield", you start importing forms. It has a widget. That widget has some constants from .. er... the model. And now you have a loop. Better import that form field class inside the def formfield() body instead of the global module scope.

3. A managers.py that refers to constants of models.py

   After all, the model needs the manager first. The manager can't start importing models.py because it was still initializing. See below, because this is the most simple situation.

4. Using ugettext() instead of ugettext_lazy.

   When you use ugettext(), the translation system needs to initialize. It runs a scan over all packages in INSTALLED_APPS, looking for a locale.XY.formats package. When your app was just initializing itself, it now gets imported again by the global module scan.

   Similar things happen with a scan for plugins, search_indexes by haystack, and other similar mechanisms.

5. Putting way too much in __init__.py.

   This is a combination of points 1 and 4, it stresses the import system because an import of a subpackage will first initialize all parent packages. In effect, a lot of code is running for a simple import and that increases the changes of having to import something from the wrong place.

The solution isn't that hard either. Once you have an idea of what is causing the loop, you remove that import statement out of the global imports (on top of the file) and place it inside a function that uses the symbol. For example:

# models.py:
from django.db import models
from mycms.managers import PageManager

class Page(models.Model)
    PUBLISHED = 1

    objects = PageManager()

    # ....


# managers.py:
from django.db import models

class PageManager(models.Manager):
    def published(self):
        from mycms.models import Page   # Import here to prevent circular imports
        return self.filter(status=Page.PUBLISHED)

In this case, you can see models.py really needs to import managers.py; without it, it can't do the static initialisation of PageManager. The other way around is not so critical. The Page model could easily be imported inside a function instead of globally.

The same applies to any other situation of import errors. The loop may include a few more packages however.

回答2:

One of the common causes of circular imports in Django is using foreign keys in modules that reference each other. Django provides a way to circumvent this by explicitly specifying a model as a string with the full application label:

# from myapp import MyAppModel  ## removed circular import

class MyModel(models.Model):
    myfk = models.ForeignKey(
        'myapp.MyAppModel',  ## avoided circular import
        null=True)

See: https://docs.djangoproject.com/en/dev/ref/models/fields/#foreignkey

回答3:

What I normally do when I encounter an import error is to work my way backwards. I'd get some "cannot import xyz from myproject.views" error, even though xyz exists just fine. Then I do two things:

• I grep my own code for every import of myproject.views and make a (mental) list of modules that import it.

• I check if I import one of those matching modules in views.py in turn. That often gives you the culprit.

A common spot where it can go wrong is your models.py. Often central to what you're doing. But make sure you try to keep your imports pointing AT models.py instead of away from it. So import models from views.py, but not the other way around.

And in urls.py, I normally import my views (because I get a nice immediate import error when I make a mistake that way). But to prevent circular import errors, you can also refer to your views with a dotted path string. But this depends on what you're doing in your urls.py.

A comment regarding placement of imports: keep them at the top of the file. If they're spread out you'll never get a clear picture of which module imports what. Just putting them all at the top (nicely sorted) could already help you pinpoint problems. Only import inside functions if necessary for solving a specific circular import.

And make your imports absolute instead of relative. I mean "from myproject.views import xyz" instead of "from views import xyz". Making it absolute combined with sorting the list of imports makes your imports more clear and neat.

回答4:

Just transforming the comment above in an answer...

If you have circular import, python -vv does the trick. Other way would be to overload the module loader (there's a link somewhere but I can't find it just now). Update: you can probably do it with the ModuleFinder

The silent failure happens because you have multiple modules with the same name. Then, python import order (based on pythonpath) is the reference. Oh, when/if you change the name, make sure you remove the .pyc too :) (it happened to me several times)

回答5:

It can help to visualize the module dependencies using pyreverse.

Install pylint (pyreverse is integrated in pylint) and graphviz (for generating png images):

apt-get install graphviz
pip install pylint

Then generate image with modules dependency graph for modules in <folder>:

pyreverse -o png <folder>

Ideally the dependency graph should flow from bottom up without any circles:

来源：https://stackoverflow.com/questions/9098787/tool-for-pinpointing-circular-imports-in-python-django