How do I unload (reload) a Python module?

Question

I have a long-running Python server and would like to be able to upgrade a service without restarting the server. What\'s the best way do do this?

if foo.py          


        

Answer 1

reload(module), but only if it's completely stand-alone. If anything else has a reference to the module (or any object belonging to the module), then you'll get subtle and curious errors caused by the old code hanging around longer than you expected, and things like isinstance not working across different versions of the same code.

If you have one-way dependencies, you must also reload all modules that depend on the reloaded module to get rid of all the references to the old code. And then reload modules that depend on the reloaded modules, recursively.

If you have circular dependencies, which is very common for example when you are dealing with reloading a package, you must unload all the modules in the group in one go. You can't do this with reload() because it will re-import each module before its dependencies have been refreshed, allowing old references to creep into new modules.

The only way to do it in this case is to hack sys.modules, which is kind of unsupported. You'd have to go through and delete each sys.modules entry you wanted to be reloaded on next import, and also delete entries whose values are None to deal with an implementation issue to do with caching failed relative imports. It's not terribly nice but as long as you have a fully self-contained set of dependencies that doesn't leave references outside its codebase, it's workable.

It's probably best to restart the server. :-)

Answer 2

Other option. See that Python default importlib.reload will just reimport the library passed as an argument. It won't reload the libraries that your lib import. If you changed a lot of files and have a somewhat complex package to import, you must do a deep reload.

If you have IPython or Jupyter installed, you can use a function to deep reload all libs:

from IPython.lib.deepreload import reload as dreload
dreload(foo)

If you don't have Jupyter, install it with this command in your shell:

pip3 install jupyter

Answer 3

if 'myModule' in sys.modules:  
    del sys.modules["myModule"]

Answer 4

This is the modern way of reloading a module:

from importlib import reload

If you want to support versions of Python older than 3.5, try this:

from sys import version_info
if version_info[0] < 3:
    pass # Python 2 has built in reload
elif version_info[0] == 3 and version_info[1] <= 4:
    from imp import reload # Python 3.0 - 3.4 
else:
    from importlib import reload # Python 3.5+

To use it, run reload(MODULE), replacing MODULE with the module you want to reload.

For example, reload(math) will reload the math module.

Answer 5

You can reload a module when it has already been imported by using the reload builtin function (Python 3.4+ only):

from importlib import reload  
import foo

while True:
    # Do some things.
    if is_changed(foo):
        foo = reload(foo)

In Python 3, reload was moved to the imp module. In 3.4, imp was deprecated in favor of importlib, and reload was added to the latter. When targeting 3 or later, either reference the appropriate module when calling reload or import it.

I think that this is what you want. Web servers like Django's development server use this so that you can see the effects of your code changes without restarting the server process itself.

To quote from the docs:

Python modules’ code is recompiled and the module-level code reexecuted, defining a new set of objects which are bound to names in the module’s dictionary. The init function of extension modules is not called a second time. As with all other objects in Python the old objects are only reclaimed after their reference counts drop to zero. The names in the module namespace are updated to point to any new or changed objects. Other references to the old objects (such as names external to the module) are not rebound to refer to the new objects and must be updated in each namespace where they occur if that is desired.

As you noted in your question, you'll have to reconstruct Foo objects if the Foo class resides in the foo module.

Answer 6

The accepted answer doesn't handle the from X import Y case. This code handles it and the standard import case as well:

def importOrReload(module_name, *names):
    import sys

    if module_name in sys.modules:
        reload(sys.modules[module_name])
    else:
        __import__(module_name, fromlist=names)

    for name in names:
        globals()[name] = getattr(sys.modules[module_name], name)

# use instead of: from dfly_parser import parseMessages
importOrReload("dfly_parser", "parseMessages")

In the reloading case, we reassign the top level names to the values stored in the newly reloaded module, which updates them.