Python internals - How do objects know about global variables?

Question

I recently discovered some interesting behavior that left me wondering about how an object knows about which global variables exist. For example, suppose I have a file "test.py":

globalVar = 1
toDelete = 2

class Test(object):
    classVar = 3

    def runTest1(self):
        print globalVar
        print toDelete
        print missingVar

    def runTest2(self):
        print self.classVar
        print toCreate
        print missingVar

Then in an interactive shell I do this:

>>> import test
>>> tester = test.Test()
>>> tester.runTest1()
1
2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 10, in runTest1
    print missingVar
NameError: global name 'missingVar' is not defined
>>> tester.runTest2()
3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 14, in runTest2
    print toCreate
NameError: global name 'toCreate' is not defined

Nothing surprising. Then I change the first few lines of "test.py" to this:

globalVar = 4 
toCreate = 5

class Test(object):
    classVar = 6

Now back to the interactive shell:

>>> reload(test) # test = reload(test) gives the same result 
<module 'test' from 'test.py'>
>>> tester.runTest1()
4
2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 10, in runTest1
    print missingVar
NameError: global name 'missingVar' is not defined
>>> tester.runTest2()
3
5
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 15, in runTest2
    print missingVar
NameError: global name 'missingVar' is not defined
>>> dir(test)
['Test', '__builtins__', '__doc__', '__file__', '__name__', '__package__', 'globalVar', 'toCreate', 'toDelete']

So tester now knows about toCreate, which showed up after tester itself was created. It still knows about toDelete because reloading a module apparently doesn't affect a global variable that was deleted. Here comes a twist:

>>> import sys
>>> import importlib
>>> del(sys.modules['test']) # remove cached version
>>> test = importlib.import_module('test') # same result if I don't have 'test = '
>>> tester.runTest1()
None
None
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 10, in runTest1
    print missingVar
NameError: global name 'missingVar' is not defined
>>> tester.runTest2()
3
None
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 15, in runTest2
    print missingVar
NameError: global name 'missingVar' is not defined
>>> dir(test)
['Test', '__builtins__', '__doc__', '__file__', '__name__', '__package__', 'globalVar', 'toCreate']

Deleting the module from sys.modules and then reimporting the module results in all the global variables becoming None.

Also of interest is that if I delete test and sys.modules['test'], it still knows about the values of the variables for a while. After a little while (which I assume is how long it takes for the module to be garbage collected) the values become None. Reimporting the module causes the garbage collection (or whatever is going on) to happen immediately.

So how does tester find out about a new global variable being created, and then once the module is gone why does it still know which variables existed even though it no longer knows what values they held?

Answer 1

Any name that is not a local (has not been assigned to in the current scope) is instead assumed to be a global. The name is looked up every time the code runs.

So at runtime the name is looked up in the global namespace, which is just a dictionary. If the name doesn't exist at that time, a NameError exception is raised.

You can see this when you disassemble a function; bytecode is shown when using the dis module:

>>> import dis
>>> def foo():
...     bar = 'baz'  # local
...     bar  # reference the local
...     baz  # reference something else; e.g. a global
... 
>>> dis.dis(foo)
  2           0 LOAD_CONST               1 ('baz')
              3 STORE_FAST               0 (bar)

  3           6 LOAD_FAST                0 (bar)
              9 POP_TOP             

  4          10 LOAD_GLOBAL              0 (baz)
             13 POP_TOP             
             14 LOAD_CONST               0 (None)
             17 RETURN_VALUE        

bar is a local (it was assigned to in the block), while baz is a global. The local is referenced by LOAD_FAST, while the global is referenced by LOAD_GLOBAL.

To do this, function objects have a function.__globals__ reference linking it to the module globals mapping; see the User-defined functions section in the datamodel documentation:

>>> foo.__globals__ is globals()
True

Python also cleans up globals when deleting modules; to prevent circular references holding up finalisation globals are rebound to None at that time (although this behaviour changed in Python 3.4). If you however maintain a reference to tester, your code will find those None values.

Your tester instance, still referencing the original class and it's methods, are still referencing their module through their function.__globals__ references. So, although you deleted the sys.modules reference to the module, triggering the module cleanup, the globals dictionary is still referenced by the class methods. This globals dictionary now holds None values for each global.