'NameError: global name is not defined' under pdb, for dictionary that does exist

Question

I've encountered an issue re scopes in a lambda function. I can successfully output foo to stdout but I get an error when using max() including a lambda - see simplified code below...

All in all, I am trying find the largest value for a nested key budget within an unknown number of first order keys.

(Pdb) foo = self.some_method()    # some_method() returns a dict, printed in the next step

(Pdb) pp foo

{'1': {'count': 1,
       'extra_data': {'activity-count': 1,
                             'budget': 0,
                             [...MORE KEY-VALUE PAIRS HERE...]
                             'version': 1},
       [...LOTS MORE KEY-VALUE PAIRS HERE...]
       'elements_total': defaultdict(<type 'int'>, {'result': 1, 'another_key': 2}),
       'extra_year_data': defaultdict(<function <lambda> at 0x10e05bd70>, {})}, 

 '2': {'count': 1,
       'extra_data': {'activity-count': 1,
                             'budget': 3,
                             [...MORE KEY-VALUE PAIRS HERE...]
                             'version': 1},
       [...LOTS MORE KEY-VALUE PAIRS HERE...]
       'elements_total': defaultdict(<type 'int'>, {'result': 1, 'another_key': 2}),
       'extra_year_data': defaultdict(<function <lambda> at 0x10e05bd70>, {})}}

(Pdb) max(foo, key=lambda x: foo[x]['extra_data']['budget'])
*** NameError: global name 'foo' is not defined

All in all, I am trying to use max(foo, key=lambda x: foo[x]['extra_data']['budget']) to find the largest value for a nested key budget within an unknown number of first order keys.

The expected result in this case could be 2 as the value for foo['2']['extra_data']['budget'] = 3 vs. foo['1']['extra_data']['budget'] = 0.

Could the error be related to the fact that some of the (unrelated) keys have defaultdicts within them?

Answer 1

You set a new local with pdb, but that is not visible to expressions using nested scopes in this debugger session. Any expression in a nested scope such as the lambda used for the key argument, using a name that is local to the current frame, would need to be a closure and will have this problem.

That's a limitation of how the debugger and Python compilation work; closures can only be created if the function that need to produce them was compiled in the same session. Since the function you are debugging was compiled without foo being a closure, it cannot be used by the lambda expression as such.

You can bind the local to the lambda (making it a local rather than a closure):

max(foo, key=lambda x, foo=foo: foo[x]['extra_data']['budget'])

See What exactly is contained within a obj.__closure__? for details on how the Python compiler creates closures.

Answer 2

There is a bug report for Python 3 (however this issue affects Python 2.7 as well as you found out) which suggests a workaround as an alternative to Martijn's solution: interact at the pdb prompt drops you into an interactive session which is populated with globals() and locals() and your lambda should work as expected.