Xpath like query for nested python dictionaries

Question

Is there a way to define a XPath type query for nested python dictionaries.

Something like this:

foo = {
  \'spam\':\'eggs\',
  \'morefoo\': {
               


        

Answer 1

If terseness is your fancy:

def xpath(root, path, sch='/'):
    return reduce(lambda acc, nxt: acc[nxt],
                  [int(x) if x.isdigit() else x for x in path.split(sch)],
                  root)

Of course, if you only have dicts, then it's simpler:

def xpath(root, path, sch='/'):
    return reduce(lambda acc, nxt: acc[nxt],
                  path.split(sch),
                  root)

Good luck finding any errors in your path spec tho ;-)

Answer 2

One of the best libraries I've been able to identify, which, in addition, is very actively developed, is an extracted project from boto: JMESPath. It has a very powerful syntax of doing things that would normally take pages of code to express.

Here are some examples:

search('foo | bar', {"foo": {"bar": "baz"}}) -> "baz"
search('foo[*].bar | [0]', {
    "foo": [{"bar": ["first1", "second1"]},
            {"bar": ["first2", "second2"]}]}) -> ["first1", "second1"]
search('foo | [0]', {"foo": [0, 1, 2]}) -> [0]

Answer 3

There is an easier way to do this now.

http://github.com/akesterson/dpath-python

$ easy_install dpath
>>> dpath.util.search(YOUR_DICTIONARY, "morefoo/morebar")

... done. Or if you don't like getting your results back in a view (merged dictionary that retains the paths), yield them instead:

$ easy_install dpath
>>> for (path, value) in dpath.util.search(YOUR_DICTIONARY, "morefoo/morebar", yielded=True)

... and done. 'value' will hold {'bacon': 'foobar'} in that case.

Answer 4

def Dict(var, *arg, **kwarg):
  """ Return the value of an (imbricated) dictionnary, if all fields exist else return "" unless "default=new_value" specified as end argument
      Avoid TypeError: argument of type 'NoneType' is not iterable
      Ex: Dict(variable_dict, 'field1', 'field2', default = 0)
  """
  for key in arg:
    if isinstance(var, dict) and key and key in var:  var = var[key]
    else:  return kwarg['default'] if kwarg and 'default' in kwarg else ""   # Allow Dict(var, tvdbid).isdigit() for example
  return kwarg['default'] if var in (None, '', 'N/A', 'null') and kwarg and 'default' in kwarg else "" if var in (None, '', 'N/A', 'null') else var

foo = {
  'spam':'eggs',
  'morefoo': {
               'bar':'soap',
               'morebar': {'bacon' : 'foobar'}
              }
   }
print Dict(foo, 'morefoo', 'morebar')
print Dict(foo, 'morefoo', 'morebar', default=None)

Have a SaveDict(value, var, *arg) function that can even append to lists in dict...

Answer 5

More work would have to be put into how the XPath-like selector would work. '/' is a valid dictionary key, so how would

foo={'/':{'/':'eggs'},'//':'ham'}

be handled?

foo.select("///")

would be ambiguous.

Answer 6

There is the newer jsonpath-rw library supporting a JSONPATH syntax but for python dictionaries and arrays, as you wished.

So your 1st example becomes:

from jsonpath_rw import parse

print( parse('$.morefoo.morebar').find(foo) )

And the 2nd:

print( parse("$.morefoo[0].morebar.bacon").find(foo) )

---

PS: An alternative simpler library also supporting dictionaries is python-json-pointer with a more XPath-like syntax.