Calculate difference in keys contained in two Python dictionaries

Question

Suppose I have two Python dictionaries - dictA and dictB. I need to find out if there are any keys which are present in dictB but not

Answer 1

This is an old question and asks a little bit less than what I needed so this answer actually solves more than this question asks. The answers in this question helped me solve the following:

1. (asked) Record differences between two dictionaries
2. Merge differences from #1 into base dictionary
3. (asked) Merge differences between two dictionaries (treat dictionary #2 as if it were a diff dictionary)
4. Try to detect item movements as well as changes
5. (asked) Do all of this recursively

All this combined with JSON makes for a pretty powerful configuration storage support.

The solution (also on github):

from collections import OrderedDict
from pprint import pprint


class izipDestinationMatching(object):
    __slots__ = ("attr", "value", "index")

    def __init__(self, attr, value, index):
        self.attr, self.value, self.index = attr, value, index

    def __repr__(self):
        return "izip_destination_matching: found match by '%s' = '%s' @ %d" % (self.attr, self.value, self.index)


def izip_destination(a, b, attrs, addMarker=True):
    """
    Returns zipped lists, but final size is equal to b with (if shorter) a padded with nulls
    Additionally also tries to find item reallocations by searching child dicts (if they are dicts) for attribute, listed in attrs)
    When addMarker == False (patching), final size will be the longer of a, b
    """
    for idx, item in enumerate(b):
        try:
            attr = next((x for x in attrs if x in item), None)  # See if the item has any of the ID attributes
            match, matchIdx = next(((orgItm, idx) for idx, orgItm in enumerate(a) if attr in orgItm and orgItm[attr] == item[attr]), (None, None)) if attr else (None, None)
            if match and matchIdx != idx and addMarker: item[izipDestinationMatching] = izipDestinationMatching(attr, item[attr], matchIdx)
        except:
            match = None
        yield (match if match else a[idx] if len(a) > idx else None), item
    if not addMarker and len(a) > len(b):
        for item in a[len(b) - len(a):]:
            yield item, item


def dictdiff(a, b, searchAttrs=[]):
    """
    returns a dictionary which represents difference from a to b
    the return dict is as short as possible:
      equal items are removed
      added / changed items are listed
      removed items are listed with value=None
    Also processes list values where the resulting list size will match that of b.
    It can also search said list items (that are dicts) for identity values to detect changed positions.
      In case such identity value is found, it is kept so that it can be re-found during the merge phase
    @param a: original dict
    @param b: new dict
    @param searchAttrs: list of strings (keys to search for in sub-dicts)
    @return: dict / list / whatever input is
    """
    if not (isinstance(a, dict) and isinstance(b, dict)):
        if isinstance(a, list) and isinstance(b, list):
            return [dictdiff(v1, v2, searchAttrs) for v1, v2 in izip_destination(a, b, searchAttrs)]
        return b
    res = OrderedDict()
    if izipDestinationMatching in b:
        keepKey = b[izipDestinationMatching].attr
        del b[izipDestinationMatching]
    else:
        keepKey = izipDestinationMatching
    for key in sorted(set(a.keys() + b.keys())):
        v1 = a.get(key, None)
        v2 = b.get(key, None)
        if keepKey == key or v1 != v2: res[key] = dictdiff(v1, v2, searchAttrs)
    if len(res) <= 1: res = dict(res)  # This is only here for pretty print (OrderedDict doesn't pprint nicely)
    return res


def dictmerge(a, b, searchAttrs=[]):
    """
    Returns a dictionary which merges differences recorded in b to base dictionary a
    Also processes list values where the resulting list size will match that of a
    It can also search said list items (that are dicts) for identity values to detect changed positions
    @param a: original dict
    @param b: diff dict to patch into a
    @param searchAttrs: list of strings (keys to search for in sub-dicts)
    @return: dict / list / whatever input is
    """
    if not (isinstance(a, dict) and isinstance(b, dict)):
        if isinstance(a, list) and isinstance(b, list):
            return [dictmerge(v1, v2, searchAttrs) for v1, v2 in izip_destination(a, b, searchAttrs, False)]
        return b
    res = OrderedDict()
    for key in sorted(set(a.keys() + b.keys())):
        v1 = a.get(key, None)
        v2 = b.get(key, None)
        #print "processing", key, v1, v2, key not in b, dictmerge(v1, v2)
        if v2 is not None: res[key] = dictmerge(v1, v2, searchAttrs)
        elif key not in b: res[key] = v1
    if len(res) <= 1: res = dict(res)  # This is only here for pretty print (OrderedDict doesn't pprint nicely)
    return res

Answer 2

Here is a solution for deep comparing 2 dictionaries keys:

def compareDictKeys(dict1, dict2):
  if type(dict1) != dict or type(dict2) != dict:
      return False

  keys1, keys2 = dict1.keys(), dict2.keys()
  diff = set(keys1) - set(keys2) or set(keys2) - set(keys1)

  if not diff:
      for key in keys1:
          if (type(dict1[key]) == dict or type(dict2[key]) == dict) and not compareDictKeys(dict1[key], dict2[key]):
              diff = True
              break

  return not diff

Answer 3

Not sure if it is still relevant but I came across this problem, my situation i just needed to return a dictionary of the changes for all nested dictionaries etc etc. Could not find a good solution out there but I did end up writing a simple function to do this. Hope this helps,

Answer 4

Here's a way that will work, allows for keys that evaluate to False, and still uses a generator expression to fall out early if possible. It's not exceptionally pretty though.

any(map(lambda x: True, (k for k in b if k not in a)))

EDIT:

THC4k posted a reply to my comment on another answer. Here's a better, prettier way to do the above:

any(True for k in b if k not in a)

Not sure how that never crossed my mind...

Answer 5

There is an other question in stackoverflow about this argument and i have to admit that there is a simple solution explained: the datadiff library of python helps printing the difference between two dictionaries.

Answer 6

Use set():

set(dictA.keys()).intersection(dictB.keys())