How I can get rid of None values in dictionary?

Question

Something like:

for (a,b) in kwargs.iteritems():
    if not b : del kwargs[a]

This code raise exception because changing of dictionary when

Answer 1

The recursive approach to also filter nested lists of dicts in the dictionary:

def filter_none(d):
if isinstance(d, dict):
    return {k: filter_none(v) for k, v in d.items() if v is not None}
elif isinstance(d, list):
    return [filter_none(v) for v in d]
else:
    return d

Sample output:

data = {'a': 'b', 'c': None, 'd':{'e': 'f', 'h': None, 'i':[{'j': 'k', 'l': None}]}}
print(filter_none(data))
>>> {'a': 'b', 'd': {'e': 'f', 'i': [{'j': 'k'}]}}

Answer 2

I like the variation of your second method:

   res = dict((a, b) for (a, b) in kwargs.iteritems() if b is not None)

it's Pythonic and I don't think that ugly. A variation of your first is:

   for (a, b) in list(kwargs.iteritems()):
       if b is None:
            del kwargs[a]

Answer 3

To anybody who may interests, here's another way to get rid of None value. Instead of deleting the key, I change the value of None with a placeholder for the same key.

One use case is applying with Spark RDD.map onto null valued JSON.

def filter_null(data, placeholder="[spark]nonexists"):
    # Replace all `None` in the dict to the value of `placeholder`
    return dict((k, filter_null(v, placeholder) if isinstance(v, dict) else v if v 
is not None else placeholder) for k, v in data.iteritems())

Sample output:

>>> filter_null({'a':None,'b':"nul", "c": {'a':None,'b':"nul"}})
{'a': '[spark]nonexists', 'c': {'a': '[spark]nonexists', 'b': 'nul'}, 'b': 'nul'}

For python3, change the iteritems() to items().

Answer 4

You can also use filter:

d = dict(a = 1, b = None, c = 3)

filtered = dict(filter(lambda item: item[1] is not None, d.items()))

print(filtered)
{'a': 1, 'c': 3}

Answer 5

d = {'a': None, 'b': 'myname', 'c': 122}
print dict(filter(lambda x:x[1], d.items()))
{'b': 'myname', 'c': 122}

Answer 6

Another way to write it is

res = dict((k,v) for k,v in kwargs.iteritems() if v is not None)

In Python3, this becomes

res = {k:v for k,v in kwargs.items() if v is not None}