Python pandas apply function if a column value is not NULL

Question

I have a dataframe (in Python 2.7, pandas 0.15.0):

df=
       A    B               C
0    NaN   11             NaN
1    two  NaN  [\'foo\', \'bar\']
2  three   3         


        

Answer 1

The problem is that pd.notnull(['foo', 'bar']) operates elementwise and returns array([ True, True], dtype=bool). Your if condition trys to convert that to a boolean, and that's when you get the exception.

To fix it, you could simply wrap the isnull statement with np.all:

df[['A','C']].apply(lambda x: my_func(x) if(np.all(pd.notnull(x[1]))) else x, axis = 1)

Now you'll see that np.all(pd.notnull(['foo', 'bar'])) is indeed True.

Answer 2

Also another way is to just use row.notnull().all() (without numpy), here is an example:

df.apply(lambda row: func1(row) if row.notnull().all() else func2(row), axis=1)

Here is a complete example on your df:

>>> d = {'A': [None, 2, 3, 4], 'B': [11, None, 33, 4], 'C': [None, ['a','b'], None, 4]}
>>> df = pd.DataFrame(d)
>>> df
     A     B       C
0  NaN  11.0    None
1  2.0   NaN  [a, b]
2  3.0  33.0    None
3  4.0   4.0       4
>>> def func1(r):
...     return 'No'
...
>>> def func2(r):
...     return 'Yes'
...
>>> df.apply(lambda row: func1(row) if row.notnull().all() else func2(row), axis=1)
0    Yes
1    Yes
2    Yes
3     No

And a friendlier screenshot :-)

Answer 3

I had a column contained lists and NaNs. So, the next one worked for me.

df.C.map(lambda x: my_func(x) if type(x) == list else x)

Answer 4

Try...

df['a'] = df['a'].apply(lambda x: x.replace(',','\,') if x != None else x)

this example just adds an escape character to a comma if the value is not None