I have a DataFrame:df as following:
row id name age url
1 e1 tom NaN http1
2 e2 john 25 NaN
3 e3 lucy NaN
You can use where with fillna and condition by isnull:
df[['age', 'url']] = df[['age', 'url']].where(df[['age', 'url']].isnull(), 1)
.fillna(0).astype(int)
print (df)
row id name age url
0 1 e1 tom 0 1
1 2 e2 john 1 0
2 3 e3 lucy 0 1
3 4 e4 tick 1 0
Or numpy.where with isnull:
df[['age', 'url']] = np.where(df[['age', 'url']].isnull(), 0, 1)
print (df)
row id name age url
0 1 e1 tom 0 1
1 2 e2 john 1 0
2 3 e3 lucy 0 1
3 4 e4 tick 1 0
Fastest solution with notnull and astype:
df[['age', 'url']] = df[['age', 'url']].notnull().astype(int)
print (df)
row id name age url
0 1 e1 tom 0 1
1 2 e2 john 1 0
2 3 e3 lucy 0 1
3 4 e4 tick 1 0
EDIT:
I try modify your solution:
df[['age', 'url']] = df[['age', 'url']].applymap(lambda x: 0 if pd.isnull(x) else 1)
print (df)
row id name age url
0 1 e1 tom 0 1
1 2 e2 john 1 0
2 3 e3 lucy 0 1
3 4 e4 tick 1 0
Timings:
len(df)=4k
:
In [127]: %timeit df[['age', 'url']] = df[['age', 'url']].applymap(lambda x: 0 if pd.isnull(x) else 1)
100 loops, best of 3: 11.2 ms per loop
In [128]: %timeit df[['age', 'url']] = np.where(df[['age', 'url']].isnull(), 0, 1)
100 loops, best of 3: 2.69 ms per loop
In [129]: %timeit df[['age', 'url']] = np.where(pd.notnull(df[['age', 'url']]), 1, 0)
100 loops, best of 3: 2.78 ms per loop
In [131]: %timeit df.loc[:, ['age', 'url']] = df[['age', 'url']].notnull() * 1
1000 loops, best of 3: 1.45 ms per loop
In [136]: %timeit df[['age', 'url']] = df[['age', 'url']].notnull().astype(int)
1000 loops, best of 3: 1.01 ms per loop
df.loc[:, ['age', 'url']] = df[['age', 'url']].notnull() * 1
df
Use np.where
with pd.notnull
to replace the missing and valid elements with 0
and 1
respectively:
In [90]:
df[['age', 'url']] = np.where(pd.notnull(df[['age', 'url']]), 1, 0)
df
Out[90]:
row id name age url
0 1 e1 tom 0 1
1 2 e2 john 1 0
2 3 e3 lucy 0 1
3 4 e4 tick 1 0