I was wondering how I can remove all indexes that containing negative values inside their column. I am using Pandas DataFrames
.
Documentation Pandas DataFr
If you want to check the values of an adjacent group of columns, for example from the second to the tenth:
df[(df.ix[:,2:10] > 0).all(1)]
You can also use a range
df[(df.ix[:,range(1,10,3)] > 0).all(1)]
and an own list of indices
mylist=[1,2,4,8]
df[(df.ix[:, mylist] > 0).all(1)]
You can use all
to check an entire row or column is True:
In [11]: df = pd.DataFrame(np.random.randn(10, 3))
In [12]: df
Out[12]:
0 1 2
0 -1.003735 0.792479 0.787538
1 -2.056750 -1.508980 0.676378
2 1.355528 0.307063 0.369505
3 1.201093 0.994041 -1.169323
4 -0.305359 0.044360 -0.085346
5 -0.684149 -0.482129 -0.598155
6 1.795011 1.231198 -0.465683
7 -0.632216 -0.075575 0.812735
8 -0.479523 -1.900072 -0.966430
9 -1.441645 -1.189408 1.338681
In [13]: (df > 0).all(1)
Out[13]:
0 False
1 False
2 True
3 False
4 False
5 False
6 False
7 False
8 False
9 False
dtype: bool
In [14]: df[(df > 0).all(1)]
Out[14]:
0 1 2
2 1.355528 0.307063 0.369505
If you only want to look at a subset of the columns, e.g.[0, 1]
:
In [15]: df[(df[[0, 1]] > 0).all(1)]
Out[15]:
0 1 2
2 1.355528 0.307063 0.369505
3 1.201093 0.994041 -1.169323
6 1.795011 1.231198 -0.465683
To use and statements inside a data-frame you just have to use a single & character and separate each condition with parenthesis.
For example:
data = data[(data['col1']>0) & (data['valuecol2']>0) & (data['valuecol3']>0)]
You could loop over the column names
for cols in data.columns.tolist()[1:]:
data = data.ix[data[cols] > 0]