what's the inverse of the quantile function on a pandas Series?

问题

The quantile functions gives us the quantile of a given pandas series s,

E.g.

s.quantile(0.9) is 4.2

Is there the inverse function (i.e. cumulative distribution) which finds the value x such that

s.quantile(x)=4

Thanks

回答1:

I had the same question as you did! I found an easy way of getting the inverse of quantile using scipy.

#libs required
from scipy import stats
import pandas as pd
import numpy as np

#generate ramdom data with same seed (to be reproducible)
np.random.seed(seed=1)
df = pd.DataFrame(np.random.uniform(0,1,(10)), columns=['a'])

#quantile function
x = df.quantile(0.5)[0]

#inverse of quantile
stats.percentileofscore(df['a'],x)

回答2:

Sorting can be expensive, if you look for a single value I'd guess you'd be better of computing it with:

s = pd.Series(np.random.uniform(size=1000))
( s < 0.7 ).astype(int).mean() # =0.7ish

There's probably a way to avoid the int(bool) shenanigan.

回答3:

There's no 1-liner that I know of, but you can achieve this with scipy:

import pandas as pd
import numpy as np
from scipy.interpolate import interp1d

# set up a sample dataframe
df = pd.DataFrame(np.random.uniform(0,1,(11)), columns=['a'])
# sort it by the desired series and caculate the percentile
sdf = df.sort('a').reset_index()
sdf['b'] = sdf.index / float(len(sdf) - 1)
# setup the interpolator using the value as the index
interp = interp1d(sdf['a'], sdf['b'])

# a is the value, b is the percentile
>>> sdf
    index         a    b
0      10  0.030469  0.0
1       3  0.144445  0.1
2       4  0.304763  0.2
3       1  0.359589  0.3
4       7  0.385524  0.4
5       5  0.538959  0.5
6       8  0.642845  0.6
7       6  0.667710  0.7
8       9  0.733504  0.8
9       2  0.905646  0.9
10      0  0.961936  1.0

Now we can see that the two functions are inverses of each other.

>>> df['a'].quantile(0.57)
0.61167933268395969
>>> interp(0.61167933268395969)
array(0.57)
>>> interp(df['a'].quantile(0.43))
array(0.43)

interp can also take in list, a numpy array, or a pandas data series, any iterator really!

回答4:

Just came across the same problem. Here's my two cents.

def inverse_percentile(arr, num):
    arr = sorted(arr)
    i_arr = [i for i, x in enumerate(arr) if x > num]

    return i_arr[0] / len(arr) if len(i_arr) > 0 else 1

回答5:

Mathematically speaking, you're trying to find the CDF or return the probability of s being smaller than or equal to a value or quantile of q:

F(q) = Pr[s <= q]

One can use numpy and try this one-line code:

np.mean(s.to_numpy() <= q)

回答6:

You can use the ECDF function from statsmodels. ECDF stands for empirical distribution function, "empirical" referring to the fact that the function it's creating is based on what is observed in your data.

Suppose you have a series s:

import pandas as pd
s = pd.Series(np.random.uniform(size=1000))

You can evaluate the CDF at 0.282:

(s <= 0.282).mean()

Or you can create the ECDF using the statsmodels function:

from statsmodels.distributions.empirical_distribution import ECDF

ecdf_s = ECDF(s)

ecdf_s

[ecdf_s(k) for k in [0.282, 0.544, 0.775]]

And check that it is the inverse of the quantiles:

s.quantile([0.25, 0.50, 0.75])

来源：https://stackoverflow.com/questions/26489134/whats-the-inverse-of-the-quantile-function-on-a-pandas-series

标签

python

pandas

quantile