Question

我是pandas / seaborn / etc的新手，并尝试使用类似https://seaborn.pydata.org/generated/seaborn.stripplot.html的示例，以不同的样式（使用seaborn）绘制我的数据子集：

>>> ax = sns.stripplot(x="day", y="total_bill", hue="smoker",
...                    data=tips, jitter=True,
...                    palette="Set2", dodge=True)

我的目标是仅绘制每个x / hue维内的离群值，即对于所示示例，我将对8个使用8个不同的百分位数截止显示不同的点列。

我有一个像这样的数据框：

        Cat  RPS   latency_ns
0       X    100     909423.0
1       X    100   14747385.0
2       X    1000  14425058.0
3       Y    100    7107907.0
4       Y    1000  21466101.0
...     ...  ...   ...

我想过滤此数据，仅留下较高的99.9％离群值。

我发现我可以做到：

df.groupby([dim1_label, dim2_label]).quantile(0.999)

要获得类似的东西

                                         latency_ns
Cat                          RPS                   
X                            10 RPS    6.463337e+07
                             100 RPS   4.400980e+07
                             1000 RPS  6.075070e+07
Y                            100 RPS   3.958944e+07
Z                            10 RPS    5.621427e+07
                             100 RPS   4.436208e+07
                             1000 RPS  6.658783e+07

但是我不确定通过合并/过滤操作从哪里去。

Answer 1

这是我创建的一个指导您的小例子。希望对您有所帮助。

代码

import numpy as np
import pandas as pd
import seaborn as sns

#create a sample data frame
n = 1000
prng = np.random.RandomState(123)

x = prng.uniform(low=1, high=5, size=(n,)).astype('int')
#print(x[:10])
#[3 2 1 3 3 2 4 3 2 2]

y = prng.normal(size=(n,))
#print(y[:10])
#[ 1.32327371 -0.00315484 -0.43065984 -0.14641577  1.16017595 -0.64151234
#-0.3002324  -0.63226078 -0.20431653  0.2136956 ]

z = prng.binomial(n=1,p=2/3,size=(n,))
#print(z[:10])
#[1 0 1 1 1 1 0 1 1 1]

#analagously to the smoking example, my df x maps day,
#y maps to total bill, and z maps to is smoker (or not)
df = pd.DataFrame(data={'x':x,'y':y,'z':z})

#df.head()

df_filtered = pd.DataFrame()

#df.groupby.quantile([0.9]) returns a scalar, unless you want to plot only a single point, use this
#if you want to plot values that are within the lower and upper bounds, then some
#conditional filtering is required, see the conditional filtering I wrote below
for i,j in df.groupby([x, z]):
    b = j.quantile([0,0.9]) #use [0.999,1] in your case
    lb = b['y'].iloc[0]
    ub = b['y'].iloc[1]
    df_temp = j[(j['y']>=lb)&(j['y']<=ub)]
    df_filtered = pd.concat([df_filtered,df_temp])


#print(df_filtered.count())
#x    897
#y    897
#z    897
#dtype: int64

输出

import matplotlib.pyplot as plt

ax = sns.stripplot(x='x', y='y', hue='z',
data=df_filtered, jitter=True,
palette="Set2", dodge=True)

plt.show()

过滤熊猫中每个类别数据中的异常值

1 个答案: