Question

我正在从一列中找到离群值并将其存储在列表中。现在我想删除所有的值列在我的列表中。如何实现呢？

这是我发现异常值的功能

outlier=[]

def detect_outliers(data):

    threshold=3
    m = np.mean(data)
    st = np.std(data)

    for i in data:
        #calculating z-score value
        z_score=(i-m)/st
        #if the z_score value is greater than threshold value than its a outlier
        if np.abs(z_score)>threshold:
            outlier.append(i)
    return outlier

This is my column in data frame

df_train_11.AMT_INCOME_TOTAL

Answer 1

import numpy as np, pandas as pd

df = pd.DataFrame(np.random.rand(10,5))

outlier_list=[]
def detect_outliers(data):
    threshold=0.5
    for i in data:
    #calculating z-score value
        z_score=(df.loc[:,i]- np.mean(df.loc[:,i])) /np.std(df.loc[:,i])
        outliers = np.abs(z_score)>threshold

        outlier_list.append(df.index[outliers].tolist())
    return outlier_list

outlier_list = detect_outliers(df)

[[1, 2, 4, 5, 6, 7, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 4, 8],
 [0, 1, 3, 4, 6, 8],
 [0, 1, 3, 5, 6, 8, 9]]

这样，您将获得每列的异常值。 outlier_list[0]为您提供[1, 2, 4, 5, 6, 7, 9]，这意味着第1,2等行与第0列的异常值。

编辑

更短的答案


df = pd.DataFrame(np.random.randn(10, 3), columns=list('ABC'))
df[((df.B - df.B.mean()) / df.B.std()).abs() < 3]

这将过滤只有一个列（例如'B'）在三个标准差以内的DataFrame。

如何从列中删除在熊猫列表中具有匹配值的行

1 个答案: