Question

我在熊猫聚合中使用lambda函数来计算加权平均值。我的问题是，如果值之一是nan，则整个结果就是该组的nan。如何避免这种情况？

df = pd.DataFrame(np.random.randn(5, 3), index=['a', 'c', 'e', 'f', 'h'],columns = ['one', 'two', 'three'])
df['four'] = 'bar'
df['five'] = df['one'] > 0
df = df.reindex(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h'])
df.loc['b','four'] ='foo'
df.loc['c','four'] ='foo'


        one       two     three four   five found
a  1.046540 -0.304646 -0.982008  bar   True   NaN
b       NaN       NaN       NaN  foo    NaN   foo
c -1.086525  1.086501  0.403910  foo  False   NaN
d       NaN       NaN       NaN  NaN    NaN   NaN
e  0.569420  0.105422  0.192559  bar   True   NaN
f  0.384400 -0.558321  0.324624  bar   True   NaN
g       NaN       NaN       NaN  NaN    NaN   NaN
h  0.656231 -2.185062  0.180535  bar   True   NaN


df.groupby('four').agg(sum=('two','sum'), weighted_avg=('one', lambda x: np.average(x, weights=df.loc[x.index, 'two'])))

           sum  weighted_avg
four                        
bar  -2.942608      0.648173
foo   1.086501           NaN

所需结果：

           sum  weighted_avg
four                        
bar  -2.942608      0.648173
foo   1.086501     -1.086525

与this question不同，这不是该列的实际值不出现的问题，这是nanmean没有加权选项的问题。

另一个数字示例：

     x      y
0   NaN   18.0
1   NaN   21.0
2   NaN   38.0
3  56.0  150.0
4  65.0  154.0

在这里，我们将只返回最后两行的加权平均值，而忽略其他包含nan的行。

Answer 1

对我来说，我正在实施this解决方案：

def f(x):
    indices = ~np.isnan(x)
    return np.average(x[indices], weights=df.loc[x.index[indices], 'two'])

df = df.groupby('four').agg(sum=('two','sum'), weighted_avg=('one', f))

print (df)
           sum  weighted_avg
four                        
bar  -2.942607      0.648173
foo   1.086501     -1.086525

编辑：

def f(x):
    indices = ~np.isnan(x)
    if indices.all():
        return np.average(x[indices], weights=df.loc[x.index[indices], 'two'])
    else:
        return np.nan

Answer 2

这似乎更可靠：

def f(x):
    indices = (~np.isnan(x)) & (~np.isnan(df[weight_column]))[x.index]
    try:
        return np.average(x[indices], weights=df.loc[x.index[indices], weight_column])
    except ZeroDivisionError:
        return np.nan

df = df.groupby('four').agg(sum=('two','sum'), weighted_avg=('one', f))

权重的nanmean来计算熊猫的加权平均值。agg

2 个答案: