Question

我有一些数据需要分组到分档中。而不是将箱子表示为0,1,2,3 ......等。我希望它输出每个bin的平均值或中位数。有没有办法做到这一点？

Answer 1

您可以通过仅计算一次bin_idx的统计信息来加速shx2的代码。

import numpy as np
x = np.tile(np.array([0.2, 9., 6.4, 3.0, 1.6]), 100000)
bins = np.array([0.0, 1.0, 2.5, 10.0])

def binstats(x, bins):
    inds = np.digitize(x, bins)
    statistics = []
    binnumber = []
    seen = set()
    for bin_idx in inds:
        if bin_idx not in seen:
            bin_arr = x[inds==bin_idx]
            statistics.append([np.mean(bin_arr), np.median(bin_arr)])
            binnumber.append(bin_idx)
            seen.add(bin_idx)
    return statistics, binnumber

statistics, binnumber = binstats(x, bins)
for (mean, median), bin_idx in zip(statistics, binnumber):
    print('{b}: {mean:.2f} {median:.2f}'.format(b=bin_idx, mean=mean, median=median))

产量

1: 0.20 0.20
3: 6.13 6.40
2: 1.60 1.60

顺便说一句，如果你有scipy，你也可以使用scipy.stats.binned_statistic，但性能不是更好：

import scipy.stats as stats

# This is a hack to return two statistics with one call to binned_statistic. It reduces the precision of the statistics to `float32`.
def onecall():
    statistics, bin_edges, binnumber = stats.binned_statistic(
        x, values=x, bins=bins,
        statistic=lambda grp: (np.array([grp.mean(), np.median(grp)])
                               .astype('float32').view('float64')))
    return statistics.view('float32').reshape(-1, 2)

def twocalls():
    means, bin_edges, binnumber = stats.binned_statistic(
        x, values=x, statistic='mean', bins=bins)
    medians, bin_edges, binnumber = stats.binned_statistic(
        x, values=x, statistic='median', bins=bins)
    return means, medians

In [284]: %timeit binstats(x, bins)
10 loops, best of 3: 85.6 ms per loop

In [285]: %timeit onecall()
10 loops, best of 3: 86.6 ms per loop

In [286]: %timeit twocalls()
10 loops, best of 3: 150 ms per loop

Answer 2

为了进行比较，以下是您使用pandas和groupby（类似于pd.cut）在np.digitize中撰写此类内容的方式：

>>> x = np.random.uniform(0, 10, 5*10**5)
>>> bins = np.array([0, 1, 2.5, 10])
>>> s = pd.Series(x)
>>> s.groupby(pd.cut(s, bins)).agg(["median", "mean"])
             median      mean
(0, 1]     0.500684  0.500641
(1, 2.5]   1.751121  1.751630
(2.5, 10]  6.243822  6.248801

[3 rows x 2 columns]

性能似乎与unutbu的numpy解决方案相当（稍微调整一下以接受args）：

>> %timeit binstats(x, bins)
10 loops, best of 3: 126 ms per loop
>>> %timeit onecall(x, bins)
10 loops, best of 3: 74.8 ms per loop
>>> %timeit twocalls(x, bins)
10 loops, best of 3: 109 ms per loop
>>> %timeit s.groupby(pd.cut(s, bins)).agg(["median", "mean"])
10 loops, best of 3: 72.5 ms per loop

如果你愿意牺牲一点优雅，你可以节省更多时间：

>>> %timeit s.groupby(np.digitize(x, bins)).agg(["median", "mean"])
10 loops, best of 3: 65.2 ms per loop

但是我没有使用pandas来表现性能，我使用它是因为它使许多常见的数据操作更加方便。

Answer 3

我没有无环路的解决方案（就像大多数numpy问题所要求的那样），但假设你没有太多的垃圾箱，并且阵列不是很大，这应该相当快：< / p>

x = np.array([0.2, 9., 6.4, 3.0, 1.6])
bins = np.array([0.0, 1.0, 2.5, 10.0])
inds = np.digitize(x, bins)
inds
=> array([1, 3, 3, 3, 2])

for bin_idx in inds:
    bin_arr = x[inds==bin_idx]
    print bin_idx, np.mean(bin_arr), np.median(bin_arr)
=>
1 0.2 0.2
3 6.13333333333 6.4
3 6.13333333333 6.4
3 6.13333333333 6.4
2 1.6 1.6

创建数组：

bin_means = np.array([ x[inds==bin_idx].mean() for bin_idx in inds ])

Answer 4

更简单，更通用的unutbu代码版本

import numpy as np
x = np.tile(np.array([0.2, 9., 6.4, 3.0, 1.6]), 100000)
bins = np.array([0.0, 1.0, 2.5, 10.0])

def binstats(x, bins, funcs):
    inds = np.digitize(x, bins)
    inds2 = np.unique(inds)
    statistics = []
    binnumber = []
    for bin_idx in inds2:
        bin_arr = x[inds==bin_idx]
        statistics.append([f(bin_arr) for f in funcs])
    return statistics, inds2

statistics, binnumber = binstats(x, bins, [np.mean, np.median])
print(statistics)
for (mean, median), bin_idx in zip(statistics, binnumber):
    print('{b}: {mean:.2f} {median:.2f}'.format(b=bin_idx, mean=mean, median=median))

这可能更合适，因为它允许您在统计信息中使用任意数量的函数。提前创建集合可能会更快。

numpy的数字化函数输出均值或中位数？

4 个答案: