Question

请帮我找一个干净的方法来创建一个现有的新阵列。如果任何类的示例数小于类中的最大示例数，则应对其进行过采样。样本应取自原始数组（没有任何区别，无论是随机还是顺序）

让我们说，初始数组就是这样：

[  2,  29,  30,   1]
[  5,  50,  46,   0]
[  1,   7,  89,   1]
[  0,  10,  92,   9]
[  4,  11,   8,   1]
[  3,  92,   1,   0]

最后一列包含类：

classes = [ 0,  1,  9]

类的分布如下：

distrib = [2, 3, 1]

我需要的是创建一个具有相同数量的所有类的样本的新数组，从原始数组中随机取出，例如

[  5,  50,  46,   0]
[  3,  92,   1,   0]
[  5,  50,  46,   0] # one example added
[  2,  29,  30,   1]
[  1,   7,  89,   1]
[  4,  11,   8,   1]
[  0,  10,  92,   9]
[  0,  10,  92,   9] # two examples
[  0,  10,  92,   9] # added

Answer 1

以下代码可以满足您的需求：

a = np.array([[  2,  29,  30,   1],
              [  5,  50,  46,   0],
              [  1,   7,  89,   1],
              [  0,  10,  92,   9],
              [  4,  11,   8,   1],
              [  3,  92,   1,   0]])

unq, unq_idx = np.unique(a[:, -1], return_inverse=True)
unq_cnt = np.bincount(unq_idx)
cnt = np.max(unq_cnt)
out = np.empty((cnt*len(unq),) + a.shape[1:], a.dtype)
for j in xrange(len(unq)):
    indices = np.random.choice(np.where(unq_idx==j)[0], cnt)
    out[j*cnt:(j+1)*cnt] = a[indices]

>>> out
array([[ 5, 50, 46,  0],
       [ 5, 50, 46,  0],
       [ 5, 50, 46,  0],
       [ 1,  7, 89,  1],
       [ 4, 11,  8,  1],
       [ 2, 29, 30,  1],
       [ 0, 10, 92,  9],
       [ 0, 10, 92,  9],
       [ 0, 10, 92,  9]])

当numpy 1.9发布时，或者你从开发分支编译时，前两行可以压缩成：

unq, unq_idx, unq_cnt = np.unique(a[:, -1], return_inverse=True,
                                  return_counts=True)

请注意，np.random.choice的工作方式，无法保证原始数组的所有行都将出现在输出中，如上例所示。如果需要，您可以执行以下操作：

unq, unq_idx = np.unique(a[:, -1], return_inverse=True)
unq_cnt = np.bincount(unq_idx)
cnt = np.max(unq_cnt)
out = np.empty((cnt*len(unq) - len(a),) + a.shape[1:], a.dtype)
slices = np.concatenate(([0], np.cumsum(cnt - unq_cnt)))
for j in xrange(len(unq)):
    indices = np.random.choice(np.where(unq_idx==j)[0], cnt - unq_cnt[j])
    out[slices[j]:slices[j+1]] = a[indices]
out = np.vstack((a, out))

>>> out
array([[ 2, 29, 30,  1],
       [ 5, 50, 46,  0],
       [ 1,  7, 89,  1],
       [ 0, 10, 92,  9],
       [ 4, 11,  8,  1],
       [ 3, 92,  1,  0],
       [ 5, 50, 46,  0],
       [ 0, 10, 92,  9],
       [ 0, 10, 92,  9]])

Answer 2

这给出了每个类的概率相等的随机分布：

distrib = np.bincount(a[:,-1])
prob = 1/distrib[a[:, -1]].astype(float)
prob /= prob.sum()

In [38]: a[np.random.choice(np.arange(len(a)), size=np.count_nonzero(distrib)*distrib.max(), p=prob)]
Out[38]: 
array([[ 5, 50, 46,  0],
       [ 4, 11,  8,  1],
       [ 0, 10, 92,  9],
       [ 0, 10, 92,  9],
       [ 2, 29, 30,  1],
       [ 0, 10, 92,  9],
       [ 3, 92,  1,  0],
       [ 1,  7, 89,  1],
       [ 1,  7, 89,  1]])

每个班级的概率相等，不保证发生率相等。

Answer 3

您可以使用imbalanced-learn软件包：

import numpy as np
from imblearn.over_sampling import RandomOverSampler

data = np.array([
    [  2,  29,  30,   1],
    [  5,  50,  46,   0],
    [  1,   7,  89,   1],
    [  0,  10,  92,   9],
    [  4,  11,   8,   1],
    [  3,  92,   1,   0]
])

ros = RandomOverSampler()

# fit_resample expects two arguments: a matrix of sample data and a vector of
# sample labels. In this case, the sample data is in the first three columns of 
# our array and the labels are in the last column
X_resampled, y_resampled = ros.fit_resample(data[:, :-1], data[:, -1])

# fit_resample returns a matrix of resampled data and a vector with the 
# corresponding labels. Combine them into a single matrix
resampled = np.column_stack((X_resampled, y_resampled))

print(resampled)

输出：

[[ 2 29 30  1]
 [ 5 50 46  0]
 [ 1  7 89  1]
 [ 0 10 92  9]
 [ 4 11  8  1]
 [ 3 92  1  0]
 [ 3 92  1  0]
 [ 0 10 92  9]
 [ 0 10 92  9]]

RandomOverSampler提供different sampling strategies，但默认设置是对除多数类之外的所有类重新采样。

平衡numpy数组与过采样

3 个答案: