Question

我想对我的数据集重新采样。这包括带有3类标签的分类转换数据。每个类别的样本数量为：

A级人数：6945
B级人数：650
C级人数：9066
TOTAl样本：16661

不带标签的数据形状为（16661，1000，256）。这意味着（1000,256）的16661个样本。我想要对数据进行上采样，直至达到多数类（即A类->（6945）

）中的样本数量

但是，在致电时：

from imblearn.over_sampling import SMOTE
print(categorical_vector.shape)
sm = SMOTE(random_state=2)
X_train_res, y_labels_res = sm.fit_sample(categorical_vector, labels.ravel())

它一直在说ValueError：找到的数组为暗3。估计器应为<= 2。

我怎样才能使数据扁平化，以使估算器可以拟合数据并且有意义呢？此外，获取X_train_res后如何展开（使用3D尺寸）？

Answer 1

from imblearn.over_sampling 
import RandomOverSampler 
import numpy as np 
oversample = RandomOverSampler(sampling_strategy='minority')

X可以是时间步进的3D数据，例如X [sample，time，feature]，而y是每个样本的二进制值。例如：（1,1），（2,1），（3,1）-> 1

X = np.array([[[1,1],[2,1],[3,1]],
             [[2,1],[3,1],[4,1]],
             [[5,1],[6,1],[7,1]],
             [[8,1],[9,1],[10,1]],
             [[11,1],[12,1],[13,1]]
             ])

y = np.array([1,0,1,1,0])

无法使用3D X值训练OVERSAMPLER，因为如果使用2D，您将获得2D数据。

Xo,yo = oversample.fit_resample(X[:,:,0], y)
Xo:
[[ 1  2  3]
 [ 2  3  4]
 [ 5  6  7]
 [ 8  9 10]
 [11 12 13]
 [ 2  3  4]]

yo:
[1 0 1 1 0 0]

但是如果您使用2D数据（样本，时间，0）来拟合模型，它将返回索引，并且足以创建3D过采样的数据

oversample.fit_resample(X[:,:,0], y)
Xo = X[oversample.sample_indices_]
yo = y[oversample.sample_indices_]

Xo:
[[[ 1  1][ 2  1][ 3  1]]
 [[ 2  1][ 3  1][ 4  1]]
 [[ 5  1][ 6  1][ 7  1]]
 [[ 8  1][ 9  1][10  1]]
 [[11  1][12  1][13  1]]
 [[ 2  1][ 3  1][ 4  1]]]
yo:
[1 0 1 1 0 0]

Answer 2

我正在考虑一个虚拟的3d数组，并自己假设一个2d数组大小，

arr = np.random.rand(160, 10, 25)
orig_shape = arr.shape
print(orig_shape)

输出：(160, 10, 25)

arr = np.reshape(arr, (arr.shape[0], arr.shape[1]))
print(arr.shape)

输出：(4000, 10)

arr = np.reshape(arr, orig_shape))
print(arr.shape)

输出：(160, 10, 25)

Answer 3

我将为2维数组创建每个点，然后将其重塑为3维数组。我已经提供了脚本。如有任何混淆，请发表评论；请回复。

x_train, y_train = zip(*train_dataset)
x_test, y_test = zip(*test_dataset)

dim_1 = np.array(x_train).shape[0]
dim_2 = np.array(x_train).shape[1]
dim_3 = np.array(x_train).shape[2]

new_dim = dim_1 * dim_2

new_x_train = np.array(x_train).reshape(new_dim, dim_3)


new_y_train = []
for i in range(len(y_train)):
    # print(y_train[i])
    new_y_train.extend([y_train[i]]*dim_2)

new_y_train = np.array(new_y_train)

# transform the dataset
oversample = SMOTE()
X_Train, Y_Train = oversample.fit_sample(new_x_train, new_y_train)
# summarize the new class distribution
counter = Counter(Y_Train)
print('The number of samples in TRAIN: ', counter)



x_train_SMOTE = X_Train.reshape(int(X_Train.shape[0]/dim_2), dim_2, dim_3)

y_train_SMOTE = []
for i in range(int(X_Train.shape[0]/dim_2)):
    # print(i)
    value_list = list(Y_Train.reshape(int(X_Train.shape[0]/dim_2), dim_2)[i])
    # print(list(set(value_list)))
    y_train_SMOTE.extend(list(set(value_list)))
    ## Check: if there is any different value in a list 
    if len(set(value_list)) != 1:
        print('\n\n********* STOP: THERE IS SOMETHING WRONG IN TRAIN ******\n\n')
    


dim_1 = np.array(x_test).shape[0]
dim_2 = np.array(x_test).shape[1]
dim_3 = np.array(x_test).shape[2]

new_dim = dim_1 * dim_2

new_x_test = np.array(x_test).reshape(new_dim, dim_3)


new_y_test = []
for i in range(len(y_test)):
    # print(y_train[i])
    new_y_test.extend([y_test[i]]*dim_2)

new_y_test = np.array(new_y_test)

# transform the dataset
oversample = SMOTE()
X_Test, Y_Test = oversample.fit_sample(new_x_test, new_y_test)
# summarize the new class distribution
counter = Counter(Y_Test)
print('The number of samples in TEST: ', counter)



x_test_SMOTE = X_Test.reshape(int(X_Test.shape[0]/dim_2), dim_2, dim_3)

y_test_SMOTE = []
for i in range(int(X_Test.shape[0]/dim_2)):
    # print(i)
    value_list = list(Y_Test.reshape(int(X_Test.shape[0]/dim_2), dim_2)[i])
    # print(list(set(value_list)))
    y_test_SMOTE.extend(list(set(value_list)))
    ## Check: if there is any different value in a list 
    if len(set(value_list)) != 1:
        print('\n\n********* STOP: THERE IS SOMETHING WRONG IN TEST ******\n\n')

重新采样数据-使用来自imblearn的SMOTE和3D numpy数组

3 个答案: