Question

我正在使用数据生成器来填充fit_generator。我的生成器将元组(x_val, y_val, val_sample_weights)作为输出，因此显示了样本权重。就像：

import numpy as np
import keras
import librosa
from time import time
import random
from config import *

class DataGenerator(keras.utils.Sequence):

    'Generates data for Keras'

    def __init__(self, dataframe, batch_size=None, dim=None, labels_dim=None,
                 n_classes=None, shuffle=True, samples=None, duration=None, sample_weights=None):
        'Initialization'
        self.dim = dim
        self.batch_size = batch_size

        self.dataframe = dataframe
        self.dataframe = self.dataframe.sample(n=len(self.dataframe))
        self.samples = samples
        self.on_epoch_end()
        self.shuffle = shuffle
        self.sample_weights = sample_weights

    def __len__(self):
        'Denotes the number of batches per epoch'
        return int(np.floor(len(self.dataframe) / self.batch_size))

    def __getitem__(self, index):
        'Generate one batch of data'
        random_pd = self.dataframe.iloc[self.batch_size*index : (index+1)*self.batch_size]
        # Generate data
        X, y = self.__data_generation(random_pd)
        return X, y
    def __data_generation(self, random_pd):
        'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels)
        # Initialization
        X = np.empty((self.batch_size, 1, self.samples))
        y = np.empty((self.batch_size, self.n_classes))
        i = 0
        while i < self.batch_size:
            for index, row in random_pd.iterrows():
                 # generate 
                y[i,] = label
                X[i,] = ...
                i += 1
        return X, y, self.sample_weights

因此它将返回X, y, self.sample_weights。

问题在于，我们会得到一个StopIteration: too many values to unpack，它会像预期的那样为2，但我给出的是3个值-实际上是我在做的事情。

Traceback (most recent call last):
File "train.py", line 438, in <module>
train()
File "train.py", line 422, in train
callbacks=callbacks
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 1315, in fit_generator
initial_epoch=initial_epoch)
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 2250, in fit_generator
max_queue_size=max_queue_size)
File "/usr/local/lib/python2.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 2383, in evaluate_generator
generator_output = next(output_generator)
File "/usr/local/lib/python2.7/dist-packages/keras/utils/data_utils.py", line 584, in get
six.raise_from(StopIteration(e), e)
File "/usr/local/lib/python2.7/dist-packages/six.py", line 737, in raise_from
raise value
StopIteration: too many values to unpack

我像往常一样通过training_generator呼叫fit_generator

history = model.fit_generator(generator=training_generator,
                                class_weight=class_weights,
                                verbose=1,
                                use_multiprocessing=True,
                                workers=24, 
                                steps_per_epoch=training_steps_per_epoch,
                                epochs=epochs,
                                validation_data=validation_generator,
                                validation_steps = validation_steps_per_epoch,
                                callbacks=callbacks
                                )

之所以这样做，是因为在使用fit_generator时无法通过sample_weight，因为方法签名仅支持class_weight-请参见https://github.com/keras-team/keras/issues/11800

Answer 1

哦。...看看你的退货单...

__data_generation返回3个值。但是您正在做X, y = self.__data_generation(random_pd)

试图将三个值(X,y,weights)分成两个变量(X,y)。

请注意，__get_item__也没有返回权重。

Keras实现了一个数据生成器，它输出sample_weight

1 个答案: