Question

我正面临有关卷积神经网络中用于文本数据分析的超参数优化的问题。首先，我将解释到目前为止的过程：在各种出色的Blog-Post的帮助下，我能够构建一个适用于我的项目的CNN。在我的项目中，我试图借助FOMC会议声明来预测VIX和S＆P 500。因此，基本上我一方面是文本数据，另一方面是财务数据（回报）。经过预处理和应用Google的Word2Vec预训练的Word嵌入程序后，我构建了以下卷积网络：

def ConvNet(embeddings, max_sequence_length, num_words, embedding_dim, trainable=False, extra_conv=True,
            lr=0.001, dropout=0.5):
    embedding_layer = Embedding(num_words,
                                embedding_dim,
                                weights=[embeddings],
                                input_length=max_sequence_length,
                                trainable=trainable)

    sequence_input = Input(shape=(max_sequence_length,), dtype='int32')
    embedded_sequences = embedding_layer(sequence_input)

    convs = []
    filter_sizes = [3, 4, 5]

    for filter_size in filter_sizes:
        l_conv = Conv1D(filters=128, kernel_size=filter_size, activation='relu')(embedded_sequences)
        l_pool = MaxPooling1D(pool_size=3)(l_conv)
        convs.append(l_pool)

    l_merge = concatenate([convs[0], convs[1], convs[2]], axis=1)

    # add a 1D convnet with global maxpooling, instead of Yoon Kim model
    conv = Conv1D(filters=128, kernel_size=3, activation='relu')(embedded_sequences)
    pool = MaxPooling1D(pool_size=3)(conv)

    if extra_conv == True:
        x = Dropout(dropout)(l_merge)
    else:
        # Original Yoon Kim model
        x = Dropout(dropout)(pool)
    x = Flatten()(x)
    x = Dense(128, activation='relu')(x)
    preds = Dense(1, activation='linear')(x)

    model = Model(sequence_input, preds)
    sgd = SGD(learning_rate = lr, momentum= 0.8)
    model.compile(loss='mean_squared_error',
                  optimizer= sgd,
                  metrics=['mean_squared_error'])
    model.summary()
    return model

model = ConvNet(train_embedding_weights, MAX_SEQUENCE_LENGTH, len(train_word_index)+1, EMBEDDING_DIM, False)

#define callbacks
early_stopping = EarlyStopping(monitor='val_loss', min_delta=0.01, patience=4, verbose=1)
callbacks_list = [early_stopping]

hist = model.fit(x_train, y_tr, epochs=5, batch_size=33, validation_split=0.2, shuffle=True, callbacks=callbacks_list)

我的模型架构如下：

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 1086)         0                                            
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 1086, 300)    532500      input_1[0][0]                    
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 1084, 128)    115328      embedding_1[0][0]                
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 1083, 128)    153728      embedding_1[0][0]                
__________________________________________________________________________________________________

conv1d_3 (Conv1D)               (None, 1082, 128)    192128      embedding_1[0][0]                


__________________________________________________________________________________________________
max_pooling1d_1 (MaxPooling1D)  (None, 361, 128)     0           conv1d_1[0][0]                   
__________________________________________________________________________________________________
max_pooling1d_2 (MaxPooling1D)  (None, 361, 128)     0           conv1d_2[0][0]                   
__________________________________________________________________________________________________
max_pooling1d_3 (MaxPooling1D)  (None, 360, 128)     0           conv1d_3[0][0]                   
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 1082, 128)    0           max_pooling1d_1[0][0]            
                                                                 max_pooling1d_2[0][0]            
                                                                 max_pooling1d_3[0][0]            
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 1082, 128)    0           concatenate_1[0][0]              
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 138496)       0           dropout_2[0][0]                  
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 128)          17727616    flatten_1[0][0]                  
__________________________________________________________________________________________________
dense_4 (Dense)                 (None, 1)            129         dense_3[0][0]                    
==================================================================================================
Total params: 18,721,429
Trainable params: 18,188,929
Non-trainable params: 532,500

模型架构：

因此，现在我面临着下一个大问题，而我真的没有办法解决以下问题：超参数的优化

所以我的具体问题是，如何执行超参数优化？

我的搜索代码是：

from hyperopt import fmin, hp, tpe, space_eval, Trials

def train_and_score(args):
    # Train the model the fixed params plus the optimization args.
    # Note that this method should return the final History object.
    test = ConvNet(embeddings=train_embedding_weights, max_sequence_length= MAX_SEQUENCE_LENGTH,
                    num_words=len(train_word_index)+1, embedding_dim= EMBEDDING_DIM,
                   trainable=False, extra_conv=True,
                   lr=args['lr'], dropout=args['dropout'])
    # Unpack and return the last validation loss from the history.
    return test['val_loss'][-1]

# Define the space to optimize over.
space = {
    'lr': hp.loguniform('lr', np.log(0.01), np.log(0.1)),
    'dropout': hp.uniform('dropout', 0, 0.5)
}

# Minimize the training score over the space.
trials = Trials()
best = fmin(fn=train_and_score,
             space=space,
             trials=trials,
             algo=tpe.suggest,
             max_evals=100)

# Print details about the best results and hyperparameters.
print(best)
print(space_eval(space, best))

特定的错误消息是：

__________________________________________________________________________________________________
  0%|          | 0/100 [00:00<?, ?trial/s, best loss=?]
job exception: 'Model' object is not subscriptable

Traceback (most recent call last):
  File "/Users/lukaskoston/Desktop/MasterarbeitFOMCAnalysis/07_Regression/CNN regression neu.py", line 262, in <module>
    max_evals=100)
  File "/Users/lukaskoston/.local/lib/python3.7/site-packages/hyperopt/fmin.py", line 482, in fmin
    show_progressbar=show_progressbar,
  File "/Users/lukaskoston/.local/lib/python3.7/site-packages/hyperopt/base.py", line 686, in fmin
    show_progressbar=show_progressbar,
  File "/Users/lukaskoston/.local/lib/python3.7/site-packages/hyperopt/fmin.py", line 509, in fmin
    rval.exhaust()
  File "/Users/lukaskoston/.local/lib/python3.7/site-packages/hyperopt/fmin.py", line 330, in exhaust
    self.run(self.max_evals - n_done, block_until_done=self.asynchronous)
  File "/Users/lukaskoston/.local/lib/python3.7/site-packages/hyperopt/fmin.py", line 286, in run
    self.serial_evaluate()
  File "/Users/lukaskoston/.local/lib/python3.7/site-packages/hyperopt/fmin.py", line 165, in serial_evaluate
    result = self.domain.evaluate(spec, ctrl)
  File "/Users/lukaskoston/.local/lib/python3.7/site-packages/hyperopt/base.py", line 894, in evaluate
    rval = self.fn(pyll_rval)
  File "/Users/lukaskoston/Desktop/MasterarbeitFOMCAnalysis/07_Regression/CNN regression neu.py", line 248, in train_and_score
    return hist['val_loss'][-1]
TypeError: 'Model' object is not subscriptable

预先感谢，卢卡斯

Answer 1

您得到的错误是因为您试图直接子集无法直接子集化的模型（例如列表或字典）。

您的ConvNet函数定义并编译了一个模型，但是它没有训练或评估它。您需要运行model.fit()进行培训，并像在发布hist = model.fit(...)的第一个脚本中那样存储培训历史记录的输出。然后，您可以将train_and_score的return语句更改为return hist.history['val_loss'][-1]。

我将首先获取您的培训代码

early_stopping = EarlyStopping(monitor='val_loss', min_delta=0.01, patience=4, verbose=1)
callbacks_list = [early_stopping]

hist = model.fit(x_train, y_tr, epochs=5, batch_size=33, validation_split=0.2, shuffle=True, callbacks=callbacks_list)

，并将其添加到模型定义之后的train_and_score函数中。然后更改return语句。

卷积神经网络中超参数的优化

1 个答案: