Question

我正在尝试对我的模型进行故障排除，该模型会产生荒谬的预测。在此过程中，我添加了另一个密集层，并且火车计时与没有它时保持相同。因此，我添加了4 x 512个神经元密集层，时间是相同的。

我使用TPU v3，但我仍然希望-计算4个额外的完全连接的网络，每个网络具有512个神经元，应该花费一些时间。

任何人都可以帮忙解释一下，或者至少给出一个调查思路

编辑： LSTM的可训练参数数量+ 1密度= 4百万； LSTM + 4密度= 500万

model = tf.keras.Sequential()
model.add(layers.LSTM(neurons, input_shape=(window_size, inputs_n), return_sequences=True)) 
model.add(layers.Dropout(0.2))
model.add(layers.LSTM(neurons))
model.add(layers.Dense(neurons, activation='relu'))
model.add(layers.Dropout(0.2))
model.add(layers.Dense(neurons, activation='relu'))
model.add(layers.Dropout(0.2))
model.add(layers.Dense(neurons, activation='relu'))
model.add(layers.Dropout(0.2))
model.add(layers.Dense(neurons, activation='relu'))
model.add(layers.Dropout(0.2))
model.add(layers.Dense(outputs_n, activation=activation))

TPU_ADDRESS1 = 'grpc://10.240.2.2:8470'

opt = tf.train.AdamOptimizer(learning_rate)

model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['categorical_accuracy'])
print(model.metrics_names)

tpu_model = tf.contrib.tpu.keras_to_tpu_model(model, 
        strategy=tf.contrib.tpu.TPUDistributionStrategy(
            tf.contrib.cluster_resolver.TPUClusterResolver(tpu = [TPU_ADDRESS1])))

Layer (type)                 Output Shape              Param #   
=================================================================
lstm_input (InputLayer)      (None, 1024, 7)           0         
_________________________________________________________________
lstm (LSTM)                  (None, 1024, 512)         1064960   
_________________________________________________________________
dropout (Dropout)            (None, 1024, 512)         0         
_________________________________________________________________
lstm_1 (LSTM)                (None, 512)               2099200   
_________________________________________________________________
dense (Dense)                (None, 512)               262656    
_________________________________________________________________
dropout_1 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 512)               262656    
_________________________________________________________________
dropout_2 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 512)               262656    
_________________________________________________________________
dropout_3 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_3 (Dense)              (None, 512)               262656    
_________________________________________________________________
dropout_4 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_4 (Dense)              (None, 4)                 2052      
=================================================================
Total params: 4,216,836
Trainable params: 4,216,836
Non-trainable params: 0
_________________________________________________________________

具有额外4个密集层的模型的训练时间与没有

0 个答案: