我实现了LSTMLM代码。 以下代码有效，但是训练时间太长 因为我们不能将Masking与CuDNN一起使用。（其他网站已经报道了这种现象。） 因此，我尝试将“制作”层替换为“嵌入”层。
model_input = Input(tensor=Xtrain)
x_mask = Masking(
input_shape=(None,timestep,vocab_size),
mask_value=0)(model_input)
x_lstm = LSTM(num_hidden_units,return_sequences=True)(x_mask)
model_output = TimeDistributed(Dense(vocab_size,activation="softmax"))(x_lstm)
model = Model(inputs=model_input, outputs=model_output)
model.compile(loss='sparse_categorical_crossentropy', optimizer=SGD(),metrics=['sparse_categorical_accuracy'],target_tensors=[Ytrain])
model_input_valid = Input(tensor=Xvalid)
x_mask_valid=Masking(
input_shape=(None,timestep,vocab_size),
mask_value=0)(model_input_valid)
x_lstm_valid = LSTM(num_hidden_units,return_sequences=True)(model_input_valid)
model_output_valid = TimeDistributed(Dense(vocab_size,activation="softmax"))(x_lstm_valid)
model_valid = Model(inputs=model_input_valid, outputs=model_output_valid)
我将这段代码更改为使用嵌入，如下所示： 但是失败了。
model_input = Input(tensor=Xtrain)
embedding1 = Embedding(
input_dim=vocab_size,
output_dim=100,
mask_zero=True,
name='embedding1')(model_input)
x_lstm = LSTM(num_hidden_units,return_sequences=True)(embedding1)
model_output = TimeDistributed(Dense(vocab_size,activation="softmax"))(x_lstm)
model = Model(inputs=model_input, outputs=model_output)
model.compile(loss='sparse_categorical_crossentropy', optimizer=SGD(),metrics=['sparse_categorical_accuracy'],target_tensors=[Ytrain])
model_input_valid = Input(tensor=Xvalid)
embedding2 = Embedding(
input_dim=vocab_size,
output_dim=100,
mask_zero=True,
name='embedding1')(model_input_valid)
x_lstm_valid = LSTM(num_hidden_units,return_sequences=True)(embedding2)
model_output_valid = TimeDistributed(Dense(vocab_size,activation="softmax"))(x_lstm_valid)
model_valid = Model(inputs=model_input_valid, outputs=model_output_valid)
错误说，
Traceback (most recent call last):
File "train_embed.py", line 397, in <module>
x_lstm = LSTM(num_hidden_units,return_sequences=True)(embedding1)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/layers/recurrent.py", line 619, in __call__
return super(RNN, self).__call__(inputs, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 757, in __call__
outputs = self.call(inputs, *args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/layers/recurrent.py", line 2237, in call
inputs, mask=mask, training=training, initial_state=initial_state)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/layers/recurrent.py", line 750, in call
input_length=timesteps)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/backend.py", line 3292, in rnn
swap_memory=True)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 3291, in while_loop
return_same_structure)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 3004, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 2939, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 3260, in <lambda>
body = lambda i, lv: (i + 1, orig_body(*lv))
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/backend.py", line 3245, in _step
new_state.set_shape(state.shape)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 561, in set_shape
raise ValueError(str(e))
ValueError: Shapes must be equal rank, but are 3 and 2
我该怎么做才能更正此代码？ 如果有人可以纠正它，可以给我建议吗？ 谢谢。