Question

TensorFlow 2.0（GPU版本），CUDA 10.0，NVIDIA GeForce RTX 2060，Windows 10（1903）

我正在进行文本分类。我使用字符嵌入和LSTM将每个单词编码为向量。这是我的模型：

import tensorflow as tf
from tensorflow.keras.models import Sequential, Model, load_model, model_from_json
from tensorflow.keras.layers import Input, Embedding, LSTM, TimeDistributed, Multiply, Masking, Concatenate, Dense
from tensorflow.keras.utils import to_categorical

import numpy as np

import re

。

target_word_length = 15
target_sentence_length = 100
mask_embedding = 27

def NetEncoder(string):

    d = dict(zip('abcdefghijklmnopqrstuvwxyz', range(1,27)))

    text_enc = []

    for w in re.findall('[a-z]+', string.lower()):
        word_enc = []
        for c in w:
            word_enc.append(d[c])
        if len(word_enc) < target_word_length:
            for i in range(target_word_length - len(word_enc)):
                word_enc.append(0)
        text_enc.append(word_enc)

    const = []
    for i in range(target_word_length):
        const.append(0)
    const[0] = mask_embedding

    sentence_length = len(text_enc)
    if sentence_length < target_sentence_length:
        for i in range(target_sentence_length - sentence_length):
            text_enc.append(const)
        mask = []
        for i in range(target_sentence_length):
            if i < sentence_length:
                mask.append([1])
            else:
                mask.append([0])

    return [np.array(text_enc, dtype=np.float32), np.array(mask, dtype=np.float32)]

。

emb = Sequential([
    Embedding(28, 3, mask_zero=True, input_shape=(target_word_length,)),
    LSTM(16, return_sequences=False)
])

input_1 = Input(shape=(target_sentence_length, target_word_length))
mask_1 = Input(shape=(target_sentence_length, 1))
net_1 = TimeDistributed(emb)(input_1)
net_1 = Multiply()([net_1, mask_1])
net_1 = Masking(mask_value=np.full(16, 0.))(net_1)
net_1 = LSTM(64, return_sequences=False)(net_1)

input_2 = Input(shape=(target_sentence_length, target_word_length))
mask_2 = Input(shape=(target_sentence_length, 1))
net_2 = TimeDistributed(emb)(input_2)
net_2 = Multiply()([net_2, mask_2])
net_2 = Masking(mask_value=np.full(16, 0.))(net_2)
net_2 = LSTM(64, return_sequences=False)(net_2)

cat = Concatenate()([net_1, net_2])
clf = Dense(32, activation='relu')(cat)
clf = Dense(2, activation='softmax')(clf)

model = Model(inputs=[input_1, mask_1, input_2, mask_2], outputs=clf)

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

[input_1, mask_1] = NetEncoder('Hello Max')
[input_2, mask_2] = NetEncoder('Hello Max')
Y = to_categorical([0], num_classes=2)

model.fit([[input_1], [mask_1], [input_2], [mask_2]], Y, epochs=10)

训练时间：每个样本23毫秒

Train on 1 samples
Epoch 1/10
4/1 [==============================================] - 16s 4s/sample - loss: 0.6840 - accuracy: 0.7500
Epoch 2/10
4/1 [==============================================] - 0s 23ms/sample - loss: 0.6618 - accuracy: 1.0000
Epoch 3/10
4/1 [==============================================] - 0s 23ms/sample - loss: 0.6335 - accuracy: 1.0000

预测时间：每个样本63毫秒

%%time
model.predict([[input_1], [mask_1], [input_2], [mask_2]])

墙壁时间：63毫秒

array（[[1.0000000e + 00，2.6085422e-08]]，dtype = float32）

保存和加载模型（＃1）

model.save('model.h5', save_format='tf', include_optimizer=False)

model_1 = load_model('model.h5')

预测时间更短：37 ms与63 ms。但是输出略有不同。

%%time
model_1.predict([[input_1], [mask_1], [input_2], [mask_2]])

墙壁时间：37.4毫秒

array（[[[1.000000e + 00，2.608547e-08]]，dtype = float32）

model_1.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model_1.fit([[input_1], [mask_1], [input_2], [mask_2]], Y, epochs=10)

训练速度降低了6倍：每个样本139 vs 23毫秒！

Train on 1 samples
Epoch 1/10
4/1 [==============================================] - 4s 1s/sample - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 2/10
4/1 [==============================================] - 1s 133ms/sample - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 3/10
4/1 [==============================================] - 1s 139ms/sample - loss: 0.0000e+00 - accuracy: 1.0000

保存和加载模型（＃2）

model.save_weights('model_V2.h5')
model_json = model.to_json()
with open('model_V2.json', "w") as json_file:
    json_file.write(model_json)
json_file.close()

json_file = open("model_V2.json", 'r')
loaded_model_json = json_file.read()
json_file.close()

model_2 = model_from_json(loaded_model_json)
#model_2.load_weights("model_V2.h5")

model_2.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model_2.fit([[input_1], [mask_1], [input_2], [mask_2]], Y, epochs=10)

训练时间：每个样本149个vs 23毫秒！

Train on 1 samples
Epoch 1/10
4/1 [==============================================] - 4s 1s/sample - loss: 0.6805 - accuracy: 0.7500
Epoch 2/10
4/1 [==============================================] - 1s 145ms/sample - loss: 0.6526 - accuracy: 1.0000
Epoch 3/10
4/1 [==============================================] - 1s 149ms/sample - loss: 0.6186 - accuracy: 1.0000

出了什么问题？

Keras：加载模型的训练慢了6倍

0 个答案: