Question

我是深度学习的新手。我正在尝试在词嵌入功能方面建立非常基本的LSTM网络。我已经为模型编写了以下代码，但无法运行。

from keras.layers import Dense, LSTM, merge, Input,Concatenate
from keras.layers.recurrent import LSTM
from keras.models import Sequential, Model
from keras.layers import Dense, Dropout, Flatten


max_sequence_size = 14
classes_num = 2

LSTM_word_1 = LSTM(100, activation='relu',recurrent_dropout = 0.25, dropout = 0.25)
lstm_word_input_1 = Input(shape=(max_sequence_size, 300))
lstm_word_out_1 = LSTM_word_1(lstm_word_input_1)


merged_feature_vectors = Dense(50, activation='sigmoid')(Dropout(0.2)(lstm_word_out_1))

predictions = Dense(classes_num, activation='softmax')(merged_feature_vectors)

my_model = Model(input=[lstm_word_input_1], output=predictions)
print my_model.summary()

我遇到的错误是ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (3019, 300)。在搜索时，我发现人们已经使用Flatten()来压缩密集层的所有二维特征（3019,300）。但我无法解决此问题。

在解释时，请告诉我尺寸的计算方式。

根据要求：

我的X_training存在尺寸问题，因此，我在下面提供了代码以消除混乱，

def makeFeatureVec(words, model, num_features):
    # Function to average all of the word vectors in a given
    # paragraph
    #
    # Pre-initialize an empty numpy array (for speed)
    featureVec = np.zeros((num_features,),dtype="float32")
    #
    nwords = 0.
    #
    # Index2word is a list that contains the names of the words in
    # the model's vocabulary. Convert it to a set, for speed
    index2word_set = set(model.wv.index2word)
    #
    # Loop over each word in the review and, if it is in the model's
    # vocaublary, add its feature vector to the total
    for word in words:
        if word in index2word_set:
            nwords = nwords + 1.
            featureVec = np.add(featureVec,model[word])
    #
    # Divide the result by the number of words to get the average
    featureVec = np.divide(featureVec,nwords)
    return featureVec

我认为下面的代码正在以这种方式初始化二维numpy数组

def getAvgFeatureVecs(reviews, model, num_features):
    # Given a set of reviews (each one a list of words), calculate
    # the average feature vector for each one and return a 2D numpy array
    #
    # Initialize a counter
    counter = 0.
    #
    # Preallocate a 2D numpy array, for speed
    reviewFeatureVecs = np.zeros((len(reviews),num_features),dtype="float32")

    for review in reviews:

       if counter%1000. == 0.:
           print "Question %d of %d" % (counter, len(reviews))

       reviewFeatureVecs[int(counter)] = makeFeatureVec(review, model, \
           num_features)

       counter = counter + 1.
    return reviewFeatureVecs


def getCleanReviews(reviews):
    clean_reviews = []
    for review in reviews["question"]:
        clean_reviews.append( KaggleWord2VecUtility.review_to_wordlist( review, remove_stopwords=True ))
    return clean_reviews

我的目标只是对我已有的评论使用gensim预训练模型进行LSTM。

trainDataVecs = getAvgFeatureVecs( getCleanReviews(train), model, num_features )

Answer 1

您应该尝试在LSTM层之前使用Embedding layer。另外，由于您已经为3019条注释预训练了300维矢量，因此可以使用此矩阵初始化嵌入层的权重。

inp_layer = Input((maxlen,))
x = Embedding(max_features, embed_size, weights=[trainDataVecs])(x)
x = LSTM(50, dropout=0.1)(x)

在这里，maxlen是注释的最大长度，max_features是数据集的唯一单词或词汇量的最大数目，embed_size是向量的维数，您的情况是300。

请注意trainDataVecs的形状应为（max_features，embed_size），因此，如果您将预先训练的单词向量加载到trainDataVecs中，则应该可以。

预训练词嵌入gensim上的LSTM网络

1 个答案: