Question

我是神经网络的新手，并且已经在文本分析领域学习它的应用程序，所以我在python中使用了一个lstm rnn用于应用程序。

在维度为20,000 * 1的数据集上训练模型（2000-是文本，1-是文本的情感）后，我得到了99％的良好准确率，之后我验证了正在工作的模型罚款（使用model.predict（）函数）。

现在只是为了测试我的模型我一直试图从数据框或包含一些文本的变量提供随机文本输入但我总是因为重新整形数组的错误而需要输入到rnn模型具有维度（1,30）。

但是当我将训练数据重新输入模型进行预测时，模型工作得非常好，为什么会这样？

link for the screenshot of error

link for image of model summary

training data

我只是被困在这里，任何建议都会帮助我学习更多关于rnn的知识，我正在将错误和rnn模型代码附加到此请求中。

谢谢

此致

Tushar Upadhyay

    import numpy as np 
    import pandas as pd 
    import keras
    import sklearn

    from sklearn.feature_extraction.text import CountVectorizer
    from keras.preprocessing.text import Tokenizer
    from keras.preprocessing.sequence import pad_sequences
    from keras.models import Sequential
    from keras.layers import Dense, Embedding, LSTM
     from sklearn.model_selection import train_test_split
    from keras.utils.np_utils import to_categorical
    import re


    data=pd.read_csv('..../twitter_tushar_data.csv')
    max_fatures = 4000
    tokenizer = Tokenizer(num_words=max_fatures, split=' ')
    tokenizer.fit_on_texts(data['tweetText'].values)
    X = tokenizer.texts_to_sequences(data['tweetText'].values)
    X = pad_sequences(X)


    embed_dim = 128
    lstm_out = 196
    model = Sequential()
    keras.layers.core.SpatialDropout1D(0.2) #used to avoid overfitting
    model.add(Embedding(max_fatures, embed_dim,input_length = X.shape[1]))
    model.add(LSTM(196, recurrent_dropout=0.2, dropout=0.2))
   model.add(Dense(2,activation='softmax'))
   model.compile(loss = 'categorical_crossentropy', optimizer='adam',metrics 
   = ['accuracy'])
   print(model.summary())
   #splitting data in training and testing parts

   Y = pd.get_dummies(data['SA']).values
   X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 
   0.30, random_state = 42)
   print(X_train.shape,Y_train.shape)
   print(X_test.shape,Y_test.shape)
   batch_size = 128
   model.fit(X_train, Y_train, epochs = 7, batch_size=batch_size, verbose = 
   2)


   validation_size = 3500

   X_validate = X_test[-validation_size:]
   Y_validate = Y_test[-validation_size:]
   X_test = X_test[:-validation_size]
   Y_test = Y_test[:-validation_size]
   score,acc = model.evaluate(X_test, Y_test, verbose = 2, batch_size = 128)
   print("score: %.2f" % (score))
   print("acc: %.2f" % (acc))


   pos_cnt, neg_cnt, pos_correct, neg_correct = 0, 0, 0, 0
   for x in range(len(X_validate)):
   result = 
 model.predict(X_validate[x].reshape(1,X_test.shape[1]),batch_size=1,verbose 
 = 2)[0]
 if np.argmax(result) == np.argmax(Y_validate[x]):
    if np.argmax(Y_validate[x]) == 0:
        neg_correct += 1
    else:
        pos_correct += 1

if np.argmax(Y_validate[x]) == 0:
    neg_cnt += 1
else:
    pos_cnt += 1
print("pos_acc", pos_correct/pos_cnt*100, "%")
print("neg_acc", neg_correct/neg_cnt*100, "%")

Answer 1

我得到了我的问题的解决方案，这只是一个标记输入正确的问题，谢谢!!下面的代码用于预测不同的用户输入..

 text=np.array(['you are a pathetic awful movie'])
 print(text.shape)
 tk=Tokenizer(num_words=4000,lower=True,split=" ")
 tk.fit_on_texts(text)

prediction=model.predict(sequence.pad_sequences(tk.texts_to_sequences(text),
maxlen=max_review_length))
print(prediction)
print(np.argmax(prediction))

重塑输入标记化文本时出错，预测lstm rnn中的情绪

1 个答案: