Question

所以我已经准备了一个情感分析模型，我试图用新的输入来预测它，但是我遇到了一个错误：

data = getFile('Cleaned Data.xlsx')

data['Description'] = data['Description'].apply(lambda x: x.lower())
data['Description'] = data['Description'].apply((lambda x: re.sub('[^a-zA-z0-9\s]','',x)))

print(data[ data['Classification'] == 1].size)
print(data[ data['Classification'] == 0].size)

for idx,row in data.iterrows():
    row[1] = row[1].replace('rt',' ')

tokenizer = Tokenizer(split=' ')
tokenizer.fit_on_texts(data['Description'].values)

#vectorizer = CountVectorizer()
#X = vectorizer.fit_transform(jobSpec['Description']).toarray()

X = tokenizer.texts_to_sequences(data['Description'].values)
X = pad_sequences(X, maxlen=100, value=0.)

display(X.shape)

vocab_size = len(tokenizer.word_index) + 1
max_length = max([len(s.split()) for s in data['Description']])

Y = pd.get_dummies(data['Classification']).values
X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.33, random_state = 42)


tokens_docs = [doc.split(" ") for doc in data['Description'].values]
all_tokens = itertools.chain.from_iterable(tokens_docs)
my_dict = {token: token if token.isdigit() else idx for idx, token in enumerate(set(all_tokens))}

print(X_train.shape,Y_train.shape)
print(X_test.shape,Y_test.shape)

raw_embedding = load_embedding('glove.6B.100d.txt')

embedding_vectors= get_weight_matrix(raw_embedding, tokenizer.word_index)
embedding_layer = Embedding(vocab_size, 100, weights=[embedding_vectors], input_length = X.shape[1], trainable=False, name="Embeddings")

display(embedding_vectors.shape)

# define model
sequence_input = Input(shape=(100,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
x = Conv1D(128, 5, activation='relu')(embedded_sequences)
x = MaxPooling1D(2)(x)
x = Conv1D(128, 5, activation='relu')(x)
x = MaxPooling1D(2)(x)
x = Conv1D(128, 5, activation='relu')(x)
x = MaxPooling1D(18)(x)
x = Flatten()(x)
x = Dense(128, activation='relu')(x)
preds = Dense(2, activation='softmax')(x)
model = Model(sequence_input, preds)
model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['acc'])


# input for which we need the embedding
input_str = "the company our client is a renowned civil actor who have consistently and safely delivered major civil infrastructure  ts across a"

# build index based on our `vocabulary`
word_to_idx = OrderedDict({w:all_tokens.index(w) for w in input_str.split() if w in all_tokens})

ynew = model.predict([1],[3],[5],[7])
display(ynew)

当我尝试使用新输入预测该模型时：

ynew = model.predict([1],[3],[5],[7])
display(ynew)

它给我一条错误消息：

ValueError: Error when checking input: expected input_29 to have shape (100,) but got array with shape (1,)

我试图将模型的形状更改为None和1，但这给了我其他新的错误。我对机器学习知识还很陌生，所以真的不确定如何解决这个问题。

任何帮助将不胜感激

Answer 1

我通过将输入更改为以下内容来修复它：

text = ['Construction workers needed for this company who has qualifications ']
#vectorizing the tweet by the pre-fitted tokenizer instance
text = tokenizer.texts_to_sequences(text)

text = pad_sequences(twt, maxlen=100, dtype='int32', value=0)
print(text)
sentiment = model.predict(text)[0]
display(sentiment)

出现错误是因为我没有填充输入以匹配输入尺寸的长度。

在keras中预测将句子中的单词索引作为输入的conv1d模型

1 个答案: