Question

我正在训练Keras LSTM进行命名实体识别。单词和字符级别都有双向LSTM。

在训练过程中，训练和测试装置的准确性都很高。训练后，我可以在测试集上运行model.evaluate()，并以99％的准确率获得高分。

但是，如果我在model.predict()上使用X_test，则该模型仅预测每个序列都是零数组（我用于填充），并且精度为60％，并且f1分数为0。这是代码：

class BiLSTM:

    def __init__(self):
        self.annotations = load_annotations('data/annotations.p')
        self.cache = format_data(self.annotations)
        self.cv_sets = cross_val_sets(self.cache['padded_sents'],     self.cache['padded_labels'])
        self.embedding_matrix = get_embedding_matrix(self.cache['word_to_integer'])
        self.char_cache = format_char_data(self.annotations,
                                           self.cache['word_to_integer'].keys(),
                                           self.cache['max_sequence_length'])
        self.model = None

    def fit_model(self, X_train, y_train, X_char_train):

        #Extract parameters from the cache
        word_to_integer = self.cache['word_to_integer']
        n_words = self.cache['n_words']
        n_tags = self.cache['n_tags']
        max_sequence_length = self.cache['max_sequence_length']
        X_char = self.char_cache['X_char']
        max_len_char = self.char_cache['max_len_char']
        n_chars = self.char_cache['n_chars']

        # (among top max_features most common words)
        batch_size = 32


        #Word input
        word_in = Input(shape=(max_sequence_length,))

        # Word embedding matrix
        embedding_matrix = get_embedding_matrix(word_to_integer)

        # Word Embedding layer
        embedding_layer = Embedding(input_dim=n_words + 1,
                                output_dim=200,
                                weights=[embedding_matrix],
                                input_length=max_sequence_length,
                                trainable=False)(word_in)

        # input and embeddings for characters
        char_in = Input(shape=(max_sequence_length, max_len_char,))
        emb_char = TimeDistributed(Embedding(input_dim=n_chars + 2, output_dim=10,
                                         input_length=max_len_char, mask_zero=True))(char_in)

        # character LSTM to get word encodings by characters
        char_enc = TimeDistributed(LSTM(units=20, return_sequences=False,
                                    recurrent_dropout=0.5))(emb_char)

        # main LSTM
        x = concatenate([embedding_layer, char_enc])
        x = SpatialDropout1D(0.3)(x)
        main_lstm = Bidirectional(LSTM(units=50, return_sequences=True,
                                   recurrent_dropout=0.6))(x)
        out = TimeDistributed(Dense(n_tags + 1, activation="softmax"))(main_lstm)

        model = Model([word_in, char_in], out)
        optimizer = Adam(lr=0.01, beta_1=0.9, beta_2=0.999)
        model.compile(optimizer=optimizer, loss="categorical_crossentropy", metrics=["acc"])


        model.fit([X_train, X_char_train], y_train,
                        batch_size=32, epochs=5, validation_split=0.2, verbose=1)

        self.model = model


    def run(self, cutoff = 0.8):
        sents = self.cache['padded_sents']
        labels = self.cache['padded_labels']
        # Train a model
        cutoff = int(sents.shape[0]*0.8)
        X_train = sents[:cutoff]
        X_test = sents[cutoff:]
        y_train = labels[:cutoff]
        y_test = labels[cutoff:]
        X_char_train = np.array(self.char_cache['X_char'])[:cutoff]
        X_char_test = np.array(self.char_cache['X_char'])[cutoff:]

        self.fit_model(X_train, y_train, X_char_train)

        # Accuracy metrics
        loss, accuracy = self.model.evaluate([X_test, X_char_test], y_test)

        print(accuracy)

        probs = self.model.predict([X_test, X_char_test])

        predicted = probs.argmax(axis=-1)
        actual = y_test.argmax(axis=-1)
        accuracy, f1 = get_metrics(actual, predicted, self.cache['integer_to_label'])
        print('acc: {}, f1: {}'.format(accuracy, f1))


if __name__ == "__main__":
    lstm = BiLSTM()
    lstm.run()

我已经进行了广泛的搜索，找不到解决方案。任何帮助表示感谢，谢谢！

Answer 1

您是否尝试为

这样的单个序列打印出单个预测

watch: {
    messages: function (val, oldVal) {
       // DOM not updated yet

       this.$nextTick(function () {
           // DOM updated

           this.first_load = true;
           this.scroll_bottom();
       });
    }
}

并将其与

进行比较

predicted = probs[0].argmax(axis = -1)

比较此结果将帮助您进行调试。如果此预测相同，则actual = y_test[0].argmax(axis = -1)必须存在一些问题。试试这个，然后发布反馈。

Keras model.predict给model.evalute提供了不同的结果

1 个答案: