Question

我是机器学习和深度学习的新手，我正在尝试使用神经网络对5个类别的文本进行分类。为此，我制作了一个字典以将单词翻译成索引，最后得到一个包含索引列表的数组。此外，我将标签更改为整数。我也做了填充和其他东西。问题在于，当我拟合模型时，精度会保持很低的水平（约0.20），并且在各个时期都不会改变。我试图更改很多参数，例如词汇量，神经元数量，辍学概率，优化程序参数等。代码的关键部分如下。

# Arrays with indexes (that works fine)
X_train = tokens_to_indexes(tokenized_tr_mrp, vocab, return_vocab=False)
X_test, vocab_dict = tokens_to_indexes(tokenized_te_mrp, vocab)

# Labels to integers
labels_dict = {}
labels_dict['Alzheimer'] = 0
labels_dict['Bladder Cancer'] = 1
labels_dict['Breast Cancer'] = 2
labels_dict['Cervical Cancer'] = 3
labels_dict['Negative'] = 4

y_train = np.array([labels_dict[i] for i in y_tr])
y_test = np.array([labels_dict[i] for i in y_te])

# One-hot encoding of labels
from keras.utils import to_categorical
encoded_train = to_categorical(y_train)
encoded_test = to_categorical(y_test)

# Padding
max_review_length = 235
X_train_pad = sequence.pad_sequences(X_train, maxlen=max_review_length)
X_test_pad = sequence.pad_sequences(X_test, maxlen=max_review_length)

# Model
# Vocab size
top_words = len(list(vocab_dict.keys()))
# Neurone type
rnn = LSTM
# dropout
set_dropout = True
p = 0.2
# embedding size
embedding_vector_length = 64
# regularization strength
L = 0.0005
# Number of neurones
N = 50

# Model
model = Sequential()
# Embedding layer
model.add(Embedding(top_words,
                   embedding_vector_length,
                   embeddings_regularizer=regularizers.l1(l=L),
                   input_length=max_review_length
                    #,embeddings_constraint=UnitNorm(axis=1)
                   ))

# Dropout layer
if set_dropout:
    model.add(Dropout(p))

# Recurrent layer
model.add(rnn(N))

# Output layer
model.add(Dense(5, activation='softmax'))

# Compilation
model.compile(loss='categorical_crossentropy',
             optimizer=Adam(lr=0.001),
             metrics=['Accuracy'])

# Split training set for validation
X_tr, X_va, y_tr_, y_va = train_test_split(X_train_pad, encoded_train, 
                                          test_size=0.3, random_state=2)

# Parameters
batch_size = 50
# N epochs
n_epocas = 20

best_val_acc  = 0
best_val_loss = 1e20
best_i        = 0

best_weights = []
acum_tr_acc = []
acum_tr_loss = []
acum_val_acc = []
acum_val_loss = []

# Training
for e in range(n_epocas):
    h = model.fit(X_tr, y_tr_,
                batch_size=batch_size,
                validation_data=(X_va, y_va),
                epochs=1, verbose=1)
  
    acum_tr_acc  = acum_tr_acc  + h.history['accuracy']
    acum_tr_loss = acum_tr_loss + h.history['loss']
    val_acc  = h.history['val_accuracy'][0]
    val_loss = h.history['val_loss'][0]
    acum_val_acc  = acum_val_acc  + [val_acc]
    acum_val_loss = acum_val_loss + [val_loss]
#   if val_acc  > best_val_acc:
    if val_loss < best_val_loss:
        best_i        = len(acum_val_acc)-1
        best_val_acc  = val_acc
        best_val_loss = val_loss
        best_weights = model.get_weights().copy()
    if len(acum_tr_acc)>1 and (len(acum_tr_acc)+1) % 1 == 0:
        if e>1:
            clear_output()

Answer 1

您发布的代码确实是错误的做法。

您可以使用当前方法为n_epocas进行训练，并添加回调以获取最佳权重（例如ModelCheckpoint），也可以使用tf.GradientTape，但使用model.fit()进行一个时期一次会导致奇怪的结果，因为优化器不知道它处于哪个时期。

我建议保留您当前的代码，但要一口气为n_epocas进行培训，并在此处报告结果（准确性+损失）。

Answer 2

有人给了我解决方案。我只需要更改此行：

model.compile(loss='categorical_crossentropy',
             optimizer=Adam(lr=0.001),
             metrics=['Accuracy'])

为此：

model.compile(loss='categorical_crossentropy',
                 optimizer=Adam(lr=0.001),
                 metrics=['acc'])

我还更改了最后一个循环中与准确性有关的行。一键编码也是必要的。

准确性并没有随着时代的发展而提高

2 个答案: