LSTM仅预测一种

Question

LSTM仅在我的测试集中预测一个类。我的训练集上达到0.99。

我尝试添加weight_decay和drop_layer，但是效果不佳。我正在使用word2vec + LSTM + FC。此外，在添加BatchNorm1d之前，我的训练集失败。

class lstmnet(nn.Module):
    def __init__(self, config, emb_weights):
        super(lstmnet, self).__init__()
        self.embedding = nn.Embedding.from_pretrained(embeddings=emb_weights, freeze=config.emb_freeze)
        self.lstm = nn.LSTM(input_size=config.input_size, hidden_size=config.hidden_size, num_layers=config.num_layers,
                            batch_first=True,
                            bidirectional=config.bidir)
        if config.bidir:
            self.l1 = nn.Sequential(nn.Linear(config.hidden_size * 2, config.l1_size), nn.BatchNorm1d(config.l1_size),
                                    nn.ReLU(True))
        else:
            self.l1 = nn.Sequential(nn.Linear(config.hidden_size, config.l1_size), nn.BatchNorm1d(config.l1_size),
                                    nn.ReLU(True))

        self.l2 = nn.Sequential(nn.Linear(config.l1_size, config.l2_size), nn.BatchNorm1d(config.l2_size),
                                nn.ReLU(True))
        self.l3 = nn.Sequential(nn.Linear(config.l2_size, config.num_classes))

    def forward(self, x):
        x = self.embedding(x)
        out, _ = self.lstm(x)
        out = out[:, -1, :]
        out = self.l1(out)
        out = self.l2(out)
        out = self.l3(out)

        return out

我的训练集很平衡。