Question

我正在用Pytorch训练NN，以预测Boston dataset的预期价格。网络看起来像这样：

from sklearn.datasets import load_boston
from torch.utils.data.dataset import Dataset
from torch.utils.data import DataLoader
import torch.nn.functional as F
import torch
import torch.nn as nn
import torch.optim as optim

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(13, 128)
        self.fc2 = nn.Linear(128, 64)
        self.fc3 = nn.Linear(64, 32)
        self.fc4 = nn.Linear(32, 16)
        self.fc5 = nn.Linear(16,1)

    def forward(self, x):
       x = self.fc1(x)
       x = self.fc2(x)
       x = F.relu(x)
       x = self.fc3(x)
       x = F.relu(x)
       x = self.fc4(x)
       x = F.relu(x)
       return self.fc5(x)

还有数据加载器：

class BostonData(Dataset):
    __xs = []
    __ys = []

    def __init__(self, train = True):
        df = load_boston()
        index = int(len(df["data"]) * 0.7)
        if train:
            self.__xs = df["data"][0:index]
            self.__ys = df["target"][0:index]
        else:
            self.__xs = df["data"][index:]
            self.__ys = df["target"][index:]

    def __getitem__(self, index):
        return self.__xs[index], self.__ys[index]

    def __len__(self):
        return len(self.__xs)

在我的第一次尝试中，我没有添加 ReLU 单位，但是经过一番研究，我发现添加它们是一种常见的做法，但对我而言却没有奏效。

这是培训代码：

dset_train = BostonData(train = True)
dset_test = BostonData(train = False)
train_loader = DataLoader(dset_train, batch_size=30, shuffle=True)
test_loader = DataLoader(dset_train, batch_size=30, shuffle=True)


optimizer = optim.Adam(net.parameters(), lr = 0.001)
criterion = torch.nn.MSELoss() 
EPOCHS = 10000

lloss = []

for epoch in range(EPOCHS):
    for trainbatch in train_loader:
        X,y = trainbatch
        net.zero_grad()
        output = net(X.float())
        loss = criterion(output, y)
        loss.backward()
        optimizer.step()
    lloss.append(loss)
    print(loss)

历经10k之后，损耗图如下所示

我没有看到任何明显的减少。我不知道我是在搞混torch.nn.MSELoss()，optimizer还是网络拓扑，所以将不胜感激。

编辑：改变学习速度和规范化数据对我不起作用。我添加了行self.__xs = (self.__xs - self.__xs.mean()) / self.__xs.std()和对lr = 0.01的更改。损失图与第一个非常相似。

lr = 0.01的相同图并在1000个历元后归一化：

Answer 1

您每个时期追加一次lloss，并且正确，但是您追加了loss（仅使用最后一批），而您应该追加avg_train_loss

尝试：

for epoch in range(EPOCHS):
    avg_train_loss = 0
    for trainbatch in train_loader:
        X,y = trainbatch
        net.zero_grad()
        output = net(X.float())
        loss = criterion(output, y)
        loss.backward()
        optimizer.step()
        avg_train_loss += loss.item() / len(train_loader)
    lloss.append(avg_train_loss)

NN回归损失值没有减少

1 个答案: