Question

我正在尝试从波士顿数据集中进行线性回归。自第一次迭代以来，MSE损失函数为nan。我尝试更改学习率和batch_size，但没有用。

from torch.utils.data import TensorDataset , DataLoader
inputs  = torch.from_numpy(Features).to(torch.float32)
targets = torch.from_numpy(target).to(torch.float32)
train_ds = TensorDataset(inputs , targets)
train_dl = DataLoader(train_ds , batch_size = 5 , shuffle = True)

model = nn.Linear(13,1)
opt = optim.SGD(model.parameters(), lr=1e-5)
loss_fn = F.mse_loss

def fit(num_epochs, model, loss_fn, opt, train_dl):
    
    # Repeat for given number of epochs
    for epoch in range(num_epochs):
        
        # Train with batches of data
        for xb,yb in train_dl:
            
            # 1. Generate predictions
            pred = model(xb)
            
            # 2. Calculate loss
            loss = loss_fn(pred, yb)
            
            # 3. Compute gradients
            loss.backward()
            
            # 4. Update parameters using gradients
            opt.step()
            
            
            # 5. Reset the gradients to zero
            opt.zero_grad()
        
        # Print the progress
        if (epoch+1) % 10 == 0:
            print('Epoch [{}/{}], Loss: {}'.format(epoch+1, num_epochs, loss.item()))


fit(100, model, loss_fn , opt , train_dl)

output

Answer 1

注意：

使用规范化：x = (x - x.mean()) / x.std()
y_train / y_test 必须是 (-1, 1) 形状。使用 y_train.view(-1, 1)（如果 y_train 是 torch.Tensor 或其他东西）
（不是您的情况，而是其他人的情况）如果您使用 torch.nn.MSELoss(reduction='sum')，那么您必须重新使用总和来表示。可以使用 torch.nn.MSELoss() 或在 train-loop 中完成：l = loss(y_pred, y) / y.shape[0]。

示例：

    ...
    loss = torch.nn.MSELoss()  
    ...
    for epoch in range(num_epochs):  
        for x, y in train_iter:  
            y_pred = model(x)  
            l = loss(y_pred, y)  
            optimizer.zero_grad()  
            l.backward()  
            optimizer.step()  
        print("epoch {} loss: {:.4f}".format(epoch + 1, l.item()))

训练期间Pytorch MSE损失函数nan

1 个答案: