为什么我的损失函数总是返回nan？

Question

我正在尝试学习ML，并试图建立一个简单的线性模型，但是当我运行它时，损失会显示为空：

所以我试图找出问题所在。如果我先打印10个y_pred，则其中只有17个有数字，其余则为null。也许我做错了，请帮忙。

import torch
from torch.utils.data import TensorDataset, DataLoader
import torch.nn as nn
import numpy as np

#Input (temp, rainfall, humidity)
inputs = np.array([[73, 67, 43], [91, 88, 64], [87, 134, 58], [102, 43, 37], [69, 96, 70], [73, 67, 43], [91, 88, 64], [87, 134, 58], [102, 43, 37], [69, 96, 70], [73, 67, 43], [91, 88, 64], [87, 134, 58], [102, 43, 37], [69, 96, 70]], dtype='float32')

#Target (apples, oranges)
targets = np.array([[56, 70], [81, 101], [119, 133], [22, 37], [103, 119], [56, 70], [81, 101], [119, 133], [22, 37], [103, 119], [56, 70], [81, 101], [119, 133], [22, 37], [103, 119]], dtype='float32')

inputs = torch.from_numpy(inputs)
targets = torch.from_numpy(targets)

#Define datasets
train_ds = TensorDataset(inputs, targets)
train_ds[0:3]

#Hyperparameters
batch_size = 5
num_epochs = 100
learning_rate = 0.01

train_dl = DataLoader(dataset=train_ds, batch_size=batch_size, shuffle=True)


model = nn.Linear(3,2) #inputs(temp, rainfall, humidity) , targets(apples, oranges)

loss_f = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

for epoch in range(num_epochs):
    for xb, yb in train_dl:
        y_pred = model(xb)
        loss = loss_f(y_pred, yb)
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()
        
    if(epoch+1) % 10 == 0:
        print(f'epoch = {epoch+1}/{num_epochs}, loss = {loss.item():.4f}')
        
print(f'Final loss = {loss.item():.4f}')

编辑： y_pred.shape = torch.Size([5, 2]), yb.shape = torch.Size([5, 2])

Answer 1

与损失功能无关。您的模型正在预测Nan和Infy的数字。可能的解决方案

• 降低学习率（例如：learning_rate = 0.001）或
• 减小批量大小（例如：batch_size = 2）或
• 使用激活功能将更多层添加到模型中，或
• 标准化输入

Answer 2

在给定信息量的情况下，我的第一次尝试就是改变

loss_f = nn.MSELoss()

到

loss_f = nn.MSELoss(size_average=True)