Question

我正在为此数据集创建分类器模型： https://archive.ics.uci.edu/ml/datasets/ILPD+%28Indian+Liver+Patient+Dataset%29 并且我在pytorch中提出了以下代码：

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
import pandas as pd
import numpy as np


ILPDataset = pd.read_csv('preprocessed-ilpd.csv')
ILPDataset["Age"] = pd.to_numeric(ILPDataset["Age"])
LabelEncoder = preprocessing.LabelEncoder()
LabelEncoder.fit(ILPDataset["Gender"])
ILPDataset["Gender"] = LabelEncoder.transform(ILPDataset["Gender"])
print(ILPDataset["Gender"].describe())
ILPDataset["AAP"] = preprocessing.scale(ILPDataset["AAP"])
ILPDataset["SgAlAm"] = preprocessing.scale(ILPDataset["SgAlAm"])
ILPDataset["SgApAm"] = preprocessing.scale(ILPDataset["SgApAm"])
Features = ["Age","Gender","TB","DB","AAP","SgAlAm","SgApAm","TP","ALB","A/G"]
ILPDFeatures = ILPDataset[Features]
ILPDTarget = ILPDataset["Selector"]

X_Train, X_Test, Y_Train, Y_Test = train_test_split(ILPDFeatures,ILPDTarget,test_size=0.2,random_state=0)

print(X_Train.shape)
print(Y_Train.shape)
print(X_Test.shape)
print(Y_Test.shape)
torch.set_default_tensor_type(torch.DoubleTensor)
TrainX = torch.from_numpy(X_Train.values).double()
TestX = torch.from_numpy(X_Test.values).double()


TrainY = torch.from_numpy(Y_Train.values).long().view(1,-1)[0]
TestY = torch.from_numpy(Y_Test.values).long().view(1,-1)[0]
TrainY.reshape(TrainY.shape[0],1)
TestY.reshape(TestY.shape[0],1)
print(X_Train.shape[1])
input_layers = X_Train.shape[1]
output_layers = 2
hidden = 100

class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()
        self.fc1 = nn.Linear(input_layers,hidden)
        self.fc2 = nn.Linear(hidden,hidden)
        self.fc3 = nn.Linear(hidden,output_layers)

    def forward(self,x):
        x = torch.sigmoid(self.fc1(x))
        x = torch.sigmoid(self.fc2(x))
        x = self.fc3(x)
        return F.softmax(x,dim=-1)

model = Net()
optimizer = torch.optim.SGD(model.parameters(),lr = 0.0001,momentum = 0.9)
loss_fn = nn.NLLLoss()
epochs_data = []
epochs = 601
print(TrainX.shape)
print(TestY.shape)
for epoch in range(1,3):
    optimizer.zero_grad()
    Ypred = model(TrainX)
    loss = loss_fn(Ypred,TrainY)
    loss.backward()
    optimizer.step()
    print(Ypred)
    if epoch%100==0:
        print("Epoch - %d, (%d%%) "%(epoch,epoch/epochs*100))

YPred_Test = model(TestX)
Loss_Test = loss_fn(YPred_Test,TestY)
print(Loss_Test.item())
print(YPred_Test)
print(YPred_Test)
print(TestY)

我对不同的LR和动量使用了不同的优化器，但是在loss.backward（）之后应用每个优化器时，它将张量中的值转换为NaN。我在SO上查找了不同的答案，但是即使尝试了一下，我仍然无法解决错误。

Answer 1

问题是您将softmax与NLLLoss一起使用，但不能一起使用。 softmax上的documentation状态：

该模块无法直接与NLLLoss配合使用，后者需要Log 在Softmax及其自身之间进行计算。改用LogSoftmax （速度更快，并且具有更好的数值属性）。

在log_softmax的最后一层使用NLLLoss，如下所示：

def forward(self,x):
    ...
    return F.log_softmax(x,dim=-1)
...
loss_fn = nn.NLLLoss()

Answer 2

该问题是由于未规范化数据集中的每一列而引起的。我不知道为什么，但归一化列可以解决问题。

应用任何优化都会导致值变为NaN

2 个答案: