RuntimeError：输入类型（torch.FloatTensor）和权重类型（torch.cuda.FloatTensor）应该相同

Question

我正在尝试如下训练以下CNN，但对于.cuda（）我仍然遇到相同的错误，并且不确定如何解决。到目前为止，这是我的大部分代码。

import matplotlib.pyplot as plt
import numpy as np
import torch
from torch import nn
from torch import optim
import torch.nn.functional as F
import torchvision
from torchvision import datasets, transforms, models
from torch.utils.data.sampler import SubsetRandomSampler


data_dir = "/home/ubuntu/ML2/ExamII/train2/"
valid_size = .2

# Normalize the test and train sets with torchvision
train_transforms = transforms.Compose([transforms.Resize(224),
                                           transforms.ToTensor(),
                                           ])

test_transforms = transforms.Compose([transforms.Resize(224),
                                          transforms.ToTensor(),
                                          ])

# ImageFolder class to load the train and test images
train_data = datasets.ImageFolder(data_dir, transform=train_transforms)
test_data = datasets.ImageFolder(data_dir, transform=test_transforms)


# Number of train images
num_train = len(train_data)
indices = list(range(num_train))
# Split = 20% of train images
split = int(np.floor(valid_size * num_train))
# Shuffle indices of train images
np.random.shuffle(indices)
# Subset indices for test and train
train_idx, test_idx = indices[split:], indices[:split]
# Samples elements randomly from a given list of indices
train_sampler = SubsetRandomSampler(train_idx)
test_sampler = SubsetRandomSampler(test_idx)
# Batch and load the images
trainloader = torch.utils.data.DataLoader(train_data, sampler=train_sampler, batch_size=1)
testloader = torch.utils.data.DataLoader(test_data, sampler=test_sampler, batch_size=1)


#print(trainloader.dataset.classes)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = models.resnet50(pretrained=True)

model.fc = nn.Sequential(nn.Linear(2048, 512),
                                 nn.ReLU(),
                                 nn.Dropout(0.2),
                                 nn.Linear(512, 10),
                                 nn.LogSigmoid())
                                 # nn.LogSoftmax(dim=1))
# criterion = nn.NLLLoss()
criterion = nn.BCELoss()
optimizer = optim.Adam(model.fc.parameters(), lr=0.003)
model.to(device)

#Train the network
for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:    # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

但是，我一直在控制台中收到此错误： RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

关于如何解决它的任何想法？我读到该模型可能尚未推送到我的GPU中，但不确定如何修复。谢谢！

Answer 1

您的神经网络位于GPU上，但您的输入却不在。因此，您需要将输入张量发送到CUDA。

inputs, labels = data
inputs, labels = inputs.cuda(), labels.cuda() # add this line

或者像这样，与其余代码保持一致：

inputs, labels = inputs.to(device), labels.to(device)

Answer 2

新的API将使用.to()方法。

优势显而易见，而且很重要。 明天您的设备可能不是“ cuda”：

• cpu
• cuda
• mkldnn
• opengl
• opencl
• ideep
• hip
• msnpu
• xla

因此，请避免使用model.cuda() 检查设备没有错

dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

或对其进行硬编码：

dev=torch.device("cuda") 

与：

dev="cuda"

通常，您可以使用以下代码：

model.to(dev)
data = data.to(dev)

Answer 3

正如前面的答案中已经提到的那样，问题可能在于您的模型是在GPU上训练的，但是已经在CPU上进行了测试。如果是这种情况，则需要像这样将模型的权重和数据从GPU移植到CPU：

device = args.device # "cuda" / "cpu"
if "cuda" in device and not torch.cuda.is_available():
    device = "cpu"
data = data.to(device)
model.to(device)

注意：在这里，我们仍然检查配置参数是否设置为GPU或CPU，以便这段代码可用于训练（在GPU上）和测试（在CPU上）。

Answer 4

   * when you get this error::RuntimeError: Input type 
   (torch.FloatTensor) and weight type (torch.cuda.FloatTensor should 
   be the same
   # Move tensors to GPU is CUDA is available
   # Check if CUDA is available

  train_on_gpu = torch.cuda.is_available()

  If train_on_gpu:
      print("CUDA is available! Training on GPU...")
  else:
      print("CUDA is not available. Training on CPU...")

 -------------------
 # Move tensors to GPU is CUDA is available
if train_on_gpu:

model.cuda()

Answer 5

首次检查cuda是否可用：

  if torch.cuda.is_available():
      device = 'cuda'
  else:
      device = 'cpu'

如果要加载某些模型，请执行以下操作：

  checkpoint = torch.load('./generator_release.pth', map_location=device)
  G = Generator().to(device)

现在您可能会遇到此错误：

RuntimeError：输入类型（torch.FloatTensor）和重量类型（torch.cuda.FloatTensor）应该相同

需要通过以下方式将输入数据的类型从torch.tensor转换为torch.cuda.tensor：

if torch.cuda.is_available():
  data = data.cuda()
result = G(data)

，然后将结果从torch.cuda.tensor转换为torch.tensor：

if torch.cuda.is_available():
    result = result.cpu()