Question

我在Windows 10上的matlab 2018b中运行的R-CNN速度更快。当我增加培训项目的数量或增加CUDA_ERROR_ILLEGAL_ADDRESS时，我会遇到MaxEpoch异常

下面是我的gpuDevice

的信息

  CUDADevice with properties:

                  Name: 'GeForce GTX 1050'
                 Index: 1
     ComputeCapability: '6.1'
        SupportsDouble: 1
         DriverVersion: 9.2000
        ToolkitVersion: 9.1000
    MaxThreadsPerBlock: 1024
      MaxShmemPerBlock: 49152
    MaxThreadBlockSize: [1024 1024 64]
           MaxGridSize: [2.1475e+09 65535 65535]
             SIMDWidth: 32
           TotalMemory: 4.2950e+09
       AvailableMemory: 3.4635e+09
   MultiprocessorCount: 5
          ClockRateKHz: 1493000
           ComputeMode: 'Default'
  GPUOverlapsTransfers: 1
KernelExecutionTimeout: 1
      CanMapHostMemory: 1
       DeviceSupported: 1
        DeviceSelected: 1

这是我的代码

latest_index =0;

for i=1:6

load (strcat('newDataset', int2str(i), '.mat')); 
len =length(vehicleDataset.imageFilename);

for j=1:len

filename = vehicleDataset.imageFilename{j};
latest_index=latest_index+1;
fulldata.imageFilename{latest_index} = filename;
fulldata.vehicle{latest_index} = vehicleDataset.vehicle{j};

end
end 

trainingDataTable = table(fulldata.imageFilename', fulldata.vehicle');
trainingDataTable.Properties.VariableNames = {'imageFilename','vehicle'};

data.trainingDataTable = trainingDataTable;
trainingDataTable(1:4,:)


% Split data into a training and test set.
idx = floor(0.6 * height(trainingDataTable));

trainingData = trainingDataTable(1:idx,:);
testData = trainingDataTable(idx:end,:);

% Create image input layer.
inputLayer = imageInputLayer([32 32 3]);

% Define the convolutional layer parameters.
filterSize = [3 3];

numFilters = 64;

% Create the middle layers.
middleLayers = [

convolution2dLayer(filterSize, numFilters, 'Padding', 1)   
reluLayer()
convolution2dLayer(filterSize, numFilters, 'Padding', 1)  
reluLayer() 
maxPooling2dLayer(3, 'Stride',2)
];

finalLayers = [

fullyConnectedLayer(128)
% Add a ReLU non-linearity.
reluLayer()

fullyConnectedLayer(width(trainingDataTable))
% Add the softmax loss layer and classification layer. 
softmaxLayer()
classificationLayer()
];

layers = [
inputLayer
middleLayers
finalLayers
];

% Options for step 1.
 optionsStage1 = trainingOptions('sgdm', ...
 'MaxEpochs', 2, ...
 'MiniBatchSize', 1, ...
 'InitialLearnRate', 1e-3, ...
 'CheckpointPath', tempdir);

% Options for step 2.
optionsStage2 = trainingOptions('sgdm', ...
'MaxEpochs', 2, ...
'MiniBatchSize', 1, ...
'InitialLearnRate', 1e-3, ...
'CheckpointPath', tempdir);

% Options for step 3.
optionsStage3 = trainingOptions('sgdm', ...
'MaxEpochs', 2, ...
'MiniBatchSize', 1, ...    
'InitialLearnRate', 1e-3, ...
'CheckpointPath', tempdir);

% Options for step 4.
optionsStage4 = trainingOptions('sgdm', ...
'MaxEpochs', 2, ...
'MiniBatchSize', 1, ...
'InitialLearnRate', 1e-3, ...
'CheckpointPath', tempdir);

 options = [
 optionsStage1
 optionsStage2
 optionsStage3
 optionsStage4
 ];

 doTrainingAndEval = true;

if doTrainingAndEval
 % Set random seed to ensure example training reproducibility.
 rng(0);

 % Train Faster R-CNN detector. Select a BoxPyramidScale of 1.2 to allow
 % for finer resolution for multiscale object detection.
 detector = trainFasterRCNNObjectDetector(trainingData, layers, options, ...
    'NegativeOverlapRange', [0 0.3], ...
    'PositiveOverlapRange', [0.6 1], ...
    'BoxPyramidScale', 1.2);

 data.detector= detector;
else

% Load pretrained detector for the example.
detector = data.detector;

end

save mix_data data

if doTrainingAndEval

% Run detector on each image in the test set and collect results.

resultsStruct = struct([]);

for i = 1:height(testData)
 % Read the image.
    I = imread(testData.imageFilename{i});
    % Run the detector.
    [bboxes, scores, labels] = detect(detector, I);

    % Collect the results.
    resultsStruct(i).Boxes = bboxes;
    resultsStruct(i).Scores = scores;
    resultsStruct(i).Labels = labels;
end

% Convert the results into a table.
results = struct2table(resultsStruct);

data.results = results;

save mix_data data

else

% Load results from disk.
results = data.results;
end

% Extract expected bounding box locations from test data.
expectedResults = testData(:, 2:end);

% Evaluate the object detector using Average Precision metric.
[ap, recall, precision] = evaluateDetectionPrecision(results, expectedResults);

% Plot precision/recall curve
figure
plot(recall,precision)

xlabel('Recall')
ylabel('Precision')

grid on
title(sprintf('Average Precision = %.2f', ap))

首先它多次打印警告并引发以下异常

警告：CUDA执行期间发生意外错误。 CUDA错误是：   CUDA_ERROR_ILLEGAL_ADDRESS   在trainFasterRCNNObjectDetector中（第320行）     在rcnn_trail中（第184行）

使用-时出错   CUDA执行期间发生意外错误。 CUDA错误是：   CUDA_ERROR_ILLEGAL_ADDRESS

vision.internal.cnn.layer.SmoothL1Loss / backwardLoss中的错误（第156行）               idx =（X>-一）＆（X <一）;   nnet.internal.cnn.DAGNetwork / computeGradientsForTraining / efficiencyBackProp中的错误（第585行）                       dLossdX = thisLayer.backwardLoss（...

nnet.internal.cnn.DAGNetwork中的错误> @（）efficientBackProp（i）（第661行）                           @（）efficiencyBackProp（i），...

nnet.internal.cnn.util.executeWithStagedGPUOOMRecovery中的错误（第11行）           [varargout {1：nOutputs}] = computeFun（）;

nnet.internal.cnn.DAGNetwork中的错误> iExecuteWithStagedGPUOOMRecovery（第1195行）   [varargout {1：nargout}] = nnet.internal.cnn.util.executeWithStagedGPUOOMRecovery（varargin {：}）;

nnet.internal.cnn.DAGNetwork / computeGradientsForTraining中的错误（第660行）                       theseGradients = iExecuteWithStagedGPUOOMRecovery（...

nnet.internal.cnn.Trainer / computeGradients中的错误（第184行）               [梯度，预测，状态] = net.computeGradientsForTraining（X，Y，               needsStatefulTraining，propagationState）;

nnet.internal.cnn.Trainer / train中的错误（第85行）                       [梯度，预测，状态] = this.computeGradients（net，X，response，                       needsStatefulTraining，propagationState）;

vision.internal.cnn.trainNetwork中的错误（第47行）   trainingNet = trainer.train（trainedNet，trainingDispatcher）;

fastRCNNObjectDetector.train中的错误（第190行）               [网络信息] = vision.internal.cnn.trainNetwork（ds，lgraph，opts，映射，               checkpointSaver）;

trainFasterRCNNObjectDetector中的错误（第410行）       [stage2Detector，fastRCNN，〜，info（2）] = fastRCNNObjectDetector.train（trainingData，fastRCNN，       options（2），iStageTwoParams（params），checkpointSaver）;

rcnn_trail中的错误（第184行）       detector = trainFasterRCNNObjectDetector（trainingData，图层，选项...

Answer 1

在与Matlab支持人员交谈之后，显然我的GPU并不是用于深度学习和神经网络的“正确” GPU。

但是，我发现问题是Windows在运行期间更改了GPU，要解决此问题，我去了INVIDIA控制面板>程序设置> 1.选择Mathworks Matlab 2.首选图形处理器选择您的GPU卡

在Matlab上运行Faster R-CNN时，CUDA_ERROR_ILLEGAL_ADDRESS

1 个答案: