解决方案

Question

我没有得到二进制分类问题的预期输出。

问题是使用二元分类将乳腺癌标记为： - 良性，或 - 恶性

它没有提供所需的输出。

首先有一个函数可以加载返回测试和训练形状数据的数据集：

x_train is of shape: (30, 381),
y_train is of shape: (1, 381),
x_test is of shape:  (30, 188),
y_test is of shape:  (1, 188).

然后有一个逻辑回归分类器的类，它预测输出。

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np

def load_dataset():
    cancer_data = load_breast_cancer()
    x_train, x_test, y_train, y_test = train_test_split(cancer_data.data, cancer_data.target, test_size=0.33)
    x_train = x_train.T
    x_test = x_test.T
    y_train = y_train.reshape(1, (len(y_train)))
    y_test = y_test.reshape(1, (len(y_test)))
    m = x_train.shape[1]
    return x_train, x_test, y_train, y_test, m

class Neural_Network():
    def __init__(self):
        np.random.seed(1)
        self.weights = np.random.rand(30, 1) * 0.01
        self.bias = np.zeros(shape=(1, 1))

    def sigmoid(self, x):
        return 1 / (1 + np.exp(-x))

    def train(self, x_train, y_train, iterations, m, learning_rate=0.5):

        for i in range(iterations):
            z = np.dot(self.weights.T, x_train) + self.bias
            a = self.sigmoid(z)

            cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))

            if (i % 500 == 0):
                print("Cost after iteration %i: %f" % (i, cost))

            dw = (1 / m) * np.dot(x_train, (a - y_train).T)
            db = (1 / m) * np.sum(a - y_train)

            self.weights = self.weights - learning_rate * dw
            self.bias = self.bias - learning_rate * db

    def predict(self, inputs):
        m = inputs.shape[1]
        y_predicted = np.zeros((1, m))
        z = np.dot(self.weights.T, inputs) + self.bias
        a = self.sigmoid(z)
        for i in range(a.shape[1]):
            y_predicted[0, i] = 1 if a[0, i] > 0.5 else 0
        return y_predicted

if __name__ == "__main__":
    '''
    step-1 : Loading data set
                 x_train is of shape: (30, 381)
                 y_train is of shape: (1, 381)
                 x_test is of shape:  (30, 188)
                 y_test is of shape:  (1, 188)
    '''

    x_train, x_test, y_train, y_test, m = load_dataset()

    neuralNet = Neural_Network()

    '''
       step-2 : Train the network
    '''

    neuralNet.train(x_train, y_train,10000,m)


    y_predicted = neuralNet.predict(x_test)

    print("Accuracy on test data: ")
    print(accuracy_score(y_test, y_predicted)*100)

提供此输出的程序：

    C:\Python36\python.exe C:/Users/LENOVO/PycharmProjects/MarkDmo001/Numpy.py
Cost after iteration 0: 5.263853
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:25: RuntimeWarning: overflow encountered in exp
  return 1 / (1 + np.exp(-x))
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:33: RuntimeWarning: divide by zero encountered in log
  cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))
C:/Users/LENOVO/PycharmProjects/MarkDmo001/logisticReg.py:33: RuntimeWarning: invalid value encountered in multiply
  cost = (-1 / m) * np.sum(y_train * np.log(a) + (1 - y_train) * np.log(1 - a))
Cost after iteration 500: nan
Cost after iteration 1000: nan
Cost after iteration 1500: nan
Cost after iteration 2000: nan
Cost after iteration 2500: nan
Cost after iteration 3000: nan
Cost after iteration 3500: nan
Cost after iteration 4000: nan
Cost after iteration 4500: nan
Cost after iteration 5000: nan
Cost after iteration 5500: nan
Cost after iteration 6000: nan
Cost after iteration 6500: nan
Cost after iteration 7000: nan
Cost after iteration 7500: nan
Cost after iteration 8000: nan
Cost after iteration 8500: nan
Cost after iteration 9000: nan
Cost after iteration 9500: nan

Accuracy: 
0.0

Answer 1

问题是爆炸渐变。您需要将输入标准化为[0, 1]。

如果你在训练数据中查看特征3和特征23，你会看到大于3000的值。在这些值与你的初始权重相乘后，它们仍然位于[0, 30]范围内。因此，在第一次迭代中，z向量仅包含值为50的正数。因此，a向量（sigmoid的输出）如下所示：

[0.9994797 0.99853904 0.99358676 0.99999973 0.98392862 0.99983016 0.99818802 ...]

因此，在第一步中，您的模型始终以高置信度预测1。但这并不总是正确的，并且模型输出导致大梯度的高概率，当您查看dw的最高值时可以看到。就我而言，

dw[3]是388
dw[23]是571

，其他值位于[0, 55]。因此，您可以清楚地看到这些特征中的大输入如何导致爆炸梯度。由于梯度下降现在朝着相反方向迈出了太大的一步，下一步中的权重不在[0, 0.01]中，而在[-285, 0.002]中，这只会使事情变得更糟。在下一次迭代中，z包含大约-100万的值，这会导致sigmoid函数溢出。

解决方案

将您的输入标准化为[0, 1]
在[-0.01, 0.01]中使用权重，以便它们大致相互抵消。否则，z中的值仍然会与您拥有的要素数呈线性关系。

至于规范化输入，您可以使用sklearn＆＃39; MinMaxScaler：

x_train, x_test, y_train, y_test, m = load_dataset()

scaler = MinMaxScaler()
x_train_normalized = scaler.fit_transform(x_train.T).T

neuralNet = Neural_Network()

'''
   step-2 : Train the network
'''

neuralNet.train(x_train_normalized, y_train,10000,m)

# Use the same transformation on the test inputs as on the training inputs
x_test_normalized = scaler.transform(x_test.T).T
y_predicted = neuralNet.predict(x_test_normalized)

.T是因为sklearn希望训练输入的形状为(num_samples, num_features)，而x_train和x_test的形状为(num_features, num_samples)。

如何获得logisitc回归的正确答案？

1 个答案:

解决方案