Question

对于相同的数据集（此处为Bupa）和参数，我得到不同的准确度。

我忽略了什么？

R 实施：

data_file = "bupa.data"
dataset = read.csv(data_file, header = FALSE)
nobs <- nrow(dataset) # 303 observations 
sample <- train <- sample(nrow(dataset), 0.95*nobs) # 227 observations
# validate <- sample(setdiff(seq_len(nrow(dataset)), train), 0.1*nobs) # 30 observations
test <- setdiff(seq_len(nrow(dataset)), train) # 76 observations
svmfit <- svm(V7~ .,data=dataset[train,],
              type="C-classification",
              kernel="linear",
              cost=1,
              cross=10)
testpr <- predict(svmfit, newdata=na.omit(dataset[test,]))
accuracy <- sum(testpr==na.omit(dataset[test,])$V7)/length(na.omit(dataset[test,])$V7)

我得到准确度： 0.94

但是当我在 python（scikit-learn）

中执行以下操作时

import numpy as np
from sklearn import cross_validation
from sklearn import datasets
import pandas as pd
from sklearn import svm, grid_search

f = open("data/bupa.data")
dataset =  np.loadtxt(fname = f, delimiter = ',')
nobs = np.shape(dataset)[0]
print("Number of Observations: %d" % nobs)
y = dataset[:,6]
X = dataset[:,:-1]
X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size=0.06, random_state=0)

clf = svm.SVC(kernel='linear', C=1).fit(X_train, y_train)
scores = cross_validation.cross_val_score(clf, X, y, cv=10, scoring='accuracy')

我的准确度 0.67

请帮帮我。

Answer 1

我遇到了同样问题的帖子 - scikit-learn和libSVM的e1071绑定之间的准确性差异很大。我认为问题是e1071缩放训练数据，然后保留缩放参数以用于预测新观察。 Scikit-learn不会这样做，并让用户意识到需要对训练和测试数据采用相同的缩放方法。我只是想在从libSVM背后的好人那里遇到并阅读this guide之后检查一下。

虽然我没有您的数据，str(svmfit)应该为您提供缩放参数（Bupa列的平均值和标准偏差）。您可以使用它们在Python中适当地扩展数据（请参阅下面的想法）。或者，您可以在Python中将整个数据集缩放在一起，然后进行测试/训练分割;无论哪种方式都应该给你现在相同的预测。

def manual_scale(a, means, sds):
    a1 = a - means
    a1 = a1/sds
    return a1

Answer 2

在Python / sklearn和R / e1071中使用支持向量回归时，x和y变量都需要缩放/不缩放。这是一个使用rpy2展示R和Python结果相等的独立示例（第一部分在R中禁用了缩放，第二部分在Python中具有“手动”缩放）：

# import modules
import matplotlib.pyplot as plt
import numpy as np
import sklearn
import sklearn.model_selection
import sklearn.datasets
import sklearn.svm
import rpy2
import rpy2.robjects
import rpy2.robjects.packages

# use R e1071 SVM function via rpy2
def RSVR(x_train, y_train, x_test,
         cost=1.0, epsilon=0.1, gamma=0.01, scale=False):

    # convert Python arrays to R matrices
    rx_train = rpy2.robjects.r['matrix'](rpy2.robjects.FloatVector(np.array(x_train).T.flatten()), nrow = len(x_train))
    ry_train = rpy2.robjects.FloatVector(np.array(y_train).flatten())
    rx_test = rpy2.robjects.r['matrix'](rpy2.robjects.FloatVector(np.array(x_test).T.flatten()), nrow = len(x_test))

    # train SVM
    e1071 = rpy2.robjects.packages.importr('e1071')
    rsvr = e1071.svm(x=rx_train,
                     y=ry_train,
                     kernel='radial',
                     cost=cost,
                     epsilon=epsilon,
                     gamma=gamma,
                     scale=scale)

    # run SVM
    predict = rpy2.robjects.r['predict']
    ry_pred = np.array(predict(rsvr, rx_test))

    return ry_pred

# define auxiliary function for plotting results
def plot_results(y_test, py_pred, ry_pred, title, lim=[-500, 500]):
    plt.title(title)
    plt.plot(lim, lim, lw=2, color='gray', zorder=-1)
    plt.scatter(y_test, py_pred, color='black', s=40, label='Python/sklearn')
    plt.scatter(y_test, ry_pred, color='orange', s=10, label='R/e1071')
    plt.xlabel('observed')
    plt.ylabel('predicted')
    plt.legend(loc=0)
    return None

# get example regression data
x_orig, y_orig = sklearn.datasets.make_regression(n_samples=100, n_features=10, random_state=42)

# split into train and test set
x_train, x_test, y_train, y_test = sklearn.model_selection.train_test_split(x_orig, y_orig, train_size=0.8)

# SVM parameters
# (identical but named differently for R/e1071 and Python/sklearn)
C = 1000.0
epsilon = 0.1
gamma = 0.01

# setup SVM and scaling classes
psvr = sklearn.svm.SVR(kernel='rbf', C=C, epsilon=epsilon, gamma=gamma)
x_sca = sklearn.preprocessing.StandardScaler()
y_sca = sklearn.preprocessing.StandardScaler()

# run R and Python SVMs without any scaling
# (see 'scale=False')
py_pred = psvr.fit(x_train, y_train).predict(x_test)
ry_pred = RSVR(x_train, y_train, x_test,
               cost=C, epsilon=epsilon, gamma=gamma, scale=False)

# scale both x and y variables
sx_train = x_sca.fit_transform(x_train)
sy_train = y_sca.fit_transform(y_train.reshape(-1, 1))[:, 0]
sx_test = x_sca.transform(x_test)
sy_test = y_sca.transform(y_test.reshape(-1, 1))[:, 0]

# run Python SVM on scaled data and invert scaling afterwards
ps_pred = psvr.fit(sx_train, sy_train).predict(sx_test)
ps_pred = y_sca.inverse_transform(ps_pred.reshape(-1, 1))[:, 0]

# run R SVM with native scaling on original/unscaled data
# (see 'scale=True')
rs_pred = RSVR(x_train, y_train, x_test,
               cost=C, epsilon=epsilon, gamma=gamma, scale=True)

# plot results
plt.subplot(121)
plot_results(y_test, py_pred, ry_pred, 'without scaling (Python/sklearn default)')
plt.subplot(122)
plot_results(y_test, ps_pred, rs_pred, 'with scaling (R/e1071 default)')
plt.tight_layout()

更新：实际上，缩放在R和Python中使用的方差定义略有不同，请参见this answer（R中的1 /（N-1）...与Python中的1 / N ...其中N是样本数量）。但是，对于典型的样本量，这应该可以忽略不计。

Answer 3

我可以证实这些陈述。确实需要对火车和测试装置应用相同的缩放。特别是我这样做了：

from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
X = sc_X.fit_transform(X)

其中X是我的训练集。然后，在准备测试集时，我只使用了从训练测试的缩放中获得的StandardScaler实例。重要的是将它用于转换，而不是用于拟合和转换（如上所述），即：

X_test = sc_X.transform(X_test)

这使得R和scikit-learn结果之间获得了实质性的一致。

python（Scikit-Learn）和R（e1071）的准确度不同

3 个答案: