我在理解如何创建sklearn变压器的子类时遇到问题。对于长代码示例，我深表歉意，我尝试使最低限度可重现，但无法重新创建错误。希望您会看到大多数代码示例都是我在编写文档。
下面在代码段中描述了该变压器。
class PCAVarThreshSelector(PCA):
"""
Description
-----------
Selects the columns that can explain a certain percentage of the variance in a data set
Authors
-------
Eden Trainor
Notes
-----
1. PCA has a principole component limit of 4459 components, no matter how many more features you put into
it this is a hrad limit of how many components it will return to you.
"""
def __init__(self,
n_components=None,
copy=True,
whiten=False,
svd_solver='auto',
tol=0.0,
iterated_power='auto',
random_state=None,
explained_variance_thresh = 0.8):
super(PCAVarThreshSelector, self).__init__(n_components, copy, whiten, svd_solver, tol, iterated_power, random_state)
self.explained_variance_thresh = explained_variance_thresh
def find_nearest_index(self, array, value):
"""
Description
-----------
Finds the index of the coefficient in an array nearest a certain value.
Args
----
array: np.ndarray, (number_of_componants,)
Array containing coeffficients
value: int,
Index of coefficient in array closset to this value is found.
Returns
-------
index: int,
Index of coefficient in array closest to value.
"""
index = (np.abs(array - value)).argmin()
return index
def fit(self, X, y = None):
"""
Description
-----------
Fits the PCA and calculates the index threshold index of the cumulative explained variance ratio array.
Args
----
X: DataFrame, (examples, features)
Pandas DataFrame containing training example features
y: array/DataFrame, (examples,)
(Optional) Training example labels
Returns
-------
self: PCAVarThreshSelector instance
Returns transfromer instance with fitted instance variables on training data.
"""
#PCA fit the dataset
super(PCAVarThreshSelector, self).fit(X)
#Get the cumulative explained variance ratio array (ascending order of cumulative variance explained)
cumulative_EVR = self.explained_variance_ratio_.cumsum()
#Finds the index corresponding to the threshold amount of variance explained
self.indx = self.find_nearest_index(array = cumulative_EVR,
value = self.explained_variance_thresh)
return self
def transform(self, X):
"""
Description
-----------
Selects all the principle components up to the threshold variance.
Args
----
X: DataFrame, (examples, features)
Pandas DataFrame containing training example features
Returns
-------
self: np.ndarray, (examples, indx)
Array containing the minimum number of principle componants required by explained_variance_thresh.
"""
all_components = super(PCAVarThreshSelector, self).transform(X) #To the sklean limit
return all_components[:, :self.indx]
我用我的数据测试了这个类，并且它在一个具有RobustScaler前端的简单管道中按预期工作。在这个简单的管道中，该类将按预期进行拟合和转换。
然后，我将简单的流水线放入带有估算器的另一个流水线中，希望对管道使用.fit（）和.score（）：
model_pipe = Pipeline([('ppp', Pipeline([('rs', RobustScaler()),
('pcavts', PCAVarThreshSelector(whiten = True))])),
('lin_reg', LinearRegression())])
管道安装正确无误。但是，当我尝试对其评分时，会出现AttributeError：
AttributeError Traceback (most recent call last)
<ipython-input-92-cf336db13fe1> in <module>()
----> 1 model_pipe.score(X_test, y_test)
~\Anaconda3\lib\site-packages\sklearn\utils\metaestimators.py in <lambda>(*args, **kwargs)
113
114 # lambda, but not partial, allows help() to work with update_wrapper
--> 115 out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
116 # update the docstring of the returned function
117 update_wrapper(out, self.fn)
~\Anaconda3\lib\site-packages\sklearn\pipeline.py in score(self, X, y, sample_weight)
484 for name, transform in self.steps[:-1]:
485 if transform is not None:
--> 486 Xt = transform.transform(Xt)
487 score_params = {}
488 if sample_weight is not None:
~\Anaconda3\lib\site-packages\sklearn\pipeline.py in _transform(self, X)
424 for name, transform in self.steps:
425 if transform is not None:
--> 426 Xt = transform.transform(Xt)
427 return Xt
428
<ipython-input-88-9153ece48646> in transform(self, X)
114 all_components = super(PCAVarThreshSelector, self).transform(X) #To the sklean limit
115
--> 116 return all_components[:, :self.indx]
117
AttributeError: 'PCAVarThreshSelector' object has no attribute 'indx'
我最初认为这与我在类中如何调用super（）有关。根据{{3}}博客文章，我认为在对管道进行.score（）编辑时，该类正在重新启动，因此在fit方法中创建的属性在评分时不再存在。 我尝试了其他一些调用父类方法的方法，包括：super（）。method，PCA.method（）以及博客文章中建议的方法，但都给出了相同的错误。
我认为，博客的解决方案特定于Python 2，而我的代码则使用Python 3。
但是，当尝试以最低限度可重现此问题的方式重现此错误时，我不再遇到该错误。
from sklearn.datasets import make_regression
from sklearn.base import TransformerMixin, BaseEstimator
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import Pipeline
X, y = make_regression() #Just some dummy regression data for demonstrative purposes.
class BaseTransformer(TransformerMixin, BaseEstimator):
def __init__(self):
print("Base Init")
def fit(self, X, y = None):
return self
def transform(self, X):
return X
class DerivedTransformer(BaseTransformer):
def __init__(self):
super(DerivedTransformer, self).__init__()
print("Dervied init")
def fit(self, X, y = None):
super(DerivedTransformer, self).fit(X, y)
self.new_attribute = 0.0001
return self
def transform(self, X):
output = super(DerivedTransformer, self).transform(X)
output += self.new_attribute
return output
base_pipeline = Pipeline([('base_transformer', BaseTransformer()),
('linear_regressor', LinearRegression())])
derived_pipeline = Pipeline([('derived_transformer', DerivedTransformer()),
('linear_regressor', LinearRegression())])
上面的代码按预期运行，没有错误。我很茫然。谁能帮我解决这个错误？
答案 0 :(得分：0)
那是因为您没有覆盖fit_transform()方法（实现）。
只需将以下部分添加到您的PCAVarThreshSelector即可解决问题：
def fit_transform(self, X, y=None):
return self.fit(X, y).transform(X)
原因：管道将尝试在所有步骤（不包括最后一个步骤）上首先调用fit_transform()方法。
此fit_transform()方法只是调用fit()然后调用transform()的简写，其定义与我上面的定义相同。
但是在某些情况下，例如scikit-learn中的PCA或CountVectorizer等，此方法的实现方式有所不同，以加快处理速度，因为：
fit()中的数据然后在transform()中再次检查数据相比，仅将数据检查/验证（和转换）为适当的形式由于您是从PCA继承的，因此当您调用model_pipe.fit()时，它将使用PCA的fit_transform()，因此永远不会转到您定义的fit()方法（因此，类对象永远不会包含任何indx属性。
但是，当您调用score()时，只会在管道的所有中间步骤上调用transform()并转到已实现的transform()。因此是错误。
如果您在fit_transform()中实现BaseTransformer有所不同，则可以使您有关BaseTransformer和DerivedTransformer的示例可重现。