Question

我在Python中使用管道分析gapminder数据集[1]但由于某种原因，imputer不会替换nan值。根据文档（“对于编码为np.nan的缺失值，使用字符串值”NaN“。”）我应该像下面这样做，但代码崩溃与“ValueError：输入包含NaN”在行“gm_cv.fit” （X_train，y_train）“。但是gm_cv是基于管道创建的，管道包含应该删除nans的插补。为什么这不起作用？

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import Imputer
from sklearn.pipeline import Pipeline
from sklearn.linear_model import ElasticNet
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import GridSearchCV

fn = 'gapminder.csv' 

df = pd.read_csv(fn, delimiter=',')

# replace empty strings with numpy nans
df.replace('', np.nan, inplace=True)
df.replace(' ', np.nan, inplace=True)

targetVariable = 'lifeexpectancy'

X = df.drop([targetVariable, 'country'], axis=1).values
y = df[targetVariable]

steps = [('imputation', Imputer(missing_values='NaN', strategy='mean', axis=0)),
         ('scaler', StandardScaler()),
         ('elasticnet', ElasticNet())]

pipeline = Pipeline(steps)

# Specify the hyperparameter space
parameters = {'elasticnet__l1_ratio': np.linspace(0,1,30)}

# Create train and test sets
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.4, random_state=98)

# Create the GridSearchCrossValidation object
gm_cv = GridSearchCV(pipeline, parameters, cv=3)

# Fit to the training set
gm_cv.fit(X_train, y_train)

# results:
r2 = gm_cv.score(X_test, y_test)
print("Tuned ElasticNet Alpha: {}".format(gm_cv.best_params_))
print("Tuned ElasticNet R squared: {}".format(r2))

完整错误的堆栈跟踪：

python.exe pipline_and_classification_II.py
Traceback (most recent call last):
  File "pipline_and_classification_II.py", line 55, in <module>
    gm_cv.fit(X_train, y_train)
  File "lib\site-packages\sklearn\model_selection\_search.py", line 639, in fit
    cv.split(X, y, groups)))
  File "lib\site-packages\sklearn\externals\joblib\parallel.py", line 779, in __call__
    while self.dispatch_one_batch(iterator):
  File "lib\site-packages\sklearn\externals\joblib\parallel.py", line 625, in dispatch_one_batch
    self._dispatch(tasks)
  File "lib\site-packages\sklearn\externals\joblib\parallel.py", line 588, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "lib\site-packages\sklearn\externals\joblib\_parallel_backends.py", line 111, in apply_async
    result = ImmediateResult(func)
  File "lib\site-packages\sklearn\externals\joblib\_parallel_backends.py", line 332, in __init__
    self.results = batch()
  File "lib\site-packages\sklearn\externals\joblib\parallel.py", line 131, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "lib\site-packages\sklearn\externals\joblib\parallel.py", line 131, in <listcomp>
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
  File "lib\site-packages\sklearn\model_selection\_validation.py", line 458, in _fit_and_score
    estimator.fit(X_train, y_train, **fit_params)
  File "lib\site-packages\sklearn\pipeline.py", line 250, in fit
    self._final_estimator.fit(Xt, y, **fit_params)
  File "lib\site-packages\sklearn\linear_model\coordinate_descent.py", line 709, in fit
    ensure_2d=False)
  File "lib\site-packages\sklearn\utils\validation.py", line 453, in check_array
    _assert_all_finite(array)
  File "lib\site-packages\sklearn\utils\validation.py", line 44, in _assert_all_finite
    " or a value too large for %r." % X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

Process finished with exit code 1

更新：缓慢调试（没有管道）显示Imputer不喜欢1d数组（如上面的代码中的y）。在使用下面的代码之前手动进行纳米去除时，它可以正常工作。

y = np.array(y)
idx = np.argwhere(np.isnan(y))
y[idx] = np.nanmean(y)

但这违背了管道的目的。任何想法如何在没有人工修补的情况下运行？

[1] http://makemeanalyst.com/download-and-learn-about-gapminder-dataset/

python管道不执行imputer

0 个答案: