Question

我正在尝试将包含DataFrame文本的列转换为一个热编码矩阵。这个工作正常，但由于我不知道的原因而停止工作。消息说：＆＃34; TypeError：＆＃39;＆gt;＆＃39; ＆＃39; str＆＃39;的实例之间不支持并且＆＃39;浮动＆＃39;＆＃34;对我来说，这似乎是无稽之谈，因为我只使用tekst数据。当我用一个小数据集重复实验时，LabelBinarizer可以正常工作并产生所需的输出。

我注意到X_train数据帧的大小为4.6 GB。我的机器只有8 GB。我应该注意一些内存限制吗？所有的数字都相当小，我应该转换为int32和float32吗？

我可以重现以下错误。但我不确定这是否提供了足够的信息。

from sklearn.preprocessing import LabelBinarizer

lb=LabelBinarizer()

s=['a','b','c','b','a']

df=pd.DataFrame (s)

df = pd.Series (s)

dd = X_train['state']

type(dd)
Out[9]: pandas.core.series.Series

type(df)
Out[10]: pandas.core.series.Series

lb.fit(dd)
Traceback (most recent call last):

  File "<ipython-input-11-5ec245111e31>", line 1, in <module>
    lb.fit(dd)

  File "C:\packages\Anaconda3\lib\site-packages\sklearn\preprocessing\label.py", line 296, in fit
    self.y_type_ = type_of_target(y)

  File "C:\packages\Anaconda3\lib\site-packages\sklearn\utils\multiclass.py", line 275, in type_of_target
    if (len(np.unique(y)) > 2) or (y.ndim >= 2 and len(y[0]) > 1):

  File "C:\packages\Anaconda3\lib\site-packages\numpy\lib\arraysetops.py", line 214, in unique
    ar.sort()

TypeError: '>' not supported between instances of 'str' and 'float'


lb.fit(df)
Out[12]: LabelBinarizer(neg_label=0, pos_label=1, sparse_output=False)

df.value_counts()
Out[13]: 
a    2
b    2
c    1
dtype: int64

dd.value_counts()
Out[14]: 
MI    228601
CA      5020
TX      2420
FL      2237
IL      1310
SC      1304
OH       967
NY       673
MN       632
GA       535
NV       484
UT       477
PA       466
NJ       395
VA       385
NC       353
MD       349
AZ       329
ME       261
OK       248
AL       215
TN       207
WA       192
MA       182
IA       159
WI       159
OR       153
MO       151
CO       147
KY       146
IN       106
AR        82
LA        81
AK        79
UK        77
NB        77
MS        64
CT        60
DC        58
ON        51
DE        50
KS        37
RI        35
SD        33
ID        33
MT        28
NM        21
BC        17
WY        12
HI        10
NH         9
VT         7
VI         6
WV         6
PR         5
QC         5
QL         3
ND         2
BL         2
Name: state, dtype: int64

len(df)
Out[15]: 5

len(dd)
Out[16]: 250306

Answer 1

也许其输入数据可能包含缺失值。

@media (max-width: 767px) {
    #nameinput {width:100%;}
}
@media (min-width:1025px){
    #nameinput{  width:300pt;}             
}

如果没有缺失值，则其工作原理如下。

from sklearn.preprocessing import LabelBinarizer
import numpy as np
import pandas as pd

lb = LabelBinarizer()

s = ['a','b','c','b','a', np.nan]
df = pd.DataFrame(s, columns=["state"])

df_binarized = lb.fit_transform(df['state'])
df_binarized

Traceback (most recent call last):
  File "/home/kuroyanagi/.pyenv/versions/anaconda3-4.4.0/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-45-f16e01b4e1be>", line 4, in <module>
    df_binarized = lb.fit_transform(df['state'])
  File "/home/kuroyanagi/.pyenv/versions/anaconda3-4.4.0/lib/python3.6/site-packages/sklearn/base.py", line 494, in fit_transform
    return self.fit(X, **fit_params).transform(X)
  File "/home/kuroyanagi/.pyenv/versions/anaconda3-4.4.0/lib/python3.6/site-packages/sklearn/preprocessing/label.py", line 296, in fit
    self.y_type_ = type_of_target(y)
  File "/home/kuroyanagi/.pyenv/versions/anaconda3-4.4.0/lib/python3.6/site-packages/sklearn/utils/multiclass.py", line 275, in type_of_target
    if (len(np.unique(y)) > 2) or (y.ndim >= 2 and len(y[0]) > 1):
  File "/home/kuroyanagi/.pyenv/versions/anaconda3-4.4.0/lib/python3.6/site-packages/numpy/lib/arraysetops.py", line 210, in unique
    return _unique1d(ar, return_index, return_inverse, return_counts)
  File "/home/kuroyanagi/.pyenv/versions/anaconda3-4.4.0/lib/python3.6/site-packages/numpy/lib/arraysetops.py", line 277, in _unique1d
ar.sort()
TypeError: '<' not supported between instances of 'float' and 'str'

由于NaN，LabelBinarizer表现不一致

1 个答案: