Question

我正在尝试编写一个函数来对Pandas数据框的指定列（说明，event_name）进行一些文本处理。我写了这段代码：

#removal of unreadable chars, unwanted spaces, words of at most length two from 'description' column and lowercase the 'description' column

def data_preprocessing(source):

    return source.replace('[^A-Za-z]',' ')
    #data['description'] = data['description'].str.replace('\W+',' ')
    return source.lower()
    return source.replace("\s\s+" , " ")
    return source.replace('\s+[a-z]{1,2}(?!\S)',' ')
    return source.replace("\s\s+" , " ")

data['description'] = data['description'].apply(lambda row: data_preprocessing(row))
data['event_name'] = data['event_name'].apply(lambda row: data_preprocessing(row))

出现以下错误：

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-94-cb5ec147833f> in <module>()
----> 1 data['description'] = data['description'].apply(lambda row: data_preprocessing(row))
      2 data['event_name'] = data['event_name'].apply(lambda row: data_preprocessing(row))
      3 
      4 #df['words']=df['words'].apply(lambda row: eliminate_space(row))
      5 

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/pandas/core/series.py in apply(self, func, convert_dtype, args, **kwds)
   2549             else:
   2550                 values = self.asobject
-> 2551                 mapped = lib.map_infer(values, f, convert=convert_dtype)
   2552 
   2553         if len(mapped) and isinstance(mapped[0], Series):

pandas/_libs/src/inference.pyx in pandas._libs.lib.map_infer()

<ipython-input-94-cb5ec147833f> in <lambda>(row)
----> 1 data['description'] = data['description'].apply(lambda row: data_preprocessing(row))
      2 data['event_name'] = data['event_name'].apply(lambda row: data_preprocessing(row))
      data['description'] = data['description'].str.replace('\W+',' ')    
<ipython-input-93-fdfec5f52a06> in data_preprocessing(source)
      3 def data_preprocessing(source):
      4 
----> 5     return source.replace('[^A-Za-z]',' ')
      6     #data['description'] = data['description'].str.replace('\W+',' ')
      7     source = source.lower()

AttributeError: 'float' object has no attribute 'replace'

如果我按以下方式编写代码，但没有功能，则可以正常运行：

data['description'] = data['description'].str.replace('[^A-Za-z]',' ')

Answer 1

要修复的两件事：

第一，当您将apply的lambda函数应用于熊猫系列时，lambda函数将应用于该系列的每个。我认为您需要以向量化的方式将功能应用于整个系列。

第二，您的函数具有多个return语句。结果，只有第一个语句return source.replace('[^A-Za-z]',' ')会运行。您需要做的是对函数内的变量source进行预处理更改，最后在最后返回修改后的source（或中间变量）。

要重写功能以在整个熊猫系列上运行，请用source.替换每次出现的source.str.。新函数定义：

def data_preprocessing(source):
    source = source.str.replace('[^A-Za-z]',' ')
    #data['description'] = data['description'].str.replace('\W+',' ')
    source = source.str.lower()
    source = source.str.replace("\s\s+" , " ")
    source = source.str.replace('\s+[a-z]{1,2}(?!\S)',' ')
    source = source.str.replace("\s\s+" , " ")
    return source

然后，代替这个：

data['description'] = data['description'].apply(lambda row: data_preprocessing(row))
data['event_name'] = data['event_name'].apply(lambda row: data_preprocessing(row))

尝试一下：

data['description'] = data_preprocessing(data['description'])
data['event_name'] = data_preprocessing(data['event_name'])

Python +数据框：AttributeError：'float'对象没有属性'replace'

1 个答案: