Question

我在python中有一个来自kaggle.com的项目。我在读取数据集时遇到问题。它有一个 csv文件。我们需要阅读它并将目标和训练部分放在数组中。

以下是前3行数据集（目标列是第19列，功能是前18列）：

user    gender  age how_tall_in_meters  weight  body_mass_index x1  
debora  Woman   46  1.62    75  28.6    -3  
debora  Woman   46  1.62    75  28.6    -3

此处未显示的目标列具有字符串值。

from pandas import read_csv
import numpy as np
from sklearn.linear_model.stochastic_gradient import SGDClassifier
from sklearn import preprocessing
import sklearn.metrics as metrics
from sklearn.cross_validation import train_test_split

#d = pd.read_csv("data.csv", dtype={'A': np.str(), 'B': np.str(), 'S': np.str()})

dataset = np.genfromtxt(open('data.csv','r'), delimiter=',', dtype='f8')[1:]
target = np.array([x[19] for x in dataset])
train = np.array([x[1:] for x in dataset])

print(target)

我得到的错误是：

Traceback (most recent call last):
  File "C:\Users\Cameron\Desktop\Project - Machine learning\datafilesforproj\SGD_classifier.py", line 12, in <module>
    dataset = np.genfromtxt(open('data.csv','r'), delimiter=',', dtype='f8')[1:]
  File "C:\Python33\lib\site-packages\numpy\lib\npyio.py", line 1380, in genfromtxt
    first_values = split_line(first_line)
  File "C:\Python33\lib\site-packages\numpy\lib\_iotools.py", line 217, in _delimited_splitter
    line = line.split(self.comments)[0]
TypeError: Can't convert 'bytes' object to str implicitly

Answer 1

对我有用的是改变线

dataset = np.genfromtxt(open('data.csv','r'), delimiter=',', dtype='f8')[1:]

到

dataset = np.genfromtxt('data.csv', delimiter=',', dtype='f8')[1:]

（不幸的是，我不太确定底层问题是什么）

Answer 2

这实际上是numpy中的一个错误，参见issue #3184

我只想复制我在那里提出的解决方法：

import functools
import io
import numpy as np
import sys

genfromtxt_old = np.genfromtxt
@functools.wraps(genfromtxt_old)
def genfromtxt_py3_fixed(f, encoding="utf-8", *args, **kwargs):
  if isinstance(f, io.TextIOBase):
    if hasattr(f, "buffer") and hasattr(f.buffer, "raw") and \
    isinstance(f.buffer.raw, io.FileIO):
      # Best case: get underlying FileIO stream (binary!) and use that
      fb = f.buffer.raw
      # Reset cursor on the underlying object to match that on wrapper
      fb.seek(f.tell())
      result = genfromtxt_old(fb, *args, **kwargs)
      # Reset cursor on wrapper to match that of the underlying object
      f.seek(fb.tell())
    else:
      # Not very good but works: Put entire contents into BytesIO object,
      # otherwise same ideas as above
      old_cursor_pos = f.tell()
      fb = io.BytesIO(bytes(f.read(), encoding=encoding))
      result = genfromtxt_old(fb, *args, **kwargs)
      f.seek(old_cursor_pos + fb.tell())
  else:
    result = genfromtxt_old(f, *args, **kwargs)
  return result

if sys.version_info >= (3,):
  np.genfromtxt = genfromtxt_py3_fixed

将它放在代码的顶部后，您可以再次使用np.genfromtxt，它在Python 3中可以正常工作。

Answer 3

根据https://mail.python.org/pipermail/python-list/2012-April/622487.html 你可能需要

import io
import sys
inpstream = io.open('data.csv','rb')
dataset = np.genfromtxt(inpstream, delimiter=',', dtype='f8')[1:]

在http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html中显示的示例中用作文件的对象属于类StringIO。不过，从函数的规范来看，我猜传递文件名应该可行。

Answer 4

您需要明确地将bytes对象解码为str对象，正如TypeError所暗示的那样。

# For instance, interpret as UTF-8 (depends on your source)
self.comments = self.comments.decode('utf-8')

Answer 5

而不是：

dataset = np.genfromtxt(open('data.csv','r'), delimiter=',', dtype='f8')[1:]

试试这个：

dataset = np.genfromtxt('C:\\\\..\\\\..\\\train.csv', delimiter=',', dtype='None')[1:]

请注意，您必须使用额外的＆＃39; \＆＃39;逃避另一个。

使用numpy.genfromtxt会产生TypeError：无法转换＆＃39;字节＆＃39;隐含地反对str

5 个答案: