Question

我有1 GB的csv文件，我无法读取该日志文件，并且在我的csv文件中的python和pandas代码中都给出了相同的错误，它的值不止一个列，因为只有一个列值而且我所有的CSV值都是数字

with open("/Users/kiya/sep_sent.csv", encoding='utf-8') as f:
for i in f: 
   print(i.strip())

另一种方法：

with open("/Users/kiya/sep_sent.csv",encoding='cp1252') as f:
    for i in f:
      print(i.strip())

Traceback (most recent call last):
  File "/Users/kiya/test8.py", line 5, in <module>
    for i in f:
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/encodings/cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 159: character maps to <undefined>

熊猫代码：

import pandas as pd
df = pd.read_csv("/Users/kiya/sep_sent.csv", encoding="utf-8")
print(df)

我的csv值如下：

错误：

0
0
0
0
0
Traceback (most recent call last):
  File "/Users/kiya//test8.py", line 4, in <module>
    for i in f:
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 52: invalid start byte

Answer 1

您也可以将encoding参数传递给read_csv

df = pd.read_csv("/Users/kiya/sep_sent.csv", encoding="utf-8")

Answer 2

使用utf-8编码打开文件，它应该可以工作：

with open("/Users/kiya/sep_sent.csv", encoding='utf-8') as f:
    for i in f: 
       print(i.strip())

在python中加载大型csv文件“'utf-8'编解码器无法解密”时出错

2 个答案: