Question

因此我尝试使用名为DeepMoji的东西来评分csv充满推文。推文必须以Unicode编码。我已经能够使用一个小数据集，但是我拥有超过200,000点的数据集，我收到此错误： UnicodeDecodeError：'utf8'编解码器无法解码254位的字节0xe2：意外的数据结束。

我尝试过的代码和解决方案如下，但是给出了同样的错误，有没有人有任何想法？

TEST_SENTENCES = []
with open('Cleaned_Data3.csv', 'rU') as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:
        TEST_SENTENCES.append(row["Tweet"])
    try:
        [x.encode('utf-8') for x in TEST_SENTENCES]
    except:
        for rows in TEST_SENTENCES: #attempt to fix the problem 
            str=unicode(str, errors='replace')

这是完整的错误代码。

Traceback (most recent call last):
  File "C:\Users\pjame\Desktop\DeepMoji-master\examples\score_texts_emojis.py", line 24, in <module>
    for row in reader:
  File "C:\Python27\lib\site-packages\unicodecsv\py2.py", line 217, in next
    row = csv.DictReader.next(self)
  File "C:\Python27\lib\csv.py", line 108, in next
    row = self.reader.next()
  File "C:\Python27\lib\site-packages\unicodecsv\py2.py", line 128, in next
    for value in row]
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe2 in position 254: unexpected end of data

从CSV / List读取时的UnicodeDecodeError：意外的数据结束

0 个答案: