' UTF8'在将查询结果写入csv时,编解码器无法解码字节0x92

时间:2017-11-29 20:14:01

标签: python utf-8 python-unicode

我正在从谷歌工作表中读取一个查询文本,该文章正在传递为" str1"下面。这是我的代码:

# get query string from google sheets
# establish database connection
cursor = conn.cursor()
cursor.execute((str1))
results1 = cursor.fetchall()
cursor.close()
from openpyxl import Workbook
wb = Workbook()
ws = wb.active
for row in results1:
    ws.append(row)

此时我收到错误,如标题所示:

  File "<stdin>", line 2, in <module>
  File "/Library/Python/2.7/site-packages/openpyxl/worksheet/worksheet.py", line 790, in append
    cell = Cell(self, row=row_idx, col_idx=col_idx, value=content)
  File "/Library/Python/2.7/site-packages/openpyxl/cell/cell.py", line 114, in __init__
    self.value = value
   File "/Library/Python/2.7/site-packages/openpyxl/cell/cell.py", line 294, in value
    self._bind_value(value)
  File "/Library/Python/2.7/site-packages/openpyxl/cell/cell.py", line 191, in _bind_value
    value = self.check_string(value)
  File "/Library/Python/2.7/site-packages/openpyxl/cell/cell.py", line 150, in check_string
    value = unicode(value, self.encoding)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 48: invalid start byte

数据与作者/文章信息有关(我们是出版商)。它包含内容ID,网站代码,署名,作者,Facebook广告链接和提取日期/时间

以下是一行数据的示例,其中包含错误:

(1693279, 'CPD', 'Morgan Dietrich', "20 Intuitive People Share Their 'Something Doesn\x92t Feel Right' Story That Turned Out To Be True", 'business.facebook.com/550634765042035/posts/…;, datetime.datetime(2017, 11, 29, 20, 49, 24))

我已经阅读了很多关于此错误的问题,但无法找出解决方案。查询结果(results1) - 成功运行 - 是这样的元组:

( (query result ro1/col1, query result ro1/col2, query result ro1/col3), 
  (query result ro2/col1, query result ro2/col2, query result ro2/col3), ... etc... )

我尝试过.encode / .decode,但他们似乎并没有在元组上工作。我已经尝试过滤掉那些不好的角色,但那并没有起作用。

我如何在世界范围内解决此问题?这些与utf8相关的错误在过去引起了我很大的烦恼,尽管我读到这一切仍然显得相当令人困惑。

失败的行是

(1693279,
 'CPD',
 'Morgan Dietrich',
 "20 Intuitive People Share Their 'Something Doesn\x92t Feel Right' Story That Turned Out To Be True",
 'https://business.facebook.com/550634765042035/posts/1223000787805426',
 datetime.datetime(2017, 11, 29, 20, 49, 24))

1 个答案:

答案 0 :(得分:0)

您有字节串数据,其中包含非ASCII和非UTF8数据。您要么必须将数据库配置为为第4列返回Unicode字符串,要么手动解码。

如果您将数据解码为Windows代码页1252(或同一范围内的其他一些Windows代码页,但1252最可能是英文文本),则0x92字节为U+2019 RIGHT SINGLE QUOTATION MARK代码点,因此您可以尝试一下:

for row in results1:
    row = list(row)  # convert to list for easier mutation
    row[3] = row[3].decode('cp1252')
    ws.append(row)