Question

我可以通过Python csv模块将MySQL表导出为CSV文件，但是没有utf-8字符。（示例：???? chars insted ąöę。

表数据采用utf-8格式（phpMyAdmin让我看到正确的数据）。

我发现一些信息，在Python中，所有数据都应该在utf-8中解码，然后通过例如unicodewritter在utf-8中编码为CSV（因为本机csv模块不支持Unicode正确）。

我尝试了很多但没有成功。

问题：是否有任何示例脚本将utf-8中的MySQL数据库导出为Python格式的utf-8格式的CSV文件？

我使用ubuntu 14.04并且mysql.connector存在问题所以我使用MySQLdb和Gord Thompson代码：

# -*- coding: utf-8 -*-
import csv
import MySQLdb
from UnicodeSupportForCsv import UnicodeWriter
import sys
reload(sys)  
sys.setdefaultencoding('utf8')
#sys.setdefaultencoding('Cp1252')

conn = MySQLdb.Connection(db='sampledb', host='localhost',           
user='sampleuser', passwd='samplepass')

crsr = conn.cursor()
crsr.execute("SELECT * FROM rfid")
with open(r'test.csv', 'wb') as csvfile:
    uw = UnicodeWriter(
    csvfile, delimiter=',',
    quotechar='"', quoting=csv.QUOTE_MINIMAL)
for row in crsr.fetchall():
    uw.writerow([unicode(col) for col in row])

错误仍然存在：UnicodeDecodeError：'utf8'编解码器无法解码位置2中的字节0xf3：无效的连续字节

Answer 1

MySQL非常适合转换字符集。但您需要告诉它使用正确的排序规则建立连接。

默认情况下，它返回将其放入数据库的方式。将所需的字符集添加到连接：

conn = MySQLdb.Connection(db='sampledb', host='localhost',           
user='sampleuser', passwd='samplepass', charset='utf-8', )

这有用吗？

Answer 2

这适用于Python 2.7.5和MySQL Connector / Python 2.0.4：

# -*- coding: utf-8 -*-
import csv
import mysql.connector
from UnicodeSupportForCsv import UnicodeWriter

conn = mysql.connector.connect(
    host='localhost', port=3307,
    user='root', password='whatever',
    database='mydb')
crsr = conn.cursor()
crsr.execute("SELECT * FROM vocabulary")
with open(r'C:\Users\Gord\Desktop\test.csv', 'wb') as csvfile:
    uw = UnicodeWriter(
        csvfile, delimiter=',',
        quotechar='"', quoting=csv.QUOTE_MINIMAL)
    for row in crsr.fetchall():
        uw.writerow([unicode(col) for col in row])

UnicodeWriter类直接来自documentation page for the csv module上的最后一个示例，我将其存储在一个名为＆＃34; UnicodeSupportForCsv.py＆＃34;的文件中：

import csv, codecs, cStringIO

class UTF8Recoder:
    """
    Iterator that reads an encoded stream and reencodes the input to UTF-8
    """
    def __init__(self, f, encoding):
        self.reader = codecs.getreader(encoding)(f)

    def __iter__(self):
        return self

    def next(self):
        return self.reader.next().encode("utf-8")

class UnicodeReader:
    """
    A CSV reader which will iterate over lines in the CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        f = UTF8Recoder(f, encoding)
        self.reader = csv.reader(f, dialect=dialect, **kwds)

    def next(self):
        row = self.reader.next()
        return [unicode(s, "utf-8") for s in row]

    def __iter__(self):
        return self

class UnicodeWriter:
    """
    A CSV writer which will write rows to CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        # Redirect output to a queue
        self.queue = cStringIO.StringIO()
        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
        self.stream = f
        self.encoder = codecs.getincrementalencoder(encoding)()

    def writerow(self, row):
        self.writer.writerow([s.encode("utf-8") for s in row])
        # Fetch UTF-8 output from the queue ...
        data = self.queue.getvalue()
        data = data.decode("utf-8")
        # ... and reencode it into the target encoding
        data = self.encoder.encode(data)
        # write to the target stream
        self.stream.write(data)
        # empty queue
        self.queue.truncate(0)

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)

Answer 3

试试这个......让你轻松

https://github.com/jdunck/python-unicodecsv

unicodecsv是Python 2.7的csv模块的替代品，它支持unicode字符串而不会有麻烦。支持的版本是python 2.6,2.7,3.3,3.4,3.5和pypy 2.4.0。

>>> import unicodecsv as csv
>>> from io import BytesIO
>>> f = BytesIO()
>>> w = csv.writer(f, encoding='utf-8')
>>> _ = w.writerow((u'é', u'ñ'))
>>> _ = f.seek(0)
>>> r = csv.reader(f, encoding='utf-8')
>>> next(r) == [u'é', u'ñ']
True

Answer 4

最终它有效！感谢： Gord Thompson 和 Prikkeldraad 。谢谢伙计们！

MySQL通过Python脚本以UTF-8的形式导出到CSV文件

4 个答案: