Question

尝试重用从Mac到Windows的有效Python代码。代码使用gzip压缩utf8字符串，并使用SQLAlchemy将输出插入blob。

但是插入后我收到以下错误：

        UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 15: character maps to <undefined>

相关部分：

        from sqlalchemy import *
        import zlib

        pcaxis_table = Table('pcaxis_data', meta, autoload=True, autoload_with=engine)

        try:
            response = urllib2.urlretrieve(url_source)
        except Exception as e:
            print url_source
            raise e

        infile = response.read()
        px_file = infile.decode('cp1252').encode('utf-8')

        cmpstr = zlib.compress(px_file)


        #out = StringIO.StringIO()
        #with gzip.GzipFile(fileobj=out, mode="w") as f:
        #    f.write(px_file)

        ins = pcaxis_table.insert(values = {'TableSQL':tableSQL,
                                            'zip_file':cmpstr, #out.getvalue()
                                            })
        ins.execute()

跟踪...（尝试将blob解码为cp1252时失败）

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Anaconda\Lib\site-packages\sqlalchemy\sql\base.py", line 386, in execute
    return e._execute_clauseelement(self, multiparams, params)
  File "C:\Anaconda\Lib\site-packages\sqlalchemy\engine\base.py", line 1758, in _execute_clauseelement
    return connection._execute_clauseelement(elem, multiparams, params)
  File "C:\Anaconda\Lib\site-packages\sqlalchemy\engine\base.py", line 826, in _execute_clauseelement
    compiled_sql, distilled_params
  File "C:\Anaconda\Lib\site-packages\sqlalchemy\engine\base.py", line 958, in _execute_context
    context)
  File "C:\Anaconda\Lib\site-packages\sqlalchemy\engine\base.py", line 1162, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "C:\Anaconda\Lib\site-packages\sqlalchemy\engine\base.py", line 951, in _execute_context
    context)
  File "C:\Anaconda\Lib\site-packages\sqlalchemy\engine\default.py", line 436, in do_execute
    cursor.execute(statement, parameters)
  File "C:\Anaconda\Lib\site-packages\pymysql\cursors.py", line 100, in execute
    query = query % escaped_args
  File "C:\Anaconda\lib\encodings\cp1252.py", line 15, in decode
    return codecs.charmap_decode(input,errors,decoding_table)
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 6: character maps to <undefined>

和MySQL表：

create table pcaxis_data(
    id int NOT NULL AUTO_INCREMENT,
    TableSQL varchar(25),
    zip_file BLOB,
    inserttime TIMESTAMP,
    PRIMARY KEY (id)
);

Answer 1

问题在于.decode('cp1252')。 Windows-1252代码页不使用所有字节（因此例如字节8f未使用且无法解码）。您可以改为使用latin1。

response实际上是Windows-1252文字吗？如果不是这样，那么解码它是没有意义的。

zlib.compress接受一个bytestring参数，response是一个bytestring，你可以直接压缩它，而不需要重新编码。

Answer 2

刚刚解决了这个问题。怎么样？将pymysql从大约0.6.0升级到0.6.3。出了什么问题？ pymysql驱动程序尝试通过转换为unicode来转义二进制数据。字节\ x08不使用UTF8或Latin1映射到unicode。这就是为什么失败了。

sqlalchemy insert blob引发UnicodeDecodeError

2 个答案: