使用Django将Unicode子类存储到MySQL

时间:2016-05-31 13:44:14

标签: python mysql django unicode

我在MySQL中保存Unicode字符时遇到了问题。

    Exception Type: UnicodeEncodeError
    Exception Value:    
'ascii' codec can't encode character u'\xed' in position 39: ordinal not in range(128)

    Exception Location: /home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/MySQLdb/connections.py in string_literal, line 204
    Python Executable:  /home/truhlik/.virtualenvs/reality/bin/python
    Python Version: 2.7.11

MySQL库中出现错误:

    def _get_string_literal():
        def string_literal(obj, dummy=None):
            # try:
            return db.string_literal(obj) ...
            # except UnicodeEncodeError:
            #    return db.string_literal(unicode(obj).encode("utf-8"))
        return string_literal

变量:

obj = u'temp/files_widget/2016-05-31-15-00/1/Sn\xedmek obrazovky po\u0159\xedzen\xfd 2016-05-23 10-34-59.png'

我认为问题与此问题有关: python - Problem storing Unicode character to MySQL with Django

我的" obj"变量不是纯Unicode,而是ImagePath类实例。

class ImagePaths(unicode):
    item_class = ImagePath

问题在于我不知道应该采用哪种方法来解决问题。

注意: 我的修复程序在上面的代码中有注释。但它不是很干净的解决方案。它直接写在MySQL库中。

更新#1:

完整追溯:

        /home/truhlik/Dropbox/web/reality/permissions/models.py in save
        super(CustomModel, self).save(*args, **kwargs) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/models/base.py in save
                       force_update=force_update, update_fields=update_fields) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/models/base.py in save_base
            updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/models/base.py in _save_table
                                      forced_update) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/models/base.py in _do_update
        return filtered._update(values) > 0 ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/models/query.py in _update
        return query.get_compiler(self.db).execute_sql(CURSOR) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py in execute_sql
        cursor = super(SQLUpdateCompiler, self).execute_sql(result_type) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py in execute_sql
            cursor.execute(sql, params) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/backends/utils.py in execute
            return super(CursorDebugWrapper, self).execute(sql, params) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/backends/utils.py in execute
                return self.cursor.execute(sql, params) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py in execute
            return self.cursor.execute(query, args) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/MySQLdb/cursors.py in execute
                query = query % tuple([db.literal(item) for item in args]) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/MySQLdb/connections.py in literal
        return self.escape(o, self.encoders) ...

/home/truhlik/.virtualenvs/reality/local/lib/python2.7/site-packages/MySQLdb/connections.py in string_literal
                return db.string_literal(obj) 

使用此连接设置:

'default': {
    'ENGINE': 'django.db.backends.mysql',
    'NAME': 'reality_devel',
    'USER': 'reality_devel',
    'HOST': 'mail.it-poradce.cz',
    'PORT': '3306',
    'OPTIONS': {
        'charset': 'utf8',
        'use_unicode': True,
    }
}

MySQL数据库已配置COLLATION 'utf8_general_ci' DEFAULT CHARACTER SET 'utf8'

发现直接向Django和MySQL报告的几个问题可能与我的问题有关。 http://bugs.mysql.com/bug.php?id=79993https://code.djangoproject.com/ticket/22377 但我不确定。

更新#2:

我正在尝试使用此字段保存图像路径。

images = files_widget.ImagesField(_(u'Obrázky'), blank=True, null=True, help_text=HELP_TEXT_IMAGES)

该领域以这种方式实施:

class FilesField(models.TextField):
    description = _("Files")
    attr_class = controllers.FilePaths

    def __init__(self, *args, **kwargs):
        self.accept = kwargs.pop('accept', None)
        super(FilesField, self).__init__(*args, **kwargs)

    def contribute_to_class(self, cls, name):
        super(FilesField, self).contribute_to_class(cls, name)
        receiver(post_save, sender=cls)(manage_files_on_disk)
        setattr(cls, self.name, controllers.FilesDescriptor(self))

    def save_form_data(self, instance, data):
        save_all_data(self, instance, data)
        super(FilesField, self).save_form_data(instance, data)

    def formfield(self, default_widget=None, **kwargs):
        if not default_widget:
            default_widget = FilesWidget(field=self, accept=self.accept)
        defaults = formfield_defaults(self, default_widget, **kwargs)
        return super(FilesField, self).formfield(**defaults)

class ImagesField(FilesField):
    description = _("Images")
    attr_class = controllers.ImagePaths

    def formfield(self, default_widget=None, **kwargs):
        if not default_widget:
            default_widget = ImagesWidget(field=self, accept=self.accept)
        defaults = formfield_defaults(self, default_widget, **kwargs)
        return super(ImagesField, self).formfield(**defaults)

它正在使用这个应用程序:django-files-widget ...所以如果你需要查看更多代码,那么你可以在GitHub上查看它。 抱歉,无法发布完整的网址。

试图找到我应该在哪里......

  

正确转换为unicode

但不要弄清楚。

更新#3:

添加结果:SHOW VARIABLES LIKE 'char%'

character_set_client - utf8mb4
character_set_connection - utf8mb4
character_set_database - utf8
character_set_filesystem - binary
character_set_results - utf8mb4
character_set_server - latin1
character_set_system - utf8
character_sets_dir - /usr/share/mysql/charsets/

2 个答案:

答案 0 :(得分:0)

看起来你正在向库代码发送一个unicode对象,它希望收到一个str对象。您发布的代码没有太多的上下文(尝试发布整个回溯)但是通过修改MySQLdb库来处理异常并不是正确的方法。您应该在调用库的客户端代码中处理它。找到追溯中的点,该点位于引发异常的行之前,然后在那里移动try: except:块。

可能导致此问题的其他因素是您使用Connection创建了use_unicode = False个对象。

编辑:下面的一些示例代码。由于你没有发布CustomClass的代码,所以我编写了以下简单的类,它的工作正常。

在models.py中:

from __future__ import unicode_literals

from django.db import models

# Create your models here.
class CustomClass(models.Model):
    path = models.CharField(max_length=200)

我使用您拥有的OPTIONS设置我的数据库。然后从shell测试:

>>> c = CustomClass()
>>> c.path = u'temp/files_widget/2016-05-31-15-00/1/Sn\xedmek obrazovky po\u0159\xedzen\xfd 2016-05-23 10-34-59.png'
>>> c.save()
>>>

也许你在代码中的某个地方做了非标准的事情导致了UnicodeEncodeError。可能在您的自定义save()方法或之前的其他客户端代码中。也许你正在使用一个自定义Field子类,它没有正确转换为unicode。

答案 1 :(得分:0)

Hex file_put_contents('logfile.txt', print_r($_REQUEST, true)); file_put_contents('logfile.txt', print_r($_GET, true)); file_put_contents('logfile.txt', print_r($_POST, true)); 不是Unicode,也不是utf8。也许你期待yo?这就是latin1编码为ed

í来自哪里?更改它以编码utf8中的内容或声明您使用的是latin1,而不是utf8。

Python提示:http://mysql.rjweb.org/doc.php/charcoll#python
Django提示:http://mysql.rjweb.org/doc.php/charcoll#other_computer_languages