python:mysql错误,"字符串值不正确:' \\ xF0 \\ x9F \\ x87 \\ xBA \\ xF0 \\ x9F ...'

时间:2017-01-24 02:14:01

标签: python

python程序,在mysql表中插入数据:

MillisecondsToStartOfTodayFromEpoch + (timeOtherDateInMilliseconds % MilliSecondsPerDay)

运行时,它出错:

is_exist_table_sql = "SHOW TABLES LIKE 'fb_public_figure_posts'"
    if cur.execute(is_exist_table_sql) == 0:
        create_sql = """CREATE TABLE fb_public_figure_posts(Post_ID varchar(32), Permalink varchar(128), Create_time varchar(32), Updated_time varchar(32), Author varchar(32),
        Author_ID bigint, Message text, Link varchar(1024), Likes int, Comments int, pf_ID bigint, foreign key(pf_ID) references fb_public_figure_info(ID))"""
        cur.execute(create_sql)

db_data = posts
if type == "public_figure_posts":
    for item in db_data:
        if "'" in item["message"]:
            item["message"] = str(item["message"]).replace("'","\\\'")
        elif '"' in item["message"]:
            item["message"] = str(item["message"]).replace('"','\\\"')
        is_exist_id_sql = "select * from fb_public_figure_posts where Post_ID = '" + item['id'] + "'"
        if cur.execute(is_exist_id_sql) == 0:
            insert_sql = "INSERT INTO fb_public_figure_posts VALUES ('{0}','{1}','{2}','{3}','{4}',{5},'{6}','{7}',{8},{9},{10})".format(item['id'],item['permalink_url'],item['created_time'],item['updated_time'],item['from']['name'],item['from']['id'],item['message'],item['link'],item['likes']['summary']['total_count'],item['comments']['summary']['total_count'],public_figure_id)
            print(insert_sql)
            cur.execute(insert_sql)

指出句子错误:

pymysql.err.InternalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x87\\xBA\\xF0\\x9F...' for column 'Message' at row 1")

总之,它显示:

 Traceback (most recent call last):
  File "C:/Python/PyCharmProject/FaceBookCrawl/FBCrawl.py", line 286, in <module>
    publicfigure_download.public_figure_posts_storage(public_figure_name)
INSERT INTO fb_public_figure_posts VALUES ('153080620724_10158392447835725','https://www.facebook.com/DonaldTrump/posts/10158392447835725:0','2017-01-01T04:59:07+0000','2017-01-23T19:52:49+0000','Donald J. Trump',153080620724,'‪TO ALL AMERICANS-‬
  File "C:\Python\PyCharmProject\FaceBookCrawl\publicfigure_download.py", line 103, in public_figure_posts_storage
‪#HappyNewYear & many blessings to you all! Looking forward to a wonderful & prosperous 2017 as we work together to #MAGA‬','https://www.facebook.com/DonaldTrump/photos/a.488852220724.393301.153080620724/10158392447835725/?type=3',158710,11045,153080620724)
    mysql_manage().public_figure_db_manage(type, posts, public_figure_id, public_figure_name)
  File "C:\Python\PyCharmProject\FaceBookCrawl\database_manage.py", line 47, in public_figure_db_manage
    cur.execute(insert_sql)
  File "C:\Python\Python36\lib\site-packages\pymysql\cursors.py", line 166, in execute
    result = self._query(query)
  File "C:\Python\Python36\lib\site-packages\pymysql\cursors.py", line 322, in _query
    conn.query(q)
  File "C:\Python\Python36\lib\site-packages\pymysql\connections.py", line 835, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "C:\Python\Python36\lib\site-packages\pymysql\connections.py", line 1019, in _read_query_result
    result.read()
  File "C:\Python\Python36\lib\site-packages\pymysql\connections.py", line 1302, in read
    first_packet = self.connection._read_packet()
  File "C:\Python\Python36\lib\site-packages\pymysql\connections.py", line 981, in _read_packet
    packet.check_error()
  File "C:\Python\Python36\lib\site-packages\pymysql\connections.py", line 393, in check_error
    err.raise_mysql_exception(self._data)
  File "C:\Python\Python36\lib\site-packages\pymysql\err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x87\\xBA\\xF0\\x9F...' for column 'Message' at row 1")

Process finished with exit code 1
你可以帮助我吗

1 个答案:

答案 0 :(得分:0)

你实际上有两个问题。第一个是您尝试插入包含数据库/表/字段编码不支持的字符的字符串 - catch是MySQL's "utf-8" encoding is NOT really utf8 compliant(惊讶,惊讶......)。链接的SO问题给出了完整的解释,您也可以check this for a migration procedure

您的第二个问题 - AFAICT不对您描述的问题负责但 会导致其他一些问题 - 是您未正确使用数据库连接器。这会使你的代码变得无用而且非常脆弱 - 正如你可能已经注意到的那样,正确的转义是一种痛苦,所以最好让db连接器完成工作 - 而且opens your app to SQL injection attacks。这里的解决方法很简单:use prepared statements。这实际上比不使用它们简单得多,并且会一举两得。