Question

我使用sqlalchemy连接mysql数据库。我的代码看起来像

engine = sqlalchemy.create_engine(sqlalchemy.engine.url.URL(**url), connect_args={'cursorclass': SSCursor})
conn = engine.connect()

由于需要一次从DB获取大量数据，我使用了SSCursor。我做了一些调查，并创建了一个连接在60秒后丢失的场景。它通过简单的代码模拟，我添加超时（60）。我得到了一个跟踪如下：

Exception during reset or similar
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/pool.py", line 687, in _finalize_fairy
    fairy._reset(pool)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/pool.py", line 829, in _reset
    pool._dialect.do_rollback(self)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/dialects/mysql/base.py", line 1598, in do_rollback
    dbapi_connection.rollback()
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 788, in rollback
    self._execute_command(COMMAND.COM_QUERY, "ROLLBACK")
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1067, in _execute_command
    raise err.InterfaceError("(0, '')")
pymysql.err.InterfaceError: (0, '')
Exception ignored in: <bound method MySQLResult.__del__ of <pymysql.connections.MySQLResult object at 0x7fed3af0d828>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1332, in __del__
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1434, in _finish_unbuffered_query
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 983, in _read_packet
  File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1014, in _read_bytes
AttributeError: 'NoneType' object has no attribute 'settimeout'

我无法弄清楚，如何确保连接在不同的时间后超时。我尝试在connect_timeout中使用connect_args，但行为没有变化。

我应该怎样做才能确保连接？

由于

Answer 1

你应该改变MySQL服务器选项，增加net_read_timeout和max_allowed_packet

的价值

https://dev.mysql.com/doc/refman/5.5/en/error-lost-connection.html

Answer 2

Traceback (most recent call last):
  File "/usr/lib/python3.6/dist-packages/pymysql/connections.py", line 1071, in __del__
  File "/usr/lib/python3.6/dist-packages/pymysql/connections.py", line 1177, in _finish_unbuffered_query
  File "/usr/lib/python3.6/dist-packages/pymysql/connections.py", line 657, in _read_packet
  File "/usr/lib/python3.6/dist-packages/pymysql/connections.py", line 688, in _read_bytes
AttributeError: 'NoneType' object has no attribute 'settimeout'

昨晚我在处理海量数据时遇到了同样的问题。我知道导致问题的原因，那一定是内存/网络问题。因此，我的解决方案是：

捕获异常
睡眠60秒
重试

                try:
                   do_something()
                except pymysql.OperationalError:
                    log.error("error %s: %d %s" % (row["id"], e.args[0], e.args[1]))
                    max_try = 10
                    try_times = 0
                    while con.open is False:
                        if try_times < max_try:
                            try_times += 1
                            time.sleep(60)
                            con.ping()
                    # retry
                    do_something()

如果您有钱，增加存储空间会有所帮助。如果您有时间，请进一步了解mysql，并一次又一次地打开内存和网络。

如果您只是想享受生活，那么请按照程序员的方式进行。这不是最好的解决方案，但是我完成了我的工作。

SSCursor

2 个答案: