我有多处理的应用程序,每个线程执行以下python代码:
while True:
db_cursor.execute("SET autocommit=0;")
db_cursor.execute("LOCK TABLES prqueue WRITE;")
db_cursor.execute("SELECT MAX(id) FROM prqueue;")
for record in db_cursor.fetchall():
if record[0]:
db_cursor.execute("DELETE FROM prqueue WHERE id='%s';" % record[0]);
#db_connector.commit()
db_cursor.execute("COMMIT;")
db_cursor.execute("UNLOCK TABLES;")
# some work that takes time
else:
db_cursor.execute("COMMIT;")
db_cursor.execute("UNLOCK TABLES;")
有时我的进程因OperationalError异常而死:'Deadlock found when trying to get lock; try restarting transaction'
。
这里最奇怪的是流程在不同的行上死亡。例如,来自其中3个的错误消息:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/crawler/Worker.py", line 122, in do_crawling
db_cursor.execute("COMMIT;")
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (1213, 'Deadlock found when trying to get lock; try restarting transaction')
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/crawler/Worker.py", line 122, in do_crawling
db_cursor.execute("COMMIT;")
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (1213, 'Deadlock found when trying to get lock; try restarting transaction')
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/crawler/Worker.py", line 109, in do_crawling
db_cursor.execute("SELECT MAX(id) FROM prqueue;")
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (1213, 'Deadlock found when trying to get lock; try restarting transaction')
我在这段代码中做错了什么?完全厌倦了所有这些僵局。
谢谢。