Question

我搜索在多线程中转换我的程序，但我没有说怎么做。确实，我想到了2个解决方案，但我认为它不是更优化：选择并在python中将结果存储在数组中，然后按线程的范围进行过滤。或者带有子句id的多线程，以便select通过线程只有一个Domain 有没有更好的方法直接在循环中制作它？

#!/usr/bin/python
import mysql.connector as mariadb
from subprocess import Popen, PIPE, STDOUT
import re
import os

mariadb_connection = mariadb.connect(user='root', password='xxxx', database='xxxx');
cursor = mariadb_connection.cursor(buffered=True)
#retrieving information

cursor.execute("SELECT Domain,Id FROM classement where Domain like '%com';")

for Domain,Id in cursor:
          counter=0
          for character in Domain:
                if (character == "."):
                        counter = counter + 1
                        if (counter==1):
                            print(Domain)
                            pingresult = Popen(['ping','-c','3', Domain], stdout=PIPE, stderr=STDOUT).communicate()[0]
                            pingresult=str(pingresult)
                            ping = pingresult.find('min/avg/max/mdev')
                            ping=pingresult[ping+19:-6]
                            print(ping)
                            cursor.execute('update classement set Ping = "%s" where Domain = "%s";' % (ping,Domain))
                            mariadb_connection.commit()
mariadb_connection.close()

Answer 1

我认为最简单的方法是在函数中包装你的循环并使用multiprocessing.dummy线程池：

from multiprocessing.dummy import Pool
...
cursor.execute( ... )
...

def process (entry):
    Domain,Id = entry
    ...
    # your code there
    ...
    cursor.execute( ... )
    mariadb_connection.commit()

pool = Pool(os.cpu_count())
# or may be N*os.cpu_count() if ping waits for
# network interaction too long

pool.map(process, cursor)
pool.join()

可能是多线程访问数据库所需的额外锁定。

用于执行查询sql

1 个答案: