Question

从How to use threading in Python?我拿了这段代码示例：

from multiprocessing.dummy import Pool as ThreadPool 
pool = ThreadPool(4) 
results = pool.map(my_function, my_array)

它完全适用于这样的urllib.request.urlopen函数并且提供了大约2-3倍的速度提升

from urllib.request import urlopen
from multiprocessing.dummy import Pool as ThreadPool
from timeit import Timer

urls = [
  'http://www.python.org', 
  'http://www.python.org/about/',
  'http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html',
  'http://www.python.org/doc/',
  'http://www.python.org/download/',
  'http://www.python.org/getit/',
  'http://www.python.org/community/',
]

def mult(urls):
    pool = ThreadPool(8) 
    results = pool.map(urlopen, urls)
    pool.close()
    pool.join()

def single(urls):
    [urlopen(url) for url in urls]

print(Timer(lambda: single(urls)).timeit(number=1))
print(Timer(lambda: mult(urls)).timeit(number=1))

但是在调用DB程序的情况下，我没有注意到多处理提供的任何加速

from multiprocessing.dummy import Pool as ThreadPool
import cx_Oracle as ora
import configparser
config = configparser.ConfigParser()
config.read('configuration.ini')
conf = config['sample_config']

dsn = ora.makedsn(conf['ip'], conf['port'], sid=conf['sid'])
connection = ora.Connection(user=conf['user'], password=conf['password'], dsn=dsn, threaded=True)
cursor = ora.Cursor(connection)

def test_function(params):
    cursor = p.connection.cursor()
    # call procedure
    cursor.callproc('Sample_PKG.my_procedure', keywordParameters=params)

dicts = [{'a': 'b'}, {'a': 'c'}] # a lot of dictionaries contains about 30 items each

pool = ThreadPool(4)
pool.map(test_function, dicts)
pool.close()
pool.join()

那么，为什么会这样呢？什么可以成为推动脚本工作的解决方案？

UPD 试图使用会话池。此示例代码正在运行

db_pool = ora.SessionPool(user=conf['user'], password=conf['password'], dsn=dsn, min=1, max=4, increment=1, threaded=True)
connection = ora.Connection(dsn=dsn, pool=db_pool)
cursor = connection.cursor()
cursor.execute("select 1 from dual")
result = cursor.fetchall()
cursor.close()
db_pool.release(connection)

但是当我替换

cursor.execute("select 1 from dual")

与

cursor.callproc('Sample_PKG.my_procedure', keywordParameters=params)

我让我的控制台挂起。我做错了吗？

Answer 1

每个Oracle连接一次只能执行一个语句。尝试使用会话池并为每个语句使用不同的连接（并希望您的DBA不介意额外的连接）。

如何使用cx_Oracle的Python多处理获得加速？

1 个答案: