Python-多处理池映射返回不会出现泡菜错误

时间:2018-09-04 13:16:06

标签: python-3.x multiprocessing

我有以下代码创建一个testrail客户端并执行testrail的GET_SUITES API调用。

我有一个调用GET_SUITES API的函数,并且我将testrail客户端和test_rail_project_id传递为参数

我正在尝试使用多处理功能在我的项目列表上执行,以加快处理速度,但我不会出现泡菜错误

我的代码:

from itertools import product

def get_suites(client, project_id):
    try:
        path = 'get_suites/{projectid}'.format(projectid=project_id)
        test_rail_response = client.send_get(path)
        return test_rail_response
    except Exception as e:
        raise Exception(str(e))

if __name__ == "__main__":
    testRailClient = APIClient(TESTRAIL_URL)
    pool = Pool(2)
    all_project_ids = [100, 200, 300]
    data = pool.starmap(get_suites, product([testRailClient], all_project_ids))

错误堆栈:

Traceback (most recent call last):
  File "main.py", line 57, in <module>
    data = pool.starmap(testrailapi.get_suites, product([testRailClient], all_project_ids))
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 274, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 424, in _handle_tasks
    put(task)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: can't pickle SSLContext objects

有什么建议吗?

谢谢

PS:我正在使用Python3.6

更新: 如建议的那样,我尝试删除API客户端作为参数,并且可以正常工作,但是当我使用“ get_suites”作为方法时,却遇到了相同的错误。请在下面查看我的更新代码

class TestRailExecution:

    def __init__(self, url, username, password):
        self.url = url
        self.username = username
        self.password = password
        self.client = APIClient(self.url)
        self.client.user = username
        self.client.password = password

    def get_suites(self, project_id):
        try:
            path = 'get_suites/{projectid}'.format(projectid=project_id)
            test_rail_response = self.client.send_get(path)
            return test_rail_response
        except Exception as e:
            raise Exception(str(e))

if __name__ == "__main__":
    testRailClient = TestRailExecution(TESTRAIL_URL, user, password)
    pool = Pool(2)
    data = pool.map(get_suites, [100, 200, 300])

0 个答案:

没有答案