如何使这个循环异步?

时间:2017-11-21 17:05:09

标签: python-3.x asynchronous python-asyncio

我现在正在使用Github API,这里有一个函数可以获取列表中每个repo的所有pull请求:

async def get_all_pulls(repos, api):
    pulls = []
    for repo in repos:
        try:
            async for pull in api.getiter(f'/repos/{org}/{repo}/pulls?state=all'):
                pull['repo'] = repo
                if pull not in pulls:
                    pulls.append(pull)
        except Exception:
            print(f"Bad repo/no access=> [{repo}]")
            continue

    return pulls

一切正常,但是一个小问题,需要花费很多时间,因为重复了回购(让他们说有30个)。

我试图让它像这样异步(确定我在使用它时在声明中摆脱for循环):

        # gather all prs for all repos

        tasks = [asyncio.ensure_future(get_all_pulls_for_repo(api, repo)) for repo in repos]
        results = await asyncio.gather(*tasks)
        # unwrap list of lists
        for res in results:
            all_pull_requests += res

但是我遇到了崩溃并说回购是坏的等等。 我想我在这里遗漏了一些重要的东西但却无法得到什么。

为什么它会因异步循环而崩溃?我可以让它发挥作用吗?

UPDATE1 : get_all_reviews上的回溯:

Traceback (most recent call last):
  File "/home/metal/Documents/projects/-git/async_git_tool.py", line 193, in <module>
    loop.run_until_complete(main())
  File "/home/metal/.pyenv/versions/3.6.0/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/home/metal/Documents/projects/-git/async_git_tool.py", line 113, in main
    reviewed = await get_all_reviews(created, api, ss_programmers)
  File "/home/metal/Documents/projects/-git/async_git_tool.py", line 181, in get_all_reviews
    async for review in api.getiter(f'/repos/{org}/{pr_repo}/pulls/{pr_number}/reviews'):
  File "/home/metal/Documents/projects/-git/venv/lib/python3.6/site-packages/gidgethub/abc.py", line 85, in getiter
    data, more = await self._make_request("GET", url, url_vars, b"", accept)
  File "/home/metal/Documents/projects/-git/venv/lib/python3.6/site-packages/gidgethub/abc.py", line 66, in _make_request
    data, self.rate_limit, more = sansio.decipher_response(*response)
  File "/home/metal/Documents/projects/-git/venv/lib/python3.6/site-packages/gidgethub/sansio.py", line 284, in decipher_response
    rate_limit = RateLimit.from_http(headers)
  File "/home/metal/Documents/projects/-git/venv/lib/python3.6/site-packages/gidgethub/sansio.py", line 226, in from_http
    limit = int(headers["x-ratelimit-limit"])
  File "multidict/_multidict.pyx", line 140, in multidict._multidict._Base.__getitem__
  File "multidict/_multidict.pyx", line 135, in multidict._multidict._Base._getone
KeyError: "Key not found: 'x-ratelimit-limit'"

这是功能本身:

    async def get_all_reviews(pulls, api, programmers):
    reviewed_pulls = []
    for pull in pulls:
        pr_repo = pull['repo']
        pr_number = str(pull['number'])

        async for review in api.getiter(f'/repos/{org}/{pr_repo}/pulls/{pr_number}/reviews'):
            if review['user']['login'] not in programmers \
                    and pull not in reviewed_pulls:
                reviewed_pulls.append(pull)

    return reviewed_pulls

我会这样称呼它:

reviewed = await get_all_reviews(softserve_created, api, ss_programmers)

1 个答案:

答案 0 :(得分:1)

你描述的想法对我来说很好:

.csproj's

为请求创建令牌,粘贴它并且您将看到PR网址列表。