python异步发布请求

时间:2018-08-06 02:19:18

标签: python python-requests python-asyncio aiohttp

我想知道是否有任何方法可以使此脚本更快,例如立即创建1000个帐户,或者至少在几秒钟内创建一个帐户。 我已经尝试过自己做一些异步工作,但是据我所知,我只是异步编程的初学者,所以能提供帮助。

import asyncio
import aiohttp


async def make_numbers(numbers, _numbers):
    for i in range(numbers, _numbers):
        yield i

async def make_account():
   url = "https://example.com/sign_up.php"
   async with aiohttp.ClientSession() as session:
          async for x in make_numbers(35691, 5000000):
              async with  session.post(url, data ={
                    "terms": 1,
                    "captcha": 1,
                    "email": "user%s@hotmail.com" % str(x),
                    "full_name": "user%s" % str(x),
                    "password": "123456",
                    "username": "auser%s" % str(x)
              }) as response:
                    data = await response.text()
                    print("-> Creating account number %d" % x)
                    print (data)

loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(make_account())
finally:
    loop.close()

2 个答案:

答案 0 :(得分:3)

问题中的代码将执行一系列的所有POST请求,这使得该代码不会比在单个线程中使用requests的速度更快。但是与requests不同,asyncio使在同一线程中并行化它们变得容易:

async def make_account():
    url = "https://example.com/sign_up.php"
    async with aiohttp.ClientSession() as session:
        post_tasks = []
        # prepare the coroutines that poat
        async for x in make_numbers(35691, 5000000):
            post_tasks.append(do_post(session, url, x))
        # now execute them all at once
        await asyncio.gather(*post_tasks)

async def do_post(session, url, x):
    async with  session.post(url, data ={
                "terms": 1,
                "captcha": 1,
                "email": "user%s@hotmail.com" % str(x),
                "full_name": "user%s" % str(x),
                "password": "123456",
                "username": "auser%s" % str(x)
          }) as response:
          data = await response.text()
          print("-> Created account number %d" % x)
          print (data)

以上代码将尝试一次发送所有POST请求。尽管有此意图,但aiohttp.ClientSession的TCP连接器将节制它,默认情况下最多允许100个同时连接。要增加或取消此限制,必须在会话上设置custom connector

答案 1 :(得分:0)

您发布的异步代码对我来说没问题。您可以通过将asyncio与多线程/多进程相结合来加快速度。

但是还有其他限制,使您无法在一秒钟内创建1000个帐户。例如,网络速度,并发连接,速率限制,服务器端的数据库IOPS。