我尝试使用并行和单独的会话每隔X秒访问一个网站,然后分析响应中的内容以查看每个会话是否应该继续。但是,一旦代码到达第二个循环,它就会失败。
import asyncio
from aiohttp import ClientSession
import logging
import time
interval = 30
instances = 2
visit_url = 'http://www.example.org'
tasks = []
logging.basicConfig(
format='%(asctime)s.%(msecs)03d %(message)s', # Log in format time.milliseconds {message}
level=logging.INFO, # Use with logging.info()
datefmt='%H:%M:%S') # Display time as Hours:Minutes:Seconds
class StopException(Exception):
pass
async def quit_app(session, task_, reason):
logging.info("[{}] {}.".format(task_, reason))
session.cookies.clear() # Reset cookies
session.headers.clear() # Reset headers
session.close() # End HTTP connection
raise StopException
async def get_status(response):
if "abow" in response:
return "success"
elif "odoap" or "daoscp" in response:
return "waiting"
elif "nullt" in response:
return "fail"
elif "issue" in response:
return "banned"
elif "pending" in response:
return "pending"
else:
return "pending"
async def initialise(headers):
session = ClientSession()
task_ = len(asyncio.Task.all_tasks()) - instances - 1
passed = False
while passed is False:
async with session as session:
async with session.get(visit_url, headers=headers, allow_redirects=True) as initial:
status = await get_status(await initial.text()) # Check HTML for status
if status == "success":
logging.info("[{}] {}.".format(task_, "Success"))
passed = True
elif status == "pending":
logging.info("[{}] {}.".format(task_, "Pending.."))
await asyncio.sleep(interval)
elif status == "waiting":
logging.info("[{}] {}.".format(task_, "Waiting..."))
await asyncio.sleep(interval)
elif status == "banned":
await quit_app(initial, task_, "Banned")
elif status == "fail":
await quit_app(initial, task_, "Failed")
if __name__ == "__main__":
headers = {
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.9',
'upgrade-insecure-asks': '1',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36'
} # Add appropriate headers
start = time.clock()
loop = asyncio.get_event_loop()
for i in range(instances):
task = asyncio.ensure_future(initialise(headers))
tasks.append(task)
loop.run_until_complete(asyncio.wait(tasks))
end = time.clock()
print("Process took {0:.4f} seconds.".format(end - start))
此代码返回以下错误:
13:56:58.604从未检索到任务异常 future:任务完成coro = initialise()完成,定义于 C:/Users/x/PycharmProjects/tests/src/aiohttp_main.py:49 exception = RuntimeError('会话已关闭',) RuntimeError:会话已关闭
我无法解决如何让会话保持活力,直到我.close()他们......
答案 0 :(得分:3)
我无法解决如何使会话保持活力直到我.close()他们......
async with
是关闭会话的请求。这意味着在你写完之后:
async with session as session:
...一旦session
的主体执行完毕,您将无法再使用async with
。这不是aiohttp特有的,它是with
在Python中的工作方式。例如,在处理文件时,with
是关闭文件的请求:
with open('data.csv') as fileobj:
# ... read stuff from fileobj
# outside the "with" block, fileobj is closed and you
# can no longer read from it
修复很简单,只需将with
移到while
循环之外。例如:
async def initialise(headers):
async with ClientSession() as session:
# the rest of the code, including the `while` loop, here
在不相关的说明中,您可能希望将len(asyncio.Task.all_tasks())
替换为您自己的全局计数器。如果您稍后将其他不相关的任务合并到事件循环中(或者第三方库为您执行此操作),则以这种方式使用Task.all_tasks()
可能会开始产生不正确的结果。