Question

我想使用协同程序来抓取和解析网页。我写了一个样本并进行测试。该程序可以在ubuntu 16.04中的python 3.5中运行良好，当所有工作完成后它将退出。源代码如下。

import aiohttp
import asyncio
from bs4 import BeautifulSoup

async def coro():
    coro_loop = asyncio.get_event_loop()
    url = u'https://www.python.org/'
    for _ in range(4):
        async with aiohttp.ClientSession(loop=coro_loop) as coro_session:
            with aiohttp.Timeout(30, loop=coro_session.loop):
                async with coro_session.get(url) as resp:
                    print('get response from url: %s' % url)
                    source_code = await resp.read()
                    soup = BeautifulSoup(source_code, 'lxml')

def main():
    loop = asyncio.get_event_loop()
    worker = loop.create_task(coro())
    try:
        loop.run_until_complete(worker)
    except KeyboardInterrupt:
        print ('keyboard interrupt')
        worker.cancel()
    finally:
        loop.stop()
        loop.run_forever()
        loop.close()

if __name__ == '__main__':
    main()

在测试时，我发现当我按Ctrl + C＆＃39;关闭程序时，会出现错误＆＃39;从未检索到任务异常＆＃39;。

^Ckeyboard interrupt
Task exception was never retrieved
future: <Task finished coro=<coro() done, defined at ./test.py:8> exception=KeyboardInterrupt()>
Traceback (most recent call last):
  File "./test.py", line 23, in main
    loop.run_until_complete(worker)
  File "/usr/lib/python3.5/asyncio/base_events.py", line 375, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.5/asyncio/base_events.py", line 345, in run_forever
    self._run_once()
  File "/usr/lib/python3.5/asyncio/base_events.py", line 1312, in _run_once
    handle._run()
  File "/usr/lib/python3.5/asyncio/events.py", line 125, in _run
    self._callback(*self._args)
  File "/usr/lib/python3.5/asyncio/tasks.py", line 307, in _wakeup
    self._step()
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "./test.py", line 17, in coro
    soup = BeautifulSoup(source_code, 'lxml')
  File "/usr/lib/python3/dist-packages/bs4/__init__.py", line 215, in __init__
    self._feed()
  File "/usr/lib/python3/dist-packages/bs4/__init__.py", line 239, in _feed
    self.builder.feed(self.markup)
  File "/usr/lib/python3/dist-packages/bs4/builder/_lxml.py", line 240, in feed
    self.parser.feed(markup)
  File "src/lxml/parser.pxi", line 1194, in lxml.etree._FeedParser.feed (src/lxml/lxml.etree.c:119773)
  File "src/lxml/parser.pxi", line 1316, in lxml.etree._FeedParser.feed (src/lxml/lxml.etree.c:119644)
  File "src/lxml/parsertarget.pxi", line 141, in lxml.etree._TargetParserContext._handleParseResult (src/lxml/lxml.etree.c:137264)
  File "src/lxml/parsertarget.pxi", line 135, in lxml.etree._TargetParserContext._handleParseResult (src/lxml/lxml.etree.c:137128)
  File "src/lxml/lxml.etree.pyx", line 324, in lxml.etree._ExceptionContext._raise_if_stored (src/lxml/lxml.etree.c:11090)
  File "src/lxml/saxparser.pxi", line 499, in lxml.etree._handleSaxData (src/lxml/lxml.etree.c:131013)
  File "src/lxml/parsertarget.pxi", line 88, in lxml.etree._PythonSaxParserTarget._handleSaxData (src/lxml/lxml.etree.c:136397)
  File "/usr/lib/python3/dist-packages/bs4/builder/_lxml.py", line 206, in data
    def data(self, content):
KeyboardInterrupt

我浏览了the offical docs of python，但还没有得到线索。我尝试在coro（）中捕获键盘中断。

try:
    soup = BeautifulSoup(source_code, 'lxml')
except KeyboardInterrupt:
    print ('capture exception')
    raise

每次尝试/除外＆＃39;在BeautifulSoup（）周围捕获KeyboardInterrupt，会发生错误。似乎BeautifulSoup导致错误。但是如何解决呢？

Answer 1

当您致电task.cancel()时，此功能实际上并未取消任务，它只是“标记”要取消的任务。当任务将恢复执行时，将启动取消任务的实际过程。 asyncio.CancelledError将立即在任务中引发，强制它被实际取消。任务将使用此异常完成它的执行。

另一方面，如果您的某些任务以静默方式完成异常（如果您没有检查任务执行结果），asyncio会发出警告。

为了避免出现问题，你应该等待任务取消接收asyncio.CancelledError（并且可能因为你不需要而取消）：

import asyncio
from contextlib import suppress


async def coro():
    # ...

def main():
    loop = asyncio.get_event_loop()
    worker = asyncio.ensure_future(coro())
    try:
        loop.run_until_complete(worker)
    except KeyboardInterrupt:
        print('keyboard interrupt')

        worker.cancel()
        with suppress(asyncio.CancelledError):
            loop.run_until_complete(worker)  # await task cancellation.
    finally:
        loop.close()

if __name__ == '__main__':
    main()

为什么BeautifulSoup与＆＃39;任务异常相关从未被检索过＆＃39;？

1 个答案: