我已经为网络爬虫创建了Flask API,并希望在收到POST请求时开始运行它,一旦它开始运行,就返回一个响应,而不是等待刮板完成。
我刚刚开始尝试在代码中实现asyncio,但是我不明白如何跳过等待响应,我已经用邮递员测试了以下代码,但仍挂起等待响应
到目前为止,这是我的代码:
import subprocess
from flask import Flask
import asyncio
application = Flask(__name__)
async def parse_request(data):
command = './portal_scrape.py -us "{user}" -p "{password}"-url "{url}"'.format(**data)
output = subprocess.check_output(['bash','-c', command])
@application.route('/scraper/run', methods=['POST'])
def init_scrape():
loop = asyncio.get_event_loop()
# I want task1 to start running and then return data to postman
task1 = asyncio.ensure_future(parse_request(request.json))
loop.run_until_complete(task1)
return jsonify({'Scraping this site: ': "url"}), 201
except Exception as e:
print(e)
if __name__ == '__main__':
application.run(host="0.0.0.0", port="8080")