Question

我有一个RESTFUL Flask API，我正在使用gunicorn，并且在向发出POST请求的人发送响应之后，我试图继续运行parse_request()，这样他们就不必等待它完成了

我不太确定这是否可以实现我想要的功能，但这是我到目前为止的代码。

from threading import Thread
import subprocess
from flask import Flask
import asyncio

application = Flask(__name__)

async def parse_request(data):
    try:
        command = './webscraper.py -us "{user}" -p "{password}" -url "{url}"'.format(**data)
        output = subprocess.check_output(['bash','-c', command])
    except Exception as e:
        print(e)

@application.route('/scraper/run', methods=['POST'])
def init_scrape():
    try:
        thread = Thread(target=parse_request, kwargs={'data': request.json})
        thread.start()
        return jsonify({'Scraping this site: ': request.json["url"]}), 201
    except Exception as e:
                print(e)


if __name__ == '__main__':
    try:
        application.run(host="0.0.0.0", port="8080")
    except Exception as e:
        print(e)

我正在发送与此类似的POST请求。 localhost:8080/scraper/run

data = {
    "user": "username",
    "password": "password",
    "url": "www.mysite.com"
}

发送POST请求时遇到的错误是这个。

/usr/lib/python3.6/threading.py:864: RuntimeWarning: coroutine 'parse_request' was never awaited
  self._target(*self._args, **self._kwargs)

Answer 1

首先，为什么要通过子进程调用webscraper.py？这是完全没有意义的。由于webscraper.py是python脚本，因此您应该从webscraper.py导入所需的函数/类并直接使用它们。这样称呼它完全打败了您想要做的事情。

接下来，您的实际问题是在异步和线程之间混为一谈。我建议您了解更多信息，但是本质上，您希望使用Quart这种类似Flask的异步版本的东西，它很适合您的情况。

from quart import Quart, response, jsonify
import asyncio
from webscraper import <Class>, <webscraper_func> # Import what you need or
import webscraper # whatever suits your needs


app = Quart(__name__)

async def parse_request(user, password, url):
    webscraper_func(user, password, url)
    return 'Success'

@app.route('/scraper/run', methods=['POST'])
async def init_scrape():

    user = request.args.get('user')
    password = request.args.get('password')
    url = request.args.get('url')

    asyncio.get_running_loop().run_in_executor(
        None, 
        parse_request(user, password, url)
    )

    return 'Success'


if __name__ == '__main__':
    app.run(host='0.0.0.0', port='8080')

我该如何解决从未等待的协程？

1 个答案: