并发POST请求附件Python

时间:2017-02-16 11:19:13

标签: python concurrency python-requests python-asyncio aiohttp

我有一台等待包含图片的请求的服务器:

@app.route("/uploader_ios", methods=['POST'])
def upload_file_ios():
    imagefile = request.files['imagefile']

我可以很容易地使用python中的请求提交帖子请求,如下所示:

url = "<myserver>/uploader_ios"
files = {'imagefile': open(fname, 'rb')}
%time requests.post(url, files=files).json()  # 2.77s

但是,我想要做的是同时提交1000或100,000个请求。我想尝试使用asyncio这样做,因为我已经能够使用它来获取请求而没有问题。但是我无法看到创建服务器接受的有效发布请求。

我的尝试如下:

import aiohttp
import asyncio
import json

# Testing with small amount
concurrent = 2
url_list = ['<myserver>/uploader_ios'] * 10

def handle_req(data):
    return json.loads(data)['English']

def chunked_http_client(num_chunks, s):
    # Use semaphore to limit number of requests
    semaphore = asyncio.Semaphore(num_chunks)
    @asyncio.coroutine
    # Return co-routine that will work asynchronously and respect
    # locking of semaphore

    def http_get(url):
        nonlocal semaphore
        with (yield from semaphore):

            # Attach files
            files = aiohttp.FormData()
            files.add_field('imagefile', open(fname, 'rb'))

            response = yield from s.request('post', url, data=files)
            print(response)

            body = yield from response.content.read()
            yield from response.wait_for_close()
        return body
    return http_get

def run_experiment(urls, _session):
    http_client = chunked_http_client(num_chunks=concurrent, s=_session)

    # http_client returns futures, save all the futures to a list
    tasks = [http_client(url) for url in urls]

    dfs_route = []

    # wait for futures to be ready then iterate over them
    for future in asyncio.as_completed(tasks):
        data = yield from future
        try:
            out = handle_req(data)
            dfs_route.append(out)
        except Exception as err:
            print("Error {0}".format(err))
    return dfs_route

with aiohttp.ClientSession() as session:  # We create a persistent connection
    loop = asyncio.get_event_loop()
    calc_routes = loop.run_until_complete(run_experiment(url_list, session))

问题是我得到的回应是:

.../uploader_ios) [400 BAD REQUEST]>

我假设这是因为我没有正确附加图像文件

0 个答案:

没有答案