python2中的多处理池无法正常工作

时间:2015-07-16 19:07:38

标签: python-2.7 python-3.x flask python-multiprocessing

我试图从8080上托管的API以多处理的形式调用其他3个API。这是我正在使用的代码(python 3.4):

import requests, json
from flask import Flask, request
from concurrent import futures

def getRequest(*args):
    return requests.get(args[0],params = args[1])

process = futures.ProcessPoolExecutor(3)
some_text =  open('lines.txt').read() # some text
params = {'text': some_text}
url1 = 'http://localhost:8081' # api2
url2 = 'http://localhost:8082' # api1
url3 = 'http://localhost:8083' # api3
g = process.map(getRequest, [[url1,params], [url2,params], [url3, params]])

上面的代码片段位于端口8080上运行的烧瓶应用程序中。我可以连接到端口8080甚至请求并行发送到端口8081,8082,8083,但这些API之间存在一些数据类型不匹配。它会自动将json.dumps(arg[0],arg[1])的数据更改为某种形式。

所有其他API单独运行良好。

这是我得到的错误:

127.0.0.1 - - [16/Jul/2015 21:58:18] "GET /?%7B%22article%22:%20%22American%20Profile%20-%20American%20Profile%20Celebrates%20The%20Intriguing%20People,%20Places%20And%20Things%20In%20Hometowns%20Across%20The%20Country%20Along%20With%20Features%20On%20Music,%20Film,%20TV,%20Seasonal%20Recipes,%20Health%20And%20Family%20Finance.%20Get%20every%20new%20post%20delivered%20to%20your%20Inbox.%20Join%20184%20other%20followers%20%5Cn%22%7D HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/prateekgupta/Desktop/project-annie/phrase_extraction/code/getNounsAndNounPhrases.py", line 18, in getKeywords
    vals = eval(request.data)
File "<string>", line 0

    ^
SyntaxError: unexpected EOF while parsing

这是端口8081(python 2.7)上的API:

# API
app=Flask(__name__)
@app.route('/', methods=['GET', 'POST'])
def getSomething():
    vals = dict(request.args)['key'][0] # its a hack. idk when to use args. ideally i will like to pass json and get json
    return json.dumps(some_function(vals), indent=4) + '\n'


# start your app here
if __name__ == '__main__':
    app.debug = True
    app.run(host='0.0.0.0', port =8081)

我尝试过使用参数的不同配置,但没有用。

1 个答案:

答案 0 :(得分:0)

evaljson.dumps不同,json.loads是(eval将有效负载视为Python代码,而JSON不是Python的正确子集。)

将您的getSomething处理程序更改为使用vals = json.loads(request.data)(甚至更好,vals = request.get_json(force=true)),事情应该按预期工作。