Docker镜像中的nginx缓冲烧瓶事件流

时间:2018-01-29 16:44:17

标签: python nginx flask event-stream event-stream-processing

我有一个带有python / flask的REST API后端,并希望在事件流中传输响应。一切都在使用nginx / uwsgi(https://hub.docker.com/r/tiangolo/uwsgi-nginx-flask/)的docker容器中运行。

API工作正常,直到事件流。似乎某些东西(可能是nginx)正在缓冲“产量”,因为在服务器完成计算并且所有内容一起发送之前,任何类型的客户端都不会收到任何内容。

我尝试使用另外的config(nginx_streaming.conf)文件调整nginx设置(根据docker图像说明):

server {
  location / {
        include uwsgi_params;
        uwsgi_request_buffering off;
  }
}

dockerfile:

FROM tiangolo/uwsgi-nginx-flask:python3.6
COPY ./app /app
COPY ./nginx_streaming.conf /etc/nginx/conf.d/nginx_streaming.conf

但是我对nginx设置并不熟悉,并确定我在这里做什么^^这至少不起作用..有什么建议吗?

我的服务器端实现:

from flask import Flask
from flask import stream_with_context, request, Response
from werkzeug.contrib.cache import SimpleCache
cache = SimpleCache()
app = Flask(__name__)

from multiprocessing import Pool, Process
@app.route("/my-app")
def myFunc():
    global cache

    arg = request.args.get(<my-arg>)
    cachekey = str(arg)
    print(cachekey)

    result = cache.get(cachekey)
    if result is not None:
        print('Result from cache')
        return result
    else:
        print('object not in Cache...calculate...')
        def calcResult():
            yield 'worker thread started\n'

            with Pool(processes=cores) as parallel_pool:
                [...]

            yield 'Somewhere in the processing'
            temp_result = doSomethingWith(

            savetocache = cache.set(cachekey, temp_result, timeout=60*60*24) #timeout in seconds

            yield 'saved to cache with key:' + cachekey +'\n'
            print(savetocache, flush=True)
            yield temp_result

        return Response(calcResult(), content_type="text/event-stream")

if __name__ == "__main__":
    # Only for debugging while developing
    app.run(host='0.0.0.0', debug=True, port=80)

2 个答案:

答案 0 :(得分:0)

我遇到了同样的问题。尝试更改

return Response(calcResult(), content_type="text/event-stream")

return Response(calcResult(), content_type="text/event-stream", headers={'X-Accel-Buffering': 'no'})

答案 1 :(得分:0)

the answer from @u-rizwan here之后,我将其添加到// RECOGNIZE URLS AND CONVERT THEM TO HYPERLINKS // Ignore if hyperlink is found in HTML attr, like "href" $('.js-replaceUrls').each(function(){ // GET THE CONTENT var str = $(this).html(); // SET THE REGEX STRING var regex = /(([a-z]+:\/\/)?(([a-z0-9\-]+\.)+([a-z]{2}|aero|arpa|biz|com|coop|edu|gov|info|int|jobs|mil|museum|name|nato|net|org|pro|travel|local|internal))(:[0-9]{1,5})?(\/[a-z0-9_\-\.~]+)*(\/([a-z0-9_\-\.]*)(\?[a-z0-9+_\-\.%=&amp;]*)?)?(#[a-zA-Z0-9!$&'()*+.=-_~:@/?]*)?)(\s+|$)/gmi; // REPLACE PLAIN TEXT LINKS BY HYPERLINKS var replaced_text = str.replace(regex, "<a href='$1' class='js-link'>$1</a>"); // ECHO LINK $(this).html(replaced_text); }); // DEFINE URLS WITHOUT "http" OR "https" var linkHasNoHttp = $(".js-link:not([href*=http],[href*=https])"); // ADD "http://" TO "href" $(linkHasNoHttp).each(function() { var linkHref = $(this).attr("href"); $(this).attr("href" , "http://" + linkHref); }); 并解决了问题:

/etc/nginx/conf.d/mysite.conf

我已将其添加到add_header X-Accel-Buffering no; 下,但是将其放置在事件流的特定位置可能是个好主意(我在这里使用低流量的Intranet用例)。

注意:如果nginx来自应用程序,则默认情况下似乎会剥离此标头:https://serverfault.com/questions/937665/does-nginx-show-x-accel-headers-in-response