Question

我正在尝试运行一个使用Confluent Kafka将消息发送到特定主题的Flask应用。当我在本地以Flask应用程序运行它时，它可以正常工作，但是当我在Nginx后面的Docker容器中运行它时，它不起作用。 send方法有效，但是当我尝试通过调用flush方法使其同步时，它“挂起”。查看输出，刷新看起来会产生另一个进程。我假设它挂起的原因是冲洗针对生产者的另一个实例运行。如果我在post块内实例化生产者，则可以解决问题。但是，实例化将花费几乎100ms的时间，这是不可接受的，因此需要全球生产者。以下是一个演示该问题的示例烧瓶应用程序。

from flask import Flask, request, jsonify
from flask_restful import Resource, Api, abort

from confluent_kafka import Producer

producer = Producer({'bootstrap.servers':"kafka.nonprod:9092"})

# Create the app and api
app = Flask(__name__)
api = Api(app)

class Gateway(Resource):
    def post(self):
        try:
            message = "This is a test message"

            producer.produce("output_topic", message)
            producer.flush()

            resp = jsonify({"response": "Okay"})
            resp.status_code = 200
            return resp

        except Exception as exception:
            print(type(exception))
            print(exception.message)
            abort(400, data=request.data, message=exception.message)

api.add_resource(Gateway, '/')

我还在这里包括Docker映像的输出：

    docker run --rm -p 80:80 gateway
/usr/lib/python2.7/site-packages/supervisor/options.py:298: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2018-10-04 16:52:07,212 CRIT Supervisor running as root (no user in config file)
2018-10-04 16:52:07,213 INFO supervisord started with pid 1
2018-10-04 16:52:08,216 INFO spawned: 'nginx' with pid 10
2018-10-04 16:52:08,219 INFO spawned: 'uwsgi' with pid 11
[uWSGI] getting INI configuration from /etc/uwsgi/uwsgi.ini
*** Starting uWSGI 2.0.17 (64bit) on [Thu Oct  4 16:52:08 2018] ***
compiled with version: 6.4.0 on 27 March 2018 12:43:27
os: Linux-4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018
nodename: 43e8ded405ee
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /gateway
detected binary path: /usr/sbin/uwsgi
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
setgid() to 101
set additional group 82 (www-data)
setuid() to 100
Python version: 2.7.15 (default, Aug 22 2018, 13:24:18)  [GCC 6.4.0]
Python main interpreter initialized at 0x7f7cce1de760
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 437520 bytes (427 KB) for 5 cores
*** Operational MODE: preforking ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x7f7cce1de760 pid: 11 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 11)
spawned uWSGI worker 1 (pid: 19, cores: 1)
2018-10-04 16:52:09,322 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-10-04 16:52:09,322 INFO success: uwsgi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
spawned uWSGI worker 2 (pid: 26, cores: 1)

我有一个使用Apache Kafka库的以前版本的应用程序。我将转换为Confluent Kafka库，以便可以利用架构注册表及其提供的所有基础架构管理工具。

使用Confluent Kafka在Flask应用中需要同步Kafka生产者

0 个答案: