通过kafka-python将消息发布到Confluent云集群

时间:2019-01-25 14:11:58

标签: python apache-kafka kafka-python

我正在使用kafka-python软件包将消息发布到kafka融合云集群。

我的代码如下:

    from kafka import KafkaProducer
    producer=KafkaProducer(
                    bootstrap_servers='pkc-epgnk.us-central1.gcp.confluent.cloud:9092',
                    security_protocol='SASL_SSL',
                    sasl_mechanism='PLAIN',
                    ssl_certfile='/usr/local/etc/openssl/cert.pem',
                    sasl_plain_username='[api_key]',
                    sasl_plain_password='[api_secret]')

    producer.prodcue(topic='file-ingestion',key=b'',value=b'test')

运行上面的代码,我得到下面的错误:

kafka.errors.NoBrokersAvailable: NoBrokersAvailable

融合的卡夫卡手册建议我为C / C ++生产者设置以下值;但是,我不能通过kafka-python生产者设置所有值

bootstrap.servers=pkc-epgnk.us-central1.gcp.confluent.cloud:9092
api.version.request=true
broker.version.fallback=0.10.0.0
api.version.fallback.ms=0
sasl.mechanisms=PLAIN
security.protocol=SASL_SSL
ssl.ca.location=/usr/local/etc/openssl/cert.pem
sasl.username=<CLUSTER_API_KEY>
sasl.password=<CLUSTER_API_SECRET>

1 个答案:

答案 0 :(得分:0)

您可能应该使用confluent-kafka-python客户端来设置这些属性,并在产品之间获得更好的集成和支持

Example code from Github

from confluent_kafka import Producer, Consumer

p = Producer({
    'bootstrap.servers': '<ccloud bootstrap servers>',
    'broker.version.fallback': '0.10.0.0',
    'api.version.fallback.ms': 0,
    'sasl.mechanisms': 'PLAIN',
    'security.protocol': 'SASL_SSL',
    'sasl.username': '<ccloud key>',
    'sasl.password': '<ccloud secret>'
})