Question

我编写了一个代码，可以在主题中以键值方式生成消息：

from kafka import KafkaProducer
from kafka.errors import KafkaError
import json

producer = KafkaProducer(bootstrap_servers=['localhost:9092'])

# produce json messages
producer = KafkaProducer(value_serializer=lambda m: json.dumps(m).encode('ascii'))

deviceId = "4bc03533ccc94065"
responseId = "c03c4851-701f-4265-aafd-eb133c09c08e"

print deviceId
print responseId

producer.send('collect-response-devices', {'deviceId': deviceId})
producer.send('collect-response-responses', {'responseId' : responseId})

def on_send_success(record_metadata):
    print(record_metadata.topic)
    print(record_metadata.partition)
    print(record_metadata.offset)

def on_send_error(excp):
    log.error('I am an errback', exc_info=excp)
    print excp
    # handle exception

# block until all async messages are sent
producer.flush()

# configure multiple retries
producer = KafkaProducer(retries=5)

但是，我的消费者会消费消息，不分配任何内容（None中的key）以及值部分中的所有内容。

from kafka import KafkaConsumer
import json

consumer = KafkaConsumer(bootstrap_servers='localhost:9092', auto_offset_reset='earliest', value_deserializer=lambda m: json.loads(m.decode('utf-8')))
consumer.subscribe(['collect-response-devices'])
for message in consumer:
  print (message.key, message.value)

这是消费者的输出：

（无，{u'deviceId'：u'4bc03533ccc94065'}）

Answer 1

根据docs，您需要在生成消息时指定密钥：

# produce keyed messages to enable hashed partitioning
producer.send('my-topic', key=b'foo', value=b'bar')

目前，您在制作邮件时未指定密钥，因此获取None

无法获得Kafka Python Consumer的密钥

1 个答案: