KafkaAvroDeserializer问题

Question

我正在尝试使用Avro Kafka Deserializer对Kafka Avro消息进行去序列化。这段代码很常见，很多用户已经在实践中使用。但我在实施同样的方面遇到的困难很少： 代码：

Properties props = new Properties();
props.put("bootstrap.servers", "broker1:9092,broker2:9092");
props.put("group.id", "CountryCounter");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.serializer", "io.confluent.kafka.serializers.KafkaAvroDeserializer");
props.put("schema.registry.url", schemaUrl);
String topic = "customerContacts"

KafkaConsumer consumer = new KafkaConsumer(createConsumerConfig(brokers, groupId, url));
consumer.subscribe(Collections.singletonList(topic));

System.out.println("Reading topic:" + topic);

while (true) {
        @SuppressWarnings("unchecked")
        ConsumerRecords<String, ATPEvent> records = (ConsumerRecords<String, ATPEvent>) consumer.poll(1000);  //1

       for (ConsumerRecord<String, ATPEvent> record: records)  //2 {

            try {
                kafkaMessageInputStream = new ByteBufferInputStream(Lists.newArrayList(ByteBuffer.wrap(record.value()))); //3
                avroBinaryDecoder = avroDecoderFactory.binaryDecoder(kafkaMessageInputStream, avroBinaryDecoder);
                avroEvent = reader.read(avroEvent, avroBinaryDecoder);
                System.out.println(avroEvent);
                kafkaMessageInputStream.close();
              } catch (Exception ex) {
                System.out.println("Unable to process event from kafka, see exception details" + ex);
              }
        }
        consumer.commitSync(); //4
}

现在，这是4个问题：

1. 我必须添加强制转换，否则会出错 Type mismatch: cannot convert from Map<String,ConsumerRecords<String,ATPEvent>> to ConsumerRecords<String,ATPEvent>

2. Can only iterate over an array or an instance of java.lang.Iterable 我不知道为什么这个？我可以这样做吗？ ：

   List<ConsumerRecord<String, ATPEvent>> records = (List<ConsumerRecord<String, ATPEvent>>) consumer.poll(1000);
   
   for (ConsumerRecord<String, ATPEvent> record: records) {
   

3. The method wrap(byte[]) in the type ByteBuffer is not applicable for the arguments (ATPEvent) 我理解这一点，但是如何将类转换为字节，还有其他方法吗？

4. The method commitSync() is undefined for the type KafkaConsumer<String,ATPEvent> 我可以使用consumer.close();

请提供解决方案2＆amp; 3，如果可能的解释为1＆amp; 4。

Answer 1

您使用哪个版本的Kafka？ 0.8.x和0.9.x

存在差异

Kafka 0.8.x：

1. 返回类型为Map<String, ConsumerRecords>（请参阅http://www.webhostingreviewjam.com/mirror/apache/kafka/0.8.2-beta/java-doc/org/apache/kafka/clients/consumer/KafkaConsumer.html）
2. 使用ConsumerRecords#records(...)获取List<ConsumerRecord>进行迭代（请参阅http://www.webhostingreviewjam.com/mirror/apache/kafka/0.8.2-beta/java-doc/org/apache/kafka/clients/consumer/ConsumerRecords.html）
3. ConsumerRecord.value()返回byte[]（请参阅http://www.webhostingreviewjam.com/mirror/apache/kafka/0.8.2-beta/java-doc/org/apache/kafka/clients/consumer/ConsumerRecord.html#value%28%29）
4. 使用仅{0. 1}中提供的commit(boolen)（请参阅http://www.webhostingreviewjam.com/mirror/apache/kafka/0.8.2-beta/java-doc/org/apache/kafka/clients/consumer/KafkaConsumer.html#commit%28boolean%29） - commitSync()

Kafka 0.9.x

1. 返回类型为ConsumerRecords（请参阅https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html）
2. ConsumerRecords实施Iterable（请参阅https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html）因此，您可以使用for(ConsumerRecord r : records)
3. ConsumerRecord#value返回T（在您的情况下为T == ATPEvent）（请参阅https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html）
4. commitSync()可在0.9.x
中找到