如何使用null键对kstream执行操作?

时间:2018-05-16 11:36:08

标签: java apache-kafka kafka-consumer-api apache-kafka-streams

我有一个像这样的kafka流对象:

[KSTREAM-SOURCE-0000000000]: null, {"id": 1, "name": "john", "age": 26, "updated_at": 1525774480752}
[KSTREAM-SOURCE-0000000000]: null, {"id": 2, "name": "jane", "age": 24, "updated_at": 1525774480784}
[KSTREAM-SOURCE-0000000000]: null, {"id": 3, "name": "julia", "age": 25, "updated_at": 1525774480827}

当我尝试执行地图操作时:

KStream<String, mysql> locationsStream = pgsql_users.map((k, v) -> new KeyValue<String, mysql>(k.toString(), new mysql ((Integer) v.get("id"), (String) v.get("name").toString(), (Integer) v.get("age")) ));

我得到一个例外:

[KSTREAM-SOURCE-0000000000]: null, {"id": 1, "name": "john", "age": 26, "updated_at": 1525774480752}
  

线程中的异常   “测试应用程式内-0d15a00c-51dc-4c1a-8293-82f47fc7ef90-StreamThread-1”   java.lang.NullPointerException at   com.aail.kafka_stream.lambda $ main $ 0(kafka_stream.java:86)at   org.apache.kafka.streams.kstream.internals.KStreamMap $ KStreamMapProcessor.process(KStreamMap.java:41)     在   org.apache.kafka.streams.processor.internals.ProcessorNode $ 1.run(ProcessorNode.java:46)     在   org.apache.kafka.streams.processor.internals.StreamsMetricsImpl.measureLatencyNs(StreamsMetricsImpl.java:208)     在   org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:124)     在   org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:85)     在   org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:80)     在   org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:216)     在   org.apache.kafka.streams.processor.internals.AssignedTasks.process(AssignedTasks.java:403)     在   org.apache.kafka.streams.processor.internals.TaskManager.process(TaskManager.java:317)     在   org.apache.kafka.streams.processor.internals.StreamThread.processAndMaybeCommit(StreamThread.java:942)     在   org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:822)     在   org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:774)     在   org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:744)

我以为我得到了这个,因为密钥为空。有人可以帮助我解决这个问题。

1 个答案:

答案 0 :(得分:2)

您使用的是k.toString(),但k为空。

您可能希望在新密钥中再次使用knull,如果您确实需要String.valueOf(k),则可以"null"(不推荐,因为它将所有消息发送到单个分区)

此外,由于键为空,您只需将byte[]设置为流的关键对象即可。无需使用反序列化器