Apache Beam kafka IO checkpoint

时间:2018-11-27 00:55:02

标签: apache-kafka offset dataflow beam

I am creating a service using apache beam(dataflow) which reads data from kafka and does some in stream transformations and write it back to kafka .

kafka topic ----> trasformations ----> kafka out topic

Here I want to write offset to kafka only after I load back data to kafka out topic.

We have a feature in akka streams where we can do this by m.committableOffset.commitScaladsl().

do we have similar thing in apache beam Kafka connector

Appreciate the help.

0 个答案:

没有答案