Question

我正在尝试创建一个Akka-Kafka消费者。与Kafka的消费者群体（Consumer Group Example doc）的文档类似，我创建了一个Actor KafkaConsumerGroupMaster 和一个actor KafkaConsumer 。

KafkaConsumerGroupMaster 演员将收到一个动物园管理员，一个主题以及要创建和收听的流量。

创建 KafkaConsumer 演员更好吗？我的想法是：（1）在组主演员的 preStart（）中。（2）作为组主的 onRecieve（）中的创建消息。

谢谢，盖

Answer 1

如果在处理某些Init消息时创建子actor（或执行任何初始化），则必须确保在由于en错误重新启动actor时再次发送该消息，例如

如果您使用preStart()方法进行初始化，则无需考虑这一点。

Answer 2

我在GlobalStart的onStart重写方法中创建了使用者线程（从play.GlobalSettings扩展）。另外，我没有使用Java线程服务执行程序。相反，我创建了一个KafkaRunnable并使用Akka.system（）。dispatcher（）。execute（）调用直接执行它。

KafkaRunnable.java

package infrastructure.kafka;
import java.nio.charset.StandardCharsets;
import play.Logger;
import controllers.OrderController;
import kafka.consumer.ConsumerIterator;
import kafka.consumer.KafkaStream;
import kafka.message.MessageAndMetadata;
public class KafkaRunnable implements Runnable {
  private KafkaStream<byte[],byte[]> m_stream;
  private int m_threadNumber;
  public KafkaRunnable(KafkaStream<byte[],byte[]> a_stream, int a_threadNumber) {
    m_threadNumber = a_threadNumber;
    m_stream = a_stream;
}

@Override
public void run() {
    Logger.info("Starting Thread: " + m_threadNumber);
    ConsumerIterator<byte[],byte[]> it = m_stream.iterator();
    while (it.hasNext()){
        MessageAndMetadata<byte[],byte[]> msg=it.next();
        String orderId=new String(msg.key(), StandardCharsets.UTF_8);
        String uri=new String(msg.message(), StandardCharsets.UTF_8);
        try{
            Logger.info("Thread " + m_threadNumber + ": OrderId:" + orderId+";URI:"+uri);
            OrderController.notifyOrder(orderId, uri);
        }catch (Throwable ex){
            Logger.error("Error in notify order. ", ex);
        }

    }
    Logger.info("Shutting down Thread: " + m_threadNumber);
  }
}

KafkaConsumer.java

package infrastructure.kafka;

import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;

import play.Logger;
import play.libs.Akka;
import kafka.consumer.Consumer;
import kafka.consumer.ConsumerConfig;
import kafka.consumer.KafkaStream;
import kafka.javaapi.consumer.ConsumerConnector;


public class KafkaConsumer{
    public static ConsumerConfig createConsumerConfig() {
        Properties props = new Properties();

        props.put("zookeeper.connect",     play.Play.application().configuration().getString("zookeeper.connect"));
        props.put("group.id", "notifyOrder");
        props.put("zookeeper.session.timeout.ms", "400");
        props.put("zookeeper.sync.time.ms", "200");
        props.put("auto.commit.interval.ms", "1000");

        return new ConsumerConfig(props);
    }   
    private static ConsumerConnector consumer;
    public static void start(int noOfThreads){
        final String topic="neworder";

        //Create Consumer config
        ConsumerConfig cfg=KafkaConsumer.createConsumerConfig();
        consumer=Consumer.createJavaConsumerConnector(cfg);
        Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
        topicCountMap.put(topic, new Integer(noOfThreads));
        Map<String, List<KafkaStream<byte[], byte[]>>> consumerMap = consumer.createMessageStreams(topicCountMap);
        List<KafkaStream<byte[], byte[]>> streams = consumerMap.get(topic);
        int threadNumber = 0;
        for (final KafkaStream<byte[], byte[]> stream : streams) {
            Akka.system().dispatcher().execute(new KafkaRunnable(stream, threadNumber));
            threadNumber++;
        }       
    }
    public static void shutDown(){
        if (consumer!=null){
            Logger.info("Initiating Kafka Consumer shutdown");
            try {
                Thread.sleep(10000);
            } catch (InterruptedException ie) {
        }

            consumer.shutdown();
            Logger.info("Kafka Consumer shutdown complete");
        }
    }
}

我应该在哪里创建一个Akka演员卡夫卡消费者？

2 个答案: