如何处理从Confluent Kafka消费者那里收到的结果?

时间:2019-07-13 04:09:53

标签: .net apache-kafka avro confluent-schema-registry confluent-kafka

嗨,我正在研究Confluent Kafka。我有一个消费者返回ConsumeResult。以下是我对消费者的实现。

package deadlock;

public class DeadlockApp {

    String s1 = "hello";
    String s2 = "world";

    Thread th1 = new Thread() {
        public void run() {
            System.out.println("Thread th1 has started");
            synchronized (s1) { //A lock is created internally (holds access of s1), lock will be released or unlocked for s1, only when it exits the block Line #23
                System.out.println("Executing first synchronized block of th1!");
                try {
                    Thread.sleep(1000);
                } catch(InterruptedException ex) {
                    System.out.println("Exception is caught in th1");
                }
                System.out.println("Waiting for the lock to be released from parrallel thread th1");
                synchronized (s2) { //As another has runned parallely Line #32, lock has been created for s2
                    System.out.println(s1 + s2);
                }

            }
            System.out.println("Thread th1 has executed");
        }
    };


    Thread th2 = new Thread() {
        public void run() {
            System.out.println("Thread th2 has started");
            synchronized (s2) { //A lock is created internally (holds access of s2), lock will be released or unlocked for s2, only when it exits the block Line #44
                System.out.println("Executing first synchronized block of th2!");
                try {
                    Thread.sleep(1000);
                } catch(InterruptedException ex) {
                    System.out.println("Exception is caught in th2");
                }
                System.out.println("Waiting for the lock to be released from parrallel thread th2");
                synchronized (s1) { //As another has runned parallely Line #11, lock has been created for s1
                    System.out.println(s1 + s2);
                }

            }
            System.out.println("Thread th2 has executed");
        }
    };

    public static void main(String[] args) {
        DeadlockApp deadLock = new DeadlockApp();
        deadLock.th1.start();
        deadLock.th2.start();
        //Line #51 and #52 runs parallely on executing the program, a lock is created inside synchronized method
        //A lock is nothing but, something like a blocker or wall, which holds access of the variable from being used by others.
        //Locked object is accessible, only when it is unlocked (i.e exiting  the synchronized block)
        //Lock cannot be created for primitive types (ex: int, float, double)
        //Dont forget to add thread.sleep(time) because if not added, then object access will not be at same time for both threads to create Deadlock (not actual runtime with lots of threads) 
        //This is a simple program, so we added sleep90 to create Deadlock, it will execute successfully, if it is removed. 
    }

    //Happy coding -- Parthasarathy S
}

我将此方法称为

public ConsumeResult<string, GenericRecord> Consume(string topic)
    {
      consumer.Subscribe(topic);
      ConsumeResult<string, GenericRecord> result;
      try
      {
        result = consumer.Consume();
        return result;
      }
      catch (Exception e)
      {
        this.logger.Error("KafkaClient", $"Error sending message '{e.Message}'");
        return null;
      }
    }

然后终于从控制器中以我的身份呼叫

  

var productEvents = this.eventDispatcher.SubscribeAsync();

现在我要处理这个结果。我知道我可以得到一个字段

public ConsumeResult<string, GenericRecord> SubscribeAsync()
    {
      return this.consumerClient.Consume(productEventTopicName);
    }

我想一次性获得所有字段。我的困惑是,是否需要根据从消费者那里获得的数据或我可以使用的现有模型来选择所需模型?如果我使用现有模型,那么如果明天会有新的字段出现在消费者结果中,那么我们的模型将如何处理呢?有人可以帮我理解吗?任何帮助,将不胜感激。谢谢。

1 个答案:

答案 0 :(得分:0)

当您使用GenericRecord时,反序列化器将与架构注册表联系,以在生产者发送新的架构信息时提取它。

但是,为了解析新字段,必须停止使用者并显式添加代码以相应地处理这些属性。这个问题不是Kafka特有的