C#无法使用有关Kafka主题的消息?

时间:2018-08-03 13:21:29

标签: c# apache-kafka confluent-kafka

我一直在研究Confluent.Kafka客户端(https://github.com/confluentinc/confluent-kafka-dotnet/)的几个示例,虽然我可以成功地使生产者将消息推送到Kafka中,但是却无法撤消任何消息与消费者。

通过用户界面,我可以看到创建了该主题,并且消息正在进入该主题(当前有10个分区和3条消息),但是我的使用者始终报告“分区结束”,而没有消耗任何消息( 3个仍然保留在该主题上,“ OnMessage”永不触发。

但是,消费者肯定可以访问这些主题,并且可以在其中一个分区上看到3条消息:

分区结束:dotnet-test-topic [6] @ 3

它只是不消耗消息并触发OnMessage()。有什么想法吗?

var conf = new Dictionary<string, object> 
{ 
    { "group.id", Guid.NewGuid().ToString() },
    { "bootstrap.servers", "mykafkacluster:9094" },
    { "sasl.mechanisms", "SCRAM-SHA-256" },
    { "security.protocol", "SASL_SSL" },
    { "sasl.username", "myuser" },
    { "sasl.password", "mypass" }
};

using (var producer = new Producer<string, string>(conf, new StringSerializer(Encoding.UTF8), new StringSerializer(Encoding.UTF8)))
{
    producer.ProduceAsync("dotnet-test-topic", "some key", "some value")
            .ContinueWith(result => 
            {
                var msg = result.Result;
                if (msg.Error.Code != ErrorCode.NoError)
                {
                    Console.WriteLine($"failed to deliver message: {msg.Error.Reason}");
                }
                else 
                {
                    Console.WriteLine($"delivered to: {result.Result.TopicPartitionOffset}");
                }
            });

    producer.Flush(TimeSpan.FromSeconds(10));
}

using (var consumer = new Consumer<string, string>(conf, new StringDeserializer(Encoding.UTF8), new StringDeserializer(Encoding.UTF8)))
{ 
    consumer.Subscribe("dotnet-test-topic");

    consumer.OnConsumeError += (_, err)
        => Console.WriteLine($"consume error: {err.Error.Reason}");

    consumer.OnMessage += (_, msg)
        => Console.WriteLine($"consumed: {msg.Value}");

    consumer.OnPartitionEOF += (_, tpo)
        => Console.WriteLine($"end of partition: {tpo}");

    while (true)
    {
        consumer.Poll(TimeSpan.FromMilliseconds(100));
    }  
}

1 个答案:

答案 0 :(得分:3)

如果没有提供以下配置,看起来OnMessage事件不会触发:

RSA

添加此内容后,我可以阅读有关该主题的消息。