卡夫卡SASL握手时间过长

时间:2018-07-09 16:09:26

标签: c# security apache-kafka confluent-kafka

说明: 使用SASL / SCRAM或SASL / PLAINTEXT进行的身份验证大约需要9秒钟才能完成。 这正常吗?

如何复制:

  • 一个Kafka经纪人实例(v1.1.0)
  • 一个执行以下操作的C#生产者(Confluent Kafka Client v0.11.4):
       

    var producerConfig =    
               PropertiesUtils.ReadPropertiesFile("producer.properties");

           using (var producer = new Producer(producerConfig, null, new StringSerializer(Encoding.UTF8)))
           {
               while (true)
               {
                   Console.Write("message: ");
                   string msg = Console.ReadLine();

                   producer.ProduceAsync("test-topic", null, msg);
               }
           }

  • 一个执行以下操作的C#使用者(Confluent Kafka Client v0.11.4):
        

    var config = PropertiesUtils.ReadPropertiesFile("consumer.properties");

            using (var consumer = new Consumer(config, null, new StringDeserializer(Encoding.UTF8)))
            {
                consumer.OnMessage += (_, msg)
                      =>
                {
                    Console.WriteLine(msg.Value);
                };

                consumer.OnError += (_, error)
                  => Console.WriteLine($"Error: {error}");

                consumer.OnConsumeError += (_, msg)
                  => Console.WriteLine($"Consume error ({msg.TopicPartitionOffset}): {msg.Error}");

                consumer.Subscribe("test-topic");

                while (true)
                {
                    try
                    {
                        consumer.Poll(TimeSpan.FromMilliseconds(1000));
                    }
                    catch(Exception e)
                    {
                        Console.WriteLine(e.Message);
                    }
                }
            }

  • server.properties:

      

    broker.id = 0
      num.network.threads = 3
      num.io.threads = 8

         

    socket.send.buffer.bytes = 102400
      socket.receive.buffer.bytes = 102400
      socket.request.max.bytes = 104857600
      session.timeout.ms = 1000

         

    group.initial.rebalance.delay.ms = 0

         

    listeners = SASL_SSL:// localhost:9093

         

    ssl.keystore.type = JKS
      ssl.keystore.location = ...
      ssl.keystore.password = ...
      ssl.key.password = ...

         

    ssl.truststore.type = JKS
      ssl.truststore.location = ...
      ssl.truststore.password = ...

         

    ssl.protocol = TLS
      ssl.enabled.protocols = TLSv1.2,TLSv1.1,TLSv1
      ssl.client.auth =必需
      security.inter.broker.protocol = SASL_SSL
      ssl.secure.random.implementation = SHA1PRNG

         

    sasl.enabled.mechanisms =普通,SCRAM-SHA-256
      sasl.mechanism.inter.broker.protocol =普通

         

    log.dirs = ...
      num.partitions = 1
      num.recovery.threads.per.data.dir = 1

         

    offsets.topic.replication.factor = 1
      transaction.state.log.replication.factor = 1
      transaction.state.log.min.isr = 1

         

    log.retention.hours = 168
      log.retention.bytes = 1073741824
      log.segment.bytes = 1073741824
      log.retention.check.interval.ms = 300000
      num.replica.fetchers = 1

         

    zookeeper.connect =本地主机:2181
      zookeeper.connection.timeout.ms = 6000
      group.initial.rebalance.delay.ms = 0

  • consumer.properties:

      

    bootstrap.servers = localhost:9093
      group.id = test-consumer-group
      fetch.min.bytes = 1
      fetch.wait.max.ms = 1
      auto.offset.reset =最新
      socket.blocking.max.ms = 1
      fetch.error.backoff.ms = 1
      ssl.ca.location = ...
      ssl.certificate.location = ...
      ssl.key.location = ...
      ssl.key.password = ..
      security.protocol = SASL_SSL
      sasl.mechanisms = PLAIN
      sasl.username = ...
      sasl.password = ...

  • producer.properties

      

    bootstrap.servers = localhost:9093
      compression.type = none
      linger.ms = 0
      重试= 0攻击= 0

         

    ssl.ca.location = ...
      ssl.certificate.location = ...
      ssl.key.location = ...
      ssl.key.password = ...

         

    security.protocol = SASL_SSL
      sasl.mechanisms = PLAIN
      sasl.username = ...
      sasl.password = ...

  • 运行使用者。从请求到完成,大约需要9秒钟才能完成SASL握手。这是日志:

      

    [2018-07-06 17:03:37,673]调试将SASL服务器状态设置为HANDSHAKE_OR_VERSIONS_REQUEST(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)   [2018-07-06 17:03:37,673]调试处理Kafka请求API_VERSIONS(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)   [2018-07-06 17:03:37,673]调试将SASL服务器状态设置为HANDSHAKE_REQUEST(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)   [2018-07-06 17:03:37,673]调试Kafka请求SASL_HANDSHAKE(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)   [2018-07-06 17:03: 37 ,674]使用客户端提供的SASL机制“ PLAIN”进行调试(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)   [2018-07-06 17:03: 46 ,805]调试将SASL服务器状态设置为AUTHENTICATE(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)   [2018-07-06 17:03:46,807]调试将SASL服务器状态设置为COMPLETE(org.apache.kafka.common.security.authenticator.SaslServerAuthenticator)

备注

  • 在运行生产者时,我观察到的持续时间也相同

  • 使用SCRAM-256进行身份验证时,我观察到了相同的持续时间

  • 在运行Java客户端(kafka-console-consumerkafka-console-producer)时,我观察到了相同的持续时间

更新: 根据{{​​3}}的建议,当我在Ubuntu虚拟机上设置代理并从Windows连接客户端时,不再观察到延迟。我想知道为什么Windows上的SSL握手和SASL身份验证每个都要花10秒!

0 个答案:

没有答案