Kafka / kubernetes关闭Zookeeper会话并在启用SASL时关闭

时间:2018-09-14 08:39:16

标签: apache-kafka apache-zookeeper

我正在Kubernetes中运行Kafka和zookeeper,并且当我启用SASL身份验证时,Kafka连接到zookeeper并在会话获得会话身份后立即关闭会话,这反过来导致容器在Kafka关闭时退出。

请注意,当PLAINTEXT是唯一使用的侦听器,但我们需要身份验证时,所有这些都可以正常工作。

我在本地使用Kafka和Zookeeper的非容器化版本进行了测试,并且该配置可以按预期工作:

非容器配置

sasl.enabled.mechanisms=SCRAM-SHA-256
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256
listeners=INTERNAL://localhost:9092
advertised.listeners=INTERNAL://localhost:9092
listener.security.protocol.map=INTERNAL:SASL_PLAINTEXT
inter.broker.listener.name=INTERNAL

容器配置:

  advertised.listeners: |- 
    INTERNAL://${POD_IP}:9092
  listener.security.protocol.map: |-
    INTERNAL:SASL_PLAINTEXT
  inter.broker.listener.name: INTERNAL
  sasl.enabled.mechanisms: SCRAM-SHA-256
  sasl.mechanism.inter.broker.protocol: SCRAM-SHA-256
  security.inter.broker.protocol: SASL_PLAINTEXT

容器日志:

[main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=zookeeper:2181 sessionTimeout=40000 watcher=io.confluent.admin.utils.ZookeeperConnectionWatcher@4edde6e5
[main-SendThread(zookeeper:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/etc/jaas/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(zookeeper:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper/100.70.125.109:2181
[main] ERROR io.confluent.admin.utils.ClusterStatus - Error occurred while connecting to Zookeeper server[zookeeper:2181]. Authentication failed.
[main-SendThread(zookeeper:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to zookeeper/100.70.125.109:2181, initiating session
[main-SendThread(zookeeper:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server zookeeper/100.70.125.109:2181, sessionid = 0x265d7206f920000, negotiated timeout = 40000
[main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x265d7206f920001 closed
[main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x265d7206f920001

我认为这可能是问题所在

[main] ERROR io.confluent.admin.utils.ClusterStatus - Error occurred while connecting to Zookeeper server[zookeeper:2181]. Authentication failed.

但是它成功建立了与Zookeeper的成功连接(Zookeeper没有身份验证)。此行也出现在非容器Kafka安装中,该安装在本地运行:

非容器日志(有效):

[2018-09-14 08:59:00,519] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@10e92f8f (org.apache.zookeeper.ZooKeeper)
[2018-09-14 08:59:00,536] WARN SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/etc/kafka/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. (org.apache.zookeeper.ClientCnxn)
[2018-09-14 08:59:00,536] INFO [ZooKeeperClient] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2018-09-14 08:59:00,537] INFO Opening socket connection to server localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2018-09-14 08:59:00,540] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient)
[2018-09-14 08:59:00,555] INFO Socket connection established to localhost/127.0.0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2018-09-14 08:59:00,564] INFO Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1000004693a0004, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2018-09-14 08:59:00,565] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.ZooKeeperClient)
[2018-09-14 08:59:00,801] INFO Cluster ID = zNddSOF5SDm4FkjP33FYNQ (kafka.server.KafkaServer)
[2018-09-14 08:59:00,892] INFO KafkaConfig values:
    advertised.host.name = null
    advertised.listeners = INTERNAL://localhost:9092
    advertised.port = null
    alter.config.policy.class.name = null

容器Zookeeper日志似乎并没有太大帮助:

2018-09-14 08:31:23,237 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /100.105.123.61:37766
2018-09-14 08:31:23,240 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /100.105.123.61:37766
2018-09-14 08:31:23,242 [myid:2] - INFO  [CommitProcessor:2:ZooKeeperServer@687] - Established session 0x265d7206f920001 with negotiated timeout 40000 for client /100.105.123.61:37766
2018-09-14 08:31:23,246 [myid:2] - INFO  [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@486] - Processed session termination for sessionid: 0x265d7206f920001
2018-09-14 08:31:23,248 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /100.105.123.61:37766 which had sessionid 0x265d7206f920001
2018-09-14 08:31:26,102 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /127.0.0.1:50326
2018-09-14 08:31:26,102 [myid:2] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@883] - Processing ruok command from /127.0.0.1:50326

按如下所述配置SCRAM配置:
此处:https://docs.confluent.io/current/kafka/authentication_sasl/authentication_sasl_scram.html

jaas配置定义为

KafkaServer {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="admin"
    password="admin-secret";
};

0 个答案:

没有答案