我正试图让kafka spark流消费者与使用SSL保护的Kafka经纪人合作。使用者在Kerberized Hadoop集群上运行。 因此，我的使用者代码使用以下配置：
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="(username)" password="(password)";
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
使用者失败，但出现以下异常
Caused by: java.lang.NullPointerException
at org.apache.kafka.common.security.plain.PlainSaslServer$PlainSaslServerFactory.getMechanismNames(PlainSaslServer.java:163) ~[kafka-clients-0.10.0-kafka-2.1.0.jar:?]
at org.apache.hadoop.security.SaslRpcServer$FastSaslServerFactory.<init>(SaslRpcServer.java:381) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.security.SaslRpcServer.init(SaslRpcServer.java:186) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:570) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.hdfs.NameNodeProxies.createNNProxyWithClientProtocol(NameNodeProxies.java:418) ~[hadoop-hdfs-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:314) ~[hadoop-hdfs-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider$DefaultProxyFactory.createProxy(ConfiguredFailoverProxyProvider.java:68) ~[hadoop-hdfs-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider.getProxy(ConfiguredFailoverProxyProvider.java:152) ~[hadoop-hdfs-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:75) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:66) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.io.retry.RetryProxy.create(RetryProxy.java:58) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:181) ~[hadoop-hdfs-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:730) ~[hadoop-hdfs-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:673) ~[hadoop-hdfs-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:155) ~[hadoop-hdfs-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2816) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:98) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2853) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2835) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:387) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) ~[hadoop-common-2.6.0-cdh5.12.1.jar:?]
at org.apache.spark.internal.io.SparkHadoopWriter.open(SparkHadoopWriter.scala:82) ~[spark-core_2.11-2.2.0.cloudera2.jar:2.2.0.cloudera2]
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1133) ~[spark-core_2.11-2.2.0.cloudera2.jar:2.2.0.cloudera2]
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1125) ~[spark-core_2.11-2.2.0.cloudera2.jar:2.2.0.cloudera2]
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) ~[spark-core_2.11-2.2.0.cloudera2.jar:2.2.0.cloudera2]
at org.apache.spark.scheduler.Task.run(Task.scala:108) ~[spark-core_2.11-2.2.0.cloudera2.jar:2.2.0.cloudera2]
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:338) ~[spark-core_2.11-2.2.0.cloudera2.jar:2.2.0.cloudera2]
... 3 more
经过一些初步分析，我发现hadoop-commons SaslRpcServer类在其FastSaslServerFactory中具有空属性映射。 kafka-clients中的PlainSaslServer类尝试从该映射中获取通向NPE的地址。
我想念什么吗？网上没有太多关于此问题的文章，因此希望获得一些指点。
答案 0 :(得分：0)
我已经解决了这个问题。此问题是由kafka-clients版本引起的。您可以将kafka-clients版本升级到0.11。详细信息为here。
kafka版本0.11.0的客户可以与版本0.10.0或更新的代理进行对话。这样您就可以放心升级！