我正在做Spark2提交,通常用于在客户端模式下执行相同操作,并且从未遇到过任何Kerberos身份验证问题。但是现在当我开始在集群中尝试相同的代码时,我收到的身份验证问题如下:
WARN UserGroupInformation :PriviledgedActionException as:user_name(auth:SIMPLE)原因:javax.security.sasl.SaslException:GSS启动失败[由GSSException引起:未提供有效凭据(机制级别:失败)找到任何Kerberos tgt)]
WARN RpcClientImpl :连接到服务器时遇到异常:javax.security.sasl.SaslException:GSS启动失败[由GSSException引起:未提供有效凭据(机制级别:无法找到任何Kerberos) TGT)]
错误RpcClientImpl :SASL身份验证失败。最可能的原因是凭据丢失或无效。考虑一下' kinit'。 javax.security.sasl.SaslException:GSS启动失败[由GSSException引起:未提供有效凭据(机制级别:无法找到任何Kerberos tgt)] 在com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211) 在org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:181) at org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.setupSaslConnection(RpcClientImpl.java:618) at org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.access $ 700(RpcClientImpl.java:163) at org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection $ 2.run(RpcClientImpl.java:744) 在org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection $ 2.run(RpcClientImpl.java:741) at java.security.AccessController.doPrivileged(Native Method) 在javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1912) at org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.setupIOstreams(RpcClientImpl.java:741) at org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.writeRequest(RpcClientImpl.java:907) at org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.tracedWriteRequest(RpcClientImpl.java:874) 在org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1243) 在org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227) at org.apache.hadoop.hbase.ipc.AbstractRpcClient $ BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos $ ClientService $ BlockingStub.scan(ClientProtos.java:34094) 在org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:400) 在org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:204) 在org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:65) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas $ RetryingRPC.call(ScannerCallableWithReplicas.java:381) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas $ RetryingRPC.call(ScannerCallableWithReplicas.java:355) 在org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) at org.apache.hadoop.hbase.client.ResultBoundedCompletionService $ QueueingFuture.run(ResultBoundedCompletionService.java:80) 在java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617) 在java.lang.Thread.run(Thread.java:745) 引起:GSSException:未提供有效凭据(机制级别:无法找到任何Kerberos tgt) at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147) at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122) at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187) at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) 在com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) ......还有26个
我在shell脚本中提交spark提交,该脚本已经运行kinit命令,如下所示:
kinit -kt keytab_file.keytab user_name @ realm
另外,我已经在使用principal的spark submit中传递了keytab文件,并且我获得了成功的身份验证日志消息。当我使用keytab在代码中登录时,我面临的问题是:
hBaseConf = HBaseConfiguration.create()
hBaseConf.set("hadoop.security.authentication", "kerberos")
hBaseConf.set("hbase.security.authentication", "kerberos")
hBaseConf.set("hbase.zookeeper.quorum", NameNode)
hBaseConf.set("hbase.zookeeper.property.client.port", "2181")
//Login Using KeyTab
UserGroupInformation.setConfiguration(hBaseConf)
UserGroupInformation.loginUserFromKeytab(userName, keyTab)
对此的任何帮助都将受到高度赞赏!