Question

我通过使用spark streaming将数据插入到启用kerberos的hbase中编写了一个程序。在一批中，我遇到了一个失败的任务。错误如下：

java.io.IOException: Login failure for myuser@example.com from keytab ./user.keytab
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1160)
    at com.framework.common.HbaseUtil$.InsertToHbase(HbaseUtil.scala:81)
    at com.framework.realtime.RDDUtil$$anonfun$dwsTodwd$2.apply(RDDUtil.scala:203)
    at com.framework.realtime.RDDUtil$$anonfun$dwsTodwd$2.apply(RDDUtil.scala:202)
    at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
    at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.auth.login.LoginException: Receive timed out
    at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:767)
    at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584)
    at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1149)
    ... 13 more
Caused by: java.net.SocketTimeoutException: Receive timed out
    at java.net.PlainDatagramSocketImpl.receive0(Native Method)
    at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:146)
    at java.net.DatagramSocket.receive(DatagramSocket.java:816)
    at sun.security.krb5.internal.UDPClient.receive(NetClient.java:207)
    at sun.security.krb5.KdcComm$KdcCommunication.run(KdcComm.java:390)
    at sun.security.krb5.KdcComm$KdcCommunication.run(KdcComm.java:343)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.krb5.KdcComm.send(KdcComm.java:327)
    at sun.security.krb5.KdcComm.send(KdcComm.java:219)
    at sun.security.krb5.KdcComm.send(KdcComm.java:191)
    at sun.security.krb5.KrbAsReqBuilder.send(KrbAsReqBuilder.java:319)
    at sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:364)
    at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:735)
    ... 25 more

但是在第二次尝试中，任务成功了。在我看来，认证过程太长，所以它失败了，而在另一次尝试中，这个过程很短。所以它就是scceed。我对么？如果是这样，请问如何解决这个问题？我的代码如下：

val ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(princ,
      keytab)

    ugi.doAs(new PrivilegedAction[Unit]() {
      def run(): Unit = {
        // TODO Auto-generated method stub
        var conn: HConnection = null
        var htable: HTableInterface = null

          conn = HConnectionManager.createConnection(conf)
          htable = conn.getTable(tableName)
          htable.setAutoFlushTo(false)
          for (record <- partitionOfRecords) {
             htable.put(record)
          }
      }
    })

Answer 1

从 Hadoop and Kerberos - the Madness beyond the Gate 章＆＃34;错误消息恐惧＆＃34; ......

收到超时

通常在像
这样的堆栈跟踪中
Caused by: java.net.SocketTimeoutException: Receive timed out
  at java.net.PlainDatagramSocketImpl.receive0(Native Method)
  ...
  at sun.security.krb5.internal.UDPClient.receive(NetClient.java:207)


... UDP套接字......切换到TCP - 至少，它会失败   更快。

就在上面：

将kerberos切换为使用TCP而不是UDP   在/etc/krb5.conf：


[libdefaults]
  udp_preference_limit = 1

一般来说，许多不稳定的Kerberos问题似乎只发生在UDP上，所以不幸的是它默认使用它......

请注意，Java还支持 kdc_timeout 配置参数，但这是一个肮脏的混乱：

MIT Kerberos documentation
在Unix / Linux文档中未提及 for BSD
仅在darkest corners of Java documentation, here for Java 9中提及，并且有一个有趣的侧面说明，默认值已从30s表达式隐式毫秒变为30秒
几个星期前，Cloudera支持团队发布了有关该设置的建议 - 因为30秒默认超时可能会导致HDFS高可用性或类似情况下的级联故障 - 但可怜的家伙他们真的不知道他们推荐什么，所以他们随机提出了＃34; 3＆＃34;或＆＃34; 3s＆＃34;或＆＃34; 3000＆＃34;显式超时值

另请注意，如果您有多个 KDC用于高可用性，并且这些KDC明确列在krb5.conf中（或通过DNS别名隐式列出）例如，使用循环规则设置）然后在＆＃34; KDC超时＆＃34; Java应该重新使用下一个KDC。除非你已达到全球超时。

java.io.IOException：从keytab登录myuser@example.com失败

1 个答案: