java.io.IOException:找不到用户配置单元的组

时间:2018-04-06 19:42:25

标签: hive hiveql usergroups

我有一个简单的测试应用程序来设置与Hive(简单身份验证)的会话。

CliSessionState localSession = new CliSessionState(hiveConf)
localSession.setIsHiveServerQuery(true)
localSession.setCurrentDatabase("LOOM")
Driver driver = new Driver(hiveConf)
SessionState.start(localSession)   ///////////////////////////// 1.

List data = new ArrayList()
CommandProcessorResponse response = driver.run("SHOW TABLES") // 2. 
driver.getResults(data)
data.each {
    println "${it}"   // I skipped table list in log output
}

driver.close()
if (localSession != null) {
    localSession.close()       ///////////////////////////////// 3.
}

一切正常。我在数据库LOOM中有一个表列表。没有错误或什么的。但是,日志输出看起来很可疑。有很多行如:

  

org.apache.hadoop.security.UserGroupInformation - 无法通过java.io.IOException获取用户配置单元的组:找不到用户配置单元的组

它是否表示我的代码,配置单元设置或其他问题?我该怎么做才能摆脱这些消息?

以下是应用日志。评论为1.2.3.的行会生成相应的行块:

1.

org.apache.hadoop.security.UserGroupInformation - hadoop login
org.apache.hadoop.security.UserGroupInformation - hadoop login commit
org.apache.hadoop.security.UserGroupInformation - Using user: "hive" with name hive
org.apache.hadoop.security.UserGroupInformation - User entry: "hive"
org.apache.hadoop.security.UserGroupInformation - Assuming keytab is managed externally since logged in from subject.
org.apache.hadoop.security.UserGroupInformation - UGI loginUser:hive (auth:SIMPLE)
org.apache.hadoop.security.UserGroupInformation - Failed to get groups for user hive by java.io.IOException: No groups found for user hive
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection

2.

org.apache.hadoop.security.UserGroupInformation - Failed to get groups for user hive by java.io.IOException: No groups found for user hive
org.apache.hadoop.security.UserGroupInformation - Failed to get groups for user hive by java.io.IOException: No groups found for user hive
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection

3.

org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection

更新

  • 日志是这个应用程序的STDOUT / STDERR我在intellij idea中本地启动了(在集群上安装hadoop / hive我用* -site.xml' s连接)

  • 我实际上在服务器上的组hive中有一个本地用户hadoop

  • 我在intellij idea中设置IOException的断点并打开stacktrace的输出。以下是我的特定异常实例的堆栈跟踪:

    Breakpoint reached
          at org.apache.hadoop.security.Groups.getGroups(Groups.java:210)
          at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1721)
          at org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator.setConf(HadoopDefaultAuthenticator.java:64)
          at org.apache.hadoop.hive.ql.security.ProxyUserAuthenticator.setConf(ProxyUserAuthenticator.java:47)
          at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
          at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
          at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthenticator(HiveUtils.java:441)
          at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:759)
          at org.apache.hadoop.hive.ql.session.SessionState.getAuthorizationMode(SessionState.java:1543)
          at org.apache.hadoop.hive.ql.session.SessionState.isAuthorizationModeV2(SessionState.java:1554)
          at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:635)
          at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:510)
          at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:320)
          at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1219)
          at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1260)
          at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1156)
          at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1146)
          at org.apache.hadoop.hive.ql.processors.CommandProcessor$run.call(Unknown Source:-1)

UPDATE2: 有Hive安全设置:

hadoop.security.authentication=simple
hive.server2.authentication=NONE

hadoop.security.group.mapping=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback
hadoop.user.group.static.mapping.overrides=dr.who=;

0 个答案:

没有答案