我有一个简单的测试应用程序来设置与Hive(简单身份验证)的会话。
CliSessionState localSession = new CliSessionState(hiveConf)
localSession.setIsHiveServerQuery(true)
localSession.setCurrentDatabase("LOOM")
Driver driver = new Driver(hiveConf)
SessionState.start(localSession) ///////////////////////////// 1.
List data = new ArrayList()
CommandProcessorResponse response = driver.run("SHOW TABLES") // 2.
driver.getResults(data)
data.each {
println "${it}" // I skipped table list in log output
}
driver.close()
if (localSession != null) {
localSession.close() ///////////////////////////////// 3.
}
一切正常。我在数据库LOOM
中有一个表列表。没有错误或什么的。但是,日志输出看起来很可疑。有很多行如:
org.apache.hadoop.security.UserGroupInformation - 无法通过java.io.IOException获取用户配置单元的组:找不到用户配置单元的组
它是否表示我的代码,配置单元设置或其他问题?我该怎么做才能摆脱这些消息?
以下是应用日志。评论为1.
,2.
,3.
的行会生成相应的行块:
1.
org.apache.hadoop.security.UserGroupInformation - hadoop login org.apache.hadoop.security.UserGroupInformation - hadoop login commit org.apache.hadoop.security.UserGroupInformation - Using user: "hive" with name hive org.apache.hadoop.security.UserGroupInformation - User entry: "hive" org.apache.hadoop.security.UserGroupInformation - Assuming keytab is managed externally since logged in from subject. org.apache.hadoop.security.UserGroupInformation - UGI loginUser:hive (auth:SIMPLE) org.apache.hadoop.security.UserGroupInformation - Failed to get groups for user hive by java.io.IOException: No groups found for user hive org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
2.
org.apache.hadoop.security.UserGroupInformation - Failed to get groups for user hive by java.io.IOException: No groups found for user hive org.apache.hadoop.security.UserGroupInformation - Failed to get groups for user hive by java.io.IOException: No groups found for user hive org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
3.
org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
更新
日志是这个应用程序的STDOUT / STDERR我在intellij idea中本地启动了(在集群上安装hadoop / hive我用* -site.xml' s连接)
我实际上在服务器上的组hive
中有一个本地用户hadoop
。
我在intellij idea中设置IOException
的断点并打开stacktrace的输出。以下是我的特定异常实例的堆栈跟踪:
Breakpoint reached at org.apache.hadoop.security.Groups.getGroups(Groups.java:210) at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1721) at org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator.setConf(HadoopDefaultAuthenticator.java:64) at org.apache.hadoop.hive.ql.security.ProxyUserAuthenticator.setConf(ProxyUserAuthenticator.java:47) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthenticator(HiveUtils.java:441) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:759) at org.apache.hadoop.hive.ql.session.SessionState.getAuthorizationMode(SessionState.java:1543) at org.apache.hadoop.hive.ql.session.SessionState.isAuthorizationModeV2(SessionState.java:1554) at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:635) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:510) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:320) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1219) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1260) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1156) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1146) at org.apache.hadoop.hive.ql.processors.CommandProcessor$run.call(Unknown Source:-1)
UPDATE2: 有Hive安全设置:
hadoop.security.authentication=simple hive.server2.authentication=NONE hadoop.security.group.mapping=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback hadoop.user.group.static.mapping.overrides=dr.who=;