Question

我正在尝试从spark scala外壳连接到以kerberized保护的Hbase集群，以下是我的代码，感谢您提供有关错误的帮助。我正在使用—文件

在spark shell中传递hdfs-site.xml，hbase-site.xml，core-site.xml和我的keytab

strategy.entry

org.apache.hadoop.hbase.classification.InterfaceAudience找不到-继续存根。

错误- -

import org.apache.hadoop.conf.Configuration 
import org.apache.hadoop.hbase.HBaseConfiguration
import org.apache.hadoop.hbase.client.Connection
import org.apache.hadoop.hbase.client.ConnectionFactory 
import org.apache.hadoop.hbase.TableName
import org.apache.hadoop.security.UserGroupInformation

val conf: Configuration = HBaseConfiguration.create()
conf.set("hbase.zookeeper.quorum", "xxxxx1@abc.com,xxxxx2@abc.com,xxxxx3@abc.com")
conf.set("zookeeper.znode.parent", "/hbase-secure") 
conf.setInt("hbase.client.scanner.caching", 10000)
conf.set("hbase.rpc.controllerfactory.class","org.apache.hadoop.hbase.ipc.RpcControllerFactory")
conf.set("hbase.rpc.controllerfactory.class","org.apache.hadoop.hbase.ipc.RpcControllerFactory")
conf.set("hadoop.security.authentication", "kerberos")
conf.set("hbase.security.authentication", "kerberos")
val userGroupInformation = UserGroupInformation.loginUserFromKeytabAndReturnUGI("XXX@abc.COM", "/u/xxxxx/XXXX.keytab")
UserGroupInformation.setLoginUser(userGroupInformation)
val connection: Connection = ConnectionFactory.createConnection(conf)
print(connection)
val admin = connection.getAdmin
val listtables = admin.listNamespaceDescriptors()
]

I see a lot of warning in the process as below 
warning: Class

Answer 1

您的Zookeeper配置缺少Zookeeper端口。添加

conf.set("hbase.zookeeper.property.clientPort", "2181")

我想建议您一些其他事情，

此后，您可能会遇到火花配置问题。复制hdfs-site.xml，hbase-site.xml，core-site.xml和yarn-site.xml（如果使用的是yarn）以触发conf文件夹。

向hbase配置对象添加资源

conf.addResource("/path/to/hbase-site.xml");

设置Java属性

//Point to the krb5.conf file. Enable Kerberos debug.
System.setProperty("java.security.krb5.conf", "/etc/krb5.conf");
System.setProperty("sun.security.krb5.debug", "true");

希望这会帮助...

来自Spark Scala的Hbase Kerberos连接

1 个答案: