Question

您好我正在尝试为安全的hbase编写一个Java客户端。我想从代码本身做kinit，因为我使用的是用户组信息类。任何人都可以指出我在哪里错了吗？

这是我尝试连接o hbase的主要方法。

我必须在CONfiguration对象中添加配置而不是使用xml，因为客户端可以位于任何地方。

请参阅以下代码：

    public static void main(String [] args) {
    try {
        System.setProperty(CommonConstants.KRB_REALM, ConfigUtil.getProperty(CommonConstants.HADOOP_CONF, "krb.realm"));
        System.setProperty(CommonConstants.KRB_KDC, ConfigUtil.getProperty(CommonConstants.HADOOP_CONF,"krb.kdc"));
        System.setProperty(CommonConstants.KRB_DEBUG, "true");

        final Configuration config = HBaseConfiguration.create();

        config.set(CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHENTICATION, AUTH_KRB);
        config.set(CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION, AUTHORIZATION);
        config.set(CommonConfigurationKeysPublic.FS_AUTOMATIC_CLOSE_KEY, AUTO_CLOSE);
        config.set(CommonConfigurationKeysPublic.FS_DEFAULT_NAME_KEY, defaultFS);
        config.set("hbase.zookeeper.quorum", ConfigUtil.getProperty(CommonConstants.HBASE_CONF, "hbase.host"));
        config.set("hbase.zookeeper.property.clientPort", ConfigUtil.getProperty(CommonConstants.HBASE_CONF, "hbase.port"));
        config.set("hbase.client.retries.number", Integer.toString(0));
        config.set("zookeeper.session.timeout", Integer.toString(6000));
        config.set("zookeeper.recovery.retry", Integer.toString(0));
        config.set("hbase.master", "gauravt-namenode.pbi.global.pvt:60000");
        config.set("zookeeper.znode.parent", "/hbase-secure");
        config.set("hbase.rpc.engine", "org.apache.hadoop.hbase.ipc.SecureRpcEngine");
        config.set("hbase.security.authentication", AUTH_KRB);
        config.set("hbase.security.authorization", AUTHORIZATION);
        config.set("hbase.master.kerberos.principal", "hbase/gauravt-namenode.pbi.global.pvt@pbi.global.pvt");
        config.set("hbase.master.keytab.file", "D:/var/lib/bda/secure/keytabs/hbase.service.keytab");
        config.set("hbase.regionserver.kerberos.principal", "hbase/gauravt-datanode2.pbi.global.pvt@pbi.global.pvt");
        config.set("hbase.regionserver.keytab.file", "D:/var/lib/bda/secure/keytabs/hbase.service.keytab");

        UserGroupInformation.setConfiguration(config);
        UserGroupInformation userGroupInformation = UserGroupInformation.loginUserFromKeytabAndReturnUGI("hbase/gauravt-datanode2.pbi.global.pvt@pbi.global.pvt", "D:/var/lib/bda/secure/keytabs/hbase.service.keytab");
        UserGroupInformation.setLoginUser(userGroupInformation);

        User user = User.create(userGroupInformation);

        user.runAs(new PrivilegedExceptionAction<Object>() {

            @Override
            public Object run() throws Exception {
                HBaseAdmin admins = new HBaseAdmin(config);

                if(admins.isTableAvailable("ambarismoketest")) {
                    System.out.println("Table is available");
                };

                HConnection connection = HConnectionManager.createConnection(config);

                HTableInterface table = connection.getTable("ambarismoketest");



                admins.close();
                System.out.println(table.get(new Get(null)));
                return table.get(new Get(null));
            }
        });
        System.out.println(UserGroupInformation.getLoginUser().getUserName());


    } catch (Exception e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

我得到以下例外：

    Caused by: org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): GSS initiate failed
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.readStatus(HBaseSaslRpcClient.java:110)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:146)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupSaslConnection(RpcClient.java:762)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.access$600(RpcClient.java:354)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:883)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:880)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:880)
... 33 more

任何指针都会有所帮助。

Answer 1

上面的工作很好，但我看到很多人都在设置Configuration对象中所有正确的属性。没有事实上的列表，我已经找到了您需要和不需要的内容，并且它非常依赖于您的群集配置。

万无一失的方法是在类路径中包含HBase配置的副本，因为您的客户端可以在您提到的任何位置。然后，您可以将资源添加到对象，而无需指定所有属性。

Configuration conf = HBaseConfiguration.create();
conf.addResource("core-site.xml");
conf.addResource("hbase-site.xml");
conf.addResource("hdfs-site.xml");

以下是支持这种方法的一些消息来源： IBM， Scalding (Scala)

另请注意，此方法并不限制您实际使用内部Zookeeper主体和密钥表，即您可以为应用程序或Active Directory用户创建密钥表，并让守护进程内部生成的密钥表之间进行身份验证。

Answer 2

不确定您是否仍需要帮助。我认为你的代码片段中缺少设置“hadoop.security.authentication”属性。

我正在使用以下代码段连接到安全HBase（在CDH5上）。你可以尝试一下。

config.set("hbase.zookeeper.quorum", zookeeperHosts);
config.set("hbase.zookeeper.property.clientPort", zookeeperPort);
config.set("hadoop.security.authentication", "kerberos");
config.set("hbase.security.authentication", "kerberos");
config.set("hbase.master.kerberos.principal", HBASE_MASTER_PRINCIPAL);
config.set("hbase.regionserver.kerberos.principal", HBASE_RS_PRINCIPAL);

UserGroupInformation.setConfiguration(config);
UserGroupInformation.loginUserFromKeytab(ZOOKEEPER_PRINCIPAL,ZOOKEEPER_KEYTAB);

HBaseAdmin admins = new HBaseAdmin(config);
TableName[] tables  = admins.listTableNames();

for(TableName table: tables){
    System.out.println(table.toString());
}

Answer 3

in Jdk 1.8, you need set 
"System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");"

config.set("hbase.zookeeper.quorum", zookeeperHosts);
config.set("hbase.zookeeper.property.clientPort", zookeeperPort);
config.set("hadoop.security.authentication", "kerberos");
config.set("hbase.security.authentication", "kerberos");
config.set("hbase.master.kerberos.principal", HBASE_MASTER_PRINCIPAL);
config.set("hbase.regionserver.kerberos.principal", HBASE_RS_PRINCIPAL);
System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");
UserGroupInformation.setConfiguration(config);
UserGroupInformation.loginUserFromKeytab(ZOOKEEPER_PRINCIPAL,ZOOKEEPER_KEYTAB);

HBaseAdmin admins = new HBaseAdmin(config);
TableName[] tables  = admins.listTableNames();

for(TableName table: tables){
    System.out.println(table.toString());
}

quote:
http://hbase.apache.org/book.html#trouble.client 
question: 142.9

Answer 4

我认为最好的是https://scalding.io/2015/02/making-your-hbase-client-work-in-a-kerberized-environment/

要使代码正常工作，您无需更改本文顶部写的任何一行，只需使您的客户端能够访问完整的HBase配置即可。这仅意味着将您正在运行的类路径更改为：

/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase/conf:target/scala-2.11/hbase-assembly-1.0.jar 这将使一切顺利运行。它特定于CDH 5.3，但您可以将其调整为适合集群配置。

PS不需要：

conf.addResource("core-site.xml");
conf.addResource("hbase-site.xml");
conf.addResource("hdfs-site.xml");

因为HBaseConfiguration具有

  public static Configuration addHbaseResources(Configuration conf) {
conf.addResource("hbase-default.xml");
conf.addResource("hbase-site.xml");

用于Secure Hbase的Java客户端

4 个答案: