Apache Phoenix jdbc驱动程序连接QueryUtil更改为localhost

时间:2017-01-12 21:24:41

标签: hadoop hbase kerberos cloudera phoenix

在Java程序中,我尝试使用Apache Phoenix jdbc驱动程序在Cloudera / Kerberos环境中连接到我们的HBase。 问题似乎是在连接期间它停止使用zookeeper仲裁和开关(或默认值?)回到

中的localhost
"INFO util.QueryUtil: Creating connection with the jdbc url: jdbc:phoenix:localhost:2181:/hbase"

步骤。

连接java代码行是:

connection = DriverManager.getConnection("jdbc:phoenix:name1,name2,name3:2181:/hbase:username@XXX.XXX:/home/username/username.keytab", props);

道具是:

HBaseConfiguration config = new HBaseConfiguration();
config.clear();
config.set("hbase.zookeeper.quorum", "name1:2181,name2:2181,name3:2181");
config.set("hbase.zookeeper.property.clientPort","2181");
config.addResource("/etc/hbase/conf/core-site.xml");
config.addResource("/etc/hbase/conf/hbase-site.xml");
config.addResource("/etc/hbase/conf/hdfs-site.xml");

Properties props = ConfigurationUtil.toProperties(config);
props.setProperty("phoenix.trace.frequency", "always");

1。运行PhoenixExample的命令是:

$ hadoop jar /home/me/workspace/phoenix-example-master/phoenixexample.jar PhoenixExample

...
17/01/11 19:44:35 INFO query.ConnectionQueryServicesImpl: Trying to connect to a secure cluster with keytab:/home/username/username.keytab                                  
17/01/11 19:44:35 INFO security.UserGroupInformation: Login successful for user username@XXX.XXX using keytab file /home/username/username.keytab                   
17/01/11 19:44:35 INFO query.ConnectionQueryServicesImpl: Successfull login to secure cluster!!                                                                             
17/01/11 19:44:35 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x54b219e4 connecting to ZooKeeper ensemble=name1:2181,name2:2181,name3:2181          
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.9.0--1, built on 10/21/2016 08:08 GMT                                           
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:host.name=hostname                                                                              
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_67                                                                                        
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation                                                                               
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.7.0_67-cloudera/jre 
...
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/lib/native                                                                         
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp                                                                                                                                              
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>                                                                                                                                               
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux                                                                                                                                                    
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64                                                                                                                                                    
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-573.18.1.el6.x86_64                                                                                                                            
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:user.name=username                                                                                                                                               
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/username                                                                                                                                         
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/me/workspace/phoenix-example-master                                                                                                         
17/01/11 19:44:35 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=name1:2181,name2:2181,name3:2181 sessionTimeout=90000 watcher=hconnection-0x54b219e40x0, quorum=name1:2181,name2:2181,name3:2181, baseZNode=/hbase                                                                                                                                                                                                                      
17/01/11 19:44:35 INFO zookeeper.ClientCnxn: Opening socket connection to server name2/XXX.XXX.XXX.XXX:2181. Will not attempt to authenticate using SASL (unknown error)                                                        
17/01/11 19:44:35 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /XXX.XXX.XXX.XXX:48959, server: name2/XXX.XXX.XXX.XXX:2181                                                              
17/01/11 19:44:35 INFO zookeeper.ClientCnxn: Session establishment complete on server name2/XXX.XXX.XXX.XXX:2181, sessionid = 0x258b142e7723f03, negotiated timeout = 60000                                                     
17/01/11 19:44:36 INFO metrics.Metrics: Initializing metrics system: phoenix                                                                                                                                                    
17/01/11 19:44:36 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2-phoenix.properties                                                                                                                            
17/01/11 19:44:36 INFO trace.PhoenixMetricsSink: Writing tracing metrics to phoenix table                                                                                                                                       
17/01/11 19:44:36 INFO trace.PhoenixMetricsSink: Phoenix tracing writer started                                                                                                                                                 
17/01/11 19:44:36 INFO impl.MetricsSinkAdapter: Sink tracing started                                                                                                                                                            
17/01/11 19:44:36 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).                                                                                                                                       
17/01/11 19:44:36 INFO impl.MetricsSystemImpl: phoenix metrics system started                                                                                                                                                   
17/01/11 19:44:36 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available                                                                                                         
17/01/11 19:44:46 INFO util.QueryUtil: Creating connection with the jdbc url: jdbc:phoenix:localhost:2181:/hbase;                                                                                                               
17/01/11 19:44:46 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum                                                                                                      
17/01/11 19:44:46 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum                                                                                                      
17/01/11 19:44:46 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x266c806b connecting to ZooKeeper ensemble=localhost:2181                                                                                
17/01/11 19:44:46 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0x266c806b0x0, quorum=localhost:2181, baseZNode=/hbase                          
17/01/11 19:44:46 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost.localdomain/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)                                              
17/01/11 19:44:46 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect                                                                                  
java.net.ConnectException: Connection refused                                                                                                                                                                                   
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)                                                                                                                                                             
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)                                                                                                                                               
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)                                                                                                                                   
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)                                                                                                                                                 
17/01/11 19:44:47 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost.localdomain/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)

注意:如果应用程序在没有任何属性的情况下运行,则会出现上面显示的相同结果。

2。按照建议,我可以让sqlline.py进行连接,但是当它尝试创建表时,它会失败,因为我们的HBase对我和大多数用户是只读的< / p>

$ cd /opt/cloudera/parcels/CLABS_PHOENIX-4.7.0-1.clabs_phoenix1.3.0.p0.000/lib/phoenix/bin
$ ./sqlline.py name1,name2,name3:2181:/hbase:username@XXX.XXX/home/username/username.keytab

Setting property: [incremental, false]                                                                                     
Setting property: [isolation, TRANSACTION_READ_COMMITTED]                                                                  
issuing: !connect jdbc:phoenix:name1,name2,name3:2181:/hbase:username@XXX.XXX:/home/username/username.keytab none none org.apache.phoenix.jdbc.PhoenixDriver                                                                                                                                            
Connecting to jdbc:phoenix:name1,name2,name3:2181:/hbase:username@XXXXX.XXX:/home/username/username.keytab                                        
SLF4J: Class path contains multiple SLF4J bindings.                                                                                                     
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CLABS_PHOENIX-4.7.0-1.clabs_phoenix1.3.0.p0.000/lib/phoenix/phoenix-4.7.0-clabs-phoenix1.3.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]                                                                                                       
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]                                                                                                                                                       
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.                                                                        
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]                                                                                    
17/01/06 15:58:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable     
Error: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions (user=username@XXX.XXX, scope=default, params=[namespace=default,table=default:SYSTEM.CATALOG,family=0],action=CREATE)                                                                                          
        at org.apache.hadoop.hbase.security.access.AccessController.requireNamespacePermission(AccessController.java:633)                               
        at org.apache.hadoop.hbase.security.access.AccessController.preCreateTable(AccessController.java:1002)                                          
        at org.apache.hadoop.hbase.master.MasterCoprocessorHost$11.call(MasterCoprocessorHost.java:216)                                                 
        at org.apache.hadoop.hbase.master.MasterCoprocessorHost.execOperation(MasterCoprocessorHost.java:1142)                                          
        at org.apache.hadoop.hbase.master.MasterCoprocessorHost.preCreateTable(MasterCoprocessorHost.java:212)                                          
        at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1521)                                                                        
        at org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)                                                     
        at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55682)                          
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)                                                                              
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)                                                                              
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)                                                                    
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165) (state=08000,code=101)                                             
org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions (user=username@XXX.XXX, scope=default, params=[namespace=default,table=default:SYSTEM.CATALOG,family=0],action=CREATE)

3. 我打印出了sqlline.py java命令字符串,并尝试使用这个为PhoenixExample类量身定制的,但它在连接中的距离更远,如图所示:

$ java -cp "/etc/hbase/conf::::/etc/hadoop/conf:/etc/hadoop/conf:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/libexec/../../hadoop/lib/*:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/libexec/../../hadoop/.//*:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/libexec/../../hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/libexec/../../hadoop-hdfs/.//*:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/libexec/../../hadoop-yarn/lib/*:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/libexec/../../hadoop-yarn/.//*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//*" -Dlog4j.configuration=file:/home/me/workspace/phoenix-example-master/log4j.properties -jar /home/me/workspace/phoenix-example-master/phoenixexample.jar PhoenixExample

...
17/01/11 19:22:13 INFO query.ConnectionQueryServicesImpl: Successfull login to secure cluster!!                                                                                                                                
17/01/11 19:22:39 DEBUG security.UserGroupInformation: hadoop login                                                                                                                                                            
17/01/11 19:22:39 DEBUG security.UserGroupInformation: hadoop login commit                                                                                                                                                     
17/01/11 19:22:39 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: username                                                                                                                                
17/01/11 19:22:39 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: username" with name username                                                                                                                
17/01/11 19:22:39 DEBUG security.UserGroupInformation: User entry: "username"                                                                                                                                                  
17/01/11 19:22:39 DEBUG security.UserGroupInformation: UGI loginUser:username (auth:SIMPLE)                                                                                                                                    
17/01/11 19:22:39 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x712cd5d3 connecting to ZooKeeper ensemble=name1:2181,name2:2181,name3:2181                                                             
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.7.0--1, built on 04/17/2016 08:12 GMT                                                                                              
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:host.name=hostname                                                                                                                                 
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_60                                                                                                                                           
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation                                                                                                                                  
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.8.0_60/jre                                                                                                                             
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/home/me/workspace/phoenix-example-master/phoenixexample.jar                                                                              
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib                                                                                  
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp                                                                                                                                             
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>                                                                                                                                              
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux                                                                                                                                                   
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64                                                                                                                                                   
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-573.18.1.el6.x86_64                                                                                                                           
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:user.name=username                                                                                                                                              
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/username                                                                                                                                        
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/me/workspace/phoenix-example-master                                                                                                        
17/01/11 19:22:39 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=name1:2181,name2:2181,name3:2181 sessionTimeout=90000 watcher=hconnection-0x712cd5d30x0, quorum=name1:2181,name2:2181,name3:2181, baseZNode=/hbase                                                                      
17/01/11 19:22:39 DEBUG zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false                                                                                                                                         
17/01/11 19:22:39 INFO zookeeper.ClientCnxn: Opening socket connection to server name1/XXX.XXX.XXX.XXX:2181. Will not attempt to authenticate using SASL (unknown error)                                                       
17/01/11 19:22:39 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /XXX.XXX.XXX.XXX:45766, server: name1/XXX.XXX.XXX.XXX:2181                                                             
17/01/11 19:22:39 DEBUG zookeeper.ClientCnxn: Session establishment request sent on name1/XXX.XXX.XXX.XXX:2181                                                                                                                 
17/01/11 19:22:39 INFO zookeeper.ClientCnxn: Session establishment complete on server name1/XXX.XXX.XXX.XXX:2181, sessionid = 0x158b142e83d40f9, negotiated timeout = 60000                                                    
17/01/11 19:22:39 DEBUG zookeeper.ZooKeeperWatcher: hconnection-0x712cd5d30x0, quorum=name1:2181,name2:2181,name3:2181, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null                   
17/01/11 19:22:39 DEBUG zookeeper.ZooKeeperWatcher: hconnection-0x712cd5d3-0x158b142e83d40f9 connected                                                                                                                         
17/01/11 19:22:41 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x158b142e83d40f9, packet:: clientPath:null serverPath:null finished:false header:: 1,3  replyHeader:: 1,180389695832,0  request:: '/hbase/hbaseid,F  response:: s{8589935364,180388782726,1468957667066,1480986044766,59,0,50,0,67,0,8589935364}                                                                                                                                       
17/01/11 19:22:41 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x158b142e83d40f9, packet:: clientPath:null serverPath:null finished:false header:: 2,4  replyHeader:: 2,180389695832,0  request:: '/hbase/hbaseid,F  response:: #ffffffff000146d61737465723a3630303030ffffffcfffffff9dfffffff2ffffff81ffffffe138773d50425546a2466313333333263372d653238382d343635652d383130302d663330613933633934643264,s{8589935364,180388782726,1468957667066,1480986044766,59,0,50,0,67,0,8589935364}                                                                                                                                                                                              
^C^C17/01/11 19:23:01 DEBUG zookeeper.ClientCnxn: Got ping response for sessionid: 0x158b142e83d40f9 after 1ms                                                                                                                 
17/01/11 19:23:06 DEBUG core.Tracer: sampler.classes = ; loaded no samplers                                                                                                                                                    
17/01/11 19:23:06 DEBUG core.Tracer: span.receiver.classes = ; loaded no span receivers                                                                                                                                        
17/01/11 19:23:21 DEBUG zookeeper.ClientCnxn: Got ping response for sessionid: 0x158b142e83d40f9 after 0ms

其他信息:

正在使用虚拟主机。

$ cat core-site.xml                                                                                                     
<?xml version="1.0" encoding="UTF-8"?>                                                                                                                                        
<!--Autogenerated by Cloudera Manager-->
<configuration>                         
  <property>                            
    <name>fs.defaultFS</name>           
    <value>hdfs://afnameservice</value> 
  </property> 
...

$ cat hbase-site.xml                                                                                                                           
<?xml version="1.0" encoding="UTF-8"?> 
<!--Autogenerated by Cloudera Manager-->
<configuration>                         
  <property>                            
    <name>hbase.rootdir</name>          
    <value>hdfs://afnameservice/hbase</value>
  </property> 

...

有人可以提供建议吗?

由于

0 个答案:

没有答案