Question

我正在三台机器上的hadoop群集上设置一个配置单元。 hadoop（2.7.1）和德比（10.11）运行良好：

hduser@master:~$ ij
ij version 10.11
ij> connect 'jdbc:derby://localhost:1527/metastore_db;create=true';
ij> select * from a;
ID
-----------

0 rows selected
ij>

但是蜂巢抱怨：

...
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
        ... 8 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
        ... 14 more
Caused by: javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby://localhost:1527/metastore_db;create=true , username = APP. Terminating connectio pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
java.sql.SQLException: No suitable driver found for jdbc:derby://localhost:1527/metastore_db;create=true
...

环境变量：

# Java
export JAVA_HOME=/usr/lib/jvm/jdk

# derby

export DERBY_HOME=/usr/local/derby
export PATH=$PATH:$DERBY_HOME/bin
export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytools.jar

# Hadoop
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"


# hive

export HIVE_HOME=/usr/local/hive
export PATH=$PATH:$HIVE_HOME/bin
export CLASSPATH=$CLASSPATH:/usr/local/hive/lib/*:.

hive-site.xml在配置中只有这个属性：

  <property>
     <name>javax.jdo.option.ConnectionURL</name>
     <value>jdbc:derby://localhost:1527/metastore_db;create=true </value>
     <description>JDBC connect string for a JDBC metastore </description>
  </property>

发生了什么？

Answer 1

你说：

 <table>

    <thead>
      <tr>
         <th>Course Code</th>
         <th>Course Description</th>
         <th>Course Type</th>
         <th>Select Course</th>
      </tr>
    </thead>

    <tbody>
     <c:forEach  var="course" items= "${myObject.course_list}" >
      <tr>
         <td><c:out value="${course.code}"></c:out></td>
         <td><c:out value="${course.name}"></c:out></td>
         <td><c:out value="${course.courseType}"></c:out></td>
     </tr>
  </c:forEach>

</tbody>
</table>

但你也说过：

export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytools.jar

因此，您为Derby的客户端/服务器配置提供了JDBC连接URL，但您提供的CLASSPATH是针对Derby的嵌入式配置。

因此您收到错误：

jdbc:derby://localhost:1527/metastore_db;create=true

您可以更改CLASSPATH，也可以更改JDBC连接URL。

如果要使用Derby的客户端/服务器配置，请将CLASSPATH切换为：

No suitable driver found

因为此配置需要JDBC客户端驱动程序。

如果要使用Derby的嵌入式配置，请单独保留CLASSPATH并将JDBC连接URL更改为：

export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derbyclient.jar

要了解有关不同Derby配置的更多信息，请阅读：http://db.apache.org/derby/docs/10.11/getstart/cgsquck70629.html

Answer 2

当我在另一个终端选项卡中打开spark-shell会话时，当我在不同的终端选项卡中已经打开会话时，我开始看到此错误。关闭此旧终端选项卡，并在当前选项卡中再次启动spark-shell会话修复了该问题。

apache hive无法连接到derby：metastore_db虽然来自derby的ij可以

2 个答案: