在Cloudera 4.7.0群集上的hbase类路径中找不到jar

时间：2015-08-24 17:06:29

标签： hbase cloudera cloudera-cdh phoenix

我正在升级运行hbase版本0.94.15-cdh-4.7.0的CDH 4.7.0群集上使用的Apache Phoenix版本。我的目标是将我们从Phoenix 3.1.0迁移到3.3.1，但是当我升级到版本3.2.2时，我的连接语句开始失败。

Phoenix Upgrade Documentation充其量只是简洁，但我的理解是使用Cloudera Manager推出新升级的jar包，然后连接到具有升级客户端的群集应该是升级所需的全部内容从一个版本到另一个版本。

升级后，我的所有测试都会通过保存连接测试。一个简单的连接语句：

SELECT A."id", A."group", B."group", B."name" 
  FROM "Test_Table_1" AS A 
  INNER JOIN "Test_Table_2" AS B 
  ON A."group" = B."group" 
  LIMIT 10

导致以下异常（发布结束时的完整堆栈跟踪）：

Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=14, exceptions:
Mon Aug 24 09:21:18 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException

在版本3.2中，phoenix将其版本的snappy升级为使用org.iq80.snappy版本0.3，但jar tf phoenix-core-3.2.2.jar未显示任何snappy类。这让我觉得类似路径中缺少了那个活泼的jar，尽管它是凤凰3.2 pom的依赖。这是麻烦开始的地方：我可以将snappy-0.3.jar添加到我的hbase集群中，但无法弄清楚如何修改类路径以包含jar。

我在Cloudera的HBase服务环境高级配置（安全阀）中包含以下内容：     HBASE_CLASSPATH="/opt/cloudera/parcels/phoenix_core/lib/java/*:$HBASE_CLASSPATH"

请注意，我的凤凰jar在/opt/cloudera/parcels/phoenix_core/lib/java目录中，所以我知道这个目录在hbase类路径中，即使我在其中一个上输入hbase classpath到终端时没有显示它数据节点。

因此，在此背景下，如何验证snappy-0.3.jar是否在类路径中，更广泛地说，如何让我的连接语句正常工作？

加入时的完整堆栈跟踪：

java.sql.SQLException: Encountered exception in sub plan [0] execution.
    at org.apache.phoenix.execute.HashJoinPlan.iterator(HashJoinPlan.java:151)
    at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:230)
    at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:221)
    at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:54)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:221)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:1032)
    at MainTest.execute(MainTest.java:87)
    at MainTest.join1(MainTest.java:297)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: java.sql.SQLException: java.util.concurrent.ExecutionException: java.lang.reflect.UndeclaredThrowableException
    at org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:202)
    at org.apache.phoenix.join.HashCacheClient.addHashCache(HashCacheClient.java:83)
    at org.apache.phoenix.execute.HashJoinPlan$HashSubPlan.execute(HashJoinPlan.java:333)
    at org.apache.phoenix.execute.HashJoinPlan$1.call(HashJoinPlan.java:130)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.UndeclaredThrowableException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:202)
    at org.apache.phoenix.cache.ServerCacheClient.addServerCache(ServerCacheClient.java:194)
    ... 7 more
Caused by: java.lang.reflect.UndeclaredThrowableException
    at com.sun.proxy.$Proxy20.addServerCache(Unknown Source)
    at org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:172)
    at org.apache.phoenix.cache.ServerCacheClient$1.call(ServerCacheClient.java:167)
    ... 4 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=14, exceptions:
Mon Aug 24 09:21:18 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:20 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:22 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:24 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:27 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:31 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:36 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:21:44 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:22:01 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:22:34 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:23:38 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:24:43 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:25:48 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException
Mon Aug 24 09:26:52 PDT 2015, org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1dfa77a9, org.apache.hadoop.ipc.RemoteException(java.lang.NoClassDefFoundError): IPC server unable to read call parameters: org/iq80/snappy/CorruptionException

    at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:187)
    at org.apache.hadoop.hbase.ipc.ExecRPCInvoker.invoke(ExecRPCInvoker.java:79)
    ... 7 more

1 个答案:

答案 0 :(得分：1)

我已经想出了一个可以防止错误的黑客工作。我解压缩了snappy-0.3.jar文件，然后将生成的类文件添加到phoenix-core-3.3.1.jar

  1. jar xf snappy-0.3.jar
  2. jar uf phoenix-core-3.3.1.jar org/iq80/snappy/*
  3. 然后修改后的凤凰核心罐被推入一个新包裹，现在加入工作！

    4. 如果有人有一个更好的答案，不涉及“滚动我自己”可以这么说，我仍然希望听到它。

相关问题
最新问题