oozie 4.3.0构建失败

时间：2017-03-24 14:12:16

标签： sqoop oozie

我需要使用sqoop-1.4.6参数将hive表导出到mysql，其中hive表有两个分区。但我发现我的hadoop集群上的oozie共享lib只支持sqoop-1.4.3。所以我尝试构建和创建符合我们版本规范的共享库。

我修改了oozie pom.xml作为我们的集群主节点的情况，但是如果我做了以下两个更改，则构建失败：

  1. 如果我将hive版本Ln 92从<hive.version>1.2.0</hive.version>更改为<hive.version>2.1.0</hive.version>，我会收到以下错误：
    2.   

    块引用

    [INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Oozie Main .................................. SUCCESS [  0.991 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [  1.843 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [  0.215 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [  0.432 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [  0.017 s]
[INFO] Apache Oozie Client ................................ SUCCESS [ 12.492 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [  1.868 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [  8.001 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [  0.733 s]
[INFO] Apache Oozie Core .................................. FAILURE [ 16.672 s]
[INFO] Apache Oozie Share Lib Streaming ................... SKIPPED
[INFO] Apache Oozie Share Lib Pig ......................... SKIPPED
[INFO] Apache Oozie Share Lib Hive ........................ SKIPPED
[INFO] Apache Oozie Share Lib Hive 2 ...................... SKIPPED
[INFO] Apache Oozie Share Lib Sqoop ....................... SKIPPED
[INFO] Apache Oozie Examples .............................. SKIPPED
[INFO] Apache Oozie Share Lib Spark ....................... SKIPPED
[INFO] Apache Oozie Share Lib ............................. SKIPPED
[INFO] Apache Oozie Docs .................................. SKIPPED
[INFO] Apache Oozie WebApp ................................ SKIPPED
[INFO] Apache Oozie Tools ................................. SKIPPED
[INFO] Apache Oozie MiniOozie ............................. SKIPPED
[INFO] Apache Oozie Distro ................................ SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests .............. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44.067 s
[INFO] Finished at: 2017-03-27T17:36:56+08:00
[INFO] Final Memory: 193M/1464M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile (default-testCompile) on project oozie-core: Compilation failure: Compilation failure:
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[817,47] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[818,32] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java:[819,78] error: cannot find symbol
[ERROR] variable firstLogEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[213,46] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[214,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[215,81] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[221,46] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[222,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[231,31] error: cannot find symbol
[ERROR] variable logEntry of type LoggingEvent
[ERROR] /tmp/oozie-4.3.0/core/src/test/java/org/apache/oozie/service/TestEventHandlerService.java:[239,31] error: cannot find symbol
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :oozie-core
    1. 如果我将hbase版本Ln 85从<hbase.version>0.94.27</hbase.version>更改为<hbase.version>1.1.1</hbase.version>，则会收到以下错误：
      2.   

      块引用

      [INFO] ------------------------------------------------------------------------
[INFO] Building Apache Oozie Core 4.3.0
[INFO] ------------------------------------------------------------------------
Downloading: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/hbase/hbase/1.1.1/hbase-1.1.1.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Oozie Main .................................. SUCCESS [  1.008 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [  1.869 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [  0.094 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [  0.287 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [  0.026 s]
[INFO] Apache Oozie Client ................................ SUCCESS [ 14.418 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [  2.331 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [  2.478 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [  0.728 s]
[INFO] Apache Oozie Core .................................. FAILURE [  0.755 s]
[INFO] Apache Oozie Share Lib Streaming ................... SKIPPED
[INFO] Apache Oozie Share Lib Pig ......................... SKIPPED
[INFO] Apache Oozie Share Lib Hive ........................ SKIPPED
[INFO] Apache Oozie Share Lib Hive 2 ...................... SKIPPED
[INFO] Apache Oozie Share Lib Sqoop ....................... SKIPPED
[INFO] Apache Oozie Examples .............................. SKIPPED
[INFO] Apache Oozie Share Lib Spark ....................... SKIPPED
[INFO] Apache Oozie Share Lib ............................. SKIPPED
[INFO] Apache Oozie Docs .................................. SKIPPED
[INFO] Apache Oozie WebApp ................................ SKIPPED
[INFO] Apache Oozie Tools ................................. SKIPPED
[INFO] Apache Oozie MiniOozie ............................. SKIPPED
[INFO] Apache Oozie Distro ................................ SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests .............. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24.879 s
[INFO] Finished at: 2017-03-27T18:01:13+08:00
[INFO] Final Memory: 110M/1420M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project oozie-core: Could not resolve dependencies for project org.apache.oozie:oozie-core:jar:4.3.0: Could not find artifact org.apache.hbase:hbase:jar:1.1.1 in alimaven (http://maven.aliyun.com/nexus/content/repositories/central/) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :oozie-core

      我正在使用附近的maven中央镜子和代理，但这不应该是问题，因为我也试过美国镜子，结果是一样的。到目前为止，我已经尽力查找文档，相关帖子，但没有运气。

      拜托，请一位，请赐教。

1 个答案:

答案 0 :(得分：0)

我无法找到构建问题的正确解决方案，所以我使用了一个丑陋的解决方法。首先解压oozie-sharelib-4.3.0.tar.gz，替换导致我的应用程序出现问题的库，然后使用oozie-setup.sh sharelib create -fs hdfs://<HDFS master node>:9000 -locallib oozie-sharelib-4.3.0.tar.gz更新共享库。

然而，这也不是正确的方法，因为那时oozie要求一个看似无穷无尽的罐子列表。

我终于回到使用shell动作从bash执行sqoop，我不得不解决一个非常奇怪的jar版本碰撞问题，这完全超出了这个问题的范围。

我希望丑陋的工作可以帮助处于紧急情况的人。

相关问题
最新问题