Question

我试图从Oozie启动一个shell作业，我将文件从hdfs映射到Hbase

shell脚本可以从命令行运行，但不能从Oozie

运行

Shell脚本：

HADOOP_CLASSPATH=`/usr/bin/hbase classpath` /usr/bin/hadoop jar /usr/lib/hbase/hbase-server.jar importtsv -Dimporttsv.separator=, -Dimporttsv.columns=ORIGINAL:FIELD1,ORIGINAL:FIELD2,ORIGINAL:FIELD3  HBASE_Table /user/U0H8048/file

错误是：

SLF4J: Class path contains multiple SLF4J bindings.
  SLF4J: Found binding in [jar:file:/cloudera/opt/cloudera/parcels/CDH-5.0.2-    1.cdh5.0.2.p0.13/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  SLF4J: Found binding in [jar:file:/cloudera/yarn/nm/filecache/2282/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  SLF4J: Found binding in [jar:file:/cloudera/yarn/nm/filecache/2288/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" java.lang.reflect.InvocationTargetException
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:54)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
  Caused by: java.io.FileNotFoundException: File does not exist: hdfs://salcls0226.com:8020/cloudera/opt/cloudera/parcels/CDH-5.0.2-1.cdh5.0.2.p0.13/lib/hbase/lib/hbase-server-0.96.1.1-cdh5.0.2.jar
  at    org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
  at    org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
      ...

有谁知道可能的解决方案？

如何指定hbase-server-0.96.1.1-cdh5.0.2.jar文件位于本地路径而不是hdfs？

由于

Answer 1

我解决了这个问题。

当工作流启动时，需要将行oozie.use.system.libpath=true 添加到Oozie读取的job.properties 文件中。

您必须将所有需要的文件添加到HDFS路径： / user / oozie / share / lib 。通常主库文件已存在。

使用Oozie将hdfs文件映射到HBase

1 个答案: