Question

我成功创建并执行了sqoop导入作业，但无法在oozie工作流程中运行它。 Sqoop将数据从RDBMS导入到HDFS中的镶木地板文件中。似乎问题与镶木地板格式有关。如果我使用--as-textfile，工作流程会毫无问题地运行。

另外，我已经将所有的镶木地板 - * .jar从SQOOP_HOME / lib复制到oozie share lib。

Sqoop 1.4.7和Oozie 4.3.1。

Sqoop职位定义

$ sqoop job --create ingest_amsp_custmaster -- import --connect "jdbc:oracle:thin:@<IP>:<PORT>/<SID>" \
--username <USER> -P \
--table CUSTMASTER \
--as-parquetfile \
--target-dir /warehouse/raw/amsp/custmaster \
--delete-target-dir \
-m 1

以下是我从错误日志中获得的内容：

java.lang.IllegalArgumentException: No enum constant com.cloudera.sqoop.SqoopOptions.FileLayout.ParquetFile
        at java.lang.Enum.valueOf(Enum.java:238)
        at org.apache.sqoop.SqoopOptions.loadProperties(SqoopOptions.java:522)
        at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.read(HsqldbJobStorage.java:299)
        at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:198)
        at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
        at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:200)
        at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:183)
        at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:64)
        at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:436)
        at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:350)
        at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:211)
        at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:254)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Intercepting System.exit(1)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

任何帮助都将不胜感激。

感谢。 Yusata。

Answer 1

在Oozie工作流程中检查Sqoop的版本。看起来它不是1.4.7：堆栈跟踪线

org.apache.sqoop.SqoopOptions.loadProperties（SqoopOptions.java:522）

对应于https://github.com/apache/sqoop/blob/20af67ef60096b17e1d9585670e5ec787eb760e2/src/java/org/apache/sqoop/SqoopOptions.java#L522

在oozie中无法运行sqoop作业（无枚举常量com.cloudera.sqoop.SqoopOptions.FileLayout.ParquetFile）

1 个答案: