Question

当我从CLI运行以下sqoop命令时，我能够将数据导出到OracleDB，但是当我使用oozie工作流运行时，同样的命令，我遇到了问题。

直接从CLI运行的命令：

sqoop export --connect jdbc:oracle:thin:@192.168.245.1:1521:XE --username HR --password HR --table HR.REVIEW_FINAL --export-dir /user/cloudera/Review/hive/review_final --input-fields-terminated-by '\001'

以下是我通过Oozie使用的内容：

<command>export --connect jdbc:oracle:thin:@192.168.245.1:1521:XE --username HR --password HR --table HR.REVIEW_FINAL --export-dir /user/cloudera/Review/hive/review_final --input-fields-terminated-by '\001'</command>

我在日志中看到的异常：

java.io.IOException: Can't export data, please check failed map task logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
at org.apache.hadoop.mapred.Child.main(Child.java:262)

引起：java.lang.RuntimeException：无法解析输入数据：

Answer 1

--input-fields-terminated-by '\001'

将其改为

--input-fields-terminated-by \001

祝你好运！

Answer 2

在我的例子中，不仅input-fields-terminated-by，而且input-null-non-string需要删除单引号。

<action name="load-to-mysql">
   <sqoop xmlns="uri:oozie:sqoop-action:0.2">
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <command>export --connect jdbc:mysql://xtradb1.t1.shadc.yosemitecloud.com:3306/report_test --username root -password root --table location_stats --export-dir /user/hive/warehouse/report/test_location_stats_${DATE} --input-null-string \\N --input-null-non-string \\N --input-fields-terminated-by \001 --input-lines-terminated-by '\n' --driver com.mysql.jdbc.Driver --update-mode allowinsert</command>
    </sqoop>
    <ok to="remove-tmp-data"/>
    <error to="fail"/>
</action>

无法使用oozie运行sqoop

2 个答案: