Question

我在Oracle上有一个表，在Hive上有相同的表（具有相关的数据类型）我正在尝试使用脚本将蜂巢表导出到Oracle：

sqoop export -D oraoop.disabled=true -Dmapred.job.queue.name=disco --connect jdbc:oracle:thin:@oracle:1521/tns  \
--username someuser  \
--password somepasswd \
--hcatalog-database hive_database  \
--hcatalog-table TABLE_ON_HIVE  \
--table TABLE_ON_ORACLE  \
--num-mappers 5

我得到一个错误：

18/08/29 08:23:10 INFO mapreduce.Job: Job job_1535519043541_1004 running in uber mode : false
18/08/29 08:23:10 INFO mapreduce.Job:  map 0% reduce 0%
18/08/29 08:23:28 INFO mapreduce.Job:  map 100% reduce 0%
18/08/29 08:28:40 INFO mapreduce.Job: Task Id : attempt_1535519043541_1004_m_000000_0, Status : FAILED
AttemptID:attempt_1535519043541_1004_m_000000_0 Timed out after 300 secs
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

18/08/29 08:28:41 INFO mapreduce.Job:  map 0% reduce 0%
18/08/29 08:28:57 INFO mapreduce.Job:  map 100% reduce 0%
18/08/29 08:34:09 INFO mapreduce.Job: Task Id : attempt_1535519043541_1004_m_000000_1, Status : FAILED
AttemptID:attempt_1535519043541_1004_m_000000_1 Timed out after 300 secs
18/08/29 08:34:10 INFO mapreduce.Job:  map 0% reduce 0%
18/08/29 08:34:28 INFO mapreduce.Job:  map 100% reduce 0%
18/08/29 08:39:39 INFO mapreduce.Job: Task Id : attempt_1535519043541_1004_m_000000_2, Status : FAILED
AttemptID:attempt_1535519043541_1004_m_000000_2 Timed out after 300 secs
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

18/08/29 08:39:40 INFO mapreduce.Job:  map 0% reduce 0%
18/08/29 08:39:56 INFO mapreduce.Job:  map 100% reduce 0%
18/08/29 08:45:11 INFO mapreduce.Job: Job job_1535519043541_1004 failed with state FAILED due to: Task failed task_1535519043541_1004_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

18/08/29 08:45:11 INFO mapreduce.Job: Counters: 9
        Job Counters
                Failed map tasks=4
                Launched map tasks=4
                Other local map tasks=3
                Rack-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=1312647
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=1312647
                Total vcore-seconds taken by all map tasks=1312647
                Total megabyte-seconds taken by all map tasks=5376602112
18/08/29 08:45:11 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
18/08/29 08:45:11 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 1,359.6067 seconds (0 bytes/sec)
18/08/29 08:45:11 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
18/08/29 08:45:11 INFO mapreduce.ExportJobBase: Exported 0 records.
18/08/29 08:45:11 ERROR tool.ExportTool: Error during export: Export job failed!

因此，我进行了两次尝试，并且导出失败，没有说明原因。您是否遇到过像我这样的问题？在哪里可以找到更多详细的日志？

毛巾

Answer 1

请尝试使用以下命令
sqoop导出--connect \
-Dmapred.job.queue.name =迪斯科\
--username sqoop \
--password sqoop \
--table emp \
--update-mode allowinsert \
--update-key id \
--export-dir table_location \
--input-fields-由'delimiter'终止

注意：--update-mode-我们可以传递两个参数“ updateonly”-更新记录。如果更新键匹配，这将更新记录。如果要进行upsert（如果存在UPDATE则为INSERT），则使用“ allowinsert”模式。例： --update-mode updateonly \->用于更新 --update-mode allowinsert \->用于upsert

Sqoop在导出到Oracle期间失败

1 个答案: