成功映射后sqoop导出失败

时间:2017-09-13 09:03:51

标签: mysql hadoop hive sqoop

1.sqoop export --connect jdbc:mysql://localhost:3306/hduser_db --username hduser 
    --password hduser --table export --export-dir /user/hive/warehouse/three --
fields-terminated-by ','

17/09/13 14:10:45 INFO mapreduce.Job:  map 0% reduce 0%
17/09/13 14:10:50 INFO mapreduce.Job:  map 100% reduce 0%
17/09/13 14:10:51 INFO mapreduce.Job: Job job_1505199140014_0033 failed with 
state FAILED due to: Task failed task_1505199140014_0033_m_000000
ob failed as tasks failed. failedMaps:1 failedReduces:0

2.17/09/13 14:10:51 INFO mapreduce.Job: Counters: 8
  

工作柜台                   失败的地图任务= 1                   启动地图任务= 1                   机架局部映射任务= 1                   占用时隙中所有映射所花费的总时间(ms)= 2947                   所有花费的总时间减少了占用的时隙(ms)= 0                   所有地图任务花费的总时间(ms)= 2947                   所有地图任务占用的总核心毫秒数= 2947                   所有映射任务占用的总兆字节数毫秒= 3017728       17/09/13 14:10:51 WARN mapreduce.Counters:Group FileSystemCounters is       藐视       17/09/13 14:10:51 INFO mapreduce.ExportJobBase:传输0字节       14.8875 s       17/09/13 14:10:51 WARN mapreduce.Counters:Group       org.apache.hadoop.mapred.Task $ C       17/09/13 14:10:51 INFO mapreduce.ExportJobBase:导出0条记录。       17/09/13 14:10:51错误tool.ExportTool:导出时出错:       出口工作失败!           在org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java)           在org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)           在org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)           在org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)           在org.apache.sqoop.Sqoop.run(Sqoop.java:147)           在org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)           在org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)           在org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)           在org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)           在org.apache.sqoop.Sqoop.main(Sqoop.java:252)

1 个答案:

答案 0 :(得分:0)

在运行导出命令时 - 必须小心以下步骤。

  1. 数据类型,源(HDFS数据)和目标(rdbms上的表)之间的列名应匹配。

  2. 我们应该在--columns参数中指定所有列名。

  3. 例如:

    sqoop export --connect jdbc:mysql:// localhost:3306 / hduser_db --username hduser --password hduser - 表格出口 --export-dir / user / hive / warehouse / three - 字段 - 终止 - ',' --columns" column1,column2,...." ;