运行sqoop导出作业时运行Oraoop

时间:2017-04-13 11:17:59

标签: java oracle hadoop sqoop bigdata

我正在运行此sqoop查询

sqoop export -D oraoop.timestamp.string=false --connect jdbc:oracle:thin:@127.0.0.1:1521:XE --username root --password manager --table person_temp4 --columns "id,first_name,last_name,email,tim" --export-dir /sqoop/person_data/avro3/ -m 4

对于这个查询,调用了Oraoop,这很奇怪,因为oraoop不支持导出或评估工作 输出是:

Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/04/13 04:01:07 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4.2.1.1.0-385
17/04/13 04:01:07 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/04/13 04:01:07 INFO manager.SqlManager: Using default fetchSize of 1000
17/04/13 04:01:08 INFO oraoop.OraOopOracleQueries: Current schema is: ROOT
17/04/13 04:01:08 INFO oraoop.OraOopManagerFactory: 
***********************************************************************
*** Using Quest® Data Connector for Oracle and Hadoop 1.6.0-cdh4-20 ***
*** Copyright 2012 Quest Software, Inc.                             ***
*** ALL RIGHTS RESERVED.                                            ***
***********************************************************************
17/04/13 04:01:08 INFO oraoop.OraOopManagerFactory: Oracle Database version: Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
17/04/13 04:01:08 INFO oraoop.OraOopManagerFactory: This Oracle database is not a RAC.
17/04/13 04:01:08 INFO Configuration.deprecation: mapred.map.max.attempts is deprecated. Instead, use mapreduce.map.maxattempts
17/04/13 04:01:08 INFO tool.CodeGenTool: Beginning code generation
17/04/13 04:01:09 INFO manager.SqlManager: Executing SQL statement: SELECT "ID","FIRST_NAME","LAST_NAME","EMAIL","TIM" FROM person_temp4 WHERE 0=1
17/04/13 04:01:09 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-root/compile/b4748ab65cf1d407986f9bee683d82db/person_temp4.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/04/13 04:01:11 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/b4748ab65cf1d407986f9bee683d82db/person_temp4.jar
17/04/13 04:01:11 INFO mapreduce.ExportJobBase: Beginning export of person_temp4
17/04/13 04:01:11 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
17/04/13 04:01:13 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
17/04/13 04:01:13 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
17/04/13 04:01:13 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
17/04/13 04:01:13 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/10.0.2.15:8050
17/04/13 04:01:14 INFO input.FileInputFormat: Total input paths to process : 1
17/04/13 04:01:14 INFO mapreduce.JobSubmitter: number of splits:1
17/04/13 04:01:14 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
17/04/13 04:01:14 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1492080500165_0005
17/04/13 04:01:15 INFO impl.YarnClientImpl: Submitted application application_1492080500165_0005
17/04/13 04:01:15 INFO mapreduce.Job: The url to track the job: http://sandbox.hortonworks.com:8088/proxy/application_1492080500165_0005/
17/04/13 04:01:15 INFO mapreduce.Job: Running job: job_1492080500165_0005
17/04/13 04:01:23 INFO mapreduce.Job: Job job_1492080500165_0005 running in uber mode : false
17/04/13 04:01:23 INFO mapreduce.Job:  map 0% reduce 0%
17/04/13 04:01:32 INFO mapreduce.Job: Task Id : attempt_1492080500165_0005_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: No such field: TIM
    at person_temp4.setField(person_temp4.java:337)
    at org.apache.sqoop.mapreduce.AvroExportMapper.toSqoopRecord(AvroExportMapper.java:120)
    at org.apache.sqoop.mapreduce.AvroExportMapper.map(AvroExportMapper.java:104)
    at org.apache.sqoop.mapreduce.AvroExportMapper.map(AvroExportMapper.java:49)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

17/04/13 04:01:38 INFO mapreduce.Job: Task Id : attempt_1492080500165_0005_m_000000_1, Status : FAILED
Error: java.lang.RuntimeException: No such field: TIM
    at person_temp4.setField(person_temp4.java:337)
    at org.apache.sqoop.mapreduce.AvroExportMapper.toSqoopRecord(AvroExportMapper.java:120)
    at org.apache.sqoop.mapreduce.AvroExportMapper.map(AvroExportMapper.java:104)
    at org.apache.sqoop.mapreduce.AvroExportMapper.map(AvroExportMapper.java:49)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

17/04/13 04:01:44 INFO mapreduce.Job: Task Id : attempt_1492080500165_0005_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: No such field: TIM
    at person_temp4.setField(person_temp4.java:337)
    at org.apache.sqoop.mapreduce.AvroExportMapper.toSqoopRecord(AvroExportMapper.java:120)
    at org.apache.sqoop.mapreduce.AvroExportMapper.map(AvroExportMapper.java:104)
    at org.apache.sqoop.mapreduce.AvroExportMapper.map(AvroExportMapper.java:49)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

17/04/13 04:01:56 INFO mapreduce.Job:  map 100% reduce 0%
17/04/13 04:01:57 INFO mapreduce.Job: Job job_1492080500165_0005 failed with state FAILED due to: Task failed task_1492080500165_0005_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

17/04/13 04:01:57 INFO mapreduce.Job: Counters: 9
    Job Counters 
        Failed map tasks=4
        Launched map tasks=4
        Other local map tasks=3
        Data-local map tasks=1
        Total time spent by all maps in occupied slots (ms)=25923
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=25923
        Total vcore-seconds taken by all map tasks=25923
        Total megabyte-seconds taken by all map tasks=6480750
17/04/13 04:01:57 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
17/04/13 04:01:57 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 44.3748 seconds (0 bytes/sec)
17/04/13 04:01:57 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
17/04/13 04:01:57 INFO mapreduce.ExportJobBase: Exported 0 records.
17/04/13 04:01:57 ERROR tool.ExportTool: Error during export: Export job failed!

为什么要为出口工作调用Oraoop? 我正在使用HDP 2.1并且已经单独安装了Oraoop。

0 个答案:

没有答案