Couchbase / hadoop连接器:sqoop job FAILED“找到接口org.apache.hadoop.mapreduce.TaskAttemptContext,但是类是预期的”

时间:2013-01-08 13:13:49

标签: hadoop couchbase sqoop

我的配置:

  • CouchBase server 2.0
  • Sqoop 1.4.2(编译为hadoop版本2.0.0)
  • Stack Hadoop CDH4.1.2

我想使用CouchBase / Hadoop连接器(http://www.couchbase.com/develop/connectors/hadoop),但是当我启动导入时出现以下错误:

sqoop-import --connect http://localhost:8091/pools --table DUMP
13/01/07 11:00:34 INFO tool.CodeGenTool: Beginning code generation
13/01/07 11:00:34 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop
Note: /tmp/sqoop-agent/compile/fccb413cf9eec92f9a716320cf268a0a/DUMP.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/01/07 11:00:42 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-agent/compile/fccb413cf9eec92f9a716320cf268a0a/DUMP.jar
13/01/07 11:00:42 INFO mapreduce.ImportJobBase: Beginning import of DUMP
13/01/07 11:00:45 WARN util.Jars: No such class couchbase doesn't use a jdbc driver available.
13/01/07 11:00:50 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2013-01-07 11:00:55.065 INFO net.spy.memcached.MemcachedConnection:  Added {QA sa=/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-01-07 11:00:55.098 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@41e22632
2013-01-07 11:00:56.266 INFO net.spy.memcached.MemcachedConnection:  Shut down memcached client
13/01/07 11:02:39 INFO mapred.JobClient: Running job: job_201212111702_0131
13/01/07 11:02:40 INFO mapred.JobClient:  map 0% reduce 0%
13/01/07 11:02:59 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000001_0,     Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:02 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000002_0, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:02 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000000_0, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:09 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000003_0, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:12 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000001_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:13 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000002_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:20 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000000_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:24 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000003_1, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:24 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000001_2, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:30 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000002_2, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:33 INFO mapred.JobClient: Task Id : attempt_201212111702_0131_m_000000_2, Status : FAILED
Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
13/01/07 11:03:39 INFO mapred.JobClient: Job complete: job_201212111702_0131
13/01/07 11:03:40 INFO mapred.JobClient: Counters: 7
13/01/07 11:03:40 INFO mapred.JobClient:   Job Counters 
13/01/07 11:03:40 INFO mapred.JobClient:     Failed map tasks=1
13/01/07 11:03:40 INFO mapred.JobClient:     Launched map tasks=14
13/01/07 11:03:40 INFO mapred.JobClient:     Rack-local map tasks=14
13/01/07 11:03:40 INFO mapred.JobClient:     Total time spent by all maps in occupied slots (ms)=74621
13/01/07 11:03:40 INFO mapred.JobClient:     Total time spent by all reduces in occupied slots (ms)=0
13/01/07 11:03:40 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
13/01/07 11:03:40 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
13/01/07 11:03:40 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 174.4617 seconds (0 bytes/sec)
13/01/07 11:03:40 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
13/01/07 11:03:40 INFO mapreduce.ImportJobBase: Retrieved 0 records.
13/01/07 11:03:40 ERROR tool.ImportTool: Error during import: Import job failed!

是否有人使用过具有相同配置的连接器并成功使其工作?

谢谢你的帮助!

马德琳

2 个答案:

答案 0 :(得分:3)

Hadoop经历了从Hadoop 1.0到Hadoop 2.0(相应地从CDH3到CDH4)的巨大代码重构。一个副作用是针对Hadoop 1.0(CDH3)编译的代码与Hadoop 2.0(CDH4)不兼容,反之亦然。但是源代码是兼容的,因此只需要使用目标Hadoop分发重新编译代码。

当您在Hadoop 2.0(CDH4)上运行为Hadoop 1.0(CDH3)编译的代码时,异常“找到接口X,但是类是预期的”非常常见。

完全相同的约束适用于连接器。没有关于Hadoop版本CouchBase连接器编译的明确信息,所以我猜它是针对Hadoop 1.0的,因此它不适用于Hadoop 2.0(CDH4)。

答案 1 :(得分:0)