Question

我试图测试Reducer吼叫

Reducer<CompositeKeyWritable, AvroValue<Class1>, AvroKey<Class2>, NullWritable>

出于这些目的，我使用测试中的下一个代码：

CustomReducer reducer = new CustomReducer();
    reduceDriver = new ReduceDriver<CompositeKeyWritable, AvroValue<Class1>, AvroKey<Class2>, NullWritable>();

    Job job = new Job();

    reduceDriver = ReduceDriver.newReduceDriver(reducer).withConfiguration(job.getConfiguration());
    reduceDriver.setReducer(reducer);

    job.setInputFormatClass(AvroKeyInputFormat.class);

    job.setMapOutputKeyClass(CompositeKeyWritable.class);
    AvroJob.setMapOutputValueSchema(job, union);

    AvroJob.setOutputKeySchema(job, Class2.SCHEMA$);
    job.setOutputFormatClass(AvroKeyOutputFormat.class);
    job.setOutputValueClass(NullWritable.class);

结果我得到了下一个错误：

java.lang.RuntimeException: java.io.EOFException
at org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:86)
at org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:97)
at org.apache.hadoop.mrunit.internal.io.Serialization.copyWithConf(Serialization.java:110)
at org.apache.hadoop.mrunit.TestDriver.copy(TestDriver.java:675)
at org.apache.hadoop.mrunit.ReduceDriverBase.addInput(ReduceDriverBase.java:167)
at org.apache.hadoop.mrunit.ReduceDriverBase.addInput(ReduceDriverBase.java:181)
at com.wellcare.caregap.hadoop.reducers.CareGapReducerTest.testReducer(CareGapReducerTest.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

工作配置有什么问题？如果有人帮我解决这些例外，我将非常感激。

Answer 1

调试方法readFields并在Class1中编写。当您使用reduceDriver.withInput(yourkey, yourvalue);

添加输入时，它似乎是在尝试读取或写入其中一个DataInput或DataOutput流的结尾之后的字段

如何在MRUnit中使用avro params测试减速机？

1 个答案: