故障单元测试自定义输入格式

时间:2015-05-12 15:02:55

标签: java unit-testing maven hadoop

我是hadoop世界的新手。我为我的应用程序开发了自定义输入格式和阅读器。我在运行单元测试时遇到问题。我按照这篇文章开发了我的单元测试:How to do unit testing of custom RecordReader and InputFormat classes?。除了InputFormat类之外,我的单元测试代码与post中的代码完全相同:


    Configuration conf = new Configuration(false);
    conf.set("fs.default.name", "file:///");
    File testFile = new File("/tmp/testfile");
    Path path = new Path(testFile.getAbsoluteFile().toURI());
    FileSplit split = new FileSplit(path, 0, testFile.length(), null);
    MyFileInputFormat inputFormat =     ReflectionUtils.newInstance(MyFileInputFormat.class, conf);
    TaskAttemptContext context = new TaskAttemptContextImpl(conf, new TaskAttemptID());
    try {
        RecordReader reader = inputFormat.createRecordReader(split, context);
        reader.initialize(split, context);
    } catch (Exception e) {

    }

当我尝试使用" mvn test"运行此测试时,我收到以下错误:


    java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration$DeprecationDelta
            at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
            at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
            at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
            at org.apache.hadoop.mapreduce.util.ConfigUtil.addDeprecatedKeys(ConfigUtil.java:54)
            at org.apache.hadoop.mapreduce.util.ConfigUtil.loadResources(ConfigUtil.java:42)
            at org.apache.hadoop.mapred.JobConf.(JobConf.java:119)
            at java.lang.Class.forName0(Native Method)
            at java.lang.Class.forName(Class.java:274)
            at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
            at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:80)
            at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
            at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)

任何人都可以帮我理解如何解决这个问题吗?

0 个答案:

没有答案