Question

我是pig / java设置的新手。我正在尝试测试apache门户网站上给出的样本单元测试。以下是代码

import java.io.IOException;

import org.apache.pig.pigunit.PigTest;
import org.junit.Test;
import org.junit.BeforeClass;
import org.apache.pig.test.Util;

import org.apache.pig.tools.parameters.ParseException;

public class PigQueriesTest {

    @BeforeClass
    public static void setUpOnce() throws Exception {
        System.getProperties().setProperty("pigunit.exectype", Util.getLocalTestMode().toString());
    }

    @Test
    public void testTop2Queries() throws ParseException, IOException {
      String[] args = {
          "n=2",
          };

      PigTest test = new PigTest("src/main/PigScripts/top_queries.pig", args);

      String[] input = {
          "yahoo",
          "yahoo",
          "yahoo",
          "twitter",
          "facebook",
          "facebook",
          "linkedin",
      };

      String[] output = {
          "(yahoo1,5)",
          "(facebook,2)",
      };

      test.assertOutput("data", input, "queries_limit", output);

    }
}

错误是 -

PigQueriesTest > testTop2Queries FAILED
    java.lang.NoClassDefFoundError at PigQueriesTest.java:46
        Caused by: java.lang.ClassNotFoundException at PigQueriesTest.java:46

即。我在第

行收到错误

 test.assertOutput("data", input, "queries_limit", output);

猪脚本

data =
    LOAD 'input'
    AS (query:CHARARRAY);

queries_group =
    GROUP data
    BY query; 

queries_count = 
    FOREACH queries_group 
    GENERATE 
        group AS query, 
        COUNT(data) AS total;

queries_ordered =
    ORDER queries_count
    BY total DESC, query;

queries_limit =
    LIMIT queries_ordered $n;

STORE queries_limit INTO 'output';

我在调试模式下运行了gradle。我得到了以下错误

10:08:14.660 [DEBUG] [TestEventLogger] PigQueriesTest > testTop2Queries FAILED
10:08:14.660 [DEBUG] [TestEventLogger]     java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.LocalExecType.getExecutionEngine(LocalExecType.java:50)
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.impl.PigContext.<init>(PigContext.java:270)
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.PigServer.<init>(PigServer.java:206)
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.PigServer.<init>(PigServer.java:202)
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.pigunit.pig.PigServer.<init>(PigServer.java:39)
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.pigunit.PigTest.getCluster(PigTest.java:144)
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.pigunit.PigTest.registerScript(PigTest.java:170)
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:290)
10:08:14.661 [DEBUG] [TestEventLogger]         at org.apache.pig.pigunit.PigTest.assertOutput(PigTest.java:285)
10:08:14.661 [DEBUG] [TestEventLogger]         at PigQueriesTest.testTop2Queries(PigQueriesTest.java:46)
10:08:14.661 [DEBUG] [TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
10:08:14.661 [DEBUG] [TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
10:08:14.661 [DEBUG] [TestEventLogger]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

我遇到了关于stackoverflow的问题，要求停止扫描类文件以解决一些与gradle相关的问题。我的gradle构建文件是

// Apply the java plugin to add support for Java
apply plugin: 'java'
apply plugin:'application'
apply plugin:'eclipse'
apply plugin: 'maven'

mainClassName = 'com.myudf.Main'

// In this section you declare where to find the dependencies of your project
repositories {
    // Use 'jcenter' for resolving your dependencies.
    // You can declare any Maven/Ivy/file repository here.
    jcenter()
}


test {
  testLogging {
    events "passed", "skipped", "failed", "standardOut", "standardError"
  }
}

//Necessary to stop gradle from scanning - GRADLE-1682 / GRADLE-3238
tasks.withType(Test) {
    scanForTestClasses = false
    include "**/*Test.class" // whatever Ant pattern matches your test class files
}


// In this section you declare the dependencies for your production and test code
dependencies {
    // The production code uses the SLF4J logging API at compile time
    compile 'org.slf4j:slf4j-api:1.7.12'
    compile group: 'org.apache.pig', name: 'pig', version: '0.14.0'

    //https://repository.cloudera.com/artifactory/cloudera-repos/
    compile group: "org.apache.hadoop", name: "hadoop-common", version: "2.6.0-cdh5.9.0"

    // Dependencies for unit tests
    compile group: 'org.apache.pig', name: 'pigunit', version: '0.14.0'
    compile group: 'junit', name: 'junit', version: '4.12'
// Declare the dependency for your favourite test framework you want to use in your tests.
    // TestNG is also supported by the Gradle Test task. Just change the
    // testCompile dependency to testCompile 'org.testng:testng:6.8.1' and add
    // 'test.useTestNG()' to your build script.
    testCompile 'junit:junit:4.12'
}

我在设置中遗漏了什么吗？

提前致谢和Manish

Answer 1

这适用于将来会遇到此问题的人。在做了一些谷歌搜索后，我能够解决这个问题。我做了两件事来修复错误

1）首先我包括在内 // https://repository.cloudera.com/artifactory/cloudera-repos/

添加上述依赖项后，我收到了错误

16:08:57.029 [DEBUG] [TestEventLogger] PigQueriesTest > testTop2Queries FAILED
16:08:57.029 [DEBUG] [TestEventLogger]     org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias queries_limit

2）通过添加分类器＆＃34; h2＆＃34;修正了上述错误在猪依赖

compile group: 'org.apache.pig', name: 'pig', version: '0.14.0', classifier: 'h2'

Pigunit给出错误classnotfound

1 个答案: