groovy hadoop jar with gradle - 包不存在错误

时间:2016-03-16 15:05:04

标签: java hadoop gradle dependencies

我试图用Gradle 2.12创建一个常规罐子。 groovy文件有如下导入语句,我将此文件放在src/main/groovy

前两个导入是java文件,其中包含org.apache.hadoop个导入语句。我将这两个文件放在src/main/java

import StartsWithCountMapper
import StartsWithCountReducer
import org.apache.hadoop.conf.Configured
import org.apache.hadoop.fs.Path
import org.apache.hadoop.io.IntWritable
import org.apache.hadoop.io.LongWritable
import org.apache.hadoop.io.Text
import org.apache.hadoop.mapreduce.Job
import org.apache.hadoop.mapreduce.Mapper
import org.apache.hadoop.mapreduce.Reducer
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat
import org.apache.hadoop.util.Tool
import org.apache.hadoop.util.ToolRunner

以下是我的build.gradle文件(使用this

apply plugin: 'groovy'
apply plugin: 'application'
version = '1.0'
mainClassName='CountGroovyJob'

repositories {
    mavenCentral()
}

dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.0.0'
    compile 'org.apache.hadoop:hadoop-common:2.7.1'

}

task Createjar(type: Jar,dependsOn:[':compileJava',':compileGroovy']) {
    from files(sourceSets.main.output.classesDir)
    from configurations.runtime.asFileTree.files.collect { zipTree(it) }

    manifest {
        attributes 'Main-Class': mainClassName
    }
}

我还试图通过将它们放在依赖项

中进行编译
     compile 'org.apache.hadoop:hadoop-client:2.0.0-mr1-cdh4.0.1'
    compile 'org.apache.hadoop:hadoop-core:1.2.1'
    compile 'org.apache.hadoop:hadoop-hdfs:2.7.1'

但我一直收到这个错误:

/Users/../../../gradle/buildSrc/src/main/java/StartsWithCountMapper.java:5: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.IntWritable;
                           ^
/Users/../../../gradle/buildSrc/src/main/java/StartsWithCountMapper.java:6: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.LongWritable;
                           ^
/Users/../../../gradle/buildSrc/src/main/java/StartsWithCountMapper.java:7: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Text;

如何在groovy和java文件中编译和创建包含所有import语句的jar?

修改 包括buildscrip {}

apply plugin: 'groovy'
apply plugin: 'application'
version = '1.0'
mainClassName='CountGroovyJob'

repositories {
    mavenCentral()
}

dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.0.0'
    compile 'org.apache.hadoop:hadoop-common:2.7.1'

}

task Createjar(type: Jar,dependsOn:[':compileJava',':compileGroovy']) {
    from files(sourceSets.main.output.classesDir)
    from configurations.runtime.asFileTree.files.collect { zipTree(it) }

    manifest {
        attributes 'Main-Class': mainClassName
    }
}

buildscript{
    repositories {
        mavenCentral()
    }

    dependencies {
        compile 'org.codehaus.groovy:groovy-all:2.0.0'
        compile 'org.apache.hadoop:hadoop-common:2.7.1'
    }
}

文件夹结构:

 grad-proj
 |
 +-- build.gradle
 |    
 +-- src
    |  
    +-- main
       |
       +-- groovy
       |
       +-- java

1 个答案:

答案 0 :(得分:2)

import语句位于buildSrc而不是main的代码中,这意味着buildscript依赖项部分需要依赖项。

而是将依赖关系应用于适用于main目录中的源的一般依赖关系部分。

添加到build.gradle:

buildscript{
    repositories {
        mavenCentral()
    }

    dependencies {
        compile 'org.codehaus.groovy:groovy-all:2.0.0'
        compile 'org.apache.hadoop:hadoop-common:2.7.1'
    }
}