难以使用Ant构建Pig

时间:2014-06-17 13:42:23

标签: java windows hadoop ant apache-pig

我是一个猪新手,一直试图让它在我的Windows 7机器上运行。我的hadoop(2.4.0)在我的本地机器上工作正常,但每当我尝试运行一个简单的Pig脚本时,每当我尝试转储文件内容时就会抛出错误。

找到类似问题的解决方案,发现here,我尝试使用Ant重建我的Pig(0.12.1)。 每当我跑

ant clean jar-withouthadoop -Dhadoopversion=23

我收到以下错误:

Buildfile: C:\pig-0.12.1-src\build.xml
......
[org.apache.ftpserver#ftpserver-core;1.0.0] in [hadoop20]
[ivy:resolve]   org.apache.mina#mina-core;2.0.0-M2 by [org.apache.mina#mina-core;2.0.0-M5] in [hadoop20]
---------------------------------------------------------------------
|                  |            modules            ||   artifacts   |
|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
|      master      |   0   |   0   |   0   |   0   ||   0   |   0   |
|      default     |  101  |   32  |   0   |   3   ||  101  |   0   |
|      runtime     |  101  |   32  |   0   |   3   ||  101  |   0   |
|      compile     |   89  |   29  |   0   |   2   ||   90  |   0   |
|       test       |  101  |   32  |   0   |   3   ||  101  |   0   |
|      javadoc     |  101  |   32  |   0   |   3   ||  101  |   0   |
|   releaseaudit   |   3   |   2   |   0   |   0   ||   3   |   0   |
|       jdiff      |   3   |   0   |   0   |   0   ||   3   |   0   |
|    checkstyle    |   10  |   3   |   0   |   0   ||   10  |   0   |
|     buildJar     |  101  |   32  |   0   |   3   ||  101  |   0   |
|     hadoop20     |   48  |   30  |   0   |   9   ||   39  |   0   |
|     hadoop23     |   40  |   13  |   0   |   0   ||   42  |   0   |
|      hbase94     |   1   |   0   |   0   |   0   ||   2   |   0   |
|      hbase95     |   7   |   0   |   0   |   0   ||   13  |   0   |
---------------------------------------------------------------------

ivy-compile:
[ivy:retrieve] :: retrieving :: org.apache.pig#pig
[ivy:retrieve]  confs: [compile]
[ivy:retrieve]  90 artifacts copied, 0 already retrieved (80170kB/407ms)
[ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
[ivy:cachepath] :: loading settings :: file = C:\pig-0.12.1-src\ivy\ivysettings.xml

init:
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\impl\logicalLayer\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\tools\pigscript\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\tools\parameters
    [mkdir] Created dir: C:\pig-0.12.1-src\build\classes
    [mkdir] Created dir: C:\pig-0.12.1-src\build\test\classes
    [mkdir] Created dir: C:\pig-0.12.1-src\test\org\apache\pig\test\utils\dotGraph\parser
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\data\parser
    [move] Moving 1 file to C:\pig-0.12.1-src\build\ivy\lib\Pig

cc-compile:
   [javacc] Java Compiler Compiler Version 4.2 (Parser Generator)
   [javacc] (type "javacc" with no arguments for help)
   [javacc] Reading from file C:\pig-0.12.1-
    ....
   [javacc] Parser generated successfully.

prepare:
    [mkdir] Created dir: C:\pig-0.12.1-src\src-gen\org\apache\pig\parser

genLexer:
     [java] error(1):  cannot write file : java.io.FileNotFoundException: C:\pig-0.12.1-src\src-gen\org\apache\pig\parser\C:\QueryLexer.java (The filename, directory name, or volume label syntax is incorrect)
     [java] java.io.FileOutputStream.open(Native Method)
     [java] java.io.FileOutputStream.<init>(FileOutputStream.java:221)
     [java] java.io.FileOutputStream.<init>(FileOutputStream.java:171)
     [java] java.io.FileWriter.<init>(FileWriter.java:90)
     [java] org.antlr.Tool.getOutputFile(Tool.java:871)
     [java] org.antlr.codegen.CodeGenerator.write(CodeGenerator.java:1262)
     [java] org.antlr.codegen.Target.genRecognizerFile(Target.java:94)
     [java] org.antlr.codegen.CodeGenerator.genRecognizer(CodeGenerator.java:460)
     [java] org.antlr.Tool.generateRecognizer(Tool.java:655)
     [java] org.antlr.Tool.process(Tool.java:468)
     [java] org.antlr.Tool.main(Tool.java:93)
BUILD FAILED
C:\pig-0.12.1-src\build.xml:495 Java returned: 1

我在这里做错了什么?我是否丢失文件,或者是build.xml中的配置问题?

编辑:我已经从cmd和cygwin终端运行此命令。它们都输出相同的错误。

EDIT2 :我正在使用Ant构建(1.9.4)

最终编辑:结果我需要将pig src存档(pig-0.12.1-src.tar.gz)移动到cygwin主目录,解压缩,cd到解压缩的存档然后运行命令。

1 个答案:

答案 0 :(得分:2)

我遇到了同样的问题。我通过在cygwin家中通过cygwin编译猪来解决它。

我的常春藤设置如下:

<ivysettings>

 <!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->

 <!--
  see http://www.jayasoft.org/ivy/doc/configuration
  -->
  <!-- you can override this property to use mirrors
          http://repo1.maven.org/maven2/
          http://mirrors.dotsrc.org/maven2
          http://ftp.ggi-project.org/pub/packages/maven2
          http://mirrors.sunsite.dk/maven2
          http://public.planetmirror.com/pub/maven2
          http://ibiblio.lsu.edu/main/pub/packages/maven2
          http://www.ibiblio.net/pub/packages/maven2
  -->
  <property name="repo.maven.org" value="${mvnrepo}" override="true"/>
  <property name="repo.jboss.org" value="http://repository.jboss.com/nexus/content/groups/public/" override="false"/>
  <property name="repo.apache.snapshots" value="http://repository.apache.org/content/groups/snapshots-group/" override="false"/>
  <property name="repo.dir" value="${user.home}/.m2/repository" override="false"/>
  <property name="maven2.pattern" value="[organisation]/[module]/[revision]/[module]-[revision](-[classifier])"/>
  <property name="maven2.pattern.ext" value="${maven2.pattern}.[ext]"/>
  <property name="snapshot.pattern" value="[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]"/>
  <property name="resolvers" value="default" override="false"/>
  <property name="force-resolve" value="false" override="false"/>
  <!-- pull in the local repository -->
  <include url="${ivy.default.conf.dir}/ivyconf-local.xml"/>
  <settings defaultResolver="${resolvers}"/>
  <resolvers>
    <ibiblio name="maven2" root="${repo.maven.org}" pattern="${maven2.pattern.ext}" m2compatible="true"/>
    <ibiblio name="jboss-maven2" root="${repo.jboss.org}" pattern="${maven2.pattern.ext}" m2compatible="true"/>
    <ibiblio name="apache-snapshots" root="${repo.apache.snapshots}" pattern="${snapshot.pattern}"
         checkmodified="true" changingPattern=".*SNAPSHOT" m2compatible="true"/>
    <filesystem name="fs" m2compatible="true" checkconsistency="false" force="${force-resolve}"
         checkmodified="true" changingPattern=".*SNAPSHOT">
      <artifact pattern="${repo.dir}/${maven2.pattern.ext}"/>
      <ivy pattern="${repo.dir}/[organisation]/[module]/[revision]/[module]-[revision].pom"/>
    </filesystem>
    <chain name="internal" checkmodified="true">
      <resolver ref="fs"/>
    </chain>
    <chain name="external" dual="true">
      <resolver ref="maven2"/>
      <resolver ref="jboss-maven2"/>
      <resolver ref="apache-snapshots"/>
    </chain>
    <chain name="default" dual="true" checkmodified="true">
      <resolver ref="internal"/>
      <resolver ref="external"/>
    </chain>
  </resolvers>
  <modules>
    <module organisation="org.apache.pig" name=".*" resolver="internal"/>
  </modules>
</ivysettings>