尝试通过Java运行Pig时获取ERROR 4010

时间:2013-02-05 00:16:58

标签: java apache-pig

我正在尝试通过Java运行Pig脚本。这是我的代码现在的样子:

public static void main(String[] args) throws JSONException, InterruptedException, IOException {    
    Properties props = new Properties();
    props.setProperty("fs.default.name", "hdfs://<some-value>:8020");
    props.setProperty("mapred.job.tracker", "<some-value>:54311");
    PigServer pigServer = new PigServer(ExecType.MAPREDUCE, props);

    Map<String, String> params = new LinkedHashMap<String, String>();
    params.put("INPUT_PATH", "hdfs://<some-input-value>");
    params.put("OUTPUT_FILE", "hdfs:///user/<some-username>/last-login-out");

    pigServer.registerScript("last-login-by-userid.pig", params);
}

但每当我运行该程序时,我得到:

Exception in thread "main" org.apache.pig.backend.executionengine.ExecException: ERROR 4010: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath). If you plan to use local mode, please put -x local option in command line.

我已将从Apache网站下载的pig-0.10.1文件夹移至Applications,并在export PIG_HOME=/Applications/pig-0.10.1中添加了~/.bash_profile

当我登录<some-value>:8020服务器时,我可以正常运行Pig脚本。

2 个答案:

答案 0 :(得分:2)

你的hadoop安装在哪里?在我的系统上,配置位于/etc/hadoop/conf下。将其添加到Java程序的类路径中。

例如,由于各种原因,我不使用已安装的pig bash脚本;我只是这样叫Pig的Main课:

java -cp /path/to/pig-0.10.0.jar:/etc/hadoop/conf org.apache.pig.Main`

答案 1 :(得分:0)

从Java运行Pig时遇到了同样的异常。在将hadoop conf目录的路径添加到项目属性库add jar / folder

之后,这已得到解决

参考:http://helpmetocode.blogspot.in/2012/04/exception-in-thread-main.html

谢谢,

卡莱