Question

我无法从Java执行pig命令。该命令用于创建关系，在关系中加载输入文件，然后将其存储在hdfs路径中的文件中。以下是我得到的错误：

15/09/23 05:02:50 WARN mapReduceLayer.Launcher: There is no log file to write to. 
15/09/23 05:02:50 ERROR mapReduceLayer.Launcher: Backend error message during job submission org.apache.pig.backend.executionengine.ExecException: ERROR 2100: file:/<username>/abhi/myinput.txt does not exist. 
    at org.apache.pig.backend.executionengine.PigSlicer.validate(PigSlicer.java:126) 
    at org.apache.pig.impl.io.ValidatingInputFileSpec.validate(ValidatingInputFileSpec.java:59) 
    at org.apache.pig.impl.io.ValidatingInputFileSpec.<init>(ValidatingInputFileSpec.java:44) 
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:230) 
    at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:810) 
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:781) 
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:730) 
    at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378) 
    at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247) 
    at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279) 
    at java.lang.Thread.run(Thread.java:745) 
15/09/23 05:02:50 ERROR mapReduceLayer.MapReduceLauncher: Failed to produce result in: "file:/user/<username>/abhi" 
15/09/23 05:02:50 INFO mapReduceLayer.MapReduceLauncher: Failed! And below is the code package oozie;

这是代码：

import java.io.IOException; 
import java.util.Properties; 
import org.apache.pig.ExecType; 
import org.apache.pig.PigServer; 
import org.apache.pig.impl.PigContext; 

public class Oozietest1 { 

    public static void main(String[] args) { 
        try { 
            Properties pr = new Properties(); 
            pr.setProperty("fs.defaultFS", "hdfs://<ip address of the resource manager>/"); 
            pr.setProperty("mapreduce.framework.name", "yarn"); 
            pr.setProperty("yarn.resourcemanager.address", "< address>:<port>"); 
            PigContext pc=new PigContext(ExecType.MAPREDUCE, pr); 
            PigServer pigServer = new PigServer(pc); 
            System.out.println(pigServer); 
            runMyQuery(pigServer, "hdfs://user/<username>/abhi/myinput.txt"); 
        } catch (IOException e) { 
            e.printStackTrace(); 
        } 
    } 

    public static void runMyQuery(PigServer pigServer, String inputFile) throws IOException { 
        pigServer.registerQuery("A = load '" + inputFile + "' using PigStorage(',') as (name);"); 
        pigServer.registerQuery("STORE A INTO '/user/<username>/abhi/' USING PigStorage (',');"); 
    } 
}

简而言之，总是尝试从本地文件系统读取文件，而应该从hdfs读取文件。如果您需要任何其他信息，请与我们联系。

尝试从java执行pig操作时出现异常

0 个答案: