Question

我正在尝试按照此链接上给出的步骤运行猪脚本 - http://www.orzota.com/pig-tutorialfor-beginners/

但是我收到了这个错误。它无法读取加载到HDFS中的文件。你能帮忙吗？错误如下 -

Failed Jobs:
JobId   Alias   Feature Message Outputs
N/A BookXRecords,CountByYear,GroupByYear    GROUP_BY,COMBINER   Message: Unexpected System Error Occured: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:225)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
    at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:458)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
    at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:240)
    at org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:121)
    at java.lang.Thread.run(Thread.java:662)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:271)
    /user/hduser/output/pig_output_bookx,

Input(s):
Failed to read data from "/user/hduser/input/BX-BooksCorrected1.txt"

Output(s):
Failed to produce result in "/user/hduser/output/pig_output_bookx"

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
null


2015-02-19 22:19:45,852 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!

Answer 1

据我了解，您刚刚下载了脚本并运行。这就是为什么您的脚本无法找到您想要运行pig脚本的确切文件的原因。请确保：

您已运行sed命令来过滤本地系统上的books.csv文件
在运行sed后命名您在脚本中提到的过滤文件（不确定您的情况是什么，但应该是BX-BooksCorrected或BX-BooksCorrected1，请检查）
然后将该文件移动到HDFS，然后尝试运行它将起作用的脚本并且不会给出错误

P.S。：通过仔细阅读错误日志，您可以了解错误的本质。快乐的Hadooping！

每年发布的书籍频率的猪脚本

1 个答案: