在hadoop上运行kmeans时发生PriviledgedActionException

时间:2015-03-20 06:29:22

标签: eclipse hadoop mapreduce k-means

我正在尝试使用此指南在hadoop上运行KMeans。

http://www.slideshare.net/titusdamaiyanti/hadoop-installation-k-means-clustering-mapreduce?qid=44b5881c-089d-474b-b01d-c35a2f91cc67&v=qf1&b=&from_search=1#likes-panel

我在eclipse-luna中运行它。当我执行时,map和reduce都显示它们完成100%。但我没有得到输出。相反,我最后得到了跟随错误。请一些人帮我解决这个问题..

15/03/20 11:29:44 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-hduser/mapred/staging/hduser378797276/.staging/job_local378797276_0002

15/03/20 11:29:44 ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:java.io.IOException: No input paths specified in job

Exception in thread "main" java.io.IOException: No input paths specified in job


at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:193)


at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:55)


at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)


at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)


at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)


at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)


at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)


at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)


at java.security.AccessController.doPrivileged(Native Method)


at javax.security.auth.Subject.doAs(Subject.java:422)


at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)


at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)


at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)


at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)


at com.clustering.mapreduce.KMeansClusteringJob.main(KMeansClusteringJob.java:114)

1 个答案:

答案 0 :(得分:0)

您必须在运行map reduce程序之前提供输入文件位置。有两种方法可以提供输入文件位置。

  1. 使用ecliple go运行configration并提供文件名作为参数
  2. 将程序转换为jar文件并在hadoop集群中运行以下命令

    hadoop jar NameOfYourJarFile InputFileLocation OutPutFileLocation `