mrjob:PipeMapRed.waitOutputThreads():子进程失败,代码为2

时间:2019-03-08 06:42:09

标签: python hadoop mrjob

我按照tutorial测试wordcount。可以直接通过python mr_word_count.py my_file.txt运行它。当我尝试在hdfs python my_job.py -r hadoop hdfs://my_home/input.txt中运行它时,它显示了以下错误。我已经尝试过thisthis之类的解决方案,以及其他一些解决方案。但是问题仍然存在。我在AWS EMR集群上运行它。

Task Id : attempt_1552008616549_0861_m_000001_0, Status : FAILED
Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 2
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:538)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:344)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)

0 个答案:

没有答案