我正在推出失败的Hadoop流媒体作业。启动它的行是:
hadoop jar $HADOOP_HOME/hadoop-streaming-2.6.0.2.2.5.3-1.jar -conf ~/HADOOP/conf/hadoop-cluster.xml -files aggregation_jobs -input /epcot -output /crowd_analytics/event_count/ -mapper "aggregation_jobs/streaming/event_count_map.py" -reducer "aggregation_jobs/streaming/event_count_reduce.py" -verbose >> output
hadoop-cluster.xml文件的内容如下:
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://fsdala12080.test.domain.com/</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>fsdala12081.test.domain.com:8032</value>
</property>
</configuration>
我收到以下错误:
java.net.UnknownHostException: ww-am04035180-010082030080.test.domain.com: ww-am04035180-010082030080.test.domain.com: unknown error
at java.net.InetAddress.getLocalHost(InetAddress.java:1484)
at org.apache.hadoop.streaming.Environment.getHost(Environment.java:121)
at org.apache.hadoop.streaming.StreamUtil.<clinit>(StreamUtil.java:176)
at org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:822)
at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:128)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.UnknownHostException: ww-am04035180-010082030080.test.domain.com: unknown error
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:907)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1302)
at java.net.InetAddress.getLocalHost(InetAddress.java:1479)
... 13 more
其中未知主机字符串的第一部分是我的本地计算机,它不在我正在谈论的集群中。顺便说一句,我可以从我的计算机与群集通信,并将文件从本地硬盘驱动器复制到该群集上的hdfs。为什么这可能不起作用的任何想法?为什么它会使我的本地主机达到峰值?我提交工作时有任何错误吗?任何指针都会有所帮助。