您好我正在尝试配置Hadoop 1.0。遵循此博客,以伪分布模式。
但是当我运行hadoop发行版中给出的pi示例时,我得到标题中提到的错误。有人可以帮助我并指导我如何解决这个问题。另外它的请求也请提出解决方案,如果可能的话,同时指出问题。
这是我通过运行jps得到的东西
8322 Jps
7611 SecondaryNameNode
7474 DataNode
7341 NameNode
这是完整的错误消息。
Number of Maps = 10
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
12/05/16 13:11:56 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 0 time(s).
12/05/16 13:11:57 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 1 time(s).
12/05/16 13:11:58 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 2 time(s).
12/05/16 13:11:59 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 3 time(s).
12/05/16 13:12:00 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 4 time(s).
12/05/16 13:12:01 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 5 time(s).
12/05/16 13:12:02 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 6 time(s).
12/05/16 13:12:03 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 7 time(s).
12/05/16 13:12:04 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 8 time(s).
12/05/16 13:12:05 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8021. Already tried 9 time(s).
java.net.ConnectException: Call to localhost/127.0.0.1:8021 failed on connection exception: java.net.ConnectException: Connection refused
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1095)
at org.apache.hadoop.ipc.Client.call(Client.java:1071)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at org.apache.hadoop.mapred.$Proxy2.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:480)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:474)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:457)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1260)
at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:297)
at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:656)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1202)
at org.apache.hadoop.ipc.Client.call(Client.java:1046)
... 24 more
答案 0 :(得分:3)
您的jps
输出显示您没有运行JobTracker或TaskTracker - 因此在尝试与jobtracker进行通信时出现连接问题。
检查jobtracker($HADOOP_HOME/logs/*.jobtracker*.log
)的日志,看看是否有错误消息说明它未启动的原因(如何启动群集 - start-all.sh
或start-dfs.sh
,也许尝试运行start-mapred.sh
以查看是否启动了地图缩减服务?)
答案 1 :(得分:0)
看起来这是一些与连接相关的问题。 你的jobtracker和trasktracker是否正常运行?
请参阅此link
修改强>
阅读权威指南中的附录A here。它具有在所有3种模式下设置hadoop服务器的完整步骤。我认为您的配置可能会产生问题,因此请使用本书中规定的那些。还要检查无密码SSH是否在您的设置上正常工作。
答案 2 :(得分:0)
我遇到同样的问题,因为hadoop 1.0.x有差异需要配置hadoop_env.sh中的两个变量:
导出HADOOP_HEAPSIZE = 2000
export HADOOP_OPTS =“ - Djava.security.krb5.realm = OX.AC.UK -Djava.security.krb5.kdc = kdc0.ox.ac.uk:kdc1.ox.ac.uk”
添加这两个变量时,在启动守护程序期间不会看到任何错误或警告,以便您可以解决问题。祝你好运!
希望我的回答可以帮助其他人:)
答案 3 :(得分:0)
从jps输出看起来TaskTracker
和JobTracker
未运行。尝试运行:
start-mapred.sh
这将启动TaskTracker
和JobTracker
并尝试重新运行作业。