在mac os x 10.8上设置hadoop的问题

时间：2013-02-18 09:13:09

标签： macos hadoop

我在我的mac上设置了一些用于安装hadoop的东西。但是有一个错误信息：

13/02/18 04:05:52 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 0 time(s).
13/02/18 04:05:53 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 1 time(s).
13/02/18 04:05:54 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 2 time(s).
13/02/18 04:05:55 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 3 time(s).
13/02/18 04:05:56 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 4 time(s).
13/02/18 04:05:57 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 5 time(s).
13/02/18 04:05:58 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 6 time(s).
13/02/18 04:05:59 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 7 time(s).
13/02/18 04:06:00 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 8 time(s).
13/02/18 04:06:01 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 9 time(s).
java.lang.RuntimeException: java.net.ConnectException: Call to localhost/127.0.0.1:9000 failed on connection exception: java.net.ConnectException: Connection refused
    at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:546)
    at org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:318)
    at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:265)
    at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
    at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.net.ConnectException: Call to localhost/127.0.0.1:9000 failed on connection exception: java.net.ConnectException: Connection refused
    at org.apache.hadoop.ipc.Client.wrapException(Client.java:1099)
    at org.apache.hadoop.ipc.Client.call(Client.java:1075)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
    at $Proxy1.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
    at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
    at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
    at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
    at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)
    at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:542)
    ... 17 more
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
    at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1206)
    at org.apache.hadoop.ipc.Client.call(Client.java:1050)
    ... 31 more

然后我输入jps来检查服务，结果是： 20635 Jps

20466 TaskTracker

20189 DataNode

20291 SecondaryNameNode

我不知道如何处理这个错误。有人可以给我一个答案吗？ 很多!!!

3 个答案:

答案 0 :(得分：2)

实际上由于IP配置错误，所有（hadoop应该运行）的进程都没有运行。我不熟悉Mac OS，但是在linux和windows上我们需要把hadoop enteries用于连接主机文件（etc / hosts）和我很确定它应该适用于Mac.Now重点是你需要将你的hadoop条目放在该文件中作为127.0.0.1这样的本地mnachine实际上你需要把它放在实际的IP上您的机器
例如
hadoop-machine 127.0.0.1 - ＆gt;（放回环回IP在这里是错误的，因为hadoop将尝试连接此IP）。
删除此127.0.0.1并在此条目前面放置您机器的实际IP。You can find ip of your mac machine eaisly。这里有一些与hadoop没有直接关系的问题，但我想它们对您有所帮助。
{ {3}}，Question 1，Question 2

答案 1 :(得分：1)

这可能会有所帮助。但首先，你必须使用命令

删除先前的安装
  

rm -rf / usr / local / Cellar /usr/local/.git＆amp;＆amp;酿造清理

然后U可以在U're Mac上重新安装Hadoop。

第1步：安装Homebrew

  

$ ruby​​ -e“$（curl -fsSL https://raw.github.com/mxcl/homebrew/go）”

第2步：安装Hadoop

  

$ brew install hadoop

假设brew安装Hadoop 1.2.1

第3步：配置Hadoop

  

$ cd /usr/local/Cellar/hadoop/1.2.1/libexec

将以下行添加到conf / hadoop-env.sh：

  

export HADOOP_OPTS =“ - Djava.security.krb5.realm = -Djava.security.krb5.kdc =”

将以下行添加到配置标记内的conf / core-site.xml：

<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>

将以下行添加到配置标记内的conf / hdfs-site.xml：

<property>
<name>dfs.replication</name>
<value>1</value>
</property>

将以下行添加到配置标记内的conf / mapred-site.xml：

 <property>
 <name>mapred.job.tracker</name>
 <value>localhost:9001</value>
 </property>

转到系统偏好设置＆gt;共享。 确保选中“远程登录”。

  

$ ssh-keygen -t rsa   $ cat~ / .ssh / id_rsa.pub＆gt;＆gt;的〜/ .ssh / authorized_keys的

步骤4：启用SSH到localhost

  

$ cat~ / .ssh / id_rsa.pub＆gt;＆gt;的〜/ .ssh / authorized_keys的

第5步：格式化Hadoop文件系统

  

$ bin / hadoop namenode -format

第6步：启动Hadoop

  

$ bin / start-all.sh

确保所有Hadoop进程都在运行：

  

$ jps

运行Hadoop示例：

还有一件事！ “brew update”会将hadoop二进制文件更新为最新版本（目前为1.2.1）。

答案 2 :(得分：1)

我在所有secondarynamenode日志文件中都有同样的错误：ConnectException: Connection refused。

但我也在namenode的日志文件中找到了这个：

2015-10-25 16:35:15,720 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /private/tmp/hadoop-admin/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.

因此我做了一个

hadoop namenode -format

问题已经消失。因此，错误消息只是namenode未成功启动的事实。

相关问题
最新问题