Question

尝试在Mac Sierra上运行单节点Hadoop。

这是我的配置：

JAVA版本https://wiki.apache.org/hadoop/HadoopJavaVersions

my-MacBook-Pro:hadoop me$ java -version
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

HDFS-SITE

<configuration>
<property>
        <name>dfs.replication</name>
        <value>1</value>
</property>
</configuration>

CORE-SITE

<configuration>
<property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value>
</property>
</configuration>

MAPRED-SITE

<configuration>
<property>
        <name>mapred.job.tracker</name>
        <value>localhost:9010</value>
</property>
</configuration>

HADOOP-ENV

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home
$ bin/hadoop  # displays the usage documentation

以下是一些额外设置：

我没有让这个活跃，有些指示推荐

#export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true -Dsun.security.krb5.debug=true -Dsun.security.spnego.debug"

默认两个其他配置已在HADOOP-ENV中活跃

# Some parts of the shell code may do special things dependent upon
# the operating system.  We have to set this here. See the next
# section as to why....
export HADOOP_OS_TYPE=${HADOOP_OS_TYPE:-$(uname -s)}


# Under certain conditions, Java on OS X will throw SCDynamicStore errors
# in the system logs.
# See HADOOP-8719 for more information.  If one needs Kerberos
# support on OS X, one will want to change/remove this extra bit.
case ${HADOOP_OS_TYPE} in
  Darwin*)
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.realm= "
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.kdc= "
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.conf= "
  ;;
esac

这是我格式化namenode的地方：

NAMENODE FORMATTED

My-MacBook-Pro:3.0.0 me$ bin/hdfs namenode -format
WARNING: /usr/local/Cellar/hadoop/3.0.0/libexec/logs does not exist. Creating.

启动namenode：

2018-04-09 17:45:15,037 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = My-MacBook-Pro.local/10.5.1.16
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 3.0.0
STARTUP_MSG:   classpath = /usr/ …
STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r c25427ceca461ee979d30edd7a4b0f50718e6533; compiled by 'andrew' on 2017-12-08T19:16Z
STARTUP_MSG:   java = 1.8.0_25
************************************************************/
2018-04-09 17:45:15,060 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2018-04-09 17:45:15,076 INFO namenode.NameNode: createNameNode [-format]
2018-04-09 17:45:15,518 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Formatting using clusterid: CID-f925aa84-648a-478b-9134-77d1766a86cb
…

格式：

2018-04-09 17:45:17,121 WARN namenode.NameNode: Encountered exception during format: 
java.io.IOException: Cannot create directory /usr/local/Cellar/hadoop/hdfs/tmp/dfs/name/current
    at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:416)
    at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:579)
    at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:601)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:167)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1169)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1610)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1720)
2018-04-09 17:45:17,128 ERROR namenode.NameNode: Failed to start namenode.

例外：

java.io.IOException: Cannot create directory /usr/local/Cellar/hadoop/hdfs/tmp/dfs/name/current
    2018-04-09 17:45:17,140 INFO namenode.NameNode: SHUTDOWN_MSG: 
    /************************************************************
    SHUTDOWN_MSG: Shutting down NameNode at My-MacBook-Pro.local/10.5.1.16
    ************************************************************/

My-MacBook-Pro:3.0.0 me$ sbin/start-dfs.sh
Starting namenodes on [localhost]

语法错误：

localhost: /Users/me/.bashrc: line 2: syntax error near unexpected token `('
localhost: /Users/me/.bashrc: line 2: `==> Searching for a previously deleted formula (in the last month)...'
Starting datanodes
localhost: /Users/me/.bashrc: line 2: syntax error near unexpected token `('
localhost: /Users/me/.bashrc: line 2: `==> Searching for a previously deleted formula (in the last month)...'
Starting secondary namenodes [My-MacBook-Pro.local]
My-MacBook-Pro.local: Warning: Permanently added the ECDSA host key for IP address '10.5.13.173' to the list of known hosts.
My-MacBook-Pro.local: /Users/me/.bashrc: line 2: syntax error near unexpected token `('
My-MacBook-Pro.local: /Users/me/.bashrc: line 2: `==> Searching for a previously deleted formula (in the last month)...'

警告：

2018-04-10 08:28:44,359 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

JPS

My-MacBook-Pro:3.0.0 me$ jps
23201 
46872 SecondaryNameNode
46730 DataNode
47039 Jps

主要想知道从哪里开始排除故障？最后一次警告看起来有问题。我在http://localhost:9000/，http://localhost:9010/，http://localhost:9870/

没有得到任何结果

使用Java 1.8在Mac上本地运行Hadoop

0 个答案: