Hadoop安装文件夹

时间:2015-12-29 15:50:36

标签: hadoop

我正在尝试设置一个hadoop多节点集群。当我检查安装文件夹时,我没有看到名为/ conf的文件夹。

我看到一个/ etc目录,里面有一个hadoop文件夹,里面有一些xml文件 -

core-site.xml
hdfs-site.xml 
master     ...etc

但没有任何主文件或conf文件夹

1 个答案:

答案 0 :(得分:1)

当前Apache Hadoop 2.x版本需要etc/hadoop目录结构。您将无法获得conf目录。以下是全新安装的Apache Hadoop 2.7.1的目录结构。

> curl https://www.apache.org/dist/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz 2>/dev/null | tar xf -

> ls -l hadoop-2.7.1/
total 48
-rw-r--r--   1 chris  wheel    15K Jun 28  2015 LICENSE.txt
-rw-r--r--   1 chris  wheel   101B Jun 28  2015 NOTICE.txt
-rw-r--r--   1 chris  wheel   1.3K Jun 28  2015 README.txt
drwxr-xr-x  13 chris  wheel   442B Jun 28  2015 bin/
drwxr-xr-x   3 chris  wheel   102B Jun 28  2015 etc/
drwxr-xr-x   7 chris  wheel   238B Jun 28  2015 include/
drwxr-xr-x   3 chris  wheel   102B Jun 28  2015 lib/
drwxr-xr-x  12 chris  wheel   408B Jun 28  2015 libexec/
drwxr-xr-x  30 chris  wheel   1.0K Jun 28  2015 sbin/
drwxr-xr-x   4 chris  wheel   136B Jun 28  2015 share/

> ls -l hadoop-2.7.1/etc/hadoop/
total 304
-rw-r--r--  1 chris  wheel   4.3K Jun 28  2015 capacity-scheduler.xml
-rw-r--r--  1 chris  wheel   1.3K Jun 28  2015 configuration.xsl
-rw-r--r--  1 chris  wheel   318B Jun 28  2015 container-executor.cfg
-rw-r--r--  1 chris  wheel   774B Jun 28  2015 core-site.xml
-rw-r--r--  1 chris  wheel   3.6K Jun 28  2015 hadoop-env.cmd
-rw-r--r--  1 chris  wheel   4.1K Jun 28  2015 hadoop-env.sh
-rw-r--r--  1 chris  wheel   2.4K Jun 28  2015 hadoop-metrics.properties
-rw-r--r--  1 chris  wheel   2.5K Jun 28  2015 hadoop-metrics2.properties
-rw-r--r--  1 chris  wheel   9.5K Jun 28  2015 hadoop-policy.xml
-rw-r--r--  1 chris  wheel   775B Jun 28  2015 hdfs-site.xml
-rw-r--r--  1 chris  wheel   1.4K Jun 28  2015 httpfs-env.sh
-rw-r--r--  1 chris  wheel   1.6K Jun 28  2015 httpfs-log4j.properties
-rw-r--r--  1 chris  wheel    21B Jun 28  2015 httpfs-signature.secret
-rw-r--r--  1 chris  wheel   620B Jun 28  2015 httpfs-site.xml
-rw-r--r--  1 chris  wheel   3.4K Jun 28  2015 kms-acls.xml
-rw-r--r--  1 chris  wheel   1.5K Jun 28  2015 kms-env.sh
-rw-r--r--  1 chris  wheel   1.6K Jun 28  2015 kms-log4j.properties
-rw-r--r--  1 chris  wheel   5.4K Jun 28  2015 kms-site.xml
-rw-r--r--  1 chris  wheel    11K Jun 28  2015 log4j.properties
-rw-r--r--  1 chris  wheel   951B Jun 28  2015 mapred-env.cmd
-rw-r--r--  1 chris  wheel   1.4K Jun 28  2015 mapred-env.sh
-rw-r--r--  1 chris  wheel   4.0K Jun 28  2015 mapred-queues.xml.template
-rw-r--r--  1 chris  wheel   758B Jun 28  2015 mapred-site.xml.template
-rw-r--r--  1 chris  wheel    10B Jun 28  2015 slaves
-rw-r--r--  1 chris  wheel   2.3K Jun 28  2015 ssl-client.xml.example
-rw-r--r--  1 chris  wheel   2.2K Jun 28  2015 ssl-server.xml.example
-rw-r--r--  1 chris  wheel   2.2K Jun 28  2015 yarn-env.cmd
-rw-r--r--  1 chris  wheel   4.5K Jun 28  2015 yarn-env.sh
-rw-r--r--  1 chris  wheel   690B Jun 28  2015 yarn-site.xml

如果您从Apache Hadoop 1.x迁移到Apache Hadoop 2.x,那么您可能希望看到一个较旧的目录布局,它具有conf目录。以下是Apache Hadoop 1.2.1的外观。

> curl https://www.apache.org/dist/hadoop/common/hadoop-1.2.1/hadoop-1.2.1-bin.tar.gz 2>/dev/null | tar xf -

> ls -l hadoop-1.2.1/
total 16680
-rw-r--r--   1 chris  wheel   482K Jul 22  2013 CHANGES.txt
-rw-r--r--   1 chris  wheel    13K Jul 22  2013 LICENSE.txt
-rw-r--r--   1 chris  wheel   101B Jul 22  2013 NOTICE.txt
-rw-r--r--   1 chris  wheel   1.3K Jul 22  2013 README.txt
drwxr-xr-x  19 chris  wheel   646B Jul 22  2013 bin/
-rw-r--r--   1 chris  wheel   118K Jul 22  2013 build.xml
drwxr-xr-x   4 chris  wheel   136B Jul 22  2013 c++/
drwxr-xr-x  19 chris  wheel   646B Jul 22  2013 conf/
drwxr-xr-x  10 chris  wheel   340B Jul 22  2013 contrib/
-rw-r--r--   1 chris  wheel   6.7K Jul 22  2013 hadoop-ant-1.2.1.jar
-rw-r--r--   1 chris  wheel   414B Jul 22  2013 hadoop-client-1.2.1.jar
-rw-r--r--   1 chris  wheel   4.0M Jul 22  2013 hadoop-core-1.2.1.jar
-rw-r--r--   1 chris  wheel   139K Jul 22  2013 hadoop-examples-1.2.1.jar
-rw-r--r--   1 chris  wheel   417B Jul 22  2013 hadoop-minicluster-1.2.1.jar
-rw-r--r--   1 chris  wheel   3.0M Jul 22  2013 hadoop-test-1.2.1.jar
-rw-r--r--   1 chris  wheel   377K Jul 22  2013 hadoop-tools-1.2.1.jar
drwxr-xr-x  13 chris  wheel   442B Jul 22  2013 ivy/
-rw-r--r--   1 chris  wheel    10K Jul 22  2013 ivy.xml
drwxr-xr-x  52 chris  wheel   1.7K Jul 22  2013 lib/
drwxr-xr-x   4 chris  wheel   136B Jul 22  2013 libexec/
drwxr-xr-x   9 chris  wheel   306B Jul 22  2013 sbin/
drwxr-xr-x   3 chris  wheel   102B Jul 22  2013 share/
drwxr-xr-x   3 chris  wheel   102B Dec 29 09:44 src/
drwxr-xr-x   9 chris  wheel   306B Jul 22  2013 webapps/

> ls -l hadoop-1.2.1/conf/
total 160
-rw-r--r--  1 chris  wheel   7.3K Jul 22  2013 capacity-scheduler.xml
-rw-r--r--  1 chris  wheel   1.1K Jul 22  2013 configuration.xsl
-rw-r--r--  1 chris  wheel   178B Jul 22  2013 core-site.xml
-rw-r--r--  1 chris  wheel   327B Jul 22  2013 fair-scheduler.xml
-rw-r--r--  1 chris  wheel   2.4K Jul 22  2013 hadoop-env.sh
-rw-r--r--  1 chris  wheel   2.0K Jul 22  2013 hadoop-metrics2.properties
-rw-r--r--  1 chris  wheel   4.5K Jul 22  2013 hadoop-policy.xml
-rw-r--r--  1 chris  wheel   178B Jul 22  2013 hdfs-site.xml
-rw-r--r--  1 chris  wheel   4.9K Jul 22  2013 log4j.properties
-rw-r--r--  1 chris  wheel   2.0K Jul 22  2013 mapred-queue-acls.xml
-rw-r--r--  1 chris  wheel   178B Jul 22  2013 mapred-site.xml
-rw-r--r--  1 chris  wheel    10B Jul 22  2013 masters
-rw-r--r--  1 chris  wheel    10B Jul 22  2013 slaves
-rw-r--r--  1 chris  wheel   2.0K Jul 22  2013 ssl-client.xml.example
-rw-r--r--  1 chris  wheel   1.9K Jul 22  2013 ssl-server.xml.example
-rw-r--r--  1 chris  wheel   3.8K Jul 22  2013 task-log4j.properties
-rw-r--r--  1 chris  wheel   382B Jul 22  2013 taskcontroller.cfg

但是,在Apache Hadoop 2.x发行版中,您将无法获得conf目录。

Hadoop 1.x中的conf和Hadoop 2.x中的etc/hadoop的内容有些类似。您将看到各种* -site.xml文件。有关配置的更多详细信息,请参阅Cluster Setup指南。