无法在Windows上配置hadoop-1.2.1

时间:2014-02-18 12:21:44

标签: java eclipse hadoop command cygwin

我可以在我的Windows系统上配置hadoop-0.19.1并按照

中给出的步骤进行操作

http://ebiquity.umbc.edu/Tutorials/Hadoop/09%20-%20unpack%20hadoop.html

现在我正在尝试按照相同的步骤配置hadoop-1.2.1但是我被命令

  

$ bin / hadoop namenode -format

它给我的错误如下所示

313159@PC213710 ~
$ cd hadoop-1.2.1

313159@PC213710 ~/hadoop-1.2.1
$ bin/hadoop namenode -format
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 15: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 19: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 21: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 26: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 30: cd: /home/31
: No such file or directory
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 32: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 35: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 87: syntax error
: unexpected end of file
Error: Could not find or load main class org.apache.hadoop.util.PlatformName
Error: Could not find or load main class org.apache.hadoop.hdfs.server.namenode.
NameNode

313159@PC213710 ~/hadoop-1.2.1
$ ssh-host-config

*** ERROR: There are still ssh processes running. Please shut them down first.

313159@PC213710 ~/hadoop-1.2.1
$ ssh-host-config

*** Query: Overwrite existing /etc/ssh_config file? (yes/no) yes
*** Info: Creating default /etc/ssh_config file
*** Query: Overwrite existing /etc/sshd_config file? (yes/no) yes
*** Info: Creating default /etc/sshd_config file
*** Info: Privilege separation is set to yes by default since OpenSSH 3.3.
*** Info: However, this requires a non-privileged account called 'sshd'.
*** Info: For more info on privilege separation read /usr/share/doc/openssh/READ
ME.privsep.
*** Query: Should privilege separation be used? (yes/no) yes
*** Info: Updating /etc/sshd_config file
*** Query: Overwrite existing /etc/inetd.d/sshd-inetd file? (yes/no) yes
*** Info: Creating default /etc/inetd.d/sshd-inetd file
*** Info: Updated /etc/inetd.d/sshd-inetd

*** Info: Sshd service is already installed.

*** Info: Host configuration finished. Have fun!

313159@PC213710 ~/hadoop-1.2.1
$ cd ..

313159@PC213710 ~
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/313159/.ssh/id_rsa):
/home/313159/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/313159/.ssh/id_rsa.
Your public key has been saved in /home/313159/.ssh/id_rsa.pub.
The key fingerprint is:
21:45:11:30:6b:3a:af:c6:4a:2d:ed:3a:bf:be:69:be 313159@PC213710
The key's randomart image is:
+--[ RSA 2048]----+
|      oo=o       |
|       +         |
|      + .        |
|     o . .       |
|    o   S        |
|   o o           |
|  o.o .          |
| ..o+o           |
|  oXE+           |
+-----------------+

313159@PC213710 ~/.ssh
$ ls -l
total 12
-rw-r--r-- 1 313159 mkpasswd 2004 Feb 18 11:59 authorized_keys
-rw------- 1 313159 mkpasswd  668 Oct 21 12:06 id_dsa
-rw-r--r-- 1 313159 mkpasswd  605 Oct 21 12:06 id_dsa.pub
-rw------- 1 313159 mkpasswd 1679 Feb 18 11:57 id_rsa
-rw-r--r-- 1 313159 mkpasswd  397 Feb 18 11:57 id_rsa.pub
-rw-r--r-- 1 313159 mkpasswd  171 Oct 21 12:00 known_hosts

313159@PC213710 ~/.ssh
$ cd ..

313159@PC213710 ~
$ cd hadoop-1.2.1

313159@PC213710 ~/hadoop-1.2.1
$ bin/hadoop namenode -format
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 15: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 19: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 21: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 26: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 30: cd: /home/31
: No such file or directory
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 32: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 35: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 87: syntax error
: unexpected end of file
Error: Could not find or load main class org.apache.hadoop.util.PlatformName
Error: Could not find or load main class org.apache.hadoop.hdfs.server.namenode.
NameNode

313159@PC213710 ~/hadoop-1.2.1
$ cd bin

313159@PC213710 ~/hadoop-1.2.1/bin
$ dos2unix *.sh
dos2unix: converting file hadoop-config.sh to Unix format ...
dos2unix: converting file hadoop-daemon.sh to Unix format ...
dos2unix: converting file hadoop-daemons.sh to Unix format ...
dos2unix: converting file slaves.sh to Unix format ...
dos2unix: converting file start-all.sh to Unix format ...
dos2unix: converting file start-balancer.sh to Unix format ...
dos2unix: converting file start-dfs.sh to Unix format ...
dos2unix: converting file start-jobhistoryserver.sh to Unix format ...
dos2unix: converting file start-mapred.sh to Unix format ...
dos2unix: converting file stop-all.sh to Unix format ...
dos2unix: converting file stop-balancer.sh to Unix format ...
dos2unix: converting file stop-dfs.sh to Unix format ...
dos2unix: converting file stop-jobhistoryserver.sh to Unix format ...
dos2unix: converting file stop-mapred.sh to Unix format ...

313159@PC213710 ~/hadoop-1.2.1/bin
$ cd ..

313159@PC213710 ~/hadoop-1.2.1
$ cd conf

313159@PC213710 ~/hadoop-1.2.1/conf
$ dos2unix *.sh
dos2unix: converting file hadoop-env.sh to Unix format ...

313159@PC213710 ~/hadoop-1.2.1/conf
$ cd ..

313159@PC213710 ~/hadoop-1.2.1
$ bin/hadoop namenode -format
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 15: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 19: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 21: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 26: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 30: cd: /home/31
: No such file or directory
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 32: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 35: $'\r': comma
nd not found
/home/313159/hadoop-1.2.1/bin/../libexec/hadoop-config.sh: line 87: syntax error
: unexpected end of file
Error: Could not find or load main class org.apache.hadoop.util.PlatformName
Error: Could not find or load main class org.apache.hadoop.hdfs.server.namenode.
NameNode

任何人都可以建议/分享我解决问题的方法 谢谢

1 个答案:

答案 0 :(得分:0)

原因可能是Unix和Windows中的换行符字符差异。

我认为答案是按照this post [可能重复]中的建议运行此文件的dos2unix命令。