Question

我已在Amazon EC2主机中安装了Apache Hadoop MapReduce 2.6.1，并且我已使用私有IP地址配置运行时。您可以在[2-5]下查看我的配置。

为了尝试简化调试问题，我已经允许安全组[6]中的所有入站流量。

问题是当我启动MapReduce时，我在SSH密钥中得到Permission denied。

我注意到我不能ssh来自同一主机的EC主持人。我该如何解决这个错误？ ubuntu@ip-XXX-XX-XX-XX: ssh ubuntu@ip-XXX-XX-XX
我想在EC2中的不同站点中运行的HDFS实例之间复制数据。使用主机的私有IP地址是否正确？

[1]我有错误

ubuntu@ip-XXX-XX-XX-XX:~/Programs/medusa-2.0$ start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
15/12/16 10:52:18 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [ip-XXX-XX-XX-XX]
ip-XXX-XX-XX-XX: Permission denied (publickey).
ip-XXX-XX-XX-XX: Permission denied (publickey).
Starting secondary namenodes [0.0.0.0]
0.0.0.0: Permission denied (publickey).
15/12/16 10:52:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
starting resourcemanager, logging to /home/ubuntu/Programs/hadoop-2.6.2/logs/yarn-ubuntu-resourcemanager-ip-XXX-XX-XX-XX.out
ip-XXX-XX-XX-XX: Permission denied (publickey).

[2] Yarn-site.xml

ubuntu@ip-XXX-XX-XX-XX:~/Programs$ cat ./hadoop/etc/hadoop/yarn-site.xml 
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
 <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property>
 <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property>  
 <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property>
 <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>ip-XXX-XX-XX-XX:8025</value> </property>
 <property> <name>yarn.resourcemanager.scheduler.address</name> <value>ip-XXX-XX-XX-XX:8030</value> </property>
 <property> <name>yarn.resourcemanager.address</name> <value>ip-XXX-XX-XX-XX:8040</value> </property>
</configuration>

[3] core-site.xml

ubuntu@ip-XXX-XX-XX-XX:~/Programs$ cat ./hadoop/etc/hadoop/core-site.xml 
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
  <property> <name>fs.default.name</name> <value>hdfs://ip-XXX-XX-XX-XX:9000</value> </property>
  <property> <name>hadoop.tmp.dir</name> <value>/tmp/hadoop-temp</value> </property>
</configuration>

[4]奴隶

ubuntu@ip-XXX-XX-XX-XX:~/Programs$ cat ./hadoop/etc/hadoop/slaves 
ip-XXX-XX-XX-XX

[5] SSH文件

ubuntu@ip-XXX-XX-XX-XX:~/Programs$ ls -alrt ~/.ssh/
total 24
-rw------- 1 ubuntu ubuntu  392 Dec 16 09:49 authorized_keys
-rw-r--r-- 1 ubuntu ubuntu  666 Dec 16 10:33 known_hosts
-rw-r--r-- 1 ubuntu ubuntu  404 Dec 16 10:33 id_rsa.pub
-rw------- 1 ubuntu ubuntu 1675 Dec 16 10:33 id_rsa
drwx------ 2 ubuntu ubuntu 4096 Dec 16 10:33 .
drwxr-xr-x 7 ubuntu ubuntu 4096 Dec 16 10:34 ..

[6]安全组

All traffic   All   All   0.0.0.0/0
SSH           TCP   22   0.0.0.0/0

Answer 1

与SSH问题有关。您尚未正确配置SSH。

你可以在运行start-all.sh程序之前测试ssh ubuntu @ ip ****。如果它不起作用，那么您需要解决ssh问题并重试。

EC2主机

1 个答案: